Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29196

Use UUIDs as dataset_ids in registry

    XMLWordPrintable

    Details

    • Story Points:
      12
    • Sprint:
      DB_S21_12
    • Team:
      Data Access and Database
    • Urgent?:
      No

      Description

      Syncing registries from different sources (such as the lightweight registry from workflows) will be simplified significantly if we switch from autoincrementing integers to UUIDs for our datasets.

      Some things to consider:

      • Where are UUIDs calculated? By Registry?
      • Should we allow UUIDs to be calculated from a datasetRef (dataset type / dataId and run name). This could be an alternative implementation of DM-21794, allowing raw data to have predictable IDs.
      • Should we allow an external source for UUID?
      • Is there a case for datastore to allocate IDs itself which can then be used by registry?

        Attachments

          Issue Links

            Activity

            tjenness Tim Jenness created issue -
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Description Syncing registries from different sources (such as the lightweight registry from workflows) will be simplified significantly if we switch from autoincrementing integers to UUIDs for our datasets.

            Some things to consider:

            * Where are UUIDs calculated? By Registry?
            * Should we allow UUIDs to be calculated from a dataId and collection name. This could be an alternative implementation of DM-21794, allowing raw data to have predictable IDs.
            * Is there a case for datastore to allocate IDs itself which can then be used by registry?



            Syncing registries from different sources (such as the lightweight registry from workflows) will be simplified significantly if we switch from autoincrementing integers to UUIDs for our datasets.

            Some things to consider:

            * Where are UUIDs calculated? By Registry?
            * Should we allow UUIDs to be calculated from a datasetRef (dataset type / dataId and run name). This could be an alternative implementation of DM-21794, allowing raw data to have predictable IDs.
            * Should we allow an external source for UUID?
            * Is there a case for datastore to allocate IDs itself which can then be used by registry?



            salnikov Andy Salnikov made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            tjenness Tim Jenness made changes -
            Remote Link This issue links to "DMTN-099 (Web Link)" [ 27936 ]
            tjenness Tim Jenness made changes -
            Link This issue relates to DM-21794 [ DM-21794 ]
            salnikov Andy Salnikov made changes -
            Epic Link DM-27791 [ 442307 ]
            Sprint DB_S21_12 [ 1065 ]
            Labels gen3-middleware
            salnikov Andy Salnikov made changes -
            Attachment DM-29196 datasets insert time sqlite.png [ 48516 ]
            Attachment DM-29196 datasets insert time postgres.png [ 48517 ]
            salnikov Andy Salnikov made changes -
            Attachment DM-29196 datasets insert time sqlite.png [ 48516 ]
            salnikov Andy Salnikov made changes -
            Attachment DM-29196 datasets insert time postgres.png [ 48517 ]
            salnikov Andy Salnikov made changes -
            salnikov Andy Salnikov made changes -
            Remote Link This issue links to "Script used for testing INSERT performance (Web Link)" [ 27943 ]
            salnikov Andy Salnikov made changes -
            yanny Brian Yanny made changes -
            Watchers Andy Salnikov, Jim Bosch, Michelle Gower, Tim Jenness [ Andy Salnikov, Jim Bosch, Michelle Gower, Tim Jenness ] Andy Salnikov, Brian Yanny, Jim Bosch, Michelle Gower, Tim Jenness [ Andy Salnikov, Brian Yanny, Jim Bosch, Michelle Gower, Tim Jenness ]
            tjenness Tim Jenness made changes -
            Summary Use UUIDs are dataset_ids in registry Use UUIDs as dataset_ids in registry
            salnikov Andy Salnikov made changes -
            Reviewers Jim Bosch [ jbosch ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-29593 [ DM-29593 ]
            jbosch Jim Bosch made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            salnikov Andy Salnikov made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            tjenness Tim Jenness made changes -
            Link This issue relates to DM-29765 [ DM-29765 ]

              People

              Assignee:
              salnikov Andy Salnikov
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              Jim Bosch
              Watchers:
              Andy Salnikov, Brian Yanny, Jim Bosch, Michelle Gower, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.