Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21764

Better encapsulate dataset storage in Registry

    XMLWordPrintable

    Details

      Description

      Move registry code involving datasets (and, as needed, runs and collections) into helper classes as the prototype. If possible, defer actually moving from monolithic to split dataset tables to DM-21766.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21766 Add per-dataset-type tables to Registry

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-24432 Add CALIBRATION collections and remove the calibration_label dimension

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-24612 Add indexes to dataset_collection tables

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-24614 Move dataset_location tables into manager/storage hierarchy

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21794 Add origin component to all autoincrement data IDs in Registry

          • Invalid
          is contained by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21231 Refactor Registry handling of dataset and associated tables

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            Restarting work on this now. I currently plan to do DM-21766 on the same branch (except they'll actually be per-sets-of-dimensions tables, not per-dataset-type), but not DM-21794 or DM-24432 (which currently seem no harder to do later).

            Show
            jbosch Jim Bosch added a comment - Restarting work on this now. I currently plan to do DM-21766 on the same branch (except they'll actually be per-sets-of-dimensions tables, not per-dataset-type), but not DM-21794 or DM-24432 (which currently seem no harder to do later).
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            Andy Salnikov, here's another fairly large review for you, but I'm hoping it will at least be familiar given that it's a refactoring pattern I know I've asked you to review before. All changes are in daf_butler (so far; running Jenkins now but I don't expect other packages to break), and most commits are small cleanups or improvements that either set up or react to the three big ones, which:

            Jira seems to picked up an irrelevant PR as well as the right one, which is https://github.com/lsst/daf_butler/pull/266.

            Show
            jbosch Jim Bosch added a comment - Andy Salnikov , here's another fairly large review for you, but I'm hoping it will at least be familiar given that it's a refactoring pattern I know I've asked you to review before. All changes are in daf_butler (so far; running Jenkins now but I don't expect other packages to break), and most commits are small cleanups or improvements that either set up or react to the three big ones, which: add ABCs for helper classes for datasets in Registry ; provide default implementations for those ; switch Registry over to using them. Jira seems to picked up an irrelevant PR as well as the right one, which is https://github.com/lsst/daf_butler/pull/266 .
            Hide
            Permalink
            salnikov Andy Salnikov added a comment -

            Looks OK, few comments on PR.

            Show
            salnikov Andy Salnikov added a comment - Looks OK, few comments on PR.

              People

              Assignee:
              jbosch Jim Bosch
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Andy Salnikov
              Watchers:
              Andy Salnikov, Jim Bosch
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.