Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22609

Start using Database in Registry

    Details

      Description

      Use the new Database class (DM-22508) as the backend for Registry, with a lot of pragmatism about how much to change.  The goal will be to make this as transparent to Butler users as possible (I don't yet see a reason why it can't be completely transparent), and stick to:

      • Making Registry a concrete class, basically just merging Registry's docs with SqlRegistry's implementations.
      • Using Database instances to create engines and connections for Registry (and connecting RegistryConfig's connectionString logic with Database's).
      • Using Database to emit all DDL (this probably means moving schema.yaml content to Python).

      At this point, we'll just brazenly steal the (protected) connection object out of the `Database` instance to let most existing SqlRegistry method implementations remain unchanged.  That will get fixed later as we port Manager objects over from the u/jbosch/prototyping branch.

      This should take care of all of the code duplication that currently exists (core/schema.py and registry/ddl.py, registries/* and registry/databases/*).

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            Tim Jenness, I'm afraid you're next up on my list of victims reviewers. This one is big, but as the PR comment explains it's also a lot smaller than it seems in terms of actual content.

            I'm still looking for breakage in downstream packages (I don't expect much, but there will probably be a little), but this will be coming your way eventually, and you're welcome to get started now if you like.

            I will be disappearing on Thursday for vacation, and while I may work a bit, I may be slow in replying to review comments, and there's no rush to get this done until after the holidays.

            Show
            jbosch Jim Bosch added a comment - Tim Jenness , I'm afraid you're next up on my list of victims reviewers. This one is big, but as the PR comment explains it's also a lot smaller than it seems in terms of actual content. I'm still looking for breakage in downstream packages (I don't expect much, but there will probably be a little), but this will be coming your way eventually, and you're welcome to get started now if you like. I will be disappearing on Thursday for vacation, and while I may work a bit, I may be slow in replying to review comments, and there's no rush to get this done until after the holidays.
            Hide
            jbosch Jim Bosch added a comment -

            Jenkins is green, with only trivial changes to obs_base, so this is now fully ready for review.

            Show
            jbosch Jim Bosch added a comment - Jenkins is green, with only trivial changes to obs_base , so this is now fully ready for review.
            Hide
            tjenness Tim Jenness added a comment -

            Sorry it took me so long. It looks fine to me. Minor comments on PR.

            Show
            tjenness Tim Jenness added a comment - Sorry it took me so long. It looks fine to me. Minor comments on PR.

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Reviewers:
                Tim Jenness
                Watchers:
                Jim Bosch, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel