Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21451

Remove DatabaseDict and vectorize Datastore/Butler ingest APIs

    XMLWordPrintable

    Details

      Description

      From #dm-middleware on slack:

      I'd like to get Datastores using the new schema stuff I added recently (TableSpec/FieldSpec) for anything they want to keep in the registry database, and I want to stop using a dict interface, as it's hard to map _setitem_ to insert and update, especially for bulk operations.  I'm thinking of:

      • Removing DatabaseDict and DatabaseDictRecordBase entirely.
      • Add abstract methods to StoredDatastoreItemInfo to translate to/from dictionaries that correspond to records to be saved.
      • Moving the logic for inserting and retrieving StoredDatastoreItemInfo from the database directly into GenericDatastoreBase.  That would probably be _insert_info and _fetch_info methods that replace _info_to_record and _record_to_info.
      • Overriding _insert_info and _fetch_info in InMemoryDatastore to save into a dict instead of letting GenericDatastoreBase's implementations write to the database.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            Tim Jenness, I know you're traveling, but you're still pretty clearly the best candidate for this review.  No rush - I'll need to land some other changes before we can gain much advantage from these, and at least the first few of those will be quite orthogonal.

            Show
            jbosch Jim Bosch added a comment - Tim Jenness , I know you're traveling, but you're still pretty clearly the best candidate for this review.  No rush - I'll need to land some other changes before we can gain much advantage from these, and at least the first few of those will be quite orthogonal.
            Hide
            jbosch Jim Bosch added a comment -

            Tim Jenness, ping: review request may have gone out while Jira wasn't sending them.

            Show
            jbosch Jim Bosch added a comment - Tim Jenness , ping: review request may have gone out while Jira wasn't sending them.
            Hide
            tjenness Tim Jenness added a comment -

            In my defense I started doing the review five minutes before your comment turned up

            Show
            tjenness Tim Jenness added a comment - In my defense I started doing the review five minutes before your comment turned up
            Hide
            tjenness Tim Jenness added a comment -

            Looks good. I have some questions on the PR.

            Show
            tjenness Tim Jenness added a comment - Looks good. I have some questions on the PR.

              People

              Assignee:
              jbosch Jim Bosch
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Tim Jenness
              Watchers:
              Jim Bosch, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.