Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15336

Decouple DataUnitRegistry from Schema


    • Story Points:
    • Sprint:
      DRP F18-4
    • Team:
      Data Release Production


      In order to use DataUnit objects inside DatasetType and DataId objects on DM-15034, we need to make it possible to construct a DataUnitRegistry without having a Registry instance.  In many ways that makes DataUnitRegistry analogous to StorageClassFactory, and hence we should follow the same approach: make it an append-only singleton.

      This is complicated by the fact that Schema and DataUnitRegistry are closely intertwined; DataUnit objects actually currently have some SQLAlchemy objects attached to them.  We'll have to remove those as part of this ticket, and instead attach them to Schema in a way that allows it to retrieve them with DataUnit objects or names as keys.

      The recommended implementation plan is roughly:

      • Move the "tables" sections out of the individual DataUnit sections in schema.yaml into the main "tables" section at the end; this should break at most a tiny bit of the parsing logic for that configuration (and possibly nothing).
      • Remove the "foreignKey" sections nested under "link" sections in the yaml; instead generate these programmatically.
      • Split the DataUnit content (now free of any SQL-specific stuff) into its own yaml file; adjust the levels of both dataUnits.yaml and schema.yaml appropriately (e.g. remove the now-superfluous "tables" section heading?).
      • Change SchemaBuilder so SQLAlchemy objects are put somewhere in Schema where they can be looked up via DataUnit name or object, instead in the DataUnit objects themselves.  Fix whatever this breaks in SqlRegistry and SqlPreflight.


          Issue Links


            No work has yet been logged on this issue.


              • Assignee:
                nlust Nate Lust
                jbosch Jim Bosch
                Jim Bosch
                Jim Bosch, Nate Lust, Pim Schellart [X] (Inactive), Tim Jenness
              • Votes:
                0 Vote for this issue
                4 Start watching this issue


                • Created:

                  Summary Panel