Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12634

Write SQL-YAML Converters for Butler Registry Schema

    Details

    • Templates:
    • Story Points:
      8
    • Team:
      Data Release Production

      Description

      Write functions/classes that convert Registry content from SQL to YAML and back, assuming the SQL schema is created separately.

      The YAML format should be designed with the expectation that it will be used
      as both:

      • the internal representation of a limited Registry (possibly nested in a file that also contains Butler and Datastore configuration)
      • a format ("the" format?) for transfers between Registries.

      This should use SQLAlchemy as much as possible, but target SQLite specifically when necessary. When interfaces that would allow different future specializations for other RDBMs are obvious, they should be added, but this should not be considered a focus at this point.

      The SQL->YAML converter should be something usable by all future SQL Registries as a way to dump their output. We do not expect the YAML->SQL converter to work for all future Registries (as some will expose the common schema via views, or need to rewrite IDs on import), but it will be used by those included in the Minimal Gen3 Butler deliverable.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                pschella Pim Schellart [X] (Inactive)
                Watchers:
                Jim Bosch, John Swinbank, Pim Schellart [X] (Inactive), Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel