Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12634

Write SQL-YAML Converters for Butler Registry Schema

    XMLWordPrintable

    Details

      Description

      Write functions/classes that convert Registry content from SQL to YAML and back, assuming the SQL schema is created separately.

      The YAML format should be designed with the expectation that it will be used
      as both:

      • the internal representation of a limited Registry (possibly nested in a file that also contains Butler and Datastore configuration)
      • a format ("the" format?) for transfers between Registries.

      This should use SQLAlchemy as much as possible, but target SQLite specifically when necessary. When interfaces that would allow different future specializations for other RDBMs are obvious, they should be added, but this should not be considered a focus at this point.

      The SQL->YAML converter should be something usable by all future SQL Registries as a way to dump their output. We do not expect the YAML->SQL converter to work for all future Registries (as some will expose the common schema via views, or need to rewrite IDs on import), but it will be used by those included in the Minimal Gen3 Butler deliverable.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            While we haven't yet gotten back to working on registry transfer formats, this old ticket made some assumptions I'm not sure I'd make now.  Should re-frame the problem in new tickets once we get there.

            Show
            jbosch Jim Bosch added a comment - While we haven't yet gotten back to working on registry transfer formats, this old ticket made some assumptions I'm not sure I'd make now.  Should re-frame the problem in new tickets once we get there.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              pschella Pim Schellart [X] (Inactive)
              Watchers:
              Jim Bosch, John Swinbank, Pim Schellart [X] (Inactive), Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.