Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4382

Port registryInfo.py from obs_subaru into Butler

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: daf_persistence, obs_base
    • Labels:
      None

      Description

      in butler (probably butlerUtils), users would like the ability to dump info from a repository's sqlite registry to text (console output). This is already implemented in obs_subaru/.../registryInfo.py, and basically just needs to be ported to butler in a sensible place. There are a few cases that assume certain columns are present and we need either to make the script more generic, or Robert Lupton suggests that maybe we need to standardize the registries.

        Attachments

          Activity

          No builds found.
          npease Nate Pease [X] (Inactive) created issue -
          npease Nate Pease [X] (Inactive) made changes -
          Field Original Value New Value
          Epic Link DM-2404 [ 16718 ]
          npease Nate Pease [X] (Inactive) made changes -
          Rank Ranked higher
          npease Nate Pease [X] (Inactive) made changes -
          Assignee Nate Pease [ npease ]
          tjenness Tim Jenness made changes -
          Description in butler (probably butlerUtils), users would like the ability to dump info from a repository's sqlite registry to text (console output). This is already implemented in obs_subaru/.../registryInfo.py, and basically just needs to be ported to butler in a sensible place. There are a few cases that assume certain columns are present and we need either to make the script more generic, or @rhl suggests that maybe we need to standardize the registries. in butler (probably butlerUtils), users would like the ability to dump info from a repository's sqlite registry to text (console output). This is already implemented in obs_subaru/.../registryInfo.py, and basically just needs to be ported to butler in a sensible place. There are a few cases that assume certain columns are present and we need either to make the script more generic, or [~rhl] suggests that maybe we need to standardize the registries.
          tjenness Tim Jenness made changes -
          Description in butler (probably butlerUtils), users would like the ability to dump info from a repository's sqlite registry to text (console output). This is already implemented in obs_subaru/.../registryInfo.py, and basically just needs to be ported to butler in a sensible place. There are a few cases that assume certain columns are present and we need either to make the script more generic, or [~rhl] suggests that maybe we need to standardize the registries. in butler (probably butlerUtils), users would like the ability to dump info from a repository's sqlite registry to text (console output). This is already implemented in {{obs_subaru/.../registryInfo.py}}, and basically just needs to be ported to butler in a sensible place. There are a few cases that assume certain columns are present and we need either to make the script more generic, or [~rhl] suggests that maybe we need to standardize the registries.
          Hide
          ktl Kian-Tat Lim added a comment -

          Standardizing registries could require limiting a lot of the flexibility that the Butler was intended to provide. It is not clear if that flexibility is still desirable, however.

          There are definitely some things in registryInfo.py that are very HSC-specific. A more generic version should probably dump all columns present in the "raw" table (or allow some means of specifying a list of columns to include or exclude), possibly with some summarization. I'm not sure that the max() functions are always appropriate.

          And we should likely use the lsst.db package and SQLalchemy for database portability, if that is necessary (otherwise just use sqlite as the CameraMapper's registries do internally).

          Show
          ktl Kian-Tat Lim added a comment - Standardizing registries could require limiting a lot of the flexibility that the Butler was intended to provide. It is not clear if that flexibility is still desirable, however. There are definitely some things in registryInfo.py that are very HSC-specific. A more generic version should probably dump all columns present in the "raw" table (or allow some means of specifying a list of columns to include or exclude), possibly with some summarization. I'm not sure that the max() functions are always appropriate. And we should likely use the lsst.db package and SQLalchemy for database portability, if that is necessary (otherwise just use sqlite as the CameraMapper's registries do internally).
          jbecla Jacek Becla made changes -
          Team Data Access and Database [ 10204 ]
          jbecla Jacek Becla made changes -
          Epic Link DM-2404 [ 16718 ]
          jbecla Jacek Becla made changes -
          Rank Ranked higher
          jbecla Jacek Becla made changes -
          Epic Link DM-4341 [ 21422 ]
          jbecla Jacek Becla made changes -
          Rank Ranked higher
          jbecla Jacek Becla made changes -
          Rank Ranked higher
          npease Nate Pease [X] (Inactive) made changes -
          Epic Link DM-4341 [ 21422 ] DM-4340 [ 21421 ]
          fritzm Fritz Mueller made changes -
          Epic Link DM-4340 [ 21421 ] DM-10678 [ 32630 ]
          fritzm Fritz Mueller made changes -
          Rank Ranked higher
          fritzm Fritz Mueller made changes -
          Rank Ranked higher
          fritzm Fritz Mueller made changes -
          Epic Link DM-10678 [ 32630 ] DM-12759 [ 36358 ]
          fritzm Fritz Mueller made changes -
          Assignee Nate Pease [ npease ]
          tjenness Tim Jenness made changes -
          Component/s daf_persistence [ 10713 ]
          Component/s butler [ 12317 ]
          tjenness Tim Jenness made changes -
          Component/s obs_base [ 10719 ]
          Hide
          tjenness Tim Jenness added a comment -

          Gen2 is dead.

          Show
          tjenness Tim Jenness added a comment - Gen2 is dead.
          tjenness Tim Jenness made changes -
          Resolution Done [ 10000 ]
          Status To Do [ 10001 ] Won't Fix [ 10405 ]

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            npease Nate Pease [X] (Inactive)
            Watchers:
            Hsin-Fang Chiang, Kian-Tat Lim, Nate Pease [X] (Inactive), Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.