Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8212

Add support for reading metadata, length and schema of a catalog

    Details

    • Type: Story
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_base
    • Labels:
      None
    • Templates:
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP F16-6
    • Team:
      Data Release Production

      Description

      The butler.get(dataset + "_md", dataId) pattern is only supported for images, not catalogs. It would be useful to allow this to work on catalogs.
      At the same time, it would be useful to add support for getting the length and schema of a catalog.

        Issue Links

          Activity

          Hide
          price Paul Price added a comment -

          John Parejko, would you mind reviewing this, please? There's some cleanup as well as adding the new functionality. It's currently going through Jenkins.

          price@price-laptop:~/LSST/obs/base (tickets/DM-8212=) $ git sub
          commit 063ae18cb95808ac13edce0dc05a0ce15c91f9b8
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Tue Nov 8 15:04:06 2016 -0500
           
              CameraMapper: simplify setting of extra dataset features
              
              Extra dataset features (e.g., "raw_sub", "raw_md", "raw_filename") are
              are created from the original mapping (e.g., "raw") by defining a bunch
              of extra methods. This patch replaces the sea of
                  if not hasattr(self, someName):
                      setattr(self, someName, someImpl)
              calls with a common function to set the implementations for each extra
              dataset feature. This clarifies the code and makes it easier to add more
              extra features.
           
           python/lsst/obs/base/cameraMapper.py | 97 +++++++++++++++++-------------------
           1 file changed, 47 insertions(+), 50 deletions(-)
           
          commit cac031cc03abf537bf3c75879da0c40a65993666
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Tue Nov 8 15:07:19 2016 -0500
           
              CameraMapper: add extra dataset features for catalogs
              
              Added "foo_md", "foo_len" and "foo_schema" if "foo" is a catalog.
              
              Note that the "foo_schema" commonly has a different meaning because it's
              usually defined explicitly if the "foo" is an important pipeline output.
           
           python/lsst/obs/base/cameraMapper.py | 15 +++++++++++++++
           tests/MinMapper2.paf                 | 10 +++++-----
           tests/testCameraMapper.py            | 33 ++++++++++++++++++++++++++++++---
           3 files changed, 50 insertions(+), 8 deletions(-)
          

          Show
          price Paul Price added a comment - John Parejko , would you mind reviewing this, please? There's some cleanup as well as adding the new functionality. It's currently going through Jenkins . price@price-laptop:~/LSST/obs/base (tickets/DM-8212=) $ git sub commit 063ae18cb95808ac13edce0dc05a0ce15c91f9b8 Author: Paul Price <price@astro.princeton.edu> Date: Tue Nov 8 15:04:06 2016 -0500   CameraMapper: simplify setting of extra dataset features Extra dataset features (e.g., "raw_sub", "raw_md", "raw_filename") are are created from the original mapping (e.g., "raw") by defining a bunch of extra methods. This patch replaces the sea of if not hasattr(self, someName): setattr(self, someName, someImpl) calls with a common function to set the implementations for each extra dataset feature. This clarifies the code and makes it easier to add more extra features.   python/lsst/obs/base/cameraMapper.py | 97 +++++++++++++++++------------------- 1 file changed, 47 insertions(+), 50 deletions(-)   commit cac031cc03abf537bf3c75879da0c40a65993666 Author: Paul Price <price@astro.princeton.edu> Date: Tue Nov 8 15:07:19 2016 -0500   CameraMapper: add extra dataset features for catalogs Added "foo_md", "foo_len" and "foo_schema" if "foo" is a catalog. Note that the "foo_schema" commonly has a different meaning because it's usually defined explicitly if the "foo" is an important pipeline output.   python/lsst/obs/base/cameraMapper.py | 15 +++++++++++++++ tests/MinMapper2.paf | 10 +++++----- tests/testCameraMapper.py | 33 ++++++++++++++++++++++++++++++--- 3 files changed, 50 insertions(+), 8 deletions(-)
          Hide
          price Paul Price added a comment -

          First Jenkins revealed a small fix was needed. A new Jenkins run passed.

          Show
          price Paul Price added a comment - First Jenkins revealed a small fix was needed. A new Jenkins run passed.
          Hide
          Parejkoj John Parejko added a comment -

          Thanks for cleaning this up. See comments on github.

          Show
          Parejkoj John Parejko added a comment - Thanks for cleaning this up. See comments on github.
          Hide
          price Paul Price added a comment -

          Made changes that were requested. A new Jenkins is running.

          Show
          price Paul Price added a comment - Made changes that were requested. A new Jenkins is running.
          Hide
          price Paul Price added a comment -

          Jenkins passed following a rebase.

          Show
          price Paul Price added a comment - Jenkins passed following a rebase.
          Hide
          price Paul Price added a comment -

          Sorry, that Jenkins was for another issue (not even mine).

          The relevant Jenkins is this one.

          Show
          price Paul Price added a comment - Sorry, that Jenkins was for another issue (not even mine). The relevant Jenkins is this one .
          Hide
          price Paul Price added a comment -

          Merged to master.

          Show
          price Paul Price added a comment - Merged to master.

            People

            • Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Reviewers:
              John Parejko
              Watchers:
              John Parejko, Paul Price
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development

                  Agile