Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8212

Add support for reading metadata, length and schema of a catalog

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_base
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP F16-6
    • Team:
      Data Release Production

      Description

      The butler.get(dataset + "_md", dataId) pattern is only supported for images, not catalogs. It would be useful to allow this to work on catalogs.
      At the same time, it would be useful to add support for getting the length and schema of a catalog.

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            John Parejko, would you mind reviewing this, please? There's some cleanup as well as adding the new functionality. It's currently going through Jenkins.

            price@price-laptop:~/LSST/obs/base (tickets/DM-8212=) $ git sub
            commit 063ae18cb95808ac13edce0dc05a0ce15c91f9b8
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue Nov 8 15:04:06 2016 -0500
             
                CameraMapper: simplify setting of extra dataset features
                
                Extra dataset features (e.g., "raw_sub", "raw_md", "raw_filename") are
                are created from the original mapping (e.g., "raw") by defining a bunch
                of extra methods. This patch replaces the sea of
                    if not hasattr(self, someName):
                        setattr(self, someName, someImpl)
                calls with a common function to set the implementations for each extra
                dataset feature. This clarifies the code and makes it easier to add more
                extra features.
             
             python/lsst/obs/base/cameraMapper.py | 97 +++++++++++++++++-------------------
             1 file changed, 47 insertions(+), 50 deletions(-)
             
            commit cac031cc03abf537bf3c75879da0c40a65993666
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue Nov 8 15:07:19 2016 -0500
             
                CameraMapper: add extra dataset features for catalogs
                
                Added "foo_md", "foo_len" and "foo_schema" if "foo" is a catalog.
                
                Note that the "foo_schema" commonly has a different meaning because it's
                usually defined explicitly if the "foo" is an important pipeline output.
             
             python/lsst/obs/base/cameraMapper.py | 15 +++++++++++++++
             tests/MinMapper2.paf                 | 10 +++++-----
             tests/testCameraMapper.py            | 33 ++++++++++++++++++++++++++++++---
             3 files changed, 50 insertions(+), 8 deletions(-)
            

            Show
            price Paul Price added a comment - John Parejko , would you mind reviewing this, please? There's some cleanup as well as adding the new functionality. It's currently going through Jenkins . price@price-laptop:~/LSST/obs/base (tickets/DM-8212=) $ git sub commit 063ae18cb95808ac13edce0dc05a0ce15c91f9b8 Author: Paul Price <price@astro.princeton.edu> Date: Tue Nov 8 15:04:06 2016 -0500   CameraMapper: simplify setting of extra dataset features Extra dataset features (e.g., "raw_sub", "raw_md", "raw_filename") are are created from the original mapping (e.g., "raw") by defining a bunch of extra methods. This patch replaces the sea of if not hasattr(self, someName): setattr(self, someName, someImpl) calls with a common function to set the implementations for each extra dataset feature. This clarifies the code and makes it easier to add more extra features.   python/lsst/obs/base/cameraMapper.py | 97 +++++++++++++++++------------------- 1 file changed, 47 insertions(+), 50 deletions(-)   commit cac031cc03abf537bf3c75879da0c40a65993666 Author: Paul Price <price@astro.princeton.edu> Date: Tue Nov 8 15:07:19 2016 -0500   CameraMapper: add extra dataset features for catalogs Added "foo_md", "foo_len" and "foo_schema" if "foo" is a catalog. Note that the "foo_schema" commonly has a different meaning because it's usually defined explicitly if the "foo" is an important pipeline output.   python/lsst/obs/base/cameraMapper.py | 15 +++++++++++++++ tests/MinMapper2.paf | 10 +++++----- tests/testCameraMapper.py | 33 ++++++++++++++++++++++++++++++--- 3 files changed, 50 insertions(+), 8 deletions(-)
            Hide
            price Paul Price added a comment -

            First Jenkins revealed a small fix was needed. A new Jenkins run passed.

            Show
            price Paul Price added a comment - First Jenkins revealed a small fix was needed. A new Jenkins run passed.
            Hide
            Parejkoj John Parejko added a comment -

            Thanks for cleaning this up. See comments on github.

            Show
            Parejkoj John Parejko added a comment - Thanks for cleaning this up. See comments on github.
            Hide
            price Paul Price added a comment -

            Made changes that were requested. A new Jenkins is running.

            Show
            price Paul Price added a comment - Made changes that were requested. A new Jenkins is running.
            Hide
            price Paul Price added a comment -

            Jenkins passed following a rebase.

            Show
            price Paul Price added a comment - Jenkins passed following a rebase.
            Hide
            price Paul Price added a comment -

            Sorry, that Jenkins was for another issue (not even mine).

            The relevant Jenkins is this one.

            Show
            price Paul Price added a comment - Sorry, that Jenkins was for another issue (not even mine). The relevant Jenkins is this one .
            Hide
            price Paul Price added a comment -

            Merged to master.

            Show
            price Paul Price added a comment - Merged to master.

              People

              Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Reviewers:
              John Parejko
              Watchers:
              John Parejko, Paul Price
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.