Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7883

Butler.get('ccdExposureId') error message unhelpful


    • Templates:
    • Team:


      When calling butler.get('ccdExposureId', dataId=dataId), if the dataId doesn't contain items that are required by mapper._computeCcdExposureId(), you get a non-helpful error. Example code below.

      This problem is shared with all of the obs_* package mapper._compute* methods: they expect specific things in the dataId, and fail with a KeyError if those are not available. The user of butler.get does not know why _computeCcdExposureId (or even bypass_ccdExposureId, which calls the latter) is being called, and so does not know where to look to determine what they're missing.

      import lsst.daf.persistence
      product_dir = getPackageDir('testdata_decam')
      butler = lsst.daf.persistence.Butler(root=product_dir+'/rawData')
      butler.get('ccdExposureId', dataId={'visit':1})
      125743 [0x7fff7b67f310] DEBUG daf.persistence.butler null - Get type=ccdExposureId keys=DataId(initialdata={'visit': 1}, tag=set()) from lsst.daf.base.PropertySet at ignored(/Users/parejkoj/lsst/lsstsw/stack/DarwinX86/testdata_decam/master-gfea4835579/rawData/ignored)
      Out[7]: ---------------------------------------------------------------------------
      AttributeError                            Traceback (most recent call last)
      /Users/parejkoj/lsst/lsstsw3/stack/DarwinX86/daf_persistence/12.1+1/python/lsst/daf/persistence/readProxy.py in __subject__(self, get_cache, set_cache)
          135     try:
      --> 136         return get_cache(self)
          137     except AttributeError:
      AttributeError: __cache__
      During handling of the above exception, another exception occurred:
      KeyError                                  Traceback (most recent call last)
      /Users/parejkoj/lsst/lsstsw3/miniconda/lib/python3.5/site-packages/IPython/core/formatters.py in __call__(self, obj)
          878                 return True
          879             # Finally look for special method names
      --> 880             method = get_real_method(obj, self.print_method)
          881             if method is not None:
          882                 method()
      /Users/parejkoj/lsst/lsstsw3/miniconda/lib/python3.5/site-packages/IPython/utils/dir2.py in get_real_method(obj, name)
           59     Returns the method or None.
           60     """
      ---> 61     if inspect.isclass(obj):
           62         return None
      /Users/parejkoj/lsst/lsstsw3/miniconda/lib/python3.5/inspect.py in isclass(object)
           76         __doc__         documentation string
           77         __module__      name of module in which this class was defined"""
      ---> 78     return isinstance(object, type)
           80 def ismethod(object):
      /Users/parejkoj/lsst/lsstsw3/stack/DarwinX86/daf_persistence/12.1+1/python/lsst/daf/persistence/readProxy.py in __getattribute__(self, attr, oga)
           41     def __getattribute__(self, attr, oga=object.__getattribute__):
      ---> 42         subject = oga(self, '__subject__')
           43         if attr == '__subject__':
           44             return subject
      /Users/parejkoj/lsst/lsstsw3/stack/DarwinX86/daf_persistence/12.1+1/python/lsst/daf/persistence/readProxy.py in __subject__(self, get_cache, set_cache)
          136         return get_cache(self)
          137     except AttributeError:
      --> 138         set_cache(self, get_callback(self)())
          139         return get_cache(self)
      /Users/parejkoj/lsst/lsstsw3/stack/DarwinX86/daf_persistence/12.1+1/python/lsst/daf/persistence/butler.py in <lambda>()
          605                     pythonType = getattr(importType, importClassString)
          606             bypassFunc = getattr(location.mapper, "bypass_" + datasetType)
      --> 607             callback = lambda: bypassFunc(datasetType, pythonType, location, dataId)
          608         else:
          609             callback = lambda: self._read(location)
      /Users/parejkoj/lsst/cameraGeom/obs_decam/python/lsst/obs/decam/decamMapper.py in bypass_ccdExposureId(self, datasetType, pythonType, location, dataId)
           91     def bypass_ccdExposureId(self, datasetType, pythonType, location, dataId):
      ---> 92         return self._computeCcdExposureId(dataId)
           94     def bypass_ccdExposureId_bits(self, datasetType, pythonType, location, dataId):
      /Users/parejkoj/lsst/cameraGeom/obs_decam/python/lsst/obs/decam/decamMapper.py in _computeCcdExposureId(self, dataId)
          102         copyId = self._transformId(dataId)
          103         visit = copyId['visit']
      --> 104         ccdnum = copyId['ccdnum']
          105         return int("%07d%02d" % (visit, ccdnum))
      /Users/parejkoj/lsst/lsstsw3/miniconda/lib/python3.5/collections/__init__.py in __getitem__(self, key)
          984         if hasattr(self.__class__, "__missing__"):
          985             return self.__class__.__missing__(self, key)
      --> 986         raise KeyError(key)
          987     def __setitem__(self, key, item): self.data[key] = item
          988     def __delitem__(self, key): del self.data[key]
      KeyError: 'ccdnum'


          Issue Links



              • Assignee:
                ktl Kian-Tat Lim
                Parejkoj John Parejko
                John Parejko, Kian-Tat Lim, Nate Pease, Simon Krughoff, Tim Jenness
              • Votes:
                0 Vote for this issue
                5 Start watching this issue


                • Created:

                  Summary Panel