Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10443

Butler sometimes looks up calibDate from filesystem not calibration registry

    Details

    • Templates:
    • Story Points:
      1
    • Team:
      External

      Description

      We've just been debugging some strange calibration registry problems. Specifically, we have exactly one master bias which has been created but not registered.

      in obs_base mapping.py:224 there is a call to

      self.registry.lookup(properties, self.tables, lookupDataId, template=self.template)
      

      and stepping into the call reveals that:

      > /ssd/lsstsw/stack_20170409/Linux64/daf_persistence/13.0-6-gf146911/python/lsst/daf/persistence/registries.py(180)lookup()
          179 
      --> 180     def lookup(self, lookupProperties, reference, dataId, **kwargs):
          181         """Perform a lookup in the registry.
       
      ipdb> a
      self = <lsst.daf.persistence.registries.PosixRegistry object at 0x7f9359ccf590>
      lookupProperties = [u'calibDate']
      reference = [u'bias']
      dataId = {(u'validStart', u'validEnd'): '2016-08-22T22:45:00.921'}
      kwargs = {'template': u'bias/%(calibDate)s/bias-%(calibDate)s.fits.gz'}
      

      (the template is of course set in the .paf file).

      The butler merrily calls PosixRegistry.LookupData(lookupProperties, dataId) which resolves to the single file that matches — and this means that it uses a regexp to look up the calibDate.

      This is not what we wanted; the bias is a calibration type (as stated in the paf file) with a validity range which has been ignored.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  price Paul Price
                  Reporter:
                  mfisherlevine Merlin Fisher-Levine
                  Reviewers:
                  Nate Pease
                  Watchers:
                  Fritz Mueller, John Swinbank, Merlin Fisher-Levine, Nate Pease, Paul Price, Robert Lupton
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  6 Start watching this issue

                  Dates

                  • Created:
                    Updated:

                    Summary Panel