Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5774

Provide more useful error message from stored config read failure

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_base
    • Labels:
      None
    • Templates:
    • Story Points:
      0.25
    • Sprint:
      DRP X16-2
    • Team:
      Data Release Production

      Description

      Sometimes in the course of development we change a Config class from what has been stored in a data repo. Subsequent runs then fail to read the Config file and produce a perplexing error message. Here's an example:

      reduceBias.py '/Users/azuri/spectra/pfs/PFS' --calib '/Users/azuri/spectra/pfs/PFS/CALIB' --output '/Users/azuri/spectra/pfs/PFS/CALIB/combined' --calibId calibVersion=bias arm=m --do-exec --id field=BIAS dateObs=2015-12-22 spectrograph=2 site=S category=A filter=m --nodes=1 --procs=1
      : Config override file does not exist: '/Users/azuri/stella-git/lsst/obs_subaru/config/bias.py'
      : Config override file does not exist: '/Users/azuri/stella-git/lsst/obs_subaru/config/pfs/bias.py'
      : input=/Users/azuri/spectra/pfs/PFS
      : calib=/Users/azuri/spectra/pfs/PFS/CALIB
      : output=/Users/azuri/spectra/pfs/PFS/CALIB/combined
      CameraMapper: Loading registry registry from /Users/azuri/spectra/pfs/PFS/CALIB/combined/_parent/registry.sqlite3
      CameraMapper: Unable to locate calibRegistry registry in root: /Users/azuri/spectra/pfs/PFS/CALIB/calibRegistry.sqlite3
      CameraMapper: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
      CameraMapper: Loading Posix registry from /Users/azuri/spectra/pfs/PFS/CALIB
      type(config) =  <class 'lsst.afw.cameraGeom.cameraConfig.CameraConfig'>
      CameraMapper: Loading registry registry from /Users/azuri/spectra/pfs/PFS/CALIB/combined/_parent/registry.sqlite3
      CameraMapper: Unable to locate calibRegistry registry in root: /Users/azuri/spectra/pfs/PFS/CALIB/calibRegistry.sqlite3
      CameraMapper: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
      CameraMapper: Loading Posix registry from /Users/azuri/spectra/pfs/PFS/CALIB
      type(config) =  <class 'lsst.afw.cameraGeom.cameraConfig.CameraConfig'>
      bias FATAL: Failed in task initialization: No module named detrends
      Traceback (most recent call last):
        File "/Users/azuri/lsstsw/stack/DarwinX86/pipe_base/2016_01.0-6-g7751869+1/python/lsst/pipe/base/cmdLineTask.py", line 285, in precall
          task.writeConfig(parsedCmd.butler, clobber=self.clobberConfig, doBackup=self.doBackup)
        File "/Users/azuri/lsstsw/stack/DarwinX86/pipe_base/2016_01.0-6-g7751869+1/python/lsst/pipe/base/cmdLineTask.py", line 494, in writeConfig
          oldConfig = butler.get(configName, immediate=True)
        File "/Users/azuri/lsstsw/stack/DarwinX86/daf_persistence/2016_01.0-8-geb625e2/python/lsst/daf/persistence/butler.py", line 370, in get
          return callback()
        File "/Users/azuri/lsstsw/stack/DarwinX86/daf_persistence/2016_01.0-8-geb625e2/python/lsst/daf/persistence/butler.py", line 365, in <lambda>
          callback = lambda: self._read(location)
        File "/Users/azuri/lsstsw/stack/DarwinX86/daf_persistence/2016_01.0-8-geb625e2/python/lsst/daf/persistence/butler.py", line 455, in _read
          results = location.repository.read(location)
        File "/Users/azuri/lsstsw/stack/DarwinX86/daf_persistence/2016_01.0-8-geb625e2/python/lsst/daf/persistence/repository.py", line 346, in read
          return self._access.read(butlerLocation)
        File "/Users/azuri/lsstsw/stack/DarwinX86/daf_persistence/2016_01.0-8-geb625e2/python/lsst/daf/persistence/access.py", line 125, in read
          return self.storage.read(butlerLocation=butlerLocation)
        File "/Users/azuri/lsstsw/stack/DarwinX86/daf_persistence/2016_01.0-8-geb625e2/python/lsst/daf/persistence/posixStorage.py", line 229, in read
          importType = __import__(importPackage, globals(), locals(), [importClassString], -1)
      ImportError: No module named detrends
      usage: reduceBias.py input [options]
      reduceBias.py: error: Error in task preparation
      

      In this case, it's because the detrends module has been renamed, but it's not obvious. We need a more helpful error message.

        Attachments

          Container Issues

            Activity

              People

              • Assignee:
                price Paul Price
                Reporter:
                price Paul Price
                Reviewers:
                Russell Owen
                Watchers:
                Andreas Ritter, Paul Price, Russell Owen
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel