Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23558

DCR coadds are missing PhotoCalib

    Details

    • Story Points:
      2
    • Sprint:
      AP S20-4 (March)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      ap_pipe fails for DCR coadds with the following error. This failure started in w_2020_07 but was fine in w_2020_06. The error below was from a run with a coadd made using w.2020.06-6-g81841ad or pipe_tasks, while the failed ap_pipe run used w.2020.07

       

      // apPipe INFO: Running ImageDifference...
      apPipe.differencer INFO: Processing DataId(initialdata={'ccdnum': 23, 'filter': 'g', 'visit': 411037, 'date': '2015-02-17', 'object': 'Blind15A_42'}, tag=set())
      apPipe.differencer.getTemplate INFO: Using skyMap tract 0
      apPipe.differencer.getTemplate INFO: Assembling 4 coadd patches
      apPipe.differencer.getTemplate INFO: exposure dimensions=(2048, 4096); coadd dimensions=(4175, 2116)
      apPipe.differencer.getTemplate INFO: Constructing DCR-matched template for patch {'datasetType': 'dcrCoadd_sub', 'bbox': Box2I(minimum=Point2I(46369, 55694), dimensions=Extent2I(1731, 406)), 'tract': 0, 'patch': '11,13', 'numSubfil
      ters': 3}
      apPipe.differencer.getTemplate INFO: Constructing DCR-matched template for patch {'datasetType': 'dcrCoadd_sub', 'bbox': Box2I(minimum=Point2I(46369, 55900), dimensions=Extent2I(1731, 1910)), 'tract': 0, 'patch': '11,14', 'numSubfi
      lters': 3}
      apPipe.differencer.getTemplate WARN: dcrCoadd_sub, tract=0, patch=12,13, numSubfilters=3, subfilter=0 does not exist
      apPipe.differencer.getTemplate WARN: dcrCoadd_sub, tract=0, patch=12,14, numSubfilters=3, subfilter=0 does not exist
      apPipe FATAL: Failed on dataId={'ccdnum': 23, 'filter': 'g', 'visit': 411037, 'date': '2015-02-17', 'hdu': 21, 'object': 'Blind15A_42'}: RuntimeError: No coadd PhotoCalib found!
      /software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/obs_decam/19.0.0-6-gb5bb71e+9/config/isr.py:95: FutureWarning: Config field ccdProcessor.isr.doAddDistortionModel is deprecated: Camera geometry is incorporate
      d when reading the raw files. This option no longer is used, and will be removed after v19.
        config.doAddDistortionModel = False  # rely on the TPV terms instead
      /software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/ap_association/19.0.0-13-g744ec97+1/python/lsst/ap/association/mapApData.py:185: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as 
      the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
        table_list = list(yaml.load_all(yaml_stream))
      /software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/pex_config/19.0.0-5-g9aa49c1/python/lsst/pex/config/configurableField.py:150: FutureWarning: Config field ccdProcessor.isr.doAddDistortionModel is deprecated: 
      Camera geometry is incorporated when reading the raw files. This option no longer is used, and will be removed after v19.
        self._value.__setattr__(name, value, at=at, label=label)
      /software/lsstsw/stack_20191101/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/site-packages/astropy/units/function/logarithmic.py:46: RuntimeWarning: invalid value encountered in log10
        return dex.to(self._function_unit, np.log10(x))
      Traceback (most recent call last):
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/pipe_base/19.0.0-9-g0ae078d/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
          result = self.runTask(task, dataRef, kwargs)
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/pipe_base/19.0.0-9-g0ae078d/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
          return task.runDataRef(dataRef, **kwargs)
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/pipe_base/19.0.0-9-g0ae078d/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/ap_pipe/19.0.0-6-g8dfed85+1/python/lsst/ap/pipe/ap_pipe.py", line 165, in runDataRef
          diffImResults = self.runDiffIm(calexpRef, templateIds)
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/pipe_base/19.0.0-9-g0ae078d/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/ap_pipe/19.0.0-6-g8dfed85+1/python/lsst/ap/pipe/ap_pipe.py", line 238, in runDiffIm
          return self.differencer.runDataRef(sensorRef, templateIdList=templateIds)
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/pipe_base/19.0.0-9-g0ae078d/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/pipe_tasks/19.0.0-27-g2f87fd56/python/lsst/pipe/tasks/imageDifference.py", line 388, in runDataRef
          template = self.getTemplate.run(exposure, sensorRef, templateIdList=templateIdList)
        File "/software/lsstsw/stack_20191101/stack/miniconda3-4.5.12-4d7b902/Linux64/ip_diffim/19.0.0-7-g2f7a0e4+1/python/lsst/ip/diffim/getTemplate.py", line 186, in run
          raise RuntimeError("No coadd PhotoCalib found!")
      RuntimeError: No coadd PhotoCalib found!
      

        Attachments

          Activity

          Hide
          sullivan Ian Sullivan added a comment -

          Fairly simple changes to add PhotoCalib to DcrCoadds. I've verified that this fixes the original bug on my machine.

           

          Pull requests are:

          https://github.com/lsst/ip_diffim/pull/149

          https://github.com/lsst/pipe_tasks/pull/357

          Show
          sullivan Ian Sullivan added a comment - Fairly simple changes to add PhotoCalib to DcrCoadds. I've verified that this fixes the original bug on my machine.   Pull requests are: https://github.com/lsst/ip_diffim/pull/149 https://github.com/lsst/pipe_tasks/pull/357
          Hide
          Parejkoj John Parejko added a comment -

          One comment/question on the ip_diffim changes.

          Show
          Parejkoj John Parejko added a comment - One comment/question on the ip_diffim changes.

            People

            • Assignee:
              sullivan Ian Sullivan
              Reporter:
              sullivan Ian Sullivan
              Reviewers:
              John Parejko
              Watchers:
              Ian Sullivan, John Parejko
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel