Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20929

Fix bug in DcrAssembleCoadd when running with slurm

    Details

    • Story Points:
      2
    • Epic Link:
    • Sprint:
      AP F19-2
    • Team:
      Alert Production

      Description

      DcrAssembleCoaddTask fails when running on Decam data on lsst-dev using slurm, though the same data runs fine using AssembleCoaddTask. I've copied the error below, which appears to be caused by the coaddInputs not being set up properly before trying to use them to compute a coaddPsf.

       

      dcrAssembleCoadd FATAL: Failed on dataId=DataId(initialdata={'filter': 'g', 'tract': 0, 'patch': '12,12'}, tag=set()): LengthError: 
        File "include/lsst/afw/table/Catalog.h", line 186, in lsst::afw::table::CatalogT<RecordT> lsst::afw::table::CatalogT<RecordT>::subset(const ndarray::Array<const bool, 1>&) const [with Recor
      dT = lsst::afw::table::ExposureRecord]
          Mask array with 4 elements applied to catalog with 8 elements {0}
      lsst::pex::exceptions::LengthError: 'Mask array with 4 elements applied to catalog with 8 elements'/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_tasks/18.0.0-8-g24ce6f0f+8/python/lsst/pipe/tasks/scaleVariance.py:147: RuntimeWarning: invalid value encountered 
      in sqrt
        snr = maskedImage.image.array/np.sqrt(variance.array)
      Traceback (most recent call last):
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.0.0-2-g0ee56d7+10/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
          result = self.runTask(task, dataRef, kwargs)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.0.0-2-g0ee56d7+10/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
          return task.runDataRef(dataRef, **kwargs)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.0.0-2-g0ee56d7+10/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_tasks/18.0.0-8-g24ce6f0f+8/python/lsst/pipe/tasks/dcrAssembleCoadd.py", line 294, in runDataRef
          warpRefList=warpRefList)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.0.0-2-g0ee56d7+10/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_tasks/18.0.0-8-g24ce6f0f+8/python/lsst/pipe/tasks/assembleCoadd.py", line 525, in runDataRef
          inputData.weightList, supplementaryData=supplementaryData)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_tasks/18.0.0-8-g24ce6f0f+8/python/lsst/pipe/tasks/dcrAssembleCoadd.py", line 471, in run
          dcrModels = self.prepareDcrInputs(templateCoadd, warpRefList, weightList)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_tasks/18.0.0-8-g24ce6f0f+8/python/lsst/pipe/tasks/dcrAssembleCoadd.py", line 395, in prepareDcrInputs
          psf = self.selectCoaddPsf(templateCoadd, warpRefList)
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_tasks/18.0.0-8-g24ce6f0f+8/python/lsst/pipe/tasks/dcrAssembleCoadd.py", line 1136, in selectCoaddPsf
          psf = measAlg.CoaddPsf(ccds[goodVisits], templateCoadd.getWcs(),
        File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/afw/18.0.0-10-g6da83990c/python/lsst/afw/table/_base.py", line 117, in __getitem__
          return self.subset(key)
      lsst.pex.exceptions.wrappers.LengthError: 
        File "include/lsst/afw/table/Catalog.h", line 186, in lsst::afw::table::CatalogT<RecordT> lsst::afw::table::CatalogT<RecordT>::subset(const ndarray::Array<const bool, 1>&) const [with Recor
      dT = lsst::afw::table::ExposureRecord]
          Mask array with 4 elements applied to catalog with 8 elements {0}
      lsst::pex::exceptions::LengthError: 'Mask array with 4 elements applied to catalog with 8 elements' 

       

        Attachments

          Issue Links

            Activity

            Hide
            sullivan Ian Sullivan added a comment -

            Pull request in pipe_tasks is here: https://github.com/lsst/pipe_tasks/pull/306

             

            Show
            sullivan Ian Sullivan added a comment - Pull request in pipe_tasks is here:  https://github.com/lsst/pipe_tasks/pull/306  
            Hide
            danielsf Scott Daniel added a comment -

            Looks good

            Show
            danielsf Scott Daniel added a comment - Looks good

              People

              • Assignee:
                sullivan Ian Sullivan
                Reporter:
                sullivan Ian Sullivan
                Reviewers:
                Scott Daniel
                Watchers:
                Ian Sullivan, Scott Daniel
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel