Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21237

BBox error in some DCR image differencing templates

    Details

    • Templates:
    • Story Points:
      2
    • Sprint:
      AP F19-5 (October), AP F19-6 (November), AP S20-1 (December)
    • Team:
      Alert Production

      Description

      Some images have been failing during image differencing using the DCR-matched template, that succeeded with a standard coadd template. These images appear to fail for patches covering the extreme edges of the area with data, but more investigation is needed. Example error message below:

       

       
      apPipe INFO: Running ImageDifference...
      apPipe.differencer INFO: Processing DataId(initialdata=\{'ccdnum': 44, 'filter': 'g', 'visit': 410929, 'date': '2015-02-17', 'object': 'Blind15A_40'}, tag=set())
      apPipe.differencer.getTemplate INFO: Using skyMap tract 0
      apPipe.differencer.getTemplate INFO: Assembling 6 coadd patches
      apPipe.differencer.getTemplate INFO: exposure dimensions=(2048, 4096); coadd dimensions=(5424, 2740)
      apPipe.differencer.getTemplate INFO: Constructing DCR-matched template for patch \{'datasetType': 'dcrCoadd_sub', 'bbox': Box2I(minimum=Point2I(60087, 34140), dimensions=Extent2I(13, 1960)), 'tract': 0, 'patch': '14,8', 'numSubfilters': 3}
      apPipe FATAL: Failed on dataId=\{'ccdnum': 44, 'filter': 'g', 'visit': 410929, 'date': '2015-02-17', 'hdu': 48, 'object': 'Blind15A_40'}: LengthError: 
       File "src/image/Image.cc", line 89, in static lsst::afw::image::ImageBase<PixelT>::_view_t lsst::afw::image::ImageBase<PixelT>::_makeSubView(const Extent2I&, const Extent2I&, const _view_t&) [with PixelT = float; lsst::afw::image::ImageBase<PixelT>::_view_t = boost::gil::image_view<boost::gil::memory_based_2d_locator<boost::gil::memory_based_step_iterator<boost::gil::pixel<float, boost::gil::layout<boost::mpl::vector1<boost::gil::gray_color_t> > >*> > >; lsst::geom::Extent2I = lsst::geom::Extent<int, 2>]
       Box2I(Point2I(-60087,-34140),lsst::geom::Extent2I(0,0)) doesn't fit in image 13x1960 \{0}
      lsst::pex::exceptions::LengthError: 'Box2I(Point2I(-60087,-34140),lsst::geom::Extent2I(0,0)) doesn't fit in image 13x1960'
       
      /software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pex_config/18.1.0-4-gc96e915/python/lsst/pex/config/config.py:1276: FutureWarning: Config field ccdProcessor.isr.doAddDistortionModel is deprecated: Camera geometry is incorporated when reading the raw files. This option no longer is used, and will be removed after v19.
       FutureWarning)
      /software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ap_association/18.1.0-8-g5734da6/python/lsst/ap/association/mapApData.py:185: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
       table_list = list(yaml.load_all(yaml_stream))
      /software/lsstsw/stack_20190330/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/site-packages/astropy/units/function/logarithmic.py:43: RuntimeWarning: invalid value encountered in log10
       return dex.to(self._function_unit, np.log10(x))
      Traceback (most recent call last):
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
       result = self.runTask(task, dataRef, kwargs)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
       return task.runDataRef(dataRef, **kwargs)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/timer.py", line 150, in wrapper
       res = func(self, *args, **keyArgs)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ap_pipe/18.1.0-4-gcd8f52c+10/python/lsst/ap/pipe/ap_pipe.py", line 202, in runDataRef
       diffImResults = self.runDiffIm(calexpRef, templateIds)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/timer.py", line 150, in wrapper
       res = func(self, *args, **keyArgs)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ap_pipe/18.1.0-4-gcd8f52c+10/python/lsst/ap/pipe/ap_pipe.py", line 276, in runDiffIm
       return self.differencer.runDataRef(sensorRef, templateIdList=templateIds)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/timer.py", line 150, in wrapper
       res = func(self, *args, **keyArgs)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_tasks/18.1.0-15-gc153667b/python/lsst/pipe/tasks/imageDifference.py", line 353, in runDataRef
       template = self.getTemplate.run(exposure, sensorRef, templateIdList=templateIdList)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/getTemplate.py", line 153, in run
       visitInfo=exposure.getInfo().getVisitInfo())
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/dcrModel.py", line 386, in buildMatchedExposure
       bbox=bbox, wcs=wcs, mask=mask)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/dcrModel.py", line 349, in buildMatchedTemplate
       refModel = self.getReferenceImage(bbox)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/dcrModel.py", line 274, in getReferenceImage
       return np.mean([model[bbox].array for model in self], axis=0)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/dcrModel.py", line 274, in <listcomp>
       return np.mean([model[bbox].array for model in self], axis=0)
       File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/afw/18.1.0-15-gbf79fb8fa/python/lsst/afw/image/slicing.py", line 262, in __getitem__
       return self.subset(box, origin=origin)
      lsst.pex.exceptions.wrappers.LengthError: 
       File "src/image/Image.cc", line 89, in static lsst::afw::image::ImageBase<PixelT>::_view_t lsst::afw::image::ImageBase<PixelT>::_makeSubView(const Extent2I&, const Extent2I&, const _view_t&) [with PixelT = float; lsst::afw::image::ImageBase<PixelT>::_view_t = boost::gil::image_view<boost::gil::memory_based_2d_locator<boost::gil::memory_based_step_iterator<boost::gil::pixel<float, boost::gil::layout<boost::mpl::vector1<boost::gil::gray_color_t> > >*> > >; lsst::geom::Extent2I = lsst::geom::Extent<int, 2>]
       Box2I(Point2I(-60087,-34140),lsst::geom::Extent2I(0,0)) doesn't fit in image 13x1960 \{0}
      lsst::pex::exceptions::LengthError: 'Box2I(Point2I(-60087,-34140),lsst::geom::Extent2I(0,0)) doesn't fit in image 13x1960'
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sullivan Ian Sullivan
                Reporter:
                sullivan Ian Sullivan
                Reviewers:
                Gabor Kovacs
                Watchers:
                Gabor Kovacs, Ian Sullivan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel