Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ip_diffim
-
Labels:
-
Story Points:2
-
Epic Link:
-
Sprint:AP F19-5 (October), AP F19-6 (November), AP S20-1 (December)
-
Team:Alert Production
Description
Some images have been failing during image differencing using the DCR-matched template, that succeeded with a standard coadd template. These images appear to fail for patches covering the extreme edges of the area with data, but more investigation is needed. Example error message below:
|
apPipe INFO: Running ImageDifference...
|
apPipe.differencer INFO: Processing DataId(initialdata=\{'ccdnum': 44, 'filter': 'g', 'visit': 410929, 'date': '2015-02-17', 'object': 'Blind15A_40'}, tag=set()) |
apPipe.differencer.getTemplate INFO: Using skyMap tract 0 |
apPipe.differencer.getTemplate INFO: Assembling 6 coadd patches |
apPipe.differencer.getTemplate INFO: exposure dimensions=(2048, 4096); coadd dimensions=(5424, 2740) |
apPipe.differencer.getTemplate INFO: Constructing DCR-matched template for patch \{'datasetType': 'dcrCoadd_sub', 'bbox': Box2I(minimum=Point2I(60087, 34140), dimensions=Extent2I(13, 1960)), 'tract': 0, 'patch': '14,8', 'numSubfilters': 3} |
apPipe FATAL: Failed on dataId=\{'ccdnum': 44, 'filter': 'g', 'visit': 410929, 'date': '2015-02-17', 'hdu': 48, 'object': 'Blind15A_40'}: LengthError: |
File "src/image/Image.cc", line 89, in static lsst::afw::image::ImageBase<PixelT>::_view_t lsst::afw::image::ImageBase<PixelT>::_makeSubView(const Extent2I&, const Extent2I&, const _view_t&) [with PixelT = float; lsst::afw::image::ImageBase<PixelT>::_view_t = boost::gil::image_view<boost::gil::memory_based_2d_locator<boost::gil::memory_based_step_iterator<boost::gil::pixel<float, boost::gil::layout<boost::mpl::vector1<boost::gil::gray_color_t> > >*> > >; lsst::geom::Extent2I = lsst::geom::Extent<int, 2>] |
Box2I(Point2I(-60087,-34140),lsst::geom::Extent2I(0,0)) doesn't fit in image 13x1960 \{0} |
lsst::pex::exceptions::LengthError: 'Box2I(Point2I(-60087,-34140),lsst::geom::Extent2I(0,0)) doesn't fit in image 13x1960' |
|
/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pex_config/18.1.0-4-gc96e915/python/lsst/pex/config/config.py:1276: FutureWarning: Config field ccdProcessor.isr.doAddDistortionModel is deprecated: Camera geometry is incorporated when reading the raw files. This option no longer is used, and will be removed after v19. |
FutureWarning)
|
/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ap_association/18.1.0-8-g5734da6/python/lsst/ap/association/mapApData.py:185: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. |
table_list = list(yaml.load_all(yaml_stream)) |
/software/lsstsw/stack_20190330/python/miniconda3-4.5.12/envs/lsst-scipipe/lib/python3.7/site-packages/astropy/units/function/logarithmic.py:43: RuntimeWarning: invalid value encountered in log10 |
return dex.to(self._function_unit, np.log10(x)) |
Traceback (most recent call last):
|
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__ |
result = self.runTask(task, dataRef, kwargs) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask |
return task.runDataRef(dataRef, **kwargs) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/timer.py", line 150, in wrapper |
res = func(self, *args, **keyArgs) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ap_pipe/18.1.0-4-gcd8f52c+10/python/lsst/ap/pipe/ap_pipe.py", line 202, in runDataRef |
diffImResults = self.runDiffIm(calexpRef, templateIds) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/timer.py", line 150, in wrapper |
res = func(self, *args, **keyArgs) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ap_pipe/18.1.0-4-gcd8f52c+10/python/lsst/ap/pipe/ap_pipe.py", line 276, in runDiffIm |
return self.differencer.runDataRef(sensorRef, templateIdList=templateIds) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_base/18.1.0-4-g6c9d669+5/python/lsst/pipe/base/timer.py", line 150, in wrapper |
res = func(self, *args, **keyArgs) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/pipe_tasks/18.1.0-15-gc153667b/python/lsst/pipe/tasks/imageDifference.py", line 353, in runDataRef |
template = self.getTemplate.run(exposure, sensorRef, templateIdList=templateIdList) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/getTemplate.py", line 153, in run |
visitInfo=exposure.getInfo().getVisitInfo()) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/dcrModel.py", line 386, in buildMatchedExposure |
bbox=bbox, wcs=wcs, mask=mask) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/dcrModel.py", line 349, in buildMatchedTemplate |
refModel = self.getReferenceImage(bbox) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/dcrModel.py", line 274, in getReferenceImage |
return np.mean([model[bbox].array for model in self], axis=0) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/ip_diffim/18.1.0-9-gae7190a+1/python/lsst/ip/diffim/dcrModel.py", line 274, in <listcomp> |
return np.mean([model[bbox].array for model in self], axis=0) |
File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/afw/18.1.0-15-gbf79fb8fa/python/lsst/afw/image/slicing.py", line 262, in __getitem__ |
return self.subset(box, origin=origin) |
lsst.pex.exceptions.wrappers.LengthError:
|
File "src/image/Image.cc", line 89, in static lsst::afw::image::ImageBase<PixelT>::_view_t lsst::afw::image::ImageBase<PixelT>::_makeSubView(const Extent2I&, const Extent2I&, const _view_t&) [with PixelT = float; lsst::afw::image::ImageBase<PixelT>::_view_t = boost::gil::image_view<boost::gil::memory_based_2d_locator<boost::gil::memory_based_step_iterator<boost::gil::pixel<float, boost::gil::layout<boost::mpl::vector1<boost::gil::gray_color_t> > >*> > >; lsst::geom::Extent2I = lsst::geom::Extent<int, 2>] |
Box2I(Point2I(-60087,-34140),lsst::geom::Extent2I(0,0)) doesn't fit in image 13x1960 \{0} |
lsst::pex::exceptions::LengthError: 'Box2I(Point2I(-60087,-34140),lsst::geom::Extent2I(0,0)) doesn't fit in image 13x1960' |
Attachments
Issue Links
- relates to
-
DM-21230 Compare ap_pipe output using higher resolution templates
- Done
This is a simple fix for some rare warnings we've seen when running image differencing using DCR templates. On further investigation, it looks like the problem is that a failing patch only had a thin strip of pixels with data, which was smaller than the buffer we clip from the edges. The fix is to skip the patch and print a log message.