Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_tasks
-
Story Points:3
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
As discussed on this thread on Slack, while processing a small subset of the RC2 dataset on DM-30927, six instances (out of 3811 total) of the following error were encountered during imageDifference
File "/home/lauren/LSST/pipe_tasks/python/lsst/pipe/tasks/imageDifference.py", line 736, in run |
templateSigma = templateExposure.getPsf().computeShape().getDeterminantRadius() |
lsst.pex.exceptions.wrappers.InvalidParameterError:
|
File "src/CoaddPsf.cc", line 250, in virtual std::shared_ptr<lsst::afw::image::Image<double> > lsst::meas::al |
gorithms::CoaddPsf::doComputeKernelImage(const Point2D&, const lsst::afw::image::Color&) const
|
Cannot compute CoaddPsf at point (31923.8, 32218); no input images at that point. {0} |
lsst::pex::exceptions::InvalidParameterError: 'Cannot compute CoaddPsf at point (31923.8, 32218); no input imag |
es at that point.'
|
This stems from a call to getPsf().computeShape() without specifying a position, so the default and highly dangerous position is used, see RFC-796 (I believe this is happening near the edge of the coadd footprint, so there are many “template” images that will have very few actual good science pixels and whatever is being returned by the default position is an area where there is no real data). It seems we have gotten lucky so far with the full RC2/DC2 runs, but this subset has revealed that this is an issue that warrants fixing.
There are a few offenders in this file:
here and here (and, with the deprecation warning added on DM-31535, a recent processing run will reveal just how many such cases we have with the message:
FutureWarning: Default position argument overload is deprecated and will be removed in version 24.0. Please explicitly specify a position.)
Attachments
Issue Links
- is triggering
-
- Done
- relates to
-
- Done
-
- Done
-
- Done
-
- Done
-
- Adopted
- mentioned in
-
Page Loading...
Activity
I was planning to create a ticket and fix these, but I could hold off if it was easier to just fix all the instances together on this ticket.
Please make a new ticket.
It has nothing to do with RFC-796
Recommended a bit more dead code removal on ip_diffim PR. Otherwise looks good.
This ticket patch was merged before w_2021_46; is the outcome of step4 failures expected to change?
From the comments of HSC-RC2 w_2021_46 versus w_2021_42 they had same number of failures in imageDifference. And one of the six dataIds, visit=23718 ccd=30, was in w_2021_46's failure list.
Just wanted to double check what's expected.
Hsin-Fang Chiang
added a comment - This ticket patch was merged before w_2021_46; is the outcome of step4 failures expected to change?
From the comments of HSC-RC2 w_2021_46 versus w_2021_42 they had same number of failures in imageDifference. And one of the six dataIds, visit=23718 ccd=30, was in w_2021_46's failure list.
Just wanted to double check what's expected. Nope. Not what I was expecting. Filing a ticket.
The fix ended up to be to switch to load in the component coadd CoaddPsfs into template's CoaddPsf with the tractWcs. Previously, I was loading in the post-warped PSFs in the detector-WCS frame, and I now see I was abusing CoaddPsf. Sorry CoaddPsf
While I was in there I also gave the computeShape call in imageDifferenceTask a position.
Passes Jenkins: https://ci.lsst.codes/job/stack-os-matrix/35370/display/redirect
I ran a 3828/3829
$OBS_LSST_DIR/pipelines/imsim/DRP.yaml#getTemplate,imageDifference,transformDiaSourceCat,consolidateDiaSourceTable into u/yusra/DC2/
DM-31777-bps.(498966[w44] - 498851
DM-31777+ 174[new inDM-31777because previously failed )/ 498966[w44] = 0.000579