Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_tasks
-
Story Points:3
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
As discussed on this thread on Slack, while processing a small subset of the RC2 dataset on DM-30927, six instances (out of 3811 total) of the following error were encountered during imageDifference
File "/home/lauren/LSST/pipe_tasks/python/lsst/pipe/tasks/imageDifference.py", line 736, in run |
templateSigma = templateExposure.getPsf().computeShape().getDeterminantRadius() |
lsst.pex.exceptions.wrappers.InvalidParameterError:
|
File "src/CoaddPsf.cc", line 250, in virtual std::shared_ptr<lsst::afw::image::Image<double> > lsst::meas::al |
gorithms::CoaddPsf::doComputeKernelImage(const Point2D&, const lsst::afw::image::Color&) const
|
Cannot compute CoaddPsf at point (31923.8, 32218); no input images at that point. {0} |
lsst::pex::exceptions::InvalidParameterError: 'Cannot compute CoaddPsf at point (31923.8, 32218); no input imag |
es at that point.'
|
This stems from a call to getPsf().computeShape() without specifying a position, so the default and highly dangerous position is used, see RFC-796 (I believe this is happening near the edge of the coadd footprint, so there are many “template” images that will have very few actual good science pixels and whatever is being returned by the default position is an area where there is no real data). It seems we have gotten lucky so far with the full RC2/DC2 runs, but this subset has revealed that this is an issue that warrants fixing.
There are a few offenders in this file:
here and here (and, with the deprecation warning added on DM-31535, a recent processing run will reveal just how many such cases we have with the message:
FutureWarning: Default position argument overload is deprecated and will be removed in version 24.0. Please explicitly specify a position.)
Attachments
Issue Links
- is triggering
-
DM-32756 Template CoaddPsf Errors after DM-31777
- Done
- relates to
-
DM-32391 Gen3 DC2 reprocessing with bps and w_2021_44
- Done
-
DM-32549 Add position to psf.getDeterminantRadius() calls
- Done
-
DM-31795 Gen3 RC2 reprocessing with bps and w_2021_38
- Done
-
DM-32024 Gen3 DC2 reprocessing with bps and w_2021_40
- Done
-
RFC-796 Default Psf position argument considered harmful
- Adopted
- mentioned in
-
Page Loading...
The fix ended up to be to switch to load in the component coadd CoaddPsfs into template's CoaddPsf with the tractWcs. Previously, I was loading in the post-warped PSFs in the detector-WCS frame, and I now see I was abusing CoaddPsf. Sorry CoaddPsf
While I was in there I also gave the computeShape call in imageDifferenceTask a position.
Passes Jenkins: https://ci.lsst.codes/job/stack-os-matrix/35370/display/redirect
I ran a 3828/3829
$OBS_LSST_DIR/pipelines/imsim/DRP.yaml#getTemplate,imageDifference,transformDiaSourceCat,consolidateDiaSourceTable into u/yusra/DC2/
DM-31777-bps.(498966[w44] - 498851
DM-31777+ 174[new inDM-31777because previously failed )/ 498966[w44] = 0.000579