Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: meas_deblender
-
Labels:None
-
Story Points:1
-
Team:Data Release Production
Description
In a validation run for hscPipe 5.0-beta5:
pprice@tiger-sumire:/tigress/pprice/hscPipe-5.0-beta5 $ measureCoaddSources.py /tigress/HSC/HSC --rerun hscPipe-5.0-beta5/20170517/wide --id tract=8764 patch=0,0 filter=HSC-I
|
root INFO: Loading config overrride file '/tigress/HSC/stack/hsc-tiger-20170322/Linux64/obs_subaru/13.0-18-gbe0b5c0/config/measureCoaddSources.py'
|
root INFO: Loading config overrride file '/tigress/HSC/stack/hsc-tiger-20170322/Linux64/obs_subaru/13.0-18-gbe0b5c0/config/hsc/measureCoaddSources.py'
|
root INFO: Running: /tigress/pprice/hscPipe-5.0-beta5/pipe_tasks/bin/measureCoaddSources.py /tigress/HSC/HSC --rerun hscPipe-5.0-beta5/20170517/wide --id tract=8764 patch=0,0 filter=HSC-I
|
/tigress/HSC/stack/hsc-tiger-20170322/Linux64/miniconda2/4.2.12.lsst1/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
|
warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
|
measureCoaddSources INFO: Read 7020 detections: DataId(initialdata={'filter': 'HSC-I', 'patch': '0,0', 'tract': 8764}, tag=set([]))
|
measureCoaddSources.deblend INFO: Deblending 7020 sources
|
measureCoaddSources FATAL: Failed on dataId=DataId(initialdata={'filter': 'HSC-I', 'patch': '0,0', 'tract': 8764}, tag=set([])):
|
Traceback (most recent call last):
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/pipe_base/13.0+10/python/lsst/pipe/base/cmdLineTask.py", line 347, in __call__
|
result = task.run(dataRef, **kwargs)
|
File "/tigress/pprice/hscPipe-5.0-beta5/pipe_tasks/python/lsst/pipe/tasks/multiBand.py", line 1086, in run
|
self.deblend.run(exposure, sources)
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/pipe_base/13.0+10/python/lsst/pipe/base/timer.py", line 121, in wrapper
|
res = func(self, *args, **keyArgs)
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/meas_deblender/13.0-3-gdadfd13+1/python/lsst/meas/deblender/deblend.py", line 238, in run
|
self.deblend(exposure, sources, psf)
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/pipe_base/13.0+10/python/lsst/pipe/base/timer.py", line 121, in wrapper
|
res = func(self, *args, **keyArgs)
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/meas_deblender/13.0-3-gdadfd13+1/python/lsst/meas/deblender/deblend.py", line 323, in deblend
|
medianSmoothTemplate=self.config.medianSmoothTemplate
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/meas_deblender/13.0-3-gdadfd13+1/python/lsst/meas/deblender/baseline.py", line 667, in deblend
|
debResult = newDeblend(debPlugins, footprint, maskedImage, psf, psffwhm, filters, log, verbose, avgNoise)
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/meas_deblender/13.0-3-gdadfd13+1/python/lsst/meas/deblender/baseline.py", line 739, in newDeblend
|
reset = debPlugins[step].run(debResult, log)
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/meas_deblender/13.0-3-gdadfd13+1/python/lsst/meas/deblender/plugins.py", line 76, in run
|
reset = self.func(debResult, log, **self.kwargs)
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/meas_deblender/13.0-3-gdadfd13+1/python/lsst/meas/deblender/plugins.py", line 152, in fitPsfs
|
dp.img, dp.varimg, psfChisqCut1, psfChisqCut2, psfChisqCut2b, tinyFootprintSize)
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/meas_deblender/13.0-3-gdadfd13+1/python/lsst/meas/deblender/plugins.py", line 371, in _fitPsf
|
sx1, sx2, sx3, sx4 = _overlap(xlo, xhi, obb.getMinX(), obb.getMaxX())
|
File "/tigress/HSC/stack/hsc-tiger-20170322/Linux64/meas_deblender/13.0-3-gdadfd13+1/python/lsst/meas/deblender/plugins.py", line 319, in _overlap
|
(xlo <= xhi) and (xmin <= xmax))
|
AssertionError
|
After a bit of poking, I believe it's due to the use of LOCAL here.
Attachments
Activity
Hsin-Fang Chiang, since you said you also ran into this, would you mind having a quick look? The patch is above — real simple.
It's currently going through Jenkins.
Paul Price
added a comment - - edited Hsin-Fang Chiang , since you said you also ran into this, would you mind having a quick look? The patch is above — real simple.
It's currently going through Jenkins. Ack! This breaks the demo, presumably because the small changes to the deblender produce small differences in the output. I'll update the demo.
Paul Price
added a comment - Ack! This breaks the demo , presumably because the small changes to the deblender produce small differences in the output. I'll update the demo. Looks good to me.
I wasn't able to build this with w_2017_17. So I tried w_2017_20 and reproduced the same error with a patch in VVDS. With your fix, it successfully finished (after spending a while measuring 22091 sources!)
Hsin-Fang Chiang
added a comment - Looks good to me.
I wasn't able to build this with w_2017_17. So I tried w_2017_20 and reproduced the same error with a patch in VVDS. With your fix, it successfully finished (after spending a while measuring 22091 sources!) Updated the demo, and now a new Jenkins is running...
pprice@tiger-sumire:~/LSST/lsst_dm_stack_demo (tickets/DM-10574=) $ git sub
|
commit 7789524c69f40bd8feefc0a0af2995f2e9e17c35
|
Author: Paul Price <price@astro.princeton.edu>
|
Date: Mon May 22 21:49:52 2017 -0400
|
|
|
update expected results following deblender changes
|
|
DM-10574 fixed a deblender bug that, when fixed, changes how
|
the deblender works ever so slightly, requiring updated results
|
for the demo.
|
|
|
expected/DarwinX86/detected-sources.txt | 6614 ++++++++++++-------------
|
expected/DarwinX86/detected-sources_small.txt | 5032 +++++++++----------
|
expected/Linux64/detected-sources.txt | 6614 ++++++++++++-------------
|
expected/Linux64/detected-sources_small.txt | 5032 +++++++++----------
|
4 files changed, 11646 insertions(+), 11646 deletions(-)
|
Paul Price
added a comment - Updated the demo, and now a new Jenkins is running...
pprice@tiger-sumire:~/LSST/lsst_dm_stack_demo (tickets/DM-10574=) $ git sub
commit 7789524c69f40bd8feefc0a0af2995f2e9e17c35
Author: Paul Price <price@astro.princeton.edu>
Date: Mon May 22 21:49:52 2017 -0400
update expected results following deblender changes
DM-10574 fixed a deblender bug that, when fixed, changes how
the deblender works ever so slightly, requiring updated results
for the demo.
expected/DarwinX86/detected-sources.txt | 6614 ++++++++++++-------------
expected/DarwinX86/detected-sources_small.txt | 5032 +++++++++----------
expected/Linux64/detected-sources.txt | 6614 ++++++++++++-------------
expected/Linux64/detected-sources_small.txt | 5032 +++++++++----------
4 files changed, 11646 insertions(+), 11646 deletions(-)
Well, that Jenkins failed. I believe it's because it's not looking at the ticket branch. John Swinbank writes:
Jenkins will only run the demo on master: you have to test it manually, merge to master, and stand by to fix it if it breaks.
Having tested it manually, I'm about to merge to master and then will re-run Jenkins.
Paul Price
added a comment - Well, that Jenkins failed. I believe it's because it's not looking at the ticket branch. John Swinbank writes:
Jenkins will only run the demo on master: you have to test it manually, merge to master, and stand by to fix it if it breaks.
Having tested it manually, I'm about to merge to master and then will re-run Jenkins. Merged, and Jenkins is running on the new master...
Paul Price
added a comment - Merged, and Jenkins is running on the new master... Jenkins passed, and all is well.
Paul Price
added a comment - Jenkins passed, and all is well. 
With LOCAL removed, it works just fine.
price@pap-laptop:~/LSST/meas_deblender (tickets/DM-10574=) $ git sub-patchcommit dba71810921104b96f8ca10ed7d402f5abf678b3Author: Paul Price <price@astro.princeton.edu>Date: Mon May 22 14:51:21 2017 -0400fitPsf: fix non-overlapping bounding boxesWas getting an AssertionError from the _overlap internal functionbecause the bounding boxes didn't overlap at all. This was surprisingbecause the list of peaks should overlap by construction. On closerinspection, found that at construction time the LOCAL bounding box waschecked, but the assertion uses the default frame (which is now PARENT).Switched to using the default (PARENT) frame for construction forconsistency (LOCAL isn't used anywhere else in the deblender).diff --git a/python/lsst/meas/deblender/plugins.py b/python/lsst/meas/deblender/plugins.pyindex b00c8a2..ddaad87 100755--- a/python/lsst/meas/deblender/plugins.py+++ b/python/lsst/meas/deblender/plugins.py@@ -279,7 +279,7 @@ def _fitPsf(fp, fmask, pk, pkF, pkres, fbb, peaks, peaksF, log, psf, psffwhm,if pkF.distanceSquared(pkF2) > R2**2:continueopsfimg = psf.computeImage(pkF2.getX(), pkF2.getY())- if not opsfimg.getBBox(afwImage.LOCAL).overlaps(stampbb):+ if not opsfimg.getBBox().overlaps(stampbb):continueotherpeaks.append(opsfimg)log.trace('%i other peaks within range', len(otherpeaks))