Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Story Points:4
-
Epic Link:
-
Sprint:DRP S18-6
-
Team:Data Release Production
Description
While running coaddDriver with the PDR1 data, using w_2018_15, I got 8 cases with the following IndexError
coaddDriver WARN: lsst-verify-worker23:129753: Caught IndexError while detection on DataId(initialdata={'tract': 16972, 'filter': 'HSC-G', 'patch': '0,7'}, tag=set()): cannot do a non-empty take from an empty axes.
|
coaddDriver INFO: lsst-verify-worker23:129753: Traceback:
|
Traceback (most recent call last):
|
File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/15.0-1-ga91101e+7/python/lsst/ctrl/pool/parallel.py", line 512, in logOperation
|
yield
|
File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_drivers/15.0-3-ga03b4ca+9/python/lsst/pipe/drivers/coaddDriver.py", line 321, in coadd
|
detResults = self.detectCoaddSources.runDetection(coadd, idFactory, expId=expId)
|
File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/15.0-5-g389937dc+5/python/lsst/pipe/tasks/multiBand.py", line 301, in runDetection
|
varScale = self.scaleVariance.run(exposure.maskedImage)
|
File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/15.0-5-g389937dc+5/python/lsst/pipe/tasks/scaleVariance.py", line 116, in run
|
factor = self.pixelBased(maskedImage)
|
File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/15.0-5-g389937dc+5/python/lsst/pipe/tasks/scaleVariance.py", line 155, in pixelBased
|
q1, q3 = np.percentile(snr[isGood], (25, 75))
|
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4269, in percentile
|
interpolation=interpolation)
|
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4011, in _ureduce
|
r = func(a, **kwargs)
|
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4386, in _percentile
|
x1 = take(ap, indices_below, axis=axis) * weights_below
|
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 134, in take
|
return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
|
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
|
return getattr(obj, method)(*args, **kwds)
|
IndexError: cannot do a non-empty take from an empty axes.
|
The traceback have similarities to but are not exactly the same as DM-13563 and DM-13517. The 8 cases include 2 in UDEEP, 5 in DEEP, and 1 in WIDE. To reproduce, their data IDs are:
UDEEP: --id tract=8766 filter=HSC-G patch=8,3 --selectId ccd=0..8^10..103 visit=9830^9832^9834^9836^9874^9878^11626^11628^11642^11644^11660^11662^42346^42348^42350^42352^42354^42356
|
UDEEP: --id tract=9571 filter=HSC-Y patch=7,7 --selectId ccd=0..8^10..103 visit=318^322^324^326^328^330^332^340^344^346^348^350^352^354^356^358^360^362^1856^1868^1870^1872^1874^1876^1880^1882^11716^11718^11720^11722^11724^11726^11728^11730^11732^11734^11736^11740^22604^22606^22608^22626^22628^22630^22632^22642^22644^22646^22648^22658^22660^22662^22664
|
DEEP: --id tract=9708 filter=HSC-G patch=7,6 --selectId ccd=0..8^10..103 visit=9796^9798^9812^9820^34488^34496^34504^34512
|
DEEP: --id tract=9812 filter=HSC-G patch=5,3 --selectId ccd=0..8^10..103 visit=29308^29310^29316^29318^29328^29332^29342^29346^29354^29358
|
DEEP: --id tract=8766 filter=HSC-I patch=0,5 --selectId ccd=0..8^10..103 visit=46860^46864^46866^14220^14222^14224^14226^14246^14248^14252^14254^14264^14266^14268
|
DEEP: --id tract=9707 filter=HSC-I patch=6,6 --selectId ccd=0..8^10..103 visit=7240^7242^7244^7246^7256^7258^7264^7266^35978^35982^35984^35988^35990^35996^35998^36004^36006
|
DEEP: --id tract=9707 filter=NB0816 patch=6,6 --selectId ccd=0..8^10..103 visit=37318^37320^37322^37324^37326^37328^37330^37796^37798^37800^38474
|
WIDE: --id tract=16972 filter=HSC-G patch=0,7 --selectId ccd=0..8^10..103 visit=29530^29536^29538^34210^34212^34214^34216^34218
|
One can run
coaddDriver.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun DM-13666/[DEEP|UDEEP|WIDE]:private/username/abcd ....
|
with the above IDs to reproduce.
Attachments
Issue Links
- relates to
-
- Done
Activity
This is a good fix, but I have suggested changes to the implementation in the GitHub PR.
Paul Price
added a comment - This is a good fix, but I have suggested changes to the implementation in the GitHub PR. 
Can you take a look at this change? The pull request is not showing up in jira yet, but it is up on github