Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ip_isr
-
Labels:None
-
Story Points:0.5
-
Epic Link:
-
Sprint:DRP S17-2
-
Team:Data Release Production
Description
The ordinates for the overscan spline interpolation can violate the requirement of monotonic increasing in the presence of masked rows, causing GSL to reject it and we end up raising an exception. For example (notice the third element):
#1 0x00002aaadf6ccae4 in lsst::afw::math::InterpolateGsl::InterpolateGsl (
|
this=0x6f4ef0, x=std::vector of length 30, capacity 32 = {...},
|
y=std::vector of length 30, capacity 32 = {...},
|
style=lsst::afw::math::Interpolate::AKIMA_SPLINE)
|
at src/math/Interpolate.cc:211
|
211 int const status = ::gsl_interp_init(_interp, &x[0], &y[0], _y.size());
|
(gdb) p x
|
$19 = std::vector of length 30, capacity 32 = {-4.6668978729026289,
|
-1.0006934865900383, -2.2712418300653598, -0.76676245210727956,
|
-0.70019157088122619, -0.63362068965517238, -0.5668103448275863, -0.5,
|
-0.43342911877394619, -0.36685823754789276, -0.30028735632183906,
|
-0.23371647509578544, -0.1669061302681992, -0.10009578544061301,
|
-0.033524904214559385, 0.033045977011494247, 0.099616858237547928,
|
0.16618773946360157, 0.23299808429118776, 0.29980842911877403,
|
0.36637931034482762, 0.43295019157088127, 0.49952107279693497,
|
0.56609195402298851, 0.6329022988505747, 0.69971264367816088,
|
0.7662835249042147, 0.83285440613026807, 0.89942528735632199,
|
0.96623563218390807}
|
This is because the code to generate these ordinates (binning the overscan data vector) is incorrect.
Attachments
Activity
Wow, I'm amazed this has never bitten us (to the point of the exception) before. Nice catch!
Out of curiosity, is it the numPerBin > 0 clause on the HSC side here:
https://github.com/HyperSuprime-Cam/ip_isr/blob/master/python/lsst/ip/isr/isr.py#L447
that takes care of this?
I presume you will run Jenkins + ci_hsc...if/when it passes, feel free to merge (I suppose there is a very slim chance that some test values could get modified?)
Thanks, Lauren.
The code on the HSC side has a different approach (not using numpy.histogram to bin the ordinates). I believe I changed it because it doesn't account for masked rows in the ordinate.
I'll run ci_hsc now.
Paul Price
added a comment - Thanks, Lauren.
The code on the HSC side has a different approach (not using numpy.histogram to bin the ordinates). I believe I changed it because it doesn't account for masked rows in the ordinate.
I'll run ci_hsc now. This passed Jenkins.
Merged to master.
Paul Price
added a comment - This passed Jenkins .
Merged to master. 
Lauren MacArthur, would you review this change, please? I think we discussed this code at some point in the past, so you're probably the most familiar with it.
pprice@tiger-sumire:~/LSST/ip/isr (tickets/DM-8982=) $ git subcommit f72963fa68f0344683b1a3c729f38f18ebda6367Author: Paul Price <price@astro.princeton.edu>Date: Thu Jan 12 14:19:16 2017 -0500overscan: fix binning of spline ordinateThe binning was incorrect in the presence of masked rows, whichcould cause the ordinate to be non-monotonic increasing, resultingin an exception. The problem was that the masking wasn't beingapplied in the binning, but it was in the scaling (the 'numBins'vector).python/lsst/ip/isr/isr.py | 6 ++++--1 file changed, 4 insertions(+), 2 deletions(-)commit da9fcc3a3e9fef3572a70ffe2ecabf15bae8d8d2Author: Paul Price <price@astro.princeton.edu>Date: Thu Jan 12 14:19:31 2017 -0500overscan: plot bad data as well as goodThis helps us identify what was clipped.python/lsst/ip/isr/isr.py | 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)