Details

Type: Story

Status: Done

Resolution: Done

Fix Version/s: None

Component/s: ip_isr

Labels:

Story Points:5

Epic Link:

Sprint:DRP S21a (Dec Jan)

Team:Data Release Production

Urgent?:No
Description
It seems some of our calexps coming out of single frame processing have nonfinite values, including NaNs and infs. Neither are marked with the UNMASKEDNAN mask nor are they interpolated and replaced with finite values, as would be expected.
To get an idea of the frequency of this occurrence, considering just the COSMOS visits included in the HSC RC2 dataset:
COSMOS_G nVisits: 17


COSMOS_R nVisits: 16

There are 2187 NaN pixels in HSCR 23704 102

There are 949 NaN pixels in HSCR 23706 102


COSMOS_I nVisits: 33

There are 5526 NaN pixels in HSCI 30494 100


COSMOS_Z nVisits: 31

There are 4320 NaN pixels in HSCZ 1166 100

There are 1899 NaN pixels in HSCZ 1168 102

There are 1464 NaN pixels in HSCZ 1176 102

There are 643 NaN pixels in HSCZ 1192 102

There are 6 NaN pixels in HSCZ 1194 101

There are 4892 NaN pixels in HSCZ 17950 100


COSMOS_Y nVisits: 52

There are 689 NaN pixels in HSCY 11736 101

So of (17 + 16 + 33 + 31 + 52)*103 = 15347 CCDs, 10 of them have NaN pixels (I wasn't actually looking for Inf when I ran the script, and it turns out they can creep in too, so there could be a handful more with nonfinite values).
It is likely of note that they are all outer corner CCDs (all with odd nQuarter rotations...but I'm really hoping this is not a rotated CCD issue...)
For a bit more context, see discussion at:
https://lsstc.slack.com/archives/C4JQP6FRS/p1615848617018600?thread_ts=1615848582.018500&cid=C4JQP6FRS
But this is the gist, using visit HSCI 30494 100 as an example:
The mess at top rightofcenter has 5526 NaN and 128 inf pixels.
Here are the mask planes:
So, not only are nonfinite values creeping in, they are not getting the (most unfortunately named) UNMASKEDNAN bit set. The only mask plane that includes the mess is NO_DATA.
I have looked at the calib images (bias, dark, flat), and they all look "fine", so this is either present at "raw", or introduced by some other method in ip_isr.