Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ip_isr
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
Edge effects in some exposures (example {exposure=36118, ccd=87}) can create a negative variance. The process of this is:
- Image data is unusually low (example has 0.0 values for the lower ~84 rows).
- Overscan is fit with a spline, and cannot fit these points (which may also be excluded from the spline fit for being too discrepant).
- Resulting overscan value in that region is higher than raw image value, resulting in a negative value prior to the variance calculation.
- Variance calculation is just I / G + RN^2 which propagates the negative to the variance.
These low pixels need to be masked. One option is to propagate the overscan excluded rows back so they can be masked. This will eliminate the majority of points, but does not guarantee that non-zero points ill fit by the overscan spline will also be removed (the points in the ramp-down region of the figure that will still result in negative variance).
This may need a config option to enable, as we should not mask negative image values while creating biases.
I've added code to allow pixels with negative variance to masked with a specified plane (defaults to BAD). This is enabled by default, as in most circumstances we wouldn't expect any negative variance pixels.
Assuming no hidden issues in `ci_hsc`, this should succeed:
https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34764/pipelinehttps://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34765/pipeline