Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ip_isr
-
Labels:None
-
Team:DM Science
Description
The code to estimate overscan levels do not take enough care to handle low signal levels. Once the noise level is small enough a simple median does not return an estimate of population median (think about the case that the pixels have equal probability of being either 1 or 2; the population median is 1.5, but the median will be 1 or 2). With enough noise this isn't much of a problem as the noise provides dithering; even then there can be biases based on the noise's pdf.
For bias levels this is an issue; for example the bias-subtracted over-scan bias frames from auxTel (taken on 2018-07-21) have amplifier medians that differ by significant fractions of a DN – and that's enough to mess up faint light measurements as some of the effect is in overscan correction used to build the bias frames.
Using a mean instead of a median "solves" this problem, but at the expense of being sensitive to CR hits (or other outliers) in the overscan.
One simple solution to the median problem is to find the mean of sets of pixels, then estimate the median of the result (this essentially raises the noise level); another — which we used in SDSS — is to form a histogram and estimate its median. This should be implemented in afwMath.Statistics as a robust median; adding a clipping option to afwMath.Statistics would also probably be fine.
The issue with a non-robust mean can be resolved by exposing the StatisticsCtrl argument and using a clipped mean.