Details
-
Type:
Improvement
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Story Points:8
-
Epic Link:
-
Team:Data Release Production
Description
Parallel overscan masking is still preventing potentially correctable amplifiers from being processed. Rework the masking to check per column (or per group of columns).
Attachments
Issue Links
Activity
I've uploaded an update: Craig Lage was correct that I had incorrectly put the masked-value "interpolation" fix in a function that wasn't being called by the MEDIAN_PER_ROW correction (but confusingly was run by the debug code). This resulted in the columns with bad bleeds continuing to have NaN values, causing the masking code to mask and interpolate these pixels (doing a poor job as there is also oversubtraction that pushed the interpolated values below the background level). I've fixed that, and although it's better (the crosstalk ghosts do not appear to have significant oversubtraction), the source bleed is still oversubtracting (~65 ADU on the shown image). Growing the input mask helps somewhat, with 2-pixel growth reducing the oversubtraction to ~25 ADU. I'm going to look at some bleeds, and see if there's a larger grow value that yields better results.
I've pushed the current version in case others want to look at it, and I'm hoping to have a cleaned up version ready tomorrow that can hopefully go into review.
I looked in more detail at that dark column in 2022121200752. The attached image Dark_Column... shows plots through the various regions. I concluded that the negative values in the PostISR image are not caused by a failure of the parallel overscan, but by the negative values that are already present in the raw values in the imaging region. I'm not sure what causes those.
Ignore this. I once again fell into the trap that the exposures get modified in place. The attached Dark_Column... is corrected and it doesn't really show anything we didn't know. It looks like we need to go +/- 3-4 columns to mask out that hot column.
I played with the grow parameter last night, and eventually settled on a 7 pixel growth radius. This still isn't perfect, but the oversubtraction is down to ~5ADU, which would be lost in the noise if the low pixels weren't correlated.
I'm going to try to get this into review, and have launched a Jenkins run to ensure there aren't any hidden errors.
Looks good pending final jenkins with ci_cpp.
It will need to be next week, but I'll make sure the MEDIAN_PER_ROW is getting the right thing (that's what I was testing with). I'll also add additional debug plots that demonstrate the results.