# ip_isr is not handling BAD pixels as expected

XMLWordPrintable

#### Details

• Type: Improvement
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
6
• Team:
Data Release Production

#### Description

As a result of the interpolation change (DM-19382), BAD pixels are now only being reset to the image median after interpolation.  However, these pixels are interpolated, and because of that, will not be set to the image median.  This shouldn't significantly change image values, but the order is not correct.

#### Activity

Hide
Colin Slater added a comment -

Looks fine to me, but I'm not sure I understand why we're resetting bad pixels and then interpolating them both, and I don't know how I would find out why we're doing that. Are we collecting this ISR knowledge anywhere? (cc: Yusra AlSayyad or Jim Bosch?)

Show
Colin Slater added a comment - Looks fine to me, but I'm not sure I understand why we're resetting bad pixels and then interpolating them both, and I don't know how I would find out why we're doing that. Are we collecting this ISR knowledge anywhere? (cc:  Yusra AlSayyad or Jim Bosch ?)
Hide

I don't know. Will this fix: https://jira.lsstcorp.org/browse/DM-19841 too?

Show
Yusra AlSayyad added a comment - I don't know. Will this fix: https://jira.lsstcorp.org/browse/DM-19841 too?
Hide
Christopher Waters added a comment -

I don't know yet.  My suspicion that this was the case prompted me to look into this (finding if nothing else the interpolation exposure bug).  Looking at the CORR for the image cited on that ticket (visit=130334/ccd=033), the data drops from the high value in amp1 approximately linearly over the transition to amp2.  My thought was that if BAD pixels aren't set to a value prior to interpolating, that it might skew the one side of the transition, resulting in what's seen.  However, most of the linear transition does not appear to have the INTRP bit set, so I'm not sure where the data values for that region come from.

I'd like to run this flagged exposure through runIsr.py myself to test, but I'm having difficulty getting the HSC archive to give it to me.  I may spot check a few other ccd=033 images that we have on disk to see if they show a similar feature.

Show
Christopher Waters added a comment - I don't know yet.  My suspicion that this was the case prompted me to look into this (finding if nothing else the interpolation exposure bug).  Looking at the CORR for the image cited on that ticket (visit=130334/ccd=033), the data drops from the high value in amp1 approximately linearly over the transition to amp2.  My thought was that if BAD pixels aren't set to a value prior to interpolating, that it might skew the one side of the transition, resulting in what's seen.  However, most of the linear transition does not appear to have the INTRP bit set, so I'm not sure where the data values for that region come from. I'd like to run this flagged exposure through runIsr.py myself to test, but I'm having difficulty getting the HSC archive to give it to me.  I may spot check a few other ccd=033 images that we have on disk to see if they show a similar feature.
Hide
Christopher Waters added a comment -

I made a comment last week on DM-20090, but the updated version of this ticket branch should resolve DM-19841 entirely.

We were not marking bad amplifiers as BAD, allowing them to be used to set the interpolation.  Marking them as such now prevents the bright amplifier edge issue, and setting the value to the image median keeps the image scaling reasonable.

As for the choice between interpolation and median setting, the logic to me is:

• Large contiguous areas are reset, as there is no useful information there, and values on either side of the area have little relation to what happens in between.
• Small areas (saturation spikes, bright columns) should be interpolated, as the image data on either side of the bad pixels should be similar to the expected value in the bad region.

I'll admit this is something of an ex-post facto justification for what we do, but I will add something to this effect into the comments of isrTask, so there's an explanation for the future.

Show
Christopher Waters added a comment - I made a comment last week on DM-20090 , but the updated version of this ticket branch should resolve DM-19841 entirely. We were not marking bad amplifiers as BAD, allowing them to be used to set the interpolation.  Marking them as such now prevents the bright amplifier edge issue, and setting the value to the image median keeps the image scaling reasonable. As for the choice between interpolation and median setting, the logic to me is: Large contiguous areas are reset, as there is no useful information there, and values on either side of the area have little relation to what happens in between. Small areas (saturation spikes, bright columns) should be interpolated, as the image data on either side of the bad pixels should be similar to the expected value in the bad region. I'll admit this is something of an ex-post facto justification for what we do, but I will add something to this effect into the comments of isrTask, so there's an explanation for the future.
Hide
John Swinbank added a comment -

Hi folks — a reminder to please make sure ticketed work has story points and epics attached. Thanks!

Show
John Swinbank added a comment - Hi folks — a reminder to please make sure ticketed work has story points and epics attached. Thanks!

#### People

Assignee:
Christopher Waters
Reporter:
Christopher Waters
Reviewers:
Colin Slater
Watchers:
Christopher Waters, Colin Slater, John Swinbank, Yusra AlSayyad