Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ip_isr
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
If there are masked pixels in the overscan, the MEDIAN_PER_ROW overscan correction code generates a warning:
python/lsst/ip/isr/overscan.py:344: UserWarning: Warning: converting a masked element to nan.
|
rowMedian = afwMath.makeStatistics(row, fitType, self.statControl).getValue()
|
Please remove this. I think the simplest fix is row = rowrow.mask.data[0] in collapseArrayMedian, but there may be other occurrences. A better solution might be to stop using np.ma in favour of using stack-native code everywhere, which understands Mask planes.
Attachments
Activity
| Field | Original Value | New Value |
|---|---|---|
| Team | Data Release Production [ 10301 ] |
| Description |
If there are masked pixels in the overscan, the \{\{MEDIAN_PER_ROW}} overscan correction code generates a warning:
\{\{python/lsst/ip/isr/overscan.py:344: UserWarning: Warning: converting a masked element to nan. rowMedian = afwMath.makeStatistics(row, fitType, self.statControl).getValue()}} Please remove this. I think the simplest fix is \{\{row = row[~row.mask].data[0]}} in collapseArrayMedian, but there may be other occurrences. A better solution might be to stop using {{np.ma}} in favour of using stack-native code everywhere, which understands {{Mask}} planes. |
If there are masked pixels in the overscan, the \{\{MEDIAN_PER_ROW}} overscan correction code generates a warning:
{code} python/lsst/ip/isr/overscan.py:344: UserWarning: Warning: converting a masked element to nan. rowMedian = afwMath.makeStatistics(row, fitType, self.statControl).getValue() {code} Please remove this. I think the simplest fix is \{\{row = row[~row.mask].data[0]}} in collapseArrayMedian, but there may be other occurrences. A better solution might be to stop using {{np.ma}} in favour of using stack-native code everywhere, which understands {{Mask}} planes. |
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
| Assignee | Christopher Waters [ cwaters ] |
| Reviewers | Robert Lupton [ rhl ] | |
| Status | In Progress [ 3 ] | In Review [ 10004 ] |
| Epic Link |
|
| Story Points | 1 |
Looks good. I've no idea why my original fix stopped working, but yours is better anyway (if we have to keep np.ma)
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
| Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
The fix above causes makeStatistics to raise. Switched to `numpy.ma.compressed()`, which though poorly named, does the right operation. Added unit test as well because the overscan outlier masking has apparently never been tested.
https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32528/pipeline