Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26414

Handle masked pixels in ip_isr's MEDIAN_PER_ROW

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_isr
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      If there are masked pixels in the overscan, the MEDIAN_PER_ROW overscan correction code generates a warning:

      python/lsst/ip/isr/overscan.py:344: UserWarning: Warning: converting a masked element to nan.
      		 
       rowMedian = afwMath.makeStatistics(row, fitType, self.statControl).getValue()

      Please remove this.  I think the simplest fix is row = rowrow.mask.data[0] in collapseArrayMedian, but there may be other occurrences.  A better solution might be to stop using np.ma in favour of using stack-native code everywhere, which understands Mask planes.

       

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          czw Christopher Waters added a comment -

          The fix above causes makeStatistics to raise.  Switched to `numpy.ma.compressed()`, which though poorly named, does the right operation.  Added unit test as well because the overscan outlier masking has apparently never been tested.

          https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32528/pipeline

          Show
          czw Christopher Waters added a comment - The fix above causes makeStatistics to raise.  Switched to `numpy.ma.compressed()`, which though poorly named, does the right operation.  Added unit test as well because the overscan outlier masking has apparently never been tested. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32528/pipeline
          Hide
          Permalink
          rhl Robert Lupton added a comment -

          Looks good. I've no idea why my original fix stopped working, but yours is better anyway (if we have to keep np.ma)

          Show
          rhl Robert Lupton added a comment - Looks good. I've no idea why my original fix stopped working, but yours is better anyway (if we have to keep np.ma)

            People

            Assignee:
            czw Christopher Waters
            Reporter:
            rhl Robert Lupton
            Reviewers:
            Robert Lupton
            Watchers:
            Christopher Waters, Robert Lupton
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.