Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-38973

Call to np.percentile in overscan.py leads to numpy warnings with 1.23

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • ip_isr
    • 1
    • Data Release Production
    • No

    Description

      With numpy 1.23, the following call gives numpy warnings. (Lots of them). https://github.com/lsst/ip_isr/blob/main/python/lsst/ip/isr/overscan.py#L620

      py.warnings WARNING: /u2/ki/erykoff/lsst-pc9_20230420/conda/envs/lsst-scipipe-6.0.0/lib/python3.10/site-packages/numpy/lib/function_base.py:4691: UserWarning: Warning: 'partition' will ignore the 'mask' of the MaskedArray.
      

      This is caused by the fact that getImage gives a masked array:
      https://github.com/lsst/ip_isr/blob/main/python/lsst/ip/isr/overscan.py#L599-L601

      (Masked arrays in numpy are a great idea but never work the way I think they should).

      We need to figure out how to call percentile while avoiding the mask. I think that maybe nanpercentile, filling with nans?

      Attachments

        Activity

          erykoff Eli Rykoff created issue -
          czw Christopher Waters made changes -
          Field Original Value New Value
          Status To Do [ 10001 ] In Progress [ 3 ]
          czw Christopher Waters made changes -
          Reviewers Eli Rykoff [ erykoff ]
          Status In Progress [ 3 ] In Review [ 10004 ]
          erykoff Eli Rykoff made changes -
          Status In Review [ 10004 ] Reviewed [ 10101 ]
          czw Christopher Waters made changes -
          Epic Link DM-32177 [ 779901 ]
          Story Points 1
          czw Christopher Waters made changes -
          Resolution Done [ 10000 ]
          Status Reviewed [ 10101 ] Done [ 10002 ]

          People

            czw Christopher Waters
            erykoff Eli Rykoff
            Eli Rykoff
            Christopher Waters, Eli Rykoff
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Jenkins

                No builds found.