Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-38901

Clear Template mask planes in image differencing

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_diffim
    • Labels:
      None
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      AP S23-5 (April), AP S23-6 (May)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Add a config option to the image differencing tasks for specific template mask planes to preserve, and clear all others.

      Mask planes we know should be preserved:

      • NO_DATA

      Mask planes we should preserve for now, and investigate whether they should be removed:

      • BAD
      • SAT

        Attachments

        1. New diffim mask.png
          New diffim mask.png
          36 kB
        2. Old diffim mask.png
          Old diffim mask.png
          126 kB
        3. Science mask.png
          Science mask.png
          73 kB

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-37386 Investigate adding CenterAll flag during detect and measure

          • To Do

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-37968 Use source flags instead of mask plane in image differencing

          • To Do
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33798 Improve bitmask propagation in ip_diffim

          • To Do

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            sullivan Ian Sullivan added a comment -

            Adding the code to clear the template mask uncovered a couple of minor bugs in the existing code, which are also fixed on this ticket. I added new unit tests to verify the behavior of the image difference mask, and ran ap_verify on the ap_verify_ci_hits2015 test dataset to verify the results on real data. Snapshots below are provided as a quick summary of the changed behavior:

            The mask of the science and template images retrieved from the butler are identical in both runs (as they should be). Mask of the science image with suppressed color scale:

            With our original code, the mask of the difference image is almost useless (same color scale):

            With the new code to clear the mask of the template, the mask of the difference image may be useful (same color scale):

            Show
            sullivan Ian Sullivan added a comment - Adding the code to clear the template mask uncovered a couple of minor bugs in the existing code, which are also fixed on this ticket. I added new unit tests to verify the behavior of the image difference mask, and ran ap_verify on the ap_verify_ci_hits2015 test dataset to verify the results on real data. Snapshots below are provided as a quick summary of the changed behavior: The mask of the science and template images retrieved from the butler are identical in both runs (as they should be). Mask of the science image with suppressed color scale: With our original code, the mask of the difference image is almost useless (same color scale): With the new code to clear the mask of the template, the mask of the difference image may be useful (same color scale):
            Hide
            Permalink
            sullivan Ian Sullivan added a comment -

            PR: https://github.com/lsst/ip_diffim/pull/257
            Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/38594/pipeline

            Show
            sullivan Ian Sullivan added a comment - PR: https://github.com/lsst/ip_diffim/pull/257 Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/38594/pipeline
            Hide
            Permalink
            bsanchez Bruno Sanchez added a comment -

            I reviewed the changes. I ask for simple clarifications with optional changes.

            Show
            bsanchez Bruno Sanchez added a comment - I reviewed the changes. I ask for simple clarifications with optional changes.
            Hide
            Permalink
            sullivan Ian Sullivan added a comment -

            Reviewed by Bruno Sanchez on Github

            Show
            sullivan Ian Sullivan added a comment - Reviewed by Bruno Sanchez on Github

              People

              Assignee:
              sullivan Ian Sullivan
              Reporter:
              sullivan Ian Sullivan
              Reviewers:
              Bruno Sanchez
              Watchers:
              Bruno Sanchez, Ian Sullivan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.