Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-25050

Fix non-normalized matching kernel case in the decorrelation afterburner

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      5
    • Epic Link:
    • Sprint:
      AP F20-2 (July), AP F20-6 (November), AP S21-2 (January)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      In DM-24371, the implemented afterburner correction implicitly assumes that the matching kernel is sum == 1 which is not usually the case, especially without scaling the science and reference image values by their photometric calibration.

      • The afterburner correction does not sum to 1, which changes the fluxes of the difference image (or shift its photometric scale).
      • The variance plane calculation of the afterburned difference image is scaled to the level of var1+var2 which is incorrect if the matching kernel contains flux rescaling, i.e. not sum == 1.
      • The same consideration is required to support a preconvolution kernel, though it needs further investigation to determine the statistical meaning of an afterburned preconvolved image.
      • Also, this causes the corrected PSF not to be automatically sum == 1, though this is corrected by the KernelPsf creation.

        Attachments

          Issue Links

          is child task of

          Bug - A problem which impairs or prevents the functions of the product. DM-21868 Debug and fix the decorrelation afterburner in ip_diffim

          • Invalid
          is duplicated by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-11994 Fix AL decorrelation when preConvolution is enabled

          • Invalid
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-28442 Fix variance plane calculation scaling in non-normalized matching kernel case

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-24371 Implement fixed correction fixed PSF support decorrelation afterburner

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            gkovacs Gabor Kovacs [X] (Inactive) added a comment -

            Added division by zero protection for the doPreConvolve==True case, though it's unlikely that it happens numerically. Instead, it is expected that high frequency noise will be added to the difference image in this case. (The decorrelation kernel is divergent in frequency space - practically we try to invert a multiplication by zero but in a "small noise/small noise" expression. The same problem as in zogy. The difference image should not be created separately in this case.)

            Show
            gkovacs Gabor Kovacs [X] (Inactive) added a comment - Added division by zero protection for the doPreConvolve==True case, though it's unlikely that it happens numerically. Instead, it is expected that high frequency noise will be added to the difference image in this case. (The decorrelation kernel is divergent in frequency space - practically we try to invert a multiplication by zero but in a "small noise/small noise" expression. The same problem as in zogy. The difference image should not be created separately in this case.)
            Hide
            Permalink
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Looks great! Some comments on the pull request.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Looks great! Some comments on the pull request.
            Hide
            Permalink
            gkovacs Gabor Kovacs [X] (Inactive) added a comment -

            Chris Morrison [X] Thanks for the review. Sorry for the confusion by accidentially pushing more commits to this branch while this ticket was in review. As discussed at stand up, I'd rather move them into a new ticket, DM-28442; those changes were incomplete anyway.

             

            Show
            gkovacs Gabor Kovacs [X] (Inactive) added a comment - Chris Morrison [X] Thanks for the review. Sorry for the confusion by accidentially pushing more commits to this branch while this ticket was in review. As discussed at stand up, I'd rather move them into a new ticket, DM-28442 ; those changes were incomplete anyway.  
            Hide
            Permalink
            gkovacs Gabor Kovacs [X] (Inactive) added a comment -

            Added +1 story point for setting up the gen3 hsc test run on lsst-devl (skyMap issue with rc2w42_ssw46 )

            Show
            gkovacs Gabor Kovacs [X] (Inactive) added a comment - Added +1 story point for setting up the gen3 hsc test run on lsst-devl (skyMap issue with rc2w42_ssw46 )

              People

              Assignee:
              gkovacs Gabor Kovacs [X] (Inactive)
              Reporter:
              gkovacs Gabor Kovacs [X] (Inactive)
              Reviewers:
              Chris Morrison [X] (Inactive)
              Watchers:
              Chris Morrison [X] (Inactive), Gabor Kovacs [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.