Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23855

Audit and debug current zogy image differencing code

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      10
    • Epic Link:
    • Sprint:
      AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP F21-1 (June), AP F21-2 (July), AP F21-3 (August), AP F21-4 (September)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      The current zogy implementation is assumed to have issues in:

      • Understand how the variance plane should propagate in zogy (convolution then whitening decorrelation).
      • Correct padding and centering of images and PSFs in Fourier space opeartions. (numerical accuracy in DFT vs. origin location and periodicity assumptions)
      • Correct calculation of the difference image variance plane. (variance of image k-space components vs. k-space components of variance plane)
      • zogy.py:384 padSize argument usage

        Attachments

          Issue Links

          is duplicated by

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. DM-9441 Additional debugging of ZOGY and A&L(decorrelation)

          • Invalid

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-10513 Understand and remove ZOGY artifacts

          • Invalid

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-11656 Fix ZOGY failure when PSFs are offset

          • Invalid
          is parent task of

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-25115 Implement zogy subtraction with non-spatial PSF

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-25174 Draft current understanding of DM's ZOGY implementation as a technical note

          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-12917 running ZOGY with coadded templates produces improper results

          • Invalid

          Bug - A problem which impairs or prevents the functions of the product. DM-21868 Debug and fix the decorrelation afterburner in ip_diffim

          • Invalid

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21920 Try out and write notebook about demonstrating basic DFT space steps for ip_diffim operations

          • Done

            Activity

            Ascending order - Click to sort in descending order
            gkovacs Gabor Kovacs [X] (Inactive) created issue -
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Field Original Value New Value
            Link This issue relates to DM-21868 [ DM-21868 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Link This issue relates to DM-21920 [ DM-21920 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Issue Type Story [ 10001 ] Bug [ 1 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Description The current zogy implementation is assumed to have issues in:
             * Understand how the variance plane should propagate in zogy (convolution then whitening decorrelation).
             * Correct padding and centering of images and PSFs in Fourier space opeartions. (numerical accuracy in DFT vs. origin location and periodicity assumptions)

             * Correct calculation of the difference image variance plane. (variance of image k-space components vs. k-space components of variance plane)             		The current zogy implementation is assumed to have issues in:
             * Understand how the variance plane should propagate in zogy (convolution then whitening decorrelation).
             * Correct padding and centering of images and PSFs in Fourier space opeartions. (numerical accuracy in DFT vs. origin location and periodicity assumptions)

             * Correct calculation of the difference image variance plane. (variance of image k-space components vs. k-space components of variance plane)
             * zogy.py:384 padSize argument usage
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Description The current zogy implementation is assumed to have issues in:
             * Understand how the variance plane should propagate in zogy (convolution then whitening decorrelation).
             * Correct padding and centering of images and PSFs in Fourier space opeartions. (numerical accuracy in DFT vs. origin location and periodicity assumptions)

             * Correct calculation of the difference image variance plane. (variance of image k-space components vs. k-space components of variance plane)
             * zogy.py:384 padSize argument usage             		The current zogy implementation is assumed to have issues in:
             * Understand how the variance plane should propagate in zogy (convolution then whitening decorrelation).
             * Correct padding and centering of images and PSFs in Fourier space opeartions. (numerical accuracy in DFT vs. origin location and periodicity assumptions)

             * Correct calculation of the difference image variance plane. (variance of image k-space components vs. k-space components of variance plane)
             * {{zogy.py:384}} padSize argument usage
            swinbank John Swinbank made changes -
            Epic Link DM-22635 [ 427744 ]
            swinbank John Swinbank made changes -
            Team Alert Production [ 10300 ]
            swinbank John Swinbank made changes -
            Sprint AP S20-5 (April) [ 986 ]
            Story Points 10
            swinbank John Swinbank made changes -
            Link This issue relates to DM-12917 [ DM-12917 ]
            swinbank John Swinbank made changes -
            Assignee Gabor Kovacs [ gkovacs ]
            swinbank John Swinbank made changes -
            Epic Link DM-22635 [ 427744 ] DM-24342 [ 433029 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            swinbank John Swinbank made changes -
            Sprint AP S20-5 (April) [ 986 ] AP S20-5 (April), AP S20-6 (May) [ 986, 987 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Link This issue is parent task of DM-25115 [ DM-25115 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Link This issue is parent task of DM-25174 [ DM-25174 ]
            swinbank John Swinbank made changes -
            Epic Link DM-24342 [ 433029 ] DM-25144 [ 435262 ]
            swinbank John Swinbank made changes -
            Sprint AP S20-5 (April), AP S20-6 (May) [ 986, 987 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June) [ 986, 987, 1019 ]
            swinbank John Swinbank made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June) [ 986, 987, 1019 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July) [ 986, 987, 1019, 1025 ]
            swinbank John Swinbank made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July) [ 986, 987, 1019, 1025 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-3 (August) [ 986, 987, 1019, 1025, 1033 ]
            swinbank John Swinbank made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-3 (August) [ 986, 987, 1019, 1025, 1033 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-4 (September) [ 986, 987, 1019, 1025, 1039 ]
            swinbank John Swinbank made changes -
            Rank Ranked lower
            swinbank John Swinbank made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-4 (September) [ 986, 987, 1019, 1025, 1039 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October) [ 986, 987, 1019, 1025, 1046 ]
            swinbank John Swinbank made changes -
            Epic Link DM-25144 [ 435262 ] DM-26804 [ 439756 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Link This issue is duplicated by DM-10513 [ DM-10513 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Link This issue is duplicated by DM-11656 [ DM-11656 ]
            gkovacs Gabor Kovacs [X] (Inactive) made changes -
            Link This issue is duplicated by DM-9441 [ DM-9441 ]
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October) [ 986, 987, 1019, 1025, 1046 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November) [ 986, 987, 1019, 1025, 1046, 1052 ]
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November) [ 986, 987, 1019, 1025, 1046, 1052 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-1 (December) [ 986, 987, 1019, 1025, 1046, 1052, 1059 ]
            sullivan Ian Sullivan made changes -
            Rank Ranked higher
            sullivan Ian Sullivan made changes -
            Epic Link DM-26804 [ 439756 ] DM-27910 [ 442603 ]
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-1 (December) [ 986, 987, 1019, 1025, 1046, 1052, 1059 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-2 (January) [ 986, 987, 1019, 1025, 1046, 1052, 1064 ]
            sullivan Ian Sullivan made changes -
            Rank Ranked lower
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-2 (January) [ 986, 987, 1019, 1025, 1046, 1052, 1064 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February) [ 986, 987, 1019, 1025, 1046, 1052, 1072 ]
            sullivan Ian Sullivan made changes -
            Rank Ranked higher
            sullivan Ian Sullivan made changes -
            Reviewers Ian Sullivan [ sullivan ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            sullivan Ian Sullivan made changes -
            Epic Link DM-27910 [ 442603 ] DM-29210 [ 459212 ]
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February) [ 986, 987, 1019, 1025, 1046, 1052, 1072 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079 ]
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084 ]
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-6 (May) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1088 ]
            sullivan Ian Sullivan made changes -
            Rank Ranked lower
            sullivan Ian Sullivan made changes -
            Rank Ranked higher
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-6 (May) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1088 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084 ]
            sullivan Ian Sullivan made changes -
            Rank Ranked lower
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP S21-6 (May) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1088 ]
            sullivan Ian Sullivan made changes -
            Epic Link DM-29210 [ 459212 ] DM-30435 [ 504823 ]
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP S21-6 (May) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1088 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP F21-1 (June) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1096 ]
            sullivan Ian Sullivan made changes -
            Rank Ranked lower
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP F21-1 (June) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1096 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP F21-1 (June), AP F21-2 (July) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1096, 1102 ]
            sullivan Ian Sullivan made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            sullivan Ian Sullivan made changes -
            Rank Ranked higher
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP F21-1 (June), AP F21-2 (July) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1096, 1102 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP F21-1 (June), AP F21-2 (July), AP F21-3 (August) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1096, 1102, 1109 ]
            sullivan Ian Sullivan made changes -
            Sprint AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP F21-1 (June), AP F21-2 (July), AP F21-3 (August) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1096, 1102, 1109 ] AP S20-5 (April), AP S20-6 (May), AP F20-1 (June), AP F20-2 (July), AP F20-5 (October), AP F20-6 (November), AP S21-3 (February), AP S21-4 (March), AP S21-5 (April), AP F21-1 (June), AP F21-2 (July), AP F21-3 (August), AP F21-4 (September) [ 986, 987, 1019, 1025, 1046, 1052, 1072, 1079, 1084, 1096, 1102, 1109, 1113 ]
            sullivan Ian Sullivan made changes -
            Assignee Gabor Kovacs [X] [ gkovacs ] Ian Sullivan [ sullivan ]
            sullivan Ian Sullivan made changes -
            Epic Link DM-30435 [ 504823 ] DM-30503 [ 510162 ]
            sullivan Ian Sullivan made changes -
            Epic Link DM-30503 [ 510162 ] DM-30435 [ 504823 ]
            sullivan Ian Sullivan made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            sullivan Ian Sullivan made changes -
            Starting 30/Aug/21

              People

              Assignee:
              sullivan Ian Sullivan
              Reporter:
              gkovacs Gabor Kovacs [X] (Inactive)
              Reviewers:
              Ian Sullivan
              Watchers:
              Gabor Kovacs [X] (Inactive), Ian Sullivan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.