Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16414

Confirm fix of DM-16392 after next RC2 processing

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_analysis
    • Labels:
      None
    • Story Points:
      2
    • Epic Link:
    • Team:
      Data Release Production

      Description

      The fix on DM-16392 is easiest to confirm on the pipe_analysis processing outputs of the RC2 bi-weekly reprocessing. The issue was that the order aperture corrections were being added to the schema was not deterministic (loop was being done on a set). As a result, when attempting to concatenate all patches of a given tract into a single catalog and using a template schema, the aperture corrections were often pointing to different key offsets, thus getting the wrong column appended into catalog. This issue was discovered in perusal of the plots output by pipe_analysis's compareCoaddAnalysis.py and compareVisitAnalysis.py when comparing outputs from one reprocessing to the previous one (between which no algorithmic changes had been made that could've caused this discrepancy, backed up by the identical flux measurements for the same points). This ticket is to capture the effort that went into diagnosing the bug and ensuring the fix was successful after the w_2018_44 – assuming the fix lands on time – is completed.

        Attachments

          Activity

          Hide
          lauren Lauren MacArthur added a comment -

          As an example, the following are the comparison plots pre-fix:

          Post-fix, these should all flatline at zero (assuming no algorithmic changes get into the next release!).

          Show
          lauren Lauren MacArthur added a comment - As an example, the following are the comparison plots pre-fix: Post-fix, these should all flatline at zero (assuming no algorithmic changes get into the next release!).
          Hide
          lauren Lauren MacArthur added a comment -

          I can now confirm that the problem was indeed fixed.  The plots comparing the aperture corrections of the w_2018_46 vs. w_2018_44 reprocessing runs, which can be found here, indeed all flatline at zero, e.g.

          Show
          lauren Lauren MacArthur added a comment - I can now confirm that the problem was indeed fixed.  The plots comparing the aperture corrections of the w_2018_46 vs. w_2018_44 reprocessing runs, which can be found here , indeed all flatline at zero, e.g.
          Hide
          lauren Lauren MacArthur added a comment -

          Ok to close this one out?

          Show
          lauren Lauren MacArthur added a comment - Ok to close this one out?

            People

            • Assignee:
              lauren Lauren MacArthur
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Yusra AlSayyad
              Watchers:
              Jim Bosch, Lauren MacArthur, Yusra AlSayyad
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: