Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18638

Check for at least 2 visits in AMx metric calculation.

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: validate_drp
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP S19-4
    • Team:
      Data Release Production

      Description

      Running `matchedVisitMetrics`for DM-17830 was yielding AM1 of 0 for HSC WIDE tracts 9615 and 9697 which have visits numbers in the single digits. AM1 is the median of the RMS of the distances between 2 stars in N visits. Closer inspection revealed that more than half of the RMSs were exactly zero, which is why the median was exactly zero.  Michael Wood-Vasey quickly found the problem which was that if a pair appeared in exactly one visit, the stdev([one distance]) was exactly 0  and not NaN.

      Require at least 2 distances before computing their stdev.

      Michael Wood-Vasey will investigate the potential bias is in the median RMS as a function of N.

      This change will increase our reported AMx reported in squash.

        Attachments

          Issue Links

            Activity

            Show
            yusra Yusra AlSayyad added a comment - BEFORE https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2019_06-jointcal-stars-only-internal/validateDrp_bad/matchedVisitMetrics/9697/HSC-G/validate-jointcal_HSC-G_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png AFTER https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2019_06-jointcal-stars-only-internal/validateDrp/matchedVisitMetrics/9697/HSC-G/validate-jointcal_HSC-G_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png
            Hide
            yusra Yusra AlSayyad added a comment -

            Do you want to review? I spent 5 minutes looking at what it'd take to build something that quacked like a matchedDataset to test the whole amx calculation, but it looked like it'd be an endeavor. I can file a ticket for that.
            Jenkins:
            https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29563/

            Show
            yusra Yusra AlSayyad added a comment - Do you want to review? I spent 5 minutes looking at what it'd take to build something that quacked like a matchedDataset to test the whole amx calculation, but it looked like it'd be an endeavor. I can file a ticket for that. Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29563/
            Hide
            Parejkoj John Parejko added a comment -

            It looks like only one line changed. It's probably worth a comment above that saying why 1 is the correct choice.

            Show
            Parejkoj John Parejko added a comment - It looks like only one line changed. It's probably worth a comment above that saying why 1 is the correct choice.
            Hide
            yusra Yusra AlSayyad added a comment -

            Added an inline comment and merged.

            Show
            yusra Yusra AlSayyad added a comment - Added an inline comment and merged.
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            Yes, this was my fault. I thought that numpy.std used 1/(n-1) and would return NaN for n=1. But I did not test this implicit assumption.

            I suggest we require 2 values and then return n/(n-1) * np.std(...) to correct the bias.

            I created DM-18751 to do this and will implement later today.

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - Yes, this was my fault. I thought that numpy.std used 1/(n-1) and would return NaN for n=1. But I did not test this implicit assumption. I suggest we require 2 values and then return n/(n-1) * np.std(...) to correct the bias. I created DM-18751 to do this and will implement later today.

              People

              • Assignee:
                yusra Yusra AlSayyad
                Reporter:
                yusra Yusra AlSayyad
                Reviewers:
                John Parejko
                Watchers:
                John Parejko, Michael Wood-Vasey, Simon Krughoff, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel