Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: validate_drp
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Sprint:DRP S19-4
-
Team:Data Release Production
Description
Running `matchedVisitMetrics`for DM-17830 was yielding AM1 of 0 for HSC WIDE tracts 9615 and 9697 which have visits numbers in the single digits. AM1 is the median of the RMS of the distances between 2 stars in N visits. Closer inspection revealed that more than half of the RMSs were exactly zero, which is why the median was exactly zero. Michael Wood-Vasey quickly found the problem which was that if a pair appeared in exactly one visit, the stdev([one distance]) was exactly 0 and not NaN.
Require at least 2 distances before computing their stdev.
Michael Wood-Vasey will investigate the potential bias is in the median RMS as a function of N.
This change will increase our reported AMx reported in squash.
Attachments
Issue Links
Activity
Do you want to review? I spent 5 minutes looking at what it'd take to build something that quacked like a matchedDataset to test the whole amx calculation, but it looked like it'd be an endeavor. I can file a ticket for that.
Jenkins:
https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29563/
It looks like only one line changed. It's probably worth a comment above that saying why 1 is the correct choice.
Added an inline comment and merged.
Yes, this was my fault. I thought that numpy.std used 1/(n-1) and would return NaN for n=1. But I did not test this implicit assumption.
I suggest we require 2 values and then return n/(n-1) * np.std(...) to correct the bias.
I created DM-18751 to do this and will implement later today.
Michael Wood-Vasey
added a comment - Yes, this was my fault. I thought that numpy.std used 1/(n-1) and would return NaN for n=1. But I did not test this implicit assumption.
I suggest we require 2 values and then return n/(n-1) * np.std(...) to correct the bias.
I created DM-18751 to do this and will implement later today.
BEFORE
https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2019_06-jointcal-stars-only-internal/validateDrp_bad/matchedVisitMetrics/9697/HSC-G/validate-jointcal_HSC-G_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png
AFTER
https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2019_06-jointcal-stars-only-internal/validateDrp/matchedVisitMetrics/9697/HSC-G/validate-jointcal_HSC-G_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png