Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: validate_drp
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Sprint:DRP S19-4
-
Team:Data Release Production
Description
Running `matchedVisitMetrics`for DM-17830 was yielding AM1 of 0 for HSC WIDE tracts 9615 and 9697 which have visits numbers in the single digits. AM1 is the median of the RMS of the distances between 2 stars in N visits. Closer inspection revealed that more than half of the RMSs were exactly zero, which is why the median was exactly zero. Michael Wood-Vasey quickly found the problem which was that if a pair appeared in exactly one visit, the stdev([one distance]) was exactly 0 and not NaN.
Require at least 2 distances before computing their stdev.
Michael Wood-Vasey will investigate the potential bias is in the median RMS as a function of N.
This change will increase our reported AMx reported in squash.
Attachments
Issue Links
Activity
| Field | Original Value | New Value |
|---|---|---|
| Epic Link |
|
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
| Attachment | download (14).png [ 37419 ] | |
| Attachment | download (13).png [ 37420 ] |
.png){kind=link}
.png){kind=link}
| Description |
Running `matchedVisitMetrics`for Require at least 2 distances before computing their stdev. [~wmwood-vasey] will investigate the potential bias is in the median RMS as a function of N. This change will increase our reported AMx reported in squash. |
Running `matchedVisitMetrics`for Require at least 2 distances before computing their stdev. [~wmwood-vasey] will investigate the potential bias is in the median RMS as a function of N. This change will increase our reported AMx reported in squash. |
| Description |
Running `matchedVisitMetrics`for Require at least 2 distances before computing their stdev. [~wmwood-vasey] will investigate the potential bias is in the median RMS as a function of N. This change will increase our reported AMx reported in squash. |
Running `matchedVisitMetrics`for Require at least 2 distances before computing their stdev. [~wmwood-vasey] will investigate the potential bias is in the median RMS as a function of N. This change will increase our reported AMx reported in squash. |
Do you want to review? I spent 5 minutes looking at what it'd take to build something that quacked like a matchedDataset to test the whole amx calculation, but it looked like it'd be an endeavor. I can file a ticket for that.
Jenkins:
https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29563/
| Reviewers | John Parejko [ parejkoj ] | |
| Status | In Progress [ 3 ] | In Review [ 10004 ] |
| Watchers | John Parejko, Yusra AlSayyad [ John Parejko, Yusra AlSayyad ] | John Parejko, Simon Krughoff, Yusra AlSayyad [ John Parejko, Simon Krughoff, Yusra AlSayyad ] |
It looks like only one line changed. It's probably worth a comment above that saying why 1 is the correct choice.
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
Added an inline comment and merged.
| Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
Michael Wood-Vasey
made changes -
Yes, this was my fault. I thought that numpy.std used 1/(n-1) and would return NaN for n=1. But I did not test this implicit assumption.
I suggest we require 2 values and then return n/(n-1) * np.std(...) to correct the bias.
I created DM-18751 to do this and will implement later today.
Michael Wood-Vasey
added a comment - Yes, this was my fault. I thought that numpy.std used 1/(n-1) and would return NaN for n=1. But I did not test this implicit assumption.
I suggest we require 2 values and then return n/(n-1) * np.std(...) to correct the bias.
I created DM-18751 to do this and will implement later today.
BEFORE
https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2019_06-jointcal-stars-only-internal/validateDrp_bad/matchedVisitMetrics/9697/HSC-G/validate-jointcal_HSC-G_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png
AFTER
https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2019_06-jointcal-stars-only-internal/validateDrp/matchedVisitMetrics/9697/HSC-G/validate-jointcal_HSC-G_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png