Details
-
Type:
Improvement
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: validate_drp
-
Labels:None
-
Story Points:0.2
-
Team:SQuaRE
Description
Return the unbiased std sqrt(1/(n-1) * sum_i((x_i*2 - <x>)*2)) definition of RMS for AMx calculations.
I suggest we require 2 values and then return n/(n-1) * np.std(...) to correct the bias.
Attachments
Issue Links
- is child task of
-
DM-18638 Check for at least 2 visits in AMx metric calculation.
- Done
Activity
Michael Wood-Vasey
added a comment - Passes Jenkins
https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29599/pipeline/ Yusra AlSayyad Thanks for the review (particularly long, drawn-out, "no, no, come back out of that rabbit hole" part).
Michael Wood-Vasey
added a comment - Yusra AlSayyad Thanks for the review (particularly long, drawn-out, "no, no, come back out of that rabbit hole" part).
Simon Krughoff When this merges, it's going to increase the AM1 and AM2 values for test cases being run through the regression monitoring. This isn't a change in pipeline performance, this is an improvement in our measurement of the performance that reveals we weren't doing as well as we thought.
For HSC, the changes, based on the stored processing of v17, in validation_data_hsc are:
| HSC | previous | |
|---|---|---|
| AM1 I | 7.456 | 8.972 |
| AM1 Y | 6.047 | 8.551 |
| AM1 R | 3.232 | 4.571 |
| AM2 I | 7.449 | 8.976 |
| AM2 Y | 6.206 | 8.777 |
| AM2 R | 3.297 | 4.663 |
These jumps are significant mostly because we have few images in these validation_data_* repos, so the difference between 1/n and 1/(n-1) is significant.
Michael Wood-Vasey
added a comment - - edited Simon Krughoff When this merges, it's going to increase the AM1 and AM2 values for test cases being run through the regression monitoring. This isn't a change in pipeline performance, this is an improvement in our measurement of the performance that reveals we weren't doing as well as we thought.
For HSC, the changes, based on the stored processing of v17, in validation_data_hsc are:
HSC
previous
DM-18751
AM1 I
7.456
8.972
AM1 Y
6.047
8.551
AM1 R
3.232
4.571
AM2 I
7.449
8.976
AM2 Y
6.206
8.777
AM2 R
3.297
4.663
These jumps are significant mostly because we have few images in these validation_data_* repos, so the difference between 1/n and 1/(n-1) is significant.
Passes Jenkins:
Will wait until after v18.0 to merge.
Michael Wood-Vasey
added a comment - Passes Jenkins:
https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29937/pipeline/47/
Will wait until after v18.0 to merge. Merged to master.
Michael Wood-Vasey
added a comment - Merged to master. 
Multiple np.std by n/(n-1) to compute sample standard deviation.