Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: meas_astrom
-
Labels:
-
Story Points:6
-
Epic Link:
-
Sprint:Alert Production S17 - 6
-
Team:Alert Production
Description
Once the validation datasets have been processed, analyze the outputs for completeness. This will include categorizing the failure modes (if any exist 
).
Things to be collected could include:
1. Pass/Fail
2. Num ref/num source
3. Rms scatter and error on that value
4. Broad categories of failures
Attachments
Issue Links
- is blocked by
-
- Done
Activity
Created ascii files for each of the processed data from HSC, CFHTLS, and HITS. These data are stored in the directory:
/project/morriscb/VALIDATE_data
2 files are stored per rerun: The visits/ccds where both matcher "succeeded" (as in produced a calExp) and where some part of the pipeline failed (i.e. the visti/ccd contains bad data or at least one of the matchers failed).
These data were created from the output files produced by singleFrameDriver.py by using the Jupyter notebook in "/home/morriscb/Parse and store output logs for matcher.ipynb".
This script could be more generailized for end to end creation of these logs files given a request from the reviewer.
Can you remind me what you mean by fail? I thought it was something more subtle than just failing to produce a calexp. I.e. the match scatter being too large would indicate failure, but maybe that comes in the analysis step.
I think we need to categorize the cases where a failure to produce a calexp happened. What were the causes of those failures? I think that needs to be part of this ticket.
Hey Simon,
By fail in this context I mean did not produce a calexp for one of 2 cases: 1) Either matcher failed to find any match 2) an earlier pipeline step failed for some reason.
The number of pipeline failures for each of the different reruns are:
CFHTLS_u: 2016 visits, 7 pipeline failures
CFHTLS_i: 2016 visits, 7 pipeline failures
CFHTLS_z: 2016 visits, 7 pipeline failures
CFHTLS: 11,700 visits, 2757 pipeline failures
HITS: 11,040 visits, 162 pipeline failures
HSC: 3952 visits, 0 pipeline failures.
A failure to match either catastrophically or with a bad scatter will be investigated in the next ticket DM-10398.
Sounds good. Ship it.
Summary data files created successfully.
Started collecting summary statics from data run with both optimistic and pessimistic matchers..