Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: meas_astrom
-
Labels:
-
Story Points:6
-
Epic Link:
-
Sprint:Alert Production S17 - 6
-
Team:Alert Production
Description
Run a large set of data through the stackified matcher. These data should be run only once and with the configuration produced by the verification process (DM-9751).
Attachments
Issue Links
Activity
| Field | Original Value | New Value |
|---|---|---|
| Epic Link |
|
| Team | Alert Production [ 10300 ] |
If the HSC RC dataset would be useful to you, so far we have two processed sets, using the stack versions w_2017_14 and w_2017_17, accessible as butler repos from:
/datasets/hsc/repo/rerun/private/hchiang2/RC/DM-10084-mosaic/ (week 14), and
/datasets/hsc/repo/rerun/private/hchiang2/RC/DM-10129 (week 17).
The visit IDs of the RC dataset are listed in the 3rd section here.
Hsin-Fang Chiang
added a comment - If the HSC RC dataset would be useful to you, so far we have two processed sets, using the stack versions w_2017_14 and w_2017_17 , accessible as butler repos from:
/datasets/hsc/repo/rerun/private/hchiang2/RC/ DM-10084 -mosaic/ (week 14), and
/datasets/hsc/repo/rerun/private/hchiang2/RC/ DM-10129 (week 17).
The visit IDs of the RC dataset are listed in the 3rd section here . Started run on HSC DEEPE data and HITS. Need to injest CFHTLS u band data for complete test.
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
| Sprint | Alert Production S17 - 6 [ 616 ] |
Finished running data sets for validation of the pessimistic matcher and comparison to optimistic matcher.
Data run are:
HSC pointing=908 - 4368 targets/ccds
HITS/DECam visit=406285-410983 - 11040 targets/ccds
CFHTLS W3 visit=704382-792946 - 11700 targets/ccds
Output are stored in:
/project/morriscb/HSC/rerun/VALDIATE
/project/morriscb/HITS/rerun/VALDIATE
/project/morriscb/CFHTLS/rerun/VALDIATE
respectively for the pessimistic matcher. Optimistic matcher results are stored in the VALIDATE_stack rerun directory.
| Reviewers | Simon Krughoff [ krughoff ] | |
| Status | In Progress [ 3 ] | In Review [ 10004 ] |
I'm a little worried that we are only looking at the best behaved filters (g and r), though I know it is a bit of a tricky thing because int he harder bands ISR plays a bigger role. I think it's fine for now, but we should keep an eye out for problems in other bands.
For completeness, I've recorded the number of calexps that got recorded for the new/old matcher. They are similar, but not the same. I hope we can figure out why these chips failed. It sounds like you have a good start on that, but we need to make sure to be able to categorize all the missing ones in the next phase.
Completed HSC – 3952/4056
Completed HITS – 10071/8676
Completed CFHTLS – 8831/8906
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
Ugh, I thought there would be a few u, i, z exposures in that set of visits. I'll run some specific u, i, z data as well. Should finish up today.
As for the missing calExps, yeah, I've seen a few that had errors from the PSF estimation but there are definite failures to match to be investigated in there. The log files can point me to those and as part of DM-10398.
By the time we get to happy hour there should be new 2000 ccds worth processed data for each band in /project/morriscb/CFHTLS/rerun/VALIDATE_uiz or /project/morriscb/CFHTLS/rerun/VALIDATE_stack_uiz for the new and stack matcher respectively. Let me know when you get a chance to take a look at it.
O.K. I've verified there are around 2000 calexps in each band in both repositories. I think we can consider this done.
Completed run of data for CFHTLS, HITS (DECam) and New Horizons (HSC).
| Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
Please coordinate with Hsin-Fang Chiang so we don't duplicate effort.