Fix Version/s: None
Sprint:Alert Production S17 - 6, Alert Production F17 - 7, AP S18-1, AP S18-2, AP S18-3
Analyze and make summary plots for the metadata collected in
DM-12440. This may involve investigating further any failure modes that may have cropped up.
- is blocked by
DM-12440 Rerun matcher validation dataset
The data here are all excellent and the resulting conclusion is very clear. I did some math to compute fractions of successful CCDs because I couldn't find those in the tables, but then discovered that the numbers were already there. So the main thing I would modify is to present this in a way that's more easily scanned to get a high level overview. JIRA is clearly not a good venue for that, so I would suggest pushing this into a technote in this ticket, even if the text surrounding it is left for a subsequent ticket. (e.g., you can make this Appendix A without anything else yet). One table of simply old percentage -> new percentage for each dataset would be the right level of detail for someone to see the conclusion, and then probably a second table could contain the detailed CCD counts and statistics.
I think you could show that the matcher is even better speed-wise if you included a bit of data on dense fields. Doesn't have to be 10k ccds, but maybe a dozen to give a general sense. The one other minor detail is to please drop insignificant digits.
I've added the summary tables to DMTN-031. They are within this commit: https://github.com/lsst-dm/dmtn-031/commit/8226d9306827b2aaa4be1dce8c7015755992bd8a
The timing comparison using HSC is currently running and should complete some time tomorrow.
Here are the timings for for the HSC, New Horizon pointing (Number: 908):
New Matcher Times: Mean=86.126 Median=15.996 Std=112.800
Old Matcher Times: Mean=68.690 Median=12.347 Std=123.853
Attached summary statistics and select summary plots to this ticket showing the performance of the new MatchPessimisticB task vs the old MatchOptimsiticB task. Let me know if the summaries are sufficiently informative and suggest any other tests to be carried out.