`ci_hsc` is a Scons-based running of many tests. Should long-term move to more Task-driven interface that can be driven from command-line, Scons, or called from outside.
This review and thinking led to the following two tickets for further work.
Continuous integration will occupy a spectrum toward validation. With a need for quick 5- minute CI checks with "small" all the way through to nightly runs with "medium" data sets, and even weekly full builds with "large" data sets full focal planes and deep coadds. ci_hsc currently occupies the "nightly", or "medium" scale data set size.
1. Provide general comparison sets for data. Both input and processed. This provides convenient comparisons for the developer and enables a range of quick-to-full-scale tests against "known" results for quick-turnaround CI up to full focal-plane coadd comparisons in nightly builds..
DM-5147 - Provide usable repos in `validation_data_*` packages.
2. Compare changes to specific measurements of objects when processing the same data through different pipelines or with different algorithms or configurations.
DM-5270 - Provide comparison routines for comparing two repos of the same data
The comparison routines in u/laurenm/
DM-4730 will map relatively easily to general cameras with some work to remove HSC-specific prints and text annotation and some robustness against cameras or sources not having certain properties.