Description
This RFC proposes to deprecate obs_sdss and the lsst_stack_demo. The stack demo processes a few SDSS frames and checks that the measurement results are identical to a saved set of measurements. obs_sdss is necessary to support the demo, and the most recent substantial usage outside the demo was in the Stripe 82 processing during 2013. The catalogs from the Stripe 82 reprocessing are still in use as one of the databases available in the science platform, which doesn't require obs_sdss, but the images are also served by the SODA service which does use obs_subaru.
Deprecating obs_sdss at this point would save the effort required to port it to Gen3. The demo also requires occasional effort by developers when merging measurement changes to update the fixed values it uses for validating the results. This testing is largely superseded by validate_drp, ap_verify, ci_hsc, and the squash framework for monitoring regressions. The demo does sometimes identify changes that unexpectedly modify measurements results (since it requires nearly bitwise-identical results), but this functionality could be added to either of the other CI packages. If this type of validation is desired, it would be substantially better to add it to ci_hsc where it can monitor more stages of the pipelines, beyond single frame measurement.
Deprecating obs_sdss will require migrating the SODA service to use HSC data. This migration should be easier than bringing obs_sdss and the Stripe 82 repo up to Gen3, and serving HSC images is a worthwhile goal on its own.
As I understand the deprecation policy, if accepted, we should announce the impending deprecation in this upcoming 19.0 release and remove the packages from lsst_distrib after the release and once the SODA service has been migrated to HSC data. This RFC will need approval by the DMCCB.
Attachments
Issue Links
- blocks
-
DM-21866 Extend Gen3 butler support for obs_sdss
- Won't Fix
- is triggering
-
- Done
- relates to
-
- Won't Fix
-
- Won't Fix
-
- To Do
-
- Done
-
- Won't Fix
-
- Implemented
- mentioned in
-
Page Loading...
-
-
-
-
-
Activity
I'm in favor of dropping obs_sdss. I think all of my arguments have been made by others, but I can definitely confirm that obs_sdss will be more difficult than most obs_ packages to make work with Gen3, because its drift-scan observing mode and multiple-filter focal plane would require us to generalize our data model substantially. Being able to assume all cameras have basically the same visit/detector/filter concepts (and relationships between those concepts) has been really helpful so far.
While the SDSS data is weird, I think it can be mapped to a model that looks like a normal camera (we might need to add a digit to the field number to represent the filter and disambiguate the field)
I think that might be doable, but I'd still love to not have to try to make it work. The other side of this is of course what we get out of it, and for that I think the relevant question is whether there are things we can only do with SDSS data. I think the answer is that there are interesting things we can only do with SDSS from a scientific perspective, but very little we can only learn from SDSS data about our algorithms right now and in the foreseeable future. That's partly because other datasets are pretty good in what they cover (especially with HSC deep fields getting substantially more interesting for time-domain science with PDR2), partly because the subset of our pipelines we can run on SDSS without heavy modification is relatively small, and partly because the things that SDSS is most uniquely good for (e.g. variability characterization) are neither things we're working on now nor things we consider major risks.
The set of datasets for scientific performance monitoring is detailed in DMTN-091, and does not include SDSS. This was presented at DMLT in October and there was agreement that this set was sufficient for scientific testing of the pipelines. These are the datasets that we need to prioritize. I agree with comments above that there are interesting things we can learn from SDSS but we need to prioritize and obs_sdss is not a priority . We should deprecate obs_sdss in R19. If someone wants to pick it up in future and port it to Gen3 it can be re-included (providing it passes all tests), however it should not be on the critical path.
I agree it is useful in lsst_stack_demo to processes a few frames and check that the measurement results are identical to a saved set of measurements, however we don't need this to be done with SDSS data. This should be replaced in lsst_stack_demo by a similar processing using one of the datasets listed in DMTN-091.
If it's not too much work I'd like to keep this alive. The stripe82 data is interesting for testing deep(ish) multi-epoch coadds in the presence of things like IR cirrus.
The stripe82 data is also valuable for difference imaging studies, and has one of the best-studied collection of variables (I think that's still true).