Details
-
Type:
RFC
-
Status: Implemented
-
Resolution: Done
-
Component/s: DM
-
Labels:None
Description
We now have the option to define mapper datasets that are the same for all cameras in a common location (the config files for the base class CameraMapper). I propose that we go ahead and do this for all common datasets - essentially anything that doesn't depend on the keys of the raw data ID - now. This will massively reduce code duplication in the mapper files, and reduce our dependence on paf files (since we'll be moving to the new YAML-ish format).
There are two possible hangups:
- We probably need to resolve
DM-6858first (we currently compare the list of centrally-defined datasets against a list hard-coded into the test). - This will change the location of some data products for at least some cameras. I think this is a good thing in the long run, as there's no reason for different cameras to put common datasets in different locations, but this opinion may not be unanimous, and it may cause some backwards compatibility problems. If these are important, I suggest we define additional backwards compability mappers that do not move these datasets for a transitional period. The good news is that a preliminary look suggests that most of our mappers already define things consistently; obs_sdss may be the only one that uses different conventions for coadd datasets.
I do not have anyone already signed up to do this work (I'm hoping to get by on the kindness of TCAMs), but this really shouldn't be more than a couple of days - there are a lot of lines to move and reformat, but it's dead simple.
I do believe doing this sooner rather than later (and in particular not waiting on the dynamic dataset definition feature) will make Science Pipelines developers' lives easier, and I think it may be helpful to Nate Pease [X] in adding support for compound datasets as well.
Implementation issue is
DM-7049, which will only make strictly backwards-compatible changes. It will also spawn new per-camera issues for non-backwards-compatible standardization.