DM-11332 introduces FITS compression, which may be configured with a variety of options (e.g., compression algorithm, tile size, quantization). We need to be able to select the FITS compression options according to the dataset because compression options relevant/optimal for calexps may not match the compression options relevant/optimal for coadds. The solution proposed in DM-11332 is the introduction of "recipes" for writing datasets.
A recipe is a set of configuration settings with a symbolic name that may be used to refer to the set. Recipes are defined in obs_base/policy/writeRecipes.yaml; obs packages may define their own custom recipes in obs_whatever/policy/writeRecipes.yaml, which overrides those defined in obs_base. Recipes may be applied to a particular dataset through the mapper policy file (typically obs_whatever/policy/WhateverMapper.yaml) by providing a recipe element for the dataset. The default recipe (used if no recipe is explicitly provided) is called default. Recipes are currently only used for the FitsStorage storage type (used for FITS images), but there's no reason why they couldn't be used for other storage types if required to configure how the writing is done.
Here is the proposed obs_base/policy/writeRecipes.yaml:
Here's an example in obs_subaru/policy/HscMapper.yaml:
This design is based on a similar design used in the Pan-STARRS Image Processing Pipeline. In that case, we defined "fits types" which could be selected by their symbolic name in the "file rules". We found that this design worked well, allowing simple selection of the parameters required for writing images. The Pan-STARRS IPP design treated image, mask and variance planes separately, but since in LSST these are treated together, they are together in the recipe.
One possibility is to leave off recipes and fold the options into the storage type, maybe something like:
It's clear that, even for this simple example, the options are rather long and would have to be repeated several times for each dataset. The use of recipes as proposed simplifies the description and removes the need for repetition.
For more information, see: