Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-440

Update calibrations in /datasets/hsc/repo and enable new HSC pipeline steps

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      We (Paul Price and Jim Bosch) would like to update the master calibrations used by default for HSC processing on lsst-dev, including both existing calibration frame types (flats, biases, darks, fringes) and two new calibration-like datasets:

      • Sky Frames: measurements of the persistent (average) sky background in HSC visits across different filters.  This will be used in the new visit-level background estimation step (https://community.lsst.org/t/sky-subtraction/2415).
      • Transmission Curves: measurements of the wavelength-dependent transmission for different parts of the optical system, which will be combined and added to Exposures in during ISR after the merge of DM-12375.

      Both of these new processing steps will be enabled by default after the necessary calibrations are added to lsst-dev, which means any processing of HSC data with the default configuration on other platforms will also required those new calibrations to be added.  Instructions for adding the sky frames can be found in the above community.lsst.org link; while adding transmission curves can be done (after DM-12375) with:

      setup obs_subaru
      installTransmissionCurves.py /path/to/repo

      Note that the above path is to the regular data repository root, not a calibRoot; transmission curves are not yet versioned (they will be in the Gen3 butler, if not earlier).

      Both of these steps will only be used in HSC processing for now.  They could be enabled for other cameras once the work is done to provide the necessary calibrations for those cameras.

      As per our policy on modifications to /datasets:

      • Description and reason for addition/change/deletion: updates to the existing calibration frames reflect our most up-to-date understanding of the camera's characteristics.  The new datasets enable new processing steps that improve the quality of the reductions.
      • Target top-level-directory for location of addition/change/deletion: /datasets/hsc/repo
      • Organization of data:
        • calibration and sky frames will go in a new subdirectory of /datasets/hsc/repo/calibs, and the /datasets/hsc/repo/CALIB symlink will be updated to point to the new subdirectory
        • transmission curves will go into /datasets/hsc/repo/transmission
      • Other necessary domain knowledge as identified by project members relating to the contents of the data: provided in the community.lsst.org link above

       

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            Adopted.

            Show
            price Paul Price added a comment - Adopted.

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Swinbank, Paul Price
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Planned End: