Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-233

Proposed change to datasets for LsstSim mapper

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      RFC for DM-7677: recommended changes to lsstSimMapper.paf file

      Request to modify some templates and dataset names to make the obs_* Mapper classes more consistent.

      DM-7049 was intended to make the Mapper datasets more consistent by centralizing datasets which could be shared across Mappers.
      The latest HscMapper.paf was used as a model. Datasets were moved to datasets.yaml and exposures.yaml in daf_butlerUtils.

      1. Datasets in the individual Mapper.paf which matched the shared ones in butlerUtils were deleted. To avoid changing any datasets deletion was only done on those matched in both name and content. Those which matched in name but not content are listed (1) below.
      2. Datasets were also flagged when it appeared that they could be shareable, but in fact were not in HscMapper at all. These are candidates to be renamed and moved to daf_butlerUtils. Some of these could also be datasets no longer in use.

      Please also see RFC-231 for datasets which are specifically connected to Coadd and Difference imaging. They were omitted from (2).

      -------------------------------------

      1. Datasets with same name but different content in lsstSimMapper.paf
      The difference is shown between the existing paf dataset and the
      proposed change to the "ref": the dataset is defined in butlerUtils.

      datasets.deepCoadd_calexp_background
           template	deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/b
                    	kgd-%(filter)s-%(tract)d-%(patch)s.fits} }
        ref:template	deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/d
                    	et_bkgd-%(filter)s-%(tract)d-%(patch)s.fits
      datasets.deepCoadd_forced_metadata
            template	deepCoadd_forced_metadata/%(filter)s/%(tract)d/%(p
                    	atch)s.boost
        ref:template	deepCoadd_forcedPhotCoadd_metadata/%(filter)s/%(tr
                    	act)d/%(patch)s.boost
      datasets.deepCoadd_forced_config
            python	lsst.meas.base.forcedCoadd.ForcedCoaddMeasurementC
                    	onfig
        ref:python	lsst.meas.base.forcedPhotCoadd.ForcedPhotCoaddConf
                    	ig
            template	config/deepCoadd_forced.py
        ref:template	config/forcedPhotCoadd.py
      datasets.deepCoadd_forced_src
            template	deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/f
                    	orced-%(filter)s-%(tract)d-%(patch)s.fits
        ref:template	deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/f
                    	orced_src-%(filter)s-%(tract)d-%(patch)s.fits
      datasets.deep_safeClipAssembleCoadd_metadata
            template	deep_safeClipAssembleCoadd_metadata/%(filter)s/%(t
                    	ract)d/%(patch)s.boost
        ref:template	deep_assembleCoadd_metadata/%(filter)s/%(tract)d/%
                    	(patch)s.boost
      datasets.forced_src_schema
            template	schema/forced.fits
        ref:template	schema/forced_src_schema.fits
      

      -------------------------------------

      2. Datasets which should be examined to see if they should be
      removed, renamed, or moved to dafButlerUtils.
      Note mappers with the same dataset name are shown:
      l=lsstSim,d=decam,s=suprimecam,S=sdss,m=megacam,t=test

      datasets.badSourceHist: Slt
      datasets.badSource_schema: Slt
      datasets.badSource: Slt
      datasets.cal_ref_cat: lt
      datasets.invalidSource_schema: Slt
      datasets.invalidSource: Slt
      datasets.measureCcd_config: lt
      datasets.measureCcd_metadata: lt
      datasets.modelfits: mlt
      datasets.modelfits_schema: mlt
      datasets.Mosaic_metadata: lt
      datasets.multiband_config: smSl
      datasets.object_schema: Slt
      datasets.object: Slt
      datasets.refcat: lt
      datasets.sourceAssoc_config: Slt
      datasets.sourceAssoc_metadata: Slt
      datasets.sourceHist: Slt
      datasets.source_schema: Slt
      datasets.source: Slt
      exposures.dcrModel: l
      

        Attachments

          Issue Links

            Activity

            Hide
            pgee Perry Gee added a comment -

            Lauren MacArthur, there is a dataset called Mosaic_metadata (no config) in this mapper, but it does not appear to be related to the MosaicTask in meas_mosaic. Can you comment on whether this dataset is in use?

            Show
            pgee Perry Gee added a comment - Lauren MacArthur , there is a dataset called Mosaic_metadata (no config) in this mapper, but it does not appear to be related to the MosaicTask in meas_mosaic. Can you comment on whether this dataset is in use?
            Hide
            pgee Perry Gee added a comment -

            This is an incorrect persistable setting in lsstSim:

            forcedPhotCoaddConfig is the correct class for deepCoadd_forced_config in lsstSim Template may be wrong too, but it is somewhat dangerous to correct so I will not.

            Show
            pgee Perry Gee added a comment - This is an incorrect persistable setting in lsstSim: forcedPhotCoaddConfig is the correct class for deepCoadd_forced_config in lsstSim Template may be wrong too, but it is somewhat dangerous to correct so I will not.
            Hide
            lauren Lauren MacArthur added a comment -

            Sorry for taking so long to respond. I have no familiarity with this obs package nor any plans for it to use meas_mosaic (although I'd highly doubt there are any such intentions seeing as jointcal will be LSST's version). However, the commit message introducing the mosaic-related datasets confirms their association with meas_mosaic:

            https://github.com/lsst/obs_lsstSim/commit/2afb17daa100266e109fd4116343f09a03dd2eaf

            and it was your commit:

            https://github.com/lsst/obs_lsstSim/commit/73ea51022c2c85bc68b3711c493d406942e1d339

            that removed the config dataset.

            Show
            lauren Lauren MacArthur added a comment - Sorry for taking so long to respond. I have no familiarity with this obs package nor any plans for it to use meas_mosaic (although I'd highly doubt there are any such intentions seeing as jointcal will be LSST's version). However, the commit message introducing the mosaic-related datasets confirms their association with meas_mosaic : https://github.com/lsst/obs_lsstSim/commit/2afb17daa100266e109fd4116343f09a03dd2eaf and it was your commit: https://github.com/lsst/obs_lsstSim/commit/73ea51022c2c85bc68b3711c493d406942e1d339 that removed the config dataset.
            Hide
            pgee Perry Gee added a comment -

            Implemented changes to this Mapper.paf from RFC-233

            Show
            pgee Perry Gee added a comment - Implemented changes to this Mapper.paf from RFC-233
            Hide
            pgee Perry Gee added a comment -

            The modifications for the last described set of changes will appear in DM-8128

            Show
            pgee Perry Gee added a comment - The modifications for the last described set of changes will appear in DM-8128

              People

              • Assignee:
                pgee Perry Gee
                Reporter:
                pgee Perry Gee
                Watchers:
                Colin Slater, Jim Bosch, John Parejko, Lauren MacArthur, Perry Gee, Russell Owen
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Planned End:

                  Summary Panel