Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-272

Make PSF-Matched and Non-PSF-Matched CoaddTempExps simultaneously

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Currently, users choose whether to make PSF-Matched or default (Non-PSF-Matched) Temp Exps and Coadds by flipping the boolean config: CoaddBaseConfig.doPsfMatch. To create both, users must run makeCoaddTempExp.py twice and assembleCoadd.py twice. This warps the calexps (a computationally expensive operation) twice. In this RFC, I propose to change the behavior and interface of MakeCoaddTempExpTask to create both types of coadds simultaneously by default, and optionally one or the other per a command-line config. By default MakeCoaddTempExpTask will hold two TempExps in memory, each calexp will be read in, warped, placed into image #1, then matched and placed into image #2.

      See github https://github.com/lsst/pipe_tasks/compare/tickets/DM-8491 for illustration. (This is not a PR)
      You may ask, why make the warps and psf-matched warps per calexp in simultaneously instead of in serial? One can imagine making a complete set of non-matched temp exps, and then the psf-matching these temp exps. This would be nice because a user could change their mind about the FWHM of the model PSF and re-match the warps for half the cost, but this isn’t feasible at the moment because: (1) Placement of calexps on the patch can cut off too much of the original calexp to retain a sufficient number of kernel candidates for accurate PSF-Matching (2) PSFs in the warps are discontinuous at chip boundaries. It is technically feasible to store enough information about the PSF models in the warps and make PSF models with known (not-rectangular) discontinuities, but this complex functionality does not yet exist in the stack.

      Comments welcome on:

      1. Config names and default behavior for makeCoaddTempExpTask and assembleCoaddTask:
        • Default behavior change: make both psfMatched and non-Psf-Matched Coadds by default and use two new configs:
          • CoaddBaseConfig.makeOnlyPsfMatched=False
          • CoaddBaseConfig.makeOnlyDirect=False
            With these configs, current default behavior of just creating non-matched coadds would be replicated with setting CoaddBaseConfig.makeOnlyDirect=True.
      2. Name of new data products:
        • deepCoadd_tempExpPsfMatched
        • deepCoaddPsfMatched
      3. New interface of WarpAndPsfMatch.run(self, exposure, wcs, modelPsf=None, maxBBox=None, destBBox=None) which currently returns a regular temp exp if modelPsf is None and a matched temp Exp if not None. If we need WarpAndPsfMatch to keep its original responsibilities, I propose returning a Struct from WarpAndPsfMatch with a warped and matched calexp called “exposurePsfMatched” and a warped exposure called “exposure.” However, WarpAndPsfMatch is so minimal I don’t know why it needs to be its own task… No tasks other than makeCoaddTempExp depend on it, and warping and psfMatching could easily be called directly from makeCoaddTempExpTask.
      4. How will this affect supertasks or pipe_drivers?

      Paul Price, you mentioned you had plans to refactor assembleCoadd and makeCoaddTempExp soon. I’d like to hear what you were planning to make sure these changes fit within your vision.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            As for nomenclature, as per DMTN-15 I'd recommend using "direct" as the word for "not PSF-matched"; in the future I anticipate having "PsfMatched", "Direct", and "Likelihood" flavors of coaddTempExps.

            If WarpAndPsfMatch gets in your way at all, I'd be happy to see it gone. We probably shouldn't make task hierarchy decisions just to make the config hierarchy simpler and less verbose, but removing it would certainly do that, which is good.

            Show
            jbosch Jim Bosch added a comment - As for nomenclature, as per DMTN-15 I'd recommend using "direct" as the word for "not PSF-matched"; in the future I anticipate having "PsfMatched", "Direct", and "Likelihood" flavors of coaddTempExps. If WarpAndPsfMatch gets in your way at all, I'd be happy to see it gone. We probably shouldn't make task hierarchy decisions just to make the config hierarchy simpler and less verbose, but removing it would certainly do that, which is good.
            Hide
            yusra Yusra AlSayyad added a comment -

            Thank you Jim Bosch. I'm on board with "Direct," and I appreciate the lack of attachment to WarpAndPsfMatch.

            Show
            yusra Yusra AlSayyad added a comment - Thank you Jim Bosch . I'm on board with "Direct," and I appreciate the lack of attachment to WarpAndPsfMatch.
            Hide
            yusra Yusra AlSayyad added a comment - - edited

            I'd also like comments on my model of how the coadds are named with a prefix and a suffix:

            [deep|goodSeeing]Coadd[Direct|PsfMatched|Likelihood]

            • The prefix has historically signified how the TempExps are assembled into coadds i.e. what subset of TempExps are used and their weighting. I leave this as-is.
            • The new suffix will contain how the TempExps were created. i.e. what convolutions have have been performed.

            To be very explicit that means we can have any of the N_suffix X N_prefix combinations:

            • deepCoaddDirect
            • deepCoaddPsfMatched
            • deepCoaddLikelihood
            • goodSeeingCoaddDirect
            • goodSeeingCoaddPsfMatched <--prob won't ever do this
            • goodSeeingCoaddLikelihood

            Corollary: This means that the "deep" in deepCoadd_tempExp doesn't mean anything, because we will assemble both deep and goodSeeing coadds from the same TempExps. I'll likely rename the deepCoadd_tempExp's at some point. I know project members who have worked on Pan-STARRS have strong opinions on what that should be, and I do not want tempExp-naming to take over this RFC. So look out for a later RFC on that after multiple simultaneous coadds are implemented.)

            Show
            yusra Yusra AlSayyad added a comment - - edited I'd also like comments on my model of how the coadds are named with a prefix and a suffix: [deep|goodSeeing] Coadd [Direct|PsfMatched|Likelihood] The prefix has historically signified how the TempExps are assembled into coadds i.e. what subset of TempExps are used and their weighting. I leave this as-is. The new suffix will contain how the TempExps were created. i.e. what convolutions have have been performed. To be very explicit that means we can have any of the N_suffix X N_prefix combinations: deepCoaddDirect deepCoaddPsfMatched deepCoaddLikelihood goodSeeingCoaddDirect goodSeeingCoaddPsfMatched <--prob won't ever do this goodSeeingCoaddLikelihood Corollary: This means that the "deep" in deepCoadd_tempExp doesn't mean anything, because we will assemble both deep and goodSeeing coadds from the same TempExps. I'll likely rename the deepCoadd_tempExp's at some point. I know project members who have worked on Pan-STARRS have strong opinions on what that should be, and I do not want tempExp-naming to take over this RFC. So look out for a later RFC on that after multiple simultaneous coadds are implemented.)
            Hide
            yusra Yusra AlSayyad added a comment - - edited

            After Jim Bosch pointed out that we anticipate 3 types of temp exps, I'm changing the xor-type configs initially proposed, to be positive "and"-type configs:

            • makeDirect
            • makePsfMatched
            • makeLikelihood (not implemented now)

            These would be more intuitive, and flexible to make any combination of three temp exps.
            This RFC will be implemented on DM-8491

            Show
            yusra Yusra AlSayyad added a comment - - edited After Jim Bosch pointed out that we anticipate 3 types of temp exps, I'm changing the xor-type configs initially proposed, to be positive "and"-type configs: makeDirect makePsfMatched makeLikelihood (not implemented now) These would be more intuitive, and flexible to make any combination of three temp exps. This RFC will be implemented on DM-8491
            Hide
            tjenness Tim Jenness added a comment -

            Yusra AlSayyad the work triggered by this RFC has been completed. Does this mean the RFC can be marked implemented?

            Show
            tjenness Tim Jenness added a comment - Yusra AlSayyad the work triggered by this RFC has been completed. Does this mean the RFC can be marked implemented?

              People

              Assignee:
              yusra Yusra AlSayyad
              Reporter:
              yusra Yusra AlSayyad
              Watchers:
              Jim Bosch, John Swinbank, Paul Price, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.