Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33982

Apply new finalized psf models to coadds/forced measurements

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_tasks
    • Labels:
      None

      Description

      When DM-33645 is done, the next step is to allow the new psfs and aperture corrections to be used downstream. The basic idea is to follow the external photocalib and wcs code from https://github.com/lsst/pipe_tasks/blob/e01ce967e736e719928a61476ba89203dfe3730f/python/lsst/pipe/tasks/makeCoaddTempExp.py#L830 and https://github.com/lsst/meas_base/blob/26961ef6c7a3bd91c4b40a0307c740244fea775e/python/lsst/meas/base/forcedPhotCcd.py#L630 and https://github.com/lsst/faro/blob/a97a9ee4e67ff257dd51f441c225d798fb7ba92e/python/lsst/faro/base/MatchedCatalogBase.py#L244 with an doApplyFinalizedPsf option (which will imply equivalent aperture corrections as well).

        Attachments

          Activity

          Hide
          erykoff Eli Rykoff added a comment -

          In order to support loading the external psf in ForcedPhotCcdTask I've had to implement what was to be DM-17062 (Yusra AlSayyad).

          Also, on further review faro does not need to be touched on this ticket, since it only uses catalogs and not the calexp or psf directly.

          Show
          erykoff Eli Rykoff added a comment - In order to support loading the external psf in ForcedPhotCcdTask I've had to implement what was to be DM-17062 ( Yusra AlSayyad ). Also, on further review faro does not need to be touched on this ticket, since it only uses catalogs and not the calexp or psf directly.
          Hide
          yusra Yusra AlSayyad added a comment -

          Couple high level comments before reviewing for real this weekend:

          1) You modified ImageDifferenceTasks's runQuantum, but ImageDifferenceFromTemplateTask is the one that we currently run in the Gen3 DRP https://github.com/lsst/drp_pipe/blob/main/ingredients/DRP.yaml#L58  and AP https://github.com/lsst/ap_pipe/blob/main/pipelines/ApPipe.yaml#L36 pipelines. That needs an edit too. 

          2) The default DRP pipelines does not run jointcal or FGCM. So for consistency sake, the ForcedPhotCcd default configs for external calibs should remain False, and the overrides in obs_subaru should remain True. 

          Show
          yusra Yusra AlSayyad added a comment - Couple high level comments before reviewing for real this weekend: 1) You modified ImageDifferenceTasks 's runQuantum, but ImageDifferenceFromTemplateTask is the one that we currently run in the Gen3  DRP https://github.com/lsst/drp_pipe/blob/main/ingredients/DRP.yaml#L58   and AP https://github.com/lsst/ap_pipe/blob/main/pipelines/ApPipe.yaml#L36  pipelines. That needs an edit too.  2) The default DRP pipelines does not  run jointcal or FGCM. So for consistency sake, the ForcedPhotCcd default configs for external calibs should remain False, and the overrides in obs_subaru should remain True. 
          Hide
          erykoff Eli Rykoff added a comment -

          Thanks. Re (2) I've updated the defaults to be False, and explicitly turned on the external calibrations in the HSC DRP pipelines.

          Re (1), I updated the code in both ImageDifferenceTasks and ImageDifferenceFromTemplateTask but boy does the github diff viewer make it difficult to see that.

          I don't think there are any edits I need to make to ap_pipe at the moment, until we turn on the finalized psf models (and is this something that ap pipe would be using anyway?).

          Finally, I don't think there are actually any tests at the moment for using the finalized psf models. I'm not sure how to do that prior to running ci with the option turned on. Any advice welcome!

          Show
          erykoff Eli Rykoff added a comment - Thanks. Re (2) I've updated the defaults to be False, and explicitly turned on the external calibrations in the HSC DRP pipelines. Re (1), I updated the code in both ImageDifferenceTasks and ImageDifferenceFromTemplateTask but boy does the github diff viewer make it difficult to see that. I don't think there are any edits I need to make to ap_pipe at the moment, until we turn on the finalized psf models (and is this something that ap pipe would be using anyway?). Finally, I don't think there are actually any tests at the moment for using the finalized psf models. I'm not sure how to do that prior to running ci with the option turned on. Any advice welcome!
          Hide
          yusra Yusra AlSayyad added a comment -

          Looks good, left a couple comments. The drp_pipe PR worried me that I wasn't clear about the consistency point about the configs:

          In a default DRP there's no FGCM/jointcal. 
          In all the obs_subaru pipelines, we use FGCM and jointcal, (even in ci_hsc with with those adjustments), which is why we have  https://github.com/lsst/obs_subaru/blob/main/config/forcedPhotCcd.py

          The only reason that we mentioned forcedPhotCcd in the gen3 yaml at all during the gen2->gen3 transition was to UNDO the  doApply*=True in ^^^ since we knew that didn't work in gen3 yet. 

          Thanks to this ticket, now those configs do work in gen3 too!  so those yaml configs can just disappear. Redundant configs are just confusing. 

          Show
          yusra Yusra AlSayyad added a comment - Looks good, left a couple comments. The drp_pipe PR worried me that I wasn't clear about the consistency point about the configs: In a default DRP there's no FGCM/jointcal.  In all the obs_subaru pipelines, we use FGCM and jointcal, (even in ci_hsc with with those adjustments), which is why we have   https://github.com/lsst/obs_subaru/blob/main/config/forcedPhotCcd.py The only reason that we mentioned forcedPhotCcd in the gen3 yaml at all during the gen2->gen3 transition was to UNDO the  doApply*=True in ^^^ since we knew that didn't work in gen3 yet.  Thanks to this ticket, now those configs do work in gen3 too!  so those yaml configs can just disappear. Redundant configs are just confusing. 
          Hide
          erykoff Eli Rykoff added a comment -
          Show
          erykoff Eli Rykoff added a comment - Final jenkins is clean https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36299/pipeline with ci_hsc, ci_imsim.

            People

            Assignee:
            erykoff Eli Rykoff
            Reporter:
            erykoff Eli Rykoff
            Reviewers:
            Yusra AlSayyad
            Watchers:
            Eli Rykoff, John Parejko, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.