# Update the pipe_analysis scripts to have the option to apply fgcm calibrations

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
10
• Sprint:
DRP F19-5, DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb)
• Team:
Data Release Production
• Urgent?:
No

#### Description

The fgcmcal "Forward Global Calibration Method" calibration is adding a new dataset type and product in the form of the usual photoCalib object (called fgcm_tract_photoCalib but currently only defined in obs_subaru, see DM-16704). The scripts in pipe_analysis require updating to allow for the option of applying fgcm photometric calibrations in lieu of the jointcal ones.

#### Activity

Hide
Lauren MacArthur added a comment -

The application here will also follow the naming scheme for configs established in DM-21308.

Show
Lauren MacArthur added a comment - The application here will also follow the naming scheme for configs established in DM-21308 .
Hide
Lauren MacArthur added a comment - - edited

The scripts on this branch now allow the user to choose which external calibrations to apply to the visit-level src catalogs. Their application and which flavor to apply are determined by config parameters (as in pipe_tasks for coaddition):

 doApplyExternalPhotoCalib externalPhotoCalibName doApplyExternalSkyWcs externalSkyWcsName 

(there is still accommodation for any vestigial meas_mosaic runs...but I doubt they will ever be used going forward).

The compareVisitAnalysis.py script can be run with, e.g.

 externalPhotoCalibName1="fgcm" externalPhotoCalibName2="jointcal" 

in order to directly compare the mapping between the two calibrations. As an example, this is the mapping between the fgcm & jointcal photometric calibrations for a COSMOS HSC-I band visit:

Most uses of the term "uber" in regards to calibrations have been replaced with "external". The one exception, as described in the commit message, is a variable named uberCalLabel as doing this change now would cause some serious rebasing nightmares for a few far-reaching tickets that our soon to land. It will be updated to externalCalLabel once a less disruptive phase is reached.

Show
Lauren MacArthur added a comment - - edited The scripts on this branch now allow the user to choose which external calibrations to apply to the visit-level src catalogs. Their application and which flavor to apply are determined by config parameters (as in pipe_tasks for coaddition): doApplyExternalPhotoCalib externalPhotoCalibName doApplyExternalSkyWcs externalSkyWcsName (there is still accommodation for any vestigial meas_mosaic runs...but I doubt they will ever be used going forward). The compareVisitAnalysis.py script can be run with, e.g. externalPhotoCalibName1 = "fgcm" externalPhotoCalibName2 = "jointcal" in order to directly compare the mapping between the two calibrations. As an example, this is the mapping between the fgcm & jointcal photometric calibrations for a COSMOS HSC-I band visit: Most uses of the term "uber" in regards to calibrations have been replaced with "external". The one exception, as described in the commit message, is a variable named uberCalLabel as doing this change now would cause some serious rebasing nightmares for a few far-reaching tickets that our soon to land. It will be updated to externalCalLabel once a less disruptive phase is reached.
Hide
Lauren MacArthur added a comment - - edited

Would you mind reviewing this?  You have seen many of the relevant plots demonstrating the functionality already, but you can dig deeper if you'd like looking at the plots in https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-21052/plots (some don't use the functionality directly, but I ran all scripts to make sure nothing outright broke and that the labels are appropriate and reasonably placed).

Oh, and a question for you in particular:  what do you think the default should be for externalPhotoCalibName : "jointcal", "fgcm", or "fgcm_tract"?

Show
Lauren MacArthur added a comment - - edited Would you mind reviewing this?  You have seen many of the relevant plots demonstrating the functionality already, but you can dig deeper if you'd like looking at the plots in https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-21052/plots  (some don't use the functionality directly, but I ran all scripts to make sure nothing outright broke and that the labels are appropriate and reasonably placed). Oh, and a question for you in particular:  what do you think the default should be for externalPhotoCalibName : "jointcal", "fgcm", or "fgcm_tract"? PR is: https://github.com/lsst-dm/pipe_analysis/pull/49
Hide
Eli Rykoff added a comment -

Looks good.

Show
Eli Rykoff added a comment - Looks good.

#### People

Assignee:
Lauren MacArthur
Reporter:
Lauren MacArthur
Reviewers:
Eli Rykoff
Watchers:
Eli Rykoff, Lauren MacArthur, Yusra AlSayyad