# Override source selection criteria in SFM calibration stages for LSSTCam-imSim

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
5
• Team:
Data Release Production
• Urgent?:
No

#### Description

In the analysis done on DM-16875, the suggestion was made to override the default config settings for the source selection criteria for consideration of a given source as a suitable candidate for the PSF modeling. In particular, the default setting of a minimum flux of 12500 (raw counts, I believe) seems way too high and, ideally, the cut would not be based on flux, but rather on S/N. Justification for this based on HSC data as well as the S/N thresholding functionality was added on DM-17043. [Note that the fluxMin value has long been overridden to 4000 in obs_subaru].

It seemed this change was widely agreed upon by those involved in the imsim/DC2 Run2.2i processing and was adopted – in addition to several other source selection overrides for other calibration phases – in at least some of the DESC workflows (see, e.g. the overrides in their "dc2/run2.2" branch run: https://github.com/lsst/obs_lsst/blob/dc2/run2.2/config/imsim/processCcd.py#L39-L42
and note the other overrides in that file). Unfortunately, during an investigation on PREOPS-885, it was noted that these overrides have not found their way into the master/main config files in obs_lsst, so they are not currently in effect for any processing based off the master/main configs. This situation should be rectified.

Some care may be required in the actual S/N threshold set and whether the other updates in the file pointed to above should also be adopted. In particular, I would love to hear from anyone involved the DC2 processing campaigns as to the validation of those choices (I added a few watchers as a first guess, but please do add/remove as you see fit!) I also have no idea what was used in the DP0.1 processing run, so if anyone could chime in on that, it would be greatly appreciated!

#### Activity

Lauren MacArthur created issue -
Field Original Value New Value
Link This issue relates to DM-16785 [ DM-16785 ]
 Link This issue relates to DM-17043 [ DM-17043 ]
 Link This issue relates to PREOPS-885 [ PREOPS-885 ]
 Attachment plot-v277060-u-base_PsfFluxSn_raw-hist.png [ 54866 ]
 Attachment plot-v193827-g-base_PsfFluxSn_raw-hist.png [ 54867 ]
 Description In the analysis done on DM-16875, the suggestion was made to override the default config settings for the source selection criteria for consideration of a given source as a suitable candidate for the PSF modeling. In particular, the default setting of a minimum flux of 12500 (raw counts, I believe) seems way too high and, ideally, the cut would not be based on flux, but rather on S/N. Justification for this based on HSC data as well as the S/N thresholding functionality was added on DM-17043. [Note that the {{fluxMin}} value has long been overridden to 4000 in {{obs_subaru}}]. It seemed this change was widely agreed upon by those involved in the {{imsim}}/DC2 Run2.2i processing and was adopted – in addition to several other source selection overrides for other calibration phases – in at least some of the DESC workflows (see, e.g. the overrides in their "dc2/run2.2" branch run: [https://github.com/lsst/obs_lsst/blob/dc2/run2.2/config/imsim/processCcd.py#L39-L42] and note the other overrides in that file). Unfortunately, during an investigation on PREOPS-885, it was noted that these overrides have not found their way into the master/main config files in {{obs_lsst}}, so they are not currently in effect for any processing based off the master/main configs. This situation should be rectified. Some care may be required in the actual S/N threshold set and whether the other updates in the file pointed to above should also be adopted. In particular, I would love to hear from anyone involved the DC2 processing campaigns as to the validation of those choices (I added a few watchers as a first guess, but please do add/remove as you see fit!) I also have no idea what was used in the DP0.1 processing run, so if anyone could chime in on that, it would be greatly appreciated! In the analysis done on DM-16875, the suggestion was made to override the default config settings for the source selection criteria for consideration of a given source as a suitable candidate for the PSF modeling. In particular, the default setting of a minimum flux of 12500 (raw counts, I believe) seems way too high and, ideally, the cut would not be based on flux, but rather on S/N. Justification for this based on HSC data as well as the S/N thresholding functionality was added on DM-17043. $Note that the {{fluxMin}} value has long been overridden to 4000 in {{obs_subaru}}$. It seemed this change was widely agreed upon by those involved in the {{imsim}}/DC2 Run2.2i processing and was adopted – in addition to several other source selection overrides for other calibration phases – in at least some of the DESC workflows (see, e.g. the overrides in their "dc2/run2.2" branch run: [https://github.com/lsst/obs_lsst/blob/dc2/run2.2/config/imsim/processCcd.py#L39-L42] and note the other overrides in that file). Unfortunately, during an investigation on PREOPS-885, it was noted that these overrides have not found their way into the master/main config files in {{obs_lsst}}, so they are not currently in effect for any processing based off the master/main configs. This situation should be rectified. Some care may be required in the actual S/N threshold set and whether the other updates in the file pointed to above should also be adopted. In particular, I would love to hear from anyone involved the DC2 processing campaigns as to the validation of those choices (I added a few watchers as a first guess, but please do add/remove as you see fit!) I also have no idea what was used in the DP0.1 processing run, so if anyone could chime in on that, it would be greatly appreciated!
 Summary Override source selection criteria in SFM calibration stages fo LSSTCam-imSim Override source selection criteria in SFM calibration stages for LSSTCam-imSim
 Attachment plot-v2334-u-base_GausssianFluxSn_raw-hist.png [ 54891 ]
 Assignee Lauren MacArthur [ lauren ]
 Status To Do [ 10001 ] In Progress [ 3 ]
 Attachment plot-v2334-u-base_GausssianFluxSn_raw-hist.png [ 54891 ]
 Attachment plot-v277060-u-base_GausssianFluxSn_raw-hist.png [ 54902 ]
 Attachment plot-v277060-u-base_PsfFluxSn_rawIcSrc-hist.png [ 54927 ]
 Attachment plot-v277060-u-base_PsfFluxSn_rawIcSrc-hist_DM-32624.png [ 54928 ]
 Attachment plot-v277060-u-psfTraceDiff-psfMagHist.png [ 54929 ]
 Attachment plot-v2336-u-psfTraceDiff-psfMagHist_DM-32624.png [ 54930 ]
 Attachment compareVisit-v2336-u-psfTrace-psfMagHist.png [ 54931 ]
 Attachment compareVisit-v2336-u-diff_base_PsfFlux-psfMagHist.png [ 54932 ]
 Attachment plot-v277060-u-matches_PSF-psfMagHist.png [ 54933 ]
 Attachment plot-v2336-u-matches_PSF-psfMagHist_DM-32624.png [ 54934 ]
 Attachment compareVisit-v2336-u-diff_base_PsfFlux_apCorr-sky-stars.png [ 54935 ]
 Attachment psfEllipRediduals_test-med-1.pdf [ 54936 ]
 Attachment psfScaledSizeScatter_test-med-1.pdf [ 54937 ]
 Attachment psfScaledSizeScatter_test-med-1.pdf [ 54937 ]
 Attachment psfEllipRediduals_test-med-1.pdf [ 54936 ]
 Attachment psfEllipRediduals_test-med-1.png [ 54938 ] Attachment psfScaledSizeScatter_test-med-1.png [ 54939 ]
 Attachment psfScaledSizeScatter_HSC_RC2.png [ 54940 ] Attachment psfEllipRediduals_HSC_RC2.png [ 54941 ]
 Reviewers Yusra AlSayyad [ yusra ] Status In Progress [ 3 ] In Review [ 10004 ]
 Labels SciencePipelines SciencePipelines backport-v23
 Status In Review [ 10004 ] Reviewed [ 10101 ]
 Resolution Done [ 10000 ] Status Reviewed [ 10101 ] Done [ 10002 ]
 Labels SciencePipelines backport-v23 SciencePipelines backport-approved backport-v23
 Labels SciencePipelines backport-approved backport-v23 SciencePipelines backport-approved backport-done backport-v23
 Story Points 5
 Epic Link DM-30540 [ 511198 ]

#### People

Assignee:
Lauren MacArthur
Reporter:
Lauren MacArthur
Reviewers:
Watchers:
Colin Slater, Eli Rykoff, Heather Kelly, James Chiang, Jim Bosch, Lauren MacArthur, Yusra AlSayyad