# Propagate astrometry and photometry visit calibration flags to coadds

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
6
• Sprint:
DRP F18-4
• Team:
Data Release Production

#### Description

DM-4878 added a mechanism for propagating flags from the input catalogues to the coadd catalogue indicating stars that were used for measuring the PSF. Since then, DM-9050 added similar calibration flags for astrometry and photometry which should also be added to the list of flags that get propagated from visits to coadds.

The default list of propagated flags is currently:

 flags = DictField(keytype=str, itemtype=float,  default={"calib_psf_candidate": 0.2, "calib_psf_used": 0.2, },  doc="Source catalog flags to propagate, with the threshold of relative occurrence.") 

• calib_astrometry_used
• calib_photometry_used
• calib_photometry_reserved

#### Activity

Hide
Lauren MacArthur added a comment - - edited

Would you mind giving this a look?  Seeing as you have been poking around the plots from pipe_analysis, you understand the motivation.

Unit testing is not really feasible as we need a fully processed (through multiband) dataset, so I added a test in ci_hsc.

The PR for pipe_tasks (which is not showing up in Jira these days) is: https://github.com/lsst/pipe_tasks/pull/217

I ran a Jenkins run before the weekend which passed. Just to be on the safe side, I also kicked one off today [UPDATE: it also passed].

Oh, and I also created a coadd and checked that the sources that get these flags set do indeed look sensible.

Show
Lauren MacArthur added a comment - - edited Would you mind giving this a look?  Seeing as you have been poking around the plots from pipe_analysis, you understand the motivation. Unit testing is not really feasible as we need a fully processed (through multiband) dataset, so I added a test in ci_hsc . The PR for pipe_tasks (which is not showing up in Jira these days) is: https://github.com/lsst/pipe_tasks/pull/217 I ran a Jenkins run before the weekend which passed . Just to be on the safe side, I also kicked one off today [UPDATE: it also passed] . Oh, and I also created a coadd and checked that the sources that get these flags set do indeed look sensible.
Hide
Meredith Rawls added a comment -

Looks good to me. The only thing I wasn't clear about is why the default values are all 0.2. If you could possibly clarify that in the docstring, that would be helpful.

Show
Meredith Rawls added a comment - Looks good to me. The only thing I wasn't clear about is why the default values are all 0.2. If you could possibly clarify that in the docstring, that would be helpful.
Hide
Lauren MacArthur added a comment -

This seemed like more than enough justification to me, and I felt the logic also applies to the flags added here.  Let me know if you disagree or think a note needs to be added elsewhere to make it more visible.

Show
Lauren MacArthur added a comment - Very fair question.  There is a lengthy discussion in the task description here: https://github.com/lsst/pipe_tasks/blob/master/python/lsst/pipe/tasks/propagateVisitFlags.py#L64-L90 This seemed like more than enough justification to me, and I felt the logic also applies to the flags added here.  Let me know if you disagree or think a note needs to be added elsewhere to make it more visible.
Hide
Meredith Rawls added a comment -

Indeed, that provides plenty of justification, but is a rather odd place for it. I think I was further confused by the default for the following config field, matchRadius, which also happens to be 0.2... arcseconds.

I'd prefer a second sentence added to the docstring along the lines of, "May be in the range 0-1; default 0.2 will flag the coadd if at least 20% of its inputs are flagged." I didn't fully understand that's what was happening here until you pointed me to the task description.

Show
Meredith Rawls added a comment - Indeed, that provides plenty of justification, but is a rather odd place for it. I think I was further confused by the default for the following config field, matchRadius, which also happens to be 0.2... arcseconds. I'd prefer a second sentence added to the docstring along the lines of, "May be in the range 0-1; default 0.2 will flag the coadd if at least 20% of its inputs are flagged." I didn't fully understand that's what was happening here until you pointed me to the task description.
Hide
Lauren MacArthur added a comment -

The config parameter doc now reads (with a --show config=propagateFlags* on a multiband.py run):

 # Source catalog flags to propagate, with the threshold of relative occurrence (valid range: [0-1], default is 0.2). Coadd object will have flag set if the fraction of input visits in which it is flagged is greater than the threshold. config.measureCoaddSources.propagateFlags.flags={'calib_psf_candidate': 0.2, 'calib_psf_used': 0.2, 'calib_psf_reserved': 0.2, 'calib_astrometry_used': 0.2, 'calib_photometry_used': 0.2, 'calib_photometry_reserved': 0.2}

Show
Lauren MacArthur added a comment - The config parameter doc now reads (with a --show config=propagateFlags* on a multiband.py run): # Source catalog flags to propagate, with the threshold of relative occurrence (valid range: [0-1], default is 0.2). Coadd object will have flag set if the fraction of input visits in which it is flagged is greater than the threshold. config.measureCoaddSources.propagateFlags.flags={'calib_psf_candidate': 0.2, 'calib_psf_used': 0.2, 'calib_psf_reserved': 0.2, 'calib_astrometry_used': 0.2, 'calib_photometry_used': 0.2, 'calib_photometry_reserved': 0.2}
Hide
Lauren MacArthur added a comment -

Thanks Meredith.  Docstring updated and merged to master.

Show
Lauren MacArthur added a comment - Thanks Meredith.  Docstring updated and merged to master.

#### People

Assignee:
Lauren MacArthur
Reporter:
Lauren MacArthur
Reviewers:
Meredith Rawls
Watchers:
John Swinbank, Lauren MacArthur, Meredith Rawls, Yusra AlSayyad