# Add an "detect_isPrimary" flag to single frame processing schema

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
9
• Epic Link:
• Sprint:
DRP F20-1 (June), DRP F20-2 (July)
• Team:
Data Release Production
• Urgent?:
No

#### Description

The coadd level catalogs include a flag:

 "detect_isPrimary", type="Flag",  doc="true if source has no children and is in the inner region of a coadd patch "  "and is in the inner region of a coadd tract "  "and is not \"detected\" in a pseudo-filter (see config.pseudoFilterList)", 

that is very useful for culling catalogs of duplicate (patch/tract overlap), non-deblended parent sources (deblend_nChild>0, and sky objects. No equivalent flag currently exists at the visit /single frame measurement level, so this ticket is to add such a flag. Currently, the only two conditions to be added to flag undesirable sources are:
deblend_nChild == 0
sky_source is False

A community post describing this feature may be found here: https://community.lsst.org/t/primary-source-flag-for-single-frame-processing/4425

#### Activity

Hide
Lee Kelvin added a comment -

Thanks for taking a look at this ticket Meredith, much appreciated. We pushed it through Jenkins yesterday without issue, so hopefully it should be good to go.

Show
Lee Kelvin added a comment - Thanks for taking a look at this ticket Meredith, much appreciated. We pushed it through Jenkins yesterday without issue, so hopefully it should be good to go.
Hide
Meredith Rawls added a comment -

This adds a useful functionality, thanks for implementing it. Please take a look at my GitHub comments. The high-level question I have is, where do these flags ultimately land? In the source catalog as a new column alongside e.g. all the pixel flags? The definition of "primary" is a little funny to me since it really means "is a real source which has no child sources," so please take every opportunity in the code to remind us that is what is going on.

Show
Meredith Rawls added a comment - This adds a useful functionality, thanks for implementing it. Please take a look at my GitHub comments. The high-level question I have is, where do these flags ultimately land? In the source catalog as a new column alongside e.g. all the pixel flags? The definition of "primary" is a little funny to me since it really means "is a real source which has no child sources," so please take every opportunity in the code to remind us that is what is going on.
Hide
Lee Kelvin added a comment -

Thanks Meredith, these look great. To answer your high-level question, these flags are output in the main single-frame catalog (i.e., butler.get('src', ...)). The output column in the table schema looks like this: (Field['Flag'](name="detect_isPrimary", doc="true if source has no children and is not a sky source"), Key['Flag'](offset=624, bit=47)).

I'll start taking a look at your comments on GitHub just now and respond further there. Wherever possible, I'll try to remind the reader that this new detect_isPrimary flag returns True for all sources which are not sky sources and which have no children. This flag is meant to allow the end-user to easily select a useful list of science sources, and so hopefully should come in handy.

Show
Lee Kelvin added a comment - Thanks Meredith, these look great. To answer your high-level question, these flags are output in the main single-frame catalog (i.e., butler.get('src', ...)). The output column in the table schema looks like this: (Field ['Flag'] (name="detect_isPrimary", doc="true if source has no children and is not a sky source"), Key ['Flag'] (offset=624, bit=47)) . I'll start taking a look at your comments on GitHub just now and respond further there. Wherever possible, I'll try to remind the reader that this new detect_isPrimary flag returns True for all sources which are not sky sources and which have no children. This flag is meant to allow the end-user to easily select a useful list of science sources, and so hopefully should come in handy.
Hide
Lee Kelvin added a comment - - edited

Thanks again for all the comments, much appreciated. I've updated the PR with the changes suggested. Does that look okay? The branch has been rebased to master, and I've ran another Jenkins as a safety check.

Show
Lee Kelvin added a comment - - edited Thanks again for all the comments, much appreciated. I've updated the PR with the changes suggested. Does that look okay? The branch has been rebased to master, and I've ran another Jenkins as a safety check.
Hide
Meredith Rawls added a comment -

Looks great, thanks for addressing my comments. Good to merge assuming the rebasing and Jenkins are happy.

Show
Meredith Rawls added a comment - Looks great, thanks for addressing my comments. Good to merge assuming the rebasing and Jenkins are happy.
Hide
Lee Kelvin added a comment -

Thanks! Merged to master and removed the branch.

Show
Lee Kelvin added a comment - Thanks! Merged to master and removed the branch.
Hide
Yusra AlSayyad added a comment -

Lee Kelvin This new column never made it into the parquet Source Table: https://github.com/lsst/obs_subaru/blob/master/policy/Source.yaml
Please open a new ticket to add.

Show
Yusra AlSayyad added a comment - Lee Kelvin This new column never made it into the parquet Source Table: https://github.com/lsst/obs_subaru/blob/master/policy/Source.yaml Please open a new ticket to add.
Hide
Lee Kelvin added a comment -

Thanks Yusra AlSayyad, ticket DM-26438 has been set up to address this.

Show
Lee Kelvin added a comment - Thanks Yusra AlSayyad , ticket DM-26438 has been set up to address this.

#### People

Assignee:
Lee Kelvin
Reporter:
Lauren MacArthur
Reviewers:
Meredith Rawls
Watchers:
Jim Bosch, Lauren MacArthur, Lee Kelvin, Meredith Rawls, Yusra AlSayyad
Votes:
0 Vote for this issue
Watchers:
5 Start watching this issue

#### Dates

Created:
Updated:
Resolved: