Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_tasks
-
Labels:
-
Story Points:9
-
Epic Link:
-
Sprint:DRP F20-1 (June), DRP F20-2 (July)
-
Team:Data Release Production
-
Urgent?:No
Description
The coadd level catalogs include a flag:
"detect_isPrimary", type="Flag", |
doc="true if source has no children and is in the inner region of a coadd patch " |
"and is in the inner region of a coadd tract " |
"and is not \"detected\" in a pseudo-filter (see config.pseudoFilterList)", |
that is very useful for culling catalogs of duplicate (patch/tract overlap), non-deblended parent sources (deblend_nChild>0, and sky objects. No equivalent flag currently exists at the visit /single frame measurement level, so this ticket is to add such a flag. Currently, the only two conditions to be added to flag undesirable sources are:
deblend_nChild == 0
sky_source is False
A community post describing this feature may be found here: https://community.lsst.org/t/primary-source-flag-for-single-frame-processing/4425
Attachments
Issue Links
- relates to
-
- Done
Activity
| Field | Original Value | New Value |
|---|---|---|
| Epic Link |
|
| Description |
The coadd level catalogs include a flag:
{code:python} "detect_isPrimary", type="Flag", doc="true if source has no children and is in the inner region of a coadd patch " "and is in the inner region of a coadd tract " "and is not \"detected\" in a pseudo-filter (see config.pseudoFilterList)", {code} that is very useful for culling catalogs of duplicate (patch/tract overlap), non-deblended parent sources ({{detect_nChild>0}}, and sky objects. No equivalent flag currently exists at the visit /single frame measurement level, so this ticket is to add such a flag. Currently, the only two conditions to be added to flag undesirable sources are: {{detect_nChild > 0}} {{sky_source is True}} |
The coadd level catalogs include a flag:
{code:python}"detect_isPrimary", type="Flag", doc="true if source has no children and is in the inner region of a coadd patch " "and is in the inner region of a coadd tract " "and is not \"detected\" in a pseudo-filter (see config.pseudoFilterList)", {code} that is very useful for culling catalogs of duplicate (patch/tract overlap), non-deblended parent sources ({{deblend_nChild>0}}, and sky objects. No equivalent flag currently exists at the visit /single frame measurement level, so this ticket is to add such a flag. Currently, the only two conditions to be added to flag undesirable sources are: {{deblend_nChild > 0}} {{sky_source is True}} |
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
| Description |
The coadd level catalogs include a flag:
{code:python}"detect_isPrimary", type="Flag", doc="true if source has no children and is in the inner region of a coadd patch " "and is in the inner region of a coadd tract " "and is not \"detected\" in a pseudo-filter (see config.pseudoFilterList)", {code} that is very useful for culling catalogs of duplicate (patch/tract overlap), non-deblended parent sources ({{deblend_nChild>0}}, and sky objects. No equivalent flag currently exists at the visit /single frame measurement level, so this ticket is to add such a flag. Currently, the only two conditions to be added to flag undesirable sources are: {{deblend_nChild > 0}} {{sky_source is True}} |
The coadd level catalogs include a flag:
{code:python}"detect_isPrimary", type="Flag", doc="true if source has no children and is in the inner region of a coadd patch " "and is in the inner region of a coadd tract " "and is not \"detected\" in a pseudo-filter (see config.pseudoFilterList)", {code} that is very useful for culling catalogs of duplicate (patch/tract overlap), non-deblended parent sources ({{deblend_nChild>0}}, and sky objects. No equivalent flag currently exists at the visit /single frame measurement level, so this ticket is to add such a flag. Currently, the only two conditions to be added to flag undesirable sources are: {{deblend_nChild == 0}} {{sky_source is False}} |
| Sprint | DRP F20-1 (June) [ 1030 ] | DRP F20-1 (June), DRP F20-2 (July) [ 1030, 1036 ] |
| Reviewers | Meredith Rawls [ mrawls ] | |
| Status | In Progress [ 3 ] | In Review [ 10004 ] |
This adds a useful functionality, thanks for implementing it. Please take a look at my GitHub comments. The high-level question I have is, where do these flags ultimately land? In the source catalog as a new column alongside e.g. all the pixel flags? The definition of "primary" is a little funny to me since it really means "is a real source which has no child sources," so please take every opportunity in the code to remind us that is what is going on.
Thanks Meredith, these look great. To answer your high-level question, these flags are output in the main single-frame catalog (i.e., butler.get('src', ...)). The output column in the table schema looks like this: (Field['Flag'](name="detect_isPrimary", doc="true if source has no children and is not a sky source"), Key['Flag'](offset=624, bit=47)).
I'll start taking a look at your comments on GitHub just now and respond further there. Wherever possible, I'll try to remind the reader that this new detect_isPrimary flag returns True for all sources which are not sky sources and which have no children. This flag is meant to allow the end-user to easily select a useful list of science sources, and so hopefully should come in handy.
Thanks again for all the comments, much appreciated. I've updated the PR with the changes suggested. Does that look okay? The branch has been rebased to master, and I've ran another Jenkins as a safety check.
Looks great, thanks for addressing my comments. Good to merge assuming the rebasing and Jenkins are happy.
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
Thanks! Merged to master and removed the branch.
| Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
| Story Points | 12 |
Lee Kelvin This new column never made it into the parquet Source Table: https://github.com/lsst/obs_subaru/blob/master/policy/Source.yaml
Please open a new ticket to add.
Thanks Yusra AlSayyad, ticket DM-26438 has been set up to address this.
| Story Points | 12 | 9 |
| Description |
The coadd level catalogs include a flag:
{code:python}"detect_isPrimary", type="Flag", doc="true if source has no children and is in the inner region of a coadd patch " "and is in the inner region of a coadd tract " "and is not \"detected\" in a pseudo-filter (see config.pseudoFilterList)", {code} that is very useful for culling catalogs of duplicate (patch/tract overlap), non-deblended parent sources ({{deblend_nChild>0}}, and sky objects. No equivalent flag currently exists at the visit /single frame measurement level, so this ticket is to add such a flag. Currently, the only two conditions to be added to flag undesirable sources are: {{deblend_nChild == 0}} {{sky_source is False}} |
The coadd level catalogs include a flag:
{code:python}"detect_isPrimary", type="Flag", doc="true if source has no children and is in the inner region of a coadd patch " "and is in the inner region of a coadd tract " "and is not \"detected\" in a pseudo-filter (see config.pseudoFilterList)", {code} that is very useful for culling catalogs of duplicate (patch/tract overlap), non-deblended parent sources ({{deblend_nChild>0}}, and sky objects. No equivalent flag currently exists at the visit /single frame measurement level, so this ticket is to add such a flag. Currently, the only two conditions to be added to flag undesirable sources are: {{deblend_nChild == 0}} {{sky_source is False}} A community post describing this feature may be found here: https://community.lsst.org/t/primary-source-flag-for-single-frame-processing/4425 |
Thanks for taking a look at this ticket Meredith, much appreciated. We pushed it through Jenkins yesterday without issue, so hopefully it should be good to go.