Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14874

Add options to select against duplicate coadd sources

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_analysis
    • Labels:
      None
    • Story Points:
      5
    • Epic Link:
    • Sprint:
      DRP F18-1, DRP F18-2
    • Team:
      Data Release Production

      Description

      We should only ever be plotting the "primary" sources for coadd QA-ing.  The description for the detect_isPrimary flag is:

      doc="true if source has no children and is in the inner region of a coadd patch and is in the inner region of a coadd tract"

      We are currently selecting on deblend_nChild = 0 (which is appropriate at the visit level).

        Attachments

          Issue Links

            Activity

            lauren Lauren MacArthur created issue -
            lauren Lauren MacArthur made changes -
            Field Original Value New Value
            Epic Link DM-14402 [ 79665 ]
            lauren Lauren MacArthur made changes -
            Risk Score 0
            lauren Lauren MacArthur made changes -
            Story Points 0.25 1
            Hide
            lauren Lauren MacArthur added a comment -

            In working on this, I noticed some cleanup that needs to be done in the use of forced vs. unforced catalogs for various plot (revealed when I became aware that the detect_isPrimary flag does not get persisted in the forced catalogs...neither do the calib_psfUsed & calib_psfCandidate flags).

            Increasing the scope to make accommodations and include these cleanups.

            Show
            lauren Lauren MacArthur added a comment - In working on this, I noticed some cleanup that needs to be done in the use of forced vs. unforced catalogs for various plot (revealed when I became aware that the detect_isPrimary flag does not get persisted in the forced catalogs...neither do the calib_psfUsed  &  calib_psfCandidate flags). Increasing the scope to make accommodations and include these cleanups.
            lauren Lauren MacArthur made changes -
            Story Points 1 3
            lauren Lauren MacArthur made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            lauren Lauren MacArthur added a comment -

            The detect_isPrimary flag is actually too strict for our per-tract analysis, so I have accommodated just selecting on detect_isPatchInner and detect_isTractInner separately (in addition to deblend_nChild = 0 and merge_peak_sky = False).

            Show
            lauren Lauren MacArthur added a comment - The detect_isPrimary flag is actually too strict for our per-tract analysis, so I have accommodated just selecting on  detect_isPatchInner and  detect_isTractInner separately (in addition to deblend_nChild = 0 and merge_peak_sky = False ).
            lauren Lauren MacArthur made changes -
            Story Points 3 5
            Hide
            lauren Lauren MacArthur added a comment -

            Seeing as you were also recently reminded of the importance of these flags, would you mind giving this a look.  I have run a few examples for coaddAnalysis and colorAnalysis based on Hsin-Fang Chiang's w_2018_22 RC2 reprocessing, so the two sets of plots can be directly compared.  As expected, the total number of sources has decreased in the rerun based on this branch, but the statistics and color locus fits are largely unchanged.  Links to mine and Hsin-Fang Chiang's plots are, e.g.:

            Coadds:

            https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-14874/w_2018_22/WIDE_GAMA15H_9615/plots/HSC-I/tract-9615/

            https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_22/pipe_analysis/qaPlots/HSC-I/tract-9615/

             

            Colors:

            https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-14874/w_2018_22/plots/color/tract-9615/

            https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_22/pipe_analysis/qaPlots/color/tract-9615/

             

            Show
            lauren Lauren MacArthur added a comment - Seeing as you were also recently reminded of the importance of these flags, would you mind giving this a look.  I have run a few examples for coaddAnalysis and colorAnalysis based on Hsin-Fang Chiang 's w_2018_22 RC2 reprocessing, so the two sets of plots can be directly compared.  As expected, the total number of sources has decreased in the rerun based on this branch, but the statistics and color locus fits are largely unchanged.  Links to mine and Hsin-Fang Chiang 's plots are, e.g.: Coadds: https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-14874/w_2018_22/WIDE_GAMA15H_9615/plots/HSC-I/tract-9615/ https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_22/pipe_analysis/qaPlots/HSC-I/tract-9615/   Colors: https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-14874/w_2018_22/plots/color/tract-9615/ https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_22/pipe_analysis/qaPlots/color/tract-9615/  
            lauren Lauren MacArthur made changes -
            Reviewers Tim Morton [ tmorton ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            lauren Lauren MacArthur made changes -
            Summary Add detect_isPrimary = True to coadd object selection criteria Add options to select against duplicate coadd sources
            Hide
            lauren Lauren MacArthur added a comment -

            Note that I changed the title of this ticket so as not to be misleading.

            Show
            lauren Lauren MacArthur added a comment - Note that I changed the title of this ticket so as not to be misleading.
            yusra Yusra AlSayyad made changes -
            Sprint DRP F18-1 [ 742 ] DRP F18-1, DRP F18-2 [ 742, 756 ]
            Hide
            tmorton Tim Morton added a comment -

            Looks good to me! (modulo those accidental extra characters at the top of that script file)

            Show
            tmorton Tim Morton added a comment - Looks good to me! (modulo those accidental extra characters at the top of that script file)
            tmorton Tim Morton made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            lauren Lauren MacArthur added a comment -

            Thanks, Tim!  Superfluous characters removed and merged to master.

            Show
            lauren Lauren MacArthur added a comment - Thanks, Tim!  Superfluous characters removed and merged to master.
            lauren Lauren MacArthur made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            lauren Lauren MacArthur made changes -
            Link This issue relates to DM-15091 [ DM-15091 ]

              People

              • Assignee:
                lauren Lauren MacArthur
                Reporter:
                lauren Lauren MacArthur
                Reviewers:
                Tim Morton
                Watchers:
                John Swinbank, Lauren MacArthur, Tim Morton, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel