Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14874

Add options to select against duplicate coadd sources

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_analysis
    • Labels:
      None
    • Story Points:
      5
    • Epic Link:
    • Sprint:
      DRP F18-1, DRP F18-2
    • Team:
      Data Release Production

      Description

      We should only ever be plotting the "primary" sources for coadd QA-ing.  The description for the detect_isPrimary flag is:

      doc="true if source has no children and is in the inner region of a coadd patch and is in the inner region of a coadd tract"

      We are currently selecting on deblend_nChild = 0 (which is appropriate at the visit level).

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment -

            In working on this, I noticed some cleanup that needs to be done in the use of forced vs. unforced catalogs for various plot (revealed when I became aware that the detect_isPrimary flag does not get persisted in the forced catalogs...neither do the calib_psfUsed & calib_psfCandidate flags).

            Increasing the scope to make accommodations and include these cleanups.

            Show
            lauren Lauren MacArthur added a comment - In working on this, I noticed some cleanup that needs to be done in the use of forced vs. unforced catalogs for various plot (revealed when I became aware that the detect_isPrimary flag does not get persisted in the forced catalogs...neither do the calib_psfUsed  &  calib_psfCandidate flags). Increasing the scope to make accommodations and include these cleanups.
            Hide
            lauren Lauren MacArthur added a comment -

            The detect_isPrimary flag is actually too strict for our per-tract analysis, so I have accommodated just selecting on detect_isPatchInner and detect_isTractInner separately (in addition to deblend_nChild = 0 and merge_peak_sky = False).

            Show
            lauren Lauren MacArthur added a comment - The detect_isPrimary flag is actually too strict for our per-tract analysis, so I have accommodated just selecting on  detect_isPatchInner and  detect_isTractInner separately (in addition to deblend_nChild = 0 and merge_peak_sky = False ).
            Hide
            lauren Lauren MacArthur added a comment -

            Seeing as you were also recently reminded of the importance of these flags, would you mind giving this a look.  I have run a few examples for coaddAnalysis and colorAnalysis based on Hsin-Fang Chiang's w_2018_22 RC2 reprocessing, so the two sets of plots can be directly compared.  As expected, the total number of sources has decreased in the rerun based on this branch, but the statistics and color locus fits are largely unchanged.  Links to mine and Hsin-Fang Chiang's plots are, e.g.:

            Coadds:

            https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-14874/w_2018_22/WIDE_GAMA15H_9615/plots/HSC-I/tract-9615/

            https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_22/pipe_analysis/qaPlots/HSC-I/tract-9615/

             

            Colors:

            https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-14874/w_2018_22/plots/color/tract-9615/

            https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_22/pipe_analysis/qaPlots/color/tract-9615/

             

            Show
            lauren Lauren MacArthur added a comment - Seeing as you were also recently reminded of the importance of these flags, would you mind giving this a look.  I have run a few examples for coaddAnalysis and colorAnalysis based on Hsin-Fang Chiang 's w_2018_22 RC2 reprocessing, so the two sets of plots can be directly compared.  As expected, the total number of sources has decreased in the rerun based on this branch, but the statistics and color locus fits are largely unchanged.  Links to mine and Hsin-Fang Chiang 's plots are, e.g.: Coadds: https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-14874/w_2018_22/WIDE_GAMA15H_9615/plots/HSC-I/tract-9615/ https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_22/pipe_analysis/qaPlots/HSC-I/tract-9615/   Colors: https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-14874/w_2018_22/plots/color/tract-9615/ https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_22/pipe_analysis/qaPlots/color/tract-9615/  
            Hide
            lauren Lauren MacArthur added a comment -

            Note that I changed the title of this ticket so as not to be misleading.

            Show
            lauren Lauren MacArthur added a comment - Note that I changed the title of this ticket so as not to be misleading.
            Hide
            tmorton Tim Morton added a comment -

            Looks good to me! (modulo those accidental extra characters at the top of that script file)

            Show
            tmorton Tim Morton added a comment - Looks good to me! (modulo those accidental extra characters at the top of that script file)
            Hide
            lauren Lauren MacArthur added a comment -

            Thanks, Tim!  Superfluous characters removed and merged to master.

            Show
            lauren Lauren MacArthur added a comment - Thanks, Tim!  Superfluous characters removed and merged to master.

              People

              • Assignee:
                lauren Lauren MacArthur
                Reporter:
                lauren Lauren MacArthur
                Reviewers:
                Tim Morton
                Watchers:
                John Swinbank, Lauren MacArthur, Tim Morton, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel