Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9953

Pixels rejected from coaddition and CCD are not masked on coadds

    Details

      Description

      Rejecting an input pixel during coaddition does not result in the CLIPPED bit being set, even though can yield bad CoaddPsf.

      We similarly do not flag objects that lie on the edges of CCDs in the coadd. This could probably be fixed by propagating the EDGE flag through to the coadds.

        Attachments

          Issue Links

            Activity

            Hide
            yusra Yusra AlSayyad added a comment -

            CLIPPED bit is being manually set in SafeClip and now CompareWarp. Remove these blocks when DM-9953 is complete:

            https://github.com/lsst/pipe_tasks/commit/a1b2df812ae7b91836085ebea70598a41e9ca156

            Show
            yusra Yusra AlSayyad added a comment - CLIPPED bit is being manually set in SafeClip and now CompareWarp. Remove these blocks when DM-9953 is complete: https://github.com/lsst/pipe_tasks/commit/a1b2df812ae7b91836085ebea70598a41e9ca156
            Hide
            price Paul Price added a comment -

            I'm adding some statistics (NCLIPPED, NMASKED), which Pim Schellart [X] may be interested in incorporating into his work.

            Show
            price Paul Price added a comment - I'm adding some statistics ( NCLIPPED , NMASKED ), which Pim Schellart [X] may be interested in incorporating into his work.
            Hide
            price Paul Price added a comment -

            Thanks for volunteering to look at this, Jim Bosch. There are commits in afw, obs_subaru and pipe_tasks. This includes stripping out background matching from assembleCoadd (because it was easy to do and has long been hanging over my head and it makes it easier to follow the code). I'm about to run this through Jenkins, but I've verified that a ci_hsc coadd's mask is what I expect.

            Show
            price Paul Price added a comment - Thanks for volunteering to look at this, Jim Bosch . There are commits in afw, obs_subaru and pipe_tasks. This includes stripping out background matching from assembleCoadd (because it was easy to do and has long been hanging over my head and it makes it easier to follow the code). I'm about to run this through Jenkins, but I've verified that a ci_hsc coadd's mask is what I expect.
            Hide
            price Paul Price added a comment -

            Here's the i-band coadd from ci_hsc, with the CLIPPED pixels in white. The edges of the CCDs are showing up, along with pixels rejected in the coadd.

            Show
            price Paul Price added a comment - Here's the i-band coadd from ci_hsc, with the CLIPPED pixels in white. The edges of the CCDs are showing up, along with pixels rejected in the coadd.
            Hide
            jbosch Jim Bosch added a comment -

            Looks good. One minor comment on how you may be able to slightly improve some test code.

            But while this is all functional, I wonder if it's an abuse of CLIPPED for it to be used for EDGE pixels (and, to lesser extent, mask-rejected pixels). Since we can now add new mask planes without much cost, it might be a good idea to do so; something like:

            • CLIPPED: one or more pixels going into the coadd was rejected by safe-clip or WarpCompare.
            • REJECTED: one or more pixels going into the coadd was rejected by mask values set in processCcd.
            • EDGE: one or more pixels going into the coadd was on the edge of a CCD
            • INEXACT_PSF: CLIPPED | REJECTED | EDGE
            Show
            jbosch Jim Bosch added a comment - Looks good. One minor comment on how you may be able to slightly improve some test code. But while this is all functional, I wonder if it's an abuse of CLIPPED for it to be used for EDGE pixels (and, to lesser extent, mask-rejected pixels). Since we can now add new mask planes without much cost, it might be a good idea to do so; something like: CLIPPED: one or more pixels going into the coadd was rejected by safe-clip or WarpCompare. REJECTED: one or more pixels going into the coadd was rejected by mask values set in processCcd. EDGE: one or more pixels going into the coadd was on the edge of a CCD INEXACT_PSF: CLIPPED | REJECTED | EDGE
            Hide
            price Paul Price added a comment -

            One additional change in obs_sdss.

            Show
            price Paul Price added a comment - One additional change in obs_sdss.
            Hide
            price Paul Price added a comment -

            The modified version is starting to go green on Jenkins.

            Show
            price Paul Price added a comment - The modified version is starting to go green on Jenkins .
            Hide
            price Paul Price added a comment -

            Merged to master.

            I should explain the changes made after offline consultation with Jim:

            • Pixels that are masked EDGE on the individual CCDs get masked SENSOR_EDGE on the coadd.
            • Pixels that are clipped/rejected (there's no point distinguishing between the two because we don't do any rejection so as to avoid messing up the PSF) are masked CLIPPED on the coadd.
            • Any pixel masked either SENSOR_EDGE or CLIPPED are also masked INEXACT_PSF.
            • Added each of these to the PixelFlags measurement in both measureCoaddSources and forcedPhotCoadd.
            Show
            price Paul Price added a comment - Merged to master. I should explain the changes made after offline consultation with Jim: Pixels that are masked EDGE on the individual CCDs get masked SENSOR_EDGE on the coadd. Pixels that are clipped/rejected (there's no point distinguishing between the two because we don't do any rejection so as to avoid messing up the PSF) are masked CLIPPED on the coadd. Any pixel masked either SENSOR_EDGE or CLIPPED are also masked INEXACT_PSF . Added each of these to the PixelFlags measurement in both measureCoaddSources and forcedPhotCoadd.

              People

              • Assignee:
                price Paul Price
                Reporter:
                jbosch Jim Bosch
                Reviewers:
                Jim Bosch
                Watchers:
                Jim Bosch, Paul Price, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel