Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-38246

Exclude edge pixels from source detection

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_diffim
    • Labels:
    • Story Points:
      4
    • Sprint:
      AP S23-4 (March), AP S23-5 (April), AP S23-6 (May)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Investigate the flagging of edge pixels when the science image is convolved during image differencing. In DM-37698 we saw a large excess of false sources around the edges of every ccd when using the new preconvolution option, which suggests that either the mask plane is not being set correctly, or the EDGE bit is not being used during detection.

        Attachments

          Issue Links

            Activity

            Hide
            sullivan Ian Sullivan added a comment -

            The main change on this ticket is the addition in meas_algorithms SourceDetectionTask that prevents detection on pixels with NO_DATA, EDGE, or BAD flags set. The changes in ip_diffim mostly just fix and expand the unit tests following that change. On a typical DECam visit, this reduces the number of diaSources by 30-50%, which substantially speeds up measurement. We should plan to later implement a fix that allows us to recover these edge sources without generating so many spurious detections, but I believe that is a low priority for now. As part of your review, please include your thoughts on that priority.

            Show
            sullivan Ian Sullivan added a comment - The main change on this ticket is the addition in meas_algorithms SourceDetectionTask that prevents detection on pixels with NO_DATA , EDGE , or BAD flags set. The changes in ip_diffim mostly just fix and expand the unit tests following that change. On a typical DECam visit, this reduces the number of diaSources by 30-50%, which substantially speeds up measurement. We should plan to later implement a fix that allows us to recover these edge sources without generating so many spurious detections, but I believe that is a low priority for now. As part of your review, please include your thoughts on that priority.
            Hide
            jbosch Jim Bosch added a comment -

            Looks good, just some very minor comments on the PRs.

            Show
            jbosch Jim Bosch added a comment - Looks good, just some very minor comments on the PRs.
            Hide
            sullivan Ian Sullivan added a comment -

            Jim Bosch Note that this will change the behavior for detection in DRP as well as for AP. I believe this change will be an improvement for both by reducing the number of junk sources, but I wanted to make sure that you were aware it would change the behavior of SourceDetectionTask in general. John pointed out that we could set the (newly-renamed) excludeMaskPlanes parameter just in DetectAndMeasureTask, and leave it as a default empty list in other cases to preserve current behavior.

            Show
            sullivan Ian Sullivan added a comment - Jim Bosch Note that this will change the behavior for detection in DRP as well as for AP. I believe this change will be an improvement for both by reducing the number of junk sources, but I wanted to make sure that you were aware it would change the behavior of SourceDetectionTask in general. John pointed out that we could set the (newly-renamed) excludeMaskPlanes parameter just in DetectAndMeasureTask , and leave it as a default empty list in other cases to preserve current behavior.
            Hide
            jbosch Jim Bosch added a comment -

            Thanks for the heads-up. I'm pretty confident that rejecting NO_DATA in all processing is a good idea, and I think the same is true of EDGE with just a bit less confidence. But I think BAD might be one of those planes that's propagated to coadds a little excessively, i.e. there might be a lot of pixels that were BAD in only one input. I do think it'd be prudent to either leave that out or check some moderately recent coadds for BAD to see what the mask plane looks like before including it in coadd-processing runs of detection. I am for including BAD in the list for all single-epoch DRP processing, because I think that'll help a lot with bad-amplifier areas, but I'm a bit worried (and this is a concern for AP as well) about small defects that were already interpolated pretty well bisecting sources, and creating two peaks where there was only one before. I still the avoiding the bad amps makes that a good trade, at least for HSC where the bad amps are pretty serious, but it might be worth a follow-up ticket.

            Show
            jbosch Jim Bosch added a comment - Thanks for the heads-up. I'm pretty confident that rejecting NO_DATA in all processing is a good idea, and I think the same is true of EDGE with just a bit less confidence. But I think BAD might be one of those planes that's propagated to coadds a little excessively, i.e. there might be a lot of pixels that were BAD in only one input. I do think it'd be prudent to either leave that out or check some moderately recent coadds for BAD to see what the mask plane looks like before including it in coadd-processing runs of detection. I am for including BAD in the list for all single-epoch DRP processing, because I think that'll help a lot with bad-amplifier areas, but I'm a bit worried (and this is a concern for AP as well) about small defects that were already interpolated pretty well bisecting sources, and creating two peaks where there was only one before. I still the avoiding the bad amps makes that a good trade, at least for HSC where the bad amps are pretty serious, but it might be worth a follow-up ticket.

              People

              Assignee:
              sullivan Ian Sullivan
              Reporter:
              sullivan Ian Sullivan
              Reviewers:
              Jim Bosch
              Watchers:
              Bruno Sanchez, Erin Howard, Ian Sullivan, Jim Bosch, John Parejko
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.