Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4821

HSC backport: Remove interpolated background before detection to reduce junk sources

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_algorithms
    • Labels:
      None
    • Story Points:
      1
    • Sprint:
      Science Pipelines DM-W16-5
    • Team:
      Data Release Production

      Description

      This is a port of HSC-1353 and HSC-1360.

      Descriptions from HSC:

      HSC-1353

      We typically get a large number of junk detections around bright objects due to noise fluctuations in the elevated background. We can try to reduce the number of junk detections by adding an additional local background subtraction before object detection. We can then add this back in after detection of footprints and peaks.

      HSC-1360

      I forgot to set the useApprox=True for the background subtraction that runs before footprint and peak detection. This will then use the Chebyshev instead of the spline.

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment - - edited

            Bob Armstrong would you mind having a look at this. I've attached two figures demonstrating the difference between running with this feature off (red crosses) and on (green circles). FYI: This was for processCcd.py runs of visit=1228 ccd=71 of the UDEEP COSMOS field.

            The decision of whether to turn it on by default (or in config overrides in camera packages) will be left until the ongoing testing is completed on the HSC side of things.

            A Jenkins build was successful.

            Show
            lauren Lauren MacArthur added a comment - - edited Bob Armstrong would you mind having a look at this. I've attached two figures demonstrating the difference between running with this feature off (red crosses) and on (green circles). FYI: This was for processCcd.py runs of visit=1228 ccd=71 of the UDEEP COSMOS field. The decision of whether to turn it on by default (or in config overrides in camera packages) will be left until the ongoing testing is completed on the HSC side of things. A Jenkins build was successful.
            Hide
            rearmstr Bob Armstrong added a comment -

            Your plots seem to indicate that turning it on makes things worse in that there are more junk detections. You ran your test on an individual frame, but I only looked at things on the coadds. This probably makes sense in that you would expect more junk for higher S/N. We probably need to test this more before we decide how/where to turn this on.

            Show
            rearmstr Bob Armstrong added a comment - Your plots seem to indicate that turning it on makes things worse in that there are more junk detections. You ran your test on an individual frame, but I only looked at things on the coadds. This probably makes sense in that you would expect more junk for higher S/N. We probably need to test this more before we decide how/where to turn this on.
            Hide
            lauren Lauren MacArthur added a comment -

            I totally agree. I was pretty sure you hadn't tried running this on single frames. It looks to me like there are trade offs to both modes, so further testing is certainly needed before (if/where) turning this feature on.

            Show
            lauren Lauren MacArthur added a comment - I totally agree. I was pretty sure you hadn't tried running this on single frames. It looks to me like there are trade offs to both modes, so further testing is certainly needed before (if/where) turning this feature on.
            Hide
            lauren Lauren MacArthur added a comment -

            I made the footprintBackground --> tempLocalBackground name change, pushed a tickets/DM-4821 branch, reran Jenkins (successful) and pushed to master.

            Show
            lauren Lauren MacArthur added a comment - I made the footprintBackground --> tempLocalBackground name change, pushed a tickets/ DM-4821 branch, reran Jenkins (successful) and pushed to master.
            Hide
            swinbank John Swinbank added a comment -

            And release notes have been updated: thanks!

            Show
            swinbank John Swinbank added a comment - And release notes have been updated: thanks!

              People

              • Assignee:
                lauren Lauren MacArthur
                Reporter:
                lauren Lauren MacArthur
                Reviewers:
                Bob Armstrong
                Watchers:
                Bob Armstrong, John Swinbank, Lauren MacArthur
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel