+1. This sounds helpful, and if some camera team prefers to run without it is easy enough to disable using an override in the appropriate obs_ package.

I think it should be turned on at the coadd stage. Lauren looked at this for single visits and there was not a firm conclusion on whether it was an improvement.

Indeed, and a few plots & discussion can be found on DM-4821.

Let me try to make a case for turning it on globally.

I worry about having different configurations depending on the depth of the image (depth seems to me to be the major distinguishing feature between single visits and coadds in this case) because it means we have to adjust the pipeline according to how long your exposures are, and single visits can have quite a range of exposure times (variations of a factor of 10 to 100) and when you add in seeing you get even more variation in depth so it's not clear why a coadd should have it turned on but a single visit should not.

If it works on coadds, which are the more difficult case, it should work on the easier single visits. I think we want to suppress junk around bright sources on the single visits as much as we want to suppress it on the coadds. In fact, when running it on a test CCD, I observe that it suppresses junk and recovers true objects around bright stars (the local background subtraction suppresses the wings of the bright star, which reduces the star's footprint size which allows the deblender to function, and we can deblend out real sources in the wings).

Now, I have noticed that with this feature turned on we detect and deblend more sources as part of satellite trails, but I think that's a separate problem which should be dealt with by running Steve's satellite trail detection code.

Here are some demonstrations. Circles are detections from running without the junk suppression, while crosses are detections from running with junk suppression. The mask plane is from running without the junk suppression (green is SAT, red is CROSSTALK, blue is DETECTED and aqua is NOT_DEBLENDED).

The surrounds of a bright star. Junk sources (one at 10 o'clock and one at 11:30) are suppressed, and real sources in the vicinity of the bright star are recovered:

A couple of bright stars and a satellite trail. Again, junk sources (from 3 o'clock to 5 o'clock on the brighter star) are suppressed, but we get detections in the satellite trail. These are arguably true peaks in the satellite trail that are now being detected and deblended because they are separated from the bright star by using a local background subtraction.

Lauren and I took a look at her examples and they don't make sense. I think we should revisit the single-frame data before using it as an argument pro or con this RFC.

Robert Lupton, when are you and Lauren MacArthur going to look at that?

That's a question for John Swinbank

I don't have anything to add in regards to whether junk suppression improves the quality of single-frame processing, but I wanted to chime in and say that I agree with Paul Price's statement that we should enable it there if it just does no harm, on the principle that it's desirable to process different depths with the same algorithms whenever possible.

I'm not totally convinced of the "does no harm" assertion. I think the figures I show on DM-4821 are slightly messier cases than those shown by Paul here in that they demonstrate that there can also be many junk sources added (while many are indeed eliminated) when this feature is activated. Even the figures here show extra detections within the bright/saturated stars. Won't these extra junk detections cause the same issues we are trying to avoid? I also see extra junk detections in the diffraction spikes of bright stars, but perhaps these would also be handled by Steve Bickerton's satellite detection code?

Here's the bright star from the same CCD Lauren ran. The left side is processed without the local background subtraction, while the right side is processed with local background subtraction. Green circles are sources in the catalog without local background subtraction, while red crosses are sources in the catalog with local background subtraction.

The main difference in the images is in the masking: the bright star isn't deblended when we don't use a local background subtraction. This serves to suppress sources in the halo (i.e., there are peaks there, but they're not deblended from the parent so they don't end up in the catalog). I would suggest that this is not a sufficient difference to prevent using the local background subtraction because the difference is coming from a shortcoming in the deblender rather than something intrinsic to the local background subtraction. Since they are bona fide peaks, I see no problem with including them in the catalog; they should be flagged or suppressed by code that deliberately deals with haloes and diffraction spikes rather than by a mere accident.

I'm more concerned about the quality of the measurements between the two. Here are the aperture correction RMSes (a measure of photometric systematics) without local background subtraction:

processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_exp: RMS 0.016637 from 48

processCcd.charImage.measureApCorr: Aperture correction for base_PsfFlux: RMS 0.020972 from 50

processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel: RMS 0.016441 from 48

processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_initial: RMS 0.020378 from 50

processCcd.charImage.measureApCorr: Aperture correction for base_GaussianFlux: RMS 0.017768 from 49

processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_dev: RMS 0.016407 from 48

and with local background subtraction:

processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_exp: RMS 0.020698 from 55

processCcd.charImage.measureApCorr: Aperture correction for base_PsfFlux: RMS 0.024550 from 56

processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel: RMS 0.020635 from 55

processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_initial: RMS 0.019872 from 55

processCcd.charImage.measureApCorr: Aperture correction for base_GaussianFlux: RMS 0.019390 from 55

processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_dev: RMS 0.020450 from 55

As you can see, using the local background subtraction typically increases the RMS by an extra 4 mmag or so, though that may be due to using more stars (55 instead of 49). The PSF model with the local background subtraction used 56/62 stars instead of 50/55 without.

In our meeting last week (including Lauren MacArthur, Robert Lupton and Bob Armstrong), we decided that this should go forward.

Indeed, and I am implementing it as part of DM-6815.

Implemented by Lauren MacArthur in DM-6815.