Details
-
Type:
RFC
-
Status: Implemented
-
Resolution: Done
-
Component/s: DM
-
Labels:None
Description
SourceDetectionConfig.doTempLocalBackground currently defaults to False, but we have reason to believe that this should be True by default:
- It suppresses junk in the wings of bright sources (e.g., compare without and with junk suppression).
- It makes the deblender much more robust: processing the HSC data of the VVDS and HectoMap fields, I was seeing about half out of a hundred or so tracts fail in the deblender (usually a single patch in the tract blowing out the memory usage); when I turned on the junk suppression and re-ran, I saw only one tract fail, and that appears to be a bona fide deblender bug.
- The only reason it's not activated by default is due to an oversight: the feature was introduced on the HSC side, where we intended to run with it on, but it was accidentally left disabled.
If you're not familiar with this parameter, see HSC-1353, where it was introduced (under a different name).
I propose to change the default behaviour to activate the junk suppression.
Attachments
Issue Links
Activity
Here's the bright star from the same CCD Lauren ran. The left side is processed without the local background subtraction, while the right side is processed with local background subtraction. Green circles are sources in the catalog without local background subtraction, while red crosses are sources in the catalog with local background subtraction.
The main difference in the images is in the masking: the bright star isn't deblended when we don't use a local background subtraction. This serves to suppress sources in the halo (i.e., there are peaks there, but they're not deblended from the parent so they don't end up in the catalog). I would suggest that this is not a sufficient difference to prevent using the local background subtraction because the difference is coming from a shortcoming in the deblender rather than something intrinsic to the local background subtraction. Since they are bona fide peaks, I see no problem with including them in the catalog; they should be flagged or suppressed by code that deliberately deals with haloes and diffraction spikes rather than by a mere accident.
I'm more concerned about the quality of the measurements between the two. Here are the aperture correction RMSes (a measure of photometric systematics) without local background subtraction:
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_exp: RMS 0.016637 from 48
|
processCcd.charImage.measureApCorr: Aperture correction for base_PsfFlux: RMS 0.020972 from 50
|
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel: RMS 0.016441 from 48
|
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_initial: RMS 0.020378 from 50
|
processCcd.charImage.measureApCorr: Aperture correction for base_GaussianFlux: RMS 0.017768 from 49
|
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_dev: RMS 0.016407 from 48
|
and with local background subtraction:
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_exp: RMS 0.020698 from 55
|
processCcd.charImage.measureApCorr: Aperture correction for base_PsfFlux: RMS 0.024550 from 56
|
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel: RMS 0.020635 from 55
|
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_initial: RMS 0.019872 from 55
|
processCcd.charImage.measureApCorr: Aperture correction for base_GaussianFlux: RMS 0.019390 from 55
|
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_dev: RMS 0.020450 from 55
|
As you can see, using the local background subtraction typically increases the RMS by an extra 4 mmag or so, though that may be due to using more stars (55 instead of 49). The PSF model with the local background subtraction used 56/62 stars instead of 50/55 without.
Paul Price
added a comment - - edited Here's the bright star from the same CCD Lauren ran. The left side is processed without the local background subtraction, while the right side is processed with local background subtraction. Green circles are sources in the catalog without local background subtraction, while red crosses are sources in the catalog with local background subtraction.
The main difference in the images is in the masking: the bright star isn't deblended when we don't use a local background subtraction. This serves to suppress sources in the halo (i.e., there are peaks there, but they're not deblended from the parent so they don't end up in the catalog). I would suggest that this is not a sufficient difference to prevent using the local background subtraction because the difference is coming from a shortcoming in the deblender rather than something intrinsic to the local background subtraction. Since they are bona fide peaks, I see no problem with including them in the catalog; they should be flagged or suppressed by code that deliberately deals with haloes and diffraction spikes rather than by a mere accident.
I'm more concerned about the quality of the measurements between the two. Here are the aperture correction RMSes (a measure of photometric systematics) without local background subtraction:
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_exp: RMS 0.016637 from 48
processCcd.charImage.measureApCorr: Aperture correction for base_PsfFlux: RMS 0.020972 from 50
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel: RMS 0.016441 from 48
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_initial: RMS 0.020378 from 50
processCcd.charImage.measureApCorr: Aperture correction for base_GaussianFlux: RMS 0.017768 from 49
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_dev: RMS 0.016407 from 48
and with local background subtraction:
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_exp: RMS 0.020698 from 55
processCcd.charImage.measureApCorr: Aperture correction for base_PsfFlux: RMS 0.024550 from 56
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel: RMS 0.020635 from 55
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_initial: RMS 0.019872 from 55
processCcd.charImage.measureApCorr: Aperture correction for base_GaussianFlux: RMS 0.019390 from 55
processCcd.charImage.measureApCorr: Aperture correction for modelfit_CModel_dev: RMS 0.020450 from 55
As you can see, using the local background subtraction typically increases the RMS by an extra 4 mmag or so, though that may be due to using more stars (55 instead of 49). The PSF model with the local background subtraction used 56/62 stars instead of 50/55 without. In our meeting last week (including Lauren MacArthur, Robert Lupton and Bob Armstrong), we decided that this should go forward.
Paul Price
added a comment - In our meeting last week (including Lauren MacArthur , Robert Lupton and Bob Armstrong ), we decided that this should go forward. Indeed, and I am implementing it as part of DM-6815.
Implemented by Lauren MacArthur in DM-6815.
Paul Price
added a comment - Implemented by Lauren MacArthur in DM-6815 . 
I'm not totally convinced of the "does no harm" assertion. I think the figures I show on
DM-4821are slightly messier cases than those shown by Paul here in that they demonstrate that there can also be many junk sources added (while many are indeed eliminated) when this feature is activated. Even the figures here show extra detections within the bright/saturated stars. Won't these extra junk detections cause the same issues we are trying to avoid? I also see extra junk detections in the diffraction spikes of bright stars, but perhaps these would also be handled by Steve Bickerton's satellite detection code?