The DC2 survey paper (Abolfathi et al. 2021) turns on the sky correction task by default (with configs originally copied from obs_subaru). However, obs_lsst does not turn on sky correction (i.e., doApplySkyCorr=False). There are two key problems with skyCorr: first, testing has struggled to reproduce the DC2 survey paper results (see DM-29715 ); and second, skyCorrectionTask is extremely memory intensive.

This ticket aims to investigate whether an alternative solution to applying sky correction task might be to instead reduce the order of the Chebyshev polynomial used to fit the background in visit-level data. As DC2 backgrounds are simple to begin with, this is a valid approach. From the DC2 survey paper (Section 6.1.2, page 22):

Other current approximations, to which we found that our results are insensitive, include ... holding the sky brightness constant across each sensor (intra-CCD sky variations were found to be less than the Poisson noise of the sky background).

In this ticket, we experiment in reducing DC2 data using Chebyshev polynomials of order 0, 1 and 2 to fit the background, and compare these outputs to the defaults currently being used (i.e., a Chebyshev polynomial of order 6). Visit lists for visits overlapping test_med_1 tract 3828, patch 24 (i.e., approximately central) are generated. The collection 2.2i/runs/test-med-1/w_2021_24/DM-30674 is used to find visits overlapping this tract/patch. In addition, three visits from tract 3263, patch 21 are also processed, to allow for a direct comparison to the dataset shown in the DC2 survey paper. The collection 2.2i/defaults/DP0 is used as the input collection for rerun data processing.

The full list of visits processed as part of this ticket are:

3828/24: g [159471, 159491, 183772]

3828/24: r [162699, 193147, 193189]

3828/24: i [174534, 192355, 204706]

3828/24: z [7997, 7998, 8003]

3828/24: y [5884, 12466, 37657]

3263/21: r [40318, 40351, 185822]

The base $OBS_LSST_DIR/pipelines/imsim/DRP.yaml pipeline is used to process data. Starting with 2.2i/defaults/DP0 as the input collection, the above visits are processed through single frame processing and multi visit.

singleFrame:

multiVisit:

The config option approxOrderX sets the approximation order in X (and Y) for background Chebyshev fitting. At the time of writing, this config option appears in the singleFrame/multiVisit tasks listed below. Here I'm setting approxOrderX=1, reducing the order of the Chebyshev polynomial from it's default value of 6. Other orders tested in addition to 1 as part of this ticket are 0 and 2. Config options which are commented out are those where useApprox=False (approxOrderX is valid only with useApprox=True, but all config options are listed here, for reference).

singleFrame:

* characterizeImage

config.background.approxOrderX=1

config.detection.background.approxOrderX=1

# config.detection.tempLocalBackground.approxOrderX=1

# config.detection.tempWideBackground.approxOrderX=1

# config.repair.cosmicray.background.approxOrderX=1

* calibrate

config.detection.background.approxOrderX=1

# config.detection.tempLocalBackground.approxOrderX=1

# config.detection.tempWideBackground.approxOrderX=1

multiVisit:

* makeWarp

config.warpAndPsfMatch.psfMatch.kernel['AL'].afwBackgroundConfig.approxOrderX=1

* assembleCoadd

config.detect.background.approxOrderX=1

# config.detect.tempLocalBackground.approxOrderX=1

# config.detect.tempWideBackground.approxOrderX=1

config.detectTemplate.background.approxOrderX=1

# config.detectTemplate.tempLocalBackground.approxOrderX=1

# config.detectTemplate.tempWideBackground.approxOrderX=1

# config.scaleWarpVariance.background.approxOrderX=1

* detection

# config.scaleVariance.background.approxOrderX=1

# config.detection.background.approxOrderX=1

# config.detection.tempLocalBackground.approxOrderX=1

# config.detection.tempWideBackground.approxOrderX=1

Additionally, to facilitate efforts led by Morgan Schmitz looking into bright star diagnostics, the following config options were adopted on this ticket too:

* measure

config.measurement.plugins['base_CircularApertureFlux'].radii=[3, 4.5, 6.0, 8, 9, 12, 13, 17, 18, 23, 25, 28, 33, 35, 38, 43, 48, 50, 53, 58, 63, 68, 70, 73, 78, 83, 88, 93, 98, 103, 108, 113, 118, 123, 128, 133, 138, 143, 148, 153, 158, 163, 168, 173, 178, 183, 188, 193, 198, 203, 208, 213, 218, 223, 228, 233, 238, 243, 248]

config.measurement.doReplaceWithNoise=False

Note that the above measurement config options should not impact the primary conclusions of this ticket.

Results for these tests are explored in the comments below. Postage stamp outputs at the visit level and coadd level are shown, and plots generated showing the differences in output photometry relative to the current defaults.

In summary, the final conclusion of this ticket is that the Chebyshev polynomial order should be reduced from its current default of 6 to the lower value of 1. Not only does this resolve the well-known sky oversubtraction issue around bright sources (the 'divot' problem), but is also physically motivated due to the simple nature of DC2 backgrounds established during DC2 image construction. A further reduction to order 0 produces undesirable effects at the coadd-level, namely, a visible imprint of the visit level detectors remain visible. It appears that not allowing the background fit to fit at least a sky gradient leads to these CCD edge effects remaining visible. Conversely, an increased background fitting order of 2 leads to residual remnant sky oversubtraction (divots) surrounding bright sources still existing in the imaging. Therefore, a reduced order of 1 appears to be the happy-medium with which to balance out all negative effects.