Per our meeting of 2019-03-26, we agreed that it's appropriate for Gabor Kovacs to spent another couple of weeks of investigation on this ticket. At that point — if the work hasn't converged —, he should summarize his results here and solicit inputs from elsewhere in the project (likely primarily from Robert Lupton).

Gabor Kovacs — it has now been a lot longer since the “couple of weeks” we mentioned on 26 March, and I know you have taken a lot of action (including talking to Robert).

Could you please update this ticket with a note of what actions you've taken, what you've discovered, and what the plan is for making progress. Then I think we can close out this ticket as mission accomplished (certainly, these bad subtractions have been examined in detail!!).

Fingerprinting pattern empirical characteristics

Fingerprinting solutions occur on the DECam images when both the
science and template images are sharp, there is little PSF width
difference between the science and the template and the science image
have higher number of sources (several hundreds compared to ~100). As
these effects appear together, we did not separate which effect is
the dominant. This could be better understood if we could generate
images with simulated sources.

The fingerprinting solution is characterized by order of magnitude
higher coefficient values in the higher order basis functions
compared to the 1st basis function (that scales the flux). The stamp
image of the kernel shows oscillations (rings) between positive and
negative values. DM-19443 provides debug plots to see the AL solution
coeffiecients at different parts of the image and their
histograms. Unfortunately, example plots of earlier test runs were
not preserved.

The spatially varying solution also inherits the oscillating nature
everywhere in the image. Also the distribution of the higher order
coefficients are wider and have a high mean across the local
solutions. ip_diffim has debug plots via afwDisplay that
can be interactively visualized ds9.

In the visually good cases, when the fingerprinting pattern is away,
the spatial kernel solution is smooth, Gaussian-like and do not have
outlying high coefficients of higher order basis functions. Test runs
of ap_pipe and lightcurve/flux plotting suggests however that the
flux values may be inaccurate. Fluxes are often around half of their
original values which may simply be a subtraction artefact, not
physical variablity.

Notes

Due to the zero normalization of the basis functions (except for the
first), these components just redistribute flux. Also their
normalization make these basis functions to have a few highly
outlying pixel values (sharp hole or peak in the middle) while their
median pixel values are low. This actually makes it plausible to get
an oscillating solution and high coefficient values.

There was no clear cut algorithm modification in DM-19820 that removed
all the visual artefacts at once on its own.

The deconvolution case use an approximation for the inverse of
Gaussian convolution by a linear combination of 3 Gaussians. We note
that the zero normalization of the basis functions may not be
compatible with the inverse approximation but this needs further
confirmation.

The exact PSF match case is not well handled in the code as in this
case the config file external sigma values are taken for the basis
functions. The close to match PSF sigmas case is limited by the
minimum sigma config values.

Future work ideas

In the light of DM-19660 and the false positives sprint in June 2019,
it may be useful to re-run ap_pipe with DM-19820 development ip_diffim
branches and to generate the current metrics of quality beyond the
occasional visual inspection of the images. Properly document, screenshot
debug displays to preserve the result of these test runs.

Study the spatial distribution of the first coefficient that carries
the flux scaling. Are there any specific pattern around the sources
that are bright but otherwise look normal? This is to examine the visually
good, but questionable difference flux cases.

This ticket only has closing comments.

Just to be clear on the terminology — you use the term “fingerprinting solution” to describe a kernel which oscillates, because it looks something like rings on a fingerprint. Is that correct?

Also — I don't think it is clear from the above, but I believe you see these fingerprints in both the deconvolution and non-deconvolution case. Is that right?

Finally, in DM-20730 you wrote a short report in an attempt to get some feedback from Robert Lupton. Did that get anywhere? I don't think we've captured his response (if any).

Otherwise, looks great — thank you!

Just to be clear on the terminology — you use the term “fingerprinting solution” to describe a kernel which oscillates, because it looks something like rings on a fingerprint. Is that correct?

"Fingerprinting" originally comes from the visual pattern in the image differences themselves. And yes, on decam images, we observed this fingerprinting in the image differences in coincidence with fingerprinting (oscillating) kernel solutions.

It is true that in first approximation, we expect that a Gaussian like PSF should be convolved by a Gaussian like smooth kernel to match the PSFs. I don't think however, that all "good" kernel solutions must be necessarily non-oscillating. As far as I recall, in one of the test debug cases with simulated and slightly shifted Gaussian sources, the kernel solutions can have some rings while the image difference still preserves the Gaussian PSF shape. Also, if we consider the deconvolution case: in Fourier space, the inversion of the image space Gaussian convolution is division by a Gaussian like image in frequency space that will contain close to zero high
frequency values (and may not be practically carried out if values are zeroes – this is the much blurring case: if the Gaussian is wide in image space and sharp in frequency space).

Also — I don't think it is clear from the above, but I believe you see these fingerprints in both the deconvolution and non-deconvolution case. Is that right?

Yes. While we just briefly focused on the deconvolution case, the patterns were around in all the images seen in case of deconvolution.

Finally, in DM-20730 you wrote a short report in an attempt to get some feedback from Robert Lupton. Did that get anywhere? I don't think we've captured his response (if any).

I'm afraid there was no follow up discussion on the report.