The mean number of false positives due to noise should be ~0.1 in each of these simulations, following the calculation in DMTN-006. Below I've switched to reporting the median number of dipoles or DIA sources instead of the mean to reject the cluster of science images with very good seeing that have a large number of dipoles in all of the plots. These images have narrower PSFs than the coadd PSF, and the number of dipoles and DIA sources for these images drop to the levels in the other images if I convolve the science image with the PSF-matching kernel instead of the template.
Median number of dipoles detected in the image differences
Seeing range |
Deep dipoles |
DCR dipoles |
0% |
9 |
1 |
10% |
11 |
1 |
21% |
10 |
1 |
33% |
15 |
2 |
46% |
13 |
2 |
61% |
13 |
2 |
77% |
13 |
3 |
95% |
17 |
3 |
Median number of DIA sources detected in the image differences
Seeing range |
Deep DIA sources |
DCR DIA sources |
0% |
27 |
2 |
10% |
31 |
2 |
21% |
29 |
3 |
33% |
27 |
4 |
46% |
26 |
4 |
61% |
50 |
5 |
77% |
65 |
5 |
95% |
109 |
5 |
Unfortunately, the PSF of the DCR-based template has complex structure when it is built using a wide range of seeing values, and this is not well captured by the PSF matching kernel. As a result there are bright oscillating residuals at the location of many sources, and these are not detected. Based on the sharp increase in the number of DIA sources above 46% seeing range, I would suggest that be a reasonable upper limit for either algorithm currently.
Because I found the number of dipoles or the number of DIA sources to be unreliable for templates built with a wide variation in seeing, I attempted to come up with a better metric to capture the quality of the subtraction. I took a single science image mask as a reference, and evaluated the mean of the absolute value of the pixels identified as detected in the reference mask for every difference image. My hope was that this would clearly show the residuals getting worse as the seeing range increased and the template got worse. The median of that residual across all of the difference images for each seeing range is tabulated below, but unfortunately this metric appears to get better despite the residuals clearly getting worse by eye.
Median residual in the difference images
Calculated using the absolute value of the residual within the footprints of sources detected in the science image.
Seeing range |
Deep residual |
DCR residual |
0% |
20.76 |
18.18 |
10% |
20.84 |
18.32 |
21% |
21.20 |
18.66 |
33% |
21.35 |
19.25 |
46% |
19.02 |
17.09 |
61% |
21.91 |
17.33 |
77% |
19.90 |
13.60 |
95% |
23.26 |
14.95 |
I generated nine sets of eight simulated images of the same field, each with the same sources and airmass range but a different range of seeing. The range of seeing in the images used to construct the DcrModel started at zero (constant seeing), and gradually increased to nearly a factor of two. The seeing range was drawn from the same OpSim simulated observations, but the seeing values were scaled to force the maximum seeing to be the given percent greater than the minimum. The DcrModel and compareWarp Coadd were both constructed using all eight of the images, with no additional cuts on seeing.
To test the DcrModel and compareWarp templates, an additional set of 24 observations were simulated for the same sources, with airmass and seeing taken directly from the OpSim simulated observations. The same 24 "science" images were used for image differencing for each of the nine "year 1" simulations using the DCR-matched template and the compareWarp Coadd as the template.
In the attached plots, the arrow starts at the number of dipoles detected in the residual image using the compareWarp template, and ends at the number of dipoles detected using the DCR-matched template. Each arrow is color-coded by the seeing of the "science" image, and there is no obvious correlation between the number of dipoles and the seeing of the science image, demonstrating that PSF matching in image differencing is working well.
Below is the average number of dipoles detected over the 24 alert images, for each range of seeing allowed in the template images.
Seeing range deep dcr
0%: 16.96 2.57
10%: 17.43 2.61
21%: 17.17 3.30
33%: 19.04 4.52
46%: 19.13 4.22
61%: 19.35 4.17
77%: 20.48 4.13
95%: 23.17 4.52