Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19660

Bug? The PSF is not warped during image differencing

    Details

      Description

      It looks like the template PSF is not warped in image differencing. See lines 407-411 of imagePsfMatch.py in ip_diffim. This leads to a much broader PSF being used for calculating the matching kernel if the template is at a finer plate scale than the science image. Also, if the PSF is elliptical, it may end up with the wrong rotation, and this may be causing an increase in "quadrupole" mis-subtractions.

        Attachments

          Issue Links

            Activity

            Hide
            mrawls Meredith Rawls added a comment -

            Ian and I took a look at this today with both his simulated images and some real HiTS images. Presently, when the template image is warped in difference imaging, the PSF is used as-provided and is not also warped in any way. So, we tried using WarpedPsf (from meas_algorithms) on the template PSF, exactly as it's implemented in pipe_tasks' WarpAndPsfMatchTask.

            For simulated images, the result by eye is a significant improvement when the template has a finer pixel scale than the science image (this means the template image has a larger FWHM than the science image as measured in pixels). With no PSF warping, the diffims have large boxy halo residuals. With PSF warping, these halos disappear, but the chronic fingerprint/ringing signature is unaffected.

            For real images, the result by eye is at least not worse. We think this is probably a second-order improvement while the main problem of the kernel fitting producing the fingerprint signatures dominates.

            The next step is to figure out if this is an appropriate way to handle PSF warping for difference imaging templates. One test that could be done, for example, is generating a bunch of ellipsoidal PSFs and checking to see that they are properly rotated when warping is implemented.

            It is worth noting that the code change we tried has been implemented in ip_diffim's zogy.py, so that supports it being a reasonable thing to do.

            Show
            mrawls Meredith Rawls added a comment - Ian and I took a look at this today with both his simulated images and some real HiTS images. Presently, when the template image is warped in difference imaging, the PSF is used as-provided and is not also warped in any way. So, we tried using WarpedPsf (from meas_algorithms) on the template PSF, exactly as it's implemented in pipe_tasks' WarpAndPsfMatchTask. For simulated images, the result by eye is a significant improvement when the template has a finer pixel scale than the science image (this means the template image has a larger FWHM than the science image as measured in pixels). With no PSF warping, the diffims have large boxy halo residuals. With PSF warping, these halos disappear, but the chronic fingerprint/ringing signature is unaffected. For real images, the result by eye is at least not worse. We think this is probably a second-order improvement while the main problem of the kernel fitting producing the fingerprint signatures dominates. The next step is to figure out if this is an appropriate way to handle PSF warping for difference imaging templates. One test that could be done, for example, is generating a bunch of ellipsoidal PSFs and checking to see that they are properly rotated when warping is implemented. It is worth noting that the code change we tried has been implemented in ip_diffim's zogy.py, so that supports it being a reasonable thing to do.
            Hide
            sullivan Ian Sullivan added a comment -

            Please take a look at these changes, which adds warping the PSF of the template to image differencing. This fixes a bug I was seeing when the template coadd was made at a higher resolution than the science images, which caused the (unwarped) template PSF to be wider than the PSF of the science image.

            If you are aware of a better way to handle warping the PSF of the template, please let me know!

            The code changes are in ip_diffim https://github.com/lsst/ip_diffim/pull/121 though there is an additional pull request in afw just to add a note clarifying that the PSF is not warped as part of warpExposure: https://github.com/lsst/afw/pull/464

            Show
            sullivan Ian Sullivan added a comment - Please take a look at these changes, which adds warping the PSF of the template to image differencing. This fixes a bug I was seeing when the template coadd was made at a higher resolution than the science images, which caused the (unwarped) template PSF to be wider than the PSF of the science image. If you are aware of a better way to handle warping the PSF of the template, please let me know! The code changes are in ip_diffim https://github.com/lsst/ip_diffim/pull/121 though there is an additional pull request in afw just to add a note clarifying that the PSF is not warped as part of warpExposure : https://github.com/lsst/afw/pull/464
            Hide
            jbosch Jim Bosch added a comment -

            Code changes look entirely reasonable; it's always good to have a correct PSF attached to an Exposure when possible.  But I wouldn't have expected that to have any effect on the difference kernel and hence the subtraction, because we should be determining those by minimizing [postage stamps of] the image difference itself, not using the PSF models at all (unless we're just talking about a ZOGY code path, I guess).  Do you understand where the template PSF is being used later in the task to produce the differences you saw?

            Show
            jbosch Jim Bosch added a comment - Code changes look entirely reasonable; it's always good to have a correct PSF attached to an Exposure when possible.  But I wouldn't have expected that to have any effect on the difference kernel and hence the subtraction, because we should be determining those by minimizing [postage stamps of] the image difference itself, not using the PSF models at all (unless we're just talking about a ZOGY code path, I guess).  Do you understand where the template PSF is being used later in the task to produce the differences you saw?
            Hide
            sullivan Ian Sullivan added a comment -

            The issue is that the kernel basis functions are calculated from the FWHM of the PSF, so if the template has a higher resolution than the science image the unwarped PSF of the template may be larger than the PSF of the science image. Then the incorrect FWHM is used in generateAlardLuptonBasisList to determine whether to perform convolution or deconvolution.

            One test failed on Jenkins, though, because the runXY0 test in test_subtractExposures supplied the template PSF FWHM as None, which short circuits most of generateAlardLuptonBasisList. With a non-None PSF the test fails, and I haven't figured out the right way to fix it.

            Show
            sullivan Ian Sullivan added a comment - The issue is that the kernel basis functions are calculated from the FWHM of the PSF, so if the template has a higher resolution than the science image the unwarped PSF of the template may be larger than the PSF of the science image. Then the incorrect FWHM is used in generateAlardLuptonBasisList  to determine whether to perform convolution or deconvolution. One test failed on Jenkins, though, because the runXY0  test in test_subtractExposures  supplied the template PSF FWHM as None, which short circuits most of  generateAlardLuptonBasisList . With a non-None PSF the test fails, and I haven't figured out the right way to fix it.

              People

              • Assignee:
                sullivan Ian Sullivan
                Reporter:
                sullivan Ian Sullivan
                Reviewers:
                Jim Bosch
                Watchers:
                Eric Bellm, Ian Sullivan, Jim Bosch, John Swinbank, Meredith Rawls
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel