Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14869

Verify ellipticity definition choice in validate_drp

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Team:
      SQuaRE

      Description

      It seems that there are two commonly used conventions to define the “ellipticity”, which Bosch et al. 2017 refer to as the “distortion” and the “shear” (see Appendix 3 of https://arxiv.org/abs/1705.06766 , particularly footnote 8 ). This difference in definition is also described in the wikipedia article https://en.wikipedia.org/wiki/Gravitational_lensing_formalism#Measures_of_ellipticity . It seems that the LSST pipeline is using the “distortion” definition based on the discussion in Bosch et al. 2017, and this appears to be confirmed by Equation 2 of Mandelbaum et al. 2017 https://arxiv.org/abs/1705.06745 . The formula currently in `validate_drp` looks to be using the “shear” definition, e.g., slide 8 of this presentation by Catherine Heymans http://www.roe.ac.uk/~heymans/KSBf90_for_DUEL/Download_Lecture_Notes_files/Lecture1_CH.pdf . That same formula for the “shear” case also appears in the wikipedia article with the epsilon symbol. Meanwhile, it seems that the SRD is using the “distortion” definition (equations 10 and 11, https://docushare.lsst.org/docushare/dsweb/Get/LPM-17 ). Therefore, it seems that for consistency, that we would also want to use the “distortion” convention in `validate_drp`. In that case, I think we would have

      e = (I_xx - I_yy + 2j*I_xy) / (I_xx + I_yy)
      e1 = np.real(e)
      e2 = np.imag(e)

      Here are the relevant line in validate_drp:

      https://github.com/lsst/validate_drp/blob/2df8e11de267ce4944ad0a12032c95bc244397fa/python/lsst/validate/drp/util.py#L95

        Attachments

          Activity

          Hide
          kbechtol Keith Bechtol added a comment -

          Jim Bosch Could you please verify that the convention to be used for defining the ellipticity is the "distortion" and that the suggested implementation above for the ellipticity components in validate_drp is correct? Thanks!

          Show
          kbechtol Keith Bechtol added a comment - Jim Bosch Could you please verify that the convention to be used for defining the ellipticity is the "distortion" and that the suggested implementation above for the ellipticity components in validate_drp is correct? Thanks!
          Hide
          jbosch Jim Bosch added a comment - - edited

          It looks like distortion is indeed the best convention (it's definitely what's in the SRD, and what the HSM Regaussianization reports).

          The above formula is correct, but you can also use afw code to convert between conventions:

          from lsst.afw.geom.ellipses import Quadrupole, SeparableDistortionTraceRadius
           
          q = Quadrupole(ixx, iyy, ixy)
          s = SeparableDistortionTraceRadius(q)
          e1 = s.getE1()
          e2 = s.getE2()
           
          
          

          While these formulae will work for the direct moments and ellipticities used by the metrics defined in the SRD and implemented in validate_drp, please also note that obtaining PSF-deconvolved ellipticities is significantly trickier, and there's no single best way to do it (HSM's Regaussianization is one algorithm).

          Show
          jbosch Jim Bosch added a comment - - edited It looks like distortion is indeed the best convention (it's definitely what's in the SRD, and what the HSM Regaussianization reports). The above formula is correct, but you can also use afw code to convert between conventions: from lsst.afw.geom.ellipses import Quadrupole, SeparableDistortionTraceRadius   q = Quadrupole(ixx, iyy, ixy) s = SeparableDistortionTraceRadius(q) e1 = s.getE1() e2 = s.getE2()   While these formulae will work for the direct moments and ellipticities used by the metrics defined in the SRD and implemented in validate_drp, please also note that obtaining PSF-deconvolved ellipticities is significantly trickier, and there's no single best way to do it (HSM's Regaussianization is one algorithm).
          Hide
          kbechtol Keith Bechtol added a comment -

          Thanks Jim Bosch

          When using w201826, I don’t see that the functions getE1 and getE2 exist for the Quadrupole class. They seem to be defined in lsst.afw.geom.ellipses.ellipticityBase but I’m not immediately seeing the inheritance to the Quadrupole class.

          Part of the motivation for doing the ellipticity calculation outside the Quadrupole class is that I wanted to take advantage of broadcasting with numpy arrays and I think the Quadrupole object only allows ellipticity calculations for one quadrupole at a time.

          Show
          kbechtol Keith Bechtol added a comment - Thanks Jim Bosch When using w201826, I don’t see that the functions getE1 and getE2 exist for the Quadrupole class. They seem to be defined in lsst.afw.geom.ellipses.ellipticityBase but I’m not immediately seeing the inheritance to the Quadrupole class. Part of the motivation for doing the ellipticity calculation outside the Quadrupole class is that I wanted to take advantage of broadcasting with numpy arrays and I think the Quadrupole object only allows ellipticity calculations for one quadrupole at a time.
          Hide
          jbosch Jim Bosch added a comment -

          Sorry - there was a typo in my snippet above - getE1() and getE2() are defined on the Separable* class, not Quadrupole.

          In any case, numpy vectorization is a reasonable excuse for writing your own.  We should probably think about finding a way to add that to afw as well, but it is indeed not supported at present.

          Show
          jbosch Jim Bosch added a comment - Sorry - there was a typo in my snippet above - getE1() and getE2() are defined on the Separable* class, not Quadrupole. In any case, numpy vectorization is a reasonable excuse for writing your own.  We should probably think about finding a way to add that to afw as well, but it is indeed not supported at present.
          Hide
          kbechtol Keith Bechtol added a comment -
          Show
          kbechtol Keith Bechtol added a comment - Pull request here: https://github.com/lsst/validate_drp/pull/84
          Hide
          jbosch Jim Bosch added a comment -

          Looks good!

          Show
          jbosch Jim Bosch added a comment - Looks good!
          Hide
          kbechtol Keith Bechtol added a comment -

          Merged. We should have a look at the ellipticity metrics output from validate_drp to see how the values have changed on reference datasets given the change in the choice of ellipticity convention.

          Show
          kbechtol Keith Bechtol added a comment - Merged. We should have a look at the ellipticity metrics output from validate_drp to see how the values have changed on reference datasets given the change in the choice of ellipticity convention.
          Hide
          wmwood-vasey Michael Wood-Vasey added a comment -

          Keith Bechtol Thanks for doing this. Calculating the change was going to be my request to do before merging this.

          Looking at the SQuaSH outputs, https://squash.lsst.codes, the quoted TE1 and TE2 values increased by a factor of ~3-5.

          Show
          wmwood-vasey Michael Wood-Vasey added a comment - Keith Bechtol Thanks for doing this. Calculating the change was going to be my request to do before merging this. Looking at the SQuaSH outputs, https://squash.lsst.codes , the quoted TE1 and TE2 values increased by a factor of ~3-5.

            People

            Assignee:
            kbechtol Keith Bechtol
            Reporter:
            kbechtol Keith Bechtol
            Reviewers:
            Jim Bosch
            Watchers:
            Colin Slater, Jim Bosch, Keith Bechtol, Leanne Guy, Michael Wood-Vasey
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.