Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3040

KPM Measurement: Residual PSF Ellipticity Correlations (TE1), FY15

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Epic Name:
      KPM: TE1, FY15
    • WBS:
      02C.04
    • Team:
      Data Release Production
    • Cycle:
      Summer 2015

      Description

      This will be measured on precursor data (almost certainly from HSC, via DM-2380) using the following procedure:

      • Run visit-level processing (i.e. ProcessCcdTask, probably via the new HSC driver ported on DM-3368).
      • Compute the residual ellipticity correlation function separately in each visit. Bob Armstrong has code we can probably use for this (and will be at LSST2015 to help). The stile project (https://github.com/msimet/Stile) may be another solution (and perhaps a better long-term one).
      • Compute the median correlation function (at separation bins) over multiple visits.

      This will be done in whichever of r or i band has better public HSC data.

      The same measurement procedure will be used for TE2 (DLP-308) in S15 (DM-3047).

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            This experiment has already been done on the HSC side, where we easily met this requirement; see the blue line in https://hsc-jira.astro.princeton.edu/jira/secure/attachment/25808/rho1_safeclip.png. We're meeting both the requirement for FY20 and the SRD minimum, actually - but not the SRD design spec, and that last bit may be really hard.

            As long as we can complete the critical pieces of the HSC merge, we should have no trouble with this requirement for a long time, and we could consider changing the schedule for the metric values a bit.

            Show
            jbosch Jim Bosch added a comment - This experiment has already been done on the HSC side, where we easily met this requirement; see the blue line in https://hsc-jira.astro.princeton.edu/jira/secure/attachment/25808/rho1_safeclip.png . We're meeting both the requirement for FY20 and the SRD minimum, actually - but not the SRD design spec, and that last bit may be really hard. As long as we can complete the critical pieces of the HSC merge, we should have no trouble with this requirement for a long time, and we could consider changing the schedule for the metric values a bit.
            Hide
            rearmstr Bob Armstrong added a comment - - edited

            Here is the ellipticity residual correlation function for the i-band data.

            Show
            rearmstr Bob Armstrong added a comment - - edited Here is the ellipticity residual correlation function for the i-band data.
            Hide
            rearmstr Bob Armstrong added a comment -

            I should also add that we do not need HSM moments to calculate this, so I don't understand the statement in the description.

            Show
            rearmstr Bob Armstrong added a comment - I should also add that we do not need HSM moments to calculate this, so I don't understand the statement in the description.
            Hide
            jbosch Jim Bosch added a comment -

            I've removed mention of HSM moments; I was mistaken about how we were calculating this on the HSC side.

            Show
            jbosch Jim Bosch added a comment - I've removed mention of HSM moments; I was mistaken about how we were calculating this on the HSC side.
            Hide
            jbosch Jim Bosch added a comment -

            John Swinbank, I think this is essentially measured; do we need to write something up as well?

            Show
            jbosch Jim Bosch added a comment - John Swinbank , I think this is essentially measured; do we need to write something up as well?
            Hide
            swinbank John Swinbank added a comment -

            I think the plan is that there will be a writeup, but exactly what form that will take remains to be seen – David Nidever [X] & Kian-Tat Lim likely have opinions.

            Show
            swinbank John Swinbank added a comment - I think the plan is that there will be a writeup, but exactly what form that will take remains to be seen – David Nidever [X] & Kian-Tat Lim likely have opinions.
            Hide
            ktl Kian-Tat Lim added a comment -

            For now, I'm actually OK with just a description of how the metric was generated (with a bit more detail of the actual experimental conditions and procedures used than the Description section above) and the value itself being put into a comment here, the ticket being placed "In Review" for Mario to verify and then declare "Done".

            Show
            ktl Kian-Tat Lim added a comment - For now, I'm actually OK with just a description of how the metric was generated (with a bit more detail of the actual experimental conditions and procedures used than the Description section above) and the value itself being put into a comment here, the ticket being placed "In Review" for Mario to verify and then declare "Done".
            Hide
            rearmstr Bob Armstrong added a comment - - edited

            The data used in this KPM was the i-band single visit data from the public HSC engineering run.

            To create the plot I took the stars from each visit that were selected for the psf modeling and computed the difference in ellipticities between the model of the psf and the sdss moments. I then computed the average dot product of the residuals for all pairs of stars at different separation distances.

            Once this was computed for each visit I then took the median from all of the i-band visits to create the points in the plot.

            Reading off the plot, the residual correlations are:

            • less than 1 arcmin: 6e-5 requirement: 500e-5
            • beween 1-5 arcmin: 2e-5 (see DM-3047) requirement: 500e-5

            Both of these are well below the requiremnt

            Show
            rearmstr Bob Armstrong added a comment - - edited The data used in this KPM was the i-band single visit data from the public HSC engineering run. To create the plot I took the stars from each visit that were selected for the psf modeling and computed the difference in ellipticities between the model of the psf and the sdss moments. I then computed the average dot product of the residuals for all pairs of stars at different separation distances. Once this was computed for each visit I then took the median from all of the i-band visits to create the points in the plot. Reading off the plot, the residual correlations are: less than 1 arcmin: 6e-5 requirement: 500e-5 beween 1-5 arcmin: 2e-5 (see DM-3047 ) requirement: 500e-5 Both of these are well below the requiremnt
            Hide
            rearmstr Bob Armstrong added a comment -

            I should add that to find all the pairs and compute the averages I used code written by Mike Jarvis called treecorr. It is designed to quickly calculate correlation functions. It was not strictly necessary to use this software for the HSC data in this KPM because the number of stars in a single visit is relatively small. However, it will be necessary when trying to do calculations on larger datasets. Stile also uses treecorr to calculate the correlation functions.

            Show
            rearmstr Bob Armstrong added a comment - I should add that to find all the pairs and compute the averages I used code written by Mike Jarvis called treecorr. It is designed to quickly calculate correlation functions. It was not strictly necessary to use this software for the HSC data in this KPM because the number of stars in a single visit is relatively small. However, it will be necessary when trying to do calculations on larger datasets. Stile also uses treecorr to calculate the correlation functions.
            Hide
            swinbank John Swinbank added a comment -

            We've now shipped the S15 release including the KPM measurement required by this epic. Is any further work required before we close it?

            Show
            swinbank John Swinbank added a comment - We've now shipped the S15 release including the KPM measurement required by this epic. Is any further work required before we close it?
            Hide
            mjuric Mario Juric added a comment -

            Thanks for the review request!

            Comments/questions:

            • I think it'd be good to add a reference to http://ls.st/srd (the "Galaxy shear measurement accuracy, and PSF ellipticity residuals") section to make it easy to trace what's being measured
            • It will be easier if the measurement code is in a git repository
            • It would be better from my PoV to put all TE* measurements into one JIRA Issue (or even a "KPM testing definition document") – having them scattered over four is a bit of a pain to review and track. I'll understand if there are management/procedure issues that make this difficult.
            • From the code: I wonder if this is correct?

              useFlag = src.schema.find('calib_psfUsed').key
              mask = src.get(useFlag)==1
              

              Does this snippet select only stars that were used to derive the PSF? If yes: I'd argue we should actually measure the residuals only on stars not used to derive the PSF model, as the goal is to establish that the interpolated PSF residuals are small anywhere in the focal plane (i.e., at locations of galaxies whose shapes we'll measure). For example, if I was evil I could devise an "interpolation scheme" where PSF at the position of each star is exactly the image of the star, while elsewhere it's zero – that would pass your test. If no: ignore.

            Show
            mjuric Mario Juric added a comment - Thanks for the review request! Comments/questions: I think it'd be good to add a reference to http://ls.st/srd (the "Galaxy shear measurement accuracy, and PSF ellipticity residuals") section to make it easy to trace what's being measured It will be easier if the measurement code is in a git repository It would be better from my PoV to put all TE* measurements into one JIRA Issue (or even a "KPM testing definition document") – having them scattered over four is a bit of a pain to review and track. I'll understand if there are management/procedure issues that make this difficult. From the code: I wonder if this is correct? useFlag = src.schema.find('calib_psfUsed').key mask = src.get(useFlag)==1 Does this snippet select only stars that were used to derive the PSF? If yes: I'd argue we should actually measure the residuals only on stars not used to derive the PSF model, as the goal is to establish that the interpolated PSF residuals are small anywhere in the focal plane (i.e., at locations of galaxies whose shapes we'll measure). For example, if I was evil I could devise an "interpolation scheme" where PSF at the position of each star is exactly the image of the star, while elsewhere it's zero – that would pass your test. If no: ignore.
            Hide
            mjuric Mario Juric added a comment -

            Noticing that this is an FY15 task, if it's more consistent with the work scheduling methodology, it'd be OK with me to resolve the above as a part of FY16 KPM measurement work. Also, similar to DM-3057 another pair of eyes (David Nidever [X]'s) on the code would be good to have.

            Show
            mjuric Mario Juric added a comment - Noticing that this is an FY15 task, if it's more consistent with the work scheduling methodology, it'd be OK with me to resolve the above as a part of FY16 KPM measurement work. Also, similar to DM-3057 another pair of eyes ( David Nidever [X] 's) on the code would be good to have.

              People

              • Assignee:
                swinbank John Swinbank
                Reporter:
                swinbank John Swinbank
                Reviewers:
                Mario Juric
                Watchers:
                Bob Armstrong, Jim Bosch, John Swinbank, Kian-Tat Lim, Lauren MacArthur, Mario Juric
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel