I understand that you don't have more time to spend on this, but the next time around we need to use aperture corrected aperture magnitudes.

This may require getting the aperture correction code running reliably, and than in turn may mean more robust algorithms for measuring the aperture fluxes.

Following an off-line conversation between Robert Lupton and Zeljko Ivezic, we will use aperture magnitudes here, and to be conservative, we'll use the RMS estimate, so the final FY15 estimate for PA1 is 11.1 mmag, which does meet the FY15 target. With this change, we also meet the minimum spec for PA2/PF1. See the notebook for details:

https://github.com/lsst/afw/blob/tickets/DM-3896/examples/repeatability.ipynb

Found a bug in the IPython notebook (fixed in the link in the comment immediately above). Since this has gone through so many modifications, here's a brand new writeup that I hope will indeed be the last one:

Our FY15 estimate for the PA1 repeatability metric is 10.6 mmag (which meets the target of 13 mmag), measured on 12-pixel aperture fluxes. This cycle, we will use the same estimate for the procCalRep, PA1ugy, and PA1riz metrics, as these represent the same quantity computed on different datasets, and we are limited to one test dataset in FY15. This dataset is a selection of i-band HyperSuprime-Cam engineering data taken in the SDSS Stripe 82 region. This dataset consists of 30s exposures, so it is somewhat similar to projected LSST data in depth, and we've thus far put more effort into tuning the pipeline on HSC data than most other pipelines. Our current calibration approach has many limitations relative to what we ultimately plan to implement for LSST, so there is still plenty of room for improvement here:

• There's currently no relative calibration being run at all.
• We have only limited correction for chromatic effects.
• There's currently no allowance for zeropoint variations smaller than the scale of a CCD (including the effects of pixel area changes). We have not included the outermost CCDs in the HSC focal plane in the processing (where the geometric factor is largest), however, so this this should not play as large a role as it otherwise might.

We also use a much simpler sample selection than that proposed by the SRD; the SRD suggests using colors and variability to ensure a pure sample of bright, low-variability stars.

We have also measured the related PA2/PF1 metric; we see PF1=20% of measurements are more than PA2=12.0 mmag from the mean, which means the final minimum spec of PA2=15.0 mmag. However, we see PF1=10% of measurements more than PA2=16.9 mmag, which does not meet the design spec (also PA2=15.0 mmag).

Annotated code to compute the metrics can be found at https://github.com/lsst/afw/blob/tickets/DM-3896/examples/repeatability.ipynb.

Hey Jim Bosch & Simon Krughoff – I am not planning to schedule any more work on the FY15 KPMs unless higher authority demands it (there might need to be a formal report produced at some point, but let's wait for requirements). I stuck all the KPM epics on the DRP side to "in review" by Mario & K-T. Since the photometric repeatability KPMs are nominally AP rather than DRP I didn't want to just do the same here, but I figured I'd let you know my thinking.

I think https://confluence.lsstcorp.org/pages/viewpage.action?pageId=41785659 is enough of a report. As far as I'm concerned, you can close all of these issues/epics.