Vishal Kasliwal [X] has produced this figure using the scripts attached.

The brief summary is that we achieve 12.49 mas here, substantially better than the 60 mas requirement. Once Vishal Kasliwal [X] has his JIRA account up and running, he will update this issue with details of the data used and processing applied.

I note the description says we should do this in magnitude bins, which we're not doing (it looks from the script as though we use a single bin spanning mags 18--19.5). Should either add the other bins or explain why we've changed the procedure.

Here are a few plots to illustrate the current status of the Relative Astrometry KPM. First we've constructed histograms of the RMS relative separation deltas for pairs of stars with ~ 5 arcmin of each other that fall within a 0.5-magnitude wide bin. To do so, we used Jim Bosch's pairwise matching script (jimBoschScript.py) to select a list of 'safe' matches. Flags related to de-blending were applied based on a recommendation by Lauren MacArthur. The relevant quantities (id, coord_ra, coord_dec, object, visit, base_PsfFlux_mag) were read out of the GroupView object safeMatches into lists which were then pickled into a single file ListOfLists.pkl to avoid having to re-run the Jim Bosch script.
To create the histograms using KPMScript_hist.py, we first read the pickle file in and then find all matches within an Annulus of central radius D using simple squared Cartesian distance. For each pair, we check to see if a. Both members lie within a given magnitude bin, and b. Both members were observed on the same visit (not always true to the dithering). The spherical distance between the members of the pair is computed for each common visit and the RMS spread in separation measured for a given pair.
To create the scatter plot using KPMScript_scatter.py, a similar routine is followed. However, no rejection is performed based on bins. Instead, all pairs of objects (regardless of magnitude) are used to compute the RMS RS scatter. The file Utilities.py is used to email the user at the end of the script.

Descriptions of individual plots follow.

Plotted above is the histogram of point sources with PSF magnitudes from 16 (inclusive) to 16.5 (exclusive). The RMS of the relative separation from 36 pairs (across visits) is 10.05 mas (miliarcsec) which is well below the S15/S16 targets of 60 and 50 MAS respectively. In fact, the current value is almost within the final spec. Note also the percentage of targets with RMS relative separation (RMS RS) under AM1+AD1 is 2.78%, well under the final AF1 spec (10%).

Plotted above is the histogram of point sources with PSF magnitudes from 16.5 (inclusive) to 17.0 (exclusive). The RMS of the relative separation from 124 pairs (across visits) is 9.89 mas which is within the final spec (10 mas). Note also the percentage of targets with RMS relative separation (RMS RS) under AM1+AD1 is 5.65%, well under the final AF1 spec (10%).

Plotted above is the histogram of point sources with PSF magnitudes from 17.0 (inclusive) to 17.5 (exclusive). The RMS of the relative separation from 358 pairs (across visits) is 12.58 mas which is within the S20 AM1 spec (15 mas), but larger than the final target. However, the percentage of targets with RMS relative separation (RMS RS) under AM1+AD1 is 4.19% which remains under the final AF1 spec (10%).

Plotted above is the histogram of point sources with PSF magnitudes from 17.5 (inclusive) to 18.0 (exclusive). The RMS of the relative separation from 871 pairs (across visits) is 11.77 mas which is within the S20 AM1 spec (15 mas), but larger than the final target. The percentage of targets with RMS relative separation (RMS RS) under AM1+AD1 is 4.13%, which continues to remain under the final AF1 spec (10%).

Plotted above is the histogram of point sources with PSF magnitudes from 18.0 (inclusive) to 18.5 (exclusive). The RMS of the relative separation from 1664 pairs (across visits) is 12.42 mas which is within the S20 AM1 spec (15 mas), but larger than the final target. The percentage of targets with RMS relative separation (RMS RS) under AM1+AD1 is 3.85%, which continues to remain under the final AF1 spec (10%).

Plotted above is the histogram of point sources with PSF magnitudes from 18.5 (inclusive) to 19.0 (exclusive). The RMS of the relative separation from 2905 pairs (across visits) is 11.95 mas which is within the S20 AM1 spec (15 mas), but larger than the final target. The percentage of targets with RMS relative separation (RMS RS) under AM1+AD1 is 3.99%, which continues to remain under the final AF1 spec (10%).

Plotted above is the histogram of point sources with PSF magnitudes from 18.0 (inclusive) to 18.5 (exclusive). The RMS of the relative separation from 4963 pairs (across visits) is 12.43 mas which is within the S20 AM1 spec (15 mas), but larger than the final target. The percentage of targets with RMS relative separation (RMS RS) under AM1+AD1 is 3.12%, which continues to remain under the final AF1 spec (10%).

Next we show a scatter plot of the PSF magnitudes of all pairs of targets. The individual plot points are colored by the RMS RS and are also sized by the RMS RS (for additional clarity) i.e. bigger, darker points represent pairs with poorer RMS RS repeatability.

Notice that there is an indication of some sort of 'binning' in magnitudes. This behavior is much more evident in the D = 20 arcmin scatter plot (https://jira.lsstcorp.org/browse/DLP-311).

NB (John Swinbank Mario Juric Robert Lupton): Mike Jarvis at UPenn (mjarvis@physics.upenn.edu) has pointed out that the final AM1 should be atleast as good as 3 MAS rather than 10 for weak lensing science. We should probably get him to give us a tentative AM1 number based on a detailed calculation and update the final science requirement and stretch goal accordingly.

We've now shipped the S15 release including the KPM measurement required by this epic. Is any further work required before we close it? Further, do you have any thoughts on Vishal Kasliwal [X]/Mike Jarvis's suggestion that the requirement on AM1 should be tightened?

John Swinbank et al., the methodology looks sound, but before we close:

• Are the various scripts that are being used somewhere in a git repository, together with sufficient instructions on how to reproduce the measurements?
• I think both I and David Nidever [X] should do a more careful code review on them to a) double-check for consistency with the letter and spirit of the SRD requirements and, b) check for any potential mistakes.

In general, I think David should audit proposed ways to measure KPMs (sorry David ) – it's always good to have another pair of eyes there, as these tests ultimately determine whether we've passed or failed. We'll probably have to have an independent review of them in sometime near the end of construction.

PS: If it's better from the scheduling point of view, it's fine with me to make the above tasks for FY16 KPM measurements.

Vishal Kasliwal [X], thanks for reminding me! Regarding Mike's comment – we've been aware of that for a while but didn't have the time to act on it yet.

If he's right (and I think he is), it will necessitate a tightening of the SRD. That makes it first my problem (to prove this needs to be done), and then a PST problem (to further review it and assess whether it can be done, given what we've built so far). I'll add it on my TODO list (I'd expect to be able to take it up sometime early next year).

Zeljko Ivezic, take a look some ~three comments up, this is about having to tighten the astrometry requirements to ~3mas (just FYI at this point).

Yes, Robert already mentioned this new 3mas goal to me some time ago. Before we can
propose to change the SRD, we need to understand quantitatively how whatever metric
was used depends on the actual value of that spec. I am puzzled by the following result.
To get to 3mas, and using error ~ FWHM/SNR, we need SNR ~ 200 or larger. The faintest
galaxies in our gold sample (i<25) will have SNR~20. So we need to go 2.5 mag brighter
to even theoretically be able to achieve 3mas. But if you go 2.5 brighter, and the cumulative
counts go down as logN = 0.31*(i-25), only about 15% of galaxies in the gold sample could
theoretically have relative astrometric errors as small as 3 mas. Hence, based on this simple
initial analysis, I conclude that 3 mas is an overkill. To convince me that I am wrong, we will
need a more quantitative description of the origin of this 3 mas requirement.

I'll try to look at this. I hadn't realized there was progress on this front since Bremerton. It would be useful to have the code that is being used to derive these metrics.

David Nidever [X] The code used to generate the plots is attached to this ticket.

jimBoschScript.py does the matching and generates ListOfLists.pkl which is a python pickled object consisting of the required data for all the matches.

KPMScript_hist.py & KPMScript_scatter.py produce the required scatter plots and histograms from ListOfLists.pkl i.e. you can avoid re-running the jimBoschScript.py and just directly use the .pkl object. This may be your best bet because the jimBoschScript.py needs HSC data to run anyway. You can change the matching parameters in the two KPMScripts to make the plots for the required annulus etc...

Hope this helps