Much progress has been made!
Key improvements include: better understanding of the instrument (as noted above); fix the derivative math, correctly this time (
DM-18014); and fix a typo in the derivative computation.
The fit is now much better behaved, and this makes a big difference in all metrics: repeatability; chromatic consistence; uniformity (in r-band compared to Gaia as a baseline).
First, an update to the instrumental plots above. With the latest fixes, I have better estimates of these "raw" plots, and this shows the form of the mirror much clearer:
Vertical lines show my "mirror intervals", where I have determined (by eye) that the rate of decline changed; the right-most line is the date of the mirror recoating.
I have done 3 separate runs for comparing repeatability stats and to compare to Gaia DR2 G-band (against HSC r-band, with stars with 0.5<g-i<1.5 and a random-forest generated color correction). These 3 runs are with no reference stars (noref); PS1 stars with s/n > 10 (ref10), and PS1 stars with s/n > 50 (ref50). PS1 stars were used with the default color corrections. In all cases HSC-R was corrected to HSC-R2 and HSC-I was corrected to HSC-I2 using the FGCM-derived transformations from the transmission curve information.
There are a few stats to compare.
First: internal repeatability of bright stars (s/n>100), after accounting for photometric error. All quantities are mmag rms for g/r/i/z/y.
- ref50: 5.4/5.1/5.0/4.2/4.5
- ref10: 7.4/6.6/5.8/5.3/6.9
- noref: 4.5/4.3/3.5/3.0/3.4
For this test, ref50 is clearly better than ref10 (the lower s/n stars are very noisy in the refcat and show some systematics as well). Interesting noref is better still, but from plots below it might be "overfitting" locally because the global uniformity isn't as good.
Second: Consistency with the reference stars. All quantities are mmag rms:
- ref50: 21.5/21.0/20.2/17.5/21.7
- ref10: 38.3/33.1/27.9/26.0/37.2
(I don't have a measurement for the noref case because of the way the code is structured; I will return to this at a later date).
Here it's clear that the ref50 is much better, well beyond what the additional photometric errors one would expect from going s/n 50 to 10.
Third: Consistency with Gaia G-band. This is the pixel-to-pixel rms over the full footprint, measured in nside=128 pixels. All quantities are mmag. Also, plots!
- ref50: 1.9
- ref10: 2.2
- noref: 3.0
- noref, before latest fixes: 5.4
The fixes have made a huge improvement in the uniformity even without any reference stars. I'm very pleased that I can tie these disparate fields together so well! But there's definitely information to be gained from the reference stars especially at large scales.
I'm going to try a run with s/n>100 to see what happens.