Agreed!

Thanks for filing this ticket. To help prioritize, may I ask if there was a specific failure mode that prompted this ticket?

Defining the "bright star" cutoff in terms of S/N is consistent with expectations, utility, and the SRD. The same cut will be used for both the astrometric and photometric metrics and plots.

They should also be isolated from other sources.

Implementation Question:
----------------------------------
To preserve flexibility and clarity about when cuts are done, and to prevent preferential rejection of less-good measurements of a object, this cutoff should be applied after sources have been matched. How would you suggest defining the S/N of an object? The most obvious possibility that occurs to me is cutting for S/N > X for a specified percentage of the observations of a source. Should that quantile be 50% (median) or something else?

The actual observed RMS (separate from reported S/N) is what we are measuring, so that's out.

Pedantic Note:
--------------------
validateDrp.py will always conflate astrometric and photometric calibration. At least the photometric metrics will always be sensitive to the astrometric calibration as astrometric calibration is implicitly part of successfully matching sources across visits. Any CCDs or visits that are outliers will have a significant number of sources rejected from the matching.

This ticket will help loosen the association in one direction, and is in keeping with the phrasing of the SRD.

I changed the title back to "bright star" because the SRD statements for both astrometric and photometric quantities apply to the "bright end" of isolated point sources.

LSST SRD, Section 3.3
"""For quantities with a strong dependence on the signal-to-noise ratio (SNR), requirements are specified at the bright end, defined here as the magnitude range between 1 mag and 4 mag fainter than the saturation limit (full well) in a given bandpass. Assuming that the faint end of this range corresponds to r = 20, and that 5σ depth is achieved at r = 24.5, the photon statistics limits on photometric and astrometric accuracy (SNR ∼ 200) are 5 millimag and 4 milliarcsec for a fiducial delivered seeing of 0.7 arcsec."""

So the SRD doesn't actually phrase it directly in terms of S/N, but S/N should be a good proxy for the intent. However, a S/N cut of >200 would be too limiting for the small datasets we're currently looking at. I suggest that the default be something like S/N >50, and that this will of course be configurable.

Russell Owen Did a full draft start up on using a SNR cut instead of

cut.

Need to:
1. update the YAML configuration files.
2. Think through goodMatches vs. safeMatches.
3. Look at sample data sets (CFHT, DECam, HSC) to verify reasonableness.

Quick review. Make sure it runs for you. Check output plots to see if you like the SNR plotting and cuts.

Uses S/N threshold instead of mag threshold to define the "bright" stars to use for the SRD and related calculations.

Plots astrometric RMS vs. SNR and mag.

Reworked the plotAstrometric and plotPhotometric functions to depend only on the variables they actually use. Updated the calls to these functions in validate.py.

This looks good. The astrometry plots are a bit narrow due to adding both an SNR and a magnitude plot.

Would it suffice to print the estimated mag at the SNR cutoff? I realize picking a zero point becomes an issue.

(The mag plot does provide a way to show the existence of insane zero points, but it's a fairly crude tool for that.)

Thanks for the feedback. I removed the astrom. vs. mag plot and instead added a SNR vs. mag plot on the photometry plot.

In the process of tying up this current ticket, I insert a quick fix on a new tickets/DM-4683 branch that removed the key=None keyword from a butler.queryMetadata call. I took the liberty of self-merging this 10-character change.

Merged to master.