Thanks for filing this ticket. To help prioritize, may I ask if there was a specific failure mode that prompted this ticket?
Defining the "bright star" cutoff in terms of S/N is consistent with expectations, utility, and the SRD. The same cut will be used for both the astrometric and photometric metrics and plots.
They should also be isolated from other sources.
To preserve flexibility and clarity about when cuts are done, and to prevent preferential rejection of less-good measurements of a object, this cutoff should be applied after sources have been matched. How would you suggest defining the S/N of an object? The most obvious possibility that occurs to me is cutting for S/N > X for a specified percentage of the observations of a source. Should that quantile be 50% (median) or something else?
The actual observed RMS (separate from reported S/N) is what we are measuring, so that's out.
validateDrp.py will always conflate astrometric and photometric calibration. At least the photometric metrics will always be sensitive to the astrometric calibration as astrometric calibration is implicitly part of successfully matching sources across visits. Any CCDs or visits that are outliers will have a significant number of sources rejected from the matching.
This ticket will help loosen the association in one direction, and is in keeping with the phrasing of the SRD.