Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: Validation
-
Labels:None
-
Story Points:0.5
-
Team:Alert Production
Description
validate_drp will test more of our code if it uses default config parameters wherever possible. To that effect I would like to ask you to eliminate all config overrides that are not essential and document the reasons for the remaining overrides.
For DECam there are no overrides that are different than the defaults, so the file can simply be emptied (for now).
For CFHT there are many overrides that are different, and an important question is whether the overrides in this package are better for CFHT data than the overrides in obs_cfht; if so, please move them to obs_cfht.
As a heads up: the default star selector is changing from "secondMoment" to "objectSize" in DM-4692 and I hope to allow that in validate_drp, since it works better and is better supported.
Sorry for the incorrect component, but validate_drp is not yet a supported component in JIRA (see DM-5004)
Attachments
Activity
Here it the output for examples/runCfhtTest.sh with tickets/DM-5005:
1295 sources in ccd: 12
|
1301 sources in ccd: 13
|
1279 sources in ccd: 14
|
1309 sources in ccd: 21
|
1389 sources in ccd: 22
|
1438 sources in ccd: 23
|
2062 sources in ccd: 12
|
2164 sources in ccd: 13
|
2441 sources in ccd: 14
|
2179 sources in ccd: 21
|
2342 sources in ccd: 22
|
2440 sources in ccd: 23
|
Median value of the astrometric scatter - all magnitudes: 59.659 mas
|
Astrometric scatter (median) - mag < 21.0 : 9.8 mas
|
Median value of the photometric scatter - all magnitudes: 116.444 mmag
|
Photometric scatter (median) - mag < 21.0 : 13.2 mmag
|
No stars found that are 199.0--201.0 arcmin apart.
|
PA1(RMS) = 18.09+-0.00 mmag
|
PA1(IQR) = 18.65+-0.57 mmag
|
minimum: PF1=20% of diffs more than PA2 = 22.73 mmag (target is < 15 mmag)
|
design : PF1=10% of diffs more than PA2 = 29.00 mmag (target is < 15 mmag)
|
stretch: PF1= 5% of diffs more than PA2 = 35.32 mmag (target is < 10 mmag)
|
Median of distribution of RMS of distance of stellar pairs.
|
DESIGN goals
|
For stars from 17.00 < mag < 21.50
|
from D = [4.00, 6.00] arcmin, AM1=7.77 mas (target is < 10 mas).
|
6.32% of sample deviates by >30 mas (target is < 10%)
|
Median of distribution of RMS of distance of stellar pairs.
|
DESIGN goals
|
For stars from 17.00 < mag < 21.50
|
from D = [19.00, 21.00] arcmin, AM2=7.15 mas (target is < 10 mas).
|
5.46% of sample deviates by >30 mas (target is < 10%)
|
which gives better astrometric results and essentially identical photometric scatter to master (the astrometric improvement is expected due to tickets/DM-5005 using the newer, default, astrometry task):
1295 sources in ccd: 12
|
1301 sources in ccd: 13
|
1279 sources in ccd: 14
|
1309 sources in ccd: 21
|
1389 sources in ccd: 22
|
1438 sources in ccd: 23
|
2062 sources in ccd: 12
|
2164 sources in ccd: 13
|
2441 sources in ccd: 14
|
2179 sources in ccd: 21
|
2342 sources in ccd: 22
|
2440 sources in ccd: 23
|
Median value of the astrometric scatter - all magnitudes: 60.449 mas
|
Astrometric scatter (median) - mag < 21.0 : 12.0 mas
|
Median value of the photometric scatter - all magnitudes: 112.331 mmag
|
Photometric scatter (median) - mag < 21.0 : 13.1 mmag
|
No stars found that are 199.0--201.0 arcmin apart.
|
PA1(RMS) = 20.34+-0.00 mmag
|
PA1(IQR) = 18.16+-0.58 mmag
|
minimum: PF1=20% of diffs more than PA2 = 24.96 mmag (target is < 15 mmag)
|
design : PF1=10% of diffs more than PA2 = 35.15 mmag (target is < 15 mmag)
|
stretch: PF1= 5% of diffs more than PA2 = 38.15 mmag (target is < 10 mmag)
|
Median of distribution of RMS of distance of stellar pairs.
|
DESIGN goals
|
For stars from 17.00 < mag < 21.50
|
from D = [4.00, 6.00] arcmin, AM1=9.65 mas (target is < 10 mas).
|
5.25% of sample deviates by >30 mas (target is < 10%)
|
Median of distribution of RMS of distance of stellar pairs.
|
DESIGN goals
|
For stars from 17.00 < mag < 21.50
|
from D = [19.00, 21.00] arcmin, AM2=8.72 mas (target is < 10 mas).
|
3.79% of sample deviates by >30 mas (target is < 10%)
|
DECam results are unaffected by the change with one exception whose change is well within sigma:
PA1(IQR) = 29.50+-1.04 mmag # DM-4962
|
PA1(IQR) = 29.32+-0.95 mmag # master
|
All changes on in validate_drp tickets/DM-5005
This looks good.
There's one really minor documentation clarification suggestion on the PR.
Merge when ready.
Michael Wood-Vasey
added a comment - This looks good.
There's one really minor documentation clarification suggestion on the PR.
Merge when ready. Thanks for catching the documentation issue. Fixed, rebased and merged to master.
I just pushed a change to obs_cfht master that tweaks a few params for improved performance. I carefully checked the output of validate_drp before doing this and it looks fine (though I encourage you to verify this). So at this point CFHT needs no config overrides on master.
I think the DECam overrides were already unnecessary, but again, I encourage you to verify that.
I suggest leaving a default override file around if you plan to keep offering the examples/runX scripts, to make it easier for users to try out the effects of temporary config overrides.
In my opinion this is a fairly important change, because it makes the results easier to understand.