Details
-
Type:
Improvement
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Team:Data Release Production
Description
While testing the fix for DM-27185 using BOT data, it was noticed that the variance of an amp in the difference image (difference of two flats at same exposure time) and the cov[0][0] entry (which should be the variance) as calculated by computing the covariances as shown in appendix A of Astier+19, have a fractional relative difference that ranges from less than 0.01% (even 0.0001% sometimes; very good agreement) in most of the cases to a few outliers with 5% or even more than 15% difference in one case. Where are these outliers coming from?
The calculations and relevant function calls are in measureMeanVarCov of ptc.py. For the Astier+19 calculations, reportCovFft and cov methods of the CovFft are the relevant functions (these ones are in astierCovPtcUtils.py).
Example (I attach a file with the full output):
3020100800182094 3020100800183094 C01 7182.92470792412 6616.198621612795 13307.308179573422 6653.654089786711 0.9943706920034497 0.5661176502706805
|
|
3020100800182094 3020100800183094 C00 7284.907886238256 6790.771311302765 13911.331684922758 6955.665842461379 0.976293494412569 2.4282150524515314
|
|
3020100800185094 3020100800186094 C10 5231.724141064015 4812.214512976904 9624.419104076962 4812.209552038481 1.000001030906566 -0.0001030905503034397
|
|
3020100800185094 3020100800186094 C11 5180.399461646926 4739.624305409257 9482.80702244531 4741.403511222655 0.9996247512346954 0.037538962979971474
|
|
3020100800185094 3020100800186094 C12 5090.304697165564 4611.828608685678 9223.934860096106 4611.967430048053 0.9999698997522249 0.0030101153827255095
|
The columns are:
expid1 | expId2 | ampName | muDiff | varDiff | covAstier[0][3] | covAstier[0][3]*0.5 |
|
varDiff/(covAstier[0][3]*0.5) | ((covAstier[0][3]*0.5 - varDiff)/varDiff)*100)
|
where the last one if the fractional relative difference (in %), and covAstier[0][3] is the variance as calculated from Astier+19 (differed by a factor of 2, at this stage of the code, from afw's varDiff ).
Command:
measurePhotonTransferCurve.py /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606 --rerun plazas/DM-27185 --id detector=94 expId=3020100800155^3020100800156^3020100800158^3020100800159^3020100800185^3020100800186^3020100800161^3020100800162^3020100800188^3020100800189^3020100800164^3020100800165^3020100800191^3020100800192^3020100800167^3020100800168^3020100800194^3020100800195^3020100800170^3020100800171^3020100800197^3020100800198^3020100800173^3020100800174^3020100800200^3020100800201^3020100800176^3020100800177^3020100800203^3020100800204^3020100800179^3020100800180^3020100800206^3020100800207^3020100800182^3020100800183^3020100800209^3020100800210^3020100800212^3020100800213^3020100800215^3020100800216 -c maxMeanSignal=120000 ptcFitType=EXPAPPROXIMATION doPhotodiode=False sigmaCutPtcOutliers=5.0 maximumRangeCovariancesAstier=8 initialNonLinearityExclusionThresholdPositive=0.5 --clobber-config --clobber-version -j 1
|
Attachments
Issue Links
Activity
This is being looked at and addressed in DM-29272 (using BOT data too,gen3 repo: BB = dB.Butler("/project/plazas/WORK/DM-23159/TEST_BOT"). Using the fix discussed in that ticket, the fractional difference in the pair 3020100800182094, 3020100800183094 for amp C00, for example, goes from 2.4282150524515314% to 0.000103049%.
(x-axis is the percentual fractional difference)