Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ci_hsc, meas_modelfit, obs_subaru
-
Labels:
-
Story Points:2
-
Epic Link:
-
Sprint:DRP S17-4
-
Team:Data Release Production
Description
When per-pixel variances are turned off, CModel likelihoods are computed without using the variance at all. This would not matter in a pure likelihood fit, but it means the prior and likelihood are not given the appropriate relative weights - and the relative weighting is not even consistent; it depends on the noise level of the image.
Since the typical effect of this is to make the prior much more informative, there is some danger that fixing this bug will cause other problems due to poorly-constrained fits. To avoid this, I'll add a configuration option to tune the relative weighting of the prior via a constant (which we could set to the typical variance level of the images to get behavior like what we have now without the inconsistency).
Attachments
Activity
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
| Sprint | DRP S17-4 [ 363 ] |
You asked for the mean variance by filter for the Wide survey. I used the attached script (meanVariance.py) to collect the median variance for each patch, and then took the median of those. The results:
- g: 0.001076
- r: 0.002587
- i: 0.003834
- z: 0.014951
- y: 0.060202
Example command-line:
pprice@tiger-sumire:~ $ python -c 'from meanVariance import MeanVarianceTask; MeanVarianceTask.parseAndSubmit()' /tigress/HSC/HSC --rerun production-20160523 --id tract=10045^15827^8279^8766^9075^9128^9318^9371^9561^9614^9802^10046^15828^8284^8767^9076^9132^9319^9375^9562^9617^9803^10047^15829^8285^8768^9077^9133^9320^9376^9563^9618^9804^10048^15830^8520^9004^9078^9134^9321^9377^9564^9619^9805^10049^15831^8521^9005^9103^9135^9345^9378^9588^9620^9806^10287^16005^8526^9006^9104^9313^9346^9556^9589^9621^9807^10288^16006^8762^9007^9106^9314^9349^9557^9592^9798^10289^16007^8763^9008^9125^9315^9368^9558^9611^9799^10290^16008^8764^9009^9126^9316^9369^9559^9612^9800^10291^16009^8765^9010^9127^9317^9370^9560^9613^9801 filter=HSC-I --cores 14 --no-versions
|
It needs a tweaked obs_base with:
pprice@tiger-sumire:~/LSST/obs/base (master *=) $ git diff
|
diff --git a/policy/exposures.yaml b/policy/exposures.yaml
|
index 2e4e85c..ad0d906 100644
|
--- a/policy/exposures.yaml
|
+++ b/policy/exposures.yaml
|
@@ -8,7 +8,7 @@ deepCoadd_calexp:
|
persistable: ExposureF
|
storage: FitsStorage
|
python: lsst.afw.image.ExposureF
|
- template: deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/calexp-%(filter)s-%(tract)d-%(patch)s.fits
|
+ template: deepCoadd/%(filter)s/%(tract)d/%(patch)s/calexp-%(filter)s-%(tract)d-%(patch)s.fits
|
level: None
|
deepDiff_differenceExp:
|
template: ''
|
Paul Price
added a comment - You asked for the mean variance by filter for the Wide survey. I used the attached script (meanVariance.py) to collect the median variance for each patch, and then took the median of those. The results:
g: 0.001076
r: 0.002587
i: 0.003834
z: 0.014951
y: 0.060202
Example command-line:
pprice@tiger-sumire:~ $ python -c 'from meanVariance import MeanVarianceTask; MeanVarianceTask.parseAndSubmit()' /tigress/HSC/HSC --rerun production-20160523 --id tract=10045^15827^8279^8766^9075^9128^9318^9371^9561^9614^9802^10046^15828^8284^8767^9076^9132^9319^9375^9562^9617^9803^10047^15829^8285^8768^9077^9133^9320^9376^9563^9618^9804^10048^15830^8520^9004^9078^9134^9321^9377^9564^9619^9805^10049^15831^8521^9005^9103^9135^9345^9378^9588^9620^9806^10287^16005^8526^9006^9104^9313^9346^9556^9589^9621^9807^10288^16006^8762^9007^9106^9314^9349^9557^9592^9798^10289^16007^8763^9008^9125^9315^9368^9558^9611^9799^10290^16008^8764^9009^9126^9316^9369^9559^9612^9800^10291^16009^8765^9010^9127^9317^9370^9560^9613^9801 filter=HSC-I --cores 14 --no-versions
It needs a tweaked obs_base with:
pprice@tiger-sumire:~/LSST/obs/base (master *=) $ git diff
diff --git a/policy/exposures.yaml b/policy/exposures.yaml
index 2e4e85c..ad0d906 100644
--- a/policy/exposures.yaml
+++ b/policy/exposures.yaml
@@ -8,7 +8,7 @@ deepCoadd_calexp:
persistable: ExposureF
storage: FitsStorage
python: lsst.afw.image.ExposureF
- template: deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/calexp-%(filter)s-%(tract)d-%(patch)s.fits
+ template: deepCoadd/%(filter)s/%(tract)d/%(patch)s/calexp-%(filter)s-%(tract)d-%(patch)s.fits
level: None
deepDiff_differenceExp:
template: ''
Paul Price
made changes -
| Attachment | meanVariance.py [ 29329 ] |
Preliminary tests on UDeep COSMOS runs look good (see attached figure):
- The run with the bug fixed but no scaling factor (middle) preserves extendedness removes the artificial pileup around PSF - CModel = 0.6.
- The run with the bug fixed and a scaling factor derived from the Wide survey i-band coadd variance restores that pileup (and the rest of the original behavior). That's despite the fact that this is UDeep data, so it doesn't look like the results are super sensitive to that factor. This is i-band data, so it's where we'd expect this scaling to do best at reproducing the old behavior, but since it's usually the reference band it's also the band where that matters the most.
So I think this is working well enough that we can do the rest of the testing as part of the usual RC validation.
| Attachment | psf_vs_cmodel.png [ 29346 ] |
Finally ready for review.
When we run the HSC RC data, we'll want to revert the configuration change in obs_subaru for the alternate run. With the configuration change in obs_subaru here we should get approximately the same behavior as we had before, but more consistent.
| Reviewers | Paul Price [ price ] | |
| Status | In Progress [ 3 ] | In Review [ 10004 ] |
Love the plot!
Some trivial comments on the GitHub PRs.
Paul Price
added a comment - Love the plot!
Some trivial comments on the GitHub PRs. Paul Price
made changes -
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
| Epic Link |
|
So, it turns out this change breaks ci_hsc: there's a check there that greater than 95% of the sources we marked as calib_psfUsed must have base_ClassificationExtendedness_value == 0 on the coadd, and with this change that number is only 94.3% (3 out of 530 sources were classified differently). I think I'm going to call that unnecessary strictness in ci_hsc rather than change the extendedness threshold, but we should consider revisiting the extendedness threshold in the RC processing.
The modified ci_hsc is now running successfully through jenkins, so I'm ready to merge. I'll leave it open a bit longer in case Paul Price wants to comment on the ci_hsc changes.
No, merge away.
Paul Price
added a comment - No, merge away. | Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
Paul Price
made changes -
| Component/s | ci_hsc [ 13140 ] | |
| Component/s | obs_subaru [ 10747 ] |
Has been in progress since last week.