Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: Data Release Production
-
Labels:None
-
Story Points:7
-
Epic Link:
-
Sprint:DRP S18-5, DRP S18-6, DRP F18-1
-
Team:Data Release Production
Description
I ran multiBandDriver.py on HSC-R 9813 3,4 to determine if any cModel configuration parameters can/should be changed to improve the running time and/or best-fit model parameters. My conclusions for the tested parameters are:
initial.nComponents (default 3): For 2, initial fits run in 0.67x the time, but the exp/dev fits run more slowly, so the total fit time is 0.94x. Initial fits are slightly worse on average (with large scatter) and a sizeable fraction of the dev/exp fits are significantly worse (deltalnP < -10).
For nComponents=6, the initial fitting time is more than twice as long, with modest increases in the exp/dev lnP (<1 for most galaxies, with scatter).
Recommendation: leave it at 3 or lower to 2.
initial.optimizer.gradientThreshold (default 1e-2): 1e-3 runs very slightly faster than default and very slightly improves fits. 1e-1 is slightly slower and also results in slightly worse initial/final fits.
Recommendation: lower to 1e-3.
initial.optimizer.maxInnerIterations (default 20): Zero impact on fits, running time for 8 is virtually identical; 50 is 1% slower.
Recommendation: leave at 20.
initial.optimizer.noSR1Term (default False): True radically changes fitting times by factors of (0.33, 4.37, 4.98)x for (initial, exp, dev) i.e. the initial fit is faster but the subsequent exp/dev are much slower.
Confusingly, the initial fits are better on average, whereas the final dev/exp are worse - this may warrant follow-up. Many of the very slow fits have faint PSF mags and may be pathological cases.
Recommendation: leave as False.
region.nFitRadiiMax (default 3): Surprisingly, nfitrad=2 initial fits are 0.5% slower than the default, whereas nfitrad=5 is 2% faster. However, the total fitting time scales by a factor of (0.86, 1.38) for the smaller/larger size.
Smaller fitting regions naturally have larger likelihoods, but I can't make a meaningful comparison between likelihoods - presumably the smaller region fits better in the common are and worse outside.
Recommendation: leave as is pending further investigation.
One common feature is that there is a non-negligible fraction of objects (usually with faint mag_psf) with very small differences in initial fit likelihood but large ones in the dev/exp fit, which should not occur frequently.
The quoted figures can be generated by the notebook created in DM-14118 (lsst-dev:/home/dtaranu/src/mine/taranu_lsst/cModelConfigs.ipynb).
Attachments
Issue Links
Activity
Field | Original Value | New Value |
---|---|---|
Epic Link |
|
Risk Score | 0 |
Status | To Do [ 10001 ] | In Progress [ 3 ] |
Sprint | DRP S18-5 [ 707 ] | DRP S18-5, DRP S18-6 [ 707, 727 ] |
Description |
I ran multiBandDriver.py on HSC-R 9813 3,4 to determine if any cModel configuration parameters can/should be changed to improve the running time and/or best-fit model parameters. My conclusions for the tested parameters are:
initial.nComponents (default 3): For 2, initial fits run in 0.67x the time, but the exp/dev fits run more slowly, so the total fit time is 0.94x. Initial fits are slightly worse on average (with large scatter) and a sizeable fraction of the dev/exp fits are significantly worse (deltalnP < -10). For nComponents=6, the initial fitting time is more than twice as long, with modest increases in the exp/dev lnP (<1 for most galaxies, with scatter). Recommendation: leave it at 3 or lower to 2. initial.optimizer.gradientThreshold (default 1e-2): 1e-3 runs very slightly faster than default and very slightly improves fits. 1e-1 is slightly slower and also results in slightly worse initial/final fits. Recommendation: lower to 1e-3. initial.optimizer.maxInnerIterations (default 20): Zero impact on fits, running time for 8 is virtually identical; 50 is 1% slower. Recommendation: leave at 20. initial.optimizer.noSR1Term (default False): True radically changes fitting times by factors of (0.33, 4.37, 4.98)x for (initial, exp, dev) i.e. the initial fit is faster but the subsequent exp/dev are much slower. Confusingly, the initial fits are better on average, whereas the final dev/exp are worse - this may warrant follow-up. Many of the very slow fits have faint PSF mags and may be pathological cases. Recommendation: leave as False. region.nFitRadiiMax (default 3): Surprisingly, nfitrad=2 initial fits are 0.5% slower than the default, whereas nfitrad=5 is 2% faster. However, the total fitting time scales by a factor of (0.86, 1.38) for the smaller/larger size. Smaller fitting regions naturally have larger likelihoods, but I can't make a meaningful comparison between likelihoods - presumably the smaller region fits better in the common are and worse outside. Recommendation: leave as is pending further investigation. One common feature is that there is a non-negligible fraction of objects (usually with faint mag_psf) with very small differences in initial fit likelihood but large ones in the dev/exp fit, which should not occur frequently. |
I ran multiBandDriver.py on HSC-R 9813 3,4 to determine if any cModel configuration parameters can/should be changed to improve the running time and/or best-fit model parameters. My conclusions for the tested parameters are:
initial.nComponents (default 3): For 2, initial fits run in 0.67x the time, but the exp/dev fits run more slowly, so the total fit time is 0.94x. Initial fits are slightly worse on average (with large scatter) and a sizeable fraction of the dev/exp fits are significantly worse (deltalnP < -10). For nComponents=6, the initial fitting time is more than twice as long, with modest increases in the exp/dev lnP (<1 for most galaxies, with scatter). Recommendation: leave it at 3 or lower to 2. initial.optimizer.gradientThreshold (default 1e-2): 1e-3 runs very slightly faster than default and very slightly improves fits. 1e-1 is slightly slower and also results in slightly worse initial/final fits. Recommendation: lower to 1e-3. initial.optimizer.maxInnerIterations (default 20): Zero impact on fits, running time for 8 is virtually identical; 50 is 1% slower. Recommendation: leave at 20. initial.optimizer.noSR1Term (default False): True radically changes fitting times by factors of (0.33, 4.37, 4.98)x for (initial, exp, dev) i.e. the initial fit is faster but the subsequent exp/dev are much slower. Confusingly, the initial fits are better on average, whereas the final dev/exp are worse - this may warrant follow-up. Many of the very slow fits have faint PSF mags and may be pathological cases. Recommendation: leave as False. region.nFitRadiiMax (default 3): Surprisingly, nfitrad=2 initial fits are 0.5% slower than the default, whereas nfitrad=5 is 2% faster. However, the total fitting time scales by a factor of (0.86, 1.38) for the smaller/larger size. Smaller fitting regions naturally have larger likelihoods, but I can't make a meaningful comparison between likelihoods - presumably the smaller region fits better in the common are and worse outside. Recommendation: leave as is pending further investigation. One common feature is that there is a non-negligible fraction of objects (usually with faint mag_psf) with very small differences in initial fit likelihood but large ones in the dev/exp fit, which should not occur frequently. The quoted figures can be generated by the notebook created in |
Reviewers | Yusra AlSayyad [ yusra ] | Jim Bosch [ jbosch ] |
Status | In Progress [ 3 ] | In Review [ 10004 ] |
Story Points | 4 | 7 |
Sprint | DRP S18-5, DRP S18-6 [ 707, 727 ] | DRP S18-5, DRP S18-6, DRP F18-1 [ 707, 727, 742 ] |
Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
Resolution | Done [ 10000 ] | |
Status | Reviewed [ 10101 ] | Done [ 10002 ] |