Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: jointcal
-
Labels:
-
Story Points:2
-
Epic Link:
-
Sprint:AP S19-3, AP S19-4, AP S19-5
-
Team:Alert Production
Description
To get more information about the details of jointcal's fitting processes, we can output the "chi2 contributions" file at each stage of the _iterate_fit() "outer" python loop. This will provide some more details, without being too overwhelming. This would be a boolean config option (named writeChi2ContributionFilesOuter?) that defaults to False. The output files would follow the same naming convention as the initial/final ones, with the fit iteration appended.
This would also be a good time to clean up how saveChi2Contributions() generates the names of the files: we should pass some kind of format string to it and use a combination of find and replace in C++ to generate the two file names for the measured and reference stars. _iterate_fit() should also output a count for each iteration, so that we can more easily associate these new files with the outer fit loop log messages (which will have that count in their filename).
We need to decide whether to write the files before or after the fit loop: the current "initial_chi2" file is written immediately after the model is constructed, before any of the "initialization steps" are taken, while the "final" step happens after all fitting is complete.
Should this option also result in the "initialization steps" also being logged, or should the existing writeChi2ContributionFiles be modified to do that? The 3-4 initialization steps are by far the largest chi2 change, and have outlier rejection turned off, so they are both qualitatively and quantitatively different from the fit loop.
Attachments
Issue Links
Activity
| Field | Original Value | New Value |
|---|---|---|
| Epic Link |
|
| Description |
To get more information about the details of jointcal's fitting processes, we can output the "chi2 contributions" file at each stage of the {{_iterate_fit()}} "outer" python loop. This will provide some more details, without being too overwhelming.
This would also be a good time to clean up how {{saveChi2Contributions()}} generates the names of the files: we should pass some kind of format string to it and use a combination of {{find}} and {{replace}} in C++ to generate the two files names for the measured and reference stars. {{_iterate_fit()}} should also output a count for each iteration, so that we can more easily associate these new files with the outer fit loop log messages (which will have that count in their filename). |
To get more information about the details of jointcal's fitting processes, we can output the "chi2 contributions" file at each stage of the {{_iterate_fit()}} "outer" python loop. This will provide some more details, without being too overwhelming. This would be a boolean config option (named {{writeChi2ContributionFilesOuter}}?) that defaults to False.
This would also be a good time to clean up how {{saveChi2Contributions()}} generates the names of the files: we should pass some kind of format string to it and use a combination of {{find}} and {{replace}} in C++ to generate the two files names for the measured and reference stars. {{_iterate_fit()}} should also output a count for each iteration, so that we can more easily associate these new files with the outer fit loop log messages (which will have that count in their filename). We need to make a call whether to write the files before or after the fit loop: the current "initial_chi2" file is written immediately after the model is constructed, before any of the "initialization steps" are taken ( It's probably easiest to output the files at the start of each loop (for {{i-1}}), since the "final" chi2 file will take care of the state after the last loop, |
| Description |
To get more information about the details of jointcal's fitting processes, we can output the "chi2 contributions" file at each stage of the {{_iterate_fit()}} "outer" python loop. This will provide some more details, without being too overwhelming. This would be a boolean config option (named {{writeChi2ContributionFilesOuter}}?) that defaults to False.
This would also be a good time to clean up how {{saveChi2Contributions()}} generates the names of the files: we should pass some kind of format string to it and use a combination of {{find}} and {{replace}} in C++ to generate the two files names for the measured and reference stars. {{_iterate_fit()}} should also output a count for each iteration, so that we can more easily associate these new files with the outer fit loop log messages (which will have that count in their filename). We need to make a call whether to write the files before or after the fit loop: the current "initial_chi2" file is written immediately after the model is constructed, before any of the "initialization steps" are taken ( It's probably easiest to output the files at the start of each loop (for {{i-1}}), since the "final" chi2 file will take care of the state after the last loop, |
To get more information about the details of jointcal's fitting processes, we can output the "chi2 contributions" file at each stage of the {{_iterate_fit()}} "outer" python loop. This will provide some more details, without being too overwhelming. This would be a boolean config option (named {{writeChi2ContributionFilesOuter}}?) that defaults to False. The output files would follow the same naming convention as the initial/final ones, with the fit iteration appended.
This would also be a good time to clean up how {{saveChi2Contributions()}} generates the names of the files: we should pass some kind of format string to it and use a combination of {{find}} and {{replace}} in C++ to generate the two file names for the measured and reference stars. {{_iterate_fit()}} should also output a count for each iteration, so that we can more easily associate these new files with the outer fit loop log messages (which will have that count in their filename). We need to make a call whether to write the files before or after the fit loop: the current "initial_chi2" file is written immediately after the model is constructed, before any of the "initialization steps" are taken, while the "final" step happens after all fitting is complete. Should this option also result in the "initialization steps" also being logged, or should the existing {{writeChi2ContributionFiles}} be modified to do that? The 3-4 initialization steps are by far the largest chi2 change, and have outlier rejection turned off, so they are both qualitatively and quantitatively different from the fit loop. |
| Summary | Add option to output chi2 files at each external fit iteration | Add option to output chi2 files at each outer fit iteration |
| Description |
To get more information about the details of jointcal's fitting processes, we can output the "chi2 contributions" file at each stage of the {{_iterate_fit()}} "outer" python loop. This will provide some more details, without being too overwhelming. This would be a boolean config option (named {{writeChi2ContributionFilesOuter}}?) that defaults to False. The output files would follow the same naming convention as the initial/final ones, with the fit iteration appended.
This would also be a good time to clean up how {{saveChi2Contributions()}} generates the names of the files: we should pass some kind of format string to it and use a combination of {{find}} and {{replace}} in C++ to generate the two file names for the measured and reference stars. {{_iterate_fit()}} should also output a count for each iteration, so that we can more easily associate these new files with the outer fit loop log messages (which will have that count in their filename). We need to make a call whether to write the files before or after the fit loop: the current "initial_chi2" file is written immediately after the model is constructed, before any of the "initialization steps" are taken, while the "final" step happens after all fitting is complete. Should this option also result in the "initialization steps" also being logged, or should the existing {{writeChi2ContributionFiles}} be modified to do that? The 3-4 initialization steps are by far the largest chi2 change, and have outlier rejection turned off, so they are both qualitatively and quantitatively different from the fit loop. |
To get more information about the details of jointcal's fitting processes, we can output the "chi2 contributions" file at each stage of the {{_iterate_fit()}} "outer" python loop. This will provide some more details, without being too overwhelming. This would be a boolean config option (named {{writeChi2ContributionFilesOuter}}?) that defaults to False. The output files would follow the same naming convention as the initial/final ones, with the fit iteration appended.
This would also be a good time to clean up how {{saveChi2Contributions()}} generates the names of the files: we should pass some kind of format string to it and use a combination of {{find}} and {{replace}} in C++ to generate the two file names for the measured and reference stars. {{_iterate_fit()}} should also output a count for each iteration, so that we can more easily associate these new files with the outer fit loop log messages (which will have that count in their filename). We need to decide whether to write the files before or after the fit loop: the current "initial_chi2" file is written immediately after the model is constructed, before any of the "initialization steps" are taken, while the "final" step happens after all fitting is complete. Should this option also result in the "initialization steps" also being logged, or should the existing {{writeChi2ContributionFiles}} be modified to do that? The 3-4 initialization steps are by far the largest chi2 change, and have outlier rejection turned off, so they are both qualitatively and quantitatively different from the fit loop. |
| Sprint | AP S19-3 [ 831 ] |
| Reviewers | Jim Bosch [ jbosch ] | |
| Status | To Do [ 10001 ] | In Review [ 10004 ] |
| Sprint | AP S19-3 [ 831 ] | AP S19-3, AP S19-4 [ 831, 832 ] |
| Sprint | AP S19-3, AP S19-4 [ 831, 832 ] | AP S19-3, AP S19-4, AP S19-5 [ 831, 832, 833 ] |
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
Thank you for the review. I've cleaned it up per your suggestions. I'll make DM-12446 a priority before doing further work on persisting the internals.
Merged and done.
| Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
Jim Bosch: would you be able to review this +200/-74 ticket? Probably easiest to review by commit: the commits are atomic, and build on each other.
This should make it easier to implement the "inner loop" outputs, so suggestions here will trickle down into that.
Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29465/pipeline