Fix Version/s: None
Sprint:AP S19-3, AP S19-4, AP S19-5
To get more information about the details of jointcal's fitting processes, we can output the "chi2 contributions" file at each stage of the _iterate_fit() "outer" python loop. This will provide some more details, without being too overwhelming. This would be a boolean config option (named writeChi2ContributionFilesOuter?) that defaults to False. The output files would follow the same naming convention as the initial/final ones, with the fit iteration appended.
This would also be a good time to clean up how saveChi2Contributions() generates the names of the files: we should pass some kind of format string to it and use a combination of find and replace in C++ to generate the two file names for the measured and reference stars. _iterate_fit() should also output a count for each iteration, so that we can more easily associate these new files with the outer fit loop log messages (which will have that count in their filename).
We need to decide whether to write the files before or after the fit loop: the current "initial_chi2" file is written immediately after the model is constructed, before any of the "initialization steps" are taken, while the "final" step happens after all fitting is complete.
Should this option also result in the "initialization steps" also being logged, or should the existing writeChi2ContributionFiles be modified to do that? The 3-4 initialization steps are by far the largest chi2 change, and have outlier rejection turned off, so they are both qualitatively and quantitatively different from the fit loop.
Thank you for the review. I've cleaned it up per your suggestions. I'll make
DM-12446 a priority before doing further work on persisting the internals.
Merged and done.
Jim Bosch: would you be able to review this +200/-74 ticket? Probably easiest to review by commit: the commits are atomic, and build on each other.
This should make it easier to implement the "inner loop" outputs, so suggestions here will trickle down into that.
Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29465/pipeline