Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15247

"goodness of fit" criterion for jointcal

    Details

    • Type: Improvement
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: jointcal
    • Team:
      Alert Production

      Description

      In order to determine whether jointcal's fit is "good", "ok", or "bad", we need to compute some sort of statistic at the end of the minimization loop. After some discussion with Daniella H., it seems a good starting approach would be to generate some simulated data using the jointcal models, fit those models, and look at the distribution of chi2s from those fits.

      Of course, we know that our models are inadequate, but given the number of degrees of freedom jointcal usually works with (1e5-1e6 or more), the standard chi2 distribution is close to a narrow Gaussian and chi2/ndof has to be very close to 1 for the fit to be considered "good" by the usual statistical definition.

      I'm not sure how to better approach this question, but it would be good for jointcal to produce some sort of log message (beyond the normal quoting of chi2) about its own confidence in the fit.

        Attachments

          Issue Links

            Activity

            There are no comments yet on this issue.

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Watchers:
                Dominique Boutigny, Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Pierre Astier
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel