Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15244

Change fluxSigma to fluxErr and similarly for apCorr and covariances

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw

      Description

      Change fluxSigma to fluxErr, apCorrSigma to apCorrErr and similarly for the diagonal elements of covariance matrices. This includes variable names, argument names and afw table field names. This is necessary in order to implement RFC-333.

      Change afw::table::CovarianceMatrixKey to use "Err" as a suffix rather than "Sigma".

      Update the FITS catalog persistence as follows:

      • Bump the schema VERSION constant from 1 to 2 in SchemaImpl.h
      • When reading version 1 catalogs make a fooErr as an alias to every fooSigma field
      • When reading version 0 catalogs make Err aliases instead of Sigma aliases.

        Attachments

          Issue Links

            Activity

            Hide
            rowen Russell Owen added a comment - - edited

            JIRA is not showing branches or pull requests for afw and pipe_tasks, as usual, so here they are:

            The reason there is one more branch than pull request is that JIRA is showing an extra, unrelated branch for obs_subaru (in addition to the correct branch). A bug that's new to me: DM-15273

            Show
            rowen Russell Owen added a comment - - edited JIRA is not showing branches or pull requests for afw and pipe_tasks, as usual, so here they are: afw: https://github.com/lsst/afw/pull/376 pipe_tasks: https://github.com/lsst/pipe_tasks/pull/210 The reason there is one more branch than pull request is that JIRA is showing an extra, unrelated branch for obs_subaru (in addition to the correct branch). A bug that's new to me: DM-15273
            Hide
            rowen Russell Owen added a comment - - edited

            Jim Bosch please review the changes in afw. Sophie Reed please look at the others. If you need help let me know – I am happy to answer questions and if it's too much work I can ask somebody else to review some of the packages.

            Show
            rowen Russell Owen added a comment - - edited Jim Bosch please review the changes in afw. Sophie Reed please look at the others. If you need help let me know – I am happy to answer questions and if it's too much work I can ask somebody else to review some of the packages.
            Hide
            jbosch Jim Bosch added a comment -

            My review is complete.

            Show
            jbosch Jim Bosch added a comment - My review is complete.
            Hide
            rowen Russell Owen added a comment - - edited

            Notes:

            • jointcal has some jupyter notebooks with code that uses the old field names. I looked into updating those notebooks, but they are not python 3 compatible (due to old style print statements) and the ones that need updating have hard-coded paths that are relevant to a particular user, so I gave up and left them alone. John Parejko may wish to weigh in on this.
            • cat has a lot of sql with the old names. I'm not sure what to do with it.
            • qserv has a lot of code with the old names. I've warned Fritz Mueller. He said old tables can be left alone.
            • qa_explorer has a jupyter notebook with the old field names as output of one cell. This should not stop anything in the notebook from running and I could not update the notebook because I could not get display_ginga installed.
            Show
            rowen Russell Owen added a comment - - edited Notes: jointcal has some jupyter notebooks with code that uses the old field names. I looked into updating those notebooks, but they are not python 3 compatible (due to old style print statements) and the ones that need updating have hard-coded paths that are relevant to a particular user, so I gave up and left them alone. John Parejko may wish to weigh in on this. cat has a lot of sql with the old names. I'm not sure what to do with it. qserv has a lot of code with the old names. I've warned Fritz Mueller . He said old tables can be left alone. qa_explorer has a jupyter notebook with the old field names as output of one cell. This should not stop anything in the notebook from running and I could not update the notebook because I could not get display_ginga installed.
            Hide
            Parejkoj John Parejko added a comment -

            The old jointcal notebooks I'm keeping around for reference purposes (in particular, Astrometry.ipynb, CheckSimAstrom.ipynb, and checkCoadd.ipynb, all of which came with the original meas_simastrom). They don't work for other reasons too (e.g., jointcal's debug output having changed), but they have been useful for coming up with ideas for debugging and plots.

            Show
            Parejkoj John Parejko added a comment - The old jointcal notebooks I'm keeping around for reference purposes (in particular, Astrometry.ipynb , CheckSimAstrom.ipynb , and checkCoadd.ipynb , all of which came with the original meas_simastrom). They don't work for other reasons too (e.g., jointcal's debug output having changed), but they have been useful for coming up with ideas for debugging and plots.
            Hide
            sophiereed Sophie Reed added a comment -

            I have looked at all the other pull requests, apart from the one I commented on they all look good.

             

            Show
            sophiereed Sophie Reed added a comment - I have looked at all the other pull requests, apart from the one I commented on they all look good.  
            Hide
            sophiereed Sophie Reed added a comment -

            I've checked the new obs_subaru one and it looks fine.

             

            Show
            sophiereed Sophie Reed added a comment - I've checked the new obs_subaru one and it looks fine.  
            Hide
            rowen Russell Owen added a comment - - edited

            Thank you very much for the reviews.

            I wrote a short community post describing the changes.

            Show
            rowen Russell Owen added a comment - - edited Thank you very much for the reviews. I wrote a short community post describing the changes.

              People

              • Assignee:
                rowen Russell Owen
                Reporter:
                rowen Russell Owen
                Reviewers:
                Jim Bosch, Sophie Reed
                Watchers:
                Jim Bosch, John Parejko, Russell Owen, Sophie Reed
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel