Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-686

Allow interface-breaking cfitsio upgrade in release 20.0

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Newer versions of cfitsio require uppercase header names as a matter of standards conformance. This necessitates modifications to all code that is expecting to find lowercase or mixed-case names in a PropertyList derived from a FITS header; it should use an uppercase name instead. Code that is inserting into a PropertyList that is to be persisted as a FITS header should also use uppercase only for consistency. We will modify the afw FITS writing code to warn if lowercase characters are detected in headers.

      Other PropertyList uses that have no FITS persistence can use any case as desired.

      We would like to adopt a new version of cfitsio prior to Release 20.0 (which is expected to be release at the end of May 2020), rather than having a one-release transition period, as this change enables the nearly-complete use of conda and conda-forge for the third-party dependencies of the Science Pipelines stack and use of the conda compilers for building it.

      This RFC explicitly permits the use of the new cfitsio for Release 20.0 instead of 21.0.

        Attachments

          Issue Links

            Activity

            Hide
            bvan Brian Van Klaveren added a comment -

            I have been trying to follow the call stack of Fits::readMeatadata->Fits::forEachKey in fits.cc stack into cfitsio 3.47, and it seems like that path does NOT uppercase on read.

            https://github.com/lsst/afw/blob/706b86db4bbcb8a2ac40b529dbf274a30fb6761b/src/fits.cc#L791

            Show
            bvan Brian Van Klaveren added a comment - I have been trying to follow the call stack of Fits::readMeatadata->Fits::forEachKey in fits.cc stack into cfitsio 3.47, and it seems like that path does NOT uppercase on read. https://github.com/lsst/afw/blob/706b86db4bbcb8a2ac40b529dbf274a30fb6761b/src/fits.cc#L791
            Hide
            bvan Brian Van Klaveren added a comment -

            after digging a bit more

            • readKey in src/fits.cc might not be actually be used, at least in afw, so I would assume it’s probably not being used anywhere else, but I can’t be positive.
            • readMetadata (and it's variants) seem to be the the predominant method used (see src/image/MaskedImage.cc and src/table/Schema.cc), and those iterate over they keys, extracting the names with fits_read_keyn, which doesn't activate the code path the performs upper-casing.

            In examining src/fits.cc, it seems a good way to do upper casing on read is to upper case key in MetadataIterationFunctor::add immediately before it's added to the PropertyList/PropertySet.

            I'm not sure of how LSST_FITS_CHECK_STATUS is used, but an additional message may be desired too.

            Show
            bvan Brian Van Klaveren added a comment - after digging a bit more readKey in src/fits.cc might not be actually be used, at least in afw, so I would assume it’s probably not being used anywhere else, but I can’t be positive. readMetadata (and it's variants) seem to be the the predominant method used (see src/image/MaskedImage.cc and src/table/Schema.cc), and those iterate over they keys, extracting the names with fits_read_keyn, which doesn't activate the code path the performs upper-casing. In examining src/fits.cc, it seems a good way to do upper casing on read is to upper case key in MetadataIterationFunctor::add immediately before it's added to the PropertyList/PropertySet. I'm not sure of how LSST_FITS_CHECK_STATUS is used, but an additional message may be desired too.
            Hide
            tjenness Tim Jenness added a comment -

            At CCB today we agreed that with the following changes we would be okay with this RFC:

            • Add warn on write if a lower case key is encountered.
            • Always upper case on read so that old files are read in the same way as new files.
            Show
            tjenness Tim Jenness added a comment - At CCB today we agreed that with the following changes we would be okay with this RFC: Add warn on write if a lower case key is encountered. Always upper case on read so that old files are read in the same way as new files.
            Hide
            bvan Brian Van Klaveren added a comment -

            I think we can adopt this now. DM-24376 was just merged which implements the CCB agreements

            Show
            bvan Brian Van Klaveren added a comment - I think we can adopt this now. DM-24376 was just merged which implements the CCB agreements
            Hide
            bvan Brian Van Klaveren added a comment -

            should there also be additional documentation on this somewhere, or is that something that will happen when preparing v20.0?

            Show
            bvan Brian Van Klaveren added a comment - should there also be additional documentation on this somewhere, or is that something that will happen when preparing v20.0?

              People

              • Assignee:
                ktl Kian-Tat Lim
                Reporter:
                ktl Kian-Tat Lim
                Watchers:
                Brian Van Klaveren, Eli Rykoff, John Parejko, John Swinbank, Kian-Tat Lim, Krzysztof Findeisen, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Planned End:

                  Summary Panel