Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7221

TUNIT header is not uniformly applied to columns in FITS table output from afw.table

    Details

    • Type: Bug
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw
    • Labels:
    • Templates:
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP F16-3
    • Team:
      Data Release Production

      Description

      In looking at the Source table outputs (SRC*.fits files) in the validation_data_hsc repository, we noticed that the headers are not consistently applied in the FITS files generated by afw.table output. In particular, the TUNIT keywords are missing for the coord_ra and coord_dec columns, making it difficult for a client to determine that the columns are persisted in units of radians, especially since the FITS convention is for the default units for angles to be degrees.

      KEYWORD: TUNITn
      REFERENCE: FITS Standard
      STATUS: reserved
      HDU: table
      VALUE: string
      COMMENT: column units
      DEFINITION: The value field shall contain a character string describing
      the physical units in which the quantity in field n, after any
      application of TSCALn and TZEROn, is expressed. The units of all FITS
      header keyword values, with the exception of measurements of angles,
      should conform with the recommendations in the IAU Style Manual. For
      angular measurements given as floating point values and specified with
      reserved keywords, degrees are the recommended units (with the units,
      if specified, given as 'deg').

      http://heasarc.gsfc.nasa.gov/docs/fcg/standard_dict.html

      Accepting that we are not conforming to that convention, the proper value for TUNITn is "rad" for data in units of radians.

      While much access to afw.table-produced files will be through the afw.table interfaces, where this issue is irrelevant as long as round-tripping is successful, for exploratory and debugging purposes it is very useful to be able to access the file contents with standard tools.

      In addition, if the FITS tables produced from afw.table are to be used as the input to the database ingest, providing units for all columns will assist the database system in providing and validating correct column metadata to its clients, whether through VO standards or other means.

        Issue Links

          Activity

          Hide
          jbosch Jim Bosch added a comment -

          Ready for review; John Parejko, could you take a look?

          This is three separate fixes all dealing with afw.table persistence, split into three commits (with another one that's purely cleanup). In addition to the two changes (angle unit writing and AR_NAME header key duplication) described above, I've also modified the angle-column reading code to better handle non-LSST and/or malformed inputs. All changes are on tickets/DM-7221 of afw.

          Show
          jbosch Jim Bosch added a comment - Ready for review; John Parejko , could you take a look? This is three separate fixes all dealing with afw.table persistence, split into three commits (with another one that's purely cleanup). In addition to the two changes (angle unit writing and AR_NAME header key duplication) described above, I've also modified the angle-column reading code to better handle non-LSST and/or malformed inputs. All changes are on tickets/ DM-7221 of afw.
          Hide
          jbosch Jim Bosch added a comment -

          Temporarily taking this out of review, because Jenkins doesn't like it (looks like I broke another test that I hadn't been re-running locally).

          Show
          jbosch Jim Bosch added a comment - Temporarily taking this out of review, because Jenkins doesn't like it (looks like I broke another test that I hadn't been re-running locally).
          Hide
          jbosch Jim Bosch added a comment -

          Fixed, and once again ready for review. I scaled by my ambitions on reading columns with angle units to only read those tagged specifically as Angles by our code back into Angle fields; converting fields without those tags was backwards incompatible, and hence a bigger change than I should attempt on this issue.

          Show
          jbosch Jim Bosch added a comment - Fixed, and once again ready for review. I scaled by my ambitions on reading columns with angle units to only read those tagged specifically as Angles by our code back into Angle fields; converting fields without those tags was backwards incompatible, and hence a bigger change than I should attempt on this issue.
          Hide
          Parejkoj John Parejko added a comment -

          Just a couple of comments in the PR, but otherwise it looks good.

          Show
          Parejkoj John Parejko added a comment - Just a couple of comments in the PR, but otherwise it looks good.
          Hide
          jbosch Jim Bosch added a comment -

          Merged to master.

          Show
          jbosch Jim Bosch added a comment - Merged to master.

            People

            • Assignee:
              jbosch Jim Bosch
              Reporter:
              gpdf Gregory Dubois-Felsmann
              Reviewers:
              John Parejko
              Watchers:
              Gregory Dubois-Felsmann, Jim Bosch, John Parejko, John Swinbank, Loi Ly, Nate Pease, Paul Price, Pim Schellart, Trey Roby
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development

                  Agile