# Calibration products need more metadata

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
4
• Epic Link:
• Sprint:
Arch 2019-02-11, Arch 2019-02-18, Arch 2019-02-25, Arch 2019-03-04, Arch 2019-03-11, Arch 2019-03-18
• Team:
Architecture

#### Description

I was testing the new metadata translator on FITS file I had lying around in testdata repositories and I see that our calibration products do not seem to have a reasonable set of headers. The only indication this is a HSC calibration file rather than an LSST one is the path name to the root directory.

 SIMPLE = T / file does conform to FITS standard  BITPIX = -32 / number of bits per data pixel  NAXIS = 2 / number of data axes  NAXIS1 = 2048 / length of data axis 1  NAXIS2 = 4176 / length of data axis 2  EXTEND = T / FITS dataset may contain extensions  COMMENT FITS (Flexible Image Transport System) format is defined in 'Astronomy COMMENT and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H  OBSTYPE = 'bias '  HIERARCH CALIB_CREATION_DATE = '2016-03-28'  HIERARCH CALIB_CREATION_TIME = '16:16:33 EDT'  HIERARCH CALIB_CREATION_ROOT = '/tigress/HSC/HSC/rerun/dm-5124/calib'  HIERARCH CALIB_INPUT_0 = '(904542,)'  HIERARCH CALIB_INPUT_1 = '(904544,)'  HIERARCH CALIB_INPUT_2 = '(904546,)'  HIERARCH CALIB_INPUT_3 = '(904548,)'  HIERARCH CALIB_INPUT_4 = '(904550,)'  HIERARCH CALIB_INPUT_5 = '(904552,)'  HIERARCH CALIB_INPUT_6 = '(904554,)'  HIERARCH CALIB_INPUT_7 = '(904556,)'  HIERARCH CALIB_INPUT_8 = '(904558,)'  HIERARCH CALIB_INPUT_9 = '(904560,)'  HIERARCH CALIB_INPUT_10 = '(904562,)'  HIERARCH CALIB_INPUT_11 = '(904564,)'  HIERARCH CALIB_INPUT_12 = '(904566,)'  CALIB_ID= 'filter=NONE calibDate=2013-11-03 ccd=50'  HIERARCH MD5_IMAGE = '6185bb72f20de7e81c45e6e6591eb6ad'  CRVAL1A = 0 / Column pixel of Reference Pixel  CRVAL2A = 0 / Row pixel of Reference Pixel  CRPIX1A = 1 / Column Pixel Coordinate of Reference  CRPIX2A = 1 / Row Pixel Coordinate of Reference  CTYPE1A = 'LINEAR ' / Type of projection  CTYPE2A = 'LINEAR ' / Type of projection  CUNIT1A = 'PIXEL ' / Column unit  CUNIT2A = 'PIXEL ' / Row unit 

At minimum there should be INSTRUME/TELESCOP headers but ideally we should be able to construct a full header based on all the input headers (propagating headers that are identical and indicating which headers define bounds). The latter is what I implemented at the JCMT Science Archive for all data products. It might be that another approach is to at least be able to generate a standard set of headers from the information the Butler (Gen 3) knows about each dataset.

#### Activity

Hide
Paul Price added a comment -

pipe_drivers changes look good. Please be sure to test them, as they do not currently get exercised by unit tests, the demo or ci_hsc. I think ci_ctio0m9 might do.

Show
Paul Price added a comment - pipe_drivers changes look good. Please be sure to test them, as they do not currently get exercised by unit tests, the demo or ci_hsc. I think ci_ctio0m9 might do.
Hide
Tim Jenness added a comment -

They work with ci_lsst (Which is where I test them). I will have a look at ci_ctio0m9.

Show
Tim Jenness added a comment - They work with ci_lsst (Which is where I test them). I will have a look at ci_ctio0m9.
Hide
Tim Jenness added a comment -

Simon Krughoff thank you for agreeing to review the changes to astro_metadata_translator.

Show
Tim Jenness added a comment - Simon Krughoff thank you for agreeing to review the changes to astro_metadata_translator.
Hide
Simon Krughoff added a comment -

As I understand it Paul Price is ok with the pipe_drivers changes. I've reviewed the other changes and have a few comments on the PR.

Show
Simon Krughoff added a comment - As I understand it Paul Price is ok with the pipe_drivers changes. I've reviewed the other changes and have a few comments on the PR.
Hide
Tim Jenness added a comment -

Thanks for the reviews. I tested with ci_ctio0m9 and tweaked the patch so that it functions properly if header translation fails. Jenkins passed and I tested again with ci_lsst. Merged.

Show
Tim Jenness added a comment - Thanks for the reviews. I tested with ci_ctio0m9 and tweaked the patch so that it functions properly if header translation fails. Jenkins passed and I tested again with ci_lsst. Merged.

#### People

Assignee:
Tim Jenness
Reporter:
Tim Jenness
Reviewers:
Paul Price, Simon Krughoff
Watchers:
Jim Bosch, John Swinbank, Merlin Fisher-Levine, Paul Price, Robert Lupton, Simon Krughoff, Tim Jenness
Votes:
0 Vote for this issue
Watchers:
7 Start watching this issue

#### Dates

Created:
Updated:
Resolved:

#### CI Builds

No builds found.