# Write metadata translators for obs_lsst cameras

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
10
• Sprint:
Arch 2018-11-12, Arch 2018-11-19, Arch 2018-11-26, Arch 2018-12-03, Arch 2018-12-10, Arch 2019-01-07, Arch 2019-01-14
• Team:
Architecture

## Description

Add translators for cameras in obs_lsst. This is needed in order for LSST data files to be used with Gen 3 butler and also to allow a simple way for NCSA to extract metadata when ingesting test files into the data backbone.

## Activity

Hide
Tim Jenness added a comment -

Michelle Gower TS8 and PhoSim translations work now on the ticket branches. If you are writing Python to use the new translators you need to import lsst.obs.lsst.translators and that will register them with the translation system (with translate_header.py use the new --packages lsst.obs.lsst.translators option to make them available).

I have not switched VisitInfo calculations to use these yet. I am currently trying to decide whether I override the ParseTask class from pipe_tasks to make it use the new translators rather than the old translators. If I do that then at least we won't be having to support the old and new translators and gen2 will be using the new ones as well.

Show
Tim Jenness added a comment - Michelle Gower TS8 and PhoSim translations work now on the ticket branches. If you are writing Python to use the new translators you need to import lsst.obs.lsst.translators and that will register them with the translation system (with translate_header.py use the new --packages lsst.obs.lsst.translators option to make them available). I have not switched VisitInfo calculations to use these yet. I am currently trying to decide whether I override the ParseTask class from pipe_tasks to make it use the new translators rather than the old translators. If I do that then at least we won't be having to support the old and new translators and gen2 will be using the new ones as well.
Hide
Tim Jenness added a comment -

I am now at the point where I have translators for PhoSim, ImSim, AuxTel, and TS8 as they exist in Robert Lupton's test repository and the headers in obs_lsst.

I have also cleaned up the WCS setting code so that it doesn't look for RATEL etc explicitly from the header again but gets the values from the VisitInfo that has already been calculated.

Ideally I would replace the gen2 translator with the new translator and I can have a look at doing that whilst the review is ongoing. That would help remove the added overhead of supporting two translators.

One remaining issue I have is that the IMSIM reduced calibration data files have no headers indicating what these data are (DM-16292): no instrument, telescope, or date, so it's difficult to translate the headers.

This is quite a big ticket to review so can I suggest that Simon Krughoff handles astro_metadata_translator and obs_base and that the obs_lsst changes are shared by Merlin Fisher-Levine and James Chiang.

It's possible that I've made mistakes in migrating logic from the gen2 butler translator.

To test the translations from the command line you can use the translatehdr.py command:

 % translate_header.py -p lsst.obs.lsst.translators data/input/phosim Analyzing data/input/phosim/raw/204595/R11/00204595-R11-S20-det042-000.fits... instrument: PhoSim telescope: LSST datetime_begin: 2022-10-05T06:53:11.357 altaz_begin:  boresight_airmass: 1.01051444909673 boresight_rotation_angle: 195.624333002684 deg boresight_rotation_coord: sky dark_time: 30.0 s datetime_end: 2022-10-05T06:53:41.357 detector_exposure_id: 40919101 detector_group: R11 detector_name: S20 detector_num: 101 detector_serial: R11_S20 exposure_id: 204595 exposure_time: 30.0 s location: (1822838.3011943279, -1062819.3254546772, -6001635.768663893) m object: unknown observation_id: 204595 observation_type: science physical_filter: i pressure: 520.0 mmHg relative_humidity: 40.0 science_program: 204595 temperature: 20.0 deg_C tracking_radec:  visit_id: 204595 

(this requires both the astro_metadata_translator and obs_lsst branches to be set up).

Thanks.

Show
Hide
Simon Krughoff added a comment -

I've reviewed my pieces, I believe. Correct me if I'm wrong. It looks very good to me. Minor comments on the PR.

Show
Simon Krughoff added a comment - I've reviewed my pieces, I believe. Correct me if I'm wrong. It looks very good to me. Minor comments on the PR.
Hide
Merlin Fisher-Levine added a comment -

I'm also done with the review I think, great stuff

Show
Merlin Fisher-Levine added a comment - I'm also done with the review I think, great stuff
Hide
Tim Jenness added a comment -

Thanks for the reviews. Merged astro_metadata_translator and obs_lsst branches. We decided against the obs_base change.

Show
Tim Jenness added a comment - Thanks for the reviews. Merged astro_metadata_translator and obs_lsst branches. We decided against the obs_base change.

## People

• Assignee:
Tim Jenness
Reporter:
Tim Jenness
Reviewers:
James Chiang, Merlin Fisher-Levine, Simon Krughoff
Watchers:
James Chiang, Jim Bosch, Kian-Tat Lim, Merlin Fisher-Levine, Michelle Gower, Simon Krughoff, Tim Jenness