Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15646

Rewrite header translation infrastructure for butler Gen 3

    XMLWordPrintable

    Details

      Description

      As decribed on confluence we need a set of classes and methods for taking a PropertyList, extracting the relevant information and creating an object summarizing that information in a standard form. Cameras can implement their own subclasses to override specific information, with the intent to reuse code wherever possible when a header has the same definition across multiple cameras.

      We have not yet decided whether this should go into obs_base or be a standalone infrastructure package.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            One advantage of the standalone repository is that this is another piece of code that could in theory be generally useful (like daf_butler). Especially if it could work with dict-like entities and did not have to be completely PropertyList based.

            Show
            tjenness Tim Jenness added a comment - One advantage of the standalone repository is that this is another piece of code that could in theory be generally useful (like daf_butler). Especially if it could work with dict-like entities and did not have to be completely PropertyList based.
            Hide
            tjenness Tim Jenness added a comment -

            John Parejko thanks for agreeing to take a look at obs_metadata. Please focus your review on the infrastructure and DECam, HSC, and MegaPrime example implementations. I've copied relevant translations from the obs packages. There is some minor code duplication that could be implemented. Jim Bosch was not keen on multiple inheritance of lots of little translator classes, so I've been pondering importing explicit helper methods and attaching them as methods directly to the class. Unlike in the obs package config/ingest.py all translation methods have the same form (trivial mappings become actual methods).

            The code is designed to be as generic as possible and assumes dict-like header access (not PropertyList at the moment, although PropertyList is handled).

            The concept here is that we have a standardized set of derived quantities, rather than letting obs packages do whatever they want (VisitInfo started on this). I've used Astropy classes throughout, and this has the added advantage of letting me do a self consistency check on the RA/Dec and AltAz headers. In theory you should not need to know which particular set of headers you have, the translation routines will work it out for you.

            The initial plan would be to replace the makeRawVisitInfo routine in daf_butler with the ObservationInfo calculation and translate the astropy versions to afw versions so that they can be attached to an exposure. Eventually the two sets of header translation code (one for visit info and one for ingest) will be removed.

            Show
            tjenness Tim Jenness added a comment - John Parejko thanks for agreeing to take a look at obs_metadata. Please focus your review on the infrastructure and DECam, HSC, and MegaPrime example implementations. I've copied relevant translations from the obs packages. There is some minor code duplication that could be implemented. Jim Bosch was not keen on multiple inheritance of lots of little translator classes, so I've been pondering importing explicit helper methods and attaching them as methods directly to the class. Unlike in the obs package config/ingest.py all translation methods have the same form (trivial mappings become actual methods). The code is designed to be as generic as possible and assumes dict-like header access (not PropertyList at the moment, although PropertyList is handled). The concept here is that we have a standardized set of derived quantities, rather than letting obs packages do whatever they want (VisitInfo started on this). I've used Astropy classes throughout, and this has the added advantage of letting me do a self consistency check on the RA/Dec and AltAz headers. In theory you should not need to know which particular set of headers you have, the translation routines will work it out for you. The initial plan would be to replace the makeRawVisitInfo routine in daf_butler with the ObservationInfo calculation and translate the astropy versions to afw versions so that they can be attached to an exposure. Eventually the two sets of header translation code (one for visit info and one for ingest) will be removed.
            Hide
            tjenness Tim Jenness added a comment -

            Thanks for the reviews. I've merged this. Any further work needed to get ingest working will be done on DM-15914.

            Show
            tjenness Tim Jenness added a comment - Thanks for the reviews. I've merged this. Any further work needed to get ingest working will be done on DM-15914 .

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              John Parejko
              Watchers:
              Jim Bosch, John Parejko, Kian-Tat Lim, Russell Owen, Simon Krughoff, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.