# Convert meas_mosaic wcs output to a format directly readable by the butler

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
2
• Sprint:
DRP S18-3
• Team:
Data Release Production

#### Description

meas_mosaic writes its WCS as FITS header with no attached image, which requires loading code to use the following pattern:

 md = butler.get("wcs_md", ...) wcs = lsst.afw.image.makeWcs(md) 

 wcs = butler.get("wcs", ...) 

Using a header-only format also limits us to FITS-standard WCS mappings.

Because Wcs inherits from afw::table::io::Persistable it already has writeFits and readFits methods that utilize our FITS binary table format, which will be able to save more complex WCS solutions. It's also compatible (or will be soon, on DM-10728) with the "FitsCatalogStorage" butler storage type, so we should be able to fix this by:

• Redefining "wcs" to be a "FitsCatalogStorage" dataset, instead of a "FitsStorage" exposure in all mappers;
• modifying meas_mosaic (and possibly jointcal, if needed) to use butler.put directly.
• modifying any code that consumes the wcs dataset to use butler.get directly.

In addition, this issue should include creating a simple command-line script that can be used to convert a data repository from the old format to the new one.

#### Activity

Hide
Lauren MacArthur added a comment -

Oh, and I think pipe_analysis is fine as is (it only checks on datasetExists, the "getting" is delegated to applyMosaicResultsCatalog in lsst.meas.mosaic.updateExposure).

Show
Lauren MacArthur added a comment - Oh, and I think pipe_analysis is fine as is (it only checks on datasetExists, the "getting" is delegated to applyMosaicResultsCatalog in lsst.meas.mosaic.updateExposure ).
Hide
Jim Bosch added a comment -

Yes, I think the backwards-compatibility code in meas_mosaic removes the need for the script to convert data repositories.

In addition to Jenkins, I'll rerun meas_mosaic on the next weekly outputs and remake at least one coadd patch; I think if that goes through, it should be sufficient to say that this works (hard to imagine this change breaking something without it resulting in a hard failure).

Show
Jim Bosch added a comment - Yes, I think the backwards-compatibility code in meas_mosaic removes the need for the script to convert data repositories.   In addition to Jenkins, I'll rerun meas_mosaic on the next weekly outputs and remake at least one coadd patch; I think if that goes through, it should be sufficient to say that this works (hard to imagine this change breaking something without it resulting in a hard failure).
Hide
Jim Bosch added a comment -

I'm going to defer the final test and merge of this to next week - since the last weekly, DM-12447 introduced an ABI change in afw, and that means that I can't do the meas_mosaic tests without either rebuilding essentially the full stack on lsst-dev or just waiting for the next weekly.  The latter is a lot easier.

Show
Jim Bosch added a comment - I'm going to defer the final test and merge of this to next week - since the last weekly, DM-12447 introduced an ABI change in afw, and that means that I can't do the meas_mosaic tests without either rebuilding essentially the full stack on lsst-dev or just waiting for the next weekly.  The latter is a lot easier.
Hide
Jim Bosch added a comment -

meas_mosaic tests mentioned have been completed; I re-ran meas_mosaic and then rebuilt a single-patch coadd.  The pixel values were identical to the same patch in the w_2018_08/DM-13532 run, and I think that's sufficient.

Running one more Jenkins job, as I think I needed to rebase some packages since the last one.

Show
Jim Bosch added a comment - meas_mosaic tests mentioned have been completed; I re-ran meas_mosaic and then rebuilt a single-patch coadd.  The pixel values were identical to the same patch in the w_2018_08/ DM-13532 run, and I think that's sufficient. Running one more Jenkins job, as I think I needed to rebase some packages since the last one.
Hide
Jim Bosch added a comment -

Merged to master.

Show
Jim Bosch added a comment - Merged to master.

#### People

Assignee:
Jim Bosch
Reporter:
Jim Bosch
Reviewers:
Lauren MacArthur
Watchers:
Hsin-Fang Chiang, Jim Bosch, John Parejko, Lauren MacArthur, Paul Price, Russell Owen