Reduce Wcs pickle size

XMLWordPrintable

Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Team:
Data Release Production

Description

The size of a Wcs pickle increased in DM-7152 (due to this commit). It needs to be reduced so as to maximise the number of inputs a coadd may have (since a list of Wcs is transmitted over the wire in the coaddDriver, and pickle in python 2 has a size that must fit in an int); we are hitting the limit when coadding Cosmos y-band data from HSC.

Activity

Hide
Paul Price added a comment -

 price@pap-laptop:~/LSST/afw (tickets/DM-9985=) $git sub commit 194fcacc3baa8f424c6dd59dfde76c82a08e44ea Author: Paul Price  Date: Tue Mar 28 14:52:05 2017 -0400    wcs: reduce pickle size    The pickle size was increased in commit 74b8006. The Exposure includes  multiple copies of the Wcs (in the header of each of image, mask,  variance, plus the archive version). But we don't have to embed the  Wcs in an Exposure --- we can just write the Wcs to FITS directly.  This decreases the size of the Wcs pickle by a factor of 5.    python/lsst/afw/image/wcs/wcsContinued.py | 13 ++-----------  1 file changed, 2 insertions(+), 11 deletions(-)  Show Paul Price added a comment - price@pap-laptop:~/LSST/afw (tickets/DM-9985=)$ git sub commit 194fcacc3baa8f424c6dd59dfde76c82a08e44ea Author: Paul Price <price@astro.princeton.edu> Date: Tue Mar 28 14:52:05 2017 -0400   wcs: reduce pickle size The pickle size was increased in commit 74b8006. The Exposure includes multiple copies of the Wcs (in the header of each of image, mask, variance, plus the archive version). But we don't have to embed the Wcs in an Exposure --- we can just write the Wcs to FITS directly. This decreases the size of the Wcs pickle by a factor of 5.   python/lsst/afw/image/wcs/wcsContinued.py | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-)
Hide
Paul Price added a comment -

Jenkins passed.

Show
Paul Price added a comment - Jenkins passed . Yusra AlSayyad , would you mind looking at this, please?
Hide

This so-called band-aid looks OK to merge.

Show
Yusra AlSayyad added a comment - This so-called band-aid looks OK to merge.
Hide
Paul Price added a comment -

Thanks for the prompt review, Yusra!

Merged to master.

Show
Paul Price added a comment - Thanks for the prompt review, Yusra! Merged to master.

People

• Assignee:
Paul Price
Reporter:
Paul Price
Reviewers:
Watchers:
John Parejko, Paul Price, Yusra AlSayyad