Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9985

Reduce Wcs pickle size

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw
    • Labels:
    • Team:
      Data Release Production

      Description

      The size of a Wcs pickle increased in DM-7152 (due to this commit). It needs to be reduced so as to maximise the number of inputs a coadd may have (since a list of Wcs is transmitted over the wire in the coaddDriver, and pickle in python 2 has a size that must fit in an int); we are hitting the limit when coadding Cosmos y-band data from HSC.

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            Thanks for the prompt review, Yusra!

            Merged to master.

            Show
            price Paul Price added a comment - Thanks for the prompt review, Yusra! Merged to master.
            Hide
            yusra Yusra AlSayyad added a comment -

            This so-called band-aid looks OK to merge.

            Show
            yusra Yusra AlSayyad added a comment - This so-called band-aid looks OK to merge.
            Hide
            price Paul Price added a comment -

            Jenkins passed.

            Yusra AlSayyad, would you mind looking at this, please?

            Show
            price Paul Price added a comment - Jenkins passed . Yusra AlSayyad , would you mind looking at this, please?
            Hide
            price Paul Price added a comment -

            price@pap-laptop:~/LSST/afw (tickets/DM-9985=) $ git sub
            commit 194fcacc3baa8f424c6dd59dfde76c82a08e44ea
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue Mar 28 14:52:05 2017 -0400
             
                wcs: reduce pickle size
                
                The pickle size was increased in commit 74b8006.  The Exposure includes
                multiple copies of the Wcs (in the header of each of image, mask,
                variance, plus the archive version). But we don't have to embed the
                Wcs in an Exposure --- we can just write the Wcs to FITS directly.
                This decreases the size of the Wcs pickle by a factor of 5.
             
             python/lsst/afw/image/wcs/wcsContinued.py | 13 ++-----------
             1 file changed, 2 insertions(+), 11 deletions(-)
            

            Show
            price Paul Price added a comment - price@pap-laptop:~/LSST/afw (tickets/DM-9985=) $ git sub commit 194fcacc3baa8f424c6dd59dfde76c82a08e44ea Author: Paul Price <price@astro.princeton.edu> Date: Tue Mar 28 14:52:05 2017 -0400   wcs: reduce pickle size The pickle size was increased in commit 74b8006. The Exposure includes multiple copies of the Wcs (in the header of each of image, mask, variance, plus the archive version). But we don't have to embed the Wcs in an Exposure --- we can just write the Wcs to FITS directly. This decreases the size of the Wcs pickle by a factor of 5.   python/lsst/afw/image/wcs/wcsContinued.py | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-)

              People

              • Assignee:
                price Paul Price
                Reporter:
                price Paul Price
                Reviewers:
                Yusra AlSayyad
                Watchers:
                John Parejko, Paul Price, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel