Details

• Type: Story
• Status: Done
• Priority: Major
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Templates:
• Story Points:
1
• Sprint:
DRP F16-6
• Team:
Data Release Production

Description

This same file should be available as:
The error is:

  MalformedArchiveError:   File "src/CoaddPsf.cc", line 358, in virtual std::shared_ptr lsst::meas::algorithms::CoaddPsf::Factory::read(const InputArchive&, const CatalogVector&) const  Archive assertion failed: catalogs.size() == 2u {0}  File "src/table/io/InputArchive.cc", line 105, in std::shared_ptr lsst::afw::table::io::InputArchive::Impl::get(int, const lsst::afw::table::io::InputArchive&) 

On lsst-qserv-dax01.ncsa.illinois.edu this can be reproduced by

 $source /software/lsstsw/stack/loadLSST.bash $ setup meas_algorithms python >>> import lsst.afw.image >>> e = lsst.afw.image.ExposureF("/datasets/gapon/data/DC_2013/coadd/deepCoadd/r/0/321,4/coadd-r-0-321,4.fits") 

Activity

Hide
Jim Bosch added a comment -

I've found the problem; it's this commit from July 2013:

commit 53e61faef61c9e5ee47d56b89e41f18325d96295
Author: Jim Bosch <jbosch@astro.princeton.edu>
Date: Wed Jul 10 13:16:53 2013 -0400

Overhaul CoaddPsf persistence to make sure everything round-trips (#2948)

This change breaks compatibility with previously-saved CoaddPsfs.

Previously, we were recomputing the average position, using slightly different
inputs to the algorithm when we unpersisted relative to when we originally
control parameters at all.

Unfortunately, it doesn't lend itself to an easy solution; it looks like we weren't saving enough information to fully reconstruct our PSF models back when this file was written. I could probably add some backwards compatibility code that would let us read the file in with a slightly incorrect PSF model, since I'm guessing rebuilding the coadds is out of scope for the PDAC, but we should probably not rely on this generation of data products any longer than we have to, since we won't ever be able to reconstruct the original PSF models exactly.

Show
Jim Bosch added a comment - I've found the problem; it's this commit from July 2013: commit 53e61faef61c9e5ee47d56b89e41f18325d96295 Author: Jim Bosch <jbosch@astro.princeton.edu> Date: Wed Jul 10 13:16:53 2013 -0400 Overhaul CoaddPsf persistence to make sure everything round-trips (#2948) This change breaks compatibility with previously-saved CoaddPsfs. Previously, we were recomputing the average position, using slightly different inputs to the algorithm when we unpersisted relative to when we originally constructed the CoaddPsf. In addition, we weren't saving the warping control parameters at all. Unfortunately, it doesn't lend itself to an easy solution; it looks like we weren't saving enough information to fully reconstruct our PSF models back when this file was written. I could probably add some backwards compatibility code that would let us read the file in with a slightly incorrect PSF model, since I'm guessing rebuilding the coadds is out of scope for the PDAC, but we should probably not rely on this generation of data products any longer than we have to, since we won't ever be able to reconstruct the original PSF models exactly.
Hide
Jim Bosch added a comment -

Russell Owen, could you take a look at this change to add more backwards compatibility to CoaddPsf and the ExposureCatalog it relies on? Some of the ExposureCatalog code should be quite familiar to you, though in order to support even older {{ExposureCatalog}}s I did switch some things around.

Changes are in afw and meas_algorithms; PRs have already been created.

Show
Jim Bosch added a comment - Russell Owen , could you take a look at this change to add more backwards compatibility to CoaddPsf and the ExposureCatalog it relies on? Some of the ExposureCatalog code should be quite familiar to you, though in order to support even older {{ExposureCatalog}}s I did switch some things around. Changes are in afw and meas_algorithms; PRs have already been created.
Hide
Russell Owen added a comment -

This looks very nice (especially the changes to afw). I requested a few trivial comment tweaks on github.

Show
Russell Owen added a comment - This looks very nice (especially the changes to afw). I requested a few trivial comment tweaks on github.
Hide
Jim Bosch added a comment -

Merged to master.

Show
Jim Bosch added a comment - Merged to master.

People

• Assignee:
Jim Bosch
Reporter:
John Gates
Reviewers:
Russell Owen
Watchers:
Jim Bosch, John Gates, John Swinbank, Russell Owen