Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: jenkins
-
Labels:None
-
Story Points:0.5
-
Team:Architecture
-
Urgent?:No
Description
Build #31303 failed on node mojave-1 in obs_decam test_ingest.py. Four test cases failed; example error is:
from lsst.afw.image import makeExposure, makeMaskedImage
|
> full = makeExposure(makeMaskedImage(self.readImage()))
|
../../stack/DarwinX86/obs_base/19.0.0-21-gaaa92db+7/python/lsst/obs/base/fitsRawFormatterBase.py:338:
|
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
self = lsst.obs.decam.rawFormatter.DarkEnergyCameraRawFormatter(FileDescriptor(Location('file:///Users/square/j/ws/stack-os-m...mage.TransmissionCurve'), 'metadata': StorageClassPropertyList('PropertyList', pytype='lsst.daf.base.PropertyList')})))
|
def readImage(self):
|
index, metadata = self._determineHDU(self.dataId['detector'])
|
> return lsst.afw.image.ImageI(self.fileDescriptor.location.path, index)
|
E lsst.pex.exceptions.wrappers.FitsError:
|
E File "src/fits.cc", line 1402, in void lsst::afw::fits::Fits::readImageImpl(int, T *, long *, long *, long *) [T = int]
|
E cfitsio error (/Users/square/j/ws/stack-os-matrix/6ab9d5fd5e/lsstsw/build/obs_decam/tests/tmpbmqgsnz8/raw/raw/r/r DECam SDSS c0002 6415.0 1480.0/415282/raw_r_r DECam SDSS c0002 6415.0 1480.fits): non-CFITSIO program error (1) : Reading image
|
E cfitsio error stack:
|
E decompression error: hit end of compressed byte stream
|
E {0}
|
E lsst::afw::fits::FitsError: 'cfitsio error (/Users/square/j/ws/stack-os-matrix/6ab9d5fd5e/lsstsw/build/obs_decam/tests/tmpbmqgsnz8/raw/raw/r/r DECam SDSS c0002 6415.0 1480.0/415282/raw_r_r DECam SDSS c0002 6415.0 1480.fits): non-CFITSIO program error (1) : Reading image
|
E cfitsio error stack:
|
E decompression error: hit end of compressed byte stream
|
E '
|
Attachments
Issue Links
- relates to
-
- Done
Activity
As a data point mojve-4 (seemingly without the "a" in the name) built obs_decam without difficulty. mojave-1 seems to be consistently failing.
It seems that one of the files in testdata_decam on mojave-1 is corrupt. Here is mojave-1:
md5 rawData/raw/*
|
MD5 (rawData/raw/c4d_150227_012718_ori-stripped-shuffled.fits.fz) = 2c3c130890ea32d7caddf3d9c5848685
|
MD5 (rawData/raw/c4d_150227_012718_ori-stripped.fits.fz) = 7cf6c8c6e977514b5141cdc156dd7fdf
|
MD5 (rawData/raw/raw.fits) = ae18d919f3bc663d5d586360009d9457
|
and here is my laptop:
$ md5 rawData/raw/*
|
MD5 (rawData/raw/c4d_150227_012718_ori-stripped-shuffled.fits.fz) = 2c3c130890ea32d7caddf3d9c5848685
|
MD5 (rawData/raw/c4d_150227_012718_ori-stripped.fits.fz) = 57e1a4ea3c16d79e51e3108a3e8d8d0a
|
MD5 (rawData/raw/raw.fits) = ae18d919f3bc663d5d586360009d9457
|
The failing test is the one that is reading c4d_150227_012718_ori-stripped.fits.fz – the tests that read the other raw files are fine.
I will remove the testdata_decam from that node and let it redownload.
Jenkins is running now on mojave-1: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/31312/pipeline
The MD5s are now reporting correctly:
MD5 (rawData/raw/c4d_150227_012718_ori-stripped-shuffled.fits.fz) = 2c3c130890ea32d7caddf3d9c5848685
|
MD5 (rawData/raw/c4d_150227_012718_ori-stripped.fits.fz) = 57e1a4ea3c16d79e51e3108a3e8d8d0a
|
MD5 (rawData/raw/raw.fits) = ae18d919f3bc663d5d586360009d9457
|
mojave-1 seems to be fixed so I will close this ticket.
Adam Thornton thanks for investigating the macOS version issue.
Thanks for resolving this, Tim!
I'm trying not to think about why the file was corrupt.
This was also seen by https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/31297/pipeline