Details
-
Type:
Story
-
Status: Invalid
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ci_hsc
-
Labels:None
-
Team:Architecture
Description
When I run scons in ci_hsc, it fails as follows:
partial([".scons/sfm-903334-16"], ["DATA/registry.sqlite3", "DATA/CALIB", ".scons/sfm"])
|
: Validating dataset processCcd_config for {'ccd': 16, 'visit': 903334}
|
CameraMapper: Loading registry registry from /raid/swinbank/src/ci_hsc/DATA/registry.sqlite3
|
CameraMapper: Loading calibRegistry registry from /raid/swinbank/src/ci_hsc/DATA/CALIB/calibRegistry.sqlite3
|
: processCcd_config exists: PASS
|
: processCcd_config readable (<class 'lsst.pipe.tasks.processCcd.ProcessCcdConfig'>): PASS
|
: Validating dataset processCcd_metadata for {'ccd': 16, 'visit': 903334}
|
: processCcd_metadata exists: PASS
|
scons: *** [.scons/sfm-903334-16] Exception : input stream error
|
Traceback (most recent call last):
|
File "/nfs/home/lsstsw/stack/Linux64/scons/2.3.0+1/lib/scons/SCons/Action.py", line 1062, in execute
|
result = self.execfunction(target=target, source=rsources, env=env)
|
File "/raid/swinbank/src/ci_hsc/python/lsst/ci/hsc/validate.py", line 116, in scons
|
return self.run(*args, **kwargs)
|
File "/raid/swinbank/src/ci_hsc/python/lsst/ci/hsc/validate.py", line 92, in run
|
self.validateDataset(dataId, ds)
|
File "/raid/swinbank/src/ci_hsc/python/lsst/ci/hsc/validate.py", line 67, in validateDataset
|
print(data.__class__)
|
File "/home/lsstsw/stack/Linux64/daf_persistence/2015_10.0-6-g5a5f333+5/python/lsst/daf/persistence/readProxy.py", line 41, in __getattribute__
|
subject = oga(self, '__subject__')
|
File "/home/lsstsw/stack/Linux64/daf_persistence/2015_10.0-6-g5a5f333+5/python/lsst/daf/persistence/readProxy.py", line 136, in __subject__
|
set_cache(self, get_callback(self)())
|
File "/home/lsstsw/stack/Linux64/daf_persistence/2015_10.0-6-g5a5f333+5/python/lsst/daf/persistence/butler.py", line 279, in <lambda>
|
callback = lambda: self._read(pythonType, location)
|
File "/home/lsstsw/stack/Linux64/daf_persistence/2015_10.0-6-g5a5f333+5/python/lsst/daf/persistence/butler.py", line 452, in _read
|
location.getCppType(), storageList, additionalData)
|
File "/home/lsstsw/stack/Linux64/daf_persistence/2015_10.0-6-g5a5f333+5/python/lsst/daf/persistence/persistenceLib.py", line 1430, in unsafeRetrieve
|
return _persistenceLib.Persistence_unsafeRetrieve(self, *args)
|
Exception: input stream error
|
scons: building terminated because of errors.
|
Attachments
Issue Links
- blocks
-
- Done
Activity
Linking DM-7049, which is moving what definitions we can from obs* to daf_butlerUtils.
Note that it's not actually possible to move all of the *_metadata policies to daf_butlerUtils, since the templates for the single-frame processing metadata are different for each camera.
Can we keep a compatibility layer that fails-back to the old-style metadata while emitting a deprecation warning? After say ~1 year this layer could be related.
This suggestion relates to the request for a deprecation implementation policy of https://jira.lsstcorp.org/browse/RFC-213.
Michael Wood-Vasey
added a comment - - edited Can we keep a compatibility layer that fails-back to the old-style metadata while emitting a deprecation warning? After say ~1 year this layer could be related.
This suggestion relates to the request for a deprecation implementation policy of https://jira.lsstcorp.org/browse/RFC-213 . If we want backwards compatbility (I'm ambivalent about that myself), I think we'd want to:
- Define new datasetTypes for YAML metadata files.
- Switch new tasks to write the new YAML metadata files instead of the boost-serialized ones.
- Remove write support for boost-serialized metadata, but keep read support.
+1 on switching to YAML and centralizing the metadata policies. That would be a great solution.