# Add skyMap support to butler export/import

XMLWordPrintable

#### Details

• Type: Story
• Status: To Do
• Resolution: Unresolved
• Fix Version/s: None
• Component/s:
• Labels:
• Team:
Architecture
• Urgent?:
No

#### Description

If you try to use butler export with a deepCoadd_skyMap then when you do the butler import you get the following error:

 LookupError: Could not fetch record for required dimension skymap via keys {'skymap': 'hsc_rings_v1'}. 

According to Jim Bosch this is related to https://github.com/lsst/daf_butler/blob/master/python/lsst/daf/butler/core/repoTransfers.py#L334

#### Activity

Hide
Jim Bosch added a comment -

Proximate cause of the error above this was passing elements=() to saveDatasets instead of leaving that off. But a deeper problem - the TODO line mentioned above - would have become a problem once that was fixed. This ticket should probably try to address all TODO items in repoTransfers.py; I think the issues that made those hard to address originally have now been resolved, and import/export are now becoming quite popular.

Show
Jim Bosch added a comment - Proximate cause of the error above this was passing elements=() to saveDatasets instead of leaving that off. But a deeper problem - the TODO line mentioned above - would have become a problem once that was fixed. This ticket should probably try to address all TODO items in repoTransfers.py; I think the issues that made those hard to address originally have now been resolved, and import/export are now becoming quite popular.
Hide
Krzysztof Findeisen added a comment - - edited

EDIT: moved to DM-25786.

Show
Krzysztof Findeisen added a comment - - edited EDIT: moved to DM-25786 .
Hide
Tim Jenness added a comment -

The problem with sphgeom is caused by import using safe_load but export dumping sphgeom objects that haven't got a default serializer registered with yaml. YAML will not import arbitrary python objects using safe_load so this can only be fixed by switching to the unsafe loader (not a good idea) or registering proper yaml serialization callbacks in sphgeom package itself.

Show
Tim Jenness added a comment - The problem with sphgeom is caused by import using safe_load but export dumping sphgeom objects that haven't got a default serializer registered with yaml. YAML will not import arbitrary python objects using safe_load so this can only be fixed by switching to the unsafe loader (not a good idea) or registering proper yaml serialization callbacks in sphgeom package itself.
Hide
Jim Bosch added a comment - - edited

I think this was actually fully fixed by DM-25786 (along with the usage change from my first comment, above). Eli Rykoff, is the original exercise that triggered this something you'd like to try again (when you have time)?

Show
Jim Bosch added a comment - - edited I think this was actually fully fixed by DM-25786 (along with the usage change from my first comment, above). Eli Rykoff , is the original exercise that triggered this something you'd like to try again (when you have time)?

#### People

Assignee:
Unassigned
Reporter:
Eli Rykoff
Watchers:
Eli Rykoff, Jim Bosch, Tim Jenness