Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ctrl_mpexec
-
Labels:
-
Story Points:2
-
Epic Link:
-
Sprint:DB_S20_02
-
Team:Data Access and Database
-
Urgent?:No
Description
Saving a QuantumGraph and run it with pipetask doesn't work. This is the same problem as DM-21724 but the fix got unfixed.
When trying to run a stored QuantumGraph pickle with
pipetask run --qgraph name.pickle -b butler.yaml the error is
_pickle.UnpicklingError: DimensionUniverse with version '0' not found. Note that DimensionUniverse objects are not truly serialized; when using pickle to transfer them between processes, an equivalent instance with the same version must already exist in the receiving process.
|
Slack: https://lsstc.slack.com/archives/C2JPT1KB7/p1590018536265400
Attachments
Issue Links
Activity
Thanks for review! Merged and closing.
I think I understand the reasoning behind DimensionUniverse, my worry now is that unit testing for even trivial things feels very involved, which potentially can lead to unexpected failures.
Thanks for fixing this!
With the two new master branches from this ticket I'm still having issues using separate pipetask commands saving a qgraph and then running it. I suspect I use the commands wrong. What would be the right way to break the pipetask run as in https://github.com/lsst/ci_hsc_gen3/blob/master/bin/pipeline.sh into two steps? I thought the following would work but still running into errors like
lsst.pipe.tasks.colorterms.ColortermNotFoundError: No colorterm found for filter 'HSC-R' with photoCatName 'ps1_pv3_3pi_20170110' = catalog 'ps1*'
|
|
pipetask qgraph -q ci_hsc.pickle -d "patch = 69" -b "$2"/butler.yaml \
|
-i calib/hsc,raw/hsc,masks/hsc,ref_cats,skymaps,shared/ci_hsc -o "$COLLECTION" \
|
-p "$CI_HSC_GEN3_DIR"/pipelines/CiHsc.yaml
|
|
|
pipetask run --qgraph ci_hsc.pickle -j "$1" -b "$2"/butler.yaml \
|
-i calib/hsc,raw/hsc,masks/hsc,ref_cats,skymaps,shared/ci_hsc -o "$COLLECTION" \
|
--register-dataset-types
|
If it's more appropriate to do this in a new ticket I can open one.
Hsin-Fang Chiang
added a comment - Thanks for fixing this!
With the two new master branches from this ticket I'm still having issues using separate pipetask commands saving a qgraph and then running it. I suspect I use the commands wrong. What would be the right way to break the pipetask run as in https://github.com/lsst/ci_hsc_gen3/blob/master/bin/pipeline.sh into two steps? I thought the following would work but still running into errors like
lsst.pipe.tasks.colorterms.ColortermNotFoundError: No colorterm found for filter 'HSC-R' with photoCatName 'ps1_pv3_3pi_20170110' = catalog 'ps1*'
pipetask qgraph -q ci_hsc.pickle -d "patch = 69" -b "$2"/butler.yaml \
-i calib/hsc,raw/hsc,masks/hsc,ref_cats,skymaps,shared/ci_hsc -o "$COLLECTION" \
-p "$CI_HSC_GEN3_DIR"/pipelines/CiHsc.yaml
pipetask run --qgraph ci_hsc.pickle -j "$1" -b "$2"/butler.yaml \
-i calib/hsc,raw/hsc,masks/hsc,ref_cats,skymaps,shared/ci_hsc -o "$COLLECTION" \
--register-dataset-types
If it's more appropriate to do this in a new ticket I can open one. Hsin-Fang Chiang, I think this deserves separate ticket, it's not obvious that it is related to what has been fixed on this ticket.
Agree. DM-25113
Hsin-Fang Chiang
added a comment - Agree. DM-25113 
Looks good; only comment on the PR is about naming.
In case you're curious, part of the reason dimension-related things are this way w.r.t. pickling is because of the way Python's multiprocessing module uses pickling to pass objects between processes - it doesn't provide any way to use one shared object (like a DimensionUniverse) when unpickling others (pickle itself has this functionality, but multiprocessing doesn't provide any way to use it). So while DimensionUniverse isn't directly a singleton, and it isn't used at all as a singleton in most other contexts, there is a global dict containing all instances, and the unpickling code for all dimension-based objects ultimately looks in that.