Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ci_hsc
-
Labels:None
-
Story Points:1
-
Team:Data Facility
Description
ci_hsc fails in my lsst-dev/lsstvc/slurm env (using 1 node) although it passes on Jenkins.
The problem arose when preSfm ran before refcat.
I think this is because preSfm, effectively processCcd.py DATA --rerun ci_hsc --doraise, needs the ref_cats link already in the repo. Running the command in a repo without the ref_cats gives this error:
root INFO: Running: /home/hchiang2/lsstsw/stack/Linux64/pipe_tasks/13.0-8-g148fcaf+1/bin/processCcd.py /home/hchiang2/ciHscTest/asMaster/ci_hsc/DATA --rerun test2 --doraise |
Traceback (most recent call last):
|
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_tasks/13.0-8-g148fcaf+1/bin/processCcd.py", line 25, in <module> |
ProcessCcdTask.parseAndRun()
|
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_base/13.0-1-g92030ba+10/python/lsst/pipe/base/cmdLineTask.py", line 482, in parseAndRun |
resultList = taskRunner.run(parsedCmd)
|
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_base/13.0-1-g92030ba+10/python/lsst/pipe/base/cmdLineTask.py", line 202, in run |
if self.precall(parsedCmd): |
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_base/13.0-1-g92030ba+10/python/lsst/pipe/base/cmdLineTask.py", line 300, in precall |
task = self.makeTask(parsedCmd=parsedCmd)
|
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_base/13.0-1-g92030ba+10/python/lsst/pipe/base/cmdLineTask.py", line 391, in makeTask |
return self.TaskClass(config=self.config, log=self.log, butler=butler) |
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_tasks/13.0-8-g148fcaf+1/python/lsst/pipe/tasks/processCcd.py", line 160, in __init__ |
astromRefObjLoader=astromRefObjLoader, photoRefObjLoader=photoRefObjLoader)
|
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_base/13.0-1-g92030ba+10/python/lsst/pipe/base/task.py", line 237, in makeSubtask |
subtask = taskField.apply(name=name, parentTask=self, **keyArgs)
|
File "/home/hchiang2/lsstsw/stack/Linux64/pex_config/13.0-1-g41367f3+1/python/lsst/pex/config/configurableField.py", line 83, in apply |
return self.target(*args, config=self.value, **kw) |
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_tasks/13.0-8-g148fcaf+1/python/lsst/pipe/tasks/calibrate.py", line 322, in __init__ |
self.makeSubtask('astromRefObjLoader', butler=butler) |
File "/home/hchiang2/lsstsw/stack/Linux64/pipe_base/13.0-1-g92030ba+10/python/lsst/pipe/base/task.py", line 237, in makeSubtask |
subtask = taskField.apply(name=name, parentTask=self, **keyArgs)
|
File "/home/hchiang2/lsstsw/stack/Linux64/pex_config/13.0-1-g41367f3+1/python/lsst/pex/config/configurableField.py", line 83, in apply |
return self.target(*args, config=self.value, **kw) |
File "/home/hchiang2/lsstsw/stack/Linux64/meas_algorithms/13.0-6-g6f8a914+2/python/lsst/meas/algorithms/loadIndexedReferenceObjects.py", line 50, in __init__ |
dataset_config = butler.get("ref_cat_config", name=self.config.ref_dataset_name, immediate=True) |
File "/home/hchiang2/lsstsw/stack/Linux64/daf_persistence/13.0-5-g87674b4/python/lsst/daf/persistence/butler.py", line 945, in get |
raise NoResults("No locations for get:", datasetType, dataId) |
lsst.daf.persistence.butlerExceptions.NoResults: No locations for get: datasetType:ref_cat_config dataId:DataId(initialdata={'name': 'ps1_pv3_3pi_20170110'}, tag=set([])) |
|
Attachments
Issue Links
- relates to
-
- Done
Activity
The implementation has changed to what we suggested.
Paul Price
added a comment - The implementation has changed to what we suggested. 
Thank you both very much for pointing me to the correct way of using scons!
I pushed the new fix but missed to comment here. Sorry for the confusion. Now test-running on lsstvc and Jenkins before merging it. Also fixed the descriptions.
Hsin-Fang Chiang
added a comment - Thank you both very much for pointing me to the correct way of using scons!
I pushed the new fix but missed to comment here. Sorry for the confusion. Now test-running on lsstvc and Jenkins before merging it. Also fixed the descriptions. Merged after passing Jenkins and the slurm env that I saw this originally.
Hsin-Fang Chiang
added a comment - Merged after passing Jenkins and the slurm env that I saw this originally. 
So now I'm confused. I thought Paul Price agreed with me that the solution proposed here probably isn't right, but then he went ahead and marked it "reviewed" without comment. This seems wrong...!