Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Story Points:2
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
As encountered in PREOPS-707, the quantum graph generation for DC2 tract=3828^3829 step4's writeForcedSourceTable failed:
File "/opt/lsst/software/stack_w_2021_40/stack/miniconda3-py38_4.9.2-0.7.0/Linux64/pipe_base/22.0.1-25-g96539fe+64e601228d/python/lsst
|
/pipe/base/graph/_implDetails.py", line 78, in addProducer
|
raise ValueError(f"Only one node is allowed to produce {key}, "
|
ValueError: Only one node is allowed to produce forcedSource@{instrument: 'LSSTCam-imSim', detector: 172, visit: 266117, ...}, sc=DataFrame], the current producer is QuantumNode(quantum=Quantum(taskName=lsst.pipe.tasks.postprocess.WriteForcedSourceTableTask, dataId={instrument: 'LSSTCam-imSim', skymap: 'DC2', detector: 172, tract: 3828, visit: 266117, ...}), taskDef=TaskDef(WriteForcedSourceTableTask, label=writeForcedSourceTable), nodeId=NodeId(number=59960, buildId='1633714767.4772997-29875'))
|
To reproduce the error on data-int.lsst.cloud RSP:
pipetask qgraph -b s3://butler-us-central1-panda-dev/dc2/butler-external.yaml -p "${OBS_LSST_DIR}/pipelines/imsim/DRP.yaml#step4" -i 2.2i/runs/test-med-1/w_2021_40/PREOPS-707 -d "instrument='LSSTCam-imSim' and skymap='DC2'"
|
It appears to happen when forced_src data of a detector exist for more than one tract.
Attachments
Activity
Hey Nate Lust, think you'll have time to get to this on Monday?
I am aiming to merge before w_2021_44 gets tagged next Wednesday.
Nate, would you have time to review this?
Sending to you because it relates to your earlier choice of making forcedPhotCcdTask ultimately run per-detector per-tract. The first transform step of merging the forcedPhotCcd and forcedPhotDiff output catalogs are now per-tract as well. Downstream processing is happy with this.
Jenkins passed here:
https://ci.lsst.codes/job/stack-os-matrix/35196/display/redirect
And I'm running a full step4 and step5 with the 2 ticket branches that affect those steps (this one and
DM-32238) here:/project/yusra/
DM-32238/submit/u/yusra/DC2/DM-32238-32124