Details
-
Type:
Bug
-
Status: Won't Fix
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ctrl_mpexec, pipe_base
-
Labels:
-
Team:Data Release Production
-
Urgent?:No
Description
When trying to run graph visualization on the pipeline in ap_verify_ci_cosmos_pdr2:*
pipetask build --config diaPipe:apdb.db_url="sqlite:///foo/apdb.db" --pipeline-dot temp.dot -p pipelines/ApVerify.yaml
|
dot temp.dot -Tpng > ApVerify.png
|
I found that many of the metrics are displayed as disconnected pieces of the pipeline graph (see attached image). Specifically, datasets like imageDifference_metadata are shown as preexisting rather than as pipeline outputs.
The desired behavior would be for metadata (and presumably config) datasets to be displayed as outputs of the appropriate task when they are used as inputs of another task. Presumably, they should still be omitted when they are not used by the pipeline.
*This example cannot be reproduced with the "generic" pipelines in ap_verify because of a configuration bug (DM-32726).
A similar issue that I'm not sure should be a separate bug report: components are also treated as preexisting datasets; for example, calexp.bbox is unrelated to calexp. Desired behavior would be for the main dataset to be presented as the "source" (in some sense) of its components.
EDIT: this has since been re-reported as
DM-34811.