Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Labels:
-
Team:Alert Production
-
Urgent?:No
Description
RFC-775 laid out most of a proposal for organizing pipeline YAML files in packages, but left some details the implementation. This included a goal of clearly separating pipelines that should be executed (and for the most part, not imported into other pipelines) from "ingredient" pipelines that are the opposite: they should not be executed, only imported.
In the implementation of drp_pipe, we walked back one aspect of the RFC-775 proposal, by leaving the "runnable" pipelines as regular YAML files, rather than some sort of expanded/resolved variant thereof, as the latter didn't seem to be a sound concept in end. But we did settle on putting the "don't execute, only import" pipeline YAML files in an "ingredients" directory with (when appropriate) some of the same per-instrument structure as the pipelines directory.
For consistency, we should make the same change to ap_pipe as well. That may just be a matter of moving pipelines that are not associated with instruments to an "ingredients" directory, but an ap_pipe expert should check that guess.
Attachments
Issue Links
- is triggered by
-
RFC-927 Clarify pipeline definition YAML directory structures
- Implemented
- relates to
-
DM-39333 Move ingredient pipeline definitions in drp_pipe to the pipelines directory
- Done
-
DM-39465 Standardize pipelines README files following RFC-927
- Done
-
DM-36796 Rename config/pipeline directories in ap_pipe and ap_verify
- Invalid
-
RFC-775 Reorganize pipelines and packages at the top of the Science Pipelines codebase
- Adopted