Details
-
Type:
Story
-
Status: To Do
-
Resolution: Unresolved
-
Fix Version/s: None
-
Component/s: ap_pipe
-
Labels:
-
Story Points:2
-
Epic Link:
-
Team:Alert Production
-
Urgent?:No
Description
In digging a bit deeper into DM-35359, John, Ken, and I (re?)discovered that all configs in ap_pipe/pipelines/ApTemplate.yaml (and other "base pipelines" in the same directory) are thrown out by HSC and LsstCamImSim due to DM-34699 and DM-31063.
What this means is, for the time being, if you want to change a pipeline configuration, you need to either add it to ap_pipe/config/$CAMERA/whateverTask.py or you need to add it to the camera-specific pipeline. This is a temporary state of affairs that nevertheless breaks how "base pipelines" are supposed to work, as a place to set default pipeline configs.
This ticket is to remedy the situation. I believe this involves the following steps:
(1) Audit the configs in ap_pipe/config, most notably the ones in ap_pipe/config/$CAMERA/whateverTask.py, to decide which ones should be task-level configs vs. AP pipeline specific configs (vs thrown out entirely)
(2) Move configs to their appropriate new home
(3) Remove the entire ap_pipe/config directory (I think?)
Then configs can only be set in two places: task defaults or pipelines themselves, neither of which overwrites the other. (Or, you know, at runtime on the command line, when living dangerously.)
Attachments
Issue Links
- is blocked by
-
- To Do
Activity
Great point Krzysztof Findeisen, thank you. Does a ticket exist for moving DRP configs from obs packages into drp_pipe? If I'm fully understanding the situation, it sounds like it would be simplest to set that as a blocker for this ticket. The idea being that only what I would call "true camera-required configs" remain in the obs packages.
Thanks, a blocker sounds perfect. I think the ticket in question is DM-31047.
Krzysztof Findeisen
added a comment - Thanks, a blocker sounds perfect. I think the ticket in question is DM-31047 .
Note that the ap_pipe/config configs can't be removed until the obs package configs are gone, as that would cause the obs configs to be applied again. I was never able to find a way to suppress the obs configs without also taking out the base pipeline ones.
As for where things are configured, the motivation for the current arrangement was that it would make it easier to synchronize any changes to the obs package configs (which are still being used as the central config location by the DRP team) when the time came to remove those. If you're planning to merge the ap_pipe/config files into the pipelines, do you have an alternative way to identify "shared" changes that should have been part of AP? I'm worried that if we don't sync anything, whoever is tasked with removing the obs configs will mistake configs that should be task-level for ones that are DRP-specific.