Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ap_pipe
-
Story Points:6
-
Epic Link:
-
Sprint:AP F20-5 (October), AP F20-6 (November)
-
Team:Alert Production
-
Urgent?:No
Description
make_apdb.py is a non-Task command-line script. To make it easy to keep database configuration consistent between it and ap_pipe.py, make_apdb.py takes exactly the same config arguments as ap_pipe.py. In Gen 2, it does this by internally reusing ApPipeConfig and two custom argparse.Action classes from CmdLineTask. All of these classes will be removed along with Gen 2.
Develop a new make_apdb.py that matches the interface used to configure the AP Pipeline (or at least the APDB) with the pipetask activator:
pipetask run -c label:config_var.sub_config=6
|
pipetask run --config label:config_var.sub_config=6
|
pipetask run -C label:path
|
pipetask run --config-file label:path
|
pipetask run --pipeline path # May configure any component, including APDB
|
It may be useful to wait until DM-21939 is done, so that we can see exactly what the new ap_pipe.py interface will look like. The current make_apdb.py will work with the Gen 3 pipeline for as long as CmdLineTask is also supported.
Unfortunately, the Gen 3 equivalents of ConfigFileAction and ConfigValueAction do not do any config processing, which is instead tightly coupled to the activator code itself. So the Gen 3 version will have to find a different way to give users a consistent UI.
Attachments
Issue Links
- is blocked by
-
- Done
- relates to
-
DM-27461 Unclear error messages in ConfigOverrides
- To Do
-
- To Do
-
- Done
-
DM-24436 Use obs and camera overrides in make_apdb.py
- Won't Fix
- mentioned in
-
Page Loading...
Activity
Andy Salnikov mentioned on DM-24584 that pipetask's UI may be changing to favor a purely file-based configuration. Once we're ready to work on this, update the requirements as appropriate.
Krzysztof Findeisen
added a comment - Andy Salnikov mentioned on DM-24584 that pipetask 's UI may be changing to favor a purely file-based configuration. Once we're ready to work on this, update the requirements as appropriate. This ticket ended up changing in spec a lot. After discussion on #dm-alert-prod, I dropped the requirement that the UI for make_apdb.py look like the one for pipetask run. That, in turn, meant there was little point in creating a separate Gen 3 script. Instead, I've altered the existing make_apdb.py to use ApdbConfig directly (a breaking change for anybody calling it) in both Gen 2 and Gen 3.
Meredith Rawls, I suggest looking at the ap_pipe changes commit by commit, since one of the larger commits is a straight copy of existing code from pipe_base.
Krzysztof Findeisen
added a comment - - edited This ticket ended up changing in spec a lot. After discussion on #dm-alert-prod , I dropped the requirement that the UI for make_apdb.py look like the one for pipetask run . That, in turn, meant there was little point in creating a separate Gen 3 script. Instead, I've altered the existing make_apdb.py to use ApdbConfig directly (a breaking change for anybody calling it) in both Gen 2 and Gen 3.
Meredith Rawls , I suggest looking at the ap_pipe changes commit by commit, since one of the larger commits is a straight copy of existing code from pipe_base . Thanks for the clear descriptions here, straightforward implementation, and updated docs. Please see my comments on GitHub.
Back In Progress, since Meredith Rawls pointed out that part of this is a problem the Gen 3 people have already solved, and I should use their solution.
Krzysztof Findeisen
added a comment - Back In Progress, since Meredith Rawls pointed out that part of this is a problem the Gen 3 people have already solved, and I should use their solution. Sorry Meredith Rawls, but could you re-review? Other than the corrections you requested, I think the only changes I made were to make_apdb.py (in particular, commit 057d803).
Krzysztof Findeisen
added a comment - - edited Sorry Meredith Rawls , but could you re-review? Other than the corrections you requested, I think the only changes I made were to make_apdb.py (in particular, commit 057d803 ). I think this is good to go now, assuming Jenkins agrees. Thanks for your work on it!
The new make_apdb.py script should support obs package overrides for the APDB, possibly through apPipe.py or diaPipe.py, as originally proposed on
DM-24436.