Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-587

Non-deprecated breaking change to AP pipeline workflow

    XMLWordPrintable

Details

    • RFC
    • Status: Implemented
    • Resolution: Done
    • DM
    • None

    Description

      Currently, ap_pipe.py tries to set up a prompt products database when it is run. This behavior is fine for Alert Production runs against an SQLite database on a single-node machine, but is expected to cause problems when moving to either a more formal (e.g., centrally managed) database system or a more complex computing environment. Therefore, DM-16606 adds a new script, make_ppdb.py, that creates a database for running ap_pipe.py in development environments. The intent is that, with an external script, ap_pipe.py can simply assume the database exists without worrying about whether it should (or can) set it up itself.

      The database creation script has been tested and reviewed, but removing the old code from ApPipeTask is a breaking change – users who previously could run ap_pipe.py with no preparation must ensure there's an available prompt products database*. We could have a deprecation period in which users are encouraged to call make_ppdb.py but ap_pipe.py also tries to set up the database (safe to try for SQLite), but that would limit our options for deploying the AP pipeline until the old code is finally removed.

      Since I understand that ap_pipe.py is starting to be used by other LSST developers, but hope the number of affected users is still low, I propose to not have a deprecation period and make ap_pipe.py require a pre-existing database as soon as DM-16606 is merged. Is there anyone who would be seriously affected by such a sudden change?

      [*Users of ap_verify.py don't need to do anything, only those who call ap_pipe.py directly.]

      Attachments

        Issue Links

          Activity

            ktl Kian-Tat Lim added a comment -

            I'm OK with this; I don't think the original code is widely used, and the new flow seems better for the long term.

            ktl Kian-Tat Lim added a comment - I'm OK with this; I don't think the original code is widely used, and the new flow seems better for the long term.

            rearmstr & rbiswas — we identified you as the two most likely people to be impacted by this RFC. If you don't complain in the next day or so, we indent to adopt it (with DM-CCB approval), so please shout now if that's a problem for you.

            swinbank John Swinbank added a comment - rearmstr & rbiswas — we identified you as the two most likely people to be impacted by this RFC. If you don't complain in the next day or so, we indent to adopt it (with DM-CCB approval), so please shout now if that's a problem for you.

            We are not currently using the ap_pipe implementation so this shouldn't affect us.

            rearmstr Bob Armstrong added a comment - We are not currently using the ap_pipe implementation so this shouldn't affect us.

            Per discussion at DMCCB#9, I have set this to “board recommended”. krzys, that means you can go ahead and adopt/implement this RFC at your leisure.

            swinbank John Swinbank added a comment - Per discussion at DMCCB#9 , I have set this to “board recommended”. krzys , that means you can go ahead and adopt/implement this RFC at your leisure.

            FYI, "Manager Approves" is not a very intuitive button for moving from "Board Recommended" to "Adopted"...

            krzys Krzysztof Findeisen added a comment - FYI, "Manager Approves" is not a very intuitive button for moving from "Board Recommended" to "Adopted"...

            People

              krzys Krzysztof Findeisen
              krzys Krzysztof Findeisen
              Bob Armstrong, John Parejko, John Swinbank, Kian-Tat Lim, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                Jenkins

                  No builds found.