Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-27117

Add dummy pipeline to ap_verify testing

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ap_verify
    • Labels:
      None
    • Story Points:
      10
    • Epic Link:
    • Sprint:
      AP S22-3 (February)
    • Team:
      Alert Production

      Description

      Mock-based testing of pipeline_driver.py has run into several problems, including over-dependence on ctrl_mpexec. It would be cleaner to create a mock pipeline whose tasks do no processing besides creating output datasets. Implement such a pipeline and use it in tests that currently try to mock the pipeline, its executor, or pipeline_driver.

      Note that ap_verify currently does some run-time configuration of the AP pipeline (e.g., database location), so the mock pipeline needs to have at least some of the same configuration hooks as the real pipeline. Also, since tasks must be importable to be included in a pipeline, they will need to go somewhere in the main python directory of ap_verify.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33734 Investigate non-processing AP pipeline overhead

          • Done
          has to be done after

          Improvement - An improvement or enhancement to an existing feature or task. DM-26140 Centralize Gen 3 pipeline configuration info for ap_verify datasets

          • Done
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-30758 Add mock DRP pipeline to pipe_base

          • To Do

          Bug - A problem which impairs or prevents the functions of the product. DM-27099 Re-bridge ap_verify and ctrl_mpexec

          • Done

          Improvement - An improvement or enhancement to an existing feature or task. DM-26239 Create Python API for pipetask

          • To Do

            Activity

            Hide
            Permalink
            salnikov Andy Salnikov added a comment -

            Looks good, few minor comments for ap_verify.

            Show
            salnikov Andy Salnikov added a comment - Looks good, few minor comments for ap_verify.

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Andy Salnikov
              Watchers:
              Andy Salnikov, Kian-Tat Lim, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.