Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14019

Allow ap_pipe to skip association

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ap_pipe
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      AP S19-2
    • Team:
      Alert Production

      Description

      Once ap_association supports it, ap_pipe should query the association database to see if a dataId has already been processed. This will allow ap_pipe to skip all processing steps that have already been performed.

        Attachments

          Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-22120 ap_verify scales poorly to large runs

          • Done

          Improvement - An improvement or enhancement to an existing feature or task. DM-17521 Add warning when ap_pipe skips association

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment -

            Hi Chris Morrison [X], could you please review this 70-line feature? Thanks!

            Show
            krzys Krzysztof Findeisen added a comment - Hi Chris Morrison [X] , could you please review this 70-line feature? Thanks!
            Hide
            Permalink
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Looks good. I've approved the pull requests. One comment that is worth making outside of those is that some warning should be communicated for allowing association to be re-run is that the association and Ppdb are temporally sensitive. That is if you lose a ccd say in processing and come back and re-do it, the resultant associations and Ppdb will not be the same. There may even be some weirdness with the DiaObjects themselves based on how ValidityStart and ValidityEnd are used in the Ppdb.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Looks good. I've approved the pull requests. One comment that is worth making outside of those is that some warning should be communicated for allowing association to be re-run is that the association and Ppdb are temporally sensitive. That is if you lose a ccd say in processing and come back and re-do it, the resultant associations and Ppdb will not be the same. There may even be some weirdness with the DiaObjects themselves based on how ValidityStart and ValidityEnd are used in the Ppdb.

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Chris Morrison [X] (Inactive)
              Watchers:
              Chris Morrison [X] (Inactive), Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.