Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15801

Detect loops in QuantumGraph

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_supertask
    • Labels:
      None

      Description

      It looks like in current implementation of graph builder it is possible to make graphs containing loops. Simplest example that was observed already is when pipeline of a single task reads and writes into the same dataset type. We are already supposed to have loop detection when we build Pipeline, maybe logic there is not robust enough and needs a small fix.

        Attachments

        1. graph.png
          32 kB
          Andy Salnikov

          Activity

          Hide
          salnikov Andy Salnikov added a comment -

          Added rendering of a graph with a loop in it:

          Show
          salnikov Andy Salnikov added a comment - Added rendering of a graph with a loop in it:
          Hide
          salnikov Andy Salnikov added a comment -

          In current implementation loops are detected when we do pipeline ordering, and it was only done when --order-pipeline option was used. Trivial fix for this is to always do pipeline ordering, but only use the result if option above is used. Verified with the trivial task that it's working as expected now.

          Show
          salnikov Andy Salnikov added a comment - In current implementation loops are detected when we do pipeline ordering, and it was only done when --order-pipeline option was used. Trivial fix for this is to always do pipeline ordering, but only use the result if option above is used. Verified with the trivial task that it's working as expected now.
          Hide
          salnikov Andy Salnikov added a comment -

          Nate, could you review this trivial fix. This is really a one-line fix in pipelineBuilder.py, the rest of the changes are some small improvements/fixes not directly related to the problem.

          This update should now cause error for the situation like you had before in your task configuration, you can probably test it with your task too. 

          Show
          salnikov Andy Salnikov added a comment - Nate, could you review this trivial fix. This is really a one-line fix in pipelineBuilder.py , the rest of the changes are some small improvements/fixes not directly related to the problem. This update should now cause error for the situation like you had before in your task configuration, you can probably test it with your task too. 
          Hide
          nlust Nate Lust added a comment -

          Small comment but looks good

          Show
          nlust Nate Lust added a comment - Small comment but looks good
          Hide
          salnikov Andy Salnikov added a comment -

          Thanks for review! Merged and done.

          Show
          salnikov Andy Salnikov added a comment - Thanks for review! Merged and done.

            People

            Assignee:
            salnikov Andy Salnikov
            Reporter:
            salnikov Andy Salnikov
            Reviewers:
            Nate Lust
            Watchers:
            Andy Salnikov, Nate Lust
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                CI Builds

                No builds found.