Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11792

Investigate loadscope test distribution option

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: pipe_tasks, sconsUtils
    • Labels:
      None
    • Templates:
    • Story Points:
      1
    • Team:
      Architecture

      Description

      Some tests have a high setup overhead which makes them incompatible with distributed testing. An example of this is the tests in pipe_tasks that create 200MB of test data and then check it. If xdist is used 200MB of data are created per process and this makes the tests significantly slower and can cause resource exhaustion.

      The ideal example would be to use a fixture to mark a class to be fixed to a node. There is an open issue on that on GitHub but for now the developers recommend running with --dist=loadscope to ensure that all classes for all tests are pinned to their own nodes.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  tjenness Tim Jenness
                  Reporter:
                  tjenness Tim Jenness
                  Watchers:
                  Russell Owen, Tim Jenness
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:

                    Summary Panel