Some tests have a high setup overhead which makes them incompatible with distributed testing. An example of this is the tests in pipe_tasks that create 200MB of test data and then check it. If xdist is used 200MB of data are created per process and this makes the tests significantly slower and can cause resource exhaustion.
The ideal example would be to use a fixture to mark a class to be fixed to a node. There is an open issue on that on GitHub but for now the developers recommend running with --dist=loadscope to ensure that all classes for all tests are pinned to their own nodes.