Following a discussion on community, I am proposing we switch test execution in sconsUtils to use pytest rather than python. A proposed implementation can be found on
- All (see next bullet) tests will be executed in a single pytest process. This leads to significant gains in startup performance.
- sconsUtils can be configured to run specific test files in a standalone pytest process.
- The SConscript file can be configured to use automatic test discovery within pytest rather than giving an explicit list of files by globbing the tests directory.
- The output from pytest will be written to standard out (and therefore appear in _build.log and on the terminal when running scons from the command line) but I will also try to write the output to tests/.tests as now.
- pytest will generate one or more JUnit format XML files to allow Jenkins to display a much more detailed testing report (including test run times).
With this enabled we can also enable flake8 testing of all code as part of standard testing. I have tested this in meas_base and shapelet and can release it as soon as
DM-11514 is merged.
I am putting a planned end of Friday with the assumption that we work on minor issues during the week. In particular we need to sort out
DM-8688 so that meas_algorithms will work), and others may be discovered.