Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-793

Allow pytest-only features in Python unit tests

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Our Python unit tests are executed via the pytest utility in all circumstances except manual execution during development.  In that last case, we still allow python tests/test_foo.py via the underlying unittest infrastructure.

      pytest has many features that extend testing capabilities beyond what unittest provides.  One example that has come up recently (in DM-31141) is parametrizing tests; pytest provides facilities for doing this, while we have had to invent our own ways of doing so on top of unittest.

      pytest is always a part of our development environment, as it is included in rubin-env.

      Accordingly, I propose that we allow the use of pytest-only features in Python unit tests, which will break the usage of python tests/test_foo.py, forcing the use of pytest tests/test_foo.py instead.  python -m pdb tests/test_foo.py can be replaced by pytest --pdb tests/test_foo.py (to debug on failure) or pytest --trace tests/test_foo.py (to debug at the start of each test).

      This is not a mandate to use pytest style or to remove unittest from our tests. We are not changing our other unittest-based extensions such as open-file-leak or specialized assertions at this time. Most developers should not need to change what they are doing at all. But where appropriate pytest functionality exists to extend our typical usage, this proposal would allow it.

        Attachments

          Issue Links

            Activity

            Hide
            jhoblitt Joshua Hoblitt added a comment -

            +1

            Show
            jhoblitt Joshua Hoblitt added a comment - +1
            Hide
            krzys Krzysztof Findeisen added a comment -

            Show
            krzys Krzysztof Findeisen added a comment -
            Hide
            Parejkoj John Parejko added a comment -

            +1

            An few additional comments on pytest options I've found particularly useful, incase there is anyone else still using python test.py instead of pytest test.py:

            • install pdbpp to get an ipython-like interface for the pytest --pdb command (and I'd suggest we could add that to our conda env).
            • use pytest -k name test.py to only run tests that match the "name" pattern. I've found this to be much easier and more flexible than specifying individual tests via ClassName.testName in the unittest style.
            • running pytest with -sv will get extended output during the run (normally pytest grabs all of stdout).
            • running with -r will get a summary of successes and failures at the end that can be useful when you have many failing tests.
            • run with -n X to run your tests with X processes to speed things up (like scons -jX.
            Show
            Parejkoj John Parejko added a comment - +1 An few additional comments on pytest options I've found particularly useful, incase there is anyone else still using python test.py instead of pytest test.py : install pdbpp to get an ipython-like interface for the pytest --pdb command (and I'd suggest we could add that to our conda env). use pytest -k name test.py to only run tests that match the "name" pattern. I've found this to be much easier and more flexible than specifying individual tests via ClassName.testName in the unittest style. running pytest with -sv will get extended output during the run (normally pytest grabs all of stdout). running with -r will get a summary of successes and failures at the end that can be useful when you have many failing tests. run with -n X to run your tests with X processes to speed things up (like scons -jX .
            Hide
            rowen Russell Owen added a comment -

            +1. In addition to the many other advantages listed, this would allow us to use bare assert instead of the plethora of assertFoo methods, which I think leads to cleaner looking unit tests.

            I am a bit concerned about the dichotomy that will arise between tests written in old style and new style (especially test classes vs free functions) but I think we just live with it.

            Show
            rowen Russell Owen added a comment - +1. In addition to the many other advantages listed, this would allow us to use bare assert instead of the plethora of assertFoo methods, which I think leads to cleaner looking unit tests. I am a bit concerned about the dichotomy that will arise between tests written in old style and new style (especially test classes vs free functions) but I think we just live with it.
            Hide
            lguy Leanne Guy added a comment -

            +1

            Show
            lguy Leanne Guy added a comment - +1
            Hide
            ktl Kian-Tat Lim added a comment -

            Looks like there are no objections. Adopting.

            Show
            ktl Kian-Tat Lim added a comment - Looks like there are no objections. Adopting.

              People

              Assignee:
              ktl Kian-Tat Lim
              Reporter:
              ktl Kian-Tat Lim
              Watchers:
              John Parejko, Joshua Hoblitt, Kian-Tat Lim, Krzysztof Findeisen, Leanne Guy, Russell Owen, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.