Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-647

Add pytest-subtests to our standard packages

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None
    • Location:
      This RFC

      Description

      I propose that we add the pytest-subtests package https://github.com/pytest-dev/pytest-subtests to our standard suite of Python packages. This allows pytest to handle unittest's subTest method correctly: running all versions even if one fails.

      Without this add-on package pytest stops the test at the first failure, even if more subTest cases remain to be run. That is not a disaster, but it does mean that a test failure will not produce as much useful information as it ought to.

      This seems especially apropos given Paul Price's recent work with iterator support in unit tests: https://developer.lsst.io/python/testing.html#decorators-for-iteration

        Attachments

          Issue Links

            Activity

            rowen Russell Owen created issue -
            Hide
            tjenness Tim Jenness added a comment -

            I have been waiting for pytest support of subTests for a long time. I strongly endorse adding this to our conda env.

            Show
            tjenness Tim Jenness added a comment - I have been waiting for pytest support of subTests for a long time. I strongly endorse adding this to our conda env.
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Watchers John Swinbank, Russell Owen, Tim Jenness [ John Swinbank, Russell Owen, Tim Jenness ] Gabriele Comoretto, John Swinbank, Kian-Tat Lim, Russell Owen, Tim Jenness [ Gabriele Comoretto, John Swinbank, Kian-Tat Lim, Russell Owen, Tim Jenness ]
            Hide
            tjenness Tim Jenness added a comment -

            It looks like pytest-subtests is not part of the default conda channel. I assume it's in conda-forge?

            Show
            tjenness Tim Jenness added a comment - It looks like pytest-subtests is not part of the default conda channel. I assume it's in conda-forge?
            Show
            bvan Brian Van Klaveren added a comment - https://anaconda.org/conda-forge/pytest-subtests
            Hide
            krzys Krzysztof Findeisen added a comment -

            A much cleaner solution to the problem than what I had in mind.

            Show
            krzys Krzysztof Findeisen added a comment - A much cleaner solution to the problem than what I had in mind.
            gcomoretto Gabriele Comoretto made changes -
            Remote Link This issue links to "Page (Confluence)" [ 22468 ]
            Hide
            tjenness Tim Jenness added a comment -

            Since this is adding to the conda environment we are flagging this RFC to CCB.

            Show
            tjenness Tim Jenness added a comment - Since this is adding to the conda environment we are flagging this RFC to CCB.
            tjenness Tim Jenness made changes -
            Status Proposed [ 10805 ] Flagged [ 10606 ]
            Hide
            tjenness Tim Jenness added a comment -

            CCB discussed this on 2019-11-20 and have no problem with adding pytest-subtests to the conda env. We recommend deferring this until after the 19.0 release has been completed.

            Show
            tjenness Tim Jenness added a comment - CCB discussed this on 2019-11-20 and have no problem with adding pytest-subtests to the conda env. We recommend deferring this until after the 19.0 release has been completed.
            Hide
            tjenness Tim Jenness added a comment -

            I have used pytest-subtests just now and it works fine for me and shows up as a single "F" in the test summary output but multiple failures reported when the tests complete.

            Show
            tjenness Tim Jenness added a comment - I have used pytest-subtests just now and it works fine for me and shows up as a single "F" in the test summary output but multiple failures reported when the tests complete.
            Hide
            tjenness Tim Jenness added a comment - - edited

            Note that in the interim for local development pip install pytest-subtests works fine and should not cause the rest of your conda installation to go wonky.

            Show
            tjenness Tim Jenness added a comment - - edited Note that in the interim for local development pip install pytest-subtests works fine and should not cause the rest of your conda installation to go wonky.
            gcomoretto Gabriele Comoretto made changes -
            Remote Link This issue links to "Page (Confluence)" [ 22603 ]
            gcomoretto Gabriele Comoretto made changes -
            Status Flagged [ 10606 ] Board Recommended [ 11405 ]
            gcomoretto Gabriele Comoretto made changes -
            Remote Link This issue links to "Page (Confluence)" [ 22658 ]
            gcomoretto Gabriele Comoretto made changes -
            Remote Link This issue links to "Page (Confluence)" [ 22715 ]
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-22640 [ DM-22640 ]
            tjenness Tim Jenness made changes -
            Status Board Recommended [ 11405 ] Adopted [ 10806 ]
            Hide
            tjenness Tim Jenness added a comment -

            Architecture team will add this package to the base system when a convenient time is found. This may happen during the post-19 major conda uprev.

            Show
            tjenness Tim Jenness added a comment - Architecture team will add this package to the base system when a convenient time is found. This may happen during the post-19 major conda uprev.
            tjenness Tim Jenness made changes -
            Link This issue relates to RFC-664 [ RFC-664 ]
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-22817 [ DM-22817 ]
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status Adopted [ 10806 ] Implemented [ 11105 ]

              People

              • Assignee:
                rowen Russell Owen
                Reporter:
                rowen Russell Owen
                Watchers:
                Brian Van Klaveren, Gabriele Comoretto, John Swinbank, Kian-Tat Lim, Krzysztof Findeisen, Russell Owen, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Planned End:

                  Summary Panel