Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-370

Use pytest for test execution in sconsUtils

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Following a discussion on community, I am proposing we switch test execution in sconsUtils to use pytest rather than python. A proposed implementation can be found on DM-11514.

      • All (see next bullet) tests will be executed in a single pytest process. This leads to significant gains in startup performance.
      • sconsUtils can be configured to run specific test files in a standalone pytest process.
      • The SConscript file can be configured to use automatic test discovery within pytest rather than giving an explicit list of files by globbing the tests directory.
      • The output from pytest will be written to standard out (and therefore appear in _build.log and on the terminal when running scons from the command line) but I will also try to write the output to tests/.tests as now.
      • pytest will generate one or more JUnit format XML files to allow Jenkins to display a much more detailed testing report (including test run times).

      With this enabled we can also enable flake8 testing of all code as part of standard testing. I have tested this in meas_base and shapelet and can release it as soon as DM-11514 is merged.

      I am putting a planned end of Friday with the assumption that we work on minor issues during the week. In particular we need to sort out DM-8688 so that meas_algorithms will work), and others may be discovered.

        Attachments

          Issue Links

            Activity

            No builds found.
            tjenness Tim Jenness created issue -
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Link This issue is triggering DM-11514 [ DM-11514 ]
            tjenness Tim Jenness made changes -
            Remote Link This issue links to "Page (Confluence)" [ 15342 ]
            Hide
            frossie Frossie Economou added a comment -

            +1 We look forward to having junit test reports to integrate into Jenkins.

            Show
            frossie Frossie Economou added a comment - +1 We look forward to having junit test reports to integrate into Jenkins.
            tjenness Tim Jenness made changes -
            Link This issue relates to RFC-229 [ RFC-229 ]
            Hide
            tjenness Tim Jenness added a comment -

            Solid progress has been made on this. I have not heard any negative responses to the work so I will adopt the RFC tomorrow. The change to use xdist for multi-process testing has lead to quite a bit of additional work fixing up tests to use temporary directories and filenames rather than fixed locations. I won't merge DM-11514 until I can build lsst_ci and lsst_sims.

            Show
            tjenness Tim Jenness added a comment - Solid progress has been made on this. I have not heard any negative responses to the work so I will adopt the RFC tomorrow. The change to use xdist for multi-process testing has lead to quite a bit of additional work fixing up tests to use temporary directories and filenames rather than fixed locations. I won't merge DM-11514 until I can build lsst_ci and lsst_sims .
            Hide
            tjenness Tim Jenness added a comment -

            The pytest migration plan has been adopted. I'll merge DM-11514 as soon as I get clean builds of everything. The blocker tickets can be found on DM-11514.

            Show
            tjenness Tim Jenness added a comment - The pytest migration plan has been adopted. I'll merge DM-11514 as soon as I get clean builds of everything. The blocker tickets can be found on DM-11514 .
            tjenness Tim Jenness made changes -
            Status Proposed [ 10805 ] Adopted [ 10806 ]
            tjenness Tim Jenness made changes -
            Remote Link This issue links to "Community discussion (Web Link)" [ 15349 ]
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status Adopted [ 10806 ] Implemented [ 11105 ]

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Watchers:
              Frossie Economou, John Swinbank, Jonathan Sick, Joshua Hoblitt, Robert Lupton, Russell Owen, Scott Daniel, Tim Jenness
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  CI Builds

                  No builds found.