Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3901

Update some tests to support nose and/or py.test

    Details

      Description

      When sconsUtils is migrated to use nose or py.test some test scripts will need to be modified because test discovery will be slightly different and the namespace of test execution will change.

      Two things to consider:

      • People would still like the option of running a test as python tests/testMe.py.
      • We have to work out how to run the memory test case.

        Attachments

          Issue Links

            Activity

            Hide
            rowen Russell Owen added a comment - - edited

            You raise good points about the parent class being run and about skipping tests that may not need to be skipped. It would be worth looking at the latter – if not on this ticket, then filing a new ticket.

            However, I was merely suggesting that the string "afwdata isn't set up" would be better than "Skipping <foo> test case as afwdata isn't set up" because the longer version is subject to copy/paste errors which could result in lying about what is being skipped.

            Show
            rowen Russell Owen added a comment - - edited You raise good points about the parent class being run and about skipping tests that may not need to be skipped. It would be worth looking at the latter – if not on this ticket, then filing a new ticket. However, I was merely suggesting that the string "afwdata isn't set up" would be better than "Skipping <foo> test case as afwdata isn't set up" because the longer version is subject to copy/paste errors which could result in lying about what is being skipped.
            Hide
            tjenness Tim Jenness added a comment -

            Thank you. I have made the suggested changes. I have also worked out how to rework the meas_astrom tests so that they do not trigger the skips in the base class. This involved splitting the base class into two and using multiple inheritance but it seems to work well. These have now been merged. Changes to other packages will now go on separate tickets.

            Show
            tjenness Tim Jenness added a comment - Thank you. I have made the suggested changes. I have also worked out how to rework the meas_astrom tests so that they do not trigger the skips in the base class. This involved splitting the base class into two and using multiple inheritance but it seems to work well. These have now been merged. Changes to other packages will now go on separate tickets.
            Hide
            tjenness Tim Jenness added a comment -

            Following up on the question from Russell Owen about memory test, the simplest option is to add this to each test file:

            class MemoryTest(utilsTests.MemoryTestCase):
                pass
            

            There will inevitably be code duplication whilst we try to retain the ability to use the old style suite() approach as well as supporting standardized unittest test discovery.

            Show
            tjenness Tim Jenness added a comment - Following up on the question from Russell Owen about memory test, the simplest option is to add this to each test file: class MemoryTest(utilsTests.MemoryTestCase): pass There will inevitably be code duplication whilst we try to retain the ability to use the old style suite() approach as well as supporting standardized unittest test discovery.
            Hide
            ktl Kian-Tat Lim added a comment -

            Adding a MemoryTest that can be run at any time during the execution of the tests does not accomplish the goals of the MemoryTestCase, which is to check for memory leakage during execution of (all) other tests. I hadn't fully realized that one result of this ticket is to have every test execute in an independent context. If that's true, then MemoryTestCase needs to be a wrapper around every single test. I'm not sure if unittest, nose, and/or py.test provide a means for specifying such a wrapper.

            Show
            ktl Kian-Tat Lim added a comment - Adding a MemoryTest that can be run at any time during the execution of the tests does not accomplish the goals of the MemoryTestCase , which is to check for memory leakage during execution of (all) other tests. I hadn't fully realized that one result of this ticket is to have every test execute in an independent context. If that's true, then MemoryTestCase needs to be a wrapper around every single test. I'm not sure if unittest , nose , and/or py.test provide a means for specifying such a wrapper.
            Hide
            tjenness Tim Jenness added a comment -

            I think py.test is the opposite. Every test runs within the same process (unlike now where a new python process is spawned per test file). This is why DM-3904 is a problem and why DM-5164 needed a fix to reset the policy. py.test runs all the tests in sequence so the memory test can end up being run multiple times. It does seem to be true that there is no way to guarantee that the memory test will be run as the final test from each test file (when py.test uses unittest for test discovery) but in aggregate the memory test will be run multiple times if each test file includes it.

            Show
            tjenness Tim Jenness added a comment - I think py.test is the opposite. Every test runs within the same process (unlike now where a new python process is spawned per test file). This is why DM-3904 is a problem and why DM-5164 needed a fix to reset the policy. py.test runs all the tests in sequence so the memory test can end up being run multiple times. It does seem to be true that there is no way to guarantee that the memory test will be run as the final test from each test file (when py.test uses unittest for test discovery) but in aggregate the memory test will be run multiple times if each test file includes it.

              People

              • Assignee:
                tjenness Tim Jenness
                Reporter:
                tjenness Tim Jenness
                Reviewers:
                Russell Owen
                Watchers:
                John Parejko, John Swinbank, Kian-Tat Lim, Michael Wood-Vasey, Russell Owen, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel