Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-1895

scons fails if a test has failed, even if it doesn't rerun tests

    Details

    • Type: Bug
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: sconsUtils
    • Labels:
      None
    • Team:
      Architecture

      Description

      If scons decides it does not need to run unit tests it will still claim the build failed if there are any failed tests in tests/.tests.

      To see this try the following:

      • setup afw but not afwData
      • put a fake failed test in afw/tests/.tests (e.g. rename an existing passed test log by adding .failed to the end)
      • scons

      It will say that there is nothing to be done for .tests but proceed to tell you that the build failed because one test failed.

      I think this is important to fix because it is very confusing and could cost developers a lot of time trying to fix tests that are not broken. I certainly wasted some time on it today.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            I agree this is a bug that should be fixed, but I'm not sure I'm the right person to do it - I'd like to get out of the business of being the sconsUtils maintainer in general, and in this case I'm not actually familiar with the code in question, as I wasn't involved in the change to the exit code when there are failed tests.

            Show
            jbosch Jim Bosch added a comment - I agree this is a bug that should be fixed, but I'm not sure I'm the right person to do it - I'd like to get out of the business of being the sconsUtils maintainer in general, and in this case I'm not actually familiar with the code in question, as I wasn't involved in the change to the exit code when there are failed tests.
            Hide
            tjenness Tim Jenness added a comment -

            It sounds like this behavior is pretty entrenched in how we use scons. I'm guessing the "should I run the tests" decision is completely decoupled from the "did any tests fail" search of the directory.

            So you are asking for the .tests directory to be ignored if the tests were not re-run. This relies on DM-2839 being fixed (so that tests are run when source code has changed).

            In some sense scons is correct in claiming that the build has failed. It's only known to have worked if the tests all passed. Is the real bug here that the tests weren't re-run when they should have done?

            Show
            tjenness Tim Jenness added a comment - It sounds like this behavior is pretty entrenched in how we use scons . I'm guessing the "should I run the tests" decision is completely decoupled from the "did any tests fail" search of the directory. So you are asking for the .tests directory to be ignored if the tests were not re-run. This relies on DM-2839 being fixed (so that tests are run when source code has changed). In some sense scons is correct in claiming that the build has failed. It's only known to have worked if the tests all passed. Is the real bug here that the tests weren't re-run when they should have done?

              People

              • Assignee:
                Unassigned
                Reporter:
                rowen Russell Owen
                Watchers:
                Jim Bosch, Russell Owen, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel