# Test failures should cause a build failure

XMLWordPrintable

## Details

• Type: Bug
• Status: Invalid
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Team:
SQuaRE

## Description

There are currently no red flashing lights when a test fails under scons — just a "failed" printed to the screen, which is easy to miss, especially when building multithreaded. A test failure should cause the entire scons build to fail, the same way a compile failure causes the build to fail.

## Activity

Hide
Robyn Allsman [X] (Inactive) added a comment -

+1

Makes it difficult for buildbot to catch the error, too, since deduction and reporting happens so removed from the actual test failure.

Show
Robyn Allsman [X] (Inactive) added a comment - +1 Makes it difficult for buildbot to catch the error, too, since deduction and reporting happens so removed from the actual test failure.
Hide
Tim Jenness added a comment -

I'm guessing this was fixed at some point given that builds now fail if tests fail. Just in case this ticket relates to a subtle difference between eups distrib install and running scons manually I forced a test to fail and scons did terminate:

 scons: done reading SConscript files. scons: Building targets ... scons: Nothing to be done for python'. running tests/cameraMapper.py... failed running tests/outputRoot.py... passed running tests/testDM-329.py... passed buildConfig(["doc/doxygen.conf"], ["doc/doxygen.conf.in"]) doxygen /Users/timj/work/lsstsw/src/daf_butlerUtils/doc/doxygen.conf 1 tests failed scons: *** [checkTestStatus] Error 1 scons: building terminated because of errors. 

It's a bit strange that it tries the doxygen step regardless but that does look like the test failure triggers bad exit status eventually.

Show
Tim Jenness added a comment - I'm guessing this was fixed at some point given that builds now fail if tests fail. Just in case this ticket relates to a subtle difference between eups distrib install and running scons manually I forced a test to fail and scons did terminate: scons: done reading SConscript files. scons: Building targets ... scons: Nothing to be done for python'. running tests/cameraMapper.py... failed running tests/outputRoot.py... passed running tests/testDM-329.py... passed buildConfig(["doc/doxygen.conf"], ["doc/doxygen.conf.in"]) doxygen /Users/timj/work/lsstsw/src/daf_butlerUtils/doc/doxygen.conf 1 tests failed scons: *** [checkTestStatus] Error 1 scons: building terminated because of errors. It's a bit strange that it tries the doxygen step regardless but that does look like the test failure triggers bad exit status eventually.
Hide
Kian-Tat Lim added a comment - - edited

As linked above, this was a duplicate of DM-1027. Not sure if Won't Fix was available at that time. Now marked "Invalid" as a duplicate.

Show
Kian-Tat Lim added a comment - - edited As linked above, this was a duplicate of DM-1027 . Not sure if Won't Fix was available at that time. Now marked "Invalid" as a duplicate.

## People

• Assignee:
Jim Bosch
Reporter:
Paul Price
Watchers:
Frossie Economou, Kian-Tat Lim, Mario Juric, Paul Price, Robert Lupton, Robyn Allsman [X] (Inactive), Tim Jenness