The output of demo2012 results in an output file which is compared against a benchmarked file. Currently the comparison allows a deviation from the benchmark based "on the the number of digits in the significands used in multiple precision arithmetic"; that number is currently set to 11.
An example using that setting is:
@ Absolute error = 5.9973406400e-1, Relative error = 9.1865836000e-4
##2566 #:25 <== 29.7751550737
##2566 #:25 ==> 29.7478269835
Additionally, the current use returns: a 'Fatal' error if the code itself fails to execute correctly; a "Warning' error if the any of the benchmarked quantities do not meet the comparison criteria; 'Success' if the comparison meets all criteria.
It is noted that the buildbot 'Warning' color indicator is not currently being displayed when the comparison fails. That is a coding error.
This Issue will:
- ensure that BUILDBOT_WARNING(S) causes the correct display color when appropriate.
This Ticket has been split into two parts. This is part 1. Part 2 is