Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-1305

Tests fail in shapelet when building on OS X 10.9

    XMLWordPrintable

Details

    Description

      When building the master on pugsley.ncsa.illinois.edu, shapelet builds successfully, but two tests fail:

      pugsley:lsstsw mjuric$ cat build/shapelet/tests/.tests/*.failed
      tests/testMatrixBuilder.py
       
      .F.....
      ======================================================================
      FAIL: testConvolvedCompoundMatrixBuilder (__main__.MatrixBuilderTestCase)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "tests/testMatrixBuilder.py", line 310, in testConvolvedCompoundMatrixBuilder
          self.assertClose(numpy.dot(matrix1D, coefficients), checkVector, rtol=1E-14)
        File "/Users/mjuric/test/lsstsw/stack/DarwinX86/utils/9.2+8/python/lsst/utils/tests.py", line 328, in assertClose
          testCase.assertFalse(failed, msg="\n".join(msg))
      AssertionError: 1/50 elements differ with rtol=1e-14, atol=2.22044604925e-16
      0.175869366369 != 0.175869366369 (diff=1.99840144433e-15/0.175869366369=1.13629876856e-14)
       
      ----------------------------------------------------------------------
      Ran 7 tests in 0.323s
       
      FAILED (failures=1)
      tests/testMultiShapelet.py
       
      ...F...
      ======================================================================
      FAIL: testConvolveGaussians (__main__.MultiShapeletTestCase)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "tests/testMultiShapelet.py", line 88, in testConvolveGaussians
          self.compareMultiShapeletFunctions(msf3a, msf3b)
        File "/Users/mjuric/test/lsstsw/build/shapelet/python/lsst/shapelet/tests.py", line 107, in compareMultiShapeletFunctions
          self.compareShapeletFunctions(sa, sb, rtolEllipse=rtolEllipse, rtolCoeff=rtolCoeff)
        File "/Users/mjuric/test/lsstsw/build/shapelet/python/lsst/shapelet/tests.py", line 86, in compareShapeletFunctions
          rtol=rtolEllipse)
        File "/Users/mjuric/test/lsstsw/stack/DarwinX86/utils/9.2+8/python/lsst/utils/tests.py", line 328, in assertClose
          testCase.assertFalse(failed, msg="\n".join(msg))
      AssertionError: 1/5 elements differ with rtol=1e-14, atol=2.22044604925e-16
      2.44929359829e-16 != 1.13310777953e-15 (diff=8.881784197e-16/1.13310777953e-15=0.783842839795)
       
      ----------------------------------------------------------------------
      Ran 7 tests in 0.131s
       
      FAILED (failures=1)

      ===============

      More info on pugsley.ncsa.illinois.edu:

      pugsley:lsstsw mjuric$ sw_vers
      ProductName: Mac OS X
      ProductVersion: 10.9.5
      BuildVersion: 13F34

      pugsley:lsstsw mjuric$ clang -v
      Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
      Target: x86_64-apple-darwin13.4.0
      Thread model: posix

      ============

      The files are in /Users/mjuric/test/lsstsw/build/shapelet/.

      Attachments

        Issue Links

          Activity

            rowen Russell Owen added a comment -

            Sorry, I missed that change to testMatrixBuilder (it was in a previous commit and I was being sloppy). That test now passes. Unfortunately testMultiShapelet.py still fails and I'm not sure if the failure is significant:

            FAIL: testConvolveGaussians (__main__.MultiShapeletTestCase)
            ----------------------------------------------------------------------
            Traceback (most recent call last):
              File "tests/testMultiShapelet.py", line 87, in testConvolveGaussians
                self.compareMultiShapeletFunctions(msf3a, msf3b)
              File "/Users/rowen/LSST/lsstsw/build/shapelet/python/lsst/shapelet/tests.py", line 107, in compareMultiShapeletFunctions
                self.compareShapeletFunctions(sa, sb, rtolEllipse=rtolEllipse, rtolCoeff=rtolCoeff)
              File "/Users/rowen/LSST/lsstsw/build/shapelet/python/lsst/shapelet/tests.py", line 86, in compareShapeletFunctions
                rtol=rtolEllipse)
              File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/utils/9.2+10/python/lsst/utils/tests.py", line 328, in assertClose
                testCase.assertFalse(failed, msg="\n".join(msg))
            AssertionError: 1/5 elements differ with rtol=1e-13, atol=2.22044604925e-16
            2.44929359829e-16 != 1.13310777953e-15 (diff=8.881784197e-16/1.13310777953e-15=0.783842839795)

            rowen Russell Owen added a comment - Sorry, I missed that change to testMatrixBuilder (it was in a previous commit and I was being sloppy). That test now passes. Unfortunately testMultiShapelet.py still fails and I'm not sure if the failure is significant: FAIL: testConvolveGaussians (__main__.MultiShapeletTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/testMultiShapelet.py", line 87, in testConvolveGaussians self.compareMultiShapeletFunctions(msf3a, msf3b) File "/Users/rowen/LSST/lsstsw/build/shapelet/python/lsst/shapelet/tests.py", line 107, in compareMultiShapeletFunctions self.compareShapeletFunctions(sa, sb, rtolEllipse=rtolEllipse, rtolCoeff=rtolCoeff) File "/Users/rowen/LSST/lsstsw/build/shapelet/python/lsst/shapelet/tests.py", line 86, in compareShapeletFunctions rtol=rtolEllipse) File "/Users/rowen/LSST/lsstsw/stack/DarwinX86/utils/9.2+10/python/lsst/utils/tests.py", line 328, in assertClose testCase.assertFalse(failed, msg="\n".join(msg)) AssertionError: 1/5 elements differ with rtol=1e-13, atol=2.22044604925e-16 2.44929359829e-16 != 1.13310777953e-15 (diff=8.881784197e-16/1.13310777953e-15=0.783842839795)
            jbosch Jim Bosch added a comment -

            It's not significant. Feel free to adjust rtol and atol as needed to get it to pass. In general, I wouldn't consider any failure above 1E-8 a serious problem for any of these tests, but I'd rather not set the tolerances any lower than they need to be in order to protect against future regressions.

            jbosch Jim Bosch added a comment - It's not significant. Feel free to adjust rtol and atol as needed to get it to pass. In general, I wouldn't consider any failure above 1E-8 a serious problem for any of these tests, but I'd rather not set the tolerances any lower than they need to be in order to protect against future regressions.
            jbosch Jim Bosch added a comment -

            Ah, actually it looks like the problem is that we need to set atol here (rtol is useless because the numbers are so close to zero) and there's no obvious code path to do that, since the actual comparison is delegated to some test utility code and it doesn't have an atol argument that's forwarded. I'll be able to return to working on this later this evening, but feel free to look into it yourself earlier if you'd like.

            jbosch Jim Bosch added a comment - Ah, actually it looks like the problem is that we need to set atol here (rtol is useless because the numbers are so close to zero) and there's no obvious code path to do that, since the actual comparison is delegated to some test utility code and it doesn't have an atol argument that's forwarded. I'll be able to return to working on this later this evening, but feel free to look into it yourself earlier if you'd like.
            jbosch Jim Bosch added a comment -

            Added one more commit to this branch with absolute tolerances in the test utility functions. Hopefully it will solve the most recent problem.

            jbosch Jim Bosch added a comment - Added one more commit to this branch with absolute tolerances in the test utility functions. Hopefully it will solve the most recent problem.
            rowen Russell Owen added a comment -

            Looks good (other than a few typos that I fixed). I rebased that fix and merged to master.

            Thank you very much for the fixes.

            rowen Russell Owen added a comment - Looks good (other than a few typos that I fixed). I rebased that fix and merged to master. Thank you very much for the fixes.

            People

              jbosch Jim Bosch
              mjuric Mario Juric
              Russell Owen
              Frossie Economou, Jim Bosch, Mario Juric, Russell Owen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.