Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5822

Afw fails unit test for convolve depending on compiler optimisation level

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      DRP X16-2, DRP X16-3, DRP F16-1
    • Team:
      Data Release Production

      Description

      On OSX 10.11.4 with Apple LLVM version 7.3.0 (clang-703.0.29) afw fails test/convolve.py with the following error when either -O0 or -O1 is enabled but works fine for -O2 and -O3.

      tests/convolve.py
       
      .....FF/Users/pschella/Development/lsst/code/afw/python/lsst/afw/image/testUtils.py:283: RuntimeWarning: invalid value encountered in isnan
        nan0 = np.isnan(filledArr0)
      /Users/pschella/Development/lsst/lsstsw/miniconda/lib/python2.7/site-packages/numpy/lib/ufunclike.py:113: RuntimeWarning: invalid value encountered in isinf
        nx.logical_and(nx.isinf(x), ~nx.signbit(x), y)
      /Users/pschella/Development/lsst/lsstsw/miniconda/lib/python2.7/site-packages/numpy/lib/ufunclike.py:176: RuntimeWarning: invalid value encountered in isinf
        nx.logical_and(nx.isinf(x), nx.signbit(x), y)
      F.F...
      ======================================================================
      FAIL: testSpatiallyVaryingAnalyticConvolve (__main__.ConvolveTestCase)
      Test in-place convolution with a spatially varying AnalyticKernel
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "tests/convolve.py", line 437, in testSpatiallyVaryingAnalyticConvolve
          rtol = rtol)
        File "tests/convolve.py", line 290, in runStdTest
          self.runBasicConvolveEdgeTest(kernel, kernelDescr)
        File "tests/convolve.py", line 317, in runBasicConvolveEdgeTest
          doVariance = True, rtol=0, atol=0, msg=msg)
        File "/Users/pschella/Development/lsst/code/afw/python/lsst/afw/image/testUtils.py", line 201, in assertMaskedImagesNearlyEqual
          testCase.fail("%s: %s" % (msg, "; ".join(errStrList)))
      AssertionError: basicConvolve(MaskedImage, kernel=Spatially Varying Gaussian Analytic Kernel using brute force) wrote to edge pixels: image planes differ: maxDiff=1.09176e+38 at position (73, 18); value=-1.09176e+38 vs. 2825.0; NaNs differ
       
      ======================================================================
      FAIL: testSpatiallyVaryingDeltaFunctionLinearCombination (__main__.ConvolveTestCase)
      Test convolution with a spatially varying LinearCombinationKernel of delta function basis kernels.
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "tests/convolve.py", line 556, in testSpatiallyVaryingDeltaFunctionLinearCombination
          rtol = rtol)
        File "tests/convolve.py", line 290, in runStdTest
          self.runBasicConvolveEdgeTest(kernel, kernelDescr)
        File "tests/convolve.py", line 317, in runBasicConvolveEdgeTest
          doVariance = True, rtol=0, atol=0, msg=msg)
        File "/Users/pschella/Development/lsst/code/afw/python/lsst/afw/image/testUtils.py", line 201, in assertMaskedImagesNearlyEqual
          testCase.fail("%s: %s" % (msg, "; ".join(errStrList)))
      AssertionError: basicConvolve(MaskedImage, kernel=Spatially varying LinearCombinationKernel of delta function kernels using brute force) wrote to edge pixels: image planes differ: maxDiff=9.06659e+36 at position (75, 29); value=9.06659e+36 vs. 2865.0
       
      ======================================================================
      FAIL: testSpatiallyVaryingGaussianLinerCombination (__main__.ConvolveTestCase)
      Test convolution with a spatially varying LinearCombinationKernel of two Gaussian basis kernels.
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "tests/convolve.py", line 523, in testSpatiallyVaryingGaussianLinerCombination
          rtol = rtol)
        File "tests/convolve.py", line 290, in runStdTest
          self.runBasicConvolveEdgeTest(kernel, kernelDescr)
        File "tests/convolve.py", line 317, in runBasicConvolveEdgeTest
          doVariance = True, rtol=0, atol=0, msg=msg)
        File "/Users/pschella/Development/lsst/code/afw/python/lsst/afw/image/testUtils.py", line 201, in assertMaskedImagesNearlyEqual
          testCase.fail("%s: %s" % (msg, "; ".join(errStrList)))
      AssertionError: basicConvolve(MaskedImage, kernel=Spatially Varying Gaussian Analytic Kernel with 3 basis kernels convolved using brute force) wrote to edge pixels: image planes differ: maxDiff=1.22472e+38 at position (74, 3); value=-1.22472e+38 vs. 2878.0; NaNs differ
       
      ======================================================================
      FAIL: testTicket873 (__main__.ConvolveTestCase)
      Demonstrate ticket 873: convolution of a MaskedImage with a spatially varying
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "tests/convolve.py", line 623, in testTicket873
          rtol = rtol)
        File "tests/convolve.py", line 290, in runStdTest
          self.runBasicConvolveEdgeTest(kernel, kernelDescr)
        File "tests/convolve.py", line 317, in runBasicConvolveEdgeTest
          doVariance = True, rtol=0, atol=0, msg=msg)
        File "/Users/pschella/Development/lsst/code/afw/python/lsst/afw/image/testUtils.py", line 201, in assertMaskedImagesNearlyEqual
          testCase.fail("%s: %s" % (msg, "; ".join(errStrList)))
      AssertionError: basicConvolve(MaskedImage, kernel=Spatially varying LinearCombinationKernel of basis kernels with low covariance, using brute force) wrote to edge pixels: image planes differ: maxDiff=3.19374e+38 at position (1, 46); value=3.19374e+38 vs. 2774.0
       
      ----------------------------------------------------------------------
      Ran 13 tests in 43.252s
       
      FAILED (failures=4)
      The following tests failed:
      /Users/pschella/Development/lsst/code/afw/tests/.tests/convolve.py.failed
      1 tests failed
      scons: *** [checkTestStatus] Error 1
      scons: building terminated because of errors.
      

        Attachments

        1. o0.png
          o0.png
          15 kB
        2. o3.png
          o3.png
          15 kB

          Issue Links

            Activity

            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Merged after checking for performance regression (none found in convolve tests) and adding comment linking to this issue.

            P.S. The Boost change didn't help indeed.

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Merged after checking for performance regression (none found in convolve tests) and adding comment linking to this issue. P.S. The Boost change didn't help indeed.
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Oh, and I do believe the story point estimate for this ticket is a bit on the low side

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Oh, and I do believe the story point estimate for this ticket is a bit on the low side
            Hide
            swinbank John Swinbank added a comment -
            Show
            swinbank John Swinbank added a comment - Brief note added to the WIP release notes at https://confluence.lsstcorp.org/display/DM/Data+Release+Production+WIP+F16+Release+Notes .
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Which reminds me, John Swinbank should we ask Frossie Economou to pull this into the release?

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Which reminds me, John Swinbank should we ask Frossie Economou to pull this into the release?
            Hide
            swinbank John Swinbank added a comment -

            My inclination would be no, because:

            • We already decided that this wasn't a blocker for the release, and I don't think your fix changes that;
            • The production of the release is already very advanced, and we don't want to delay it unless it's for something really very urgent.

            However, if Frossie Economou is seeing this and has some spare time to pull it in, I certainly wouldn't object.

            Show
            swinbank John Swinbank added a comment - My inclination would be no, because: We already decided that this wasn't a blocker for the release, and I don't think your fix changes that; The production of the release is already very advanced, and we don't want to delay it unless it's for something really very urgent. However, if Frossie Economou is seeing this and has some spare time to pull it in, I certainly wouldn't object.

              People

              Assignee:
              pschella Pim Schellart [X] (Inactive)
              Reporter:
              pschella Pim Schellart [X] (Inactive)
              Reviewers:
              John Swinbank
              Watchers:
              Jim Bosch, John Swinbank, Nate Lust, Pim Schellart [X] (Inactive), Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.