Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31611

Improve reproducibility in faro ellipKPM test_te1

    XMLWordPrintable

    Details

    • Team:
      DM Science
    • Urgent?:
      No

      Description

      Currently, the `test_te1` assertion tests for exact equality. Due to the floating point nature of these outputs and how they're generated on machines of varying architecture, exact equality cannot be guaranteed. Therefore, this ticket will replace the test for exact equality with the `assertAlmostEqual` constraint.

        Attachments

          Issue Links

            Activity

            Hide
            wittgen Matthias Wittgen added a comment -

            Implementing assertAlmostEqual, loosening the test conditions and understanding why tiger fails the test are three different tickets in my opinion.

            Show
            wittgen Matthias Wittgen added a comment - Implementing assertAlmostEqual, loosening the test conditions and understanding why tiger fails the test are three different tickets in my opinion.
            Hide
            ctslater Colin Slater added a comment - - edited

            I suspect that this is a recurrence of DM-29030, where treecorr needs the parameter brute=True in unit tests to reduce machine-dependent variation. If that doesn't work, we should probably drop this test (it's not worth this level of false-positive test failures).

            Show
            ctslater Colin Slater added a comment - - edited I suspect that this is a recurrence of DM-29030 , where treecorr needs the parameter brute=True in unit tests to reduce machine-dependent variation. If that doesn't work, we should probably drop this test (it's not worth this level of false-positive test failures).
            Hide
            wittgen Matthias Wittgen added a comment - - edited

            Does DM-29030 explain why a binary installation on Springdale 7.9 fails the unit test at Princeton, but not on a freshly installed Springdale 7.9 VM? The underlying HW used is different but similar in architecture (x86_64). I would expect the conda/release install to yield reproducible results on Intel/AMD x86_64 unless there's something different in the run time setup.
            Some environment override, different libraries picked up.

            Show
            wittgen Matthias Wittgen added a comment - - edited Does DM-29030 explain why a binary installation on Springdale 7.9 fails the unit test at Princeton, but not on a freshly installed Springdale 7.9 VM? The underlying HW used is different but similar in architecture (x86_64). I would expect the conda/release install to yield reproducible results on Intel/AMD x86_64 unless there's something different in the run time setup. Some environment override, different libraries picked up.
            Hide
            ctslater Colin Slater added a comment -

            I have pushed a fix to the branch that sets brute=true in treecor, and makes the tests pass on jenkins, lsst-devl, and at least one machine that they were previously failing on. If someone wants to try the branch on any of the machines that were previously failing, I'd be interested to know if they still produce different results.

            Show
            ctslater Colin Slater added a comment - I have pushed a fix to the branch that sets brute=true in treecor, and makes the tests pass on jenkins, lsst-devl, and at least one machine that they were previously failing on. If someone wants to try the branch on any of the machines that were previously failing, I'd be interested to know if they still produce different results.
            Hide
            ctslater Colin Slater added a comment -

            Looks like Eli has reviewed on the PR.

            Show
            ctslater Colin Slater added a comment - Looks like Eli has reviewed on the PR.

              People

              Assignee:
              ctslater Colin Slater
              Reporter:
              lskelvin Lee Kelvin
              Reviewers:
              Eli Rykoff
              Watchers:
              Colin Slater, Eli Rykoff, John Parejko, Lee Kelvin, Matthias Wittgen, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.