# Improve reproducibility in faro ellipKPM test_te1

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Team:
DM Science
• Urgent?:
No

#### Description

Currently, the test_te1 assertion tests for exact equality. Due to the floating point nature of these outputs and how they're generated on machines of varying architecture, exact equality cannot be guaranteed. Therefore, this ticket will replace the test for exact equality with the assertAlmostEqual constraint.

#### Activity

Hide
Matthias Wittgen added a comment -

Implementing assertAlmostEqual, loosening the test conditions and understanding why tiger fails the test are three different tickets in my opinion.

Show
Matthias Wittgen added a comment - Implementing assertAlmostEqual, loosening the test conditions and understanding why tiger fails the test are three different tickets in my opinion.
Hide
Colin Slater added a comment - - edited

I suspect that this is a recurrence of DM-29030, where treecorr needs the parameter brute=True in unit tests to reduce machine-dependent variation. If that doesn't work, we should probably drop this test (it's not worth this level of false-positive test failures).

Show
Colin Slater added a comment - - edited I suspect that this is a recurrence of DM-29030 , where treecorr needs the parameter brute=True in unit tests to reduce machine-dependent variation. If that doesn't work, we should probably drop this test (it's not worth this level of false-positive test failures).
Hide
Matthias Wittgen added a comment - - edited

Does DM-29030 explain why a binary installation on Springdale 7.9 fails the unit test at Princeton, but not on a freshly installed Springdale 7.9 VM? The underlying HW used is different but similar in architecture (x86_64). I would expect the conda/release install to yield reproducible results on Intel/AMD x86_64 unless there's something different in the run time setup.
Some environment override, different libraries picked up.

Show
Matthias Wittgen added a comment - - edited Does DM-29030 explain why a binary installation on Springdale 7.9 fails the unit test at Princeton, but not on a freshly installed Springdale 7.9 VM? The underlying HW used is different but similar in architecture (x86_64). I would expect the conda/release install to yield reproducible results on Intel/AMD x86_64 unless there's something different in the run time setup. Some environment override, different libraries picked up.
Hide
Colin Slater added a comment -

I have pushed a fix to the branch that sets brute=true in treecor, and makes the tests pass on jenkins, lsst-devl, and at least one machine that they were previously failing on. If someone wants to try the branch on any of the machines that were previously failing, I'd be interested to know if they still produce different results.

Show
Colin Slater added a comment - I have pushed a fix to the branch that sets brute=true in treecor, and makes the tests pass on jenkins, lsst-devl, and at least one machine that they were previously failing on. If someone wants to try the branch on any of the machines that were previously failing, I'd be interested to know if they still produce different results.
Hide
Colin Slater added a comment -

Looks like Eli has reviewed on the PR.

Show
Colin Slater added a comment - Looks like Eli has reviewed on the PR.

#### People

Assignee:
Colin Slater
Reporter:
Lee Kelvin
Reviewers:
Eli Rykoff
Watchers:
Colin Slater, Eli Rykoff, John Parejko, Lee Kelvin, Matthias Wittgen, Tim Jenness