Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9707

Please make assertFloatsAlmostEqual more flexible

    XMLWordPrintable

    Details

      Description

      lsst.utils.assertFloatsAlmostEqual cannot handle lists of floats. Thus it cannot handle values returned from C++ as std::vector. Similarly for tuples of floats, lists of lists of floats, etc., though those may be rarer.

      The usual workaround is to use self.assertTrue(numpy.allclose(arr1, arr2)) but it would be nice not to have to do that.

      Also please consider adding a equal_nan flag like numpy.allclose. In both cases it is worth considering using numpy allclose as the code that performs the test.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            As Jim says above, I don't think we want to advocate using the numpy.allclose or numpy.testing.assert_array_almost_equal, because they both have a much less stringent default tolerance, and the numpy.testing modules defines things completely differently (number of decimals, instead of relative/absolute tolerance).

            Show
            Parejkoj John Parejko added a comment - As Jim says above, I don't think we want to advocate using the numpy.allclose or numpy.testing.assert_array_almost_equal , because they both have a much less stringent default tolerance, and the numpy.testing modules defines things completely differently (number of decimals, instead of relative/absolute tolerance).
            Hide
            jbosch Jim Bosch added a comment -

            Disappointing to hear that numpy.testing uses number of decimals instead of relative/absolute tolerance. Otherwise I'd have been on board with migrating in that direction.

            An ignoreNaNs option was added on DM-29370; I'm not sure if that's exactly what was requested here, but I imagine it's close enough. Sounds like there's at least a small amount of work to do here to support non-array sequence arguments, and that's still desired.

            Show
            jbosch Jim Bosch added a comment - Disappointing to hear that numpy.testing uses number of decimals instead of relative/absolute tolerance. Otherwise I'd have been on board with migrating in that direction. An ignoreNaNs option was added on DM-29370 ; I'm not sure if that's exactly what was requested here, but I imagine it's close enough. Sounds like there's at least a small amount of work to do here to support non-array sequence arguments, and that's still desired.
            Hide
            rowen Russell Owen added a comment - - edited

            Maybe I am misunderstanding, but numpy.testing.assert_allclose offers absolute and relative tolerance parameters, as does numpy.allclose (though I default values are different). I don't even see an option for specifying the number of decimal points in numpy.testing.assert_allclose. Here is the documentation I found: https://numpy.org/doc/stable/reference/generated/numpy.testing.assert_allclose.html#numpy-testing-assert-allclose

            I have been burned by assert_allclose's default value of atol=0. This means that arrays that should be nearly zero can fail validation for perfectly reasonable values. But of course you have to know what a reasonable variation is in order to set a good non-zero value of atol. I always use assert_allclose if I can, but I also try to remember to set atol.

            Show
            rowen Russell Owen added a comment - - edited Maybe I am misunderstanding, but numpy.testing.assert_allclose offers absolute and relative tolerance parameters, as does numpy.allclose (though I default values are different). I don't even see an option for specifying the number of decimal points in numpy.testing.assert_allclose. Here is the documentation I found: https://numpy.org/doc/stable/reference/generated/numpy.testing.assert_allclose.html#numpy-testing-assert-allclose I have been burned by assert_allclose's default value of atol=0. This means that arrays that should be nearly zero can fail validation for perfectly reasonable values. But of course you have to know what a reasonable variation is in order to set a good non-zero value of atol. I always use assert_allclose if I can, but I also try to remember to set atol.
            Hide
            Parejkoj John Parejko added a comment -

            I very specifically referred to "numpy.testing.assert_array_almost_equal": https://numpy.org/doc/stable/reference/generated/numpy.testing.assert_array_almost_equal.html

            numpy's allclose defaults to rtol=1e-7, which is not very tight.

            Show
            Parejkoj John Parejko added a comment - I very specifically referred to "numpy.testing.assert_array_almost_equal": https://numpy.org/doc/stable/reference/generated/numpy.testing.assert_array_almost_equal.html numpy's allclose defaults to rtol=1e-7, which is not very tight.
            Hide
            rowen Russell Owen added a comment - - edited

            John Parejko I was answering Jim Bosch statement "numpy.testing uses number of decimals instead of relative/absolute tolerance". I think that statement is too broad.

            In any case the docs you point to explicitly say that assert_arrays_almost_equal should be avoided in favor of assert_allclose or two other options I have never heard of before.

            Note that numpy.allclose and numpy.testing.assert_allclose are different (though very similar) functions, with different default tolerance values. Perhaps it is best to always specify the tolerances explicitly.

            Show
            rowen Russell Owen added a comment - - edited John Parejko I was answering Jim Bosch statement "numpy.testing uses number of decimals instead of relative/absolute tolerance". I think that statement is too broad. In any case the docs you point to explicitly say that assert_arrays_almost_equal should be avoided in favor of assert_allclose or two other options I have never heard of before. Note that numpy.allclose and numpy.testing.assert_allclose are different (though very similar) functions, with different default tolerance values. Perhaps it is best to always specify the tolerances explicitly.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              rowen Russell Owen
              Watchers:
              Jim Bosch, John Parejko, John Swinbank, Russell Owen, Simon Krughoff, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.