Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-6278

Investigate proper precision for afw::image::Image pixel transforms

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: afw
    • Templates:
    • Story Points:
      1
    • Team:
      Data Release Production

      Description

      The various pixel based transforms in afw/src/image/Image.cc were converted from using boost::lambda to C++11 lambda per DM-6091.

      At many places the previous implementation contained implicit casts (through boost::ret) of intermediate results to PixelT (e.g. float).
      In particular this affects opperations such as result = l + c*r where l is the left hand side image, r is the right hand side image and c a double constant.
      When calculated at double precision (e.g. without the casts, which are not needed with C++11 lambdas) the result is slightly different and this causes tests/testProcessCcd.py to fail on self.assertAlmostEqual(psfIyy, 2.17386182921239, places=7) which is only equal up to the fifth place.

      In order to not break existing behaviour I added explicit casts to PixelT for intermediate results. But this approach is questionable as the end result will be less accurate then possible. The aim of this ticket is to decide which approach is best:

      1. Calculate at full precision and modify the test case.
      2. Cast intermediate results to final precision (as it is done now).
      3. Do something else?

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  pschella Pim Schellart [X] (Inactive)
                  Watchers:
                  Jim Bosch, John Swinbank, Pim Schellart [X] (Inactive)
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:

                    Summary Panel