Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-6278

Investigate proper precision for afw::image::Image pixel transforms


    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: afw
    • Templates:
    • Story Points:
    • Team:
      Data Release Production


      The various pixel based transforms in afw/src/image/Image.cc were converted from using boost::lambda to C++11 lambda per DM-6091.

      At many places the previous implementation contained implicit casts (through boost::ret) of intermediate results to PixelT (e.g. float).
      In particular this affects opperations such as result = l + c*r where l is the left hand side image, r is the right hand side image and c a double constant.
      When calculated at double precision (e.g. without the casts, which are not needed with C++11 lambdas) the result is slightly different and this causes tests/testProcessCcd.py to fail on self.assertAlmostEqual(psfIyy, 2.17386182921239, places=7) which is only equal up to the fifth place.

      In order to not break existing behaviour I added explicit casts to PixelT for intermediate results. But this approach is questionable as the end result will be less accurate then possible. The aim of this ticket is to decide which approach is best:

      1. Calculate at full precision and modify the test case.
      2. Cast intermediate results to final precision (as it is done now).
      3. Do something else?


          Container Issues

            Issue Links



                • Assignee:
                  pschella Pim Schellart [X] (Inactive)
                  Jim Bosch, John Swinbank, Pim Schellart [X] (Inactive)
                • Votes:
                  0 Vote for this issue
                  3 Start watching this issue


                  • Created:

                    Summary Panel