Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-39836

Speed up FitAffineWcsTask

    XMLWordPrintable

    Details

    • Story Points:
      5
    • Epic Link:
    • Sprint:
      Ops Pipelines 2023
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      Trying out FitAffineWcsTask in the new CalibrateImage and comparing runtimes with the old Characterize and Calibrate showed me that FitAffineWcsTask is about 20x slower than FitTanSipWcsTask. Most of that time is spent in some scipy.optimize calls: there might be faster ones we could use instead.

      We still haven't switched to use the new affine fitter anywhere, but I think we should do so soon.

      This is at least a partial blocker to using the affine fitter for AP, as previously astrometry was a negligible part of our pipeline (less than half a second), and with the affine fitter it's about 5 seconds.

        Attachments

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29513 Confirm that HSC full focal plane distorations are used in AP processing and test that FitAffineWcsTask works as expected.

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-27858 Validate new Gaia AstrometryTask defaults for HSC

          • To Do

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            Looks great!  I'm glad turned out to be as simple as it seems to have been.  A few comments (mostly about code comments) on the PR.

             

            Show
            jbosch Jim Bosch added a comment - Looks great!  I'm glad turned out to be as simple as it seems to have been.  A few comments (mostly about code comments) on the PR.  
            Hide
            Permalink
            nlust Nate Lust added a comment -

            Final solution speeds this task up by about a factor of 36x, and makes it about 3 times faster than FitTanSipWcsTask (at least in the limited test case).

             

            John Parejko Do you think you could run this again in the context the ticket was created and see how it performs now?

            Show
            nlust Nate Lust added a comment - Final solution speeds this task up by about a factor of 36x, and makes it about 3 times faster than FitTanSipWcsTask (at least in the limited test case).   John Parejko Do you think you could run this again in the context the ticket was created and see how it performs now?
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks for the quick fix! Yes, this now makes FitAffineWcsTask take a bit less time than FitTanSipWcsTask on my test dataset. Nice work! AstrometryTask is back down to about 0.6s, most of which is in matchPessimisticB. This is now far down the list of things we want to optimize further, so I can stop thinking about astrometry for a while.

            Show
            Parejkoj John Parejko added a comment - Thanks for the quick fix! Yes, this now makes FitAffineWcsTask take a bit less time than FitTanSipWcsTask on my test dataset. Nice work! AstrometryTask is back down to about 0.6s, most of which is in matchPessimisticB. This is now far down the list of things we want to optimize further, so I can stop thinking about astrometry for a while.

              People

              Assignee:
              nlust Nate Lust
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Jim Bosch
              Watchers:
              Clare Saunders, Ian Sullivan, Jim Bosch, John Parejko, Nate Lust, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.