Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11805

Improve performance of MaxPsfWcsSelectImagesTask

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Science Pipelines
    • Labels:
      None
    • Story Points:
      4
    • Sprint:
      Alert Production F17 - 9, Alert Production F17 - 10
    • Team:
      Alert Production

      Description

      The good-seeing ImageSelector ported in DM-10977 seems to be extremely slow compared to passing the visit ids directly to selectID. This ticket is to investigate the cause of the slowness and see if it can be mitigated.

        Attachments

          Activity

          Hide
          ebellm Eric Bellm added a comment - - edited

          I note the comment in runDataRef from Simon Krughoff:

          We use "convexHull" instead of generating a SphericalConvexPolygon
          directly because the standard for the inputs to SphericalConvexPolygon
          are pretty high and we don't want to be responsible for reaching them.
          If "convexHull" is found to be too slow, we can revise this.

          Show
          ebellm Eric Bellm added a comment - - edited I note the comment in runDataRef from Simon Krughoff : We use "convexHull" instead of generating a SphericalConvexPolygon directly because the standard for the inputs to SphericalConvexPolygon are pretty high and we don't want to be responsible for reaching them. If "convexHull" is found to be too slow, we can revise this.
          Hide
          ebellm Eric Bellm added a comment -

          A further suggestion is that the current code calls dataRef.get('calexp'); if this could be replaced with dataRef.get('calexp_psf') it would presumably be faster.

          Show
          ebellm Eric Bellm added a comment - A further suggestion is that the current code calls dataRef.get('calexp') ; if this could be replaced with dataRef.get('calexp_psf') it would presumably be faster.
          Hide
          ebellm Eric Bellm added a comment -

          I investigated the performance of the task on a small dataset by inspecting the cProfile statistics. The bulk of the 26 second runtime is taken by convolve, warpExposure, and persist. Sorting by cumulative time (which includes the time in that function and all called functions), the MaxPsfWcsSelectImagesTask.runDataRef call takes only 0.3 seconds.

          I therefore conclude that any potential savings from changing the code covered by this ticket (including either of the two fixes identified above) would give only ~1% speedup, so I am marking this as "Won't Fix."

          (There is some possibility that performance hot spots in this module would appear with a larger number of input images, but it's not obvious to me how or if it would be dominant. I will check the profiles on larger runs as needed.)

          Show
          ebellm Eric Bellm added a comment - I investigated the performance of the task on a small dataset by inspecting the cProfile statistics. The bulk of the 26 second runtime is taken by convolve , warpExposure , and persist . Sorting by cumulative time (which includes the time in that function and all called functions), the MaxPsfWcsSelectImagesTask.runDataRef call takes only 0.3 seconds. I therefore conclude that any potential savings from changing the code covered by this ticket (including either of the two fixes identified above) would give only ~1% speedup, so I am marking this as "Won't Fix." (There is some possibility that performance hot spots in this module would appear with a larger number of input images, but it's not obvious to me how or if it would be dominant. I will check the profiles on larger runs as needed.)

            People

            • Assignee:
              ebellm Eric Bellm
              Reporter:
              ebellm Eric Bellm
              Watchers:
              Eric Bellm
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: