Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13912

Remove sparse matrices in the new deblender

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_deblender
    • Labels:
      None

      Description

      The construction of sparse matrices in scipy is noticeably slow and currently causes the largest slowdown in performance for the new deblender (scarlet). We have optimized most of the code so that sparse matrices aren't needed, with the exception of the translation operators and PSF convolution operator. Since all of the sparse matrices we are using are essentially of the same form, banded diagonal matrices that can be thought of a single image window projected onto the morphology, we should develop a class that performs this simple operation for us. This is the last large boost we can expect in performance for the deblender and will give us a reasonable idea of the runtime of the algorithm.

       

      This will require some refactoring of the data structures in scarlet, namely to keep the morphology as an image as opposed to a flattened vector, which was only done to allow us to use linear operators.

        Attachments

          Activity

          Hide
          fred3m Fred Moolekamp added a comment -

          Modified scarlet to use 2D morphologies (instead of flattened matrices) and analytic operators to take the place of the sparse matrices. This has improved the runtime by ~25-30%.

          Show
          fred3m Fred Moolekamp added a comment - Modified scarlet to use 2D morphologies (instead of flattened matrices) and analytic operators to take the place of the sparse matrices. This has improved the runtime by ~25-30%.
          Hide
          pmelchior Peter Melchior added a comment -

          Done. We shaved of ~50% of the runtime and isolated the expensive call in one method, which can be further optimized as needed.

          Show
          pmelchior Peter Melchior added a comment - Done. We shaved of ~50% of the runtime and isolated the expensive call in one method, which can be further optimized as needed.

            People

            • Assignee:
              fred3m Fred Moolekamp
              Reporter:
              fred3m Fred Moolekamp
              Reviewers:
              Peter Melchior
              Watchers:
              Fred Moolekamp, Peter Melchior
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel