Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11554

Improve Block-SDMM convergence checks

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_deblender
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      DRP F17-3, DRP F17-4, DRP F17-5, DRP F17-6, DRP S18-1, DRP S18-2, DRP S18-3, DRP S18-4
    • Team:
      Data Release Production

      Description

      Our Block-SDMM algorithm has always had certain blends that never met the convergence criteria even when a model appeared to quickly converge to a solution. Further investigation revealed that this behavior occurs because the traditional check for convergence in an ADMM-like algorithm breaks down when the dual variable (Z) is a projection onto zero. In this case, which is true of the symmetry operator L, the residual is R^2 = (LX-Z)^2 (where X is the matrix we are solving for), which is compared to the primal error e^2=e_rel^2 * max(Z_k+1 - Z_k, LX^2)=e_rel^2 * LX^2 = e_rel**2 * R^2, where e_rel is the maximum relative error allowed.

      The convergence criteria is R^2<= e^2, which is never true in the case where Z_k is a projection onto a single value (in this case zero). The current algorithm just checks that R^2<=e_rel**2, which does not scale properly with the number of objects and size of the image, causing the issues we are seeing with convergence in certain blends.

      This ticket is to devise a mathematically sound and accurate check that the algorithm has converged, even in cases where the constraint cannot be met (for example the solution is not symmetric), including a flag to note that a particular peak (or blend) failed to meet the constraint.

        Attachments

          Issue Links

            Activity

            Hide
            fred3m Fred Moolekamp added a comment -

            Much of this work is being implement by Peter Melchior in his work on a new deblender branch. As long as testing on this branch shows that the convergence issue has been solved for the deblender this ticket can be closed (I adjusted the number of story points for work already completed).

            Show
            fred3m Fred Moolekamp added a comment - Much of this work is being implement by Peter Melchior in his work on a new deblender branch. As long as testing on this branch shows that the convergence issue has been solved for the deblender this ticket can be closed (I adjusted the number of story points for work already completed).
            Hide
            fred3m Fred Moolekamp added a comment -

            Peter Melchior are you ok with me marking this as "Won't Fix?" I believe that the convergence checks are right now, using the algorithm that you completed in the fall, but we never spent any time optimizing our choice of e_abs. Now that we don't use the prox_g constraints I think that this ticket is irrelevant.

            Show
            fred3m Fred Moolekamp added a comment - Peter Melchior are you ok with me marking this as "Won't Fix?" I believe that the convergence checks are right now, using the algorithm that you completed in the fall, but we never spent any time optimizing our choice of e_abs . Now that we don't use the prox_g constraints I think that this ticket is irrelevant.
            Hide
            pmelchior Peter Melchior added a comment -

            Yes, fine, this has become largely obsolete.

            Show
            pmelchior Peter Melchior added a comment - Yes, fine, this has become largely obsolete.

              People

              • Assignee:
                fred3m Fred Moolekamp
                Reporter:
                fred3m Fred Moolekamp
                Watchers:
                Fred Moolekamp, Peter Melchior
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel