Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11139

Update deblender to use new proxmin API

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_deblender
    • Labels:
      None

      Description

      Peter Melchior and his summer student Max Jerdee have been making improvements to the proxmin package, which contains the GLMM algorithm used in the deblender. This ticket will update the deblender repo to use the new API.

        Attachments

          Activity

          Hide
          fred3m Fred Moolekamp added a comment -

          The API has been updated, including the use of (optionally) using gradients in X and Y in place of radial monotonicity. https://github.com/lsst-dm/deblender_examples/blob/master/test1.ipynb for the results on a single field.

          Show
          fred3m Fred Moolekamp added a comment - The API has been updated, including the use of (optionally) using gradients in X and Y in place of radial monotonicity. https://github.com/lsst-dm/deblender_examples/blob/master/test1.ipynb for the results on a single field.
          Hide
          fred3m Fred Moolekamp added a comment -

          Peter, would you mind checking off that the code uses the new deblender API (I still have a "strict monotonicity" parameter that I use in the stack to enforce strict monotonicity, which is why that parameter still exists in the notebook).

          Show
          fred3m Fred Moolekamp added a comment - Peter, would you mind checking off that the code uses the new deblender API (I still have a "strict monotonicity" parameter that I use in the stack to enforce strict monotonicity, which is why that parameter still exists in the notebook).
          Hide
          pmelchior Peter Melchior added a comment -

          1) I don't think the new X,Y constraints are flexible enough. They seem to work fine for close-to circular objects, but create strong artifacts for objects with large ellipticities. That's pretty much expected because the break in monotonicity direction is shaped like a + fixed at the objects centroid. Everything that has an angle to it, will get these blocky artifacts.

          2) I don't fully understand what API you're calling here as there's the wrapper `proximal.ExposureDeblend` around the actual call to NMF or GLMM.

          Show
          pmelchior Peter Melchior added a comment - 1) I don't think the new X,Y constraints are flexible enough. They seem to work fine for close-to circular objects, but create strong artifacts for objects with large ellipticities. That's pretty much expected because the break in monotonicity direction is shaped like a + fixed at the objects centroid. Everything that has an angle to it, will get these blocky artifacts. 2) I don't fully understand what API you're calling here as there's the wrapper `proximal.ExposureDeblend` around the actual call to NMF or GLMM.
          Hide
          fred3m Fred Moolekamp added a comment -

          1) I agree that they aren't perfect, but the blockiness in mostly in the noisy regions and they do some things better than radial monotonicity (no donuts in the residual, lower overall residuals and better flux matches to the data). But this discussion is outside the scope of this ticket and now that we have multiple monotonicity constraints we can evaluate them in a future ticket.

          2) ExposureDeblend is an object that keeps track of all of the stack exposure images and deblended parents. By now it's pretty complicated, but essentially it contains images, catalogs, etc. for a given image and each blend is a `DeblendedParent` object.

          Starting at line 315, this commit shows the features that were updated to work with the new API.

          Show
          fred3m Fred Moolekamp added a comment - 1) I agree that they aren't perfect, but the blockiness in mostly in the noisy regions and they do some things better than radial monotonicity (no donuts in the residual, lower overall residuals and better flux matches to the data). But this discussion is outside the scope of this ticket and now that we have multiple monotonicity constraints we can evaluate them in a future ticket. 2) ExposureDeblend is an object that keeps track of all of the stack exposure images and deblended parents. By now it's pretty complicated, but essentially it contains images, catalogs, etc. for a given image and each blend is a `DeblendedParent` object. Starting at line 315, this commit shows the features that were updated to work with the new API.
          Hide
          pmelchior Peter Melchior added a comment -

          1) Agreed

          2) I'm looking at lsst/meas/deblender/proximal.py and there are redundant things as well as missing things.

          • The entire block (Line 316 to 338) is not needed because the prox_S operator is constructed in deblender.nmf.deblend
          • The usePsf argument is None even though it should be boolean and passing the psfs through kwargs is very implicit and really not needed here.
          • Unless you want to go with the kwargs route, then I suggest that you collect all arguments from DeblendedParent.deblend that are passed to nmf and don't affect the work of this member function. That would be (by my count) maxiter, psfThresh, l0_thresh, and l1_thresh. This way you'd make quite clear what this method does besides running the nmf.
          • I believe the peaks argument and later treatment is redundant because the peaks are already present at construction of the class.
          • strict_constraints is not supported any longer and should log a warning.
          Show
          pmelchior Peter Melchior added a comment - 1) Agreed 2) I'm looking at lsst/meas/deblender/proximal.py and there are redundant things as well as missing things. The entire block (Line 316 to 338) is not needed because the prox_S operator is constructed in deblender.nmf.deblend The usePsf argument is None even though it should be boolean and passing the psfs through kwargs is very implicit and really not needed here. Unless you want to go with the kwargs route, then I suggest that you collect all arguments from DeblendedParent.deblend that are passed to nmf and don't affect the work of this member function. That would be (by my count) maxiter, psfThresh, l0_thresh, and l1_thresh. This way you'd make quite clear what this method does besides running the nmf. I believe the peaks argument and later treatment is redundant because the peaks are already present at construction of the class. strict_constraints is not supported any longer and should log a warning.
          Hide
          pmelchior Peter Melchior added a comment -

          After talking to Fred, we agreed on removing the duplicate keywords maxiter, psfThresh, l0_thresh, and l1_thresh in favor of kwargs that get passed onto the actual nmf deblender.

          Also, if the prox_S operator is overwritten before the NMF is called, a message to that effect should be logged.

          Show
          pmelchior Peter Melchior added a comment - After talking to Fred, we agreed on removing the duplicate keywords maxiter, psfThresh, l0_thresh, and l1_thresh in favor of kwargs that get passed onto the actual nmf deblender. Also, if the prox_S operator is overwritten before the NMF is called, a message to that effect should be logged.
          Show
          fred3m Fred Moolekamp added a comment - I made the requested updates, see https://github.com/lsst/meas_deblender/blob/d39cda089c98cd548f634c0608c6f95fbceacfc5/python/lsst/meas/deblender/proximal.py#L252-L341 .

            People

            • Assignee:
              fred3m Fred Moolekamp
              Reporter:
              fred3m Fred Moolekamp
              Reviewers:
              Peter Melchior
              Watchers:
              Fred Moolekamp, Peter Melchior
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel