Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12783

Normalize weights differently for morphology and SED updates

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_deblender
    • Labels:
      None

      Description

      It is believed that the problems with bad colors in HSC-Z and Y that we observed in DM-12776 are due to the way we are using inverse variance maps to weight gradient steps in the deblender. When two bands have substantially different mean variance, certain colors converge more quickly than other colors, and at all times are capable of taking larger steps toward the minimum, which we believe results in poor colors in the bands with higher variance. Similarly, images with widely varying pixel variances could also see some pixels reaching convergence more quickly than others (although we have not observed this directly).

      To combat this Peter Melchior and I propose to normalize the weights for SED updates along each band, so that only pixel by pixel variations are weighted for each SED update, and normalize the weights for morphologies along each single band image. This should give us more uniform convergence and hopefully solve DM-12776 as well.

        Attachments

          Issue Links

            Activity

            Hide
            fred3m Fred Moolekamp added a comment -

            I updated the deblender with commit 1d87a84 and proxmin with commit 7343928 to use different weights for A and S.

            I tested this on an HSC field where we first saw this problem come up and the results now have more accurate colors.

            Show
            fred3m Fred Moolekamp added a comment - I updated the deblender with commit 1d87a84 and proxmin with commit 7343928 to use different weights for A and S. I tested this on an HSC field where we first saw this problem come up and the results now have more accurate colors.
            Hide
            pmelchior Peter Melchior added a comment -

            I have added a version of this with three weight matrices (original, pixel-normalized, band-normalized) to the scarlet branch of deblender. It also deals with partially and fully saturated pixels (meaning one/several bands vs all bands saturated). For the former, we need to ignore the colors of those pixels, for the latter ones there cannot be an update the changes them because we do not have information about them. My fix should make sure that they don't harm the rest of the pixels, but this requires additional testing.

            I've sent you the respective code, can you make sure it's working as intended, please.

            Show
            pmelchior Peter Melchior added a comment - I have added a version of this with three weight matrices (original, pixel-normalized, band-normalized) to the scarlet branch of deblender. It also deals with partially and fully saturated pixels (meaning one/several bands vs all bands saturated). For the former, we need to ignore the colors of those pixels, for the latter ones there cannot be an update the changes them because we do not have information about them. My fix should make sure that they don't harm the rest of the pixels, but this requires additional testing. I've sent you the respective code, can you make sure it's working as intended, please.
            Hide
            fred3m Fred Moolekamp added a comment -

            I verified that the algorithm you implemented in the scarlet branch also works with the older API (see the latest commit).

            Can I close this ticket now?

            Show
            fred3m Fred Moolekamp added a comment - I verified that the algorithm you implemented in the scarlet branch also works with the older API (see the latest commit ). Can I close this ticket now?

              People

              • Assignee:
                fred3m Fred Moolekamp
                Reporter:
                fred3m Fred Moolekamp
                Reviewers:
                Peter Melchior
                Watchers:
                Fred Moolekamp, Peter Melchior
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel