Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11909

Test functionality of deblender task

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_deblender
    • Labels:
      None
    • Story Points:
      5
    • Sprint:
      DRP F17-4, DRP F17-5, DRP F17-6, DRP S18-1, DRP S18-2, DRP S18-3, DRP S18-4, DRP S18-5, DRP S18-6, DRP F18-1, DRP F19-2
    • Team:
      Data Release Production

      Description

      DM-11329 involved creating a deblender task to run the new deblender on HSC data. This ticket is designed to run that task on a few HSC fields to test its performance before the larger task of running the deblender on a large patch of data and analyzing the results (DM-11330).

        Attachments

          Issue Links

            Activity

            Hide
            fred3m Fred Moolekamp added a comment -

            The notebook compareHSC_new.ipynb displays a comparison of the flux calculated using the new deblender templates, the flux conserved results from the new deblender, and the old deblender.

            It also shows a visual comparison of the models from the new deblender and data for

            1. outliers (24 objects whose flux differs from the old deblender by more than 5%)
            2. blends (all blends with a non-negligible flux overlap, sorted from most blended to least blended

            Any sources marked with a red label are outliers (differ in flux by more than 5%).

            My preliminary analysis is as follows (many of these items we knew already):

            1. The biggest problem is false colors due to the warping done when the coadds are PSF matched. I don't see a way around this, since it breaks the fundamental assumption of our algorithm, ie. that each source can be modeled as a morphology multiplied by an SED. So a poorly modeled bright object will affect neighboring sources that try to model the colors from the brighter object, causing them to have larger footprints and unrealistic colors.
            2. Non-detections are a major problem and we will have to have an iterative detection scheme built into the pipeline
            3. Overall the deblender performs well, but to really understand how it handles heavily blended regions we have to run it on regular coadds (since we can't trust the colors from PSF matched coadds for now).

            So my recommendation (and Peter Melchior's) is to upgrade to his latest changes in the deblender, which do a better job setting the convergence and background levels and convolve each object with the PSF in the deblender. This should be a relatively simple change to make and will give us a better understanding of the current performance of the deblender.

            Show
            fred3m Fred Moolekamp added a comment - The notebook compareHSC_new.ipynb displays a comparison of the flux calculated using the new deblender templates, the flux conserved results from the new deblender, and the old deblender. It also shows a visual comparison of the models from the new deblender and data for outliers (24 objects whose flux differs from the old deblender by more than 5%) blends (all blends with a non-negligible flux overlap, sorted from most blended to least blended Any sources marked with a red label are outliers (differ in flux by more than 5%). My preliminary analysis is as follows (many of these items we knew already): The biggest problem is false colors due to the warping done when the coadds are PSF matched. I don't see a way around this, since it breaks the fundamental assumption of our algorithm, ie. that each source can be modeled as a morphology multiplied by an SED. So a poorly modeled bright object will affect neighboring sources that try to model the colors from the brighter object, causing them to have larger footprints and unrealistic colors. Non-detections are a major problem and we will have to have an iterative detection scheme built into the pipeline Overall the deblender performs well, but to really understand how it handles heavily blended regions we have to run it on regular coadds (since we can't trust the colors from PSF matched coadds for now). So my recommendation (and Peter Melchior 's) is to upgrade to his latest changes in the deblender, which do a better job setting the convergence and background levels and convolve each object with the PSF in the deblender. This should be a relatively simple change to make and will give us a better understanding of the current performance of the deblender.
            Hide
            pmelchior Peter Melchior added a comment -

            Fred and I went through both lists today, and I have a few comments.

            Re 1) from Fred's list above: I don't believe that's warping, but regardless: the cores of bright objects often have very bad colors with non-zero weights (meaning: they haven't been properly flagged). That's a problem.

            Re 2): there are non-detections of even really bright objects. For the life of me, I don't know what happened with those. That's an even bigger problem.

            The second list in the notebook is more instructive of flaws in the deblender or its configuration. I have categorized those into ~4 different cases, which we should check in detail. I believe we understand all cases, but only a close inspection will tell. If true, we have strategies for dealing with these cases.

            @fred3m: Can you export the images of the second list together with the corresponding weights and the peak coordinates into, say, numpy or fits, so that we can run those offline as a testbed?

            Show
            pmelchior Peter Melchior added a comment - Fred and I went through both lists today, and I have a few comments. Re 1) from Fred's list above: I don't believe that's warping, but regardless: the cores of bright objects often have very bad colors with non-zero weights (meaning: they haven't been properly flagged). That's a problem. Re 2): there are non-detections of even really bright objects. For the life of me, I don't know what happened with those. That's an even bigger problem. The second list in the notebook is more instructive of flaws in the deblender or its configuration. I have categorized those into ~4 different cases, which we should check in detail. I believe we understand all cases, but only a close inspection will tell. If true, we have strategies for dealing with these cases. @fred3m: Can you export the images of the second list together with the corresponding weights and the peak coordinates into, say, numpy or fits, so that we can run those offline as a testbed?
            Hide
            fred3m Fred Moolekamp added a comment -

            Peter Melchior, I'm assuming that you want the coadded images that are not psf matched, is that correct? I'll put them in the https://github.com/lsst/testdata_deblender repo along with the other files you requested.

            Show
            fred3m Fred Moolekamp added a comment - Peter Melchior , I'm assuming that you want the coadded images that are not psf matched, is that correct? I'll put them in the https://github.com/lsst/testdata_deblender repo along with the other files you requested.
            Hide
            pmelchior Peter Melchior added a comment -

            I'd be content with the PSF-matched ones for now (apples-to-apples), and the second list (substantial overlap) doesn't have the worst-case failures.

            But if you can also give me the coadds before PSF-matching and the PSF images for those areas, I'd take them as well so that we can also do the PSF deconvolution offline.

            Show
            pmelchior Peter Melchior added a comment - I'd be content with the PSF-matched ones for now (apples-to-apples), and the second list (substantial overlap) doesn't have the worst-case failures. But if you can also give me the coadds before PSF-matching and the PSF images for those areas, I'd take them as well so that we can also do the PSF deconvolution offline.
            Hide
            fred3m Fred Moolekamp added a comment - - edited

            See https://github.com/lsst/testdata_deblender/tree/master/real_data/hsc_cosmos.
            To get the data, weight map, and catalog for each blend use:
            data = np.load(filename)
            images = data["images"]
            weights = data["weights"]
            peaks = data["peaks"]
            psfs = data["psfs"]

            The filenames correspond to the parent id from the catalog created on the psf_matched coadds.

            Show
            fred3m Fred Moolekamp added a comment - - edited See https://github.com/lsst/testdata_deblender/tree/master/real_data/hsc_cosmos . To get the data, weight map, and catalog for each blend use: data = np.load(filename) images = data ["images"] weights = data ["weights"] peaks = data ["peaks"] psfs = data ["psfs"] The filenames correspond to the parent id from the catalog created on the psf_matched coadds.
            Hide
            fred3m Fred Moolekamp added a comment -

            Upon further inspection the work described in this ticket was basically handled during the deblender sprint, as meas_extensions_scarlet has now been demonstrated to run on multiple HSC fields. So DM-11330 will be used to run scarlet on a more targeted set of patches with DM-12413 used to analyze the results.

            Show
            fred3m Fred Moolekamp added a comment - Upon further inspection the work described in this ticket was basically handled during the deblender sprint, as meas_extensions_scarlet has now been demonstrated to run on multiple HSC fields. So DM-11330 will be used to run scarlet on a more targeted set of patches with DM-12413 used to analyze the results.

              People

              • Assignee:
                fred3m Fred Moolekamp
                Reporter:
                fred3m Fred Moolekamp
                Watchers:
                Fred Moolekamp, Jim Bosch, John Swinbank, Peter Melchior, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: