Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8624

Evaluate the deblender using simulations

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      In addition to testing the blender using observed fields, it would also be wise to create synthetic objects using galsim or SynPipe where we know what the underlying structures are and compare that to the results from running the deblender.

      The goal of this ticket is to evaluate galsim and Synpipe to see if one or the other (or both) will be useful in debugging certain failure cases with the deblender.

        Attachments

          Activity

          Hide
          rearmstr Bob Armstrong added a comment - - edited

          I looked at this using a Synpipe run that was done at IPMU. My specific goal was to reproduce some of the problems we found in DM-8059, but where we had truth information. This would allow us to isolate where the problem was occurring.

          I took a number of patches from a single tract and found all of the true objects that were in blends with more than 25 peaks. This was to try to find the problems we encountered in galaxy clusters. I compared the measured cmodel flux to the true flux as a function of true magnitude:

          This plot shows that except for some notable outliers, the cmodel magnitudes look pretty reasonable. I investigated a number of outliers and found that a majority of these are cases where the fake galaxy was inserted on top of a real galaxy or it was inserted near a bright star. I also tried to examine the deblended templates and didn't find any of the problems I found in DM-8059. This may be because I was not actually selecting clusters of galaxies. We have a maximum limit of the number of peaks per footprint of 30, but for clusters you would expect more. This means we can't select on footprints that have more than 30 peaks.

          I think this means that we may want to investigate using galsim to create groups of objects that look more like actual galaxy clusters.

          Show
          rearmstr Bob Armstrong added a comment - - edited I looked at this using a Synpipe run that was done at IPMU. My specific goal was to reproduce some of the problems we found in DM-8059 , but where we had truth information. This would allow us to isolate where the problem was occurring. I took a number of patches from a single tract and found all of the true objects that were in blends with more than 25 peaks. This was to try to find the problems we encountered in galaxy clusters. I compared the measured cmodel flux to the true flux as a function of true magnitude: This plot shows that except for some notable outliers, the cmodel magnitudes look pretty reasonable. I investigated a number of outliers and found that a majority of these are cases where the fake galaxy was inserted on top of a real galaxy or it was inserted near a bright star. I also tried to examine the deblended templates and didn't find any of the problems I found in DM-8059 . This may be because I was not actually selecting clusters of galaxies. We have a maximum limit of the number of peaks per footprint of 30, but for clusters you would expect more. This means we can't select on footprints that have more than 30 peaks. I think this means that we may want to investigate using galsim to create groups of objects that look more like actual galaxy clusters.
          Hide
          rearmstr Bob Armstrong added a comment -

          I looked which objects were being selected and found that none of them were actually clusters of galaxies. Part of the problem is that we had the local background subtraction turned off - meaning that all bright objects had large halos of objects around them.

          I have now started generating multi-color images using simulations using galsim that can more accurately address the problem.

          Show
          rearmstr Bob Armstrong added a comment - I looked which objects were being selected and found that none of them were actually clusters of galaxies. Part of the problem is that we had the local background subtraction turned off - meaning that all bright objects had large halos of objects around them. I have now started generating multi-color images using simulations using galsim that can more accurately address the problem.
          Hide
          rearmstr Bob Armstrong added a comment -

          I have finished the script to create multi-color images from realistic galaxy templates using galsim. This is able to reproduce the same type of failures we saw in cluster environments. I don't think I was able to learn much more having this simulations, because we have an idea of where the problem is now. But this will be useful for testing new deblending schemes with some more realistic situations with known failures where we know what the truth is.

          I think we can close this issue.

          Show
          rearmstr Bob Armstrong added a comment - I have finished the script to create multi-color images from realistic galaxy templates using galsim. This is able to reproduce the same type of failures we saw in cluster environments. I don't think I was able to learn much more having this simulations, because we have an idea of where the problem is now. But this will be useful for testing new deblending schemes with some more realistic situations with known failures where we know what the truth is. I think we can close this issue.
          Hide
          swinbank John Swinbank added a comment -

          Bob Armstrong — where is the script you produced? If you were to get a better offer and decide to leave the project tomorrow, how could we reproduce your work?

          Show
          swinbank John Swinbank added a comment - Bob Armstrong — where is the script you produced? If you were to get a better offer and decide to leave the project tomorrow, how could we reproduce your work?
          Hide
          rearmstr Bob Armstrong added a comment -

          Ah, yes. I was planning on sharing it with Fred, but I can post it here on the issue as well. Is there a better place to put this?

          Show
          rearmstr Bob Armstrong added a comment - Ah, yes. I was planning on sharing it with Fred, but I can post it here on the issue as well. Is there a better place to put this?
          Hide
          swinbank John Swinbank added a comment -

          I think on the issue is fine for now. If it is (or becomes) something we want to preserve and work on in the long term, it should go into a repository in the lsst-dm organization.

          Show
          swinbank John Swinbank added a comment - I think on the issue is fine for now. If it is (or becomes) something we want to preserve and work on in the long term, it should go into a repository in the lsst-dm organization.

            People

            • Assignee:
              rearmstr Bob Armstrong
              Reporter:
              fred3m Fred Moolekamp
              Watchers:
              Bob Armstrong, Fred Moolekamp, John Swinbank
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel