Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4550

Add SDSS anti-shredding algorithm to deblender

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      8
    • Sprint:
      DRP F16-5, DRP F16-6
    • Team:
      Data Release Production

      Description

      Robert Lupton identifies this as a priority on HipChat (HSC room, 2015-12-04, 18:17 Eastern).

        Attachments

        1. example1.png
          example1.png
          500 kB
        2. example2.png
          example2.png
          444 kB
        3. example3.png
          example3.png
          390 kB
        4. example4.png
          example4.png
          340 kB
        5. example5.png
          example5.png
          470 kB

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment -

            Looks great Bob Armstrong. A few minor comments on the PR and a request for more test coverage (which can be pushed to another ticket if you'd like).

            Show
            lauren Lauren MacArthur added a comment - Looks great Bob Armstrong . A few minor comments on the PR and a request for more test coverage (which can be pushed to another ticket if you'd like).
            Hide
            rearmstr Bob Armstrong added a comment -

            Here are a couple of examples of what the algorithm does from a list of merging galaxies in HSC that are known to have their flux underestimated as compared to SDSS. (You can find a list of these in DM-8139).

            In the following pictures I have labeled

            • blue "x" = peak the deblender thought was a PSF
            • green "+" = peak that is not a PSF
            • purple "o" = objects identified as having their template degenerate with another one.
            • red "x" = object that had underestimated flux compared to SDSS

            Here are a few examples of cases that look like this is helping to not shred galaxies


            Here are some examples where objects were identified as having a degenerate template, but perhaps should not have been.

            Show
            rearmstr Bob Armstrong added a comment - Here are a couple of examples of what the algorithm does from a list of merging galaxies in HSC that are known to have their flux underestimated as compared to SDSS. (You can find a list of these in DM-8139 ). In the following pictures I have labeled blue "x" = peak the deblender thought was a PSF green "+" = peak that is not a PSF purple "o" = objects identified as having their template degenerate with another one. red "x" = object that had underestimated flux compared to SDSS Here are a few examples of cases that look like this is helping to not shred galaxies Here are some examples where objects were identified as having a degenerate template, but perhaps should not have been.
            Hide
            swinbank John Swinbank added a comment -

            That sounds fine to me.

            Show
            swinbank John Swinbank added a comment - That sounds fine to me.
            Hide
            lauren Lauren MacArthur added a comment -

            Oh, I didn't mean to imply the documentation should be attached to the ticket, just that if the location of it (whether temporary or not) doesn't easily accommodate demonstrative figures, a few of the latter attached to the ticket would nice.

            Show
            lauren Lauren MacArthur added a comment - Oh, I didn't mean to imply the documentation should be attached to the ticket, just that if the location of it (whether temporary or not) doesn't easily accommodate demonstrative figures, a few of the latter attached to the ticket would nice.
            Hide
            swinbank John Swinbank added a comment - - edited

            I'd be fine with a few "before" and "after" figures attached to the ticket.

            Hmm, I'm not sure I would. At least, I'd like to discuss in more detail what our longer term plans for the deblender and its documentation will be before agreeing to that. In particular:

            1. Does the "complete rewrite" literally involve throwing away everything and starting again with completely new algorithms? If not, it seems like anything we can capture on this new algorithm now will continue to be useful;
            2. While I think Lauren is asking for documentation on this ticket that helps her understand the impact of the work that has been carried out, I'd actually like to see documentation we can present to folks trying to use the deblender to get work done in the future;
            3. There is some documentation in the doc/design.tex file written by Dustin. That hasn't been touched in a couple of years; I've not read it properly, I don't know if it reflects reality, and I'm not sure if it's actually in a format that we'd find useful, but we should at least consider updating it to reflect this new functionality.
            Show
            swinbank John Swinbank added a comment - - edited I'd be fine with a few "before" and "after" figures attached to the ticket. Hmm, I'm not sure I would. At least, I'd like to discuss in more detail what our longer term plans for the deblender and its documentation will be before agreeing to that. In particular: Does the "complete rewrite" literally involve throwing away everything and starting again with completely new algorithms? If not, it seems like anything we can capture on this new algorithm now will continue to be useful; While I think Lauren is asking for documentation on this ticket that helps her understand the impact of the work that has been carried out, I'd actually like to see documentation we can present to folks trying to use the deblender to get work done in the future; There is some documentation in the doc/design.tex file written by Dustin. That hasn't been touched in a couple of years; I've not read it properly, I don't know if it reflects reality, and I'm not sure if it's actually in a format that we'd find useful, but we should at least consider updating it to reflect this new functionality.

              People

              • Assignee:
                rearmstr Bob Armstrong
                Reporter:
                swinbank John Swinbank
                Reviewers:
                Lauren MacArthur
                Watchers:
                Bob Armstrong, John Swinbank, Lauren MacArthur
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel