Details

    • Type: Story
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_base
    • Labels:
    • Templates:
    • Story Points:
      5
    • Sprint:
      DRP F16-3, DRP F16-4, DRP F16-5
    • Team:
      Data Release Production

      Description

      The deblender sometimes gets into trouble with cluster galaxies, and the deblended fluxes aren't accurate. In that case it helps to have measurements on the image without any deblending having been performed. This is a feature used in HSC's mid-2016 production run afterburner, ticket HSC-1400. This feature should be ported for use in LSST.

        Issue Links

          Activity

          Hide
          price Paul Price added a comment -

          I've got an implementation and will work on cleaning it up for submission. Here's a demonstration, showing that the ratio of PSF fluxes measured in the deblended (regular) and undeblended modes tracks the blendedness value.

          Show
          price Paul Price added a comment - I've got an implementation and will work on cleaning it up for submission. Here's a demonstration, showing that the ratio of PSF fluxes measured in the deblended (regular) and undeblended modes tracks the blendedness value.
          Hide
          price Paul Price added a comment -

          Aperture correction is working now too. Here's another nice plot showing the relation between blendedness and the ratio of the deblended and undeblended photometry, this time as a function of magnitude. The cloud of blended sources at bright magnitudes are mostly due to saturated stars (e.g., the bleed trail splits the star into two sources which are bright and fairly well blended with each other).

          Show
          price Paul Price added a comment - Aperture correction is working now too. Here's another nice plot showing the relation between blendedness and the ratio of the deblended and undeblended photometry, this time as a function of magnitude. The cloud of blended sources at bright magnitudes are mostly due to saturated stars (e.g., the bleed trail splits the star into two sources which are bright and fairly well blended with each other).
          Hide
          price Paul Price added a comment -

          Jim Bosch, can you have a look at what I've got so far? I would like to write a test for the feature and also activate the feature in obs_subaru before I'm done, but I would like to get your sign-off on the feature itself first. Changes are in meas_base. I've branched from the latest weekly instead of the most recent commit in order to make it possible to run with the shared stack, but I don't think that affects anything.

          price@price-laptop:~/LSST/meas/base (tickets/DM-6785=) $ git sub
          commit 30b550c2894e936d85d34dc16e106e5b082a225b
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Wed Sep 7 15:58:04 2016 -0400
           
              provide option to measure on undeblended image
              
              In some cases, the deblender doesn't do a good job, or we want to
              compare what's measured by our code with a different code that
              doesn't do deblending; so we measure on the undeblended image.
              
              Refactored the guts of the measurement (mainly the exception handling)
              in the BaseMeasurementTask to make that easy. SingleFrameMeasurementTask
              will now measure on the undeblended image using a new list of plugins
              in SingleFrameMeasurementConfig.undeblended; the results will appear
              in the catalog with columns named after the measurement algorithms but
              with the SingleFrameMeasurementConfig.undeblendedPrefix prepended.
           
           python/lsst/meas/base/baseMeasurement.py | 81 +++++++++++++++++++++++---------
           python/lsst/meas/base/sfm.py             | 32 ++++++++++++-
           2 files changed, 89 insertions(+), 24 deletions(-)
           
          commit bf7b746aa9a3cf9f3fdc893b57aaeafd5a888f27
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Thu Sep 8 20:57:05 2016 -0400
           
              applyApCorr: allow proxy aperture corrections
              
              Allow the aperture correction model for one measurement to be applied to
              another. The aperture correction values are only recorded once.
              
              We will use this feature to apply aperture corrections to the undeblended
              measurements.
           
           python/lsst/meas/base/applyApCorr.py | 63 +++++++++++++++++++++++-------------
           1 file changed, 40 insertions(+), 23 deletions(-)
           
          commit 3e966e90a70334e8c00a1d29163998ad543dc22c
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Thu Sep 8 21:07:14 2016 -0400
           
              applyApCorr: correct undeblended measurements with proxy model
              
              Use the aperture correction model from the deblended measurements as a
              proxy to correct the undeblended measurements. The mechanism used is a
              bit more general, doing an aperture correction with a proxy model for any
              prefix specified in the new config parameter.
           
           python/lsst/meas/base/applyApCorr.py | 11 +++++++++++
           1 file changed, 11 insertions(+)
          

          Show
          price Paul Price added a comment - Jim Bosch , can you have a look at what I've got so far? I would like to write a test for the feature and also activate the feature in obs_subaru before I'm done, but I would like to get your sign-off on the feature itself first. Changes are in meas_base. I've branched from the latest weekly instead of the most recent commit in order to make it possible to run with the shared stack, but I don't think that affects anything. price@price-laptop:~/LSST/meas/base (tickets/DM-6785=) $ git sub commit 30b550c2894e936d85d34dc16e106e5b082a225b Author: Paul Price <price@astro.princeton.edu> Date: Wed Sep 7 15:58:04 2016 -0400   provide option to measure on undeblended image In some cases, the deblender doesn't do a good job, or we want to compare what's measured by our code with a different code that doesn't do deblending; so we measure on the undeblended image. Refactored the guts of the measurement (mainly the exception handling) in the BaseMeasurementTask to make that easy. SingleFrameMeasurementTask will now measure on the undeblended image using a new list of plugins in SingleFrameMeasurementConfig.undeblended; the results will appear in the catalog with columns named after the measurement algorithms but with the SingleFrameMeasurementConfig.undeblendedPrefix prepended.   python/lsst/meas/base/baseMeasurement.py | 81 +++++++++++++++++++++++--------- python/lsst/meas/base/sfm.py | 32 ++++++++++++- 2 files changed, 89 insertions(+), 24 deletions(-)   commit bf7b746aa9a3cf9f3fdc893b57aaeafd5a888f27 Author: Paul Price <price@astro.princeton.edu> Date: Thu Sep 8 20:57:05 2016 -0400   applyApCorr: allow proxy aperture corrections Allow the aperture correction model for one measurement to be applied to another. The aperture correction values are only recorded once. We will use this feature to apply aperture corrections to the undeblended measurements.   python/lsst/meas/base/applyApCorr.py | 63 +++++++++++++++++++++++------------- 1 file changed, 40 insertions(+), 23 deletions(-)   commit 3e966e90a70334e8c00a1d29163998ad543dc22c Author: Paul Price <price@astro.princeton.edu> Date: Thu Sep 8 21:07:14 2016 -0400   applyApCorr: correct undeblended measurements with proxy model Use the aperture correction model from the deblended measurements as a proxy to correct the undeblended measurements. The mechanism used is a bit more general, doing an aperture correction with a proxy model for any prefix specified in the new config parameter.   python/lsst/meas/base/applyApCorr.py | 11 +++++++++++ 1 file changed, 11 insertions(+)
          Hide
          price Paul Price added a comment -

          After looking at what I had, Jim suggested I use a plugin composed of plugins to implement the functionality instead. I did that and found that the circular dependencies (from putting the UndeblendedPlugin into the registry and containing a registry) was very difficult to reconcile, at least without changing a lot of things. I'm reverting to the previous proposal. I just need to clean it up and add some tests.

          Show
          price Paul Price added a comment - After looking at what I had, Jim suggested I use a plugin composed of plugins to implement the functionality instead. I did that and found that the circular dependencies (from putting the UndeblendedPlugin into the registry and containing a registry) was very difficult to reconcile, at least without changing a lot of things. I'm reverting to the previous proposal. I just need to clean it up and add some tests.
          Hide
          price Paul Price added a comment - - edited

          Jim Bosch, I've added a test and the aliases you requested. Would you please give this a proper review? It has passed Jenkins on python 3 and python 2.

          price@price-laptop:~/LSST/meas/base (tickets/DM-6785=) $ git sub
          commit b64fe9448d2f86925f2e341d03af803dfd3c3413
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Wed Sep 7 15:58:04 2016 -0400
           
              provide option to measure on undeblended image
              
              In some cases, the deblender doesn't do a good job, or we want to
              compare what's measured by our code with a different code that
              doesn't do deblending; so we measure on the undeblended image.
              
              Refactored the guts of the measurement (mainly the exception handling)
              in the BaseMeasurementTask to make that easy. SingleFrameMeasurementTask
              will now measure on the undeblended image using a new list of plugins
              in SingleFrameMeasurementConfig.undeblended; the results will appear
              in the catalog with columns named after the measurement algorithms but
              with the SingleFrameMeasurementConfig.undeblendedPrefix prepended.
           
           python/lsst/meas/base/baseMeasurement.py | 81 +++++++++++++++++++++++---------
           python/lsst/meas/base/sfm.py             | 32 ++++++++++++-
           2 files changed, 89 insertions(+), 24 deletions(-)
           
          commit 0a5ef13cd73fd3519262fd4ffa7e9a87d19408eb
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Thu Sep 8 20:57:05 2016 -0400
           
              applyApCorr: allow proxy aperture corrections
              
              Allow the aperture correction model for one measurement to be applied to
              another. The aperture correction values are only recorded once.
              
              We will use this feature to apply aperture corrections to the undeblended
              measurements.
           
           python/lsst/meas/base/applyApCorr.py | 66 ++++++++++++++++++++++++------------
           1 file changed, 45 insertions(+), 21 deletions(-)
           
          commit 8e729344cfe6a550fae03734ae1ad7d9ed3561bf
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Thu Sep 8 21:07:14 2016 -0400
           
              applyApCorr: correct undeblended measurements with proxy model
              
              Use the aperture correction model from the deblended measurements as a
              proxy to correct the undeblended measurements. The mechanism used is a
              bit more general, doing an aperture correction with a proxy model for any
              prefix specified in the new config parameter.
           
           python/lsst/meas/base/applyApCorr.py | 11 +++++++++++
           1 file changed, 11 insertions(+)
           
          commit bd34fb788d675125ff28e4babb5f598c90fa4810
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Fri Sep 23 16:29:40 2016 -0400
           
              add test for undeblended measurement and aperture correction
           
           tests/testMeasureSources.py | 119 ++++++++++++++++++++++++++++++++++++++++++++
           1 file changed, 119 insertions(+)
          

          Show
          price Paul Price added a comment - - edited Jim Bosch , I've added a test and the aliases you requested. Would you please give this a proper review? It has passed Jenkins on python 3 and python 2 . price@price-laptop:~/LSST/meas/base (tickets/DM-6785=) $ git sub commit b64fe9448d2f86925f2e341d03af803dfd3c3413 Author: Paul Price <price@astro.princeton.edu> Date: Wed Sep 7 15:58:04 2016 -0400   provide option to measure on undeblended image In some cases, the deblender doesn't do a good job, or we want to compare what's measured by our code with a different code that doesn't do deblending; so we measure on the undeblended image. Refactored the guts of the measurement (mainly the exception handling) in the BaseMeasurementTask to make that easy. SingleFrameMeasurementTask will now measure on the undeblended image using a new list of plugins in SingleFrameMeasurementConfig.undeblended; the results will appear in the catalog with columns named after the measurement algorithms but with the SingleFrameMeasurementConfig.undeblendedPrefix prepended.   python/lsst/meas/base/baseMeasurement.py | 81 +++++++++++++++++++++++--------- python/lsst/meas/base/sfm.py | 32 ++++++++++++- 2 files changed, 89 insertions(+), 24 deletions(-)   commit 0a5ef13cd73fd3519262fd4ffa7e9a87d19408eb Author: Paul Price <price@astro.princeton.edu> Date: Thu Sep 8 20:57:05 2016 -0400   applyApCorr: allow proxy aperture corrections Allow the aperture correction model for one measurement to be applied to another. The aperture correction values are only recorded once. We will use this feature to apply aperture corrections to the undeblended measurements.   python/lsst/meas/base/applyApCorr.py | 66 ++++++++++++++++++++++++------------ 1 file changed, 45 insertions(+), 21 deletions(-)   commit 8e729344cfe6a550fae03734ae1ad7d9ed3561bf Author: Paul Price <price@astro.princeton.edu> Date: Thu Sep 8 21:07:14 2016 -0400   applyApCorr: correct undeblended measurements with proxy model Use the aperture correction model from the deblended measurements as a proxy to correct the undeblended measurements. The mechanism used is a bit more general, doing an aperture correction with a proxy model for any prefix specified in the new config parameter.   python/lsst/meas/base/applyApCorr.py | 11 +++++++++++ 1 file changed, 11 insertions(+)   commit bd34fb788d675125ff28e4babb5f598c90fa4810 Author: Paul Price <price@astro.princeton.edu> Date: Fri Sep 23 16:29:40 2016 -0400   add test for undeblended measurement and aperture correction   tests/testMeasureSources.py | 119 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+)
          Hide
          jbosch Jim Bosch added a comment -

          I've left some comments on the GitHub PR. Feel free to push back if you disagree or create a new issue for anything you think is a good idea but out of scope.

          Show
          jbosch Jim Bosch added a comment - I've left some comments on the GitHub PR. Feel free to push back if you disagree or create a new issue for anything you think is a good idea but out of scope.
          Hide
          price Paul Price added a comment -

          Jim Bosch, I believe I've addressed all your comments. Would you like to have a look at the result?

          Show
          price Paul Price added a comment - Jim Bosch , I believe I've addressed all your comments. Would you like to have a look at the result?
          Hide
          jbosch Jim Bosch added a comment -

          Looks good, merge whenever you're ready.

          Show
          jbosch Jim Bosch added a comment - Looks good, merge whenever you're ready.
          Hide
          price Paul Price added a comment -

          Thanks Jim.

          Merged to master.

          Show
          price Paul Price added a comment - Thanks Jim. Merged to master.

            People

            • Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Reviewers:
              Jim Bosch
              Watchers:
              Jim Bosch, Paul Price
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development

                  Agile