# Port parent/child measurement from HSC

## Details

• Type: Story
• Status: Done
• Priority: Major
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Templates:
• Story Points:
5
• Sprint:
DRP F16-3, DRP F16-4, DRP F16-5
• Team:
Data Release Production

## Description

The deblender sometimes gets into trouble with cluster galaxies, and the deblended fluxes aren't accurate. In that case it helps to have measurements on the image without any deblending having been performed. This is a feature used in HSC's mid-2016 production run afterburner, ticket HSC-1400. This feature should be ported for use in LSST.

## Attachments

1. undeblended.png
19 kB
2. undeblended2.png
27 kB

## Activity

Hide
Paul Price added a comment -

I've got an implementation and will work on cleaning it up for submission. Here's a demonstration, showing that the ratio of PSF fluxes measured in the deblended (regular) and undeblended modes tracks the blendedness value.

Show
Paul Price added a comment - I've got an implementation and will work on cleaning it up for submission. Here's a demonstration, showing that the ratio of PSF fluxes measured in the deblended (regular) and undeblended modes tracks the blendedness value.
Hide
Paul Price added a comment -

Aperture correction is working now too. Here's another nice plot showing the relation between blendedness and the ratio of the deblended and undeblended photometry, this time as a function of magnitude. The cloud of blended sources at bright magnitudes are mostly due to saturated stars (e.g., the bleed trail splits the star into two sources which are bright and fairly well blended with each other).

Show
Paul Price added a comment - Aperture correction is working now too. Here's another nice plot showing the relation between blendedness and the ratio of the deblended and undeblended photometry, this time as a function of magnitude. The cloud of blended sources at bright magnitudes are mostly due to saturated stars (e.g., the bleed trail splits the star into two sources which are bright and fairly well blended with each other).
Hide
Paul Price added a comment -

Jim Bosch, can you have a look at what I've got so far? I would like to write a test for the feature and also activate the feature in obs_subaru before I'm done, but I would like to get your sign-off on the feature itself first. Changes are in meas_base. I've branched from the latest weekly instead of the most recent commit in order to make it possible to run with the shared stack, but I don't think that affects anything.

 price@price-laptop:~/LSST/meas/base (tickets/DM-6785=) $git sub commit 30b550c2894e936d85d34dc16e106e5b082a225b Author: Paul Price  Date: Wed Sep 7 15:58:04 2016 -0400    provide option to measure on undeblended image    In some cases, the deblender doesn't do a good job, or we want to  compare what's measured by our code with a different code that  doesn't do deblending; so we measure on the undeblended image.    Refactored the guts of the measurement (mainly the exception handling)  in the BaseMeasurementTask to make that easy. SingleFrameMeasurementTask  will now measure on the undeblended image using a new list of plugins  in SingleFrameMeasurementConfig.undeblended; the results will appear  in the catalog with columns named after the measurement algorithms but  with the SingleFrameMeasurementConfig.undeblendedPrefix prepended.    python/lsst/meas/base/baseMeasurement.py | 81 +++++++++++++++++++++++---------  python/lsst/meas/base/sfm.py | 32 ++++++++++++-  2 files changed, 89 insertions(+), 24 deletions(-)   commit bf7b746aa9a3cf9f3fdc893b57aaeafd5a888f27 Author: Paul Price  Date: Thu Sep 8 20:57:05 2016 -0400    applyApCorr: allow proxy aperture corrections    Allow the aperture correction model for one measurement to be applied to  another. The aperture correction values are only recorded once.    We will use this feature to apply aperture corrections to the undeblended  measurements.    python/lsst/meas/base/applyApCorr.py | 63 +++++++++++++++++++++++-------------  1 file changed, 40 insertions(+), 23 deletions(-)   commit 3e966e90a70334e8c00a1d29163998ad543dc22c Author: Paul Price  Date: Thu Sep 8 21:07:14 2016 -0400    applyApCorr: correct undeblended measurements with proxy model    Use the aperture correction model from the deblended measurements as a  proxy to correct the undeblended measurements. The mechanism used is a  bit more general, doing an aperture correction with a proxy model for any  prefix specified in the new config parameter.    python/lsst/meas/base/applyApCorr.py | 11 +++++++++++  1 file changed, 11 insertions(+)  Show Paul Price added a comment - Jim Bosch , can you have a look at what I've got so far? I would like to write a test for the feature and also activate the feature in obs_subaru before I'm done, but I would like to get your sign-off on the feature itself first. Changes are in meas_base. I've branched from the latest weekly instead of the most recent commit in order to make it possible to run with the shared stack, but I don't think that affects anything. price@price-laptop:~/LSST/meas/base (tickets/DM-6785=)$ git sub commit 30b550c2894e936d85d34dc16e106e5b082a225b Author: Paul Price <price@astro.princeton.edu> Date: Wed Sep 7 15:58:04 2016 -0400   provide option to measure on undeblended image In some cases, the deblender doesn't do a good job, or we want to compare what's measured by our code with a different code that doesn't do deblending; so we measure on the undeblended image. Refactored the guts of the measurement (mainly the exception handling) in the BaseMeasurementTask to make that easy. SingleFrameMeasurementTask will now measure on the undeblended image using a new list of plugins in SingleFrameMeasurementConfig.undeblended; the results will appear in the catalog with columns named after the measurement algorithms but with the SingleFrameMeasurementConfig.undeblendedPrefix prepended.   python/lsst/meas/base/baseMeasurement.py | 81 +++++++++++++++++++++++--------- python/lsst/meas/base/sfm.py | 32 ++++++++++++- 2 files changed, 89 insertions(+), 24 deletions(-)   commit bf7b746aa9a3cf9f3fdc893b57aaeafd5a888f27 Author: Paul Price <price@astro.princeton.edu> Date: Thu Sep 8 20:57:05 2016 -0400   applyApCorr: allow proxy aperture corrections Allow the aperture correction model for one measurement to be applied to another. The aperture correction values are only recorded once. We will use this feature to apply aperture corrections to the undeblended measurements.   python/lsst/meas/base/applyApCorr.py | 63 +++++++++++++++++++++++------------- 1 file changed, 40 insertions(+), 23 deletions(-)   commit 3e966e90a70334e8c00a1d29163998ad543dc22c Author: Paul Price <price@astro.princeton.edu> Date: Thu Sep 8 21:07:14 2016 -0400   applyApCorr: correct undeblended measurements with proxy model Use the aperture correction model from the deblended measurements as a proxy to correct the undeblended measurements. The mechanism used is a bit more general, doing an aperture correction with a proxy model for any prefix specified in the new config parameter.   python/lsst/meas/base/applyApCorr.py | 11 +++++++++++ 1 file changed, 11 insertions(+)
Hide
Paul Price added a comment -

After looking at what I had, Jim suggested I use a plugin composed of plugins to implement the functionality instead. I did that and found that the circular dependencies (from putting the UndeblendedPlugin into the registry and containing a registry) was very difficult to reconcile, at least without changing a lot of things. I'm reverting to the previous proposal. I just need to clean it up and add some tests.

Show
Paul Price added a comment - After looking at what I had, Jim suggested I use a plugin composed of plugins to implement the functionality instead. I did that and found that the circular dependencies (from putting the UndeblendedPlugin into the registry and containing a registry) was very difficult to reconcile, at least without changing a lot of things. I'm reverting to the previous proposal. I just need to clean it up and add some tests.
Hide
Paul Price added a comment - - edited

Jim Bosch, I've added a test and the aliases you requested. Would you please give this a proper review? It has passed Jenkins on python 3 and python 2.

 price@price-laptop:~/LSST/meas/base (tickets/DM-6785=) $git sub commit b64fe9448d2f86925f2e341d03af803dfd3c3413 Author: Paul Price  Date: Wed Sep 7 15:58:04 2016 -0400    provide option to measure on undeblended image    In some cases, the deblender doesn't do a good job, or we want to  compare what's measured by our code with a different code that  doesn't do deblending; so we measure on the undeblended image.    Refactored the guts of the measurement (mainly the exception handling)  in the BaseMeasurementTask to make that easy. SingleFrameMeasurementTask  will now measure on the undeblended image using a new list of plugins  in SingleFrameMeasurementConfig.undeblended; the results will appear  in the catalog with columns named after the measurement algorithms but  with the SingleFrameMeasurementConfig.undeblendedPrefix prepended.    python/lsst/meas/base/baseMeasurement.py | 81 +++++++++++++++++++++++---------  python/lsst/meas/base/sfm.py | 32 ++++++++++++-  2 files changed, 89 insertions(+), 24 deletions(-)   commit 0a5ef13cd73fd3519262fd4ffa7e9a87d19408eb Author: Paul Price  Date: Thu Sep 8 20:57:05 2016 -0400    applyApCorr: allow proxy aperture corrections    Allow the aperture correction model for one measurement to be applied to  another. The aperture correction values are only recorded once.    We will use this feature to apply aperture corrections to the undeblended  measurements.    python/lsst/meas/base/applyApCorr.py | 66 ++++++++++++++++++++++++------------  1 file changed, 45 insertions(+), 21 deletions(-)   commit 8e729344cfe6a550fae03734ae1ad7d9ed3561bf Author: Paul Price  Date: Thu Sep 8 21:07:14 2016 -0400    applyApCorr: correct undeblended measurements with proxy model    Use the aperture correction model from the deblended measurements as a  proxy to correct the undeblended measurements. The mechanism used is a  bit more general, doing an aperture correction with a proxy model for any  prefix specified in the new config parameter.    python/lsst/meas/base/applyApCorr.py | 11 +++++++++++  1 file changed, 11 insertions(+)   commit bd34fb788d675125ff28e4babb5f598c90fa4810 Author: Paul Price  Date: Fri Sep 23 16:29:40 2016 -0400    add test for undeblended measurement and aperture correction    tests/testMeasureSources.py | 119 ++++++++++++++++++++++++++++++++++++++++++++  1 file changed, 119 insertions(+)  Show Paul Price added a comment - - edited Jim Bosch , I've added a test and the aliases you requested. Would you please give this a proper review? It has passed Jenkins on python 3 and python 2 . price@price-laptop:~/LSST/meas/base (tickets/DM-6785=)$ git sub commit b64fe9448d2f86925f2e341d03af803dfd3c3413 Author: Paul Price <price@astro.princeton.edu> Date: Wed Sep 7 15:58:04 2016 -0400   provide option to measure on undeblended image In some cases, the deblender doesn't do a good job, or we want to compare what's measured by our code with a different code that doesn't do deblending; so we measure on the undeblended image. Refactored the guts of the measurement (mainly the exception handling) in the BaseMeasurementTask to make that easy. SingleFrameMeasurementTask will now measure on the undeblended image using a new list of plugins in SingleFrameMeasurementConfig.undeblended; the results will appear in the catalog with columns named after the measurement algorithms but with the SingleFrameMeasurementConfig.undeblendedPrefix prepended.   python/lsst/meas/base/baseMeasurement.py | 81 +++++++++++++++++++++++--------- python/lsst/meas/base/sfm.py | 32 ++++++++++++- 2 files changed, 89 insertions(+), 24 deletions(-)   commit 0a5ef13cd73fd3519262fd4ffa7e9a87d19408eb Author: Paul Price <price@astro.princeton.edu> Date: Thu Sep 8 20:57:05 2016 -0400   applyApCorr: allow proxy aperture corrections Allow the aperture correction model for one measurement to be applied to another. The aperture correction values are only recorded once. We will use this feature to apply aperture corrections to the undeblended measurements.   python/lsst/meas/base/applyApCorr.py | 66 ++++++++++++++++++++++++------------ 1 file changed, 45 insertions(+), 21 deletions(-)   commit 8e729344cfe6a550fae03734ae1ad7d9ed3561bf Author: Paul Price <price@astro.princeton.edu> Date: Thu Sep 8 21:07:14 2016 -0400   applyApCorr: correct undeblended measurements with proxy model Use the aperture correction model from the deblended measurements as a proxy to correct the undeblended measurements. The mechanism used is a bit more general, doing an aperture correction with a proxy model for any prefix specified in the new config parameter.   python/lsst/meas/base/applyApCorr.py | 11 +++++++++++ 1 file changed, 11 insertions(+)   commit bd34fb788d675125ff28e4babb5f598c90fa4810 Author: Paul Price <price@astro.princeton.edu> Date: Fri Sep 23 16:29:40 2016 -0400   add test for undeblended measurement and aperture correction   tests/testMeasureSources.py | 119 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+)
Hide
Jim Bosch added a comment -

I've left some comments on the GitHub PR. Feel free to push back if you disagree or create a new issue for anything you think is a good idea but out of scope.

Show
Jim Bosch added a comment - I've left some comments on the GitHub PR. Feel free to push back if you disagree or create a new issue for anything you think is a good idea but out of scope.
Hide
Paul Price added a comment -

Jim Bosch, I believe I've addressed all your comments. Would you like to have a look at the result?

Show
Paul Price added a comment - Jim Bosch , I believe I've addressed all your comments. Would you like to have a look at the result?
Hide
Jim Bosch added a comment -

Looks good, merge whenever you're ready.

Show
Jim Bosch added a comment - Looks good, merge whenever you're ready.
Hide
Paul Price added a comment -

Thanks Jim.

Merged to master.

Show
Paul Price added a comment - Thanks Jim. Merged to master.

## People

• Assignee:
Paul Price
Reporter:
Paul Price
Reviewers:
Jim Bosch
Watchers:
Jim Bosch, Paul Price