Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7889

Activate HSC afterburner functionality

    XMLWordPrintable

    Details

      Description

      We have just merged the functionality of the HSC afterburner (DM-6784, DM-6785). We need to add configuration options in obs_subaru to activate the features in the coadds.

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            This was a LOT more work than I was expecting because, in addition to adding a configuration file, it also required:

            • adapting meas_extensions_convolved to work with forced measurement (I changed it to wrap a base class, similar to how C++ plugins work; Jim Bosch may be interested in adapting the wrapper into meas_base for general use).
            • adapting meas_base to do undeblended measurement in the ForcedMeasurementTask as well as the SingleFrameMeasurementTask.
            • changing meas_extensions_photometryKron to make the source of the Kron radius (for forced measurement) configurable (because there's no undeblended_ext_photometryKron_KronFlux_radius column in the reference).

            Lauren MacArthur, would you mind having a look at these, please? If you feel it's a bit much, we can split it up.

            price@price-laptop:~/LSST/obs/subaru (tickets/DM-7889=) $ git sub
            commit 0e9fb31f1d7ae0a4ab85401a6c9c979b3364cbcc
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Fri Nov 4 14:17:45 2016 -0400
             
                activate measurements used in s16a production afterburner
                
                The HSC s16a production's afterburner included two measurements features
                which have been ported to LSST: measurement on the undeblended image (to
                allow potentially useful measurements where the deblender fails, and to
                compare with software that doesn't use a deblender), and measurement of
                convolved fluxes (to provide fluxes in multiple bands with a common
                seeing, e.g., for photometric redshifts). This patch activates these
                features for HSC and Suprime-Cam.
             
             config/convolvedFluxes.py     |  9 +++++++++
             config/forcedPhotCoadd.py     | 27 +++++++++++++++++++++++++++
             config/measureCoaddSources.py |  1 +
             config/processCcd.py          |  6 ++++++
             4 files changed, 43 insertions(+)
             
             
            price@price-laptop:~/LSST/meas/extensions/convolved (tickets/DM-7889=) $ git subcommit 5c904dca6e5f32c4c3a3f1b31e4e5afa44a42a43
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Mon Nov 7 20:49:48 2016 -0600
             
                wrap ConvolvedFluxPlugin to allow use in forced measurement
                
                Change the ConvolvedFluxPlugin class into a base class that we then
                wrap with the appropriate plugin base class. There's also a little
                bit of fiddling required to give the forced measurement a slightly
                different signature.
                
                In order to do the forced measurement, we need to add transformation
                from the reference frame to the measurement frame. Since we're
                already doing forced measurement (center is provided, aperture shape
                and radius are provided), moved the calculation into a 'measureForced'
                method, and have 'measure' call that with a null transformation.
                
                Modified the test to check that forced measurement also works.
             
             python/lsst/meas/extensions/convolved/convolved.py | 203 ++++++++++++++-------
             tests/test_convolved.py                            |  76 ++++++--
             2 files changed, 201 insertions(+), 78 deletions(-)
             
             
            price@price-laptop:~/LSST/meas/base (tickets/DM-7889=) $ git sub
            commit 7d67323726849bf986d3c0a7261dbdd8e32e2c94
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Mon Nov 7 20:47:05 2016 -0600
             
                refactor undeblended measurement into BaseMeasurementTask
                
                This makes it available in the ForcedMeasurementTask as well as
                SingleFrameMeasurementTask. Unfortunately, the actual measurement
                iteration cannot live in BaseMeasurementTask because the forced
                measurement requires iterating through two catalogs in sync, while
                the single frame measurement only has one.
             
             python/lsst/meas/base/baseMeasurement.py   | 24 ++++++++++++++++++++++++
             python/lsst/meas/base/forcedMeasurement.py | 13 +++++++++++++
             python/lsst/meas/base/sfm.py               | 30 +++++++-----------------------
             3 files changed, 44 insertions(+), 23 deletions(-)
             
             
            price@price-laptop:~/LSST/meas/extensions/photometryKron (tickets/DM-7889=) $ git sub
            commit dab9e3e939b0b34bcadced5fd5a9faa493ca8eda
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue Nov 15 21:25:37 2016 -0500
             
                make source of Kron radius configurable for forced mode
                
                The source of the Kron radius was hard-wired to _name+"_radius",
                but this doesn't work for the undeblended version (which has a
                different name that doesn't exist in the reference catalog).
                Making the source configurable solves this problem.
             
             include/lsst/meas/extensions/photometryKron.h | 5 ++++-
             src/KronPhotometry.cc                         | 2 +-
             2 files changed, 5 insertions(+), 2 deletions(-)
            

            Show
            price Paul Price added a comment - This was a LOT more work than I was expecting because, in addition to adding a configuration file, it also required: adapting meas_extensions_convolved to work with forced measurement (I changed it to wrap a base class, similar to how C++ plugins work; Jim Bosch may be interested in adapting the wrapper into meas_base for general use). adapting meas_base to do undeblended measurement in the ForcedMeasurementTask as well as the SingleFrameMeasurementTask . changing meas_extensions_photometryKron to make the source of the Kron radius (for forced measurement) configurable (because there's no undeblended_ext_photometryKron_KronFlux_radius column in the reference). Lauren MacArthur , would you mind having a look at these, please? If you feel it's a bit much, we can split it up. price@price-laptop:~/LSST/obs/subaru (tickets/DM-7889=) $ git sub commit 0e9fb31f1d7ae0a4ab85401a6c9c979b3364cbcc Author: Paul Price <price@astro.princeton.edu> Date: Fri Nov 4 14:17:45 2016 -0400   activate measurements used in s16a production afterburner The HSC s16a production's afterburner included two measurements features which have been ported to LSST: measurement on the undeblended image (to allow potentially useful measurements where the deblender fails, and to compare with software that doesn't use a deblender), and measurement of convolved fluxes (to provide fluxes in multiple bands with a common seeing, e.g., for photometric redshifts). This patch activates these features for HSC and Suprime-Cam.   config/convolvedFluxes.py | 9 +++++++++ config/forcedPhotCoadd.py | 27 +++++++++++++++++++++++++++ config/measureCoaddSources.py | 1 + config/processCcd.py | 6 ++++++ 4 files changed, 43 insertions(+)     price@price-laptop:~/LSST/meas/extensions/convolved (tickets/DM-7889=) $ git subcommit 5c904dca6e5f32c4c3a3f1b31e4e5afa44a42a43 Author: Paul Price <price@astro.princeton.edu> Date: Mon Nov 7 20:49:48 2016 -0600   wrap ConvolvedFluxPlugin to allow use in forced measurement Change the ConvolvedFluxPlugin class into a base class that we then wrap with the appropriate plugin base class. There's also a little bit of fiddling required to give the forced measurement a slightly different signature. In order to do the forced measurement, we need to add transformation from the reference frame to the measurement frame. Since we're already doing forced measurement (center is provided, aperture shape and radius are provided), moved the calculation into a 'measureForced' method, and have 'measure' call that with a null transformation. Modified the test to check that forced measurement also works.   python/lsst/meas/extensions/convolved/convolved.py | 203 ++++++++++++++------- tests/test_convolved.py | 76 ++++++-- 2 files changed, 201 insertions(+), 78 deletions(-)     price@price-laptop:~/LSST/meas/base (tickets/DM-7889=) $ git sub commit 7d67323726849bf986d3c0a7261dbdd8e32e2c94 Author: Paul Price <price@astro.princeton.edu> Date: Mon Nov 7 20:47:05 2016 -0600   refactor undeblended measurement into BaseMeasurementTask This makes it available in the ForcedMeasurementTask as well as SingleFrameMeasurementTask. Unfortunately, the actual measurement iteration cannot live in BaseMeasurementTask because the forced measurement requires iterating through two catalogs in sync, while the single frame measurement only has one.   python/lsst/meas/base/baseMeasurement.py | 24 ++++++++++++++++++++++++ python/lsst/meas/base/forcedMeasurement.py | 13 +++++++++++++ python/lsst/meas/base/sfm.py | 30 +++++++----------------------- 3 files changed, 44 insertions(+), 23 deletions(-)     price@price-laptop:~/LSST/meas/extensions/photometryKron (tickets/DM-7889=) $ git sub commit dab9e3e939b0b34bcadced5fd5a9faa493ca8eda Author: Paul Price <price@astro.princeton.edu> Date: Tue Nov 15 21:25:37 2016 -0500   make source of Kron radius configurable for forced mode The source of the Kron radius was hard-wired to _name+"_radius", but this doesn't work for the undeblended version (which has a different name that doesn't exist in the reference catalog). Making the source configurable solves this problem.   include/lsst/meas/extensions/photometryKron.h | 5 ++++- src/KronPhotometry.cc | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)
            Hide
            price Paul Price added a comment -

            David Reiss, would you mind reviewing this, please? Lauren MacArthur is swamped.

            Show
            price Paul Price added a comment - David Reiss , would you mind reviewing this, please? Lauren MacArthur is swamped.
            Hide
            reiss David Reiss added a comment -

            Sorry for the delay. This was quite an extensive set of changes but I only have a few minor comments (in the pull requests). Assuming this all passes Jenkins, I'd say it's good to go.

            Show
            reiss David Reiss added a comment - Sorry for the delay. This was quite an extensive set of changes but I only have a few minor comments (in the pull requests). Assuming this all passes Jenkins, I'd say it's good to go.
            Hide
            price Paul Price added a comment -

            Thanks David Reiss.

            I've addressed the two small comments on meas_base, rebased everything against origin/master, and am in the process of doing a final Jenkins run before merging.

            Show
            price Paul Price added a comment - Thanks David Reiss . I've addressed the two small comments on meas_base, rebased everything against origin/master , and am in the process of doing a final Jenkins run before merging.
            Hide
            price Paul Price added a comment -

            Jenkins passed.

            Merged to master.

            Show
            price Paul Price added a comment - Jenkins passed . Merged to master.

              People

              Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Reviewers:
              David Reiss
              Watchers:
              David Reiss, Lauren MacArthur, Paul Price
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.