Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12243

Add base_InputCount to forced coadd measurement

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_base
    • Labels:
    • Team:
      External

      Description

      In HSC-1424, Mineo-san reports:

      Matsuoka-san wants to find drop-out objects by such a search condition as "the object was not detected in this band though there were two or more input images there." It is not feasible in hscPipe 5.4 since inputCount_value is zero if inputCount_flag_badCentroid is true, no matter how many inputs there actually were. This problem did not exist in hscPipe 4 because inputCount was measured also in forced measurement. In 5.4, inputCount is measured only in unforced measurement, and inputCount_flag_badCentroid can be true even if it is not in other bands. Please measure inputCount in forced measurement. ("Setting inputCount_value to a valid value no matter how bad the centroid is" may suffice though.)

        Attachments

          Activity

          Hide
          price Paul Price added a comment -

          Nate Lust, would you mind reviewing this please?

          price@pap-laptop:~/LSST/meas_base (tickets/DM-12243 $=) $ git sub
          commit f94f1b0c32e30a7eddf1391977bd8e01c0e3b60d
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Mon Oct 16 18:49:36 2017 -0400
           
              add input count to forced coadd measurement
              
              Having it only in unforced measurement isn't as helpful because it can be
              skipped if the centroid is flagged, leaving us without a count measurement.
           
           python/lsst/meas/base/forcedPhotCoadd.py | 1 +
           1 file changed, 1 insertion(+)
           
          commit 5e673485206058bdac85f79b2708fca823c552cf (HEAD -> tickets/DM-12243, origin/tickets/DM-12243)
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Tue Oct 17 13:00:54 2017 -0400
           
              inputCount: only fail if the centroid is actually bad
              
              Even if the centroid has been declared to be bad (through the flag), still
              attempt to measure the number of inputs at the position, as this might
              provide something useful.
           
           python/lsst/meas/base/plugins.py |  5 +++--
           tests/test_InputCount.py         | 46 +++++++++++++++++++++++++++++-----------
           2 files changed, 37 insertions(+), 14 deletions(-)
          

          Show
          price Paul Price added a comment - Nate Lust , would you mind reviewing this please? price@pap-laptop:~/LSST/meas_base (tickets/DM-12243 $=) $ git sub commit f94f1b0c32e30a7eddf1391977bd8e01c0e3b60d Author: Paul Price <price@astro.princeton.edu> Date: Mon Oct 16 18:49:36 2017 -0400   add input count to forced coadd measurement Having it only in unforced measurement isn't as helpful because it can be skipped if the centroid is flagged, leaving us without a count measurement.   python/lsst/meas/base/forcedPhotCoadd.py | 1 + 1 file changed, 1 insertion(+)   commit 5e673485206058bdac85f79b2708fca823c552cf (HEAD -> tickets/DM-12243, origin/tickets/DM-12243) Author: Paul Price <price@astro.princeton.edu> Date: Tue Oct 17 13:00:54 2017 -0400   inputCount: only fail if the centroid is actually bad Even if the centroid has been declared to be bad (through the flag), still attempt to measure the number of inputs at the position, as this might provide something useful.   python/lsst/meas/base/plugins.py | 5 +++-- tests/test_InputCount.py | 46 +++++++++++++++++++++++++++++----------- 2 files changed, 37 insertions(+), 14 deletions(-)
          Hide
          price Paul Price added a comment -

          Ah, this is more complicated than simply adding the algorithm to the forced measurement config — the algorithm isn't defined as operating on forced measurement, only on single frame measurement.

          Taking this back so I can fix it.

          Show
          price Paul Price added a comment - Ah, this is more complicated than simply adding the algorithm to the forced measurement config — the algorithm isn't defined as operating on forced measurement, only on single frame measurement. Taking this back so I can fix it.
          Hide
          price Paul Price added a comment - - edited

          OK, this now passes Jenkins after some work to make plugins work on both single-frame and forced measurement.

          price@pap-laptop:~/LSST/meas_base (tickets/DM-12243 $=) $ git sub
          commit f94f1b0c32e30a7eddf1391977bd8e01c0e3b60d
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Mon Oct 16 18:49:36 2017 -0400
           
              add input count to forced coadd measurement
              
              Having it only in unforced measurement isn't as helpful because it can be
              skipped if the centroid is flagged, leaving us without a count measurement.
           
           python/lsst/meas/base/forcedPhotCoadd.py | 1 +
           1 file changed, 1 insertion(+)
           
          commit 5e673485206058bdac85f79b2708fca823c552cf
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Tue Oct 17 13:00:54 2017 -0400
           
              inputCount: only fail if the centroid is actually bad
              
              Even if the centroid has been declared to be bad (through the flag), still
              attempt to measure the number of inputs at the position, as this might
              provide something useful.
           
           python/lsst/meas/base/plugins.py |  5 +++--
           tests/test_InputCount.py         | 46 +++++++++++++++++++++++++++++-----------
           2 files changed, 37 insertions(+), 14 deletions(-)
           
          commit 1dd45f03aa32a52ec7188fb371dc4fc15c84d3c0
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Tue Oct 17 21:18:05 2017 -0400
           
              add standard __future__ imports
           
           python/lsst/meas/base/plugins.py  | 2 ++
           python/lsst/meas/base/wrappers.py | 2 ++
           2 files changed, 4 insertions(+)
           
          commit 355e8b223d21c7c51ab403190ffca486de659c8a
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Tue Oct 17 21:18:40 2017 -0400
           
              wrappers: add factories to produce SFM/forced plugins from generic
              
              Some measurement plugins are really generic in the sense that it
              doesn't matter whether they are run as a single-frame or forced
              plugin, but the plumbing is slightly different (most notably, the
              signature of the 'measure' method). To work around this, added a
              GenericPlugin class and methods that will use that to produce the
              appropriate SingleFramePlugin and ForcedPlugin classes so common
              code can be used for both.
           
           python/lsst/meas/base/wrappers.py | 208 +++++++++++++++++++++++++++++++++++++-
           1 file changed, 204 insertions(+), 4 deletions(-)
           
          commit 3789c89a43f749115c385409c68cd2176e07293a
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Tue Oct 17 21:23:29 2017 -0400
           
              plugins: convert variance and inputCount plugins to generic
              
              These previously only worked in single-frame measurement, but they would
              be useful in forced measurement too. Using the new GenericPlugin, we can
              use the same code to achieve both. This necessitated some small changes in
              the error handling.
           
           python/lsst/meas/base/plugins.py | 54 +++++++++++++++++++++-------------------
           tests/test_InputCount.py         |  2 +-
           tests/test_Variance.py           |  5 ++--
           3 files changed, 32 insertions(+), 29 deletions(-)
           
          commit 150813dc62f35875581ae9b45e2bcddfa1920715 (HEAD -> tickets/DM-12243, origin/tickets/DM-12243)
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Tue Oct 17 21:25:20 2017 -0400
           
              add variance measurement to forced coadd measurement
           
           python/lsst/meas/base/forcedPhotCoadd.py | 2 +-
           1 file changed, 1 insertion(+), 1 deletion(-)
          

          Show
          price Paul Price added a comment - - edited OK, this now passes Jenkins after some work to make plugins work on both single-frame and forced measurement. price@pap-laptop:~/LSST/meas_base (tickets/DM-12243 $=) $ git sub commit f94f1b0c32e30a7eddf1391977bd8e01c0e3b60d Author: Paul Price <price@astro.princeton.edu> Date: Mon Oct 16 18:49:36 2017 -0400   add input count to forced coadd measurement Having it only in unforced measurement isn't as helpful because it can be skipped if the centroid is flagged, leaving us without a count measurement.   python/lsst/meas/base/forcedPhotCoadd.py | 1 + 1 file changed, 1 insertion(+)   commit 5e673485206058bdac85f79b2708fca823c552cf Author: Paul Price <price@astro.princeton.edu> Date: Tue Oct 17 13:00:54 2017 -0400   inputCount: only fail if the centroid is actually bad Even if the centroid has been declared to be bad (through the flag), still attempt to measure the number of inputs at the position, as this might provide something useful.   python/lsst/meas/base/plugins.py | 5 +++-- tests/test_InputCount.py | 46 +++++++++++++++++++++++++++++----------- 2 files changed, 37 insertions(+), 14 deletions(-)   commit 1dd45f03aa32a52ec7188fb371dc4fc15c84d3c0 Author: Paul Price <price@astro.princeton.edu> Date: Tue Oct 17 21:18:05 2017 -0400   add standard __future__ imports   python/lsst/meas/base/plugins.py | 2 ++ python/lsst/meas/base/wrappers.py | 2 ++ 2 files changed, 4 insertions(+)   commit 355e8b223d21c7c51ab403190ffca486de659c8a Author: Paul Price <price@astro.princeton.edu> Date: Tue Oct 17 21:18:40 2017 -0400   wrappers: add factories to produce SFM/forced plugins from generic Some measurement plugins are really generic in the sense that it doesn't matter whether they are run as a single-frame or forced plugin, but the plumbing is slightly different (most notably, the signature of the 'measure' method). To work around this, added a GenericPlugin class and methods that will use that to produce the appropriate SingleFramePlugin and ForcedPlugin classes so common code can be used for both.   python/lsst/meas/base/wrappers.py | 208 +++++++++++++++++++++++++++++++++++++- 1 file changed, 204 insertions(+), 4 deletions(-)   commit 3789c89a43f749115c385409c68cd2176e07293a Author: Paul Price <price@astro.princeton.edu> Date: Tue Oct 17 21:23:29 2017 -0400   plugins: convert variance and inputCount plugins to generic These previously only worked in single-frame measurement, but they would be useful in forced measurement too. Using the new GenericPlugin, we can use the same code to achieve both. This necessitated some small changes in the error handling.   python/lsst/meas/base/plugins.py | 54 +++++++++++++++++++++------------------- tests/test_InputCount.py | 2 +- tests/test_Variance.py | 5 ++-- 3 files changed, 32 insertions(+), 29 deletions(-)   commit 150813dc62f35875581ae9b45e2bcddfa1920715 (HEAD -> tickets/DM-12243, origin/tickets/DM-12243) Author: Paul Price <price@astro.princeton.edu> Date: Tue Oct 17 21:25:20 2017 -0400   add variance measurement to forced coadd measurement   python/lsst/meas/base/forcedPhotCoadd.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
          Hide
          nlust Nate Lust added a comment -

          A few things up on github for you to look at

          Show
          nlust Nate Lust added a comment - A few things up on github for you to look at
          Hide
          price Paul Price added a comment -

          Made the change you suggested (great idea!) and now it passes Jenkins again.

          Show
          price Paul Price added a comment - Made the change you suggested (great idea!) and now it passes Jenkins again.
          Hide
          nlust Nate Lust added a comment -

          looks good, thanks for the changes

          Show
          nlust Nate Lust added a comment - looks good, thanks for the changes
          Hide
          price Paul Price added a comment -

          Thanks Nate.

          Merged to master.

          Show
          price Paul Price added a comment - Thanks Nate. Merged to master.

            People

            • Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Reviewers:
              Nate Lust
              Watchers:
              Nate Lust, Paul Price
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel