Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4887

Refactor measurement afterburners into a new plugin system


    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_base
    • Labels:
    • Templates:
    • Story Points:
    • Sprint:
      DRP X16-2
    • Team:
      Data Release Production


      Some of the operations we currently run as part of measurement (or would like to) share some features that make them a bit different from most plugin algorithms:

      • They must be run after at least some other high-level plugins, and may be run after all of them.
      • They do not require access to pixel data, as they derive their outputs entirely from other plugins' catalog outputs.
      • They may require an aggregation stage of some sort to be run on the regular plugin output before they can be run.

      Some examples include:

      • Star/Galaxy classification (with training done after measurement and before classification).
      • Applying aperture corrections (estimating the correction must be done first).
      • BFD's P, Q, R statistics (requires a prior estimated from deep data).

      We should move these algorithms to a new plugin system that's run by a new subtask, allowing these plugins to be run entirely separately from SingleFrameMeasurementTask. This will simplify some of the currently contorted logic required to make S/G classification happen after aperture correction, while making room for hierarchical inference algorithms like BFD and Bayesian S/G classification in the future.

      (We will not be able to support BFD immediately, as this will also require changes to our parallelization approach, but this will be a step in the right direction).

      This work should probably be delayed until after the HSC merge and Russell Owen's rewrite of ProcessCcdTask are complete, but it's conceivable that this refactoring could solve emergent problems there and be worth doing earlier as a result.


          Issue Links



              • Assignee:
                nlust Nate Lust
                jbosch Jim Bosch
                Paul Price
                Bob Armstrong, Jim Bosch, John Swinbank, Lauren MacArthur, Nate Lust, Paul Price
              • Votes:
                0 Vote for this issue
                6 Start watching this issue


                • Created:

                  Summary Panel