Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4887

Refactor measurement afterburners into a new plugin system

    Details

    • Type: Story
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_base
    • Labels:
      None
    • Templates:
    • Story Points:
      8
    • Sprint:
      DRP X16-2
    • Team:
      Data Release Production

      Description

      Some of the operations we currently run as part of measurement (or would like to) share some features that make them a bit different from most plugin algorithms:

      • They must be run after at least some other high-level plugins, and may be run after all of them.
      • They do not require access to pixel data, as they derive their outputs entirely from other plugins' catalog outputs.
      • They may require an aggregation stage of some sort to be run on the regular plugin output before they can be run.

      Some examples include:

      • Star/Galaxy classification (with training done after measurement and before classification).
      • Applying aperture corrections (estimating the correction must be done first).
      • BFD's P, Q, R statistics (requires a prior estimated from deep data).

      We should move these algorithms to a new plugin system that's run by a new subtask, allowing these plugins to be run entirely separately from SingleFrameMeasurementTask. This will simplify some of the currently contorted logic required to make S/G classification happen after aperture correction, while making room for hierarchical inference algorithms like BFD and Bayesian S/G classification in the future.

      (We will not be able to support BFD immediately, as this will also require changes to our parallelization approach, but this will be a step in the right direction).

      This work should probably be delayed until after the HSC merge and Russell Owen's rewrite of ProcessCcdTask are complete, but it's conceivable that this refactoring could solve emergent problems there and be worth doing earlier as a result.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  nlust Nate Lust
                  Reporter:
                  jbosch Jim Bosch
                  Reviewers:
                  Paul Price
                  Watchers:
                  Bob Armstrong, Jim Bosch, John Swinbank, Lauren MacArthur, Nate Lust, Paul Price
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  6 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel