The middleware team's approach is to write a separate class adapter for each high-level CmdLineTask (including CoaddTask, ProcessCcdTask, and IsrTask). I gather that subtasks will interact with their parent tasks like before, i.e. they will not be aware of the PipelineTask interface at all. The adapter will request inputs through, e.g., PipelineTask.getInputDatasetTypes, though it is not clear for what purpose; the adapter must be able to function without a QuantumGraphGenerator to match up the inputs and outputs.

The proposed design is fairly labor-intensive and requires a custom class for each adaptee, making it scale poorly to an indefinite (and hopefully growing) number of MetricTasks. Therefore, it is still worthwhile to research our own solution, particularly since we can use known properties of MetricTask to simplify the design (e.g., no MetricTask will create catalogs, so a specialized adapter need not support catalog schemas).

TL;DR: it is almost certainly possible to create an adapter as proposed in DMTN-098, but it will require a lot of some boilerplate.

My intent was that the adapter could be implemented as a Python class decorator, something like:

Gen3Task = PipelineTaskAdapter([extra info])(OriginalTaskClass)

pipeline.task[1] = Gen3Task

pipeline.task[1].config.[field] = [configuration info]

I assumed that a Pipeline will be configured using the existing Config framework, much like subtasks are. This is in principle possible: a Pipeline is a list of TaskDef, each of which includes a task type object and a config. However, it's not clear how a Pipeline will be configured in the final system; I suspect the middleware group has not yet thought about a user interface.

The current placeholder (CmdLineFwk) is configured by passing task names through command line arguments, and requires that all tasks be importable classes. The decorator approach can be made to work with such a system, but it requires promoting the Gen3Task = PipelineTaskAdapter(...) declaration from a local (possibly anonymous) object in a config file to a publicly visible object in some Python module, adding a lot of API clutter.

We can avoid supporting some optional PipelineTask features on the grounds that MetricTask will never need them. However, even a minimal PipelineTask still needs a substantial amount of information that cannot be derived from the original Gen 2 task:

• input and output dataset types: there must be one for each task input as well as the (single) output, and each must specify three fields: the type name, the data units (e.g., "ccd", "visit", "tract"), and the Butler 3 storage class. Of these three, only the type names can be inferred from the wrapped MetricTask.
  • It may be possible to preemptively include this information in the Gen 2 configs, as appears to have been done in DM-15663. However, I'm not completely sure this won't cause problems with the config is loaded without access to a Butler 3 repository. Done in DM-16822, no ill effects so far.
• PipelineTaskConfig.quantum: this appears to be a replacement for user-provided data IDs. This field must specify a list of data units (presumably not redundant with those in the input spec), and may provide an SQL query that selects the data to be processed. It may be possible to have the latter default to empty, then overwrite it in the user config file. Given that CmdLineFwk bypasses this field, and it only appears in one example (where it is set to None), I don't have a clear picture of how PipelineTaskConfig.quantum.sql is meant to be used. Nate Lust says this field should not be exposed to, let alone set by, the task user.

In short, we may need to treat adapted MetricTasks as if they were classes in their own right, and we need to provide a substantial amount of information either as arguments to the adapter or as mandatory config options. Both add complexity to a feature that was intended to keep the code simple.