Details
-
Type:
RFC
-
Status: Withdrawn
-
Resolution: Done
-
Component/s: DM
-
Labels:None
Description
As part of the ap_verify work we are writing a pipeline-agnostic framework for the code that actually computes metrics. The framework will provide code for computing metrics given an arbitrary (processed) repository as well as the ability to integrate said code into Gen 2 and Gen 3 pipelines. The system is being prototyped in DM-16017.
We'd like to put the new framework in its own lsst_distrib package to keep it independent of specific pipelines, the more abstract verify framework, or the metrics definitions in verify_metrics. We propose to keep with the current verify naming convention by calling the new package verify_measurements, but we understand that the term "measurement" is falling out of favor and are open to alternatives.
Attachments
Issue Links
Activity
Hi Krzysztof Findeisen, right I should not have used TimingMetricTask specifically. The general idea of using mixins in lsst.verify is to add metric related code that is not in MetricTask but that still could be used in subclasses of MetricTask or other classes.
Example:
from lsst.verify import MetricTask, OneMetricMixin, AnotherMetricMixin
|
|
|
class MyMetricTask(MetricTask, OneMetricMixin, AnotherMetricMixin):
|
pass
|
|
|
class AnotherTask(OneMetricMixin):
|
pass
|
sorry, didn't want to deviate that much from the scope of the RFC.
I just realized that TimingMetricTask, or anything else that uses verify classes, must not go in pipe_base, because that would introduce a circular dependency between pipe_base and verify. 
That suggests we really should have a new package for "generic" concrete MetricTasks, but it also suggests that the details should wait until I've done more prototyping. I'll withdraw this RFC for now.
Krzysztof Findeisen
added a comment - I just realized that TimingMetricTask , or anything else that uses verify classes, must not go in pipe_base , because that would introduce a circular dependency between pipe_base and verify .
That suggests we really should have a new package for "generic" concrete MetricTasks , but it also suggests that the details should wait until I've done more prototyping. I'll withdraw this RFC for now. Per Jonathan Sick's recommendation, I'll put MetricTask and its support code into lsst.verify, and include specific MetricTasks in pipelines packages (mostly ap_association and ap_verify for our existing metrics). If in the future we still have a case for a package dedicated to metric-computing code, I'll open another RFC then.
Krzysztof Findeisen
added a comment - - edited Per Jonathan Sick 's recommendation, I'll put MetricTask and its support code into lsst.verify , and include specific MetricTasks in pipelines packages (mostly ap_association and ap_verify for our existing metrics). If in the future we still have a case for a package dedicated to metric-computing code, I'll open another RFC then. Thanks Krzysztof Findeisen, I appreciate your willingness to change plans slightly and do this.
Hi Angelo Fausti, I'm not sure what you mean by making it a mixin. TimingMetricTask is a MetricTask specialized for computing timing metrics, and in the interest of flexibility (and that of doing one thing and doing it well) each MetricTask object should be responsible for computing the value of one, and only one, metric.