Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30869

Modernize MetricTask for better Gen 3 workflow

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: verify
    • Labels:
      None

      Description

      Change the specification for lsst.verify.MetricTask as described in RFC-788. In addition, make the following concrete code changes throughout Science Pipelines:

      • Remove MetricTask.runQuantum. Rewrite all known overrides of runQuantum to be based off of (and preferably delegate to) PipelineTask.runQuantum.
      • Stop supporting optional inputs in concrete MetricTasks. This code was only being used in unit tests, so it can be removed without harm.

        Attachments

          Issue Links

            Activity

            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            John Parejko discovered that lsst.verify.tasks.TimingMetricTask causes an empty quantum graph if the task being timed was never run. Investigate whether this is related to the introduction of optional inputs and/or NoWorkFound in the Middleware framework. Note that, in his scenario, the pipeline was structurally incapable of generating the expected metadata, while the "return if irrelevant" policy was intended to handle pipeline failures.

            Show
            krzys Krzysztof Findeisen added a comment - - edited John Parejko discovered that lsst.verify.tasks.TimingMetricTask causes an empty quantum graph if the task being timed was never run. Investigate whether this is related to the introduction of optional inputs and/or NoWorkFound in the Middleware framework. Note that, in his scenario, the pipeline was structurally incapable of generating the expected metadata, while the "return if irrelevant" policy was intended to handle pipeline failures.
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            Thanks for offering to review this, Jim Bosch! Note that there is one PR, ap_pipe#125, that didn't get picked up by Jira.

            Show
            krzys Krzysztof Findeisen added a comment - - edited Thanks for offering to review this, Jim Bosch ! Note that there is one PR, ap_pipe#125 , that didn't get picked up by Jira.
            Hide
            krzys Krzysztof Findeisen added a comment -

            Also, FTR: I had to drop the "Remove MetricTask.runQuantum" bullet after a Slack discussion revealed that it's still needed for None output support.

            Show
            krzys Krzysztof Findeisen added a comment - Also, FTR: I had to drop the "Remove MetricTask.runQuantum " bullet after a Slack discussion revealed that it's still needed for None output support.
            Hide
            jbosch Jim Bosch added a comment -

            Looks great!  Nice simplifications, and as usual your commit discipline and messages are much appreciated.  My only comment (on the ip_diffim PR) is perhaps the smallest, most trivial comment it is possible to make on a PR.

             

            Show
            jbosch Jim Bosch added a comment - Looks great!  Nice simplifications, and as usual your commit discipline and messages are much appreciated.  My only comment (on the ip_diffim PR) is perhaps the smallest, most trivial comment it is possible to make on a PR.  

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Jim Bosch
              Watchers:
              Jim Bosch, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.