Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16822

Use pipe.base.*DatasetConfig in MetricTask configs

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ap_verify, verify
    • Labels:
      None
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      AP S19-2
    • Team:
      Alert Production

      Description

      Currently, the classes in lsst.verify.compatibility do not use any Gen 3 elements, including lsst.pipe.base.InputDatasetConfig and lsst.pipe.base.OutputDatasetConfig. However, Christopher Waters suggested that these config classes can be used in Gen 2 task configs without inadvertent side effects. Adopting them as part of the MetricTask API now will make it much easier to transition to Gen 3.

      This ticket covers the following work:

      • Update MetricTask documentation to require that subclasses use *DatasetConfig appropriately.
      • Rewrite MetricTask input/output methods to use the configs in a way that is compatible with Gen 2 repositories.
      • Rewrite TimingMetricTask (and any other extant MetricTasks) in terms of *DatasetConfig. This should reduce the amount of source code at the cost of an extra config field.

        Attachments

          Issue Links

          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-16017 Prototype a metrics-handling Task

          • Done
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-16503 Investigate requirements for Butler 3 interoperability

          • Done

            Activity

            Ascending order - Click to sort in descending order
            No builds found.
            krzys Krzysztof Findeisen created issue -
            krzys Krzysztof Findeisen made changes -
            Field Original Value New Value
            Epic Link DM-16713 [ 235321 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-16017 [ DM-16017 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-16503 [ DM-16503 ]
            krzys Krzysztof Findeisen made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment -

            Hi Christopher Waters, would you be willing to review this 100-line change? This is Gen 2 use of InputDatasetConfig, with the expectation that individual MetricTask subclasses can be migrated or adapted to Gen 3 later (ideally, migration would just be a matter of switching their base class to a Gen 3 MetricTask, once it exists).

            Show
            krzys Krzysztof Findeisen added a comment - Hi Christopher Waters , would you be willing to review this 100-line change? This is Gen 2 use of InputDatasetConfig , with the expectation that individual MetricTask subclasses can be migrated or adapted to Gen 3 later (ideally, migration would just be a matter of switching their base class to a Gen 3 MetricTask , once it exists).
            krzys Krzysztof Findeisen made changes -
            Reviewers Christopher Waters [ cwaters ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            Permalink
            czw Christopher Waters added a comment -

            Looks good.

            Beware of potential code changes before this is converted to a full `pipelineTask`, but that's just a warning that the Gen3 code is not totally finalized.

            Show
            czw Christopher Waters added a comment - Looks good. Beware of potential code changes before this is converted to a full `pipelineTask`, but that's just a warning that the Gen3 code is not totally finalized.
            czw Christopher Waters made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment -

            I know, but I figure if the configs are stable enough to use in major tasks, then adopting them now will be less work than trying to retrofit them later. Thanks for the review!

            Show
            krzys Krzysztof Findeisen added a comment - I know, but I figure if the configs are stable enough to use in major tasks, then adopting them now will be less work than trying to retrofit them later. Thanks for the review!
            krzys Krzysztof Findeisen made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Christopher Waters
              Watchers:
              Christopher Waters, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.