Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16017

Prototype a metrics-handling Task

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Alert Production, verify
    • Labels:
      None
    • Story Points:
      6
    • Epic Link:
    • Sprint:
      AP F18-6, AP S19-1
    • Team:
      Alert Production

      Description

      Implement a minimal version of a measurement Task, and transfer responsibility for some of ap.verify.measurements' metrics to the task. Exact specifications TBD, but the initial version might only support e.g. source catalog queries.

        Attachments

          Issue Links

            Activity

            No builds found.
            krzys Krzysztof Findeisen created issue -
            krzys Krzysztof Findeisen made changes -
            Field Original Value New Value
            Epic Link DM-14431 [ 80156 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-16016 [ DM-16016 ]
            krzys Krzysztof Findeisen made changes -
            Risk Score 0
            krzys Krzysztof Findeisen made changes -
            Rank Ranked higher
            krzys Krzysztof Findeisen made changes -
            Summary Prototype a measurement Task Prototype a metrics-handling Task
            krzys Krzysztof Findeisen made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            Depending on the resolution of DM-16503, the MetricTask API may need to change (e.g., getInputDatasetTypes -> getInputDatasetTypesGen2). However, at the time this issue is completed, there should be few enough MetricTask subclasses that the extra work is manageable.
            I realized later that there's a simpler, and much more robust way to structure the code that eliminates the need for an everything-at-once conversion – have two MetricTask interfaces, one that's always Gen 2 and one that's always Gen 3. I've edited DMTN-098 accordingly.

            (Naturally, only the Gen 2 interface will be implemented now)

            Show
            krzys Krzysztof Findeisen added a comment - - edited Depending on the resolution of DM-16503 , the MetricTask API may need to change (e.g., getInputDatasetTypes -> getInputDatasetTypesGen2 ). However, at the time this issue is completed, there should be few enough MetricTask subclasses that the extra work is manageable. I realized later that there's a simpler, and much more robust way to structure the code that eliminates the need for an everything-at-once conversion – have two MetricTask interfaces, one that's always Gen 2 and one that's always Gen 3. I've edited DMTN-098 accordingly. (Naturally, only the Gen 2 interface will be implemented now)
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-16503 [ DM-16503 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-16535 [ DM-16535 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-16536 [ DM-16536 ]
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            The final scope of this ticket will be:

            • The full Gen 2 MetricTask interface will be implemented and tested.
            • Only timing metrics will be implemented.
            • The Configs for the timing metrics will be as simple as possible, with no attention paid to minimizing future code duplication or making the Gen 3 transition easier.
            • MetricsControllerTask will have a hard-coded list of timing metrics, with no support for registering additional metrics.
            • MetricTask and MetricsControllerTask will have hard-coded equivalents for the metadata requested in DM-11321 and DM-16333.

            These limitations will be lifted by later work, linked as blocked by this ticket.

            Show
            krzys Krzysztof Findeisen added a comment - - edited The final scope of this ticket will be: The full Gen 2 MetricTask interface will be implemented and tested. Only timing metrics will be implemented. The Configs for the timing metrics will be as simple as possible, with no attention paid to minimizing future code duplication or making the Gen 3 transition easier. MetricsControllerTask will have a hard-coded list of timing metrics, with no support for registering additional metrics. MetricTask and MetricsControllerTask will have hard-coded equivalents for the metadata requested in DM-11321 and DM-16333 . These limitations will be lifted by later work, linked as blocked by this ticket.
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-16642 [ DM-16642 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is triggered by RFC-550 [ RFC-550 ]
            swinbank John Swinbank made changes -
            Epic Link DM-14431 [ 80156 ] DM-16713 [ 235321 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-16728 [ DM-16728 ]
            krzys Krzysztof Findeisen made changes -
            Component/s verify [ 15038 ]
            swinbank John Swinbank made changes -
            Sprint AP F18-6 [ 751 ] AP F18-6, AP S19-1 [ 751, 825 ]
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            Hi Jonathan Sick and Christopher Waters, would you be willing to review this ticket? Since you have complementary perspectives (verify vs. Gen 3 middleware), I'd appreciate it if you could both look over the changes to all three packages (verify, ap_verify, and dmtn-098).

            Show
            krzys Krzysztof Findeisen added a comment - - edited Hi Jonathan Sick and Christopher Waters , would you be willing to review this ticket? Since you have complementary perspectives (verify vs. Gen 3 middleware), I'd appreciate it if you could both look over the changes to all three packages ( verify , ap_verify , and dmtn-098 ).
            krzys Krzysztof Findeisen made changes -
            Reviewers Christopher Waters, Jonathan Sick [ cwaters, jsick ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue has to be done after DM-16519 [ DM-16519 ]
            Hide
            czw Christopher Waters added a comment -

            Given the ticket scope, I don't think I have any major concerns for this with respect to Gen2.  I'm not sure how difficult the conversion to Gen3 will be, but I've made a quick note where I have the largest concern about future trouble.  You're likely already aware of this, but I wanted to raise the point.

            Show
            czw Christopher Waters added a comment - Given the ticket scope, I don't think I have any major concerns for this with respect to Gen2.  I'm not sure how difficult the conversion to Gen3 will be, but I've made a quick note where I have the largest concern about future trouble.  You're likely already aware of this, but I wanted to raise the point.
            czw Christopher Waters made changes -
            Reviewers Christopher Waters, Jonathan Sick [ cwaters, jsick ] Jonathan Sick [ jsick ]
            Hide
            jsick Jonathan Sick added a comment -

            This looks good, thanks.

            Show
            jsick Jonathan Sick added a comment - This looks good, thanks.
            jsick Jonathan Sick made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            krzys Krzysztof Findeisen added a comment -

            Thank you both for your feedback!

            Show
            krzys Krzysztof Findeisen added a comment - Thank you both for your feedback!
            krzys Krzysztof Findeisen made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-16822 [ DM-16822 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-16826 [ DM-16826 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-16827 [ DM-16827 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-16828 [ DM-16828 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-16835 [ DM-16835 ]

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Jonathan Sick
              Watchers:
              Christopher Waters, Eric Bellm, John Swinbank, Jonathan Sick, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.