Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12033

Add section to LDM-294 describing regression monitoring

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Add a paragraph to LDM-294 about how the SST should handle regression monitoring. The draft is:

      All KPMs and other regression monitoring metrics should be presented to the SQuaSH tool and recalculated on a regular cadence (daily if possible). On a biweekly bases, the SST will review the traces of the various metrics to attempt to identify regressions in performance over the past two week period. If regressions are identified in any of the metrics, the SST will identify a team to take lead on investigating the regression and will communicate with the DM Project Manager to schedule the necessary resources.

        Attachments

          Activity

          Hide
          mjuric Mario Juric added a comment -

          Thanks Simon, but I continue to worry that a committee shouldn't have a responsibility, but a person; still like my proposed wording better (with the modification pointed out by Wil):

          All KPMs and other regression monitoring metrics will be calculated on a regular cadence (daily if possible). They are monitored by the SQuaRE scientist, with status periodically reported to the System Science Team (SST). The SQuaRE scientist brings up any major regressions to the attention of the SST, along with an initial assessment of the problem. The SST may recommend further actions to the DM Project Manager and/or Scientist, if necessary. These include performing additional testing, broader root cause analysis, documenting the regression, or recommendations on the priority of fixing the regression relative to presently scheduled work.

          Think it mirrors how we operate with the RFCs where Tim is the responsible person to keep things going, with unresolved issues flowed up to the DMLT for further discussion & recommendation to Wil.

          Show
          mjuric Mario Juric added a comment - Thanks Simon, but I continue to worry that a committee shouldn't have a responsibility, but a person; still like my proposed wording better (with the modification pointed out by Wil): All KPMs and other regression monitoring metrics will be calculated on a regular cadence (daily if possible). They are monitored by the SQuaRE scientist, with status periodically reported to the System Science Team (SST). The SQuaRE scientist brings up any major regressions to the attention of the SST, along with an initial assessment of the problem. The SST may recommend further actions to the DM Project Manager and/or Scientist, if necessary. These include performing additional testing, broader root cause analysis, documenting the regression, or recommendations on the priority of fixing the regression relative to presently scheduled work. Think it mirrors how we operate with the RFCs where Tim is the responsible person to keep things going, with unresolved issues flowed up to the DMLT for further discussion & recommendation to Wil.
          Hide
          womullan Wil O'Mullane added a comment -

          Taking some of the proposed actions and prefixing with Simon's original idea "The SST has the responsibility of monitoring the overall system for whether.." would reflect everything .

          I suggest we have invested enough effort in crafting this phrase at this point.

          Show
          womullan Wil O'Mullane added a comment - Taking some of the proposed actions and prefixing with Simon's original idea "The SST has the responsibility of monitoring the overall system for whether.." would reflect everything . I suggest we have invested enough effort in crafting this phrase at this point.
          Hide
          krughoff Simon Krughoff added a comment -

          I've adopted Mario's wording with some clarifications.

          Show
          krughoff Simon Krughoff added a comment - I've adopted Mario's wording with some clarifications.
          Hide
          womullan Wil O'Mullane added a comment -

          looks good to me

          Show
          womullan Wil O'Mullane added a comment - looks good to me
          Hide
          krughoff Simon Krughoff added a comment -

          merged.

          Show
          krughoff Simon Krughoff added a comment - merged.

            People

            • Assignee:
              krughoff Simon Krughoff
              Reporter:
              krughoff Simon Krughoff
              Reviewers:
              Wil O'Mullane
              Watchers:
              Leanne Guy, Mario Juric, Simon Krughoff, Wil O'Mullane
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel