Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8477

SQuaSH in-task metric capture

    XMLWordPrintable

    Details

    • Epic Name:
      sqre-s17-squash-1
    • Story Points:
      67.5
    • WBS:
      1.02C.10.02
    • Team:
      SQuaRE
    • Cycle:
      Spring 2017

      Description

      This epic covers the next round of features to SQuaSH, SQuaRE's quality control harness. Following consultation with the Pipeline teams, the main goal will be to allow the capture of metrics from in-task execution (extending the current functionality, which involves capture from afterburners).

      The specific deliverables will be clarified after the architectural sprint, DM-8474.

      JS 33% AF 33% AT 33%

        Attachments

          Issue Links

          is blocked by

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. DM-8476 DM Management/Community support ad-hoc requests

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-8359 Port ctrl_pool to log package

          • Done
          relates to

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. DM-9539 jointcal validation framework integration (continued from S17)

          • Done

          Milestone - An overall project milestone DLP-352 Metrics of Interest integrated in running stack or post-tests

          • Major - Major loss of function.
          • Closed

          Stories in Epic (Custom Issue Matrix)

          Key Summary Story Points Assignee Status
           
          		DM-10357

          Verification reporting facility in lsst.verify framework

          		 3.5 Jonathan Sick Done
           
          		DM-10306

          Verification Framework metadata system

          		 2.8 Jonathan Sick Done
           
          		DM-10261

          Port post-qa into lsst.verify framework

          		 2.8 Jonathan Sick Done
           
          		DM-10199

          Improve Measurement class integration in lsst.verify framework

          		 5.6 Jonathan Sick Done
           
          		DM-10195

          Improve comparison handling in Name and SpecificationSet classes of verify framework

          		 0.5 Jonathan Sick Done
           
          		DM-10113

          Rename validate_base to verify and validate_metrics to verify_metrics

          		 3.2 Jonathan Sick Done
           
          		DM-10095

          Fix jointcal verification framework YAML specification definitions

          		 0.2 Jonathan Sick Done
           
          		DM-10089

          Simplify validate_base Blob into a concrete collection type

          		 1 Jonathan Sick Done
           
          		DM-10747

          Verification Framework presentation

          		 1.4 Jonathan Sick Done
           
          		DM-10625

          lsst.verify unit test failure unique to Jenkins env

          		 1.2 Jonathan Sick Done
           
          		DM-10428

          SQR-019 Technote demonstrating the lsst.verify API

          		 3.5 Jonathan Sick Done
           
          		DM-9606

          Fix validate.metrics Python import

          		 0.3 Jonathan Sick Done
           
          		DM-9559

          Create a specifications access API for validate_metrics specs

          		 6.7 Jonathan Sick Done
           
          		DM-9532

          Create a metric definitions access API

          		 1.8 Jonathan Sick Done
           
          		DM-9530

          Create metric definitions package

          		 6.3 Jonathan Sick Done
           
          		DM-9492

          Metricdeviation parameters are wrong

          		 0.5 Adam Thornton Done
           
          		DM-9879

          validate_base coding style and unittest cleanups

          		 0.4 Jonathan Sick Done
           
          		DM-9877

          Refactor existing validate_base specification into a ThresholdSpecification

          		 1.25 Jonathan Sick Done
           
          		DM-9850

          Create a Name class for semantic metric and specification name handling

          		 3.3 Jonathan Sick Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            frossie Frossie Economou added a comment -

            I have placed a block from DM-8359 after discussions with John Swinbank and Kian-Tat Limas it is likely that the primary method for capture will be via the stack logging facility.

            Show
            frossie Frossie Economou added a comment - I have placed a block from DM-8359 after discussions with John Swinbank and Kian-Tat Lim as it is likely that the primary method for capture will be via the stack logging facility.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Note that this will be the merge ticket for our work on validate_base, to prevent our changes from conflicting with Michael Wood-Vasey's validate_drp work.

            Show
            Parejkoj John Parejko added a comment - Note that this will be the merge ticket for our work on validate_base, to prevent our changes from conflicting with Michael Wood-Vasey 's validate_drp work.
            Hide
            Permalink
            frossie Frossie Economou added a comment -

            This was a significant amount of work to develop a framework for describing metrics and specifications for verification work. This work allows for the decoupling of the metric generation from the code that is used to verify the metrics, monitor their value and generate report.

            This framework gives us the ability to create machine-parsable specification libraries in a modular way. It includes metric definitions, subsets of data that are appropriate to those metrics and an extensible class for specifying boundaries (eg thresholds) for those specifications.

            A summary of this work is in the slides attached to DM-10747 and documented in SQR-019.

            Show
            frossie Frossie Economou added a comment - This was a significant amount of work to develop a framework for describing metrics and specifications for verification work. This work allows for the decoupling of the metric generation from the code that is used to verify the metrics, monitor their value and generate report. This framework gives us the ability to create machine-parsable specification libraries in a modular way. It includes metric definitions, subsets of data that are appropriate to those metrics and an extensible class for specifying boundaries (eg thresholds) for those specifications. A summary of this work is in the slides attached to DM-10747 and documented in SQR-019.

              People

              Assignee:
              jsick Jonathan Sick
              Reporter:
              frossie Frossie Economou
              Watchers:
              Frossie Economou, John Parejko
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.