Fix Version/s: None
Component/s: Developer Infrastructure, Verification
This epic covers the next round of features to SQuaSH, SQuaRE's quality control harness. Following consultation with the Pipeline teams, the main goal will be to allow the capture of metrics from in-task execution (extending the current functionality, which involves capture from afterburners).
The specific deliverables will be clarified after the architectural sprint,
JS 33% AF 33% AT 33%
Note that this will be the merge ticket for our work on validate_base, to prevent our changes from conflicting with Michael Wood-Vasey's validate_drp work.
This was a significant amount of work to develop a framework for describing metrics and specifications for verification work. This work allows for the decoupling of the metric generation from the code that is used to verify the metrics, monitor their value and generate report.
This framework gives us the ability to create machine-parsable specification libraries in a modular way. It includes metric definitions, subsets of data that are appropriate to those metrics and an extensible class for specifying boundaries (eg thresholds) for those specifications.
A summary of this work is in the slides attached to
DM-10747 and documented in SQR-019.
I have placed a block from
DM-8359after discussions with John Swinbank and Kian-Tat Limas it is likely that the primary method for capture will be via the stack logging facility.