Fix Version/s: None
Component/s: Developer Infrastructure, QA
This epic covers work to deliver the following improvements to the SQuaSH prototype stood up in X16:
- drilldown 1 level (time series->histogram)
- multiple testdata options (requires jenkins, backend, dashboard extension)
- processccd + validate_drp pseudo-workflow
- pseudo-provenance (track manifest.txt - real LSST provenance system will be swapped in for extensive functionality when available)
- is cloned by
DM-7605 SQuaSH ad-hoc usability improvements
I have tested the deployment on the staging environment and ignoring some minor UX nitpicks it's ready to greenlight this for deployment in the next SQuaSH upgrade window.
The "drilldown" supports allows the inspection of sources that were used in calculating a particular metric in plots similar to those presented in http://dmtn-008.lsst.io/en/latest/ . It also allows for an interactive change of the cut-off to indicate the relative contribution of these sources in linked graphs. The CI system is mocking a workflow that links the processing of data with an afterburner calculating the metrics. The "provenance" of the metrics is tracked so that the user can be taken to the code changes that were involved in the metric shown in the regression graph. Additionally, extensive re-engineering to decompose the application in separate bokeh apps and to support the validate_base split-off from validate_drp was performed.
Great work Angelo. Looking forward to seeing it in prod.
In this epic we extended SQuaSH to support multiple datasets and shipped the "Code Changes" feature and implemented the validate_drp plots using Bokeh as a proof of concept.
Examples of validate_drp plots using Bokeh are available in my testing instance: