# Implement validate_drp static plots in Bokeh as proof-of-concept for SQUASH

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s: None
• Labels:
• Story Points:
10
• Team:
SQuaRE

#### Description

This ticket will implement a plot from validate_drp in the QA Dashboard as a proof-of-concept for how existing matplotlib plots can be re-implemented in Bokeh with data from the QA database.

Stretch goals (maybe for a future ticket) will be to overplot the validate_drp output of one job against another’s to understand performance changes.

#### Activity

Hide
Angelo Fausti added a comment -

Jonathan Sick this can be interesting for the serialization of plot information in validate_drp, there is an open issue for bokeh to support JSON data structures following the Vega-Lite specification
https://github.com/bokeh/bokeh/issues/4844

Show
Angelo Fausti added a comment - Jonathan Sick this can be interesting for the serialization of plot information in validate_drp, there is an open issue for bokeh to support JSON data structures following the Vega-Lite specification https://github.com/bokeh/bokeh/issues/4844
Hide
Jonathan Sick added a comment -

That's really interesting! We could add an class, kind of like Blobs in the currently-proposed measurement API, that could describe plots as vega-lite JSON. That way we could replace the matplotlib code in validate_drp with vega-lite that could be rendered both client-side from validate_drp and server side with bokeh. I guess we could create a ticket to start making and shipping Vega-Lite to SQUASH now, and then once Bokeh is able to render Vega-Lite we should switch the detailed data plots to that.

Show
Jonathan Sick added a comment - That's really interesting! We could add an class, kind of like Blobs in the currently-proposed measurement API, that could describe plots as vega-lite JSON. That way we could replace the matplotlib code in validate_drp with vega-lite that could be rendered both client-side from validate_drp and server side with bokeh. I guess we could create a ticket to start making and shipping Vega-Lite to SQUASH now, and then once Bokeh is able to render Vega-Lite we should switch the detailed data plots to that.
Hide
Angelo Fausti added a comment -

I agree, we can start by doing some of the validate_drp static plots following Vega-Lite specification.

Show
Angelo Fausti added a comment - I agree, we can start by doing some of the validate_drp static plots following Vega-Lite specification.
Hide
Angelo Fausti added a comment - - edited

Changing bokeh app to directory style for convenience to handle multiple apps:
http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#directory-format

Proposed reoganization has three independent apps served by the same bokeh server, for astrometry plots, photometry plots and regression testing

   astrometry/  main.py  theme.yaml   photometry/  main.py  theme.yaml   regression/ (old metrics app)  main.py  theme.yaml    

Show
Angelo Fausti added a comment - - edited Changing bokeh app to directory style for convenience to handle multiple apps: http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#directory-format Proposed reoganization has three independent apps served by the same bokeh server, for astrometry plots, photometry plots and regression testing   astrometry/ main.py theme.yaml photometry/ main.py theme.yaml   regression/ (old metrics app) main.py theme.yaml
Hide
Jonathan Sick added a comment - - edited

in the final JSON sample for the full REST json, blobs is an object/dict. Do you want the keys of this dict to be the blob identifier? this would make it easy to look up from a measurement.

If so, the actions that post-qa needs to do to shim it's native format to the SQUASH format is:

1. Convert the blobs array to an object keyed by identifier.
2. Convert the measurements array to an an array of objects with fields: 1) metric name, and 2) array of corresponding measurements.

Thinking of the last one, it may make more sense to simply make measurements an object keyed by metric names,

 {  "measurements": {  {"AM1": [], <- array of measurement objects  ...}  } } 

Moving discussion to DM-7043.

Show
Jonathan Sick added a comment - - edited in the final JSON sample for the full REST json, blobs is an object/dict. Do you want the keys of this dict to be the blob identifier ? this would make it easy to look up from a measurement. If so, the actions that post-qa needs to do to shim it's native format to the SQUASH format is: Convert the blobs array to an object keyed by identifier . Convert the measurements array to an an array of objects with fields: 1) metric name, and 2) array of corresponding measurements. Thinking of the last one, it may make more sense to simply make measurements an object keyed by metric names, { "measurements": { {"AM1": [], <- array of measurement objects ...} } } Moving discussion to DM-7043 .
Hide
Angelo Fausti added a comment -

In order to extend squash drill down capabilities we implemented validade_drp static plots in bokeh

They are available in my test environment:

note that the parameters metric, dataset and Job Id are fixed in those examples.

In DM-8478 we will continue this implementation connecting those plots with the measurements in the regression app

https://angelo-squash-bokeh.lsst.codes/regression

Show
Angelo Fausti added a comment - In order to extend squash drill down capabilities we implemented validade_drp static plots in bokeh They are available in my test environment: https://angelo-squash-bokeh.lsst.codes/astrometry https://angelo-squash-bokeh.lsst.codes/photometry note that the parameters metric, dataset and Job Id are fixed in those examples. In DM-8478 we will continue this implementation connecting those plots with the measurements in the regression app https://angelo-squash-bokeh.lsst.codes/regression

#### People

Assignee:
Angelo Fausti
Reporter:
Jonathan Sick
Watchers:
Angelo Fausti, Jonathan Sick