Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5945

Implement validate_drp static plots in Bokeh as proof-of-concept for SQUASH

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      This ticket will implement a plot from validate_drp in the QA Dashboard as a proof-of-concept for how existing matplotlib plots can be re-implemented in Bokeh with data from the QA database.

      Stretch goals (maybe for a future ticket) will be to overplot the validate_drp output of one job against another’s to understand performance changes.

        Attachments

          Issue Links

            Activity

            Hide
            afausti Angelo Fausti added a comment -

            Jonathan Sick this can be interesting for the serialization of plot information in validate_drp, there is an open issue for bokeh to support JSON data structures following the Vega-Lite specification
            https://github.com/bokeh/bokeh/issues/4844

            Show
            afausti Angelo Fausti added a comment - Jonathan Sick this can be interesting for the serialization of plot information in validate_drp, there is an open issue for bokeh to support JSON data structures following the Vega-Lite specification https://github.com/bokeh/bokeh/issues/4844
            Hide
            jsick Jonathan Sick added a comment -

            That's really interesting! We could add an class, kind of like Blobs in the currently-proposed measurement API, that could describe plots as vega-lite JSON. That way we could replace the matplotlib code in validate_drp with vega-lite that could be rendered both client-side from validate_drp and server side with bokeh. I guess we could create a ticket to start making and shipping Vega-Lite to SQUASH now, and then once Bokeh is able to render Vega-Lite we should switch the detailed data plots to that.

            Show
            jsick Jonathan Sick added a comment - That's really interesting! We could add an class, kind of like Blobs in the currently-proposed measurement API, that could describe plots as vega-lite JSON. That way we could replace the matplotlib code in validate_drp with vega-lite that could be rendered both client-side from validate_drp and server side with bokeh. I guess we could create a ticket to start making and shipping Vega-Lite to SQUASH now, and then once Bokeh is able to render Vega-Lite we should switch the detailed data plots to that.
            Hide
            afausti Angelo Fausti added a comment -

            I agree, we can start by doing some of the validate_drp static plots following Vega-Lite specification.

            Show
            afausti Angelo Fausti added a comment - I agree, we can start by doing some of the validate_drp static plots following Vega-Lite specification.
            Hide
            afausti Angelo Fausti added a comment - - edited

            Changing bokeh app to directory style for convenience to handle multiple apps:
            http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#directory-format

            Proposed reoganization has three independent apps served by the same bokeh server, for astrometry plots, photometry plots and regression testing

             
            astrometry/
               main.py
               theme.yaml
               
            photometry/
               main.py
               theme.yaml
             
            regression/  (old metrics app)
               main.py
               theme.yaml
             
            
            

            Show
            afausti Angelo Fausti added a comment - - edited Changing bokeh app to directory style for convenience to handle multiple apps: http://bokeh.pydata.org/en/latest/docs/user_guide/server.html#directory-format Proposed reoganization has three independent apps served by the same bokeh server, for astrometry plots, photometry plots and regression testing   astrometry/ main.py theme.yaml photometry/ main.py theme.yaml   regression/ (old metrics app) main.py theme.yaml  
            Hide
            jsick Jonathan Sick added a comment - - edited

            in the final JSON sample for the full REST json, blobs is an object/dict. Do you want the keys of this dict to be the blob identifier? this would make it easy to look up from a measurement.

            If so, the actions that post-qa needs to do to shim it's native format to the SQUASH format is:

            1. Convert the blobs array to an object keyed by identifier.
            2. Convert the measurements array to an an array of objects with fields: 1) metric name, and 2) array of corresponding measurements.

            Thinking of the last one, it may make more sense to simply make measurements an object keyed by metric names,

            {
              "measurements": {
                {"AM1": [], <- array of measurement objects
                ...}
              }
            }
            

            Moving discussion to DM-7043.

            Show
            jsick Jonathan Sick added a comment - - edited in the final JSON sample for the full REST json, blobs is an object/dict. Do you want the keys of this dict to be the blob identifier ? this would make it easy to look up from a measurement. If so, the actions that post-qa needs to do to shim it's native format to the SQUASH format is: Convert the blobs array to an object keyed by identifier . Convert the measurements array to an an array of objects with fields: 1) metric name, and 2) array of corresponding measurements. Thinking of the last one, it may make more sense to simply make measurements an object keyed by metric names, { "measurements": { {"AM1": [], <- array of measurement objects ...} } } Moving discussion to DM-7043 .
            Hide
            afausti Angelo Fausti added a comment -

            In order to extend squash drill down capabilities we implemented validade_drp static plots in bokeh

            They are available in my test environment:

            https://angelo-squash-bokeh.lsst.codes/astrometry
            https://angelo-squash-bokeh.lsst.codes/photometry

            note that the parameters metric, dataset and Job Id are fixed in those examples.

            In DM-8478 we will continue this implementation connecting those plots with the measurements in the regression app

            https://angelo-squash-bokeh.lsst.codes/regression

            Show
            afausti Angelo Fausti added a comment - In order to extend squash drill down capabilities we implemented validade_drp static plots in bokeh They are available in my test environment: https://angelo-squash-bokeh.lsst.codes/astrometry https://angelo-squash-bokeh.lsst.codes/photometry note that the parameters metric, dataset and Job Id are fixed in those examples. In DM-8478 we will continue this implementation connecting those plots with the measurements in the regression app https://angelo-squash-bokeh.lsst.codes/regression

              People

              • Assignee:
                afausti Angelo Fausti
                Reporter:
                jsick Jonathan Sick
                Watchers:
                Angelo Fausti, Jonathan Sick
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: