Angelo Fausti added a comment - 20/Jul/16 4:38 PM Implemented new model to associate job measurements to a dataset.

Angelo Fausti added a comment - 04/Aug/16 12:10 AM - edited Frossie Economou , Jonathan Sick , Joshua Hoblitt , J Matt Peterson [X] Update on the status of this ticket. My intention is to implement interactions in the SQuaSH dashboard using bokeh widgets, in DM-70006 I have updated bokh to 0.12.1 that brings a number of improvements and new layout capabilities http://bokeh.pydata.org/en/latest/docs/user_guide/interaction/widgets.html the advantages are clear, we can have small apps for each view of the dashboard written purely in bokeh and very modular (each bokeh app in a single file). For instance, for dataset selection, I am using the bokeh select widget which in python is really two lines of code: # add ui component   dataset_select = Select(title = "Data Set:" , value = self .datasets[ 0 ], options = self .datasets, width = 100 ) dataset_select.on_change( "value" , self .on_dataset_change) and the layout with the plot is done like this   self.layout = column( widgetbox(dataset_select), widgetbox(metric_radio_group), plot) Another advantage of this approach is that bokeh server gets a 'data source' with all the data we want to display, and any change to the 'data source' automatically updates in the plot, that is the basic mechanism for adding interactions. However... the way bokeh app is embedded in Django in the current implemetation of SQuaSH, it oes not respond to "on change" or any other event. It took me a full day to figure out the solution for that which is also suggested in this post Reference: https://groups.google.com/a/continuum.io/forum/#!topic/bokeh/GtRQklgJ3qs There are different ways to embed bokeh apps, as described here http://bokeh.pydata.org/en/latest/docs/user_guide/embed.html and the alternative which works is simpler and it is the way to go in my opinion, i.e having the bokeh serving the bokeh app and adding the bokeh script in the Django template using   script = autoload_server(app_path = "/metrics" )   bokeh_script = autoload_server( None , app_path = "/metrics" ) context.update(bokeh_script = bokeh_script) the template code in Django is just this:   < div class = "container" > < div class = "row" > < div class = "page-header" > < h1 >KPM CI< small > Tracking the impact of code changes in the Key Performance Metrics </ small ></ h1 > </ div > </ div > < div class = "panel-body plot" >{{ bokeh_script|safe }}</ div > </ div >   where 'bokeh_script' contains the bokeh_app. As a result of that, I have implemented the dataset and metric selection as part of the bokeh app, removing all the previous Django template code associated with that - SQuaSH looks much simpler now. Having standalone bokeh apps composing SQuaSH is a good architecture choice. The next thing that is necessary ( and will be described in another ticket ) is to get the data from the Django REST API instead of using the Django models directly. As we discussed before, this way we decouple the dashboard from the component that is serving the data and will be easier in the future to switch from our propotype QA database to the one served by Webserv with a similar API. My intention is to make these changes in SQuaSH soon rather than later and would like to discuss that in our meeting this afternoon.

Angelo Fausti added a comment - 10/Aug/16 4:06 PM Joshua Hoblitt would you like to review? there are some changes that might be relevant for deployment, like package updates and the new method to start the bokeh server . More info at https://github.com/lsst-sqre/qa-dashboard/pull/11

Angelo Fausti added a comment - 10/Aug/16 8:34 PM Joshua Hoblitt so here is the migration exercise in dev mode, two new fields 'ci_dataset' and 'ci_label' were added to the Job model. The migration will make the necessary changes in the db and populat the new fields. I have created the migration files as follows: $ python manage.py makemigrations You are trying to add a non-nullable field 'ci_dataset' to job without a default; we can't do that (the database needs something to populate existing rows). Please select a fix: 1) Provide a one-off default now (will be set on all existing rows) 2) Quit, and let me add a default in models.py Select an option: 1 Please enter the default value now, as valid Python >>> 'cfht' You are trying to add a non-nullable field 'ci_label' to job without a default; we can't do that (the database needs something to populate existing rows). Please select a fix: 1) Provide a one-off default now (will be set on all existing rows) 2) Quit, and let me add a default in models.py Select an option: 1 Please enter the default value now, as valid Python The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now() >>> 'centos-7' Migrations for 'dashboard' : 0003_auto_20160811_0319.py: - Add field ci_dataset to job - Add field ci_label to job ( env ) they are under squash/dashborad/migrations (just pushed) Then, in production everything you should do is to apply the migrations with the command: $ python manage.py migrate and it should be ok.

Angelo Fausti added a comment - 11/Aug/16 12:28 PM Changed title to better reflect the scope of the work done.

Angelo Fausti added a comment - 11/Aug/16 7:41 PM Adresses some of the issues in the PR.

Angelo Fausti added a comment - 14/Aug/16 8:40 PM - edited Updated technote after changes in this ticket http://sqr-009.lsst.io/en/latest/