Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7005

List the code changes from job to job and link the package names with the corresponding git url

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Motivated by the deviation seen from build 156 to 157 in https://squash.lsst.codes/AM1 (caused by a commit in meas_algorithms package), and other deviations seen later as well, we want to show the list of packages that changed in the current build with respect to the previous build and link to the the git commit URLs to easily identify what caused that deviation.

        Attachments

          Activity

          Hide
          afausti Angelo Fausti added a comment - - edited

          Another motivation to work on this issue now is that the jobs api endpoint returns the whole list of packages for each build.

          http://sqr-009.lsst.io/en/latest/#the-dashboard-api

          Instead, we should have another API endpoint to retrieve just the necessary data to feed the metrics app. Example:

          api/measurements (return required information for all measurements)

          The api/jobs?ci_dataset filter should be implemented in this new endpoint as well. Example:

          api/measurements/?ci_dataset='chft' (return required information only for cfht data set measurements)

          In addition each result will include only the packages in the current build that differs from the packages in the previous build for comparison, reducing the information retrieved.

          The git_url, the git_commit and the git_branch will also be returned.

          Expected results:

          • Metrics app will load faster
          • The user will be able to drill down to the code changes in git hub if unexpected deviations in a metric measurement is seen.
          • get_measurements_by_dataset() in viz/service.py must be adapted to get the measurements data from the new api endpoint.

          Note that api/jobs still will be used to ingest data from validate_drp.

          Show
          afausti Angelo Fausti added a comment - - edited Another motivation to work on this issue now is that the jobs api endpoint returns the whole list of packages for each build. http://sqr-009.lsst.io/en/latest/#the-dashboard-api Instead, we should have another API endpoint to retrieve just the necessary data to feed the metrics app. Example: api/measurements (return required information for all measurements) The api/jobs?ci_dataset filter should be implemented in this new endpoint as well. Example: api/measurements/?ci_dataset='chft' (return required information only for cfht data set measurements) In addition each result will include only the packages in the current build that differs from the packages in the previous build for comparison, reducing the information retrieved. The git_url, the git_commit and the git_branch will also be returned. Expected results: Metrics app will load faster The user will be able to drill down to the code changes in git hub if unexpected deviations in a metric measurement is seen. get_measurements_by_dataset() in viz/service.py must be adapted to get the measurements data from the new api endpoint. Note that api/jobs still will be used to ingest data from validate_drp.
          Hide
          afausti Angelo Fausti added a comment - - edited

          Implementing "measurements" API endpoint with filter by dataset.

          api/measurements/?ci_dataset='chft'

          and reviewing API naming

          http://www.restapitutorial.com/lessons/restfulresourcenaming.html

          Show
          afausti Angelo Fausti added a comment - - edited Implementing "measurements" API endpoint with filter by dataset. api/measurements/?ci_dataset='chft' and reviewing API naming http://www.restapitutorial.com/lessons/restfulresourcenaming.html
          Hide
          afausti Angelo Fausti added a comment - - edited

          Since measurements is in the jobs context, the following URIs also should be implemented for reading, updating or deleting measurements given the job id

          GET api/jobs/<id>/measurements
          PUT api/jobs/<id>/measurements
          DELETE api/jobs/<id>/measurements

          this is is probably out of the scope of this ticket, but could be implemented using 'hierarchy' in Django Rest Framework.

          Show
          afausti Angelo Fausti added a comment - - edited Since measurements is in the jobs context, the following URIs also should be implemented for reading, updating or deleting measurements given the job id GET api/jobs/<id>/measurements PUT api/jobs/<id>/measurements DELETE api/jobs/<id>/measurements this is is probably out of the scope of this ticket, but could be implemented using 'hierarchy' in Django Rest Framework.
          Show
          afausti Angelo Fausti added a comment - - edited http://www.django-rest-framework.org/topics/documenting-your-api/#self-describing-apis
          Hide
          afausti Angelo Fausti added a comment - - edited

          Created measurements API endpoint using a specific viewset and a serializer to retrieve the data consumed by the metrics app.

          This way the data transformations required for the metrics app are implemented directly in the serializer avoiding additional data manipulation as in the previous implementation of get_measurements_by_dataset() function (see https://jira.lsstcorp.org/browse/DM-7612)

          Amazing how many lines of code you save when you lear how to use DRF correctly, the solution uses the serializerMethodField (see http://www.django-rest-framework.org/api-guide/fields/#serializermethodfield)

          Now the final URL filtering by dataset and metric looks like

          http://localhost:8000/dashboard/api/measurements/?job__ci_dataset=cfht&metric=AM1
          which returns fields from different models

            "results": [
                  {
                      "value": 5.0,
                      "ci_id": "1",
                      "ci_url": "https://ci.lsst.codes/job/ci_cfht/1/",
                      "date": "2016-09-15T00:00:00.001Z"
                  },
                  {
                      "value": 10.0,
                      "ci_id": "2",
                      "ci_url": "https://ci.lsst.codes/job/ci_cfht/2/",
                      "date": "2016-09-15T08:00:00.001Z"
                  },
                  {
                      "value": 8.0,
                      "ci_id": "3",
                      "ci_url": "https://ci.lsst.codes/job/ci_cfht/3/",
                      "date": "2016-09-15T16:00:00.001Z"
                  },
          

          Show
          afausti Angelo Fausti added a comment - - edited Created measurements API endpoint using a specific viewset and a serializer to retrieve the data consumed by the metrics app. This way the data transformations required for the metrics app are implemented directly in the serializer avoiding additional data manipulation as in the previous implementation of get_measurements_by_dataset() function (see https://jira.lsstcorp.org/browse/DM-7612 ) Amazing how many lines of code you save when you lear how to use DRF correctly, the solution uses the serializerMethodField (see http://www.django-rest-framework.org/api-guide/fields/#serializermethodfield ) Now the final URL filtering by dataset and metric looks like http://localhost:8000/dashboard/api/measurements/?job__ci_dataset=cfht&metric=AM1 which returns fields from different models "results": [ { "value": 5.0, "ci_id": "1", "ci_url": "https://ci.lsst.codes/job/ci_cfht/1/", "date": "2016-09-15T00:00:00.001Z" }, { "value": 10.0, "ci_id": "2", "ci_url": "https://ci.lsst.codes/job/ci_cfht/2/", "date": "2016-09-15T08:00:00.001Z" }, { "value": 8.0, "ci_id": "3", "ci_url": "https://ci.lsst.codes/job/ci_cfht/3/", "date": "2016-09-15T16:00:00.001Z" },
          Hide
          afausti Angelo Fausti added a comment - - edited
          • finished (need real test though) a method to return the list of packages that changed in a given job with respect to the previous one (check diffs in the git commit sha of each package) the measurements api returns the required info for those packages

          {
              "count": 6,
              "next": null,
              "previous": null,
              "results": [
                  {
                      "value": 4.0,
                      "ci_id": "1",
                      "ci_url": "https://ci.lsst.codes/job/ci_cfht/1/",
                      "date": "2016-09-15T00:00:00.001Z",
                      "changed_packages": {
                          "previous_job": [
                              {
                                  "git_url": "https://github.com/lsst/afw.git",
                                  "name": "afw",
                                  "git_commit": "fc355a99abe3425003b0e5fbe1e13a39644b1e95",
                                  "ci_id": "1",
                                  "build_version": "b2000"
                              }
                          ],
                          "current_job": [
                              {
                                  "git_url": "https://github.com/lsst/afw.git",
                                  "name": "afw",
                                  "git_commit": "fc355a99abe3425003b0e5fbe1e13a39644b1e95",
                                  "ci_id": "1",
                                  "build_version": "b2000"
                              }
                          ]
                      }
                  },
                  {
                      "value": 6.0,
                      "ci_id": "2",
                      "ci_url": "https://ci.lsst.codes/job/ci_cfht/2/",
                      "date": "2016-09-15T08:00:00.001Z",
                      "changed_packages": {
                          "previous_job": [
                              {
                                  "git_url": "https://github.com/lsst/afw.git",
                                  "name": "afw",
                                  "git_commit": "fc355a99abe3425003b0e5fbe1e13a39644b1e95",
                                  "ci_id": "1",
                                  "build_version": "b2000"
                              }
                          ],
                          "current_job": [
                              {
                                  "git_url": "https://github.com/lsst/afw.git",
                                  "name": "afw",
                                  "git_commit": "fc355a99abe3425003b0e5fbe1e13a39644b1e95",
                                  "ci_id": "2",
                                  "build_version": "b2000"
                              }
                          ]
                      }
                  },
          
          

          Show
          afausti Angelo Fausti added a comment - - edited finished (need real test though) a method to return the list of packages that changed in a given job with respect to the previous one (check diffs in the git commit sha of each package) the measurements api returns the required info for those packages { "count": 6, "next": null, "previous": null, "results": [ { "value": 4.0, "ci_id": "1", "ci_url": "https://ci.lsst.codes/job/ci_cfht/1/", "date": "2016-09-15T00:00:00.001Z", "changed_packages": { "previous_job": [ { "git_url": "https://github.com/lsst/afw.git", "name": "afw", "git_commit": "fc355a99abe3425003b0e5fbe1e13a39644b1e95", "ci_id": "1", "build_version": "b2000" } ], "current_job": [ { "git_url": "https://github.com/lsst/afw.git", "name": "afw", "git_commit": "fc355a99abe3425003b0e5fbe1e13a39644b1e95", "ci_id": "1", "build_version": "b2000" } ] } }, { "value": 6.0, "ci_id": "2", "ci_url": "https://ci.lsst.codes/job/ci_cfht/2/", "date": "2016-09-15T08:00:00.001Z", "changed_packages": { "previous_job": [ { "git_url": "https://github.com/lsst/afw.git", "name": "afw", "git_commit": "fc355a99abe3425003b0e5fbe1e13a39644b1e95", "ci_id": "1", "build_version": "b2000" } ], "current_job": [ { "git_url": "https://github.com/lsst/afw.git", "name": "afw", "git_commit": "fc355a99abe3425003b0e5fbe1e13a39644b1e95", "ci_id": "2", "build_version": "b2000" } ] } }, implemented bokeh data table to present the results (any suggestion here?) still learning how to update the bokeh data table connected with the hover tool, that is possible using a custom JS callback as explained here http://bokeh.pydata.org/en/latest/docs/user_guide/interaction/callbacks.html#customjs-for-hover updated bokeh to the newest version 0.12.3 there is a css issue with bootstrap and bokeh data tables (pending)
          Hide
          afausti Angelo Fausti added a comment -

          Logged more 6 SP for week Oct 17-21

          Show
          afausti Angelo Fausti added a comment - Logged more 6 SP for week Oct 17-21
          Hide
          afausti Angelo Fausti added a comment -

          I think I've found the right solution to connect the measurements (dots in the time series plot) with the data table listing the code changes. The bokeh data table has a nice feature called " scroll_to_selection" so that if you select the data point by any method the corresponding value is highlighted in the table. So if we change the behaviour of the "Tap Select" tool to just select the data points it works nicely. Then instead of open the corresponding Jenkins URL on Tap, the ci_url can be linked from the table by clicking on the corresponding "Job ID" and of course the package name can be linked to the github commit URL

          Show
          afausti Angelo Fausti added a comment - I think I've found the right solution to connect the measurements (dots in the time series plot) with the data table listing the code changes. The bokeh data table has a nice feature called " scroll_to_selection" so that if you select the data point by any method the corresponding value is highlighted in the table. So if we change the behaviour of the "Tap Select" tool to just select the data points it works nicely. Then instead of open the corresponding Jenkins URL on Tap, the ci_url can be linked from the table by clicking on the corresponding "Job ID" and of course the package name can be linked to the github commit URL
          Hide
          afausti Angelo Fausti added a comment -

          The implementation is available for testing at

          https://angelo-squash-squash.lsst.codes

          Look at the measurements API endpoint and notice that it returns just the data consumed by the metrics app then try the interaction between the data points in the plot and in the data table.

          Show
          afausti Angelo Fausti added a comment - The implementation is available for testing at https://angelo-squash-squash.lsst.codes Look at the measurements API endpoint and notice that it returns just the data consumed by the metrics app then try the interaction between the data points in the plot and in the data table.
          Hide
          afausti Angelo Fausti added a comment -

          Logged more 6 SP for week Oct 31 - Nov 4

          • Return packages that changed with respect to the previous job in the measurements serializer, this implementation uses python Set to get the package differences
          • Changed get_meas_by_dataset_and_metric() in services.py to get list os package names and corresponding git urls
          • Use bokeh HTMLTemplateFormatter to link package names with the git urls in the metrics app data table (Package column)
          • Added more test data

          Final implementation for this ticket is available here:
          https://angelo-squash-squash.lsst.codes

          Show
          afausti Angelo Fausti added a comment - Logged more 6 SP for week Oct 31 - Nov 4 Return packages that changed with respect to the previous job in the measurements serializer, this implementation uses python Set to get the package differences Changed get_meas_by_dataset_and_metric() in services.py to get list os package names and corresponding git urls Use bokeh HTMLTemplateFormatter to link package names with the git urls in the metrics app data table (Package column) Added more test data Final implementation for this ticket is available here: https://angelo-squash-squash.lsst.codes

            People

            • Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel