Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15240

Create resource in the squash-restful-api for the KPMs dashboard

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: squash
    • Labels:
      None

      Description

      Implement the /kpms API resource to feed the KPMs dashboard. It should be possible to query metrics and specification tags as well as specification query metadata stored as JSON fields in the SQuaSH database.

      API changes required by DM-14751

        Attachments

          Issue Links

          is child task of

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-14751 Implement KPM dashboard in squash-bokeh

          • Invalid
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15474 Implement HTTP PUT for metrics and specifications resources

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15165 Add new tags for validate_drp metric definition and specifications

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15386 Add specification thresholds back to the code_changes app

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            afausti Angelo Fausti added a comment - - edited

            Example of query:

            Return metrics and specifications for a given job_id and dataset_name, look for metrics tagged as photometry and with measurements on filter_name r only.

            http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=photometry&filter_name=r

            Return metrics for a given job_id and dataset_name look for metrics tagged as astrometry and design specifications only.

            http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=astrometry&specification_tag=design
             
            Return metrics and specifications for a given job_id and dataset_name, look for metrics tagged as photometry, with measurements on filter_name r and design specifications only.

            http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=photometry&specification_tag=design&filter_name=r

            Show
            afausti Angelo Fausti added a comment - - edited Example of query: Return metrics and specifications for a given job_id and dataset_name, look for metrics tagged as photometry and with measurements on filter_name r only. http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=photometry&filter_name=r Return metrics for a given job_id and dataset_name look for metrics tagged as astrometry and design specifications only. http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=astrometry&specification_tag=design   Return metrics and specifications for a given job_id and dataset_name, look for metrics tagged as photometry, with measurements on filter_name r and design specifications only. http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=photometry&specification_tag=design&filter_name=r
            Hide
            Permalink
            afausti Angelo Fausti added a comment - - edited

            SQLAlchemy works well for querying JSON fields on MySQL 5.7.

            For filtering based on a key/value pair the sintax looks like:

            expr = Specification.metadata_query['filter_name'] == filter_name
            		 
            queryset = queryset.filter(expr)

            For filtering based on data contained in arrays:

            expr = func.json_contains(Metric.tags, '"{}"'.format(metric_tag))
            		 
            queryset = queryset.filter(expr)

            Note the weird syntax for the candidate value, it must be within double quotes.

            More information here:

            Show
            afausti Angelo Fausti added a comment - - edited SQLAlchemy works well for querying JSON fields on MySQL 5.7. For filtering based on a key/value pair the sintax looks like: expr = Specification.metadata_query[ 'filter_name' ] = = filter_name queryset = queryset. filter (expr) For filtering based on data contained in arrays: expr = func.json_contains(Metric.tags, '"{}"' .format(metric_tag)) queryset = queryset.filter(expr) Note the weird syntax for the candidate value, it must be within double quotes. More information here: https://dev.mysql.com/doc/refman/5.7/en/json-search-functions.html https://stackoverflow.com/questions/33513625/invalid-json-text-in-argument-2-json-contains-in-mysql-5-7-8
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            See PR https://github.com/lsst-sqre/squash-restful-api/pull/25

            Show
            afausti Angelo Fausti added a comment - See PR https://github.com/lsst-sqre/squash-restful-api/pull/25

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.