Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15240

Create resource in the squash-restful-api for the KPMs dashboard

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: squash
    • Labels:
      None

      Description

      Implement the /kpms API resource to feed the KPMs dashboard. It should be possible to query metrics and specification tags as well as specification query metadata stored as JSON fields in the SQuaSH database.

      API changes required by DM-14751

        Attachments

          Issue Links

            Activity

            No builds found.
            afausti Angelo Fausti created issue -
            afausti Angelo Fausti made changes -
            Field Original Value New Value
            Epic Link DM-14312 [ 63991 ]
            afausti Angelo Fausti made changes -
            Risk Score 0
            afausti Angelo Fausti made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            afausti Angelo Fausti added a comment - - edited

            Example of query:

            Return metrics and specifications for a given job_id and dataset_name, look for metrics tagged as photometry and with measurements on filter_name r only.

            http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=photometry&filter_name=r

            Return metrics for a given job_id and dataset_name look for metrics tagged as astrometry and design specifications only.

            http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=astrometry&specification_tag=design
             
            Return metrics and specifications for a given job_id and dataset_name, look for metrics tagged as photometry, with measurements on filter_name r and design specifications only.

            http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=photometry&specification_tag=design&filter_name=r

            Show
            afausti Angelo Fausti added a comment - - edited Example of query: Return metrics and specifications for a given job_id and dataset_name, look for metrics tagged as photometry and with measurements on filter_name r only. http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=photometry&filter_name=r Return metrics for a given job_id and dataset_name look for metrics tagged as astrometry and design specifications only. http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=astrometry&specification_tag=design   Return metrics and specifications for a given job_id and dataset_name, look for metrics tagged as photometry, with measurements on filter_name r and design specifications only. http://localhost:5000/kpms?job_id=2&dataset_name=validation_data_cfht&metric_tag=photometry&specification_tag=design&filter_name=r
            Hide
            afausti Angelo Fausti added a comment - - edited

            SQLAlchemy works well for querying JSON fields on MySQL 5.7.

            For filtering based on a key/value pair the sintax looks like:

            expr = Specification.metadata_query['filter_name'] == filter_name
            queryset = queryset.filter(expr)
            

            For filtering based on data contained in arrays:

            expr = func.json_contains(Metric.tags, '"{}"'.format(metric_tag))
            queryset = queryset.filter(expr)
            

            Note the weird syntax for the candidate value, it must be within double quotes.

            More information here:

            Show
            afausti Angelo Fausti added a comment - - edited SQLAlchemy works well for querying JSON fields on MySQL 5.7. For filtering based on a key/value pair the sintax looks like: expr = Specification.metadata_query[ 'filter_name' ] = = filter_name queryset = queryset. filter (expr) For filtering based on data contained in arrays: expr = func.json_contains(Metric.tags, '"{}"' .format(metric_tag)) queryset = queryset.filter(expr) Note the weird syntax for the candidate value, it must be within double quotes. More information here: https://dev.mysql.com/doc/refman/5.7/en/json-search-functions.html https://stackoverflow.com/questions/33513625/invalid-json-text-in-argument-2-json-contains-in-mysql-5-7-8
            afausti Angelo Fausti made changes -
            Link This issue is child task of DM-14751 [ DM-14751 ]
            Show
            afausti Angelo Fausti added a comment - See PR https://github.com/lsst-sqre/squash-restful-api/pull/25
            afausti Angelo Fausti made changes -
            Resolution Done [ 10000 ]
            Status In Progress [ 3 ] Done [ 10002 ]
            afausti Angelo Fausti made changes -
            Description A new /kpms resource to retrieve the data needed by the KPMs dashboard. I should be possible to query metrics and specification tags as well as specification query metadata stored as JSON fields in the SQuaSH database. A new /kpms resource to retrieve the data needed by the KPMs dashboard. I should be possible to query metrics and specification tags as well as specification query metadata stored as JSON fields in the SQuaSH database.

            API changes required by DM-1471
            afausti Angelo Fausti made changes -
            Description A new /kpms resource to retrieve the data needed by the KPMs dashboard. I should be possible to query metrics and specification tags as well as specification query metadata stored as JSON fields in the SQuaSH database.

            API changes required by DM-1471
            A new /kpms resource to retrieve the data needed by the KPMs dashboard. I should be possible to query metrics and specification tags as well as specification query metadata stored as JSON fields in the SQuaSH database.

            API changes required by DM-14751
            afausti Angelo Fausti made changes -
            Description A new /kpms resource to retrieve the data needed by the KPMs dashboard. I should be possible to query metrics and specification tags as well as specification query metadata stored as JSON fields in the SQuaSH database.

            API changes required by DM-14751
            Implement the /kpms API resource to feed the KPMs dashboard. It should be possible to query metrics and specification tags as well as specification query metadata stored as JSON fields in the SQuaSH database.

            API changes required by DM-14751
            afausti Angelo Fausti made changes -
            Component/s squash [ 14169 ]
            afausti Angelo Fausti made changes -
            Link This issue relates to DM-15165 [ DM-15165 ]
            afausti Angelo Fausti made changes -
            Link This issue relates to DM-15386 [ DM-15386 ]
            afausti Angelo Fausti made changes -
            Link This issue relates to DM-15474 [ DM-15474 ]

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.