# Update post-qa to submit new JSON from validate_drp measurement API

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
4
• Team:
SQuaRE

## Description

DM-6629 introduced a new measurement API. The post-qa app needs to be updated to shim and upload this JSON to the SQUASH API.

Note that this ticket only updates the shims for the new measurement api's JSON to the current SQUASH API. A separate ticket will be needed to then adapt this the output schema for a more fully-featured SQUASH API.

This ticket should use JSON Schema to validate that the shims are successfully producing the JSON that is expected by the SQUASH API.

## Activity

Hide
Angelo Fausti added a comment - - edited

Related to the JSON generated by post_qa, right now it looks like the example below

{
"ci_id": "177",
"ci_name": "demo",
"ci_url": "https://ci.lsst.codes/job/validate_drp/dataset=cfht,label=centos-7,python=py2/177/",
"date": "2016-07-27T16:32:08.300213+00:00",
"measurements": [

{ "metric": "AM1", "value": 7.137208394754365 }

,

ci_name should assume the value validate_drp right?

and I am planning to add

"ci_dataset": "cfht"

and

"ci_label": "centos-7"

to implement the ability of filtering the data by dataset and platform in the dashboard (related to ticket DM-6992).

do you agree with this change?

Show
Angelo Fausti added a comment - - edited Jonathan Sick , Joshua Hoblitt Related to the JSON generated by post_qa, right now it looks like the example below { "ci_id": "177", "ci_name": "demo", "ci_url": "https://ci.lsst.codes/job/validate_drp/dataset=cfht,label=centos-7,python=py2/177/", "date": "2016-07-27T16:32:08.300213+00:00", "measurements": [ { "metric": "AM1", "value": 7.137208394754365 } , ci_name should assume the value validate_drp right? and I am planning to add "ci_dataset": "cfht" and "ci_label": "centos-7" to implement the ability of filtering the data by dataset and platform in the dashboard (related to ticket DM-6992 ). do you agree with this change?
Hide
Jonathan Sick added a comment - - edited

Here's the line that sets ci_name:

https://github.com/lsst-sqre/post-qa/blob/master/postqa/jenkinsenv.py#L26

In other words, we just need a $DATASET environment variable in the Jenkins environment that indicates the current dataset. I think "ci_name" and your "ci_dataset" are pretty much the same thing? "ci_label" seems like a good idea. I'll have to find out how I can get that from the Jenkins environment. Update: Ah, it might make sense for "ci_name" to be the name of the Jenkins project. Again, I'll have to find out how to get that from the environment. Show Jonathan Sick added a comment - - edited Here's the line that sets ci_name : https://github.com/lsst-sqre/post-qa/blob/master/postqa/jenkinsenv.py#L26 In other words, we just need a$DATASET environment variable in the Jenkins environment that indicates the current dataset. I think "ci_name" and your "ci_dataset" are pretty much the same thing? "ci_label" seems like a good idea. I'll have to find out how I can get that from the Jenkins environment. Update : Ah, it might make sense for "ci_name" to be the name of the Jenkins project. Again, I'll have to find out how to get that from the environment.
Hide
Angelo Fausti added a comment - - edited

I think so, we could have:

"ci_name" : "validate_drp",
"ci_dataset": "cfht"
"ci_label": "centos-7"

I would say you could use the Jenkins API given the ci_id

https://ci.lsst.codes/job/validate_drp/177/api/json?pretty=true

but "dataset" and "label" only appear enconded in the url...

Show
Angelo Fausti added a comment - - edited I think so, we could have: "ci_name" : "validate_drp", "ci_dataset": "cfht" "ci_label": "centos-7" I would say you could use the Jenkins API given the ci_id https://ci.lsst.codes/job/validate_drp/177/api/json?pretty=true but "dataset" and "label" only appear enconded in the url...
Hide
Jonathan Sick added a comment -

Can I call the API from inside the Jenkins job though? post-qa runs on Jenkins itself.

Show
Jonathan Sick added a comment - Can I call the API from inside the Jenkins job though? post-qa runs on Jenkins itself.
Hide
Angelo Fausti added a comment -

oh, this link provides all the enviroment variables that should be available for post_qa

https://ci.lsst.codes/job/validate_drp/177/dataset=cfht,label=centos-7,python=py2/injectedEnvVars/

there's already PRODUCT, dataset and label (but these two should be uppercase)

Show
Angelo Fausti added a comment - oh, this link provides all the enviroment variables that should be available for post_qa https://ci.lsst.codes/job/validate_drp/177/dataset=cfht,label=centos-7,python=py2/injectedEnvVars/ there's already PRODUCT , dataset and label (but these two should be uppercase)
Hide
Angelo Fausti added a comment -

The changes discussed above will be implemented in a separate ticket DM-7098.

Show
Angelo Fausti added a comment - The changes discussed above will be implemented in a separate ticket DM-7098 .

## People

• Assignee:
Jonathan Sick
Reporter:
Jonathan Sick
Reviewers:
J Matt Peterson [X] (Inactive)
Watchers:
Angelo Fausti, J Matt Peterson [X] (Inactive), Jonathan Sick