Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: squash
-
Labels:None
-
Story Points:1.4
-
Epic Link:
-
Team:SQuaRE
Description
There are at least three options of timestamps that we can use to map the lsst.verify run time to InfluxDB.
- CI pipeline run time
- Verification Job time
- InfluxDB creation time
We are currently using the "Verification Job time" which turns not to be the right choice because we are running a different verification job for each dataID. As consequence, results that should have the same timestamp get inserted in InfluxDB at different times. A better choice for the InfluxDB timestamp seems to be the CI pipeline run time.
Attachments
Issue Links
Activity
For the record the query to display metric values per ccd:
SELECT "ap_association.totalUnassociatedDiaObjects" FROM "squash-demo"."autogen"."ap_association" WHERE time > :dashboardTime: AND ("ccdnum"='10' OR "ccdnum"='5' OR "ccdnum"='56' OR "ccdnum"='60') GROUP BY "ccdnum"
|
and the query to aggregate metric values across multiple ccds:
SELECT mean("ap_association.totalUnassociatedDiaObjects") AS "mean_ap_association.totalUnassociatedDiaObjects" FROM "squash-demo"."autogen"."ap_association" WHERE time > :dashboardTime: GROUP BY time(1d)
|
Results for this aggregation test are currently available under the SQuaRE organization in Chronograf.
Simon Krughoff would like to have a look at these results in Chronograf?
I will have a look.
Looks great.
In the PR https://github.com/lsst-sqre/squash-restful-api/pull/45 we updated the influxdb_data_model.ipynb notebook to use the CI pipeline runtime as the InfluxDB timestamp.
The squash-demo database was re-created and we showed that aggregation of metric values across multiple data ids work just fine, using mean() and GROUP BY time() InfluxQL functions.