# Mapping lsst.verify run time to InfluxDB

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
1.4
• Team:
SQuaRE

#### Description

There are at least three options of timestamps that we can use to map the lsst.verify run time to InfluxDB.

• CI pipeline run time
• Verification Job time
• InfluxDB creation time

We are currently using the "Verification Job time" which turns not to be the right choice because we are running a different verification job for each dataID. As consequence, results that should have the same timestamp get inserted in InfluxDB at different times. A better choice for the InfluxDB timestamp seems to be the CI pipeline run time.

#### Activity

Hide
Angelo Fausti added a comment -

In the PR https://github.com/lsst-sqre/squash-restful-api/pull/45 we updated the influxdb_data_model.ipynb notebook to use the CI pipeline runtime as the InfluxDB timestamp.

The squash-demo database was re-created and we showed that aggregation of metric values across multiple data ids work just fine, using mean() and GROUP BY time() InfluxQL functions.

Show
Angelo Fausti added a comment - In the PR https://github.com/lsst-sqre/squash-restful-api/pull/45 we updated the influxdb_data_model.ipynb notebook to use the CI pipeline runtime as the InfluxDB timestamp. The squash-demo database was re-created and we showed that aggregation of metric values across multiple data ids work just fine, using mean() and GROUP BY time() InfluxQL functions.
Hide
Angelo Fausti added a comment - - edited

For the record the query to display metric values per ccd:

 SELECT "ap_association.totalUnassociatedDiaObjects" FROM "squash-demo"."autogen"."ap_association" WHERE time > :dashboardTime: AND ("ccdnum"='10' OR "ccdnum"='5' OR "ccdnum"='56' OR "ccdnum"='60') GROUP BY "ccdnum" 

and the query to aggregate metric values across multiple ccds:

 SELECT mean("ap_association.totalUnassociatedDiaObjects") AS "mean_ap_association.totalUnassociatedDiaObjects" FROM "squash-demo"."autogen"."ap_association" WHERE time > :dashboardTime: GROUP BY time(1d) 

Results for this aggregation test are currently available under the SQuaRE organization in Chronograf.

Show
Angelo Fausti added a comment - - edited For the record the query to display metric values per ccd: SELECT "ap_association.totalUnassociatedDiaObjects" FROM "squash-demo"."autogen"."ap_association" WHERE time > :dashboardTime: AND ("ccdnum"='10' OR "ccdnum"='5' OR "ccdnum"='56' OR "ccdnum"='60') GROUP BY "ccdnum" and the query to aggregate metric values across multiple ccds: SELECT mean("ap_association.totalUnassociatedDiaObjects") AS "mean_ap_association.totalUnassociatedDiaObjects" FROM "squash-demo"."autogen"."ap_association" WHERE time > :dashboardTime: GROUP BY time(1d) Results for this aggregation test are currently available under the SQuaRE organization in Chronograf.
Hide
Angelo Fausti added a comment -

Simon Krughoff would like to have a look at these results in Chronograf?

Show
Angelo Fausti added a comment - Simon Krughoff would like to have a look at these results in Chronograf?
Hide
Simon Krughoff added a comment -

I will have a look.

Show
Simon Krughoff added a comment - I will have a look.
Hide
Simon Krughoff added a comment -

Looks great.

Show
Simon Krughoff added a comment - Looks great.

#### People

Assignee:
Angelo Fausti
Reporter:
Angelo Fausti
Reviewers:
Simon Krughoff
Watchers:
Angelo Fausti, John Swinbank, Simon Krughoff