Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18103

Revisit SQuaSH InfluxDB data model

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: squash
    • Labels:
      None
    • Templates:
    • Epic Link:
    • Team:
      SQuaRE

      Description

      This ticket revisits the current SQuaSH InfluxDB data model as implemented in DM-16300, summarizes ongoing changes and converge to a final proposal.

      The mapping between lsst.verify concepts to InfluxDB concepts:

      • lsst.verify package -> InfluxDB measurement
      • lsst.verify metadata -> InfluxDB tag
      • lsst.verify metric value - > InfluxDB field
      • Jenkins pipeline runtime -> InfluxDB timestamp

      Comments:

      • It is not possible to combine metric values (i.e. do math operations with metric values) that are in different measurements.
      • It seems more appropriate to add metadata like ci_id and squash_id as InfluxDB fields and not tags
      • It seems useful to have the timestamp as an InfluxDB field as well because it is not possible to do math with timestamps in InfluxDB. See this InfluxDB community post.
      • Define an appropriate time precision for the SQuaSH database in InfluxDB
      • The InlfuxDB schema is flat. It means that job metadata and environment metadata will be equally stored as InfluxDB tags.
      • Different execution environments (CI, LDF) can store metadata in SQuaSH preserving the original keys. At InfluxDB we can have a common data model so that we present an unified view of the different execution environment to the users (e.g ci_id and {run_id}} mapped to run_id).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                afausti Angelo Fausti
                Reporter:
                afausti Angelo Fausti
                Watchers:
                Angelo Fausti, Krzysztof Findeisen, Simon Krughoff, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel