Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: QA
-
Labels:
-
Story Points:5
-
Epic Link:
-
Team:SQuaRE
Description
In DM-6629, a new measurement API was introduced into validate_drp that established a JSON data model for metrics, specifications of metrics, measurements, and general blob datasets. The intent of that work is to enable rich plots in the SQUASH dashboard, with access to data behind measurements. The new data model also clarifies the subtleties of metric specifications (filter dependence, and dependence on other specifications). This ticket will incorporate validate_drp’s new data model into the SQUASH database and API.
Also related to DM-7041, which will update the post-qa tool that submits validate_drp json to the SQUASH API.
Attachments
Issue Links
- has to be started together with
-
- Done
- is parent task of
-
- Done
- is triggered by
-
- Done
Activity
This looks fine. Some comments are in the PR. The main need I see is documenting the schema of these JSON fields (maybe in SQR-008) so that it's clear what data is in them (there's a slight difference between the json schema made by validate_base and what finally ends up in these JSON fields and it's good to make it clear).
Thanks, yes I will document that properly in the SQuaSH technote, basically the mapping of validade_base Job to SQUASH API is the following
1) the content of blobs is ingested into the Job.data field (make sense to change the field name to blobs)
2) the content of measurements is ingested in the Measurement.metadata field, with the exception of the metric specification which is ingested in the Metric table for normalization
3) the measurement value is kept in a separate field in the Measurement table for convenience
4) the metrics specification has unit, description, operator, specs and reference which are all individual fields in the Metric table.
Applied PR comments
Here is how the API endpoints look like after changing the model and serializers.
Notice that measurements metadata is not present in the Metrics App endpoint. We kept just the fields required for this app minimizing the amount of data returned by this endpoint.