Fix Version/s: None
validate_drp computes metrics and generates JSON that, through the post-qa tool, is submitted to the SQuaSH REST API for persistence and display in a web app.
A previous ticket,
DM-6086, we bolted on a JSON serialization scheme compatible with SQuaSH. However, this approach was not well integrated with validate_drp. We want a framework/API where serialization is handled consistently and integrally with metric computations. This includes the semantic serialization of computational parameters and reduced datasets.
This API can be applied beyond validate_drp as a means for metrics and integration tests to be submitted to SQuaSH as well.
- is triggering
DM-7041 Update post-qa to submit new JSON from validate_drp measurement API
DM-7042 validate_base API refinement
DM-7043 Update SQuaSH database model and JSON API with concepts from validate_drp measurement API
- relates to
DM-6917 Write User Guide for new validate_drp metric/measurement API
DM-7692 Create validate_base package
DM-8711 Deploy validate_drp F16 work to Jenkins validate_drp Job
I’m putting this ticket into review, finally, and I’d like everyone involved in SQUASH and validate_drp to take a look since I think think this will be the API that everyone uses to implement metrics and measurements that get presented to the SQUASH dashboard and DB, in validate_drp or elsewhere in the stack.
Take a look at the measurement classes (e.g., AMxMeasurement) in the calcsrd module to get a feel for what implementing measurements is like from a developer perspective. I’ve used a tiny bit of metaprogramming to making the data model easy to use for developers.
I’ve also put together prototype user docs in a separate ticket, but they’re available at https://validate-drp.lsst.io/v/DM-6917/
The PR also includes sample plots re-implemented with the new data model.
Jonathan Sick This is going to take me another of couple of days to get to. This review caught me at the LSST DESC collaboration meeting, and I'm still working my way out of the backlog from that.
No worries Michael Wood-Vasey; no one else has commented on it yet either.
To be honest, that makes me worry more. My goal is for you to get feedback, not for me to not be later than others
Because of the difficult nature of such a large code review (and the pressing needs of EVM), let’s pivot this review:
- I will close this ticket, along with the associated doc ticket
- I will merge this PR, along with
DM-6917, onto a u/sqre/measurement-api integration branch in validate_drp.
- I will convene a meeting next week (week of Aug 1, 2016). The purpose of this meeting will be to treat
DM-6629and DM-6917as a design prototype prototype for a measurement API in validate_drp. In this meeting I’ll present the motivation, features, and other salient points about this prototype. Then we will discuss whether this is a design we want to use in validate_drp and other packages making metric measurements that might be report to SQUASH. We’ll also discuss how this proposed data model impacts SQuaSH. Everyone already named as a reviewer on this ticket is explicitly invited to this design review. I’d also like to invite Tim Jenness to represent the interests of DM Architecture.
Any objections before I go ahead and mark this ticket as done?
Merged to u/sqre/measurement-api. We’ll review the new API proposal presented here on that branch.
See PR for sample plots, print output, and JSON.
Docs are being prototyped at https://validate-drp.lsst.io/v/DM-6917/