Can you please review this? It's not perfect, but it should be enough to get you going on the Specifications. I've flattened my commits down nicely, I hope. The PR for validate_base goes to
DM-8477, as we'd decided earlier.
- Some tests fail (e.g. test_job) because they still assume the MeasurementBase structure that was there before. I only cleaned up Measurement and Metric, not the others.
- bin/validate_metrics_repo.py is terribly named, given everything (as is Metric.validate()), so suggestions there are welcome. Although, if we're no-longer going to call this "validate_base", maybe it's not a problem? It does succeed for $VALIDATE_METRICS_DIR/metrics as it is on this ticket, so that's nice!
- test_output_measurements.py fails, because auto-loading of metrics from validate_metrics isn't implemented. It would be pretty easy to do at this point, but this gets back to the dependency question we've been wondering about. I can finish that code in
DM-9534, since the code in jointcal is already in place. What to actually do with that output is another question, possibly best answered by Angelo Fausti.
One final question: I'm currently not ensuring consistency between fully-qualified measurement names and metric names. We might not be able to do that until test_output_measurements.py is working with auto-loading of validate_metrics, but we'll have to decide exactly how that is supposed to behave.