Details
-
Type:
Story
-
Status: To Do
-
Resolution: Unresolved
-
Fix Version/s: None
-
Component/s: verify
-
Labels:None
-
Epic Link:
-
Team:SQuaRE
Description
This is a description of some of the interactions I've had with lsst.verify over the last few weeks. I'm trying to capture the relevant experience so we can triage and determine what we want to do going forward. Note that these mostly have to do with specs, at the moment.
- When dealing with a few levels of inheritance, it's very easy to get confused. It would be nice to be able to introspect what partials went into a particular spec definitions.
- We have been considering the YAML descriptions to be the primary implementation mechanism for defining specifications. With multiple levels of hierarchy, it's easy to make mistakes, via simple typos. We should think about whether we want to continue with YAML as the source of primacy (as opposed to code for example).
- We need some tooling for linting. A good start on this is in
DM-18900. Essentially all of my problems that weren't simple typos were due to either redefinitions of specs, or ids of partials shadowing each other. - Currently, there are a few different styles for spec definitions. I think it would be good to provide advice on best practices as well as to homogenize the definitions we have now.
- The validate_drp specs are very complicated because of so called "compound" specs. These are specs that depend both on the release level, e.g. FY19, and the SRD level, e.g. design. I think we should examine whether these kinds of compound specs should even be defined the way we are currently doing it.
Further stories may appear in the comments.
Attachments
Issue Links
- mentioned in
-
Page Loading...
Activity
Is this still a SQuaRE ticket or should it be reassigned or closed?
In the spirit of the above notes, I've run into some issues with defining partial metrics cleanly in our YAML files. For example, in ap_association.yaml we have a number of source count metrics, as well as several overlapping sets of tags (CCD-level metric, dataset-level metric, difference imaging metric).
However, because every node in the YAML file must correspond to a real metric, these concepts end up getting jumbled up. Ideally one would like to define a generic source count metric (which constrains units and documentation) and a generic CCD-level metric (which constrains tags), and then define a CCD-level source count metric as a combination of these two templates. Instead, a prototypical (CCD-level) metric defines the source count anchor, and any source count metric that is not CCD-level needs to override its tags accordingly. This coupling makes the file harder to read, understand, or modify.
If we do keep the YAML format, we might consider changing how the file is parsed to allow more flexibility in the YAML (e.g., by allowing a kind of "scratch space" that can define anchors but is ignored by lsst.verify).