It would be useful to have “sensitivity tests" for metrics. This will require some thought to implement, but the basic idea is to develop a method to perturb some relevant values (and to decide which values are "relevant" for each metric) and capture how much metrics change in response. This will provide context when interpreting metrics.