Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ap_verify
-
Labels:None
-
Story Points:6
-
Epic Link:
-
Sprint:AP F18-4
-
Team:Alert Production
Description
As proposed in DM-13970, ap_verify should have a script that handles interaction between Jenkins (which should only need to know dataset names) and ap_verify itself. We should be able to model our script on validate_drp's solution, although the standardized nature of ap_verify datasets may simplify the implementation.
Hi Joshua Hoblitt, can you take a look at the script and see if it suits your needs? The script takes only one argument, the name of the dataset to be processed, and produces a number of metrics .json files as output.
On my laptop, a run of run_ci_dataset.sh -d CI-HiTS2015 takes just under 8 minutes. This is without parallelism. The six metrics .json files take up 1.1 MiB combined; note that these files have somewhat awkward names (e.g., ap_verify.visit=411371 ccdnum=56 filter=g.verify.json).