The verify_ap system will bring together the Minimum Viable System (MVS) with LSST's verify framework. Its primary role is to automatically process a test dataset through the Level 1 pipeline via continuous integration so that Alert Production (AP) performance metrics can be monitored. It will also be straightforward for any user to run on a test dataset of their choice. The main components of verify_ap are summarized nicely in the Venn Diagram by Eric Bellm on the MVS Confluence Page.
Currently, a prototype version of the MVS portion of verify_ap lives on github in lsst-dm/decam_hits. The intention is to rename this repo to ap_pipe (which is called by verify_ap) once it more closely resembles a finished product.
Our first goal is to have a functional running demonstration of verify_ap in time for the August 2017 Project & Community "All Hands" Meeting. The work is being done by a core team in UW DM, led by Eric and Simon: Meredith, David, Chris, and Krzysztof.
The main pieces that need to be done are grouped into epics as follows:
DM-9676: Promote prototype AP system to verify_ap
DM-10770: Implement initial metrics for MVS
DM-10771: Identify and procure datasets for calculating metrics for MVS
DM-10773: Design and implement MVS for alert production
My (Meredith's) main role in the next several weeks is to get the MVS in a more useful state so that it can be plugged into the verify_ap system. David is laying the groundwork for verify_ap datasets. The initial test dataset needs to be easily accessible by a continuous integration system, and it also should be straightforward to use verify_ap with other datasets in the future. Chris' piece fits in at the end of the proto-MVS, which currently yields DIASources but will need to create the final data product, DIAObjects. Krzysztof is working on initial metrics, such as runtime measures and counting the number of DIAObjects created. He will also begin making a high-level verify_ap package from which the MVS and friends can be called.