Meredith Rawls added a comment - 11/Aug/17 7:05 PM We have ingestion and calib ingestion working! Parts of it are a bit messy, but if one pulls the DM-11390 branch of ap_verify and ap_pipe , and does setup for both as well as for ap_verify_hits2015 , it is possible to run this from the command line and have it work through calibration ingestion: python ap_verify/bin.src/ap_verify.py --dataset HiTS2015 --output your_desired_output_dir --dataIdString "visit=410985" --silent (Note that the visit number doesn't matter just yet because ingestion and calib ingestion do not use any dataId info and instead they just ingest everything in the dataset repo provided.)

Meredith Rawls added a comment - 14/Sep/17 7:01 PM Thanks to help from Colin Slater , both ap_pipe and ap_verify may now successfully be run from the command line, for all steps from ingestion through difference imaging. For now, the template visit is hardwired to 410929 and the template ccdnum is hardwired to 25. This will be fixed when coadds are accepted as templates in DM-11422 . To try it out: setup obs_decam , ap_pipe , ap_verify_hits2015 , and ap_verify (as well as a non-buggy daf_persistence ) cd to wherever you cloned ap_pipe and ap_verify for ap_pipe alone: python ap_pipe/bin.src/ap_pipe.py -d ap_verify_hits2015/ -o zomg5 -i "visit=410985 ccdnum=25" (you should be able to choose a different visit from the Blind15A_40 field if you prefer) for ap_verify : python ap_verify/bin.src/ap_verify.py --dataset HiTS2015 --output zomg6 --dataIdString "visit=410985 ccdnum=25" --silent (again, you may choose a different Blind15A_40 visit if desired) Both of these have successfully produced output on lsst-dev in /project/mrawls/prototype_ap/ in the zomg5 and zomg6 directories, respectively. However, after ap_verify finishes running all the parts of ap_pipe , it does throw the following exception ap.verify.pipeline_driver.run_ap_pipe INFO: Pipeline complete Traceback (most recent call last): File "ap_verify/bin.src/ap_verify.py", line 27, in <module> run_ap_verify() File "/project/mrawls/prototype_ap/ap_verify/python/lsst/ap/verify/ap_verify.py", line 162, in run_ap_verify _measure_final_properties(metadata, job) File "/project/mrawls/prototype_ap/ap_verify/python/lsst/ap/verify/ap_verify.py", line 140, in _measure_final_properties measurements = measure_from_metadata(metadata) File "/project/mrawls/prototype_ap/ap_verify/python/lsst/ap/verify/measurements/compute_metrics.py", line 60, in measure_from_metadata for metric, task in timing_map: File "/home/mrawls/daf_persistence/python/lsst/daf/persistence/policy.py", line 168, in __getitem__ for key in name.split('.'): AttributeError: 'int' object has no attribute 'split' Once this is resolved ( Krzysztof Findeisen ?), and I write a few tests for ap_pipe as promised, this ticket will be ready for review. I plan to upate the DMTN-039 technote for ap_pipe as part of DM-11422 when the coadd template functionality is implemented.

Meredith Rawls added a comment - 20/Sep/17 7:33 PM Krzysztof Findeisen , please let me know if you would rather not do this review.

Krzysztof Findeisen added a comment - 22/Sep/17 12:04 PM I've left comments on the various branches. I would still like to see ap_verify not need to know about ap_pipe 's output organization, but hiding that inside ap_pipe is probably best left to a separate ticket.