If possible, using SLURM per DM-14259; if that turns out to be difficult, we shouldn't let it block this but rather should run on a single node.

I manually ingested the ap_verify_hits2015 dataset with no major issues. I'm not using slurm this time, so ap_pipe.py will be chugging along for a bit on lsst-dev.

The initial stab at this is on lsst-dev in /project/mrawls/hits2015/rerun/take1. It looks like about a third of the visits didn't process due to a problem with the calib validity ranges. The error was "No registry for lookup" for a bunch of cpBiases. I thought –validity 999 was adequate during calib ingestion, but it doesn't seem to be. I manually compared the calibRegistry with the one Eric Bellm made previously and his cpBias validity ranges are adequate to encompass all the data while mine are not.

As Eric Bellm, John Swinbank, and I discussed in person, this ticket was successfully completed by doing a partial reprocessing on lsst-dev in /project/mrawls/hits2015. A new ticket to troubleshoot why ~1/3 of the data didn't process and fix the problem has been created as DM-14761.

Hi Meredith Rawls, could you just run your notebook from DM-13687 on this output and drop it in the ap_pipe/u/mrawls/notebooks-etc branch? No need to update the code, but it would be nice to have at least a small tangible record of the status of this reprocessing while we wait for DM-14762.

As we discussed, I actually did have to update my notebook analysis code since the database schema changed significantly (in a good way!). I figure I may as well include this latest notebook with a few preliminary light curves in a new notebooks directory in ap_pipe. The PR I just made does this. If you disagree I am happy to move it somewhere else.

Hurrah! Great work. Onwards to SLURM-ing (which will hopefully clean up the templates)...