My attempt to add post-qa json archiving to the validate_drp job didn't work due to required post-qa params (DM-9421). validate_drp has been fixed up and a new test build is running.

The validate_drp changes are now working in a sandbox env.

The decam dataset was failing on master and was disabled. I've rebased test branch for jenkins-dm-jobs on the current master. Assuming this builds cleanly over the weekend, I am planning to [finally]] deploy it to master on Monday. Note that the el7 build slaves will need to be re-provisioned at that time to increase the available disk space.

The changes to jenkins-dm-jobs have been merged and the production el7 slaves have been re-provisioned with 1.5TiB of disk.

This issue is being left open until it has been confirmed (tomorrow) that validate_drp has successfully run overnight on its new schedule.

For the last two nights, the valiation_data_hsc build has run ~14 hours because it landed on two different slaves (el7-3 & el7-1). This isn't unexpected, however, I'm going to move the start time up to 1700 project time. I also noticed how large the validation_data_hsc output json files are – several 100MiBs combined. I'm going update the job to compress them.