Type: Technical task
Fix Version/s: None
Component/s: cp_pipe, daf_butler, ip_isr, obs_base, pipe_base, pipe_tasks
Team:Data Release Production
DM-24432 subtasks will have added all butler-level functionality needed to use CALIBRATION collections, so this ticket is mostly about fixing downstream code (particular QG generation and various DatasetType definitions) before finally removing the calibration_label dimension from daf_butler.
Status report: I think I've got everything but pipelines_check rebased, through review, and passing tests locally. I'm still waiting on my local ci_hsc_gen3 to finish running, though, to in order to generate the new data for pipelines_check. And once pipelines_check is done locally, I'll need one more Jenkins run for it all.
I did just notice Simon Krughoff's comment on
DM-26738 about Gen3 breakage to the new validate_drp_gen3 package, and if I can I'd like to test that I haven't broken that as well. From what I know about it, I don't think I will break it, since it runs gen2to3 conversion and I'm not breaking Gen2 repos, but I'd like to check if I can. However, I don't actually see it in the lsst or lsst-dm GitHub orgs. Anyone know where I can find it, or better, how I can run it through Jenkins?
pipelines_check is updated and two Jenkins runs are off to the races (stagged because ci_hsc was ready first and will probably take longer)
- lsst_distrib + lsst_ci: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32718/pipeline/
- lsst_distrib + ci_hsc: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32717/pipeline
The PR for pipelines_check is here if anyone else is still following along, but I'll merge that without review if all tests pass, and deal with any style concerns later.
I think https://github.com/lsst-dmsst/metric-pipeline-tasks is the package Simon fixed today.
I have no objection to merging today; other tickets have kept ap_verify from running in Gen 3 this week anyway.