Thanks Sophie, and thanks Chris for your help too. This ticket removes unnecessary wcs and photoCalib code from processCcdWithFakes.py, and corrects a problematic typo that crops up further down the line. I can confirm that I'm able to successfully run both ap_verify and DRPFakes with this ticket.
For testing, I first set up:
setup lsst_distrib -t w_2021_43
|
setup -j -r ap_verify
|
setup -j -r ap_verify_ci_cosmos_pdr2
|
setup -j -r pipe_tasks
|
setup -j -r obs_subaru
|
setup -j -r ci_hsc_gen3
|
(all on their master branches). I believe these are all the repos which were touched since DM-31491 dropped, plus the required ap_verify repos.
Next, I confirm that running on 'master':
ap_verify.py -j 8 \
|
--dataset ap_verify_ci_cosmos_pdr2 \
|
--output /project/lskelvin/apverify/run/ \
|
--gen3 \
|
--data-query "visit in (59150, 59160) and band='g'" \
|
-p $AP_VERIFY_CI_COSMOS_PDR2_DIR/pipelines/ApVerifyWithFakes.yaml \
|
2>&1 | tee log-ApVerifyWithFakes.log
|
fails with the above joincal error message.
Switching pipe_tasks from 'master' to 'tickets/DM-32376' and then re-running now allows for ap_verify to complete successfully, with outputs as expected.
I also ran on this ticket branch:
pipetask --long-log run --register-dataset-types \
|
-b /repo/main \
|
-i HSC/runs/RC2/w_2021_42/DM-32248,u/sr525/ssiCats \
|
-o u/lskelvin/ap_verify_fakes_test_ticket \
|
-p $OBS_SUBARU_DIR/pipelines/DRPFakes.yaml \
|
-d "instrument='HSC' AND skymap='hsc_rings_v1' AND band='i' AND detector=1 AND visit=1228" \
|
2>&1 | tee log-DRPFakes-ticket.log
|
which also appears to complete successfully, producing outputs as expected.
Assuming Jenkins doesn't produce any red flags, then I think this is good to merge. Thanks!
Looking at this, I notice that the dimensions of the new fake externalSkyWcsTractCatalog https://github.com/lsst/pipe_tasks/blob/master/python/lsst/pipe/tasks/processCcdWithFakes.py#L73-L79
are different than the dimensions of the old externalSkyWcsTractCatalog
https://github.com/lsst/pipe_tasks/blob/master/python/lsst/pipe/tasks/makeCoaddTempExp.py#L617-L623
And I think this might be the problem. The original one is missing skymap as a dimension; does this mean that this is wrong?