Would you be willing to review this small config update, Ken? I'll make the PR momentarily.

PR is here: https://github.com/lsst/ap_pipe/pull/116

I did not run Jenkins since I don't think it has any ability to vet pipeline configs.

I'll pull the ticket branch and test myself just to make sure it will work from a fresh install.

I ran a test on a small bit of the HSC data in the 2021 diffim sprint and it looks like it's working well. 

An interesting update on this issue: I don't think it's quite as simple as we first thought. Since I couldn't figure out why I didn't see the same failure when working on DM-33911, I did another test where I set up weekly 25 (has the fix from 35162 but not this one) and then ran exactly what Meredith Rawls tried modulo the output collection name and restricting to a single patch for speed:

pipetask --long-log run -j 12 -b /repo/dc2 -d "skymap='DC2' AND tract=4431 AND patch IN (9)" -i 2.2i/defaults -o u/kherner/DM-35359_w25 -p $AP_PIPE_DIR/pipelines/LsstCamImSim/ApTemplate.yaml --register-dataset-types --clobber-outputs

Sure enough I could reproduce the problem. Then, without any changes, I tried doing something very similar on some of what goes into the diffim sprint datasets:

pipetask run -j8 -i HSC/runs/RC2/w_2022_22/DM-34983 -b /repo/main -d "tract=9813 AND instrument='HSC' AND skymap='hsc_rings_v1' AND patch=40" -p $AP_PIPE_DIR/pipelines/HyperSuprimeCam/ApTemplate.yaml -o u/kherner/DM-35359_w25 --register-dataset-types --clobber-outputs

but that worked! You'll immediately see two differences:

1) Different pipeline yaml files

2) Different input collections (from different repos)

 As for the first, they do indeed pull in (very) different makeWarp.py files ($AP_PIPE_DIR/config/LSSTCam-imSim/makeWarp.py and $AP_PIPE_DIR/config/HSC/makeWarp.py), but there is nothing about doApplyFinalizedPsf in either one! Maybe it's getting set indirectly by some other setting that's different between the two (I would have to look at the code again) but I'm not sure what that mechanism would be. As for the second, it's not clear why the input collection would matter.

Finally, I don't think any of that is a reason not to go ahead and merge this change in, since it certainly does what we want, but I wanted to bring it up in case we're missing something more sinister going on behind the scenes.

I also think it's good to merge this, but I wanted to link in two related tickets which are probably related to the trouble: DM-31063 and its followup DM-34699.

We figured out the source of the discrepancy! /repo/main has registered the finalized_psf_ap_corr_catalog dataset type thanks to myriad DRP reruns, and in fact, the "kitchen sink" input collection Ken was using included instances of those datasets. We were therefore both semi-unknowingly running with doApplyFinalizedPsf = True. I say "semi-unknowingly" because, as DM-35504 now explains, we exist in a config limbo-land in which configs set in ap_pipe/pipelines/ApTemplate.py get blown away and overwritten by configs set in ap_pipe/config/$CAMERA/whateverTask.py. 

I'm going to add a warning at the top of the "base" ApTemplate and ApPipe warning about this state of affairs that can be removed whenever DM-35504 is done.