Making you reviewer on this Meredith so you can as me directly on Jira if you want me to look into some other output from the CI datasets.

Thanks Chris. So far it looks like we have the following for w18:

HSC visit 59160 detector 51
gen2 src count: 4024
gen3 src count: 4046
gen2 DiaSrc count: 163
gen3 DiaSrc count: 198

DECam visit 411371 detector 60
gen2 & gen3 src count: 2455 (consistent!)
gen2 DiaSrc count: 137
gen3 DiaSrc count: 255 --> I can explain this gen2/gen3 difference by doScaleDiffimVariance being turned on in gen3 only

And the following for w15:

HSC visit 59150 detector 50
gen2 src count: 3880
gen3 src count: 3870
gen2 DiaSrc count: 239
gen3 DiaSrc count: 274

DECam visit 411420 detector 5
gen2 & gen3 src count: 2108 (consistent!)
gen2 & gen3 DiaSrc count: 132 (consistent!)

I am confident that the remaining gen2/gen3 discrepancies are all happening in HSC, so we don't need to worry about the HiTS dataset anymore. Did you pull new changes to the cosmos CI dataset before running ap_verify? I merged a change about a week ago. If you did, I can state that the remaining gen2/gen3 HSC src and DiaSrc count discrepancies are independent of doScaleDiffimVariance. I'm not sure what to look at next.

I pulled the ap_verify_ci_cosmos_pdr2 data this week and just check and the repo I used is currently even with master.

Run 2 more checks. Simple image stats are the same for raws in both HiTS and HSC between Gen2/3 however the calexp stats differ for HSC pointing towards settings in calibrate and earlier causing some of the differences.

Great. Thanks for digging into this and sharing the notebook. It looks like the discrepancy is just HSC, in one of the "processCcd" (isr, characterizeImage, and/or calibrate) steps. I'll continue the investigation on DM-29344.