In the single frame processing, among the 320 visits (103 ccds each, as ccd=9 was excluded), there were 81 reproducible fatal failures. Their dataIds are:

• --id visit=278 ccd=95 --id visit=280 ccd=22 --id visit=280 ccd=69 --id visit=284 ccd=61 --id visit=1206 ccd=77 --id visit=1212 ccd=103 --id visit=24308 ccd=29 --id visit=28366 ccd=101 --id visit=28372 ccd=0 --id visit=28372 ccd=100 --id visit=28374 ccd=0 --id visit=28374 ccd=101 --id visit=28376 ccd=69 --id visit=28378 ccd=10 --id visit=28378 ccd=101 --id visit=28380 ccd=0 --id visit=28380 ccd=101 --id visit=28382 ccd=98 --id visit=28382 ccd=101 --id visit=28386 ccd=52 --id visit=28386 ccd=64 --id visit=28386 ccd=100 --id visit=28388 ccd=101 --id visit=28390 ccd=102 --id visit=28392 ccd=100 --id visit=28392 ccd=101 --id visit=28392 ccd=102 --id visit=28394 ccd=93 --id visit=28394 ccd=101 --id visit=28396 ccd=90 --id visit=28396 ccd=102 --id visit=28398 ccd=10 --id visit=28398 ccd=95 --id visit=28398 ccd=101 --id visit=28400 ccd=5 --id visit=28400 ccd=10 --id visit=28400 ccd=15 --id visit=28400 ccd=18 --id visit=28400 ccd=23 --id visit=28400 ccd=26 --id visit=28400 ccd=28 --id visit=28400 ccd=40 --id visit=28400 ccd=53 --id visit=28400 ccd=55 --id visit=28400 ccd=61 --id visit=28400 ccd=68 --id visit=28400 ccd=77 --id visit=28400 ccd=80 --id visit=28400 ccd=84 --id visit=28400 ccd=89 --id visit=28400 ccd=93 --id visit=28400 ccd=94 --id visit=28400 ccd=95 --id visit=28400 ccd=99 --id visit=28400 ccd=100 --id visit=28400 ccd=101 --id visit=28400 ccd=102 --id visit=28402 ccd=0 --id visit=28402 ccd=101 --id visit=29324 ccd=99 --id visit=28360 ccd=100 --id visit=28360 ccd=101 --id visit=9868 ccd=68 --id visit=9868 ccd=69 --id visit=9868 ccd=76 --id visit=9868 ccd=77 --id visit=9890 ccd=38 --id visit=9912 ccd=103 --id visit=11582 ccd=69 --id visit=7344 ccd=67 --id visit=19468 ccd=69 --id visit=6478 ccd=99 --id visit=6528 ccd=24 --id visit=6528 ccd=67 --id visit=6528 ccd=75 --id visit=9736 ccd=59 --id visit=9736 ccd=66 --id visit=9736 ccd=67 --id visit=17738 ccd=69 --id visit=17738 ccd=77 --id visit=17750 ccd=58

Out of the 81 failures:

• 62 failed with: "Unable to match sources"
• 4 failed with: "No objects passed our cuts for consideration as psf stars"
• 4 failed with: "No sources remaining in match list after magnitude limit cuts"
• 11 failed with: "PSF star selector found [123] candidates"

The rerun log of the 81 failed processCcd is attached.

Mosaic

Current master of meas_mosaic (f4b19d1) built with w_2017_25 is used. All 3 tracts completed.

Coadd

Compared to DM-10129, not every patch has coadd products generated with w_2017_25. Now there are more calexps being removed from coaddition because "because size scatter is too large" or "because median e residual too large".

COSMOS: In tract 9813, the coadd products have 75 patches in HSC-G, 74 in HSC-R, 79 in HSC-I, 79 in HSC-Y, 79 in HSC-Z, and 76 in NB0921. The empty patches are:

• HSC-G: 0,0^8,0^0,1^0,7^0,8^3,8
• HSC-I: 0,0^0,8
• HSC-R: 0,0^0,8^8,0^0,1^0,7^1,8^8,8
• HSC-Y: 0,0^0,8
• HSC-Z: 0,0^0,8
• NB0921: 0,0^0,7^0,8^1,8^8,8

Processing with stack w_2017_17 in DM-10129 did generate coadd files in 0,1 and 3,8 in the HSC-G filter. Otherwise the same.

WIDE:
In tract 8766 filter HSC-R, patches 7,8 and 0,8 are empty.
In tract 8767 filter HSC-R, these patches are empty: 8,7^0,8^1,8^2,8^8,8

MultiBand

Due to the bug of DM-11091 from DM-9998, multiBandDriver would attempt to do coadd source detection on empty patches and fail because no coadd data exist in those patches. The error messages were like this: "NoResults: No locations for get: datasetType:deepCoadd dataId ...")

For this ticket, I am continuing by only doing the multiband processing for patches where coadds of all filters exist. That is, specifying the safe patch IDs only in the multiBandDriver command line --id.

As only patches where coadds of all filters exist, there are:

• (COSMOS) 73 patches in tract=9813.
• (WIDE) 79 patches in tract=8766 and 76 patches in 8767.
  and other patches are ignored in the multiband processing.

The processed results are at this repo: /project/hsc_rc/DM-11020
Logs are at: /project/hsc_rc/DM-11020/logs

Everything here is moved to the new path /project/hsc_rc/w_2017_25/DM-11020 so it's easier to be found without the ticket number.