I compared Lauren MacArthur's list and the data available at /datasets/hsc/. Below are the visit numbers to be included. Missing visits are noted. Some missing visits are not really the newest data. Lauren MacArthur may you please take a quick look of the missing visits; is it okay not including those?

(1) Cosmos to full depth: (part of SSP_UDEEP_COSMOS)

• HSC-G: 11690..11712:2^29324^29326^29336^29340^29350
  (missing 29352)
• HSC-R: 1202..1220:2^23692^23694^23704^23706^23716^23718
• HSC-I: 1228..1232:2^1238..1248:2^19658^19660^19662^19680^19682^19684^19694^19696^19698^19708^19710^19712^30482..30504:2
  (missing 1236)

• HSC-Y: 318^322^324..332:2^344..362:2^1868^1870..1876:2^1880^1882^11718..11740:2^22602..22608:2^22626..22632:2^22642..22648:2^22658..22664:2
  (missing 274..302:2^306..316:2^320^334^342^364^366^368^370^1858..1862:2^1878^11742)

• HSC-Z: 1166..1194:2^17900..17908:2^17926..17934:2^17944..17952:2^17962
  (missing 28354..28402:2)

• NB0921: 23038..23056:2^23594..23606:2^24298..24310:2^25810..25816:2

(2) Two tracts of wide: (part of SSP_WIDE)

• HSC-G: 9852^9856^9860^9868^9870^9888^9898^9900^9904^9906^9912^11568^11572^11576^11582^11588^11590^11596^11598
  (missing 9864^9890)

• HSC-R: 11442^11446^11450^11470^11476^11478^11506^11508^11532^11534
• HSC-I: 7300^7304^7308^7318^7322^7338^7340^7344^7348^7358^7360^7374^7384^7386^19468^19470^19482^19484^19486

• HSC-Z: 9708^9712^9716^9724^9726^9730^9732^9736^9740^9750^9752^9764^9772^9774^17738^17740^17750^17752^17754

• HSC-Y: 6478^6482^6486^6496^6498^6522^6524^6528^6532^6544^6546^6568^13152^13154

The missing visits are also a mystery to me. A quick look at HCS-1361 indicates those visits are indeed included in the RC dataset. I must defer to Paul Price as to wether their omission was intentional or accidental.
[But note that Paul is likely unavailable for comment for ~2 weeks]

If you are in a rush to get some processing going, it's probably ok to run with the missing visits, but the outputs may not be as useful if we want the original RC dataset as our basis for quality analyses.

From John Swinbank, I understand that the HSC stack has not converged yet, and a stack newer than w_2017_14 will be wanted for the full processing. So the processing in this ticket would be "just" an exercise rather than actually needing QA. I've created DM-10129 which will use an agreed version. Meanwhile I'm trying to obtain those missing visits in DM-10128

I noticed that Lauren MacArthur's RC visits are different from the verification data Paul Price used for testing hscPipe 5.0-beta1 on Perseus (from his group email "Running verification data" on Apr 1). Paul included more visits in COSMOS. Also the tracts of WIDE are different.

I'll stick to Lauren MacArthur RC dataset. The visit IDs have been copied and pasted to https://confluence.lsstcorp.org/display/DM/S17B+HSC+PDR1+reprocessing

All 320 visits, including those newly added in DM-10128, have been processed through singleFrameDriver using the w_2017_14 stack on lsstvc and its default hsc configs; ccd=9 was excluded as it's known to be bad.
There were 46 (reproducible) failures; their dataIds are (visit: ccd):
278: 95, 280: 22, 69, 284: 61, 1206: 77, 6478: 99, 6528: 24, 67, 7344: 67, 9736: 67, 9868: 76, 17738: 69, 17750: 58, 19468: 69, 24308: 29, 28376: 69, 28380: 0, 28382: 101, 28392: 102, 28394: 93, 28396: 102, 28398: 95, 101, 28400: 5 ,10 ,15 ,23 ,26 ,40 ,53 ,55 ,61 ,68 ,77 ,84 ,89 ,92 ,93 ,94 ,95 ,99 ,100 ,101 ,102, 29324: 99, 29326: 47

The current master of meas_mosaic does not work; minimally I need two changes, both of which are already part of DM-9862; I pushed the used version to branch u/hfc/DM-10084. With that branch, I still see errors in writeCatalog when "--diagnostics" is used, so I went without it. Also, although MosaicTask ran, it did not output wcs/fcr files for every CCD of all input visits. Especially the edge CCDs were missed more often. I'm not sure if that is expected but will wait for DM-9862 to fix the master first.

I went on the processing with and without the meas_mosaic wcs/fcr. For the WIDE subset, I was able to process through coaddDriver and multiBandDriver. Expected output data for all 162 patches in tract 8766 and 8767 were generated. I'm still working on the COSMOS subset.

A skymap generated by running makeSkyMap.py with hsc defaults is used. The skymap is different from what Lauren used in her notes, and it put COSMOS in tract 9813.

As noted by Paul, the COSMOS HSC-Y band coadd needs more resources than others. Nonetheless they all processed through. For this COSMOS subset, everything falls within tract 9813. There are no results in some edge patches, as expected. A plot showing where the (HSC-R) visits are w.r.t. the tract/patch/skymap is uploaded. The outputs have 78 patches in HSC-G, 74 in HSC-R, 79 in HSC-I, 79 in HSC-Y, 79 in HSC-Z, and 76 in NB0921. As suggested by Paul, I use a config override config.assembleCoadd.subregionSize = [10000, 50]; I used it for all coaddDrivers although it's probably only needed for COSMOS HSC-Y for the RC dataset.

The processed results, with or without the meas_mosaic step, are accessible as butler repos at

/datasets/hsc/repo/rerun/private/hchiang2/RC/DM-10084-mosaic/
/datasets/hsc/repo/rerun/private/hchiang2/RC/DM-10084-noMosaic/

(note: These butler repos cannot be read with their real paths e.g. /project/hsc_rc/DM-10084-mosaic because of DM-10268)