w_2017_28 was not released last weekend so it's not yet available in the shared stack on lsst-dev so far. Therefore I cannot start this run yet. As there is a regularly scheduled maintenance this Thursday this ticket may be further delayed.

w_2017_28 just became available on the shared stack. With it I have scons-ed the current mater of meas_mosaic (f4b19d1) and submitted the first batch of jobs. The slurm queue has been occupied so it may be a while before these jobs run.

Hitting IHS-341 again with No space left on device at /project

Single Frame Processing

81 CCDs failed, 62 from COSMOS and 19 from WIDE. The failed data IDs are the same as in w_2017_25 (DM-11020) and w_2017_26 (DM-11165).

Mosaic

Completed.

coadd

In COSMOS tract 9813, the coadd products have 78 patches in HSC-G, 79 in HSC-I, 74 in HSC-R, 79 in HSC-Y, 79 in HSC-Z, and 76 in NB0921; in total 465 patches. The empty patches of tract 9813 are:

• patch 0,0 HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921
• patch 0,8 HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921
• patch 0,1 HSC-R
• patch 0,7 HSC-R,NB0921
• patch 1,8 HSC-R,NB0921
• patch 8,0 HSC-G,HSC-R
• patch 8,8 HSC-R,NB0921

In WIDE, both tract 8766 and 8767 have all 81 patches of coadd products in all 5 bands.

Compared with the coadd by w_2017_26 (DM-11165), some empty patches are now filled. Much more calexps are getting selected for making coadds; in other words, fewer calexps are being removed by PsfWcsSelectImagesTask. This is expected by the merge of DM-11243.

MULTIBAND

Completed: 81 patches in WIDE tracts 8766 and 8767, 79 patches in COSMOS tract 9813.

Because the inode shortage at /project earlier this week, jobs starting with mosaic were run and outputted to /scratch space. A job to sync up the repos is pending in the lsstvc slurm queue.

Logs have been copied to /project/hsc_rc/w_2017_28/DM-11184/logs/

Syncing has finished.

The nominal repo path is /datasets/hsc/repo/rerun/RC/w_2017_28/DM-11184/
(realpath: /project/hsc_rc/w_2017_28/DM-11184)

With w_2017_28 the mosaic jobs had many warnings about the missed matches, e.g.

Persisted match record with ID 111331498155159102 not found in catalog 1

This has been reported in DM-11356.