Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11186

Reprocess RC with w_2017_32

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Reprocess the HSC RC dataset using w_2017_32 following the same steps as in DM-10129/DM-11020

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Started single frame processing with w_2017_32 that just became available in the shared stack an hour ago

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Started single frame processing with w_2017_32 that just became available in the shared stack an hour ago
            Hide
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            Got many 'No space left on device' errors; '/project' seems to be getting full.

            Had to remove some files to continue.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - - edited Got many 'No space left on device' errors; '/project' seems to be getting full. Had to remove some files to continue.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            singleFrameDriver/prcessCcd failed in 10 CCDs at WIDE, and 36 CCDs at COSMOS. This is expected by the config changes in DM-11090. The failed dataIds are the same as in week 30 (DM-11185).

            The failed data IDs are:

            --id visit=278 ccd=95 --id visit=280 ccd=22 --id visit=280 ccd=69 --id visit=284 ccd=61 --id visit=1206 ccd=77 --id visit=24308 ccd=29 --id visit=28376 ccd=69 --id visit=28380 ccd=0 --id visit=28382 ccd=101 --id visit=28392 ccd=101 --id visit=28392 ccd=102 --id visit=28394 ccd=93 --id visit=28396 ccd=102 --id visit=28398 ccd=95 --id visit=28398 ccd=101 --id visit=28400 ccd=5 --id visit=28400 ccd=10 --id visit=28400 ccd=15 --id visit=28400 ccd=23 --id visit=28400 ccd=26 --id visit=28400 ccd=40 --id visit=28400 ccd=53 --id visit=28400 ccd=55 --id visit=28400 ccd=61 --id visit=28400 ccd=68 --id visit=28400 ccd=77 --id visit=28400 ccd=84 --id visit=28400 ccd=89 --id visit=28400 ccd=93 --id visit=28400 ccd=94 --id visit=28400 ccd=95 --id visit=28400 ccd=99 --id visit=28400 ccd=100 --id visit=28400 ccd=101 --id visit=28400 ccd=102 --id visit=29324 ccd=99 --id visit=9868 ccd=76 --id visit=11582 ccd=76 --id visit=7344 ccd=67 --id visit=19468 ccd=69 --id visit=6478 ccd=99 --id visit=6528 ccd=24 --id visit=6528 ccd=67 --id visit=9736 ccd=67 --id visit=17738 ccd=69 --id visit=17750 ccd=58
            

            Show
            hchiang2 Hsin-Fang Chiang added a comment - singleFrameDriver/prcessCcd failed in 10 CCDs at WIDE, and 36 CCDs at COSMOS. This is expected by the config changes in DM-11090 . The failed dataIds are the same as in week 30 ( DM-11185 ). The failed data IDs are: --id visit= 278 ccd= 95 --id visit= 280 ccd= 22 --id visit= 280 ccd= 69 --id visit= 284 ccd= 61 --id visit= 1206 ccd= 77 --id visit= 24308 ccd= 29 --id visit= 28376 ccd= 69 --id visit= 28380 ccd= 0 --id visit= 28382 ccd= 101 --id visit= 28392 ccd= 101 --id visit= 28392 ccd= 102 --id visit= 28394 ccd= 93 --id visit= 28396 ccd= 102 --id visit= 28398 ccd= 95 --id visit= 28398 ccd= 101 --id visit= 28400 ccd= 5 --id visit= 28400 ccd= 10 --id visit= 28400 ccd= 15 --id visit= 28400 ccd= 23 --id visit= 28400 ccd= 26 --id visit= 28400 ccd= 40 --id visit= 28400 ccd= 53 --id visit= 28400 ccd= 55 --id visit= 28400 ccd= 61 --id visit= 28400 ccd= 68 --id visit= 28400 ccd= 77 --id visit= 28400 ccd= 84 --id visit= 28400 ccd= 89 --id visit= 28400 ccd= 93 --id visit= 28400 ccd= 94 --id visit= 28400 ccd= 95 --id visit= 28400 ccd= 99 --id visit= 28400 ccd= 100 --id visit= 28400 ccd= 101 --id visit= 28400 ccd= 102 --id visit= 29324 ccd= 99 --id visit= 9868 ccd= 76 --id visit= 11582 ccd= 76 --id visit= 7344 ccd= 67 --id visit= 19468 ccd= 69 --id visit= 6478 ccd= 99 --id visit= 6528 ccd= 24 --id visit= 6528 ccd= 67 --id visit= 9736 ccd= 67 --id visit= 17738 ccd= 69 --id visit= 17750 ccd= 58
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Out of the 46 singleFrameDriver/prcessCcd fatal failures:

            • 28 failed with "Unable to match sources"
            • 3 failed with "No sources remaining in match list after magnitude limit cuts"
            • 4 failed with "No objects passed our cuts for consideration as psf stars"
            • 11 failed with "PSF star selector found [123] candidates"

            A retry log with the 46 failures is attached.

            In mosaic jobs, still see many warnings like afw.table WARN: Persisted match record with ID 111331498155159102 not found in catalog 1. as before (DM-11356).

            In coadd and multiband, the numbers of products and the empty patches are the same as w_2017_28 (DM-11184) (as in this comment). No fatal errors were seen in the logs.

            I had to remove files from /project to make space for this new run. Free space is running low.

            The output repo is at /datasets/hsc/repo/rerun/RC/w_2017_32/DM-11186/
            Logs are at /datasets/hsc/repo/rerun/RC/w_2017_32/DM-11186/logs

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Out of the 46 singleFrameDriver/prcessCcd fatal failures: 28 failed with "Unable to match sources" 3 failed with "No sources remaining in match list after magnitude limit cuts" 4 failed with "No objects passed our cuts for consideration as psf stars" 11 failed with "PSF star selector found [123] candidates" A retry log with the 46 failures is attached. In mosaic jobs, still see many warnings like afw.table WARN: Persisted match record with ID 111331498155159102 not found in catalog 1. as before ( DM-11356 ). In coadd and multiband, the numbers of products and the empty patches are the same as w_2017_28 ( DM-11184 ) ( as in this comment ). No fatal errors were seen in the logs. I had to remove files from /project to make space for this new run. Free space is running low. The output repo is at /datasets/hsc/repo/rerun/RC/w_2017_32/ DM-11186 / Logs are at /datasets/hsc/repo/rerun/RC/w_2017_32/ DM-11186 /logs

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: