Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13066

Reprocess RC1 with w_2018_02

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Reprocess the HSC RC1 dataset, as defined in DMTR-31 Sect. 4.1 , using Stack w_2018_02. Steps include makeSkyMap.py, singleFrameDriver.py, mosaic.py, coaddDriver.py, and multiBandDriver.py (DM-10129/DM-11020).

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            started last saturday

            Show
            hchiang2 Hsin-Fang Chiang added a comment - started last saturday
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            SingleFrameDriver/prcessCcd failed in 11 CCDs at WIDE, and 19 CCDs at COSMOS. Their data IDs are:

            --id visit=278 ccd=95 --id visit=280 ccd=61 --id visit=280 ccd=69 --id visit=280 ccd=103 --id visit=284 ccd=61 --id visit=11698 ccd=68 --id visit=17934 ccd=1 --id visit=23600 ccd=1 --id visit=28376 ccd=69 --id visit=28382 ccd=101 --id visit=28392 ccd=101 --id visit=28396 ccd=102 --id visit=28398 ccd=95 --id visit=28398 ccd=101 --id visit=28400 ccd=53 --id visit=28400 ccd=61 --id visit=28400 ccd=95 --id visit=28400 ccd=100 --id visit=28400 ccd=101 --id visit=9868 ccd=76 --id visit=11582 ccd=76 --id visit=7344 ccd=67 --id visit=19468 ccd=69 --id visit=6478 ccd=99 --id visit=6528 ccd=24 --id visit=6528 ccd=67 --id visit=9708 ccd=99 --id visit=9736 ccd=67 --id visit=17738 ccd=69 --id visit=17750 ccd=58
            

            Out of the 30 singleFrameDriver/prcessCcd fatal failures:

            • 5 failed with "Unable to match sources"
            • 4 failed with "No objects passed our cuts for consideration as psf stars"
            • 3 failed with "RuntimeError: No matches to use for photocal"
            • 18 failed with "PSF star selector found [123] candidates"

            Compared to w_2017_50 (DM-12929), there is one fewer "RuntimeError: No matches to use for photocal":
            visit=23596 ccd=6 (filter=NB0921) failed with w_2017_50 but passed since w_2017_51. Another two differences are visit=280 ccd=61 and visit=280 ccd=22.

            The difference happened between w_2017_50 and w_2017_51:

            • Passed with w_2017_50 but failed with w_2017_51
              --id visit=280 ccd=61
            • Failed with w_2017_50 but passed with w_2017_51
              --id visit=280 ccd=22
              --id visit=23596 ccd=6

            visit=280 (filter=HSC-Y) is known to be a problematic visit and Lauren MacArthur has already flagged it out for RC2.

            The changes are likely due to DM-12979.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - SingleFrameDriver/prcessCcd failed in 11 CCDs at WIDE, and 19 CCDs at COSMOS. Their data IDs are: --id visit= 278 ccd= 95 --id visit= 280 ccd= 61 --id visit= 280 ccd= 69 --id visit= 280 ccd= 103 --id visit= 284 ccd= 61 --id visit= 11698 ccd= 68 --id visit= 17934 ccd= 1 --id visit= 23600 ccd= 1 --id visit= 28376 ccd= 69 --id visit= 28382 ccd= 101 --id visit= 28392 ccd= 101 --id visit= 28396 ccd= 102 --id visit= 28398 ccd= 95 --id visit= 28398 ccd= 101 --id visit= 28400 ccd= 53 --id visit= 28400 ccd= 61 --id visit= 28400 ccd= 95 --id visit= 28400 ccd= 100 --id visit= 28400 ccd= 101 --id visit= 9868 ccd= 76 --id visit= 11582 ccd= 76 --id visit= 7344 ccd= 67 --id visit= 19468 ccd= 69 --id visit= 6478 ccd= 99 --id visit= 6528 ccd= 24 --id visit= 6528 ccd= 67 --id visit= 9708 ccd= 99 --id visit= 9736 ccd= 67 --id visit= 17738 ccd= 69 --id visit= 17750 ccd= 58 Out of the 30 singleFrameDriver/prcessCcd fatal failures: 5 failed with "Unable to match sources" 4 failed with "No objects passed our cuts for consideration as psf stars" 3 failed with "RuntimeError: No matches to use for photocal" 18 failed with "PSF star selector found [123] candidates" Compared to w_2017_50 ( DM-12929 ), there is one fewer "RuntimeError: No matches to use for photocal": visit=23596 ccd=6 (filter=NB0921) failed with w_2017_50 but passed since w_2017_51. Another two differences are visit=280 ccd=61 and visit=280 ccd=22 . The difference happened between w_2017_50 and w_2017_51: Passed with w_2017_50 but failed with w_2017_51 --id visit=280 ccd=61 Failed with w_2017_50 but passed with w_2017_51 --id visit=280 ccd=22 --id visit=23596 ccd=6 visit=280 (filter=HSC-Y) is known to be a problematic visit and Lauren MacArthur has already flagged it out for RC2. The changes are likely due to DM-12979 .
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Attached are a log of the 30 processCcd failures, and screen shots of the shallow visit=280.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Attached are a log of the 30 processCcd failures, and screen shots of the shallow visit=280.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            The numbers of coadd/multiband products and the empty patches are the same as w_2017_50 (DM-12929, this comment). No fatal errors were seen in the logs; many warnings as before.

            The output repo is available at /datasets/hsc/repo/rerun/RC/w_2018_02/DM-13066/

            Logs have been copied to /datasets/hsc/repo/rerun/RC/w_2018_02/DM-13066/logs/

            Show
            hchiang2 Hsin-Fang Chiang added a comment - - edited The numbers of coadd/multiband products and the empty patches are the same as w_2017_50 ( DM-12929 , this comment ). No fatal errors were seen in the logs; many warnings as before. The output repo is available at /datasets/hsc/repo/rerun/RC/w_2018_02/ DM-13066 / Logs have been copied to  /datasets/hsc/repo/rerun/RC/w_2018_02/ DM-13066 /logs/

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: