Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12212

Reprocess RC with w_2017_42

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Reprocess the HSC RC dataset, as defined in DMTR-31 Sect. 4.1 , using Stack w_2017_42. Steps include makeSkyMap.py, singleFrameDriver.py, mosaic.py, coaddDriver.py, and multiBandDriver.py (DM-10129/DM-11020).

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            SingleFrameDriver/prcessCcd failed fatally in 29 CCDs, same as in DM-11980 / w_2017_40 / this comment

            Show
            hchiang2 Hsin-Fang Chiang added a comment - SingleFrameDriver/prcessCcd failed fatally in 29 CCDs, same as in DM-11980 / w_2017_40 / this comment
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            The numbers of coadd/multiband products and the empty patches are the same as w_2017_28 (DM-11184 , this comment). No fatal errors were seen in the logs; many warnings as before.

            In coadd, one common warning is like this:

            29925 WARN  2017-10-23T20:25:36.221-0500 coaddDriver.makeCoaddTempExp: Warp DataId(initialdata={'filter': 'HSC-G', 'tract': 9813, 'visit': 29336, 'patch': '8,0'}, tag=set([])) could not be created
            

            There are 413 of them with w_2017_42. They are followed by a warning like this:

            29925 WARN  2017-10-23T20:51:00.985-0500 coaddDriver.assembleCoadd: Could not find deepCoadd_directWarp DataId(initialdata={'filter': 'HSC-G', 'tract': 9813, 'visit': 29336, 'patch': '8,0'}, tag=set([])); skipping it
            

            The warp is not created because:

            29925 INFO  2017-10-23T20:25:36.219-0500 coaddDriver.makeCoaddTempExp: directWarp has 0 good pixels (0.0%)
            

            I did not check them all, but suspect many of them could be cases like this: https://confluence.lsstcorp.org/display/~hchiang2/Collection+of+edge+cases

            Another common warning in coadd is:

            WARN  2017-10-23T20:40:02.319-0500 coaddDriver.assembleCoadd ()(assembleCoadd.py:835)- Expected to see filter in metadata
            

            In multiband, one common warning, seen 48 times in the log, is:

            10200 WARN  2017-10-24T20:34:37.391-0500 multiBandDriver.measureCoaddSources: Patch DataId(initialdata={'filter': ... , 'patch':..., 'tract':....}, tag=set([])) contains 1 large footprints that were not deblended
            

            A few of this is seen:

            10289 WARN  2017-10-24T22:24:21.607-0500 meas_deblender.baseline: Skipping peak at (30862.0, 34736.0): no unmasked pixels nearby
            

            Show
            hchiang2 Hsin-Fang Chiang added a comment - The numbers of coadd/multiband products and the empty patches are the same as w_2017_28 ( DM-11184 , this comment ). No fatal errors were seen in the logs; many warnings as before. In coadd, one common warning is like this: 29925 WARN 2017 - 10 -23T20: 25 : 36.221 - 0500 coaddDriver.makeCoaddTempExp: Warp DataId(initialdata={ 'filter' : 'HSC-G' , 'tract' : 9813 , 'visit' : 29336 , 'patch' : '8,0' }, tag=set([])) could not be created There are 413 of them with w_2017_42 . They are followed by a warning like this: 29925 WARN 2017 - 10 -23T20: 51 : 00.985 - 0500 coaddDriver.assembleCoadd: Could not find deepCoadd_directWarp DataId(initialdata={ 'filter' : 'HSC-G' , 'tract' : 9813 , 'visit' : 29336 , 'patch' : '8,0' }, tag=set([])); skipping it The warp is not created because: 29925 INFO 2017 - 10 -23T20: 25 : 36.219 - 0500 coaddDriver.makeCoaddTempExp: directWarp has 0 good pixels ( 0.0 %) I did not check them all, but suspect many of them could be cases like this: https://confluence.lsstcorp.org/display/~hchiang2/Collection+of+edge+cases Another common warning in coadd is: WARN 2017 - 10 -23T20: 40 : 02.319 - 0500 coaddDriver.assembleCoadd ()(assembleCoadd.py: 835 )- Expected to see filter in metadata In multiband, one common warning, seen 48 times in the log, is: 10200 WARN 2017 - 10 -24T20: 34 : 37.391 - 0500 multiBandDriver.measureCoaddSources: Patch DataId(initialdata={ 'filter' : ... , 'patch' :..., 'tract' :....}, tag=set([])) contains 1 large footprints that were not deblended A few of this is seen: 10289 WARN 2017 - 10 -24T22: 24 : 21.607 - 0500 meas_deblender.baseline: Skipping peak at ( 30862.0 , 34736.0 ): no unmasked pixels nearby
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            The output repo is available at /datasets/hsc/repo/rerun/RC/w_2017_42/DM-12212/

            Logs have been copied to /datasets/hsc/repo/rerun/RC/w_2017_42/DM-12212/logs/

            Show
            hchiang2 Hsin-Fang Chiang added a comment - The output repo is available at /datasets/hsc/repo/rerun/RC/w_2017_42/ DM-12212 / Logs have been copied to /datasets/hsc/repo/rerun/RC/w_2017_42/ DM-12212 /logs/

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: