Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14286

CompareWarpAssembleCoadd RuntimeError when No PsfMatched warps found at edge cases

    XMLWordPrintable

Details

    • Story
    • Status: To Do
    • Resolution: Unresolved
    • None
    • None

    Description

      In some edge cases, there are good pixels for directWarp but 0 good pixels for psfMatchedWarp. The deepCoadd_directWarp files are written, but no deepCoadd_psfMatchedWarp files are written. Then the following message about not being able to find psf-Matched warps is seen: (from DM-12391)

      coaddDriver WARN: lsst-verify-worker17:90693: Caught RuntimeError while coadding DataId(initialdata={'tract': 9463, 'filter': 'NB0816', 'patch': '7,0'}, tag=set()): No PsfMatched warps were found to build the template coadd which is
                    required to run CompareWarpAssembleCoaddTask. To continue assembling this type of coadd,
                    first either rerun makeCoaddTempExp with config.makePsfMatched=True or
                    coaddDriver with config.makeCoadTempExp.makePsfMatched=True, before assembleCoadd.
       
                    Alternatively, to use another algorithm with existing warps, retarget the CoaddDriverConfig to
                    another algorithm like:
       
                      from lsst.pipe.tasks.assembleCoadd import SafeClipAssembleCoaddTask
                      config.assemble.retarget(SafeClipAssembleCoaddTask)
       
      coaddDriver INFO: lsst-verify-worker17:90693: Traceback:
      Traceback (most recent call last):
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/15.0-1-ga91101e+7/python/lsst/ctrl/pool/parallel.py", line 512, in logOperation
          yield
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_drivers/15.0-3-ga03b4ca+9/python/lsst/pipe/drivers/coaddDriver.py", line 299, in coadd
          coaddResults = self.assembleCoadd.run(patchRef, selectDataList)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/15.0-4-g654b129+5/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/15.0-5-g389937dc+5/python/lsst/pipe/tasks/assembleCoadd.py", line 361, in run
          supplementaryData = self.makeSupplementaryData(dataRef, selectDataList)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/15.0-5-g389937dc+5/python/lsst/pipe/tasks/assembleCoadd.py", line 1607, in makeSupplementaryData
          raise RuntimeError(message)
      RuntimeError: No PsfMatched warps were found to build the template coadd which is
                    required to run CompareWarpAssembleCoaddTask. To continue assembling this type of coadd,
                    first either rerun makeCoaddTempExp with config.makePsfMatched=True or
                    coaddDriver with config.makeCoadTempExp.makePsfMatched=True, before assembleCoadd.
       
                    Alternatively, to use another algorithm with existing warps, retarget the CoaddDriverConfig to
                    another algorithm like:
       
                      from lsst.pipe.tasks.assembleCoadd import SafeClipAssembleCoaddTask
                      config.assemble.retarget(SafeClipAssembleCoaddTask)
      

      In the PDR1 run with w_2018_15, it's seen in these 5 patches, 4 in DEEP and 1 in WIDE. To reproduce, one can run

      coaddDriver.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun DM-13666/[DEEP|UDEEP|WIDE]:private/username/abcd  .... 
      

      with the following data IDs:

      DEEP: --id tract=9463 filter=NB0816 patch=7,0 --selectId ccd=0..8^10..103 visit=37328^37330^37796^37798^37800^37802^37806^37808^37810^38474^38476^38478
      DEEP: --id tract=8525 filter=HSC-Z patch=1,5 --selectId ccd=0..8^10..103 visit=15038^15042^15046^15048^15060^15062^15072^15074^15076^44566^44568^44574^44576^44580^44594^44596^44598^44600^44602^44604^44606
      DEEP: --id tract=9465 filter=HSC-Z patch=5,6 --selectId ccd=0..8^10..103 visit=9628^9630^9648^9656^9660^9664^9672^36512^36516^36520^36524^36528^36532^36536^36544
      DEEP: --id tract=9465 filter=HSC-Z patch=4,8 --selectId ccd=0..8^10..103 visit=9628^9630^9648^9656^9660^9664^9672^36512^36516^36520^36524^36528^36532^36536^36544
      WIDE: --id tract=16821 filter=HSC-I patch=2,6 --selectId ccd=0..8^10..103 visit=30598^30600^30602^30604^30606^30608^30610
      

      Attachments

        1. rCoaB-89463.o138845
          298 kB
        2. rCoaB-I16821.o138849
          175 kB
        3. rCoaB-Z8525.o138846
          504 kB
        4. rCoaB-Z9465a.o138847
          367 kB
        5. rCoaB-Z9465b.o138848
          364 kB

        Issue Links

          Activity

            I'm attaching the 5 log files from running the 5 cases. 

            hchiang2 Hsin-Fang Chiang added a comment - I'm attaching the 5 log files from running the 5 cases. 
            tjenness Tim Jenness added a comment -

            This is four years old and used Gen 2. Did we have any similar problem to this for DP0.2 processing?

            tjenness Tim Jenness added a comment - This is four years old and used Gen 2. Did we have any similar problem to this for DP0.2 processing?

            People

              Unassigned Unassigned
              hchiang2 Hsin-Fang Chiang
              Hsin-Fang Chiang, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Jenkins

                  No builds found.