Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14303

detectCoaddSources RuntimeError: Variance rescaling factor exceeds configured limit

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None

      Description

      While running coaddDriver with the PDR1 data using w_2018_15, at an edge patch 'tract': 9940, 'filter': 'HSC-I', 'patch': '3,8' of the WIDE layer, a RuntimeError was raised:

      coaddDriver.detectCoaddSources.scaleVariance WARN: Pixel-based variance rescaling factor (20.418020) exceeds configured limit (10.000000); trying image-based method
      coaddDriver WARN: lsst-verify-worker17:101609: Caught RuntimeError while detection on DataId(initialdata={'tract': 9940, 'filter': 'HSC-I', 'patch': '3,8'}, tag=set()): Variance rescaling factor (20.491061) exceeds configured limit (10.000000)
      coaddDriver INFO: lsst-verify-worker17:101609: Traceback:
      Traceback (most recent call last):
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/ctrl_pool/15.0-1-ga91101e+7/python/lsst/ctrl/pool/parallel.py", line 512, in logOperation
          yield
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_drivers/15.0-3-ga03b4ca+9/python/lsst/pipe/drivers/coaddDriver.py", line 321, in coadd
          detResults = self.detectCoaddSources.runDetection(coadd, idFactory, expId=expId)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/15.0-5-g389937dc+5/python/lsst/pipe/tasks/multiBand.py", line 301, in runDetection
          varScale = self.scaleVariance.run(exposure.maskedImage)
        File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/15.0-5-g389937dc+5/python/lsst/pipe/tasks/scaleVariance.py", line 123, in run
          (factor, self.config.limit))
      RuntimeError: Variance rescaling factor (20.491061) exceeds configured limit (10.000000)
      

      deepCoadd_calexp was not made for this patch, while both deepCoadd_directWarp and deepCoadd_psfMatchedWarp exist. This patch is at the edge of the coverage and only have few pixels at one corner (from makeCoaddTempExp logs: directWarp has 174578 good pixels (1.0%) psfMatchedWarp has 138454 good pixels (0.8%)).

        Attachments

          Issue Links

            Activity

            Hide
            yusra Yusra AlSayyad added a comment -

            Marking as Won't Fix. Aborting here was the right behavior when we don't have enough pixels. And the plumbing is sufficiently different in gen3 (quantum per visit/patch instead of per tract) that there's no benefit from changing the old plumbing.

            Show
            yusra Yusra AlSayyad added a comment - Marking as Won't Fix. Aborting here was the right behavior when we don't have enough pixels. And the plumbing is sufficiently different in gen3 (quantum per visit/patch instead of per tract) that there's no benefit from changing the old plumbing.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.