Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13274

Deblender sometimes fails to model second object, or crashes

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_deblender
    • Labels:
      None
    • Templates:
    • Story Points:
      2
    • Sprint:
      DRP S18-2, DRP S18-3, DRP S18-4
    • Team:
      Data Release Production

      Description

      This issue was reported by Erin Sheldon's student. The problem was that some blends failed during initialization with a ValueError:

      /Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/operators.py:37: RuntimeWarning: invalid value encountered in true_divide
      /Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/source.py:149: RuntimeWarning: invalid value encountered in less
        morph[morph<0] = 0
      /Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/source.py:161: RuntimeWarning: invalid value encountered in greater
        ypix, xpix = np.where(morph>blend._bg_rms[band]/2)
      /Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/source.py:163: RuntimeWarning: invalid value encountered in greater
        ypix, xpix = np.where(morph>0)
      Traceback (most recent call last):
        File "galsim_deblend.py", line 194, in <module>
          model,mod2 = make_model(img,bg_rms,B,coords)
        File "galsim_deblend.py", line 80, in make_model
          blend = scarlet.Blend(sources, img, bg_rms=bg_rms)
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/blend.py", line 76, in __init__
          self.init_sources()
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/blend.py", line 255, in init_sources
          self.sources[m].init_source(self, self._img)
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/source.py", line 579, in init_source
          self.init_func(self, blend, img)
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/source.py", line 164, in init_templates
          Ny = np.max(ypix)-np.min(ypix)
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 2272, in amax
          out=out, **kwargs)
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/numpy/core/_methods.py", line 26, in _amax
          return umr_maximum(a, axis, None, out, keepdims)
      ValueError: zero-size array to reduction operation maximum which has no identity
      

      And other times failed during the fit with a LinAlgError:

      /Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/utils.py:340: RuntimeWarning: divide by zero encountered in double_scalars
      /Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/utils.py:340: RuntimeWarning: invalid value encountered in multiply
      /Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/utils.py:310: RuntimeWarning: divide by zero encountered in double_scalars
      /Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/utils.py:310: RuntimeWarning: invalid value encountered in multiply
      /Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/utils.py:358: RuntimeWarning: divide by zero encountered in double_scalars
      /Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/utils.py:360: RuntimeWarning: divide by zero encountered in double_scalars
      /Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/utils.py:360: RuntimeWarning: invalid value encountered in true_divide
      Traceback (most recent call last):
        File "galsim_deblend.py", line 194, in <module>
          model,mod2 = make_model(img,bg_rms,B,coords)
        File "galsim_deblend.py", line 92, in make_model
          blend.fit(200)#, e_rel=1e-1)
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/blend.py", line 229, in fit
          e_rel=self._e_rel, e_abs=self._e_abs, accelerated=accelerated, traceback=traceback)
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/algorithms.py", line 487, in bsdmm
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/blend.py", line 541, in _steps_f
          self._stepAS = [self._cbAS[block](block) for block in [0,1]]
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/blend.py", line 541, in <listcomp>
          self._stepAS = [self._cbAS[block](block) for block in [0,1]]
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/proxmin-0.4.3-py3.6.egg/proxmin/utils.py", line 222, in __call__
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/scarlet-0.1.d2e9b44-py3.6-macosx-10.7-x86_64.egg/scarlet/blend.py", line 486, in _one_over_lipschitz
          LA = np.real(np.linalg.eigvals(SSigma_1S).max())
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 889, in eigvals
          _assertFinite(a)
        File "/Users/lorena/anaconda3/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 217, in _assertFinite
          raise LinAlgError("Array must not contain infs or NaNs")
      numpy.linalg.linalg.LinAlgError: Array must not contain infs or NaNs

       

      The root cause of both issues is that the new deblender takes coordinates as (y,x) while the user was passing (x,y) (see the discussion on github for more).

       

      The first error was thrown because the incorrect position given had no flux above the noise level at the peak, so the source couldn't be initialized. We should implement a check for this to warn the user and have some fallback initialization.

       

      The second error is caused by a problem with fitting positions for sources that have no flux, which Peter Melchior has fixed in a soon to be merged branch.

       

      I'll make the change to the initialization error and close this ticket once Peter Melchior's branch has been merged.

        Attachments

          Activity

            People

            • Assignee:
              fred3m Fred Moolekamp
              Reporter:
              fred3m Fred Moolekamp
              Reviewers:
              Peter Melchior
              Watchers:
              Fred Moolekamp, Peter Melchior
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel