Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11614

Jointcal does not work with pytest

    Details

    • Templates:
    • Team:
      Architecture

      Description

      Jointcal fails the test_jointcal_decam if test_jointcal_cfht or test_jointcal_cfht_minimal are run first.

      ========================================================================================================== test session starts ==========================================================================================================
      platform darwin -- Python 3.6.0, pytest-3.2.0, py-1.4.34, pluggy-0.4.0
      rootdir: /Volumes/G-RAID with Thunderbolt/transient/lsstsw3/build/jointcal, inifile:
      plugins: session2file-0.1.9, forked-0.3.dev0+g1dd93f6.d20170815, xdist-1.19.2.dev0+g459d52e.d20170815, flake8-0.8.1
      collected 9 items 
       
      tests/test_jointcal_cfht.py .....
      tests/test_jointcal_decam.py FF..
       
      =============================================================================================================== FAILURES ================================================================================================================
      _____________________________________________________________________________________________ JointcalTestDECAM.test_jointcalTask_2_visits ______________________________________________________________________________________________
       
      self = <test_jointcal_decam.JointcalTestDECAM testMethod=test_jointcalTask_2_visits>
       
          def test_jointcalTask_2_visits(self):
              # NOTE: The relative RMS limit was empirically determined from the
              # first run of jointcal on this data. We should always do better than
              # this in the future!
              relative_error = 32e-3*u.arcsecond
              pa1 = 0.14
              # NOTE: decam fits are currently not converging; the chi2 jumps around, so skip that Metric.
              metrics = {'collectedAstrometryRefStars': 8194,
                         'collectedPhotometryRefStars': 8194,
                         'selectedAstrometryRefStars': 8194,
                         'selectedPhotometryRefStars': 8194,
                         'associatedAstrometryFittedStars': 8241,
                         'associatedPhotometryFittedStars': 8241,
                         'selectedAstrometryFittedStars': 2261,
                         'selectedPhotometryFittedStars': 2261,
                         'selectedAstrometryCcdImageList': 17,
                         'selectedPhotometryCcdImageList': 17,
                         'astrometryFinalChi2': None,
                         'astrometryFinalNdof': 4306,
                         'photometryFinalChi2': None,
                         'photometryFinalNdof': 2391,
                         }
          
      >       self._testJointcalTask(2, relative_error, self.dist_rms_absolute, pa1, metrics=metrics)
       
      tests/test_jointcal_decam.py:80: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
      tests/jointcalTestBase.py:134: in _testJointcalTask
          result = self._runJointcalTask(nCatalogs, caller, metrics=metrics)
      tests/jointcalTestBase.py:180: in _runJointcalTask
          result = jointcal.JointcalTask.parseAndRun(args=args, doReturnResults=True, config=self.config)
      /Users/timj/work/lsstsw3/stack/DarwinX86/pipe_base/13.0-12-gaf0c0ec+2/python/lsst/pipe/base/cmdLineTask.py:526: in parseAndRun
          parsedCmd = argumentParser.parse_args(config=config, args=args, log=log, override=cls.applyOverrides)
      /Users/timj/work/lsstsw3/stack/DarwinX86/pipe_base/13.0-12-gaf0c0ec+2/python/lsst/pipe/base/argumentParser.py:514: in parse_args
          namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs)
      /Users/timj/work/lsstsw3/stack/DarwinX86/daf_persistence/13.0-31-g48013df/python/lsst/daf/persistence/butler.py:543: in __init__
          repoData.repo = Repository(repoData)
      /Users/timj/work/lsstsw3/stack/DarwinX86/daf_persistence/13.0-31-g48013df/python/lsst/daf/persistence/repository.py:143: in __init__
          self._initMapper(repoData)
      /Users/timj/work/lsstsw3/stack/DarwinX86/daf_persistence/13.0-31-g48013df/python/lsst/daf/persistence/repository.py:172: in _initMapper
          **mapperArgs)
      /Users/timj/work/lsstsw3/stack/DarwinX86/obs_decam/13.0-20-g5323ca4/python/lsst/obs/decam/decamMapper.py:60: in __init__
          afwImageUtils.defineFilter('u', lambdaEff=350, alias=['u DECam c0006 3500.0 1000.0'])
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
       
      name = 'u', lambdaEff = 350, alias = ['u DECam c0006 3500.0 1000.0'], force = False
       
          def defineFilter(name, lambdaEff, alias=[], force=False):
              """Define a filter and its properties in the filter registry"""
      >       prop = FilterProperty(name, lambdaEff, force)
      E       lsst.pex.exceptions.wrappers.RuntimeError: 
      E         File "src/image/Filter.cc", line 74, in void lsst::afw::image::FilterProperty::_insert(bool)
      E           Filter u is already defined {0}
      E       lsst::pex::exceptions::RuntimeError: 'Filter u is already defined'
       
      /Users/timj/work/lsstsw3/stack/DarwinX86/afw/13.0-94-ga1c444047/python/lsst/afw/image/utils.py:110: RuntimeError
      --------------------------------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------------------------------
      CameraMapper INFO: Loading exposure registry from /Users/timj/work/lsstsw3/stack/DarwinX86/testdata_jointcal/master-g9b8753233a+1/decam/registry.sqlite3
      CameraMapper INFO: Loading Posix calib registry from /Users/timj/work/lsstsw3/stack/DarwinX86/testdata_jointcal/master-g9b8753233a+1/decam
      _____________________________________________________________________________________ JointcalTestDECAM.test_jointcalTask_2_visits_constrainedPoly ______________________________________________________________________________________
       
      self = <test_jointcal_decam.JointcalTestDECAM testMethod=test_jointcalTask_2_visits_constrainedPoly>
       
          def test_jointcalTask_2_visits_constrainedPoly(self):
              self.config = lsst.jointcal.jointcal.JointcalConfig()
              self.config.astrometryModel = "constrainedPoly"
              self.config.doPhotometry = False
              self.jointcalStatistics.do_photometry = False
          
              # NOTE: The relative RMS limit was empirically determined from the
              # first run of jointcal on this data. We should always do better than
              # this in the future!
              relative_error = 20e-3*u.arcsecond
              pa1 = None
              metrics = {'collectedAstrometryRefStars': 8194,
                         'selectedAstrometryRefStars': 8194,
                         'associatedAstrometryFittedStars': 8241,
                         'selectedAstrometryFittedStars': 2261,
                         'selectedAstrometryCcdImageList': 17,
                         'astrometryFinalChi2': 5106.2,
                         'astrometryFinalNdof': 4530,
                         }
          
      >       self._testJointcalTask(2, relative_error, self.dist_rms_absolute, pa1, metrics=metrics)
       
      tests/test_jointcal_decam.py:102: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
      tests/jointcalTestBase.py:134: in _testJointcalTask
          result = self._runJointcalTask(nCatalogs, caller, metrics=metrics)
      tests/jointcalTestBase.py:180: in _runJointcalTask
          result = jointcal.JointcalTask.parseAndRun(args=args, doReturnResults=True, config=self.config)
      /Users/timj/work/lsstsw3/stack/DarwinX86/pipe_base/13.0-12-gaf0c0ec+2/python/lsst/pipe/base/cmdLineTask.py:526: in parseAndRun
          parsedCmd = argumentParser.parse_args(config=config, args=args, log=log, override=cls.applyOverrides)
      /Users/timj/work/lsstsw3/stack/DarwinX86/pipe_base/13.0-12-gaf0c0ec+2/python/lsst/pipe/base/argumentParser.py:514: in parse_args
          namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs)
      /Users/timj/work/lsstsw3/stack/DarwinX86/daf_persistence/13.0-31-g48013df/python/lsst/daf/persistence/butler.py:543: in __init__
          repoData.repo = Repository(repoData)
      /Users/timj/work/lsstsw3/stack/DarwinX86/daf_persistence/13.0-31-g48013df/python/lsst/daf/persistence/repository.py:143: in __init__
          self._initMapper(repoData)
      /Users/timj/work/lsstsw3/stack/DarwinX86/daf_persistence/13.0-31-g48013df/python/lsst/daf/persistence/repository.py:172: in _initMapper
          **mapperArgs)
      /Users/timj/work/lsstsw3/stack/DarwinX86/obs_decam/13.0-20-g5323ca4/python/lsst/obs/decam/decamMapper.py:60: in __init__
          afwImageUtils.defineFilter('u', lambdaEff=350, alias=['u DECam c0006 3500.0 1000.0'])
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
       
      name = 'u', lambdaEff = 350, alias = ['u DECam c0006 3500.0 1000.0'], force = False
       
          def defineFilter(name, lambdaEff, alias=[], force=False):
              """Define a filter and its properties in the filter registry"""
      >       prop = FilterProperty(name, lambdaEff, force)
      E       lsst.pex.exceptions.wrappers.RuntimeError: 
      E         File "src/image/Filter.cc", line 74, in void lsst::afw::image::FilterProperty::_insert(bool)
      E           Filter u is already defined {0}
      E       lsst::pex::exceptions::RuntimeError: 'Filter u is already defined'
       
      /Users/timj/work/lsstsw3/stack/DarwinX86/afw/13.0-94-ga1c444047/python/lsst/afw/image/utils.py:110: RuntimeError
      --------------------------------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------------------------------
      CameraMapper INFO: Loading exposure registry from /Users/timj/work/lsstsw3/stack/DarwinX86/testdata_jointcal/master-g9b8753233a+1/decam/registry.sqlite3
      CameraMapper INFO: Loading Posix calib registry from /Users/timj/work/lsstsw3/stack/DarwinX86/testdata_jointcal/master-g9b8753233a+1/decam
      =========================================================================================================== warnings summary ============================================================================================================
      tests/test_jointcal_cfht.py::JointcalTestCFHT::test_jointcalTask_2_visits
        /Users/timj/work/lsstsw3/stack/DarwinX86/meas_algorithms/13.0-18-gc4ad4228+4/python/lsst/meas/algorithms/astrometrySourceSelector.py:137: RuntimeWarning: invalid value encountered in greater
          return sourceCat.get(self.fluxKey)/sourceCat.get(self.fluxSigmaKey) > self.config.minSnr
       
      tests/test_jointcal_cfht.py::JointcalTestCFHT::test_jointcalTask_2_visits_constrainedPhotometry_no_astrometry
        /Users/timj/work/lsstsw3/stack/DarwinX86/meas_algorithms/13.0-18-gc4ad4228+4/python/lsst/meas/algorithms/astrometrySourceSelector.py:137: RuntimeWarning: invalid value encountered in greater
          return sourceCat.get(self.fluxKey)/sourceCat.get(self.fluxSigmaKey) > self.config.minSnr
       
      tests/test_jointcal_cfht.py::JointcalTestCFHT::test_jointcalTask_2_visits_constrainedPoly
        /Users/timj/work/lsstsw3/stack/DarwinX86/meas_algorithms/13.0-18-gc4ad4228+4/python/lsst/meas/algorithms/astrometrySourceSelector.py:137: RuntimeWarning: invalid value encountered in greater
          return sourceCat.get(self.fluxKey)/sourceCat.get(self.fluxSigmaKey) > self.config.minSnr
       
      -- Docs: http://doc.pytest.org/en/latest/warnings.html
      =========================================================================================== 2 failed, 7 passed, 3 warnings in 126.41 seconds ============================================================================================
      

      The u filter is being defined but I assume the u filter was set up for CFHT. None of the other tests care.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tjenness Tim Jenness
                Reporter:
                tjenness Tim Jenness
                Reviewers:
                John Parejko
                Watchers:
                Colin Slater, John Parejko, Simon Krughoff, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel