Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16289

validate_drp seems to fail on Python 3.7

    XMLWordPrintable

Details

    • Story
    • Status: Invalid
    • Resolution: Done
    • None
    • None
    • None
    • SQuaRE

    Description

      I have tried out the fix from DM-15462 and whilst it does work on my numpy1.15 system, I get now a new failure on python 3.7 when running lsst_ci:

      tests/test_obs.py:50: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      ../../stack/DarwinX86/utils/16.0-13-g80874fd+2/python/lsst/utils/tests.py:297: in assertExecutable
          self.fail(msg)
      E   AssertionError: DECam Quick Test failed
      ----------------------------- Captured stdout call -----------------------------
      Running executable 'scripts/runDecamQuickTest.sh' with arguments "-- --noplot"...
      Ingesting Raw data
      root INFO: Loading config overrride file '/Users/timj/work/lsstsw3/stack/DarwinX86/obs_decam/16.0-11-g0300050/config/ingest.py'
      CameraMapper INFO: Loading Posix exposure registry from /Users/timj/work/lsstsw3/build/lsst_ci/DecamQuick/input
      ingest INFO: /Users/timj/work/lsstsw3/stack/DarwinX86/validation_data_decam/16.0-1-gd5cb667+2/instcal/instcal0176837.fits.fz --<link>--> /Users/timj/work/lsstsw3/build/lsst_ci/DecamQuick/input/0176837/instcal0176837.fits.fz
      ingest INFO: /Users/timj/work/lsstsw3/stack/DarwinX86/validation_data_decam/16.0-1-gd5cb667+2/dqmask/dqmask0176837.fits.fz --<link>--> /Users/timj/work/lsstsw3/build/lsst_ci/DecamQuick/input/0176837/dqmask0176837.fits.fz
      ingest INFO: /Users/timj/work/lsstsw3/stack/DarwinX86/validation_data_decam/16.0-1-gd5cb667+2/wtmap/wtmap0176837.fits.fz --<link>--> /Users/timj/work/lsstsw3/build/lsst_ci/DecamQuick/input/0176837/wtmap0176837.fits.fz
      ingest INFO: /Users/timj/work/lsstsw3/stack/DarwinX86/validation_data_decam/16.0-1-gd5cb667+2/instcal/instcal0176846.fits.fz --<link>--> /Users/timj/work/lsstsw3/build/lsst_ci/DecamQuick/input/0176846/instcal0176846.fits.fz
      ingest INFO: /Users/timj/work/lsstsw3/stack/DarwinX86/validation_data_decam/16.0-1-gd5cb667+2/dqmask/dqmask0176846.fits.fz --<link>--> /Users/timj/work/lsstsw3/build/lsst_ci/DecamQuick/input/0176846/dqmask0176846.fits.fz
      ingest INFO: /Users/timj/work/lsstsw3/stack/DarwinX86/validation_data_decam/16.0-1-gd5cb667+2/wtmap/wtmap0176846.fits.fz --<link>--> /Users/timj/work/lsstsw3/build/lsst_ci/DecamQuick/input/0176846/wtmap0176846.fits.fz
      running singleFrameDriver
      validating
      CameraMapper INFO: Loading exposure registry from /Users/timj/work/lsstsw3/build/lsst_ci/DecamQuick/input/registry.sqlite3
      No locations for get: datasetType:calexp_calib dataId:DataId(initialdata={'filter': 'z', 'visit': 176837, 'ccdnum': 10}, tag=set())
      Could not open calibrated image file for  {'filter': 'z', 'visit': 176837, 'ccdnum': 10}
      Skipping this dataId.
      No locations for get: datasetType:calexp_calib dataId:DataId(initialdata={'filter': 'z', 'visit': 176846, 'ccdnum': 10}, tag=set())
      Could not open calibrated image file for  {'filter': 'z', 'visit': 176846, 'ccdnum': 10}
      Skipping this dataId.
       
      ----------------------------- Captured stderr call -----------------------------
      Traceback (most recent call last):
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/validate_drp/16.0-12-g7d69e54+1/bin/validateDrp.py", line 104, in <module>
          validate.run(args.repo, **kwargs)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/validate_drp/16.0-12-g7d69e54+1/python/lsst/validate/drp/validate.py", line 152, in run
          metrics_package=metrics_package, **kwargs)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/validate_drp/16.0-12-g7d69e54+1/python/lsst/validate/drp/validate.py", line 243, in runOneRepo
          metrics_package=metrics_package, **kwargs)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/validate_drp/16.0-12-g7d69e54+1/python/lsst/validate/drp/validate.py", line 301, in runOneFilter
          skipTEx=skipTEx)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/validate_drp/16.0-12-g7d69e54+1/python/lsst/validate/drp/matchreduce.py", line 126, in build_matched_dataset
          useJointCal=useJointCal, skipTEx=skipTEx)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/validate_drp/16.0-12-g7d69e54+1/python/lsst/validate/drp/matchreduce.py", line 293, in _loadAndMatchCatalogs
          matchCat = mmatch.finish()
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/afw/16.0-55-ged26810fc/python/lsst/afw/table/multiMatch.py", line 123, in finish
          for record in self.result:
      TypeError: 'NoneType' object is not iterable
       generated xml file: /Users/timj/work/lsstsw3/build/lsst_ci/tests/.tests/pytest-lsst_ci.xml 
      

      The DECam and CFHT tests fail in the same way.

      Attachments

        Issue Links

          Activity

            tjenness Tim Jenness created issue -
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Link This issue relates to DM-15462 [ DM-15462 ]
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status To Do [ 10001 ] Invalid [ 11005 ]

            People

              Unassigned Unassigned
              tjenness Tim Jenness
              Frossie Economou, Jeffrey Carlin, Simon Krughoff (Inactive), Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.