Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8832

Butler mapper issue in validate_drp since Jenkins build 663

    XMLWordPrintable

    Details

      Description

      Since build 663 on Jenkins we're seeing a new Butler-related issue in validate_drp even with the CFHT dataset. Could there be a butler change that is triggering this issue? This would have happened January 3-4 2017.

      [py2] $ /bin/bash -e /tmp/hudson8266472982640135803.sh
      notice: lsstsw tools have been set up.
      Ingesting Raw data
      root INFO: Loading config overrride file '/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/obs_cfht/12.1-14-gddeedbd+20/config/ingest.py'
      CameraMapper INFO: Unable to locate registry registry in root: /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input/registry.sqlite3
      CameraMapper INFO: Unable to locate registry registry in current dir: ./registry.sqlite3
      CameraMapper INFO: Loading Posix registry from /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input
      CameraMapper INFO: Unable to locate calibRegistry registry in root: /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input/calibRegistry.sqlite3
      CameraMapper INFO: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
      CameraMapper INFO: Loading Posix registry from /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input
      /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validation_data_cfht/master-g2016f8e221+2/raw/849375p.fits.fz --<link>--> /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input/raw/06AL01/D3/2006-05-20/r/849375p.fits.fz
      /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validation_data_cfht/master-g2016f8e221+2/raw/850587p.fits.fz --<link>--> /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input/raw/06AL01/D3/2006-06-02/r/850587p.fits.fz
      running processCcd
      validating
      Traceback (most recent call last):
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/bin/validateDrp.py", line 95, in <module>
          validate.run(args.repo, **kwargs)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/python/lsst/validate/drp/validate.py", line 104, in run
          **kwargs)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/python/lsst/validate/drp/validate.py", line 204, in runOneFilter
          verbose=verbose)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/python/lsst/validate/drp/matchreduce.py", line 147, in __init__
          repo, dataIds, matchRadius)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/python/lsst/validate/drp/matchreduce.py", line 175, in _loadAndMatchCatalogs
          butler = dafPersist.Butler(repo)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/daf_persistence/12.1-17-g9654cba+1/python/lsst/daf/persistence/butler.py", line 304, in __init__
          self._addRepo(args, inout='out', defaultMapper=defaultMapper, butlerIOParents=butlerIOParents)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/daf_persistence/12.1-17-g9654cba+1/python/lsst/daf/persistence/butler.py", line 399, in _addRepo
          "Could not infer mapper and one not specified in repositoryArgs:%s" % args)
      RuntimeError: Could not infer mapper and one not specified in repositoryArgs:RepositoryArgs(root='Cfht/output', cfgRoot=None, mapper=None, mapperArgs={}, tags=set([]), mode='rw', policy=None)
      Validation failed
      Build step 'Execute shell' marked build as failure
      [PostBuildScript] - Execution post build scripts.
      [py2] $ /bin/sh -xe /tmp/hudson1544818061461777296.sh
      ++ lsof -d 200 -t
      + Z=
      + [[ ! -z '' ]]
      + rm -rf /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/.lockDir
      Archiving artifacts
      [INFO] HipChat notification sent to the following rooms: Bot: Jenkins
      Finished: FAILURE
      

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment -

            Yes, I'm sure it is.

            Show
            swinbank John Swinbank added a comment - Yes, I'm sure it is.
            Hide
            swinbank John Swinbank added a comment -

            Hey Simon Krughoff, sorry to bother you but I wondered if there was an ETA on this review? If you'd rather I pass it off to somebody else, let me know.

            Show
            swinbank John Swinbank added a comment - Hey Simon Krughoff , sorry to bother you but I wondered if there was an ETA on this review? If you'd rather I pass it off to somebody else, let me know.
            Hide
            krughoff Simon Krughoff added a comment -

            Looks perfect. Merge away.

            Show
            krughoff Simon Krughoff added a comment - Looks perfect. Merge away.
            Hide
            swinbank John Swinbank added a comment -

            Merged. Thanks Simon!

            Show
            swinbank John Swinbank added a comment - Merged. Thanks Simon!
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            The jenkins validate_drp job is passing again!

            Show
            jhoblitt Joshua Hoblitt added a comment - The jenkins validate_drp job is passing again!

              People

              Assignee:
              swinbank John Swinbank
              Reporter:
              jsick Jonathan Sick
              Reviewers:
              Simon Krughoff
              Watchers:
              Angelo Fausti, Jim Bosch, John Swinbank, Jonathan Sick, Joshua Hoblitt, Michael Wood-Vasey, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.