Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8832

Butler mapper issue in validate_drp since Jenkins build 663

    Details

    • Templates:
    • Story Points:
      1
    • Sprint:
      DRP S17-2
    • Team:
      Data Release Production

      Description

      Since build 663 on Jenkins we're seeing a new Butler-related issue in validate_drp even with the CFHT dataset. Could there be a butler change that is triggering this issue? This would have happened January 3-4 2017.

      [py2] $ /bin/bash -e /tmp/hudson8266472982640135803.sh
      notice: lsstsw tools have been set up.
      Ingesting Raw data
      root INFO: Loading config overrride file '/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/obs_cfht/12.1-14-gddeedbd+20/config/ingest.py'
      CameraMapper INFO: Unable to locate registry registry in root: /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input/registry.sqlite3
      CameraMapper INFO: Unable to locate registry registry in current dir: ./registry.sqlite3
      CameraMapper INFO: Loading Posix registry from /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input
      CameraMapper INFO: Unable to locate calibRegistry registry in root: /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input/calibRegistry.sqlite3
      CameraMapper INFO: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
      CameraMapper INFO: Loading Posix registry from /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input
      /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validation_data_cfht/master-g2016f8e221+2/raw/849375p.fits.fz --<link>--> /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input/raw/06AL01/D3/2006-05-20/r/849375p.fits.fz
      /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validation_data_cfht/master-g2016f8e221+2/raw/850587p.fits.fz --<link>--> /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/validate_drp/Cfht/input/raw/06AL01/D3/2006-06-02/r/850587p.fits.fz
      running processCcd
      validating
      Traceback (most recent call last):
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/bin/validateDrp.py", line 95, in <module>
          validate.run(args.repo, **kwargs)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/python/lsst/validate/drp/validate.py", line 104, in run
          **kwargs)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/python/lsst/validate/drp/validate.py", line 204, in runOneFilter
          verbose=verbose)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/python/lsst/validate/drp/matchreduce.py", line 147, in __init__
          repo, dataIds, matchRadius)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/validate_drp/master-gf3b529ce90+2/python/lsst/validate/drp/matchreduce.py", line 175, in _loadAndMatchCatalogs
          butler = dafPersist.Butler(repo)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/daf_persistence/12.1-17-g9654cba+1/python/lsst/daf/persistence/butler.py", line 304, in __init__
          self._addRepo(args, inout='out', defaultMapper=defaultMapper, butlerIOParents=butlerIOParents)
        File "/home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/Linux64/daf_persistence/12.1-17-g9654cba+1/python/lsst/daf/persistence/butler.py", line 399, in _addRepo
          "Could not infer mapper and one not specified in repositoryArgs:%s" % args)
      RuntimeError: Could not infer mapper and one not specified in repositoryArgs:RepositoryArgs(root='Cfht/output', cfgRoot=None, mapper=None, mapperArgs={}, tags=set([]), mode='rw', policy=None)
      Validation failed
      Build step 'Execute shell' marked build as failure
      [PostBuildScript] - Execution post build scripts.
      [py2] $ /bin/sh -xe /tmp/hudson1544818061461777296.sh
      ++ lsof -d 200 -t
      + Z=
      + [[ ! -z '' ]]
      + rm -rf /home/jenkins-slave/workspace/validate_drp/dataset/cfht/label/centos-7/python/py2/lsstsw/stack/.lockDir
      Archiving artifacts
      [INFO] HipChat notification sent to the following rooms: Bot: Jenkins
      Finished: FAILURE
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                swinbank John Swinbank
                Reporter:
                jsick Jonathan Sick
                Reviewers:
                Simon Krughoff
                Watchers:
                Angelo Fausti, Jim Bosch, John Swinbank, Jonathan Sick, Joshua Hoblitt, Michael Wood-Vasey, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel