Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11691

Test failure with butler in obs_sdss

    Details

    • Templates:
    • Story Points:
      1
    • Sprint:
      DB_S17_8
    • Team:
      Data Access and Database

      Description

      In a Jenkins run last night the Mac failed with an error in obs_sdss:

      RuntimeError: The RepositoryArgs and RepositoryCfg must match for writable repositories, RepositoryCfg:RepositoryCfg(root='/Users/square/jenkins/workspace/stack-os-matrix/osx.py3/lsstsw/build/obs_sdss', mapper=<lsst.obs.sdss.sdssMapper.SdssMapper object at 0x141d71320>, mapperArgs={}, parents=[], policy=None), RepositoryArgs:RepositoryArgs(root='/Users/square/jenkins/workspace/stack-os-matrix/osx.py3/lsstsw/build/obs_sdss', cfgRoot=None, mapper=<lsst.obs.sdss.sdssMapper.SdssMapper object at 0x141d4d320>, mapperArgs=None, tags=set(), mode='rw', policy=None)
      Stacktrace
      self = <testGetId.GetIdTestCase testMethod=testId>
          def setUp(self):
              self.bf = dafPersist.ButlerFactory(mapper=SdssMapper(root="."))
      >       self.butler = self.bf.create()
      tests/testGetId.py:37: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      ../../stack/DarwinX86/daf_persistence/13.0-32-g7b14ddd/python/lsst/daf/persistence/butlerFactory.py:79: in create
          return Butler(root=root, mapper=self.mapper)
      ../../stack/DarwinX86/daf_persistence/13.0-32-g7b14ddd/python/lsst/daf/persistence/butler.py:521: in __init__
          self._getCfgs(repoDataList)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      self = <[AttributeError("'Butler' object has no attribute '_repos'") raised in repr()] Butler object at 0x141d4d6d8>
      repoDataList = [RepoData(id=5399557680,repoArgs=RepositoryArgs(root='/Users/square/jenkins/workspace/stack-os-matrix/osx.py3/lsstsw/b...fg=None,cfgOrigin=None,cfgRoot=None,repo=None,parentRepoDatas=[],isV1Repository=False,role=output,parentRegistry=None)]
          def _getCfgs(self, repoDataList):
              """Get or make a RepositoryCfg for each RepoData, and add the cfg to the RepoData.
                  If the cfg exists, compare values. If values match then use the cfg as an "existing" cfg. If the
                  values do not match, use the cfg as a "nested" cfg.
                  If the cfg does not exist, the RepositoryArgs must be for a writable repository.
          
                  Parameters
                  ----------
                  repoDataList : list of RepoData
                      The RepoData that are output and inputs of this Butler
          
                  Raises
                  ------
                  RuntimeError
                      If the passed-in RepositoryArgs indicate an existing repository but other cfg parameters in those
                      RepositoryArgs don't
                      match the existing repository's cfg a RuntimeError will be raised.
                  """
              def cfgMatchesArgs(args, cfg):
                  """Test if there are any values in an RepositoryArgs that conflict with the values in a cfg"""
                  if args.mapper is not None and cfg.mapper != args.mapper:
                      return False
                  if args.mapperArgs is not None and cfg.mapperArgs != args.mapperArgs:
                      return False
                  if args.policy is not None and cfg.policy != args.policy:
                      return False
                  return True
          
              for repoData in repoDataList:
                  cfg, isOldButlerRepository = self._getRepositoryCfg(repoData.repoArgs)
                  if cfg is None:
                      if 'w' not in repoData.repoArgs.mode:
                          raise RuntimeError(
                              "No cfg found for read-only input repository at {}".format(repoData.repoArgs.cfgRoot))
                      repoData.setCfg(cfg=RepositoryCfg.makeFromArgs(repoData.repoArgs),
                                      origin='new',
                                      root=repoData.repoArgs.cfgRoot,
                                      isV1Repository=isOldButlerRepository)
                  else:
                      if 'w' in repoData.repoArgs.mode:
                          # if it's an output repository, the RepositoryArgs must match the existing cfg.
                          if not cfgMatchesArgs(repoData.repoArgs, cfg):
                              raise RuntimeError(("The RepositoryArgs and RepositoryCfg must match for writable " +
                                                  "repositories, RepositoryCfg:{}, RepositoryArgs:{}").format(
      >                                               cfg, repoData.repoArgs))
      E                       RuntimeError: The RepositoryArgs and RepositoryCfg must match for writable repositories, RepositoryCfg:RepositoryCfg(root='/Users/square/jenkins/workspace/stack-os-matrix/osx.py3/lsstsw/build/obs_sdss', mapper=<lsst.obs.sdss.sdssMapper.SdssMapper object at 0x141d71320>, mapperArgs={}, parents=[], policy=None), RepositoryArgs:RepositoryArgs(root='/Users/square/jenkins/workspace/stack-os-matrix/osx.py3/lsstsw/build/obs_sdss', cfgRoot=None, mapper=<lsst.obs.sdss.sdssMapper.SdssMapper object at 0x141d4d320>, mapperArgs=None, tags=set(), mode='rw', policy=None)
      ../../stack/DarwinX86/daf_persistence/13.0-32-g7b14ddd/python/lsst/daf/persistence/butler.py:799: RuntimeError
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                npease Nate Pease
                Reporter:
                tjenness Tim Jenness
                Reviewers:
                Tim Jenness
                Watchers:
                Nate Pease, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel