Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12117

repositoryCfg.yaml input root not backwards compatible

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: butler
    • Labels:
      None

      Description

      The following butler command fails on a dataset on lsst-dev that was written with w_2017_17, when using a more recent stack (in this case, w_2017_39). It also fails if root=PATH is used instead of just passing the path as the first arg. See discussion beginning here on slack: https://lsstc.slack.com/archives/C3UCAEW3D/p1507226925000049

      lsst.daf.persistence.Butler("/datasets/hsc/repo/rerun/DM-10404/SFM/")
      

      On the other hand, if the input is specified as "inputs={'root':PATH}" it succeeds. Either RepositoryCfg_v1 should have its version number bumped, or the newer Butler should transparently read the older .yaml.

      Full stack trace of the failure:

      In [7]: lsst.daf.persistence.Butler(root="/datasets/hsc/repo/rerun/DM-10404/SFM/
         ...: ")
      ---------------------------------------------------------------------------
      AttributeError                            Traceback (most recent call last)
      <ipython-input-7-04365c25b492> in <module>()
      ----> 1 lsst.daf.persistence.Butler(root="/datasets/hsc/repo/rerun/DM-10404/SFM/")
       
      /software/lsstsw/stack/Linux64/daf_persistence/13.0-40-g460649b+5/python/lsst/daf/persistence/butler.pyc in __init__(self, root, mapper, inputs, outputs, **mapperArgs)
          523         self._addParents(repoDataList)
          524
      --> 525         self._setAndVerifyParentsLists(repoDataList)
          526
          527         self._setDefaultMapper(repoDataList)
       
      /software/lsstsw/stack/Linux64/daf_persistence/13.0-40-g460649b+5/python/lsst/daf/persistence/butler.pyc in _setAndVerifyParentsLists(self, repoDataList)
          900                 repoData.cfg.addParents(parents)
          901             elif repoData.cfgOrigin in ('existing', 'nested'):
      --> 902                 if repoData.cfg.parents != parents:
          903                     try:
          904                         repoData.cfg.extendParents(parents)
       
      /software/lsstsw/stack/Linux64/daf_persistence/13.0-40-g460649b+5/python/lsst/daf/persistence/repositoryCfg.pyc in __eq__(self, other)
           95         if not other:
           96             return False
      ---> 97         return self.root == other.root and \
           98             self.mapper == other.mapper and \
           99             self.mapperArgs == other.mapperArgs and \
       
      AttributeError: 'str' object has no attribute 'root'
      

        Attachments

          Issue Links

            Activity

            Hide
            npease Nate Pease added a comment -

            John, can you please code review. Ignore the obs_subaru branch, it won't be checked in (but can't be deleted), only daf_persistence needs to be reviewed. Thx.

            Show
            npease Nate Pease added a comment - John, can you please code review. Ignore the obs_subaru branch, it won't be checked in (but can't be deleted), only daf_persistence needs to be reviewed. Thx.
            Hide
            npease Nate Pease added a comment -

            John Parejko changes are uploaded. I'll wait for you to change the ticket to "reviewed" before merging.

            Show
            npease Nate Pease added a comment - John Parejko changes are uploaded. I'll wait for you to change the ticket to "reviewed" before merging.
            Hide
            npease Nate Pease added a comment -

            John Parejko, are you done reviewing?

            Show
            npease Nate Pease added a comment - John Parejko , are you done reviewing?
            Hide
            Parejkoj John Parejko added a comment -

            Thanks. This looks better. Merge away!

            Show
            Parejkoj John Parejko added a comment - Thanks. This looks better. Merge away!
            Hide
            npease Nate Pease added a comment -

            changes to obs_hsc that were on that package's branch tickets/DM-12117 were done by Paul Price in a different ticket.

            Show
            npease Nate Pease added a comment - changes to obs_hsc that were on that package's branch tickets/ DM-12117 were done by Paul Price in a different ticket.

              People

              • Assignee:
                npease Nate Pease
                Reporter:
                Parejkoj John Parejko
                Reviewers:
                John Parejko
                Watchers:
                Eli Rykoff, Fritz Mueller, Hsin-Fang Chiang, John Parejko, John Swinbank, Joshua Meyers, Kian-Tat Lim, Nate Pease, Paul Price, Robert Lupton
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel