Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18601

Investigate newer SQLAlchemy versions

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Third Party Software
    • Labels:
      None
    • Templates:
    • Story Points:
      2
    • Sprint:
      Arch 2019-04-01, Arch 2019-04-08
    • Team:
      Architecture

      Description

      With the merge of DM-18367 we are getting many DeprecationWarnings from sqlalchemy when running the test in daf_butler (about 85 of them). This seems to be down to some new code paths being used from within sqlalchemy.

      We are currently running 1.2.16. On this ticket I will see if 1.2.18 gets rid of the warnings and possibly try the new 1.3 series.

      The current warning comes during schema creation:

        File "/Users/timj/work/lsstsw3/build/daf_butler/python/lsst/daf/butler/core/registry.py", line 177, in fromConfig
          return cls(registryConfig, schemaConfig, dimensionConfig, create=create)
        File "/Users/timj/work/lsstsw3/build/daf_butler/python/lsst/daf/butler/registries/sqliteRegistry.py", line 94, in __init__
          super().__init__(registryConfig, schemaConfig, dimensionConfig, create)
        File "/Users/timj/work/lsstsw3/build/daf_butler/python/lsst/daf/butler/registries/sqlRegistry.py", line 82, in __init__
          self._createTables(self._schema, self._connection)
        File "/Users/timj/work/lsstsw3/build/daf_butler/python/lsst/daf/butler/registries/sqlRegistry.py", line 154, in _createTables
          schema.metadata.create_all(connection)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/schema.py", line 4201, in create_all
          ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/engine/base.py", line 1593, in _run_visitor
          visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/visitors.py", line 130, in traverse_single
          return meth(obj, **kw)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
          [t for t in tables if self._can_create_table(t)]
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/ddl.py", line 1174, in sort_tables_and_constraints
          deterministic_order=True,
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/util/topological.py", line 50, in sort
          for set_ in sort_as_subsets(tuples, allitems, deterministic_order):
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/util/topological.py", line 36, in sort_as_subsets
          _gen_edges(edges),
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/exc.py", line 139, in __init__
          message += " (%s)" % ", ".join(repr(s) for s in cycles)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/exc.py", line 139, in <genexpr>
          message += " (%s)" % ", ".join(repr(s) for s in cycles)
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/schema.py", line 692, in __repr__
          + ["%s=%s" % (k, repr(getattr(self, k))) for k in ["schema"]]
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/schema.py", line 691, in <listcomp>
          + [repr(x) for x in self.columns]
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/schema.py", line 1400, in __repr__
          + ["%s=%s" % (k, repr(getattr(self, k))) for k in kwarg]
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/schema.py", line 1400, in <listcomp>
          + ["%s=%s" % (k, repr(getattr(self, k))) for k in kwarg]
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/sql/schema.py", line 112, in __repr__
          return util.generic_repr(self, omit_kwarg=["info"])
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/util/langhelpers.py", line 556, in generic_repr
          val = getattr(obj, arg, missing)
        File "<string>", line 2, in quote
        File "/Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/util/deprecations.py", line 118, in warned
          warnings.warn(message, wtype, stacklevel=3)
        File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.7/warnings.py", line 110, in _showwarnmsg
          msg.file, msg.line)
        File "xxx.py", line 12, in warn_with_traceback
          traceback.print_stack(file=log)
      /Users/timj/work/lsstsw3/stack/DarwinX86/sqlalchemy/1.2.16/lib/python/SQLAlchemy-1.2.16-py3.7-macosx-10.7-x86_64.egg/sqlalchemy/util/langhelpers.py:556: SADeprecationWarning: Use ``<obj>.name.quote``
        val = getattr(obj, arg, missing)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tjenness Tim Jenness
                Reporter:
                tjenness Tim Jenness
                Reviewers:
                Michelle Gower
                Watchers:
                Andy Salnikov, Hsin-Fang Chiang, Michelle Gower, Nate Lust, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel