Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33085

Fix cache key warning in butler sqlalchemy usage

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DB_S22_12
    • Team:
      Data Access and Database
    • Urgent?:
      No

      Description

      Quentin Le Boulc'h reports the following new warning when using sqlalchemy v1.4.28:

      $ butler import /sps/lsst/users/leboulch/Gen3-DC2 /sps/lsst/datasets/rubin/previews/dp0.2/calib --export-file /sps/lsst/datasets/rubin/previews/dp0.2/calib/export.yaml
      		 
      py.warnings WARNING: /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v23.0.0/stack/miniconda3-py38_4.9.2-0.7.0/Linux64/daf_butler/23.0.0+d18c45d440/python/lsst/daf/butler/registry/interfaces/_database.py:1640: SAWarning: TypeDecorator Base64Region() will not produce a cache key because the ``cache_ok`` attribute is not set to True.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)

        Attachments

          Issue Links

          is duplicated by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33061 Ensure daf_butler SQLAlchemy usage enables caching

          • Invalid

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            See also DM-33061.

            Show
            tjenness Tim Jenness added a comment - See also DM-33061 .
            Hide
            Permalink
            salnikov Andy Salnikov added a comment -

            There are few similar warnings that I see in unit tests:

              .../daf_butler/python/lsst/daf/butler/registry/databases/sqlite.py:465: SAWarning: 
                Class _Replace will not make use of SQL compilation caching as it does not set the 'inherit_cache' attribute to ``True``. 
                This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. 
                Set this attribute to True if this object can make use of the cache key generated by the superclass.  Alternatively, this attribute may be set to False which will disable this warning.
            		 
             
            		 
              .../daf_butler/python/lsst/daf/butler/registry/interfaces/_database.py:1706: SAWarning:
            		 
                Class _TimestampColumnElement will not make use of SQL compilation caching as it does not set the 'inherit_cache' attribute to ``True``.  
                This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  
                Set this attribute to True if this object can make use of the cache key generated by the superclass.  Alternatively, this attribute may be set to False which will disable this warning.
            		 
             
            		 
              .../daf_butler/python/lsst/daf/butler/registry/databases/sqlite.py:476: SAWarning:
            		 
                Class _Ensure will not make use of SQL compilation caching as it does not set the 'inherit_cache' attribute to ``True``.  
                This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  
                Set this attribute to True if this object can make use of the cache key generated by the superclass.  Alternatively, this attribute may be set to False which will disable this warning.
            		 
             
            		 
              .../daf_butler/python/lsst/daf/butler/registry/interfaces/_database.py:1706: SAWarning: 
                TypeDecorator Base64Region() will not produce a cache key because the ``cache_ok`` attribute is not set to True.  
                This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  
                Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning.

            Show
            salnikov Andy Salnikov added a comment - There are few similar warnings that I see in unit tests: .../daf_butler/python/lsst/daf/butler/registry/databases/sqlite.py:465: SAWarning: Class _Replace will not make use of SQL compilation caching as it does not set the 'inherit_cache' attribute to ``True``. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this object can make use of the cache key generated by the superclass. Alternatively, this attribute may be set to False which will disable this warning.   .../daf_butler/python/lsst/daf/butler/registry/interfaces/_database.py:1706: SAWarning: Class _TimestampColumnElement will not make use of SQL compilation caching as it does not set the 'inherit_cache' attribute to ``True``. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this object can make use of the cache key generated by the superclass. Alternatively, this attribute may be set to False which will disable this warning.   .../daf_butler/python/lsst/daf/butler/registry/databases/sqlite.py:476: SAWarning: Class _Ensure will not make use of SQL compilation caching as it does not set the 'inherit_cache' attribute to ``True``. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this object can make use of the cache key generated by the superclass. Alternatively, this attribute may be set to False which will disable this warning.   .../daf_butler/python/lsst/daf/butler/registry/interfaces/_database.py:1706: SAWarning: TypeDecorator Base64Region() will not produce a cache key because the ``cache_ok`` attribute is not set to True. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning.
            Hide
            Permalink
            salnikov Andy Salnikov added a comment -

            Jim Bosch, could you check it when you have a minute. It's mostly trivial, but I also tried to improve caching of _TimestampColumnElement based on my understanding how it works. Documentation for caching is not perfect and I don't think it makes a big difference in the end, but at least it does not break anything. Jenkins is OK and I do not see any warnings now.

            Show
            salnikov Andy Salnikov added a comment - Jim Bosch , could you check it when you have a minute. It's mostly trivial, but I also tried to improve caching of _TimestampColumnElement based on my understanding how it works. Documentation for caching is not perfect and I don't think it makes a big difference in the end, but at least it does not break anything. Jenkins is OK and I do not see any warnings now.

              People

              Assignee:
              salnikov Andy Salnikov
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              Jim Bosch
              Watchers:
              Andy Salnikov, Jim Bosch, Quentin Le Boulc'h, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.