Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8035

daf_persistence test failures with gcc 5.2 on el5

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: daf_persistence
    • Labels:
      None

      Description

      The majority of the daf_persistence tests fail when attempting to build a conda package via conda-lsst on el5 with gcc 5.2.

      Traceback (most recent call last):
        File "tests/reposInButler.py", line 36, in <module>
          import lsst.daf.persistence as dp
        File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/__init__.py", line 28, in <module>
          from .persistenceLib import *
        File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/persistenceLib.py", line 27, in <module>
          _persistenceLib = swig_import_helper()
        File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/persistenceLib.py", line 26, in swig_import_helper
          return importlib.import_module('_persistenceLib')
        File "/home/jhoblitt/conda-lsst/miniconda/envs/_build/lib/python2.7/importlib/__init__.py", line 37, in import_module
          __import__(name)
      ImportError: No module named _persistenceLib
      tests/butlerSubset.py
      

      I'm at a loss as why the tests are failing in this environment. The library is being successfully built.

      $ ls -la /home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/_persistenceLib.so 
      -rwxrwxr-x 1 jhoblitt jhoblitt 9716560 Oct 19 14:47 /home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/_persistenceLib.so
      

      $ swig -copyright
       
      SWIG Version 3.0.10
      Copyright (c) 1995-1998
      University of Utah and the Regents of the University of California
      Copyright (c) 1998-2005
      University of Chicago
      Copyright (c) 2005-2006
      Arizona Board of Regents (University of Arizona)
      

        Attachments

          Issue Links

            Activity

            Hide
            jhoblitt Joshua Hoblitt added a comment -

            $ python -c 'import lsst.daf.persistence._persistenceLib'
            Traceback (most recent call last):
              File "<string>", line 1, in <module>
              File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/__init__.py", line 28, in <module>
                from .persistenceLib import *
              File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/persistenceLib.py", line 27, in <module>
                _persistenceLib = swig_import_helper()
              File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/persistenceLib.py", line 26, in swig_import_helper
                return importlib.import_module('_persistenceLib')
              File "/home/jhoblitt/conda-lsst/miniconda/envs/_build/lib/python2.7/importlib/__init__.py", line 37, in import_module
                __import__(name)
            ImportError: No module named _persistenceLib
            

            Show
            jhoblitt Joshua Hoblitt added a comment - $ python -c 'import lsst.daf.persistence._persistenceLib' Traceback (most recent call last): File "<string>" , line 1 , in <module> File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/__init__.py" , line 28 , in <module> from .persistenceLib import * File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/persistenceLib.py" , line 27 , in <module> _persistenceLib = swig_import_helper() File "/home/jhoblitt/conda-lsst/miniconda/conda-bld/work/python/lsst/daf/persistence/persistenceLib.py" , line 26 , in swig_import_helper return importlib.import_module( '_persistenceLib' ) File "/home/jhoblitt/conda-lsst/miniconda/envs/_build/lib/python2.7/importlib/__init__.py" , line 37 , in import_module __import__(name) ImportError: No module named _persistenceLib
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            Per slack discussion with Tim Jenness, I tried copying _persistenceLib.so into an otherwise empty dir and importing it directly

             
            $ python -c 'import _persistenceLib'
            Traceback (most recent call last):
              File "<string>", line 1, in <module>
            ImportError: /home/jhoblitt/conda-lsst/miniconda/conda-bld/work/lib/libdaf_persistence.so: symbol mysql_stmt_bind_result, version libmysqlclient_16 not defined in file libmysqlclient.so.18 with link time reference
            
            

            Show
            jhoblitt Joshua Hoblitt added a comment - Per slack discussion with Tim Jenness , I tried copying _persistenceLib.so into an otherwise empty dir and importing it directly   $ python -c 'import _persistenceLib' Traceback (most recent call last): File "<string>" , line 1 , in <module> ImportError: /home/jhoblitt/conda-lsst/miniconda/conda-bld/work/lib/libdaf_persistence.so: symbol mysql_stmt_bind_result, version libmysqlclient_16 not defined in file libmysqlclient.so. 18 with link time reference
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            The symbol is definitely in there:

             
            $ objdump -T ./conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so.18.0.0 | grep mysql_stmt_bind_result
            000000000002aad0 g    DF .text	0000000000000000  libmysqlclient_16 mysql_stmt_bind_result
            000000000002aad0 g    DF .text	0000000000000194  libmysqlclient_18 mysql_stmt_bind_result
            
            

            It is also present in mariadb-libs-10.0.27-1.fc23.x86_64, which is installed on my desktop:

            $ objdump -T libmysqlclient.so.18.0.0  | grep mysql_stmt_bind_result
            0000000000026990 g    DF .text	0000000000000000 (libmysqlclient_16) mysql_stmt_bind_result
            0000000000026990 g    DF .text	00000000000001b3  libmysqlclient_18 mysql_stmt_bind_result
            

            Show
            jhoblitt Joshua Hoblitt added a comment - The symbol is definitely in there:   $ objdump -T ./conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so. 18.0 . 0 | grep mysql_stmt_bind_result 000000000002aad0 g DF .text 0000000000000000 libmysqlclient_16 mysql_stmt_bind_result 000000000002aad0 g DF .text 0000000000000194 libmysqlclient_18 mysql_stmt_bind_result It is also present in mariadb-libs-10.0.27-1.fc23.x86_64 , which is installed on my desktop: $ objdump -T libmysqlclient.so. 18.0 . 0 | grep mysql_stmt_bind_result 0000000000026990 g DF .text 0000000000000000 (libmysqlclient_16) mysql_stmt_bind_result 0000000000026990 g DF .text 00000000000001b3 libmysqlclient_18 mysql_stmt_bind_result
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            This is from the jenkins el6-1 build slave:

            $ objdump -T /home/jenkins-slave/workspace/stack-os-matrix/label/centos-6/python/py2/lsstsw/stack/Linux64/mariadbclient/10.1.11.lsst3/lib/libmysqlclient.so.18.0.0 | grep mysql_stmt_bind_result
            000000000002a340 g    DF .text	0000000000000000 (libmysqlclient_16) mysql_stmt_bind_result
            000000000002a340 g    DF .text	0000000000000194  libmysqlclient_18 mysql_stmt_bind_result
             
            $ ldd -v /home/jenkins-slave/workspace/stack-os-matrix/label/centos-6/python/py2/lsstsw/stack/Linux64/daf_persistence/tickets.DM-7719-gce0ab30d4b/lib/libdaf_persistence.so | grep mysql
            	libmysqlclient.so.18 => /home/jenkins-slave/workspace/stack-os-matrix/label/centos-6/python/py2/lsstsw/stack/Linux64/mariadbclient/10.1.11.lsst3/lib/libmysqlclient.so.18 (0x00007fd69f07d000)
            		libmysqlclient.so.18 (libmysqlclient_18) => /home/jenkins-slave/workspace/stack-os-matrix/label/centos-6/python/py2/lsstsw/stack/Linux64/mariadbclient/10.1.11.lsst3/lib/libmysqlclient.so.18
            	/home/jenkins-slave/workspace/stack-os-matrix/label/centos-6/python/py2/lsstsw/stack/Linux64/mariadbclient/10.1.11.lsst3/lib/libmysqlclient.so.18:
            

            And this is in the conda build env:

            $ ldd -v ./conda-lsst/miniconda/conda-bld/work/lib/libdaf_persistence.so | grep mysql
            	libmysqlclient.so.18 => /home/jhoblitt/conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so.18 (0x00007f7ae9159000)
            		libmysqlclient.so.18 (libmysqlclient_18) => /home/jhoblitt/conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so.18
            		libmysqlclient.so.18 (libmysqlclient_16) => /home/jhoblitt/conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so.18
            	/home/jhoblitt/conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so.18:
            

            So it looks like there is some odd linking going on.

            Show
            jhoblitt Joshua Hoblitt added a comment - This is from the jenkins el6-1 build slave: $ objdump -T /home/jenkins-slave/workspace/stack-os-matrix/label/centos- 6 /python/py2/lsstsw/stack/Linux64/mariadbclient/ 10.1 . 11 .lsst3/lib/libmysqlclient.so. 18.0 . 0 | grep mysql_stmt_bind_result 000000000002a340 g DF .text 0000000000000000 (libmysqlclient_16) mysql_stmt_bind_result 000000000002a340 g DF .text 0000000000000194 libmysqlclient_18 mysql_stmt_bind_result   $ ldd -v /home/jenkins-slave/workspace/stack-os-matrix/label/centos- 6 /python/py2/lsstsw/stack/Linux64/daf_persistence/tickets.DM- 7719 -gce0ab30d4b/lib/libdaf_persistence.so | grep mysql libmysqlclient.so. 18 => /home/jenkins-slave/workspace/stack-os-matrix/label/centos- 6 /python/py2/lsstsw/stack/Linux64/mariadbclient/ 10.1 . 11 .lsst3/lib/libmysqlclient.so. 18 ( 0x00007fd69f07d000 ) libmysqlclient.so. 18 (libmysqlclient_18) => /home/jenkins-slave/workspace/stack-os-matrix/label/centos- 6 /python/py2/lsstsw/stack/Linux64/mariadbclient/ 10.1 . 11 .lsst3/lib/libmysqlclient.so. 18 /home/jenkins-slave/workspace/stack-os-matrix/label/centos- 6 /python/py2/lsstsw/stack/Linux64/mariadbclient/ 10.1 . 11 .lsst3/lib/libmysqlclient.so. 18 : And this is in the conda build env: $ ldd -v ./conda-lsst/miniconda/conda-bld/work/lib/libdaf_persistence.so | grep mysql libmysqlclient.so. 18 => /home/jhoblitt/conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so. 18 ( 0x00007f7ae9159000 ) libmysqlclient.so. 18 (libmysqlclient_18) => /home/jhoblitt/conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so. 18 libmysqlclient.so. 18 (libmysqlclient_16) => /home/jhoblitt/conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so. 18 /home/jhoblitt/conda-lsst/miniconda/envs/_build/opt/lsst/mariadbclient/lib/libmysqlclient.so. 18 : So it looks like there is some odd linking going on.
            Hide
            jmatt J Matt Peterson [X] (Inactive) added a comment -

            This is fixed through using a patch that Joshua Hoblitt contributed and using MariaDBClient 10.1.18.

            Show
            jmatt J Matt Peterson [X] (Inactive) added a comment - This is fixed through using a patch that Joshua Hoblitt contributed and using MariaDBClient 10.1.18.

              People

              • Assignee:
                jmatt J Matt Peterson [X] (Inactive)
                Reporter:
                jhoblitt Joshua Hoblitt
                Watchers:
                Frossie Economou, J Matt Peterson [X] (Inactive), Joshua Hoblitt, Nate Pease, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel