Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3161

Implement MySQL-based KVInterface

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:
      None
    • Story Points:
      12
    • Sprint:
      DB_S15_07, DB_S15_08
    • Team:
      Data Access and Database

      Description

      This story covers adding mysql-based implementation of KVInterface. The implementation will be done in C++, and it will be exposed to the python layer.

        Attachments

          Issue Links

            Activity

            Hide
            npease Nate Pease [X] (Inactive) added a comment -

            This implements KvInterfaceImplMySql and a 'unit' test for it. It adds a schema file, and I think names it in the correct place so that when a czar starts the table will be created in the correct server.

            Show
            npease Nate Pease [X] (Inactive) added a comment - This implements KvInterfaceImplMySql and a 'unit' test for it. It adds a schema file, and I think names it in the correct place so that when a czar starts the table will be created in the correct server.
            Hide
            salnikov Andy Salnikov added a comment -

            OK, I left bunch of comment in PR, I believe it needs a bit of fixing. Will be happy to review it again.

            Show
            salnikov Andy Salnikov added a comment - OK, I left bunch of comment in PR, I believe it needs a bit of fixing. Will be happy to review it again.
            Hide
            salnikov Andy Salnikov added a comment -

            Nate, I checked it once again, it looks better now, still I think there are places when we pass unescaped strings to SQL, please check my comments.

            There is a bigger issue though, please do not merge until we resolve it. When I try to start qserv services in this branch czar and wmgr fail with the exceptions like:

              File "/usr/local/home/salnikov/dm-3161/bin/qservWmgr.py", line 40, in <module>
                from lsst.qserv.wmgr import auth, config, dbMgr, procMgr, xrdMgr
              File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/wmgr/config.py", line 41, in <module>
                from lsst.qserv.admin.qservAdmin import QservAdmin
              File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/admin/qservAdmin.py", line 44, in <module>
                from lsst.qserv.css.kvInterface import KvInterface, KvException
              File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/css/__init__.py", line 31, in <module>
                import cssLib
              File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/css/cssLib.py", line 28, in <module>
                _cssLib = swig_import_helper()
              File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/css/cssLib.py", line 24, in swig_import_helper
                _mod = imp.load_module('_cssLib', fp, pathname, description)
            ImportError: /usr/local/home/salnikov/dm-3161/lib/libqserv_css.so: undefined symbol: _ZN4lsst5qserv3sql14SqlTransaction5abortERNS1_14SqlErrorObjectE
            

            It says that libqserv_css.so library has unresolved symbols that belong to sql module. It is kind of obvious that libqserv_css.so should depend on sql/mysql now, but the situation is complicated by the mess in our shared libraries. I do not know how to resolve this immediately, but we should discuss that soon.

            Show
            salnikov Andy Salnikov added a comment - Nate, I checked it once again, it looks better now, still I think there are places when we pass unescaped strings to SQL, please check my comments. There is a bigger issue though, please do not merge until we resolve it. When I try to start qserv services in this branch czar and wmgr fail with the exceptions like: File "/usr/local/home/salnikov/dm-3161/bin/qservWmgr.py", line 40, in <module> from lsst.qserv.wmgr import auth, config, dbMgr, procMgr, xrdMgr File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/wmgr/config.py", line 41, in <module> from lsst.qserv.admin.qservAdmin import QservAdmin File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/admin/qservAdmin.py", line 44, in <module> from lsst.qserv.css.kvInterface import KvInterface, KvException File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/css/__init__.py", line 31, in <module> import cssLib File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/css/cssLib.py", line 28, in <module> _cssLib = swig_import_helper() File "/usr/local/home/salnikov/dm-3161/lib/python/lsst/qserv/css/cssLib.py", line 24, in swig_import_helper _mod = imp.load_module('_cssLib', fp, pathname, description) ImportError: /usr/local/home/salnikov/dm-3161/lib/libqserv_css.so: undefined symbol: _ZN4lsst5qserv3sql14SqlTransaction5abortERNS1_14SqlErrorObjectE It says that libqserv_css.so library has unresolved symbols that belong to sql module. It is kind of obvious that libqserv_css.so should depend on sql/mysql now, but the situation is complicated by the mess in our shared libraries. I do not know how to resolve this immediately, but we should discuss that soon.
            Hide
            salnikov Andy Salnikov added a comment -

            With shared libs issue resolved I think we can proceed with merging (if comments in PR are resolved).

            Show
            salnikov Andy Salnikov added a comment - With shared libs issue resolved I think we can proceed with merging (if comments in PR are resolved).

              People

              Assignee:
              npease Nate Pease [X] (Inactive)
              Reporter:
              fritzm Fritz Mueller
              Reviewers:
              Andy Salnikov
              Watchers:
              Andy Salnikov, Jacek Becla, Nate Pease [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.