Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-28490

Qserv Replication controller crashes if no DNS entry exists for the MySQL service

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:
      None

      Description

      Problem

      The current implementation of the MySQL service within the Qserv Replication System's Framework doesn't seem to handle non-existing DNS entries for the MySQL services. The problem is seen as (for example):

      2021-01-25T16:45:01.601Z  LWP 40    INFO   MASTER CONTROLLERadmin-auth-key= auth-key= config=mysql://qsreplica:@qserv-repl-db:3306/qservReplica db-allow-reconnect=1 db-max-reconnects=1 db-reconnect-timeout=3600 db-transaction-timeout=3600 health-probe-interval=120 instance-id= qserv-db-password=****** qserv-sync-timeout=1800 replicas=0 replication-interval=1200 worker-config-timeout=600 worker-evict-timeout=3600 worker-response-timeout=60 debug=0 permanent-worker-delete=0 purge=0 qserv-sync-force=0 
      2021-01-25T16:45:01.605Z  LWP 40    ERROR  Connection[2]::_connect(_inTransaction=0,_connectTimeoutSec=3600)  unrecoverable error at: Connection[2]::_connectOnce(_inTransaction=0,_connectionAttempt=1)  mysql_real_connect() failed, error: Unknown MySQL server host 'qserv-repl-db' (-2), errno: 2005
      2021-01-25T16:45:01.605Z  LWP 40    ERROR  DatabaseServices::create  failed to instantiate MySQL-based database services, error: Connection[2]::_connectOnce(_inTransaction=0,_connectionAttempt=1)  mysql_real_connect() failed, error: Unknown MySQL server host 'qserv-repl-db' (-2), errno: 2005, no such service will be available to the application.
      main()  the application failed, exception: DatabaseServices::create  failed to instantiate MySQL-based database services, error: Connection[2]::_connectOnce(_inTransaction=0,_connectionAttempt=1)  mysql_real_connect() failed, error: Unknown MySQL server host 'qserv-repl-db' (-2), errno: 2005
      

      Solution

      Handle MySQL error code CR_UNKNOWN_HOST in the implementation of the MySQL Connection class. More details on the code can be found at: https://docs.oracle.com/cd/E19078-01/mysql/mysql-refman-5.1/error-handling.html#error_cr_unknown_host

        Attachments

          Activity

          Show
          gapon Igor Gaponenko added a comment - PR: https://github.com/lsst/qserv/pull/595
          Hide
          salnikov Andy Salnikov added a comment -

          The link that you pasted (https://docs.oracle.com/cd/E19078-01/mysql/mysql-refman-5.1/error-handling.html#error_cr_unknown_host) does not give useful info, did you mean something different?

          Show
          salnikov Andy Salnikov added a comment - The link that you pasted ( https://docs.oracle.com/cd/E19078-01/mysql/mysql-refman-5.1/error-handling.html#error_cr_unknown_host ) does not give useful info, did you mean something different?
          Hide
          salnikov Andy Salnikov added a comment -

          Looks OK but somehow I felt obliged to say something critical on my review

          Show
          salnikov Andy Salnikov added a comment - Looks OK but somehow I felt obliged to say something critical on my review

            People

            Assignee:
            gapon Igor Gaponenko
            Reporter:
            gapon Igor Gaponenko
            Reviewers:
            Andy Salnikov
            Watchers:
            Andy Salnikov, Fritz Mueller, Igor Gaponenko, Nate Pease
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: