Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16224

Reduce excessive database traffic from the Replication Controller



    • Improvement
    • Status: Done
    • Resolution: Done
    • None
    • Qserv


      A development effort associated with this ticket is meant to address one of the side-effects introduced during the code review of DM-14262. The new version of the code (which is now formally correct) puts too much unnecessary load onto the MariaDB service when updating persistent states of the Replication-Qserv synchronization requests. This results in million rows of mostly unneeded data stored in the databases each time such synchronization is happening. Each such row contains names of a databases and chunk numbers which are sent from the Replication system to the Qserv workers. Another problem with the implementation is the rapid growth of the database space used by the Replication system (roughly 200 GB over 48 hours).

      In this ticket the detailed reporting of the database names and chunks numbers will be replaced with a simple counter of replicas sent to Qserv workers by each such request.

      The second improvement is to be made in the default configuration of the master Replication Controller. The frequency of checks made by the Cluster Health Monitoring and Replication threads is going to be significantly reduces.


        Issue Links


            Self reviewed the ticket due to minor changes in the code

            gapon Igor Gaponenko added a comment - Self reviewed the ticket due to minor changes in the code


              gapon Igor Gaponenko
              gapon Igor Gaponenko
              Igor Gaponenko
              Fritz Mueller, Igor Gaponenko, Vaikunth Thukral
              0 Vote for this issue
              3 Start watching this issue




                  No builds found.