Fix Version/s: None
Team:Data Access and Database
The current version of the Replication system has a bit complicated way of configuring connections to the Qserv czar’s and Qserv worker’s MySQL services. In particular, some connection parameters to the services, such as host, port, user (and in some cases - database) need to be pre-configured in the persistent configuration of the system, while a value of the parameter password is passed into a component via a command-line option as shown below:
On top of that, some algorithms may also use the hardcoded string for the MySQL root account when opening connections.
Another problem with this way of configuring the connections is that a lack of scalability since adding a new database requires adding configuration parameters for each type (or instance) of a connection.
Altogether this complicates the process of configuring the system, and it also makes it difficult to debug problems when a connection can't be established.
Hence, the idea is to use a similar approach that was taken for configuring the replication system with the connection string modeled after one of Python's sqlalchemy. The string has the following general syntax:
This is an example of how the string could look like:
In the proposed implementation the connection configuration mechanism will be unified to rely upon the connection strings instead of the multiple disjoined parameters scattered in the Configuration and command-line options. The corresponding configuration parameters will be removed from the database. Existing command-line option --qserv-db-password will get eliminated, and the following command-line options will be added:
Consider adding the third command-line option for configuring connections to the Qserv's mysql-proxy service for submitting queries. This will be needed by a yet-to-be-added REST service used by the Qserv Web Dashboard for submitting queries to Qserv. The proposed command-line option will be supported by the Master Controller :
The proposed enhancement won't require making significant changes to the code since the current implementation of the system already has an infrastructure for parsing, validating, and using the connection strings for making MySQL connections.
Changes to the code will include:
- Removing obsolete command-line option(s).
- Adding the new command-line options qserv-czar-db, qserv-worker-db, and qserv-mysql-proxy.
- Adding static "setter" and "getter" methods to the class Configuration for these parameters.
- Migrating existing clients (within the system) to use the new parameters.
- Removing support for the obsolete configuration parameters from the transient schema definition at the class ConfigurationSchema.
- Remove unused data members dbHost, dbPort and dbUser from the worker descriptor WorkerInfo and prom relevant classes.
Update the documentation on configuring and using the Replication/Ingest system: