Details
Description
Currently mysql connection from clients is handled by mysql-proxy which passes queries to czar and returns query results back to user. This proxy-czar combination has some significant issues which limit what we can do with it:
- very limited possibility to generate data on proxy side (it has to result from SQL query of some sort)
- proxy has very little information about result data and cannot do transformations on that
It looks like we can achieve better result if we can implement our own proxy which can work at the mysql wire-level protocol and integrate that proxy directly into czar. This would eliminate one server process from our current setup and should help both performance and stability.
Attachments
Issue Links
Activity
Hi K-T,
you are absolutely right that there is more than one mysql proxy out there (I personally wrote more than one). My impression is that most are focused on high performance and availability and are not very big on customization. I did some research and mysql-proxy actually looks like the most customizable one and even with that we are struggling to do what we would like to do. I'm adding Jacek and Daniel, maybe they can add more historical insight on mysql-proxy .
Anyways, we have something that is kind of working now, there is a wish to improve that but it's not clear if we have time/resources to do it. I opened this epic as a reminder in case we actually find it useful to start working on that.
There didn't used to be a number of alternative proxy implementations. But it seems that there are two that we might look at: MaxScale and ProxySQL. Both seem lower in maturity than mysqlproxy. In both, the focus is on scalability and load-balancing, and customizability is focused on rewriting queries (I think one of them has a parser). I guess if one of them is C++, we can find a good place to cut-in. Another thing we could do is to let ourselves hack on mysqlproxy at the C level--I don't think there will be any upstream changes to worry about.
Related: we talked about replacing xml-rpc proxy-czar protocol with RESTful interface. See https://confluence.lsstcorp.org/display/DM/Qserv+Hangout+2015-02-04
Comments related to MaxScale - see DM-2057
Fritz Mueller — the DMCCB wonders whether this ticket is still relevant.
I guess after 6 years we are not going to do anything like it. I think current idea is that if we are ever to replace mysql-proxy we'll get rid of mysql protocol altogether.
Please do not implement the MySQL protocol or even try to feed it directly. Look for other proxies; there are a number out there, including some that may be more customizable than mysql-proxy.