Details
-
Type:
Story
-
Status: To Do
-
Resolution: Unresolved
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Team:Data Access and Database
Description
Calling mmap and mlock causes queries to run significantly faster, including solo queries, which is unexpected. Further, the mlock call must complete before the query is passed to mysql and only one mlock call can be be running at a time or the speed benefit vanishes. This is not the expected behavior and it would be good to know exactly what is happening. At this point, this has only been tested on the in2p3 cluster.
Attachments
Activity
Still relevant; low priority.
While a speedup was expected, it was unexpected that solo queries finished about 40% faster with the tables locked in memory with both MariadDB and MySQL. This likely comes down to how those programs access storage in full table scans. While mlock is supposed to allow lazy locking (lock the pages of the file into memory as they are read) the Linux Kernel doesn't really support this, so all the pages of the file must be in memory before locking. Andy Hanushevsky knows more about this. It would be interesting to so see how much affect using mlock has on a system using SSD storage.
John Gates
added a comment - While a speedup was expected, it was unexpected that solo queries finished about 40% faster with the tables locked in memory with both MariadDB and MySQL. This likely comes down to how those programs access storage in full table scans. While mlock is supposed to allow lazy locking (lock the pages of the file into memory as they are read) the Linux Kernel doesn't really support this, so all the pages of the file must be in memory before locking. Andy Hanushevsky knows more about this. It would be interesting to so see how much affect using mlock has on a system using SSD storage. 
This issue has been escalated to the DMCCB since it seems no work has been done for a long time.
Is it still a valid issue?
Please address with a comment or a change of status.