# Add jemalloc to eups distrib, to enable use as a potential dependancy for qserv

XMLWordPrintable

#### Details

• Type: RFC
• Status: Implemented
• Resolution: Done
• Component/s:
• Labels:
None

#### Description

jemalloc is a drop in mallc replacement for C/C++. The primary advantage here is that it provides application memory profiling with little cost. Once installed, enabling the profiling features is a matter of setting environent variables. jemalloc getting started

#### Activity

Hide
Robert Lupton added a comment -

While debugging mallocs are very useful they tend not to be as efficient as system mallocs. I don't know anything about jemalloc, is it only used via dynamic loading patching if enabled? And how will this work on OS/X platforms (where I think that the system malloc has much of this built into – see mallocinfo)?

Show
Robert Lupton added a comment - While debugging mallocs are very useful they tend not to be as efficient as system mallocs. I don't know anything about jemalloc , is it only used via dynamic loading patching if enabled? And how will this work on OS/X platforms (where I think that the system malloc has much of this built into – see mallocinfo )?
Hide
Brian Van Klaveren added a comment -

jemalloc is actually the main malloc of FreeBSD, so it is a system malloc.

Show
Brian Van Klaveren added a comment - jemalloc is actually the main malloc of FreeBSD, so it is a system malloc.
Hide
Robert Lupton added a comment -

So on FreeBSD we don't need it. I'm worried that on some hardware the vendor may have "value added" a custom malloc that knows about e.g. their memory backplane.

Show
Robert Lupton added a comment - So on FreeBSD we don't need it. I'm worried that on some hardware the vendor may have "value added" a custom malloc that knows about e.g. their memory backplane.
Hide
John Gates added a comment -

This is a good explanation for why jemalloc is useful for memory profiling Debugging Java Native Memory Leaks

The library is only used if you set the LD_PRELOAD environment variable and the profiling only occurs if you set the MALLOC_CONF environment variable appropriately. Otherwise, it uses the system malloc.

So, if you want to use the memory profiling, do this before starting the program. It should generate a report every time a GB of memory is allocated.

 export LD_PRELOAD=/usr/local/lib/libjemalloc.so export MALLOC_CONF='prof:true,prof_prefix:/qserv/run/var/log/jeprof.out,lg_prof_interval:31,lg_prof_sample:22' 

Show
John Gates added a comment - This is a good explanation for why jemalloc is useful for memory profiling Debugging Java Native Memory Leaks The library is only used if you set the LD_PRELOAD environment variable and the profiling only occurs if you set the MALLOC_CONF environment variable appropriately. Otherwise, it uses the system malloc. So, if you want to use the memory profiling, do this before starting the program. It should generate a report every time a GB of memory is allocated. export LD_PRELOAD=/usr/local/lib/libjemalloc.so export MALLOC_CONF='prof:true,prof_prefix:/qserv/run/var/log/jeprof.out,lg_prof_interval:31,lg_prof_sample:22'
Hide
Fritz Mueller added a comment -

Please note in title above, the proposed dependency add is only for Qserv

Show
Fritz Mueller added a comment - Please note in title above, the proposed dependency add is only for Qserv
Hide
Tim Jenness added a comment -

I see no reason why this RFC can't be adopted. Is the implementation ticket DM-10971? (for switching qserv).

Show
Tim Jenness added a comment - I see no reason why this RFC can't be adopted. Is the implementation ticket DM-10971 ? (for switching qserv).
Hide
Fritz Mueller added a comment -

jemalloc was packaged and published in ticket DM-11214. This has allowed us to use jemalloc for the intended debug purposes in branch builds. It remains to be seen whether we'll cut Qserv over to jemalloc in general, but that decision and work is not under the umbrella of this RFC.

Show
Fritz Mueller added a comment - jemalloc was packaged and published in ticket DM-11214 . This has allowed us to use jemalloc for the intended debug purposes in branch builds. It remains to be seen whether we'll cut Qserv over to jemalloc in general, but that decision and work is not under the umbrella of this RFC.

#### People

Assignee:
John Gates
Reporter:
John Gates
Watchers:
Andy Hanushevsky, Brian Van Klaveren, Fritz Mueller, John Gates, Robert Lupton, Tim Jenness
1 Vote for this issue
Watchers:
6 Start watching this issue

#### Dates

Created:
Updated:
Resolved:
Planned End:

#### Jenkins

No builds found.