How would I build a shared object file?

[SamSaffron]

Trying to get my head around bazel is there a way of building a tcmalloc.so shared object file from this project?

I know the recommendation is just to compile applications with tcmalloc directly but for my use case: https://github.com/SamSaffron/allocator_bench I would like to do a side by side comparison to perftools and jemalloc that are LD_PRELOADed.

Also not against experimenting with a statically compiled ruby including tcmalloc, if we can prove it is faster / better maybe Ruby folks would be open to adding it.

[ckennelly]

Modifying the BUILD file to no longer have the linkstatic=1 attribute should produce a libtcmalloc.so.

That said, all of TCMalloc's dependencies (in Abseil) will end up being dynamically linked in that case, so the performance overhead will be higher than necessary.

[gaffneyc]

@SamSaffron I had the same idea after seeing it on HackerNews yesterday. I was able to build a shared library using code in #16.

Benchmarks were run on Linux 5.6.7, AMD 2700, compiled with gcc 9.3.0

ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
built-in mem: 170800 duration: 4.936174764
built-in mem (MALLOC_ARENA_MAX=2): 139292 duration: 5.318339345
tcmalloc 2.7 mem: 172408 duration: 4.401484569
tcmalloc 2.7.90 mem: 173216 duration: 4.773377773
jemalloc 5.0.0 mem: 169692 duration: 4.470789708
jemalloc 5.0.1 mem: 135116 duration: 4.678362483
jemalloc 5.1.0 mem: 168668 duration: 4.454680265
jemalloc 5.2.1 mem: 139296 duration: 4.780960854
google/tcmalloc mem: 197184 duration: 6.906723601

[karanaggarwal1994]

How to link abseil in tcmalloc.so?

[MaskRay]

Having a documented way to build a self-contained tcmalloc.so or tcmalloc.a will be very useful.

I am playing with lld with different malloc implementations today (https://gist.github.com/MaskRay/219effe23a767b85059097f863ebc085). For many allocators using them is simply

ld.lld @response.txt -o lld.glibc
ld.lld ~/Dev/mimalloc/out/release/libmimalloc.a @response.txt -o lld.mi
ld.lld ~/Dev/jemalloc/out/release/lib/libjemalloc.a @response.txt -o lld.je
ld.lld ~/Dev/snmalloc/out/release/libsnmallocshim-static.a @response.txt /usr/lib/x86_64-linux-gnu/libatomic.so.1 -o lld.sn

It seems that if a project doesn't use Bazel, it's very difficult to plug tcmalloc into it. (Yeah I know llvm-project has an unofficial util/bazel/ but still it's very unclear how to make it use tcmalloc.)

[junyer]

AFAIK, creating a shared library with Bazel requires a cc_binary() rule with, at minimum, linkshared = True; see this part of the Bazel documentation. Depending on the situation, wrangling symbol visibility might also be necessary; see this part of the pybind_extension() implementation. One could start off by cloning //tcmalloc:tcmalloc as //tcmalloc:tcmalloc.so and then tweaking the latter until it works well enough for one's purposes.

[jesHrz]

Following this instruction solves the problem.

Briefly, adding a new cc_binary target called libtcmalloc.so that depends on the the target tcmalloc with linkshared = 1 at the end of the BUILD, and then bazel build //tcmalloc:libtcmalloc.so will generate the dynamic library.

cc_binary(
    name = "libtcmalloc.so",
    deps = [":tcmalloc"],
    linkshared = 1,
    copts = TCMALLOC_DEFAULT_COPTS,
)

A new question: How can I access the headers outside of the tcmalloc project?

[lano1106]

@SamSaffron I had the same idea after seeing it on HackerNews yesterday. I was able to build a shared library using code in #16.

Benchmarks were run on Linux 5.6.7, AMD 2700, compiled with gcc 9.3.0

ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
built-in mem: 170800 duration: 4.936174764
built-in mem (MALLOC_ARENA_MAX=2): 139292 duration: 5.318339345
tcmalloc 2.7 mem: 172408 duration: 4.401484569
tcmalloc 2.7.90 mem: 173216 duration: 4.773377773
jemalloc 5.0.0 mem: 169692 duration: 4.470789708
jemalloc 5.0.1 mem: 135116 duration: 4.678362483
jemalloc 5.1.0 mem: 168668 duration: 4.454680265
jemalloc 5.2.1 mem: 139296 duration: 4.780960854
google/tcmalloc mem: 197184 duration: 6.906723601

this is an old benchmark... Is there more recent benchmark that compares the latest TCMalloc version performance against other allocators? I had the assumption that the latest version was at least as good as the gperf one and certainly better with all the new bells and whistles such as rseq, THP support and C++14 sized delete... but since I am having such a hard time playing with the new version, I need to make sure that the effort is worth the trouble... I am currently doubting after having spent a full day learning bazel and getting results not working well...

I do not care if the allocation time is slightly slower if the use of THP means less TLB lookups, less page faults and makes the overall process better but those things have to be measured. With all the rich documentation that the TCMalloc team is providing, I am surprised that there is no performance numbers published anywhere... gperf TCMalloc had a lot of benchmarks numbers to compare it against pt2malloc...

https://gperftools.github.io/gperftools/tcmalloc.html

[xiedeacc]

Following this instruction solves the problem.

Briefly, adding a new cc_binary target called libtcmalloc.so that depends on the the target tcmalloc with linkshared = 1 at the end of the BUILD, and then bazel build //tcmalloc:libtcmalloc.so will generate the dynamic library.

cc_binary(
    name = "libtcmalloc.so",
    deps = [":tcmalloc"],
    linkshared = 1,
    copts = TCMALLOC_DEFAULT_COPTS,
)

A new question: How can I access the headers outside of the tcmalloc project?

just copy tcmalloc directory under tcmalloc repo to /usr/local/include, or add -I point to tcmalloc repo path

[xiedeacc]

@SamSaffron I had the same idea after seeing it on HackerNews yesterday. I was able to build a shared library using code in #16.
Benchmarks were run on Linux 5.6.7, AMD 2700, compiled with gcc 9.3.0

ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
built-in mem: 170800 duration: 4.936174764
built-in mem (MALLOC_ARENA_MAX=2): 139292 duration: 5.318339345
tcmalloc 2.7 mem: 172408 duration: 4.401484569
tcmalloc 2.7.90 mem: 173216 duration: 4.773377773
jemalloc 5.0.0 mem: 169692 duration: 4.470789708
jemalloc 5.0.1 mem: 135116 duration: 4.678362483
jemalloc 5.1.0 mem: 168668 duration: 4.454680265
jemalloc 5.2.1 mem: 139296 duration: 4.780960854
google/tcmalloc mem: 197184 duration: 6.906723601

this is an old benchmark... Is there more recent benchmark that compares the latest TCMalloc version performance against other allocators? I had the assumption that the latest version was at least as good as the gperf one and certainly better with all the new bells and whistles such as rseq, THP support and C++14 sized delete... but since I am having such a hard time playing with the new version, I need to make sure that the effort is worth the trouble... I am currently doubting after having spent a full day learning bazel and getting results not working well...

I do not care if the allocation time is slightly slower if the use of THP means less TLB lookups and makes the overall process better but those things have to be measured. With all the rich documentation that the TCMalloc team is providing, I am surprised that there is no performance numbers published anywhere... gperf TCMalloc had a lot of benchmarks numbers to compare it against pt2malloc...

comment #linkstatic = 1, still works

[lano1106]

Following this instruction solves the problem.

Briefly, adding a new cc_binary target called libtcmalloc.so that depends on the the target tcmalloc with linkshared = 1 at the end of the BUILD, and then bazel build //tcmalloc:libtcmalloc.so will generate the dynamic library.

cc_binary(
    name = "libtcmalloc.so",
    deps = [":tcmalloc"],
    linkshared = 1,
    copts = TCMALLOC_DEFAULT_COPTS,
)

A new question: How can I access the headers outside of the tcmalloc project?

there is something not quite right... I am not very fond of bazel for being very opaque about what it is doing. This recipe to create a shared library will create one. With one small but critical omission. With the help of @fweimer-rh

he made me discover that bazel was not including the elf soname field in the created shared lib. This omission will stop LD_PRELOAD to work. Unless you happen to also have the libtcmalloc.so within the LD_LIBRARY_PATH search paths.

to find if you have the problem, you can use:
$ readelf -Wd libtcmalloc.so | grep SONAME

if the output is empty, you have a halfbacked shared lib...

maybe there is a way to specify bazel to include the soname but I don't know enough the tool to know how...

[xiedeacc]

Wl,soname may works

[xiedeacc]

cc_binary(
name = "libtcmalloc.so",
linkopts = ["-Wl,-soname,libtcmalloc.so"],
linkshared = 1,
deps = [":tcmalloc"],
)

this works

[lano1106]

I am going to give you the nickname of bazel master!

thx a lot... I am determined to make tcmalloc work with my project but it may take few days...

the other issue is the ODR violation if so lib user is also using Abseil that Chris Kennelly did report in some other issues...
it is the case of my project...