Re: Trivial program size inflation

[RVP <rvp%SDF.ORG@localhost>]

On Mon, 3 Jul 2023, Taylor R Campbell wrote:

> However, if I try to link something that calls malloc or realloc with
> -static -lbsdmalloc, ld objects:
>
> $ cat null.c
> #include <stdlib.h>
> int main(void) { return malloc(1) != NULL; }
>
> $ make null LDLIBS=-static\ -lbsdmalloc\ -Wl,-Map=null.map
> cc -O2    -o null null.c -static -lbsdmalloc -Wl,-Map=null.map
> ld: /usr/lib/libc.a(jemalloc.o): in function `malloc':
> jemalloc.c:(.text+0x30a6): multiple definition of `malloc'; /usr/lib/libbsdmalloc.a(malloc.o):malloc.c:(.text+0x0): first defined here
> ld: /usr/lib/libc.a(jemalloc.o): in function `realloc':
> jemalloc.c:(.text+0x49c6): multiple definition of `realloc'; /usr/lib/libbsdmalloc.a(malloc.o):malloc.c:(.text+0x282): first defined here
> ld: /usr/lib/libc.a(jemalloc.o): in function `free':
> jemalloc.c:(.text+0x4ca0): multiple definition of `free'; /usr/lib/libbsdmalloc.a(malloc.o):malloc.c:(.text+0x22b): first defined here
>
> calloc, free, posix_memalign, and aligned_alloc all work fine here.
> Not sure why calling malloc or realloc still causes jemalloc.o to be
> pulled in, in the presence of -lbsdmalloc.
>
> Not investigating further; if someone wants to investigate and make
> sure this actually works, feel free to commit this patch.

What's happening here is the result of a combination of
a) the standard symbol resolution procedure pointed out by Mouse, and
b) the alloc functions in libc.a not being marked as being "weak".

A simple setup to show what's going on is:

```
$ cat foo.c
#include <stdlib.h>
int main(void) { return calloc(1, 4096) == NULL; }

$ cat malloc.c
#include <stddef.h>
void* malloc(size_t size) { return NULL; }
void* realloc(void* ptr, size_t size) { return NULL; }
void free(void* ptr) { return; }

$ cat calloc.c
#include <stddef.h>
void* calloc(size_t n, size_t size) { return NULL; }

$
```

Note that I've split calloc() into its own file.
Now we compile and run:

```
$ cc -c calloc.c malloc.c
$ ar cru libmm.a calloc.o malloc.o
$ cc -static -o foo foo.c -L. -lmm
ld: /usr/lib/libc.a(jemalloc.o): in function `calloc':
jemalloc.c:(.text+0x4a9e): multiple definition of `calloc'; ./libmm.a(calloc.o):calloc.c:(.text+0x0): first defined here
$
```

If you change calloc() to malloc() in foo.c, you get what we've already
seen before:

```
$ cc -static -o foo foo.c -L. -lmm
ld: /usr/lib/libc.a(jemalloc.o): in function `malloc':
jemalloc.c:(.text+0x39cb): multiple definition of `malloc'; ./libmm.a(malloc.o):malloc.c:(.text+0x0): first defined here
ld: /usr/lib/libc.a(jemalloc.o): in function `realloc':
jemalloc.c:(.text+0x5376): multiple definition of `realloc'; ./libmm.a(malloc.o):malloc.c:(.text+0xf): first defined here
ld: /usr/lib/libc.a(jemalloc.o): in function `free':
jemalloc.c:(.text+0x562e): multiple definition of `free'; ./libmm.a(malloc.o):malloc.c:(.text+0x22): first defined here
```

You can work around this static-linking goofiness by either

a) doing what Mouse did:

```
$ cc -static -o foo foo.c -L. -Wl,-whole-archive -lmm -Wl,-no-whole-archive
```
or,

b) doing what Christos does for libgnumalloc in src/external/gpl2/libmalloc/lib/combined.c

(But, even here, since calloc.c has not been #include'd, we get:

```
$ cc -static -o foo foo.c -lgnumalloc
ld: /usr/lib/libc.a(jemalloc.o): in function `malloc':
jemalloc.c:(.text+0x39cb): multiple definition of `malloc'; /usr/lib/libgnumalloc.a(combined.o):combined.c:(.text+0x5a4): first defined here
ld: /usr/lib/libc.a(jemalloc.o): in function `realloc':
jemalloc.c:(.text+0x5376): multiple definition of `realloc'; /usr/lib/libgnumalloc.a(combined.o):combined.c:(.text+0xab2): first defined here
ld: /usr/lib/libc.a(jemalloc.o): in function `free':
jemalloc.c:(.text+0x562e): multiple definition of `free'; /usr/lib/libgnumalloc.a(combined.o):combined.c:(.text+0x45): first defined here
$
```

This too, should be fixed.)

Really! This is the sort of static-library wretchedness kre@ warned about...

So, we should:

a) collapse the newly added functions back into a single file.
or,

b) make the libc alloc functions "weak" like they are on FreeBSD (I was surprised
that they were not in NetBSD--I thought the historical Unix practice was to make
them weak to facilitate user overrides...):

```
$ uname -a
FreeBSD x202e 13.2-RELEASE-p1 FreeBSD 13.2-RELEASE-p1 GENERIC amd64
$ nm -g /usr/lib/libc.a |
> grep -E ' [TW] (free|(m|c|re)alloc|posix_memalign|aligned_alloc)$'
0000000000002060 W aligned_alloc
0000000000002650 W calloc
0000000000004580 W free
00000000000018c0 W malloc
00000000000019b0 W posix_memalign
0000000000002e80 W realloc
$
```

Of course, both a) + b) is fine too!

Incidentally, only _some_ of these functions are weak in Glibc (even in the
latest version). Dunno why:

```
$ uname -a
Linux X202E 5.0.0-38-generic #41-Ubuntu SMP Tue Dec 3 00:27:35 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ nm -g /lib/x86_64-linux-gnu/libc.a 2>/dev/null |
> grep -E ' [TW] (free|(m|c|re)alloc|posix_memalign|aligned_alloc)$'
0000000000006420 W aligned_alloc
00000000000064d0 W calloc
0000000000005d50 T free
00000000000055f0 T malloc
0000000000007470 W posix_memalign
0000000000005fd0 T realloc
$
```

HTH,

-RVP