malloc() exceeds RLIMIT_DATA

[Graham Percival <gperciva%tarsnap.com@localhost>]

(attachment included this time, sorry!)

Greetings,

Thomas Klausner (wiz@) suggested that I write to this list.  It appears that
NetBSD's malloc() will allocate more than RLIMIT_DATA's rlim_max.

The attached use-memory.c will:

1) call getrlimit(RLIMIT_DATA, ...); to find the "soft" and "hard" limits,
   as per
   http://pubs.opengroup.org/onlinepubs/9699919799/functions/getrlimit.html
2) attempt to allocate twice the "soft" limit.
3) attempt to allocate twice the "hard" limit.

I expected it to fail at #3 and possibly #2, but instead it succeeded all the
time.

I tested it on NetBSD 8.1 inside Virtualbox (2GB memory, 4GB hard drive).
wiz tested it on his NetBSD hardware (32GB RAM, 148GB swap), although I'm not
certain exactly which version of NetBSD he used.

Sample output:

localhost: {4} cc use-memory.c
localhost: {5} ./a.out
soft limit Mb:  256
hard limit Mb:  8192
--
try malloc:     512 MB,         536870912 b
allocated Mb:   512     536870912
x is:   0x7b34c9900000
--
try malloc:     16384 MB,       17179869184 b
allocated Mb:   16384   17179869184
x is:   0x7b30e9900000
localhost: {6} 


In case the larger context is helpful, this arose while testing the scrypt
1.2.99 pre-release.  Scrypt attempts to use getrlimit(RLIMIT_DATA, ...) to
find out how much memory is available for encryption or decryption.
http://www.tarsnap.com/scrypt.html

Cheers,
- Graham Percival

#include <sys/resource.h>

#include <stdio.h>
#include <stdlib.h>

static void getmem(size_t want)
{
	void * x = NULL;

	printf("--\n");
	printf("try malloc:\t%zu MB, \t%zu b\n", want / 1024 / 1024, want);

	if ((x = malloc(want)) == NULL) {
		printf("Could not allocate memory\n");
		exit (1);
	}

	printf("allocated Mb:\t%zu\t%zu\n", want / 1024 / 1024, want);
	printf("x is:\t%p\n", x);

	free(x);
}


int main() {
	struct rlimit rlp;

	if (getrlimit(RLIMIT_DATA, &rlp)) {
		printf("getrlimit() failed\n");
	}

	printf("soft limit Mb:\t%zu\n", rlp.rlim_cur / 1024 / 1024);
	printf("hard limit Mb:\t%zu\n", rlp.rlim_max / 1024 / 1024);

	/* Can we exceed the soft limit? */
	getmem(2 * rlp.rlim_cur);

	/* Can we exceed the hard limit? */
	getmem(2 * rlp.rlim_max);

	exit (0);
}