Subject: Re: Replacement for grep(1) (part 2)
To: Matthew Dillon <dillon@apollo.backplane.com>
From: Chris G. Demetriou <cgd@netbsd.org>
List: tech-userlevel
Date: 07/13/1999 14:42:05
Matthew Dillon <dillon@apollo.backplane.com> writes:

>     If you don't have the disk necessary for a standard overcommit model to
>     work, you definitely do not have the disk necessary for a non-overcommit 
>     model to work.

I'd _really_ like to know how you figure this.

text    data    bss     dec     hex     filename
45024   4096    392     49512   c168    /bin/cat
311264  12288   9900    333452  5168c   /bin/sh
212960  4096    28492   245548  3bf2c   inetd.static
458720  12288   73716   544724  84fd4   sendmail.static

None of these are particularly huge.  Dynamically linked binaries
inflate things somewhat, of course, but even there:

442336  12288   35780   490404  77ba4   /usr/lib/libc.so.12.40

the data and bss per library just aren't that huge.

Of course, in addition to the sizes of the data and bss sections, you
need to make sure you've got enough extra for dynamically allocated
data, and for stack pages, and for a few other tidbits of dynamically
allocated storage, but the end result is a system which, even without
overcommit, can fit into a reasonable amount of backing store.


>     Read the part again where I mentioned the swap requirements for a 
>     non-overcommit model to operate at the same level as the current model.

So, I just went back and looked at my mail folder for this NetBSD
mailing list (tech-userlevel), which has about a week and a half's
worth of messages.

Nowhere did I see what amounts to anything other than hand-waving
claims that you'll have to allocate much, much more backing store than
you currently need to, and claims that that's unacceptable for general
purpose computing environments.  If you have a more specific analysis
that you'd like me/us to read, please point us at it more specifically.

Regarding those claims:

* not all the world's a general purpose computing environment,

* while you certainly need to allocate more backing store than you
would with overcommit, it's _not_ ridiculously more for most
applications(+), and, finally,

* even if you are not willing to pay that price, there _are_ people
who are quite willing to pay that price to get the benefits that they
see (whether it's a matter of perception or not, from their
perspective they may as well be real) of such a scheme.


(+): obviously, there are some applications for which no-overcommit is
just silly.  However, 'normal' UNIX applications by and large allocate
memory (or map files writeable/private, or map anonymous memory) to
actually use it.  I.e. if 'cat' or 'inetd' or 'sendmail' allocates a
page from the system, it's almost certainly going write something to
it, and, while there are undoubtedly a few pages that aren't written
to, they are by far the majority.  And, of course, once the page has
been written, it's no longer reserved, it's committed.  8-)


I would honestly love to know: what do you see huge numbers of
reserved pages being reserved for, if they're not actually being
committed, by 'average' UNIX applications (for any definition of
average that excludes applications which do memory based computation
on sparse dasta).


cgd
-- 
Chris Demetriou - cgd@netbsd.org - http://www.netbsd.org/People/Pages/cgd.html
Disclaimer: Not speaking for NetBSD, just expressing my own opinion.