Subject: Re: RFC: migration to a fully dynamically linked system
To: None <tech-userlevel@netbsd.org>
From: Greg A. Woods <woods@weird.com>
List: tech-userlevel
Date: 12/21/2001 16:42:12
[ On Friday, December 21, 2001 at 17:23:00 (+0000), Ben Harris wrote: ]
> Subject: Re: RFC: migration to a fully dynamically linked system
>
> In article <20011221121958.B3408@wasabisystems.com> you write:
> >  3. Dynamically link everything against /lib
> 
> Another pro you haven't mentioned is that the memory usage of programs in   
> /sbin and /bin will decrease in this scenario.  At the moment, each of these
> programs has to have a separate in-memory copy of all the bits of libc it
> cares about.  If they were dynamically-linked, they could share them with
> each other and with the rest of the system.  Since the programs in /bin
> especially are used quite a lot (sh, ls, cp, mv), this should be a
> noticeable win.

I suggest you make some real measurements before you make such a claim.

Most of the small programs in /sbin and /bin are not long lived and may
not often run simultaneously either -- indeed many are so short-lived
that the additional overhead of ld.so often can double their wall-clock
run-time!

Most of the programs with longer lifietimes are those that also
frequently run simultaneously and which thus share not only all of the
libc code linked into them, but _all_ of their code, _all_ of the time.

Furthermore the overhead of dynamic linking for small programs can
actually cause more I/Os and more paging activity than it takes to run
the same program from a static linked binary.

This creates a very dynamic and unpredictable scenario.  The specific
application mix in a given system environment must be measured to
determine whether dynamic linking will be any win or not.  It's not
anywhere near as cut&dried a conclusion as most everyone seems to think.

I have measured some of these differences.  I have built, and am
running, a couple of fully statically linked systems.  One estimate I
obtaine by comparing the total runtime of a kernel build suggested that
the static linked system was indeed about 20% faster.  This was on a
sparc-20 with 320MB of RAM.  If I'm not mistaken this is very similar to
the experience reported some time ago about static linking on m68k.

Even the ratio of CPU and memory speed to disk speed can affect the
results.  In some systems even a static-linked xterm can start up faster
than a dynamic linked one, and if you've got more than about three or
four running simultaneously then you're actually going to use just as
little real memory as you would if they were dynamically linked!

The biggest sure savings of dynamic linking are only in the disk space
it takes to store the binary images.  If you happen to be using modern
high-speed high-capacity disks then the value of this "win" isn't so
obvious, especially when compared with all the associated disadvantages
and overheads.  If disk space is your only "cost" then by all means
dynamically link everything in sight.  However if reduced performance is
also a cost then you've got to make real-world measurements to determine
what should be statically linked and what should be dynamically linked.
There are some rules of thumb you can used to guide you, such as the
frequency of program invocation, and the number of simultaneous process
instances of the same program, but beyond that the rules get very fuzzy.

-- 
								Greg A. Woods

+1 416 218-0098;  <gwoods@acm.org>;  <g.a.woods@ieee.org>;  <woods@robohack.ca>
Planix, Inc. <woods@planix.com>; VE3TCP; Secrets of the Weird <woods@weird.com>