On Mon, 17 May 1999, Mika Nystroem wrote:

> Hi Mike,
> 
> I don't mean to be rude here, but I think you should consider
> getting the "Sorting and Searching" volume from Knuth's series if
> you want to run programs that use tens of gigabytes of temporary
> storage.  (Look at the sections about out-of-core algorithms.)

Yes, thanks, I have that volume, and have even discussed it with Don some.


> We've got some 1 GB FreeBSD machines here (SMP), and we often want
> ...
> quite bad.  When these jobs get going, all kinds of terribly annoying
> (but not very surprising things) happen.. NFS gets slow, xterms

It's not surprising at all, is it? I mean, resource constraints do impose
their price!  But isn't it nice that the job can be run at all?

> are almost unusable, and the disks, of course, go absolutely crazy
> for about half an hour.  (The machine we mainly beat up like this
> is a dual PPro200/512k with 1 GB of RAM, 5 GB of swap, and a 16 GB
> filesystem ccd'd over four disks.  All the disks are fast/wide 7200
> rpm.)


>Using swap space to make up for the fact that you're a lazy programmer
                                                        ^^^^^^^^^^^^^^^

Thank you for that high praise!  Damn straight, I'm a lazy programmer, and
I expect the OS to do as much of the work for me as possible.  

> is simply not advisable (at least if you easily get
>bored of watching the disk activity light flicker).  


> Almost without exception, even a dumb out-of-core algorithm (like
> a little malloc() wrapper someone mentioned, or that you could
> probably whip up in a couple of hours) will perform better than
> swap space which has to "guess" what you're going to do with the
> memory next, whereas you probably have information about it that
> you just can't tell the VM system.  (If you wondered, the guy who
> wrote the circuit simulator has this excuse:  he says that it's
> not his fault that the program uses swap because the correct solution
> is just to buy more RAM.  Too bad that we couldn't find any 128
> meg FPM modules...)

I don't have any particular problem having the computer "guess" what I'm
going to do next; isn't this what "speculative execution" is all about at
the instruction set level?  Isn't that what the swapping algorithms do now?

I'm interested in having the machine do more and more; we don't program in
binary anymore; we use a nice portable assembly language like C.

The question is really: where do you want your abstractions?  Built into
the core of the OS, or layered on top of what is already there?

Since I've already shown that DynaSwap is merely an upward-compatible
extension to swap, and we've all agreed that if you don't have to swap,
you don't swap, what are the technical reasons that this is a Bad Idea?

Thanks, Mika, for your thoughts on this.

-Mike

>    Regards,
>      Mika Nystrom <mika@cs.caltech.edu>
>      Asynchronous Systems Architecture Project
>      Department of Computer Science
>      California Institute of Technology