On Sat, Feb 26, 2005 at 03:06:42PM -0600, Tracy Di Marco White wrote:
> 
> In message <4220E340.6000601@tastylime.net>, Jeff Rizzo writes:
> >I'm setting up a machine for a mirror of anoncvs, and it has 4G of RAM.  
> >I'm trying to take maximum advantage of it to keep the CVS repository in 
> >RAM (in case it's not clear, this is the NetBSD CVS repository, which is 
> >over 3G is size) to maximize performance.
> >
> >Despite various tweaks I've made to things, I'm having a hard time 
> >getting the file cache (which is the one I assume I should care about) 
> >much over 1.5G - consequently things get pretty slow at times.  Can 
> >anyone who might already be running an anoncvs mirror suggest some 
> >settings to start with?  I've also bumped kern.maxvnodes to 500000.
> 
> The applicable things in /etc/sysctl.conf:
> kern.maxvnodes=524288
> vm.filemax=90
> vm.anonmin=5
> vm.anonmax=10

You will need some other things, too.

First, it's basically imperative that you put the repository on a filesystem
that has a 4k blocksize and 512-byte fragment size.  I suggest you
carefully constrain the size of the filesystem so you don't get a totally
insane number of cylinder groups.  Since you know approximately how many
inodes you need, as well, you can adjust the number of bytes per inode to 
get the cylinder group count down, too.  Twice as many inodes as are currently
used by the repository should be enough to hold you for a long time.

You will also want to ensure that there is sufficient memory allocated to the
metadata cache.  We do this on nbftp (which runs 2.0; nbanoncvs currently runs
1.6):

vm.bufmem_lowater=268435456

Note that with a very very small change to vfs_bio to add a buffer size for
512-byte fragments (the way the code is now, they will take up 1k) you
will get significantly more efficient use of the metadata cache.

All this will help, but it will not be enough, by itself, to give you
acceptable performance on a busy anoncvs server.

For that, you will need at least one other thing:  You will need to put
/anon-root/tmp on a separate physical disk or disks.  Again, make this a
4k/512 filesystem.  Mount it async and newfs it fresh at every boot (mark
it noauto in /etc/fstab, newfs and mount it from /etc/rc.local or a separate
rc script).

If you have separate disk(s) for the anoncvs data itself, you can
probably get away with putting /anon-root/tmp  on the same physical
disk(s) as / (if you are running without a chroot for the anoncvs data,
you are living very dangerously but in that case apply what I said about
"/anon-root/tmp" to "/tmp").

You'll want to be sure all filesystems on the machine are mounted noatime,
and I would strongly recommend mounting the actual anoncvs data softdep to
keep the cost of syncing down.

If all this does not prove to be enough we have a modified cvs executable
available that we could provide to you.  But try the stuff Tracy and I have
described first and see how it goes.

-- 
 Thor Lancelot Simon	                                      tls@rek.tjls.com

"The inconsistency is startling, though admittedly, if consistency is to be
 abandoned or transcended, there is no problem."		- Noam Chomsky