Subject: Re: rsync'able repository with crypto/xsrc/pkgsrc ?
To: Todd Whitesel <toddpw@best.com>
From: Feico Dillema <dillema@acm.org>
List: current-users
Date: 01/19/2000 12:30:13
On Wed, Jan 19, 2000 at 02:28:49AM -0800, Todd Whitesel wrote:
> > Rather than answering your question I'll explain to you why the master
> > NetBSD AnonCVS repository does *not* let you do this.  We don't let you
> > do it because of the extreme I/O load rsync places on the machine by
> > busting the namei cache and causing tens of thousands of lookups to need
> > to hit the actual inodes which are scattered all over the disk.
> 
> Is _that_ the reason? There are lots of ways to fix that!
> 
> 1. There could be a separate rsync.netbsd.org which does one nightly rsync
> from AnonCVS and serves that to everyone else via rsync. Configure a
> ridiculously huge namei cache on the rsync server. End of problem.
> 
> 2. The repository currently available by rsync du's at about 614 megs, tars
> up to about 625 megs, and gzip -9's to about 140 megs. A daily process could
> tar up the various anoncvs modules and put them up for FTP on another server.
> A simple 7-day rotation scheme would protect against the occasional script
> failure.
> 
> 3. Tell SUP how to push the entire repository as a collection. A huge one,
> but still just a collection.
> 
> 
> Lately I'm finding that I really enjoy having a complete repository on my
> internal network, but right now that limits me to the base system sources,
> apparently.
> 
> Would hardware donations of some kind help here?

Our server, currently offering anoncvs and cvsup access, has room
enough for some extra load. Although the number of anoncvs users is
slowly but steadily growing, it's still fairly lightly loaded. So,
adding rsync access would not be a real problem I think. We could
always limit the amount of users to an acceptable level if it runs out 
hand...

I haven't set it up due to lack of time and unfamiliarity with rsync and 
because I wasn't sure whether it offered any real added value next to the 
anoncvs and cvsup access we're already offering (I guess it does now). In
other words, we may support rsync access in the future but sofar it's
relatively low on my ToDo list. If it's high on yours, you could
give me a hand to speed things up. I could give you an account on our
server, and set it up in little time. If you'd like that, just send me
a private email about it.

Feico.