Subject: Re: high-volume INN server on netbsd, mmap(), and expire
To: None <current-users@NetBSD.ORG>
From: Alexis Rosen <alexis@panix.com>
List: current-users
Date: 08/04/1997 14:28:42
On Mon, Aug 04, 1997 at 02:52:08PM +0300, Jarkko Torppa said:
> > I'm running INN 1.5.1 on netbsd 1.2. It's time to upgrade both of them,
> > and I have two questions that I was hoping someone might be able to answer:
> > 
> > 1) mmap()- Has anyone had any problem with using the mmap() support in the
> > dbz library? What about the shared-active patch or actived? As I understand
> > it, this *should* work just fine. Does it? I'd *really* hate to find out
> > the hard way...
> 
> mmap has worked fine for me, you have to enable the actsync option in there 
> though. Shared active did'nt compile for me when (1.4unoff) I tried it and I 
> havent tred again.

?? Actsync is for updating an active file from another host. Do you mean
ACT_STYLE should be set to MMAP?

> > 2) I have expire built and running, and for the last year or two it has
> > worked fine, cranking through a 300-400MB history file in 10-20 minutes.
> > However, I can't reproduce this binary! I've built expire from 1.5, 1.5.1,
> > and even 1.4u4, but no matter what I try, I get a version that runs just
> > fine, but ~150-200 *times* slower than the expire I'm using.
> 
> Check the ulimits on the expire, if it can't do dbzincore for the history db 
> it is slow.

No kidding...

And Perry said:
> Maybe you don't have things set to do the "list expired files to a
> file first" thing...

Scott suggests the same thing:
> Hmm.  I'm really shooting in the dark here, but did you verify that the
> `delayrm' option is being specified for the news.daily script?

David offers this:
>         I've had this same problem.  The news.daily run is a real
> memory pig. You need to unlimit the datasize right at the start of
> news.daily and then the expire run will complete in the proper amount
> of time. [explains about ulimit]

I wish it were as simple as any of these. News.daily has had a ulimit for
ages, and I always use delayrm (I run multiple parallel fastrms on separate
spool disks- it's very effective).

In fact, on the *same* machine, if I type
	/usr/local/news/bin/expire -n -z /tmp/arts &
I can see it rocketing through the expire file at many MB/min. (Although,
interestingly, the machine can take up to a minute to ls the NEWSLIB while
it's going.) If I type
	/usr/local/news/bin/expire.new -n -z /tmp/arts &
the machine ambles through the history file at perhaps 250-500KB/min. I do
these things one after the other, immediately, with all environmental factors
exactly the same (ulimit, disk characteristics, etc.).

Since nobody said "oh, I know what's wrong, it's...", I'd like to ask
those who wrote who have a working expire to send me their config.data files.
I'll see if I can compare them against mine and turn up what's wrong.

Thanks,
/a

---
Alexis Rosen   Owner/Sysadmin,
PANIX Public Access Unix & Internet, NYC.
alexis@panix.com