Subject: Re: high-volume INN server on netbsd, mmap(), and expire
To: Alexis Rosen <alexis@panix.com>
From: Brian K. Zuzga <boogles@tolkien.mit.edu>
List: current-users
Date: 08/04/1997 16:23:48
I had a very similar problem.  It did turn out to be a ulimit problem,
even though I thought I had everything set up correctly.

I narrowed it down by doing the following:

1. Running ktrace on the expire to see what it was doing.  I noticed
that it was reading and writing to the history file for every article
line it was processing.  This was a sign to that the dbz "first table"
caching wasn't working.

2. I put in a simple printf around where it malloc's the first table,
and noticed it was failing.  It proceeded silently, but slowly.  This
seems like the kind of thing a warning might be useful for.

This happened to be a FreeBSD box and I had to double MAXDSIZ.  Things
went much more smoothly after that and I sprinkled a few more
ritualistic unlimits around just to be sure.

I hope this helps.

-boogles

>>>>> "Alexis" == Alexis Rosen <alexis@panix.com> writes:

    Alexis> I wish it were as simple as any of these. News.daily has
    Alexis> had a ulimit for ages, and I always use delayrm (I run
    Alexis> multiple parallel fastrms on separate spool disks- it's
    Alexis> very effective).

    Alexis> In fact, on the *same* machine, if I type
    Alexis> /usr/local/news/bin/expire -n -z /tmp/arts & I can see it
    Alexis> rocketing through the expire file at many
    Alexis> MB/min. (Although, interestingly, the machine can take up
    Alexis> to a minute to ls the NEWSLIB while it's going.) If I type
    Alexis> /usr/local/news/bin/expire.new -n -z /tmp/arts & the
    Alexis> machine ambles through the history file at perhaps
    Alexis> 250-500KB/min. I do these things one after the other,
    Alexis> immediately, with all environmental factors exactly the
    Alexis> same (ulimit, disk characteristics, etc.).