Thanks to everyone who responded to this. Problem solved, and it's not what
I would have expected...

alexis@panix.com (Alexis Rosen) writes:
>I'm running INN 1.5.1 on netbsd 1.2. It's time to upgrade both of them,
>and I have two questions that I was hoping someone might be able to answer:

>1) mmap()- Has anyone had any problem with using the mmap() support in the
>dbz library? What about the shared-active patch or actived? As I understand
>it, this *should* work just fine. Does it? I'd *really* hate to find out
>the hard way...

Consensus is that this is OK. Various people reminded me to set various
options in INND, all of which I'd done, and all of which have worked fine
for me for ages (including dbz, as I found out- see below). What I was
really interested in was the shared-active patch; nobody so far admits to
running it.

>2) I have expire built and running, and for the last year or two it has
>worked fine, cranking through a 300-400MB history file in 10-20 minutes.
>However, I can't reproduce this binary! I've built expire from 1.5, 1.5.1,
>and even 1.4u4, but no matter what I try, I get a version that runs just
>fine, but ~150-200 *times* slower than the expire I'm using. This is pretty
>embarassing but not being able to expire would be even more embarassing
>so I'm appealing to the list for help- what options are *you* using to
>expire? (As I was typing this, I wondered if it was gcc's optimization.
>-O2, -O3, and no optimization all produce the same results.)

Now *this* was mindbogglingly simple, and yet only Richard Todd knew about
it. what I can't figure out is why INN 1.5.1 works for *anyone* under 1.2
or later without this change...

The problem is that dbz doesn't know that off_t is a 64-bit int now. And
therefore it doesn't know about the type of fseek. So fseek silently fails
early on. Details left to those who care to read the code, it's not very
interesting.

As for *why* this is a problem, well... it seems that some time between
INN1.5b1 and INN1.5 releases, the code of dbz.c was changed, even though
the version number wasn't changed (it remained 3.2). In 1.5b1 and earlier,
you could define FUNNY_SEEKS which would have the critical side effect of
including unistd.h. (Actually, that was the main effect- but the intention
was to get SEEK_SET, and the critical side effect was to get the prototype
of fseek().) From INN1.5 on, FUNNY_SEEKS was eliminated, unistd.h didn't
get included, and dbz performed like a dog.

The simple fix is to inlude unistd.h early on. I put if right before the
#ifdef SEEK_SET just because that's where it would happen if FUNNY_SEEKS
still existed. No particular reason to pick this exact spot, though.

I'm still mystified that others haven't run into this; perhaps *all* of
us last built 1.5.1 before off_t became 64 bits, and that's why our binaries
continue to work? This doesn't seem convincing...

Oh, BTW, 11MB/min on a P166 means that there's still something wrong. On a
150MHz machine with 64MB (woefully inadequate for expire while innd is
still running, so it swaps like crazy, so bad that the machine looks like
it's totally hung for several minutes at a time while it's expiring), expire
blows through a 270MB history file at >30MB/min. If I pause innd, it does
almost 55MB/min.

Again, thanks to everyone who helped out.

/a

---
Alexis Rosen   Owner/Sysadmin,
PANIX Public Access Unix & Internet, NYC.
alexis@panix.com