"Jordan K. Hubbard" <jkh@time.cdrom.com> writes:

> Unless someone has optimized the way passwd database manipulation is
> done in NetBSD, every time one of those users changes their password
> or otherwise causes the database to be rebuilt, great delays will
> occur as your 30K password database is rebuilt.

One thing to consider here is this.  When using a hash format Berkeley
DB file you can't get above about 100 inserts/sec.  If you switch to a
btree format you get many more.

Perhaps we have outgrown the hash format?

To insert 30,000 "made up" names (this is 1/4 the work pwd_mkdb does,
since it inserts 4 copies -- by name and uid in each of the private and
public databases) into a hash database:

2.578u 1.837s 0:28.53 15.4% 0+0k 561+2953io 9pf+0w

That's 29 seconds time four == 114.12 seconds.

To do the same thing with a btree:

1.896u 0.127s 0:02.20 91.3% 0+0k 26+288io 13pf+0w

Or 8.8 seconds for a real password database rebuild.

--Michael

Here's the btree flavor of the program.  Compile with either -DUSE_HASH
or -DUSE_BTREE.  The 30000.words file consists of 30000 random 4-8 character
names, selected at random from usenet postings.  They are unordered and
consist of only lowercase letters.

#include <sys/param.h>
#include <sys/stat.h>

#include <db.h>
#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <limits.h>
#include <pwd.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <util.h>

HASHINFO hashinfo = {
        4096,           /* bsize */
        32,             /* ffactor */
        256,            /* nelem */
        2048 * 1024,    /* cachesize */
        NULL,           /* hash() */
        0               /* lorder */
};

int
main(int argc, char **argv)
{
  static struct passwd pwd;                       /* password structure */
  DB *dp, *edp;
  DBT data, key;
  FILE *fp, *oldfp;
  char buf[1024];
  char buf2[1024];
  char keybuf[1024];
  int  uid;
  
  fp = fopen("30000.words", "r");
  if (fp == NULL)
    errx(1, "Cannot open 30000.words");

#ifdef USE_HASH
  dp = dbopen("30000.words.hash", O_RDWR|O_CREAT, 0600, DB_HASH, &hashinfo);
#endif
#ifdef USE_BTREE
  dp = dbopen("30000.words.btree", O_RDWR|O_CREAT, 0600, DB_BTREE, NULL);
#endif

  while (fgets(buf, 1024, fp) != NULL) {
    buf[strlen(buf) - 1] = '\0';
    snprintf(buf2, 1024,
	     "%s:%s:%d:3001:qmail:/var/qmail/alias:/sbin/nologin",
	     buf, buf, uid);

    snprintf(keybuf, 1024, "%s", buf);

    data.data = buf2;
    data.size = strlen(buf2);
    key.data = keybuf;
    key.size = strlen(keybuf);
    if ((dp->put)(dp, &key, &data, R_NOOVERWRITE) == -1)
      errx(1, "put");

    uid++;
  }

  (dp->close)(dp);
}