Subject: Re: 30000 users?
To: Michael Graff <explorer@flame.org>
From: Andrew Wheadon <andrew@wipux2.wifo.uni-mannheim.de>
List: current-users
Date: 03/04/1997 11:47:48
> > Well I just tried it on a real passwd with 1178 entries, and 
> > it was half the speed. All I did was change gen/getpwent.c 
> > to use DB_BTREE instead of DB_HASH at both the two dbopens
> > and pwd_mkdb from DB_HASH, &openinfo to DB_BTREE, NULL, and
> > it took twice as long to remake the database, and change passwords.
> btree took twice as long for you?
Yup, actually even longer here's the output:
First I chroot and use a sh compiled with a BTREE-libc.a, and use a
BTREE-ls which doesn't list the owner because there is no pwd.db then
I make a BTREE-pwd.db with BTREE-pwd_mkdb and ls works. Makeing pwd.db
takes a long time. Then I use a HASH-pwd_mkdb.old to remake pwd.db and a
HASH-ls.old to list it. the HASH-pwd_mkdb is much faster. Now since 
"passwd" just seems to run pwd_mkdb it is just a slow with BTREE.
It would be great if you can prove me wrong.
Cheerio

<root@wipux2:~> chroot test bin/sh
<\u@\h:\w> cd etc;ls -l
total 346
-rw-------  1 0  0  98017 Mar  4 10:22 master.passwd
-rw-r--r--  1 0  0  78189 Mar  4 10:22 passwd
<\u@\h:\w> ../bin/time ../bin/pwd_mkdb master.passwd
       18.68 real         0.59 user         2.55 sys
<\u@\h:\w> ls -l
total 2586
-rw-------  1 root  0   98017 Mar  4 10:22 master.passwd
-rw-r--r--  1 root  0   78189 Mar  4 10:22 passwd
-rw-r--r--  1 root  0  540672 Mar  4 10:23 pwd.db
-rw-------  1 root  0  589824 Mar  4 10:23 spwd.db
<\u@\h:\w> cd etc;../bin/time ../bin/pwd_mkdb.old master.passwd
        1.93 real         0.47 user         0.39 sys
<\u@\h:\w> ls.old -l
total 2506
-rw-------  1 root  0   98017 Mar  4 10:22 master.passwd
-rw-r--r--  1 root  0   78189 Mar  4 10:22 passwd
-rw-r--r--  1 root  0  618496 Mar  4 10:28 pwd.db
-rw-------  1 root  0  622592 Mar  4 10:28 spwd.db
<\u@\h:\w>


Here's my changes:
getpwent.c
--- gen/getpwent.c.old	Tue Mar  4 11:33:39 1997
+++ gen/getpwent.c	Tue Mar  4 11:33:54 1997
@@ -97,7 +97,7 @@
 
 	/* initialize the exclusion table if needed. */
 	if(__ypexclude == (DB *)NULL) {
-		__ypexclude = dbopen(NULL, O_RDWR, 600, DB_HASH, NULL);
+		__ypexclude = dbopen(NULL, O_RDWR, 600, DB_BTREE, NULL);
 		if(__ypexclude == (DB *)NULL)
 			return(1);
 	}
@@ -890,7 +890,7 @@
 	__ypmode = YPMODE_NONE;
 #endif
 	p = (geteuid()) ? _PATH_MP_DB : _PATH_SMP_DB;
-	_pw_db = dbopen(p, O_RDONLY, 0, DB_HASH, NULL);
+	_pw_db = dbopen(p, O_RDONLY, 0, DB_BTREE, NULL);
 	if (_pw_db)
 		return(1);
 	if (!warned)


and pwd_mkdb.new
diff -u pwd_mkdb/Makefile pwd_mkdb.new/Makefile
--- pwd_mkdb/Makefile	Thu May 16 20:34:52 1996
+++ pwd_mkdb.new/Makefile	Sun Mar  2 23:01:54 1997
@@ -4,8 +4,9 @@
 PROG=	pwd_mkdb
 SRCS=	pwd_mkdb.c getpwent.c
 MAN=	pwd_mkdb.8
-LDADD+=	-lutil
+LDADD+=	-lutil /tmp/libc.a
 DPADD+=	${LIBUTIL}
+LDSTATIC=-static
 
 .PATH:	${.CURDIR}/../../lib/libc/gen
 
Common subdirectories: pwd_mkdb/obj and pwd_mkdb.new/obj
diff -u pwd_mkdb/pwd_mkdb.c pwd_mkdb.new/pwd_mkdb.c
--- pwd_mkdb/pwd_mkdb.c	Mon Nov 25 13:24:50 1996
+++ pwd_mkdb.new/pwd_mkdb.c	Sun Mar  2 23:51:15 1997
@@ -147,8 +147,14 @@
 
 	/* Open the temporary insecure password database. */
 	(void)snprintf(buf, sizeof(buf), "%s%s.tmp", prefix, _PATH_MP_DB);
+#define BTREE
+#ifdef BTREE
+	dp = dbopen(buf,
+	    O_RDWR|O_CREAT|O_EXCL, PERM_INSECURE, DB_BTREE, NULL);
+#else /* HASH */
 	dp = dbopen(buf,
 	    O_RDWR|O_CREAT|O_EXCL, PERM_INSECURE, DB_HASH, &openinfo);
+#endif /* ! BTREE */
 	if (dp == NULL)
 		error(buf);
 	clean = FILE_INSECURE;
@@ -272,8 +278,14 @@
 
 	/* Open the temporary encrypted password database. */
 	(void)snprintf(buf, sizeof(buf), "%s%s.tmp", prefix, _PATH_SMP_DB);
+#define BTREE
+#ifdef BTREE
+	edp = dbopen(buf,
+	    O_RDWR|O_CREAT|O_EXCL, PERM_SECURE, DB_BTREE, NULL);
+#else /* BTREE */
 	edp = dbopen(buf,
 	    O_RDWR|O_CREAT|O_EXCL, PERM_SECURE, DB_HASH, &openinfo);
+#endif /* !BTREE */
 	if (!edp)
 		error(buf);
 	clean = FILE_SECURE;
-- 
The cost of living hasn't affected its popularity. (Mark Twain)
current	release=doc host=netbsd.wifo.uni-mannheim.de \ "NetBSD-current mirror"
hostbase=/mit/ftp/pub/NetBSD base=/usr prefix=/usr backup delete use-rel-suffix