Subject: ksh globbing uses up unnecessary resources
To: None <netbsd-users@netbsd.org>
From: Jeremy C. Reed <reed@reedmedia.net>
List: netbsd-users
Date: 03/16/2001 23:15:14
I believe that the ksh file globbing/expansion uses up unnecessary
resources. On my AMD K-6 450 with 128MB ram running 1.5.1_ALPHA, my
machine locked up (I could ping it); and after waiting for over an hour I
had to reset the power. This can occur if you do '*/../*/../' several
times. I assume I could have stopped this problem by changing my user
resource limits (they are the defaults). (This is a similar problem
mentioned several times on Bugtraq.)

I say "unnecessary", because even if the globbing for a long item worked,
I am guessing that most likely it will be too long to be useful.

Basically globbing is the expansion of a pattern (*?[) to pathnames, for
example: "ls *.[ch]"; some info can be found in the glob(3) manual page.

Here is a patch that may help. I am submitting it here, because I am
interested if this is even useful. I don't use ksh -- but I want to learn.
This patch needs work: I am thinking that a fix should really generate
an error -- or at least not generate any result.

/bin/sh (ash?) has the similar problem which I also patched, but that
patch has some problems. (I also patched libc's glob.c using a very
similar example to FreeBSD's fix; this patch is only two lines -- I also
put it below.)

Please share your comments and your advice.

--- eval.c.orig	Fri Mar 16 19:24:40 2001
+++ eval.c	Fri Mar 16 20:22:25 2001
@@ -40,6 +40,8 @@
 #define IFS_WS		1	/* have seen IFS white-space */
 #define IFS_NWS		2	/* have seen IFS non-white-space */
 
+#define MAX_GLOBS	10000	/* don't let globs become out of control */
+
 static	int	varsub ARGS((Expand *xp, char *sp, char *word, int *stypep, int *slenp));
 static	int	comsub ARGS((Expand *xp, char *cp));
 static	char   *trimsub ARGS((char *str, char *pat, int how));
@@ -54,6 +56,7 @@
 static void	alt_expand ARGS((XPtrV *wp, char *start, char *exp_start,
 				 char *end, int fdo));
 #endif
+static	int	total_globs;
 
 /* compile and expand word */
 char *
@@ -975,6 +978,7 @@
 	char *xp;
 
 	Xinit(xs, xp, 256, ATEMP);
+	total_globs = 0;
 	globit(&xs, &xp, cp, wp, markdirs ? GF_MARKDIR : GF_NONE);
 	Xfree(xs, xp);
 
@@ -997,6 +1001,8 @@
 	/* This to allow long expansions to be interrupted */
 	intrcheck();
 
+	if (total_globs > MAX_GLOBS) return;
+
 	if (sp == NULL) {	/* end of source path */
 		/* We only need to check if the file exists if a pattern
 		 * is followed by a non-pattern (eg, foo*x/bar; no check
@@ -1099,6 +1105,7 @@
 		/* xp = *xpp;	   copy_non_glob() may have re-alloc'd xs */
 		*xp = '\0';
 		prefix_len = Xlength(*xs, xp);
+		total_globs++;
 		dirp = ksh_opendir(prefix_len ? Xstring(*xs, xp) : ".");
 		if (dirp == NULL)
 			goto Nodir;



--- /usr/src/lib/libc/gen/__glob13.c    Fri Mar 16 14:22:37 2001
+++ /usr/src/lib/libc/gen/__glob13.c.orig       Fri Mar 16 09:10:33 2001
@@ -102,9 +102,6 @@
     "warning: reference to compatibility globfree(); include <glob.h> for
correct reference")
 #endif
 
-/* only allow a max amount of entries */
-#define MAX_GLOBENTRIES 10000
-
 #define        DOLLAR          '$'
 #define        DOT             '.'
 #define        EOS             '\0'
@@ -721,8 +718,6 @@
 
        _DIAGASSERT(path != NULL);
        _DIAGASSERT(pglob != NULL);
-
-       if (pglob->gl_pathc > MAX_GLOBENTRIES) return (GLOB_ABORTED);
 
        newsize = sizeof(*pathv) * (2 + pglob->gl_pathc + pglob->gl_offs);
        pathv = pglob->gl_pathv ?


   Jeremy C. Reed
   http://www.reedmedia.net/