Subject: ksh globbing uses up unnecessary resources
To: None <netbsd-users@netbsd.org>
From: Jeremy C. Reed <reed@reedmedia.net>
List: netbsd-users
Date: 03/16/2001 23:15:14
I believe that the ksh file globbing/expansion uses up unnecessary
resources. On my AMD K-6 450 with 128MB ram running 1.5.1_ALPHA, my
machine locked up (I could ping it); and after waiting for over an hour I
had to reset the power. This can occur if you do '*/../*/../' several
times. I assume I could have stopped this problem by changing my user
resource limits (they are the defaults). (This is a similar problem
mentioned several times on Bugtraq.)
I say "unnecessary", because even if the globbing for a long item worked,
I am guessing that most likely it will be too long to be useful.
Basically globbing is the expansion of a pattern (*?[) to pathnames, for
example: "ls *.[ch]"; some info can be found in the glob(3) manual page.
Here is a patch that may help. I am submitting it here, because I am
interested if this is even useful. I don't use ksh -- but I want to learn.
This patch needs work: I am thinking that a fix should really generate
an error -- or at least not generate any result.
/bin/sh (ash?) has the similar problem which I also patched, but that
patch has some problems. (I also patched libc's glob.c using a very
similar example to FreeBSD's fix; this patch is only two lines -- I also
put it below.)
Please share your comments and your advice.
--- eval.c.orig Fri Mar 16 19:24:40 2001
+++ eval.c Fri Mar 16 20:22:25 2001
@@ -40,6 +40,8 @@
#define IFS_WS 1 /* have seen IFS white-space */
#define IFS_NWS 2 /* have seen IFS non-white-space */
+#define MAX_GLOBS 10000 /* don't let globs become out of control */
+
static int varsub ARGS((Expand *xp, char *sp, char *word, int *stypep, int *slenp));
static int comsub ARGS((Expand *xp, char *cp));
static char *trimsub ARGS((char *str, char *pat, int how));
@@ -54,6 +56,7 @@
static void alt_expand ARGS((XPtrV *wp, char *start, char *exp_start,
char *end, int fdo));
#endif
+static int total_globs;
/* compile and expand word */
char *
@@ -975,6 +978,7 @@
char *xp;
Xinit(xs, xp, 256, ATEMP);
+ total_globs = 0;
globit(&xs, &xp, cp, wp, markdirs ? GF_MARKDIR : GF_NONE);
Xfree(xs, xp);
@@ -997,6 +1001,8 @@
/* This to allow long expansions to be interrupted */
intrcheck();
+ if (total_globs > MAX_GLOBS) return;
+
if (sp == NULL) { /* end of source path */
/* We only need to check if the file exists if a pattern
* is followed by a non-pattern (eg, foo*x/bar; no check
@@ -1099,6 +1105,7 @@
/* xp = *xpp; copy_non_glob() may have re-alloc'd xs */
*xp = '\0';
prefix_len = Xlength(*xs, xp);
+ total_globs++;
dirp = ksh_opendir(prefix_len ? Xstring(*xs, xp) : ".");
if (dirp == NULL)
goto Nodir;
--- /usr/src/lib/libc/gen/__glob13.c Fri Mar 16 14:22:37 2001
+++ /usr/src/lib/libc/gen/__glob13.c.orig Fri Mar 16 09:10:33 2001
@@ -102,9 +102,6 @@
"warning: reference to compatibility globfree(); include <glob.h> for
correct reference")
#endif
-/* only allow a max amount of entries */
-#define MAX_GLOBENTRIES 10000
-
#define DOLLAR '$'
#define DOT '.'
#define EOS '\0'
@@ -721,8 +718,6 @@
_DIAGASSERT(path != NULL);
_DIAGASSERT(pglob != NULL);
-
- if (pglob->gl_pathc > MAX_GLOBENTRIES) return (GLOB_ABORTED);
newsize = sizeof(*pathv) * (2 + pglob->gl_pathc + pglob->gl_offs);
pathv = pglob->gl_pathv ?
Jeremy C. Reed
http://www.reedmedia.net/