Subject: Re: Memory leak?
To: None <current-users@NetBSD.ORG>
From: der Mouse <mouse@Collatz.McRCIM.McGill.EDU>
List: current-users
Date: 02/12/1996 13:39:35
>> Regular rebooting because of a kernel 'feature' doesn't sound very
>> professional to me. :)
> Indeed.  SunOS drives me nuts on busy machines because the kernel (or
> mountd or automount or something) fails to roll the NFS minor device
> number back around to the beginning of the device address list.

Having fairly recently attacked this very problem under SunOS, I feel
moderately well qualified to comment.

We were seeing essentially this:

- Client machine accesses /var/spool/mail, which is (after symlink
   chasing) automounted from mailhost:/var/spool/mail with an amd map
   reading
	host!=${key};type:=host;fs:=${autodir}/${rhost}/root;rhost:=${key}
- On mailhost, the /var/spool/mail path crosses filesystem mount points
   at /, /var, and /var/spool/mail.  Of these, / and /var/spool/mail
   are exported; /var isn't.
- The client mounts mailhost:/ and mailhost:/var/spool/mail
   (/var/spool/mail exists in mailhost:/ underneath the /var mount.)
- Eventually, amd decides to unmount these mounts.  unmount(2) on the
   /var/spool/mail mount point returns EINVAL, which is supposed to
   mean there's no mount there.  The unmount on / fails, of course,
   because /var/spool/mail is mounted under it.  amd decides, of
   course, that some bozo unmounted the former by hand, and that
   there's some process with an outstanding reference to the latter; it
   then remounts the rest of mailhost's mount points (it unmounts some
   of them successfully before getting the error), including (since it
   thinks it got unmounted) /var/spool/mail.
- Repeat the previous paragraph every N seconds, where N is however
   often amd retries failed unmounts that have timed out.

Each time around the loop, the filesystem gets mounted one more time.
You don't see this when running "mount" to look at the mount table,
because that is nothing but a reformatting of "cat /etc/mtab".  If you
write a kmem groveler to read the kernel's idea of the mount list,
you'll see that the mounts just pile up, one or two at a time.  As soon
as the kernel has 255 mounts outstanding, you start seeing the symptom
you first mentioned: minor number collisions.  It's not a matter of not
wrapping; the problem is that all 255 possible minors are in use, so no
matter what it picks it'll collide with _some_ other mount.  The kernel
doesn't - can't - know that some of those mount points are completely
inaccessible, nor that /etc/mtab doesn't include them.

I don't know whether the bug is in the client or the server.  We found
that having mailhost export /var worked around it; I have no idea why
that papered over whatever is really wrong.

This is one reason I really like NetBSD's not maintaining the mount
table in user-land - when you have the same information maintained two
places, they can get out of sync.  Here, part of the problem was that
the kernel reality was not matched by the user-land /etc/mtab.

					der Mouse

			    mouse@collatz.mcrcim.mcgill.edu