Subject: sys_select() EBADF bug
To: None <tech-kern@netbsd.org>
From: Tad Hunt <tad@entrisphere.com>
List: tech-kern
Date: 11/14/2002 15:16:47
Current code:

	sys_select()
	{
		...
		if (SCARG(uap, nd) > p->p_fd->fd_nfiles) {
			/* forgiving; slightly wrong */
			SCARG(uap, nd) = p->p_fd->fd_nfiles;
		}
		...
	}


Proposed fix:

	sys_select()
	{
		...
		/*
		 * selscan properly handles fd's which are set
		 * but beyond the end of the ofiles array by
		 * returning EBADF.  A process cannot select on
		 * more fd's than it is allowed to open.
		 */
		if (SCARG(uap, nd) > p->p_rlimit[RLIMIT_NOFILE].rlim_cur)
			return (EINVAL);
		...
	}

Isn't the above code from sys_select() which is is described as
"slightly wrong" in the comments, correct if rewritten as shown?
This way, selscan() can return the EBADF error for file descriptors
between p->p_fd->fd_nfiles and the rlimit.

Currently, select will happily block forever if a bad fd is in the
list and greater than fd_nfiles.  selscan() was already rewritten
to use fd_getfile(), which correctly handles a fd beyond the end
of the array.

This way, if the process puts a fd > the number of open files in,
it will still get an EBADF error back from select(2).

NetBSD currently returns EINVAL if "nd" < 0 (though the manpage
makes no mention of this)

I think that the right answer is to return EINVAL if "nd" is bigger
than the current rlimit, and EBADF if a fd is set between the number
of open files and the rlimit.

Solaris manpages state that select returns EINVAL if the nfds parameter
is >= FD_SETSIZE.

According to the FreeBSD manpages, they return EINVAL for an "invalid
nfds".  (Though looking at the source via the FreeBSD cvsweb seems
to say that an invalid nfds is only one that is less than 0, the
same as NetBSD).

OpenBSD seems to only return EINVAL for a bad timer.  It silently
replaces an out of range "nd" with fd_nfiles. 

I recommend returning EINVAL if "nd" >= p->p_rlimit[RLIMIT_NOFILE].rlim_cur,
and letting selscan() handle fd's between fd_nfiles and the rlimit.

-Tad