Re: kern/57504 (select with large enough bogus fd number set hangs instead of failing with EBADF)

[kre%NetBSD.org@localhost]

Synopsis: select with large enough bogus fd number set hangs instead of failing with EBADF

State-Changed-From-To: open->analyzed
State-Changed-By: kre%NetBSD.org@localhost
State-Changed-When: Mon, 23 Sep 2024 21:34:56 +0000
State-Changed-Why:
I have code running now which seems to fix this issue.

For now I am just using the test program (selwhat) provided in the PR, but
before committing I need to make some better tests (ones which actually do
a select() intended to work!).   I made one mod to the source, I added
	#define FD_SETSIZE 40000
before any of the #includes (so I can test operation with much larger
arg values).

On my system I have:

jacaranda$ ulimit -H -n
6668

And with that:

/tmp/selwhat 19
ret=-1
errno=9 Bad file descriptor
/tmp/selwhat 20
ret=-1
errno=9 Bad file descriptor
/tmp/selwhat 500
ret=-1
errno=9 Bad file descriptor
/tmp/selwhat 6668
ret=-1
errno=9 Bad file descriptor
/tmp/selwhat 6923
ret=-1
errno=9 Bad file descriptor
/tmp/selwhat 6924
ret=-1
errno=22 Invalid argument
/tmp/selwhat 20000
ret=-1
errno=22 Invalid argument


The code returns EINVAL if the fd arg for select() is >= $(ulimit -H -n) + FD_SETSIZE
(the kernel version of FD_SETSIZE, hence 256).  6668+256 == 2924.   Allowing the
extra is intended to allow applications to calculate the maximum FD_SETSIZE that
will work based upon the hard limit of file descriptors, and round up, and also to
allow some slop should that limit have been reduced after a very high numbered fd
was opened.

That the limit exists is just to prevent applications causing the kernel to do
excess work by supplying INT_MAX as the fd number.   If EINVAL is returned, no
checking of the fd_sets supplied is performed.   Otherwise, EBADF is returned
only if there is a set bit that doesn't represent an open fd (so passing fd == 6900
with only the bit representing fd==0 set, assuming stdin is open, won't generate
an error).

Are any comments on how this should operate, or changes that could be
made (I could use the kernel's "maxfiles" as the limit below which fds are
checked for example, rather than basing it on the process's resource limit)?