RE: [PATCH v2] fs/select: add vmalloc fallback for select(2)

[David Laight <David.Laight@ACULAB.COM>]

From: Vlastimil Babka
> Sent: 26 September 2016 11:02
> On 09/23/2016 03:35 PM, David Laight wrote:
> > From: Vlastimil Babka
> >> Sent: 23 September 2016 10:59
> > ...
> >> > I suspect that fdt->max_fds is an upper bound for the highest fd the
> >> > process has open - not the RLIMIT_NOFILE value.
> >>
> >> I gathered that the highest fd effectively limits the number of files,
> >> so it's the same. I might be wrong.
> >
> > An application can reduce RLIMIT_NOFILE below that of an open file.
> 
> OK, I did some more digging in the code, and my understanding is that:
> 
> - fdt->max_fds is the current size of the fdtable, which isn't allocated upfront
> to match the limit, but grows as needed. This means it's OK for
> core_sys_select() to silently cap nfds, as it knows there are no fd's with
> higher number in the fdtable, so it's a performance optimization.

Not entirely, if any bits are set for fd above fdt->max_fds then select()
call should fail - fd not open.

> However, to
> match what the manpage says, there should be another check against RLIMIT_NOFILE
> to return -EINVAL, which there isn't, AFAICS.
> 
> - fdtable is expanded (and fdt->max_fds bumped) by
> expand_files()->expand_fdtable() which checks against fs.nr_open sysctl, which
> seems to be 1048576 where I checked.
> 
> - callers of expand_files(), such as dup(), check the rlimit(RLIMIT_NOFILE) to
> limit the expansion.
> 
> So yeah, application can reduce RLIMIT_NOFILE, but it has no effect on fdtable
> and fdt->max_fds that is already above the limit. Select syscall would have to
> check the rlimit to conform to the manpage. Or (rather?) we should fix the manpage.

I think the manpage should be fixed (delete that clause).
Then add code to the system call to scan the high bit sets (above fdt->max_fds)
for any non-zero bytes. This can be done into a small buffer.

> As for the original vmalloc() flood concern, I still think we're safe, as
> ordinary users are limited by RLIMIT_NOFILE way below sizes that would need
> vmalloc(), and root has many other options to DOS the system (or worse).

Some processes need very high numbers of fd.
Likely they don't use select() on them, but trashing performance if they
do is a bit silly.
Trying to slit the 3 masks first seems sensible.

	David