* select fails to verify all file descriptors are valid @ 2017-03-14 16:11 Matthew Wilcox 2017-03-15 5:34 ` Carlos O'Donell [not found] ` <20170314161138.GC4033-PfSpb0PWhxZc2C7mugBRk2EX/6BAtgUQ@public.gmane.org> 0 siblings, 2 replies; 4+ messages in thread From: Matthew Wilcox @ 2017-03-14 16:11 UTC (permalink / raw) To: linux-api-u79uwXL29TY76Z2rM5mHXA; +Cc: linux-fsdevel-u79uwXL29TY76Z2rM5mHXA Quoting the manpage: int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout); nfds is the highest-numbered file descriptor in any of the three sets, plus 1. EBADF An invalid file descriptor was given in one of the sets. (Per‐ haps a file descriptor that was already closed, or one on which an error has occurred.) That's not quite how Linux behaves. We only check the fd_set up to the maximum number of fds allocated to this task: rcu_read_lock(); fdt = files_fdtable(current->files); max_fds = fdt->max_fds; rcu_read_unlock(); if (n > max_fds) n = max_fds; (then we copy in up to 'n' bits worth of bitmaps). It is pretty straightforward to demonstrate that Linux doesn't check: int main(void) { int ret; struct timeval tv = { }; fd_set fds; FD_ZERO(&fds); FD_SETFD(FD_SETSIZE - 1, &fds); ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv); assert(ret == -1 && errno == EBADF); return 0; } Linux has behaved this way since 2.6.12, and I can't be bothered to get out the historical git trees to find out what happened before 2005. So ... if I change this behaviour by checking all the file descriptors, I do stand a chance of breaking an application. On the other hand, that application could already have been broken by the shell deciding to open a really high file descriptor (I'm looking at you, bash), which the program then inherits. Worth fixing this bug? Worth documenting this bug, at least? ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: select fails to verify all file descriptors are valid 2017-03-14 16:11 select fails to verify all file descriptors are valid Matthew Wilcox @ 2017-03-15 5:34 ` Carlos O'Donell [not found] ` <4859e5e8-4cc1-626c-3294-e5f529fe41d3-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> [not found] ` <20170314161138.GC4033-PfSpb0PWhxZc2C7mugBRk2EX/6BAtgUQ@public.gmane.org> 1 sibling, 1 reply; 4+ messages in thread From: Carlos O'Donell @ 2017-03-15 5:34 UTC (permalink / raw) To: Matthew Wilcox, linux-api, Michael Kerrisk; +Cc: linux-fsdevel On 03/14/2017 12:11 PM, Matthew Wilcox wrote: Hello Matthew :-) > Quoting the manpage: > > int select(int nfds, fd_set *readfds, fd_set *writefds, > fd_set *exceptfds, struct timeval *timeout); > > nfds is the highest-numbered file descriptor in any of the three sets, > plus 1. OK. > EBADF An invalid file descriptor was given in one of the sets. (Per‐ > haps a file descriptor that was already closed, or one on which > an error has occurred.) > > That's not quite how Linux behaves. We only check the fd_set up to the > maximum number of fds allocated to this task: > > rcu_read_lock(); > fdt = files_fdtable(current->files); > max_fds = fdt->max_fds; > rcu_read_unlock(); > if (n > max_fds) > n = max_fds; > > (then we copy in up to 'n' bits worth of bitmaps). Right, and that doesn't match the POSIX requirement either. http://pubs.opengroup.org/onlinepubs/9699919799/functions/pselect.html ~~~ The nfds argument specifies the range of descriptors to be tested. The first nfds descriptors shall be checked in each set; that is, the descriptors from zero through nfds-1 in the descriptor sets shall be examined. ~~~ That is to say you should "test" `nfds - 1` descriptors, potentially returning EBADF for invalid descriptors, not some random number that depends on how many you have open right now. > It is pretty straightforward to demonstrate that Linux doesn't check: > > int main(void) > { > int ret; > struct timeval tv = { }; > fd_set fds; > FD_ZERO(&fds); > FD_SETFD(FD_SETSIZE - 1, &fds); > ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv); > assert(ret == -1 && errno == EBADF); > return 0; > } And one that compiles and works... cat >> select-verify.c <<EOF #include <stdio.h> #include <stdlib.h> #include <sys/select.h> #include <sys/time.h> #include <sys/types.h> #include <unistd.h> #include <assert.h> #include <errno.h> int main(void) { int ret; struct timeval tv = {0, 0}; fd_set fds; FD_ZERO(&fds); FD_SET(FD_SETSIZE - 1, &fds); ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv); assert(ret == -1 && errno == EBADF); return 0; } EOF gcc -O0 -g3 -Wall -pedantic -o ./select-verify ./select-verify.c ./select-verify select-verify: ./select-verify.c:19: main: Assertion `ret == -1 && errno == EBADF' failed. Aborted (core dumped) > Linux has behaved this way since 2.6.12, and I can't be bothered to get > out the historical git trees to find out what happened before 2005. > > So ... if I change this behaviour by checking all the file descriptors, I > do stand a chance of breaking an application. On the other hand, that > application could already have been broken by the shell deciding to open > a really high file descriptor (I'm looking at you, bash), which the program > then inherits. The user application would have to: (a) Use a sloppy limit for nfds e.g. `FD_SETSIZE - 1`. _and_ (b) Have an fd selected that they were not interested in selecting. Exactly what the test application above does. > Worth fixing this bug? Worth documenting this bug, at least? I would fix it on the grounds of making the interface as robust as it was designed to be. Right now it randomly depends on your task max_fds if it's going to work or not. That's pretty terrible. I'm including Michael Kerrisk here if he has any comments since he touched the manual page several times ;-) -- Cheers, Carlos. ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <4859e5e8-4cc1-626c-3294-e5f529fe41d3-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: select fails to verify all file descriptors are valid [not found] ` <4859e5e8-4cc1-626c-3294-e5f529fe41d3-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2017-03-24 14:41 ` Michael Kerrisk (man-pages) 0 siblings, 0 replies; 4+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-03-24 14:41 UTC (permalink / raw) To: Carlos O'Donell, Matthew Wilcox, linux-api-u79uwXL29TY76Z2rM5mHXA Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, linux-fsdevel-u79uwXL29TY76Z2rM5mHXA Hi Carlos, On 03/15/2017 06:34 AM, Carlos O'Donell wrote: > On 03/14/2017 12:11 PM, Matthew Wilcox wrote: > > Hello Matthew :-) > >> Quoting the manpage: >> >> int select(int nfds, fd_set *readfds, fd_set *writefds, >> fd_set *exceptfds, struct timeval *timeout); >> >> nfds is the highest-numbered file descriptor in any of the three sets, >> plus 1. > > OK. > >> EBADF An invalid file descriptor was given in one of the sets. (Per‐ >> haps a file descriptor that was already closed, or one on which >> an error has occurred.) >> >> That's not quite how Linux behaves. We only check the fd_set up to the >> maximum number of fds allocated to this task: >> >> rcu_read_lock(); >> fdt = files_fdtable(current->files); >> max_fds = fdt->max_fds; >> rcu_read_unlock(); >> if (n > max_fds) >> n = max_fds; >> >> (then we copy in up to 'n' bits worth of bitmaps). > > Right, and that doesn't match the POSIX requirement either. > > http://pubs.opengroup.org/onlinepubs/9699919799/functions/pselect.html > ~~~ > The nfds argument specifies the range of descriptors to be tested. > The first nfds descriptors shall be checked in each set; that is, > the descriptors from zero through nfds-1 in the descriptor sets > shall be examined. > ~~~ > > That is to say you should "test" `nfds - 1` descriptors, potentially > returning EBADF for invalid descriptors, not some random number that > depends on how many you have open right now. Thanks for checking POSIX! >> It is pretty straightforward to demonstrate that Linux doesn't check: >> >> int main(void) >> { >> int ret; >> struct timeval tv = { }; >> fd_set fds; >> FD_ZERO(&fds); >> FD_SETFD(FD_SETSIZE - 1, &fds); >> ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv); >> assert(ret == -1 && errno == EBADF); >> return 0; >> } > > And one that compiles and works... > > cat >> select-verify.c <<EOF > #include <stdio.h> > #include <stdlib.h> > #include <sys/select.h> > #include <sys/time.h> > #include <sys/types.h> > #include <unistd.h> > #include <assert.h> > #include <errno.h> > > int > main(void) > { > int ret; > struct timeval tv = {0, 0}; > fd_set fds; > FD_ZERO(&fds); > FD_SET(FD_SETSIZE - 1, &fds); > ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv); > assert(ret == -1 && errno == EBADF); > return 0; > } > EOF > gcc -O0 -g3 -Wall -pedantic -o ./select-verify ./select-verify.c > ./select-verify > select-verify: ./select-verify.c:19: main: Assertion `ret == -1 && errno == EBADF' failed. > Aborted (core dumped) And thanks for the CWE. >> Linux has behaved this way since 2.6.12, and I can't be bothered to get >> out the historical git trees to find out what happened before 2005. >> >> So ... if I change this behaviour by checking all the file descriptors, I >> do stand a chance of breaking an application. On the other hand, that >> application could already have been broken by the shell deciding to open >> a really high file descriptor (I'm looking at you, bash), which the program >> then inherits. > > The user application would have to: > > (a) Use a sloppy limit for nfds e.g. `FD_SETSIZE - 1`. > > _and_ > > (b) Have an fd selected that they were not interested in selecting. > > Exactly what the test application above does. > >> Worth fixing this bug? Worth documenting this bug, at least? > > I would fix it on the grounds of making the interface as robust as > it was designed to be. > > Right now it randomly depends on your task max_fds if it's going to > work or not. That's pretty terrible. It's certainly at least sloppy. Of course, fixing this might conceivably break some (broken) code. But, the violation of POSIX suggests we should at least try a fix and see if anything does get broken (presumably, it's rather unlikely). > I'm including Michael Kerrisk here if he has any comments since he > touched the manual page several times ;-) Thanks ;-). Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <20170314161138.GC4033-PfSpb0PWhxZc2C7mugBRk2EX/6BAtgUQ@public.gmane.org>]
* Re: select fails to verify all file descriptors are valid [not found] ` <20170314161138.GC4033-PfSpb0PWhxZc2C7mugBRk2EX/6BAtgUQ@public.gmane.org> @ 2017-03-24 14:40 ` Michael Kerrisk (man-pages) 0 siblings, 0 replies; 4+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-03-24 14:40 UTC (permalink / raw) To: Matthew Wilcox, linux-api-u79uwXL29TY76Z2rM5mHXA Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, linux-fsdevel-u79uwXL29TY76Z2rM5mHXA, Carlos O'Donell Hi Matthew, On 03/14/2017 05:11 PM, Matthew Wilcox wrote: > > Quoting the manpage: > > int select(int nfds, fd_set *readfds, fd_set *writefds, > fd_set *exceptfds, struct timeval *timeout); > > nfds is the highest-numbered file descriptor in any of the three sets, > plus 1. > > EBADF An invalid file descriptor was given in one of the sets. (Per‐ > haps a file descriptor that was already closed, or one on which > an error has occurred.) > > That's not quite how Linux behaves. We only check the fd_set up to the > maximum number of fds allocated to this task: > > rcu_read_lock(); > fdt = files_fdtable(current->files); > max_fds = fdt->max_fds; > rcu_read_unlock(); > if (n > max_fds) > n = max_fds; > > (then we copy in up to 'n' bits worth of bitmaps). > > It is pretty straightforward to demonstrate that Linux doesn't check: > > int main(void) > { > int ret; > struct timeval tv = { }; > fd_set fds; > FD_ZERO(&fds); > FD_SETFD(FD_SETSIZE - 1, &fds); > ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv); > assert(ret == -1 && errno == EBADF); > return 0; > } > > Linux has behaved this way since 2.6.12, and I can't be bothered to get > out the historical git trees to find out what happened before 2005. Looking back, this behavior seems to have been present at least as far back as 2.4.0: if (n > current->files->max_fdset) n = current->files->max_fdset; My reading on the 2.2.0 source suggests this odd behavior did not occur there. > So ... if I change this behaviour by checking all the file descriptors, I > do stand a chance of breaking an application. On the other hand, that > application could already have been broken by the shell deciding to open > a really high file descriptor (I'm looking at you, bash), which the program > then inherits. I'm not sure that bash's FD 255 oddity is a problem here, since the close-on-exec flag is set on that FD. > Worth fixing this bug? Worth documenting this bug, at least? Certainly worth documenting. Fixing? I'm less sure. But, Carlos does have a point about POSIX... I've updated the man page to say: nfds should be set to the highest-numbered file descriptor in any of the three sets, plus 1. The indicated file descriptors in each set are checked, up to this limit (but see BUGS). ... EBADF An invalid file descriptor was given in one of the sets. (Perhaps a file descriptor that was already closed, or one on which an error has occurred.) However, see BUGS. ... BUGS According to POSIX, select() should check all specified file descriptors in the three file descriptor sets, up to the limit nfds-1. However, the current implementation ignores any file descriptor in these sets that is greater than the maximum file descriptor number that the process currently has open. According to POSIX, any such file descriptor that is specified in one of the sets should result in the error EBADF. Cheers, Michael > -- > To unsubscribe from this list: send the line "unsubscribe linux-api" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-03-24 14:41 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-03-14 16:11 select fails to verify all file descriptors are valid Matthew Wilcox 2017-03-15 5:34 ` Carlos O'Donell [not found] ` <4859e5e8-4cc1-626c-3294-e5f529fe41d3-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2017-03-24 14:41 ` Michael Kerrisk (man-pages) [not found] ` <20170314161138.GC4033-PfSpb0PWhxZc2C7mugBRk2EX/6BAtgUQ@public.gmane.org> 2017-03-24 14:40 ` Michael Kerrisk (man-pages)
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).