From: KOSAKI Motohiro <kosaki.motohiro@gmail.com> To: "H. Peter Anvin" <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org>, Alexey Dobriyan <adobriyan@gmail.com>, akpm@linux-foundation.org, viro@zeniv.linux.org.uk, drepper@gmail.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH] nextfd(2) Date: Wed, 11 Apr 2012 15:46:00 -0400 [thread overview] Message-ID: <CAHGf_=qBY1uVqWiDoOQY5UrxqK8G47+J_o=HQAbutGZNdhX5Ug@mail.gmail.com> (raw) In-Reply-To: <4F85C96B.2070803@zytor.com> >>> But it still has the same braindamage: one system call per loop >>> invocation, and we can do better. I would much rather see fdwalk() in SUS. >> >> Why would we bother to do better? >> >> System calls are cheap, and usually you actually do want to do >> something about the fd, so you actually want to iterate over them. >> >> I'd much rather have simple cheap interfaces than anything else. If >> SuS has a F_NEXT fcntl, let's just do that thing. Much simpler than >> doing something more complex and then just having to emulate the >> simple thing in user space anyway. >> >> If a standard interface exists, we should just use it. >> > > I went back and looked at the post, and also the discussion on the SUS > mailing list. > > The proposal for FD_NEXT was rejected with some serious vitriol. > > fdwalk() was considered just more palatable since there is an existing > implementation (in Solaris) and since it might be possible to provide a > way to hide specific fds from fdwalk(), but a much bigger issue raised > is that *ALL* of these interfaces are inherently broken. Closing random > file descriptors is: > > a) inherently racy in a multithreaded environment; I would say two things. 1) I know and I agree we _can_ misuse the interface. but many already existed interface also can be misused. 2) As I already explained this can be used correctly. So, I have a question. Why do you bother a possibility of misuse? Of if you didn't point out misuse, can you please point out a real world use case of multi threads + fd interation? > b) unsafe because there might be file descriptors used by libc itself. I agree this. Even though almost developer don't use libc message catalogue and we can avoid such issue by using nextfd() + fcntl(O_CLOEXEC). > Instead, from the resolution text: > >> Therefore, the rest of this proposal seeks to document the problem >> with closing arbitrary file descriptors, and a new bugid will be >> opened to propose standardizing some recent interfaces and interface >> extensions first appearing in Linux (new interfaces such as pipe2( ), >> accept4( ), mkostemps( ), ..., and extensions like fopen(,"we")) to >> guarantee the atomic creation of file descriptors with the cloexec >> bit already set, as was already done in the standard with O_CLOEXEC >> in open( ) and F_DUPFD_CLOEXEC in fcntl( ). See also 0000368 for >> a related proposal to require CLOEXEC on hidden descriptors. > > I say we ask the new glibc people to provide fdwalk() since it already > has an implementation history (and because it can be implemented without > new system calls, thereby working on old kernels), but the *big* > takeaway from this is that if there is way to create a file descriptor > so that it doesn't have O_CLOEXEC set from the very beginning, *that* is > what we need to fix. Yeah, I don't think fdwalk() is problematic. It's an option if I understand Alexey's mail correctly. but I disagree almost all developers should fix a design and rewrite their applications. In theory, they can avoid glibc or they can rewrite all of their code or avoid linux. but there is one problem. unrealistic.
WARNING: multiple messages have this Message-ID (diff)
From: KOSAKI Motohiro <kosaki.motohiro@gmail.com> To: "H. Peter Anvin" <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org>, Alexey Dobriyan <adobriyan@gmail.com>, akpm@linux-foundation.org, viro@zeniv.linux.org.uk, drepper@gmail.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH] nextfd(2) Date: Wed, 11 Apr 2012 15:46:00 -0400 [thread overview] Message-ID: <CAHGf_=qBY1uVqWiDoOQY5UrxqK8G47+J_o=HQAbutGZNdhX5Ug@mail.gmail.com> (raw) In-Reply-To: <4F85C96B.2070803@zytor.com> >>> But it still has the same braindamage: one system call per loop >>> invocation, and we can do better. I would much rather see fdwalk() in SUS. >> >> Why would we bother to do better? >> >> System calls are cheap, and usually you actually do want to do >> something about the fd, so you actually want to iterate over them. >> >> I'd much rather have simple cheap interfaces than anything else. If >> SuS has a F_NEXT fcntl, let's just do that thing. Much simpler than >> doing something more complex and then just having to emulate the >> simple thing in user space anyway. >> >> If a standard interface exists, we should just use it. >> > > I went back and looked at the post, and also the discussion on the SUS > mailing list. > > The proposal for FD_NEXT was rejected with some serious vitriol. > > fdwalk() was considered just more palatable since there is an existing > implementation (in Solaris) and since it might be possible to provide a > way to hide specific fds from fdwalk(), but a much bigger issue raised > is that *ALL* of these interfaces are inherently broken. Closing random > file descriptors is: > > a) inherently racy in a multithreaded environment; I would say two things. 1) I know and I agree we _can_ misuse the interface. but many already existed interface also can be misused. 2) As I already explained this can be used correctly. So, I have a question. Why do you bother a possibility of misuse? Of if you didn't point out misuse, can you please point out a real world use case of multi threads + fd interation? > b) unsafe because there might be file descriptors used by libc itself. I agree this. Even though almost developer don't use libc message catalogue and we can avoid such issue by using nextfd() + fcntl(O_CLOEXEC). > Instead, from the resolution text: > >> Therefore, the rest of this proposal seeks to document the problem >> with closing arbitrary file descriptors, and a new bugid will be >> opened to propose standardizing some recent interfaces and interface >> extensions first appearing in Linux (new interfaces such as pipe2( ), >> accept4( ), mkostemps( ), ..., and extensions like fopen(,"we")) to >> guarantee the atomic creation of file descriptors with the cloexec >> bit already set, as was already done in the standard with O_CLOEXEC >> in open( ) and F_DUPFD_CLOEXEC in fcntl( ). See also 0000368 for >> a related proposal to require CLOEXEC on hidden descriptors. > > I say we ask the new glibc people to provide fdwalk() since it already > has an implementation history (and because it can be implemented without > new system calls, thereby working on old kernels), but the *big* > takeaway from this is that if there is way to create a file descriptor > so that it doesn't have O_CLOEXEC set from the very beginning, *that* is > what we need to fix. Yeah, I don't think fdwalk() is problematic. It's an option if I understand Alexey's mail correctly. but I disagree almost all developers should fix a design and rewrite their applications. In theory, they can avoid glibc or they can rewrite all of their code or avoid linux. but there is one problem. unrealistic. -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2012-04-11 19:46 UTC|newest] Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top 2012-04-01 12:57 [PATCH] nextfd(2) Alexey Dobriyan 2012-04-01 13:58 ` Konstantin Khlebnikov 2012-04-01 21:30 ` Alexey Dobriyan 2012-04-02 0:09 ` Alan Cox 2012-04-02 8:38 ` Konstantin Khlebnikov 2012-04-02 9:26 ` Cyrill Gorcunov 2012-04-01 15:43 ` Eric Dumazet 2012-04-01 21:31 ` Alexey Dobriyan 2012-04-01 21:36 ` Alan Cox 2012-04-01 17:20 ` Linus Torvalds 2012-04-01 18:28 ` Valentin Nechayev 2012-04-01 21:33 ` Alexey Dobriyan 2012-04-01 19:21 ` Arnd Bergmann 2012-04-01 21:35 ` Alexey Dobriyan 2012-04-01 22:05 ` H. Peter Anvin 2012-04-04 12:13 ` Arnd Bergmann 2012-04-01 22:03 ` H. Peter Anvin 2012-04-01 22:13 ` H. Peter Anvin 2012-04-02 0:08 ` Alan Cox 2012-04-30 9:58 ` Valentin Nechayev 2012-04-02 1:19 ` Kyle Moffett 2012-04-02 1:19 ` Kyle Moffett 2012-04-02 1:37 ` H. Peter Anvin 2012-04-02 11:37 ` Ulrich Drepper 2012-04-06 9:54 ` Alexey Dobriyan 2012-04-06 9:54 ` Alexey Dobriyan 2012-04-06 15:27 ` Colin Walters 2012-04-06 16:14 ` H. Peter Anvin 2012-04-06 20:16 ` Alexey Dobriyan 2012-04-06 20:33 ` H. Peter Anvin 2012-04-06 21:02 ` H. Peter Anvin 2012-04-12 10:54 ` Alexey Dobriyan 2012-04-12 10:54 ` Alexey Dobriyan 2012-04-12 11:11 ` Alan Cox 2012-04-12 11:11 ` Alan Cox 2012-04-12 13:35 ` Alexey Dobriyan 2012-04-12 13:51 ` H. Peter Anvin 2012-04-12 19:21 ` Alexey Dobriyan 2012-04-12 14:09 ` Eric Dumazet 2012-04-06 16:23 ` H. Peter Anvin 2012-04-07 21:21 ` Ben Pfaff 2012-04-11 0:12 ` KOSAKI Motohiro 2012-04-11 0:12 ` KOSAKI Motohiro 2012-04-11 0:09 ` KOSAKI Motohiro 2012-04-11 17:58 ` H. Peter Anvin 2012-04-11 18:04 ` Linus Torvalds 2012-04-11 18:04 ` Linus Torvalds 2012-04-11 18:11 ` H. Peter Anvin 2012-04-11 19:46 ` KOSAKI Motohiro [this message] 2012-04-11 19:46 ` KOSAKI Motohiro 2012-04-11 19:49 ` H. Peter Anvin 2012-04-11 20:23 ` KOSAKI Motohiro 2012-04-11 20:32 ` H. Peter Anvin 2012-04-17 18:12 ` KOSAKI Motohiro 2012-04-11 18:00 ` H. Peter Anvin 2012-04-11 19:20 ` KOSAKI Motohiro 2012-04-11 19:20 ` KOSAKI Motohiro 2012-04-11 19:22 ` H. Peter Anvin 2012-04-11 19:26 ` KOSAKI Motohiro 2012-04-11 19:28 ` H. Peter Anvin 2012-04-11 19:31 ` KOSAKI Motohiro 2012-04-11 19:32 ` H. Peter Anvin 2012-04-02 23:17 ` KOSAKI Motohiro 2012-04-02 23:56 ` H. Peter Anvin 2012-04-04 11:51 ` Ulrich Drepper 2012-04-04 16:38 ` KOSAKI Motohiro 2012-04-04 16:43 ` Ulrich Drepper 2012-04-04 17:07 ` KOSAKI Motohiro 2012-04-04 17:49 ` Ulrich Drepper 2012-04-04 18:08 ` KOSAKI Motohiro 2012-04-04 16:31 ` KOSAKI Motohiro 2012-04-04 17:10 ` Colin Walters 2012-04-04 17:25 ` Colin Walters 2012-04-04 23:35 ` KOSAKI Motohiro 2012-04-04 18:44 ` H. Peter Anvin 2012-04-03 19:21 ` Colin Walters 2012-04-04 3:01 ` Al Viro 2012-04-04 17:10 ` KOSAKI Motohiro
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAHGf_=qBY1uVqWiDoOQY5UrxqK8G47+J_o=HQAbutGZNdhX5Ug@mail.gmail.com' \ --to=kosaki.motohiro@gmail.com \ --cc=adobriyan@gmail.com \ --cc=akpm@linux-foundation.org \ --cc=drepper@gmail.com \ --cc=hpa@zytor.com \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=torvalds@linux-foundation.org \ --cc=viro@zeniv.linux.org.uk \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.