From: "H. Peter Anvin" <hpa@zytor.com>
To: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
Cc: Alexey Dobriyan <adobriyan@gmail.com>,
akpm@linux-foundation.org, viro@zeniv.linux.org.uk,
torvalds@linux-foundation.org, drepper@gmail.com,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH] nextfd(2)
Date: Mon, 02 Apr 2012 16:56:50 -0700 [thread overview]
Message-ID: <4F7A3CC2.1040200@zytor.com> (raw)
In-Reply-To: <CAHGf_=pgpRrQA3-YVtS=h-Bjq2LyTFmYtKGxxWgwP0XWexnh2g@mail.gmail.com>
On 04/02/2012 04:17 PM, KOSAKI Motohiro wrote:
>
> Sorry for the long delay comment. I realized this thread now. I think
> /proc no mount case is not good explanation for the worth of this patch. The problem
> is, we can't use opendir() after fork() if an app has multi threads.
>
> SUS clearly say so,
> http://pubs.opengroup.org/onlinepubs/009695399/functions/fork.html
>
> we can only call async-signal-safe functions after fork() when multi threads and
> opendir() call malloc() internally.
>
> As far as I know, OpenJDK has a such fork-readdir-exec code and it can
> make deadlock
> when spawnning a new process. Unfortunately Java language perfeter to
> make a lot of threads rather than other language.
>
> This patch can solve such multi threaded case.
>
> offtopic, glibc malloc is a slightly clever. It reinitialize its
> internal lock when fork by using thread_atfork() hook. It mean glibc malloc can be used after
> fork() and the technique can avoid this issue. But, glibc malloc still has several
> performance problem and many people prefer to use jemalloc or google malloc instead. Then,
> they hit an old issue, bah.
>
OK, so what you're saying here is:
Linux doesn't actually have a problem unless:
1. You use the library implementation of opendir/readdir/closedir;
2. You use a nonstandard malloc for the platform which doesn't
correctly set up fork hooks (which I would consider a bug);
You can deal with this in one of two ways:
2. Fix your malloc().
1. Use the low level open()/getdents()/close() functions instead of
opendir()/readdir()/closedir().
> and I've received a request that linux aim fdwalk() several times. Example,
It doesn't sound very hard to implement fdwalk() in terms of
open/getdents/close without using malloc; since the fdwalk() interface
lets you use the stack for storage. You can then implement closefrom()
in terms of fdwalk(). Something like this (untested):
int fdwalk(int (*func)(void *, int), void *cd)
{
char buf[4096]; /* ... could be less... */
const char *p, *q;
const struct linux_dirent *dp
int dfd, fd;
unsigned char c;
int rv = 0;
int sz;
dfd = open("/proc/self/fd", O_RDONLY|O_DIRECTORY|O_CLOEXEC);
if (dfd < 0)
return -1;
/*** XXX: may want to check for procfs magic here ***/
while ((sz = getdents(dfd, buf, sizeof buf)) > 0) {
p = buf;
while (sz > offsetof(struct linux_dirent, d_name)) {
dp = (const struct linux_dirent *)p;
if (sz < dp->d_reclen)
break;
q = dp->d_name;
p += dp->d_reclen;
sz -= dp->d_reclen;
fd = 0;
while (q < p && (c = *q++)) {
c -= '0';
if (c >= 10)
goto skip;
fd = fd*10 + c;
}
if (fd != dfd)
rv = func(cd, fd);
skip:
;
}
}
if (close(dfd))
return -1;
return rv;
}
next prev parent reply other threads:[~2012-04-02 23:57 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-04-01 12:57 [PATCH] nextfd(2) Alexey Dobriyan
2012-04-01 13:58 ` Konstantin Khlebnikov
2012-04-01 21:30 ` Alexey Dobriyan
2012-04-02 0:09 ` Alan Cox
2012-04-02 8:38 ` Konstantin Khlebnikov
2012-04-02 9:26 ` Cyrill Gorcunov
2012-04-01 15:43 ` Eric Dumazet
2012-04-01 21:31 ` Alexey Dobriyan
2012-04-01 21:36 ` Alan Cox
2012-04-01 17:20 ` Linus Torvalds
2012-04-01 18:28 ` Valentin Nechayev
2012-04-01 21:33 ` Alexey Dobriyan
2012-04-01 19:21 ` Arnd Bergmann
2012-04-01 21:35 ` Alexey Dobriyan
2012-04-01 22:05 ` H. Peter Anvin
2012-04-04 12:13 ` Arnd Bergmann
2012-04-01 22:03 ` H. Peter Anvin
2012-04-01 22:13 ` H. Peter Anvin
2012-04-02 0:08 ` Alan Cox
2012-04-30 9:58 ` Valentin Nechayev
2012-04-02 1:19 ` Kyle Moffett
2012-04-02 1:37 ` H. Peter Anvin
2012-04-02 11:37 ` Ulrich Drepper
2012-04-06 9:54 ` Alexey Dobriyan
2012-04-06 15:27 ` Colin Walters
2012-04-06 16:14 ` H. Peter Anvin
2012-04-06 20:16 ` Alexey Dobriyan
2012-04-06 20:33 ` H. Peter Anvin
2012-04-06 21:02 ` H. Peter Anvin
2012-04-12 10:54 ` Alexey Dobriyan
2012-04-12 11:11 ` Alan Cox
2012-04-12 13:35 ` Alexey Dobriyan
2012-04-12 13:51 ` H. Peter Anvin
2012-04-12 19:21 ` Alexey Dobriyan
2012-04-12 14:09 ` Eric Dumazet
2012-04-06 16:23 ` H. Peter Anvin
2012-04-07 21:21 ` Ben Pfaff
2012-04-11 0:12 ` KOSAKI Motohiro
2012-04-11 0:09 ` KOSAKI Motohiro
2012-04-11 17:58 ` H. Peter Anvin
2012-04-11 18:04 ` Linus Torvalds
2012-04-11 18:11 ` H. Peter Anvin
2012-04-11 19:46 ` KOSAKI Motohiro
2012-04-11 19:49 ` H. Peter Anvin
2012-04-11 20:23 ` KOSAKI Motohiro
2012-04-11 20:32 ` H. Peter Anvin
2012-04-17 18:12 ` KOSAKI Motohiro
2012-04-11 18:00 ` H. Peter Anvin
2012-04-11 19:20 ` KOSAKI Motohiro
2012-04-11 19:22 ` H. Peter Anvin
2012-04-11 19:26 ` KOSAKI Motohiro
2012-04-11 19:28 ` H. Peter Anvin
2012-04-11 19:31 ` KOSAKI Motohiro
2012-04-11 19:32 ` H. Peter Anvin
2012-04-02 23:17 ` KOSAKI Motohiro
2012-04-02 23:56 ` H. Peter Anvin [this message]
2012-04-04 11:51 ` Ulrich Drepper
2012-04-04 16:38 ` KOSAKI Motohiro
2012-04-04 16:43 ` Ulrich Drepper
2012-04-04 17:07 ` KOSAKI Motohiro
2012-04-04 17:49 ` Ulrich Drepper
2012-04-04 18:08 ` KOSAKI Motohiro
2012-04-04 16:31 ` KOSAKI Motohiro
2012-04-04 17:10 ` Colin Walters
2012-04-04 17:25 ` Colin Walters
2012-04-04 23:35 ` KOSAKI Motohiro
2012-04-04 18:44 ` H. Peter Anvin
2012-04-03 19:21 ` Colin Walters
2012-04-04 3:01 ` Al Viro
2012-04-04 17:10 ` KOSAKI Motohiro
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4F7A3CC2.1040200@zytor.com \
--to=hpa@zytor.com \
--cc=adobriyan@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=drepper@gmail.com \
--cc=kosaki.motohiro@gmail.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).