From: Jakub Kicinski <kuba@kernel.org>
To: Max Neunhoeffer <max@arangodb.com>
Cc: netdev@vger.kernel.org, linux-fsdevel@vger.kernel.org,
LKML <linux-kernel@vger.kernel.org>,
Roman Penyaev <rpenyaev@suse.de>,
Christopher Kohlhoff <chris.kohlhoff@clearpool.io>,
viro@zeniv.linux.org.uk
Subject: Re: epoll_wait misses edge-triggered eventfd events: bug in Linux 5.3 and 5.4
Date: Mon, 3 Feb 2020 08:48:21 -0800 [thread overview]
Message-ID: <20200203084821.7a672861@cakuba.hsd1.ca.comcast.net> (raw)
In-Reply-To: <20200203151536.caf6n4b2ymvtssmh@tux>
On Mon, 3 Feb 2020 16:15:36 +0100, Max Neunhoeffer wrote:
> Dear Jakub and all,
>
> I have done a git bisect and found that this commit introduced the epoll
> bug:
>
> https://github.com/torvalds/linux/commit/a218cc4914209ac14476cb32769b31a556355b22
>
> I Cc the author of the commit.
Awesome, thanks a lot for doing that! Hopefully Roman can take a look
soon. Breaking boost::asio seems like a pretty serious regression.
> This makes sense, since the commit introduces a new rwlock to reduce
> contention in ep_poll_callback. I do not fully understand the details
> but this sounds all very close to this bug.
>
> I have also verified that the bug is still present in the latest master
> branch in Linus' repository.
>
> Furthermore, Chris Kohlhoff has provided yet another reproducing program
> which is no longer using edge-triggered but standard level-triggered
> events and epoll_wait. This makes the bug all the more urgent, since
> potentially more programs could run into this problem and could end up
> with sleeping barbers.
>
> I have added all the details to the bugzilla bugreport:
>
> https://bugzilla.kernel.org/show_bug.cgi?id=205933
>
> Hopefully, we can resolve this now equipped with this amount of information.
>
> Best regards,
> Max.
>
> On 20/02/01 12:16, Jakub Kicinski wrote:
> > On Fri, 31 Jan 2020 14:57:30 +0100, Max Neunhoeffer wrote:
> > > Dear All,
> > >
> > > I believe I have found a bug in Linux 5.3 and 5.4 in epoll_wait/epoll_ctl
> > > when an eventfd together with edge-triggered or the EPOLLONESHOT policy
> > > is used. If an epoll_ctl call to rearm the eventfd happens approximately
> > > at the same time as the epoll_wait goes to sleep, the event can be lost,
> > > even though proper protection through a mutex is employed.
> > >
> > > The details together with two programs showing the problem can be found
> > > here:
> > >
> > > https://bugzilla.kernel.org/show_bug.cgi?id=205933
> > >
> > > Older kernels seem not to have this problem, although I did not test all
> > > versions. I know that 4.15 and 5.0 do not show the problem.
> > >
> > > Note that this method of using epoll_wait/eventfd is used by
> > > boost::asio to wake up event loops in case a new completion handler
> > > is posted to an io_service, so this is probably relevant for many
> > > applications.
> > >
> > > Any help with this would be appreciated.
> >
> > Could be networking related but let's CC FS folks just in case.
> >
> > Would you be able to perform bisection to narrow down the search
> > for a buggy change?
next prev parent reply other threads:[~2020-02-03 16:48 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-31 13:57 epoll_wait misses edge-triggered eventfd events: bug in Linux 5.3 and 5.4 Max Neunhoeffer
2020-02-01 20:16 ` Jakub Kicinski
2020-02-03 15:15 ` Max Neunhoeffer
2020-02-03 16:48 ` Jakub Kicinski [this message]
2020-02-03 17:33 ` Roman Penyaev
2020-02-03 21:03 ` Max Neunhöffer
2020-02-04 10:35 ` Roman Penyaev
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200203084821.7a672861@cakuba.hsd1.ca.comcast.net \
--to=kuba@kernel.org \
--cc=chris.kohlhoff@clearpool.io \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=max@arangodb.com \
--cc=netdev@vger.kernel.org \
--cc=rpenyaev@suse.de \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).