From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, dave@stgolabs.net,
edumazet@google.com, guantaol@google.com, khazhy@google.com,
linux-mm@kvack.org, mm-commits@vger.kernel.org,
soheil@google.com, torvalds@linux-foundation.org,
viro@zeniv.linux.org.uk, willemb@google.com
Subject: [patch 13/15] epoll: check ep_events_available() upon timeout
Date: Sun, 01 Nov 2020 17:08:04 -0800 [thread overview]
Message-ID: <20201102010804.uENOQsZO9%akpm@linux-foundation.org> (raw)
In-Reply-To: <20201101170656.48abbd5e88375219f868af5e@linux-foundation.org>
From: Soheil Hassas Yeganeh <soheil@google.com>
Subject: epoll: check ep_events_available() upon timeout
After abc610e01c66 ("fs/epoll: avoid barrier after an epoll_wait(2)
timeout"), we break out of the ep_poll loop upon timeout, without checking
whether there is any new events available. Prior to that patch-series we
always called ep_events_available() after exiting the loop.
This can cause races and missed wakeups. For example, consider the
following scenario reported by Guantao Liu:
Suppose we have an eventfd added using EPOLLET to an epollfd.
Thread 1: Sleeps for just below 5ms and then writes to an eventfd.
Thread 2: Calls epoll_wait with a timeout of 5 ms. If it sees an
event of the eventfd, it will write back on that fd.
Thread 3: Calls epoll_wait with a negative timeout.
Prior to abc610e01c66, it is guaranteed that Thread 3 will wake up either
by Thread 1 or Thread 2. After abc610e01c66, Thread 3 can be blocked
indefinitely if Thread 2 sees a timeout right before the write to the
eventfd by Thread 1. Thread 2 will be woken up from
schedule_hrtimeout_range and, with evail 0, it will not call
ep_send_events().
To fix this issue, while holding the lock, try to remove the thread that
timed out the wait queue and check whether it was woken up or not.
Link: https://lkml.kernel.org/r/20201028180202.952079-1-soheil.kdev@gmail.com
Fixes: abc610e01c66 ("fs/epoll: avoid barrier after an epoll_wait(2) timeout")
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Reported-by: Guantao Liu <guantaol@google.com>
Tested-by: Guantao Liu <guantaol@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Acked-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
fs/eventpoll.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
--- a/fs/eventpoll.c~epoll-check-ep_events_available-upon-timeout
+++ a/fs/eventpoll.c
@@ -1907,7 +1907,21 @@ fetch_events:
if (!schedule_hrtimeout_range(to, slack, HRTIMER_MODE_ABS)) {
timed_out = 1;
- break;
+ __set_current_state(TASK_RUNNING);
+ /*
+ * Acquire the lock and try to remove this thread from
+ * the wait queue. If this thread is not on the wait
+ * queue, it has woken up after its timeout ended
+ * before it could re-acquire the lock. In that case,
+ * try to harvest some events.
+ */
+ write_lock_irq(&ep->lock);
+ if (!list_empty(&wait.entry))
+ __remove_wait_queue(&ep->wq, &wait);
+ else
+ eavail = 1;
+ write_unlock_irq(&ep->lock);
+ goto send_events;
}
/* We were woken up, thus go and try to harvest some events */
_
next prev parent reply other threads:[~2020-11-02 1:08 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-02 1:06 incoming Andrew Morton
2020-11-02 1:07 ` [patch 01/15] mm/mremap_pages: fix static key devmap_managed_key updates Andrew Morton
2020-11-02 1:07 ` [patch 02/15] hugetlb_cgroup: fix reservation accounting Andrew Morton
2020-11-02 1:07 ` [patch 03/15] mm: memcontrol: correct the NR_ANON_THPS counter of hierarchical memcg Andrew Morton
2020-11-02 1:07 ` [patch 04/15] mm: memcg: link page counters to root if use_hierarchy is false Andrew Morton
2020-11-02 1:07 ` [patch 05/15] kasan: adopt KUNIT tests to SW_TAGS mode Andrew Morton
2020-11-02 1:07 ` [patch 06/15] mm: mempolicy: fix potential pte_unmap_unlock pte error Andrew Morton
2020-11-02 1:07 ` [patch 07/15] ptrace: fix task_join_group_stop() for the case when current is traced Andrew Morton
2020-11-02 1:07 ` [patch 08/15] lib/crc32test: remove extra local_irq_disable/enable Andrew Morton
2020-11-02 1:07 ` [patch 09/15] mm/truncate.c: make __invalidate_mapping_pages() static Andrew Morton
2020-11-02 1:07 ` [patch 10/15] kthread_worker: prevent queuing delayed work from timer_fn when it is being canceled Andrew Morton
2020-11-02 1:07 ` [patch 11/15] mm, oom: keep oom_adj under or at upper limit when printing Andrew Morton
2020-11-02 1:08 ` [patch 12/15] mm: always have io_remap_pfn_range() set pgprot_decrypted() Andrew Morton
2020-11-02 1:08 ` Andrew Morton [this message]
2020-11-02 17:08 ` [patch 13/15] epoll: check ep_events_available() upon timeout Linus Torvalds
2020-11-02 17:48 ` Soheil Hassas Yeganeh
2020-11-02 18:51 ` Linus Torvalds
2020-11-02 19:38 ` Linus Torvalds
2020-11-02 19:54 ` Soheil Hassas Yeganeh
2020-11-02 20:12 ` Linus Torvalds
2020-11-02 1:08 ` [patch 14/15] epoll: add a selftest for epoll timeout race Andrew Morton
2020-11-02 1:08 ` [patch 15/15] kernel/hung_task.c: make type annotations consistent Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201102010804.uENOQsZO9%akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=dave@stgolabs.net \
--cc=edumazet@google.com \
--cc=guantaol@google.com \
--cc=khazhy@google.com \
--cc=linux-mm@kvack.org \
--cc=mm-commits@vger.kernel.org \
--cc=soheil@google.com \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
--cc=willemb@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).