From: Peter Zijlstra <peterz@infradead.org>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: Dima Zavin <dmitriyz@waymo.com>,
Christopher Lameter <cl@linux.com>, Li Zefan <lizefan@huawei.com>,
Pekka Enberg <penberg@kernel.org>,
David Rientjes <rientjes@google.com>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Andrew Morton <akpm@linux-foundation.org>,
cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, Cliff Spradlin <cspradlin@waymo.com>,
Mel Gorman <mgorman@techsingularity.net>
Subject: Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled()
Date: Fri, 28 Jul 2017 11:30:47 +0200 [thread overview]
Message-ID: <20170728093047.ykgbufjj74xa5x3r@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <41954034-9de1-de8e-f915-51a4b0334f98@suse.cz>
On Fri, Jul 28, 2017 at 09:45:16AM +0200, Vlastimil Babka wrote:
> [+CC PeterZ]
>
> On 07/27/2017 06:46 PM, Dima Zavin wrote:
> > In codepaths that use the begin/retry interface for reading
> > mems_allowed_seq with irqs disabled, there exists a race condition that
> > stalls the patch process after only modifying a subset of the
> > static_branch call sites.
> >
> > This problem manifested itself as a dead lock in the slub
> > allocator, inside get_any_partial. The loop reads
> > mems_allowed_seq value (via read_mems_allowed_begin),
> > performs the defrag operation, and then verifies the consistency
> > of mem_allowed via the read_mems_allowed_retry and the cookie
> > returned by xxx_begin. The issue here is that both begin and retry
> > first check if cpusets are enabled via cpusets_enabled() static branch.
> > This branch can be rewritted dynamically (via cpuset_inc) if a new
> > cpuset is created. The x86 jump label code fully synchronizes across
> > all CPUs for every entry it rewrites. If it rewrites only one of the
> > callsites (specifically the one in read_mems_allowed_retry) and then
> > waits for the smp_call_function(do_sync_core) to complete while a CPU is
> > inside the begin/retry section with IRQs off and the mems_allowed value
> > is changed, we can hang. This is because begin() will always return 0
> > (since it wasn't patched yet) while retry() will test the 0 against
> > the actual value of the seq counter.
>
> Hm I wonder if there are other static branch users potentially having
> similar problem. Then it would be best to fix this at static branch
> level. Any idea, Peter? An inelegant solution would be to have indicate
> static_branch_(un)likely() callsites ordering for the patching. I.e.
> here we would make sure that read_mems_allowed_begin() callsites are
> patched before read_mems_allowed_retry() when enabling the static key,
> and the opposite order when disabling the static key.
I'm not aware of any other sure ordering requirements. But you can
manually create this order by using 2 static keys. Then flip them in the
desired order.
next prev parent reply other threads:[~2017-07-28 9:31 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-26 16:50 [RFC PATCH] mm/slub: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin
2017-07-26 17:02 ` Christopher Lameter
2017-07-26 19:54 ` Dima Zavin
2017-07-27 16:46 ` [PATCH v2] cpuset: " Dima Zavin
2017-07-27 19:48 ` Andrew Morton
2017-07-27 21:41 ` Dima Zavin
2017-07-27 19:51 ` Andrew Morton
2017-07-27 21:41 ` Dima Zavin
2017-07-28 7:45 ` Vlastimil Babka
2017-07-28 8:48 ` Dima Zavin
2017-07-28 9:30 ` Peter Zijlstra [this message]
2017-07-28 14:05 ` Vlastimil Babka
2017-07-28 16:52 ` Dima Zavin
2017-07-31 4:01 ` [PATCH v3] " Dima Zavin
2017-07-31 4:04 ` Dima Zavin
2017-07-31 8:02 ` Vlastimil Babka
2017-07-31 9:05 ` Dima Zavin
2017-07-29 4:56 ` [PATCH v2] " kbuild test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170728093047.ykgbufjj74xa5x3r@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=cl@linux.com \
--cc=cspradlin@waymo.com \
--cc=dmitriyz@waymo.com \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lizefan@huawei.com \
--cc=mgorman@techsingularity.net \
--cc=penberg@kernel.org \
--cc=rientjes@google.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).