From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: linux-kernel@vger.kernel.org, linux-rt-users@vger.kernel.org,
Carsten Emde <C.Emde@osadl.org>, Christoph Lameter <cl@linux.com>,
Pekka Enberg <penberg@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Subject: [PATCH 09/12] FIX [2/2] slub: Tid must be retrieved from the percpu area of the current processor
Date: Wed, 13 Feb 2013 17:13:48 +0100 [thread overview]
Message-ID: <1360772031-27186-10-git-send-email-bigeasy@linutronix.de> (raw)
In-Reply-To: <1360772031-27186-1-git-send-email-bigeasy@linutronix.de>
From: Christoph Lameter <cl@linux.com>
As Steven Rostedt has pointer out: Rescheduling could occur on a differnet processor
after the determination of the per cpu pointer and before the tid is retrieved.
This could result in allocation from the wrong node in slab_alloc.
The effect is much more severe in slab_free() where we could free to the freelist
of the wrong page.
The window for something like that occurring is pretty small but it is possible.
Signed-off-by: Christoph Lameter <cl@linux.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Pekka Enberg <penberg@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
mm/slub.c | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/mm/slub.c b/mm/slub.c
index fbf6810..634aabc 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2313,13 +2313,18 @@ static __always_inline void *slab_alloc(struct kmem_cache *s,
return NULL;
redo:
-
/*
* Must read kmem_cache cpu data via this cpu ptr. Preemption is
* enabled. We may switch back and forth between cpus while
* reading from one cpu area. That does not matter as long
* as we end up on the original cpu again when doing the cmpxchg.
+ *
+ * Preemption is disabled for the retrieval of the tid because that
+ * must occur from the current processor. We cannot allow rescheduling
+ * on a different processor between the determination of the pointer
+ * and the retrieval of the tid.
*/
+ preempt_disable();
c = __this_cpu_ptr(s->cpu_slab);
/*
@@ -2329,7 +2334,7 @@ static __always_inline void *slab_alloc(struct kmem_cache *s,
* linked list in between.
*/
tid = c->tid;
- barrier();
+ preempt_enable();
object = c->freelist;
if (unlikely(!object || !node_match(c, node)))
@@ -2575,10 +2580,11 @@ static __always_inline void slab_free(struct kmem_cache *s,
* data is retrieved via this pointer. If we are on the same cpu
* during the cmpxchg then the free will succedd.
*/
+ preempt_disable();
c = __this_cpu_ptr(s->cpu_slab);
tid = c->tid;
- barrier();
+ preempt_enable();
if (likely(page == c->page)) {
set_freepointer(s, object, c->freelist);
--
1.7.10.4
next prev parent reply other threads:[~2013-02-13 16:14 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-02-13 16:13 [PREEMPT RT] SLUB and split softirq lock for v3.4-rt Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 01/12] softirq: Make serving softirqs a task flag Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 02/12] softirq: Split handling function Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 03/12] softirq: Split softirq locks Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 04/12] softirq: Adapt NOHZ softirq pending check to new RT scheme Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 05/12] softirq: Add more debugging Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 06/12] softirq: Fix nohz pending issue for real Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 07/12] net: Use local_bh_disable in netif_rx_ni() Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 08/12] FIX [1/2] slub: Do not dereference NULL pointer in node_match Sebastian Andrzej Siewior
2013-02-13 16:13 ` Sebastian Andrzej Siewior [this message]
2013-02-13 16:13 ` [PATCH 10/12] slub: Use correct cpu_slab on dead cpu Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 11/12] mm: Enable SLUB for RT Sebastian Andrzej Siewior
2013-02-13 16:13 ` [PATCH 12/12] slub: Enable irqs for __GFP_WAIT Sebastian Andrzej Siewior
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1360772031-27186-10-git-send-email-bigeasy@linutronix.de \
--to=bigeasy@linutronix.de \
--cc=C.Emde@osadl.org \
--cc=cl@linux.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=penberg@kernel.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).