From: Vlastimil Babka <vbabka@suse.cz>
To: Mike Galbraith <efault@gmx.de>,
Thomas Gleixner <tglx@linutronix.de>,
LKML <linux-kernel@vger.kernel.org>
Cc: linux-rt-users@vger.kernel.org,
Mel Gorman <mgorman@techsingularity.net>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Subject: Re: [rfc/patch] mm/slub: restore/expand unfreeze_partials() local exclusion scope
Date: Sun, 25 Jul 2021 21:12:49 +0200 [thread overview]
Message-ID: <73f032c2-70f1-77b6-9fd2-9aca52fd5b4d@suse.cz> (raw)
In-Reply-To: <18ca0ce9-3407-61e1-31d6-5c48e80eb5bb@suse.cz>
On 7/25/21 6:27 PM, Vlastimil Babka wrote:
>>> Hope fixing that helps then?
>>
>> Yeah, though RT should perhaps be pinned across release/re-acquire?
>
> Probably not necessary, this_cpu_cmpxchg() will effectively recognize
> being moved to a different CPU.
> Might also move __unfreeze_partials() out of the whole loop to avoid the
> relock. Yeah that should be better.
>
>> Actually, local locks should rediscover the recursion handling skills
>> they long had so such RT specific hole poking isn't necessary. There
>> previously would have been no ifdef+typo there for eyeballs to miss and
>> miss and miss.
>
> Hm, now I'm realizing that local_lock() on !RT is just
> preempt_disable(), i.e. equivalent to get_cpu_ptr(), so some of the
> ifdeffery could go away?
How much will this explode? Thanks.
----8<----
From 99808b198bdf867951131bb9d1ca1bd1cd12b8c4 Mon Sep 17 00:00:00 2001
From: Vlastimil Babka <vbabka@suse.cz>
Date: Fri, 23 Jul 2021 23:17:18 +0200
Subject: [PATCH] PREEMPT_RT+SLUB_CPU_PARTIAL fix attempt
---
mm/slub.c | 34 +++++++++++++++++++++++++++++++---
1 file changed, 31 insertions(+), 3 deletions(-)
diff --git a/mm/slub.c b/mm/slub.c
index 581004a5aca9..d12a50b5ee6f 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2437,13 +2437,19 @@ static void unfreeze_partials(struct kmem_cache *s)
{
struct page *partial_page;
+#ifndef CONFIG_PREEMPT_RT
do {
partial_page = this_cpu_read(s->cpu_slab->partial);
} while (partial_page &&
this_cpu_cmpxchg(s->cpu_slab->partial, partial_page, NULL)
!= partial_page);
-
+#else
+ local_lock(&s->cpu_slab->lock);
+ partial_page = this_cpu_read(s->cpu_slab->partial);
+ this_cpu_write(s->cpu_slab->partial, NULL);
+ local_unlock(&s->cpu_slab->lock);
+#endif
if (partial_page)
__unfreeze_partials(s, partial_page);
}
@@ -2479,10 +2485,15 @@ static void put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
{
#ifdef CONFIG_SLUB_CPU_PARTIAL
struct page *oldpage;
+ struct page *page_to_unfreeze = NULL;
int pages;
int pobjects;
- slub_get_cpu_ptr(s->cpu_slab);
+ /*
+ * On !RT we just want to disable preemption, on RT we need the lock
+ * for real. This happens to match local_lock() semantics.
+ */
+ local_lock(&s->cpu_slab->lock);
do {
pages = 0;
pobjects = 0;
@@ -2496,7 +2507,16 @@ static void put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
* partial array is full. Move the existing
* set to the per node partial list.
*/
+#ifndef CONFIG_PREEMPT_RT
unfreeze_partials(s);
+#else
+ /*
+ * Postpone unfreezing until we drop the local
+ * lock to avoid relocking.
+ */
+ page_to_unfreeze = oldpage;
+#endif
+
oldpage = NULL;
pobjects = 0;
pages = 0;
@@ -2511,9 +2531,17 @@ static void put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
page->pobjects = pobjects;
page->next = oldpage;
+#ifndef CONFIG_PREEMPT_RT
} while (this_cpu_cmpxchg(s->cpu_slab->partial, oldpage, page)
!= oldpage);
- slub_put_cpu_ptr(s->cpu_slab);
+#else
+ this_cpu_write(s->cpu_slab->partial, page);
+ } while (false);
+#endif
+
+ local_unlock(&s->cpu_slab->lock);
+ if (page_to_unfreeze)
+ __unfreeze_partials(s, page_to_unfreeze);
#endif /* CONFIG_SLUB_CPU_PARTIAL */
}
--
2.32.0
next prev parent reply other threads:[~2021-07-25 19:13 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-08 9:42 [ANNOUNCE] v5.13-rt1 Thomas Gleixner
2021-07-09 5:20 ` [patch] mm/slub: Fix kmem_cache_alloc_bulk() error path Mike Galbraith
2021-07-09 5:20 ` [patch] mm/slub: Replace do_slab_free() local_lock_irqsave/restore() calls in PREEMPT_RT scope Mike Galbraith
2021-07-09 5:21 ` [rfc/patch] mm/slub: restore/expand unfreeze_partials() local exclusion scope Mike Galbraith
2021-07-09 5:25 ` Mike Galbraith
2021-07-09 19:28 ` Thomas Gleixner
2021-07-10 1:12 ` Mike Galbraith
2021-07-15 16:34 ` Mike Galbraith
2021-07-17 14:58 ` [patch] v2 " Mike Galbraith
2021-07-18 7:58 ` Vlastimil Babka
2021-07-18 8:11 ` Mike Galbraith
2021-07-18 15:43 ` Mike Galbraith
2021-07-18 21:19 ` Vlastimil Babka
2021-07-19 4:01 ` Mike Galbraith
2021-07-19 13:15 ` Mike Galbraith
2021-07-20 2:46 ` kernel test robot
2021-07-20 8:56 ` [rfc/patch] " Vlastimil Babka
2021-07-20 11:26 ` Mike Galbraith
2021-07-21 4:56 ` Mike Galbraith
2021-07-21 8:44 ` Vlastimil Babka
2021-07-21 9:33 ` Mike Galbraith
2021-07-23 22:39 ` Vlastimil Babka
2021-07-24 2:25 ` Mike Galbraith
2021-07-25 14:09 ` Mike Galbraith
2021-07-25 14:16 ` Vlastimil Babka
2021-07-25 15:02 ` Mike Galbraith
2021-07-25 16:27 ` Vlastimil Babka
2021-07-25 19:12 ` Vlastimil Babka [this message]
2021-07-25 19:34 ` Vlastimil Babka
2021-07-26 10:04 ` Mike Galbraith
2021-07-26 17:00 ` Mike Galbraith
2021-07-26 21:26 ` Vlastimil Babka
2021-07-27 4:09 ` Mike Galbraith
2021-07-28 16:59 ` Vlastimil Babka
2021-07-29 4:51 ` Mike Galbraith
2021-07-29 9:51 ` Vlastimil Babka
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=73f032c2-70f1-77b6-9fd2-9aca52fd5b4d@suse.cz \
--to=vbabka@suse.cz \
--cc=bigeasy@linutronix.de \
--cc=efault@gmx.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).