From: Sergey Senozhatsky <senozhatsky@chromium.org>
To: Minchan Kim <minchan@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>
Cc: Yosry Ahmed <yosryahmed@google.com>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Sergey Senozhatsky <senozhatsky@chromium.org>
Subject: [PATCHv2] zsmalloc: allow only one active pool compaction context
Date: Mon, 17 Apr 2023 22:54:20 +0900 [thread overview]
Message-ID: <20230417135420.1836741-1-senozhatsky@chromium.org> (raw)
zsmalloc pool can be compacted concurrently by many contexts,
e.g.
cc1 handle_mm_fault()
do_anonymous_page()
__alloc_pages_slowpath()
try_to_free_pages()
do_try_to_free_pages(
lru_gen_shrink_node()
shrink_slab()
do_shrink_slab()
zs_shrinker_scan()
zs_compact()
This creates unnecessary contention as all those processes
compete for access to the same classes. A single compaction
process is enough. Moreover contention that is created by
multiple compaction processes impact other zsmalloc functions,
e.g. zs_malloc(), since zsmalloc uses "global" pool->lock to
synchronize access to pool.
Introduce pool compaction mutex and permit only one compaction
context at a time. This reduces overall pool->lock contention.
/proc/lock-stat after make -j$((`nproc`+1)) linux kernel for
&pool->lock#3:
Base Patched
------------------------------------------
con-bounces 2035730 1540066
contentions 2343871 1774348
waittime-min 0.10 0.10
waittime-max 4004216.24 2745.22
waittime-total 101334168.29 67865414.91
waittime-avg 43.23 38.25
acq-bounces 2895765 2186745
acquisitions 6247686 5136943
holdtime-min 0.07 0.07
holdtime-max 2605507.97 482439.16
holdtime-total 9998599.59 5107151.01
holdtime-avg 1.60 0.99
Test run time:
Base
2775.15user 1709.13system 2:13.82elapsed 3350%CPU
Patched
2608.25user 1439.03system 2:03.63elapsed 3273%CPU
Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org>
---
mm/zsmalloc.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index cc81dfba05a0..dfec2fc6a30f 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -264,6 +264,7 @@ struct zs_pool {
struct work_struct free_work;
#endif
spinlock_t lock;
+ atomic_t compaction_in_progress;
};
struct zspage {
@@ -2274,6 +2275,9 @@ unsigned long zs_compact(struct zs_pool *pool)
struct size_class *class;
unsigned long pages_freed = 0;
+ if (atomic_xchg(&pool->compaction_in_progress, 1))
+ return 0;
+
for (i = ZS_SIZE_CLASSES - 1; i >= 0; i--) {
class = pool->size_class[i];
if (class->index != i)
@@ -2281,6 +2285,7 @@ unsigned long zs_compact(struct zs_pool *pool)
pages_freed += __zs_compact(pool, class);
}
atomic_long_add(pages_freed, &pool->stats.pages_compacted);
+ atomic_set(&pool->compaction_in_progress, 0);
return pages_freed;
}
@@ -2388,6 +2393,7 @@ struct zs_pool *zs_create_pool(const char *name)
init_deferred_free(pool);
spin_lock_init(&pool->lock);
+ atomic_set(&pool->compaction_in_progress, 0);
pool->name = kstrdup(name, GFP_KERNEL);
if (!pool->name)
--
2.40.0.634.g4ca3ef3211-goog
next reply other threads:[~2023-04-17 13:54 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-17 13:54 Sergey Senozhatsky [this message]
2023-04-17 18:32 ` [PATCHv2] zsmalloc: allow only one active pool compaction context Yosry Ahmed
2023-04-17 23:58 ` Sergey Senozhatsky
2023-04-18 0:41 ` Andrew Morton
2023-04-18 2:53 ` Yosry Ahmed
2023-04-18 11:24 ` Sergey Senozhatsky
2023-04-18 19:37 ` Yosry Ahmed
2023-04-18 3:05 ` Sergey Senozhatsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230417135420.1836741-1-senozhatsky@chromium.org \
--to=senozhatsky@chromium.org \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan@kernel.org \
--cc=yosryahmed@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.