From: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
To: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>
Cc: LKML <linux-kernel@vger.kernel.org>, Baoquan He <bhe@redhat.com>,
Lorenzo Stoakes <lstoakes@gmail.com>,
Christoph Hellwig <hch@infradead.org>,
Matthew Wilcox <willy@infradead.org>,
"Liam R . Howlett" <Liam.Howlett@oracle.com>,
Dave Chinner <david@fromorbit.com>,
"Paul E . McKenney" <paulmck@kernel.org>,
Joel Fernandes <joel@joelfernandes.org>,
Uladzislau Rezki <urezki@gmail.com>,
Oleksiy Avramchenko <oleksiy.avramchenko@sony.com>
Subject: [PATCH v2 0/9] Mitigate a vmap lock contention v2
Date: Tue, 29 Aug 2023 10:11:33 +0200 [thread overview]
Message-ID: <20230829081142.3619-1-urezki@gmail.com> (raw)
Hello, folk!
This is the v2, the series which tends to minimize the vmap
lock contention. It is based on the tag: v6.5-rc6. Here you
can find a documentation about it:
wget ftp://vps418301.ovh.net/incoming/Fix_a_vmalloc_lock_contention_in_SMP_env_v2.pdf
even though it is a bit outdated(it follows v1), it still gives a
good overview on the problem and how it can be solved. On demand
and by request i can update it.
The v1 is here: https://lore.kernel.org/linux-mm/ZIAqojPKjChJTssg@pc636/T/
Delta v1 -> v2:
- open coded locking;
- switch to array of nodes instead of per-cpu definition;
- density is 2 cores per one node(not equal to number of CPUs);
- VAs first go back(free path) to an owner node and later to
a global heap if a block is fully freed, nid is saved in va->flags;
- add helpers to drain lazily-freed areas faster, if high pressure;
- picked al Reviewed-by.
Test on AMD Ryzen Threadripper 3970X 32-Core Processor:
sudo ./test_vmalloc.sh run_test_mask=127 nr_threads=64
<v6.5-rc6 perf>
94.17% 0.90% [kernel] [k] _raw_spin_lock
93.27% 93.05% [kernel] [k] native_queued_spin_lock_slowpath
74.69% 0.25% [kernel] [k] __vmalloc_node_range
72.64% 0.01% [kernel] [k] __get_vm_area_node
72.04% 0.89% [kernel] [k] alloc_vmap_area
42.17% 0.00% [kernel] [k] vmalloc
32.53% 0.00% [kernel] [k] __vmalloc_node
24.91% 0.25% [kernel] [k] vfree
24.32% 0.01% [kernel] [k] remove_vm_area
22.63% 0.21% [kernel] [k] find_unlink_vmap_area
15.51% 0.00% [unknown] [k] 0xffffffffc09a74ac
14.35% 0.00% [kernel] [k] ret_from_fork_asm
14.35% 0.00% [kernel] [k] ret_from_fork
14.35% 0.00% [kernel] [k] kthread
<v6.5-rc6 perf>
vs
<v6.5-rc6+v2 perf>
74.32% 2.42% [kernel] [k] __vmalloc_node_range
69.58% 0.01% [kernel] [k] vmalloc
54.21% 1.17% [kernel] [k] __alloc_pages_bulk
48.13% 47.91% [kernel] [k] clear_page_orig
43.60% 0.01% [unknown] [k] 0xffffffffc082f16f
32.06% 0.00% [kernel] [k] ret_from_fork_asm
32.06% 0.00% [kernel] [k] ret_from_fork
32.06% 0.00% [kernel] [k] kthread
31.30% 0.00% [unknown] [k] 0xffffffffc082f889
22.98% 4.16% [kernel] [k] vfree
14.36% 0.28% [kernel] [k] __get_vm_area_node
13.43% 3.35% [kernel] [k] alloc_vmap_area
10.86% 0.04% [kernel] [k] remove_vm_area
8.89% 2.75% [kernel] [k] _raw_spin_lock
7.19% 0.00% [unknown] [k] 0xffffffffc082fba3
6.65% 1.37% [kernel] [k] free_unref_page
6.13% 6.11% [kernel] [k] native_queued_spin_lock_slowpath
<v6.5-rc6+v2 perf>
On smaller systems, for example, 8xCPU Hikey960 board the
contention is not that high and is approximately ~16 percent.
Uladzislau Rezki (Sony) (9):
mm: vmalloc: Add va_alloc() helper
mm: vmalloc: Rename adjust_va_to_fit_type() function
mm: vmalloc: Move vmap_init_free_space() down in vmalloc.c
mm: vmalloc: Remove global vmap_area_root rb-tree
mm: vmalloc: Remove global purge_vmap_area_root rb-tree
mm: vmalloc: Offload free_vmap_area_lock lock
mm: vmalloc: Support multiple nodes in vread_iter
mm: vmalloc: Support multiple nodes in vmallocinfo
mm: vmalloc: Set nr_nodes/node_size based on CPU-cores
mm/vmalloc.c | 929 +++++++++++++++++++++++++++++++++++++--------------
1 file changed, 683 insertions(+), 246 deletions(-)
--
2.30.2
next reply other threads:[~2023-08-29 8:12 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-29 8:11 Uladzislau Rezki (Sony) [this message]
2023-08-29 8:11 ` [PATCH v2 1/9] mm: vmalloc: Add va_alloc() helper Uladzislau Rezki (Sony)
2023-09-06 5:51 ` Baoquan He
2023-09-06 15:06 ` Uladzislau Rezki
2023-08-29 8:11 ` [PATCH v2 2/9] mm: vmalloc: Rename adjust_va_to_fit_type() function Uladzislau Rezki (Sony)
2023-09-06 5:51 ` Baoquan He
2023-09-06 16:27 ` Uladzislau Rezki
2023-08-29 8:11 ` [PATCH v2 3/9] mm: vmalloc: Move vmap_init_free_space() down in vmalloc.c Uladzislau Rezki (Sony)
2023-09-06 5:52 ` Baoquan He
2023-09-06 16:29 ` Uladzislau Rezki
2023-08-29 8:11 ` [PATCH v2 4/9] mm: vmalloc: Remove global vmap_area_root rb-tree Uladzislau Rezki (Sony)
2023-08-29 14:30 ` kernel test robot
2023-08-30 14:48 ` Uladzislau Rezki
2023-09-07 2:17 ` Baoquan He
2023-09-07 2:17 ` Baoquan He
2023-09-07 9:38 ` Baoquan He
2023-09-07 9:38 ` Baoquan He
2023-09-07 9:40 ` Uladzislau Rezki
2023-09-07 9:40 ` Uladzislau Rezki
2023-09-07 9:39 ` Uladzislau Rezki
2023-09-07 9:39 ` Uladzislau Rezki
2023-09-07 9:58 ` Baoquan He
2023-09-07 9:58 ` Baoquan He
2023-09-08 1:51 ` HAGIO KAZUHITO(萩尾 一仁)
2023-09-08 1:51 ` HAGIO KAZUHITO(萩尾 一仁)
2023-09-08 4:43 ` Baoquan He
2023-09-08 4:43 ` Baoquan He
2023-09-08 5:01 ` HAGIO KAZUHITO(萩尾 一仁)
2023-09-08 5:01 ` HAGIO KAZUHITO(萩尾 一仁)
2023-09-08 6:44 ` Baoquan He
2023-09-08 6:44 ` Baoquan He
2023-09-08 11:25 ` Uladzislau Rezki
2023-09-08 11:25 ` Uladzislau Rezki
2023-09-08 11:38 ` Baoquan He
2023-09-08 11:38 ` Baoquan He
2023-09-08 13:23 ` Uladzislau Rezki
2023-09-08 13:23 ` Uladzislau Rezki
2023-09-11 2:38 ` Baoquan He
2023-09-11 16:53 ` Uladzislau Rezki
2023-09-12 13:19 ` Baoquan He
2023-08-29 8:11 ` [PATCH v2 5/9] mm: vmalloc: Remove global purge_vmap_area_root rb-tree Uladzislau Rezki (Sony)
2023-09-11 2:57 ` Baoquan He
2023-09-11 17:00 ` Uladzislau Rezki
2023-08-29 8:11 ` [PATCH v2 6/9] mm: vmalloc: Offload free_vmap_area_lock lock Uladzislau Rezki (Sony)
2023-09-06 6:04 ` Baoquan He
2023-09-06 19:16 ` Uladzislau Rezki
2023-09-07 0:06 ` Baoquan He
2023-09-07 9:33 ` Uladzislau Rezki
2023-09-11 3:25 ` Baoquan He
2023-09-11 17:10 ` Uladzislau Rezki
2023-09-12 13:21 ` Baoquan He
2023-08-29 8:11 ` [PATCH v2 7/9] mm: vmalloc: Support multiple nodes in vread_iter Uladzislau Rezki (Sony)
2023-09-11 3:58 ` Baoquan He
2023-09-11 18:16 ` Uladzislau Rezki
2023-09-12 13:42 ` Baoquan He
2023-09-13 15:42 ` Uladzislau Rezki
2023-09-14 3:02 ` Baoquan He
2023-09-14 3:36 ` Baoquan He
2023-09-14 3:38 ` Baoquan He
2023-09-13 10:59 ` Baoquan He
2023-09-13 15:38 ` Uladzislau Rezki
2023-08-29 8:11 ` [PATCH v2 8/9] mm: vmalloc: Support multiple nodes in vmallocinfo Uladzislau Rezki (Sony)
2023-09-15 13:02 ` Baoquan He
2023-09-15 18:32 ` Uladzislau Rezki
2023-08-29 8:11 ` [PATCH v2 9/9] mm: vmalloc: Set nr_nodes/node_size based on CPU-cores Uladzislau Rezki (Sony)
2023-09-15 13:03 ` Baoquan He
2023-09-15 18:31 ` Uladzislau Rezki
2023-08-31 1:15 ` [PATCH v2 0/9] Mitigate a vmap lock contention v2 Baoquan He
2023-08-31 16:26 ` Uladzislau Rezki
2023-09-04 14:55 ` Uladzislau Rezki
2023-09-04 19:53 ` Andrew Morton
2023-09-05 6:53 ` Uladzislau Rezki
2023-09-06 20:04 ` Lorenzo Stoakes
2023-09-07 9:15 ` Uladzislau Rezki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230829081142.3619-1-urezki@gmail.com \
--to=urezki@gmail.com \
--cc=Liam.Howlett@oracle.com \
--cc=akpm@linux-foundation.org \
--cc=bhe@redhat.com \
--cc=david@fromorbit.com \
--cc=hch@infradead.org \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lstoakes@gmail.com \
--cc=oleksiy.avramchenko@sony.com \
--cc=paulmck@kernel.org \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.