linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Punit Agrawal <punit.agrawal@bytedance.com>
To: Suren Baghdasaryan <surenb@google.com>
Cc: akpm@linux-foundation.org, michel@lespinasse.org,
	jglisse@google.com, mhocko@suse.com, vbabka@suse.cz,
	hannes@cmpxchg.org, mgorman@techsingularity.net,
	dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com,
	peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org,
	mingo@redhat.com, will@kernel.org, luto@kernel.org,
	songliubraving@fb.com, peterx@redhat.com, david@redhat.com,
	dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de,
	kent.overstreet@linux.dev, punit.agrawal@bytedance.com,
	lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com,
	axelrasmussen@google.com, joelaf@google.com, minchan@google.com,
	rppt@kernel.org, jannh@google.com, shakeelb@google.com,
	tatashin@google.com, edumazet@google.com, gthelen@google.com,
	gurua@google.com, arjunroy@google.com, soheil@google.com,
	leewalsh@google.com, posk@google.com, linux-mm@kvack.org,
	linux-arm-kernel@lists.infradead.org,
	linuxppc-dev@lists.ozlabs.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, kernel-team@android.com
Subject: Re: [External] [PATCH v2 00/33] Per-VMA locks
Date: Wed, 15 Feb 2023 17:32:58 +0000	[thread overview]
Message-ID: <87r0uqq1f9.fsf@stealth> (raw)
In-Reply-To: <20230127194110.533103-1-surenb@google.com> (Suren Baghdasaryan's message of "Fri, 27 Jan 2023 11:40:37 -0800")

Suren Baghdasaryan <surenb@google.com> writes:

> Previous version:
> v1: https://lore.kernel.org/all/20230109205336.3665937-1-surenb@google.com/
> RFC: https://lore.kernel.org/all/20220901173516.702122-1-surenb@google.com/
>
> LWN article describing the feature:
> https://lwn.net/Articles/906852/
>
> Per-vma locks idea that was discussed during SPF [1] discussion at LSF/MM
> last year [2], which concluded with suggestion that “a reader/writer
> semaphore could be put into the VMA itself; that would have the effect of
> using the VMA as a sort of range lock. There would still be contention at
> the VMA level, but it would be an improvement.” This patchset implements
> this suggested approach.

I took the patches for a spin on a 2-socket 32 core (64 threads) system
with Intel 8336C (Ice Lake) and 512GB of RAM.

For the initial testing, "pft-threads" from the mm-tests suite[0] was
used. The test mmaps a memory region (~100GB on the test system) and
triggers access by a number of threads executing in parallel. For each
degree of parallelism, the test is repeated 10 times to get a better
feel for the behaviour. Below is an excerpt of the harmonic mean
reported by 'compare_kernel' script[1] included with mm-tests.

The first column is results for mm-unstable as of 2023-02-10, the second
column is the patches posted here while the third column includes
optimizations to reclaim some of the observed regression.

From the results, there is a drop in page fault/second for low number of
CPUs but good improvement with higher CPUs.

                                        6.2.0-rc4                6.2.0-rc4                6.2.0-rc4
                             mm-unstable-20230210                   pvl-v2               pvl-v2+opt

Hmean     faults/cpu-1     898792.9338 (   0.00%)   894597.0474 *  -0.47%*   895933.2782 *  -0.32%*
Hmean     faults/cpu-4     751903.9803 (   0.00%)   677764.2975 *  -9.86%*   688643.8163 *  -8.41%*
Hmean     faults/cpu-7     612275.5663 (   0.00%)   565363.4137 *  -7.66%*   597538.9396 *  -2.41%*
Hmean     faults/cpu-12    434460.9074 (   0.00%)   410974.2708 *  -5.41%*   452501.4290 *   4.15%*
Hmean     faults/cpu-21    291475.5165 (   0.00%)   293936.8460 (   0.84%)   308712.2434 *   5.91%*
Hmean     faults/cpu-30    218021.3980 (   0.00%)   228265.0559 *   4.70%*   241897.5225 *  10.95%*
Hmean     faults/cpu-48    141798.5030 (   0.00%)   162322.5972 *  14.47%*   166081.9459 *  17.13%*
Hmean     faults/cpu-79     90060.9577 (   0.00%)   107028.7779 *  18.84%*   109810.4488 *  21.93%*
Hmean     faults/cpu-110    64729.3561 (   0.00%)    80597.7246 *  24.51%*    83134.0679 *  28.43%*
Hmean     faults/cpu-128    55740.1334 (   0.00%)    68395.4426 *  22.70%*    69248.2836 *  24.23%*

Hmean     faults/sec-1     898781.7694 (   0.00%)   894247.3174 *  -0.50%*   894440.3118 *  -0.48%*
Hmean     faults/sec-4    2965588.9697 (   0.00%)  2683651.5664 *  -9.51%*  2726450.9710 *  -8.06%*
Hmean     faults/sec-7    4144512.3996 (   0.00%)  3891644.2128 *  -6.10%*  4099918.8601 (  -1.08%)
Hmean     faults/sec-12   4969513.6934 (   0.00%)  4829731.4355 *  -2.81%*  5264682.7371 *   5.94%*
Hmean     faults/sec-21   5814379.4789 (   0.00%)  5941405.3116 *   2.18%*  6263716.3903 *   7.73%*
Hmean     faults/sec-30   6153685.3709 (   0.00%)  6489311.6634 *   5.45%*  6910843.5858 *  12.30%*
Hmean     faults/sec-48   6197953.1327 (   0.00%)  7216320.7727 *  16.43%*  7412782.2927 *  19.60%*
Hmean     faults/sec-79   6167135.3738 (   0.00%)  7425927.1022 *  20.41%*  7637042.2198 *  23.83%*
Hmean     faults/sec-110  6264768.2247 (   0.00%)  7813329.3863 *  24.72%*  7984344.4005 *  27.45%*
Hmean     faults/sec-128  6460727.8216 (   0.00%)  7875664.8999 *  21.90%*  8049910.3601 *  24.60%*

[0] https://github.com/gormanm/mmtests
[1] https://github.com/gormanm/mmtests/blob/master/compare-kernels.sh

  parent reply	other threads:[~2023-02-15 17:33 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-27 19:40 [PATCH v2 00/33] Per-VMA locks Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 01/33] maple_tree: Be more cautious about dead nodes Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 02/33] maple_tree: Detect dead nodes in mas_start() Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 03/33] maple_tree: Fix freeing of nodes in rcu mode Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 04/33] maple_tree: remove extra smp_wmb() from mas_dead_leaves() Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 05/33] maple_tree: Fix write memory barrier of nodes once dead for RCU mode Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 06/33] maple_tree: Add smp_rmb() to dead node detection Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 07/33] mm: Enable maple tree RCU mode by default Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 08/33] mm: introduce CONFIG_PER_VMA_LOCK Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 09/33] mm: rcu safe VMA freeing Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 10/33] mm: move mmap_lock assert function definitions Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 11/33] mm: add per-VMA lock and helper functions to control it Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 12/33] mm: mark VMA as being written when changing vm_flags Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 13/33] mm/mmap: move VMA locking before vma_adjust_trans_huge call Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 14/33] mm/khugepaged: write-lock VMA while collapsing a huge page Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 15/33] mm/mmap: write-lock VMAs before merging, splitting or expanding them Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 16/33] mm/mmap: write-lock VMA before shrinking or expanding it Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 17/33] mm/mremap: write-lock VMA while remapping it to a new address range Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 18/33] mm: write-lock VMAs before removing them from VMA tree Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 19/33] mm: conditionally write-lock VMA in free_pgtables Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 20/33] mm/mmap: write-lock adjacent VMAs if they can grow into unmapped area Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 21/33] kernel/fork: assert no VMA readers during its destruction Suren Baghdasaryan
2023-01-27 19:40 ` [PATCH v2 22/33] mm/mmap: prevent pagefault handler from racing with mmu_notifier registration Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 23/33] mm: introduce lock_vma_under_rcu to be used from arch-specific code Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 24/33] mm: fall back to mmap_lock if vma->anon_vma is not yet set Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 25/33] mm: add FAULT_FLAG_VMA_LOCK flag Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 26/33] mm: prevent do_swap_page from handling page faults under VMA lock Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 27/33] mm: prevent userfaults to be handled under per-vma lock Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 28/33] mm: introduce per-VMA lock statistics Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 29/33] x86/mm: try VMA lock-based page fault handling first Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 30/33] arm64/mm: " Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 31/33] powerc/mm: " Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 32/33] mm/mmap: free vm_area_struct without call_rcu in exit_mmap Suren Baghdasaryan
2023-01-27 19:41 ` [PATCH v2 33/33] mm: separate vma->lock from vm_area_struct Suren Baghdasaryan
2023-01-27 22:51 ` [PATCH v2 00/33] Per-VMA locks Andrew Morton
2023-01-27 23:26   ` Matthew Wilcox
2023-01-28  0:00     ` Suren Baghdasaryan
2023-02-14 16:47       ` Suren Baghdasaryan
2023-02-15 17:32 ` Punit Agrawal [this message]
2023-02-15 17:39   ` [External] " Suren Baghdasaryan
2023-02-28 12:06   ` Punit Agrawal
2023-02-28 18:08     ` Suren Baghdasaryan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87r0uqq1f9.fsf@stealth \
    --to=punit.agrawal@bytedance.com \
    --cc=akpm@linux-foundation.org \
    --cc=arjunroy@google.com \
    --cc=axelrasmussen@google.com \
    --cc=bigeasy@linutronix.de \
    --cc=dave@stgolabs.net \
    --cc=david@redhat.com \
    --cc=dhowells@redhat.com \
    --cc=edumazet@google.com \
    --cc=gthelen@google.com \
    --cc=gurua@google.com \
    --cc=hannes@cmpxchg.org \
    --cc=hughd@google.com \
    --cc=jannh@google.com \
    --cc=jglisse@google.com \
    --cc=joelaf@google.com \
    --cc=kent.overstreet@linux.dev \
    --cc=kernel-team@android.com \
    --cc=ldufour@linux.ibm.com \
    --cc=leewalsh@google.com \
    --cc=liam.howlett@oracle.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=lstoakes@gmail.com \
    --cc=luto@kernel.org \
    --cc=mgorman@techsingularity.net \
    --cc=mhocko@suse.com \
    --cc=michel@lespinasse.org \
    --cc=minchan@google.com \
    --cc=mingo@redhat.com \
    --cc=paulmck@kernel.org \
    --cc=peterjung1337@gmail.com \
    --cc=peterx@redhat.com \
    --cc=peterz@infradead.org \
    --cc=posk@google.com \
    --cc=rientjes@google.com \
    --cc=rppt@kernel.org \
    --cc=shakeelb@google.com \
    --cc=soheil@google.com \
    --cc=songliubraving@fb.com \
    --cc=surenb@google.com \
    --cc=tatashin@google.com \
    --cc=vbabka@suse.cz \
    --cc=will@kernel.org \
    --cc=willy@infradead.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).