From: Sergey Senozhatsky <senozhatsky@chromium.org>
To: Nhat Pham <nphamcs@gmail.com>
Cc: akpm@linux-foundation.org, hannes@cmpxchg.org,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org,
sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com
Subject: Re: [PATCH v7 3/6] zsmalloc: Consolidate zs_pool's migrate_lock and size_class's locks
Date: Tue, 29 Nov 2022 13:01:15 +0900 [thread overview]
Message-ID: <Y4WECyoq3fP/cOmi@google.com> (raw)
In-Reply-To: <20221128191616.1261026-4-nphamcs@gmail.com>
On (22/11/28 11:16), Nhat Pham wrote:
> Currently, zsmalloc has a hierarchy of locks, which includes a
> pool-level migrate_lock, and a lock for each size class. We have to
> obtain both locks in the hotpath in most cases anyway, except for
> zs_malloc. This exception will no longer exist when we introduce a LRU
> into the zs_pool for the new writeback functionality - we will need to
> obtain a pool-level lock to synchronize LRU handling even in zs_malloc.
>
> In preparation for zsmalloc writeback, consolidate these locks into a
> single pool-level lock, which drastically reduces the complexity of
> synchronization in zsmalloc.
>
> We have also benchmarked the lock consolidation to see the performance
> effect of this change on zram.
>
> First, we ran a synthetic FS workload on a server machine with 36 cores
> (same machine for all runs), using
>
> fs_mark -d ../zram1mnt -s 100000 -n 2500 -t 32 -k
>
> before and after for btrfs and ext4 on zram (FS usage is 80%).
>
> Here is the result (unit is file/second):
>
> With lock consolidation (btrfs):
> Average: 13520.2, Median: 13531.0, Stddev: 137.5961482019028
>
> Without lock consolidation (btrfs):
> Average: 13487.2, Median: 13575.0, Stddev: 309.08283679298665
>
> With lock consolidation (ext4):
> Average: 16824.4, Median: 16839.0, Stddev: 89.97388510006668
>
> Without lock consolidation (ext4)
> Average: 16958.0, Median: 16986.0, Stddev: 194.7370021336469
>
> As you can see, we observe a 0.3% regression for btrfs, and a 0.9%
> regression for ext4. This is a small, barely measurable difference in my
> opinion.
>
> For a more realistic scenario, we also tries building the kernel on zram.
> Here is the time it takes (in seconds):
>
> With lock consolidation (btrfs):
> real
> Average: 319.6, Median: 320.0, Stddev: 0.8944271909999159
> user
> Average: 6894.2, Median: 6895.0, Stddev: 25.528415540334656
> sys
> Average: 521.4, Median: 522.0, Stddev: 1.51657508881031
>
> Without lock consolidation (btrfs):
> real
> Average: 319.8, Median: 320.0, Stddev: 0.8366600265340756
> user
> Average: 6896.6, Median: 6899.0, Stddev: 16.04057355583023
> sys
> Average: 520.6, Median: 521.0, Stddev: 1.140175425099138
>
> With lock consolidation (ext4):
> real
> Average: 320.0, Median: 319.0, Stddev: 1.4142135623730951
> user
> Average: 6896.8, Median: 6878.0, Stddev: 28.621670111997307
> sys
> Average: 521.2, Median: 521.0, Stddev: 1.7888543819998317
>
> Without lock consolidation (ext4)
> real
> Average: 319.6, Median: 319.0, Stddev: 0.8944271909999159
> user
> Average: 6886.2, Median: 6887.0, Stddev: 16.93221781102523
> sys
> Average: 520.4, Median: 520.0, Stddev: 1.140175425099138
>
> The difference is entirely within the noise of a typical run on zram. This
> hardly justifies the complexity of maintaining both the pool lock and
> the class lock. In fact, for writeback, we would need to introduce yet
> another lock to prevent data races on the pool's LRU, further
> complicating the lock handling logic. IMHO, it is just better to
> collapse all of these into a single pool-level lock.
>
> Suggested-by: Johannes Weiner <hannes@cmpxchg.org>
> Signed-off-by: Nhat Pham <nphamcs@gmail.com>
> Acked-by: Minchan Kim <minchan@kernel.org>
> Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Sergey Senozhatsky <senozhatsky@chromium.org>
next prev parent reply other threads:[~2022-11-29 4:01 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-28 19:16 [PATCH v7 0/6] Implement writeback for zsmalloc Nhat Pham
2022-11-28 19:16 ` [PATCH v7 1/6] zswap: fix writeback lock ordering " Nhat Pham
2022-11-29 3:47 ` Sergey Senozhatsky
2022-11-28 19:16 ` [PATCH v7 2/6] zpool: clean out dead code Nhat Pham
2022-11-28 19:16 ` [PATCH v7 3/6] zsmalloc: Consolidate zs_pool's migrate_lock and size_class's locks Nhat Pham
2022-11-29 4:01 ` Sergey Senozhatsky [this message]
2022-11-28 19:16 ` [PATCH v7 4/6] zsmalloc: Add a LRU to zs_pool to keep track of zspages in LRU order Nhat Pham
2022-11-28 19:25 ` Johannes Weiner
2022-11-29 3:50 ` Sergey Senozhatsky
2022-11-29 11:53 ` Vitaly Wool
2022-11-29 14:03 ` Sergey Senozhatsky
2022-11-29 15:54 ` Johannes Weiner
2022-11-30 15:23 ` Vitaly Wool
2022-11-28 19:16 ` [PATCH v7 5/6] zsmalloc: Add zpool_ops field to zs_pool to store evict handlers Nhat Pham
2022-11-28 19:27 ` Johannes Weiner
2022-11-29 3:53 ` Sergey Senozhatsky
2022-11-28 19:16 ` [PATCH v7 6/6] zsmalloc: Implement writeback mechanism for zsmalloc Nhat Pham
2022-11-28 19:28 ` Johannes Weiner
2022-11-29 3:58 ` Sergey Senozhatsky
2023-01-03 4:57 ` [PATCH v7 0/6] Implement writeback " Thomas Weißschuh
2023-01-06 20:32 ` Nhat Pham
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y4WECyoq3fP/cOmi@google.com \
--to=senozhatsky@chromium.org \
--cc=akpm@linux-foundation.org \
--cc=ddstreet@ieee.org \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan@kernel.org \
--cc=ngupta@vflare.org \
--cc=nphamcs@gmail.com \
--cc=sjenning@redhat.com \
--cc=vitaly.wool@konsulko.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).