Re: [LSF/MM/BPF TOPIC] Swap Abstraction / Native Zswap

From: Chris Li <chrisl@kernel.org>
To: "Huang, Ying" <ying.huang@intel.com>
Cc: Yosry Ahmed <yosryahmed@google.com>,
	lsf-pc@lists.linux-foundation.org,
	Johannes Weiner <hannes@cmpxchg.org>,
	Linux-MM <linux-mm@kvack.org>, Michal Hocko <mhocko@kernel.org>,
	Shakeel Butt <shakeelb@google.com>,
	David Rientjes <rientjes@google.com>,
	Hugh Dickins <hughd@google.com>,
	Seth Jennings <sjenning@redhat.com>,
	Dan Streetman <ddstreet@ieee.org>,
	Vitaly Wool <vitaly.wool@konsulko.com>,
	Yang Shi <shy828301@gmail.com>, Peter Xu <peterx@redhat.com>,
	Minchan Kim <minchan@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Aneesh Kumar K V <aneesh.kumar@linux.ibm.com>,
	Michal Hocko <mhocko@suse.com>, Wei Xu <weixugc@google.com>
Subject: Re: [LSF/MM/BPF TOPIC] Swap Abstraction / Native Zswap
Date: Fri, 10 Mar 2023 15:14:23 -0800	[thread overview]
Message-ID: <ZAu5z7j/oxnnR4/J@google.com> (raw)
In-Reply-To: <87y1o571aa.fsf@yhuang6-desk2.ccr.corp.intel.com>

On Fri, Mar 10, 2023 at 11:06:37AM +0800, Huang, Ying wrote:
> > Unfortunately it's a little bit more. 24 is the extra overhead.
> >
> > Today we have an xarray entry for each swapped out page, that either
> > has the swapcache pointer or the shadow entry.
> >
> > With this implementation, we have an xarray entry for each swapped out
> > page, that has a pointer to the swap_desc.
> >
> > Ignoring the overhead of the xarray itself, we have (8 + 24) / (8 + 1) = 3.5556.
> 
> OK.  I see.  We can only hold 8 bytes for each xarray entry.  To save
> memory usage, we can allocate multiple swap_desc (e.g., 16) for each
> xarray entry.  Then the memory usage of xarray becomes 1/N.

The xarray look up key is the swap offset from the swap entry. If you
put more than one swap_desc under the one xarray entry. It will mean
all those different swap_descs will share a swap offset.

> >> > Yeah, I initially thought we would only need the swp_entry_t ->
> >> > swap_desc reverse mapping for readahead, and that we can only store
> >> > that for spinning disks, but I was wrong. We need for other things as
> >> > well today: swapoff, when trying to find an empty swap slot and we
> >> > start trying to free swap slots used only by the swapcache. However, I
> >> > think both of these cases can be fixed (I can share more details if
> >> > you want). If everything goes well we should only need to maintain the
> >> > reverse mapping (extra overhead above 24 bytes) for swap files on
> >> > spinning disks for readahead.
> >> >
> >> >>
> >> >> Looking forward to your discussion.
> 
> Per my understanding, the indirection is to make it easy to move
> (swapped) pages among swap devices based on hot/cold.  This is similar
> as the target of memory tiering.  It appears that we can extend the
> memory tiering (mm/memory-tiers.c) framework to cover swap devices too?
> Is it possible for zswap to be faster than some slow memory media?

I just took a look at the mm/memory-tier.c. It does not look like it
can cover block device swap without major overhauling.

Chris