Re: [LSF/MM/BPF TOPIC] Swap Abstraction / Native Zswap

From: "Huang, Ying" <ying.huang@intel.com>
To: Yosry Ahmed <yosryahmed@google.com>
Cc: Chris Li <chrisl@kernel.org>,
	 lsf-pc@lists.linux-foundation.org,
	Johannes Weiner <hannes@cmpxchg.org>,
	 Linux-MM <linux-mm@kvack.org>, Michal Hocko <mhocko@kernel.org>,
	 Shakeel Butt <shakeelb@google.com>,
	David Rientjes <rientjes@google.com>,
	 Hugh Dickins <hughd@google.com>,
	Seth Jennings <sjenning@redhat.com>,
	 Dan Streetman <ddstreet@ieee.org>,
	Vitaly Wool <vitaly.wool@konsulko.com>,
	 Yang Shi <shy828301@gmail.com>, Peter Xu <peterx@redhat.com>,
	 Minchan Kim <minchan@kernel.org>,
	 Andrew Morton <akpm@linux-foundation.org>,
	 Aneesh Kumar K V <aneesh.kumar@linux.ibm.com>,
	 Michal Hocko <mhocko@suse.com>,  Wei Xu <weixugc@google.com>
Subject: Re: [LSF/MM/BPF TOPIC] Swap Abstraction / Native Zswap
Date: Thu, 16 Mar 2023 09:42:48 +0800	[thread overview]
Message-ID: <87fsa55v53.fsf@yhuang6-desk2.ccr.corp.intel.com> (raw)
In-Reply-To: <CAJD7tkb4zK+ZzrQqgdTHE1r-Br9DKEER-gW3DFfJ+Sh9Rgs6+g@mail.gmail.com> (Yosry Ahmed's message of "Wed, 15 Mar 2023 00:41:45 -0700")

Yosry Ahmed <yosryahmed@google.com> writes:

> On Sun, Mar 12, 2023 at 6:11 PM Huang, Ying <ying.huang@intel.com> wrote:
>>
>> Chris Li <chrisl@kernel.org> writes:
>>
>> > On Fri, Mar 10, 2023 at 11:06:37AM +0800, Huang, Ying wrote:
>> >> > Unfortunately it's a little bit more. 24 is the extra overhead.
>> >> >
>> >> > Today we have an xarray entry for each swapped out page, that either
>> >> > has the swapcache pointer or the shadow entry.
>> >> >
>> >> > With this implementation, we have an xarray entry for each swapped out
>> >> > page, that has a pointer to the swap_desc.
>> >> >
>> >> > Ignoring the overhead of the xarray itself, we have (8 + 24) / (8 + 1) = 3.5556.
>> >>
>> >> OK.  I see.  We can only hold 8 bytes for each xarray entry.  To save
>> >> memory usage, we can allocate multiple swap_desc (e.g., 16) for each
>> >> xarray entry.  Then the memory usage of xarray becomes 1/N.
>> >
>> > The xarray look up key is the swap offset from the swap entry. If you
>> > put more than one swap_desc under the one xarray entry. It will mean
>> > all those different swap_descs will share a swap offset.
>>
>> For example, if we allocate 16 swap_desc for each xarray entry.  Then,
>> we can use (swap_desc_index >> 4) as key to lookup xarray, and
>> (swap_desc_index & 0x15) to index inside 16 swap_desc for the xarray
>> entry.
>
> With this approach we save (16 - 1) * 8 = 120 bytes per 16 swap_descs,
> but only if we use the space for 16 swap_desc's fully. As pages are
> swapped in, we can easily end up with less than 16 swap_descs for one
> xarray entry, how do we deal with such fragmentation?
>
> At 11 swap_descs per xarray entry, we are not saving any memory
> (wasting 24 * 5 = 120 bytes). Below 11 swap_descs, we are wasting more
> memory than we are saving.

Yes.  This approach isn't very fragmentation friendly.  xarray isn't
too.  So, I think we should try to reduce fragmentation anyway, for
example, reuse freed swap_desc IDs ASAP, or allocate less swap_desc per
xarray entry (e.g., 4), etc.

Best Regards,
Huang, Ying