Re: [LSF/MM/BPF TOPIC] Swap Abstraction / Native Zswap

["Huang, Ying" <ying.huang@intel.com>]

Chris Li <chrisl@kernel.org> writes:

> On Mon, Mar 20, 2023 at 10:55:03AM +0800, Huang, Ying wrote:
>> >
>> > How so? With the indirection enabled, the page tables & page cache
>> > have the swap id (or swap_desc index), which can point to a swap entry
>> > or a zswap entry -- which can change when the page is moved between
>> > zswap & swapfiles. How is xarray (a) indexed by the swap entry in this
>> > case? Shouldn't be indexed by the abstract swap id so that the
>> > writeback from zswap is transparent?
>> 
>> In my mind,
>> 
>> - swap core will define a abstract interface to swap implementations
>>   (zswap, swap device/file, maybe more in the future), like VFS.
>
> I like your idea very much.

Thanks!

>> 
>> - zswap will be a special swap implementation (compressing instead of
>>   writing to disk).
>
> Agree.
>
>> 
>> - swap core will manage the indirection layer and swap cache.
>
> Agree, those are very good points.
>
>> 
>> - swap core can move swap pages between swap implementations (e.g., from
>>   zswap to a swap device, or from one swap device to another swap
>>   device) with the help of the indirection layer.
>  
> We need to carefully design the swap cache that, when moving between
> swap implementaions, there will be one shared swap cache. The current
> swap cache belongs to swap devices, so two devices will have the same
> page in two swap caches.

We can remove a page from the swap cache for the swap device A, then
insert the page into the swap cache for the swap device B.  The swap
entry will be changed too.

>> In this design, the writeback from zswap becomes moving swapped pages
>> from zswap to a swap device.
>
> Ack.
>
>> 
>> If my understanding were correct, your suggestion is kind of moving
>> zswap logic to the swap core?  And zswap will be always at a higher
>> layer on top of swap device/file?
>
> It seems that way to me. I will let Yosry confirm that.
>
>> > I am not sure how this works with zswap. Currently swap_map[]
>> > implementation is specific for swapfiles, it does not work for zswap
>> > unless we implement separate swap counting logic for zswap &
>> > swapfiles. Same for the swapcache, it currently supports being indexed
>> > by a swap entry, it would need to support being indexed by a swap id,
>> > or have a separate swap cache for zswap. Having separate
>> > implementation would add complexity, and we would need to perform
>> > handoffs of the swap count/cache when a page is moved from zswap to a
>> > swapfile.
>> 
>> We can allocate a swap entry for each swapped page in zswap.
>
> One thing to consider when moving page from zswap to swap file, is the
> zswap swap entry the same entry as the swap file entry.

I think that the swap entry will be changed after moving.  Swap entry is
kind of local to a swap device.  While the swap desc ID isn't changed,
that is why we need the indirection layer.

>> > I think for this proposal, there are only 2 hardcoded tiers. Zswap is
>> > fast, swapfile is slow. In the future, we can support more dynamic
>> > tiering if the need arises.
>> 
>> We can start from a simple implementation.  And I think that it's better
>> to consider the general design too.  Try not to make it impossible now.
>
> In my mind there are a few usage cases:
> 1) using only swap file.
> 2) using only zswap, no swap file.
> 3) Using zswap + swap file (SSD).
>
> The swap core should handle both 3 cases well with minial memory waste.

Yes.  Agree.

Best Regards,
Huang, Ying