From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB828C77B62 for ; Tue, 4 Apr 2023 08:47:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 74A8D6B0071; Tue, 4 Apr 2023 04:47:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D3676B0074; Tue, 4 Apr 2023 04:47:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54CDB6B0075; Tue, 4 Apr 2023 04:47:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3DCB36B0071 for ; Tue, 4 Apr 2023 04:47:41 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 09B9BC0C03 for ; Tue, 4 Apr 2023 08:47:41 +0000 (UTC) X-FDA: 80643080322.30.5A6737E Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) by imf03.hostedemail.com (Postfix) with ESMTP id 1CBCD2001C for ; Tue, 4 Apr 2023 08:47:38 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="c2/Bvost"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.41 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680598059; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hegNkkLH5md92O36LjRBVdOTObpSkQ9V2LcaXC4qAF4=; b=JeDaMhkDUeP+n40TVZ67Aob1fNPsKfhZ9M12nYlQAJvy0dgtp1zRCUGtAJbXDnprbukfZl qmpro9PtuAyUETy+7ZwfRvax0MBtL8UnHj6fXQBHQ/JtiJTDCQ4spFmGqzWwPFLvB+4e+Y jPq4d6kL0DyAhWYD/wXb5zKlg+nxFWM= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="c2/Bvost"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.41 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680598059; a=rsa-sha256; cv=none; b=aH/waYH2YcMAYU4Czn26YxqHKCWCBwit6oUmA601vNnzSEsDCH24EG9pnE9VQdGL58XYtB wTC6Yr+MmlAad4wINIxrj4d8wyHdJUY1+Q8uSI7ArY2/aJEF75B1z4mBy8yDw4OF5XGh/A aBpN5vHqBxKHoFheiK4wI5etsqTx6/g= Received: by mail-ed1-f41.google.com with SMTP id er13so86514484edb.9 for ; Tue, 04 Apr 2023 01:47:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680598057; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=hegNkkLH5md92O36LjRBVdOTObpSkQ9V2LcaXC4qAF4=; b=c2/BvostpTODJVwPvkkY5bi02bdAsUMFVmf2BCOYzXo8OCKIghaly89xi3GV3wkv84 QCxEZ4/wHCaavZ1rM70XNY3aYm4ecxhwNZaCE8RM1tauXXk2AQiUBLPno2JeT/nUn4qd g9IAAvTSf6Ars4HG7BljP/aaqObfoJXcnedJ4dHz6bceFwNJ7f8CuNDrNMHHwotaNhyy YT+KUPHeY9fk0tJaKhNY71Ac98PCj0XTPSxMQ9lKenX6eCExjTdJDLSlmq3ukM4zujeQ 66PyO1DLVa0nPv83wo1iXqAb/yTbNxgCZBGg1/MBj/8LtQ6HTp8nlML8BLHh0Gzn0jBI kPGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680598057; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hegNkkLH5md92O36LjRBVdOTObpSkQ9V2LcaXC4qAF4=; b=nUNEnxLgyl35gZku14w8+uq2gFlj6qB8zQLSpkk1HjCS8IDYwfd23hRL32yC+55dKH A54WkW6LB79KmN0T9yndMM1d3sA1n93wLkTV5/eNClszsuGyDBC1pkNMYvWQ6XdC5cBg RhGknf8aBQDdnQretlc81du4Odnzy/X2GQOzN/yfk+DRkMzGXgXViOE0K3lSwGadMbws uSYtMK6LSozFc+IZVusHvT5+OMQ2bG13hJ62lensozfUMNPDebWMS9PqbpRDJvpEEQDo 6WECjl5TM6fXWOQJdYBnid+RfEidd68vt7Tv1i2MWIJ7Br3sl1IfTpAUSUUf2MTPQRTD aizg== X-Gm-Message-State: AAQBX9cNgZxCsr9IPStRxdqBXZ58xjdoUs09yNUvipSH8IYGjb9Xpt6k nciDk4vAZmzZPt2Zeh/ux5bP3zdEEE9qHjLfaqTOSw== X-Google-Smtp-Source: AKy350YJwgHtHQwrlHyZxE1B7G7R2PnkyROLnBimgpjy6SH2RpyD36GGxbbhhciVAmYbrBbEzc7OOFaDUEdK7EeYvGI= X-Received: by 2002:a17:907:c00e:b0:933:7658:8b44 with SMTP id ss14-20020a170907c00e00b0093376588b44mr795827ejc.15.1680598057349; Tue, 04 Apr 2023 01:47:37 -0700 (PDT) MIME-Version: 1.0 References: <87edpbq96g.fsf@yhuang6-desk2.ccr.corp.intel.com> <87jzz1pfb3.fsf@yhuang6-desk2.ccr.corp.intel.com> <87fs9ppdhz.fsf@yhuang6-desk2.ccr.corp.intel.com> <87bkkcpckw.fsf@yhuang6-desk2.ccr.corp.intel.com> <87y1n8xe2g.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87y1n8xe2g.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Yosry Ahmed Date: Tue, 4 Apr 2023 01:47:00 -0700 Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Swap Abstraction / Native Zswap To: "Huang, Ying" Cc: Chris Li , lsf-pc@lists.linux-foundation.org, Johannes Weiner , Linux-MM , Michal Hocko , Shakeel Butt , David Rientjes , Hugh Dickins , Seth Jennings , Dan Streetman , Vitaly Wool , Yang Shi , Peter Xu , Minchan Kim , Andrew Morton , Aneesh Kumar K V , Michal Hocko , Wei Xu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1CBCD2001C X-Stat-Signature: xyhcyhu8o3eb7gunqaceerxsqsmrubzo X-HE-Tag: 1680598058-580205 X-HE-Meta: U2FsdGVkX19EMAGA+4eibU8tSEspfTBrO/KfEN/j1GSPUyR4n6R72vRLvaj+5JscF6plF0De7Z46SIUhA+CZ5D4oUn8Pl9l9zGQAhwyfkEJ1ecRoVgtl+wxe8vUXdnXjtymXBeFexqOJzeXpWNCXG1ond1+ZvBteVybWYRRigy2gZGNjz5tWwFGX40LKAXsbSzmh9n2LNkmh0KISsr/y2M4rwbwrHYQwNenKjZ/HKQYAOn/p5FTVYaaxpnRxwtdmmRrqqbANMQyezQAlLLBJuX5ooz59NXojevz1q4DfJxD0KKMaEZOau9FdD2JNAEu+/gZ+6Er2cWprououBnuVN1EQq9CLLy4Dri93J8vrIn2uKPiI/EH5V/BYXaoZTSV2Rf9dISDUXpGZVFnVhbQLKKodPEjcyjwc27V1E+yPLTtR9tJC7u1ofAzZQ7Kyc+Sw6ySA8sk5A5hs2AWEE3oDOtYXwSWCsN4frPaSy24DAXeqmitFPqf/I1RbmOn3J23kbNMArDH02tXo+CvznHjqe9TDOjb3TF7+H5qVCizKqkKM1wCrG9sKSUyfD80Hxwi0tjbVf1DcOqHT0sP+goFcDTIpb2B8PCupy+OAV+ugidgSXyKrJ0BtQGiUXjbwbQoauu5HzFrIZv3KU8UinEFttZqL1AVhQkwGhTfb50KM/E1ytTSB+KAgHFKnLYN4zla/YqMf2B5a9LBbCw0SrGIjHsPytOKqez6RpWFSL5ZBqp7Xa/5r41gJMr7PO6mxqlefrRqk07OQtJuBXrBI5l8p7Ng32uJj5T6OC3oA5orSaD0791dRzYRNxQU6y30tJteIMIhJNtFEkpiylhM6OpReUzKEK41NcbZUfn4RE/Ng9vqzf4pLsD+41tuCAcUPEL0jcLHu5vrkzevg6yQLLicuf/7gCOvGkMJTLV2NemrjF+HCIMa5gwYFs6pV31CV0AGJFslFOREm9xzhNwMeeTD HGQuiz+h 5jetBs6qvVflPlKY9JQUNFRmseNDN/7l1CYahWhuD4BbQmEa4XUe61y5Sk9BPtbZXKXEetrPu4L90pETtBXMEeEa/MAdPQNv6JIiZPutIOI+3WqEoGVrMcMtYNLlfFVFTbTqBh9MYQcywvsd0Vh0AavSU9oMHBl15jViYnz29/EDazdhLtcWrwHy7wMeT6Pu1fl82FbPgOi80H+iq7XeBHzsZYt5Ox4OebLpbaLBD6PxH0RIvrbASB4kS9Md20kjNcFc12GMTMjlXfFUOq9jhsDYJ1FHJuNhChElrqTHwqCC5Vmn9uqFflcfpic6GY5DiokqNypftGQMDCMryfyXUogcl28JrxZ54z0PDPiSNDT+5eJuHlqtBAeeLdQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 4, 2023 at 1:12=E2=80=AFAM Huang, Ying w= rote: > > Yosry Ahmed writes: > > > On Tue, Mar 28, 2023 at 6:33=E2=80=AFPM Huang, Ying wrote: > >> > >> Yosry Ahmed writes: > >> > >> > On Tue, Mar 28, 2023 at 2:32=E2=80=AFPM Chris Li = wrote: > >> >> > >> >> On Tue, Mar 28, 2023 at 02:01:09PM -0700, Yosry Ahmed wrote: > >> >> > On Tue, Mar 28, 2023 at 1:50=E2=80=AFPM Chris Li wrote: > >> >> > > > >> >> > > On Tue, Mar 28, 2023 at 12:59:31AM -0700, Yosry Ahmed wrote: > >> >> > > > > > I don't have a problem with this approach, it is not real= ly clean as > >> >> > > > > > we still treat zswap as a swapfile and have to deal with = a lot of > >> >> > > > > > unnecessary code like swap slots handling and whatnot. > >> >> > > > > > >> >> > > > > These are existing code? > >> >> > > > >> >> > > Yes. The ghost swap file are existing code used in Google for m= any years. > >> >> > > > >> >> > > > I was referring to the fact that today with zswap being tied = to > >> >> > > > swapfiles we do some necessary work such as searching for swa= p slots > >> >> > > > during swapout. The initial swap_desc approach aimed to avoid= that. > >> >> > > > With this minimal ghost swapfile approach we retain this unfa= vorable > >> >> > > > behavior. > >> >> > > > >> >> > > Can you explain how you can avoid the free swap entry search > >> >> > > in the swap descriptor world? > >> >> > > >> >> > For zswap, in the swap descriptor world, you just need to allocat= e a > >> >> > struct zswap_entry and have the swap descriptor point to it. No n= eed > >> >> > for swap slot management since we are not tied to a swapfile and = pages > >> >> > in zswap do not have a specific position. > >> >> > >> >> Your swap descriptor will be using one swp_entry_t, which get from = the PTE > >> >> to lookup, right? That is the swap entry I am talking about. You ju= st > >> >> substitute zswap swap entry with the swap descriptor swap entry. > >> >> You still need to allocate from the free swap entry space at least = once. > >> > > >> > Oh, you mean the swap ID space. We just need to find an unused ID, w= e > >> > can simply use an allocating xarray > >> > (https://docs.kernel.org/core-api/xarray.html#allocating-xarrays). > >> > This is simpler than keeping track of swap slots in a swapfile. > >> > >> If we want to implement the swap entry management inside the zswap > >> implementation (instead of reusing swap_map[]), then the allocating > >> xarray can be used too. Some per-entry data (such as swap count, etc.= ) > >> can be stored there. I understanding that this isn't perfect (one mor= e > >> xarray looking up, one more data structure, etc.), but this is a choic= e > >> too. > > > > My main concern here would be having two separate swap counting > > implementations -- although it might not be the end of the world. > > This isn't a big issue for me. For file systems, there are duplicated > functionality in different file system implementation, such as free > block space management. Instead, I hope we can design better swap > implementation in the future. > > > It would be useful to consider all the options. So far, I think we > > have been discussing 3 alternatives: > > > > (a) The initial swap_desc proposal. > > My main concern for the initial swap_desc proposal is that the zswap > code is put in swap core instead of zswap implementation per my > understanding. So zswap isn't another swap implementation encapsulated > with a common interface. Please correct me if my understanding isn't > correct. > > If so, the flexibility of the swap system is the cost. For example, > zswap may be always at the highest priority among all swap devices. We > can move the cold page from zswap to some swap device. But we cannot > move the cold page from some swap device to zswap. Not really. In the swap_desc proposal, I intended to have struct swap_desc contain either a swap device entry (swp_entry_t) or a frontswap entry (a pointer). zswap implementation would not be in the swap core, instead, we would have two swap implementations: swap devices and frontswap/zswap -- each of which implement a common swap API. We can use one of the free bits to distinguish the type of the underlying entry (swp_entry_t or pointer to frontswap/zswap entry). We can start by only supporting moving pages from frontswap/zswap to swap devices, but I don't see why the same design would not support pages moving in the other direction if the need arises. The number of free bits in swp_entry_t and pointers is limited (2 bits on 32-bit systems, 3 bits on 64-bit systems), so there are only a handful of different swap types we can support with the swap_desc design, but we only need two to begin with. If in the future we need more, we can add an indirection layer then or expand swap_desc -- or we can encode the data within the swap device itself (how it compares to frontswap/zswap). In summary, the swap_desc proposal does NOT involve moving zswap code to core swap, it involves a generic swap API with two implementations: swap devices and frontswap/zswap. The only problems I see with the swap_desc design are: - Extra overhead for users using swapfiles only. - A bigger leap from what we have today than other ideas proposed (e.g. virtual swap device for zswap). > > > Maybe compression is always faster than any other swap devices, so we > will never need the flexibility. Maybe the cost to hide zswap behind a > common interface is unacceptable. I'm open to these. But please > provide the evidence, and maybe data. > > Best Regards, > Huang, Ying > > > (b) Add an optional indirection layer that can move swap entries > > between swap devices and add a virtual swap device for zswap in the > > kernel. > > (c) Add an optional indirection layer that can move entries between > > different swap backends. Swap backends would be zswap & swap devices > > for now. Zswap needs to implement swap entry management, swap > > counting, etc. > > > > Does this accurately summarize what we have discussed so far? > >