From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D841C7618D for ; Thu, 6 Apr 2023 01:42:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8CEED6B0071; Wed, 5 Apr 2023 21:42:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87DD66B0074; Wed, 5 Apr 2023 21:42:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7466D6B0075; Wed, 5 Apr 2023 21:42:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6506C6B0071 for ; Wed, 5 Apr 2023 21:42:13 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 331DF140428 for ; Thu, 6 Apr 2023 01:42:13 +0000 (UTC) X-FDA: 80649265746.26.7151AB9 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf22.hostedemail.com (Postfix) with ESMTP id 8A261C000F for ; Thu, 6 Apr 2023 01:42:10 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="JAP/h3Te"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf22.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680745331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0Ax9KVMb1nGr9YIXjtUkpFw8hQrVLGZ93pSAtuf7a8Q=; b=CLDBIYwE+6jlrguaN67m7svjMAWFSIdI5huH1hKr2GdkphUID3wh5qACDOjOztoIl1/iWn tOzw43yWEl3U0SDQn6PBS0d4JlFferRw0lELlcKfWeevL5xDnpV4AGkbXLis1IBEu7BkOM ZNbFqKJPSnciUDLnegi4ooDWkXSTcuI= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="JAP/h3Te"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf22.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680745331; a=rsa-sha256; cv=none; b=e+EoFH4pTQLWJioApS8GtF951Pax4+SnKNl0IdIkBQxsLu6G+lbIO8lXnPbhudDbBKDq18 GLd7EcDbFW8S8oOAK+zYDbZHrejSdGmF7E2C97gyTGzRYL7SQvGjdWI12E364UHzYbsU0m /DPWkNHV3FjRBWlat19xqVOCEsIEy7U= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1680745330; x=1712281330; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version:content-transfer-encoding; bh=MGya5zfhVlWAz/RoC9xfzfNCwI538gXLnFKEu8ktfDs=; b=JAP/h3TehXTt+ww257/wH3rjs+7rb7obUx60NYBH1eea8s59tB5DYAua oiGcckgA8URQmdRNd4PKi4MhnhQa5ANfOK0OtV29tD+kpWg+tG6GuV5CE fP/E2Jp0afU6nFBKeNmGDA4hK9CDvVD3zWUBxiNj1wueKug89/888TGsr /4RKmImN7v5m7SQB1+Py2oI+AmM7QVuUY8zhyKGNG5dKgj4GQLS3p1dQk NGDjrJcSrdxQbC8mk8x4lp97OqfVfm1Fu+EcDTDPN9t5/arMc3UYJS250 tTV61UlDJvySIBrKbkJDX4bW9MC72axTpF2cL5ftLTc8a0F3uymd9rszB g==; X-IronPort-AV: E=McAfee;i="6600,9927,10671"; a="322262966" X-IronPort-AV: E=Sophos;i="5.98,322,1673942400"; d="scan'208";a="322262966" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Apr 2023 18:42:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10671"; a="637093477" X-IronPort-AV: E=Sophos;i="5.98,322,1673942400"; d="scan'208";a="637093477" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Apr 2023 18:42:02 -0700 From: "Huang, Ying" To: Yosry Ahmed Cc: Chris Li , lsf-pc@lists.linux-foundation.org, Johannes Weiner , Linux-MM , Michal Hocko , Shakeel Butt , David Rientjes , Hugh Dickins , Seth Jennings , Dan Streetman , Vitaly Wool , Yang Shi , Peter Xu , Minchan Kim , Andrew Morton , Aneesh Kumar K V , Michal Hocko , Wei Xu Subject: Re: [LSF/MM/BPF TOPIC] Swap Abstraction / Native Zswap References: <87edpbq96g.fsf@yhuang6-desk2.ccr.corp.intel.com> <87jzz1pfb3.fsf@yhuang6-desk2.ccr.corp.intel.com> <87fs9ppdhz.fsf@yhuang6-desk2.ccr.corp.intel.com> <87bkkcpckw.fsf@yhuang6-desk2.ccr.corp.intel.com> <87y1n8xe2g.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Thu, 06 Apr 2023 09:40:45 +0800 In-Reply-To: (Yosry Ahmed's message of "Tue, 4 Apr 2023 01:47:00 -0700") Message-ID: <87pm8hyehu.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 8A261C000F X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: gsjs67xx1cbcut51n9sepzuxspm9fndi X-HE-Tag: 1680745330-196529 X-HE-Meta: U2FsdGVkX18VnY26TWmYDGOIz0zv3G03Zu/cZq8SbB3xO63ybrX6qiBVho5ozSTilhe5+bMewvEQMtLvfNxE5FczFMgQPWKdwWPy+rcX/Od0Mip3gE3JOGqMpmXtM2H1I6SMi2Da/xA/kup+5/v6CX8VEv1WW2s/GzBrK60VFXGRsc7hrgcm9BwPLuamkIC0mJmqFdS/fCdnnQBeR28hKM56BIHPzjrjVVs2WdsMDk/XYJtkXAsNGOvETLywFxrieeLRecbigCeR8QhEWG+8ErAlwD2kKwYKZWU0aaZXkLLWA+tsNAr25YQzY3YzPP3HHBnyNps5nSogxoCCzux3Um8CTtmWkNsglXaVnoXhVN6pen+s5I1wrm+NGxLJgg9xXpL8TYPTH6z+2pJz2llVF5K46U/9bsiagimBGdGJ9G7TYjWUwa2fw1R/4FSZFO1thtzShhCRDzIrnoNojw3FvGr95reiQLJy5n+NviA1NOsi2v0cM580kx4SuzDLp738TORLE1SUcWLyKLlPukr/6ZNTMS7GqF7xGt4a8rjYCsIl4ePFo2Nw6qzL2gj39pP4Vu/pt2eoOtBghNLiDCUStRfg68rsKf8Qu2P9tAn2gx4hQP/GWmIvl5BoDkX9FqTSZD8BGNxBcbAhsxfWGYm9Koz9bEIHEKgTS+V4FweQfVuONyKsbvLKkkiY9jLZ9zzrBm51Ue2XkH+9hq9vvmm8+xCKBirRuytqL98q9nUG/PiKAlQ+9N07DdAoqLL58yq8MBlx6aXbhx2POgTB/Bi4Fd4Tc0PmwYqWVG0CXkI/Zosh7nos+2wqyECYmhYJcaVEM6iUaFJ94m5H2PIBvZQR8/+anhkk7LCe6iKH69hvdJWWl0F99gu6YnXX9prJuM3A8GjXLdU1BMZ+rbI1maGY/S1YdBfuB6gp+aoSXWueNPM86P5+m20DmboqOHHe+cLEdrX2VTLQvYRRbX1XD2D LxZyYcmn FmXkEmNB88scacFnzBjaDV11QgzP2lT3s+Q4698ejugIkzMaiauQuU4P+/P/xBiR87NH+9T9UdrPj3Woxt8Dkc15ZpvNKQuoUukwTn5G/vvqlz+PFzdia9s0DyASbaUF6syt2P7+8U0r8fJ1+xjk5CbziVwd4TkJajyTYoTtNTZe6fgBF2nBohwPQXhQ2Oc/tU0GKe2c4DpgRSzqbElHbSsRnrYV2XF7Gj1o3ZISdrRZdR1YP82uu9CSqKjgzs2jGITtZEycRQ0VoKt0YnTG+3asUdhbWcvU4NVPl9PCpKwarH4g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Yosry Ahmed writes: > On Tue, Apr 4, 2023 at 1:12=E2=80=AFAM Huang, Ying = wrote: >> >> Yosry Ahmed writes: >> >> > On Tue, Mar 28, 2023 at 6:33=E2=80=AFPM Huang, Ying wrote: >> >> >> >> Yosry Ahmed writes: >> >> >> >> > On Tue, Mar 28, 2023 at 2:32=E2=80=AFPM Chris Li wrote: >> >> >> >> >> >> On Tue, Mar 28, 2023 at 02:01:09PM -0700, Yosry Ahmed wrote: >> >> >> > On Tue, Mar 28, 2023 at 1:50=E2=80=AFPM Chris Li wrote: >> >> >> > > >> >> >> > > On Tue, Mar 28, 2023 at 12:59:31AM -0700, Yosry Ahmed wrote: >> >> >> > > > > > I don't have a problem with this approach, it is not rea= lly clean as >> >> >> > > > > > we still treat zswap as a swapfile and have to deal with= a lot of >> >> >> > > > > > unnecessary code like swap slots handling and whatnot. >> >> >> > > > > >> >> >> > > > > These are existing code? >> >> >> > > >> >> >> > > Yes. The ghost swap file are existing code used in Google for = many years. >> >> >> > > >> >> >> > > > I was referring to the fact that today with zswap being tied= to >> >> >> > > > swapfiles we do some necessary work such as searching for sw= ap slots >> >> >> > > > during swapout. The initial swap_desc approach aimed to avoi= d that. >> >> >> > > > With this minimal ghost swapfile approach we retain this unf= avorable >> >> >> > > > behavior. >> >> >> > > >> >> >> > > Can you explain how you can avoid the free swap entry search >> >> >> > > in the swap descriptor world? >> >> >> > >> >> >> > For zswap, in the swap descriptor world, you just need to alloca= te a >> >> >> > struct zswap_entry and have the swap descriptor point to it. No = need >> >> >> > for swap slot management since we are not tied to a swapfile and= pages >> >> >> > in zswap do not have a specific position. >> >> >> >> >> >> Your swap descriptor will be using one swp_entry_t, which get from= the PTE >> >> >> to lookup, right? That is the swap entry I am talking about. You j= ust >> >> >> substitute zswap swap entry with the swap descriptor swap entry. >> >> >> You still need to allocate from the free swap entry space at least= once. >> >> > >> >> > Oh, you mean the swap ID space. We just need to find an unused ID, = we >> >> > can simply use an allocating xarray >> >> > (https://docs.kernel.org/core-api/xarray.html#allocating-xarrays). >> >> > This is simpler than keeping track of swap slots in a swapfile. >> >> >> >> If we want to implement the swap entry management inside the zswap >> >> implementation (instead of reusing swap_map[]), then the allocating >> >> xarray can be used too. Some per-entry data (such as swap count, etc= .) >> >> can be stored there. I understanding that this isn't perfect (one mo= re >> >> xarray looking up, one more data structure, etc.), but this is a choi= ce >> >> too. >> > >> > My main concern here would be having two separate swap counting >> > implementations -- although it might not be the end of the world. >> >> This isn't a big issue for me. For file systems, there are duplicated >> functionality in different file system implementation, such as free >> block space management. Instead, I hope we can design better swap >> implementation in the future. >> >> > It would be useful to consider all the options. So far, I think we >> > have been discussing 3 alternatives: >> > >> > (a) The initial swap_desc proposal. >> >> My main concern for the initial swap_desc proposal is that the zswap >> code is put in swap core instead of zswap implementation per my >> understanding. So zswap isn't another swap implementation encapsulated >> with a common interface. Please correct me if my understanding isn't >> correct. >> >> If so, the flexibility of the swap system is the cost. For example, >> zswap may be always at the highest priority among all swap devices. We >> can move the cold page from zswap to some swap device. But we cannot >> move the cold page from some swap device to zswap. > > > Not really. In the swap_desc proposal, I intended to have struct > swap_desc contain either a swap device entry (swp_entry_t) or a > frontswap entry (a pointer). zswap implementation would not be in the > swap core, instead, we would have two swap implementations: swap > devices and frontswap/zswap -- each of which implement a common swap > API. We can use one of the free bits to distinguish the type of the > underlying entry (swp_entry_t or pointer to frontswap/zswap entry). > > We can start by only supporting moving pages from frontswap/zswap to > swap devices, but I don't see why the same design would not support > pages moving in the other direction if the need arises. > > The number of free bits in swp_entry_t and pointers is limited (2 bits > on 32-bit systems, 3 bits on 64-bit systems), so there are only a > handful of different swap types we can support with the swap_desc > design, but we only need two to begin with. If in the future we need > more, we can add an indirection layer then or expand swap_desc -- or > we can encode the data within the swap device itself (how it compares > to frontswap/zswap). > > In summary, the swap_desc proposal does NOT involve moving zswap code > to core swap, it involves a generic swap API with two implementations: > swap devices and frontswap/zswap. This eliminate the main concerns for me! Thanks! > The only problems I see with the swap_desc design are: > - Extra overhead for users using swapfiles only. > - A bigger leap from what we have today than other ideas proposed > (e.g. virtual swap device for zswap). Yes. Best Regards, Huang, Ying >> >> >> Maybe compression is always faster than any other swap devices, so we >> will never need the flexibility. Maybe the cost to hide zswap behind a >> common interface is unacceptable. I'm open to these. But please >> provide the evidence, and maybe data. >> >> Best Regards, >> Huang, Ying >> >> > (b) Add an optional indirection layer that can move swap entries >> > between swap devices and add a virtual swap device for zswap in the >> > kernel. >> > (c) Add an optional indirection layer that can move entries between >> > different swap backends. Swap backends would be zswap & swap devices >> > for now. Zswap needs to implement swap entry management, swap >> > counting, etc. >> > >> > Does this accurately summarize what we have discussed so far? >> >