From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1490C76196 for ; Tue, 28 Mar 2023 20:00:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 912756B0071; Tue, 28 Mar 2023 16:00:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C27C6B0072; Tue, 28 Mar 2023 16:00:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78CB36B0074; Tue, 28 Mar 2023 16:00:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 685B76B0071 for ; Tue, 28 Mar 2023 16:00:37 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2D322140A5D for ; Tue, 28 Mar 2023 20:00:37 +0000 (UTC) X-FDA: 80619374514.21.C35535E Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by imf20.hostedemail.com (Postfix) with ESMTP id 4C2BD1C0008 for ; Tue, 28 Mar 2023 20:00:34 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=WrXPFekm; spf=pass (imf20.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.49 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680033634; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NlguKQob/kh+Q6i57OoFGGW5vg1ySsiMNE41lp3hkCU=; b=tTL06PCdaVpaP/jqAYr1O6l09smPOEXGCbnTS0Nb7/0Hyabh77zhDPAts+hCgwsm9Hm53y G9tMb4z6mSpQ/g+r22KwXjeZHXFVyYy4EG6dNbHp1fwYq1nFtjVqzxaJn7aJNUPO1DiW9m JBavzRBfEyhp/w3KAE/IfWp5lGOdqSs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=WrXPFekm; spf=pass (imf20.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.49 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680033634; a=rsa-sha256; cv=none; b=2ViADH+KEwb9HOIJbETpljKlfZ/a/7F7yJxBAcAXevSq2l+wRyTvNEVmo1FrRxgS6yiFsn V948bLjv9vt516OF6LZG8ZpN0RlLIm6oNeeTyk5+Vct6qFIT+Etu4v1EPMTv5TGNVFUBdZ S3s6atIfFh/SdcDXRNn0nXkZ/n6Q5iI= Received: by mail-ed1-f49.google.com with SMTP id r11so54418682edd.5 for ; Tue, 28 Mar 2023 13:00:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680033633; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=NlguKQob/kh+Q6i57OoFGGW5vg1ySsiMNE41lp3hkCU=; b=WrXPFekm/xr5nq9Sbr2kL9jnUx6Y8iHiYzliI46IkR20klQw+H1R6K5guQ1m6Rad+A dtMaI+cL2spPqPfHV0GynycLrb/dVLzt7AvVg6DJC1SwiJX2WdYiQMPXuPwfDuYUmyjr j/6C7/B+MCbH4c/2gf7qmCIepiah4jp7bIa4oJCozRXou1IlfSq+Tc4PWKsUuAQ051Wf KeEd4Nt7W+rZogp0/5G5QD86GfYl9vrnjd0kvJq6IzagTXIlTeUfYcPVu/Ve5/dCn9h9 eefaqv7B4XeHWa3mCMfAKjaYgvpaCCAr2vsMDjQIZWcLLcYVuxQ0ux551pp/eKbxPi5F Vbsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680033633; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NlguKQob/kh+Q6i57OoFGGW5vg1ySsiMNE41lp3hkCU=; b=bfSrmNviLQWhCncUXF2zW8Z+jJpQW4niwUt8Oha88vUOTTBmjbS0L+/kCRdtckG+0u z5mKZ+f6wcUE9N1CUY0tOmBNyTHq00muV4tyuQFXQx+XotyHHm+jtFoIYQTYdAeXiGSE T2Rjh1Dl4/TQC7H6F7bvaq1AWVuacmDrOizeqB51Q8esDA1KLLX96AVl6tXtuy0dfvj1 Dp1HJHMiyNG6aIx049aYl6wRiABW+c/9XW3ENzN/7wV+tXLtbUa0i2Z/m2MGN6cxpENq 9yOHBG1+hNQTiMybTxmp/gxCfJxHeEUjdErBc9blU0Png993EdgMon3YUEXDQoSU5mQQ 47NQ== X-Gm-Message-State: AAQBX9fX1kbgTCiOzg4C6NNWK50MJPYmCGGDRYZWcnknOxkAj4sIFoBH TS9ZvUcgApZWBAdUUoXmYJhqyiL1H6C+B4wZ12luPw== X-Google-Smtp-Source: AKy350a0fNyg08y3astL37O/MFSTBj6M0aVpHp4GA3UJNiXZaaQNEk3jMNJOJoJyuOwgLbcIZn/WbI7T3OKrXo7uO/k= X-Received: by 2002:a50:8ace:0:b0:502:49bf:7e8d with SMTP id k14-20020a508ace000000b0050249bf7e8dmr3411364edk.8.1680033632586; Tue, 28 Mar 2023 13:00:32 -0700 (PDT) MIME-Version: 1.0 References: <87sfduri1j.fsf@yhuang6-desk2.ccr.corp.intel.com> <87edpbq96g.fsf@yhuang6-desk2.ccr.corp.intel.com> <87jzz1pfb3.fsf@yhuang6-desk2.ccr.corp.intel.com> <87fs9ppdhz.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: From: Yosry Ahmed Date: Tue, 28 Mar 2023 12:59:55 -0700 Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Swap Abstraction / Native Zswap To: Johannes Weiner Cc: "Huang, Ying" , Chris Li , lsf-pc@lists.linux-foundation.org, Linux-MM , Michal Hocko , Shakeel Butt , David Rientjes , Hugh Dickins , Seth Jennings , Dan Streetman , Vitaly Wool , Yang Shi , Peter Xu , Minchan Kim , Andrew Morton , Aneesh Kumar K V , Michal Hocko , Wei Xu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4C2BD1C0008 X-Rspam-User: X-Stat-Signature: sqh6edadadappzu5g3qpdkhuus6pqtbx X-HE-Tag: 1680033634-382083 X-HE-Meta: U2FsdGVkX19QuOeOWwCZ8Xg1rm9UGqk7uvMr/jgTx7VfTRRKEuLCqOV8/RM5nj/wo7qCJzUcI8d+2j1fcwvjIL+RA2WaMKo4Lp3pu+g1TWG5ZtuOY/vLNI6jHlDo/kC7r3UC/rDcTBdFmGdxYtMVmgpF3cm3SR33fI6dM7XwSnaVn239lBXH9uhRdGVb6ovPSor9AGCD2LTYo6C2B0TzlSxUu4FLiP3Z92gEgtKCU4fhGolvlPh8Uooew1ft00gLnD4ybaB2vlDeu8cJu2uuRkXlzXi7/hqNEyiy/X3g84CFxmm84T4lLVFM+9Z2SY6OVFW1BV0YbMjorK4e+gIzm3YnkJ0RPBKjcHfanXt27KEa7MihKAdrkWu41GiigT8mmmymUaUXspIq95nXUkoFhGDGaP47xHSkwIZ7IquH1cgdLuTyTVS6vwEZu6Qsmvi4NJi4Qo6LG93cwWz61hh7ZMldvcbGRhHBKXSMR3etotT4JfLmCk6jIJ/4xdP+ocS3piB2C9HB2I2kQMIYIyOniyntd8HlUJqs4/SUvnHnromQxhqcinZfCxXhV45rnI3hVrNJAGHuNeOa4iSiwq/rItg2v4XZHWJwSfsuvAPIXadcEa9osAqs2jgLbV5PJ+6UuUXaFptlX7aNSfKmyy7Qtwt8nKczE8nl33T143FQIGNKyuGO7UwIaZIsnd6GaCBouLrjYZe0luvdE3h09I+JppSzymzOO0bOTJBfLLa3euZV1ity2oULx1+jPNuvtJ+QkFryB5DK/S8PPMbO4W82RnvToXmgGNBXmO2ke6iFzC5CY1wDLPoUra0Rpl4d1Y0HedIPEO3Iw+rRP0aAqRje9FOWgm5yN27X8o+9fMfCNNhmC8KknkiVnzVJPl5GJtaHD/XLHk5PrOIAiLST921iY3IPeB8RCB/hNIQ47xuVla2fmv3kHfzT5vhkarQqyjyqpGL/MKKs12TdQ4kOqsw Ikn2xXyf pSutv5G63KBHH7M3of1XXfz29qGISHCIoLd7uXv8i75+qlFb4P7PHsxqaQy5lz/eRXvyK1RS8+0+NC6SHzIrO3Z7OGXTr9Nzcf2YxeJoS9/I3G9XiKLEWWNQAxM+yHgpCtLZ3DoJcsblbBGV9HMxv6pmBt4TFRBw9/uptiMcbsjeq4JT6AB4asqX1DL80z2928oG6OFJPiLWRvbl0DIuk0e7Tt4sga8w+1nvOvjzDhoxrwiWAWhAXCeMYMGWDU4gnghFjqVcwXxraMIbZ18T+Rm9rlXDyMdnOGN2S/yI1jeJeGIlM0l3VBS7zM45XXD2FTS/2BtAaPdtdU9KDQcqePUA1jLPHmhOFtP8Z X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 28, 2023 at 7:14=E2=80=AFAM Johannes Weiner wrote: > > On Tue, Mar 28, 2023 at 12:59:31AM -0700, Yosry Ahmed wrote: > > On Tue, Mar 28, 2023 at 12:01=E2=80=AFAM Huang, Ying wrote: > > > Yosry Ahmed writes: > > > > We also have to unnecessarily limit the size of zswap with the size= of > > > > this fake swapfile. > > > > > > I guess you need to limit the size of zswap anyway, because you need = to > > > decide when to start to writeback or moving to the lower tiers. > > > > zswap has a knob to limit its size, but based on the actual memory > > usage of zswap (i.e the size of compressed pages). There is ongoing > > work as well to autotune this if I remember correctly. Having to deal > > with both the limit on compressed memory and the limited on the > > uncompressed size of swapped pages is cumbersome. Again, we already > > have this behavior today, but the initial swap_desc proposal aimed to > > avoid it. > > Right. > > The optimal size of the zswap pool on top of a swapfile depends on the > size and compressibility of the warm set of the workload: data that's > too cold for regular memory yet too hot for swap. This is obviously > highly dynamic, and even varies over time within individual jobs. > > With this proposal, we'd have to provision a static swap map for the > highest expected offloading rate and compression ratio on every host > of a shared pool. On 256G machines that would put the fixed overhead > at a couple of hundred MB if I counted right. > > Not the end of the world I guess. And I agree it would make for > simpler initial patches. OTOH, it would add more quirks to the swap > code instead of cleaning it up. And given how common compressed memory > setups are nowadays, it still feels like it's trading off too far in > favor of regular swap setups at the expense of compression. Right, I don't like adding more quirks to the swap code. I guess for Android and ChromeOS, even though they are using compressed memory, it is zram not zswap, so any extra overhead by swap_descs for normal swap setups would also affect Android -- so that's something to think about. > > So it wouldn't be my first preference. But it sounds workable. If we settle on this as a first step, perhaps to avoid any ABI changes we can have the kernel create a virtual swap device for zswap if it is enabled, without userspace interfering or having to do swapon on a sparse swapfile like we do today with ghost swapfiles at Google. We can then implement indirection logic that only supports moving pages between swap devices -- and perhaps only restrict it to only support the virtual zswap swap device as a top tier initially. The only user visible effect would be that if the user has zswap enabled and did not configure a swapfile, zswap would start compressing pages regardless, but that's what we're hoping for anyway -- I wouldn't think this is a breaking change. This also wouldn't be my first preference, but it seems like a smaller step from what we have today. As long as we don't have ABI dependencies we can always come back and change it later I suppose.