From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F21CC35247 for ; Tue, 4 Feb 2020 18:56:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 28FD42082E for ; Tue, 4 Feb 2020 18:56:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ElZmrd3Q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 28FD42082E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A91116B0005; Tue, 4 Feb 2020 13:56:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A42AF6B0006; Tue, 4 Feb 2020 13:56:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9591B6B0007; Tue, 4 Feb 2020 13:56:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0107.hostedemail.com [216.40.44.107]) by kanga.kvack.org (Postfix) with ESMTP id 7D30E6B0005 for ; Tue, 4 Feb 2020 13:56:39 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1C702181AC9C6 for ; Tue, 4 Feb 2020 18:56:39 +0000 (UTC) X-FDA: 76453350918.12.frogs58_57197b5b3c351 X-HE-Tag: frogs58_57197b5b3c351 X-Filterd-Recvd-Size: 6753 Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Tue, 4 Feb 2020 18:56:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580842598; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VG9fJQCSTH0iBN3XQzGN/nWQWYqUcHLExT95A3q2HzM=; b=ElZmrd3Qpp6CauRViwEigueudVczFL5caPejk2kgg441znh02OQsJaqwYeZGmbD5k5rt0C +6J2wnq4HheOamycnB5yP3uEVXkuqnyx/Ng8N+TYEAJ3P2KiqugxQ17IjmH30TGLdAkWwc T6IbOKazlF3BI3C27WdLbrci3JJH4eA= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-355-5K3S41ftNUiNFNvtn4-5qQ-1; Tue, 04 Feb 2020 13:56:27 -0500 Received: by mail-qk1-f199.google.com with SMTP id q2so5792112qkq.19 for ; Tue, 04 Feb 2020 10:56:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=nh+DZ4Y0u+XTvfkHliPjLCQG0zaBs8Iokr4BO5qL5oE=; b=EpOWa6B64bEALDpMOUhkOx0WLtZe81rQA/GTW1OBeOeF6mvQP7EBSjRmDjW9DfS+R9 ZskW4cEz/YY4M+7VAUBXGYEMG+OnBLG9GyMvBw1Gntcz7oGPQozeOP/RzPxzcgHTGP6r TdxraB0YBR94fEoSAGSUeroTiAFpRQjq38da2T2F0QHD5WvsaqmiT6Kyv6KH1jB7NeRB jB61ohtGxBAJogTYwHNDBh11qqu9d+j5fqO/iJsqjBJR7u91Nnp16jNxsDQLlSMDwu7y b+Iy3EGLVKICMS1FMHBPb3VaMQ2PhkHnULhf2+FqQbXJwFam7ocbgBlyAvysb1v9QXCi W1FQ== X-Gm-Message-State: APjAAAVO629j9xTK1Y7BZEgcd/e8XEYZzE7xjnrTuzQ0oIhCWB3dmbx+ 5ADzc4oNULd+z6oaUGZ5Q9SwZFRoK/ShBSDjWR+b6jbtQHJzpraDjr35bKauG8G/LQ/fv2ESyaA d2DN3ehRJ2tg= X-Received: by 2002:a0c:f193:: with SMTP id m19mr136392qvl.154.1580842587493; Tue, 04 Feb 2020 10:56:27 -0800 (PST) X-Google-Smtp-Source: APXvYqwnU58vkOrjzUqa3857ou1cEE6izA8TOEXDujVIQTzgSk3wVjHWzJ8LdVI0/LhMBhOqOXxhmg== X-Received: by 2002:a0c:f193:: with SMTP id m19mr136375qvl.154.1580842587234; Tue, 04 Feb 2020 10:56:27 -0800 (PST) Received: from redhat.com (bzq-79-176-41-183.red.bezeqint.net. [79.176.41.183]) by smtp.gmail.com with ESMTPSA id s22sm11834992qke.19.2020.02.04.10.56.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2020 10:56:26 -0800 (PST) Date: Tue, 4 Feb 2020 13:56:21 -0500 From: "Michael S. Tsirkin" To: Tyler Sanderson Cc: David Hildenbrand , Alexander Duyck , "Wang, Wei W" , "virtualization@lists.linux-foundation.org" , David Rientjes , "linux-mm@kvack.org" , Michal Hocko , namit@vmware.com Subject: Re: Balloon pressuring page cache Message-ID: <20200204135533-mutt-send-email-mst@kernel.org> References: <75d4594f-0864-5172-a0f8-f97affedb366@redhat.com> <286AC319A985734F985F78AFA26841F73E3F8A02@shsmsx102.ccr.corp.intel.com> <20200203080520-mutt-send-email-mst@kernel.org> <5ac131de8e3b7fc1fafd05a61feb5f6889aeb917.camel@linux.intel.com> <20200203120225-mutt-send-email-mst@kernel.org> <74cc25a6-cefb-c580-8e59-5b76fb680bf4@redhat.com> MIME-Version: 1.0 In-Reply-To: X-MC-Unique: 5K3S41ftNUiNFNvtn4-5qQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Feb 04, 2020 at 10:52:42AM -0800, Tyler Sanderson wrote: >=20 >=20 > On Tue, Feb 4, 2020 at 12:29 AM David Hildenbrand wrot= e: >=20 > On 03.02.20 21:32, Tyler Sanderson wrote: > > There were apparently good reasons for moving away from OOM notifie= r > > callback: > > https://lkml.org/lkml/2018/7/12/314 > > https://lkml.org/lkml/2018/8/2/322 > > > > In particular the OOM notifier is worse than the shrinker because: >=20 > The issue is that DEFLATE_ON_OOM is under-specified. >=20 > > > >=A0 1. It is last-resort, which means the system has already gone th= rough > >=A0 =A0 =A0heroics to prevent OOM. Those heroic reclaim efforts are = expensive > >=A0 =A0 =A0and impact application performance. >=20 > That's *exactly* what "deflate on OOM" suggests. >=20 >=20 > It seems there are some use cases where "deflate on OOM" is desired and o= thers > where "deflate on pressure" is desired. > This suggests adding a new feature bit "DEFLATE_ON_PRESSURE" that registe= rs the > shrinker, and reverting DEFLATE_ON_OOM to use the OOM notifier callback. >=20 > This lets users configure the balloon for their use case. Right. Let's not repeat past mistakes and let's try to specify this new one properly though :) >=20 >=20 > Assume you are using virtio-balloon for some weird way of memory > hotunplug (which is what some people do) and you want to minimize the > footprint of your guest. Then you really only want to give the guest > more memory (or rather, let it take back memory automatically in this > case) in case it really needs more memory. It should try to reclaim f= irst. >=20 > Under-specified. >=20 >=20 > >=A0 2. It lacks understanding of NUMA or other OOM constraints. >=20 > Ballooning in general lacks the understanding of NUMA. >=20 > >=A0 3. It has a higher potential for bugs due to the subtlety=A0of t= he > >=A0 =A0 =A0callback context. >=20 > While that is a valid point, it doesn't explain why existing > functionality is changed. >=20 > Personally, I think DEFLATE_ON_OOM should never have been introduced = (at > least not in this form). >=20 > I'm actually not sure how you would safely do memory overcommit without > DEFLATE_ON_OOM. So I think it unlocks a huge use case. > =A0 >=20 >=20 >=20 > -- > Thanks, >=20 > David / dhildenb >=20 >=20 From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: Balloon pressuring page cache Date: Tue, 4 Feb 2020 13:56:21 -0500 Message-ID: <20200204135533-mutt-send-email-mst@kernel.org> References: <75d4594f-0864-5172-a0f8-f97affedb366@redhat.com> <286AC319A985734F985F78AFA26841F73E3F8A02@shsmsx102.ccr.corp.intel.com> <20200203080520-mutt-send-email-mst@kernel.org> <5ac131de8e3b7fc1fafd05a61feb5f6889aeb917.camel@linux.intel.com> <20200203120225-mutt-send-email-mst@kernel.org> <74cc25a6-cefb-c580-8e59-5b76fb680bf4@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: Content-Disposition: inline List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" To: Tyler Sanderson Cc: "virtualization@lists.linux-foundation.org" , "linux-mm@kvack.org" , namit@vmware.com, David Rientjes , Alexander Duyck , Michal Hocko List-Id: virtualization@lists.linuxfoundation.org On Tue, Feb 04, 2020 at 10:52:42AM -0800, Tyler Sanderson wrote: > = > = > On Tue, Feb 4, 2020 at 12:29 AM David Hildenbrand wrot= e: > = > On 03.02.20 21:32, Tyler Sanderson wrote: > > There were apparently good reasons for moving away from OOM notifier > > callback: > > https://lkml.org/lkml/2018/7/12/314 > > https://lkml.org/lkml/2018/8/2/322 > > > > In particular the OOM notifier is worse than the shrinker because: > = > The issue is that DEFLATE_ON_OOM is under-specified. > = > > > >=C2=A0 1. It is last-resort, which means the system has already gone= through > >=C2=A0 =C2=A0 =C2=A0heroics to prevent OOM. Those heroic reclaim eff= orts are expensive > >=C2=A0 =C2=A0 =C2=A0and impact application performance. > = > That's *exactly* what "deflate on OOM" suggests. > = > = > It seems there are some use cases where "deflate on OOM" is desired and o= thers > where "deflate on pressure" is desired. > This suggests adding a new feature bit "DEFLATE_ON_PRESSURE" that registe= rs the > shrinker, and reverting DEFLATE_ON_OOM to use the OOM notifier callback. > = > This lets users configure the balloon for their use case. Right. Let's not repeat past mistakes and let's try to specify this new one properly though :) > = > = > Assume you are using virtio-balloon for some weird way of memory > hotunplug (which is what some people do) and you want to minimize the > footprint of your guest. Then you really only want to give the guest > more memory (or rather, let it take back memory automatically in this > case) in case it really needs more memory. It should try to reclaim f= irst. > = > Under-specified. > = > = > >=C2=A0 2. It lacks understanding of NUMA or other OOM constraints. > = > Ballooning in general lacks the understanding of NUMA. > = > >=C2=A0 3. It has a higher potential for bugs due to the subtlety=C2= =A0of the > >=C2=A0 =C2=A0 =C2=A0callback context. > = > While that is a valid point, it doesn't explain why existing > functionality is changed. > = > Personally, I think DEFLATE_ON_OOM should never have been introduced = (at > least not in this form). > = > I'm actually not sure how you would safely do memory overcommit without > DEFLATE_ON_OOM. So I think it unlocks a huge use case. > =C2=A0 > = > = > = > -- > Thanks, > = > David / dhildenb > = > =