From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CF28C433DB for ; Fri, 19 Feb 2021 19:23:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C2A0F64E77 for ; Fri, 19 Feb 2021 19:23:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C2A0F64E77 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4A90F6B0005; Fri, 19 Feb 2021 14:23:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 459A56B006C; Fri, 19 Feb 2021 14:23:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 348F46B006E; Fri, 19 Feb 2021 14:23:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0183.hostedemail.com [216.40.44.183]) by kanga.kvack.org (Postfix) with ESMTP id 181E36B0005 for ; Fri, 19 Feb 2021 14:23:17 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C074E8248D51 for ; Fri, 19 Feb 2021 19:23:16 +0000 (UTC) X-FDA: 77835990792.22.B616FAF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 2AFB6E0001AF for ; Fri, 19 Feb 2021 19:23:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613762595; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=fdCSAN64npUd+P1YDfJ63MqsRK7B7rJbfhaQ54M1yUc=; b=UB1kTviubUmLmit1ExQoe6Ps1l/QqotAnFIrwA/4wyXXOafBwpSqGohS2uaZ21LALMYZy4 J2N+gSI/27XoLHCIcw/OlHgz797flJz26uFzknDWLgWhwqrw/0vbJu5vcTC+RYz15mg7+O Oz2adYb5WxS1V/jQIr/i//BNQriWDsI= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-401-mCGqOtKaMeGNOKIhbxme2A-1; Fri, 19 Feb 2021 14:23:14 -0500 X-MC-Unique: mCGqOtKaMeGNOKIhbxme2A-1 Received: by mail-qk1-f197.google.com with SMTP id b27so4226428qkl.11 for ; Fri, 19 Feb 2021 11:23:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=fdCSAN64npUd+P1YDfJ63MqsRK7B7rJbfhaQ54M1yUc=; b=dJv6Mp3iTkRM0WxffdV40n7DpDf8doS+y/OLpKfHQuVKaB95CeaAeGwdeK1Q9d69R9 lP/bqzSGZCRF+XpklOGj4iktLA3iE48ydQWGW6lSCF5OM3xSu6mxwi59ulLo90ZlV/d+ oEnXVJotnojIoj6yQOeKNLiEF+N1YeWUsknPwgnmHOIn4W+icFPVXPa8INbGihHWOy/p kzOex8gKUJBoHIOaJPhcOj9yRS4MEgymjiq65DQyKsyfm0sTEUqot5fKRfE5PFz0+dEw xWTgfJlBN2TPET1g50qppJPvr3WcQF2rbkPqQTxcqXE7QccYx5t8EJhWt7cjLRZ8hgru yTOw== X-Gm-Message-State: AOAM530pTYB0Hy9csgDWrMhv/EFxjPemFrT9n2f3PfVG3LNgeL4cFCv5 pl4M/4A/nf20hklr+p6vJPD7FBOo/4WGFBqyuNIeT347pRZVL52nLoDvvyyoF5rVCrfP2Veg41c EWpUkhUPrzh0= X-Received: by 2002:a05:622a:354:: with SMTP id r20mr10488498qtw.99.1613762593478; Fri, 19 Feb 2021 11:23:13 -0800 (PST) X-Google-Smtp-Source: ABdhPJwlUOJtda3AQE2S/fUnpHXarn/y+dc9GJx5VJzTxxMAuX0PQH2qaOfwl33zpotPFLv912Hqlg== X-Received: by 2002:a05:622a:354:: with SMTP id r20mr10488454qtw.99.1613762593193; Fri, 19 Feb 2021 11:23:13 -0800 (PST) Received: from xz-x1 (bras-vprn-toroon474qw-lp130-25-174-95-95-253.dsl.bell.ca. [174.95.95.253]) by smtp.gmail.com with ESMTPSA id l24sm5994647qtj.50.2021.02.19.11.23.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Feb 2021 11:23:12 -0800 (PST) Date: Fri, 19 Feb 2021 14:23:10 -0500 From: Peter Xu To: David Hildenbrand Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Arnd Bergmann , Michal Hocko , Oscar Salvador , Matthew Wilcox , Andrea Arcangeli , Minchan Kim , Jann Horn , Jason Gunthorpe , Dave Hansen , Hugh Dickins , Rik van Riel , "Michael S . Tsirkin" , "Kirill A . Shutemov" , Vlastimil Babka , Richard Henderson , Ivan Kokshaysky , Matt Turner , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Chris Zankel , Max Filippov , linux-alpha@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linux-xtensa@linux-xtensa.org, linux-arch@vger.kernel.org Subject: Re: [PATCH RFC] mm/madvise: introduce MADV_POPULATE to prefault/prealloc memory Message-ID: <20210219192310.GI6669@xz-x1> References: <20210217154844.12392-1-david@redhat.com> <20210218225904.GB6669@xz-x1> <20210219163157.GF6669@xz-x1> <41444eb8-8bb8-8d5b-4cec-be7fa7530d0e@redhat.com> MIME-Version: 1.0 In-Reply-To: <41444eb8-8bb8-8d5b-4cec-be7fa7530d0e@redhat.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Stat-Signature: ee3nog3wfmyktteq4ahdp3rmh6dt67pt X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 2AFB6E0001AF Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=216.205.24.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1613762595-542628 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Feb 19, 2021 at 06:13:47PM +0100, David Hildenbrand wrote: > On 19.02.21 17:31, Peter Xu wrote: > > On Fri, Feb 19, 2021 at 09:20:16AM +0100, David Hildenbrand wrote: > > > On 18.02.21 23:59, Peter Xu wrote: > > > > Hi, David, > > > > > > > > On Wed, Feb 17, 2021 at 04:48:44PM +0100, David Hildenbrand wrote: > > > > > When we manage sparse memory mappings dynamically in user space - also > > > > > sometimes involving MADV_NORESERVE - we want to dynamically populate/ > > > > > discard memory inside such a sparse memory region. Example users are > > > > > hypervisors (especially implementing memory ballooning or similar > > > > > technologies like virtio-mem) and memory allocators. In addition, we want > > > > > to fail in a nice way if populating does not succeed because we are out of > > > > > backend memory (which can happen easily with file-based mappings, > > > > > especially tmpfs and hugetlbfs). [1] > > E.g., can we simply ask the kernel "how much memory this process can still > > allocate", then get a number out of it? I'm not sure whether it can be done > > Anything like that is completely racy and unreliable. The failure path won't be racy imho - If we can detect current process doesn't have enough memory budget, it'll be more efficient to fail even before trying to populate any memory and then drop part of them again. But I see your point - indeed it's good to guarantee the guest won't crash at any point of further guest side memory access. Another question: can the user actually specify arbitrary max-length for the virtio-mem device (which decides the maximum memory this device could possibly consume)? I thought we should check that first before realizing the device and we really shouldn't fail any guest memory access if that check passed. Feel free to correct me.. [...] > > > > I think there's a special case if QEMU fork() with a MAP_PRIVATE hugetlbfs > > mapping, that could cause the memory accouting to be delayed until COW happens. > > That would be kind of weird. I'd assume the reservation gets properly done > during fork() - just like for VM_ACCOUNT. AFAIK VM_ACCOUNT is never applied for hugetlbfs. Neither do I know any accounting done for hugetlbfs during fork(), if not taking the pinned pages into account - that is definitely a special case. > > > However that's definitely not the case for QEMU since QEMU won't work at all as > > late as that point. > > > > IOW, for hugetlbfs I don't know why we need to populate the pages at all if we > > simply want to know "whether we do still have enough space".. And IIUC 2) > > above is the major issue you'd like to solve too. > > To avoid page faults at runtime on access I think. Reservation <= > Preallocation. Yes. Besides my above question regarding max-length of virtio-mem device: we care most about private mappings of hugetlbfs/shmem here, am I right? I'm thinking why we'd need MAP_PRIVATE of these at all for VM context. It's definitely not the major scenario when they're used shared with either ovs or any non-qemu process, because then MAP_SHARED is a must. Then if we use them privately, can we simply always make it MAP_SHARED? IMHO MAP_PRIVATE could be helpful only if we'd like the COW scemantics, so it means when there're something already, we'd like to keep that snapshot but trigger page copy when writes. But is that the case for a VM memory backend which should be always zeroed by default? Then, I'm wondering can we simply avoid bothering with VM_PRIVATE on these file-backed memory at all - then we'll naturally get fallocate() on hand, which seems already working for us. Thanks, -- Peter Xu