From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F01DCC5DF62 for ; Wed, 6 Nov 2019 12:16:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B269A217F4 for ; Wed, 6 Nov 2019 12:16:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B269A217F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 432556B000A; Wed, 6 Nov 2019 07:16:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E3056B000C; Wed, 6 Nov 2019 07:16:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D1906B000D; Wed, 6 Nov 2019 07:16:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0202.hostedemail.com [216.40.44.202]) by kanga.kvack.org (Postfix) with ESMTP id 183D66B000A for ; Wed, 6 Nov 2019 07:16:09 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id A39E0180AD815 for ; Wed, 6 Nov 2019 12:16:08 +0000 (UTC) X-FDA: 76125749616.27.heart98_52b17cfb1fc0d X-HE-Tag: heart98_52b17cfb1fc0d X-Filterd-Recvd-Size: 5885 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Wed, 6 Nov 2019 12:16:08 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 31B6EB41B; Wed, 6 Nov 2019 12:16:06 +0000 (UTC) Date: Wed, 6 Nov 2019 13:16:05 +0100 From: Michal Hocko To: akpm@linux-foundation.org Cc: aarcange@redhat.com, alexander.h.duyck@linux.intel.com, dan.j.williams@intel.com, dave.hansen@intel.com, david@redhat.com, konrad.wilk@oracle.com, lcapitulino@redhat.com, mgorman@techsingularity.net, mm-commits@vger.kernel.org, mst@redhat.com, osalvador@suse.de, pagupta@redhat.com, pbonzini@redhat.com, riel@surriel.com, vbabka@suse.cz, wei.w.wang@intel.com, willy@infradead.org, yang.zhang.wz@gmail.com, linux-mm@kvack.org Subject: Re: + mm-introduce-reported-pages.patch added to -mm tree Message-ID: <20191106121605.GH8314@dhcp22.suse.cz> References: <20191106000547.juQRi83gi%akpm@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191106000547.juQRi83gi%akpm@linux-foundation.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: I didn't have time to read through newer versions of this patch series but I remember there were concerns about this functionality being pulled into the page allocator previously both by me and Mel [1][2]. Have those been addressed? I do not see an ack from Mel or any other MM people. Is there really a consensus that we want something like that living in the allocator? There has also been a different approach discussed and from [3] (referenced by the cover letter) I can only see : Then Nitesh's solution had changed to the bitmap approach[7]. However it : has been pointed out that this solution doesn't deal with sparse memory, : hotplug, and various other issues. which looks more like something to be done than a fundamental roadblocks. [1] http://lkml.kernel.org/r/20190912163525.GV2739@techsingularity.net [2] http://lkml.kernel.org/r/20190912091925.GM4023@dhcp22.suse.cz [3] http://lkml.kernel.org/r/29f43d5796feed0dec8e8bb98b187d9dac03b900.camel@linux.intel.com On Tue 05-11-19 16:05:47, Andrew Morton wrote: > From: Alexander Duyck > Subject: mm: introduce Reported pages > > In order to pave the way for free page reporting in virtualized > environments we will need a way to get pages out of the free lists and > identify those pages after they have been returned. To accomplish this, > this patch adds the concept of a Reported Buddy, which is essentially > meant to just be the Uptodate flag used in conjunction with the Buddy page > type. > > It adds a set of pointers we shall call "reported_boundary" which > represent the upper boundary between the unreported and reported pages. > The general idea is that in order for a page to cross from one side of the > boundary to the other it will need to verify that it went through the > reporting process. Ultimately a free list has been fully processed when > the boundary has been moved from the tail all they way up to occupying the > first entry in the list. Without this we would have to manually walk the > entire page list until we have find a page that hasn't been reported. In > my testing this adds as much as 18% additional overhead which would make > this unattractive as a solution. > > One limitation to this approach is that it is essentially a linear search > and in the case of the free lists we can have pages added to either the > head or the tail of the list. In order to place limits on this we only > allow pages to be added before the reported_boundary instead of adding to > the tail itself. An added advantage to this approach is that we should be > reducing the overall memory footprint of the guest as it will be more > likely to recycle warm pages versus trying to allocate the reported pages > that were likely evicted from the guest memory. > > Since we will only be reporting one zone at a time we keep the boundary > limited to being defined for just the zone we are currently reporting > pages from. Doing this we can keep the number of additional pointers > needed quite small. To flag that the boundaries are in place we use a > single bit in the zone to indicate that reporting and the boundaries are > active. > > We store the index of the boundary pointer used to track the reported page > in the page->index value. Doing this we can avoid unnecessary computation > to determine the index value again. There should be no issues with this > as the value is unused when the page is in the buddy allocator, and is > reset as soon as the page is removed from the free list. > > Link: http://lkml.kernel.org/r/20191105220219.15144.69031.stgit@localhost.localdomain > Signed-off-by: Alexander Duyck > Cc: Andrea Arcangeli > Cc: Dan Williams > Cc: Dave Hansen > Cc: David Hildenbrand > Cc: > Cc: Luiz Capitulino > Cc: Matthew Wilcox > Cc: Mel Gorman > Cc: Michael S. Tsirkin > Cc: Michal Hocko > Cc: Oscar Salvador > Cc: Pankaj Gupta > Cc: Paolo Bonzini > Cc: Rik van Riel > Cc: Vlastimil Babka > Cc: Wei Wang > Cc: Yang Zhang > Signed-off-by: Andrew Morton -- Michal Hocko SUSE Labs