From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96865C4CEC5 for ; Thu, 12 Sep 2019 16:35:42 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6ADF3206A5 for ; Thu, 12 Sep 2019 16:35:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="B8VlFnz6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6ADF3206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=techsingularity.net Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2OOz5757bB2MWkRAeitSJPUHZ+CuEOcpWmIv+MYvT9Y=; b=B8VlFnz67ylMCx iAUf/f/+96Mo62J0BvBjQlRtDjgI2XhlTOpzE5Hb8arojAkOyUWHs35Oz0KSldvgORnPJqrsH6/uT Notz/xw/0SGBvaMbjphlHzblKjT2lRcDwqDZfuBZDl0NiqCmACxU3YS0VD/v3OhuGAHs/63pLOYD8 DxWgzXGjaQbv+RcTsU15I2f/fe1toWdycTlx8JF7v7nPjJDQZvt9ujHXSUZQLbs9RNa59ziBBG9tx 0LL6O+ygzhfNRIF5vM5kGEeVmmcsF6HFmFr07+wE8jiElpBkGUsQP03LyXGmj5z1JVJDpqS1u60fr QuqtJMYg3DMUDNCt7L6g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1i8S4L-0004iT-PF; Thu, 12 Sep 2019 16:35:41 +0000 Received: from outbound-smtp02.blacknight.com ([81.17.249.8]) by bombadil.infradead.org with esmtps (Exim 4.92.2 #3 (Red Hat Linux)) id 1i8S4H-0004hK-E7 for linux-arm-kernel@lists.infradead.org; Thu, 12 Sep 2019 16:35:40 +0000 Received: from mail.blacknight.com (pemlinmail02.blacknight.ie [81.17.254.11]) by outbound-smtp02.blacknight.com (Postfix) with ESMTPS id E65F498BD8 for ; Thu, 12 Sep 2019 17:35:26 +0100 (IST) Received: (qmail 8291 invoked from network); 12 Sep 2019 16:35:26 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.19.210]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 12 Sep 2019 16:35:26 -0000 Date: Thu, 12 Sep 2019 17:35:25 +0100 From: Mel Gorman To: Michal Hocko Subject: Re: [PATCH v9 0/8] stg mail -e --version=v9 \ Message-ID: <20190912163525.GV2739@techsingularity.net> References: <20190907172225.10910.34302.stgit@localhost.localdomain> <20190910124209.GY2063@dhcp22.suse.cz> <20190910144713.GF2063@dhcp22.suse.cz> <20190910175213.GD4023@dhcp22.suse.cz> <1d7de9f9f4074f67c567dbb4cc1497503d739e30.camel@linux.intel.com> <20190911113619.GP4023@dhcp22.suse.cz> <20190912091925.GM4023@dhcp22.suse.cz> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190912091925.GM4023@dhcp22.suse.cz> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190912_093537_747096_E69ABCA2 X-CRM114-Status: GOOD ( 31.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yang Zhang , Pankaj Gupta , kvm list , "Michael S. Tsirkin" , Catalin Marinas , Alexander Duyck , lcapitulino@redhat.com, linux-mm , Alexander Duyck , will@kernel.org, Andrea Arcangeli , virtio-dev@lists.oasis-open.org, David Hildenbrand , Matthew Wilcox , "Wang, Wei W" , Mel Gorman , ying.huang@intel.com, Rik van Riel , Vlastimil Babka , Dan Williams , linux-arm-kernel@lists.infradead.org, Oscar Salvador , Nitesh Narayan Lal , Konrad Rzeszutek Wilk , Dave Hansen , LKML , Paolo Bonzini , Andrew Morton , Fengguang Wu , "Kirill A. Shutemov" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Sep 12, 2019 at 11:19:25AM +0200, Michal Hocko wrote: > On Wed 11-09-19 08:12:03, Alexander Duyck wrote: > > On Wed, Sep 11, 2019 at 4:36 AM Michal Hocko wrote: > > > > > > On Tue 10-09-19 14:23:40, Alexander Duyck wrote: > > > [...] > > > > We don't put any limitations on the allocator other then that it needs to > > > > clean up the metadata on allocation, and that it cannot allocate a page > > > > that is in the process of being reported since we pulled it from the > > > > free_list. If the page is a "Reported" page then it decrements the > > > > reported_pages count for the free_area and makes sure the page doesn't > > > > exist in the "Boundary" array pointer value, if it does it moves the > > > > "Boundary" since it is pulling the page. > > > > > > This is still a non-trivial limitation on the page allocation from an > > > external code IMHO. I cannot give any explicit reason why an ordering on > > > the free list might matter (well except for page shuffling which uses it > > > to make physical memory pattern allocation more random) but the > > > architecture seems hacky and dubious to be honest. It shoulds like the > > > whole interface has been developed around a very particular and single > > > purpose optimization. > > > > How is this any different then the code that moves a page that will > > likely be merged to the tail though? > > I guess you are referring to the page shuffling. If that is the case > then this is an integral part of the allocator for a reason and it is > very well obvious in the code including the consequences. I do not > really like an idea of hiding similar constrains behind a generic > looking feature which is completely detached from the allocator and so > any future change of the allocator might subtly break it. > It's not just that, compaction pokes into the free_area information as well and directly takes pages from the free list without going through the page allocator itself. It assumes that a free page is a free page and only takes the zone and migratetype into account. > > In our case the "Reported" page is likely going to be much more > > expensive to allocate and use then a standard page because it will be > > faulted back in. In such a case wouldn't it make sense for us to want > > to keep the pages that don't require faults ahead of those pages in > > the free_list so that they are more likely to be allocated? > > OK, I was suspecting this would pop out. And this is exactly why I > didn't like an idea of an external code imposing a non obvious constrains > to the allocator. You simply cannot count with any ordering with the > page allocator. Indeed not. It can be arbitrary and compaction can interfere with the ordering as well. While in theory that could be addressed by always going through an interface maintained by the page allocator, it would be tricky to test the virtio case in particular. > We used to distinguish cache hot/cold pages in the past > and pushed pages to the specific end of the free list but that has been > removed. That was always best effort too, not a hard guarantee. It was eventually removed as the cost of figuring out the ordering exceeded the benefit. > There are other potential changes like that possible. Shuffling > is a good recent example. > > Anyway I am not a maintainer of this code. I would really like to hear > opinions from Mel and Vlastimil here (now CCed - the thread starts > http://lkml.kernel.org/r/20190907172225.10910.34302.stgit@localhost.localdomain. I worry that poking too much into the internal state of the allocator will be fragile long-term. There is the arch alloc/free hooks but they are typically about protections only and does not interfere with the internal state of the allocator. Compaction pokes in as well but once the page is off the free list, the page allocator no longer cares so again there is on interference with the internal state. If the state is interefered with externally, it becomes unclear what happens if things like page merging is deferred in a way the allocator cannot control as high-order allocation requests may fail for example. For THP, it would not matter but failed allocation reports when pages are on the freelist, but unsuitable for allocation because of the reported state, would be hard to debug. Similarly, latency issues due to a reported page being picked for allocation but requiring communication with the hypervisor will be difficult to debug and atomic allocations may fail entirely. Finally, if merging was broken for reported/unreported pages, it could be a long time before such bugs were fixed. That's a lot of caveats to optimise communication about unused free pages to the allocator. I didn't read the patches particularly carefully but it was not clear why a best effort was not made to track free pages and if the metadata maintenance for that fills then do exhaustive searches for remaining pages. It might be difficult to stabilise that as the metadata may overflow again while the exhaustive search takes place. Much would depend on the frequency that pages are entering/leaving reported state. -- Mel Gorman SUSE Labs _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel