From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02121C4321E for ; Fri, 7 Sep 2018 14:44:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BC69C2083D for ; Fri, 7 Sep 2018 14:44:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BC69C2083D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729950AbeIGTZs (ORCPT ); Fri, 7 Sep 2018 15:25:48 -0400 Received: from foss.arm.com ([217.140.101.70]:32980 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727821AbeIGTZr (ORCPT ); Fri, 7 Sep 2018 15:25:47 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 103D218A; Fri, 7 Sep 2018 07:44:33 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D455E3F575; Fri, 7 Sep 2018 07:44:32 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 078CE1AE3828; Fri, 7 Sep 2018 15:44:47 +0100 (BST) Date: Fri, 7 Sep 2018 15:44:47 +0100 From: Will Deacon To: Ard Biesheuvel Cc: Jia He , Russell King , Catalin Marinas , Mark Rutland , Andrew Morton , Michal Hocko , Wei Yang , Kees Cook , Laura Abbott , Vladimir Murzin , Philip Derrin , AKASHI Takahiro , James Morse , Steve Capper , Gioh Kim , Vlastimil Babka , Mel Gorman , Johannes Weiner , Kemi Wang , Petr Tesarik , YASUAKI ISHIMATSU , Andrey Ryabinin , Nikolay Borisov , Daniel Jordan , Daniel Vacek , Eugeniu Rosca , linux-arm-kernel , Linux Kernel Mailing List , Linux-MM , Jia He Subject: Re: [PATCH v11 0/3] remain and optimize memblock_next_valid_pfn on arm and arm64 Message-ID: <20180907144447.GD12788@arm.com> References: <1534907237-2982-1-git-send-email-jia.he@hxt-semitech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Sep 06, 2018 at 01:24:22PM +0200, Ard Biesheuvel wrote: > On 22 August 2018 at 05:07, Jia He wrote: > > Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns > > where possible") optimized the loop in memmap_init_zone(). But it causes > > possible panic bug. So Daniel Vacek reverted it later. > > > > But as suggested by Daniel Vacek, it is fine to using memblock to skip > > gaps and finding next valid frame with CONFIG_HAVE_ARCH_PFN_VALID. > > > > More from what Daniel said: > > "On arm and arm64, memblock is used by default. But generic version of > > pfn_valid() is based on mem sections and memblock_next_valid_pfn() does > > not always return the next valid one but skips more resulting in some > > valid frames to be skipped (as if they were invalid). And that's why > > kernel was eventually crashing on some !arm machines." > > > > About the performance consideration: > > As said by James in b92df1de5, > > "I have tested this patch on a virtual model of a Samurai CPU with a > > sparse memory map. The kernel boot time drops from 109 to 62 seconds." > > Thus it would be better if we remain memblock_next_valid_pfn on arm/arm64. > > > > Besides we can remain memblock_next_valid_pfn, there is still some room > > for improvement. After this set, I can see the time overhead of memmap_init > > is reduced from 27956us to 13537us in my armv8a server(QDF2400 with 96G > > memory, pagesize 64k). I believe arm server will benefit more if memory is > > larger than TBs > > > > OK so we can summarize the benefits of this series as follows: > - boot time on a virtual model of a Samurai CPU drops from 109 to 62 seconds > - boot time on a QDF2400 arm64 server with 96 GB of RAM drops by ~15 > *milliseconds* > > Google was not very helpful in figuring out what a Samurai CPU is and > why we should care about the boot time of Linux running on a virtual > model of it, and the 15 ms speedup is not that compelling either. > > Apologies to Jia that it took 11 revisions to reach this conclusion, > but in /my/ opinion, tweaking the fragile memblock/pfn handling code > for this reason is totally unjustified, and we're better off > disregarding these patches. Oh, we're talking about a *simulator* for the significant boot time improvement here? I didn't realise that, so I agree that the premise of this patch set looks pretty questionable given how much "fun" we've had with the memmap on arm and arm64. Will From mboxrd@z Thu Jan 1 00:00:00 1970 From: will.deacon@arm.com (Will Deacon) Date: Fri, 7 Sep 2018 15:44:47 +0100 Subject: [PATCH v11 0/3] remain and optimize memblock_next_valid_pfn on arm and arm64 In-Reply-To: References: <1534907237-2982-1-git-send-email-jia.he@hxt-semitech.com> Message-ID: <20180907144447.GD12788@arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Thu, Sep 06, 2018 at 01:24:22PM +0200, Ard Biesheuvel wrote: > On 22 August 2018 at 05:07, Jia He wrote: > > Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns > > where possible") optimized the loop in memmap_init_zone(). But it causes > > possible panic bug. So Daniel Vacek reverted it later. > > > > But as suggested by Daniel Vacek, it is fine to using memblock to skip > > gaps and finding next valid frame with CONFIG_HAVE_ARCH_PFN_VALID. > > > > More from what Daniel said: > > "On arm and arm64, memblock is used by default. But generic version of > > pfn_valid() is based on mem sections and memblock_next_valid_pfn() does > > not always return the next valid one but skips more resulting in some > > valid frames to be skipped (as if they were invalid). And that's why > > kernel was eventually crashing on some !arm machines." > > > > About the performance consideration: > > As said by James in b92df1de5, > > "I have tested this patch on a virtual model of a Samurai CPU with a > > sparse memory map. The kernel boot time drops from 109 to 62 seconds." > > Thus it would be better if we remain memblock_next_valid_pfn on arm/arm64. > > > > Besides we can remain memblock_next_valid_pfn, there is still some room > > for improvement. After this set, I can see the time overhead of memmap_init > > is reduced from 27956us to 13537us in my armv8a server(QDF2400 with 96G > > memory, pagesize 64k). I believe arm server will benefit more if memory is > > larger than TBs > > > > OK so we can summarize the benefits of this series as follows: > - boot time on a virtual model of a Samurai CPU drops from 109 to 62 seconds > - boot time on a QDF2400 arm64 server with 96 GB of RAM drops by ~15 > *milliseconds* > > Google was not very helpful in figuring out what a Samurai CPU is and > why we should care about the boot time of Linux running on a virtual > model of it, and the 15 ms speedup is not that compelling either. > > Apologies to Jia that it took 11 revisions to reach this conclusion, > but in /my/ opinion, tweaking the fragile memblock/pfn handling code > for this reason is totally unjustified, and we're better off > disregarding these patches. Oh, we're talking about a *simulator* for the significant boot time improvement here? I didn't realise that, so I agree that the premise of this patch set looks pretty questionable given how much "fun" we've had with the memmap on arm and arm64. Will