From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D388C4361B for ; Mon, 7 Dec 2020 10:40:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1923123339 for ; Mon, 7 Dec 2020 10:40:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726320AbgLGKkl (ORCPT ); Mon, 7 Dec 2020 05:40:41 -0500 Received: from foss.arm.com ([217.140.110.172]:46698 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725842AbgLGKkk (ORCPT ); Mon, 7 Dec 2020 05:40:40 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1D43B1042; Mon, 7 Dec 2020 02:39:55 -0800 (PST) Received: from [10.163.86.92] (unknown [10.163.86.92]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0BC2A3F66B; Mon, 7 Dec 2020 02:39:50 -0800 (PST) Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page map To: Mike Rapoport , Ard Biesheuvel Cc: Barry Song , Linux ARM , Steve Capper , Marc Zyngier , Linux Kernel Mailing List , Wei Li , Catalin Marinas , butao@hisilicon.com, Will Deacon , Nicolas Saenz Julienne , fengbaopeng2@hisilicon.com References: <20201204014443.43329-1-liwei213@huawei.com> <20201204111347.GA844@willie-the-truck> <390f5f441d99a832f4b2425b46f6d971@kernel.org> <20201207094215.GC1112728@linux.ibm.com> <20201207100426.GE1112728@linux.ibm.com> From: Anshuman Khandual Message-ID: <39d72e1c-b3b3-89d3-1a1a-3ee222d40761@arm.com> Date: Mon, 7 Dec 2020 16:09:52 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20201207100426.GE1112728@linux.ibm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/7/20 3:34 PM, Mike Rapoport wrote: > On Mon, Dec 07, 2020 at 10:49:26AM +0100, Ard Biesheuvel wrote: >> On Mon, 7 Dec 2020 at 10:42, Mike Rapoport wrote: >>> >>> On Mon, Dec 07, 2020 at 09:35:06AM +0000, Marc Zyngier wrote: >>>> On 2020-12-07 09:09, Ard Biesheuvel wrote: >>>>> (+ Marc) >>>>> >>>>> On Fri, 4 Dec 2020 at 12:14, Will Deacon wrote: >>>>>> >>>>>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote: >>>>>>> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP >>>>>>> do not free the reserved memory for the page map, decrease the section >>>>>>> size can reduce the waste of reserved memory. >>>>>>> >>>>>>> Signed-off-by: Wei Li >>>>>>> Signed-off-by: Baopeng Feng >>>>>>> Signed-off-by: Xia Qing >>>>>>> --- >>>>>>> arch/arm64/include/asm/sparsemem.h | 2 +- >>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>>>> >>>>>>> diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h >>>>>>> index 1f43fcc79738..8963bd3def28 100644 >>>>>>> --- a/arch/arm64/include/asm/sparsemem.h >>>>>>> +++ b/arch/arm64/include/asm/sparsemem.h >>>>>>> @@ -7,7 +7,7 @@ >>>>>>> >>>>>>> #ifdef CONFIG_SPARSEMEM >>>>>>> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS >>>>>>> -#define SECTION_SIZE_BITS 30 >>>>>>> +#define SECTION_SIZE_BITS 27 >>>>>> >>>>>> We chose '30' to avoid running out of bits in the page flags. What >>>>>> changed? >>>>>> >>>>>> With this patch, I can trigger: >>>>>> >>>>>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds >>>>>> SECTION_SIZE >>>>>> #error Allocator MAX_ORDER exceeds SECTION_SIZE >>>>>> >>>>>> if I bump up NR_CPUS and NODES_SHIFT. >>>>>> >>>>> >>>>> Does this mean we will run into problems with the GICv3 ITS LPI tables >>>>> again if we are forced to reduce MAX_ORDER to fit inside >>>>> SECTION_SIZE_BITS? >>>> >>>> Most probably. We are already massively constraint on platforms >>>> such as TX1, and dividing the max allocatable range by 8 isn't >>>> going to make it work any better... >>> >>> I don't think MAX_ORDER should shrink. Even if SECTION_SIZE_BITS is >>> reduced it should accomodate the existing MAX_ORDER. >>> >>> My two pennies. >>> >> >> But include/linux/mmzone.h:1170 has this: >> >> #if (MAX_ORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS >> #error Allocator MAX_ORDER exceeds SECTION_SIZE >> #endif >> >> and Will managed to trigger it after applying this patch. > > Right, because with 64K pages section size of 27 bits is not enough to > accomodate MAX_ORDER (2^13 pages of 64K). > > Which means that definition of SECTION_SIZE_BITS should take MAX_ORDER > into account either statically with > > #ifdef ARM64_4K_PAGES > #define SECTION_SIZE_BITS > #elif ARM64_16K_PAGES > #define SECTION_SIZE_BITS > #elif ARM64_64K_PAGES > #define SECTION_SIZE_BITS > #else > #error "and what is the page size?" > #endif > > or dynamically, like e.g. ia64 does: > > #ifdef CONFIG_FORCE_MAX_ZONEORDER > #if ((CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS) > #undef SECTION_SIZE_BITS > #define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) > #endif I had proposed the same on the other thread here. But with this the SECTION_SIZE_BITS becomes 22 in case of 4K page size reducing to an extent where PMD based vmemmap mapping could not be created. Though have not looked into much details yet. Using CONFIG_FORCE_MAX_ZONEORDER seems to the right thing to do. But if that does not reasonably work for 4K pages, we might have to hard code it as 27 to have huge page vmemmap mappings. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11CE2C4361B for ; Mon, 7 Dec 2020 10:41:15 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 800C423341 for ; Mon, 7 Dec 2020 10:41:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 800C423341 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VBfHOrb/zbsi4VlBkhO6E1Ce2i1vQwjnX4WY98M6Tbs=; b=Vf+o006pWe8ffrvJApukCkQnJ 7QUQvsehKDea5iGBTWxPV54U2nRh8rJ/aomUYKLq7KeMKgV5CyBvaBOmdYbRbCbuHJjbBZ2rZMih8 vZlKatsbjGz+bGqh1yAVP9jp7CGe6PPhzfU625FnFnNNUprJKNkFDmmtpw5gPjWIQGyNCSPLRSrzt svwRjJztQ6klsvgePRxoh2VJgMIVt4fsRDGcF5idC2DT1EKSdiELNwmeAhEJnBsKibSCvMxwLjPHK y2S/NUfFDFlQE6mDVyDracoSU3D44FEZo+V9mXcXjubqE+WwmPgGWk2V8xK0pkU8NQ+Yl57MAqDnM Pr9LLk7bA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmDw2-0007RM-Dx; Mon, 07 Dec 2020 10:40:02 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmDvz-0007Ph-Vg for linux-arm-kernel@lists.infradead.org; Mon, 07 Dec 2020 10:40:01 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1D43B1042; Mon, 7 Dec 2020 02:39:55 -0800 (PST) Received: from [10.163.86.92] (unknown [10.163.86.92]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0BC2A3F66B; Mon, 7 Dec 2020 02:39:50 -0800 (PST) Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory reserved for the page map To: Mike Rapoport , Ard Biesheuvel References: <20201204014443.43329-1-liwei213@huawei.com> <20201204111347.GA844@willie-the-truck> <390f5f441d99a832f4b2425b46f6d971@kernel.org> <20201207094215.GC1112728@linux.ibm.com> <20201207100426.GE1112728@linux.ibm.com> From: Anshuman Khandual Message-ID: <39d72e1c-b3b3-89d3-1a1a-3ee222d40761@arm.com> Date: Mon, 7 Dec 2020 16:09:52 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20201207100426.GE1112728@linux.ibm.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201207_054000_133171_E6BA1545 X-CRM114-Status: GOOD ( 24.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Barry Song , Steve Capper , Marc Zyngier , Linux Kernel Mailing List , fengbaopeng2@hisilicon.com, Nicolas Saenz Julienne , Wei Li , Catalin Marinas , Will Deacon , Linux ARM , butao@hisilicon.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 12/7/20 3:34 PM, Mike Rapoport wrote: > On Mon, Dec 07, 2020 at 10:49:26AM +0100, Ard Biesheuvel wrote: >> On Mon, 7 Dec 2020 at 10:42, Mike Rapoport wrote: >>> >>> On Mon, Dec 07, 2020 at 09:35:06AM +0000, Marc Zyngier wrote: >>>> On 2020-12-07 09:09, Ard Biesheuvel wrote: >>>>> (+ Marc) >>>>> >>>>> On Fri, 4 Dec 2020 at 12:14, Will Deacon wrote: >>>>>> >>>>>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote: >>>>>>> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP >>>>>>> do not free the reserved memory for the page map, decrease the section >>>>>>> size can reduce the waste of reserved memory. >>>>>>> >>>>>>> Signed-off-by: Wei Li >>>>>>> Signed-off-by: Baopeng Feng >>>>>>> Signed-off-by: Xia Qing >>>>>>> --- >>>>>>> arch/arm64/include/asm/sparsemem.h | 2 +- >>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>>>> >>>>>>> diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h >>>>>>> index 1f43fcc79738..8963bd3def28 100644 >>>>>>> --- a/arch/arm64/include/asm/sparsemem.h >>>>>>> +++ b/arch/arm64/include/asm/sparsemem.h >>>>>>> @@ -7,7 +7,7 @@ >>>>>>> >>>>>>> #ifdef CONFIG_SPARSEMEM >>>>>>> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS >>>>>>> -#define SECTION_SIZE_BITS 30 >>>>>>> +#define SECTION_SIZE_BITS 27 >>>>>> >>>>>> We chose '30' to avoid running out of bits in the page flags. What >>>>>> changed? >>>>>> >>>>>> With this patch, I can trigger: >>>>>> >>>>>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds >>>>>> SECTION_SIZE >>>>>> #error Allocator MAX_ORDER exceeds SECTION_SIZE >>>>>> >>>>>> if I bump up NR_CPUS and NODES_SHIFT. >>>>>> >>>>> >>>>> Does this mean we will run into problems with the GICv3 ITS LPI tables >>>>> again if we are forced to reduce MAX_ORDER to fit inside >>>>> SECTION_SIZE_BITS? >>>> >>>> Most probably. We are already massively constraint on platforms >>>> such as TX1, and dividing the max allocatable range by 8 isn't >>>> going to make it work any better... >>> >>> I don't think MAX_ORDER should shrink. Even if SECTION_SIZE_BITS is >>> reduced it should accomodate the existing MAX_ORDER. >>> >>> My two pennies. >>> >> >> But include/linux/mmzone.h:1170 has this: >> >> #if (MAX_ORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS >> #error Allocator MAX_ORDER exceeds SECTION_SIZE >> #endif >> >> and Will managed to trigger it after applying this patch. > > Right, because with 64K pages section size of 27 bits is not enough to > accomodate MAX_ORDER (2^13 pages of 64K). > > Which means that definition of SECTION_SIZE_BITS should take MAX_ORDER > into account either statically with > > #ifdef ARM64_4K_PAGES > #define SECTION_SIZE_BITS > #elif ARM64_16K_PAGES > #define SECTION_SIZE_BITS > #elif ARM64_64K_PAGES > #define SECTION_SIZE_BITS > #else > #error "and what is the page size?" > #endif > > or dynamically, like e.g. ia64 does: > > #ifdef CONFIG_FORCE_MAX_ZONEORDER > #if ((CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS) > #undef SECTION_SIZE_BITS > #define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) > #endif I had proposed the same on the other thread here. But with this the SECTION_SIZE_BITS becomes 22 in case of 4K page size reducing to an extent where PMD based vmemmap mapping could not be created. Though have not looked into much details yet. Using CONFIG_FORCE_MAX_ZONEORDER seems to the right thing to do. But if that does not reasonably work for 4K pages, we might have to hard code it as 27 to have huge page vmemmap mappings. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel