From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2DF4C63777 for ; Fri, 20 Nov 2020 10:40:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 45CD022264 for ; Fri, 20 Nov 2020 10:40:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="QXuoT0YG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727543AbgKTKj4 (ORCPT ); Fri, 20 Nov 2020 05:39:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727430AbgKTKjz (ORCPT ); Fri, 20 Nov 2020 05:39:55 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A748EC061A04 for ; Fri, 20 Nov 2020 02:39:55 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id d17so4650916plr.5 for ; Fri, 20 Nov 2020 02:39:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Tp1Yn6PcYwDveyfmiNUMjqoohGn5JgQdU6bb6b54s7c=; b=QXuoT0YGe3HDxz+HXbH17h6UKx5WhqJBvvvjMJBBZq2FNc1zrSX59gmBtx3vzBZalN JT69SsXhQtlq1zgI7hllhxrbDFAWelFFMocFYKNiAHFPd5YNckbSBCW1yYHdh5TG2aQL 86Qjz9M3UfKWKIdCX60Vb3vV9Nw/Si4ZgfNwJiqW8V5+i+oGqzDL+Uld/Y+1WfRtkHSK WRDFJirytKoexo3twKJEkEJiKOJNBPqHqr5pHmcKGY+1KcjFueMlE++7eWtExSbgVy0R C5Ottl56Y88HOgabvGotKsHw4tK3CnHE2/6KOZHcaAaWTxdEIcT6NkeMwHIkSdKIXeuS lwZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Tp1Yn6PcYwDveyfmiNUMjqoohGn5JgQdU6bb6b54s7c=; b=sZgUZdfrBufUB6NWwWwcckd6/mP2HFkhslFz+dF82towoRPDL4En1VJdv1/hdVSawN ZFlJYHc9/wZ5H8jZRNQ1rLPJYQXaelqfWVEhvcpp7hQeCf2uHJ+NCtlVKEx7irLVD6K8 XRz5sJ6HHnGyyOT7j2gAZSi8o1JWCaCXw20n6TO5A91xPjDR6r/LGH6mz/fl/ej7F82z qjolovGIHGoG1TPn5e6W4USws10beNr7t/wOYYAz0G7W+JN52hqV0oG19I2j3l9pB7bO pf8DSzz9xB1WmPY9wWselSUIzzsx24IbVWUqOSCg8ZgOpdEsU7Uz4nn2rWwVMCMyCrzB I+Tw== X-Gm-Message-State: AOAM531ym3brJEYDm8bJS3N6Ii/QD2qE4u5lyMNX+vayu7FQBbtb+AUs 6IL7/yKASqXKaPIKfvX7LZc5kcD8u/PZ6KDyyD0jTg== X-Google-Smtp-Source: ABdhPJwyeHponJyLYkFShPtIMQjhm14hpbv9nuNdyb0slhePQUiE+rmZ2muvx54FRFWkF7TSl9xDPloHXXLtI1r7PYw= X-Received: by 2002:a17:902:c14b:b029:d6:ab18:108d with SMTP id 11-20020a170902c14bb02900d6ab18108dmr13501415plj.20.1605868795137; Fri, 20 Nov 2020 02:39:55 -0800 (PST) MIME-Version: 1.0 References: <20201120064325.34492-1-songmuchun@bytedance.com> <20201120064325.34492-18-songmuchun@bytedance.com> <20201120082212.GG3200@dhcp22.suse.cz> In-Reply-To: <20201120082212.GG3200@dhcp22.suse.cz> From: Muchun Song Date: Fri, 20 Nov 2020 18:39:12 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v5 17/21] mm/hugetlb: Add a kernel parameter hugetlb_free_vmemmap To: Michal Hocko Cc: Jonathan Corbet , Mike Kravetz , Thomas Gleixner , mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, Peter Zijlstra , viro@zeniv.linux.org.uk, Andrew Morton , paulmck@kernel.org, mchehab+huawei@kernel.org, pawan.kumar.gupta@linux.intel.com, Randy Dunlap , oneukum@suse.com, anshuman.khandual@arm.com, jroedel@suse.de, Mina Almasry , David Rientjes , Matthew Wilcox , Oscar Salvador , "Song Bao Hua (Barry Song)" , Xiongchun duan , linux-doc@vger.kernel.org, LKML , Linux Memory Management List , linux-fsdevel Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Nov 20, 2020 at 4:22 PM Michal Hocko wrote: > > On Fri 20-11-20 14:43:21, Muchun Song wrote: > > Add a kernel parameter hugetlb_free_vmemmap to disable the feature of > > freeing unused vmemmap pages associated with each hugetlb page on boot. > > As replied to the config patch. This is fine but I would argue that the > default should be flipped. Saving memory is nice but it comes with > overhead and therefore should be an opt-in. The config option should > only guard compile time dependencies not a user choice. Got it. The default will be flipped in the next version. Thanks. > > > Signed-off-by: Muchun Song > > --- > > Documentation/admin-guide/kernel-parameters.txt | 9 +++++++++ > > Documentation/admin-guide/mm/hugetlbpage.rst | 3 +++ > > mm/hugetlb_vmemmap.c | 21 +++++++++++++++++++++ > > 3 files changed, 33 insertions(+) > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > > index 5debfe238027..ccf07293cb63 100644 > > --- a/Documentation/admin-guide/kernel-parameters.txt > > +++ b/Documentation/admin-guide/kernel-parameters.txt > > @@ -1551,6 +1551,15 @@ > > Documentation/admin-guide/mm/hugetlbpage.rst. > > Format: size[KMG] > > > > + hugetlb_free_vmemmap= > > + [KNL] When CONFIG_HUGETLB_PAGE_FREE_VMEMMAP is set, > > + this controls freeing unused vmemmap pages associated > > + with each HugeTLB page. > > + Format: { on (default) | off } > > + > > + on: enable the feature > > + off: disable the feature > > + > > hung_task_panic= > > [KNL] Should the hung task detector generate panics. > > Format: 0 | 1 > > diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst > > index f7b1c7462991..7d6129ee97dd 100644 > > --- a/Documentation/admin-guide/mm/hugetlbpage.rst > > +++ b/Documentation/admin-guide/mm/hugetlbpage.rst > > @@ -145,6 +145,9 @@ default_hugepagesz > > > > will all result in 256 2M huge pages being allocated. Valid default > > huge page size is architecture dependent. > > +hugetlb_free_vmemmap > > + When CONFIG_HUGETLB_PAGE_FREE_VMEMMAP is set, this disables freeing > > + unused vmemmap pages associated each HugeTLB page. > > > > When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages`` > > indicates the current number of pre-allocated huge pages of the default size. > > diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c > > index 3629165d8158..c958699d1393 100644 > > --- a/mm/hugetlb_vmemmap.c > > +++ b/mm/hugetlb_vmemmap.c > > @@ -144,6 +144,22 @@ static inline bool vmemmap_pmd_huge(pmd_t *pmd) > > } > > #endif > > > > +static bool hugetlb_free_vmemmap_disabled __initdata; > > + > > +static int __init early_hugetlb_free_vmemmap_param(char *buf) > > +{ > > + if (!buf) > > + return -EINVAL; > > + > > + if (!strcmp(buf, "off")) > > + hugetlb_free_vmemmap_disabled = true; > > + else if (strcmp(buf, "on")) > > + return -EINVAL; > > + > > + return 0; > > +} > > +early_param("hugetlb_free_vmemmap", early_hugetlb_free_vmemmap_param); > > + > > static inline unsigned int vmemmap_pages_per_hpage(struct hstate *h) > > { > > return free_vmemmap_pages_per_hpage(h) + RESERVE_VMEMMAP_NR; > > @@ -541,6 +557,11 @@ void __init hugetlb_vmemmap_init(struct hstate *h) > > unsigned int order = huge_page_order(h); > > unsigned int vmemmap_pages; > > > > + if (hugetlb_free_vmemmap_disabled) { > > + pr_info("disable free vmemmap pages for %s\n", h->name); > > + return; > > + } > > + > > vmemmap_pages = ((1 << order) * sizeof(struct page)) >> PAGE_SHIFT; > > /* > > * The head page and the first tail page are not to be freed to buddy > > -- > > 2.11.0 > > -- > Michal Hocko > SUSE Labs -- Yours, Muchun From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85589C56202 for ; Fri, 20 Nov 2020 10:39:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CAEB422255 for ; Fri, 20 Nov 2020 10:39:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="QXuoT0YG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CAEB422255 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0FC7D6B005D; Fri, 20 Nov 2020 05:39:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D3D06B0068; Fri, 20 Nov 2020 05:39:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2B146B006C; Fri, 20 Nov 2020 05:39:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0129.hostedemail.com [216.40.44.129]) by kanga.kvack.org (Postfix) with ESMTP id B911A6B005D for ; Fri, 20 Nov 2020 05:39:57 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 3CE5F1EF1 for ; Fri, 20 Nov 2020 10:39:57 +0000 (UTC) X-FDA: 77504451234.05.can97_5e0bdda2734a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 1A8CA1801D007 for ; Fri, 20 Nov 2020 10:39:57 +0000 (UTC) X-HE-Tag: can97_5e0bdda2734a X-Filterd-Recvd-Size: 7918 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Fri, 20 Nov 2020 10:39:56 +0000 (UTC) Received: by mail-pl1-f195.google.com with SMTP id bj5so3841393plb.4 for ; Fri, 20 Nov 2020 02:39:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Tp1Yn6PcYwDveyfmiNUMjqoohGn5JgQdU6bb6b54s7c=; b=QXuoT0YGe3HDxz+HXbH17h6UKx5WhqJBvvvjMJBBZq2FNc1zrSX59gmBtx3vzBZalN JT69SsXhQtlq1zgI7hllhxrbDFAWelFFMocFYKNiAHFPd5YNckbSBCW1yYHdh5TG2aQL 86Qjz9M3UfKWKIdCX60Vb3vV9Nw/Si4ZgfNwJiqW8V5+i+oGqzDL+Uld/Y+1WfRtkHSK WRDFJirytKoexo3twKJEkEJiKOJNBPqHqr5pHmcKGY+1KcjFueMlE++7eWtExSbgVy0R C5Ottl56Y88HOgabvGotKsHw4tK3CnHE2/6KOZHcaAaWTxdEIcT6NkeMwHIkSdKIXeuS lwZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Tp1Yn6PcYwDveyfmiNUMjqoohGn5JgQdU6bb6b54s7c=; b=L5v8x34PPCex1CS+FMtM+rPiCAWLD3L1SoAuHOmp5gbdL0TkX4/G0VDf6Hyf9LYw9X 3xmiKDTwmPcUMp1HMjyGFvZwW4YTc1CMxZqoD21l8RQwbE5aMLN1EcSYa8Oh0U/gmoJG TUjKWH6Oh+aXu5PH3bLLAV2GQo1PQyLpkzBMwLJvVhatENnTSacvtiey2qq3W6O0xocT jYhqnjpSzNjizSrG5yFkq+9BYpzfp9xY+0KM9kqhmyS0+BzRe7vVDKoUciU64uEx3GVO 1xI+Sy6zG5ZDHVxgLilBpXIhgvygFKhQdr+np88jt0oRB+OGIUNCUkedEcy7O+RhQoup RVfA== X-Gm-Message-State: AOAM531qMDGiLb15N0enAlq/Ax2Z93d0kEnpDaspO+xOqPQhB9se29qR N9NkFvWlFUCZnl82tFzI9HDRrdecNDBc1OhYrAsJdw== X-Google-Smtp-Source: ABdhPJwyeHponJyLYkFShPtIMQjhm14hpbv9nuNdyb0slhePQUiE+rmZ2muvx54FRFWkF7TSl9xDPloHXXLtI1r7PYw= X-Received: by 2002:a17:902:c14b:b029:d6:ab18:108d with SMTP id 11-20020a170902c14bb02900d6ab18108dmr13501415plj.20.1605868795137; Fri, 20 Nov 2020 02:39:55 -0800 (PST) MIME-Version: 1.0 References: <20201120064325.34492-1-songmuchun@bytedance.com> <20201120064325.34492-18-songmuchun@bytedance.com> <20201120082212.GG3200@dhcp22.suse.cz> In-Reply-To: <20201120082212.GG3200@dhcp22.suse.cz> From: Muchun Song Date: Fri, 20 Nov 2020 18:39:12 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v5 17/21] mm/hugetlb: Add a kernel parameter hugetlb_free_vmemmap To: Michal Hocko Cc: Jonathan Corbet , Mike Kravetz , Thomas Gleixner , mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, Peter Zijlstra , viro@zeniv.linux.org.uk, Andrew Morton , paulmck@kernel.org, mchehab+huawei@kernel.org, pawan.kumar.gupta@linux.intel.com, Randy Dunlap , oneukum@suse.com, anshuman.khandual@arm.com, jroedel@suse.de, Mina Almasry , David Rientjes , Matthew Wilcox , Oscar Salvador , "Song Bao Hua (Barry Song)" , Xiongchun duan , linux-doc@vger.kernel.org, LKML , Linux Memory Management List , linux-fsdevel Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Nov 20, 2020 at 4:22 PM Michal Hocko wrote: > > On Fri 20-11-20 14:43:21, Muchun Song wrote: > > Add a kernel parameter hugetlb_free_vmemmap to disable the feature of > > freeing unused vmemmap pages associated with each hugetlb page on boot. > > As replied to the config patch. This is fine but I would argue that the > default should be flipped. Saving memory is nice but it comes with > overhead and therefore should be an opt-in. The config option should > only guard compile time dependencies not a user choice. Got it. The default will be flipped in the next version. Thanks. > > > Signed-off-by: Muchun Song > > --- > > Documentation/admin-guide/kernel-parameters.txt | 9 +++++++++ > > Documentation/admin-guide/mm/hugetlbpage.rst | 3 +++ > > mm/hugetlb_vmemmap.c | 21 +++++++++++++++++++++ > > 3 files changed, 33 insertions(+) > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > > index 5debfe238027..ccf07293cb63 100644 > > --- a/Documentation/admin-guide/kernel-parameters.txt > > +++ b/Documentation/admin-guide/kernel-parameters.txt > > @@ -1551,6 +1551,15 @@ > > Documentation/admin-guide/mm/hugetlbpage.rst. > > Format: size[KMG] > > > > + hugetlb_free_vmemmap= > > + [KNL] When CONFIG_HUGETLB_PAGE_FREE_VMEMMAP is set, > > + this controls freeing unused vmemmap pages associated > > + with each HugeTLB page. > > + Format: { on (default) | off } > > + > > + on: enable the feature > > + off: disable the feature > > + > > hung_task_panic= > > [KNL] Should the hung task detector generate panics. > > Format: 0 | 1 > > diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst > > index f7b1c7462991..7d6129ee97dd 100644 > > --- a/Documentation/admin-guide/mm/hugetlbpage.rst > > +++ b/Documentation/admin-guide/mm/hugetlbpage.rst > > @@ -145,6 +145,9 @@ default_hugepagesz > > > > will all result in 256 2M huge pages being allocated. Valid default > > huge page size is architecture dependent. > > +hugetlb_free_vmemmap > > + When CONFIG_HUGETLB_PAGE_FREE_VMEMMAP is set, this disables freeing > > + unused vmemmap pages associated each HugeTLB page. > > > > When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages`` > > indicates the current number of pre-allocated huge pages of the default size. > > diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c > > index 3629165d8158..c958699d1393 100644 > > --- a/mm/hugetlb_vmemmap.c > > +++ b/mm/hugetlb_vmemmap.c > > @@ -144,6 +144,22 @@ static inline bool vmemmap_pmd_huge(pmd_t *pmd) > > } > > #endif > > > > +static bool hugetlb_free_vmemmap_disabled __initdata; > > + > > +static int __init early_hugetlb_free_vmemmap_param(char *buf) > > +{ > > + if (!buf) > > + return -EINVAL; > > + > > + if (!strcmp(buf, "off")) > > + hugetlb_free_vmemmap_disabled = true; > > + else if (strcmp(buf, "on")) > > + return -EINVAL; > > + > > + return 0; > > +} > > +early_param("hugetlb_free_vmemmap", early_hugetlb_free_vmemmap_param); > > + > > static inline unsigned int vmemmap_pages_per_hpage(struct hstate *h) > > { > > return free_vmemmap_pages_per_hpage(h) + RESERVE_VMEMMAP_NR; > > @@ -541,6 +557,11 @@ void __init hugetlb_vmemmap_init(struct hstate *h) > > unsigned int order = huge_page_order(h); > > unsigned int vmemmap_pages; > > > > + if (hugetlb_free_vmemmap_disabled) { > > + pr_info("disable free vmemmap pages for %s\n", h->name); > > + return; > > + } > > + > > vmemmap_pages = ((1 << order) * sizeof(struct page)) >> PAGE_SHIFT; > > /* > > * The head page and the first tail page are not to be freed to buddy > > -- > > 2.11.0 > > -- > Michal Hocko > SUSE Labs -- Yours, Muchun