From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAF31C4320A for ; Thu, 19 Aug 2021 07:01:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 606CB6101A for ; Thu, 19 Aug 2021 07:01:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 606CB6101A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 078256B0072; Thu, 19 Aug 2021 03:01:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 026796B0073; Thu, 19 Aug 2021 03:01:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E30C36B0074; Thu, 19 Aug 2021 03:01:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0181.hostedemail.com [216.40.44.181]) by kanga.kvack.org (Postfix) with ESMTP id C92816B0072 for ; Thu, 19 Aug 2021 03:01:30 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 667AC181D0289 for ; Thu, 19 Aug 2021 07:01:30 +0000 (UTC) X-FDA: 78490934340.01.78333F9 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf05.hostedemail.com (Postfix) with ESMTP id 19468506A0AF for ; Thu, 19 Aug 2021 07:01:29 +0000 (UTC) Received: by mail-pg1-f175.google.com with SMTP id q2so4972248pgt.6 for ; Thu, 19 Aug 2021 00:01:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OTxY68Ufdm8UNqRMLWBsmc/KZ1S77UCqKKx7OF0ikR0=; b=udSVxlwMVtCgOdal9skEcrtOOlkbpjElZ0fwLOo8FwTTHT/Jnxl5qRS//fZ+inFeq4 N6U7m4ES00lY1g5Z3nWRJfQISLuH9mIWJkadx3e5lZRPgwjcsKr+nZbLba83Gqh5MrUe LVIOKysRnfasE0FG2JvPu1aoqc/k41nHT5qNQ7/bwHmx92z2o/Ewg+mIzMcD10ng3GMa pldchSUP694vbNs590DJbMGSx4cVT7R4kbvTonTKSkCeZ23CLFCa4RjfpGw2o142Cdvt 4o86NswXubR6AROFGWZ4FzR0zWuXqtYb+mTvoWHt8RuRgrfQ5JZjhYC/+VbK0V3Vtc32 xn9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OTxY68Ufdm8UNqRMLWBsmc/KZ1S77UCqKKx7OF0ikR0=; b=KtfEnV3d+JfeAuEithbM0GgKaknf+raj0wYR5tLXjvKy5TdRvSmOykmZWsuojE4zqY 0MIaZysEAji4RQD47tLlfvP5P8KCK1ZWpA7UfqsaSfA++4cSteNSjKkZgO0U6RZBYf3N 3cv4KnFiHwTzoSaYKNMrV4bUjH1N/PQuxSf8bwwrwr6m43Ro7RfzWU4+0WRPZLXzygEB hCp2LF2cmUhiSDA7qCZOb2jnY3oQT4UWmcvF3Cjj07AFRnWul9Sx8XgXsNhKZyz58Nlb 2FKuumaRbSjzzrjunywwEbtE/XN+DsXJ5rSwSHJRrbCujIwL/3WT9jjT94VUdbMim87i rYsw== X-Gm-Message-State: AOAM533tkXM9GADTwG/VbNc0CTkGV+/mLGr1ImYI0BKzpnYkJlcDUiro Xee5XQNa4x6nqafniT5ykC30Bg== X-Google-Smtp-Source: ABdhPJz6BNvUtnn0IBiNO/ph123G8mcdkJEcNpmKIzPsWUA724ZXomsEWhuBZTvbLwPKu0WhJiQCVA== X-Received: by 2002:aa7:86d5:0:b0:3e1:abc7:890b with SMTP id h21-20020aa786d5000000b003e1abc7890bmr13279465pfo.4.1629356489253; Thu, 19 Aug 2021 00:01:29 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id t30sm2490395pgl.47.2021.08.19.00.01.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Aug 2021 00:01:28 -0700 (PDT) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org, osalvador@suse.de, mhocko@suse.com, song.bao.hua@hisilicon.com, david@redhat.com, chenhuang5@huawei.com, bodeddub@amazon.com, corbet@lwn.net, willy@infradead.org Cc: duanxiongchun@bytedance.com, fam.zheng@bytedance.com, smuchun@gmail.com, zhengqi.arch@bytedance.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH v2 2/4] mm: hugetlb: replace hugetlb_free_vmemmap_enabled with a static_key Date: Thu, 19 Aug 2021 14:58:29 +0800 Message-Id: <20210819065831.43186-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210819065831.43186-1-songmuchun@bytedance.com> References: <20210819065831.43186-1-songmuchun@bytedance.com> MIME-Version: 1.0 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=bytedance-com.20150623.gappssmtp.com header.s=20150623 header.b=udSVxlwM; spf=pass (imf05.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 19468506A0AF X-Stat-Signature: ethc1tpuaxq3g7kpzcx47h3b8yrc1pik X-HE-Tag: 1629356489-813520 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The page_head_if_fake() is used throughout memory management and the conditional check requires checking a global variable, although the overhead of this check may be small, it increases when the memory cache comes under pressure. Also, the global variable will not be modified after system boot, so it is very appropriate to use static key machanism. Signed-off-by: Muchun Song --- include/linux/hugetlb.h | 6 +++++- include/linux/page-flags.h | 6 ++++-- mm/hugetlb_vmemmap.c | 10 +++++----- 3 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index f7ca1a3870ea..ee3ddf3d12cf 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1057,7 +1057,11 @@ static inline void set_huge_swap_pte_at(struct mm_= struct *mm, unsigned long addr #endif /* CONFIG_HUGETLB_PAGE */ =20 #ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP -extern bool hugetlb_free_vmemmap_enabled; +DECLARE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, + hugetlb_free_vmemmap_enabled_key); +#define hugetlb_free_vmemmap_enabled \ + static_key_enabled(&hugetlb_free_vmemmap_enabled_key) + #else #define hugetlb_free_vmemmap_enabled false #endif diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 7b1a918ebd43..d68d2cf30d76 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -185,7 +185,8 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H =20 #ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP -extern bool hugetlb_free_vmemmap_enabled; +DECLARE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, + hugetlb_free_vmemmap_enabled_key); =20 /* * If the feature of freeing some vmemmap pages associated with each Hug= eTLB @@ -204,7 +205,8 @@ extern bool hugetlb_free_vmemmap_enabled; */ static __always_inline const struct page *page_head_if_fake(const struct= page *page) { - if (!hugetlb_free_vmemmap_enabled) + if (!static_branch_maybe(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, + &hugetlb_free_vmemmap_enabled_key)) return page; =20 /* diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 527bcaa44a48..5b80129c684c 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -188,9 +188,9 @@ #define RESERVE_VMEMMAP_NR 1U #define RESERVE_VMEMMAP_SIZE (RESERVE_VMEMMAP_NR << PAGE_SHIFT) =20 -bool hugetlb_free_vmemmap_enabled __read_mostly =3D - IS_ENABLED(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON); -EXPORT_SYMBOL(hugetlb_free_vmemmap_enabled); +DEFINE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, + hugetlb_free_vmemmap_enabled_key); +EXPORT_SYMBOL(hugetlb_free_vmemmap_enabled_key); =20 static int __init early_hugetlb_free_vmemmap_param(char *buf) { @@ -204,9 +204,9 @@ static int __init early_hugetlb_free_vmemmap_param(ch= ar *buf) return -EINVAL; =20 if (!strcmp(buf, "on")) - hugetlb_free_vmemmap_enabled =3D true; + static_branch_enable(&hugetlb_free_vmemmap_enabled_key); else if (!strcmp(buf, "off")) - hugetlb_free_vmemmap_enabled =3D false; + static_branch_disable(&hugetlb_free_vmemmap_enabled_key); else return -EINVAL; =20 --=20 2.11.0