From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 098ECC433DB for ; Tue, 9 Mar 2021 21:43:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8976E64FC0 for ; Tue, 9 Mar 2021 21:43:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8976E64FC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1BC338D0149; Tue, 9 Mar 2021 16:43:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1459E8D0148; Tue, 9 Mar 2021 16:43:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB2E78D0149; Tue, 9 Mar 2021 16:43:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0177.hostedemail.com [216.40.44.177]) by kanga.kvack.org (Postfix) with ESMTP id C95098D000F for ; Tue, 9 Mar 2021 16:43:11 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 77D6A611A for ; Tue, 9 Mar 2021 21:43:11 +0000 (UTC) X-FDA: 77901661782.02.3BCDFE3 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf16.hostedemail.com (Postfix) with ESMTP id A8D0D801914A for ; Tue, 9 Mar 2021 21:43:10 +0000 (UTC) Received: by mail-pj1-f50.google.com with SMTP id a22-20020a17090aa516b02900c1215e9b33so5746558pjq.5 for ; Tue, 09 Mar 2021 13:43:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uJhcYxfyQggjoFPfbV3IBwgqNX6derqJ/0OIfES3kng=; b=jL8KGxLZpq6kPeT9F+1qoL3DOavcSq+bJ2zvXDYPWXl+6YiuZBCwwRfCE7vfScpXYo JLjeD2AAKSyeha5oGCTqtO44VFZbSaDU8csNlwFLUVsaobMtQxYqvBKu4ZTMPr3ykAR6 o5hUKHMrDsMF1SIVJQ8Kd1W1dLLgNv1pQqr3M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uJhcYxfyQggjoFPfbV3IBwgqNX6derqJ/0OIfES3kng=; b=Exh2eaT+8gaJCG8yb2O1evdIQuXHUJq09QbFXv5G/1rEFc++p/0dCmJVpZvfpbxM4z ZpWcfYZiGGAyYNwUAxUtwUCUpceXlPOMYpxi3Hcv3nG1k81jG2hKTbr13zN8Mr3sBzn3 JbogvaT4OCa7cQHb63khTfRBQ6KQqWWrx1gotcKi8IVjQxU/2IoH+16fRnxApkUr/Lf0 AHAgNBWcA81sMjHLpAJOSSeCgn4bWLPA7PlLKDN+i7hrRjB8yuYMrXnNjlQwCANu5431 gJhBYIoqdULTKBZe3vZVaOnnmGStEzls0T7ioPOw/dOqbVlXzgNASSFaLhEhF/si8I4k 3YQQ== X-Gm-Message-State: AOAM531SKR8C7YDuWF9OVqTpR8GuGjlTCNl+ph5tJXlqGF3PGpHeLebD FlV19kYuBL9u+7HFag9Zh9GWrg== X-Google-Smtp-Source: ABdhPJzbrDwJvtujX+Yvv98LI2Jlt3lOh8R9CznfILmERfEu/fMLAFatr8X7tXLsHQfwfaOl+hu6bQ== X-Received: by 2002:a17:902:ec83:b029:e3:ec1f:9def with SMTP id x3-20020a170902ec83b02900e3ec1f9defmr107248plg.11.1615326189697; Tue, 09 Mar 2021 13:43:09 -0800 (PST) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id d20sm3872315pjv.47.2021.03.09.13.43.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 13:43:09 -0800 (PST) From: Kees Cook To: Thomas Gleixner Cc: Kees Cook , stable@vger.kernel.org, Elena Reshetova , x86@kernel.org, Andy Lutomirski , Peter Zijlstra , Catalin Marinas , Will Deacon , Mark Rutland , Alexander Potapenko , Alexander Popov , Ard Biesheuvel , Jann Horn , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vlastimil Babka , David Hildenbrand , Mike Rapoport , Andrew Morton , Jonathan Corbet , Randy Dunlap Subject: [PATCH v5 1/7] mm: Restore init_on_* static branch defaults Date: Tue, 9 Mar 2021 13:42:55 -0800 Message-Id: <20210309214301.678739-2-keescook@chromium.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309214301.678739-1-keescook@chromium.org> References: <20210309214301.678739-1-keescook@chromium.org> MIME-Version: 1.0 X-Patch-Hashes: v=1; h=sha256; g=28f3f839c24b05670130f4ff25d00e52971fd524; i=F1m1wEnux/IiRlF/sb2Zhr2mAPnBBAdImHuXCXK/7kg=; m=obdLQlMNg29JMC3GLL05bs7+HEYTRXfdcOPcqLYs1G0=; p=5DbOxVDVK+l3oSqEiw8KVTN/ODiEU3l6O/gw4X79jro= X-Patch-Sig: m=pgp; i=keescook@chromium.org; s=0x0x8972F4DFDC6DC026; b=iQIzBAABCgAdFiEEpcP2jyKd1g9yPm4TiXL039xtwCYFAmBH6+MACgkQiXL039xtwCZfHA//cAa PLXroyUxGbCfvAX2WTtwU6C5LPDtgM2cyDkhcMFkbY7jpjetZH58AgmsDVICQO5Gr1o6pcLanTaCu g9wn5kl+oYBuX+AM6N/RBjx2uuFb9H8vxS1sbup+S68TfmMPoa+8iLJYnDdQ5FeUCLzgIgdIoWyGC 1ebqyiytd8kEuZ54soVilY8JhdJ9Uyy2lj2CNuLncTdTZpB30J86I1fpIp8kG/1EqzwfOsrfK5WNE PlLp/tQYnPZw2LAxJg4fxq1OPvdXYXjOY6kVcx/tCaY3k4sEZjf+CA+N5ubqNZNQrcPo6yoUCuP34 iJpjSPR6EpsPimnEBVqI5dOnKcRbrQO/B96RbgbSqdGJckEiZk+/3x98oHJ7x0ZtQW/IDK82joRRf ZufV3KvwN1iLF2M0I1POqcKAk+9OF8mG/RA6pVK16vJmCA1iHTfLbz2JDUfWSu5rhFOYPCYs806V7 K8BpUEJz//Q38t6OXtzd/NuChkoMeEokmktZlFzmzLlk9qbAIWUrFHp8ECIbjYPNmaQ16YtU79tl6 qyAIlTkn5odVmnJxSTIKCZr7RwyszaDRBqPdPFSymmS/YdTKm+85L4N+AFtvX4V0NSjnSsYd5F0K3 q/LmHpjXj0aH6bVSWXXL2ifyidh60bHGGtZiwcWsIgkXi2mW0iYpkirvlgZc3gz4= X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A8D0D801914A X-Stat-Signature: iqc5b6yaap39p5p9j7c3xoebeenry7jy Received-SPF: none (chromium.org>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-pj1-f50.google.com; client-ip=209.85.216.50 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1615326190-463319 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Choosing the initial state of static branches changes the assembly layout (if the condition is expected to be likely, inline, or unlikely, out of line via a jump). The _TRUE/_FALSE defines for CONFIG_INIT_ON_*_DEFAULT_O= N were accidentally removed. These need to stay so that the CONFIG controls the pessimization of the resulting static branch NOP/JMP locations. Fixes: 04013513cc84 ("mm, page_alloc: do not rely on the order of page_po= ison and init_on_alloc/free parameters") Cc: stable@vger.kernel.org Signed-off-by: Kees Cook --- include/linux/mm.h | 8 ++++++++ mm/page_alloc.c | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 77e64e3eac80..b3317d91ee8e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2871,7 +2871,11 @@ static inline void kernel_poison_pages(struct page= *page, int numpages) { } static inline void kernel_unpoison_pages(struct page *page, int numpages= ) { } #endif =20 +#ifdef CONFIG_INIT_ON_ALLOC_DEFAULT_ON +DECLARE_STATIC_KEY_TRUE(init_on_alloc); +#else DECLARE_STATIC_KEY_FALSE(init_on_alloc); +#endif static inline bool want_init_on_alloc(gfp_t flags) { if (static_branch_unlikely(&init_on_alloc)) @@ -2879,7 +2883,11 @@ static inline bool want_init_on_alloc(gfp_t flags) return flags & __GFP_ZERO; } =20 +#ifdef CONFIG_INIT_ON_FREE_DEFAULT_ON +DECLARE_STATIC_KEY_TRUE(init_on_free); +#else DECLARE_STATIC_KEY_FALSE(init_on_free); +#endif static inline bool want_init_on_free(void) { return static_branch_unlikely(&init_on_free); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3e4b29ee2b1e..f2d474a844cf 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -167,10 +167,18 @@ unsigned long totalcma_pages __read_mostly; =20 int percpu_pagelist_fraction; gfp_t gfp_allowed_mask __read_mostly =3D GFP_BOOT_MASK; +#ifdef CONFIG_INIT_ON_ALLOC_DEFAULT_ON +DEFINE_STATIC_KEY_TRUE(init_on_alloc); +#else DEFINE_STATIC_KEY_FALSE(init_on_alloc); +#endif EXPORT_SYMBOL(init_on_alloc); =20 +#ifdef CONFIG_INIT_ON_FREE_DEFAULT_ON +DEFINE_STATIC_KEY_TRUE(init_on_free); +#else DEFINE_STATIC_KEY_FALSE(init_on_free); +#endif EXPORT_SYMBOL(init_on_free); =20 static bool _init_on_alloc_enabled_early __read_mostly --=20 2.25.1