From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7C3BC433E0 for ; Wed, 12 Aug 2020 01:32:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6203C2076C for ; Wed, 12 Aug 2020 01:32:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="d0Jy6be0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6203C2076C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 095158D000E; Tue, 11 Aug 2020 21:32:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F356F8D0001; Tue, 11 Aug 2020 21:32:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD9938D000E; Tue, 11 Aug 2020 21:32:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0100.hostedemail.com [216.40.44.100]) by kanga.kvack.org (Postfix) with ESMTP id C3ACD8D0001 for ; Tue, 11 Aug 2020 21:32:08 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 91DFA8248047 for ; Wed, 12 Aug 2020 01:32:08 +0000 (UTC) X-FDA: 77140190736.17.mask68_341796e26fe7 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 67C6F180D0181 for ; Wed, 12 Aug 2020 01:32:08 +0000 (UTC) X-HE-Tag: mask68_341796e26fe7 X-Filterd-Recvd-Size: 3685 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Wed, 12 Aug 2020 01:32:08 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1831022BEA; Wed, 12 Aug 2020 01:32:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597195927; bh=S4eWXnieZaxpnJaxp000pVSgN/+QdvHB6PoW6lKw2WA=; h=Date:From:To:Subject:In-Reply-To:From; b=d0Jy6be081ifQtSm4HsFRgc40hwbOTh/hYBMgMykVS4k8KjiFZiyzTTGamJF5JJRl 6vbRRAODJiAZvpJaa3ST+6H0kaLHbWtiRnLlSl8BGql6myfRNk8odN0XEs4mZrB/AT 4bqKSBdNe/rLWKJjFbB21SiNmBO/TaP7XcigXDN8= Date: Tue, 11 Aug 2020 18:32:06 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, longman@redhat.com, mathieu.desnoyers@efficios.com, mingo@redhat.com, mm-commits@vger.kernel.org, peterz@infradead.org, torvalds@linux-foundation.org, walken@google.com Subject: [patch 035/165] include/linux/sched/mm.h: optimize current_gfp_context() Message-ID: <20200812013206.tt3v-eUQV%akpm@linux-foundation.org> In-Reply-To: <20200811182949.e12ae9a472e3b5e27e16ad6c@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Queue-Id: 67C6F180D0181 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Waiman Long Subject: include/linux/sched/mm.h: optimize current_gfp_context() The current_gfp_context() converts a number of PF_MEMALLOC_* per-process flags into the corresponding GFP_* flags for memory allocation. In that function, current->flags is accessed 3 times. That may lead to duplicated access of the same memory location. This is not usually a problem with minimal debug config options on as the compiler can optimize away the duplicated memory accesses. With most of the debug config options on, however, that may not be the case. For example, the x86-64 object size of the __need_fs_reclaim() in a debug kernel that calls current_gfp_context() was 309 bytes. With this patch applied, the object size is reduced to 202 bytes. This is a saving of 107 bytes and will probably be slightly faster too. Use READ_ONCE() to access current->flags to prevent the compiler from possibly accessing current->flags multiple times. Link: http://lkml.kernel.org/r/20200618212936.9776-1-longman@redhat.com Signed-off-by: Waiman Long Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Mathieu Desnoyers Cc: Michel Lespinasse Signed-off-by: Andrew Morton --- include/linux/sched/mm.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) --- a/include/linux/sched/mm.h~sched-mm-optimize-current_gfp_context +++ a/include/linux/sched/mm.h @@ -178,14 +178,16 @@ static inline bool in_vfork(struct task_ */ static inline gfp_t current_gfp_context(gfp_t flags) { - if (unlikely(current->flags & (PF_MEMALLOC_NOIO | PF_MEMALLOC_NOFS))) { + unsigned int pflags = READ_ONCE(current->flags); + + if (unlikely(pflags & (PF_MEMALLOC_NOIO | PF_MEMALLOC_NOFS))) { /* * NOIO implies both NOIO and NOFS and it is a weaker context * so always make sure it makes precedence */ - if (current->flags & PF_MEMALLOC_NOIO) + if (pflags & PF_MEMALLOC_NOIO) flags &= ~(__GFP_IO | __GFP_FS); - else if (current->flags & PF_MEMALLOC_NOFS) + else if (pflags & PF_MEMALLOC_NOFS) flags &= ~__GFP_FS; } return flags; _