From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, linux-mm@kvack.org,
longman@redhat.com, mathieu.desnoyers@efficios.com,
mingo@redhat.com, mm-commits@vger.kernel.org,
peterz@infradead.org, torvalds@linux-foundation.org,
walken@google.com
Subject: [patch 035/165] include/linux/sched/mm.h: optimize current_gfp_context()
Date: Tue, 11 Aug 2020 18:32:06 -0700 [thread overview]
Message-ID: <20200812013206.tt3v-eUQV%akpm@linux-foundation.org> (raw)
In-Reply-To: <20200811182949.e12ae9a472e3b5e27e16ad6c@linux-foundation.org>
From: Waiman Long <longman@redhat.com>
Subject: include/linux/sched/mm.h: optimize current_gfp_context()
The current_gfp_context() converts a number of PF_MEMALLOC_* per-process
flags into the corresponding GFP_* flags for memory allocation. In that
function, current->flags is accessed 3 times. That may lead to duplicated
access of the same memory location.
This is not usually a problem with minimal debug config options on as the
compiler can optimize away the duplicated memory accesses. With most of
the debug config options on, however, that may not be the case. For
example, the x86-64 object size of the __need_fs_reclaim() in a debug
kernel that calls current_gfp_context() was 309 bytes. With this patch
applied, the object size is reduced to 202 bytes. This is a saving of 107
bytes and will probably be slightly faster too.
Use READ_ONCE() to access current->flags to prevent the compiler from
possibly accessing current->flags multiple times.
Link: http://lkml.kernel.org/r/20200618212936.9776-1-longman@redhat.com
Signed-off-by: Waiman Long <longman@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Michel Lespinasse <walken@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
include/linux/sched/mm.h | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
--- a/include/linux/sched/mm.h~sched-mm-optimize-current_gfp_context
+++ a/include/linux/sched/mm.h
@@ -178,14 +178,16 @@ static inline bool in_vfork(struct task_
*/
static inline gfp_t current_gfp_context(gfp_t flags)
{
- if (unlikely(current->flags & (PF_MEMALLOC_NOIO | PF_MEMALLOC_NOFS))) {
+ unsigned int pflags = READ_ONCE(current->flags);
+
+ if (unlikely(pflags & (PF_MEMALLOC_NOIO | PF_MEMALLOC_NOFS))) {
/*
* NOIO implies both NOIO and NOFS and it is a weaker context
* so always make sure it makes precedence
*/
- if (current->flags & PF_MEMALLOC_NOIO)
+ if (pflags & PF_MEMALLOC_NOIO)
flags &= ~(__GFP_IO | __GFP_FS);
- else if (current->flags & PF_MEMALLOC_NOFS)
+ else if (pflags & PF_MEMALLOC_NOFS)
flags &= ~__GFP_FS;
}
return flags;
_
next prev parent reply other threads:[~2020-08-12 1:32 UTC|newest]
Thread overview: 168+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-12 1:29 incoming Andrew Morton
2020-08-12 1:30 ` [patch 001/165] percpu: return number of released bytes from pcpu_free_area() Andrew Morton
2020-08-12 1:30 ` [patch 002/165] mm: memcg/percpu: account percpu memory to memory cgroups Andrew Morton
2020-08-12 1:30 ` [patch 003/165] mm: memcg/percpu: per-memcg percpu memory statistics Andrew Morton
2020-08-12 1:30 ` [patch 004/165] mm: memcg: charge memcg percpu memory to the parent cgroup Andrew Morton
2020-08-12 1:30 ` [patch 006/165] mm/hugetlb: add mempolicy check in the reservation routine Andrew Morton
2020-08-12 1:30 ` [patch 007/165] mm/vmscan: make active/inactive ratio as 1:1 for anon lru Andrew Morton
2020-08-12 1:30 ` [patch 008/165] mm/vmscan: protect the workingset on anonymous LRU Andrew Morton
2020-08-12 1:30 ` [patch 009/165] mm/workingset: prepare the workingset detection infrastructure for anon LRU Andrew Morton
2020-08-12 1:30 ` [patch 010/165] mm/swapcache: support to handle the shadow entries Andrew Morton
2020-08-12 1:30 ` [patch 011/165] mm/swap: implement workingset detection for anonymous LRU Andrew Morton
2020-08-12 1:30 ` [patch 012/165] mm/vmscan: restore active/inactive ratio " Andrew Morton
2020-08-12 1:30 ` [patch 013/165] /proc/PID/smaps: consistent whitespace output format Andrew Morton
2020-08-12 1:31 ` [patch 014/165] mm: proactive compaction Andrew Morton
2020-08-12 1:31 ` [patch 015/165] mm: fix compile error due to COMPACTION_HPAGE_ORDER Andrew Morton
2020-08-12 1:31 ` [patch 016/165] mm: use unsigned types for fragmentation score Andrew Morton
2020-08-12 1:31 ` [patch 017/165] mm/compaction: correct the comments of compact_defer_shift Andrew Morton
2020-08-12 1:31 ` [patch 018/165] mm: mempolicy: fix kerneldoc of numa_map_to_online_node() Andrew Morton
2020-08-12 1:31 ` [patch 019/165] mm/mempolicy.c: check parameters first in kernel_get_mempolicy Andrew Morton
2020-08-12 1:31 ` [patch 020/165] include/linux/mempolicy.h: fix typo Andrew Morton
2020-08-12 1:31 ` [patch 021/165] mm, oom: make the calculation of oom badness more accurate Andrew Morton
2020-08-12 1:31 ` [patch 022/165] doc, mm: sync up oom_score_adj documentation Andrew Morton
2020-08-12 1:31 ` [patch 023/165] doc, mm: clarify /proc/<pid>/oom_score value range Andrew Morton
2020-08-12 1:31 ` [patch 024/165] mm, oom: show process exiting information in __oom_kill_process() Andrew Morton
2020-08-12 1:31 ` [patch 025/165] hugetlbfs: prevent filesystem stacking of hugetlbfs Andrew Morton
2020-08-12 1:31 ` [patch 026/165] hugetlbfs: remove call to huge_pte_alloc without i_mmap_rwsem Andrew Morton
2020-08-12 1:31 ` [patch 027/165] mm/migrate: optimize migrate_vma_setup() for holes Andrew Morton
2020-08-12 1:31 ` [patch 028/165] mm/migrate: add migrate-shared test for migrate_vma_*() Andrew Morton
2020-08-12 1:31 ` [patch 029/165] mm: thp: remove debug_cow switch Andrew Morton
2020-08-12 1:31 ` [patch 030/165] mm/vmstat: add events for THP migration without split Andrew Morton
2020-08-12 1:31 ` [patch 031/165] mm/cma.c: fix NULL pointer dereference when cma could not be activated Andrew Morton
2020-08-12 1:31 ` [patch 032/165] mm: cma: fix the name of CMA areas Andrew Morton
2020-08-12 1:32 ` [patch 033/165] mm: hugetlb: fix the name of hugetlb CMA Andrew Morton
2020-08-12 1:32 ` [patch 034/165] cma: don't quit at first error when activating reserved areas Andrew Morton
2020-08-12 1:32 ` Andrew Morton [this message]
2020-08-12 1:32 ` [patch 036/165] mm: mmu_notifier: fix and extend kerneldoc Andrew Morton
2020-08-12 1:32 ` [patch 037/165] x86/mm: use max memory block size on bare metal Andrew Morton
2020-08-12 1:32 ` [patch 038/165] mm/memory_hotplug: introduce default dummy memory_add_physaddr_to_nid() Andrew Morton
2020-08-12 1:32 ` [patch 039/165] mm/memory_hotplug: fix unpaired mem_hotplug_begin/done Andrew Morton
2020-08-12 1:32 ` [patch 040/165] mm, memory_hotplug: update pcp lists everytime onlining a memory block Andrew Morton
2020-08-12 1:32 ` [patch 041/165] mm: drop duplicated words in <linux/pgtable.h> Andrew Morton
2020-08-12 1:32 ` [patch 042/165] mm: drop duplicated words in <linux/mm.h> Andrew Morton
2020-08-12 1:32 ` [patch 043/165] include/linux/highmem.h: fix duplicated words in a comment Andrew Morton
2020-08-12 1:32 ` [patch 044/165] include/linux/frontswap.h: drop duplicated word " Andrew Morton
2020-08-12 1:32 ` [patch 045/165] include/linux/memcontrol.h: drop duplicate word and fix spello Andrew Morton
2020-08-12 1:32 ` [patch 046/165] sh/mm: drop unused MAX_PHYSADDR_BITS Andrew Morton
2020-08-12 1:32 ` [patch 047/165] sparc: " Andrew Morton
2020-08-12 1:32 ` [patch 048/165] mm/compaction.c: delete duplicated word Andrew Morton
2020-08-12 1:32 ` [patch 049/165] mm/filemap.c: " Andrew Morton
2020-08-12 1:32 ` [patch 050/165] mm/hmm.c: " Andrew Morton
2020-08-12 1:32 ` [patch 051/165] mm/hugetlb.c: delete duplicated words Andrew Morton
2020-08-12 1:33 ` [patch 052/165] mm/memcontrol.c: " Andrew Morton
2020-08-12 1:33 ` [patch 053/165] mm/memory.c: " Andrew Morton
2020-08-12 1:33 ` [patch 054/165] mm/migrate.c: delete duplicated word Andrew Morton
2020-08-12 1:33 ` [patch 055/165] mm/nommu.c: delete duplicated words Andrew Morton
2020-08-12 1:33 ` [patch 056/165] mm/page_alloc.c: delete or fix " Andrew Morton
2020-08-12 1:33 ` [patch 057/165] mm/shmem.c: delete duplicated word Andrew Morton
2020-08-12 1:33 ` [patch 058/165] mm/slab_common.c: " Andrew Morton
2020-08-12 1:33 ` [patch 059/165] mm/usercopy.c: " Andrew Morton
2020-08-12 1:33 ` [patch 060/165] mm/vmscan.c: delete or fix duplicated words Andrew Morton
2020-08-12 1:33 ` [patch 061/165] mm/zpool.c: delete duplicated word and fix grammar Andrew Morton
2020-08-12 1:33 ` [patch 062/165] mm/zsmalloc.c: fix duplicated words Andrew Morton
2020-08-12 1:33 ` [patch 063/165] syscalls: use uaccess_kernel in addr_limit_user_check Andrew Morton
2020-08-12 1:33 ` [patch 064/165] nds32: use uaccess_kernel in show_regs Andrew Morton
2020-08-12 1:33 ` [patch 065/165] riscv: include <asm/pgtable.h> in <asm/uaccess.h> Andrew Morton
2020-08-12 1:33 ` [patch 066/165] uaccess: remove segment_eq Andrew Morton
2020-08-12 1:33 ` [patch 067/165] uaccess: add force_uaccess_{begin,end} helpers Andrew Morton
2020-08-12 1:33 ` [patch 068/165] exec: use force_uaccess_begin during exec and exit Andrew Morton
2020-08-12 1:33 ` [patch 069/165] alpha: fix annotation of io{read,write}{16,32}be() Andrew Morton
2020-08-12 1:33 ` [patch 070/165] include/linux/compiler-clang.h: drop duplicated word in a comment Andrew Morton
2020-08-12 1:34 ` [patch 071/165] include/linux/exportfs.h: " Andrew Morton
2020-08-12 1:34 ` [patch 072/165] include/linux/async_tx.h: " Andrew Morton
2020-08-12 1:34 ` [patch 073/165] include/linux/xz.h: drop duplicated word Andrew Morton
2020-08-12 1:34 ` [patch 074/165] kernel: add a kernel_wait helper Andrew Morton
2020-08-12 1:34 ` [patch 075/165] ./Makefile: add debug option to enable function aligned on 32 bytes Andrew Morton
2020-08-12 1:34 ` [patch 076/165] kernel.h: remove duplicate include of asm/div64.h Andrew Morton
2020-08-12 1:34 ` [patch 077/165] include/: replace HTTP links with HTTPS ones Andrew Morton
2020-08-12 1:34 ` [patch 078/165] include/linux/poison.h: remove obsolete comment Andrew Morton
2020-08-12 1:34 ` [patch 079/165] sparse: group the defines by functionality Andrew Morton
2020-08-12 1:34 ` [patch 080/165] lib/bitmap.c: fix bitmap_cut() for partial overlapping case Andrew Morton
2020-08-12 1:34 ` [patch 081/165] lib/test_bitmap.c: add test for bitmap_cut() Andrew Morton
2020-08-12 1:34 ` [patch 082/165] lib/generic-radix-tree.c: remove unneeded __rcu Andrew Morton
2020-08-12 1:34 ` [patch 083/165] lib/test_bitops: do the full test during module init Andrew Morton
2020-08-12 1:34 ` [patch 084/165] lib/test_lockup.c: make symbol 'test_works' static Andrew Morton
2020-08-12 1:34 ` [patch 085/165] lib/Kconfig.debug: make TEST_LOCKUP depend on module Andrew Morton
2020-08-12 1:34 ` [patch 086/165] lib/test_lockup.c: fix return value of test_lockup_init() Andrew Morton
2020-08-12 1:34 ` [patch 087/165] lib/: replace HTTP links with HTTPS ones Andrew Morton
2020-08-12 1:34 ` [patch 088/165] kstrto*: correct documentation references to simple_strto*() Andrew Morton
2020-08-12 1:34 ` [patch 089/165] kstrto*: do not describe simple_strto*() as obsolete/replaced Andrew Morton
2020-08-12 1:35 ` [patch 090/165] lz4: fix kernel decompression speed Andrew Morton
2020-08-12 17:54 ` Linus Torvalds
2020-08-14 17:20 ` Arvind Sankar
2020-08-12 1:35 ` [patch 091/165] lib/test_bits.c: add tests of GENMASK Andrew Morton
2020-08-12 13:39 ` William Breathitt Gray
2020-08-12 1:35 ` [patch 092/165] checkpatch: add test for possible misuse of IS_ENABLED() without CONFIG_ Andrew Morton
2020-08-12 1:35 ` [patch 093/165] checkpatch: add --fix option for ASSIGN_IN_IF Andrew Morton
2020-08-12 1:35 ` [patch 094/165] checkpatch: fix CONST_STRUCT when const_structs.checkpatch is missing Andrew Morton
2020-08-12 1:35 ` [patch 095/165] checkpatch: add test for repeated words Andrew Morton
2020-08-12 1:35 ` [patch 096/165] checkpatch: remove missing switch/case break test Andrew Morton
2020-08-12 1:35 ` [patch 097/165] autofs: fix doubled word Andrew Morton
2020-08-12 1:35 ` [patch 098/165] fs/minix: check return value of sb_getblk() Andrew Morton
2020-08-12 1:35 ` [patch 099/165] fs/minix: don't allow getting deleted inodes Andrew Morton
2020-08-12 1:35 ` [patch 100/165] fs/minix: reject too-large maximum file size Andrew Morton
2020-08-12 1:35 ` [patch 101/165] fs/minix: set s_maxbytes correctly Andrew Morton
2020-08-12 1:35 ` [patch 102/165] fs/minix: fix block limit check for V1 filesystems Andrew Morton
2020-08-12 1:35 ` [patch 103/165] fs/minix: remove expected error message in block_to_path() Andrew Morton
2020-08-12 1:35 ` [patch 104/165] nilfs2: only call unlock_new_inode() if I_NEW Andrew Morton
2020-08-12 1:35 ` [patch 105/165] nilfs2: convert __nilfs_msg to integrate the level and format Andrew Morton
2020-08-12 1:35 ` [patch 106/165] nilfs2: use a more common logging style Andrew Morton
2020-08-12 1:35 ` [patch 107/165] fs/ufs: avoid potential u32 multiplication overflow Andrew Morton
2020-08-12 1:35 ` [patch 108/165] fatfs: switch write_lock to read_lock in fat_ioctl_get_attributes Andrew Morton
2020-08-12 1:35 ` [patch 109/165] VFAT/FAT/MSDOS FILESYSTEM: replace HTTP links with HTTPS ones Andrew Morton
2020-08-12 1:36 ` [patch 110/165] fat: fix fat_ra_init() for data clusters == 0 Andrew Morton
2020-08-12 1:36 ` [patch 111/165] fs/signalfd.c: fix inconsistent return codes for signalfd4 Andrew Morton
2020-08-12 1:36 ` [patch 112/165] selftests: kmod: use variable NAME in kmod_test_0001() Andrew Morton
2020-08-12 1:36 ` [patch 113/165] kmod: remove redundant "be an" in the comment Andrew Morton
2020-08-12 1:36 ` [patch 114/165] test_kmod: avoid potential double free in trigger_config_run_type() Andrew Morton
2020-08-12 1:36 ` [patch 115/165] coredump: add %f for executable filename Andrew Morton
2020-08-12 1:36 ` [patch 116/165] exec: change uselib(2) IS_SREG() failure to EACCES Andrew Morton
2020-08-12 1:36 ` [patch 117/165] exec: move S_ISREG() check earlier Andrew Morton
2020-08-12 1:36 ` [patch 118/165] exec: move path_noexec() " Andrew Morton
2020-08-12 1:36 ` [patch 119/165] kdump: append kernel build-id string to VMCOREINFO Andrew Morton
2020-08-12 1:36 ` [patch 120/165] drivers/rapidio/devices/rio_mport_cdev.c: use struct_size() helper Andrew Morton
2020-08-12 1:36 ` [patch 121/165] drivers/rapidio/rio-scan.c: " Andrew Morton
2020-08-12 1:36 ` [patch 122/165] rapidio/rio_mport_cdev: use array_size() helper in copy_{from,to}_user() Andrew Morton
2020-08-12 1:36 ` [patch 123/165] kernel/panic.c: make oops_may_print() return bool Andrew Morton
2020-08-12 1:36 ` [patch 124/165] lib/Kconfig.debug: fix typo in the help text of CONFIG_PANIC_TIMEOUT Andrew Morton
2020-08-12 1:36 ` [patch 125/165] panic: make print_oops_end_marker() static Andrew Morton
2020-08-12 1:36 ` [patch 126/165] kcov: unconditionally add -fno-stack-protector to compiler options Andrew Morton
2020-08-12 1:36 ` [patch 127/165] kcov: make some symbols static Andrew Morton
2020-08-12 1:37 ` [patch 128/165] scripts/gdb: fix python 3.8 SyntaxWarning Andrew Morton
2020-08-12 1:37 ` [patch 129/165] ipc: uninline functions Andrew Morton
2020-08-12 1:37 ` [patch 130/165] ipc/shm.c: remove the superfluous break Andrew Morton
2020-08-12 1:37 ` [patch 131/165] mm/page_isolation: prefer the node of the source page Andrew Morton
2020-08-12 1:37 ` [patch 132/165] mm/migrate: move migration helper from .h to .c Andrew Morton
2020-08-12 1:37 ` [patch 133/165] mm/hugetlb: unify migration callbacks Andrew Morton
2020-08-12 1:37 ` [patch 134/165] mm/migrate: clear __GFP_RECLAIM to make the migration callback consistent with regular THP allocations Andrew Morton
2020-08-12 1:37 ` [patch 135/165] mm/migrate: introduce a standard migration target allocation function Andrew Morton
2020-08-12 1:37 ` [patch 136/165] mm/mempolicy: use a standard migration target allocation callback Andrew Morton
2020-08-12 1:37 ` [patch 137/165] mm/page_alloc: remove a wrapper for alloc_migration_target() Andrew Morton
2020-08-12 1:37 ` [patch 138/165] mm/gup: restrict CMA region by using allocation scope API Andrew Morton
2020-08-12 1:37 ` [patch 139/165] mm/hugetlb: make hugetlb migration callback CMA aware Andrew Morton
2020-08-12 1:37 ` [patch 140/165] mm/gup: use a standard migration target allocation callback Andrew Morton
2020-08-12 1:37 ` [patch 141/165] mm: do page fault accounting in handle_mm_fault Andrew Morton
2020-08-12 1:37 ` [patch 142/165] mm/alpha: use general page fault accounting Andrew Morton
2020-08-12 1:37 ` [patch 143/165] mm/arc: " Andrew Morton
2020-08-12 1:37 ` [patch 144/165] mm/arm: " Andrew Morton
2020-08-12 1:37 ` [patch 145/165] mm/arm64: " Andrew Morton
2020-08-12 1:38 ` [patch 146/165] mm/csky: " Andrew Morton
2020-08-12 1:38 ` [patch 147/165] mm/hexagon: " Andrew Morton
2020-08-12 1:38 ` [patch 148/165] mm/ia64: " Andrew Morton
2020-08-12 1:38 ` [patch 149/165] mm/m68k: " Andrew Morton
2020-08-12 1:38 ` [patch 150/165] mm/microblaze: " Andrew Morton
2020-08-12 1:38 ` [patch 151/165] mm/mips: " Andrew Morton
2020-08-12 1:38 ` [patch 152/165] mm/nds32: " Andrew Morton
2020-08-12 1:38 ` [patch 153/165] mm/nios2: " Andrew Morton
2020-08-12 1:38 ` [patch 154/165] mm/openrisc: " Andrew Morton
2020-08-12 1:38 ` [patch 155/165] mm/parisc: " Andrew Morton
2020-08-12 1:38 ` [patch 156/165] mm/powerpc: " Andrew Morton
2020-08-12 1:38 ` [patch 157/165] mm/riscv: " Andrew Morton
2020-08-12 1:38 ` [patch 158/165] mm/s390: " Andrew Morton
2020-08-12 1:38 ` [patch 159/165] mm/sh: " Andrew Morton
2020-08-12 1:38 ` [patch 160/165] mm/sparc32: " Andrew Morton
2020-08-12 1:38 ` [patch 161/165] mm/sparc64: " Andrew Morton
2020-08-12 1:38 ` [patch 162/165] mm/x86: " Andrew Morton
2020-08-12 1:38 ` [patch 163/165] mm/xtensa: " Andrew Morton
2020-08-12 1:38 ` [patch 164/165] mm: clean up the last pieces of page fault accountings Andrew Morton
2020-08-12 1:39 ` [patch 165/165] mm/gup: remove task_struct pointer for all gup code Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200812013206.tt3v-eUQV%akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=linux-mm@kvack.org \
--cc=longman@redhat.com \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@redhat.com \
--cc=mm-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=torvalds@linux-foundation.org \
--cc=walken@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).