* + mm-page_alloc-optimize-code-layout-for-__alloc_pages_bulk.patch added to -mm tree
@ 2021-03-28 22:30 akpm
0 siblings, 0 replies; only message in thread
From: akpm @ 2021-03-28 22:30 UTC (permalink / raw)
To: alexander.duyck, alobakin, brouer, chuck.lever, davem, hch,
ilias.apalodimas, mgorman, mm-commits, vbabka, willy
The patch titled
Subject: mm/page_alloc: optimize code layout for __alloc_pages_bulk
has been added to the -mm tree. Its filename is
mm-page_alloc-optimize-code-layout-for-__alloc_pages_bulk.patch
This patch should soon appear at
https://ozlabs.org/~akpm/mmots/broken-out/mm-page_alloc-optimize-code-layout-for-__alloc_pages_bulk.patch
and later at
https://ozlabs.org/~akpm/mmotm/broken-out/mm-page_alloc-optimize-code-layout-for-__alloc_pages_bulk.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Jesper Dangaard Brouer <brouer@redhat.com>
Subject: mm/page_alloc: optimize code layout for __alloc_pages_bulk
Looking at perf-report and ASM-code for __alloc_pages_bulk() it is clear
that the code activated is suboptimal. The compiler guesses wrong and
places unlikely code at the beginning. Due to the use of WARN_ON_ONCE()
macro the UD2 asm instruction is added to the code, which confuse the
I-cache prefetcher in the CPU.
[mgorman@techsingularity.net: minor changes and rebasing]
Link: https://lkml.kernel.org/r/20210325114228.27719-5-mgorman@techsingularity.net
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Reviewed-by: Alexander Lobakin <alobakin@pm.me>
Cc: Alexander Duyck <alexander.duyck@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Chuck Lever <chuck.lever@oracle.com>
Cc: David Miller <davem@davemloft.net>
Cc: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
mm/page_alloc.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
--- a/mm/page_alloc.c~mm-page_alloc-optimize-code-layout-for-__alloc_pages_bulk
+++ a/mm/page_alloc.c
@@ -5036,7 +5036,7 @@ unsigned long __alloc_pages_bulk(gfp_t g
unsigned int alloc_flags;
int nr_populated = 0;
- if (WARN_ON_ONCE(nr_pages <= 0))
+ if (unlikely(nr_pages <= 0))
return 0;
/*
@@ -5083,7 +5083,7 @@ unsigned long __alloc_pages_bulk(gfp_t g
* If there are no allowed local zones that meets the watermarks then
* try to allocate a single page and reclaim if necessary.
*/
- if (!zone)
+ if (unlikely(!zone))
goto failed;
/* Attempt the batch allocation */
@@ -5101,7 +5101,7 @@ unsigned long __alloc_pages_bulk(gfp_t g
page = __rmqueue_pcplist(zone, ac.migratetype, alloc_flags,
pcp, pcp_list);
- if (!page) {
+ if (unlikely(!page)) {
/* Try and get at least one page */
if (!nr_populated)
goto failed_irq;
_
Patches currently in -mm which might be from brouer@redhat.com are
mm-page_alloc-optimize-code-layout-for-__alloc_pages_bulk.patch
mm-page_alloc-inline-__rmqueue_pcplist.patch
net-page_pool-refactor-dma_map-into-own-function-page_pool_dma_map.patch
net-page_pool-use-alloc_pages_bulk-in-refill-code-path.patch
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2021-03-28 22:31 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-28 22:30 + mm-page_alloc-optimize-code-layout-for-__alloc_pages_bulk.patch added to -mm tree akpm
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.