* mm, page_alloc: Swap likely to unlikely as code logic is different for next_zones_zonelist()
@ 2017-01-06 1:01 Steven Rostedt
2017-01-06 8:00 ` Vlastimil Babka
2017-01-06 10:16 ` Mel Gorman
0 siblings, 2 replies; 4+ messages in thread
From: Steven Rostedt @ 2017-01-06 1:01 UTC (permalink / raw)
To: LKML
Cc: Mel Gorman, Andrew Morton, Linus Torvalds, Dirk Hohndel,
Vlastimil Babka, Jesper Dangaard Brouer
Commit 682a3385e773 "mm, page_alloc: inline the fast path of the
zonelist iterator" changed how next_zones_zonelist() is called, by
adding a static inline function to do the fast path. This function adds:
if (likely(!nodes && zonelist_zone_idx(z) <= highest_zoneidx))
return z;
return __next_zones_zonelist(z, highest_zoneidx, nodes);
Where __next_zones_zonelist() is only called when nodes is not NULL or
zonelist_zone_idx(z) is less than highest_zoneidx.
The original next_zone_zonelist() was converted to
__next_zones_zonelist() but it still maintained:
if (likely(nodes == NULL))
Which is now actually a very unlikely, as it is only called with nodes
equal to NULL when zonelist_zone_idx(z) is greater than highest_zoneidx.
Before this commit, this if had this statistic:
correct incorrect % Function File Line
------- --------- - -------- ---- ----
837895 446078 34 next_zones_zonelist mmzone.c 63
After this commit, it has:
correct incorrect % Function File Line
------- --------- - -------- ---- ----
10 173840 99 __next_zones_zonelist mmzone.c 63
Thus, the if statement is now much more unlikely than it ever was as a
likely.
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
diff --git a/mm/mmzone.c b/mm/mmzone.c
index 5652be858e5e..a51c0a67ea3d 100644
--- a/mm/mmzone.c
+++ b/mm/mmzone.c
@@ -60,7 +60,7 @@ struct zoneref *__next_zones_zonelist(struct zoneref *z,
* Find the next suitable zone to use for the allocation.
* Only filter based on nodemask if it's set
*/
- if (likely(nodes == NULL))
+ if (unlikely(nodes == NULL))
while (zonelist_zone_idx(z) > highest_zoneidx)
z++;
else
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: mm, page_alloc: Swap likely to unlikely as code logic is different for next_zones_zonelist()
2017-01-06 1:01 mm, page_alloc: Swap likely to unlikely as code logic is different for next_zones_zonelist() Steven Rostedt
@ 2017-01-06 8:00 ` Vlastimil Babka
2017-01-06 14:41 ` Steven Rostedt
2017-01-06 10:16 ` Mel Gorman
1 sibling, 1 reply; 4+ messages in thread
From: Vlastimil Babka @ 2017-01-06 8:00 UTC (permalink / raw)
To: Steven Rostedt, LKML
Cc: Mel Gorman, Andrew Morton, Linus Torvalds, Dirk Hohndel,
Jesper Dangaard Brouer
On 01/06/2017 02:01 AM, Steven Rostedt wrote:
> Commit 682a3385e773 "mm, page_alloc: inline the fast path of the
> zonelist iterator" changed how next_zones_zonelist() is called, by
> adding a static inline function to do the fast path. This function adds:
>
> if (likely(!nodes && zonelist_zone_idx(z) <= highest_zoneidx))
Your profiling stats posted online suggested even this one was wrong:
function.file.line correct incorrect percent
next_zones_zonelist.mmzone.h.965 216840653 25415261543 %0.85
Was it bogus then, or is there other explanation? I have speculated that
this might be due to multiple cpusets configured in the system.
> return z;
> return __next_zones_zonelist(z, highest_zoneidx, nodes);
>
> Where __next_zones_zonelist() is only called when nodes is not NULL or
> zonelist_zone_idx(z) is less than highest_zoneidx.
>
> The original next_zone_zonelist() was converted to
> __next_zones_zonelist() but it still maintained:
>
> if (likely(nodes == NULL))
>
> Which is now actually a very unlikely, as it is only called with nodes
> equal to NULL when zonelist_zone_idx(z) is greater than highest_zoneidx.
>
> Before this commit, this if had this statistic:
>
> correct incorrect % Function File Line
> ------- --------- - -------- ---- ----
> 837895 446078 34 next_zones_zonelist mmzone.c 63
>
> After this commit, it has:
>
> correct incorrect % Function File Line
> ------- --------- - -------- ---- ----
> 10 173840 99 __next_zones_zonelist mmzone.c 63
>
> Thus, the if statement is now much more unlikely than it ever was as a
> likely.
>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Anyway this makes sense.
Acked-by: Vlastimil Babka <vbabka@suse.cz>
> ---
> diff --git a/mm/mmzone.c b/mm/mmzone.c
> index 5652be858e5e..a51c0a67ea3d 100644
> --- a/mm/mmzone.c
> +++ b/mm/mmzone.c
> @@ -60,7 +60,7 @@ struct zoneref *__next_zones_zonelist(struct zoneref *z,
> * Find the next suitable zone to use for the allocation.
> * Only filter based on nodemask if it's set
> */
> - if (likely(nodes == NULL))
> + if (unlikely(nodes == NULL))
> while (zonelist_zone_idx(z) > highest_zoneidx)
> z++;
> else
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: mm, page_alloc: Swap likely to unlikely as code logic is different for next_zones_zonelist()
2017-01-06 8:00 ` Vlastimil Babka
@ 2017-01-06 14:41 ` Steven Rostedt
0 siblings, 0 replies; 4+ messages in thread
From: Steven Rostedt @ 2017-01-06 14:41 UTC (permalink / raw)
To: Vlastimil Babka
Cc: LKML, Mel Gorman, Andrew Morton, Linus Torvalds, Dirk Hohndel,
Jesper Dangaard Brouer
On Fri, 6 Jan 2017 09:00:23 +0100
Vlastimil Babka <vbabka@suse.cz> wrote:
> On 01/06/2017 02:01 AM, Steven Rostedt wrote:
> > Commit 682a3385e773 "mm, page_alloc: inline the fast path of the
> > zonelist iterator" changed how next_zones_zonelist() is called, by
> > adding a static inline function to do the fast path. This function adds:
> >
> > if (likely(!nodes && zonelist_zone_idx(z) <= highest_zoneidx))
>
> Your profiling stats posted online suggested even this one was wrong:
>
> function.file.line correct incorrect percent
> next_zones_zonelist.mmzone.h.965 216840653 25415261543 %0.85
>
> Was it bogus then, or is there other explanation? I have speculated that
> this might be due to multiple cpusets configured in the system.
I didn't have any cpusets configured on either system.
But the number is exaggerated, as it too was affected by the "constant"
optimization by gcc. But when I fixed that, it still gives me about
50%. Probably best just to remove the likely() in that case.
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
>
> Anyway this makes sense.
>
> Acked-by: Vlastimil Babka <vbabka@suse.cz>
Thanks!
-- Steve
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: mm, page_alloc: Swap likely to unlikely as code logic is different for next_zones_zonelist()
2017-01-06 1:01 mm, page_alloc: Swap likely to unlikely as code logic is different for next_zones_zonelist() Steven Rostedt
2017-01-06 8:00 ` Vlastimil Babka
@ 2017-01-06 10:16 ` Mel Gorman
1 sibling, 0 replies; 4+ messages in thread
From: Mel Gorman @ 2017-01-06 10:16 UTC (permalink / raw)
To: Steven Rostedt
Cc: LKML, Andrew Morton, Linus Torvalds, Dirk Hohndel,
Vlastimil Babka, Jesper Dangaard Brouer
On Thu, Jan 05, 2017 at 08:01:02PM -0500, Steven Rostedt wrote:
> Commit 682a3385e773 "mm, page_alloc: inline the fast path of the
> zonelist iterator" changed how next_zones_zonelist() is called, by
> adding a static inline function to do the fast path. This function adds:
>
> if (likely(!nodes && zonelist_zone_idx(z) <= highest_zoneidx))
> return z;
> return __next_zones_zonelist(z, highest_zoneidx, nodes);
>
> Where __next_zones_zonelist() is only called when nodes is not NULL or
> zonelist_zone_idx(z) is less than highest_zoneidx.
>
> The original next_zone_zonelist() was converted to
> __next_zones_zonelist() but it still maintained:
>
> if (likely(nodes == NULL))
>
> Which is now actually a very unlikely, as it is only called with nodes
> equal to NULL when zonelist_zone_idx(z) is greater than highest_zoneidx.
>
> Before this commit, this if had this statistic:
>
> correct incorrect % Function File Line
> ------- --------- - -------- ---- ----
> 837895 446078 34 next_zones_zonelist mmzone.c 63
>
> After this commit, it has:
>
> correct incorrect % Function File Line
> ------- --------- - -------- ---- ----
> 10 173840 99 __next_zones_zonelist mmzone.c 63
>
> Thus, the if statement is now much more unlikely than it ever was as a
> likely.
>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Well spotted;
Acked-by: Mel Gorman <mgorman@techsingularity.net>
--
Mel Gorman
SUSE Labs
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-01-06 14:42 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-06 1:01 mm, page_alloc: Swap likely to unlikely as code logic is different for next_zones_zonelist() Steven Rostedt
2017-01-06 8:00 ` Vlastimil Babka
2017-01-06 14:41 ` Steven Rostedt
2017-01-06 10:16 ` Mel Gorman
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.