* [PATCH] x86/topology: Fallback to SMT level only once
@ 2016-08-28 18:19 Jiri Olsa
2016-08-29 10:40 ` Peter Zijlstra
0 siblings, 1 reply; 4+ messages in thread
From: Jiri Olsa @ 2016-08-28 18:19 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Jean-Pierre Lozi, Jirka Hladky, Petr Surý,
lkml, Ingo Molnar, Thomas Gleixner, H. Peter Anvin
Jirka, Peter and Jean-Pierre reported performance drop on
some cpus after making cpu offline and online again.
The reason is the kernel logic that falls back to SMT
level topology if more than one node is detected within
CPU package. During the system boot this logic cuts out
the DIE topology level and numa code adds NUMA level
on top of this.
After the boot if you reboot make the cpu offline and online
again, this logic resets the SMT level topology, removing
whole NUMA level stuff.
Ensuring the SMT topology fallback happens only once during
the boot so the NUMA topology level is kept once it's built.
This problem is one of the issues reported in the wastedcores
article [1]. There's similar patch to this issue attached
to the article [2].
[1] https://github.com/jplozi/wastedcores
[2] https://github.com/jplozi/wastedcores/blob/master/patches/missing_sched_domains_linux_4.1.patch
Cc: Jean-Pierre Lozi <jplozi@unice.fr>
Cc: Jirka Hladky <jhladky@redhat.com>
Cc: Petr Surý <psury@redhat.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
arch/x86/kernel/smpboot.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 2a6e84a30a54..f2a769b2b3fe 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -487,7 +487,17 @@ static struct sched_domain_topology_level numa_inside_package_topology[] = {
*/
static void primarily_use_numa_for_topology(void)
{
- set_sched_topology(numa_inside_package_topology);
+ static bool once;
+
+ /*
+ * We need to run it only during boot, once we are
+ * here due to getting cpu online again we have already
+ * NUMA topology setup done.
+ */
+ if (!once) {
+ set_sched_topology(numa_inside_package_topology);
+ once = true;
+ }
}
void set_cpu_sibling_map(int cpu)
--
2.7.4
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] x86/topology: Fallback to SMT level only once
2016-08-28 18:19 [PATCH] x86/topology: Fallback to SMT level only once Jiri Olsa
@ 2016-08-29 10:40 ` Peter Zijlstra
2016-08-29 11:09 ` Jiri Olsa
2016-08-29 12:20 ` Jirka Hladky
0 siblings, 2 replies; 4+ messages in thread
From: Peter Zijlstra @ 2016-08-29 10:40 UTC (permalink / raw)
To: Jiri Olsa
Cc: Jean-Pierre Lozi, Jirka Hladky, Petr Surý,
lkml, Ingo Molnar, Thomas Gleixner, H. Peter Anvin
On Sun, Aug 28, 2016 at 08:19:46PM +0200, Jiri Olsa wrote:
> Jirka, Peter and Jean-Pierre reported performance drop on
> some cpus after making cpu offline and online again.
>
> The reason is the kernel logic that falls back to SMT
> level topology if more than one node is detected within
> CPU package. During the system boot this logic cuts out
> the DIE topology level and numa code adds NUMA level
> on top of this.
Its not SMT topology, and back when I asked if he had CoD enabled or
such he said not.
See also:
http://lkml.kernel.org/r/1471559812-19967-3-git-send-email-srinivas.pandruvada@linux.intel.com
Arguably, that should have been split in two patches, but alas..
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] x86/topology: Fallback to SMT level only once
2016-08-29 10:40 ` Peter Zijlstra
@ 2016-08-29 11:09 ` Jiri Olsa
2016-08-29 12:20 ` Jirka Hladky
1 sibling, 0 replies; 4+ messages in thread
From: Jiri Olsa @ 2016-08-29 11:09 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Jiri Olsa, Jean-Pierre Lozi, Jirka Hladky, Petr Surý,
lkml, Ingo Molnar, Thomas Gleixner, H. Peter Anvin
On Mon, Aug 29, 2016 at 12:40:47PM +0200, Peter Zijlstra wrote:
> On Sun, Aug 28, 2016 at 08:19:46PM +0200, Jiri Olsa wrote:
> > Jirka, Peter and Jean-Pierre reported performance drop on
> > some cpus after making cpu offline and online again.
> >
> > The reason is the kernel logic that falls back to SMT
> > level topology if more than one node is detected within
> > CPU package. During the system boot this logic cuts out
> > the DIE topology level and numa code adds NUMA level
> > on top of this.
>
> Its not SMT topology, and back when I asked if he had CoD enabled or
> such he said not.
>
>
> See also:
>
> http://lkml.kernel.org/r/1471559812-19967-3-git-send-email-srinivas.pandruvada@linux.intel.com
>
> Arguably, that should have been split in two patches, but alas..
ok, that change will work for us as well
are you going to pull it?
thanks,
jirka
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] x86/topology: Fallback to SMT level only once
2016-08-29 10:40 ` Peter Zijlstra
2016-08-29 11:09 ` Jiri Olsa
@ 2016-08-29 12:20 ` Jirka Hladky
1 sibling, 0 replies; 4+ messages in thread
From: Jirka Hladky @ 2016-08-29 12:20 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Jiri Olsa, Jean-Pierre Lozi, Petr Surý,
lkml, Ingo Molnar, Thomas Gleixner, H. Peter Anvin
Hi Peter,
yes, initially I have reported the issue to occur on Intel E5v3 CPU
(that CPU does not have CoD) but it has turned to be a fluctuation of
results. After repeating the test 10 times it has turned out that
Intel E5v3 CPU is not affected. I'm sorry for that.
I have then rerun the test on Opteron 6272 (same CPU as used by
authors of the paper) and there was performance degradation by factor
9x. Jirka has then provided the patch.
Thanks a lot
Jirka
On Mon, Aug 29, 2016 at 12:40 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> On Sun, Aug 28, 2016 at 08:19:46PM +0200, Jiri Olsa wrote:
>> Jirka, Peter and Jean-Pierre reported performance drop on
>> some cpus after making cpu offline and online again.
>>
>> The reason is the kernel logic that falls back to SMT
>> level topology if more than one node is detected within
>> CPU package. During the system boot this logic cuts out
>> the DIE topology level and numa code adds NUMA level
>> on top of this.
>
> Its not SMT topology, and back when I asked if he had CoD enabled or
> such he said not.
>
>
> See also:
>
> http://lkml.kernel.org/r/1471559812-19967-3-git-send-email-srinivas.pandruvada@linux.intel.com
>
> Arguably, that should have been split in two patches, but alas..
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2016-08-29 12:20 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-28 18:19 [PATCH] x86/topology: Fallback to SMT level only once Jiri Olsa
2016-08-29 10:40 ` Peter Zijlstra
2016-08-29 11:09 ` Jiri Olsa
2016-08-29 12:20 ` Jirka Hladky
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).