From: Mike Galbraith <efault@gmx.de>
To: Wanpeng Li <kernellwp@gmail.com>,
linux-kernel@vger.kernel.org, kvm <kvm@vger.kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Radim Krcmar <rkrcmar@redhat.com>,
Frederic Weisbecker <fweisbec@gmail.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@kernel.org>
Subject: Re: unixbench context switch perfomance & cpu topology
Date: Mon, 22 Jan 2018 13:08:59 +0100 [thread overview]
Message-ID: <1516622939.24679.5.camel@gmx.de> (raw)
In-Reply-To: <CANRm+CwpGoSTLNG3D8KN+fgfc+20-gFND7xGAYo2b5EuwzPeOg@mail.gmail.com>
On Mon, 2018-01-22 at 19:47 +0800, Wanpeng Li wrote:
> Hi all,
>
> We can observe unixbench context switch performance is heavily
> influenced by cpu topology which is exposed to the guest. the score is
> posted below, bigger is better, both the guest and the host kernel are
> 3.15-rc3(we can also reproduce against centos 7.4 693 guest/host), LLC
> is exposed to the guest, kvm adaptive halt-polling is default enabled,
> then start a guest w/ 8 logical cpus.
>
>
>
> unixbench context switch
> -smp 8, sockets=8, cores=1, threads=1 382036
> -smp 8, sockets=4, cores=2, threads=1 132480
> -smp 8, sockets=2, cores=4, threads=1 128032
> -smp 8, sockets=2, cores=2, threads=2 131767
> -smp 8, sockets=1, cores=4, threads=2 132742
> -smp 8, sockets=1, cores=4, threads=2 (guest w/ nohz=off idle=poll) 331471
>
> I can observe there are a lot of reschedule IPIs sent from one vCPU to
> another vCPU, the context switch workload switches between running and
> idle frequently which results in HLT instruction in the idle path, I
> use idle=poll to avoid vmexit due to HLT and to avoid reschedule IPIs
> since idle task checks TIF_NEED_RESCHED flags in a loop, nohz=off can
> stop to program lapic timer/other nohz stuffs. Any idea why sockets=8
> can get best performance?
Probably because with that topology, there is no shared llc, thus no
cross-core scheduling, micro-benchmark waker/wakee are stacked. If
your benchmark does nothing but schedule, stacking makes beautiful (but
utterly meaningless) numbers.
-Mike
next prev parent reply other threads:[~2018-01-22 12:09 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-22 11:47 unixbench context switch perfomance & cpu topology Wanpeng Li
2018-01-22 12:08 ` Mike Galbraith [this message]
2018-01-22 12:27 ` Wanpeng Li
2018-01-22 13:37 ` Mike Galbraith
2018-01-23 10:36 ` Wanpeng Li
2018-01-23 13:49 ` Mike Galbraith
2018-01-24 8:07 ` Wanpeng Li
2018-01-22 12:53 ` Peter Zijlstra
2018-01-23 10:33 ` Wanpeng Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1516622939.24679.5.camel@gmx.de \
--to=efault@gmx.de \
--cc=fweisbec@gmail.com \
--cc=kernellwp@gmail.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rkrcmar@redhat.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).