From: Yury Norov <ynorov@caviumnetworks.com>
To: Wanpeng Li <kernellwp@gmail.com>
Cc: linux-kernel@vger.kernel.org, kvm <kvm@vger.kernel.org>,
LAK <linux-arm-kernel@lists.infradead.org>,
Andrew Morton <akpm@linux-foundation.org>,
Ashish Kalra <Ashish.Kalra@cavium.com>,
Christoffer Dall <christoffer.dall@linaro.org>,
Geert Uytterhoeven <geert@linux-m68k.org>,
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
Linu Cherian <Linu.Cherian@cavium.com>,
Shih-Wei Li <shihwei@cs.columbia.edu>,
Sunil Goutham <Sunil.Goutham@cavium.com>
Subject: Re: [PATCH v2] IPI performance benchmark
Date: Fri, 22 Dec 2017 09:09:40 +0300 [thread overview]
Message-ID: <20171222060940.f2gjsjelqacdyec3@yury-thinkpad> (raw)
In-Reply-To: <CANRm+Cx6Lm8r2Oej+-EX=Dz76bvu81rVKD-RU0=LRSNs3dw7mw@mail.gmail.com>
On Wed, Dec 20, 2017 at 02:44:25PM +0800, Wanpeng Li wrote:
> Hi Yury,
> 2017-12-19 16:50 GMT+08:00 Yury Norov <ynorov@caviumnetworks.com>:
> > This benchmark sends many IPIs in different modes and measures
> > time for IPI delivery (first column), and total time, ie including
> > time to acknowledge the receive by sender (second column).
> >
> > The scenarios are:
> > Dry-run: do everything except actually sending IPI. Useful
> > to estimate system overhead.
> > Self-IPI: Send IPI to self CPU.
> > Normal IPI: Send IPI to some other CPU.
> > Broadcast IPI: Send broadcast IPI to all online CPUs.
> > Broadcast lock: Send broadcast IPI to all online CPUs and force them
> > acquire/release spinlock.
> >
> > The raw output looks like this:
> > [ 155.363374] Dry-run: 0, 2999696 ns
> > [ 155.429162] Self-IPI: 30385328, 65589392 ns
> > [ 156.060821] Normal IPI: 566914128, 631453008 ns
> > [ 158.384427] Broadcast IPI: 0, 2323368720 ns
> > [ 160.831850] Broadcast lock: 0, 2447000544 ns
> >
> > For virtualized guests, sending and reveiving IPIs causes guest exit.
> > I used this test to measure performance impact on KVM subsystem of
> > Christoffer Dall's series "Optimize KVM/ARM for VHE systems" [1].
> >
> > Test machine is ThunderX2, 112 online CPUs. Below the results normalized
> > to host dry-run time, broadcast lock results omitted. Smaller - better.
>
> Could you test on a x86 box? I see a lot of calltraces on my haswell
> client host, there is no calltrace in the guest, however, I can still
> observe "Invalid parameters" warning when insmod this module. In
> addition, the x86 box fails to boot when ipi_benchmark is buildin.
EINVAL is returned intentionally to let user run test again without
annoying rmmod.
WARNING: multiple messages have this Message-ID (diff)
From: ynorov@caviumnetworks.com (Yury Norov)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v2] IPI performance benchmark
Date: Fri, 22 Dec 2017 09:09:40 +0300 [thread overview]
Message-ID: <20171222060940.f2gjsjelqacdyec3@yury-thinkpad> (raw)
In-Reply-To: <CANRm+Cx6Lm8r2Oej+-EX=Dz76bvu81rVKD-RU0=LRSNs3dw7mw@mail.gmail.com>
On Wed, Dec 20, 2017 at 02:44:25PM +0800, Wanpeng Li wrote:
> Hi Yury,
> 2017-12-19 16:50 GMT+08:00 Yury Norov <ynorov@caviumnetworks.com>:
> > This benchmark sends many IPIs in different modes and measures
> > time for IPI delivery (first column), and total time, ie including
> > time to acknowledge the receive by sender (second column).
> >
> > The scenarios are:
> > Dry-run: do everything except actually sending IPI. Useful
> > to estimate system overhead.
> > Self-IPI: Send IPI to self CPU.
> > Normal IPI: Send IPI to some other CPU.
> > Broadcast IPI: Send broadcast IPI to all online CPUs.
> > Broadcast lock: Send broadcast IPI to all online CPUs and force them
> > acquire/release spinlock.
> >
> > The raw output looks like this:
> > [ 155.363374] Dry-run: 0, 2999696 ns
> > [ 155.429162] Self-IPI: 30385328, 65589392 ns
> > [ 156.060821] Normal IPI: 566914128, 631453008 ns
> > [ 158.384427] Broadcast IPI: 0, 2323368720 ns
> > [ 160.831850] Broadcast lock: 0, 2447000544 ns
> >
> > For virtualized guests, sending and reveiving IPIs causes guest exit.
> > I used this test to measure performance impact on KVM subsystem of
> > Christoffer Dall's series "Optimize KVM/ARM for VHE systems" [1].
> >
> > Test machine is ThunderX2, 112 online CPUs. Below the results normalized
> > to host dry-run time, broadcast lock results omitted. Smaller - better.
>
> Could you test on a x86 box? I see a lot of calltraces on my haswell
> client host, there is no calltrace in the guest, however, I can still
> observe "Invalid parameters" warning when insmod this module. In
> addition, the x86 box fails to boot when ipi_benchmark is buildin.
EINVAL is returned intentionally to let user run test again without
annoying rmmod.
next prev parent reply other threads:[~2017-12-22 6:10 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-19 8:50 [PATCH v2] IPI performance benchmark Yury Norov
2017-12-19 8:50 ` Yury Norov
2017-12-19 9:26 ` Philippe Ombredanne
2017-12-19 9:26 ` Philippe Ombredanne
2017-12-19 9:26 ` Philippe Ombredanne
2017-12-19 10:28 ` Yury Norov
2017-12-19 10:28 ` Yury Norov
2017-12-19 10:28 ` Yury Norov
2017-12-19 23:51 ` Andrew Morton
2017-12-19 23:51 ` Andrew Morton
2017-12-20 6:44 ` Wanpeng Li
2017-12-20 6:44 ` Wanpeng Li
2017-12-21 19:02 ` Yury Norov
2017-12-21 19:02 ` Yury Norov
2017-12-22 6:17 ` Wanpeng Li
2017-12-22 6:09 ` Yury Norov [this message]
2017-12-22 6:09 ` Yury Norov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171222060940.f2gjsjelqacdyec3@yury-thinkpad \
--to=ynorov@caviumnetworks.com \
--cc=Ashish.Kalra@cavium.com \
--cc=Linu.Cherian@cavium.com \
--cc=Sunil.Goutham@cavium.com \
--cc=akpm@linux-foundation.org \
--cc=christoffer.dall@linaro.org \
--cc=geert@linux-m68k.org \
--cc=kernellwp@gmail.com \
--cc=konrad.wilk@oracle.com \
--cc=kvm@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=shihwei@cs.columbia.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.