kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Wanpeng Li <kernellwp@gmail.com>
To: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>, kvm <kvm@vger.kernel.org>
Subject: IPI broadcast latency in the guest is worse when AVIC is enabled
Date: Tue, 11 Aug 2020 10:04:46 +0800	[thread overview]
Message-ID: <CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com> (raw)

Hi all,

We found that the IPI broadcast latency in the guest when AVIC=1,
exposing xapic is worse than when AVIC=0, exposing xapic. The host is
AMD ROME, 2 sockets, 96 cores, 192 threads, the VM is 180 vCPUs. The
guest boots with kvm-hint-dedicated=on, --overcommit cpu-pm=on, -smp
180,sockets=2,cores=45,threads=2, l3-cache=on qemu command-line, the
pCPU which vCPU is running on is isolated. Both the guest and host
kernel are 5.8 Linus' tree. (Note, if you fails to boot with
--overcommit cpu-pm=on, you can comments out commit e72436bc3a52, I
have a report here, https://lkml.org/lkml/2020/7/8/308)

IPI microbenchmark(https://lkml.org/lkml/2017/12/19/141, Destination
Shorthand is All excluding self)

avic0_xapic:   12313907508.50 ns
avic1_xapic:   19106424733.30 ns
avic0_x2apic: 13073988486.00 ns


ebizzy -M  (Destination Shorthand is All excluding self)

avic0_xapic
9416 records/s
real 10.00 s
user  4.80 s
sys  1693.25 s

avic1_xapic
18157 records/s
real 10.00 s
user 10.69 s
sys  1779.80 s

avic0_x2apic
74507 records/s
real 10.00 s
user 48.98 s
sys  1752.12 s


./hackbench -l 1000000  (Destination Shorthand is Destination)

avic0_xapic
Time: 121.339

avic1_xapic
Time: 117.840

avic0_x2apic
Time: 118.753

Any thoughts?

    Wanpeng

             reply	other threads:[~2020-08-11  2:05 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-11  2:04 Wanpeng Li [this message]
2020-08-11  8:28 ` IPI broadcast latency in the guest is worse when AVIC is enabled Paolo Bonzini
2020-08-13  7:59   ` Wanpeng Li

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com' \
    --to=kernellwp@gmail.com \
    --cc=Suravee.Suthikulpanit@amd.com \
    --cc=kvm@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).