All of lore.kernel.org
 help / color / mirror / Atom feed
From: Qu Wenruo <wqu@suse.com>
To: kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org,
	Linux ARM <linux-arm-kernel@lists.infradead.org>
Subject: Any way to disable KVM VHE extension?
Date: Thu, 15 Jul 2021 16:55:27 +0800	[thread overview]
Message-ID: <37f873cf-1b39-ea7f-a5e7-6feb0200dd4c@suse.com> (raw)

Hi,

Recently I'm playing around the Nvidia Xavier AGX board, which has VHE 
extension support.

In theory, considering the CPU and memory, it should be pretty powerful 
compared to boards like RPI CM4.

But to my surprise, KVM runs pretty poor on Xavier.

Just booting the edk2 firmware could take over 10s, and 20s to fully 
boot the kernel.
Even my VM on RPI CM4 has way faster boot time, even just running on 
PCIE2.0 x1 lane NVME, and just 4 2.1Ghz A72 core.

This is definitely out of my expectation, I double checked to be sure 
that it's running in KVM mode.

But further digging shows that, since Xavier AGX CPU supports VHE, kvm 
is running in VHE mode other than HYP mode on CM4.

Is there anyway to manually disable VHE mode to test the more common HYP 
mode on Xavier?

BTW, this is the dmesg related to KVM on Xavier, running v5.13 upstream 
kernel, with 64K page size:
[    0.852357] kvm [1]: IPA Size Limit: 40 bits
[    0.857378] kvm [1]: vgic interrupt IRQ9
[    0.862122] kvm: pmu event creation failed -2
[    0.866734] kvm [1]: VHE mode initialized successfully

While on CM4, the host runs v5.12.10 upstream kernel (with downstream 
dtb), with 4K page size:
[    1.276818] kvm [1]: IPA Size Limit: 44 bits
[    1.278425] kvm [1]: vgic interrupt IRQ9
[    1.278620] kvm [1]: Hyp mode initialized successfully

Could it be the PAGE size causing problem?

Thanks,
Qu


WARNING: multiple messages have this Message-ID (diff)
From: Qu Wenruo <wqu@suse.com>
To: kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org,
	Linux ARM <linux-arm-kernel@lists.infradead.org>
Subject: Any way to disable KVM VHE extension?
Date: Thu, 15 Jul 2021 16:55:27 +0800	[thread overview]
Message-ID: <37f873cf-1b39-ea7f-a5e7-6feb0200dd4c@suse.com> (raw)

Hi,

Recently I'm playing around the Nvidia Xavier AGX board, which has VHE 
extension support.

In theory, considering the CPU and memory, it should be pretty powerful 
compared to boards like RPI CM4.

But to my surprise, KVM runs pretty poor on Xavier.

Just booting the edk2 firmware could take over 10s, and 20s to fully 
boot the kernel.
Even my VM on RPI CM4 has way faster boot time, even just running on 
PCIE2.0 x1 lane NVME, and just 4 2.1Ghz A72 core.

This is definitely out of my expectation, I double checked to be sure 
that it's running in KVM mode.

But further digging shows that, since Xavier AGX CPU supports VHE, kvm 
is running in VHE mode other than HYP mode on CM4.

Is there anyway to manually disable VHE mode to test the more common HYP 
mode on Xavier?

BTW, this is the dmesg related to KVM on Xavier, running v5.13 upstream 
kernel, with 64K page size:
[    0.852357] kvm [1]: IPA Size Limit: 40 bits
[    0.857378] kvm [1]: vgic interrupt IRQ9
[    0.862122] kvm: pmu event creation failed -2
[    0.866734] kvm [1]: VHE mode initialized successfully

While on CM4, the host runs v5.12.10 upstream kernel (with downstream 
dtb), with 4K page size:
[    1.276818] kvm [1]: IPA Size Limit: 44 bits
[    1.278425] kvm [1]: vgic interrupt IRQ9
[    1.278620] kvm [1]: Hyp mode initialized successfully

Could it be the PAGE size causing problem?

Thanks,
Qu

_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

WARNING: multiple messages have this Message-ID (diff)
From: Qu Wenruo <wqu@suse.com>
To: kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org,
	Linux ARM <linux-arm-kernel@lists.infradead.org>
Subject: Any way to disable KVM VHE extension?
Date: Thu, 15 Jul 2021 16:55:27 +0800	[thread overview]
Message-ID: <37f873cf-1b39-ea7f-a5e7-6feb0200dd4c@suse.com> (raw)

Hi,

Recently I'm playing around the Nvidia Xavier AGX board, which has VHE 
extension support.

In theory, considering the CPU and memory, it should be pretty powerful 
compared to boards like RPI CM4.

But to my surprise, KVM runs pretty poor on Xavier.

Just booting the edk2 firmware could take over 10s, and 20s to fully 
boot the kernel.
Even my VM on RPI CM4 has way faster boot time, even just running on 
PCIE2.0 x1 lane NVME, and just 4 2.1Ghz A72 core.

This is definitely out of my expectation, I double checked to be sure 
that it's running in KVM mode.

But further digging shows that, since Xavier AGX CPU supports VHE, kvm 
is running in VHE mode other than HYP mode on CM4.

Is there anyway to manually disable VHE mode to test the more common HYP 
mode on Xavier?

BTW, this is the dmesg related to KVM on Xavier, running v5.13 upstream 
kernel, with 64K page size:
[    0.852357] kvm [1]: IPA Size Limit: 40 bits
[    0.857378] kvm [1]: vgic interrupt IRQ9
[    0.862122] kvm: pmu event creation failed -2
[    0.866734] kvm [1]: VHE mode initialized successfully

While on CM4, the host runs v5.12.10 upstream kernel (with downstream 
dtb), with 4K page size:
[    1.276818] kvm [1]: IPA Size Limit: 44 bits
[    1.278425] kvm [1]: vgic interrupt IRQ9
[    1.278620] kvm [1]: Hyp mode initialized successfully

Could it be the PAGE size causing problem?

Thanks,
Qu


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

             reply	other threads:[~2021-07-15  8:55 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-15  8:55 Qu Wenruo [this message]
2021-07-15  8:55 ` Any way to disable KVM VHE extension? Qu Wenruo
2021-07-15  8:55 ` Qu Wenruo
2021-07-15  8:57 ` Qu Wenruo
2021-07-15  8:57   ` Qu Wenruo
2021-07-15  8:57   ` Qu Wenruo
2021-07-15  9:28 ` Robin Murphy
2021-07-15  9:28   ` Robin Murphy
2021-07-15  9:28   ` Robin Murphy
2021-07-15  9:44   ` Qu Wenruo
2021-07-15  9:44     ` Qu Wenruo
2021-07-15  9:44     ` Qu Wenruo
2021-07-15 10:00     ` Robin Murphy
2021-07-15 10:00       ` Robin Murphy
2021-07-15 10:00       ` Robin Murphy
2021-07-15 10:06       ` Mark Rutland
2021-07-15 10:06         ` Mark Rutland
2021-07-15 10:06         ` Mark Rutland
2021-07-15 10:49     ` Marc Zyngier
2021-07-15 10:49       ` Marc Zyngier
2021-07-15 10:49       ` Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=37f873cf-1b39-ea7f-a5e7-6feb0200dd4c@suse.com \
    --to=wqu@suse.com \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-arm-kernel@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.