All of lore.kernel.org
 help / color / mirror / Atom feed
From: Janne Karhunen <janne.karhunen@gmail.com>
To: Marc Zyngier <maz@kernel.org>
Cc: Christoph Hellwig <hch@infradead.org>,
	Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Andrew Walbran <qwandor@google.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Will Deacon <will@kernel.org>, Tejun Heo <tj@kernel.org>,
	Dennis Zhou <dennis@kernel.org>, Christoph Lameter <cl@linux.com>,
	kernel-team@android.com, kvmarm@lists.cs.columbia.edu,
	linux-arm-kernel <linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH v1 00/24] Opt-in always-on nVHE hypervisor
Date: Tue, 19 Jan 2021 15:17:29 +0200	[thread overview]
Message-ID: <CAE=NcraN8NcnJEneno=U6dPj3esHSjChfqu9XKLNPE49ULhxwQ@mail.gmail.com> (raw)
In-Reply-To: <b174c468e3df6dc7874b9ab886b38009@kernel.org>

On Tue, Nov 10, 2020 at 1:19 PM Marc Zyngier <maz@kernel.org> wrote:

> > Why?  I thought we were trying to kill nVHE off now that newer CPUs
> > provide the saner virtualization extensions?
>
> We can't kill nVHE at all, because that is the only game in town.
> You can't even buy a decent machine with VHE, no matter how much money
> you put on the table.

As I mentioned it earlier, we did this type of nVHE hypervisor and the
proof of concept is here:
https://github.com/jkrh/kvms

See the README. It runs successfully on multiple pieces of arm64
hardware and provides a tiny QEMU based development environment via
the makefiles for the QEMU 'max' CPU. The code is rough, the amount of
man hours put to it is not sky high, but it does run. I'll update a
new kernel patch to patches/ dir for one of the later kernels
hopefully next week, up to now we have only supported kernels between
4.9 .... 5.6 as this is what our development hardware(s) run with. It
requires a handful of hooks in the kvm code, but the actual kvm calls
are just rerouted back to the kernel symbols. This way the hypervisor
itself can be kept very tiny.

The s2 page tables are fully owned by the hyp and the guests are
unmapped from the host memory when configured with the option (we call
it host blinding). Multiple VMs can be run without pinning them into
the memory.  It also provides a tiny out of tree driver prototype stub
to protect the critical sections of the kernel memory beyond the
kernel's own reach. There are still holes in the implementation such
as the virtio-mapback handling via whitelisting and paging integrity
checks, and many things are not quite all the way there yet. One step
at a time.


--
Janne

WARNING: multiple messages have this Message-ID (diff)
From: Janne Karhunen <janne.karhunen@gmail.com>
To: Marc Zyngier <maz@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Andrew Walbran <qwandor@google.com>,
	kernel-team@android.com,
	Catalin Marinas <catalin.marinas@arm.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Christoph Hellwig <hch@infradead.org>, Tejun Heo <tj@kernel.org>,
	Dennis Zhou <dennis@kernel.org>, Christoph Lameter <cl@linux.com>,
	Will Deacon <will@kernel.org>,
	kvmarm@lists.cs.columbia.edu,
	linux-arm-kernel <linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH v1 00/24] Opt-in always-on nVHE hypervisor
Date: Tue, 19 Jan 2021 15:17:29 +0200	[thread overview]
Message-ID: <CAE=NcraN8NcnJEneno=U6dPj3esHSjChfqu9XKLNPE49ULhxwQ@mail.gmail.com> (raw)
In-Reply-To: <b174c468e3df6dc7874b9ab886b38009@kernel.org>

On Tue, Nov 10, 2020 at 1:19 PM Marc Zyngier <maz@kernel.org> wrote:

> > Why?  I thought we were trying to kill nVHE off now that newer CPUs
> > provide the saner virtualization extensions?
>
> We can't kill nVHE at all, because that is the only game in town.
> You can't even buy a decent machine with VHE, no matter how much money
> you put on the table.

As I mentioned it earlier, we did this type of nVHE hypervisor and the
proof of concept is here:
https://github.com/jkrh/kvms

See the README. It runs successfully on multiple pieces of arm64
hardware and provides a tiny QEMU based development environment via
the makefiles for the QEMU 'max' CPU. The code is rough, the amount of
man hours put to it is not sky high, but it does run. I'll update a
new kernel patch to patches/ dir for one of the later kernels
hopefully next week, up to now we have only supported kernels between
4.9 .... 5.6 as this is what our development hardware(s) run with. It
requires a handful of hooks in the kvm code, but the actual kvm calls
are just rerouted back to the kernel symbols. This way the hypervisor
itself can be kept very tiny.

The s2 page tables are fully owned by the hyp and the guests are
unmapped from the host memory when configured with the option (we call
it host blinding). Multiple VMs can be run without pinning them into
the memory.  It also provides a tiny out of tree driver prototype stub
to protect the critical sections of the kernel memory beyond the
kernel's own reach. There are still holes in the implementation such
as the virtio-mapback handling via whitelisting and paging integrity
checks, and many things are not quite all the way there yet. One step
at a time.


--
Janne
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

WARNING: multiple messages have this Message-ID (diff)
From: Janne Karhunen <janne.karhunen@gmail.com>
To: Marc Zyngier <maz@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Andrew Walbran <qwandor@google.com>,
	kernel-team@android.com,
	Catalin Marinas <catalin.marinas@arm.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Christoph Hellwig <hch@infradead.org>, Tejun Heo <tj@kernel.org>,
	Dennis Zhou <dennis@kernel.org>, Christoph Lameter <cl@linux.com>,
	Will Deacon <will@kernel.org>,
	kvmarm@lists.cs.columbia.edu,
	linux-arm-kernel <linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH v1 00/24] Opt-in always-on nVHE hypervisor
Date: Tue, 19 Jan 2021 15:17:29 +0200	[thread overview]
Message-ID: <CAE=NcraN8NcnJEneno=U6dPj3esHSjChfqu9XKLNPE49ULhxwQ@mail.gmail.com> (raw)
In-Reply-To: <b174c468e3df6dc7874b9ab886b38009@kernel.org>

On Tue, Nov 10, 2020 at 1:19 PM Marc Zyngier <maz@kernel.org> wrote:

> > Why?  I thought we were trying to kill nVHE off now that newer CPUs
> > provide the saner virtualization extensions?
>
> We can't kill nVHE at all, because that is the only game in town.
> You can't even buy a decent machine with VHE, no matter how much money
> you put on the table.

As I mentioned it earlier, we did this type of nVHE hypervisor and the
proof of concept is here:
https://github.com/jkrh/kvms

See the README. It runs successfully on multiple pieces of arm64
hardware and provides a tiny QEMU based development environment via
the makefiles for the QEMU 'max' CPU. The code is rough, the amount of
man hours put to it is not sky high, but it does run. I'll update a
new kernel patch to patches/ dir for one of the later kernels
hopefully next week, up to now we have only supported kernels between
4.9 .... 5.6 as this is what our development hardware(s) run with. It
requires a handful of hooks in the kvm code, but the actual kvm calls
are just rerouted back to the kernel symbols. This way the hypervisor
itself can be kept very tiny.

The s2 page tables are fully owned by the hyp and the guests are
unmapped from the host memory when configured with the option (we call
it host blinding). Multiple VMs can be run without pinning them into
the memory.  It also provides a tiny out of tree driver prototype stub
to protect the critical sections of the kernel memory beyond the
kernel's own reach. There are still holes in the implementation such
as the virtio-mapback handling via whitelisting and paging integrity
checks, and many things are not quite all the way there yet. One step
at a time.


--
Janne

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  reply	other threads:[~2021-01-19 13:30 UTC|newest]

Thread overview: 166+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-09 11:32 [PATCH v1 00/24] Opt-in always-on nVHE hypervisor David Brazdil
2020-11-09 11:32 ` David Brazdil
2020-11-09 11:32 ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 01/24] psci: Accessor for configured PSCI version David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-10 14:43   ` Marc Zyngier
2020-11-10 14:43     ` Marc Zyngier
2020-11-10 14:43     ` Marc Zyngier
2020-11-11 11:31     ` David Brazdil
2020-11-11 11:31       ` David Brazdil
2020-11-11 11:31       ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 02/24] psci: Accessor for configured PSCI function IDs David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 03/24] arm64: Move MAIR_EL1_SET to asm/memory.h David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-10 14:49   ` Marc Zyngier
2020-11-10 14:49     ` Marc Zyngier
2020-11-10 14:49     ` Marc Zyngier
2020-11-09 11:32 ` [PATCH v1 04/24] kvm: arm64: Initialize MAIR_EL2 using a constant David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 05/24] kvm: arm64: Add .hyp.data..ro_after_init ELF section David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 06/24] kvm: arm64: Support per_cpu_ptr in nVHE hyp code David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-10 15:08   ` Marc Zyngier
2020-11-10 15:08     ` Marc Zyngier
2020-11-10 15:08     ` Marc Zyngier
2020-11-11 12:32     ` David Brazdil
2020-11-11 12:32       ` David Brazdil
2020-11-11 12:32       ` David Brazdil
2020-11-11 12:46       ` Marc Zyngier
2020-11-11 12:46         ` Marc Zyngier
2020-11-11 12:46         ` Marc Zyngier
2020-11-09 11:32 ` [PATCH v1 07/24] kvm: arm64: Create nVHE copy of cpu_logical_map David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-10 15:24   ` Marc Zyngier
2020-11-10 15:24     ` Marc Zyngier
2020-11-10 15:24     ` Marc Zyngier
2020-11-11 13:03     ` David Brazdil
2020-11-11 13:03       ` David Brazdil
2020-11-11 13:03       ` David Brazdil
2020-11-11 13:29       ` Marc Zyngier
2020-11-11 13:29         ` Marc Zyngier
2020-11-11 13:29         ` Marc Zyngier
2020-11-11 13:45         ` David Brazdil
2020-11-11 13:45           ` David Brazdil
2020-11-11 13:45           ` David Brazdil
2020-11-11 13:52           ` Marc Zyngier
2020-11-11 13:52             ` Marc Zyngier
2020-11-11 13:52             ` Marc Zyngier
2020-11-09 11:32 ` [PATCH v1 08/24] kvm: arm64: Move hyp-init params to a per-CPU struct David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-10 15:30   ` Marc Zyngier
2020-11-10 15:30     ` Marc Zyngier
2020-11-10 15:30     ` Marc Zyngier
2020-11-09 11:32 ` [PATCH v1 09/24] kvm: arm64: Refactor handle_trap to use a switch David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 10/24] kvm: arm64: Extract parts of el2_setup into a macro David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-10 15:56   ` Marc Zyngier
2020-11-10 15:56     ` Marc Zyngier
2020-11-10 15:56     ` Marc Zyngier
2020-11-16 17:56     ` David Brazdil
2020-11-16 17:56       ` David Brazdil
2020-11-16 17:56       ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 11/24] kvm: arm64: Add SMC handler in nVHE EL2 David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-11 12:12   ` Marc Zyngier
2020-11-11 12:12     ` Marc Zyngier
2020-11-11 12:12     ` Marc Zyngier
2020-11-09 11:32 ` [PATCH v1 12/24] kvm: arm64: Extract __do_hyp_init into a helper function David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 13/24] kvm: arm64: Add CPU entry point in nVHE hyp David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-11 11:57   ` Marc Zyngier
2020-11-11 11:57     ` Marc Zyngier
2020-11-11 11:57     ` Marc Zyngier
2020-11-16 11:49     ` David Brazdil
2020-11-16 11:49       ` David Brazdil
2020-11-16 11:49       ` David Brazdil
2020-11-16 12:40       ` Marc Zyngier
2020-11-16 12:40         ` Marc Zyngier
2020-11-16 12:40         ` Marc Zyngier
2020-11-09 11:32 ` [PATCH v1 14/24] kvm: arm64: Add function to enter host from KVM nVHE hyp code David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 15/24] kvm: arm64: Bootstrap PSCI SMC handler in nVHE EL2 David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-11 13:18   ` Marc Zyngier
2020-11-11 13:18     ` Marc Zyngier
2020-11-11 13:18     ` Marc Zyngier
2020-11-09 11:32 ` [PATCH v1 16/24] kvm: arm64: Add offset for hyp VA <-> PA conversion David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-11 13:47   ` Marc Zyngier
2020-11-11 13:47     ` Marc Zyngier
2020-11-11 13:47     ` Marc Zyngier
2020-11-09 11:32 ` [PATCH v1 17/24] kvm: arm64: Add __hyp_pa_symbol helper macro David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 16:59   ` Quentin Perret
2020-11-09 16:59     ` Quentin Perret
2020-11-09 16:59     ` Quentin Perret
2020-11-09 18:10     ` Marc Zyngier
2020-11-09 18:10       ` Marc Zyngier
2020-11-09 18:10       ` Marc Zyngier
2020-11-10  9:24       ` David Brazdil
2020-11-10  9:24         ` David Brazdil
2020-11-10  9:24         ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 18/24] kvm: arm64: Forward safe PSCI SMCs coming from host David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 19/24] kvm: arm64: Intercept host's PSCI_CPU_ON SMCs David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 20/24] kvm: arm64: Intercept host's CPU_SUSPEND PSCI SMCs David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 21/24] kvm: arm64: Add kvm-arm.protected early kernel parameter David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 22/24] kvm: arm64: Keep nVHE EL2 vector installed David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 23/24] kvm: arm64: Trap host SMCs in protected mode David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-10  5:02   ` kernel test robot
2020-11-10  5:02     ` kernel test robot
2020-11-10  5:02     ` kernel test robot
2020-11-10  5:02     ` kernel test robot
2020-11-10  9:03   ` Marc Zyngier
2020-11-10  9:03     ` Marc Zyngier
2020-11-10  9:03     ` Marc Zyngier
2020-11-10 13:00     ` David Brazdil
2020-11-10 13:00       ` David Brazdil
2020-11-10 13:00       ` David Brazdil
2020-11-09 11:32 ` [PATCH v1 24/24] kvm: arm64: Fix EL2 mode availability checks David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-09 11:32   ` David Brazdil
2020-11-10 10:15 ` [PATCH v1 00/24] Opt-in always-on nVHE hypervisor Christoph Hellwig
2020-11-10 10:15   ` Christoph Hellwig
2020-11-10 10:15   ` Christoph Hellwig
2020-11-10 11:18   ` Marc Zyngier
2020-11-10 11:18     ` Marc Zyngier
2020-11-10 11:18     ` Marc Zyngier
2021-01-19 13:17     ` Janne Karhunen [this message]
2021-01-19 13:17       ` Janne Karhunen
2021-01-19 13:17       ` Janne Karhunen
2020-11-11 14:32 ` Marc Zyngier
2020-11-11 14:32   ` Marc Zyngier
2020-11-11 14:32   ` Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAE=NcraN8NcnJEneno=U6dPj3esHSjChfqu9XKLNPE49ULhxwQ@mail.gmail.com' \
    --to=janne.karhunen@gmail.com \
    --cc=catalin.marinas@arm.com \
    --cc=cl@linux.com \
    --cc=dennis@kernel.org \
    --cc=hch@infradead.org \
    --cc=kernel-team@android.com \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lorenzo.pieralisi@arm.com \
    --cc=maz@kernel.org \
    --cc=qwandor@google.com \
    --cc=tj@kernel.org \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.