From: Yury Norov <ynorov@caviumnetworks.com>
To: Marc Zyngier <marc.zyngier@arm.com>
Cc: kvm@vger.kernel.org, Shih-Wei Li <shihwei@cs.columbia.edu>,
kvmarm@lists.cs.columbia.edu,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v2 00/36] Optimize KVM/ARM for VHE systems
Date: Mon, 11 Dec 2017 18:14:33 +0300 [thread overview]
Message-ID: <20171211151433.3mmqiwau5wepknxl@yury-thinkpad> (raw)
In-Reply-To: <e238af17-d63c-52cd-c411-c2d3f3d86994@arm.com>
On Mon, Dec 11, 2017 at 02:56:01PM +0000, Marc Zyngier wrote:
> On 11/12/17 14:43, Yury Norov wrote:
> > Hi Christoffer,
> >
> > On Thu, Dec 07, 2017 at 06:05:54PM +0100, Christoffer Dall wrote:
> >> This series redesigns parts of KVM/ARM to optimize the performance on
> >> VHE systems. The general approach is to try to do as little work as
> >> possible when transitioning between the VM and the hypervisor. This has
> >> the benefit of lower latency when waiting for interrupts and delivering
> >> virtual interrupts, and reduces the overhead of emulating behavior and
> >> I/O in the host kernel.
> >>
> >> Patches 01 through 04 are not VHE specific, but rework parts of KVM/ARM
> >> that can be generally improved. We then add infrastructure to move more
> >> logic into vcpu_load and vcpu_put, we improve handling of VFP and debug
> >> registers.
> >>
> >> We then introduce a new world-switch function for VHE systems, which we
> >> can tweak and optimize for VHE systems. To do that, we rework a lot of
> >> the system register save/restore handling and emulation code that may
> >> need access to system registers, so that we can defer as many system
> >> register save/restore operations to vcpu_load and vcpu_put, and move
> >> this logic out of the VHE world switch function.
> >>
> >> We then optimize the configuration of traps. On non-VHE systems, both
> >> the host and VM kernels run in EL1, but because the host kernel should
> >> have full access to the underlying hardware, but the VM kernel should
> >> not, we essentially make the host kernel more privileged than the VM
> >> kernel despite them both running at the same privilege level by enabling
> >> VE traps when entering the VM and disabling those traps when exiting the
> >> VM. On VHE systems, the host kernel runs in EL2 and has full access to
> >> the hardware (as much as allowed by secure side software), and is
> >> unaffected by the trap configuration. That means we can configure the
> >> traps for VMs running in EL1 once, and don't have to switch them on and
> >> off for every entry/exit to/from the VM.
> >>
> >> Finally, we improve our VGIC handling by moving all save/restore logic
> >> out of the VHE world-switch, and we make it possible to truly only
> >> evaluate if the AP list is empty and not do *any* VGIC work if that is
> >> the case, and only do the minimal amount of work required in the course
> >> of the VGIC processing when we have virtual interrupts in flight.
> >>
> >> The patches are based on v4.15-rc1 plus the fixes sent for v4.15-rc3
> >> [1], the level-triggered mapped interrupts support series [2], and the
> >> first five patches of James' SDEI series [3], a single SVE patch that
> >> moves the CPU ID reg trap setup out of the world-switch path, and v3 of
> >> my vcpu load/put series [4].
> >>
> >> I've given the patches a fair amount of testing on Thunder-X, Mustang,
> >> Seattle, and TC2 (32-bit) for non-VHE testing, and tested VHE
> >> functionality on the Foundation model, running both 64-bit VMs and
> >> 32-bit VMs side-by-side and using both GICv3-on-GICv3 and
> >> GICv2-on-GICv3.
> >>
> >> The patches are also available in the vhe-optimize-v2 branch on my
> >> kernel.org repository [5].
> >>
> >> Changes since v1:
> >> - Rebased on v4.15-rc1 and newer versions of other dependencies,
> >> including the vcpu load/put approach taken for KVM.
> >> - Addressed review comments from v1 (detailed changelogs are in the
> >> individual patches).
> >>
> >> Thanks,
> >> -Christoffer
> >>
> >> [1]: git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm kvm-arm-fixes-for-v4.15-1
> >> [2]: git://git.kernel.org/pub/scm/linux/kernel/git/cdall/linux.git level-mapped-v6
> >> [3]: git://linux-arm.org/linux-jm.git sdei/v5/base
> >> [4]: git://git.kernel.org/pub/scm/linux/kernel/git/cdall/linux.git vcpu-load-put-v3
> >> [5]: git://git.kernel.org/pub/scm/linux/kernel/git/cdall/linux.git vhe-optimize-v2
> >
> > I just submitted the benchmark I used to test your v1 and v2 series':
> > https://lkml.org/lkml/2017/12/11/364
> >
> > On ThunderX2, 112 online CPUs test results for v1 are like this:
> >
> > Host, v4.14:
> > Dry-run: 0 1
> > Self-IPI: 9 18
> > Normal IPI: 81 110
> > Broadcast IPI: 0 2106
> >
> > Guest, v4.14:
> > Dry-run: 0 1
> > Self-IPI: 10 18
> > Normal IPI: 305 525
> > Broadcast IPI: 0 9729
> >
> > Guest, v4.14 + VHE:
> > Dry-run: 0 1
> > Self-IPI: 9 18
> > Normal IPI: 176 343
> > Broadcast IPI: 0 9885
> >
> > And for v2.
> >
> > Host, v4.15:
> > Dry-run: 0 1
> > Self-IPI: 9 18
> > Normal IPI: 79 108
> > Broadcast IPI: 0 2102
> >
> > Guest, v4.15-rc:
> > Dry-run: 0 1
> > Self-IPI: 9 18
> > Normal IPI: 291 526
> > Broadcast IPI: 0 10439
> >
> > Guest, v4.15-rc + VHE:
> > Dry-run: 0 2
> > Self-IPI: 14 28
> > Normal IPI: 370 569
> > Broadcast IPI: 0 11688
> >
> > All times are normalized to v1 host dry-run time. Smaller - better.
> >
> > Results for v1 and v2 may vary because kernel version is changed.
> > What makes us worry is slowing down the "Normal IPI" test observed in
> > v2 series.
> It'd be interesting if you could profile your system to find our where
> you're spending time. My own tests, with a different benchmark, did show
> a 40% reduction in the number of *cycles*.
40% reduction is what I also observed for v1, to be specific - 42%.
So I was surprised when found v2 slower than vanilla kernel. Did you
observe 40% reduction for v2 or v1, or both?
I also think to switch to *cycles* as it (doubtly) might be CPU
frequency scaling issue, and do some profiling.
Yury
WARNING: multiple messages have this Message-ID (diff)
From: ynorov@caviumnetworks.com (Yury Norov)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v2 00/36] Optimize KVM/ARM for VHE systems
Date: Mon, 11 Dec 2017 18:14:33 +0300 [thread overview]
Message-ID: <20171211151433.3mmqiwau5wepknxl@yury-thinkpad> (raw)
In-Reply-To: <e238af17-d63c-52cd-c411-c2d3f3d86994@arm.com>
On Mon, Dec 11, 2017 at 02:56:01PM +0000, Marc Zyngier wrote:
> On 11/12/17 14:43, Yury Norov wrote:
> > Hi Christoffer,
> >
> > On Thu, Dec 07, 2017 at 06:05:54PM +0100, Christoffer Dall wrote:
> >> This series redesigns parts of KVM/ARM to optimize the performance on
> >> VHE systems. The general approach is to try to do as little work as
> >> possible when transitioning between the VM and the hypervisor. This has
> >> the benefit of lower latency when waiting for interrupts and delivering
> >> virtual interrupts, and reduces the overhead of emulating behavior and
> >> I/O in the host kernel.
> >>
> >> Patches 01 through 04 are not VHE specific, but rework parts of KVM/ARM
> >> that can be generally improved. We then add infrastructure to move more
> >> logic into vcpu_load and vcpu_put, we improve handling of VFP and debug
> >> registers.
> >>
> >> We then introduce a new world-switch function for VHE systems, which we
> >> can tweak and optimize for VHE systems. To do that, we rework a lot of
> >> the system register save/restore handling and emulation code that may
> >> need access to system registers, so that we can defer as many system
> >> register save/restore operations to vcpu_load and vcpu_put, and move
> >> this logic out of the VHE world switch function.
> >>
> >> We then optimize the configuration of traps. On non-VHE systems, both
> >> the host and VM kernels run in EL1, but because the host kernel should
> >> have full access to the underlying hardware, but the VM kernel should
> >> not, we essentially make the host kernel more privileged than the VM
> >> kernel despite them both running at the same privilege level by enabling
> >> VE traps when entering the VM and disabling those traps when exiting the
> >> VM. On VHE systems, the host kernel runs in EL2 and has full access to
> >> the hardware (as much as allowed by secure side software), and is
> >> unaffected by the trap configuration. That means we can configure the
> >> traps for VMs running in EL1 once, and don't have to switch them on and
> >> off for every entry/exit to/from the VM.
> >>
> >> Finally, we improve our VGIC handling by moving all save/restore logic
> >> out of the VHE world-switch, and we make it possible to truly only
> >> evaluate if the AP list is empty and not do *any* VGIC work if that is
> >> the case, and only do the minimal amount of work required in the course
> >> of the VGIC processing when we have virtual interrupts in flight.
> >>
> >> The patches are based on v4.15-rc1 plus the fixes sent for v4.15-rc3
> >> [1], the level-triggered mapped interrupts support series [2], and the
> >> first five patches of James' SDEI series [3], a single SVE patch that
> >> moves the CPU ID reg trap setup out of the world-switch path, and v3 of
> >> my vcpu load/put series [4].
> >>
> >> I've given the patches a fair amount of testing on Thunder-X, Mustang,
> >> Seattle, and TC2 (32-bit) for non-VHE testing, and tested VHE
> >> functionality on the Foundation model, running both 64-bit VMs and
> >> 32-bit VMs side-by-side and using both GICv3-on-GICv3 and
> >> GICv2-on-GICv3.
> >>
> >> The patches are also available in the vhe-optimize-v2 branch on my
> >> kernel.org repository [5].
> >>
> >> Changes since v1:
> >> - Rebased on v4.15-rc1 and newer versions of other dependencies,
> >> including the vcpu load/put approach taken for KVM.
> >> - Addressed review comments from v1 (detailed changelogs are in the
> >> individual patches).
> >>
> >> Thanks,
> >> -Christoffer
> >>
> >> [1]: git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm kvm-arm-fixes-for-v4.15-1
> >> [2]: git://git.kernel.org/pub/scm/linux/kernel/git/cdall/linux.git level-mapped-v6
> >> [3]: git://linux-arm.org/linux-jm.git sdei/v5/base
> >> [4]: git://git.kernel.org/pub/scm/linux/kernel/git/cdall/linux.git vcpu-load-put-v3
> >> [5]: git://git.kernel.org/pub/scm/linux/kernel/git/cdall/linux.git vhe-optimize-v2
> >
> > I just submitted the benchmark I used to test your v1 and v2 series':
> > https://lkml.org/lkml/2017/12/11/364
> >
> > On ThunderX2, 112 online CPUs test results for v1 are like this:
> >
> > Host, v4.14:
> > Dry-run: 0 1
> > Self-IPI: 9 18
> > Normal IPI: 81 110
> > Broadcast IPI: 0 2106
> >
> > Guest, v4.14:
> > Dry-run: 0 1
> > Self-IPI: 10 18
> > Normal IPI: 305 525
> > Broadcast IPI: 0 9729
> >
> > Guest, v4.14 + VHE:
> > Dry-run: 0 1
> > Self-IPI: 9 18
> > Normal IPI: 176 343
> > Broadcast IPI: 0 9885
> >
> > And for v2.
> >
> > Host, v4.15:
> > Dry-run: 0 1
> > Self-IPI: 9 18
> > Normal IPI: 79 108
> > Broadcast IPI: 0 2102
> >
> > Guest, v4.15-rc:
> > Dry-run: 0 1
> > Self-IPI: 9 18
> > Normal IPI: 291 526
> > Broadcast IPI: 0 10439
> >
> > Guest, v4.15-rc + VHE:
> > Dry-run: 0 2
> > Self-IPI: 14 28
> > Normal IPI: 370 569
> > Broadcast IPI: 0 11688
> >
> > All times are normalized to v1 host dry-run time. Smaller - better.
> >
> > Results for v1 and v2 may vary because kernel version is changed.
> > What makes us worry is slowing down the "Normal IPI" test observed in
> > v2 series.
> It'd be interesting if you could profile your system to find our where
> you're spending time. My own tests, with a different benchmark, did show
> a 40% reduction in the number of *cycles*.
40% reduction is what I also observed for v1, to be specific - 42%.
So I was surprised when found v2 slower than vanilla kernel. Did you
observe 40% reduction for v2 or v1, or both?
I also think to switch to *cycles* as it (doubtly) might be CPU
frequency scaling issue, and do some profiling.
Yury
next prev parent reply other threads:[~2017-12-11 15:14 UTC|newest]
Thread overview: 158+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-07 17:05 [PATCH v2 00/36] Optimize KVM/ARM for VHE systems Christoffer Dall
2017-12-07 17:05 ` Christoffer Dall
2017-12-07 17:05 ` [PATCH v2 01/36] KVM: arm64: Avoid storing the vcpu pointer on the stack Christoffer Dall
2017-12-07 17:05 ` Christoffer Dall
2017-12-09 17:19 ` Marc Zyngier
2017-12-09 17:19 ` Marc Zyngier
2017-12-11 9:30 ` Christoffer Dall
2017-12-11 9:30 ` Christoffer Dall
2017-12-11 9:35 ` Marc Zyngier
2017-12-11 9:35 ` Marc Zyngier
2017-12-07 17:05 ` [PATCH v2 02/36] KVM: arm64: Rework hyp_panic for VHE and non-VHE Christoffer Dall
2017-12-07 17:05 ` Christoffer Dall
2017-12-09 17:24 ` Marc Zyngier
2017-12-09 17:24 ` Marc Zyngier
2017-12-07 17:05 ` [PATCH v2 03/36] KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag Christoffer Dall
2017-12-07 17:05 ` Christoffer Dall
2017-12-07 17:05 ` [PATCH v2 04/36] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines Christoffer Dall
2017-12-07 17:05 ` Christoffer Dall
2017-12-07 17:05 ` [PATCH v2 05/36] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs Christoffer Dall
2017-12-07 17:05 ` Christoffer Dall
2017-12-09 17:30 ` Marc Zyngier
2017-12-09 17:30 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 06/36] KVM: arm64: Defer restoring host VFP state to vcpu_put Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-09 17:37 ` Marc Zyngier
2017-12-09 17:37 ` Marc Zyngier
2017-12-11 9:31 ` Christoffer Dall
2017-12-11 9:31 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 07/36] KVM: arm64: Move debug dirty flag calculation out of world switch Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-09 19:20 ` Marc Zyngier
2017-12-09 19:20 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 08/36] KVM: arm64: Slightly improve debug save/restore functions Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-09 19:37 ` Marc Zyngier
2017-12-09 19:37 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 09/36] KVM: arm64: Improve debug register save/restore flow Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 9:40 ` Marc Zyngier
2017-12-11 9:40 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 10/36] KVM: arm64: Factor out fault info population and gic workarounds Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 9:45 ` Marc Zyngier
2017-12-11 9:45 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 11/36] KVM: arm64: Introduce VHE-specific kvm_vcpu_run Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 9:53 ` Marc Zyngier
2017-12-11 9:53 ` Marc Zyngier
2017-12-14 12:12 ` Christoffer Dall
2017-12-14 12:12 ` Christoffer Dall
2017-12-14 12:17 ` Marc Zyngier
2017-12-14 12:17 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 12/36] KVM: arm64: Remove kern_hyp_va() use in VHE switch function Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 9:54 ` Marc Zyngier
2017-12-11 9:54 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 13/36] KVM: arm64: Don't deactivate VM on VHE systems Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 9:58 ` Marc Zyngier
2017-12-11 9:58 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 14/36] KVM: arm64: Remove noop calls to timer save/restore from VHE switch Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 10:02 ` Marc Zyngier
2017-12-11 10:02 ` Marc Zyngier
2017-12-14 12:30 ` Christoffer Dall
2017-12-14 12:30 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 15/36] KVM: arm64: Move userspace system registers into separate function Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 10:14 ` Marc Zyngier
2017-12-11 10:14 ` Marc Zyngier
2017-12-14 12:53 ` Christoffer Dall
2017-12-14 12:53 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 16/36] KVM: arm64: Rewrite sysreg alternatives to static keys Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 10:15 ` Marc Zyngier
2017-12-11 10:15 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 17/36] KVM: arm64: Introduce separate VHE/non-VHE sysreg save/restore functions Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 10:22 ` Marc Zyngier
2017-12-11 10:22 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 18/36] KVM: arm/arm64: Remove leftover comment from kvm_vcpu_run_vhe Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 10:30 ` Marc Zyngier
2017-12-11 10:30 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 19/36] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 10:40 ` Marc Zyngier
2017-12-11 10:40 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 20/36] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 10:44 ` Marc Zyngier
2017-12-11 10:44 ` Marc Zyngier
2017-12-14 13:46 ` Christoffer Dall
2017-12-14 13:46 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 21/36] KVM: arm64: Change 32-bit handling of VM system registers Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 10:57 ` Marc Zyngier
2017-12-11 10:57 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 22/36] KVM: arm64: Prepare to handle traps on deferred VM sysregs Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 11:10 ` Marc Zyngier
2017-12-11 11:10 ` Marc Zyngier
2017-12-11 11:24 ` Christoffer Dall
2017-12-11 11:24 ` Christoffer Dall
2017-12-11 11:46 ` Marc Zyngier
2017-12-11 11:46 ` Marc Zyngier
2017-12-12 13:08 ` Marc Zyngier
2017-12-12 13:08 ` Marc Zyngier
2017-12-12 15:46 ` Christoffer Dall
2017-12-12 15:46 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 23/36] KVM: arm64: Prepare to handle traps on deferred EL0 sysregs Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 24/36] KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 25/36] KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 26/36] KVM: arm64: Defer saving/restoring system registers to vcpu load/put on VHE Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 13:20 ` Marc Zyngier
2017-12-11 13:20 ` Marc Zyngier
2017-12-15 16:29 ` Christoffer Dall
2017-12-15 16:29 ` Christoffer Dall
2017-12-29 16:39 ` Christoffer Dall
2017-12-29 16:39 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 27/36] KVM: arm64: Move common VHE/non-VHE trap config in separate functions Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 13:53 ` Marc Zyngier
2017-12-11 13:53 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 28/36] KVM: arm64: Configure FPSIMD traps on vcpu load/put for VHE Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 14:18 ` Marc Zyngier
2017-12-11 14:18 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 29/36] KVM: arm64: Configure c15, PMU, and debug register traps on cpu " Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 30/36] KVM: arm64: Separate activate_traps and deactive_traps for VHE and non-VHE Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 31/36] KVM: arm/arm64: Get rid of vgic_elrsr Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 32/36] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 33/36] KVM: arm/arm64: Move arm64-only vgic-v2-sr.c file to arm64 Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 34/36] KVM: arm/arm64: Handle VGICv3 save/restore from the main VGIC code on VHE Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 35/36] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 36/36] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs Christoffer Dall
2017-12-07 17:06 ` Christoffer Dall
2017-12-11 14:43 ` [PATCH v2 00/36] Optimize KVM/ARM for VHE systems Yury Norov
2017-12-11 14:43 ` Yury Norov
2017-12-11 14:56 ` Marc Zyngier
2017-12-11 14:56 ` Marc Zyngier
2017-12-11 15:14 ` Yury Norov [this message]
2017-12-11 15:14 ` Yury Norov
2017-12-11 15:34 ` Christoffer Dall
2017-12-11 15:34 ` Christoffer Dall
2017-12-11 16:30 ` Yury Norov
2017-12-11 16:30 ` Yury Norov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171211151433.3mmqiwau5wepknxl@yury-thinkpad \
--to=ynorov@caviumnetworks.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=marc.zyngier@arm.com \
--cc=shihwei@cs.columbia.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.