Re: Re: [PATCH v2 00/94] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support

Re: Re: [PATCH v2 00/94] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support

[George Cherian]

> -----Original Message-----
> From: Marc Zyngier <maz@kernel.org>
> Sent: Friday, April 3, 2020 1:32 PM
> To: George Cherian <gcherian@marvell.com>
> Cc: Dave.Martin@arm.com; alexandru.elisei@arm.com;
> andre.przywara@arm.com; christoffer.dall@arm.com;
> james.morse@arm.com; jintack@cs.columbia.edu;
> julien.thierry.kdev@gmail.com; kvm@vger.kernel.org;
> kvmarm@lists.cs.columbia.edu; linux-arm-kernel@lists.infradead.org;
> suzuki.poulose@arm.com; Anil Kumar Reddy H <areddy3@marvell.com>;
> Ganapatrao Kulkarni <gkulkarni@marvell.com>
> Subject: Re: [PATCH v2 00/94] KVM: arm64: ARMv8.3/8.4 Nested
> Virtualization support
> 
> 
> ----------------------------------------------------------------------
> Hi George,
> 
> On 2020-04-03 08:27, George Cherian wrote:
> > Hi Marc,
> >
> > On 2/11/20 9:48 AM, Marc Zyngier wrote:
> >> This is a major rework of the NV series that I posted over 6 months
> >> ago[1], and a lot has changed since then:
> >>
> >> - Early ARMv8.4-NV support
> >> - ARMv8.4-TTL support in host and guest
> >> - ARMv8.5-GTG support in host and guest
> >> - Lots of comments addressed after the review
> >> - Rebased on v5.6-rc1
> >> - Way too many patches
> >>
> >> In my defence, the whole of the NV code is still smaller that the
> >> 32bit KVM/arm code I'm about to remove, so I feel less bad inflicting
> >> this on everyone! ;-)
> >>
> >> >From a functionality perspective, you can expect a L2 guest to work,
> >> but don't even think of L3, as we only partially emulate the
> >> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug,
> >> PMU, as well as anything that would require a Stage-1 PTW. What we
> >> want to achieve is that with NV disabled, there is no performance
> >> overhead and no regression.
> >>
> >> The series is roughly divided in 5 parts: exception handling, memory
> >> virtualization, interrupts and timers for ARMv8.3, followed by the
> >> ARMv8.4 support. There are of course some dependencies, but you'll
> >> hopefully get the gist of it.
> >>
> >> For the most courageous of you, I've put out a branch[2]. Of course,
> >> you'll need some userspace. Andre maintains a hacked version of
> >> kvmtool[3] that takes a --nested option, allowing the guest to be
> >> started at EL2. You can run the whole stack in the Foundation model.
> >> Don't be in a hurry ;-).
> >>
> > The full series was tested on both Foundation model as well as Marvell
> > ThunderX3
> > Emulation Platform.
> > Basic boot testing done for Guest Hypervisor and Guest Guest.
> >
> > Tested-by:  George Cherian <george.cherian@marvell.com>
> 
> Thanks for having given this a go.
> 
> However, without more details, it is pretty hard to find out what you have
> tested.
> What sort of guest have you booted, with what configuration, what
> workloads did you run in the L2 guests and what are the architectural
> features that TX3 implements?
> 

We have tried the following configurations and tests (GH - Guest Hypervisor GG- Guest Guest).
1 - configuration: Host:8cpus/4GB Mem, GH:4vcpus/3GB, GG: 2vcpus/2GB
Ran hackbench and Large Malloc tests (1GB allocations) across HOST,GH and GG. 

2 - configuration: Host:8cpus/4GB Mem, GH:1vcpu/3GB, GG: 1vcpu/2GB
Ran hackbench and Large Malloc tests across HOST,GH and GG. Host:

We used QEMU for all these testing. 

TX3 implements v8.4 Enhanced Nested Virtualization Support.

> The last point is specially important, as the NV architecture spans two major
> revisions of the architecture and affects tons of other extensions that are
> themselves optional. Without any detail on that front, I have no idea what
> the coverage of your testing is.
> 
> Thanks,
> 
>          M.
> --
> Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Re: [PATCH v2 00/94] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support

[Marc Zyngier]

On 2020-04-03 10:36, George Cherian wrote:
>> -----Original Message-----
>> From: Marc Zyngier <maz@kernel.org>
>> Sent: Friday, April 3, 2020 1:32 PM
>> To: George Cherian <gcherian@marvell.com>
>> Cc: Dave.Martin@arm.com; alexandru.elisei@arm.com;
>> andre.przywara@arm.com; christoffer.dall@arm.com;
>> james.morse@arm.com; jintack@cs.columbia.edu;
>> julien.thierry.kdev@gmail.com; kvm@vger.kernel.org;
>> kvmarm@lists.cs.columbia.edu; linux-arm-kernel@lists.infradead.org;
>> suzuki.poulose@arm.com; Anil Kumar Reddy H <areddy3@marvell.com>;
>> Ganapatrao Kulkarni <gkulkarni@marvell.com>
>> Subject: Re: [PATCH v2 00/94] KVM: arm64: ARMv8.3/8.4 Nested
>> Virtualization support
>> 
>> 
>> ----------------------------------------------------------------------
>> Hi George,
>> 
>> On 2020-04-03 08:27, George Cherian wrote:
>> > Hi Marc,
>> >
>> > On 2/11/20 9:48 AM, Marc Zyngier wrote:
>> >> This is a major rework of the NV series that I posted over 6 months
>> >> ago[1], and a lot has changed since then:
>> >>
>> >> - Early ARMv8.4-NV support
>> >> - ARMv8.4-TTL support in host and guest
>> >> - ARMv8.5-GTG support in host and guest
>> >> - Lots of comments addressed after the review
>> >> - Rebased on v5.6-rc1
>> >> - Way too many patches
>> >>
>> >> In my defence, the whole of the NV code is still smaller that the
>> >> 32bit KVM/arm code I'm about to remove, so I feel less bad inflicting
>> >> this on everyone! ;-)
>> >>
>> >> >From a functionality perspective, you can expect a L2 guest to work,
>> >> but don't even think of L3, as we only partially emulate the
>> >> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug,
>> >> PMU, as well as anything that would require a Stage-1 PTW. What we
>> >> want to achieve is that with NV disabled, there is no performance
>> >> overhead and no regression.
>> >>
>> >> The series is roughly divided in 5 parts: exception handling, memory
>> >> virtualization, interrupts and timers for ARMv8.3, followed by the
>> >> ARMv8.4 support. There are of course some dependencies, but you'll
>> >> hopefully get the gist of it.
>> >>
>> >> For the most courageous of you, I've put out a branch[2]. Of course,
>> >> you'll need some userspace. Andre maintains a hacked version of
>> >> kvmtool[3] that takes a --nested option, allowing the guest to be
>> >> started at EL2. You can run the whole stack in the Foundation model.
>> >> Don't be in a hurry ;-).
>> >>
>> > The full series was tested on both Foundation model as well as Marvell
>> > ThunderX3
>> > Emulation Platform.
>> > Basic boot testing done for Guest Hypervisor and Guest Guest.
>> >
>> > Tested-by:  George Cherian <george.cherian@marvell.com>
>> 
>> Thanks for having given this a go.
>> 
>> However, without more details, it is pretty hard to find out what you 
>> have
>> tested.
>> What sort of guest have you booted, with what configuration, what
>> workloads did you run in the L2 guests and what are the architectural
>> features that TX3 implements?
>> 
> 
> We have tried the following configurations and tests (GH - Guest
> Hypervisor GG- Guest Guest).
> 1 - configuration: Host:8cpus/4GB Mem, GH:4vcpus/3GB, GG: 2vcpus/2GB
> Ran hackbench and Large Malloc tests (1GB allocations) across HOST,GH 
> and GG.
> 
> 2 - configuration: Host:8cpus/4GB Mem, GH:1vcpu/3GB, GG: 1vcpu/2GB
> Ran hackbench and Large Malloc tests across HOST,GH and GG. Host:
> 
> We used QEMU for all these testing.

Interesting. So you have added NV support to QEMU? Please be aware that
the userspace side is pretty incomplete and subject to changes.

> TX3 implements v8.4 Enhanced Nested Virtualization Support.

Hmm. Ok. This doesn't really answer my question in terms of what other
features the CPU has that would be affected by NV, but I guess that's
all we'll get at this point.

Thanks,

         M.

> 
>> The last point is specially important, as the NV architecture spans 
>> two major
>> revisions of the architecture and affects tons of other extensions 
>> that are
>> themselves optional. Without any detail on that front, I have no idea 
>> what
>> the coverage of your testing is.
>> 
>> Thanks,
>> 
>>          M.
>> --
>> Jazz is not dead. It just smells funny...

-- 
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm