Re: AMD Ryzen KVM/NPT/IOMMU issue

["Bridgman, John" <John.Bridgman-5C7GfCeVMHo@public.gmane.org>]

[-- Attachment #1.1: Type: text/plain, Size: 3603 bytes --]


________________________________
From: Alex Williamson <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Sent: June 28, 2017 3:08 PM
To: Suthikulpanit, Suravee
Cc: Steven Walter; Nick Sarnie; Paolo Bonzini; iommu-cunTk1MwBs9QetFLy7KEmy65B3kUBaIG@public.gmane.org.org; Matthias Ehrenfeuchter; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bridgman, John
Subject: Re: AMD Ryzen KVM/NPT/IOMMU issue

On Thu, 29 Jun 2017 01:53:57 +0700
Suravee Suthikulpanit <Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org> wrote:

> On 6/29/17 00:26, Steven Walter wrote:
> >> So, I'm trying to reproduce this issue on the Ryzen system w/ the following
> >> setup:
> >>
> >>   * Host kernel v4.11 (with this patch https://lkml.org/lkml/2017/6/23/295)
> >>
> >>   * guest VM RHEL7.3
> >>
> >>   * guest graphic driver = radeon
> >>
> >>   * qemu-system-x86_64 --version
> >>     QEMU emulator version 2.9.50 (v2.9.0-1659-g577caa2-dirty)
> >>
> >>   * kvm-amd npt=1
> >>
> >>   * dGPU is 08:00.0 VGA compatible controller: Advanced Micro Devices, Inc.
> >> [AMD/ATI] Tobago PRO [Radeon R7 360 / R9 360 OEM] (rev 81)
> >>
> >>   * qemu-system-x86_64 -smp 4 -enable-kvm -M q35 -m 4096 -cpu host -bios
> >> /usr/share/qemu/bios.bin -device
> >> ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1
> >> -drive file=/sandbox/vm-images/rhel7.3.qcow2,if=virtio,id=disk0 -net none
> >> -vga none -nodefaults -device
> >> vfio-pci,host=08:00.0,x-vga=on,addr=0.0,multifunction=on,bus=root.1,romfile=/sandbox/vm-images/vbios.rom
> >> -usb -device usb-host,hostbus=3,hostport=1 -device
> >> usb-host,hostbus=3,hostport=3 -device vfio-pci,host=0000:08:00.1 -device
> >> vfio-pci,host=0000:09:00.0
> >>
> >> With this setup, I am able to pass-through the dGPU and run the following
> >> test:
> >>   * Starting up the guest w/ full GNOME GUI on the attached monitor.
> >>   * glxgears (running @ 60 FPS)
> >>   * Playing 1080p HD video on Youtube
> >>
> >> I am not noticing issues here. What kind of test are you running in the
> >> guest VM?
> > Try running the open source game "torcs" inside the VM.  I think
> > you'll find that there's a very noticeable performance different
> > between npt=1 and npt=0
>
> Hm.. actually torcs seems to be running fine w/ npt=1 setup. Although I think my
> driving skill is ~20% worse :(

>A clarification on the issue, it's not that these games/benchmarks
>don't work or aren't playable, it's that they run slower (as measured by
>framerate) with npt=1 vs npt=0.  A virtualized guest on AMD hardware is
>hindered either by lower graphics performance (npt=1) or higher CPU
>virtualization overhead (npt=0) for high intensity games or graphics
>workloads.  Intel's equivalent ept feature does not have this issue.  I
>would encourage looking at the framerate for one mode vs the other
>before drawing any conclusions on whether it's working well.  Thanks,
>Alex

One more data point - Nick did some testing with Xen enabling/disabling npt and
found that (a) performance was not affected much whether npt was on or off, and
(b) performance with npt on was pretty close to KVM performance with npt off.

This suggests something specific to KVM that doesn't play well with npt, although
I have no idea what that might be. I was going to talk to our folks to see if they had
any suggestions re: ways to narrow down where the performance impact is coming
from, but I ended up going off sick instead.

Haven't gone through the whole thread here yet so apologies if this has already been
mentioned.

[-- Attachment #1.2: Type: text/html, Size: 5218 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]