[Qemu-devel] vfio/pci: IGD assignment

[Qemu-devel] vfio/pci: IGD assignment

[nicolas prochazka]

Hello,
Since I upgrade my linux kernel from linux 4.8rc1 to 4.8.0, I can observe
regression with VFIO/PCI IGD Assignment .
All works very well with linux 4.8rc1 ( broadwell/skylake cpu + win10 guest
/ Linux Guest ) .
Since linux 4.8.0, Windows guest screen is a " blink pixel , multi color
blink , a matrix revisited.

Only linux kernel change between two test,
no log in qemu or dmesg.

Is there modification to vfio in kernel between 4.8rc1 and 4.8.0 ? , can i
test something.

Qemu version is 2.7.0
my libvirt configuration :
<domain type='kvm' id='2' xmlns:qemu='
http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>win10_sabrina</name>
  <uuid>7e3cc74f-e3c9-3753-a4dc-a63d5ff8c415</uuid>
  <memory unit='KiB'>14418944</memory>
  <currentMemory unit='KiB'>14417969</currentMemory>
  <memoryBacking>
    <hugepages/>
  </memoryBacking>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type>
    <loader readonly='yes' type='rom'>/usr/share/seabios/bios.bin</loader>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hap/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-kvm.igd</emulator>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw'/>
      <source protocol='nbd'>
        <host transport='unix' socket='/tmp/win10_sabrina.ctl'/>
      </source>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw'/>
      <source protocol='nbd'>
        <host transport='unix' socket='/tmp/swap.ctl'/>
      </source>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='ac:de:50:7d:88:85'/>
      <source bridge='switch0'/>
      <vlan>
        <tag id='0'/>
      </vlan>
      <virtualport type='openvswitch'>
        <parameters interfaceid='1fa42874-b2aa-44c6-88da-2203088ce6f9'/>
      </virtualport>
      <target dev='vnet0'/>
      <model type='e1000'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
    </interface>
    <channel type='unix'>
      <source mode='bind'
path='/var/lib/libvirt/qemu/channel/target/win10_sabrina.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0'
state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x14' function='0x0'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x3'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x0'/>
    </hostdev>
    <memballoon model='none'>
      <alias name='balloon0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-netdev
user,id=user.0,restrict=on,hostfwd=tcp:127.0.0.1:5901-:5900 -device
e1000,netdev=user.0'/>
  </qemu:commandline>
</domain>


qemu cmd line : qemu-nbd --aio=native --discard=unmap --detect-zeroes=on
--cache=none /data/VMS/win10_sabrina/hda.qcow2 -k /tmp/win10_sabrina.ctl -t
13930 root       0:00 qemu-nbd --cache=unsafe -s --discard=unmap
--detect-zeroes=on /swap.ntfs -k /tmp/swap.ctl -t
14013 root     305:17 /usr/bin/qemu-system-x86_64.igd -name win10_sabrina
-S -machine pc-i440fx-2.7,accel=kvm,usb=off -cpu
host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff -bios
/usr/share/seabios/bios.bin -m 14081 -mem-prealloc -mem-path
/dev/hugepages/libvirt/qemu -realtime mlock=off -smp
2,sockets=1,cores=2,threads=1 -uuid 7e3cc74f-e3c9-3753-a4dc-a63d5ff8c415
-nographic -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/win10_sabrina.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime
-no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global
PIIX4_PM.disable_s4=1 -boot strict=on -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive
file=nbd:unix:/tmp/win10_sabrina.ctl,if=none,id=drive-ide0-0-0,format=raw
-device
ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive
file=nbd:unix:/tmp/swap.ctl,if=none,id=drive-ide0-0-1,format=raw -device
ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev
tap,fd=20,id=hostnet0 -device
e1000,netdev=hostnet0,id=net0,mac=ac:de:50:7d:88:85,bus=pci.0,addr=0x3
-chardev
socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/win10_sabrina.org.qemu.guest_agent.0,server,nowait
-device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
-device vfio-pci,host=00:02.0,id=hostdev0,bus=pci.0,addr=0x2 -device
vfio-pci,host=00:14.0,id=hostdev1,bus=pci.0,addr=0x5 -device
vfio-pci,host=00:1f.3,id=hostdev2,bus=pci.0,addr=0x6 -netdev
user,id=user.0,restrict=on,hostfwd=tcp:127.0.0.1:5901-:5900 -device
e1000,netdev=user.0 -msg timestamp=on



Regards,
Nicolas Prochazka

Re: [Qemu-devel] vfio/pci: IGD assignment

[Alex Williamson]

On Thu, 6 Oct 2016 08:06:24 +0200
nicolas prochazka <prochazka.nicolas@gmail.com> wrote:

> Hello,
> Since I upgrade my linux kernel from linux 4.8rc1 to 4.8.0, I can observe
> regression with VFIO/PCI IGD Assignment .
> All works very well with linux 4.8rc1 ( broadwell/skylake cpu + win10 guest
> / Linux Guest ) .
> Since linux 4.8.0, Windows guest screen is a " blink pixel , multi color
> blink , a matrix revisited.
> 
> Only linux kernel change between two test,
> no log in qemu or dmesg.
> 
> Is there modification to vfio in kernel between 4.8rc1 and 4.8.0 ? , can i
> test something.

Hmm, I was running 4.8-rc5 on my system, I upgraded to v4.8 and it
still works.  There's only one change in drivers/vfio from rc1 to 4.8:

$ git log --oneline v4.8-rc1..v4.8 drivers/vfio
c8952a7 vfio/pci: Fix NULL pointer oops in error interrupt setup handling

I notice you're also assigning device 1f.3, which means you're using
the acs override patch, which means you're not running a stock v4.8.
Thanks,

Alex

Re: [Qemu-devel] vfio/pci: IGD assignment

[nicolas prochazka]

Hello,
I'm just add
{ PCI_VENDOR_ID_INTEL, 0x9d70, pci_quirk_mf_endpoint_acs },
in drivers/pci/quirks.c   for the sound card on intel nuc skylake.
This issues is present on broadwell, but it's very strange behaviour
because it depends how the windows igd drivers seems to be installed.
I'm trying to investigate, now i known there's no vfio modification in
kernel code
Regards,
Nicolas

2016-10-06 17:56 GMT+02:00 Alex Williamson <alex.williamson@redhat.com>:

> On Thu, 6 Oct 2016 08:06:24 +0200
> nicolas prochazka <prochazka.nicolas@gmail.com> wrote:
>
> > Hello,
> > Since I upgrade my linux kernel from linux 4.8rc1 to 4.8.0, I can observe
> > regression with VFIO/PCI IGD Assignment .
> > All works very well with linux 4.8rc1 ( broadwell/skylake cpu + win10
> guest
> > / Linux Guest ) .
> > Since linux 4.8.0, Windows guest screen is a " blink pixel , multi color
> > blink , a matrix revisited.
> >
> > Only linux kernel change between two test,
> > no log in qemu or dmesg.
> >
> > Is there modification to vfio in kernel between 4.8rc1 and 4.8.0 ? , can
> i
> > test something.
>
> Hmm, I was running 4.8-rc5 on my system, I upgraded to v4.8 and it
> still works.  There's only one change in drivers/vfio from rc1 to 4.8:
>
> $ git log --oneline v4.8-rc1..v4.8 drivers/vfio
> c8952a7 vfio/pci: Fix NULL pointer oops in error interrupt setup handling
>
> I notice you're also assigning device 1f.3, which means you're using
> the acs override patch, which means you're not running a stock v4.8.
> Thanks,
>
> Alex
>