Re: [Qemu-devel] [PATCH v9 05/11] vfio: add check host bus reset is support or not

[Dou Liyang <douly.fnst@cn.fujitsu.com>]

Hi Alex,

At 09/01/2016 03:56 AM, Alex Williamson wrote:
> On Tue, 19 Jul 2016 15:38:23 +0800
> Zhou Jie <zhoujie2011@cn.fujitsu.com> wrote:
>
>> From: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
>>
>> When assigning a vfio device with AER enabled, we must check whether
>> the device supports a host bus reset (ie. hot reset) as this may be
>> used by the guest OS in order to recover the device from an AER
>> error.  QEMU must therefore have the ability to perform a physical
>> host bus reset using the existing vfio APIs in response to a virtual
>> bus reset in the VM.  A physical bus reset affects all of the devices
>> on the host bus, therefore we place a few simplifying configuration
>> restriction on the VM:
>>
>>  - All physical devices affected by a bus reset must be assigned to
>>    the VM with AER enabled on each and be configured on the same
>>    virtual bus in the VM.
>>
>>  - No devices unaffected by the bus reset, be they physical, emulated,
>>    or paravirtual may be configured on the same virtual bus as a
>>    device supporting AER signaling through vfio.
>>
>> In other words users wishing to enable AER on a multifunction device
>> need to assign all functions of the device to the same virtual bus
>> and enable AER support for each device.  The easiest way to
>> accomplish this is to identity map the physical functions to virtual
>> functions with multifunction enabled on the virtual device.
>
> Why am I able to start the following VM with aer=on for the vfio-pci
> devices?

In my opinion, because, when we check out this situation in vfio_init,
our code just do nothing except going to out_teardown. so the devices 
which are behind a PCIe-to-PCI bridge will be regarded as the PCI devices.

>
> # lspci -tv
> -[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
>            +-01.0  Device 1234:1111
>            +-1c.0-[01]--
>            +-1d.0-[02]--+-01.0  Intel Corporation 82576 Gigabit Network Connection
>            |            \-01.1  Intel Corporation 82576 Gigabit Network Connection
>            ...
>
> # lspci -vvv -s 1d.0
> 00:1d.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge (prog-if 00 [Normal decode])
>
> The devices are behind a PCIe-to-PCI bridge, so shouldn't specifying
> aer=on for the vfio-pci devices cause a configuration error?

Yes, I think it may makes users confuse.

I added a error_report for it.

Then, I test many passible combinations, such as:

pci-bridge
     |- vfio-pci

-device pci-bridge,chassis_nr=3,id=pci.3,bus=pcie.0,addr=0x1d \
-device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.3,addr=1 \

error:
vfio: Unable to enable AER for device 0000:06:00.0, device does not 
support AER signaling

--------------------------------------------------------------------

pci-bridge
     |- vfio-pci
     |- vfio-pci

-device pci-bridge,chassis_nr=3,id=pci.3,bus=pcie.0,addr=0x1d \
-device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.3,addr=1 \
-device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=pci.3,addr=1.1 \

error:
vfio: Unable to enable AER for device 0000:06:00.0, device does not 
support AER signaling

--------------------------------------------------------------------
ioh3420
   |- pci-bridge
       |- vfio-pci
       |- vfio-pci

-device ioh3420,bus=pcie.0,addr=1c.0,port=1,id=bridge1,chassis=1 \
-device pci-bridge,chassis_nr=3,id=pci.1,bus=bridge1,addr=0x1d \
-device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.1,addr=1 \
-device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=pci.1,addr=1.1 \

error:
vfio: Unable to enable AER for device 0000:06:00.0, device does not 
support AER signaling

--------------------------------------------------------------------
just for test:
pci-bridge
   |- ioh3420
       |- vfio-pci
       |- vfio-pci
-device pci-bridge,chassis_nr=3,id=pci.1,bus=pcie.0,addr=0x1d \
-device ioh3420,bus=pci.1,addr=1c.0,port=1,id=bridge1,chassis=1 \
-device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=bridge1,addr=1 \
-device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=bridge1,addr=1.1 \

error:
vfio: Unable to enable AER for device 0000:06:00.0, parent bus does not 
support AER signaling

--------------------------------------------------------------------
ioh3420
    |- vfio-pci
    |- vfio-pci

-device ioh3420,bus=pcie.0,addr=1c.0,port=1,id=bridge1,chassis=1 \
-device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=bridge1,addr=1 \
-device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=bridge1,addr=1.1 \

OK
--------------------------------------------------------------------
...

Thanks,
Dou

>

[...]


>