Re: [Qemu-devel] [PATCH v8 11/12] vfio: register aer resume notification handler for aer resume

[Zhou Jie <zhoujie2011@cn.fujitsu.com>]

Hi Alex,

On 2016/6/30 2:22, Alex Williamson wrote:
> On Wed, 29 Jun 2016 16:54:05 +0800
> Zhou Jie <zhoujie2011@cn.fujitsu.com> wrote:
>
>> Hi Alex,
>>
>>> And yet we have struct pci_dev.broken_intx_masking and we test for
>>> working DisINTx via pci_intx_mask_supported() rather than simply
>>> looking for a PCIe device.  Some devices are broken and some simply
>>> don't follow the spec, so you're going to need to deal with that or
>>> exclude those devices.
>> For those devices I have no way to disable the INTx.
>
> disable_irq()?  Clearly vfio-pci already manages these types of devices
> and can disable INTx.  This is why I keep suggesting that maybe tearing
> the interrupt setup down completely is a more complete and reliable
> approach than masking in the command register.  Unless we're going to
> exclude such devices from supporting this mode (which I don't condone),
> we must deal with them.
Thank you for tell me that.
Yes, I can use disable_irq to disable the pci device irq.
But should I enable the irq after reset?
I will dig into it.

Sincerely
Zhou Jie

>>> How does that happen, aren't we notifying the user at the point the
>>> error occurs, while the device is still in the process or being reset?
>>> My question is how does the user know that the host reset is complete
>>> in order to begin their own re-initialization?
>> I will add a state in "struct vfio_pci_device".
>> The state is set when the device can not work such as a aer error
>>   occured.
>> And the state is clear when the device can work such as resume
>>   received.
>> Return the state when user get info by vfio_pci_ioctl.
>>
>>>> The interrupt status will be cleared by hardware.
>>>> So the hardware is the same as the state when the
>>>> vfio device fd is opened.
>>>
>>> The PCI-core in Linux will save and restore the device state around
>>> reset, how do we know that vfio-pci itself is not racing that reset and
>>> whether PCI-core will restore the state including our interrupt masking
>>> or a state without it?  Do we need to restore the state to the one we
>>> saved when we originally opened the device?  Shouldn't that mean we
>>> teardown the interrupt setup the user had prior to the error event?
>> For above you said.
>> Maybe disable the interrupt is not a good idea.
>> Think about what will happend in the interrupt handler.
>> Maybe read/write configure space and region bar.
>> I will make the configure space read only.
>> Do nothing for region bar which used by userd.
>
> I'm thinking that vfio-pci will be attempting to mask the interrupts
> via the PCI command register, which is potentially in a state of flux
> due to the host reset and yet we're somehow expecting that our write to
> the command register sticks.  We certainly have the ability to a)
> discard interrupts received between AER error and resume, and b) if we
> want to be consistent with requiring the user to reinitialize the
> device, then the user interrupt setup should likely be torn down.
> Thanks,