Re: [Qemu-devel] [PATCH v8 11/12] vfio: register aer resume notification handler for aer resume

[Zhou Jie <zhoujie2011@cn.fujitsu.com>]

Hi Alex,

On 2016/6/28 11:58, Alex Williamson wrote:
> On Tue, 28 Jun 2016 11:26:33 +0800
> Zhou Jie <zhoujie2011@cn.fujitsu.com> wrote:
>
>> Hi Alex,
>>
>>> The INTx/MSI part needs further definition for the user.  Are we
>>> actually completely tearing down interrupts with the expectation that
>>> the user will re-enable them or are we just masking them such that the
>>> user needs to unmask?  Also note that not all devices support DisINTx.
>>
>> After reset, the "Bus Master Enable" bit of "Command Register"
>> should be cleared, so MSI/MSI- X interrupt Messages is still disabled.
>> After reset, the "Interrupt Disable" bit of "Command Register"
>> should be cleared, so INTx interrupts is enabled.
>> If the device doesn't support INTx, "Interrupt Disable" bit will
>> hardware to 0, it is OK here.
>>
>> After fatal-error occurs, the user should reset the device and
>> reinitialize the device.
>> So I disable the interrupt before host reset the device,
>> and let user to do the reinitialization.
>
> I'm dubious here.  When DisINTx is not supported by the device or it's
> marked broken in host quirks, then we can't trust the device to stop
> sending INTx.  It's hardwired to zero, meaning that it doesn't work or
> it's been found to be broken in other ways.  So COMMAND register
> masking is not sufficient for all devices.
For Endpoints that generate INTx interrupts, this bit is required.
For Endpoints that do not generate IN Tx interrupts this bit is
optional.  If not implemented, this bit must be hardwired to 0b.
For Root Ports, Switch Ports, and Bridges that generate INTx
interrupts on their own behalf, this bit is required.

The above is from "7.5.1.1." of "PCI Express Base Specification 3.1a".
So I think "Interrupt Disable" bit must be supported by the device
which can generate INTx interrupts.

> Also, any time we start
> changing the state of the device from what the user expects, we risk
> consistency problems.  We need to consider how the user last saw the
> device and whether we can legitimately expect them to handle the device
> in a new state.  If we expect the user to re-initialize the device then
> would it be more correct to teardown all interrupt signaling such that
> the device is effectively in the same state as initial handoff when the
> vfio device fd is opened?
Before the user re-initialize the device, host has reseted the device.
The interrupt status will be cleared by hardware.
So the hardware is the same as the state when the
vfio device fd is opened.

> How will the user know when the device is
> ready to be reset?  Which of the ioctls that you're blocking can they
> poll w/o any unwanted side-effects or awkward interactions?  Should
> flag bits in the device info ioctl indicate not only support for this
> behavior but also the current status?  Thanks,
I can block the reset ioctl and config write.
I will not add flag for the device current status,
because I don't depend on user to prevent awkward interactions.

Sincerely
Zhou Jie