Re: [Qemu-devel] [PATCH v8 11/12] vfio: register aer resume notification handler for aer resume

From: Alex Williamson <alex.williamson@redhat.com>
To: Zhou Jie <zhoujie2011@cn.fujitsu.com>
Cc: izumi.taku@jp.fujitsu.com, caoj.fnst@cn.fujitsu.com,
	Chen Fan <fan.chen@easystack.cn>,
	qemu-devel@nongnu.org, mst@redhat.com
Subject: Re: [Qemu-devel] [PATCH v8 11/12] vfio: register aer resume notification handler for aer resume
Date: Mon, 27 Jun 2016 21:58:08 -0600	[thread overview]
Message-ID: <20160627215808.1531a774@t450s.home> (raw)
In-Reply-To: <d52923dc-afb9-9dc1-ac0e-bae645141843@cn.fujitsu.com>

On Tue, 28 Jun 2016 11:26:33 +0800
Zhou Jie <zhoujie2011@cn.fujitsu.com> wrote:

> Hi Alex,
> 
> > The INTx/MSI part needs further definition for the user.  Are we
> > actually completely tearing down interrupts with the expectation that
> > the user will re-enable them or are we just masking them such that the
> > user needs to unmask?  Also note that not all devices support DisINTx.  
> 
> After reset, the "Bus Master Enable" bit of "Command Register"
> should be cleared, so MSI/MSI- X interrupt Messages is still disabled.
> After reset, the "Interrupt Disable" bit of "Command Register"
> should be cleared, so INTx interrupts is enabled.
> If the device doesn't support INTx, "Interrupt Disable" bit will
> hardware to 0, it is OK here.
> 
> After fatal-error occurs, the user should reset the device and
> reinitialize the device.
> So I disable the interrupt before host reset the device,
> and let user to do the reinitialization.

I'm dubious here.  When DisINTx is not supported by the device or it's
marked broken in host quirks, then we can't trust the device to stop
sending INTx.  It's hardwired to zero, meaning that it doesn't work or
it's been found to be broken in other ways.  So COMMAND register
masking is not sufficient for all devices.  Also, any time we start
changing the state of the device from what the user expects, we risk
consistency problems.  We need to consider how the user last saw the
device and whether we can legitimately expect them to handle the device
in a new state.  If we expect the user to re-initialize the device then
would it be more correct to teardown all interrupt signaling such that
the device is effectively in the same state as initial handoff when the
vfio device fd is opened?  How will the user know when the device is
ready to be reset?  Which of the ioctls that you're blocking can they
poll w/o any unwanted side-effects or awkward interactions?  Should
flag bits in the device info ioctl indicate not only support for this
behavior but also the current status?  Thanks,

Alex