Re: [PATCH V5 17/25] vfio-pci: cpr part 2

From: Alex Williamson <alex.williamson@redhat.com>
To: Steven Sistare <steven.sistare@oracle.com>
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	"Jason Zeng" <jason.zeng@linux.intel.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Juan Quintela" <quintela@redhat.com>,
	qemu-devel@nongnu.org, "Eric Blake" <eblake@redhat.com>,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	"Marc-André Lureau" <marcandre.lureau@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Markus Armbruster" <armbru@redhat.com>
Subject: Re: [PATCH V5 17/25] vfio-pci: cpr part 2
Date: Mon, 19 Jul 2021 12:10:21 -0600	[thread overview]
Message-ID: <20210719121021.6babb9ff.alex.williamson@redhat.com> (raw)
In-Reply-To: <be26577b-8df6-a8b5-fe06-ea33552aa472@oracle.com>

On Mon, 19 Jul 2021 13:44:08 -0400
Steven Sistare <steven.sistare@oracle.com> wrote:

> On 7/16/2021 4:51 PM, Alex Williamson wrote:
> > On Wed,  7 Jul 2021 10:20:26 -0700
> > Steve Sistare <steven.sistare@oracle.com> wrote:
> >   
> >> Finish cpr for vfio-pci by preserving eventfd's and vector state.
> >>
> >> Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
> >> ---
> >>  hw/vfio/pci.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> >>  1 file changed, 116 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
> >> index 0f5c542..07bd360 100644
> >> --- a/hw/vfio/pci.c
> >> +++ b/hw/vfio/pci.c  
> > ...  
> >> @@ -3295,14 +3329,91 @@ static void vfio_merge_config(VFIOPCIDevice  
> > *vdev)  
> >>      g_free(phys_config);
> >>  }
> >>  
> >> +static int vfio_pci_pre_save(void *opaque)
> >> +{
> >> +    VFIOPCIDevice *vdev = opaque;
> >> +    PCIDevice *pdev = &vdev->pdev;
> >> +    int i;
> >> +
> >> +    if (vfio_pci_read_config(pdev, PCI_INTERRUPT_PIN, 1)) {
> >> +        error_report("%s: cpr does not support vfio-pci INTX",
> >> +                     vdev->vbasedev.name);
> >> +    }  
> > 
> > You're not only not supporting INTx, but devices that support INTx, so
> > this only works on VFs.  Why?  Is this just out of scope or is there
> > something fundamentally difficult about it?
> > 
> > This makes me suspect there's a gap in INTx routing setup if it's more
> > than just another eventfd to store and setup.  If we hot-add a device
> > using INTx after cpr restart, are we going to find problems?  Thanks,  
> 
> It could be supported, but requires more code (several event fd's plus other state in VFIOINTx
> to save and restore) for a case that does not seem very useful (a directly assigned device that
> only supports INTx ?). 

It's not testing that the device *only* supports INTx, it's testing
that the device supports INTx _at_all_.  That effectively means this
excludes anything other than an SR-IOV VF.  There are plenty of valid
and useful cases of assigning PFs, most of which support INTx even if
we don't expect that's their primary operational mode.  Thanks,

Alex