On Fri, Jul 24, 2020 at 12:57:22PM -0400, Jag Raman wrote:
> > On Jul 1, 2020, at 5:20 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > On Sat, Jun 27, 2020 at 10:09:34AM -0700, elena.ufimtseva@oracle.com wrote:
> > In a 1:1 model the CONNECT_DEV message is not necessary because each
> > socket is already associated with a specific remote device (e.g. qemu -M
> > remote -object mplink,dev=lsi-scsi-1,sockpath=/tmp/lsi-scsi-1.sock).
> > Connecting to the socket already indicates which device we are talking
> > to.
> > 
> > The N:1 model will work but it's more complex. There is a main socket
> > that is used for CONNECT_DEV (anything else?) and we need to worry about
> > the lifecycle of the per-device sockets that are passed over the main
> > socket.
> 
> The main socket is only used for CONNECT_DEV. The CONNECT_DEV
> message sticks a fd to the remote device.
> 
> We are using the following command-line in the remote process:
> qemu-system-x86_64 -machine remote,fd=4 -device lsi53c895a,id=lsi1 ...
> 
> The alternative approach would be to let the orchestrator to assign fds for
> each remote device. In this approach, we would specify an ‘fd’ for each
> device object like below:
> qemu-system-x86_64 -machine remote -device lsi53c895a,id=lsi1,fd=4 …
> 
> The alternative approach would entail changes to the DeviceState struct
> and qdev_device_add() function, which we thought is not preferable. Could
> you please share your thoughts on this?

I suggest dropping multi-device support for now. It will be implemented
differently with VFIO-over-socket anyway, so it's not worth investing
much time into.

The main socket approach needs authentication support if multiple guests
share a remote device emulation process. Otherwise guest A can access
guest B's devices.

It's simpler if each device has a separate UNIX domain socket. It is not
necessary to modify lsi53c895a in order to do this. Either the socket
can be associated with the remote PCIe port (although I think the
current code implements the older PCI Local Bus instead of PCIe) or a
separate -object mpqlink,device=lsi1,fd=4 object can be defined (I think
that's the syntax I've shared in the past).

For now though, just using the -machine remote,fd=4 approach is fine -
but limited to 1 device.

Stefan