On Wed, Nov 13, 2019 at 11:14:50AM -0500, Jag Raman wrote:
> On 11/11/2019 11:21 AM, Stefan Hajnoczi wrote:
> > On Thu, Oct 24, 2019 at 05:09:13AM -0400, Jagannathan Raman wrote:
> > > Using a separate communication channel for MMIO helps
> > > with improving Performance
> > 
> > Why?
> 
> Typical initiation of IO operations involves multiple MMIO accesses per
> IO operation. In some legacy devices like LSI, the completion of the IO
> operations is also accomplished by polling on MMIO registers. Therefore,
> MMIO traffic can be hefty in some cases and contribute to Performance.
> 
> Having a dedicated channel for MMIO ensures that it doesn't have to
> compete with other messages to the remote process, especially when there
> are multiple devices emulated by a single remote process.

A vCPU doing a polling read on an MMIO register will cause a BAR_READ
message to be sent to the remote process.  The vCPU thread waits for the
response to this message.

When there are multiple remote devices each has its own socket, so
communication with different remote processes does not interfere.

The only scenarios I can think of are:
1. Interference within a single device between vCPUs and/or the QEMU
   monitor.
2. A single process serving multiple devices that is implemented in a
   way such that different devices interfere with each other.

It sounds like you are saying the problem is #2, but this is still
unclear to me.  If the remote process can be implemented in a way such
that there is no interference when each device has a special MMIO
socket, then why can't it be implemented in a way such that there is no
interference when each device's main socket is used (each device has
it's own!).

Maybe I've missed the point.  It would be good if you could explain in
more detail.

Stefan