On Mon, Feb 24, 2020 at 03:55:41PM -0500, Jagannathan Raman wrote:
> From: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> 
> Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
> Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
> ---
>  docs/qemu-multiprocess.txt | 86 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 86 insertions(+)
>  create mode 100644 docs/qemu-multiprocess.txt
> 
> diff --git a/docs/qemu-multiprocess.txt b/docs/qemu-multiprocess.txt
> new file mode 100644
> index 0000000..f156177
> --- /dev/null
> +++ b/docs/qemu-multiprocess.txt
> @@ -0,0 +1,86 @@
> +Multi-process QEMU
> +==================
> +
> +This document describes how to configure and use multi-process qemu.
> +For the design document refer to docs/devel/qemu-multiprocess.
> +
> +1) Configuration
> +----------------
> +
> +To enable support for multi-process add --enable-mpqemu
> +to the list of options for the "configure" script.
> +
> +
> +2) Usage
> +--------
> +
> +To start qemu with devices intended to run in a separate emulation
> +process without libvirtd support, the following should be used on QEMU
> +command line. As of now, we only support the emulation of lsi53c895a
> +in a separate process
> +
> +* Since parts of the RAM are shared between QEMU & remote process, a
> +  memory-backend-file is required to facilitate this, as follows:
> +
> +  -object memory-backend-file,id=mem,mem-path=/dev/shm/,size=4096M,share=on

memory-backend-memfd is more convenient.  It doesn't require a mem-path
and share=on is the default.

> +
> +* The devices to be emulated in the separate process are defined as
> +  before with addition of "rid" suboption that serves as a remote group
> +  identificator.
> +
> +  -device <device options>,rid="remote process id"
> +
> +  For example, for non multi-process qemu:
> +    -device lsi53c895a,id=scsi0 device
> +    -device scsi-hd,drive=drive0,bus=scsi0.0,scsi-id=0
> +    -drive id=drive0,file=data-disk.img
> +
> +  and for multi-process qemu and no libvirt
> +  support (i.e. QEMU forks child processes):
> +    -device lsi53c895a,id=scsi0,rid=0
> +    -device scsi-hd,drive=drive0,bus=scsi0.0,scsi-id=0,rid=0

This approach is invasive:
 * lsi53c895a should not need to be modified with a new rid= option.
 * QEMU should not know about the scsi-hd device or drive0.  Only the
   device emulation process needs to know about scsi-hd.

In order to cleanly separate QEMU and the device emulation process
syntax like this is needed:

  -object remote-device,id=rid0,...
  -device remote-pci-device,id=scsi0,remote-device=rid0

The "remote-device" object could be part of remote-pci-device, but
keeping it separate may be useful in the future in order to support
things like reconnection.

The generic "remote-pci-device" device handles any remote PCI device,
not just the LSI SCSI controller.

Do you agree with this approach?

> +* The command-line options for the remote process are added to the "command"
> +  suboption of the newly added "-remote" option. 
> +
> +   -remote [socket],rid=0,exec="...",command="..."

QEMU has been using the -object TYPE syntax instead of adding new -TYPE
command-line options.  This gives you object_add hotplug for free, for
example.  I suggest using -object remote-device,id=,exec=,command=,
instead of -remote.

> +
> +  The drives to be emulated by the remote process are specified as part of
> +  this command sub-option. The device to be used to connect to the monitor
> +  is also specified as part of this suboption.
> +
> +  For example, the following option adds a drive and monitor to the remote
> +  process:
> +  -remote rid=0,exec="qemu-scsi-dev",command="-drive id=drive0,,file=data-disk.img -monitor unix:/home/qmp-sock,,server,,nowait"
> +
> +  Note: There's an issue with this "command" sub-option which we are in the
> +  process of fixing. To work around this issue, it requires additional
> +  "comma" characters as illustrated above, and in the example below.

command= (which could be called args= for clarity) will be difficult to
use not just because of comma escaping but also because of double-quote
escaping.  How do you pass a command-line argument that contains spaces?