From: elena.ufimtseva@oracle.com
To: qemu-devel@nongnu.org
Cc: elena.ufimtseva@oracle.com, fam@euphon.net,
swapnil.ingle@nutanix.com, john.g.johnson@oracle.com,
kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com,
mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com,
felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com,
konrad.wilk@oracle.com, dgilbert@redhat.com,
liran.alon@oracle.com, stefanha@redhat.com,
thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com,
berrange@redhat.com, mreitz@redhat.com,
ross.lagerwall@citrix.com, marcandre.lureau@gmail.com,
pbonzini@redhat.com
Subject: [PATCH v7 21/21] multi-process: add configure and usage information
Date: Sat, 27 Jun 2020 10:09:43 -0700 [thread overview]
Message-ID: <3624d64f8943aea7f9867c10159e15b42cac77ff.1593273671.git.elena.ufimtseva@oracle.com> (raw)
In-Reply-To: <cover.1593273671.git.elena.ufimtseva@oracle.com>
From: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
---
MAINTAINERS | 2 +
docs/multi-process.rst | 71 ++++++++++++++++++++++++++++
scripts/mpqemu-launcher-perf-mode.py | 67 ++++++++++++++++++++++++++
scripts/mpqemu-launcher.py | 47 ++++++++++++++++++
4 files changed, 187 insertions(+)
create mode 100644 docs/multi-process.rst
create mode 100644 scripts/mpqemu-launcher-perf-mode.py
create mode 100755 scripts/mpqemu-launcher.py
diff --git a/MAINTAINERS b/MAINTAINERS
index a65a5dd279..60c21d8210 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2959,6 +2959,8 @@ F: include/hw/pci/memory-sync.h
F: hw/remote/iohub.c
F: include/hw/remote/iohub.h
F: docs/devel/multi-process.rst
+F: scripts/mpqemu-launcher.py
+F: scripts/mpqemu-launcher-perf-mode.py
Build and test automation
-------------------------
diff --git a/docs/multi-process.rst b/docs/multi-process.rst
new file mode 100644
index 0000000000..0fa643a511
--- /dev/null
+++ b/docs/multi-process.rst
@@ -0,0 +1,71 @@
+Multi-process QEMU
+==================
+
+This document describes how to configure and use multi-process qemu.
+For the design document refer to docs/devel/qemu-multiprocess.
+
+1) Configuration
+----------------
+
+To enable support for multi-process add --enable-mpqemu
+to the list of options for the "configure" script.
+
+
+2) Usage
+--------
+
+Multi-process QEMU requires an orchestrator to launch. Please refer to a
+light-weight python based orchestrator for mpqemu in
+scripts/mpqemu-launcher.py to lauch QEMU in multi-process mode.
+
+Please refer to scripts/mpqemu-launcher-perf-mode.py for running the remote
+process in performance mode.
+
+As of now, we only support the emulation of lsi53c895a in a separate process.
+
+Following is a description of command-line used to launch mpqemu.
+
+* Orchestrator:
+
+ - The Orchestrator creates a unix socketpair
+
+ - It launches the remote process and passes one of the
+ sockets to it via command-line.
+
+ - It then launches QEMU and specifies the other socket as an option
+ to the Proxy device object
+
+* Remote Process:
+
+ - QEMU can enter remote process mode by using the "remote" machine
+ option. The socket suboption specifies the channel in which the
+ remote process received messages
+
+ - The remaining options are no different from how one launches QEMU with
+ devices. The only other requirement is each PCI device must have a
+ unique ID specified to it. This is needed to pair remote device with the
+ Proxy object.
+
+ - Example command-line for the remote process is as follows:
+
+ /usr/bin/qemu-system-x86_64 \
+ -machine remote,socket=4 \
+ -device lsi53c895a,id=lsi0 \
+ -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2 \
+ -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0
+
+* QEMU:
+
+ - Since parts of the RAM are shared between QEMU & remote process, a
+ memory-backend-memfd is required to facilitate this, as follows:
+
+ -object memory-backend-memfd,id=mem,size=2G
+
+ - A "pci-proxy-dev" device is created for each of the PCI devices emulated
+ in the remote process. A "socket" sub-option specifies the other end of
+ unix channel created by orchestrator. The "id" sub-option must be specified
+ and should be the same as the "id" specified for the remote PCI device
+
+ - Example commandline for QEMU is as follows:
+
+ -device pci-proxy-dev,id=lsi0,socket=3
diff --git a/scripts/mpqemu-launcher-perf-mode.py b/scripts/mpqemu-launcher-perf-mode.py
new file mode 100644
index 0000000000..583f65ca99
--- /dev/null
+++ b/scripts/mpqemu-launcher-perf-mode.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+import socket
+import os
+import subprocess
+import time
+
+PROC_QEMU='/usr/bin/qemu-system-x86_64'
+
+proxy_1, remote_1 = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
+proxy_2, remote_2 = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
+
+remote_cmd_1 = [ PROC_QEMU, \
+ '-machine', 'remote,socket='+str(remote_1.fileno()), \
+ '-device', 'lsi53c895a,id=lsi1', \
+ '-drive', 'id=drive_image1,file=/test-1.qcow2', \
+ '-device', 'scsi-hd,id=drive1,drive=drive_image1,bus=lsi1.0,' \
+ 'scsi-id=0', \
+ '-nographic', \
+ ]
+
+remote_cmd_2 = [ PROC_QEMU, \
+ '-machine', 'remote,socket='+str(remote_2.fileno()), \
+ '-device', 'lsi53c895a,id=lsi2', \
+ '-drive', 'id=drive_image2,file=/test-2.qcow2', \
+ '-device', 'scsi-hd,id=drive2,drive=drive_image2,bus=lsi2.0,' \
+ 'scsi-id=0', \
+ '-nographic', \
+ ]
+
+proxy_cmd = [ PROC_QEMU, \
+ '-name', 'OL7.4', \
+ '-machine', 'q35,accel=kvm', \
+ '-smp', 'sockets=1,cores=1,threads=1', \
+ '-m', '2048', \
+ '-object', 'memory-backend-memfd,id=sysmem-file,size=2G', \
+ '-numa', 'node,memdev=sysmem-file', \
+ '-device', 'virtio-scsi-pci,id=virtio_scsi_pci0', \
+ '-drive', 'id=drive_image1,if=none,format=qcow2,' \
+ 'file=/home/ol7-hdd-1.qcow2', \
+ '-device', 'scsi-hd,id=image1,drive=drive_image1,' \
+ 'bus=virtio_scsi_pci0.0', \
+ '-boot', 'd', \
+ '-vnc', ':0', \
+ '-device', 'pci-proxy-dev,id=lsi1,fd='+str(proxy_1.fileno()), \
+ '-device', 'pci-proxy-dev,id=lsi2,fd='+str(proxy_2.fileno()), \
+ ]
+
+
+pid = os.fork();
+if pid == 0:
+ # In remote_1
+ print('Launching Remote process 1');
+ process = subprocess.Popen(remote_cmd_1, pass_fds=[remote_1.fileno()])
+ os._exit(0)
+
+
+pid = os.fork();
+if pid == 0:
+ # In remote_2
+ print('Launching Remote process 2');
+ process = subprocess.Popen(remote_cmd_2, pass_fds=[remote_2.fileno()])
+ os._exit(0)
+
+
+print('Launching Proxy process');
+process = subprocess.Popen(proxy_cmd, pass_fds=[proxy_1.fileno(),
+ proxy_2.fileno()])
diff --git a/scripts/mpqemu-launcher.py b/scripts/mpqemu-launcher.py
new file mode 100755
index 0000000000..751b51eae7
--- /dev/null
+++ b/scripts/mpqemu-launcher.py
@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+import socket
+import os
+import subprocess
+import time
+
+PROC_QEMU='/usr/bin/qemu-system-x86_64'
+
+proxy, remote = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM)
+
+remote_cmd = [ PROC_QEMU, \
+ '-machine', 'remote,socket='+str(remote.fileno()), \
+ '-device', 'lsi53c895a,id=lsi1', \
+ '-drive', 'id=drive_image1,file=/build/ol7-nvme-test-1.qcow2', \
+ '-device', 'scsi-hd,id=drive1,drive=drive_image1,bus=lsi1.0,' \
+ 'scsi-id=0', \
+ '-nographic', \
+ ]
+
+proxy_cmd = [ PROC_QEMU, \
+ '-name', 'OL7.4', \
+ '-machine', 'q35,accel=kvm', \
+ '-smp', 'sockets=1,cores=1,threads=1', \
+ '-m', '2048', \
+ '-object', 'memory-backend-memfd,id=sysmem-file,size=2G', \
+ '-numa', 'node,memdev=sysmem-file', \
+ '-device', 'virtio-scsi-pci,id=virtio_scsi_pci0', \
+ '-drive', 'id=drive_image1,if=none,format=qcow2,' \
+ 'file=/home/ol7-hdd-1.qcow2', \
+ '-device', 'scsi-hd,id=image1,drive=drive_image1,' \
+ 'bus=virtio_scsi_pci0.0', \
+ '-boot', 'd', \
+ '-vnc', ':0', \
+ '-device', 'pci-proxy-dev,id=lsi1,fd='+str(proxy.fileno()), \
+ ]
+
+
+pid = os.fork();
+
+if pid:
+ # In Proxy
+ print('Launching QEMU with Proxy object');
+ process = subprocess.Popen(proxy_cmd, pass_fds=[proxy.fileno()])
+else:
+ # In remote
+ print('Launching Remote process');
+ process = subprocess.Popen(remote_cmd, pass_fds=[remote.fileno()])
--
2.25.GIT
next prev parent reply other threads:[~2020-06-27 17:18 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-27 17:09 [PATCH v7 00/21] Initial support for multi-process qemu elena.ufimtseva
2020-06-27 17:09 ` [PATCH v7 01/21] memory: alloc RAM from file at offset elena.ufimtseva
2020-06-30 14:59 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 02/21] multi-process: Add config option for multi-process QEMU elena.ufimtseva
2020-06-30 14:57 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 03/21] multi-process: setup PCI host bridge for remote device elena.ufimtseva
2020-06-30 15:17 ` Stefan Hajnoczi
2020-07-09 14:23 ` Jag Raman
2020-06-27 17:09 ` [PATCH v7 04/21] multi-process: setup a machine object for remote device process elena.ufimtseva
2020-06-30 15:26 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 05/21] multi-process: add qio channel function to transmit elena.ufimtseva
2020-06-30 15:29 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 06/21] multi-process: define MPQemuMsg format and transmission functions elena.ufimtseva
2020-06-30 15:53 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 07/21] multi-process: add co-routines to communicate with remote elena.ufimtseva
2020-06-30 18:31 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 08/21] multi-process: Initialize communication channel at the remote end elena.ufimtseva
2020-07-01 6:44 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 09/21] multi-process: Initialize message handler in remote device elena.ufimtseva
2020-07-01 6:53 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 10/21] multi-process: setup memory manager for " elena.ufimtseva
2020-07-01 7:58 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 11/21] multi-process: introduce proxy object elena.ufimtseva
2020-07-01 8:58 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 12/21] multi-process: Connect Proxy Object with device in the remote process elena.ufimtseva
2020-07-01 9:20 ` Stefan Hajnoczi
2020-07-24 16:57 ` Jag Raman
2020-07-27 13:18 ` Stefan Hajnoczi
2020-07-27 13:22 ` Michael S. Tsirkin
2020-07-31 18:31 ` Jag Raman
2020-06-27 17:09 ` [PATCH v7 13/21] multi-process: Forward PCI config space acceses to " elena.ufimtseva
2020-07-01 9:40 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 14/21] multi-process: PCI BAR read/write handling for proxy & remote endpoints elena.ufimtseva
2020-07-01 10:41 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 15/21] multi-process: Synchronize remote memory elena.ufimtseva
2020-07-01 10:55 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 16/21] multi-process: create IOHUB object to handle irq elena.ufimtseva
2020-07-02 12:09 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 17/21] multi-process: Retrieve PCI info from remote process elena.ufimtseva
2020-07-02 12:59 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 18/21] multi-process: heartbeat messages to remote elena.ufimtseva
2020-07-02 13:16 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 19/21] multi-process: perform device reset in the remote process elena.ufimtseva
2020-07-02 13:19 ` Stefan Hajnoczi
2020-06-27 17:09 ` [PATCH v7 20/21] multi-process: add the concept description to docs/devel/qemu-multiprocess elena.ufimtseva
2020-07-02 13:31 ` Stefan Hajnoczi
2020-06-27 17:09 ` elena.ufimtseva [this message]
2020-07-02 13:26 ` [PATCH v7 21/21] multi-process: add configure and usage information Stefan Hajnoczi
2020-07-02 13:40 ` [PATCH v7 00/21] Initial support for multi-process qemu Stefan Hajnoczi
2020-07-09 14:16 ` Jag Raman
2020-07-13 11:21 ` Stefan Hajnoczi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3624d64f8943aea7f9867c10159e15b42cac77ff.1593273671.git.elena.ufimtseva@oracle.com \
--to=elena.ufimtseva@oracle.com \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=ehabkost@redhat.com \
--cc=fam@euphon.net \
--cc=felipe@nutanix.com \
--cc=jag.raman@oracle.com \
--cc=john.g.johnson@oracle.com \
--cc=kanth.ghatraju@oracle.com \
--cc=konrad.wilk@oracle.com \
--cc=kraxel@redhat.com \
--cc=kwolf@redhat.com \
--cc=liran.alon@oracle.com \
--cc=marcandre.lureau@gmail.com \
--cc=mreitz@redhat.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=ross.lagerwall@citrix.com \
--cc=rth@twiddle.net \
--cc=stefanha@redhat.com \
--cc=swapnil.ingle@nutanix.com \
--cc=thanos.makatos@nutanix.com \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).