From: Jagannathan Raman <jag.raman@oracle.com>
To: qemu-devel@nongnu.org
Cc: elena.ufimtseva@oracle.com, fam@euphon.net,
swapnil.ingle@nutanix.com, john.g.johnson@oracle.com,
kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com,
mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com,
felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com,
konrad.wilk@oracle.com, dgilbert@redhat.com,
liran.alon@oracle.com, stefanha@redhat.com,
thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com,
berrange@redhat.com, mreitz@redhat.com,
ross.lagerwall@citrix.com, marcandre.lureau@gmail.com,
pbonzini@redhat.com
Subject: [PATCH v5 00/50] Initial support for multi-process qemu
Date: Mon, 24 Feb 2020 15:54:51 -0500 [thread overview]
Message-ID: <cover.1582576372.git.jag.raman@oracle.com> (raw)
Hello
Started with the presentation in October 2017 made by Marc-Andre (Red Hat)
and Konrad Wilk (Oracle) [1], and continued by Jag's BoF at KVM Forum 2018,
the multi-process project is now available and presented in this patchset.
This first series enables the emulation of lsi53c895a in a separate process.
We posted the Proof Of Concept patches [2] before the BoF session in 2018.
Subsequently, we posted RFC v1 [3], RFC v2 [4], RFC v3 [5] and RFC v4 [6].
John & Elena presented the status of this project in KVM Forum 2019. We
appreciate the in-person and email feedback we received to improve this
patchset. We also received valuable feedback and direction on future
improvements from the bi-weekly KVM community conference. We have
incorporated all the feedback in the current version of the series, v5.
Following people contributed to this patchset:
John G Johnson <john.g.johnson@oracle.com>
Jagannathan Raman <jag.raman@oracle.com>
Elena Ufimtseva <elena.ufimtseva@oracle.com>
Kanth Ghatraju <kanth.ghatraju@oracle.com>
Konrad Wilk <konrad.wilk@oracle.com>
For full concept writeup about QEMU disaggregation, refer to
docs/devel/qemu-multiprocess.rst. Please refer to
docs/qemu-multiprocess.txt for usage information.
We are planning on making the following improvements in the future to the experimental
Qemu multi-process:
- Asynchronous communication channel;
- Performance improvements;
- Libvirt support;
- Enforcement of security policies and privileges control;
We welcome all your ideas, concerns, and questions for this patchset.
Thank you!
[1]: http://events17.linuxfoundation.org/sites/events/files/slides/KVM%20FORUM%20multi-process.pdf
[1]: https://www.youtube.com/watch?v=Kq1-coHh7lg
[2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html
[3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html
[4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html
[5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html
[6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html
Elena Ufimtseva (23):
multi-process: add a command line option for debug file
multi-process: introduce proxy object
mutli-process: build remote command line args
multi-process: configure remote side devices
multi-process: add qdev_proxy_add to create proxy devices
multi-process: remote: add setup_devices msg processing
multi-process: remote: use fd for socket from parent process
multi-process: remote: add create_done condition
multi-process: add processing of remote device command line
multi-process: refractor vl.c code
multi-process: add remote option
multi-process: add remote options parser
multi-process: add parse_cmdline in remote process
multi-process: send heartbeat messages to remote
multi-process: handle heartbeat messages in remote process
multi-process/mon: choose HMP commands based on target
multi-process/mig: Load VMSD in the proxy object
multi-process/mig: refactor runstate_check into common file
multi-process/mig: Synchronize runstate of remote process
multi-process/mig: Restore the VMSD in remote process
multi-process: Enable support for multiple devices in remote
multi-process: Validate incoming commands from Proxy
multi-process: add configure and usage information
Jagannathan Raman (26):
multi-process: memory: alloc RAM from file at offset
multi-process: Refactor machine_init and exit notifiers
multi-process: Add stub functions to facilate build of multi-process
multi-process: Add config option for multi-process QEMU
multi-process: build system for remote device process
multi-process: define mpqemu-link object
multi-process: add functions to synchronize proxy and remote endpoints
multi-process: setup PCI host bridge for remote device
multi-process: setup a machine object for remote device process
multi-process: setup memory manager for remote device
multi-process: remote process initialization
multi-process: PCI BAR read/write handling for proxy & remote
endpoints
multi-process: Synchronize remote memory
multi-process: create IOHUB object to handle irq
multi-process: Retrieve PCI info from remote process
multi-process: Introduce build flags to separate remote process code
multi-process: Use separate MMIO communication channel
multi-process: perform device reset in the remote process
multi-process/mon: stub functions to enable QMP module for remote
process
multi-process/mon: enable QMP module support in the remote process
multi-process/mon: Refactor monitor/chardev functions out of vl.c
multi-process/mon: Initialize QMP module for remote processes
multi-process: prevent duplicate memory initialization in remote
multi-process/mig: build migration module in the remote process
multi-process/mig: Enable VMSD save in the Proxy object
multi-process/mig: Send VMSD of remote to the Proxy object
John G Johnson (1):
multi-process: add the concept description to
docs/devel/qemu-multiprocess
Makefile | 2 +
Makefile.objs | 45 ++
Makefile.target | 105 +++-
accel/Makefile.objs | 2 +
accel/stubs/kvm-stub.c | 5 +
accel/stubs/tcg-stub.c | 107 ++++
backends/Makefile.objs | 2 +
block/Makefile.objs | 2 +
chardev/char.c | 14 +
configure | 15 +
docs/devel/index.rst | 1 +
docs/devel/qemu-multiprocess.rst | 1102 ++++++++++++++++++++++++++++++++++++++
docs/qemu-multiprocess.txt | 86 +++
exec.c | 31 +-
hmp-commands-info.hx | 10 +
hmp-commands.hx | 25 +-
hw/Makefile.objs | 7 +
hw/block/Makefile.objs | 2 +
hw/core/Makefile.objs | 19 +
hw/nvram/Makefile.objs | 2 +
hw/pci/Makefile.objs | 4 +
hw/proxy/memory-sync.c | 212 ++++++++
hw/proxy/qemu-proxy.c | 906 +++++++++++++++++++++++++++++++
hw/scsi/Makefile.objs | 2 +
include/chardev/char.h | 1 +
include/exec/address-spaces.h | 2 +
include/exec/ram_addr.h | 4 +-
include/hw/pci/pci_ids.h | 3 +
include/hw/proxy/memory-sync.h | 37 ++
include/hw/proxy/qemu-proxy.h | 107 ++++
include/hw/qdev-core.h | 2 +
include/io/mpqemu-link.h | 218 ++++++++
include/monitor/monitor.h | 2 +
include/monitor/qdev.h | 26 +
include/qemu-common.h | 8 +
include/qemu/log.h | 1 +
include/qemu/mmap-alloc.h | 3 +-
include/remote/iohub.h | 50 ++
include/remote/machine.h | 32 ++
include/remote/memory.h | 20 +
include/remote/pcihost.h | 45 ++
include/sysemu/runstate.h | 3 +
include/sysemu/sysemu.h | 2 +
io/Makefile.objs | 2 +
io/mpqemu-link.c | 410 ++++++++++++++
memory.c | 2 +-
migration/Makefile.objs | 13 +
migration/savevm.c | 63 +++
migration/savevm.h | 3 +
monitor/Makefile.objs | 4 +
monitor/misc.c | 84 +--
monitor/monitor-internal.h | 38 ++
monitor/monitor.c | 40 +-
net/Makefile.objs | 2 +
qapi/Makefile.objs | 2 +
qdev-monitor.c | 259 ++++++++-
qemu-options.hx | 21 +
qom/Makefile.objs | 4 +
remote/Makefile.objs | 6 +
remote/iohub.c | 148 +++++
remote/machine.c | 99 ++++
remote/memory.c | 63 +++
remote/pcihost.c | 64 +++
remote/remote-main.c | 625 +++++++++++++++++++++
remote/remote-opts.c | 115 ++++
remote/remote-opts.h | 15 +
rules.mak | 2 +-
runstate.c | 41 ++
scripts/hxtool | 44 +-
softmmu/vl.c | 299 ++++-------
stubs/audio.c | 12 +
stubs/gdbstub.c | 23 +
stubs/machine-init-done.c | 4 +
stubs/migration.c | 211 ++++++++
stubs/monitor.c | 72 +++
stubs/net-stub.c | 121 +++++
stubs/qapi-misc.c | 43 ++
stubs/qapi-target.c | 55 ++
stubs/replay.c | 26 +
stubs/runstate-check.c | 3 +
stubs/ui-stub.c | 130 +++++
stubs/vl-stub.c | 195 +++++++
stubs/vmstate.c | 19 +
stubs/xen-mapcache.c | 22 +
ui/Makefile.objs | 2 +
util/Makefile.objs | 2 +
util/log.c | 2 +
util/mmap-alloc.c | 7 +-
util/notify.c | 43 ++
util/oslib-posix.c | 2 +-
vl-parse.c | 164 ++++++
vl.h | 43 ++
92 files changed, 6698 insertions(+), 245 deletions(-)
create mode 100644 docs/devel/qemu-multiprocess.rst
create mode 100644 docs/qemu-multiprocess.txt
create mode 100644 hw/proxy/memory-sync.c
create mode 100644 hw/proxy/qemu-proxy.c
create mode 100644 include/hw/proxy/memory-sync.h
create mode 100644 include/hw/proxy/qemu-proxy.h
create mode 100644 include/io/mpqemu-link.h
create mode 100644 include/remote/iohub.h
create mode 100644 include/remote/machine.h
create mode 100644 include/remote/memory.h
create mode 100644 include/remote/pcihost.h
create mode 100644 io/mpqemu-link.c
create mode 100644 remote/Makefile.objs
create mode 100644 remote/iohub.c
create mode 100644 remote/machine.c
create mode 100644 remote/memory.c
create mode 100644 remote/pcihost.c
create mode 100644 remote/remote-main.c
create mode 100644 remote/remote-opts.c
create mode 100644 remote/remote-opts.h
create mode 100644 runstate.c
mode change 100644 => 100755 scripts/hxtool
create mode 100644 stubs/audio.c
create mode 100644 stubs/migration.c
create mode 100644 stubs/net-stub.c
create mode 100644 stubs/qapi-misc.c
create mode 100644 stubs/qapi-target.c
create mode 100644 stubs/ui-stub.c
create mode 100644 stubs/vl-stub.c
create mode 100644 stubs/xen-mapcache.c
create mode 100644 vl-parse.c
create mode 100644 vl.h
--
1.8.3.1
next reply other threads:[~2020-02-24 20:58 UTC|newest]
Thread overview: 117+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-24 20:54 Jagannathan Raman [this message]
2020-02-24 20:54 ` [PATCH v5 01/50] multi-process: memory: alloc RAM from file at offset Jagannathan Raman
2020-03-03 19:51 ` Dr. David Alan Gilbert
2020-03-04 18:24 ` Jag Raman
2020-02-24 20:54 ` [PATCH v5 02/50] multi-process: Refactor machine_init and exit notifiers Jagannathan Raman
2020-03-29 16:45 ` Marc-André Lureau
2020-02-24 20:54 ` [PATCH v5 03/50] multi-process: add a command line option for debug file Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 04/50] multi-process: Add stub functions to facilate build of multi-process Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 05/50] multi-process: Add config option for multi-process QEMU Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 06/50] multi-process: build system for remote device process Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 07/50] multi-process: define mpqemu-link object Jagannathan Raman
2020-03-10 16:09 ` Stefan Hajnoczi
2020-03-10 18:26 ` Elena Ufimtseva
2020-03-16 11:26 ` Stefan Hajnoczi
2020-02-24 20:54 ` [PATCH v5 08/50] multi-process: add functions to synchronize proxy and remote endpoints Jagannathan Raman
2020-03-03 19:56 ` Dr. David Alan Gilbert
2020-03-04 18:42 ` Jag Raman
2020-03-04 19:46 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 09/50] multi-process: setup PCI host bridge for remote device Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 10/50] multi-process: setup a machine object for remote device process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 11/50] multi-process: setup memory manager for remote device Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 12/50] multi-process: remote process initialization Jagannathan Raman
2020-03-04 10:29 ` Dr. David Alan Gilbert
2020-03-04 18:45 ` Jag Raman
2020-03-04 19:00 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 13/50] multi-process: introduce proxy object Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 14/50] mutli-process: build remote command line args Jagannathan Raman
2020-03-02 17:36 ` Philippe Mathieu-Daudé
2020-03-02 17:47 ` Daniel P. Berrangé
2020-03-02 22:39 ` Elena Ufimtseva
2020-03-04 11:00 ` Daniel P. Berrangé
2020-03-04 16:25 ` Elena Ufimtseva
2020-03-04 16:33 ` Daniel P. Berrangé
2020-03-04 22:37 ` Elena Ufimtseva
2020-03-05 8:21 ` Daniel P. Berrangé
2020-03-06 16:25 ` Stefan Hajnoczi
2020-03-04 10:09 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 15/50] multi-process: PCI BAR read/write handling for proxy & remote endpoints Jagannathan Raman
2020-03-04 10:47 ` Dr. David Alan Gilbert
2020-03-04 19:05 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 16/50] multi-process: Synchronize remote memory Jagannathan Raman
2020-03-04 11:53 ` Dr. David Alan Gilbert
2020-03-04 19:35 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 17/50] multi-process: create IOHUB object to handle irq Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 18/50] multi-process: configure remote side devices Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 19/50] multi-process: Retrieve PCI info from remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 20/50] multi-process: add qdev_proxy_add to create proxy devices Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 21/50] multi-process: remote: add setup_devices msg processing Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 22/50] multi-process: remote: use fd for socket from parent process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 23/50] multi-process: remote: add create_done condition Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 24/50] multi-process: add processing of remote device command line Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 25/50] multi-process: Introduce build flags to separate remote process code Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 26/50] multi-process: refractor vl.c code Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 27/50] multi-process: add remote option Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 28/50] multi-process: add remote options parser Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 29/50] multi-process: add parse_cmdline in remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 30/50] multi-process: send heartbeat messages to remote Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 31/50] multi-process: handle heartbeat messages in remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 32/50] multi-process: Use separate MMIO communication channel Jagannathan Raman
2020-03-06 16:52 ` Stefan Hajnoczi
2020-03-10 18:04 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 33/50] multi-process: perform device reset in the remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 34/50] multi-process/mon: choose HMP commands based on target Jagannathan Raman
2020-03-05 10:39 ` Dr. David Alan Gilbert
2020-03-05 15:38 ` Jag Raman
2020-03-05 15:50 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 35/50] multi-process/mon: stub functions to enable QMP module for remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 36/50] multi-process/mon: enable QMP module support in the " Jagannathan Raman
2020-03-05 10:43 ` Dr. David Alan Gilbert
2020-03-05 15:40 ` Jag Raman
2020-03-05 15:52 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 37/50] multi-process/mon: Refactor monitor/chardev functions out of vl.c Jagannathan Raman
2020-03-05 10:51 ` Dr. David Alan Gilbert
2020-03-05 15:41 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 38/50] multi-process/mon: Initialize QMP module for remote processes Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 39/50] multi-process: prevent duplicate memory initialization in remote Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 40/50] multi-process/mig: build migration module in the remote process Jagannathan Raman
2020-03-04 15:58 ` Dr. David Alan Gilbert
2020-03-04 19:45 ` Jag Raman
2020-03-04 19:52 ` Dr. David Alan Gilbert
2020-03-04 20:23 ` Jag Raman
2020-03-05 10:10 ` Dr. David Alan Gilbert
2020-03-05 17:07 ` Elena Ufimtseva
2020-03-05 17:19 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 41/50] multi-process/mig: Enable VMSD save in the Proxy object Jagannathan Raman
2020-03-05 12:31 ` Dr. David Alan Gilbert
2020-03-05 16:48 ` Jag Raman
2020-03-05 17:04 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 42/50] multi-process/mig: Send VMSD of remote to " Jagannathan Raman
2020-03-05 14:39 ` Dr. David Alan Gilbert
2020-03-05 16:32 ` Elena Ufimtseva
2020-02-24 20:55 ` [PATCH v5 43/50] multi-process/mig: Load VMSD in the proxy object Jagannathan Raman
2020-03-05 15:28 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 44/50] multi-process/mig: refactor runstate_check into common file Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 45/50] multi-process/mig: Synchronize runstate of remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 46/50] multi-process/mig: Restore the VMSD in " Jagannathan Raman
2020-03-05 16:05 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 47/50] multi-process: Enable support for multiple devices in remote Jagannathan Raman
2020-02-28 16:44 ` Stefan Hajnoczi
2020-03-02 19:28 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 48/50] multi-process: Validate incoming commands from Proxy Jagannathan Raman
2020-02-27 17:18 ` Stefan Hajnoczi
2020-02-28 17:53 ` Elena Ufimtseva
2020-02-24 20:55 ` [PATCH v5 49/50] multi-process: add the concept description to docs/devel/qemu-multiprocess Jagannathan Raman
2020-02-27 17:11 ` Stefan Hajnoczi
2020-02-28 18:44 ` Elena Ufimtseva
2020-02-24 20:55 ` [PATCH v5 50/50] multi-process: add configure and usage information Jagannathan Raman
2020-02-27 16:58 ` Stefan Hajnoczi
2020-02-28 18:43 ` Elena Ufimtseva
2020-03-06 16:42 ` Stefan Hajnoczi
2020-02-24 21:59 ` [PATCH v5 00/50] Initial support for multi-process qemu no-reply
2020-02-24 22:03 ` no-reply
2020-02-24 22:23 ` no-reply
2020-03-01 11:57 ` Alex Bennée
2020-03-02 15:28 ` Jag Raman
2020-03-02 16:29 ` Alex Bennée
2020-03-02 16:53 ` Jag Raman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1582576372.git.jag.raman@oracle.com \
--to=jag.raman@oracle.com \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=ehabkost@redhat.com \
--cc=elena.ufimtseva@oracle.com \
--cc=fam@euphon.net \
--cc=felipe@nutanix.com \
--cc=john.g.johnson@oracle.com \
--cc=kanth.ghatraju@oracle.com \
--cc=konrad.wilk@oracle.com \
--cc=kraxel@redhat.com \
--cc=kwolf@redhat.com \
--cc=liran.alon@oracle.com \
--cc=marcandre.lureau@gmail.com \
--cc=mreitz@redhat.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=ross.lagerwall@citrix.com \
--cc=rth@twiddle.net \
--cc=stefanha@redhat.com \
--cc=swapnil.ingle@nutanix.com \
--cc=thanos.makatos@nutanix.com \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.