QEMU-Devel Archive on lore.kernel.org
 help / color / Atom feed
From: "Alex Bennée" <alex.bennee@linaro.org>
To: peter.maydell@linaro.org
Cc: "Richard Henderson" <richard.henderson@linaro.org>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	qemu-devel@nongnu.org
Subject: [PULL v2 21/73] docs/devel: add plugins.rst design document
Date: Fri, 25 Oct 2019 07:36:21 +0100
Message-ID: <20191025063713.23374-22-alex.bennee@linaro.org> (raw)
In-Reply-To: <20191025063713.23374-1-alex.bennee@linaro.org>

This is mostly extracted from Emilio's more verbose commit comments
with some additional verbiage from me.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index 1ec61fcfed9..2ff058bae38 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -22,3 +22,4 @@ Contents:
    decodetree
    secure-coding-practices
    tcg
+   plugins
diff --git a/docs/devel/plugins.rst b/docs/devel/plugins.rst
new file mode 100644
index 00000000000..b18fb6729e3
--- /dev/null
+++ b/docs/devel/plugins.rst
@@ -0,0 +1,112 @@
+..
+   Copyright (C) 2017, Emilio G. Cota <cota@braap.org>
+   Copyright (c) 2019, Linaro Limited
+   Written by Emilio Cota and Alex Bennée
+
+================
+QEMU TCG Plugins
+================
+
+QEMU TCG plugins provide a way for users to run experiments taking
+advantage of the total system control emulation can have over a guest.
+It provides a mechanism for plugins to subscribe to events during
+translation and execution and optionally callback into the plugin
+during these events. TCG plugins are unable to change the system state
+only monitor it passively. However they can do this down to an
+individual instruction granularity including potentially subscribing
+to all load and store operations.
+
+API Stability
+=============
+
+This is a new feature for QEMU and it does allow people to develop
+out-of-tree plugins that can be dynamically linked into a running QEMU
+process. However the project reserves the right to change or break the
+API should it need to do so. The best way to avoid this is to submit
+your plugin upstream so they can be updated if/when the API changes.
+
+
+Exposure of QEMU internals
+--------------------------
+
+The plugin architecture actively avoids leaking implementation details
+about how QEMU's translation works to the plugins. While there are
+conceptions such as translation time and translation blocks the
+details are opaque to plugins. The plugin is able to query select
+details of instructions and system configuration only through the
+exported *qemu_plugin* functions. The types used to describe
+instructions and events are opaque to the plugins themselves.
+
+Usage
+=====
+
+The QEMU binary needs to be compiled for plugin support:
+
+::
+    configure --enable-plugins
+
+Once built a program can be run with multiple plugins loaded each with
+their own arguments:
+
+::
+    $QEMU $OTHER_QEMU_ARGS \
+      -plugin tests/plugin/libhowvec.so,arg=inline,arg=hint \
+      -plugin tests/plugin/libhotblocks.so
+
+Arguments are plugin specific and can be used to modify their
+behaviour. In this case the howvec plugin is being asked to use inline
+ops to count and break down the hint instructions by type.
+
+Plugin Life cycle
+=================
+
+First the plugin is loaded and the public qemu_plugin_install function
+is called. The plugin will then register callbacks for various plugin
+events. Generally plugins will register a handler for the *atexit*
+if they want to dump a summary of collected information once the
+program/system has finished running.
+
+When a registered event occurs the plugin callback is invoked. The
+callbacks may provide additional information. In the case of a
+translation event the plugin has an option to enumerate the
+instructions in a block of instructions and optionally register
+callbacks to some or all instructions when they are executed.
+
+There is also a facility to add an inline event where code to
+increment a counter can be directly inlined with the translation.
+Currently only a simple increment is supported. This is not atomic so
+can miss counts. If you want absolute precision you should use a
+callback which can then ensure atomicity itself.
+
+Finally when QEMU exits all the registered *atexit* callbacks are
+invoked.
+
+Internals
+=========
+
+Locking
+-------
+
+We have to ensure we cannot deadlock, particularly under MTTCG. For
+this we acquire a lock when called from plugin code. We also keep the
+list of callbacks under RCU so that we do not have to hold the lock
+when calling the callbacks. This is also for performance, since some
+callbacks (e.g. memory access callbacks) might be called very
+frequently.
+
+  * A consequence of this is that we keep our own list of CPUs, so that
+    we do not have to worry about locking order wrt cpu_list_lock.
+  * Use a recursive lock, since we can get registration calls from
+    callbacks.
+
+As a result registering/unregistering callbacks is "slow", since it
+takes a lock. But this is very infrequent; we want performance when
+calling (or not calling) callbacks, not when registering them. Using
+RCU is great for this.
+
+We support the uninstallation of a plugin at any time (e.g. from
+plugin callbacks). This allows plugins to remove themselves if they no
+longer want to instrument the code. This operation is asynchronous
+which means callbacks may still occur after the uninstall operation is
+requested. The plugin isn't completely uninstalled until the safe work
+has executed while all vCPUs are quiescent.
-- 
2.20.1



  parent reply index

Thread overview: 84+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-25  6:36 [PULL v2 00/73] tcg plugins and testing updates Alex Bennée
2019-10-25  6:36 ` [PULL v2 01/73] travis.yml: reduce scope of the --enable-debug build Alex Bennée
2019-10-25  6:36 ` [PULL v2 02/73] travis.yml: Add libvdeplug-dev to compile-test net/vde.c Alex Bennée
2019-10-25  6:36 ` [PULL v2 03/73] travis.yml: Use libsdl2 instead of libsdl1.2, and install libsdl2-image Alex Bennée
2019-10-25  6:36 ` [PULL v2 04/73] travis.yml: Use newer version of libgnutls and libpng Alex Bennée
2019-10-25  6:36 ` [PULL v2 05/73] travis.yml: Fix the ccache lines Alex Bennée
2019-10-25  6:36 ` [PULL v2 06/73] travis.yml: Test the release tarball Alex Bennée
2019-10-25  6:36 ` [PULL v2 07/73] travis.yml: bump Xcode 10 to latest dot release Alex Bennée
2019-10-25  6:36 ` [PULL v2 08/73] cirrus.yml: add latest Xcode build target Alex Bennée
2019-10-25  6:36 ` [PULL v2 09/73] tests/vm: netbsd autoinstall, using serial console Alex Bennée
2019-10-25  6:36 ` [PULL v2 10/73] tests/vm: Let subclasses disable IPv6 Alex Bennée
2019-10-25  6:36 ` [PULL v2 11/73] tests/vm/netbsd: Disable IPv6 Alex Bennée
2019-10-25  6:36 ` [PULL v2 12/73] travis.yml: cache the clang sanitizer build Alex Bennée
2019-10-25  6:36 ` [PULL v2 13/73] gitlab-ci.yml: Use libvdeplug-dev to compile-test the VDE network backend Alex Bennée
2019-10-25  6:36 ` [PULL v2 14/73] travis.yml: --enable-debug-tcg to check-tcg Alex Bennée
2019-10-25  6:36 ` [PULL v2 15/73] tests/docker: set HOST_ARCH if we don't have ARCH Alex Bennée
2019-10-25  6:36 ` [PULL v2 16/73] tests/docker: update Travis image to a more current version Alex Bennée
2019-10-25  6:36 ` [PULL v2 17/73] trace: expand mem_info:size_shift to 4 bits Alex Bennée
2019-10-25  6:36 ` [PULL v2 18/73] trace: add mmu_index to mem_info Alex Bennée
2019-10-25  6:36 ` [PULL v2 19/73] cpu: introduce cpu_in_exclusive_context() Alex Bennée
2019-10-25  6:36 ` [PULL v2 20/73] translate-all: use cpu_in_exclusive_work_context() in tb_flush Alex Bennée
2019-10-25  6:36 ` Alex Bennée [this message]
2019-10-25  6:36 ` [PULL v2 22/73] plugin: add user-facing API Alex Bennée
2019-10-25  6:36 ` [PULL v2 23/73] plugin: add core code Alex Bennée
2019-10-25  6:36 ` [PULL v2 24/73] plugin: add implementation of the api Alex Bennée
2019-10-25  6:36 ` [PULL v2 25/73] queue: add QTAILQ_REMOVE_SEVERAL Alex Bennée
2019-10-25  6:36 ` [PULL v2 26/73] cputlb: document get_page_addr_code Alex Bennée
2019-10-25  6:36 ` [PULL v2 27/73] cputlb: introduce get_page_addr_code_hostp Alex Bennée
2019-10-25  6:36 ` [PULL v2 28/73] tcg: add tcg_gen_st_ptr Alex Bennée
2019-10-25  6:36 ` [PULL v2 29/73] plugin-gen: add module for TCG-related code Alex Bennée
2019-10-25  6:36 ` [PULL v2 30/73] atomic_template: add inline trace/plugin helpers Alex Bennée
2019-10-25  6:36 ` [PULL v2 31/73] tcg: let plugins instrument virtual memory accesses Alex Bennée
2019-10-25  6:36 ` [PULL v2 32/73] plugins: implement helpers for resolving hwaddr Alex Bennée
2019-10-25  6:36 ` [PULL v2 33/73] translate-all: notify plugin code of tb_flush Alex Bennée
2019-10-25  6:36 ` [PULL v2 34/73] *-user: notify plugin of exit Alex Bennée
2019-10-25  6:36 ` [PULL v2 35/73] *-user: plugin syscalls Alex Bennée
2019-10-25  6:36 ` [PULL v2 36/73] cpu: hook plugin vcpu events Alex Bennée
2019-10-25  6:36 ` [PULL v2 37/73] plugin-gen: add plugin_insn_append Alex Bennée
2019-10-25  6:36 ` [PULL v2 38/73] cputlb: ensure _cmmu helper functions follow the naming standard Alex Bennée
2019-10-25  6:36 ` [PULL v2 39/73] translator: add translator_ld{ub,sw,uw,l,q} Alex Bennée
2019-10-25  6:36 ` [PULL v2 40/73] target/arm: fetch code with translator_ld Alex Bennée
2019-10-25  6:36 ` [PULL v2 41/73] target/ppc: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 42/73] target/sh4: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 43/73] target/i386: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 44/73] target/hppa: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 45/73] target/m68k: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 46/73] target/alpha: " Alex Bennée
2019-10-25  7:40   ` USB-audio sound issues with qemu-system-ppc in Linux and Windows Howard Spoelstra
2019-10-25  6:36 ` [PULL v2 47/73] target/riscv: fetch code with translator_ld Alex Bennée
2019-10-25  6:36 ` [PULL v2 48/73] target/sparc: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 49/73] target/xtensa: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 50/73] target/openrisc: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 51/73] translator: inject instrumentation from plugins Alex Bennée
2019-10-25  6:36 ` [PULL v2 52/73] configure: add --enable-plugins Alex Bennée
2019-10-25  6:36 ` [PULL v2 53/73] plugin: add API symbols to qemu-plugins.symbols Alex Bennée
2019-10-25  6:36 ` [PULL v2 54/73] plugin: expand the plugin_init function to include an info block Alex Bennée
2019-10-25  6:36 ` [PULL v2 55/73] plugin: add qemu_plugin_insn_disas helper Alex Bennée
2019-10-25  6:36 ` [PULL v2 56/73] plugin: add qemu_plugin_outs helper Alex Bennée
2019-10-25  6:36 ` [PULL v2 57/73] vl: support -plugin option Alex Bennée
2019-10-25  6:36 ` [PULL v2 58/73] linux-user: " Alex Bennée
2019-10-25  6:36 ` [PULL v2 59/73] tests/plugin: add sample plugins Alex Bennée
2019-10-25  6:37 ` [PULL v2 60/73] tests/tcg/Makefile.target: fix path to config-host.mak Alex Bennée
2019-10-25  6:37 ` [PULL v2 61/73] tests/tcg: set QEMU_OPTS for all cris runs Alex Bennée
2019-10-25  6:37 ` [PULL v2 62/73] tests/tcg: move "virtual" tests to EXTRA_TESTS Alex Bennée
2019-10-25  6:37 ` [PULL v2 63/73] tests/tcg: drop test-i386-fprem from TESTS when not SLOW Alex Bennée
2019-10-25  6:37 ` [PULL v2 64/73] tests/tcg: enable plugin testing Alex Bennée
2019-10-25  6:37 ` [PULL v2 65/73] tests/plugin: add a hotblocks plugin Alex Bennée
2019-10-25  6:37 ` [PULL v2 66/73] tests/plugin: add instruction execution breakdown Alex Bennée
2019-10-25  6:37 ` [PULL v2 67/73] tests/plugin: add hotpages to analyse memory access patterns Alex Bennée
2019-10-25  6:37 ` [PULL v2 68/73] accel/stubs: reduce headers from tcg-stub Alex Bennée
2019-10-25  6:37 ` [PULL v2 69/73] include/exec: wrap cpu_ldst.h in CONFIG_TCG Alex Bennée
2019-10-25  6:37 ` [PULL v2 70/73] .travis.yml: add --enable-plugins tests Alex Bennée
2019-10-25  6:37 ` [PULL v2 71/73] scripts/checkpatch.pl: don't complain about (foo, /* empty */) Alex Bennée
2019-10-25  6:37 ` [PULL v2 72/73] MAINTAINERS: add me for the TCG plugins code Alex Bennée
2019-10-25  6:37 ` [PULL v2 73/73] travis.yml: enable linux-gcc-debug-tcg cache Alex Bennée
2019-10-25 12:59 ` [PULL v2 00/73] tcg plugins and testing updates Markus Armbruster
2019-10-25 15:04   ` Alex Bennée
2019-10-25 20:23     ` Markus Armbruster
2019-10-27 19:44       ` Peter Maydell
2019-10-28  9:07         ` Alex Bennée
2019-11-06 12:42       ` Markus Armbruster
2019-11-08 17:23         ` Peter Maydell
2019-10-25 16:53 ` Peter Maydell
2019-10-25 19:38   ` Alex Bennée

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191025063713.23374-22-alex.bennee@linaro.org \
    --to=alex.bennee@linaro.org \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=richard.henderson@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

QEMU-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/qemu-devel/0 qemu-devel/git/0.git
	git clone --mirror https://lore.kernel.org/qemu-devel/1 qemu-devel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 qemu-devel qemu-devel/ https://lore.kernel.org/qemu-devel \
		qemu-devel@nongnu.org
	public-inbox-index qemu-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.nongnu.qemu-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git