All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH for 4.2 v5 00/55] Support for TCG plugins
@ 2019-10-14 10:48 Alex Bennée
  2019-10-14 10:48 ` [PATCH v5 01/55] trace: expand mem_info:size_shift to 4 bits Alex Bennée
                   ` (55 more replies)
  0 siblings, 56 replies; 94+ messages in thread
From: Alex Bennée @ 2019-10-14 10:48 UTC (permalink / raw)
  To: qemu-devel; +Cc: aaron, cota, Alex Bennée, robert.foley, peter.puhov

Hi,

This is the latest iteration of the TCG plugins series. From the documentation:

  QEMU TCG plugins provide a way for users to run experiments taking
  advantage of the total system control emulation can have over a guest.
  It provides a mechanism for plugins to subscribe to events during
  translation and execution and optionally callback into the plugin
  during these events. TCG plugins are unable to change the system state
  only monitor it passively. However they can do this down to an
  individual instruction granularity including potentially subscribing
  to all load and store operations.

I think I've addressed most of the comments from the v4 posting. I've
made a couple of changes to API which I intend to keep as separate
patches in the series to aid review. They are:

  qemu_plugin_install - new info block
  qemu_plugin_outs - output a string via QEMU's log system

The first allows plugins to know about the guest architecture - in a
fairly simple way by exposing TARGET_NAME. Its enough that a plugin
could decide not to install anything if it is not what it is expecting
but not so much that it has detailed information about the target. For
that sort of subtly you will have to pass details down via the plugin
options. I did briefly look at exposing ElfMachine information but
that got too gnarly. The only user in the examples is the howvec
plugin which will now run on all architectures although it can only
classify aarch64 and sparc64 instructions into groups.

The ability of the plugins to be fully "multiarch" aware meant I could
enable check-tcg to run most tests with each available plugin. This
proved very effective at exposing some bugs in the code that didn't
show up on my main test architecture (ARM). I can still trigger
crashes in sparc64-linux-user but rth informs me it is currently quite
broken so I skip it for now in the CI config.

The qemu_plugin_outs patch is mostly mechanical but it does neaten up
the output of plugins and prevent them just spamming stdout. Currently
the output goes via the logging system but I can envisage future
tweaks for system emulation which could push the output via a chardev
and allow nicer integration with test harnesses.

One requested feature that didn't make it in was a way to introspect
the state of the registers. It's not a deal breaker as the plugin can
always decode the instruction itself and has access to all
loads/stores so could track the data itself. I didn't want to hold up
the merging of what we have given softfreeze is so close so I think
this is something to think about for the next cycle. I'm open to ideas
of the best way to do this. One idle thought I had was giving the
plugin the ability to trigger a debug exception which would then punt
the problem to the gdbstub. I'm not sure if triggering a non-guest
visible exception violates the passive monitoring contract of TCG
plugins.

I've added some more words to the documentation to hopefully ally
fears about TCG plugins being used to end-run around the GPL.

The only other change of note is fixing a bunch of races by using
__thread for per-CPU data structures. There are more details bellow
the --- comments in each patch.

I'm hoping to get it merged this cycle so please review:

  02 - trace add mmu_index to mem_info
  17 - plugins implement helpers for resolving hwaddr
  40 - tests tcg Makefile.target fix path to config host
  41 - tests tcg set QEMU_OPTS for all cris runs
  42 - tests tcg move virtual tests to EXTRA_TESTS
  43 - tests tcg drop test i386 fprem from TESTS when no
  44 - tests tcg enable plugin testing
  45 - tests plugin add a hotblocks plugin
  47 - tests plugin add instruction execution breakdown
  48 - tests plugin add hotpages plugin to breakdown mem
  51 - plugins expand the plugin_init function to includ
  52 - plugins make howvec plugin more generic
  53 - plugins add sparc64 instruction classification ta
  54 - plugins add qemu_plugin_outs and use it
  55 - .travis.yml add enable plugins tests

Alex Bennée (21):
  trace: add mmu_index to mem_info
  docs/devel: add plugins.rst design document
  configure: add --enable-plugins (MOVE TO END)
  plugin: add implementation of the api
  plugins: implement helpers for resolving hwaddr
  tests/tcg/Makefile.target: fix path to config-host.mak
  tests/tcg: set QEMU_OPTS for all cris runs
  tests/tcg: move "virtual" tests to EXTRA_TESTS
  tests/tcg: drop test-i386-fprem from TESTS when not SLOW
  tests/tcg: enable plugin testing
  tests/plugin: add a hotblocks plugin
  plugin: add qemu_plugin_insn_disas helper
  tests/plugin: add instruction execution breakdown
  tests/plugin: add hotpages plugin to breakdown memory access patterns
  accel/stubs: reduce headers from tcg-stub
  include/exec: wrap cpu_ldst.h in CONFIG_TCG
  plugins: expand the plugin_init function to include an info block
  plugins: make howvec plugin more generic
  plugins: add sparc64 instruction classification table
  plugins: add qemu_plugin_outs and use it
  .travis.yml: add --enable-plugins tests

Emilio G. Cota (32):
  trace: expand mem_info:size_shift to 4 bits
  cpu: introduce cpu_in_exclusive_context()
  translate-all: use cpu_in_exclusive_work_context() in tb_flush
  plugin: add user-facing API
  plugin: add core code
  queue: add QTAILQ_REMOVE_SEVERAL
  cputlb: document get_page_addr_code
  cputlb: introduce get_page_addr_code_hostp
  tcg: add tcg_gen_st_ptr
  plugin-gen: add module for TCG-related code
  atomic_template: add inline trace/plugin helpers
  tcg: let plugins instrument virtual memory accesses
  translate-all: notify plugin code of tb_flush
  *-user: notify plugin of exit
  *-user: plugin syscalls
  cpu: hook plugin vcpu events
  plugin-gen: add plugin_insn_append
  translator: add translator_ld{ub,sw,uw,l,q}
  target/arm: fetch code with translator_ld
  target/ppc: fetch code with translator_ld
  target/sh4: fetch code with translator_ld
  target/i386: fetch code with translator_ld
  target/hppa: fetch code with translator_ld
  target/m68k: fetch code with translator_ld
  target/alpha: fetch code with translator_ld
  target/riscv: fetch code with translator_ld
  target/sparc: fetch code with translator_ld
  target/xtensa: fetch code with translator_ld
  target/openrisc: fetch code with translator_ld
  translator: inject instrumentation from plugins
  plugin: add API symbols to qemu-plugins.symbols
  tests/plugin: add sample plugins

Lluís Vilanova (2):
  vl: support -plugin option
  linux-user: support -plugin option

 .travis.yml                               |  15 +
 Makefile                                  |  16 +-
 Makefile.target                           |   2 +
 accel/stubs/tcg-stub.c                    |   1 -
 accel/tcg/Makefile.objs                   |   1 +
 accel/tcg/atomic_common.inc.c             |  54 ++
 accel/tcg/atomic_template.h               |  94 ++-
 accel/tcg/cpu-exec.c                      |   8 +-
 accel/tcg/cputlb.c                        |  53 +-
 accel/tcg/plugin-gen.c                    | 932 ++++++++++++++++++++++
 accel/tcg/plugin-helpers.h                |   5 +
 accel/tcg/translate-all.c                 |  15 +-
 accel/tcg/translator.c                    |  20 +
 accel/tcg/user-exec.c                     |   3 +
 bsd-user/syscall.c                        |  24 +-
 configure                                 |  86 ++
 cpus-common.c                             |   4 +
 cpus.c                                    |  10 +
 disas.c                                   | 110 +++
 docs/devel/index.rst                      |   1 +
 docs/devel/plugins.rst                    | 112 +++
 exec.c                                    |   2 +
 hw/core/cpu.c                             |   2 +
 include/disas/disas.h                     |   2 +
 include/exec/cpu-defs.h                   |   1 +
 include/exec/cpu_ldst.h                   |  11 +
 include/exec/cpu_ldst_template.h          |  37 +-
 include/exec/cpu_ldst_useronly_template.h |  29 +-
 include/exec/exec-all.h                   |  84 +-
 include/exec/helper-gen.h                 |   1 +
 include/exec/helper-proto.h               |   1 +
 include/exec/helper-tcg.h                 |   1 +
 include/exec/plugin-gen.h                 |  71 ++
 include/exec/translator.h                 |  58 +-
 include/hw/core/cpu.h                     |  19 +
 include/qemu/bswap.h                      |   5 +
 include/qemu/log.h                        |   1 +
 include/qemu/plugin.h                     | 261 ++++++
 include/qemu/qemu-plugin.h                | 388 +++++++++
 include/qemu/queue.h                      |  10 +
 include/user/syscall-trace.h              |  40 +
 linux-user/exit.c                         |   1 +
 linux-user/main.c                         |  18 +
 linux-user/syscall.c                      |   7 +-
 plugins/.gitignore                        |   2 +
 plugins/Makefile.objs                     |  21 +
 plugins/api.c                             | 334 ++++++++
 plugins/core.c                            | 504 ++++++++++++
 plugins/loader.c                          | 384 +++++++++
 plugins/plugin.h                          |  95 +++
 plugins/qemu-plugins.symbols              |  40 +
 qemu-options.hx                           |  17 +
 scripts/tracetool/transform.py            |   1 +
 target/alpha/translate.c                  |   2 +-
 target/arm/arm_ldst.h                     |  15 +-
 target/hppa/translate.c                   |   2 +-
 target/i386/translate.c                   |  10 +-
 target/m68k/translate.c                   |   2 +-
 target/openrisc/translate.c               |   2 +-
 target/ppc/translate.c                    |   8 +-
 target/riscv/translate.c                  |   2 +-
 target/sh4/translate.c                    |   4 +-
 target/sparc/translate.c                  |   2 +-
 target/xtensa/translate.c                 |   4 +-
 tcg/tcg-op.c                              |  40 +-
 tcg/tcg-op.h                              |  16 +
 tcg/tcg-opc.h                             |   3 +
 tcg/tcg.c                                 |  22 +
 tcg/tcg.h                                 |  23 +
 tests/Makefile.include                    |  11 +-
 tests/plugin/Makefile                     |  31 +
 tests/plugin/bb.c                         |  64 ++
 tests/plugin/empty.c                      |  30 +
 tests/plugin/hotblocks.c                  | 143 ++++
 tests/plugin/hotpages.c                   | 174 ++++
 tests/plugin/howvec.c                     | 351 ++++++++
 tests/plugin/insn.c                       |  61 ++
 tests/plugin/mem.c                        |  97 +++
 tests/tcg/Makefile.target                 |  43 +-
 tests/tcg/aarch64/Makefile.softmmu-target |   2 +-
 tests/tcg/aarch64/Makefile.target         |   6 +
 tests/tcg/arm/Makefile.softmmu-target     |   1 +
 tests/tcg/arm/Makefile.target             |   6 +
 tests/tcg/cris/Makefile.target            |   2 +-
 tests/tcg/i386/Makefile.target            |  11 +-
 trace-events                              |   8 +-
 trace/mem-internal.h                      |  39 +-
 trace/mem.h                               |   7 +-
 util/log.c                                |   3 +
 vl.c                                      |  11 +
 90 files changed, 5111 insertions(+), 161 deletions(-)
 create mode 100644 accel/tcg/atomic_common.inc.c
 create mode 100644 accel/tcg/plugin-gen.c
 create mode 100644 accel/tcg/plugin-helpers.h
 create mode 100644 docs/devel/plugins.rst
 create mode 100644 include/exec/plugin-gen.h
 create mode 100644 include/qemu/plugin.h
 create mode 100644 include/qemu/qemu-plugin.h
 create mode 100644 include/user/syscall-trace.h
 create mode 100644 plugins/.gitignore
 create mode 100644 plugins/Makefile.objs
 create mode 100644 plugins/api.c
 create mode 100644 plugins/core.c
 create mode 100644 plugins/loader.c
 create mode 100644 plugins/plugin.h
 create mode 100644 plugins/qemu-plugins.symbols
 create mode 100644 tests/plugin/Makefile
 create mode 100644 tests/plugin/bb.c
 create mode 100644 tests/plugin/empty.c
 create mode 100644 tests/plugin/hotblocks.c
 create mode 100644 tests/plugin/hotpages.c
 create mode 100644 tests/plugin/howvec.c
 create mode 100644 tests/plugin/insn.c
 create mode 100644 tests/plugin/mem.c

-- 
2.20.1



^ permalink raw reply	[flat|nested] 94+ messages in thread

end of thread, other threads:[~2019-10-18 18:33 UTC | newest]

Thread overview: 94+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-14 10:48 [PATCH for 4.2 v5 00/55] Support for TCG plugins Alex Bennée
2019-10-14 10:48 ` [PATCH v5 01/55] trace: expand mem_info:size_shift to 4 bits Alex Bennée
2019-10-14 14:35   ` Richard Henderson
2019-10-14 10:48 ` [PATCH v5 02/55] trace: add mmu_index to mem_info Alex Bennée
2019-10-14 14:53   ` Richard Henderson
2019-10-15 11:15     ` Alex Bennée
2019-10-14 10:48 ` [PATCH v5 03/55] cpu: introduce cpu_in_exclusive_context() Alex Bennée
2019-10-14 10:48 ` [PATCH v5 04/55] translate-all: use cpu_in_exclusive_work_context() in tb_flush Alex Bennée
2019-10-14 10:48 ` [PATCH v5 05/55] docs/devel: add plugins.rst design document Alex Bennée
2019-10-14 10:48 ` [PATCH v5 06/55] configure: add --enable-plugins (MOVE TO END) Alex Bennée
2019-10-14 10:49 ` [PATCH v5 07/55] plugin: add user-facing API Alex Bennée
2019-10-14 10:49 ` [PATCH v5 08/55] plugin: add core code Alex Bennée
2019-10-14 10:49 ` [PATCH v5 09/55] plugin: add implementation of the api Alex Bennée
2019-10-14 10:49 ` [PATCH v5 10/55] queue: add QTAILQ_REMOVE_SEVERAL Alex Bennée
2019-10-14 10:49 ` [PATCH v5 11/55] cputlb: document get_page_addr_code Alex Bennée
2019-10-14 10:49 ` [PATCH v5 12/55] cputlb: introduce get_page_addr_code_hostp Alex Bennée
2019-10-14 10:49 ` [PATCH v5 13/55] tcg: add tcg_gen_st_ptr Alex Bennée
2019-10-14 10:49 ` [PATCH v5 14/55] plugin-gen: add module for TCG-related code Alex Bennée
2019-10-14 15:23   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 15/55] atomic_template: add inline trace/plugin helpers Alex Bennée
2019-10-14 10:49 ` [PATCH v5 16/55] tcg: let plugins instrument virtual memory accesses Alex Bennée
2019-10-14 10:49 ` [PATCH v5 17/55] plugins: implement helpers for resolving hwaddr Alex Bennée
2019-10-14 15:45   ` Richard Henderson
2019-10-14 15:54   ` Peter Maydell
2019-10-14 16:34     ` Alex Bennée
2019-10-14 16:56       ` Peter Maydell
2019-10-14 10:49 ` [PATCH v5 18/55] translate-all: notify plugin code of tb_flush Alex Bennée
2019-10-14 10:49 ` [PATCH v5 19/55] *-user: notify plugin of exit Alex Bennée
2019-10-14 10:49 ` [PATCH v5 20/55] *-user: plugin syscalls Alex Bennée
2019-10-14 10:49 ` [PATCH v5 21/55] cpu: hook plugin vcpu events Alex Bennée
2019-10-14 10:49 ` [PATCH v5 22/55] plugin-gen: add plugin_insn_append Alex Bennée
2019-10-14 10:49 ` [PATCH v5 23/55] translator: add translator_ld{ub,sw,uw,l,q} Alex Bennée
2019-10-14 16:08   ` Richard Henderson
2019-10-14 17:31   ` Peter Maydell
2019-10-15 18:55     ` Alex Bennée
2019-10-15 21:34       ` Alex Bennée
2019-10-15 19:03     ` Alex Bennée
2019-10-14 10:49 ` [PATCH v5 24/55] target/arm: fetch code with translator_ld Alex Bennée
2019-10-14 10:49 ` [PATCH v5 25/55] target/ppc: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 26/55] target/sh4: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 27/55] target/i386: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 28/55] target/hppa: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 29/55] target/m68k: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 30/55] target/alpha: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 31/55] target/riscv: " Alex Bennée
2019-10-14 10:49   ` Alex Bennée
2019-10-14 17:59   ` Alistair Francis
2019-10-14 17:59     ` Alistair Francis
2019-10-18 18:32     ` Palmer Dabbelt
2019-10-18 18:32       ` Palmer Dabbelt
2019-10-14 10:49 ` [PATCH v5 32/55] target/sparc: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 33/55] target/xtensa: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 34/55] target/openrisc: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 35/55] translator: inject instrumentation from plugins Alex Bennée
2019-10-14 10:49 ` [PATCH v5 36/55] plugin: add API symbols to qemu-plugins.symbols Alex Bennée
2019-10-14 16:13   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 37/55] vl: support -plugin option Alex Bennée
2019-10-14 10:49 ` [PATCH v5 38/55] linux-user: " Alex Bennée
2019-10-14 10:49 ` [PATCH v5 39/55] tests/plugin: add sample plugins Alex Bennée
2019-10-14 16:14   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 40/55] tests/tcg/Makefile.target: fix path to config-host.mak Alex Bennée
2019-10-14 16:15   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 41/55] tests/tcg: set QEMU_OPTS for all cris runs Alex Bennée
2019-10-14 16:16   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 42/55] tests/tcg: move "virtual" tests to EXTRA_TESTS Alex Bennée
2019-10-14 16:16   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 43/55] tests/tcg: drop test-i386-fprem from TESTS when not SLOW Alex Bennée
2019-10-14 16:44   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 44/55] tests/tcg: enable plugin testing Alex Bennée
2019-10-14 16:46   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 45/55] tests/plugin: add a hotblocks plugin Alex Bennée
2019-10-14 16:49   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 46/55] plugin: add qemu_plugin_insn_disas helper Alex Bennée
2019-10-14 10:49 ` [PATCH v5 47/55] tests/plugin: add instruction execution breakdown Alex Bennée
2019-10-14 16:50   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 48/55] tests/plugin: add hotpages plugin to breakdown memory access patterns Alex Bennée
2019-10-14 16:51   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 49/55] accel/stubs: reduce headers from tcg-stub Alex Bennée
2019-10-14 10:49 ` [PATCH v5 50/55] include/exec: wrap cpu_ldst.h in CONFIG_TCG Alex Bennée
2019-10-14 10:49 ` [PATCH v5 51/55] plugins: expand the plugin_init function to include an info block Alex Bennée
2019-10-14 16:54   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 52/55] plugins: make howvec plugin more generic Alex Bennée
2019-10-14 16:59   ` Richard Henderson
2019-10-14 17:14     ` Alex Bennée
2019-10-14 17:39       ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 53/55] plugins: add sparc64 instruction classification table Alex Bennée
2019-10-14 17:01   ` Richard Henderson
2019-10-15 19:09     ` Alex Bennée
2019-10-15 19:37       ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 54/55] plugins: add qemu_plugin_outs and use it Alex Bennée
2019-10-14 17:03   ` Richard Henderson
2019-10-14 10:49 ` [PATCH v5 55/55] .travis.yml: add --enable-plugins tests Alex Bennée
2019-10-14 17:04   ` Richard Henderson
2019-10-15  4:36 ` [PATCH for 4.2 v5 00/55] Support for TCG plugins no-reply

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.