[RFC PATCH 00/12] Introduce QEMU userspace ebpf support

[RFC PATCH 00/12] Introduce QEMU userspace ebpf support

[Zhang Chen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 5490 bytes --]

Hi All,

    The goal of this series is to bring the power of ebpf to QEMU.
It makes QEMU have the ability to extend the capabilities without
requiring changing source code. Just need to load the eBPF binary
file even at VM runtime. And already have some userspace ebpf
implementation like: Intel DPDK eBPF, windows eBPF, etc..
The original idea suggested by Jason Wang.

    eBPF is a revolutionary technology with origins in the Linux kernel
that can run sandboxed programs in an operating system kernel. It is
used to safely and efficiently extend the capabilities of the kernel
without requiring to change kernel source code or load kernel
modules.(from https://ebpf.io/)

    KVM already got benefits from it, but QEMU did not. Hence we want
to bring the power of eBPF to QEMU. It can load binary eBPF program
even when VM running. At the same time, add some hooks in QEMU as
the user space eBPF load point. Do the things on different layers.

   That’s the advantages of kernel eBPF. Most of the functions can be
implemented in QEMU. This series just a start of the Power of Programmability.

    1). Safety:

    Building on the foundation of seeing and understanding all system
    calls and combining that with a packet and socket-level view of all
    networking operations allows for revolutionary new approaches to
    securing systems.

    2). Tracing & Profiling:

    The ability to attach eBPF programs to trace points as well as kernel
    and user application probe points allows unprecedented visibility into
    the runtime behavior of applications and the system itself.

    3). Networking:

    The combination of programmability and efficiency makes eBPF a natural
    fit for all packet processing requirements of networking solutions.

    4). Observability & Monitoring:

    Instead of relying on static counters and gauges exposed by the
    perating system, eBPF enables the collection & in-kernel aggregation
    of custom metrics and generation of visibility events based on a wide
    range of possible sources.

    QEMU userspace ebpf design based on ubpf project (https://github.com/iovisor/ubpf).
The most mature userspace ebpf implementation. This project officially
support by iovisor(Like BCC and bpftrace). This project includes an eBPF
assembler, disassembler, interpreter (for all platforms), and JIT compiler
(for x86-64 and Arm64 targets). Qemu userspace ebpf make the ubpf project
as the git submodule.

    Current implementation support load ebpf program and run it in
net/filter-ubpf module, this filter can support any user defined rules
to hanle network packet. At the same time, it's easy for other developers
to use the ubpf infrastructue in QEMU's other modules from the function
in /ebpf/ubpf.c, and it support JIT.

    For the uBPF License is Apache License 2.0, It's OK to compatible
with QEMU’s GPLv2 LICENSE same as mason.

    TODO: Need to add more comments and test-case for ubpf, current
implementation not include ebpf verifier. But I think maybe it's not
a big problem, current ebpf load/unload API exposed by QMP command.
Qemu is a userspace program, if someone want to hack QEMU, no need to
load a malicious ubpf program, it can hack QEMU code or crash QEMU on
host directly(different from kernel ebpf needs strict inspection, but
yes, it still need basic check).

Any comments are welcome.

Thanks
Chen


Zhang Chen (12):
  configure: Add iovisor/ubpf project as a submodule for QEMU
  meson: Add ubpf build config and misc
  ebpf/uBPF: Introduce userspace ebpf data structure
  ebpf/uBPF: Introduce ubpf initialize functions
  ebpf/uBPF: Add qemu_prepare_ubpf to load ebpf binary
  ebpf/uBPF: Add qemu_ubpf_run_once excute real ebpf program
  net/filter: Introduce filter-ubpf module
  qapi: Add FilterUbpfProperties and qemu-options
  softmmu/vl.c: Add filter-ubpf for netdev as other netfilters
  net/filter-ubpf.c: run the ubpf program to handle network packet
  docs/devel: Add userspace-ebpf.rst
  test/qtest: Add ubpf basic test case

 .gitmodules                         |   3 +
 configure                           |  20 +++
 docs/devel/userspace-ebpf.rst       | 106 ++++++++++++++
 ebpf/meson.build                    |   1 +
 ebpf/ubpf-stub.c                    |  35 +++++
 ebpf/ubpf.c                         | 217 ++++++++++++++++++++++++++++
 ebpf/ubpf.h                         |  44 ++++++
 meson.build                         |  47 ++++++
 meson_options.txt                   |   3 +
 net/filter-ubpf.c                   | 185 ++++++++++++++++++++++++
 net/meson.build                     |   1 +
 qapi/qom.json                       |  18 +++
 qemu-options.hx                     |   6 +
 scripts/coverity-scan/COMPONENTS.md |   3 +
 scripts/meson-buildoptions.sh       |   5 +
 softmmu/vl.c                        |   3 +-
 tests/qtest/demo_ubpf.o             | Bin 0 -> 544 bytes
 tests/qtest/integer_5.mem           | Bin 0 -> 4 bytes
 tests/qtest/meson.build             |   3 +-
 tests/qtest/ubpf-test.c             |  64 ++++++++
 ubpf                                |   1 +
 21 files changed, 763 insertions(+), 2 deletions(-)
 create mode 100644 docs/devel/userspace-ebpf.rst
 create mode 100644 ebpf/ubpf-stub.c
 create mode 100644 ebpf/ubpf.c
 create mode 100644 ebpf/ubpf.h
 create mode 100644 net/filter-ubpf.c
 create mode 100644 tests/qtest/demo_ubpf.o
 create mode 100644 tests/qtest/integer_5.mem
 create mode 100644 tests/qtest/ubpf-test.c
 create mode 160000 ubpf

-- 
2.25.1

[RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Zhang Chen]

Make iovisor/ubpf project be a git submodule for QEMU.
It will auto clone ubpf project when configure QEMU.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 .gitmodules |  3 +++
 configure   | 20 ++++++++++++++++++++
 ubpf        |  1 +
 3 files changed, 24 insertions(+)
 create mode 160000 ubpf

diff --git a/.gitmodules b/.gitmodules
index b8bff47df8..30fb082f39 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -64,3 +64,6 @@
 [submodule "tests/lcitool/libvirt-ci"]
 	path = tests/lcitool/libvirt-ci
 	url = https://gitlab.com/libvirt/libvirt-ci.git
+[submodule "ubpf"]
+	path = ubpf
+	url = https://github.com/iovisor/ubpf.git
diff --git a/configure b/configure
index e69537c756..7dde1429dc 100755
--- a/configure
+++ b/configure
@@ -326,6 +326,7 @@ else
   slirp="auto"
 fi
 fdt="auto"
+ubpf="auto"
 
 # 2. Automatically enable/disable other options
 tcg="enabled"
@@ -820,6 +821,14 @@ for opt do
   ;;
   --enable-slirp=*) slirp="$optarg"
   ;;
+  --disable-ubpf) ubpf="disabled"
+  ;;
+  --enable-ubpf) ubpf="enabled"
+  ;;
+  --enable-ubpf=git) ubpf="internal"
+  ;;
+  --enable-ubpf=*) ubpf="$optarg"
+  ;;
   --disable-tcg) tcg="disabled"
                  plugins="no"
   ;;
@@ -2176,6 +2185,16 @@ if test "$have_ubsan" = "yes"; then
   QEMU_LDFLAGS="-fsanitize=undefined $QEMU_LDFLAGS"
 fi
 
+##########################################
+# check for ubpf
+
+case "$ubpf" in
+  auto | enabled | internal)
+    # Simpler to always update submodule, even if not needed.
+    git_submodules="${git_submodules} ubpf"
+    ;;
+esac
+
 ##########################################
 
 # Exclude --warn-common with TSan to suppress warnings from the TSan libraries.
@@ -2664,6 +2683,7 @@ if test "$skip_meson" = no; then
   # QEMU options
   test "$cfi" != false && meson_option_add "-Dcfi=$cfi"
   test "$fdt" != auto && meson_option_add "-Dfdt=$fdt"
+  test "$ubpf" != auto && meson_option_add "-Dubpf=$ubpf"
   test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
   test "$qemu_suffix" != qemu && meson_option_add "-Dqemu_suffix=$qemu_suffix"
   test "$slirp" != auto && meson_option_add "-Dslirp=$slirp"
diff --git a/ubpf b/ubpf
new file mode 160000
index 0000000000..0dd334daf4
--- /dev/null
+++ b/ubpf
@@ -0,0 +1 @@
+Subproject commit 0dd334daf4849137fa40d2b7676d2bf920d5c81d
-- 
2.25.1

[RFC PATCH 02/12] meson: Add ubpf build config and misc

[Zhang Chen]

Make meson to build iovisor/ubpf code in Qemu.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 meson.build                         | 47 +++++++++++++++++++++++++++++
 meson_options.txt                   |  3 ++
 scripts/coverity-scan/COMPONENTS.md |  3 ++
 scripts/meson-buildoptions.sh       |  5 +++
 4 files changed, 58 insertions(+)

diff --git a/meson.build b/meson.build
index 21cd949082..f370c1aba7 100644
--- a/meson.build
+++ b/meson.build
@@ -2717,9 +2717,53 @@ if not fdt.found() and fdt_required.length() > 0
   error('fdt not available but required by targets ' + ', '.join(fdt_required))
 endif
 
+ubpf = not_found
+ubpf_opt = 'disabled'
+if have_system
+  ubpf_opt = get_option('ubpf')
+  if ubpf_opt in ['enabled', 'auto', 'system']
+    have_internal = fs.exists(meson.current_source_dir() / 'ubpf/vm/Makefile')
+    ubpf = dependency('ubpf', kwargs: static_kwargs,
+                      method: 'pkg-config',
+                      required: ubpf_opt == 'system' or
+                      ubpf_opt == 'enabled' and not have_internal)
+    if ubpf.found()
+      ubpf_opt = 'system'
+    elif have_internal
+      ubpf_opt = 'internal'
+    else
+      ubpf_opt = 'disabled'
+    endif
+  endif
+  if ubpf_opt == 'internal'
+    ubpf_data = configuration_data()
+
+    ubpf_files = files(
+      'ubpf/vm/ubpf_jit_x86_64.c',
+      'ubpf/vm/ubpf_vm.c',
+      'ubpf/vm/ubpf_loader.c',
+    )
+
+    ubpf_cargs = [
+      '-Wno-error', '-w',
+      '-include', 'ubpf-defs.h'
+    ]
+
+    configure_file(output: 'ubpf-defs.h', configuration: ubpf_data)
+    ubpf_inc = include_directories('ubpf/vm', 'ubpf/vm/inc')
+    libubpf = static_library('ubpf',
+                             sources: ubpf_files,
+                             c_args: ubpf_cargs,
+                             include_directories: ubpf_inc)
+    ubpf = declare_dependency(link_with: libubpf,
+                              include_directories: ubpf_inc)
+  endif
+endif
+
 config_host_data.set('CONFIG_CAPSTONE', capstone.found())
 config_host_data.set('CONFIG_FDT', fdt.found())
 config_host_data.set('CONFIG_SLIRP', slirp.found())
+config_host_data.set('CONFIG_UBPF', ubpf.found())
 
 #####################
 # Generated sources #
@@ -3046,6 +3090,8 @@ subdir('softmmu')
 common_ss.add(capstone)
 specific_ss.add(files('cpu.c', 'disas.c', 'gdbstub.c'), capstone)
 
+common_ss.add(ubpf)
+
 # Work around a gcc bug/misfeature wherein constant propagation looks
 # through an alias:
 #   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99696
@@ -3911,6 +3957,7 @@ summary_info += {'libudev':           libudev}
 # Dummy dependency, keep .found()
 summary_info += {'FUSE lseek':        fuse_lseek.found()}
 summary_info += {'selinux':           selinux}
+summary_info += {'ubpf support':      ubpf_opt == 'internal' ? ubpf_opt : ubpf}
 summary(summary_info, bool_yn: true, section: 'Dependencies')
 
 if not supported_cpus.contains(cpu)
diff --git a/meson_options.txt b/meson_options.txt
index 2de94af037..1eb9164857 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -262,6 +262,9 @@ option('slirp', type: 'combo', value: 'auto',
 option('fdt', type: 'combo', value: 'auto',
        choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
        description: 'Whether and how to find the libfdt library')
+option('ubpf', type: 'combo', value: 'auto',
+       choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
+       description: 'Whether and how to find the ubpf library')
 
 option('selinux', type: 'feature', value: 'auto',
        description: 'SELinux support in qemu-nbd')
diff --git a/scripts/coverity-scan/COMPONENTS.md b/scripts/coverity-scan/COMPONENTS.md
index 183f26a32c..dd28116674 100644
--- a/scripts/coverity-scan/COMPONENTS.md
+++ b/scripts/coverity-scan/COMPONENTS.md
@@ -72,6 +72,9 @@ char
 capstone
   ~ (/qemu)?(/capstone/.*)
 
+ubpf
+  ~ (/qemu)?(/ubpf/vm/.*)
+
 crypto
   ~ (/qemu)?((/include)?/crypto/.*|/hw/.*/crypto.*)
 
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 00ea4d8cd1..044dde1cff 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -37,6 +37,8 @@ meson_options_help() {
   printf "%s\n" '                           getrandom()'
   printf "%s\n" '  --enable-slirp[=CHOICE]  Whether and how to find the slirp library'
   printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
+  printf "%s\n" '  --enable-ubpf[=CHOICE]   Whether and how to find the ubpf library'
+  printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
   printf "%s\n" '  --enable-strip           Strip targets on install'
   printf "%s\n" '  --enable-tcg-interpreter TCG with bytecode interpreter (slow)'
   printf "%s\n" '  --enable-trace-backends=CHOICES'
@@ -379,6 +381,9 @@ _meson_option_parse() {
     --enable-slirp=*) quote_sh "-Dslirp=$2" ;;
     --enable-slirp-smbd) printf "%s" -Dslirp_smbd=enabled ;;
     --disable-slirp-smbd) printf "%s" -Dslirp_smbd=disabled ;;
+    --enable-ubpf) printf "%s" -Dubpf=enabled ;;
+    --disable-ubpf) printf "%s" -Dubpf=disabled ;;
+    --enable-ubpf=*) quote_sh "-Dubpf=$2" ;;
     --enable-smartcard) printf "%s" -Dsmartcard=enabled ;;
     --disable-smartcard) printf "%s" -Dsmartcard=disabled ;;
     --enable-snappy) printf "%s" -Dsnappy=enabled ;;
-- 
2.25.1

[RFC PATCH 03/12] ebpf/uBPF: Introduce userspace ebpf data structure

[Zhang Chen]

Add ebpf/ubpf.h for the UbpfState.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 ebpf/ubpf.h | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
 create mode 100644 ebpf/ubpf.h

diff --git a/ebpf/ubpf.h b/ebpf/ubpf.h
new file mode 100644
index 0000000000..2562fff503
--- /dev/null
+++ b/ebpf/ubpf.h
@@ -0,0 +1,37 @@
+/*
+ * QEMU Userspace eBPF Header
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <chen.zhang@intel.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_UBPF_H
+#define QEMU_UBPF_H
+
+#include <ubpf.h>
+#include <math.h>
+#include <elf.h>
+
+#define MAX_LEN (1024 * 1024)
+
+typedef struct UbpfState {
+    bool jit;
+    char *code_path;
+    void *code;
+    size_t code_len;
+    char *target_path;
+    void *target;
+    size_t target_len;
+    struct ubpf_vm *vm;
+    ubpf_jit_fn fn;
+    int type;
+    char *func;
+} UbpfState;
+
+#endif /* QEMU_UBPF_H */
-- 
2.25.1

[RFC PATCH 04/12] ebpf/uBPF: Introduce ubpf initialize functions

[Zhang Chen]

Introduce ubpf.c/ubpf-stub.c with basic read and init_jit functions.
Add ubpf related .c files to meson.build.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 ebpf/meson.build |   1 +
 ebpf/ubpf-stub.c |  24 +++++++++++
 ebpf/ubpf.c      | 101 +++++++++++++++++++++++++++++++++++++++++++++++
 ebpf/ubpf.h      |   4 ++
 4 files changed, 130 insertions(+)
 create mode 100644 ebpf/ubpf-stub.c
 create mode 100644 ebpf/ubpf.c

diff --git a/ebpf/meson.build b/ebpf/meson.build
index 2dd0fd8948..f4457fbd28 100644
--- a/ebpf/meson.build
+++ b/ebpf/meson.build
@@ -1 +1,2 @@
 softmmu_ss.add(when: libbpf, if_true: files('ebpf_rss.c'), if_false: files('ebpf_rss-stub.c'))
+softmmu_ss.add(when: ubpf, if_true: files('ubpf.c'), if_false: files('ubpf-stub.c'))
diff --git a/ebpf/ubpf-stub.c b/ebpf/ubpf-stub.c
new file mode 100644
index 0000000000..2e8bf15b91
--- /dev/null
+++ b/ebpf/ubpf-stub.c
@@ -0,0 +1,24 @@
+/*
+ * QEMU Userspace eBPF Stub File
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <chen.zhang@intel.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path)
+{
+    return 0;
+}
+
+bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path)
+{
+    return 0;
+}
+
+void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit) {}
diff --git a/ebpf/ubpf.c b/ebpf/ubpf.c
new file mode 100644
index 0000000000..38a6530903
--- /dev/null
+++ b/ebpf/ubpf.c
@@ -0,0 +1,101 @@
+/*
+ * QEMU Userspace eBPF Support
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <chen.zhang@intel.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "ebpf/ubpf.h"
+
+static void *qemu_ubpf_read(const char *path, size_t maxlen, size_t *len)
+{
+    FILE *file;
+    size_t offset = 0, rv;
+    void *data;
+
+    if (!strcmp(path, "-")) {
+        file = fdopen(STDIN_FILENO, "r");
+    } else {
+        file = fopen(path, "r");
+    }
+
+    if (file == NULL) {
+        error_report("Failed to open %s: %s", path, strerror(errno));
+        return NULL;
+    }
+
+    data = g_malloc0(maxlen);
+
+    while ((rv = fread(data + offset, 1, maxlen - offset, file)) > 0) {
+        offset += rv;
+    }
+
+    if (ferror(file)) {
+        error_report("Failed to read %s: %s", path, strerror(errno));
+        goto err;
+    }
+
+    if (!feof(file)) {
+        error_report("Failed to read %s because it is too large"
+                     " (max %u bytes)", path, (unsigned)maxlen);
+        goto err;
+    }
+
+    fclose(file);
+    if (len) {
+        *len = offset;
+    }
+    return data;
+
+err:
+    fclose(file);
+    free(data);
+    return false;
+}
+
+/* Read Userspace eBPF binary file to QEMU */
+bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path)
+{
+    if (!path) {
+        return false;
+    }
+    u_ebpf->code_path = path;
+
+    u_ebpf->code = qemu_ubpf_read(u_ebpf->code_path, MAX_LEN,
+                                  &u_ebpf->code_len);
+    if (u_ebpf->code) {
+        return true;
+    } else {
+        return false;
+    }
+}
+
+/* Read Userspace eBPF target */
+bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path)
+{
+    if (!path) {
+        return false;
+    }
+    u_ebpf->target_path = path;
+
+    u_ebpf->target = qemu_ubpf_read(u_ebpf->target_path, MAX_LEN,
+                                    &u_ebpf->target_len);
+    if (u_ebpf->target) {
+        return true;
+    } else {
+        return false;
+    }
+}
+
+void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit)
+{
+    u_ebpf->jit = jit;
+}
diff --git a/ebpf/ubpf.h b/ebpf/ubpf.h
index 2562fff503..808c02565c 100644
--- a/ebpf/ubpf.h
+++ b/ebpf/ubpf.h
@@ -34,4 +34,8 @@ typedef struct UbpfState {
     char *func;
 } UbpfState;
 
+bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path);
+bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path);
+void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit);
+
 #endif /* QEMU_UBPF_H */
-- 
2.25.1

[RFC PATCH 05/12] ebpf/uBPF: Add qemu_prepare_ubpf to load ebpf binary

[Zhang Chen]

The qemu_prepare_ubpf() can load user defined userspace ebpf binary
file to Qemu userspace ebpf VM but not run it. The ebpf program
will triggered in the hook point.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 ebpf/ubpf-stub.c |   5 +++
 ebpf/ubpf.c      | 100 +++++++++++++++++++++++++++++++++++++++++++++++
 ebpf/ubpf.h      |   1 +
 3 files changed, 106 insertions(+)

diff --git a/ebpf/ubpf-stub.c b/ebpf/ubpf-stub.c
index 2e8bf15b91..885bd954b7 100644
--- a/ebpf/ubpf-stub.c
+++ b/ebpf/ubpf-stub.c
@@ -22,3 +22,8 @@ bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path)
 }
 
 void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit) {}
+
+int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path)
+{
+    return 0;
+}
diff --git a/ebpf/ubpf.c b/ebpf/ubpf.c
index 38a6530903..d65fffeda3 100644
--- a/ebpf/ubpf.c
+++ b/ebpf/ubpf.c
@@ -99,3 +99,103 @@ void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit)
 {
     u_ebpf->jit = jit;
 }
+
+static uint64_t gather_bytes(uint8_t a, uint8_t b, uint8_t c,
+                             uint8_t d, uint8_t e)
+{
+    return ((uint64_t)a << 32) |
+           ((uint32_t)b << 24) |
+           ((uint32_t)c << 16) |
+           ((uint16_t)d << 8) |
+           e;
+}
+
+static void trash_registers(void)
+{
+    /* Overwrite all caller-save registers */
+    asm(
+        "mov $0xf0, %rax;"
+        "mov $0xf1, %rcx;"
+        "mov $0xf2, %rdx;"
+        "mov $0xf3, %rsi;"
+        "mov $0xf4, %rdi;"
+        "mov $0xf5, %r8;"
+        "mov $0xf6, %r9;"
+        "mov $0xf7, %r10;"
+        "mov $0xf8, %r11;"
+    );
+}
+
+static uint32_t sqrti(uint32_t x)
+{
+    return sqrt(x);
+}
+
+static uint64_t unwind(uint64_t i)
+{
+    return i;
+}
+
+static void register_functions(struct ubpf_vm *vm)
+{
+    ubpf_register(vm, 0, "gather_bytes", gather_bytes);
+    ubpf_register(vm, 1, "memfrob", memfrob);
+    ubpf_register(vm, 2, "trash_registers", trash_registers);
+    ubpf_register(vm, 3, "sqrti", sqrti);
+    ubpf_register(vm, 4, "strcmp_ext", strcmp);
+    ubpf_register(vm, 5, "unwind", unwind);
+    ubpf_set_unwind_function_index(vm, 5);
+}
+
+int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path)
+{
+    bool is_elf;
+    char *errmsg;
+    int ret;
+
+    if (!qemu_ubpf_read_code(u_ebpf, code_path)) {
+        error_report("Ubpf failed to read code");
+        return -1;
+    }
+
+    u_ebpf->vm = ubpf_create();
+    if (!u_ebpf->vm) {
+        error_report("Failed to create ubpf VM");
+        return -1;
+    }
+
+    register_functions(u_ebpf->vm);
+
+    /*
+     * The ELF magic corresponds to an RSH instruction with an offset,
+     * which is invalid.
+     */
+     is_elf = u_ebpf->code_len >= SELFMAG && !memcmp(u_ebpf->code,
+                                                     ELFMAG, SELFMAG);
+
+    if (is_elf) {
+        ret = ubpf_load_elf(u_ebpf->vm, u_ebpf->code,
+                            u_ebpf->code_len, &errmsg);
+    } else {
+        ret = ubpf_load(u_ebpf->vm, u_ebpf->code,
+                        u_ebpf->code_len, &errmsg);
+    }
+
+    if (ret < 0) {
+        error_report("Failed to load ubpf code: %s ", errmsg);
+        free(errmsg);
+        ubpf_destroy(u_ebpf->vm);
+        return -1;
+    }
+
+    if (u_ebpf->jit) {
+        u_ebpf->fn = ubpf_compile(u_ebpf->vm, &errmsg);
+        if (u_ebpf->fn == NULL) {
+            error_report("Failed to ubpf compile: %s", errmsg);
+            free(errmsg);
+            return -1;
+        }
+    }
+
+    return 0;
+}
diff --git a/ebpf/ubpf.h b/ebpf/ubpf.h
index 808c02565c..9a35efbeb6 100644
--- a/ebpf/ubpf.h
+++ b/ebpf/ubpf.h
@@ -37,5 +37,6 @@ typedef struct UbpfState {
 bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path);
 bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path);
 void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit);
+int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path);
 
 #endif /* QEMU_UBPF_H */
-- 
2.25.1

[RFC PATCH 06/12] ebpf/uBPF: Add qemu_ubpf_run_once excute real ebpf program

[Zhang Chen]

Before running this function, we need to ensure that the
userspace ebpf program has been loaded correctly.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 ebpf/ubpf-stub.c |  6 ++++++
 ebpf/ubpf.c      | 16 ++++++++++++++++
 ebpf/ubpf.h      |  2 ++
 3 files changed, 24 insertions(+)

diff --git a/ebpf/ubpf-stub.c b/ebpf/ubpf-stub.c
index 885bd954b7..70da421629 100644
--- a/ebpf/ubpf-stub.c
+++ b/ebpf/ubpf-stub.c
@@ -27,3 +27,9 @@ int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path)
 {
     return 0;
 }
+
+uint64_t qemu_ubpf_run_once(UbpfState *u_ebpf, void *target,
+                            size_t target_len)
+{
+    return 0;
+}
diff --git a/ebpf/ubpf.c b/ebpf/ubpf.c
index d65fffeda3..8ac513c7ed 100644
--- a/ebpf/ubpf.c
+++ b/ebpf/ubpf.c
@@ -199,3 +199,19 @@ int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path)
 
     return 0;
 }
+
+uint64_t qemu_ubpf_run_once(UbpfState *u_ebpf, void *target,
+                            size_t target_len)
+{
+    uint64_t result;
+
+    if (u_ebpf->jit) {
+        result = u_ebpf->fn(target, target_len);
+    } else {
+        if (ubpf_exec(u_ebpf->vm, target, target_len, &result) < 0) {
+            result = UINT64_MAX;
+        }
+    }
+
+    return result;
+}
diff --git a/ebpf/ubpf.h b/ebpf/ubpf.h
index 9a35efbeb6..fc40e84e51 100644
--- a/ebpf/ubpf.h
+++ b/ebpf/ubpf.h
@@ -38,5 +38,7 @@ bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path);
 bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path);
 void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit);
 int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path);
+uint64_t qemu_ubpf_run_once(UbpfState *u_ebpf, void *target,
+                            size_t target_len);
 
 #endif /* QEMU_UBPF_H */
-- 
2.25.1

[RFC PATCH 07/12] net/filter: Introduce filter-ubpf module

[Zhang Chen]

The filter-ubpf module able to load user defined ebpf program
to handle network packet based on filter framework.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 net/filter-ubpf.c | 149 ++++++++++++++++++++++++++++++++++++++++++++++
 net/meson.build   |   1 +
 2 files changed, 150 insertions(+)
 create mode 100644 net/filter-ubpf.c

diff --git a/net/filter-ubpf.c b/net/filter-ubpf.c
new file mode 100644
index 0000000000..c63a021759
--- /dev/null
+++ b/net/filter-ubpf.c
@@ -0,0 +1,149 @@
+/*
+ * QEMU Userspace eBPF Support
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <chen.zhang@intel.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "net/filter.h"
+#include "net/net.h"
+#include "qapi/error.h"
+#include "qom/object.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "trace.h"
+#include "ebpf/ubpf.h"
+
+#define TYPE_FILTER_UBPF "filter-ubpf"
+OBJECT_DECLARE_SIMPLE_TYPE(FiliterUbpfState, FILTER_UBPF)
+
+struct FiliterUbpfState {
+    NetFilterState parent_obj;
+    bool ip_mode;
+    char *handler;
+    UbpfState ubpf;
+};
+
+static ssize_t filter_ubpf_receive_iov(NetFilterState *nf,
+                                       NetClientState *sender,
+                                       unsigned flags,
+                                       const struct iovec *iov,
+                                       int iovcnt,
+                                       NetPacketSent *sent_cb)
+{
+    /* TODO: handle packet by loaded userspace ebpf program */
+
+    return 0;
+}
+
+static void filter_ubpf_cleanup(NetFilterState *nf)
+{
+    /* cleanup */
+}
+
+static void filter_ubpf_setup(NetFilterState *nf, Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(nf);
+
+    if (s->handler == NULL) {
+        error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND, "filter-ubpf parameter"\
+                  " 'ubpf-handler' cannot be empty");
+        return;
+    }
+
+    qemu_ubpf_init_jit(&s->ubpf, true);
+
+    if (qemu_ubpf_prepare(&s->ubpf, s->handler)) {
+        error_set(errp, ERROR_CLASS_DEVICE_NOT_ACTIVE, "filter-ubpf parameter"\
+                  " 'ubpf-handler' cannot be load");
+        return;
+    }
+}
+
+static char *filter_ubpf_get_handler(Object *obj, Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    return g_strdup(s->handler);
+}
+
+static void filter_ubpf_set_handler(Object *obj,
+                                    const char *value,
+                                    Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    g_free(s->handler);
+    s->handler = g_strdup(value);
+    if (!s->handler) {
+        error_setg(errp, "filter ubpf needs 'ubpf-handler' "
+                   "property set");
+        return;
+    }
+}
+
+static bool filter_ubpf_get_mode(Object *obj, Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    return s->ip_mode;
+}
+
+static void filter_ubpf_set_mode(Object *obj, bool value, Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    s->ip_mode = value;
+}
+
+static void filter_ubpf_class_init(ObjectClass *oc, void *data)
+{
+    NetFilterClass *nfc = NETFILTER_CLASS(oc);
+
+    object_class_property_add_str(oc, "ubpf-handler",
+                                  filter_ubpf_get_handler,
+                                  filter_ubpf_set_handler);
+    object_class_property_add_bool(oc, "ip-mode",
+                                   filter_ubpf_get_mode,
+                                   filter_ubpf_set_mode);
+
+    nfc->setup = filter_ubpf_setup;
+    nfc->cleanup = filter_ubpf_cleanup;
+    nfc->receive_iov = filter_ubpf_receive_iov;
+}
+
+static void filter_ubpf_init(Object *obj)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    /* Filter-ubpf default is ip_mode */
+    s->ip_mode = true;
+}
+
+static void filter_ubpf_fini(Object *obj)
+{
+    /* do some thing */
+}
+
+static const TypeInfo filter_ubpf_info = {
+    .name = TYPE_FILTER_UBPF,
+    .parent = TYPE_NETFILTER,
+    .class_init = filter_ubpf_class_init,
+    .instance_init = filter_ubpf_init,
+    .instance_finalize = filter_ubpf_fini,
+    .instance_size = sizeof(FiliterUbpfState),
+};
+
+static void register_types(void)
+{
+    type_register_static(&filter_ubpf_info);
+}
+
+type_init(register_types);
diff --git a/net/meson.build b/net/meson.build
index 754e2d1d40..177078fa7a 100644
--- a/net/meson.build
+++ b/net/meson.build
@@ -14,6 +14,7 @@ softmmu_ss.add(files(
   'queue.c',
   'socket.c',
   'util.c',
+  'filter-ubpf.c',
 ))
 
 softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('filter-replay.c'))
-- 
2.25.1

[RFC PATCH 08/12] qapi: Add FilterUbpfProperties and qemu-options

[Zhang Chen]

Add filter-ubpf related QOM and qemu-options.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 qapi/qom.json   | 18 ++++++++++++++++++
 qemu-options.hx |  6 ++++++
 2 files changed, 24 insertions(+)

diff --git a/qapi/qom.json b/qapi/qom.json
index 6a653c6636..820a5218e8 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -444,6 +444,22 @@
   'base': 'NetfilterProperties',
   'data': { '*vnet_hdr_support': 'bool' } }
 
+##
+# @FilterUbpfProperties:
+#
+# Properties for filter-ubpf objects.
+#
+# @ip-mode: if true, IP packet handle mode is enabled(default: true).
+#
+# @ubpf-handler: The filename where the userspace ebpf packets handler.
+#
+# Since: 7.1
+##
+{ 'struct': 'FilterUbpfProperties',
+  'base': 'NetfilterProperties',
+  'data': { '*ip-mode': 'bool',
+            '*ubpf-handler': 'str' } }
+
 ##
 # @InputBarrierProperties:
 #
@@ -845,6 +861,7 @@
     'filter-redirector',
     'filter-replay',
     'filter-rewriter',
+    'filter-ubpf',
     'input-barrier',
     { 'name': 'input-linux',
       'if': 'CONFIG_LINUX' },
@@ -911,6 +928,7 @@
       'filter-redirector':          'FilterRedirectorProperties',
       'filter-replay':              'NetfilterProperties',
       'filter-rewriter':            'FilterRewriterProperties',
+      'filter-ubpf':                'FilterUbpfProperties',
       'input-barrier':              'InputBarrierProperties',
       'input-linux':                { 'type': 'InputLinuxProperties',
                                       'if': 'CONFIG_LINUX' },
diff --git a/qemu-options.hx b/qemu-options.hx
index 60cf188da4..3dfb858867 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -5080,6 +5080,12 @@ SRST
         stored. The file format is libpcap, so it can be analyzed with
         tools such as tcpdump or Wireshark.
 
+    ``-object filter-ubpf,id=id,netdev=dev,ubpf-handler=filename[,ip-mode][,position=head|tail|id=<id>][,insert=behind|before]``
+        filter-ubpf is the userspace ebpf network traffic handler on netdev dev
+        from the userspace ebpf handler file specified by filename.
+        If disable ip_mode, the loaded ebpf program will handle raw
+        network packet.
+
     ``-object colo-compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chardevid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@var{ms}][,expired_scan_cycle=@var{ms}][,max_queue_size=@var{size}]``
         Colo-compare gets packet from primary\_in chardevid and
         secondary\_in, then compare whether the payload of primary packet
-- 
2.25.1

[RFC PATCH 09/12] softmmu/vl.c: Add filter-ubpf for netdev as other netfilters

[Zhang Chen]

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 softmmu/vl.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 4c1e94b00e..d924fb1c71 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1822,7 +1822,8 @@ static bool object_create_early(const char *type)
         g_str_equal(type, "filter-redirector") ||
         g_str_equal(type, "colo-compare") ||
         g_str_equal(type, "filter-rewriter") ||
-        g_str_equal(type, "filter-replay")) {
+        g_str_equal(type, "filter-replay") ||
+        g_str_equal(type, "filter-ubpf")) {
         return false;
     }
 
-- 
2.25.1

[RFC PATCH 10/12] net/filter-ubpf.c: run the ubpf program to handle network packet

[Zhang Chen]

Run the loaded userspace ebpf program with the packet.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 net/filter-ubpf.c | 40 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 38 insertions(+), 2 deletions(-)

diff --git a/net/filter-ubpf.c b/net/filter-ubpf.c
index c63a021759..554cc24d8f 100644
--- a/net/filter-ubpf.c
+++ b/net/filter-ubpf.c
@@ -20,6 +20,8 @@
 #include "qemu/error-report.h"
 #include "trace.h"
 #include "ebpf/ubpf.h"
+#include "colo.h"
+#include "util.h"
 
 #define TYPE_FILTER_UBPF "filter-ubpf"
 OBJECT_DECLARE_SIMPLE_TYPE(FiliterUbpfState, FILTER_UBPF)
@@ -38,9 +40,43 @@ static ssize_t filter_ubpf_receive_iov(NetFilterState *nf,
                                        int iovcnt,
                                        NetPacketSent *sent_cb)
 {
-    /* TODO: handle packet by loaded userspace ebpf program */
+    FiliterUbpfState *s = FILTER_UBPF(nf);
+    size_t size;
+    char *buf;
+    Packet *pkt = NULL;
+    uint64_t result;
+
+    size = iov_size(iov, iovcnt);
+    if (!size) {
+        return 0;
+    }
+
+    buf = g_malloc(size);
+    if (unlikely(iov_to_buf(iov, iovcnt, 0, buf, size) != size)) {
+        g_free(buf);
+        return 0;
+    }
+
+    pkt = packet_new_nocopy(buf, size, 0);
 
-    return 0;
+    if (parse_packet_early(pkt)) {
+        packet_destroy(pkt, NULL);
+        pkt = NULL;
+        return 0;
+    }
+
+    if (s->ip_mode) {
+        result = qemu_ubpf_run_once(&s->ubpf, pkt->ip, sizeof(struct ip));
+    } else {
+        result = qemu_ubpf_run_once(&s->ubpf, pkt->data, pkt->size);
+    }
+
+    /* If result == 1, means trigger the ebpf program rules */
+    if (result) {
+        return -1;
+    } else {
+        return 0;
+    }
 }
 
 static void filter_ubpf_cleanup(NetFilterState *nf)
-- 
2.25.1

[RFC PATCH 11/12] docs/devel: Add userspace-ebpf.rst

[Zhang Chen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 4278 bytes --]

Introduce userspace ebpf basic knowledge.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 docs/devel/userspace-ebpf.rst | 106 ++++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)
 create mode 100644 docs/devel/userspace-ebpf.rst

diff --git a/docs/devel/userspace-ebpf.rst b/docs/devel/userspace-ebpf.rst
new file mode 100644
index 0000000000..41eb9b04d6
--- /dev/null
+++ b/docs/devel/userspace-ebpf.rst
@@ -0,0 +1,106 @@
+===========================
+Userspace eBPF support
+===========================
+
+eBPF is a revolutionary technology with origins in the Linux kernel that
+can run sandboxed programs in an operating system kernel. It is used to
+safely and efficiently extend the capabilities of the kernel without
+requiring to change kernel source code or load kernel
+modules.(from https://ebpf.io/)
+
+Recently, I worked on QEMU net filter related jobs, like netfilter/iptables
+in kernel. We noticed kernel extend the netfilter original cBPF to eBPF,
+
+It make Linux kernel have the ability to load code dynamically. Why not
+enable user space eBPF in QEMU? It can load binary eBPF program even
+when VM running. Add some hooks in QEMU as the user space eBPF load point.
+Do the things on different layers. The original idea from Jason Wang.
+
+
+That’s the advantages of kernel eBPF. Most of the functions can be
+implemented in QEMU. The Power of Programmability.
+
+    1). Safety:
+
+    Building on the foundation of seeing and understanding all system
+    calls and combining that with a packet and socket-level view of all
+    networking operations allows for revolutionary new approaches to
+    securing systems.
+
+    2). Tracing & Profiling:
+
+    The ability to attach eBPF programs to trace points as well as kernel
+    and user application probe points allows unprecedented visibility into
+    the runtime behavior of applications and the system itself.
+
+    3). Networking:
+
+    The combination of programmability and efficiency makes eBPF a natural
+    fit for all packet processing requirements of networking solutions.
+
+    4). Observability & Monitoring:
+
+    Instead of relying on static counters and gauges exposed by the
+    perating system, eBPF enables the collection & in-kernel aggregation
+    of custom metrics and generation of visibility events based on a wide
+    range of possible sources.
+
+    Qemu userspace ebpf design based on ubpf project (https://github.com/iovisor/ubpf).
+    The most mature userspace ebpf implementation. This project officially
+    support by iovisor(Like BCC and bpftrace). Qemu userspace ebpf make
+    the ubpf project as the git submodule.
+
+    Current implementation support load ebpf program and run it in
+    filter-ubpf module, developer can easy reuse the ubpf function in
+    Qemu's other modules from the function in /ebpf/ubpf.c, And it support JIT.
+    For the uBPF License is Apache License 2.0, It's OK to compatible
+    with QEMU’s GPLv2 LICENSE same as mason.
+
+    How to use it:
+    1. Write your ebpf C program. For example filter dst IP:
+
+    bpf_filter.c
+
+#include <arpa/inet.h>
+#include <stdint.h>
+
+#define ONE_ONE_ONE_ONE 0x01010101
+
+struct ipv4_header {
+    uint8_t ver_ihl;
+    uint8_t tos;
+    uint16_t total_length;
+    uint16_t id;
+    uint16_t frag;
+    uint8_t ttl;
+    uint8_t proto;
+    uint16_t csum;
+    uint32_t src;
+    uint32_t dst;
+};
+
+int is_dst_one_one_one_one(void *opaque) {
+    struct ipv4_header *ipv4_header = (struct ipv4_header*)opaque;
+
+    if (ntohl(ipv4_header->dst) == ONE_ONE_ONE_ONE) {
+        return 1;
+    }
+
+    return 0;
+}
+
+    2. Build it with clang:
+      clang -O2 -target bpf -c bpf_filter.c -o ip_dst.o
+
+    3. Load it with Qemu filter-ubpf:
+      -object filter-ubpf,netdev=hn0,id=ubpf1,queue=tx,ip-mode=on,
+       ubpf-handler=ip_dst.o
+
+    4. Boot the VM and it will filt IP dst 1.1.1.1 packet.
+
+
+TODO: Need to add more comments and test-case for ubpf, current
+      implementation not include ebpf verifier. Qemu is a userspace
+      program, not like kernel ebpf run code in kernel space, I think
+      if the someone want to hack Qemu code no need to load a malicious
+      ubpf program, he can hack Qemu code directly.
-- 
2.25.1

[RFC PATCH 12/12] test/qtest: Add ubpf basic test case

[Zhang Chen]

TODO: This test case does not work. Need add ubpf.h header in qtest
compile "-I ../ubpf/vm -I ../ubpf/vm/inc".
I'm not sure if we need it in qtest. Because normal tests/qtest
not including external module test case like fdt. Or we just
need a qtest case for filter-ubpf module.
This test will load pre-compiled ebpf binary and run it in QEMU.

Signed-off-by: Zhang Chen <chen.zhang@intel.com>
---
 tests/qtest/demo_ubpf.o   | Bin 0 -> 544 bytes
 tests/qtest/integer_5.mem | Bin 0 -> 4 bytes
 tests/qtest/meson.build   |   3 +-
 tests/qtest/ubpf-test.c   |  64 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 66 insertions(+), 1 deletion(-)
 create mode 100644 tests/qtest/demo_ubpf.o
 create mode 100644 tests/qtest/integer_5.mem
 create mode 100644 tests/qtest/ubpf-test.c

diff --git a/tests/qtest/demo_ubpf.o b/tests/qtest/demo_ubpf.o
new file mode 100644
index 0000000000000000000000000000000000000000..960a411c224348548db42d9ae2716ae3ef4ea249
GIT binary patch
literal 544
zcmb<-^>JfjWMqH=MuzVU2p&w7f#Csy$>0EHJ20>URVE5RB+`KtNZ(Wl7bhtPlwo1`
z_#a&XJ5WG~fe9`w0b}Wvq*jzLBo(B^7Zl~EGw9{yl;y@Jrlb@VXQnfh0>yPpQj1IU
zk{R@hONvSolYn$(E{LWM&;lC6jK!!0P%$esIrOjt@j;jkO`QXj5BDdO&w-{66uitn
o|MP)V1G3ZtWDbzc0_CIIZv+%agepQ)1eEE4qz|MHW<Shb0E~Jd)c^nh

literal 0
HcmV?d00001

diff --git a/tests/qtest/integer_5.mem b/tests/qtest/integer_5.mem
new file mode 100644
index 0000000000000000000000000000000000000000..a786e127004dd9e94e88fda7742d248237ad8885
GIT binary patch
literal 4
LcmZQ&U|;|M02lxU

literal 0
HcmV?d00001

diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 31287a9173..9cc629e22b 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -94,7 +94,8 @@ qtests_i386 = \
    'migration-test',
    'test-x86-cpuid-compat',
    'numa-test',
-   'test-filter-redirector'
+   'test-filter-redirector',
+   'ubpf-test'
   ]
 
 if dbus_display
diff --git a/tests/qtest/ubpf-test.c b/tests/qtest/ubpf-test.c
new file mode 100644
index 0000000000..6e70a99320
--- /dev/null
+++ b/tests/qtest/ubpf-test.c
@@ -0,0 +1,64 @@
+/*
+ * QEMU Userspace eBPF test case
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <chen.zhang@intel.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "libqtest.h"
+#include "ebpf/ubpf.h"
+
+/*
+ * Demo userspace ebpf program
+ * The test binary use clang to build this source code:
+ * demo_ubpf.c
+ *
+ * #include <stdint.h>
+ *
+ * static uint32_t double_it(uint32_t a)
+ * {
+ *      return (a * 2);
+ * }
+ *
+ * uint32_t bpf_prog(int32_t *arg) {
+ *       uint32_t result = 0;
+ *       result = double_it(*arg);
+ *
+ *       return result;
+ * }
+ *
+ * Build the userspace ebpf program binary file:
+ * clang -O2 -target bpf -c demo_ubpf.c -o demo_ubpf.o
+ *
+ * The external terget source:
+ * printf "%b" '\x05\x00\x00\x00' > integer_5.mem
+ *
+ */
+
+int main(int argc, char **argc)
+{
+    UbpfState u_ebpf;
+    char program_path[] = "demo_ubpf.o";
+    /* uBPF can read target from internal source or external source*/
+    char target_path[] = "integer_5.mem";
+
+    qemu_ubpf_init_jit(&u_ebpf, true);
+
+    g_assert_cmpuint(qemu_ubpf_prepare(&u_ebpf, program_path), ==, 0);
+
+    g_assert_true(qemu_ubpf_read_target(&u_ebpf, target_path));
+
+    g_assert_cmpuint(qemu_run_ubpf_once(&u_ebpf, u_ebpf.target,
+		     u_ebpf.target_len), ==, 10);
+
+    ubpf_destroy(u_ebpf.vm);
+
+    return 0;
+}
-- 
2.25.1

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Daniel P. Berrangé]

On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> Make iovisor/ubpf project be a git submodule for QEMU.
> It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we
don't have any back compat issues. We should just expect the distros
to ship ubpf if they want their QEMU builds to take advantage of it.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case

[Thomas Huth]

On 17/06/2022 09.36, Zhang Chen wrote:
> TODO: This test case does not work. Need add ubpf.h header in qtest
> compile "-I ../ubpf/vm -I ../ubpf/vm/inc".
> I'm not sure if we need it in qtest. Because normal tests/qtest
> not including external module test case like fdt. Or we just
> need a qtest case for filter-ubpf module.
> This test will load pre-compiled ebpf binary and run it in QEMU.
> 
> Signed-off-by: Zhang Chen <chen.zhang@intel.com>
> ---
[...]
> diff --git a/tests/qtest/ubpf-test.c b/tests/qtest/ubpf-test.c
> new file mode 100644
> index 0000000000..6e70a99320
> --- /dev/null
> +++ b/tests/qtest/ubpf-test.c
> @@ -0,0 +1,64 @@
> +/*
> + * QEMU Userspace eBPF test case
> + *
> + * Copyright(C) 2022 Intel Corporation.
> + *
> + * Author:
> + *  Zhang Chen <chen.zhang@intel.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "libqtest.h"
> +#include "ebpf/ubpf.h"
> +
> +/*
> + * Demo userspace ebpf program
> + * The test binary use clang to build this source code:
> + * demo_ubpf.c
> + *
> + * #include <stdint.h>
> + *
> + * static uint32_t double_it(uint32_t a)
> + * {
> + *      return (a * 2);
> + * }
> + *
> + * uint32_t bpf_prog(int32_t *arg) {
> + *       uint32_t result = 0;
> + *       result = double_it(*arg);
> + *
> + *       return result;
> + * }
> + *
> + * Build the userspace ebpf program binary file:
> + * clang -O2 -target bpf -c demo_ubpf.c -o demo_ubpf.o
> + *
> + * The external terget source:
> + * printf "%b" '\x05\x00\x00\x00' > integer_5.mem
> + *
> + */
> +
> +int main(int argc, char **argc)
> +{
> +    UbpfState u_ebpf;
> +    char program_path[] = "demo_ubpf.o";
> +    /* uBPF can read target from internal source or external source*/
> +    char target_path[] = "integer_5.mem";
> +
> +    qemu_ubpf_init_jit(&u_ebpf, true);
> +
> +    g_assert_cmpuint(qemu_ubpf_prepare(&u_ebpf, program_path), ==, 0);
> +
> +    g_assert_true(qemu_ubpf_read_target(&u_ebpf, target_path));
> +
> +    g_assert_cmpuint(qemu_run_ubpf_once(&u_ebpf, u_ebpf.target,
> +		     u_ebpf.target_len), ==, 10);
> +
> +    ubpf_destroy(u_ebpf.vm);
> +
> +    return 0;
> +}

Apart from the #include "libqtest.h" there is nothing related to qtest in 
here ... should this maybe rather go into test/unit/ instead?

  Thomas

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Zhang, Chen]

> -----Original Message-----
> From: Daniel P. Berrangé <berrange@redhat.com>
> Sent: Friday, June 17, 2022 4:05 PM
> To: Zhang, Chen <chen.zhang@intel.com>
> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>; Peter Maydell
> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>; Laurent
> Vivier <lvivier@redhat.com>; Yuri Benditovich
> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> <andrew@daynix.com>
> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> submodule for QEMU
> 
> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> > Make iovisor/ubpf project be a git submodule for QEMU.
> > It will auto clone ubpf project when configure QEMU.
> 
> I don't think we need todo this. As it is brand new functionality we don't have
> any back compat issues. We should just expect the distros to ship ubpf if
> they want their QEMU builds to take advantage of it.
> 

Yes, agree. It's the best way to use the uBPF project. 
But current status is distros(ubuntu, RHEL...) does not ship
the iovisor/ubpf like the iovisor/bcc. So I have to do it.
Or do you have any better suggestions? 

Thanks
Chen

> 
> With regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|

Re: [RFC PATCH 08/12] qapi: Add FilterUbpfProperties and qemu-options

[Markus Armbruster]

Zhang Chen <chen.zhang@intel.com> writes:

> Add filter-ubpf related QOM and qemu-options.
>
> Signed-off-by: Zhang Chen <chen.zhang@intel.com>
> ---
>  qapi/qom.json   | 18 ++++++++++++++++++
>  qemu-options.hx |  6 ++++++
>  2 files changed, 24 insertions(+)
>
> diff --git a/qapi/qom.json b/qapi/qom.json
> index 6a653c6636..820a5218e8 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -444,6 +444,22 @@
>    'base': 'NetfilterProperties',
>    'data': { '*vnet_hdr_support': 'bool' } }
>  
> +##
> +# @FilterUbpfProperties:
> +#
> +# Properties for filter-ubpf objects.
> +#
> +# @ip-mode: if true, IP packet handle mode is enabled(default: true).

Space between "enabled" and "(default: true)", please.

I'm not sure about the name @ip-mode.  A mode tends to be an enum.  A
boolean tends to be a flag, like @disable-packed-handle-mode.  Note that
I reverted the sense there, to make the default false.

> +#
> +# @ubpf-handler: The filename where the userspace ebpf packets handler.
> +#
> +# Since: 7.1
> +##
> +{ 'struct': 'FilterUbpfProperties',
> +  'base': 'NetfilterProperties',
> +  'data': { '*ip-mode': 'bool',
> +            '*ubpf-handler': 'str' } }
> +
>  ##
>  # @InputBarrierProperties:
>  #
> @@ -845,6 +861,7 @@
>      'filter-redirector',
>      'filter-replay',
>      'filter-rewriter',
> +    'filter-ubpf',
>      'input-barrier',
>      { 'name': 'input-linux',
>        'if': 'CONFIG_LINUX' },
> @@ -911,6 +928,7 @@
>        'filter-redirector':          'FilterRedirectorProperties',
>        'filter-replay':              'NetfilterProperties',
>        'filter-rewriter':            'FilterRewriterProperties',
> +      'filter-ubpf':                'FilterUbpfProperties',
>        'input-barrier':              'InputBarrierProperties',
>        'input-linux':                { 'type': 'InputLinuxProperties',
>                                        'if': 'CONFIG_LINUX' },
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 60cf188da4..3dfb858867 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -5080,6 +5080,12 @@ SRST
>          stored. The file format is libpcap, so it can be analyzed with
>          tools such as tcpdump or Wireshark.
>  
> +    ``-object filter-ubpf,id=id,netdev=dev,ubpf-handler=filename[,ip-mode][,position=head|tail|id=<id>][,insert=behind|before]``
> +        filter-ubpf is the userspace ebpf network traffic handler on netdev dev
> +        from the userspace ebpf handler file specified by filename.

I believe the conventional capitalization is eBPF.

> +        If disable ip_mode, the loaded ebpf program will handle raw

Markup: ``ip_mode``.

> +        network packet.

Suggest something like "If ``ip_mode`` is off, the eBPF program is fed
raw network packets" (hope I'm not misinterpreting things).

> +
>      ``-object colo-compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chardevid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@var{ms}][,expired_scan_cycle=@var{ms}][,max_queue_size=@var{size}]``
>          Colo-compare gets packet from primary\_in chardevid and
>          secondary\_in, then compare whether the payload of primary packet

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Daniel P. Berrangé]

On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> 
> 
> > -----Original Message-----
> > From: Daniel P. Berrangé <berrange@redhat.com>
> > Sent: Friday, June 17, 2022 4:05 PM
> > To: Zhang, Chen <chen.zhang@intel.com>
> > Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
> > Armbruster <armbru@redhat.com>; Peter Maydell
> > <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>; Laurent
> > Vivier <lvivier@redhat.com>; Yuri Benditovich
> > <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > <andrew@daynix.com>
> > Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> > submodule for QEMU
> > 
> > On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> > > Make iovisor/ubpf project be a git submodule for QEMU.
> > > It will auto clone ubpf project when configure QEMU.
> > 
> > I don't think we need todo this. As it is brand new functionality we don't have
> > any back compat issues. We should just expect the distros to ship ubpf if
> > they want their QEMU builds to take advantage of it.
> > 
> 
> Yes, agree. It's the best way to use the uBPF project. 
> But current status is distros(ubuntu, RHEL...) does not ship
> the iovisor/ubpf like the iovisor/bcc. So I have to do it.
> Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for
it IMHO.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Thomas Huth]

On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
>>
>>
>>> -----Original Message-----
>>> From: Daniel P. Berrangé <berrange@redhat.com>
>>> Sent: Friday, June 17, 2022 4:05 PM
>>> To: Zhang, Chen <chen.zhang@intel.com>
>>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
>>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
>>> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
>>> Armbruster <armbru@redhat.com>; Peter Maydell
>>> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>; Laurent
>>> Vivier <lvivier@redhat.com>; Yuri Benditovich
>>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
>>> <andrew@daynix.com>
>>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
>>> submodule for QEMU
>>>
>>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
>>>> Make iovisor/ubpf project be a git submodule for QEMU.
>>>> It will auto clone ubpf project when configure QEMU.
>>>
>>> I don't think we need todo this. As it is brand new functionality we don't have
>>> any back compat issues. We should just expect the distros to ship ubpf if
>>> they want their QEMU builds to take advantage of it.
>>>
>>
>> Yes, agree. It's the best way to use the uBPF project.
>> But current status is distros(ubuntu, RHEL...) does not ship
>> the iovisor/ubpf like the iovisor/bcc. So I have to do it.
>> Or do you have any better suggestions?
> 
> If distros want to support the functionality, they can add packages for
> it IMHO.

Yes, let's please avoid new submodules. Submodules can sometimes be a real 
PITA (e.g. if you forget to update before rsync'ing your code to a machine 
that has limited internet access), and if users install QEMU from sources, 
they can also install ubpf from sources, too.
And if distros want to support this feature, they can package ubpf on their 
own, as Daniel said.

  Thomas

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Zhang, Chen]

> -----Original Message-----
> From: Thomas Huth <thuth@redhat.com>
> Sent: Monday, June 20, 2022 4:47 PM
> To: Daniel P. Berrangé <berrange@redhat.com>; Zhang, Chen
> <chen.zhang@intel.com>
> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>; Peter Maydell
> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> <andrew@daynix.com>
> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> submodule for QEMU
> 
> On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> > On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: Daniel P. Berrangé <berrange@redhat.com>
> >>> Sent: Friday, June 17, 2022 4:05 PM
> >>> To: Zhang, Chen <chen.zhang@intel.com>
> >>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> >>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> >>> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> >>> Markus Armbruster <armbru@redhat.com>; Peter Maydell
> >>> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>;
> Laurent
> >>> Vivier <lvivier@redhat.com>; Yuri Benditovich
> >>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> >>> <andrew@daynix.com>
> >>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project
> >>> as a submodule for QEMU
> >>>
> >>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> >>>> Make iovisor/ubpf project be a git submodule for QEMU.
> >>>> It will auto clone ubpf project when configure QEMU.
> >>>
> >>> I don't think we need todo this. As it is brand new functionality we
> >>> don't have any back compat issues. We should just expect the distros
> >>> to ship ubpf if they want their QEMU builds to take advantage of it.
> >>>
> >>
> >> Yes, agree. It's the best way to use the uBPF project.
> >> But current status is distros(ubuntu, RHEL...) does not ship the
> >> iovisor/ubpf like the iovisor/bcc. So I have to do it.
> >> Or do you have any better suggestions?
> >
> > If distros want to support the functionality, they can add packages
> > for it IMHO.
> 
> Yes, let's please avoid new submodules. Submodules can sometimes be a
> real PITA (e.g. if you forget to update before rsync'ing your code to a
> machine that has limited internet access), and if users install QEMU from
> sources, they can also install ubpf from sources, too.
> And if distros want to support this feature, they can package ubpf on their
> own, as Daniel said.

Hi Daniel and Thomas,

I don't know much the background history of QEMU submodules, but meson build
is a submodule for QEMU too. It means user can't install QEMU from sources
with limited internet access. 
And back to Daniel's comments,  Yes, the best way is distros add the ubpf packages,
But maybe it's too late to implement new features for us. We can introduce the submodule now and
auto change to the distros's lib when distros add it.  For example QEMU's submodule SLIRP do it in the same way.
It's already added by most distros and still as a QEMU submodule. It make user experience the latest technology
with no other dependencies. uBPF infrastructure have the ability to extend the capabilities without requiring
changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler,
interpreter, and JIT compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Thanks
Chen


> 
>   Thomas

RE: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case

[Zhang, Chen]

> -----Original Message-----
> From: Thomas Huth <thuth@redhat.com>
> Sent: Friday, June 17, 2022 5:34 PM
> To: Zhang, Chen <chen.zhang@intel.com>; Jason Wang
> <jasowang@redhat.com>; qemu-dev <qemu-devel@nongnu.org>; Paolo
> Bonzini <pbonzini@redhat.com>; Daniel P. Berrangé
> <berrange@redhat.com>; Eduardo Habkost <eduardo@habkost.net>; Eric
> Blake <eblake@redhat.com>; Markus Armbruster <armbru@redhat.com>
> Cc: Peter Maydell <peter.maydell@linaro.org>; Laurent Vivier
> <lvivier@redhat.com>; Yuri Benditovich <yuri.benditovich@daynix.com>;
> Andrew Melnychenko <andrew@daynix.com>
> Subject: Re: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case
> 
> On 17/06/2022 09.36, Zhang Chen wrote:
> > TODO: This test case does not work. Need add ubpf.h header in qtest
> > compile "-I ../ubpf/vm -I ../ubpf/vm/inc".
> > I'm not sure if we need it in qtest. Because normal tests/qtest not
> > including external module test case like fdt. Or we just need a qtest
> > case for filter-ubpf module.
> > This test will load pre-compiled ebpf binary and run it in QEMU.
> >
> > Signed-off-by: Zhang Chen <chen.zhang@intel.com>
> > ---
> [...]
> > diff --git a/tests/qtest/ubpf-test.c b/tests/qtest/ubpf-test.c new
> > file mode 100644 index 0000000000..6e70a99320
> > --- /dev/null
> > +++ b/tests/qtest/ubpf-test.c
> > @@ -0,0 +1,64 @@
> > +/*
> > + * QEMU Userspace eBPF test case
> > + *
> > + * Copyright(C) 2022 Intel Corporation.
> > + *
> > + * Author:
> > + *  Zhang Chen <chen.zhang@intel.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + *
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "libqtest.h"
> > +#include "ebpf/ubpf.h"
> > +
> > +/*
> > + * Demo userspace ebpf program
> > + * The test binary use clang to build this source code:
> > + * demo_ubpf.c
> > + *
> > + * #include <stdint.h>
> > + *
> > + * static uint32_t double_it(uint32_t a)
> > + * {
> > + *      return (a * 2);
> > + * }
> > + *
> > + * uint32_t bpf_prog(int32_t *arg) {
> > + *       uint32_t result = 0;
> > + *       result = double_it(*arg);
> > + *
> > + *       return result;
> > + * }
> > + *
> > + * Build the userspace ebpf program binary file:
> > + * clang -O2 -target bpf -c demo_ubpf.c -o demo_ubpf.o
> > + *
> > + * The external terget source:
> > + * printf "%b" '\x05\x00\x00\x00' > integer_5.mem
> > + *
> > + */
> > +
> > +int main(int argc, char **argc)
> > +{
> > +    UbpfState u_ebpf;
> > +    char program_path[] = "demo_ubpf.o";
> > +    /* uBPF can read target from internal source or external source*/
> > +    char target_path[] = "integer_5.mem";
> > +
> > +    qemu_ubpf_init_jit(&u_ebpf, true);
> > +
> > +    g_assert_cmpuint(qemu_ubpf_prepare(&u_ebpf, program_path), ==,
> > + 0);
> > +
> > +    g_assert_true(qemu_ubpf_read_target(&u_ebpf, target_path));
> > +
> > +    g_assert_cmpuint(qemu_run_ubpf_once(&u_ebpf, u_ebpf.target,
> > +		     u_ebpf.target_len), ==, 10);
> > +
> > +    ubpf_destroy(u_ebpf.vm);
> > +
> > +    return 0;
> > +}
> 
> Apart from the #include "libqtest.h" there is nothing related to qtest in
> here ... should this maybe rather go into test/unit/ instead?

Rethink about it, I think you are right.
The only issue is can we involve submodule's header file in tests/unit?
I can't find meson/fdt/SLIRP test cases in the tests.

Thanks
Chen

> 
>   Thomas

RE: [RFC PATCH 08/12] qapi: Add FilterUbpfProperties and qemu-options

[Zhang, Chen]

> -----Original Message-----
> From: Markus Armbruster <armbru@redhat.com>
> Sent: Monday, June 20, 2022 3:45 PM
> To: Zhang, Chen <chen.zhang@intel.com>
> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Daniel
> P.Berrangé <berrange@redhat.com>; Eduardo Habkost
> <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Peter Maydell
> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>; Laurent
> Vivier <lvivier@redhat.com>; Yuri Benditovich
> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> <andrew@daynix.com>
> Subject: Re: [RFC PATCH 08/12] qapi: Add FilterUbpfProperties and qemu-
> options
> 
> Zhang Chen <chen.zhang@intel.com> writes:
> 
> > Add filter-ubpf related QOM and qemu-options.
> >
> > Signed-off-by: Zhang Chen <chen.zhang@intel.com>
> > ---
> >  qapi/qom.json   | 18 ++++++++++++++++++
> >  qemu-options.hx |  6 ++++++
> >  2 files changed, 24 insertions(+)
> >
> > diff --git a/qapi/qom.json b/qapi/qom.json index
> > 6a653c6636..820a5218e8 100644
> > --- a/qapi/qom.json
> > +++ b/qapi/qom.json
> > @@ -444,6 +444,22 @@
> >    'base': 'NetfilterProperties',
> >    'data': { '*vnet_hdr_support': 'bool' } }
> >
> > +##
> > +# @FilterUbpfProperties:
> > +#
> > +# Properties for filter-ubpf objects.
> > +#
> > +# @ip-mode: if true, IP packet handle mode is enabled(default: true).
> 
> Space between "enabled" and "(default: true)", please.
> 
> I'm not sure about the name @ip-mode.  A mode tends to be an enum.  A
> boolean tends to be a flag, like @disable-packed-handle-mode.  Note that I
> reverted the sense there, to make the default false.

Thanks for your review Markus~

Makes sense. Current mode just include ip-mode and raw-mode, it looks
Need change to a enum mode for it, maybe will add other mode in the future.

> 
> > +#
> > +# @ubpf-handler: The filename where the userspace ebpf packets
> handler.
> > +#
> > +# Since: 7.1
> > +##
> > +{ 'struct': 'FilterUbpfProperties',
> > +  'base': 'NetfilterProperties',
> > +  'data': { '*ip-mode': 'bool',
> > +            '*ubpf-handler': 'str' } }
> > +
> >  ##
> >  # @InputBarrierProperties:
> >  #
> > @@ -845,6 +861,7 @@
> >      'filter-redirector',
> >      'filter-replay',
> >      'filter-rewriter',
> > +    'filter-ubpf',
> >      'input-barrier',
> >      { 'name': 'input-linux',
> >        'if': 'CONFIG_LINUX' },
> > @@ -911,6 +928,7 @@
> >        'filter-redirector':          'FilterRedirectorProperties',
> >        'filter-replay':              'NetfilterProperties',
> >        'filter-rewriter':            'FilterRewriterProperties',
> > +      'filter-ubpf':                'FilterUbpfProperties',
> >        'input-barrier':              'InputBarrierProperties',
> >        'input-linux':                { 'type': 'InputLinuxProperties',
> >                                        'if': 'CONFIG_LINUX' }, diff
> > --git a/qemu-options.hx b/qemu-options.hx index 60cf188da4..3dfb858867
> > 100644
> > --- a/qemu-options.hx
> > +++ b/qemu-options.hx
> > @@ -5080,6 +5080,12 @@ SRST
> >          stored. The file format is libpcap, so it can be analyzed with
> >          tools such as tcpdump or Wireshark.
> >
> > +    ``-object filter-ubpf,id=id,netdev=dev,ubpf-handler=filename[,ip-
> mode][,position=head|tail|id=<id>][,insert=behind|before]``
> > +        filter-ubpf is the userspace ebpf network traffic handler on netdev
> dev
> > +        from the userspace ebpf handler file specified by filename.
> 
> I believe the conventional capitalization is eBPF.

Yes, will fix it.

> 
> > +        If disable ip_mode, the loaded ebpf program will handle raw
> 
> Markup: ``ip_mode``.

OK.

> 
> > +        network packet.
> 
> Suggest something like "If ``ip_mode`` is off, the eBPF program is fed raw
> network packets" (hope I'm not misinterpreting things).

OK. I will address your comments in next version.

Thanks
Chen

> 
> > +
> >      ``-object colo-
> compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chard
> evid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@
> var{ms}][,expired_scan_cycle=@var{ms}][,max_queue_size=@var{size}]``
> >          Colo-compare gets packet from primary\_in chardevid and
> >          secondary\_in, then compare whether the payload of primary
> > packet

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Thomas Huth]

On 20/06/2022 11.29, Zhang, Chen wrote:
> 
> 
>> -----Original Message-----
>> From: Thomas Huth <thuth@redhat.com>
>> Sent: Monday, June 20, 2022 4:47 PM
>> To: Daniel P. Berrangé <berrange@redhat.com>; Zhang, Chen
>> <chen.zhang@intel.com>
>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
>> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
>> Armbruster <armbru@redhat.com>; Peter Maydell
>> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
>> Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
>> <andrew@daynix.com>
>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
>> submodule for QEMU
>>
>> On 20/06/2022 10.11, Daniel P. Berrangé wrote:
>>> On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Daniel P. Berrangé <berrange@redhat.com>
>>>>> Sent: Friday, June 17, 2022 4:05 PM
>>>>> To: Zhang, Chen <chen.zhang@intel.com>
>>>>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
>>>>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
>>>>> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
>>>>> Markus Armbruster <armbru@redhat.com>; Peter Maydell
>>>>> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>;
>> Laurent
>>>>> Vivier <lvivier@redhat.com>; Yuri Benditovich
>>>>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
>>>>> <andrew@daynix.com>
>>>>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project
>>>>> as a submodule for QEMU
>>>>>
>>>>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
>>>>>> Make iovisor/ubpf project be a git submodule for QEMU.
>>>>>> It will auto clone ubpf project when configure QEMU.
>>>>>
>>>>> I don't think we need todo this. As it is brand new functionality we
>>>>> don't have any back compat issues. We should just expect the distros
>>>>> to ship ubpf if they want their QEMU builds to take advantage of it.
>>>>>
>>>>
>>>> Yes, agree. It's the best way to use the uBPF project.
>>>> But current status is distros(ubuntu, RHEL...) does not ship the
>>>> iovisor/ubpf like the iovisor/bcc. So I have to do it.
>>>> Or do you have any better suggestions?
>>>
>>> If distros want to support the functionality, they can add packages
>>> for it IMHO.
>>
>> Yes, let's please avoid new submodules. Submodules can sometimes be a
>> real PITA (e.g. if you forget to update before rsync'ing your code to a
>> machine that has limited internet access), and if users install QEMU from
>> sources, they can also install ubpf from sources, too.
>> And if distros want to support this feature, they can package ubpf on their
>> own, as Daniel said.
> 
> Hi Daniel and Thomas,
> 
> I don't know much the background history of QEMU submodules, but meson build
> is a submodule for QEMU too. It means user can't install QEMU from sources
> with limited internet access.

There is no written policy, but I think the general consensus is that we 
only ship code in submodules if:

1) It's not available in a required version in distros yet

and

2) it is essentially required to build QEMU (like meson) or if the feature 
has been part of the QEMU sources before and then moved to a separate 
repository (like slirp).

We ship meson as a submodule since we require some meson features that are 
not available with the meson versions in the distros yet. Once the distros 
catch up, we'll likely remove the meson submodule from QEMU.

> And back to Daniel's comments,  Yes, the best way is distros add the ubpf packages,
> But maybe it's too late to implement new features for us. We can introduce the submodule now and
> auto change to the distros's lib when distros add it.  For example QEMU's submodule SLIRP do it in the same way.

slirp used to be part of the QEMU repository, but then has been moved to a 
separate project a while ago. However, at that point in time there weren't 
any packages ins distros yet, so we had to include it as a submodule.

Now that the distros ship it, too, we're planning to remove the slirp 
submodule from QEMU soon, see:

  https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html

> It make user experience the latest technology
> with no other dependencies.

Well, that's only true if we update the submodule in QEMU regularly. If we 
forget to update, we could easily miss some important (maybe even security 
related) fixes from the upstream projects. This can be a nightmare for 
distros, when they then have to go around and look into each and every 
projects whether they embed a certain code module that needs a CVE fix. It's 
better if it can be fixed in one central spot instead.

> uBPF infrastructure have the ability to extend the capabilities without requiring
> changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler,
> interpreter, and JIT compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Not sure whether I understood that statement right ... nobody said that QEMU 
should not allow it - we just suggested to rely on a system installation of 
ubpf instead of embedding the code. Or is that not possible?? (I don't know 
that project yet - isn't it possible to compile it as a shared library?)

  Thomas

Re: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case

[Thomas Huth]

On 20/06/2022 11.31, Zhang, Chen wrote:
> 
>> -----Original Message-----
>> From: Thomas Huth <thuth@redhat.com>
>> Sent: Friday, June 17, 2022 5:34 PM
>> To: Zhang, Chen <chen.zhang@intel.com>; Jason Wang
>> <jasowang@redhat.com>; qemu-dev <qemu-devel@nongnu.org>; Paolo
>> Bonzini <pbonzini@redhat.com>; Daniel P. Berrangé
>> <berrange@redhat.com>; Eduardo Habkost <eduardo@habkost.net>; Eric
>> Blake <eblake@redhat.com>; Markus Armbruster <armbru@redhat.com>
>> Cc: Peter Maydell <peter.maydell@linaro.org>; Laurent Vivier
>> <lvivier@redhat.com>; Yuri Benditovich <yuri.benditovich@daynix.com>;
>> Andrew Melnychenko <andrew@daynix.com>
>> Subject: Re: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case
>>
>> On 17/06/2022 09.36, Zhang Chen wrote:
>>> TODO: This test case does not work. Need add ubpf.h header in qtest
>>> compile "-I ../ubpf/vm -I ../ubpf/vm/inc".
>>> I'm not sure if we need it in qtest. Because normal tests/qtest not
>>> including external module test case like fdt. Or we just need a qtest
>>> case for filter-ubpf module.
>>> This test will load pre-compiled ebpf binary and run it in QEMU.
>>>
>>> Signed-off-by: Zhang Chen <chen.zhang@intel.com>
>>> ---
[...]
>> Apart from the #include "libqtest.h" there is nothing related to qtest in
>> here ... should this maybe rather go into test/unit/ instead?
> 
> Rethink about it, I think you are right.
> The only issue is can we involve submodule's header file in tests/unit?
> I can't find meson/fdt/SLIRP test cases in the tests.

Well, the test should only be enabled in meson.build if the CONFIG_UBPF 
switch is also enabled, then you can be sure that the header files of ubpf 
are also available when it gets compiled.

  Thomas

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Daniel P. Berrangé]

On Mon, Jun 20, 2022 at 09:29:05AM +0000, Zhang, Chen wrote:
> 
> > 
> > On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> > > On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> > >>
> > >>
> > >>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> > >>>> Make iovisor/ubpf project be a git submodule for QEMU.
> > >>>> It will auto clone ubpf project when configure QEMU.
> > >>>
> > >>> I don't think we need todo this. As it is brand new functionality we
> > >>> don't have any back compat issues. We should just expect the distros
> > >>> to ship ubpf if they want their QEMU builds to take advantage of it.
> > >>>
> > >>
> > >> Yes, agree. It's the best way to use the uBPF project.
> > >> But current status is distros(ubuntu, RHEL...) does not ship the
> > >> iovisor/ubpf like the iovisor/bcc. So I have to do it.
> > >> Or do you have any better suggestions?
> > >
> > > If distros want to support the functionality, they can add packages
> > > for it IMHO.
> > 
> > Yes, let's please avoid new submodules. Submodules can sometimes be a
> > real PITA (e.g. if you forget to update before rsync'ing your code to a
> > machine that has limited internet access), and if users install QEMU from
> > sources, they can also install ubpf from sources, too.
> > And if distros want to support this feature, they can package ubpf on their
> > own, as Daniel said.
> 
> Hi Daniel and Thomas,
> 
> I don't know much the background history of QEMU submodules, but meson build
> is a submodule for QEMU too. It means user can't install QEMU from sources
> with limited internet access. 
> And back to Daniel's comments,  Yes, the best way is distros add the ubpf packages,
> But maybe it's too late to implement new features for us. We can introduce the submodule now and
> auto change to the distros's lib when distros add it.  For example QEMU's submodule SLIRP do it in the same way.
> It's already added by most distros and still as a QEMU submodule. It make user experience the latest technology
> with no other dependencies. uBPF infrastructure have the ability to extend the capabilities without requiring
> changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler,
> interpreter, and JIT compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Slirp is a different scenario. That was functionality that was historically
integrated into QEMU and was then spun out into a standalone project. Since
we had existing users on existing distro releases dependant on Slirp, we wanted
to give a smooth upgrade experiance by bundling Slirp. Essentially the goal was
to avoid the regression if someone deployed new QEMU on existing distros.

Meson is a fairly similar scenario. We wanted to swap the build system in
QEMU over to Meson, and that change would affect all existing users of QEMU.
Many distros didn't have a new enough meson, and so bundling it in QEMU enables
us to give a smooth upgrade path without any regression for existing users
on existing distros.

This patch, however, is proposing an entirely new piece of functionality
that has no existing users and even once present will be used by relatively
few users compartively speaking. As such there is no upgrade compatibility /
regression scenario that we need to worry about. Anyone interested in the
new functionality can be reasonably asked to either wait for the distro to
package it, or build it themselves. 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Zhang, Chen]

> -----Original Message-----
> From: Daniel P. Berrangé <berrange@redhat.com>
> Sent: Monday, June 20, 2022 6:01 PM
> To: Zhang, Chen <chen.zhang@intel.com>
> Cc: Thomas Huth <thuth@redhat.com>; Jason Wang
> <jasowang@redhat.com>; qemu-dev <qemu-devel@nongnu.org>; Paolo
> Bonzini <pbonzini@redhat.com>; Eduardo Habkost <eduardo@habkost.net>;
> Eric Blake <eblake@redhat.com>; Markus Armbruster
> <armbru@redhat.com>; Peter Maydell <peter.maydell@linaro.org>; Laurent
> Vivier <lvivier@redhat.com>; Yuri Benditovich
> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> <andrew@daynix.com>
> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> submodule for QEMU
> 
> On Mon, Jun 20, 2022 at 09:29:05AM +0000, Zhang, Chen wrote:
> >
> > >
> > > On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> > > > On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> > > >>
> > > >>
> > > >>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> > > >>>> Make iovisor/ubpf project be a git submodule for QEMU.
> > > >>>> It will auto clone ubpf project when configure QEMU.
> > > >>>
> > > >>> I don't think we need todo this. As it is brand new
> > > >>> functionality we don't have any back compat issues. We should
> > > >>> just expect the distros to ship ubpf if they want their QEMU builds to
> take advantage of it.
> > > >>>
> > > >>
> > > >> Yes, agree. It's the best way to use the uBPF project.
> > > >> But current status is distros(ubuntu, RHEL...) does not ship the
> > > >> iovisor/ubpf like the iovisor/bcc. So I have to do it.
> > > >> Or do you have any better suggestions?
> > > >
> > > > If distros want to support the functionality, they can add
> > > > packages for it IMHO.
> > >
> > > Yes, let's please avoid new submodules. Submodules can sometimes be
> > > a real PITA (e.g. if you forget to update before rsync'ing your code
> > > to a machine that has limited internet access), and if users install
> > > QEMU from sources, they can also install ubpf from sources, too.
> > > And if distros want to support this feature, they can package ubpf
> > > on their own, as Daniel said.
> >
> > Hi Daniel and Thomas,
> >
> > I don't know much the background history of QEMU submodules, but
> meson
> > build is a submodule for QEMU too. It means user can't install QEMU
> > from sources with limited internet access.
> > And back to Daniel's comments,  Yes, the best way is distros add the
> > ubpf packages, But maybe it's too late to implement new features for
> > us. We can introduce the submodule now and auto change to the distros's
> lib when distros add it.  For example QEMU's submodule SLIRP do it in the
> same way.
> > It's already added by most distros and still as a QEMU submodule. It
> > make user experience the latest technology with no other dependencies.
> > uBPF infrastructure have the ability to extend the capabilities
> > without requiring changing source code. If we not allow it, we have to re-
> implement all the eBPF assembler, disassembler, interpreter, and JIT
> compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by
> license issue).
> 
> Slirp is a different scenario. That was functionality that was historically
> integrated into QEMU and was then spun out into a standalone project. Since
> we had existing users on existing distro releases dependant on Slirp, we
> wanted to give a smooth upgrade experiance by bundling Slirp. Essentially
> the goal was to avoid the regression if someone deployed new QEMU on
> existing distros.
> 
> Meson is a fairly similar scenario. We wanted to swap the build system in
> QEMU over to Meson, and that change would affect all existing users of
> QEMU.
> Many distros didn't have a new enough meson, and so bundling it in QEMU
> enables us to give a smooth upgrade path without any regression for existing
> users on existing distros.
> 
> This patch, however, is proposing an entirely new piece of functionality that
> has no existing users and even once present will be used by relatively few
> users compartively speaking. As such there is no upgrade compatibility /
> regression scenario that we need to worry about. Anyone interested in the
> new functionality can be reasonably asked to either wait for the distro to
> package it, or build it themselves.
> 

OK, got your point.
For this series, should we introduce an external library "libubpf" in QEMU configure?
If this library is found, the relevant files will be compiled and the feature can be enabled in QEMU.

Thanks
Chen

> With regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Zhang, Chen]

> -----Original Message-----
> From: Thomas Huth <thuth@redhat.com>
> Sent: Monday, June 20, 2022 5:44 PM
> To: Zhang, Chen <chen.zhang@intel.com>; Daniel P. Berrangé
> <berrange@redhat.com>
> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>; Peter Maydell
> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> <andrew@daynix.com>
> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> submodule for QEMU
> 
> On 20/06/2022 11.29, Zhang, Chen wrote:
> >
> >
> >> -----Original Message-----
> >> From: Thomas Huth <thuth@redhat.com>
> >> Sent: Monday, June 20, 2022 4:47 PM
> >> To: Daniel P. Berrangé <berrange@redhat.com>; Zhang, Chen
> >> <chen.zhang@intel.com>
> >> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> >> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> >> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> Markus
> >> Armbruster <armbru@redhat.com>; Peter Maydell
> >> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> >> Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> >> <andrew@daynix.com>
> >> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as
> >> a submodule for QEMU
> >>
> >> On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> >>> On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Daniel P. Berrangé <berrange@redhat.com>
> >>>>> Sent: Friday, June 17, 2022 4:05 PM
> >>>>> To: Zhang, Chen <chen.zhang@intel.com>
> >>>>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> >>>>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> >>>>> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> >>>>> Markus Armbruster <armbru@redhat.com>; Peter Maydell
> >>>>> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>;
> >> Laurent
> >>>>> Vivier <lvivier@redhat.com>; Yuri Benditovich
> >>>>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> >>>>> <andrew@daynix.com>
> >>>>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project
> >>>>> as a submodule for QEMU
> >>>>>
> >>>>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> >>>>>> Make iovisor/ubpf project be a git submodule for QEMU.
> >>>>>> It will auto clone ubpf project when configure QEMU.
> >>>>>
> >>>>> I don't think we need todo this. As it is brand new functionality
> >>>>> we don't have any back compat issues. We should just expect the
> >>>>> distros to ship ubpf if they want their QEMU builds to take advantage
> of it.
> >>>>>
> >>>>
> >>>> Yes, agree. It's the best way to use the uBPF project.
> >>>> But current status is distros(ubuntu, RHEL...) does not ship the
> >>>> iovisor/ubpf like the iovisor/bcc. So I have to do it.
> >>>> Or do you have any better suggestions?
> >>>
> >>> If distros want to support the functionality, they can add packages
> >>> for it IMHO.
> >>
> >> Yes, let's please avoid new submodules. Submodules can sometimes be a
> >> real PITA (e.g. if you forget to update before rsync'ing your code to
> >> a machine that has limited internet access), and if users install
> >> QEMU from sources, they can also install ubpf from sources, too.
> >> And if distros want to support this feature, they can package ubpf on
> >> their own, as Daniel said.
> >
> > Hi Daniel and Thomas,
> >
> > I don't know much the background history of QEMU submodules, but
> meson
> > build is a submodule for QEMU too. It means user can't install QEMU
> > from sources with limited internet access.
> 
> There is no written policy, but I think the general consensus is that we only
> ship code in submodules if:
> 
> 1) It's not available in a required version in distros yet
> 
> and
> 
> 2) it is essentially required to build QEMU (like meson) or if the feature has
> been part of the QEMU sources before and then moved to a separate
> repository (like slirp).
> 
> We ship meson as a submodule since we require some meson features that
> are not available with the meson versions in the distros yet. Once the distros
> catch up, we'll likely remove the meson submodule from QEMU.
> 
> > And back to Daniel's comments,  Yes, the best way is distros add the
> > ubpf packages, But maybe it's too late to implement new features for
> > us. We can introduce the submodule now and auto change to the distros's
> lib when distros add it.  For example QEMU's submodule SLIRP do it in the
> same way.
> 
> slirp used to be part of the QEMU repository, but then has been moved to a
> separate project a while ago. However, at that point in time there weren't
> any packages ins distros yet, so we had to include it as a submodule.
> 
> Now that the distros ship it, too, we're planning to remove the slirp
> submodule from QEMU soon, see:
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html
> 
> > It make user experience the latest technology with no other
> > dependencies.
> 
> Well, that's only true if we update the submodule in QEMU regularly. If we
> forget to update, we could easily miss some important (maybe even security
> related) fixes from the upstream projects. This can be a nightmare for distros,
> when they then have to go around and look into each and every projects
> whether they embed a certain code module that needs a CVE fix. It's better
> if it can be fixed in one central spot instead.
> 
> > uBPF infrastructure have the ability to extend the capabilities
> > without requiring changing source code. If we not allow it, we have to
> > re-implement all the eBPF assembler, disassembler, interpreter, and JIT
> compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by
> license issue).
> 
> Not sure whether I understood that statement right ... nobody said that
> QEMU should not allow it - we just suggested to rely on a system installation
> of ubpf instead of embedding the code. Or is that not possible?? (I don't
> know that project yet - isn't it possible to compile it as a shared library?)

Thanks for your details explanation.
It looks better to introduce the uBPF shared library for QEMU.
For example:
./configure --ubpf-lib=path

If yes, I think it's fine for me.

Thanks
Chen

> 
>   Thomas

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Daniel P. Berrangé]

On Mon, Jun 20, 2022 at 10:29:14AM +0000, Zhang, Chen wrote:
> 
> 
> > -----Original Message-----
> > From: Thomas Huth <thuth@redhat.com>
> > Sent: Monday, June 20, 2022 5:44 PM
> > To: Zhang, Chen <chen.zhang@intel.com>; Daniel P. Berrangé
> > <berrange@redhat.com>
> > Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
> > Armbruster <armbru@redhat.com>; Peter Maydell
> > <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> > Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > <andrew@daynix.com>
> > Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> > submodule for QEMU
> > 
> > On 20/06/2022 11.29, Zhang, Chen wrote:
> > >
> > >
> > >> -----Original Message-----
> > >> From: Thomas Huth <thuth@redhat.com>
> > >> Sent: Monday, June 20, 2022 4:47 PM
> > >> To: Daniel P. Berrangé <berrange@redhat.com>; Zhang, Chen
> > >> <chen.zhang@intel.com>
> > >> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > >> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > >> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> > Markus
> > >> Armbruster <armbru@redhat.com>; Peter Maydell
> > >> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>; Yuri
> > >> Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > >> <andrew@daynix.com>
> > >> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as
> > >> a submodule for QEMU
> > >>
> > >> On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> > >>> On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> > >>>>
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: Daniel P. Berrangé <berrange@redhat.com>
> > >>>>> Sent: Friday, June 17, 2022 4:05 PM
> > >>>>> To: Zhang, Chen <chen.zhang@intel.com>
> > >>>>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > >>>>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > >>>>> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> > >>>>> Markus Armbruster <armbru@redhat.com>; Peter Maydell
> > >>>>> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>;
> > >> Laurent
> > >>>>> Vivier <lvivier@redhat.com>; Yuri Benditovich
> > >>>>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > >>>>> <andrew@daynix.com>
> > >>>>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project
> > >>>>> as a submodule for QEMU
> > >>>>>
> > >>>>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> > >>>>>> Make iovisor/ubpf project be a git submodule for QEMU.
> > >>>>>> It will auto clone ubpf project when configure QEMU.
> > >>>>>
> > >>>>> I don't think we need todo this. As it is brand new functionality
> > >>>>> we don't have any back compat issues. We should just expect the
> > >>>>> distros to ship ubpf if they want their QEMU builds to take advantage
> > of it.
> > >>>>>
> > >>>>
> > >>>> Yes, agree. It's the best way to use the uBPF project.
> > >>>> But current status is distros(ubuntu, RHEL...) does not ship the
> > >>>> iovisor/ubpf like the iovisor/bcc. So I have to do it.
> > >>>> Or do you have any better suggestions?
> > >>>
> > >>> If distros want to support the functionality, they can add packages
> > >>> for it IMHO.
> > >>
> > >> Yes, let's please avoid new submodules. Submodules can sometimes be a
> > >> real PITA (e.g. if you forget to update before rsync'ing your code to
> > >> a machine that has limited internet access), and if users install
> > >> QEMU from sources, they can also install ubpf from sources, too.
> > >> And if distros want to support this feature, they can package ubpf on
> > >> their own, as Daniel said.
> > >
> > > Hi Daniel and Thomas,
> > >
> > > I don't know much the background history of QEMU submodules, but
> > meson
> > > build is a submodule for QEMU too. It means user can't install QEMU
> > > from sources with limited internet access.
> > 
> > There is no written policy, but I think the general consensus is that we only
> > ship code in submodules if:
> > 
> > 1) It's not available in a required version in distros yet
> > 
> > and
> > 
> > 2) it is essentially required to build QEMU (like meson) or if the feature has
> > been part of the QEMU sources before and then moved to a separate
> > repository (like slirp).
> > 
> > We ship meson as a submodule since we require some meson features that
> > are not available with the meson versions in the distros yet. Once the distros
> > catch up, we'll likely remove the meson submodule from QEMU.
> > 
> > > And back to Daniel's comments,  Yes, the best way is distros add the
> > > ubpf packages, But maybe it's too late to implement new features for
> > > us. We can introduce the submodule now and auto change to the distros's
> > lib when distros add it.  For example QEMU's submodule SLIRP do it in the
> > same way.
> > 
> > slirp used to be part of the QEMU repository, but then has been moved to a
> > separate project a while ago. However, at that point in time there weren't
> > any packages ins distros yet, so we had to include it as a submodule.
> > 
> > Now that the distros ship it, too, we're planning to remove the slirp
> > submodule from QEMU soon, see:
> > 
> >   https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html
> > 
> > > It make user experience the latest technology with no other
> > > dependencies.
> > 
> > Well, that's only true if we update the submodule in QEMU regularly. If we
> > forget to update, we could easily miss some important (maybe even security
> > related) fixes from the upstream projects. This can be a nightmare for distros,
> > when they then have to go around and look into each and every projects
> > whether they embed a certain code module that needs a CVE fix. It's better
> > if it can be fixed in one central spot instead.
> > 
> > > uBPF infrastructure have the ability to extend the capabilities
> > > without requiring changing source code. If we not allow it, we have to
> > > re-implement all the eBPF assembler, disassembler, interpreter, and JIT
> > compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by
> > license issue).
> > 
> > Not sure whether I understood that statement right ... nobody said that
> > QEMU should not allow it - we just suggested to rely on a system installation
> > of ubpf instead of embedding the code. Or is that not possible?? (I don't
> > know that project yet - isn't it possible to compile it as a shared library?)
> 
> Thanks for your details explanation.
> It looks better to introduce the uBPF shared library for QEMU.
> For example:
> ./configure --ubpf-lib=path

I've not looked, so maybe it already does this, but ideally  'uBPF'
would ship a 'pkg-config'  file, so that apps can automatically find
it and set the right cflags/libs etc for the compiler. For configure
integration, normally we'd expect it to be --enable-ubpf/--disable-ubpf,
with it automatically enabling itself if the pkg-config file is found.
Take a look at handling of some existing libraries we depend on for
examples.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

[Zhang, Chen]

> -----Original Message-----
> From: Daniel P. Berrangé <berrange@redhat.com>
> Sent: Monday, June 20, 2022 6:37 PM
> To: Zhang, Chen <chen.zhang@intel.com>
> Cc: Thomas Huth <thuth@redhat.com>; Jason Wang
> <jasowang@redhat.com>; qemu-dev <qemu-devel@nongnu.org>; Paolo
> Bonzini <pbonzini@redhat.com>; Eduardo Habkost <eduardo@habkost.net>;
> Eric Blake <eblake@redhat.com>; Markus Armbruster
> <armbru@redhat.com>; Peter Maydell <peter.maydell@linaro.org>; Laurent
> Vivier <lvivier@redhat.com>; Yuri Benditovich
> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> <andrew@daynix.com>
> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a
> submodule for QEMU
> 
> On Mon, Jun 20, 2022 at 10:29:14AM +0000, Zhang, Chen wrote:
> >
> >
> > > -----Original Message-----
> > > From: Thomas Huth <thuth@redhat.com>
> > > Sent: Monday, June 20, 2022 5:44 PM
> > > To: Zhang, Chen <chen.zhang@intel.com>; Daniel P. Berrangé
> > > <berrange@redhat.com>
> > > Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > > devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > > Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> > > Markus Armbruster <armbru@redhat.com>; Peter Maydell
> > > <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>;
> > > Yuri Benditovich <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > > <andrew@daynix.com>
> > > Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project
> > > as a submodule for QEMU
> > >
> > > On 20/06/2022 11.29, Zhang, Chen wrote:
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Thomas Huth <thuth@redhat.com>
> > > >> Sent: Monday, June 20, 2022 4:47 PM
> > > >> To: Daniel P. Berrangé <berrange@redhat.com>; Zhang, Chen
> > > >> <chen.zhang@intel.com>
> > > >> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > > >> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Eduardo
> > > >> Habkost <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>;
> > > Markus
> > > >> Armbruster <armbru@redhat.com>; Peter Maydell
> > > >> <peter.maydell@linaro.org>; Laurent Vivier <lvivier@redhat.com>;
> > > >> Yuri Benditovich <yuri.benditovich@daynix.com>; Andrew
> > > >> Melnychenko <andrew@daynix.com>
> > > >> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf
> > > >> project as a submodule for QEMU
> > > >>
> > > >> On 20/06/2022 10.11, Daniel P. Berrangé wrote:
> > > >>> On Mon, Jun 20, 2022 at 05:59:06AM +0000, Zhang, Chen wrote:
> > > >>>>
> > > >>>>
> > > >>>>> -----Original Message-----
> > > >>>>> From: Daniel P. Berrangé <berrange@redhat.com>
> > > >>>>> Sent: Friday, June 17, 2022 4:05 PM
> > > >>>>> To: Zhang, Chen <chen.zhang@intel.com>
> > > >>>>> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> > > >>>>> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>;
> > > >>>>> Eduardo Habkost <eduardo@habkost.net>; Eric Blake
> > > >>>>> <eblake@redhat.com>; Markus Armbruster
> <armbru@redhat.com>;
> > > >>>>> Peter Maydell <peter.maydell@linaro.org>; Thomas Huth
> > > >>>>> <thuth@redhat.com>;
> > > >> Laurent
> > > >>>>> Vivier <lvivier@redhat.com>; Yuri Benditovich
> > > >>>>> <yuri.benditovich@daynix.com>; Andrew Melnychenko
> > > >>>>> <andrew@daynix.com>
> > > >>>>> Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf
> > > >>>>> project as a submodule for QEMU
> > > >>>>>
> > > >>>>> On Fri, Jun 17, 2022 at 03:36:19PM +0800, Zhang Chen wrote:
> > > >>>>>> Make iovisor/ubpf project be a git submodule for QEMU.
> > > >>>>>> It will auto clone ubpf project when configure QEMU.
> > > >>>>>
> > > >>>>> I don't think we need todo this. As it is brand new
> > > >>>>> functionality we don't have any back compat issues. We should
> > > >>>>> just expect the distros to ship ubpf if they want their QEMU
> > > >>>>> builds to take advantage
> > > of it.
> > > >>>>>
> > > >>>>
> > > >>>> Yes, agree. It's the best way to use the uBPF project.
> > > >>>> But current status is distros(ubuntu, RHEL...) does not ship
> > > >>>> the iovisor/ubpf like the iovisor/bcc. So I have to do it.
> > > >>>> Or do you have any better suggestions?
> > > >>>
> > > >>> If distros want to support the functionality, they can add
> > > >>> packages for it IMHO.
> > > >>
> > > >> Yes, let's please avoid new submodules. Submodules can sometimes
> > > >> be a real PITA (e.g. if you forget to update before rsync'ing
> > > >> your code to a machine that has limited internet access), and if
> > > >> users install QEMU from sources, they can also install ubpf from
> sources, too.
> > > >> And if distros want to support this feature, they can package
> > > >> ubpf on their own, as Daniel said.
> > > >
> > > > Hi Daniel and Thomas,
> > > >
> > > > I don't know much the background history of QEMU submodules, but
> > > meson
> > > > build is a submodule for QEMU too. It means user can't install
> > > > QEMU from sources with limited internet access.
> > >
> > > There is no written policy, but I think the general consensus is
> > > that we only ship code in submodules if:
> > >
> > > 1) It's not available in a required version in distros yet
> > >
> > > and
> > >
> > > 2) it is essentially required to build QEMU (like meson) or if the
> > > feature has been part of the QEMU sources before and then moved to a
> > > separate repository (like slirp).
> > >
> > > We ship meson as a submodule since we require some meson features
> > > that are not available with the meson versions in the distros yet.
> > > Once the distros catch up, we'll likely remove the meson submodule from
> QEMU.
> > >
> > > > And back to Daniel's comments,  Yes, the best way is distros add
> > > > the ubpf packages, But maybe it's too late to implement new
> > > > features for us. We can introduce the submodule now and auto
> > > > change to the distros's
> > > lib when distros add it.  For example QEMU's submodule SLIRP do it
> > > in the same way.
> > >
> > > slirp used to be part of the QEMU repository, but then has been
> > > moved to a separate project a while ago. However, at that point in
> > > time there weren't any packages ins distros yet, so we had to include it as
> a submodule.
> > >
> > > Now that the distros ship it, too, we're planning to remove the
> > > slirp submodule from QEMU soon, see:
> > >
> > >
> > > https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html
> > >
> > > > It make user experience the latest technology with no other
> > > > dependencies.
> > >
> > > Well, that's only true if we update the submodule in QEMU regularly.
> > > If we forget to update, we could easily miss some important (maybe
> > > even security
> > > related) fixes from the upstream projects. This can be a nightmare
> > > for distros, when they then have to go around and look into each and
> > > every projects whether they embed a certain code module that needs a
> > > CVE fix. It's better if it can be fixed in one central spot instead.
> > >
> > > > uBPF infrastructure have the ability to extend the capabilities
> > > > without requiring changing source code. If we not allow it, we
> > > > have to re-implement all the eBPF assembler, disassembler,
> > > > interpreter, and JIT
> > > compiler like DPDK userspace eBPF support (DPDK can't use ubpf
> > > project by license issue).
> > >
> > > Not sure whether I understood that statement right ... nobody said
> > > that QEMU should not allow it - we just suggested to rely on a
> > > system installation of ubpf instead of embedding the code. Or is
> > > that not possible?? (I don't know that project yet - isn't it
> > > possible to compile it as a shared library?)
> >
> > Thanks for your details explanation.
> > It looks better to introduce the uBPF shared library for QEMU.
> > For example:
> > ./configure --ubpf-lib=path
> 
> I've not looked, so maybe it already does this, but ideally  'uBPF'
> would ship a 'pkg-config'  file, so that apps can automatically find it and set
> the right cflags/libs etc for the compiler. For configure integration, normally
> we'd expect it to be --enable-ubpf/--disable-ubpf, with it automatically
> enabling itself if the pkg-config file is found.
> Take a look at handling of some existing libraries we depend on for examples.
> 

OK, thanks your comments. I reviewed uBPF code, but no pkg-config file.
The make install just copy the libubpf.so/.a to /usr/lib.
Fortunately, writing this file looks easy. I will try to do it and make QEMU use the libubpf.so like "pixman".

Thanks
Chen
 

> With regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|

Re: [RFC PATCH 00/12] Introduce QEMU userspace ebpf support

[Andrew Melnichenko]

Hi all,
Nice idea.
It would be great if future patches would add the BPF map support(if
uBPF allows it).

On Fri, Jun 17, 2022 at 10:51 AM Zhang Chen <chen.zhang@intel.com> wrote:
>
> Hi All,
>
>     The goal of this series is to bring the power of ebpf to QEMU.
> It makes QEMU have the ability to extend the capabilities without
> requiring changing source code. Just need to load the eBPF binary
> file even at VM runtime. And already have some userspace ebpf
> implementation like: Intel DPDK eBPF, windows eBPF, etc..
> The original idea suggested by Jason Wang.
>
>     eBPF is a revolutionary technology with origins in the Linux kernel
> that can run sandboxed programs in an operating system kernel. It is
> used to safely and efficiently extend the capabilities of the kernel
> without requiring to change kernel source code or load kernel
> modules.(from https://ebpf.io/)
>
>     KVM already got benefits from it, but QEMU did not. Hence we want
> to bring the power of eBPF to QEMU. It can load binary eBPF program
> even when VM running. At the same time, add some hooks in QEMU as
> the user space eBPF load point. Do the things on different layers.
>
>    That’s the advantages of kernel eBPF. Most of the functions can be
> implemented in QEMU. This series just a start of the Power of Programmability.
>
>     1). Safety:
>
>     Building on the foundation of seeing and understanding all system
>     calls and combining that with a packet and socket-level view of all
>     networking operations allows for revolutionary new approaches to
>     securing systems.
>
>     2). Tracing & Profiling:
>
>     The ability to attach eBPF programs to trace points as well as kernel
>     and user application probe points allows unprecedented visibility into
>     the runtime behavior of applications and the system itself.
>
>     3). Networking:
>
>     The combination of programmability and efficiency makes eBPF a natural
>     fit for all packet processing requirements of networking solutions.
>
>     4). Observability & Monitoring:
>
>     Instead of relying on static counters and gauges exposed by the
>     perating system, eBPF enables the collection & in-kernel aggregation
>     of custom metrics and generation of visibility events based on a wide
>     range of possible sources.
>
>     QEMU userspace ebpf design based on ubpf project (https://github.com/iovisor/ubpf).
> The most mature userspace ebpf implementation. This project officially
> support by iovisor(Like BCC and bpftrace). This project includes an eBPF
> assembler, disassembler, interpreter (for all platforms), and JIT compiler
> (for x86-64 and Arm64 targets). Qemu userspace ebpf make the ubpf project
> as the git submodule.
>
>     Current implementation support load ebpf program and run it in
> net/filter-ubpf module, this filter can support any user defined rules
> to hanle network packet. At the same time, it's easy for other developers
> to use the ubpf infrastructue in QEMU's other modules from the function
> in /ebpf/ubpf.c, and it support JIT.
>
>     For the uBPF License is Apache License 2.0, It's OK to compatible
> with QEMU’s GPLv2 LICENSE same as mason.
>
>     TODO: Need to add more comments and test-case for ubpf, current
> implementation not include ebpf verifier. But I think maybe it's not
> a big problem, current ebpf load/unload API exposed by QMP command.
> Qemu is a userspace program, if someone want to hack QEMU, no need to
> load a malicious ubpf program, it can hack QEMU code or crash QEMU on
> host directly(different from kernel ebpf needs strict inspection, but
> yes, it still need basic check).
>
> Any comments are welcome.
>
> Thanks
> Chen
>
>
> Zhang Chen (12):
>   configure: Add iovisor/ubpf project as a submodule for QEMU
>   meson: Add ubpf build config and misc
>   ebpf/uBPF: Introduce userspace ebpf data structure
>   ebpf/uBPF: Introduce ubpf initialize functions
>   ebpf/uBPF: Add qemu_prepare_ubpf to load ebpf binary
>   ebpf/uBPF: Add qemu_ubpf_run_once excute real ebpf program
>   net/filter: Introduce filter-ubpf module
>   qapi: Add FilterUbpfProperties and qemu-options
>   softmmu/vl.c: Add filter-ubpf for netdev as other netfilters
>   net/filter-ubpf.c: run the ubpf program to handle network packet
>   docs/devel: Add userspace-ebpf.rst
>   test/qtest: Add ubpf basic test case
>
>  .gitmodules                         |   3 +
>  configure                           |  20 +++
>  docs/devel/userspace-ebpf.rst       | 106 ++++++++++++++
>  ebpf/meson.build                    |   1 +
>  ebpf/ubpf-stub.c                    |  35 +++++
>  ebpf/ubpf.c                         | 217 ++++++++++++++++++++++++++++
>  ebpf/ubpf.h                         |  44 ++++++
>  meson.build                         |  47 ++++++
>  meson_options.txt                   |   3 +
>  net/filter-ubpf.c                   | 185 ++++++++++++++++++++++++
>  net/meson.build                     |   1 +
>  qapi/qom.json                       |  18 +++
>  qemu-options.hx                     |   6 +
>  scripts/coverity-scan/COMPONENTS.md |   3 +
>  scripts/meson-buildoptions.sh       |   5 +
>  softmmu/vl.c                        |   3 +-
>  tests/qtest/demo_ubpf.o             | Bin 0 -> 544 bytes
>  tests/qtest/integer_5.mem           | Bin 0 -> 4 bytes
>  tests/qtest/meson.build             |   3 +-
>  tests/qtest/ubpf-test.c             |  64 ++++++++
>  ubpf                                |   1 +
>  21 files changed, 763 insertions(+), 2 deletions(-)
>  create mode 100644 docs/devel/userspace-ebpf.rst
>  create mode 100644 ebpf/ubpf-stub.c
>  create mode 100644 ebpf/ubpf.c
>  create mode 100644 ebpf/ubpf.h
>  create mode 100644 net/filter-ubpf.c
>  create mode 100644 tests/qtest/demo_ubpf.o
>  create mode 100644 tests/qtest/integer_5.mem
>  create mode 100644 tests/qtest/ubpf-test.c
>  create mode 160000 ubpf
>
> --
> 2.25.1
>

RE: [RFC PATCH 00/12] Introduce QEMU userspace ebpf support

[Zhang, Chen]

> -----Original Message-----
> From: Andrew Melnichenko <andrew@daynix.com>
> Sent: Wednesday, June 29, 2022 6:43 PM
> To: Zhang, Chen <chen.zhang@intel.com>
> Cc: Jason Wang <jasowang@redhat.com>; qemu-dev <qemu-
> devel@nongnu.org>; Paolo Bonzini <pbonzini@redhat.com>; Daniel P.
> Berrangé <berrange@redhat.com>; Eduardo Habkost
> <eduardo@habkost.net>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>; Peter Maydell
> <peter.maydell@linaro.org>; Thomas Huth <thuth@redhat.com>; Laurent
> Vivier <lvivier@redhat.com>; Yuri Benditovich
> <yuri.benditovich@daynix.com>
> Subject: Re: [RFC PATCH 00/12] Introduce QEMU userspace ebpf support
> 
> Hi all,
> Nice idea.
> It would be great if future patches would add the BPF map support(if uBPF
> allows it).

The BPF map support is very useful.  But current uBPF project don't support this yet.
According to the previous discussion with Thomas and Daniel, we should avoid
Introduce new git submodule for QEMU. And related general discussion:
Why we should avoid new submodules if possible:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg897339.html

I think for the future development, we have to submit patch to the uBPF project.

Thanks
Chen

> 
> On Fri, Jun 17, 2022 at 10:51 AM Zhang Chen <chen.zhang@intel.com> wrote:
> >
> > Hi All,
> >
> >     The goal of this series is to bring the power of ebpf to QEMU.
> > It makes QEMU have the ability to extend the capabilities without
> > requiring changing source code. Just need to load the eBPF binary file
> > even at VM runtime. And already have some userspace ebpf
> > implementation like: Intel DPDK eBPF, windows eBPF, etc..
> > The original idea suggested by Jason Wang.
> >
> >     eBPF is a revolutionary technology with origins in the Linux
> > kernel that can run sandboxed programs in an operating system kernel.
> > It is used to safely and efficiently extend the capabilities of the
> > kernel without requiring to change kernel source code or load kernel
> > modules.(from https://ebpf.io/)
> >
> >     KVM already got benefits from it, but QEMU did not. Hence we want
> > to bring the power of eBPF to QEMU. It can load binary eBPF program
> > even when VM running. At the same time, add some hooks in QEMU as
> the
> > user space eBPF load point. Do the things on different layers.
> >
> >    That’s the advantages of kernel eBPF. Most of the functions can be
> > implemented in QEMU. This series just a start of the Power of
> Programmability.
> >
> >     1). Safety:
> >
> >     Building on the foundation of seeing and understanding all system
> >     calls and combining that with a packet and socket-level view of all
> >     networking operations allows for revolutionary new approaches to
> >     securing systems.
> >
> >     2). Tracing & Profiling:
> >
> >     The ability to attach eBPF programs to trace points as well as kernel
> >     and user application probe points allows unprecedented visibility into
> >     the runtime behavior of applications and the system itself.
> >
> >     3). Networking:
> >
> >     The combination of programmability and efficiency makes eBPF a natural
> >     fit for all packet processing requirements of networking solutions.
> >
> >     4). Observability & Monitoring:
> >
> >     Instead of relying on static counters and gauges exposed by the
> >     perating system, eBPF enables the collection & in-kernel aggregation
> >     of custom metrics and generation of visibility events based on a wide
> >     range of possible sources.
> >
> >     QEMU userspace ebpf design based on ubpf project
> (https://github.com/iovisor/ubpf).
> > The most mature userspace ebpf implementation. This project officially
> > support by iovisor(Like BCC and bpftrace). This project includes an
> > eBPF assembler, disassembler, interpreter (for all platforms), and JIT
> > compiler (for x86-64 and Arm64 targets). Qemu userspace ebpf make the
> > ubpf project as the git submodule.
> >
> >     Current implementation support load ebpf program and run it in
> > net/filter-ubpf module, this filter can support any user defined rules
> > to hanle network packet. At the same time, it's easy for other
> > developers to use the ubpf infrastructue in QEMU's other modules from
> > the function in /ebpf/ubpf.c, and it support JIT.
> >
> >     For the uBPF License is Apache License 2.0, It's OK to compatible
> > with QEMU’s GPLv2 LICENSE same as mason.
> >
> >     TODO: Need to add more comments and test-case for ubpf, current
> > implementation not include ebpf verifier. But I think maybe it's not a
> > big problem, current ebpf load/unload API exposed by QMP command.
> > Qemu is a userspace program, if someone want to hack QEMU, no need to
> > load a malicious ubpf program, it can hack QEMU code or crash QEMU on
> > host directly(different from kernel ebpf needs strict inspection, but
> > yes, it still need basic check).
> >
> > Any comments are welcome.
> >
> > Thanks
> > Chen
> >
> >
> > Zhang Chen (12):
> >   configure: Add iovisor/ubpf project as a submodule for QEMU
> >   meson: Add ubpf build config and misc
> >   ebpf/uBPF: Introduce userspace ebpf data structure
> >   ebpf/uBPF: Introduce ubpf initialize functions
> >   ebpf/uBPF: Add qemu_prepare_ubpf to load ebpf binary
> >   ebpf/uBPF: Add qemu_ubpf_run_once excute real ebpf program
> >   net/filter: Introduce filter-ubpf module
> >   qapi: Add FilterUbpfProperties and qemu-options
> >   softmmu/vl.c: Add filter-ubpf for netdev as other netfilters
> >   net/filter-ubpf.c: run the ubpf program to handle network packet
> >   docs/devel: Add userspace-ebpf.rst
> >   test/qtest: Add ubpf basic test case
> >
> >  .gitmodules                         |   3 +
> >  configure                           |  20 +++
> >  docs/devel/userspace-ebpf.rst       | 106 ++++++++++++++
> >  ebpf/meson.build                    |   1 +
> >  ebpf/ubpf-stub.c                    |  35 +++++
> >  ebpf/ubpf.c                         | 217 ++++++++++++++++++++++++++++
> >  ebpf/ubpf.h                         |  44 ++++++
> >  meson.build                         |  47 ++++++
> >  meson_options.txt                   |   3 +
> >  net/filter-ubpf.c                   | 185 ++++++++++++++++++++++++
> >  net/meson.build                     |   1 +
> >  qapi/qom.json                       |  18 +++
> >  qemu-options.hx                     |   6 +
> >  scripts/coverity-scan/COMPONENTS.md |   3 +
> >  scripts/meson-buildoptions.sh       |   5 +
> >  softmmu/vl.c                        |   3 +-
> >  tests/qtest/demo_ubpf.o             | Bin 0 -> 544 bytes
> >  tests/qtest/integer_5.mem           | Bin 0 -> 4 bytes
> >  tests/qtest/meson.build             |   3 +-
> >  tests/qtest/ubpf-test.c             |  64 ++++++++
> >  ubpf                                |   1 +
> >  21 files changed, 763 insertions(+), 2 deletions(-)  create mode
> > 100644 docs/devel/userspace-ebpf.rst  create mode 100644
> > ebpf/ubpf-stub.c  create mode 100644 ebpf/ubpf.c  create mode 100644
> > ebpf/ubpf.h  create mode 100644 net/filter-ubpf.c  create mode 100644
> > tests/qtest/demo_ubpf.o  create mode 100644
> tests/qtest/integer_5.mem
> > create mode 100644 tests/qtest/ubpf-test.c  create mode 160000 ubpf
> >
> > --
> > 2.25.1
> >