[RFC v3 00/32] Rust binding for QAPI and qemu-ga QMP handler examples

[marcandre.lureau@redhat.com]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Hi,

Among the QEMU developers, there is a desire to use Rust. (see previous
thread from Stefan "Why QEMU should move from C to Rust", the rust-vmm
related projects and other experiments).

Thanks to our QAPI type system and the associate code generator, it is possible
to create Rust bindings for the generated C types (also called FFI binding) and
functions (rust-bindgen could probably do a similar job, but it is unnecessary
given the FFI generator is quite straightforward).

More interesting (but more complicated) is to expose a Rust API, and provide
convenient conversions FFI<->Rust. Taking inspiration from glib-rs binding, I
implemented a simplified version of the FromGlib/ToGlib traits, with simpler
ownership model, sufficient for QAPI needs.

The usage is as such:

- from_qemu_none(ptr: *const sys::P) -> T
  Return a Rust type T for a const ffi pointer P.

- from_qemu_full(ptr: *mut sys::P) -> T
  Return a Rust type T for a ffi pointer P, taking ownership.

- T::to_qemu_none() -> Stash<P>
  Returns a borrowed ffi pointer P (using a Stash to destroy "glue"
  storage data, if any).

- T::to_qemu_full() -> P
  Returns a ffi pointer P. (P resources are leaked/passed to C/ffi)

With the Rust QAPI bindings, we can implement QMP callbacks in idiomatic Rust.
With the translation traits, we can also bind manually more types and implement
other functions (QOM is my next target).

Since Rust doesn't cover all the platforms QEMU supports (see first patch and
https://doc.rust-lang.org/nightly/rustc/platform-support.html), the Rust code
should be optional. Nevertheless, it is desirable to write sensitive code in
Rust first, and a C version if the code/feature is not optional and not specific
to a platform where Rust is Tier-1 or 2.

v3 (RFC):
 - rebased (was waiting for qapi 'if' conditions to be merged)
 - now covers the various QEMU QAPI schemas (QEMU schema is left out of this
   series, but test schema is the complete)
 - numerous improvements, patch reorgnisation, cleanups etc

v2 (PoC):
 - split the original patch in smaller patches and more digestable form
 - dropped the DBus interface experiment from this series
 - various build-sys improvements, new configure options --with-rust(-target)
 - various attempts at better meson integration, finally satisfied enough with the
   current solution, which handles getting link flags from Rust sanely.
   (more meson stuff to come to handle config-host/features mapping etc).
 - rebased, QGA QMP now uses unions, added support for it.
 - start a common crate (which re-surfaced issues with foreign types and traits,
   worked around with a NewPtr wrapper)
 - explicit errors when ifcond are presents (after various unsucessful attempts,
   I will try to tackle it in v3)
 - mingw cross compilation support
 - some attempts to add it to CI
 - actually implement {get,set}-vcpus
 - vendor the external crates

Marc-André Lureau (32):
  RFC: docs: add supported host CPUs section
  build-sys: add HAVE_IPPROTO_MPTCP
  scripts/qapi: teach c_param_type() to return const argument type
  glib-compat: add G_SIZEOF_MEMBER
  scripts/qapi: add QAPISchemaVisitor.visit_module_end
  scripts/qapi: add a CABI module
  scripts/qapi: generate CABI dump for C types
  tests: build qapi-cabi (C ABI dump)
  build-sys: add i686 cpu target
  build-sys: add --with-rust{-target} & basic build infrastructure
  build-sys: add a cargo-wrapper script
  rust: provide a common crate for QEMU
  rust: use vendored-sources
  scripts/qapi: add QAPISchemaIfCond.rsgen()
  scripts/qapi: strip trailing whitespaces
  scripts/qapi: add Rust FFI bindings generation
  scripts/qapi: learn to generate ABI dump for Rust FFI
  tests: generate Rust bindings
  tests: check Rust and C CABI diffs
  scripts/qapi: generate high-level Rust bindings
  tests/rust: build a common library, checking bindings compile
  qga: build qapi-cabi binary (ABI from C)
  qga/rust: build and link an empty static library
  qga/rust: generate QGA QAPI types FFI bindings
  qga/rust: build a qga-cabi-rs executable (ABI from Rust)
  qga/rust: check the Rust C binding
  qga/rust: build high-level Rust QAPI types
  qga/rust: implement get-host-name in Rust (example)
  qga/rust: implement {get,set}-vcpus in Rust (example)
  tests/vm: add Rust to FreeBSD VM
  tests/vm: bump fedora VM to f32
  tests/vm: add Rust to Fedora

 docs/about/build-platforms.rst |  28 +
 configure                      |  32 +-
 meson.build                    |  49 +-
 qapi/sockets.json              |   2 +-
 include/glib-compat.h          |   7 +
 include/qemu/osdep.h           |  10 -
 io/dns-resolver.c              |   2 +-
 qga/commands-posix.c           | 152 ------
 qga/commands-win32.c           |  71 ---
 qga/commands.c                 |  34 +-
 qga/qapi-cabi.c                |   9 +
 tests/qapi-cabi.c              |   9 +
 tests/unit/test-qga.c          |   4 +
 util/oslib-posix.c             |  35 --
 util/oslib-win32.c             |  13 -
 util/qemu-sockets.c            |   6 +-
 .cargo/config.toml.in          |   5 +
 .cargo/meson.build             |   5 +
 .gitignore                     |   1 +
 .gitmodules                    |   4 +
 Cargo.lock                     | 119 ++++
 Cargo.toml                     |   6 +
 meson_options.txt              |   5 +
 qga/Cargo.toml                 |  24 +
 qga/lib.rs                     |   5 +
 qga/meson.build                |  64 ++-
 qga/qapi-cabi.rs               |   6 +
 qga/qapi.rs                    |   6 +
 qga/qapi_ffi.rs                |   8 +
 qga/qmp/hostname.rs            |   9 +
 qga/qmp/mod.rs                 |  28 +
 qga/qmp/vcpus.rs               | 161 ++++++
 rust/common/Cargo.toml         |  11 +
 rust/common/src/error.rs       | 113 ++++
 rust/common/src/ffi.rs         |  93 ++++
 rust/common/src/lib.rs         |  21 +
 rust/common/src/qemu.rs        | 101 ++++
 rust/common/src/qmp.rs         |   0
 rust/common/src/translate.rs   | 482 ++++++++++++++++
 rust/vendored                  |   1 +
 scripts/archive-source.sh      |   2 +-
 scripts/cargo_wrapper.py       | 183 +++++++
 scripts/configh_to_cfg.py      |  44 ++
 scripts/diff_commands.py       |  40 ++
 scripts/qapi/cabi.py           | 187 +++++++
 scripts/qapi/common.py         |  16 +
 scripts/qapi/gen.py            |   6 +-
 scripts/qapi/main.py           |  25 +-
 scripts/qapi/rs.py             | 262 +++++++++
 scripts/qapi/rs_ffi.py         | 446 +++++++++++++++
 scripts/qapi/rs_types.py       | 966 +++++++++++++++++++++++++++++++++
 scripts/qapi/schema.py         |  23 +-
 scripts/qapi/types.py          |  58 +-
 tests/Cargo.toml               |  17 +
 tests/lib.rs                   |   2 +
 tests/meson.build              |  72 ++-
 tests/qapi-cabi.rs             |   5 +
 tests/qapi.rs                  |  11 +
 tests/qapi_ffi.rs              |   8 +
 tests/vm/fedora                |  12 +-
 tests/vm/freebsd               |   1 +
 61 files changed, 3794 insertions(+), 333 deletions(-)
 create mode 100644 qga/qapi-cabi.c
 create mode 100644 tests/qapi-cabi.c
 create mode 100644 .cargo/config.toml.in
 create mode 100644 .cargo/meson.build
 create mode 100644 Cargo.lock
 create mode 100644 Cargo.toml
 create mode 100644 qga/Cargo.toml
 create mode 100644 qga/lib.rs
 create mode 100644 qga/qapi-cabi.rs
 create mode 100644 qga/qapi.rs
 create mode 100644 qga/qapi_ffi.rs
 create mode 100644 qga/qmp/hostname.rs
 create mode 100644 qga/qmp/mod.rs
 create mode 100644 qga/qmp/vcpus.rs
 create mode 100644 rust/common/Cargo.toml
 create mode 100644 rust/common/src/error.rs
 create mode 100644 rust/common/src/ffi.rs
 create mode 100644 rust/common/src/lib.rs
 create mode 100644 rust/common/src/qemu.rs
 create mode 100644 rust/common/src/qmp.rs
 create mode 100644 rust/common/src/translate.rs
 create mode 160000 rust/vendored
 create mode 100644 scripts/cargo_wrapper.py
 create mode 100644 scripts/configh_to_cfg.py
 create mode 100644 scripts/diff_commands.py
 create mode 100644 scripts/qapi/cabi.py
 create mode 100644 scripts/qapi/rs.py
 create mode 100644 scripts/qapi/rs_ffi.py
 create mode 100644 scripts/qapi/rs_types.py
 create mode 100644 tests/Cargo.toml
 create mode 100644 tests/lib.rs
 create mode 100644 tests/qapi-cabi.rs
 create mode 100644 tests/qapi.rs
 create mode 100644 tests/qapi_ffi.rs

-- 
2.33.0.113.g6c40894d24