How to clone CPUState in a new thread?

How to clone CPUState in a new thread?

[Michael Goffioul]

[-- Attachment #1.1: Type: text/plain, Size: 2896 bytes --]

[originally posted on qemu-discuss]

=== (initial)

Hi,

I'm working on a project that wants to replace houdini (ARM-to-x86
translation layer for Android from Intel) with a free open-source
implementation. I'm trying to leverage qemu user-mode to achieve that, but
it requires code changes to allow executing dynamically loaded functions
instead of running a single executable.

In a nutshell, using ideas from unicorn-engine, I've enhanced CPUARMState
with a stop address. Whenever this address is encountered in the
translator, it generates a YIELD exception, which then makes the cpu_loop
to exit.

It works fine for simple cases, but I'm having trouble with multi-threading
aspect. Threads created from the native/ARM side do seem to work properly.
The problem is when a new Java thread (not created from native/ARM)
attempts to execute native code. The QEMU engine has been initialized in
the main thread, but new Java threads do not have access to thread-local
variable thread_cpu.

I've tried (maybe naively) to recreate what the clone syscall is doing to
create a new CPUState/CPUArchState object, usable from the new thread, but
executing any ARM code quickly lead to a crash. I suppose I'm doing
something wrong, or missing something to properly initiale a new cpu. I'm
hoping that someone could help me solve this problem.

I've attached the current QEMU patch I'm using, most of the Android glue
layer is in linux-user/main.c. It contains a set of utility functions that
my Android native bridge implementation is using.

=== (follow-up)

Basically Houdini implements the native bridge interface, as defined here:
https://android.googlesource.com/platform/system/core/+/refs/tags/android-10.0.0_r11/libnativebridge/include/nativebridge/native_bridge.h#172
It allows running Android APK that contains ARM-compiled native/JNI code on
an Android-x86 OS. It does so by taking care of loading the ARM .so JNI
files are providing trampoline stubs to the Android runtime JVM. It does
not expose the host native .so to the emulated code, instead it provides a
set of ARM-compiled core libraries from Android: it is actually very
similar to running dynamically linked code in qemu-user with a chroot'ed
ARM environment. Actual interaction with the native host is happening
mostly/only through binder socket.

To initialize the qemu-user engine, I make it load a custom ARM .so/ELF
file that uses the Android linker (from the ARM pseudo chroot environment)
as interpreter. This allows me to delegate all dynamic linking aspects.

So far, the emulation is working fine and I'm able to run simple
ARM-compiled apps on Android-x86, even if the native code spawns new
threads. My current (hopefully last) problem is when a Java thread,
different than the one that initialized the qemu engine) is trying to run
native code. I need to setup a new CPUState/CPUArchState instance for this
Java thread.

[-- Attachment #1.2: Type: text/html, Size: 3282 bytes --]

[-- Attachment #2: qemu-android.diff.bz2 --]
[-- Type: application/x-bzip, Size: 7533 bytes --]