qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [RFC 00/19] Add virtual device fuzzing support
@ 2019-07-25  3:23 Oleinik, Alexander
  2019-07-25  3:23 ` [Qemu-devel] [RFC 01/19] fuzz: add configure option and linker objects Oleinik, Alexander
                   ` (21 more replies)
  0 siblings, 22 replies; 49+ messages in thread
From: Oleinik, Alexander @ 2019-07-25  3:23 UTC (permalink / raw)
  To: qemu-devel; +Cc: pbonzini, bsd, superirishdonkey, stefanha, Oleinik,  Alexander

As part of Google Summer of Code 2019, I'm working on integrating
fuzzing of virtual devices into QEMU [1]. This is a highly WIP patchset
adding this functionality.

Fuzzers provide random data to a program and monitor its execution for
errors. Coverage-guided fuzzers also observe the parts of the program
that are exercised by each input, and use this information to
mutate/guide the inputs to reach additional parts of the program. They
are quite effective for finding bugs in a wide range of software. 

Summary:
 - The virtual-device fuzzers use libfuzzer [2] for coverage-guided
   in-process fuzzing.
 - To fuzz a device, create a new fuzz "target" - i.e. a function that
   exercises QEMU based on inputs provided by the fuzzer.
 - Fuzz targets rely on qtest and libqos to turn inputs into actions.
 - Since libfuzzer does in-process fuzzing, the QEMU state needs to be
   reset after each fuzz run. These patches provide three methods for
   resetting state.
 - There are currently few targets, but they have already helped
   discover bugs in the console, and virtio-net, and have reproduced
   previously-reported vulnerabilities.

Here are some main implementation details:
 - The fuzzing occurs within a single process. QTest and QOS are
   modified so the QTest client and server coexist within the same
   process. They communicate with each other through direct function
   calls. Similar to qtest, the fuzzer uses a lightweight accelerator to
   skip CPU emulation. The fuzzing target is responsible for manually
   executing the main loop.
 - Since the same process is reused for many fuzzing runs, QEMU state
   needs to be reset at the end of each run. There are currently three
   implemented options for resetting state: 
   1. Reboot the guest between runs.
      Pros: Straightforward and fast for simple fuzz targets. 
      Cons: Depending on the device, does not reset all device state. If
      the device requires some initialization prior to being ready for
      fuzzing (common for QOS-based targets), this initialization needs
      to be done after each reboot.
      Example target: --virtio-net-ctrl-fuzz
   2. vmsave the state to RAM, once, and restore it after each run.
      Alternatively, only save the device state
      (savevm.c:qemu_save_device_state)
      Pros: Do not need to initialize devices prior to each run.
      VMStateDescriptions often specify more state than the device
      resetting functions called during reboots.
      Cons: Restoring state is often slower than rebooting. There is
      currently no way to save the QOS object state, so the objects
      usually needs to be re-allocated, defeating the purpose of
      one-time device initialization.
      Example target: --qtest-fuzz
   3. Run each test case in a separate forked process and copy the 
      coverage information back to the parent. This is fairly similar to
      AFL's "deferred" fork-server mode [3]
      Pros: Relatively fast. Devices only need to be initialized once.
      No need to do slow reboots or vmloads.
      Cons: Not officially supported by libfuzzer and the implementation
      is very flimsy. Does not work well for devices that rely on
      dedicated threads.
      Example target: --qtest-fork-fuzz
 - Fuzz targets are registered using QEMU's module system, similar to
   QOS test cases. Base qtest targets are registed with fuzz_add_target
   and QOS-based targets with fuzz_add_qos_target.
 - There are two entry points for the fuzzer:
    LLVMFuzzerInitialize: Run once, prior to fuzzing. Here, we set up
   qtest/qos, register the fuzz targets and partially execute vl.c:main.
   This is also where we would take a snapshot, if using the vmsave
   approach to resetting.
    LLVMFuzzerTestOneInput: Run for each fuzzing input. This function is
   responsible for taking care of device initialization, calling the
   actual fuzz target, and resetting state at the end of each run.
   Both of these functions are defined in tests/fuzz/fuzz.c
 - There are many libfuzzer flags which should be used to configure the
   coverage metrics and storage of interesting fuzz inputs. [2] These
   flags can also be helpful in evaluating fuzzing performance through
   metrics such as inputs/seconds and line-coverage.

Here are some key issues with the current state of the code:
 - The patches change vl.c, main-loop.c, qtest.c, tests/libqtest.c,
   savevm.c, memory.c. I wrapped the changes with #ifdef CONFIG_FUZZ,
   but many of these changes can and should be avoided.
 - tests/fuzz/qos_helpers.c is largely a copy of tests/qos-test.c.
 - The fuzzer is not properly integrated into the build system.
   Currently I simply added all of the necessary objects to
   target/i386/Makefile.objs, but there should be a simple way to build
   for other arches. The binary needs to be linked against libqemuutil,
   libqtest, qos and the qos objects, and the requirements for softmmu
   targets.
 - Some of the fuzz targets leak memory during state-resetting that need
   to be tracked down and fixed. 
 - As mentioned already, running each test in a separate process does
   not seem to be supported by libfuzzer, and the implementation
   reflects this (tests/fuzz/fuzzer_hooks.c)
 - The existing fuzz targets should be cleaned up as they have issues
   with memory alignment and contain redundant checks. The should also
   use qtest's clock_step. The fork fuzz targets are dependant on
   a hard-coded section size.

Building and running:
Libfuzzer requires clang.
  $ CC=clang-7 CXX=clang++-7 ./configure --enable-fuzzing
  $ make i386-softmmu/all
  $ i386-softmmu/qemu-system-i386 --qtest-dma-fuzz -detect_leaks=0

Here "qtest-dma-fuzz" is the fuzz target name. Running qemu-system-i386
without any arguments should print all of the available fuzz targets.
The -help=1 command prints out the available libfuzzer options.

There are more details, including instructions for adding new fuzz
targets in docs/devel/fuzzing.txt

In the coming weeks I would like to fix the issues listed above, more
fuzzing targets, and ideally work on getting QEMU into oss-fuzz[4],
where it can be fuzzed continuously.

I appreciate any feedback. Thanks
-Alex

[1] https://wiki.qemu.org/Internships/ProjectIdeas/QtestOssFuzz
[2] Trophy Case section: http://lcamtuf.coredump.cx/afl/
[3] https://llvm.org/docs/LibFuzzer.html
[4] https://github.com/mirrorer/afl/blob/master/llvm_mode/README.llvm#L82
[5] https://github.com/google/oss-fuzz


Alexander Oleinik (19):
  fuzz: add configure option and linker objects
  fuzz: add FUZZ_TARGET type to qemu module system
  fuzz: add fuzz accelerator
  fuzz: Add qos support to fuzz targets
  fuzz: expose qemu_savevm_state & skip state header
  fuzz: Add ramfile for fast vmstate/vmload
  fuzz: Modify libqtest to directly invoke qtest.c
  fuzz: add shims to intercept libfuzzer init
  fuzz: use mtree_info to find mapped addresses
  fuzz: expose real_main (aka regular vl.c:main)
  fuzz: add direct send/receive in qtest client
  fuzz: hard-code all of the needed files for build
  fuzz: add ctrl vq support to virtio-net in libqos
  fuzz: hard-code a main-loop timeout
  fuzz: add fuzz accelerator type
  fuzz: add general fuzzer entrypoints
  fuzz: add general qtest fuzz target
  fuzz: Add virtio-net tx and ctrl fuzz targets
  fuzz: Add documentation about the fuzzer to docs/

 accel/fuzz.c                 |  47 ++++++
 configure                    |  11 ++
 docs/devel/fuzzing.txt       | 145 +++++++++++++++++
 include/qemu/module.h        |   7 +-
 include/sysemu/fuzz.h        |  15 ++
 include/sysemu/qtest.h       |   7 +-
 include/sysemu/sysemu.h      |   4 +
 memory.c                     |  34 ++++
 migration/savevm.c           |   8 +-
 migration/savevm.h           |   3 +
 qtest.c                      |  19 ++-
 target/i386/Makefile.objs    |  19 +++
 tests/fuzz/fuzz.c            | 262 +++++++++++++++++++++++++++++++
 tests/fuzz/fuzz.h            |  96 ++++++++++++
 tests/fuzz/fuzzer_hooks.c    | 106 +++++++++++++
 tests/fuzz/fuzzer_hooks.h    |   9 ++
 tests/fuzz/qos_fuzz.c        |  63 ++++++++
 tests/fuzz/qos_fuzz.h        |  29 ++++
 tests/fuzz/qos_helpers.c     | 295 +++++++++++++++++++++++++++++++++++
 tests/fuzz/qos_helpers.h     |  17 ++
 tests/fuzz/qtest_fuzz.c      | 261 +++++++++++++++++++++++++++++++
 tests/fuzz/qtest_fuzz.h      |  38 +++++
 tests/fuzz/ramfile.c         | 127 +++++++++++++++
 tests/fuzz/ramfile.h         |  20 +++
 tests/fuzz/virtio-net-fuzz.c | 226 +++++++++++++++++++++++++++
 tests/libqos/virtio-net.c    |   2 +-
 tests/libqtest.c             |  53 ++++++-
 tests/libqtest.h             |   6 +
 util/main-loop.c             |   3 +
 vl.c                         |  21 ++-
 30 files changed, 1945 insertions(+), 8 deletions(-)
 create mode 100644 accel/fuzz.c
 create mode 100644 docs/devel/fuzzing.txt
 create mode 100644 include/sysemu/fuzz.h
 create mode 100644 tests/fuzz/fuzz.c
 create mode 100644 tests/fuzz/fuzz.h
 create mode 100644 tests/fuzz/fuzzer_hooks.c
 create mode 100644 tests/fuzz/fuzzer_hooks.h
 create mode 100644 tests/fuzz/qos_fuzz.c
 create mode 100644 tests/fuzz/qos_fuzz.h
 create mode 100644 tests/fuzz/qos_helpers.c
 create mode 100644 tests/fuzz/qos_helpers.h
 create mode 100644 tests/fuzz/qtest_fuzz.c
 create mode 100644 tests/fuzz/qtest_fuzz.h
 create mode 100644 tests/fuzz/ramfile.c
 create mode 100644 tests/fuzz/ramfile.h
 create mode 100644 tests/fuzz/virtio-net-fuzz.c

-- 
2.20.1



^ permalink raw reply	[flat|nested] 49+ messages in thread

end of thread, other threads:[~2019-08-06 10:01 UTC | newest]

Thread overview: 49+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-25  3:23 [Qemu-devel] [RFC 00/19] Add virtual device fuzzing support Oleinik, Alexander
2019-07-25  3:23 ` [Qemu-devel] [RFC 01/19] fuzz: add configure option and linker objects Oleinik, Alexander
2019-07-25  9:39   ` Paolo Bonzini
2019-07-25  3:23 ` [Qemu-devel] [RFC 02/19] fuzz: add FUZZ_TARGET type to qemu module system Oleinik, Alexander
2019-07-26 12:32   ` Stefan Hajnoczi
2019-07-25  3:23 ` [Qemu-devel] [RFC 03/19] fuzz: add fuzz accelerator Oleinik, Alexander
2019-07-26 10:33   ` Paolo Bonzini
2019-07-26 12:35   ` Stefan Hajnoczi
2019-07-25  3:23 ` [Qemu-devel] [RFC 04/19] fuzz: Add qos support to fuzz targets Oleinik, Alexander
2019-07-26 10:39   ` Paolo Bonzini
2019-07-25  3:23 ` [Qemu-devel] [RFC 05/19] fuzz: expose qemu_savevm_state & skip state header Oleinik, Alexander
2019-07-25 13:22   ` Dr. David Alan Gilbert
2019-07-25  3:23 ` [Qemu-devel] [RFC 07/19] fuzz: Modify libqtest to directly invoke qtest.c Oleinik, Alexander
2019-07-25  9:04   ` Thomas Huth
2019-07-25  9:33     ` Paolo Bonzini
2019-07-26 12:49     ` Stefan Hajnoczi
2019-07-26 12:56   ` Stefan Hajnoczi
2019-07-26 21:50     ` Paolo Bonzini
2019-07-25  3:23 ` [Qemu-devel] [RFC 06/19] fuzz: Add ramfile for fast vmstate/vmload Oleinik, Alexander
2019-07-26 12:47   ` Stefan Hajnoczi
2019-07-26 19:36     ` Oleinik, Alexander
2019-07-26 19:54       ` Paolo Bonzini
2019-07-25  3:23 ` [Qemu-devel] [RFC 08/19] fuzz: add shims to intercept libfuzzer init Oleinik, Alexander
2019-07-25  8:21   ` Paolo Bonzini
2019-07-26 12:59     ` Stefan Hajnoczi
2019-07-25  3:23 ` [Qemu-devel] [RFC 09/19] fuzz: use mtree_info to find mapped addresses Oleinik, Alexander
2019-07-26 13:04   ` Stefan Hajnoczi
2019-07-26 21:51     ` Paolo Bonzini
2019-07-25  3:23 ` [Qemu-devel] [RFC 10/19] fuzz: expose real_main (aka regular vl.c:main) Oleinik, Alexander
2019-07-25  9:38   ` Paolo Bonzini
2019-07-25  3:23 ` [Qemu-devel] [RFC 11/19] fuzz: add direct send/receive in qtest client Oleinik, Alexander
2019-07-25  9:10   ` Thomas Huth
2019-07-25  3:23 ` [Qemu-devel] [RFC 12/19] fuzz: hard-code all of the needed files for build Oleinik, Alexander
2019-07-25  3:23 ` [Qemu-devel] [RFC 13/19] fuzz: add ctrl vq support to virtio-net in libqos Oleinik, Alexander
2019-07-25 16:25   ` John Snow
2019-07-25 17:05     ` Oleinik, Alexander
2019-07-26 13:09       ` Stefan Hajnoczi
2019-07-25  3:23 ` [Qemu-devel] [RFC 14/19] fuzz: hard-code a main-loop timeout Oleinik, Alexander
2019-07-25  9:40   ` Paolo Bonzini
2019-07-25  3:23 ` [Qemu-devel] [RFC 15/19] fuzz: add fuzz accelerator type Oleinik, Alexander
2019-07-25  3:23 ` [Qemu-devel] [RFC 16/19] fuzz: add general fuzzer entrypoints Oleinik, Alexander
2019-07-25 17:53   ` Philippe Mathieu-Daudé
2019-07-25  3:23 ` [Qemu-devel] [RFC 17/19] fuzz: add general qtest fuzz target Oleinik, Alexander
2019-07-25  3:24 ` [Qemu-devel] [RFC 18/19] fuzz: Add virtio-net tx and ctrl fuzz targets Oleinik, Alexander
2019-07-25  3:24 ` [Qemu-devel] [RFC 19/19] fuzz: Add documentation about the fuzzer to docs/ Oleinik, Alexander
2019-07-26 13:19   ` Stefan Hajnoczi
2019-07-25  3:41 ` [Qemu-devel] [RFC 00/19] Add virtual device fuzzing support no-reply
2019-07-26 13:24 ` Stefan Hajnoczi
2019-08-06  9:59 ` jiade zhang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).