Announcing the Berserker toolkit for (semi-)automated fs fuzz testing

Announcing the Berserker toolkit for (semi-)automated fs fuzz testing

[Sami Liedes]

[-- Attachment #1: Type: text/plain, Size: 2576 bytes --]

This is an announcement of the first release of the Berserker toolkit
for (semi-)automated fuzz testing and testcase minimization of Linux
kernel filesystem implementations.

The toolkit consists of the following components and their
documentation:

* Debian sid (unstable) based root filesystem image with scripts
  inside and set to run automatically that will test filesystems based
  on kernel commandline parameters, intended to be run inside a
  virtual machine (KVM); a 32-bit x86 system

* berserker-testfs.py, a script to automate running fuzz tests inside
  KVM on a filesystem image (simply give as parameters the filesystem
  type, a working filesystem image and a kernel bzImage - see --help).
  This script takes care of running KVM and interpreting the output.
  Its return values make it suitable for using in "git bisect run". By
  default the VM will fuzz and run until it has produced a crash.

* berserker-minimize.sh (and fuzz-minimize used by it), a program to
  derive a crash-inducing test case with minimal differences to a
  pristine filesystem image by repeatedly automatically running
  berserker-testfs.py on different images. Takes as input the kernel
  image to use, a pristine filesystem image and a fuzzed filesystem
  image that causes the kernel to crash.

To get the source:

   git clone http://www.niksula.hut.fi/~sliedes/berserker/berserker.git

The repository contains a script (download-binaries.sh) that downloads
some files (*at least until my university gets unhappy with the
bandwidth used):

* the root filesystem (hda.autotest; 112 MiB compressed, 501 MiB
  uncompressed); and for quick start

* a vanilla 3.3.4 bzImage for amd64 suitable for use with the system
  (config file included in the git repository)

* testimg.ext4, a 10 MiB ext4 filesystem with the required layout
  (described in more detail in the README file)

The actual fuzzer used is zzuf (Debian package zzuf) by Sam Hocevar. I
believe it is similar in spirit to fsfuzzer which appears to be more
familiar to the kernel community; zzuf was chosen because I was more
familiar with it and because it was packaged for Debian.

Two examples of (what appears to be) ext4 bugs found with this
toolkit, both reproducible with a 1-bit difference to a pristine
filesystem:

  http://www.spinics.net/lists/linux-ext4/msg31850.html
     (sorry, by mistake the subject doesn't quite reflect the bug...)

  http://www.spinics.net/lists/linux-ext4/msg31853.html

See the included README file for a very quick introduction and some
more detailed documentation.

	Sami Liedes

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Announcing the Berserker toolkit for (semi-)automated fs fuzz testing

[Richard W.M. Jones]

On Sat, Apr 28, 2012 at 04:39:04AM +0300, Sami Liedes wrote:
> This is an announcement of the first release of the Berserker toolkit
> for (semi-)automated fuzz testing and testcase minimization of Linux
> kernel filesystem implementations.

This looks interesting, and it's very necessary to find these bugs in
filesystem drivers to improve the security around virtualization and
untrusted disk images.

I wrote something similar a while back, trying to use systemtap to do
feedback-directed fuzz testing.  By putting a tracepoint on every line
in the filesystem module, you can find out how many lines of code are
actually executed when mounting/using the fuzzed filesystem and use
that as a cost function for feedback (maximizing the # of lines
executed, rather than blindly fuzzing).  Unfortunately it doesn't work
yet because of a bug in Linux[1].

Anyway I was going to say: why not use libguestfs to provide a simpler
framework for running KVM and the fuzzer?

Rich.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=713248

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: Announcing the Berserker toolkit for (semi-)automated fs fuzz testing

[Sami Liedes]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

On Sun, May 06, 2012 at 11:27:27AM +0100, Richard W.M. Jones wrote:
> I wrote something similar a while back, trying to use systemtap to do
> feedback-directed fuzz testing.  By putting a tracepoint on every line
> in the filesystem module, you can find out how many lines of code are
> actually executed when mounting/using the fuzzed filesystem and use
> that as a cost function for feedback (maximizing the # of lines
> executed, rather than blindly fuzzing).  Unfortunately it doesn't work
> yet because of a bug in Linux[1].

I guess something like that could also be used for trying to generate
a small(ish) set of test case filesystem images that maximally
exercise the code.

> Anyway I was going to say: why not use libguestfs to provide a simpler
> framework for running KVM and the fuzzer?

I didn't even know about libguestfs. Very interesting; I need to take
a closer look at it.

I'd be glad quite glad if I could somehow get rid of the root
filesystem image, or to build and update it automatically. Right now
it's a headache for version control purposes.

	Sami

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]