2017-03-24 13:56 GMT+01:00 Wei Liu <wei.liu2@citrix.com>:
On Fri, Mar 24, 2017 at 08:34:46AM +0100, Felix Schmoll wrote:
> Hi,
>
> this time for real:
>
> so I've been reading up on the task quite a bit now and I'd be thankful if
> you could clarify what exactly you're looking for with the execution path.
> The AFL-fuzzer seems to make use just of a rather simple representation of
> that (https://lwn.net/Articles/674854/), so you would just have to insert
> this snippet:
>
>   cur_location = <COMPILE_TIME_RANDOM>;
>   shared_mem[cur_location ^ prev_location]++;
>   prev_location = cur_location >> 1;
>

Yes, that's the basic idea for AFL.

However, you can't just do that for the hypervisor. AFL is geared
towards fuzzing userspace libraries. It is not possible to just use AFL
toolchain to compile Xen and expect it to work. It will fail miserably.
You can't even get Xen to compile with afl-gcc or afl-clang.

You would need to get the execution path with some other method instead
of relying on the AFL toolchain. Using gcc's trace-pc support is good
for that.

> at every edge, i.e. into *__sanitizer_cov_trace_pc,* and somehow set up a
> shared memory section between kernel (where you'd run the fuzzer) and
> hypervisor.

Using shared memory is also an idea. But as you already point out, that
might not be applicable to other fuzzer.

AFL won't run inside guest kernel though. It will remain a userspace
program.

> That might however just be true for AFL and not for other
> coverage-guided fuzzers, so maybe that's the problem. For the fuzzer to
> work I'd also have to implement some templates on what hypercalls are
> available and what arguments they expect, but that would still not fill
> three months, so I'm assuming that I'm fundamentally missing something here.
>

My idea is that:

1. Extract execution path from hypervisor via a hypercall.
2. Write a small stub userspace program to turn the raw data into a form
   that can be consumed by AFL (or any other fuzzer you have in mind).
3. Use the same small stub program to drive a domU to execute the
   test case provided by AFL.

Some of the code in #2 and #3 is going to be fuzzer-specific (the
template you talked about), but we should be able to make the
architecture as generic as possible.

That's only a high-level description. There is actually a lot of work to
be done. It would be good if we can upstream #1 within 3 months.  If we
can get all three implemented, that would be really good.

So just one last time to be clear about this: You can't just ignore interrupts and write all other edges to a shared memory region, like the KCOV feature the syzkaller uses does, but have to find a way to clean the execution path from whatever Xen is doing under the hood, which is what makes it difficult? Or is it that afl-gcc is actually doing much more than inserting that snippet?

Is there any particular format you're thinking of for the execution path, i.e. can the three-line snippet be used, or would that already belong to #2, and you would want to have something like a sequence of left-/right-branch-taken decisions?

Also, just for my general understanding, one would in the end still have to build some infrastructure similar to what syzkaller does to actually run the hypervisor, i.e. some virtualisation environment to run the hypervisor in, and so on, right?

Lastly, do you have any suggestions for what would be a good midterm-deliverable? Based on the assumption that the answer to my first question is affirmative I was thinking of a thorough idea on how the hypercall is implemented.

Thanks once again
Felix
Wei.