Re: [PATCH v3 4/5] Added build and install scripts

[Alexander Graf <graf@amazon.com>]

On 24.06.19 16:24, Sam Caccavale wrote:
> install_afl.sh installs AFL locally and emits AFLPATH,
> build.sh, and run.sh build and run respectively
> 
> ---
> 
> v1 -> v2:
>   - Introduced this patch
> 
> v2 -> v3:
>   - Moved non-essential development scripts to a later patch
> 
> Signed-off-by: Sam Caccavale <samcacc@amazon.de>
> ---
>   tools/fuzz/x86ie/scripts/afl-many       | 31 +++++++++++++++++++++++
>   tools/fuzz/x86ie/scripts/build.sh       | 33 +++++++++++++++++++++++++
>   tools/fuzz/x86ie/scripts/install_afl.sh | 17 +++++++++++++
>   tools/fuzz/x86ie/scripts/run.sh         | 10 ++++++++
>   4 files changed, 91 insertions(+)
>   create mode 100755 tools/fuzz/x86ie/scripts/afl-many
>   create mode 100755 tools/fuzz/x86ie/scripts/build.sh
>   create mode 100755 tools/fuzz/x86ie/scripts/install_afl.sh
>   create mode 100755 tools/fuzz/x86ie/scripts/run.sh
> 
> diff --git a/tools/fuzz/x86ie/scripts/afl-many b/tools/fuzz/x86ie/scripts/afl-many
> new file mode 100755
> index 000000000000..e55ff115a777
> --- /dev/null
> +++ b/tools/fuzz/x86ie/scripts/afl-many
> @@ -0,0 +1,31 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0+
> +# This is for running AFL over NPROC or `nproc` cores with normal AFL options ex:
> +# ulimit -Sv $[21999999999 << 10]; ./tools/fuzz/x86ie/scripts/afl-many -m 22000000000 -i $FUZZDIR/in -o $FUZZDIR/out tools/fuzz/x86ie/afl-harness @@
> +
> +export AFL_NO_AFFINITY=1
> +
> +while [ -z "$sync_dir" ]; do
> +  while getopts ":o:" opt; do
> +    case "${opt}" in
> +      o)
> +        sync_dir="${OPTARG}"
> +        ;;
> +      *)
> +        ;;
> +    esac
> +  done
> +  ((OPTIND++))
> +  [ $OPTIND -gt $# ] && break
> +done
> +
> +# AFL/linux do some weird stuff with core affinity and will often run
> +# N processes over < N virtual cores.  In order to avoid that, we taskset
> +# each process to its own core.
> +for i in $(seq 1 $(( ${NPROC:-$(nproc)} - 1)) ); do
> +    taskset -c "$i" ./afl-fuzz -S "slave$i" $@ >/dev/null 2>&1 &
> +done
> +taskset -c 0 ./afl-fuzz -M master $@ >/dev/null 2>&1 &
> +
> +watch -n1 "echo \"Executing '$AFLPATH/afl-fuzz $@' on ${NPROC:-$(nproc)} cores.\" && $AFLPATH/afl-whatsup -s ${sync_dir}"
> +pkill afl-fuzz
> diff --git a/tools/fuzz/x86ie/scripts/build.sh b/tools/fuzz/x86ie/scripts/build.sh
> new file mode 100755
> index 000000000000..032762bf56ef
> --- /dev/null
> +++ b/tools/fuzz/x86ie/scripts/build.sh
> @@ -0,0 +1,33 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0+
> +# Run from root of linux via `./tools/fuzz/x86ie/scripts/build.sh`
> +
> +kernel_objects="arch/x86/kvm/emulate.o arch/x86/lib/retpoline.o lib/find_bit.o"
> +
> +disable() { sed -i -r "/\b$1\b/c\# $1" .config; }
> +enable() { sed -i -r "/\b$1\b/c\\$1=y" .config; }
> +
> +make ${CC:+ "CC=$CC"} ${DEBUG:+ "DEBUG=1"} defconfig
> +
> +enable "CONFIG_DEBUG_INFO"
> +enable "CONFIG_STACKPROTECTOR"
> +
> +yes ' ' | make ${CC:+ "CC=$CC"} ${DEBUG:+ "DEBUG=1"} $kernel_objects
> +
> +omit_arg () { args=$(echo "$args" | sed "s/ $1//g"); }
> +add_arg () { args+=" $1"; }
> +
> +rebuild () {
> +  args="$(head -1 $(dirname $1)/.$(basename $1).cmd | sed -e 's/.*:= //g')"
> +  omit_arg "-mcmodel=kernel"
> +  omit_arg "-mpreferred-stack-boundary=3"
> +  add_arg "-fsanitize=address"
> +  echo -e "Rebuilding $1 with \n$args"
> +  eval "$args"
> +}
> +
> +for object in $kernel_objects; do
> +  rebuild $object
> +done
> +
> +make ${CC:+ "CC=$CC"} ${DEBUG:+ "DEBUG=1"} tools/fuzz
> diff --git a/tools/fuzz/x86ie/scripts/install_afl.sh b/tools/fuzz/x86ie/scripts/install_afl.sh
> new file mode 100755
> index 000000000000..3bdbdf2a040b
> --- /dev/null
> +++ b/tools/fuzz/x86ie/scripts/install_afl.sh
> @@ -0,0 +1,17 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0+
> +# Can be run where ever, but usually run from linux root:
> +# `source ./tools/fuzz/x86ie/scripts/install_afl.sh`
> +# (must be sourced to get the AFLPATH envvar, otherwise set manually)
> +
> +wget http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz
> +mkdir -p afl
> +tar xzf afl-latest.tgz -C afl --strip-components 1
> +
> +pushd afl
> +set AFL_USE_ASAN
> +make clean all
> +export AFLPATH="$(pwd)"
> +popd
> +
> +sudo bash -c "echo core >/proc/sys/kernel/core_pattern"

What is this? :)

Surely if it's important to generate core dumps, it's not only important 
during installation, no?

Alex

> diff --git a/tools/fuzz/x86ie/scripts/run.sh b/tools/fuzz/x86ie/scripts/run.sh
> new file mode 100755
> index 000000000000..0571cd524c01
> --- /dev/null
> +++ b/tools/fuzz/x86ie/scripts/run.sh
> @@ -0,0 +1,10 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0+
> +
> +FUZZDIR="${FUZZDIR:-$(pwd)/fuzz}"
> +
> +mkdir -p $FUZZDIR/in
> +cp tools/fuzz/x86ie/rand_sample.bin $FUZZDIR/in
> +mkdir -p $FUZZDIR/out
> +
> +screen bash -c "ulimit -Sv $[21999999999 << 10]; ./tools/fuzz/x86ie/scripts/afl-many -m 22000000000 -i $FUZZDIR/in -o $FUZZDIR/out tools/fuzz/x86ie/afl-harness @@"
>