All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] fuzz: add oss-fuzz build.sh script
@ 2020-06-05 17:40 Alexander Bulekov
  2020-06-05 17:47 ` Alexander Bulekov
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Alexander Bulekov @ 2020-06-05 17:40 UTC (permalink / raw)
  To: qemu-devel; +Cc: darren.kenny, bsd, f4bug, stefanha, Alexander Bulekov

It is neater to keep this in the QEMU repo, since any change that
requires an update to the oss-fuzz build configuration, can make the
necessary changes in the same series.

Suggested-by: Philippe Mathieu-Daude <f4bug@amsat.org>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
---
 scripts/oss-fuzz/build.sh | 47 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100755 scripts/oss-fuzz/build.sh

diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh
new file mode 100755
index 0000000000..7be6dcce4c
--- /dev/null
+++ b/scripts/oss-fuzz/build.sh
@@ -0,0 +1,47 @@
+#!/bin/sh
+#
+# Update syscall_nr.h files from linux headers asm-generic/unistd.h
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+# build project
+# e.g.
+# ./autogen.sh
+# ./configure
+# make -j$(nproc) all
+
+# build fuzzers
+# e.g.
+# $CXX $CXXFLAGS -std=c++11 -Iinclude \
+#     /path/to/name_of_fuzzer.cc -o $OUT/name_of_fuzzer \
+#     $LIB_FUZZING_ENGINE /path/to/library.a
+
+mkdir -p $OUT/lib/              # Shared libraries
+
+# Build once to get the list of dynamic lib paths, and copy them over
+./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" \
+    --extra-cflags="$CFLAGS -U __OPTIMIZE__ "
+make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz
+
+for i in $(ldd ./i386-softmmu/qemu-fuzz-i386  | cut -f3 -d' '); do 
+    cp $i $OUT/lib/
+done
+rm ./i386-softmmu/qemu-fuzz-i386
+
+# Build a second time to build the final binary with correct rpath
+./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" \
+    --extra-cflags="$CFLAGS -U __OPTIMIZE__" \
+    --extra-ldflags="-Wl,-rpath,'\$\$ORIGIN/lib'"
+make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz
+
+# Copy over the datadir
+cp  -r ./pc-bios/ $OUT/pc-bios
+
+# Copy over the qemu-fuzz-i386, naming it according to each available fuzz
+# target (See 05509c8e6d fuzz: select fuzz target using executable name)
+for target in $(./i386-softmmu/qemu-fuzz-i386 | awk '$1 ~ /\*/  {print $2}');
+do
+    cp ./i386-softmmu/qemu-fuzz-i386 $OUT/qemu-fuzz-i386-target-$target
+done
-- 
2.26.2



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] fuzz: add oss-fuzz build.sh script
  2020-06-05 17:40 [PATCH] fuzz: add oss-fuzz build.sh script Alexander Bulekov
@ 2020-06-05 17:47 ` Alexander Bulekov
  2020-06-05 17:56 ` Philippe Mathieu-Daudé
  2020-06-05 20:47 ` no-reply
  2 siblings, 0 replies; 6+ messages in thread
From: Alexander Bulekov @ 2020-06-05 17:47 UTC (permalink / raw)
  To: qemu-devel; +Cc: darren.kenny, bsd, f4bug, stefanha

On 200605 1340, Alexander Bulekov wrote:
> It is neater to keep this in the QEMU repo, since any change that
> requires an update to the oss-fuzz build configuration, can make the
> necessary changes in the same series.
> 
> Suggested-by: Philippe Mathieu-Daude <f4bug@amsat.org>
> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> ---
>  scripts/oss-fuzz/build.sh | 47 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
>  create mode 100755 scripts/oss-fuzz/build.sh
> 
> diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh
> new file mode 100755
> index 0000000000..7be6dcce4c
> --- /dev/null
> +++ b/scripts/oss-fuzz/build.sh
> @@ -0,0 +1,47 @@
> +#!/bin/sh
> +#
> +# Update syscall_nr.h files from linux headers asm-generic/unistd.h
This is obviously wrong... Sending v2.

> +#
> +# This code is licensed under the GPL version 2 or later.  See
> +# the COPYING file in the top-level directory.
> +#
> +
> +# build project
> +# e.g.
> +# ./autogen.sh
> +# ./configure
> +# make -j$(nproc) all
> +
> +# build fuzzers
> +# e.g.
> +# $CXX $CXXFLAGS -std=c++11 -Iinclude \
> +#     /path/to/name_of_fuzzer.cc -o $OUT/name_of_fuzzer \
> +#     $LIB_FUZZING_ENGINE /path/to/library.a
> +
> +mkdir -p $OUT/lib/              # Shared libraries
> +
> +# Build once to get the list of dynamic lib paths, and copy them over
> +./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" \
> +    --extra-cflags="$CFLAGS -U __OPTIMIZE__ "
> +make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz
> +
> +for i in $(ldd ./i386-softmmu/qemu-fuzz-i386  | cut -f3 -d' '); do 
> +    cp $i $OUT/lib/
> +done
> +rm ./i386-softmmu/qemu-fuzz-i386
> +
> +# Build a second time to build the final binary with correct rpath
> +./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" \
> +    --extra-cflags="$CFLAGS -U __OPTIMIZE__" \
> +    --extra-ldflags="-Wl,-rpath,'\$\$ORIGIN/lib'"
> +make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz
> +
> +# Copy over the datadir
> +cp  -r ./pc-bios/ $OUT/pc-bios
> +
> +# Copy over the qemu-fuzz-i386, naming it according to each available fuzz
> +# target (See 05509c8e6d fuzz: select fuzz target using executable name)
> +for target in $(./i386-softmmu/qemu-fuzz-i386 | awk '$1 ~ /\*/  {print $2}');
> +do
> +    cp ./i386-softmmu/qemu-fuzz-i386 $OUT/qemu-fuzz-i386-target-$target
> +done
> -- 
> 2.26.2
> 


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] fuzz: add oss-fuzz build.sh script
  2020-06-05 17:40 [PATCH] fuzz: add oss-fuzz build.sh script Alexander Bulekov
  2020-06-05 17:47 ` Alexander Bulekov
@ 2020-06-05 17:56 ` Philippe Mathieu-Daudé
  2020-06-05 18:41   ` Alexander Bulekov
  2020-06-08 17:49   ` Alexander Bulekov
  2020-06-05 20:47 ` no-reply
  2 siblings, 2 replies; 6+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-06-05 17:56 UTC (permalink / raw)
  To: Alexander Bulekov, qemu-devel; +Cc: darren.kenny, bsd, stefanha

On 6/5/20 7:40 PM, Alexander Bulekov wrote:
> It is neater to keep this in the QEMU repo, since any change that
> requires an update to the oss-fuzz build configuration, can make the
> necessary changes in the same series.
> 
> Suggested-by: Philippe Mathieu-Daude <f4bug@amsat.org>

'Philippe Mathieu-Daudé' ;)

> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> ---
>  scripts/oss-fuzz/build.sh | 47 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
>  create mode 100755 scripts/oss-fuzz/build.sh
> 
> diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh
> new file mode 100755
> index 0000000000..7be6dcce4c
> --- /dev/null
> +++ b/scripts/oss-fuzz/build.sh
> @@ -0,0 +1,47 @@
> +#!/bin/sh
> +#
> +# Update syscall_nr.h files from linux headers asm-generic/unistd.h

Hmmm?

> +#
> +# This code is licensed under the GPL version 2 or later.  See
> +# the COPYING file in the top-level directory.
> +#
> +
> +# build project

Please mention this file use consumed by
https://github.com/google/oss-fuzz/.../projects/qemu/Dockerfile

> +# e.g.
> +# ./autogen.sh
> +# ./configure
> +# make -j$(nproc) all
> +
> +# build fuzzers
> +# e.g.
> +# $CXX $CXXFLAGS -std=c++11 -Iinclude \
> +#     /path/to/name_of_fuzzer.cc -o $OUT/name_of_fuzzer \
> +#     $LIB_FUZZING_ENGINE /path/to/library.a
> +
> +mkdir -p $OUT/lib/              # Shared libraries

Maybe rename OUT -> DEST_DIR?

> +
> +# Build once to get the list of dynamic lib paths, and copy them over
> +./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" \
> +    --extra-cflags="$CFLAGS -U __OPTIMIZE__ "

So we use an in-tree build.

Still we could set some SRCDIR=./

> +make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz
> +
> +for i in $(ldd ./i386-softmmu/qemu-fuzz-i386  | cut -f3 -d' '); do 
> +    cp $i $OUT/lib/
> +done
> +rm ./i386-softmmu/qemu-fuzz-i386
> +
> +# Build a second time to build the final binary with correct rpath
> +./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" \
> +    --extra-cflags="$CFLAGS -U __OPTIMIZE__" \
> +    --extra-ldflags="-Wl,-rpath,'\$\$ORIGIN/lib'"
> +make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz
> +
> +# Copy over the datadir
> +cp  -r ./pc-bios/ $OUT/pc-bios

"make install-datadir"?

> +
> +# Copy over the qemu-fuzz-i386, naming it according to each available fuzz
> +# target (See 05509c8e6d fuzz: select fuzz target using executable name)
> +for target in $(./i386-softmmu/qemu-fuzz-i386 | awk '$1 ~ /\*/  {print $2}');
> +do
> +    cp ./i386-softmmu/qemu-fuzz-i386 $OUT/qemu-fuzz-i386-target-$target

There seems to be an extra 'target'.

> +done
> 

Or "make install", not sure.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] fuzz: add oss-fuzz build.sh script
  2020-06-05 17:56 ` Philippe Mathieu-Daudé
@ 2020-06-05 18:41   ` Alexander Bulekov
  2020-06-08 17:49   ` Alexander Bulekov
  1 sibling, 0 replies; 6+ messages in thread
From: Alexander Bulekov @ 2020-06-05 18:41 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: darren.kenny, bsd, qemu-devel, stefanha

On 200605 1956, Philippe Mathieu-Daudé wrote:
> On 6/5/20 7:40 PM, Alexander Bulekov wrote:
> > It is neater to keep this in the QEMU repo, since any change that
> > requires an update to the oss-fuzz build configuration, can make the
> > necessary changes in the same series.
> > 
> > Suggested-by: Philippe Mathieu-Daude <f4bug@amsat.org>
> 
> 'Philippe Mathieu-Daudé' ;)

Oops - Sorry.

> 
> > Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> > ---
> >  scripts/oss-fuzz/build.sh | 47 +++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 47 insertions(+)
> >  create mode 100755 scripts/oss-fuzz/build.sh
> > 
> > diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh
> > new file mode 100755
> > index 0000000000..7be6dcce4c
> > --- /dev/null
> > +++ b/scripts/oss-fuzz/build.sh
> > @@ -0,0 +1,47 @@
> > +#!/bin/sh
> > +#
> > +# Update syscall_nr.h files from linux headers asm-generic/unistd.h
> 
> Hmmm?

Fixed in v2.

> 
> > +#
> > +# This code is licensed under the GPL version 2 or later.  See
> > +# the COPYING file in the top-level directory.
> > +#
> > +
> > +# build project
> 
> Please mention this file use consumed by
> https://github.com/google/oss-fuzz/.../projects/qemu/Dockerfile
> 
> > +# e.g.
> > +# ./autogen.sh
> > +# ./configure
> > +# make -j$(nproc) all
> > +
> > +# build fuzzers
> > +# e.g.
> > +# $CXX $CXXFLAGS -std=c++11 -Iinclude \
> > +#     /path/to/name_of_fuzzer.cc -o $OUT/name_of_fuzzer \
> > +#     $LIB_FUZZING_ENGINE /path/to/library.a
> > +
> > +mkdir -p $OUT/lib/              # Shared libraries
> 
> Maybe rename OUT -> DEST_DIR?

$OUT is something specified by OSS-Fuzz, when it runs the script in
docker. If its better, I can do DEST_DIR=$OUT

> > +
> > +# Build once to get the list of dynamic lib paths, and copy them over
> > +./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" \
> > +    --extra-cflags="$CFLAGS -U __OPTIMIZE__ "
> 
> So we use an in-tree build.
> 
> Still we could set some SRCDIR=./
I can change it to build in ./build/ or even an out-of-tree build, if
thats neater..

> 
> > +make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz
> > +
> > +for i in $(ldd ./i386-softmmu/qemu-fuzz-i386  | cut -f3 -d' '); do 
> > +    cp $i $OUT/lib/
> > +done
> > +rm ./i386-softmmu/qemu-fuzz-i386
> > +
> > +# Build a second time to build the final binary with correct rpath
> > +./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" \
> > +    --extra-cflags="$CFLAGS -U __OPTIMIZE__" \
> > +    --extra-ldflags="-Wl,-rpath,'\$\$ORIGIN/lib'"
> > +make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz
> > +
> > +# Copy over the datadir
> > +cp  -r ./pc-bios/ $OUT/pc-bios
> 
> "make install-datadir"?
With something like: ./configure --datadir="$OUT/pc-bios/"
Ok.

> 
> > +
> > +# Copy over the qemu-fuzz-i386, naming it according to each available fuzz
> > +# target (See 05509c8e6d fuzz: select fuzz target using executable name)
> > +for target in $(./i386-softmmu/qemu-fuzz-i386 | awk '$1 ~ /\*/  {print $2}');
> > +do
> > +    cp ./i386-softmmu/qemu-fuzz-i386 $OUT/qemu-fuzz-i386-target-$target
> 
> There seems to be an extra 'target'.
I don't think so, unless i'm missing something.
We do a strstr(argv[0], "-target-") in fuzz.c The
targets need to be named:
qemu-fuzz-i386-target-virtio-net-socket
qemu-fuzz-i386-target-i440fx-qos-fork-fuzz
etc..

Thanks
-Alex

> > +done
> > 
> 
> Or "make install", not sure.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] fuzz: add oss-fuzz build.sh script
  2020-06-05 17:40 [PATCH] fuzz: add oss-fuzz build.sh script Alexander Bulekov
  2020-06-05 17:47 ` Alexander Bulekov
  2020-06-05 17:56 ` Philippe Mathieu-Daudé
@ 2020-06-05 20:47 ` no-reply
  2 siblings, 0 replies; 6+ messages in thread
From: no-reply @ 2020-06-05 20:47 UTC (permalink / raw)
  To: alxndr; +Cc: darren.kenny, qemu-devel, f4bug, alxndr, bsd, stefanha

Patchew URL: https://patchew.org/QEMU/20200605174036.4527-1-alxndr@bu.edu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20200605174036.4527-1-alxndr@bu.edu
Subject: [PATCH] fuzz: add oss-fuzz build.sh script
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20200605174036.4527-1-alxndr@bu.edu -> patchew/20200605174036.4527-1-alxndr@bu.edu
Switched to a new branch 'test'
8109a86 fuzz: add oss-fuzz build.sh script

=== OUTPUT BEGIN ===
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#16: 
new file mode 100755

ERROR: trailing whitespace
#48: FILE: scripts/oss-fuzz/build.sh:28:
+for i in $(ldd ./i386-softmmu/qemu-fuzz-i386  | cut -f3 -d' '); do $

total: 1 errors, 1 warnings, 47 lines checked

Commit 8109a8627d68 (fuzz: add oss-fuzz build.sh script) has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200605174036.4527-1-alxndr@bu.edu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] fuzz: add oss-fuzz build.sh script
  2020-06-05 17:56 ` Philippe Mathieu-Daudé
  2020-06-05 18:41   ` Alexander Bulekov
@ 2020-06-08 17:49   ` Alexander Bulekov
  1 sibling, 0 replies; 6+ messages in thread
From: Alexander Bulekov @ 2020-06-08 17:49 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: darren.kenny, bsd, qemu-devel, stefanha

On 200605 1956, Philippe Mathieu-Daudé wrote:
> On 6/5/20 7:40 PM, Alexander Bulekov wrote:
-cut-
> "make install-datadir"?
I think this just sets up the datadir for subsequent copies:

install-datadir:
    $(INSTALL_DIR) "$(DESTDIR)$(qemu_datadir)"

The actual copy happens in install:

install: all $(if $(BUILD_DOCS),install-doc) \
	install-datadir install-localstatedir install-includedir 
...
ifneq ($(BLOBS),)
	set -e; for x in $(BLOBS); do \
		$(INSTALL_DATA) $(SRC_PATH)/pc-bios/$$x "$(DESTDIR)$(qemu_datadir)"; \
	done
...

> 
> > +
> > +# Copy over the qemu-fuzz-i386, naming it according to each available fuzz
> > +# target (See 05509c8e6d fuzz: select fuzz target using executable name)
> > +for target in $(./i386-softmmu/qemu-fuzz-i386 | awk '$1 ~ /\*/  {print $2}');
> > +do
> > +    cp ./i386-softmmu/qemu-fuzz-i386 $OUT/qemu-fuzz-i386-target-$target
> 
> There seems to be an extra 'target'.
> 
> > +done
> > 
> 
> Or "make install", not sure.

If I can get this to work, hopefully it will also take care of the
datadir.
Thanks
-Alex


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-06-08 17:52 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-05 17:40 [PATCH] fuzz: add oss-fuzz build.sh script Alexander Bulekov
2020-06-05 17:47 ` Alexander Bulekov
2020-06-05 17:56 ` Philippe Mathieu-Daudé
2020-06-05 18:41   ` Alexander Bulekov
2020-06-08 17:49   ` Alexander Bulekov
2020-06-05 20:47 ` no-reply

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.