So far we neither compile-tested nor run any of the new fuzzers in our CI, which led to some build failures of the fuzzer code in the past weeks. To avoid this problem, add a job to compile the fuzzer code and run some loops (which likely don't find any new bugs via fuzzing, but at least we know that the code can still be run). A nice side-effect of this test is that the leak tests are enabled here, so we should now notice some of the memory leaks in our code base earlier. Signed-off-by: Thomas Huth <thuth@redhat.com> --- .gitlab-ci.yml | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5eeba2791b..e96f8794b9 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -161,9 +161,27 @@ build-clang: IMAGE: fedora CONFIGURE_ARGS: --cc=clang --cxx=clang++ TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu - ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user + ppc-softmmu s390x-softmmu arm-linux-user MAKE_CHECK_ARGS: check +build-fuzzer: + <<: *native_build_job_definition + variables: + IMAGE: fedora + script: + - mkdir build + - cd build + - ../configure --cc=clang --cxx=clang++ --enable-fuzzing + --target-list=x86_64-softmmu + - make -j"$JOBS" all check-build x86_64-softmmu/fuzz + - make check + - for fuzzer in i440fx-qos-fork-fuzz i440fx-qos-noreset-fuzz + i440fx-qtest-reboot-fuzz virtio-scsi-flags-fuzz virtio-scsi-fuzz ; do + echo Testing ${fuzzer} ... ; + x86_64-softmmu/qemu-fuzz-x86_64 --fuzz-target=${fuzzer} -runs=1000 + || exit 1 ; + done + build-tci: <<: *native_build_job_definition variables: -- 2.18.1
On 16/07/2020 12.09, Thomas Huth wrote:
> So far we neither compile-tested nor run any of the new fuzzers in our CI,
> which led to some build failures of the fuzzer code in the past weeks.
> To avoid this problem, add a job to compile the fuzzer code and run some
> loops (which likely don't find any new bugs via fuzzing, but at least we
> know that the code can still be run).
>
> A nice side-effect of this test is that the leak tests are enabled here,
> so we should now notice some of the memory leaks in our code base earlier.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> .gitlab-ci.yml | 20 +++++++++++++++++++-
> 1 file changed, 19 insertions(+), 1 deletion(-)
Note: This patch needs two other patches merged first to work correctly:
- 'fuzz: Expect the cmdline in a freeable GString' from Alexander
- 'qom: Plug memory leak in "info qom-tree"' from Markus
Otherwise the test will fail due to detected memory leaks.
Thomas
On 200716 1209, Thomas Huth wrote: > So far we neither compile-tested nor run any of the new fuzzers in our CI, > which led to some build failures of the fuzzer code in the past weeks. > To avoid this problem, add a job to compile the fuzzer code and run some > loops (which likely don't find any new bugs via fuzzing, but at least we > know that the code can still be run). > > A nice side-effect of this test is that the leak tests are enabled here, > so we should now notice some of the memory leaks in our code base earlier. > > Signed-off-by: Thomas Huth <thuth@redhat.com> Thank you for this, Thomas. I just sent a patch to add a job that runs similar tests with the build-script that we use on oss-fuzz Patch: <20200716163330.29141-1-alxndr@bu.edu> I know that these jobs have a lot of overlap, but there are enough quirks in the oss-fuzz build-script that I, personally, don't think they are redundant. A couple notes below, and I haven't been able to test on my own fork of qemu on gitlab, yet due to some pipeline errors, but otherwise Reviewed-by: Alexander Bulekov <alxndr@bu.edu> > --- > .gitlab-ci.yml | 20 +++++++++++++++++++- > 1 file changed, 19 insertions(+), 1 deletion(-) > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index 5eeba2791b..e96f8794b9 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -161,9 +161,27 @@ build-clang: > IMAGE: fedora > CONFIGURE_ARGS: --cc=clang --cxx=clang++ > TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu > - ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user > + ppc-softmmu s390x-softmmu arm-linux-user > MAKE_CHECK_ARGS: check > > +build-fuzzer: > + <<: *native_build_job_definition > + variables: > + IMAGE: fedora > + script: > + - mkdir build > + - cd build > + - ../configure --cc=clang --cxx=clang++ --enable-fuzzing > + --target-list=x86_64-softmmu https://lists.nongnu.org/archive/html/qemu-devel/2020-07/msg02310.html When/if this gets merged, enable-fuzzing won't build with AddressSanitizer, by default, so I would add --enable-sanitizers, just to be safe. > + - make -j"$JOBS" all check-build x86_64-softmmu/fuzz > + - make check > + - for fuzzer in i440fx-qos-fork-fuzz i440fx-qos-noreset-fuzz > + i440fx-qtest-reboot-fuzz virtio-scsi-flags-fuzz virtio-scsi-fuzz ; do Any reason for these particular fuzzers? I know the virtio-net ones find crashes pretty quickly, but I dont think that causes a non-zero exit. > + echo Testing ${fuzzer} ... ; > + x86_64-softmmu/qemu-fuzz-x86_64 --fuzz-target=${fuzzer} -runs=1000 > + || exit 1 ; > + done > + > build-tci: > <<: *native_build_job_definition > variables: > -- > 2.18.1 >
On 16/07/2020 18.46, Alexander Bulekov wrote: > On 200716 1209, Thomas Huth wrote: >> So far we neither compile-tested nor run any of the new fuzzers in our CI, >> which led to some build failures of the fuzzer code in the past weeks. >> To avoid this problem, add a job to compile the fuzzer code and run some >> loops (which likely don't find any new bugs via fuzzing, but at least we >> know that the code can still be run). >> >> A nice side-effect of this test is that the leak tests are enabled here, >> so we should now notice some of the memory leaks in our code base earlier. >> >> Signed-off-by: Thomas Huth <thuth@redhat.com> > > Thank you for this, Thomas. I just sent a patch to add a job that runs > similar tests with the build-script that we use on oss-fuzz > Patch: <20200716163330.29141-1-alxndr@bu.edu> Good idea! ... but it does not work quite yet, I gave it a try and ended up here: https://gitlab.com/huth/qemu/-/jobs/643353500 > I know that these jobs have a lot of overlap, but there are enough > quirks in the oss-fuzz build-script that I, personally, don't think > they are redundant. I think we should finally go with your approach and compile the fuzzing test via the script. But since that seems to need some more work first, let's go with my patch now, so that we have something for 5.1-rc1, and then when your patch is ready, replace my "build-fuzzer" job with yours, ok? >> +build-fuzzer: >> + <<: *native_build_job_definition >> + variables: >> + IMAGE: fedora >> + script: >> + - mkdir build >> + - cd build >> + - ../configure --cc=clang --cxx=clang++ --enable-fuzzing >> + --target-list=x86_64-softmmu > > https://lists.nongnu.org/archive/html/qemu-devel/2020-07/msg02310.html > When/if this gets merged, enable-fuzzing won't build with > AddressSanitizer, by default, so I would add --enable-sanitizers, just > to be safe. Ok, thanks, I'll add that. >> + - make -j"$JOBS" all check-build x86_64-softmmu/fuzz >> + - make check >> + - for fuzzer in i440fx-qos-fork-fuzz i440fx-qos-noreset-fuzz >> + i440fx-qtest-reboot-fuzz virtio-scsi-flags-fuzz virtio-scsi-fuzz ; do > > Any reason for these particular fuzzers? I know the virtio-net ones find > crashes pretty quickly, but I dont think that causes a non-zero exit. I did not include the virtio-net fuzzers because I read that warning "May result in network traffic emitted from the process. Run in an isolated network environment." in the help text ... so I was not sure whether they are really suitable for the CI on the gitlab machines? Thomas