[PATCH-for-5.1 0/2] fuzz: broken build fixes

[PATCH-for-5.1 0/2] fuzz: broken build fixes

[Alexander Bulekov]

Hi,
These fix build-breaking problems with --enable-fuzzing
The first patch prevents --enable-fuzzing from overwriting CFLAGS.
The second patch adds a missing header to fuzz.c
-Alex

Alexander Bulekov (2):
  configure: do not clobber CFLAGS with --enable-fuzzing
  fuzz: add missing header for rcu_enable_atfork

 configure               | 2 +-
 tests/qtest/fuzz/fuzz.c | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

-- 
2.26.2

[PATCH-for-5.1 1/2] configure: do not clobber CFLAGS with --enable-fuzzing

[Alexander Bulekov]

When configuring with --enable-fuzzing, we overwrote the CFLAGS
added by all the preceding checks. Instead of overwriting CFLAGS, append
the ones we need.

Fixes: adc28027ff ("fuzz: add configure flag --enable-fuzzing")
Reported-by: Li Qiang <liq3ea@163.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
---
 configure | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index ee6c3c6792..078ebaa9f3 100755
--- a/configure
+++ b/configure
@@ -7898,7 +7898,7 @@ if test "$fuzzing" = "yes" ; then
   if test "$have_fuzzer" = "yes"; then
     FUZZ_LDFLAGS=" -fsanitize=address,fuzzer"
     FUZZ_CFLAGS=" -fsanitize=address,fuzzer"
-    CFLAGS=" -fsanitize=address,fuzzer-no-link"
+    CFLAGS="$CFLAGS -fsanitize=address,fuzzer-no-link"
   else
     error_exit "Your compiler doesn't support -fsanitize=address,fuzzer"
     exit 1
-- 
2.26.2

Re: [PATCH-for-5.1 1/2] configure: do not clobber CFLAGS with --enable-fuzzing

[Li Qiang]

Alexander Bulekov <alxndr@bu.edu> 于2020年7月9日周四 上午4:02写道：
>
> When configuring with --enable-fuzzing, we overwrote the CFLAGS
> added by all the preceding checks. Instead of overwriting CFLAGS, append
> the ones we need.
>
> Fixes: adc28027ff ("fuzz: add configure flag --enable-fuzzing")
> Reported-by: Li Qiang <liq3ea@163.com>
> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>


Tested-by: Li Qiang <liq3ea@gmail.com>
Reviewed-by: Li Qiang <liq3ea@gmail.com>

> ---
>  configure | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/configure b/configure
> index ee6c3c6792..078ebaa9f3 100755
> --- a/configure
> +++ b/configure
> @@ -7898,7 +7898,7 @@ if test "$fuzzing" = "yes" ; then
>    if test "$have_fuzzer" = "yes"; then
>      FUZZ_LDFLAGS=" -fsanitize=address,fuzzer"
>      FUZZ_CFLAGS=" -fsanitize=address,fuzzer"
> -    CFLAGS=" -fsanitize=address,fuzzer-no-link"
> +    CFLAGS="$CFLAGS -fsanitize=address,fuzzer-no-link"
>    else
>      error_exit "Your compiler doesn't support -fsanitize=address,fuzzer"
>      exit 1
> --
> 2.26.2
>
>

Re: [PATCH-for-5.1 1/2] configure: do not clobber CFLAGS with --enable-fuzzing

[Philippe Mathieu-Daudé]

On 7/8/20 10:01 PM, Alexander Bulekov wrote:
> When configuring with --enable-fuzzing, we overwrote the CFLAGS
> added by all the preceding checks. Instead of overwriting CFLAGS, append
> the ones we need.
> 
> Fixes: adc28027ff ("fuzz: add configure flag --enable-fuzzing")
> Reported-by: Li Qiang <liq3ea@163.com>
> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> ---
>  configure | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/configure b/configure
> index ee6c3c6792..078ebaa9f3 100755
> --- a/configure
> +++ b/configure
> @@ -7898,7 +7898,7 @@ if test "$fuzzing" = "yes" ; then
>    if test "$have_fuzzer" = "yes"; then
>      FUZZ_LDFLAGS=" -fsanitize=address,fuzzer"
>      FUZZ_CFLAGS=" -fsanitize=address,fuzzer"
> -    CFLAGS=" -fsanitize=address,fuzzer-no-link"
> +    CFLAGS="$CFLAGS -fsanitize=address,fuzzer-no-link"
>    else
>      error_exit "Your compiler doesn't support -fsanitize=address,fuzzer"
>      exit 1
> 

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

[PATCH-for-5.1 2/2] fuzz: add missing header for rcu_enable_atfork

[Alexander Bulekov]

In 45222b9a90, I fixed a broken check for rcu_enable_atfork introduced
in d6919e4cb6. I added a call to rcu_enable_atfork after the
call to qemu_init in fuzz.c, but forgot to include the corresponding
header, breaking --enable-fuzzing --enable-werror builds.

Fixes: 45222b9a90 ("fuzz: fix broken qtest check at rcu_disable_atfork")
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
---
 tests/qtest/fuzz/fuzz.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
index a36d9038e0..0b66e43409 100644
--- a/tests/qtest/fuzz/fuzz.c
+++ b/tests/qtest/fuzz/fuzz.c
@@ -19,6 +19,7 @@
 #include "sysemu/runstate.h"
 #include "sysemu/sysemu.h"
 #include "qemu/main-loop.h"
+#include "qemu/rcu.h"
 #include "tests/qtest/libqtest.h"
 #include "tests/qtest/libqos/qgraph.h"
 #include "fuzz.h"
-- 
2.26.2

Re: [PATCH-for-5.1 2/2] fuzz: add missing header for rcu_enable_atfork

[Philippe Mathieu-Daudé]

On 7/8/20 10:01 PM, Alexander Bulekov wrote:
> In 45222b9a90, I fixed a broken check for rcu_enable_atfork introduced
> in d6919e4cb6. I added a call to rcu_enable_atfork after the
> call to qemu_init in fuzz.c, but forgot to include the corresponding
> header, breaking --enable-fuzzing --enable-werror builds.
> 
> Fixes: 45222b9a90 ("fuzz: fix broken qtest check at rcu_disable_atfork")
> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> ---
>  tests/qtest/fuzz/fuzz.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
> index a36d9038e0..0b66e43409 100644
> --- a/tests/qtest/fuzz/fuzz.c
> +++ b/tests/qtest/fuzz/fuzz.c
> @@ -19,6 +19,7 @@
>  #include "sysemu/runstate.h"
>  #include "sysemu/sysemu.h"
>  #include "qemu/main-loop.h"
> +#include "qemu/rcu.h"
>  #include "tests/qtest/libqtest.h"
>  #include "tests/qtest/libqos/qgraph.h"
>  #include "fuzz.h"
> 

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH-for-5.1 2/2] fuzz: add missing header for rcu_enable_atfork

[Philippe Mathieu-Daudé]

On 7/9/20 7:03 AM, Philippe Mathieu-Daudé wrote:
> On 7/8/20 10:01 PM, Alexander Bulekov wrote:
>> In 45222b9a90, I fixed a broken check for rcu_enable_atfork introduced
>> in d6919e4cb6. I added a call to rcu_enable_atfork after the
>> call to qemu_init in fuzz.c, but forgot to include the corresponding
>> header, breaking --enable-fuzzing --enable-werror builds.
>>
>> Fixes: 45222b9a90 ("fuzz: fix broken qtest check at rcu_disable_atfork")
>> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
>> ---
>>  tests/qtest/fuzz/fuzz.c | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
>> index a36d9038e0..0b66e43409 100644
>> --- a/tests/qtest/fuzz/fuzz.c
>> +++ b/tests/qtest/fuzz/fuzz.c
>> @@ -19,6 +19,7 @@
>>  #include "sysemu/runstate.h"
>>  #include "sysemu/sysemu.h"
>>  #include "qemu/main-loop.h"
>> +#include "qemu/rcu.h"
>>  #include "tests/qtest/libqtest.h"
>>  #include "tests/qtest/libqos/qgraph.h"
>>  #include "fuzz.h"
>>
> 
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> 

Please add the include to softmmu/vl.c too.

Re: [PATCH-for-5.1 2/2] fuzz: add missing header for rcu_enable_atfork

[Thomas Huth]

On 09/07/2020 07.09, Philippe Mathieu-Daudé wrote:
> On 7/9/20 7:03 AM, Philippe Mathieu-Daudé wrote:
>> On 7/8/20 10:01 PM, Alexander Bulekov wrote:
>>> In 45222b9a90, I fixed a broken check for rcu_enable_atfork introduced
>>> in d6919e4cb6. I added a call to rcu_enable_atfork after the
>>> call to qemu_init in fuzz.c, but forgot to include the corresponding
>>> header, breaking --enable-fuzzing --enable-werror builds.
>>>
>>> Fixes: 45222b9a90 ("fuzz: fix broken qtest check at rcu_disable_atfork")
>>> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
>>> ---
>>>  tests/qtest/fuzz/fuzz.c | 1 +
>>>  1 file changed, 1 insertion(+)
>>>
>>> diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
>>> index a36d9038e0..0b66e43409 100644
>>> --- a/tests/qtest/fuzz/fuzz.c
>>> +++ b/tests/qtest/fuzz/fuzz.c
>>> @@ -19,6 +19,7 @@
>>>  #include "sysemu/runstate.h"
>>>  #include "sysemu/sysemu.h"
>>>  #include "qemu/main-loop.h"
>>> +#include "qemu/rcu.h"
>>>  #include "tests/qtest/libqtest.h"
>>>  #include "tests/qtest/libqos/qgraph.h"
>>>  #include "fuzz.h"
>>>
>>
>> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>>
> 
> Please add the include to softmmu/vl.c too.

Why? Did you run into compile problems here, too?

 Thomas

Re: [PATCH-for-5.1 2/2] fuzz: add missing header for rcu_enable_atfork

[Philippe Mathieu-Daudé]

On 7/9/20 7:15 AM, Thomas Huth wrote:
> On 09/07/2020 07.09, Philippe Mathieu-Daudé wrote:
>> On 7/9/20 7:03 AM, Philippe Mathieu-Daudé wrote:
>>> On 7/8/20 10:01 PM, Alexander Bulekov wrote:
>>>> In 45222b9a90, I fixed a broken check for rcu_enable_atfork introduced
>>>> in d6919e4cb6. I added a call to rcu_enable_atfork after the
>>>> call to qemu_init in fuzz.c, but forgot to include the corresponding
>>>> header, breaking --enable-fuzzing --enable-werror builds.
>>>>
>>>> Fixes: 45222b9a90 ("fuzz: fix broken qtest check at rcu_disable_atfork")
>>>> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
>>>> ---
>>>>  tests/qtest/fuzz/fuzz.c | 1 +
>>>>  1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
>>>> index a36d9038e0..0b66e43409 100644
>>>> --- a/tests/qtest/fuzz/fuzz.c
>>>> +++ b/tests/qtest/fuzz/fuzz.c
>>>> @@ -19,6 +19,7 @@
>>>>  #include "sysemu/runstate.h"
>>>>  #include "sysemu/sysemu.h"
>>>>  #include "qemu/main-loop.h"
>>>> +#include "qemu/rcu.h"
>>>>  #include "tests/qtest/libqtest.h"
>>>>  #include "tests/qtest/libqos/qgraph.h"
>>>>  #include "fuzz.h"
>>>>
>>>
>>> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>>>
>>
>> Please add the include to softmmu/vl.c too.
> 
> Why? Did you run into compile problems here, too?

No, because it is unexpectedly pulled by "exec/memory.h".

You are right however this is unrelated to 45222b9a90,
it comes from 73c6e4013b ("rcu: completely disable pthread_atfork
callbacks as soon as possible"), so I'll send a separate patch.

Re: [PATCH-for-5.1 2/2] fuzz: add missing header for rcu_enable_atfork

[Thomas Huth]

On 08/07/2020 22.01, Alexander Bulekov wrote:
> In 45222b9a90, I fixed a broken check for rcu_enable_atfork introduced
> in d6919e4cb6. I added a call to rcu_enable_atfork after the
> call to qemu_init in fuzz.c, but forgot to include the corresponding
> header, breaking --enable-fuzzing --enable-werror builds.
> 
> Fixes: 45222b9a90 ("fuzz: fix broken qtest check at rcu_disable_atfork")
> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> ---
>  tests/qtest/fuzz/fuzz.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
> index a36d9038e0..0b66e43409 100644
> --- a/tests/qtest/fuzz/fuzz.c
> +++ b/tests/qtest/fuzz/fuzz.c
> @@ -19,6 +19,7 @@
>  #include "sysemu/runstate.h"
>  #include "sysemu/sysemu.h"
>  #include "qemu/main-loop.h"
> +#include "qemu/rcu.h"
>  #include "tests/qtest/libqtest.h"
>  #include "tests/qtest/libqos/qgraph.h"
>  #include "fuzz.h"

D'oh, mea culpa, I also apparently did not properly compile test that
patch :-( I think we need a CI job that at least compile tests the
fuzzing code - I can look into that once Alex Bennée's current testing
pull request has been merged.

Alexander, is there also a way to run a fuzzer just for some few
minutes? E.g. a fuzzing test that finishes quickly, or an option to
limit the time that a test is running? If so, we could also add that
quick test to the CI pipeline, to make sure that the fuzzer code does
not only compile, but is also able to run (at least a little bit).

For this patch here:
Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH-for-5.1 2/2] fuzz: add missing header for rcu_enable_atfork

[Alexander Bulekov]

On 200709 0718, Thomas Huth wrote:
> On 08/07/2020 22.01, Alexander Bulekov wrote:
> > In 45222b9a90, I fixed a broken check for rcu_enable_atfork introduced
> > in d6919e4cb6. I added a call to rcu_enable_atfork after the
> > call to qemu_init in fuzz.c, but forgot to include the corresponding
> > header, breaking --enable-fuzzing --enable-werror builds.
> > 
> > Fixes: 45222b9a90 ("fuzz: fix broken qtest check at rcu_disable_atfork")
> > Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> > ---
> >  tests/qtest/fuzz/fuzz.c | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
> > index a36d9038e0..0b66e43409 100644
> > --- a/tests/qtest/fuzz/fuzz.c
> > +++ b/tests/qtest/fuzz/fuzz.c
> > @@ -19,6 +19,7 @@
> >  #include "sysemu/runstate.h"
> >  #include "sysemu/sysemu.h"
> >  #include "qemu/main-loop.h"
> > +#include "qemu/rcu.h"
> >  #include "tests/qtest/libqtest.h"
> >  #include "tests/qtest/libqos/qgraph.h"
> >  #include "fuzz.h"
> 
> D'oh, mea culpa, I also apparently did not properly compile test that
> patch :-( I think we need a CI job that at least compile tests the
> fuzzing code - I can look into that once Alex Bennée's current testing
> pull request has been merged.

My bad - I should have done a clean build with a version of clang
that doesn't require me to -disable-werror

> Alexander, is there also a way to run a fuzzer just for some few
> minutes? E.g. a fuzzing test that finishes quickly, or an option to
> limit the time that a test is running? If so, we could also add that
> quick test to the CI pipeline, to make sure that the fuzzer code does
> not only compile, but is also able to run (at least a little bit).

Yes. I think the sequence could look something like:
CC=clang CXX=clang++ ../configure --enable-fuzzing --enable-sanitizers \
             --enable-werror
make i386-softmmu/fuzz
./i386-softmmu/qemu-fuzz-i386 --fuzz-target=i440fx-qtest-reboot-fuzz -runs=5000

This will run the i440fx fuzzer over 5000 inputs which should finish in
a second or so. I don't expect it to actually find any crashes in the
i440fx in such a short period, so, ideally, all errors would be
fuzzer-related.

Where can I get started with building out a CI job for this?

One aside: running this right now, QEMU exits and AddressSanitizer
complains about some leaks. There is a patch in Paolo's PR that should
fix this, but I was surprised that existing CI tests didn't catch it. Is
leak detection usually disabled in CI?

> For this patch here:
> Reviewed-by: Thomas Huth <thuth@redhat.com>

Thanks!
-Alex

Re: [PATCH-for-5.1 2/2] fuzz: add missing header for rcu_enable_atfork

[Thomas Huth]

On 09/07/2020 15.38, Alexander Bulekov wrote:
> On 200709 0718, Thomas Huth wrote:
>> On 08/07/2020 22.01, Alexander Bulekov wrote:
>>> In 45222b9a90, I fixed a broken check for rcu_enable_atfork introduced
>>> in d6919e4cb6. I added a call to rcu_enable_atfork after the
>>> call to qemu_init in fuzz.c, but forgot to include the corresponding
>>> header, breaking --enable-fuzzing --enable-werror builds.
>>>
>>> Fixes: 45222b9a90 ("fuzz: fix broken qtest check at rcu_disable_atfork")
>>> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
>>> ---
>>>  tests/qtest/fuzz/fuzz.c | 1 +
>>>  1 file changed, 1 insertion(+)
>>>
>>> diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
>>> index a36d9038e0..0b66e43409 100644
>>> --- a/tests/qtest/fuzz/fuzz.c
>>> +++ b/tests/qtest/fuzz/fuzz.c
>>> @@ -19,6 +19,7 @@
>>>  #include "sysemu/runstate.h"
>>>  #include "sysemu/sysemu.h"
>>>  #include "qemu/main-loop.h"
>>> +#include "qemu/rcu.h"
>>>  #include "tests/qtest/libqtest.h"
>>>  #include "tests/qtest/libqos/qgraph.h"
>>>  #include "fuzz.h"
>>
>> D'oh, mea culpa, I also apparently did not properly compile test that
>> patch :-( I think we need a CI job that at least compile tests the
>> fuzzing code - I can look into that once Alex Bennée's current testing
>> pull request has been merged.
> 
> My bad - I should have done a clean build with a version of clang
> that doesn't require me to -disable-werror
> 
>> Alexander, is there also a way to run a fuzzer just for some few
>> minutes? E.g. a fuzzing test that finishes quickly, or an option to
>> limit the time that a test is running? If so, we could also add that
>> quick test to the CI pipeline, to make sure that the fuzzer code does
>> not only compile, but is also able to run (at least a little bit).
> 
> Yes. I think the sequence could look something like:
> CC=clang CXX=clang++ ../configure --enable-fuzzing --enable-sanitizers \
>              --enable-werror
> make i386-softmmu/fuzz
> ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=i440fx-qtest-reboot-fuzz -runs=5000
> 
> This will run the i440fx fuzzer over 5000 inputs which should finish in
> a second or so. I don't expect it to actually find any crashes in the
> i440fx in such a short period, so, ideally, all errors would be
> fuzzer-related.
> 
> Where can I get started with building out a CI job for this?

I'd suggest to use gitlab, since we're currently focusing on that for
our CI. So get an account on gitlab, clone the qemu repository there
(https://gitlab.com/qemu-project/qemu) to your account, and then you
should almost be ready to go: Edit the .gitlab-ci.yml file in the
repository, and once you push your local branch to the gitlab server,
you should see the jobs running in the "CI / CD" section. (Not sure
anymore whether you have to enable the CI manually for your project,
though, but it should not be too hard to find that setting if that's the
case)

> One aside: running this right now, QEMU exits and AddressSanitizer
> complains about some leaks. There is a patch in Paolo's PR that should
> fix this, but I was surprised that existing CI tests didn't catch it. Is
> leak detection usually disabled in CI?

I'm not aware of any CI tests that is currently using leak detection ...
so it's certainly welcome if we get more test coverage here!

 Thomas