* [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
@ 2020-08-31 15:32 Thomas Huth
2020-08-31 16:40 ` Alexander Bulekov
2020-08-31 21:31 ` Philippe Mathieu-Daudé
0 siblings, 2 replies; 7+ messages in thread
From: Thomas Huth @ 2020-08-31 15:32 UTC (permalink / raw)
To: qemu-devel, Alexander Bulekov
Cc: Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Bandan Das, Stefan Hajnoczi,
Paolo Bonzini, Alex Bennée
The fuzzer job finishes quite early, so we can run the unit tests and
qtests with -fsanitize=address here without extending the total test time.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
.gitlab-ci.yml | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 17f1f8fad9..417fda6909 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -256,13 +256,14 @@ build-oss-fuzz:
- mkdir build-oss-fuzz
- CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
./scripts/oss-fuzz/build.sh
+ - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
- for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f
| grep -v slirp); do
grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ;
echo Testing ${fuzzer} ... ;
- ASAN_OPTIONS="fast_unwind_on_malloc=0"
- "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
+ "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
done
+ - cd build-oss-fuzz && make check-qtest-i386 check-unit
build-tci:
<<: *native_build_job_definition
--
2.18.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
2020-08-31 15:32 [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job Thomas Huth
@ 2020-08-31 16:40 ` Alexander Bulekov
2020-09-01 6:07 ` Thomas Huth
2020-08-31 21:31 ` Philippe Mathieu-Daudé
1 sibling, 1 reply; 7+ messages in thread
From: Alexander Bulekov @ 2020-08-31 16:40 UTC (permalink / raw)
To: Thomas Huth
Cc: Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Alex Bennée
On 200831 1732, Thomas Huth wrote:
> The fuzzer job finishes quite early, so we can run the unit tests and
> qtests with -fsanitize=address here without extending the total test time.
>
Sounds good to me, though its too bad that this is limited to i386 and
we aren't using the --enable-sanitizers configure argument due to the
strange oss-fuzz build script.
Speaking of testing with sanitizers, does it make sense to have a job
that does check-qtest with --enable-tsan, now that we have
ThreadSanitizer support?
Can --enable-sanitizers --enable-tsan be added to some existing job,
without severely increasing the pipeline's exeuction time?
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> .gitlab-ci.yml | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 17f1f8fad9..417fda6909 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -256,13 +256,14 @@ build-oss-fuzz:
> - mkdir build-oss-fuzz
> - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
> ./scripts/oss-fuzz/build.sh
> + - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
> - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f
> | grep -v slirp); do
> grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ;
> echo Testing ${fuzzer} ... ;
> - ASAN_OPTIONS="fast_unwind_on_malloc=0"
> - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> done
> + - cd build-oss-fuzz && make check-qtest-i386 check-unit
>
> build-tci:
> <<: *native_build_job_definition
> --
> 2.18.2
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
2020-08-31 16:40 ` Alexander Bulekov
@ 2020-09-01 6:07 ` Thomas Huth
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Huth @ 2020-09-01 6:07 UTC (permalink / raw)
To: Alexander Bulekov
Cc: Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Alex Bennée
On 31/08/2020 18.40, Alexander Bulekov wrote:
> On 200831 1732, Thomas Huth wrote:
>> The fuzzer job finishes quite early, so we can run the unit tests and
>> qtests with -fsanitize=address here without extending the total test time.
>>
>
> Sounds good to me, though its too bad that this is limited to i386 and
> we aren't using the --enable-sanitizers configure argument due to the
> strange oss-fuzz build script.
>
> Speaking of testing with sanitizers, does it make sense to have a job
> that does check-qtest with --enable-tsan, now that we have
> ThreadSanitizer support?
>
> Can --enable-sanitizers --enable-tsan be added to some existing job,
> without severely increasing the pipeline's exeuction time?
Good idea, I'll have a try when I've got some spare time again (probably
not this week, though, it's a pretty busy one for me).
> Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Thanks!
Thomas
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
2020-08-31 15:32 [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job Thomas Huth
2020-08-31 16:40 ` Alexander Bulekov
@ 2020-08-31 21:31 ` Philippe Mathieu-Daudé
2020-09-01 6:04 ` Thomas Huth
1 sibling, 1 reply; 7+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-08-31 21:31 UTC (permalink / raw)
To: Thomas Huth
Cc: Alex Bennée, qemu-devel@nongnu.org Developers,
Wainer dos Santos Moschetta, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Philippe Mathieu-Daudé
[-- Attachment #1: Type: text/plain, Size: 1414 bytes --]
Hi Thomas,
Le lun. 31 août 2020 17:33, Thomas Huth <thuth@redhat.com> a écrit :
> The fuzzer job finishes quite early, so we can run the unit tests and
> qtests with -fsanitize=address here without extending the total test time.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> .gitlab-ci.yml | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 17f1f8fad9..417fda6909 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -256,13 +256,14 @@ build-oss-fuzz:
> - mkdir build-oss-fuzz
> - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
> ./scripts/oss-fuzz/build.sh
> + - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
> - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f
> | grep -v slirp); do
> grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 ||
> continue ;
> echo Testing ${fuzzer} ... ;
> - ASAN_OPTIONS="fast_unwind_on_malloc=0"
> - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> done
> + - cd build-oss-fuzz && make check-qtest-i386 check-unit
>
As this does not use the fuzzer main entry point, what is the point of
running that?
> build-tci:
> <<: *native_build_job_definition
> --
> 2.18.2
>
>
>
[-- Attachment #2: Type: text/html, Size: 2225 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
2020-08-31 21:31 ` Philippe Mathieu-Daudé
@ 2020-09-01 6:04 ` Thomas Huth
2020-09-01 6:12 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 7+ messages in thread
From: Thomas Huth @ 2020-09-01 6:04 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Alex Bennée, qemu-devel@nongnu.org Developers,
Wainer dos Santos Moschetta, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Philippe Mathieu-Daudé
On 31/08/2020 23.31, Philippe Mathieu-Daudé wrote:
> Hi Thomas,
>
> Le lun. 31 août 2020 17:33, Thomas Huth <thuth@redhat.com
> <mailto:thuth@redhat.com>> a écrit :
>
> The fuzzer job finishes quite early, so we can run the unit tests and
> qtests with -fsanitize=address here without extending the total test
> time.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com <mailto:thuth@redhat.com>>
> ---
> .gitlab-ci.yml | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 17f1f8fad9..417fda6909 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -256,13 +256,14 @@ build-oss-fuzz:
> - mkdir build-oss-fuzz
> - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
> ./scripts/oss-fuzz/build.sh
> + - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
> - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable
> -type f
> | grep -v slirp); do
> grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 ||
> continue ;
> echo Testing ${fuzzer} ... ;
> - ASAN_OPTIONS="fast_unwind_on_malloc=0"
> - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> done
> + - cd build-oss-fuzz && make check-qtest-i386 check-unit
>
>
> As this does not use the fuzzer main entry point, what is the point of
> running that?
Read the friendly patch description ;-) - the idea here is that we run
the qtests and unit tests with -fsanitize=address in one of the
pipelines. We could also use a different task for this, but since the
fuzzer tests finish quite early and we already compiled the code with
-fsanitize=address here anyway, it's IMHO simply a good fit here.
Thomas
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
2020-09-01 6:04 ` Thomas Huth
@ 2020-09-01 6:12 ` Philippe Mathieu-Daudé
2020-09-01 6:19 ` Thomas Huth
0 siblings, 1 reply; 7+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-09-01 6:12 UTC (permalink / raw)
To: Thomas Huth
Cc: Philippe Mathieu-Daudé,
qemu-devel@nongnu.org Developers, Wainer dos Santos Moschetta,
Alexander Bulekov, Bandan Das, Stefan Hajnoczi, Paolo Bonzini,
Alex Bennée
On Tue, Sep 1, 2020 at 8:05 AM Thomas Huth <thuth@redhat.com> wrote:
>
> On 31/08/2020 23.31, Philippe Mathieu-Daudé wrote:
> > Hi Thomas,
> >
> > Le lun. 31 août 2020 17:33, Thomas Huth <thuth@redhat.com
> > <mailto:thuth@redhat.com>> a écrit :
> >
> > The fuzzer job finishes quite early, so we can run the unit tests and
> > qtests with -fsanitize=address here without extending the total test
> > time.
> >
> > Signed-off-by: Thomas Huth <thuth@redhat.com <mailto:thuth@redhat.com>>
> > ---
> > .gitlab-ci.yml | 5 +++--
> > 1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index 17f1f8fad9..417fda6909 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -256,13 +256,14 @@ build-oss-fuzz:
> > - mkdir build-oss-fuzz
> > - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
> > ./scripts/oss-fuzz/build.sh
> > + - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
> > - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable
> > -type f
> > | grep -v slirp); do
> > grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 ||
> > continue ;
> > echo Testing ${fuzzer} ... ;
> > - ASAN_OPTIONS="fast_unwind_on_malloc=0"
> > - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> > + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
> > done
> > + - cd build-oss-fuzz && make check-qtest-i386 check-unit
> >
> >
> > As this does not use the fuzzer main entry point, what is the point of
> > running that?
>
> Read the friendly patch description ;-) - the idea here is that we run
> the qtests and unit tests with -fsanitize=address in one of the
> pipelines.
Sorry, the description was not that obvious to me.
Can you add a comment before the 'make check*' line?
# Unrelated to fuzzer: run tests with -fsanitize=address
> We could also use a different task for this, but since the
> fuzzer tests finish quite early and we already compiled the code with
> -fsanitize=address here anyway, it's IMHO simply a good fit here.
>
> Thomas
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
2020-09-01 6:12 ` Philippe Mathieu-Daudé
@ 2020-09-01 6:19 ` Thomas Huth
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Huth @ 2020-09-01 6:19 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Philippe Mathieu-Daudé,
qemu-devel@nongnu.org Developers, Wainer dos Santos Moschetta,
Alexander Bulekov, Bandan Das, Stefan Hajnoczi, Paolo Bonzini,
Alex Bennée
On 01/09/2020 08.12, Philippe Mathieu-Daudé wrote:
> On Tue, Sep 1, 2020 at 8:05 AM Thomas Huth <thuth@redhat.com> wrote:
>>
>> On 31/08/2020 23.31, Philippe Mathieu-Daudé wrote:
>>> Hi Thomas,
>>>
>>> Le lun. 31 août 2020 17:33, Thomas Huth <thuth@redhat.com
>>> <mailto:thuth@redhat.com>> a écrit :
>>>
>>> The fuzzer job finishes quite early, so we can run the unit tests and
>>> qtests with -fsanitize=address here without extending the total test
>>> time.
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com <mailto:thuth@redhat.com>>
>>> ---
>>> .gitlab-ci.yml | 5 +++--
>>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>> index 17f1f8fad9..417fda6909 100644
>>> --- a/.gitlab-ci.yml
>>> +++ b/.gitlab-ci.yml
>>> @@ -256,13 +256,14 @@ build-oss-fuzz:
>>> - mkdir build-oss-fuzz
>>> - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"
>>> ./scripts/oss-fuzz/build.sh
>>> + - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
>>> - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable
>>> -type f
>>> | grep -v slirp); do
>>> grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 ||
>>> continue ;
>>> echo Testing ${fuzzer} ... ;
>>> - ASAN_OPTIONS="fast_unwind_on_malloc=0"
>>> - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
>>> + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
>>> done
>>> + - cd build-oss-fuzz && make check-qtest-i386 check-unit
>>>
>>>
>>> As this does not use the fuzzer main entry point, what is the point of
>>> running that?
>>
>> Read the friendly patch description ;-) - the idea here is that we run
>> the qtests and unit tests with -fsanitize=address in one of the
>> pipelines.
>
> Sorry, the description was not that obvious to me.
> Can you add a comment before the 'make check*' line?
>
> # Unrelated to fuzzer: run tests with -fsanitize=address
Sure, will do.
Thomas
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-09-01 6:20 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-31 15:32 [PATCH] gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job Thomas Huth
2020-08-31 16:40 ` Alexander Bulekov
2020-09-01 6:07 ` Thomas Huth
2020-08-31 21:31 ` Philippe Mathieu-Daudé
2020-09-01 6:04 ` Thomas Huth
2020-09-01 6:12 ` Philippe Mathieu-Daudé
2020-09-01 6:19 ` Thomas Huth
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.