From: "Philippe Mathieu-Daudé" <philmd@redhat.com>
To: Robert Foley <robert.foley@linaro.org>, qemu-devel@nongnu.org
Cc: peter.puhov@linaro.org, cota@braap.org, alex.bennee@linaro.org,
Lingfeng Yang <lfy@google.com>
Subject: Re: [PATCH 01/19] configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext
Date: Sat, 23 May 2020 18:55:44 +0200 [thread overview]
Message-ID: <cc897d62-5346-ebee-3d1e-ef5df3fc9b3f@redhat.com> (raw)
In-Reply-To: <20200522160755.886-2-robert.foley@linaro.org>
Hi Robert,
On 5/22/20 6:07 PM, Robert Foley wrote:
> From: Lingfeng Yang <lfy@google.com>
>
> We tried running QEMU under tsan in 2016, but tsan's lack of support for
> longjmp-based fibers was a blocker:
> https://groups.google.com/forum/#!topic/thread-sanitizer/se0YuzfWazw
>
> Fortunately, thread sanitizer gained fiber support in early 2019:
> https://reviews.llvm.org/D54889
>
> This patch brings tsan support upstream by importing the patch that annotated
> QEMU's coroutines as tsan fibers in Android's QEMU fork:
> https://android-review.googlesource.com/c/platform/external/qemu/+/844675
>
> Tested with '--enable-tsan --cc=clang-9 --cxx=clang++-9 --disable-werror'
> configure flags.
>
> Signed-off-by: Lingfeng Yang <lfy@google.com>
> Signed-off-by: Emilio G. Cota <cota@braap.org>
> [cota: minor modifications + configure changes]
> Signed-off-by: Robert Foley <robert.foley@linaro.org>
> [RF: minor changes to clean up checkpatch warnings/errors]
> ---
> configure | 39 ++++++++++++++++
> util/coroutine-ucontext.c | 97 +++++++++++++++++++++++++++++++++++----
> 2 files changed, 127 insertions(+), 9 deletions(-)
>
> diff --git a/configure b/configure
> index 26084fc53a..c95c54fb48 100755
> --- a/configure
> +++ b/configure
> @@ -395,6 +395,7 @@ gprof="no"
> debug_tcg="no"
> debug="no"
> sanitizers="no"
> +tsan="no"
> fortify_source=""
> strip_opt="yes"
> tcg_interpreter="no"
> @@ -1150,6 +1151,10 @@ for opt do
> ;;
> --disable-sanitizers) sanitizers="no"
> ;;
> + --enable-tsan) tsan="yes"
> + ;;
> + --disable-tsan) tsan="no"
> + ;;
> --enable-sparse) sparse="yes"
> ;;
> --disable-sparse) sparse="no"
> @@ -1750,6 +1755,7 @@ Advanced options (experts only):
> --with-pkgversion=VERS use specified string as sub-version of the package
> --enable-debug enable common debug build options
> --enable-sanitizers enable default sanitizers
> + --enable-tsan enable thread sanitizer
> --disable-strip disable stripping binaries
> --disable-werror disable compilation abort on warning
> --disable-stack-protector disable compiler-provided stack protection
> @@ -6176,6 +6182,27 @@ if test "$fuzzing" = "yes" ; then
> fi
> fi
>
> +# Thread sanitizer is, for now, much noisier than the other sanitizers;
> +# keep it separate until that is not the case.
> +have_tsan=no
> +have_tsan_iface_fiber=no
> +if test "$tsan" = "yes" ; then
> + write_c_skeleton
> + if compile_prog "$CPU_CFLAGS -Werror -fsanitize=thread" "" ; then
> + have_tsan=yes
> + fi
> + cat > $TMPC << EOF
> +#include <sanitizer/tsan_interface.h>
> +int main(void) {
> + __tsan_create_fiber(0);
> + return 0;
> +}
> +EOF
> + if compile_prog "$CPU_CFLAGS -Werror -fsanitize=thread" "" ; then
> + have_tsan_iface_fiber=yes
> + fi
> +fi
> +
> ##########################################
> # check for libpmem
>
> @@ -6277,6 +6304,14 @@ if test "$have_asan" = "yes"; then
> "Without code annotation, the report may be inferior."
> fi
> fi
> +if test "$have_tsan" = "yes" ; then
> + if test "$have_tsan_iface_fiber" = "yes" ; then
> + QEMU_CFLAGS="-fsanitize=thread $QEMU_CFLAGS"
> + QEMU_LDFLAGS="-fsanitize=thread $QEMU_LDFLAGS"
> + else
> + echo "Cannot enable TSAN due to missing fiber annotation interface."
I tried your series and there were no changes anywhere, then I looked at
how TSan work, started to debug, to finally realize my build was not
using TSan (clang8). Please use to something such:
if test "$tsan" = "yes" ; then
error_exit "Cannot enable TSAN due to missing fiber" \
"annotation interface."
fi
> + fi
> +fi
> if test "$have_ubsan" = "yes"; then
> QEMU_CFLAGS="-fsanitize=undefined $QEMU_CFLAGS"
> QEMU_LDFLAGS="-fsanitize=undefined $QEMU_LDFLAGS"
> @@ -7365,6 +7400,10 @@ if test "$have_asan_iface_fiber" = "yes" ; then
> echo "CONFIG_ASAN_IFACE_FIBER=y" >> $config_host_mak
> fi
>
> +if test "$have_tsan" = "yes" && test "$have_tsan_iface_fiber" = "yes" ; then
> + echo "CONFIG_TSAN=y" >> $config_host_mak
> +fi
> +
> if test "$has_environ" = "yes" ; then
> echo "CONFIG_HAS_ENVIRON=y" >> $config_host_mak
> fi
[...]
next prev parent reply other threads:[~2020-05-23 16:56 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-22 16:07 [PATCH 00/19] Add Thread Sanitizer support to QEMU Robert Foley
2020-05-22 16:07 ` [PATCH 01/19] configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext Robert Foley
2020-05-23 16:55 ` Philippe Mathieu-Daudé [this message]
2020-05-26 12:38 ` Robert Foley
2020-06-17 14:24 ` Stefan Hajnoczi
2020-06-17 15:56 ` Alex Bennée
2020-06-17 21:27 ` Robert Foley
2020-05-22 16:07 ` [PATCH 02/19] cpu: convert queued work to a QSIMPLEQ Robert Foley
2020-05-24 10:20 ` Philippe Mathieu-Daudé
2020-05-26 15:01 ` Robert Foley
2020-05-22 16:07 ` [PATCH 03/19] thread: add qemu_spin_destroy Robert Foley
2020-05-22 16:07 ` [PATCH 04/19] cputlb: destroy CPUTLB with tlb_destroy Robert Foley
2020-05-22 16:07 ` [PATCH 05/19] qht: call qemu_spin_destroy for head buckets Robert Foley
2020-05-22 16:07 ` [PATCH 06/19] tcg: call qemu_spin_destroy for tb->jmp_lock Robert Foley
2020-05-22 16:07 ` [PATCH 07/19] translate-all: call qemu_spin_destroy for PageDesc Robert Foley
2020-05-22 16:07 ` [PATCH 08/19] thread: add tsan annotations to QemuSpin Robert Foley
2020-05-22 16:07 ` [PATCH 09/19] tests/docker: Added docker build support for TSan Robert Foley
2020-05-22 16:07 ` [PATCH 10/19] include/qemu: Added tsan.h for annotations Robert Foley
2020-05-23 17:20 ` Emilio G. Cota
2020-05-23 21:37 ` Emilio G. Cota
2020-05-22 16:07 ` [PATCH 11/19] accel/tcg: Fixed tsan warnings related to parallel_cpus Robert Foley
2020-05-23 17:21 ` Emilio G. Cota
2020-05-22 16:07 ` [PATCH 12/19] configure: added tsan support for blacklist Robert Foley
2020-05-23 17:27 ` Emilio G. Cota
2020-05-26 14:07 ` Robert Foley
2020-05-22 16:07 ` [PATCH 13/19] accel/tcg: Fixed tsan warnings Robert Foley
2020-05-23 20:06 ` Emilio G. Cota
2020-05-26 15:14 ` Robert Foley
2020-05-26 18:47 ` Paolo Bonzini
2020-05-22 16:07 ` [PATCH 14/19] util/async: " Robert Foley
2020-05-23 20:12 ` Emilio G. Cota
2020-05-26 15:19 ` Robert Foley
2020-05-26 10:32 ` Stefan Hajnoczi
2020-05-26 15:21 ` Robert Foley
2020-05-22 16:07 ` [PATCH 15/19] qht: Fix " Robert Foley
2020-05-23 20:44 ` Emilio G. Cota
2020-05-22 16:07 ` [PATCH 16/19] util: fixed tsan warnings in thread_pool.c Robert Foley
2020-05-26 20:18 ` Paolo Bonzini
2020-05-22 16:07 ` [PATCH 17/19] util: Added tsan annotate for thread name Robert Foley
2020-05-23 20:29 ` Emilio G. Cota
2020-05-22 16:07 ` [PATCH 18/19] target/arm: Fix tsan warning in cpu.c Robert Foley
2020-05-22 17:44 ` Peter Maydell
2020-05-22 21:33 ` Robert Foley
2020-05-22 22:36 ` Peter Maydell
2020-05-23 17:18 ` Emilio G. Cota
2020-05-26 14:01 ` Robert Foley
2020-05-22 16:07 ` [PATCH 19/19] docs: Added details on TSan to testing.rst Robert Foley
2020-05-23 20:29 ` Emilio G. Cota
2020-05-22 21:07 ` [PATCH 00/19] Add Thread Sanitizer support to QEMU no-reply
2020-05-23 21:36 ` Emilio G. Cota
2020-05-26 15:18 ` Robert Foley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cc897d62-5346-ebee-3d1e-ef5df3fc9b3f@redhat.com \
--to=philmd@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=cota@braap.org \
--cc=lfy@google.com \
--cc=peter.puhov@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=robert.foley@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).