* Re: [PATCH v3] arm64: Enable KCSAN
2021-12-02 14:33 [PATCH v3] arm64: Enable KCSAN Kefeng Wang
@ 2021-12-02 14:36 ` Marco Elver
2021-12-02 14:44 ` Mark Rutland
2021-12-02 16:13 ` Joey Gouly
2021-12-02 16:56 ` Nathan Chancellor
2 siblings, 1 reply; 8+ messages in thread
From: Marco Elver @ 2021-12-02 14:36 UTC (permalink / raw)
To: Kefeng Wang
Cc: Catalin Marinas, Will Deacon, linux-arm-kernel, linux-kernel,
mark.rutland
On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
> This patch enables KCSAN for arm64, with updates to build rules
> to not use KCSAN for several incompatible compilation units.
>
> Resent GCC version(at least GCC10) made outline-atomics as the
s/Resent/Recent/
> default option(unlike Clang), which will cause linker errors
> for kernel/kcsan/core.o. Disables the out-of-line atomics by
> no-outline-atomics to fix the linker errors.
>
> Meanwhile, as Mark said[1], there is a specific issue on arm64
> about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed
> by Clang 12, add CLANG_VERSION check. And also some latent issues
> are need to be fixed which isn't just a KCSAN problem, we make
> the KCSAN depends on EXPERT for now.
>
> Tested selftest and kcsan_test(built with GCC11 and Clang 13),
> and all passed.
>
> [1] https://lkml.org/lkml/2021/12/1/354
Please use lore/kernel.org permalinks. For this one it'd be:
https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N
(But I think if this is the final version of the patch, hopefully a
maintainer can amend the commit message.)
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Acked-by: Marco Elver <elver@google.com> # kernel/kcsan
is still valid, given nothing changed there. I leave the rest to Mark.
Thanks,
-- Marco
> ---
> Tested on Qemu with clang 13 / gcc 11, based on 5.16-rc3.
>
> [ 0.221518] kcsan: enabled early
> [ 0.222422] kcsan: strict mode configured
> ...
> [ 5.839223] kcsan: selftest: 3/3 tests passed
> ...
> [ 517.895102] # kcsan: pass:24 fail:0 skip:0 total:24
> [ 517.896393] # Totals: pass:168 fail:0 skip:0 total:168
> [ 517.897502] ok 1 - kcsan
>
> v3:
> - add EXPERT and CLANG_VERSION depends suggested by Mark Rutland
> v2:
> - tested on GCC11 and disable outline-atomics for kernel/kcsan/core.c
> suggested by Marco Elver
>
> arch/arm64/Kconfig | 1 +
> arch/arm64/kernel/vdso/Makefile | 1 +
> arch/arm64/kvm/hyp/nvhe/Makefile | 1 +
> kernel/kcsan/Makefile | 1 +
> 4 files changed, 4 insertions(+)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 4ff73299f8a9..3a7b17742cde 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -150,6 +150,7 @@ config ARM64
> select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN
> select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN
> select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE)
> + select HAVE_ARCH_KCSAN if EXPERT && (CC_IS_GCC || CLANG_VERSION >= 120000)
> select HAVE_ARCH_KFENCE
> select HAVE_ARCH_KGDB
> select HAVE_ARCH_MMAP_RND_BITS
> diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
> index 700767dfd221..60813497a381 100644
> --- a/arch/arm64/kernel/vdso/Makefile
> +++ b/arch/arm64/kernel/vdso/Makefile
> @@ -32,6 +32,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO
> CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS) \
> $(CC_FLAGS_LTO)
> KASAN_SANITIZE := n
> +KCSAN_SANITIZE := n
> UBSAN_SANITIZE := n
> OBJECT_FILES_NON_STANDARD := y
> KCOV_INSTRUMENT := n
> diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile
> index c3c11974fa3b..24b2c2425b38 100644
> --- a/arch/arm64/kvm/hyp/nvhe/Makefile
> +++ b/arch/arm64/kvm/hyp/nvhe/Makefile
> @@ -89,6 +89,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI)
> # cause crashes. Just disable it.
> GCOV_PROFILE := n
> KASAN_SANITIZE := n
> +KCSAN_SANITIZE := n
> UBSAN_SANITIZE := n
> KCOV_INSTRUMENT := n
>
> diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile
> index c2bb07f5bcc7..e893b0e1d62a 100644
> --- a/kernel/kcsan/Makefile
> +++ b/kernel/kcsan/Makefile
> @@ -8,6 +8,7 @@ CFLAGS_REMOVE_debugfs.o = $(CC_FLAGS_FTRACE)
> CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
>
> CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \
> + $(call cc-option,-mno-outline-atomics) \
> -fno-stack-protector -DDISABLE_BRANCH_PROFILING
>
> obj-y := core.o debugfs.o report.o
> --
> 2.26.2
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] arm64: Enable KCSAN
2021-12-02 14:36 ` Marco Elver
@ 2021-12-02 14:44 ` Mark Rutland
2021-12-03 2:29 ` Kefeng Wang
0 siblings, 1 reply; 8+ messages in thread
From: Mark Rutland @ 2021-12-02 14:44 UTC (permalink / raw)
To: Marco Elver
Cc: Kefeng Wang, Catalin Marinas, Will Deacon, linux-arm-kernel,
linux-kernel, joey.gouly
On Thu, Dec 02, 2021 at 03:36:06PM +0100, Marco Elver wrote:
> On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
> >
> > This patch enables KCSAN for arm64, with updates to build rules
> > to not use KCSAN for several incompatible compilation units.
> >
> > Resent GCC version(at least GCC10) made outline-atomics as the
>
> s/Resent/Recent/
>
> > default option(unlike Clang), which will cause linker errors
> > for kernel/kcsan/core.o. Disables the out-of-line atomics by
> > no-outline-atomics to fix the linker errors.
> >
> > Meanwhile, as Mark said[1], there is a specific issue on arm64
> > about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed
> > by Clang 12, add CLANG_VERSION check. And also some latent issues
> > are need to be fixed which isn't just a KCSAN problem, we make
> > the KCSAN depends on EXPERT for now.
> >
> > Tested selftest and kcsan_test(built with GCC11 and Clang 13),
> > and all passed.
> >
> > [1] https://lkml.org/lkml/2021/12/1/354
>
> Please use lore/kernel.org permalinks. For this one it'd be:
> https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N
>
> (But I think if this is the final version of the patch, hopefully a
> maintainer can amend the commit message.)
>
> > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>
> Acked-by: Marco Elver <elver@google.com> # kernel/kcsan
>
> is still valid, given nothing changed there. I leave the rest to Mark.
I want to do some compiler / config testing before I ack this (just to
make sure there isn't some latent issue I've forgotten about), but
otherwise I think this should be fine.
I'll try to have that done in the next few days.
Any other review/testing would be appreciated!
Thanks,
Mark.
>
> Thanks,
> -- Marco
>
> > ---
> > Tested on Qemu with clang 13 / gcc 11, based on 5.16-rc3.
> >
> > [ 0.221518] kcsan: enabled early
> > [ 0.222422] kcsan: strict mode configured
> > ...
> > [ 5.839223] kcsan: selftest: 3/3 tests passed
> > ...
> > [ 517.895102] # kcsan: pass:24 fail:0 skip:0 total:24
> > [ 517.896393] # Totals: pass:168 fail:0 skip:0 total:168
> > [ 517.897502] ok 1 - kcsan
> >
> > v3:
> > - add EXPERT and CLANG_VERSION depends suggested by Mark Rutland
> > v2:
> > - tested on GCC11 and disable outline-atomics for kernel/kcsan/core.c
> > suggested by Marco Elver
> >
> > arch/arm64/Kconfig | 1 +
> > arch/arm64/kernel/vdso/Makefile | 1 +
> > arch/arm64/kvm/hyp/nvhe/Makefile | 1 +
> > kernel/kcsan/Makefile | 1 +
> > 4 files changed, 4 insertions(+)
> >
> > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > index 4ff73299f8a9..3a7b17742cde 100644
> > --- a/arch/arm64/Kconfig
> > +++ b/arch/arm64/Kconfig
> > @@ -150,6 +150,7 @@ config ARM64
> > select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN
> > select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN
> > select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE)
> > + select HAVE_ARCH_KCSAN if EXPERT && (CC_IS_GCC || CLANG_VERSION >= 120000)
> > select HAVE_ARCH_KFENCE
> > select HAVE_ARCH_KGDB
> > select HAVE_ARCH_MMAP_RND_BITS
> > diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
> > index 700767dfd221..60813497a381 100644
> > --- a/arch/arm64/kernel/vdso/Makefile
> > +++ b/arch/arm64/kernel/vdso/Makefile
> > @@ -32,6 +32,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO
> > CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS) \
> > $(CC_FLAGS_LTO)
> > KASAN_SANITIZE := n
> > +KCSAN_SANITIZE := n
> > UBSAN_SANITIZE := n
> > OBJECT_FILES_NON_STANDARD := y
> > KCOV_INSTRUMENT := n
> > diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile
> > index c3c11974fa3b..24b2c2425b38 100644
> > --- a/arch/arm64/kvm/hyp/nvhe/Makefile
> > +++ b/arch/arm64/kvm/hyp/nvhe/Makefile
> > @@ -89,6 +89,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI)
> > # cause crashes. Just disable it.
> > GCOV_PROFILE := n
> > KASAN_SANITIZE := n
> > +KCSAN_SANITIZE := n
> > UBSAN_SANITIZE := n
> > KCOV_INSTRUMENT := n
> >
> > diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile
> > index c2bb07f5bcc7..e893b0e1d62a 100644
> > --- a/kernel/kcsan/Makefile
> > +++ b/kernel/kcsan/Makefile
> > @@ -8,6 +8,7 @@ CFLAGS_REMOVE_debugfs.o = $(CC_FLAGS_FTRACE)
> > CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
> >
> > CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \
> > + $(call cc-option,-mno-outline-atomics) \
> > -fno-stack-protector -DDISABLE_BRANCH_PROFILING
> >
> > obj-y := core.o debugfs.o report.o
> > --
> > 2.26.2
> >
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] arm64: Enable KCSAN
2021-12-02 14:44 ` Mark Rutland
@ 2021-12-03 2:29 ` Kefeng Wang
2021-12-03 10:41 ` Mark Rutland
0 siblings, 1 reply; 8+ messages in thread
From: Kefeng Wang @ 2021-12-03 2:29 UTC (permalink / raw)
To: Mark Rutland, Marco Elver
Cc: Catalin Marinas, Will Deacon, linux-arm-kernel, linux-kernel, joey.gouly
On 2021/12/2 22:44, Mark Rutland wrote:
> On Thu, Dec 02, 2021 at 03:36:06PM +0100, Marco Elver wrote:
>> On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>>> This patch enables KCSAN for arm64, with updates to build rules
>>> to not use KCSAN for several incompatible compilation units.
>>>
>>> Resent GCC version(at least GCC10) made outline-atomics as the
>> s/Resent/Recent/
>>
>>> default option(unlike Clang), which will cause linker errors
>>> for kernel/kcsan/core.o. Disables the out-of-line atomics by
>>> no-outline-atomics to fix the linker errors.
>>>
>>> Meanwhile, as Mark said[1], there is a specific issue on arm64
>>> about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed
>>> by Clang 12, add CLANG_VERSION check. And also some latent issues
>>> are need to be fixed which isn't just a KCSAN problem, we make
>>> the KCSAN depends on EXPERT for now.
>>>
>>> Tested selftest and kcsan_test(built with GCC11 and Clang 13),
>>> and all passed.
>>>
>>> [1] https://lkml.org/lkml/2021/12/1/354
>> Please use lore/kernel.org permalinks. For this one it'd be:
>> https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N
>>
>> (But I think if this is the final version of the patch, hopefully a
>> maintainer can amend the commit message.)
>>
>>> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>> Acked-by: Marco Elver <elver@google.com> # kernel/kcsan
>>
>> is still valid, given nothing changed there. I leave the rest to Mark.
> I want to do some compiler / config testing before I ack this (just to
> make sure there isn't some latent issue I've forgotten about), but
> otherwise I think this should be fine.
>
> I'll try to have that done in the next few days.
I will wait for some time, thanks Marco/Mark.
>
> Any other review/testing would be appreciated!
As Nathan points, commit 8cdd23c23c3d ("arm64: Restrict ARM64_BTI_KERNEL
to clang 12.0.0 and newer"), so need to add Clang version check, which
is v2 does.
is there some other requirement that we need this check, what's your option,
thanks.
>
> Thanks,
> Mark.
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v3] arm64: Enable KCSAN
2021-12-03 2:29 ` Kefeng Wang
@ 2021-12-03 10:41 ` Mark Rutland
2021-12-10 6:58 ` Kefeng Wang
0 siblings, 1 reply; 8+ messages in thread
From: Mark Rutland @ 2021-12-03 10:41 UTC (permalink / raw)
To: Kefeng Wang
Cc: Marco Elver, Catalin Marinas, Will Deacon, linux-arm-kernel,
linux-kernel, joey.gouly
On Fri, Dec 03, 2021 at 10:29:22AM +0800, Kefeng Wang wrote:
>
> On 2021/12/2 22:44, Mark Rutland wrote:
> > On Thu, Dec 02, 2021 at 03:36:06PM +0100, Marco Elver wrote:
> > > On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
> > > > This patch enables KCSAN for arm64, with updates to build rules
> > > > to not use KCSAN for several incompatible compilation units.
> > > >
> > > > Resent GCC version(at least GCC10) made outline-atomics as the
> > > s/Resent/Recent/
> > >
> > > > default option(unlike Clang), which will cause linker errors
> > > > for kernel/kcsan/core.o. Disables the out-of-line atomics by
> > > > no-outline-atomics to fix the linker errors.
> > > >
> > > > Meanwhile, as Mark said[1], there is a specific issue on arm64
> > > > about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed
> > > > by Clang 12, add CLANG_VERSION check. And also some latent issues
> > > > are need to be fixed which isn't just a KCSAN problem, we make
> > > > the KCSAN depends on EXPERT for now.
> > > >
> > > > Tested selftest and kcsan_test(built with GCC11 and Clang 13),
> > > > and all passed.
> > > >
> > > > [1] https://lkml.org/lkml/2021/12/1/354
> > > Please use lore/kernel.org permalinks. For this one it'd be:
> > > https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N
> > >
> > > (But I think if this is the final version of the patch, hopefully a
> > > maintainer can amend the commit message.)
> > >
> > > > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> > > Acked-by: Marco Elver <elver@google.com> # kernel/kcsan
> > >
> > > is still valid, given nothing changed there. I leave the rest to Mark.
> > I want to do some compiler / config testing before I ack this (just to
> > make sure there isn't some latent issue I've forgotten about), but
> > otherwise I think this should be fine.
> >
> > I'll try to have that done in the next few days.
>
> I will wait for some time, thanks Marco/Mark.
>
> >
> > Any other review/testing would be appreciated!
>
> As Nathan points, commit 8cdd23c23c3d ("arm64: Restrict ARM64_BTI_KERNEL
>
> to clang 12.0.0 and newer"), so need to add Clang version check, which is v2
> does.
>
> is there some other requirement that we need this check, what's your option,
I'm not immediately aware of another reason, so I'm fine with not checking that
for KCSAN -- If I discover another reason I will let you know.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v3] arm64: Enable KCSAN
2021-12-03 10:41 ` Mark Rutland
@ 2021-12-10 6:58 ` Kefeng Wang
0 siblings, 0 replies; 8+ messages in thread
From: Kefeng Wang @ 2021-12-10 6:58 UTC (permalink / raw)
To: Mark Rutland
Cc: Marco Elver, Catalin Marinas, Will Deacon, linux-arm-kernel,
linux-kernel, joey.gouly
On 2021/12/3 18:41, Mark Rutland wrote:
> On Fri, Dec 03, 2021 at 10:29:22AM +0800, Kefeng Wang wrote:
>> On 2021/12/2 22:44, Mark Rutland wrote:
>>> On Thu, Dec 02, 2021 at 03:36:06PM +0100, Marco Elver wrote:
>>>> On Thu, 2 Dec 2021 at 15:23, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>>>>> This patch enables KCSAN for arm64, with updates to build rules
>>>>> to not use KCSAN for several incompatible compilation units.
>>>>>
>>>>> Resent GCC version(at least GCC10) made outline-atomics as the
>>>> s/Resent/Recent/
>>>>
>>>>> default option(unlike Clang), which will cause linker errors
>>>>> for kernel/kcsan/core.o. Disables the out-of-line atomics by
>>>>> no-outline-atomics to fix the linker errors.
>>>>>
>>>>> Meanwhile, as Mark said[1], there is a specific issue on arm64
>>>>> about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed
>>>>> by Clang 12, add CLANG_VERSION check. And also some latent issues
>>>>> are need to be fixed which isn't just a KCSAN problem, we make
>>>>> the KCSAN depends on EXPERT for now.
>>>>>
>>>>> Tested selftest and kcsan_test(built with GCC11 and Clang 13),
>>>>> and all passed.
>>>>>
>>>>> [1] https://lkml.org/lkml/2021/12/1/354
>>>> Please use lore/kernel.org permalinks. For this one it'd be:
>>>> https://lkml.kernel.org/r/YadiUPpJ0gADbiHQ@FVFF77S0Q05N
>>>>
>>>> (But I think if this is the final version of the patch, hopefully a
>>>> maintainer can amend the commit message.)
>>>>
>>>>> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>>>> Acked-by: Marco Elver <elver@google.com> # kernel/kcsan
>>>>
>>>> is still valid, given nothing changed there. I leave the rest to Mark.
>>> I want to do some compiler / config testing before I ack this (just to
>>> make sure there isn't some latent issue I've forgotten about), but
>>> otherwise I think this should be fine.
>>>
>>> I'll try to have that done in the next few days.
>> I will wait for some time, thanks Marco/Mark.
Hi Mark, I will send v4 to address the comments in v3, is there new
update about this?
>>
>>> Any other review/testing would be appreciated!
>> As Nathan points, commit 8cdd23c23c3d ("arm64: Restrict ARM64_BTI_KERNEL
>>
>> to clang 12.0.0 and newer"), so need to add Clang version check, which is v2
>> does.
>>
>> is there some other requirement that we need this check, what's your option,
> I'm not immediately aware of another reason, so I'm fine with not checking that
> for KCSAN -- If I discover another reason I will let you know.
Ok, will drop the Clang version check in v4.
Thanks.
>
> Thanks,
> Mark.
> .
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] arm64: Enable KCSAN
2021-12-02 14:33 [PATCH v3] arm64: Enable KCSAN Kefeng Wang
2021-12-02 14:36 ` Marco Elver
@ 2021-12-02 16:13 ` Joey Gouly
2021-12-02 16:56 ` Nathan Chancellor
2 siblings, 0 replies; 8+ messages in thread
From: Joey Gouly @ 2021-12-02 16:13 UTC (permalink / raw)
To: Kefeng Wang
Cc: Marco Elver, Catalin Marinas, Will Deacon, linux-arm-kernel,
linux-kernel, mark.rutland, nd
Hi all,
On Thu, Dec 02, 2021 at 10:33:31PM +0800, Kefeng Wang wrote:
> This patch enables KCSAN for arm64, with updates to build rules
> to not use KCSAN for several incompatible compilation units.
>
> Resent GCC version(at least GCC10) made outline-atomics as the
> default option(unlike Clang), which will cause linker errors
> for kernel/kcsan/core.o. Disables the out-of-line atomics by
> no-outline-atomics to fix the linker errors.
>
> Meanwhile, as Mark said[1], there is a specific issue on arm64
> about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed
> by Clang 12, add CLANG_VERSION check. And also some latent issues
> are need to be fixed which isn't just a KCSAN problem, we make
> the KCSAN depends on EXPERT for now.
>
> Tested selftest and kcsan_test(built with GCC11 and Clang 13),
> and all passed.
>
> [1] https://lkml.org/lkml/2021/12/1/354
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Confirming that I get KCSAN errors such as:
[ 285.188576] ==================================================================
[ 285.189514] BUG: KCSAN: data-race in el0_svc_common.constprop.0+0x84/0x14c
[ 285.190599]
[ 285.191092] race at unknown origin, with read to 0xffff0000e3ce3700 of 8 bytes by task 19946 on cpu 5:
[ 285.192576] el0_svc_common.constprop.0+0x84/0x14c
[ 285.193601] do_el0_svc+0x30/0x40
[ 285.194457] el0_svc+0x3c/0x70
[ 285.194954] el0t_64_sync_handler+0x9c/0x120
[ 285.195618] el0t_64_sync+0x18c/0x190
[ 285.196371]
[ 285.196823] value changed: 0x0000000000000008 -> 0x000000000000000a
[ 285.197674]
[ 285.198172] Reported by Kernel Concurrency Sanitizer on:
[ 285.198941] CPU: 5 PID: 19946 Comm: stress-ng Not tainted 5.16.0-rc3+ #842 95517f70e70f029451088b9dec0b9074aa29ae52
[ 285.201191] ==================================================================
This particular case is fixed by Mark's thread flag series [1], which is in linux-next now.
Tested with gcc 11 and clang 14.0 (built from git) on qemu and FVP.
Tested-by: Joey Gouly <joey.gouly@arm.com>
Thanks,
Joey
[1] https://lore.kernel.org/lkml/20211129130653.2037928-1-mark.rutland@arm.com/
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] arm64: Enable KCSAN
2021-12-02 14:33 [PATCH v3] arm64: Enable KCSAN Kefeng Wang
2021-12-02 14:36 ` Marco Elver
2021-12-02 16:13 ` Joey Gouly
@ 2021-12-02 16:56 ` Nathan Chancellor
2 siblings, 0 replies; 8+ messages in thread
From: Nathan Chancellor @ 2021-12-02 16:56 UTC (permalink / raw)
To: Kefeng Wang
Cc: Marco Elver, Catalin Marinas, Will Deacon, linux-arm-kernel,
linux-kernel, mark.rutland, llvm
On Thu, Dec 02, 2021 at 10:33:31PM +0800, Kefeng Wang wrote:
> This patch enables KCSAN for arm64, with updates to build rules
> to not use KCSAN for several incompatible compilation units.
>
> Resent GCC version(at least GCC10) made outline-atomics as the
> default option(unlike Clang), which will cause linker errors
> for kernel/kcsan/core.o. Disables the out-of-line atomics by
> no-outline-atomics to fix the linker errors.
>
> Meanwhile, as Mark said[1], there is a specific issue on arm64
> about ARM64_BTI with Clang 11 if KCSAN enabled, which is fixed
> by Clang 12, add CLANG_VERSION check. And also some latent issues
I do not think this addition is necessary because we make BTI depend on
clang 12 or newer after commit 8cdd23c23c3d ("arm64: Restrict
ARM64_BTI_KERNEL to clang 12.0.0 and newer") upstream. I figured it was
better to go after the root cause rather than add a bunch of version
checks like this to the various sanitizers.
> are need to be fixed which isn't just a KCSAN problem, we make
> the KCSAN depends on EXPERT for now.
>
> Tested selftest and kcsan_test(built with GCC11 and Clang 13),
> and all passed.
>
> [1] https://lkml.org/lkml/2021/12/1/354
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> ---
> Tested on Qemu with clang 13 / gcc 11, based on 5.16-rc3.
>
> [ 0.221518] kcsan: enabled early
> [ 0.222422] kcsan: strict mode configured
> ...
> [ 5.839223] kcsan: selftest: 3/3 tests passed
> ...
> [ 517.895102] # kcsan: pass:24 fail:0 skip:0 total:24
> [ 517.896393] # Totals: pass:168 fail:0 skip:0 total:168
> [ 517.897502] ok 1 - kcsan
>
> v3:
> - add EXPERT and CLANG_VERSION depends suggested by Mark Rutland
> v2:
> - tested on GCC11 and disable outline-atomics for kernel/kcsan/core.c
> suggested by Marco Elver
>
> arch/arm64/Kconfig | 1 +
> arch/arm64/kernel/vdso/Makefile | 1 +
> arch/arm64/kvm/hyp/nvhe/Makefile | 1 +
> kernel/kcsan/Makefile | 1 +
> 4 files changed, 4 insertions(+)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 4ff73299f8a9..3a7b17742cde 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -150,6 +150,7 @@ config ARM64
> select HAVE_ARCH_KASAN_VMALLOC if HAVE_ARCH_KASAN
> select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN
> select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE)
> + select HAVE_ARCH_KCSAN if EXPERT && (CC_IS_GCC || CLANG_VERSION >= 120000)
> select HAVE_ARCH_KFENCE
> select HAVE_ARCH_KGDB
> select HAVE_ARCH_MMAP_RND_BITS
> diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
> index 700767dfd221..60813497a381 100644
> --- a/arch/arm64/kernel/vdso/Makefile
> +++ b/arch/arm64/kernel/vdso/Makefile
> @@ -32,6 +32,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO
> CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS) \
> $(CC_FLAGS_LTO)
> KASAN_SANITIZE := n
> +KCSAN_SANITIZE := n
> UBSAN_SANITIZE := n
> OBJECT_FILES_NON_STANDARD := y
> KCOV_INSTRUMENT := n
> diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile
> index c3c11974fa3b..24b2c2425b38 100644
> --- a/arch/arm64/kvm/hyp/nvhe/Makefile
> +++ b/arch/arm64/kvm/hyp/nvhe/Makefile
> @@ -89,6 +89,7 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI)
> # cause crashes. Just disable it.
> GCOV_PROFILE := n
> KASAN_SANITIZE := n
> +KCSAN_SANITIZE := n
> UBSAN_SANITIZE := n
> KCOV_INSTRUMENT := n
>
> diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile
> index c2bb07f5bcc7..e893b0e1d62a 100644
> --- a/kernel/kcsan/Makefile
> +++ b/kernel/kcsan/Makefile
> @@ -8,6 +8,7 @@ CFLAGS_REMOVE_debugfs.o = $(CC_FLAGS_FTRACE)
> CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
>
> CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \
> + $(call cc-option,-mno-outline-atomics) \
> -fno-stack-protector -DDISABLE_BRANCH_PROFILING
>
> obj-y := core.o debugfs.o report.o
> --
> 2.26.2
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread