Re: KCSAN Support on ARM64 Kernel

Re: KCSAN Support on ARM64 Kernel

[Dmitry Vyukov]

On Mon, Oct 14, 2019 at 7:11 AM <sgrover@codeaurora.org> wrote:
>
> Hi Dmitry,
>
> I am from Qualcomm Linux Security Team, just going through KCSAN and found that there was a thread for arm64 support (https://lkml.org/lkml/2019/9/20/804).
>
> Can you please tell me if KCSAN is supported on ARM64 now? Can I just rebase the KCSAN branch on top of our let’s say android mainline kernel, enable the config and run syzkaller on that for finding race conditions?
>
> It would be very helpful if you reply, we want to setup this for finding issues on our proprietary modules that are not part of kernel mainline.
>
> Regards,
>
> Sachin Grover

+more people re KCSAN on ARM64

Re: KCSAN Support on ARM64 Kernel

[Marco Elver]

On Mon, 14 Oct 2019 at 10:40, Dmitry Vyukov <dvyukov@google.com> wrote:
>
> On Mon, Oct 14, 2019 at 7:11 AM <sgrover@codeaurora.org> wrote:
> >
> > Hi Dmitry,
> >
> > I am from Qualcomm Linux Security Team, just going through KCSAN and found that there was a thread for arm64 support (https://lkml.org/lkml/2019/9/20/804).
> >
> > Can you please tell me if KCSAN is supported on ARM64 now? Can I just rebase the KCSAN branch on top of our let’s say android mainline kernel, enable the config and run syzkaller on that for finding race conditions?
> >
> > It would be very helpful if you reply, we want to setup this for finding issues on our proprietary modules that are not part of kernel mainline.
> >
> > Regards,
> >
> > Sachin Grover
>
> +more people re KCSAN on ARM64

KCSAN does not yet have ARM64 support. Once it's upstream, I would
expect that Mark's patches (from repo linked in LKML thread) will just
cleanly apply to enable ARM64 support.

Thanks,
-- Marco

RE: KCSAN Support on ARM64 Kernel

[sgrover]

Hi Marco,

When can we expect upstream of KCSAN on kernel mainline. Any timeline?

Regards,
Sachin Grover

-----Original Message-----
From: Marco Elver <elver@google.com> 
Sent: Monday, 14 October, 2019 2:40 PM
To: Dmitry Vyukov <dvyukov@google.com>
Cc: sgrover@codeaurora.org; kasan-dev <kasan-dev@googlegroups.com>; LKML <linux-kernel@vger.kernel.org>; Paul E. McKenney <paulmck@linux.ibm.com>; Will Deacon <willdeacon@google.com>; Andrea Parri <parri.andrea@gmail.com>; Alan Stern <stern@rowland.harvard.edu>; Mark Rutland <mark.rutland@arm.com>
Subject: Re: KCSAN Support on ARM64 Kernel

On Mon, 14 Oct 2019 at 10:40, Dmitry Vyukov <dvyukov@google.com> wrote:
>
> On Mon, Oct 14, 2019 at 7:11 AM <sgrover@codeaurora.org> wrote:
> >
> > Hi Dmitry,
> >
> > I am from Qualcomm Linux Security Team, just going through KCSAN and found that there was a thread for arm64 support (https://lkml.org/lkml/2019/9/20/804).
> >
> > Can you please tell me if KCSAN is supported on ARM64 now? Can I just rebase the KCSAN branch on top of our let’s say android mainline kernel, enable the config and run syzkaller on that for finding race conditions?
> >
> > It would be very helpful if you reply, we want to setup this for finding issues on our proprietary modules that are not part of kernel mainline.
> >
> > Regards,
> >
> > Sachin Grover
>
> +more people re KCSAN on ARM64

KCSAN does not yet have ARM64 support. Once it's upstream, I would expect that Mark's patches (from repo linked in LKML thread) will just cleanly apply to enable ARM64 support.

Thanks,
-- Marco

Re: KCSAN Support on ARM64 Kernel

[Marco Elver]

Hi Sachin,

My plan was to send patches upstream within the month.

Thanks,
-- Marco

On Mon, 14 Oct 2019 at 11:30, <sgrover@codeaurora.org> wrote:
>
> Hi Marco,
>
> When can we expect upstream of KCSAN on kernel mainline. Any timeline?
>
> Regards,
> Sachin Grover
>
> -----Original Message-----
> From: Marco Elver <elver@google.com>
> Sent: Monday, 14 October, 2019 2:40 PM
> To: Dmitry Vyukov <dvyukov@google.com>
> Cc: sgrover@codeaurora.org; kasan-dev <kasan-dev@googlegroups.com>; LKML <linux-kernel@vger.kernel.org>; Paul E. McKenney <paulmck@linux.ibm.com>; Will Deacon <willdeacon@google.com>; Andrea Parri <parri.andrea@gmail.com>; Alan Stern <stern@rowland.harvard.edu>; Mark Rutland <mark.rutland@arm.com>
> Subject: Re: KCSAN Support on ARM64 Kernel
>
> On Mon, 14 Oct 2019 at 10:40, Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > On Mon, Oct 14, 2019 at 7:11 AM <sgrover@codeaurora.org> wrote:
> > >
> > > Hi Dmitry,
> > >
> > > I am from Qualcomm Linux Security Team, just going through KCSAN and found that there was a thread for arm64 support (https://lkml.org/lkml/2019/9/20/804).
> > >
> > > Can you please tell me if KCSAN is supported on ARM64 now? Can I just rebase the KCSAN branch on top of our let’s say android mainline kernel, enable the config and run syzkaller on that for finding race conditions?
> > >
> > > It would be very helpful if you reply, we want to setup this for finding issues on our proprietary modules that are not part of kernel mainline.
> > >
> > > Regards,
> > >
> > > Sachin Grover
> >
> > +more people re KCSAN on ARM64
>
> KCSAN does not yet have ARM64 support. Once it's upstream, I would expect that Mark's patches (from repo linked in LKML thread) will just cleanly apply to enable ARM64 support.
>
> Thanks,
> -- Marco
>

Re: KCSAN Support on ARM64 Kernel

[Mark Rutland]

On Mon, Oct 14, 2019 at 11:09:40AM +0200, Marco Elver wrote:
> On Mon, 14 Oct 2019 at 10:40, Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > On Mon, Oct 14, 2019 at 7:11 AM <sgrover@codeaurora.org> wrote:
> > >
> > > Hi Dmitry,
> > >
> > > I am from Qualcomm Linux Security Team, just going through KCSAN
> > > and found that there was a thread for arm64 support
> > > (https://lkml.org/lkml/2019/9/20/804).
> > >
> > > Can you please tell me if KCSAN is supported on ARM64 now? Can I
> > > just rebase the KCSAN branch on top of our let’s say android
> > > mainline kernel, enable the config and run syzkaller on that for
> > > finding race conditions?
> > >
> > > It would be very helpful if you reply, we want to setup this for
> > > finding issues on our proprietary modules that are not part of
> > > kernel mainline.
> > >
> > > Regards,
> > >
> > > Sachin Grover
> >
> > +more people re KCSAN on ARM64
> 
> KCSAN does not yet have ARM64 support. Once it's upstream, I would
> expect that Mark's patches (from repo linked in LKML thread) will just
> cleanly apply to enable ARM64 support.

Once the core kcsan bits are ready, I'll rebase the arm64 patch atop.
I'm expecting some things to change as part of review, so it'd be great
to see that posted ASAP.

For arm64 I'm not expecting major changes (other than those necessary to
handle the arm64 atomic rework that went in to v5.4-rc1)

FWIW, I was able to run Syzkaller atop of my arm64/kcsan branch, but
it's very noisy as it has none of the core fixes.

Thanks,
Mark.

RE: KCSAN Support on ARM64 Kernel

[sgrover]

Hi All,

Is there any update in Arm64 support of KCSAN.

Regards,
Sachin Grover

-----Original Message-----
From: Mark Rutland <mark.rutland@arm.com> 
Sent: Monday, 14 October, 2019 3:50 PM
To: Marco Elver <elver@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>; sgrover@codeaurora.org; kasan-dev <kasan-dev@googlegroups.com>; LKML <linux-kernel@vger.kernel.org>; Paul E. McKenney <paulmck@linux.ibm.com>; Will Deacon <willdeacon@google.com>; Andrea Parri <parri.andrea@gmail.com>; Alan Stern <stern@rowland.harvard.edu>
Subject: Re: KCSAN Support on ARM64 Kernel

On Mon, Oct 14, 2019 at 11:09:40AM +0200, Marco Elver wrote:
> On Mon, 14 Oct 2019 at 10:40, Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > On Mon, Oct 14, 2019 at 7:11 AM <sgrover@codeaurora.org> wrote:
> > >
> > > Hi Dmitry,
> > >
> > > I am from Qualcomm Linux Security Team, just going through KCSAN 
> > > and found that there was a thread for arm64 support 
> > > (https://lkml.org/lkml/2019/9/20/804).
> > >
> > > Can you please tell me if KCSAN is supported on ARM64 now? Can I 
> > > just rebase the KCSAN branch on top of our let’s say android 
> > > mainline kernel, enable the config and run syzkaller on that for 
> > > finding race conditions?
> > >
> > > It would be very helpful if you reply, we want to setup this for 
> > > finding issues on our proprietary modules that are not part of 
> > > kernel mainline.
> > >
> > > Regards,
> > >
> > > Sachin Grover
> >
> > +more people re KCSAN on ARM64
> 
> KCSAN does not yet have ARM64 support. Once it's upstream, I would 
> expect that Mark's patches (from repo linked in LKML thread) will just 
> cleanly apply to enable ARM64 support.

Once the core kcsan bits are ready, I'll rebase the arm64 patch atop.
I'm expecting some things to change as part of review, so it'd be great to see that posted ASAP.

For arm64 I'm not expecting major changes (other than those necessary to handle the arm64 atomic rework that went in to v5.4-rc1)

FWIW, I was able to run Syzkaller atop of my arm64/kcsan branch, but it's very noisy as it has none of the core fixes.

Thanks,
Mark.

Re: KCSAN Support on ARM64 Kernel

[Marco Elver]

We're in the process of upstreaming KCSAN, which will simplify a lot
of things including Arm64 support. So far KCSAN is not yet in
mainline, but I assume when that happens (if things go well, very
soon) it should be trivial to add Arm64 support based on Mark's
prototype.

Thanks,
-- Marco

On Mon, 2 Dec 2019 at 06:07, <sgrover@codeaurora.org> wrote:
>
> Hi All,
>
> Is there any update in Arm64 support of KCSAN.
>
> Regards,
> Sachin Grover
>
> -----Original Message-----
> From: Mark Rutland <mark.rutland@arm.com>
> Sent: Monday, 14 October, 2019 3:50 PM
> To: Marco Elver <elver@google.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>; sgrover@codeaurora.org; kasan-dev <kasan-dev@googlegroups.com>; LKML <linux-kernel@vger.kernel.org>; Paul E. McKenney <paulmck@linux.ibm.com>; Will Deacon <willdeacon@google.com>; Andrea Parri <parri.andrea@gmail.com>; Alan Stern <stern@rowland.harvard.edu>
> Subject: Re: KCSAN Support on ARM64 Kernel
>
> On Mon, Oct 14, 2019 at 11:09:40AM +0200, Marco Elver wrote:
> > On Mon, 14 Oct 2019 at 10:40, Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > On Mon, Oct 14, 2019 at 7:11 AM <sgrover@codeaurora.org> wrote:
> > > >
> > > > Hi Dmitry,
> > > >
> > > > I am from Qualcomm Linux Security Team, just going through KCSAN
> > > > and found that there was a thread for arm64 support
> > > > (https://lkml.org/lkml/2019/9/20/804).
> > > >
> > > > Can you please tell me if KCSAN is supported on ARM64 now? Can I
> > > > just rebase the KCSAN branch on top of our let’s say android
> > > > mainline kernel, enable the config and run syzkaller on that for
> > > > finding race conditions?
> > > >
> > > > It would be very helpful if you reply, we want to setup this for
> > > > finding issues on our proprietary modules that are not part of
> > > > kernel mainline.
> > > >
> > > > Regards,
> > > >
> > > > Sachin Grover
> > >
> > > +more people re KCSAN on ARM64
> >
> > KCSAN does not yet have ARM64 support. Once it's upstream, I would
> > expect that Mark's patches (from repo linked in LKML thread) will just
> > cleanly apply to enable ARM64 support.
>
> Once the core kcsan bits are ready, I'll rebase the arm64 patch atop.
> I'm expecting some things to change as part of review, so it'd be great to see that posted ASAP.
>
> For arm64 I'm not expecting major changes (other than those necessary to handle the arm64 atomic rework that went in to v5.4-rc1)
>
> FWIW, I was able to run Syzkaller atop of my arm64/kcsan branch, but it's very noisy as it has none of the core fixes.
>
> Thanks,
> Mark.
>

Re: KCSAN Support on ARM64 Kernel

[Mark Rutland]

Hi Mukesh,

In future *please* reply in plaintext rather than HTML.

On Thu, Dec 12, 2019 at 04:22:30PM +0000, Mukesh Ojha wrote:
> On 10/14/2019 3:49 PM, Mark Rutland wrote:
> > Once the core kcsan bits are ready, I'll rebase the arm64 patch atop.
> > I'm expecting some things to change as part of review, so it'd be great
> > to see that posted ASAP.
> > 
> > For arm64 I'm not expecting major changes (other than those necessary to
> > handle the arm64 atomic rework that went in to v5.4-rc1)
> 
> Hi Mark,
> 
> Are the below patches enough for kcsan to be working on arm64 ?

That depends on what branch you're using as a base. My arm64/kcsan
branch worked for me as-is, but as I mentioned that was /very/ noisy.
Both the kcsan code and the arm64 code have moved on since then, and I
have no idea how well that would backport.

I had a quick go at porting my arm64 patch atop the kcsan branch in
Paul's tree, and that doesn't get as far as producing earlycon output,
so more work will be necessary to investigate and debug that.

I hope to look at that, but I don't think I'll have the chance to do so
before the end of next week.

> I am not sure about the one you are mentioning about "atomic rework patches
> which went in 5.4 rc1" .

There were a number of patches from Andrew Murray reworking the arm64
atomic implementation. See:

$ git log v5.3..v5.4-rc1 --author='Andrew Murray' -- arch/arm64

With those patches applied, my change to arch/arm64/lib/Makefile is
unnecessary and can be dropped.

Thanks,
Mark.

> 2019-10-03 	arm64, kcsan: enable KCSAN for arm64 <https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/commit/?h=arm64/kcsan&id=ae1d089527027ce710e464105a73eb0db27d7875>arm64/kcsan <https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/log/?h=arm64/kcsan>
> 	Mark Rutland 	5 	-1/+5
> 
> 	
> 	
> 	
> 	
> 2019-09-24 	locking/atomics, kcsan: Add KCSAN instrumentation <https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/commit/?h=arm64/kcsan&id=8b3b76ec443b9af7e55994a163bb6f4aee016f09>
> 	Marco Elver 	2 	-2/+199
> 2019-09-24 	asm-generic, kcsan: Add KCSAN instrumentation for bitops <https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/commit/?h=arm64/kcsan&id=50c23ad00c040927e71c8943d4eb7d52e9f77762>
> 	Marco Elver 	1 	-0/+18
> 2019-09-24 	seqlock, kcsan: Add annotations for KCSAN <https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/commit/?h=arm64/kcsan&id=e2b32e1a3b397bffcb6afbe86f6fe55e2040a34a>
> 	Marco Elver 	1 	-5/+42
> 2019-09-24 	build, kcsan: Add KCSAN build exceptions <https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/commit/?h=arm64/kcsan&id=35a907033244099a71f17d28e9ffaca92f714463>
> 	Marco Elver 	3 	-0/+17
> 2019-09-24 	objtool, kcsan: Add KCSAN runtime functions to whitelist <https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/commit/?h=arm64/kcsan&id=3afc592ca7ebd9c13c939c98b995763345e85e08>
> 	Marco Elver 	1 	-0/+17
> 2019-09-24 	kcsan: Add Kernel Concurrency Sanitizer infrastructure <https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/commit/?h=arm64/kcsan&id=73d893b441dc3e5c1645884a19b46a1bfd4fd692>
> 	Marco Elver
> 

Re: KCSAN Support on ARM64 Kernel

[Marco Elver]

On Mon, 14 Oct 2019 at 11:31, Marco Elver <elver@google.com> wrote:
> My plan was to send patches upstream within the month.
[...]
> On Mon, 14 Oct 2019 at 11:30, <sgrover@codeaurora.org> wrote:
[...]
> > When can we expect upstream of KCSAN on kernel mainline. Any timeline?
[...]
> > > > Can you please tell me if KCSAN is supported on ARM64 now? Can I just rebase the KCSAN branch on top of our let’s say android mainline kernel, enable the config and run syzkaller on that for finding race conditions?
[...]
> > KCSAN does not yet have ARM64 support. Once it's upstream, I would expect that Mark's patches (from repo linked in LKML thread) will just cleanly apply to enable ARM64 support.

Just FYI, KCSAN is in mainline now. I believe porting it to other
architectures has also become much simpler due to its reworked
ONCE/atomic support relying on proper compiler instrumentation instead
of other tricks.

Thanks,
-- Marco