Re: [RFC PATCH] UML: add support for KASAN under x86_64

From: Dmitry Vyukov <dvyukov@google.com>
To: Johannes Berg <johannes@sipsolutions.net>
Cc: Patricia Alfonso <trishalfonso@google.com>,
	Richard Weinberger <richard@nod.at>,
	Jeff Dike <jdike@addtoit.com>,
	Brendan Higgins <brendanhiggins@google.com>,
	LKML <linux-kernel@vger.kernel.org>,
	kasan-dev <kasan-dev@googlegroups.com>,
	linux-um@lists.infradead.org, David Gow <davidgow@google.com>,
	Andrey Ryabinin <aryabinin@virtuozzo.com>,
	anton.ivanov@cambridgegreys.com
Subject: Re: [RFC PATCH] UML: add support for KASAN under x86_64
Date: Fri, 17 Jan 2020 11:03:07 +0100	[thread overview]
Message-ID: <CACT4Y+bsaZoPC1Q7_rV-e_aO=LVPA-cE3btT_VARStWYk6dcPA@mail.gmail.com> (raw)
In-Reply-To: <CACT4Y+atPME1RYvusmr2EQpv_mNkKJ2_LjMeANv0HxF=+Uu5hw@mail.gmail.com>

On Fri, Jan 17, 2020 at 10:59 AM Dmitry Vyukov <dvyukov@google.com> wrote:
>
> On Thu, Jan 16, 2020 at 10:39 PM Patricia Alfonso
> <trishalfonso@google.com> wrote:
> >
> > On Thu, Jan 16, 2020 at 1:23 AM Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > On Thu, Jan 16, 2020 at 10:20 AM Johannes Berg
> > > <johannes@sipsolutions.net> wrote:
> > > >
> > > > On Thu, 2020-01-16 at 10:18 +0100, Dmitry Vyukov wrote:
> > > > >
> > > > > Looking at this problem and at the number of KASAN_SANITIZE := n in
> > > > > Makefiles (some of which are pretty sad, e.g. ignoring string.c,
> > > > > kstrtox.c, vsprintf.c -- that's where the bugs are!), I think we
> > > > > initialize KASAN too late. I think we need to do roughly what we do in
> > > > > user-space asan (because it is user-space asan!). Constructors run
> > > > > before main and it's really good, we need to initialize KASAN from
> > > > > these constructors. Or if that's not enough in all cases, also add own
> > > > > constructor/.preinit array entry to initialize as early as possible.
> > > >
> >
> > I am not too happy with the number of KASAN_SANITIZE := n's either.
> > This sounds like a good idea. Let me look into it; I am not familiar
> > with constructors or .preint array.
> >
> > > > We even control the linker in this case, so we can put something into
> > > > the .preinit array *first*.
> > >
> > > Even better! If we can reliably put something before constructors, we
> > > don't even need lazy init in constructors.
> > >
> > > > > All we need to do is to call mmap syscall, there is really no
> > > > > dependencies on anything kernel-related.
> > > >
> > > > OK. I wasn't really familiar with those details.
> > > >
> > > > > This should resolve the problem with constructors (after they
> > > > > initialize KASAN, they can proceed to do anything they need) and it
> > > > > should get rid of most KASAN_SANITIZE (in particular, all of
> > > > > lib/Makefile and kernel/Makefile) and should fix stack instrumentation
> > > > > (in case it does not work now). The only tiny bit we should not
> > > > > instrument is the path from constructor up to mmap call.
> >
> > This sounds like a great solution. I am getting this KASAN report:
> > "BUG: KASAN: stack-out-of-bounds in syscall_stub_data+0x2a5/0x2c7",
> > which is probably because of this stack instrumentation problem you
> > point out.
>
> [reposting to the list]
>
> If that part of the code I mentioned is instrumented, manifestation
> would be different -- stack instrumentation will try to access shadow,
> shadow is not mapped yet, so it would crash on the shadow access.
>
> What you are seeing looks like, well, a kernel bug where it does a bad
> stack access. Maybe it's KASAN actually _working_? :)

Though, stack instrumentation may have issues with longjmp-like things.
I would suggest first turning off stack instrumentation and getting
that work. Solving problems one-by-one is always easier.
If you need help debugging this, please post more info: patch, what
you are doing, full kernel output (preferably from start, if it's not
too lengthy).