Re: [RFC PATCH 1/2] Port KASAN Tests to KUnit

From: Brendan Higgins <brendanhiggins@google.com>
To: Kees Cook <keescook@chromium.org>
Cc: Dmitry Vyukov <dvyukov@google.com>,
	Patricia Alfonso <trishalfonso@google.com>,
	Andrey Ryabinin <aryabinin@virtuozzo.com>,
	David Gow <davidgow@google.com>, Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	LKML <linux-kernel@vger.kernel.org>,
	kasan-dev <kasan-dev@googlegroups.com>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	KUnit Development <kunit-dev@googlegroups.com>
Subject: Re: [RFC PATCH 1/2] Port KASAN Tests to KUnit
Date: Mon, 2 Mar 2020 14:36:48 -0800	[thread overview]
Message-ID: <CAFd5g47OHZ-6Fao+JOMES+aPd2vyWXSS0zKCkSwL6XczN4R7aQ@mail.gmail.com> (raw)
In-Reply-To: <CAGXu5jKbpbH4sm4sv-74iHa+VzWuvF5v3ci7R-KVt+StRpMESg@mail.gmail.com>

On Mon, Mar 2, 2020 at 9:52 AM Kees Cook <keescook@chromium.org> wrote:
>
> On Sat, Feb 29, 2020 at 10:39 PM Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > On Sat, Feb 29, 2020 at 2:56 AM Patricia Alfonso
> > <trishalfonso@google.com> wrote:
> > > On Thu, Feb 27, 2020 at 6:19 AM Dmitry Vyukov <dvyukov@google.com> wrote:
> > > >
> > > > .On Thu, Feb 27, 2020 at 3:44 AM Patricia Alfonso
> > > > > -       pr_info("out-of-bounds in copy_from_user()\n");
> > > > > -       unused = copy_from_user(kmem, usermem, size + 1);
> > > >
> > > > Why is all of this removed?
> > > > Most of these tests are hard earned and test some special corner cases.
> > > >
> > > I just moved it inside IS_MODULE(CONFIG_TEST_KASAN) instead because I
> > > don't think there is a way to rewrite this without it being a module.
> >
> > You mean these are unconditionally crashing the machine? If yes,
> > please add a comment about this.
> >
> > Theoretically we could have a notion of "death tests" similar to gunit:
> > https://stackoverflow.com/questions/3698718/what-are-google-test-death-tests
> > KUnit test runner wrapper would need to spawn a separete process per
> > each such test. Under non-KUnit test runner these should probably be
> > disabled by default and only run if specifically requested (a-la
> > --gunit_filter/--gunit_also_run_disabled_tests).
> > Could also be used to test other things that unconditionally panic,
> > e.g. +Kees may be happy for unit tests for some of the
> > hardening/fortification features.
> > I am not asking to bundle this with this change of course.
>
> A bunch of LKDTM tests can kill the system too. I collected the list
> when building the selftest script for LKDTM:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/lkdtm/tests.txt
>
> I'm all for unittests (I have earlier kind-of-unit-tests in
> lib/test_user_copy.c lib/test_overflow.c etc), but most of LKDTM is

<Minor tangent (sorry)>

I took a brief look at lib/test_user_copy.c, it looks like it doesn't
use TAP formatted output. How do you feel about someone converting
them over to use KUnit? If nothing else, it would be good getting all
the unit-ish tests to output in the same format.

I proposed converting over some of the runtime tests over to KUnit as
a LKMP project (Linux Kernel Mentorship Program) here:

https://wiki.linuxfoundation.org/lkmp/lkmp_project_list#convert_runtime_tests_to_kunit_tests

I am curious what you think about this.

</Minor tangent>

> designed to be full system-behavior testing ("does the system correct
> BUG the current thread, when some deeper system state is violated?")

Makes sense.

Thanks!