Re: [PATCH linux-kselftest/test v1] apparmor: add AppArmor KUnit tests for policy unpack

From: Kees Cook <keescook@chromium.org>
To: Luis Chamberlain <mcgrof@kernel.org>
Cc: Brendan Higgins <brendanhiggins@google.com>,
	Alan Maguire <alan.maguire@oracle.com>,
	Matthias Maennich <maennich@google.com>, shuah <shuah@kernel.org>,
	John Johansen <john.johansen@canonical.com>,
	jmorris@namei.org, serge@hallyn.com,
	Iurii Zaikin <yzaikin@google.com>,
	David Gow <davidgow@google.com>, Theodore Ts'o <tytso@mit.edu>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-security-module@vger.kernel.org,
	KUnit Development <kunit-dev@googlegroups.com>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	Mike Salvatore <mike.salvatore@canonical.com>
Subject: Re: [PATCH linux-kselftest/test v1] apparmor: add AppArmor KUnit tests for policy unpack
Date: Wed, 30 Oct 2019 12:09:40 -0700	[thread overview]
Message-ID: <201910301205.74EC2A226D@keescook> (raw)
In-Reply-To: <20191024101529.GK11244@42.do-not-panic.com>

On Thu, Oct 24, 2019 at 10:15:29AM +0000, Luis Chamberlain wrote:
> On Wed, Oct 23, 2019 at 05:42:18PM -0700, Brendan Higgins wrote:
> > With that, I think the best solution in this case will be the
> > "__visible_for_testing" route. It has no overhead when testing is
> > turned off (in fact it is no different in anyway when testing is
> > turned off). The downsides I see are:
> > 
> > 1) You may not be able to test non-module code not compiled for
> > testing later with the test modules that Alan is working on (But the
> > only way I think that will work is by preventing the symbol from being
> > inlined, right?).
> > 
> > 2) I think "__visible_for_testing" will be prone to abuse. Here, I
> > think there are reasons why we might want to expose these symbols for
> > testing, but not otherwise. Nevertheless, I think most symbols that
> > should be tested should probably be made visible by default. Since you
> > usually only want to test your public interfaces. I could very well
> > see this getting used as a kludge that gets used far too frequently.
> 
> There are two parts to your statement on 2):
> 
>   a) possible abuse of say __visible_for_testing

I really don't like the idea of littering the kernel with these. It'll
also require chunks in header files wrapped in #ifdefs. This is really
ugly.

>   b) you typically only want to test your public interfaces

True, but being able to test the little helper functions is a nice
starting point and a good building block.

Why can't unit tests live with the code they're testing? They're already
logically tied together; what's the harm there? This needn't be the case
for ALL tests, etc. The test driver could still live externally. The
test in the other .c would just have exported functions... ?

-- 
Kees Cook