Re: [PATCH linux-kselftest/test v1] apparmor: add AppArmor KUnit tests for policy unpack

From: John Johansen <john.johansen@canonical.com>
To: Kees Cook <keescook@chromium.org>, Luis Chamberlain <mcgrof@kernel.org>
Cc: Brendan Higgins <brendanhiggins@google.com>,
	Alan Maguire <alan.maguire@oracle.com>,
	Matthias Maennich <maennich@google.com>, shuah <shuah@kernel.org>,
	jmorris@namei.org, serge@hallyn.com,
	Iurii Zaikin <yzaikin@google.com>,
	David Gow <davidgow@google.com>, Theodore Ts'o <tytso@mit.edu>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-security-module@vger.kernel.org,
	KUnit Development <kunit-dev@googlegroups.com>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	Mike Salvatore <mike.salvatore@canonical.com>
Subject: Re: [PATCH linux-kselftest/test v1] apparmor: add AppArmor KUnit tests for policy unpack
Date: Wed, 30 Oct 2019 18:37:24 -0700	[thread overview]
Message-ID: <f6673ccf-30e3-449f-dd8b-7b71fe829f43@canonical.com> (raw)
In-Reply-To: <201910301205.74EC2A226D@keescook>

On 10/30/19 12:09 PM, Kees Cook wrote:
> On Thu, Oct 24, 2019 at 10:15:29AM +0000, Luis Chamberlain wrote:
>> On Wed, Oct 23, 2019 at 05:42:18PM -0700, Brendan Higgins wrote:
>>> With that, I think the best solution in this case will be the
>>> "__visible_for_testing" route. It has no overhead when testing is
>>> turned off (in fact it is no different in anyway when testing is
>>> turned off). The downsides I see are:
>>>
>>> 1) You may not be able to test non-module code not compiled for
>>> testing later with the test modules that Alan is working on (But the
>>> only way I think that will work is by preventing the symbol from being
>>> inlined, right?).
>>>
>>> 2) I think "__visible_for_testing" will be prone to abuse. Here, I
>>> think there are reasons why we might want to expose these symbols for
>>> testing, but not otherwise. Nevertheless, I think most symbols that
>>> should be tested should probably be made visible by default. Since you
>>> usually only want to test your public interfaces. I could very well
>>> see this getting used as a kludge that gets used far too frequently.
>>
>> There are two parts to your statement on 2):
>>
>>   a) possible abuse of say __visible_for_testing
> 
> I really don't like the idea of littering the kernel with these. It'll
> also require chunks in header files wrapped in #ifdefs. This is really
> ugly.
> 
>>   b) you typically only want to test your public interfaces
>
I would disagree with this. Helper/lib functions benefit from testing,
and ensuring they work as expected. Having tests on these helps catch
errors when you fix bugs or make improvements. If you want its indirect
testing of public interfaces.

> True, but being able to test the little helper functions is a nice
> starting point and a good building block.
> 
yeah its a nice building block

> Why can't unit tests live with the code they're testing? They're already
> logically tied together; what's the harm there? This needn't be the case
> for ALL tests, etc. The test driver could still live externally. The
> test in the other .c would just have exported functions... ?
> 

they can, its my preference too. Or if the tests must live separate I
don't even mind the abomination of including a test .c that contains
the tests for the private fns only.

In the end though, I just want to see more unit testing and will happily
take what ever the community decides is the way to go.