linux-kselftest.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Brendan Higgins <brendanhiggins@google.com>
To: "Theodore Y. Ts'o" <tytso@mit.edu>
Cc: shuah <shuah@kernel.org>,
	John Johansen <john.johansen@canonical.com>,
	jmorris@namei.org, serge@hallyn.com,
	Kees Cook <keescook@chromium.org>,
	Alan Maguire <alan.maguire@oracle.com>,
	Iurii Zaikin <yzaikin@google.com>,
	David Gow <davidgow@google.com>,
	Luis Chamberlain <mcgrof@kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-security-module@vger.kernel.org,
	KUnit Development <kunit-dev@googlegroups.com>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	Mike Salvatore <mike.salvatore@canonical.com>
Subject: Re: [PATCH linux-kselftest/test v1] apparmor: add AppArmor KUnit tests for policy unpack
Date: Fri, 18 Oct 2019 14:41:38 -0700	[thread overview]
Message-ID: <CAFd5g45LmnbD7L4LqdbfBV5YR377e81m61+z==RKCGjWBFqDGQ@mail.gmail.com> (raw)
In-Reply-To: <20191018162519.GH21137@mit.edu>

On Fri, Oct 18, 2019 at 9:25 AM Theodore Y. Ts'o <tytso@mit.edu> wrote:
>
> On Thu, Oct 17, 2019 at 05:43:07PM -0700, Brendan Higgins wrote:
> > > +config SECURITY_APPARMOR_TEST
> > > +   bool "Build KUnit tests for policy_unpack.c"
> > > +   default n
> > > +   depends on KUNIT && SECURITY_APPARMOR
> >
> > Ted, here is an example where doing select on direct dependencies is
> > tricky because SECURITY_APPARMOR has a number of indirect dependencies.
>
> Well, that could be solved by adding a select on all of the indirect
> dependencies.  I did get your point about the fact that we could have

In this particular case that would work.

> cases where the indirect dependencies might conflict with one another.
> That's going to be a tough situation regardless of whether we have a
> sat-solver or a human who has to struggle with that situation.

But yeah, that's the real problem.

> It's also going to be a bit sad because it means that we won't be able
> to create a single config that could be used to run all the kunit
> tests when a user pushes a change to a Gerrit server for review.  :-/

Yeah...well, we can do the next best thing and generate a set of
kunitconfigs that in sum will run all the tests. Not nearly as nice,
but it's the next best thing, right? If you think about it, it's
really not all that different from the eventual goal of having many
independent test binaries.

> I suppose that if we use a strict definition of "unit tests", and we
> assume that all of the tests impacted by a change in foo/bar/baz.c
> will be found in foo/bar/baz-test.c, or maybe foo/bar/*-test.c, we can
> automate the generation of the kunitconfig file, perhaps?

Possibly. I have some friends on the TAP team (automated testing team
within Google), and it sounds like that is actually a pretty hard
problem, but something that is at least possible. Still, it would be
nice to have a way to periodically run all the tests.

> The other sad bit about having mutually exclusive config options is
> that we can't easily "run all KUinit tests" for some kind of test
> spinner or zero-day bot.
>
> I'm not sure there's a good solution to that issue, though.

I think, as I mentioned above, the best we can do is probably have a
thing which generates a set of kunitconfigs that in sum will run all
the tests.

Thoughts?

  reply	other threads:[~2019-10-18 21:41 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-18  0:18 Brendan Higgins
2019-10-18  0:33 ` Iurii Zaikin
2019-10-30 18:59   ` Kees Cook
2019-11-06  0:35     ` Brendan Higgins
2019-11-06  0:37       ` Brendan Higgins
2019-10-18  0:43 ` Brendan Higgins
2019-10-18 16:25   ` Theodore Y. Ts'o
2019-10-18 21:41     ` Brendan Higgins [this message]
2019-10-30 19:02       ` Kees Cook
2019-10-31  9:01         ` Brendan Higgins
2019-10-18 12:29 ` Luis Chamberlain
2019-10-19 12:56   ` Alan Maguire
2019-10-19 18:36     ` Luis Chamberlain
2019-10-24  0:42     ` Brendan Higgins
2019-10-24 10:15       ` Luis Chamberlain
2019-10-30 19:09         ` Kees Cook
2019-10-30 20:11           ` Iurii Zaikin
2019-10-31  1:40             ` John Johansen
2019-10-31  9:33             ` Brendan Higgins
2019-10-31 18:40               ` Kees Cook
2019-11-05 16:43               ` Mike Salvatore
2019-11-05 23:59                 ` Brendan Higgins
2019-10-31  1:37           ` John Johansen
2019-10-31  9:17           ` Brendan Higgins
2019-11-01 12:30             ` Alan Maguire
2019-11-05 23:44               ` Brendan Higgins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAFd5g45LmnbD7L4LqdbfBV5YR377e81m61+z==RKCGjWBFqDGQ@mail.gmail.com' \
    --to=brendanhiggins@google.com \
    --cc=alan.maguire@oracle.com \
    --cc=davidgow@google.com \
    --cc=jmorris@namei.org \
    --cc=john.johansen@canonical.com \
    --cc=keescook@chromium.org \
    --cc=kunit-dev@googlegroups.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-security-module@vger.kernel.org \
    --cc=mcgrof@kernel.org \
    --cc=mike.salvatore@canonical.com \
    --cc=serge@hallyn.com \
    --cc=shuah@kernel.org \
    --cc=tytso@mit.edu \
    --cc=yzaikin@google.com \
    --subject='Re: [PATCH linux-kselftest/test v1] apparmor: add AppArmor KUnit tests for policy unpack' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).