Re: [Ksummit-discuss] [CORE TOPIC] Testing

[Steven Rostedt <rostedt@goodmis.org>]

On Tue, 7 Jul 2015 14:14:11 +0100
Mark Brown <broonie@sirena.org.uk> wrote:

> On Tue, Jul 07, 2015 at 04:02:13PM +0300, Alexey Dobriyan wrote:
> > On Tue, Jul 7, 2015 at 12:24 PM, Mark Brown <broonie@kernel.org> wrote:
> 
> > >  - Should we start carrying config fragments upstream designed to
> > >    support testing, things like the distro config fragments that keep
> > >    getting discussed are one example here but there's other things like
> > >    collections of debug options we could be looking at.
> 
> > This will gravitate everyone to running the same config which is the opposite
> > of what you want.
> 
> Perhaps, perhaps not - it's not an unequivocal thing either way.  The
> more barriers there are to enabling things the more likely it is that
> people just won't bother in the first place (or that they'll run into
> somme problem and give up before they get things working) and it's not
> clear that having to figure these things out is always a good use of
> people's time.

The testing/selftests tests should have three results: PASS, FAIL,
UNSUPPORTED. The UNSUPPORTED is what should be returned if the kernel
configuration doesn't have the needed features configured. For example,
if you run the ftrace selftests without function tracing enabled, all
the tests that test the function tracer return UNSUPPORTED.

Perhaps we should have a central location that each test needs to add
the required configuration for it to be properly tested. Then if users
want to test various subsystems, they would look in this location for
the proper configs (be it a directory that has files of the tests they
represent, and contain the configs needed). Then there should be no
real barrier for people to run these tests.

Of course if the test requires certain hardware, or a file system, then
that should be properly documented.

-- Steve