Re: [PATCH v5 linux-kselftest-test 3/6] kunit: allow kunit tests to be loaded as a module

From: Alan Maguire <alan.maguire@oracle.com>
To: Iurii Zaikin <yzaikin@google.com>
Cc: Alan Maguire <alan.maguire@oracle.com>,
	"Theodore Y. Ts'o" <tytso@mit.edu>,
	Brendan Higgins <brendanhiggins@google.com>,
	David Gow <davidgow@google.com>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	KUnit Development <kunit-dev@googlegroups.com>,
	Kees Cook <keescook@chromium.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Masahiro Yamada <yamada.masahiro@socionext.com>,
	catalin.marinas@arm.com, joe.lawrence@redhat.com,
	penguin-kernel@i-love.sakura.ne.jp, urezki@gmail.com,
	andriy.shevchenko@linux.intel.com,
	Jonathan Corbet <corbet@lwn.net>,
	adilger.kernel@dilger.ca, Luis Chamberlain <mcgrof@kernel.org>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	Knut Omang <knut.omang@oracle.com>
Subject: Re: [PATCH v5 linux-kselftest-test 3/6] kunit: allow kunit tests to be loaded as a module
Date: Fri, 6 Dec 2019 13:53:38 +0000 (GMT)	[thread overview]
Message-ID: <alpine.LRH.2.20.1912061349001.28856@dhcp-10-175-208-120.vpn.oracle.com> (raw)
In-Reply-To: <CAAXuY3qekjWBUTxzAjCs+87nVXpigvzqm7TpG7MtJagTSG-xtg@mail.gmail.com>

On Wed, 4 Dec 2019, Iurii Zaikin wrote:

> > I've also got a patch that I was hoping to send out soon
> > that might help.  The idea is that each test suite would create
> > a debugfs representation under /sys/kernel/debug/kunit;
> > specifically:
> >
> > /sys/kernel/debug/kunit/results/<suite>
> > /sys/kernel/debug/kunit/results/<suite>-tests
> >
> > ...where cat'ing the former shows the full set of results,
> > and the latter is a directory within which we can display
> > individual test results in test-case-specific files.
> >
> > This is all done by ensuring that when tests log information,
> > they log to a per-test-case log buffer as well as to dmesg.
> >
> > If the above sounds useful, I'll try and polish up the patch
> > for submission. Thanks!
> What would be the best way for kunit_tool to:
> 1. Know that the tests have completed as QEMU will be just sitting
> there with kernel complaining about the absence of init (or running
> whatever we give it as init)?
> 2. Read the test results from debugfs under QEMU virtual machine while
> the kernel is still there?
> I think supplying an init script/binary that copies the
> /sys/kernel/debug/kunit/results/* to a 9p shared dir set up by
> kunit_tool would work but it would add a step of cross-compiling and
> packaging a userspace binary.
> 

I confess I'd only been thinking about supporting the case of a user 
modprobe-ing a kunit test suite module directly and wanting a clean set 
of results separated from other dmesg output. However the scheme you 
describe does seem workable for the UML case also.  With the 
late_initcalls the builtin kunit suites will likely run early in boot but perhaps we could tweak the 
semantics such that the full results debugfs file is not populated until 
the tests have run to simplify script-based probing. I'll try some 
experiments with the debugfs patch once it's ready. Thanks!

Alan 