Re: Linux Testing Microconference at LPC

From: Andrea Parri <andrea.parri@amarulasolutions.com>
To: Mark Rutland <mark.rutland@arm.com>
Cc: Dhaval Giani <dhaval.giani@gmail.com>,
	Sasha Levin <alexander.levin@microsoft.com>,
	shuah <shuah@kernel.org>, Kevin Hilman <khilman@baylibre.com>,
	Tim Bird <tbird20d@gmail.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	"Carpenter,Dan" <dan.carpenter@oracle.com>,
	willy@infradead.org, gustavo.padovan@collabora.co.uk,
	Dmitry Vyukov <dvyukov@google.com>,
	knut.omang@oracle.com, Nick Desaulniers <ndesaulniers@google.com>,
	"Paul E. McKenney" <paulmck@linux.ibm.com>,
	Alan Stern <stern@rowland.harvard.edu>
Subject: Re: Linux Testing Microconference at LPC
Date: Sun, 12 May 2019 02:40:08 +0200	[thread overview]
Message-ID: <20190512004008.GA6062@andrea> (raw)
In-Reply-To: <20190423102250.GA56999@lakrids.cambridge.arm.com>

On Tue, Apr 23, 2019 at 11:22:50AM +0100, Mark Rutland wrote:
> On Thu, Apr 11, 2019 at 10:37:51AM -0700, Dhaval Giani wrote:
> > Hi Folks,
> > 
> > This is a call for participation for the Linux Testing microconference
> > at LPC this year.
> > 
> > For those who were at LPC last year, as the closing panel mentioned,
> > testing is probably the next big push needed to improve quality. From
> > getting more selftests in, to regression testing to ensure we don't
> > break realtime as more of PREEMPT_RT comes in, to more stable distros,
> > we need more testing around the kernel.
> > 
> > We have talked about different efforts around testing, such as fuzzing
> > (using syzkaller and trinity), automating fuzzing with syzbot, 0day
> > testing, test frameworks such as ktests, smatch to find bugs in the
> > past. We want to push this discussion further this year and are
> > interested in hearing from you what you want to talk about, and where
> > kernel testing needs to go next.
> 
> I'd be interested to discuss what we could do with annotations and
> compiler instrumentation to make the kernel more amenable to static and
> dynamic analysis (and to some extent, documenting implicit
> requirements).
> 
> One idea that I'd like to explore in the context of RT is to annotate
> function signatures with their required IRQ/preempt context, such that
> we could dynamically check whether those requirements were violated
> (even if it didn't happen to cause a problem at that point in time), and
> static analysis would be able to find some obviously broken usage. I had
> some rough ideas of how to do the dynamic part atop/within ftrace. Maybe
> there are similar problems elsewhere.
> 
> I know that some clang folk were interested in similar stuff. IIRC Nick
> Desaulniers was interested in whether clang's thread safety analysis
> tooling could be applied to the kernel (e.g. based on lockdep
> annotations).

FWIW, I'd also be interested in discussing these developments.

There have been several activities/projects related to such "tooling"
(thread safety analysis) recently:  I could point out the (brand new)
Google Summer of Code "Applying Clang Thread Safety Analyser to Linux
Kernel" project [1] and (for the "dynamic analysis" side) the efforts
to revive the Kernel Thread sanitizer [2].  I should also mention the
efforts to add (support for) "unmarked" accesses and to formalize the
notion of "data race" in the memory consistency model [3].

So, again, I'd welcome a discussion on these works/ideas.

Thanks,
  Andrea

[1] https://summerofcode.withgoogle.com/projects/#5358212549705728
    https://github.com/ClangBuiltLinux/thread-safety-analysis
[2] https://github.com/google/ktsan/commits/ktsan
[3] https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=dev&id=c602b9e58cb9c13f260791dd7da6687e06809923
    https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=dev&id=3b1fe99c68b5673879a8018a46b23f431e4d4b7a
    https://lkml.kernel.org/r/Pine.LNX.4.44L0.1903191459270.1593-200000@iolanthe.rowland.org