Re: what trees/branches to test on syzbot

From: Fengguang Wu <fengguang.wu@intel.com>
To: Dmitry Vyukov <dvyukov@google.com>
Cc: Guenter Roeck <groeck@google.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Theodore Ts'o <tytso@mit.edu>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	syzkaller <syzkaller@googlegroups.com>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	David Miller <davem@davemloft.net>
Subject: Re: what trees/branches to test on syzbot
Date: Fri, 19 Jan 2018 09:48:03 +0800	[thread overview]
Message-ID: <20180119014803.n75l5vrxlpifm3sc@wfg-t540p.sh.intel.com> (raw)
In-Reply-To: <CACT4Y+aGMoobn069+Lq1BT2YGqi9qYY9vHFtiXT2DLsJ5ZUh9Q@mail.gmail.com>

Hi Dmitry,

On Tue, Jan 16, 2018 at 10:58:51AM +0100, Dmitry Vyukov wrote:
>On Tue, Jan 16, 2018 at 10:45 AM, Guenter Roeck <groeck@google.com> wrote:
>> On Mon, Jan 15, 2018 at 11:51 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
>>> Hello,
>>>
>>> Several people proposed that linux-next should not be tested on
>>> syzbot. While some people suggested that it needs to test as many
>>> trees as possible. I've initially included linux-next as it is a
>>> staging area before upstream tree, with the intention that patches are
>>> _tested_ there, is they are not tested there, bugs enter upstream
>>> tree. And then it takes much longer to get fix into other trees.
>>>
>>> So the question is: what trees/branches should be tested? Preferably
>>> in priority order as syzbot can't test all of them.
>>>
>>
>> I always thought that -next existed specifically to give people a
>> chance to test the code in it. Maybe the question is where to report
>> the test results ?
>
>FTR, from Guenter on another thread:
>
>> Interesting. Assuming that refers to linux-next, not linux-net, that
>> may explain why linux-next tends to deteriorate. I wonder if I should
>> drop it from my testing as well. I'll be happy to follow whatever the
>> result of this exchange is and do the same.
>
>If we agree on some list of important branches, and what branches
>specifically should not be tested with automatic reporting, I think it
>will benefit everybody.
>+Fengguang, can you please share your list and rationale behind it?

0-day aims to aggressively test as much tree and branches as possible,
including various developer trees, maintainer, linux-next, mainline and
stable trees. Here are the complete list of 800+ trees we monitored:

https://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git/tree/repo/linux

The rationale is obvious. IMHO what really matters here is about
capability rather than rationale: that policy heavily relies on the
fundamental capability of auto bisecting. Once regressions are
bisected, we know the owners of problem to auto send report to, ie.
the first bad commit's author and committer.

For the bugs that cannot be bisected, they tend to be old ones and
we report more often on mainline tree than linux-next.

Thanks,
Fengguang