Re: what trees/branches to test on syzbot

From: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
To: Guenter Roeck <groeck@google.com>, Dmitry Vyukov <dvyukov@google.com>
Cc: "Theodore Ts'o" <tytso@mit.edu>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	syzkaller <syzkaller@googlegroups.com>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	David Miller <davem@davemloft.net>,
	kbuild test robot <fengguang.wu@intel.com>
Subject: Re: what trees/branches to test on syzbot
Date: Wed, 27 Jun 2018 05:37:00 +0900	[thread overview]
Message-ID: <876e38d9-95e5-0a93-9c26-7287151b8b53@i-love.sakura.ne.jp> (raw)
In-Reply-To: <CABXOdTe1HCy78w+G4LuN8S7Ad5kvUxwHsiP2a=8aW-jEvpjdug@mail.gmail.com>

On 2018/06/26 23:54, Guenter Roeck wrote:
> On Tue, Jun 26, 2018 at 7:38 AM Dmitry Vyukov <dvyukov@google.com> wrote:
>>
>> On Tue, Jun 26, 2018 at 4:16 PM, Theodore Y. Ts'o <tytso@mit.edu> wrote:
>>> On Tue, Jun 26, 2018 at 07:54:53PM +0900, Tetsuo Handa wrote:
>>>> I hope we can accept NOW either "reviving linux-next.git" or "allowing debug printk()
>>>> patches for linux.git". For example, "INFO: task hung in __sb_start_write" got 900
>>>> crashes in 81 days but still unable to find a reproducer. Dmitry tried to reproduce
>>>> locally with debug printk() patches but not yet successful. I think that testing with
>>>> http://lkml.kernel.org/r/f91e1c82-9693-cca3-4ab7-ecd9d9881fb4@i-love.sakura.ne.jp
>>>> on linux.git or linux-next.git is the only realistic way for debugging this bug.
>>>> More we postpone revival of the linux-next, more syzbot reports we will get...
>>>
>>> Here's a proposal for adding linux-next back:
>>>
>>> *) Subsystems or maintainers need to have a way to opt out of getting
>>>    spammed with Syzkaller reports that have no reproducer.  More often
>>>    than not, they are not actionable, and just annoy the maintainers,
>>>    with the net result that they tune out all Syzkaller reports as
>>>    noise.
>>
>> False. You can count yourself. 2/3 are actionable and fixed.
>>
> 
> Problem is that some if not many of the other 1/3 will be considered
> noise, and even some of the 2/3 will be considered noise because they
> have already been fixed by the time they are reported. Same problem as
> with, say, stable tree merges: People don't see the thousands of bug
> fixes inherited with such merges, but they do see the two or three
> regressions. Plus, of course, one can not prove that the thousands of
> bug fixes did any good because the fixed bugs are not observable
> anymore. The only remedy is to try to reduce regressions down to zero
> (or, of course, stop using/merging stable releases).
> 
> The same applies here: People won't see the good, they only see the
> noise. This is pretty much the reason why I all but stopped reporting
> build/boot failures on -next. You would have to reduce the noise
> almost down to zero for people to stop complaining, and you would have
> to be _really_ sure that the problem was not already fixed or reported
> elsewhere.
> 
> Guenter
> 

I think that syzbot can stop deciding email recipients and leave it to those who
diagnose bugs, for the ratio of sending to wrong subsystem maintainers is not low.
For example, syzbot assumed that "INFO: task hung in __get_super" is a fs layer bug.
But I think that the problem is in more lower layers (block or mm or locking layer).
The root cause could even be just overstressed due to instructions enabled by
CONFIG_KCOV_ENABLE_COMPARISONS=y.