Re: [RFC] syzbot process

From: Dmitry Vyukov <dvyukov@google.com>
To: Stephan Mueller <smueller@chronox.de>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>,
	LKML <linux-kernel@vger.kernel.org>,
	syzkaller <syzkaller@googlegroups.com>,
	Eric Dumazet <edumazet@google.com>,
	Eric Biggers <ebiggers3@gmail.com>,
	Kostya Serebryany <kcc@google.com>,
	Alexander Potapenko <glider@google.com>,
	andreyknvl <andreyknvl@google.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>,
	David Miller <davem@davemloft.net>,
	Willem de Bruijn <willemb@google.com>,
	Guenter Roeck <groeck@google.com>
Subject: Re: [RFC] syzbot process
Date: Thu, 28 Dec 2017 11:19:20 +0100	[thread overview]
Message-ID: <CACT4Y+bvTzef_1ppdN2E5X71dUdA_z=MYUQu70ERSLtjzQVkvA@mail.gmail.com> (raw)
In-Reply-To: <2591445.sHVZGWs4yY@tauon.chronox.de>

On Thu, Dec 21, 2017 at 2:36 PM, Stephan Mueller <smueller@chronox.de> wrote:
> Am Donnerstag, 21. Dezember 2017, 14:22:40 CET schrieb Andrey Ryabinin:
>
> Hi Andrey,
>
>> 2017-12-21 15:52 GMT+03:00 Dmitry Vyukov <dvyukov@google.com>:
>> > Any other proposals, thoughts, ideas?
>>
>> a) Assume that patches send in replies to the bug report are fixes.
>>
>> b) Almost the same as your "syzbot-fix: HASH"  proposal, but slightly
>> closer to normal kernel development workflow.
>>      Add hash/bug id into the From field of email, i.e.
>>
>>      instead of
>>      From: syzbot <syzkaller@googlegroups.com>
>>
>>      make it
>>      From: syzbot-{hash} <syzkaller@googlegroups.com>
>>
>>      And ask to include "Reported-by: syzbot-{hash}
>> <syzkaller@googlegroups.com>" tag in a changelog.
>>
>> a) doesn't exclude b) or "#syz: fix " emails, and vise versa
>
> One additional suggestion: Rerun all already generated reproducer tests on,
> say, each updated kernel (e.g. newer rc or even full new version). If an issue
> is detected again, send a reply to the original message to indicate that the
> issue is still there. Note, I sometimes even fear that a patch that ought to
> fix the reported issue may not completely fix it considering races.
>
> The problem with the current approach (at least to me) is that on mailing
> lists with some volume, such reports get easily buried.

Hi Stephan,

We've considered a similar idea, but there are 2 problems with it:
1. For some bugs syzbot doesn't have reproducers, so it simply can't
retest (though, more than half of them is still perfectly actionable,
e.g. LOCKDEP/KASAN reports contain enough information to rootcause,
some WARNINGs/BUGs/GPFs clearly point to simple issues like missed
input checks, off-by-ones, etc).
2. Lots of bugs are due to races and can't be reproduced with 100%
probability, or code can slightly change so the old reproducer doesn't
trigger the bug anymore. For racy bugs in the worst case syzbot will
close and remail the bug each day (which obviously won't be warmly
welcomed).