syzbot reporting less duplicates

syzbot reporting less duplicates

[Dmitry Vyukov]

Hi,

FYI syzbot should report less duplicates now.
A significant number of reported duplicates was one of common
complaints. syzbot will now merge some slightly differently looking
crashes together, for example, these 2 recently reported bugs will be
merged into a single bug and reported once in future:

KASAN: use-after-free Read in skb_segment
KASAN: slab-out-of-bounds Read in skb_segment

You can see an example here:
https://syzkaller.appspot.com/bug?id=9936b32dd3a4a278f06a2cb07eb13df9e113ca84
which contains crashes for both "BUG: unable to handle kernel paging
request in cdev_del" and "general protection fault in cdev_del".

Hope you will find this useful.

If you are interested in heuristics details, grep for "alt:" here:
https://github.com/google/syzkaller/blob/master/pkg/report/linux.go
Crashes with the same alt title are merged together. In short,
currently there is one cluster for "bad access in function foo" (GPF,
fault, KASAN, KMSAN, corrupted list, etc); another cluster for passive
hangs (task hung, task can't die); and another cluster for active
stalls (rcu stall, cpu hang). These rules are based on analysis of all
previously reported bugs and duplicates among them.

Thanks

Re: syzbot reporting less duplicates

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

On Mon 2021-02-01 11:52:12, Dmitry Vyukov wrote:
> Hi,
> 
> FYI syzbot should report less duplicates now.
> A significant number of reported duplicates was one of common
> complaints. syzbot will now merge some slightly differently looking
> crashes together, for example, these 2 recently reported bugs will be
> merged into a single bug and reported once in future:
> 
> KASAN: use-after-free Read in skb_segment
> KASAN: slab-out-of-bounds Read in skb_segment

Could we please get common prefix (like syzbot: KASAN:....) so that
the bulk of emails is easier to remove?

Thanks,
								Pavel
-- 
http://www.livejournal.com/~pavelmachek

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: syzbot reporting less duplicates

[bobwxc]

On Wed, Feb 03, 2021 at 05:05:43PM +0100, Pavel Machek wrote:
> On Mon 2021-02-01 11:52:12, Dmitry Vyukov wrote:
> Could we please get common prefix (like syzbot: KASAN:....) so that
> the bulk of emails is easier to remove?
There are several bots testing on the kernel, maybe we should give a prefix 
format for all bot.
Also we can use mail-address to fliter email, but it's still a little 
inconvenient.

bobwxc

Re: syzbot reporting less duplicates

[Dmitry Vyukov]

On Wed, Feb 3, 2021 at 6:39 PM bobwxc <bobwxc@email.cn> wrote:
>
> On Wed, Feb 03, 2021 at 05:05:43PM +0100, Pavel Machek wrote:
> > On Mon 2021-02-01 11:52:12, Dmitry Vyukov wrote:
> > Could we please get common prefix (like syzbot: KASAN:....) so that
> > the bulk of emails is easier to remove?
> There are several bots testing on the kernel, maybe we should give a prefix
> format for all bot.
> Also we can use mail-address to fliter email, but it's still a little
> inconvenient.

Hi Pavel, bobwxc,

Yes, I was wondering if syzbot in From/To/CC can be used for
filtering? I assume email clients that can filter based on subject can
also filter based on From/To/CC.
Does anybody filter syzbot emails? Maybe you can share what works best?

I am not sure a common prefix for all bots is useful because it
supports only all or nothing. There are also some bots that
maintainers use now that seem to be fundamental to the process, if one
is ignoring them, then they are effectively ignoring what the
maintainer is saying.

Re: syzbot reporting less duplicates

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1539 bytes --]

Hi!
On Wed 2021-02-03 19:22:34, Dmitry Vyukov wrote:
> On Wed, Feb 3, 2021 at 6:39 PM bobwxc <bobwxc@email.cn> wrote:
> >
> > On Wed, Feb 03, 2021 at 05:05:43PM +0100, Pavel Machek wrote:
> > > On Mon 2021-02-01 11:52:12, Dmitry Vyukov wrote:
> > > Could we please get common prefix (like syzbot: KASAN:....) so that
> > > the bulk of emails is easier to remove?
> > There are several bots testing on the kernel, maybe we should give a prefix
> > format for all bot.
> > Also we can use mail-address to fliter email, but it's still a little
> > inconvenient.
> 
> Hi Pavel, bobwxc,
> 
> Yes, I was wondering if syzbot in From/To/CC can be used for
> filtering? I assume email clients that can filter based on subject can
> also filter based on From/To/CC.
> Does anybody filter syzbot emails? Maybe you can share what works
> best?

From does not really work. So... syzbot reports for subsystems I don't
maintain are uninteresting, and so is the resulting discussion.

While filtering on "From:" is easy for initial report, it does not
make it easy to remove follow up discussion.n 

> I am not sure a common prefix for all bots is useful because it
> supports only all or nothing. There are also some bots that
> maintainers use now that seem to be fundamental to the process, if one
> is ignoring them, then they are effectively ignoring what the
> maintainer is saying.

I'm pretty sure common prefix for all bots is useful.

Best regards,
							Pavel
-- 
http://www.livejournal.com/~pavelmachek

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: syzbot reporting less duplicates

[Dmitry Vyukov]

On Wed, Feb 3, 2021 at 8:24 PM Pavel Machek <pavel@ucw.cz> wrote:
>
> Hi!
> On Wed 2021-02-03 19:22:34, Dmitry Vyukov wrote:
> > On Wed, Feb 3, 2021 at 6:39 PM bobwxc <bobwxc@email.cn> wrote:
> > >
> > > On Wed, Feb 03, 2021 at 05:05:43PM +0100, Pavel Machek wrote:
> > > > On Mon 2021-02-01 11:52:12, Dmitry Vyukov wrote:
> > > > Could we please get common prefix (like syzbot: KASAN:....) so that
> > > > the bulk of emails is easier to remove?
> > > There are several bots testing on the kernel, maybe we should give a prefix
> > > format for all bot.
> > > Also we can use mail-address to fliter email, but it's still a little
> > > inconvenient.
> >
> > Hi Pavel, bobwxc,
> >
> > Yes, I was wondering if syzbot in From/To/CC can be used for
> > filtering? I assume email clients that can filter based on subject can
> > also filter based on From/To/CC.
> > Does anybody filter syzbot emails? Maybe you can share what works
> > best?
>
> From does not really work. So... syzbot reports for subsystems I don't
> maintain are uninteresting, and so is the resulting discussion.
>
> While filtering on "From:" is easy for initial report, it does not
> make it easy to remove follow up discussion.n


I've filed https://github.com/google/syzkaller/issues/2435 to track
the subject feature request.
I thought that maybe filtering based on From/To/CC should work right
away, it should capture follow up discussions as well.



> > I am not sure a common prefix for all bots is useful because it
> > supports only all or nothing. There are also some bots that
> > maintainers use now that seem to be fundamental to the process, if one
> > is ignoring them, then they are effectively ignoring what the
> > maintainer is saying.
>
> I'm pretty sure common prefix for all bots is useful.
>
> Best regards,
>                                                         Pavel
> --
> http://www.livejournal.com/~pavelmachek

Re: syzbot reporting less duplicates

[Dmitry Vyukov]

On Wed, Feb 3, 2021 at 8:24 PM Pavel Machek <pavel@ucw.cz> wrote:
>
> Hi!
> On Wed 2021-02-03 19:22:34, Dmitry Vyukov wrote:
> > On Wed, Feb 3, 2021 at 6:39 PM bobwxc <bobwxc@email.cn> wrote:
> > >
> > > On Wed, Feb 03, 2021 at 05:05:43PM +0100, Pavel Machek wrote:
> > > > On Mon 2021-02-01 11:52:12, Dmitry Vyukov wrote:
> > > > Could we please get common prefix (like syzbot: KASAN:....) so that
> > > > the bulk of emails is easier to remove?
> > > There are several bots testing on the kernel, maybe we should give a prefix
> > > format for all bot.
> > > Also we can use mail-address to fliter email, but it's still a little
> > > inconvenient.
> >
> > Hi Pavel, bobwxc,
> >
> > Yes, I was wondering if syzbot in From/To/CC can be used for
> > filtering? I assume email clients that can filter based on subject can
> > also filter based on From/To/CC.
> > Does anybody filter syzbot emails? Maybe you can share what works
> > best?
>
> From does not really work. So... syzbot reports for subsystems I don't
> maintain are uninteresting, and so is the resulting discussion.
>
> While filtering on "From:" is easy for initial report, it does not
> make it easy to remove follow up discussion.n
>
> > I am not sure a common prefix for all bots is useful because it
> > supports only all or nothing. There are also some bots that
> > maintainers use now that seem to be fundamental to the process, if one
> > is ignoring them, then they are effectively ignoring what the
> > maintainer is saying.
>
> I'm pretty sure common prefix for all bots is useful.

This is now implemented and deployed:
https://github.com/google/syzkaller/pull/2472

Here is the first example report with the prefix:
https://lore.kernel.org/lkml/0000000000002592aa05bccabae5@google.com/T/#u