KASAN: null-ptr-deref Read in reclaim_high

KASAN: null-ptr-deref Read in reclaim_high

[syzbot]

syzbot has bisected this bug to:

commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
Author: Shakeel Butt <shakeelb@google.com>
Date:   Wed Jan 9 22:02:21 2019 +0000

     memcg: schedule high reclaim for remote memcgs on high_work

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
git tree:       linux-next
final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
userspace arch: amd64
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000

Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on  
high_work")

KASAN: null-ptr-deref Read in reclaim_high

[syzbot]

syzbot has bisected this bug to:

commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
Author: Shakeel Butt <shakeelb@google.com>
Date:   Wed Jan 9 22:02:21 2019 +0000

     memcg: schedule high reclaim for remote memcgs on high_work

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
git tree:       linux-next
final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
userspace arch: amd64
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000

Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on  
high_work")

Re: KASAN: null-ptr-deref Read in reclaim_high

[Andrew Morton]

On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:

> syzbot has bisected this bug to:
> 
> commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> Author: Shakeel Butt <shakeelb@google.com>
> Date:   Wed Jan 9 22:02:21 2019 +0000
> 
>      memcg: schedule high reclaim for remote memcgs on high_work
> 
> bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> git tree:       linux-next
> final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> userspace arch: amd64
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> 
> Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on  
> high_work")

The following patch
memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
might have fixed this.  Was it applied?

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
<akpm@linux-foundation.org> wrote:
>
> On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
>
> > syzbot has bisected this bug to:
> >
> > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > Author: Shakeel Butt <shakeelb@google.com>
> > Date:   Wed Jan 9 22:02:21 2019 +0000
> >
> >      memcg: schedule high reclaim for remote memcgs on high_work
> >
> > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > git tree:       linux-next
> > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > userspace arch: amd64
> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> >
> > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > high_work")
>
> The following patch
> memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> might have fixed this.  Was it applied?

Hi Andrew,

You mean if the patch was applied during the bisection?
No, it wasn't. Bisection is very specifically done on the same tree
where the bug was hit. There are already too many factors that make
the result flaky/wrong/inconclusive without changing the tree state.
Now, if syzbot would know about any pending fix for this bug, then it
would not do the bisection at all. But it have not seen any patch in
upstream/linux-next with the Reported-by tag, nor it received any syz
fix commands for this bugs. Should have been it aware of the fix? How?

Thanks

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
<akpm@linux-foundation.org> wrote:
>
> On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
>
> > syzbot has bisected this bug to:
> >
> > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > Author: Shakeel Butt <shakeelb@google.com>
> > Date:   Wed Jan 9 22:02:21 2019 +0000
> >
> >      memcg: schedule high reclaim for remote memcgs on high_work
> >
> > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > git tree:       linux-next
> > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > userspace arch: amd64
> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> >
> > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > high_work")
>
> The following patch
> memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> might have fixed this.  Was it applied?

Hi Andrew,

You mean if the patch was applied during the bisection?
No, it wasn't. Bisection is very specifically done on the same tree
where the bug was hit. There are already too many factors that make
the result flaky/wrong/inconclusive without changing the tree state.
Now, if syzbot would know about any pending fix for this bug, then it
would not do the bisection at all. But it have not seen any patch in
upstream/linux-next with the Reported-by tag, nor it received any syz
fix commands for this bugs. Should have been it aware of the fix? How?

Thanks

Re: KASAN: null-ptr-deref Read in reclaim_high

[Andrew Morton]

On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:

> On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> <akpm@linux-foundation.org> wrote:
> >
> > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> >
> > > syzbot has bisected this bug to:
> > >
> > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > Author: Shakeel Butt <shakeelb@google.com>
> > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > >
> > >      memcg: schedule high reclaim for remote memcgs on high_work
> > >
> > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > git tree:       linux-next
> > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > userspace arch: amd64
> > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > >
> > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > high_work")
> >
> > The following patch
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > might have fixed this.  Was it applied?
> 
> Hi Andrew,
> 
> You mean if the patch was applied during the bisection?
> No, it wasn't. Bisection is very specifically done on the same tree
> where the bug was hit. There are already too many factors that make
> the result flaky/wrong/inconclusive without changing the tree state.
> Now, if syzbot would know about any pending fix for this bug, then it
> would not do the bisection at all. But it have not seen any patch in
> upstream/linux-next with the Reported-by tag, nor it received any syz
> fix commands for this bugs. Should have been it aware of the fix? How?

memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
added to linux-next on Jan 10.  I take it that this bug was hit when
testing the entire linux-next tree, so we can assume that
memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
does not fix it, correct?

In which case, over to Shakeel!

Re: KASAN: null-ptr-deref Read in reclaim_high

[Eric Biggers]

On Mon, Mar 11, 2019 at 11:25:41PM -0700, Andrew Morton wrote:
> On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> 
> > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > <akpm@linux-foundation.org> wrote:
> > >
> > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > >
> > > > syzbot has bisected this bug to:
> > > >
> > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > >
> > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > >
> > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > git tree:       linux-next
> > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > userspace arch: amd64
> > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > >
> > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > high_work")
> > >
> > > The following patch
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > might have fixed this.  Was it applied?
> > 
> > Hi Andrew,
> > 
> > You mean if the patch was applied during the bisection?
> > No, it wasn't. Bisection is very specifically done on the same tree
> > where the bug was hit. There are already too many factors that make
> > the result flaky/wrong/inconclusive without changing the tree state.
> > Now, if syzbot would know about any pending fix for this bug, then it
> > would not do the bisection at all. But it have not seen any patch in
> > upstream/linux-next with the Reported-by tag, nor it received any syz
> > fix commands for this bugs. Should have been it aware of the fix? How?
> 
> memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> added to linux-next on Jan 10.  I take it that this bug was hit when
> testing the entire linux-next tree, so we can assume that
> memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> does not fix it, correct?
> 
> In which case, over to Shakeel!
> 

I don't understand what happened here.  First, the syzbot report doesn't say
which linux-next version was tested (which it should), but I get:

$ git tag --contains 29a4b8e275d1f10c51c7891362877ef6cffae9e7
next-20190110
next-20190111
next-20190114
next-20190115
next-20190116

That's almost 2 months old, yet this bug was just reported now.  Why?

- Eric

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 7:43 AM Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Mon, Mar 11, 2019 at 11:25:41PM -0700, Andrew Morton wrote:
> > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > >
> > > > > syzbot has bisected this bug to:
> > > > >
> > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > >
> > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > >
> > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > git tree:       linux-next
> > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > userspace arch: amd64
> > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > >
> > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > high_work")
> > > >
> > > > The following patch
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > might have fixed this.  Was it applied?
> > >
> > > Hi Andrew,
> > >
> > > You mean if the patch was applied during the bisection?
> > > No, it wasn't. Bisection is very specifically done on the same tree
> > > where the bug was hit. There are already too many factors that make
> > > the result flaky/wrong/inconclusive without changing the tree state.
> > > Now, if syzbot would know about any pending fix for this bug, then it
> > > would not do the bisection at all. But it have not seen any patch in
> > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > fix commands for this bugs. Should have been it aware of the fix? How?
> >
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > added to linux-next on Jan 10.  I take it that this bug was hit when
> > testing the entire linux-next tree, so we can assume that
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > does not fix it, correct?
> >
> > In which case, over to Shakeel!
> >
>
> I don't understand what happened here.  First, the syzbot report doesn't say
> which linux-next version was tested (which it should), but I get:
>
> $ git tag --contains 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> next-20190110
> next-20190111
> next-20190114
> next-20190115
> next-20190116
>
> That's almost 2 months old, yet this bug was just reported now.  Why?

Hi Eric,

This bug was reported on Jan 10:
https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ

The start revision of the bisection process (provided) is the same
that was used to create the reproducer. The end revision and bisection
log are provided in the email.

How can we improve the format to make it more clear?

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 7:43 AM Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Mon, Mar 11, 2019 at 11:25:41PM -0700, Andrew Morton wrote:
> > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > >
> > > > > syzbot has bisected this bug to:
> > > > >
> > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > >
> > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > >
> > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > git tree:       linux-next
> > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > userspace arch: amd64
> > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > >
> > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > high_work")
> > > >
> > > > The following patch
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > might have fixed this.  Was it applied?
> > >
> > > Hi Andrew,
> > >
> > > You mean if the patch was applied during the bisection?
> > > No, it wasn't. Bisection is very specifically done on the same tree
> > > where the bug was hit. There are already too many factors that make
> > > the result flaky/wrong/inconclusive without changing the tree state.
> > > Now, if syzbot would know about any pending fix for this bug, then it
> > > would not do the bisection at all. But it have not seen any patch in
> > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > fix commands for this bugs. Should have been it aware of the fix? How?
> >
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > added to linux-next on Jan 10.  I take it that this bug was hit when
> > testing the entire linux-next tree, so we can assume that
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > does not fix it, correct?
> >
> > In which case, over to Shakeel!
> >
>
> I don't understand what happened here.  First, the syzbot report doesn't say
> which linux-next version was tested (which it should), but I get:
>
> $ git tag --contains 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> next-20190110
> next-20190111
> next-20190114
> next-20190115
> next-20190116
>
> That's almost 2 months old, yet this bug was just reported now.  Why?

Hi Eric,

This bug was reported on Jan 10:
https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ

The start revision of the bisection process (provided) is the same
that was used to create the reproducer. The end revision and bisection
log are provided in the email.

How can we improve the format to make it more clear?

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
>
> > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > <akpm@linux-foundation.org> wrote:
> > >
> > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > >
> > > > syzbot has bisected this bug to:
> > > >
> > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > >
> > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > >
> > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > git tree:       linux-next
> > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > userspace arch: amd64
> > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > >
> > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > high_work")
> > >
> > > The following patch
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > might have fixed this.  Was it applied?
> >
> > Hi Andrew,
> >
> > You mean if the patch was applied during the bisection?
> > No, it wasn't. Bisection is very specifically done on the same tree
> > where the bug was hit. There are already too many factors that make
> > the result flaky/wrong/inconclusive without changing the tree state.
> > Now, if syzbot would know about any pending fix for this bug, then it
> > would not do the bisection at all. But it have not seen any patch in
> > upstream/linux-next with the Reported-by tag, nor it received any syz
> > fix commands for this bugs. Should have been it aware of the fix? How?
>
> memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> added to linux-next on Jan 10.  I take it that this bug was hit when
> testing the entire linux-next tree, so we can assume that
> memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> does not fix it, correct?
> In which case, over to Shakeel!

Jan 10 is exactly when this bug was reported:
https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a

We don't know if that patch fixed the bug or not because nobody tested
the reproducer with that patch.

It seems that the problem here is that nobody associated the fix with
the bug report. So people looking at open bug reports will spend time
again and again debugging this just to find that this was fixed months
ago. syzbot also doesn't have a chance to realize that this is fixed
and bisection is not necessary anymore. It also won't confirm/disprove
that the fix actually fixes the bug because even if the crash will
continue to happen it will look like the old crash just continues to
happen, so nothing to notify about.

Associating fixes with bug reports solves all these problems for
humans and bots.

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
>
> > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > <akpm@linux-foundation.org> wrote:
> > >
> > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > >
> > > > syzbot has bisected this bug to:
> > > >
> > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > >
> > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > >
> > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > git tree:       linux-next
> > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > userspace arch: amd64
> > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > >
> > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > high_work")
> > >
> > > The following patch
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > might have fixed this.  Was it applied?
> >
> > Hi Andrew,
> >
> > You mean if the patch was applied during the bisection?
> > No, it wasn't. Bisection is very specifically done on the same tree
> > where the bug was hit. There are already too many factors that make
> > the result flaky/wrong/inconclusive without changing the tree state.
> > Now, if syzbot would know about any pending fix for this bug, then it
> > would not do the bisection at all. But it have not seen any patch in
> > upstream/linux-next with the Reported-by tag, nor it received any syz
> > fix commands for this bugs. Should have been it aware of the fix? How?
>
> memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> added to linux-next on Jan 10.  I take it that this bug was hit when
> testing the entire linux-next tree, so we can assume that
> memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> does not fix it, correct?
> In which case, over to Shakeel!

Jan 10 is exactly when this bug was reported:
https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a

We don't know if that patch fixed the bug or not because nobody tested
the reproducer with that patch.

It seems that the problem here is that nobody associated the fix with
the bug report. So people looking at open bug reports will spend time
again and again debugging this just to find that this was fixed months
ago. syzbot also doesn't have a chance to realize that this is fixed
and bisection is not necessary anymore. It also won't confirm/disprove
that the fix actually fixes the bug because even if the crash will
continue to happen it will look like the old crash just continues to
happen, so nothing to notify about.

Associating fixes with bug reports solves all these problems for
humans and bots.

Re: KASAN: null-ptr-deref Read in reclaim_high

[Shakeel Butt]

On Tue, Mar 12, 2019 at 1:33 AM Dmitry Vyukov <dvyukov@google.com> wrote:
>
> On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > >
> > > > > syzbot has bisected this bug to:
> > > > >
> > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > >
> > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > >
> > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > git tree:       linux-next
> > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > userspace arch: amd64
> > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > >
> > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > high_work")
> > > >
> > > > The following patch
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > might have fixed this.  Was it applied?
> > >
> > > Hi Andrew,
> > >
> > > You mean if the patch was applied during the bisection?
> > > No, it wasn't. Bisection is very specifically done on the same tree
> > > where the bug was hit. There are already too many factors that make
> > > the result flaky/wrong/inconclusive without changing the tree state.
> > > Now, if syzbot would know about any pending fix for this bug, then it
> > > would not do the bisection at all. But it have not seen any patch in
> > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > fix commands for this bugs. Should have been it aware of the fix? How?
> >
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > added to linux-next on Jan 10.  I take it that this bug was hit when
> > testing the entire linux-next tree, so we can assume that
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > does not fix it, correct?
> > In which case, over to Shakeel!
>
> Jan 10 is exactly when this bug was reported:
> https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
>
> We don't know if that patch fixed the bug or not because nobody tested
> the reproducer with that patch.
>
> It seems that the problem here is that nobody associated the fix with
> the bug report. So people looking at open bug reports will spend time
> again and again debugging this just to find that this was fixed months
> ago. syzbot also doesn't have a chance to realize that this is fixed
> and bisection is not necessary anymore. It also won't confirm/disprove
> that the fix actually fixes the bug because even if the crash will
> continue to happen it will look like the old crash just continues to
> happen, so nothing to notify about.
>
> Associating fixes with bug reports solves all these problems for
> humans and bots.

Should we add "Reported-by" for syzbot reports on linux-next patches
as well? Please note that these patches are in flux and might be
dropped or completely changed before merging into Linus tree.

Also should syzbot drop bug reports on older linux-next trees if it
can not be repro'ed in the latest linux-next tree? IMHO yes.

Shakeel

Re: KASAN: null-ptr-deref Read in reclaim_high

[Shakeel Butt]

On Tue, Mar 12, 2019 at 1:33 AM Dmitry Vyukov <dvyukov@google.com> wrote:
>
> On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > >
> > > > > syzbot has bisected this bug to:
> > > > >
> > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > >
> > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > >
> > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > git tree:       linux-next
> > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > userspace arch: amd64
> > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > >
> > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > high_work")
> > > >
> > > > The following patch
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > might have fixed this.  Was it applied?
> > >
> > > Hi Andrew,
> > >
> > > You mean if the patch was applied during the bisection?
> > > No, it wasn't. Bisection is very specifically done on the same tree
> > > where the bug was hit. There are already too many factors that make
> > > the result flaky/wrong/inconclusive without changing the tree state.
> > > Now, if syzbot would know about any pending fix for this bug, then it
> > > would not do the bisection at all. But it have not seen any patch in
> > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > fix commands for this bugs. Should have been it aware of the fix? How?
> >
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > added to linux-next on Jan 10.  I take it that this bug was hit when
> > testing the entire linux-next tree, so we can assume that
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > does not fix it, correct?
> > In which case, over to Shakeel!
>
> Jan 10 is exactly when this bug was reported:
> https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
>
> We don't know if that patch fixed the bug or not because nobody tested
> the reproducer with that patch.
>
> It seems that the problem here is that nobody associated the fix with
> the bug report. So people looking at open bug reports will spend time
> again and again debugging this just to find that this was fixed months
> ago. syzbot also doesn't have a chance to realize that this is fixed
> and bisection is not necessary anymore. It also won't confirm/disprove
> that the fix actually fixes the bug because even if the crash will
> continue to happen it will look like the old crash just continues to
> happen, so nothing to notify about.
>
> Associating fixes with bug reports solves all these problems for
> humans and bots.

Should we add "Reported-by" for syzbot reports on linux-next patches
as well? Please note that these patches are in flux and might be
dropped or completely changed before merging into Linus tree.

Also should syzbot drop bug reports on older linux-next trees if it
can not be repro'ed in the latest linux-next tree? IMHO yes.

Shakeel

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 2:46 PM Shakeel Butt <shakeelb@google.com> wrote:
>
> On Tue, Mar 12, 2019 at 1:33 AM Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> > >
> > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > >
> > > > > > syzbot has bisected this bug to:
> > > > > >
> > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > >
> > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > >
> > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > git tree:       linux-next
> > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > userspace arch: amd64
> > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > >
> > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > high_work")
> > > > >
> > > > > The following patch
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > might have fixed this.  Was it applied?
> > > >
> > > > Hi Andrew,
> > > >
> > > > You mean if the patch was applied during the bisection?
> > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > where the bug was hit. There are already too many factors that make
> > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > would not do the bisection at all. But it have not seen any patch in
> > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > >
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > testing the entire linux-next tree, so we can assume that
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > does not fix it, correct?
> > > In which case, over to Shakeel!
> >
> > Jan 10 is exactly when this bug was reported:
> > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> >
> > We don't know if that patch fixed the bug or not because nobody tested
> > the reproducer with that patch.
> >
> > It seems that the problem here is that nobody associated the fix with
> > the bug report. So people looking at open bug reports will spend time
> > again and again debugging this just to find that this was fixed months
> > ago. syzbot also doesn't have a chance to realize that this is fixed
> > and bisection is not necessary anymore. It also won't confirm/disprove
> > that the fix actually fixes the bug because even if the crash will
> > continue to happen it will look like the old crash just continues to
> > happen, so nothing to notify about.
> >
> > Associating fixes with bug reports solves all these problems for
> > humans and bots.
>
> Should we add "Reported-by" for syzbot reports on linux-next patches
> as well? Please note that these patches are in flux and might be
> dropped or completely changed before merging into Linus tree.

Reported-by will work, but may be confusing. It was discussed that for
squashed fixed Tested-by tag may be more appropriate and will also
work.
For dropped patches we don't have a better way other than marking it
as invalid for now.

> Also should syzbot drop bug reports on older linux-next trees if it
> can not be repro'ed in the latest linux-next tree? IMHO yes.

Please file an issue for this at:
https://github.com/google/syzkaller/issues
So far we don't have retesting in any form. Some bugs don't have
repros, so can't be retested. Any closing should probably happen after
long enough timeout to avoid spam, so bisection for them will most
likely happen anyway.
I can't promise any ETA for linux-next-specific work. Testing of
linux-next happens mostly because it's done on a rules not
significantly different from everything else (other trees, forks and
OSes).

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 2:46 PM Shakeel Butt <shakeelb@google.com> wrote:
>
> On Tue, Mar 12, 2019 at 1:33 AM Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> > >
> > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > >
> > > > > > syzbot has bisected this bug to:
> > > > > >
> > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > >
> > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > >
> > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > git tree:       linux-next
> > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > userspace arch: amd64
> > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > >
> > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > high_work")
> > > > >
> > > > > The following patch
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > might have fixed this.  Was it applied?
> > > >
> > > > Hi Andrew,
> > > >
> > > > You mean if the patch was applied during the bisection?
> > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > where the bug was hit. There are already too many factors that make
> > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > would not do the bisection at all. But it have not seen any patch in
> > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > >
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > testing the entire linux-next tree, so we can assume that
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > does not fix it, correct?
> > > In which case, over to Shakeel!
> >
> > Jan 10 is exactly when this bug was reported:
> > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> >
> > We don't know if that patch fixed the bug or not because nobody tested
> > the reproducer with that patch.
> >
> > It seems that the problem here is that nobody associated the fix with
> > the bug report. So people looking at open bug reports will spend time
> > again and again debugging this just to find that this was fixed months
> > ago. syzbot also doesn't have a chance to realize that this is fixed
> > and bisection is not necessary anymore. It also won't confirm/disprove
> > that the fix actually fixes the bug because even if the crash will
> > continue to happen it will look like the old crash just continues to
> > happen, so nothing to notify about.
> >
> > Associating fixes with bug reports solves all these problems for
> > humans and bots.
>
> Should we add "Reported-by" for syzbot reports on linux-next patches
> as well? Please note that these patches are in flux and might be
> dropped or completely changed before merging into Linus tree.

Reported-by will work, but may be confusing. It was discussed that for
squashed fixed Tested-by tag may be more appropriate and will also
work.
For dropped patches we don't have a better way other than marking it
as invalid for now.

> Also should syzbot drop bug reports on older linux-next trees if it
> can not be repro'ed in the latest linux-next tree? IMHO yes.

Please file an issue for this at:
https://github.com/google/syzkaller/issues
So far we don't have retesting in any form. Some bugs don't have
repros, so can't be retested. Any closing should probably happen after
long enough timeout to avoid spam, so bisection for them will most
likely happen anyway.
I can't promise any ETA for linux-next-specific work. Testing of
linux-next happens mostly because it's done on a rules not
significantly different from everything else (other trees, forks and
OSes).

Re: KASAN: null-ptr-deref Read in reclaim_high

[Eric Biggers]

Hi Dmitry,

On Tue, Mar 12, 2019 at 09:21:09AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> On Tue, Mar 12, 2019 at 7:43 AM Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > On Mon, Mar 11, 2019 at 11:25:41PM -0700, Andrew Morton wrote:
> > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > >
> > > > > > syzbot has bisected this bug to:
> > > > > >
> > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > >
> > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > >
> > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > git tree:       linux-next
> > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > userspace arch: amd64
> > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > >
> > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > high_work")
> > > > >
> > > > > The following patch
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > might have fixed this.  Was it applied?
> > > >
> > > > Hi Andrew,
> > > >
> > > > You mean if the patch was applied during the bisection?
> > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > where the bug was hit. There are already too many factors that make
> > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > would not do the bisection at all. But it have not seen any patch in
> > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > >
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > testing the entire linux-next tree, so we can assume that
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > does not fix it, correct?
> > >
> > > In which case, over to Shakeel!
> > >
> >
> > I don't understand what happened here.  First, the syzbot report doesn't say
> > which linux-next version was tested (which it should), but I get:
> >
> > $ git tag --contains 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > next-20190110
> > next-20190111
> > next-20190114
> > next-20190115
> > next-20190116
> >
> > That's almost 2 months old, yet this bug was just reported now.  Why?
> 
> Hi Eric,
> 
> This bug was reported on Jan 10:
> https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> 
> The start revision of the bisection process (provided) is the same
> that was used to create the reproducer. The end revision and bisection
> log are provided in the email.
> 
> How can we improve the format to make it more clear?
> 

syzbot started a new thread rather than sending the bisection result in the
existing thread.  So I thought it was a new bug report, as did everyone else
probably.

- Eric

Re: KASAN: null-ptr-deref Read in reclaim_high

[Eric Biggers]

On Tue, Mar 12, 2019 at 09:33:44AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > >
> > > > > syzbot has bisected this bug to:
> > > > >
> > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > >
> > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > >
> > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > git tree:       linux-next
> > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > userspace arch: amd64
> > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > >
> > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > high_work")
> > > >
> > > > The following patch
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > might have fixed this.  Was it applied?
> > >
> > > Hi Andrew,
> > >
> > > You mean if the patch was applied during the bisection?
> > > No, it wasn't. Bisection is very specifically done on the same tree
> > > where the bug was hit. There are already too many factors that make
> > > the result flaky/wrong/inconclusive without changing the tree state.
> > > Now, if syzbot would know about any pending fix for this bug, then it
> > > would not do the bisection at all. But it have not seen any patch in
> > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > fix commands for this bugs. Should have been it aware of the fix? How?
> >
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > added to linux-next on Jan 10.  I take it that this bug was hit when
> > testing the entire linux-next tree, so we can assume that
> > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > does not fix it, correct?
> > In which case, over to Shakeel!
> 
> Jan 10 is exactly when this bug was reported:
> https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> 
> We don't know if that patch fixed the bug or not because nobody tested
> the reproducer with that patch.
> 
> It seems that the problem here is that nobody associated the fix with
> the bug report. So people looking at open bug reports will spend time
> again and again debugging this just to find that this was fixed months
> ago. syzbot also doesn't have a chance to realize that this is fixed
> and bisection is not necessary anymore. It also won't confirm/disprove
> that the fix actually fixes the bug because even if the crash will
> continue to happen it will look like the old crash just continues to
> happen, so nothing to notify about.
> 
> Associating fixes with bug reports solves all these problems for
> humans and bots.
> 

I think syzbot needs to be more aggressive about invalidating old bug reports on
linux-next, e.g. automatically invalidate linux-next bugs that no longer occur
after a few weeks even if there is a reproducer.  Patches get added, changed,
and removed in linux-next every day.  Bugs that syzbot runs into on linux-next
are often obvious enough that they get reported by other people too, resulting
in bugs being fixed or dropped without people ever seeing the syzbot report.
How do you propose that people associate fixes with syzbot reports when they
never saw the syzbot report in the first place?

This is a problem on mainline too, of course.  But we *know* it's a more severe
problem on linux-next, and that a bug like this that only ever happened on
linux-next and stopped happening 2 months ago, is much less likely to be
relevant than a bug in mainline.  Kernel developers don't have time to examine
every single syzbot report so you need to help them out by reducing the noise.

- Eric

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 11:31 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> Hi Dmitry,
>
> On Tue, Mar 12, 2019 at 09:21:09AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > On Tue, Mar 12, 2019 at 7:43 AM Eric Biggers <ebiggers@kernel.org> wrote:
> > >
> > > On Mon, Mar 11, 2019 at 11:25:41PM -0700, Andrew Morton wrote:
> > > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > > >
> > > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > > <akpm@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > > >
> > > > > > > syzbot has bisected this bug to:
> > > > > > >
> > > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > > >
> > > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > > >
> > > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > > git tree:       linux-next
> > > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > > userspace arch: amd64
> > > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > > >
> > > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > > high_work")
> > > > > >
> > > > > > The following patch
> > > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > > might have fixed this.  Was it applied?
> > > > >
> > > > > Hi Andrew,
> > > > >
> > > > > You mean if the patch was applied during the bisection?
> > > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > > where the bug was hit. There are already too many factors that make
> > > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > > would not do the bisection at all. But it have not seen any patch in
> > > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > > >
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > > testing the entire linux-next tree, so we can assume that
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > does not fix it, correct?
> > > >
> > > > In which case, over to Shakeel!
> > > >
> > >
> > > I don't understand what happened here.  First, the syzbot report doesn't say
> > > which linux-next version was tested (which it should), but I get:
> > >
> > > $ git tag --contains 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > next-20190110
> > > next-20190111
> > > next-20190114
> > > next-20190115
> > > next-20190116
> > >
> > > That's almost 2 months old, yet this bug was just reported now.  Why?
> >
> > Hi Eric,
> >
> > This bug was reported on Jan 10:
> > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> >
> > The start revision of the bisection process (provided) is the same
> > that was used to create the reproducer. The end revision and bisection
> > log are provided in the email.
> >
> > How can we improve the format to make it more clear?
> >
>
> syzbot started a new thread rather than sending the bisection result in the
> existing thread.  So I thought it was a new bug report, as did everyone else
> probably.

There were not In-reply-to headers in first few bisect reports. This
should be fixed now. E.g. this one should be properly threaded:
https://groups.google.com/forum/#!topic/syzkaller-bugs/r2t3i5E78Mw

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 11:31 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> Hi Dmitry,
>
> On Tue, Mar 12, 2019 at 09:21:09AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > On Tue, Mar 12, 2019 at 7:43 AM Eric Biggers <ebiggers@kernel.org> wrote:
> > >
> > > On Mon, Mar 11, 2019 at 11:25:41PM -0700, Andrew Morton wrote:
> > > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > > >
> > > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > > <akpm@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > > >
> > > > > > > syzbot has bisected this bug to:
> > > > > > >
> > > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > > >
> > > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > > >
> > > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > > git tree:       linux-next
> > > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > > userspace arch: amd64
> > > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > > >
> > > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > > high_work")
> > > > > >
> > > > > > The following patch
> > > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > > might have fixed this.  Was it applied?
> > > > >
> > > > > Hi Andrew,
> > > > >
> > > > > You mean if the patch was applied during the bisection?
> > > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > > where the bug was hit. There are already too many factors that make
> > > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > > would not do the bisection at all. But it have not seen any patch in
> > > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > > >
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > > testing the entire linux-next tree, so we can assume that
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > does not fix it, correct?
> > > >
> > > > In which case, over to Shakeel!
> > > >
> > >
> > > I don't understand what happened here.  First, the syzbot report doesn't say
> > > which linux-next version was tested (which it should), but I get:
> > >
> > > $ git tag --contains 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > next-20190110
> > > next-20190111
> > > next-20190114
> > > next-20190115
> > > next-20190116
> > >
> > > That's almost 2 months old, yet this bug was just reported now.  Why?
> >
> > Hi Eric,
> >
> > This bug was reported on Jan 10:
> > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> >
> > The start revision of the bisection process (provided) is the same
> > that was used to create the reproducer. The end revision and bisection
> > log are provided in the email.
> >
> > How can we improve the format to make it more clear?
> >
>
> syzbot started a new thread rather than sending the bisection result in the
> existing thread.  So I thought it was a new bug report, as did everyone else
> probably.

There were not In-reply-to headers in first few bisect reports. This
should be fixed now. E.g. this one should be properly threaded:
https://groups.google.com/forum/#!topic/syzkaller-bugs/r2t3i5E78Mw

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 11:50 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Tue, Mar 12, 2019 at 09:33:44AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> > >
> > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > >
> > > > > > syzbot has bisected this bug to:
> > > > > >
> > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > >
> > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > >
> > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > git tree:       linux-next
> > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > userspace arch: amd64
> > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > >
> > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > high_work")
> > > > >
> > > > > The following patch
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > might have fixed this.  Was it applied?
> > > >
> > > > Hi Andrew,
> > > >
> > > > You mean if the patch was applied during the bisection?
> > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > where the bug was hit. There are already too many factors that make
> > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > would not do the bisection at all. But it have not seen any patch in
> > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > >
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > testing the entire linux-next tree, so we can assume that
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > does not fix it, correct?
> > > In which case, over to Shakeel!
> >
> > Jan 10 is exactly when this bug was reported:
> > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> >
> > We don't know if that patch fixed the bug or not because nobody tested
> > the reproducer with that patch.
> >
> > It seems that the problem here is that nobody associated the fix with
> > the bug report. So people looking at open bug reports will spend time
> > again and again debugging this just to find that this was fixed months
> > ago. syzbot also doesn't have a chance to realize that this is fixed
> > and bisection is not necessary anymore. It also won't confirm/disprove
> > that the fix actually fixes the bug because even if the crash will
> > continue to happen it will look like the old crash just continues to
> > happen, so nothing to notify about.
> >
> > Associating fixes with bug reports solves all these problems for
> > humans and bots.
> >
>
> I think syzbot needs to be more aggressive about invalidating old bug reports on
> linux-next, e.g. automatically invalidate linux-next bugs that no longer occur
> after a few weeks even if there is a reproducer.  Patches get added, changed,
> and removed in linux-next every day.  Bugs that syzbot runs into on linux-next
> are often obvious enough that they get reported by other people too, resulting
> in bugs being fixed or dropped without people ever seeing the syzbot report.
> How do you propose that people associate fixes with syzbot reports when they
> never saw the syzbot report in the first place?
>
> This is a problem on mainline too, of course.  But we *know* it's a more severe
> problem on linux-next, and that a bug like this that only ever happened on
> linux-next and stopped happening 2 months ago, is much less likely to be
> relevant than a bug in mainline.  Kernel developers don't have time to examine
> every single syzbot report so you need to help them out by reducing the noise.

Please file an issue for this at https://github.com/google/syzkaller/issues

I also wonder how does this work for all other kernel bugs reports?
syzbot is not the only one reporting kernel bugs and we don't want to
invent new rules here.

Also note that what happens now may be not representative of what will
happen in a steady mode later. Now syzbot bisects old bugs accumulated
over 1+ year. Later if it reports a bug, it should bisect sooner. So
all of what happens in this bug report won't take place.

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Tue, Mar 12, 2019 at 11:50 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Tue, Mar 12, 2019 at 09:33:44AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> > >
> > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > >
> > > > > > syzbot has bisected this bug to:
> > > > > >
> > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > >
> > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > >
> > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > git tree:       linux-next
> > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > userspace arch: amd64
> > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > >
> > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > high_work")
> > > > >
> > > > > The following patch
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > might have fixed this.  Was it applied?
> > > >
> > > > Hi Andrew,
> > > >
> > > > You mean if the patch was applied during the bisection?
> > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > where the bug was hit. There are already too many factors that make
> > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > would not do the bisection at all. But it have not seen any patch in
> > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > >
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > testing the entire linux-next tree, so we can assume that
> > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > does not fix it, correct?
> > > In which case, over to Shakeel!
> >
> > Jan 10 is exactly when this bug was reported:
> > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> >
> > We don't know if that patch fixed the bug or not because nobody tested
> > the reproducer with that patch.
> >
> > It seems that the problem here is that nobody associated the fix with
> > the bug report. So people looking at open bug reports will spend time
> > again and again debugging this just to find that this was fixed months
> > ago. syzbot also doesn't have a chance to realize that this is fixed
> > and bisection is not necessary anymore. It also won't confirm/disprove
> > that the fix actually fixes the bug because even if the crash will
> > continue to happen it will look like the old crash just continues to
> > happen, so nothing to notify about.
> >
> > Associating fixes with bug reports solves all these problems for
> > humans and bots.
> >
>
> I think syzbot needs to be more aggressive about invalidating old bug reports on
> linux-next, e.g. automatically invalidate linux-next bugs that no longer occur
> after a few weeks even if there is a reproducer.  Patches get added, changed,
> and removed in linux-next every day.  Bugs that syzbot runs into on linux-next
> are often obvious enough that they get reported by other people too, resulting
> in bugs being fixed or dropped without people ever seeing the syzbot report.
> How do you propose that people associate fixes with syzbot reports when they
> never saw the syzbot report in the first place?
>
> This is a problem on mainline too, of course.  But we *know* it's a more severe
> problem on linux-next, and that a bug like this that only ever happened on
> linux-next and stopped happening 2 months ago, is much less likely to be
> relevant than a bug in mainline.  Kernel developers don't have time to examine
> every single syzbot report so you need to help them out by reducing the noise.

Please file an issue for this at https://github.com/google/syzkaller/issues

I also wonder how does this work for all other kernel bugs reports?
syzbot is not the only one reporting kernel bugs and we don't want to
invent new rules here.

Also note that what happens now may be not representative of what will
happen in a steady mode later. Now syzbot bisects old bugs accumulated
over 1+ year. Later if it reports a bug, it should bisect sooner. So
all of what happens in this bug report won't take place.

Re: KASAN: null-ptr-deref Read in reclaim_high

[Eric Biggers]

On Wed, Mar 13, 2019 at 09:24:21AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> On Tue, Mar 12, 2019 at 11:50 PM Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > On Tue, Mar 12, 2019 at 09:33:44AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > > On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > > >
> > > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > > <akpm@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > > >
> > > > > > > syzbot has bisected this bug to:
> > > > > > >
> > > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > > >
> > > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > > >
> > > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > > git tree:       linux-next
> > > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > > userspace arch: amd64
> > > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > > >
> > > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > > high_work")
> > > > > >
> > > > > > The following patch
> > > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > > might have fixed this.  Was it applied?
> > > > >
> > > > > Hi Andrew,
> > > > >
> > > > > You mean if the patch was applied during the bisection?
> > > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > > where the bug was hit. There are already too many factors that make
> > > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > > would not do the bisection at all. But it have not seen any patch in
> > > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > > >
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > > testing the entire linux-next tree, so we can assume that
> > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > does not fix it, correct?
> > > > In which case, over to Shakeel!
> > >
> > > Jan 10 is exactly when this bug was reported:
> > > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> > > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > >
> > > We don't know if that patch fixed the bug or not because nobody tested
> > > the reproducer with that patch.
> > >
> > > It seems that the problem here is that nobody associated the fix with
> > > the bug report. So people looking at open bug reports will spend time
> > > again and again debugging this just to find that this was fixed months
> > > ago. syzbot also doesn't have a chance to realize that this is fixed
> > > and bisection is not necessary anymore. It also won't confirm/disprove
> > > that the fix actually fixes the bug because even if the crash will
> > > continue to happen it will look like the old crash just continues to
> > > happen, so nothing to notify about.
> > >
> > > Associating fixes with bug reports solves all these problems for
> > > humans and bots.
> > >
> >
> > I think syzbot needs to be more aggressive about invalidating old bug reports on
> > linux-next, e.g. automatically invalidate linux-next bugs that no longer occur
> > after a few weeks even if there is a reproducer.  Patches get added, changed,
> > and removed in linux-next every day.  Bugs that syzbot runs into on linux-next
> > are often obvious enough that they get reported by other people too, resulting
> > in bugs being fixed or dropped without people ever seeing the syzbot report.
> > How do you propose that people associate fixes with syzbot reports when they
> > never saw the syzbot report in the first place?
> >
> > This is a problem on mainline too, of course.  But we *know* it's a more severe
> > problem on linux-next, and that a bug like this that only ever happened on
> > linux-next and stopped happening 2 months ago, is much less likely to be
> > relevant than a bug in mainline.  Kernel developers don't have time to examine
> > every single syzbot report so you need to help them out by reducing the noise.
> 
> Please file an issue for this at https://github.com/google/syzkaller/issues

I filed https://github.com/google/syzkaller/issues/1054

> 
> I also wonder how does this work for all other kernel bugs reports?
> syzbot is not the only one reporting kernel bugs and we don't want to
> invent new rules here.

Well, I think you already know the answer to that.  There's no unified bug
tracking system for all kernel subsystems, so in the worst case bugs/features
just get ignored until someone cares to bring it up again.  I know you want to
change that, but the larger problem is that there aren't enough people able and
funded to do the work.  For the kernel overall (some subsystems are better, OFC)
there so many low-quality, duplicate, or irrelevant reports/requests that no one
can keep up.  That means maintainers have to focus on the highest priority
reports/requests, such as the ones that are clearly relevant and get continued
discussion, vs. some random problem someone had 2 years ago.  Just putting stuff
on a bug tracker does not magically make people work on it.

I think the reality is that until people can actually be funded to immediately
analyze every syzbot report, syzbot needs to be designed to help developers
focus on the reports most likely to still be actual bugs.  That means
automatically closing bugs where the crash is no longer occurring, especially if
it was on linux-next; and sending reminders if the crash is still occurring.

> 
> Also note that what happens now may be not representative of what will
> happen in a steady mode later. Now syzbot bisects old bugs accumulated
> over 1+ year. Later if it reports a bug, it should bisect sooner. So
> all of what happens in this bug report won't take place.
> 

Sure, but I think there will continue to be syzbot reports that the relevant
people either don't see, or don't have time or expertise to look into.  This is
especially true when the same bug is filed as many different bug reports.

- Eric

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Wed, Mar 13, 2019 at 7:16 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Wed, Mar 13, 2019 at 09:24:21AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > On Tue, Mar 12, 2019 at 11:50 PM Eric Biggers <ebiggers@kernel.org> wrote:
> > >
> > > On Tue, Mar 12, 2019 at 09:33:44AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > > > On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > > > >
> > > > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > > > <akpm@linux-foundation.org> wrote:
> > > > > > >
> > > > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > > > >
> > > > > > > > syzbot has bisected this bug to:
> > > > > > > >
> > > > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > > > >
> > > > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > > > >
> > > > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > > > git tree:       linux-next
> > > > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > > > userspace arch: amd64
> > > > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > > > >
> > > > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > > > high_work")
> > > > > > >
> > > > > > > The following patch
> > > > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > > > might have fixed this.  Was it applied?
> > > > > >
> > > > > > Hi Andrew,
> > > > > >
> > > > > > You mean if the patch was applied during the bisection?
> > > > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > > > where the bug was hit. There are already too many factors that make
> > > > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > > > would not do the bisection at all. But it have not seen any patch in
> > > > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > > > >
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > > > testing the entire linux-next tree, so we can assume that
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > does not fix it, correct?
> > > > > In which case, over to Shakeel!
> > > >
> > > > Jan 10 is exactly when this bug was reported:
> > > > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> > > > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > >
> > > > We don't know if that patch fixed the bug or not because nobody tested
> > > > the reproducer with that patch.
> > > >
> > > > It seems that the problem here is that nobody associated the fix with
> > > > the bug report. So people looking at open bug reports will spend time
> > > > again and again debugging this just to find that this was fixed months
> > > > ago. syzbot also doesn't have a chance to realize that this is fixed
> > > > and bisection is not necessary anymore. It also won't confirm/disprove
> > > > that the fix actually fixes the bug because even if the crash will
> > > > continue to happen it will look like the old crash just continues to
> > > > happen, so nothing to notify about.
> > > >
> > > > Associating fixes with bug reports solves all these problems for
> > > > humans and bots.
> > > >
> > >
> > > I think syzbot needs to be more aggressive about invalidating old bug reports on
> > > linux-next, e.g. automatically invalidate linux-next bugs that no longer occur
> > > after a few weeks even if there is a reproducer.  Patches get added, changed,
> > > and removed in linux-next every day.  Bugs that syzbot runs into on linux-next
> > > are often obvious enough that they get reported by other people too, resulting
> > > in bugs being fixed or dropped without people ever seeing the syzbot report.
> > > How do you propose that people associate fixes with syzbot reports when they
> > > never saw the syzbot report in the first place?
> > >
> > > This is a problem on mainline too, of course.  But we *know* it's a more severe
> > > problem on linux-next, and that a bug like this that only ever happened on
> > > linux-next and stopped happening 2 months ago, is much less likely to be
> > > relevant than a bug in mainline.  Kernel developers don't have time to examine
> > > every single syzbot report so you need to help them out by reducing the noise.
> >
> > Please file an issue for this at https://github.com/google/syzkaller/issues
>
> I filed https://github.com/google/syzkaller/issues/1054

Thanks.

> > I also wonder how does this work for all other kernel bugs reports?
> > syzbot is not the only one reporting kernel bugs and we don't want to
> > invent new rules here.
>
> Well, I think you already know the answer to that.  There's no unified bug
> tracking system for all kernel subsystems, so in the worst case bugs/features
> just get ignored until someone cares to bring it up again.  I know you want to
> change that, but the larger problem is that there aren't enough people able and
> funded to do the work.  For the kernel overall (some subsystems are better, OFC)
> there so many low-quality, duplicate, or irrelevant reports/requests that no one
> can keep up.  That means maintainers have to focus on the highest priority

Interesting point in "distributed" kernel testing versus "the" kernel testing.
Obviously if we have 50 dispersed testing efforts we do get tons of
duplicates. And these reports are of low quality because nobody wants
to invest into 1/50-th of testing with unclear future nor duplicate
work 50x. This results in tons of unproductive work for everybody.
E.g. all the work on syzbot bisection. All of this was already done
multiple times. And I am sure Linus already explained everything he
explains to me multiple times before. But this still does not get us
anywhere because it's just syzbot, so does not benefit anything else
and Linus will need to explain the same again and again...

> reports/requests, such as the ones that are clearly relevant and get continued
> discussion, vs. some random problem someone had 2 years ago.  Just putting stuff
> on a bug tracker does not magically make people work on it.
>
> I think the reality is that until people can actually be funded to immediately
> analyze every syzbot report, syzbot needs to be designed to help developers
> focus on the reports most likely to still be actual bugs.  That means
> automatically closing bugs where the crash is no longer occurring, especially if
> it was on linux-next; and sending reminders if the crash is still occurring.
> >
> > Also note that what happens now may be not representative of what will
> > happen in a steady mode later. Now syzbot bisects old bugs accumulated
> > over 1+ year. Later if it reports a bug, it should bisect sooner. So
> > all of what happens in this bug report won't take place.
> >
>
> Sure, but I think there will continue to be syzbot reports that the relevant
> people either don't see, or don't have time or expertise to look into.  This is
> especially true when the same bug is filed as many different bug reports.
>
> - Eric

Re: KASAN: null-ptr-deref Read in reclaim_high

[Dmitry Vyukov]

On Wed, Mar 13, 2019 at 7:16 PM Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Wed, Mar 13, 2019 at 09:24:21AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > On Tue, Mar 12, 2019 at 11:50 PM Eric Biggers <ebiggers@kernel.org> wrote:
> > >
> > > On Tue, Mar 12, 2019 at 09:33:44AM +0100, 'Dmitry Vyukov' via syzkaller-bugs wrote:
> > > > On Tue, Mar 12, 2019 at 7:25 AM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Tue, 12 Mar 2019 07:08:38 +0100 Dmitry Vyukov <dvyukov@google.com> wrote:
> > > > >
> > > > > > On Tue, Mar 12, 2019 at 12:37 AM Andrew Morton
> > > > > > <akpm@linux-foundation.org> wrote:
> > > > > > >
> > > > > > > On Mon, 11 Mar 2019 06:08:01 -0700 syzbot <syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com> wrote:
> > > > > > >
> > > > > > > > syzbot has bisected this bug to:
> > > > > > > >
> > > > > > > > commit 29a4b8e275d1f10c51c7891362877ef6cffae9e7
> > > > > > > > Author: Shakeel Butt <shakeelb@google.com>
> > > > > > > > Date:   Wed Jan 9 22:02:21 2019 +0000
> > > > > > > >
> > > > > > > >      memcg: schedule high reclaim for remote memcgs on high_work
> > > > > > > >
> > > > > > > > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=155bf5db200000
> > > > > > > > start commit:   29a4b8e2 memcg: schedule high reclaim for remote memcgs on..
> > > > > > > > git tree:       linux-next
> > > > > > > > final crash:    https://syzkaller.appspot.com/x/report.txt?x=175bf5db200000
> > > > > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=135bf5db200000
> > > > > > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
> > > > > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > > > > > > userspace arch: amd64
> > > > > > > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
> > > > > > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000
> > > > > > > >
> > > > > > > > Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com
> > > > > > > > Fixes: 29a4b8e2 ("memcg: schedule high reclaim for remote memcgs on
> > > > > > > > high_work")
> > > > > > >
> > > > > > > The following patch
> > > > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > > > might have fixed this.  Was it applied?
> > > > > >
> > > > > > Hi Andrew,
> > > > > >
> > > > > > You mean if the patch was applied during the bisection?
> > > > > > No, it wasn't. Bisection is very specifically done on the same tree
> > > > > > where the bug was hit. There are already too many factors that make
> > > > > > the result flaky/wrong/inconclusive without changing the tree state.
> > > > > > Now, if syzbot would know about any pending fix for this bug, then it
> > > > > > would not do the bisection at all. But it have not seen any patch in
> > > > > > upstream/linux-next with the Reported-by tag, nor it received any syz
> > > > > > fix commands for this bugs. Should have been it aware of the fix? How?
> > > > >
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch was
> > > > > added to linux-next on Jan 10.  I take it that this bug was hit when
> > > > > testing the entire linux-next tree, so we can assume that
> > > > > memcg-schedule-high-reclaim-for-remote-memcgs-on-high_work-v3.patch
> > > > > does not fix it, correct?
> > > > > In which case, over to Shakeel!
> > > >
> > > > Jan 10 is exactly when this bug was reported:
> > > > https://groups.google.com/forum/#!msg/syzkaller-bugs/5YkhNUg2PFY/4-B5M7bDCAAJ
> > > > https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
> > > >
> > > > We don't know if that patch fixed the bug or not because nobody tested
> > > > the reproducer with that patch.
> > > >
> > > > It seems that the problem here is that nobody associated the fix with
> > > > the bug report. So people looking at open bug reports will spend time
> > > > again and again debugging this just to find that this was fixed months
> > > > ago. syzbot also doesn't have a chance to realize that this is fixed
> > > > and bisection is not necessary anymore. It also won't confirm/disprove
> > > > that the fix actually fixes the bug because even if the crash will
> > > > continue to happen it will look like the old crash just continues to
> > > > happen, so nothing to notify about.
> > > >
> > > > Associating fixes with bug reports solves all these problems for
> > > > humans and bots.
> > > >
> > >
> > > I think syzbot needs to be more aggressive about invalidating old bug reports on
> > > linux-next, e.g. automatically invalidate linux-next bugs that no longer occur
> > > after a few weeks even if there is a reproducer.  Patches get added, changed,
> > > and removed in linux-next every day.  Bugs that syzbot runs into on linux-next
> > > are often obvious enough that they get reported by other people too, resulting
> > > in bugs being fixed or dropped without people ever seeing the syzbot report.
> > > How do you propose that people associate fixes with syzbot reports when they
> > > never saw the syzbot report in the first place?
> > >
> > > This is a problem on mainline too, of course.  But we *know* it's a more severe
> > > problem on linux-next, and that a bug like this that only ever happened on
> > > linux-next and stopped happening 2 months ago, is much less likely to be
> > > relevant than a bug in mainline.  Kernel developers don't have time to examine
> > > every single syzbot report so you need to help them out by reducing the noise.
> >
> > Please file an issue for this at https://github.com/google/syzkaller/issues
>
> I filed https://github.com/google/syzkaller/issues/1054

Thanks.

> > I also wonder how does this work for all other kernel bugs reports?
> > syzbot is not the only one reporting kernel bugs and we don't want to
> > invent new rules here.
>
> Well, I think you already know the answer to that.  There's no unified bug
> tracking system for all kernel subsystems, so in the worst case bugs/features
> just get ignored until someone cares to bring it up again.  I know you want to
> change that, but the larger problem is that there aren't enough people able and
> funded to do the work.  For the kernel overall (some subsystems are better, OFC)
> there so many low-quality, duplicate, or irrelevant reports/requests that no one
> can keep up.  That means maintainers have to focus on the highest priority

Interesting point in "distributed" kernel testing versus "the" kernel testing.
Obviously if we have 50 dispersed testing efforts we do get tons of
duplicates. And these reports are of low quality because nobody wants
to invest into 1/50-th of testing with unclear future nor duplicate
work 50x. This results in tons of unproductive work for everybody.
E.g. all the work on syzbot bisection. All of this was already done
multiple times. And I am sure Linus already explained everything he
explains to me multiple times before. But this still does not get us
anywhere because it's just syzbot, so does not benefit anything else
and Linus will need to explain the same again and again...

> reports/requests, such as the ones that are clearly relevant and get continued
> discussion, vs. some random problem someone had 2 years ago.  Just putting stuff
> on a bug tracker does not magically make people work on it.
>
> I think the reality is that until people can actually be funded to immediately
> analyze every syzbot report, syzbot needs to be designed to help developers
> focus on the reports most likely to still be actual bugs.  That means
> automatically closing bugs where the crash is no longer occurring, especially if
> it was on linux-next; and sending reminders if the crash is still occurring.
> >
> > Also note that what happens now may be not representative of what will
> > happen in a steady mode later. Now syzbot bisects old bugs accumulated
> > over 1+ year. Later if it reports a bug, it should bisect sooner. So
> > all of what happens in this bug report won't take place.
> >
>
> Sure, but I think there will continue to be syzbot reports that the relevant
> people either don't see, or don't have time or expertise to look into.  This is
> especially true when the same bug is filed as many different bug reports.
>
> - Eric

KASAN: null-ptr-deref Read in reclaim_high

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit:    6cab33afc3dd Add linux-next specific files for 20190110
git tree:       linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=178b287b400000
kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
compiler:       gcc (GCC) 9.0.0 20181231 (experimental)
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com

==================================================================
BUG: KASAN: null-ptr-deref in atomic64_read  
include/generated/atomic-instrumented.h:836 [inline]
BUG: KASAN: null-ptr-deref in atomic_long_read  
include/generated/atomic-long.h:28 [inline]
BUG: KASAN: null-ptr-deref in page_counter_read  
include/linux/page_counter.h:47 [inline]
BUG: KASAN: null-ptr-deref in reclaim_high.constprop.0+0xa6/0x1e0  
mm/memcontrol.c:2149
Read of size 8 at addr 0000000000000138 by task syz-executor037/7964

CPU: 1 PID: 7964 Comm: syz-executor037 Not tainted 5.0.0-rc1-next-20190110  
#9
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS  
Google 01/01/2011
Call Trace:
  __dump_stack lib/dump_stack.c:77 [inline]
  dump_stack+0x1db/0x2d0 lib/dump_stack.c:113
  kasan_report.cold+0x5/0x40 mm/kasan/report.c:321
  check_memory_region_inline mm/kasan/generic.c:185 [inline]
  check_memory_region+0x123/0x190 mm/kasan/generic.c:191
  kasan_check_read+0x11/0x20 mm/kasan/common.c:100
  atomic64_read include/generated/atomic-instrumented.h:836 [inline]
  atomic_long_read include/generated/atomic-long.h:28 [inline]
  page_counter_read include/linux/page_counter.h:47 [inline]
  reclaim_high.constprop.0+0xa6/0x1e0 mm/memcontrol.c:2149
  mem_cgroup_handle_over_high+0xc1/0x180 mm/memcontrol.c:2178
  tracehook_notify_resume include/linux/tracehook.h:190 [inline]
  exit_to_usermode_loop+0x299/0x3b0 arch/x86/entry/common.c:166
  prepare_exit_to_usermode arch/x86/entry/common.c:197 [inline]
  syscall_return_slowpath+0x519/0x5f0 arch/x86/entry/common.c:268
  ret_from_fork+0x15/0x50 arch/x86/entry/entry_64.S:344
RIP: 0033:0x44034a
Code: Bad RIP value.
RSP: 002b:00007ffc31cd3040 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000044034a
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
RBP: 00007ffc31cd3060 R08: 0000000000000001 R09: 0000000002027880
R10: 0000000002027b50 R11: 0000000000000246 R12: 0000000000000001
R13: 000000000000cc59 R14: 0000000000000000 R15: 0000000000000000
==================================================================


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with  
syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches

KASAN: null-ptr-deref Read in reclaim_high

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit:    6cab33afc3dd Add linux-next specific files for 20190110
git tree:       linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=178b287b400000
kernel config:  https://syzkaller.appspot.com/x/.config?x=611f89e5b6868db
dashboard link: https://syzkaller.appspot.com/bug?extid=fa11f9da42b46cea3b4a
compiler:       gcc (GCC) 9.0.0 20181231 (experimental)
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14259017400000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=141630a0c00000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+fa11f9da42b46cea3b4a@syzkaller.appspotmail.com

==================================================================
BUG: KASAN: null-ptr-deref in atomic64_read  
include/generated/atomic-instrumented.h:836 [inline]
BUG: KASAN: null-ptr-deref in atomic_long_read  
include/generated/atomic-long.h:28 [inline]
BUG: KASAN: null-ptr-deref in page_counter_read  
include/linux/page_counter.h:47 [inline]
BUG: KASAN: null-ptr-deref in reclaim_high.constprop.0+0xa6/0x1e0  
mm/memcontrol.c:2149
Read of size 8 at addr 0000000000000138 by task syz-executor037/7964

CPU: 1 PID: 7964 Comm: syz-executor037 Not tainted 5.0.0-rc1-next-20190110  
#9
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS  
Google 01/01/2011
Call Trace:
  __dump_stack lib/dump_stack.c:77 [inline]
  dump_stack+0x1db/0x2d0 lib/dump_stack.c:113
  kasan_report.cold+0x5/0x40 mm/kasan/report.c:321
  check_memory_region_inline mm/kasan/generic.c:185 [inline]
  check_memory_region+0x123/0x190 mm/kasan/generic.c:191
  kasan_check_read+0x11/0x20 mm/kasan/common.c:100
  atomic64_read include/generated/atomic-instrumented.h:836 [inline]
  atomic_long_read include/generated/atomic-long.h:28 [inline]
  page_counter_read include/linux/page_counter.h:47 [inline]
  reclaim_high.constprop.0+0xa6/0x1e0 mm/memcontrol.c:2149
  mem_cgroup_handle_over_high+0xc1/0x180 mm/memcontrol.c:2178
  tracehook_notify_resume include/linux/tracehook.h:190 [inline]
  exit_to_usermode_loop+0x299/0x3b0 arch/x86/entry/common.c:166
  prepare_exit_to_usermode arch/x86/entry/common.c:197 [inline]
  syscall_return_slowpath+0x519/0x5f0 arch/x86/entry/common.c:268
  ret_from_fork+0x15/0x50 arch/x86/entry/entry_64.S:344
RIP: 0033:0x44034a
Code: Bad RIP value.
RSP: 002b:00007ffc31cd3040 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000044034a
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
RBP: 00007ffc31cd3060 R08: 0000000000000001 R09: 0000000002027880
R10: 0000000002027b50 R11: 0000000000000246 R12: 0000000000000001
R13: 000000000000cc59 R14: 0000000000000000 R15: 0000000000000000
==================================================================


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with  
syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches