Re: kernel BUG at mm/vmalloc.c:LINE! (2)

Re: kernel BUG at mm/vmalloc.c:LINE! (2)

[Qian Cai]

On Mon, Jul 20, 2020 at 10:06:18PM +0200, Uladzislau Rezki wrote:
> On Mon, Jul 20, 2020 at 09:48:21AM -0700, syzbot wrote:
> > syzbot has found a reproducer for the following issue on:
> > 
> > HEAD commit:    ab8be66e Add linux-next specific files for 20200720
> > git tree:       linux-next
> > console output: https://syzkaller.appspot.com/x/log.txt?x=161a0cc8900000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=c4bf77d63d0cf88c
> > dashboard link: https://syzkaller.appspot.com/bug?extid=5f326d255ca648131f87
> > compiler:       gcc (GCC) 10.1.0-syz 20200507
> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=151192bb100000
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=12d7a873100000
> > 
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+5f326d255ca648131f87@syzkaller.appspotmail.com
> > 
> > ------------[ cut here ]------------
> > kernel BUG at mm/vmalloc.c:3089!
> > invalid opcode: 0000 [#1] PREEMPT SMP KASAN
> > CPU: 0 PID: 12 Comm: kworker/0:1 Not tainted 5.8.0-rc6-next-20200720-syzkaller #0
> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> > Workqueue: events pcpu_balance_workfn
> > RIP: 0010:free_vm_area mm/vmalloc.c:3089 [inline]
> > RIP: 0010:free_vm_area mm/vmalloc.c:3085 [inline]
> > RIP: 0010:pcpu_free_vm_areas+0x96/0xc0 mm/vmalloc.c:3432
> > Code: 75 48 48 8b 2b 48 8d 7d 08 48 89 f8 48 c1 e8 03 42 80 3c 30 00 75 2c 48 8b 7d 08 e8 c4 c8 ff ff 48 39 c5 74 a5 e8 ea c3 c9 ff <0f> 0b e8 e3 c3 c9 ff 4c 89 ff 5b 5d 41 5c 41 5d 41 5e 41 5f e9 71
> > RSP: 0018:ffffc90000d2fba8 EFLAGS: 00010293
> > RAX: 0000000000000000 RBX: ffff8880a801be00 RCX: 0000000000000000
> > RDX: ffff8880a95fa300 RSI: ffffffff81aa7c76 RDI: 0000000000000001
> > RBP: ffff8880a2b38180 R08: 0000000000000000 R09: ffffffff89cfecc3
> > R10: fffffbfff139fd98 R11: 0000000000000000 R12: 0000000000000000
> > R13: 0000000000000001 R14: dffffc0000000000 R15: ffff8880a801be00
> > FS:  0000000000000000(0000) GS:ffff8880ae600000(0000) knlGS:0000000000000000
> > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > CR2: 00000000004c8e48 CR3: 00000000a4c08000 CR4: 00000000001506f0
> > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > Call Trace:
> >  pcpu_destroy_chunk mm/percpu-vm.c:366 [inline]
> >  __pcpu_balance_workfn mm/percpu.c:1982 [inline]
> >  pcpu_balance_workfn+0x8b3/0x1310 mm/percpu.c:2069
> >  process_one_work+0x94c/0x1670 kernel/workqueue.c:2269
> >  worker_thread+0x64c/0x1120 kernel/workqueue.c:2415
> >  kthread+0x3b5/0x4a0 kernel/kthread.c:292
> >  ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
> > Modules linked in:
> > ---[ end trace 6a2e56ec52e1f480 ]---
> > RIP: 0010:free_vm_area mm/vmalloc.c:3089 [inline]
> > RIP: 0010:free_vm_area mm/vmalloc.c:3085 [inline]
> > RIP: 0010:pcpu_free_vm_areas+0x96/0xc0 mm/vmalloc.c:3432
> > Code: 75 48 48 8b 2b 48 8d 7d 08 48 89 f8 48 c1 e8 03 42 80 3c 30 00 75 2c 48 8b 7d 08 e8 c4 c8 ff ff 48 39 c5 74 a5 e8 ea c3 c9 ff <0f> 0b e8 e3 c3 c9 ff 4c 89 ff 5b 5d 41 5c 41 5d 41 5e 41 5f e9 71
> > RSP: 0018:ffffc90000d2fba8 EFLAGS: 00010293
> > RAX: 0000000000000000 RBX: ffff8880a801be00 RCX: 0000000000000000
> > RDX: ffff8880a95fa300 RSI: ffffffff81aa7c76 RDI: 0000000000000001
> > RBP: ffff8880a2b38180 R08: 0000000000000000 R09: ffffffff89cfecc3
> > R10: fffffbfff139fd98 R11: 0000000000000000 R12: 0000000000000000
> > R13: 0000000000000001 R14: dffffc0000000000 R15: ffff8880a801be00
> > FS:  0000000000000000(0000) GS:ffff8880ae600000(0000) knlGS:0000000000000000
> > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > CR2: 00000000004c8e48 CR3: 00000000a4c08000 CR4: 00000000001506f0
> > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > 
> That is because of below revert:
> 
> <snip>
> commit bdbfb1d52d5e576c1d275fd8ab59b677011229e8
> Author: Ingo Molnar <mingo@kernel.org>
> Date:   Sun Jun 7 21:12:51 2020 +0200
> 
>     Revert "mm/vmalloc: modify struct vmap_area to reduce its size"
>     
>     This reverts commit 688fcbfc06e4fdfbb7e1d5a942a1460fe6379d2d.
>     
>     Signed-off-by: Ingo Molnar <mingo@kernel.org>
>     
>     Conflicts:
>             mm/vmalloc.c
> <snip>
> 
> I can check further, but it can be it was not correctly reverted,
> because everything should work just fine even with the revert,
> though i i do not understand a reason of reverting.

Vlad, how sure are you about this? We also start to trigger this now on
linux-next, but the reverting patch surely looks like doggy without any useful
information in the commit description.

Re: kernel BUG at mm/vmalloc.c:LINE! (2)

[Uladzislau Rezki]

> > > syzbot has found a reproducer for the following issue on:
> > > 
> > > HEAD commit:    ab8be66e Add linux-next specific files for 20200720
> > > git tree:       linux-next
> > > console output: https://syzkaller.appspot.com/x/log.txt?x=161a0cc8900000
> > > kernel config:  https://syzkaller.appspot.com/x/.config?x=c4bf77d63d0cf88c
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=5f326d255ca648131f87
> > > compiler:       gcc (GCC) 10.1.0-syz 20200507
> > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=151192bb100000
> > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=12d7a873100000
> > > 
> > > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > > Reported-by: syzbot+5f326d255ca648131f87@syzkaller.appspotmail.com
> > > 
> > > ------------[ cut here ]------------
> > > kernel BUG at mm/vmalloc.c:3089!
> > > invalid opcode: 0000 [#1] PREEMPT SMP KASAN
> > > CPU: 0 PID: 12 Comm: kworker/0:1 Not tainted 5.8.0-rc6-next-20200720-syzkaller #0
> > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> > > Workqueue: events pcpu_balance_workfn
> > > RIP: 0010:free_vm_area mm/vmalloc.c:3089 [inline]
> > > RIP: 0010:free_vm_area mm/vmalloc.c:3085 [inline]
> > > RIP: 0010:pcpu_free_vm_areas+0x96/0xc0 mm/vmalloc.c:3432
> > > Code: 75 48 48 8b 2b 48 8d 7d 08 48 89 f8 48 c1 e8 03 42 80 3c 30 00 75 2c 48 8b 7d 08 e8 c4 c8 ff ff 48 39 c5 74 a5 e8 ea c3 c9 ff <0f> 0b e8 e3 c3 c9 ff 4c 89 ff 5b 5d 41 5c 41 5d 41 5e 41 5f e9 71
> > > RSP: 0018:ffffc90000d2fba8 EFLAGS: 00010293
> > > RAX: 0000000000000000 RBX: ffff8880a801be00 RCX: 0000000000000000
> > > RDX: ffff8880a95fa300 RSI: ffffffff81aa7c76 RDI: 0000000000000001
> > > RBP: ffff8880a2b38180 R08: 0000000000000000 R09: ffffffff89cfecc3
> > > R10: fffffbfff139fd98 R11: 0000000000000000 R12: 0000000000000000
> > > R13: 0000000000000001 R14: dffffc0000000000 R15: ffff8880a801be00
> > > FS:  0000000000000000(0000) GS:ffff8880ae600000(0000) knlGS:0000000000000000
> > > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > CR2: 00000000004c8e48 CR3: 00000000a4c08000 CR4: 00000000001506f0
> > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > > Call Trace:
> > >  pcpu_destroy_chunk mm/percpu-vm.c:366 [inline]
> > >  __pcpu_balance_workfn mm/percpu.c:1982 [inline]
> > >  pcpu_balance_workfn+0x8b3/0x1310 mm/percpu.c:2069
> > >  process_one_work+0x94c/0x1670 kernel/workqueue.c:2269
> > >  worker_thread+0x64c/0x1120 kernel/workqueue.c:2415
> > >  kthread+0x3b5/0x4a0 kernel/kthread.c:292
> > >  ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
> > > Modules linked in:
> > > ---[ end trace 6a2e56ec52e1f480 ]---
> > > RIP: 0010:free_vm_area mm/vmalloc.c:3089 [inline]
> > > RIP: 0010:free_vm_area mm/vmalloc.c:3085 [inline]
> > > RIP: 0010:pcpu_free_vm_areas+0x96/0xc0 mm/vmalloc.c:3432
> > > Code: 75 48 48 8b 2b 48 8d 7d 08 48 89 f8 48 c1 e8 03 42 80 3c 30 00 75 2c 48 8b 7d 08 e8 c4 c8 ff ff 48 39 c5 74 a5 e8 ea c3 c9 ff <0f> 0b e8 e3 c3 c9 ff 4c 89 ff 5b 5d 41 5c 41 5d 41 5e 41 5f e9 71
> > > RSP: 0018:ffffc90000d2fba8 EFLAGS: 00010293
> > > RAX: 0000000000000000 RBX: ffff8880a801be00 RCX: 0000000000000000
> > > RDX: ffff8880a95fa300 RSI: ffffffff81aa7c76 RDI: 0000000000000001
> > > RBP: ffff8880a2b38180 R08: 0000000000000000 R09: ffffffff89cfecc3
> > > R10: fffffbfff139fd98 R11: 0000000000000000 R12: 0000000000000000
> > > R13: 0000000000000001 R14: dffffc0000000000 R15: ffff8880a801be00
> > > FS:  0000000000000000(0000) GS:ffff8880ae600000(0000) knlGS:0000000000000000
> > > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > CR2: 00000000004c8e48 CR3: 00000000a4c08000 CR4: 00000000001506f0
> > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > > 
> > That is because of below revert:
> > 
> > <snip>
> > commit bdbfb1d52d5e576c1d275fd8ab59b677011229e8
> > Author: Ingo Molnar <mingo@kernel.org>
> > Date:   Sun Jun 7 21:12:51 2020 +0200
> > 
> >     Revert "mm/vmalloc: modify struct vmap_area to reduce its size"
> >     
> >     This reverts commit 688fcbfc06e4fdfbb7e1d5a942a1460fe6379d2d.
> >     
> >     Signed-off-by: Ingo Molnar <mingo@kernel.org>
> >     
> >     Conflicts:
> >             mm/vmalloc.c
> > <snip>
> > 
> > I can check further, but it can be it was not correctly reverted,
> > because everything should work just fine even with the revert,
> > though i i do not understand a reason of reverting.
> 
> Vlad, how sure are you about this? We also start to trigger this now on
> linux-next, but the reverting patch surely looks like doggy without any useful
> information in the commit description.
>
Hello, Andrew, Qian.

I am not aware of reason of the revert, though i tried to get through Ingo.
I can send out a patch that fixes the revert. Another option to drop the
revert, but it is up to Andrew and Ingo.

Andrew, could you please comment on?

Thank you in advance!

--
Vlad Rezki

Re: kernel BUG at mm/vmalloc.c:LINE! (2)

[Andrew Morton]

On Wed, 22 Jul 2020 16:46:50 +0200 Uladzislau Rezki <urezki@gmail.com> wrote:

> > > I can check further, but it can be it was not correctly reverted,
> > > because everything should work just fine even with the revert,
> > > though i i do not understand a reason of reverting.
> > 
> > Vlad, how sure are you about this? We also start to trigger this now on
> > linux-next, but the reverting patch surely looks like doggy without any useful
> > information in the commit description.
> >
> Hello, Andrew, Qian.
> 
> I am not aware of reason of the revert, though i tried to get through Ingo.
> I can send out a patch that fixes the revert. Another option to drop the
> revert, but it is up to Andrew and Ingo.
> 
> Andrew, could you please comment on?

All a bit mysterious.  I think it's best that we revert this from
linux-next until we hear from Ingo.  I queued a patch - I expect
Stephen will see and grab it, thanks.

Re: kernel BUG at mm/vmalloc.c:LINE! (2)

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 326 bytes --]

Hi Andrew,

On Thu, 23 Jul 2020 19:50:29 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
> All a bit mysterious.  I think it's best that we revert this from
> linux-next until we hear from Ingo.  I queued a patch - I expect
> Stephen will see and grab it, thanks.

Wiil do.

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: kernel BUG at mm/vmalloc.c:LINE! (2)

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 602 bytes --]

Hi Andrew,

On Thu, 23 Jul 2020 19:50:29 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Wed, 22 Jul 2020 16:46:50 +0200 Uladzislau Rezki <urezki@gmail.com> wrote:
> 
> All a bit mysterious.  I think it's best that we revert this from
> linux-next until we hear from Ingo.  I queued a patch - I expect
> Stephen will see and grab it, thanks.

In the end I actually did the revert (of the revert) in the merge of
the tip tree (so that -next will bisect better if necessary).  So you
will not need the revert in your quilt series after today.

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: kernel BUG at mm/vmalloc.c:LINE! (2)

[Thomas Gleixner]

Stephen Rothwell <sfr@canb.auug.org.au> writes:
>> All a bit mysterious.  I think it's best that we revert this from
>> linux-next until we hear from Ingo.  I queued a patch - I expect
>> Stephen will see and grab it, thanks.
>
> In the end I actually did the revert (of the revert) in the merge of
> the tip tree (so that -next will bisect better if necessary).  So you
> will not need the revert in your quilt series after today.

Sigh. I have no idea why this was in tip auto-latest. I just
reintegrated that branch and the annoyance should be gone now.

Sorry for not paying attention.

Thanks,

        tglx