Linux-RDMA Archive on lore.kernel.org
 help / color / Atom feed
* KASAN: use-after-free Read in cma_cancel_operation, rdma_listen
@ 2021-04-13  3:36 Hao Sun
  2021-04-13 13:33 ` Jason Gunthorpe
  0 siblings, 1 reply; 7+ messages in thread
From: Hao Sun @ 2021-04-13  3:36 UTC (permalink / raw)
  To: dledford, jgg, linux-rdma; +Cc: leon, linux-kernel


[-- Attachment #1: Type: text/plain, Size: 14075 bytes --]

Hi

When using Healer(https://github.com/SunHao-0/healer/tree/dev) to fuzz
the Linux kernel, I found two use-after-free bugs which have been
reported a long time ago by Syzbot.
Although the corresponding patches have been merged into upstream,
these two bugs can still be triggered easily.
The original information about Syzbot report can be found here:
https://syzkaller.appspot.com/bug?id=8dc0bcd9dd6ec915ba10b3354740eb420884acaa
https://syzkaller.appspot.com/bug?id=95f89b8fb9fdc42e28ad586e657fea074e4e719b

Current report Information:
commit:   89698becf06d341a700913c3d89ce2a914af69a2
version:   linux 5.12
git tree:    upstream
kernel config, and full log can be found in the attached file.

use-after-free Read in rdma_listen
==================================================================
BUG: KASAN: use-after-free in __list_add_valid+0x93/0xa0 lib/list_debug.c:26
Read of size 8 at addr ffff88801a69b1e0 by task syz-executor.0/7768

CPU: 0 PID: 7768 Comm:  executor.0 Not tainted 5.12.0-rc7+ #15
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.13.0-1ubuntu1.1 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0xfa/0x151 lib/dump_stack.c:120
 print_address_description.constprop.0.cold+0x82/0x32c mm/kasan/report.c:232
 __kasan_report mm/kasan/report.c:399 [inline]
 kasan_report.cold+0x7c/0xd8 mm/kasan/report.c:416
 __list_add_valid+0x93/0xa0 lib/list_debug.c:26
 __list_add include/linux/list.h:67 [inline]
 list_add_tail include/linux/list.h:100 [inline]
 cma_listen_on_all drivers/infiniband/core/cma.c:2557 [inline]
 rdma_listen+0x6f4/0xd80 drivers/infiniband/core/cma.c:3751
 ucma_listen+0x16a/0x210 drivers/infiniband/core/ucma.c:1102
 ucma_write+0x259/0x350 drivers/infiniband/core/ucma.c:1732
 vfs_write+0x22a/0xa40 fs/read_write.c:603
 ksys_write+0x1ee/0x250 fs/read_write.c:658
 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x469c1d
Code: 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48
89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d
01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f1482460198 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 000000000057bf60 RCX: 0000000000469c1d
RDX: 0000000000000010 RSI: 0000000020000000 RDI: 0000000000000003
RBP: 00000000004d4a17 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000057bf60
R13: 00007ffcf543ebcf R14: 00007ffcf543ed70 R15: 00007f1482460300

Allocated by task 7765:
 kasan_save_stack+0x1b/0x40 mm/kasan/common.c:38
 kasan_set_track mm/kasan/common.c:46 [inline]
 set_alloc_info mm/kasan/common.c:427 [inline]
 ____kasan_kmalloc mm/kasan/common.c:506 [inline]
 __kasan_kmalloc+0x7a/0x90 mm/kasan/common.c:515
 kasan_kmalloc include/linux/kasan.h:233 [inline]
 kmem_cache_alloc_trace+0x19f/0x350 mm/slub.c:2934
 kmalloc include/linux/slab.h:554 [inline]
 kzalloc include/linux/slab.h:684 [inline]
 __rdma_create_id+0x5b/0x550 drivers/infiniband/core/cma.c:844
 rdma_create_user_id+0x79/0xd0 drivers/infiniband/core/cma.c:897
 ucma_create_id+0x162/0x370 drivers/infiniband/core/ucma.c:461
 ucma_write+0x259/0x350 drivers/infiniband/core/ucma.c:1732
 vfs_write+0x22a/0xa40 fs/read_write.c:603
 ksys_write+0x1ee/0x250 fs/read_write.c:658
 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xae

Freed by task 7766:
 kasan_save_stack+0x1b/0x40 mm/kasan/common.c:38
 kasan_set_track+0x1c/0x30 mm/kasan/common.c:46
 kasan_set_free_info+0x20/0x30 mm/kasan/generic.c:357
 ____kasan_slab_free mm/kasan/common.c:360 [inline]
 ____kasan_slab_free mm/kasan/common.c:325 [inline]
 __kasan_slab_free+0xdb/0x110 mm/kasan/common.c:367
 kasan_slab_free include/linux/kasan.h:199 [inline]
 slab_free_hook mm/slub.c:1562 [inline]
 slab_free_freelist_hook mm/slub.c:1600 [inline]
 slab_free mm/slub.c:3161 [inline]
 kfree+0xf8/0x430 mm/slub.c:4213
 ucma_close_id+0x4c/0x90 drivers/infiniband/core/ucma.c:185
 ucma_destroy_private_ctx+0x887/0xb00 drivers/infiniband/core/ucma.c:576
 ucma_close+0x10a/0x180 drivers/infiniband/core/ucma.c:1797
 __fput+0x288/0x920 fs/file_table.c:280
 task_work_run+0xe0/0x1a0 kernel/task_work.c:140
 exit_task_work include/linux/task_work.h:30 [inline]
 do_exit+0xbf9/0x2e10 kernel/exit.c:825
 do_group_exit+0x125/0x340 kernel/exit.c:922
 get_signal+0x4d5/0x2570 kernel/signal.c:2781
 arch_do_signal_or_restart+0x2e2/0x1e50 arch/x86/kernel/signal.c:789
 handle_signal_work kernel/entry/common.c:147 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:171 [inline]
 exit_to_user_mode_prepare+0x161/0x270 kernel/entry/common.c:208
 __syscall_exit_to_user_mode_work kernel/entry/common.c:290 [inline]
 syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:301
 entry_SYSCALL_64_after_hwframe+0x44/0xae

The buggy address belongs to the object at ffff88801a69b000
 which belongs to the cache kmalloc-2k of size 2048
The buggy address is located 480 bytes inside of
 2048-byte region [ffff88801a69b000, ffff88801a69b800)
The buggy address belongs to the page:
page:00000000204c7c36 refcount:1 mapcount:0 mapping:0000000000000000
index:0x0 pfn:0x1a698
head:00000000204c7c36 order:3 compound_mapcount:0 compound_pincount:0
flags: 0xfff00000010200(slab|head)
raw: 00fff00000010200 dead000000000100 dead000000000122 ffff88800fc42000
raw: 0000000000000000 0000000080080008 00000001ffffffff 0000000000000000
page dumped because: kasan: bad access detected

Memory state around the buggy address:
 ffff88801a69b080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff88801a69b100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff88801a69b180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                                                       ^
 ffff88801a69b200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff88801a69b280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb

The following reproduction program(Syzlang format) can trigger above crash:
# {Threaded:false Collide:false Repeat:true RepeatTimes:0 Procs:1
Slowdown:1 Sandbox:none Fault:false FaultCall:-1 FaultNth:0 Leak:false
NetInjection:true NetDevices:true NetReset:true Cgroups:true
BinfmtMisc:true CloseFDs:true KCSAN:false DevlinkPCI:true USB:true
VhciInjection:true Wifi:true IEEE802154:true Sysctl:true
UseTmpDir:true HandleSegv:true Repro:false Trace:false}

r0 = openat$rdma_cm(0xffffffffffffff9c,
&(0x7f0000000040)='/dev/infiniband/rdma_cm\x00', 0x2, 0x0)
write$RDMA_USER_CM_CMD_CREATE_ID(r0, &(0x7f0000000300)={0x0, 0x18,
0xfa00, {0x1, &(0x7f00000002c0)={<r1=>0x0}, 0x2, 0x6}}, 0x20)
write$RDMA_USER_CM_CMD_RESOLVE_IP(r0, &(0x7f0000000380)={0x3, 0x40,
0xfa00, {{0xa, 0x4e22, 0xaa6b1d92a66f187b, @loopback,
0x815d88844dd0205c}, {0xa, 0x4e24, 0xf101e177c7e2a46b, @private2,
0xf948b998a4649c8a}, r1, 0xab244bf96747ef83}}, 0x48)
write$RDMA_USER_CM_CMD_LISTEN(r0, &(0x7f0000000000)={0x7, 0x8, 0xfa00,
{r1, 0x97bf0a2859a6b2e3}}, 0x10)
write$RDMA_USER_CM_CMD_LISTEN(r0, &(0x7f00000001c0)={0x7, 0x8, 0xfa00,
{r1, 0x1}}, 0x10)


use-after-free Read in cma_cancel_operation
==================================================================
BUG: KASAN: use-after-free in __list_del_entry_valid+0xe0/0xf0
lib/list_debug.c:51
Read of size 8 at addr ffff88801da7d1e0 by task syz-executor.1/11033

CPU: 0 PID: 11033 Comm: syz-executor.1 Not tainted 5.12.0-rc7+ #15
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.13.0-1ubuntu1.1 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0xfa/0x151 lib/dump_stack.c:120
 print_address_description.constprop.0.cold+0x82/0x32c mm/kasan/report.c:232
 __kasan_report mm/kasan/report.c:399 [inline]
 kasan_report.cold+0x7c/0xd8 mm/kasan/report.c:416
 __list_del_entry_valid+0xe0/0xf0 lib/list_debug.c:51
 __list_del_entry include/linux/list.h:132 [inline]
 list_del include/linux/list.h:146 [inline]
 cma_cancel_listens drivers/infiniband/core/cma.c:1758 [inline]
 cma_cancel_operation drivers/infiniband/core/cma.c:1786 [inline]
 cma_cancel_operation+0x2bf/0xa00 drivers/infiniband/core/cma.c:1774
 _destroy_id+0x24/0x870 drivers/infiniband/core/cma.c:1853
 ucma_close_id+0x4c/0x90 drivers/infiniband/core/ucma.c:185
 ucma_destroy_private_ctx+0x887/0xb00 drivers/infiniband/core/ucma.c:576
 ucma_close+0x10a/0x180 drivers/infiniband/core/ucma.c:1797
 __fput+0x288/0x920 fs/file_table.c:280
 task_work_run+0xe0/0x1a0 kernel/task_work.c:140
 exit_task_work include/linux/task_work.h:30 [inline]
 do_exit+0xbf9/0x2e10 kernel/exit.c:825
 do_group_exit+0x125/0x340 kernel/exit.c:922
 get_signal+0x4d5/0x2570 kernel/signal.c:2781
 arch_do_signal_or_restart+0x2e2/0x1e50 arch/x86/kernel/signal.c:789
 handle_signal_work kernel/entry/common.c:147 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:171 [inline]
 exit_to_user_mode_prepare+0x161/0x270 kernel/entry/common.c:208
 __syscall_exit_to_user_mode_work kernel/entry/common.c:290 [inline]
 syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:301
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x469c1d
Code: Unable to access opcode bytes at RIP 0x469bf3.
RSP: 002b:00007f1847648218 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: 0000000000000001 RBX: 000000000057c008 RCX: 0000000000469c1d
RDX: 00000000000f4240 RSI: 0000000000000081 RDI: 000000000057c014
RBP: 000000000057c010 R08: 0000000000000016 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 000000000057c014
R13: 00007ffe31f70c4f R14: 00007ffe31f70df0 R15: 00007f1847648300

Allocated by task 11019:
 kasan_save_stack+0x1b/0x40 mm/kasan/common.c:38
 kasan_set_track mm/kasan/common.c:46 [inline]
 set_alloc_info mm/kasan/common.c:427 [inline]
 ____kasan_kmalloc mm/kasan/common.c:506 [inline]
 __kasan_kmalloc+0x7a/0x90 mm/kasan/common.c:515
 kasan_kmalloc include/linux/kasan.h:233 [inline]
 kmem_cache_alloc_trace+0x19f/0x350 mm/slub.c:2934
 kmalloc include/linux/slab.h:554 [inline]
 kzalloc include/linux/slab.h:684 [inline]
 __rdma_create_id+0x5b/0x550 drivers/infiniband/core/cma.c:844
 rdma_create_user_id+0x79/0xd0 drivers/infiniband/core/cma.c:897
 ucma_create_id+0x162/0x370 drivers/infiniband/core/ucma.c:461
 ucma_write+0x259/0x350 drivers/infiniband/core/ucma.c:1732
 vfs_write+0x22a/0xa40 fs/read_write.c:603
 ksys_write+0x1ee/0x250 fs/read_write.c:658
 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xae

Freed by task 11027:
 kasan_save_stack+0x1b/0x40 mm/kasan/common.c:38
 kasan_set_track+0x1c/0x30 mm/kasan/common.c:46
 kasan_set_free_info+0x20/0x30 mm/kasan/generic.c:357
 ____kasan_slab_free mm/kasan/common.c:360 [inline]
 ____kasan_slab_free mm/kasan/common.c:325 [inline]
 __kasan_slab_free+0xdb/0x110 mm/kasan/common.c:367
 kasan_slab_free include/linux/kasan.h:199 [inline]
 slab_free_hook mm/slub.c:1562 [inline]
 slab_free_freelist_hook mm/slub.c:1600 [inline]
 slab_free mm/slub.c:3161 [inline]
 kfree+0xf8/0x430 mm/slub.c:4213
 ucma_close_id+0x4c/0x90 drivers/infiniband/core/ucma.c:185
 ucma_destroy_private_ctx+0x887/0xb00 drivers/infiniband/core/ucma.c:576
 ucma_close+0x10a/0x180 drivers/infiniband/core/ucma.c:1797
 __fput+0x288/0x920 fs/file_table.c:280
 task_work_run+0xe0/0x1a0 kernel/task_work.c:140
 exit_task_work include/linux/task_work.h:30 [inline]
 do_exit+0xbf9/0x2e10 kernel/exit.c:825
 do_group_exit+0x125/0x340 kernel/exit.c:922
 get_signal+0x4d5/0x2570 kernel/signal.c:2781
 arch_do_signal_or_restart+0x2e2/0x1e50 arch/x86/kernel/signal.c:789
 handle_signal_work kernel/entry/common.c:147 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:171 [inline]
 exit_to_user_mode_prepare+0x161/0x270 kernel/entry/common.c:208
 __syscall_exit_to_user_mode_work kernel/entry/common.c:290 [inline]
 syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:301
 ret_from_fork+0x15/0x30 arch/x86/entry/entry_64.S:287

The buggy address belongs to the object at ffff88801da7d000
 which belongs to the cache kmalloc-2k of size 2048
The buggy address is located 480 bytes inside of
 2048-byte region [ffff88801da7d000, ffff88801da7d800)
The buggy address belongs to the page:
page:00000000d8c23c50 refcount:1 mapcount:0 mapping:0000000000000000
index:0x0 pfn:0x1da78
head:00000000d8c23c50 order:3 compound_mapcount:0 compound_pincount:0
flags: 0xfff00000010200(slab|head)
raw: 00fff00000010200 dead000000000100 dead000000000122 ffff88800fc42000
raw: 0000000000000000 0000000000080008 00000001ffffffff 0000000000000000
page dumped because: kasan: bad access detected

Memory state around the buggy address:
 ffff88801da7d080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff88801da7d100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff88801da7d180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                                                       ^
 ffff88801da7d200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff88801da7d280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb

The following reproduction program(Syzlang format) can trigger above crash:
# {Threaded:true Collide:false Repeat:true RepeatTimes:0 Procs:2
Slowdown:1 Sandbox:none Fault:false FaultCall:-1 FaultNth:0 Leak:false
NetInjection:true NetDevices:true NetReset:true Cgroups:true
BinfmtMisc:true CloseFDs:true KCSAN:false DevlinkPCI:true USB:true
VhciInjection:true Wifi:true IEEE802154:true Sysctl:true
UseTmpDir:true HandleSegv:true Repro:false Trace:false}

r0 = openat$rdma_cm(0xffffffffffffff9c,
&(0x7f0000000040)='/dev/infiniband/rdma_cm\x00', 0x2, 0x0)
write$RDMA_USER_CM_CMD_CREATE_ID(r0, &(0x7f0000000300)={0x0, 0x18,
0xfa00, {0x2, &(0x7f00000002c0)={<r1=>0x0}, 0x2, 0x4}}, 0x20)
write$RDMA_USER_CM_CMD_RESOLVE_IP(r0, &(0x7f0000000380)={0x3, 0x40,
0xfa00, {{0xa, 0x4e22, 0xcaa4d8d2aba4fcd8, @loopback, 0x1}, {0xa,
0x4e20, 0x0, @private2, 0x1}, r1, 0x78fe24d7c8b32e4c}}, 0x48)
write$RDMA_USER_CM_CMD_BIND_IP(r0, &(0x7f0000000000)={0x2, 0x28,
0xfa00, {0x0, {0xa, 0x4e21, 0xbb6fb906443e717, @mcast2,
0xb2c44fba8234f958}, r1}}, 0x30)
write$RDMA_USER_CM_CMD_LISTEN(r0, &(0x7f0000000000)={0x7, 0x8, 0xfa00,
{r1, 0x96677ad888e911ea}}, 0x10)

[-- Attachment #2: log2 --]
[-- Type: application/octet-stream, Size: 10812 bytes --]

[-- Attachment #3: log1 --]
[-- Type: application/octet-stream, Size: 9542 bytes --]

[-- Attachment #4: config --]
[-- Type: application/octet-stream, Size: 221433 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: KASAN: use-after-free Read in cma_cancel_operation, rdma_listen
  2021-04-13  3:36 KASAN: use-after-free Read in cma_cancel_operation, rdma_listen Hao Sun
@ 2021-04-13 13:33 ` Jason Gunthorpe
  2021-04-13 13:42   ` Hao Sun
  0 siblings, 1 reply; 7+ messages in thread
From: Jason Gunthorpe @ 2021-04-13 13:33 UTC (permalink / raw)
  To: Hao Sun; +Cc: dledford, linux-rdma, leon, linux-kernel

On Tue, Apr 13, 2021 at 11:36:41AM +0800, Hao Sun wrote:
> Hi
> 
> When using Healer(https://github.com/SunHao-0/healer/tree/dev) to fuzz
> the Linux kernel, I found two use-after-free bugs which have been
> reported a long time ago by Syzbot.
> Although the corresponding patches have been merged into upstream,
> these two bugs can still be triggered easily.
> The original information about Syzbot report can be found here:
> https://syzkaller.appspot.com/bug?id=8dc0bcd9dd6ec915ba10b3354740eb420884acaa
> https://syzkaller.appspot.com/bug?id=95f89b8fb9fdc42e28ad586e657fea074e4e719b

Then why hasn't syzbot seen this in a year's time? Seems strange
 
> Current report Information:
> commit:   89698becf06d341a700913c3d89ce2a914af69a2

What commit is this?

Jason

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: KASAN: use-after-free Read in cma_cancel_operation, rdma_listen
  2021-04-13 13:33 ` Jason Gunthorpe
@ 2021-04-13 13:42   ` Hao Sun
  2021-04-13 13:44     ` Jason Gunthorpe
  0 siblings, 1 reply; 7+ messages in thread
From: Hao Sun @ 2021-04-13 13:42 UTC (permalink / raw)
  To: Jason Gunthorpe; +Cc: dledford, linux-rdma, leon, linux-kernel

Jason Gunthorpe <jgg@ziepe.ca> 于2021年4月13日周二 下午9:34写道:
>
> On Tue, Apr 13, 2021 at 11:36:41AM +0800, Hao Sun wrote:
> > Hi
> >
> > When using Healer(https://github.com/SunHao-0/healer/tree/dev) to fuzz
> > the Linux kernel, I found two use-after-free bugs which have been
> > reported a long time ago by Syzbot.
> > Although the corresponding patches have been merged into upstream,
> > these two bugs can still be triggered easily.
> > The original information about Syzbot report can be found here:
> > https://syzkaller.appspot.com/bug?id=8dc0bcd9dd6ec915ba10b3354740eb420884acaa
> > https://syzkaller.appspot.com/bug?id=95f89b8fb9fdc42e28ad586e657fea074e4e719b
>
> Then why hasn't syzbot seen this in a year's time? Seems strange
>

Seems strange to me too, but the fact is that the reproduction program
in attachment can trigger these two bugs quickly.

> > Current report Information:
> > commit:   89698becf06d341a700913c3d89ce2a914af69a2
>
> What commit is this?

The latest commit in the master branch,
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: KASAN: use-after-free Read in cma_cancel_operation, rdma_listen
  2021-04-13 13:42   ` Hao Sun
@ 2021-04-13 13:44     ` Jason Gunthorpe
  2021-04-13 14:09       ` Haakon Bugge
  2021-04-13 14:19       ` Hao Sun
  0 siblings, 2 replies; 7+ messages in thread
From: Jason Gunthorpe @ 2021-04-13 13:44 UTC (permalink / raw)
  To: Hao Sun; +Cc: dledford, linux-rdma, leon, linux-kernel

On Tue, Apr 13, 2021 at 09:42:43PM +0800, Hao Sun wrote:
> Jason Gunthorpe <jgg@ziepe.ca> 于2021年4月13日周二 下午9:34写道:
> >
> > On Tue, Apr 13, 2021 at 11:36:41AM +0800, Hao Sun wrote:
> > > Hi
> > >
> > > When using Healer(https://github.com/SunHao-0/healer/tree/dev) to fuzz
> > > the Linux kernel, I found two use-after-free bugs which have been
> > > reported a long time ago by Syzbot.
> > > Although the corresponding patches have been merged into upstream,
> > > these two bugs can still be triggered easily.
> > > The original information about Syzbot report can be found here:
> > > https://syzkaller.appspot.com/bug?id=8dc0bcd9dd6ec915ba10b3354740eb420884acaa
> > > https://syzkaller.appspot.com/bug?id=95f89b8fb9fdc42e28ad586e657fea074e4e719b
> >
> > Then why hasn't syzbot seen this in a year's time? Seems strange
> >
> 
> Seems strange to me too, but the fact is that the reproduction program
> in attachment can trigger these two bugs quickly.

Do you have this in the C format?

Jason

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: KASAN: use-after-free Read in cma_cancel_operation, rdma_listen
  2021-04-13 13:44     ` Jason Gunthorpe
@ 2021-04-13 14:09       ` Haakon Bugge
  2021-04-13 14:19       ` Hao Sun
  1 sibling, 0 replies; 7+ messages in thread
From: Haakon Bugge @ 2021-04-13 14:09 UTC (permalink / raw)
  To: Jason Gunthorpe; +Cc: Hao Sun, dledford, OFED mailing list, leon, linux-kernel



> On 13 Apr 2021, at 15:44, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> 
> On Tue, Apr 13, 2021 at 09:42:43PM +0800, Hao Sun wrote:
>> Jason Gunthorpe <jgg@ziepe.ca> 于2021年4月13日周二 下午9:34写道:
>>> 
>>> On Tue, Apr 13, 2021 at 11:36:41AM +0800, Hao Sun wrote:
>>>> Hi
>>>> 
>>>> When using Healer(https://github.com/SunHao-0/healer/tree/dev) to fuzz
>>>> the Linux kernel, I found two use-after-free bugs which have been
>>>> reported a long time ago by Syzbot.
>>>> Although the corresponding patches have been merged into upstream,
>>>> these two bugs can still be triggered easily.
>>>> The original information about Syzbot report can be found here:
>>>> https://syzkaller.appspot.com/bug?id=8dc0bcd9dd6ec915ba10b3354740eb420884acaa
>>>> https://syzkaller.appspot.com/bug?id=95f89b8fb9fdc42e28ad586e657fea074e4e719b
>>> 
>>> Then why hasn't syzbot seen this in a year's time? Seems strange
>>> 
>> 
>> Seems strange to me too, but the fact is that the reproduction program
>> in attachment can trigger these two bugs quickly.
> 
> Do you have this in the C format?

How can a commit changing arch m68k provoke a bug when running on x86_64?


Håkon

> 
> Jason


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: KASAN: use-after-free Read in cma_cancel_operation, rdma_listen
  2021-04-13 13:44     ` Jason Gunthorpe
  2021-04-13 14:09       ` Haakon Bugge
@ 2021-04-13 14:19       ` Hao Sun
  2021-04-19 17:50         ` Jason Gunthorpe
  1 sibling, 1 reply; 7+ messages in thread
From: Hao Sun @ 2021-04-13 14:19 UTC (permalink / raw)
  To: Jason Gunthorpe; +Cc: dledford, linux-rdma, leon, linux-kernel


[-- Attachment #1: Type: text/plain, Size: 1913 bytes --]

Jason Gunthorpe <jgg@ziepe.ca> 于2021年4月13日周二 下午9:45写道:
>
> On Tue, Apr 13, 2021 at 09:42:43PM +0800, Hao Sun wrote:
> > Jason Gunthorpe <jgg@ziepe.ca> 于2021年4月13日周二 下午9:34写道:
> > >
> > > On Tue, Apr 13, 2021 at 11:36:41AM +0800, Hao Sun wrote:
> > > > Hi
> > > >
> > > > When using Healer(https://github.com/SunHao-0/healer/tree/dev) to fuzz
> > > > the Linux kernel, I found two use-after-free bugs which have been
> > > > reported a long time ago by Syzbot.
> > > > Although the corresponding patches have been merged into upstream,
> > > > these two bugs can still be triggered easily.
> > > > The original information about Syzbot report can be found here:
> > > > https://syzkaller.appspot.com/bug?id=8dc0bcd9dd6ec915ba10b3354740eb420884acaa
> > > > https://syzkaller.appspot.com/bug?id=95f89b8fb9fdc42e28ad586e657fea074e4e719b
> > >
> > > Then why hasn't syzbot seen this in a year's time? Seems strange
> > >
> >
> > Seems strange to me too, but the fact is that the reproduction program
> > in attachment can trigger these two bugs quickly.
>
> Do you have this in the C format?
>

Just tried to use syz-prog2c to convert the repro-prog to C format.
The repro program of  rdma_listen was successfully reproduced
(uploaded in attachment), the other one failed. it looks like
syz-prog2c may not be able to do the equivalent conversion.
You can use syz-execprog to execute the reprogram directly, this
method can reproduce both crashes, I have tried it.

For reproduction prog of  cma_cancel_operation,  using syz-execprog
can run this reproduction program directly:
 ./syz-execprog  -threaded -repeat 0 -procs 2 -slowdown 1 -enable tun
-enable netdev -enable resetnet -enable cgroups -enable binfmt-misc
-enable close_fds -enable devlinkpci -enable usb -enable vhci -enable
wifi -enable ieee802154 -enable sysctl repro.prog

[-- Attachment #2: repro_rdma.c --]
[-- Type: text/plain, Size: 9921 bytes --]

// autogenerated by syzkaller (https://github.com/google/syzkaller)

#define _GNU_SOURCE

#include <dirent.h>
#include <endian.h>
#include <errno.h>
#include <fcntl.h>
#include <setjmp.h>
#include <signal.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mount.h>
#include <sys/prctl.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>

static __thread int skip_segv;
static __thread jmp_buf segv_env;

static void segv_handler(int sig, siginfo_t* info, void* ctx)
{
  uintptr_t addr = (uintptr_t)info->si_addr;
  const uintptr_t prog_start = 1 << 20;
  const uintptr_t prog_end = 100 << 20;
  int skip = __atomic_load_n(&skip_segv, __ATOMIC_RELAXED) != 0;
  int valid = addr < prog_start || addr > prog_end;
  if (skip && valid) {
    _longjmp(segv_env, 1);
  }
  exit(sig);
}

static void install_segv_handler(void)
{
  struct sigaction sa;
  memset(&sa, 0, sizeof(sa));
  sa.sa_handler = SIG_IGN;
  syscall(SYS_rt_sigaction, 0x20, &sa, NULL, 8);
  syscall(SYS_rt_sigaction, 0x21, &sa, NULL, 8);
  memset(&sa, 0, sizeof(sa));
  sa.sa_sigaction = segv_handler;
  sa.sa_flags = SA_NODEFER | SA_SIGINFO;
  sigaction(SIGSEGV, &sa, NULL);
  sigaction(SIGBUS, &sa, NULL);
}

#define NONFAILING(...)                                                        \
  ({                                                                           \
    int ok = 1;                                                                \
    __atomic_fetch_add(&skip_segv, 1, __ATOMIC_SEQ_CST);                       \
    if (_setjmp(segv_env) == 0) {                                              \
      __VA_ARGS__;                                                             \
    } else                                                                     \
      ok = 0;                                                                  \
    __atomic_fetch_sub(&skip_segv, 1, __ATOMIC_SEQ_CST);                       \
    ok;                                                                        \
  })

static void sleep_ms(uint64_t ms)
{
  usleep(ms * 1000);
}

static uint64_t current_time_ms(void)
{
  struct timespec ts;
  if (clock_gettime(CLOCK_MONOTONIC, &ts))
    exit(1);
  return (uint64_t)ts.tv_sec * 1000 + (uint64_t)ts.tv_nsec / 1000000;
}

static void use_temporary_dir(void)
{
  char tmpdir_template[] = "./syzkaller.XXXXXX";
  char* tmpdir = mkdtemp(tmpdir_template);
  if (!tmpdir)
    exit(1);
  if (chmod(tmpdir, 0777))
    exit(1);
  if (chdir(tmpdir))
    exit(1);
}

static bool write_file(const char* file, const char* what, ...)
{
  char buf[1024];
  va_list args;
  va_start(args, what);
  vsnprintf(buf, sizeof(buf), what, args);
  va_end(args);
  buf[sizeof(buf) - 1] = 0;
  int len = strlen(buf);
  int fd = open(file, O_WRONLY | O_CLOEXEC);
  if (fd == -1)
    return false;
  if (write(fd, buf, len) != len) {
    int err = errno;
    close(fd);
    errno = err;
    return false;
  }
  close(fd);
  return true;
}

#define FS_IOC_SETFLAGS _IOW('f', 2, long)
static void remove_dir(const char* dir)
{
  int iter = 0;
  DIR* dp = 0;
retry:
  while (umount2(dir, MNT_DETACH) == 0) {
  }
  dp = opendir(dir);
  if (dp == NULL) {
    if (errno == EMFILE) {
      exit(1);
    }
    exit(1);
  }
  struct dirent* ep = 0;
  while ((ep = readdir(dp))) {
    if (strcmp(ep->d_name, ".") == 0 || strcmp(ep->d_name, "..") == 0)
      continue;
    char filename[FILENAME_MAX];
    snprintf(filename, sizeof(filename), "%s/%s", dir, ep->d_name);
    while (umount2(filename, MNT_DETACH) == 0) {
    }
    struct stat st;
    if (lstat(filename, &st))
      exit(1);
    if (S_ISDIR(st.st_mode)) {
      remove_dir(filename);
      continue;
    }
    int i;
    for (i = 0;; i++) {
      if (unlink(filename) == 0)
        break;
      if (errno == EPERM) {
        int fd = open(filename, O_RDONLY);
        if (fd != -1) {
          long flags = 0;
          if (ioctl(fd, FS_IOC_SETFLAGS, &flags) == 0) {
          }
          close(fd);
          continue;
        }
      }
      if (errno == EROFS) {
        break;
      }
      if (errno != EBUSY || i > 100)
        exit(1);
      if (umount2(filename, MNT_DETACH))
        exit(1);
    }
  }
  closedir(dp);
  for (int i = 0;; i++) {
    if (rmdir(dir) == 0)
      break;
    if (i < 100) {
      if (errno == EPERM) {
        int fd = open(dir, O_RDONLY);
        if (fd != -1) {
          long flags = 0;
          if (ioctl(fd, FS_IOC_SETFLAGS, &flags) == 0) {
          }
          close(fd);
          continue;
        }
      }
      if (errno == EROFS) {
        break;
      }
      if (errno == EBUSY) {
        if (umount2(dir, MNT_DETACH))
          exit(1);
        continue;
      }
      if (errno == ENOTEMPTY) {
        if (iter < 100) {
          iter++;
          goto retry;
        }
      }
    }
    exit(1);
  }
}

static void kill_and_wait(int pid, int* status)
{
  kill(-pid, SIGKILL);
  kill(pid, SIGKILL);
  for (int i = 0; i < 100; i++) {
    if (waitpid(-1, status, WNOHANG | __WALL) == pid)
      return;
    usleep(1000);
  }
  DIR* dir = opendir("/sys/fs/fuse/connections");
  if (dir) {
    for (;;) {
      struct dirent* ent = readdir(dir);
      if (!ent)
        break;
      if (strcmp(ent->d_name, ".") == 0 || strcmp(ent->d_name, "..") == 0)
        continue;
      char abort[300];
      snprintf(abort, sizeof(abort), "/sys/fs/fuse/connections/%s/abort",
               ent->d_name);
      int fd = open(abort, O_WRONLY);
      if (fd == -1) {
        continue;
      }
      if (write(fd, abort, 1) < 0) {
      }
      close(fd);
    }
    closedir(dir);
  } else {
  }
  while (waitpid(-1, status, __WALL) != pid) {
  }
}

static void setup_test()
{
  prctl(PR_SET_PDEATHSIG, SIGKILL, 0, 0, 0);
  setpgrp();
  write_file("/proc/self/oom_score_adj", "1000");
}

static void execute_one(void);

#define WAIT_FLAGS __WALL

static void loop(void)
{
  int iter = 0;
  for (;; iter++) {
    char cwdbuf[32];
    sprintf(cwdbuf, "./%d", iter);
    if (mkdir(cwdbuf, 0777))
      exit(1);
    int pid = fork();
    if (pid < 0)
      exit(1);
    if (pid == 0) {
      if (chdir(cwdbuf))
        exit(1);
      setup_test();
      execute_one();
      exit(0);
    }
    int status = 0;
    uint64_t start = current_time_ms();
    for (;;) {
      if (waitpid(-1, &status, WNOHANG | WAIT_FLAGS) == pid)
        break;
      sleep_ms(1);
      if (current_time_ms() - start < 5000) {
        continue;
      }
      kill_and_wait(pid, &status);
      break;
    }
    remove_dir(cwdbuf);
  }
}

uint64_t r[2] = {0xffffffffffffffff, 0xffffffffffffffff};

void execute_one(void)
{
  intptr_t res = 0;
  NONFAILING(memcpy((void*)0x20000040, "/dev/infiniband/rdma_cm\000", 24));
  res = syscall(__NR_openat, 0xffffffffffffff9cul, 0x20000040ul, 2ul, 0ul);
  if (res != -1)
    r[0] = res;
  NONFAILING(*(uint32_t*)0x20000300 = 0);
  NONFAILING(*(uint16_t*)0x20000304 = 0x18);
  NONFAILING(*(uint16_t*)0x20000306 = 0xfa00);
  NONFAILING(*(uint64_t*)0x20000308 = 1);
  NONFAILING(*(uint64_t*)0x20000310 = 0x200002c0);
  NONFAILING(*(uint16_t*)0x20000318 = 2);
  NONFAILING(*(uint8_t*)0x2000031a = 6);
  NONFAILING(*(uint8_t*)0x2000031b = 0);
  NONFAILING(*(uint8_t*)0x2000031c = 0);
  NONFAILING(*(uint8_t*)0x2000031d = 0);
  NONFAILING(*(uint8_t*)0x2000031e = 0);
  NONFAILING(*(uint8_t*)0x2000031f = 0);
  res = syscall(__NR_write, r[0], 0x20000300ul, 0x20ul);
  if (res != -1)
    NONFAILING(r[1] = *(uint32_t*)0x200002c0);
  NONFAILING(*(uint32_t*)0x20000380 = 3);
  NONFAILING(*(uint16_t*)0x20000384 = 0x40);
  NONFAILING(*(uint16_t*)0x20000386 = 0xfa00);
  NONFAILING(*(uint16_t*)0x20000388 = 0xa);
  NONFAILING(*(uint16_t*)0x2000038a = htobe16(0x4e22));
  NONFAILING(*(uint32_t*)0x2000038c = htobe32(0xa66f187b));
  NONFAILING(*(uint64_t*)0x20000390 = htobe64(0));
  NONFAILING(*(uint64_t*)0x20000398 = htobe64(1));
  NONFAILING(*(uint32_t*)0x200003a0 = 0x4dd0205c);
  NONFAILING(*(uint16_t*)0x200003a4 = 0xa);
  NONFAILING(*(uint16_t*)0x200003a6 = htobe16(0x4e24));
  NONFAILING(*(uint32_t*)0x200003a8 = htobe32(0xc7e2a46b));
  NONFAILING(*(uint8_t*)0x200003ac = 0xfc);
  NONFAILING(*(uint8_t*)0x200003ad = 2);
  NONFAILING(*(uint8_t*)0x200003ae = 0);
  NONFAILING(*(uint8_t*)0x200003af = 0);
  NONFAILING(*(uint8_t*)0x200003b0 = 0);
  NONFAILING(*(uint8_t*)0x200003b1 = 0);
  NONFAILING(*(uint8_t*)0x200003b2 = 0);
  NONFAILING(*(uint8_t*)0x200003b3 = 0);
  NONFAILING(*(uint8_t*)0x200003b4 = 0);
  NONFAILING(*(uint8_t*)0x200003b5 = 0);
  NONFAILING(*(uint8_t*)0x200003b6 = 0);
  NONFAILING(*(uint8_t*)0x200003b7 = 0);
  NONFAILING(*(uint8_t*)0x200003b8 = 0);
  NONFAILING(*(uint8_t*)0x200003b9 = 0);
  NONFAILING(*(uint8_t*)0x200003ba = 0);
  NONFAILING(*(uint8_t*)0x200003bb = 0);
  NONFAILING(*(uint32_t*)0x200003bc = 0xa4649c8a);
  NONFAILING(*(uint32_t*)0x200003c0 = r[1]);
  NONFAILING(*(uint32_t*)0x200003c4 = 0x6747ef83);
  syscall(__NR_write, r[0], 0x20000380ul, 0x48ul);
  NONFAILING(*(uint32_t*)0x20000000 = 7);
  NONFAILING(*(uint16_t*)0x20000004 = 8);
  NONFAILING(*(uint16_t*)0x20000006 = 0xfa00);
  NONFAILING(*(uint32_t*)0x20000008 = r[1]);
  NONFAILING(*(uint32_t*)0x2000000c = 0x59a6b2e3);
  syscall(__NR_write, r[0], 0x20000000ul, 0x10ul);
  NONFAILING(*(uint32_t*)0x200001c0 = 7);
  NONFAILING(*(uint16_t*)0x200001c4 = 8);
  NONFAILING(*(uint16_t*)0x200001c6 = 0xfa00);
  NONFAILING(*(uint32_t*)0x200001c8 = r[1]);
  NONFAILING(*(uint32_t*)0x200001cc = 1);
  syscall(__NR_write, r[0], 0x200001c0ul, 0x10ul);
}
int main(void)
{
  syscall(__NR_mmap, 0x1ffff000ul, 0x1000ul, 0ul, 0x32ul, -1, 0ul);
  syscall(__NR_mmap, 0x20000000ul, 0x1000000ul, 7ul, 0x32ul, -1, 0ul);
  syscall(__NR_mmap, 0x21000000ul, 0x1000ul, 0ul, 0x32ul, -1, 0ul);
  install_segv_handler();
  use_temporary_dir();
  loop();
  return 0;
}

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: KASAN: use-after-free Read in cma_cancel_operation, rdma_listen
  2021-04-13 14:19       ` Hao Sun
@ 2021-04-19 17:50         ` Jason Gunthorpe
  0 siblings, 0 replies; 7+ messages in thread
From: Jason Gunthorpe @ 2021-04-19 17:50 UTC (permalink / raw)
  To: Hao Sun; +Cc: dledford, linux-rdma, leon, linux-kernel

On Tue, Apr 13, 2021 at 10:19:25PM +0800, Hao Sun wrote:
> Jason Gunthorpe <jgg@ziepe.ca> 于2021年4月13日周二 下午9:45写道:
> >
> > On Tue, Apr 13, 2021 at 09:42:43PM +0800, Hao Sun wrote:
> > > Jason Gunthorpe <jgg@ziepe.ca> 于2021年4月13日周二 下午9:34写道:
> > > >
> > > > On Tue, Apr 13, 2021 at 11:36:41AM +0800, Hao Sun wrote:
> > > > > Hi
> > > > >
> > > > > When using Healer(https://github.com/SunHao-0/healer/tree/dev) to fuzz
> > > > > the Linux kernel, I found two use-after-free bugs which have been
> > > > > reported a long time ago by Syzbot.
> > > > > Although the corresponding patches have been merged into upstream,
> > > > > these two bugs can still be triggered easily.
> > > > > The original information about Syzbot report can be found here:
> > > > > https://syzkaller.appspot.com/bug?id=8dc0bcd9dd6ec915ba10b3354740eb420884acaa
> > > > > https://syzkaller.appspot.com/bug?id=95f89b8fb9fdc42e28ad586e657fea074e4e719b
> > > >
> > > > Then why hasn't syzbot seen this in a year's time? Seems strange
> > > >
> > >
> > > Seems strange to me too, but the fact is that the reproduction program
> > > in attachment can trigger these two bugs quickly.
> >
> > Do you have this in the C format?
> >
> 
> Just tried to use syz-prog2c to convert the repro-prog to C format.
> The repro program of  rdma_listen was successfully reproduced
> (uploaded in attachment), the other one failed. it looks like
> syz-prog2c may not be able to do the equivalent conversion.
> You can use syz-execprog to execute the reprogram directly, this
> method can reproduce both crashes, I have tried it.

I tried this program and it reliably deadlocks my kernel :|

So there is certainly something here

Jason

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-13  3:36 KASAN: use-after-free Read in cma_cancel_operation, rdma_listen Hao Sun
2021-04-13 13:33 ` Jason Gunthorpe
2021-04-13 13:42   ` Hao Sun
2021-04-13 13:44     ` Jason Gunthorpe
2021-04-13 14:09       ` Haakon Bugge
2021-04-13 14:19       ` Hao Sun
2021-04-19 17:50         ` Jason Gunthorpe

Linux-RDMA Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rdma/0 linux-rdma/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rdma linux-rdma/ https://lore.kernel.org/linux-rdma \
		linux-rdma@vger.kernel.org
	public-inbox-index linux-rdma

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rdma


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git