* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update
@ 2017-03-10 13:38 ` Andrey Konovalov
0 siblings, 0 replies; 23+ messages in thread
From: Andrey Konovalov @ 2017-03-10 13:38 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář,
Christoffer Dall, Marc Zyngier, Catalin Marinas, Will Deacon,
Ingo Molnar, Michal Hocko, Christian Borntraeger,
Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm,
linux-arm-kernel, kvmarm, LKML
Cc: Dmitry Vyukov, Kostya Serebryany, syzkaller
Hi,
I've got the following error report while fuzzing the kernel with syzkaller.
On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8).
Unfortunately I can't reproduce it.
==================================================================
BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63
Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615
CPU: 1 PID: 26615 Comm: syz-executor Not tainted
4.11.0-rc1-next-20170308-xc2-dirty #3
Hardware name: Hardkernel ODROID-C2 (DT)
Call trace:
[<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505
[<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228
[<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline]
[<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52
[<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250
[<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349
[<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline]
[<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393
[<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63
[<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124
[<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488
arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817
[<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8
arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026
[<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58
arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075
[<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region
arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline]
[<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308
arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960
[<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline]
[<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685
[<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline]
[<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691
[<ffff200008083f70>] el0_svc_naked+0x24/0x28
Allocated by task 26657:
save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133
save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216
save_stack mm/kasan/kasan.c:515 [inline]
set_track mm/kasan/kasan.c:527 [inline]
kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619
kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557
slab_post_alloc_hook mm/slab.h:456 [inline]
slab_alloc_node mm/slub.c:2718 [inline]
slab_alloc mm/slub.c:2726 [inline]
kmem_cache_alloc+0x144/0x230 mm/slub.c:2731
__split_vma+0x118/0x608 mm/mmap.c:2515
do_munmap+0x194/0x9b0 mm/mmap.c:2636
Freed by task 26657:
save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133
save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216
save_stack mm/kasan/kasan.c:515 [inline]
set_track mm/kasan/kasan.c:527 [inline]
kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592
slab_free_hook mm/slub.c:1357 [inline]
slab_free_freelist_hook mm/slub.c:1379 [inline]
slab_free mm/slub.c:2961 [inline]
kmem_cache_free+0x80/0x258 mm/slub.c:2983
__vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28
The buggy address belongs to the object at ffff80003b9a2000
which belongs to the cache vm_area_struct(647:session-6.scope) of size 184
The buggy address is located 64 bytes inside of
184-byte region [ffff80003b9a2000, ffff80003b9a20b8)
The buggy address belongs to the page:
page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0
flags: 0xfffc00000000100(slab)
raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010
raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980
page dumped because: kasan: bad access detected
page->mem_cgroup:ffff80005ac99980
Memory state around the buggy address:
ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb
ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================
^ permalink raw reply [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-10 13:38 ` Andrey Konovalov 0 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-03-10 13:38 UTC (permalink / raw) To: linux-arm-kernel Hi, I've got the following error report while fuzzing the kernel with syzkaller. On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). Unfortunately I can't reproduce it. ================================================================== BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 CPU: 1 PID: 26615 Comm: syz-executor Not tainted 4.11.0-rc1-next-20170308-xc2-dirty #3 Hardware name: Hardkernel ODROID-C2 (DT) Call trace: [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 [<ffff200008083f70>] el0_svc_naked+0x24/0x28 Allocated by task 26657: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:515 [inline] set_track mm/kasan/kasan.c:527 [inline] kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 slab_post_alloc_hook mm/slab.h:456 [inline] slab_alloc_node mm/slub.c:2718 [inline] slab_alloc mm/slub.c:2726 [inline] kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 __split_vma+0x118/0x608 mm/mmap.c:2515 do_munmap+0x194/0x9b0 mm/mmap.c:2636 Freed by task 26657: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:515 [inline] set_track mm/kasan/kasan.c:527 [inline] kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 slab_free_hook mm/slub.c:1357 [inline] slab_free_freelist_hook mm/slub.c:1379 [inline] slab_free mm/slub.c:2961 [inline] kmem_cache_free+0x80/0x258 mm/slub.c:2983 __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 The buggy address belongs to the object at ffff80003b9a2000 which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 The buggy address is located 64 bytes inside of 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) The buggy address belongs to the page: page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 flags: 0xfffc00000000100(slab) raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 page dumped because: kasan: bad access detected page->mem_cgroup:ffff80005ac99980 Memory state around the buggy address: ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update 2017-03-10 13:38 ` Andrey Konovalov @ 2017-03-10 15:50 ` Andrey Konovalov -1 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-03-10 15:50 UTC (permalink / raw) To: Paolo Bonzini, Radim Krčmář, Christoffer Dall, Marc Zyngier, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML Cc: Dmitry Vyukov, Kostya Serebryany, syzkaller On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: > Hi, > > I've got the following error report while fuzzing the kernel with syzkaller. > > On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). > > Unfortunately I can't reproduce it. > > ================================================================== > BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 > Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 > > CPU: 1 PID: 26615 Comm: syz-executor Not tainted > 4.11.0-rc1-next-20170308-xc2-dirty #3 > Hardware name: Hardkernel ODROID-C2 (DT) > Call trace: > [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 > [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 > [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] > [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 > [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 > [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 > [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] > [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 > [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 > [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 > [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 > arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 > [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 > arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 > [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 > arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 > [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region > arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] > [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 > arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 > [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] > [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 > [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] > [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 > [<ffff200008083f70>] el0_svc_naked+0x24/0x28 > > Allocated by task 26657: > save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 > save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 > save_stack mm/kasan/kasan.c:515 [inline] > set_track mm/kasan/kasan.c:527 [inline] > kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 > kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 > slab_post_alloc_hook mm/slab.h:456 [inline] > slab_alloc_node mm/slub.c:2718 [inline] > slab_alloc mm/slub.c:2726 [inline] > kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 > __split_vma+0x118/0x608 mm/mmap.c:2515 > do_munmap+0x194/0x9b0 mm/mmap.c:2636 > Freed by task 26657: > save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 > save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 > save_stack mm/kasan/kasan.c:515 [inline] > set_track mm/kasan/kasan.c:527 [inline] > kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 > slab_free_hook mm/slub.c:1357 [inline] > slab_free_freelist_hook mm/slub.c:1379 [inline] > slab_free mm/slub.c:2961 [inline] > kmem_cache_free+0x80/0x258 mm/slub.c:2983 > __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 > > The buggy address belongs to the object at ffff80003b9a2000 > which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 > The buggy address is located 64 bytes inside of > 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) > The buggy address belongs to the page: > page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 > flags: 0xfffc00000000100(slab) > raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 > raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 > page dumped because: kasan: bad access detected > page->mem_cgroup:ffff80005ac99980 > > Memory state around the buggy address: > ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff > ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb > ^ > ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb > ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb > ================================================================== Another one that looks related and doesn't have parts of stack traces missing: ================================================================== BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 CPU: 2 PID: 4360 Comm: syz-executor Not tainted 4.11.0-rc1-next-20170308-xc2-dirty #3 Hardware name: Hardkernel ODROID-C2 (DT) Call trace: [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 [<ffff200008083f70>] el0_svc_naked+0x24/0x28 Allocated by task 4365: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:515 [inline] set_track mm/kasan/kasan.c:527 [inline] kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 slab_post_alloc_hook mm/slab.h:456 [inline] slab_alloc_node mm/slub.c:2718 [inline] slab_alloc mm/slub.c:2726 [inline] kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 __split_vma+0x118/0x608 mm/mmap.c:2515 do_munmap+0x194/0x9b0 mm/mmap.c:2636 mmap_region+0x138/0xc78 mm/mmap.c:1616 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2122 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 Freed by task 4365: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:515 [inline] set_track mm/kasan/kasan.c:527 [inline] kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 slab_free_hook mm/slub.c:1357 [inline] slab_free_freelist_hook mm/slub.c:1379 [inline] slab_free mm/slub.c:2961 [inline] kmem_cache_free+0x80/0x258 mm/slub.c:2983 __vma_adjust+0x6b0/0xff8 mm/mmap.c:890 vma_merge+0x880/0xa40 mm/mmap.c:1135 mmap_region+0x1f4/0xc78 mm/mmap.c:1633 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2122 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 The buggy address belongs to the object at ffff800031a03e88 which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 The buggy address is located 8 bytes inside of 184-byte region [ffff800031a03e88, ffff800031a03f40) The buggy address belongs to the page: page:ffff7e0000c680c0 count:1 mapcount:0 mapping: (null) index:0x0 flags: 0xfffc00000000100(slab) raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000100100010 raw: dead000000000100 dead000000000200 ffff80005228d000 ffff800052540000 page dumped because: kasan: bad access detected page->mem_cgroup:ffff800052540000 Memory state around the buggy address: ffff800031a03d80: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ffff800031a03e00: 00 00 00 00 00 00 00 00 00 fc fc fc fc fc fc fc >ffff800031a03e80: fc fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff800031a03f00: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc ffff800031a03f80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ================================================================== ^ permalink raw reply [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-10 15:50 ` Andrey Konovalov 0 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-03-10 15:50 UTC (permalink / raw) To: linux-arm-kernel On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: > Hi, > > I've got the following error report while fuzzing the kernel with syzkaller. > > On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). > > Unfortunately I can't reproduce it. > > ================================================================== > BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 > Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 > > CPU: 1 PID: 26615 Comm: syz-executor Not tainted > 4.11.0-rc1-next-20170308-xc2-dirty #3 > Hardware name: Hardkernel ODROID-C2 (DT) > Call trace: > [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 > [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 > [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] > [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 > [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 > [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 > [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] > [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 > [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 > [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 > [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 > arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 > [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 > arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 > [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 > arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 > [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region > arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] > [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 > arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 > [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] > [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 > [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] > [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 > [<ffff200008083f70>] el0_svc_naked+0x24/0x28 > > Allocated by task 26657: > save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 > save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 > save_stack mm/kasan/kasan.c:515 [inline] > set_track mm/kasan/kasan.c:527 [inline] > kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 > kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 > slab_post_alloc_hook mm/slab.h:456 [inline] > slab_alloc_node mm/slub.c:2718 [inline] > slab_alloc mm/slub.c:2726 [inline] > kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 > __split_vma+0x118/0x608 mm/mmap.c:2515 > do_munmap+0x194/0x9b0 mm/mmap.c:2636 > Freed by task 26657: > save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 > save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 > save_stack mm/kasan/kasan.c:515 [inline] > set_track mm/kasan/kasan.c:527 [inline] > kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 > slab_free_hook mm/slub.c:1357 [inline] > slab_free_freelist_hook mm/slub.c:1379 [inline] > slab_free mm/slub.c:2961 [inline] > kmem_cache_free+0x80/0x258 mm/slub.c:2983 > __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 > > The buggy address belongs to the object at ffff80003b9a2000 > which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 > The buggy address is located 64 bytes inside of > 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) > The buggy address belongs to the page: > page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 > flags: 0xfffc00000000100(slab) > raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 > raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 > page dumped because: kasan: bad access detected > page->mem_cgroup:ffff80005ac99980 > > Memory state around the buggy address: > ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff > ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb > ^ > ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb > ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb > ================================================================== Another one that looks related and doesn't have parts of stack traces missing: ================================================================== BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 CPU: 2 PID: 4360 Comm: syz-executor Not tainted 4.11.0-rc1-next-20170308-xc2-dirty #3 Hardware name: Hardkernel ODROID-C2 (DT) Call trace: [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 [<ffff200008083f70>] el0_svc_naked+0x24/0x28 Allocated by task 4365: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:515 [inline] set_track mm/kasan/kasan.c:527 [inline] kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 slab_post_alloc_hook mm/slab.h:456 [inline] slab_alloc_node mm/slub.c:2718 [inline] slab_alloc mm/slub.c:2726 [inline] kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 __split_vma+0x118/0x608 mm/mmap.c:2515 do_munmap+0x194/0x9b0 mm/mmap.c:2636 mmap_region+0x138/0xc78 mm/mmap.c:1616 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2122 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 Freed by task 4365: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:515 [inline] set_track mm/kasan/kasan.c:527 [inline] kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 slab_free_hook mm/slub.c:1357 [inline] slab_free_freelist_hook mm/slub.c:1379 [inline] slab_free mm/slub.c:2961 [inline] kmem_cache_free+0x80/0x258 mm/slub.c:2983 __vma_adjust+0x6b0/0xff8 mm/mmap.c:890 vma_merge+0x880/0xa40 mm/mmap.c:1135 mmap_region+0x1f4/0xc78 mm/mmap.c:1633 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2122 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 The buggy address belongs to the object at ffff800031a03e88 which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 The buggy address is located 8 bytes inside of 184-byte region [ffff800031a03e88, ffff800031a03f40) The buggy address belongs to the page: page:ffff7e0000c680c0 count:1 mapcount:0 mapping: (null) index:0x0 flags: 0xfffc00000000100(slab) raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000100100010 raw: dead000000000100 dead000000000200 ffff80005228d000 ffff800052540000 page dumped because: kasan: bad access detected page->mem_cgroup:ffff800052540000 Memory state around the buggy address: ffff800031a03d80: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ffff800031a03e00: 00 00 00 00 00 00 00 00 00 fc fc fc fc fc fc fc >ffff800031a03e80: fc fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff800031a03f00: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc ffff800031a03f80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ================================================================== ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update 2017-03-10 15:50 ` Andrey Konovalov (?) @ 2017-03-10 18:37 ` Suzuki K Poulose -1 siblings, 0 replies; 23+ messages in thread From: Suzuki K Poulose @ 2017-03-10 18:37 UTC (permalink / raw) To: Andrey Konovalov, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Marc Zyngier, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML Cc: Dmitry Vyukov, Kostya Serebryany, syzkaller On 10/03/17 15:50, Andrey Konovalov wrote: > On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >> Hi, >> >> I've got the following error report while fuzzing the kernel with syzkaller. >> >> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >> >> Unfortunately I can't reproduce it. >> >> ================================================================== >> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >> >> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >> 4.11.0-rc1-next-20170308-xc2-dirty #3 >> Hardware name: Hardkernel ODROID-C2 (DT) >> Call trace: >> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >> >> Allocated by task 26657: >> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >> save_stack mm/kasan/kasan.c:515 [inline] >> set_track mm/kasan/kasan.c:527 [inline] >> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >> slab_post_alloc_hook mm/slab.h:456 [inline] >> slab_alloc_node mm/slub.c:2718 [inline] >> slab_alloc mm/slub.c:2726 [inline] >> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >> __split_vma+0x118/0x608 mm/mmap.c:2515 >> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >> Freed by task 26657: >> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >> save_stack mm/kasan/kasan.c:515 [inline] >> set_track mm/kasan/kasan.c:527 [inline] >> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >> slab_free_hook mm/slub.c:1357 [inline] >> slab_free_freelist_hook mm/slub.c:1379 [inline] >> slab_free mm/slub.c:2961 [inline] >> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >> >> The buggy address belongs to the object at ffff80003b9a2000 >> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >> The buggy address is located 64 bytes inside of >> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >> The buggy address belongs to the page: >> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >> flags: 0xfffc00000000100(slab) >> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >> page dumped because: kasan: bad access detected >> page->mem_cgroup:ffff80005ac99980 >> >> Memory state around the buggy address: >> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> ^ >> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> ================================================================== > > Another one that looks related and doesn't have parts of stack traces missing: > > ================================================================== > BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 > Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 > > CPU: 2 PID: 4360 Comm: syz-executor Not tainted > 4.11.0-rc1-next-20170308-xc2-dirty #3 > Hardware name: Hardkernel ODROID-C2 (DT) > Call trace: > [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 > [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 > [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] > [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 > [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 > [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 > [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] > [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 > [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 > [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 > arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 It looks like we don't take the mmap_sem before calling find_vma() in stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing the race, with probably the test trying to unmap ranges in between. Suzuki ^ permalink raw reply [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-10 18:37 ` Suzuki K Poulose 0 siblings, 0 replies; 23+ messages in thread From: Suzuki K Poulose @ 2017-03-10 18:37 UTC (permalink / raw) To: linux-arm-kernel On 10/03/17 15:50, Andrey Konovalov wrote: > On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >> Hi, >> >> I've got the following error report while fuzzing the kernel with syzkaller. >> >> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >> >> Unfortunately I can't reproduce it. >> >> ================================================================== >> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >> >> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >> 4.11.0-rc1-next-20170308-xc2-dirty #3 >> Hardware name: Hardkernel ODROID-C2 (DT) >> Call trace: >> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >> >> Allocated by task 26657: >> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >> save_stack mm/kasan/kasan.c:515 [inline] >> set_track mm/kasan/kasan.c:527 [inline] >> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >> slab_post_alloc_hook mm/slab.h:456 [inline] >> slab_alloc_node mm/slub.c:2718 [inline] >> slab_alloc mm/slub.c:2726 [inline] >> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >> __split_vma+0x118/0x608 mm/mmap.c:2515 >> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >> Freed by task 26657: >> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >> save_stack mm/kasan/kasan.c:515 [inline] >> set_track mm/kasan/kasan.c:527 [inline] >> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >> slab_free_hook mm/slub.c:1357 [inline] >> slab_free_freelist_hook mm/slub.c:1379 [inline] >> slab_free mm/slub.c:2961 [inline] >> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >> >> The buggy address belongs to the object at ffff80003b9a2000 >> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >> The buggy address is located 64 bytes inside of >> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >> The buggy address belongs to the page: >> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >> flags: 0xfffc00000000100(slab) >> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >> page dumped because: kasan: bad access detected >> page->mem_cgroup:ffff80005ac99980 >> >> Memory state around the buggy address: >> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> ^ >> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> ================================================================== > > Another one that looks related and doesn't have parts of stack traces missing: > > ================================================================== > BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 > Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 > > CPU: 2 PID: 4360 Comm: syz-executor Not tainted > 4.11.0-rc1-next-20170308-xc2-dirty #3 > Hardware name: Hardkernel ODROID-C2 (DT) > Call trace: > [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 > [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 > [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] > [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 > [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 > [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 > [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] > [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 > [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 > [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 > arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 It looks like we don't take the mmap_sem before calling find_vma() in stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing the race, with probably the test trying to unmap ranges in between. Suzuki ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-10 18:37 ` Suzuki K Poulose 0 siblings, 0 replies; 23+ messages in thread From: Suzuki K Poulose @ 2017-03-10 18:37 UTC (permalink / raw) To: Andrey Konovalov, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Marc Zyngier, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML Cc: Kostya Serebryany, syzkaller, Dmitry Vyukov On 10/03/17 15:50, Andrey Konovalov wrote: > On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >> Hi, >> >> I've got the following error report while fuzzing the kernel with syzkaller. >> >> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >> >> Unfortunately I can't reproduce it. >> >> ================================================================== >> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >> >> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >> 4.11.0-rc1-next-20170308-xc2-dirty #3 >> Hardware name: Hardkernel ODROID-C2 (DT) >> Call trace: >> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >> >> Allocated by task 26657: >> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >> save_stack mm/kasan/kasan.c:515 [inline] >> set_track mm/kasan/kasan.c:527 [inline] >> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >> slab_post_alloc_hook mm/slab.h:456 [inline] >> slab_alloc_node mm/slub.c:2718 [inline] >> slab_alloc mm/slub.c:2726 [inline] >> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >> __split_vma+0x118/0x608 mm/mmap.c:2515 >> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >> Freed by task 26657: >> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >> save_stack mm/kasan/kasan.c:515 [inline] >> set_track mm/kasan/kasan.c:527 [inline] >> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >> slab_free_hook mm/slub.c:1357 [inline] >> slab_free_freelist_hook mm/slub.c:1379 [inline] >> slab_free mm/slub.c:2961 [inline] >> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >> >> The buggy address belongs to the object at ffff80003b9a2000 >> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >> The buggy address is located 64 bytes inside of >> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >> The buggy address belongs to the page: >> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >> flags: 0xfffc00000000100(slab) >> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >> page dumped because: kasan: bad access detected >> page->mem_cgroup:ffff80005ac99980 >> >> Memory state around the buggy address: >> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> ^ >> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >> ================================================================== > > Another one that looks related and doesn't have parts of stack traces missing: > > ================================================================== > BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 > Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 > > CPU: 2 PID: 4360 Comm: syz-executor Not tainted > 4.11.0-rc1-next-20170308-xc2-dirty #3 > Hardware name: Hardkernel ODROID-C2 (DT) > Call trace: > [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 > [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 > [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] > [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 > [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 > [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 > [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] > [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 > [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 > [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 > arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 It looks like we don't take the mmap_sem before calling find_vma() in stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing the race, with probably the test trying to unmap ranges in between. Suzuki ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update 2017-03-10 18:37 ` Suzuki K Poulose (?) @ 2017-03-13 9:58 ` Marc Zyngier -1 siblings, 0 replies; 23+ messages in thread From: Marc Zyngier @ 2017-03-13 9:58 UTC (permalink / raw) To: Suzuki K Poulose, Andrey Konovalov, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML Cc: Dmitry Vyukov, Kostya Serebryany, syzkaller On 10/03/17 18:37, Suzuki K Poulose wrote: > On 10/03/17 15:50, Andrey Konovalov wrote: >> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>> Hi, >>> >>> I've got the following error report while fuzzing the kernel with syzkaller. >>> >>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>> >>> Unfortunately I can't reproduce it. >>> >>> ================================================================== >>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>> >>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>> Hardware name: Hardkernel ODROID-C2 (DT) >>> Call trace: >>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>> >>> Allocated by task 26657: >>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>> save_stack mm/kasan/kasan.c:515 [inline] >>> set_track mm/kasan/kasan.c:527 [inline] >>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>> slab_post_alloc_hook mm/slab.h:456 [inline] >>> slab_alloc_node mm/slub.c:2718 [inline] >>> slab_alloc mm/slub.c:2726 [inline] >>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>> Freed by task 26657: >>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>> save_stack mm/kasan/kasan.c:515 [inline] >>> set_track mm/kasan/kasan.c:527 [inline] >>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>> slab_free_hook mm/slub.c:1357 [inline] >>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>> slab_free mm/slub.c:2961 [inline] >>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>> >>> The buggy address belongs to the object at ffff80003b9a2000 >>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>> The buggy address is located 64 bytes inside of >>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>> The buggy address belongs to the page: >>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>> flags: 0xfffc00000000100(slab) >>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>> page dumped because: kasan: bad access detected >>> page->mem_cgroup:ffff80005ac99980 >>> >>> Memory state around the buggy address: >>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>> ^ >>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>> ================================================================== >> >> Another one that looks related and doesn't have parts of stack traces missing: >> >> ================================================================== >> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >> >> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >> 4.11.0-rc1-next-20170308-xc2-dirty #3 >> Hardware name: Hardkernel ODROID-C2 (DT) >> Call trace: >> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 > > It looks like we don't take the mmap_sem before calling find_vma() in > stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing > the race, with probably the test trying to unmap ranges in between. That indeed seems like a possible failure mode. The annoying thing is that we're not exactly in a position to take mmap_sem in stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may have to hold mmap_sem while iterating over all the memslots. How about the following (very lightly tested): diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index 962616fd4ddd..2006a79d5912 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) int idx; idx = srcu_read_lock(&kvm->srcu); + down_read(¤t->mm->mmap_sem); spin_lock(&kvm->mmu_lock); slots = kvm_memslots(kvm); @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) stage2_unmap_memslot(kvm, memslot); spin_unlock(&kvm->mmu_lock); + up_read(¤t->mm->mmap_sem); srcu_read_unlock(&kvm->srcu, idx); } @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, * | memory region | * +--------------------------------------------+ */ + down_read(¤t->mm->mmap_sem); do { struct vm_area_struct *vma = find_vma(current->mm, hva); hva_t vm_start, vm_end; @@ -1857,7 +1860,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, } while (hva < reg_end); if (change == KVM_MR_FLAGS_ONLY) - return ret; + goto out; spin_lock(&kvm->mmu_lock); if (ret) @@ -1865,6 +1868,9 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, else stage2_flush_memslot(kvm, memslot); spin_unlock(&kvm->mmu_lock); + +out: + up_read(¤t->mm->mmap_sem); return ret; } I'm much more worried about the other report, as I don't really see yet how it happens. Coffee required. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply related [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-13 9:58 ` Marc Zyngier 0 siblings, 0 replies; 23+ messages in thread From: Marc Zyngier @ 2017-03-13 9:58 UTC (permalink / raw) To: linux-arm-kernel On 10/03/17 18:37, Suzuki K Poulose wrote: > On 10/03/17 15:50, Andrey Konovalov wrote: >> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>> Hi, >>> >>> I've got the following error report while fuzzing the kernel with syzkaller. >>> >>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>> >>> Unfortunately I can't reproduce it. >>> >>> ================================================================== >>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>> >>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>> Hardware name: Hardkernel ODROID-C2 (DT) >>> Call trace: >>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>> >>> Allocated by task 26657: >>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>> save_stack mm/kasan/kasan.c:515 [inline] >>> set_track mm/kasan/kasan.c:527 [inline] >>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>> slab_post_alloc_hook mm/slab.h:456 [inline] >>> slab_alloc_node mm/slub.c:2718 [inline] >>> slab_alloc mm/slub.c:2726 [inline] >>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>> Freed by task 26657: >>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>> save_stack mm/kasan/kasan.c:515 [inline] >>> set_track mm/kasan/kasan.c:527 [inline] >>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>> slab_free_hook mm/slub.c:1357 [inline] >>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>> slab_free mm/slub.c:2961 [inline] >>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>> >>> The buggy address belongs to the object at ffff80003b9a2000 >>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>> The buggy address is located 64 bytes inside of >>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>> The buggy address belongs to the page: >>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>> flags: 0xfffc00000000100(slab) >>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>> page dumped because: kasan: bad access detected >>> page->mem_cgroup:ffff80005ac99980 >>> >>> Memory state around the buggy address: >>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>> ^ >>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>> ================================================================== >> >> Another one that looks related and doesn't have parts of stack traces missing: >> >> ================================================================== >> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >> >> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >> 4.11.0-rc1-next-20170308-xc2-dirty #3 >> Hardware name: Hardkernel ODROID-C2 (DT) >> Call trace: >> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 > > It looks like we don't take the mmap_sem before calling find_vma() in > stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing > the race, with probably the test trying to unmap ranges in between. That indeed seems like a possible failure mode. The annoying thing is that we're not exactly in a position to take mmap_sem in stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may have to hold mmap_sem while iterating over all the memslots. How about the following (very lightly tested): diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index 962616fd4ddd..2006a79d5912 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) int idx; idx = srcu_read_lock(&kvm->srcu); + down_read(¤t->mm->mmap_sem); spin_lock(&kvm->mmu_lock); slots = kvm_memslots(kvm); @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) stage2_unmap_memslot(kvm, memslot); spin_unlock(&kvm->mmu_lock); + up_read(¤t->mm->mmap_sem); srcu_read_unlock(&kvm->srcu, idx); } @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, * | memory region | * +--------------------------------------------+ */ + down_read(¤t->mm->mmap_sem); do { struct vm_area_struct *vma = find_vma(current->mm, hva); hva_t vm_start, vm_end; @@ -1857,7 +1860,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, } while (hva < reg_end); if (change == KVM_MR_FLAGS_ONLY) - return ret; + goto out; spin_lock(&kvm->mmu_lock); if (ret) @@ -1865,6 +1868,9 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, else stage2_flush_memslot(kvm, memslot); spin_unlock(&kvm->mmu_lock); + +out: + up_read(¤t->mm->mmap_sem); return ret; } I'm much more worried about the other report, as I don't really see yet how it happens. Coffee required. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-13 9:58 ` Marc Zyngier 0 siblings, 0 replies; 23+ messages in thread From: Marc Zyngier @ 2017-03-13 9:58 UTC (permalink / raw) To: Suzuki K Poulose, Andrey Konovalov, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML Cc: Kostya Serebryany, syzkaller, Dmitry Vyukov On 10/03/17 18:37, Suzuki K Poulose wrote: > On 10/03/17 15:50, Andrey Konovalov wrote: >> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>> Hi, >>> >>> I've got the following error report while fuzzing the kernel with syzkaller. >>> >>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>> >>> Unfortunately I can't reproduce it. >>> >>> ================================================================== >>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>> >>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>> Hardware name: Hardkernel ODROID-C2 (DT) >>> Call trace: >>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>> >>> Allocated by task 26657: >>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>> save_stack mm/kasan/kasan.c:515 [inline] >>> set_track mm/kasan/kasan.c:527 [inline] >>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>> slab_post_alloc_hook mm/slab.h:456 [inline] >>> slab_alloc_node mm/slub.c:2718 [inline] >>> slab_alloc mm/slub.c:2726 [inline] >>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>> Freed by task 26657: >>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>> save_stack mm/kasan/kasan.c:515 [inline] >>> set_track mm/kasan/kasan.c:527 [inline] >>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>> slab_free_hook mm/slub.c:1357 [inline] >>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>> slab_free mm/slub.c:2961 [inline] >>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>> >>> The buggy address belongs to the object at ffff80003b9a2000 >>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>> The buggy address is located 64 bytes inside of >>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>> The buggy address belongs to the page: >>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>> flags: 0xfffc00000000100(slab) >>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>> page dumped because: kasan: bad access detected >>> page->mem_cgroup:ffff80005ac99980 >>> >>> Memory state around the buggy address: >>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>> ^ >>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>> ================================================================== >> >> Another one that looks related and doesn't have parts of stack traces missing: >> >> ================================================================== >> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >> >> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >> 4.11.0-rc1-next-20170308-xc2-dirty #3 >> Hardware name: Hardkernel ODROID-C2 (DT) >> Call trace: >> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 > > It looks like we don't take the mmap_sem before calling find_vma() in > stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing > the race, with probably the test trying to unmap ranges in between. That indeed seems like a possible failure mode. The annoying thing is that we're not exactly in a position to take mmap_sem in stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may have to hold mmap_sem while iterating over all the memslots. How about the following (very lightly tested): diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index 962616fd4ddd..2006a79d5912 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) int idx; idx = srcu_read_lock(&kvm->srcu); + down_read(¤t->mm->mmap_sem); spin_lock(&kvm->mmu_lock); slots = kvm_memslots(kvm); @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) stage2_unmap_memslot(kvm, memslot); spin_unlock(&kvm->mmu_lock); + up_read(¤t->mm->mmap_sem); srcu_read_unlock(&kvm->srcu, idx); } @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, * | memory region | * +--------------------------------------------+ */ + down_read(¤t->mm->mmap_sem); do { struct vm_area_struct *vma = find_vma(current->mm, hva); hva_t vm_start, vm_end; @@ -1857,7 +1860,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, } while (hva < reg_end); if (change == KVM_MR_FLAGS_ONLY) - return ret; + goto out; spin_lock(&kvm->mmu_lock); if (ret) @@ -1865,6 +1868,9 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, else stage2_flush_memslot(kvm, memslot); spin_unlock(&kvm->mmu_lock); + +out: + up_read(¤t->mm->mmap_sem); return ret; } I'm much more worried about the other report, as I don't really see yet how it happens. Coffee required. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update 2017-03-13 9:58 ` Marc Zyngier @ 2017-03-14 11:03 ` Suzuki K Poulose -1 siblings, 0 replies; 23+ messages in thread From: Suzuki K Poulose @ 2017-03-14 11:03 UTC (permalink / raw) To: Marc Zyngier, Andrey Konovalov, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML Cc: Dmitry Vyukov, Kostya Serebryany, syzkaller On 13/03/17 09:58, Marc Zyngier wrote: > On 10/03/17 18:37, Suzuki K Poulose wrote: >> On 10/03/17 15:50, Andrey Konovalov wrote: >>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>> Hi, >>>> >>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>> >>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>> >>>> Unfortunately I can't reproduce it. >>>> >>>> ================================================================== >>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>> >>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>> Call trace: >>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>> >>>> Allocated by task 26657: >>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>> save_stack mm/kasan/kasan.c:515 [inline] >>>> set_track mm/kasan/kasan.c:527 [inline] >>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>> slab_alloc_node mm/slub.c:2718 [inline] >>>> slab_alloc mm/slub.c:2726 [inline] >>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>> Freed by task 26657: >>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>> save_stack mm/kasan/kasan.c:515 [inline] >>>> set_track mm/kasan/kasan.c:527 [inline] >>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>> slab_free_hook mm/slub.c:1357 [inline] >>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>> slab_free mm/slub.c:2961 [inline] >>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>> >>>> The buggy address belongs to the object at ffff80003b9a2000 >>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>> The buggy address is located 64 bytes inside of >>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>> The buggy address belongs to the page: >>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>> flags: 0xfffc00000000100(slab) >>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>> page dumped because: kasan: bad access detected >>>> page->mem_cgroup:ffff80005ac99980 >>>> >>>> Memory state around the buggy address: >>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>> ^ >>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>> ================================================================== >>> >>> Another one that looks related and doesn't have parts of stack traces missing: >>> >>> ================================================================== >>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>> >>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>> Hardware name: Hardkernel ODROID-C2 (DT) >>> Call trace: >>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >> >> It looks like we don't take the mmap_sem before calling find_vma() in >> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >> the race, with probably the test trying to unmap ranges in between. > > That indeed seems like a possible failure mode. The annoying thing is > that we're not exactly in a position to take mmap_sem in > stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may > have to hold mmap_sem while iterating over all the memslots. > > How about the following (very lightly tested): > > diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c > index 962616fd4ddd..2006a79d5912 100644 > --- a/arch/arm/kvm/mmu.c > +++ b/arch/arm/kvm/mmu.c > @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) > int idx; > > idx = srcu_read_lock(&kvm->srcu); > + down_read(¤t->mm->mmap_sem); > spin_lock(&kvm->mmu_lock); > > slots = kvm_memslots(kvm); > @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) > stage2_unmap_memslot(kvm, memslot); > > spin_unlock(&kvm->mmu_lock); > + up_read(¤t->mm->mmap_sem); > srcu_read_unlock(&kvm->srcu, idx); > } > > @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, > * | memory region | > * +--------------------------------------------+ > */ > + down_read(¤t->mm->mmap_sem); > do { > struct vm_area_struct *vma = find_vma(current->mm, hva); > hva_t vm_start, vm_end; I have added the following hunk : @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, pa += vm_start - vma->vm_start; /* IO region dirty page logging not allowed */ - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) - return -EINVAL; + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { + ret = -EINVAL; + goto out; + } ret = kvm_phys_addr_ioremap(kvm, gpa, pa, vm_end - vm_start, > > I'm much more worried about the other report, as I don't really see yet > how it happens. Coffee required. I have something about that one. Will soon post a patch for it. Suzuki ^ permalink raw reply [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-14 11:03 ` Suzuki K Poulose 0 siblings, 0 replies; 23+ messages in thread From: Suzuki K Poulose @ 2017-03-14 11:03 UTC (permalink / raw) To: linux-arm-kernel On 13/03/17 09:58, Marc Zyngier wrote: > On 10/03/17 18:37, Suzuki K Poulose wrote: >> On 10/03/17 15:50, Andrey Konovalov wrote: >>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>> Hi, >>>> >>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>> >>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>> >>>> Unfortunately I can't reproduce it. >>>> >>>> ================================================================== >>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>> >>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>> Call trace: >>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>> >>>> Allocated by task 26657: >>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>> save_stack mm/kasan/kasan.c:515 [inline] >>>> set_track mm/kasan/kasan.c:527 [inline] >>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>> slab_alloc_node mm/slub.c:2718 [inline] >>>> slab_alloc mm/slub.c:2726 [inline] >>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>> Freed by task 26657: >>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>> save_stack mm/kasan/kasan.c:515 [inline] >>>> set_track mm/kasan/kasan.c:527 [inline] >>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>> slab_free_hook mm/slub.c:1357 [inline] >>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>> slab_free mm/slub.c:2961 [inline] >>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>> >>>> The buggy address belongs to the object at ffff80003b9a2000 >>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>> The buggy address is located 64 bytes inside of >>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>> The buggy address belongs to the page: >>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>> flags: 0xfffc00000000100(slab) >>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>> page dumped because: kasan: bad access detected >>>> page->mem_cgroup:ffff80005ac99980 >>>> >>>> Memory state around the buggy address: >>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>> ^ >>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>> ================================================================== >>> >>> Another one that looks related and doesn't have parts of stack traces missing: >>> >>> ================================================================== >>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>> >>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>> Hardware name: Hardkernel ODROID-C2 (DT) >>> Call trace: >>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >> >> It looks like we don't take the mmap_sem before calling find_vma() in >> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >> the race, with probably the test trying to unmap ranges in between. > > That indeed seems like a possible failure mode. The annoying thing is > that we're not exactly in a position to take mmap_sem in > stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may > have to hold mmap_sem while iterating over all the memslots. > > How about the following (very lightly tested): > > diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c > index 962616fd4ddd..2006a79d5912 100644 > --- a/arch/arm/kvm/mmu.c > +++ b/arch/arm/kvm/mmu.c > @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) > int idx; > > idx = srcu_read_lock(&kvm->srcu); > + down_read(¤t->mm->mmap_sem); > spin_lock(&kvm->mmu_lock); > > slots = kvm_memslots(kvm); > @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) > stage2_unmap_memslot(kvm, memslot); > > spin_unlock(&kvm->mmu_lock); > + up_read(¤t->mm->mmap_sem); > srcu_read_unlock(&kvm->srcu, idx); > } > > @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, > * | memory region | > * +--------------------------------------------+ > */ > + down_read(¤t->mm->mmap_sem); > do { > struct vm_area_struct *vma = find_vma(current->mm, hva); > hva_t vm_start, vm_end; I have added the following hunk : @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, pa += vm_start - vma->vm_start; /* IO region dirty page logging not allowed */ - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) - return -EINVAL; + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { + ret = -EINVAL; + goto out; + } ret = kvm_phys_addr_ioremap(kvm, gpa, pa, vm_end - vm_start, > > I'm much more worried about the other report, as I don't really see yet > how it happens. Coffee required. I have something about that one. Will soon post a patch for it. Suzuki ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update 2017-03-14 11:03 ` Suzuki K Poulose (?) @ 2017-03-14 12:26 ` Marc Zyngier -1 siblings, 0 replies; 23+ messages in thread From: Marc Zyngier @ 2017-03-14 12:26 UTC (permalink / raw) To: Suzuki K Poulose, Andrey Konovalov, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML Cc: Dmitry Vyukov, Kostya Serebryany, syzkaller On 14/03/17 11:03, Suzuki K Poulose wrote: > On 13/03/17 09:58, Marc Zyngier wrote: >> On 10/03/17 18:37, Suzuki K Poulose wrote: >>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>> Hi, >>>>> >>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>> >>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>> >>>>> Unfortunately I can't reproduce it. >>>>> >>>>> ================================================================== >>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>> >>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>> Call trace: >>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>> >>>>> Allocated by task 26657: >>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>> slab_alloc mm/slub.c:2726 [inline] >>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>> Freed by task 26657: >>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>> slab_free mm/slub.c:2961 [inline] >>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>> >>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>> The buggy address is located 64 bytes inside of >>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>> The buggy address belongs to the page: >>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>> flags: 0xfffc00000000100(slab) >>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>> page dumped because: kasan: bad access detected >>>>> page->mem_cgroup:ffff80005ac99980 >>>>> >>>>> Memory state around the buggy address: >>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>> ^ >>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>> ================================================================== >>>> >>>> Another one that looks related and doesn't have parts of stack traces missing: >>>> >>>> ================================================================== >>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>> >>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>> Call trace: >>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>> >>> It looks like we don't take the mmap_sem before calling find_vma() in >>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>> the race, with probably the test trying to unmap ranges in between. >> >> That indeed seems like a possible failure mode. The annoying thing is >> that we're not exactly in a position to take mmap_sem in >> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >> have to hold mmap_sem while iterating over all the memslots. >> >> How about the following (very lightly tested): >> >> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >> index 962616fd4ddd..2006a79d5912 100644 >> --- a/arch/arm/kvm/mmu.c >> +++ b/arch/arm/kvm/mmu.c >> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >> int idx; >> >> idx = srcu_read_lock(&kvm->srcu); >> + down_read(¤t->mm->mmap_sem); >> spin_lock(&kvm->mmu_lock); >> >> slots = kvm_memslots(kvm); >> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >> stage2_unmap_memslot(kvm, memslot); >> >> spin_unlock(&kvm->mmu_lock); >> + up_read(¤t->mm->mmap_sem); >> srcu_read_unlock(&kvm->srcu, idx); >> } >> >> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >> * | memory region | >> * +--------------------------------------------+ >> */ >> + down_read(¤t->mm->mmap_sem); >> do { >> struct vm_area_struct *vma = find_vma(current->mm, hva); >> hva_t vm_start, vm_end; > > I have added the following hunk : > > @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, > pa += vm_start - vma->vm_start; > > /* IO region dirty page logging not allowed */ > - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) > - return -EINVAL; > + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { > + ret = -EINVAL; > + goto out; > + } > > ret = kvm_phys_addr_ioremap(kvm, gpa, pa, > vm_end - vm_start, > Ah, of course... Thanks for pointing that out, I'll fix it as I post a proper patch. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-14 12:26 ` Marc Zyngier 0 siblings, 0 replies; 23+ messages in thread From: Marc Zyngier @ 2017-03-14 12:26 UTC (permalink / raw) To: linux-arm-kernel On 14/03/17 11:03, Suzuki K Poulose wrote: > On 13/03/17 09:58, Marc Zyngier wrote: >> On 10/03/17 18:37, Suzuki K Poulose wrote: >>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>> Hi, >>>>> >>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>> >>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>> >>>>> Unfortunately I can't reproduce it. >>>>> >>>>> ================================================================== >>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>> >>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>> Call trace: >>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>> >>>>> Allocated by task 26657: >>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>> slab_alloc mm/slub.c:2726 [inline] >>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>> Freed by task 26657: >>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>> slab_free mm/slub.c:2961 [inline] >>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>> >>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>> The buggy address is located 64 bytes inside of >>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>> The buggy address belongs to the page: >>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>> flags: 0xfffc00000000100(slab) >>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>> page dumped because: kasan: bad access detected >>>>> page->mem_cgroup:ffff80005ac99980 >>>>> >>>>> Memory state around the buggy address: >>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>> ^ >>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>> ================================================================== >>>> >>>> Another one that looks related and doesn't have parts of stack traces missing: >>>> >>>> ================================================================== >>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>> >>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>> Call trace: >>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>> >>> It looks like we don't take the mmap_sem before calling find_vma() in >>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>> the race, with probably the test trying to unmap ranges in between. >> >> That indeed seems like a possible failure mode. The annoying thing is >> that we're not exactly in a position to take mmap_sem in >> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >> have to hold mmap_sem while iterating over all the memslots. >> >> How about the following (very lightly tested): >> >> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >> index 962616fd4ddd..2006a79d5912 100644 >> --- a/arch/arm/kvm/mmu.c >> +++ b/arch/arm/kvm/mmu.c >> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >> int idx; >> >> idx = srcu_read_lock(&kvm->srcu); >> + down_read(¤t->mm->mmap_sem); >> spin_lock(&kvm->mmu_lock); >> >> slots = kvm_memslots(kvm); >> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >> stage2_unmap_memslot(kvm, memslot); >> >> spin_unlock(&kvm->mmu_lock); >> + up_read(¤t->mm->mmap_sem); >> srcu_read_unlock(&kvm->srcu, idx); >> } >> >> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >> * | memory region | >> * +--------------------------------------------+ >> */ >> + down_read(¤t->mm->mmap_sem); >> do { >> struct vm_area_struct *vma = find_vma(current->mm, hva); >> hva_t vm_start, vm_end; > > I have added the following hunk : > > @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, > pa += vm_start - vma->vm_start; > > /* IO region dirty page logging not allowed */ > - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) > - return -EINVAL; > + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { > + ret = -EINVAL; > + goto out; > + } > > ret = kvm_phys_addr_ioremap(kvm, gpa, pa, > vm_end - vm_start, > Ah, of course... Thanks for pointing that out, I'll fix it as I post a proper patch. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-03-14 12:26 ` Marc Zyngier 0 siblings, 0 replies; 23+ messages in thread From: Marc Zyngier @ 2017-03-14 12:26 UTC (permalink / raw) To: Suzuki K Poulose, Andrey Konovalov, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML Cc: Kostya Serebryany, syzkaller, Dmitry Vyukov On 14/03/17 11:03, Suzuki K Poulose wrote: > On 13/03/17 09:58, Marc Zyngier wrote: >> On 10/03/17 18:37, Suzuki K Poulose wrote: >>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>> Hi, >>>>> >>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>> >>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>> >>>>> Unfortunately I can't reproduce it. >>>>> >>>>> ================================================================== >>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>> >>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>> Call trace: >>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>> >>>>> Allocated by task 26657: >>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>> slab_alloc mm/slub.c:2726 [inline] >>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>> Freed by task 26657: >>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>> slab_free mm/slub.c:2961 [inline] >>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>> >>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>> The buggy address is located 64 bytes inside of >>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>> The buggy address belongs to the page: >>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>> flags: 0xfffc00000000100(slab) >>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>> page dumped because: kasan: bad access detected >>>>> page->mem_cgroup:ffff80005ac99980 >>>>> >>>>> Memory state around the buggy address: >>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>> ^ >>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>> ================================================================== >>>> >>>> Another one that looks related and doesn't have parts of stack traces missing: >>>> >>>> ================================================================== >>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>> >>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>> Call trace: >>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>> >>> It looks like we don't take the mmap_sem before calling find_vma() in >>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>> the race, with probably the test trying to unmap ranges in between. >> >> That indeed seems like a possible failure mode. The annoying thing is >> that we're not exactly in a position to take mmap_sem in >> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >> have to hold mmap_sem while iterating over all the memslots. >> >> How about the following (very lightly tested): >> >> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >> index 962616fd4ddd..2006a79d5912 100644 >> --- a/arch/arm/kvm/mmu.c >> +++ b/arch/arm/kvm/mmu.c >> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >> int idx; >> >> idx = srcu_read_lock(&kvm->srcu); >> + down_read(¤t->mm->mmap_sem); >> spin_lock(&kvm->mmu_lock); >> >> slots = kvm_memslots(kvm); >> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >> stage2_unmap_memslot(kvm, memslot); >> >> spin_unlock(&kvm->mmu_lock); >> + up_read(¤t->mm->mmap_sem); >> srcu_read_unlock(&kvm->srcu, idx); >> } >> >> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >> * | memory region | >> * +--------------------------------------------+ >> */ >> + down_read(¤t->mm->mmap_sem); >> do { >> struct vm_area_struct *vma = find_vma(current->mm, hva); >> hva_t vm_start, vm_end; > > I have added the following hunk : > > @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, > pa += vm_start - vma->vm_start; > > /* IO region dirty page logging not allowed */ > - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) > - return -EINVAL; > + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { > + ret = -EINVAL; > + goto out; > + } > > ret = kvm_phys_addr_ioremap(kvm, gpa, pa, > vm_end - vm_start, > Ah, of course... Thanks for pointing that out, I'll fix it as I post a proper patch. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update 2017-03-14 12:26 ` Marc Zyngier (?) @ 2017-04-11 15:26 ` Andrey Konovalov -1 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-04-11 15:26 UTC (permalink / raw) To: Marc Zyngier Cc: Suzuki K Poulose, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML, Dmitry Vyukov, Kostya Serebryany, syzkaller On Tue, Mar 14, 2017 at 1:26 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > On 14/03/17 11:03, Suzuki K Poulose wrote: >> On 13/03/17 09:58, Marc Zyngier wrote: >>> On 10/03/17 18:37, Suzuki K Poulose wrote: >>>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>>> Hi, >>>>>> >>>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>>> >>>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>>> >>>>>> Unfortunately I can't reproduce it. >>>>>> >>>>>> ================================================================== >>>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>>> >>>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>> Call trace: >>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>>> >>>>>> Allocated by task 26657: >>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>>> slab_alloc mm/slub.c:2726 [inline] >>>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>>> Freed by task 26657: >>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>>> slab_free mm/slub.c:2961 [inline] >>>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>>> >>>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>>> The buggy address is located 64 bytes inside of >>>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>>> The buggy address belongs to the page: >>>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>>> flags: 0xfffc00000000100(slab) >>>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>>> page dumped because: kasan: bad access detected >>>>>> page->mem_cgroup:ffff80005ac99980 >>>>>> >>>>>> Memory state around the buggy address: >>>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>> ^ >>>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>> ================================================================== >>>>> >>>>> Another one that looks related and doesn't have parts of stack traces missing: >>>>> >>>>> ================================================================== >>>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>>> >>>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>> Call trace: >>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>> >>>> It looks like we don't take the mmap_sem before calling find_vma() in >>>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>>> the race, with probably the test trying to unmap ranges in between. >>> >>> That indeed seems like a possible failure mode. The annoying thing is >>> that we're not exactly in a position to take mmap_sem in >>> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >>> have to hold mmap_sem while iterating over all the memslots. >>> >>> How about the following (very lightly tested): >>> >>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >>> index 962616fd4ddd..2006a79d5912 100644 >>> --- a/arch/arm/kvm/mmu.c >>> +++ b/arch/arm/kvm/mmu.c >>> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>> int idx; >>> >>> idx = srcu_read_lock(&kvm->srcu); >>> + down_read(¤t->mm->mmap_sem); >>> spin_lock(&kvm->mmu_lock); >>> >>> slots = kvm_memslots(kvm); >>> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>> stage2_unmap_memslot(kvm, memslot); >>> >>> spin_unlock(&kvm->mmu_lock); >>> + up_read(¤t->mm->mmap_sem); >>> srcu_read_unlock(&kvm->srcu, idx); >>> } >>> >>> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>> * | memory region | >>> * +--------------------------------------------+ >>> */ >>> + down_read(¤t->mm->mmap_sem); >>> do { >>> struct vm_area_struct *vma = find_vma(current->mm, hva); >>> hva_t vm_start, vm_end; >> >> I have added the following hunk : >> >> @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >> pa += vm_start - vma->vm_start; >> >> /* IO region dirty page logging not allowed */ >> - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) >> - return -EINVAL; >> + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { >> + ret = -EINVAL; >> + goto out; >> + } >> >> ret = kvm_phys_addr_ioremap(kvm, gpa, pa, >> vm_end - vm_start, >> > > Ah, of course... Thanks for pointing that out, I'll fix it as I post a > proper patch. > > Thanks, > > M. > -- > Jazz is not dead. It just smells funny... Hi, I've got this report again on linux-next 4c3c5cd02318 (Apr 5). It seems that it's still not fixed. Thanks! ================================================================== BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 Read of size 8 at addr ffff80004d53aae8 by task syz-executor/23095 CPU: 0 PID: 23095 Comm: syz-executor Not tainted 4.11.0-rc5-next-20170405-xc2-09030-g4c3c5cd02318-dirty #4 Hardware name: Hardkernel ODROID-C2 (DT) Call trace: [<ffff20000808fb90>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 [<ffff20000808fff0>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 [<ffff2000088d5bb0>] __dump_stack lib/dump_stack.c:16 [inline] [<ffff2000088d5bb0>] dump_stack+0x110/0x168 lib/dump_stack.c:52 [<ffff200008403d98>] print_address_description+0x60/0x248 mm/kasan/report.c:252 [<ffff2000084042a8>] kasan_report_error mm/kasan/report.c:351 [inline] [<ffff2000084042a8>] kasan_report+0x218/0x300 mm/kasan/report.c:408 [<ffff200008404408>] __asan_report_load8_noabort+0x18/0x20 mm/kasan/report.c:429 [<ffff20000837444c>] vmacache_update+0x114/0x118 mm/vmacache.c:63 [<ffff2000083991f0>] find_vma+0xf8/0x150 mm/mmap.c:2124 [<ffff2000080dc59c>] kvm_arch_prepare_memory_region+0x2ac/0x488 arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1812 [<ffff2000080c2c18>] __kvm_set_memory_region+0x4a0/0x1340 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1029 [<ffff2000080c3af0>] kvm_set_memory_region+0x38/0x58 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1078 [<ffff2000080c7784>] kvm_vm_ioctl_set_memory_region arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1090 [inline] [<ffff2000080c7784>] kvm_vm_ioctl+0xb94/0x1308 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2963 [<ffff20000847e590>] vfs_ioctl fs/ioctl.c:45 [inline] [<ffff20000847e590>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 [<ffff20000847f4d0>] SYSC_ioctl fs/ioctl.c:700 [inline] [<ffff20000847f4d0>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 [<ffff200008083f70>] el0_svc_naked+0x24/0x28 Allocated by task 23107: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:513 [inline] set_track mm/kasan/kasan.c:525 [inline] kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:617 kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:555 slab_post_alloc_hook mm/slab.h:457 [inline] slab_alloc_node mm/slub.c:2718 [inline] slab_alloc mm/slub.c:2726 [inline] kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 __split_vma+0x118/0x608 mm/mmap.c:2515 do_munmap+0x194/0x9b0 mm/mmap.c:2636 mmap_region+0x138/0xc60 mm/mmap.c:1616 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2149 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 Freed by task 23107: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:513 [inline] set_track mm/kasan/kasan.c:525 [inline] kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:590 slab_free_hook mm/slub.c:1357 [inline] slab_free_freelist_hook mm/slub.c:1379 [inline] slab_free mm/slub.c:2961 [inline] kmem_cache_free+0x80/0x260 mm/slub.c:2983 __vma_adjust+0x6b0/0xff8 mm/mmap.c:890 vma_merge+0x880/0xa40 mm/mmap.c:1135 mmap_region+0x1f4/0xc60 mm/mmap.c:1633 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2149 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 The buggy address belongs to the object at ffff80004d53aaa8 which belongs to the cache vm_area_struct(673:session-6.scope) of size 184 The buggy address is located 64 bytes inside of 184-byte region [ffff80004d53aaa8, ffff80004d53ab60) The buggy address belongs to the page: page:ffff7e0001354e80 count:1 mapcount:0 mapping: (null) index:0x0 flags: 0xfffc00000000100(slab) raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000100100010 raw: dead000000000100 dead000000000200 ffff800048440200 ffff800056825500 page dumped because: kasan: bad access detected page->mem_cgroup:ffff800056825500 Memory state around the buggy address: ffff80004d53a980: fc fc fc fc fc fc fb fb fb fb fb fb fb fb fb fb ffff80004d53aa00: fb fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc >ffff80004d53aa80: fc fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb ^ ffff80004d53ab00: fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc ffff80004d53ab80: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== ^ permalink raw reply [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-04-11 15:26 ` Andrey Konovalov 0 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-04-11 15:26 UTC (permalink / raw) To: linux-arm-kernel On Tue, Mar 14, 2017 at 1:26 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > On 14/03/17 11:03, Suzuki K Poulose wrote: >> On 13/03/17 09:58, Marc Zyngier wrote: >>> On 10/03/17 18:37, Suzuki K Poulose wrote: >>>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>>> Hi, >>>>>> >>>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>>> >>>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>>> >>>>>> Unfortunately I can't reproduce it. >>>>>> >>>>>> ================================================================== >>>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>>> >>>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>> Call trace: >>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>>> >>>>>> Allocated by task 26657: >>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>>> slab_alloc mm/slub.c:2726 [inline] >>>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>>> Freed by task 26657: >>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>>> slab_free mm/slub.c:2961 [inline] >>>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>>> >>>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>>> The buggy address is located 64 bytes inside of >>>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>>> The buggy address belongs to the page: >>>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>>> flags: 0xfffc00000000100(slab) >>>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>>> page dumped because: kasan: bad access detected >>>>>> page->mem_cgroup:ffff80005ac99980 >>>>>> >>>>>> Memory state around the buggy address: >>>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>> ^ >>>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>> ================================================================== >>>>> >>>>> Another one that looks related and doesn't have parts of stack traces missing: >>>>> >>>>> ================================================================== >>>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>>> >>>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>> Call trace: >>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>> >>>> It looks like we don't take the mmap_sem before calling find_vma() in >>>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>>> the race, with probably the test trying to unmap ranges in between. >>> >>> That indeed seems like a possible failure mode. The annoying thing is >>> that we're not exactly in a position to take mmap_sem in >>> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >>> have to hold mmap_sem while iterating over all the memslots. >>> >>> How about the following (very lightly tested): >>> >>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >>> index 962616fd4ddd..2006a79d5912 100644 >>> --- a/arch/arm/kvm/mmu.c >>> +++ b/arch/arm/kvm/mmu.c >>> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>> int idx; >>> >>> idx = srcu_read_lock(&kvm->srcu); >>> + down_read(¤t->mm->mmap_sem); >>> spin_lock(&kvm->mmu_lock); >>> >>> slots = kvm_memslots(kvm); >>> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>> stage2_unmap_memslot(kvm, memslot); >>> >>> spin_unlock(&kvm->mmu_lock); >>> + up_read(¤t->mm->mmap_sem); >>> srcu_read_unlock(&kvm->srcu, idx); >>> } >>> >>> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>> * | memory region | >>> * +--------------------------------------------+ >>> */ >>> + down_read(¤t->mm->mmap_sem); >>> do { >>> struct vm_area_struct *vma = find_vma(current->mm, hva); >>> hva_t vm_start, vm_end; >> >> I have added the following hunk : >> >> @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >> pa += vm_start - vma->vm_start; >> >> /* IO region dirty page logging not allowed */ >> - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) >> - return -EINVAL; >> + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { >> + ret = -EINVAL; >> + goto out; >> + } >> >> ret = kvm_phys_addr_ioremap(kvm, gpa, pa, >> vm_end - vm_start, >> > > Ah, of course... Thanks for pointing that out, I'll fix it as I post a > proper patch. > > Thanks, > > M. > -- > Jazz is not dead. It just smells funny... Hi, I've got this report again on linux-next 4c3c5cd02318 (Apr 5). It seems that it's still not fixed. Thanks! ================================================================== BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 Read of size 8 at addr ffff80004d53aae8 by task syz-executor/23095 CPU: 0 PID: 23095 Comm: syz-executor Not tainted 4.11.0-rc5-next-20170405-xc2-09030-g4c3c5cd02318-dirty #4 Hardware name: Hardkernel ODROID-C2 (DT) Call trace: [<ffff20000808fb90>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 [<ffff20000808fff0>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 [<ffff2000088d5bb0>] __dump_stack lib/dump_stack.c:16 [inline] [<ffff2000088d5bb0>] dump_stack+0x110/0x168 lib/dump_stack.c:52 [<ffff200008403d98>] print_address_description+0x60/0x248 mm/kasan/report.c:252 [<ffff2000084042a8>] kasan_report_error mm/kasan/report.c:351 [inline] [<ffff2000084042a8>] kasan_report+0x218/0x300 mm/kasan/report.c:408 [<ffff200008404408>] __asan_report_load8_noabort+0x18/0x20 mm/kasan/report.c:429 [<ffff20000837444c>] vmacache_update+0x114/0x118 mm/vmacache.c:63 [<ffff2000083991f0>] find_vma+0xf8/0x150 mm/mmap.c:2124 [<ffff2000080dc59c>] kvm_arch_prepare_memory_region+0x2ac/0x488 arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1812 [<ffff2000080c2c18>] __kvm_set_memory_region+0x4a0/0x1340 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1029 [<ffff2000080c3af0>] kvm_set_memory_region+0x38/0x58 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1078 [<ffff2000080c7784>] kvm_vm_ioctl_set_memory_region arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1090 [inline] [<ffff2000080c7784>] kvm_vm_ioctl+0xb94/0x1308 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2963 [<ffff20000847e590>] vfs_ioctl fs/ioctl.c:45 [inline] [<ffff20000847e590>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 [<ffff20000847f4d0>] SYSC_ioctl fs/ioctl.c:700 [inline] [<ffff20000847f4d0>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 [<ffff200008083f70>] el0_svc_naked+0x24/0x28 Allocated by task 23107: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:513 [inline] set_track mm/kasan/kasan.c:525 [inline] kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:617 kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:555 slab_post_alloc_hook mm/slab.h:457 [inline] slab_alloc_node mm/slub.c:2718 [inline] slab_alloc mm/slub.c:2726 [inline] kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 __split_vma+0x118/0x608 mm/mmap.c:2515 do_munmap+0x194/0x9b0 mm/mmap.c:2636 mmap_region+0x138/0xc60 mm/mmap.c:1616 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2149 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 Freed by task 23107: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:513 [inline] set_track mm/kasan/kasan.c:525 [inline] kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:590 slab_free_hook mm/slub.c:1357 [inline] slab_free_freelist_hook mm/slub.c:1379 [inline] slab_free mm/slub.c:2961 [inline] kmem_cache_free+0x80/0x260 mm/slub.c:2983 __vma_adjust+0x6b0/0xff8 mm/mmap.c:890 vma_merge+0x880/0xa40 mm/mmap.c:1135 mmap_region+0x1f4/0xc60 mm/mmap.c:1633 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2149 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 The buggy address belongs to the object at ffff80004d53aaa8 which belongs to the cache vm_area_struct(673:session-6.scope) of size 184 The buggy address is located 64 bytes inside of 184-byte region [ffff80004d53aaa8, ffff80004d53ab60) The buggy address belongs to the page: page:ffff7e0001354e80 count:1 mapcount:0 mapping: (null) index:0x0 flags: 0xfffc00000000100(slab) raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000100100010 raw: dead000000000100 dead000000000200 ffff800048440200 ffff800056825500 page dumped because: kasan: bad access detected page->mem_cgroup:ffff800056825500 Memory state around the buggy address: ffff80004d53a980: fc fc fc fc fc fc fb fb fb fb fb fb fb fb fb fb ffff80004d53aa00: fb fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc >ffff80004d53aa80: fc fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb ^ ffff80004d53ab00: fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc ffff80004d53ab80: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-04-11 15:26 ` Andrey Konovalov 0 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-04-11 15:26 UTC (permalink / raw) To: Marc Zyngier Cc: Kostya Serebryany, Michal Hocko, Lorenzo Stoakes, kvm, Catalin Marinas, Will Deacon, LKML, Markus Elfring, Christian Borntraeger, syzkaller, kvmarm, linux-arm-kernel, Suraj Jitindar Singh, Dmitry Vyukov, Paolo Bonzini, Ingo Molnar On Tue, Mar 14, 2017 at 1:26 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > On 14/03/17 11:03, Suzuki K Poulose wrote: >> On 13/03/17 09:58, Marc Zyngier wrote: >>> On 10/03/17 18:37, Suzuki K Poulose wrote: >>>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>>> Hi, >>>>>> >>>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>>> >>>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>>> >>>>>> Unfortunately I can't reproduce it. >>>>>> >>>>>> ================================================================== >>>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>>> >>>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>> Call trace: >>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>>> >>>>>> Allocated by task 26657: >>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>>> slab_alloc mm/slub.c:2726 [inline] >>>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>>> Freed by task 26657: >>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>>> slab_free mm/slub.c:2961 [inline] >>>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>>> >>>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>>> The buggy address is located 64 bytes inside of >>>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>>> The buggy address belongs to the page: >>>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>>> flags: 0xfffc00000000100(slab) >>>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>>> page dumped because: kasan: bad access detected >>>>>> page->mem_cgroup:ffff80005ac99980 >>>>>> >>>>>> Memory state around the buggy address: >>>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>> ^ >>>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>> ================================================================== >>>>> >>>>> Another one that looks related and doesn't have parts of stack traces missing: >>>>> >>>>> ================================================================== >>>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>>> >>>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>> Call trace: >>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>> >>>> It looks like we don't take the mmap_sem before calling find_vma() in >>>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>>> the race, with probably the test trying to unmap ranges in between. >>> >>> That indeed seems like a possible failure mode. The annoying thing is >>> that we're not exactly in a position to take mmap_sem in >>> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >>> have to hold mmap_sem while iterating over all the memslots. >>> >>> How about the following (very lightly tested): >>> >>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >>> index 962616fd4ddd..2006a79d5912 100644 >>> --- a/arch/arm/kvm/mmu.c >>> +++ b/arch/arm/kvm/mmu.c >>> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>> int idx; >>> >>> idx = srcu_read_lock(&kvm->srcu); >>> + down_read(¤t->mm->mmap_sem); >>> spin_lock(&kvm->mmu_lock); >>> >>> slots = kvm_memslots(kvm); >>> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>> stage2_unmap_memslot(kvm, memslot); >>> >>> spin_unlock(&kvm->mmu_lock); >>> + up_read(¤t->mm->mmap_sem); >>> srcu_read_unlock(&kvm->srcu, idx); >>> } >>> >>> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>> * | memory region | >>> * +--------------------------------------------+ >>> */ >>> + down_read(¤t->mm->mmap_sem); >>> do { >>> struct vm_area_struct *vma = find_vma(current->mm, hva); >>> hva_t vm_start, vm_end; >> >> I have added the following hunk : >> >> @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >> pa += vm_start - vma->vm_start; >> >> /* IO region dirty page logging not allowed */ >> - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) >> - return -EINVAL; >> + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { >> + ret = -EINVAL; >> + goto out; >> + } >> >> ret = kvm_phys_addr_ioremap(kvm, gpa, pa, >> vm_end - vm_start, >> > > Ah, of course... Thanks for pointing that out, I'll fix it as I post a > proper patch. > > Thanks, > > M. > -- > Jazz is not dead. It just smells funny... Hi, I've got this report again on linux-next 4c3c5cd02318 (Apr 5). It seems that it's still not fixed. Thanks! ================================================================== BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 Read of size 8 at addr ffff80004d53aae8 by task syz-executor/23095 CPU: 0 PID: 23095 Comm: syz-executor Not tainted 4.11.0-rc5-next-20170405-xc2-09030-g4c3c5cd02318-dirty #4 Hardware name: Hardkernel ODROID-C2 (DT) Call trace: [<ffff20000808fb90>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 [<ffff20000808fff0>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 [<ffff2000088d5bb0>] __dump_stack lib/dump_stack.c:16 [inline] [<ffff2000088d5bb0>] dump_stack+0x110/0x168 lib/dump_stack.c:52 [<ffff200008403d98>] print_address_description+0x60/0x248 mm/kasan/report.c:252 [<ffff2000084042a8>] kasan_report_error mm/kasan/report.c:351 [inline] [<ffff2000084042a8>] kasan_report+0x218/0x300 mm/kasan/report.c:408 [<ffff200008404408>] __asan_report_load8_noabort+0x18/0x20 mm/kasan/report.c:429 [<ffff20000837444c>] vmacache_update+0x114/0x118 mm/vmacache.c:63 [<ffff2000083991f0>] find_vma+0xf8/0x150 mm/mmap.c:2124 [<ffff2000080dc59c>] kvm_arch_prepare_memory_region+0x2ac/0x488 arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1812 [<ffff2000080c2c18>] __kvm_set_memory_region+0x4a0/0x1340 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1029 [<ffff2000080c3af0>] kvm_set_memory_region+0x38/0x58 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1078 [<ffff2000080c7784>] kvm_vm_ioctl_set_memory_region arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1090 [inline] [<ffff2000080c7784>] kvm_vm_ioctl+0xb94/0x1308 arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2963 [<ffff20000847e590>] vfs_ioctl fs/ioctl.c:45 [inline] [<ffff20000847e590>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 [<ffff20000847f4d0>] SYSC_ioctl fs/ioctl.c:700 [inline] [<ffff20000847f4d0>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 [<ffff200008083f70>] el0_svc_naked+0x24/0x28 Allocated by task 23107: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:513 [inline] set_track mm/kasan/kasan.c:525 [inline] kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:617 kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:555 slab_post_alloc_hook mm/slab.h:457 [inline] slab_alloc_node mm/slub.c:2718 [inline] slab_alloc mm/slub.c:2726 [inline] kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 __split_vma+0x118/0x608 mm/mmap.c:2515 do_munmap+0x194/0x9b0 mm/mmap.c:2636 mmap_region+0x138/0xc60 mm/mmap.c:1616 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2149 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 Freed by task 23107: save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 save_stack mm/kasan/kasan.c:513 [inline] set_track mm/kasan/kasan.c:525 [inline] kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:590 slab_free_hook mm/slub.c:1357 [inline] slab_free_freelist_hook mm/slub.c:1379 [inline] slab_free mm/slub.c:2961 [inline] kmem_cache_free+0x80/0x260 mm/slub.c:2983 __vma_adjust+0x6b0/0xff8 mm/mmap.c:890 vma_merge+0x880/0xa40 mm/mmap.c:1135 mmap_region+0x1f4/0xc60 mm/mmap.c:1633 do_mmap+0x3cc/0x848 mm/mmap.c:1453 do_mmap_pgoff include/linux/mm.h:2149 [inline] vm_mmap_pgoff+0xec/0x120 mm/util.c:309 SYSC_mmap_pgoff mm/mmap.c:1503 [inline] SyS_mmap_pgoff+0x220/0x420 mm/mmap.c:1461 sys_mmap+0x58/0x80 arch/arm64/kernel/sys.c:37 el0_svc_naked+0x24/0x28 The buggy address belongs to the object at ffff80004d53aaa8 which belongs to the cache vm_area_struct(673:session-6.scope) of size 184 The buggy address is located 64 bytes inside of 184-byte region [ffff80004d53aaa8, ffff80004d53ab60) The buggy address belongs to the page: page:ffff7e0001354e80 count:1 mapcount:0 mapping: (null) index:0x0 flags: 0xfffc00000000100(slab) raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000100100010 raw: dead000000000100 dead000000000200 ffff800048440200 ffff800056825500 page dumped because: kasan: bad access detected page->mem_cgroup:ffff800056825500 Memory state around the buggy address: ffff80004d53a980: fc fc fc fc fc fc fb fb fb fb fb fb fb fb fb fb ffff80004d53aa00: fb fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc >ffff80004d53aa80: fc fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb ^ ffff80004d53ab00: fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc ffff80004d53ab80: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update 2017-04-11 15:26 ` Andrey Konovalov @ 2017-04-11 15:36 ` Marc Zyngier -1 siblings, 0 replies; 23+ messages in thread From: Marc Zyngier @ 2017-04-11 15:36 UTC (permalink / raw) To: Andrey Konovalov Cc: Suzuki K Poulose, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML, Dmitry Vyukov, Kostya Serebryany, syzkaller On 11/04/17 16:26, Andrey Konovalov wrote: > On Tue, Mar 14, 2017 at 1:26 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >> On 14/03/17 11:03, Suzuki K Poulose wrote: >>> On 13/03/17 09:58, Marc Zyngier wrote: >>>> On 10/03/17 18:37, Suzuki K Poulose wrote: >>>>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>>>> >>>>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>>>> >>>>>>> Unfortunately I can't reproduce it. >>>>>>> >>>>>>> ================================================================== >>>>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>>>> >>>>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>>> Call trace: >>>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>>>> >>>>>>> Allocated by task 26657: >>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>>>> slab_alloc mm/slub.c:2726 [inline] >>>>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>>>> Freed by task 26657: >>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>>>> slab_free mm/slub.c:2961 [inline] >>>>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>>>> >>>>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>>>> The buggy address is located 64 bytes inside of >>>>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>>>> The buggy address belongs to the page: >>>>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>>>> flags: 0xfffc00000000100(slab) >>>>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>>>> page dumped because: kasan: bad access detected >>>>>>> page->mem_cgroup:ffff80005ac99980 >>>>>>> >>>>>>> Memory state around the buggy address: >>>>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>> ^ >>>>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>> ================================================================== >>>>>> >>>>>> Another one that looks related and doesn't have parts of stack traces missing: >>>>>> >>>>>> ================================================================== >>>>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>>>> >>>>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>> Call trace: >>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>> >>>>> It looks like we don't take the mmap_sem before calling find_vma() in >>>>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>>>> the race, with probably the test trying to unmap ranges in between. >>>> >>>> That indeed seems like a possible failure mode. The annoying thing is >>>> that we're not exactly in a position to take mmap_sem in >>>> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >>>> have to hold mmap_sem while iterating over all the memslots. >>>> >>>> How about the following (very lightly tested): >>>> >>>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >>>> index 962616fd4ddd..2006a79d5912 100644 >>>> --- a/arch/arm/kvm/mmu.c >>>> +++ b/arch/arm/kvm/mmu.c >>>> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>> int idx; >>>> >>>> idx = srcu_read_lock(&kvm->srcu); >>>> + down_read(¤t->mm->mmap_sem); >>>> spin_lock(&kvm->mmu_lock); >>>> >>>> slots = kvm_memslots(kvm); >>>> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>> stage2_unmap_memslot(kvm, memslot); >>>> >>>> spin_unlock(&kvm->mmu_lock); >>>> + up_read(¤t->mm->mmap_sem); >>>> srcu_read_unlock(&kvm->srcu, idx); >>>> } >>>> >>>> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>> * | memory region | >>>> * +--------------------------------------------+ >>>> */ >>>> + down_read(¤t->mm->mmap_sem); >>>> do { >>>> struct vm_area_struct *vma = find_vma(current->mm, hva); >>>> hva_t vm_start, vm_end; >>> >>> I have added the following hunk : >>> >>> @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>> pa += vm_start - vma->vm_start; >>> >>> /* IO region dirty page logging not allowed */ >>> - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) >>> - return -EINVAL; >>> + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { >>> + ret = -EINVAL; >>> + goto out; >>> + } >>> >>> ret = kvm_phys_addr_ioremap(kvm, gpa, pa, >>> vm_end - vm_start, >>> >> >> Ah, of course... Thanks for pointing that out, I'll fix it as I post a >> proper patch. >> >> Thanks, >> >> M. >> -- >> Jazz is not dead. It just smells funny... > > Hi, > > I've got this report again on linux-next 4c3c5cd02318 (Apr 5). > > It seems that it's still not fixed. > > Thanks! > > ================================================================== > BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 > Read of size 8 at addr ffff80004d53aae8 by task syz-executor/23095 > > CPU: 0 PID: 23095 Comm: syz-executor Not tainted > 4.11.0-rc5-next-20170405-xc2-09030-g4c3c5cd02318-dirty #4 The fixes went into -rc6. Can you please give it a go? Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-04-11 15:36 ` Marc Zyngier 0 siblings, 0 replies; 23+ messages in thread From: Marc Zyngier @ 2017-04-11 15:36 UTC (permalink / raw) To: linux-arm-kernel On 11/04/17 16:26, Andrey Konovalov wrote: > On Tue, Mar 14, 2017 at 1:26 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >> On 14/03/17 11:03, Suzuki K Poulose wrote: >>> On 13/03/17 09:58, Marc Zyngier wrote: >>>> On 10/03/17 18:37, Suzuki K Poulose wrote: >>>>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>>>> >>>>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>>>> >>>>>>> Unfortunately I can't reproduce it. >>>>>>> >>>>>>> ================================================================== >>>>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>>>> >>>>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>>> Call trace: >>>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>>>> >>>>>>> Allocated by task 26657: >>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>>>> slab_alloc mm/slub.c:2726 [inline] >>>>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>>>> Freed by task 26657: >>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>>>> slab_free mm/slub.c:2961 [inline] >>>>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>>>> >>>>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>>>> The buggy address is located 64 bytes inside of >>>>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>>>> The buggy address belongs to the page: >>>>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>>>> flags: 0xfffc00000000100(slab) >>>>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>>>> page dumped because: kasan: bad access detected >>>>>>> page->mem_cgroup:ffff80005ac99980 >>>>>>> >>>>>>> Memory state around the buggy address: >>>>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>> ^ >>>>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>> ================================================================== >>>>>> >>>>>> Another one that looks related and doesn't have parts of stack traces missing: >>>>>> >>>>>> ================================================================== >>>>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>>>> >>>>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>> Call trace: >>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>> >>>>> It looks like we don't take the mmap_sem before calling find_vma() in >>>>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>>>> the race, with probably the test trying to unmap ranges in between. >>>> >>>> That indeed seems like a possible failure mode. The annoying thing is >>>> that we're not exactly in a position to take mmap_sem in >>>> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >>>> have to hold mmap_sem while iterating over all the memslots. >>>> >>>> How about the following (very lightly tested): >>>> >>>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >>>> index 962616fd4ddd..2006a79d5912 100644 >>>> --- a/arch/arm/kvm/mmu.c >>>> +++ b/arch/arm/kvm/mmu.c >>>> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>> int idx; >>>> >>>> idx = srcu_read_lock(&kvm->srcu); >>>> + down_read(¤t->mm->mmap_sem); >>>> spin_lock(&kvm->mmu_lock); >>>> >>>> slots = kvm_memslots(kvm); >>>> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>> stage2_unmap_memslot(kvm, memslot); >>>> >>>> spin_unlock(&kvm->mmu_lock); >>>> + up_read(¤t->mm->mmap_sem); >>>> srcu_read_unlock(&kvm->srcu, idx); >>>> } >>>> >>>> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>> * | memory region | >>>> * +--------------------------------------------+ >>>> */ >>>> + down_read(¤t->mm->mmap_sem); >>>> do { >>>> struct vm_area_struct *vma = find_vma(current->mm, hva); >>>> hva_t vm_start, vm_end; >>> >>> I have added the following hunk : >>> >>> @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>> pa += vm_start - vma->vm_start; >>> >>> /* IO region dirty page logging not allowed */ >>> - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) >>> - return -EINVAL; >>> + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { >>> + ret = -EINVAL; >>> + goto out; >>> + } >>> >>> ret = kvm_phys_addr_ioremap(kvm, gpa, pa, >>> vm_end - vm_start, >>> >> >> Ah, of course... Thanks for pointing that out, I'll fix it as I post a >> proper patch. >> >> Thanks, >> >> M. >> -- >> Jazz is not dead. It just smells funny... > > Hi, > > I've got this report again on linux-next 4c3c5cd02318 (Apr 5). > > It seems that it's still not fixed. > > Thanks! > > ================================================================== > BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 > Read of size 8 at addr ffff80004d53aae8 by task syz-executor/23095 > > CPU: 0 PID: 23095 Comm: syz-executor Not tainted > 4.11.0-rc5-next-20170405-xc2-09030-g4c3c5cd02318-dirty #4 The fixes went into -rc6. Can you please give it a go? Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update 2017-04-11 15:36 ` Marc Zyngier (?) @ 2017-04-11 15:41 ` Andrey Konovalov -1 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-04-11 15:41 UTC (permalink / raw) To: Marc Zyngier Cc: Suzuki K Poulose, Paolo Bonzini, Radim Krčmář, Christoffer Dall, Catalin Marinas, Will Deacon, Ingo Molnar, Michal Hocko, Christian Borntraeger, Suraj Jitindar Singh, Markus Elfring, Lorenzo Stoakes, kvm, linux-arm-kernel, kvmarm, LKML, Dmitry Vyukov, Kostya Serebryany, syzkaller On Tue, Apr 11, 2017 at 5:36 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > On 11/04/17 16:26, Andrey Konovalov wrote: >> On Tue, Mar 14, 2017 at 1:26 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >>> On 14/03/17 11:03, Suzuki K Poulose wrote: >>>> On 13/03/17 09:58, Marc Zyngier wrote: >>>>> On 10/03/17 18:37, Suzuki K Poulose wrote: >>>>>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>>>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>>>>> >>>>>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>>>>> >>>>>>>> Unfortunately I can't reproduce it. >>>>>>>> >>>>>>>> ================================================================== >>>>>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>>>>> >>>>>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>>>> Call trace: >>>>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>>>>> >>>>>>>> Allocated by task 26657: >>>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>>>>> slab_alloc mm/slub.c:2726 [inline] >>>>>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>>>>> Freed by task 26657: >>>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>>>>> slab_free mm/slub.c:2961 [inline] >>>>>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>>>>> >>>>>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>>>>> The buggy address is located 64 bytes inside of >>>>>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>>>>> The buggy address belongs to the page: >>>>>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>>>>> flags: 0xfffc00000000100(slab) >>>>>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>>>>> page dumped because: kasan: bad access detected >>>>>>>> page->mem_cgroup:ffff80005ac99980 >>>>>>>> >>>>>>>> Memory state around the buggy address: >>>>>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>>> ^ >>>>>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>>> ================================================================== >>>>>>> >>>>>>> Another one that looks related and doesn't have parts of stack traces missing: >>>>>>> >>>>>>> ================================================================== >>>>>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>>>>> >>>>>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>>> Call trace: >>>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>> >>>>>> It looks like we don't take the mmap_sem before calling find_vma() in >>>>>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>>>>> the race, with probably the test trying to unmap ranges in between. >>>>> >>>>> That indeed seems like a possible failure mode. The annoying thing is >>>>> that we're not exactly in a position to take mmap_sem in >>>>> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >>>>> have to hold mmap_sem while iterating over all the memslots. >>>>> >>>>> How about the following (very lightly tested): >>>>> >>>>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >>>>> index 962616fd4ddd..2006a79d5912 100644 >>>>> --- a/arch/arm/kvm/mmu.c >>>>> +++ b/arch/arm/kvm/mmu.c >>>>> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>>> int idx; >>>>> >>>>> idx = srcu_read_lock(&kvm->srcu); >>>>> + down_read(¤t->mm->mmap_sem); >>>>> spin_lock(&kvm->mmu_lock); >>>>> >>>>> slots = kvm_memslots(kvm); >>>>> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>>> stage2_unmap_memslot(kvm, memslot); >>>>> >>>>> spin_unlock(&kvm->mmu_lock); >>>>> + up_read(¤t->mm->mmap_sem); >>>>> srcu_read_unlock(&kvm->srcu, idx); >>>>> } >>>>> >>>>> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>>> * | memory region | >>>>> * +--------------------------------------------+ >>>>> */ >>>>> + down_read(¤t->mm->mmap_sem); >>>>> do { >>>>> struct vm_area_struct *vma = find_vma(current->mm, hva); >>>>> hva_t vm_start, vm_end; >>>> >>>> I have added the following hunk : >>>> >>>> @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>> pa += vm_start - vma->vm_start; >>>> >>>> /* IO region dirty page logging not allowed */ >>>> - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) >>>> - return -EINVAL; >>>> + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { >>>> + ret = -EINVAL; >>>> + goto out; >>>> + } >>>> >>>> ret = kvm_phys_addr_ioremap(kvm, gpa, pa, >>>> vm_end - vm_start, >>>> >>> >>> Ah, of course... Thanks for pointing that out, I'll fix it as I post a >>> proper patch. >>> >>> Thanks, >>> >>> M. >>> -- >>> Jazz is not dead. It just smells funny... >> >> Hi, >> >> I've got this report again on linux-next 4c3c5cd02318 (Apr 5). >> >> It seems that it's still not fixed. >> >> Thanks! >> >> ================================================================== >> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >> Read of size 8 at addr ffff80004d53aae8 by task syz-executor/23095 >> >> CPU: 0 PID: 23095 Comm: syz-executor Not tainted >> 4.11.0-rc5-next-20170405-xc2-09030-g4c3c5cd02318-dirty #4 > > The fixes went into -rc6. Can you please give it a go? Ah, OK, I assumed they were in the linux-next revision I used. Thanks! > > Thanks, > > M. > -- > Jazz is not dead. It just smells funny... > > -- > You received this message because you are subscribed to the Google Groups "syzkaller" group. > To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. ^ permalink raw reply [flat|nested] 23+ messages in thread
* kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-04-11 15:41 ` Andrey Konovalov 0 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-04-11 15:41 UTC (permalink / raw) To: linux-arm-kernel On Tue, Apr 11, 2017 at 5:36 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > On 11/04/17 16:26, Andrey Konovalov wrote: >> On Tue, Mar 14, 2017 at 1:26 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >>> On 14/03/17 11:03, Suzuki K Poulose wrote: >>>> On 13/03/17 09:58, Marc Zyngier wrote: >>>>> On 10/03/17 18:37, Suzuki K Poulose wrote: >>>>>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>>>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>>>>> >>>>>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>>>>> >>>>>>>> Unfortunately I can't reproduce it. >>>>>>>> >>>>>>>> ================================================================== >>>>>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>>>>> >>>>>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>>>> Call trace: >>>>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>>>>> >>>>>>>> Allocated by task 26657: >>>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>>>>> slab_alloc mm/slub.c:2726 [inline] >>>>>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>>>>> Freed by task 26657: >>>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>>>>> slab_free mm/slub.c:2961 [inline] >>>>>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>>>>> >>>>>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>>>>> The buggy address is located 64 bytes inside of >>>>>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>>>>> The buggy address belongs to the page: >>>>>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>>>>> flags: 0xfffc00000000100(slab) >>>>>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>>>>> page dumped because: kasan: bad access detected >>>>>>>> page->mem_cgroup:ffff80005ac99980 >>>>>>>> >>>>>>>> Memory state around the buggy address: >>>>>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>>> ^ >>>>>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>>> ================================================================== >>>>>>> >>>>>>> Another one that looks related and doesn't have parts of stack traces missing: >>>>>>> >>>>>>> ================================================================== >>>>>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>>>>> >>>>>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>>> Call trace: >>>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>> >>>>>> It looks like we don't take the mmap_sem before calling find_vma() in >>>>>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>>>>> the race, with probably the test trying to unmap ranges in between. >>>>> >>>>> That indeed seems like a possible failure mode. The annoying thing is >>>>> that we're not exactly in a position to take mmap_sem in >>>>> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >>>>> have to hold mmap_sem while iterating over all the memslots. >>>>> >>>>> How about the following (very lightly tested): >>>>> >>>>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >>>>> index 962616fd4ddd..2006a79d5912 100644 >>>>> --- a/arch/arm/kvm/mmu.c >>>>> +++ b/arch/arm/kvm/mmu.c >>>>> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>>> int idx; >>>>> >>>>> idx = srcu_read_lock(&kvm->srcu); >>>>> + down_read(¤t->mm->mmap_sem); >>>>> spin_lock(&kvm->mmu_lock); >>>>> >>>>> slots = kvm_memslots(kvm); >>>>> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>>> stage2_unmap_memslot(kvm, memslot); >>>>> >>>>> spin_unlock(&kvm->mmu_lock); >>>>> + up_read(¤t->mm->mmap_sem); >>>>> srcu_read_unlock(&kvm->srcu, idx); >>>>> } >>>>> >>>>> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>>> * | memory region | >>>>> * +--------------------------------------------+ >>>>> */ >>>>> + down_read(¤t->mm->mmap_sem); >>>>> do { >>>>> struct vm_area_struct *vma = find_vma(current->mm, hva); >>>>> hva_t vm_start, vm_end; >>>> >>>> I have added the following hunk : >>>> >>>> @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>> pa += vm_start - vma->vm_start; >>>> >>>> /* IO region dirty page logging not allowed */ >>>> - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) >>>> - return -EINVAL; >>>> + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { >>>> + ret = -EINVAL; >>>> + goto out; >>>> + } >>>> >>>> ret = kvm_phys_addr_ioremap(kvm, gpa, pa, >>>> vm_end - vm_start, >>>> >>> >>> Ah, of course... Thanks for pointing that out, I'll fix it as I post a >>> proper patch. >>> >>> Thanks, >>> >>> M. >>> -- >>> Jazz is not dead. It just smells funny... >> >> Hi, >> >> I've got this report again on linux-next 4c3c5cd02318 (Apr 5). >> >> It seems that it's still not fixed. >> >> Thanks! >> >> ================================================================== >> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >> Read of size 8 at addr ffff80004d53aae8 by task syz-executor/23095 >> >> CPU: 0 PID: 23095 Comm: syz-executor Not tainted >> 4.11.0-rc5-next-20170405-xc2-09030-g4c3c5cd02318-dirty #4 > > The fixes went into -rc6. Can you please give it a go? Ah, OK, I assumed they were in the linux-next revision I used. Thanks! > > Thanks, > > M. > -- > Jazz is not dead. It just smells funny... > > -- > You received this message because you are subscribed to the Google Groups "syzkaller" group. > To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/d/optout. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update @ 2017-04-11 15:41 ` Andrey Konovalov 0 siblings, 0 replies; 23+ messages in thread From: Andrey Konovalov @ 2017-04-11 15:41 UTC (permalink / raw) To: Marc Zyngier Cc: Kostya Serebryany, Michal Hocko, Lorenzo Stoakes, kvm, Catalin Marinas, Will Deacon, LKML, Markus Elfring, Christian Borntraeger, syzkaller, kvmarm, linux-arm-kernel, Suraj Jitindar Singh, Dmitry Vyukov, Paolo Bonzini, Ingo Molnar On Tue, Apr 11, 2017 at 5:36 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > On 11/04/17 16:26, Andrey Konovalov wrote: >> On Tue, Mar 14, 2017 at 1:26 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >>> On 14/03/17 11:03, Suzuki K Poulose wrote: >>>> On 13/03/17 09:58, Marc Zyngier wrote: >>>>> On 10/03/17 18:37, Suzuki K Poulose wrote: >>>>>> On 10/03/17 15:50, Andrey Konovalov wrote: >>>>>>> On Fri, Mar 10, 2017 at 2:38 PM, Andrey Konovalov <andreyknvl@google.com> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I've got the following error report while fuzzing the kernel with syzkaller. >>>>>>>> >>>>>>>> On linux-next commit 56b8bad5e066c23e8fa273ef5fba50bd3da2ace8 (Mar 8). >>>>>>>> >>>>>>>> Unfortunately I can't reproduce it. >>>>>>>> >>>>>>>> ================================================================== >>>>>>>> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>>> Read of size 8 at addr ffff80003b9a2040 by task syz-executor/26615 >>>>>>>> >>>>>>>> CPU: 1 PID: 26615 Comm: syz-executor Not tainted >>>>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>>>> Call trace: >>>>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>>>> [<ffff200008383f64>] vmacache_update+0x114/0x118 mm/vmacache.c:63 >>>>>>>> [<ffff2000083a9000>] find_vma+0xf8/0x150 mm/mmap.c:2124 >>>>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>>>> [<ffff2000080c2920>] __kvm_set_memory_region+0x3d8/0x12b8 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1026 >>>>>>>> [<ffff2000080c3838>] kvm_set_memory_region+0x38/0x58 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1075 >>>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl_set_memory_region >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:1087 [inline] >>>>>>>> [<ffff2000080c747c>] kvm_vm_ioctl+0xb94/0x1308 >>>>>>>> arch/arm64/kvm/../../../virt/kvm/kvm_main.c:2960 >>>>>>>> [<ffff20000848f928>] vfs_ioctl fs/ioctl.c:45 [inline] >>>>>>>> [<ffff20000848f928>] do_vfs_ioctl+0x128/0xfc0 fs/ioctl.c:685 >>>>>>>> [<ffff200008490868>] SYSC_ioctl fs/ioctl.c:700 [inline] >>>>>>>> [<ffff200008490868>] SyS_ioctl+0xa8/0xb8 fs/ioctl.c:691 >>>>>>>> [<ffff200008083f70>] el0_svc_naked+0x24/0x28 >>>>>>>> >>>>>>>> Allocated by task 26657: >>>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>>> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:619 >>>>>>>> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:557 >>>>>>>> slab_post_alloc_hook mm/slab.h:456 [inline] >>>>>>>> slab_alloc_node mm/slub.c:2718 [inline] >>>>>>>> slab_alloc mm/slub.c:2726 [inline] >>>>>>>> kmem_cache_alloc+0x144/0x230 mm/slub.c:2731 >>>>>>>> __split_vma+0x118/0x608 mm/mmap.c:2515 >>>>>>>> do_munmap+0x194/0x9b0 mm/mmap.c:2636 >>>>>>>> Freed by task 26657: >>>>>>>> save_stack_trace_tsk+0x0/0x330 arch/arm64/kernel/stacktrace.c:133 >>>>>>>> save_stack_trace+0x20/0x30 arch/arm64/kernel/stacktrace.c:216 >>>>>>>> save_stack mm/kasan/kasan.c:515 [inline] >>>>>>>> set_track mm/kasan/kasan.c:527 [inline] >>>>>>>> kasan_slab_free+0x84/0x198 mm/kasan/kasan.c:592 >>>>>>>> slab_free_hook mm/slub.c:1357 [inline] >>>>>>>> slab_free_freelist_hook mm/slub.c:1379 [inline] >>>>>>>> slab_free mm/slub.c:2961 [inline] >>>>>>>> kmem_cache_free+0x80/0x258 mm/slub.c:2983 >>>>>>>> __vma_adjust+0x6b0/0xf mm/mmap.c:890] el0_svc_naked+0x24/0x28 >>>>>>>> >>>>>>>> The buggy address belongs to the object at ffff80003b9a2000 >>>>>>>> which belongs to the cache vm_area_struct(647:session-6.scope) of size 184 >>>>>>>> The buggy address is located 64 bytes inside of >>>>>>>> 184-byte region [ffff80003b9a2000, ffff80003b9a20b8) >>>>>>>> The buggy address belongs to the page: >>>>>>>> page:ffff7e0000ee6880 count:1 mapcount:0 mapping: (null) index:0x0 >>>>>>>> flags: 0xfffc00000000100(slab) >>>>>>>> raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010 >>>>>>>> raw: 0000000000000000 0000000c00000001 ffff80005a5cc600 ffff80005ac99980 >>>>>>>> page dumped because: kasan: bad access detected >>>>>>>> page->mem_cgroup:ffff80005ac99980 >>>>>>>> >>>>>>>> Memory state around the buggy address: >>>>>>>> ffff80003b9a1f00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>>> ffff80003b9a1f80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >>>>>>>>> ffff80003b9a2000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>>> ^ >>>>>>>> ffff80003b9a2080: fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc fb >>>>>>>> ffff80003b9a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >>>>>>>> ================================================================== >>>>>>> >>>>>>> Another one that looks related and doesn't have parts of stack traces missing: >>>>>>> >>>>>>> ================================================================== >>>>>>> BUG: KASAN: use-after-free in find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>>> Read of size 8 at addr ffff800031a03e90 by task syz-executor/4360 >>>>>>> >>>>>>> CPU: 2 PID: 4360 Comm: syz-executor Not tainted >>>>>>> 4.11.0-rc1-next-20170308-xc2-dirty #3 >>>>>>> Hardware name: Hardkernel ODROID-C2 (DT) >>>>>>> Call trace: >>>>>>> [<ffff20000808fbb0>] dump_backtrace+0x0/0x440 arch/arm64/kernel/traps.c:505 >>>>>>> [<ffff200008090010>] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:228 >>>>>>> [<ffff2000088e9578>] __dump_stack lib/dump_stack.c:16 [inline] >>>>>>> [<ffff2000088e9578>] dump_stack+0x110/0x168 lib/dump_stack.c:52 >>>>>>> [<ffff200008414018>] print_address_description+0x60/0x248 mm/kasan/report.c:250 >>>>>>> [<ffff2000084142e8>] kasan_report_error+0xe8/0x250 mm/kasan/report.c:349 >>>>>>> [<ffff200008414564>] kasan_report mm/kasan/report.c:372 [inline] >>>>>>> [<ffff200008414564>] __asan_report_load8_noabort+0x3c/0x48 mm/kasan/report.c:393 >>>>>>> [<ffff2000083a9048>] find_vma+0x140/0x150 mm/mmap.c:2114 >>>>>>> [<ffff2000080dc19c>] kvm_arch_prepare_memory_region+0x2ac/0x488 >>>>>>> arch/arm64/kvm/../../../arch/arm/kvm/mmu.c:1817 >>>>>> >>>>>> It looks like we don't take the mmap_sem before calling find_vma() in >>>>>> stage2_unmap_memslot() and in kvm_arch_prepare_memory_region(), which is causing >>>>>> the race, with probably the test trying to unmap ranges in between. >>>>> >>>>> That indeed seems like a possible failure mode. The annoying thing is >>>>> that we're not exactly in a position to take mmap_sem in >>>>> stage2_unmap_memslot, since we hold the kvm->mmu_lock spinlock. We may >>>>> have to hold mmap_sem while iterating over all the memslots. >>>>> >>>>> How about the following (very lightly tested): >>>>> >>>>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c >>>>> index 962616fd4ddd..2006a79d5912 100644 >>>>> --- a/arch/arm/kvm/mmu.c >>>>> +++ b/arch/arm/kvm/mmu.c >>>>> @@ -803,6 +803,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>>> int idx; >>>>> >>>>> idx = srcu_read_lock(&kvm->srcu); >>>>> + down_read(¤t->mm->mmap_sem); >>>>> spin_lock(&kvm->mmu_lock); >>>>> >>>>> slots = kvm_memslots(kvm); >>>>> @@ -810,6 +811,7 @@ void stage2_unmap_vm(struct kvm *kvm) >>>>> stage2_unmap_memslot(kvm, memslot); >>>>> >>>>> spin_unlock(&kvm->mmu_lock); >>>>> + up_read(¤t->mm->mmap_sem); >>>>> srcu_read_unlock(&kvm->srcu, idx); >>>>> } >>>>> >>>>> @@ -1813,6 +1815,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>>> * | memory region | >>>>> * +--------------------------------------------+ >>>>> */ >>>>> + down_read(¤t->mm->mmap_sem); >>>>> do { >>>>> struct vm_area_struct *vma = find_vma(current->mm, hva); >>>>> hva_t vm_start, vm_end; >>>> >>>> I have added the following hunk : >>>> >>>> @@ -1844,8 +1847,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>> pa += vm_start - vma->vm_start; >>>> >>>> /* IO region dirty page logging not allowed */ >>>> - if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) >>>> - return -EINVAL; >>>> + if (memslot->flags & KVM_MEM_LOG_DIRTY_PAGES) { >>>> + ret = -EINVAL; >>>> + goto out; >>>> + } >>>> >>>> ret = kvm_phys_addr_ioremap(kvm, gpa, pa, >>>> vm_end - vm_start, >>>> >>> >>> Ah, of course... Thanks for pointing that out, I'll fix it as I post a >>> proper patch. >>> >>> Thanks, >>> >>> M. >>> -- >>> Jazz is not dead. It just smells funny... >> >> Hi, >> >> I've got this report again on linux-next 4c3c5cd02318 (Apr 5). >> >> It seems that it's still not fixed. >> >> Thanks! >> >> ================================================================== >> BUG: KASAN: use-after-free in vmacache_update+0x114/0x118 mm/vmacache.c:63 >> Read of size 8 at addr ffff80004d53aae8 by task syz-executor/23095 >> >> CPU: 0 PID: 23095 Comm: syz-executor Not tainted >> 4.11.0-rc5-next-20170405-xc2-09030-g4c3c5cd02318-dirty #4 > > The fixes went into -rc6. Can you please give it a go? Ah, OK, I assumed they were in the linux-next revision I used. Thanks! > > Thanks, > > M. > -- > Jazz is not dead. It just smells funny... > > -- > You received this message because you are subscribed to the Google Groups "syzkaller" group. > To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2017-04-11 15:41 UTC | newest] Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-03-10 13:38 kvm/arm64: use-after-free in kvm_vm_ioctl/vmacache_update Andrey Konovalov 2017-03-10 13:38 ` Andrey Konovalov 2017-03-10 15:50 ` Andrey Konovalov 2017-03-10 15:50 ` Andrey Konovalov 2017-03-10 18:37 ` Suzuki K Poulose 2017-03-10 18:37 ` Suzuki K Poulose 2017-03-10 18:37 ` Suzuki K Poulose 2017-03-13 9:58 ` Marc Zyngier 2017-03-13 9:58 ` Marc Zyngier 2017-03-13 9:58 ` Marc Zyngier 2017-03-14 11:03 ` Suzuki K Poulose 2017-03-14 11:03 ` Suzuki K Poulose 2017-03-14 12:26 ` Marc Zyngier 2017-03-14 12:26 ` Marc Zyngier 2017-03-14 12:26 ` Marc Zyngier 2017-04-11 15:26 ` Andrey Konovalov 2017-04-11 15:26 ` Andrey Konovalov 2017-04-11 15:26 ` Andrey Konovalov 2017-04-11 15:36 ` Marc Zyngier 2017-04-11 15:36 ` Marc Zyngier 2017-04-11 15:41 ` Andrey Konovalov 2017-04-11 15:41 ` Andrey Konovalov 2017-04-11 15:41 ` Andrey Konovalov
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.