* [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu
@ 2021-08-07 13:44 syzbot
2021-08-07 20:50 ` syzbot
2021-08-09 19:22 ` Pavel Skripkin
0 siblings, 2 replies; 8+ messages in thread
From: syzbot @ 2021-08-07 13:44 UTC (permalink / raw)
To: coreteam, davem, fw, kadlec, kuba, linux-kernel, netdev,
netfilter-devel, pablo, syzkaller-bugs
Hello,
syzbot found the following issue on:
HEAD commit: 894d6f401b21 Merge tag 'spi-fix-v5.14-rc4' of git://git.ke..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=17c622fa300000
kernel config: https://syzkaller.appspot.com/x/.config?x=343fd21f6f4da2d6
dashboard link: https://syzkaller.appspot.com/bug?extid=649e339fa6658ee623d3
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.1
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=110319aa300000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1142fac9d00000
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+649e339fa6658ee623d3@syzkaller.appspotmail.com
==================================================================
BUG: KASAN: use-after-free in instrument_atomic_read_write include/linux/instrumented.h:101 [inline]
BUG: KASAN: use-after-free in atomic_dec_and_test include/asm-generic/atomic-instrumented.h:542 [inline]
BUG: KASAN: use-after-free in nf_conntrack_put include/linux/netfilter/nf_conntrack_common.h:33 [inline]
BUG: KASAN: use-after-free in nf_ct_put include/net/netfilter/nf_conntrack.h:176 [inline]
BUG: KASAN: use-after-free in nft_ct_tmpl_put_pcpu+0x135/0x1e0 net/netfilter/nft_ct.c:356
Write of size 4 at addr ffff88803d750400 by task syz-executor409/9789
CPU: 0 PID: 9789 Comm: syz-executor409 Not tainted 5.14.0-rc4-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:105
print_address_description.constprop.0.cold+0x6c/0x309 mm/kasan/report.c:233
__kasan_report mm/kasan/report.c:419 [inline]
kasan_report.cold+0x83/0xdf mm/kasan/report.c:436
check_region_inline mm/kasan/generic.c:183 [inline]
kasan_check_range+0x13d/0x180 mm/kasan/generic.c:189
instrument_atomic_read_write include/linux/instrumented.h:101 [inline]
atomic_dec_and_test include/asm-generic/atomic-instrumented.h:542 [inline]
nf_conntrack_put include/linux/netfilter/nf_conntrack_common.h:33 [inline]
nf_ct_put include/net/netfilter/nf_conntrack.h:176 [inline]
nft_ct_tmpl_put_pcpu+0x135/0x1e0 net/netfilter/nft_ct.c:356
__nft_ct_set_destroy net/netfilter/nft_ct.c:529 [inline]
__nft_ct_set_destroy net/netfilter/nft_ct.c:518 [inline]
nft_ct_set_init+0x41e/0x750 net/netfilter/nft_ct.c:614
nf_tables_newexpr net/netfilter/nf_tables_api.c:2742 [inline]
nft_expr_init+0x145/0x2d0 net/netfilter/nf_tables_api.c:2780
nft_set_elem_expr_alloc+0x27/0x280 net/netfilter/nf_tables_api.c:5284
nf_tables_newset+0x208a/0x32f0 net/netfilter/nf_tables_api.c:4389
nfnetlink_rcv_batch+0x1710/0x25f0 net/netfilter/nfnetlink.c:513
nfnetlink_rcv_skb_batch net/netfilter/nfnetlink.c:634 [inline]
nfnetlink_rcv+0x3af/0x420 net/netfilter/nfnetlink.c:652
netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
sock_sendmsg_nosec net/socket.c:703 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:723
____sys_sendmsg+0x6e8/0x810 net/socket.c:2392
___sys_sendmsg+0xf3/0x170 net/socket.c:2446
__sys_sendmsg+0xe5/0x1b0 net/socket.c:2475
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x444819
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 41 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c4 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fff0ba410d8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 0000000000444819
RDX: 0000000000000000 RSI: 00000000200000c0 RDI: 0000000000000003
RBP: 0000000000000000 R08: 00000000000f4240 R09: 00000000000f4240
R10: 00007fff0ba40b60 R11: 0000000000000246 R12: 00007fff0ba41100
R13: 00000000000f4240 R14: 000000000003754e R15: 00007fff0ba410f4
Allocated by task 9789:
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:38
kasan_set_track mm/kasan/common.c:46 [inline]
set_alloc_info mm/kasan/common.c:434 [inline]
____kasan_kmalloc mm/kasan/common.c:513 [inline]
____kasan_kmalloc mm/kasan/common.c:472 [inline]
__kasan_kmalloc+0x9b/0xd0 mm/kasan/common.c:522
kmalloc include/linux/slab.h:591 [inline]
kzalloc include/linux/slab.h:721 [inline]
nf_ct_tmpl_alloc+0x8d/0x270 net/netfilter/nf_conntrack_core.c:569
nft_ct_tmpl_alloc_pcpu net/netfilter/nft_ct.c:371 [inline]
nft_ct_set_init+0x4d6/0x750 net/netfilter/nft_ct.c:567
nf_tables_newexpr net/netfilter/nf_tables_api.c:2742 [inline]
nft_expr_init+0x145/0x2d0 net/netfilter/nf_tables_api.c:2780
nft_set_elem_expr_alloc+0x27/0x280 net/netfilter/nf_tables_api.c:5284
nf_tables_newset+0x208a/0x32f0 net/netfilter/nf_tables_api.c:4389
nfnetlink_rcv_batch+0x1710/0x25f0 net/netfilter/nfnetlink.c:513
nfnetlink_rcv_skb_batch net/netfilter/nfnetlink.c:634 [inline]
nfnetlink_rcv+0x3af/0x420 net/netfilter/nfnetlink.c:652
netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
sock_sendmsg_nosec net/socket.c:703 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:723
____sys_sendmsg+0x6e8/0x810 net/socket.c:2392
___sys_sendmsg+0xf3/0x170 net/socket.c:2446
__sys_sendmsg+0xe5/0x1b0 net/socket.c:2475
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
Freed by task 9788:
kasan_save_stack+0x1b/0x40 mm/kasan/common.c:38
kasan_set_track+0x1c/0x30 mm/kasan/common.c:46
kasan_set_free_info+0x20/0x30 mm/kasan/generic.c:360
____kasan_slab_free mm/kasan/common.c:366 [inline]
____kasan_slab_free mm/kasan/common.c:328 [inline]
__kasan_slab_free+0xfb/0x130 mm/kasan/common.c:374
kasan_slab_free include/linux/kasan.h:230 [inline]
slab_free_hook mm/slub.c:1625 [inline]
slab_free_freelist_hook+0xdf/0x240 mm/slub.c:1650
slab_free mm/slub.c:3210 [inline]
kfree+0xe4/0x530 mm/slub.c:4264
nf_ct_tmpl_free net/netfilter/nf_conntrack_core.c:590 [inline]
destroy_conntrack+0x222/0x2c0 net/netfilter/nf_conntrack_core.c:613
nf_conntrack_destroy+0xab/0x230 net/netfilter/core.c:677
nf_conntrack_put include/linux/netfilter/nf_conntrack_common.h:34 [inline]
nf_ct_put include/net/netfilter/nf_conntrack.h:176 [inline]
nft_ct_tmpl_put_pcpu+0x15e/0x1e0 net/netfilter/nft_ct.c:356
__nft_ct_set_destroy net/netfilter/nft_ct.c:529 [inline]
__nft_ct_set_destroy net/netfilter/nft_ct.c:518 [inline]
nft_ct_set_init+0x41e/0x750 net/netfilter/nft_ct.c:614
nf_tables_newexpr net/netfilter/nf_tables_api.c:2742 [inline]
nft_expr_init+0x145/0x2d0 net/netfilter/nf_tables_api.c:2780
nft_set_elem_expr_alloc+0x27/0x280 net/netfilter/nf_tables_api.c:5284
nf_tables_newset+0x208a/0x32f0 net/netfilter/nf_tables_api.c:4389
nfnetlink_rcv_batch+0x1710/0x25f0 net/netfilter/nfnetlink.c:513
nfnetlink_rcv_skb_batch net/netfilter/nfnetlink.c:634 [inline]
nfnetlink_rcv+0x3af/0x420 net/netfilter/nfnetlink.c:652
netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
sock_sendmsg_nosec net/socket.c:703 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:723
____sys_sendmsg+0x6e8/0x810 net/socket.c:2392
___sys_sendmsg+0xf3/0x170 net/socket.c:2446
__sys_sendmsg+0xe5/0x1b0 net/socket.c:2475
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
The buggy address belongs to the object at ffff88803d750400
which belongs to the cache kmalloc-512 of size 512
The buggy address is located 0 bytes inside of
512-byte region [ffff88803d750400, ffff88803d750600)
The buggy address belongs to the page:
page:ffffea0000f5d400 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x3d750
head:ffffea0000f5d400 order:2 compound_mapcount:0 compound_pincount:0
flags: 0xfff00000010200(slab|head|node=0|zone=1|lastcpupid=0x7ff)
raw: 00fff00000010200 0000000000000000 dead000000000122 ffff888010841c80
raw: 0000000000000000 0000000080100010 00000001ffffffff 0000000000000000
page dumped because: kasan: bad access detected
page_owner tracks the page as allocated
page last allocated via order 2, migratetype Unmovable, gfp_mask 0xd20c0(__GFP_IO|__GFP_FS|__GFP_NOWARN|__GFP_NORETRY|__GFP_COMP|__GFP_NOMEMALLOC), pid 9789, ts 226704064982, free_ts 0
prep_new_page mm/page_alloc.c:2436 [inline]
get_page_from_freelist+0xa72/0x2f80 mm/page_alloc.c:4169
__alloc_pages+0x1b2/0x500 mm/page_alloc.c:5391
alloc_pages+0x18c/0x2a0 mm/mempolicy.c:2244
alloc_slab_page mm/slub.c:1688 [inline]
allocate_slab+0x32e/0x4b0 mm/slub.c:1828
new_slab mm/slub.c:1891 [inline]
new_slab_objects mm/slub.c:2637 [inline]
___slab_alloc+0x4ba/0x820 mm/slub.c:2800
__slab_alloc.constprop.0+0xa7/0xf0 mm/slub.c:2840
slab_alloc_node mm/slub.c:2922 [inline]
slab_alloc mm/slub.c:2964 [inline]
kmem_cache_alloc_trace+0x30f/0x3c0 mm/slub.c:2981
kmalloc include/linux/slab.h:591 [inline]
kzalloc include/linux/slab.h:721 [inline]
nf_ct_tmpl_alloc+0x8d/0x270 net/netfilter/nf_conntrack_core.c:569
nft_ct_tmpl_alloc_pcpu net/netfilter/nft_ct.c:371 [inline]
nft_ct_set_init+0x4d6/0x750 net/netfilter/nft_ct.c:567
nf_tables_newexpr net/netfilter/nf_tables_api.c:2742 [inline]
nft_expr_init+0x145/0x2d0 net/netfilter/nf_tables_api.c:2780
nft_set_elem_expr_alloc+0x27/0x280 net/netfilter/nf_tables_api.c:5284
nf_tables_newset+0x208a/0x32f0 net/netfilter/nf_tables_api.c:4389
nfnetlink_rcv_batch+0x1710/0x25f0 net/netfilter/nfnetlink.c:513
nfnetlink_rcv_skb_batch net/netfilter/nfnetlink.c:634 [inline]
nfnetlink_rcv+0x3af/0x420 net/netfilter/nfnetlink.c:652
netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline]
netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340
netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929
page_owner free stack trace missing
Memory state around the buggy address:
ffff88803d750300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
ffff88803d750380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>ffff88803d750400: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff88803d750480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff88803d750500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu 2021-08-07 13:44 [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu syzbot @ 2021-08-07 20:50 ` syzbot 2021-08-09 19:22 ` Pavel Skripkin 1 sibling, 0 replies; 8+ messages in thread From: syzbot @ 2021-08-07 20:50 UTC (permalink / raw) To: coreteam, davem, fw, kadlec, kuba, linux-kernel, netdev, netfilter-devel, pablo, syzkaller-bugs syzbot has bisected this issue to: commit 65038428b2c6c5be79d3f78a6b79c0cdc3a58a41 Author: Pablo Neira Ayuso <pablo@netfilter.org> Date: Tue Mar 17 13:13:46 2020 +0000 netfilter: nf_tables: allow to specify stateful expression in set definition bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1145a881300000 start commit: 894d6f401b21 Merge tag 'spi-fix-v5.14-rc4' of git://git.ke.. git tree: upstream final oops: https://syzkaller.appspot.com/x/report.txt?x=1345a881300000 console output: https://syzkaller.appspot.com/x/log.txt?x=1545a881300000 kernel config: https://syzkaller.appspot.com/x/.config?x=343fd21f6f4da2d6 dashboard link: https://syzkaller.appspot.com/bug?extid=649e339fa6658ee623d3 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=110319aa300000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1142fac9d00000 Reported-by: syzbot+649e339fa6658ee623d3@syzkaller.appspotmail.com Fixes: 65038428b2c6 ("netfilter: nf_tables: allow to specify stateful expression in set definition") For information about bisection process see: https://goo.gl/tpsmEJ#bisection ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu 2021-08-07 13:44 [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu syzbot 2021-08-07 20:50 ` syzbot @ 2021-08-09 19:22 ` Pavel Skripkin 2021-08-09 20:39 ` Florian Westphal 2021-08-10 2:59 ` syzbot 1 sibling, 2 replies; 8+ messages in thread From: Pavel Skripkin @ 2021-08-09 19:22 UTC (permalink / raw) To: syzbot, coreteam, davem, fw, kadlec, kuba, linux-kernel, netdev, netfilter-devel, pablo, syzkaller-bugs [-- Attachment #1: Type: text/plain, Size: 9453 bytes --] On 8/7/21 4:44 PM, syzbot wrote: > Hello, > > syzbot found the following issue on: > > HEAD commit: 894d6f401b21 Merge tag 'spi-fix-v5.14-rc4' of git://git.ke.. > git tree: upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=17c622fa300000 > kernel config: https://syzkaller.appspot.com/x/.config?x=343fd21f6f4da2d6 > dashboard link: https://syzkaller.appspot.com/bug?extid=649e339fa6658ee623d3 > compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.1 > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=110319aa300000 > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1142fac9d00000 > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > Reported-by: syzbot+649e339fa6658ee623d3@syzkaller.appspotmail.com > > ================================================================== > BUG: KASAN: use-after-free in instrument_atomic_read_write include/linux/instrumented.h:101 [inline] > BUG: KASAN: use-after-free in atomic_dec_and_test include/asm-generic/atomic-instrumented.h:542 [inline] > BUG: KASAN: use-after-free in nf_conntrack_put include/linux/netfilter/nf_conntrack_common.h:33 [inline] > BUG: KASAN: use-after-free in nf_ct_put include/net/netfilter/nf_conntrack.h:176 [inline] > BUG: KASAN: use-after-free in nft_ct_tmpl_put_pcpu+0x135/0x1e0 net/netfilter/nft_ct.c:356 > Write of size 4 at addr ffff88803d750400 by task syz-executor409/9789 > > CPU: 0 PID: 9789 Comm: syz-executor409 Not tainted 5.14.0-rc4-syzkaller #0 > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 > Call Trace: > __dump_stack lib/dump_stack.c:88 [inline] > dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:105 > print_address_description.constprop.0.cold+0x6c/0x309 mm/kasan/report.c:233 > __kasan_report mm/kasan/report.c:419 [inline] > kasan_report.cold+0x83/0xdf mm/kasan/report.c:436 > check_region_inline mm/kasan/generic.c:183 [inline] > kasan_check_range+0x13d/0x180 mm/kasan/generic.c:189 > instrument_atomic_read_write include/linux/instrumented.h:101 [inline] > atomic_dec_and_test include/asm-generic/atomic-instrumented.h:542 [inline] > nf_conntrack_put include/linux/netfilter/nf_conntrack_common.h:33 [inline] > nf_ct_put include/net/netfilter/nf_conntrack.h:176 [inline] > nft_ct_tmpl_put_pcpu+0x135/0x1e0 net/netfilter/nft_ct.c:356 (*) > __nft_ct_set_destroy net/netfilter/nft_ct.c:529 [inline] > __nft_ct_set_destroy net/netfilter/nft_ct.c:518 [inline] > nft_ct_set_init+0x41e/0x750 net/netfilter/nft_ct.c:614 > nf_tables_newexpr net/netfilter/nf_tables_api.c:2742 [inline] > nft_expr_init+0x145/0x2d0 net/netfilter/nf_tables_api.c:2780 > nft_set_elem_expr_alloc+0x27/0x280 net/netfilter/nf_tables_api.c:5284 > nf_tables_newset+0x208a/0x32f0 net/netfilter/nf_tables_api.c:4389 > nfnetlink_rcv_batch+0x1710/0x25f0 net/netfilter/nfnetlink.c:513 > nfnetlink_rcv_skb_batch net/netfilter/nfnetlink.c:634 [inline] > nfnetlink_rcv+0x3af/0x420 net/netfilter/nfnetlink.c:652 > netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline] > netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340 > netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929 > sock_sendmsg_nosec net/socket.c:703 [inline] > sock_sendmsg+0xcf/0x120 net/socket.c:723 > ____sys_sendmsg+0x6e8/0x810 net/socket.c:2392 > ___sys_sendmsg+0xf3/0x170 net/socket.c:2446 > __sys_sendmsg+0xe5/0x1b0 net/socket.c:2475 > do_syscall_x64 arch/x86/entry/common.c:50 [inline] > do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 > entry_SYSCALL_64_after_hwframe+0x44/0xae > RIP: 0033:0x444819 [snip] > Freed by task 9788: > kasan_save_stack+0x1b/0x40 mm/kasan/common.c:38 > kasan_set_track+0x1c/0x30 mm/kasan/common.c:46 > kasan_set_free_info+0x20/0x30 mm/kasan/generic.c:360 > ____kasan_slab_free mm/kasan/common.c:366 [inline] > ____kasan_slab_free mm/kasan/common.c:328 [inline] > __kasan_slab_free+0xfb/0x130 mm/kasan/common.c:374 > kasan_slab_free include/linux/kasan.h:230 [inline] > slab_free_hook mm/slub.c:1625 [inline] > slab_free_freelist_hook+0xdf/0x240 mm/slub.c:1650 > slab_free mm/slub.c:3210 [inline] > kfree+0xe4/0x530 mm/slub.c:4264 > nf_ct_tmpl_free net/netfilter/nf_conntrack_core.c:590 [inline] > destroy_conntrack+0x222/0x2c0 net/netfilter/nf_conntrack_core.c:613 > nf_conntrack_destroy+0xab/0x230 net/netfilter/core.c:677 > nf_conntrack_put include/linux/netfilter/nf_conntrack_common.h:34 [inline] > nf_ct_put include/net/netfilter/nf_conntrack.h:176 [inline] > nft_ct_tmpl_put_pcpu+0x15e/0x1e0 net/netfilter/nft_ct.c:356 I think, there a missing lock in this function: for_each_possible_cpu(cpu) { ct = per_cpu(nft_ct_pcpu_template, cpu); if (!ct) break; nf_ct_put(ct); per_cpu(nft_ct_pcpu_template, cpu) = NULL; } Syzbot hit a UAF in nft_ct_tmpl_put_pcpu() (*), but freed template should be NULL. So I suspect following scenario: CPU0: CPU1: = per_cpu() = per_cpu() nf_ct_put per_cpu = NULL nf_ct_put() * UAF * #syz test git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master With regards, Pavel Skripkin > __nft_ct_set_destroy net/netfilter/nft_ct.c:529 [inline] > __nft_ct_set_destroy net/netfilter/nft_ct.c:518 [inline] > nft_ct_set_init+0x41e/0x750 net/netfilter/nft_ct.c:614 > nf_tables_newexpr net/netfilter/nf_tables_api.c:2742 [inline] > nft_expr_init+0x145/0x2d0 net/netfilter/nf_tables_api.c:2780 > nft_set_elem_expr_alloc+0x27/0x280 net/netfilter/nf_tables_api.c:5284 > nf_tables_newset+0x208a/0x32f0 net/netfilter/nf_tables_api.c:4389 > nfnetlink_rcv_batch+0x1710/0x25f0 net/netfilter/nfnetlink.c:513 > nfnetlink_rcv_skb_batch net/netfilter/nfnetlink.c:634 [inline] > nfnetlink_rcv+0x3af/0x420 net/netfilter/nfnetlink.c:652 > netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline] > netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340 > netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929 > sock_sendmsg_nosec net/socket.c:703 [inline] > sock_sendmsg+0xcf/0x120 net/socket.c:723 > ____sys_sendmsg+0x6e8/0x810 net/socket.c:2392 > ___sys_sendmsg+0xf3/0x170 net/socket.c:2446 > __sys_sendmsg+0xe5/0x1b0 net/socket.c:2475 > do_syscall_x64 arch/x86/entry/common.c:50 [inline] > do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 > entry_SYSCALL_64_after_hwframe+0x44/0xae > > The buggy address belongs to the object at ffff88803d750400 > which belongs to the cache kmalloc-512 of size 512 > The buggy address is located 0 bytes inside of > 512-byte region [ffff88803d750400, ffff88803d750600) > The buggy address belongs to the page: > page:ffffea0000f5d400 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x3d750 > head:ffffea0000f5d400 order:2 compound_mapcount:0 compound_pincount:0 > flags: 0xfff00000010200(slab|head|node=0|zone=1|lastcpupid=0x7ff) > raw: 00fff00000010200 0000000000000000 dead000000000122 ffff888010841c80 > raw: 0000000000000000 0000000080100010 00000001ffffffff 0000000000000000 > page dumped because: kasan: bad access detected > page_owner tracks the page as allocated > page last allocated via order 2, migratetype Unmovable, gfp_mask 0xd20c0(__GFP_IO|__GFP_FS|__GFP_NOWARN|__GFP_NORETRY|__GFP_COMP|__GFP_NOMEMALLOC), pid 9789, ts 226704064982, free_ts 0 > prep_new_page mm/page_alloc.c:2436 [inline] > get_page_from_freelist+0xa72/0x2f80 mm/page_alloc.c:4169 > __alloc_pages+0x1b2/0x500 mm/page_alloc.c:5391 > alloc_pages+0x18c/0x2a0 mm/mempolicy.c:2244 > alloc_slab_page mm/slub.c:1688 [inline] > allocate_slab+0x32e/0x4b0 mm/slub.c:1828 > new_slab mm/slub.c:1891 [inline] > new_slab_objects mm/slub.c:2637 [inline] > ___slab_alloc+0x4ba/0x820 mm/slub.c:2800 > __slab_alloc.constprop.0+0xa7/0xf0 mm/slub.c:2840 > slab_alloc_node mm/slub.c:2922 [inline] > slab_alloc mm/slub.c:2964 [inline] > kmem_cache_alloc_trace+0x30f/0x3c0 mm/slub.c:2981 > kmalloc include/linux/slab.h:591 [inline] > kzalloc include/linux/slab.h:721 [inline] > nf_ct_tmpl_alloc+0x8d/0x270 net/netfilter/nf_conntrack_core.c:569 > nft_ct_tmpl_alloc_pcpu net/netfilter/nft_ct.c:371 [inline] > nft_ct_set_init+0x4d6/0x750 net/netfilter/nft_ct.c:567 > nf_tables_newexpr net/netfilter/nf_tables_api.c:2742 [inline] > nft_expr_init+0x145/0x2d0 net/netfilter/nf_tables_api.c:2780 > nft_set_elem_expr_alloc+0x27/0x280 net/netfilter/nf_tables_api.c:5284 > nf_tables_newset+0x208a/0x32f0 net/netfilter/nf_tables_api.c:4389 > nfnetlink_rcv_batch+0x1710/0x25f0 net/netfilter/nfnetlink.c:513 > nfnetlink_rcv_skb_batch net/netfilter/nfnetlink.c:634 [inline] > nfnetlink_rcv+0x3af/0x420 net/netfilter/nfnetlink.c:652 > netlink_unicast_kernel net/netlink/af_netlink.c:1314 [inline] > netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1340 > netlink_sendmsg+0x86d/0xdb0 net/netlink/af_netlink.c:1929 > page_owner free stack trace missing > > Memory state around the buggy address: > ffff88803d750300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > ffff88803d750380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc >>ffff88803d750400: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb > ^ > ffff88803d750480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb > ffff88803d750500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb > ================================================================== > [-- Attachment #2: 0001-netfilter-add-mutex-to-protect-nft_ct_pcpu_template.patch --] [-- Type: text/x-patch, Size: 2541 bytes --] From 39275bfc1bac5758752ba2bf4df68e244590589b Mon Sep 17 00:00:00 2001 From: Pavel Skripkin <paskripkin@gmail.com> Date: Mon, 9 Aug 2021 22:13:44 +0300 Subject: [PATCH] netfilter: add mutex to protect nft_ct_pcpu_template /* ... */ Signed-off-by: Pavel Skripkin <paskripkin@gmail.com> --- net/netfilter/nft_ct.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/net/netfilter/nft_ct.c b/net/netfilter/nft_ct.c index 337e22d8b40b..09f03036e5c8 100644 --- a/net/netfilter/nft_ct.c +++ b/net/netfilter/nft_ct.c @@ -10,6 +10,7 @@ #include <linux/init.h> #include <linux/module.h> #include <linux/netlink.h> +#include <linux/mutex.h> #include <linux/netfilter.h> #include <linux/netfilter/nf_tables.h> #include <net/netfilter/nf_tables.h> @@ -41,6 +42,7 @@ struct nft_ct_helper_obj { #ifdef CONFIG_NF_CONNTRACK_ZONES static DEFINE_PER_CPU(struct nf_conn *, nft_ct_pcpu_template); static unsigned int nft_ct_pcpu_template_refcnt __read_mostly; +static DEFINE_MUTEX(nft_ct_pcpu_template_mutex); #endif static u64 nft_ct_get_eval_counter(const struct nf_conn_counter *c, @@ -344,11 +346,13 @@ static const struct nla_policy nft_ct_policy[NFTA_CT_MAX + 1] = { }; #ifdef CONFIG_NF_CONNTRACK_ZONES -static void nft_ct_tmpl_put_pcpu(void) +static void nft_ct_tmpl_put_pcpu(bool lock) { struct nf_conn *ct; int cpu; + if (lock) + mutex_lock(&nft_ct_pcpu_template_mutex); for_each_possible_cpu(cpu) { ct = per_cpu(nft_ct_pcpu_template, cpu); if (!ct) @@ -356,6 +360,8 @@ static void nft_ct_tmpl_put_pcpu(void) nf_ct_put(ct); per_cpu(nft_ct_pcpu_template, cpu) = NULL; } + if (lock) + mutex_unlock(&nft_ct_pcpu_template_mutex); } static bool nft_ct_tmpl_alloc_pcpu(void) @@ -367,16 +373,18 @@ static bool nft_ct_tmpl_alloc_pcpu(void) if (nft_ct_pcpu_template_refcnt) return true; + mutex_lock(&nft_ct_pcpu_template_mutex); for_each_possible_cpu(cpu) { tmp = nf_ct_tmpl_alloc(&init_net, &zone, GFP_KERNEL); if (!tmp) { - nft_ct_tmpl_put_pcpu(); + nft_ct_tmpl_put_pcpu(false); return false; } atomic_set(&tmp->ct_general.use, 1); per_cpu(nft_ct_pcpu_template, cpu) = tmp; } + mutex_unlock(&nft_ct_pcpu_template_mutex); return true; } @@ -526,7 +534,7 @@ static void __nft_ct_set_destroy(const struct nft_ctx *ctx, struct nft_ct *priv) #ifdef CONFIG_NF_CONNTRACK_ZONES case NFT_CT_ZONE: if (--nft_ct_pcpu_template_refcnt == 0) - nft_ct_tmpl_put_pcpu(); + nft_ct_tmpl_put_pcpu(true); break; #endif default: -- 2.32.0 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu 2021-08-09 19:22 ` Pavel Skripkin @ 2021-08-09 20:39 ` Florian Westphal 2021-08-09 21:16 ` Pavel Skripkin 2021-08-10 2:59 ` syzbot 1 sibling, 1 reply; 8+ messages in thread From: Florian Westphal @ 2021-08-09 20:39 UTC (permalink / raw) To: Pavel Skripkin Cc: syzbot, coreteam, davem, fw, kadlec, kuba, linux-kernel, netdev, netfilter-devel, pablo, syzkaller-bugs Pavel Skripkin <paskripkin@gmail.com> wrote: > I think, there a missing lock in this function: > > for_each_possible_cpu(cpu) { > ct = per_cpu(nft_ct_pcpu_template, cpu); > if (!ct) > break; > nf_ct_put(ct); > per_cpu(nft_ct_pcpu_template, cpu) = NULL; > > } > > Syzbot hit a UAF in nft_ct_tmpl_put_pcpu() (*), but freed template should be > NULL. > > So I suspect following scenario: > > > CPU0: CPU1: > = per_cpu() > = per_cpu() > > nf_ct_put > per_cpu = NULL > nf_ct_put() > * UAF * Yes and no. The above is fine since pcpu will return different pointers for cpu 0 and 1. The race is between two different net namespaces that race when changing nft_ct_pcpu_template_refcnt. This happens since commit f102d66b335a417d4848da9441f585695a838934 netfilter: nf_tables: use dedicated mutex to guard transactions Before this, all transactions were serialized by a global mutex, now we only serialize transactions in the same netns. Its probably best to add DEFINE_MUTEX(nft_ct_pcpu_mutex) and then acquire that when we need to inc/dec the nft_ct_pcpu_template_refcnt so we can't have two distinct cpus hitting a zero refcount. Would you send a patch for this? Thanks. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu 2021-08-09 20:39 ` Florian Westphal @ 2021-08-09 21:16 ` Pavel Skripkin 2021-08-09 21:40 ` Florian Westphal 2021-08-10 3:18 ` syzbot 0 siblings, 2 replies; 8+ messages in thread From: Pavel Skripkin @ 2021-08-09 21:16 UTC (permalink / raw) To: Florian Westphal Cc: syzbot, coreteam, davem, kadlec, kuba, linux-kernel, netdev, netfilter-devel, pablo, syzkaller-bugs [-- Attachment #1: Type: text/plain, Size: 1864 bytes --] On 8/9/21 11:39 PM, Florian Westphal wrote: > Pavel Skripkin <paskripkin@gmail.com> wrote: >> I think, there a missing lock in this function: >> >> for_each_possible_cpu(cpu) { >> ct = per_cpu(nft_ct_pcpu_template, cpu); (*) >> if (!ct) >> break; >> nf_ct_put(ct); >> per_cpu(nft_ct_pcpu_template, cpu) = NULL; >> >> } >> >> Syzbot hit a UAF in nft_ct_tmpl_put_pcpu() (*), but freed template should be >> NULL. >> >> So I suspect following scenario: >> >> >> CPU0: CPU1: >> = per_cpu() >> = per_cpu() >> >> nf_ct_put >> per_cpu = NULL >> nf_ct_put() >> * UAF * Hi, Florian! > > Yes and no. The above is fine since pcpu will return different pointers > for cpu 0 and 1. > Dumb question: why per_cpu() will return 2 different pointers for CPU 1 and CPU 0? As I understand for_each_possible_cpu() will iterate over all CPUs which could ever be enabled. So, we can hit situation when 2 concurrent processes call per_cpu() with same cpu value (*). > The race is between two different net namespaces that race when > changing nft_ct_pcpu_template_refcnt. > This happens since > > commit f102d66b335a417d4848da9441f585695a838934 > netfilter: nf_tables: use dedicated mutex to guard transactions > > Before this, all transactions were serialized by a global mutex, > now we only serialize transactions in the same netns. > > Its probably best to add > DEFINE_MUTEX(nft_ct_pcpu_mutex) and then acquire that when we need to > inc/dec the nft_ct_pcpu_template_refcnt so we can't have two distinct > cpus hitting a zero refcount. > > Would you send a patch for this? > Anyway, I think, moving locking a bit higher is good here, let's test it. I will prepare a patch, if it will pass syzbot testing, thanks! #syz test git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master With regards, Pavel Skripkin [-- Attachment #2: 0001-netfiler-protect-nft_ct_pcpu_template_refcnt-with-mu.patch --] [-- Type: text/x-patch, Size: 1635 bytes --] From 616e99fd3ac738b2b5e43c5bc57f6f8cc7a49da0 Mon Sep 17 00:00:00 2001 From: Pavel Skripkin <paskripkin@gmail.com> Date: Tue, 10 Aug 2021 00:13:38 +0300 Subject: [PATCH] netfiler: protect nft_ct_pcpu_template_refcnt with mutex /* .... */ Signed-off-by: Pavel Skripkin <paskripkin@gmail.com> --- net/netfilter/nft_ct.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/net/netfilter/nft_ct.c b/net/netfilter/nft_ct.c index 337e22d8b40b..99b1de14ff7e 100644 --- a/net/netfilter/nft_ct.c +++ b/net/netfilter/nft_ct.c @@ -41,6 +41,7 @@ struct nft_ct_helper_obj { #ifdef CONFIG_NF_CONNTRACK_ZONES static DEFINE_PER_CPU(struct nf_conn *, nft_ct_pcpu_template); static unsigned int nft_ct_pcpu_template_refcnt __read_mostly; +static DEFINE_MUTEX(nft_ct_pcpu_mutex); #endif static u64 nft_ct_get_eval_counter(const struct nf_conn_counter *c, @@ -525,8 +526,10 @@ static void __nft_ct_set_destroy(const struct nft_ctx *ctx, struct nft_ct *priv) #endif #ifdef CONFIG_NF_CONNTRACK_ZONES case NFT_CT_ZONE: + mutex_lock(&nft_ct_pcpu_mutex); if (--nft_ct_pcpu_template_refcnt == 0) nft_ct_tmpl_put_pcpu(); + mutex_unlock(&nft_ct_pcpu_mutex); break; #endif default: @@ -564,9 +567,13 @@ static int nft_ct_set_init(const struct nft_ctx *ctx, #endif #ifdef CONFIG_NF_CONNTRACK_ZONES case NFT_CT_ZONE: - if (!nft_ct_tmpl_alloc_pcpu()) + mutex_lock(&nft_ct_pcpu_mutex); + if (!nft_ct_tmpl_alloc_pcpu()) { + mutex_unlock(&nft_ct_pcpu_mutex); return -ENOMEM; + } nft_ct_pcpu_template_refcnt++; + mutex_unlock(&nft_ct_pcpu_mutex); len = sizeof(u16); break; #endif -- 2.32.0 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu 2021-08-09 21:16 ` Pavel Skripkin @ 2021-08-09 21:40 ` Florian Westphal 2021-08-10 3:18 ` syzbot 1 sibling, 0 replies; 8+ messages in thread From: Florian Westphal @ 2021-08-09 21:40 UTC (permalink / raw) To: Pavel Skripkin Cc: Florian Westphal, syzbot, coreteam, davem, kadlec, kuba, linux-kernel, netdev, netfilter-devel, pablo, syzkaller-bugs Pavel Skripkin <paskripkin@gmail.com> wrote: > Dumb question: why per_cpu() will return 2 different pointers for CPU 1 and > CPU 0? As I understand for_each_possible_cpu() will iterate over all > CPUs which could ever be enabled. So, we can hit situation when 2 concurrent > processes call per_cpu() with same cpu value (*). Yes, that is what I was trying to say, the race is that we can have > 1 processes here ever since the global transaction mutex was removed in 2018. > Anyway, I think, moving locking a bit higher is good here, let's test it. I > will prepare a patch, if it will pass syzbot testing, thanks! It looks correct to me, thanks. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu 2021-08-09 21:16 ` Pavel Skripkin 2021-08-09 21:40 ` Florian Westphal @ 2021-08-10 3:18 ` syzbot 1 sibling, 0 replies; 8+ messages in thread From: syzbot @ 2021-08-10 3:18 UTC (permalink / raw) To: coreteam, davem, fw, kadlec, kuba, linux-kernel, netdev, netfilter-devel, pablo, paskripkin, syzkaller-bugs Hello, syzbot has tested the proposed patch and the reproducer did not trigger any issue: Reported-and-tested-by: syzbot+649e339fa6658ee623d3@syzkaller.appspotmail.com Tested on: commit: 9a73fa37 Merge branch 'for-5.14-fixes' of git://git.ke.. git tree: upstream kernel config: https://syzkaller.appspot.com/x/.config?x=e3a20bae04b96ccd dashboard link: https://syzkaller.appspot.com/bug?extid=649e339fa6658ee623d3 compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.1 patch: https://syzkaller.appspot.com/x/patch.diff?x=12d6baf6300000 Note: testing is done by a robot and is best-effort only. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu 2021-08-09 19:22 ` Pavel Skripkin 2021-08-09 20:39 ` Florian Westphal @ 2021-08-10 2:59 ` syzbot 1 sibling, 0 replies; 8+ messages in thread From: syzbot @ 2021-08-10 2:59 UTC (permalink / raw) To: coreteam, davem, fw, kadlec, kuba, linux-kernel, netdev, netfilter-devel, pablo, paskripkin, syzkaller-bugs Hello, syzbot has tested the proposed patch and the reproducer did not trigger any issue: Reported-and-tested-by: syzbot+649e339fa6658ee623d3@syzkaller.appspotmail.com Tested on: commit: 9a73fa37 Merge branch 'for-5.14-fixes' of git://git.ke.. git tree: upstream kernel config: https://syzkaller.appspot.com/x/.config?x=e3a20bae04b96ccd dashboard link: https://syzkaller.appspot.com/bug?extid=649e339fa6658ee623d3 compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.1 patch: https://syzkaller.appspot.com/x/patch.diff?x=153511fa300000 Note: testing is done by a robot and is best-effort only. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-08-10 3:18 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-07 13:44 [syzbot] KASAN: use-after-free Write in nft_ct_tmpl_put_pcpu syzbot 2021-08-07 20:50 ` syzbot 2021-08-09 19:22 ` Pavel Skripkin 2021-08-09 20:39 ` Florian Westphal 2021-08-09 21:16 ` Pavel Skripkin 2021-08-09 21:40 ` Florian Westphal 2021-08-10 3:18 ` syzbot 2021-08-10 2:59 ` syzbot
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).