* [PATCH net] net: rds: force to destroy connection if t_sock is NULL in rds_tcp_kill_sock().
@ 2019-03-28 9:10 Mao Wenan
2019-03-28 17:14 ` Santosh Shilimkar
2019-03-29 0:18 ` David Miller
0 siblings, 2 replies; 3+ messages in thread
From: Mao Wenan @ 2019-03-28 9:10 UTC (permalink / raw)
To: santosh.shilimkar, davem; +Cc: netdev, linux-rdma, rds-devel, linux-kernel
When it is to cleanup net namespace, rds_tcp_exit_net() will call
rds_tcp_kill_sock(), if t_sock is NULL, it will not call
rds_conn_destroy(), rds_conn_path_destroy() and rds_tcp_conn_free() to free
connection, and the worker cp_conn_w is not stopped, afterwards the net is freed in
net_drop_ns(); While cp_conn_w rds_connect_worker() will call rds_tcp_conn_path_connect()
and reference 'net' which has already been freed.
In rds_tcp_conn_path_connect(), rds_tcp_set_callbacks() will set t_sock = sock before
sock->ops->connect, but if connect() is failed, it will call
rds_tcp_restore_callbacks() and set t_sock = NULL, if connect is always
failed, rds_connect_worker() will try to reconnect all the time, so
rds_tcp_kill_sock() will never to cancel worker cp_conn_w and free the
connections.
Therefore, the condition !tc->t_sock is not needed if it is going to do
cleanup_net->rds_tcp_exit_net->rds_tcp_kill_sock, because tc->t_sock is always
NULL, and there is on other path to cancel cp_conn_w and free
connection. So this patch is to fix this.
rds_tcp_kill_sock():
...
if (net != c_net || !tc->t_sock)
...
==================================================================
BUG: KASAN: use-after-free in inet_create+0xbcc/0xd28
net/ipv4/af_inet.c:340
Read of size 4 at addr ffff8003496a4684 by task kworker/u8:4/3721
CPU: 3 PID: 3721 Comm: kworker/u8:4 Not tainted 5.1.0 #11
Hardware name: linux,dummy-virt (DT)
Workqueue: krdsd rds_connect_worker
Call trace:
dump_backtrace+0x0/0x3c0 arch/arm64/kernel/time.c:53
show_stack+0x28/0x38 arch/arm64/kernel/traps.c:152
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x120/0x188 lib/dump_stack.c:113
print_address_description+0x68/0x278 mm/kasan/report.c:253
kasan_report_error mm/kasan/report.c:351 [inline]
kasan_report+0x21c/0x348 mm/kasan/report.c:409
__asan_report_load4_noabort+0x30/0x40 mm/kasan/report.c:429
inet_create+0xbcc/0xd28 net/ipv4/af_inet.c:340
__sock_create+0x4f8/0x770 net/socket.c:1276
sock_create_kern+0x50/0x68 net/socket.c:1322
rds_tcp_conn_path_connect+0x2b4/0x690 net/rds/tcp_connect.c:114
rds_connect_worker+0x108/0x1d0 net/rds/threads.c:175
process_one_work+0x6e8/0x1700 kernel/workqueue.c:2153
worker_thread+0x3b0/0xdd0 kernel/workqueue.c:2296
kthread+0x2f0/0x378 kernel/kthread.c:255
ret_from_fork+0x10/0x18 arch/arm64/kernel/entry.S:1117
Allocated by task 687:
save_stack mm/kasan/kasan.c:448 [inline]
set_track mm/kasan/kasan.c:460 [inline]
kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:553
kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:490
slab_post_alloc_hook mm/slab.h:444 [inline]
slab_alloc_node mm/slub.c:2705 [inline]
slab_alloc mm/slub.c:2713 [inline]
kmem_cache_alloc+0x14c/0x388 mm/slub.c:2718
kmem_cache_zalloc include/linux/slab.h:697 [inline]
net_alloc net/core/net_namespace.c:384 [inline]
copy_net_ns+0xc4/0x2d0 net/core/net_namespace.c:424
create_new_namespaces+0x300/0x658 kernel/nsproxy.c:107
unshare_nsproxy_namespaces+0xa0/0x198 kernel/nsproxy.c:206
ksys_unshare+0x340/0x628 kernel/fork.c:2577
__do_sys_unshare kernel/fork.c:2645 [inline]
__se_sys_unshare kernel/fork.c:2643 [inline]
__arm64_sys_unshare+0x38/0x58 kernel/fork.c:2643
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall arch/arm64/kernel/syscall.c:47 [inline]
el0_svc_common+0x168/0x390 arch/arm64/kernel/syscall.c:83
el0_svc_handler+0x60/0xd0 arch/arm64/kernel/syscall.c:129
el0_svc+0x8/0xc arch/arm64/kernel/entry.S:960
Freed by task 264:
save_stack mm/kasan/kasan.c:448 [inline]
set_track mm/kasan/kasan.c:460 [inline]
__kasan_slab_free+0x114/0x220 mm/kasan/kasan.c:521
kasan_slab_free+0x10/0x18 mm/kasan/kasan.c:528
slab_free_hook mm/slub.c:1370 [inline]
slab_free_freelist_hook mm/slub.c:1397 [inline]
slab_free mm/slub.c:2952 [inline]
kmem_cache_free+0xb8/0x3a8 mm/slub.c:2968
net_free net/core/net_namespace.c:400 [inline]
net_drop_ns.part.6+0x78/0x90 net/core/net_namespace.c:407
net_drop_ns net/core/net_namespace.c:406 [inline]
cleanup_net+0x53c/0x6d8 net/core/net_namespace.c:569
process_one_work+0x6e8/0x1700 kernel/workqueue.c:2153
worker_thread+0x3b0/0xdd0 kernel/workqueue.c:2296
kthread+0x2f0/0x378 kernel/kthread.c:255
ret_from_fork+0x10/0x18 arch/arm64/kernel/entry.S:1117
The buggy address belongs to the object at ffff8003496a3f80
which belongs to the cache net_namespace of size 7872
The buggy address is located 1796 bytes inside of
7872-byte region [ffff8003496a3f80, ffff8003496a5e40)
The buggy address belongs to the page:
page:ffff7e000d25a800 count:1 mapcount:0 mapping:ffff80036ce4b000
index:0x0 compound_mapcount: 0
flags: 0xffffe0000008100(slab|head)
raw: 0ffffe0000008100 dead000000000100 dead000000000200 ffff80036ce4b000
raw: 0000000000000000 0000000080040004 00000001ffffffff 0000000000000000
page dumped because: kasan: bad access detected
Memory state around the buggy address:
ffff8003496a4580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff8003496a4600: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff8003496a4680: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff8003496a4700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff8003496a4780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================
Fixes: 467fa15356ac("RDS-TCP: Support multiple RDS-TCP listen endpoints, one per netns.")
Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Mao Wenan <maowenan@huawei.com>
---
net/rds/tcp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/rds/tcp.c b/net/rds/tcp.c
index fd2694174607..faf726e00e27 100644
--- a/net/rds/tcp.c
+++ b/net/rds/tcp.c
@@ -608,7 +608,7 @@ static void rds_tcp_kill_sock(struct net *net)
list_for_each_entry_safe(tc, _tc, &rds_tcp_conn_list, t_tcp_node) {
struct net *c_net = read_pnet(&tc->t_cpath->cp_conn->c_net);
- if (net != c_net || !tc->t_sock)
+ if (net != c_net)
continue;
if (!list_has_conn(&tmp_list, tc->t_cpath->cp_conn)) {
list_move_tail(&tc->t_tcp_node, &tmp_list);
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH net] net: rds: force to destroy connection if t_sock is NULL in rds_tcp_kill_sock().
2019-03-28 9:10 [PATCH net] net: rds: force to destroy connection if t_sock is NULL in rds_tcp_kill_sock() Mao Wenan
@ 2019-03-28 17:14 ` Santosh Shilimkar
2019-03-29 0:18 ` David Miller
1 sibling, 0 replies; 3+ messages in thread
From: Santosh Shilimkar @ 2019-03-28 17:14 UTC (permalink / raw)
To: Mao Wenan, davem; +Cc: netdev, linux-rdma, rds-devel, linux-kernel
On 3/28/2019 2:10 AM, Mao Wenan wrote:
> When it is to cleanup net namespace, rds_tcp_exit_net() will call
> rds_tcp_kill_sock(), if t_sock is NULL, it will not call
> rds_conn_destroy(), rds_conn_path_destroy() and rds_tcp_conn_free() to free
> connection, and the worker cp_conn_w is not stopped, afterwards the net is freed in
> net_drop_ns(); While cp_conn_w rds_connect_worker() will call rds_tcp_conn_path_connect()
> and reference 'net' which has already been freed.
>
> In rds_tcp_conn_path_connect(), rds_tcp_set_callbacks() will set t_sock = sock before
> sock->ops->connect, but if connect() is failed, it will call
> rds_tcp_restore_callbacks() and set t_sock = NULL, if connect is always
> failed, rds_connect_worker() will try to reconnect all the time, so
> rds_tcp_kill_sock() will never to cancel worker cp_conn_w and free the
> connections.
>
> Therefore, the condition !tc->t_sock is not needed if it is going to do
> cleanup_net->rds_tcp_exit_net->rds_tcp_kill_sock, because tc->t_sock is always
> NULL, and there is on other path to cancel cp_conn_w and free
> connection. So this patch is to fix this.
>
> rds_tcp_kill_sock():
> ...
> if (net != c_net || !tc->t_sock)
> ...
> ==================================================================
> BUG: KASAN: use-after-free in inet_create+0xbcc/0xd28
> net/ipv4/af_inet.c:340
> Read of size 4 at addr ffff8003496a4684 by task kworker/u8:4/3721
>
> CPU: 3 PID: 3721 Comm: kworker/u8:4 Not tainted 5.1.0 #11
> Hardware name: linux,dummy-virt (DT)
> Workqueue: krdsd rds_connect_worker
> Call trace:
> dump_backtrace+0x0/0x3c0 arch/arm64/kernel/time.c:53
> show_stack+0x28/0x38 arch/arm64/kernel/traps.c:152
> __dump_stack lib/dump_stack.c:77 [inline]
> dump_stack+0x120/0x188 lib/dump_stack.c:113
> print_address_description+0x68/0x278 mm/kasan/report.c:253
> kasan_report_error mm/kasan/report.c:351 [inline]
> kasan_report+0x21c/0x348 mm/kasan/report.c:409
> __asan_report_load4_noabort+0x30/0x40 mm/kasan/report.c:429
> inet_create+0xbcc/0xd28 net/ipv4/af_inet.c:340
> __sock_create+0x4f8/0x770 net/socket.c:1276
> sock_create_kern+0x50/0x68 net/socket.c:1322
> rds_tcp_conn_path_connect+0x2b4/0x690 net/rds/tcp_connect.c:114
> rds_connect_worker+0x108/0x1d0 net/rds/threads.c:175
> process_one_work+0x6e8/0x1700 kernel/workqueue.c:2153
> worker_thread+0x3b0/0xdd0 kernel/workqueue.c:2296
> kthread+0x2f0/0x378 kernel/kthread.c:255
> ret_from_fork+0x10/0x18 arch/arm64/kernel/entry.S:1117
>
> Allocated by task 687:
> save_stack mm/kasan/kasan.c:448 [inline]
> set_track mm/kasan/kasan.c:460 [inline]
> kasan_kmalloc+0xd4/0x180 mm/kasan/kasan.c:553
> kasan_slab_alloc+0x14/0x20 mm/kasan/kasan.c:490
> slab_post_alloc_hook mm/slab.h:444 [inline]
> slab_alloc_node mm/slub.c:2705 [inline]
> slab_alloc mm/slub.c:2713 [inline]
> kmem_cache_alloc+0x14c/0x388 mm/slub.c:2718
> kmem_cache_zalloc include/linux/slab.h:697 [inline]
> net_alloc net/core/net_namespace.c:384 [inline]
> copy_net_ns+0xc4/0x2d0 net/core/net_namespace.c:424
> create_new_namespaces+0x300/0x658 kernel/nsproxy.c:107
> unshare_nsproxy_namespaces+0xa0/0x198 kernel/nsproxy.c:206
> ksys_unshare+0x340/0x628 kernel/fork.c:2577
> __do_sys_unshare kernel/fork.c:2645 [inline]
> __se_sys_unshare kernel/fork.c:2643 [inline]
> __arm64_sys_unshare+0x38/0x58 kernel/fork.c:2643
> __invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
> invoke_syscall arch/arm64/kernel/syscall.c:47 [inline]
> el0_svc_common+0x168/0x390 arch/arm64/kernel/syscall.c:83
> el0_svc_handler+0x60/0xd0 arch/arm64/kernel/syscall.c:129
> el0_svc+0x8/0xc arch/arm64/kernel/entry.S:960
>
> Freed by task 264:
> save_stack mm/kasan/kasan.c:448 [inline]
> set_track mm/kasan/kasan.c:460 [inline]
> __kasan_slab_free+0x114/0x220 mm/kasan/kasan.c:521
> kasan_slab_free+0x10/0x18 mm/kasan/kasan.c:528
> slab_free_hook mm/slub.c:1370 [inline]
> slab_free_freelist_hook mm/slub.c:1397 [inline]
> slab_free mm/slub.c:2952 [inline]
> kmem_cache_free+0xb8/0x3a8 mm/slub.c:2968
> net_free net/core/net_namespace.c:400 [inline]
> net_drop_ns.part.6+0x78/0x90 net/core/net_namespace.c:407
> net_drop_ns net/core/net_namespace.c:406 [inline]
> cleanup_net+0x53c/0x6d8 net/core/net_namespace.c:569
> process_one_work+0x6e8/0x1700 kernel/workqueue.c:2153
> worker_thread+0x3b0/0xdd0 kernel/workqueue.c:2296
> kthread+0x2f0/0x378 kernel/kthread.c:255
> ret_from_fork+0x10/0x18 arch/arm64/kernel/entry.S:1117
>
> The buggy address belongs to the object at ffff8003496a3f80
> which belongs to the cache net_namespace of size 7872
> The buggy address is located 1796 bytes inside of
> 7872-byte region [ffff8003496a3f80, ffff8003496a5e40)
> The buggy address belongs to the page:
> page:ffff7e000d25a800 count:1 mapcount:0 mapping:ffff80036ce4b000
> index:0x0 compound_mapcount: 0
> flags: 0xffffe0000008100(slab|head)
> raw: 0ffffe0000008100 dead000000000100 dead000000000200 ffff80036ce4b000
> raw: 0000000000000000 0000000080040004 00000001ffffffff 0000000000000000
> page dumped because: kasan: bad access detected
>
> Memory state around the buggy address:
> ffff8003496a4580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ffff8003496a4600: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>> ffff8003496a4680: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ^
> ffff8003496a4700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ffff8003496a4780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ==================================================================
>
> Fixes: 467fa15356ac("RDS-TCP: Support multiple RDS-TCP listen endpoints, one per netns.")
>
> Reported-by: Hulk Robot <hulkci@huawei.com>
> Signed-off-by: Mao Wenan <maowenan@huawei.com>
> ---
Looks reasonable fix to me. Thanks !!
Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH net] net: rds: force to destroy connection if t_sock is NULL in rds_tcp_kill_sock().
2019-03-28 9:10 [PATCH net] net: rds: force to destroy connection if t_sock is NULL in rds_tcp_kill_sock() Mao Wenan
2019-03-28 17:14 ` Santosh Shilimkar
@ 2019-03-29 0:18 ` David Miller
1 sibling, 0 replies; 3+ messages in thread
From: David Miller @ 2019-03-29 0:18 UTC (permalink / raw)
To: maowenan; +Cc: santosh.shilimkar, netdev, linux-rdma, rds-devel, linux-kernel
From: Mao Wenan <maowenan@huawei.com>
Date: Thu, 28 Mar 2019 17:10:56 +0800
> When it is to cleanup net namespace, rds_tcp_exit_net() will call
> rds_tcp_kill_sock(), if t_sock is NULL, it will not call
> rds_conn_destroy(), rds_conn_path_destroy() and rds_tcp_conn_free() to free
> connection, and the worker cp_conn_w is not stopped, afterwards the net is freed in
> net_drop_ns(); While cp_conn_w rds_connect_worker() will call rds_tcp_conn_path_connect()
> and reference 'net' which has already been freed.
>
> In rds_tcp_conn_path_connect(), rds_tcp_set_callbacks() will set t_sock = sock before
> sock->ops->connect, but if connect() is failed, it will call
> rds_tcp_restore_callbacks() and set t_sock = NULL, if connect is always
> failed, rds_connect_worker() will try to reconnect all the time, so
> rds_tcp_kill_sock() will never to cancel worker cp_conn_w and free the
> connections.
>
> Therefore, the condition !tc->t_sock is not needed if it is going to do
> cleanup_net->rds_tcp_exit_net->rds_tcp_kill_sock, because tc->t_sock is always
> NULL, and there is on other path to cancel cp_conn_w and free
> connection. So this patch is to fix this.
>
> rds_tcp_kill_sock():
> ...
> if (net != c_net || !tc->t_sock)
> ...
> ==================================================================
> BUG: KASAN: use-after-free in inet_create+0xbcc/0xd28
...
> ==================================================================
>
> Fixes: 467fa15356ac("RDS-TCP: Support multiple RDS-TCP listen endpoints, one per netns.")
> Reported-by: Hulk Robot <hulkci@huawei.com>
> Signed-off-by: Mao Wenan <maowenan@huawei.com>
Applied and queued up for -stable.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-03-29 0:18 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-28 9:10 [PATCH net] net: rds: force to destroy connection if t_sock is NULL in rds_tcp_kill_sock() Mao Wenan
2019-03-28 17:14 ` Santosh Shilimkar
2019-03-29 0:18 ` David Miller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).