* [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock
2019-09-19 20:14 [PATCH net v2 0/3] Fix Qdisc destroy issues caused by adding fine-grained locking to filter API Vlad Buslov
@ 2019-09-19 20:14 ` Vlad Buslov
2019-09-19 22:49 ` Eric Dumazet
2019-09-20 1:05 ` Cong Wang
2019-09-19 20:14 ` [PATCH net v2 2/3] net: sched: multiq: " Vlad Buslov
2019-09-19 20:14 ` [PATCH net v2 3/3] net: sched: sch_sfb: " Vlad Buslov
2 siblings, 2 replies; 9+ messages in thread
From: Vlad Buslov @ 2019-09-19 20:14 UTC (permalink / raw)
To: netdev; +Cc: jhs, xiyou.wangcong, jiri, davem, Vlad Buslov
Recent changes that removed rtnl dependency from rules update path of tc
also made tcf_block_put() function sleeping. This function is called from
ops->destroy() of several Qdisc implementations, which in turn is called by
qdisc_put(). Some Qdiscs call qdisc_put() while holding sch tree spinlock,
which results sleeping-while-atomic BUG.
Steps to reproduce for htb:
tc qdisc add dev ens1f0 root handle 1: htb default 12
tc class add dev ens1f0 parent 1: classid 1:1 htb rate 100kbps ceil 100kbps
tc qdisc add dev ens1f0 parent 1:1 handle 40: sfq perturb 10
tc class add dev ens1f0 parent 1:1 classid 1:2 htb rate 100kbps ceil 100kbps
Resulting dmesg:
[ 4791.148551] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:909
[ 4791.151354] in_atomic(): 1, irqs_disabled(): 0, pid: 27273, name: tc
[ 4791.152805] INFO: lockdep is turned off.
[ 4791.153605] CPU: 19 PID: 27273 Comm: tc Tainted: G W 5.3.0-rc8+ #721
[ 4791.154336] Hardware name: Supermicro SYS-2028TP-DECR/X10DRT-P, BIOS 2.0b 03/30/2017
[ 4791.155075] Call Trace:
[ 4791.155803] dump_stack+0x85/0xc0
[ 4791.156529] ___might_sleep.cold+0xac/0xbc
[ 4791.157251] __mutex_lock+0x5b/0x960
[ 4791.157966] ? console_unlock+0x363/0x5d0
[ 4791.158676] ? tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 4791.159395] ? tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 4791.160103] tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 4791.160815] tcf_block_put_ext.part.0+0x21/0x50
[ 4791.161530] tcf_block_put+0x50/0x70
[ 4791.162233] sfq_destroy+0x15/0x50 [sch_sfq]
[ 4791.162936] qdisc_destroy+0x5f/0x160
[ 4791.163642] htb_change_class.cold+0x5df/0x69d [sch_htb]
[ 4791.164505] tc_ctl_tclass+0x19d/0x480
[ 4791.165360] rtnetlink_rcv_msg+0x170/0x4b0
[ 4791.166191] ? netlink_deliver_tap+0x95/0x400
[ 4791.166907] ? rtnl_dellink+0x2d0/0x2d0
[ 4791.167625] netlink_rcv_skb+0x49/0x110
[ 4791.168345] netlink_unicast+0x171/0x200
[ 4791.169058] netlink_sendmsg+0x224/0x3f0
[ 4791.169771] sock_sendmsg+0x5e/0x60
[ 4791.170475] ___sys_sendmsg+0x2ae/0x330
[ 4791.171183] ? ___sys_recvmsg+0x159/0x1f0
[ 4791.171894] ? do_wp_page+0x9c/0x790
[ 4791.172595] ? __handle_mm_fault+0xcd3/0x19e0
[ 4791.173309] __sys_sendmsg+0x59/0xa0
[ 4791.174024] do_syscall_64+0x5c/0xb0
[ 4791.174725] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 4791.175435] RIP: 0033:0x7f0aa41497b8
[ 4791.176129] Code: 89 02 48 c7 c0 ff ff ff ff eb bb 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 65 8f 0c 00 8b 00 85 c0 75 17 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 89 5
4
[ 4791.177532] RSP: 002b:00007fff4e37d588 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 4791.178243] RAX: ffffffffffffffda RBX: 000000005d8132f7 RCX: 00007f0aa41497b8
[ 4791.178947] RDX: 0000000000000000 RSI: 00007fff4e37d5f0 RDI: 0000000000000003
[ 4791.179662] RBP: 0000000000000000 R08: 0000000000000001 R09: 00000000020149a0
[ 4791.180382] R10: 0000000000404eda R11: 0000000000000246 R12: 0000000000000001
[ 4791.181100] R13: 000000000047f640 R14: 0000000000000000 R15: 0000000000000000
Extend sch API with new qdisc_put_empty() function that has same
implementation as regular qdisc_put() but skips parts that reset qdisc and
free all packet buffers from gso_skb and skb_bad_txq queues. In
htb_change_class() function save parent->leaf.q to local temporary variable
and put reference to it after sch tree lock is released in order not to
call potentially sleeping cls API in atomic section. This is safe to do
because Qdisc has already been reset by qdisc_purge_queue() inside sch tree
lock critical section.
Fixes: c266f64dbfa2 ("net: sched: protect block state with mutex")
Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
---
Notes:
Changes V1 -> V2:
- Extend sch API with new qdisc_put_empty() function that has same
implementation as regular qdisc_put() but skips parts that reset qdisc
and free all packet buffers from gso_skb and skb_bad_txq queues.
- Use new qdisc_put_empty() instead of regular qdisc_put() in
htb_change_class().
include/net/sch_generic.h | 1 +
net/sched/sch_generic.c | 37 +++++++++++++++++++++++++------------
net/sched/sch_htb.c | 4 +++-
3 files changed, 29 insertions(+), 13 deletions(-)
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index 43f5b7ed02bd..5687c342989c 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -623,6 +623,7 @@ struct Qdisc *dev_graft_qdisc(struct netdev_queue *dev_queue,
struct Qdisc *qdisc);
void qdisc_reset(struct Qdisc *qdisc);
void qdisc_put(struct Qdisc *qdisc);
+void qdisc_put_empty(struct Qdisc *qdisc);
void qdisc_put_unlocked(struct Qdisc *qdisc);
void qdisc_tree_reduce_backlog(struct Qdisc *qdisc, int n, int len);
#ifdef CONFIG_NET_SCHED
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 17bd8f539bc7..3e2cc8d97efe 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -951,7 +951,7 @@ static void qdisc_free_cb(struct rcu_head *head)
qdisc_free(q);
}
-static void qdisc_destroy(struct Qdisc *qdisc)
+static void qdisc_destroy(struct Qdisc *qdisc, bool reset)
{
const struct Qdisc_ops *ops = qdisc->ops;
struct sk_buff *skb, *tmp;
@@ -962,7 +962,7 @@ static void qdisc_destroy(struct Qdisc *qdisc)
qdisc_put_stab(rtnl_dereference(qdisc->stab));
#endif
gen_kill_estimator(&qdisc->rate_est);
- if (ops->reset)
+ if (reset && ops->reset)
ops->reset(qdisc);
if (ops->destroy)
ops->destroy(qdisc);
@@ -970,20 +970,22 @@ static void qdisc_destroy(struct Qdisc *qdisc)
module_put(ops->owner);
dev_put(qdisc_dev(qdisc));
- skb_queue_walk_safe(&qdisc->gso_skb, skb, tmp) {
- __skb_unlink(skb, &qdisc->gso_skb);
- kfree_skb_list(skb);
- }
+ if (reset) {
+ skb_queue_walk_safe(&qdisc->gso_skb, skb, tmp) {
+ __skb_unlink(skb, &qdisc->gso_skb);
+ kfree_skb_list(skb);
+ }
- skb_queue_walk_safe(&qdisc->skb_bad_txq, skb, tmp) {
- __skb_unlink(skb, &qdisc->skb_bad_txq);
- kfree_skb_list(skb);
+ skb_queue_walk_safe(&qdisc->skb_bad_txq, skb, tmp) {
+ __skb_unlink(skb, &qdisc->skb_bad_txq);
+ kfree_skb_list(skb);
+ }
}
call_rcu(&qdisc->rcu, qdisc_free_cb);
}
-void qdisc_put(struct Qdisc *qdisc)
+static void __qdisc_put(struct Qdisc *qdisc, bool reset)
{
if (!qdisc)
return;
@@ -992,10 +994,21 @@ void qdisc_put(struct Qdisc *qdisc)
!refcount_dec_and_test(&qdisc->refcnt))
return;
- qdisc_destroy(qdisc);
+ qdisc_destroy(qdisc, reset);
+}
+
+void qdisc_put(struct Qdisc *qdisc)
+{
+ __qdisc_put(qdisc, true);
}
EXPORT_SYMBOL(qdisc_put);
+void qdisc_put_empty(struct Qdisc *qdisc)
+{
+ __qdisc_put(qdisc, false);
+}
+EXPORT_SYMBOL(qdisc_put_empty);
+
/* Version of qdisc_put() that is called with rtnl mutex unlocked.
* Intended to be used as optimization, this function only takes rtnl lock if
* qdisc reference counter reached zero.
@@ -1007,7 +1020,7 @@ void qdisc_put_unlocked(struct Qdisc *qdisc)
!refcount_dec_and_rtnl_lock(&qdisc->refcnt))
return;
- qdisc_destroy(qdisc);
+ qdisc_destroy(qdisc, true);
rtnl_unlock();
}
EXPORT_SYMBOL(qdisc_put_unlocked);
diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c
index 7bcf20ef9145..44551f485ac8 100644
--- a/net/sched/sch_htb.c
+++ b/net/sched/sch_htb.c
@@ -1302,6 +1302,7 @@ static int htb_change_class(struct Qdisc *sch, u32 classid,
struct htb_class *cl = (struct htb_class *)*arg, *parent;
struct nlattr *opt = tca[TCA_OPTIONS];
struct nlattr *tb[TCA_HTB_MAX + 1];
+ struct Qdisc *parent_qdisc = NULL;
struct tc_htb_opt *hopt;
u64 rate64, ceil64;
int warn = 0;
@@ -1401,7 +1402,7 @@ static int htb_change_class(struct Qdisc *sch, u32 classid,
if (parent && !parent->level) {
/* turn parent into inner node */
qdisc_purge_queue(parent->leaf.q);
- qdisc_put(parent->leaf.q);
+ parent_qdisc = parent->leaf.q;
if (parent->prio_activity)
htb_deactivate(q, parent);
@@ -1480,6 +1481,7 @@ static int htb_change_class(struct Qdisc *sch, u32 classid,
cl->cbuffer = PSCHED_TICKS2NS(hopt->cbuffer);
sch_tree_unlock(sch);
+ qdisc_put_empty(parent_qdisc);
if (warn)
pr_warn("HTB: quantum of class %X is %s. Consider r2q change.\n",
--
2.21.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock
2019-09-19 20:14 ` [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock Vlad Buslov
@ 2019-09-19 22:49 ` Eric Dumazet
2019-09-20 6:34 ` Vlad Buslov
2019-09-20 1:05 ` Cong Wang
1 sibling, 1 reply; 9+ messages in thread
From: Eric Dumazet @ 2019-09-19 22:49 UTC (permalink / raw)
To: Vlad Buslov, netdev; +Cc: jhs, xiyou.wangcong, jiri, davem
On 9/19/19 1:14 PM, Vlad Buslov wrote:
> Recent changes that removed rtnl dependency from rules update path of tc
> also made tcf_block_put() function sleeping. This function is called from
> ops->destroy() of several Qdisc implementations, which in turn is called by
> qdisc_put(). Some Qdiscs call qdisc_put() while holding sch tree spinlock,
> which results sleeping-while-atomic BUG.
>
Note that calling qdisc_put() while holding sch tree lock can also
trigger deadlocks.
For example sch_pie.c has a del_timer_sync() in pie_destroy(),
while the pie_timer() timer handler acquires the root_lock.
(there are other cases like that, SFQ for example)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock
2019-09-19 22:49 ` Eric Dumazet
@ 2019-09-20 6:34 ` Vlad Buslov
0 siblings, 0 replies; 9+ messages in thread
From: Vlad Buslov @ 2019-09-20 6:34 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Vlad Buslov, netdev, jhs, xiyou.wangcong, jiri, davem
On Fri 20 Sep 2019 at 01:49, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On 9/19/19 1:14 PM, Vlad Buslov wrote:
>> Recent changes that removed rtnl dependency from rules update path of tc
>> also made tcf_block_put() function sleeping. This function is called from
>> ops->destroy() of several Qdisc implementations, which in turn is called by
>> qdisc_put(). Some Qdiscs call qdisc_put() while holding sch tree spinlock,
>> which results sleeping-while-atomic BUG.
>>
>
>
> Note that calling qdisc_put() while holding sch tree lock can also
> trigger deadlocks.
>
> For example sch_pie.c has a del_timer_sync() in pie_destroy(),
> while the pie_timer() timer handler acquires the root_lock.
>
> (there are other cases like that, SFQ for example)
These and other examples of sleeping calls in cls APIs used by
tcf_block_put() that I described in one of my previous emails make me
think that putting might_sleep() at the beginning of qdisc_put() would
be a good idea, instead of waiting for syzbot to find correct
combination to trigger a crash.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock
2019-09-19 20:14 ` [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock Vlad Buslov
2019-09-19 22:49 ` Eric Dumazet
@ 2019-09-20 1:05 ` Cong Wang
2019-09-20 6:27 ` Vlad Buslov
1 sibling, 1 reply; 9+ messages in thread
From: Cong Wang @ 2019-09-20 1:05 UTC (permalink / raw)
To: Vlad Buslov
Cc: Linux Kernel Network Developers, Jamal Hadi Salim, Jiri Pirko,
David Miller
On Thu, Sep 19, 2019 at 1:14 PM Vlad Buslov <vladbu@mellanox.com> wrote:
> Notes:
> Changes V1 -> V2:
>
> - Extend sch API with new qdisc_put_empty() function that has same
> implementation as regular qdisc_put() but skips parts that reset qdisc
> and free all packet buffers from gso_skb and skb_bad_txq queues.
I don't understand why you need a new API here, as long as qdisc_reset()
gets called before releasing sch tree lock, the ->reset() inside qdisc_put(),
after releasing sch tree lock, should be a nop, right?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock
2019-09-20 1:05 ` Cong Wang
@ 2019-09-20 6:27 ` Vlad Buslov
2019-09-20 16:53 ` Cong Wang
0 siblings, 1 reply; 9+ messages in thread
From: Vlad Buslov @ 2019-09-20 6:27 UTC (permalink / raw)
To: Cong Wang
Cc: Vlad Buslov, Linux Kernel Network Developers, Jamal Hadi Salim,
Jiri Pirko, David Miller
On Fri 20 Sep 2019 at 04:05, Cong Wang <xiyou.wangcong@gmail.com> wrote:
> On Thu, Sep 19, 2019 at 1:14 PM Vlad Buslov <vladbu@mellanox.com> wrote:
>> Notes:
>> Changes V1 -> V2:
>>
>> - Extend sch API with new qdisc_put_empty() function that has same
>> implementation as regular qdisc_put() but skips parts that reset qdisc
>> and free all packet buffers from gso_skb and skb_bad_txq queues.
>
> I don't understand why you need a new API here, as long as qdisc_reset()
> gets called before releasing sch tree lock, the ->reset() inside qdisc_put(),
> after releasing sch tree lock, should be a nop, right?
Yes, but I wanted to make it explicit, so anyone else looking at the
code of those Qdiscs would know that manual reset with appropriate
locking is required. And it didn't require much new code because
qdisc_put() and qidsc_put_empty() just reuse same __qdisc_put(). I'll
revert it back, if you suggest that original approach is better.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock
2019-09-20 6:27 ` Vlad Buslov
@ 2019-09-20 16:53 ` Cong Wang
0 siblings, 0 replies; 9+ messages in thread
From: Cong Wang @ 2019-09-20 16:53 UTC (permalink / raw)
To: Vlad Buslov
Cc: Linux Kernel Network Developers, Jamal Hadi Salim, Jiri Pirko,
David Miller
On Thu, Sep 19, 2019 at 11:27 PM Vlad Buslov <vladbu@mellanox.com> wrote:
>
>
> On Fri 20 Sep 2019 at 04:05, Cong Wang <xiyou.wangcong@gmail.com> wrote:
> > On Thu, Sep 19, 2019 at 1:14 PM Vlad Buslov <vladbu@mellanox.com> wrote:
> >> Notes:
> >> Changes V1 -> V2:
> >>
> >> - Extend sch API with new qdisc_put_empty() function that has same
> >> implementation as regular qdisc_put() but skips parts that reset qdisc
> >> and free all packet buffers from gso_skb and skb_bad_txq queues.
> >
> > I don't understand why you need a new API here, as long as qdisc_reset()
> > gets called before releasing sch tree lock, the ->reset() inside qdisc_put(),
> > after releasing sch tree lock, should be a nop, right?
>
> Yes, but I wanted to make it explicit, so anyone else looking at the
> code of those Qdiscs would know that manual reset with appropriate
> locking is required. And it didn't require much new code because
> qdisc_put() and qidsc_put_empty() just reuse same __qdisc_put(). I'll
> revert it back, if you suggest that original approach is better.
It is unnecessary for -net/-stable. And you can always add a comment
to explain this if it is not clear.
Thanks.
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH net v2 2/3] net: sched: multiq: don't call qdisc_put() while holding tree lock
2019-09-19 20:14 [PATCH net v2 0/3] Fix Qdisc destroy issues caused by adding fine-grained locking to filter API Vlad Buslov
2019-09-19 20:14 ` [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock Vlad Buslov
@ 2019-09-19 20:14 ` Vlad Buslov
2019-09-19 20:14 ` [PATCH net v2 3/3] net: sched: sch_sfb: " Vlad Buslov
2 siblings, 0 replies; 9+ messages in thread
From: Vlad Buslov @ 2019-09-19 20:14 UTC (permalink / raw)
To: netdev; +Cc: jhs, xiyou.wangcong, jiri, davem, Vlad Buslov
Recent changes that removed rtnl dependency from rules update path of tc
also made tcf_block_put() function sleeping. This function is called from
ops->destroy() of several Qdisc implementations, which in turn is called by
qdisc_put(). Some Qdiscs call qdisc_put() while holding sch tree spinlock,
which results sleeping-while-atomic BUG.
Steps to reproduce for multiq:
tc qdisc add dev ens1f0 root handle 1: multiq
tc qdisc add dev ens1f0 parent 1:10 handle 50: sfq perturb 10
ethtool -L ens1f0 combined 2
tc qdisc change dev ens1f0 root handle 1: multiq
Resulting dmesg:
[ 5539.419344] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:909
[ 5539.420945] in_atomic(): 1, irqs_disabled(): 0, pid: 27658, name: tc
[ 5539.422435] INFO: lockdep is turned off.
[ 5539.423904] CPU: 21 PID: 27658 Comm: tc Tainted: G W 5.3.0-rc8+ #721
[ 5539.425400] Hardware name: Supermicro SYS-2028TP-DECR/X10DRT-P, BIOS 2.0b 03/30/2017
[ 5539.426911] Call Trace:
[ 5539.428380] dump_stack+0x85/0xc0
[ 5539.429823] ___might_sleep.cold+0xac/0xbc
[ 5539.431262] __mutex_lock+0x5b/0x960
[ 5539.432682] ? tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 5539.434103] ? __nla_validate_parse+0x51/0x840
[ 5539.435493] ? tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 5539.436903] tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 5539.438327] tcf_block_put_ext.part.0+0x21/0x50
[ 5539.439752] tcf_block_put+0x50/0x70
[ 5539.441165] sfq_destroy+0x15/0x50 [sch_sfq]
[ 5539.442570] qdisc_destroy+0x5f/0x160
[ 5539.444000] multiq_tune+0x14a/0x420 [sch_multiq]
[ 5539.445421] tc_modify_qdisc+0x324/0x840
[ 5539.446841] rtnetlink_rcv_msg+0x170/0x4b0
[ 5539.448269] ? netlink_deliver_tap+0x95/0x400
[ 5539.449691] ? rtnl_dellink+0x2d0/0x2d0
[ 5539.451116] netlink_rcv_skb+0x49/0x110
[ 5539.452522] netlink_unicast+0x171/0x200
[ 5539.453914] netlink_sendmsg+0x224/0x3f0
[ 5539.455304] sock_sendmsg+0x5e/0x60
[ 5539.456686] ___sys_sendmsg+0x2ae/0x330
[ 5539.458071] ? ___sys_recvmsg+0x159/0x1f0
[ 5539.459461] ? do_wp_page+0x9c/0x790
[ 5539.460846] ? __handle_mm_fault+0xcd3/0x19e0
[ 5539.462263] __sys_sendmsg+0x59/0xa0
[ 5539.463661] do_syscall_64+0x5c/0xb0
[ 5539.465044] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 5539.466454] RIP: 0033:0x7f1fe08177b8
[ 5539.467863] Code: 89 02 48 c7 c0 ff ff ff ff eb bb 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 65 8f 0c 00 8b 00 85 c0 75 17 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 89 5
4
[ 5539.470906] RSP: 002b:00007ffe812de5d8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 5539.472483] RAX: ffffffffffffffda RBX: 000000005d8135e3 RCX: 00007f1fe08177b8
[ 5539.474069] RDX: 0000000000000000 RSI: 00007ffe812de640 RDI: 0000000000000003
[ 5539.475655] RBP: 0000000000000000 R08: 0000000000000001 R09: 000000000182e9b0
[ 5539.477203] R10: 0000000000404eda R11: 0000000000000246 R12: 0000000000000001
[ 5539.478699] R13: 000000000047f640 R14: 0000000000000000 R15: 0000000000000000
Rearrange locking in multiq_tune() in following ways:
- In loop that removes Qdiscs from disabled queues, call
qdisc_purge_queue() instead of qdisc_tree_flush_backlog() on Qdisc that
is being destroyed. Save the Qdisc in temporary allocated array and call
qdisc_put_empty() on each element of the array after sch tree lock is
released. This is safe to do because Qdiscs have already been reset by
qdisc_purge_queue() inside sch tree lock critical section.
- Do the same change for second loop that initializes Qdiscs for newly
enabled queues in multiq_tune() function. Since sch tree lock is obtained
and released on each iteration of this loop, just call qdisc_put_empty()
directly outside of critical section. Don't verify that old Qdisc is not
noop_qdisc before releasing reference to it because such check is already
performed by qdisc_put*() functions.
Fixes: c266f64dbfa2 ("net: sched: protect block state with mutex")
Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
---
Notes:
Changes V1 -> V2:
- Refactor first loop in multiq_tune() to save child Qdiscs that are being
removed into an array and call qdisc_put_empty() on all of its elements
after sch tree lock critical section. Revert the change in V1 that
obtained and released sch tree lock on every loop iteration.
- Use qdisc_purge_queue() instead of qdisc_tree_flush_backlog() to properly
reset Qdiscs inside sch tree lock critical section.
- Use qdisc_put_empty() in both loops of multiq_tune() instead of regular
qdisc_put().
net/sched/sch_multiq.c | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/net/sched/sch_multiq.c b/net/sched/sch_multiq.c
index e1087746f6a2..52595b40efbd 100644
--- a/net/sched/sch_multiq.c
+++ b/net/sched/sch_multiq.c
@@ -174,7 +174,8 @@ static int multiq_tune(struct Qdisc *sch, struct nlattr *opt,
{
struct multiq_sched_data *q = qdisc_priv(sch);
struct tc_multiq_qopt *qopt;
- int i;
+ struct Qdisc **removed;
+ int i, n_removed = 0;
if (!netif_is_multiqueue(qdisc_dev(sch)))
return -EOPNOTSUPP;
@@ -185,6 +186,11 @@ static int multiq_tune(struct Qdisc *sch, struct nlattr *opt,
qopt->bands = qdisc_dev(sch)->real_num_tx_queues;
+ removed = kmalloc(sizeof(*removed) * (q->max_bands - q->bands),
+ GFP_KERNEL);
+ if (!removed)
+ return -ENOMEM;
+
sch_tree_lock(sch);
q->bands = qopt->bands;
for (i = q->bands; i < q->max_bands; i++) {
@@ -192,13 +198,17 @@ static int multiq_tune(struct Qdisc *sch, struct nlattr *opt,
struct Qdisc *child = q->queues[i];
q->queues[i] = &noop_qdisc;
- qdisc_tree_flush_backlog(child);
- qdisc_put(child);
+ qdisc_purge_queue(child);
+ removed[n_removed++] = child;
}
}
sch_tree_unlock(sch);
+ for (i = 0; i < n_removed; i++)
+ qdisc_put_empty(removed[i]);
+ kfree(removed);
+
for (i = 0; i < q->bands; i++) {
if (q->queues[i] == &noop_qdisc) {
struct Qdisc *child, *old;
@@ -213,11 +223,10 @@ static int multiq_tune(struct Qdisc *sch, struct nlattr *opt,
if (child != &noop_qdisc)
qdisc_hash_add(child, true);
- if (old != &noop_qdisc) {
- qdisc_tree_flush_backlog(old);
- qdisc_put(old);
- }
+ if (old != &noop_qdisc)
+ qdisc_purge_queue(old);
sch_tree_unlock(sch);
+ qdisc_put_empty(old);
}
}
}
--
2.21.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH net v2 3/3] net: sched: sch_sfb: don't call qdisc_put() while holding tree lock
2019-09-19 20:14 [PATCH net v2 0/3] Fix Qdisc destroy issues caused by adding fine-grained locking to filter API Vlad Buslov
2019-09-19 20:14 ` [PATCH net v2 1/3] net: sched: sch_htb: don't call qdisc_put() while holding tree lock Vlad Buslov
2019-09-19 20:14 ` [PATCH net v2 2/3] net: sched: multiq: " Vlad Buslov
@ 2019-09-19 20:14 ` Vlad Buslov
2 siblings, 0 replies; 9+ messages in thread
From: Vlad Buslov @ 2019-09-19 20:14 UTC (permalink / raw)
To: netdev
Cc: jhs, xiyou.wangcong, jiri, davem, Vlad Buslov,
syzbot+ac54455281db908c581e
Recent changes that removed rtnl dependency from rules update path of tc
also made tcf_block_put() function sleeping. This function is called from
ops->destroy() of several Qdisc implementations, which in turn is called by
qdisc_put(). Some Qdiscs call qdisc_put() while holding sch tree spinlock,
which results sleeping-while-atomic BUG.
Steps to reproduce for sfb:
tc qdisc add dev ens1f0 handle 1: root sfb
tc qdisc add dev ens1f0 parent 1:10 handle 50: sfq perturb 10
tc qdisc change dev ens1f0 root handle 1: sfb
Resulting dmesg:
[ 7265.938717] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:909
[ 7265.940152] in_atomic(): 1, irqs_disabled(): 0, pid: 28579, name: tc
[ 7265.941455] INFO: lockdep is turned off.
[ 7265.942744] CPU: 11 PID: 28579 Comm: tc Tainted: G W 5.3.0-rc8+ #721
[ 7265.944065] Hardware name: Supermicro SYS-2028TP-DECR/X10DRT-P, BIOS 2.0b 03/30/2017
[ 7265.945396] Call Trace:
[ 7265.946709] dump_stack+0x85/0xc0
[ 7265.947994] ___might_sleep.cold+0xac/0xbc
[ 7265.949282] __mutex_lock+0x5b/0x960
[ 7265.950543] ? tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 7265.951803] ? tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 7265.953022] tcf_chain0_head_change_cb_del.isra.0+0x1b/0xf0
[ 7265.954248] tcf_block_put_ext.part.0+0x21/0x50
[ 7265.955478] tcf_block_put+0x50/0x70
[ 7265.956694] sfq_destroy+0x15/0x50 [sch_sfq]
[ 7265.957898] qdisc_destroy+0x5f/0x160
[ 7265.959099] sfb_change+0x175/0x330 [sch_sfb]
[ 7265.960304] tc_modify_qdisc+0x324/0x840
[ 7265.961503] rtnetlink_rcv_msg+0x170/0x4b0
[ 7265.962692] ? netlink_deliver_tap+0x95/0x400
[ 7265.963876] ? rtnl_dellink+0x2d0/0x2d0
[ 7265.965064] netlink_rcv_skb+0x49/0x110
[ 7265.966251] netlink_unicast+0x171/0x200
[ 7265.967427] netlink_sendmsg+0x224/0x3f0
[ 7265.968595] sock_sendmsg+0x5e/0x60
[ 7265.969753] ___sys_sendmsg+0x2ae/0x330
[ 7265.970916] ? ___sys_recvmsg+0x159/0x1f0
[ 7265.972074] ? do_wp_page+0x9c/0x790
[ 7265.973233] ? __handle_mm_fault+0xcd3/0x19e0
[ 7265.974407] __sys_sendmsg+0x59/0xa0
[ 7265.975591] do_syscall_64+0x5c/0xb0
[ 7265.976753] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 7265.977938] RIP: 0033:0x7f229069f7b8
[ 7265.979117] Code: 89 02 48 c7 c0 ff ff ff ff eb bb 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 65 8f 0c 00 8b 00 85 c0 75 17 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 89 5
4
[ 7265.981681] RSP: 002b:00007ffd7ed2d158 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 7265.983001] RAX: ffffffffffffffda RBX: 000000005d813ca1 RCX: 00007f229069f7b8
[ 7265.984336] RDX: 0000000000000000 RSI: 00007ffd7ed2d1c0 RDI: 0000000000000003
[ 7265.985682] RBP: 0000000000000000 R08: 0000000000000001 R09: 000000000165c9a0
[ 7265.987021] R10: 0000000000404eda R11: 0000000000000246 R12: 0000000000000001
[ 7265.988309] R13: 000000000047f640 R14: 0000000000000000 R15: 0000000000000000
In sfb_change() function use qdisc_purge_queue() instead of
qdisc_tree_flush_backlog() to properly reset old child Qdisc and save
pointer to it into local temporary variable. Put reference to Qdisc after
sch tree lock is released in order not to call potentially sleeping cls API
in atomic section. This is safe to do because Qdisc has already been reset
by qdisc_purge_queue() inside sch tree lock critical section.
Reported-by: syzbot+ac54455281db908c581e@syzkaller.appspotmail.com
Fixes: c266f64dbfa2 ("net: sched: protect block state with mutex")
Suggested-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
---
Notes:
Changes V1 -> V2:
- Use qdisc_purge_queue() instead of qdisc_tree_flush_backlog() to properly
reset Qdiscs inside sch tree lock critical section.
- Call qdisc_put_empty() instead of regular qdisc_put() after sch tree lock
is released.
net/sched/sch_sfb.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/net/sched/sch_sfb.c b/net/sched/sch_sfb.c
index 1dff8506a715..23b2e492c587 100644
--- a/net/sched/sch_sfb.c
+++ b/net/sched/sch_sfb.c
@@ -488,7 +488,7 @@ static int sfb_change(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
struct sfb_sched_data *q = qdisc_priv(sch);
- struct Qdisc *child;
+ struct Qdisc *child, *old;
struct nlattr *tb[TCA_SFB_MAX + 1];
const struct tc_sfb_qopt *ctl = &sfb_default_ops;
u32 limit;
@@ -518,8 +518,8 @@ static int sfb_change(struct Qdisc *sch, struct nlattr *opt,
qdisc_hash_add(child, true);
sch_tree_lock(sch);
- qdisc_tree_flush_backlog(q->qdisc);
- qdisc_put(q->qdisc);
+ qdisc_purge_queue(q->qdisc);
+ old = q->qdisc;
q->qdisc = child;
q->rehash_interval = msecs_to_jiffies(ctl->rehash_interval);
@@ -542,6 +542,7 @@ static int sfb_change(struct Qdisc *sch, struct nlattr *opt,
sfb_init_perturbation(1, q);
sch_tree_unlock(sch);
+ qdisc_put_empty(old);
return 0;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 9+ messages in thread