* [PATCH net v2 1/2] net: netem: fix backlog accounting for corrupted GSO frames
2019-06-17 18:11 [PATCH net v2 0/2] net: netem: fix issues with corrupting GSO frames Jakub Kicinski
@ 2019-06-17 18:11 ` Jakub Kicinski
2019-06-17 22:32 ` Cong Wang
2019-06-17 18:11 ` [PATCH net v2 2/2] net: netem: fix use after free and double free with packet corruption Jakub Kicinski
1 sibling, 1 reply; 5+ messages in thread
From: Jakub Kicinski @ 2019-06-17 18:11 UTC (permalink / raw)
To: davem, xiyou.wangcong
Cc: stephen, jhs, jiri, netem, netdev, oss-drivers, edumazet, posk, nhorman
When GSO frame has to be corrupted netem uses skb_gso_segment()
to produce the list of frames, and re-enqueues the segments one
by one. The backlog length has to be adjusted to account for
new frames.
The current calculation is incorrect, leading to wrong backlog
lengths in the parent qdisc (both bytes and packets), and
incorrect packet backlog count in netem itself.
Parent backlog goes negative, netem's packet backlog counts
all non-first segments twice (thus remaining non-zero even
after qdisc is emptied).
Move the variables used to count the adjustment into local
scope to make 100% sure they aren't used at any stage in
backports.
Fixes: 6071bd1aa13e ("netem: Segment GSO packets on enqueue")
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Dirk van der Merwe <dirk.vandermerwe@netronome.com>
---
net/sched/sch_netem.c | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index 956ff3da81f4..3b3e2d772c3b 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -439,8 +439,7 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
struct netem_skb_cb *cb;
struct sk_buff *skb2;
struct sk_buff *segs = NULL;
- unsigned int len = 0, last_len, prev_len = qdisc_pkt_len(skb);
- int nb = 0;
+ unsigned int prev_len = qdisc_pkt_len(skb);
int count = 1;
int rc = NET_XMIT_SUCCESS;
int rc_drop = NET_XMIT_DROP;
@@ -497,6 +496,7 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
segs = netem_segment(skb, sch, to_free);
if (!segs)
return rc_drop;
+ qdisc_skb_cb(segs)->pkt_len = segs->len;
} else {
segs = skb;
}
@@ -593,6 +593,11 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
finish_segs:
if (segs) {
+ unsigned int len, last_len;
+ int nb = 0;
+
+ len = skb->len;
+
while (segs) {
skb2 = segs->next;
skb_mark_not_on_list(segs);
@@ -608,9 +613,7 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
}
segs = skb2;
}
- sch->q.qlen += nb;
- if (nb > 1)
- qdisc_tree_reduce_backlog(sch, 1 - nb, prev_len - len);
+ qdisc_tree_reduce_backlog(sch, -nb, prev_len - len);
}
return NET_XMIT_SUCCESS;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH net v2 2/2] net: netem: fix use after free and double free with packet corruption
2019-06-17 18:11 [PATCH net v2 0/2] net: netem: fix issues with corrupting GSO frames Jakub Kicinski
2019-06-17 18:11 ` [PATCH net v2 1/2] net: netem: fix backlog accounting for corrupted " Jakub Kicinski
@ 2019-06-17 18:11 ` Jakub Kicinski
2019-06-17 22:33 ` Cong Wang
1 sibling, 1 reply; 5+ messages in thread
From: Jakub Kicinski @ 2019-06-17 18:11 UTC (permalink / raw)
To: davem, xiyou.wangcong
Cc: stephen, jhs, jiri, netem, netdev, oss-drivers, edumazet, posk, nhorman
Brendan reports that the use of netem's packet corruption capability
leads to strange crashes. This seems to be caused by
commit d66280b12bd7 ("net: netem: use a list in addition to rbtree")
which uses skb->next pointer to construct a fast-path queue of
in-order skbs.
Packet corruption code has to invoke skb_gso_segment() in case
of skbs in need of GSO. skb_gso_segment() returns a list of
skbs. If next pointers of the skbs on that list do not get cleared
fast path list may point to freed skbs or skbs which are also on
the RB tree.
Let's say skb gets segmented into 3 frames:
A -> B -> C
A gets hooked to the t_head t_tail list by tfifo_enqueue(), but it's
next pointer didn't get cleared so we have:
h t
|/
A -> B -> C
Now if B and C get also get enqueued successfully all is fine, because
tfifo_enqueue() will overwrite the list in order. IOW:
Enqueue B:
h t
| |
A -> B C
Enqueue C:
h t
| |
A -> B -> C
But if B and C get reordered we may end up with:
h t RB tree
|/ |
A -> B -> C B
\
C
Or if they get dropped just:
h t
|/
A -> B -> C
where A and B are already freed.
To reproduce either limit has to be set low to cause freeing of
segs or reorders have to happen (due to delay jitter).
Note that we only have to mark the first segment as not on the
list, "finish_segs" handling of other frags already does that.
Another caveat is that qdisc_drop_all() still has to free all
segments correctly in case of drop of first segment, therefore
we re-link segs before calling it.
v2:
- re-link before drop, v1 was leaking non-first segs if limit
was hit at the first seg
- better commit message which lead to discovering the above :)
Reported-by: Brendan Galloway <brendan.galloway@netronome.com>
Fixes: d66280b12bd7 ("net: netem: use a list in addition to rbtree")
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Dirk van der Merwe <dirk.vandermerwe@netronome.com>
---
net/sched/sch_netem.c | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index 3b3e2d772c3b..b17f2ed970e2 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -493,17 +493,14 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
*/
if (q->corrupt && q->corrupt >= get_crandom(&q->corrupt_cor)) {
if (skb_is_gso(skb)) {
- segs = netem_segment(skb, sch, to_free);
- if (!segs)
+ skb = netem_segment(skb, sch, to_free);
+ if (!skb)
return rc_drop;
- qdisc_skb_cb(segs)->pkt_len = segs->len;
- } else {
- segs = skb;
+ segs = skb->next;
+ skb_mark_not_on_list(skb);
+ qdisc_skb_cb(skb)->pkt_len = skb->len;
}
- skb = segs;
- segs = segs->next;
-
skb = skb_unshare(skb, GFP_ATOMIC);
if (unlikely(!skb)) {
qdisc_qstats_drop(sch);
@@ -520,6 +517,8 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
}
if (unlikely(sch->q.qlen >= sch->limit)) {
+ /* re-link segs, so that qdisc_drop_all() frees them all */
+ skb->next = segs;
qdisc_drop_all(skb, sch, to_free);
return rc_drop;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 5+ messages in thread