From: Jakub Kicinski <jakub.kicinski@netronome.com>
To: Cong Wang <xiyou.wangcong@gmail.com>
Cc: David Miller <davem@davemloft.net>,
Stephen Hemminger <stephen@networkplumber.org>,
Jamal Hadi Salim <jhs@mojatatu.com>,
Jiri Pirko <jiri@resnulli.us>,
netem@lists.linux-foundation.org,
Linux Kernel Network Developers <netdev@vger.kernel.org>,
oss-drivers@netronome.com, Eric Dumazet <edumazet@google.com>,
posk@google.com
Subject: Re: [PATCH net] net: netem: fix use after free and double free with packet corruption
Date: Fri, 14 Jun 2019 17:16:44 -0700 [thread overview]
Message-ID: <20190614171644.766547d1@cakuba.netronome.com> (raw)
In-Reply-To: <CAM_iQpXoKnP+Xj0CMQf08nBCnbPEVu=uTbgCk98C380pYSUetA@mail.gmail.com>
On Fri, 14 Jun 2019 09:40:18 -0700, Cong Wang wrote:
> On Wed, Jun 12, 2019 at 11:52 AM Jakub Kicinski wrote:
> >
> > Brendan reports that the use of netem's packet corruption capability
> > leads to strange crashes. This seems to be caused by
> > commit d66280b12bd7 ("net: netem: use a list in addition to rbtree")
> > which uses skb->next pointer to construct a fast-path queue of
> > in-order skbs.
> >
> > Packet corruption code has to invoke skb_gso_segment() in case
> > of skbs in need of GSO. skb_gso_segment() returns a list of
> > skbs. If next pointers of the skbs on that list do not get cleared
> > fast path list goes into the weeds and tries to access the next
> > segment skb multiple times.
>
> Mind to be more specific? How could it be accessed multiple times?
You're right, the commit message is not great :S
So we segment an skb and get a list:
A -> B -> C
And then we hook in A to the t_head t_tail list:
h t
|/
A -> B -> C
Now if B and C get also get enqueued successfully all is fine, because
we will overwrite the list in order. IOW:
h t
| |
A -> B -> C
h t
| |
A -> B -> C
But if B and C get reordered we may end up with
h t RB
|/ |
A -> B -> C B
\
C
Or if they get dropped (overlimits) just:
h t
|/
A -> B -> C
while A and B are already freed.
> > Reported-by: Brendan Galloway <brendan.galloway@netronome.com>
> > Fixes: d66280b12bd7 ("net: netem: use a list in addition to rbtree")
> > Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
> > Reviewed-by: Dirk van der Merwe <dirk.vandermerwe@netronome.com>
> > ---
> > net/sched/sch_netem.c | 11 ++++-------
> > 1 file changed, 4 insertions(+), 7 deletions(-)
> >
> > diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
> > index 956ff3da81f4..1fd4405611e5 100644
> > --- a/net/sched/sch_netem.c
> > +++ b/net/sched/sch_netem.c
> > @@ -494,16 +494,13 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
> > */
> > if (q->corrupt && q->corrupt >= get_crandom(&q->corrupt_cor)) {
> > if (skb_is_gso(skb)) {
> > - segs = netem_segment(skb, sch, to_free);
> > - if (!segs)
> > + skb = netem_segment(skb, sch, to_free);
> > + if (!skb)
> > return rc_drop;
> > - } else {
> > - segs = skb;
> > + segs = skb->next;
> > + skb_mark_not_on_list(skb);
> > }
> >
> > - skb = segs;
> > - segs = segs->next;
> > -
>
> I don't see how this works when we hit goto finish_segs?
> Either goto finish_segs can be removed or needs to be fixed?
Note that I'm removing the else branch. So for non-GSO we end up with:
skb = original
segs = NULL
for GSO we end up with:
skb = first seg (->next == NULL)
segs = second seg (->next == third, etc.)
So should work all as is?
next prev parent reply other threads:[~2019-06-15 0:16 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-12 18:51 [PATCH net] net: netem: fix use after free and double free with packet corruption Jakub Kicinski
2019-06-14 16:40 ` Cong Wang
2019-06-15 0:16 ` Jakub Kicinski [this message]
2019-06-15 2:08 ` David Miller
2019-06-15 20:25 ` Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190614171644.766547d1@cakuba.netronome.com \
--to=jakub.kicinski@netronome.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=jhs@mojatatu.com \
--cc=jiri@resnulli.us \
--cc=netdev@vger.kernel.org \
--cc=netem@lists.linux-foundation.org \
--cc=oss-drivers@netronome.com \
--cc=posk@google.com \
--cc=stephen@networkplumber.org \
--cc=xiyou.wangcong@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).