From: Eric Dumazet <eric.dumazet@gmail.com>
To: Denys Fedoryshchenko <nuclearcat@nuclearcat.com>
Cc: Cong Wang <cwang@twopensource.com>,
Jamal Hadi Salim <jhs@mojatatu.com>,
"David S. Miller" <davem@davemloft.net>,
netdev <netdev@vger.kernel.org>,
linux-kernel@vger.kernel.org
Subject: Re: kernel panic in 4.2.3, rb_erase in sch_fq
Date: Tue, 03 Nov 2015 20:46:21 -0800 [thread overview]
Message-ID: <1446612381.4184.7.camel@edumazet-glaptop2.roam.corp.google.com> (raw)
In-Reply-To: <0705e2b76150c28341d7e1915433450d@visp.net.lb>
On Wed, 2015-11-04 at 06:25 +0200, Denys Fedoryshchenko wrote:
> On 2015-11-04 00:06, Cong Wang wrote:
> > On Mon, Nov 2, 2015 at 6:11 AM, Denys Fedoryshchenko
> > <nuclearcat@nuclearcat.com> wrote:
> >> Hi!
> >>
> >> Actually seems i was getting this panic for a while (once per week) on
> >> loaded pppoe server, but just now was able to get full panic message.
> >> After checking commit logs on sch_fq.c i didnt seen any fixes, so
> >> probably
> >> upgrading to newer kernel wont help?
> >
> >
> > Can you share your `tc qdisc show dev xxxx` with us? And how to
> > reproduce
> > it? I tried to setup htb+fq and then flip the interface back and forth
> > but I don't
> > see any crash.
> My guess it wont be easy to reproduce, it is happening on box with 4.5k
> interfaces, that constantly create/delete interfaces,
> and even with that this problem may happen once per day, or may not
> happen for 1 week.
>
> Here is script that is being fired after new ppp interface detected. But
> pppoe process are independent from
> process that are "establishing" shapers.
It is probably a generic bug. sch_fq seems OK to me.
Somehow nobody tries to change qdisc hundred times per second ;)
Could you try following patch ?
It seems to 'fix' the issue for me.
diff --git a/net/core/dev.c b/net/core/dev.c
index 8ce3f74cd6b9..bf136103bc7b 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2880,6 +2880,12 @@ static inline int __dev_xmit_skb(struct sk_buff *skb, struct Qdisc *q,
spin_lock(&q->busylock);
spin_lock(root_lock);
+ if (unlikely(q != rcu_dereference_bh(txq->qdisc))) {
+ pr_err_ratelimited("Arg, qdisc changed ! state %lx\n", q->state);
+ kfree_skb(skb);
+ rc = NET_XMIT_DROP;
+ goto end;
+ }
if (unlikely(test_bit(__QDISC_STATE_DEACTIVATED, &q->state))) {
kfree_skb(skb);
rc = NET_XMIT_DROP;
@@ -2913,6 +2919,7 @@ static inline int __dev_xmit_skb(struct sk_buff *skb, struct Qdisc *q,
__qdisc_run(q);
}
}
+end:
spin_unlock(root_lock);
if (unlikely(contended))
spin_unlock(&q->busylock);
next prev parent reply other threads:[~2015-11-04 4:46 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-02 14:11 kernel panic in 4.2.3, rb_erase in sch_fq Denys Fedoryshchenko
2015-11-02 15:24 ` Eric Dumazet
2015-11-02 15:58 ` Denys Fedoryshchenko
2015-11-02 16:12 ` Eric Dumazet
2015-11-02 16:21 ` Denys Fedoryshchenko
2015-11-03 22:06 ` Cong Wang
2015-11-04 4:25 ` Denys Fedoryshchenko
2015-11-04 4:46 ` Eric Dumazet [this message]
2015-11-04 4:58 ` Eric Dumazet
2015-11-04 5:13 ` Denys Fedoryshchenko
2015-11-13 21:41 ` Denys Fedoryshchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1446612381.4184.7.camel@edumazet-glaptop2.roam.corp.google.com \
--to=eric.dumazet@gmail.com \
--cc=cwang@twopensource.com \
--cc=davem@davemloft.net \
--cc=jhs@mojatatu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=nuclearcat@nuclearcat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).