From: Yunsheng Lin <linyunsheng@huawei.com>
To: Jakub Kicinski <kuba@kernel.org>
Cc: <davem@davemloft.net>, <olteanv@gmail.com>, <ast@kernel.org>,
<daniel@iogearbox.net>, <andriin@fb.com>, <edumazet@google.com>,
<weiwan@google.com>, <cong.wang@bytedance.com>,
<ap420073@gmail.com>, <netdev@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, <linuxarm@openeuler.org>,
<mkl@pengutronix.de>, <linux-can@vger.kernel.org>
Subject: Re: [Linuxarm] Re: [RFC v2] net: sched: implement TCQ_F_CAN_BYPASS for lockless qdisc
Date: Tue, 16 Mar 2021 11:47:25 +0800 [thread overview]
Message-ID: <3ddec762-19c8-6743-43dd-3e44f91fd113@huawei.com> (raw)
In-Reply-To: <3838b7c2-c32f-aeda-702a-5cb8f712ec0c@huawei.com>
On 2021/3/16 8:35, Yunsheng Lin wrote:
> On 2021/3/16 2:53, Jakub Kicinski wrote:
>> On Mon, 15 Mar 2021 11:10:18 +0800 Yunsheng Lin wrote:
>>> @@ -606,6 +623,11 @@ static const u8 prio2band[TC_PRIO_MAX + 1] = {
>>> */
>>> struct pfifo_fast_priv {
>>> struct skb_array q[PFIFO_FAST_BANDS];
>>> +
>>> + /* protect against data race between enqueue/dequeue and
>>> + * qdisc->empty setting
>>> + */
>>> + spinlock_t lock;
>>> };
>>>
>>> static inline struct skb_array *band2list(struct pfifo_fast_priv *priv,
>>> @@ -623,7 +645,10 @@ static int pfifo_fast_enqueue(struct sk_buff *skb, struct Qdisc *qdisc,
>>> unsigned int pkt_len = qdisc_pkt_len(skb);
>>> int err;
>>>
>>> - err = skb_array_produce(q, skb);
>>> + spin_lock(&priv->lock);
>>> + err = __ptr_ring_produce(&q->ring, skb);
>>> + WRITE_ONCE(qdisc->empty, false);
>>> + spin_unlock(&priv->lock);
>>>
>>> if (unlikely(err)) {
>>> if (qdisc_is_percpu_stats(qdisc))
>>> @@ -642,6 +667,7 @@ static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc)
>>> struct sk_buff *skb = NULL;
>>> int band;
>>>
>>> + spin_lock(&priv->lock);
>>> for (band = 0; band < PFIFO_FAST_BANDS && !skb; band++) {
>>> struct skb_array *q = band2list(priv, band);
>>>
>>> @@ -655,6 +681,7 @@ static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc)
>>> } else {
>>> WRITE_ONCE(qdisc->empty, true);
>>> }
>>> + spin_unlock(&priv->lock);
>>>
>>> return skb;
>>> }
>>
>> I thought pfifo was supposed to be "lockless" and this change
>> re-introduces a lock between producer and consumer, no?
>
> Yes, the lock breaks the "lockless" of the lockless qdisc for now
> I do not how to solve the below data race locklessly:
>
> CPU1: CPU2:
> dequeue skb .
> . .
> . enqueue skb
> . .
> . WRITE_ONCE(qdisc->empty, false);
> . .
> . .
> WRITE_ONCE(qdisc->empty, true);
>
> If the above happens, the qdisc->empty is true even if the qdisc has some
> skb, which may cuase out of order or packet stuck problem.
>
> It seems we may need to update ptr_ring' status(empty or not) while
> enqueuing/dequeuing atomically in the ptr_ring implementation.
>
> Any better idea?
It seems we can use __ptr_ring_empty() within the qdisc->seqlock protection,
because qdisc->seqlock is clearly served as r->consumer_lock.
>
>>
>> .
>>
> _______________________________________________
> Linuxarm mailing list -- linuxarm@openeuler.org
> To unsubscribe send an email to linuxarm-leave@openeuler.org
>
next prev parent reply other threads:[~2021-03-16 3:48 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-13 2:47 [PATCH RFC] net: sched: implement TCQ_F_CAN_BYPASS for lockless qdisc Yunsheng Lin
2021-03-14 0:03 ` Vladimir Oltean
2021-03-14 10:15 ` Marc Kleine-Budde
2021-03-15 0:50 ` Yunsheng Lin
2021-03-15 3:10 ` [RFC v2] " Yunsheng Lin
2021-03-15 12:29 ` Vladimir Oltean
2021-03-15 13:09 ` Marc Kleine-Budde
2021-03-15 18:53 ` Jakub Kicinski
2021-03-16 0:35 ` Yunsheng Lin
2021-03-16 3:47 ` Yunsheng Lin [this message]
2021-03-16 8:15 ` Eric Dumazet
2021-03-16 12:36 ` Yunsheng Lin
2021-03-16 22:48 ` Cong Wang
2021-03-17 1:14 ` Yunsheng Lin
2021-03-17 13:35 ` Toke Høiland-Jørgensen
2021-03-17 13:45 ` Jason A. Donenfeld
2021-03-18 7:33 ` [Linuxarm] " Yunsheng Lin
2021-03-19 18:15 ` Cong Wang
2021-03-22 0:55 ` Yunsheng Lin
2021-03-24 1:49 ` Cong Wang
2021-03-24 2:36 ` Yunsheng Lin
2021-03-19 19:03 ` Jason A. Donenfeld
2021-03-22 1:05 ` Yunsheng Lin
2021-03-18 7:10 ` Ahmad Fatoum
2021-03-18 7:46 ` Yunsheng Lin
2021-03-18 9:09 ` Ahmad Fatoum
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3ddec762-19c8-6743-43dd-3e44f91fd113@huawei.com \
--to=linyunsheng@huawei.com \
--cc=andriin@fb.com \
--cc=ap420073@gmail.com \
--cc=ast@kernel.org \
--cc=cong.wang@bytedance.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=linux-can@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxarm@openeuler.org \
--cc=mkl@pengutronix.de \
--cc=netdev@vger.kernel.org \
--cc=olteanv@gmail.com \
--cc=weiwan@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).