From: Dave Taht <dave.taht@gmail.com>
To: Jakob Unterwurzacher <jakob.unterwurzacher@theobroma-systems.com>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
John Fastabend <john.fastabend@gmail.com>,
"David S. Miller" <davem@davemloft.net>,
"linux-can@vger.kernel.org" <linux-can@vger.kernel.org>,
Martin Elshuber <martin.elshuber@theobroma-systems.com>
Subject: Re: [bug, bisected] pfifo_fast causes packet reordering
Date: Tue, 13 Mar 2018 11:35:46 -0700 [thread overview]
Message-ID: <CAA93jw5FSbvg8AZ=EmneW8juqd8LCgp=xUrSxYm2GBEucKkT3w@mail.gmail.com> (raw)
In-Reply-To: <946dbe16-a2eb-eca8-8069-468859ccc78d@theobroma-systems.com>
On Tue, Mar 13, 2018 at 11:24 AM, Jakob Unterwurzacher
<jakob.unterwurzacher@theobroma-systems.com> wrote:
> During stress-testing our "ucan" USB/CAN adapter SocketCAN driver on Linux
> v4.16-rc4-383-ged58d66f60b3 we observed that a small fraction of packets are
> delivered out-of-order.
>
> We have tracked the problem down to the driver interface level, and it seems
> that the driver's net_device_ops.ndo_start_xmit() function gets the packets
> handed over in the wrong order.
>
> This behavior was not observed on Linux v4.15 and I have bisected the
> problem down to this patch:
>
>> commit c5ad119fb6c09b0297446be05bd66602fa564758
>> Author: John Fastabend <john.fastabend@gmail.com>
>> Date: Thu Dec 7 09:58:19 2017 -0800
>>
>> net: sched: pfifo_fast use skb_array
>>
>> This converts the pfifo_fast qdisc to use the skb_array data structure
>> and set the lockless qdisc bit. pfifo_fast is the first qdisc to
>> support
>> the lockless bit that can be a child of a qdisc requiring locking. So
>> we add logic to clear the lock bit on initialization in these cases
>> when
>> the qdisc graft operation occurs.
>>
>> This also removes the logic used to pick the next band to dequeue from
>> and instead just checks a per priority array for packets from top
>> priority
>> to lowest. This might need to be a bit more clever but seems to work
>> for now.
>>
>> Signed-off-by: John Fastabend <john.fastabend@gmail.com>
>> Signed-off-by: David S. Miller <davem@davemloft.net>
>
>
> The patch does not revert cleanly, but moving to one commit earlier makes
> the problem go away.
>
> Selecting the "fq" scheduler instead of "pfifo_fast" makes the problem go
> away as well.
I am of course, a fan of obsoleting pfifo_fast. There's no good reason
for it anymore.
>
> Is this an unintended side-effect of the patch or is there something the
> driver has to do to request in-order delivery?
>
> Thanks,
> Jakob
--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
next prev parent reply other threads:[~2018-03-13 18:35 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-13 18:24 [bug, bisected] pfifo_fast causes packet reordering Jakob Unterwurzacher
2018-03-13 18:35 ` Dave Taht [this message]
2018-03-14 4:03 ` John Fastabend
2018-03-14 10:09 ` Jakob Unterwurzacher
2018-03-15 18:08 ` Jakob Unterwurzacher
2018-03-15 22:30 ` John Fastabend
2018-03-16 10:26 ` Jakob Unterwurzacher
2018-03-19 6:07 ` Alexander Stein
2018-03-19 12:32 ` Paolo Abeni
2018-03-19 12:56 ` Jakob Unterwurzacher
2018-03-21 10:01 ` Jakob Unterwurzacher
2018-03-21 18:43 ` John Fastabend
2018-03-21 19:44 ` Jakob Unterwurzacher
2018-03-21 20:52 ` John Fastabend
2018-03-22 10:16 ` Jakob Unterwurzacher
2018-03-24 14:26 ` John Fastabend
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAA93jw5FSbvg8AZ=EmneW8juqd8LCgp=xUrSxYm2GBEucKkT3w@mail.gmail.com' \
--to=dave.taht@gmail.com \
--cc=davem@davemloft.net \
--cc=jakob.unterwurzacher@theobroma-systems.com \
--cc=john.fastabend@gmail.com \
--cc=linux-can@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=martin.elshuber@theobroma-systems.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).