From: Jakub Kicinski <kuba@kernel.org>
To: Sven Van Asbroeck <thesven73@gmail.com>
Cc: "Bryan Whitehead" <bryan.whitehead@microchip.com>,
UNGLinuxDriver@microchip.com,
"David S Miller" <davem@davemloft.net>,
"Andrew Lunn" <andrew@lunn.ch>,
"Alexey Denisov" <rtgbnm@gmail.com>,
"Sergej Bauer" <sbauer@blackbox.su>,
"Tim Harvey" <tharvey@gateworks.com>,
"Anders Rønningen" <anders@ronningen.priv.no>,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH net-next v1 1/6] lan743x: boost performance on cpu archs w/o dma cache snooping
Date: Fri, 29 Jan 2021 14:01:20 -0800 [thread overview]
Message-ID: <20210129140120.29ae5062@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> (raw)
In-Reply-To: <20210129195240.31871-2-TheSven73@gmail.com>
On Fri, 29 Jan 2021 14:52:35 -0500 Sven Van Asbroeck wrote:
> From: Sven Van Asbroeck <thesven73@gmail.com>
>
> The buffers in the lan743x driver's receive ring are always 9K,
> even when the largest packet that can be received (the mtu) is
> much smaller. This performs particularly badly on cpu archs
> without dma cache snooping (such as ARM): each received packet
> results in a 9K dma_{map|unmap} operation, which is very expensive
> because cpu caches need to be invalidated.
>
> Careful measurement of the driver rx path on armv7 reveals that
> the cpu spends the majority of its time waiting for cache
> invalidation.
>
> Optimize as follows:
>
> 1. set rx ring buffer size equal to the mtu. this limits the
> amount of cache that needs to be invalidated per dma_map().
>
> 2. when dma_unmap()ping, skip cpu sync. Sync only the packet data
> actually received, the size of which the chip will indicate in
> its rx ring descriptors. this limits the amount of cache that
> needs to be invalidated per dma_unmap().
>
> These optimizations double the rx performance on armv7.
> Third parties report 3x rx speedup on armv8.
>
> Performance on dma cache snooping architectures (such as x86)
> is expected to stay the same.
>
> Tested with iperf3 on a freescale imx6qp + lan7430, both sides
> set to mtu 1500 bytes, measure rx performance:
>
> Before:
> [ ID] Interval Transfer Bandwidth Retr
> [ 4] 0.00-20.00 sec 550 MBytes 231 Mbits/sec 0
> After:
> [ ID] Interval Transfer Bandwidth Retr
> [ 4] 0.00-20.00 sec 1.33 GBytes 570 Mbits/sec 0
>
> Test by Anders Roenningen (anders@ronningen.priv.no) on armv8,
> rx iperf3:
> Before 102 Mbits/sec
> After 279 Mbits/sec
>
> Signed-off-by: Sven Van Asbroeck <thesven73@gmail.com>
You may need to rebase to see this:
drivers/net/ethernet/microchip/lan743x_main.c:2123:41: warning: restricted __le32 degrades to integer
next prev parent reply other threads:[~2021-01-29 22:02 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-29 19:52 [PATCH net-next v1 0/6] lan743x speed boost Sven Van Asbroeck
2021-01-29 19:52 ` [PATCH net-next v1 1/6] lan743x: boost performance on cpu archs w/o dma cache snooping Sven Van Asbroeck
2021-01-29 20:36 ` Andrew Lunn
2021-01-29 22:49 ` Sven Van Asbroeck
2021-01-29 22:01 ` Jakub Kicinski [this message]
2021-01-29 22:46 ` Sven Van Asbroeck
2021-01-30 22:10 ` Bryan.Whitehead
2021-01-30 23:59 ` Sven Van Asbroeck
2021-01-31 0:14 ` Sven Van Asbroeck
[not found] ` <20210204060210.2362-1-hdanton@sina.com>
2021-02-05 9:31 ` Christoph Hellwig
2021-02-05 14:01 ` Sven Van Asbroeck
2021-02-05 12:44 ` Sergej Bauer
2021-02-05 14:07 ` Sven Van Asbroeck
2021-02-05 15:09 ` Sergej Bauer
2021-02-05 16:39 ` Sven Van Asbroeck
2021-02-05 16:59 ` Sergej Bauer
2021-01-29 19:52 ` [PATCH net-next v1 2/6] lan743x: support rx multi-buffer packets Sven Van Asbroeck
2021-01-29 22:11 ` Willem de Bruijn
2021-01-29 23:02 ` Sven Van Asbroeck
2021-01-29 23:08 ` Willem de Bruijn
2021-01-29 23:10 ` Sven Van Asbroeck
2021-01-31 7:06 ` Bryan.Whitehead
2021-01-31 15:25 ` Sven Van Asbroeck
2021-02-01 18:04 ` Bryan.Whitehead
2021-02-03 18:53 ` Sven Van Asbroeck
2021-02-03 20:14 ` Bryan.Whitehead
2021-02-03 20:25 ` Sven Van Asbroeck
2021-02-03 20:41 ` Bryan.Whitehead
2021-01-29 19:52 ` [PATCH net-next v1 3/6] lan743x: allow mtu change while network interface is up Sven Van Asbroeck
2021-01-29 19:52 ` [PATCH net-next v1 4/6] TEST ONLY: lan743x: limit rx ring buffer size to 500 bytes Sven Van Asbroeck
2021-01-29 19:52 ` [PATCH net-next v1 5/6] TEST ONLY: lan743x: skb_alloc failure test Sven Van Asbroeck
2021-01-29 19:52 ` [PATCH net-next v1 6/6] TEST ONLY: lan743x: skb_trim " Sven Van Asbroeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210129140120.29ae5062@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com \
--to=kuba@kernel.org \
--cc=UNGLinuxDriver@microchip.com \
--cc=anders@ronningen.priv.no \
--cc=andrew@lunn.ch \
--cc=bryan.whitehead@microchip.com \
--cc=davem@davemloft.net \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=rtgbnm@gmail.com \
--cc=sbauer@blackbox.su \
--cc=tharvey@gateworks.com \
--cc=thesven73@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).