From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
To: netdev@vger.kernel.org
Cc: "Michael S. Tsirkin" <mst@redhat.com>,
Jason Wang <jasowang@redhat.com>,
Xuan Zhuo <xuanzhuo@linux.alibaba.com>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Jesper Dangaard Brouer <hawk@kernel.org>,
John Fastabend <john.fastabend@gmail.com>,
virtualization@lists.linux.dev, bpf@vger.kernel.org
Subject: [PATCH net-next 00/17] virtio-net: support AF_XDP zero copy (3/3)
Date: Tue, 16 Jan 2024 17:42:56 +0800 [thread overview]
Message-ID: <20240116094313.119939-1-xuanzhuo@linux.alibaba.com> (raw)
This is the third part of virtio-net support AF_XDP zero copy.
The whole patch set
http://lore.kernel.org/all/20231229073108.57778-1-xuanzhuo@linux.alibaba.com
## AF_XDP
XDP socket(AF_XDP) is an excellent bypass kernel network framework. The zero
copy feature of xsk (XDP socket) needs to be supported by the driver. The
performance of zero copy is very good. mlx5 and intel ixgbe already support
this feature, This patch set allows virtio-net to support xsk's zerocopy xmit
feature.
At present, we have completed some preparation:
1. vq-reset (virtio spec and kernel code)
2. virtio-core premapped dma
3. virtio-net xdp refactor
So it is time for Virtio-Net to complete the support for the XDP Socket
Zerocopy.
Virtio-net can not increase the queue num at will, so xsk shares the queue with
kernel.
On the other hand, Virtio-Net does not support generate interrupt from driver
manually, so when we wakeup tx xmit, we used some tips. If the CPU run by TX
NAPI last time is other CPUs, use IPI to wake up NAPI on the remote CPU. If it
is also the local CPU, then we wake up napi directly.
This patch set includes some refactor to the virtio-net to let that to support
AF_XDP.
## performance
ENV: Qemu with vhost-user(polling mode).
Host CPU: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz
### virtio PMD in guest with testpmd
testpmd> show port stats all
######################## NIC statistics for port 0 ########################
RX-packets: 19531092064 RX-missed: 0 RX-bytes: 1093741155584
RX-errors: 0
RX-nombuf: 0
TX-packets: 5959955552 TX-errors: 0 TX-bytes: 371030645664
Throughput (since last show)
Rx-pps: 8861574 Rx-bps: 3969985208
Tx-pps: 8861493 Tx-bps: 3969962736
############################################################################
### AF_XDP PMD in guest with testpmd
testpmd> show port stats all
######################## NIC statistics for port 0 ########################
RX-packets: 68152727 RX-missed: 0 RX-bytes: 3816552712
RX-errors: 0
RX-nombuf: 0
TX-packets: 68114967 TX-errors: 33216 TX-bytes: 3814438152
Throughput (since last show)
Rx-pps: 6333196 Rx-bps: 2837272088
Tx-pps: 6333227 Tx-bps: 2837285936
############################################################################
But AF_XDP consumes more CPU for tx and rx napi(100% and 86%).
## maintain
I am currently a reviewer for virtio-net. I commit to maintain AF_XDP support in
virtio-net.
Please review.
Thanks.
v3
1. virtio introduces helpers for virtio-net sq using premapped dma
2. xsk has more complete support for merge mode
3. fix some problems
v2
1. wakeup uses the way of GVE. No send ipi to wakeup napi on remote cpu.
2. remove rcu. Because we synchronize all operat, so the rcu is not needed.
3. split the commit "move to virtio_net.h" in last patch set. Just move the
struct/api to header when we use them.
4. add comments for some code
v1:
1. remove two virtio commits. Push this patchset to net-next
2. squash "virtio_net: virtnet_poll_tx support rescheduled" to xsk: support tx
3. fix some warnings
Xuan Zhuo (17):
virtio_net: separate virtnet_rx_resize()
virtio_net: separate virtnet_tx_resize()
virtio_net: xsk: bind/unbind xsk
virtio_net: xsk: prevent disable tx napi
virtio_net: move some api to header
virtio_net: xsk: tx: support xmit xsk buffer
virtio_net: xsk: tx: support wakeup
virtio_net: xsk: tx: handle the transmitted xsk buffer
virtio_net: xsk: tx: free the unused xsk buffer
virtio_net: separate receive_mergeable
virtio_net: separate receive_buf
virtio_net: xsk: rx: support fill with xsk buffer
virtio_net: xsk: rx: support recv merge mode
virtio_net: xsk: rx: support recv small mode
virtio_net: xsk: rx: free the unused xsk buffer
virtio_net: update tx timeout record
virtio_net: xdp_features add NETDEV_XDP_ACT_XSK_ZEROCOPY
drivers/net/virtio/Makefile | 2 +-
drivers/net/virtio/main.c | 409 +++++++++++----------
drivers/net/virtio/virtio_net.h | 140 +++++++
drivers/net/virtio/xsk.c | 622 ++++++++++++++++++++++++++++++++
drivers/net/virtio/xsk.h | 32 ++
5 files changed, 1014 insertions(+), 191 deletions(-)
create mode 100644 drivers/net/virtio/xsk.c
create mode 100644 drivers/net/virtio/xsk.h
--
2.32.0.3.g01195cf9f
next reply other threads:[~2024-01-16 9:43 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-16 9:42 Xuan Zhuo [this message]
2024-01-16 9:42 ` [PATCH net-next 01/17] virtio_net: separate virtnet_rx_resize() Xuan Zhuo
2024-01-16 9:42 ` [PATCH net-next 02/17] virtio_net: separate virtnet_tx_resize() Xuan Zhuo
2024-01-16 9:42 ` [PATCH net-next 03/17] virtio_net: xsk: bind/unbind xsk Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 04/17] virtio_net: xsk: prevent disable tx napi Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 05/17] virtio_net: move some api to header Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 06/17] virtio_net: xsk: tx: support xmit xsk buffer Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 07/17] virtio_net: xsk: tx: support wakeup Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 08/17] virtio_net: xsk: tx: handle the transmitted xsk buffer Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 09/17] virtio_net: xsk: tx: free the unused " Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 10/17] virtio_net: separate receive_mergeable Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 11/17] virtio_net: separate receive_buf Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 12/17] virtio_net: xsk: rx: support fill with xsk buffer Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 13/17] virtio_net: xsk: rx: support recv merge mode Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 14/17] virtio_net: xsk: rx: support recv small mode Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 15/17] virtio_net: xsk: rx: free the unused xsk buffer Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 16/17] virtio_net: update tx timeout record Xuan Zhuo
2024-01-16 9:43 ` [PATCH net-next 17/17] virtio_net: xdp_features add NETDEV_XDP_ACT_XSK_ZEROCOPY Xuan Zhuo
2024-01-16 12:37 ` [PATCH net-next 00/17] virtio-net: support AF_XDP zero copy (3/3) Paolo Abeni
2024-01-16 15:07 ` Jakub Kicinski
2024-01-16 20:46 ` Michael S. Tsirkin
2024-01-17 5:53 ` Xuan Zhuo
2024-01-17 5:55 ` Xuan Zhuo
2024-01-22 4:24 ` Jason Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240116094313.119939-1-xuanzhuo@linux.alibaba.com \
--to=xuanzhuo@linux.alibaba.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=hawk@kernel.org \
--cc=jasowang@redhat.com \
--cc=john.fastabend@gmail.com \
--cc=kuba@kernel.org \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=virtualization@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).