* Re: [PATCH net-next v2 2/7] virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx [not found] <1610970895.0597434-1-xuanzhuo@linux.alibaba.com> @ 2021-01-20 3:23 ` Jason Wang 0 siblings, 0 replies; 3+ messages in thread From: Jason Wang @ 2021-01-20 3:23 UTC (permalink / raw) To: Xuan Zhuo Cc: Michael S. Tsirkin, David S. Miller, Jakub Kicinski, Björn Töpel, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, virtualization, bpf, netdev On 2021/1/18 下午7:54, Xuan Zhuo wrote: > On Mon, 18 Jan 2021 14:45:16 +0800, Jason Wang <jasowang@redhat.com> wrote: >> On 2021/1/16 上午10:59, Xuan Zhuo wrote: >>> If support xsk, a new ptr will be recovered during the >>> process of freeing the old ptr. In order to distinguish between ctx sent >>> by XDP_TX and ctx sent by xsk, a struct is added here to distinguish >>> between these two situations. virtnet_xdp_type.type It is used to >>> distinguish different ctx, and virtnet_xdp_type.offset is used to record >>> the offset between "true ctx" and virtnet_xdp_type. >>> >>> The newly added virtnet_xsk_hdr will be used for xsk. >>> >>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com> >> >> Any reason that you can't simply encode the type in the pointer itself >> as we used to do? >> >> #define VIRTIO_XSK_FLAG BIT(1) >> >> ? > Since xdp socket does not use xdp_frame, we will encounter three data types when > recycling: skb, xdp_frame, xdp socket. Just to make sure we are in the same page. Currently, the pointer type is encoded with 1 bit in the pointer. Can we simply use 2 bit to distinguish skb, xdp, xsk? Thanks > ^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH netdev 0/5] virtio-net support xdp socket zero copy xmit @ 2021-01-05 9:11 Xuan Zhuo 2021-01-16 2:59 ` [PATCH net-next v2 0/7] " Xuan Zhuo 0 siblings, 1 reply; 3+ messages in thread From: Xuan Zhuo @ 2021-01-05 9:11 UTC (permalink / raw) To: netdev Cc: dust.li, tonylu, Michael S. Tsirkin, Jason Wang, David S. Miller, Jakub Kicinski, Björn Töpel, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, open list:VIRTIO CORE AND NET DRIVERS, open list, open list:XDP SOCKETS (AF_XDP) The first patch made some adjustments to xsk. The second patch itself can be used as an independent patch to solve the problem that XDP may fail to load when the number of queues is insufficient. The third to last patch implements support for xsk in virtio-net. A practical problem with virtio is that tx interrupts are not very reliable. There will always be some missing or delayed tx interrupts. So I specially added a point timer to solve this problem. Of course, considering performance issues, The timer only triggers when the ring of the network card is full. Regarding the issue of virtio-net supporting xsk's zero copy rx, I am also developing it, but I found that the modification may be relatively large, so I consider this patch set to be separated from the code related to xsk zero copy rx. Xuan Zhuo (5): xsk: support get page for drv virtio-net: support XDP_TX when not more queues virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx xsk, virtio-net: prepare for support xsk virtio-net, xsk: virtio-net support xsk zero copy tx drivers/net/virtio_net.c | 643 +++++++++++++++++++++++++++++++++++++++----- include/linux/netdevice.h | 1 + include/net/xdp_sock_drv.h | 10 + include/net/xsk_buff_pool.h | 1 + net/xdp/xsk_buff_pool.c | 10 +- 5 files changed, 597 insertions(+), 68 deletions(-) -- 1.8.3.1 ^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH net-next v2 0/7] virtio-net support xdp socket zero copy xmit 2021-01-05 9:11 [PATCH netdev 0/5] virtio-net support xdp socket zero copy xmit Xuan Zhuo @ 2021-01-16 2:59 ` Xuan Zhuo 2021-01-16 2:59 ` [PATCH net-next v2 2/7] virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx Xuan Zhuo 0 siblings, 1 reply; 3+ messages in thread From: Xuan Zhuo @ 2021-01-16 2:59 UTC (permalink / raw) To: netdev Cc: Michael S. Tsirkin, Jason Wang, David S. Miller, Jakub Kicinski, Björn Töpel, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, virtualization, bpf XDP socket is an excellent by pass kernel network transmission framework. The zero copy feature of xsk (XDP socket) needs to be supported by the driver. The performance of zero copy is very good. mlx5 and intel ixgbe already support this feature, This patch set allows virtio-net to support xsk's zerocopy xmit feature. And xsk's zerocopy rx has made major changes to virtio-net, and I hope to submit it after this patch set are received. Compared with other drivers, virtio-net does not directly obtain the dma address, so I first obtain the xsk page, and then pass the page to virtio. When recycling the sent packets, we have to distinguish between skb and xdp. Now we have to distinguish between skb, xdp, xsk. So the second patch solves this problem first. The last four patches are used to support xsk zerocopy in virtio-net: 1. support xsk enable/disable 2. realize the function of xsk packet sending 3. implement xsk wakeup callback 4. set xsk completed when packet sent done ---------------- Performance Testing ------------ The udp package tool implemented by the interface of xsk vs sockperf(kernel udp) for performance testing: xsk zero copy in virtio-net: CPU PPS MSGSIZE 28.7% 3833857 64 38.5% 3689491 512 38.9% 2787096 1456 xsk without zero copy in virtio-net: CPU PPS MSGSIZE 100% 1916747 64 100% 1775988 512 100% 1440054 1456 sockperf: CPU PPS MSGSIZE 100% 713274 64 100% 701024 512 100% 695832 1456 Xuan Zhuo (7): xsk: support get page for drv virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx xsk, virtio-net: prepare for support xsk zerocopy xmit virtio-net, xsk: support xsk enable/disable virtio-net, xsk: realize the function of xsk packet sending virtio-net, xsk: implement xsk wakeup callback virtio-net, xsk: set xsk completed when packet sent done drivers/net/virtio_net.c | 559 +++++++++++++++++++++++++++++++++++++++----- include/linux/netdevice.h | 1 + include/net/xdp_sock_drv.h | 10 + include/net/xsk_buff_pool.h | 1 + net/xdp/xsk_buff_pool.c | 10 +- 5 files changed, 523 insertions(+), 58 deletions(-) -- 1.8.3.1 ^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH net-next v2 2/7] virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx 2021-01-16 2:59 ` [PATCH net-next v2 0/7] " Xuan Zhuo @ 2021-01-16 2:59 ` Xuan Zhuo 2021-01-18 6:45 ` Jason Wang 0 siblings, 1 reply; 3+ messages in thread From: Xuan Zhuo @ 2021-01-16 2:59 UTC (permalink / raw) To: netdev Cc: Michael S. Tsirkin, Jason Wang, David S. Miller, Jakub Kicinski, Björn Töpel, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, virtualization, bpf If support xsk, a new ptr will be recovered during the process of freeing the old ptr. In order to distinguish between ctx sent by XDP_TX and ctx sent by xsk, a struct is added here to distinguish between these two situations. virtnet_xdp_type.type It is used to distinguish different ctx, and virtnet_xdp_type.offset is used to record the offset between "true ctx" and virtnet_xdp_type. The newly added virtnet_xsk_hdr will be used for xsk. Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com> --- drivers/net/virtio_net.c | 75 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 60 insertions(+), 15 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index ba8e637..e707c31 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -94,6 +94,22 @@ struct virtnet_rq_stats { u64 kicks; }; +enum { + XDP_TYPE_XSK, + XDP_TYPE_TX, +}; + +struct virtnet_xdp_type { + int offset:24; + unsigned type:8; +}; + +struct virtnet_xsk_hdr { + struct virtnet_xdp_type type; + struct virtio_net_hdr_mrg_rxbuf hdr; + u32 len; +}; + #define VIRTNET_SQ_STAT(m) offsetof(struct virtnet_sq_stats, m) #define VIRTNET_RQ_STAT(m) offsetof(struct virtnet_rq_stats, m) @@ -251,14 +267,19 @@ static bool is_xdp_frame(void *ptr) return (unsigned long)ptr & VIRTIO_XDP_FLAG; } -static void *xdp_to_ptr(struct xdp_frame *ptr) +static void *xdp_to_ptr(struct virtnet_xdp_type *ptr) { return (void *)((unsigned long)ptr | VIRTIO_XDP_FLAG); } -static struct xdp_frame *ptr_to_xdp(void *ptr) +static struct virtnet_xdp_type *ptr_to_xtype(void *ptr) +{ + return (struct virtnet_xdp_type *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); +} + +static void *xtype_get_ptr(struct virtnet_xdp_type *xdptype) { - return (struct xdp_frame *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); + return (char *)xdptype + xdptype->offset; } /* Converting between virtqueue no. and kernel tx/rx queue no. @@ -459,11 +480,16 @@ static int __virtnet_xdp_xmit_one(struct virtnet_info *vi, struct xdp_frame *xdpf) { struct virtio_net_hdr_mrg_rxbuf *hdr; + struct virtnet_xdp_type *xdptype; int err; - if (unlikely(xdpf->headroom < vi->hdr_len)) + if (unlikely(xdpf->headroom < vi->hdr_len + sizeof(*xdptype))) return -EOVERFLOW; + xdptype = (struct virtnet_xdp_type *)(xdpf + 1); + xdptype->offset = (char *)xdpf - (char *)xdptype; + xdptype->type = XDP_TYPE_TX; + /* Make room for virtqueue hdr (also change xdpf->headroom?) */ xdpf->data -= vi->hdr_len; /* Zero header and leave csum up to XDP layers */ @@ -473,7 +499,7 @@ static int __virtnet_xdp_xmit_one(struct virtnet_info *vi, sg_init_one(sq->sg, xdpf->data, xdpf->len); - err = virtqueue_add_outbuf(sq->vq, sq->sg, 1, xdp_to_ptr(xdpf), + err = virtqueue_add_outbuf(sq->vq, sq->sg, 1, xdp_to_ptr(xdptype), GFP_ATOMIC); if (unlikely(err)) return -ENOSPC; /* Caller handle free/refcnt */ @@ -523,8 +549,11 @@ static int virtnet_xdp_xmit(struct net_device *dev, /* Free up any pending old buffers before queueing new ones. */ while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { if (likely(is_xdp_frame(ptr))) { - struct xdp_frame *frame = ptr_to_xdp(ptr); + struct virtnet_xdp_type *xtype; + struct xdp_frame *frame; + xtype = ptr_to_xtype(ptr); + frame = xtype_get_ptr(xtype); bytes += frame->len; xdp_return_frame(frame); } else { @@ -1373,24 +1402,34 @@ static int virtnet_receive(struct receive_queue *rq, int budget, static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi) { - unsigned int len; unsigned int packets = 0; unsigned int bytes = 0; - void *ptr; + unsigned int len; + struct virtnet_xdp_type *xtype; + struct xdp_frame *frame; + struct virtnet_xsk_hdr *xskhdr; + struct sk_buff *skb; + void *ptr; while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { if (likely(!is_xdp_frame(ptr))) { - struct sk_buff *skb = ptr; + skb = ptr; pr_debug("Sent skb %p\n", skb); bytes += skb->len; napi_consume_skb(skb, in_napi); } else { - struct xdp_frame *frame = ptr_to_xdp(ptr); + xtype = ptr_to_xtype(ptr); - bytes += frame->len; - xdp_return_frame(frame); + if (xtype->type == XDP_TYPE_XSK) { + xskhdr = (struct virtnet_xsk_hdr *)xtype; + bytes += xskhdr->len; + } else { + frame = xtype_get_ptr(xtype); + xdp_return_frame(frame); + bytes += frame->len; + } } packets++; } @@ -2659,10 +2698,16 @@ static void free_unused_bufs(struct virtnet_info *vi) for (i = 0; i < vi->max_queue_pairs; i++) { struct virtqueue *vq = vi->sq[i].vq; while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) { - if (!is_xdp_frame(buf)) + if (!is_xdp_frame(buf)) { dev_kfree_skb(buf); - else - xdp_return_frame(ptr_to_xdp(buf)); + } else { + struct virtnet_xdp_type *xtype; + + xtype = ptr_to_xtype(buf); + + if (xtype->type != XDP_TYPE_XSK) + xdp_return_frame(xtype_get_ptr(xtype)); + } } } -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH net-next v2 2/7] virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx 2021-01-16 2:59 ` [PATCH net-next v2 2/7] virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx Xuan Zhuo @ 2021-01-18 6:45 ` Jason Wang 0 siblings, 0 replies; 3+ messages in thread From: Jason Wang @ 2021-01-18 6:45 UTC (permalink / raw) To: Xuan Zhuo, netdev Cc: Michael S. Tsirkin, David S. Miller, Jakub Kicinski, Björn Töpel, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh, virtualization, bpf On 2021/1/16 上午10:59, Xuan Zhuo wrote: > If support xsk, a new ptr will be recovered during the > process of freeing the old ptr. In order to distinguish between ctx sent > by XDP_TX and ctx sent by xsk, a struct is added here to distinguish > between these two situations. virtnet_xdp_type.type It is used to > distinguish different ctx, and virtnet_xdp_type.offset is used to record > the offset between "true ctx" and virtnet_xdp_type. > > The newly added virtnet_xsk_hdr will be used for xsk. > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com> Any reason that you can't simply encode the type in the pointer itself as we used to do? #define VIRTIO_XSK_FLAG BIT(1) ? > --- > drivers/net/virtio_net.c | 75 ++++++++++++++++++++++++++++++++++++++---------- > 1 file changed, 60 insertions(+), 15 deletions(-) > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index ba8e637..e707c31 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -94,6 +94,22 @@ struct virtnet_rq_stats { > u64 kicks; > }; > > +enum { > + XDP_TYPE_XSK, > + XDP_TYPE_TX, > +}; > + > +struct virtnet_xdp_type { > + int offset:24; > + unsigned type:8; > +}; > + > +struct virtnet_xsk_hdr { > + struct virtnet_xdp_type type; > + struct virtio_net_hdr_mrg_rxbuf hdr; > + u32 len; > +}; > + > #define VIRTNET_SQ_STAT(m) offsetof(struct virtnet_sq_stats, m) > #define VIRTNET_RQ_STAT(m) offsetof(struct virtnet_rq_stats, m) > > @@ -251,14 +267,19 @@ static bool is_xdp_frame(void *ptr) > return (unsigned long)ptr & VIRTIO_XDP_FLAG; > } > > -static void *xdp_to_ptr(struct xdp_frame *ptr) > +static void *xdp_to_ptr(struct virtnet_xdp_type *ptr) > { > return (void *)((unsigned long)ptr | VIRTIO_XDP_FLAG); > } > > -static struct xdp_frame *ptr_to_xdp(void *ptr) > +static struct virtnet_xdp_type *ptr_to_xtype(void *ptr) > +{ > + return (struct virtnet_xdp_type *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); > +} > + > +static void *xtype_get_ptr(struct virtnet_xdp_type *xdptype) > { > - return (struct xdp_frame *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); > + return (char *)xdptype + xdptype->offset; > } > > /* Converting between virtqueue no. and kernel tx/rx queue no. > @@ -459,11 +480,16 @@ static int __virtnet_xdp_xmit_one(struct virtnet_info *vi, > struct xdp_frame *xdpf) > { > struct virtio_net_hdr_mrg_rxbuf *hdr; > + struct virtnet_xdp_type *xdptype; > int err; > > - if (unlikely(xdpf->headroom < vi->hdr_len)) > + if (unlikely(xdpf->headroom < vi->hdr_len + sizeof(*xdptype))) > return -EOVERFLOW; > > + xdptype = (struct virtnet_xdp_type *)(xdpf + 1); > + xdptype->offset = (char *)xdpf - (char *)xdptype; > + xdptype->type = XDP_TYPE_TX; > + > /* Make room for virtqueue hdr (also change xdpf->headroom?) */ > xdpf->data -= vi->hdr_len; > /* Zero header and leave csum up to XDP layers */ > @@ -473,7 +499,7 @@ static int __virtnet_xdp_xmit_one(struct virtnet_info *vi, > > sg_init_one(sq->sg, xdpf->data, xdpf->len); > > - err = virtqueue_add_outbuf(sq->vq, sq->sg, 1, xdp_to_ptr(xdpf), > + err = virtqueue_add_outbuf(sq->vq, sq->sg, 1, xdp_to_ptr(xdptype), > GFP_ATOMIC); > if (unlikely(err)) > return -ENOSPC; /* Caller handle free/refcnt */ > @@ -523,8 +549,11 @@ static int virtnet_xdp_xmit(struct net_device *dev, > /* Free up any pending old buffers before queueing new ones. */ > while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { > if (likely(is_xdp_frame(ptr))) { > - struct xdp_frame *frame = ptr_to_xdp(ptr); > + struct virtnet_xdp_type *xtype; > + struct xdp_frame *frame; > > + xtype = ptr_to_xtype(ptr); > + frame = xtype_get_ptr(xtype); > bytes += frame->len; > xdp_return_frame(frame); > } else { > @@ -1373,24 +1402,34 @@ static int virtnet_receive(struct receive_queue *rq, int budget, > > static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi) > { > - unsigned int len; > unsigned int packets = 0; > unsigned int bytes = 0; > - void *ptr; > + unsigned int len; > + struct virtnet_xdp_type *xtype; > + struct xdp_frame *frame; > + struct virtnet_xsk_hdr *xskhdr; > + struct sk_buff *skb; > + void *ptr; > > while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { > if (likely(!is_xdp_frame(ptr))) { > - struct sk_buff *skb = ptr; > + skb = ptr; > > pr_debug("Sent skb %p\n", skb); > > bytes += skb->len; > napi_consume_skb(skb, in_napi); > } else { > - struct xdp_frame *frame = ptr_to_xdp(ptr); > + xtype = ptr_to_xtype(ptr); > > - bytes += frame->len; > - xdp_return_frame(frame); > + if (xtype->type == XDP_TYPE_XSK) { > + xskhdr = (struct virtnet_xsk_hdr *)xtype; > + bytes += xskhdr->len; > + } else { > + frame = xtype_get_ptr(xtype); > + xdp_return_frame(frame); > + bytes += frame->len; > + } > } > packets++; > } > @@ -2659,10 +2698,16 @@ static void free_unused_bufs(struct virtnet_info *vi) > for (i = 0; i < vi->max_queue_pairs; i++) { > struct virtqueue *vq = vi->sq[i].vq; > while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) { > - if (!is_xdp_frame(buf)) > + if (!is_xdp_frame(buf)) { > dev_kfree_skb(buf); > - else > - xdp_return_frame(ptr_to_xdp(buf)); > + } else { > + struct virtnet_xdp_type *xtype; > + > + xtype = ptr_to_xtype(buf); > + > + if (xtype->type != XDP_TYPE_XSK) > + xdp_return_frame(xtype_get_ptr(xtype)); > + } > } > } > ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-01-20 3:25 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <1610970895.0597434-1-xuanzhuo@linux.alibaba.com> 2021-01-20 3:23 ` [PATCH net-next v2 2/7] virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx Jason Wang 2021-01-05 9:11 [PATCH netdev 0/5] virtio-net support xdp socket zero copy xmit Xuan Zhuo 2021-01-16 2:59 ` [PATCH net-next v2 0/7] " Xuan Zhuo 2021-01-16 2:59 ` [PATCH net-next v2 2/7] virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx Xuan Zhuo 2021-01-18 6:45 ` Jason Wang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).