From: "Michael S. Tsirkin" <mst@redhat.com> To: Xuan Zhuo <xuanzhuo@linux.alibaba.com> Cc: virtualization@lists.linux-foundation.org, Jason Wang <jasowang@redhat.com>, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, Alexei Starovoitov <ast@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, Jesper Dangaard Brouer <hawk@kernel.org>, John Fastabend <john.fastabend@gmail.com>, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: Re: [PATCH vhost v10 00/10] virtio core prepares for AF_XDP Date: Thu, 22 Jun 2023 15:38:11 -0400 [thread overview] Message-ID: <20230622153730-mutt-send-email-mst@kernel.org> (raw) In-Reply-To: <20230602092206.50108-1-xuanzhuo@linux.alibaba.com> On Fri, Jun 02, 2023 at 05:21:56PM +0800, Xuan Zhuo wrote: > ## About DMA APIs > > Now, virtio may can not work with DMA APIs when virtio features do not have > VIRTIO_F_ACCESS_PLATFORM. > > 1. I tried to let DMA APIs return phy address by virtio-device. But DMA APIs just > work with the "real" devices. > 2. I tried to let xsk support callballs to get phy address from virtio-net > driver as the dma address. But the maintainers of xsk may want to use dma-buf > to replace the DMA APIs. I think that may be a larger effort. We will wait > too long. > > So rethinking this, firstly, we can support premapped-dma only for devices with > VIRTIO_F_ACCESS_PLATFORM. In the case of af-xdp, if the users want to use it, > they have to update the device to support VIRTIO_F_RING_RESET, and they can also > enable the device's VIRTIO_F_ACCESS_PLATFORM feature. > > Thanks for the help from Christoph. > > ================= > > XDP socket(AF_XDP) is an excellent bypass kernel network framework. The zero > copy feature of xsk (XDP socket) needs to be supported by the driver. The > performance of zero copy is very good. > > ENV: Qemu with vhost. > > vhost cpu | Guest APP CPU |Guest Softirq CPU | PPS > -----------------------------|---------------|------------------|------------ > xmit by sockperf: 90% | 100% | | 318967 > xmit by xsk: 100% | 30% | 33% | 1192064 > recv by sockperf: 100% | 68% | 100% | 692288 > recv by xsk: 100% | 33% | 43% | 771670 > > Before achieving the function of Virtio-Net, we also have to let virtio core > support these features: So by itself, this doesn't do this. But what effect does all this overhead have on performance? > 1. virtio core support premapped > 2. virtio core support reset per-queue > 3. introduce DMA APIs to virtio core > > Please review. > > Thanks. > > v10: > 1. support to set vq to premapped mode, then the vq just handles the premapped request. > 2. virtio-net support to do dma mapping in advance > > v9: > 1. use flag to distinguish the premapped operations. no do judgment by sg. > > v8: > 1. vring_sg_address: check by sg_page(sg) not dma_address. Because 0 is a valid dma address > 2. remove unused code from vring_map_one_sg() > > v7: > 1. virtqueue_dma_dev() return NULL when virtio is without DMA API. > > v6: > 1. change the size of the flags to u32. > > v5: > 1. fix for error handler > 2. add flags to record internal dma mapping > > v4: > 1. rename map_inter to dma_map_internal > 2. fix: Excess function parameter 'vq' description in 'virtqueue_dma_dev' > > v3: > 1. add map_inter to struct desc state to reocrd whether virtio core do dma map > > v2: > 1. based on sgs[0]->dma_address to judgment is premapped > 2. based on extra.addr to judgment to do unmap for no-indirect desc > 3. based on indir_desc to judgment to do unmap for indirect desc > 4. rename virtqueue_get_dma_dev to virtqueue_dma_dev > > v1: > 1. expose dma device. NO introduce the api for dma and sync > 2. split some commit for review. > > > > > Xuan Zhuo (10): > virtio_ring: put mapping error check in vring_map_one_sg > virtio_ring: introduce virtqueue_set_premapped() > virtio_ring: split: support add premapped buf > virtio_ring: packed: support add premapped buf > virtio_ring: split-detach: support return dma info to driver > virtio_ring: packed-detach: support return dma info to driver > virtio_ring: introduce helpers for premapped > virtio_ring: introduce virtqueue_dma_dev() > virtio_ring: introduce virtqueue_add_sg() > virtio_net: support dma premapped > > drivers/net/virtio_net.c | 163 ++++++++++-- > drivers/virtio/virtio_ring.c | 493 +++++++++++++++++++++++++++++++---- > include/linux/virtio.h | 34 +++ > 3 files changed, 612 insertions(+), 78 deletions(-) > > -- > 2.32.0.3.g01195cf9f
WARNING: multiple messages have this Message-ID (diff)
From: "Michael S. Tsirkin" <mst@redhat.com> To: Xuan Zhuo <xuanzhuo@linux.alibaba.com> Cc: Jesper Dangaard Brouer <hawk@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, netdev@vger.kernel.org, John Fastabend <john.fastabend@gmail.com>, Alexei Starovoitov <ast@kernel.org>, virtualization@lists.linux-foundation.org, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, bpf@vger.kernel.org, Paolo Abeni <pabeni@redhat.com>, "David S. Miller" <davem@davemloft.net> Subject: Re: [PATCH vhost v10 00/10] virtio core prepares for AF_XDP Date: Thu, 22 Jun 2023 15:38:11 -0400 [thread overview] Message-ID: <20230622153730-mutt-send-email-mst@kernel.org> (raw) In-Reply-To: <20230602092206.50108-1-xuanzhuo@linux.alibaba.com> On Fri, Jun 02, 2023 at 05:21:56PM +0800, Xuan Zhuo wrote: > ## About DMA APIs > > Now, virtio may can not work with DMA APIs when virtio features do not have > VIRTIO_F_ACCESS_PLATFORM. > > 1. I tried to let DMA APIs return phy address by virtio-device. But DMA APIs just > work with the "real" devices. > 2. I tried to let xsk support callballs to get phy address from virtio-net > driver as the dma address. But the maintainers of xsk may want to use dma-buf > to replace the DMA APIs. I think that may be a larger effort. We will wait > too long. > > So rethinking this, firstly, we can support premapped-dma only for devices with > VIRTIO_F_ACCESS_PLATFORM. In the case of af-xdp, if the users want to use it, > they have to update the device to support VIRTIO_F_RING_RESET, and they can also > enable the device's VIRTIO_F_ACCESS_PLATFORM feature. > > Thanks for the help from Christoph. > > ================= > > XDP socket(AF_XDP) is an excellent bypass kernel network framework. The zero > copy feature of xsk (XDP socket) needs to be supported by the driver. The > performance of zero copy is very good. > > ENV: Qemu with vhost. > > vhost cpu | Guest APP CPU |Guest Softirq CPU | PPS > -----------------------------|---------------|------------------|------------ > xmit by sockperf: 90% | 100% | | 318967 > xmit by xsk: 100% | 30% | 33% | 1192064 > recv by sockperf: 100% | 68% | 100% | 692288 > recv by xsk: 100% | 33% | 43% | 771670 > > Before achieving the function of Virtio-Net, we also have to let virtio core > support these features: So by itself, this doesn't do this. But what effect does all this overhead have on performance? > 1. virtio core support premapped > 2. virtio core support reset per-queue > 3. introduce DMA APIs to virtio core > > Please review. > > Thanks. > > v10: > 1. support to set vq to premapped mode, then the vq just handles the premapped request. > 2. virtio-net support to do dma mapping in advance > > v9: > 1. use flag to distinguish the premapped operations. no do judgment by sg. > > v8: > 1. vring_sg_address: check by sg_page(sg) not dma_address. Because 0 is a valid dma address > 2. remove unused code from vring_map_one_sg() > > v7: > 1. virtqueue_dma_dev() return NULL when virtio is without DMA API. > > v6: > 1. change the size of the flags to u32. > > v5: > 1. fix for error handler > 2. add flags to record internal dma mapping > > v4: > 1. rename map_inter to dma_map_internal > 2. fix: Excess function parameter 'vq' description in 'virtqueue_dma_dev' > > v3: > 1. add map_inter to struct desc state to reocrd whether virtio core do dma map > > v2: > 1. based on sgs[0]->dma_address to judgment is premapped > 2. based on extra.addr to judgment to do unmap for no-indirect desc > 3. based on indir_desc to judgment to do unmap for indirect desc > 4. rename virtqueue_get_dma_dev to virtqueue_dma_dev > > v1: > 1. expose dma device. NO introduce the api for dma and sync > 2. split some commit for review. > > > > > Xuan Zhuo (10): > virtio_ring: put mapping error check in vring_map_one_sg > virtio_ring: introduce virtqueue_set_premapped() > virtio_ring: split: support add premapped buf > virtio_ring: packed: support add premapped buf > virtio_ring: split-detach: support return dma info to driver > virtio_ring: packed-detach: support return dma info to driver > virtio_ring: introduce helpers for premapped > virtio_ring: introduce virtqueue_dma_dev() > virtio_ring: introduce virtqueue_add_sg() > virtio_net: support dma premapped > > drivers/net/virtio_net.c | 163 ++++++++++-- > drivers/virtio/virtio_ring.c | 493 +++++++++++++++++++++++++++++++---- > include/linux/virtio.h | 34 +++ > 3 files changed, 612 insertions(+), 78 deletions(-) > > -- > 2.32.0.3.g01195cf9f _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2023-06-22 19:38 UTC|newest] Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-06-02 9:21 [PATCH vhost v10 00/10] virtio core prepares for AF_XDP Xuan Zhuo 2023-06-02 9:21 ` Xuan Zhuo 2023-06-02 9:21 ` [PATCH vhost v10 01/10] virtio_ring: put mapping error check in vring_map_one_sg Xuan Zhuo 2023-06-02 9:21 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-02 9:21 ` [PATCH vhost v10 02/10] virtio_ring: introduce virtqueue_set_premapped() Xuan Zhuo 2023-06-02 9:21 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:50 ` Xuan Zhuo 2023-06-27 8:50 ` Xuan Zhuo 2023-06-27 14:56 ` Michael S. Tsirkin 2023-06-27 14:56 ` Michael S. Tsirkin 2023-06-28 1:34 ` Xuan Zhuo 2023-06-28 1:34 ` Xuan Zhuo 2023-06-02 9:21 ` [PATCH vhost v10 03/10] virtio_ring: split: support add premapped buf Xuan Zhuo 2023-06-02 9:21 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 9:01 ` Xuan Zhuo 2023-06-27 9:01 ` Xuan Zhuo 2023-06-28 4:07 ` Jason Wang 2023-06-28 6:00 ` Xuan Zhuo 2023-06-28 6:51 ` Jason Wang 2023-06-02 9:22 ` [PATCH vhost v10 04/10] virtio_ring: packed: " Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 9:05 ` Xuan Zhuo 2023-06-27 9:05 ` Xuan Zhuo 2023-06-02 9:22 ` [PATCH vhost v10 05/10] virtio_ring: split-detach: support return dma info to driver Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-22 19:36 ` Michael S. Tsirkin 2023-06-22 19:36 ` Michael S. Tsirkin 2023-06-25 2:10 ` Xuan Zhuo 2023-06-25 2:10 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 9:21 ` Xuan Zhuo 2023-06-27 9:21 ` Xuan Zhuo 2023-06-02 9:22 ` [PATCH vhost v10 06/10] virtio_ring: packed-detach: " Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-02 11:40 ` Michael S. Tsirkin 2023-06-02 11:40 ` Michael S. Tsirkin 2023-06-02 9:22 ` [PATCH vhost v10 07/10] virtio_ring: introduce helpers for premapped Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-04 13:45 ` Michael S. Tsirkin 2023-06-04 13:45 ` Michael S. Tsirkin 2023-06-05 2:06 ` Xuan Zhuo 2023-06-05 2:06 ` Xuan Zhuo 2023-06-05 5:38 ` Michael S. Tsirkin 2023-06-05 5:38 ` Michael S. Tsirkin 2023-06-06 2:01 ` Xuan Zhuo 2023-06-06 2:01 ` Xuan Zhuo 2023-06-22 19:29 ` Michael S. Tsirkin 2023-06-22 19:29 ` Michael S. Tsirkin 2023-06-02 9:22 ` [PATCH vhost v10 08/10] virtio_ring: introduce virtqueue_dma_dev() Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-02 9:22 ` [PATCH vhost v10 09/10] virtio_ring: introduce virtqueue_add_sg() Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-02 9:22 ` [PATCH vhost v10 10/10] virtio_net: support dma premapped Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-03 6:31 ` Jakub Kicinski 2023-06-05 2:10 ` Xuan Zhuo 2023-06-05 2:10 ` Xuan Zhuo 2023-06-05 5:44 ` Michael S. Tsirkin 2023-06-05 5:44 ` Michael S. Tsirkin 2023-06-06 2:11 ` Xuan Zhuo 2023-06-06 2:11 ` Xuan Zhuo 2023-06-22 12:15 ` Michael S. Tsirkin 2023-06-22 12:15 ` Michael S. Tsirkin 2023-06-25 2:43 ` Xuan Zhuo 2023-06-25 2:43 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 9:23 ` Xuan Zhuo 2023-06-27 9:23 ` Xuan Zhuo 2023-06-03 6:29 ` [PATCH vhost v10 00/10] virtio core prepares for AF_XDP Jakub Kicinski 2023-06-05 1:58 ` Xuan Zhuo 2023-06-05 1:58 ` Xuan Zhuo 2023-06-07 14:05 ` Christoph Hellwig 2023-06-07 14:05 ` Christoph Hellwig 2023-06-07 20:15 ` Michael S. Tsirkin 2023-06-07 20:15 ` Michael S. Tsirkin 2023-06-21 6:42 ` Xuan Zhuo 2023-06-21 6:42 ` Xuan Zhuo 2023-06-25 7:19 ` Jason Wang 2023-06-25 7:19 ` Jason Wang 2023-06-22 19:38 ` Michael S. Tsirkin [this message] 2023-06-22 19:38 ` Michael S. Tsirkin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230622153730-mutt-send-email-mst@kernel.org \ --to=mst@redhat.com \ --cc=ast@kernel.org \ --cc=bpf@vger.kernel.org \ --cc=daniel@iogearbox.net \ --cc=davem@davemloft.net \ --cc=edumazet@google.com \ --cc=hawk@kernel.org \ --cc=jasowang@redhat.com \ --cc=john.fastabend@gmail.com \ --cc=kuba@kernel.org \ --cc=netdev@vger.kernel.org \ --cc=pabeni@redhat.com \ --cc=virtualization@lists.linux-foundation.org \ --cc=xuanzhuo@linux.alibaba.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.