From: "Michael S. Tsirkin" <mst@redhat.com> To: Xuan Zhuo <xuanzhuo@linux.alibaba.com> Cc: Jesper Dangaard Brouer <hawk@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, netdev@vger.kernel.org, John Fastabend <john.fastabend@gmail.com>, Alexei Starovoitov <ast@kernel.org>, virtualization@lists.linux-foundation.org, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, bpf@vger.kernel.org, Paolo Abeni <pabeni@redhat.com>, "David S. Miller" <davem@davemloft.net> Subject: Re: [PATCH vhost v10 02/10] virtio_ring: introduce virtqueue_set_premapped() Date: Tue, 27 Jun 2023 10:56:54 -0400 [thread overview] Message-ID: <20230627105503-mutt-send-email-mst@kernel.org> (raw) In-Reply-To: <1687855801.1280077-4-xuanzhuo@linux.alibaba.com> On Tue, Jun 27, 2023 at 04:50:01PM +0800, Xuan Zhuo wrote: > On Tue, 27 Jun 2023 16:03:23 +0800, Jason Wang <jasowang@redhat.com> wrote: > > On Fri, Jun 2, 2023 at 5:22 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote: > > > > > > This helper allows the driver change the dma mode to premapped mode. > > > Under the premapped mode, the virtio core do not do dma mapping > > > internally. > > > > > > This just work when the use_dma_api is true. If the use_dma_api is false, > > > the dma options is not through the DMA APIs, that is not the standard > > > way of the linux kernel. > > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com> > > > --- > > > drivers/virtio/virtio_ring.c | 40 ++++++++++++++++++++++++++++++++++++ > > > include/linux/virtio.h | 2 ++ > > > 2 files changed, 42 insertions(+) > > > > > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c > > > index 72ed07a604d4..2afdfb9e3e30 100644 > > > --- a/drivers/virtio/virtio_ring.c > > > +++ b/drivers/virtio/virtio_ring.c > > > @@ -172,6 +172,9 @@ struct vring_virtqueue { > > > /* Host publishes avail event idx */ > > > bool event; > > > > > > + /* Do DMA mapping by driver */ > > > + bool premapped; > > > + > > > /* Head of free buffer list. */ > > > unsigned int free_head; > > > /* Number we've added since last sync. */ > > > @@ -2059,6 +2062,7 @@ static struct virtqueue *vring_create_virtqueue_packed( > > > vq->packed_ring = true; > > > vq->dma_dev = dma_dev; > > > vq->use_dma_api = vring_use_dma_api(vdev); > > > + vq->premapped = false; > > > > > > vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && > > > !context; > > > @@ -2548,6 +2552,7 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, > > > #endif > > > vq->dma_dev = dma_dev; > > > vq->use_dma_api = vring_use_dma_api(vdev); > > > + vq->premapped = false; > > > > > > vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && > > > !context; > > > @@ -2691,6 +2696,41 @@ int virtqueue_resize(struct virtqueue *_vq, u32 num, > > > } > > > EXPORT_SYMBOL_GPL(virtqueue_resize); > > > > > > +/** > > > + * virtqueue_set_premapped - set the vring premapped mode > > > + * @_vq: the struct virtqueue we're talking about. > > > + * > > > + * Enable the premapped mode of the vq. > > > + * > > > + * The vring in premapped mode does not do dma internally, so the driver must > > > + * do dma mapping in advance. The driver must pass the dma_address through > > > + * dma_address of scatterlist. When the driver got a used buffer from > > > + * the vring, it has to unmap the dma address. So the driver must call > > > + * virtqueue_get_buf_premapped()/virtqueue_detach_unused_buf_premapped(). > > > + * > > > + * This must be called before adding any buf to vring. > > > > And any old buffer should be detached? > > I mean that before adding any buf, So there are not old buffer. > Oh. So put this in the same sentence: This function must be called immediately after creating the vq, or after vq reset, and before adding any buffers to it. > > > > > + * So this should be called immediately after init vq or vq reset. Do you really need to call this again after each reset? > > Any way to detect and warn in this case? (not a must if it's too > > expensive to do the check) > > > I can try to check whether the qeueu is empty. > > > > > > > + * > > > + * Caller must ensure we don't call this with other virtqueue operations > > > + * at the same time (except where noted). > > > + * > > > + * Returns zero or a negative error. > > > + * 0: success. > > > + * -EINVAL: vring does not use the dma api, so we can not enable premapped mode. > > > + */ > > > +int virtqueue_set_premapped(struct virtqueue *_vq) > > > +{ > > > + struct vring_virtqueue *vq = to_vvq(_vq); > > > + > > > + if (!vq->use_dma_api) > > > + return -EINVAL; > > > + > > > + vq->premapped = true; > > > > I guess there should be a way to disable it. Would it be useful for > > the case when AF_XDP sockets were destroyed? > > Yes. > > When we reset the queue, the vq->premapped will be set to 0. > > The is called after find_vqs or reset vq. > > Thanks. > > > > > > > Thanks > > > > > > > + > > > + return 0; > > > +} > > > +EXPORT_SYMBOL_GPL(virtqueue_set_premapped); > > > + > > > /* Only available for split ring */ > > > struct virtqueue *vring_new_virtqueue(unsigned int index, > > > unsigned int num, > > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h > > > index b93238db94e3..1fc0e1023bd4 100644 > > > --- a/include/linux/virtio.h > > > +++ b/include/linux/virtio.h > > > @@ -78,6 +78,8 @@ bool virtqueue_enable_cb(struct virtqueue *vq); > > > > > > unsigned virtqueue_enable_cb_prepare(struct virtqueue *vq); > > > > > > +int virtqueue_set_premapped(struct virtqueue *_vq); > > > + > > > bool virtqueue_poll(struct virtqueue *vq, unsigned); > > > > > > bool virtqueue_enable_cb_delayed(struct virtqueue *vq); > > > -- > > > 2.32.0.3.g01195cf9f > > > > > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
WARNING: multiple messages have this Message-ID (diff)
From: "Michael S. Tsirkin" <mst@redhat.com> To: Xuan Zhuo <xuanzhuo@linux.alibaba.com> Cc: Jason Wang <jasowang@redhat.com>, virtualization@lists.linux-foundation.org, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, Alexei Starovoitov <ast@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, Jesper Dangaard Brouer <hawk@kernel.org>, John Fastabend <john.fastabend@gmail.com>, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: Re: [PATCH vhost v10 02/10] virtio_ring: introduce virtqueue_set_premapped() Date: Tue, 27 Jun 2023 10:56:54 -0400 [thread overview] Message-ID: <20230627105503-mutt-send-email-mst@kernel.org> (raw) In-Reply-To: <1687855801.1280077-4-xuanzhuo@linux.alibaba.com> On Tue, Jun 27, 2023 at 04:50:01PM +0800, Xuan Zhuo wrote: > On Tue, 27 Jun 2023 16:03:23 +0800, Jason Wang <jasowang@redhat.com> wrote: > > On Fri, Jun 2, 2023 at 5:22 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote: > > > > > > This helper allows the driver change the dma mode to premapped mode. > > > Under the premapped mode, the virtio core do not do dma mapping > > > internally. > > > > > > This just work when the use_dma_api is true. If the use_dma_api is false, > > > the dma options is not through the DMA APIs, that is not the standard > > > way of the linux kernel. > > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com> > > > --- > > > drivers/virtio/virtio_ring.c | 40 ++++++++++++++++++++++++++++++++++++ > > > include/linux/virtio.h | 2 ++ > > > 2 files changed, 42 insertions(+) > > > > > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c > > > index 72ed07a604d4..2afdfb9e3e30 100644 > > > --- a/drivers/virtio/virtio_ring.c > > > +++ b/drivers/virtio/virtio_ring.c > > > @@ -172,6 +172,9 @@ struct vring_virtqueue { > > > /* Host publishes avail event idx */ > > > bool event; > > > > > > + /* Do DMA mapping by driver */ > > > + bool premapped; > > > + > > > /* Head of free buffer list. */ > > > unsigned int free_head; > > > /* Number we've added since last sync. */ > > > @@ -2059,6 +2062,7 @@ static struct virtqueue *vring_create_virtqueue_packed( > > > vq->packed_ring = true; > > > vq->dma_dev = dma_dev; > > > vq->use_dma_api = vring_use_dma_api(vdev); > > > + vq->premapped = false; > > > > > > vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && > > > !context; > > > @@ -2548,6 +2552,7 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, > > > #endif > > > vq->dma_dev = dma_dev; > > > vq->use_dma_api = vring_use_dma_api(vdev); > > > + vq->premapped = false; > > > > > > vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && > > > !context; > > > @@ -2691,6 +2696,41 @@ int virtqueue_resize(struct virtqueue *_vq, u32 num, > > > } > > > EXPORT_SYMBOL_GPL(virtqueue_resize); > > > > > > +/** > > > + * virtqueue_set_premapped - set the vring premapped mode > > > + * @_vq: the struct virtqueue we're talking about. > > > + * > > > + * Enable the premapped mode of the vq. > > > + * > > > + * The vring in premapped mode does not do dma internally, so the driver must > > > + * do dma mapping in advance. The driver must pass the dma_address through > > > + * dma_address of scatterlist. When the driver got a used buffer from > > > + * the vring, it has to unmap the dma address. So the driver must call > > > + * virtqueue_get_buf_premapped()/virtqueue_detach_unused_buf_premapped(). > > > + * > > > + * This must be called before adding any buf to vring. > > > > And any old buffer should be detached? > > I mean that before adding any buf, So there are not old buffer. > Oh. So put this in the same sentence: This function must be called immediately after creating the vq, or after vq reset, and before adding any buffers to it. > > > > > + * So this should be called immediately after init vq or vq reset. Do you really need to call this again after each reset? > > Any way to detect and warn in this case? (not a must if it's too > > expensive to do the check) > > > I can try to check whether the qeueu is empty. > > > > > > > + * > > > + * Caller must ensure we don't call this with other virtqueue operations > > > + * at the same time (except where noted). > > > + * > > > + * Returns zero or a negative error. > > > + * 0: success. > > > + * -EINVAL: vring does not use the dma api, so we can not enable premapped mode. > > > + */ > > > +int virtqueue_set_premapped(struct virtqueue *_vq) > > > +{ > > > + struct vring_virtqueue *vq = to_vvq(_vq); > > > + > > > + if (!vq->use_dma_api) > > > + return -EINVAL; > > > + > > > + vq->premapped = true; > > > > I guess there should be a way to disable it. Would it be useful for > > the case when AF_XDP sockets were destroyed? > > Yes. > > When we reset the queue, the vq->premapped will be set to 0. > > The is called after find_vqs or reset vq. > > Thanks. > > > > > > > Thanks > > > > > > > + > > > + return 0; > > > +} > > > +EXPORT_SYMBOL_GPL(virtqueue_set_premapped); > > > + > > > /* Only available for split ring */ > > > struct virtqueue *vring_new_virtqueue(unsigned int index, > > > unsigned int num, > > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h > > > index b93238db94e3..1fc0e1023bd4 100644 > > > --- a/include/linux/virtio.h > > > +++ b/include/linux/virtio.h > > > @@ -78,6 +78,8 @@ bool virtqueue_enable_cb(struct virtqueue *vq); > > > > > > unsigned virtqueue_enable_cb_prepare(struct virtqueue *vq); > > > > > > +int virtqueue_set_premapped(struct virtqueue *_vq); > > > + > > > bool virtqueue_poll(struct virtqueue *vq, unsigned); > > > > > > bool virtqueue_enable_cb_delayed(struct virtqueue *vq); > > > -- > > > 2.32.0.3.g01195cf9f > > > > >
next prev parent reply other threads:[~2023-06-27 14:57 UTC|newest] Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-06-02 9:21 [PATCH vhost v10 00/10] virtio core prepares for AF_XDP Xuan Zhuo 2023-06-02 9:21 ` Xuan Zhuo 2023-06-02 9:21 ` [PATCH vhost v10 01/10] virtio_ring: put mapping error check in vring_map_one_sg Xuan Zhuo 2023-06-02 9:21 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-02 9:21 ` [PATCH vhost v10 02/10] virtio_ring: introduce virtqueue_set_premapped() Xuan Zhuo 2023-06-02 9:21 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:50 ` Xuan Zhuo 2023-06-27 8:50 ` Xuan Zhuo 2023-06-27 14:56 ` Michael S. Tsirkin [this message] 2023-06-27 14:56 ` Michael S. Tsirkin 2023-06-28 1:34 ` Xuan Zhuo 2023-06-28 1:34 ` Xuan Zhuo 2023-06-02 9:21 ` [PATCH vhost v10 03/10] virtio_ring: split: support add premapped buf Xuan Zhuo 2023-06-02 9:21 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 9:01 ` Xuan Zhuo 2023-06-27 9:01 ` Xuan Zhuo 2023-06-28 4:07 ` Jason Wang 2023-06-28 6:00 ` Xuan Zhuo 2023-06-28 6:51 ` Jason Wang 2023-06-02 9:22 ` [PATCH vhost v10 04/10] virtio_ring: packed: " Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 9:05 ` Xuan Zhuo 2023-06-27 9:05 ` Xuan Zhuo 2023-06-02 9:22 ` [PATCH vhost v10 05/10] virtio_ring: split-detach: support return dma info to driver Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-22 19:36 ` Michael S. Tsirkin 2023-06-22 19:36 ` Michael S. Tsirkin 2023-06-25 2:10 ` Xuan Zhuo 2023-06-25 2:10 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 9:21 ` Xuan Zhuo 2023-06-27 9:21 ` Xuan Zhuo 2023-06-02 9:22 ` [PATCH vhost v10 06/10] virtio_ring: packed-detach: " Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-02 11:40 ` Michael S. Tsirkin 2023-06-02 11:40 ` Michael S. Tsirkin 2023-06-02 9:22 ` [PATCH vhost v10 07/10] virtio_ring: introduce helpers for premapped Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-04 13:45 ` Michael S. Tsirkin 2023-06-04 13:45 ` Michael S. Tsirkin 2023-06-05 2:06 ` Xuan Zhuo 2023-06-05 2:06 ` Xuan Zhuo 2023-06-05 5:38 ` Michael S. Tsirkin 2023-06-05 5:38 ` Michael S. Tsirkin 2023-06-06 2:01 ` Xuan Zhuo 2023-06-06 2:01 ` Xuan Zhuo 2023-06-22 19:29 ` Michael S. Tsirkin 2023-06-22 19:29 ` Michael S. Tsirkin 2023-06-02 9:22 ` [PATCH vhost v10 08/10] virtio_ring: introduce virtqueue_dma_dev() Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-02 9:22 ` [PATCH vhost v10 09/10] virtio_ring: introduce virtqueue_add_sg() Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-02 9:22 ` [PATCH vhost v10 10/10] virtio_net: support dma premapped Xuan Zhuo 2023-06-02 9:22 ` Xuan Zhuo 2023-06-03 6:31 ` Jakub Kicinski 2023-06-05 2:10 ` Xuan Zhuo 2023-06-05 2:10 ` Xuan Zhuo 2023-06-05 5:44 ` Michael S. Tsirkin 2023-06-05 5:44 ` Michael S. Tsirkin 2023-06-06 2:11 ` Xuan Zhuo 2023-06-06 2:11 ` Xuan Zhuo 2023-06-22 12:15 ` Michael S. Tsirkin 2023-06-22 12:15 ` Michael S. Tsirkin 2023-06-25 2:43 ` Xuan Zhuo 2023-06-25 2:43 ` Xuan Zhuo 2023-06-27 8:03 ` Jason Wang 2023-06-27 8:03 ` Jason Wang 2023-06-27 9:23 ` Xuan Zhuo 2023-06-27 9:23 ` Xuan Zhuo 2023-06-03 6:29 ` [PATCH vhost v10 00/10] virtio core prepares for AF_XDP Jakub Kicinski 2023-06-05 1:58 ` Xuan Zhuo 2023-06-05 1:58 ` Xuan Zhuo 2023-06-07 14:05 ` Christoph Hellwig 2023-06-07 14:05 ` Christoph Hellwig 2023-06-07 20:15 ` Michael S. Tsirkin 2023-06-07 20:15 ` Michael S. Tsirkin 2023-06-21 6:42 ` Xuan Zhuo 2023-06-21 6:42 ` Xuan Zhuo 2023-06-25 7:19 ` Jason Wang 2023-06-25 7:19 ` Jason Wang 2023-06-22 19:38 ` Michael S. Tsirkin 2023-06-22 19:38 ` Michael S. Tsirkin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230627105503-mutt-send-email-mst@kernel.org \ --to=mst@redhat.com \ --cc=ast@kernel.org \ --cc=bpf@vger.kernel.org \ --cc=daniel@iogearbox.net \ --cc=davem@davemloft.net \ --cc=edumazet@google.com \ --cc=hawk@kernel.org \ --cc=john.fastabend@gmail.com \ --cc=kuba@kernel.org \ --cc=netdev@vger.kernel.org \ --cc=pabeni@redhat.com \ --cc=virtualization@lists.linux-foundation.org \ --cc=xuanzhuo@linux.alibaba.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.