From: Xuan Zhuo <xuanzhuo@linux.alibaba.com> To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" <mst@redhat.com>, Jason Wang <jasowang@redhat.com>, "David S. Miller" <davem@davemloft.net>, Jakub Kicinski <kuba@kernel.org> Subject: [PATCH 2/3] virtio: cache indirect desc for packed Date: Wed, 27 Oct 2021 14:19:12 +0800 [thread overview] Message-ID: <20211027061913.76276-3-xuanzhuo@linux.alibaba.com> (raw) In-Reply-To: <20211027061913.76276-1-xuanzhuo@linux.alibaba.com> In the case of using indirect, indirect desc must be allocated and released each time, which increases a lot of cpu overhead. Here, a cache is added for indirect. If the number of indirect desc to be applied for is less than VIRT_QUEUE_CACHE_DESC_NUM, the desc array with the size of VIRT_QUEUE_CACHE_DESC_NUM is fixed and cached for reuse. Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com> --- drivers/virtio/virtio_ring.c | 57 +++++++++++++++++++++++++++++++----- 1 file changed, 49 insertions(+), 8 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 0b9a8544b0e8..4fd7bd5bcd70 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1074,10 +1074,45 @@ static void vring_unmap_desc_packed(const struct vring_virtqueue *vq, } } -static struct vring_packed_desc *alloc_indirect_packed(unsigned int total_sg, +static void desc_cache_chain_free_packed(void *chain) +{ + struct vring_packed_desc *desc; + + while (chain) { + desc = chain; + chain = (void *)desc->addr; + kfree(desc); + } +} + +static void desc_cache_put_packed(struct vring_virtqueue *vq, + struct vring_packed_desc *desc, int n) +{ + if (vq->use_desc_cache && n <= VIRT_QUEUE_CACHE_DESC_NUM) { + desc->addr = (u64)vq->desc_cache_chain; + vq->desc_cache_chain = desc; + } else { + kfree(desc); + } +} + +static struct vring_packed_desc *alloc_indirect_packed(struct vring_virtqueue *vq, + unsigned int total_sg, gfp_t gfp) { struct vring_packed_desc *desc; + unsigned int n; + + if (vq->use_desc_cache && total_sg <= VIRT_QUEUE_CACHE_DESC_NUM) { + if (vq->desc_cache_chain) { + desc = vq->desc_cache_chain; + vq->desc_cache_chain = (void *)desc->addr; + return desc; + } + n = VIRT_QUEUE_CACHE_DESC_NUM; + } else { + n = total_sg; + } /* * We require lowmem mappings for the descriptors because @@ -1086,7 +1121,7 @@ static struct vring_packed_desc *alloc_indirect_packed(unsigned int total_sg, */ gfp &= ~__GFP_HIGHMEM; - desc = kmalloc_array(total_sg, sizeof(struct vring_packed_desc), gfp); + desc = kmalloc_array(n, sizeof(struct vring_packed_desc), gfp); return desc; } @@ -1106,7 +1141,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, dma_addr_t addr; head = vq->packed.next_avail_idx; - desc = alloc_indirect_packed(total_sg, gfp); + desc = alloc_indirect_packed(vq, total_sg, gfp); if (unlikely(vq->vq.num_free < 1)) { pr_debug("Can't add buf len 1 - avail = 0\n"); @@ -1197,7 +1232,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, for (i = 0; i < err_idx; i++) vring_unmap_desc_packed(vq, &desc[i]); - kfree(desc); + desc_cache_put_packed(vq, desc, total_sg); END_USE(vq); return -ENOMEM; @@ -1422,20 +1457,22 @@ static void detach_buf_packed(struct vring_virtqueue *vq, } if (vq->indirect) { - u32 len; + u32 len, n; /* Free the indirect table, if any, now that it's unmapped. */ desc = state->indir_desc; if (!desc) return; + n = len / sizeof(struct vring_packed_desc); + if (vq->use_dma_api) { len = vq->packed.desc_extra[id].len; - for (i = 0; i < len / sizeof(struct vring_packed_desc); - i++) + for (i = 0; i < n; i++) vring_unmap_desc_packed(vq, &desc[i]); } - kfree(desc); + + desc_cache_put_packed(vq, desc, n); state->indir_desc = NULL; } else if (ctx) { *ctx = state->indir_desc; @@ -1753,6 +1790,8 @@ static struct virtqueue *vring_create_virtqueue_packed( vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !context; vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX); + vq->desc_cache_chain = NULL; + vq->use_desc_cache = vdev->desc_cache; if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) vq->weak_barriers = false; @@ -2374,6 +2413,8 @@ void vring_del_virtqueue(struct virtqueue *_vq) kfree(vq->split.desc_state); kfree(vq->split.desc_extra); desc_cache_chain_free_split(vq->desc_cache_chain); + } else { + desc_cache_chain_free_packed(vq->desc_cache_chain); } kfree(vq); } -- 2.31.0
WARNING: multiple messages have this Message-ID (diff)
From: Xuan Zhuo <xuanzhuo@linux.alibaba.com> To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: Jakub Kicinski <kuba@kernel.org>, "David S. Miller" <davem@davemloft.net>, "Michael S. Tsirkin" <mst@redhat.com> Subject: [PATCH 2/3] virtio: cache indirect desc for packed Date: Wed, 27 Oct 2021 14:19:12 +0800 [thread overview] Message-ID: <20211027061913.76276-3-xuanzhuo@linux.alibaba.com> (raw) In-Reply-To: <20211027061913.76276-1-xuanzhuo@linux.alibaba.com> In the case of using indirect, indirect desc must be allocated and released each time, which increases a lot of cpu overhead. Here, a cache is added for indirect. If the number of indirect desc to be applied for is less than VIRT_QUEUE_CACHE_DESC_NUM, the desc array with the size of VIRT_QUEUE_CACHE_DESC_NUM is fixed and cached for reuse. Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com> --- drivers/virtio/virtio_ring.c | 57 +++++++++++++++++++++++++++++++----- 1 file changed, 49 insertions(+), 8 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 0b9a8544b0e8..4fd7bd5bcd70 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1074,10 +1074,45 @@ static void vring_unmap_desc_packed(const struct vring_virtqueue *vq, } } -static struct vring_packed_desc *alloc_indirect_packed(unsigned int total_sg, +static void desc_cache_chain_free_packed(void *chain) +{ + struct vring_packed_desc *desc; + + while (chain) { + desc = chain; + chain = (void *)desc->addr; + kfree(desc); + } +} + +static void desc_cache_put_packed(struct vring_virtqueue *vq, + struct vring_packed_desc *desc, int n) +{ + if (vq->use_desc_cache && n <= VIRT_QUEUE_CACHE_DESC_NUM) { + desc->addr = (u64)vq->desc_cache_chain; + vq->desc_cache_chain = desc; + } else { + kfree(desc); + } +} + +static struct vring_packed_desc *alloc_indirect_packed(struct vring_virtqueue *vq, + unsigned int total_sg, gfp_t gfp) { struct vring_packed_desc *desc; + unsigned int n; + + if (vq->use_desc_cache && total_sg <= VIRT_QUEUE_CACHE_DESC_NUM) { + if (vq->desc_cache_chain) { + desc = vq->desc_cache_chain; + vq->desc_cache_chain = (void *)desc->addr; + return desc; + } + n = VIRT_QUEUE_CACHE_DESC_NUM; + } else { + n = total_sg; + } /* * We require lowmem mappings for the descriptors because @@ -1086,7 +1121,7 @@ static struct vring_packed_desc *alloc_indirect_packed(unsigned int total_sg, */ gfp &= ~__GFP_HIGHMEM; - desc = kmalloc_array(total_sg, sizeof(struct vring_packed_desc), gfp); + desc = kmalloc_array(n, sizeof(struct vring_packed_desc), gfp); return desc; } @@ -1106,7 +1141,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, dma_addr_t addr; head = vq->packed.next_avail_idx; - desc = alloc_indirect_packed(total_sg, gfp); + desc = alloc_indirect_packed(vq, total_sg, gfp); if (unlikely(vq->vq.num_free < 1)) { pr_debug("Can't add buf len 1 - avail = 0\n"); @@ -1197,7 +1232,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, for (i = 0; i < err_idx; i++) vring_unmap_desc_packed(vq, &desc[i]); - kfree(desc); + desc_cache_put_packed(vq, desc, total_sg); END_USE(vq); return -ENOMEM; @@ -1422,20 +1457,22 @@ static void detach_buf_packed(struct vring_virtqueue *vq, } if (vq->indirect) { - u32 len; + u32 len, n; /* Free the indirect table, if any, now that it's unmapped. */ desc = state->indir_desc; if (!desc) return; + n = len / sizeof(struct vring_packed_desc); + if (vq->use_dma_api) { len = vq->packed.desc_extra[id].len; - for (i = 0; i < len / sizeof(struct vring_packed_desc); - i++) + for (i = 0; i < n; i++) vring_unmap_desc_packed(vq, &desc[i]); } - kfree(desc); + + desc_cache_put_packed(vq, desc, n); state->indir_desc = NULL; } else if (ctx) { *ctx = state->indir_desc; @@ -1753,6 +1790,8 @@ static struct virtqueue *vring_create_virtqueue_packed( vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !context; vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX); + vq->desc_cache_chain = NULL; + vq->use_desc_cache = vdev->desc_cache; if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) vq->weak_barriers = false; @@ -2374,6 +2413,8 @@ void vring_del_virtqueue(struct virtqueue *_vq) kfree(vq->split.desc_state); kfree(vq->split.desc_extra); desc_cache_chain_free_split(vq->desc_cache_chain); + } else { + desc_cache_chain_free_packed(vq->desc_cache_chain); } kfree(vq); } -- 2.31.0 _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2021-10-27 6:19 UTC|newest] Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-10-27 6:19 [PATCH 0/3] virtio support cache indirect desc Xuan Zhuo 2021-10-27 6:19 ` Xuan Zhuo 2021-10-27 6:19 ` [PATCH 1/3] virtio: cache indirect desc for split Xuan Zhuo 2021-10-27 6:19 ` Xuan Zhuo 2021-10-27 8:55 ` Michael S. Tsirkin 2021-10-27 8:55 ` Michael S. Tsirkin 2021-10-27 9:03 ` Xuan Zhuo 2021-10-27 16:33 ` Dongli Zhang 2021-10-27 16:33 ` Dongli Zhang 2021-10-27 19:36 ` Michael S. Tsirkin 2021-10-27 19:36 ` Michael S. Tsirkin 2021-10-27 17:07 ` Michael S. Tsirkin 2021-10-27 17:07 ` Michael S. Tsirkin 2021-10-28 2:16 ` Jason Wang 2021-10-28 2:16 ` Jason Wang 2021-10-28 6:16 ` Xuan Zhuo 2021-10-31 14:47 ` Michael S. Tsirkin 2021-10-31 14:47 ` Michael S. Tsirkin 2021-10-27 23:02 ` kernel test robot 2021-10-27 23:02 ` kernel test robot 2021-10-27 23:02 ` kernel test robot 2021-10-28 0:57 ` kernel test robot 2021-10-28 0:57 ` kernel test robot 2021-10-28 0:57 ` kernel test robot 2021-10-27 6:19 ` Xuan Zhuo [this message] 2021-10-27 6:19 ` [PATCH 2/3] virtio: cache indirect desc for packed Xuan Zhuo 2021-10-28 0:28 ` kernel test robot 2021-10-28 0:28 ` kernel test robot 2021-10-28 0:28 ` kernel test robot 2021-10-28 3:51 ` kernel test robot 2021-10-28 3:51 ` kernel test robot 2021-10-28 3:51 ` kernel test robot 2021-10-28 7:38 ` kernel test robot 2021-10-28 7:38 ` kernel test robot 2021-10-28 7:38 ` kernel test robot 2021-10-27 6:19 ` [PATCH 3/3] virtio-net: enable virtio indirect cache Xuan Zhuo 2021-10-27 6:19 ` Xuan Zhuo 2021-10-27 15:55 ` Jakub Kicinski 2021-10-28 1:57 ` Xuan Zhuo 2021-10-28 2:28 ` Jason Wang 2021-10-28 2:28 ` Jason Wang
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20211027061913.76276-3-xuanzhuo@linux.alibaba.com \ --to=xuanzhuo@linux.alibaba.com \ --cc=davem@davemloft.net \ --cc=jasowang@redhat.com \ --cc=kuba@kernel.org \ --cc=mst@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=virtualization@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.