From: Dragos Tatulea <dtatulea@nvidia.com>
To: "Michael S . Tsirkin" <mst@redhat.com>,
Jason Wang <jasowang@redhat.com>,
Eugenio Perez Martin <eperezma@redhat.com>,
Si-Wei Liu <si-wei.liu@oracle.com>,
Saeed Mahameed <saeedm@nvidia.com>,
Leon Romanovsky <leon@kernel.org>,
<virtualization@lists.linux-foundation.org>
Cc: Dragos Tatulea <dtatulea@nvidia.com>, <kvm@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, Parav Pandit <parav@nvidia.com>,
Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Subject: [PATCH vhost v4 16/16] vdpa/mlx5: Update cvq iotlb mapping on ASID change
Date: Wed, 18 Oct 2023 20:14:55 +0300 [thread overview]
Message-ID: <20231018171456.1624030-18-dtatulea@nvidia.com> (raw)
In-Reply-To: <20231018171456.1624030-2-dtatulea@nvidia.com>
For the following sequence:
- cvq group is in ASID 0
- .set_map(1, cvq_iotlb)
- .set_group_asid(cvq_group, 1)
... the cvq mapping from ASID 0 will be used. This is not always correct
behaviour.
This patch adds support for the above mentioned flow by saving the iotlb
on each .set_map and updating the cvq iotlb with it on a cvq group change.
Acked-by: Jason Wang <jasowang@redhat.com>
Acked-by: Eugenio Pérez <eperezma@redhat.com>
Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
---
drivers/vdpa/mlx5/core/mlx5_vdpa.h | 2 ++
drivers/vdpa/mlx5/core/mr.c | 26 ++++++++++++++++++++++++++
drivers/vdpa/mlx5/net/mlx5_vnet.c | 9 ++++++++-
3 files changed, 36 insertions(+), 1 deletion(-)
diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h
index ae09296f4270..db988ced5a5d 100644
--- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h
+++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h
@@ -32,6 +32,8 @@ struct mlx5_vdpa_mr {
unsigned long num_directs;
unsigned long num_klms;
+ struct vhost_iotlb *iotlb;
+
bool user_mr;
};
diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c
index 4a3df865df40..66530e28f327 100644
--- a/drivers/vdpa/mlx5/core/mr.c
+++ b/drivers/vdpa/mlx5/core/mr.c
@@ -502,6 +502,8 @@ static void _mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_
destroy_user_mr(mvdev, mr);
else
destroy_dma_mr(mvdev, mr);
+
+ vhost_iotlb_free(mr->iotlb);
}
void mlx5_vdpa_destroy_mr(struct mlx5_vdpa_dev *mvdev,
@@ -561,6 +563,30 @@ static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev,
else
err = create_dma_mr(mvdev, mr);
+ if (err)
+ return err;
+
+ mr->iotlb = vhost_iotlb_alloc(0, 0);
+ if (!mr->iotlb) {
+ err = -ENOMEM;
+ goto err_mr;
+ }
+
+ err = dup_iotlb(mr->iotlb, iotlb);
+ if (err)
+ goto err_iotlb;
+
+ return 0;
+
+err_iotlb:
+ vhost_iotlb_free(mr->iotlb);
+
+err_mr:
+ if (iotlb)
+ destroy_user_mr(mvdev, mr);
+ else
+ destroy_dma_mr(mvdev, mr);
+
return err;
}
diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
index 87dd0ba76899..5774f4adb7c4 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
@@ -3159,12 +3159,19 @@ static int mlx5_set_group_asid(struct vdpa_device *vdev, u32 group,
unsigned int asid)
{
struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
+ int err = 0;
if (group >= MLX5_VDPA_NUMVQ_GROUPS)
return -EINVAL;
mvdev->group2asid[group] = asid;
- return 0;
+
+ mutex_lock(&mvdev->mr_mtx);
+ if (group == MLX5_VDPA_CVQ_GROUP && mvdev->mr[asid])
+ err = mlx5_vdpa_update_cvq_iotlb(mvdev, mvdev->mr[asid]->iotlb, asid);
+ mutex_unlock(&mvdev->mr_mtx);
+
+ return err;
}
static const struct vdpa_config_ops mlx5_vdpa_ops = {
--
2.41.0
next prev parent reply other threads:[~2023-10-18 17:18 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-18 17:14 [PATCH vhost v4 00/16] vdpa: Add support for vq descriptor mappings Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 01/16] vdpa/mlx5: Expose descriptor group mkey hw capability Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 02/16] vdpa: introduce dedicated descriptor group for virtqueue Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 03/16] vhost-vdpa: introduce descriptor group backend feature Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 05/16] vdpa/mlx5: Create helper function for dma mappings Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 06/16] vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 07/16] vdpa/mlx5: Take cvq iotlb lock during refresh Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 08/16] vdpa/mlx5: Collapse "dvq" mr add/delete functions Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 09/16] vdpa/mlx5: Rename mr destroy functions Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 10/16] vdpa/mlx5: Allow creation/deletion of any given mr struct Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 11/16] vdpa/mlx5: Move mr mutex out of " Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 12/16] vdpa/mlx5: Improve mr update flow Dragos Tatulea
2023-10-18 17:21 ` Dragos Tatulea
2023-10-20 16:01 ` Eugenio Perez Martin
2023-10-23 8:07 ` Dragos Tatulea
2023-10-24 14:50 ` Eugenio Perez Martin
2023-10-18 17:14 ` [PATCH vhost v4 13/16] vdpa/mlx5: Introduce mr for vq descriptor Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 14/16] vdpa/mlx5: Enable hw support for vq descriptor mapping Dragos Tatulea
2023-10-18 17:14 ` [PATCH vhost v4 15/16] vdpa/mlx5: Make iotlb helper functions more generic Dragos Tatulea
2023-10-18 17:14 ` Dragos Tatulea [this message]
2023-10-18 17:24 ` [PATCH vhost v4 00/16] vdpa: Add support for vq descriptor mappings Michael S. Tsirkin
2023-10-19 23:47 ` Si-Wei Liu
2023-10-24 6:45 ` Lei Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231018171456.1624030-18-dtatulea@nvidia.com \
--to=dtatulea@nvidia.com \
--cc=eperezma@redhat.com \
--cc=jasowang@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mst@redhat.com \
--cc=parav@nvidia.com \
--cc=saeedm@nvidia.com \
--cc=si-wei.liu@oracle.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=xuanzhuo@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).