[dpdk-dev] [PATCH] vhost: make iotlb cache name unique among multi processes

[dpdk-dev] [PATCH] vhost: make iotlb cache name unique among multi processes

[Itsuro Oda]

Currently, iotlb cache name is comprised of vid and virtqueue
index. For example, "iotlb_cache_0_0". Because vid is assigned
per process, iotlb cache name is not unique among multi processes.
For example a secondary process uses a vhost
(ex. eth_vhost0,iface=/tmp/sock0) and another secondary process
uses a vhost (ex. eth_vhost1,iface=/tmp/sock1), iotlb cache
name of both vhost ("iotlb_cache_0_0") are same and as a result
iotlb cache is broken.

This patch makes iotlb cache name unique among milti processes
by using the interface name not vid to comprise iotlb cache name.
Since the length of interface name is variable, this patch uses
hash value calculated by the interface name.

Fixes: d012d1f293f4 (vhost: add IOTLB helper functions)
Cc: stable@dpdk.org

Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
 lib/librte_vhost/iotlb.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lib/librte_vhost/iotlb.c b/lib/librte_vhost/iotlb.c
index bc1758528..0992c145b 100644
--- a/lib/librte_vhost/iotlb.c
+++ b/lib/librte_vhost/iotlb.c
@@ -6,6 +6,7 @@
 #include <numaif.h>
 #endif
 
+#include <rte_jhash.h>
 #include <rte_tailq.h>
 
 #include "iotlb.h"
@@ -288,6 +289,7 @@ vhost_user_iotlb_init(struct virtio_net *dev, int vq_index)
 	char pool_name[RTE_MEMPOOL_NAMESIZE];
 	struct vhost_virtqueue *vq = dev->virtqueue[vq_index];
 	int socket = 0;
+	uint32_t val;
 
 	if (vq->iotlb_pool) {
 		/*
@@ -308,8 +310,10 @@ vhost_user_iotlb_init(struct virtio_net *dev, int vq_index)
 	TAILQ_INIT(&vq->iotlb_list);
 	TAILQ_INIT(&vq->iotlb_pending_list);
 
-	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%d_%d",
-			dev->vid, vq_index);
+	val = rte_jhash(dev->ifname, strlen(dev->ifname), 0);
+	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%08x_%d",
+			val, vq_index);
+	VHOST_LOG_CONFIG(DEBUG, "IOTLB cache name: %s\n", pool_name);
 
 	/* If already created, free it and recreate */
 	vq->iotlb_pool = rte_mempool_lookup(pool_name);
-- 
2.17.0

Re: [dpdk-dev] [PATCH] vhost: make iotlb cache name unique among multi processes

[Ye Xiaolong]

On 03/10, Itsuro Oda wrote:
>Currently, iotlb cache name is comprised of vid and virtqueue
>index. For example, "iotlb_cache_0_0". Because vid is assigned
>per process, iotlb cache name is not unique among multi processes.
>For example a secondary process uses a vhost
>(ex. eth_vhost0,iface=/tmp/sock0) and another secondary process
>uses a vhost (ex. eth_vhost1,iface=/tmp/sock1), iotlb cache
>name of both vhost ("iotlb_cache_0_0") are same and as a result
>iotlb cache is broken.
>
>This patch makes iotlb cache name unique among milti processes
>by using the interface name not vid to comprise iotlb cache name.
>Since the length of interface name is variable, this patch uses
>hash value calculated by the interface name.
>
>Fixes: d012d1f293f4 (vhost: add IOTLB helper functions)
>Cc: stable@dpdk.org
>
>Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
>---
> lib/librte_vhost/iotlb.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
>diff --git a/lib/librte_vhost/iotlb.c b/lib/librte_vhost/iotlb.c
>index bc1758528..0992c145b 100644
>--- a/lib/librte_vhost/iotlb.c
>+++ b/lib/librte_vhost/iotlb.c
>@@ -6,6 +6,7 @@
> #include <numaif.h>
> #endif
> 
>+#include <rte_jhash.h>
> #include <rte_tailq.h>
> 
> #include "iotlb.h"
>@@ -288,6 +289,7 @@ vhost_user_iotlb_init(struct virtio_net *dev, int vq_index)
> 	char pool_name[RTE_MEMPOOL_NAMESIZE];
> 	struct vhost_virtqueue *vq = dev->virtqueue[vq_index];
> 	int socket = 0;
>+	uint32_t val;
> 
> 	if (vq->iotlb_pool) {
> 		/*
>@@ -308,8 +310,10 @@ vhost_user_iotlb_init(struct virtio_net *dev, int vq_index)
> 	TAILQ_INIT(&vq->iotlb_list);
> 	TAILQ_INIT(&vq->iotlb_pending_list);
> 
>-	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%d_%d",
>-			dev->vid, vq_index);
>+	val = rte_jhash(dev->ifname, strlen(dev->ifname), 0);
>+	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%08x_%d",
>+			val, vq_index);
>+	VHOST_LOG_CONFIG(DEBUG, "IOTLB cache name: %s\n", pool_name);
> 
> 	/* If already created, free it and recreate */
> 	vq->iotlb_pool = rte_mempool_lookup(pool_name);
>-- 
>2.17.0
>

Acked-by: Xiaolong Ye <xiaolong.ye@intel.com>

Re: [dpdk-dev] [PATCH] vhost: make iotlb cache name unique among multi processes

[Van Haaren, Harry]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Ye Xiaolong
> Sent: Tuesday, March 10, 2020 11:32 AM
> To: Itsuro Oda <oda@valinux.co.jp>
> Cc: dev@dpdk.org; maxime.coquelin@redhat.com; Wang, Zhihong
> <zhihong.wang@intel.com>; stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] vhost: make iotlb cache name unique among
> multi processes
> 
> On 03/10, Itsuro Oda wrote:
> >Currently, iotlb cache name is comprised of vid and virtqueue
> >index. For example, "iotlb_cache_0_0". Because vid is assigned
> >per process, iotlb cache name is not unique among multi processes.
> >For example a secondary process uses a vhost
> >(ex. eth_vhost0,iface=/tmp/sock0) and another secondary process
> >uses a vhost (ex. eth_vhost1,iface=/tmp/sock1), iotlb cache
> >name of both vhost ("iotlb_cache_0_0") are same and as a result
> >iotlb cache is broken.
> >
> >This patch makes iotlb cache name unique among milti processes
> >by using the interface name not vid to comprise iotlb cache name.
> >Since the length of interface name is variable, this patch uses
> >hash value calculated by the interface name.
> >
> >Fixes: d012d1f293f4 (vhost: add IOTLB helper functions)
> >Cc: stable@dpdk.org
> >
> >Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
> >---
> > lib/librte_vhost/iotlb.c | 8 ++++++--
> > 1 file changed, 6 insertions(+), 2 deletions(-)
> >
> >diff --git a/lib/librte_vhost/iotlb.c b/lib/librte_vhost/iotlb.c
> >index bc1758528..0992c145b 100644
> >--- a/lib/librte_vhost/iotlb.c
> >+++ b/lib/librte_vhost/iotlb.c
> >@@ -6,6 +6,7 @@
> > #include <numaif.h>
> > #endif
> >
> >+#include <rte_jhash.h>
> > #include <rte_tailq.h>
> >
> > #include "iotlb.h"
> >@@ -288,6 +289,7 @@ vhost_user_iotlb_init(struct virtio_net *dev, int
> vq_index)
> > 	char pool_name[RTE_MEMPOOL_NAMESIZE];
> > 	struct vhost_virtqueue *vq = dev->virtqueue[vq_index];
> > 	int socket = 0;
> >+	uint32_t val;
> >
> > 	if (vq->iotlb_pool) {
> > 		/*
> >@@ -308,8 +310,10 @@ vhost_user_iotlb_init(struct virtio_net *dev, int
> vq_index)
> > 	TAILQ_INIT(&vq->iotlb_list);
> > 	TAILQ_INIT(&vq->iotlb_pending_list);
> >
> >-	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%d_%d",
> >-			dev->vid, vq_index);
> >+	val = rte_jhash(dev->ifname, strlen(dev->ifname), 0);
> >+	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%08x_%d",
> >+			val, vq_index);
> >+	VHOST_LOG_CONFIG(DEBUG, "IOTLB cache name: %s\n", pool_name);

Although very unlikely, what would happen if there is a hash-collision?

For example imagine two different names hash to the same "val", from
my understanding they will now use the same IOTLB but should not share one.

<snip>

Re: [dpdk-dev] [PATCH] vhost: make iotlb cache name unique among multi processes

[Maxime Coquelin]

On 3/10/20 1:44 PM, Van Haaren, Harry wrote:
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Ye Xiaolong
>> Sent: Tuesday, March 10, 2020 11:32 AM
>> To: Itsuro Oda <oda@valinux.co.jp>
>> Cc: dev@dpdk.org; maxime.coquelin@redhat.com; Wang, Zhihong
>> <zhihong.wang@intel.com>; stable@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH] vhost: make iotlb cache name unique among
>> multi processes
>>
>> On 03/10, Itsuro Oda wrote:
>>> Currently, iotlb cache name is comprised of vid and virtqueue
>>> index. For example, "iotlb_cache_0_0". Because vid is assigned
>>> per process, iotlb cache name is not unique among multi processes.
>>> For example a secondary process uses a vhost
>>> (ex. eth_vhost0,iface=/tmp/sock0) and another secondary process
>>> uses a vhost (ex. eth_vhost1,iface=/tmp/sock1), iotlb cache
>>> name of both vhost ("iotlb_cache_0_0") are same and as a result
>>> iotlb cache is broken.
>>>
>>> This patch makes iotlb cache name unique among milti processes
>>> by using the interface name not vid to comprise iotlb cache name.
>>> Since the length of interface name is variable, this patch uses
>>> hash value calculated by the interface name.
>>>
>>> Fixes: d012d1f293f4 (vhost: add IOTLB helper functions)
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
>>> ---
>>> lib/librte_vhost/iotlb.c | 8 ++++++--
>>> 1 file changed, 6 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/lib/librte_vhost/iotlb.c b/lib/librte_vhost/iotlb.c
>>> index bc1758528..0992c145b 100644
>>> --- a/lib/librte_vhost/iotlb.c
>>> +++ b/lib/librte_vhost/iotlb.c
>>> @@ -6,6 +6,7 @@
>>> #include <numaif.h>
>>> #endif
>>>
>>> +#include <rte_jhash.h>
>>> #include <rte_tailq.h>
>>>
>>> #include "iotlb.h"
>>> @@ -288,6 +289,7 @@ vhost_user_iotlb_init(struct virtio_net *dev, int
>> vq_index)
>>> 	char pool_name[RTE_MEMPOOL_NAMESIZE];
>>> 	struct vhost_virtqueue *vq = dev->virtqueue[vq_index];
>>> 	int socket = 0;
>>> +	uint32_t val;
>>>
>>> 	if (vq->iotlb_pool) {
>>> 		/*
>>> @@ -308,8 +310,10 @@ vhost_user_iotlb_init(struct virtio_net *dev, int
>> vq_index)
>>> 	TAILQ_INIT(&vq->iotlb_list);
>>> 	TAILQ_INIT(&vq->iotlb_pending_list);
>>>
>>> -	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%d_%d",
>>> -			dev->vid, vq_index);
>>> +	val = rte_jhash(dev->ifname, strlen(dev->ifname), 0);
>>> +	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%08x_%d",
>>> +			val, vq_index);
>>> +	VHOST_LOG_CONFIG(DEBUG, "IOTLB cache name: %s\n", pool_name);
> 
> Although very unlikely, what would happen if there is a hash-collision?
> 
> For example imagine two different names hash to the same "val", from
> my understanding they will now use the same IOTLB but should not share one.
> 
> <snip>
> 

+1.

Instead of a hash, maybe use the process ID:
snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%u_%d_%d",
	 pid, dev->vid, vq_index);

[dpdk-dev] [PATCH v2] vhost: make iotlb cache name unique among multi processes

[Itsuro Oda]

Currently, iotlb cache name is comprised of vid and virtqueue
index. For example, "iotlb_cache_0_0". Because vid is assigned
per process, iotlb cache name is not unique among multi processes.
For example a secondary process uses a vhost
(ex. eth_vhost0,iface=/tmp/sock0) and another secondary process
uses a vhost (ex. eth_vhost1,iface=/tmp/sock1), iotlb cache
name of both vhost ("iotlb_cache_0_0") are same and as a result
iotlb cache is broken.

This patch makes iotlb cache name unique among milti processes
by adding process id to the iotlb cache name.

The prefix of the name is shortend to "iotlb_" since the maximum
length of pool name is 23 bytes (RTE_MEMPOOL_NAMESIZE is 24).
Note that pid_t is 4 bytes (thus max 10 digits), vid is max 4
digits (MAX_VHOST_DEVICE is 1024) and vq_index is 1 digit. It is
just 23 characters in maximum.

Fixes: d012d1f293f4 (vhost: add IOTLB helper functions)
Cc: stable@dpdk.org

Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
v2:
* use the process id to make the iotlb cache name unique.

 lib/librte_vhost/iotlb.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/librte_vhost/iotlb.c b/lib/librte_vhost/iotlb.c
index bc1758528..7af5b8c8d 100644
--- a/lib/librte_vhost/iotlb.c
+++ b/lib/librte_vhost/iotlb.c
@@ -308,8 +308,9 @@ vhost_user_iotlb_init(struct virtio_net *dev, int vq_index)
 	TAILQ_INIT(&vq->iotlb_list);
 	TAILQ_INIT(&vq->iotlb_pending_list);
 
-	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%d_%d",
-			dev->vid, vq_index);
+	snprintf(pool_name, sizeof(pool_name), "iotlb_%d_%d_%d",
+			getpid(), dev->vid, vq_index);
+	VHOST_LOG_CONFIG(DEBUG, "IOTLB cache name: %s\n", pool_name);
 
 	/* If already created, free it and recreate */
 	vq->iotlb_pool = rte_mempool_lookup(pool_name);
-- 
2.17.0

[dpdk-dev] [PATCH v3] vhost: make iotlb cache name unique among multi processes

[Itsuro Oda]

Currently, iotlb cache name is comprised of vid and virtqueue
index. For example, "iotlb_cache_0_0". Because vid is assigned
per process, iotlb cache name is not unique among multi processes.
For example a secondary process uses a vhost
(ex. eth_vhost0,iface=/tmp/sock0) and another secondary process
uses a vhost (ex. eth_vhost1,iface=/tmp/sock1), iotlb cache
name of both vhost ("iotlb_cache_0_0") are same and as a result
iotlb cache is broken.

This patch makes iotlb cache name unique among milti processes
by adding process id to the iotlb cache name.

The prefix of the name is shortend to "iotlb_" since the maximum
length of pool name is 25 bytes (RTE_MEMPOOL_NAMESIZE is 26).
Note that it is just 25 characters in maximum at the moment.
Here,
* pid_t == int: max 10 digits.
* vid < MAX_VHOST_DECICE(1024): max 4 digits.
* vq_index < VHOST_MAX_VRING(256): max 3 digits.

Fixes: d012d1f293f4 (vhost: add IOTLB helper functions)
Cc: stable@dpdk.org

Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
---
v3:
* change format.
* fix commit message.

v2:
* use the process id to make the iotlb cache name unique.

 lib/librte_vhost/iotlb.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/librte_vhost/iotlb.c b/lib/librte_vhost/iotlb.c
index bc1758528..5b3a0c090 100644
--- a/lib/librte_vhost/iotlb.c
+++ b/lib/librte_vhost/iotlb.c
@@ -308,8 +308,9 @@ vhost_user_iotlb_init(struct virtio_net *dev, int vq_index)
 	TAILQ_INIT(&vq->iotlb_list);
 	TAILQ_INIT(&vq->iotlb_pending_list);
 
-	snprintf(pool_name, sizeof(pool_name), "iotlb_cache_%d_%d",
-			dev->vid, vq_index);
+	snprintf(pool_name, sizeof(pool_name), "iotlb_%u_%d_%d",
+			getpid(), dev->vid, vq_index);
+	VHOST_LOG_CONFIG(DEBUG, "IOTLB cache name: %s\n", pool_name);
 
 	/* If already created, free it and recreate */
 	vq->iotlb_pool = rte_mempool_lookup(pool_name);
-- 
2.17.0

Re: [dpdk-dev] [PATCH v3] vhost: make iotlb cache name unique among multi processes

[Maxime Coquelin]

On 3/12/20 12:19 AM, Itsuro Oda wrote:
> Currently, iotlb cache name is comprised of vid and virtqueue
> index. For example, "iotlb_cache_0_0". Because vid is assigned
> per process, iotlb cache name is not unique among multi processes.
> For example a secondary process uses a vhost
> (ex. eth_vhost0,iface=/tmp/sock0) and another secondary process
> uses a vhost (ex. eth_vhost1,iface=/tmp/sock1), iotlb cache
> name of both vhost ("iotlb_cache_0_0") are same and as a result
> iotlb cache is broken.
> 
> This patch makes iotlb cache name unique among milti processes
> by adding process id to the iotlb cache name.
> 
> The prefix of the name is shortend to "iotlb_" since the maximum
> length of pool name is 25 bytes (RTE_MEMPOOL_NAMESIZE is 26).
> Note that it is just 25 characters in maximum at the moment.
> Here,
> * pid_t == int: max 10 digits.
> * vid < MAX_VHOST_DECICE(1024): max 4 digits.
> * vq_index < VHOST_MAX_VRING(256): max 3 digits.
> 
> Fixes: d012d1f293f4 (vhost: add IOTLB helper functions)
> Cc: stable@dpdk.org
> 
> Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
> ---
> v3:
> * change format.
> * fix commit message.
> 
> v2:
> * use the process id to make the iotlb cache name unique.
> 
>  lib/librte_vhost/iotlb.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 


Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [dpdk-dev] [PATCH v3] vhost: make iotlb cache name unique among multi processes

[Maxime Coquelin]

On 3/12/20 12:19 AM, Itsuro Oda wrote:
> Currently, iotlb cache name is comprised of vid and virtqueue
> index. For example, "iotlb_cache_0_0". Because vid is assigned
> per process, iotlb cache name is not unique among multi processes.
> For example a secondary process uses a vhost
> (ex. eth_vhost0,iface=/tmp/sock0) and another secondary process
> uses a vhost (ex. eth_vhost1,iface=/tmp/sock1), iotlb cache
> name of both vhost ("iotlb_cache_0_0") are same and as a result
> iotlb cache is broken.
> 
> This patch makes iotlb cache name unique among milti processes
> by adding process id to the iotlb cache name.
> 
> The prefix of the name is shortend to "iotlb_" since the maximum
> length of pool name is 25 bytes (RTE_MEMPOOL_NAMESIZE is 26).
> Note that it is just 25 characters in maximum at the moment.
> Here,
> * pid_t == int: max 10 digits.
> * vid < MAX_VHOST_DECICE(1024): max 4 digits.
> * vq_index < VHOST_MAX_VRING(256): max 3 digits.
> 
> Fixes: d012d1f293f4 (vhost: add IOTLB helper functions)
> Cc: stable@dpdk.org
> 
> Signed-off-by: Itsuro Oda <oda@valinux.co.jp>
> ---
> v3:
> * change format.
> * fix commit message.
> 
> v2:
> * use the process id to make the iotlb cache name unique.
> 
>  lib/librte_vhost/iotlb.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
Applied to dpdk-next-virtio/master.

Thanks,
Maxime