[PATCH rdma-rc 00/10] Collection of fixes for 5.3

[PATCH rdma-rc 00/10] Collection of fixes for 5.3

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Hi,

This is small patch set of fixes targeted for 5.3 and stable@.

Thanks

Moni Shoua (1):
  IB/mlx5: Prevent concurrent MR updates during invalidation

Parav Pandit (4):
  RDMA/core: Annotate destroy of mutex to ensure that it is released as
    unlocked
  IB/mlx5: Avoid unnecessary typecast
  IB/core: Fix querying total rdma stats
  IB/counters: Initialize port counter and annotate mutex_destroy

Yishai Hadas (5):
  IB/mlx5: Fix unreg_umr to ignore the mkey state
  IB/mlx5: Use direct mkey destroy command upon UMR unreg failure
  IB/mlx5: Fix unreg_umr to set a device PD
  IB/mlx5: Fix clean_mr() to work in the expected order
  IB/mlx5: Fix RSS Toeplitz function to be specification aligned

 drivers/infiniband/core/cache.c        |  1 +
 drivers/infiniband/core/cma_configfs.c |  8 +++++++-
 drivers/infiniband/core/counters.c     | 19 +++++++++++--------
 drivers/infiniband/core/device.c       |  3 +++
 drivers/infiniband/core/user_mad.c     |  2 +-
 drivers/infiniband/core/uverbs_main.c  |  4 ++++
 drivers/infiniband/core/verbs.c        |  1 +
 drivers/infiniband/hw/mlx5/main.c      |  2 +-
 drivers/infiniband/hw/mlx5/mlx5_ib.h   |  1 +
 drivers/infiniband/hw/mlx5/mr.c        | 23 ++++++++++++++---------
 drivers/infiniband/hw/mlx5/odp.c       |  3 ++-
 drivers/infiniband/hw/mlx5/qp.c        | 13 ++++++++-----
 12 files changed, 54 insertions(+), 26 deletions(-)

--
2.20.1

[PATCH rdma-rc 01/10] RDMA/core: Annotate destroy of mutex to ensure that it is released as unlocked

[Leon Romanovsky]

From: Parav Pandit <parav@mellanox.com>

While compiled with CONFIG_DEBUG_MUTEXES, the kernel ensures that mutex
is not held during destroy.
Hence add mutex_destroy() for mutexes used in RDMA modules.

Signed-off-by: Parav Pandit <parav@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/core/cache.c        | 1 +
 drivers/infiniband/core/cma_configfs.c | 8 +++++++-
 drivers/infiniband/core/device.c       | 3 +++
 drivers/infiniband/core/user_mad.c     | 2 +-
 drivers/infiniband/core/uverbs_main.c  | 4 ++++
 drivers/infiniband/core/verbs.c        | 1 +
 6 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
index 18e476b3ced0..00fb3eacda19 100644
--- a/drivers/infiniband/core/cache.c
+++ b/drivers/infiniband/core/cache.c
@@ -810,6 +810,7 @@ static void release_gid_table(struct ib_device *device,
 	if (leak)
 		return;

+	mutex_destroy(&table->lock);
 	kfree(table->data_vec);
 	kfree(table);
 }
diff --git a/drivers/infiniband/core/cma_configfs.c b/drivers/infiniband/core/cma_configfs.c
index 3ec2c415bb70..8b0b5ae22e4c 100644
--- a/drivers/infiniband/core/cma_configfs.c
+++ b/drivers/infiniband/core/cma_configfs.c
@@ -342,12 +342,18 @@ static struct configfs_subsystem cma_subsys = {

 int __init cma_configfs_init(void)
 {
+	int ret;
+
 	config_group_init(&cma_subsys.su_group);
 	mutex_init(&cma_subsys.su_mutex);
-	return configfs_register_subsystem(&cma_subsys);
+	ret = configfs_register_subsystem(&cma_subsys);
+	if (ret)
+		mutex_destroy(&cma_subsys.su_mutex);
+	return ret;
 }

 void __exit cma_configfs_exit(void)
 {
 	configfs_unregister_subsystem(&cma_subsys);
+	mutex_destroy(&cma_subsys.su_mutex);
 }
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index c56993e5e5ea..c3576c7d2e8f 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -497,6 +497,9 @@ static void ib_device_release(struct device *device)
 			  rcu_head);
 	}

+	mutex_destroy(&dev->unregistration_lock);
+	mutex_destroy(&dev->compat_devs_mutex);
+
 	xa_destroy(&dev->compat_devs);
 	xa_destroy(&dev->client_data);
 	kfree_rcu(dev, rcu_head);
diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
index 9f8a48016b41..e0512aef033c 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -1038,7 +1038,7 @@ static int ib_umad_close(struct inode *inode, struct file *filp)
 				ib_unregister_mad_agent(file->agent[i]);

 	mutex_unlock(&file->port->file_mutex);
-
+	mutex_destroy(&file->mutex);
 	kfree(file);
 	return 0;
 }
diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
index 11c13c1381cf..02b57240176c 100644
--- a/drivers/infiniband/core/uverbs_main.c
+++ b/drivers/infiniband/core/uverbs_main.c
@@ -120,6 +120,8 @@ static void ib_uverbs_release_dev(struct device *device)

 	uverbs_destroy_api(dev->uapi);
 	cleanup_srcu_struct(&dev->disassociate_srcu);
+	mutex_destroy(&dev->lists_mutex);
+	mutex_destroy(&dev->xrcd_tree_mutex);
 	kfree(dev);
 }

@@ -212,6 +214,8 @@ void ib_uverbs_release_file(struct kref *ref)

 	if (file->disassociate_page)
 		__free_pages(file->disassociate_page, 0);
+	mutex_destroy(&file->umap_lock);
+	mutex_destroy(&file->ucontext_lock);
 	kfree(file);
 }

diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 92349bf37589..f974b6854224 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -2259,6 +2259,7 @@ int ib_dealloc_xrcd(struct ib_xrcd *xrcd, struct ib_udata *udata)
 		if (ret)
 			return ret;
 	}
+	mutex_destroy(&xrcd->tgt_qp_mutex);

 	return xrcd->device->ops.dealloc_xrcd(xrcd, udata);
 }
--
2.20.1

[PATCH rdma-rc 02/10] IB/mlx5: Fix unreg_umr to ignore the mkey state

[Leon Romanovsky]

From: Yishai Hadas <yishaih@mellanox.com>

Fix unreg_umr to ignore the mkey state and do not fail if was freed.
This prevents a case that a user space application already changed the
mkey state to free and then the UMR operation will fail leaving the mkey
in an un-applicable state.

Cc: <stable@vger.kernel.org> # 3.19
Fixes: 968e78dd9644 ("IB/mlx5: Enhance UMR support to allow partial page table update")
Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Artemy Kovalyov <artemyko@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h |  1 +
 drivers/infiniband/hw/mlx5/mr.c      |  4 ++--
 drivers/infiniband/hw/mlx5/qp.c      | 12 ++++++++----
 3 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index c482f19958b3..f6a53455bf8b 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -481,6 +481,7 @@ struct mlx5_umr_wr {
 	u64				length;
 	int				access_flags;
 	u32				mkey;
+	u8				ignore_free_state:1;
 };

 static inline const struct mlx5_umr_wr *umr_wr(const struct ib_send_wr *wr)
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 20ece6e0b2fc..266edaf8029d 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1372,10 +1372,10 @@ static int unreg_umr(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr)
 	if (mdev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR)
 		return 0;

-	umrwr.wr.send_flags = MLX5_IB_SEND_UMR_DISABLE_MR |
-			      MLX5_IB_SEND_UMR_FAIL_IF_FREE;
+	umrwr.wr.send_flags = MLX5_IB_SEND_UMR_DISABLE_MR;
 	umrwr.wr.opcode = MLX5_IB_WR_UMR;
 	umrwr.mkey = mr->mmkey.key;
+	umrwr.ignore_free_state = 1;

 	return mlx5_ib_post_send_wait(dev, &umrwr);
 }
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 2a97619ed603..615cc6771516 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -4295,10 +4295,14 @@ static int set_reg_umr_segment(struct mlx5_ib_dev *dev,

 	memset(umr, 0, sizeof(*umr));

-	if (wr->send_flags & MLX5_IB_SEND_UMR_FAIL_IF_FREE)
-		umr->flags = MLX5_UMR_CHECK_FREE; /* fail if free */
-	else
-		umr->flags = MLX5_UMR_CHECK_NOT_FREE; /* fail if not free */
+	if (!umrwr->ignore_free_state) {
+		if (wr->send_flags & MLX5_IB_SEND_UMR_FAIL_IF_FREE)
+			 /* fail if free */
+			umr->flags = MLX5_UMR_CHECK_FREE;
+		else
+			/* fail if not free */
+			umr->flags = MLX5_UMR_CHECK_NOT_FREE;
+	}

 	umr->xlt_octowords = cpu_to_be16(get_xlt_octo(umrwr->xlt_size));
 	if (wr->send_flags & MLX5_IB_SEND_UMR_UPDATE_XLT) {
--
2.20.1

[PATCH rdma-rc 03/10] IB/mlx5: Use direct mkey destroy command upon UMR unreg failure

[Leon Romanovsky]

From: Yishai Hadas <yishaih@mellanox.com>

Use direct firmware command to destroy the mkey in case the unreg UMR
operation has failed.

This prevents a case that an mkey will leak out from the cache post a
failure to be destroyed by a UMR WR.

In case the MR cache limit didn't reach a call to add another entry to
the cache instead of the destroyed one is issued.

In addition, replaced a warn message to WARN_ON() as this flow is fatal
and can't happen unless some bug around.

Cc: <stable@vger.kernel.org> # 4.10
Fixes: 49780d42dfc9 ("IB/mlx5: Expose MR cache for mlx5_ib")
Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Artemy Kovalyov <artemyko@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/mr.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 266edaf8029d..b83361aebf28 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -545,13 +545,16 @@ void mlx5_mr_cache_free(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr)
 		return;

 	c = order2idx(dev, mr->order);
-	if (c < 0 || c >= MAX_MR_CACHE_ENTRIES) {
-		mlx5_ib_warn(dev, "order %d, cache index %d\n", mr->order, c);
-		return;
-	}
+	WARN_ON(c < 0 || c >= MAX_MR_CACHE_ENTRIES);

-	if (unreg_umr(dev, mr))
+	if (unreg_umr(dev, mr)) {
+		mr->allocated_from_cache = false;
+		destroy_mkey(dev, mr);
+		ent = &cache->ent[c];
+		if (ent->cur < ent->limit)
+			queue_work(cache->wq, &ent->work);
 		return;
+	}

 	ent = &cache->ent[c];
 	spin_lock_irq(&ent->lock);
--
2.20.1

[PATCH rdma-rc 04/10] IB/mlx5: Fix unreg_umr to set a device PD

[Leon Romanovsky]

From: Yishai Hadas <yishaih@mellanox.com>

Fix unreg_umr to set a device PD (i.e. UMR PD) which can't be given
outside to the kernel.

As the MR addresses are still in place we must ensure that this MR can't
be used by a user space that will get the original PD number and guess
the MR lkey.

Cc: <stable@vger.kernel.org> # 3.10
Fixes: e126ba97dba9 ("mlx5: Add driver for Mellanox Connect-IB adapters")
Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Artemy Kovalyov <artemyko@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/mr.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index b83361aebf28..7274a9b9df58 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1375,8 +1375,10 @@ static int unreg_umr(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr)
 	if (mdev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR)
 		return 0;

-	umrwr.wr.send_flags = MLX5_IB_SEND_UMR_DISABLE_MR;
+	umrwr.wr.send_flags = MLX5_IB_SEND_UMR_DISABLE_MR |
+			      MLX5_IB_SEND_UMR_UPDATE_PD_ACCESS;
 	umrwr.wr.opcode = MLX5_IB_WR_UMR;
+	umrwr.pd = dev->umrc.pd;
 	umrwr.mkey = mr->mmkey.key;
 	umrwr.ignore_free_state = 1;

--
2.20.1

[PATCH rdma-rc 05/10] IB/mlx5: Fix clean_mr() to work in the expected order

[Leon Romanovsky]

From: Yishai Hadas <yishaih@mellanox.com>

Any dma map underlying the MR should only be freed once the MR is fenced
at the hardware.

As of the above we first destroy the MKEY and just after that can safely
call to dma_unmap_single().

Cc: <stable@vger.kernel.org> # 4.3
Fixes: 8a187ee52b04 ("IB/mlx5: Support the new memory registration API")
Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Artemy Kovalyov <artemyko@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/mr.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 7274a9b9df58..2c77456f359f 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1582,10 +1582,10 @@ static void clean_mr(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr)
 		mr->sig = NULL;
 	}

-	mlx5_free_priv_descs(mr);
-
-	if (!allocated_from_cache)
+	if (!allocated_from_cache) {
 		destroy_mkey(dev, mr);
+		mlx5_free_priv_descs(mr);
+	}
 }

 static void dereg_mr(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr)
--
2.20.1

[PATCH rdma-rc 06/10] IB/mlx5: Fix RSS Toeplitz function to be specification aligned

[Leon Romanovsky]

From: Yishai Hadas <yishaih@mellanox.com>

The specification for the Toeplitz function doesn't mention to set the
key explicitly to be symmetric. In case a symmetric functionality is
required a symmetric key can be simply used.

Fix the code accordingly.

Cc: <stable@vger.kernel.org> # 4.7
Fixes: 28d6137008b2aa ("IB/mlx5: Add RSS QP support")
Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Alex Vainman <alexv@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 615cc6771516..379328b2598f 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -1713,7 +1713,6 @@ static int create_rss_raw_qp_tir(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp,
 		}

 		MLX5_SET(tirc, tirc, rx_hash_fn, MLX5_RX_HASH_FN_TOEPLITZ);
-		MLX5_SET(tirc, tirc, rx_hash_symmetric, 1);
 		memcpy(rss_key, ucmd.rx_hash_key, len);
 		break;
 	}
--
2.20.1

[PATCH rdma-rc 07/10] IB/mlx5: Prevent concurrent MR updates during invalidation

[Leon Romanovsky]

From: Moni Shoua <monis@mellanox.com>

Device requires that memory registration work requests that update the
address translation table of a MR will be fenced if posted together.
This scenario can happen when address ranges are invalidated by the
mmu in separate concurrent calls to the invalidation callback.

We prefer to block concurrent address updates for a single MR over
fencing since making the decision if a WQE needs fencing will be more
expensive and fencing all WQEs is a too radical choice.

Fixes: b4cfe447d47b ("IB/mlx5: Implement on demand paging by adding support for MMU notifiers")
Signed-off-by: Moni Shoua <monis@mellanox.com>
Reviewed-by: Artemy Kovalyov <artemyko@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/odp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c
index 5b642d81e617..b54b851d54e8 100644
--- a/drivers/infiniband/hw/mlx5/odp.c
+++ b/drivers/infiniband/hw/mlx5/odp.c
@@ -246,7 +246,7 @@ void mlx5_ib_invalidate_range(struct ib_umem_odp *umem_odp, unsigned long start,
 	 * overwrite the same MTTs.  Concurent invalidations might race us,
 	 * but they will write 0s as well, so no difference in the end result.
 	 */
-
+	mutex_lock(&umem_odp->umem_mutex);
 	for (addr = start; addr < end; addr += BIT(umem_odp->page_shift)) {
 		idx = (addr - ib_umem_start(umem_odp)) >> umem_odp->page_shift;
 		/*
@@ -278,6 +278,7 @@ void mlx5_ib_invalidate_range(struct ib_umem_odp *umem_odp, unsigned long start,
 				   idx - blk_start_idx + 1, 0,
 				   MLX5_IB_UPD_XLT_ZAP |
 				   MLX5_IB_UPD_XLT_ATOMIC);
+	mutex_unlock(&umem_odp->umem_mutex);
 	/*
 	 * We are now sure that the device will not access the
 	 * memory. We can safely unmap it, and mark it as dirty if
--
2.20.1

[PATCH rdma-rc 08/10] IB/mlx5: Avoid unnecessary typecast

[Leon Romanovsky]

From: Parav Pandit <parav@mellanox.com>

IB device pointer is already available while deallocating IB device,
Hence do not typecast it.

Signed-off-by: Parav Pandit <parav@mellanox.com>
Reviewed-by: Daniel Jurgens <danielj@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index c2a5780cb394..6cdd98ec9c1e 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -6933,7 +6933,7 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev)
 	dev->port = kcalloc(num_ports, sizeof(*dev->port),
 			     GFP_KERNEL);
 	if (!dev->port) {
-		ib_dealloc_device((struct ib_device *)dev);
+		ib_dealloc_device(&dev->ib_dev);
 		return NULL;
 	}

--
2.20.1

[PATCH rdma-rc 09/10] IB/core: Fix querying total rdma stats

[Leon Romanovsky]

From: Parav Pandit <parav@mellanox.com>

rdma_counter_init() may fail for a device. In such case while calculating
total sum, ignore NULL hstats.

This fixes below observed call trace.

BUG: kernel NULL pointer dereference, address: 00000000000000a0
PGD 8000001009b30067 P4D 8000001009b30067 PUD 10549c9067 PMD 0
Oops: 0000 [#1] SMP PTI
CPU: 55 PID: 20887 Comm: cat Kdump: loaded Not tainted 5.2.0-rc6-jdc+ #13
RIP: 0010:rdma_counter_get_hwstat_value+0xf2/0x150 [ib_core]
Call Trace:
 show_hw_stats+0x5e/0x130 [ib_core]
 dev_attr_show+0x15/0x50
 sysfs_kf_seq_show+0xc6/0x1a0
 seq_read+0x132/0x370
 vfs_read+0x89/0x140
 ksys_read+0x5c/0xd0
 do_syscall_64+0x5a/0x240
 entry_SYSCALL_64_after_hwframe+0x49/0xbe

Fixes: f34a55e497e81 ("RDMA/core: Get sum value of all counters when perform a sysfs stat read")
Signed-off-by: Parav Pandit <parav@mellanox.com>
Reviewed-by: Mark Zhang <markz@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/core/counters.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/infiniband/core/counters.c b/drivers/infiniband/core/counters.c
index 01faef7bc061..c7d445635476 100644
--- a/drivers/infiniband/core/counters.c
+++ b/drivers/infiniband/core/counters.c
@@ -393,6 +393,9 @@ u64 rdma_counter_get_hwstat_value(struct ib_device *dev, u8 port, u32 index)
 	u64 sum;

 	port_counter = &dev->port_data[port].port_counter;
+	if (!port_counter->hstats)
+		return 0;
+
 	sum = get_running_counters_hwstat_sum(dev, port, index);
 	sum += port_counter->hstats->value[index];

--
2.20.1

[PATCH rdma-rc 10/10] IB/counters: Initialize port counter and annotate mutex_destroy

[Leon Romanovsky]

From: Parav Pandit <parav@mellanox.com>

Annotate mutex_destroy for port counters during counters release
operation and during error unwinding during init flow.

Also port counter object should be initialized even if alloc_stats is
unsupported, so that other QP bind operations can avoid call trace
if they try to bind QP on RDMA device which doesn't support counters.

Fixes: f34a55e497e81 ("RDMA/core: Get sum value of all counters when perform a sysfs stat read")
Signed-off-by: Parav Pandit <parav@mellanox.com>
Reviewed-by: Mark Zhang <markz@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/core/counters.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/core/counters.c b/drivers/infiniband/core/counters.c
index c7d445635476..d60416f0bf3a 100644
--- a/drivers/infiniband/core/counters.c
+++ b/drivers/infiniband/core/counters.c
@@ -595,9 +595,9 @@ int rdma_counter_get_mode(struct ib_device *dev, u8 port,
 void rdma_counter_init(struct ib_device *dev)
 {
 	struct rdma_port_counter *port_counter;
-	u32 port;
+	u32 port, i;

-	if (!dev->ops.alloc_hw_stats || !dev->port_data)
+	if (!dev->port_data)
 		return;

 	rdma_for_each_port(dev, port) {
@@ -605,6 +605,9 @@ void rdma_counter_init(struct ib_device *dev)
 		port_counter->mode.mode = RDMA_COUNTER_MODE_NONE;
 		mutex_init(&port_counter->lock);

+		if (!dev->ops.alloc_hw_stats)
+			continue;
+
 		port_counter->hstats = dev->ops.alloc_hw_stats(dev, port);
 		if (!port_counter->hstats)
 			goto fail;
@@ -613,13 +616,12 @@ void rdma_counter_init(struct ib_device *dev)
 	return;

 fail:
-	rdma_for_each_port(dev, port) {
+	for (i = port; i >= rdma_start_port(dev); i--) {
 		port_counter = &dev->port_data[port].port_counter;
 		kfree(port_counter->hstats);
 		port_counter->hstats = NULL;
+		mutex_destroy(&port_counter->lock);
 	}
-
-	return;
 }

 void rdma_counter_release(struct ib_device *dev)
@@ -627,11 +629,9 @@ void rdma_counter_release(struct ib_device *dev)
 	struct rdma_port_counter *port_counter;
 	u32 port;

-	if (!dev->ops.alloc_hw_stats)
-		return;
-
 	rdma_for_each_port(dev, port) {
 		port_counter = &dev->port_data[port].port_counter;
 		kfree(port_counter->hstats);
+		mutex_destroy(&port_counter->lock);
 	}
 }
--
2.20.1

Re: [PATCH rdma-rc 10/10] IB/counters: Initialize port counter and annotate mutex_destroy

[Jason Gunthorpe]

On Tue, Jul 23, 2019 at 09:57:33AM +0300, Leon Romanovsky wrote:
> From: Parav Pandit <parav@mellanox.com>
> 
> Annotate mutex_destroy for port counters during counters release
> operation and during error unwinding during init flow.
> 
> Also port counter object should be initialized even if alloc_stats is
> unsupported, so that other QP bind operations can avoid call trace
> if they try to bind QP on RDMA device which doesn't support counters.
> 
> Fixes: f34a55e497e81 ("RDMA/core: Get sum value of all counters when perform a sysfs stat read")
> Signed-off-by: Parav Pandit <parav@mellanox.com>
> Reviewed-by: Mark Zhang <markz@mellanox.com>
> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
> ---
>  drivers/infiniband/core/counters.c | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)

Don't put two things in a for-rc patch, I split it.

Jason

Re: [PATCH rdma-rc 00/10] Collection of fixes for 5.3

[Jason Gunthorpe]

On Tue, Jul 23, 2019 at 09:57:23AM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
> 
> Hi,
> 
> This is small patch set of fixes targeted for 5.3 and stable@.
> 
> Thanks
> 
> Moni Shoua (1):
>   IB/mlx5: Prevent concurrent MR updates during invalidation
> 
> Parav Pandit (4):
>   IB/core: Fix querying total rdma stats
>   IB/counters: Initialize port counter and annotate mutex_destroy
> 
> Yishai Hadas (5):
>   IB/mlx5: Fix unreg_umr to ignore the mkey state
>   IB/mlx5: Use direct mkey destroy command upon UMR unreg failure
>   IB/mlx5: Fix unreg_umr to set a device PD
>   IB/mlx5: Fix clean_mr() to work in the expected order
>   IB/mlx5: Fix RSS Toeplitz function to be specification aligned

I took the above for for-rc

>   IB/mlx5: Avoid unnecessary typecast
>   RDMA/core: Annotate destroy of mutex to ensure that it is released as
>     unlocked

These are not really -rc patches, I took them to for-next

Jason

RE: [PATCH rdma-rc 00/10] Collection of fixes for 5.3

[Parav Pandit]

> -----Original Message-----
> From: Jason Gunthorpe <jgg@ziepe.ca>
> Sent: Thursday, July 25, 2019 8:43 PM
> To: Leon Romanovsky <leon@kernel.org>
> Cc: Doug Ledford <dledford@redhat.com>; Leon Romanovsky
> <leonro@mellanox.com>; RDMA mailing list <linux-rdma@vger.kernel.org>;
> Alex Vainman <alexv@mellanox.com>; Artemy Kovalyov
> <artemyko@mellanox.com>; Daniel Jurgens <danielj@mellanox.com>; Eli
> Cohen <eli@mellanox.com>; Haggai Eran <haggaie@mellanox.com>; Mark
> Zhang <markz@mellanox.com>; Moni Shoua <monis@mellanox.com>; Parav
> Pandit <parav@mellanox.com>; Sagi Grimberg <sagig@mellanox.com>; Yishai
> Hadas <yishaih@mellanox.com>
> Subject: Re: [PATCH rdma-rc 00/10] Collection of fixes for 5.3
> 
> On Tue, Jul 23, 2019 at 09:57:23AM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > Hi,
> >
> > This is small patch set of fixes targeted for 5.3 and stable@.
> >
> > Thanks
> >
> > Moni Shoua (1):
> >   IB/mlx5: Prevent concurrent MR updates during invalidation
> >
> > Parav Pandit (4):
> >   IB/core: Fix querying total rdma stats
> >   IB/counters: Initialize port counter and annotate mutex_destroy
> >
> > Yishai Hadas (5):
> >   IB/mlx5: Fix unreg_umr to ignore the mkey state
> >   IB/mlx5: Use direct mkey destroy command upon UMR unreg failure
> >   IB/mlx5: Fix unreg_umr to set a device PD
> >   IB/mlx5: Fix clean_mr() to work in the expected order
> >   IB/mlx5: Fix RSS Toeplitz function to be specification aligned
> 
> I took the above for for-rc
> 
> >   IB/mlx5: Avoid unnecessary typecast
> >   RDMA/core: Annotate destroy of mutex to ensure that it is released as
> >     unlocked
> 
> These are not really -rc patches, I took them to for-next
> 
Thank you Jason.