* [PATCH v2 for-next 0/5] Misc patches for RTRS
@ 2022-07-12 10:31 Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 1/5] RDMA/rtrs-srv: Fix modinfo output for stringify Md Haris Iqbal
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: Md Haris Iqbal @ 2022-07-12 10:31 UTC (permalink / raw)
To: linux-rdma; +Cc: leon, jgg, haris.iqbal, jinpu.wang
Hi Jason, hi Leon,
Please consider to include following changes to the next merge window.
The patchset is organized as follows:
1: change to make stringify work for a module param
2: a change to avoid disable/enable of preemption
3: replace normal stats structure with percpu version
4: Fixes some checkpatch warnings
5: removes allocation and usage of mempool
Jack Wang (2):
RDMA/rtrs-srv: Fix modinfo output for stringify
RDMA/rtrs-srv: Do not use mempool for page allocation
Md Haris Iqbal (1):
RDMA/rtrs-clt: Replace list_next_or_null_rr_rcu with an inline
function
Santosh Kumar Pradhan (2):
RDMA/rtrs-clt: Use this_cpu_ API for stats
RDMA/rtrs-srv: Use per-cpu variables for rdma stats
drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 14 ++------
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 35 +++++++++-----------
drivers/infiniband/ulp/rtrs/rtrs-pri.h | 21 ++++++------
drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c | 32 +++++++++++++-----
drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c | 2 ++
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 32 ++++++++----------
drivers/infiniband/ulp/rtrs/rtrs-srv.h | 15 +++++----
7 files changed, 78 insertions(+), 73 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2 for-next 1/5] RDMA/rtrs-srv: Fix modinfo output for stringify
2022-07-12 10:31 [PATCH v2 for-next 0/5] Misc patches for RTRS Md Haris Iqbal
@ 2022-07-12 10:31 ` Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 2/5] RDMA/rtrs-clt: Use this_cpu_ API for stats Md Haris Iqbal
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Md Haris Iqbal @ 2022-07-12 10:31 UTC (permalink / raw)
To: linux-rdma; +Cc: leon, jgg, haris.iqbal, jinpu.wang, Aleksei Marov
From: Jack Wang <jinpu.wang@ionos.com>
stringify works with define, not enum.
Fixes: 91fddedd439c ("RDMA/rtrs: private headers with rtrs protocol structs and helpers")
Cc: jinpu.wang@ionos.com
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Reviewed-by: Aleksei Marov <aleksei.marov@ionos.com>
---
v1 -> v2: No change
drivers/infiniband/ulp/rtrs/rtrs-pri.h | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-pri.h b/drivers/infiniband/ulp/rtrs/rtrs-pri.h
index 9a1e5c2ae55c..ac0df734eba8 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-pri.h
+++ b/drivers/infiniband/ulp/rtrs/rtrs-pri.h
@@ -23,6 +23,17 @@
#define RTRS_PROTO_VER_STRING __stringify(RTRS_PROTO_VER_MAJOR) "." \
__stringify(RTRS_PROTO_VER_MINOR)
+/*
+ * Max IB immediate data size is 2^28 (MAX_IMM_PAYL_BITS)
+ * and the minimum chunk size is 4096 (2^12).
+ * So the maximum sess_queue_depth is 65536 (2^16) in theory.
+ * But mempool_create, create_qp and ib_post_send fail with
+ * "cannot allocate memory" error if sess_queue_depth is too big.
+ * Therefore the pratical max value of sess_queue_depth is
+ * somewhere between 1 and 65534 and it depends on the system.
+ */
+#define MAX_SESS_QUEUE_DEPTH 65535
+
enum rtrs_imm_const {
MAX_IMM_TYPE_BITS = 4,
MAX_IMM_TYPE_MASK = ((1 << MAX_IMM_TYPE_BITS) - 1),
@@ -46,16 +57,6 @@ enum {
MAX_PATHS_NUM = 128,
- /*
- * Max IB immediate data size is 2^28 (MAX_IMM_PAYL_BITS)
- * and the minimum chunk size is 4096 (2^12).
- * So the maximum sess_queue_depth is 65536 (2^16) in theory.
- * But mempool_create, create_qp and ib_post_send fail with
- * "cannot allocate memory" error if sess_queue_depth is too big.
- * Therefore the pratical max value of sess_queue_depth is
- * somewhere between 1 and 65534 and it depends on the system.
- */
- MAX_SESS_QUEUE_DEPTH = 65535,
MIN_CHUNK_SIZE = 8192,
RTRS_HB_INTERVAL_MS = 5000,
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 for-next 2/5] RDMA/rtrs-clt: Use this_cpu_ API for stats
2022-07-12 10:31 [PATCH v2 for-next 0/5] Misc patches for RTRS Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 1/5] RDMA/rtrs-srv: Fix modinfo output for stringify Md Haris Iqbal
@ 2022-07-12 10:31 ` Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 3/5] RDMA/rtrs-srv: Use per-cpu variables for rdma stats Md Haris Iqbal
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Md Haris Iqbal @ 2022-07-12 10:31 UTC (permalink / raw)
To: linux-rdma
Cc: leon, jgg, haris.iqbal, jinpu.wang, Santosh Kumar Pradhan,
Christoph Lameter, Guoqing Jiang
From: Santosh Kumar Pradhan <santosh.pradhan@ionos.com>
Use this_cpu_x() for increasing/adding a percpu counter through a
percpu pointer without the need to disable/enable preemption.
Suggested-by: Christoph Lameter <cl@linux.com>
Signed-off-by: Santosh Kumar Pradhan <santosh.pradhan@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Acked-by: Guoqing Jiang <guoqing.jiang@linux.dev>
Reviewed-by: Christoph Lameter <cl@linux.com>
---
v1 -> v2: Added Acked-by, Reviewed-by.
drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 14 +++-----------
1 file changed, 3 insertions(+), 11 deletions(-)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
index 385a19846c24..1e6ffafa2db3 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
@@ -32,11 +32,7 @@ void rtrs_clt_update_wc_stats(struct rtrs_clt_con *con)
void rtrs_clt_inc_failover_cnt(struct rtrs_clt_stats *stats)
{
- struct rtrs_clt_stats_pcpu *s;
-
- s = get_cpu_ptr(stats->pcpu_stats);
- s->rdma.failover_cnt++;
- put_cpu_ptr(stats->pcpu_stats);
+ this_cpu_inc(stats->pcpu_stats->rdma.failover_cnt);
}
int rtrs_clt_stats_migration_from_cnt_to_str(struct rtrs_clt_stats *stats, char *buf)
@@ -169,12 +165,8 @@ int rtrs_clt_reset_all_stats(struct rtrs_clt_stats *s, bool enable)
static inline void rtrs_clt_update_rdma_stats(struct rtrs_clt_stats *stats,
size_t size, int d)
{
- struct rtrs_clt_stats_pcpu *s;
-
- s = get_cpu_ptr(stats->pcpu_stats);
- s->rdma.dir[d].cnt++;
- s->rdma.dir[d].size_total += size;
- put_cpu_ptr(stats->pcpu_stats);
+ this_cpu_inc(stats->pcpu_stats->rdma.dir[d].cnt);
+ this_cpu_add(stats->pcpu_stats->rdma.dir[d].size_total, size);
}
void rtrs_clt_update_all_stats(struct rtrs_clt_io_req *req, int dir)
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 for-next 3/5] RDMA/rtrs-srv: Use per-cpu variables for rdma stats
2022-07-12 10:31 [PATCH v2 for-next 0/5] Misc patches for RTRS Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 1/5] RDMA/rtrs-srv: Fix modinfo output for stringify Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 2/5] RDMA/rtrs-clt: Use this_cpu_ API for stats Md Haris Iqbal
@ 2022-07-12 10:31 ` Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 4/5] RDMA/rtrs-clt: Replace list_next_or_null_rr_rcu with an inline function Md Haris Iqbal
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Md Haris Iqbal @ 2022-07-12 10:31 UTC (permalink / raw)
To: linux-rdma; +Cc: leon, jgg, haris.iqbal, jinpu.wang, Santosh Kumar Pradhan
From: Santosh Kumar Pradhan <santosh.pradhan@ionos.com>
Convert server stat counters from atomic to per-cpu variables.
Signed-off-by: Santosh Kumar Pradhan <santosh.pradhan@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
---
v1 -> v2: No change
drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c | 32 +++++++++++++++-----
drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c | 2 ++
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 9 +++++-
drivers/infiniband/ulp/rtrs/rtrs-srv.h | 15 ++++-----
4 files changed, 42 insertions(+), 16 deletions(-)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c
index 44b1c1652131..2aff1213a19d 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c
@@ -14,9 +14,14 @@
int rtrs_srv_reset_rdma_stats(struct rtrs_srv_stats *stats, bool enable)
{
if (enable) {
- struct rtrs_srv_stats_rdma_stats *r = &stats->rdma_stats;
+ int cpu;
+ struct rtrs_srv_stats_rdma_stats *r;
+
+ for_each_possible_cpu(cpu) {
+ r = per_cpu_ptr(stats->rdma_stats, cpu);
+ memset(r, 0, sizeof(*r));
+ }
- memset(r, 0, sizeof(*r));
return 0;
}
@@ -25,11 +30,22 @@ int rtrs_srv_reset_rdma_stats(struct rtrs_srv_stats *stats, bool enable)
ssize_t rtrs_srv_stats_rdma_to_str(struct rtrs_srv_stats *stats, char *page)
{
- struct rtrs_srv_stats_rdma_stats *r = &stats->rdma_stats;
+ int cpu;
+ struct rtrs_srv_stats_rdma_stats sum;
+ struct rtrs_srv_stats_rdma_stats *r;
+
+ memset(&sum, 0, sizeof(sum));
+
+ for_each_possible_cpu(cpu) {
+ r = per_cpu_ptr(stats->rdma_stats, cpu);
+
+ sum.dir[READ].cnt += r->dir[READ].cnt;
+ sum.dir[READ].size_total += r->dir[READ].size_total;
+ sum.dir[WRITE].cnt += r->dir[WRITE].cnt;
+ sum.dir[WRITE].size_total += r->dir[WRITE].size_total;
+ }
- return sysfs_emit(page, "%lld %lld %lld %lldn %u\n",
- (s64)atomic64_read(&r->dir[READ].cnt),
- (s64)atomic64_read(&r->dir[READ].size_total),
- (s64)atomic64_read(&r->dir[WRITE].cnt),
- (s64)atomic64_read(&r->dir[WRITE].size_total), 0);
+ return sysfs_emit(page, "%llu %llu %llu %llu\n",
+ sum.dir[READ].cnt, sum.dir[READ].size_total,
+ sum.dir[WRITE].cnt, sum.dir[WRITE].size_total);
}
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c b/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c
index b94ae12c2795..2a3c9ac64a42 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c
@@ -220,6 +220,8 @@ static void rtrs_srv_path_stats_release(struct kobject *kobj)
stats = container_of(kobj, struct rtrs_srv_stats, kobj_stats);
+ free_percpu(stats->rdma_stats);
+
kfree(stats);
}
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c
index 24024bce2566..8278d3600a36 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c
@@ -1513,6 +1513,7 @@ static void free_path(struct rtrs_srv_path *srv_path)
kobject_del(&srv_path->kobj);
kobject_put(&srv_path->kobj);
} else {
+ free_percpu(srv_path->stats->rdma_stats);
kfree(srv_path->stats);
kfree(srv_path);
}
@@ -1755,13 +1756,17 @@ static struct rtrs_srv_path *__alloc_path(struct rtrs_srv_sess *srv,
if (!srv_path->stats)
goto err_free_sess;
+ srv_path->stats->rdma_stats = alloc_percpu(struct rtrs_srv_stats_rdma_stats);
+ if (!srv_path->stats->rdma_stats)
+ goto err_free_stats;
+
srv_path->stats->srv_path = srv_path;
srv_path->dma_addr = kcalloc(srv->queue_depth,
sizeof(*srv_path->dma_addr),
GFP_KERNEL);
if (!srv_path->dma_addr)
- goto err_free_stats;
+ goto err_free_percpu;
srv_path->s.con = kcalloc(con_num, sizeof(*srv_path->s.con),
GFP_KERNEL);
@@ -1813,6 +1818,8 @@ static struct rtrs_srv_path *__alloc_path(struct rtrs_srv_sess *srv,
kfree(srv_path->s.con);
err_free_dma_addr:
kfree(srv_path->dma_addr);
+err_free_percpu:
+ free_percpu(srv_path->stats->rdma_stats);
err_free_stats:
kfree(srv_path->stats);
err_free_sess:
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.h b/drivers/infiniband/ulp/rtrs/rtrs-srv.h
index 6292e87f6afd..186a63c217df 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-srv.h
+++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.h
@@ -12,6 +12,7 @@
#include <linux/device.h>
#include <linux/refcount.h>
+#include <linux/percpu.h>
#include "rtrs-pri.h"
/*
@@ -29,15 +30,15 @@ enum rtrs_srv_state {
*/
struct rtrs_srv_stats_rdma_stats {
struct {
- atomic64_t cnt;
- atomic64_t size_total;
+ u64 cnt;
+ u64 size_total;
} dir[2];
};
struct rtrs_srv_stats {
- struct kobject kobj_stats;
- struct rtrs_srv_stats_rdma_stats rdma_stats;
- struct rtrs_srv_path *srv_path;
+ struct kobject kobj_stats;
+ struct rtrs_srv_stats_rdma_stats __percpu *rdma_stats;
+ struct rtrs_srv_path *srv_path;
};
struct rtrs_srv_con {
@@ -130,8 +131,8 @@ void close_path(struct rtrs_srv_path *srv_path);
static inline void rtrs_srv_update_rdma_stats(struct rtrs_srv_stats *s,
size_t size, int d)
{
- atomic64_inc(&s->rdma_stats.dir[d].cnt);
- atomic64_add(size, &s->rdma_stats.dir[d].size_total);
+ this_cpu_inc(s->rdma_stats->dir[d].cnt);
+ this_cpu_add(s->rdma_stats->dir[d].size_total, size);
}
/* functions which are implemented in rtrs-srv-stats.c */
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 for-next 4/5] RDMA/rtrs-clt: Replace list_next_or_null_rr_rcu with an inline function
2022-07-12 10:31 [PATCH v2 for-next 0/5] Misc patches for RTRS Md Haris Iqbal
` (2 preceding siblings ...)
2022-07-12 10:31 ` [PATCH v2 for-next 3/5] RDMA/rtrs-srv: Use per-cpu variables for rdma stats Md Haris Iqbal
@ 2022-07-12 10:31 ` Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 5/5] RDMA/rtrs-srv: Do not use mempool for page allocation Md Haris Iqbal
2022-07-18 9:29 ` [PATCH v2 for-next 0/5] Misc patches for RTRS Leon Romanovsky
5 siblings, 0 replies; 7+ messages in thread
From: Md Haris Iqbal @ 2022-07-12 10:31 UTC (permalink / raw)
To: linux-rdma; +Cc: leon, jgg, haris.iqbal, jinpu.wang
removes list_next_or_null_rr_rcu macro to fix below warnings.
That macro is used only twice.
CHECK:MACRO_ARG_REUSE: Macro argument reuse 'head' - possible side-effects?
CHECK:MACRO_ARG_REUSE: Macro argument reuse 'ptr' - possible side-effects?
CHECK:MACRO_ARG_REUSE: Macro argument reuse 'memb' - possible side-effects?
Replaces that macro with an inline function.
Fixes: 6a98d71daea1 ("RDMA/rtrs: client: main functionality")
Cc: jinpu.wang@ionos.com
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 35 ++++++++++++--------------
1 file changed, 16 insertions(+), 19 deletions(-)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
index 9809c3883979..525f083fcaeb 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
@@ -740,25 +740,25 @@ struct path_it {
struct rtrs_clt_path *(*next_path)(struct path_it *it);
};
-/**
- * list_next_or_null_rr_rcu - get next list element in round-robin fashion.
+/*
+ * rtrs_clt_get_next_path_or_null - get clt path from the list or return NULL
* @head: the head for the list.
- * @ptr: the list head to take the next element from.
- * @type: the type of the struct this is embedded in.
- * @memb: the name of the list_head within the struct.
+ * @clt_path: The element to take the next clt_path from.
*
- * Next element returned in round-robin fashion, i.e. head will be skipped,
+ * Next clt path returned in round-robin fashion, i.e. head will be skipped,
* but if list is observed as empty, NULL will be returned.
*
- * This primitive may safely run concurrently with the _rcu list-mutation
+ * This function may safely run concurrently with the _rcu list-mutation
* primitives such as list_add_rcu() as long as it's guarded by rcu_read_lock().
*/
-#define list_next_or_null_rr_rcu(head, ptr, type, memb) \
-({ \
- list_next_or_null_rcu(head, ptr, type, memb) ?: \
- list_next_or_null_rcu(head, READ_ONCE((ptr)->next), \
- type, memb); \
-})
+static inline struct rtrs_clt_path *
+rtrs_clt_get_next_path_or_null(struct list_head *head, struct rtrs_clt_path *clt_path)
+{
+ return list_next_or_null_rcu(head, &clt_path->s.entry, typeof(*clt_path), s.entry) ?:
+ list_next_or_null_rcu(head,
+ READ_ONCE((&clt_path->s.entry)->next),
+ typeof(*clt_path), s.entry);
+}
/**
* get_next_path_rr() - Returns path in round-robin fashion.
@@ -789,10 +789,8 @@ static struct rtrs_clt_path *get_next_path_rr(struct path_it *it)
path = list_first_or_null_rcu(&clt->paths_list,
typeof(*path), s.entry);
else
- path = list_next_or_null_rr_rcu(&clt->paths_list,
- &path->s.entry,
- typeof(*path),
- s.entry);
+ path = rtrs_clt_get_next_path_or_null(&clt->paths_list, path);
+
rcu_assign_pointer(*ppcpu_path, path);
return path;
@@ -2277,8 +2275,7 @@ static void rtrs_clt_remove_path_from_arr(struct rtrs_clt_path *clt_path)
* removed. If @sess is the last element, then @next is NULL.
*/
rcu_read_lock();
- next = list_next_or_null_rr_rcu(&clt->paths_list, &clt_path->s.entry,
- typeof(*next), s.entry);
+ next = rtrs_clt_get_next_path_or_null(&clt->paths_list, clt_path);
rcu_read_unlock();
/*
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 for-next 5/5] RDMA/rtrs-srv: Do not use mempool for page allocation
2022-07-12 10:31 [PATCH v2 for-next 0/5] Misc patches for RTRS Md Haris Iqbal
` (3 preceding siblings ...)
2022-07-12 10:31 ` [PATCH v2 for-next 4/5] RDMA/rtrs-clt: Replace list_next_or_null_rr_rcu with an inline function Md Haris Iqbal
@ 2022-07-12 10:31 ` Md Haris Iqbal
2022-07-18 9:29 ` [PATCH v2 for-next 0/5] Misc patches for RTRS Leon Romanovsky
5 siblings, 0 replies; 7+ messages in thread
From: Md Haris Iqbal @ 2022-07-12 10:31 UTC (permalink / raw)
To: linux-rdma
Cc: leon, jgg, haris.iqbal, jinpu.wang, kernel test robot, Gioh Kim,
Guoqing Jiang
From: Jack Wang <jinpu.wang@ionos.com>
The mempool is for guaranteed memory allocation during
extreme VM load (see the header of mempool.c of the kernel).
But rtrs-srv allocates pages only when creating new session.
There is no need to use the mempool.
With the removal of mempool, rtrs-server no longer need to reserve
huge mount of memory, this will avoid error like this:
https://lore.kernel.org/lkml/20220620020727.GA3669@xsang-OptiPlex-9020/
Reported-by: kernel test robot <oliver.sang@intel.com>
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Acked-by: Guoqing Jiang <guoqing.jiang@linux.dev>
---
v1 -> v2: Added Reported-by, Acked-by.
Changed email link to lore.kernel.org
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 23 ++++++-----------------
1 file changed, 6 insertions(+), 17 deletions(-)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c
index 8278d3600a36..34c03bde5064 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c
@@ -11,7 +11,6 @@
#define pr_fmt(fmt) KBUILD_MODNAME " L" __stringify(__LINE__) ": " fmt
#include <linux/module.h>
-#include <linux/mempool.h>
#include "rtrs-srv.h"
#include "rtrs-log.h"
@@ -26,11 +25,7 @@ MODULE_LICENSE("GPL");
#define DEFAULT_SESS_QUEUE_DEPTH 512
#define MAX_HDR_SIZE PAGE_SIZE
-/* We guarantee to serve 10 paths at least */
-#define CHUNK_POOL_SZ 10
-
static struct rtrs_rdma_dev_pd dev_pd;
-static mempool_t *chunk_pool;
struct class *rtrs_dev_class;
static struct rtrs_srv_ib_ctx ib_ctx;
@@ -1358,7 +1353,7 @@ static void free_srv(struct rtrs_srv_sess *srv)
WARN_ON(refcount_read(&srv->refcount));
for (i = 0; i < srv->queue_depth; i++)
- mempool_free(srv->chunks[i], chunk_pool);
+ __free_pages(srv->chunks[i], get_order(max_chunk_size));
kfree(srv->chunks);
mutex_destroy(&srv->paths_mutex);
mutex_destroy(&srv->paths_ev_mutex);
@@ -1411,7 +1406,8 @@ static struct rtrs_srv_sess *get_or_create_srv(struct rtrs_srv_ctx *ctx,
goto err_free_srv;
for (i = 0; i < srv->queue_depth; i++) {
- srv->chunks[i] = mempool_alloc(chunk_pool, GFP_KERNEL);
+ srv->chunks[i] = alloc_pages(GFP_KERNEL,
+ get_order(max_chunk_size));
if (!srv->chunks[i])
goto err_free_chunks;
}
@@ -1424,7 +1420,7 @@ static struct rtrs_srv_sess *get_or_create_srv(struct rtrs_srv_ctx *ctx,
err_free_chunks:
while (i--)
- mempool_free(srv->chunks[i], chunk_pool);
+ __free_pages(srv->chunks[i], get_order(max_chunk_size));
kfree(srv->chunks);
err_free_srv:
@@ -2273,14 +2269,10 @@ static int __init rtrs_server_init(void)
err);
return err;
}
- chunk_pool = mempool_create_page_pool(sess_queue_depth * CHUNK_POOL_SZ,
- get_order(max_chunk_size));
- if (!chunk_pool)
- return -ENOMEM;
rtrs_dev_class = class_create(THIS_MODULE, "rtrs-server");
if (IS_ERR(rtrs_dev_class)) {
err = PTR_ERR(rtrs_dev_class);
- goto out_chunk_pool;
+ goto out_err;
}
rtrs_wq = alloc_workqueue("rtrs_server_wq", 0, 0);
if (!rtrs_wq) {
@@ -2292,9 +2284,7 @@ static int __init rtrs_server_init(void)
out_dev_class:
class_destroy(rtrs_dev_class);
-out_chunk_pool:
- mempool_destroy(chunk_pool);
-
+out_err:
return err;
}
@@ -2302,7 +2292,6 @@ static void __exit rtrs_server_exit(void)
{
destroy_workqueue(rtrs_wq);
class_destroy(rtrs_dev_class);
- mempool_destroy(chunk_pool);
rtrs_rdma_dev_pd_deinit(&dev_pd);
}
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v2 for-next 0/5] Misc patches for RTRS
2022-07-12 10:31 [PATCH v2 for-next 0/5] Misc patches for RTRS Md Haris Iqbal
` (4 preceding siblings ...)
2022-07-12 10:31 ` [PATCH v2 for-next 5/5] RDMA/rtrs-srv: Do not use mempool for page allocation Md Haris Iqbal
@ 2022-07-18 9:29 ` Leon Romanovsky
5 siblings, 0 replies; 7+ messages in thread
From: Leon Romanovsky @ 2022-07-18 9:29 UTC (permalink / raw)
To: Md Haris Iqbal; +Cc: linux-rdma, jgg, jinpu.wang
On Tue, Jul 12, 2022 at 12:31:08PM +0200, Md Haris Iqbal wrote:
> Hi Jason, hi Leon,
>
> Please consider to include following changes to the next merge window.
>
> The patchset is organized as follows:
> 1: change to make stringify work for a module param
> 2: a change to avoid disable/enable of preemption
> 3: replace normal stats structure with percpu version
> 4: Fixes some checkpatch warnings
> 5: removes allocation and usage of mempool
>
> Jack Wang (2):
> RDMA/rtrs-srv: Fix modinfo output for stringify
> RDMA/rtrs-srv: Do not use mempool for page allocation
>
> Md Haris Iqbal (1):
> RDMA/rtrs-clt: Replace list_next_or_null_rr_rcu with an inline
> function
>
> Santosh Kumar Pradhan (2):
> RDMA/rtrs-clt: Use this_cpu_ API for stats
> RDMA/rtrs-srv: Use per-cpu variables for rdma stats
>
> drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 14 ++------
> drivers/infiniband/ulp/rtrs/rtrs-clt.c | 35 +++++++++-----------
> drivers/infiniband/ulp/rtrs/rtrs-pri.h | 21 ++++++------
> drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c | 32 +++++++++++++-----
> drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c | 2 ++
> drivers/infiniband/ulp/rtrs/rtrs-srv.c | 32 ++++++++----------
> drivers/infiniband/ulp/rtrs/rtrs-srv.h | 15 +++++----
> 7 files changed, 78 insertions(+), 73 deletions(-)
I took the patches, but please next time do not use vertical space
alignment for new code.
Thanks
>
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-07-18 9:29 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-12 10:31 [PATCH v2 for-next 0/5] Misc patches for RTRS Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 1/5] RDMA/rtrs-srv: Fix modinfo output for stringify Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 2/5] RDMA/rtrs-clt: Use this_cpu_ API for stats Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 3/5] RDMA/rtrs-srv: Use per-cpu variables for rdma stats Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 4/5] RDMA/rtrs-clt: Replace list_next_or_null_rr_rcu with an inline function Md Haris Iqbal
2022-07-12 10:31 ` [PATCH v2 for-next 5/5] RDMA/rtrs-srv: Do not use mempool for page allocation Md Haris Iqbal
2022-07-18 9:29 ` [PATCH v2 for-next 0/5] Misc patches for RTRS Leon Romanovsky
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).