From: Jack Wang <jinpu.wang@ionos.com>
To: linux-rdma@vger.kernel.org
Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com,
jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com,
Gioh Kim <gi-oh.kim@ionos.com>
Subject: [PATCH for-next 08/10] RDMA/rtrs-clt: Fix counting inflight IO
Date: Fri, 30 Jul 2021 15:18:30 +0200 [thread overview]
Message-ID: <20210730131832.118865-9-jinpu.wang@ionos.com> (raw)
In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com>
From: Gioh Kim <gi-oh.kim@ionos.com>
There are mis-match at counting inflight IO after changing the
multipath policy.
For example, we started fio test with round-robin policy and then
we changed the policy to min-inflight. IOs created under the RR policy
is finished under the min-inflight policy and inflight counter
only decreased. So the counter would be negative value.
And also we started fio test with min-inflight policy and
changed the policy to the round-robin. IOs created under the
min-inflight policy increased the inflight IO counter but the
inflight IO counter was not decreased because the policy was
the round-robin when IO was finished.
So it should count IOs only if the IO is created under the
min-inflight policy. It should not care the policy when the IO
is finished.
This patch adds a field mp_policy in struct rtrs_clt_io_req and
stores the multipath policy when an object of rtrs_clt_io_req is
created. Then rtrs-clt checks the mp_policy of only struct
rtrs_clt_io_req instead of the struct rtrs_clt.
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 2 +-
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 7 ++++---
drivers/infiniband/ulp/rtrs/rtrs-clt.h | 1 +
3 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
index 4f1980a3608a..61d5e0018392 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
@@ -174,7 +174,7 @@ void rtrs_clt_update_all_stats(struct rtrs_clt_io_req *req, int dir)
len = req->usr_len + req->data_len;
rtrs_clt_update_rdma_stats(stats, len, dir);
- if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT)
+ if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
atomic_inc(&stats->inflight);
}
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
index 3cf7118a1c00..5cce727abca0 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
@@ -438,7 +438,7 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno,
}
if (!refcount_dec_and_test(&req->ref))
return;
- if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT)
+ if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
atomic_dec(&sess->stats->inflight);
req->in_use = false;
@@ -964,6 +964,7 @@ static void rtrs_clt_init_req(struct rtrs_clt_io_req *req,
req->need_inv_comp = false;
req->inv_errno = 0;
refcount_set(&req->ref, 1);
+ req->mp_policy = sess->clt->mp_policy;
iov_iter_kvec(&iter, READ, vec, 1, usr_len);
len = _copy_from_iter(req->iu->buf, usr_len, &iter);
@@ -1154,7 +1155,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req)
"Write request failed: error=%d path=%s [%s:%u]\n",
ret, kobject_name(&sess->kobj), sess->hca_name,
sess->hca_port);
- if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT)
+ if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
atomic_dec(&sess->stats->inflight);
if (req->sg_cnt)
ib_dma_unmap_sg(sess->s.dev->ib_dev, req->sglist,
@@ -1260,7 +1261,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req)
"Read request failed: error=%d path=%s [%s:%u]\n",
ret, kobject_name(&sess->kobj), sess->hca_name,
sess->hca_port);
- if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT)
+ if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
atomic_dec(&sess->stats->inflight);
req->need_inv = false;
if (req->sg_cnt)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.h b/drivers/infiniband/ulp/rtrs/rtrs-clt.h
index c31f920e6d10..6d81aae53df4 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.h
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.h
@@ -101,6 +101,7 @@ struct rtrs_clt_io_req {
unsigned int usr_len;
void *priv;
bool in_use;
+ enum rtrs_mp_policy mp_policy;
struct rtrs_clt_con *con;
struct rtrs_sg_desc *desc;
struct ib_sge *sge;
--
2.25.1
next prev parent reply other threads:[~2021-07-30 13:18 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-30 13:18 [PATCH for-next 00/10] Misc update for RTRS Jack Wang
2021-07-30 13:18 ` [PATCH for-next 01/10] RDMA/rtrs-clt: During add_path change for_new_clt according to path_num Jack Wang
2021-08-02 6:40 ` Leon Romanovsky
2021-08-02 14:17 ` Haris Iqbal
2021-07-30 13:18 ` [PATCH for-next 02/10] RDMA/rtrs-srv: Prevent sysfs error with path name "ctl" Jack Wang
2021-08-02 6:47 ` Leon Romanovsky
2021-08-02 14:24 ` Haris Iqbal
2021-08-02 16:34 ` Leon Romanovsky
2021-07-30 13:18 ` [PATCH for-next 03/10] RDMA/rtrs: Use sysfs_emit instead of s*printf function for sysfs show Jack Wang
2021-08-02 6:52 ` Leon Romanovsky
2021-08-02 14:18 ` Haris Iqbal
2021-07-30 13:18 ` [PATCH for-next 04/10] RDMA/rtrs: Remove unused functions Jack Wang
2021-08-02 6:53 ` Leon Romanovsky
2021-07-30 13:18 ` [PATCH for-next 05/10] RDMA/rtrs: Fix warning when use poll mode Jack Wang
2021-08-02 7:06 ` Leon Romanovsky
2021-08-02 14:18 ` Haris Iqbal
2021-07-30 13:18 ` [PATCH for-next 06/10] RDMA/rtrs: Remove len parameter from helper print functions of sysfs Jack Wang
2021-08-02 7:08 ` Leon Romanovsky
2021-08-02 14:34 ` Haris Iqbal
2021-07-30 13:18 ` [PATCH for-next 07/10] RDMA/rtrs: Remove all likely and unlikely Jack Wang
2021-08-02 7:10 ` Leon Romanovsky
2021-07-30 13:18 ` Jack Wang [this message]
2021-08-02 7:22 ` [PATCH for-next 08/10] RDMA/rtrs-clt: Fix counting inflight IO Leon Romanovsky
2021-07-30 13:18 ` [PATCH for-next 09/10] RDMA/rtrs: Add support to disable an IB port on the storage side Jack Wang
2021-08-02 7:29 ` Leon Romanovsky
2021-08-02 14:31 ` Haris Iqbal
2021-08-02 16:35 ` Leon Romanovsky
2021-08-02 17:43 ` Haris Iqbal
2021-08-06 1:22 ` Jason Gunthorpe
2021-08-06 10:14 ` Haris Iqbal
2021-07-30 13:18 ` [PATCH for-next 10/10] RDMA/rtrs: remove (void) casting for functions Jack Wang
2021-08-02 7:32 ` Leon Romanovsky
2021-08-02 14:16 ` Haris Iqbal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210730131832.118865-9-jinpu.wang@ionos.com \
--to=jinpu.wang@ionos.com \
--cc=bvanassche@acm.org \
--cc=dledford@redhat.com \
--cc=gi-oh.kim@ionos.com \
--cc=haris.iqbal@ionos.com \
--cc=jgg@ziepe.ca \
--cc=leon@kernel.org \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).