From: Md Haris Iqbal <haris.iqbal@ionos.com>
To: linux-rdma@vger.kernel.org
Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com,
jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com,
Gioh Kim <gi-oh.kim@ionos.com>,
Leon Romanovsky <leonro@nvidia.com>
Subject: [PATCH v2 for-next 5/6] RDMA/rtrs-clt: Fix counting inflight IO
Date: Fri, 6 Aug 2021 13:21:11 +0200 [thread overview]
Message-ID: <20210806112112.124313-6-haris.iqbal@ionos.com> (raw)
In-Reply-To: <20210806112112.124313-1-haris.iqbal@ionos.com>
From: Gioh Kim <gi-oh.kim@ionos.com>
There are mis-match at counting inflight IO after changing the
multipath policy.
For example, we started fio test with round-robin policy and then
we changed the policy to min-inflight. IOs created under the RR policy
is finished under the min-inflight policy and inflight counter
only decreased. So the counter would be negative value.
And also we started fio test with min-inflight policy and
changed the policy to the round-robin. IOs created under the
min-inflight policy increased the inflight IO counter but the
inflight IO counter was not decreased because the policy was
the round-robin when IO was finished.
So it should count IOs only if the IO is created under the
min-inflight policy. It should not care the policy when the IO
is finished.
This patch adds a field mp_policy in struct rtrs_clt_io_req and
stores the multipath policy when an object of rtrs_clt_io_req is
created. Then rtrs-clt checks the mp_policy of only struct
rtrs_clt_io_req instead of the struct rtrs_clt.
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 2 +-
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 7 ++++---
drivers/infiniband/ulp/rtrs/rtrs-clt.h | 1 +
3 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
index b660c96a3039..5e780bdd763d 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c
@@ -180,7 +180,7 @@ void rtrs_clt_update_all_stats(struct rtrs_clt_io_req *req, int dir)
len = req->usr_len + req->data_len;
rtrs_clt_update_rdma_stats(stats, len, dir);
- if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT)
+ if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
atomic_inc(&stats->inflight);
}
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
index d3e1173e2acd..ef2904226ea2 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
@@ -438,7 +438,7 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno,
}
if (!refcount_dec_and_test(&req->ref))
return;
- if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT)
+ if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
atomic_dec(&sess->stats->inflight);
req->in_use = false;
@@ -964,6 +964,7 @@ static void rtrs_clt_init_req(struct rtrs_clt_io_req *req,
req->need_inv_comp = false;
req->inv_errno = 0;
refcount_set(&req->ref, 1);
+ req->mp_policy = sess->clt->mp_policy;
iov_iter_kvec(&iter, READ, vec, 1, usr_len);
len = _copy_from_iter(req->iu->buf, usr_len, &iter);
@@ -1154,7 +1155,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req)
"Write request failed: error=%d path=%s [%s:%u]\n",
ret, kobject_name(&sess->kobj), sess->hca_name,
sess->hca_port);
- if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT)
+ if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
atomic_dec(&sess->stats->inflight);
if (req->sg_cnt)
ib_dma_unmap_sg(sess->s.dev->ib_dev, req->sglist,
@@ -1260,7 +1261,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req)
"Read request failed: error=%d path=%s [%s:%u]\n",
ret, kobject_name(&sess->kobj), sess->hca_name,
sess->hca_port);
- if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT)
+ if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
atomic_dec(&sess->stats->inflight);
req->need_inv = false;
if (req->sg_cnt)
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.h b/drivers/infiniband/ulp/rtrs/rtrs-clt.h
index 72f9136e3c24..9dc819885ec7 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.h
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.h
@@ -101,6 +101,7 @@ struct rtrs_clt_io_req {
unsigned int usr_len;
void *priv;
bool in_use;
+ enum rtrs_mp_policy mp_policy;
struct rtrs_clt_con *con;
struct rtrs_sg_desc *desc;
struct ib_sge *sge;
--
2.25.1
next prev parent reply other threads:[~2021-08-06 11:21 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-06 11:21 [PATCH v2 for-next 0/6] Misc update for RTRS Md Haris Iqbal
2021-08-06 11:21 ` [PATCH v2 for-next 1/6] RDMA/rtrs-clt: During add_path change for_new_clt according to path_num Md Haris Iqbal
2021-08-08 8:34 ` Leon Romanovsky
2021-08-06 11:21 ` [PATCH v2 for-next 2/6] RDMA/rtrs: Remove unused functions Md Haris Iqbal
2021-08-06 11:21 ` [PATCH v2 for-next 3/6] RDMA/rtrs: Fix warning when use poll mode Md Haris Iqbal
2021-08-08 9:03 ` Leon Romanovsky
[not found] ` <CAD+HZHXM_MvTrNtAm=egQwKFhyHAi5WHDcXhTC0wvSegHbd4sg@mail.gmail.com>
2021-08-08 11:30 ` Leon Romanovsky
2021-08-10 10:54 ` Haris Iqbal
2021-08-06 11:21 ` [PATCH v2 for-next 4/6] RDMA/rtrs: Remove all likely and unlikely Md Haris Iqbal
2021-08-06 11:21 ` Md Haris Iqbal [this message]
2021-08-06 11:21 ` [PATCH v2 for-next 6/6] RDMA/rtrs: remove (void) casting for functions Md Haris Iqbal
2021-08-20 18:00 ` [PATCH v2 for-next 0/6] Misc update for RTRS Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210806112112.124313-6-haris.iqbal@ionos.com \
--to=haris.iqbal@ionos.com \
--cc=bvanassche@acm.org \
--cc=dledford@redhat.com \
--cc=gi-oh.kim@ionos.com \
--cc=jgg@ziepe.ca \
--cc=jinpu.wang@ionos.com \
--cc=leon@kernel.org \
--cc=leonro@nvidia.com \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).