[PATCH 1/1] nvme-rdma: Use unlikely macro in the fast path

[PATCH 1/1] nvme-rdma: Use unlikely macro in the fast path

[Max Gurtovoy]

This patch slightly improves performance (mainly for small block sizes).

Signed-off-by: Max Gurtovoy <maxg at mellanox.com>
---
 drivers/nvme/host/rdma.c |   12 ++++++------
 1 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index da04df1..8540dbb 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -856,7 +856,7 @@ static void nvme_rdma_unmap_data(struct nvme_rdma_queue *queue,
 
 	if (req->mr->need_inval) {
 		res = nvme_rdma_inv_rkey(queue, req);
-		if (res < 0) {
+		if (unlikely(res < 0)) {
 			dev_err(ctrl->ctrl.device,
 				"Queueing INV WR for rkey %#x failed (%d)\n",
 				req->mr->rkey, res);
@@ -921,7 +921,7 @@ static int nvme_rdma_map_sg_fr(struct nvme_rdma_queue *queue,
 	int nr;
 
 	nr = ib_map_mr_sg(req->mr, req->sg_table.sgl, count, NULL, PAGE_SIZE);
-	if (nr < count) {
+	if (unlikely(nr < count)) {
 		if (nr < 0)
 			return nr;
 		return -EINVAL;
@@ -1057,7 +1057,7 @@ static int nvme_rdma_post_send(struct nvme_rdma_queue *queue,
 		first = &wr;
 
 	ret = ib_post_send(queue->qp, first, &bad_wr);
-	if (ret) {
+	if (unlikely(ret)) {
 		dev_err(queue->ctrl->ctrl.device,
 			     "%s failed with error code %d\n", __func__, ret);
 	}
@@ -1083,7 +1083,7 @@ static int nvme_rdma_post_recv(struct nvme_rdma_queue *queue,
 	wr.num_sge  = 1;
 
 	ret = ib_post_recv(queue->qp, &wr, &bad_wr);
-	if (ret) {
+	if (unlikely(ret)) {
 		dev_err(queue->ctrl->ctrl.device,
 			"%s failed with error code %d\n", __func__, ret);
 	}
@@ -1443,7 +1443,7 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
 	blk_mq_start_request(rq);
 
 	err = nvme_rdma_map_data(queue, rq, c);
-	if (err < 0) {
+	if (unlikely(err < 0)) {
 		dev_err(queue->ctrl->ctrl.device,
 			     "Failed to map data (%d)\n", err);
 		nvme_cleanup_cmd(rq);
@@ -1457,7 +1457,7 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
 		flush = true;
 	err = nvme_rdma_post_send(queue, sqe, req->sge, req->num_sge,
 			req->mr->need_inval ? &req->reg_wr.wr : NULL, flush);
-	if (err) {
+	if (unlikely(err)) {
 		nvme_rdma_unmap_data(queue, rq);
 		goto err;
 	}
-- 
1.7.1

[PATCH 1/1] nvme-rdma: Use unlikely macro in the fast path

[Sagi Grimberg]

> This patch slightly improves performance (mainly for small block sizes).

This looks fine,

Looks fine,

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>

Since you mentioned it, can you quantify what that bought us?

[PATCH 1/1] nvme-rdma: Use unlikely macro in the fast path

[Max Gurtovoy]

On 8/15/2017 12:08 PM, Sagi Grimberg wrote:
>> This patch slightly improves performance (mainly for small block sizes).
>
> This looks fine,
>
> Looks fine,
>
> Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
>
> Since you mentioned it, can you quantify what that bought us?

1 subsystem, 1 ns (null blk backing device)


bs      (w/w.o)
---    ---------
512:    912k/900k
1k:     912k/900k
2k :    911k/901k
4k:     922k/906k
8k:     568k/569k
16k:    288k/288k

-Max

[PATCH 1/1] nvme-rdma: Use unlikely macro in the fast path

[Sagi Grimberg]

> 1 subsystem, 1 ns (null blk backing device)
> 
> 
> bs      (w/w.o)
> ---    ---------
> 512:    912k/900k
> 1k:     912k/900k
> 2k :    911k/901k
> 4k:     922k/906k

That's somewhat surprising...

Thanks

[PATCH 1/1] nvme-rdma: Use unlikely macro in the fast path

[Christoph Hellwig]

Thanks,

applied to nvme-4.14.