All of lore.kernel.org
 help / color / mirror / Atom feed
From: Max Gurtovoy <maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
To: linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org,
	keith.busch-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	hch-jcswGhMUV9g@public.gmane.org,
	linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org
Cc: vladimirk-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org,
	maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org
Subject: [PATCH v2 4/4] nvme-rdma: add support for arbitrary sg lists mapping
Date: Thu, 13 Apr 2017 18:56:17 +0300	[thread overview]
Message-ID: <1492098977-5231-5-git-send-email-maxg@mellanox.com> (raw)
In-Reply-To: <1492098977-5231-1-git-send-email-maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

If the device support arbitrary sg list mapping (device
cap IB_DEVICE_SG_GAPS_REG set) we allocate the memory regions with
IB_MR_TYPE_SG_GAPS and allow the block layer to send us gappy bio
vectors. We use this functionality in iSER driver as well.

Signed-off-by: Max Gurtovoy <maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/nvme/host/rdma.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 47a479f..1c6f974 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -294,8 +294,8 @@ static int nvme_rdma_reinit_request(void *data, struct request *rq)
 
 	ib_dereg_mr(req->mr);
 
-	req->mr = ib_alloc_mr(dev->pd, IB_MR_TYPE_MEM_REG,
-			ctrl->max_fr_pages);
+	req->mr = ib_alloc_mr(dev->pd, ib_get_sg_mr_type(dev->dev),
+			      ctrl->max_fr_pages);
 	if (IS_ERR(req->mr)) {
 		ret = PTR_ERR(req->mr);
 		req->mr = NULL;
@@ -348,8 +348,8 @@ static int __nvme_rdma_init_request(struct nvme_rdma_ctrl *ctrl,
 	if (ret)
 		return ret;
 
-	req->mr = ib_alloc_mr(dev->pd, IB_MR_TYPE_MEM_REG,
-			ctrl->max_fr_pages);
+	req->mr = ib_alloc_mr(dev->pd, ib_get_sg_mr_type(ibdev),
+			      ctrl->max_fr_pages);
 	if (IS_ERR(req->mr)) {
 		ret = PTR_ERR(req->mr);
 		goto out_free_qe;
@@ -1608,6 +1608,9 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl)
 	ctrl->ctrl.sqsize =
 		min_t(int, NVME_CAP_MQES(ctrl->cap) + 1, ctrl->ctrl.sqsize);
 
+	if (ctrl->device->dev->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG)
+		ctrl->ctrl.sg_gaps_support = true;
+
 	error = nvme_enable_ctrl(&ctrl->ctrl, ctrl->cap);
 	if (error)
 		goto out_cleanup_queue;
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WARNING: multiple messages have this Message-ID (diff)
From: maxg@mellanox.com (Max Gurtovoy)
Subject: [PATCH v2 4/4] nvme-rdma: add support for arbitrary sg lists mapping
Date: Thu, 13 Apr 2017 18:56:17 +0300	[thread overview]
Message-ID: <1492098977-5231-5-git-send-email-maxg@mellanox.com> (raw)
In-Reply-To: <1492098977-5231-1-git-send-email-maxg@mellanox.com>

If the device support arbitrary sg list mapping (device
cap IB_DEVICE_SG_GAPS_REG set) we allocate the memory regions with
IB_MR_TYPE_SG_GAPS and allow the block layer to send us gappy bio
vectors. We use this functionality in iSER driver as well.

Signed-off-by: Max Gurtovoy <maxg at mellanox.com>
Reviewed-by: Leon Romanovsky <leon at kernel.org>
---
 drivers/nvme/host/rdma.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 47a479f..1c6f974 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -294,8 +294,8 @@ static int nvme_rdma_reinit_request(void *data, struct request *rq)
 
 	ib_dereg_mr(req->mr);
 
-	req->mr = ib_alloc_mr(dev->pd, IB_MR_TYPE_MEM_REG,
-			ctrl->max_fr_pages);
+	req->mr = ib_alloc_mr(dev->pd, ib_get_sg_mr_type(dev->dev),
+			      ctrl->max_fr_pages);
 	if (IS_ERR(req->mr)) {
 		ret = PTR_ERR(req->mr);
 		req->mr = NULL;
@@ -348,8 +348,8 @@ static int __nvme_rdma_init_request(struct nvme_rdma_ctrl *ctrl,
 	if (ret)
 		return ret;
 
-	req->mr = ib_alloc_mr(dev->pd, IB_MR_TYPE_MEM_REG,
-			ctrl->max_fr_pages);
+	req->mr = ib_alloc_mr(dev->pd, ib_get_sg_mr_type(ibdev),
+			      ctrl->max_fr_pages);
 	if (IS_ERR(req->mr)) {
 		ret = PTR_ERR(req->mr);
 		goto out_free_qe;
@@ -1608,6 +1608,9 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl)
 	ctrl->ctrl.sqsize =
 		min_t(int, NVME_CAP_MQES(ctrl->cap) + 1, ctrl->ctrl.sqsize);
 
+	if (ctrl->device->dev->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG)
+		ctrl->ctrl.sg_gaps_support = true;
+
 	error = nvme_enable_ctrl(&ctrl->ctrl, ctrl->cap);
 	if (error)
 		goto out_cleanup_queue;
-- 
1.7.1

  parent reply	other threads:[~2017-04-13 15:56 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-04-13 15:56 [PATCH v2 0/4] arbitrary sg lists support Max Gurtovoy
2017-04-13 15:56 ` Max Gurtovoy
     [not found] ` <1492098977-5231-1-git-send-email-maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2017-04-13 15:56   ` [PATCH v2 1/4] IB/core: Add inline function to get sg mr type Max Gurtovoy
2017-04-13 15:56     ` Max Gurtovoy
     [not found]     ` <1492098977-5231-2-git-send-email-maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2017-04-20 11:24       ` Sagi Grimberg
2017-04-20 11:24         ` Sagi Grimberg
2017-04-13 15:56   ` [PATCH v2 2/4] IB/iser: Use ib_get_sg_mr_type() helper Max Gurtovoy
2017-04-13 15:56     ` Max Gurtovoy
2017-04-13 15:56   ` [PATCH v2 3/4] nvme: enable SG gaps support Max Gurtovoy
2017-04-13 15:56     ` Max Gurtovoy
     [not found]     ` <1492098977-5231-4-git-send-email-maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2017-04-20 11:27       ` Sagi Grimberg
2017-04-20 11:27         ` Sagi Grimberg
     [not found]         ` <2bd673ee-8455-32b7-1bad-34fa9eebb076-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-04-21  6:44           ` Christoph Hellwig
2017-04-21  6:44             ` Christoph Hellwig
     [not found]             ` <20170421064427.GB6095-jcswGhMUV9g@public.gmane.org>
2017-04-23  7:20               ` Sagi Grimberg
2017-04-23  7:20                 ` Sagi Grimberg
     [not found]                 ` <9ff1a2e8-f84f-c772-e294-2339eee8e37d-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-04-23  8:45                   ` Christoph Hellwig
2017-04-23  8:45                     ` Christoph Hellwig
     [not found]                     ` <20170423084553.GA24434-jcswGhMUV9g@public.gmane.org>
2017-04-23  9:45                       ` Sagi Grimberg
2017-04-23  9:45                         ` Sagi Grimberg
     [not found]                         ` <9fe3f5fa-4496-3819-0288-08a073dd1a1a-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-04-23 13:23                           ` Max Gurtovoy
2017-04-23 13:23                             ` Max Gurtovoy
2017-04-24  7:02                           ` Christoph Hellwig
2017-04-24  7:02                             ` Christoph Hellwig
2017-04-13 15:56   ` Max Gurtovoy [this message]
2017-04-13 15:56     ` [PATCH v2 4/4] nvme-rdma: add support for arbitrary sg lists mapping Max Gurtovoy
2017-04-13 16:31   ` [PATCH v2 0/4] arbitrary sg lists support Bart Van Assche
2017-04-13 16:31     ` Bart Van Assche
     [not found]     ` <1492101079.5176.9.camel-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2017-04-13 16:46       ` Max Gurtovoy
2017-04-13 16:46         ` Max Gurtovoy
     [not found]         ` <9a8bc957-dcc0-2e54-d631-9f05f96e0f18-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2017-04-13 16:59           ` Laurence Oberman
2017-04-13 16:59             ` Laurence Oberman
2017-04-20 11:30       ` Sagi Grimberg
2017-04-20 11:30         ` Sagi Grimberg
     [not found]         ` <c52b5ce7-5346-419d-26f2-0d4d45d20471-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-04-23 10:34           ` Leon Romanovsky
2017-04-23 10:34             ` Leon Romanovsky
     [not found]             ` <20170423103439.GC14088-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-04-23 11:49               ` Sagi Grimberg
2017-04-23 11:49                 ` Sagi Grimberg
     [not found]                 ` <d697d360-99ca-c3ed-08e2-b89cc50a15b0-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-04-23 12:12                   ` Leon Romanovsky
2017-04-23 12:12                     ` Leon Romanovsky
     [not found]                     ` <20170423121254.GD14088-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-04-23 12:28                       ` Sagi Grimberg
2017-04-23 12:28                         ` Sagi Grimberg
     [not found]                         ` <a3264968-2d9e-bda5-1f23-74da248cf4d5-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-04-23 13:35                           ` Max Gurtovoy
2017-04-23 13:35                             ` Max Gurtovoy
     [not found]                             ` <9a9668fa-87a7-e730-27e2-8a2b3e7fbc36-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2017-04-23 14:35                               ` Sagi Grimberg
2017-04-23 14:35                                 ` Sagi Grimberg
     [not found]                                 ` <8ed506ba-ae7b-af27-2d2a-9a434c22353c-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-05-01 18:50                                   ` Doug Ledford
2017-05-01 18:50                                     ` Doug Ledford
     [not found]                                     ` <1493664645.3041.184.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2017-05-02 16:14                                       ` Max Gurtovoy
2017-05-02 16:14                                         ` Max Gurtovoy
     [not found]                                         ` <4a1c94b5-fe13-8b34-762f-e7465182ed27-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2017-05-02 16:39                                           ` Laurence Oberman
2017-05-02 16:39                                             ` Laurence Oberman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1492098977-5231-5-git-send-email-maxg@mellanox.com \
    --to=maxg-vpraknaxozvwk0htik3j/w@public.gmane.org \
    --cc=hch-jcswGhMUV9g@public.gmane.org \
    --cc=keith.busch-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
    --cc=leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=vladimirk-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.