From: Santosh Shilimkar <santosh.shilimkar@oracle.com>
To: netdev@vger.kernel.org, davem@davemloft.net
Cc: linux-kernel@vger.kernel.org,
Santosh Shilimkar <santosh.shilimkar@oracle.com>
Subject: [net-next][PATCH v3 11/13] RDS: IB: add Fastreg MR (FRMR) detection support
Date: Tue, 1 Mar 2016 15:20:52 -0800 [thread overview]
Message-ID: <1456874454-10355-12-git-send-email-santosh.shilimkar@oracle.com> (raw)
In-Reply-To: <1456874454-10355-1-git-send-email-santosh.shilimkar@oracle.com>
Discovere Fast Memmory Registration support using IB device
IB_DEVICE_MEM_MGT_EXTENSIONS. Certain HCA might support just FRMR
or FMR or both FMR and FRWR. In case both mr type are supported,
default FMR is used.
Default MR is still kept as FMR against what everyone else
is following. Default will be changed to FRMR once the
RDS performance with FRMR is comparable with FMR. The
work is in progress for the same.
Signed-off-by: Santosh Shilimkar <ssantosh@kernel.org>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
---
net/rds/ib.c | 10 ++++++++++
net/rds/ib.h | 4 ++++
net/rds/ib_mr.h | 1 +
3 files changed, 15 insertions(+)
diff --git a/net/rds/ib.c b/net/rds/ib.c
index bb32cb9..b5342fd 100644
--- a/net/rds/ib.c
+++ b/net/rds/ib.c
@@ -140,6 +140,12 @@ static void rds_ib_add_one(struct ib_device *device)
rds_ibdev->max_wrs = device->attrs.max_qp_wr;
rds_ibdev->max_sge = min(device->attrs.max_sge, RDS_IB_MAX_SGE);
+ rds_ibdev->has_fr = (device->attrs.device_cap_flags &
+ IB_DEVICE_MEM_MGT_EXTENSIONS);
+ rds_ibdev->has_fmr = (device->alloc_fmr && device->dealloc_fmr &&
+ device->map_phys_fmr && device->unmap_fmr);
+ rds_ibdev->use_fastreg = (rds_ibdev->has_fr && !rds_ibdev->has_fmr);
+
rds_ibdev->fmr_max_remaps = device->attrs.max_map_per_fmr?: 32;
rds_ibdev->max_1m_mrs = device->attrs.max_mr ?
min_t(unsigned int, (device->attrs.max_mr / 2),
@@ -178,6 +184,10 @@ static void rds_ib_add_one(struct ib_device *device)
rds_ibdev->fmr_max_remaps, rds_ibdev->max_1m_mrs,
rds_ibdev->max_8k_mrs);
+ pr_info("RDS/IB: %s: %s supported and preferred\n",
+ device->name,
+ rds_ibdev->use_fastreg ? "FRMR" : "FMR");
+
INIT_LIST_HEAD(&rds_ibdev->ipaddr_list);
INIT_LIST_HEAD(&rds_ibdev->conn_list);
diff --git a/net/rds/ib.h b/net/rds/ib.h
index 62fe7d5..c5eddc2 100644
--- a/net/rds/ib.h
+++ b/net/rds/ib.h
@@ -200,6 +200,10 @@ struct rds_ib_device {
struct list_head conn_list;
struct ib_device *dev;
struct ib_pd *pd;
+ bool has_fmr;
+ bool has_fr;
+ bool use_fastreg;
+
unsigned int max_mrs;
struct rds_ib_mr_pool *mr_1m_pool;
struct rds_ib_mr_pool *mr_8k_pool;
diff --git a/net/rds/ib_mr.h b/net/rds/ib_mr.h
index add7725..2f9b9c3 100644
--- a/net/rds/ib_mr.h
+++ b/net/rds/ib_mr.h
@@ -93,6 +93,7 @@ struct rds_ib_mr_pool {
extern struct workqueue_struct *rds_ib_mr_wq;
extern unsigned int rds_ib_mr_1m_pool_size;
extern unsigned int rds_ib_mr_8k_pool_size;
+extern bool prefer_frmr;
struct rds_ib_mr_pool *rds_ib_create_mr_pool(struct rds_ib_device *rds_dev,
int npages);
--
1.9.1
next prev parent reply other threads:[~2016-03-01 23:21 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-01 23:20 RDS: Major clean-up with couple of new features for 4.6 Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 01/13] RDS: Drop stale iWARP RDMA transport Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 02/13] RDS: Add support for SO_TIMESTAMP for incoming messages Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 03/13] MAINTAINERS: update RDS entry Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 04/13] RDS: IB: Remove the RDS_IB_SEND_OP dependency Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 05/13] RDS: IB: Re-organise ibmr code Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 06/13] RDS: IB: create struct rds_ib_fmr Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 07/13] RDS: IB: move FMR code to its own file Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 08/13] RDS: IB: add connection info to ibmr Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 09/13] RDS: IB: handle the RDMA CM time wait event Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 10/13] RDS: IB: add mr reused stats Santosh Shilimkar
2016-03-01 23:20 ` Santosh Shilimkar [this message]
2016-03-01 23:20 ` [net-next][PATCH v3 12/13] RDS: IB: allocate extra space on queues for FRMR support Santosh Shilimkar
2016-03-01 23:20 ` [net-next][PATCH v3 13/13] RDS: IB: Support Fastreg MR (FRMR) memory registration mode Santosh Shilimkar
2016-03-02 19:13 ` RDS: Major clean-up with couple of new features for 4.6 David Miller
2016-03-02 19:15 ` santosh shilimkar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1456874454-10355-12-git-send-email-santosh.shilimkar@oracle.com \
--to=santosh.shilimkar@oracle.com \
--cc=davem@davemloft.net \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).