From: Gal Pressman <galpress@amazon.com>
To: Doug Ledford <dledford@redhat.com>, Jason Gunthorpe <jgg@ziepe.ca>
Cc: <linux-rdma@vger.kernel.org>,
Daniel Kranzdorf <dkkranzd@amazon.com>,
Yossi Leybovich <sleybo@amazon.com>,
Gal Pressman <galpress@amazon.com>
Subject: [PATCH for-next 4/4] RDMA/efa: Expose RDMA read related attributes
Date: Tue, 10 Sep 2019 14:43:01 +0100 [thread overview]
Message-ID: <20190910134301.4194-5-galpress@amazon.com> (raw)
In-Reply-To: <20190910134301.4194-1-galpress@amazon.com>
From: Daniel Kranzdorf <dkkranzd@amazon.com>
Query the device attributes for RDMA operations, including maximum
transfer size and maximum number of SGEs per RDMA WR, and report them
back to the userspace library.
Signed-off-by: Daniel Kranzdorf <dkkranzd@amazon.com>
Reviewed-by: Yossi Leybovich <sleybo@amazon.com>
Signed-off-by: Gal Pressman <galpress@amazon.com>
---
drivers/infiniband/hw/efa/efa_admin_cmds_defs.h | 17 +++++++++++++++++
drivers/infiniband/hw/efa/efa_com_cmd.c | 2 ++
drivers/infiniband/hw/efa/efa_com_cmd.h | 2 ++
drivers/infiniband/hw/efa/efa_verbs.c | 2 ++
include/uapi/rdma/efa-abi.h | 3 +++
5 files changed, 26 insertions(+)
diff --git a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
index 7fa9d532db61..e96bcb16bd2b 100644
--- a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
+++ b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
@@ -562,6 +562,16 @@ struct efa_admin_feature_device_attr_desc {
/* Indicates how many bits are used virtual address access */
u8 virt_addr_width;
+
+ /*
+ * 0 : rdma_read - If set, RDMA Read is supported on
+ * TX queues
+ * 31:1 : reserved - MBZ
+ */
+ u32 device_caps;
+
+ /* Max RDMA transfer size in bytes */
+ u32 max_rdma_size;
};
struct efa_admin_feature_queue_attr_desc {
@@ -608,6 +618,9 @@ struct efa_admin_feature_queue_attr_desc {
/* The maximum size of LLQ in bytes */
u32 max_llq_size;
+
+ /* Maximum number of SGEs for a single RDMA read WQE */
+ u16 max_wr_rdma_sges;
};
struct efa_admin_feature_aenq_desc {
@@ -622,6 +635,7 @@ struct efa_admin_feature_network_attr_desc {
/* Raw address data in network byte order */
u8 addr[16];
+ /* max packet payload size in bytes */
u32 mtu;
};
@@ -797,4 +811,7 @@ struct efa_admin_mmio_req_read_less_resp {
/* get_set_feature_common_desc */
#define EFA_ADMIN_GET_SET_FEATURE_COMMON_DESC_SELECT_MASK GENMASK(1, 0)
+/* feature_device_attr_desc */
+#define EFA_ADMIN_FEATURE_DEVICE_ATTR_DESC_RDMA_READ_MASK BIT(0)
+
#endif /* _EFA_ADMIN_CMDS_H_ */
diff --git a/drivers/infiniband/hw/efa/efa_com_cmd.c b/drivers/infiniband/hw/efa/efa_com_cmd.c
index 520c9d920f9e..0e6796f11e52 100644
--- a/drivers/infiniband/hw/efa/efa_com_cmd.c
+++ b/drivers/infiniband/hw/efa/efa_com_cmd.c
@@ -444,6 +444,7 @@ int efa_com_get_device_attr(struct efa_com_dev *edev,
result->phys_addr_width = resp.u.device_attr.phys_addr_width;
result->virt_addr_width = resp.u.device_attr.virt_addr_width;
result->db_bar = resp.u.device_attr.db_bar;
+ result->max_rdma_size = resp.u.device_attr.max_rdma_size;
if (result->admin_api_version < 1) {
ibdev_err_ratelimited(
@@ -477,6 +478,7 @@ int efa_com_get_device_attr(struct efa_com_dev *edev,
result->max_ah = resp.u.queue_attr.max_ah;
result->max_llq_size = resp.u.queue_attr.max_llq_size;
result->sub_cqs_per_cq = resp.u.queue_attr.sub_cqs_per_cq;
+ result->max_wr_rdma_sge = resp.u.queue_attr.max_wr_rdma_sges;
err = efa_com_get_feature(edev, &resp, EFA_ADMIN_NETWORK_ATTR);
if (err) {
diff --git a/drivers/infiniband/hw/efa/efa_com_cmd.h b/drivers/infiniband/hw/efa/efa_com_cmd.h
index d119186c41d0..4dd3452d0b41 100644
--- a/drivers/infiniband/hw/efa/efa_com_cmd.h
+++ b/drivers/infiniband/hw/efa/efa_com_cmd.h
@@ -121,9 +121,11 @@ struct efa_com_get_device_attr_result {
u32 max_pd;
u32 max_ah;
u32 max_llq_size;
+ u32 max_rdma_size;
u16 sub_cqs_per_cq;
u16 max_sq_sge;
u16 max_rq_sge;
+ u16 max_wr_rdma_sge;
u8 db_bar;
};
diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
index 0011e2c1c8a6..188fcb41218d 100644
--- a/drivers/infiniband/hw/efa/efa_verbs.c
+++ b/drivers/infiniband/hw/efa/efa_verbs.c
@@ -313,6 +313,8 @@ int efa_query_device(struct ib_device *ibdev,
resp.max_rq_sge = dev_attr->max_rq_sge;
resp.max_sq_wr = dev_attr->max_sq_depth;
resp.max_rq_wr = dev_attr->max_rq_depth;
+ resp.max_rdma_size = dev_attr->max_rdma_size;
+ resp.max_wr_rdma_sge = dev_attr->max_wr_rdma_sge;
err = ib_copy_to_udata(udata, &resp,
min(sizeof(resp), udata->outlen));
diff --git a/include/uapi/rdma/efa-abi.h b/include/uapi/rdma/efa-abi.h
index 9599a2a62be8..269424c2a8a7 100644
--- a/include/uapi/rdma/efa-abi.h
+++ b/include/uapi/rdma/efa-abi.h
@@ -96,6 +96,9 @@ struct efa_ibv_ex_query_device_resp {
__u32 max_rq_wr;
__u16 max_sq_sge;
__u16 max_rq_sge;
+ __u32 max_rdma_size;
+ __u16 max_wr_rdma_sge;
+ __u8 reserved_b0[2];
};
#endif /* EFA_ABI_USER_H */
--
2.23.0
next prev parent reply other threads:[~2019-09-10 13:43 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-10 13:42 [PATCH for-next 0/4] EFA RDMA read support Gal Pressman
2019-09-10 13:42 ` [PATCH for-next 1/4] RDMA/efa: Fix incorrect error print Gal Pressman
2019-09-16 17:37 ` Jason Gunthorpe
2019-09-10 13:42 ` [PATCH for-next 2/4] RDMA/efa: Store network attributes in device attributes Gal Pressman
2019-09-10 13:43 ` [PATCH for-next 3/4] RDMA/efa: Support remote read access in MR registration Gal Pressman
2019-09-10 13:43 ` Gal Pressman [this message]
2019-10-20 7:03 ` [PATCH for-next 0/4] EFA RDMA read support Gal Pressman
2019-10-21 17:31 ` Jason Gunthorpe
2019-10-23 9:55 ` Gal Pressman
2019-10-23 14:35 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190910134301.4194-5-galpress@amazon.com \
--to=galpress@amazon.com \
--cc=dkkranzd@amazon.com \
--cc=dledford@redhat.com \
--cc=jgg@ziepe.ca \
--cc=linux-rdma@vger.kernel.org \
--cc=sleybo@amazon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).