* [PATCH for-next V2 0/5] User-space time-stamping support for mlx5_ib
@ 2015-12-15 18:30 Matan Barak
[not found] ` <1450204213-5630-1-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
0 siblings, 1 reply; 15+ messages in thread
From: Matan Barak @ 2015-12-15 18:30 UTC (permalink / raw)
To: Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud, Haggai Eran, Matan Barak
Hi Eli,
This patch-set adds user-space support for time-stamping in mlx5_ib.
It implements the necessary API:
(a) ib_create_cq_ex - Add support for CQ creation flags
(b) ib_query_device - return timestamp_mask and hca_core_clock.
We also add support for mmaping the HCA's free running clock.
In order to do so, we use the response of the vendor's extended
part in init_ucontext. This allows us to pass the page offset
of the free running clock register to the user-space driver.
In order to implement it in a future extensible manner, we use the
same mechanism of verbs extensions to the mlx5 vendor part as well.
Regards,
Matan
Changes from v1:
* Change ib_is_udata_cleared to use memchr_inv.
Changes from v0:
* Limit mmap PAGE_SIZE to 4K (security wise).
* Optimize ib_is_udata_cleared.
* Pass hca_core_clock_offset in the vendor's response part of init_ucontext.
Matan Barak (5):
IB/mlx5: Add create_cq extended command
IB/core: Add ib_is_udata_cleared
IB/mlx5: Add support for hca_core_clock and timestamp_mask
IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext
IB/mlx5: Mmap the HCA's core clock register to user-space
drivers/infiniband/hw/mlx5/cq.c | 7 ++++
drivers/infiniband/hw/mlx5/main.c | 67 +++++++++++++++++++++++++++++++-----
drivers/infiniband/hw/mlx5/mlx5_ib.h | 7 +++-
drivers/infiniband/hw/mlx5/user.h | 12 +++++--
include/linux/mlx5/device.h | 7 ++--
include/linux/mlx5/mlx5_ifc.h | 9 +++--
include/rdma/ib_verbs.h | 27 +++++++++++++++
7 files changed, 120 insertions(+), 16 deletions(-)
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH for-next V2 1/5] IB/mlx5: Add create_cq extended command
[not found] ` <1450204213-5630-1-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2015-12-15 18:30 ` Matan Barak
2015-12-15 18:30 ` [PATCH for-next V2 2/5] IB/core: Add ib_is_udata_cleared Matan Barak
` (4 subsequent siblings)
5 siblings, 0 replies; 15+ messages in thread
From: Matan Barak @ 2015-12-15 18:30 UTC (permalink / raw)
To: Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud, Haggai Eran, Matan Barak
In order to create a CQ that supports timestamp, mlx5 needs to
support the extended create CQ command with the timestamp flag.
Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Eli Cohen <eli-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
drivers/infiniband/hw/mlx5/cq.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 3dfd287..186debf 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -743,6 +743,10 @@ static void destroy_cq_kernel(struct mlx5_ib_dev *dev, struct mlx5_ib_cq *cq)
mlx5_db_free(dev->mdev, &cq->db);
}
+enum {
+ CQ_CREATE_FLAGS_SUPPORTED = IB_CQ_FLAGS_TIMESTAMP_COMPLETION
+};
+
struct ib_cq *mlx5_ib_create_cq(struct ib_device *ibdev,
const struct ib_cq_init_attr *attr,
struct ib_ucontext *context,
@@ -766,6 +770,9 @@ struct ib_cq *mlx5_ib_create_cq(struct ib_device *ibdev,
if (entries < 0)
return ERR_PTR(-EINVAL);
+ if (attr->flags & ~CQ_CREATE_FLAGS_SUPPORTED)
+ return ERR_PTR(-EOPNOTSUPP);
+
entries = roundup_pow_of_two(entries + 1);
if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
return ERR_PTR(-EINVAL);
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH for-next V2 2/5] IB/core: Add ib_is_udata_cleared
[not found] ` <1450204213-5630-1-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-15 18:30 ` [PATCH for-next V2 1/5] IB/mlx5: Add create_cq extended command Matan Barak
@ 2015-12-15 18:30 ` Matan Barak
[not found] ` <1450204213-5630-3-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-15 18:30 ` [PATCH for-next V2 3/5] IB/mlx5: Add support for hca_core_clock and timestamp_mask Matan Barak
` (3 subsequent siblings)
5 siblings, 1 reply; 15+ messages in thread
From: Matan Barak @ 2015-12-15 18:30 UTC (permalink / raw)
To: Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud, Haggai Eran, Matan Barak
Extending core and vendor verb commands require us to check that the
unknown part of the user's given command is all zeros.
Adding ib_is_udata_cleared in order to do so.
Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
include/rdma/ib_verbs.h | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 9a68a19..33ab4eb 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -50,6 +50,8 @@
#include <linux/workqueue.h>
#include <linux/socket.h>
#include <uapi/linux/if_ether.h>
+#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/atomic.h>
#include <linux/mmu_notifier.h>
@@ -1887,6 +1889,31 @@ static inline int ib_copy_to_udata(struct ib_udata *udata, void *src, size_t len
return copy_to_user(udata->outbuf, src, len) ? -EFAULT : 0;
}
+static inline bool ib_is_udata_cleared(struct ib_udata *udata,
+ size_t offset,
+ size_t len)
+{
+ const void __user *p = udata->inbuf + offset;
+ bool ret = false;
+ u8 *buf;
+
+ if (len > USHRT_MAX)
+ return false;
+
+ buf = kmalloc(len, GFP_KERNEL);
+ if (!buf)
+ return false;
+
+ if (copy_from_user(buf, p, len))
+ goto free;
+
+ ret = !memchr_inv(buf, 0, len);
+
+free:
+ kfree(buf);
+ return ret;
+}
+
/**
* ib_modify_qp_is_ok - Check that the supplied attribute mask
* contains all required attributes and no attributes not allowed for
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH for-next V2 3/5] IB/mlx5: Add support for hca_core_clock and timestamp_mask
[not found] ` <1450204213-5630-1-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-15 18:30 ` [PATCH for-next V2 1/5] IB/mlx5: Add create_cq extended command Matan Barak
2015-12-15 18:30 ` [PATCH for-next V2 2/5] IB/core: Add ib_is_udata_cleared Matan Barak
@ 2015-12-15 18:30 ` Matan Barak
[not found] ` <1450204213-5630-4-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-15 18:30 ` [PATCH for-next V2 4/5] IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext Matan Barak
` (2 subsequent siblings)
5 siblings, 1 reply; 15+ messages in thread
From: Matan Barak @ 2015-12-15 18:30 UTC (permalink / raw)
To: Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud, Haggai Eran, Matan Barak
Reporting the hca_core_clock (in kHZ) and the timestamp_mask in
query_device extended verb. timestamp_mask is used by users in order
to know what is the valid range of the raw timestamps, while
hca_core_clock reports the clock frequency that is used for
timestamps.
Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Moshe Lazer <moshel-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
drivers/infiniband/hw/mlx5/main.c | 2 ++
include/linux/mlx5/mlx5_ifc.h | 9 ++++++---
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 7e97cb5..c707c43 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -293,6 +293,8 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
props->max_mcast_grp;
props->max_map_per_fmr = INT_MAX; /* no limit in ConnectIB */
+ props->hca_core_clock = MLX5_CAP_GEN(mdev, device_frequency_khz);
+ props->timestamp_mask = 0x7FFFFFFFFFFFFFFFULL;
#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
if (MLX5_CAP_GEN(mdev, pg))
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 1565324..1a74114 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -794,15 +794,18 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 reserved_63[0x8];
u8 log_uar_page_sz[0x10];
- u8 reserved_64[0x100];
+ u8 reserved_64[0x20];
+ u8 device_frequency_mhz[0x20];
+ u8 device_frequency_khz[0x20];
+ u8 reserved_65[0xa0];
- u8 reserved_65[0x1f];
+ u8 reserved_66[0x1f];
u8 cqe_zip[0x1];
u8 cqe_zip_timeout[0x10];
u8 cqe_zip_max_num[0x10];
- u8 reserved_66[0x220];
+ u8 reserved_67[0x220];
};
enum {
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH for-next V2 4/5] IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext
[not found] ` <1450204213-5630-1-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
` (2 preceding siblings ...)
2015-12-15 18:30 ` [PATCH for-next V2 3/5] IB/mlx5: Add support for hca_core_clock and timestamp_mask Matan Barak
@ 2015-12-15 18:30 ` Matan Barak
[not found] ` <1450204213-5630-5-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-15 18:30 ` [PATCH for-next V2 5/5] IB/mlx5: Mmap the HCA's core clock register to user-space Matan Barak
2015-12-23 17:16 ` [PATCH for-next V2 0/5] User-space time-stamping support for mlx5_ib Or Gerlitz
5 siblings, 1 reply; 15+ messages in thread
From: Matan Barak @ 2015-12-15 18:30 UTC (permalink / raw)
To: Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud, Haggai Eran, Matan Barak
Pass hca_core_clock_offset to user-space is mandatory in order to
let the user-space read the free-running clock register from the
right offset in the memory mapped page.
Passing this value is done by changing the vendor's command
and response of init_ucontext to be in extensible form.
Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Moshe Lazer <moshel-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
drivers/infiniband/hw/mlx5/main.c | 37 ++++++++++++++++++++++++++++--------
drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +++
drivers/infiniband/hw/mlx5/user.h | 12 ++++++++++--
include/linux/mlx5/device.h | 7 +++++--
4 files changed, 47 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index c707c43..917363c 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -582,8 +582,8 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(struct ib_device *ibdev,
struct ib_udata *udata)
{
struct mlx5_ib_dev *dev = to_mdev(ibdev);
- struct mlx5_ib_alloc_ucontext_req_v2 req;
- struct mlx5_ib_alloc_ucontext_resp resp;
+ struct mlx5_ib_alloc_ucontext_req_v2 req = {};
+ struct mlx5_ib_alloc_ucontext_resp resp = {};
struct mlx5_ib_ucontext *context;
struct mlx5_uuar_info *uuari;
struct mlx5_uar *uars;
@@ -598,20 +598,19 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(struct ib_device *ibdev,
if (!dev->ib_active)
return ERR_PTR(-EAGAIN);
- memset(&req, 0, sizeof(req));
reqlen = udata->inlen - sizeof(struct ib_uverbs_cmd_hdr);
if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req))
ver = 0;
- else if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req_v2))
+ else if (reqlen >= sizeof(struct mlx5_ib_alloc_ucontext_req_v2))
ver = 2;
else
return ERR_PTR(-EINVAL);
- err = ib_copy_from_udata(&req, udata, reqlen);
+ err = ib_copy_from_udata(&req, udata, min(reqlen, sizeof(req)));
if (err)
return ERR_PTR(err);
- if (req.flags || req.reserved)
+ if (req.flags)
return ERR_PTR(-EINVAL);
if (req.total_num_uuars > MLX5_MAX_UUARS)
@@ -620,6 +619,14 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(struct ib_device *ibdev,
if (req.total_num_uuars == 0)
return ERR_PTR(-EINVAL);
+ if (req.comp_mask)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ if (reqlen > sizeof(req) &&
+ !ib_is_udata_cleared(udata, sizeof(req),
+ udata->inlen - sizeof(req)))
+ return ERR_PTR(-EOPNOTSUPP);
+
req.total_num_uuars = ALIGN(req.total_num_uuars,
MLX5_NON_FP_BF_REGS_PER_PAGE);
if (req.num_low_latency_uuars > req.total_num_uuars - 1)
@@ -635,6 +642,8 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(struct ib_device *ibdev,
resp.max_send_wqebb = 1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz);
resp.max_recv_wr = 1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz);
resp.max_srq_recv_wr = 1 << MLX5_CAP_GEN(dev->mdev, log_max_srq_sz);
+ resp.response_length = min(offsetof(typeof(resp), response_length) +
+ sizeof(resp.response_length), udata->outlen);
context = kzalloc(sizeof(*context), GFP_KERNEL);
if (!context)
@@ -685,8 +694,20 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(struct ib_device *ibdev,
resp.tot_uuars = req.total_num_uuars;
resp.num_ports = MLX5_CAP_GEN(dev->mdev, num_ports);
- err = ib_copy_to_udata(udata, &resp,
- sizeof(resp) - sizeof(resp.reserved));
+
+ if (field_avail(typeof(resp), reserved2, udata->outlen))
+ resp.response_length += sizeof(resp.reserved2);
+
+ if (field_avail(typeof(resp), hca_core_clock_offset, udata->outlen)) {
+ resp.comp_mask |=
+ MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_CORE_CLOCK_OFFSET;
+ resp.hca_core_clock_offset =
+ offsetof(struct mlx5_init_seg, internal_timer_h) %
+ PAGE_SIZE;
+ resp.response_length += sizeof(resp.hca_core_clock_offset);
+ }
+
+ err = ib_copy_to_udata(udata, &resp, resp.response_length);
if (err)
goto out_uars;
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 6333472..43b3c58 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -55,6 +55,9 @@ pr_err("%s:%s:%d:(pid %d): " format, (dev)->ib_dev.name, __func__, \
pr_warn("%s:%s:%d:(pid %d): " format, (dev)->ib_dev.name, __func__, \
__LINE__, current->pid, ##arg)
+#define field_avail(type, fld, sz) (offsetof(type, fld) + \
+ sizeof(((type *)0)->fld) <= (sz))
+
enum {
MLX5_IB_MMAP_CMD_SHIFT = 8,
MLX5_IB_MMAP_CMD_MASK = 0xff,
diff --git a/drivers/infiniband/hw/mlx5/user.h b/drivers/infiniband/hw/mlx5/user.h
index 76fb7b9..e22a35f 100644
--- a/drivers/infiniband/hw/mlx5/user.h
+++ b/drivers/infiniband/hw/mlx5/user.h
@@ -66,7 +66,11 @@ struct mlx5_ib_alloc_ucontext_req_v2 {
__u32 total_num_uuars;
__u32 num_low_latency_uuars;
__u32 flags;
- __u32 reserved;
+ __u32 comp_mask;
+};
+
+enum mlx5_ib_alloc_ucontext_resp_mask {
+ MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_CORE_CLOCK_OFFSET = 1UL << 0,
};
struct mlx5_ib_alloc_ucontext_resp {
@@ -80,7 +84,11 @@ struct mlx5_ib_alloc_ucontext_resp {
__u32 max_recv_wr;
__u32 max_srq_recv_wr;
__u16 num_ports;
- __u16 reserved;
+ __u16 reserved1;
+ __u32 comp_mask;
+ __u32 response_length;
+ __u32 reserved2;
+ __u64 hca_core_clock_offset;
};
struct mlx5_ib_alloc_pd_resp {
diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index 0b473cb..cc74a8a 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -442,9 +442,12 @@ struct mlx5_init_seg {
__be32 rsvd1[120];
__be32 initializing;
struct health_buffer health;
- __be32 rsvd2[884];
+ __be32 rsvd2[880];
+ __be32 internal_timer_h;
+ __be32 internal_timer_l;
+ __be32 rsvd3[2];
__be32 health_counter;
- __be32 rsvd3[1019];
+ __be32 rsvd4[1019];
__be64 ieee1588_clk;
__be32 ieee1588_clk_type;
__be32 clr_intx;
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH for-next V2 5/5] IB/mlx5: Mmap the HCA's core clock register to user-space
[not found] ` <1450204213-5630-1-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
` (3 preceding siblings ...)
2015-12-15 18:30 ` [PATCH for-next V2 4/5] IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext Matan Barak
@ 2015-12-15 18:30 ` Matan Barak
[not found] ` <1450204213-5630-6-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-23 17:16 ` [PATCH for-next V2 0/5] User-space time-stamping support for mlx5_ib Or Gerlitz
5 siblings, 1 reply; 15+ messages in thread
From: Matan Barak @ 2015-12-15 18:30 UTC (permalink / raw)
To: Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud, Haggai Eran, Matan Barak
In order to read the HCA's current cycles register, we need
to map it to user-space. Add support to map this register
via mmap command.
Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Moshe Lazer <moshel-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
drivers/infiniband/hw/mlx5/main.c | 28 ++++++++++++++++++++++++++++
drivers/infiniband/hw/mlx5/mlx5_ib.h | 4 +++-
2 files changed, 31 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 917363c..d037a72 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -810,6 +810,34 @@ static int mlx5_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vm
case MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES:
return -ENOSYS;
+ case MLX5_IB_MMAP_CORE_CLOCK:
+ {
+ phys_addr_t pfn;
+
+ if (vma->vm_end - vma->vm_start != PAGE_SIZE)
+ return -EINVAL;
+
+ if (vma->vm_flags & (VM_WRITE | VM_EXEC))
+ return -EPERM;
+
+ /* Don't expose to user-space information it shouldn't have */
+ if (PAGE_SIZE > 4096)
+ return -EOPNOTSUPP;
+
+ vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+ pfn = (dev->mdev->iseg_base +
+ offsetof(struct mlx5_init_seg, internal_timer_h)) >>
+ PAGE_SHIFT;
+ if (io_remap_pfn_range(vma, vma->vm_start, pfn,
+ PAGE_SIZE, vma->vm_page_prot))
+ return -EAGAIN;
+
+ mlx5_ib_dbg(dev, "mapped internal timer at 0x%lx, PA 0x%llx\n",
+ vma->vm_start,
+ (unsigned long long)pfn << PAGE_SHIFT);
+ break;
+ }
+
default:
return -EINVAL;
}
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 43b3c58..a87f312 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -65,7 +65,9 @@ enum {
enum mlx5_ib_mmap_cmd {
MLX5_IB_MMAP_REGULAR_PAGE = 0,
- MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1, /* always last */
+ MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1,
+ /* 5 is chosen in order to be compatible with old versions of libmlx5 */
+ MLX5_IB_MMAP_CORE_CLOCK = 5,
};
enum {
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 2/5] IB/core: Add ib_is_udata_cleared
[not found] ` <1450204213-5630-3-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2015-12-16 14:35 ` Haggai Eran
0 siblings, 0 replies; 15+ messages in thread
From: Haggai Eran @ 2015-12-16 14:35 UTC (permalink / raw)
To: Matan Barak, Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud
On 15/12/2015 20:30, Matan Barak wrote:
> Extending core and vendor verb commands require us to check that the
> unknown part of the user's given command is all zeros.
> Adding ib_is_udata_cleared in order to do so.
>
> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Haggai Eran <haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 3/5] IB/mlx5: Add support for hca_core_clock and timestamp_mask
[not found] ` <1450204213-5630-4-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2015-12-16 14:43 ` Sagi Grimberg
[not found] ` <567178A6.7040700-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
0 siblings, 1 reply; 15+ messages in thread
From: Sagi Grimberg @ 2015-12-16 14:43 UTC (permalink / raw)
To: Matan Barak, Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud, Haggai Eran
> Reporting the hca_core_clock (in kHZ) and the timestamp_mask in
> query_device extended verb. timestamp_mask is used by users in order
> to know what is the valid range of the raw timestamps, while
> hca_core_clock reports the clock frequency that is used for
> timestamps.
Hi Matan,
Shouldn't this patch come last?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 5/5] IB/mlx5: Mmap the HCA's core clock register to user-space
[not found] ` <1450204213-5630-6-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2015-12-16 14:47 ` Sagi Grimberg
0 siblings, 0 replies; 15+ messages in thread
From: Sagi Grimberg @ 2015-12-16 14:47 UTC (permalink / raw)
To: Matan Barak, Eli Cohen
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford, Eran Ben Elisha,
Yann Droneaud, Haggai Eran
> enum mlx5_ib_mmap_cmd {
> MLX5_IB_MMAP_REGULAR_PAGE = 0,
> - MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1, /* always last */
> + MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1,
> + /* 5 is chosen in order to be compatible with old versions of libmlx5 */
> + MLX5_IB_MMAP_CORE_CLOCK = 5,
> };
Overall the patches look good so I'd suggest not to apply atop of
the contig pages patchset from Yishai which obviously involves some
debate. Although if this bit is the only conflict then perhaps doug can
take care of it...
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 3/5] IB/mlx5: Add support for hca_core_clock and timestamp_mask
[not found] ` <567178A6.7040700-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
@ 2015-12-16 14:50 ` Matan Barak
0 siblings, 0 replies; 15+ messages in thread
From: Matan Barak @ 2015-12-16 14:50 UTC (permalink / raw)
To: Sagi Grimberg
Cc: Matan Barak, Eli Cohen, linux-rdma, Doug Ledford,
Eran Ben Elisha, Yann Droneaud, Haggai Eran
On Wed, Dec 16, 2015 at 4:43 PM, Sagi Grimberg <sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>
>> Reporting the hca_core_clock (in kHZ) and the timestamp_mask in
>> query_device extended verb. timestamp_mask is used by users in order
>> to know what is the valid range of the raw timestamps, while
>> hca_core_clock reports the clock frequency that is used for
>> timestamps.
>
>
> Hi Matan,
>
> Shouldn't this patch come last?
>
Not necessarily. In order to support completion timestamping (that's
what defined in this query_device patch), we only need create_cq_ex in
mlx5_ib.
The down stream patches adds support for reading the HCA core clock
(via query_values).
One could have completion timestamping support without having
ibv_query_values support.
Thanks for taking a look.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 4/5] IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext
[not found] ` <1450204213-5630-5-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2015-12-22 21:37 ` Or Gerlitz
[not found] ` <CAJ3xEMguJeQnrHAM9WCXZf+NyNOwBGCfpka5ywUtfLHT1wL3ig-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 1 reply; 15+ messages in thread
From: Or Gerlitz @ 2015-12-22 21:37 UTC (permalink / raw)
To: Matan Barak; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Moshe Lazer
On Tue, Dec 15, 2015 at 8:30 PM, Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> Pass hca_core_clock_offset to user-space is mandatory in order to
> let the user-space read the free-running clock register from the
> right offset in the memory mapped page.
> Passing this value is done by changing the vendor's command
> and response of init_ucontext to be in extensible form.
Is "old" (unpatched) libmlx5 still operational over "new" (patched)
mlx5 IB driver?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 4/5] IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext
[not found] ` <CAJ3xEMguJeQnrHAM9WCXZf+NyNOwBGCfpka5ywUtfLHT1wL3ig-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2015-12-23 9:03 ` Or Gerlitz
[not found] ` <567A6370.1030008-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
0 siblings, 1 reply; 15+ messages in thread
From: Or Gerlitz @ 2015-12-23 9:03 UTC (permalink / raw)
To: Matan Barak; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Moshe Lazer
On 12/22/2015 11:37 PM, Or Gerlitz wrote:
> On Tue, Dec 15, 2015 at 8:30 PM, Matan Barak<matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
>> >Pass hca_core_clock_offset to user-space is mandatory in order to
>> >let the user-space read the free-running clock register from the
>> >right offset in the memory mapped page.
>> >Passing this value is done by changing the vendor's command
>> >and response of init_ucontext to be in extensible form.
> Is "old" (unpatched) libmlx5 still operational over "new" (patched)
> mlx5 IB driver?
and same question for the other way around as well
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 4/5] IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext
[not found] ` <567A6370.1030008-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2015-12-23 9:58 ` Matan Barak
0 siblings, 0 replies; 15+ messages in thread
From: Matan Barak @ 2015-12-23 9:58 UTC (permalink / raw)
To: Or Gerlitz; +Cc: Matan Barak, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Moshe Lazer
On Wed, Dec 23, 2015 at 11:03 AM, Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> On 12/22/2015 11:37 PM, Or Gerlitz wrote:
>>
>> On Tue, Dec 15, 2015 at 8:30 PM, Matan Barak<matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
>>>
>>> >Pass hca_core_clock_offset to user-space is mandatory in order to
>>> >let the user-space read the free-running clock register from the
>>> >right offset in the memory mapped page.
>>> >Passing this value is done by changing the vendor's command
>>> >and response of init_ucontext to be in extensible form.
>>
>> Is "old" (unpatched) libmlx5 still operational over "new" (patched)
>> mlx5 IB driver?
>
>
> and same question for the other way around as well
>
new kernel, old lib - response length is initialized to
min(offsetof(typeof(resp), response_length) +
sizeof(resp.response_length), udata->outlen);
I this case, we initialize it to udata->outlen, so the user-space gets
exactly the same information as before.
new lib, old kernel - the response is cleared before it is sent to the
kernel. The command size has stayed the same.
So - (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req_v2)) still holds true.
Since the kernel isn't familiar with the new response fields, it
doesn't copy them. Hence the response's comp_mask is still zero and
libmlx5 knows that hca_core_clock isn't supported.
Matan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 0/5] User-space time-stamping support for mlx5_ib
[not found] ` <1450204213-5630-1-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
` (4 preceding siblings ...)
2015-12-15 18:30 ` [PATCH for-next V2 5/5] IB/mlx5: Mmap the HCA's core clock register to user-space Matan Barak
@ 2015-12-23 17:16 ` Or Gerlitz
[not found] ` <CAJ3xEMiJJ0HcxDn_xfPDRK8V1cfN5HNVm9dt6gPaw2iJwD8jhA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
5 siblings, 1 reply; 15+ messages in thread
From: Or Gerlitz @ 2015-12-23 17:16 UTC (permalink / raw)
To: Matan Barak
Cc: Eli Cohen, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Doug Ledford,
Eran Ben Elisha, Yann Droneaud, Haggai Eran
On Tue, Dec 15, 2015 at 8:30 PM, Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> This patch-set adds user-space support for time-stamping in mlx5_ib.
> It implements the necessary API:
> (a) ib_create_cq_ex - Add support for CQ creation flags
> (b) ib_query_device - return timestamp_mask and hca_core_clock.
> We also add support for mmaping the HCA's free running clock.
> In order to do so, we use the response of the vendor's extended
> part in init_ucontext. This allows us to pass the page offset
> of the free running clock register to the user-space driver.
> In order to implement it in a future extensible manner, we use the
> same mechanism of verbs extensions to the mlx5 vendor part as well.
Hi Doug,
As Matan is now the maintainer of the mlx5_ib driver and this series
doesn't add any IB/core changes, we see no reason that it can't go
into 4.5. make sense?
Or
> Changes from v1:
> * Change ib_is_udata_cleared to use memchr_inv.
>
> Changes from v0:
> * Limit mmap PAGE_SIZE to 4K (security wise).
> * Optimize ib_is_udata_cleared.
> * Pass hca_core_clock_offset in the vendor's response part of init_ucontext.
>
> Matan Barak (5):
> IB/mlx5: Add create_cq extended command
> IB/core: Add ib_is_udata_cleared
> IB/mlx5: Add support for hca_core_clock and timestamp_mask
> IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext
> IB/mlx5: Mmap the HCA's core clock register to user-space
>
> drivers/infiniband/hw/mlx5/cq.c | 7 ++++
> drivers/infiniband/hw/mlx5/main.c | 67 +++++++++++++++++++++++++++++++-----
> drivers/infiniband/hw/mlx5/mlx5_ib.h | 7 +++-
> drivers/infiniband/hw/mlx5/user.h | 12 +++++--
> include/linux/mlx5/device.h | 7 ++--
> include/linux/mlx5/mlx5_ifc.h | 9 +++--
> include/rdma/ib_verbs.h | 27 +++++++++++++++
> 7 files changed, 120 insertions(+), 16 deletions(-)
>
> --
> 2.1.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH for-next V2 0/5] User-space time-stamping support for mlx5_ib
[not found] ` <CAJ3xEMiJJ0HcxDn_xfPDRK8V1cfN5HNVm9dt6gPaw2iJwD8jhA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2015-12-24 4:35 ` Doug Ledford
0 siblings, 0 replies; 15+ messages in thread
From: Doug Ledford @ 2015-12-24 4:35 UTC (permalink / raw)
To: Or Gerlitz, Matan Barak
Cc: Eli Cohen, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Eran Ben Elisha,
Yann Droneaud, Haggai Eran
[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]
On 12/23/2015 12:16 PM, Or Gerlitz wrote:
> On Tue, Dec 15, 2015 at 8:30 PM, Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
>
>> This patch-set adds user-space support for time-stamping in mlx5_ib.
>> It implements the necessary API:
>> (a) ib_create_cq_ex - Add support for CQ creation flags
>> (b) ib_query_device - return timestamp_mask and hca_core_clock.
>
>> We also add support for mmaping the HCA's free running clock.
>> In order to do so, we use the response of the vendor's extended
>> part in init_ucontext. This allows us to pass the page offset
>> of the free running clock register to the user-space driver.
>> In order to implement it in a future extensible manner, we use the
>> same mechanism of verbs extensions to the mlx5 vendor part as well.
>
> Hi Doug,
>
> As Matan is now the maintainer of the mlx5_ib driver and this series
> doesn't add any IB/core changes, we see no reason that it can't go
> into 4.5. make sense?
I didn't see anything wrong with the set. It's been picked up for 4.5.
--
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
GPG KeyID: 0E572FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2015-12-24 4:35 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-12-15 18:30 [PATCH for-next V2 0/5] User-space time-stamping support for mlx5_ib Matan Barak
[not found] ` <1450204213-5630-1-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-15 18:30 ` [PATCH for-next V2 1/5] IB/mlx5: Add create_cq extended command Matan Barak
2015-12-15 18:30 ` [PATCH for-next V2 2/5] IB/core: Add ib_is_udata_cleared Matan Barak
[not found] ` <1450204213-5630-3-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-16 14:35 ` Haggai Eran
2015-12-15 18:30 ` [PATCH for-next V2 3/5] IB/mlx5: Add support for hca_core_clock and timestamp_mask Matan Barak
[not found] ` <1450204213-5630-4-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-16 14:43 ` Sagi Grimberg
[not found] ` <567178A6.7040700-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2015-12-16 14:50 ` Matan Barak
2015-12-15 18:30 ` [PATCH for-next V2 4/5] IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext Matan Barak
[not found] ` <1450204213-5630-5-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-22 21:37 ` Or Gerlitz
[not found] ` <CAJ3xEMguJeQnrHAM9WCXZf+NyNOwBGCfpka5ywUtfLHT1wL3ig-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-12-23 9:03 ` Or Gerlitz
[not found] ` <567A6370.1030008-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-23 9:58 ` Matan Barak
2015-12-15 18:30 ` [PATCH for-next V2 5/5] IB/mlx5: Mmap the HCA's core clock register to user-space Matan Barak
[not found] ` <1450204213-5630-6-git-send-email-matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-12-16 14:47 ` Sagi Grimberg
2015-12-23 17:16 ` [PATCH for-next V2 0/5] User-space time-stamping support for mlx5_ib Or Gerlitz
[not found] ` <CAJ3xEMiJJ0HcxDn_xfPDRK8V1cfN5HNVm9dt6gPaw2iJwD8jhA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-12-24 4:35 ` Doug Ledford
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.