* [PATCH v2 0/8] SIF related verbs patches
@ 2016-09-16 18:31 Knut Omang
2016-09-16 18:31 ` [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered Knut Omang
` (3 more replies)
0 siblings, 4 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma, linux-kernel, Knut Omang
This patch set contains the changes and bug fixes necessary
to support Oracle's new Infiniband HCA (SIF) from the kernel side.
The exception is patch 3 which is a useful consolidation and simplification
made possible by patch 2 and the replacement of the dma_attr type with
a plain bitmask.
The changes to other rdma drivers are just trivial consequences of
the extended umem_get and create_ah calls and should hopefully be
uncontroversial.
Patches 4-7 are related to corresponding libibverbs patches.
These make minor changes (fixing end alignment) to some uverbs
data structures, but these are carefully made to be backward
binary compatible between user and kernel for existing provider
libraries.
Changes since v1:
- Improved explanation for patch 1
- Fix a remaining provider case with ib_create_ah, detected by kbuild test robot
- Relax length checks in ib_uverbs_reg_mr to avoid breaking binary compatibility
with user space provider libraries built with old libibverbs.
- Removed "ib_verbs: Add a new qp create flag to request features for Ethernet over IB"
Dag Moxnes (1):
ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered
Knut Omang (7):
ib_umem: Add a new, more generic ib_umem_get_attrs
ib_umem: With the new ib_umem_get_attrs, simplify ib_umem_get
ib: Add udata argument to create_ah
ib_uverbs: Add padding to end align ib_uverbs_reg_mr_resp
ib_uverbs: Avoid vendor specific masking of attributes in query_qp
ib_{uverbs/core}: add new ib_create_qp_ex with udata arg
ib_uverbs: Support for kernel implementation of XRC
drivers/infiniband/core/core_priv.h | 3 +-
drivers/infiniband/core/mad.c | 6 ++-
drivers/infiniband/core/smi.h | 6 +--
drivers/infiniband/core/umem.c | 23 +++---
drivers/infiniband/core/uverbs_cmd.c | 81 ++++++++++++---------
drivers/infiniband/core/verbs.c | 15 +++-
drivers/infiniband/hw/cxgb3/iwch_provider.c | 5 +-
drivers/infiniband/hw/cxgb4/mem.c | 2 +-
drivers/infiniband/hw/cxgb4/provider.c | 3 +-
drivers/infiniband/hw/i40iw/i40iw_verbs.c | 5 +-
drivers/infiniband/hw/mlx4/ah.c | 3 +-
drivers/infiniband/hw/mlx4/cq.c | 4 +-
drivers/infiniband/hw/mlx4/doorbell.c | 2 +-
drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 +-
drivers/infiniband/hw/mlx4/mr.c | 5 +-
drivers/infiniband/hw/mlx4/qp.c | 21 ++++-
drivers/infiniband/hw/mlx4/srq.c | 2 +-
drivers/infiniband/hw/mlx5/ah.c | 3 +-
drivers/infiniband/hw/mlx5/cq.c | 12 +--
drivers/infiniband/hw/mlx5/doorbell.c | 2 +-
drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +-
drivers/infiniband/hw/mlx5/mr.c | 2 +-
drivers/infiniband/hw/mlx5/qp.c | 21 ++++-
drivers/infiniband/hw/mlx5/srq.c | 2 +-
drivers/infiniband/hw/mthca/mthca_provider.c | 11 ++-
drivers/infiniband/hw/nes/nes_verbs.c | 5 +-
drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 5 +-
drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 3 +-
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 2 +-
drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 3 +-
drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 3 +-
drivers/infiniband/sw/rdmavt/mr.c | 2 +-
drivers/infiniband/sw/rxe/rxe_mr.c | 2 +-
drivers/infiniband/sw/rxe/rxe_verbs.c | 3 +-
include/rdma/ib_umem.h | 19 ++++-
include/rdma/ib_verbs.h | 15 +++-
include/uapi/rdma/ib_user_verbs.h | 3 +-
37 files changed, 215 insertions(+), 95 deletions(-)
--
git-series 0.8.10
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
@ 2016-09-16 18:31 ` Knut Omang
[not found] ` <66d69383a3376018d99c025cd188150f6673b209.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
` (2 subsequent siblings)
3 siblings, 1 reply; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Knut Omang, Sean Hefty, Hal Rosenstock,
Ira Weiny, Sagi Grimberg, Christoph Hellwig, Bart Van Assche,
Dag Moxnes, Mark Bloch, Dean Luick
From: Dag Moxnes <dag.moxnes@oracle.com>
The process_mad function is an optional IB driver entry point
allows a driver to intercept or modify MAD traffic.
This fix allows MAD traffic to flow down to the device also
when MAD traffic is completely handled by the device and
no process_mad function is provided.
SIF, the new Oracle Infiniband HCA, is the first HCA
where the device itself makes all decision wrt MAD processing.
Up till now devices either supports MAD, and do then
implement the process_mad entry point, or do not
support MAD at all, and then do not implement process_mad.
SIF introduces a 3rd case: Supports MAD
but do not terminate any MAD requests in the driver.
This case is not handled well by the current code.
The problem is that the handle_outgoing_dr_smp function
has an implicit assumption that some packets are handled
by the process_mad function itself.
There is no way to provide return values from the process_mad
function that ensures that packets are always forwarded to the device,
so the only viable solution without breaking the API
seems to be to not implement process_mad.
---
drivers/infiniband/core/mad.c | 6 ++++++
drivers/infiniband/core/smi.h | 6 ++----
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 2d49228..ece33ec 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -813,6 +813,12 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
goto out;
}
+ /* If device does not define the optional process_mad function it means that
+ * everything is handled in hardware:
+ */
+ if (!device->process_mad)
+ goto out;
+
local = kmalloc(sizeof *local, GFP_ATOMIC);
if (!local) {
ret = -ENOMEM;
diff --git a/drivers/infiniband/core/smi.h b/drivers/infiniband/core/smi.h
index 33c91c8..16a9f9a 100644
--- a/drivers/infiniband/core/smi.h
+++ b/drivers/infiniband/core/smi.h
@@ -67,8 +67,7 @@ static inline enum smi_action smi_check_local_smp(struct ib_smp *smp,
{
/* C14-9:3 -- We're at the end of the DR segment of path */
/* C14-9:4 -- Hop Pointer = Hop Count + 1 -> give to SMA/SM */
- return ((device->process_mad &&
- !ib_get_smp_direction(smp) &&
+ return ((!ib_get_smp_direction(smp) &&
(smp->hop_ptr == smp->hop_cnt + 1)) ?
IB_SMI_HANDLE : IB_SMI_DISCARD);
}
@@ -82,8 +81,7 @@ static inline enum smi_action smi_check_local_returning_smp(struct ib_smp *smp,
{
/* C14-13:3 -- We're at the end of the DR segment of path */
/* C14-13:4 -- Hop Pointer == 0 -> give to SM */
- return ((device->process_mad &&
- ib_get_smp_direction(smp) &&
+ return ((ib_get_smp_direction(smp) &&
!smp->hop_ptr) ? IB_SMI_HANDLE : IB_SMI_DISCARD);
}
--
git-series 0.8.10
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 2/8] ib_umem: Add a new, more generic ib_umem_get_attrs
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
@ 2016-09-16 18:31 ` Knut Omang
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
` (2 subsequent siblings)
3 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Knut Omang, Sean Hefty,
Hal Rosenstock, Robin Murphy, Hans-Christian Noren Egtvedt,
Vineet Gupta, Joerg Roedel, Gerald Schaefer, Krzysztof Kozlowski,
Dave Hansen
This call allows a full range of DMA attributes and also
DMA direction to be supplied and is just a refactor of the old ib_umem_get.
Reimplement ib_umem_get using the new generic call,
now a trivial implementation.
---
drivers/infiniband/core/umem.c | 23 +++++++++++++++--------
include/rdma/ib_umem.h | 15 ++++++++++++++-
2 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index c68746c..699a0f7 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -52,7 +52,7 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
if (umem->nmap > 0)
ib_dma_unmap_sg(dev, umem->sg_head.sgl,
umem->nmap,
- DMA_BIDIRECTIONAL);
+ umem->dir);
for_each_sg(umem->sg_head.sgl, sg, umem->npages, i) {
@@ -82,6 +82,17 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
size_t size, int access, int dmasync)
{
+ unsigned long dma_attrs = 0;
+ if (dmasync)
+ dma_attrs |= DMA_ATTR_WRITE_BARRIER;
+ return ib_umem_get_attrs(context, addr, size, access, DMA_BIDIRECTIONAL, dma_attrs);
+}
+EXPORT_SYMBOL(ib_umem_get);
+
+struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
+ size_t size, int access, enum dma_data_direction dir,
+ unsigned long dma_attrs)
+{
struct ib_umem *umem;
struct page **page_list;
struct vm_area_struct **vma_list;
@@ -91,16 +102,11 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
unsigned long npages;
int ret;
int i;
- unsigned long dma_attrs = 0;
struct scatterlist *sg, *sg_list_start;
int need_release = 0;
- if (dmasync)
- dma_attrs |= DMA_ATTR_WRITE_BARRIER;
-
if (!size)
return ERR_PTR(-EINVAL);
-
/*
* If the combination of the addr and size requested for this memory
* region causes an integer overflow, return error.
@@ -121,6 +127,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->address = addr;
umem->page_size = PAGE_SIZE;
umem->pid = get_task_pid(current, PIDTYPE_PID);
+ umem->dir = dir;
/*
* We ask for writable memory if any of the following
* access flags are set. "Local write" and "remote write"
@@ -213,7 +220,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->nmap = ib_dma_map_sg_attrs(context->device,
umem->sg_head.sgl,
umem->npages,
- DMA_BIDIRECTIONAL,
+ dir,
dma_attrs);
if (umem->nmap <= 0) {
@@ -239,7 +246,7 @@ out:
return ret < 0 ? ERR_PTR(ret) : umem;
}
-EXPORT_SYMBOL(ib_umem_get);
+EXPORT_SYMBOL(ib_umem_get_attrs);
static void ib_umem_account(struct work_struct *work)
{
diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
index 2d83cfd..2876679 100644
--- a/include/rdma/ib_umem.h
+++ b/include/rdma/ib_umem.h
@@ -36,6 +36,7 @@
#include <linux/list.h>
#include <linux/scatterlist.h>
#include <linux/workqueue.h>
+#include <linux/dma-direction.h>
struct ib_ucontext;
struct ib_umem_odp;
@@ -47,6 +48,7 @@ struct ib_umem {
int page_size;
int writable;
int hugetlb;
+ enum dma_data_direction dir;
struct work_struct work;
struct pid *pid;
struct mm_struct *mm;
@@ -81,9 +83,12 @@ static inline size_t ib_umem_num_pages(struct ib_umem *umem)
}
#ifdef CONFIG_INFINIBAND_USER_MEM
-
struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
size_t size, int access, int dmasync);
+struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
+ size_t size, int access,
+ enum dma_data_direction dir,
+ unsigned long dma_attrs);
void ib_umem_release(struct ib_umem *umem);
int ib_umem_page_count(struct ib_umem *umem);
int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
@@ -98,6 +103,14 @@ static inline struct ib_umem *ib_umem_get(struct ib_ucontext *context,
int access, int dmasync) {
return ERR_PTR(-EINVAL);
}
+static inline struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context,
+ unsigned long addr,
+ size_t size, int access,
+ enum dma_data_direction dir,
+ unsigned long dma_attrs)
+{
+ return ERR_PTR(-EINVAL);
+}
static inline void ib_umem_release(struct ib_umem *umem) { }
static inline int ib_umem_page_count(struct ib_umem *umem) { return 0; }
static inline int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
--
git-series 0.8.10
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 2/8] ib_umem: Add a new, more generic ib_umem_get_attrs
@ 2016-09-16 18:31 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Knut Omang, Sean Hefty, Hal Rosenstock,
Robin Murphy, Hans-Christian Noren Egtvedt, Vineet Gupta,
Joerg Roedel, Gerald Schaefer, Krzysztof Kozlowski, Dave Hansen
This call allows a full range of DMA attributes and also
DMA direction to be supplied and is just a refactor of the old ib_umem_get.
Reimplement ib_umem_get using the new generic call,
now a trivial implementation.
---
drivers/infiniband/core/umem.c | 23 +++++++++++++++--------
include/rdma/ib_umem.h | 15 ++++++++++++++-
2 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index c68746c..699a0f7 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -52,7 +52,7 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
if (umem->nmap > 0)
ib_dma_unmap_sg(dev, umem->sg_head.sgl,
umem->nmap,
- DMA_BIDIRECTIONAL);
+ umem->dir);
for_each_sg(umem->sg_head.sgl, sg, umem->npages, i) {
@@ -82,6 +82,17 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
size_t size, int access, int dmasync)
{
+ unsigned long dma_attrs = 0;
+ if (dmasync)
+ dma_attrs |= DMA_ATTR_WRITE_BARRIER;
+ return ib_umem_get_attrs(context, addr, size, access, DMA_BIDIRECTIONAL, dma_attrs);
+}
+EXPORT_SYMBOL(ib_umem_get);
+
+struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
+ size_t size, int access, enum dma_data_direction dir,
+ unsigned long dma_attrs)
+{
struct ib_umem *umem;
struct page **page_list;
struct vm_area_struct **vma_list;
@@ -91,16 +102,11 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
unsigned long npages;
int ret;
int i;
- unsigned long dma_attrs = 0;
struct scatterlist *sg, *sg_list_start;
int need_release = 0;
- if (dmasync)
- dma_attrs |= DMA_ATTR_WRITE_BARRIER;
-
if (!size)
return ERR_PTR(-EINVAL);
-
/*
* If the combination of the addr and size requested for this memory
* region causes an integer overflow, return error.
@@ -121,6 +127,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->address = addr;
umem->page_size = PAGE_SIZE;
umem->pid = get_task_pid(current, PIDTYPE_PID);
+ umem->dir = dir;
/*
* We ask for writable memory if any of the following
* access flags are set. "Local write" and "remote write"
@@ -213,7 +220,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->nmap = ib_dma_map_sg_attrs(context->device,
umem->sg_head.sgl,
umem->npages,
- DMA_BIDIRECTIONAL,
+ dir,
dma_attrs);
if (umem->nmap <= 0) {
@@ -239,7 +246,7 @@ out:
return ret < 0 ? ERR_PTR(ret) : umem;
}
-EXPORT_SYMBOL(ib_umem_get);
+EXPORT_SYMBOL(ib_umem_get_attrs);
static void ib_umem_account(struct work_struct *work)
{
diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
index 2d83cfd..2876679 100644
--- a/include/rdma/ib_umem.h
+++ b/include/rdma/ib_umem.h
@@ -36,6 +36,7 @@
#include <linux/list.h>
#include <linux/scatterlist.h>
#include <linux/workqueue.h>
+#include <linux/dma-direction.h>
struct ib_ucontext;
struct ib_umem_odp;
@@ -47,6 +48,7 @@ struct ib_umem {
int page_size;
int writable;
int hugetlb;
+ enum dma_data_direction dir;
struct work_struct work;
struct pid *pid;
struct mm_struct *mm;
@@ -81,9 +83,12 @@ static inline size_t ib_umem_num_pages(struct ib_umem *umem)
}
#ifdef CONFIG_INFINIBAND_USER_MEM
-
struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
size_t size, int access, int dmasync);
+struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
+ size_t size, int access,
+ enum dma_data_direction dir,
+ unsigned long dma_attrs);
void ib_umem_release(struct ib_umem *umem);
int ib_umem_page_count(struct ib_umem *umem);
int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
@@ -98,6 +103,14 @@ static inline struct ib_umem *ib_umem_get(struct ib_ucontext *context,
int access, int dmasync) {
return ERR_PTR(-EINVAL);
}
+static inline struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context,
+ unsigned long addr,
+ size_t size, int access,
+ enum dma_data_direction dir,
+ unsigned long dma_attrs)
+{
+ return ERR_PTR(-EINVAL);
+}
static inline void ib_umem_release(struct ib_umem *umem) { }
static inline int ib_umem_page_count(struct ib_umem *umem) { return 0; }
static inline int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
--
git-series 0.8.10
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 3/8] ib_umem: With the new ib_umem_get_attrs, simplify ib_umem_get
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
@ 2016-09-16 18:31 ` Knut Omang
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
` (2 subsequent siblings)
3 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Knut Omang, Sean Hefty,
Hal Rosenstock, Steve Wise, Faisal Latif, Chien Tin Tung,
Mustafa Ismail, Shiraz Saleem, Tatyana Nikolova, Yishai Hadas,
Matan Barak, Leon Romanovsky, Selvin Xavier, Devesh Sharma,
Mitesh Ahuja, Dennis Dalessandro, Joerg Roedel, Hans-C
Instead of having most drivers provide a specialized
and weakly typed "boolean" argument
dmasync = 0 argument which are only set to nonzero 3 places,
instead let these special use cases use the
more generic ib_umem_get_attrs which gives access to
the full range of dma attributes and also dma direction,
and eliminate the dmasync parameter from the ib_umem_get call,
Signed-off-by: Knut Omang <knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
drivers/infiniband/core/umem.c | 8 ++------
drivers/infiniband/hw/cxgb3/iwch_provider.c | 2 +-
drivers/infiniband/hw/cxgb4/mem.c | 2 +-
drivers/infiniband/hw/i40iw/i40iw_verbs.c | 2 +-
drivers/infiniband/hw/mlx4/cq.c | 4 ++--
drivers/infiniband/hw/mlx4/doorbell.c | 2 +-
drivers/infiniband/hw/mlx4/mr.c | 5 ++---
drivers/infiniband/hw/mlx4/qp.c | 2 +-
drivers/infiniband/hw/mlx4/srq.c | 2 +-
drivers/infiniband/hw/mlx5/cq.c | 12 +++++++-----
drivers/infiniband/hw/mlx5/doorbell.c | 2 +-
drivers/infiniband/hw/mlx5/mr.c | 2 +-
drivers/infiniband/hw/mlx5/qp.c | 4 ++--
drivers/infiniband/hw/mlx5/srq.c | 2 +-
drivers/infiniband/hw/mthca/mthca_provider.c | 8 ++++++--
drivers/infiniband/hw/nes/nes_verbs.c | 2 +-
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 2 +-
drivers/infiniband/sw/rdmavt/mr.c | 2 +-
drivers/infiniband/sw/rxe/rxe_mr.c | 2 +-
include/rdma/ib_umem.h | 4 ++--
20 files changed, 36 insertions(+), 35 deletions(-)
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 699a0f7..938c940 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -77,15 +77,11 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
* @addr: userspace virtual address to start at
* @size: length of region to pin
* @access: IB_ACCESS_xxx flags for memory being pinned
- * @dmasync: flush in-flight DMA when the memory region is written
*/
struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
- size_t size, int access, int dmasync)
+ size_t size, int access)
{
- unsigned long dma_attrs = 0;
- if (dmasync)
- dma_attrs |= DMA_ATTR_WRITE_BARRIER;
- return ib_umem_get_attrs(context, addr, size, access, DMA_BIDIRECTIONAL, dma_attrs);
+ return ib_umem_get_attrs(context, addr, size, access, DMA_BIDIRECTIONAL, 0);
}
EXPORT_SYMBOL(ib_umem_get);
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 3edb806..4225479 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -576,7 +576,7 @@ static struct ib_mr *iwch_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
mhp->rhp = rhp;
- mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc, 0);
+ mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc);
if (IS_ERR(mhp->umem)) {
err = PTR_ERR(mhp->umem);
kfree(mhp);
diff --git a/drivers/infiniband/hw/cxgb4/mem.c b/drivers/infiniband/hw/cxgb4/mem.c
index 0b91b0f..7a30b55 100644
--- a/drivers/infiniband/hw/cxgb4/mem.c
+++ b/drivers/infiniband/hw/cxgb4/mem.c
@@ -509,7 +509,7 @@ struct ib_mr *c4iw_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
mhp->rhp = rhp;
- mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc, 0);
+ mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc);
if (IS_ERR(mhp->umem)) {
err = PTR_ERR(mhp->umem);
kfree_skb(mhp->dereg_skb);
diff --git a/drivers/infiniband/hw/i40iw/i40iw_verbs.c b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
index 2360338..3cf1c23 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_verbs.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
@@ -1703,7 +1703,7 @@ static struct ib_mr *i40iw_reg_user_mr(struct ib_pd *pd,
if (length > I40IW_MAX_MR_SIZE)
return ERR_PTR(-EINVAL);
- region = ib_umem_get(pd->uobject->context, start, length, acc, 0);
+ region = ib_umem_get(pd->uobject->context, start, length, acc);
if (IS_ERR(region))
return (struct ib_mr *)region;
diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index d6fc8a6..8c3ef00 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -141,8 +141,8 @@ static int mlx4_ib_get_cq_umem(struct mlx4_ib_dev *dev, struct ib_ucontext *cont
int err;
int cqe_size = dev->dev->caps.cqe_size;
- *umem = ib_umem_get(context, buf_addr, cqe * cqe_size,
- IB_ACCESS_LOCAL_WRITE, 1);
+ *umem = ib_umem_get_attrs(context, buf_addr, cqe * cqe_size,
+ IB_ACCESS_LOCAL_WRITE, DMA_BIDIRECTIONAL, DMA_ATTR_WRITE_BARRIER);
if (IS_ERR(*umem))
return PTR_ERR(*umem);
diff --git a/drivers/infiniband/hw/mlx4/doorbell.c b/drivers/infiniband/hw/mlx4/doorbell.c
index c517409..f87aa5f 100644
--- a/drivers/infiniband/hw/mlx4/doorbell.c
+++ b/drivers/infiniband/hw/mlx4/doorbell.c
@@ -62,7 +62,7 @@ int mlx4_ib_db_map_user(struct mlx4_ib_ucontext *context, unsigned long virt,
page->user_virt = (virt & PAGE_MASK);
page->refcnt = 0;
page->umem = ib_umem_get(&context->ibucontext, virt & PAGE_MASK,
- PAGE_SIZE, 0, 0);
+ PAGE_SIZE, 0);
if (IS_ERR(page->umem)) {
err = PTR_ERR(page->umem);
kfree(page);
diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c
index 5d73989..31ee0f7 100644
--- a/drivers/infiniband/hw/mlx4/mr.c
+++ b/drivers/infiniband/hw/mlx4/mr.c
@@ -148,7 +148,7 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
/* Force registering the memory as writable. */
/* Used for memory re-registeration. HCA protects the access */
mr->umem = ib_umem_get(pd->uobject->context, start, length,
- access_flags | IB_ACCESS_LOCAL_WRITE, 0);
+ access_flags | IB_ACCESS_LOCAL_WRITE);
if (IS_ERR(mr->umem)) {
err = PTR_ERR(mr->umem);
goto err_free;
@@ -230,8 +230,7 @@ int mlx4_ib_rereg_user_mr(struct ib_mr *mr, int flags,
ib_umem_release(mmr->umem);
mmr->umem = ib_umem_get(mr->uobject->context, start, length,
mr_access_flags |
- IB_ACCESS_LOCAL_WRITE,
- 0);
+ IB_ACCESS_LOCAL_WRITE);
if (IS_ERR(mmr->umem)) {
err = PTR_ERR(mmr->umem);
/* Prevent mlx4_ib_dereg_mr from free'ing invalid pointer */
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index 768085f..363b1cb 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -742,7 +742,7 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd,
goto err;
qp->umem = ib_umem_get(pd->uobject->context, ucmd.buf_addr,
- qp->buf_size, 0, 0);
+ qp->buf_size, 0);
if (IS_ERR(qp->umem)) {
err = PTR_ERR(qp->umem);
goto err;
diff --git a/drivers/infiniband/hw/mlx4/srq.c b/drivers/infiniband/hw/mlx4/srq.c
index 0597f3e..31c297b 100644
--- a/drivers/infiniband/hw/mlx4/srq.c
+++ b/drivers/infiniband/hw/mlx4/srq.c
@@ -115,7 +115,7 @@ struct ib_srq *mlx4_ib_create_srq(struct ib_pd *pd,
}
srq->umem = ib_umem_get(pd->uobject->context, ucmd.buf_addr,
- buf_size, 0, 0);
+ buf_size, 0);
if (IS_ERR(srq->umem)) {
err = PTR_ERR(srq->umem);
goto err_srq;
diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 35a9f71..c505f0b 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -776,9 +776,11 @@ static int create_cq_user(struct mlx5_ib_dev *dev, struct ib_udata *udata,
*cqe_size = ucmd.cqe_size;
- cq->buf.umem = ib_umem_get(context, ucmd.buf_addr,
- entries * ucmd.cqe_size,
- IB_ACCESS_LOCAL_WRITE, 1);
+ cq->buf.umem = ib_umem_get_attrs(context, ucmd.buf_addr,
+ entries * ucmd.cqe_size,
+ IB_ACCESS_LOCAL_WRITE,
+ DMA_BIDIRECTIONAL,
+ DMA_ATTR_WRITE_BARRIER);
if (IS_ERR(cq->buf.umem)) {
err = PTR_ERR(cq->buf.umem);
return err;
@@ -1137,8 +1139,8 @@ static int resize_user(struct mlx5_ib_dev *dev, struct mlx5_ib_cq *cq,
if (ucmd.reserved0 || ucmd.reserved1)
return -EINVAL;
- umem = ib_umem_get(context, ucmd.buf_addr, entries * ucmd.cqe_size,
- IB_ACCESS_LOCAL_WRITE, 1);
+ umem = ib_umem_get_attrs(context, ucmd.buf_addr, entries * ucmd.cqe_size,
+ IB_ACCESS_LOCAL_WRITE, DMA_BIDIRECTIONAL, DMA_ATTR_WRITE_BARRIER);
if (IS_ERR(umem)) {
err = PTR_ERR(umem);
return err;
diff --git a/drivers/infiniband/hw/mlx5/doorbell.c b/drivers/infiniband/hw/mlx5/doorbell.c
index a0e4e6d..b9f8158 100644
--- a/drivers/infiniband/hw/mlx5/doorbell.c
+++ b/drivers/infiniband/hw/mlx5/doorbell.c
@@ -64,7 +64,7 @@ int mlx5_ib_db_map_user(struct mlx5_ib_ucontext *context, unsigned long virt,
page->user_virt = (virt & PAGE_MASK);
page->refcnt = 0;
page->umem = ib_umem_get(&context->ibucontext, virt & PAGE_MASK,
- PAGE_SIZE, 0, 0);
+ PAGE_SIZE, 0);
if (IS_ERR(page->umem)) {
err = PTR_ERR(page->umem);
kfree(page);
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 6f7e347..baca6e7 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -822,7 +822,7 @@ static struct ib_umem *mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
{
struct mlx5_ib_dev *dev = to_mdev(pd->device);
struct ib_umem *umem = ib_umem_get(pd->uobject->context, start, length,
- access_flags, 0);
+ access_flags);
if (IS_ERR(umem)) {
mlx5_ib_err(dev, "umem get failed (%ld)\n", PTR_ERR(umem));
return (void *)umem;
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index f3c943f..3a48d9d 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -632,7 +632,7 @@ static int mlx5_ib_umem_get(struct mlx5_ib_dev *dev,
{
int err;
- *umem = ib_umem_get(pd->uobject->context, addr, size, 0, 0);
+ *umem = ib_umem_get(pd->uobject->context, addr, size, 0);
if (IS_ERR(*umem)) {
mlx5_ib_dbg(dev, "umem_get failed\n");
return PTR_ERR(*umem);
@@ -684,7 +684,7 @@ static int create_user_rq(struct mlx5_ib_dev *dev, struct ib_pd *pd,
context = to_mucontext(pd->uobject->context);
rwq->umem = ib_umem_get(pd->uobject->context, ucmd->buf_addr,
- rwq->buf_size, 0, 0);
+ rwq->buf_size, 0);
if (IS_ERR(rwq->umem)) {
mlx5_ib_dbg(dev, "umem_get failed\n");
err = PTR_ERR(rwq->umem);
diff --git a/drivers/infiniband/hw/mlx5/srq.c b/drivers/infiniband/hw/mlx5/srq.c
index ed6ac52..f2f614c 100644
--- a/drivers/infiniband/hw/mlx5/srq.c
+++ b/drivers/infiniband/hw/mlx5/srq.c
@@ -112,7 +112,7 @@ static int create_srq_user(struct ib_pd *pd, struct mlx5_ib_srq *srq,
srq->wq_sig = !!(ucmd.flags & MLX5_SRQ_FLAG_SIGNATURE);
srq->umem = ib_umem_get(pd->uobject->context, ucmd.buf_addr, buf_size,
- 0, 0);
+ 0);
if (IS_ERR(srq->umem)) {
mlx5_ib_dbg(dev, "failed umem get, size %d\n", buf_size);
err = PTR_ERR(srq->umem);
diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c
index da2335f..86c60dc 100644
--- a/drivers/infiniband/hw/mthca/mthca_provider.c
+++ b/drivers/infiniband/hw/mthca/mthca_provider.c
@@ -910,6 +910,7 @@ static struct ib_mr *mthca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
int i, k, entry;
int err = 0;
int write_mtt_size;
+ unsigned long dma_attrs = 0;
if (udata->inlen - sizeof (struct ib_uverbs_cmd_hdr) < sizeof ucmd) {
if (!to_mucontext(pd->uobject->context)->reg_mr_warned) {
@@ -926,8 +927,11 @@ static struct ib_mr *mthca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
if (!mr)
return ERR_PTR(-ENOMEM);
- mr->umem = ib_umem_get(pd->uobject->context, start, length, acc,
- ucmd.mr_attrs & MTHCA_MR_DMASYNC);
+ if (ucmd.mr_attrs & MTHCA_MR_DMASYNC)
+ dma_attrs |= DMA_ATTR_WRITE_BARRIER;
+
+ mr->umem = ib_umem_get_attrs(pd->uobject->context, start, length, acc,
+ DMA_BIDIRECTIONAL, dma_attrs);
if (IS_ERR(mr->umem)) {
err = PTR_ERR(mr->umem);
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index bd69125..f417e1d 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -2170,7 +2170,7 @@ static struct ib_mr *nes_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
u8 stag_key;
int first_page = 1;
- region = ib_umem_get(pd->uobject->context, start, length, acc, 0);
+ region = ib_umem_get(pd->uobject->context, start, length, acc);
if (IS_ERR(region)) {
return (struct ib_mr *)region;
}
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
index b1a3d91..5c80765 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
@@ -967,7 +967,7 @@ struct ib_mr *ocrdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,
mr = kzalloc(sizeof(*mr), GFP_KERNEL);
if (!mr)
return ERR_PTR(status);
- mr->umem = ib_umem_get(ibpd->uobject->context, start, len, acc, 0);
+ mr->umem = ib_umem_get(ibpd->uobject->context, start, len, acc);
if (IS_ERR(mr->umem)) {
status = -EFAULT;
goto umem_err;
diff --git a/drivers/infiniband/sw/rdmavt/mr.c b/drivers/infiniband/sw/rdmavt/mr.c
index 80c4b6b..564724a 100644
--- a/drivers/infiniband/sw/rdmavt/mr.c
+++ b/drivers/infiniband/sw/rdmavt/mr.c
@@ -369,7 +369,7 @@ struct ib_mr *rvt_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
return ERR_PTR(-EINVAL);
umem = ib_umem_get(pd->uobject->context, start, length,
- mr_access_flags, 0);
+ mr_access_flags);
if (IS_ERR(umem))
return (void *)umem;
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index f3dab65..f4d47fd 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -169,7 +169,7 @@ int rxe_mem_init_user(struct rxe_dev *rxe, struct rxe_pd *pd, u64 start,
void *vaddr;
int err;
- umem = ib_umem_get(pd->ibpd.uobject->context, start, length, access, 0);
+ umem = ib_umem_get(pd->ibpd.uobject->context, start, length, access);
if (IS_ERR(umem)) {
pr_warn("err %d from rxe_umem_get\n",
(int)PTR_ERR(umem));
diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
index 2876679..472c2f6 100644
--- a/include/rdma/ib_umem.h
+++ b/include/rdma/ib_umem.h
@@ -84,7 +84,7 @@ static inline size_t ib_umem_num_pages(struct ib_umem *umem)
#ifdef CONFIG_INFINIBAND_USER_MEM
struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
- size_t size, int access, int dmasync);
+ size_t size, int access);
struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
size_t size, int access,
enum dma_data_direction dir,
@@ -100,7 +100,7 @@ int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
static inline struct ib_umem *ib_umem_get(struct ib_ucontext *context,
unsigned long addr, size_t size,
- int access, int dmasync) {
+ int access) {
return ERR_PTR(-EINVAL);
}
static inline struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context,
--
git-series 0.8.10
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 3/8] ib_umem: With the new ib_umem_get_attrs, simplify ib_umem_get
@ 2016-09-16 18:31 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Knut Omang, Sean Hefty, Hal Rosenstock,
Steve Wise, Faisal Latif, Chien Tin Tung, Mustafa Ismail,
Shiraz Saleem, Tatyana Nikolova, Yishai Hadas, Matan Barak,
Leon Romanovsky, Selvin Xavier, Devesh Sharma, Mitesh Ahuja,
Dennis Dalessandro, Joerg Roedel, Hans-Christian Noren Egtvedt,
Robin Murphy, Vineet Gupta, Ingo Molnar, Dave Hansen,
Krzysztof Kozlowski, Ira Weiny, Sagi Grimberg, Haggai Eran,
Amir Vadai, Moni Shoua
Instead of having most drivers provide a specialized
and weakly typed "boolean" argument
dmasync = 0 argument which are only set to nonzero 3 places,
instead let these special use cases use the
more generic ib_umem_get_attrs which gives access to
the full range of dma attributes and also dma direction,
and eliminate the dmasync parameter from the ib_umem_get call,
Signed-off-by: Knut Omang <knut.omang@oracle.com>
---
drivers/infiniband/core/umem.c | 8 ++------
drivers/infiniband/hw/cxgb3/iwch_provider.c | 2 +-
drivers/infiniband/hw/cxgb4/mem.c | 2 +-
drivers/infiniband/hw/i40iw/i40iw_verbs.c | 2 +-
drivers/infiniband/hw/mlx4/cq.c | 4 ++--
drivers/infiniband/hw/mlx4/doorbell.c | 2 +-
drivers/infiniband/hw/mlx4/mr.c | 5 ++---
drivers/infiniband/hw/mlx4/qp.c | 2 +-
drivers/infiniband/hw/mlx4/srq.c | 2 +-
drivers/infiniband/hw/mlx5/cq.c | 12 +++++++-----
drivers/infiniband/hw/mlx5/doorbell.c | 2 +-
drivers/infiniband/hw/mlx5/mr.c | 2 +-
drivers/infiniband/hw/mlx5/qp.c | 4 ++--
drivers/infiniband/hw/mlx5/srq.c | 2 +-
drivers/infiniband/hw/mthca/mthca_provider.c | 8 ++++++--
drivers/infiniband/hw/nes/nes_verbs.c | 2 +-
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 2 +-
drivers/infiniband/sw/rdmavt/mr.c | 2 +-
drivers/infiniband/sw/rxe/rxe_mr.c | 2 +-
include/rdma/ib_umem.h | 4 ++--
20 files changed, 36 insertions(+), 35 deletions(-)
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 699a0f7..938c940 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -77,15 +77,11 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
* @addr: userspace virtual address to start at
* @size: length of region to pin
* @access: IB_ACCESS_xxx flags for memory being pinned
- * @dmasync: flush in-flight DMA when the memory region is written
*/
struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
- size_t size, int access, int dmasync)
+ size_t size, int access)
{
- unsigned long dma_attrs = 0;
- if (dmasync)
- dma_attrs |= DMA_ATTR_WRITE_BARRIER;
- return ib_umem_get_attrs(context, addr, size, access, DMA_BIDIRECTIONAL, dma_attrs);
+ return ib_umem_get_attrs(context, addr, size, access, DMA_BIDIRECTIONAL, 0);
}
EXPORT_SYMBOL(ib_umem_get);
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 3edb806..4225479 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -576,7 +576,7 @@ static struct ib_mr *iwch_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
mhp->rhp = rhp;
- mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc, 0);
+ mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc);
if (IS_ERR(mhp->umem)) {
err = PTR_ERR(mhp->umem);
kfree(mhp);
diff --git a/drivers/infiniband/hw/cxgb4/mem.c b/drivers/infiniband/hw/cxgb4/mem.c
index 0b91b0f..7a30b55 100644
--- a/drivers/infiniband/hw/cxgb4/mem.c
+++ b/drivers/infiniband/hw/cxgb4/mem.c
@@ -509,7 +509,7 @@ struct ib_mr *c4iw_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
mhp->rhp = rhp;
- mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc, 0);
+ mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc);
if (IS_ERR(mhp->umem)) {
err = PTR_ERR(mhp->umem);
kfree_skb(mhp->dereg_skb);
diff --git a/drivers/infiniband/hw/i40iw/i40iw_verbs.c b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
index 2360338..3cf1c23 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_verbs.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
@@ -1703,7 +1703,7 @@ static struct ib_mr *i40iw_reg_user_mr(struct ib_pd *pd,
if (length > I40IW_MAX_MR_SIZE)
return ERR_PTR(-EINVAL);
- region = ib_umem_get(pd->uobject->context, start, length, acc, 0);
+ region = ib_umem_get(pd->uobject->context, start, length, acc);
if (IS_ERR(region))
return (struct ib_mr *)region;
diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index d6fc8a6..8c3ef00 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -141,8 +141,8 @@ static int mlx4_ib_get_cq_umem(struct mlx4_ib_dev *dev, struct ib_ucontext *cont
int err;
int cqe_size = dev->dev->caps.cqe_size;
- *umem = ib_umem_get(context, buf_addr, cqe * cqe_size,
- IB_ACCESS_LOCAL_WRITE, 1);
+ *umem = ib_umem_get_attrs(context, buf_addr, cqe * cqe_size,
+ IB_ACCESS_LOCAL_WRITE, DMA_BIDIRECTIONAL, DMA_ATTR_WRITE_BARRIER);
if (IS_ERR(*umem))
return PTR_ERR(*umem);
diff --git a/drivers/infiniband/hw/mlx4/doorbell.c b/drivers/infiniband/hw/mlx4/doorbell.c
index c517409..f87aa5f 100644
--- a/drivers/infiniband/hw/mlx4/doorbell.c
+++ b/drivers/infiniband/hw/mlx4/doorbell.c
@@ -62,7 +62,7 @@ int mlx4_ib_db_map_user(struct mlx4_ib_ucontext *context, unsigned long virt,
page->user_virt = (virt & PAGE_MASK);
page->refcnt = 0;
page->umem = ib_umem_get(&context->ibucontext, virt & PAGE_MASK,
- PAGE_SIZE, 0, 0);
+ PAGE_SIZE, 0);
if (IS_ERR(page->umem)) {
err = PTR_ERR(page->umem);
kfree(page);
diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c
index 5d73989..31ee0f7 100644
--- a/drivers/infiniband/hw/mlx4/mr.c
+++ b/drivers/infiniband/hw/mlx4/mr.c
@@ -148,7 +148,7 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
/* Force registering the memory as writable. */
/* Used for memory re-registeration. HCA protects the access */
mr->umem = ib_umem_get(pd->uobject->context, start, length,
- access_flags | IB_ACCESS_LOCAL_WRITE, 0);
+ access_flags | IB_ACCESS_LOCAL_WRITE);
if (IS_ERR(mr->umem)) {
err = PTR_ERR(mr->umem);
goto err_free;
@@ -230,8 +230,7 @@ int mlx4_ib_rereg_user_mr(struct ib_mr *mr, int flags,
ib_umem_release(mmr->umem);
mmr->umem = ib_umem_get(mr->uobject->context, start, length,
mr_access_flags |
- IB_ACCESS_LOCAL_WRITE,
- 0);
+ IB_ACCESS_LOCAL_WRITE);
if (IS_ERR(mmr->umem)) {
err = PTR_ERR(mmr->umem);
/* Prevent mlx4_ib_dereg_mr from free'ing invalid pointer */
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index 768085f..363b1cb 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -742,7 +742,7 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd,
goto err;
qp->umem = ib_umem_get(pd->uobject->context, ucmd.buf_addr,
- qp->buf_size, 0, 0);
+ qp->buf_size, 0);
if (IS_ERR(qp->umem)) {
err = PTR_ERR(qp->umem);
goto err;
diff --git a/drivers/infiniband/hw/mlx4/srq.c b/drivers/infiniband/hw/mlx4/srq.c
index 0597f3e..31c297b 100644
--- a/drivers/infiniband/hw/mlx4/srq.c
+++ b/drivers/infiniband/hw/mlx4/srq.c
@@ -115,7 +115,7 @@ struct ib_srq *mlx4_ib_create_srq(struct ib_pd *pd,
}
srq->umem = ib_umem_get(pd->uobject->context, ucmd.buf_addr,
- buf_size, 0, 0);
+ buf_size, 0);
if (IS_ERR(srq->umem)) {
err = PTR_ERR(srq->umem);
goto err_srq;
diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 35a9f71..c505f0b 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -776,9 +776,11 @@ static int create_cq_user(struct mlx5_ib_dev *dev, struct ib_udata *udata,
*cqe_size = ucmd.cqe_size;
- cq->buf.umem = ib_umem_get(context, ucmd.buf_addr,
- entries * ucmd.cqe_size,
- IB_ACCESS_LOCAL_WRITE, 1);
+ cq->buf.umem = ib_umem_get_attrs(context, ucmd.buf_addr,
+ entries * ucmd.cqe_size,
+ IB_ACCESS_LOCAL_WRITE,
+ DMA_BIDIRECTIONAL,
+ DMA_ATTR_WRITE_BARRIER);
if (IS_ERR(cq->buf.umem)) {
err = PTR_ERR(cq->buf.umem);
return err;
@@ -1137,8 +1139,8 @@ static int resize_user(struct mlx5_ib_dev *dev, struct mlx5_ib_cq *cq,
if (ucmd.reserved0 || ucmd.reserved1)
return -EINVAL;
- umem = ib_umem_get(context, ucmd.buf_addr, entries * ucmd.cqe_size,
- IB_ACCESS_LOCAL_WRITE, 1);
+ umem = ib_umem_get_attrs(context, ucmd.buf_addr, entries * ucmd.cqe_size,
+ IB_ACCESS_LOCAL_WRITE, DMA_BIDIRECTIONAL, DMA_ATTR_WRITE_BARRIER);
if (IS_ERR(umem)) {
err = PTR_ERR(umem);
return err;
diff --git a/drivers/infiniband/hw/mlx5/doorbell.c b/drivers/infiniband/hw/mlx5/doorbell.c
index a0e4e6d..b9f8158 100644
--- a/drivers/infiniband/hw/mlx5/doorbell.c
+++ b/drivers/infiniband/hw/mlx5/doorbell.c
@@ -64,7 +64,7 @@ int mlx5_ib_db_map_user(struct mlx5_ib_ucontext *context, unsigned long virt,
page->user_virt = (virt & PAGE_MASK);
page->refcnt = 0;
page->umem = ib_umem_get(&context->ibucontext, virt & PAGE_MASK,
- PAGE_SIZE, 0, 0);
+ PAGE_SIZE, 0);
if (IS_ERR(page->umem)) {
err = PTR_ERR(page->umem);
kfree(page);
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 6f7e347..baca6e7 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -822,7 +822,7 @@ static struct ib_umem *mr_umem_get(struct ib_pd *pd, u64 start, u64 length,
{
struct mlx5_ib_dev *dev = to_mdev(pd->device);
struct ib_umem *umem = ib_umem_get(pd->uobject->context, start, length,
- access_flags, 0);
+ access_flags);
if (IS_ERR(umem)) {
mlx5_ib_err(dev, "umem get failed (%ld)\n", PTR_ERR(umem));
return (void *)umem;
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index f3c943f..3a48d9d 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -632,7 +632,7 @@ static int mlx5_ib_umem_get(struct mlx5_ib_dev *dev,
{
int err;
- *umem = ib_umem_get(pd->uobject->context, addr, size, 0, 0);
+ *umem = ib_umem_get(pd->uobject->context, addr, size, 0);
if (IS_ERR(*umem)) {
mlx5_ib_dbg(dev, "umem_get failed\n");
return PTR_ERR(*umem);
@@ -684,7 +684,7 @@ static int create_user_rq(struct mlx5_ib_dev *dev, struct ib_pd *pd,
context = to_mucontext(pd->uobject->context);
rwq->umem = ib_umem_get(pd->uobject->context, ucmd->buf_addr,
- rwq->buf_size, 0, 0);
+ rwq->buf_size, 0);
if (IS_ERR(rwq->umem)) {
mlx5_ib_dbg(dev, "umem_get failed\n");
err = PTR_ERR(rwq->umem);
diff --git a/drivers/infiniband/hw/mlx5/srq.c b/drivers/infiniband/hw/mlx5/srq.c
index ed6ac52..f2f614c 100644
--- a/drivers/infiniband/hw/mlx5/srq.c
+++ b/drivers/infiniband/hw/mlx5/srq.c
@@ -112,7 +112,7 @@ static int create_srq_user(struct ib_pd *pd, struct mlx5_ib_srq *srq,
srq->wq_sig = !!(ucmd.flags & MLX5_SRQ_FLAG_SIGNATURE);
srq->umem = ib_umem_get(pd->uobject->context, ucmd.buf_addr, buf_size,
- 0, 0);
+ 0);
if (IS_ERR(srq->umem)) {
mlx5_ib_dbg(dev, "failed umem get, size %d\n", buf_size);
err = PTR_ERR(srq->umem);
diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c
index da2335f..86c60dc 100644
--- a/drivers/infiniband/hw/mthca/mthca_provider.c
+++ b/drivers/infiniband/hw/mthca/mthca_provider.c
@@ -910,6 +910,7 @@ static struct ib_mr *mthca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
int i, k, entry;
int err = 0;
int write_mtt_size;
+ unsigned long dma_attrs = 0;
if (udata->inlen - sizeof (struct ib_uverbs_cmd_hdr) < sizeof ucmd) {
if (!to_mucontext(pd->uobject->context)->reg_mr_warned) {
@@ -926,8 +927,11 @@ static struct ib_mr *mthca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
if (!mr)
return ERR_PTR(-ENOMEM);
- mr->umem = ib_umem_get(pd->uobject->context, start, length, acc,
- ucmd.mr_attrs & MTHCA_MR_DMASYNC);
+ if (ucmd.mr_attrs & MTHCA_MR_DMASYNC)
+ dma_attrs |= DMA_ATTR_WRITE_BARRIER;
+
+ mr->umem = ib_umem_get_attrs(pd->uobject->context, start, length, acc,
+ DMA_BIDIRECTIONAL, dma_attrs);
if (IS_ERR(mr->umem)) {
err = PTR_ERR(mr->umem);
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index bd69125..f417e1d 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -2170,7 +2170,7 @@ static struct ib_mr *nes_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
u8 stag_key;
int first_page = 1;
- region = ib_umem_get(pd->uobject->context, start, length, acc, 0);
+ region = ib_umem_get(pd->uobject->context, start, length, acc);
if (IS_ERR(region)) {
return (struct ib_mr *)region;
}
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
index b1a3d91..5c80765 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
@@ -967,7 +967,7 @@ struct ib_mr *ocrdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,
mr = kzalloc(sizeof(*mr), GFP_KERNEL);
if (!mr)
return ERR_PTR(status);
- mr->umem = ib_umem_get(ibpd->uobject->context, start, len, acc, 0);
+ mr->umem = ib_umem_get(ibpd->uobject->context, start, len, acc);
if (IS_ERR(mr->umem)) {
status = -EFAULT;
goto umem_err;
diff --git a/drivers/infiniband/sw/rdmavt/mr.c b/drivers/infiniband/sw/rdmavt/mr.c
index 80c4b6b..564724a 100644
--- a/drivers/infiniband/sw/rdmavt/mr.c
+++ b/drivers/infiniband/sw/rdmavt/mr.c
@@ -369,7 +369,7 @@ struct ib_mr *rvt_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
return ERR_PTR(-EINVAL);
umem = ib_umem_get(pd->uobject->context, start, length,
- mr_access_flags, 0);
+ mr_access_flags);
if (IS_ERR(umem))
return (void *)umem;
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index f3dab65..f4d47fd 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -169,7 +169,7 @@ int rxe_mem_init_user(struct rxe_dev *rxe, struct rxe_pd *pd, u64 start,
void *vaddr;
int err;
- umem = ib_umem_get(pd->ibpd.uobject->context, start, length, access, 0);
+ umem = ib_umem_get(pd->ibpd.uobject->context, start, length, access);
if (IS_ERR(umem)) {
pr_warn("err %d from rxe_umem_get\n",
(int)PTR_ERR(umem));
diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
index 2876679..472c2f6 100644
--- a/include/rdma/ib_umem.h
+++ b/include/rdma/ib_umem.h
@@ -84,7 +84,7 @@ static inline size_t ib_umem_num_pages(struct ib_umem *umem)
#ifdef CONFIG_INFINIBAND_USER_MEM
struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
- size_t size, int access, int dmasync);
+ size_t size, int access);
struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
size_t size, int access,
enum dma_data_direction dir,
@@ -100,7 +100,7 @@ int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
static inline struct ib_umem *ib_umem_get(struct ib_ucontext *context,
unsigned long addr, size_t size,
- int access, int dmasync) {
+ int access) {
return ERR_PTR(-EINVAL);
}
static inline struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context,
--
git-series 0.8.10
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 4/8] ib: Add udata argument to create_ah
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
@ 2016-09-16 18:31 ` Knut Omang
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
` (2 subsequent siblings)
3 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Knut Omang, Sean Hefty,
Hal Rosenstock, Steve Wise, Faisal Latif, Chien Tin Tung,
Mustafa Ismail, Shiraz Saleem, Tatyana Nikolova, Yishai Hadas,
Matan Barak, Leon Romanovsky, Selvin Xavier, Devesh Sharma,
Mitesh Ahuja, Christian Benvenuti, Dave Goodell
Most of the ib device driver entry points supports optional
device specific parameter transfer between user space and kernel space
via the udata argument - add a similar argument for ib_create_ah.
Update all infiniband drivers to include this agument
in their driver entry point implementation.
---
drivers/infiniband/core/uverbs_cmd.c | 10 +++++++++-
drivers/infiniband/core/verbs.c | 2 +-
drivers/infiniband/hw/cxgb3/iwch_provider.c | 3 ++-
drivers/infiniband/hw/cxgb4/provider.c | 3 ++-
drivers/infiniband/hw/i40iw/i40iw_verbs.c | 3 ++-
drivers/infiniband/hw/mlx4/ah.c | 3 ++-
drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 ++-
drivers/infiniband/hw/mlx5/ah.c | 3 ++-
drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 ++-
drivers/infiniband/hw/mthca/mthca_provider.c | 3 ++-
drivers/infiniband/hw/nes/nes_verbs.c | 3 ++-
drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 5 +++--
drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 3 ++-
drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 3 ++-
drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 3 ++-
drivers/infiniband/sw/rxe/rxe_verbs.c | 3 ++-
include/rdma/ib_verbs.h | 3 ++-
17 files changed, 41 insertions(+), 18 deletions(-)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index f664731..dbc5885 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -2874,6 +2874,7 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file,
struct ib_pd *pd;
struct ib_ah *ah;
struct ib_ah_attr attr;
+ struct ib_udata udata;
int ret;
if (out_len < sizeof resp)
@@ -2882,6 +2883,10 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file,
if (copy_from_user(&cmd, buf, sizeof cmd))
return -EFAULT;
+ INIT_UDATA(&udata, buf + sizeof cmd,
+ (unsigned long) cmd.response + sizeof resp,
+ in_len - sizeof cmd, out_len - sizeof resp);
+
uobj = kmalloc(sizeof *uobj, GFP_KERNEL);
if (!uobj)
return -ENOMEM;
@@ -2908,14 +2913,17 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file,
memset(&attr.dmac, 0, sizeof(attr.dmac));
memcpy(attr.grh.dgid.raw, cmd.attr.grh.dgid, 16);
- ah = ib_create_ah(pd, &attr);
+ ah = pd->device->create_ah(pd, &attr, &udata);
if (IS_ERR(ah)) {
ret = PTR_ERR(ah);
goto err_put;
}
+ ah->device = pd->device;
+ ah->pd = pd;
ah->uobject = uobj;
uobj->object = ah;
+ atomic_inc(&pd->usecnt);
ret = idr_add_uobj(&ib_uverbs_ah_idr, uobj);
if (ret)
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index f2b776e..6efe23d 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -293,7 +293,7 @@ struct ib_ah *ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
{
struct ib_ah *ah;
- ah = pd->device->create_ah(pd, ah_attr);
+ ah = pd->device->create_ah(pd, ah_attr, NULL);
if (!IS_ERR(ah)) {
ah->device = pd->device;
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 4225479..ffdf530 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -62,7 +62,8 @@
#include "common.h"
static struct ib_ah *iwch_ah_create(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr)
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
return ERR_PTR(-ENOSYS);
}
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index df127ce..b476df0 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -59,7 +59,8 @@ module_param(fastreg_support, int, 0644);
MODULE_PARM_DESC(fastreg_support, "Advertise fastreg support (default=1)");
static struct ib_ah *c4iw_ah_create(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr)
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
return ERR_PTR(-ENOSYS);
}
diff --git a/drivers/infiniband/hw/i40iw/i40iw_verbs.c b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
index 3cf1c23..c814a2e 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_verbs.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
@@ -2564,7 +2564,8 @@ static int i40iw_query_pkey(struct ib_device *ibdev,
* @ah_attr: address handle attributes
*/
static struct ib_ah *i40iw_create_ah(struct ib_pd *ibpd,
- struct ib_ah_attr *attr)
+ struct ib_ah_attr *attr,
+ struct ib_udata *udata)
{
return ERR_PTR(-ENOSYS);
}
diff --git a/drivers/infiniband/hw/mlx4/ah.c b/drivers/infiniband/hw/mlx4/ah.c
index 5fc6233..1ea2ba3 100644
--- a/drivers/infiniband/hw/mlx4/ah.c
+++ b/drivers/infiniband/hw/mlx4/ah.c
@@ -124,7 +124,8 @@ static struct ib_ah *create_iboe_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr
return &ah->ibah;
}
-struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
+struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
struct mlx4_ib_ah *ah;
struct ib_ah *ret;
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h
index 7c5832e..c3812e3 100644
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -740,7 +740,8 @@ int mlx4_ib_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags);
void __mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq);
void mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq);
-struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr);
+struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata);
int mlx4_ib_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr);
int mlx4_ib_destroy_ah(struct ib_ah *ah);
diff --git a/drivers/infiniband/hw/mlx5/ah.c b/drivers/infiniband/hw/mlx5/ah.c
index 745efa4..b5d0f9a 100644
--- a/drivers/infiniband/hw/mlx5/ah.c
+++ b/drivers/infiniband/hw/mlx5/ah.c
@@ -64,7 +64,8 @@ static struct ib_ah *create_ib_ah(struct mlx5_ib_dev *dev,
return &ah->ibah;
}
-struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
+struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
struct mlx5_ib_ah *ah;
struct mlx5_ib_dev *dev = to_mdev(pd->device);
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index a59034a..82a1ccd 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -726,7 +726,8 @@ void mlx5_ib_free_srq_wqe(struct mlx5_ib_srq *srq, int wqe_index);
int mlx5_MAD_IFC(struct mlx5_ib_dev *dev, int ignore_mkey, int ignore_bkey,
u8 port, const struct ib_wc *in_wc, const struct ib_grh *in_grh,
const void *in_mad, void *response_mad);
-struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr);
+struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata);
int mlx5_ib_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr);
int mlx5_ib_destroy_ah(struct ib_ah *ah);
struct ib_srq *mlx5_ib_create_srq(struct ib_pd *pd,
diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c
index 86c60dc..9618e32 100644
--- a/drivers/infiniband/hw/mthca/mthca_provider.c
+++ b/drivers/infiniband/hw/mthca/mthca_provider.c
@@ -409,7 +409,8 @@ static int mthca_dealloc_pd(struct ib_pd *pd)
}
static struct ib_ah *mthca_ah_create(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr)
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
int err;
struct mthca_ah *ah;
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index f417e1d..76862c9 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -771,7 +771,8 @@ static int nes_dealloc_pd(struct ib_pd *ibpd)
/**
* nes_create_ah
*/
-static struct ib_ah *nes_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
+static struct ib_ah *nes_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
return ERR_PTR(-ENOSYS);
}
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
index 797362a..c65ac29 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
@@ -154,7 +154,8 @@ static inline int set_av_attr(struct ocrdma_dev *dev, struct ocrdma_ah *ah,
return status;
}
-struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr)
+struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr,
+ struct ib_udata *udata)
{
u32 *ahid_addr;
int status;
@@ -203,7 +204,7 @@ struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr)
&sgid_attr.ndev->ifindex,
NULL);
if (status) {
- pr_err("%s(): Failed to resolve dmac from gid."
+ pr_err("%s(): Failed to resolve dmac from gid."
"status = %d\n", __func__, status);
goto av_conf_err;
}
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_ah.h b/drivers/infiniband/hw/ocrdma/ocrdma_ah.h
index 3856dd4..327b888 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_ah.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_ah.h
@@ -50,7 +50,8 @@ enum {
OCRDMA_AH_L3_TYPE_MASK = 0x03,
OCRDMA_AH_L3_TYPE_SHIFT = 0x1D /* 29 bits */
};
-struct ib_ah *ocrdma_create_ah(struct ib_pd *, struct ib_ah_attr *);
+struct ib_ah *ocrdma_create_ah(struct ib_pd *, struct ib_ah_attr *,
+ struct ib_udata *);
int ocrdma_destroy_ah(struct ib_ah *);
int ocrdma_query_ah(struct ib_ah *, struct ib_ah_attr *);
int ocrdma_modify_ah(struct ib_ah *, struct ib_ah_attr *);
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
index a5bfbba..9051193 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
+++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
@@ -738,7 +738,8 @@ int usnic_ib_mmap(struct ib_ucontext *context,
/* In ib callbacks section - Start of stub funcs */
struct ib_ah *usnic_ib_create_ah(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr)
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
usnic_dbg("\n");
return ERR_PTR(-EPERM);
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
index 0d9d2e6..1eecd8e 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
+++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
@@ -75,7 +75,8 @@ int usnic_ib_dealloc_ucontext(struct ib_ucontext *ibcontext);
int usnic_ib_mmap(struct ib_ucontext *context,
struct vm_area_struct *vma);
struct ib_ah *usnic_ib_create_ah(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr);
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata);
int usnic_ib_destroy_ah(struct ib_ah *ah);
int usnic_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
struct ib_send_wr **bad_wr);
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index 4552be9..64ade0b 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -314,7 +314,8 @@ static int rxe_init_av(struct rxe_dev *rxe, struct ib_ah_attr *attr,
return err;
}
-static struct ib_ah *rxe_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr)
+static struct ib_ah *rxe_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr,
+ struct ib_udata *udata)
{
int err;
struct rxe_dev *rxe = to_rdev(ibpd->device);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 8e90dd2..192d591 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1879,7 +1879,8 @@ struct ib_device {
struct ib_udata *udata);
int (*dealloc_pd)(struct ib_pd *pd);
struct ib_ah * (*create_ah)(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr);
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata);
int (*modify_ah)(struct ib_ah *ah,
struct ib_ah_attr *ah_attr);
int (*query_ah)(struct ib_ah *ah,
--
git-series 0.8.10
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 4/8] ib: Add udata argument to create_ah
@ 2016-09-16 18:31 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Knut Omang, Sean Hefty, Hal Rosenstock,
Steve Wise, Faisal Latif, Chien Tin Tung, Mustafa Ismail,
Shiraz Saleem, Tatyana Nikolova, Yishai Hadas, Matan Barak,
Leon Romanovsky, Selvin Xavier, Devesh Sharma, Mitesh Ahuja,
Christian Benvenuti, Dave Goodell, Sagi Grimberg, Majd Dibbiny,
Eran Ben Elisha, Or Gerlitz, Bart Van Assche, Somnath Kotur,
Jason Gunthorpe, Dennis Dalessandro, Ira Weiny, Moni Shoua,
Haggai Eran, Amir Vadai, Kamal Heib
Most of the ib device driver entry points supports optional
device specific parameter transfer between user space and kernel space
via the udata argument - add a similar argument for ib_create_ah.
Update all infiniband drivers to include this agument
in their driver entry point implementation.
---
drivers/infiniband/core/uverbs_cmd.c | 10 +++++++++-
drivers/infiniband/core/verbs.c | 2 +-
drivers/infiniband/hw/cxgb3/iwch_provider.c | 3 ++-
drivers/infiniband/hw/cxgb4/provider.c | 3 ++-
drivers/infiniband/hw/i40iw/i40iw_verbs.c | 3 ++-
drivers/infiniband/hw/mlx4/ah.c | 3 ++-
drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 ++-
drivers/infiniband/hw/mlx5/ah.c | 3 ++-
drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 ++-
drivers/infiniband/hw/mthca/mthca_provider.c | 3 ++-
drivers/infiniband/hw/nes/nes_verbs.c | 3 ++-
drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 5 +++--
drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 3 ++-
drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 3 ++-
drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 3 ++-
drivers/infiniband/sw/rxe/rxe_verbs.c | 3 ++-
include/rdma/ib_verbs.h | 3 ++-
17 files changed, 41 insertions(+), 18 deletions(-)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index f664731..dbc5885 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -2874,6 +2874,7 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file,
struct ib_pd *pd;
struct ib_ah *ah;
struct ib_ah_attr attr;
+ struct ib_udata udata;
int ret;
if (out_len < sizeof resp)
@@ -2882,6 +2883,10 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file,
if (copy_from_user(&cmd, buf, sizeof cmd))
return -EFAULT;
+ INIT_UDATA(&udata, buf + sizeof cmd,
+ (unsigned long) cmd.response + sizeof resp,
+ in_len - sizeof cmd, out_len - sizeof resp);
+
uobj = kmalloc(sizeof *uobj, GFP_KERNEL);
if (!uobj)
return -ENOMEM;
@@ -2908,14 +2913,17 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *file,
memset(&attr.dmac, 0, sizeof(attr.dmac));
memcpy(attr.grh.dgid.raw, cmd.attr.grh.dgid, 16);
- ah = ib_create_ah(pd, &attr);
+ ah = pd->device->create_ah(pd, &attr, &udata);
if (IS_ERR(ah)) {
ret = PTR_ERR(ah);
goto err_put;
}
+ ah->device = pd->device;
+ ah->pd = pd;
ah->uobject = uobj;
uobj->object = ah;
+ atomic_inc(&pd->usecnt);
ret = idr_add_uobj(&ib_uverbs_ah_idr, uobj);
if (ret)
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index f2b776e..6efe23d 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -293,7 +293,7 @@ struct ib_ah *ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
{
struct ib_ah *ah;
- ah = pd->device->create_ah(pd, ah_attr);
+ ah = pd->device->create_ah(pd, ah_attr, NULL);
if (!IS_ERR(ah)) {
ah->device = pd->device;
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 4225479..ffdf530 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -62,7 +62,8 @@
#include "common.h"
static struct ib_ah *iwch_ah_create(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr)
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
return ERR_PTR(-ENOSYS);
}
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index df127ce..b476df0 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -59,7 +59,8 @@ module_param(fastreg_support, int, 0644);
MODULE_PARM_DESC(fastreg_support, "Advertise fastreg support (default=1)");
static struct ib_ah *c4iw_ah_create(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr)
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
return ERR_PTR(-ENOSYS);
}
diff --git a/drivers/infiniband/hw/i40iw/i40iw_verbs.c b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
index 3cf1c23..c814a2e 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_verbs.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
@@ -2564,7 +2564,8 @@ static int i40iw_query_pkey(struct ib_device *ibdev,
* @ah_attr: address handle attributes
*/
static struct ib_ah *i40iw_create_ah(struct ib_pd *ibpd,
- struct ib_ah_attr *attr)
+ struct ib_ah_attr *attr,
+ struct ib_udata *udata)
{
return ERR_PTR(-ENOSYS);
}
diff --git a/drivers/infiniband/hw/mlx4/ah.c b/drivers/infiniband/hw/mlx4/ah.c
index 5fc6233..1ea2ba3 100644
--- a/drivers/infiniband/hw/mlx4/ah.c
+++ b/drivers/infiniband/hw/mlx4/ah.c
@@ -124,7 +124,8 @@ static struct ib_ah *create_iboe_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr
return &ah->ibah;
}
-struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
+struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
struct mlx4_ib_ah *ah;
struct ib_ah *ret;
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h
index 7c5832e..c3812e3 100644
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -740,7 +740,8 @@ int mlx4_ib_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags);
void __mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq);
void mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq);
-struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr);
+struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata);
int mlx4_ib_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr);
int mlx4_ib_destroy_ah(struct ib_ah *ah);
diff --git a/drivers/infiniband/hw/mlx5/ah.c b/drivers/infiniband/hw/mlx5/ah.c
index 745efa4..b5d0f9a 100644
--- a/drivers/infiniband/hw/mlx5/ah.c
+++ b/drivers/infiniband/hw/mlx5/ah.c
@@ -64,7 +64,8 @@ static struct ib_ah *create_ib_ah(struct mlx5_ib_dev *dev,
return &ah->ibah;
}
-struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
+struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
struct mlx5_ib_ah *ah;
struct mlx5_ib_dev *dev = to_mdev(pd->device);
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index a59034a..82a1ccd 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -726,7 +726,8 @@ void mlx5_ib_free_srq_wqe(struct mlx5_ib_srq *srq, int wqe_index);
int mlx5_MAD_IFC(struct mlx5_ib_dev *dev, int ignore_mkey, int ignore_bkey,
u8 port, const struct ib_wc *in_wc, const struct ib_grh *in_grh,
const void *in_mad, void *response_mad);
-struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr);
+struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata);
int mlx5_ib_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr);
int mlx5_ib_destroy_ah(struct ib_ah *ah);
struct ib_srq *mlx5_ib_create_srq(struct ib_pd *pd,
diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c
index 86c60dc..9618e32 100644
--- a/drivers/infiniband/hw/mthca/mthca_provider.c
+++ b/drivers/infiniband/hw/mthca/mthca_provider.c
@@ -409,7 +409,8 @@ static int mthca_dealloc_pd(struct ib_pd *pd)
}
static struct ib_ah *mthca_ah_create(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr)
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
int err;
struct mthca_ah *ah;
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index f417e1d..76862c9 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -771,7 +771,8 @@ static int nes_dealloc_pd(struct ib_pd *ibpd)
/**
* nes_create_ah
*/
-static struct ib_ah *nes_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
+static struct ib_ah *nes_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
return ERR_PTR(-ENOSYS);
}
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
index 797362a..c65ac29 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
@@ -154,7 +154,8 @@ static inline int set_av_attr(struct ocrdma_dev *dev, struct ocrdma_ah *ah,
return status;
}
-struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr)
+struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr,
+ struct ib_udata *udata)
{
u32 *ahid_addr;
int status;
@@ -203,7 +204,7 @@ struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr)
&sgid_attr.ndev->ifindex,
NULL);
if (status) {
- pr_err("%s(): Failed to resolve dmac from gid."
+ pr_err("%s(): Failed to resolve dmac from gid."
"status = %d\n", __func__, status);
goto av_conf_err;
}
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_ah.h b/drivers/infiniband/hw/ocrdma/ocrdma_ah.h
index 3856dd4..327b888 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_ah.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_ah.h
@@ -50,7 +50,8 @@ enum {
OCRDMA_AH_L3_TYPE_MASK = 0x03,
OCRDMA_AH_L3_TYPE_SHIFT = 0x1D /* 29 bits */
};
-struct ib_ah *ocrdma_create_ah(struct ib_pd *, struct ib_ah_attr *);
+struct ib_ah *ocrdma_create_ah(struct ib_pd *, struct ib_ah_attr *,
+ struct ib_udata *);
int ocrdma_destroy_ah(struct ib_ah *);
int ocrdma_query_ah(struct ib_ah *, struct ib_ah_attr *);
int ocrdma_modify_ah(struct ib_ah *, struct ib_ah_attr *);
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
index a5bfbba..9051193 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
+++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
@@ -738,7 +738,8 @@ int usnic_ib_mmap(struct ib_ucontext *context,
/* In ib callbacks section - Start of stub funcs */
struct ib_ah *usnic_ib_create_ah(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr)
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata)
{
usnic_dbg("\n");
return ERR_PTR(-EPERM);
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
index 0d9d2e6..1eecd8e 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
+++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
@@ -75,7 +75,8 @@ int usnic_ib_dealloc_ucontext(struct ib_ucontext *ibcontext);
int usnic_ib_mmap(struct ib_ucontext *context,
struct vm_area_struct *vma);
struct ib_ah *usnic_ib_create_ah(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr);
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata);
int usnic_ib_destroy_ah(struct ib_ah *ah);
int usnic_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
struct ib_send_wr **bad_wr);
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index 4552be9..64ade0b 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -314,7 +314,8 @@ static int rxe_init_av(struct rxe_dev *rxe, struct ib_ah_attr *attr,
return err;
}
-static struct ib_ah *rxe_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr)
+static struct ib_ah *rxe_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr,
+ struct ib_udata *udata)
{
int err;
struct rxe_dev *rxe = to_rdev(ibpd->device);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 8e90dd2..192d591 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1879,7 +1879,8 @@ struct ib_device {
struct ib_udata *udata);
int (*dealloc_pd)(struct ib_pd *pd);
struct ib_ah * (*create_ah)(struct ib_pd *pd,
- struct ib_ah_attr *ah_attr);
+ struct ib_ah_attr *ah_attr,
+ struct ib_udata *udata);
int (*modify_ah)(struct ib_ah *ah,
struct ib_ah_attr *ah_attr);
int (*query_ah)(struct ib_ah *ah,
--
git-series 0.8.10
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 5/8] ib_uverbs: Add padding to end align ib_uverbs_reg_mr_resp
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
2016-09-16 18:31 ` [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered Knut Omang
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
@ 2016-09-16 18:31 ` Knut Omang
2016-09-20 10:45 ` Yishai Hadas
2016-09-16 18:31 ` [PATCH v2 8/8] ib_uverbs: Support for kernel implementation of XRC Knut Omang
3 siblings, 1 reply; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Knut Omang, Sean Hefty, Hal Rosenstock,
Matan Barak, Sagi Grimberg, Yishai Hadas, Leon Romanovsky,
Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
The ib_uverbs_reg_mr_resp structure was not 64 bit end aligned
as required by the protocol. This causes alignment issues
if a device specific driver needs to transfer extra response
arguments.
Avoid breaking backward compatibility by improving the handling
of the length checking in ib_uverbs_reg_mr to consider the case
where the kernel has been updated, but user space still has
the old length without padding.
Signed-off-by: Knut Omang <knut.omang@oracle.com>
---
drivers/infiniband/core/uverbs_cmd.c | 10 ++++++----
include/uapi/rdma/ib_user_verbs.h | 1 +
2 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index dbc5885..fa8a717 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -967,16 +967,18 @@ ssize_t ib_uverbs_reg_mr(struct ib_uverbs_file *file,
struct ib_pd *pd;
struct ib_mr *mr;
int ret;
+ size_t resp_size = sizeof resp;
- if (out_len < sizeof resp)
- return -ENOSPC;
+ if (out_len < resp_size) {
+ resp_size = out_len;
+ }
if (copy_from_user(&cmd, buf, sizeof cmd))
return -EFAULT;
INIT_UDATA(&udata, buf + sizeof cmd,
(unsigned long) cmd.response + sizeof resp,
- in_len - sizeof cmd, out_len - sizeof resp);
+ in_len - sizeof cmd, out_len - resp_size);
if ((cmd.start & ~PAGE_MASK) != (cmd.hca_va & ~PAGE_MASK))
return -EINVAL;
@@ -1030,7 +1032,7 @@ ssize_t ib_uverbs_reg_mr(struct ib_uverbs_file *file,
resp.mr_handle = uobj->id;
if (copy_to_user((void __user *) (unsigned long) cmd.response,
- &resp, sizeof resp)) {
+ &resp, resp_size)) {
ret = -EFAULT;
goto err_copy;
}
diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h
index 7f035f4..6b8c9c0 100644
--- a/include/uapi/rdma/ib_user_verbs.h
+++ b/include/uapi/rdma/ib_user_verbs.h
@@ -307,6 +307,7 @@ struct ib_uverbs_reg_mr_resp {
__u32 mr_handle;
__u32 lkey;
__u32 rkey;
+ __u32 reserved;
};
struct ib_uverbs_rereg_mr {
--
git-series 0.8.10
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 6/8] ib_uverbs: Avoid vendor specific masking of attributes in query_qp
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
@ 2016-09-16 18:31 ` Knut Omang
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
` (2 subsequent siblings)
3 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Knut Omang, Sean Hefty,
Hal Rosenstock, Yishai Hadas, Matan Barak, Leon Romanovsky,
Sagi Grimberg, Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
This commit removes the implementation and use of the modify_qp_mask
helper function from the generic OFED implementation and into individual
device drivers.
Like with use of the ib_modify_qp_is_ok function it should be up to
each device driver how to handle bits set in the attribute masks.
With the modify_qp_mask function applied in the generic code,
drivers would not see the bits that the user process actually sets.
The restrictions imposed by the filter are also beyond what
is imposed by the Infiniband standard, and would also limit
future drivers or hardware from checking for unsupported or
invalid settings.
---
drivers/infiniband/core/uverbs_cmd.c | 19 ++-----------------
drivers/infiniband/hw/mlx4/qp.c | 19 ++++++++++++++++++-
drivers/infiniband/hw/mlx5/qp.c | 17 +++++++++++++++++
3 files changed, 37 insertions(+), 18 deletions(-)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index fa8a717..7033d10 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -2316,20 +2316,6 @@ out:
return ret ? ret : in_len;
}
-/* Remove ignored fields set in the attribute mask */
-static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
-{
- switch (qp_type) {
- case IB_QPT_XRC_INI:
- return mask & ~(IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER);
- case IB_QPT_XRC_TGT:
- return mask & ~(IB_QP_MAX_QP_RD_ATOMIC | IB_QP_RETRY_CNT |
- IB_QP_RNR_RETRY);
- default:
- return mask;
- }
-}
-
ssize_t ib_uverbs_modify_qp(struct ib_uverbs_file *file,
struct ib_device *ib_dev,
const char __user *buf, int in_len,
@@ -2407,10 +2393,9 @@ ssize_t ib_uverbs_modify_qp(struct ib_uverbs_file *file,
ret = ib_resolve_eth_dmac(qp, attr, &cmd.attr_mask);
if (ret)
goto release_qp;
- ret = qp->device->modify_qp(qp, attr,
- modify_qp_mask(qp->qp_type, cmd.attr_mask), &udata);
+ ret = qp->device->modify_qp(qp, attr, cmd.attr_mask, &udata);
} else {
- ret = ib_modify_qp(qp, attr, modify_qp_mask(qp->qp_type, cmd.attr_mask));
+ ret = ib_modify_qp(qp, attr, cmd.attr_mask);
}
if (ret)
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index 363b1cb..3a5061f 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -2154,6 +2154,20 @@ out:
return err;
}
+/* filter out ignored fields set in the attribute mask */
+static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
+{
+ switch (qp_type) {
+ case IB_QPT_XRC_INI:
+ return mask & ~(IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER);
+ case IB_QPT_XRC_TGT:
+ return mask & ~(IB_QP_MAX_QP_RD_ATOMIC | IB_QP_RETRY_CNT |
+ IB_QP_RNR_RETRY);
+ default:
+ return mask;
+ }
+}
+
static int _mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
int attr_mask, struct ib_udata *udata)
{
@@ -2162,6 +2176,10 @@ static int _mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
enum ib_qp_state cur_state, new_state;
int err = -EINVAL;
int ll;
+
+ if (udata)
+ attr_mask = modify_qp_mask(ibqp->qp_type, attr_mask);
+
mutex_lock(&qp->mutex);
cur_state = attr_mask & IB_QP_CUR_STATE ? attr->cur_qp_state : qp->state;
@@ -3501,4 +3519,3 @@ out:
mutex_unlock(&qp->mutex);
return err;
}
-
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 3a48d9d..5006ab0 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -2737,6 +2737,20 @@ out:
return err;
}
+/* filter out ignored fields set in the attribute mask */
+static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
+{
+ switch (qp_type) {
+ case IB_QPT_XRC_INI:
+ return mask & ~(IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER);
+ case IB_QPT_XRC_TGT:
+ return mask & ~(IB_QP_MAX_QP_RD_ATOMIC | IB_QP_RETRY_CNT |
+ IB_QP_RNR_RETRY);
+ default:
+ return mask;
+ }
+}
+
int mlx5_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
int attr_mask, struct ib_udata *udata)
{
@@ -2751,6 +2765,9 @@ int mlx5_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
if (ibqp->rwq_ind_tbl)
return -ENOSYS;
+ if (udata)
+ attr_mask = modify_qp_mask(ibqp->qp_type, attr_mask);
+
if (unlikely(ibqp->qp_type == IB_QPT_GSI))
return mlx5_ib_gsi_modify_qp(ibqp, attr, attr_mask);
--
git-series 0.8.10
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 6/8] ib_uverbs: Avoid vendor specific masking of attributes in query_qp
@ 2016-09-16 18:31 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Knut Omang, Sean Hefty, Hal Rosenstock,
Yishai Hadas, Matan Barak, Leon Romanovsky, Sagi Grimberg,
Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
This commit removes the implementation and use of the modify_qp_mask
helper function from the generic OFED implementation and into individual
device drivers.
Like with use of the ib_modify_qp_is_ok function it should be up to
each device driver how to handle bits set in the attribute masks.
With the modify_qp_mask function applied in the generic code,
drivers would not see the bits that the user process actually sets.
The restrictions imposed by the filter are also beyond what
is imposed by the Infiniband standard, and would also limit
future drivers or hardware from checking for unsupported or
invalid settings.
---
drivers/infiniband/core/uverbs_cmd.c | 19 ++-----------------
drivers/infiniband/hw/mlx4/qp.c | 19 ++++++++++++++++++-
drivers/infiniband/hw/mlx5/qp.c | 17 +++++++++++++++++
3 files changed, 37 insertions(+), 18 deletions(-)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index fa8a717..7033d10 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -2316,20 +2316,6 @@ out:
return ret ? ret : in_len;
}
-/* Remove ignored fields set in the attribute mask */
-static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
-{
- switch (qp_type) {
- case IB_QPT_XRC_INI:
- return mask & ~(IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER);
- case IB_QPT_XRC_TGT:
- return mask & ~(IB_QP_MAX_QP_RD_ATOMIC | IB_QP_RETRY_CNT |
- IB_QP_RNR_RETRY);
- default:
- return mask;
- }
-}
-
ssize_t ib_uverbs_modify_qp(struct ib_uverbs_file *file,
struct ib_device *ib_dev,
const char __user *buf, int in_len,
@@ -2407,10 +2393,9 @@ ssize_t ib_uverbs_modify_qp(struct ib_uverbs_file *file,
ret = ib_resolve_eth_dmac(qp, attr, &cmd.attr_mask);
if (ret)
goto release_qp;
- ret = qp->device->modify_qp(qp, attr,
- modify_qp_mask(qp->qp_type, cmd.attr_mask), &udata);
+ ret = qp->device->modify_qp(qp, attr, cmd.attr_mask, &udata);
} else {
- ret = ib_modify_qp(qp, attr, modify_qp_mask(qp->qp_type, cmd.attr_mask));
+ ret = ib_modify_qp(qp, attr, cmd.attr_mask);
}
if (ret)
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index 363b1cb..3a5061f 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -2154,6 +2154,20 @@ out:
return err;
}
+/* filter out ignored fields set in the attribute mask */
+static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
+{
+ switch (qp_type) {
+ case IB_QPT_XRC_INI:
+ return mask & ~(IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER);
+ case IB_QPT_XRC_TGT:
+ return mask & ~(IB_QP_MAX_QP_RD_ATOMIC | IB_QP_RETRY_CNT |
+ IB_QP_RNR_RETRY);
+ default:
+ return mask;
+ }
+}
+
static int _mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
int attr_mask, struct ib_udata *udata)
{
@@ -2162,6 +2176,10 @@ static int _mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
enum ib_qp_state cur_state, new_state;
int err = -EINVAL;
int ll;
+
+ if (udata)
+ attr_mask = modify_qp_mask(ibqp->qp_type, attr_mask);
+
mutex_lock(&qp->mutex);
cur_state = attr_mask & IB_QP_CUR_STATE ? attr->cur_qp_state : qp->state;
@@ -3501,4 +3519,3 @@ out:
mutex_unlock(&qp->mutex);
return err;
}
-
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 3a48d9d..5006ab0 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -2737,6 +2737,20 @@ out:
return err;
}
+/* filter out ignored fields set in the attribute mask */
+static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
+{
+ switch (qp_type) {
+ case IB_QPT_XRC_INI:
+ return mask & ~(IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER);
+ case IB_QPT_XRC_TGT:
+ return mask & ~(IB_QP_MAX_QP_RD_ATOMIC | IB_QP_RETRY_CNT |
+ IB_QP_RNR_RETRY);
+ default:
+ return mask;
+ }
+}
+
int mlx5_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
int attr_mask, struct ib_udata *udata)
{
@@ -2751,6 +2765,9 @@ int mlx5_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
if (ibqp->rwq_ind_tbl)
return -ENOSYS;
+ if (udata)
+ attr_mask = modify_qp_mask(ibqp->qp_type, attr_mask);
+
if (unlikely(ibqp->qp_type == IB_QPT_GSI))
return mlx5_ib_gsi_modify_qp(ibqp, attr, attr_mask);
--
git-series 0.8.10
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 7/8] ib_{uverbs/core}: add new ib_create_qp_ex with udata arg
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
@ 2016-09-16 18:31 ` Knut Omang
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
` (2 subsequent siblings)
3 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Knut Omang, Sean Hefty,
Hal Rosenstock, Matan Barak, Mark Bloch, Leon Romanovsky,
Haggai Eran, Sagi Grimberg, Yishai Hadas, Majd Dibbiny,
Eran Ben Elisha, Or Gerlitz, Bart Van Assche, Steve Wise,
Somnath Kotur
Necessary to get device specific arguments through to XRC QPs.
Added new local header file to serve as support interface
between ib_core and ib_uverbs.
Right now there is a lot of duplicate setup code in uverbs_cmd.c
on the ib_uverbs side and verbs.c on the ib_core side. This commit
is a quick fix to have XRC support working, but similar calls
can be added to consolidate the code for other parts of the API.
---
drivers/infiniband/core/core_priv.h | 3 +++
drivers/infiniband/core/uverbs_cmd.c | 2 +-
drivers/infiniband/core/verbs.c | 13 +++++++++++--
3 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h
index 19d499d..0819006 100644
--- a/drivers/infiniband/core/core_priv.h
+++ b/drivers/infiniband/core/core_priv.h
@@ -152,5 +152,8 @@ int ib_nl_handle_set_timeout(struct sk_buff *skb,
struct netlink_callback *cb);
int ib_nl_handle_ip_res_resp(struct sk_buff *skb,
struct netlink_callback *cb);
+struct ib_qp *ib_create_qp_ex(struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr,
+ struct ib_udata *udata);
#endif /* _CORE_PRIV_H */
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 7033d10..7f4470e 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1907,7 +1907,7 @@ static int create_qp(struct ib_uverbs_file *file,
}
if (cmd->qp_type == IB_QPT_XRC_TGT)
- qp = ib_create_qp(pd, &attr);
+ qp = ib_create_qp_ex(pd, &attr, uhw);
else
qp = device->create_qp(pd, &attr, uhw);
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 6efe23d..8f71b00 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -751,8 +751,9 @@ static struct ib_qp *ib_create_xrc_qp(struct ib_qp *qp,
return qp;
}
-struct ib_qp *ib_create_qp(struct ib_pd *pd,
- struct ib_qp_init_attr *qp_init_attr)
+struct ib_qp *ib_create_qp_ex(struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr,
+ struct ib_udata *udata)
{
struct ib_device *device = pd ? pd->device : qp_init_attr->xrcd->device;
struct ib_qp *qp;
@@ -836,6 +837,14 @@ struct ib_qp *ib_create_qp(struct ib_pd *pd,
return qp;
}
+EXPORT_SYMBOL(ib_create_qp_ex);
+
+
+struct ib_qp *ib_create_qp(struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr)
+{
+ return ib_create_qp_ex(pd, qp_init_attr,NULL);
+}
EXPORT_SYMBOL(ib_create_qp);
static const struct {
--
git-series 0.8.10
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 7/8] ib_{uverbs/core}: add new ib_create_qp_ex with udata arg
@ 2016-09-16 18:31 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Knut Omang, Sean Hefty, Hal Rosenstock,
Matan Barak, Mark Bloch, Leon Romanovsky, Haggai Eran,
Sagi Grimberg, Yishai Hadas, Majd Dibbiny, Eran Ben Elisha,
Or Gerlitz, Bart Van Assche, Steve Wise, Somnath Kotur
Necessary to get device specific arguments through to XRC QPs.
Added new local header file to serve as support interface
between ib_core and ib_uverbs.
Right now there is a lot of duplicate setup code in uverbs_cmd.c
on the ib_uverbs side and verbs.c on the ib_core side. This commit
is a quick fix to have XRC support working, but similar calls
can be added to consolidate the code for other parts of the API.
---
drivers/infiniband/core/core_priv.h | 3 +++
drivers/infiniband/core/uverbs_cmd.c | 2 +-
drivers/infiniband/core/verbs.c | 13 +++++++++++--
3 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h
index 19d499d..0819006 100644
--- a/drivers/infiniband/core/core_priv.h
+++ b/drivers/infiniband/core/core_priv.h
@@ -152,5 +152,8 @@ int ib_nl_handle_set_timeout(struct sk_buff *skb,
struct netlink_callback *cb);
int ib_nl_handle_ip_res_resp(struct sk_buff *skb,
struct netlink_callback *cb);
+struct ib_qp *ib_create_qp_ex(struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr,
+ struct ib_udata *udata);
#endif /* _CORE_PRIV_H */
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 7033d10..7f4470e 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1907,7 +1907,7 @@ static int create_qp(struct ib_uverbs_file *file,
}
if (cmd->qp_type == IB_QPT_XRC_TGT)
- qp = ib_create_qp(pd, &attr);
+ qp = ib_create_qp_ex(pd, &attr, uhw);
else
qp = device->create_qp(pd, &attr, uhw);
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 6efe23d..8f71b00 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -751,8 +751,9 @@ static struct ib_qp *ib_create_xrc_qp(struct ib_qp *qp,
return qp;
}
-struct ib_qp *ib_create_qp(struct ib_pd *pd,
- struct ib_qp_init_attr *qp_init_attr)
+struct ib_qp *ib_create_qp_ex(struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr,
+ struct ib_udata *udata)
{
struct ib_device *device = pd ? pd->device : qp_init_attr->xrcd->device;
struct ib_qp *qp;
@@ -836,6 +837,14 @@ struct ib_qp *ib_create_qp(struct ib_pd *pd,
return qp;
}
+EXPORT_SYMBOL(ib_create_qp_ex);
+
+
+struct ib_qp *ib_create_qp(struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr)
+{
+ return ib_create_qp_ex(pd, qp_init_attr,NULL);
+}
EXPORT_SYMBOL(ib_create_qp);
static const struct {
--
git-series 0.8.10
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v2 8/8] ib_uverbs: Support for kernel implementation of XRC
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
` (2 preceding siblings ...)
2016-09-16 18:31 ` [PATCH v2 5/8] ib_uverbs: Add padding to end align ib_uverbs_reg_mr_resp Knut Omang
@ 2016-09-16 18:31 ` Knut Omang
[not found] ` <88ffb8c9407a71069b52e155dde308a36dfaf247.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
3 siblings, 1 reply; 29+ messages in thread
From: Knut Omang @ 2016-09-16 18:31 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Knut Omang, Sean Hefty, Hal Rosenstock,
Matan Barak, Sagi Grimberg, Leon Romanovsky, Yishai Hadas,
Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
Extends the kernel/user space interface for work requests to also provide
the XRC shared receive queue number. Necessary to support
kernel level implementation of user verbs for XRC.
Requires a corresponding libibverbs change to support XRC.
Also fix kernel support for XRC broken by commit
"IB: remove xrc_remote_srq_num from struct ib_send_wr"
which removed a field needed to support kernel side XRC as part of an effort
to trim a work request to sizes dependent of the actual request instead
of the union approach.
With this commit I try to follow the pattern outlined by that cleanup
to also support kernel side XRC. Since XRC attributes are associated with
QP type and are (almost) orthogonal to the type of request, the XRC
specific attribute(s) would have to be applicable to several different
work request type specific subtypes. Since the subtypes have different sizes,
putting the xrc specific attributes at the end would require accessor functions
and to keep explicit track of the size of the subtype used.
The chosen solution is to introduce the xrc specific attributes at the top of
the struct instead, this way type checking is taking care of most issues,
except that extra care is needed at deallocation time. Note that this requires
padding of the xrc specific attributes that matches the size of struct ib_sge,
to avoid that the ALIGN() calls used to ensure that the scatter list of the
work is aligned does not extend beyond the size of the allocates space:
struct ib_xrc_wr
{
<xrc specific part>;
struct ib_send_wr wr;
}
< subtype extensions will still extend ib_send_wr down here >
Signed-off-by: Knut Omang <knut.omang@oracle.com>
---
drivers/infiniband/core/uverbs_cmd.c | 40 +++++++++++++++++++++++------
include/rdma/ib_verbs.h | 12 +++++++++-
include/uapi/rdma/ib_user_verbs.h | 2 +-
3 files changed, 46 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 7f4470e..152e617 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -2471,12 +2471,30 @@ ssize_t ib_uverbs_destroy_qp(struct ib_uverbs_file *file,
return in_len;
}
-static void *alloc_wr(size_t wr_size, __u32 num_sge)
+static void *alloc_wr(struct ib_qp *qp, size_t wr_size, __u32 num_sge)
{
- return kmalloc(ALIGN(wr_size, sizeof (struct ib_sge)) +
- num_sge * sizeof (struct ib_sge), GFP_KERNEL);
+ void *wr;
+ size_t xrc_ext = qp->qp_type == IB_QPT_XRC_INI ?
+ sizeof(struct ib_xrc_wr) - sizeof(struct ib_send_wr) :
+ 0;
+
+ wr = kmalloc(ALIGN(wr_size + xrc_ext, sizeof (struct ib_sge)) +
+ num_sge * sizeof (struct ib_sge), GFP_KERNEL);
+ if (unlikely(!wr))
+ return wr;
+
+ return wr + xrc_ext;
};
+static void free_wr(struct ib_qp *qp, struct ib_send_wr *wr)
+{
+ void *d;
+ if (unlikely(!wr))
+ return;
+ d = qp->qp_type == IB_QPT_XRC_INI ? xrc_wr(wr) : (void *)wr;
+ kfree(d);
+}
+
ssize_t ib_uverbs_post_send(struct ib_uverbs_file *file,
struct ib_device *ib_dev,
const char __user *buf, int in_len,
@@ -2511,6 +2529,7 @@ ssize_t ib_uverbs_post_send(struct ib_uverbs_file *file,
goto out;
is_ud = qp->qp_type == IB_QPT_UD;
+
sg_ind = 0;
last = NULL;
for (i = 0; i < cmd.wr_count; ++i) {
@@ -2536,7 +2555,7 @@ ssize_t ib_uverbs_post_send(struct ib_uverbs_file *file,
}
next_size = sizeof(*ud);
- ud = alloc_wr(next_size, user_wr->num_sge);
+ ud = alloc_wr(qp, next_size, user_wr->num_sge);
if (!ud) {
ret = -ENOMEM;
goto out_put;
@@ -2558,7 +2577,7 @@ ssize_t ib_uverbs_post_send(struct ib_uverbs_file *file,
struct ib_rdma_wr *rdma;
next_size = sizeof(*rdma);
- rdma = alloc_wr(next_size, user_wr->num_sge);
+ rdma = alloc_wr(qp, next_size, user_wr->num_sge);
if (!rdma) {
ret = -ENOMEM;
goto out_put;
@@ -2573,7 +2592,7 @@ ssize_t ib_uverbs_post_send(struct ib_uverbs_file *file,
struct ib_atomic_wr *atomic;
next_size = sizeof(*atomic);
- atomic = alloc_wr(next_size, user_wr->num_sge);
+ atomic = alloc_wr(qp, next_size, user_wr->num_sge);
if (!atomic) {
ret = -ENOMEM;
goto out_put;
@@ -2589,7 +2608,7 @@ ssize_t ib_uverbs_post_send(struct ib_uverbs_file *file,
user_wr->opcode == IB_WR_SEND_WITH_IMM ||
user_wr->opcode == IB_WR_SEND_WITH_INV) {
next_size = sizeof(*next);
- next = alloc_wr(next_size, user_wr->num_sge);
+ next = alloc_wr(qp, next_size, user_wr->num_sge);
if (!next) {
ret = -ENOMEM;
goto out_put;
@@ -2607,6 +2626,11 @@ ssize_t ib_uverbs_post_send(struct ib_uverbs_file *file,
next->ex.invalidate_rkey = user_wr->ex.invalidate_rkey;
}
+ if (qp->qp_type == IB_QPT_XRC_INI) {
+ struct ib_xrc_wr *xrc = xrc_wr(next);
+ xrc->remote_srqn = user_wr->xrc_remote_srq_num;
+ }
+
if (!last)
wr = next;
else
@@ -2655,7 +2679,7 @@ out_put:
if (is_ud && ud_wr(wr)->ah)
put_ah_read(ud_wr(wr)->ah);
next = wr->next;
- kfree(wr);
+ free_wr(qp, wr);
wr = next;
}
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 192d591..093d68e 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1276,6 +1276,18 @@ static inline struct ib_sig_handover_wr *sig_handover_wr(struct ib_send_wr *wr)
return container_of(wr, struct ib_sig_handover_wr, wr);
}
+struct ib_xrc_wr {
+ u32 remote_srqn;
+ u32 reserved1;
+ u64 reserved2;
+ struct ib_send_wr wr;
+};
+
+static inline struct ib_xrc_wr *xrc_wr(struct ib_send_wr *wr)
+{
+ return container_of(wr, struct ib_xrc_wr, wr);
+}
+
struct ib_recv_wr {
struct ib_recv_wr *next;
union {
diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h
index 6b8c9c0..db74a5e 100644
--- a/include/uapi/rdma/ib_user_verbs.h
+++ b/include/uapi/rdma/ib_user_verbs.h
@@ -725,6 +725,8 @@ struct ib_uverbs_send_wr {
__u32 reserved;
} ud;
} wr;
+ __u32 xrc_remote_srq_num;
+ __u32 reserved;
};
struct ib_uverbs_post_send {
--
git-series 0.8.10
^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH v2 8/8] ib_uverbs: Support for kernel implementation of XRC
2016-09-16 18:31 ` [PATCH v2 8/8] ib_uverbs: Support for kernel implementation of XRC Knut Omang
@ 2016-09-16 19:31 ` Jason Gunthorpe
0 siblings, 0 replies; 29+ messages in thread
From: Jason Gunthorpe @ 2016-09-16 19:31 UTC (permalink / raw)
To: Knut Omang
Cc: Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Sean Hefty, Hal Rosenstock,
Matan Barak, Sagi Grimberg, Leon Romanovsky, Yishai Hadas,
Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
On Fri, Sep 16, 2016 at 08:31:19PM +0200, Knut Omang wrote:
> +++ b/include/uapi/rdma/ib_user_verbs.h
> @@ -725,6 +725,8 @@ struct ib_uverbs_send_wr {
> __u32 reserved;
> } ud;
> } wr;
> + __u32 xrc_remote_srq_num;
> + __u32 reserved;
> };
You still need to discuss why this is OK...
Are you doing all this just to be able to use the existing post_send user
space path in the common code? Is there a kernel XRC user planned?
Another approach would be to implement post_send in your driver and
not rely on this common code path.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 8/8] ib_uverbs: Support for kernel implementation of XRC
@ 2016-09-16 19:31 ` Jason Gunthorpe
0 siblings, 0 replies; 29+ messages in thread
From: Jason Gunthorpe @ 2016-09-16 19:31 UTC (permalink / raw)
To: Knut Omang
Cc: Doug Ledford, linux-rdma, linux-kernel, Sean Hefty,
Hal Rosenstock, Matan Barak, Sagi Grimberg, Leon Romanovsky,
Yishai Hadas, Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
On Fri, Sep 16, 2016 at 08:31:19PM +0200, Knut Omang wrote:
> +++ b/include/uapi/rdma/ib_user_verbs.h
> @@ -725,6 +725,8 @@ struct ib_uverbs_send_wr {
> __u32 reserved;
> } ud;
> } wr;
> + __u32 xrc_remote_srq_num;
> + __u32 reserved;
> };
You still need to discuss why this is OK...
Are you doing all this just to be able to use the existing post_send user
space path in the common code? Is there a kernel XRC user planned?
Another approach would be to implement post_send in your driver and
not rely on this common code path.
Jason
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 8/8] ib_uverbs: Support for kernel implementation of XRC
2016-09-16 19:31 ` Jason Gunthorpe
@ 2016-09-16 19:49 ` Knut Omang
-1 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 19:49 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Sean Hefty, Hal Rosenstock,
Matan Barak, Sagi Grimberg, Leon Romanovsky, Yishai Hadas,
Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
On Fri, 2016-09-16 at 13:31 -0600, Jason Gunthorpe wrote:
> On Fri, Sep 16, 2016 at 08:31:19PM +0200, Knut Omang wrote:
> > +++ b/include/uapi/rdma/ib_user_verbs.h
> > @@ -725,6 +725,8 @@ struct ib_uverbs_send_wr {
> > __u32 reserved;
> > } ud;
> > } wr;
> > + __u32 xrc_remote_srq_num;
> > + __u32 reserved;
> > };
>
> You still need to discuss why this is OK...
>
> Are you doing all this just to be able to use the existing post_send user
> space path in the common code? Is there a kernel XRC user planned?
Yes, with this patch, the SIF user level provider library is able to
use either kernel verbs or user verbs on a per QP basis. This is a
very useful capability that the verbs API neatly lends itself almost
completely to, except for this particular missing piece.
> Another approach would be to implement post_send in your driver and
> not rely on this common code path.
I agree this would be possible, but that would have been
to implement a generic feature which seems a natural completion
of what is already in there in provider specific code, something
I would have thought from other discussions here is considered bad
practice?
Thanks,
Knut
>
> Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 8/8] ib_uverbs: Support for kernel implementation of XRC
@ 2016-09-16 19:49 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 19:49 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Doug Ledford, linux-rdma, linux-kernel, Sean Hefty,
Hal Rosenstock, Matan Barak, Sagi Grimberg, Leon Romanovsky,
Yishai Hadas, Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
On Fri, 2016-09-16 at 13:31 -0600, Jason Gunthorpe wrote:
> On Fri, Sep 16, 2016 at 08:31:19PM +0200, Knut Omang wrote:
> > +++ b/include/uapi/rdma/ib_user_verbs.h
> > @@ -725,6 +725,8 @@ struct ib_uverbs_send_wr {
> > __u32 reserved;
> > } ud;
> > } wr;
> > + __u32 xrc_remote_srq_num;
> > + __u32 reserved;
> > };
>
> You still need to discuss why this is OK...
>
> Are you doing all this just to be able to use the existing post_send user
> space path in the common code? Is there a kernel XRC user planned?
Yes, with this patch, the SIF user level provider library is able to
use either kernel verbs or user verbs on a per QP basis. This is a
very useful capability that the verbs API neatly lends itself almost
completely to, except for this particular missing piece.
> Another approach would be to implement post_send in your driver and
> not rely on this common code path.
I agree this would be possible, but that would have been
to implement a generic feature which seems a natural completion
of what is already in there in provider specific code, something
I would have thought from other discussions here is considered bad
practice?
Thanks,
Knut
>
> Jason
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered
2016-09-16 18:31 ` [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered Knut Omang
@ 2016-09-16 20:28 ` Santosh Shilimkar
0 siblings, 0 replies; 29+ messages in thread
From: Santosh Shilimkar @ 2016-09-16 20:28 UTC (permalink / raw)
To: Knut Omang, Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Sean Hefty, Hal Rosenstock,
Ira Weiny, Sagi Grimberg, Christoph Hellwig, Bart Van Assche,
Dag Moxnes, Mark Bloch, Dean Luick
On 9/16/2016 11:31 AM, Knut Omang wrote:
> From: Dag Moxnes <dag.moxnes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>
> The process_mad function is an optional IB driver entry point
> allows a driver to intercept or modify MAD traffic.
>
> This fix allows MAD traffic to flow down to the device also
> when MAD traffic is completely handled by the device and
> no process_mad function is provided.
>
> SIF, the new Oracle Infiniband HCA, is the first HCA
> where the device itself makes all decision wrt MAD processing.
> Up till now devices either supports MAD, and do then
> implement the process_mad entry point, or do not
> support MAD at all, and then do not implement process_mad.
>
> SIF introduces a 3rd case: Supports MAD
> but do not terminate any MAD requests in the driver.
> This case is not handled well by the current code.
>
> The problem is that the handle_outgoing_dr_smp function
> has an implicit assumption that some packets are handled
> by the process_mad function itself.
>
> There is no way to provide return values from the process_mad
> function that ensures that packets are always forwarded to the device,
> so the only viable solution without breaking the API
> seems to be to not implement process_mad.
No SOBs ?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered
@ 2016-09-16 20:28 ` Santosh Shilimkar
0 siblings, 0 replies; 29+ messages in thread
From: Santosh Shilimkar @ 2016-09-16 20:28 UTC (permalink / raw)
To: Knut Omang, Doug Ledford
Cc: linux-rdma, linux-kernel, Sean Hefty, Hal Rosenstock, Ira Weiny,
Sagi Grimberg, Christoph Hellwig, Bart Van Assche, Dag Moxnes,
Mark Bloch, Dean Luick
On 9/16/2016 11:31 AM, Knut Omang wrote:
> From: Dag Moxnes <dag.moxnes@oracle.com>
>
> The process_mad function is an optional IB driver entry point
> allows a driver to intercept or modify MAD traffic.
>
> This fix allows MAD traffic to flow down to the device also
> when MAD traffic is completely handled by the device and
> no process_mad function is provided.
>
> SIF, the new Oracle Infiniband HCA, is the first HCA
> where the device itself makes all decision wrt MAD processing.
> Up till now devices either supports MAD, and do then
> implement the process_mad entry point, or do not
> support MAD at all, and then do not implement process_mad.
>
> SIF introduces a 3rd case: Supports MAD
> but do not terminate any MAD requests in the driver.
> This case is not handled well by the current code.
>
> The problem is that the handle_outgoing_dr_smp function
> has an implicit assumption that some packets are handled
> by the process_mad function itself.
>
> There is no way to provide return values from the process_mad
> function that ensures that packets are always forwarded to the device,
> so the only viable solution without breaking the API
> seems to be to not implement process_mad.
No SOBs ?
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 0/8] SIF related verbs patches
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
@ 2016-09-16 20:30 ` Santosh Shilimkar
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
` (2 subsequent siblings)
3 siblings, 0 replies; 29+ messages in thread
From: Santosh Shilimkar @ 2016-09-16 20:30 UTC (permalink / raw)
To: Knut Omang, Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA
Hi Knut,
On 9/16/2016 11:31 AM, Knut Omang wrote:
> This patch set contains the changes and bug fixes necessary
> to support Oracle's new Infiniband HCA (SIF) from the kernel side.
>
> The exception is patch 3 which is a useful consolidation and simplification
> made possible by patch 2 and the replacement of the dma_attr type with
> a plain bitmask.
>
> The changes to other rdma drivers are just trivial consequences of
> the extended umem_get and create_ah calls and should hopefully be
> uncontroversial.
>
> Patches 4-7 are related to corresponding libibverbs patches.
> These make minor changes (fixing end alignment) to some uverbs
> data structures, but these are carefully made to be backward
> binary compatible between user and kernel for existing provider
> libraries.
>
You have missed SOBs on few patches accidentally. You might want to
fix that for next post :-)
regards,
Santosh
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 0/8] SIF related verbs patches
@ 2016-09-16 20:30 ` Santosh Shilimkar
0 siblings, 0 replies; 29+ messages in thread
From: Santosh Shilimkar @ 2016-09-16 20:30 UTC (permalink / raw)
To: Knut Omang, Doug Ledford; +Cc: linux-rdma, linux-kernel
Hi Knut,
On 9/16/2016 11:31 AM, Knut Omang wrote:
> This patch set contains the changes and bug fixes necessary
> to support Oracle's new Infiniband HCA (SIF) from the kernel side.
>
> The exception is patch 3 which is a useful consolidation and simplification
> made possible by patch 2 and the replacement of the dma_attr type with
> a plain bitmask.
>
> The changes to other rdma drivers are just trivial consequences of
> the extended umem_get and create_ah calls and should hopefully be
> uncontroversial.
>
> Patches 4-7 are related to corresponding libibverbs patches.
> These make minor changes (fixing end alignment) to some uverbs
> data structures, but these are carefully made to be backward
> binary compatible between user and kernel for existing provider
> libraries.
>
You have missed SOBs on few patches accidentally. You might want to
fix that for next post :-)
regards,
Santosh
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered
2016-09-16 20:28 ` Santosh Shilimkar
@ 2016-09-16 20:40 ` Knut Omang
-1 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 20:40 UTC (permalink / raw)
To: Santosh Shilimkar, Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Sean Hefty, Hal Rosenstock,
Ira Weiny, Sagi Grimberg, Christoph Hellwig, Bart Van Assche,
Dag Moxnes, Mark Bloch, Dean Luick
On Fri, 2016-09-16 at 13:28 -0700, Santosh Shilimkar wrote:
> On 9/16/2016 11:31 AM, Knut Omang wrote:
> > From: Dag Moxnes <dag.moxnes-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> >
> > The process_mad function is an optional IB driver entry point
> > allows a driver to intercept or modify MAD traffic.
> >
> > This fix allows MAD traffic to flow down to the device also
> > when MAD traffic is completely handled by the device and
> > no process_mad function is provided.
> >
> > SIF, the new Oracle Infiniband HCA, is the first HCA
> > where the device itself makes all decision wrt MAD processing.
> > Up till now devices either supports MAD, and do then
> > implement the process_mad entry point, or do not
> > support MAD at all, and then do not implement process_mad.
> >
> > SIF introduces a 3rd case: Supports MAD
> > but do not terminate any MAD requests in the driver.
> > This case is not handled well by the current code.
> >
> > The problem is that the handle_outgoing_dr_smp function
> > has an implicit assumption that some packets are handled
> > by the process_mad function itself.
> >
> > There is no way to provide return values from the process_mad
> > function that ensures that packets are always forwarded to the device,
> > so the only viable solution without breaking the API
> > seems to be to not implement process_mad.
>
> No SOBs ?
is unfortunately recurring for several of the patches due to a missing -ns to format-patch.
Will fix,
Thanks,
Knut
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered
@ 2016-09-16 20:40 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 20:40 UTC (permalink / raw)
To: Santosh Shilimkar, Doug Ledford
Cc: linux-rdma, linux-kernel, Sean Hefty, Hal Rosenstock, Ira Weiny,
Sagi Grimberg, Christoph Hellwig, Bart Van Assche, Dag Moxnes,
Mark Bloch, Dean Luick
On Fri, 2016-09-16 at 13:28 -0700, Santosh Shilimkar wrote:
> On 9/16/2016 11:31 AM, Knut Omang wrote:
> > From: Dag Moxnes <dag.moxnes@oracle.com>
> >
> > The process_mad function is an optional IB driver entry point
> > allows a driver to intercept or modify MAD traffic.
> >
> > This fix allows MAD traffic to flow down to the device also
> > when MAD traffic is completely handled by the device and
> > no process_mad function is provided.
> >
> > SIF, the new Oracle Infiniband HCA, is the first HCA
> > where the device itself makes all decision wrt MAD processing.
> > Up till now devices either supports MAD, and do then
> > implement the process_mad entry point, or do not
> > support MAD at all, and then do not implement process_mad.
> >
> > SIF introduces a 3rd case: Supports MAD
> > but do not terminate any MAD requests in the driver.
> > This case is not handled well by the current code.
> >
> > The problem is that the handle_outgoing_dr_smp function
> > has an implicit assumption that some packets are handled
> > by the process_mad function itself.
> >
> > There is no way to provide return values from the process_mad
> > function that ensures that packets are always forwarded to the device,
> > so the only viable solution without breaking the API
> > seems to be to not implement process_mad.
>
> No SOBs ?
is unfortunately recurring for several of the patches due to a missing -ns to format-patch.
Will fix,
Thanks,
Knut
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 0/8] SIF related verbs patches
2016-09-16 20:30 ` Santosh Shilimkar
(?)
@ 2016-09-16 20:42 ` Knut Omang
-1 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-16 20:42 UTC (permalink / raw)
To: Santosh Shilimkar, Doug Ledford; +Cc: linux-rdma, linux-kernel
On Fri, 2016-09-16 at 13:30 -0700, Santosh Shilimkar wrote:
> Hi Knut,
>
> On 9/16/2016 11:31 AM, Knut Omang wrote:
> > This patch set contains the changes and bug fixes necessary
> > to support Oracle's new Infiniband HCA (SIF) from the kernel side.
> >
> > The exception is patch 3 which is a useful consolidation and simplification
> > made possible by patch 2 and the replacement of the dma_attr type with
> > a plain bitmask.
> >
> > The changes to other rdma drivers are just trivial consequences of
> > the extended umem_get and create_ah calls and should hopefully be
> > uncontroversial.
> >
> > Patches 4-7 are related to corresponding libibverbs patches.
> > These make minor changes (fixing end alignment) to some uverbs
> > data structures, but these are carefully made to be backward
> > binary compatible between user and kernel for existing provider
> > libraries.
> >
> You have missed SOBs on few patches accidentally. You might want to
> fix that for next post :-)
>
> regards,
> Santosh
An unfortunate side effect of getting going with the otherwise excellent
git series tools that the -ns option I have been using to 'git format-patch'
git lost.
I have fixed the series for the next post,
Thanks for spotting it,
Knut
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 5/8] ib_uverbs: Add padding to end align ib_uverbs_reg_mr_resp
2016-09-16 18:31 ` [PATCH v2 5/8] ib_uverbs: Add padding to end align ib_uverbs_reg_mr_resp Knut Omang
@ 2016-09-20 10:45 ` Yishai Hadas
2016-09-20 11:02 ` Knut Omang
0 siblings, 1 reply; 29+ messages in thread
From: Yishai Hadas @ 2016-09-20 10:45 UTC (permalink / raw)
To: Knut Omang
Cc: Doug Ledford, linux-rdma, linux-kernel, Sean Hefty,
Hal Rosenstock, Matan Barak, Sagi Grimberg, Yishai Hadas,
Leon Romanovsky, Majd Dibbiny, Eran Ben Elisha, Or Gerlitz
On 9/16/2016 9:31 PM, Knut Omang wrote:
> The ib_uverbs_reg_mr_resp structure was not 64 bit end aligned
> as required by the protocol. This causes alignment issues
> if a device specific driver needs to transfer extra response
> arguments.
>
> Avoid breaking backward compatibility by improving the handling
> of the length checking in ib_uverbs_reg_mr to consider the case
> where the kernel has been updated, but user space still has
> the old length without padding.
>
> Signed-off-by: Knut Omang <knut.omang@oracle.com>
> ---
> drivers/infiniband/core/uverbs_cmd.c | 10 ++++++----
> include/uapi/rdma/ib_user_verbs.h | 1 +
> 2 files changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
> index dbc5885..fa8a717 100644
> --- a/drivers/infiniband/core/uverbs_cmd.c
> +++ b/drivers/infiniband/core/uverbs_cmd.c
> @@ -967,16 +967,18 @@ ssize_t ib_uverbs_reg_mr(struct ib_uverbs_file *file,
> struct ib_pd *pd;
> struct ib_mr *mr;
> int ret;
> + size_t resp_size = sizeof resp;
>
> - if (out_len < sizeof resp)
> - return -ENOSPC;
> + if (out_len < resp_size) {
> + resp_size = out_len;
> + }
You don't preserve the minimum response size error checking in your code
(i.e -ENOSPC). We can expect an error if outlen is less than the
offset of rkey in ib_uverbs_reg_mr_resp.
Please note that below call to copy_to_user will succeed as it copies
now based on resp_size and won't return the basic mandatory output.
> if (copy_from_user(&cmd, buf, sizeof cmd))
> return -EFAULT;
>
> INIT_UDATA(&udata, buf + sizeof cmd,
> (unsigned long) cmd.response + sizeof resp,
> - in_len - sizeof cmd, out_len - sizeof resp);
> + in_len - sizeof cmd, out_len - resp_size);
>
> if ((cmd.start & ~PAGE_MASK) != (cmd.hca_va & ~PAGE_MASK))
> return -EINVAL;
> @@ -1030,7 +1032,7 @@ ssize_t ib_uverbs_reg_mr(struct ib_uverbs_file *file,
> resp.mr_handle = uobj->id;
>
> if (copy_to_user((void __user *) (unsigned long) cmd.response,
> - &resp, sizeof resp)) {
> + &resp, resp_size)) {
> ret = -EFAULT;
> goto err_copy;
> }
> diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h
> index 7f035f4..6b8c9c0 100644
> --- a/include/uapi/rdma/ib_user_verbs.h
> +++ b/include/uapi/rdma/ib_user_verbs.h
> @@ -307,6 +307,7 @@ struct ib_uverbs_reg_mr_resp {
> __u32 mr_handle;
> __u32 lkey;
> __u32 rkey;
> + __u32 reserved;
> };
>
> struct ib_uverbs_rereg_mr {
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 5/8] ib_uverbs: Add padding to end align ib_uverbs_reg_mr_resp
2016-09-20 10:45 ` Yishai Hadas
@ 2016-09-20 11:02 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2016-09-20 11:02 UTC (permalink / raw)
To: Yishai Hadas
Cc: Doug Ledford, linux-rdma, linux-kernel, Sean Hefty, Hal Rosenstock
On Tue, 2016-09-20 at 13:45 +0300, Yishai Hadas wrote:
> On 9/16/2016 9:31 PM, Knut Omang wrote:
> >
> > The ib_uverbs_reg_mr_resp structure was not 64 bit end aligned
> > as required by the protocol. This causes alignment issues
> > if a device specific driver needs to transfer extra response
> > arguments.
> >
> > Avoid breaking backward compatibility by improving the handling
> > of the length checking in ib_uverbs_reg_mr to consider the case
> > where the kernel has been updated, but user space still has
> > the old length without padding.
> >
> > Signed-off-by: Knut Omang <knut.omang@oracle.com>
> > ---
> > drivers/infiniband/core/uverbs_cmd.c | 10 ++++++----
> > include/uapi/rdma/ib_user_verbs.h | 1 +
> > 2 files changed, 7 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
> > index dbc5885..fa8a717 100644
> > --- a/drivers/infiniband/core/uverbs_cmd.c
> > +++ b/drivers/infiniband/core/uverbs_cmd.c
> > @@ -967,16 +967,18 @@ ssize_t ib_uverbs_reg_mr(struct ib_uverbs_file *file,
> > struct ib_pd *pd;
> > struct ib_mr *mr;
> > int ret;
> > + size_t resp_size = sizeof resp;
> >
> > - if (out_len < sizeof resp)
> > - return -ENOSPC;
> > + if (out_len < resp_size) {
> > + resp_size = out_len;
> > + }
> You don't preserve the minimum response size error checking in your code
> (i.e -ENOSPC). We can expect an error if outlen is less than the
> offset of rkey in ib_uverbs_reg_mr_resp.
I agree, the minimal size needs to be the minimal supported size,
in particular since the memory for cmd may contain arbitrary stack content.
Good catch!
> Please note that below call to copy_to_user will succeed as it copies
> now based on resp_size and won't return the basic mandatory output.
Get it - will fix,
Thanks,
Knut
>
> >
> > if (copy_from_user(&cmd, buf, sizeof cmd))
> > return -EFAULT;
> >
> > INIT_UDATA(&udata, buf + sizeof cmd,
> > (unsigned long) cmd.response + sizeof resp,
> > - in_len - sizeof cmd, out_len - sizeof resp);
> > + in_len - sizeof cmd, out_len - resp_size);
> >
> > if ((cmd.start & ~PAGE_MASK) != (cmd.hca_va & ~PAGE_MASK))
> > return -EINVAL;
> > @@ -1030,7 +1032,7 @@ ssize_t ib_uverbs_reg_mr(struct ib_uverbs_file *file,
> > resp.mr_handle = uobj->id;
> >
> > if (copy_to_user((void __user *) (unsigned long) cmd.response,
> > - &resp, sizeof resp)) {
> > + &resp, resp_size)) {
> > ret = -EFAULT;
> > goto err_copy;
> > }
> > diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h
> > index 7f035f4..6b8c9c0 100644
> > --- a/include/uapi/rdma/ib_user_verbs.h
> > +++ b/include/uapi/rdma/ib_user_verbs.h
> > @@ -307,6 +307,7 @@ struct ib_uverbs_reg_mr_resp {
> > __u32 mr_handle;
> > __u32 lkey;
> > __u32 rkey;
> > + __u32 reserved;
> > };
> >
> > struct ib_uverbs_rereg_mr {
> >
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 2/8] ib_umem: Add a new, more generic ib_umem_get_attrs
2016-09-16 18:31 ` Knut Omang
@ 2017-04-05 15:20 ` Knut Omang
-1 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2017-04-05 15:20 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Sean Hefty, Hal Rosenstock,
Robin Murphy, Hans-Christian Noren Egtvedt, Vineet Gupta,
Joerg Roedel, Gerald Schaefer, Krzysztof Kozlowski, Dave Hansen
Dough,
This patch and the next in this set is IMHO together a generic enhancement/cleanup,
is it something you would consider to take just as that?
I am looking at suggesting a simplified, less target specific version of this set
to get it off my chest,
Thanks,
Knut
On Fri, 2016-09-16 at 20:31 +0200, Knut Omang wrote:
> This call allows a full range of DMA attributes and also
> DMA direction to be supplied and is just a refactor of the old ib_umem_get.
> Reimplement ib_umem_get using the new generic call,
> now a trivial implementation.
> ---
> drivers/infiniband/core/umem.c | 23 +++++++++++++++--------
> include/rdma/ib_umem.h | 15 ++++++++++++++-
> 2 files changed, 29 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
> index c68746c..699a0f7 100644
> --- a/drivers/infiniband/core/umem.c
> +++ b/drivers/infiniband/core/umem.c
> @@ -52,7 +52,7 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
> if (umem->nmap > 0)
> ib_dma_unmap_sg(dev, umem->sg_head.sgl,
> umem->nmap,
> - DMA_BIDIRECTIONAL);
> + umem->dir);
>
> for_each_sg(umem->sg_head.sgl, sg, umem->npages, i) {
>
> @@ -82,6 +82,17 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
> struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> size_t size, int access, int dmasync)
> {
> + unsigned long dma_attrs = 0;
> + if (dmasync)
> + dma_attrs |= DMA_ATTR_WRITE_BARRIER;
> + return ib_umem_get_attrs(context, addr, size, access, DMA_BIDIRECTIONAL, dma_attrs);
> +}
> +EXPORT_SYMBOL(ib_umem_get);
> +
> +struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
> + size_t size, int access, enum dma_data_direction dir,
> + unsigned long dma_attrs)
> +{
> struct ib_umem *umem;
> struct page **page_list;
> struct vm_area_struct **vma_list;
> @@ -91,16 +102,11 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> unsigned long npages;
> int ret;
> int i;
> - unsigned long dma_attrs = 0;
> struct scatterlist *sg, *sg_list_start;
> int need_release = 0;
>
> - if (dmasync)
> - dma_attrs |= DMA_ATTR_WRITE_BARRIER;
> -
> if (!size)
> return ERR_PTR(-EINVAL);
> -
> /*
> * If the combination of the addr and size requested for this memory
> * region causes an integer overflow, return error.
> @@ -121,6 +127,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> umem->address = addr;
> umem->page_size = PAGE_SIZE;
> umem->pid = get_task_pid(current, PIDTYPE_PID);
> + umem->dir = dir;
> /*
> * We ask for writable memory if any of the following
> * access flags are set. "Local write" and "remote write"
> @@ -213,7 +220,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> umem->nmap = ib_dma_map_sg_attrs(context->device,
> umem->sg_head.sgl,
> umem->npages,
> - DMA_BIDIRECTIONAL,
> + dir,
> dma_attrs);
>
> if (umem->nmap <= 0) {
> @@ -239,7 +246,7 @@ out:
>
> return ret < 0 ? ERR_PTR(ret) : umem;
> }
> -EXPORT_SYMBOL(ib_umem_get);
> +EXPORT_SYMBOL(ib_umem_get_attrs);
>
> static void ib_umem_account(struct work_struct *work)
> {
> diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
> index 2d83cfd..2876679 100644
> --- a/include/rdma/ib_umem.h
> +++ b/include/rdma/ib_umem.h
> @@ -36,6 +36,7 @@
> #include <linux/list.h>
> #include <linux/scatterlist.h>
> #include <linux/workqueue.h>
> +#include <linux/dma-direction.h>
>
> struct ib_ucontext;
> struct ib_umem_odp;
> @@ -47,6 +48,7 @@ struct ib_umem {
> int page_size;
> int writable;
> int hugetlb;
> + enum dma_data_direction dir;
> struct work_struct work;
> struct pid *pid;
> struct mm_struct *mm;
> @@ -81,9 +83,12 @@ static inline size_t ib_umem_num_pages(struct ib_umem *umem)
> }
>
> #ifdef CONFIG_INFINIBAND_USER_MEM
> -
> struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> size_t size, int access, int dmasync);
> +struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
> + size_t size, int access,
> + enum dma_data_direction dir,
> + unsigned long dma_attrs);
> void ib_umem_release(struct ib_umem *umem);
> int ib_umem_page_count(struct ib_umem *umem);
> int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
> @@ -98,6 +103,14 @@ static inline struct ib_umem *ib_umem_get(struct ib_ucontext *context,
> int access, int dmasync) {
> return ERR_PTR(-EINVAL);
> }
> +static inline struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context,
> + unsigned long addr,
> + size_t size, int access,
> + enum dma_data_direction dir,
> + unsigned long dma_attrs)
> +{
> + return ERR_PTR(-EINVAL);
> +}
> static inline void ib_umem_release(struct ib_umem *umem) { }
> static inline int ib_umem_page_count(struct ib_umem *umem) { return 0; }
> static inline int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v2 2/8] ib_umem: Add a new, more generic ib_umem_get_attrs
@ 2017-04-05 15:20 ` Knut Omang
0 siblings, 0 replies; 29+ messages in thread
From: Knut Omang @ 2017-04-05 15:20 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Sean Hefty, Hal Rosenstock,
Robin Murphy, Hans-Christian Noren Egtvedt, Vineet Gupta,
Joerg Roedel, Gerald Schaefer, Krzysztof Kozlowski, Dave Hansen
Dough,
This patch and the next in this set is IMHO together a generic enhancement/cleanup,
is it something you would consider to take just as that?
I am looking at suggesting a simplified, less target specific version of this set
to get it off my chest,
Thanks,
Knut
On Fri, 2016-09-16 at 20:31 +0200, Knut Omang wrote:
> This call allows a full range of DMA attributes and also
> DMA direction to be supplied and is just a refactor of the old ib_umem_get.
> Reimplement ib_umem_get using the new generic call,
> now a trivial implementation.
> ---
> drivers/infiniband/core/umem.c | 23 +++++++++++++++--------
> include/rdma/ib_umem.h | 15 ++++++++++++++-
> 2 files changed, 29 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
> index c68746c..699a0f7 100644
> --- a/drivers/infiniband/core/umem.c
> +++ b/drivers/infiniband/core/umem.c
> @@ -52,7 +52,7 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
> if (umem->nmap > 0)
> ib_dma_unmap_sg(dev, umem->sg_head.sgl,
> umem->nmap,
> - DMA_BIDIRECTIONAL);
> + umem->dir);
>
> for_each_sg(umem->sg_head.sgl, sg, umem->npages, i) {
>
> @@ -82,6 +82,17 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
> struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> size_t size, int access, int dmasync)
> {
> + unsigned long dma_attrs = 0;
> + if (dmasync)
> + dma_attrs |= DMA_ATTR_WRITE_BARRIER;
> + return ib_umem_get_attrs(context, addr, size, access, DMA_BIDIRECTIONAL, dma_attrs);
> +}
> +EXPORT_SYMBOL(ib_umem_get);
> +
> +struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
> + size_t size, int access, enum dma_data_direction dir,
> + unsigned long dma_attrs)
> +{
> struct ib_umem *umem;
> struct page **page_list;
> struct vm_area_struct **vma_list;
> @@ -91,16 +102,11 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> unsigned long npages;
> int ret;
> int i;
> - unsigned long dma_attrs = 0;
> struct scatterlist *sg, *sg_list_start;
> int need_release = 0;
>
> - if (dmasync)
> - dma_attrs |= DMA_ATTR_WRITE_BARRIER;
> -
> if (!size)
> return ERR_PTR(-EINVAL);
> -
> /*
> * If the combination of the addr and size requested for this memory
> * region causes an integer overflow, return error.
> @@ -121,6 +127,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> umem->address = addr;
> umem->page_size = PAGE_SIZE;
> umem->pid = get_task_pid(current, PIDTYPE_PID);
> + umem->dir = dir;
> /*
> * We ask for writable memory if any of the following
> * access flags are set. "Local write" and "remote write"
> @@ -213,7 +220,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> umem->nmap = ib_dma_map_sg_attrs(context->device,
> umem->sg_head.sgl,
> umem->npages,
> - DMA_BIDIRECTIONAL,
> + dir,
> dma_attrs);
>
> if (umem->nmap <= 0) {
> @@ -239,7 +246,7 @@ out:
>
> return ret < 0 ? ERR_PTR(ret) : umem;
> }
> -EXPORT_SYMBOL(ib_umem_get);
> +EXPORT_SYMBOL(ib_umem_get_attrs);
>
> static void ib_umem_account(struct work_struct *work)
> {
> diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
> index 2d83cfd..2876679 100644
> --- a/include/rdma/ib_umem.h
> +++ b/include/rdma/ib_umem.h
> @@ -36,6 +36,7 @@
> #include <linux/list.h>
> #include <linux/scatterlist.h>
> #include <linux/workqueue.h>
> +#include <linux/dma-direction.h>
>
> struct ib_ucontext;
> struct ib_umem_odp;
> @@ -47,6 +48,7 @@ struct ib_umem {
> int page_size;
> int writable;
> int hugetlb;
> + enum dma_data_direction dir;
> struct work_struct work;
> struct pid *pid;
> struct mm_struct *mm;
> @@ -81,9 +83,12 @@ static inline size_t ib_umem_num_pages(struct ib_umem *umem)
> }
>
> #ifdef CONFIG_INFINIBAND_USER_MEM
> -
> struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
> size_t size, int access, int dmasync);
> +struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context, unsigned long addr,
> + size_t size, int access,
> + enum dma_data_direction dir,
> + unsigned long dma_attrs);
> void ib_umem_release(struct ib_umem *umem);
> int ib_umem_page_count(struct ib_umem *umem);
> int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
> @@ -98,6 +103,14 @@ static inline struct ib_umem *ib_umem_get(struct ib_ucontext *context,
> int access, int dmasync) {
> return ERR_PTR(-EINVAL);
> }
> +static inline struct ib_umem *ib_umem_get_attrs(struct ib_ucontext *context,
> + unsigned long addr,
> + size_t size, int access,
> + enum dma_data_direction dir,
> + unsigned long dma_attrs)
> +{
> + return ERR_PTR(-EINVAL);
> +}
> static inline void ib_umem_release(struct ib_umem *umem) { }
> static inline int ib_umem_page_count(struct ib_umem *umem) { return 0; }
> static inline int ib_umem_copy_from(void *dst, struct ib_umem *umem, size_t offset,
^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2017-04-05 15:21 UTC | newest]
Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-16 18:31 [PATCH v2 0/8] SIF related verbs patches Knut Omang
2016-09-16 18:31 ` [PATCH v2 1/8] ib_mad: incoming sminfo SMPs gets discarded if no process_mad function is registered Knut Omang
[not found] ` <66d69383a3376018d99c025cd188150f6673b209.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2016-09-16 20:28 ` Santosh Shilimkar
2016-09-16 20:28 ` Santosh Shilimkar
[not found] ` <b57491e1-e36b-c331-8360-557310d15002-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2016-09-16 20:40 ` Knut Omang
2016-09-16 20:40 ` Knut Omang
[not found] ` <cover.15d71ac2c57534f1170c2c48374d3841ed75e676.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2016-09-16 18:31 ` [PATCH v2 2/8] ib_umem: Add a new, more generic ib_umem_get_attrs Knut Omang
2016-09-16 18:31 ` Knut Omang
[not found] ` <53550a232af32c5c97ba9fb70faacc2c64d8fceb.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-04-05 15:20 ` Knut Omang
2017-04-05 15:20 ` Knut Omang
2016-09-16 18:31 ` [PATCH v2 3/8] ib_umem: With the new ib_umem_get_attrs, simplify ib_umem_get Knut Omang
2016-09-16 18:31 ` Knut Omang
2016-09-16 18:31 ` [PATCH v2 4/8] ib: Add udata argument to create_ah Knut Omang
2016-09-16 18:31 ` Knut Omang
2016-09-16 18:31 ` [PATCH v2 6/8] ib_uverbs: Avoid vendor specific masking of attributes in query_qp Knut Omang
2016-09-16 18:31 ` Knut Omang
2016-09-16 18:31 ` [PATCH v2 7/8] ib_{uverbs/core}: add new ib_create_qp_ex with udata arg Knut Omang
2016-09-16 18:31 ` Knut Omang
2016-09-16 20:30 ` [PATCH v2 0/8] SIF related verbs patches Santosh Shilimkar
2016-09-16 20:30 ` Santosh Shilimkar
2016-09-16 20:42 ` Knut Omang
2016-09-16 18:31 ` [PATCH v2 5/8] ib_uverbs: Add padding to end align ib_uverbs_reg_mr_resp Knut Omang
2016-09-20 10:45 ` Yishai Hadas
2016-09-20 11:02 ` Knut Omang
2016-09-16 18:31 ` [PATCH v2 8/8] ib_uverbs: Support for kernel implementation of XRC Knut Omang
[not found] ` <88ffb8c9407a71069b52e155dde308a36dfaf247.1474049924.git-series.knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2016-09-16 19:31 ` Jason Gunthorpe
2016-09-16 19:31 ` Jason Gunthorpe
[not found] ` <20160916193102.GB28859-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2016-09-16 19:49 ` Knut Omang
2016-09-16 19:49 ` Knut Omang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.