[PATCH RFC 00/10] IB/core: SG IOCTL based RDMA ABI

[PATCH RFC 00/10] IB/core: SG IOCTL based RDMA ABI

[Matan Barak]

The ideas presented here are based on our previous series in addition to some
ideas presented in OFVWG, Sean's series, Linux Plumbers 2017 discussions and
other discussions held in Openfabrics Alliance 2017 conference.

This patch series adds ioctl() interface to the existing write() interface and
provide an easy route to backport this change to legacy supported systems.
Analyzing the current uverbs role in dispatching and parsing commands, we find
that:
(a) uverbs validates the basic properties of the command.
(b) uverbs is responsible of doing all the IDR and uobject management and
    locking. It's also responsible for handling completion FDs.
(c) uverbs transforms the user<-->kernel ABI to kernel API.

(a) and (b) are valid for every kABI. Although the nature of commands could
change, they still have to be validated and transform to kernel pointers.
In order to avoid duplications between the various drivers, we would like to
keep (a) and (b) as shared code.

In addition, this is a good time to expand the ABI to be more scalable, so we
added a few goals:
(1) Command's attributes shall be extensible in an easy one. Either by allowing
    drivers to have their own extensible set of attributes or core code
    extensible attributes.
(2) Each driver may have specific type system (i.e QP, CQ, ....). It could extend
    this type system in the future. Try to avoid duplicating existing types or
    actions.

Thus, in order to allow this flexibility, we decide giving (a) and (b) as a
common infrastructure, but use per-driver guidelines in order to do that
parsing and uobject management. Handlers are also set by the drivers
themselves (though they can point to either shared common code) or
driver specific code.

We introduce a hierarchal object-method-attributes structure. Adding an
entity to this hierarchy doesn't affect the rest of the interface.
Such a hierarchy could be rooted in a specific device and describes both the
common features and features which are unique to this specific device.
This hierarchy is actually a per-device parsing tree, composed of three
layers - objects, actions and attributes. Each such layer contains two
groups - common entities and hardware specific entities. This way, a
device could add hardware specific actions to a common object, it could
add hardware specific objects, etc. Abstractions which really make sense,
should go to the common section. This means that we still need to be able to
pass optional parameters. In order to enable optional parameters, each command
is composed of a header and a bunch of TLVs to pass the attributes of this
command. The supported attribute classes are:
* PTR_IN (command) [in case of a small buffer, we could pass the data inlined]
* PTR_OUT (response)
* IDR_OBJECT
* FD_OBJECT
We differentiate between blobs and objects in order to allow a generic piece of
code in the kernel to do some syntactic validations and translate the given
user object id to a kernel structure. This could really help in sharing code
between different handlers.

Scatter gather was chosen in order to allow us not to recompile user space
drivers. By using pointers to driver specific data, we could just use it
without introduce copying data and without changing the user-space driver at
all.

We elevate the locking and IDR changes accepted to linux-rdma in this series.
Since types are no longer enforced by the common infrastructure, there is no
point of pre-allocating common IDR types in the common code. Instead, we
provide an API for driver to add new types. We use one IDR per context
for all its IDR types. The driver declared all its supported types, their
free function and release order. After that, all uboject, exclusive access
and types are handled automatically for the driver by the infrastructure.

When putting the pieces together, we have per-device parsing tree, that actually
describes all the objects, actions and attributes a device supports by using a
descriptive language. A command is given by the user-space, as a header plus an
array of Type-Length-Pointer/Object attributes. The ioctl callback executes a
generic code that shares as much logic between the various verbs handlers as
possible. This generic code gets the command input from the user-space and by
reading the device's parsing tree, it could syntactically validate it, grab all
required objects, lock them, call the right handler and then
commit/unlock/rollback the result, depending on the handler's result. Having
such a flexible extensible mechanism, that allows introducing new common and
hardware-specific to existing common attributes, but also allows adding new
hardware-specific entities, enhances the support for device diversity quite
vastly.

This series lays the foundations of such an infrastructure. It demonstrate a few
verbs handlers that use this new infrastructure for current features. We don't
demonstrate how to add device specific features, but it's fairly simple - just
introduce a device specific root, re-use all sub-trees you need and add/replace
whatever required for your device (this could be later enhanced by a introducing
a dynamic parse-tree merge).

Future work should treat other uverbs related subsystem (such as RDMA-CM)
similarly. When implementing this infrastructure for RDMA-CM, we may need to
replace ib_device with an ioctl_device and ib_ucontext with ioctl_context.

Another future enhancement is to use the parse tree in order to introduce an
enhanced query mechanism. Instead of having a bit for every new feature, we
could allow the user-space to read the parse tree and query if
types/actions/attributes are actually supported by this particular device.

Regards,
Matan

Matan Barak (10):
  IB/core: Add a generic way to execute an operation on a uobject
  IB/core: Add support to finalize objects in one transaction
  IB/core: Add new ioctl interface
  IB/core: Declare a type instead of declaring only type attributes
  IB/core: Add DEVICE type and root types structure
  IB/core: Initialize uverbs types specification
  IB/core: Add macros for declaring actions and attributes
  IB/core: Add ability to explicitly destroy an uobject
  IB/core: Add uverbs types, actions, handlers and attributes
  IB/core: Expose ioctl interface through experimental Kconfig

 drivers/infiniband/Kconfig                   |    7 +
 drivers/infiniband/core/Makefile             |    2 +-
 drivers/infiniband/core/core_priv.h          |   14 +
 drivers/infiniband/core/rdma_core.c          |  174 +++++
 drivers/infiniband/core/rdma_core.h          |   39 +
 drivers/infiniband/core/uverbs.h             |    9 +
 drivers/infiniband/core/uverbs_cmd.c         |   21 +-
 drivers/infiniband/core/uverbs_ioctl.c       |  409 ++++++++++
 drivers/infiniband/core/uverbs_main.c        |    9 +
 drivers/infiniband/core/uverbs_std_types.c   | 1076 ++++++++++++++++++++++++--
 drivers/infiniband/hw/cxgb3/iwch_provider.c  |    5 +
 drivers/infiniband/hw/cxgb4/provider.c       |    5 +
 drivers/infiniband/hw/hns/hns_roce_main.c    |    5 +
 drivers/infiniband/hw/i40iw/i40iw_verbs.c    |    5 +
 drivers/infiniband/hw/mlx4/main.c            |    5 +
 drivers/infiniband/hw/mlx5/main.c            |    5 +
 drivers/infiniband/hw/mthca/mthca_provider.c |    5 +
 drivers/infiniband/hw/nes/nes_verbs.c        |    5 +
 drivers/infiniband/hw/ocrdma/ocrdma_main.c   |    5 +
 drivers/infiniband/hw/usnic/usnic_ib_main.c  |    5 +
 include/rdma/ib_verbs.h                      |    2 +
 include/rdma/uverbs_ioctl.h                  |  289 +++++++
 include/rdma/uverbs_std_types.h              |  212 ++++-
 include/rdma/uverbs_types.h                  |   39 +-
 include/uapi/rdma/ib_user_verbs.h            |   40 +
 include/uapi/rdma/rdma_user_ioctl.h          |   25 +
 26 files changed, 2315 insertions(+), 102 deletions(-)
 create mode 100644 drivers/infiniband/core/uverbs_ioctl.c
 create mode 100644 include/rdma/uverbs_ioctl.h

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 01/10] IB/core: Add a generic way to execute an operation on a uobject

[Matan Barak]

The ioctl infrastructure treats all user-objects in the same manner.
It gets an id from the user-space and by using the object type and
operation mentioned in the action specification, it executes this
operation. An operation is split to two stages. The first one is
carried out before executing the handler and the second one is
executed afterwards.

In order to abstract these details from the ioctl infrastructure
layer, we add uverbs_get_uobject_from_context and
uverbs_finalize_object functions which corresponds to the first
and second stages respectively.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/rdma_core.c | 51 ++++++++++++++++++++++++++++++++++++
 drivers/infiniband/core/rdma_core.h | 16 ++++++++++++
 include/rdma/uverbs_ioctl.h         | 52 +++++++++++++++++++++++++++++++++++++
 3 files changed, 119 insertions(+)
 create mode 100644 include/rdma/uverbs_ioctl.h

diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
index 41c31a2..269fa7f 100644
--- a/drivers/infiniband/core/rdma_core.c
+++ b/drivers/infiniband/core/rdma_core.c
@@ -35,6 +35,7 @@
 #include <rdma/ib_verbs.h>
 #include <rdma/uverbs_types.h>
 #include <linux/rcupdate.h>
+#include <rdma/uverbs_ioctl.h>
 #include "uverbs.h"
 #include "core_priv.h"
 #include "rdma_core.h"
@@ -625,3 +626,53 @@ void uverbs_initialize_ucontext(struct ib_ucontext *ucontext)
 	.needs_kfree_rcu = false,
 };
 
+struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
+						   struct ib_ucontext *ucontext,
+						   enum uverbs_idr_access access,
+						   int id)
+{
+	switch (access) {
+	case UVERBS_ACCESS_READ:
+		return rdma_lookup_get_uobject(type_attrs, ucontext, id, false);
+	case UVERBS_ACCESS_DESTROY:
+	case UVERBS_ACCESS_WRITE:
+		return rdma_lookup_get_uobject(type_attrs, ucontext, id, true);
+	case UVERBS_ACCESS_NEW:
+		return rdma_alloc_begin_uobject(type_attrs, ucontext);
+	default:
+		WARN_ON(true);
+		return ERR_PTR(-EOPNOTSUPP);
+	}
+}
+
+int uverbs_finalize_object(struct ib_uobject *uobj,
+			   enum uverbs_idr_access access,
+			   bool commit)
+{
+	int ret = 0;
+
+	switch (access) {
+	case UVERBS_ACCESS_READ:
+		rdma_lookup_put_uobject(uobj, false);
+		break;
+	case UVERBS_ACCESS_WRITE:
+		rdma_lookup_put_uobject(uobj, true);
+		break;
+	case UVERBS_ACCESS_DESTROY:
+		if (commit)
+			ret = rdma_remove_commit_uobject(uobj);
+		else
+			rdma_lookup_put_uobject(uobj, true);
+		break;
+	case UVERBS_ACCESS_NEW:
+		if (commit)
+			ret = rdma_alloc_commit_uobject(uobj);
+		else
+			rdma_alloc_abort_uobject(uobj);
+		break;
+	default:
+		WARN_ON(true);
+	}
+
+	return ret;
+}
diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
index 1b82e7f..5a1da24 100644
--- a/drivers/infiniband/core/rdma_core.h
+++ b/drivers/infiniband/core/rdma_core.h
@@ -39,6 +39,7 @@
 
 #include <linux/idr.h>
 #include <rdma/uverbs_types.h>
+#include <rdma/uverbs_ioctl.h>
 #include <rdma/ib_verbs.h>
 #include <linux/mutex.h>
 
@@ -75,4 +76,19 @@
  */
 void uverbs_close_fd(struct file *f);
 
+/*
+ * Get an ib_uobject that corresponds to the given id from ucontext, assuming
+ * the object is from the given type. Lock it to the required access.
+ * This function could create (access == NEW) or destroy (access == DESTROY)
+ * objects if required. The action will be finalized only when
+ * uverbs_finalize_object is called.
+ */
+struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
+						   struct ib_ucontext *ucontext,
+						   enum uverbs_idr_access access,
+						   int id);
+int uverbs_finalize_object(struct ib_uobject *uobj,
+			   enum uverbs_idr_access access,
+			   bool commit);
+
 #endif /* RDMA_CORE_H */
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
new file mode 100644
index 0000000..a18468e
--- /dev/null
+++ b/include/rdma/uverbs_ioctl.h
@@ -0,0 +1,52 @@
+/*
+ * Copyright (c) 2016, Mellanox Technologies inc.  All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _UVERBS_IOCTL_
+#define _UVERBS_IOCTL_
+
+#include <rdma/uverbs_types.h>
+
+/*
+ * =======================================
+ *	Verbs action specifications
+ * =======================================
+ */
+
+enum uverbs_idr_access {
+	UVERBS_ACCESS_READ,
+	UVERBS_ACCESS_WRITE,
+	UVERBS_ACCESS_NEW,
+	UVERBS_ACCESS_DESTROY
+};
+
+#endif
+
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC 01/10] IB/core: Add a generic way to execute an operation on a uobject

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 6492 bytes --]

On Wed, Apr 19, 2017 at 06:20:16PM +0300, Matan Barak wrote:
> The ioctl infrastructure treats all user-objects in the same manner.
> It gets an id from the user-space and by using the object type and
> operation mentioned in the action specification, it executes this
> operation. An operation is split to two stages. The first one is
> carried out before executing the handler and the second one is
> executed afterwards.
>
> In order to abstract these details from the ioctl infrastructure
> layer, we add uverbs_get_uobject_from_context and
> uverbs_finalize_object functions which corresponds to the first
> and second stages respectively.
>
> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
>  drivers/infiniband/core/rdma_core.c | 51 ++++++++++++++++++++++++++++++++++++
>  drivers/infiniband/core/rdma_core.h | 16 ++++++++++++
>  include/rdma/uverbs_ioctl.h         | 52 +++++++++++++++++++++++++++++++++++++
>  3 files changed, 119 insertions(+)
>  create mode 100644 include/rdma/uverbs_ioctl.h
>
> diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
> index 41c31a2..269fa7f 100644
> --- a/drivers/infiniband/core/rdma_core.c
> +++ b/drivers/infiniband/core/rdma_core.c
> @@ -35,6 +35,7 @@
>  #include <rdma/ib_verbs.h>
>  #include <rdma/uverbs_types.h>
>  #include <linux/rcupdate.h>
> +#include <rdma/uverbs_ioctl.h>
>  #include "uverbs.h"
>  #include "core_priv.h"
>  #include "rdma_core.h"
> @@ -625,3 +626,53 @@ void uverbs_initialize_ucontext(struct ib_ucontext *ucontext)
>  	.needs_kfree_rcu = false,
>  };
>
> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
> +						   struct ib_ucontext *ucontext,
> +						   enum uverbs_idr_access access,
> +						   int id)
> +{
> +	switch (access) {
> +	case UVERBS_ACCESS_READ:
> +		return rdma_lookup_get_uobject(type_attrs, ucontext, id, false);
> +	case UVERBS_ACCESS_DESTROY:
> +	case UVERBS_ACCESS_WRITE:
> +		return rdma_lookup_get_uobject(type_attrs, ucontext, id, true);
> +	case UVERBS_ACCESS_NEW:
> +		return rdma_alloc_begin_uobject(type_attrs, ucontext);
> +	default:
> +		WARN_ON(true);
> +		return ERR_PTR(-EOPNOTSUPP);
> +	}
> +}
> +
> +int uverbs_finalize_object(struct ib_uobject *uobj,
> +			   enum uverbs_idr_access access,
> +			   bool commit)
> +{
> +	int ret = 0;
> +
> +	switch (access) {
> +	case UVERBS_ACCESS_READ:
> +		rdma_lookup_put_uobject(uobj, false);
> +		break;
> +	case UVERBS_ACCESS_WRITE:
> +		rdma_lookup_put_uobject(uobj, true);
> +		break;
> +	case UVERBS_ACCESS_DESTROY:
> +		if (commit)
> +			ret = rdma_remove_commit_uobject(uobj);
> +		else
> +			rdma_lookup_put_uobject(uobj, true);
> +		break;
> +	case UVERBS_ACCESS_NEW:
> +		if (commit)
> +			ret = rdma_alloc_commit_uobject(uobj);
> +		else
> +			rdma_alloc_abort_uobject(uobj);
> +		break;
> +	default:
> +		WARN_ON(true);
> +	}
> +
> +	return ret;
> +}
> diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
> index 1b82e7f..5a1da24 100644
> --- a/drivers/infiniband/core/rdma_core.h
> +++ b/drivers/infiniband/core/rdma_core.h
> @@ -39,6 +39,7 @@
>
>  #include <linux/idr.h>
>  #include <rdma/uverbs_types.h>
> +#include <rdma/uverbs_ioctl.h>
>  #include <rdma/ib_verbs.h>
>  #include <linux/mutex.h>
>
> @@ -75,4 +76,19 @@
>   */
>  void uverbs_close_fd(struct file *f);
>
> +/*
> + * Get an ib_uobject that corresponds to the given id from ucontext, assuming
> + * the object is from the given type. Lock it to the required access.
> + * This function could create (access == NEW) or destroy (access == DESTROY)
> + * objects if required. The action will be finalized only when
> + * uverbs_finalize_object is called.
> + */
> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
> +						   struct ib_ucontext *ucontext,
> +						   enum uverbs_idr_access access,
> +						   int id);
> +int uverbs_finalize_object(struct ib_uobject *uobj,
> +			   enum uverbs_idr_access access,
> +			   bool commit);
> +
>  #endif /* RDMA_CORE_H */
> diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
> new file mode 100644
> index 0000000..a18468e
> --- /dev/null
> +++ b/include/rdma/uverbs_ioctl.h
> @@ -0,0 +1,52 @@
> +/*
> + * Copyright (c) 2016, Mellanox Technologies inc.  All rights reserved.
> + *
> + * This software is available to you under a choice of one of two
> + * licenses.  You may choose to be licensed under the terms of the GNU
> + * General Public License (GPL) Version 2, available from the file
> + * COPYING in the main directory of this source tree, or the
> + * OpenIB.org BSD license below:
> + *
> + *     Redistribution and use in source and binary forms, with or
> + *     without modification, are permitted provided that the following
> + *     conditions are met:
> + *
> + *      - Redistributions of source code must retain the above
> + *        copyright notice, this list of conditions and the following
> + *        disclaimer.
> + *
> + *      - Redistributions in binary form must reproduce the above
> + *        copyright notice, this list of conditions and the following
> + *        disclaimer in the documentation and/or other materials
> + *        provided with the distribution.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> + * SOFTWARE.
> + */
> +
> +#ifndef _UVERBS_IOCTL_
> +#define _UVERBS_IOCTL_
> +
> +#include <rdma/uverbs_types.h>
> +
> +/*
> + * =======================================
> + *	Verbs action specifications
> + * =======================================
> + */
> +
> +enum uverbs_idr_access {
> +	UVERBS_ACCESS_READ,
> +	UVERBS_ACCESS_WRITE,
> +	UVERBS_ACCESS_NEW,
> +	UVERBS_ACCESS_DESTROY
> +};

Does it make sense to have MODIFY too?

> +
> +#endif
> +
> --
> 1.8.3.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH RFC 01/10] IB/core: Add a generic way to execute an operation on a uobject

[Matan Barak]

On Thu, May 4, 2017 at 1:23 PM, Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> On Wed, Apr 19, 2017 at 06:20:16PM +0300, Matan Barak wrote:
>> The ioctl infrastructure treats all user-objects in the same manner.
>> It gets an id from the user-space and by using the object type and
>> operation mentioned in the action specification, it executes this
>> operation. An operation is split to two stages. The first one is
>> carried out before executing the handler and the second one is
>> executed afterwards.
>>
>> In order to abstract these details from the ioctl infrastructure
>> layer, we add uverbs_get_uobject_from_context and
>> uverbs_finalize_object functions which corresponds to the first
>> and second stages respectively.
>>
>> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>> ---
>>  drivers/infiniband/core/rdma_core.c | 51 ++++++++++++++++++++++++++++++++++++
>>  drivers/infiniband/core/rdma_core.h | 16 ++++++++++++
>>  include/rdma/uverbs_ioctl.h         | 52 +++++++++++++++++++++++++++++++++++++
>>  3 files changed, 119 insertions(+)
>>  create mode 100644 include/rdma/uverbs_ioctl.h
>>
>> diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
>> index 41c31a2..269fa7f 100644
>> --- a/drivers/infiniband/core/rdma_core.c
>> +++ b/drivers/infiniband/core/rdma_core.c
>> @@ -35,6 +35,7 @@
>>  #include <rdma/ib_verbs.h>
>>  #include <rdma/uverbs_types.h>
>>  #include <linux/rcupdate.h>
>> +#include <rdma/uverbs_ioctl.h>
>>  #include "uverbs.h"
>>  #include "core_priv.h"
>>  #include "rdma_core.h"
>> @@ -625,3 +626,53 @@ void uverbs_initialize_ucontext(struct ib_ucontext *ucontext)
>>       .needs_kfree_rcu = false,
>>  };
>>
>> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
>> +                                                struct ib_ucontext *ucontext,
>> +                                                enum uverbs_idr_access access,
>> +                                                int id)
>> +{
>> +     switch (access) {
>> +     case UVERBS_ACCESS_READ:
>> +             return rdma_lookup_get_uobject(type_attrs, ucontext, id, false);
>> +     case UVERBS_ACCESS_DESTROY:
>> +     case UVERBS_ACCESS_WRITE:
>> +             return rdma_lookup_get_uobject(type_attrs, ucontext, id, true);
>> +     case UVERBS_ACCESS_NEW:
>> +             return rdma_alloc_begin_uobject(type_attrs, ucontext);
>> +     default:
>> +             WARN_ON(true);
>> +             return ERR_PTR(-EOPNOTSUPP);
>> +     }
>> +}
>> +
>> +int uverbs_finalize_object(struct ib_uobject *uobj,
>> +                        enum uverbs_idr_access access,
>> +                        bool commit)
>> +{
>> +     int ret = 0;
>> +
>> +     switch (access) {
>> +     case UVERBS_ACCESS_READ:
>> +             rdma_lookup_put_uobject(uobj, false);
>> +             break;
>> +     case UVERBS_ACCESS_WRITE:
>> +             rdma_lookup_put_uobject(uobj, true);
>> +             break;
>> +     case UVERBS_ACCESS_DESTROY:
>> +             if (commit)
>> +                     ret = rdma_remove_commit_uobject(uobj);
>> +             else
>> +                     rdma_lookup_put_uobject(uobj, true);
>> +             break;
>> +     case UVERBS_ACCESS_NEW:
>> +             if (commit)
>> +                     ret = rdma_alloc_commit_uobject(uobj);
>> +             else
>> +                     rdma_alloc_abort_uobject(uobj);
>> +             break;
>> +     default:
>> +             WARN_ON(true);
>> +     }
>> +
>> +     return ret;
>> +}
>> diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
>> index 1b82e7f..5a1da24 100644
>> --- a/drivers/infiniband/core/rdma_core.h
>> +++ b/drivers/infiniband/core/rdma_core.h
>> @@ -39,6 +39,7 @@
>>
>>  #include <linux/idr.h>
>>  #include <rdma/uverbs_types.h>
>> +#include <rdma/uverbs_ioctl.h>
>>  #include <rdma/ib_verbs.h>
>>  #include <linux/mutex.h>
>>
>> @@ -75,4 +76,19 @@
>>   */
>>  void uverbs_close_fd(struct file *f);
>>
>> +/*
>> + * Get an ib_uobject that corresponds to the given id from ucontext, assuming
>> + * the object is from the given type. Lock it to the required access.
>> + * This function could create (access == NEW) or destroy (access == DESTROY)
>> + * objects if required. The action will be finalized only when
>> + * uverbs_finalize_object is called.
>> + */
>> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
>> +                                                struct ib_ucontext *ucontext,
>> +                                                enum uverbs_idr_access access,
>> +                                                int id);
>> +int uverbs_finalize_object(struct ib_uobject *uobj,
>> +                        enum uverbs_idr_access access,
>> +                        bool commit);
>> +
>>  #endif /* RDMA_CORE_H */
>> diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
>> new file mode 100644
>> index 0000000..a18468e
>> --- /dev/null
>> +++ b/include/rdma/uverbs_ioctl.h
>> @@ -0,0 +1,52 @@
>> +/*
>> + * Copyright (c) 2016, Mellanox Technologies inc.  All rights reserved.
>> + *
>> + * This software is available to you under a choice of one of two
>> + * licenses.  You may choose to be licensed under the terms of the GNU
>> + * General Public License (GPL) Version 2, available from the file
>> + * COPYING in the main directory of this source tree, or the
>> + * OpenIB.org BSD license below:
>> + *
>> + *     Redistribution and use in source and binary forms, with or
>> + *     without modification, are permitted provided that the following
>> + *     conditions are met:
>> + *
>> + *      - Redistributions of source code must retain the above
>> + *        copyright notice, this list of conditions and the following
>> + *        disclaimer.
>> + *
>> + *      - Redistributions in binary form must reproduce the above
>> + *        copyright notice, this list of conditions and the following
>> + *        disclaimer in the documentation and/or other materials
>> + *        provided with the distribution.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
>> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
>> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
>> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
>> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
>> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
>> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
>> + * SOFTWARE.
>> + */
>> +
>> +#ifndef _UVERBS_IOCTL_
>> +#define _UVERBS_IOCTL_
>> +
>> +#include <rdma/uverbs_types.h>
>> +
>> +/*
>> + * =======================================
>> + *   Verbs action specifications
>> + * =======================================
>> + */
>> +
>> +enum uverbs_idr_access {
>> +     UVERBS_ACCESS_READ,
>> +     UVERBS_ACCESS_WRITE,
>> +     UVERBS_ACCESS_NEW,
>> +     UVERBS_ACCESS_DESTROY
>> +};
>
> Does it make sense to have MODIFY too?
>

No, this just set the locking schema for the uobject. When the uobject
is locked for write (i.e. exclusive lock), you're free to modify the
underlying object.

>> +
>> +#endif
>> +
>> --
>> 1.8.3.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC 01/10] IB/core: Add a generic way to execute an operation on a uobject

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 8324 bytes --]

On Thu, May 04, 2017 at 01:37:33PM +0300, Matan Barak wrote:
> On Thu, May 4, 2017 at 1:23 PM, Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> > On Wed, Apr 19, 2017 at 06:20:16PM +0300, Matan Barak wrote:
> >> The ioctl infrastructure treats all user-objects in the same manner.
> >> It gets an id from the user-space and by using the object type and
> >> operation mentioned in the action specification, it executes this
> >> operation. An operation is split to two stages. The first one is
> >> carried out before executing the handler and the second one is
> >> executed afterwards.
> >>
> >> In order to abstract these details from the ioctl infrastructure
> >> layer, we add uverbs_get_uobject_from_context and
> >> uverbs_finalize_object functions which corresponds to the first
> >> and second stages respectively.
> >>
> >> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> >> ---
> >>  drivers/infiniband/core/rdma_core.c | 51 ++++++++++++++++++++++++++++++++++++
> >>  drivers/infiniband/core/rdma_core.h | 16 ++++++++++++
> >>  include/rdma/uverbs_ioctl.h         | 52 +++++++++++++++++++++++++++++++++++++
> >>  3 files changed, 119 insertions(+)
> >>  create mode 100644 include/rdma/uverbs_ioctl.h
> >>
> >> diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
> >> index 41c31a2..269fa7f 100644
> >> --- a/drivers/infiniband/core/rdma_core.c
> >> +++ b/drivers/infiniband/core/rdma_core.c
> >> @@ -35,6 +35,7 @@
> >>  #include <rdma/ib_verbs.h>
> >>  #include <rdma/uverbs_types.h>
> >>  #include <linux/rcupdate.h>
> >> +#include <rdma/uverbs_ioctl.h>
> >>  #include "uverbs.h"
> >>  #include "core_priv.h"
> >>  #include "rdma_core.h"
> >> @@ -625,3 +626,53 @@ void uverbs_initialize_ucontext(struct ib_ucontext *ucontext)
> >>       .needs_kfree_rcu = false,
> >>  };
> >>
> >> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
> >> +                                                struct ib_ucontext *ucontext,
> >> +                                                enum uverbs_idr_access access,
> >> +                                                int id)
> >> +{
> >> +     switch (access) {
> >> +     case UVERBS_ACCESS_READ:
> >> +             return rdma_lookup_get_uobject(type_attrs, ucontext, id, false);
> >> +     case UVERBS_ACCESS_DESTROY:
> >> +     case UVERBS_ACCESS_WRITE:
> >> +             return rdma_lookup_get_uobject(type_attrs, ucontext, id, true);
> >> +     case UVERBS_ACCESS_NEW:
> >> +             return rdma_alloc_begin_uobject(type_attrs, ucontext);
> >> +     default:
> >> +             WARN_ON(true);
> >> +             return ERR_PTR(-EOPNOTSUPP);
> >> +     }
> >> +}
> >> +
> >> +int uverbs_finalize_object(struct ib_uobject *uobj,
> >> +                        enum uverbs_idr_access access,
> >> +                        bool commit)
> >> +{
> >> +     int ret = 0;
> >> +
> >> +     switch (access) {
> >> +     case UVERBS_ACCESS_READ:
> >> +             rdma_lookup_put_uobject(uobj, false);
> >> +             break;
> >> +     case UVERBS_ACCESS_WRITE:
> >> +             rdma_lookup_put_uobject(uobj, true);
> >> +             break;
> >> +     case UVERBS_ACCESS_DESTROY:
> >> +             if (commit)
> >> +                     ret = rdma_remove_commit_uobject(uobj);
> >> +             else
> >> +                     rdma_lookup_put_uobject(uobj, true);
> >> +             break;
> >> +     case UVERBS_ACCESS_NEW:
> >> +             if (commit)
> >> +                     ret = rdma_alloc_commit_uobject(uobj);
> >> +             else
> >> +                     rdma_alloc_abort_uobject(uobj);
> >> +             break;
> >> +     default:
> >> +             WARN_ON(true);
> >> +     }
> >> +
> >> +     return ret;
> >> +}
> >> diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
> >> index 1b82e7f..5a1da24 100644
> >> --- a/drivers/infiniband/core/rdma_core.h
> >> +++ b/drivers/infiniband/core/rdma_core.h
> >> @@ -39,6 +39,7 @@
> >>
> >>  #include <linux/idr.h>
> >>  #include <rdma/uverbs_types.h>
> >> +#include <rdma/uverbs_ioctl.h>
> >>  #include <rdma/ib_verbs.h>
> >>  #include <linux/mutex.h>
> >>
> >> @@ -75,4 +76,19 @@
> >>   */
> >>  void uverbs_close_fd(struct file *f);
> >>
> >> +/*
> >> + * Get an ib_uobject that corresponds to the given id from ucontext, assuming
> >> + * the object is from the given type. Lock it to the required access.
> >> + * This function could create (access == NEW) or destroy (access == DESTROY)
> >> + * objects if required. The action will be finalized only when
> >> + * uverbs_finalize_object is called.
> >> + */
> >> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
> >> +                                                struct ib_ucontext *ucontext,
> >> +                                                enum uverbs_idr_access access,
> >> +                                                int id);
> >> +int uverbs_finalize_object(struct ib_uobject *uobj,
> >> +                        enum uverbs_idr_access access,
> >> +                        bool commit);
> >> +
> >>  #endif /* RDMA_CORE_H */
> >> diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
> >> new file mode 100644
> >> index 0000000..a18468e
> >> --- /dev/null
> >> +++ b/include/rdma/uverbs_ioctl.h
> >> @@ -0,0 +1,52 @@
> >> +/*
> >> + * Copyright (c) 2016, Mellanox Technologies inc.  All rights reserved.
> >> + *
> >> + * This software is available to you under a choice of one of two
> >> + * licenses.  You may choose to be licensed under the terms of the GNU
> >> + * General Public License (GPL) Version 2, available from the file
> >> + * COPYING in the main directory of this source tree, or the
> >> + * OpenIB.org BSD license below:
> >> + *
> >> + *     Redistribution and use in source and binary forms, with or
> >> + *     without modification, are permitted provided that the following
> >> + *     conditions are met:
> >> + *
> >> + *      - Redistributions of source code must retain the above
> >> + *        copyright notice, this list of conditions and the following
> >> + *        disclaimer.
> >> + *
> >> + *      - Redistributions in binary form must reproduce the above
> >> + *        copyright notice, this list of conditions and the following
> >> + *        disclaimer in the documentation and/or other materials
> >> + *        provided with the distribution.
> >> + *
> >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> >> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> >> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> >> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
> >> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
> >> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
> >> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> >> + * SOFTWARE.
> >> + */
> >> +
> >> +#ifndef _UVERBS_IOCTL_
> >> +#define _UVERBS_IOCTL_
> >> +
> >> +#include <rdma/uverbs_types.h>
> >> +
> >> +/*
> >> + * =======================================
> >> + *   Verbs action specifications
> >> + * =======================================
> >> + */
> >> +
> >> +enum uverbs_idr_access {
> >> +     UVERBS_ACCESS_READ,
> >> +     UVERBS_ACCESS_WRITE,
> >> +     UVERBS_ACCESS_NEW,
> >> +     UVERBS_ACCESS_DESTROY
> >> +};
> >
> > Does it make sense to have MODIFY too?
> >
>
> No, this just set the locking schema for the uobject. When the uobject
> is locked for write (i.e. exclusive lock), you're free to modify the
> underlying object.

So why do you need NEW and DESTROY? It can be locked with WRITE.

>
> >> +
> >> +#endif
> >> +
> >> --
> >> 1.8.3.1
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH RFC 01/10] IB/core: Add a generic way to execute an operation on a uobject

[Matan Barak]

On Thu, May 4, 2017 at 3:28 PM, Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> On Thu, May 04, 2017 at 01:37:33PM +0300, Matan Barak wrote:
>> On Thu, May 4, 2017 at 1:23 PM, Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
>> > On Wed, Apr 19, 2017 at 06:20:16PM +0300, Matan Barak wrote:
>> >> The ioctl infrastructure treats all user-objects in the same manner.
>> >> It gets an id from the user-space and by using the object type and
>> >> operation mentioned in the action specification, it executes this
>> >> operation. An operation is split to two stages. The first one is
>> >> carried out before executing the handler and the second one is
>> >> executed afterwards.
>> >>
>> >> In order to abstract these details from the ioctl infrastructure
>> >> layer, we add uverbs_get_uobject_from_context and
>> >> uverbs_finalize_object functions which corresponds to the first
>> >> and second stages respectively.
>> >>
>> >> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>> >> ---
>> >>  drivers/infiniband/core/rdma_core.c | 51 ++++++++++++++++++++++++++++++++++++
>> >>  drivers/infiniband/core/rdma_core.h | 16 ++++++++++++
>> >>  include/rdma/uverbs_ioctl.h         | 52 +++++++++++++++++++++++++++++++++++++
>> >>  3 files changed, 119 insertions(+)
>> >>  create mode 100644 include/rdma/uverbs_ioctl.h
>> >>
>> >> diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
>> >> index 41c31a2..269fa7f 100644
>> >> --- a/drivers/infiniband/core/rdma_core.c
>> >> +++ b/drivers/infiniband/core/rdma_core.c
>> >> @@ -35,6 +35,7 @@
>> >>  #include <rdma/ib_verbs.h>
>> >>  #include <rdma/uverbs_types.h>
>> >>  #include <linux/rcupdate.h>
>> >> +#include <rdma/uverbs_ioctl.h>
>> >>  #include "uverbs.h"
>> >>  #include "core_priv.h"
>> >>  #include "rdma_core.h"
>> >> @@ -625,3 +626,53 @@ void uverbs_initialize_ucontext(struct ib_ucontext *ucontext)
>> >>       .needs_kfree_rcu = false,
>> >>  };
>> >>
>> >> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
>> >> +                                                struct ib_ucontext *ucontext,
>> >> +                                                enum uverbs_idr_access access,
>> >> +                                                int id)
>> >> +{
>> >> +     switch (access) {
>> >> +     case UVERBS_ACCESS_READ:
>> >> +             return rdma_lookup_get_uobject(type_attrs, ucontext, id, false);
>> >> +     case UVERBS_ACCESS_DESTROY:
>> >> +     case UVERBS_ACCESS_WRITE:
>> >> +             return rdma_lookup_get_uobject(type_attrs, ucontext, id, true);
>> >> +     case UVERBS_ACCESS_NEW:
>> >> +             return rdma_alloc_begin_uobject(type_attrs, ucontext);
>> >> +     default:
>> >> +             WARN_ON(true);
>> >> +             return ERR_PTR(-EOPNOTSUPP);
>> >> +     }
>> >> +}
>> >> +
>> >> +int uverbs_finalize_object(struct ib_uobject *uobj,
>> >> +                        enum uverbs_idr_access access,
>> >> +                        bool commit)
>> >> +{
>> >> +     int ret = 0;
>> >> +
>> >> +     switch (access) {
>> >> +     case UVERBS_ACCESS_READ:
>> >> +             rdma_lookup_put_uobject(uobj, false);
>> >> +             break;
>> >> +     case UVERBS_ACCESS_WRITE:
>> >> +             rdma_lookup_put_uobject(uobj, true);
>> >> +             break;
>> >> +     case UVERBS_ACCESS_DESTROY:
>> >> +             if (commit)
>> >> +                     ret = rdma_remove_commit_uobject(uobj);
>> >> +             else
>> >> +                     rdma_lookup_put_uobject(uobj, true);
>> >> +             break;
>> >> +     case UVERBS_ACCESS_NEW:
>> >> +             if (commit)
>> >> +                     ret = rdma_alloc_commit_uobject(uobj);
>> >> +             else
>> >> +                     rdma_alloc_abort_uobject(uobj);
>> >> +             break;
>> >> +     default:
>> >> +             WARN_ON(true);
>> >> +     }
>> >> +
>> >> +     return ret;
>> >> +}
>> >> diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
>> >> index 1b82e7f..5a1da24 100644
>> >> --- a/drivers/infiniband/core/rdma_core.h
>> >> +++ b/drivers/infiniband/core/rdma_core.h
>> >> @@ -39,6 +39,7 @@
>> >>
>> >>  #include <linux/idr.h>
>> >>  #include <rdma/uverbs_types.h>
>> >> +#include <rdma/uverbs_ioctl.h>
>> >>  #include <rdma/ib_verbs.h>
>> >>  #include <linux/mutex.h>
>> >>
>> >> @@ -75,4 +76,19 @@
>> >>   */
>> >>  void uverbs_close_fd(struct file *f);
>> >>
>> >> +/*
>> >> + * Get an ib_uobject that corresponds to the given id from ucontext, assuming
>> >> + * the object is from the given type. Lock it to the required access.
>> >> + * This function could create (access == NEW) or destroy (access == DESTROY)
>> >> + * objects if required. The action will be finalized only when
>> >> + * uverbs_finalize_object is called.
>> >> + */
>> >> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
>> >> +                                                struct ib_ucontext *ucontext,
>> >> +                                                enum uverbs_idr_access access,
>> >> +                                                int id);
>> >> +int uverbs_finalize_object(struct ib_uobject *uobj,
>> >> +                        enum uverbs_idr_access access,
>> >> +                        bool commit);
>> >> +
>> >>  #endif /* RDMA_CORE_H */
>> >> diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
>> >> new file mode 100644
>> >> index 0000000..a18468e
>> >> --- /dev/null
>> >> +++ b/include/rdma/uverbs_ioctl.h
>> >> @@ -0,0 +1,52 @@
>> >> +/*
>> >> + * Copyright (c) 2016, Mellanox Technologies inc.  All rights reserved.
>> >> + *
>> >> + * This software is available to you under a choice of one of two
>> >> + * licenses.  You may choose to be licensed under the terms of the GNU
>> >> + * General Public License (GPL) Version 2, available from the file
>> >> + * COPYING in the main directory of this source tree, or the
>> >> + * OpenIB.org BSD license below:
>> >> + *
>> >> + *     Redistribution and use in source and binary forms, with or
>> >> + *     without modification, are permitted provided that the following
>> >> + *     conditions are met:
>> >> + *
>> >> + *      - Redistributions of source code must retain the above
>> >> + *        copyright notice, this list of conditions and the following
>> >> + *        disclaimer.
>> >> + *
>> >> + *      - Redistributions in binary form must reproduce the above
>> >> + *        copyright notice, this list of conditions and the following
>> >> + *        disclaimer in the documentation and/or other materials
>> >> + *        provided with the distribution.
>> >> + *
>> >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
>> >> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
>> >> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
>> >> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
>> >> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
>> >> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
>> >> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
>> >> + * SOFTWARE.
>> >> + */
>> >> +
>> >> +#ifndef _UVERBS_IOCTL_
>> >> +#define _UVERBS_IOCTL_
>> >> +
>> >> +#include <rdma/uverbs_types.h>
>> >> +
>> >> +/*
>> >> + * =======================================
>> >> + *   Verbs action specifications
>> >> + * =======================================
>> >> + */
>> >> +
>> >> +enum uverbs_idr_access {
>> >> +     UVERBS_ACCESS_READ,
>> >> +     UVERBS_ACCESS_WRITE,
>> >> +     UVERBS_ACCESS_NEW,
>> >> +     UVERBS_ACCESS_DESTROY
>> >> +};
>> >
>> > Does it make sense to have MODIFY too?
>> >
>>
>> No, this just set the locking schema for the uobject. When the uobject
>> is locked for write (i.e. exclusive lock), you're free to modify the
>> underlying object.
>
> So why do you need NEW and DESTROY? It can be locked with WRITE.
>

The whole concept is based on specifications. When you specify a
command, you need to specify its arguments as well.
If these arguments are of object type, you need to specify the
required state of the uobject.
When your handle is being executed, uobjects are already in the
correct state as required by the command specification.
So, if you implement a command that creates a QP, you need to tell the
infrastructure that you want to allocate a new uobject - this is
CREATE.
If you want to destroy an object, you specify a DESTROY command. If
your handler finishes successfully, both the uobject and the object
are automatically destroyed by the infrastructure.

Matan

>>
>> >> +
>> >> +#endif
>> >> +
>> >> --
>> >> 1.8.3.1
>> >>
>> >> --
>> >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC 01/10] IB/core: Add a generic way to execute an operation on a uobject

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 10053 bytes --]

On Thu, May 04, 2017 at 06:43:49PM +0300, Matan Barak wrote:
> On Thu, May 4, 2017 at 3:28 PM, Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> > On Thu, May 04, 2017 at 01:37:33PM +0300, Matan Barak wrote:
> >> On Thu, May 4, 2017 at 1:23 PM, Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> >> > On Wed, Apr 19, 2017 at 06:20:16PM +0300, Matan Barak wrote:
> >> >> The ioctl infrastructure treats all user-objects in the same manner.
> >> >> It gets an id from the user-space and by using the object type and
> >> >> operation mentioned in the action specification, it executes this
> >> >> operation. An operation is split to two stages. The first one is
> >> >> carried out before executing the handler and the second one is
> >> >> executed afterwards.
> >> >>
> >> >> In order to abstract these details from the ioctl infrastructure
> >> >> layer, we add uverbs_get_uobject_from_context and
> >> >> uverbs_finalize_object functions which corresponds to the first
> >> >> and second stages respectively.
> >> >>
> >> >> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> >> >> ---
> >> >>  drivers/infiniband/core/rdma_core.c | 51 ++++++++++++++++++++++++++++++++++++
> >> >>  drivers/infiniband/core/rdma_core.h | 16 ++++++++++++
> >> >>  include/rdma/uverbs_ioctl.h         | 52 +++++++++++++++++++++++++++++++++++++
> >> >>  3 files changed, 119 insertions(+)
> >> >>  create mode 100644 include/rdma/uverbs_ioctl.h
> >> >>
> >> >> diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
> >> >> index 41c31a2..269fa7f 100644
> >> >> --- a/drivers/infiniband/core/rdma_core.c
> >> >> +++ b/drivers/infiniband/core/rdma_core.c
> >> >> @@ -35,6 +35,7 @@
> >> >>  #include <rdma/ib_verbs.h>
> >> >>  #include <rdma/uverbs_types.h>
> >> >>  #include <linux/rcupdate.h>
> >> >> +#include <rdma/uverbs_ioctl.h>
> >> >>  #include "uverbs.h"
> >> >>  #include "core_priv.h"
> >> >>  #include "rdma_core.h"
> >> >> @@ -625,3 +626,53 @@ void uverbs_initialize_ucontext(struct ib_ucontext *ucontext)
> >> >>       .needs_kfree_rcu = false,
> >> >>  };
> >> >>
> >> >> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
> >> >> +                                                struct ib_ucontext *ucontext,
> >> >> +                                                enum uverbs_idr_access access,
> >> >> +                                                int id)
> >> >> +{
> >> >> +     switch (access) {
> >> >> +     case UVERBS_ACCESS_READ:
> >> >> +             return rdma_lookup_get_uobject(type_attrs, ucontext, id, false);
> >> >> +     case UVERBS_ACCESS_DESTROY:
> >> >> +     case UVERBS_ACCESS_WRITE:
> >> >> +             return rdma_lookup_get_uobject(type_attrs, ucontext, id, true);
> >> >> +     case UVERBS_ACCESS_NEW:
> >> >> +             return rdma_alloc_begin_uobject(type_attrs, ucontext);
> >> >> +     default:
> >> >> +             WARN_ON(true);
> >> >> +             return ERR_PTR(-EOPNOTSUPP);
> >> >> +     }
> >> >> +}
> >> >> +
> >> >> +int uverbs_finalize_object(struct ib_uobject *uobj,
> >> >> +                        enum uverbs_idr_access access,
> >> >> +                        bool commit)
> >> >> +{
> >> >> +     int ret = 0;
> >> >> +
> >> >> +     switch (access) {
> >> >> +     case UVERBS_ACCESS_READ:
> >> >> +             rdma_lookup_put_uobject(uobj, false);
> >> >> +             break;
> >> >> +     case UVERBS_ACCESS_WRITE:
> >> >> +             rdma_lookup_put_uobject(uobj, true);
> >> >> +             break;
> >> >> +     case UVERBS_ACCESS_DESTROY:
> >> >> +             if (commit)
> >> >> +                     ret = rdma_remove_commit_uobject(uobj);
> >> >> +             else
> >> >> +                     rdma_lookup_put_uobject(uobj, true);
> >> >> +             break;
> >> >> +     case UVERBS_ACCESS_NEW:
> >> >> +             if (commit)
> >> >> +                     ret = rdma_alloc_commit_uobject(uobj);
> >> >> +             else
> >> >> +                     rdma_alloc_abort_uobject(uobj);
> >> >> +             break;
> >> >> +     default:
> >> >> +             WARN_ON(true);
> >> >> +     }
> >> >> +
> >> >> +     return ret;
> >> >> +}
> >> >> diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
> >> >> index 1b82e7f..5a1da24 100644
> >> >> --- a/drivers/infiniband/core/rdma_core.h
> >> >> +++ b/drivers/infiniband/core/rdma_core.h
> >> >> @@ -39,6 +39,7 @@
> >> >>
> >> >>  #include <linux/idr.h>
> >> >>  #include <rdma/uverbs_types.h>
> >> >> +#include <rdma/uverbs_ioctl.h>
> >> >>  #include <rdma/ib_verbs.h>
> >> >>  #include <linux/mutex.h>
> >> >>
> >> >> @@ -75,4 +76,19 @@
> >> >>   */
> >> >>  void uverbs_close_fd(struct file *f);
> >> >>
> >> >> +/*
> >> >> + * Get an ib_uobject that corresponds to the given id from ucontext, assuming
> >> >> + * the object is from the given type. Lock it to the required access.
> >> >> + * This function could create (access == NEW) or destroy (access == DESTROY)
> >> >> + * objects if required. The action will be finalized only when
> >> >> + * uverbs_finalize_object is called.
> >> >> + */
> >> >> +struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
> >> >> +                                                struct ib_ucontext *ucontext,
> >> >> +                                                enum uverbs_idr_access access,
> >> >> +                                                int id);
> >> >> +int uverbs_finalize_object(struct ib_uobject *uobj,
> >> >> +                        enum uverbs_idr_access access,
> >> >> +                        bool commit);
> >> >> +
> >> >>  #endif /* RDMA_CORE_H */
> >> >> diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
> >> >> new file mode 100644
> >> >> index 0000000..a18468e
> >> >> --- /dev/null
> >> >> +++ b/include/rdma/uverbs_ioctl.h
> >> >> @@ -0,0 +1,52 @@
> >> >> +/*
> >> >> + * Copyright (c) 2016, Mellanox Technologies inc.  All rights reserved.
> >> >> + *
> >> >> + * This software is available to you under a choice of one of two
> >> >> + * licenses.  You may choose to be licensed under the terms of the GNU
> >> >> + * General Public License (GPL) Version 2, available from the file
> >> >> + * COPYING in the main directory of this source tree, or the
> >> >> + * OpenIB.org BSD license below:
> >> >> + *
> >> >> + *     Redistribution and use in source and binary forms, with or
> >> >> + *     without modification, are permitted provided that the following
> >> >> + *     conditions are met:
> >> >> + *
> >> >> + *      - Redistributions of source code must retain the above
> >> >> + *        copyright notice, this list of conditions and the following
> >> >> + *        disclaimer.
> >> >> + *
> >> >> + *      - Redistributions in binary form must reproduce the above
> >> >> + *        copyright notice, this list of conditions and the following
> >> >> + *        disclaimer in the documentation and/or other materials
> >> >> + *        provided with the distribution.
> >> >> + *
> >> >> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> >> >> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> >> >> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> >> >> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
> >> >> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
> >> >> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
> >> >> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> >> >> + * SOFTWARE.
> >> >> + */
> >> >> +
> >> >> +#ifndef _UVERBS_IOCTL_
> >> >> +#define _UVERBS_IOCTL_
> >> >> +
> >> >> +#include <rdma/uverbs_types.h>
> >> >> +
> >> >> +/*
> >> >> + * =======================================
> >> >> + *   Verbs action specifications
> >> >> + * =======================================
> >> >> + */
> >> >> +
> >> >> +enum uverbs_idr_access {
> >> >> +     UVERBS_ACCESS_READ,
> >> >> +     UVERBS_ACCESS_WRITE,
> >> >> +     UVERBS_ACCESS_NEW,
> >> >> +     UVERBS_ACCESS_DESTROY
> >> >> +};
> >> >
> >> > Does it make sense to have MODIFY too?
> >> >
> >>
> >> No, this just set the locking schema for the uobject. When the uobject
> >> is locked for write (i.e. exclusive lock), you're free to modify the
> >> underlying object.
> >
> > So why do you need NEW and DESTROY? It can be locked with WRITE.
> >
>
> The whole concept is based on specifications. When you specify a
> command, you need to specify its arguments as well.
> If these arguments are of object type, you need to specify the
> required state of the uobject.
> When your handle is being executed, uobjects are already in the
> correct state as required by the command specification.
> So, if you implement a command that creates a QP, you need to tell the
> infrastructure that you want to allocate a new uobject - this is
> CREATE.
> If you want to destroy an object, you specify a DESTROY command. If
> your handler finishes successfully, both the uobject and the object
> are automatically destroyed by the infrastructure.

Thanks for the explanations.

>
> Matan
>
> >>
> >> >> +
> >> >> +#endif
> >> >> +
> >> >> --
> >> >> 1.8.3.1
> >> >>
> >> >> --
> >> >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> >> >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> >> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH RFC 02/10] IB/core: Add support to finalize objects in one transaction

[Matan Barak]

The new ioctl based infrastructure either commits or rollbacks
all objects of the command as one transaction. In order to do
that, we introduce a notion of dealing with a collection of
objects that are related to a specific action.

This also requires adding a notion of an action and attribute.
An action contains a groups of attributes, where each group
contains several attributes.

When declaring these actions and attributes, we actually declare
their specifications. When a command is executed, we actually
allocates some space to hold auxiliary information.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/rdma_core.c | 43 ++++++++++++++++++++++
 drivers/infiniband/core/rdma_core.h | 20 ++++++++++-
 include/rdma/uverbs_ioctl.h         | 72 +++++++++++++++++++++++++++++++++++++
 3 files changed, 134 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
index 269fa7f..78ffd8c 100644
--- a/drivers/infiniband/core/rdma_core.c
+++ b/drivers/infiniband/core/rdma_core.c
@@ -676,3 +676,46 @@ int uverbs_finalize_object(struct ib_uobject *uobj,
 
 	return ret;
 }
+
+int uverbs_finalize_objects(struct uverbs_attr_array *attr_array,
+			    size_t num,
+			    const struct uverbs_action *action,
+			    bool commit)
+{
+	unsigned int i;
+	int ret = 0;
+
+	for (i = 0; i < num; i++) {
+		struct uverbs_attr_array *attr_spec_array = &attr_array[i];
+		const struct uverbs_attr_spec_group *attr_spec_group =
+			action->attr_groups[i];
+		unsigned int j;
+
+		for (j = 0; j < attr_spec_array->num_attrs; j++) {
+			struct uverbs_attr *attr = &attr_spec_array->attrs[j];
+			struct uverbs_attr_spec *spec = &attr_spec_group->attrs[j];
+
+			if (!uverbs_is_valid(attr_spec_array, j))
+				continue;
+
+			if (spec->type == UVERBS_ATTR_TYPE_IDR ||
+			    spec->type == UVERBS_ATTR_TYPE_FD) {
+				int current_ret;
+
+				/*
+				 * refcounts should be handled at the object
+				 * level and not at the uobject level. Refcounts
+				 * of the objects themselves are done in
+				 * handlers.
+				 */
+				current_ret = uverbs_finalize_object(attr->obj_attr.uobject,
+								     spec->obj.access,
+								     commit);
+				if (!ret)
+					ret = current_ret;
+			}
+		}
+	}
+	return ret;
+}
+
diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
index 5a1da24..0aebc47 100644
--- a/drivers/infiniband/core/rdma_core.h
+++ b/drivers/infiniband/core/rdma_core.h
@@ -81,7 +81,7 @@
  * the object is from the given type. Lock it to the required access.
  * This function could create (access == NEW) or destroy (access == DESTROY)
  * objects if required. The action will be finalized only when
- * uverbs_finalize_object is called.
+ * uverbs_finalize_object or uverbs_finalize_objects is called.
  */
 struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type *type_attrs,
 						   struct ib_ucontext *ucontext,
@@ -90,5 +90,23 @@ struct ib_uobject *uverbs_get_uobject_from_context(const struct uverbs_obj_type
 int uverbs_finalize_object(struct ib_uobject *uobj,
 			   enum uverbs_idr_access access,
 			   bool commit);
+/*
+ * Note that certain finalize stages could return a status:
+ *   (a) alloc_commit could return a failure if the object is committed at the
+ *       same time when the context is destroyed.
+ *   (b) remove_commit could fail if the object wasn't destroyed successfully.
+ * Since multiple objects could be finalized in one transaction, it is very NOT
+ * recommended to have several finalize actions which have side affects.
+ * For example, it's NOT recommended to have a certain action which has both
+ * a commit action and a destroy action or two destroy objects in the same
+ * action. The rule of thumb is to have one destroy or commit action with
+ * multiple lookups.
+ * The first non zero return value of finalize_object is returned from this
+ * function.
+ */
+int uverbs_finalize_objects(struct uverbs_attr_array *attr_array,
+			    size_t num,
+			    const struct uverbs_action *action,
+			    bool success);
 
 #endif /* RDMA_CORE_H */
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index a18468e..1f84f30 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -41,6 +41,12 @@
  * =======================================
  */
 
+enum uverbs_attr_type {
+	UVERBS_ATTR_TYPE_NA,
+	UVERBS_ATTR_TYPE_IDR,
+	UVERBS_ATTR_TYPE_FD,
+};
+
 enum uverbs_idr_access {
 	UVERBS_ACCESS_READ,
 	UVERBS_ACCESS_WRITE,
@@ -48,5 +54,71 @@ enum uverbs_idr_access {
 	UVERBS_ACCESS_DESTROY
 };
 
+struct uverbs_attr_spec {
+	enum uverbs_attr_type		type;
+	union {
+		u16				len;
+		struct {
+			u16			obj_type;
+			u8			access;
+		} obj;
+	};
+};
+
+struct uverbs_attr_spec_group {
+	struct uverbs_attr_spec		*attrs;
+	size_t				num_attrs;
+};
+
+struct uverbs_action {
+	const struct uverbs_attr_spec_group		**attr_groups;
+	size_t						num_groups;
+};
+
+/* =================================================
+ *              Parsing infrastructure
+ * =================================================
+ */
+
+struct uverbs_fd_attr {
+	int		fd;
+};
+
+struct uverbs_uobj_attr {
+	/*  idr handle */
+	u32	idr;
+};
+
+struct uverbs_obj_attr {
+	/* pointer to the kernel descriptor -> type, access, etc */
+	struct ib_uverbs_attr __user	*uattr;
+	const struct uverbs_type_alloc_action	*type;
+	struct ib_uobject		*uobject;
+	union {
+		struct uverbs_fd_attr		fd;
+		struct uverbs_uobj_attr		uobj;
+	};
+};
+
+struct uverbs_attr {
+	union {
+		struct uverbs_obj_attr	obj_attr;
+	};
+};
+
+/* output of one validator */
+struct uverbs_attr_array {
+	unsigned long *valid_bitmap;
+	size_t num_attrs;
+	/* arrays of attrubytes, index is the id i.e SEND_CQ */
+	struct uverbs_attr *attrs;
+};
+
+static inline bool uverbs_is_valid(const struct uverbs_attr_array *attr_array,
+				   unsigned int idx)
+{
+	return test_bit(idx, attr_array->valid_bitmap);
+}
+
 #endif
 
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH RFC 02/10] IB/core: Add support to finalize objects in one transaction

[Amrani, Ram]

> The new ioctl based infrastructure either commits or rollbacks
> all objects of the command as one transaction. In order to do
> that, we introduce a notion of dealing with a collection of
> objects that are related to a specific action.
> 
> This also requires adding a notion of an action and attribute.
> An action contains a groups of attributes, where each group
> contains several attributes.
> 
> When declaring these actions and attributes, we actually declare
> their specifications. When a command is executed, we actually
> allocates some space to hold auxiliary information.
> 
> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---

Matan, thanks for the RFC!

If I got this correctly each object will go through three phases - get, handler, and a put.
I don't quite understand how a batch operation, like destroy QPs, can be undone after the handler phase.
I do see it working if at first multiple gets are performed and one of them fails.
In that case undoing is easy because the handlers weren't invoked yet.

In the case were some user-objects failed the operation how is this reflected upwards?

Also, I wonder, is there another intention behind batch operations except speed?


Thanks,
Ram
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC 02/10] IB/core: Add support to finalize objects in one transaction

[Matan Barak]

On Thu, May 4, 2017 at 5:50 PM, Amrani, Ram <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
>> The new ioctl based infrastructure either commits or rollbacks
>> all objects of the command as one transaction. In order to do
>> that, we introduce a notion of dealing with a collection of
>> objects that are related to a specific action.
>>
>> This also requires adding a notion of an action and attribute.
>> An action contains a groups of attributes, where each group
>> contains several attributes.
>>
>> When declaring these actions and attributes, we actually declare
>> their specifications. When a command is executed, we actually
>> allocates some space to hold auxiliary information.
>>
>> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>> ---
>
> Matan, thanks for the RFC!
>
> If I got this correctly each object will go through three phases - get, handler, and a put.
> I don't quite understand how a batch operation, like destroy QPs, can be undone after the handler phase.
> I do see it working if at first multiple gets are performed and one of them fails.
> In that case undoing is easy because the handlers weren't invoked yet.
>
> In the case were some user-objects failed the operation how is this reflected upwards?
>
> Also, I wonder, is there another intention behind batch operations except speed?
>

It's really not recommended to batch create/destroy. The reason is
exactly what you've pointer out.
If you batch several "destroy objects" and the n'th one fail, you
can't unwind the successful ones.

So basically, we want to support a semantic which is similar to what
we have today - create a single object or destroy a single object.
In this case, the pre-handler stage locks the dependencies of this
object (for example, in create_qp you lock the pd and cq so they won't
be destroyed)
and create the uobject for the QP. The handler itself can assume the
requirements it stated in the specifications are filled and just
create the QP and
tie the uobject to the QP object. In the post-handler stage we commit
the QP's uobject and unlock the dependencies (assuming the handler
increased
the required refcounts).
Destroying an object is similar. The only different is that the
destruction itself isn't done by the handler, but in the
post-handler's code (to share this code between
regular "destroy" calls with process tear-down and hardware removal).

>
> Thanks,
> Ram

- Matan

> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH RFC 02/10] IB/core: Add support to finalize objects in one transaction

[Amrani, Ram]

> >> The new ioctl based infrastructure either commits or rollbacks
> >> all objects of the command as one transaction. In order to do
> >> that, we introduce a notion of dealing with a collection of
> >> objects that are related to a specific action.
> >>
> >> This also requires adding a notion of an action and attribute.
> >> An action contains a groups of attributes, where each group
> >> contains several attributes.
> >>
> >> When declaring these actions and attributes, we actually declare
> >> their specifications. When a command is executed, we actually
> >> allocates some space to hold auxiliary information.
> >>
> >> Signed-off-by: Matan Barak <matanb@mellanox.com>
> >> ---
> >
> > Matan, thanks for the RFC!
> >
> > If I got this correctly each object will go through three phases - get, handler, and a put.
> > I don't quite understand how a batch operation, like destroy QPs, can be undone after the handler phase.
> > I do see it working if at first multiple gets are performed and one of them fails.
> > In that case undoing is easy because the handlers weren't invoked yet.
> >
> > In the case were some user-objects failed the operation how is this reflected upwards?
> >
> > Also, I wonder, is there another intention behind batch operations except speed?
> >
> 
> It's really not recommended to batch create/destroy. The reason is
> exactly what you've pointer out.
> If you batch several "destroy objects" and the n'th one fail, you
> can't unwind the successful ones.
> 
> So basically, we want to support a semantic which is similar to what
> we have today - create a single object or destroy a single object.
> In this case, the pre-handler stage locks the dependencies of this
> object (for example, in create_qp you lock the pd and cq so they won't
> be destroyed)
> and create the uobject for the QP. The handler itself can assume the
> requirements it stated in the specifications are filled and just
> create the QP and
> tie the uobject to the QP object. In the post-handler stage we commit
> the QP's uobject and unlock the dependencies (assuming the handler
> increased
> the required refcounts).
> Destroying an object is similar. The only different is that the
> destruction itself isn't done by the handler, but in the
> post-handler's code (to share this code between
> regular "destroy" calls with process tear-down and hardware removal).
 
I understand there are two sets of objects here. Let's make sure I'm not confusing them.

(1) A collection of user-objects passed via ioctl. This is indicated in the first paragraph.
But as you indicated now we shouldn't support this. So why (apparently) do we?

(2) A collection of user-objects that should be locked for the creation/deletion/modification
of another that was requested via ioctl.
In this case the handler doesn't need to be invoked at all for the collection.
We can easily roll back the "get" phase, if failed during.
We don't expect the "put" phase to fail, if it will fail for some reason. Then this won't really
be handled as the handler was already invoked.

Thanks,
Ram

Re: [PATCH RFC 02/10] IB/core: Add support to finalize objects in one transaction

[Matan Barak]

On Sun, May 7, 2017 at 3:02 PM, Amrani, Ram <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
>> >> The new ioctl based infrastructure either commits or rollbacks
>> >> all objects of the command as one transaction. In order to do
>> >> that, we introduce a notion of dealing with a collection of
>> >> objects that are related to a specific action.
>> >>
>> >> This also requires adding a notion of an action and attribute.
>> >> An action contains a groups of attributes, where each group
>> >> contains several attributes.
>> >>
>> >> When declaring these actions and attributes, we actually declare
>> >> their specifications. When a command is executed, we actually
>> >> allocates some space to hold auxiliary information.
>> >>
>> >> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>> >> ---
>> >
>> > Matan, thanks for the RFC!
>> >
>> > If I got this correctly each object will go through three phases - get, handler, and a put.
>> > I don't quite understand how a batch operation, like destroy QPs, can be undone after the handler phase.
>> > I do see it working if at first multiple gets are performed and one of them fails.
>> > In that case undoing is easy because the handlers weren't invoked yet.
>> >
>> > In the case were some user-objects failed the operation how is this reflected upwards?
>> >
>> > Also, I wonder, is there another intention behind batch operations except speed?
>> >
>>
>> It's really not recommended to batch create/destroy. The reason is
>> exactly what you've pointer out.
>> If you batch several "destroy objects" and the n'th one fail, you
>> can't unwind the successful ones.
>>
>> So basically, we want to support a semantic which is similar to what
>> we have today - create a single object or destroy a single object.
>> In this case, the pre-handler stage locks the dependencies of this
>> object (for example, in create_qp you lock the pd and cq so they won't
>> be destroyed)
>> and create the uobject for the QP. The handler itself can assume the
>> requirements it stated in the specifications are filled and just
>> create the QP and
>> tie the uobject to the QP object. In the post-handler stage we commit
>> the QP's uobject and unlock the dependencies (assuming the handler
>> increased
>> the required refcounts).
>> Destroying an object is similar. The only different is that the
>> destruction itself isn't done by the handler, but in the
>> post-handler's code (to share this code between
>> regular "destroy" calls with process tear-down and hardware removal).
>
> I understand there are two sets of objects here. Let's make sure I'm not confusing them.
>
> (1) A collection of user-objects passed via ioctl. This is indicated in the first paragraph.
> But as you indicated now we shouldn't support this. So why (apparently) do we?
>
> (2) A collection of user-objects that should be locked for the creation/deletion/modification
> of another that was requested via ioctl.
> In this case the handler doesn't need to be invoked at all for the collection.
> We can easily roll back the "get" phase, if failed during.
> We don't expect the "put" phase to fail, if it will fail for some reason. Then this won't really
> be handled as the handler was already invoked.
>

The infrastructure is agnostic to whether the objects are common or
driver specific.
It actually gives you a (hopefully) convenient way to invoke verbs
handlers in the kernel.
Each handler is a function which could have some arguments. However,
we can't pass kernel pointers
from user-space to kernel and we can't trust the user-space from
executing two calls concurrently that
could use an object and destroy it at the same time. Currently
(current infrastructure), when you write such a handler, you need
to open code this yourself - map ids to objects and lock them.

What we propose here is to have some additional info to the handler.
This info could be thought as the function's deceleration.
This additional info makes the infrastructure validates syntactically
your attributes, map them to the actual kernel pointers and lock them.
Since the kernel developer writes this "additional info"
(specification), it can make sure only one "DESTROY" or "CREATE"
object exists per a specification (to avoid the behavior you
mentioned).

A command handling always consists of 3 stages: pre, handler and post
(for all handlers).
So, overall, if you have a create based command, the "pre" stage
creates the uobject and locks its dependencies. If the handler fails,
this is totally reversible (unlock dependencies and destroy the
uobject).
If you have a destroy command, the "pre" stage locks the uobject for
exclusive access. If the handler fails, it just unlocks the object. If
it's successful, the "post" stage actually destroys it.
In other commands, the "pre" stage just locks the uobjects and
obviously it's reversible.

I hope that answers your questions.

> Thanks,
> Ram
>
>

- Matan
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH RFC 02/10] IB/core: Add support to finalize objects in one transaction

[Amrani, Ram]

> The infrastructure is agnostic to whether the objects are common or
> driver specific.
> It actually gives you a (hopefully) convenient way to invoke verbs
> handlers in the kernel.
> Each handler is a function which could have some arguments. However,
> we can't pass kernel pointers
> from user-space to kernel and we can't trust the user-space from
> executing two calls concurrently that
> could use an object and destroy it at the same time. Currently
> (current infrastructure), when you write such a handler, you need
> to open code this yourself - map ids to objects and lock them.
> 
> What we propose here is to have some additional info to the handler.
> This info could be thought as the function's deceleration.
> This additional info makes the infrastructure validates syntactically
> your attributes, map them to the actual kernel pointers and lock them.
> Since the kernel developer writes this "additional info"
> (specification), it can make sure only one "DESTROY" or "CREATE"
> object exists per a specification (to avoid the behavior you
> mentioned).
> 
> A command handling always consists of 3 stages: pre, handler and post
> (for all handlers).
> So, overall, if you have a create based command, the "pre" stage
> creates the uobject and locks its dependencies. If the handler fails,
> this is totally reversible (unlock dependencies and destroy the
> uobject).
> If you have a destroy command, the "pre" stage locks the uobject for
> exclusive access. If the handler fails, it just unlocks the object. If
> it's successful, the "post" stage actually destroys it.
> In other commands, the "pre" stage just locks the uobjects and
> obviously it's reversible.
> 
> I hope that answers your questions.
> 

I'm in sync with you.
Thanks for further explaining.

- Ram

[PATCH RFC 03/10] IB/core: Add new ioctl interface

[Matan Barak]

In this ioctl interface, processing the command starts from
properties of the command and fetching the appropriate user objects
before calling the handler.

Parsing and validation is done according to a specifier declared by
the driver's code. In the driver, all supported types are declared.
These types are separated to different type groups, each could be
declared in a different place (for example, common types and driver
specific types).

For each type we list all supported actions. Similarly to types,
actions are separated to actions groups too. Each group is declared
separately. This could be used in order to add actions to an existing
type.

Each action has a specifies a handler, which could be either a
standard command or a driver specific command.
Along with the handler, a group of attributes is specified as well.
This group lists all supported attributes and is used for automatic
fetching and validation of the command, response and its related
objects.

When a group of elements is used, the high bits of the elements ids
are used in order to calculate the group index. Then, these high bits
are masked out in order to have a zero based namespace for every
group. This is mandatory for compact representation and O(1) array
access.

A group of attributes is actually an array of attributes. Each
attribute has a type (PTR_IN, PTR_OUT, IDR and FD) and a length.
Attributes could be validated through some attributes, like:
(*) Minimum size / Exact size
(*) Fops for FD
(*) Object type for IDR

If an IDR/fd attribute is specified, the kernel also states the object
type and the required access (NEW, WRITE, READ or DESTROY).
All uobject/fd management is done automatically by the infrastructure,
meaning - the infrastructure will fail concurrent commands that at
least one of them requires concurrent access (WRITE/DESTROY),
synchronize actions with device removals (dissociate context events)
and take care of reference counting (increase/decrease) for concurrent
actions invocation. The reference counts on the actual kernel objects
shall be handled by the handlers.

 types
+--------+
|        |
|        |   actions                                                                +--------+
|        |   group      action      action_spec                           +-----+   |len     |
+--------+  +------+[d]+-------+   +----------------+[d]+------------+    |attr1+-> |type    |
| type   +> |action+-> | spec  +-> +  attr_groups   +-> |common sec  +--> +-----+   |idr_type|
+--------+  +------+   |handler|   |                |   +------------+    |attr2|   |access  |
|        |  |      |   +-------+   +----------------+   |device sec  |    +-----+   +--------+
|        |  |      |                                    +------------+
|        |  +------+
|        |
|        |
|        |
|        |
|        |
|        |
|        |
|        |
|        |
|        |
+--------+

[d] = distribute ids to groups using the high order bits

The right types table is also chosen by using the high bits from
uverbs_types_groups.

Once validation and object fetching (or creation) completed, we call
the handler:
int (*handler)(struct ib_device *ib_dev, struct ib_ucontext *ucontext,
               struct uverbs_attr_array *ctx, size_t num);

Where ctx is an array of uverbs_attr_array. Each element in this array
is an array of attributes which corresponds to one group of attributes.
For example, in the usually used case:

 ctx                               core
+----------------------------+     +------------+
| core: uverbs_attr_array    +---> | valid      |
+----------------------------+     | cmd_attr   |
| driver: uverbs_attr_array  |     +------------+
|----------------------------+--+  | valid      |
                                |  | cmd_attr   |
                                |  +------------+
                                |  | valid      |
                                |  | obj_attr   |
                                |  +------------+
                                |
                                |  vendor
                                |  +------------+
                                +> | valid      |
                                   | cmd_attr   |
                                   +------------+
                                   | valid      |
                                   | cmd_attr   |
                                   +------------+
                                   | valid      |
                                   | obj_attr   |
                                   +------------+

Ctx array's indices corresponds to the attributes groups order. The indices
of core and driver corresponds to the attributes name spaces of each
group. Thus, we could think of the following as one object:
1. Set of attribute specification (with their attribute IDs)
2. Attribute group which owns (1) specifications
3. A function which could handle this attributes which the handler
   could call
4. The allocation descriptor of this type uverbs_obj_type.

Upon success of a handler invocation, reference count of uobjects and
use count will be a updated automatically according to the
specification.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/Makefile       |   2 +-
 drivers/infiniband/core/rdma_core.c    |  45 +++++
 drivers/infiniband/core/rdma_core.h    |   5 +
 drivers/infiniband/core/uverbs_ioctl.c | 351 +++++++++++++++++++++++++++++++++
 include/rdma/ib_verbs.h                |   2 +
 include/rdma/uverbs_ioctl.h            |  65 ++++--
 include/uapi/rdma/rdma_user_ioctl.h    |  25 +++
 7 files changed, 481 insertions(+), 14 deletions(-)
 create mode 100644 drivers/infiniband/core/uverbs_ioctl.c

diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
index 6ebd9ad..e18f2f8 100644
--- a/drivers/infiniband/core/Makefile
+++ b/drivers/infiniband/core/Makefile
@@ -30,4 +30,4 @@ ib_umad-y :=			user_mad.o
 ib_ucm-y :=			ucm.o
 
 ib_uverbs-y :=			uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
-				rdma_core.o uverbs_std_types.o
+				rdma_core.o uverbs_std_types.o uverbs_ioctl.o
diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
index 78ffd8c..a6e35b3 100644
--- a/drivers/infiniband/core/rdma_core.c
+++ b/drivers/infiniband/core/rdma_core.c
@@ -40,6 +40,51 @@
 #include "core_priv.h"
 #include "rdma_core.h"
 
+int uverbs_group_idx(u16 *id, unsigned int ngroups)
+{
+	int ret = (*id & UVERBS_ID_RESERVED_MASK) >> UVERBS_ID_RESERVED_SHIFT;
+
+	if (ret >= ngroups)
+		return -EINVAL;
+
+	*id &= ~UVERBS_ID_RESERVED_MASK;
+	return ret;
+}
+
+const struct uverbs_type *uverbs_get_type(const struct ib_device *ibdev,
+					  uint16_t type)
+{
+	const struct uverbs_root *groups = ibdev->specs_root;
+	const struct uverbs_type_group *types;
+	int ret = uverbs_group_idx(&type, groups->num_groups);
+
+	if (ret < 0)
+		return NULL;
+
+	types = groups->type_groups[ret];
+
+	if (type >= types->num_types)
+		return NULL;
+
+	return types->types[type];
+}
+
+const struct uverbs_action *uverbs_get_action(const struct uverbs_type *type,
+					      uint16_t action)
+{
+	const struct uverbs_action_group *action_group;
+	int ret = uverbs_group_idx(&action, type->num_groups);
+
+	if (ret < 0)
+		return NULL;
+
+	action_group = type->action_groups[ret];
+	if (action >= action_group->num_actions)
+		return NULL;
+
+	return action_group->actions[action];
+}
+
 void uverbs_uobject_get(struct ib_uobject *uobject)
 {
 	kref_get(&uobject->ref);
diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
index 0aebc47..82db2bc 100644
--- a/drivers/infiniband/core/rdma_core.h
+++ b/drivers/infiniband/core/rdma_core.h
@@ -43,6 +43,11 @@
 #include <rdma/ib_verbs.h>
 #include <linux/mutex.h>
 
+int uverbs_group_idx(u16 *id, unsigned int ngroups);
+const struct uverbs_type *uverbs_get_type(const struct ib_device *ibdev,
+					  uint16_t type);
+const struct uverbs_action *uverbs_get_action(const struct uverbs_type *type,
+					      uint16_t action);
 /*
  * These functions initialize the context and cleanups its uobjects.
  * The context has a list of objects which is protected by a mutex
diff --git a/drivers/infiniband/core/uverbs_ioctl.c b/drivers/infiniband/core/uverbs_ioctl.c
new file mode 100644
index 0000000..3465a18
--- /dev/null
+++ b/drivers/infiniband/core/uverbs_ioctl.c
@@ -0,0 +1,351 @@
+/*
+ * Copyright (c) 2017, Mellanox Technologies inc.  All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <rdma/rdma_user_ioctl.h>
+#include <rdma/uverbs_ioctl.h>
+#include "rdma_core.h"
+#include "uverbs.h"
+
+static int uverbs_process_attr(struct ib_device *ibdev,
+			       struct ib_ucontext *ucontext,
+			       const struct ib_uverbs_attr *uattr,
+			       u16 attr_id,
+			       const struct uverbs_attr_spec_group *attr_spec_group,
+			       struct uverbs_attr_array *attr_array,
+			       struct ib_uverbs_attr __user *uattr_ptr)
+{
+	const struct uverbs_attr_spec *spec;
+	struct uverbs_attr *e;
+	const struct uverbs_type *type;
+	struct uverbs_obj_attr *o_attr;
+	struct uverbs_attr *elements = attr_array->attrs;
+
+	if (uattr->reserved)
+		return -EINVAL;
+
+	if (attr_id >= attr_spec_group->num_attrs) {
+		if (uattr->flags & UVERBS_ATTR_F_MANDATORY)
+			return -EINVAL;
+		else
+			return 0;
+	}
+
+	spec = &attr_spec_group->attrs[attr_id];
+	e = &elements[attr_id];
+
+	switch (spec->type) {
+	case UVERBS_ATTR_TYPE_PTR_IN:
+	case UVERBS_ATTR_TYPE_PTR_OUT:
+		if (uattr->len < spec->len ||
+		    (!(spec->flags & UVERBS_ATTR_SPEC_F_MIN_SZ) &&
+		     uattr->len > spec->len))
+			return -EINVAL;
+
+		e->ptr_attr.ptr = (void * __user)uattr->data;
+		e->ptr_attr.len = uattr->len;
+		break;
+
+	case UVERBS_ATTR_TYPE_IDR:
+		if (uattr->data >> 32)
+			return -EINVAL;
+	/* fall through */
+	case UVERBS_ATTR_TYPE_FD:
+		if (uattr->len != 0 || !ucontext || uattr->data > INT_MAX)
+			return -EINVAL;
+
+		o_attr = &e->obj_attr;
+		type = uverbs_get_type(ibdev, spec->obj.obj_type);
+		if (!type)
+			return -EINVAL;
+		o_attr->type = type->type_attrs;
+		o_attr->uattr = uattr_ptr;
+
+		o_attr->id = (int)uattr->data;
+		o_attr->uobject = uverbs_get_uobject_from_context(
+					o_attr->type,
+					ucontext,
+					spec->obj.access,
+					o_attr->id);
+
+		if (IS_ERR(o_attr->uobject))
+			return -EINVAL;
+
+		if (spec->obj.access == UVERBS_ACCESS_NEW) {
+			u64 id = o_attr->uobject->id;
+
+			if (put_user(id, &o_attr->uattr->data)) {
+				uverbs_finalize_object(o_attr->uobject,
+						       UVERBS_ACCESS_NEW,
+						       false);
+				return -EFAULT;
+			}
+		}
+
+		break;
+	default:
+		return -EOPNOTSUPP;
+	};
+
+	set_bit(attr_id, attr_array->valid_bitmap);
+	return 0;
+}
+
+static int uverbs_uattrs_process(struct ib_device *ibdev,
+				 struct ib_ucontext *ucontext,
+				 const struct ib_uverbs_attr *uattrs,
+				 size_t num_uattrs,
+				 const struct uverbs_action *action,
+				 struct uverbs_attr_array *attr_array,
+				 struct ib_uverbs_attr __user *uattr_ptr)
+{
+	size_t i;
+	int ret = 0;
+	int num_given_groups = 0;
+
+	for (i = 0; i < num_uattrs; i++) {
+		const struct ib_uverbs_attr *uattr = &uattrs[i];
+		u16 attr_id = uattr->attr_id;
+		const struct uverbs_attr_spec_group *attr_spec_group;
+
+		ret = uverbs_group_idx(&attr_id, action->num_groups);
+		if (ret < 0) {
+			if (uattr->flags & UVERBS_ATTR_F_MANDATORY)
+				return ret;
+
+			continue;
+		}
+
+		if (ret >= num_given_groups)
+			num_given_groups = ret + 1;
+
+		attr_spec_group = action->attr_groups[ret];
+		ret = uverbs_process_attr(ibdev, ucontext, uattr, attr_id,
+					  attr_spec_group, &attr_array[ret],
+					  uattr_ptr++);
+		if (ret) {
+			uverbs_finalize_objects(attr_array,
+						num_given_groups,
+						action, false);
+			return ret;
+		}
+	}
+
+	return ret ?: num_given_groups;
+}
+
+static int uverbs_validate_kernel_mandatory(const struct uverbs_action *action,
+					    struct uverbs_attr_array *attr_array,
+					    unsigned int num_given_groups)
+{
+	unsigned int i;
+
+	for (i = 0; i < num_given_groups; i++) {
+		const struct uverbs_attr_spec_group *attr_spec_group =
+			action->attr_groups[i];
+
+		if (!bitmap_subset(attr_spec_group->mandatory_attrs_bitmask,
+				   attr_array[i].valid_bitmap,
+				   attr_spec_group->num_attrs))
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int uverbs_handle_action(struct ib_uverbs_attr __user *uattr_ptr,
+				const struct ib_uverbs_attr *uattrs,
+				size_t num_uattrs,
+				struct ib_device *ibdev,
+				struct ib_uverbs_file *ufile,
+				const struct uverbs_action *action,
+				struct uverbs_attr_array *attr_array)
+{
+	int ret;
+	int finalize_ret;
+	int num_given_groups;
+
+	num_given_groups = uverbs_uattrs_process(ibdev, ufile->ucontext, uattrs,
+						 num_uattrs, action, attr_array,
+						 uattr_ptr);
+	if (num_given_groups <= 0)
+		return -EINVAL;
+
+	ret = uverbs_validate_kernel_mandatory(action, attr_array,
+					       num_given_groups);
+	if (ret)
+		goto cleanup;
+
+	ret = action->handler(ibdev, ufile, attr_array, num_given_groups);
+cleanup:
+	finalize_ret = uverbs_finalize_objects(attr_array, num_given_groups,
+					       action, !ret);
+
+	return ret ? ret : finalize_ret;
+}
+
+#define UVERBS_OPTIMIZE_USING_STACK_SZ  256
+long ib_uverbs_cmd_verbs(struct ib_device *ib_dev,
+			 struct ib_uverbs_file *file,
+			 struct ib_uverbs_ioctl_hdr *hdr,
+			 void __user *buf)
+{
+	const struct uverbs_type *type;
+	const struct uverbs_action *action;
+	long err = 0;
+	unsigned int i;
+	struct {
+		struct ib_uverbs_attr		*uattrs;
+		struct uverbs_attr_array	*uverbs_attr_array;
+	} *ctx = NULL;
+	struct uverbs_attr *curr_attr;
+	unsigned long *curr_bitmap;
+	size_t ctx_size;
+#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
+	uintptr_t data[UVERBS_OPTIMIZE_USING_STACK_SZ / sizeof(uintptr_t)];
+#endif
+
+	if (hdr->reserved)
+		return -EINVAL;
+
+	type = uverbs_get_type(ib_dev, hdr->object_type);
+	if (!type)
+		return -EOPNOTSUPP;
+
+	action = uverbs_get_action(type, hdr->action);
+	if (!action)
+		return -EOPNOTSUPP;
+
+	if ((action->flags & UVERBS_ACTION_FLAG_CREATE_ROOT) ^ !file->ucontext)
+		return -EINVAL;
+
+	ctx_size = sizeof(*ctx) +
+		   sizeof(struct uverbs_attr_array) * action->num_groups +
+		   sizeof(*ctx->uattrs) * hdr->num_attrs +
+		   sizeof(*ctx->uverbs_attr_array->attrs) *
+		   action->num_child_attrs +
+		   sizeof(*ctx->uverbs_attr_array->valid_bitmap) *
+			(action->num_child_attrs / BITS_PER_LONG +
+			 action->num_groups);
+
+#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
+	if (ctx_size <= UVERBS_OPTIMIZE_USING_STACK_SZ)
+		ctx = (void *)data;
+
+	if (!ctx)
+#endif
+	ctx = kmalloc(ctx_size, GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	ctx->uverbs_attr_array = (void *)ctx + sizeof(*ctx);
+	ctx->uattrs = (void *)(ctx->uverbs_attr_array +
+			       action->num_groups);
+	curr_attr = (void *)(ctx->uattrs + hdr->num_attrs);
+	curr_bitmap = (void *)(curr_attr + action->num_child_attrs);
+
+	/*
+	 * We just fill the pointers and num_attrs here. The data itself will be
+	 * filled at a later stage (uverbs_process_attr)
+	 */
+	for (i = 0; i < action->num_groups; i++) {
+		unsigned int curr_num_attrs = action->attr_groups[i]->num_attrs;
+
+		ctx->uverbs_attr_array[i].attrs = curr_attr;
+		curr_attr += curr_num_attrs;
+		ctx->uverbs_attr_array[i].num_attrs = curr_num_attrs;
+		ctx->uverbs_attr_array[i].valid_bitmap = curr_bitmap;
+		bitmap_zero(curr_bitmap, curr_num_attrs);
+		curr_bitmap += BITS_TO_LONGS(curr_num_attrs);
+	}
+
+	err = copy_from_user(ctx->uattrs, buf,
+			     sizeof(*ctx->uattrs) * hdr->num_attrs);
+	if (err) {
+		err = -EFAULT;
+		goto out;
+	}
+
+	err = uverbs_handle_action(buf, ctx->uattrs, hdr->num_attrs, ib_dev,
+				   file, action, ctx->uverbs_attr_array);
+out:
+#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
+	if (ctx_size > UVERBS_OPTIMIZE_USING_STACK_SZ)
+#endif
+	kfree(ctx);
+	return err;
+}
+
+#define IB_UVERBS_MAX_CMD_SZ 4096
+
+long ib_uverbs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
+{
+	struct ib_uverbs_file *file = filp->private_data;
+	struct ib_uverbs_ioctl_hdr __user *user_hdr =
+		(struct ib_uverbs_ioctl_hdr __user *)arg;
+	struct ib_uverbs_ioctl_hdr hdr;
+	struct ib_device *ib_dev;
+	int srcu_key;
+	long err;
+
+	srcu_key = srcu_read_lock(&file->device->disassociate_srcu);
+	ib_dev = srcu_dereference(file->device->ib_dev,
+				  &file->device->disassociate_srcu);
+	if (!ib_dev) {
+		err = -EIO;
+		goto out;
+	}
+
+	if (cmd == RDMA_VERBS_IOCTL) {
+		err = copy_from_user(&hdr, user_hdr, sizeof(hdr));
+
+		if (err || hdr.length > IB_UVERBS_MAX_CMD_SZ ||
+		    hdr.length != sizeof(hdr) + hdr.num_attrs * sizeof(struct ib_uverbs_attr)) {
+			err = -EINVAL;
+			goto out;
+		}
+
+		/* currently there are no flags supported */
+		if (hdr.flags) {
+			err = -EOPNOTSUPP;
+			goto out;
+		}
+
+		err = ib_uverbs_cmd_verbs(ib_dev, file, &hdr,
+					  (__user void *)arg + sizeof(hdr));
+	} else {
+		err = -ENOIOCTLCMD;
+	}
+out:
+	srcu_read_unlock(&file->device->disassociate_srcu, srcu_key);
+
+	return err;
+}
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 3a8e058..44cd98b 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2165,6 +2165,8 @@ struct ib_device {
 	 */
 	int (*get_port_immutable)(struct ib_device *, u8, struct ib_port_immutable *);
 	void (*get_dev_fw_str)(struct ib_device *, char *str, size_t str_len);
+
+	struct uverbs_root                      *specs_root;
 };
 
 struct ib_client {
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index 1f84f30..71a6b84 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -41,8 +41,13 @@
  * =======================================
  */
 
+#define UVERBS_ID_RESERVED_MASK 0xF000
+#define UVERBS_ID_RESERVED_SHIFT 12
+
 enum uverbs_attr_type {
 	UVERBS_ATTR_TYPE_NA,
+	UVERBS_ATTR_TYPE_PTR_IN,
+	UVERBS_ATTR_TYPE_PTR_OUT,
 	UVERBS_ATTR_TYPE_IDR,
 	UVERBS_ATTR_TYPE_FD,
 };
@@ -54,8 +59,14 @@ enum uverbs_idr_access {
 	UVERBS_ACCESS_DESTROY
 };
 
+enum uverbs_attr_spec_flags {
+	UVERBS_ATTR_SPEC_F_MANDATORY	= 1U << 0,
+	UVERBS_ATTR_SPEC_F_MIN_SZ	= 1U << 1,
+};
+
 struct uverbs_attr_spec {
 	enum uverbs_attr_type		type;
+	u8				flags;
 	union {
 		u16				len;
 		struct {
@@ -68,11 +79,45 @@ struct uverbs_attr_spec {
 struct uverbs_attr_spec_group {
 	struct uverbs_attr_spec		*attrs;
 	size_t				num_attrs;
+	/* populate at runtime */
+	unsigned long			*mandatory_attrs_bitmask;
+};
+
+struct uverbs_attr_array;
+struct ib_uverbs_file;
+
+enum uverbs_action_flags {
+	UVERBS_ACTION_FLAG_CREATE_ROOT = 1 << 0,
 };
 
 struct uverbs_action {
-	const struct uverbs_attr_spec_group		**attr_groups;
+	struct uverbs_attr_spec_group			**attr_groups;
 	size_t						num_groups;
+	size_t						num_child_attrs;
+	u32 flags;
+	int (*handler)(struct ib_device *ib_dev, struct ib_uverbs_file *ufile,
+		       struct uverbs_attr_array *ctx, size_t num);
+};
+
+struct uverbs_action_group {
+	size_t					num_actions;
+	struct uverbs_action			**actions;
+};
+
+struct uverbs_type {
+	size_t					num_groups;
+	const struct uverbs_action_group	**action_groups;
+	const struct uverbs_obj_type		*type_attrs;
+};
+
+struct uverbs_type_group {
+	size_t					num_types;
+	const struct uverbs_type		**types;
+};
+
+struct uverbs_root {
+	const struct uverbs_type_group		**type_groups;
+	size_t					num_groups;
 };
 
 /* =================================================
@@ -80,28 +125,22 @@ struct uverbs_action {
  * =================================================
  */
 
-struct uverbs_fd_attr {
-	int		fd;
-};
-
-struct uverbs_uobj_attr {
-	/*  idr handle */
-	u32	idr;
+struct uverbs_ptr_attr {
+	void	* __user ptr;
+	u16		len;
 };
 
 struct uverbs_obj_attr {
 	/* pointer to the kernel descriptor -> type, access, etc */
 	struct ib_uverbs_attr __user	*uattr;
-	const struct uverbs_type_alloc_action	*type;
+	const struct uverbs_obj_type	*type;
 	struct ib_uobject		*uobject;
-	union {
-		struct uverbs_fd_attr		fd;
-		struct uverbs_uobj_attr		uobj;
-	};
+	int				id;
 };
 
 struct uverbs_attr {
 	union {
+		struct uverbs_ptr_attr	ptr_attr;
 		struct uverbs_obj_attr	obj_attr;
 	};
 };
diff --git a/include/uapi/rdma/rdma_user_ioctl.h b/include/uapi/rdma/rdma_user_ioctl.h
index 9388125..12663f6 100644
--- a/include/uapi/rdma/rdma_user_ioctl.h
+++ b/include/uapi/rdma/rdma_user_ioctl.h
@@ -43,6 +43,31 @@
 /* Legacy name, for user space application which already use it */
 #define IB_IOCTL_MAGIC		RDMA_IOCTL_MAGIC
 
+#define RDMA_VERBS_IOCTL \
+	_IOWR(RDMA_IOCTL_MAGIC, 1, struct ib_uverbs_ioctl_hdr)
+
+enum ib_uverbs_attr_flags {
+	UVERBS_ATTR_F_MANDATORY = 1U << 0,
+};
+
+struct ib_uverbs_attr {
+	__u16 attr_id;		/* command specific type attribute */
+	__u16 len;		/* NA for idr */
+	__u16 flags;		/* combination of uverbs_attr_flags */
+	__u16 reserved;
+	__u64 data;		/* ptr to command, inline data or idr/fd */
+};
+
+struct ib_uverbs_ioctl_hdr {
+	__u16 length;
+	__u16 flags;
+	__u16 object_type;
+	__u16 reserved;		/* future use for driver_id */
+	__u16 action;
+	__u16 num_attrs;
+	struct ib_uverbs_attr  attrs[0];
+};
+
 /*
  * General blocks assignments
  * It is closed on purpose do not expose it it user space
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC 03/10] IB/core: Add new ioctl interface

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 24432 bytes --]

On Wed, Apr 19, 2017 at 06:20:18PM +0300, Matan Barak wrote:
> In this ioctl interface, processing the command starts from
> properties of the command and fetching the appropriate user objects
> before calling the handler.
>
> Parsing and validation is done according to a specifier declared by
> the driver's code. In the driver, all supported types are declared.
> These types are separated to different type groups, each could be
> declared in a different place (for example, common types and driver
> specific types).
>
> For each type we list all supported actions. Similarly to types,
> actions are separated to actions groups too. Each group is declared
> separately. This could be used in order to add actions to an existing
> type.
>
> Each action has a specifies a handler, which could be either a
> standard command or a driver specific command.
> Along with the handler, a group of attributes is specified as well.
> This group lists all supported attributes and is used for automatic
> fetching and validation of the command, response and its related
> objects.
>
> When a group of elements is used, the high bits of the elements ids
> are used in order to calculate the group index. Then, these high bits
> are masked out in order to have a zero based namespace for every
> group. This is mandatory for compact representation and O(1) array
> access.
>
> A group of attributes is actually an array of attributes. Each
> attribute has a type (PTR_IN, PTR_OUT, IDR and FD) and a length.
> Attributes could be validated through some attributes, like:
> (*) Minimum size / Exact size
> (*) Fops for FD
> (*) Object type for IDR
>
> If an IDR/fd attribute is specified, the kernel also states the object
> type and the required access (NEW, WRITE, READ or DESTROY).
> All uobject/fd management is done automatically by the infrastructure,
> meaning - the infrastructure will fail concurrent commands that at
> least one of them requires concurrent access (WRITE/DESTROY),
> synchronize actions with device removals (dissociate context events)
> and take care of reference counting (increase/decrease) for concurrent
> actions invocation. The reference counts on the actual kernel objects
> shall be handled by the handlers.
>
>  types
> +--------+
> |        |
> |        |   actions                                                                +--------+
> |        |   group      action      action_spec                           +-----+   |len     |
> +--------+  +------+[d]+-------+   +----------------+[d]+------------+    |attr1+-> |type    |
> | type   +> |action+-> | spec  +-> +  attr_groups   +-> |common sec  +--> +-----+   |idr_type|
> +--------+  +------+   |handler|   |                |   +------------+    |attr2|   |access  |
> |        |  |      |   +-------+   +----------------+   |device sec  |    +-----+   +--------+
> |        |  |      |                                    +------------+
> |        |  +------+
> |        |
> |        |
> |        |
> |        |
> |        |
> |        |
> |        |
> |        |
> |        |
> |        |
> +--------+
>
> [d] = distribute ids to groups using the high order bits
>
> The right types table is also chosen by using the high bits from
> uverbs_types_groups.
>
> Once validation and object fetching (or creation) completed, we call
> the handler:
> int (*handler)(struct ib_device *ib_dev, struct ib_ucontext *ucontext,
>                struct uverbs_attr_array *ctx, size_t num);
>
> Where ctx is an array of uverbs_attr_array. Each element in this array
> is an array of attributes which corresponds to one group of attributes.
> For example, in the usually used case:
>
>  ctx                               core
> +----------------------------+     +------------+
> | core: uverbs_attr_array    +---> | valid      |
> +----------------------------+     | cmd_attr   |
> | driver: uverbs_attr_array  |     +------------+
> |----------------------------+--+  | valid      |
>                                 |  | cmd_attr   |
>                                 |  +------------+
>                                 |  | valid      |
>                                 |  | obj_attr   |
>                                 |  +------------+
>                                 |
>                                 |  vendor
>                                 |  +------------+
>                                 +> | valid      |
>                                    | cmd_attr   |
>                                    +------------+
>                                    | valid      |
>                                    | cmd_attr   |
>                                    +------------+
>                                    | valid      |
>                                    | obj_attr   |
>                                    +------------+
>
> Ctx array's indices corresponds to the attributes groups order. The indices
> of core and driver corresponds to the attributes name spaces of each
> group. Thus, we could think of the following as one object:
> 1. Set of attribute specification (with their attribute IDs)
> 2. Attribute group which owns (1) specifications
> 3. A function which could handle this attributes which the handler
>    could call
> 4. The allocation descriptor of this type uverbs_obj_type.
>
> Upon success of a handler invocation, reference count of uobjects and
> use count will be a updated automatically according to the
> specification.
>
> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
>  drivers/infiniband/core/Makefile       |   2 +-
>  drivers/infiniband/core/rdma_core.c    |  45 +++++
>  drivers/infiniband/core/rdma_core.h    |   5 +
>  drivers/infiniband/core/uverbs_ioctl.c | 351 +++++++++++++++++++++++++++++++++
>  include/rdma/ib_verbs.h                |   2 +
>  include/rdma/uverbs_ioctl.h            |  65 ++++--
>  include/uapi/rdma/rdma_user_ioctl.h    |  25 +++
>  7 files changed, 481 insertions(+), 14 deletions(-)
>  create mode 100644 drivers/infiniband/core/uverbs_ioctl.c
>
> diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
> index 6ebd9ad..e18f2f8 100644
> --- a/drivers/infiniband/core/Makefile
> +++ b/drivers/infiniband/core/Makefile
> @@ -30,4 +30,4 @@ ib_umad-y :=			user_mad.o
>  ib_ucm-y :=			ucm.o
>
>  ib_uverbs-y :=			uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
> -				rdma_core.o uverbs_std_types.o
> +				rdma_core.o uverbs_std_types.o uverbs_ioctl.o
> diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
> index 78ffd8c..a6e35b3 100644
> --- a/drivers/infiniband/core/rdma_core.c
> +++ b/drivers/infiniband/core/rdma_core.c
> @@ -40,6 +40,51 @@
>  #include "core_priv.h"
>  #include "rdma_core.h"
>
> +int uverbs_group_idx(u16 *id, unsigned int ngroups)
> +{
> +	int ret = (*id & UVERBS_ID_RESERVED_MASK) >> UVERBS_ID_RESERVED_SHIFT;
> +
> +	if (ret >= ngroups)
> +		return -EINVAL;
> +
> +	*id &= ~UVERBS_ID_RESERVED_MASK;
> +	return ret;
> +}
> +
> +const struct uverbs_type *uverbs_get_type(const struct ib_device *ibdev,
> +					  uint16_t type)
> +{
> +	const struct uverbs_root *groups = ibdev->specs_root;
> +	const struct uverbs_type_group *types;
> +	int ret = uverbs_group_idx(&type, groups->num_groups);
> +
> +	if (ret < 0)
> +		return NULL;
> +
> +	types = groups->type_groups[ret];
> +
> +	if (type >= types->num_types)
> +		return NULL;
> +
> +	return types->types[type];
> +}
> +
> +const struct uverbs_action *uverbs_get_action(const struct uverbs_type *type,
> +					      uint16_t action)
> +{
> +	const struct uverbs_action_group *action_group;
> +	int ret = uverbs_group_idx(&action, type->num_groups);
> +
> +	if (ret < 0)
> +		return NULL;
> +
> +	action_group = type->action_groups[ret];
> +	if (action >= action_group->num_actions)
> +		return NULL;
> +
> +	return action_group->actions[action];
> +}
> +
>  void uverbs_uobject_get(struct ib_uobject *uobject)
>  {
>  	kref_get(&uobject->ref);
> diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
> index 0aebc47..82db2bc 100644
> --- a/drivers/infiniband/core/rdma_core.h
> +++ b/drivers/infiniband/core/rdma_core.h
> @@ -43,6 +43,11 @@
>  #include <rdma/ib_verbs.h>
>  #include <linux/mutex.h>
>
> +int uverbs_group_idx(u16 *id, unsigned int ngroups);
> +const struct uverbs_type *uverbs_get_type(const struct ib_device *ibdev,
> +					  uint16_t type);
> +const struct uverbs_action *uverbs_get_action(const struct uverbs_type *type,
> +					      uint16_t action);
>  /*
>   * These functions initialize the context and cleanups its uobjects.
>   * The context has a list of objects which is protected by a mutex
> diff --git a/drivers/infiniband/core/uverbs_ioctl.c b/drivers/infiniband/core/uverbs_ioctl.c
> new file mode 100644
> index 0000000..3465a18
> --- /dev/null
> +++ b/drivers/infiniband/core/uverbs_ioctl.c
> @@ -0,0 +1,351 @@
> +/*
> + * Copyright (c) 2017, Mellanox Technologies inc.  All rights reserved.
> + *
> + * This software is available to you under a choice of one of two
> + * licenses.  You may choose to be licensed under the terms of the GNU
> + * General Public License (GPL) Version 2, available from the file
> + * COPYING in the main directory of this source tree, or the
> + * OpenIB.org BSD license below:
> + *
> + *     Redistribution and use in source and binary forms, with or
> + *     without modification, are permitted provided that the following
> + *     conditions are met:
> + *
> + *      - Redistributions of source code must retain the above
> + *        copyright notice, this list of conditions and the following
> + *        disclaimer.
> + *
> + *      - Redistributions in binary form must reproduce the above
> + *        copyright notice, this list of conditions and the following
> + *        disclaimer in the documentation and/or other materials
> + *        provided with the distribution.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> + * SOFTWARE.
> + */
> +
> +#include <rdma/rdma_user_ioctl.h>
> +#include <rdma/uverbs_ioctl.h>
> +#include "rdma_core.h"
> +#include "uverbs.h"
> +
> +static int uverbs_process_attr(struct ib_device *ibdev,
> +			       struct ib_ucontext *ucontext,
> +			       const struct ib_uverbs_attr *uattr,
> +			       u16 attr_id,
> +			       const struct uverbs_attr_spec_group *attr_spec_group,
> +			       struct uverbs_attr_array *attr_array,
> +			       struct ib_uverbs_attr __user *uattr_ptr)
> +{
> +	const struct uverbs_attr_spec *spec;
> +	struct uverbs_attr *e;
> +	const struct uverbs_type *type;
> +	struct uverbs_obj_attr *o_attr;
> +	struct uverbs_attr *elements = attr_array->attrs;
> +
> +	if (uattr->reserved)
> +		return -EINVAL;
> +
> +	if (attr_id >= attr_spec_group->num_attrs) {
> +		if (uattr->flags & UVERBS_ATTR_F_MANDATORY)
> +			return -EINVAL;
> +		else
> +			return 0;
> +	}
> +
> +	spec = &attr_spec_group->attrs[attr_id];
> +	e = &elements[attr_id];
> +
> +	switch (spec->type) {
> +	case UVERBS_ATTR_TYPE_PTR_IN:
> +	case UVERBS_ATTR_TYPE_PTR_OUT:
> +		if (uattr->len < spec->len ||
> +		    (!(spec->flags & UVERBS_ATTR_SPEC_F_MIN_SZ) &&
> +		     uattr->len > spec->len))
> +			return -EINVAL;
> +
> +		e->ptr_attr.ptr = (void * __user)uattr->data;
> +		e->ptr_attr.len = uattr->len;
> +		break;
> +
> +	case UVERBS_ATTR_TYPE_IDR:
> +		if (uattr->data >> 32)
> +			return -EINVAL;
> +	/* fall through */
> +	case UVERBS_ATTR_TYPE_FD:
> +		if (uattr->len != 0 || !ucontext || uattr->data > INT_MAX)
> +			return -EINVAL;
> +
> +		o_attr = &e->obj_attr;
> +		type = uverbs_get_type(ibdev, spec->obj.obj_type);
> +		if (!type)
> +			return -EINVAL;
> +		o_attr->type = type->type_attrs;
> +		o_attr->uattr = uattr_ptr;
> +
> +		o_attr->id = (int)uattr->data;
> +		o_attr->uobject = uverbs_get_uobject_from_context(
> +					o_attr->type,
> +					ucontext,
> +					spec->obj.access,
> +					o_attr->id);
> +
> +		if (IS_ERR(o_attr->uobject))
> +			return -EINVAL;
> +
> +		if (spec->obj.access == UVERBS_ACCESS_NEW) {
> +			u64 id = o_attr->uobject->id;
> +
> +			if (put_user(id, &o_attr->uattr->data)) {
> +				uverbs_finalize_object(o_attr->uobject,
> +						       UVERBS_ACCESS_NEW,
> +						       false);
> +				return -EFAULT;
> +			}
> +		}
> +
> +		break;
> +	default:
> +		return -EOPNOTSUPP;
> +	};
> +
> +	set_bit(attr_id, attr_array->valid_bitmap);
> +	return 0;
> +}
> +
> +static int uverbs_uattrs_process(struct ib_device *ibdev,
> +				 struct ib_ucontext *ucontext,
> +				 const struct ib_uverbs_attr *uattrs,
> +				 size_t num_uattrs,
> +				 const struct uverbs_action *action,
> +				 struct uverbs_attr_array *attr_array,
> +				 struct ib_uverbs_attr __user *uattr_ptr)
> +{
> +	size_t i;
> +	int ret = 0;
> +	int num_given_groups = 0;
> +
> +	for (i = 0; i < num_uattrs; i++) {
> +		const struct ib_uverbs_attr *uattr = &uattrs[i];
> +		u16 attr_id = uattr->attr_id;
> +		const struct uverbs_attr_spec_group *attr_spec_group;
> +
> +		ret = uverbs_group_idx(&attr_id, action->num_groups);
> +		if (ret < 0) {
> +			if (uattr->flags & UVERBS_ATTR_F_MANDATORY)
> +				return ret;
> +
> +			continue;
> +		}
> +
> +		if (ret >= num_given_groups)
> +			num_given_groups = ret + 1;
> +
> +		attr_spec_group = action->attr_groups[ret];
> +		ret = uverbs_process_attr(ibdev, ucontext, uattr, attr_id,
> +					  attr_spec_group, &attr_array[ret],
> +					  uattr_ptr++);
> +		if (ret) {
> +			uverbs_finalize_objects(attr_array,
> +						num_given_groups,
> +						action, false);
> +			return ret;
> +		}
> +	}
> +
> +	return ret ?: num_given_groups;
> +}
> +
> +static int uverbs_validate_kernel_mandatory(const struct uverbs_action *action,
> +					    struct uverbs_attr_array *attr_array,
> +					    unsigned int num_given_groups)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < num_given_groups; i++) {
> +		const struct uverbs_attr_spec_group *attr_spec_group =
> +			action->attr_groups[i];
> +
> +		if (!bitmap_subset(attr_spec_group->mandatory_attrs_bitmask,
> +				   attr_array[i].valid_bitmap,
> +				   attr_spec_group->num_attrs))
> +			return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +static int uverbs_handle_action(struct ib_uverbs_attr __user *uattr_ptr,
> +				const struct ib_uverbs_attr *uattrs,
> +				size_t num_uattrs,
> +				struct ib_device *ibdev,
> +				struct ib_uverbs_file *ufile,
> +				const struct uverbs_action *action,
> +				struct uverbs_attr_array *attr_array)
> +{
> +	int ret;
> +	int finalize_ret;
> +	int num_given_groups;
> +
> +	num_given_groups = uverbs_uattrs_process(ibdev, ufile->ucontext, uattrs,
> +						 num_uattrs, action, attr_array,
> +						 uattr_ptr);
> +	if (num_given_groups <= 0)
> +		return -EINVAL;
> +
> +	ret = uverbs_validate_kernel_mandatory(action, attr_array,
> +					       num_given_groups);
> +	if (ret)
> +		goto cleanup;
> +
> +	ret = action->handler(ibdev, ufile, attr_array, num_given_groups);
> +cleanup:
> +	finalize_ret = uverbs_finalize_objects(attr_array, num_given_groups,
> +					       action, !ret);
> +
> +	return ret ? ret : finalize_ret;
> +}
> +
> +#define UVERBS_OPTIMIZE_USING_STACK_SZ  256
> +long ib_uverbs_cmd_verbs(struct ib_device *ib_dev,
> +			 struct ib_uverbs_file *file,
> +			 struct ib_uverbs_ioctl_hdr *hdr,
> +			 void __user *buf)
> +{
> +	const struct uverbs_type *type;
> +	const struct uverbs_action *action;
> +	long err = 0;
> +	unsigned int i;
> +	struct {
> +		struct ib_uverbs_attr		*uattrs;
> +		struct uverbs_attr_array	*uverbs_attr_array;
> +	} *ctx = NULL;
> +	struct uverbs_attr *curr_attr;
> +	unsigned long *curr_bitmap;
> +	size_t ctx_size;
> +#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
> +	uintptr_t data[UVERBS_OPTIMIZE_USING_STACK_SZ / sizeof(uintptr_t)];
> +#endif
> +
> +	if (hdr->reserved)
> +		return -EINVAL;
> +
> +	type = uverbs_get_type(ib_dev, hdr->object_type);
> +	if (!type)
> +		return -EOPNOTSUPP;
> +
> +	action = uverbs_get_action(type, hdr->action);
> +	if (!action)
> +		return -EOPNOTSUPP;
> +
> +	if ((action->flags & UVERBS_ACTION_FLAG_CREATE_ROOT) ^ !file->ucontext)
> +		return -EINVAL;
> +
> +	ctx_size = sizeof(*ctx) +
> +		   sizeof(struct uverbs_attr_array) * action->num_groups +
> +		   sizeof(*ctx->uattrs) * hdr->num_attrs +
> +		   sizeof(*ctx->uverbs_attr_array->attrs) *
> +		   action->num_child_attrs +
> +		   sizeof(*ctx->uverbs_attr_array->valid_bitmap) *
> +			(action->num_child_attrs / BITS_PER_LONG +
> +			 action->num_groups);
> +
> +#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
> +	if (ctx_size <= UVERBS_OPTIMIZE_USING_STACK_SZ)
> +		ctx = (void *)data;
> +
> +	if (!ctx)
> +#endif
> +	ctx = kmalloc(ctx_size, GFP_KERNEL);
> +	if (!ctx)
> +		return -ENOMEM;
> +
> +	ctx->uverbs_attr_array = (void *)ctx + sizeof(*ctx);
> +	ctx->uattrs = (void *)(ctx->uverbs_attr_array +
> +			       action->num_groups);
> +	curr_attr = (void *)(ctx->uattrs + hdr->num_attrs);
> +	curr_bitmap = (void *)(curr_attr + action->num_child_attrs);
> +
> +	/*
> +	 * We just fill the pointers and num_attrs here. The data itself will be
> +	 * filled at a later stage (uverbs_process_attr)
> +	 */
> +	for (i = 0; i < action->num_groups; i++) {
> +		unsigned int curr_num_attrs = action->attr_groups[i]->num_attrs;
> +
> +		ctx->uverbs_attr_array[i].attrs = curr_attr;
> +		curr_attr += curr_num_attrs;
> +		ctx->uverbs_attr_array[i].num_attrs = curr_num_attrs;
> +		ctx->uverbs_attr_array[i].valid_bitmap = curr_bitmap;
> +		bitmap_zero(curr_bitmap, curr_num_attrs);
> +		curr_bitmap += BITS_TO_LONGS(curr_num_attrs);
> +	}
> +
> +	err = copy_from_user(ctx->uattrs, buf,
> +			     sizeof(*ctx->uattrs) * hdr->num_attrs);
> +	if (err) {
> +		err = -EFAULT;
> +		goto out;
> +	}
> +
> +	err = uverbs_handle_action(buf, ctx->uattrs, hdr->num_attrs, ib_dev,
> +				   file, action, ctx->uverbs_attr_array);
> +out:
> +#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
> +	if (ctx_size > UVERBS_OPTIMIZE_USING_STACK_SZ)
> +#endif
> +	kfree(ctx);

What is the purpose of UVERBS_OPTIMIZE_USING_STACK_SZ?
And something wrong with this "if" and kfree after that.


> +	return err;
> +}
> +
> +#define IB_UVERBS_MAX_CMD_SZ 4096
> +
> +long ib_uverbs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
> +{
> +	struct ib_uverbs_file *file = filp->private_data;
> +	struct ib_uverbs_ioctl_hdr __user *user_hdr =
> +		(struct ib_uverbs_ioctl_hdr __user *)arg;
> +	struct ib_uverbs_ioctl_hdr hdr;
> +	struct ib_device *ib_dev;
> +	int srcu_key;
> +	long err;
> +
> +	srcu_key = srcu_read_lock(&file->device->disassociate_srcu);
> +	ib_dev = srcu_dereference(file->device->ib_dev,
> +				  &file->device->disassociate_srcu);
> +	if (!ib_dev) {
> +		err = -EIO;
> +		goto out;
> +	}
> +
> +	if (cmd == RDMA_VERBS_IOCTL) {
> +		err = copy_from_user(&hdr, user_hdr, sizeof(hdr));
> +
> +		if (err || hdr.length > IB_UVERBS_MAX_CMD_SZ ||
> +		    hdr.length != sizeof(hdr) + hdr.num_attrs * sizeof(struct ib_uverbs_attr)) {
> +			err = -EINVAL;
> +			goto out;
> +		}
> +
> +		/* currently there are no flags supported */
> +		if (hdr.flags) {
> +			err = -EOPNOTSUPP;
> +			goto out;
> +		}
> +
> +		err = ib_uverbs_cmd_verbs(ib_dev, file, &hdr,
> +					  (__user void *)arg + sizeof(hdr));
> +	} else {
> +		err = -ENOIOCTLCMD;
> +	}
> +out:
> +	srcu_read_unlock(&file->device->disassociate_srcu, srcu_key);
> +
> +	return err;
> +}
> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> index 3a8e058..44cd98b 100644
> --- a/include/rdma/ib_verbs.h
> +++ b/include/rdma/ib_verbs.h
> @@ -2165,6 +2165,8 @@ struct ib_device {
>  	 */
>  	int (*get_port_immutable)(struct ib_device *, u8, struct ib_port_immutable *);
>  	void (*get_dev_fw_str)(struct ib_device *, char *str, size_t str_len);
> +
> +	struct uverbs_root                      *specs_root;
>  };
>
>  struct ib_client {
> diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
> index 1f84f30..71a6b84 100644
> --- a/include/rdma/uverbs_ioctl.h
> +++ b/include/rdma/uverbs_ioctl.h
> @@ -41,8 +41,13 @@
>   * =======================================
>   */
>
> +#define UVERBS_ID_RESERVED_MASK 0xF000
> +#define UVERBS_ID_RESERVED_SHIFT 12
> +
>  enum uverbs_attr_type {
>  	UVERBS_ATTR_TYPE_NA,
> +	UVERBS_ATTR_TYPE_PTR_IN,
> +	UVERBS_ATTR_TYPE_PTR_OUT,
>  	UVERBS_ATTR_TYPE_IDR,
>  	UVERBS_ATTR_TYPE_FD,
>  };
> @@ -54,8 +59,14 @@ enum uverbs_idr_access {
>  	UVERBS_ACCESS_DESTROY
>  };
>
> +enum uverbs_attr_spec_flags {
> +	UVERBS_ATTR_SPEC_F_MANDATORY	= 1U << 0,
> +	UVERBS_ATTR_SPEC_F_MIN_SZ	= 1U << 1,
> +};
> +
>  struct uverbs_attr_spec {
>  	enum uverbs_attr_type		type;
> +	u8				flags;
>  	union {
>  		u16				len;
>  		struct {
> @@ -68,11 +79,45 @@ struct uverbs_attr_spec {
>  struct uverbs_attr_spec_group {
>  	struct uverbs_attr_spec		*attrs;
>  	size_t				num_attrs;
> +	/* populate at runtime */
> +	unsigned long			*mandatory_attrs_bitmask;
> +};
> +
> +struct uverbs_attr_array;
> +struct ib_uverbs_file;
> +
> +enum uverbs_action_flags {
> +	UVERBS_ACTION_FLAG_CREATE_ROOT = 1 << 0,
>  };
>
>  struct uverbs_action {
> -	const struct uverbs_attr_spec_group		**attr_groups;
> +	struct uverbs_attr_spec_group			**attr_groups;
>  	size_t						num_groups;
> +	size_t						num_child_attrs;
> +	u32 flags;
> +	int (*handler)(struct ib_device *ib_dev, struct ib_uverbs_file *ufile,
> +		       struct uverbs_attr_array *ctx, size_t num);
> +};
> +
> +struct uverbs_action_group {
> +	size_t					num_actions;
> +	struct uverbs_action			**actions;
> +};
> +
> +struct uverbs_type {
> +	size_t					num_groups;
> +	const struct uverbs_action_group	**action_groups;
> +	const struct uverbs_obj_type		*type_attrs;
> +};
> +
> +struct uverbs_type_group {
> +	size_t					num_types;
> +	const struct uverbs_type		**types;
> +};
> +
> +struct uverbs_root {
> +	const struct uverbs_type_group		**type_groups;
> +	size_t					num_groups;
>  };
>
>  /* =================================================
> @@ -80,28 +125,22 @@ struct uverbs_action {
>   * =================================================
>   */
>
> -struct uverbs_fd_attr {
> -	int		fd;
> -};
> -
> -struct uverbs_uobj_attr {
> -	/*  idr handle */
> -	u32	idr;
> +struct uverbs_ptr_attr {
> +	void	* __user ptr;
> +	u16		len;
>  };
>
>  struct uverbs_obj_attr {
>  	/* pointer to the kernel descriptor -> type, access, etc */
>  	struct ib_uverbs_attr __user	*uattr;
> -	const struct uverbs_type_alloc_action	*type;
> +	const struct uverbs_obj_type	*type;
>  	struct ib_uobject		*uobject;
> -	union {
> -		struct uverbs_fd_attr		fd;
> -		struct uverbs_uobj_attr		uobj;
> -	};
> +	int				id;
>  };
>
>  struct uverbs_attr {
>  	union {
> +		struct uverbs_ptr_attr	ptr_attr;
>  		struct uverbs_obj_attr	obj_attr;
>  	};
>  };
> diff --git a/include/uapi/rdma/rdma_user_ioctl.h b/include/uapi/rdma/rdma_user_ioctl.h
> index 9388125..12663f6 100644
> --- a/include/uapi/rdma/rdma_user_ioctl.h
> +++ b/include/uapi/rdma/rdma_user_ioctl.h
> @@ -43,6 +43,31 @@
>  /* Legacy name, for user space application which already use it */
>  #define IB_IOCTL_MAGIC		RDMA_IOCTL_MAGIC
>
> +#define RDMA_VERBS_IOCTL \
> +	_IOWR(RDMA_IOCTL_MAGIC, 1, struct ib_uverbs_ioctl_hdr)
> +
> +enum ib_uverbs_attr_flags {
> +	UVERBS_ATTR_F_MANDATORY = 1U << 0,
> +};
> +
> +struct ib_uverbs_attr {
> +	__u16 attr_id;		/* command specific type attribute */
> +	__u16 len;		/* NA for idr */
> +	__u16 flags;		/* combination of uverbs_attr_flags */
> +	__u16 reserved;
> +	__u64 data;		/* ptr to command, inline data or idr/fd */
> +};
> +
> +struct ib_uverbs_ioctl_hdr {
> +	__u16 length;
> +	__u16 flags;
> +	__u16 object_type;
> +	__u16 reserved;		/* future use for driver_id */
> +	__u16 action;
> +	__u16 num_attrs;
> +	struct ib_uverbs_attr  attrs[0];
> +};
> +
>  /*
>   * General blocks assignments
>   * It is closed on purpose do not expose it it user space
> --
> 1.8.3.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH RFC 03/10] IB/core: Add new ioctl interface

[Matan Barak]

  is

On Mon, May 8, 2017 at 9:06 AM, Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:
> On Wed, Apr 19, 2017 at 06:20:18PM +0300, Matan Barak wrote:
>> In this ioctl interface, processing the command starts from
>> properties of the command and fetching the appropriate user objects
>> before calling the handler.
>>
>> Parsing and validation is done according to a specifier declared by
>> the driver's code. In the driver, all supported types are declared.
>> These types are separated to different type groups, each could be
>> declared in a different place (for example, common types and driver
>> specific types).
>>
>> For each type we list all supported actions. Similarly to types,
>> actions are separated to actions groups too. Each group is declared
>> separately. This could be used in order to add actions to an existing
>> type.
>>
>> Each action has a specifies a handler, which could be either a
>> standard command or a driver specific command.
>> Along with the handler, a group of attributes is specified as well.
>> This group lists all supported attributes and is used for automatic
>> fetching and validation of the command, response and its related
>> objects.
>>
>> When a group of elements is used, the high bits of the elements ids
>> are used in order to calculate the group index. Then, these high bits
>> are masked out in order to have a zero based namespace for every
>> group. This is mandatory for compact representation and O(1) array
>> access.
>>
>> A group of attributes is actually an array of attributes. Each
>> attribute has a type (PTR_IN, PTR_OUT, IDR and FD) and a length.
>> Attributes could be validated through some attributes, like:
>> (*) Minimum size / Exact size
>> (*) Fops for FD
>> (*) Object type for IDR
>>
>> If an IDR/fd attribute is specified, the kernel also states the object
>> type and the required access (NEW, WRITE, READ or DESTROY).
>> All uobject/fd management is done automatically by the infrastructure,
>> meaning - the infrastructure will fail concurrent commands that at
>> least one of them requires concurrent access (WRITE/DESTROY),
>> synchronize actions with device removals (dissociate context events)
>> and take care of reference counting (increase/decrease) for concurrent
>> actions invocation. The reference counts on the actual kernel objects
>> shall be handled by the handlers.
>>
>>  types
>> +--------+
>> |        |
>> |        |   actions                                                                +--------+
>> |        |   group      action      action_spec                           +-----+   |len     |
>> +--------+  +------+[d]+-------+   +----------------+[d]+------------+    |attr1+-> |type    |
>> | type   +> |action+-> | spec  +-> +  attr_groups   +-> |common sec  +--> +-----+   |idr_type|
>> +--------+  +------+   |handler|   |                |   +------------+    |attr2|   |access  |
>> |        |  |      |   +-------+   +----------------+   |device sec  |    +-----+   +--------+
>> |        |  |      |                                    +------------+
>> |        |  +------+
>> |        |
>> |        |
>> |        |
>> |        |
>> |        |
>> |        |
>> |        |
>> |        |
>> |        |
>> |        |
>> +--------+
>>
>> [d] = distribute ids to groups using the high order bits
>>
>> The right types table is also chosen by using the high bits from
>> uverbs_types_groups.
>>
>> Once validation and object fetching (or creation) completed, we call
>> the handler:
>> int (*handler)(struct ib_device *ib_dev, struct ib_ucontext *ucontext,
>>                struct uverbs_attr_array *ctx, size_t num);
>>
>> Where ctx is an array of uverbs_attr_array. Each element in this array
>> is an array of attributes which corresponds to one group of attributes.
>> For example, in the usually used case:
>>
>>  ctx                               core
>> +----------------------------+     +------------+
>> | core: uverbs_attr_array    +---> | valid      |
>> +----------------------------+     | cmd_attr   |
>> | driver: uverbs_attr_array  |     +------------+
>> |----------------------------+--+  | valid      |
>>                                 |  | cmd_attr   |
>>                                 |  +------------+
>>                                 |  | valid      |
>>                                 |  | obj_attr   |
>>                                 |  +------------+
>>                                 |
>>                                 |  vendor
>>                                 |  +------------+
>>                                 +> | valid      |
>>                                    | cmd_attr   |
>>                                    +------------+
>>                                    | valid      |
>>                                    | cmd_attr   |
>>                                    +------------+
>>                                    | valid      |
>>                                    | obj_attr   |
>>                                    +------------+
>>
>> Ctx array's indices corresponds to the attributes groups order. The indices
>> of core and driver corresponds to the attributes name spaces of each
>> group. Thus, we could think of the following as one object:
>> 1. Set of attribute specification (with their attribute IDs)
>> 2. Attribute group which owns (1) specifications
>> 3. A function which could handle this attributes which the handler
>>    could call
>> 4. The allocation descriptor of this type uverbs_obj_type.
>>
>> Upon success of a handler invocation, reference count of uobjects and
>> use count will be a updated automatically according to the
>> specification.
>>
>> Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>> ---
>>  drivers/infiniband/core/Makefile       |   2 +-
>>  drivers/infiniband/core/rdma_core.c    |  45 +++++
>>  drivers/infiniband/core/rdma_core.h    |   5 +
>>  drivers/infiniband/core/uverbs_ioctl.c | 351 +++++++++++++++++++++++++++++++++
>>  include/rdma/ib_verbs.h                |   2 +
>>  include/rdma/uverbs_ioctl.h            |  65 ++++--
>>  include/uapi/rdma/rdma_user_ioctl.h    |  25 +++
>>  7 files changed, 481 insertions(+), 14 deletions(-)
>>  create mode 100644 drivers/infiniband/core/uverbs_ioctl.c
>>
>> diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
>> index 6ebd9ad..e18f2f8 100644
>> --- a/drivers/infiniband/core/Makefile
>> +++ b/drivers/infiniband/core/Makefile
>> @@ -30,4 +30,4 @@ ib_umad-y :=                        user_mad.o
>>  ib_ucm-y :=                  ucm.o
>>
>>  ib_uverbs-y :=                       uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
>> -                             rdma_core.o uverbs_std_types.o
>> +                             rdma_core.o uverbs_std_types.o uverbs_ioctl.o
>> diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
>> index 78ffd8c..a6e35b3 100644
>> --- a/drivers/infiniband/core/rdma_core.c
>> +++ b/drivers/infiniband/core/rdma_core.c
>> @@ -40,6 +40,51 @@
>>  #include "core_priv.h"
>>  #include "rdma_core.h"
>>
>> +int uverbs_group_idx(u16 *id, unsigned int ngroups)
>> +{
>> +     int ret = (*id & UVERBS_ID_RESERVED_MASK) >> UVERBS_ID_RESERVED_SHIFT;
>> +
>> +     if (ret >= ngroups)
>> +             return -EINVAL;
>> +
>> +     *id &= ~UVERBS_ID_RESERVED_MASK;
>> +     return ret;
>> +}
>> +
>> +const struct uverbs_type *uverbs_get_type(const struct ib_device *ibdev,
>> +                                       uint16_t type)
>> +{
>> +     const struct uverbs_root *groups = ibdev->specs_root;
>> +     const struct uverbs_type_group *types;
>> +     int ret = uverbs_group_idx(&type, groups->num_groups);
>> +
>> +     if (ret < 0)
>> +             return NULL;
>> +
>> +     types = groups->type_groups[ret];
>> +
>> +     if (type >= types->num_types)
>> +             return NULL;
>> +
>> +     return types->types[type];
>> +}
>> +
>> +const struct uverbs_action *uverbs_get_action(const struct uverbs_type *type,
>> +                                           uint16_t action)
>> +{
>> +     const struct uverbs_action_group *action_group;
>> +     int ret = uverbs_group_idx(&action, type->num_groups);
>> +
>> +     if (ret < 0)
>> +             return NULL;
>> +
>> +     action_group = type->action_groups[ret];
>> +     if (action >= action_group->num_actions)
>> +             return NULL;
>> +
>> +     return action_group->actions[action];
>> +}
>> +
>>  void uverbs_uobject_get(struct ib_uobject *uobject)
>>  {
>>       kref_get(&uobject->ref);
>> diff --git a/drivers/infiniband/core/rdma_core.h b/drivers/infiniband/core/rdma_core.h
>> index 0aebc47..82db2bc 100644
>> --- a/drivers/infiniband/core/rdma_core.h
>> +++ b/drivers/infiniband/core/rdma_core.h
>> @@ -43,6 +43,11 @@
>>  #include <rdma/ib_verbs.h>
>>  #include <linux/mutex.h>
>>
>> +int uverbs_group_idx(u16 *id, unsigned int ngroups);
>> +const struct uverbs_type *uverbs_get_type(const struct ib_device *ibdev,
>> +                                       uint16_t type);
>> +const struct uverbs_action *uverbs_get_action(const struct uverbs_type *type,
>> +                                           uint16_t action);
>>  /*
>>   * These functions initialize the context and cleanups its uobjects.
>>   * The context has a list of objects which is protected by a mutex
>> diff --git a/drivers/infiniband/core/uverbs_ioctl.c b/drivers/infiniband/core/uverbs_ioctl.c
>> new file mode 100644
>> index 0000000..3465a18
>> --- /dev/null
>> +++ b/drivers/infiniband/core/uverbs_ioctl.c
>> @@ -0,0 +1,351 @@
>> +/*
>> + * Copyright (c) 2017, Mellanox Technologies inc.  All rights reserved.
>> + *
>> + * This software is available to you under a choice of one of two
>> + * licenses.  You may choose to be licensed under the terms of the GNU
>> + * General Public License (GPL) Version 2, available from the file
>> + * COPYING in the main directory of this source tree, or the
>> + * OpenIB.org BSD license below:
>> + *
>> + *     Redistribution and use in source and binary forms, with or
>> + *     without modification, are permitted provided that the following
>> + *     conditions are met:
>> + *
>> + *      - Redistributions of source code must retain the above
>> + *        copyright notice, this list of conditions and the following
>> + *        disclaimer.
>> + *
>> + *      - Redistributions in binary form must reproduce the above
>> + *        copyright notice, this list of conditions and the following
>> + *        disclaimer in the documentation and/or other materials
>> + *        provided with the distribution.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
>> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
>> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
>> + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
>> + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
>> + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
>> + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
>> + * SOFTWARE.
>> + */
>> +
>> +#include <rdma/rdma_user_ioctl.h>
>> +#include <rdma/uverbs_ioctl.h>
>> +#include "rdma_core.h"
>> +#include "uverbs.h"
>> +
>> +static int uverbs_process_attr(struct ib_device *ibdev,
>> +                            struct ib_ucontext *ucontext,
>> +                            const struct ib_uverbs_attr *uattr,
>> +                            u16 attr_id,
>> +                            const struct uverbs_attr_spec_group *attr_spec_group,
>> +                            struct uverbs_attr_array *attr_array,
>> +                            struct ib_uverbs_attr __user *uattr_ptr)
>> +{
>> +     const struct uverbs_attr_spec *spec;
>> +     struct uverbs_attr *e;
>> +     const struct uverbs_type *type;
>> +     struct uverbs_obj_attr *o_attr;
>> +     struct uverbs_attr *elements = attr_array->attrs;
>> +
>> +     if (uattr->reserved)
>> +             return -EINVAL;
>> +
>> +     if (attr_id >= attr_spec_group->num_attrs) {
>> +             if (uattr->flags & UVERBS_ATTR_F_MANDATORY)
>> +                     return -EINVAL;
>> +             else
>> +                     return 0;
>> +     }
>> +
>> +     spec = &attr_spec_group->attrs[attr_id];
>> +     e = &elements[attr_id];
>> +
>> +     switch (spec->type) {
>> +     case UVERBS_ATTR_TYPE_PTR_IN:
>> +     case UVERBS_ATTR_TYPE_PTR_OUT:
>> +             if (uattr->len < spec->len ||
>> +                 (!(spec->flags & UVERBS_ATTR_SPEC_F_MIN_SZ) &&
>> +                  uattr->len > spec->len))
>> +                     return -EINVAL;
>> +
>> +             e->ptr_attr.ptr = (void * __user)uattr->data;
>> +             e->ptr_attr.len = uattr->len;
>> +             break;
>> +
>> +     case UVERBS_ATTR_TYPE_IDR:
>> +             if (uattr->data >> 32)
>> +                     return -EINVAL;
>> +     /* fall through */
>> +     case UVERBS_ATTR_TYPE_FD:
>> +             if (uattr->len != 0 || !ucontext || uattr->data > INT_MAX)
>> +                     return -EINVAL;
>> +
>> +             o_attr = &e->obj_attr;
>> +             type = uverbs_get_type(ibdev, spec->obj.obj_type);
>> +             if (!type)
>> +                     return -EINVAL;
>> +             o_attr->type = type->type_attrs;
>> +             o_attr->uattr = uattr_ptr;
>> +
>> +             o_attr->id = (int)uattr->data;
>> +             o_attr->uobject = uverbs_get_uobject_from_context(
>> +                                     o_attr->type,
>> +                                     ucontext,
>> +                                     spec->obj.access,
>> +                                     o_attr->id);
>> +
>> +             if (IS_ERR(o_attr->uobject))
>> +                     return -EINVAL;
>> +
>> +             if (spec->obj.access == UVERBS_ACCESS_NEW) {
>> +                     u64 id = o_attr->uobject->id;
>> +
>> +                     if (put_user(id, &o_attr->uattr->data)) {
>> +                             uverbs_finalize_object(o_attr->uobject,
>> +                                                    UVERBS_ACCESS_NEW,
>> +                                                    false);
>> +                             return -EFAULT;
>> +                     }
>> +             }
>> +
>> +             break;
>> +     default:
>> +             return -EOPNOTSUPP;
>> +     };
>> +
>> +     set_bit(attr_id, attr_array->valid_bitmap);
>> +     return 0;
>> +}
>> +
>> +static int uverbs_uattrs_process(struct ib_device *ibdev,
>> +                              struct ib_ucontext *ucontext,
>> +                              const struct ib_uverbs_attr *uattrs,
>> +                              size_t num_uattrs,
>> +                              const struct uverbs_action *action,
>> +                              struct uverbs_attr_array *attr_array,
>> +                              struct ib_uverbs_attr __user *uattr_ptr)
>> +{
>> +     size_t i;
>> +     int ret = 0;
>> +     int num_given_groups = 0;
>> +
>> +     for (i = 0; i < num_uattrs; i++) {
>> +             const struct ib_uverbs_attr *uattr = &uattrs[i];
>> +             u16 attr_id = uattr->attr_id;
>> +             const struct uverbs_attr_spec_group *attr_spec_group;
>> +
>> +             ret = uverbs_group_idx(&attr_id, action->num_groups);
>> +             if (ret < 0) {
>> +                     if (uattr->flags & UVERBS_ATTR_F_MANDATORY)
>> +                             return ret;
>> +
>> +                     continue;
>> +             }
>> +
>> +             if (ret >= num_given_groups)
>> +                     num_given_groups = ret + 1;
>> +
>> +             attr_spec_group = action->attr_groups[ret];
>> +             ret = uverbs_process_attr(ibdev, ucontext, uattr, attr_id,
>> +                                       attr_spec_group, &attr_array[ret],
>> +                                       uattr_ptr++);
>> +             if (ret) {
>> +                     uverbs_finalize_objects(attr_array,
>> +                                             num_given_groups,
>> +                                             action, false);
>> +                     return ret;
>> +             }
>> +     }
>> +
>> +     return ret ?: num_given_groups;
>> +}
>> +
>> +static int uverbs_validate_kernel_mandatory(const struct uverbs_action *action,
>> +                                         struct uverbs_attr_array *attr_array,
>> +                                         unsigned int num_given_groups)
>> +{
>> +     unsigned int i;
>> +
>> +     for (i = 0; i < num_given_groups; i++) {
>> +             const struct uverbs_attr_spec_group *attr_spec_group =
>> +                     action->attr_groups[i];
>> +
>> +             if (!bitmap_subset(attr_spec_group->mandatory_attrs_bitmask,
>> +                                attr_array[i].valid_bitmap,
>> +                                attr_spec_group->num_attrs))
>> +                     return -EINVAL;
>> +     }
>> +
>> +     return 0;
>> +}
>> +
>> +static int uverbs_handle_action(struct ib_uverbs_attr __user *uattr_ptr,
>> +                             const struct ib_uverbs_attr *uattrs,
>> +                             size_t num_uattrs,
>> +                             struct ib_device *ibdev,
>> +                             struct ib_uverbs_file *ufile,
>> +                             const struct uverbs_action *action,
>> +                             struct uverbs_attr_array *attr_array)
>> +{
>> +     int ret;
>> +     int finalize_ret;
>> +     int num_given_groups;
>> +
>> +     num_given_groups = uverbs_uattrs_process(ibdev, ufile->ucontext, uattrs,
>> +                                              num_uattrs, action, attr_array,
>> +                                              uattr_ptr);
>> +     if (num_given_groups <= 0)
>> +             return -EINVAL;
>> +
>> +     ret = uverbs_validate_kernel_mandatory(action, attr_array,
>> +                                            num_given_groups);
>> +     if (ret)
>> +             goto cleanup;
>> +
>> +     ret = action->handler(ibdev, ufile, attr_array, num_given_groups);
>> +cleanup:
>> +     finalize_ret = uverbs_finalize_objects(attr_array, num_given_groups,
>> +                                            action, !ret);
>> +
>> +     return ret ? ret : finalize_ret;
>> +}
>> +
>> +#define UVERBS_OPTIMIZE_USING_STACK_SZ  256
>> +long ib_uverbs_cmd_verbs(struct ib_device *ib_dev,
>> +                      struct ib_uverbs_file *file,
>> +                      struct ib_uverbs_ioctl_hdr *hdr,
>> +                      void __user *buf)
>> +{
>> +     const struct uverbs_type *type;
>> +     const struct uverbs_action *action;
>> +     long err = 0;
>> +     unsigned int i;
>> +     struct {
>> +             struct ib_uverbs_attr           *uattrs;
>> +             struct uverbs_attr_array        *uverbs_attr_array;
>> +     } *ctx = NULL;
>> +     struct uverbs_attr *curr_attr;
>> +     unsigned long *curr_bitmap;
>> +     size_t ctx_size;
>> +#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
>> +     uintptr_t data[UVERBS_OPTIMIZE_USING_STACK_SZ / sizeof(uintptr_t)];
>> +#endif
>> +
>> +     if (hdr->reserved)
>> +             return -EINVAL;
>> +
>> +     type = uverbs_get_type(ib_dev, hdr->object_type);
>> +     if (!type)
>> +             return -EOPNOTSUPP;
>> +
>> +     action = uverbs_get_action(type, hdr->action);
>> +     if (!action)
>> +             return -EOPNOTSUPP;
>> +
>> +     if ((action->flags & UVERBS_ACTION_FLAG_CREATE_ROOT) ^ !file->ucontext)
>> +             return -EINVAL;
>> +
>> +     ctx_size = sizeof(*ctx) +
>> +                sizeof(struct uverbs_attr_array) * action->num_groups +
>> +                sizeof(*ctx->uattrs) * hdr->num_attrs +
>> +                sizeof(*ctx->uverbs_attr_array->attrs) *
>> +                action->num_child_attrs +
>> +                sizeof(*ctx->uverbs_attr_array->valid_bitmap) *
>> +                     (action->num_child_attrs / BITS_PER_LONG +
>> +                      action->num_groups);
>> +
>> +#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
>> +     if (ctx_size <= UVERBS_OPTIMIZE_USING_STACK_SZ)
>> +             ctx = (void *)data;
>> +
>> +     if (!ctx)
>> +#endif
>> +     ctx = kmalloc(ctx_size, GFP_KERNEL);
>> +     if (!ctx)
>> +             return -ENOMEM;
>> +
>> +     ctx->uverbs_attr_array = (void *)ctx + sizeof(*ctx);
>> +     ctx->uattrs = (void *)(ctx->uverbs_attr_array +
>> +                            action->num_groups);
>> +     curr_attr = (void *)(ctx->uattrs + hdr->num_attrs);
>> +     curr_bitmap = (void *)(curr_attr + action->num_child_attrs);
>> +
>> +     /*
>> +      * We just fill the pointers and num_attrs here. The data itself will be
>> +      * filled at a later stage (uverbs_process_attr)
>> +      */
>> +     for (i = 0; i < action->num_groups; i++) {
>> +             unsigned int curr_num_attrs = action->attr_groups[i]->num_attrs;
>> +
>> +             ctx->uverbs_attr_array[i].attrs = curr_attr;
>> +             curr_attr += curr_num_attrs;
>> +             ctx->uverbs_attr_array[i].num_attrs = curr_num_attrs;
>> +             ctx->uverbs_attr_array[i].valid_bitmap = curr_bitmap;
>> +             bitmap_zero(curr_bitmap, curr_num_attrs);
>> +             curr_bitmap += BITS_TO_LONGS(curr_num_attrs);
>> +     }
>> +
>> +     err = copy_from_user(ctx->uattrs, buf,
>> +                          sizeof(*ctx->uattrs) * hdr->num_attrs);
>> +     if (err) {
>> +             err = -EFAULT;
>> +             goto out;
>> +     }
>> +
>> +     err = uverbs_handle_action(buf, ctx->uattrs, hdr->num_attrs, ib_dev,
>> +                                file, action, ctx->uverbs_attr_array);
>> +out:
>> +#ifdef UVERBS_OPTIMIZE_USING_STACK_SZ
>> +     if (ctx_size > UVERBS_OPTIMIZE_USING_STACK_SZ)
>> +#endif
>> +     kfree(ctx);
>
> What is the purpose of UVERBS_OPTIMIZE_USING_STACK_SZ?
> And something wrong with this "if" and kfree after that.
>
>

In order to avoid allocations in the command execution path (to make
it faster), small commands could be copied
straight to the stack. So, this define acually means that if a command
header is less than UVERBS_OPTIMIZE_USING_STACK_SZ bytes,
just copy it. In case the command is bigger (or this size isn't
defined), you need to allocate some space.
Obviously, if you allocated something, you need to free it. This
happens always except for cases where UVERBS_OPTIMIZE_USING_STACK_SZ
is more than the
required size (as in these cases we store the command on the stack).
This is exactly what we do here.

BTW, we could define it as zero in order to disable it, but then you
could get a warning of an always true statement :)

>> +     return err;
>> +}
>> +
>> +#define IB_UVERBS_MAX_CMD_SZ 4096
>> +
>> +long ib_uverbs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
>> +{
>> +     struct ib_uverbs_file *file = filp->private_data;
>> +     struct ib_uverbs_ioctl_hdr __user *user_hdr =
>> +             (struct ib_uverbs_ioctl_hdr __user *)arg;
>> +     struct ib_uverbs_ioctl_hdr hdr;
>> +     struct ib_device *ib_dev;
>> +     int srcu_key;
>> +     long err;
>> +
>> +     srcu_key = srcu_read_lock(&file->device->disassociate_srcu);
>> +     ib_dev = srcu_dereference(file->device->ib_dev,
>> +                               &file->device->disassociate_srcu);
>> +     if (!ib_dev) {
>> +             err = -EIO;
>> +             goto out;
>> +     }
>> +
>> +     if (cmd == RDMA_VERBS_IOCTL) {
>> +             err = copy_from_user(&hdr, user_hdr, sizeof(hdr));
>> +
>> +             if (err || hdr.length > IB_UVERBS_MAX_CMD_SZ ||
>> +                 hdr.length != sizeof(hdr) + hdr.num_attrs * sizeof(struct ib_uverbs_attr)) {
>> +                     err = -EINVAL;
>> +                     goto out;
>> +             }
>> +
>> +             /* currently there are no flags supported */
>> +             if (hdr.flags) {
>> +                     err = -EOPNOTSUPP;
>> +                     goto out;
>> +             }
>> +
>> +             err = ib_uverbs_cmd_verbs(ib_dev, file, &hdr,
>> +                                       (__user void *)arg + sizeof(hdr));
>> +     } else {
>> +             err = -ENOIOCTLCMD;
>> +     }
>> +out:
>> +     srcu_read_unlock(&file->device->disassociate_srcu, srcu_key);
>> +
>> +     return err;
>> +}
>> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
>> index 3a8e058..44cd98b 100644
>> --- a/include/rdma/ib_verbs.h
>> +++ b/include/rdma/ib_verbs.h
>> @@ -2165,6 +2165,8 @@ struct ib_device {
>>        */
>>       int (*get_port_immutable)(struct ib_device *, u8, struct ib_port_immutable *);
>>       void (*get_dev_fw_str)(struct ib_device *, char *str, size_t str_len);
>> +
>> +     struct uverbs_root                      *specs_root;
>>  };
>>
>>  struct ib_client {
>> diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
>> index 1f84f30..71a6b84 100644
>> --- a/include/rdma/uverbs_ioctl.h
>> +++ b/include/rdma/uverbs_ioctl.h
>> @@ -41,8 +41,13 @@
>>   * =======================================
>>   */
>>
>> +#define UVERBS_ID_RESERVED_MASK 0xF000
>> +#define UVERBS_ID_RESERVED_SHIFT 12
>> +
>>  enum uverbs_attr_type {
>>       UVERBS_ATTR_TYPE_NA,
>> +     UVERBS_ATTR_TYPE_PTR_IN,
>> +     UVERBS_ATTR_TYPE_PTR_OUT,
>>       UVERBS_ATTR_TYPE_IDR,
>>       UVERBS_ATTR_TYPE_FD,
>>  };
>> @@ -54,8 +59,14 @@ enum uverbs_idr_access {
>>       UVERBS_ACCESS_DESTROY
>>  };
>>
>> +enum uverbs_attr_spec_flags {
>> +     UVERBS_ATTR_SPEC_F_MANDATORY    = 1U << 0,
>> +     UVERBS_ATTR_SPEC_F_MIN_SZ       = 1U << 1,
>> +};
>> +
>>  struct uverbs_attr_spec {
>>       enum uverbs_attr_type           type;
>> +     u8                              flags;
>>       union {
>>               u16                             len;
>>               struct {
>> @@ -68,11 +79,45 @@ struct uverbs_attr_spec {
>>  struct uverbs_attr_spec_group {
>>       struct uverbs_attr_spec         *attrs;
>>       size_t                          num_attrs;
>> +     /* populate at runtime */
>> +     unsigned long                   *mandatory_attrs_bitmask;
>> +};
>> +
>> +struct uverbs_attr_array;
>> +struct ib_uverbs_file;
>> +
>> +enum uverbs_action_flags {
>> +     UVERBS_ACTION_FLAG_CREATE_ROOT = 1 << 0,
>>  };
>>
>>  struct uverbs_action {
>> -     const struct uverbs_attr_spec_group             **attr_groups;
>> +     struct uverbs_attr_spec_group                   **attr_groups;
>>       size_t                                          num_groups;
>> +     size_t                                          num_child_attrs;
>> +     u32 flags;
>> +     int (*handler)(struct ib_device *ib_dev, struct ib_uverbs_file *ufile,
>> +                    struct uverbs_attr_array *ctx, size_t num);
>> +};
>> +
>> +struct uverbs_action_group {
>> +     size_t                                  num_actions;
>> +     struct uverbs_action                    **actions;
>> +};
>> +
>> +struct uverbs_type {
>> +     size_t                                  num_groups;
>> +     const struct uverbs_action_group        **action_groups;
>> +     const struct uverbs_obj_type            *type_attrs;
>> +};
>> +
>> +struct uverbs_type_group {
>> +     size_t                                  num_types;
>> +     const struct uverbs_type                **types;
>> +};
>> +
>> +struct uverbs_root {
>> +     const struct uverbs_type_group          **type_groups;
>> +     size_t                                  num_groups;
>>  };
>>
>>  /* =================================================
>> @@ -80,28 +125,22 @@ struct uverbs_action {
>>   * =================================================
>>   */
>>
>> -struct uverbs_fd_attr {
>> -     int             fd;
>> -};
>> -
>> -struct uverbs_uobj_attr {
>> -     /*  idr handle */
>> -     u32     idr;
>> +struct uverbs_ptr_attr {
>> +     void    * __user ptr;
>> +     u16             len;
>>  };
>>
>>  struct uverbs_obj_attr {
>>       /* pointer to the kernel descriptor -> type, access, etc */
>>       struct ib_uverbs_attr __user    *uattr;
>> -     const struct uverbs_type_alloc_action   *type;
>> +     const struct uverbs_obj_type    *type;
>>       struct ib_uobject               *uobject;
>> -     union {
>> -             struct uverbs_fd_attr           fd;
>> -             struct uverbs_uobj_attr         uobj;
>> -     };
>> +     int                             id;
>>  };
>>
>>  struct uverbs_attr {
>>       union {
>> +             struct uverbs_ptr_attr  ptr_attr;
>>               struct uverbs_obj_attr  obj_attr;
>>       };
>>  };
>> diff --git a/include/uapi/rdma/rdma_user_ioctl.h b/include/uapi/rdma/rdma_user_ioctl.h
>> index 9388125..12663f6 100644
>> --- a/include/uapi/rdma/rdma_user_ioctl.h
>> +++ b/include/uapi/rdma/rdma_user_ioctl.h
>> @@ -43,6 +43,31 @@
>>  /* Legacy name, for user space application which already use it */
>>  #define IB_IOCTL_MAGIC               RDMA_IOCTL_MAGIC
>>
>> +#define RDMA_VERBS_IOCTL \
>> +     _IOWR(RDMA_IOCTL_MAGIC, 1, struct ib_uverbs_ioctl_hdr)
>> +
>> +enum ib_uverbs_attr_flags {
>> +     UVERBS_ATTR_F_MANDATORY = 1U << 0,
>> +};
>> +
>> +struct ib_uverbs_attr {
>> +     __u16 attr_id;          /* command specific type attribute */
>> +     __u16 len;              /* NA for idr */
>> +     __u16 flags;            /* combination of uverbs_attr_flags */
>> +     __u16 reserved;
>> +     __u64 data;             /* ptr to command, inline data or idr/fd */
>> +};
>> +
>> +struct ib_uverbs_ioctl_hdr {
>> +     __u16 length;
>> +     __u16 flags;
>> +     __u16 object_type;
>> +     __u16 reserved;         /* future use for driver_id */
>> +     __u16 action;
>> +     __u16 num_attrs;
>> +     struct ib_uverbs_attr  attrs[0];
>> +};
>> +
>>  /*
>>   * General blocks assignments
>>   * It is closed on purpose do not expose it it user space
>> --
>> 1.8.3.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 04/10] IB/core: Declare a type instead of declaring only type attributes

[Matan Barak]

Switch all uverbs_type_attrs_xxxx with DECLARE_UVERBS_TYPE
macros. This will be later used in order to embed the types
specific actions in the types as well.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/uverbs_std_types.c | 89 ++++++++++++------------------
 include/rdma/uverbs_ioctl.h                |  4 ++
 include/rdma/uverbs_std_types.h            | 34 ++++++------
 include/rdma/uverbs_types.h                | 38 ++++++++-----
 4 files changed, 80 insertions(+), 85 deletions(-)

diff --git a/drivers/infiniband/core/uverbs_std_types.c b/drivers/infiniband/core/uverbs_std_types.c
index e3338b1..283c655 100644
--- a/drivers/infiniband/core/uverbs_std_types.c
+++ b/drivers/infiniband/core/uverbs_std_types.c
@@ -209,67 +209,50 @@ int uverbs_hot_unplug_completion_event_file(struct ib_uobject_file *uobj_file,
 	return 0;
 };
 
-const struct uverbs_obj_fd_type uverbs_type_attrs_comp_channel = {
-	.type = UVERBS_TYPE_ALLOC_FD(sizeof(struct ib_uverbs_completion_event_file), 0),
-	.context_closed = uverbs_hot_unplug_completion_event_file,
-	.fops = &uverbs_event_fops,
-	.name = "[infinibandevent]",
-	.flags = O_RDONLY,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_comp_channel,
+		    &UVERBS_TYPE_ALLOC_FD(0,
+					  sizeof(struct ib_uverbs_completion_event_file),
+					  uverbs_hot_unplug_completion_event_file,
+					  &uverbs_event_fops,
+					  "[infinibandevent]", O_RDONLY));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_cq = {
-	.type = UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_ucq_object), 0),
-	.destroy_object = uverbs_free_cq,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_cq,
+		    &UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_ucq_object), 0,
+					      uverbs_free_cq));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_qp = {
-	.type = UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uqp_object), 0),
-	.destroy_object = uverbs_free_qp,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_qp,
+		    &UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uqp_object), 0,
+					      uverbs_free_qp));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_mw = {
-	.type = UVERBS_TYPE_ALLOC_IDR(0),
-	.destroy_object = uverbs_free_mw,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_mw,
+		    &UVERBS_TYPE_ALLOC_IDR(0, uverbs_free_mw));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_mr = {
-	/* 1 is used in order to free the MR after all the MWs */
-	.type = UVERBS_TYPE_ALLOC_IDR(1),
-	.destroy_object = uverbs_free_mr,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_mr,
+		    /* 1 is used in order to free the MR after all the MWs */
+		    &UVERBS_TYPE_ALLOC_IDR(1, uverbs_free_mr));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_srq = {
-	.type = UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_usrq_object), 0),
-	.destroy_object = uverbs_free_srq,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_srq,
+		    &UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_usrq_object), 0,
+					      uverbs_free_srq));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_ah = {
-	.type = UVERBS_TYPE_ALLOC_IDR(0),
-	.destroy_object = uverbs_free_ah,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_ah,
+		    &UVERBS_TYPE_ALLOC_IDR(0, uverbs_free_ah));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_flow = {
-	.type = UVERBS_TYPE_ALLOC_IDR(0),
-	.destroy_object = uverbs_free_flow,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_flow,
+		    &UVERBS_TYPE_ALLOC_IDR(0, uverbs_free_flow));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_wq = {
-	.type = UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uwq_object), 0),
-	.destroy_object = uverbs_free_wq,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_wq,
+		    &UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uwq_object), 0,
+					      uverbs_free_wq));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_rwq_ind_table = {
-	.type = UVERBS_TYPE_ALLOC_IDR(0),
-	.destroy_object = uverbs_free_rwq_ind_tbl,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_rwq_ind_table,
+		    &UVERBS_TYPE_ALLOC_IDR(0, uverbs_free_rwq_ind_tbl));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_xrcd = {
-	.type = UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uxrcd_object), 0),
-	.destroy_object = uverbs_free_xrcd,
-};
+DECLARE_UVERBS_TYPE(uverbs_type_xrcd,
+		    &UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uxrcd_object), 0,
+					      uverbs_free_xrcd));
+
+DECLARE_UVERBS_TYPE(uverbs_type_pd,
+		    /* 2 is used in order to free the PD after MRs */
+		    &UVERBS_TYPE_ALLOC_IDR(2, uverbs_free_pd));
 
-const struct uverbs_obj_idr_type uverbs_type_attrs_pd = {
-	/* 2 is used in order to free the PD after MRs */
-	.type = UVERBS_TYPE_ALLOC_IDR(2),
-	.destroy_object = uverbs_free_pd,
-};
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index 71a6b84..b6f2bd9 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -120,6 +120,10 @@ struct uverbs_root {
 	size_t					num_groups;
 };
 
+#define DECLARE_UVERBS_TYPE(name, _type_attrs)			\
+	const struct uverbs_type name = {				\
+		.type_attrs = _type_attrs,				\
+	}
 /* =================================================
  *              Parsing infrastructure
  * =================================================
diff --git a/include/rdma/uverbs_std_types.h b/include/rdma/uverbs_std_types.h
index 7771ce9..5abb873 100644
--- a/include/rdma/uverbs_std_types.h
+++ b/include/rdma/uverbs_std_types.h
@@ -35,18 +35,18 @@
 
 #include <rdma/uverbs_types.h>
 
-extern const struct uverbs_obj_fd_type uverbs_type_attrs_comp_channel;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_cq;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_qp;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_rwq_ind_table;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_wq;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_srq;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_ah;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_flow;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_mr;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_mw;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_pd;
-extern const struct uverbs_obj_idr_type uverbs_type_attrs_xrcd;
+extern const struct uverbs_type uverbs_type_comp_channel;
+extern const struct uverbs_type uverbs_type_cq;
+extern const struct uverbs_type uverbs_type_qp;
+extern const struct uverbs_type uverbs_type_rwq_ind_table;
+extern const struct uverbs_type uverbs_type_wq;
+extern const struct uverbs_type uverbs_type_srq;
+extern const struct uverbs_type uverbs_type_ah;
+extern const struct uverbs_type uverbs_type_flow;
+extern const struct uverbs_type uverbs_type_mr;
+extern const struct uverbs_type uverbs_type_mw;
+extern const struct uverbs_type uverbs_type_pd;
+extern const struct uverbs_type uverbs_type_xrcd;
 
 static inline struct ib_uobject *__uobj_get(const struct uverbs_obj_type *type,
 					    bool write,
@@ -56,22 +56,22 @@ static inline struct ib_uobject *__uobj_get(const struct uverbs_obj_type *type,
 	return rdma_lookup_get_uobject(type, ucontext, id, write);
 }
 
-#define uobj_get_type(_type) uverbs_type_attrs_##_type.type
+#define uobj_get_type(_type) uverbs_type_##_type.type_attrs
 
 #define uobj_get_read(_type, _id, _ucontext)				\
-	 __uobj_get(&(_type), false, _ucontext, _id)
+	 __uobj_get(_type, false, _ucontext, _id)
 
 #define uobj_get_obj_read(_type, _id, _ucontext)			\
 ({									\
 	struct ib_uobject *uobj =					\
-		__uobj_get(&uobj_get_type(_type),			\
+		__uobj_get(uverbs_type_##_type.type_attrs,		\
 			   false, _ucontext, _id);			\
 									\
 	(struct ib_##_type *)(IS_ERR(uobj) ? NULL : uobj->object);	\
 })
 
 #define uobj_get_write(_type, _id, _ucontext)				\
-	 __uobj_get(&(_type), true, _ucontext, _id)
+	 __uobj_get(_type, true, _ucontext, _id)
 
 static inline void uobj_put_read(struct ib_uobject *uobj)
 {
@@ -108,7 +108,7 @@ static inline struct ib_uobject *__uobj_alloc(const struct uverbs_obj_type *type
 }
 
 #define uobj_alloc(_type, ucontext)	\
-	__uobj_alloc(&(_type), ucontext)
+	__uobj_alloc(_type, ucontext)
 
 #endif
 
diff --git a/include/rdma/uverbs_types.h b/include/rdma/uverbs_types.h
index 351ea18..9760b6d 100644
--- a/include/rdma/uverbs_types.h
+++ b/include/rdma/uverbs_types.h
@@ -151,22 +151,30 @@ struct uverbs_obj_fd_type {
 
 #define UVERBS_BUILD_BUG_ON(cond) (sizeof(char[1 - 2 * !!(cond)]) -	\
 				   sizeof(char))
-#define UVERBS_TYPE_ALLOC_FD(_size, _order)				 \
-	{								 \
-		.destroy_order = _order,				 \
-		.type_class = &uverbs_fd_class,				 \
-		.obj_size = (_size) +					 \
-			  UVERBS_BUILD_BUG_ON((_size) <			 \
-					      sizeof(struct ib_uobject_file)),\
-	}
-#define UVERBS_TYPE_ALLOC_IDR_SZ(_size, _order)				\
-	{								\
+#define UVERBS_TYPE_ALLOC_FD(_order, _obj_size, _context_closed, _fops, _name, _flags)\
+	((&((const struct uverbs_obj_fd_type)				\
+	 {.type = {							\
+		.destroy_order = _order,				\
+		.type_class = &uverbs_fd_class,				\
+		.obj_size = (_obj_size) +				\
+			UVERBS_BUILD_BUG_ON((_obj_size) < sizeof(struct ib_uobject_file)), \
+	 },								\
+	 .context_closed = _context_closed,				\
+	 .fops = _fops,							\
+	 .name = _name,							\
+	 .flags = _flags}))->type)
+#define UVERBS_TYPE_ALLOC_IDR_SZ(_size, _order, _destroy_object)	\
+	((&((const struct uverbs_obj_idr_type)				\
+	 {.type = {							\
 		.destroy_order = _order,				\
 		.type_class = &uverbs_idr_class,			\
 		.obj_size = (_size) +					\
-			  UVERBS_BUILD_BUG_ON((_size) <			\
-					      sizeof(struct ib_uobject)), \
-	}
-#define UVERBS_TYPE_ALLOC_IDR(_order)					\
-	 UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uobject), _order)
+			UVERBS_BUILD_BUG_ON((_size) <			\
+					    sizeof(struct ib_uobject))	\
+	 },								\
+	 .destroy_object = _destroy_object,}))->type)
+#define UVERBS_TYPE_ALLOC_IDR(_order, _destroy_object)			\
+	 UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uobject), _order,	\
+				  _destroy_object)
+
 #endif
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 05/10] IB/core: Add DEVICE type and root types structure

[Matan Barak]

This adds the DEVICE type. This type support creating the context all
objects are created from. Moreover, it supports executing actions
which are related to the device itself, such as QUERY_DEVICE.
The only related object to this type is a singleton (per file
instance) context.

All standard types are put in the root structure.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/uverbs_std_types.c | 19 +++++++++++++++++++
 include/rdma/uverbs_ioctl.h                | 11 +++++++++++
 include/rdma/uverbs_std_types.h            | 19 +++++++++++++++++++
 3 files changed, 49 insertions(+)

diff --git a/drivers/infiniband/core/uverbs_std_types.c b/drivers/infiniband/core/uverbs_std_types.c
index 283c655..22e77aa 100644
--- a/drivers/infiniband/core/uverbs_std_types.c
+++ b/drivers/infiniband/core/uverbs_std_types.c
@@ -256,3 +256,22 @@ int uverbs_hot_unplug_completion_event_file(struct ib_uobject_file *uobj_file,
 		    /* 2 is used in order to free the PD after MRs */
 		    &UVERBS_TYPE_ALLOC_IDR(2, uverbs_free_pd));
 
+DECLARE_UVERBS_TYPE(uverbs_type_device, NULL);
+
+DECLARE_UVERBS_TYPES(uverbs_common_types,
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_DEVICE, uverbs_type_device),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_PD, uverbs_type_pd),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_MR, uverbs_type_mr),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_COMP_CHANNEL, uverbs_type_comp_channel),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_CQ, uverbs_type_cq),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_QP, uverbs_type_qp),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_AH, uverbs_type_ah),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_MW, uverbs_type_mw),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_SRQ, uverbs_type_srq),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_FLOW, uverbs_type_flow),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_WQ, uverbs_type_wq),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_RWQ_IND_TBL,
+				     uverbs_type_rwq_ind_table),
+		     ADD_UVERBS_TYPE(UVERBS_TYPE_XRCD, uverbs_type_xrcd),
+);
+EXPORT_SYMBOL(uverbs_common_types);
diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index b6f2bd9..5c3aa4f 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -124,6 +124,17 @@ struct uverbs_root {
 	const struct uverbs_type name = {				\
 		.type_attrs = _type_attrs,				\
 	}
+#define _UVERBS_TYPE_SZ(...)						\
+	(sizeof((const struct uverbs_type *[]){__VA_ARGS__}) /	\
+	 sizeof(const struct uverbs_type *))
+#define ADD_UVERBS_TYPE(type_idx, type_ptr)				\
+	[type_idx] = ((const struct uverbs_type * const)&(type_ptr))
+#define UVERBS_TYPES(...)  ((const struct uverbs_type_group)		\
+	{.num_types = _UVERBS_TYPE_SZ(__VA_ARGS__),			\
+	 .types = (const struct uverbs_type *[]){__VA_ARGS__} })
+#define DECLARE_UVERBS_TYPES(name, ...)				\
+	const struct uverbs_type_group name = UVERBS_TYPES(__VA_ARGS__)
+
 /* =================================================
  *              Parsing infrastructure
  * =================================================
diff --git a/include/rdma/uverbs_std_types.h b/include/rdma/uverbs_std_types.h
index 5abb873..1b633f1 100644
--- a/include/rdma/uverbs_std_types.h
+++ b/include/rdma/uverbs_std_types.h
@@ -35,6 +35,23 @@
 
 #include <rdma/uverbs_types.h>
 
+enum uverbs_common_types {
+	UVERBS_TYPE_DEVICE, /* No instances of DEVICE are allowed */
+	UVERBS_TYPE_PD,
+	UVERBS_TYPE_COMP_CHANNEL,
+	UVERBS_TYPE_CQ,
+	UVERBS_TYPE_QP,
+	UVERBS_TYPE_SRQ,
+	UVERBS_TYPE_AH,
+	UVERBS_TYPE_MR,
+	UVERBS_TYPE_MW,
+	UVERBS_TYPE_FLOW,
+	UVERBS_TYPE_XRCD,
+	UVERBS_TYPE_RWQ_IND_TBL,
+	UVERBS_TYPE_WQ,
+	UVERBS_TYPE_LAST,
+};
+
 extern const struct uverbs_type uverbs_type_comp_channel;
 extern const struct uverbs_type uverbs_type_cq;
 extern const struct uverbs_type uverbs_type_qp;
@@ -47,6 +64,8 @@
 extern const struct uverbs_type uverbs_type_mw;
 extern const struct uverbs_type uverbs_type_pd;
 extern const struct uverbs_type uverbs_type_xrcd;
+extern const struct uverbs_type uverbs_type_device;
+extern const struct uverbs_type_group uverbs_common_types;
 
 static inline struct ib_uobject *__uobj_get(const struct uverbs_obj_type *type,
 					    bool write,
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 06/10] IB/core: Initialize uverbs types specification

[Matan Barak]

In order to accelerate the validation and parsing process, we
calculate the number of attributes of all groups in an action
and the mandatory attributes bitmask in advance.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/uverbs.h       |  3 ++
 drivers/infiniband/core/uverbs_ioctl.c | 58 ++++++++++++++++++++++++++++++++++
 drivers/infiniband/core/uverbs_main.c  |  3 ++
 3 files changed, 64 insertions(+)

diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index a3230b6..6e59a00 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -191,7 +191,10 @@ struct ib_ucq_object {
 	u32			async_events_reported;
 };
 
+struct uverbs_type_group;
+
 extern const struct file_operations uverbs_event_fops;
+void uverbs_initialize_type_group(const struct uverbs_type_group *type_group);
 void ib_uverbs_init_event_queue(struct ib_uverbs_event_queue *ev_queue);
 struct file *ib_uverbs_alloc_async_event_file(struct ib_uverbs_file *uverbs_file,
 					      struct ib_device *ib_dev);
diff --git a/drivers/infiniband/core/uverbs_ioctl.c b/drivers/infiniband/core/uverbs_ioctl.c
index 3465a18..18c0799 100644
--- a/drivers/infiniband/core/uverbs_ioctl.c
+++ b/drivers/infiniband/core/uverbs_ioctl.c
@@ -349,3 +349,61 @@ long ib_uverbs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 
 	return err;
 }
+
+static void uverbs_initialize_action(struct uverbs_action *action)
+{
+	size_t attr_group_idx;
+
+	for (attr_group_idx = 0; attr_group_idx < action->num_groups;
+	     attr_group_idx++) {
+		struct uverbs_attr_spec_group *attr_group =
+			action->attr_groups[attr_group_idx];
+		size_t attr_idx;
+
+		if (!attr_group)
+			continue;
+		action->num_child_attrs += attr_group->num_attrs;
+		for (attr_idx = 0; attr_idx < attr_group->num_attrs;
+		     attr_idx++) {
+			struct uverbs_attr_spec *attr =
+				&attr_group->attrs[attr_idx];
+
+			if (attr->flags & UVERBS_ATTR_SPEC_F_MANDATORY)
+				set_bit(attr_idx,
+					attr_group->mandatory_attrs_bitmask);
+		}
+	}
+}
+
+void uverbs_initialize_type_group(const struct uverbs_type_group *type_group)
+{
+	size_t type_idx;
+
+	for (type_idx = 0; type_idx < type_group->num_types; type_idx++) {
+		const struct uverbs_type *type = type_group->types[type_idx];
+		size_t action_group_idx;
+
+		if (!type)
+			continue;
+		for (action_group_idx = 0;
+		     action_group_idx < type->num_groups;
+		     action_group_idx++) {
+			const struct uverbs_action_group *action_group =
+				type->action_groups[action_group_idx];
+			size_t action_idx;
+
+			if (!action_group)
+				continue;
+			for (action_idx = 0;
+			     action_idx < action_group->num_actions;
+			     action_idx++) {
+				struct uverbs_action *action =
+					action_group->actions[action_idx];
+
+				if (!action)
+					continue;
+				uverbs_initialize_action(action);
+			}
+		}
+	}
+}
diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
index b282daf..44c4d92 100644
--- a/drivers/infiniband/core/uverbs_main.c
+++ b/drivers/infiniband/core/uverbs_main.c
@@ -45,6 +45,7 @@
 #include <linux/cdev.h>
 #include <linux/anon_inodes.h>
 #include <linux/slab.h>
+#include <rdma/uverbs_std_types.h>
 
 #include <linux/uaccess.h>
 
@@ -1253,6 +1254,8 @@ static int __init ib_uverbs_init(void)
 {
 	int ret;
 
+	uverbs_initialize_type_group(&uverbs_common_types);
+
 	ret = register_chrdev_region(IB_UVERBS_BASE_DEV, IB_UVERBS_MAX_DEVICES,
 				     "infiniband_verbs");
 	if (ret) {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 07/10] IB/core: Add macros for declaring actions and attributes

[Matan Barak]

This patch adds macros for declaring action groups, actions,
attribute groups and attributes. These definitions are later
used by downstream patches to declare some of the common types.

In addition, we add some helper inline functions to copy_{from,to}
user-space buffers and check if an attribute is valid.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 include/rdma/uverbs_ioctl.h | 113 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 112 insertions(+), 1 deletion(-)

diff --git a/include/rdma/uverbs_ioctl.h b/include/rdma/uverbs_ioctl.h
index 5c3aa4f..5beb1e1 100644
--- a/include/rdma/uverbs_ioctl.h
+++ b/include/rdma/uverbs_ioctl.h
@@ -34,6 +34,11 @@
 #define _UVERBS_IOCTL_
 
 #include <rdma/uverbs_types.h>
+#include <linux/uaccess.h>
+#include <rdma/rdma_user_ioctl.h>
+
+struct uverbs_object_type;
+struct uverbs_uobject_type;
 
 /*
  * =======================================
@@ -120,9 +125,74 @@ struct uverbs_root {
 	size_t					num_groups;
 };
 
-#define DECLARE_UVERBS_TYPE(name, _type_attrs)			\
+#define UA_FLAGS(_flags)  .flags = _flags
+#define UVERBS_ATTR(_id, _len, _type, ...)				\
+	[_id] = {.len = _len, .type = _type, ##__VA_ARGS__}
+#define UVERBS_ATTR_PTR_IN_SZ(_id, _len, ...)				\
+	UVERBS_ATTR(_id, _len, UVERBS_ATTR_TYPE_PTR_IN, ##__VA_ARGS__)
+#define UVERBS_ATTR_PTR_IN(_id, _type, ...)				\
+	UVERBS_ATTR_PTR_IN_SZ(_id, sizeof(_type), ##__VA_ARGS__)
+#define UVERBS_ATTR_PTR_OUT_SZ(_id, _len, ...)				\
+	UVERBS_ATTR(_id, _len, UVERBS_ATTR_TYPE_PTR_OUT, ##__VA_ARGS__)
+#define UVERBS_ATTR_PTR_OUT(_id, _type, ...)				\
+	UVERBS_ATTR_PTR_OUT_SZ(_id, sizeof(_type), ##__VA_ARGS__)
+#define UVERBS_ATTR_IDR(_id, _idr_type, _access, ...)			\
+	[_id] = {.type = UVERBS_ATTR_TYPE_IDR,				\
+		 .obj = {.obj_type = _idr_type,				\
+			 .access = _access				\
+		 }, ##__VA_ARGS__ }
+#define UVERBS_ATTR_FD(_id, _fd_type, _access, ...)			\
+	[_id] = {.type = UVERBS_ATTR_TYPE_FD,				\
+		 .obj = {.obj_type = _fd_type,				\
+			 .access = (_access) + BUILD_BUG_ON_ZERO(	\
+				(_access) != UVERBS_ACCESS_NEW &&	\
+				(_access) != UVERBS_ACCESS_READ)	\
+		 }, ##__VA_ARGS__ }
+#define _UVERBS_ATTR_SPEC_SZ(...)					\
+	(sizeof((struct uverbs_attr_spec[]){__VA_ARGS__}) /	\
+	 sizeof(struct uverbs_attr_spec))
+#define UVERBS_ATTR_SPEC(...)					\
+	((struct uverbs_attr_spec_group)				\
+	 {.attrs = (struct uverbs_attr_spec[]){__VA_ARGS__},		\
+	  .num_attrs = _UVERBS_ATTR_SPEC_SZ(__VA_ARGS__),		\
+	 .mandatory_attrs_bitmask =					\
+		((unsigned long[BITS_TO_LONGS(_UVERBS_ATTR_SPEC_SZ(__VA_ARGS__))]) {})})
+#define DECLARE_UVERBS_ATTR_SPEC(name, ...)			\
+	struct uverbs_attr_spec_group name =			\
+		UVERBS_ATTR_SPEC(__VA_ARGS__)
+#define _UVERBS_ATTR_ACTION_SPEC_SZ(...)				  \
+	(sizeof((struct uverbs_attr_spec_group *[]){__VA_ARGS__}) /	  \
+	 sizeof(struct uverbs_attr_spec_group *))
+#define _UVERBS_ACTION(_handler, _flags, ...)				       \
+	((struct uverbs_action) {					       \
+		.flags = _flags,					       \
+		.handler = _handler,					       \
+		.num_groups =	_UVERBS_ATTR_ACTION_SPEC_SZ(__VA_ARGS__),      \
+		.attr_groups = (struct uverbs_attr_spec_group *[]){__VA_ARGS__} })
+#define UVERBS_ACTION(_handler, ...)			\
+	_UVERBS_ACTION(_handler, 0, __VA_ARGS__)
+#define UVERBS_CTX_ACTION(_handler, ...)			\
+	_UVERBS_ACTION(_handler, UVERBS_ACTION_FLAG_CREATE_ROOT, __VA_ARGS__)
+#define _UVERBS_ACTIONS_SZ(...)					\
+	(sizeof((struct uverbs_action *[]){__VA_ARGS__}) /	\
+	 sizeof(struct uverbs_action *))
+#define ADD_UVERBS_ACTION(action_idx, _handler,  ...)		\
+	[action_idx] = &UVERBS_ACTION(_handler, __VA_ARGS__)
+#define ADD_UVERBS_CTX_ACTION(action_idx, _handler,  ...)	\
+	[action_idx] = &UVERBS_CTX_ACTION(_handler, __VA_ARGS__)
+#define UVERBS_ACTIONS(...)						\
+	((const struct uverbs_action_group)			\
+	  {.num_actions = _UVERBS_ACTIONS_SZ(__VA_ARGS__),		\
+	   .actions = (struct uverbs_action *[]){__VA_ARGS__} })
+#define _UVERBS_ACTIONS_GROUP_SZ(...)					\
+	(sizeof((const struct uverbs_action_group*[]){__VA_ARGS__}) / \
+	 sizeof(const struct uverbs_action_group *))
+
+#define DECLARE_UVERBS_TYPE(name, _type_attrs, ...)			\
 	const struct uverbs_type name = {				\
 		.type_attrs = _type_attrs,				\
+		.num_groups = _UVERBS_ACTIONS_GROUP_SZ(__VA_ARGS__),	\
+		.action_groups = (const struct uverbs_action_group *[]){__VA_ARGS__} \
 	}
 #define _UVERBS_TYPE_SZ(...)						\
 	(sizeof((const struct uverbs_type *[]){__VA_ARGS__}) /	\
@@ -135,6 +205,17 @@ struct uverbs_root {
 #define DECLARE_UVERBS_TYPES(name, ...)				\
 	const struct uverbs_type_group name = UVERBS_TYPES(__VA_ARGS__)
 
+#define _UVERBS_TYPES_SZ(...)						\
+	(sizeof((const struct uverbs_type_group *[]){__VA_ARGS__}) /	\
+	 sizeof(const struct uverbs_type_group *))
+
+#define UVERBS_TYPES_GROUP(...)						\
+	((const struct uverbs_root){				\
+		.type_groups = (const struct uverbs_type_group *[]){__VA_ARGS__},\
+		.num_groups = _UVERBS_TYPES_SZ(__VA_ARGS__)})
+#define DECLARE_UVERBS_TYPES_GROUP(name, ...)		\
+	const struct uverbs_root name = UVERBS_TYPES_GROUP(__VA_ARGS__)
+
 /* =================================================
  *              Parsing infrastructure
  * =================================================
@@ -174,5 +255,35 @@ static inline bool uverbs_is_valid(const struct uverbs_attr_array *attr_array,
 	return test_bit(idx, attr_array->valid_bitmap);
 }
 
+/* TODO: Add debug version for these macros/inline func */
+static inline int uverbs_copy_to(struct uverbs_attr_array *attr_array,
+				 size_t idx, const void *from)
+{
+	if (!uverbs_is_valid(attr_array, idx))
+		return -ENOENT;
+
+	return copy_to_user(attr_array->attrs[idx].ptr_attr.ptr, from,
+			    attr_array->attrs[idx].ptr_attr.len) ? -EFAULT : 0;
+}
+
+#define uverbs_copy_from(to, attr_array, idx)				\
+	(uverbs_is_valid((attr_array), idx) ?				\
+	 (sizeof(*to) <= sizeof(((struct ib_uverbs_attr *)0)->data) ?\
+	  (memcpy(to, &(attr_array)->attrs[idx].ptr_attr.ptr,		\
+		 (attr_array)->attrs[idx].ptr_attr.len), 0) :		\
+	  (copy_from_user((to), (attr_array)->attrs[idx].ptr_attr.ptr,	\
+			 (attr_array)->attrs[idx].ptr_attr.len) ?	\
+	   -EFAULT : 0)) : -ENOENT)
+#define uverbs_get_attr(to, attr_array, idx)				\
+	(uverbs_is_valid((attr_array), idx) ?				\
+	 (sizeof(to) <= sizeof(((struct ib_uverbs_attr *)0)->data) ? \
+	  (sizeof(to) == sizeof((&(to))[0]) ?				\
+	   ((to) = *(typeof(to) *)&(attr_array)->attrs[idx].ptr_attr.ptr, 0) :\
+	   (memcpy(&(to), &(attr_array)->attrs[idx].ptr_attr.ptr,	\
+		 (attr_array)->attrs[idx].ptr_attr.len), 0)) :		\
+	  (copy_from_user(&(to), (attr_array)->attrs[idx].ptr_attr.ptr,	\
+			 (attr_array)->attrs[idx].ptr_attr.len) ?	\
+	   -EFAULT : 0)) : -ENOENT)
+
 #endif
 
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 08/10] IB/core: Add ability to explicitly destroy an uobject

[Matan Barak]

When some objects are destroyed, we need to extract their status at
destruction. In order to have the latest and correct status, the
underlying object should be destroyed, but we should keep the uobject
alive and read this information off the uobject.
We introduce a rdma_explicit_destroy function. This function destroys
the class type object (for example, the IDR class type which destroys
the underlying object as well) and then convert the uobject to be of a
null class type. This uobject will then be destroyed as any other
uobject.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/rdma_core.c | 35 +++++++++++++++++++++++++++++++++++
 include/rdma/uverbs_types.h         |  1 +
 2 files changed, 36 insertions(+)

diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c
index a6e35b3..c456107 100644
--- a/drivers/infiniband/core/rdma_core.c
+++ b/drivers/infiniband/core/rdma_core.c
@@ -450,6 +450,41 @@ int __must_check rdma_remove_commit_uobject(struct ib_uobject *uobj)
 	return ret;
 }
 
+static int null_obj_type_class_remove_commit(struct ib_uobject *uobj,
+					     enum rdma_remove_reason why)
+{
+	return 0;
+}
+
+const struct uverbs_obj_type null_obj_type = {
+	.type_class = &((const struct uverbs_obj_type_class){
+			.remove_commit = null_obj_type_class_remove_commit,
+			/* be cautious */
+			.needs_kfree_rcu = true}),
+};
+
+int rdma_explicit_destroy(struct ib_uobject *uobject)
+{
+	int ret;
+	struct ib_ucontext *ucontext = uobject->context;
+
+	/* Cleanup is running. Calling this should have been impossible */
+	if (!down_read_trylock(&ucontext->cleanup_rwsem)) {
+		WARN(true, "ib_uverbs: Cleanup is running while removing an uobject\n");
+		return 0;
+	}
+	lockdep_check(uobject, true);
+	ret = uobject->type->type_class->remove_commit(uobject,
+						       RDMA_REMOVE_DESTROY);
+	if (ret)
+		return ret;
+
+	uobject->type = &null_obj_type;
+
+	up_read(&ucontext->cleanup_rwsem);
+	return 0;
+}
+
 static void alloc_commit_idr_uobject(struct ib_uobject *uobj)
 {
 	uverbs_uobject_add(uobj);
diff --git a/include/rdma/uverbs_types.h b/include/rdma/uverbs_types.h
index 9760b6d..cc04ec6 100644
--- a/include/rdma/uverbs_types.h
+++ b/include/rdma/uverbs_types.h
@@ -129,6 +129,7 @@ struct ib_uobject *rdma_alloc_begin_uobject(const struct uverbs_obj_type *type,
 void rdma_alloc_abort_uobject(struct ib_uobject *uobj);
 int __must_check rdma_remove_commit_uobject(struct ib_uobject *uobj);
 int rdma_alloc_commit_uobject(struct ib_uobject *uobj);
+int rdma_explicit_destroy(struct ib_uobject *uobject);
 
 struct uverbs_obj_fd_type {
 	/*
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 09/10] IB/core: Add uverbs types, actions, handlers and attributes

[Matan Barak]

We add the common (core) code for init context, query device, reg_mr,
create_cq, create_qp, modify_qp, create_comp_channel, init_pd,
destroy_cq, destroy_qp, dereg_mr, dealloc_pd and query_port.
This includes the following parts:
* Macros for defining commands and validators
* For each command
    * type declarations
          - destruction order
          - free function
          - uverbs action group
    * actions
    * handlers
    * attributes

Drivers could use the these attributes, actions or types when they
want to alter or add a new type. They could use the uverbs handler
directly in the action (or just wrap it in the driver's custom code).

Currently we use ib_udata to pass vendor specific information to the
driver. This should probably be refactored in the future.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/core_priv.h          |  14 +
 drivers/infiniband/core/uverbs.h             |   4 +
 drivers/infiniband/core/uverbs_cmd.c         |  21 +-
 drivers/infiniband/core/uverbs_std_types.c   | 984 ++++++++++++++++++++++++++-
 drivers/infiniband/hw/cxgb3/iwch_provider.c  |   5 +
 drivers/infiniband/hw/cxgb4/provider.c       |   5 +
 drivers/infiniband/hw/hns/hns_roce_main.c    |   5 +
 drivers/infiniband/hw/i40iw/i40iw_verbs.c    |   5 +
 drivers/infiniband/hw/mlx4/main.c            |   5 +
 drivers/infiniband/hw/mlx5/main.c            |   5 +
 drivers/infiniband/hw/mthca/mthca_provider.c |   5 +
 drivers/infiniband/hw/nes/nes_verbs.c        |   5 +
 drivers/infiniband/hw/ocrdma/ocrdma_main.c   |   5 +
 drivers/infiniband/hw/usnic/usnic_ib_main.c  |   5 +
 include/rdma/uverbs_std_types.h              | 159 +++++
 include/uapi/rdma/ib_user_verbs.h            |  40 ++
 16 files changed, 1248 insertions(+), 24 deletions(-)

diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h
index cb7d372..1f41225 100644
--- a/drivers/infiniband/core/core_priv.h
+++ b/drivers/infiniband/core/core_priv.h
@@ -176,4 +176,18 @@ int ib_nl_handle_set_timeout(struct sk_buff *skb,
 int ib_nl_handle_ip_res_resp(struct sk_buff *skb,
 			     struct netlink_callback *cb);
 
+/* Remove ignored fields set in the attribute mask */
+static inline int modify_qp_mask(enum ib_qp_type qp_type, int mask)
+{
+	switch (qp_type) {
+	case IB_QPT_XRC_INI:
+		return mask & ~(IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER);
+	case IB_QPT_XRC_TGT:
+		return mask & ~(IB_QP_MAX_QP_RD_ATOMIC | IB_QP_RETRY_CNT |
+				IB_QP_RNR_RETRY);
+	default:
+		return mask;
+	}
+}
+
 #endif /* _CORE_PRIV_H */
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index 6e59a00..ef37a16 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -218,6 +218,10 @@ int ib_uverbs_dealloc_xrcd(struct ib_uverbs_device *dev, struct ib_xrcd *xrcd,
 			   enum rdma_remove_reason why);
 
 int uverbs_dealloc_mw(struct ib_mw *mw);
+void uverbs_copy_query_dev_fields(struct ib_device *ib_dev,
+				  struct ib_uverbs_query_device_resp *resp,
+				  struct ib_device_attr *attr);
+
 void ib_uverbs_detach_umcast(struct ib_qp *qp,
 			     struct ib_uqp_object *uobj);
 
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index e2fee04..7f2beb1 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -173,8 +173,7 @@ ssize_t ib_uverbs_get_context(struct ib_uverbs_file *file,
 	return ret;
 }
 
-static void copy_query_dev_fields(struct ib_uverbs_file *file,
-				  struct ib_device *ib_dev,
+void uverbs_copy_query_dev_fields(struct ib_device *ib_dev,
 				  struct ib_uverbs_query_device_resp *resp,
 				  struct ib_device_attr *attr)
 {
@@ -235,7 +234,7 @@ ssize_t ib_uverbs_query_device(struct ib_uverbs_file *file,
 		return -EFAULT;
 
 	memset(&resp, 0, sizeof resp);
-	copy_query_dev_fields(file, ib_dev, &resp, &ib_dev->attrs);
+	uverbs_copy_query_dev_fields(ib_dev, &resp, &ib_dev->attrs);
 
 	if (copy_to_user((void __user *) (unsigned long) cmd.response,
 			 &resp, sizeof resp))
@@ -1889,20 +1888,6 @@ ssize_t ib_uverbs_query_qp(struct ib_uverbs_file *file,
 	return ret ? ret : in_len;
 }
 
-/* Remove ignored fields set in the attribute mask */
-static int modify_qp_mask(enum ib_qp_type qp_type, int mask)
-{
-	switch (qp_type) {
-	case IB_QPT_XRC_INI:
-		return mask & ~(IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER);
-	case IB_QPT_XRC_TGT:
-		return mask & ~(IB_QP_MAX_QP_RD_ATOMIC | IB_QP_RETRY_CNT |
-				IB_QP_RNR_RETRY);
-	default:
-		return mask;
-	}
-}
-
 static int modify_qp(struct ib_uverbs_file *file,
 		     struct ib_uverbs_ex_modify_qp *cmd, struct ib_udata *udata)
 {
@@ -3740,7 +3725,7 @@ int ib_uverbs_ex_query_device(struct ib_uverbs_file *file,
 	if (err)
 		return err;
 
-	copy_query_dev_fields(file, ib_dev, &resp.base, &attr);
+	uverbs_copy_query_dev_fields(ib_dev, &resp.base, &attr);
 
 	if (ucore->outlen < resp.response_length + sizeof(resp.odp_caps))
 		goto end;
diff --git a/drivers/infiniband/core/uverbs_std_types.c b/drivers/infiniband/core/uverbs_std_types.c
index 22e77aa..6ac2b46 100644
--- a/drivers/infiniband/core/uverbs_std_types.c
+++ b/drivers/infiniband/core/uverbs_std_types.c
@@ -37,6 +37,7 @@
 #include <linux/file.h>
 #include "rdma_core.h"
 #include "uverbs.h"
+#include "core_priv.h"
 
 int uverbs_free_ah(struct ib_uobject *uobject,
 		   enum rdma_remove_reason why)
@@ -209,27 +210,978 @@ int uverbs_hot_unplug_completion_event_file(struct ib_uobject_file *uobj_file,
 	return 0;
 };
 
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_uhw_compat_spec,
+	UVERBS_ATTR_PTR_IN_SZ(UVERBS_UHW_IN, 0, UA_FLAGS(UVERBS_ATTR_SPEC_F_MIN_SZ)),
+	UVERBS_ATTR_PTR_OUT_SZ(UVERBS_UHW_OUT, 0, UA_FLAGS(UVERBS_ATTR_SPEC_F_MIN_SZ)));
+
+static void create_udata(struct uverbs_attr_array *ctx, size_t num,
+			 struct ib_udata *udata)
+{
+	/*
+	 * This is for ease of conversion. The purpose is to convert all drivers
+	 * to use uverbs_attr_array instead of ib_udata.
+	 * Assume attr == 0 is input and attr == 1 is output.
+	 */
+	void * __user inbuf;
+	size_t inbuf_len = 0;
+	void * __user outbuf;
+	size_t outbuf_len = 0;
+
+	if (num >= UVERBS_UHW_NUM) {
+		struct uverbs_attr_array *driver = &ctx[UVERBS_UDATA_DRIVER_DATA_GROUP];
+
+		if (uverbs_is_valid(driver, UVERBS_UHW_IN)) {
+			inbuf = driver->attrs[UVERBS_UHW_IN].ptr_attr.ptr;
+			inbuf_len = driver->attrs[UVERBS_UHW_IN].ptr_attr.len;
+		}
+
+		if (driver->num_attrs >= UVERBS_UHW_OUT &&
+		    uverbs_is_valid(driver, UVERBS_UHW_OUT)) {
+			outbuf = driver->attrs[UVERBS_UHW_OUT].ptr_attr.ptr;
+			outbuf_len = driver->attrs[UVERBS_UHW_OUT].ptr_attr.len;
+		}
+	}
+	INIT_UDATA_BUF_OR_NULL(udata, inbuf, outbuf, inbuf_len, outbuf_len);
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_get_context_spec,
+	UVERBS_ATTR_PTR_OUT(GET_CONTEXT_RESP,
+			    struct ib_uverbs_get_context_resp));
+
+int uverbs_get_context(struct ib_device *ib_dev,
+		       struct ib_uverbs_file *file,
+		       struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_udata uhw;
+	struct ib_uverbs_get_context_resp resp;
+	struct ib_ucontext		 *ucontext;
+	struct file			 *filp;
+	int ret;
+
+	if (!uverbs_is_valid(common, GET_CONTEXT_RESP))
+		return -EINVAL;
+
+	/* Temporary, only until drivers get the new uverbs_attr_array */
+	create_udata(ctx, num, &uhw);
+
+	mutex_lock(&file->mutex);
+
+	if (file->ucontext) {
+		ret = -EINVAL;
+		goto err;
+	}
+
+	ucontext = ib_dev->alloc_ucontext(ib_dev, &uhw);
+	if (IS_ERR(ucontext)) {
+		ret = PTR_ERR(ucontext);
+		goto err;
+	}
+
+	ucontext->device = ib_dev;
+	/* ufile is required when some objects are released */
+	ucontext->ufile = file;
+	uverbs_initialize_ucontext(ucontext);
+
+	rcu_read_lock();
+	ucontext->tgid = get_task_pid(current->group_leader, PIDTYPE_PID);
+	rcu_read_unlock();
+	ucontext->closing = 0;
+
+#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
+	ucontext->umem_tree = RB_ROOT;
+	init_rwsem(&ucontext->umem_rwsem);
+	ucontext->odp_mrs_count = 0;
+	INIT_LIST_HEAD(&ucontext->no_private_counters);
+
+	if (!(ib_dev->attrs.device_cap_flags & IB_DEVICE_ON_DEMAND_PAGING))
+		ucontext->invalidate_range = NULL;
+
+#endif
+
+	resp.num_comp_vectors = file->device->num_comp_vectors;
+
+	ret = get_unused_fd_flags(O_CLOEXEC);
+	if (ret < 0)
+		goto err_free;
+	resp.async_fd = ret;
+
+	filp = ib_uverbs_alloc_async_event_file(file, ib_dev);
+	if (IS_ERR(filp)) {
+		ret = PTR_ERR(filp);
+		goto err_fd;
+	}
+
+	ret = uverbs_copy_to(common, GET_CONTEXT_RESP, &resp);
+	if (ret)
+		goto err_file;
+
+	file->ucontext = ucontext;
+	ucontext->ufile = file;
+
+	fd_install(resp.async_fd, filp);
+
+	mutex_unlock(&file->mutex);
+
+	return 0;
+
+err_file:
+	ib_uverbs_free_async_event_file(file);
+	fput(filp);
+
+err_fd:
+	put_unused_fd(resp.async_fd);
+
+err_free:
+	put_pid(ucontext->tgid);
+	ib_dev->dealloc_ucontext(ucontext);
+err:
+	mutex_unlock(&file->mutex);
+	return ret;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_query_device_spec,
+	UVERBS_ATTR_PTR_OUT(QUERY_DEVICE_RESP, struct ib_uverbs_query_device_resp),
+	UVERBS_ATTR_PTR_OUT(QUERY_DEVICE_ODP, struct ib_uverbs_odp_caps),
+	UVERBS_ATTR_PTR_OUT(QUERY_DEVICE_TIMESTAMP_MASK, u64),
+	UVERBS_ATTR_PTR_OUT(QUERY_DEVICE_HCA_CORE_CLOCK, u64),
+	UVERBS_ATTR_PTR_OUT(QUERY_DEVICE_CAP_FLAGS, u64));
+
+int uverbs_query_device_handler(struct ib_device *ib_dev,
+				struct ib_uverbs_file *file,
+				struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_device_attr attr = {};
+	struct ib_udata uhw;
+	int err;
+
+	/* Temporary, only until drivers get the new uverbs_attr_array */
+	create_udata(ctx, num, &uhw);
+
+	err = ib_dev->query_device(ib_dev, &attr, &uhw);
+	if (err)
+		return err;
+
+	if (uverbs_is_valid(common, QUERY_DEVICE_RESP)) {
+		struct ib_uverbs_query_device_resp resp = {};
+
+		uverbs_copy_query_dev_fields(ib_dev, &resp, &attr);
+		if (uverbs_copy_to(common, QUERY_DEVICE_RESP, &resp))
+			return -EFAULT;
+	}
+
+#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
+	if (uverbs_is_valid(common, QUERY_DEVICE_ODP)) {
+		struct ib_uverbs_odp_caps odp_caps;
+
+		odp_caps.general_caps = attr.odp_caps.general_caps;
+		odp_caps.per_transport_caps.rc_odp_caps =
+			attr.odp_caps.per_transport_caps.rc_odp_caps;
+		odp_caps.per_transport_caps.uc_odp_caps =
+			attr.odp_caps.per_transport_caps.uc_odp_caps;
+		odp_caps.per_transport_caps.ud_odp_caps =
+			attr.odp_caps.per_transport_caps.ud_odp_caps;
+
+		if (uverbs_copy_to(common, QUERY_DEVICE_ODP, &odp_caps))
+			return -EFAULT;
+	}
+#endif
+	if (uverbs_copy_to(common, QUERY_DEVICE_TIMESTAMP_MASK,
+			   &attr.timestamp_mask) == -EFAULT)
+		return -EFAULT;
+
+	if (uverbs_copy_to(common, QUERY_DEVICE_HCA_CORE_CLOCK,
+			   &attr.hca_core_clock) == -EFAULT)
+		return -EFAULT;
+
+	if (uverbs_copy_to(common, QUERY_DEVICE_CAP_FLAGS,
+			   &attr.device_cap_flags) == -EFAULT)
+		return -EFAULT;
+
+	return 0;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_query_port_spec,
+	UVERBS_ATTR_PTR_IN(QUERY_PORT_PORT_NUM, __u8),
+	UVERBS_ATTR_PTR_OUT(QUERY_PORT_RESP, struct ib_uverbs_query_port_resp,
+			    UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_query_port_handler(struct ib_device *ib_dev,
+			      struct ib_uverbs_file *file,
+			      struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_uverbs_query_port_resp resp = {};
+	struct ib_port_attr              attr;
+	u8				 port_num;
+	int				 ret;
+
+	ret = uverbs_copy_from(&port_num, common, QUERY_PORT_PORT_NUM);
+	if (ret)
+		return ret;
+
+	ret = ib_query_port(ib_dev, port_num, &attr);
+	if (ret)
+		return ret;
+
+	resp.state	     = attr.state;
+	resp.max_mtu	     = attr.max_mtu;
+	resp.active_mtu      = attr.active_mtu;
+	resp.gid_tbl_len     = attr.gid_tbl_len;
+	resp.port_cap_flags  = attr.port_cap_flags;
+	resp.max_msg_sz      = attr.max_msg_sz;
+	resp.bad_pkey_cntr   = attr.bad_pkey_cntr;
+	resp.qkey_viol_cntr  = attr.qkey_viol_cntr;
+	resp.pkey_tbl_len    = attr.pkey_tbl_len;
+	resp.lid	     = attr.lid;
+	resp.sm_lid	     = attr.sm_lid;
+	resp.lmc	     = attr.lmc;
+	resp.max_vl_num      = attr.max_vl_num;
+	resp.sm_sl	     = attr.sm_sl;
+	resp.subnet_timeout  = attr.subnet_timeout;
+	resp.init_type_reply = attr.init_type_reply;
+	resp.active_width    = attr.active_width;
+	resp.active_speed    = attr.active_speed;
+	resp.phys_state      = attr.phys_state;
+	resp.link_layer      = rdma_port_get_link_layer(ib_dev, port_num);
+
+	return uverbs_copy_to(common, QUERY_PORT_RESP, &resp);
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_alloc_pd_spec,
+	UVERBS_ATTR_IDR(ALLOC_PD_HANDLE, UVERBS_TYPE_PD,
+			UVERBS_ACCESS_NEW,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_alloc_pd_handler(struct ib_device *ib_dev,
+			    struct ib_uverbs_file *file,
+			    struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_ucontext *ucontext = file->ucontext;
+	struct ib_udata uhw;
+	struct ib_uobject *uobject;
+	struct ib_pd *pd;
+
+	/* Temporary, only until drivers get the new uverbs_attr_array */
+	create_udata(ctx, num, &uhw);
+
+	pd = ib_dev->alloc_pd(ib_dev, ucontext, &uhw);
+	if (IS_ERR(pd))
+		return PTR_ERR(pd);
+
+	uobject = common->attrs[ALLOC_PD_HANDLE].obj_attr.uobject;
+	pd->device  = ib_dev;
+	pd->uobject = uobject;
+	pd->__internal_mr = NULL;
+	uobject->object = pd;
+	atomic_set(&pd->usecnt, 0);
+
+	return 0;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_dealloc_pd_spec,
+	UVERBS_ATTR_IDR(DEALLOC_PD_HANDLE, UVERBS_TYPE_PD,
+			UVERBS_ACCESS_DESTROY,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_dealloc_pd_handler(struct ib_device *ib_dev,
+			      struct ib_uverbs_file *file,
+			      struct uverbs_attr_array *ctx, size_t num)
+{
+	return 0;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_reg_mr_spec,
+	UVERBS_ATTR_IDR(REG_MR_HANDLE, UVERBS_TYPE_MR, UVERBS_ACCESS_NEW,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_IDR(REG_MR_PD_HANDLE, UVERBS_TYPE_PD, UVERBS_ACCESS_READ,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_IN(REG_MR_CMD, struct ib_uverbs_ioctl_reg_mr,
+			   UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_OUT(REG_MR_RESP, struct ib_uverbs_ioctl_reg_mr_resp,
+			    UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_reg_mr_handler(struct ib_device *ib_dev,
+			  struct ib_uverbs_file *file,
+			  struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_uverbs_ioctl_reg_mr		cmd;
+	struct ib_uverbs_ioctl_reg_mr_resp	resp;
+	struct ib_udata uhw;
+	struct ib_uobject *uobject;
+	struct ib_pd                *pd;
+	struct ib_mr                *mr;
+	int                          ret;
+
+	if (uverbs_copy_from(&cmd, common, REG_MR_CMD))
+		return -EFAULT;
+
+	if ((cmd.start & ~PAGE_MASK) != (cmd.hca_va & ~PAGE_MASK))
+		return -EINVAL;
+
+	ret = ib_check_mr_access(cmd.access_flags);
+	if (ret)
+		return ret;
+
+	/* Temporary, only until drivers get the new uverbs_attr_array */
+	create_udata(ctx, num, &uhw);
+
+	uobject = common->attrs[REG_MR_HANDLE].obj_attr.uobject;
+	pd = common->attrs[REG_MR_PD_HANDLE].obj_attr.uobject->object;
+
+	if (cmd.access_flags & IB_ACCESS_ON_DEMAND) {
+		if (!(pd->device->attrs.device_cap_flags &
+		      IB_DEVICE_ON_DEMAND_PAGING)) {
+			pr_debug("ODP support not available\n");
+			return -EINVAL;
+		}
+	}
+
+	mr = pd->device->reg_user_mr(pd, cmd.start, cmd.length, cmd.hca_va,
+				     cmd.access_flags, &uhw);
+	if (IS_ERR(mr))
+		return PTR_ERR(mr);
+
+	mr->device  = pd->device;
+	mr->pd      = pd;
+	mr->uobject = uobject;
+	atomic_inc(&pd->usecnt);
+	uobject->object = mr;
+
+	resp.lkey      = mr->lkey;
+	resp.rkey      = mr->rkey;
+
+	if (uverbs_copy_to(common, REG_MR_RESP, &resp)) {
+		ret = -EFAULT;
+		goto err;
+	}
+
+	return 0;
+
+err:
+	ib_dereg_mr(mr);
+	return ret;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_dereg_mr_spec,
+	UVERBS_ATTR_IDR(DEREG_MR_HANDLE, UVERBS_TYPE_MR, UVERBS_ACCESS_DESTROY,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_dereg_mr_handler(struct ib_device *ib_dev,
+			    struct ib_uverbs_file *file,
+			    struct uverbs_attr_array *ctx, size_t num)
+{
+	return 0;
+};
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_create_comp_channel_spec,
+	UVERBS_ATTR_FD(CREATE_COMP_CHANNEL_FD, UVERBS_TYPE_COMP_CHANNEL,
+		       UVERBS_ACCESS_NEW,
+		       UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_create_comp_channel_handler(struct ib_device *ib_dev,
+				       struct ib_uverbs_file *file,
+				       struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_uverbs_completion_event_file *ev_file;
+	struct ib_uobject *uobj =
+		common->attrs[CREATE_COMP_CHANNEL_FD].obj_attr.uobject;
+
+	kref_get(&uobj->ref);
+	ev_file = container_of(uobj,
+			       struct ib_uverbs_completion_event_file,
+			       uobj_file.uobj);
+	ib_uverbs_init_event_queue(&ev_file->ev_queue);
+
+	return 0;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_create_cq_spec,
+	UVERBS_ATTR_IDR(CREATE_CQ_HANDLE, UVERBS_TYPE_CQ, UVERBS_ACCESS_NEW,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_IN(CREATE_CQ_CQE, u32,
+			   UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_IN(CREATE_CQ_USER_HANDLE, u64),
+	UVERBS_ATTR_FD(CREATE_CQ_COMP_CHANNEL, UVERBS_TYPE_COMP_CHANNEL, UVERBS_ACCESS_READ),
+	/*
+	 * Currently, COMP_VECTOR is mandatory, but that could be lifted in the
+	 * future.
+	 */
+	UVERBS_ATTR_PTR_IN(CREATE_CQ_COMP_VECTOR, u32,
+			   UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_IN(CREATE_CQ_FLAGS, u32),
+	UVERBS_ATTR_PTR_OUT(CREATE_CQ_RESP_CQE, u32,
+			    UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_create_cq_handler(struct ib_device *ib_dev,
+			     struct ib_uverbs_file *file,
+			     struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_ucontext *ucontext = file->ucontext;
+	struct ib_ucq_object           *obj;
+	struct ib_udata uhw;
+	int ret;
+	u64 user_handle = 0;
+	struct ib_cq_init_attr attr = {};
+	struct ib_cq                   *cq;
+	struct ib_uverbs_completion_event_file    *ev_file = NULL;
+
+	ret = uverbs_copy_from(&attr.comp_vector, common, CREATE_CQ_COMP_VECTOR);
+	if (!ret)
+		ret = uverbs_copy_from(&attr.cqe, common, CREATE_CQ_CQE);
+	if (ret)
+		return ret;
+
+	/* Optional params, if they don't exist, we get -ENOENT and skip them */
+	if (uverbs_copy_from(&attr.flags, common, CREATE_CQ_FLAGS) == -EFAULT ||
+	    uverbs_copy_from(&user_handle, common, CREATE_CQ_USER_HANDLE) == -EFAULT)
+		return -EFAULT;
+
+	if (uverbs_is_valid(common, CREATE_CQ_COMP_CHANNEL)) {
+		struct ib_uobject *ev_file_uobj =
+			common->attrs[CREATE_CQ_COMP_CHANNEL].obj_attr.uobject;
+
+		ev_file = container_of(ev_file_uobj,
+				       struct ib_uverbs_completion_event_file,
+				       uobj_file.uobj);
+		kref_get(&ev_file_uobj->ref);
+	}
+
+	if (attr.comp_vector >= ucontext->ufile->device->num_comp_vectors)
+		return -EINVAL;
+
+	obj = container_of(common->attrs[CREATE_CQ_HANDLE].obj_attr.uobject,
+			   typeof(*obj), uobject);
+	obj->uverbs_file	   = ucontext->ufile;
+	obj->comp_events_reported  = 0;
+	obj->async_events_reported = 0;
+	INIT_LIST_HEAD(&obj->comp_list);
+	INIT_LIST_HEAD(&obj->async_list);
+
+	/* Temporary, only until drivers get the new uverbs_attr_array */
+	create_udata(ctx, num, &uhw);
+
+	cq = ib_dev->create_cq(ib_dev, &attr, ucontext, &uhw);
+	if (IS_ERR(cq))
+		return PTR_ERR(cq);
+
+	cq->device        = ib_dev;
+	cq->uobject       = &obj->uobject;
+	cq->comp_handler  = ib_uverbs_comp_handler;
+	cq->event_handler = ib_uverbs_cq_event_handler;
+	cq->cq_context    = &ev_file->ev_queue;
+	obj->uobject.object = cq;
+	obj->uobject.user_handle = user_handle;
+	atomic_set(&cq->usecnt, 0);
+
+	ret = uverbs_copy_to(common, CREATE_CQ_RESP_CQE, &cq->cqe);
+	if (ret)
+		goto err;
+
+	return 0;
+err:
+	ib_destroy_cq(cq);
+	return ret;
+};
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_destroy_cq_spec,
+	UVERBS_ATTR_IDR(DESTROY_CQ_HANDLE, UVERBS_TYPE_CQ,
+			UVERBS_ACCESS_DESTROY,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_OUT(DESTROY_CQ_RESP, struct ib_uverbs_destroy_cq_resp,
+			    UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_destroy_cq_handler(struct ib_device *ib_dev,
+			      struct ib_uverbs_file *file,
+			      struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_uverbs_destroy_cq_resp resp;
+	struct ib_uobject *uobj =
+		common->attrs[DESTROY_CQ_HANDLE].obj_attr.uobject;
+	struct ib_ucq_object *obj = container_of(uobj, struct ib_ucq_object,
+						 uobject);
+	int ret;
+
+	ret = rdma_explicit_destroy(uobj);
+	if (ret)
+		return ret;
+
+	resp.comp_events_reported  = obj->comp_events_reported;
+	resp.async_events_reported = obj->async_events_reported;
+
+	WARN_ON(uverbs_copy_to(common, DESTROY_CQ_RESP, &resp));
+	return 0;
+}
+
+static int qp_fill_attrs(struct ib_qp_init_attr *attr, struct ib_ucontext *ctx,
+			 const struct ib_uverbs_ioctl_create_qp *cmd,
+			 u32 create_flags)
+{
+	if (create_flags & ~(IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK |
+			     IB_QP_CREATE_CROSS_CHANNEL |
+			     IB_QP_CREATE_MANAGED_SEND |
+			     IB_QP_CREATE_MANAGED_RECV |
+			     IB_QP_CREATE_SCATTER_FCS))
+		return -EINVAL;
+
+	attr->create_flags = create_flags;
+	attr->event_handler = ib_uverbs_qp_event_handler;
+	attr->qp_context = ctx->ufile;
+	attr->sq_sig_type = cmd->sq_sig_all ? IB_SIGNAL_ALL_WR :
+		IB_SIGNAL_REQ_WR;
+	attr->qp_type = cmd->qp_type;
+
+	attr->cap.max_send_wr     = cmd->max_send_wr;
+	attr->cap.max_recv_wr     = cmd->max_recv_wr;
+	attr->cap.max_send_sge    = cmd->max_send_sge;
+	attr->cap.max_recv_sge    = cmd->max_recv_sge;
+	attr->cap.max_inline_data = cmd->max_inline_data;
+
+	return 0;
+}
+
+static void qp_init_uqp(struct ib_uqp_object *obj)
+{
+	obj->uevent.events_reported     = 0;
+	INIT_LIST_HEAD(&obj->uevent.event_list);
+	INIT_LIST_HEAD(&obj->mcast_list);
+}
+
+static int qp_write_resp(const struct ib_qp_init_attr *attr,
+			 const struct ib_qp *qp,
+			 struct uverbs_attr_array *common)
+{
+	struct ib_uverbs_ioctl_create_qp_resp resp = {
+		.qpn = qp->qp_num,
+		.max_recv_sge    = attr->cap.max_recv_sge,
+		.max_send_sge    = attr->cap.max_send_sge,
+		.max_recv_wr     = attr->cap.max_recv_wr,
+		.max_send_wr     = attr->cap.max_send_wr,
+		.max_inline_data = attr->cap.max_inline_data};
+
+	return uverbs_copy_to(common, CREATE_QP_RESP, &resp);
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_create_qp_spec,
+	UVERBS_ATTR_IDR(CREATE_QP_HANDLE, UVERBS_TYPE_QP, UVERBS_ACCESS_NEW,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_IDR(CREATE_QP_PD_HANDLE, UVERBS_TYPE_PD, UVERBS_ACCESS_READ,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_IDR(CREATE_QP_SEND_CQ, UVERBS_TYPE_CQ, UVERBS_ACCESS_READ,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_IDR(CREATE_QP_RECV_CQ, UVERBS_TYPE_CQ, UVERBS_ACCESS_READ,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_IDR(CREATE_QP_SRQ, UVERBS_TYPE_SRQ, UVERBS_ACCESS_READ),
+	UVERBS_ATTR_PTR_IN(CREATE_QP_USER_HANDLE, u64),
+	UVERBS_ATTR_PTR_IN(CREATE_QP_CMD, struct ib_uverbs_ioctl_create_qp),
+	UVERBS_ATTR_PTR_IN(CREATE_QP_CMD_FLAGS, u32),
+	UVERBS_ATTR_PTR_OUT(CREATE_QP_RESP, struct ib_uverbs_ioctl_create_qp_resp,
+			    UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_create_qp_handler(struct ib_device *ib_dev,
+			     struct ib_uverbs_file *file,
+			     struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_ucontext *ucontext = file->ucontext;
+	struct ib_uqp_object           *obj;
+	struct ib_udata uhw;
+	int ret;
+	u64 user_handle = 0;
+	u32 create_flags = 0;
+	struct ib_uverbs_ioctl_create_qp cmd;
+	struct ib_qp_init_attr attr = {};
+	struct ib_qp                   *qp;
+	struct ib_pd			*pd;
+
+	ret = uverbs_copy_from(&cmd, common, CREATE_QP_CMD);
+	if (ret)
+		return ret;
+
+	/* Optional params */
+	if (uverbs_copy_from(&create_flags, common, CREATE_QP_CMD_FLAGS) == -EFAULT ||
+	    uverbs_copy_from(&user_handle, common, CREATE_QP_USER_HANDLE) == -EFAULT)
+		return -EFAULT;
+
+	if (cmd.qp_type == IB_QPT_XRC_INI) {
+		cmd.max_recv_wr = 0;
+		cmd.max_recv_sge = 0;
+	}
+
+	ret = qp_fill_attrs(&attr, ucontext, &cmd, create_flags);
+	if (ret)
+		return ret;
+
+	pd = common->attrs[CREATE_QP_PD_HANDLE].obj_attr.uobject->object;
+	attr.send_cq = common->attrs[CREATE_QP_SEND_CQ].obj_attr.uobject->object;
+	attr.recv_cq = common->attrs[CREATE_QP_RECV_CQ].obj_attr.uobject->object;
+	if (uverbs_is_valid(common, CREATE_QP_SRQ))
+		attr.srq = common->attrs[CREATE_QP_SRQ].obj_attr.uobject->object;
+	obj = (struct ib_uqp_object *)common->attrs[CREATE_QP_HANDLE].obj_attr.uobject;
+
+	obj->uxrcd = NULL;
+	if (attr.srq && attr.srq->srq_type != IB_SRQT_BASIC)
+		return -EINVAL;
+
+	qp_init_uqp(obj);
+	create_udata(ctx, num, &uhw);
+	qp = pd->device->create_qp(pd, &attr, &uhw);
+	if (IS_ERR(qp))
+		return PTR_ERR(qp);
+	qp->real_qp	  = qp;
+	qp->device	  = pd->device;
+	qp->pd		  = pd;
+	qp->send_cq	  = attr.send_cq;
+	qp->recv_cq	  = attr.recv_cq;
+	qp->srq		  = attr.srq;
+	qp->event_handler = attr.event_handler;
+	qp->qp_context	  = attr.qp_context;
+	qp->qp_type	  = attr.qp_type;
+	atomic_set(&qp->usecnt, 0);
+	atomic_inc(&pd->usecnt);
+	atomic_inc(&attr.send_cq->usecnt);
+	if (attr.recv_cq)
+		atomic_inc(&attr.recv_cq->usecnt);
+	if (attr.srq)
+		atomic_inc(&attr.srq->usecnt);
+	qp->uobject = &obj->uevent.uobject;
+	obj->uevent.uobject.object = qp;
+	obj->uevent.uobject.user_handle = user_handle;
+
+	ret = qp_write_resp(&attr, qp, common);
+	if (ret) {
+		ib_destroy_qp(qp);
+		return ret;
+	}
+
+	return 0;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_create_qp_xrc_tgt_spec,
+	UVERBS_ATTR_IDR(CREATE_QP_XRC_TGT_HANDLE, UVERBS_TYPE_QP, UVERBS_ACCESS_NEW,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_IDR(CREATE_QP_XRC_TGT_XRCD, UVERBS_TYPE_XRCD, UVERBS_ACCESS_READ,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_IN(CREATE_QP_XRC_TGT_USER_HANDLE, u64),
+	UVERBS_ATTR_PTR_IN(CREATE_QP_XRC_TGT_CMD, struct ib_uverbs_ioctl_create_qp),
+	UVERBS_ATTR_PTR_IN(CREATE_QP_XRC_TGT_CMD_FLAGS, u32),
+	UVERBS_ATTR_PTR_OUT(CREATE_QP_XRC_TGT_RESP, struct ib_uverbs_ioctl_create_qp_resp,
+			    UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_create_qp_xrc_tgt_handler(struct ib_device *ib_dev,
+				     struct ib_uverbs_file *file,
+				     struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_ucontext *ucontext = file->ucontext;
+	struct ib_uqp_object           *obj;
+	int ret;
+	u64 user_handle = 0;
+	u32 create_flags = 0;
+	struct ib_uverbs_ioctl_create_qp cmd;
+	struct ib_qp_init_attr attr = {};
+	struct ib_qp                   *qp;
+
+	ret = uverbs_copy_from(&cmd, common, CREATE_QP_XRC_TGT_CMD);
+	if (ret)
+		return ret;
+
+	/* Optional params */
+	if (uverbs_copy_from(&create_flags, common, CREATE_QP_CMD_FLAGS) == -EFAULT ||
+	    uverbs_copy_from(&user_handle, common, CREATE_QP_USER_HANDLE) == -EFAULT)
+		return -EFAULT;
+
+	ret = qp_fill_attrs(&attr, ucontext, &cmd, create_flags);
+	if (ret)
+		return ret;
+
+	obj = (struct ib_uqp_object *)common->attrs[CREATE_QP_HANDLE].obj_attr.uobject;
+	obj->uxrcd = container_of(common->attrs[CREATE_QP_XRC_TGT_XRCD].obj_attr.uobject,
+				  struct ib_uxrcd_object, uobject);
+	attr.xrcd = obj->uxrcd->uobject.object;
+
+	qp_init_uqp(obj);
+	qp = ib_create_qp(NULL, &attr);
+	if (IS_ERR(qp))
+		return PTR_ERR(qp);
+	qp->uobject = &obj->uevent.uobject;
+	obj->uevent.uobject.object = qp;
+	obj->uevent.uobject.user_handle = user_handle;
+	atomic_inc(&obj->uxrcd->refcnt);
+
+	ret = qp_write_resp(&attr, qp, common);
+	if (ret) {
+		ib_destroy_qp(qp);
+		return ret;
+	}
+
+	return 0;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_destroy_qp_spec,
+	UVERBS_ATTR_IDR(DESTROY_QP_HANDLE, UVERBS_TYPE_QP,
+			UVERBS_ACCESS_DESTROY,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_OUT(DESTROY_QP_EVENTS_REPORTED,
+			    __u32,
+			    UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)));
+
+int uverbs_destroy_qp_handler(struct ib_device *ib_dev,
+			      struct ib_uverbs_file *file,
+			      struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_uobject *uobj =
+		common->attrs[DESTROY_QP_HANDLE].obj_attr.uobject;
+	struct ib_uqp_object *obj = container_of(uobj, struct ib_uqp_object,
+						 uevent.uobject);
+	int ret;
+
+	ret = rdma_explicit_destroy(uobj);
+	if (ret)
+		return ret;
+
+	WARN_ON(uverbs_copy_to(common, DESTROY_QP_EVENTS_REPORTED,
+			       &obj->uevent.events_reported));
+	return 0;
+}
+
+DECLARE_UVERBS_ATTR_SPEC(
+	uverbs_modify_qp_spec,
+	UVERBS_ATTR_IDR(MODIFY_QP_HANDLE, UVERBS_TYPE_QP, UVERBS_ACCESS_WRITE,
+			UA_FLAGS(UVERBS_ATTR_SPEC_F_MANDATORY)),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_STATE, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_CUR_STATE, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_EN_SQD_ASYNC_NOTIFY, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_ACCESS_FLAGS, u32),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_PKEY_INDEX, u16),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_PORT, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_QKEY, u32),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_AV, struct ib_uverbs_qp_dest),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_PATH_MTU, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_TIMEOUT, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_RETRY_CNT, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_RNR_RETRY, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_RQ_PSN, u32),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_MAX_RD_ATOMIC, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_ALT_PATH, struct ib_uverbs_qp_alt_path),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_MIN_RNR_TIMER, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_SQ_PSN, u32),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_MAX_DEST_RD_ATOMIC, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_PATH_MIG_STATE, u8),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_DEST_QPN, u32),
+	UVERBS_ATTR_PTR_IN(MODIFY_QP_RATE_LIMIT, u32));
+
+int uverbs_modify_qp_handler(struct ib_device *ib_dev,
+			     struct ib_uverbs_file *file,
+			     struct uverbs_attr_array *ctx, size_t num)
+{
+	struct uverbs_attr_array *common = &ctx[0];
+	struct ib_udata            uhw;
+	struct ib_qp              *qp;
+	struct ib_qp_attr         *attr;
+	struct ib_uverbs_qp_dest  av;
+	struct ib_uverbs_qp_alt_path alt_path;
+	u32 attr_mask = 0;
+	int ret = 0;
+
+	qp = common->attrs[MODIFY_QP_HANDLE].obj_attr.uobject->object;
+	attr = kzalloc(sizeof(*attr), GFP_KERNEL);
+	if (!attr)
+		return -ENOMEM;
+
+#define MODIFY_QP_CPY(_param, _fld, _attr)				\
+	({								\
+		int ret = uverbs_copy_from(_fld, common, _param);	\
+		if (!ret)						\
+			attr_mask |= _attr;				\
+		ret == -EFAULT ? ret : 0;				\
+	})
+
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_STATE, &attr->qp_state,
+				   IB_QP_STATE);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_CUR_STATE, &attr->cur_qp_state,
+				   IB_QP_CUR_STATE);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_EN_SQD_ASYNC_NOTIFY,
+				   &attr->en_sqd_async_notify,
+				   IB_QP_EN_SQD_ASYNC_NOTIFY);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_ACCESS_FLAGS,
+				   &attr->qp_access_flags, IB_QP_ACCESS_FLAGS);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_PKEY_INDEX, &attr->pkey_index,
+				   IB_QP_PKEY_INDEX);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_PORT, &attr->port_num, IB_QP_PORT);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_QKEY, &attr->qkey, IB_QP_QKEY);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_PATH_MTU, &attr->path_mtu,
+				   IB_QP_PATH_MTU);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_TIMEOUT, &attr->timeout,
+				   IB_QP_TIMEOUT);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_RETRY_CNT, &attr->retry_cnt,
+				   IB_QP_RETRY_CNT);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_RNR_RETRY, &attr->rnr_retry,
+				   IB_QP_RNR_RETRY);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_RQ_PSN, &attr->rq_psn,
+				   IB_QP_RQ_PSN);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_MAX_RD_ATOMIC,
+				   &attr->max_rd_atomic,
+				   IB_QP_MAX_QP_RD_ATOMIC);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_MIN_RNR_TIMER,
+				   &attr->min_rnr_timer, IB_QP_MIN_RNR_TIMER);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_SQ_PSN, &attr->sq_psn,
+				   IB_QP_SQ_PSN);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_MAX_DEST_RD_ATOMIC,
+				   &attr->max_dest_rd_atomic,
+				   IB_QP_MAX_DEST_RD_ATOMIC);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_PATH_MIG_STATE,
+				   &attr->path_mig_state, IB_QP_PATH_MIG_STATE);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_DEST_QPN, &attr->dest_qp_num,
+				   IB_QP_DEST_QPN);
+	ret = ret ?: MODIFY_QP_CPY(MODIFY_QP_RATE_LIMIT, &attr->rate_limit,
+				   IB_QP_RATE_LIMIT);
+
+	if (ret)
+		goto err;
+
+	ret = uverbs_copy_from(&av, common, MODIFY_QP_AV);
+	if (!ret) {
+		attr_mask |= IB_QP_AV;
+		memcpy(attr->ah_attr.grh.dgid.raw, av.dgid, 16);
+		attr->ah_attr.grh.flow_label        = av.flow_label;
+		attr->ah_attr.grh.sgid_index        = av.sgid_index;
+		attr->ah_attr.grh.hop_limit         = av.hop_limit;
+		attr->ah_attr.grh.traffic_class     = av.traffic_class;
+		attr->ah_attr.dlid		    = av.dlid;
+		attr->ah_attr.sl		    = av.sl;
+		attr->ah_attr.src_path_bits	    = av.src_path_bits;
+		attr->ah_attr.static_rate	    = av.static_rate;
+		attr->ah_attr.ah_flags		    = av.is_global ? IB_AH_GRH : 0;
+		attr->ah_attr.port_num		    = av.port_num;
+	} else if (ret == -EFAULT) {
+		goto err;
+	}
+
+	ret = uverbs_copy_from(&alt_path, common, MODIFY_QP_ALT_PATH);
+	if (!ret) {
+		attr_mask |= IB_QP_ALT_PATH;
+		memcpy(attr->alt_ah_attr.grh.dgid.raw, alt_path.dest.dgid, 16);
+		attr->alt_ah_attr.grh.flow_label    = alt_path.dest.flow_label;
+		attr->alt_ah_attr.grh.sgid_index    = alt_path.dest.sgid_index;
+		attr->alt_ah_attr.grh.hop_limit     = alt_path.dest.hop_limit;
+		attr->alt_ah_attr.grh.traffic_class = alt_path.dest.traffic_class;
+		attr->alt_ah_attr.dlid		    = alt_path.dest.dlid;
+		attr->alt_ah_attr.sl		    = alt_path.dest.sl;
+		attr->alt_ah_attr.src_path_bits     = alt_path.dest.src_path_bits;
+		attr->alt_ah_attr.static_rate       = alt_path.dest.static_rate;
+		attr->alt_ah_attr.ah_flags	    = alt_path.dest.is_global ? IB_AH_GRH : 0;
+		attr->alt_ah_attr.port_num	    = alt_path.dest.port_num;
+		attr->alt_pkey_index		    = alt_path.pkey_index;
+		attr->alt_port_num		    = alt_path.port_num;
+		attr->alt_timeout		    = alt_path.timeout;
+	} else if (ret == -EFAULT) {
+		goto err;
+	}
+
+	create_udata(ctx, num, &uhw);
+
+	if (qp->real_qp == qp) {
+		if (attr_mask & IB_QP_AV) {
+			ret = ib_resolve_eth_dmac(qp->device, &attr->ah_attr);
+			if (ret)
+				goto err;
+		}
+		ret = qp->device->modify_qp(qp, attr,
+					    modify_qp_mask(qp->qp_type,
+							   attr_mask),
+					    &uhw);
+	} else {
+		ret = ib_modify_qp(qp, attr, modify_qp_mask(qp->qp_type,
+							    attr_mask));
+	}
+
+	if (ret)
+		goto err;
+
+	return 0;
+err:
+	kfree(attr);
+	return ret;
+}
+
 DECLARE_UVERBS_TYPE(uverbs_type_comp_channel,
 		    &UVERBS_TYPE_ALLOC_FD(0,
 					  sizeof(struct ib_uverbs_completion_event_file),
 					  uverbs_hot_unplug_completion_event_file,
 					  &uverbs_event_fops,
-					  "[infinibandevent]", O_RDONLY));
+					  "[infinibandevent]", O_RDONLY),
+		    &UVERBS_ACTIONS(
+			ADD_UVERBS_ACTION(UVERBS_COMP_CHANNEL_CREATE,
+					  uverbs_create_comp_channel_handler,
+					  &uverbs_create_comp_channel_spec)));
 
 DECLARE_UVERBS_TYPE(uverbs_type_cq,
 		    &UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_ucq_object), 0,
-					      uverbs_free_cq));
+					      uverbs_free_cq),
+		    &UVERBS_ACTIONS(
+			ADD_UVERBS_ACTION(UVERBS_CQ_CREATE,
+					  uverbs_create_cq_handler,
+					  &uverbs_create_cq_spec,
+					  &uverbs_uhw_compat_spec),
+			ADD_UVERBS_ACTION(UVERBS_CQ_DESTROY,
+					  uverbs_destroy_cq_handler,
+					  &uverbs_destroy_cq_spec)));
 
 DECLARE_UVERBS_TYPE(uverbs_type_qp,
 		    &UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uqp_object), 0,
-					      uverbs_free_qp));
+					      uverbs_free_qp),
+		    &UVERBS_ACTIONS(
+			ADD_UVERBS_ACTION(UVERBS_QP_CREATE,
+					  uverbs_create_qp_handler,
+					  &uverbs_create_qp_spec,
+					  &uverbs_uhw_compat_spec),
+			ADD_UVERBS_ACTION(UVERBS_QP_CREATE_XRC_TGT,
+					  uverbs_create_qp_xrc_tgt_handler,
+					  &uverbs_create_qp_xrc_tgt_spec),
+			ADD_UVERBS_ACTION(UVERBS_QP_MODIFY,
+					  uverbs_modify_qp_handler,
+					  &uverbs_modify_qp_spec,
+					  &uverbs_uhw_compat_spec),
+			ADD_UVERBS_ACTION(UVERBS_QP_DESTROY,
+					  uverbs_destroy_qp_handler,
+					  &uverbs_destroy_qp_spec)),
+);
 
 DECLARE_UVERBS_TYPE(uverbs_type_mw,
 		    &UVERBS_TYPE_ALLOC_IDR(0, uverbs_free_mw));
 
 DECLARE_UVERBS_TYPE(uverbs_type_mr,
 		    /* 1 is used in order to free the MR after all the MWs */
-		    &UVERBS_TYPE_ALLOC_IDR(1, uverbs_free_mr));
+		    &UVERBS_TYPE_ALLOC_IDR(1, uverbs_free_mr),
+		    &UVERBS_ACTIONS(
+			ADD_UVERBS_ACTION(UVERBS_MR_REG, uverbs_reg_mr_handler,
+					  &uverbs_reg_mr_spec,
+					  &uverbs_uhw_compat_spec),
+			ADD_UVERBS_ACTION(UVERBS_MR_DEREG,
+					  uverbs_dereg_mr_handler,
+					  &uverbs_dereg_mr_spec)));
 
 DECLARE_UVERBS_TYPE(uverbs_type_srq,
 		    &UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_usrq_object), 0,
@@ -254,9 +1206,29 @@ int uverbs_hot_unplug_completion_event_file(struct ib_uobject_file *uobj_file,
 
 DECLARE_UVERBS_TYPE(uverbs_type_pd,
 		    /* 2 is used in order to free the PD after MRs */
-		    &UVERBS_TYPE_ALLOC_IDR(2, uverbs_free_pd));
+		    &UVERBS_TYPE_ALLOC_IDR(2, uverbs_free_pd),
+		    &UVERBS_ACTIONS(
+			ADD_UVERBS_ACTION(UVERBS_PD_ALLOC,
+					  uverbs_alloc_pd_handler,
+					  &uverbs_alloc_pd_spec,
+					  &uverbs_uhw_compat_spec),
+			ADD_UVERBS_ACTION(UVERBS_PD_DEALLOC,
+					  uverbs_dealloc_pd_handler,
+					  &uverbs_dealloc_pd_spec)));
 
-DECLARE_UVERBS_TYPE(uverbs_type_device, NULL);
+DECLARE_UVERBS_TYPE(uverbs_type_device, NULL,
+		    &UVERBS_ACTIONS(
+			ADD_UVERBS_CTX_ACTION(UVERBS_DEVICE_ALLOC_CONTEXT,
+					      uverbs_get_context,
+					      &uverbs_get_context_spec,
+					      &uverbs_uhw_compat_spec),
+			ADD_UVERBS_ACTION(UVERBS_DEVICE_QUERY,
+					  &uverbs_query_device_handler,
+					  &uverbs_query_device_spec,
+					  &uverbs_uhw_compat_spec),
+			ADD_UVERBS_ACTION(UVERBS_DEVICE_PORT_QUERY,
+					  &uverbs_query_port_handler,
+					  &uverbs_query_port_spec)));
 
 DECLARE_UVERBS_TYPES(uverbs_common_types,
 		     ADD_UVERBS_TYPE(UVERBS_TYPE_DEVICE, uverbs_type_device),
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 86ecd3e..fd25303 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -53,6 +53,8 @@
 #include <rdma/ib_smi.h>
 #include <rdma/ib_umem.h>
 #include <rdma/ib_user_verbs.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 
 #include "cxio_hal.h"
 #include "iwch.h"
@@ -1353,6 +1355,8 @@ static void get_dev_fw_ver_str(struct ib_device *ibdev, char *str,
 	snprintf(str, str_len, "%s", info.fw_version);
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 int iwch_register_device(struct iwch_dev *dev)
 {
 	int ret;
@@ -1446,6 +1450,7 @@ int iwch_register_device(struct iwch_dev *dev)
 	memcpy(dev->ibdev.iwcm->ifname, dev->rdev.t3cdev_p->lldev->name,
 	       sizeof(dev->ibdev.iwcm->ifname));
 
+	dev->ibdev.specs_root = (struct uverbs_root *)&root;
 	ret = ib_register_device(&dev->ibdev, NULL);
 	if (ret)
 		goto bail1;
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index df64417..0c3f560 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -51,6 +51,8 @@
 #include <rdma/ib_smi.h>
 #include <rdma/ib_umem.h>
 #include <rdma/ib_user_verbs.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 
 #include "iw_cxgb4.h"
 
@@ -533,6 +535,8 @@ static void get_dev_fw_str(struct ib_device *dev, char *str,
 		 FW_HDR_FW_VER_BUILD_G(c4iw_dev->rdev.lldi.fw_vers));
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 int c4iw_register_device(struct c4iw_dev *dev)
 {
 	int ret;
@@ -626,6 +630,7 @@ int c4iw_register_device(struct c4iw_dev *dev)
 	memcpy(dev->ibdev.iwcm->ifname, dev->rdev.lldi.ports[0]->name,
 	       sizeof(dev->ibdev.iwcm->ifname));
 
+	dev->ibdev.specs_root = (struct uverbs_root *)&root;
 	ret = ib_register_device(&dev->ibdev, NULL);
 	if (ret)
 		goto bail1;
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index c3b41f9..dee7754 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -37,6 +37,8 @@
 #include <rdma/ib_smi.h>
 #include <rdma/ib_user_verbs.h>
 #include <rdma/ib_cache.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 #include "hns_roce_common.h"
 #include "hns_roce_device.h"
 #include <rdma/hns-abi.h>
@@ -424,6 +426,8 @@ static void hns_roce_unregister_device(struct hns_roce_dev *hr_dev)
 	ib_unregister_device(&hr_dev->ib_dev);
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 static int hns_roce_register_device(struct hns_roce_dev *hr_dev)
 {
 	int ret;
@@ -507,6 +511,7 @@ static int hns_roce_register_device(struct hns_roce_dev *hr_dev)
 	/* OTHERS */
 	ib_dev->get_port_immutable	= hns_roce_port_immutable;
 
+	dev->ib_dev.specs_root = (struct uverbs_root *)&root;
 	ret = ib_register_device(ib_dev, NULL);
 	if (ret) {
 		dev_err(dev, "ib_register_device failed!\n");
diff --git a/drivers/infiniband/hw/i40iw/i40iw_verbs.c b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
index 9b28499..94c57bf 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_verbs.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
@@ -43,6 +43,8 @@
 #include <rdma/ib_verbs.h>
 #include <rdma/iw_cm.h>
 #include <rdma/ib_user_verbs.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 #include <rdma/ib_umem.h>
 #include "i40iw.h"
 
@@ -2859,6 +2861,8 @@ void i40iw_destroy_rdma_device(struct i40iw_ib_device *iwibdev)
 	ib_dealloc_device(&iwibdev->ibdev);
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 /**
  * i40iw_register_rdma_device - register iwarp device to IB
  * @iwdev: iwarp device
@@ -2873,6 +2877,7 @@ int i40iw_register_rdma_device(struct i40iw_device *iwdev)
 		return -ENOMEM;
 	iwibdev = iwdev->iwibdev;
 
+	iwibdev->ibdev.specs_root = (struct uverbs_root *)&root;
 	ret = ib_register_device(&iwibdev->ibdev, NULL);
 	if (ret)
 		goto error;
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index fba94df..33ef9c8 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -48,6 +48,8 @@
 
 #include <rdma/ib_smi.h>
 #include <rdma/ib_user_verbs.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 #include <rdma/ib_addr.h>
 #include <rdma/ib_cache.h>
 
@@ -2576,6 +2578,8 @@ static void get_fw_ver_str(struct ib_device *device, char *str,
 		 (int) dev->dev->caps.fw_ver & 0xffff);
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 static void *mlx4_ib_add(struct mlx4_dev *dev)
 {
 	struct mlx4_ib_dev *ibdev;
@@ -2858,6 +2862,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
 	if (mlx4_ib_alloc_diag_counters(ibdev))
 		goto err_steer_free_bitmap;
 
+	ibdev->ib_dev.specs_root = (struct uverbs_root *)&root;
 	if (ib_register_device(&ibdev->ib_dev, NULL))
 		goto err_diag_counters;
 
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 4dc0a87..0dc6da9 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -57,6 +57,8 @@
 #include <linux/mlx5/fs.h>
 #include <linux/mlx5/vport.h>
 #include "mlx5_ib.h"
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 
 #define DRIVER_NAME "mlx5_ib"
 #define DRIVER_VERSION "2.2-1"
@@ -3319,6 +3321,8 @@ static int mlx5_ib_get_hw_stats(struct ib_device *ibdev,
 	return port->q_cnts.num_counters;
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 static void *mlx5_ib_add(struct mlx5_core_dev *mdev)
 {
 	struct mlx5_ib_dev *dev;
@@ -3540,6 +3544,7 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev)
 	if (err)
 		goto err_bfreg;
 
+	dev->ib_dev.specs_root = (struct uverbs_root *)&root;
 	err = ib_register_device(&dev->ib_dev, NULL);
 	if (err)
 		goto err_fp_bfreg;
diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c
index 22d0e6e..4758b38 100644
--- a/drivers/infiniband/hw/mthca/mthca_provider.c
+++ b/drivers/infiniband/hw/mthca/mthca_provider.c
@@ -37,6 +37,8 @@
 #include <rdma/ib_smi.h>
 #include <rdma/ib_umem.h>
 #include <rdma/ib_user_verbs.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 
 #include <linux/sched.h>
 #include <linux/slab.h>
@@ -1190,6 +1192,8 @@ static void get_dev_fw_str(struct ib_device *device, char *str,
 		 (int) dev->fw_ver & 0xffff);
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 int mthca_register_device(struct mthca_dev *dev)
 {
 	int ret;
@@ -1297,6 +1301,7 @@ int mthca_register_device(struct mthca_dev *dev)
 
 	mutex_init(&dev->cap_mask_mutex);
 
+	dev->ib_dev.specs_root = (struct uverbs_root *)&root;
 	ret = ib_register_device(&dev->ib_dev, NULL);
 	if (ret)
 		return ret;
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index ccf0a4c..db42cc8 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -41,6 +41,8 @@
 #include <rdma/ib_verbs.h>
 #include <rdma/iw_cm.h>
 #include <rdma/ib_user_verbs.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 
 #include "nes.h"
 
@@ -3852,6 +3854,8 @@ void nes_destroy_ofa_device(struct nes_ib_device *nesibdev)
 }
 
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 /**
  * nes_register_ofa_device
  */
@@ -3862,6 +3866,7 @@ int nes_register_ofa_device(struct nes_ib_device *nesibdev)
 	struct nes_adapter *nesadapter = nesdev->nesadapter;
 	int i, ret;
 
+	nesvnic->nesibdev->ibdev.specs_root = (struct uverbs_root *)&root;
 	ret = ib_register_device(&nesvnic->nesibdev->ibdev, NULL);
 	if (ret) {
 		return ret;
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_main.c b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
index 57c9a2a..ea4831e 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_main.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
@@ -44,6 +44,8 @@
 #include <linux/idr.h>
 #include <rdma/ib_verbs.h>
 #include <rdma/ib_user_verbs.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 #include <rdma/ib_addr.h>
 #include <rdma/ib_mad.h>
 
@@ -116,6 +118,8 @@ static void get_dev_fw_str(struct ib_device *device, char *str,
 	snprintf(str, str_len, "%s", &dev->attr.fw_ver[0]);
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 static int ocrdma_register_device(struct ocrdma_dev *dev)
 {
 	strlcpy(dev->ibdev.name, "ocrdma%d", IB_DEVICE_NAME_MAX);
@@ -219,6 +223,7 @@ static int ocrdma_register_device(struct ocrdma_dev *dev)
 		dev->ibdev.destroy_srq = ocrdma_destroy_srq;
 		dev->ibdev.post_srq_recv = ocrdma_post_srq_recv;
 	}
+	dev->ibdev.specs_root = (struct uverbs_root *)&root;
 	return ib_register_device(&dev->ibdev, NULL);
 }
 
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_main.c b/drivers/infiniband/hw/usnic/usnic_ib_main.c
index c0c1e8b..a4699c4 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_main.c
+++ b/drivers/infiniband/hw/usnic/usnic_ib_main.c
@@ -48,6 +48,8 @@
 #include <linux/netdevice.h>
 
 #include <rdma/ib_user_verbs.h>
+#include <rdma/uverbs_ioctl.h>
+#include <rdma/uverbs_std_types.h>
 #include <rdma/ib_addr.h>
 
 #include "usnic_abi.h"
@@ -348,6 +350,8 @@ static void usnic_get_dev_fw_str(struct ib_device *device,
 	snprintf(str, str_len, "%s", info.fw_version);
 }
 
+DECLARE_UVERBS_TYPES_GROUP(root, &uverbs_common_types);
+
 /* Start of PF discovery section */
 static void *usnic_ib_device_add(struct pci_dev *dev)
 {
@@ -434,6 +438,7 @@ static void *usnic_ib_device_add(struct pci_dev *dev)
 	us_ibdev->ib_dev.get_dev_fw_str     = usnic_get_dev_fw_str;
 
 
+	us_ibdev->ib_dev.specs_root = (struct uverbs_root *)&root;
 	if (ib_register_device(&us_ibdev->ib_dev, NULL))
 		goto err_fwd_dealloc;
 
diff --git a/include/rdma/uverbs_std_types.h b/include/rdma/uverbs_std_types.h
index 1b633f1..04d64c9 100644
--- a/include/rdma/uverbs_std_types.h
+++ b/include/rdma/uverbs_std_types.h
@@ -35,6 +35,15 @@
 
 #include <rdma/uverbs_types.h>
 
+#define UVERBS_UDATA_DRIVER_DATA_GROUP	1
+#define UVERBS_UDATA_DRIVER_DATA_FLAG	BIT(UVERBS_ID_RESERVED_SHIFT)
+
+enum {
+	UVERBS_UHW_IN,
+	UVERBS_UHW_OUT,
+	UVERBS_UHW_NUM
+};
+
 enum uverbs_common_types {
 	UVERBS_TYPE_DEVICE, /* No instances of DEVICE are allowed */
 	UVERBS_TYPE_PD,
@@ -52,6 +61,156 @@ enum uverbs_common_types {
 	UVERBS_TYPE_LAST,
 };
 
+enum uverbs_create_qp_cmd_attr_ids {
+	CREATE_QP_HANDLE,
+	CREATE_QP_PD_HANDLE,
+	CREATE_QP_SEND_CQ,
+	CREATE_QP_RECV_CQ,
+	CREATE_QP_SRQ,
+	CREATE_QP_USER_HANDLE,
+	CREATE_QP_CMD,
+	CREATE_QP_CMD_FLAGS,
+	CREATE_QP_RESP
+};
+
+enum uverbs_destroy_qp_cmd_attr_ids {
+	DESTROY_QP_HANDLE,
+	DESTROY_QP_EVENTS_REPORTED,
+};
+
+enum uverbs_create_cq_cmd_attr_ids {
+	CREATE_CQ_HANDLE,
+	CREATE_CQ_CQE,
+	CREATE_CQ_USER_HANDLE,
+	CREATE_CQ_COMP_CHANNEL,
+	CREATE_CQ_COMP_VECTOR,
+	CREATE_CQ_FLAGS,
+	CREATE_CQ_RESP_CQE,
+};
+
+enum uverbs_destroy_cq_cmd_attr_ids {
+	DESTROY_CQ_HANDLE,
+	DESTROY_CQ_RESP
+};
+
+enum uverbs_create_qp_xrc_tgt_cmd_attr_ids {
+	CREATE_QP_XRC_TGT_HANDLE,
+	CREATE_QP_XRC_TGT_XRCD,
+	CREATE_QP_XRC_TGT_USER_HANDLE,
+	CREATE_QP_XRC_TGT_CMD,
+	CREATE_QP_XRC_TGT_CMD_FLAGS,
+	CREATE_QP_XRC_TGT_RESP
+};
+
+enum uverbs_modify_qp_cmd_attr_ids {
+	MODIFY_QP_HANDLE,
+	MODIFY_QP_STATE,
+	MODIFY_QP_CUR_STATE,
+	MODIFY_QP_EN_SQD_ASYNC_NOTIFY,
+	MODIFY_QP_ACCESS_FLAGS,
+	MODIFY_QP_PKEY_INDEX,
+	MODIFY_QP_PORT,
+	MODIFY_QP_QKEY,
+	MODIFY_QP_AV,
+	MODIFY_QP_PATH_MTU,
+	MODIFY_QP_TIMEOUT,
+	MODIFY_QP_RETRY_CNT,
+	MODIFY_QP_RNR_RETRY,
+	MODIFY_QP_RQ_PSN,
+	MODIFY_QP_MAX_RD_ATOMIC,
+	MODIFY_QP_ALT_PATH,
+	MODIFY_QP_MIN_RNR_TIMER,
+	MODIFY_QP_SQ_PSN,
+	MODIFY_QP_MAX_DEST_RD_ATOMIC,
+	MODIFY_QP_PATH_MIG_STATE,
+	MODIFY_QP_DEST_QPN,
+	MODIFY_QP_RATE_LIMIT,
+};
+
+enum uverbs_create_comp_channel_cmd_attr_ids {
+	CREATE_COMP_CHANNEL_FD,
+};
+
+enum uverbs_get_context_cmd_attr_ids {
+	GET_CONTEXT_RESP,
+};
+
+enum uverbs_query_device_cmd_attr_ids {
+	QUERY_DEVICE_RESP,
+	QUERY_DEVICE_ODP,
+	QUERY_DEVICE_TIMESTAMP_MASK,
+	QUERY_DEVICE_HCA_CORE_CLOCK,
+	QUERY_DEVICE_CAP_FLAGS,
+};
+
+enum uverbs_query_port_cmd_attr_ids {
+	QUERY_PORT_PORT_NUM,
+	QUERY_PORT_RESP,
+};
+
+enum uverbs_alloc_pd_cmd_attr_ids {
+	ALLOC_PD_HANDLE,
+};
+
+enum uverbs_dealloc_pd_cmd_attr_ids {
+	DEALLOC_PD_HANDLE,
+};
+
+enum uverbs_reg_mr_cmd_attr_ids {
+	REG_MR_HANDLE,
+	REG_MR_PD_HANDLE,
+	REG_MR_CMD,
+	REG_MR_RESP
+};
+
+enum uverbs_dereg_mr_cmd_attr_ids {
+	DEREG_MR_HANDLE,
+};
+
+enum uverbs_actions_mr_ops {
+	UVERBS_MR_REG,
+	UVERBS_MR_DEREG,
+};
+
+extern const struct uverbs_action_group uverbs_actions_mr;
+
+enum uverbs_actions_comp_channel_ops {
+	UVERBS_COMP_CHANNEL_CREATE,
+};
+
+extern const struct uverbs_action_group uverbs_actions_comp_channel;
+
+enum uverbs_actions_cq_ops {
+	UVERBS_CQ_CREATE,
+	UVERBS_CQ_DESTROY,
+};
+
+extern const struct uverbs_action_group uverbs_actions_cq;
+
+enum uverbs_actions_qp_ops {
+	UVERBS_QP_CREATE,
+	UVERBS_QP_CREATE_XRC_TGT,
+	UVERBS_QP_MODIFY,
+	UVERBS_QP_DESTROY,
+};
+
+extern const struct uverbs_action_group uverbs_actions_qp;
+
+enum uverbs_actions_pd_ops {
+	UVERBS_PD_ALLOC,
+	UVERBS_PD_DEALLOC
+};
+
+extern const struct uverbs_action_group uverbs_actions_pd;
+
+enum uverbs_actions_device_ops {
+	UVERBS_DEVICE_ALLOC_CONTEXT,
+	UVERBS_DEVICE_QUERY,
+	UVERBS_DEVICE_PORT_QUERY,
+};
+
+extern const struct uverbs_action_group uverbs_actions_device;
+
 extern const struct uverbs_type uverbs_type_comp_channel;
 extern const struct uverbs_type uverbs_type_cq;
 extern const struct uverbs_type uverbs_type_qp;
diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h
index 997f904..31b8666 100644
--- a/include/uapi/rdma/ib_user_verbs.h
+++ b/include/uapi/rdma/ib_user_verbs.h
@@ -318,12 +318,25 @@ struct ib_uverbs_reg_mr {
 	__u64 driver_data[0];
 };
 
+struct ib_uverbs_ioctl_reg_mr {
+	__u64 start;
+	__u64 length;
+	__u64 hca_va;
+	__u32 access_flags;
+	__u32 reserved;
+};
+
 struct ib_uverbs_reg_mr_resp {
 	__u32 mr_handle;
 	__u32 lkey;
 	__u32 rkey;
 };
 
+struct ib_uverbs_ioctl_reg_mr_resp {
+	__u32 lkey;
+	__u32 rkey;
+};
+
 struct ib_uverbs_rereg_mr {
 	__u64 response;
 	__u32 mr_handle;
@@ -581,6 +594,17 @@ struct ib_uverbs_ex_create_qp {
 	__u32  reserved1;
 };
 
+struct ib_uverbs_ioctl_create_qp {
+	__u32 max_send_wr;
+	__u32 max_recv_wr;
+	__u32 max_send_sge;
+	__u32 max_recv_sge;
+	__u32 max_inline_data;
+	__u8  sq_sig_all;
+	__u8  qp_type;
+	__u16 reserved;
+};
+
 struct ib_uverbs_open_qp {
 	__u64 response;
 	__u64 user_handle;
@@ -603,6 +627,15 @@ struct ib_uverbs_create_qp_resp {
 	__u32 reserved;
 };
 
+struct ib_uverbs_ioctl_create_qp_resp {
+	__u32 qpn;
+	__u32 max_send_wr;
+	__u32 max_recv_wr;
+	__u32 max_send_sge;
+	__u32 max_recv_sge;
+	__u32 max_inline_data;
+};
+
 struct ib_uverbs_ex_create_qp_resp {
 	struct ib_uverbs_create_qp_resp base;
 	__u32 comp_mask;
@@ -628,6 +661,13 @@ struct ib_uverbs_qp_dest {
 	__u8  port_num;
 };
 
+struct ib_uverbs_qp_alt_path {
+	struct ib_uverbs_qp_dest dest;
+	__u16 pkey_index;
+	__u8  port_num;
+	__u8  timeout;
+};
+
 struct ib_uverbs_query_qp {
 	__u64 response;
 	__u32 qp_handle;
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC 10/10] IB/core: Expose ioctl interface through experimental Kconfig

[Matan Barak]

Add CONFIG_INFINIBAND_EXP_USER_ACCESS that enables the ioctl
interface. This interface is experimental and is subject to change.

Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/Kconfig            | 7 +++++++
 drivers/infiniband/core/uverbs.h      | 2 ++
 drivers/infiniband/core/uverbs_main.c | 6 ++++++
 3 files changed, 15 insertions(+)

diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index 66f8602..cabbb11 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -34,6 +34,13 @@ config INFINIBAND_USER_ACCESS
 	  libibverbs, libibcm and a hardware driver library from
 	  <http://www.openfabrics.org/git/>.
 
+config INFINIBAND_EXP_USER_ACCESS
+	bool "Allow experimental support for Infiniband ABI"
+	depends on INFINIBAND_USER_ACCESS
+	---help---
+	  IOCTL based ABI support for Infiniband. This allows userspace
+	  to invoke the experimental IOCTL based ABI.
+
 config INFINIBAND_USER_MEM
 	bool
 	depends on INFINIBAND_USER_ACCESS != n
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index ef37a16..2d1330f 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -225,6 +225,8 @@ void uverbs_copy_query_dev_fields(struct ib_device *ib_dev,
 void ib_uverbs_detach_umcast(struct ib_qp *qp,
 			     struct ib_uqp_object *uobj);
 
+long ib_uverbs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg);
+
 struct ib_uverbs_flow_spec {
 	union {
 		union {
diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
index 44c4d92..99e70cb 100644
--- a/drivers/infiniband/core/uverbs_main.c
+++ b/drivers/infiniband/core/uverbs_main.c
@@ -950,6 +950,9 @@ static int ib_uverbs_close(struct inode *inode, struct file *filp)
 	.open	 = ib_uverbs_open,
 	.release = ib_uverbs_close,
 	.llseek	 = no_llseek,
+#if IS_ENABLED(CONFIG_INFINIBAND_EXP_USER_ACCESS)
+	.unlocked_ioctl = ib_uverbs_ioctl,
+#endif
 };
 
 static const struct file_operations uverbs_mmap_fops = {
@@ -959,6 +962,9 @@ static int ib_uverbs_close(struct inode *inode, struct file *filp)
 	.open	 = ib_uverbs_open,
 	.release = ib_uverbs_close,
 	.llseek	 = no_llseek,
+#if IS_ENABLED(CONFIG_INFINIBAND_EXP_USER_ACCESS)
+	.unlocked_ioctl = ib_uverbs_ioctl,
+#endif
 };
 
 static struct ib_client uverbs_client = {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html