From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-rdma-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
X-Spam-Level: 
X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,
	SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham
	autolearn_force=no version=3.4.0
Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 95783C63798
	for <linux-rdma@archiver.kernel.org>; Tue, 24 Nov 2020 10:32:13 +0000 (UTC)
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by mail.kernel.org (Postfix) with ESMTP id 380B12076B
	for <linux-rdma@archiver.kernel.org>; Tue, 24 Nov 2020 10:32:13 +0000 (UTC)
Authentication-Results: mail.kernel.org;
	dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="OkIBq2Z8"
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1732085AbgKXKb6 (ORCPT <rfc822;linux-rdma@archiver.kernel.org>);
        Tue, 24 Nov 2020 05:31:58 -0500
Received: from hqnvemgate25.nvidia.com ([216.228.121.64]:10440 "EHLO
        hqnvemgate25.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1732084AbgKXKb6 (ORCPT
        <rfc822;linux-rdma@vger.kernel.org>); Tue, 24 Nov 2020 05:31:58 -0500
Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, AES256-SHA)
        id <B5fbce11e0000>; Tue, 24 Nov 2020 02:31:58 -0800
Received: from [172.27.1.196] (10.124.1.5) by HQMAIL107.nvidia.com
 (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Tue, 24 Nov
 2020 10:31:49 +0000
Subject: Re: [PATCH rdma-core 1/5] verbs: Support dma-buf based memory region
To:     Jianxin Xiong <jianxin.xiong@intel.com>
CC:     <linux-rdma@vger.kernel.org>, <dri-devel@lists.freedesktop.org>,
        "Doug Ledford" <dledford@redhat.com>,
        Jason Gunthorpe <jgg@ziepe.ca>,
        "Leon Romanovsky" <leon@kernel.org>,
        Sumit Semwal <sumit.semwal@linaro.org>,
        Christian Koenig <christian.koenig@amd.com>,
        Daniel Vetter <daniel.vetter@intel.com>,
        Yishai Hadas <yishaih@nvidia.com>
References: <1606153984-104583-1-git-send-email-jianxin.xiong@intel.com>
 <1606153984-104583-2-git-send-email-jianxin.xiong@intel.com>
From:   Yishai Hadas <yishaih@nvidia.com>
Message-ID: <fd567f71-4f10-5792-24b4-3f1860c19b20@nvidia.com>
Date:   Tue, 24 Nov 2020 12:31:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.1
MIME-Version: 1.0
In-Reply-To: <1606153984-104583-2-git-send-email-jianxin.xiong@intel.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Language: en-US
X-Originating-IP: [10.124.1.5]
X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To
 HQMAIL107.nvidia.com (172.20.187.13)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
        t=1606213918; bh=Nav52tbi4Xsp+SMqZok2bGwTkWgq7nnB8SgQLZhuBg0=;
        h=Subject:To:CC:References:From:Message-ID:Date:User-Agent:
         MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:
         Content-Language:X-Originating-IP:X-ClientProxiedBy;
        b=OkIBq2Z8TsR0Dcx38zER42g6C4gHO6ESMh0Za/FIwFzOdWNmnY9nZLm3inJ8Zn+o1
         zwDY2hJeGxrAgt8bZcN7TsMnnB/sc/hzxJG6shCgAqxsB13foH7tUQJXx0vDz4uX2o
         x89Rbo0QCQ5cWMzBigF65ZUXOzvybIWD8o5Wq9ZR3NrlkuBekPKYDi2ga0KF5qZQO3
         +QqcobLOsBP0ElQwS1ZgzGPhFjJIK0rKESTW2MgdUZMjFIghtcXI2S5MFEyLarEA+S
         f6bAy6JLye4rMNNKn/Asxc5448O8hcCC2wa17cOzRF2Uk3XSvGNPaHw/llBrWTxiRr
         /5tz5V34Yu2/A==
Precedence: bulk
List-ID: <linux-rdma.vger.kernel.org>
X-Mailing-List: linux-rdma@vger.kernel.org

On 11/23/2020 7:53 PM, Jianxin Xiong wrote:
> Add new API function and new provider method for registering dma-buf
> based memory region. Update the man page and bump the API version.
>
> Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
> ---
>   kernel-headers/rdma/ib_user_ioctl_cmds.h | 14 ++++++++++++
>   libibverbs/cmd_mr.c                      | 38 +++++++++++++++++++++++++=
+++++++
>   libibverbs/driver.h                      |  7 ++++++
>   libibverbs/dummy_ops.c                   | 11 +++++++++
>   libibverbs/libibverbs.map.in             |  6 +++++
>   libibverbs/man/ibv_reg_mr.3              | 21 ++++++++++++++++--
>   libibverbs/verbs.c                       | 19 ++++++++++++++++
>   libibverbs/verbs.h                       | 10 +++++++++
>   8 files changed, 124 insertions(+), 2 deletions(-)
>
> diff --git a/kernel-headers/rdma/ib_user_ioctl_cmds.h b/kernel-headers/rd=
ma/ib_user_ioctl_cmds.h
> index 7968a18..dafc7eb 100644
> --- a/kernel-headers/rdma/ib_user_ioctl_cmds.h
> +++ b/kernel-headers/rdma/ib_user_ioctl_cmds.h
> @@ -1,5 +1,6 @@
>   /*
>    * Copyright (c) 2018, Mellanox Technologies inc.  All rights reserved.
> + * Copyright (c) 2020, Intel Corporation. All rights reserved.
>    *
>    * This software is available to you under a choice of one of two
>    * licenses.  You may choose to be licensed under the terms of the GNU
> @@ -251,6 +252,7 @@ enum uverbs_methods_mr {
>   	UVERBS_METHOD_MR_DESTROY,
>   	UVERBS_METHOD_ADVISE_MR,
>   	UVERBS_METHOD_QUERY_MR,
> +	UVERBS_METHOD_REG_DMABUF_MR,
>   };
>  =20
>   enum uverbs_attrs_mr_destroy_ids {
> @@ -272,6 +274,18 @@ enum uverbs_attrs_query_mr_cmd_attr_ids {
>   	UVERBS_ATTR_QUERY_MR_RESP_IOVA,
>   };
>  =20
> +enum uverbs_attrs_reg_dmabuf_mr_cmd_attr_ids {
> +	UVERBS_ATTR_REG_DMABUF_MR_HANDLE,
> +	UVERBS_ATTR_REG_DMABUF_MR_PD_HANDLE,
> +	UVERBS_ATTR_REG_DMABUF_MR_OFFSET,
> +	UVERBS_ATTR_REG_DMABUF_MR_LENGTH,
> +	UVERBS_ATTR_REG_DMABUF_MR_IOVA,
> +	UVERBS_ATTR_REG_DMABUF_MR_FD,
> +	UVERBS_ATTR_REG_DMABUF_MR_ACCESS_FLAGS,
> +	UVERBS_ATTR_REG_DMABUF_MR_RESP_LKEY,
> +	UVERBS_ATTR_REG_DMABUF_MR_RESP_RKEY,
> +};
> +
>   enum uverbs_attrs_create_counters_cmd_attr_ids {
>   	UVERBS_ATTR_CREATE_COUNTERS_HANDLE,
>   };
> diff --git a/libibverbs/cmd_mr.c b/libibverbs/cmd_mr.c
> index 42dbe42..91ce2ef 100644
> --- a/libibverbs/cmd_mr.c
> +++ b/libibverbs/cmd_mr.c
> @@ -1,5 +1,6 @@
>   /*
>    * Copyright (c) 2018 Mellanox Technologies, Ltd.  All rights reserved.
> + * Copyright (c) 2020 Intel Corporation.  All rights reserved.
>    *
>    * This software is available to you under a choice of one of two
>    * licenses.  You may choose to be licensed under the terms of the GNU
> @@ -116,3 +117,40 @@ int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs=
_mr *vmr,
>   	return 0;
>   }
>  =20
> +int ibv_cmd_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, size_t len=
gth,
> +			  uint64_t iova, int fd, int access,
> +			  struct verbs_mr *vmr)
> +{
> +	DECLARE_COMMAND_BUFFER(cmdb, UVERBS_OBJECT_MR,
> +			       UVERBS_METHOD_REG_DMABUF_MR,
> +			       9);
> +	struct ib_uverbs_attr *handle;
> +	uint32_t lkey, rkey;
> +	int ret;
> +
> +	handle =3D fill_attr_out_obj(cmdb, UVERBS_ATTR_REG_DMABUF_MR_HANDLE);
> +	fill_attr_out_ptr(cmdb, UVERBS_ATTR_REG_DMABUF_MR_RESP_LKEY, &lkey);
> +	fill_attr_out_ptr(cmdb, UVERBS_ATTR_REG_DMABUF_MR_RESP_RKEY, &rkey);
> +
> +	fill_attr_in_obj(cmdb, UVERBS_ATTR_REG_DMABUF_MR_PD_HANDLE, pd->handle)=
;
> +	fill_attr_in_uint64(cmdb, UVERBS_ATTR_REG_DMABUF_MR_OFFSET, offset);
> +	fill_attr_in_uint64(cmdb, UVERBS_ATTR_REG_DMABUF_MR_LENGTH, length);
> +	fill_attr_in_uint64(cmdb, UVERBS_ATTR_REG_DMABUF_MR_IOVA, iova);
> +	fill_attr_in_uint32(cmdb, UVERBS_ATTR_REG_DMABUF_MR_FD, fd);
> +	fill_attr_in_uint32(cmdb, UVERBS_ATTR_REG_DMABUF_MR_ACCESS_FLAGS, acces=
s);
> +
> +	ret =3D execute_ioctl(pd->context, cmdb);
> +	if (ret)
> +		return errno;
> +
> +	vmr->ibv_mr.handle =3D
> +		read_attr_obj(UVERBS_ATTR_REG_DMABUF_MR_HANDLE, handle);
> +	vmr->ibv_mr.context =3D pd->context;
> +	vmr->ibv_mr.lkey    =3D lkey;
> +	vmr->ibv_mr.rkey    =3D rkey;
> +	vmr->ibv_mr.pd	    =3D pd;
> +	vmr->ibv_mr.addr    =3D (void *)offset;
> +	vmr->ibv_mr.length  =3D length;
> +	vmr->mr_type        =3D IBV_MR_TYPE_MR;
> +	return 0;
> +}
> diff --git a/libibverbs/driver.h b/libibverbs/driver.h
> index ab80f4b..d6a9d0a 100644
> --- a/libibverbs/driver.h
> +++ b/libibverbs/driver.h
> @@ -2,6 +2,7 @@
>    * Copyright (c) 2004, 2005 Topspin Communications.  All rights reserve=
d.
>    * Copyright (c) 2005, 2006 Cisco Systems, Inc.  All rights reserved.
>    * Copyright (c) 2005 PathScale, Inc.  All rights reserved.
> + * Copyright (c) 2020 Intel Corporation. All rights reserved.
>    *
>    * This software is available to you under a choice of one of two
>    * licenses.  You may choose to be licensed under the terms of the GNU
> @@ -373,6 +374,9 @@ struct verbs_context_ops {
>   	struct ibv_mr *(*reg_dm_mr)(struct ibv_pd *pd, struct ibv_dm *dm,
>   				    uint64_t dm_offset, size_t length,
>   				    unsigned int access);
> +	struct ibv_mr *(*reg_dmabuf_mr)(struct ibv_pd *pd, uint64_t offset,
> +					size_t length, uint64_t iova,
> +					int fd, int access);
>   	struct ibv_mr *(*reg_mr)(struct ibv_pd *pd, void *addr, size_t length,
>   				 uint64_t hca_va, int access);
>   	int (*req_notify_cq)(struct ibv_cq *cq, int solicited_only);
> @@ -498,6 +502,9 @@ int ibv_cmd_advise_mr(struct ibv_pd *pd,
>   		      uint32_t flags,
>   		      struct ibv_sge *sg_list,
>   		      uint32_t num_sge);
> +int ibv_cmd_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, size_t len=
gth,
> +			  uint64_t iova, int fd, int access,
> +			  struct verbs_mr *vmr);
>   int ibv_cmd_alloc_mw(struct ibv_pd *pd, enum ibv_mw_type type,
>   		     struct ibv_mw *mw, struct ibv_alloc_mw *cmd,
>   		     size_t cmd_size,
> diff --git a/libibverbs/dummy_ops.c b/libibverbs/dummy_ops.c
> index e5af9e4..dca96d2 100644
> --- a/libibverbs/dummy_ops.c
> +++ b/libibverbs/dummy_ops.c
> @@ -1,5 +1,6 @@
>   /*
>    * Copyright (c) 2017 Mellanox Technologies, Inc.  All rights reserved.
> + * Copyright (c) 2020 Intel Corporation.  All rights reserved.
>    *
>    * This software is available to you under a choice of one of two
>    * licenses.  You may choose to be licensed under the terms of the GNU
> @@ -452,6 +453,14 @@ static struct ibv_mr *reg_mr(struct ibv_pd *pd, void=
 *addr, size_t length,
>   	return NULL;
>   }
>  =20
> +static struct ibv_mr *reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset,
> +				    size_t length, uint64_t iova,
> +				    int fd, int access)
> +{
> +	errno =3D ENOSYS;
> +	return NULL;
> +}
> +
>   static int req_notify_cq(struct ibv_cq *cq, int solicited_only)
>   {
>   	return EOPNOTSUPP;
> @@ -560,6 +569,7 @@ const struct verbs_context_ops verbs_dummy_ops =3D {
>   	query_srq,
>   	read_counters,
>   	reg_dm_mr,
> +	reg_dmabuf_mr,
>   	reg_mr,
>   	req_notify_cq,
>   	rereg_mr,
> @@ -689,6 +699,7 @@ void verbs_set_ops(struct verbs_context *vctx,
>   	SET_PRIV_OP_IC(vctx, set_ece);
>   	SET_PRIV_OP_IC(vctx, unimport_mr);
>   	SET_PRIV_OP_IC(vctx, unimport_pd);
> +	SET_OP(vctx, reg_dmabuf_mr);
>  =20
>   #undef SET_OP
>   #undef SET_OP2
> diff --git a/libibverbs/libibverbs.map.in b/libibverbs/libibverbs.map.in
> index b5ccaca..f67e1ef 100644
> --- a/libibverbs/libibverbs.map.in
> +++ b/libibverbs/libibverbs.map.in
> @@ -148,6 +148,11 @@ IBVERBS_1.11 {
>   		_ibv_query_gid_table;
>   } IBVERBS_1.10;
>  =20
> +IBVERBS_1.12 {
> +	global:
> +		ibv_reg_dmabuf_mr;
> +} IBVERBS_1.11;
> +
>   /* If any symbols in this stanza change ABI then the entire staza gets =
a new symbol
>      version. See the top level CMakeLists.txt for this setting. */
>  =20
> @@ -211,6 +216,7 @@ IBVERBS_PRIVATE_@IBVERBS_PABI_VERSION@ {
>   		ibv_cmd_query_srq;
>   		ibv_cmd_read_counters;
>   		ibv_cmd_reg_dm_mr;
> +		ibv_cmd_reg_dmabuf_mr;
>   		ibv_cmd_reg_mr;
>   		ibv_cmd_req_notify_cq;
>   		ibv_cmd_rereg_mr;
> diff --git a/libibverbs/man/ibv_reg_mr.3 b/libibverbs/man/ibv_reg_mr.3
> index 2bfc955..4975c79 100644
> --- a/libibverbs/man/ibv_reg_mr.3
> +++ b/libibverbs/man/ibv_reg_mr.3
> @@ -3,7 +3,7 @@
>   .\"
>   .TH IBV_REG_MR 3 2006-10-31 libibverbs "Libibverbs Programmer's Manual"
>   .SH "NAME"
> -ibv_reg_mr, ibv_reg_mr_iova, ibv_dereg_mr \- register or deregister a me=
mory region (MR)
> +ibv_reg_mr, ibv_reg_mr_iova, ibv_reg_dmabuf_mr, ibv_dereg_mr \- register=
 or deregister a memory region (MR)
>   .SH "SYNOPSIS"
>   .nf
>   .B #include <infiniband/verbs.h>
> @@ -15,6 +15,9 @@ ibv_reg_mr, ibv_reg_mr_iova, ibv_dereg_mr \- register o=
r deregister a memory reg
>   .BI "                               size_t " "length" ", uint64_t " "hc=
a_va" ,
>   .BI "                               int " "access" );
>   .sp
> +.BI "struct ibv_mr *ibv_reg_dmabuf_mr(struct ibv_pd " "*pd" ", uint64_t =
" "offset" ,
> +.BI "                                 size_t " "length" ", int " "access=
" );
> +.sp


This misses the 'fd' parameter.

>   .BI "int ibv_dereg_mr(struct ibv_mr " "*mr" );
>   .fi
>   .SH "DESCRIPTION"
> @@ -71,11 +74,25 @@ a lkey or rkey. The offset in the memory region is co=
mputed as 'addr +
>   (iova - hca_va)'. Specifying 0 for hca_va has the same effect as
>   IBV_ACCESS_ZERO_BASED.
>   .PP
> +.B ibv_reg_dmabuf_mr()
> +registers a dma-buf based memory region (MR) associated with the protect=
ion domain
> +.I pd\fR.
> +The MR starts at
> +.I offset
> +of the dma-buf and its size is
> +.I length\fR.
> +The dma-buf is identified by the file descriptor
> +.I fd\fR.
> +The argument
> +.I access
> +describes the desired memory protection attributes; it is similar to the=
 ibv_reg_mr case except that only the following flags are supported:
> +.B IBV_ACCESS_LOCAL_WRITE, IBV_ACCESS_REMOTE_WRITE, IBV_ACCESS_REMOTE_RE=
AD, IBV_ACCESS_REMOTE_ATOMIC, IBV_ACCESS_RELAXED_ORDERING.
> +.PP
>   .B ibv_dereg_mr()
>   deregisters the MR
>   .I mr\fR.
>   .SH "RETURN VALUE"
> -.B ibv_reg_mr() / ibv_reg_mr_iova()
> +.B ibv_reg_mr() / ibv_reg_mr_iova() / ibv_reg_dmabuf_mr()
>   returns a pointer to the registered MR, or NULL if the request fails.
>   The local key (\fBL_Key\fR) field
>   .B lkey
> diff --git a/libibverbs/verbs.c b/libibverbs/verbs.c
> index 2b0ede8..84ddac7 100644
> --- a/libibverbs/verbs.c
> +++ b/libibverbs/verbs.c
> @@ -1,6 +1,7 @@
>   /*
>    * Copyright (c) 2005 Topspin Communications.  All rights reserved.
>    * Copyright (c) 2006, 2007 Cisco Systems, Inc.  All rights reserved.
> + * Copyright (c) 2020 Intel Corperation.  All rights reserved.
>    *
>    * This software is available to you under a choice of one of two
>    * licenses.  You may choose to be licensed under the terms of the GNU
> @@ -367,6 +368,24 @@ void ibv_unimport_mr(struct ibv_mr *mr)
>   	get_ops(mr->context)->unimport_mr(mr);
>   }
>  =20
> +LATEST_SYMVER_FUNC(ibv_reg_dmabuf_mr, 1_12, "IBVERBS_1.12",
> +		   struct ibv_mr *,
> +		   struct ibv_pd *pd, uint64_t offset,
> +		   size_t length, int fd, int access)
> +{


 =C2=A0ibv_dereg_mr() -> ibv_dofork_range() doesn't seem to be applicable i=
n=20
this case, right ? if offset / addr won't be 0 it may be activated on=20
the address range.

Do we rely on applications to *not* turn on fork support in libibverbs ?=20
(e.g. call ibv_fork_init()), what if some other registrations may need=20
being fork safe ?

> +	struct ibv_mr *mr;
> +
> +	mr =3D get_ops(pd->context)->reg_dmabuf_mr(pd, offset, length, offset,
> +						 fd, access);
> +	if (mr) {
> +		mr->context =3D pd->context;
> +		mr->pd      =3D pd;
> +		mr->addr    =3D (void *)offset;
> +		mr->length  =3D length;
> +	}
> +	return mr;
> +}
> +
>   LATEST_SYMVER_FUNC(ibv_rereg_mr, 1_1, "IBVERBS_1.1",
>   		   int,
>   		   struct ibv_mr *mr, int flags,
> diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
> index ee57e05..3961b1e 100644
> --- a/libibverbs/verbs.h
> +++ b/libibverbs/verbs.h
> @@ -3,6 +3,7 @@
>    * Copyright (c) 2004, 2011-2012 Intel Corporation.  All rights reserve=
d.
>    * Copyright (c) 2005, 2006, 2007 Cisco Systems, Inc.  All rights reser=
ved.
>    * Copyright (c) 2005 PathScale, Inc.  All rights reserved.
> + * Copyright (c) 2020 Intel Corporation.  All rights reserved.
>    *
>    * This software is available to you under a choice of one of two
>    * licenses.  You may choose to be licensed under the terms of the GNU
> @@ -2133,6 +2134,9 @@ struct verbs_context {
>   	struct ibv_xrcd *	(*open_xrcd)(struct ibv_context *context,
>   					     struct ibv_xrcd_init_attr *xrcd_init_attr);
>   	int			(*close_xrcd)(struct ibv_xrcd *xrcd);
> +	struct ibv_mr *		(*reg_dmabuf_mr)(struct ibv_pd *pd, uint64_t offset,
> +						 size_t length, uint64_t iova,
> +						 int fd, int access);
>   	uint64_t _ABI_placeholder3;
>   	size_t   sz;			/* Must be immediately before struct ibv_context */
>   	struct ibv_context context;	/* Must be last field in the struct */
> @@ -2535,6 +2539,12 @@ __ibv_reg_mr_iova(struct ibv_pd *pd, void *addr, s=
ize_t length, uint64_t iova,
>   			  __builtin_constant_p(                                \
>   				  ((access) & IBV_ACCESS_OPTIONAL_RANGE) =3D=3D 0))
>  =20
> +/**
> + * ibv_reg_dmabuf_mr - Register a dambuf-based memory region
> + */
> +struct ibv_mr *ibv_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, siz=
e_t length,
> +				 int fd, int access);
> +
>   enum ibv_rereg_mr_err_code {
>   	/* Old MR is valid, invalid input */
>   	IBV_REREG_MR_ERR_INPUT =3D -1,



From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <SRS0=SROE=E7=lists.freedesktop.org=dri-devel-bounces@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
X-Spam-Level: 
X-Spam-Status: No, score=-15.1 required=3.0 tests=BAYES_00,DKIM_INVALID,
	DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH,
	MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,
	USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0
Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 3F6A4C56201
	for <dri-devel@archiver.kernel.org>; Wed, 25 Nov 2020 08:12:11 +0000 (UTC)
Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by mail.kernel.org (Postfix) with ESMTPS id B8C8520857
	for <dri-devel@archiver.kernel.org>; Wed, 25 Nov 2020 08:12:10 +0000 (UTC)
Authentication-Results: mail.kernel.org;
	dkim=fail reason="signature verification failed" (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="OkIBq2Z8"
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B8C8520857
Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com
Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org
Received: from gabe.freedesktop.org (localhost [127.0.0.1])
	by gabe.freedesktop.org (Postfix) with ESMTP id 9F37A6E844;
	Wed, 25 Nov 2020 08:11:53 +0000 (UTC)
Received: from hqnvemgate25.nvidia.com (hqnvemgate25.nvidia.com
 [216.228.121.64])
 by gabe.freedesktop.org (Postfix) with ESMTPS id 8472588FB2
 for <dri-devel@lists.freedesktop.org>; Tue, 24 Nov 2020 10:31:58 +0000 (UTC)
Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by
 hqnvemgate25.nvidia.com (using TLS: TLSv1.2, AES256-SHA)
 id <B5fbce11e0000>; Tue, 24 Nov 2020 02:31:58 -0800
Received: from [172.27.1.196] (10.124.1.5) by HQMAIL107.nvidia.com
 (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Tue, 24 Nov
 2020 10:31:49 +0000
Subject: Re: [PATCH rdma-core 1/5] verbs: Support dma-buf based memory region
To: Jianxin Xiong <jianxin.xiong@intel.com>
References: <1606153984-104583-1-git-send-email-jianxin.xiong@intel.com>
 <1606153984-104583-2-git-send-email-jianxin.xiong@intel.com>
From: Yishai Hadas <yishaih@nvidia.com>
Message-ID: <fd567f71-4f10-5792-24b4-3f1860c19b20@nvidia.com>
Date: Tue, 24 Nov 2020 12:31:46 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.1
MIME-Version: 1.0
In-Reply-To: <1606153984-104583-2-git-send-email-jianxin.xiong@intel.com>
Content-Language: en-US
X-Originating-IP: [10.124.1.5]
X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To
 HQMAIL107.nvidia.com (172.20.187.13)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
 t=1606213918; bh=Nav52tbi4Xsp+SMqZok2bGwTkWgq7nnB8SgQLZhuBg0=;
 h=Subject:To:CC:References:From:Message-ID:Date:User-Agent:
 MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:
 Content-Language:X-Originating-IP:X-ClientProxiedBy;
 b=OkIBq2Z8TsR0Dcx38zER42g6C4gHO6ESMh0Za/FIwFzOdWNmnY9nZLm3inJ8Zn+o1
 zwDY2hJeGxrAgt8bZcN7TsMnnB/sc/hzxJG6shCgAqxsB13foH7tUQJXx0vDz4uX2o
 x89Rbo0QCQ5cWMzBigF65ZUXOzvybIWD8o5Wq9ZR3NrlkuBekPKYDi2ga0KF5qZQO3
 +QqcobLOsBP0ElQwS1ZgzGPhFjJIK0rKESTW2MgdUZMjFIghtcXI2S5MFEyLarEA+S
 f6bAy6JLye4rMNNKn/Asxc5448O8hcCC2wa17cOzRF2Uk3XSvGNPaHw/llBrWTxiRr
 /5tz5V34Yu2/A==
X-Mailman-Approved-At: Wed, 25 Nov 2020 08:11:52 +0000
X-BeenThere: dri-devel@lists.freedesktop.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Direct Rendering Infrastructure - Development
 <dri-devel.lists.freedesktop.org>
List-Unsubscribe: <https://lists.freedesktop.org/mailman/options/dri-devel>,
 <mailto:dri-devel-request@lists.freedesktop.org?subject=unsubscribe>
List-Archive: <https://lists.freedesktop.org/archives/dri-devel>
List-Post: <mailto:dri-devel@lists.freedesktop.org>
List-Help: <mailto:dri-devel-request@lists.freedesktop.org?subject=help>
List-Subscribe: <https://lists.freedesktop.org/mailman/listinfo/dri-devel>,
 <mailto:dri-devel-request@lists.freedesktop.org?subject=subscribe>
Cc: Yishai Hadas <yishaih@nvidia.com>, Leon
 Romanovsky <leon@kernel.org>, linux-rdma@vger.kernel.org,
 dri-devel@lists.freedesktop.org, Christian Koenig <christian.koenig@amd.com>,
 Jason Gunthorpe <jgg@ziepe.ca>, Doug
 Ledford <dledford@redhat.com>, Daniel Vetter <daniel.vetter@intel.com>
Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Errors-To: dri-devel-bounces@lists.freedesktop.org
Sender: "dri-devel" <dri-devel-bounces@lists.freedesktop.org>

T24gMTEvMjMvMjAyMCA3OjUzIFBNLCBKaWFueGluIFhpb25nIHdyb3RlOgo+IEFkZCBuZXcgQVBJ
IGZ1bmN0aW9uIGFuZCBuZXcgcHJvdmlkZXIgbWV0aG9kIGZvciByZWdpc3RlcmluZyBkbWEtYnVm
Cj4gYmFzZWQgbWVtb3J5IHJlZ2lvbi4gVXBkYXRlIHRoZSBtYW4gcGFnZSBhbmQgYnVtcCB0aGUg
QVBJIHZlcnNpb24uCj4KPiBTaWduZWQtb2ZmLWJ5OiBKaWFueGluIFhpb25nIDxqaWFueGluLnhp
b25nQGludGVsLmNvbT4KPiAtLS0KPiAgIGtlcm5lbC1oZWFkZXJzL3JkbWEvaWJfdXNlcl9pb2N0
bF9jbWRzLmggfCAxNCArKysrKysrKysrKysKPiAgIGxpYmlidmVyYnMvY21kX21yLmMgICAgICAg
ICAgICAgICAgICAgICAgfCAzOCArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKwo+ICAg
bGliaWJ2ZXJicy9kcml2ZXIuaCAgICAgICAgICAgICAgICAgICAgICB8ICA3ICsrKysrKwo+ICAg
bGliaWJ2ZXJicy9kdW1teV9vcHMuYyAgICAgICAgICAgICAgICAgICB8IDExICsrKysrKysrKwo+
ICAgbGliaWJ2ZXJicy9saWJpYnZlcmJzLm1hcC5pbiAgICAgICAgICAgICB8ICA2ICsrKysrCj4g
ICBsaWJpYnZlcmJzL21hbi9pYnZfcmVnX21yLjMgICAgICAgICAgICAgIHwgMjEgKysrKysrKysr
KysrKysrKy0tCj4gICBsaWJpYnZlcmJzL3ZlcmJzLmMgICAgICAgICAgICAgICAgICAgICAgIHwg
MTkgKysrKysrKysrKysrKysrKwo+ICAgbGliaWJ2ZXJicy92ZXJicy5oICAgICAgICAgICAgICAg
ICAgICAgICB8IDEwICsrKysrKysrKwo+ICAgOCBmaWxlcyBjaGFuZ2VkLCAxMjQgaW5zZXJ0aW9u
cygrKSwgMiBkZWxldGlvbnMoLSkKPgo+IGRpZmYgLS1naXQgYS9rZXJuZWwtaGVhZGVycy9yZG1h
L2liX3VzZXJfaW9jdGxfY21kcy5oIGIva2VybmVsLWhlYWRlcnMvcmRtYS9pYl91c2VyX2lvY3Rs
X2NtZHMuaAo+IGluZGV4IDc5NjhhMTguLmRhZmM3ZWIgMTAwNjQ0Cj4gLS0tIGEva2VybmVsLWhl
YWRlcnMvcmRtYS9pYl91c2VyX2lvY3RsX2NtZHMuaAo+ICsrKyBiL2tlcm5lbC1oZWFkZXJzL3Jk
bWEvaWJfdXNlcl9pb2N0bF9jbWRzLmgKPiBAQCAtMSw1ICsxLDYgQEAKPiAgIC8qCj4gICAgKiBD
b3B5cmlnaHQgKGMpIDIwMTgsIE1lbGxhbm94IFRlY2hub2xvZ2llcyBpbmMuICBBbGwgcmlnaHRz
IHJlc2VydmVkLgo+ICsgKiBDb3B5cmlnaHQgKGMpIDIwMjAsIEludGVsIENvcnBvcmF0aW9uLiBB
bGwgcmlnaHRzIHJlc2VydmVkLgo+ICAgICoKPiAgICAqIFRoaXMgc29mdHdhcmUgaXMgYXZhaWxh
YmxlIHRvIHlvdSB1bmRlciBhIGNob2ljZSBvZiBvbmUgb2YgdHdvCj4gICAgKiBsaWNlbnNlcy4g
IFlvdSBtYXkgY2hvb3NlIHRvIGJlIGxpY2Vuc2VkIHVuZGVyIHRoZSB0ZXJtcyBvZiB0aGUgR05V
Cj4gQEAgLTI1MSw2ICsyNTIsNyBAQCBlbnVtIHV2ZXJic19tZXRob2RzX21yIHsKPiAgIAlVVkVS
QlNfTUVUSE9EX01SX0RFU1RST1ksCj4gICAJVVZFUkJTX01FVEhPRF9BRFZJU0VfTVIsCj4gICAJ
VVZFUkJTX01FVEhPRF9RVUVSWV9NUiwKPiArCVVWRVJCU19NRVRIT0RfUkVHX0RNQUJVRl9NUiwK
PiAgIH07Cj4gICAKPiAgIGVudW0gdXZlcmJzX2F0dHJzX21yX2Rlc3Ryb3lfaWRzIHsKPiBAQCAt
MjcyLDYgKzI3NCwxOCBAQCBlbnVtIHV2ZXJic19hdHRyc19xdWVyeV9tcl9jbWRfYXR0cl9pZHMg
ewo+ICAgCVVWRVJCU19BVFRSX1FVRVJZX01SX1JFU1BfSU9WQSwKPiAgIH07Cj4gICAKPiArZW51
bSB1dmVyYnNfYXR0cnNfcmVnX2RtYWJ1Zl9tcl9jbWRfYXR0cl9pZHMgewo+ICsJVVZFUkJTX0FU
VFJfUkVHX0RNQUJVRl9NUl9IQU5ETEUsCj4gKwlVVkVSQlNfQVRUUl9SRUdfRE1BQlVGX01SX1BE
X0hBTkRMRSwKPiArCVVWRVJCU19BVFRSX1JFR19ETUFCVUZfTVJfT0ZGU0VULAo+ICsJVVZFUkJT
X0FUVFJfUkVHX0RNQUJVRl9NUl9MRU5HVEgsCj4gKwlVVkVSQlNfQVRUUl9SRUdfRE1BQlVGX01S
X0lPVkEsCj4gKwlVVkVSQlNfQVRUUl9SRUdfRE1BQlVGX01SX0ZELAo+ICsJVVZFUkJTX0FUVFJf
UkVHX0RNQUJVRl9NUl9BQ0NFU1NfRkxBR1MsCj4gKwlVVkVSQlNfQVRUUl9SRUdfRE1BQlVGX01S
X1JFU1BfTEtFWSwKPiArCVVWRVJCU19BVFRSX1JFR19ETUFCVUZfTVJfUkVTUF9SS0VZLAo+ICt9
Owo+ICsKPiAgIGVudW0gdXZlcmJzX2F0dHJzX2NyZWF0ZV9jb3VudGVyc19jbWRfYXR0cl9pZHMg
ewo+ICAgCVVWRVJCU19BVFRSX0NSRUFURV9DT1VOVEVSU19IQU5ETEUsCj4gICB9Owo+IGRpZmYg
LS1naXQgYS9saWJpYnZlcmJzL2NtZF9tci5jIGIvbGliaWJ2ZXJicy9jbWRfbXIuYwo+IGluZGV4
IDQyZGJlNDIuLjkxY2UyZWYgMTAwNjQ0Cj4gLS0tIGEvbGliaWJ2ZXJicy9jbWRfbXIuYwo+ICsr
KyBiL2xpYmlidmVyYnMvY21kX21yLmMKPiBAQCAtMSw1ICsxLDYgQEAKPiAgIC8qCj4gICAgKiBD
b3B5cmlnaHQgKGMpIDIwMTggTWVsbGFub3ggVGVjaG5vbG9naWVzLCBMdGQuICBBbGwgcmlnaHRz
IHJlc2VydmVkLgo+ICsgKiBDb3B5cmlnaHQgKGMpIDIwMjAgSW50ZWwgQ29ycG9yYXRpb24uICBB
bGwgcmlnaHRzIHJlc2VydmVkLgo+ICAgICoKPiAgICAqIFRoaXMgc29mdHdhcmUgaXMgYXZhaWxh
YmxlIHRvIHlvdSB1bmRlciBhIGNob2ljZSBvZiBvbmUgb2YgdHdvCj4gICAgKiBsaWNlbnNlcy4g
IFlvdSBtYXkgY2hvb3NlIHRvIGJlIGxpY2Vuc2VkIHVuZGVyIHRoZSB0ZXJtcyBvZiB0aGUgR05V
Cj4gQEAgLTExNiwzICsxMTcsNDAgQEAgaW50IGlidl9jbWRfcXVlcnlfbXIoc3RydWN0IGlidl9w
ZCAqcGQsIHN0cnVjdCB2ZXJic19tciAqdm1yLAo+ICAgCXJldHVybiAwOwo+ICAgfQo+ICAgCj4g
K2ludCBpYnZfY21kX3JlZ19kbWFidWZfbXIoc3RydWN0IGlidl9wZCAqcGQsIHVpbnQ2NF90IG9m
ZnNldCwgc2l6ZV90IGxlbmd0aCwKPiArCQkJICB1aW50NjRfdCBpb3ZhLCBpbnQgZmQsIGludCBh
Y2Nlc3MsCj4gKwkJCSAgc3RydWN0IHZlcmJzX21yICp2bXIpCj4gK3sKPiArCURFQ0xBUkVfQ09N
TUFORF9CVUZGRVIoY21kYiwgVVZFUkJTX09CSkVDVF9NUiwKPiArCQkJICAgICAgIFVWRVJCU19N
RVRIT0RfUkVHX0RNQUJVRl9NUiwKPiArCQkJICAgICAgIDkpOwo+ICsJc3RydWN0IGliX3V2ZXJi
c19hdHRyICpoYW5kbGU7Cj4gKwl1aW50MzJfdCBsa2V5LCBya2V5Owo+ICsJaW50IHJldDsKPiAr
Cj4gKwloYW5kbGUgPSBmaWxsX2F0dHJfb3V0X29iaihjbWRiLCBVVkVSQlNfQVRUUl9SRUdfRE1B
QlVGX01SX0hBTkRMRSk7Cj4gKwlmaWxsX2F0dHJfb3V0X3B0cihjbWRiLCBVVkVSQlNfQVRUUl9S
RUdfRE1BQlVGX01SX1JFU1BfTEtFWSwgJmxrZXkpOwo+ICsJZmlsbF9hdHRyX291dF9wdHIoY21k
YiwgVVZFUkJTX0FUVFJfUkVHX0RNQUJVRl9NUl9SRVNQX1JLRVksICZya2V5KTsKPiArCj4gKwlm
aWxsX2F0dHJfaW5fb2JqKGNtZGIsIFVWRVJCU19BVFRSX1JFR19ETUFCVUZfTVJfUERfSEFORExF
LCBwZC0+aGFuZGxlKTsKPiArCWZpbGxfYXR0cl9pbl91aW50NjQoY21kYiwgVVZFUkJTX0FUVFJf
UkVHX0RNQUJVRl9NUl9PRkZTRVQsIG9mZnNldCk7Cj4gKwlmaWxsX2F0dHJfaW5fdWludDY0KGNt
ZGIsIFVWRVJCU19BVFRSX1JFR19ETUFCVUZfTVJfTEVOR1RILCBsZW5ndGgpOwo+ICsJZmlsbF9h
dHRyX2luX3VpbnQ2NChjbWRiLCBVVkVSQlNfQVRUUl9SRUdfRE1BQlVGX01SX0lPVkEsIGlvdmEp
Owo+ICsJZmlsbF9hdHRyX2luX3VpbnQzMihjbWRiLCBVVkVSQlNfQVRUUl9SRUdfRE1BQlVGX01S
X0ZELCBmZCk7Cj4gKwlmaWxsX2F0dHJfaW5fdWludDMyKGNtZGIsIFVWRVJCU19BVFRSX1JFR19E
TUFCVUZfTVJfQUNDRVNTX0ZMQUdTLCBhY2Nlc3MpOwo+ICsKPiArCXJldCA9IGV4ZWN1dGVfaW9j
dGwocGQtPmNvbnRleHQsIGNtZGIpOwo+ICsJaWYgKHJldCkKPiArCQlyZXR1cm4gZXJybm87Cj4g
Kwo+ICsJdm1yLT5pYnZfbXIuaGFuZGxlID0KPiArCQlyZWFkX2F0dHJfb2JqKFVWRVJCU19BVFRS
X1JFR19ETUFCVUZfTVJfSEFORExFLCBoYW5kbGUpOwo+ICsJdm1yLT5pYnZfbXIuY29udGV4dCA9
IHBkLT5jb250ZXh0Owo+ICsJdm1yLT5pYnZfbXIubGtleSAgICA9IGxrZXk7Cj4gKwl2bXItPmli
dl9tci5ya2V5ICAgID0gcmtleTsKPiArCXZtci0+aWJ2X21yLnBkCSAgICA9IHBkOwo+ICsJdm1y
LT5pYnZfbXIuYWRkciAgICA9ICh2b2lkICopb2Zmc2V0Owo+ICsJdm1yLT5pYnZfbXIubGVuZ3Ro
ICA9IGxlbmd0aDsKPiArCXZtci0+bXJfdHlwZSAgICAgICAgPSBJQlZfTVJfVFlQRV9NUjsKPiAr
CXJldHVybiAwOwo+ICt9Cj4gZGlmZiAtLWdpdCBhL2xpYmlidmVyYnMvZHJpdmVyLmggYi9saWJp
YnZlcmJzL2RyaXZlci5oCj4gaW5kZXggYWI4MGY0Yi4uZDZhOWQwYSAxMDA2NDQKPiAtLS0gYS9s
aWJpYnZlcmJzL2RyaXZlci5oCj4gKysrIGIvbGliaWJ2ZXJicy9kcml2ZXIuaAo+IEBAIC0yLDYg
KzIsNyBAQAo+ICAgICogQ29weXJpZ2h0IChjKSAyMDA0LCAyMDA1IFRvcHNwaW4gQ29tbXVuaWNh
dGlvbnMuICBBbGwgcmlnaHRzIHJlc2VydmVkLgo+ICAgICogQ29weXJpZ2h0IChjKSAyMDA1LCAy
MDA2IENpc2NvIFN5c3RlbXMsIEluYy4gIEFsbCByaWdodHMgcmVzZXJ2ZWQuCj4gICAgKiBDb3B5
cmlnaHQgKGMpIDIwMDUgUGF0aFNjYWxlLCBJbmMuICBBbGwgcmlnaHRzIHJlc2VydmVkLgo+ICsg
KiBDb3B5cmlnaHQgKGMpIDIwMjAgSW50ZWwgQ29ycG9yYXRpb24uIEFsbCByaWdodHMgcmVzZXJ2
ZWQuCj4gICAgKgo+ICAgICogVGhpcyBzb2Z0d2FyZSBpcyBhdmFpbGFibGUgdG8geW91IHVuZGVy
IGEgY2hvaWNlIG9mIG9uZSBvZiB0d28KPiAgICAqIGxpY2Vuc2VzLiAgWW91IG1heSBjaG9vc2Ug
dG8gYmUgbGljZW5zZWQgdW5kZXIgdGhlIHRlcm1zIG9mIHRoZSBHTlUKPiBAQCAtMzczLDYgKzM3
NCw5IEBAIHN0cnVjdCB2ZXJic19jb250ZXh0X29wcyB7Cj4gICAJc3RydWN0IGlidl9tciAqKCpy
ZWdfZG1fbXIpKHN0cnVjdCBpYnZfcGQgKnBkLCBzdHJ1Y3QgaWJ2X2RtICpkbSwKPiAgIAkJCQkg
ICAgdWludDY0X3QgZG1fb2Zmc2V0LCBzaXplX3QgbGVuZ3RoLAo+ICAgCQkJCSAgICB1bnNpZ25l
ZCBpbnQgYWNjZXNzKTsKPiArCXN0cnVjdCBpYnZfbXIgKigqcmVnX2RtYWJ1Zl9tcikoc3RydWN0
IGlidl9wZCAqcGQsIHVpbnQ2NF90IG9mZnNldCwKPiArCQkJCQlzaXplX3QgbGVuZ3RoLCB1aW50
NjRfdCBpb3ZhLAo+ICsJCQkJCWludCBmZCwgaW50IGFjY2Vzcyk7Cj4gICAJc3RydWN0IGlidl9t
ciAqKCpyZWdfbXIpKHN0cnVjdCBpYnZfcGQgKnBkLCB2b2lkICphZGRyLCBzaXplX3QgbGVuZ3Ro
LAo+ICAgCQkJCSB1aW50NjRfdCBoY2FfdmEsIGludCBhY2Nlc3MpOwo+ICAgCWludCAoKnJlcV9u
b3RpZnlfY3EpKHN0cnVjdCBpYnZfY3EgKmNxLCBpbnQgc29saWNpdGVkX29ubHkpOwo+IEBAIC00
OTgsNiArNTAyLDkgQEAgaW50IGlidl9jbWRfYWR2aXNlX21yKHN0cnVjdCBpYnZfcGQgKnBkLAo+
ICAgCQkgICAgICB1aW50MzJfdCBmbGFncywKPiAgIAkJICAgICAgc3RydWN0IGlidl9zZ2UgKnNn
X2xpc3QsCj4gICAJCSAgICAgIHVpbnQzMl90IG51bV9zZ2UpOwo+ICtpbnQgaWJ2X2NtZF9yZWdf
ZG1hYnVmX21yKHN0cnVjdCBpYnZfcGQgKnBkLCB1aW50NjRfdCBvZmZzZXQsIHNpemVfdCBsZW5n
dGgsCj4gKwkJCSAgdWludDY0X3QgaW92YSwgaW50IGZkLCBpbnQgYWNjZXNzLAo+ICsJCQkgIHN0
cnVjdCB2ZXJic19tciAqdm1yKTsKPiAgIGludCBpYnZfY21kX2FsbG9jX213KHN0cnVjdCBpYnZf
cGQgKnBkLCBlbnVtIGlidl9td190eXBlIHR5cGUsCj4gICAJCSAgICAgc3RydWN0IGlidl9tdyAq
bXcsIHN0cnVjdCBpYnZfYWxsb2NfbXcgKmNtZCwKPiAgIAkJICAgICBzaXplX3QgY21kX3NpemUs
Cj4gZGlmZiAtLWdpdCBhL2xpYmlidmVyYnMvZHVtbXlfb3BzLmMgYi9saWJpYnZlcmJzL2R1bW15
X29wcy5jCj4gaW5kZXggZTVhZjllNC4uZGNhOTZkMiAxMDA2NDQKPiAtLS0gYS9saWJpYnZlcmJz
L2R1bW15X29wcy5jCj4gKysrIGIvbGliaWJ2ZXJicy9kdW1teV9vcHMuYwo+IEBAIC0xLDUgKzEs
NiBAQAo+ICAgLyoKPiAgICAqIENvcHlyaWdodCAoYykgMjAxNyBNZWxsYW5veCBUZWNobm9sb2dp
ZXMsIEluYy4gIEFsbCByaWdodHMgcmVzZXJ2ZWQuCj4gKyAqIENvcHlyaWdodCAoYykgMjAyMCBJ
bnRlbCBDb3Jwb3JhdGlvbi4gIEFsbCByaWdodHMgcmVzZXJ2ZWQuCj4gICAgKgo+ICAgICogVGhp
cyBzb2Z0d2FyZSBpcyBhdmFpbGFibGUgdG8geW91IHVuZGVyIGEgY2hvaWNlIG9mIG9uZSBvZiB0
d28KPiAgICAqIGxpY2Vuc2VzLiAgWW91IG1heSBjaG9vc2UgdG8gYmUgbGljZW5zZWQgdW5kZXIg
dGhlIHRlcm1zIG9mIHRoZSBHTlUKPiBAQCAtNDUyLDYgKzQ1MywxNCBAQCBzdGF0aWMgc3RydWN0
IGlidl9tciAqcmVnX21yKHN0cnVjdCBpYnZfcGQgKnBkLCB2b2lkICphZGRyLCBzaXplX3QgbGVu
Z3RoLAo+ICAgCXJldHVybiBOVUxMOwo+ICAgfQo+ICAgCj4gK3N0YXRpYyBzdHJ1Y3QgaWJ2X21y
ICpyZWdfZG1hYnVmX21yKHN0cnVjdCBpYnZfcGQgKnBkLCB1aW50NjRfdCBvZmZzZXQsCj4gKwkJ
CQkgICAgc2l6ZV90IGxlbmd0aCwgdWludDY0X3QgaW92YSwKPiArCQkJCSAgICBpbnQgZmQsIGlu
dCBhY2Nlc3MpCj4gK3sKPiArCWVycm5vID0gRU5PU1lTOwo+ICsJcmV0dXJuIE5VTEw7Cj4gK30K
PiArCj4gICBzdGF0aWMgaW50IHJlcV9ub3RpZnlfY3Eoc3RydWN0IGlidl9jcSAqY3EsIGludCBz
b2xpY2l0ZWRfb25seSkKPiAgIHsKPiAgIAlyZXR1cm4gRU9QTk9UU1VQUDsKPiBAQCAtNTYwLDYg
KzU2OSw3IEBAIGNvbnN0IHN0cnVjdCB2ZXJic19jb250ZXh0X29wcyB2ZXJic19kdW1teV9vcHMg
PSB7Cj4gICAJcXVlcnlfc3JxLAo+ICAgCXJlYWRfY291bnRlcnMsCj4gICAJcmVnX2RtX21yLAo+
ICsJcmVnX2RtYWJ1Zl9tciwKPiAgIAlyZWdfbXIsCj4gICAJcmVxX25vdGlmeV9jcSwKPiAgIAly
ZXJlZ19tciwKPiBAQCAtNjg5LDYgKzY5OSw3IEBAIHZvaWQgdmVyYnNfc2V0X29wcyhzdHJ1Y3Qg
dmVyYnNfY29udGV4dCAqdmN0eCwKPiAgIAlTRVRfUFJJVl9PUF9JQyh2Y3R4LCBzZXRfZWNlKTsK
PiAgIAlTRVRfUFJJVl9PUF9JQyh2Y3R4LCB1bmltcG9ydF9tcik7Cj4gICAJU0VUX1BSSVZfT1Bf
SUModmN0eCwgdW5pbXBvcnRfcGQpOwo+ICsJU0VUX09QKHZjdHgsIHJlZ19kbWFidWZfbXIpOwo+
ICAgCj4gICAjdW5kZWYgU0VUX09QCj4gICAjdW5kZWYgU0VUX09QMgo+IGRpZmYgLS1naXQgYS9s
aWJpYnZlcmJzL2xpYmlidmVyYnMubWFwLmluIGIvbGliaWJ2ZXJicy9saWJpYnZlcmJzLm1hcC5p
bgo+IGluZGV4IGI1Y2NhY2EuLmY2N2UxZWYgMTAwNjQ0Cj4gLS0tIGEvbGliaWJ2ZXJicy9saWJp
YnZlcmJzLm1hcC5pbgo+ICsrKyBiL2xpYmlidmVyYnMvbGliaWJ2ZXJicy5tYXAuaW4KPiBAQCAt
MTQ4LDYgKzE0OCwxMSBAQCBJQlZFUkJTXzEuMTEgewo+ICAgCQlfaWJ2X3F1ZXJ5X2dpZF90YWJs
ZTsKPiAgIH0gSUJWRVJCU18xLjEwOwo+ICAgCj4gK0lCVkVSQlNfMS4xMiB7Cj4gKwlnbG9iYWw6
Cj4gKwkJaWJ2X3JlZ19kbWFidWZfbXI7Cj4gK30gSUJWRVJCU18xLjExOwo+ICsKPiAgIC8qIElm
IGFueSBzeW1ib2xzIGluIHRoaXMgc3RhbnphIGNoYW5nZSBBQkkgdGhlbiB0aGUgZW50aXJlIHN0
YXphIGdldHMgYSBuZXcgc3ltYm9sCj4gICAgICB2ZXJzaW9uLiBTZWUgdGhlIHRvcCBsZXZlbCBD
TWFrZUxpc3RzLnR4dCBmb3IgdGhpcyBzZXR0aW5nLiAqLwo+ICAgCj4gQEAgLTIxMSw2ICsyMTYs
NyBAQCBJQlZFUkJTX1BSSVZBVEVfQElCVkVSQlNfUEFCSV9WRVJTSU9OQCB7Cj4gICAJCWlidl9j
bWRfcXVlcnlfc3JxOwo+ICAgCQlpYnZfY21kX3JlYWRfY291bnRlcnM7Cj4gICAJCWlidl9jbWRf
cmVnX2RtX21yOwo+ICsJCWlidl9jbWRfcmVnX2RtYWJ1Zl9tcjsKPiAgIAkJaWJ2X2NtZF9yZWdf
bXI7Cj4gICAJCWlidl9jbWRfcmVxX25vdGlmeV9jcTsKPiAgIAkJaWJ2X2NtZF9yZXJlZ19tcjsK
PiBkaWZmIC0tZ2l0IGEvbGliaWJ2ZXJicy9tYW4vaWJ2X3JlZ19tci4zIGIvbGliaWJ2ZXJicy9t
YW4vaWJ2X3JlZ19tci4zCj4gaW5kZXggMmJmYzk1NS4uNDk3NWM3OSAxMDA2NDQKPiAtLS0gYS9s
aWJpYnZlcmJzL21hbi9pYnZfcmVnX21yLjMKPiArKysgYi9saWJpYnZlcmJzL21hbi9pYnZfcmVn
X21yLjMKPiBAQCAtMyw3ICszLDcgQEAKPiAgIC5cIgo+ICAgLlRIIElCVl9SRUdfTVIgMyAyMDA2
LTEwLTMxIGxpYmlidmVyYnMgIkxpYmlidmVyYnMgUHJvZ3JhbW1lcidzIE1hbnVhbCIKPiAgIC5T
SCAiTkFNRSIKPiAtaWJ2X3JlZ19tciwgaWJ2X3JlZ19tcl9pb3ZhLCBpYnZfZGVyZWdfbXIgXC0g
cmVnaXN0ZXIgb3IgZGVyZWdpc3RlciBhIG1lbW9yeSByZWdpb24gKE1SKQo+ICtpYnZfcmVnX21y
LCBpYnZfcmVnX21yX2lvdmEsIGlidl9yZWdfZG1hYnVmX21yLCBpYnZfZGVyZWdfbXIgXC0gcmVn
aXN0ZXIgb3IgZGVyZWdpc3RlciBhIG1lbW9yeSByZWdpb24gKE1SKQo+ICAgLlNIICJTWU5PUFNJ
UyIKPiAgIC5uZgo+ICAgLkIgI2luY2x1ZGUgPGluZmluaWJhbmQvdmVyYnMuaD4KPiBAQCAtMTUs
NiArMTUsOSBAQCBpYnZfcmVnX21yLCBpYnZfcmVnX21yX2lvdmEsIGlidl9kZXJlZ19tciBcLSBy
ZWdpc3RlciBvciBkZXJlZ2lzdGVyIGEgbWVtb3J5IHJlZwo+ICAgLkJJICIgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgc2l6ZV90ICIgImxlbmd0aCIgIiwgdWludDY0X3QgIiAiaGNhX3Zh
IiAsCj4gICAuQkkgIiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnQgIiAiYWNjZXNz
IiApOwo+ICAgLnNwCj4gKy5CSSAic3RydWN0IGlidl9tciAqaWJ2X3JlZ19kbWFidWZfbXIoc3Ry
dWN0IGlidl9wZCAiICIqcGQiICIsIHVpbnQ2NF90ICIgIm9mZnNldCIgLAo+ICsuQkkgIiAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNpemVfdCAiICJsZW5ndGgiICIsIGludCAiICJh
Y2Nlc3MiICk7Cj4gKy5zcAoKClRoaXMgbWlzc2VzIHRoZSAnZmQnIHBhcmFtZXRlci4KCj4gICAu
QkkgImludCBpYnZfZGVyZWdfbXIoc3RydWN0IGlidl9tciAiICIqbXIiICk7Cj4gICAuZmkKPiAg
IC5TSCAiREVTQ1JJUFRJT04iCj4gQEAgLTcxLDExICs3NCwyNSBAQCBhIGxrZXkgb3IgcmtleS4g
VGhlIG9mZnNldCBpbiB0aGUgbWVtb3J5IHJlZ2lvbiBpcyBjb21wdXRlZCBhcyAnYWRkciArCj4g
ICAoaW92YSAtIGhjYV92YSknLiBTcGVjaWZ5aW5nIDAgZm9yIGhjYV92YSBoYXMgdGhlIHNhbWUg
ZWZmZWN0IGFzCj4gICBJQlZfQUNDRVNTX1pFUk9fQkFTRUQuCj4gICAuUFAKPiArLkIgaWJ2X3Jl
Z19kbWFidWZfbXIoKQo+ICtyZWdpc3RlcnMgYSBkbWEtYnVmIGJhc2VkIG1lbW9yeSByZWdpb24g
KE1SKSBhc3NvY2lhdGVkIHdpdGggdGhlIHByb3RlY3Rpb24gZG9tYWluCj4gKy5JIHBkXGZSLgo+
ICtUaGUgTVIgc3RhcnRzIGF0Cj4gKy5JIG9mZnNldAo+ICtvZiB0aGUgZG1hLWJ1ZiBhbmQgaXRz
IHNpemUgaXMKPiArLkkgbGVuZ3RoXGZSLgo+ICtUaGUgZG1hLWJ1ZiBpcyBpZGVudGlmaWVkIGJ5
IHRoZSBmaWxlIGRlc2NyaXB0b3IKPiArLkkgZmRcZlIuCj4gK1RoZSBhcmd1bWVudAo+ICsuSSBh
Y2Nlc3MKPiArZGVzY3JpYmVzIHRoZSBkZXNpcmVkIG1lbW9yeSBwcm90ZWN0aW9uIGF0dHJpYnV0
ZXM7IGl0IGlzIHNpbWlsYXIgdG8gdGhlIGlidl9yZWdfbXIgY2FzZSBleGNlcHQgdGhhdCBvbmx5
IHRoZSBmb2xsb3dpbmcgZmxhZ3MgYXJlIHN1cHBvcnRlZDoKPiArLkIgSUJWX0FDQ0VTU19MT0NB
TF9XUklURSwgSUJWX0FDQ0VTU19SRU1PVEVfV1JJVEUsIElCVl9BQ0NFU1NfUkVNT1RFX1JFQUQs
IElCVl9BQ0NFU1NfUkVNT1RFX0FUT01JQywgSUJWX0FDQ0VTU19SRUxBWEVEX09SREVSSU5HLgo+
ICsuUFAKPiAgIC5CIGlidl9kZXJlZ19tcigpCj4gICBkZXJlZ2lzdGVycyB0aGUgTVIKPiAgIC5J
IG1yXGZSLgo+ICAgLlNIICJSRVRVUk4gVkFMVUUiCj4gLS5CIGlidl9yZWdfbXIoKSAvIGlidl9y
ZWdfbXJfaW92YSgpCj4gKy5CIGlidl9yZWdfbXIoKSAvIGlidl9yZWdfbXJfaW92YSgpIC8gaWJ2
X3JlZ19kbWFidWZfbXIoKQo+ICAgcmV0dXJucyBhIHBvaW50ZXIgdG8gdGhlIHJlZ2lzdGVyZWQg
TVIsIG9yIE5VTEwgaWYgdGhlIHJlcXVlc3QgZmFpbHMuCj4gICBUaGUgbG9jYWwga2V5IChcZkJM
X0tleVxmUikgZmllbGQKPiAgIC5CIGxrZXkKPiBkaWZmIC0tZ2l0IGEvbGliaWJ2ZXJicy92ZXJi
cy5jIGIvbGliaWJ2ZXJicy92ZXJicy5jCj4gaW5kZXggMmIwZWRlOC4uODRkZGFjNyAxMDA2NDQK
PiAtLS0gYS9saWJpYnZlcmJzL3ZlcmJzLmMKPiArKysgYi9saWJpYnZlcmJzL3ZlcmJzLmMKPiBA
QCAtMSw2ICsxLDcgQEAKPiAgIC8qCj4gICAgKiBDb3B5cmlnaHQgKGMpIDIwMDUgVG9wc3BpbiBD
b21tdW5pY2F0aW9ucy4gIEFsbCByaWdodHMgcmVzZXJ2ZWQuCj4gICAgKiBDb3B5cmlnaHQgKGMp
IDIwMDYsIDIwMDcgQ2lzY28gU3lzdGVtcywgSW5jLiAgQWxsIHJpZ2h0cyByZXNlcnZlZC4KPiAr
ICogQ29weXJpZ2h0IChjKSAyMDIwIEludGVsIENvcnBlcmF0aW9uLiAgQWxsIHJpZ2h0cyByZXNl
cnZlZC4KPiAgICAqCj4gICAgKiBUaGlzIHNvZnR3YXJlIGlzIGF2YWlsYWJsZSB0byB5b3UgdW5k
ZXIgYSBjaG9pY2Ugb2Ygb25lIG9mIHR3bwo+ICAgICogbGljZW5zZXMuICBZb3UgbWF5IGNob29z
ZSB0byBiZSBsaWNlbnNlZCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVQo+IEBAIC0zNjcsNiAr
MzY4LDI0IEBAIHZvaWQgaWJ2X3VuaW1wb3J0X21yKHN0cnVjdCBpYnZfbXIgKm1yKQo+ICAgCWdl
dF9vcHMobXItPmNvbnRleHQpLT51bmltcG9ydF9tcihtcik7Cj4gICB9Cj4gICAKPiArTEFURVNU
X1NZTVZFUl9GVU5DKGlidl9yZWdfZG1hYnVmX21yLCAxXzEyLCAiSUJWRVJCU18xLjEyIiwKPiAr
CQkgICBzdHJ1Y3QgaWJ2X21yICosCj4gKwkJICAgc3RydWN0IGlidl9wZCAqcGQsIHVpbnQ2NF90
IG9mZnNldCwKPiArCQkgICBzaXplX3QgbGVuZ3RoLCBpbnQgZmQsIGludCBhY2Nlc3MpCj4gK3sK
CgogwqBpYnZfZGVyZWdfbXIoKSAtPiBpYnZfZG9mb3JrX3JhbmdlKCkgZG9lc24ndCBzZWVtIHRv
IGJlIGFwcGxpY2FibGUgaW4gCnRoaXMgY2FzZSwgcmlnaHQgPyBpZiBvZmZzZXQgLyBhZGRyIHdv
bid0IGJlIDAgaXQgbWF5IGJlIGFjdGl2YXRlZCBvbiAKdGhlIGFkZHJlc3MgcmFuZ2UuCgpEbyB3
ZSByZWx5IG9uIGFwcGxpY2F0aW9ucyB0byAqbm90KiB0dXJuIG9uIGZvcmsgc3VwcG9ydCBpbiBs
aWJpYnZlcmJzID8gCihlLmcuIGNhbGwgaWJ2X2ZvcmtfaW5pdCgpKSwgd2hhdCBpZiBzb21lIG90
aGVyIHJlZ2lzdHJhdGlvbnMgbWF5IG5lZWQgCmJlaW5nIGZvcmsgc2FmZSA/Cgo+ICsJc3RydWN0
IGlidl9tciAqbXI7Cj4gKwo+ICsJbXIgPSBnZXRfb3BzKHBkLT5jb250ZXh0KS0+cmVnX2RtYWJ1
Zl9tcihwZCwgb2Zmc2V0LCBsZW5ndGgsIG9mZnNldCwKPiArCQkJCQkJIGZkLCBhY2Nlc3MpOwo+
ICsJaWYgKG1yKSB7Cj4gKwkJbXItPmNvbnRleHQgPSBwZC0+Y29udGV4dDsKPiArCQltci0+cGQg
ICAgICA9IHBkOwo+ICsJCW1yLT5hZGRyICAgID0gKHZvaWQgKilvZmZzZXQ7Cj4gKwkJbXItPmxl
bmd0aCAgPSBsZW5ndGg7Cj4gKwl9Cj4gKwlyZXR1cm4gbXI7Cj4gK30KPiArCj4gICBMQVRFU1Rf
U1lNVkVSX0ZVTkMoaWJ2X3JlcmVnX21yLCAxXzEsICJJQlZFUkJTXzEuMSIsCj4gICAJCSAgIGlu
dCwKPiAgIAkJICAgc3RydWN0IGlidl9tciAqbXIsIGludCBmbGFncywKPiBkaWZmIC0tZ2l0IGEv
bGliaWJ2ZXJicy92ZXJicy5oIGIvbGliaWJ2ZXJicy92ZXJicy5oCj4gaW5kZXggZWU1N2UwNS4u
Mzk2MWIxZSAxMDA2NDQKPiAtLS0gYS9saWJpYnZlcmJzL3ZlcmJzLmgKPiArKysgYi9saWJpYnZl
cmJzL3ZlcmJzLmgKPiBAQCAtMyw2ICszLDcgQEAKPiAgICAqIENvcHlyaWdodCAoYykgMjAwNCwg
MjAxMS0yMDEyIEludGVsIENvcnBvcmF0aW9uLiAgQWxsIHJpZ2h0cyByZXNlcnZlZC4KPiAgICAq
IENvcHlyaWdodCAoYykgMjAwNSwgMjAwNiwgMjAwNyBDaXNjbyBTeXN0ZW1zLCBJbmMuICBBbGwg
cmlnaHRzIHJlc2VydmVkLgo+ICAgICogQ29weXJpZ2h0IChjKSAyMDA1IFBhdGhTY2FsZSwgSW5j
LiAgQWxsIHJpZ2h0cyByZXNlcnZlZC4KPiArICogQ29weXJpZ2h0IChjKSAyMDIwIEludGVsIENv
cnBvcmF0aW9uLiAgQWxsIHJpZ2h0cyByZXNlcnZlZC4KPiAgICAqCj4gICAgKiBUaGlzIHNvZnR3
YXJlIGlzIGF2YWlsYWJsZSB0byB5b3UgdW5kZXIgYSBjaG9pY2Ugb2Ygb25lIG9mIHR3bwo+ICAg
ICogbGljZW5zZXMuICBZb3UgbWF5IGNob29zZSB0byBiZSBsaWNlbnNlZCB1bmRlciB0aGUgdGVy
bXMgb2YgdGhlIEdOVQo+IEBAIC0yMTMzLDYgKzIxMzQsOSBAQCBzdHJ1Y3QgdmVyYnNfY29udGV4
dCB7Cj4gICAJc3RydWN0IGlidl94cmNkICoJKCpvcGVuX3hyY2QpKHN0cnVjdCBpYnZfY29udGV4
dCAqY29udGV4dCwKPiAgIAkJCQkJICAgICBzdHJ1Y3QgaWJ2X3hyY2RfaW5pdF9hdHRyICp4cmNk
X2luaXRfYXR0cik7Cj4gICAJaW50CQkJKCpjbG9zZV94cmNkKShzdHJ1Y3QgaWJ2X3hyY2QgKnhy
Y2QpOwo+ICsJc3RydWN0IGlidl9tciAqCQkoKnJlZ19kbWFidWZfbXIpKHN0cnVjdCBpYnZfcGQg
KnBkLCB1aW50NjRfdCBvZmZzZXQsCj4gKwkJCQkJCSBzaXplX3QgbGVuZ3RoLCB1aW50NjRfdCBp
b3ZhLAo+ICsJCQkJCQkgaW50IGZkLCBpbnQgYWNjZXNzKTsKPiAgIAl1aW50NjRfdCBfQUJJX3Bs
YWNlaG9sZGVyMzsKPiAgIAlzaXplX3QgICBzejsJCQkvKiBNdXN0IGJlIGltbWVkaWF0ZWx5IGJl
Zm9yZSBzdHJ1Y3QgaWJ2X2NvbnRleHQgKi8KPiAgIAlzdHJ1Y3QgaWJ2X2NvbnRleHQgY29udGV4
dDsJLyogTXVzdCBiZSBsYXN0IGZpZWxkIGluIHRoZSBzdHJ1Y3QgKi8KPiBAQCAtMjUzNSw2ICsy
NTM5LDEyIEBAIF9faWJ2X3JlZ19tcl9pb3ZhKHN0cnVjdCBpYnZfcGQgKnBkLCB2b2lkICphZGRy
LCBzaXplX3QgbGVuZ3RoLCB1aW50NjRfdCBpb3ZhLAo+ICAgCQkJICBfX2J1aWx0aW5fY29uc3Rh
bnRfcCggICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFwKPiAgIAkJCQkgICgoYWNjZXNz
KSAmIElCVl9BQ0NFU1NfT1BUSU9OQUxfUkFOR0UpID09IDApKQo+ICAgCj4gKy8qKgo+ICsgKiBp
YnZfcmVnX2RtYWJ1Zl9tciAtIFJlZ2lzdGVyIGEgZGFtYnVmLWJhc2VkIG1lbW9yeSByZWdpb24K
PiArICovCj4gK3N0cnVjdCBpYnZfbXIgKmlidl9yZWdfZG1hYnVmX21yKHN0cnVjdCBpYnZfcGQg
KnBkLCB1aW50NjRfdCBvZmZzZXQsIHNpemVfdCBsZW5ndGgsCj4gKwkJCQkgaW50IGZkLCBpbnQg
YWNjZXNzKTsKPiArCj4gICBlbnVtIGlidl9yZXJlZ19tcl9lcnJfY29kZSB7Cj4gICAJLyogT2xk
IE1SIGlzIHZhbGlkLCBpbnZhbGlkIGlucHV0ICovCj4gICAJSUJWX1JFUkVHX01SX0VSUl9JTlBV
VCA9IC0xLAoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
CmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpo
dHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbAo=