Re: [PATCH rdma-next v3 0/4] Query GID table API

From: Jason Gunthorpe <jgg@nvidia.com>
To: Leon Romanovsky <leon@kernel.org>
Cc: Doug Ledford <dledford@redhat.com>,
	Leon Romanovsky <leonro@nvidia.com>,
	Ariel Elior <aelior@marvell.com>,
	Avihai Horon <avihaih@nvidia.com>, <linux-kernel@vger.kernel.org>,
	<linux-rdma@vger.kernel.org>,
	Michal Kalderon <mkalderon@marvell.com>
Subject: Re: [PATCH rdma-next v3 0/4] Query GID table API
Date: Thu, 1 Oct 2020 21:32:18 -0300	[thread overview]
Message-ID: <20201002003218.GB1286456@nvidia.com> (raw)
In-Reply-To: <20200923165015.2491894-1-leon@kernel.org>

On Wed, Sep 23, 2020 at 07:50:11PM +0300, Leon Romanovsky wrote:
> When an application is not using RDMA CM and if it is using multiple RDMA
> devices with one or more RoCE ports, finding the right GID table entry is
> a long process.
> 
> For example, with two RoCE dual-port devices in a system, when IP
> failover is used between two RoCE ports, searching a suitable GID
> entry for a given source IP, matching netdevice of given RoCEv1/v2 type
> requires iterating over all 4 ports * 256 entry GID table.
> 
> Even though the best first match GID table for given criteria is used,
> when the matching entry is on the 4th port, it requires reading
> 3 ports * 256 entries * 3 files (GID, netdev, type) = 2304 files.
> 
> The GID table needs to be referred on every QP creation during IP
> failover on other netdevice of an RDMA device.
> 
> In an alternative approach, a GID cache may be maintained and updated on
> GID change event was reported by the kernel. However, it comes with below
> two limitations:
> (a) Maintain a thread per application process instance to listen and update
>  the cache.
> (b) Without the thread, on cache miss event, query the GID table. Even in
>  this approach, if multiple processes are used, a GID cache needs to be
>  maintained on a per-process basis. With a large number of processes,
>  this method doesn't scale.
> 
> Hence, we introduce this series of patches, which introduces an API to
> query the complete GID tables of an RDMA device, that returns all valid
> GID table entries.
> 
> This is done through single ioctl, eliminating 2304 read, 2304 open and
> 2304 close system calls to just a total of 2 calls (one for each device).
> 
> While at it, we also introduce an API to query an individual GID entry
> over ioctl interface, which provides all GID attributes information.
> 
> Thanks
> 
> Avihai Horon (4):
>   RDMA/core: Change rdma_get_gid_attr returned error code
>   RDMA/core: Modify enum ib_gid_type and enum rdma_network_type
>   RDMA/core: Introduce new GID table query API
>   RDMA/uverbs: Expose the new GID query API to user space

I made the edit to fix the locking, please check it

Applied to for-next

Thanks,
Jason