Re: [PATCH v1 09/20] xprtrdma: Limit the number of rpcrdma_mws

[Trond Myklebust <trondmy-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org>]

On 6/8/16, 10:54, "linux-nfs-owner@vger.kernel.org on behalf of Tom Talpey" <linux-nfs-owner@vger.kernel.org on behalf of tom@talpey.com> wrote:

>On 6/7/2016 6:01 PM, Jason Gunthorpe wrote:
>> On Tue, Jun 07, 2016 at 05:51:04PM -0400, Chuck Lever wrote:
>>
>>> There is a practical number of MRs that can be allocated
>>> per device, I thought. And each MR consumes some amount
>>> of host memory.
>>
>>> xprtrdma is happy to allocate thousands of MRs per QP/PD
>>> pair, but that isn't practical as you start adding more
>>> transports/connections.
>>
>> Yes, this is all sane and valid.
>>
>> Typically you'd target a concurrency limit per QP and allocate
>> everything (# MRs, sq,rq,cq depth, etc) in accordance with that goal,
>> I'd also suggest that target concurrency is probably a user tunable..
>
>This is already present - it's the RPC slot table depth. It would be
>great, IMO, to make this a mount parameter since it is transport- and
>also server-dependent, but currently I believe it is just a kernel
>RPC module parameter.

That would be a mount parameter which would only apply to RDMA since everything else uses dynamic slot allocation these days. I’d prefer to avoid that.

Trond