Re: [PATCH v1 09/20] xprtrdma: Limit the number of rpcrdma_mws

[Tom Talpey <tom-CLs1Zie5N5HQT0dZR+AlfA@public.gmane.org>]

On 6/8/2016 1:53 PM, Chuck Lever wrote:
>
>> On Jun 8, 2016, at 1:40 PM, Jason Gunthorpe <jgunthorpe@obsidianresearch.com> wrote:
>>
>> On Wed, Jun 08, 2016 at 03:06:56PM +0000, Trond Myklebust wrote:
>>
>>>   That would be a mount parameter which would only apply to RDMA since
>>>   everything else uses dynamic slot allocation these days. Iād prefer to
>>>   avoid that.
>>
>> The only reason RDMA cannot allocate MRs on the fly is
>> performance.. Maybe a dynamic scheme is the right answer..
>
> Dynamically managing the MR free list should be straightforward:
>
> 1. Allocate a minimum number of MRs at transport creation time
>
> 2. If workload causes the transport to exhaust its supply of
>     MRs, kick a worker thread to create some more
>
> 3. Maybe add a shrinker that can trim the MR list when the
>     system is under memory pressure

SMB Direct in Windows Server 2012R2 does this, and it works
extremely well. We reduced MR demand by approximately a factor
of 50, since it's rare that many connections are active at
high queue depths.

Tom.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html