Re: Kernel fast memory registration API proposal [RFC]

[Sagi Grimberg <sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>]

On 7/15/2015 5:32 PM, Chuck Lever wrote:
>
> On Jul 15, 2015, at 4:01 AM, Sagi Grimberg <sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>
>> On 7/14/2015 8:09 PM, Jason Gunthorpe wrote:
>>> On Tue, Jul 14, 2015 at 07:55:39PM +0300, Sagi Grimberg wrote:
>>>
>>>> But, if people think that it's better to have an API that does implicit
>>>> posting always without notification, and then silently consume error or
>>>> flush completions. I can try and look at it as well.
>>>
>>> Can we do FMR transparently if we bundle the post? If yes, I'd call
>>> that a winner..
>>
>> Doing FMR transparently is not possible as the unmap flow is scheduling.
>> Unlike NFS, iSER unmaps from a soft-IRQ context, SRP unmaps from
>> hard-IRQ context.
>
> The context in which RPC/RDMA performs FMR unmap mustn’t sleep.
> RPC/RDMA is in roughly the same situation as the other initiators.
>
>
>> Changing the context to thread context is not
>> acceptable. The best we can do is using FMR_POOLs transparently.
>> Other than polluting the API and its semantics I suspect people will
>> have other problems with it (leaving the MRs open).
>
> Count me in that group.
>
> I would rather not build a non-deterministic delay into the
> unmap interface. Using a pool or having map do an implicit
> unmap are both solutions I’d rather avoid.
>
> In both situations, MRs can be left mapped indefinitely if,
> say, the workload pauses.
>
>
>> I suggest to start with what I proposed. And in a later stage (if we
>> still think its needed) we can have a higher level API that hides the
>> post, something like:
>
>> rdma_reg_sg(struct ib_qp *qp,
>>             struct ib_mr *mr,
>>             struct scatterlist *sg,
>>             int sg_nents,
>>             u64 offset,
>>             u64 length,
>>             int access_flags)
>
> I still wonder what “length” means in the context of a scatterlist.

Would byte_count be a more explanatory name?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html