Re: [PATCH WIP 28/43] IB/core: Introduce new fast registration API

[Sagi Grimberg <sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>]

>> 1. if register - call ib_map_mr_sg (which calls dma_map_sg)
>>     else do dma_map_sg
>> 2. if registered - call ib_dma_unmap_sg (which calles dma_unmap_sg)
>>     else do dma_unmap_sg
>
>  From what I've seen in the ULPs the flow control is generally such
> that the MR is 'consumed' even if it isn't used by a send.

Not really. if registration is not needed, an MR is not consumed. In
fact, in svcrdma the IB code path never uses those, and the iWARP code
path always use those for RDMA_READs and not RDMA_WRITEs. Also, isert
use those only when signature is enabled and registration is required.

>
> So lkey usage is simply split into things that absolutely don't need a
> MR, and things that maybe do. The maybe side can go ahead and always
> consume the MR resource, but optimize the implementation to a SG list
> to avoid a performance hit.
>
> Then the whole API becomes symmetric. The ULP says, 'here is a
> scatterlist list and a lkey MR, make me a ib_sg list' and the core
> either packes it as is into the sg, or it spins up the MR and packs
> that.

Always consuming an MR resource is an extra lock acquire given these
are always kept in a pool structure.

>> I'm thinking we should keep dma_map_sg out of ib_map_mr_sg, and leave
>> it to the ULP like it does today (at least in the first stage...)
>
> I'm fine with first stage, but we still really do need to figure how
> how to get better code sharing in our API here..
>
> Maybe we can do the rkey side right away until we can figure out how
> to harmonize the rkey sg/mr usage?

I'm fine with that. I agree we still need to do better.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html