On Jun 3, 2018, at 9:59 PM, Alexey Lyashkov <alexey.lyashkov@gmail.com> wrote:
> 
>> On Sun, Jun 03 2018, Dilger, Andreas wrote:
>> 
>>> LNet is originally based on a high-performance networking stack called
>>> Portals (v3, http://www.cs.sandia.gov/Portals/), with additions for LNet
>>> routing to allow cross-network bridging.
>>> 
>>> A critical part of LNet is that it is for RDMA and not packet-based
>>> messages.  Everything in Lustre is structured around RDMA.  Of course,
>>> RDMA is not possible with TCP
> 
> To be clear. Soft IB (aka Soft RoCE) driver is part of OFED stack from 4.8
> (or 4.9).  So RDMA API now is possible with TCP networks.

Well, strictly speaking RoCE still isn't possible with TCP networks.  RoCE v1
is an Ethernet layer protocol (not IP based), while RoCE v2 is UDP/IP based.

Cheers, Andreas