directing soft iWARP traffic through a secure tunnel

directing soft iWARP traffic through a secure tunnel

[Chuck Lever]

Hi-

This might sound crazy, but bear with me.

The NFS community is starting to hold virtual interoperability testing
events to replace our in-person events that are not feasible due to
pandemic-related travel restrictions. I'm told other communities have
started doing the same.

The virtual event is being held on a private network that is set up
using OpenVPN across a large geographical area. I attach my test
systems to the VPN to access test systems run by others at other
companies.

We'd like to continue to include NFS/RDMA testing at these events.
This means either RoCEv2 or iWARP, since obviously we can't create
an ad hoc wide-area InfiniBand infrastructure.

Because the VPN is operating over long distances, we've decided to
start with iWARP. However, we are stumbling when it comes to directing
the siw driver's traffic onto the tun0 device:

[root@oracle-100 ~]# rdma link add siw0 type siw netdev tun0
error: Invalid argument
[root@oracle-100 ~]#

Has anyone else tried to do this, and what was the approach? Or does
siw not yet have this capability?

Thanks in advance.


--
Chuck Lever

Re: directing soft iWARP traffic through a secure tunnel

[Bernard Metzler]

-----"Chuck Lever" <chuck.lever@oracle.com> wrote: -----

>To: "linux-rdma" <linux-rdma@vger.kernel.org>
>From: "Chuck Lever" <chuck.lever@oracle.com>
>Date: 02/11/2021 08:38PM
>Cc: "Benjamin Coddington" <bcodding@redhat.com>
>Subject: [EXTERNAL] directing soft iWARP traffic through a secure
>tunnel
>
>Hi-
>
>This might sound crazy, but bear with me.
>
>The NFS community is starting to hold virtual interoperability
>testing
>events to replace our in-person events that are not feasible due to
>pandemic-related travel restrictions. I'm told other communities have
>started doing the same.
>
>The virtual event is being held on a private network that is set up
>using OpenVPN across a large geographical area. I attach my test
>systems to the VPN to access test systems run by others at other
>companies.
>
>We'd like to continue to include NFS/RDMA testing at these events.
>This means either RoCEv2 or iWARP, since obviously we can't create
>an ad hoc wide-area InfiniBand infrastructure.
>
>Because the VPN is operating over long distances, we've decided to
>start with iWARP. However, we are stumbling when it comes to
>directing
>the siw driver's traffic onto the tun0 device:
>
>[root@oracle-100 ~]# rdma link add siw0 type siw netdev tun0
>error: Invalid argument
>[root@oracle-100 ~]#
>
>Has anyone else tried to do this, and what was the approach? Or does
>siw not yet have this capability?
>

Hi Chuck

right. Attaching siw is currently restricted to some physical
device types. This now appears a useless limitation, since
it prevents its usage in the given setup, where it would
be just useful...
Relaxing that limitation is a rather simple code change in siw
- but that would not help you asap?

In any case I'd be happy to help with a fix, but participants
would have to rebuild the siw module...probably no option?

Best,
Bernard.

Re: directing soft iWARP traffic through a secure tunnel

[Benjamin Coddington]

On 12 Feb 2021, at 7:37, Bernard Metzler wrote:

> -----"Chuck Lever" <chuck.lever@oracle.com> wrote: -----
>
>> To: "linux-rdma" <linux-rdma@vger.kernel.org>
>> From: "Chuck Lever" <chuck.lever@oracle.com>
>> Date: 02/11/2021 08:38PM
>> Cc: "Benjamin Coddington" <bcodding@redhat.com>
>> Subject: [EXTERNAL] directing soft iWARP traffic through a secure
>> tunnel
>>
>> Hi-
>>
>> This might sound crazy, but bear with me.
>>
>> The NFS community is starting to hold virtual interoperability
>> testing
>> events to replace our in-person events that are not feasible due to
>> pandemic-related travel restrictions. I'm told other communities have
>> started doing the same.
>>
>> The virtual event is being held on a private network that is set up
>> using OpenVPN across a large geographical area. I attach my test
>> systems to the VPN to access test systems run by others at other
>> companies.
>>
>> We'd like to continue to include NFS/RDMA testing at these events.
>> This means either RoCEv2 or iWARP, since obviously we can't create
>> an ad hoc wide-area InfiniBand infrastructure.
>>
>> Because the VPN is operating over long distances, we've decided to
>> start with iWARP. However, we are stumbling when it comes to
>> directing
>> the siw driver's traffic onto the tun0 device:
>>
>> [root@oracle-100 ~]# rdma link add siw0 type siw netdev tun0
>> error: Invalid argument
>> [root@oracle-100 ~]#
>>
>> Has anyone else tried to do this, and what was the approach? Or does
>> siw not yet have this capability?
>>
>
> Hi Chuck
>
> right. Attaching siw is currently restricted to some physical
> device types. This now appears a useless limitation, since
> it prevents its usage in the given setup, where it would
> be just useful...
> Relaxing that limitation is a rather simple code change in siw
> - but that would not help you asap?
>
> In any case I'd be happy to help with a fix, but participants
> would have to rebuild the siw module...probably no option?

We can rebuild it and test it for you.

Ben

Re: directing soft iWARP traffic through a secure tunnel

[Chuck Lever]

> On Feb 12, 2021, at 7:37 AM, Bernard Metzler <bmt@zurich.ibm.com> wrote:
> 
> -----"Chuck Lever" <chuck.lever@oracle.com> wrote: -----
> 
>> To: "linux-rdma" <linux-rdma@vger.kernel.org>
>> From: "Chuck Lever" <chuck.lever@oracle.com>
>> Date: 02/11/2021 08:38PM
>> Cc: "Benjamin Coddington" <bcodding@redhat.com>
>> Subject: [EXTERNAL] directing soft iWARP traffic through a secure
>> tunnel
>> 
>> Hi-
>> 
>> This might sound crazy, but bear with me.
>> 
>> The NFS community is starting to hold virtual interoperability
>> testing
>> events to replace our in-person events that are not feasible due to
>> pandemic-related travel restrictions. I'm told other communities have
>> started doing the same.
>> 
>> The virtual event is being held on a private network that is set up
>> using OpenVPN across a large geographical area. I attach my test
>> systems to the VPN to access test systems run by others at other
>> companies.
>> 
>> We'd like to continue to include NFS/RDMA testing at these events.
>> This means either RoCEv2 or iWARP, since obviously we can't create
>> an ad hoc wide-area InfiniBand infrastructure.
>> 
>> Because the VPN is operating over long distances, we've decided to
>> start with iWARP. However, we are stumbling when it comes to
>> directing
>> the siw driver's traffic onto the tun0 device:
>> 
>> [root@oracle-100 ~]# rdma link add siw0 type siw netdev tun0
>> error: Invalid argument
>> [root@oracle-100 ~]#
>> 
>> Has anyone else tried to do this, and what was the approach? Or does
>> siw not yet have this capability?
>> 
> 
> Hi Chuck
> 
> right. Attaching siw is currently restricted to some physical
> device types. This now appears a useless limitation, since
> it prevents its usage in the given setup, where it would
> be just useful...
> Relaxing that limitation is a rather simple code change in siw
> - but that would not help you asap?
> 
> In any case I'd be happy to help with a fix, but participants
> would have to rebuild the siw module...probably no option?

Participants bring code and build infrastructure. A patch now
would be great, and we can provide you with Tested-by: !


--
Chuck Lever

Re: Re: directing soft iWARP traffic through a secure tunnel

[Bernard Metzler]

-----"Chuck Lever" <chuck.lever@oracle.com> wrote: -----

>To: "Bernard Metzler" <bmt@zurich.ibm.com>
>From: "Chuck Lever" <chuck.lever@oracle.com>
>Date: 02/12/2021 02:04PM
>Cc: "linux-rdma" <linux-rdma@vger.kernel.org>, "Benjamin Coddington"
><bcodding@redhat.com>
>Subject: [EXTERNAL] Re: directing soft iWARP traffic through a secure
>tunnel
>
>
>> On Feb 12, 2021, at 7:37 AM, Bernard Metzler <bmt@zurich.ibm.com>
>wrote:
>> 
>> -----"Chuck Lever" <chuck.lever@oracle.com> wrote: -----
>> 
>>> To: "linux-rdma" <linux-rdma@vger.kernel.org>
>>> From: "Chuck Lever" <chuck.lever@oracle.com>
>>> Date: 02/11/2021 08:38PM
>>> Cc: "Benjamin Coddington" <bcodding@redhat.com>
>>> Subject: [EXTERNAL] directing soft iWARP traffic through a secure
>>> tunnel
>>> 
>>> Hi-
>>> 
>>> This might sound crazy, but bear with me.
>>> 
>>> The NFS community is starting to hold virtual interoperability
>>> testing
>>> events to replace our in-person events that are not feasible due
>to
>>> pandemic-related travel restrictions. I'm told other communities
>have
>>> started doing the same.
>>> 
>>> The virtual event is being held on a private network that is set
>up
>>> using OpenVPN across a large geographical area. I attach my test
>>> systems to the VPN to access test systems run by others at other
>>> companies.
>>> 
>>> We'd like to continue to include NFS/RDMA testing at these events.
>>> This means either RoCEv2 or iWARP, since obviously we can't create
>>> an ad hoc wide-area InfiniBand infrastructure.
>>> 
>>> Because the VPN is operating over long distances, we've decided to
>>> start with iWARP. However, we are stumbling when it comes to
>>> directing
>>> the siw driver's traffic onto the tun0 device:
>>> 
>>> [root@oracle-100 ~]# rdma link add siw0 type siw netdev tun0
>>> error: Invalid argument
>>> [root@oracle-100 ~]#
>>> 
>>> Has anyone else tried to do this, and what was the approach? Or
>does
>>> siw not yet have this capability?
>>> 
>> 
>> Hi Chuck
>> 
>> right. Attaching siw is currently restricted to some physical
>> device types. This now appears a useless limitation, since
>> it prevents its usage in the given setup, where it would
>> be just useful...
>> Relaxing that limitation is a rather simple code change in siw
>> - but that would not help you asap?
>> 
>> In any case I'd be happy to help with a fix, but participants
>> would have to rebuild the siw module...probably no option?
>
>Participants bring code and build infrastructure. A patch now
>would be great, and we can provide you with Tested-by: !
>
>
How does this tunnel device look like? What does
ifconfig or ip show? Probably NOARP and no
HW address?

Or should I better setup a VPN client to your network
to see what is needed?

Best,
Bernard.

Re: directing soft iWARP traffic through a secure tunnel

[Benjamin Coddington]

On 12 Feb 2021, at 8:19, Bernard Metzler wrote:

> How does this tunnel device look like? What does
> ifconfig or ip show? Probably NOARP and no
> HW address?

right:

# ip link show tun0
3: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc 
fq_codel state UNKNOWN mode DEFAULT group default qlen 500

# ifconfig tun0
tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1500
         inet6 fe80::6bbf:1def:b134:2eef  prefixlen 64  scopeid 
0x20<link>
         inet6 fd51:5f56:d79b:a64e:64cc:5641:2916:7c4  prefixlen 64  
scopeid 0x0<global>
         unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
txqueuelen 500  (UNSPEC)
         RX packets 751  bytes 118096 (115.3 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 781  bytes 124142 (121.2 KiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

> Or should I better setup a VPN client to your network
> to see what is needed?

I can provide you with an openvpn config file that you can use to 
connect to
our network under separate cover if you'd like.

Ben

RE: directing soft iWARP traffic through a secure tunnel

[Bernard Metzler]

-----"Benjamin Coddington" <bcodding@redhat.com> wrote: -----

>To: "Bernard Metzler" <BMT@zurich.ibm.com>
>From: "Benjamin Coddington" <bcodding@redhat.com>
>Date: 02/12/2021 02:27PM
>Cc: "Chuck Lever" <chuck.lever@oracle.com>, "linux-rdma"
><linux-rdma@vger.kernel.org>
>Subject: [EXTERNAL] Re: directing soft iWARP traffic through a secure
>tunnel
>
>On 12 Feb 2021, at 8:19, Bernard Metzler wrote:
>
>> How does this tunnel device look like? What does
>> ifconfig or ip show? Probably NOARP and no
>> HW address?
>
>right:
>
># ip link show tun0
>3: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc 
>fq_codel state UNKNOWN mode DEFAULT group default qlen 500
>
># ifconfig tun0
>tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1500
>         inet6 fe80::6bbf:1def:b134:2eef  prefixlen 64  scopeid 
>0x20<link>
>         inet6 fd51:5f56:d79b:a64e:64cc:5641:2916:7c4  prefixlen 64  
>scopeid 0x0<global>
>         unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
>txqueuelen 500  (UNSPEC)
>         RX packets 751  bytes 118096 (115.3 KiB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 781  bytes 124142 (121.2 KiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
>> Or should I better setup a VPN client to your network
>> to see what is needed?
>
>I can provide you with an openvpn config file that you can use to 
>connect to
>our network under separate cover if you'd like.
>
great.
let's take this off the list and come back with a
proposed 3 lines patch. But, actually, I hope we
do not see problems with the iwcm regarding address resolution.

Best,
Bernard.

Re: directing soft iWARP traffic through a secure tunnel

[Bernard Metzler]

-----"Chuck Lever" <chuck.lever@oracle.com> wrote: -----

>To: "linux-rdma" <linux-rdma@vger.kernel.org>
>From: "Chuck Lever" <chuck.lever@oracle.com>
>Date: 02/11/2021 08:38PM
>Cc: "Benjamin Coddington" <bcodding@redhat.com>
>Subject: [EXTERNAL] directing soft iWARP traffic through a secure
>tunnel
>
>Hi-
>
>This might sound crazy, but bear with me.
>
>The NFS community is starting to hold virtual interoperability
>testing
>events to replace our in-person events that are not feasible due to
>pandemic-related travel restrictions. I'm told other communities have
>started doing the same.
>
>The virtual event is being held on a private network that is set up
>using OpenVPN across a large geographical area. I attach my test
>systems to the VPN to access test systems run by others at other
>companies.
>
>We'd like to continue to include NFS/RDMA testing at these events.
>This means either RoCEv2 or iWARP, since obviously we can't create
>an ad hoc wide-area InfiniBand infrastructure.
>
>Because the VPN is operating over long distances, we've decided to
>start with iWARP. However, we are stumbling when it comes to
>directing
>the siw driver's traffic onto the tun0 device:
>
>[root@oracle-100 ~]# rdma link add siw0 type siw netdev tun0
>error: Invalid argument
>[root@oracle-100 ~]#
>
>Has anyone else tried to do this, and what was the approach? Or does
>siw not yet have this capability?
>
>Thanks in advance.
>
>

After some very little ad-hoc patching of siw and RDMA core,
we get RDMA connectivity over the VPN. For siw, all what is
needed is to enable attaching to other netdev's than type
ARPHRD_ETHER, in particular allowing ARPHRD_NONE. It is
questionable, if software RDMA drivers need any such limitation,
if running on top of TCP or UDP.

The RDMA core currently does not accept a zero gid, which would
be the result of attaching to a tunnel w/o hardware address. I
tentatively disabled that code for tunnel devices (simply by
allowing a zero gid for device names starting with 'tun'), but
a serious solution would be different.

I think enabling software RDMA over TUN/TAP devices makes a lot
of sense, as exemplified by enabling this virtual NFS/RDMA test
event. If we agree on that, next step would be to come up with
the right way to do it. Would it make sense to start from the
rdma_link_layer enum, which already knows type
IB_LINK_LAYER_UNSPECIFIED ? RDMA drivers can report the link
type via device->ops.get_link_layer - see
rdma_port_get_link_layer(). Asking the RDMA core experts -
would a gid of zero have any side effects or bad implications,
since that ID is by no means unique?

Many thanks,
Bernard.

Re: directing soft iWARP traffic through a secure tunnel

[Jason Gunthorpe]

On Tue, Feb 16, 2021 at 12:27:33PM +0000, Bernard Metzler wrote:

> rdma_port_get_link_layer(). Asking the RDMA core experts -
> would a gid of zero have any side effects or bad implications,
> since that ID is by no means unique?

Yeah, it is clearly not ok. Generate a random private GUID?

Jason

Re: Re: directing soft iWARP traffic through a secure tunnel

[Bernard Metzler]

-----"Jason Gunthorpe" <jgg@ziepe.ca> wrote: -----

>To: "Bernard Metzler" <BMT@zurich.ibm.com>
>From: "Jason Gunthorpe" <jgg@ziepe.ca>
>Date: 02/16/2021 07:12PM
>Cc: "Chuck Lever" <chuck.lever@oracle.com>, "linux-rdma"
><linux-rdma@vger.kernel.org>, "Benjamin Coddington"
><bcodding@redhat.com>
>Subject: [EXTERNAL] Re: directing soft iWARP traffic through a secure
>tunnel
>
>On Tue, Feb 16, 2021 at 12:27:33PM +0000, Bernard Metzler wrote:
>
>> rdma_port_get_link_layer(). Asking the RDMA core experts -
>> would a gid of zero have any side effects or bad implications,
>> since that ID is by no means unique?
>
>Yeah, it is clearly not ok. Generate a random private GUID?
>
Right. But that's unfortunately not sufficient to get
devices w/o hardware addresses handled by the CM code.
The net_device remains w/o HW address, and that is where
CM gets the source HW address from (rdma_copy_src_l2_addr()
etc.). I don't have a good idea yet how to get it working.

Actually, all this GID and GUID and friends for iWARP
CM looks more like squeezing things into InfiniBand terms,
where we could just rely on plain ARP and IP 
(ARP resolve interface, see if there is an RDMA device
bound to, done)... or do I miss something?


On another related topic - the RDMA CM should let
the provider know if an IPv6 address is to be bound
AFONLY. This becomes needed in the current NFS/RDMA
over iWarp case, where NFS listeners bind to both IPv4
and IPv6 address with same port number. The code
correctly calls rdma_set_afonly() on the cmid, but
there is no mechanic to pass that information down
to the provider. So the second bind fails.
I'll send a small patch which enables that for
iwcm and siw, hoping for acceptance.

Many thanks,
Bernard.

Re: Re: directing soft iWARP traffic through a secure tunnel

[Jason Gunthorpe]

On Fri, Feb 19, 2021 at 01:06:26PM +0000, Bernard Metzler wrote:

> Actually, all this GID and GUID and friends for iWARP
> CM looks more like squeezing things into InfiniBand terms,
> where we could just rely on plain ARP and IP 
> (ARP resolve interface, see if there is an RDMA device
> bound to, done)... or do I miss something?

I don't know how iWarp cM works very well, it would not be surprising
if the gid table code has gained general rocee behaviors that are not
applicable to iwarp modes.

With Steve gone I don't think there is really anyone left that even
really knows how the iWarp stuff works??

Jason

Re: directing soft iWARP traffic through a secure tunnel

[Tom Talpey]

On 2/19/2021 8:57 AM, Jason Gunthorpe wrote:
> On Fri, Feb 19, 2021 at 01:06:26PM +0000, Bernard Metzler wrote:
> 
>> Actually, all this GID and GUID and friends for iWARP
>> CM looks more like squeezing things into InfiniBand terms,
>> where we could just rely on plain ARP and IP
>> (ARP resolve interface, see if there is an RDMA device
>> bound to, done)... or do I miss something?
> 
> I don't know how iWarp cM works very well, it would not be surprising
> if the gid table code has gained general rocee behaviors that are not
> applicable to iwarp modes.

iWarp doesn't really need a CM, it is capable of peer-to-peer without
any need to assign connection and queuepair ID's. The CM infrastructure
basically just implements a state machine to allow upper layers to have
a consistent connection API.

I'm with Bernard here, forcing iWarp to use CM is a fairly unnatural
act. Assigning a GID/GUID is unnecessary from a protocol perspective.

Tom.

> With Steve gone I don't think there is really anyone left that even
> really knows how the iWarp stuff works??
> 
> Jason
> 

Re: Re: directing soft iWARP traffic through a secure tunnel

[Bernard Metzler]

-----"Tom Talpey" <tom@talpey.com> wrote: -----

>To: "Jason Gunthorpe" <jgg@ziepe.ca>, "Bernard Metzler"
><BMT@zurich.ibm.com>
>From: "Tom Talpey" <tom@talpey.com>
>Date: 02/19/2021 03:15PM
>Cc: "Chuck Lever" <chuck.lever@oracle.com>, "linux-rdma"
><linux-rdma@vger.kernel.org>, "Benjamin Coddington"
><bcodding@redhat.com>
>Subject: [EXTERNAL] Re: directing soft iWARP traffic through a secure
>tunnel
>
>On 2/19/2021 8:57 AM, Jason Gunthorpe wrote:
>> On Fri, Feb 19, 2021 at 01:06:26PM +0000, Bernard Metzler wrote:
>> 
>>> Actually, all this GID and GUID and friends for iWARP
>>> CM looks more like squeezing things into InfiniBand terms,
>>> where we could just rely on plain ARP and IP
>>> (ARP resolve interface, see if there is an RDMA device
>>> bound to, done)... or do I miss something?
>> 
>> I don't know how iWarp cM works very well, it would not be
>surprising
>> if the gid table code has gained general rocee behaviors that are
>not
>> applicable to iwarp modes.
>
>iWarp doesn't really need a CM, it is capable of peer-to-peer without
>any need to assign connection and queuepair ID's. The CM
>infrastructure
>basically just implements a state machine to allow upper layers to
>have
>a consistent connection API.

Well hardware iWarp need someone to organize taking away ports
from kernel TCP which are bound to RNIC's.

>
>I'm with Bernard here, forcing iWarp to use CM is a fairly unnatural
>act. Assigning a GID/GUID is unnecessary from a protocol perspective.
>
>
>> With Steve gone I don't think there is really anyone left that even
>> really knows how the iWarp stuff works??
>> 

Cleaning up the iWarp path of it might be a complex undertaking.
I don't think going down that path solves the issue soon enough
for NFS/RDMA folks. But I will spend some time trying to wrap
my head around it...

Best,
Bernard.

Re: directing soft iWARP traffic through a secure tunnel

[Chuck Lever]

> On Feb 19, 2021, at 9:34 AM, Bernard Metzler <BMT@zurich.ibm.com> wrote:
> 
> -----"Tom Talpey" <tom@talpey.com> wrote: -----
> 
>> To: "Jason Gunthorpe" <jgg@ziepe.ca>, "Bernard Metzler"
>> <BMT@zurich.ibm.com>
>> From: "Tom Talpey" <tom@talpey.com>
>> Date: 02/19/2021 03:15PM
>> Cc: "Chuck Lever" <chuck.lever@oracle.com>, "linux-rdma"
>> <linux-rdma@vger.kernel.org>, "Benjamin Coddington"
>> <bcodding@redhat.com>
>> Subject: [EXTERNAL] Re: directing soft iWARP traffic through a secure
>> tunnel
>> 
>> On 2/19/2021 8:57 AM, Jason Gunthorpe wrote:
>>> On Fri, Feb 19, 2021 at 01:06:26PM +0000, Bernard Metzler wrote:
>>> 
>>>> Actually, all this GID and GUID and friends for iWARP
>>>> CM looks more like squeezing things into InfiniBand terms,
>>>> where we could just rely on plain ARP and IP
>>>> (ARP resolve interface, see if there is an RDMA device
>>>> bound to, done)... or do I miss something?
>>> 
>>> I don't know how iWarp cM works very well, it would not be
>> surprising
>>> if the gid table code has gained general rocee behaviors that are
>> not
>>> applicable to iwarp modes.
>> 
>> iWarp doesn't really need a CM, it is capable of peer-to-peer without
>> any need to assign connection and queuepair ID's. The CM
>> infrastructure
>> basically just implements a state machine to allow upper layers to
>> have
>> a consistent connection API.
> 
> Well hardware iWarp need someone to organize taking away ports
> from kernel TCP which are bound to RNIC's.
> 
>> 
>> I'm with Bernard here, forcing iWarp to use CM is a fairly unnatural
>> act. Assigning a GID/GUID is unnecessary from a protocol perspective.
>> 
>> 
>>> With Steve gone I don't think there is really anyone left that even
>>> really knows how the iWarp stuff works??
>>> 
> 
> Cleaning up the iWarp path of it might be a complex undertaking.
> I don't think going down that path solves the issue soon enough
> for NFS/RDMA folks.

We have a temporary solution for the upcoming event. There will
be more such events in the future.


> But I will spend some time trying to wrap
> my head around it...
> 
> Best,
> Bernard.

--
Chuck Lever