Re: How to use SMB Direct

[Xiaoli Feng <xifeng@redhat.com>]

Hello Hyeoncheol,

When use the cifsd you provide, are there any required for Samba server?
Now I have two machines that have Mellanox ConnectX-3. And install 5.5.0-rc1+.
If one machine install the cifsd. Then I can use the SMB Direct without the 
concern of Samba server. Is it right?

As I know, seems there are two projects for SMB Direct(kernel part). One is
the cifsd. The others is Metze's repo. Which one is planning to go into linux
tree or both are? 

https://github.com/cifsd-team/cifsd.git
https://git.samba.org/?p=metze/linux/smbdirect.git;a=shortlog;h=refs/heads/smbdirect-work-in-progress

Thanks.

----- Original Message -----
> From: "Hyeoncheol Lee" <hyc.lee@gmail.com>
> To: "Xiaoli Feng" <xifeng@redhat.com>
> Cc: "ronnie sahlberg" <ronniesahlberg@gmail.com>, "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> <linux-cifs@vger.kernel.org>, linux-cifsd-devel@lists.sourceforge.net
> Sent: Thursday, December 19, 2019 2:47:40 PM
> Subject: Re: How to use SMB Direct
> 
> Hello Xiaoli,
> 
> cifsd is an in-kernel SMB server which supports SMB Direct.
> If you want to test SMB Direct with cifs, you can use cifsd.
> 
> Currently we have tested SMB Direct between two old fashioned
> Mellanox ConnectX devices which are connected directly,
> and between two soft RoCE devices in kernel.
> 
> You can get cifsd's code from the following url, and
> If you have any questions, we will help you.
> 
> https://github.com/cifsd-team/cifsd.git
> 
> Thank you.
> 
> 
> 2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <xifeng@redhat.com>님이 작성:
> >
> > Thanks Ronnie for the information. Last question: Is it supported that
> > mount the windows file
> > server(has rdma hardware) with SMB Direct in linux client(has rdma
> > hardware)?
> >
> > ----- Original Message -----
> > > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > > To: "Xiaoli Feng" <xifeng@redhat.com>
> > > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> > > <linux-cifs@vger.kernel.org>
> > > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > > Subject: Re: How to use SMB Direct
> > >
> > > I don't think samba supports SMB Direct.
> > > Metze used to have a private repo with some experimental patches but I
> > > don't think it
> > > has landed in official samba yet.
> > >
> > > For a linux server, there is an experimental kernel based server
> > > called cifsd which should support SMBDirect
> > > but is incomplete in other areas. It is not part of linus tree yet
> > > (but soon I hope).
> > >
> > > Windows servers support SMBDirect but I am not aware of any soft-rdma
> > > support so you might be
> > > limited to just using real hw for any tests.
> > >
> > > Linux serverside SMBD support is not ready afaik.
> > >
> > > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > > >
> > > > Hello Tom,
> > > >
> > > > Then I try to use IP to mount, it show this error:
> > > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > port=5445
> > > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > port=445
> > > >
> > > > Client:
> > > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > > mount error(2): No such file or directory
> > > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > > >
> > > > Server:
> > > > $ ib addr |grep 172.31.0.250/
> > > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > > >
> > > > Samba version is 4.11.2. And firewalld is stopped.
> > > >
> > > > ----- Original Message -----
> > > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > > Subject: RE: How to use SMB Direct
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > > On
> > > > > > Behalf Of Xiaoli Feng
> > > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > > To: linux-cifs@vger.kernel.org
> > > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > > >
> > > > > > Hello guys,
> > > > > >
> > > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it
> > > > > > works
> > > > > > in
> > > > > > upstream.
> > > > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel.
> > > > > > The
> > > > > > smb.conf is:
> > > > > > [cifs]
> > > > > > path=/mnt/cifs
> > > > > > writeable=yes
> > > > > >
> > > > > > Then I try to mount the share on another rdma machine with
> > > > > > 5.5.0-rc1+
> > > > > > kernel.
> > > > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > > > >
> > > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > > rdma_resolve_addr()
> > > > > > completed -113"
> > > > >
> > > > > Errno 113 is "no route to host". Sounds like a network or address
> > > > > issue.
> > > > >
> > > > > Tom.
> > > > >
> > > > > > Does SMB Direct work fine in upstream?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > > > $ ibstat
> > > > > > CA 'mlx4_0'
> > > > > >     CA type: MT4099
> > > > > >     Number of ports: 2
> > > > > >     Firmware version: 2.42.5000
> > > > > >     Hardware version: 1
> > > > > >     Node GUID: 0xf4521403007be0e0
> > > > > >     System image GUID: 0xf4521403007be0e3
> > > > > >     Port 1:
> > > > > >             State: Active
> > > > > >             Physical state: LinkUp
> > > > > >             Rate: 56
> > > > > >             Base lid: 29
> > > > > >             LMC: 0
> > > > > >             SM lid: 1
> > > > > >             Capability mask: 0x0259486a
> > > > > >             Port GUID: 0xf4521403007be0e1
> > > > > >             Link layer: InfiniBand
> > > > > >     Port 2:
> > > > > >             State: Active
> > > > > >             Physical state: LinkUp
> > > > > >             Rate: 40
> > > > > >             Base lid: 44
> > > > > >             LMC: 1
> > > > > >             SM lid: 36
> > > > > >             Capability mask: 0x02594868
> > > > > >             Port GUID: 0xf4521403007be0e2
> > > > > >             Link layer: InfiniBand
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards!
> > > > > > XiaoLi Feng 冯小丽
> > > > > >
> > > > > > Red Hat Software (Beijing) Co.,Ltd
> > > > > > filesystem-qe Team
> > > > > > IRC:xifeng，#channel: fs-qe
> > > > > > Tel:+86-10-8388112
> > > > > > 9/F, Raycom
> > > > >
> > > > >
> > > >
> > >
> > >
> >
> 
>