linux-cifs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* How to use SMB Direct
       [not found] <1327532317.1529923.1576509501382.JavaMail.zimbra@redhat.com>
@ 2019-12-16 15:30 ` Xiaoli Feng
  2019-12-16 17:11   ` Tom Talpey
  0 siblings, 1 reply; 14+ messages in thread
From: Xiaoli Feng @ 2019-12-16 15:30 UTC (permalink / raw)
  To: linux-cifs

Hello guys,

I'd like to test SMB Direct. But it's failed. I'm not sure if it works in upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yes

Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
   mount //$RDMA/cifs cifs -o user=root,password=$password,rdma

It's failed because of "CIFS VFS: smbd_create_id:614 rdma_resolve_addr() completed -113"
Does SMB Direct work fine in upstream?

Thanks.

$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
CONFIG_CIFS_SMB_DIRECT=y
$ ibstat
CA 'mlx4_0'
	CA type: MT4099
	Number of ports: 2
	Firmware version: 2.42.5000
	Hardware version: 1
	Node GUID: 0xf4521403007be0e0
	System image GUID: 0xf4521403007be0e3
	Port 1:
		State: Active
		Physical state: LinkUp
		Rate: 56
		Base lid: 29
		LMC: 0
		SM lid: 1
		Capability mask: 0x0259486a
		Port GUID: 0xf4521403007be0e1
		Link layer: InfiniBand
	Port 2:
		State: Active
		Physical state: LinkUp
		Rate: 40
		Base lid: 44
		LMC: 1
		SM lid: 36
		Capability mask: 0x02594868
		Port GUID: 0xf4521403007be0e2
		Link layer: InfiniBand


-- 
Best regards!
XiaoLi Feng 冯小丽

Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom


^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: How to use SMB Direct
  2019-12-16 15:30 ` How to use SMB Direct Xiaoli Feng
@ 2019-12-16 17:11   ` Tom Talpey
  2019-12-18  2:48     ` Xiaoli Feng
  0 siblings, 1 reply; 14+ messages in thread
From: Tom Talpey @ 2019-12-16 17:11 UTC (permalink / raw)
  To: Xiaoli Feng, linux-cifs

> -----Original Message-----
> From: linux-cifs-owner@vger.kernel.org <linux-cifs-owner@vger.kernel.org> On
> Behalf Of Xiaoli Feng
> Sent: Monday, December 16, 2019 10:30 AM
> To: linux-cifs@vger.kernel.org
> Subject: [EXTERNAL] How to use SMB Direct
> 
> Hello guys,
> 
> I'd like to test SMB Direct. But it's failed. I'm not sure if it works in upstream.
> I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
> smb.conf is:
> [cifs]
> path=/mnt/cifs
> writeable=yes
> 
> Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
>    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> 
> It's failed because of "CIFS VFS: smbd_create_id:614 rdma_resolve_addr()
> completed -113"

Errno 113 is "no route to host". Sounds like a network or address issue.

Tom.

> Does SMB Direct work fine in upstream?
> 
> Thanks.
> 
> $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> CONFIG_CIFS_SMB_DIRECT=y
> $ ibstat
> CA 'mlx4_0'
> 	CA type: MT4099
> 	Number of ports: 2
> 	Firmware version: 2.42.5000
> 	Hardware version: 1
> 	Node GUID: 0xf4521403007be0e0
> 	System image GUID: 0xf4521403007be0e3
> 	Port 1:
> 		State: Active
> 		Physical state: LinkUp
> 		Rate: 56
> 		Base lid: 29
> 		LMC: 0
> 		SM lid: 1
> 		Capability mask: 0x0259486a
> 		Port GUID: 0xf4521403007be0e1
> 		Link layer: InfiniBand
> 	Port 2:
> 		State: Active
> 		Physical state: LinkUp
> 		Rate: 40
> 		Base lid: 44
> 		LMC: 1
> 		SM lid: 36
> 		Capability mask: 0x02594868
> 		Port GUID: 0xf4521403007be0e2
> 		Link layer: InfiniBand
> 
> 
> --
> Best regards!
> XiaoLi Feng 冯小丽
> 
> Red Hat Software (Beijing) Co.,Ltd
> filesystem-qe Team
> IRC:xifeng,#channel: fs-qe
> Tel:+86-10-8388112
> 9/F, Raycom


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-16 17:11   ` Tom Talpey
@ 2019-12-18  2:48     ` Xiaoli Feng
  2019-12-18  8:12       ` ronnie sahlberg
  0 siblings, 1 reply; 14+ messages in thread
From: Xiaoli Feng @ 2019-12-18  2:48 UTC (permalink / raw)
  To: Tom Talpey; +Cc: linux-cifs

Hello Tom,

Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445
[79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=445

Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
mount error(2): No such file or directory
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

Server:
$ ib addr |grep 172.31.0.250/
mlx4_ib0:		Link UP, Interface UP	172.31.0.250/24

Samba version is 4.11.2. And firewalld is stopped.

----- Original Message -----
> From: "Tom Talpey" <ttalpey@microsoft.com>
> To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> Sent: Tuesday, December 17, 2019 1:11:24 AM
> Subject: RE: How to use SMB Direct
> 
> > -----Original Message-----
> > From: linux-cifs-owner@vger.kernel.org <linux-cifs-owner@vger.kernel.org>
> > On
> > Behalf Of Xiaoli Feng
> > Sent: Monday, December 16, 2019 10:30 AM
> > To: linux-cifs@vger.kernel.org
> > Subject: [EXTERNAL] How to use SMB Direct
> > 
> > Hello guys,
> > 
> > I'd like to test SMB Direct. But it's failed. I'm not sure if it works in
> > upstream.
> > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
> > smb.conf is:
> > [cifs]
> > path=/mnt/cifs
> > writeable=yes
> > 
> > Then I try to mount the share on another rdma machine with 5.5.0-rc1+
> > kernel.
> >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > 
> > It's failed because of "CIFS VFS: smbd_create_id:614 rdma_resolve_addr()
> > completed -113"
> 
> Errno 113 is "no route to host". Sounds like a network or address issue.
> 
> Tom.
> 
> > Does SMB Direct work fine in upstream?
> > 
> > Thanks.
> > 
> > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > CONFIG_CIFS_SMB_DIRECT=y
> > $ ibstat
> > CA 'mlx4_0'
> > 	CA type: MT4099
> > 	Number of ports: 2
> > 	Firmware version: 2.42.5000
> > 	Hardware version: 1
> > 	Node GUID: 0xf4521403007be0e0
> > 	System image GUID: 0xf4521403007be0e3
> > 	Port 1:
> > 		State: Active
> > 		Physical state: LinkUp
> > 		Rate: 56
> > 		Base lid: 29
> > 		LMC: 0
> > 		SM lid: 1
> > 		Capability mask: 0x0259486a
> > 		Port GUID: 0xf4521403007be0e1
> > 		Link layer: InfiniBand
> > 	Port 2:
> > 		State: Active
> > 		Physical state: LinkUp
> > 		Rate: 40
> > 		Base lid: 44
> > 		LMC: 1
> > 		SM lid: 36
> > 		Capability mask: 0x02594868
> > 		Port GUID: 0xf4521403007be0e2
> > 		Link layer: InfiniBand
> > 
> > 
> > --
> > Best regards!
> > XiaoLi Feng 冯小丽
> > 
> > Red Hat Software (Beijing) Co.,Ltd
> > filesystem-qe Team
> > IRC:xifeng,#channel: fs-qe
> > Tel:+86-10-8388112
> > 9/F, Raycom
> 
> 


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-18  2:48     ` Xiaoli Feng
@ 2019-12-18  8:12       ` ronnie sahlberg
  2019-12-19  2:34         ` Xiaoli Feng
  0 siblings, 1 reply; 14+ messages in thread
From: ronnie sahlberg @ 2019-12-18  8:12 UTC (permalink / raw)
  To: Xiaoli Feng; +Cc: Tom Talpey, linux-cifs

I don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I
don't think it
has landed in official samba yet.

For a linux server, there is an experimental kernel based server
called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet
(but soon I hope).

Windows servers support SMBDirect but I am not aware of any soft-rdma
support so you might be
limited to just using real hw for any tests.

Linux serverside SMBD support is not ready afaik.

On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
>
> Hello Tom,
>
> Then I try to use IP to mount, it show this error:
> [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445
> [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=445
>
> Client:
> $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> mount error(2): No such file or directory
> Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
>
> Server:
> $ ib addr |grep 172.31.0.250/
> mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
>
> Samba version is 4.11.2. And firewalld is stopped.
>
> ----- Original Message -----
> > From: "Tom Talpey" <ttalpey@microsoft.com>
> > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > Subject: RE: How to use SMB Direct
> >
> > > -----Original Message-----
> > > From: linux-cifs-owner@vger.kernel.org <linux-cifs-owner@vger.kernel.org>
> > > On
> > > Behalf Of Xiaoli Feng
> > > Sent: Monday, December 16, 2019 10:30 AM
> > > To: linux-cifs@vger.kernel.org
> > > Subject: [EXTERNAL] How to use SMB Direct
> > >
> > > Hello guys,
> > >
> > > I'd like to test SMB Direct. But it's failed. I'm not sure if it works in
> > > upstream.
> > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
> > > smb.conf is:
> > > [cifs]
> > > path=/mnt/cifs
> > > writeable=yes
> > >
> > > Then I try to mount the share on another rdma machine with 5.5.0-rc1+
> > > kernel.
> > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > >
> > > It's failed because of "CIFS VFS: smbd_create_id:614 rdma_resolve_addr()
> > > completed -113"
> >
> > Errno 113 is "no route to host". Sounds like a network or address issue.
> >
> > Tom.
> >
> > > Does SMB Direct work fine in upstream?
> > >
> > > Thanks.
> > >
> > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > CONFIG_CIFS_SMB_DIRECT=y
> > > $ ibstat
> > > CA 'mlx4_0'
> > >     CA type: MT4099
> > >     Number of ports: 2
> > >     Firmware version: 2.42.5000
> > >     Hardware version: 1
> > >     Node GUID: 0xf4521403007be0e0
> > >     System image GUID: 0xf4521403007be0e3
> > >     Port 1:
> > >             State: Active
> > >             Physical state: LinkUp
> > >             Rate: 56
> > >             Base lid: 29
> > >             LMC: 0
> > >             SM lid: 1
> > >             Capability mask: 0x0259486a
> > >             Port GUID: 0xf4521403007be0e1
> > >             Link layer: InfiniBand
> > >     Port 2:
> > >             State: Active
> > >             Physical state: LinkUp
> > >             Rate: 40
> > >             Base lid: 44
> > >             LMC: 1
> > >             SM lid: 36
> > >             Capability mask: 0x02594868
> > >             Port GUID: 0xf4521403007be0e2
> > >             Link layer: InfiniBand
> > >
> > >
> > > --
> > > Best regards!
> > > XiaoLi Feng 冯小丽
> > >
> > > Red Hat Software (Beijing) Co.,Ltd
> > > filesystem-qe Team
> > > IRC:xifeng,#channel: fs-qe
> > > Tel:+86-10-8388112
> > > 9/F, Raycom
> >
> >
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-18  8:12       ` ronnie sahlberg
@ 2019-12-19  2:34         ` Xiaoli Feng
  2019-12-19  2:58           ` ronnie sahlberg
  2019-12-19  6:47           ` Hyeoncheol Lee
  0 siblings, 2 replies; 14+ messages in thread
From: Xiaoli Feng @ 2019-12-19  2:34 UTC (permalink / raw)
  To: ronnie sahlberg; +Cc: Tom Talpey, linux-cifs

Thanks Ronnie for the information. Last question: Is it supported that mount the windows file 
server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)? 

----- Original Message -----
> From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> To: "Xiaoli Feng" <xifeng@redhat.com>
> Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs" <linux-cifs@vger.kernel.org>
> Sent: Wednesday, December 18, 2019 4:12:32 PM
> Subject: Re: How to use SMB Direct
> 
> I don't think samba supports SMB Direct.
> Metze used to have a private repo with some experimental patches but I
> don't think it
> has landed in official samba yet.
> 
> For a linux server, there is an experimental kernel based server
> called cifsd which should support SMBDirect
> but is incomplete in other areas. It is not part of linus tree yet
> (but soon I hope).
> 
> Windows servers support SMBDirect but I am not aware of any soft-rdma
> support so you might be
> limited to just using real hw for any tests.
> 
> Linux serverside SMBD support is not ready afaik.
> 
> On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> >
> > Hello Tom,
> >
> > Then I try to use IP to mount, it show this error:
> > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > port=5445
> > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > port=445
> >
> > Client:
> > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > mount error(2): No such file or directory
> > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> >
> > Server:
> > $ ib addr |grep 172.31.0.250/
> > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> >
> > Samba version is 4.11.2. And firewalld is stopped.
> >
> > ----- Original Message -----
> > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > Subject: RE: How to use SMB Direct
> > >
> > > > -----Original Message-----
> > > > From: linux-cifs-owner@vger.kernel.org
> > > > <linux-cifs-owner@vger.kernel.org>
> > > > On
> > > > Behalf Of Xiaoli Feng
> > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > To: linux-cifs@vger.kernel.org
> > > > Subject: [EXTERNAL] How to use SMB Direct
> > > >
> > > > Hello guys,
> > > >
> > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it works
> > > > in
> > > > upstream.
> > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
> > > > smb.conf is:
> > > > [cifs]
> > > > path=/mnt/cifs
> > > > writeable=yes
> > > >
> > > > Then I try to mount the share on another rdma machine with 5.5.0-rc1+
> > > > kernel.
> > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > >
> > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > rdma_resolve_addr()
> > > > completed -113"
> > >
> > > Errno 113 is "no route to host". Sounds like a network or address issue.
> > >
> > > Tom.
> > >
> > > > Does SMB Direct work fine in upstream?
> > > >
> > > > Thanks.
> > > >
> > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > $ ibstat
> > > > CA 'mlx4_0'
> > > >     CA type: MT4099
> > > >     Number of ports: 2
> > > >     Firmware version: 2.42.5000
> > > >     Hardware version: 1
> > > >     Node GUID: 0xf4521403007be0e0
> > > >     System image GUID: 0xf4521403007be0e3
> > > >     Port 1:
> > > >             State: Active
> > > >             Physical state: LinkUp
> > > >             Rate: 56
> > > >             Base lid: 29
> > > >             LMC: 0
> > > >             SM lid: 1
> > > >             Capability mask: 0x0259486a
> > > >             Port GUID: 0xf4521403007be0e1
> > > >             Link layer: InfiniBand
> > > >     Port 2:
> > > >             State: Active
> > > >             Physical state: LinkUp
> > > >             Rate: 40
> > > >             Base lid: 44
> > > >             LMC: 1
> > > >             SM lid: 36
> > > >             Capability mask: 0x02594868
> > > >             Port GUID: 0xf4521403007be0e2
> > > >             Link layer: InfiniBand
> > > >
> > > >
> > > > --
> > > > Best regards!
> > > > XiaoLi Feng 冯小丽
> > > >
> > > > Red Hat Software (Beijing) Co.,Ltd
> > > > filesystem-qe Team
> > > > IRC:xifeng,#channel: fs-qe
> > > > Tel:+86-10-8388112
> > > > 9/F, Raycom
> > >
> > >
> >
> 
> 


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-19  2:34         ` Xiaoli Feng
@ 2019-12-19  2:58           ` ronnie sahlberg
  2019-12-19  6:47           ` Hyeoncheol Lee
  1 sibling, 0 replies; 14+ messages in thread
From: ronnie sahlberg @ 2019-12-19  2:58 UTC (permalink / raw)
  To: Xiaoli Feng; +Cc: Tom Talpey, linux-cifs

On Thu, Dec 19, 2019 at 12:34 PM Xiaoli Feng <xifeng@redhat.com> wrote:
>
> Thanks Ronnie for the information. Last question: Is it supported that mount the windows file
> server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?

Yeah, that should definitely work. I think Steve, or a colleague, has
run through the same set of the tests in our buildbot
(manually?) in that configuration.

If you can please test the work-in-progress cifsd kernel server.
It should supposedly work with soft-rdma and if it does then that
could be a way where we could
integrate SMBDirect testing in the buildbot.

I haven't had any time to research cifsd or set it up and test unfortunately :-(

regards
ronnie sahlberg

>
> ----- Original Message -----
> > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > To: "Xiaoli Feng" <xifeng@redhat.com>
> > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs" <linux-cifs@vger.kernel.org>
> > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > Subject: Re: How to use SMB Direct
> >
> > I don't think samba supports SMB Direct.
> > Metze used to have a private repo with some experimental patches but I
> > don't think it
> > has landed in official samba yet.
> >
> > For a linux server, there is an experimental kernel based server
> > called cifsd which should support SMBDirect
> > but is incomplete in other areas. It is not part of linus tree yet
> > (but soon I hope).
> >
> > Windows servers support SMBDirect but I am not aware of any soft-rdma
> > support so you might be
> > limited to just using real hw for any tests.
> >
> > Linux serverside SMBD support is not ready afaik.
> >
> > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > >
> > > Hello Tom,
> > >
> > > Then I try to use IP to mount, it show this error:
> > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > port=5445
> > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > port=445
> > >
> > > Client:
> > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > mount error(2): No such file or directory
> > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > >
> > > Server:
> > > $ ib addr |grep 172.31.0.250/
> > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > >
> > > Samba version is 4.11.2. And firewalld is stopped.
> > >
> > > ----- Original Message -----
> > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > Subject: RE: How to use SMB Direct
> > > >
> > > > > -----Original Message-----
> > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > On
> > > > > Behalf Of Xiaoli Feng
> > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > To: linux-cifs@vger.kernel.org
> > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > >
> > > > > Hello guys,
> > > > >
> > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it works
> > > > > in
> > > > > upstream.
> > > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
> > > > > smb.conf is:
> > > > > [cifs]
> > > > > path=/mnt/cifs
> > > > > writeable=yes
> > > > >
> > > > > Then I try to mount the share on another rdma machine with 5.5.0-rc1+
> > > > > kernel.
> > > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > > >
> > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > rdma_resolve_addr()
> > > > > completed -113"
> > > >
> > > > Errno 113 is "no route to host". Sounds like a network or address issue.
> > > >
> > > > Tom.
> > > >
> > > > > Does SMB Direct work fine in upstream?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > > $ ibstat
> > > > > CA 'mlx4_0'
> > > > >     CA type: MT4099
> > > > >     Number of ports: 2
> > > > >     Firmware version: 2.42.5000
> > > > >     Hardware version: 1
> > > > >     Node GUID: 0xf4521403007be0e0
> > > > >     System image GUID: 0xf4521403007be0e3
> > > > >     Port 1:
> > > > >             State: Active
> > > > >             Physical state: LinkUp
> > > > >             Rate: 56
> > > > >             Base lid: 29
> > > > >             LMC: 0
> > > > >             SM lid: 1
> > > > >             Capability mask: 0x0259486a
> > > > >             Port GUID: 0xf4521403007be0e1
> > > > >             Link layer: InfiniBand
> > > > >     Port 2:
> > > > >             State: Active
> > > > >             Physical state: LinkUp
> > > > >             Rate: 40
> > > > >             Base lid: 44
> > > > >             LMC: 1
> > > > >             SM lid: 36
> > > > >             Capability mask: 0x02594868
> > > > >             Port GUID: 0xf4521403007be0e2
> > > > >             Link layer: InfiniBand
> > > > >
> > > > >
> > > > > --
> > > > > Best regards!
> > > > > XiaoLi Feng 冯小丽
> > > > >
> > > > > Red Hat Software (Beijing) Co.,Ltd
> > > > > filesystem-qe Team
> > > > > IRC:xifeng,#channel: fs-qe
> > > > > Tel:+86-10-8388112
> > > > > 9/F, Raycom
> > > >
> > > >
> > >
> >
> >
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-19  2:34         ` Xiaoli Feng
  2019-12-19  2:58           ` ronnie sahlberg
@ 2019-12-19  6:47           ` Hyeoncheol Lee
  2019-12-19  6:56             ` ronnie sahlberg
                               ` (2 more replies)
  1 sibling, 3 replies; 14+ messages in thread
From: Hyeoncheol Lee @ 2019-12-19  6:47 UTC (permalink / raw)
  To: Xiaoli Feng; +Cc: ronnie sahlberg, Tom Talpey, linux-cifs, linux-cifsd-devel

Hello Xiaoli,

cifsd is an in-kernel SMB server which supports SMB Direct.
If you want to test SMB Direct with cifs, you can use cifsd.

Currently we have tested SMB Direct between two old fashioned
Mellanox ConnectX devices which are connected directly,
and between two soft RoCE devices in kernel.

You can get cifsd's code from the following url, and
If you have any questions, we will help you.

https://github.com/cifsd-team/cifsd.git

Thank you.


2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <xifeng@redhat.com>님이 작성:
>
> Thanks Ronnie for the information. Last question: Is it supported that mount the windows file
> server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?
>
> ----- Original Message -----
> > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > To: "Xiaoli Feng" <xifeng@redhat.com>
> > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs" <linux-cifs@vger.kernel.org>
> > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > Subject: Re: How to use SMB Direct
> >
> > I don't think samba supports SMB Direct.
> > Metze used to have a private repo with some experimental patches but I
> > don't think it
> > has landed in official samba yet.
> >
> > For a linux server, there is an experimental kernel based server
> > called cifsd which should support SMBDirect
> > but is incomplete in other areas. It is not part of linus tree yet
> > (but soon I hope).
> >
> > Windows servers support SMBDirect but I am not aware of any soft-rdma
> > support so you might be
> > limited to just using real hw for any tests.
> >
> > Linux serverside SMBD support is not ready afaik.
> >
> > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > >
> > > Hello Tom,
> > >
> > > Then I try to use IP to mount, it show this error:
> > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > port=5445
> > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > port=445
> > >
> > > Client:
> > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > mount error(2): No such file or directory
> > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > >
> > > Server:
> > > $ ib addr |grep 172.31.0.250/
> > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > >
> > > Samba version is 4.11.2. And firewalld is stopped.
> > >
> > > ----- Original Message -----
> > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > Subject: RE: How to use SMB Direct
> > > >
> > > > > -----Original Message-----
> > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > On
> > > > > Behalf Of Xiaoli Feng
> > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > To: linux-cifs@vger.kernel.org
> > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > >
> > > > > Hello guys,
> > > > >
> > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it works
> > > > > in
> > > > > upstream.
> > > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
> > > > > smb.conf is:
> > > > > [cifs]
> > > > > path=/mnt/cifs
> > > > > writeable=yes
> > > > >
> > > > > Then I try to mount the share on another rdma machine with 5.5.0-rc1+
> > > > > kernel.
> > > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > > >
> > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > rdma_resolve_addr()
> > > > > completed -113"
> > > >
> > > > Errno 113 is "no route to host". Sounds like a network or address issue.
> > > >
> > > > Tom.
> > > >
> > > > > Does SMB Direct work fine in upstream?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > > $ ibstat
> > > > > CA 'mlx4_0'
> > > > >     CA type: MT4099
> > > > >     Number of ports: 2
> > > > >     Firmware version: 2.42.5000
> > > > >     Hardware version: 1
> > > > >     Node GUID: 0xf4521403007be0e0
> > > > >     System image GUID: 0xf4521403007be0e3
> > > > >     Port 1:
> > > > >             State: Active
> > > > >             Physical state: LinkUp
> > > > >             Rate: 56
> > > > >             Base lid: 29
> > > > >             LMC: 0
> > > > >             SM lid: 1
> > > > >             Capability mask: 0x0259486a
> > > > >             Port GUID: 0xf4521403007be0e1
> > > > >             Link layer: InfiniBand
> > > > >     Port 2:
> > > > >             State: Active
> > > > >             Physical state: LinkUp
> > > > >             Rate: 40
> > > > >             Base lid: 44
> > > > >             LMC: 1
> > > > >             SM lid: 36
> > > > >             Capability mask: 0x02594868
> > > > >             Port GUID: 0xf4521403007be0e2
> > > > >             Link layer: InfiniBand
> > > > >
> > > > >
> > > > > --
> > > > > Best regards!
> > > > > XiaoLi Feng 冯小丽
> > > > >
> > > > > Red Hat Software (Beijing) Co.,Ltd
> > > > > filesystem-qe Team
> > > > > IRC:xifeng,#channel: fs-qe
> > > > > Tel:+86-10-8388112
> > > > > 9/F, Raycom
> > > >
> > > >
> > >
> >
> >
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-19  6:47           ` Hyeoncheol Lee
@ 2019-12-19  6:56             ` ronnie sahlberg
  2019-12-19  7:48               ` Hyunchul Lee
  2019-12-19  7:01             ` [Linux-cifsd-devel] " Namjae Jeon
  2019-12-20  2:59             ` Xiaoli Feng
  2 siblings, 1 reply; 14+ messages in thread
From: ronnie sahlberg @ 2019-12-19  6:56 UTC (permalink / raw)
  To: Hyeoncheol Lee; +Cc: Xiaoli Feng, Tom Talpey, linux-cifs, linux-cifsd-devel

On Thu, Dec 19, 2019 at 4:47 PM Hyeoncheol Lee <hyc.lee@gmail.com> wrote:
>
> Hello Xiaoli,
>
> cifsd is an in-kernel SMB server which supports SMB Direct.
> If you want to test SMB Direct with cifs, you can use cifsd.
>
> Currently we have tested SMB Direct between two old fashioned
> Mellanox ConnectX devices which are connected directly,
> and between two soft RoCE devices in kernel.

Thanks Hyeoncheol,

This is very interesting!

We have a buildbot upstream for the cifs client where we test every
set of patches before they go to linus :
http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/302

I havent had time to test with your cifsd yet but as you say you can
use soft-RoCE
you are saying it will work doing soft-RoCE between one VM running
cifs.ko and a basic virtio NIC
to another VM running your cifsd and also using a basic virtio NIC?

I would really want to set up so we can do RDMA tests for cifs.ko, and
by accident cifsd, in our buildbot.


regards
ronnie sahlberg


>
> You can get cifsd's code from the following url, and
> If you have any questions, we will help you.
>
> https://github.com/cifsd-team/cifsd.git
>
> Thank you.
>
>
> 2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <xifeng@redhat.com>님이 작성:
> >
> > Thanks Ronnie for the information. Last question: Is it supported that mount the windows file
> > server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?
> >
> > ----- Original Message -----
> > > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > > To: "Xiaoli Feng" <xifeng@redhat.com>
> > > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs" <linux-cifs@vger.kernel.org>
> > > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > > Subject: Re: How to use SMB Direct
> > >
> > > I don't think samba supports SMB Direct.
> > > Metze used to have a private repo with some experimental patches but I
> > > don't think it
> > > has landed in official samba yet.
> > >
> > > For a linux server, there is an experimental kernel based server
> > > called cifsd which should support SMBDirect
> > > but is incomplete in other areas. It is not part of linus tree yet
> > > (but soon I hope).
> > >
> > > Windows servers support SMBDirect but I am not aware of any soft-rdma
> > > support so you might be
> > > limited to just using real hw for any tests.
> > >
> > > Linux serverside SMBD support is not ready afaik.
> > >
> > > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > > >
> > > > Hello Tom,
> > > >
> > > > Then I try to use IP to mount, it show this error:
> > > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > port=5445
> > > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > port=445
> > > >
> > > > Client:
> > > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > > mount error(2): No such file or directory
> > > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > > >
> > > > Server:
> > > > $ ib addr |grep 172.31.0.250/
> > > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > > >
> > > > Samba version is 4.11.2. And firewalld is stopped.
> > > >
> > > > ----- Original Message -----
> > > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > > Subject: RE: How to use SMB Direct
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > > On
> > > > > > Behalf Of Xiaoli Feng
> > > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > > To: linux-cifs@vger.kernel.org
> > > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > > >
> > > > > > Hello guys,
> > > > > >
> > > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it works
> > > > > > in
> > > > > > upstream.
> > > > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
> > > > > > smb.conf is:
> > > > > > [cifs]
> > > > > > path=/mnt/cifs
> > > > > > writeable=yes
> > > > > >
> > > > > > Then I try to mount the share on another rdma machine with 5.5.0-rc1+
> > > > > > kernel.
> > > > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > > > >
> > > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > > rdma_resolve_addr()
> > > > > > completed -113"
> > > > >
> > > > > Errno 113 is "no route to host". Sounds like a network or address issue.
> > > > >
> > > > > Tom.
> > > > >
> > > > > > Does SMB Direct work fine in upstream?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > > > $ ibstat
> > > > > > CA 'mlx4_0'
> > > > > >     CA type: MT4099
> > > > > >     Number of ports: 2
> > > > > >     Firmware version: 2.42.5000
> > > > > >     Hardware version: 1
> > > > > >     Node GUID: 0xf4521403007be0e0
> > > > > >     System image GUID: 0xf4521403007be0e3
> > > > > >     Port 1:
> > > > > >             State: Active
> > > > > >             Physical state: LinkUp
> > > > > >             Rate: 56
> > > > > >             Base lid: 29
> > > > > >             LMC: 0
> > > > > >             SM lid: 1
> > > > > >             Capability mask: 0x0259486a
> > > > > >             Port GUID: 0xf4521403007be0e1
> > > > > >             Link layer: InfiniBand
> > > > > >     Port 2:
> > > > > >             State: Active
> > > > > >             Physical state: LinkUp
> > > > > >             Rate: 40
> > > > > >             Base lid: 44
> > > > > >             LMC: 1
> > > > > >             SM lid: 36
> > > > > >             Capability mask: 0x02594868
> > > > > >             Port GUID: 0xf4521403007be0e2
> > > > > >             Link layer: InfiniBand
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards!
> > > > > > XiaoLi Feng 冯小丽
> > > > > >
> > > > > > Red Hat Software (Beijing) Co.,Ltd
> > > > > > filesystem-qe Team
> > > > > > IRC:xifeng,#channel: fs-qe
> > > > > > Tel:+86-10-8388112
> > > > > > 9/F, Raycom
> > > > >
> > > > >
> > > >
> > >
> > >
> >

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [Linux-cifsd-devel] How to use SMB Direct
  2019-12-19  6:47           ` Hyeoncheol Lee
  2019-12-19  6:56             ` ronnie sahlberg
@ 2019-12-19  7:01             ` Namjae Jeon
  2019-12-20  2:59             ` Xiaoli Feng
  2 siblings, 0 replies; 14+ messages in thread
From: Namjae Jeon @ 2019-12-19  7:01 UTC (permalink / raw)
  To: 'Xiaoli Feng'
  Cc: 'linux-cifs', 'Tom Talpey',
	linux-cifsd-devel, 'ronnie sahlberg',
	'Hyeoncheol Lee'

Please read README file to build cifsd

And SMB direct disabled by default.
Enable smb direct support on config.

<M>   CIFS server support 
 [*]     Support for SMB Direct protocol 

> -----Original Message-----
> From: Hyeoncheol Lee <hyc.lee@gmail.com>
> Sent: Thursday, December 19, 2019 3:48 PM
> To: Xiaoli Feng <xifeng@redhat.com>
> Cc: linux-cifs <linux-cifs@vger.kernel.org>; Tom Talpey
> <ttalpey@microsoft.com>; linux-cifsd-devel@lists.sourceforge.net; ronnie
> sahlberg <ronniesahlberg@gmail.com>
> Subject: Re: [Linux-cifsd-devel] How to use SMB Direct
> 
> Hello Xiaoli,
> 
> cifsd is an in-kernel SMB server which supports SMB Direct.
> If you want to test SMB Direct with cifs, you can use cifsd.
> 
> Currently we have tested SMB Direct between two old fashioned Mellanox
> ConnectX devices which are connected directly, and between two soft RoCE
> devices in kernel.
> 
> You can get cifsd's code from the following url, and If you have any
> questions, we will help you.
> 
> https://protect2.fireeye.com/url?k=02be8777-5f6a3b1f-02bf0c38-
> 0cc47a3356b2-b295524a9bc31dd5&u=https://github.com/cifsd-team/cifsd.git
> 
> Thank you.
> 
> 
> 2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <xifeng@redhat.com>님이 작성:
> >
> > Thanks Ronnie for the information. Last question: Is it supported that
> > mount the windows file server(has rdma hardware) with SMB Direct in
> linux client(has rdma hardware)?
> >
> > ----- Original Message -----
> > > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > > To: "Xiaoli Feng" <xifeng@redhat.com>
> > > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> > > <linux-cifs@vger.kernel.org>
> > > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > > Subject: Re: How to use SMB Direct
> > >
> > > I don't think samba supports SMB Direct.
> > > Metze used to have a private repo with some experimental patches but
> > > I don't think it has landed in official samba yet.
> > >
> > > For a linux server, there is an experimental kernel based server
> > > called cifsd which should support SMBDirect but is incomplete in
> > > other areas. It is not part of linus tree yet (but soon I hope).
> > >
> > > Windows servers support SMBDirect but I am not aware of any
> > > soft-rdma support so you might be limited to just using real hw for
> > > any tests.
> > >
> > > Linux serverside SMBD support is not ready afaik.
> > >
> > > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > > >
> > > > Hello Tom,
> > > >
> > > > Then I try to use IP to mount, it show this error:
> > > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect
> > > > failed
> > > > port=5445
> > > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect
> > > > failed
> > > > port=445
> > > >
> > > > Client:
> > > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > > mount error(2): No such file or directory Refer to the
> > > > mount.cifs(8) manual page (e.g. man mount.cifs)
> > > >
> > > > Server:
> > > > $ ib addr |grep 172.31.0.250/
> > > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > > >
> > > > Samba version is 4.11.2. And firewalld is stopped.
> > > >
> > > > ----- Original Message -----
> > > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > > To: "Xiaoli Feng" <xifeng@redhat.com>,
> > > > > linux-cifs@vger.kernel.org
> > > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > > Subject: RE: How to use SMB Direct
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > > On
> > > > > > Behalf Of Xiaoli Feng
> > > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > > To: linux-cifs@vger.kernel.org
> > > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > > >
> > > > > > Hello guys,
> > > > > >
> > > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if
> > > > > > it works in upstream.
> > > > > > I setup samba server on one rdma machine with 5.5.0-rc1+
> > > > > > kernel. The smb.conf is:
> > > > > > [cifs]
> > > > > > path=/mnt/cifs
> > > > > > writeable=yes
> > > > > >
> > > > > > Then I try to mount the share on another rdma machine with
> > > > > > 5.5.0-rc1+ kernel.
> > > > > >    mount //$RDMA/cifs cifs -o
> > > > > > user=root,password=$password,rdma
> > > > > >
> > > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > > rdma_resolve_addr()
> > > > > > completed -113"
> > > > >
> > > > > Errno 113 is "no route to host". Sounds like a network or address
> issue.
> > > > >
> > > > > Tom.
> > > > >
> > > > > > Does SMB Direct work fine in upstream?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > > CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0'
> > > > > >     CA type: MT4099
> > > > > >     Number of ports: 2
> > > > > >     Firmware version: 2.42.5000
> > > > > >     Hardware version: 1
> > > > > >     Node GUID: 0xf4521403007be0e0
> > > > > >     System image GUID: 0xf4521403007be0e3
> > > > > >     Port 1:
> > > > > >             State: Active
> > > > > >             Physical state: LinkUp
> > > > > >             Rate: 56
> > > > > >             Base lid: 29
> > > > > >             LMC: 0
> > > > > >             SM lid: 1
> > > > > >             Capability mask: 0x0259486a
> > > > > >             Port GUID: 0xf4521403007be0e1
> > > > > >             Link layer: InfiniBand
> > > > > >     Port 2:
> > > > > >             State: Active
> > > > > >             Physical state: LinkUp
> > > > > >             Rate: 40
> > > > > >             Base lid: 44
> > > > > >             LMC: 1
> > > > > >             SM lid: 36
> > > > > >             Capability mask: 0x02594868
> > > > > >             Port GUID: 0xf4521403007be0e2
> > > > > >             Link layer: InfiniBand
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards!
> > > > > > XiaoLi Feng 冯小丽
> > > > > >
> > > > > > Red Hat Software (Beijing) Co.,Ltd filesystem-qe Team
> > > > > > IRC:xifeng,#channel: fs-qe
> > > > > > Tel:+86-10-8388112
> > > > > > 9/F, Raycom
> > > > >
> > > > >
> > > >
> > >
> > >
> >
> 
> 
> _______________________________________________
> Linux-cifsd-devel mailing list
> Linux-cifsd-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-cifsd-devel



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-19  6:56             ` ronnie sahlberg
@ 2019-12-19  7:48               ` Hyunchul Lee
  0 siblings, 0 replies; 14+ messages in thread
From: Hyunchul Lee @ 2019-12-19  7:48 UTC (permalink / raw)
  To: ronnie sahlberg; +Cc: Xiaoli Feng, Tom Talpey, linux-cifs, linux-cifsd-devel

Hello Ronnie,

2019년 12월 19일 (목) 오후 3:57, ronnie sahlberg <ronniesahlberg@gmail.com>님이 작성:
>
> On Thu, Dec 19, 2019 at 4:47 PM Hyeoncheol Lee <hyc.lee@gmail.com> wrote:
> >
> > Hello Xiaoli,
> >
> > cifsd is an in-kernel SMB server which supports SMB Direct.
> > If you want to test SMB Direct with cifs, you can use cifsd.
> >
> > Currently we have tested SMB Direct between two old fashioned
> > Mellanox ConnectX devices which are connected directly,
> > and between two soft RoCE devices in kernel.
>
> Thanks Hyeoncheol,
>
> This is very interesting!
>
> We have a buildbot upstream for the cifs client where we test every
> set of patches before they go to linus :
> http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/302
>
> I havent had time to test with your cifsd yet but as you say you can
> use soft-RoCE
> you are saying it will work doing soft-RoCE between one VM running
> cifs.ko and a basic virtio NIC
> to another VM running your cifsd and also using a basic virtio NIC?

Yes, I had tested SMB Direct between the two VMs that are configured
as you said.

>
> I would really want to set up so we can do RDMA tests for cifs.ko, and
> by accident cifsd, in our buildbot.
>

We will help you. but before that, our cifsd's xfstests test-cases are
a different
with your cifs' test-cases. your some test-cases may be failed for cifsd.
our test-cases were suggested by Steve in the past.

So we want to know the list of your test-cases and check that the cases will
succeed for cifsd.
And while we check the test-cases, the common test-cases between cifs and cifsd
can be tested in the buildbot.

Thank you.
>
> regards
> ronnie sahlberg
>
>
> >
> > You can get cifsd's code from the following url, and
> > If you have any questions, we will help you.
> >
> > https://github.com/cifsd-team/cifsd.git
> >
> > Thank you.
> >
> >
> > 2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <xifeng@redhat.com>님이 작성:
> > >
> > > Thanks Ronnie for the information. Last question: Is it supported that mount the windows file
> > > server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?
> > >
> > > ----- Original Message -----
> > > > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > > > To: "Xiaoli Feng" <xifeng@redhat.com>
> > > > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs" <linux-cifs@vger.kernel.org>
> > > > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > > > Subject: Re: How to use SMB Direct
> > > >
> > > > I don't think samba supports SMB Direct.
> > > > Metze used to have a private repo with some experimental patches but I
> > > > don't think it
> > > > has landed in official samba yet.
> > > >
> > > > For a linux server, there is an experimental kernel based server
> > > > called cifsd which should support SMBDirect
> > > > but is incomplete in other areas. It is not part of linus tree yet
> > > > (but soon I hope).
> > > >
> > > > Windows servers support SMBDirect but I am not aware of any soft-rdma
> > > > support so you might be
> > > > limited to just using real hw for any tests.
> > > >
> > > > Linux serverside SMBD support is not ready afaik.
> > > >
> > > > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > > > >
> > > > > Hello Tom,
> > > > >
> > > > > Then I try to use IP to mount, it show this error:
> > > > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > > port=5445
> > > > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > > port=445
> > > > >
> > > > > Client:
> > > > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > > > mount error(2): No such file or directory
> > > > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > > > >
> > > > > Server:
> > > > > $ ib addr |grep 172.31.0.250/
> > > > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > > > >
> > > > > Samba version is 4.11.2. And firewalld is stopped.
> > > > >
> > > > > ----- Original Message -----
> > > > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > > > Subject: RE: How to use SMB Direct
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > > > On
> > > > > > > Behalf Of Xiaoli Feng
> > > > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > > > To: linux-cifs@vger.kernel.org
> > > > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > > > >
> > > > > > > Hello guys,
> > > > > > >
> > > > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it works
> > > > > > > in
> > > > > > > upstream.
> > > > > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
> > > > > > > smb.conf is:
> > > > > > > [cifs]
> > > > > > > path=/mnt/cifs
> > > > > > > writeable=yes
> > > > > > >
> > > > > > > Then I try to mount the share on another rdma machine with 5.5.0-rc1+
> > > > > > > kernel.
> > > > > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > > > > >
> > > > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > > > rdma_resolve_addr()
> > > > > > > completed -113"
> > > > > >
> > > > > > Errno 113 is "no route to host". Sounds like a network or address issue.
> > > > > >
> > > > > > Tom.
> > > > > >
> > > > > > > Does SMB Direct work fine in upstream?
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > > > > $ ibstat
> > > > > > > CA 'mlx4_0'
> > > > > > >     CA type: MT4099
> > > > > > >     Number of ports: 2
> > > > > > >     Firmware version: 2.42.5000
> > > > > > >     Hardware version: 1
> > > > > > >     Node GUID: 0xf4521403007be0e0
> > > > > > >     System image GUID: 0xf4521403007be0e3
> > > > > > >     Port 1:
> > > > > > >             State: Active
> > > > > > >             Physical state: LinkUp
> > > > > > >             Rate: 56
> > > > > > >             Base lid: 29
> > > > > > >             LMC: 0
> > > > > > >             SM lid: 1
> > > > > > >             Capability mask: 0x0259486a
> > > > > > >             Port GUID: 0xf4521403007be0e1
> > > > > > >             Link layer: InfiniBand
> > > > > > >     Port 2:
> > > > > > >             State: Active
> > > > > > >             Physical state: LinkUp
> > > > > > >             Rate: 40
> > > > > > >             Base lid: 44
> > > > > > >             LMC: 1
> > > > > > >             SM lid: 36
> > > > > > >             Capability mask: 0x02594868
> > > > > > >             Port GUID: 0xf4521403007be0e2
> > > > > > >             Link layer: InfiniBand
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards!
> > > > > > > XiaoLi Feng 冯小丽
> > > > > > >
> > > > > > > Red Hat Software (Beijing) Co.,Ltd
> > > > > > > filesystem-qe Team
> > > > > > > IRC:xifeng,#channel: fs-qe
> > > > > > > Tel:+86-10-8388112
> > > > > > > 9/F, Raycom
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-19  6:47           ` Hyeoncheol Lee
  2019-12-19  6:56             ` ronnie sahlberg
  2019-12-19  7:01             ` [Linux-cifsd-devel] " Namjae Jeon
@ 2019-12-20  2:59             ` Xiaoli Feng
  2019-12-20  3:26               ` Steve French
  2019-12-20  4:33               ` Hyunchul Lee
  2 siblings, 2 replies; 14+ messages in thread
From: Xiaoli Feng @ 2019-12-20  2:59 UTC (permalink / raw)
  To: Hyeoncheol Lee, metze
  Cc: ronnie sahlberg, Tom Talpey, linux-cifs, linux-cifsd-devel, Steve French

Hello Hyeoncheol,

When use the cifsd you provide, are there any required for Samba server?
Now I have two machines that have Mellanox ConnectX-3. And install 5.5.0-rc1+.
If one machine install the cifsd. Then I can use the SMB Direct without the 
concern of Samba server. Is it right?

As I know, seems there are two projects for SMB Direct(kernel part). One is
the cifsd. The others is Metze's repo. Which one is planning to go into linux
tree or both are? 

https://github.com/cifsd-team/cifsd.git
https://git.samba.org/?p=metze/linux/smbdirect.git;a=shortlog;h=refs/heads/smbdirect-work-in-progress

Thanks.

----- Original Message -----
> From: "Hyeoncheol Lee" <hyc.lee@gmail.com>
> To: "Xiaoli Feng" <xifeng@redhat.com>
> Cc: "ronnie sahlberg" <ronniesahlberg@gmail.com>, "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> <linux-cifs@vger.kernel.org>, linux-cifsd-devel@lists.sourceforge.net
> Sent: Thursday, December 19, 2019 2:47:40 PM
> Subject: Re: How to use SMB Direct
> 
> Hello Xiaoli,
> 
> cifsd is an in-kernel SMB server which supports SMB Direct.
> If you want to test SMB Direct with cifs, you can use cifsd.
> 
> Currently we have tested SMB Direct between two old fashioned
> Mellanox ConnectX devices which are connected directly,
> and between two soft RoCE devices in kernel.
> 
> You can get cifsd's code from the following url, and
> If you have any questions, we will help you.
> 
> https://github.com/cifsd-team/cifsd.git
> 
> Thank you.
> 
> 
> 2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <xifeng@redhat.com>님이 작성:
> >
> > Thanks Ronnie for the information. Last question: Is it supported that
> > mount the windows file
> > server(has rdma hardware) with SMB Direct in linux client(has rdma
> > hardware)?
> >
> > ----- Original Message -----
> > > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > > To: "Xiaoli Feng" <xifeng@redhat.com>
> > > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> > > <linux-cifs@vger.kernel.org>
> > > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > > Subject: Re: How to use SMB Direct
> > >
> > > I don't think samba supports SMB Direct.
> > > Metze used to have a private repo with some experimental patches but I
> > > don't think it
> > > has landed in official samba yet.
> > >
> > > For a linux server, there is an experimental kernel based server
> > > called cifsd which should support SMBDirect
> > > but is incomplete in other areas. It is not part of linus tree yet
> > > (but soon I hope).
> > >
> > > Windows servers support SMBDirect but I am not aware of any soft-rdma
> > > support so you might be
> > > limited to just using real hw for any tests.
> > >
> > > Linux serverside SMBD support is not ready afaik.
> > >
> > > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > > >
> > > > Hello Tom,
> > > >
> > > > Then I try to use IP to mount, it show this error:
> > > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > port=5445
> > > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > port=445
> > > >
> > > > Client:
> > > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > > mount error(2): No such file or directory
> > > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > > >
> > > > Server:
> > > > $ ib addr |grep 172.31.0.250/
> > > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > > >
> > > > Samba version is 4.11.2. And firewalld is stopped.
> > > >
> > > > ----- Original Message -----
> > > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > > Subject: RE: How to use SMB Direct
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > > On
> > > > > > Behalf Of Xiaoli Feng
> > > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > > To: linux-cifs@vger.kernel.org
> > > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > > >
> > > > > > Hello guys,
> > > > > >
> > > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it
> > > > > > works
> > > > > > in
> > > > > > upstream.
> > > > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel.
> > > > > > The
> > > > > > smb.conf is:
> > > > > > [cifs]
> > > > > > path=/mnt/cifs
> > > > > > writeable=yes
> > > > > >
> > > > > > Then I try to mount the share on another rdma machine with
> > > > > > 5.5.0-rc1+
> > > > > > kernel.
> > > > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > > > >
> > > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > > rdma_resolve_addr()
> > > > > > completed -113"
> > > > >
> > > > > Errno 113 is "no route to host". Sounds like a network or address
> > > > > issue.
> > > > >
> > > > > Tom.
> > > > >
> > > > > > Does SMB Direct work fine in upstream?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > > > $ ibstat
> > > > > > CA 'mlx4_0'
> > > > > >     CA type: MT4099
> > > > > >     Number of ports: 2
> > > > > >     Firmware version: 2.42.5000
> > > > > >     Hardware version: 1
> > > > > >     Node GUID: 0xf4521403007be0e0
> > > > > >     System image GUID: 0xf4521403007be0e3
> > > > > >     Port 1:
> > > > > >             State: Active
> > > > > >             Physical state: LinkUp
> > > > > >             Rate: 56
> > > > > >             Base lid: 29
> > > > > >             LMC: 0
> > > > > >             SM lid: 1
> > > > > >             Capability mask: 0x0259486a
> > > > > >             Port GUID: 0xf4521403007be0e1
> > > > > >             Link layer: InfiniBand
> > > > > >     Port 2:
> > > > > >             State: Active
> > > > > >             Physical state: LinkUp
> > > > > >             Rate: 40
> > > > > >             Base lid: 44
> > > > > >             LMC: 1
> > > > > >             SM lid: 36
> > > > > >             Capability mask: 0x02594868
> > > > > >             Port GUID: 0xf4521403007be0e2
> > > > > >             Link layer: InfiniBand
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards!
> > > > > > XiaoLi Feng 冯小丽
> > > > > >
> > > > > > Red Hat Software (Beijing) Co.,Ltd
> > > > > > filesystem-qe Team
> > > > > > IRC:xifeng,#channel: fs-qe
> > > > > > Tel:+86-10-8388112
> > > > > > 9/F, Raycom
> > > > >
> > > > >
> > > >
> > >
> > >
> >
> 
> 


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-20  2:59             ` Xiaoli Feng
@ 2019-12-20  3:26               ` Steve French
  2019-12-20 13:25                 ` Stefan Metzmacher
  2019-12-20  4:33               ` Hyunchul Lee
  1 sibling, 1 reply; 14+ messages in thread
From: Steve French @ 2019-12-20  3:26 UTC (permalink / raw)
  To: Xiaoli Feng
  Cc: Hyeoncheol Lee, Stefan (metze) Metzmacher, ronnie sahlberg,
	Tom Talpey, linux-cifs, linux-cifsd-devel, Namjae Jeon

The work of Metze's on RDMA/smbdirect extensions to Samba, rely in
part on kernel code.   I can help merge that into the kernel but I
would like some agreement between Metze and Long Li (and Tom etc.) on
what would be useful to be shared between cifs.ko and the proposed
RDMA helpe driver for Samba.   From discussions this fall, it is
probably too early to consider the merge request, but I am very open
to following up discussions on this.

The alternative RDMA/smbdirect implementation in cifsd (the proposed
smb3 kernel server written by Namjae and other) is very exciting and
an interesting topic, but likely unrelated to the Samba server
implementation that Metze is working on.   The issues involved in
merging cifsd (the SMB3 kernel server) into the mainline kernel are
quite different than those related to Samba's RDMA implementation, and
instead mostly have to deal with addressing review comments from
others (I have sent namjae various suggestions which he appears to be
addressing).   More review feedback of the cifsd kernel server will be
very helpful - but the main issues are not RDMA related.


On Thu, Dec 19, 2019 at 9:00 PM Xiaoli Feng <xifeng@redhat.com> wrote:
>
> Hello Hyeoncheol,
>
> When use the cifsd you provide, are there any required for Samba server?
> Now I have two machines that have Mellanox ConnectX-3. And install 5.5.0-rc1+.
> If one machine install the cifsd. Then I can use the SMB Direct without the
> concern of Samba server. Is it right?
>
> As I know, seems there are two projects for SMB Direct(kernel part). One is
> the cifsd. The others is Metze's repo. Which one is planning to go into linux
> tree or both are?
>
> https://github.com/cifsd-team/cifsd.git
> https://git.samba.org/?p=metze/linux/smbdirect.git;a=shortlog;h=refs/heads/smbdirect-work-in-progress
>
> Thanks.
>
> ----- Original Message -----
> > From: "Hyeoncheol Lee" <hyc.lee@gmail.com>
> > To: "Xiaoli Feng" <xifeng@redhat.com>
> > Cc: "ronnie sahlberg" <ronniesahlberg@gmail.com>, "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> > <linux-cifs@vger.kernel.org>, linux-cifsd-devel@lists.sourceforge.net
> > Sent: Thursday, December 19, 2019 2:47:40 PM
> > Subject: Re: How to use SMB Direct
> >
> > Hello Xiaoli,
> >
> > cifsd is an in-kernel SMB server which supports SMB Direct.
> > If you want to test SMB Direct with cifs, you can use cifsd.
> >
> > Currently we have tested SMB Direct between two old fashioned
> > Mellanox ConnectX devices which are connected directly,
> > and between two soft RoCE devices in kernel.
> >
> > You can get cifsd's code from the following url, and
> > If you have any questions, we will help you.
> >
> > https://github.com/cifsd-team/cifsd.git
> >
> > Thank you.
> >
> >
> > 2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <xifeng@redhat.com>님이 작성:
> > >
> > > Thanks Ronnie for the information. Last question: Is it supported that
> > > mount the windows file
> > > server(has rdma hardware) with SMB Direct in linux client(has rdma
> > > hardware)?
> > >
> > > ----- Original Message -----
> > > > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > > > To: "Xiaoli Feng" <xifeng@redhat.com>
> > > > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> > > > <linux-cifs@vger.kernel.org>
> > > > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > > > Subject: Re: How to use SMB Direct
> > > >
> > > > I don't think samba supports SMB Direct.
> > > > Metze used to have a private repo with some experimental patches but I
> > > > don't think it
> > > > has landed in official samba yet.
> > > >
> > > > For a linux server, there is an experimental kernel based server
> > > > called cifsd which should support SMBDirect
> > > > but is incomplete in other areas. It is not part of linus tree yet
> > > > (but soon I hope).
> > > >
> > > > Windows servers support SMBDirect but I am not aware of any soft-rdma
> > > > support so you might be
> > > > limited to just using real hw for any tests.
> > > >
> > > > Linux serverside SMBD support is not ready afaik.
> > > >
> > > > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > > > >
> > > > > Hello Tom,
> > > > >
> > > > > Then I try to use IP to mount, it show this error:
> > > > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > > port=5445
> > > > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > > port=445
> > > > >
> > > > > Client:
> > > > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > > > mount error(2): No such file or directory
> > > > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > > > >
> > > > > Server:
> > > > > $ ib addr |grep 172.31.0.250/
> > > > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > > > >
> > > > > Samba version is 4.11.2. And firewalld is stopped.
> > > > >
> > > > > ----- Original Message -----
> > > > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > > > Subject: RE: How to use SMB Direct
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > > > On
> > > > > > > Behalf Of Xiaoli Feng
> > > > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > > > To: linux-cifs@vger.kernel.org
> > > > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > > > >
> > > > > > > Hello guys,
> > > > > > >
> > > > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it
> > > > > > > works
> > > > > > > in
> > > > > > > upstream.
> > > > > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel.
> > > > > > > The
> > > > > > > smb.conf is:
> > > > > > > [cifs]
> > > > > > > path=/mnt/cifs
> > > > > > > writeable=yes
> > > > > > >
> > > > > > > Then I try to mount the share on another rdma machine with
> > > > > > > 5.5.0-rc1+
> > > > > > > kernel.
> > > > > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > > > > >
> > > > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > > > rdma_resolve_addr()
> > > > > > > completed -113"
> > > > > >
> > > > > > Errno 113 is "no route to host". Sounds like a network or address
> > > > > > issue.
> > > > > >
> > > > > > Tom.
> > > > > >
> > > > > > > Does SMB Direct work fine in upstream?
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > > > > $ ibstat
> > > > > > > CA 'mlx4_0'
> > > > > > >     CA type: MT4099
> > > > > > >     Number of ports: 2
> > > > > > >     Firmware version: 2.42.5000
> > > > > > >     Hardware version: 1
> > > > > > >     Node GUID: 0xf4521403007be0e0
> > > > > > >     System image GUID: 0xf4521403007be0e3
> > > > > > >     Port 1:
> > > > > > >             State: Active
> > > > > > >             Physical state: LinkUp
> > > > > > >             Rate: 56
> > > > > > >             Base lid: 29
> > > > > > >             LMC: 0
> > > > > > >             SM lid: 1
> > > > > > >             Capability mask: 0x0259486a
> > > > > > >             Port GUID: 0xf4521403007be0e1
> > > > > > >             Link layer: InfiniBand
> > > > > > >     Port 2:
> > > > > > >             State: Active
> > > > > > >             Physical state: LinkUp
> > > > > > >             Rate: 40
> > > > > > >             Base lid: 44
> > > > > > >             LMC: 1
> > > > > > >             SM lid: 36
> > > > > > >             Capability mask: 0x02594868
> > > > > > >             Port GUID: 0xf4521403007be0e2
> > > > > > >             Link layer: InfiniBand
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards!
> > > > > > > XiaoLi Feng 冯小丽
> > > > > > >
> > > > > > > Red Hat Software (Beijing) Co.,Ltd
> > > > > > > filesystem-qe Team
> > > > > > > IRC:xifeng,#channel: fs-qe
> > > > > > > Tel:+86-10-8388112
> > > > > > > 9/F, Raycom
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
> >
>


-- 
Thanks,

Steve

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-20  2:59             ` Xiaoli Feng
  2019-12-20  3:26               ` Steve French
@ 2019-12-20  4:33               ` Hyunchul Lee
  1 sibling, 0 replies; 14+ messages in thread
From: Hyunchul Lee @ 2019-12-20  4:33 UTC (permalink / raw)
  To: Xiaoli Feng
  Cc: metze, ronnie sahlberg, Tom Talpey, linux-cifs,
	linux-cifsd-devel, Steve French

Hello Xiaoli,

2019년 12월 20일 (금) 오후 12:00, Xiaoli Feng <xifeng@redhat.com>님이 작성:
>
> Hello Hyeoncheol,
>
> When use the cifsd you provide, are there any required for Samba server?
> Now I have two machines that have Mellanox ConnectX-3. And install 5.5.0-rc1+.
> If one machine install the cifsd. Then I can use the SMB Direct without the
> concern of Samba server. Is it right?
>
Yes, you can use the SMB Direct without Samba server.

Following are instructions for cifsd setup.
And cifsd utils and a detailed description can be found in
https://github.com/cifsd-team/cifsd-tools

-  Build and install cifsd kernel driver
modprobe cifsd

- Create user/password for SMB share
cifsuseradd -a <Enter USERNAME for SMB share access>
Enter password for SMB share access

- Create /etc/cifs/smb.conf file, add SMB share in smb.conf file
Refer smb.conf.example

- Start cifsd user space daemon
cifsd

- Access share from Windows or Linux using CIFS

Thank you.

> As I know, seems there are two projects for SMB Direct(kernel part). One is
> the cifsd. The others is Metze's repo. Which one is planning to go into linux
> tree or both are?
>
> https://github.com/cifsd-team/cifsd.git
> https://git.samba.org/?p=metze/linux/smbdirect.git;a=shortlog;h=refs/heads/smbdirect-work-in-progress
>
> Thanks.
>
> ----- Original Message -----
> > From: "Hyeoncheol Lee" <hyc.lee@gmail.com>
> > To: "Xiaoli Feng" <xifeng@redhat.com>
> > Cc: "ronnie sahlberg" <ronniesahlberg@gmail.com>, "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> > <linux-cifs@vger.kernel.org>, linux-cifsd-devel@lists.sourceforge.net
> > Sent: Thursday, December 19, 2019 2:47:40 PM
> > Subject: Re: How to use SMB Direct
> >
> > Hello Xiaoli,
> >
> > cifsd is an in-kernel SMB server which supports SMB Direct.
> > If you want to test SMB Direct with cifs, you can use cifsd.
> >
> > Currently we have tested SMB Direct between two old fashioned
> > Mellanox ConnectX devices which are connected directly,
> > and between two soft RoCE devices in kernel.
> >
> > You can get cifsd's code from the following url, and
> > If you have any questions, we will help you.
> >
> > https://github.com/cifsd-team/cifsd.git
> >
> > Thank you.
> >
> >
> > 2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <xifeng@redhat.com>님이 작성:
> > >
> > > Thanks Ronnie for the information. Last question: Is it supported that
> > > mount the windows file
> > > server(has rdma hardware) with SMB Direct in linux client(has rdma
> > > hardware)?
> > >
> > > ----- Original Message -----
> > > > From: "ronnie sahlberg" <ronniesahlberg@gmail.com>
> > > > To: "Xiaoli Feng" <xifeng@redhat.com>
> > > > Cc: "Tom Talpey" <ttalpey@microsoft.com>, "linux-cifs"
> > > > <linux-cifs@vger.kernel.org>
> > > > Sent: Wednesday, December 18, 2019 4:12:32 PM
> > > > Subject: Re: How to use SMB Direct
> > > >
> > > > I don't think samba supports SMB Direct.
> > > > Metze used to have a private repo with some experimental patches but I
> > > > don't think it
> > > > has landed in official samba yet.
> > > >
> > > > For a linux server, there is an experimental kernel based server
> > > > called cifsd which should support SMBDirect
> > > > but is incomplete in other areas. It is not part of linus tree yet
> > > > (but soon I hope).
> > > >
> > > > Windows servers support SMBDirect but I am not aware of any soft-rdma
> > > > support so you might be
> > > > limited to just using real hw for any tests.
> > > >
> > > > Linux serverside SMBD support is not ready afaik.
> > > >
> > > > On Wed, Dec 18, 2019 at 12:49 PM Xiaoli Feng <xifeng@redhat.com> wrote:
> > > > >
> > > > > Hello Tom,
> > > > >
> > > > > Then I try to use IP to mount, it show this error:
> > > > > [79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > > port=5445
> > > > > [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed
> > > > > port=445
> > > > >
> > > > > Client:
> > > > > $ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma
> > > > > mount error(2): No such file or directory
> > > > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > > > >
> > > > > Server:
> > > > > $ ib addr |grep 172.31.0.250/
> > > > > mlx4_ib0:               Link UP, Interface UP   172.31.0.250/24
> > > > >
> > > > > Samba version is 4.11.2. And firewalld is stopped.
> > > > >
> > > > > ----- Original Message -----
> > > > > > From: "Tom Talpey" <ttalpey@microsoft.com>
> > > > > > To: "Xiaoli Feng" <xifeng@redhat.com>, linux-cifs@vger.kernel.org
> > > > > > Sent: Tuesday, December 17, 2019 1:11:24 AM
> > > > > > Subject: RE: How to use SMB Direct
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: linux-cifs-owner@vger.kernel.org
> > > > > > > <linux-cifs-owner@vger.kernel.org>
> > > > > > > On
> > > > > > > Behalf Of Xiaoli Feng
> > > > > > > Sent: Monday, December 16, 2019 10:30 AM
> > > > > > > To: linux-cifs@vger.kernel.org
> > > > > > > Subject: [EXTERNAL] How to use SMB Direct
> > > > > > >
> > > > > > > Hello guys,
> > > > > > >
> > > > > > > I'd like to test SMB Direct. But it's failed. I'm not sure if it
> > > > > > > works
> > > > > > > in
> > > > > > > upstream.
> > > > > > > I setup samba server on one rdma machine with 5.5.0-rc1+ kernel.
> > > > > > > The
> > > > > > > smb.conf is:
> > > > > > > [cifs]
> > > > > > > path=/mnt/cifs
> > > > > > > writeable=yes
> > > > > > >
> > > > > > > Then I try to mount the share on another rdma machine with
> > > > > > > 5.5.0-rc1+
> > > > > > > kernel.
> > > > > > >    mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
> > > > > > >
> > > > > > > It's failed because of "CIFS VFS: smbd_create_id:614
> > > > > > > rdma_resolve_addr()
> > > > > > > completed -113"
> > > > > >
> > > > > > Errno 113 is "no route to host". Sounds like a network or address
> > > > > > issue.
> > > > > >
> > > > > > Tom.
> > > > > >
> > > > > > > Does SMB Direct work fine in upstream?
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > > $ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT
> > > > > > > CONFIG_CIFS_SMB_DIRECT=y
> > > > > > > $ ibstat
> > > > > > > CA 'mlx4_0'
> > > > > > >     CA type: MT4099
> > > > > > >     Number of ports: 2
> > > > > > >     Firmware version: 2.42.5000
> > > > > > >     Hardware version: 1
> > > > > > >     Node GUID: 0xf4521403007be0e0
> > > > > > >     System image GUID: 0xf4521403007be0e3
> > > > > > >     Port 1:
> > > > > > >             State: Active
> > > > > > >             Physical state: LinkUp
> > > > > > >             Rate: 56
> > > > > > >             Base lid: 29
> > > > > > >             LMC: 0
> > > > > > >             SM lid: 1
> > > > > > >             Capability mask: 0x0259486a
> > > > > > >             Port GUID: 0xf4521403007be0e1
> > > > > > >             Link layer: InfiniBand
> > > > > > >     Port 2:
> > > > > > >             State: Active
> > > > > > >             Physical state: LinkUp
> > > > > > >             Rate: 40
> > > > > > >             Base lid: 44
> > > > > > >             LMC: 1
> > > > > > >             SM lid: 36
> > > > > > >             Capability mask: 0x02594868
> > > > > > >             Port GUID: 0xf4521403007be0e2
> > > > > > >             Link layer: InfiniBand
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards!
> > > > > > > XiaoLi Feng 冯小丽
> > > > > > >
> > > > > > > Red Hat Software (Beijing) Co.,Ltd
> > > > > > > filesystem-qe Team
> > > > > > > IRC:xifeng,#channel: fs-qe
> > > > > > > Tel:+86-10-8388112
> > > > > > > 9/F, Raycom
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
> >
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: How to use SMB Direct
  2019-12-20  3:26               ` Steve French
@ 2019-12-20 13:25                 ` Stefan Metzmacher
  0 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2019-12-20 13:25 UTC (permalink / raw)
  To: Steve French, Xiaoli Feng
  Cc: Hyeoncheol Lee, ronnie sahlberg, Tom Talpey, linux-cifs,
	linux-cifsd-devel, Namjae Jeon


[-- Attachment #1.1: Type: text/plain, Size: 1259 bytes --]

Am 20.12.19 um 04:26 schrieb Steve French:
> The work of Metze's on RDMA/smbdirect extensions to Samba, rely in
> part on kernel code.   I can help merge that into the kernel but I
> would like some agreement between Metze and Long Li (and Tom etc.) on
> what would be useful to be shared between cifs.ko and the proposed
> RDMA helpe driver for Samba.   From discussions this fall, it is
> probably too early to consider the merge request, but I am very open
> to following up discussions on this.

Yes, it's too early. Sadly I still don't have funding to drive this
further. But once I have time for it again I'll try to get this to
a state where we no longer have memory leaks and pass the microsoft
testsuite and manual testing. Once that's done it should be pretty
easy to plug this also into cifs.ko (I already have patches and it
worked basically a few month ago) and also into the cifsd code.
So that we can compare how the implementations perform and if
mine is able to perform in a similar way we can start to think
about merging it to the kernel have just one implementation for the
SMB-Direct protocol that is usable for all 4 cases (cifs.ko and
smbclient as client and Samba's smbd and the kernel cifsd as server).

metze


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2019-12-20 13:55 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <1327532317.1529923.1576509501382.JavaMail.zimbra@redhat.com>
2019-12-16 15:30 ` How to use SMB Direct Xiaoli Feng
2019-12-16 17:11   ` Tom Talpey
2019-12-18  2:48     ` Xiaoli Feng
2019-12-18  8:12       ` ronnie sahlberg
2019-12-19  2:34         ` Xiaoli Feng
2019-12-19  2:58           ` ronnie sahlberg
2019-12-19  6:47           ` Hyeoncheol Lee
2019-12-19  6:56             ` ronnie sahlberg
2019-12-19  7:48               ` Hyunchul Lee
2019-12-19  7:01             ` [Linux-cifsd-devel] " Namjae Jeon
2019-12-20  2:59             ` Xiaoli Feng
2019-12-20  3:26               ` Steve French
2019-12-20 13:25                 ` Stefan Metzmacher
2019-12-20  4:33               ` Hyunchul Lee

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).