Re: [SPDK] NVMf/TCP target using VPP

* Re: [SPDK] NVMf/TCP target using VPP
@ 2018-12-29  0:18 Yang, Ziye
  0 siblings, 0 replies; 12+ messages in thread
From: Yang, Ziye @ 2018-12-29  0:18 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 8816 bytes --]

Hi Paul,

Will submit a patch to revise the NVMe-oF target part doc later for adding the TCP configuration description. When the patch is ready, will notify you.

Thanks.

Best Regards
Ziye Yang 

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Luse, Paul E
Sent: Saturday, December 29, 2018 8:16 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] NVMf/TCP target using VPP

Great Ziye, thanks. Yeah that was my comment that RDMA was documented but TCP was not but later as opposed to now? Thanks for separating out the VPP stuff!!!

-Pual

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye
Sent: Friday, December 28, 2018 5:13 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] NVMf/TCP target using VPP

Hi Paul,

For the configuration of RDMA in NVMe-oF is already available on this page:
https://spdk.io/doc/nvmf.html  We need to update with TCP transport later, currently we only have RDMA support description. 

For VPP integration, we need to have a separate page, to avoid the misunderstanding from users. It is not only for iSCSI target, but can be working with any SPDK program which use the sock programing interface defined in (spdk/lib/sock) folder.

Best Regards
Ziye Yang 

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Luse, Paul E
Sent: Saturday, December 29, 2018 12:20 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] NVMf/TCP target using VPP

Awesome, thanks! As a follow-up don't we need something in the NVMEoF section to make it super clear (and easy) for folks to know how to setup for RDMA or TCP?

Thx
PAul

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye
Sent: Thursday, December 27, 2018 10:53 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] NVMf/TCP target using VPP

A patch: https://review.gerrithub.io/#/c/spdk/spdk/+/438372/ is submitted to put the VPP part into a separate page first.

Best Regards
Ziye Yang 

-----Original Message-----
From: Yang, Ziye 
Sent: Friday, December 28, 2018 9:30 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: RE: NVMf/TCP target using VPP

Hi Paul,

My opinion is that: we can make the VPP configuration general and move this doc to a separate page instead of putting it in iSCSI pages..  Actually for NVMe-oF compiled with VPP,  if the TCP transport listens to an address (managed VPP), it will use VPP socket.

Best Regards
Ziye Yang 

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Luse, Paul E
Sent: Thursday, December 27, 2018 9:59 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] NVMf/TCP target using VPP

Hi Ziye,

I looked, well not exhaustively, for NVMe-oF TCP docs in the SPDK repo and couldn't find anything.  Do they exist somewhere or do we need to get a Trello card up there to have them created?

Thanks!
Paul

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye
Sent: Wednesday, December 26, 2018 9:19 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] NVMf/TCP target using VPP

Hi Ramaraj,

Besides for NVMe-oF tcp Transport, I recommend that you first use loop back to test.  See the details in: (https://spdk.io/doc/iscsi.html). This is to make sure that you can do this, then you can replace the tap device with real DPDK PMD device.  I verified the following steps for NVMe-oF TCP, it is OK, and confirm that we leverage the VPP stack. 

SPDK supports VPP version 18.01.1.

1. Building VPP (optional)
Please skip this step if using already built packages.

Clone and checkout VPP

git clone https://gerrit.fd.io/r/vpp && cd vpp git checkout v18.01.1 Install VPP build dependencies

make install-dep
Build and create .rpm packages

make pkg-rpm
Alternatively, build and create .deb packages

make pkg-deb
Packages can be found in vpp/build-root/ directory.

For more in depth instructions please see Building section in VPP documentation

Please note: VPP 18.01.1 does not support OpenSSL 1.1. It is suggested to install a compatibility package for compilation time.

sudo dnf install -y --allowerasing compat-openssl10-devel Then reinstall latest OpenSSL devel package:

sudo dnf install -y --allowerasing openssl-devel 2. Installing VPP Packages can be installed from distribution repository or built in previous step. Minimal set of packages consists of vpp, vpp-lib and vpp-devel.

Note: Please remove or modify /etc/sysctl.d/80-vpp.conf file with appropriate values dependent on number of hugepages that will be used on system.

3. Running VPP
VPP takes over any network interfaces that were bound to userspace driver, for details please see DPDK guide on Binding and Unbinding Network Ports to/from the Kernel Modules.

VPP is installed as service and disabled by default. To start VPP with default config:

sudo systemctl start vpp
Alternatively, use vpp binary directly

sudo vpp unix {cli-listen /run/vpp/cli.sock} A usefull tool is vppctl, that allows to control running VPP instance. Either by entering VPP configuration prompt

sudo vppctl
Or, by sending single command directly. For example to display interfaces within VPP:

sudo vppctl show interface
Example: Tap interfaces on single host
For functional test purpose a virtual tap interface can be created, so no additional network hardware is required. This will allow network communication between SPDK iSCSI target using VPP end of tap and kernel iSCSI initiator using the kernel part of tap. A single host is used in this scenario.

Create tap interface via VPP

vppctl tap connect tap0
vppctl set interface state tapcli-0 up
vppctl set interface ip address tapcli-0 10.0.0.1/24 vppctl show int addr Assign address on kernel interface

sudo ip addr add 10.0.0.2/24 dev tap0
sudo ip link set tap0 up
To verify connectivity

ping 10.0.0.1
4. Building SPDK with VPP
Support for VPP can be built into SPDK by using configuration option.

configure --with-vpp
Alternatively, directory with built libraries can be pointed at and will be used for compilation instead of installed packages.

configure --with-vpp=/path/to/vpp/repo/build-root/vpp

Best Regards
Ziye Yang 

-----Original Message-----
From: Yang, Ziye
Sent: Thursday, December 27, 2018 10:55 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: RE: NVMf/TCP target using VPP

Hi Ramaraj,

For NVMe-of TCP target side, we can use VPP to test. 
For NVMe-of TCP host via perf program, we may still need some efforts to make it happen.

Best Regards
Ziye Yang 

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Ramaraj Pandian
Sent: Thursday, December 27, 2018 2:54 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] NVMf/TCP target using VPP

I see SPDK supports VPP, is that in development stage or can it be used for some testing?

Is the configuration setup for nvmf_tgt same as iscsci_tgt?

-          Unbind network from kernel using dpdk tools

-          Use the same network pci for VPP config

-          Start vpp

-          Start nvmf_tgt with VPP enabled

On initiator side:

-          Use spdk perf with tcp enabled to do io.

With above setup, perf failing even connecting to target but if I use kernel/posix tcp on both target and initialtor, it works fine. Am I missing any configuration setup?

Thanks
Ramaraj

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk

^ permalink raw reply	[flat|nested] 12+ messages in thread