From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jianfeng Tan Subject: [PATCH v4 0/8] virtio_user as an alternative exception path Date: Fri, 13 Jan 2017 12:18:33 +0000 Message-ID: <1484309921-116526-1-git-send-email-jianfeng.tan@intel.com> References: <1480689075-66977-1-git-send-email-jianfeng.tan@intel.com> Cc: yuanhan.liu@linux.intel.com, ferruh.yigit@intel.com, cunming.liang@intel.com, Jianfeng Tan To: dev@dpdk.org Return-path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 15ACE106A for ; Fri, 13 Jan 2017 14:48:58 +0100 (CET) In-Reply-To: <1480689075-66977-1-git-send-email-jianfeng.tan@intel.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" v4: - Fix a clang compiling error by removing "NULL" line in the definition of vhost_msg_strings. This error does not show up when it's defined as a static variable, so not necessary to fix it in stable branch. - Query kernel to get how many regions are supported, default 64 regions. - Set TUNSETSNDBUF to INT_MAX. - When get_features, unmask those backend-specific feature bits. - Remove VHOST_KERNEL_MAX_QUEUES (8) restriction, but due to another restriction by VIRTIO_MAX_VIRTQUEUES (8), we still cannot configure more than 8 queues. - Add a howto document. v3: - Drop the patch to postpone driver ok sending patch, superseded it with a bug fix to disable all virtqueues and re-init the device. (you might wonder why not just send reset owner msg. Under my test, it causes spinlock deadlock problem when killing the program). - Avoid compiling error on 32-bit system for pointer convert. - Fix a bug in patch "abstract virtio user backend ops", vhostfd is not properly assigned. - Fix a "MQ cannot be used" bug in v2, which is related to strip some feature bits that vhost kernel does not recognize. - Update release note. v2: (Lots of them are from yuanhan's comment) - Add offloding feature. - Add multiqueue support. - Add a new patch to postpone the sending of driver ok notification. - Put fix patch ahead of the whole patch series. - Split original 0001 patch into 0003 and 0004 patches. - Remove the original vhost_internal design, just add those into struct virtio_user_dev for simplicity. - Reword "control" to "send_request". - Reword "host_features" to "device_features". In v16.07, we upstreamed a virtual device, virtio_user (with vhost-user as the backend). The path to go with a vhost-kernel backend has been dropped for bad performance comparing to vhost-user and code simplicity. But after a second thought, virtio_user + vhost-kernel is a good candidate as an exceptional path, such as KNI, which exchanges packets with kernel networking stack. - maintenance: vhost-net (kernel) is upstreamed and extensively used kernel module. We don't need any out-of-tree module like KNI. - performance: as with KNI, this solution would use one or more kthreads to send/receive packets from user space DPDK applications, which has little impact on user space polling thread (except that it might enter into kernel space to wake up those kthreads if necessary). - features: vhost-net is born to be a networking solution, which has lots of networking related featuers, like multi queue, tso, multi-seg mbuf, etc. How to test: Here is a simple test case for: _______testpmd_______ iperf-c/iperf-s -- tap0 -- [virtio_user0 -- ixgbe] | | | iperf-s/iperf-c -- ixgbe (kernel driver, eth3) Step 1: Remove MAC addres setting in testpmd csum fwd engine. diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c index 57e6ae2..225c7a8 100644 --- a/app/test-pmd/csumonly.c +++ b/app/test-pmd/csumonly.c @@ -706,10 +706,12 @@ pkt_burst_checksum_forward(struct fwd_stream *fs) * and inner headers */ eth_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *); +#if 0 ether_addr_copy(&peer_eth_addrs[fs->peer_addr], ð_hdr->d_addr); ether_addr_copy(&ports[fs->tx_port].eth_addr, ð_hdr->s_addr); +#endif parse_ethernet(eth_hdr, &info); l3_hdr = (char *)eth_hdr + info.l2_len; Step 2: bind one ixgbe with igb_uio, and start testpmd $(testpmd) -c 0xc -n 4 \ --vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \ -- -i --txqflags=0x0 --disable-hw-vlan --enable-lro \ --enable-rx-cksum --rxd=1024 --txd=1024 To test multiqueue, start testpmd like this: $(testpmd) -c 0xc -n 4 \ --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 \ -- -i --txqflags=0x0 --disable-hw-vlan --enable-lro --enable-rx-cksum \ --txq=2 --rxq=2 --rxd=1024 --txd=1024 Step 3: start testpmd: (port 0 is ixgbe, port 1 is virtio_user) # set fwd csum # csum set ip hw 0 # csum set tcp hw 0 # csum set ip sw 1 # csum set tcp hw 1 # tso set 1448 0 # tso set 1448 1 # start Step 4: start the other end with below script $ ip netns add ns1 $ ip link set eth3 netns ns1 $ ip netns exec ns1 ifconfig eth3 1.1.1.2/24 up $ ip netns exec ns1 taskset 0xf0 iperf3 -s -i 1 Use below command if you change the position of iperf-c and iperf-s. $ ip netns exec ns1 numactl -N 0 iperf3 -c 1.1.1.3 -i 1 -t 30 Step 5: up the tap0 and configure ip $ ifconfig tap0 1.1.1.3/24 up Step 6: start test $ iperf3 -c 1.1.1.2 -i 1 -t 30 Signed-off-by: Jianfeng Tan Jianfeng Tan (8): net/virtio_user: fix wrongly get/set features net/virtio_user: fix not properly reset device net/virtio_user: move vhost user specific code net/virtio_user: abstract virtio user backend ops net/virtio_user: add vhost kernel support net/virtio_user: enable offloading net/virtio_user: enable multiqueue with vhost kernel doc: add guide to use virtio_user as exceptional path .../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes doc/guides/prog_guide/index.rst | 1 + .../prog_guide/virtio_user_as_exceptional_path.rst | 104 ++++++ doc/guides/rel_notes/release_17_02.rst | 20 + drivers/net/virtio/Makefile | 2 + drivers/net/virtio/virtio_ethdev.h | 5 + drivers/net/virtio/virtio_user/vhost.h | 51 +-- drivers/net/virtio/virtio_user/vhost_kernel.c | 401 +++++++++++++++++++++ drivers/net/virtio/virtio_user/vhost_kernel_tap.c | 133 +++++++ drivers/net/virtio/virtio_user/vhost_kernel_tap.h | 67 ++++ drivers/net/virtio/virtio_user/vhost_user.c | 98 +++-- drivers/net/virtio/virtio_user/virtio_user_dev.c | 153 +++++--- drivers/net/virtio/virtio_user/virtio_user_dev.h | 15 +- drivers/net/virtio/virtio_user_ethdev.c | 20 +- 14 files changed, 945 insertions(+), 125 deletions(-) create mode 100644 doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png create mode 100644 doc/guides/prog_guide/virtio_user_as_exceptional_path.rst create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel_tap.c create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel_tap.h -- 2.7.4