* [PATCH 0/3] virtio_user as an alternative exception path
@ 2016-12-02 14:31 Jianfeng Tan
2016-12-02 14:31 ` [PATCH 1/3] net/virtio_user: add vhost layer Jianfeng Tan
` (6 more replies)
0 siblings, 7 replies; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-02 14:31 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
In v16.07, we upstreamed a virtual device, virtio_user (with vhost-user
as the backend). The path to go with a vhost-kernel backend has been
dropped for bad performance comparing to vhost-user and code simplicity.
But after a second thought, virtio_user + vhost-kernel is a good
candidate as an exceptional path, such as KNI, which exchanges packets
with kernel networking stack.
- maintenance: vhost-net (kernel) is upstreamed and extensively used
kernel module. We don't need any out-of-tree module like KNI.
- performance: as with KNI, this solution would use one or more
kthreads to send/receive packets from user space DPDK applications,
which has little impact on user space polling thread (except that
it might enter into kernel space to wake up those kthreads if
necessary.
- features: vhost-net is born to be a networking solution, which has
lots of networking related featuers, like multi queue, tso, multi-seg
mbuf, etc.
Known issues for current version:
- Multiqueue not supported yet.
- Offloading is completely enabled yet; to enhance, we will translate
the virtio header info into mbuf metadata when receiving packets
from kernel; and translate mbuf metadata info into virtio header
when sending packets to kernel.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Jianfeng Tan (3):
net/virtio_user: add vhost layer
net/virtio_user: add vhost kernel support
net/virtio_user: fix wrongly set features
drivers/net/virtio/Makefile | 2 +
drivers/net/virtio/virtio_user/vhost.c | 181 ++++++++++
drivers/net/virtio/virtio_user/vhost.h | 73 ++--
drivers/net/virtio/virtio_user/vhost_kernel.c | 413 +++++++++++++++++++++++
drivers/net/virtio/virtio_user/vhost_user.c | 81 +++--
drivers/net/virtio/virtio_user/virtio_user_dev.c | 49 ++-
drivers/net/virtio/virtio_user/virtio_user_dev.h | 3 +-
drivers/net/virtio/virtio_user_ethdev.c | 4 +-
8 files changed, 708 insertions(+), 98 deletions(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost.c
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
--
2.7.4
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH 1/3] net/virtio_user: add vhost layer
2016-12-02 14:31 [PATCH 0/3] virtio_user as an alternative exception path Jianfeng Tan
@ 2016-12-02 14:31 ` Jianfeng Tan
2016-12-08 7:21 ` Yuanhan Liu
2016-12-02 14:31 ` [PATCH 2/3] net/virtio_user: add vhost kernel support Jianfeng Tan
` (5 subsequent siblings)
6 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-02 14:31 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
To support vhost kernel as the backend of net_virtio_user in comming
patches, we abstract a vhost layer to hide all vhost_user functions.
- Move vhost_user specific structs and macros into vhost_user.c,
and only keep common definitions in vhost.h;
- Add a struct vhost_internal, and an array to store vhost_user and
vhost_kernel backends; in multiqueue case, vhost_user has only
one vhost FD, but vhost_kernel would have multiple FDs (equal to
# of queues), so we turn to use an id to index the backend info;
- Add a struct vhost_ops depending on different type of backends.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/Makefile | 1 +
drivers/net/virtio/virtio_user/vhost.c | 162 +++++++++++++++++++++++
drivers/net/virtio/virtio_user/vhost.h | 68 +++++-----
drivers/net/virtio/virtio_user/vhost_user.c | 81 ++++++++----
drivers/net/virtio/virtio_user/virtio_user_dev.c | 47 +++----
drivers/net/virtio/virtio_user/virtio_user_dev.h | 2 +-
6 files changed, 266 insertions(+), 95 deletions(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost.c
diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile
index 97972a6..17f7129 100644
--- a/drivers/net/virtio/Makefile
+++ b/drivers/net/virtio/Makefile
@@ -59,6 +59,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_rxtx_simple_neon.c
endif
ifeq ($(CONFIG_RTE_VIRTIO_USER),y)
+SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_user.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/virtio_user_dev.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user_ethdev.c
diff --git a/drivers/net/virtio/virtio_user/vhost.c b/drivers/net/virtio/virtio_user/vhost.c
new file mode 100644
index 0000000..09e2e92
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost.c
@@ -0,0 +1,162 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include "vhost.h"
+
+#define VHOST_MAX_DEVICES 8
+
+static struct vhost_internal internals[VHOST_MAX_DEVICES];
+
+static inline int
+vhost_internal_valid_vid(int vid)
+{
+ if (vid < 0 || vid >= VHOST_MAX_DEVICES)
+ return 0;
+
+ if (!internals[vid].ops)
+ return 0;
+
+ return 1;
+}
+
+static int
+vhost_internal_alloc(void)
+{
+ int i;
+
+ for (i = 0; i < VHOST_MAX_DEVICES; ++i)
+ if (internals[i].ops == NULL)
+ break;
+
+ if (i >= VHOST_MAX_DEVICES)
+ return -1;
+
+ internals[i].features = 0;
+
+ internals[i].vhostfd = -1;
+
+ return 0;
+}
+
+static void
+vhost_internal_free(int id)
+{
+ internals[id].ops = NULL;
+
+ if (internals[id].vhostfd >= 0)
+ close(internals[id].vhostfd);
+}
+
+
+static int
+is_vhost_user_by_type(const char *path)
+{
+ struct stat sb;
+
+ if (stat(path, &sb) == -1)
+ return 0;
+
+ return S_ISSOCK(sb.st_mode);
+}
+
+int
+vhost_setup(const char *path)
+{
+ int ret = -1;
+ int vid = vhost_internal_alloc();
+
+ if (vid < 0) {
+ PMD_DRV_LOG(ERR, "exceeding %d devices", VHOST_MAX_DEVICES);
+ return -1;
+ }
+
+ if (is_vhost_user_by_type(path)) {
+ ret = vhost_ops_user.setup(&internals[vid], path);
+ internals[vid].ops = &vhost_ops_user;
+ }
+
+ if (ret < 0)
+ vhost_internal_free(vid);
+
+ return vid;
+}
+
+static const char * const vhost_msg_strings[] = {
+ [VHOST_USER_SET_OWNER] = "VHOST_USER_SET_OWNER",
+ [VHOST_USER_RESET_OWNER] = "VHOST_USER_RESET_OWNER",
+ [VHOST_USER_SET_FEATURES] = "VHOST_USER_SET_FEATURES",
+ [VHOST_USER_GET_FEATURES] = "VHOST_USER_GET_FEATURES",
+ [VHOST_USER_SET_VRING_CALL] = "VHOST_USER_SET_VRING_CALL",
+ [VHOST_USER_SET_VRING_NUM] = "VHOST_USER_SET_VRING_NUM",
+ [VHOST_USER_SET_VRING_BASE] = "VHOST_USER_SET_VRING_BASE",
+ [VHOST_USER_GET_VRING_BASE] = "VHOST_USER_GET_VRING_BASE",
+ [VHOST_USER_SET_VRING_ADDR] = "VHOST_USER_SET_VRING_ADDR",
+ [VHOST_USER_SET_VRING_KICK] = "VHOST_USER_SET_VRING_KICK",
+ [VHOST_USER_SET_MEM_TABLE] = "VHOST_USER_SET_MEM_TABLE",
+ [VHOST_USER_SET_VRING_ENABLE] = "VHOST_USER_SET_VRING_ENABLE",
+ NULL,
+};
+
+int
+vhost_call(int vid, enum vhost_user_request req, void *arg)
+{
+ RTE_SET_USED(vhost_msg_strings);
+ PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
+
+ if (!vhost_internal_valid_vid(vid))
+ return -1;
+
+ return internals[vid].ops->control(&internals[vid], req, arg);
+}
+
+int
+vhost_enable_queue_pair(int vid, uint16_t pair_idx, int enable)
+{
+ if (!vhost_internal_valid_vid(vid))
+ return -1;
+
+ return internals[vid].ops->enable_qp(&internals[vid], pair_idx, enable);
+}
+
+void
+vhost_close(int vid)
+{
+ if (!vhost_internal_valid_vid(vid))
+ return;
+
+ vhost_internal_free(vid);
+}
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 7adb55f..b476ecc 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -42,8 +42,6 @@
#include "../virtio_logs.h"
#include "../virtqueue.h"
-#define VHOST_MEMORY_MAX_NREGIONS 8
-
struct vhost_vring_state {
unsigned int index;
unsigned int num;
@@ -105,42 +103,38 @@ struct vhost_memory_region {
uint64_t mmap_offset;
};
-struct vhost_memory {
- uint32_t nregions;
- uint32_t padding;
- struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
+struct vhost_ops;
+
+struct vhost_internal {
+ /* for both vhost-user and vhost-kernel */
+ uint64_t features;
+ struct vhost_ops *ops;
+
+ /* for vhost-user */
+ int vhostfd;
+
+ /* for vhost-kernel */
+};
+
+typedef int (*vhost_setup_t)(struct vhost_internal *internal,
+ const char *path);
+typedef int (*vhost_control_t)(struct vhost_internal *internal,
+ enum vhost_user_request req,
+ void *arg);
+typedef int (*vhost_enable_qp_t)(struct vhost_internal *internal,
+ uint16_t pair_idx,
+ int enable);
+
+struct vhost_ops {
+ vhost_setup_t setup;
+ vhost_control_t control;
+ vhost_enable_qp_t enable_qp;
};
-struct vhost_user_msg {
- enum vhost_user_request request;
-
-#define VHOST_USER_VERSION_MASK 0x3
-#define VHOST_USER_REPLY_MASK (0x1 << 2)
- uint32_t flags;
- uint32_t size; /* the following payload size */
- union {
-#define VHOST_USER_VRING_IDX_MASK 0xff
-#define VHOST_USER_VRING_NOFD_MASK (0x1 << 8)
- uint64_t u64;
- struct vhost_vring_state state;
- struct vhost_vring_addr addr;
- struct vhost_memory memory;
- } payload;
- int fds[VHOST_MEMORY_MAX_NREGIONS];
-} __attribute((packed));
-
-#define VHOST_USER_HDR_SIZE offsetof(struct vhost_user_msg, payload.u64)
-#define VHOST_USER_PAYLOAD_SIZE \
- (sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
-
-/* The version of the protocol we support */
-#define VHOST_USER_VERSION 0x1
-
-#define VHOST_USER_F_PROTOCOL_FEATURES 30
-#define VHOST_USER_MQ (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)
-
-int vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg);
-int vhost_user_setup(const char *path);
-int vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable);
+struct vhost_ops vhost_ops_user;
+int vhost_setup(const char *path);
+int vhost_call(int vid, enum vhost_user_request req, void *arg);
+int vhost_enable_queue_pair(int vid, uint16_t pair_idx, int enable);
+void vhost_close(int vid);
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 082e821..afd18ec 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -42,6 +42,38 @@
#include "vhost.h"
+/* The version of the protocol we support */
+#define VHOST_USER_VERSION 0x1
+#define VHOST_MEMORY_MAX_NREGIONS 8
+
+struct vhost_memory {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
+};
+
+struct vhost_user_msg {
+ enum vhost_user_request request;
+
+#define VHOST_USER_VERSION_MASK 0x3
+#define VHOST_USER_REPLY_MASK (0x1 << 2)
+ uint32_t flags;
+ uint32_t size; /* the following payload size */
+ union {
+#define VHOST_USER_VRING_IDX_MASK 0xff
+#define VHOST_USER_VRING_NOFD_MASK (0x1 << 8)
+ uint64_t u64;
+ struct vhost_vring_state state;
+ struct vhost_vring_addr addr;
+ struct vhost_memory memory;
+ } payload;
+ int fds[VHOST_MEMORY_MAX_NREGIONS];
+} __attribute((packed));
+
+#define VHOST_USER_HDR_SIZE offsetof(struct vhost_user_msg, payload.u64)
+#define VHOST_USER_PAYLOAD_SIZE \
+ (sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
+
static int
vhost_user_write(int fd, void *buf, int len, int *fds, int fd_num)
{
@@ -223,24 +255,10 @@ prepare_vhost_memory_user(struct vhost_user_msg *msg, int fds[])
static struct vhost_user_msg m;
-static const char * const vhost_msg_strings[] = {
- [VHOST_USER_SET_OWNER] = "VHOST_USER_SET_OWNER",
- [VHOST_USER_RESET_OWNER] = "VHOST_USER_RESET_OWNER",
- [VHOST_USER_SET_FEATURES] = "VHOST_USER_SET_FEATURES",
- [VHOST_USER_GET_FEATURES] = "VHOST_USER_GET_FEATURES",
- [VHOST_USER_SET_VRING_CALL] = "VHOST_USER_SET_VRING_CALL",
- [VHOST_USER_SET_VRING_NUM] = "VHOST_USER_SET_VRING_NUM",
- [VHOST_USER_SET_VRING_BASE] = "VHOST_USER_SET_VRING_BASE",
- [VHOST_USER_GET_VRING_BASE] = "VHOST_USER_GET_VRING_BASE",
- [VHOST_USER_SET_VRING_ADDR] = "VHOST_USER_SET_VRING_ADDR",
- [VHOST_USER_SET_VRING_KICK] = "VHOST_USER_SET_VRING_KICK",
- [VHOST_USER_SET_MEM_TABLE] = "VHOST_USER_SET_MEM_TABLE",
- [VHOST_USER_SET_VRING_ENABLE] = "VHOST_USER_SET_VRING_ENABLE",
- NULL,
-};
-
-int
-vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
+static int
+vhost_user_sock(struct vhost_internal *internal,
+ enum vhost_user_request req,
+ void *arg)
{
struct vhost_user_msg msg;
struct vhost_vring_file *file = 0;
@@ -248,11 +266,9 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
int fds[VHOST_MEMORY_MAX_NREGIONS];
int fd_num = 0;
int i, len;
+ int vhostfd = internal->vhostfd;
RTE_SET_USED(m);
- RTE_SET_USED(vhost_msg_strings);
-
- PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
msg.request = req;
msg.flags = VHOST_USER_VERSION;
@@ -323,8 +339,8 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
len = VHOST_USER_HDR_SIZE + msg.size;
if (vhost_user_write(vhostfd, &msg, len, fds, fd_num) < 0) {
- PMD_DRV_LOG(ERR, "%s failed: %s",
- vhost_msg_strings[req], strerror(errno));
+ PMD_DRV_LOG(ERR, "fail to write vhost user msg: %s",
+ strerror(errno));
return -1;
}
@@ -378,8 +394,8 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
* - (-1) if fail to set up;
* - (>=0) if successful, and it is the fd to vhostfd.
*/
-int
-vhost_user_setup(const char *path)
+static int
+vhost_user_setup(struct vhost_internal *internal, const char *path)
{
int fd;
int flag;
@@ -404,11 +420,14 @@ vhost_user_setup(const char *path)
return -1;
}
+ internal->vhostfd = fd;
+
return fd;
}
-int
-vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
+static int
+vhost_user_enable_queue_pair(struct vhost_internal *internal,
+ uint16_t pair_idx, int enable)
{
int i;
@@ -418,10 +437,16 @@ vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
.num = enable,
};
- if (vhost_user_sock(vhostfd,
+ if (vhost_user_sock(internal,
VHOST_USER_SET_VRING_ENABLE, &state))
return -1;
}
return 0;
}
+
+struct vhost_ops vhost_ops_user = {
+ .setup = vhost_user_setup,
+ .control = vhost_user_sock,
+ .enable_qp = vhost_user_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index e239e0e..3aef5f6 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -64,7 +64,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = callfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_CALL, &file);
+ vhost_call(dev->vid, VHOST_USER_SET_VRING_CALL, &file);
dev->callfds[queue_sel] = callfd;
return 0;
@@ -88,12 +88,12 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.index = queue_sel;
state.num = vring->num;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_NUM, &state);
+ vhost_call(dev->vid, VHOST_USER_SET_VRING_NUM, &state);
state.num = 0; /* no reservation */
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_BASE, &state);
+ vhost_call(dev->vid, VHOST_USER_SET_VRING_BASE, &state);
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_ADDR, &addr);
+ vhost_call(dev->vid, VHOST_USER_SET_VRING_ADDR, &addr);
/* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
* lastly because vhost depends on this msg to judge if
@@ -106,7 +106,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = kickfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_KICK, &file);
+ vhost_call(dev->vid, VHOST_USER_SET_VRING_KICK, &file);
dev->kickfds[queue_sel] = kickfd;
return 0;
@@ -146,21 +146,18 @@ virtio_user_start_device(struct virtio_user_dev *dev)
if (virtio_user_queue_setup(dev, virtio_user_create_queue) < 0)
goto error;
- /* Step 1: set features
- * Make sure VHOST_USER_F_PROTOCOL_FEATURES is added if mq is enabled,
- * and VIRTIO_NET_F_MAC is stripped.
+ /* Step 1: set feature
+ * Strip VIRTIO_NET_F_MAC, as MAC address is handled in vdev init.
*/
features = dev->features;
- if (dev->max_queue_pairs > 1)
- features |= VHOST_USER_MQ;
features &= ~(1ull << VIRTIO_NET_F_MAC);
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
+ ret = vhost_call(dev->vid, VHOST_USER_SET_FEATURES, &features);
if (ret < 0)
goto error;
PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
/* Step 2: share memory regions */
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_MEM_TABLE, NULL);
+ ret = vhost_call(dev->vid, VHOST_USER_SET_MEM_TABLE, NULL);
if (ret < 0)
goto error;
@@ -171,7 +168,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
/* Step 4: enable queues
* we enable the 1st queue pair by default.
*/
- vhost_user_enable_queue_pair(dev->vhostfd, 0, 1);
+ vhost_enable_queue_pair(dev->vid, 0, 1);
return 0;
error:
@@ -181,7 +178,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
int virtio_user_stop_device(struct virtio_user_dev *dev)
{
- return vhost_user_sock(dev->vhostfd, VHOST_USER_RESET_OWNER, NULL);
+ return vhost_call(dev->vid, VHOST_USER_RESET_OWNER, NULL);
}
static inline void
@@ -215,19 +212,18 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
dev->queue_size = queue_size;
dev->mac_specified = 0;
parse_mac(dev, mac);
- dev->vhostfd = -1;
- dev->vhostfd = vhost_user_setup(dev->path);
- if (dev->vhostfd < 0) {
+ dev->vid = vhost_setup(dev->path);
+ if (dev->vid < 0) {
PMD_INIT_LOG(ERR, "backend set up fails");
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_SET_OWNER, NULL) < 0) {
+ if (vhost_call(dev->vid, VHOST_USER_SET_OWNER, NULL) < 0) {
PMD_INIT_LOG(ERR, "set_owner fails: %s", strerror(errno));
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_GET_FEATURES,
+ if (vhost_call(dev->vid, VHOST_USER_GET_FEATURES,
&dev->features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
@@ -253,13 +249,6 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
PMD_INIT_LOG(INFO, "vhost does not support ctrl-q");
}
- if (dev->max_queue_pairs > 1) {
- if (!(dev->features & VHOST_USER_MQ)) {
- PMD_INIT_LOG(ERR, "MQ not supported by the backend");
- return -1;
- }
- }
-
return 0;
}
@@ -273,7 +262,7 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
close(dev->kickfds[i]);
}
- close(dev->vhostfd);
+ vhost_close(dev->vid);
}
static uint8_t
@@ -289,9 +278,9 @@ virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs)
}
for (i = 0; i < q_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 1);
+ ret |= vhost_enable_queue_pair(dev->vid, i, 1);
for (i = q_pairs; i < dev->max_queue_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+ ret |= vhost_enable_queue_pair(dev->vid, i, 0);
dev->queue_pairs = q_pairs;
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 33690b5..80efb6e 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -39,7 +39,7 @@
#include "../virtio_ring.h"
struct virtio_user_dev {
- int vhostfd;
+ int vid;
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int kickfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int mac_specified;
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH 2/3] net/virtio_user: add vhost kernel support
2016-12-02 14:31 [PATCH 0/3] virtio_user as an alternative exception path Jianfeng Tan
2016-12-02 14:31 ` [PATCH 1/3] net/virtio_user: add vhost layer Jianfeng Tan
@ 2016-12-02 14:31 ` Jianfeng Tan
2016-12-02 14:31 ` [PATCH 3/3] net/virtio_user: fix wrongly set features Jianfeng Tan
` (4 subsequent siblings)
6 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-02 14:31 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
This patch add support vhost kernel support under vhost abstraction
layer.
Three main hook functions are added:
- vhost_kernel_setup() to open char device, each vq pair (Rx and Tx)
would need open it once;
- vhost_kernel_ioctl() to communicate control messages with vhost
kernel module;
- vhost_kernel_enable_queue_pair() to open tap device and set it
as the backend of corresonding vhost fd (that is to say, vq pair).
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/Makefile | 1 +
drivers/net/virtio/virtio_user/vhost.c | 21 +-
drivers/net/virtio/virtio_user/vhost.h | 5 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 413 ++++++++++++++++++++++++++
4 files changed, 439 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile
index 17f7129..f671f1f 100644
--- a/drivers/net/virtio/Makefile
+++ b/drivers/net/virtio/Makefile
@@ -61,6 +61,7 @@ endif
ifeq ($(CONFIG_RTE_VIRTIO_USER),y)
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_user.c
+SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_kernel.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/virtio_user_dev.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user_ethdev.c
endif
diff --git a/drivers/net/virtio/virtio_user/vhost.c b/drivers/net/virtio/virtio_user/vhost.c
index 09e2e92..5384992 100644
--- a/drivers/net/virtio/virtio_user/vhost.c
+++ b/drivers/net/virtio/virtio_user/vhost.c
@@ -56,7 +56,7 @@ vhost_internal_valid_vid(int vid)
static int
vhost_internal_alloc(void)
{
- int i;
+ int i, q;
for (i = 0; i < VHOST_MAX_DEVICES; ++i)
if (internals[i].ops == NULL)
@@ -69,16 +69,32 @@ vhost_internal_alloc(void)
internals[i].vhostfd = -1;
+ for (q = 0; q < VHOST_KERNEL_MAX_QUEUES; ++q) {
+ internals[i].vhostfds[q] = -1;
+ internals[i].tapfds[q] = -1;
+ }
+
return 0;
}
static void
vhost_internal_free(int id)
{
+ int q;
+
internals[id].ops = NULL;
if (internals[id].vhostfd >= 0)
close(internals[id].vhostfd);
+
+ for (q = 0; q < VHOST_KERNEL_MAX_QUEUES; ++q) {
+ if (internals[id].vhostfds[q] >= 0)
+ close(internals[id].vhostfds[q]);
+ if (internals[id].tapfds[q] >= 0)
+ close(internals[id].tapfds[q]);
+ }
+ if (internals[id].ifname)
+ free(internals[id].ifname);
}
@@ -107,6 +123,9 @@ vhost_setup(const char *path)
if (is_vhost_user_by_type(path)) {
ret = vhost_ops_user.setup(&internals[vid], path);
internals[vid].ops = &vhost_ops_user;
+ } else {
+ ret = vhost_ops_kernel.setup(&internals[vid], path);
+ internals[vid].ops = &vhost_ops_kernel;
}
if (ret < 0)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index b476ecc..b6fd092 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -114,6 +114,10 @@ struct vhost_internal {
int vhostfd;
/* for vhost-kernel */
+ char *ifname;
+#define VHOST_KERNEL_MAX_QUEUES 8
+ int vhostfds[VHOST_KERNEL_MAX_QUEUES];
+ int tapfds[VHOST_KERNEL_MAX_QUEUES];
};
typedef int (*vhost_setup_t)(struct vhost_internal *internal,
@@ -132,6 +136,7 @@ struct vhost_ops {
};
struct vhost_ops vhost_ops_user;
+struct vhost_ops vhost_ops_kernel;
int vhost_setup(const char *path);
int vhost_call(int vid, enum vhost_user_request req, void *arg);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
new file mode 100644
index 0000000..9a9e8bd
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -0,0 +1,413 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <net/if.h>
+#include <string.h>
+#include <errno.h>
+
+#include <rte_memory.h>
+#include <rte_eal_memconfig.h>
+
+#include "vhost.h"
+
+struct vhost_memory_kernel {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[0];
+};
+
+/* vhost kernel ioctls */
+#define VHOST_VIRTIO 0xAF
+#define VHOST_GET_FEATURES _IOR(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_FEATURES _IOW(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
+#define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
+#define VHOST_SET_MEM_TABLE _IOW(VHOST_VIRTIO, 0x03, struct vhost_memory_kernel)
+#define VHOST_SET_LOG_BASE _IOW(VHOST_VIRTIO, 0x04, __u64)
+#define VHOST_SET_LOG_FD _IOW(VHOST_VIRTIO, 0x07, int)
+#define VHOST_SET_VRING_NUM _IOW(VHOST_VIRTIO, 0x10, struct vhost_vring_state)
+#define VHOST_SET_VRING_ADDR _IOW(VHOST_VIRTIO, 0x11, struct vhost_vring_addr)
+#define VHOST_SET_VRING_BASE _IOW(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_GET_VRING_BASE _IOWR(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_SET_VRING_KICK _IOW(VHOST_VIRTIO, 0x20, struct vhost_vring_file)
+#define VHOST_SET_VRING_CALL _IOW(VHOST_VIRTIO, 0x21, struct vhost_vring_file)
+#define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
+#define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct vhost_vring_file)
+
+/* TUN ioctls */
+#define TUNSETIFF _IOW('T', 202, int)
+#define TUNGETFEATURES _IOR('T', 207, unsigned int)
+#define TUNSETOFFLOAD _IOW('T', 208, unsigned int)
+#define TUNGETIFF _IOR('T', 210, unsigned int)
+#define TUNSETSNDBUF _IOW('T', 212, int)
+#define TUNGETVNETHDRSZ _IOR('T', 215, int)
+#define TUNSETVNETHDRSZ _IOW('T', 216, int)
+#define TUNSETQUEUE _IOW('T', 217, int)
+#define TUNSETVNETLE _IOW('T', 220, int)
+#define TUNSETVNETBE _IOW('T', 222, int)
+
+/* TUNSETIFF ifr flags */
+#define IFF_TAP 0x0002
+#define IFF_NO_PI 0x1000
+#define IFF_ONE_QUEUE 0x2000
+#define IFF_VNET_HDR 0x4000
+#define IFF_MULTI_QUEUE 0x0100
+#define IFF_ATTACH_QUEUE 0x0200
+#define IFF_DETACH_QUEUE 0x0400
+
+/* Features for GSO (TUNSETOFFLOAD). */
+#define TUN_F_CSUM 0x01 /* You can hand me unchecksummed packets. */
+#define TUN_F_TSO4 0x02 /* I can handle TSO for IPv4 packets */
+#define TUN_F_TSO6 0x04 /* I can handle TSO for IPv6 packets */
+#define TUN_F_TSO_ECN 0x08 /* I can handle TSO with ECN bits. */
+#define TUN_F_UFO 0x10 /* I can handle UFO packets */
+
+/* Constants */
+#define TUN_DEF_SNDBUF (1ull << 20)
+#define PATH_NET_TUN "/dev/net/tun"
+#define VHOST_KERNEL_MAX_REGIONS 64
+
+static uint64_t vhost_req_user_to_kernel[] = {
+ [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
+ [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
+ [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
+ [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
+ [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
+ [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
+ [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
+ [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
+ [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
+ [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
+ [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
+};
+
+/* By default, vhost kernel module allows 64 regions, but DPDK allows
+ * 256 segments. As a relief, below function merges those virtually
+ * adjacent memsegs into one region.
+ */
+static struct vhost_memory_kernel *
+prepare_vhost_memory_kernel(void)
+{
+ uint32_t i, j, k = 0;
+ struct rte_memseg *seg;
+ struct vhost_memory_region *mr;
+ struct vhost_memory_kernel *vm;
+
+ vm = malloc(sizeof(struct vhost_memory_kernel) +
+ VHOST_KERNEL_MAX_REGIONS *
+ sizeof(struct vhost_memory_region));
+
+ for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
+ seg = &rte_eal_get_configuration()->mem_config->memseg[i];
+ if (!seg->addr)
+ break;
+
+ int new_region = 1;
+
+ for (j = 0; j < k; ++j) {
+ mr = &vm->regions[j];
+
+ if (mr->userspace_addr + mr->memory_size ==
+ (uint64_t)seg->addr) {
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+
+ if ((uint64_t)seg->addr + seg->len ==
+ mr->userspace_addr) {
+ mr->guest_phys_addr = (uint64_t)seg->addr;
+ mr->userspace_addr = (uint64_t)seg->addr;
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+ }
+
+ if (new_region == 0)
+ continue;
+
+ mr = &vm->regions[k++];
+ mr->guest_phys_addr = (uint64_t)seg->addr; /* use vaddr here! */
+ mr->userspace_addr = (uint64_t)seg->addr;
+ mr->memory_size = seg->len;
+ mr->mmap_offset = 0;
+
+ if (k >= VHOST_KERNEL_MAX_REGIONS) {
+ free(vm);
+ return NULL;
+ }
+ }
+
+ vm->nregions = k;
+ vm->padding = 0;
+ return vm;
+}
+
+static const uint64_t guest_offloads_mask =
+ (1ULL << VIRTIO_NET_F_GUEST_CSUM) |
+ (1ULL << VIRTIO_NET_F_GUEST_TSO4) |
+ (1ULL << VIRTIO_NET_F_GUEST_TSO6) |
+ (1ULL << VIRTIO_NET_F_GUEST_ECN) |
+ (1ULL << VIRTIO_NET_F_GUEST_UFO);
+
+static int
+vhost_kernel_ioctl(struct vhost_internal *internal,
+ enum vhost_user_request req,
+ void *arg)
+{
+ int i, ret = -1;
+ uint64_t req_kernel;
+ struct vhost_memory_kernel *vm = NULL;
+
+ req_kernel = vhost_req_user_to_kernel[req];
+
+ if (req_kernel == VHOST_SET_MEM_TABLE) {
+ vm = prepare_vhost_memory_kernel();
+ if (!vm)
+ return -1;
+ arg = (void *)vm;
+ }
+
+ /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
+ if (req_kernel == VHOST_SET_FEATURES)
+ *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+
+ for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
+ if (internal->vhostfds[i] < 0)
+ continue;
+
+ ret = ioctl(internal->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+
+ if (!ret && req_kernel == VHOST_SET_FEATURES)
+ internal->features = *((uint64_t *)arg);
+
+ /* with tap as the backend, all these features are supported but not
+ * claimed by vhost-net, so we add them back when reporting to upper
+ * layer
+ */
+ if (!ret && req_kernel == VHOST_GET_FEATURES)
+ *((uint64_t *)arg) |= guest_offloads_mask;
+
+ if (vm)
+ free(vm);
+
+ return ret;
+}
+
+/**
+ * Set up environment to talk with a vhost kernel backend.
+ * @param path
+ * - The path to vhost net (kernel) character file.
+ *
+ * @return
+ * - (-1) if fail to set up;
+ * - (>=0) if successful.
+ */
+static int
+vhost_kernel_setup(struct vhost_internal *internal, const char *path)
+{
+ int vhostfd;
+ uint32_t q;
+ uint32_t queue_pairs = 1;
+
+ for (q = 0; q < queue_pairs; ++q) {
+ vhostfd = open(path, O_RDWR);
+ if (vhostfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s, %s",
+ path, strerror(errno));
+ return -1;
+ }
+
+ internal->vhostfds[q] = vhostfd;
+ }
+
+ return 0;
+}
+
+static int
+_vhost_kernel_enable_queue_pair(int vhostfd, int tapfd)
+{
+ struct vhost_vring_file f;
+
+ f.fd = tapfd;
+ f.index = 0;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ f.index = 1;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_enable_queue_pair(struct vhost_internal *internal,
+ uint16_t pair_idx,
+ int enable)
+{
+ unsigned int offload =
+ TUN_F_CSUM |
+ TUN_F_TSO4 |
+ TUN_F_TSO6 |
+ TUN_F_TSO_ECN |
+ TUN_F_UFO;
+ unsigned int features;
+ int sndbuf = TUN_DEF_SNDBUF;
+ struct ifreq ifr;
+ int hdr_size;
+ int vhostfd;
+ int tapfd;
+ int req_mq;
+
+
+ vhostfd = internal->vhostfds[pair_idx];
+
+ if (!enable) {
+ if (internal->tapfds[pair_idx]) {
+ close(internal->tapfds[pair_idx]);
+ internal->tapfds[pair_idx] = -1;
+ }
+ return _vhost_kernel_enable_queue_pair(vhostfd, -1);
+ }
+
+ if ((internal->features & (1ULL << VIRTIO_NET_F_MRG_RXBUF)) ||
+ (internal->features & (1ULL << VIRTIO_F_VERSION_1)))
+ hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
+ else
+ hdr_size = sizeof(struct virtio_net_hdr);
+
+ req_mq = !!(internal->features & (1ULL << VIRTIO_NET_F_MQ));
+
+ /* TODO:
+ * 1. get/set offload capability, tap_probe_has_ufo, tap_fd_set_offload
+ * 2. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
+ * 3. get number of memory regions from vhost module parameter
+ * max_mem_regions, supported in newer version linux kernel
+ */
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ /* Construct ifr */
+ memset(&ifr, 0, sizeof(ifr));
+ ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
+
+ if (ioctl(tapfd, TUNGETFEATURES, &features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ goto error;
+ }
+ if (features & IFF_ONE_QUEUE)
+ ifr.ifr_flags |= IFF_ONE_QUEUE;
+
+ /* Let tap instead of vhost-net handle vnet header, as the latter does
+ * not support offloading. And in this case, we should not set feature
+ * bit VHOST_NET_F_VIRTIO_NET_HDR.
+ */
+ if (features & IFF_VNET_HDR) {
+ ifr.ifr_flags |= IFF_VNET_HDR;
+ } else {
+ PMD_DRV_LOG(ERR, "TAP does not support IFF_VNET_HDR");
+ goto error;
+ }
+
+ if (req_mq) {
+ if (features & IFF_MULTI_QUEUE)
+ ifr.ifr_flags |= IFF_MULTI_QUEUE;
+ else {
+ PMD_DRV_LOG(ERR, "multiqueue not supported by kernel");
+ goto error;
+ }
+ }
+
+ if (internal->ifname)
+ strncpy(ifr.ifr_name, internal->ifname, IFNAMSIZ);
+ else
+ strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
+ if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
+ PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ fcntl(tapfd, F_SETFL, O_NONBLOCK);
+
+ if (ioctl(tapfd, TUNSETVNETHDRSZ, &hdr_size) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETVNETHDRSZ failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (ioctl(tapfd, TUNSETSNDBUF, &sndbuf) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETSNDBUF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (_vhost_kernel_enable_queue_pair(vhostfd, tapfd) < 0)
+ goto error;
+
+ /* Only try best to set offload */
+ if (ioctl(tapfd, TUNSETOFFLOAD, offload) != 0)
+ PMD_DRV_LOG(ERR, "TUNSETOFFLOAD ioctl() failed: %s",
+ strerror(errno));
+
+ internal->tapfds[pair_idx] = tapfd;
+ if (!internal->ifname)
+ internal->ifname = strdup(ifr.ifr_name);
+
+ return 0;
+error:
+ return -1;
+}
+
+struct vhost_ops vhost_ops_kernel = {
+ .setup = vhost_kernel_setup,
+ .control = vhost_kernel_ioctl,
+ .enable_qp = vhost_kernel_enable_queue_pair
+};
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH 3/3] net/virtio_user: fix wrongly set features
2016-12-02 14:31 [PATCH 0/3] virtio_user as an alternative exception path Jianfeng Tan
2016-12-02 14:31 ` [PATCH 1/3] net/virtio_user: add vhost layer Jianfeng Tan
2016-12-02 14:31 ` [PATCH 2/3] net/virtio_user: add vhost kernel support Jianfeng Tan
@ 2016-12-02 14:31 ` Jianfeng Tan
2016-12-08 8:32 ` Yuanhan Liu
2016-12-02 14:44 ` [PATCH 0/3] virtio_user as an alternative exception path Thomas Monjalon
` (3 subsequent siblings)
6 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-02 14:31 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
Before the commit 86d59b21468a ("net/virtio: support LRO"), features
in virtio PMD, is decided and properly set at device initialization
and will not be changed. But afterward, features could be changed in
virtio_dev_configure(), and will be re-negotiated if it's changed.
In virtio_user, host features is obtained at device initialization
only once, but we did not store it. So the added feature bits in
re-negotiation will fail.
Fixes: e9efa4d93821 ("net/virtio-user: add new virtual PCI driver")
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 2 +-
drivers/net/virtio/virtio_user/virtio_user_dev.h | 1 +
drivers/net/virtio/virtio_user_ethdev.c | 4 ++--
3 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 3aef5f6..a9157a9 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -224,7 +224,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
}
if (vhost_call(dev->vid, VHOST_USER_GET_FEATURES,
- &dev->features) < 0) {
+ &dev->host_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
}
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 80efb6e..d219432 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -47,6 +47,7 @@ struct virtio_user_dev {
uint32_t queue_pairs;
uint32_t queue_size;
uint64_t features;
+ uint64_t host_features;
uint8_t status;
uint8_t mac_addr[ETHER_ADDR_LEN];
char path[PATH_MAX];
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 406beea..cfe2bfc 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -117,7 +117,7 @@ virtio_user_get_features(struct virtio_hw *hw)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- return dev->features;
+ return dev->host_features;
}
static void
@@ -125,7 +125,7 @@ virtio_user_set_features(struct virtio_hw *hw, uint64_t features)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- dev->features = features;
+ dev->features = features & dev->host_features;
}
static uint8_t
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: [PATCH 0/3] virtio_user as an alternative exception path
2016-12-02 14:31 [PATCH 0/3] virtio_user as an alternative exception path Jianfeng Tan
` (2 preceding siblings ...)
2016-12-02 14:31 ` [PATCH 3/3] net/virtio_user: fix wrongly set features Jianfeng Tan
@ 2016-12-02 14:44 ` Thomas Monjalon
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
` (2 subsequent siblings)
6 siblings, 0 replies; 73+ messages in thread
From: Thomas Monjalon @ 2016-12-02 14:44 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, yuanhan.liu, ferruh.yigit, cunming.liang
2016-12-02 14:31, Jianfeng Tan:
> In v16.07, we upstreamed a virtual device, virtio_user (with vhost-user
> as the backend). The path to go with a vhost-kernel backend has been
> dropped for bad performance comparing to vhost-user and code simplicity.
>
> But after a second thought, virtio_user + vhost-kernel is a good
> candidate as an exceptional path, such as KNI, which exchanges packets
> with kernel networking stack.
> - maintenance: vhost-net (kernel) is upstreamed and extensively used
> kernel module. We don't need any out-of-tree module like KNI.
> - performance: as with KNI, this solution would use one or more
> kthreads to send/receive packets from user space DPDK applications,
> which has little impact on user space polling thread (except that
> it might enter into kernel space to wake up those kthreads if
> necessary.
> - features: vhost-net is born to be a networking solution, which has
> lots of networking related featuers, like multi queue, tso, multi-seg
> mbuf, etc.
That's a really interesting trial.
Have I already said that I don't like KNI? ;)
> Known issues for current version:
> - Multiqueue not supported yet.
> - Offloading is completely enabled yet; to enhance, we will translate
Is a "not" missing in this sentence?
> the virtio header info into mbuf metadata when receiving packets
> from kernel; and translate mbuf metadata info into virtio header
> when sending packets to kernel.
So what is the ambition for 17.02?
Do you think it could be integrated?
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH 1/3] net/virtio_user: add vhost layer
2016-12-02 14:31 ` [PATCH 1/3] net/virtio_user: add vhost layer Jianfeng Tan
@ 2016-12-08 7:21 ` Yuanhan Liu
0 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-08 7:21 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Dec 02, 2016 at 02:31:13PM +0000, Jianfeng Tan wrote:
> To support vhost kernel as the backend of net_virtio_user in comming
> patches, we abstract a vhost layer to hide all vhost_user functions.
> - Move vhost_user specific structs and macros into vhost_user.c,
> and only keep common definitions in vhost.h;
Do that in another patch.
> - Add a struct vhost_internal, and an array to store vhost_user and
> vhost_kernel backends; in multiqueue case, vhost_user has only
> one vhost FD, but vhost_kernel would have multiple FDs (equal to
> # of queues), so we turn to use an id to index the backend info;
It's okay to add a struct for that, but what's the point of referencing
it by an id? Instead, you could directly allocate one and return it.
This would also remove the hardcoded limit: 8 devices at most.
> - Add a struct vhost_ops depending on different type of backends.
Firstly, the "control" method is not well named, IMO. Something like
"send_request" is better.
And there is another way to achieve that: instead of introducing one
callback (control, or send_request) for all requests, you could
define one callback for each request. QEMU switched to that way
few releases ago. In that way, you don't have to do the request
translation at least.
Last, please make one patch for each of 3 items. It helps review.
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH 3/3] net/virtio_user: fix wrongly set features
2016-12-02 14:31 ` [PATCH 3/3] net/virtio_user: fix wrongly set features Jianfeng Tan
@ 2016-12-08 8:32 ` Yuanhan Liu
0 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-08 8:32 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Dec 02, 2016 at 02:31:15PM +0000, Jianfeng Tan wrote:
> Before the commit 86d59b21468a ("net/virtio: support LRO"), features
> in virtio PMD, is decided and properly set at device initialization
> and will not be changed. But afterward, features could be changed in
> virtio_dev_configure(), and will be re-negotiated if it's changed.
>
> In virtio_user, host features is obtained at device initialization
> only once, but we did not store it. So the added feature bits in
> re-negotiation will fail.
I think you misunderstood the features negotiation logic here, since
the beginning.
The feature negotiation is a work of 3 parts (the virtio device, the
virtio driver and the vhost). Each part has it's own feature bits
to claim what kind of features it supports.
On the init stage, the driver will have a sync with the device, and
that's what the "get_features" and "set_features" for. Once they have
come to an agreement on the negotiated features, it will have a
last sync with the vhost backend, through the GET_FEATURES and
SET_FEATURES request.
Last, we would have a features bits that would work for all of the
3 parts.
That said, you should not mix the driver's features bits with the
device one. It's Okay to introduce host_features, but it should not
be in the driver layer, it should be in the device layer. And naming
it to something like "device_features" is better, IMO.
And maybe you should re-visit the whole virtio_user features negotiation
logic, to see what can be improved.
Besides, I would put the bug fixing patch in the first of a series,
so that it has minimal chance to introduce conflics while backporting
it to a stable branch.
--yliu
>
> Fixes: e9efa4d93821 ("net/virtio-user: add new virtual PCI driver")
>
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> ---
> drivers/net/virtio/virtio_user/virtio_user_dev.c | 2 +-
> drivers/net/virtio/virtio_user/virtio_user_dev.h | 1 +
> drivers/net/virtio/virtio_user_ethdev.c | 4 ++--
> 3 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index 3aef5f6..a9157a9 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -224,7 +224,7 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
> }
>
> if (vhost_call(dev->vid, VHOST_USER_GET_FEATURES,
> - &dev->features) < 0) {
> + &dev->host_features) < 0) {
> PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
> return -1;
> }
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index 80efb6e..d219432 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -47,6 +47,7 @@ struct virtio_user_dev {
> uint32_t queue_pairs;
> uint32_t queue_size;
> uint64_t features;
> + uint64_t host_features;
> uint8_t status;
> uint8_t mac_addr[ETHER_ADDR_LEN];
> char path[PATH_MAX];
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
> index 406beea..cfe2bfc 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -117,7 +117,7 @@ virtio_user_get_features(struct virtio_hw *hw)
> {
> struct virtio_user_dev *dev = virtio_user_get_dev(hw);
>
> - return dev->features;
> + return dev->host_features;
> }
>
> static void
> @@ -125,7 +125,7 @@ virtio_user_set_features(struct virtio_hw *hw, uint64_t features)
> {
> struct virtio_user_dev *dev = virtio_user_get_dev(hw);
>
> - dev->features = features;
> + dev->features = features & dev->host_features;
> }
>
> static uint8_t
> --
> 2.7.4
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v2 0/7] virtio_user as an alternative exception path
2016-12-02 14:31 [PATCH 0/3] virtio_user as an alternative exception path Jianfeng Tan
` (3 preceding siblings ...)
2016-12-02 14:44 ` [PATCH 0/3] virtio_user as an alternative exception path Thomas Monjalon
@ 2016-12-23 7:14 ` Jianfeng Tan
2016-12-23 7:14 ` [PATCH v2 1/7] net/virtio_user: fix wrongly set features Jianfeng Tan
` (6 more replies)
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
6 siblings, 7 replies; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-23 7:14 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
v2: (Lots of them are from yuanhan's comment)
- Add offloding feature.
- Add multiqueue support.
- Add a new patch to postpone the sending of driver ok notification.
- Put fix patch ahead of the whole patch series.
- Split original 0001 patch into 0003 and 0004 patches.
- Remove the original vhost_internal design, just add those into
struct virtio_user_dev for simplicity.
- Reword "control" to "send_request".
- Reword "host_features" to "device_features".
In v16.07, we upstreamed a virtual device, virtio_user (with vhost-user
as the backend). The path to go with a vhost-kernel backend has been
dropped for bad performance comparing to vhost-user and code simplicity.
But after a second thought, virtio_user + vhost-kernel is a good
candidate as an exceptional path, such as KNI, which exchanges packets
with kernel networking stack.
- maintenance: vhost-net (kernel) is upstreamed and extensively used
kernel module. We don't need any out-of-tree module like KNI.
- performance: as with KNI, this solution would use one or more
kthreads to send/receive packets from user space DPDK applications,
which has little impact on user space polling thread (except that
it might enter into kernel space to wake up those kthreads if
necessary).
- features: vhost-net is born to be a networking solution, which has
lots of networking related featuers, like multi queue, tso, multi-seg
mbuf, etc.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Jianfeng Tan (7):
net/virtio_user: fix wrongly set features
net/virtio_user: postpone DRIVER OK notification
net/virtio_user: move vhost user specific code
net/virtio_user: abstract virtio user backend ops
net/virtio_user: add vhost kernel support
net/virtio_user: enable offloading
net/virtio_user: enable multiqueue with vhost kernel
drivers/net/virtio/Makefile | 1 +
drivers/net/virtio/virtio_ethdev.c | 3 +-
drivers/net/virtio/virtio_user/vhost.h | 51 +--
drivers/net/virtio/virtio_user/vhost_kernel.c | 476 +++++++++++++++++++++++
drivers/net/virtio/virtio_user/vhost_user.c | 86 ++--
drivers/net/virtio/virtio_user/virtio_user_dev.c | 122 +++---
drivers/net/virtio/virtio_user/virtio_user_dev.h | 16 +-
drivers/net/virtio/virtio_user_ethdev.c | 4 +-
8 files changed, 642 insertions(+), 117 deletions(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
--
2.7.4
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v2 1/7] net/virtio_user: fix wrongly set features
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
@ 2016-12-23 7:14 ` Jianfeng Tan
2016-12-23 7:14 ` [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification Jianfeng Tan
` (5 subsequent siblings)
6 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-23 7:14 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
Before the commit 86d59b21468a ("net/virtio: support LRO"), features
in virtio PMD, is decided and properly set at device initialization
and will not be changed. But afterward, features could be changed in
virtio_dev_configure(), and will be re-negotiated if it's changed.
In virtio_user, device features is obtained at driver probe phase
only once, but we did not store it. So the added feature bits in
re-negotiation will fail.
To fix it, we store it down, and will be used to feature negotiation
either at device initialization phase or device configure phase.
Fixes: e9efa4d93821 ("net/virtio-user: add new virtual PCI driver")
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 31 +++++++++++-------------
drivers/net/virtio/virtio_user/virtio_user_dev.h | 5 +++-
drivers/net/virtio/virtio_user_ethdev.c | 4 +--
3 files changed, 20 insertions(+), 20 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index e239e0e..af5d5be 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -228,29 +228,26 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
}
if (vhost_user_sock(dev->vhostfd, VHOST_USER_GET_FEATURES,
- &dev->features) < 0) {
+ &dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
}
if (dev->mac_specified)
- dev->features |= (1ull << VIRTIO_NET_F_MAC);
+ dev->device_features |= (1ull << VIRTIO_NET_F_MAC);
- if (!cq) {
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
- /* Also disable features depends on VIRTIO_NET_F_CTRL_VQ */
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_RX);
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_VLAN);
- dev->features &= ~(1ull << VIRTIO_NET_F_GUEST_ANNOUNCE);
- dev->features &= ~(1ull << VIRTIO_NET_F_MQ);
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
- } else {
- /* vhost user backend does not need to know ctrl-q, so
- * actually we need add this bit into features. However,
- * DPDK vhost-user does send features with this bit, so we
- * check it instead of OR it for now.
+ if (cq) {
+ /* device does not really need to know anything about CQ,
+ * so if necessary, we just claim to support CQ
*/
- if (!(dev->features & (1ull << VIRTIO_NET_F_CTRL_VQ)))
- PMD_INIT_LOG(INFO, "vhost does not support ctrl-q");
+ dev->device_features |= (1ull << VIRTIO_NET_F_CTRL_VQ);
+ } else {
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
+ /* Also disable features depends on VIRTIO_NET_F_CTRL_VQ */
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_RX);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_VLAN);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_GUEST_ANNOUNCE);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_MQ);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
}
if (dev->max_queue_pairs > 1) {
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 33690b5..28fc788 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -46,7 +46,10 @@ struct virtio_user_dev {
uint32_t max_queue_pairs;
uint32_t queue_pairs;
uint32_t queue_size;
- uint64_t features;
+ uint64_t features; /* the negotiated features with driver,
+ * and will be sync with device
+ */
+ uint64_t device_features; /* supported features by device */
uint8_t status;
uint8_t mac_addr[ETHER_ADDR_LEN];
char path[PATH_MAX];
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 406beea..c5db719 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -117,7 +117,7 @@ virtio_user_get_features(struct virtio_hw *hw)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- return dev->features;
+ return dev->device_features;
}
static void
@@ -125,7 +125,7 @@ virtio_user_set_features(struct virtio_hw *hw, uint64_t features)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- dev->features = features;
+ dev->features = features & dev->device_features;
}
static uint8_t
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
2016-12-23 7:14 ` [PATCH v2 1/7] net/virtio_user: fix wrongly set features Jianfeng Tan
@ 2016-12-23 7:14 ` Jianfeng Tan
2016-12-26 6:27 ` Yuanhan Liu
2016-12-23 7:14 ` [PATCH v2 3/7] net/virtio_user: move vhost user specific code Jianfeng Tan
` (4 subsequent siblings)
6 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-23 7:14 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
In driver probe phase, we obtain device information; and then virtio
driver will initialize device and stores info, like negotiated
features, in vhost_user layer; finally, vhost_user gets DRIVER_OK
notification from virtio driver, and then sync with backend device.
Previously, DRIVER_OK could be sent twice: 1. when ether layer invokes
eth_device_init to initialize device; 2. when user configures it with
different configuration from that of previous.
Since we can only depend on DRIVER_OK notification to sync with backend
device, we postpone it to virtio_dev_start when everything is settled.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_ethdev.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 079fd6c..d54b1bb 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1276,7 +1276,6 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
ret = virtio_alloc_queues(eth_dev);
if (ret < 0)
return ret;
- vtpci_reinit_complete(hw);
if (pci_dev)
PMD_INIT_LOG(DEBUG, "port %d vendorID=0x%x deviceID=0x%x",
@@ -1474,6 +1473,8 @@ virtio_dev_start(struct rte_eth_dev *dev)
struct virtnet_tx *txvq __rte_unused;
struct virtio_hw *hw = dev->data->dev_private;
+ vtpci_reinit_complete(hw);
+
/* check if lsc interrupt feature is enabled */
if (dev->data->dev_conf.intr_conf.lsc) {
if (!(dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC)) {
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 3/7] net/virtio_user: move vhost user specific code
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
2016-12-23 7:14 ` [PATCH v2 1/7] net/virtio_user: fix wrongly set features Jianfeng Tan
2016-12-23 7:14 ` [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification Jianfeng Tan
@ 2016-12-23 7:14 ` Jianfeng Tan
2016-12-26 6:28 ` Yuanhan Liu
2016-12-23 7:14 ` [PATCH v2 4/7] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
` (3 subsequent siblings)
6 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-23 7:14 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
To support vhost kernel as the backend of net_virtio_user in coming
patches, we move vhost_user specific structs and macros into
vhost_user.c, and only keep common definitions in vhost.h.
Besides, remove VHOST_USER_MQ feature check, it will be added back
in following multiqueue patch.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost.h | 36 ------------------------
drivers/net/virtio/virtio_user/vhost_user.c | 32 +++++++++++++++++++++
drivers/net/virtio/virtio_user/virtio_user_dev.c | 9 ------
3 files changed, 32 insertions(+), 45 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 7adb55f..e54ac35 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -42,8 +42,6 @@
#include "../virtio_logs.h"
#include "../virtqueue.h"
-#define VHOST_MEMORY_MAX_NREGIONS 8
-
struct vhost_vring_state {
unsigned int index;
unsigned int num;
@@ -105,40 +103,6 @@ struct vhost_memory_region {
uint64_t mmap_offset;
};
-struct vhost_memory {
- uint32_t nregions;
- uint32_t padding;
- struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
-};
-
-struct vhost_user_msg {
- enum vhost_user_request request;
-
-#define VHOST_USER_VERSION_MASK 0x3
-#define VHOST_USER_REPLY_MASK (0x1 << 2)
- uint32_t flags;
- uint32_t size; /* the following payload size */
- union {
-#define VHOST_USER_VRING_IDX_MASK 0xff
-#define VHOST_USER_VRING_NOFD_MASK (0x1 << 8)
- uint64_t u64;
- struct vhost_vring_state state;
- struct vhost_vring_addr addr;
- struct vhost_memory memory;
- } payload;
- int fds[VHOST_MEMORY_MAX_NREGIONS];
-} __attribute((packed));
-
-#define VHOST_USER_HDR_SIZE offsetof(struct vhost_user_msg, payload.u64)
-#define VHOST_USER_PAYLOAD_SIZE \
- (sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
-
-/* The version of the protocol we support */
-#define VHOST_USER_VERSION 0x1
-
-#define VHOST_USER_F_PROTOCOL_FEATURES 30
-#define VHOST_USER_MQ (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)
-
int vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg);
int vhost_user_setup(const char *path);
int vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable);
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 082e821..295ce16 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -42,6 +42,38 @@
#include "vhost.h"
+/* The version of the protocol we support */
+#define VHOST_USER_VERSION 0x1
+
+#define VHOST_MEMORY_MAX_NREGIONS 8
+struct vhost_memory {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
+};
+
+struct vhost_user_msg {
+ enum vhost_user_request request;
+
+#define VHOST_USER_VERSION_MASK 0x3
+#define VHOST_USER_REPLY_MASK (0x1 << 2)
+ uint32_t flags;
+ uint32_t size; /* the following payload size */
+ union {
+#define VHOST_USER_VRING_IDX_MASK 0xff
+#define VHOST_USER_VRING_NOFD_MASK (0x1 << 8)
+ uint64_t u64;
+ struct vhost_vring_state state;
+ struct vhost_vring_addr addr;
+ struct vhost_memory memory;
+ } payload;
+ int fds[VHOST_MEMORY_MAX_NREGIONS];
+} __attribute((packed));
+
+#define VHOST_USER_HDR_SIZE offsetof(struct vhost_user_msg, payload.u64)
+#define VHOST_USER_PAYLOAD_SIZE \
+ (sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
+
static int
vhost_user_write(int fd, void *buf, int len, int *fds, int fd_num)
{
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index af5d5be..66a8ffe 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -151,8 +151,6 @@ virtio_user_start_device(struct virtio_user_dev *dev)
* and VIRTIO_NET_F_MAC is stripped.
*/
features = dev->features;
- if (dev->max_queue_pairs > 1)
- features |= VHOST_USER_MQ;
features &= ~(1ull << VIRTIO_NET_F_MAC);
ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
if (ret < 0)
@@ -250,13 +248,6 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
}
- if (dev->max_queue_pairs > 1) {
- if (!(dev->features & VHOST_USER_MQ)) {
- PMD_INIT_LOG(ERR, "MQ not supported by the backend");
- return -1;
- }
- }
-
return 0;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 4/7] net/virtio_user: abstract virtio user backend ops
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
` (2 preceding siblings ...)
2016-12-23 7:14 ` [PATCH v2 3/7] net/virtio_user: move vhost user specific code Jianfeng Tan
@ 2016-12-23 7:14 ` Jianfeng Tan
2016-12-26 6:41 ` Yuanhan Liu
2016-12-23 7:14 ` [PATCH v2 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
` (2 subsequent siblings)
6 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-23 7:14 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
Add a struct virtio_user_backend_ops to abstract three kinds of backend
operations:
- setup, create the unix socket connection;
- send_request, sync messages with backend;
- enable_qp, enable some queue pair.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost.h | 19 +++++--
drivers/net/virtio/virtio_user/vhost_user.c | 54 ++++++++-----------
drivers/net/virtio/virtio_user/virtio_user_dev.c | 66 +++++++++++++++++-------
drivers/net/virtio/virtio_user/virtio_user_dev.h | 7 +++
4 files changed, 93 insertions(+), 53 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index e54ac35..bd67133 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -103,8 +103,21 @@ struct vhost_memory_region {
uint64_t mmap_offset;
};
-int vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg);
-int vhost_user_setup(const char *path);
-int vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable);
+struct virtio_user_dev;
+typedef int (*vhost_setup_t)(struct virtio_user_dev *dev);
+typedef int (*vhost_send_request_t)(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg);
+typedef int (*vhost_enable_qp_t)(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable);
+
+struct virtio_user_backend_ops {
+ vhost_setup_t setup;
+ vhost_send_request_t send_request;
+ vhost_enable_qp_t enable_qp;
+};
+
+struct virtio_user_backend_ops ops_user;
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 295ce16..faea1b6 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -41,6 +41,7 @@
#include <errno.h>
#include "vhost.h"
+#include "virtio_user_dev.h"
/* The version of the protocol we support */
#define VHOST_USER_VERSION 0x1
@@ -255,24 +256,10 @@ prepare_vhost_memory_user(struct vhost_user_msg *msg, int fds[])
static struct vhost_user_msg m;
-static const char * const vhost_msg_strings[] = {
- [VHOST_USER_SET_OWNER] = "VHOST_USER_SET_OWNER",
- [VHOST_USER_RESET_OWNER] = "VHOST_USER_RESET_OWNER",
- [VHOST_USER_SET_FEATURES] = "VHOST_USER_SET_FEATURES",
- [VHOST_USER_GET_FEATURES] = "VHOST_USER_GET_FEATURES",
- [VHOST_USER_SET_VRING_CALL] = "VHOST_USER_SET_VRING_CALL",
- [VHOST_USER_SET_VRING_NUM] = "VHOST_USER_SET_VRING_NUM",
- [VHOST_USER_SET_VRING_BASE] = "VHOST_USER_SET_VRING_BASE",
- [VHOST_USER_GET_VRING_BASE] = "VHOST_USER_GET_VRING_BASE",
- [VHOST_USER_SET_VRING_ADDR] = "VHOST_USER_SET_VRING_ADDR",
- [VHOST_USER_SET_VRING_KICK] = "VHOST_USER_SET_VRING_KICK",
- [VHOST_USER_SET_MEM_TABLE] = "VHOST_USER_SET_MEM_TABLE",
- [VHOST_USER_SET_VRING_ENABLE] = "VHOST_USER_SET_VRING_ENABLE",
- NULL,
-};
-
-int
-vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
+static int
+vhost_user_sock(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg)
{
struct vhost_user_msg msg;
struct vhost_vring_file *file = 0;
@@ -280,11 +267,9 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
int fds[VHOST_MEMORY_MAX_NREGIONS];
int fd_num = 0;
int i, len;
+ int vhostfd = dev->vhostfd;
RTE_SET_USED(m);
- RTE_SET_USED(vhost_msg_strings);
-
- PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
msg.request = req;
msg.flags = VHOST_USER_VERSION;
@@ -355,8 +340,8 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
len = VHOST_USER_HDR_SIZE + msg.size;
if (vhost_user_write(vhostfd, &msg, len, fds, fd_num) < 0) {
- PMD_DRV_LOG(ERR, "%s failed: %s",
- vhost_msg_strings[req], strerror(errno));
+ PMD_DRV_LOG(ERR, "fail to write vhost user msg: %s",
+ strerror(errno));
return -1;
}
@@ -403,15 +388,13 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
/**
* Set up environment to talk with a vhost user backend.
- * @param path
- * - The path to vhost user unix socket file.
*
* @return
* - (-1) if fail to set up;
* - (>=0) if successful, and it is the fd to vhostfd.
*/
-int
-vhost_user_setup(const char *path)
+static int
+vhost_user_setup(struct virtio_user_dev *dev)
{
int fd;
int flag;
@@ -429,7 +412,7 @@ vhost_user_setup(const char *path)
memset(&un, 0, sizeof(un));
un.sun_family = AF_UNIX;
- snprintf(un.sun_path, sizeof(un.sun_path), "%s", path);
+ snprintf(un.sun_path, sizeof(un.sun_path), "%s", dev->path);
if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno));
close(fd);
@@ -439,8 +422,10 @@ vhost_user_setup(const char *path)
return fd;
}
-int
-vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
+static int
+vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable)
{
int i;
@@ -450,10 +435,15 @@ vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
.num = enable,
};
- if (vhost_user_sock(vhostfd,
- VHOST_USER_SET_VRING_ENABLE, &state))
+ if (vhost_user_sock(dev, VHOST_USER_SET_VRING_ENABLE, &state))
return -1;
}
return 0;
}
+
+struct virtio_user_backend_ops ops_user = {
+ .setup = vhost_user_setup,
+ .send_request = vhost_user_sock,
+ .enable_qp = vhost_user_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 66a8ffe..a818c29 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -39,6 +39,9 @@
#include <sys/mman.h>
#include <unistd.h>
#include <sys/eventfd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
#include "vhost.h"
#include "virtio_user_dev.h"
@@ -64,7 +67,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = callfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_CALL, &file);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_CALL, &file);
dev->callfds[queue_sel] = callfd;
return 0;
@@ -88,12 +91,12 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.index = queue_sel;
state.num = vring->num;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_NUM, &state);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
state.num = 0; /* no reservation */
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_BASE, &state);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_ADDR, &addr);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
/* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
* lastly because vhost depends on this msg to judge if
@@ -106,7 +109,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = kickfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_KICK, &file);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_KICK, &file);
dev->kickfds[queue_sel] = kickfd;
return 0;
@@ -147,18 +150,17 @@ virtio_user_start_device(struct virtio_user_dev *dev)
goto error;
/* Step 1: set features
- * Make sure VHOST_USER_F_PROTOCOL_FEATURES is added if mq is enabled,
- * and VIRTIO_NET_F_MAC is stripped.
+ * Strip VIRTIO_NET_F_MAC, as MAC address is handled in vdev init.
*/
features = dev->features;
features &= ~(1ull << VIRTIO_NET_F_MAC);
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
+ ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
if (ret < 0)
goto error;
PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
/* Step 2: share memory regions */
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_MEM_TABLE, NULL);
+ ret = dev->ops->send_request(dev, VHOST_USER_SET_MEM_TABLE, NULL);
if (ret < 0)
goto error;
@@ -169,7 +171,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
/* Step 4: enable queues
* we enable the 1st queue pair by default.
*/
- vhost_user_enable_queue_pair(dev->vhostfd, 0, 1);
+ dev->ops->enable_qp(dev, 0, 1);
return 0;
error:
@@ -179,7 +181,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
int virtio_user_stop_device(struct virtio_user_dev *dev)
{
- return vhost_user_sock(dev->vhostfd, VHOST_USER_RESET_OWNER, NULL);
+ return dev->ops->send_request(dev, VHOST_USER_RESET_OWNER, NULL);
}
static inline void
@@ -203,6 +205,36 @@ parse_mac(struct virtio_user_dev *dev, const char *mac)
}
}
+static int
+is_vhost_user_by_type(const char *path)
+{
+ struct stat sb;
+
+ if (stat(path, &sb) == -1)
+ return 0;
+
+ return S_ISSOCK(sb.st_mode);
+}
+
+static int
+virtio_user_dev_setup(struct virtio_user_dev *dev)
+{
+ uint32_t i;
+
+ dev->vhostfd = -1;
+ for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
+ dev->kickfds[i] = -1;
+ dev->callfds[i] = -1;
+ }
+
+ if (is_vhost_user_by_type(dev->path)) {
+ dev->ops = &ops_user;
+ return dev->ops->setup(dev);
+ }
+
+ return -1;
+}
+
int
virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
int cq, int queue_size, const char *mac)
@@ -213,19 +245,17 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
dev->queue_size = queue_size;
dev->mac_specified = 0;
parse_mac(dev, mac);
- dev->vhostfd = -1;
- dev->vhostfd = vhost_user_setup(dev->path);
- if (dev->vhostfd < 0) {
+ if (virtio_user_dev_setup(dev) < 0) {
PMD_INIT_LOG(ERR, "backend set up fails");
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_SET_OWNER, NULL) < 0) {
+ if (dev->ops->send_request(dev, VHOST_USER_SET_OWNER, NULL) < 0) {
PMD_INIT_LOG(ERR, "set_owner fails: %s", strerror(errno));
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_GET_FEATURES,
+ if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
&dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
@@ -277,9 +307,9 @@ virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs)
}
for (i = 0; i < q_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 1);
+ ret |= dev->ops->enable_qp(dev, i, 1);
for (i = q_pairs; i < dev->max_queue_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+ ret |= dev->ops->enable_qp(dev, i, 0);
dev->queue_pairs = q_pairs;
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 28fc788..503a496 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -37,9 +37,15 @@
#include <limits.h>
#include "../virtio_pci.h"
#include "../virtio_ring.h"
+#include "vhost.h"
struct virtio_user_dev {
+ /* for vhost_user backend */
int vhostfd;
+
+ /* for vhost_kernel backend */
+
+ /* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int kickfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int mac_specified;
@@ -54,6 +60,7 @@ struct virtio_user_dev {
uint8_t mac_addr[ETHER_ADDR_LEN];
char path[PATH_MAX];
struct vring vrings[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
+ struct virtio_user_backend_ops *ops;
};
int virtio_user_start_device(struct virtio_user_dev *dev);
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
` (3 preceding siblings ...)
2016-12-23 7:14 ` [PATCH v2 4/7] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
@ 2016-12-23 7:14 ` Jianfeng Tan
2016-12-26 7:44 ` Yuanhan Liu
2017-01-09 4:39 ` Jason Wang
2016-12-23 7:14 ` [PATCH v2 6/7] net/virtio_user: enable offloading Jianfeng Tan
2016-12-23 7:14 ` [PATCH v2 7/7] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
6 siblings, 2 replies; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-23 7:14 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
This patch add support vhost kernel as the backend for virtio_user.
Three main hook functions are added:
- vhost_kernel_setup() to open char device, each vq pair needs one
vhostfd;
- vhost_kernel_ioctl() to communicate control messages with vhost
kernel module;
- vhost_kernel_enable_queue_pair() to open tap device and set it
as the backend of corresonding vhost fd (that is to say, vq pair).
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/Makefile | 1 +
drivers/net/virtio/virtio_user/vhost.h | 2 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 364 +++++++++++++++++++++++
drivers/net/virtio/virtio_user/virtio_user_dev.c | 21 +-
drivers/net/virtio/virtio_user/virtio_user_dev.h | 4 +
5 files changed, 388 insertions(+), 4 deletions(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile
index 97972a6..faeffb2 100644
--- a/drivers/net/virtio/Makefile
+++ b/drivers/net/virtio/Makefile
@@ -60,6 +60,7 @@ endif
ifeq ($(CONFIG_RTE_VIRTIO_USER),y)
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_user.c
+SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_kernel.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/virtio_user_dev.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user_ethdev.c
endif
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index bd67133..ffab13a 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -120,4 +120,6 @@ struct virtio_user_backend_ops {
};
struct virtio_user_backend_ops ops_user;
+struct virtio_user_backend_ops ops_kernel;
+
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
new file mode 100644
index 0000000..8984c5c
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -0,0 +1,364 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <net/if.h>
+#include <string.h>
+#include <errno.h>
+
+#include <rte_memory.h>
+#include <rte_eal_memconfig.h>
+
+#include "vhost.h"
+#include "virtio_user_dev.h"
+
+struct vhost_memory_kernel {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[0];
+};
+
+/* vhost kernel ioctls */
+#define VHOST_VIRTIO 0xAF
+#define VHOST_GET_FEATURES _IOR(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_FEATURES _IOW(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
+#define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
+#define VHOST_SET_MEM_TABLE _IOW(VHOST_VIRTIO, 0x03, struct vhost_memory_kernel)
+#define VHOST_SET_LOG_BASE _IOW(VHOST_VIRTIO, 0x04, __u64)
+#define VHOST_SET_LOG_FD _IOW(VHOST_VIRTIO, 0x07, int)
+#define VHOST_SET_VRING_NUM _IOW(VHOST_VIRTIO, 0x10, struct vhost_vring_state)
+#define VHOST_SET_VRING_ADDR _IOW(VHOST_VIRTIO, 0x11, struct vhost_vring_addr)
+#define VHOST_SET_VRING_BASE _IOW(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_GET_VRING_BASE _IOWR(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_SET_VRING_KICK _IOW(VHOST_VIRTIO, 0x20, struct vhost_vring_file)
+#define VHOST_SET_VRING_CALL _IOW(VHOST_VIRTIO, 0x21, struct vhost_vring_file)
+#define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
+#define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct vhost_vring_file)
+
+/* TUN ioctls */
+#define TUNSETIFF _IOW('T', 202, int)
+#define TUNGETFEATURES _IOR('T', 207, unsigned int)
+#define TUNSETOFFLOAD _IOW('T', 208, unsigned int)
+#define TUNGETIFF _IOR('T', 210, unsigned int)
+#define TUNSETSNDBUF _IOW('T', 212, int)
+#define TUNGETVNETHDRSZ _IOR('T', 215, int)
+#define TUNSETVNETHDRSZ _IOW('T', 216, int)
+#define TUNSETQUEUE _IOW('T', 217, int)
+#define TUNSETVNETLE _IOW('T', 220, int)
+#define TUNSETVNETBE _IOW('T', 222, int)
+
+/* TUNSETIFF ifr flags */
+#define IFF_TAP 0x0002
+#define IFF_NO_PI 0x1000
+#define IFF_ONE_QUEUE 0x2000
+#define IFF_VNET_HDR 0x4000
+#define IFF_MULTI_QUEUE 0x0100
+#define IFF_ATTACH_QUEUE 0x0200
+#define IFF_DETACH_QUEUE 0x0400
+
+/* Constants */
+#define TUN_DEF_SNDBUF (1ull << 20)
+#define PATH_NET_TUN "/dev/net/tun"
+#define VHOST_KERNEL_MAX_REGIONS 64
+
+static uint64_t vhost_req_user_to_kernel[] = {
+ [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
+ [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
+ [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
+ [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
+ [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
+ [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
+ [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
+ [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
+ [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
+ [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
+ [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
+};
+
+/* By default, vhost kernel module allows 64 regions, but DPDK allows
+ * 256 segments. As a relief, below function merges those virtually
+ * adjacent memsegs into one region.
+ */
+static struct vhost_memory_kernel *
+prepare_vhost_memory_kernel(void)
+{
+ uint32_t i, j, k = 0;
+ struct rte_memseg *seg;
+ struct vhost_memory_region *mr;
+ struct vhost_memory_kernel *vm;
+
+ vm = malloc(sizeof(struct vhost_memory_kernel) +
+ VHOST_KERNEL_MAX_REGIONS *
+ sizeof(struct vhost_memory_region));
+
+ for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
+ seg = &rte_eal_get_configuration()->mem_config->memseg[i];
+ if (!seg->addr)
+ break;
+
+ int new_region = 1;
+
+ for (j = 0; j < k; ++j) {
+ mr = &vm->regions[j];
+
+ if (mr->userspace_addr + mr->memory_size ==
+ (uint64_t)seg->addr) {
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+
+ if ((uint64_t)seg->addr + seg->len ==
+ mr->userspace_addr) {
+ mr->guest_phys_addr = (uint64_t)seg->addr;
+ mr->userspace_addr = (uint64_t)seg->addr;
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+ }
+
+ if (new_region == 0)
+ continue;
+
+ mr = &vm->regions[k++];
+ mr->guest_phys_addr = (uint64_t)seg->addr; /* use vaddr here! */
+ mr->userspace_addr = (uint64_t)seg->addr;
+ mr->memory_size = seg->len;
+ mr->mmap_offset = 0;
+
+ if (k >= VHOST_KERNEL_MAX_REGIONS) {
+ free(vm);
+ return NULL;
+ }
+ }
+
+ vm->nregions = k;
+ vm->padding = 0;
+ return vm;
+}
+
+static int
+vhost_kernel_ioctl(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg)
+{
+ int i, ret = -1;
+ uint64_t req_kernel;
+ struct vhost_memory_kernel *vm = NULL;
+
+ req_kernel = vhost_req_user_to_kernel[req];
+
+ if (req_kernel == VHOST_SET_MEM_TABLE) {
+ vm = prepare_vhost_memory_kernel();
+ if (!vm)
+ return -1;
+ arg = (void *)vm;
+ }
+
+ /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
+ if (req_kernel == VHOST_SET_FEATURES)
+ *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+
+ for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
+ if (dev->vhostfds[i] < 0)
+ continue;
+
+ ret = ioctl(dev->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+
+ if (vm)
+ free(vm);
+
+ return ret;
+}
+
+/**
+ * Set up environment to talk with a vhost kernel backend.
+ *
+ * @return
+ * - (-1) if fail to set up;
+ * - (>=0) if successful.
+ */
+static int
+vhost_kernel_setup(struct virtio_user_dev *dev)
+{
+ int vhostfd;
+ uint32_t q;
+
+ for (q = 0; q < dev->max_queue_pairs; ++q) {
+ vhostfd = open(dev->path, O_RDWR);
+ if (vhostfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s, %s",
+ dev->path, strerror(errno));
+ return -1;
+ }
+
+ dev->vhostfds[q] = vhostfd;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_set_backend(int vhostfd, int tapfd)
+{
+ struct vhost_vring_file f;
+
+ f.fd = tapfd;
+ f.index = 0;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ f.index = 1;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable)
+{
+ unsigned int features;
+ int sndbuf = TUN_DEF_SNDBUF;
+ struct ifreq ifr;
+ int hdr_size;
+ int vhostfd;
+ int tapfd;
+
+ vhostfd = dev->vhostfds[pair_idx];
+
+ if (!enable) {
+ if (dev->tapfds[pair_idx]) {
+ close(dev->tapfds[pair_idx]);
+ dev->tapfds[pair_idx] = -1;
+ }
+ return vhost_kernel_set_backend(vhostfd, -1);
+ } else if (dev->tapfds[pair_idx] >= 0) {
+ return 0;
+ }
+
+ if ((dev->features & (1ULL << VIRTIO_NET_F_MRG_RXBUF)) ||
+ (dev->features & (1ULL << VIRTIO_F_VERSION_1)))
+ hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
+ else
+ hdr_size = sizeof(struct virtio_net_hdr);
+
+ /* TODO:
+ * 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
+ * 2. get number of memory regions from vhost module parameter
+ * max_mem_regions, supported in newer version linux kernel
+ */
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ /* Construct ifr */
+ memset(&ifr, 0, sizeof(ifr));
+ ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
+
+ if (ioctl(tapfd, TUNGETFEATURES, &features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ goto error;
+ }
+ if (features & IFF_ONE_QUEUE)
+ ifr.ifr_flags |= IFF_ONE_QUEUE;
+
+ /* Let tap instead of vhost-net handle vnet header, as the latter does
+ * not support offloading. And in this case, we should not set feature
+ * bit VHOST_NET_F_VIRTIO_NET_HDR.
+ */
+ if (features & IFF_VNET_HDR) {
+ ifr.ifr_flags |= IFF_VNET_HDR;
+ } else {
+ PMD_DRV_LOG(ERR, "TAP does not support IFF_VNET_HDR");
+ goto error;
+ }
+
+ if (dev->ifname)
+ strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
+ else
+ strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
+ if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
+ PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ fcntl(tapfd, F_SETFL, O_NONBLOCK);
+
+ if (ioctl(tapfd, TUNSETVNETHDRSZ, &hdr_size) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETVNETHDRSZ failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (ioctl(tapfd, TUNSETSNDBUF, &sndbuf) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETSNDBUF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (vhost_kernel_set_backend(vhostfd, tapfd) < 0)
+ goto error;
+
+ dev->tapfds[pair_idx] = tapfd;
+ if (!dev->ifname)
+ dev->ifname = strdup(ifr.ifr_name);
+
+ return 0;
+error:
+ return -1;
+}
+
+struct virtio_user_backend_ops ops_kernel = {
+ .setup = vhost_kernel_setup,
+ .send_request = vhost_kernel_ioctl,
+ .enable_qp = vhost_kernel_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index a818c29..c718b85 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -219,7 +219,7 @@ is_vhost_user_by_type(const char *path)
static int
virtio_user_dev_setup(struct virtio_user_dev *dev)
{
- uint32_t i;
+ uint32_t i, q;
dev->vhostfd = -1;
for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
@@ -227,12 +227,18 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
dev->callfds[i] = -1;
}
+ for (q = 0; q < VHOST_KERNEL_MAX_QUEUES; ++q) {
+ dev->vhostfds[q] = -1;
+ dev->tapfds[q] = -1;
+ }
+
if (is_vhost_user_by_type(dev->path)) {
dev->ops = &ops_user;
- return dev->ops->setup(dev);
+ } else {
+ dev->ops = &ops_kernel;
}
- return -1;
+ return dev->ops->setup(dev);
}
int
@@ -284,7 +290,9 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
void
virtio_user_dev_uninit(struct virtio_user_dev *dev)
{
- uint32_t i;
+ uint32_t i, q;
+
+ dev->ops->send_request(dev, VHOST_USER_RESET_OWNER, NULL);
for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
close(dev->callfds[i]);
@@ -292,6 +300,11 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
}
close(dev->vhostfd);
+
+ for (q = 0; q < VHOST_KERNEL_MAX_QUEUES; ++q) {
+ close(dev->vhostfds[q]);
+ close(dev->tapfds[q]);
+ }
}
static uint8_t
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 503a496..148b2e6 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -44,6 +44,10 @@ struct virtio_user_dev {
int vhostfd;
/* for vhost_kernel backend */
+ char *ifname;
+#define VHOST_KERNEL_MAX_QUEUES 8
+ int vhostfds[VHOST_KERNEL_MAX_QUEUES];
+ int tapfds[VHOST_KERNEL_MAX_QUEUES];
/* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 6/7] net/virtio_user: enable offloading
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
` (4 preceding siblings ...)
2016-12-23 7:14 ` [PATCH v2 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
@ 2016-12-23 7:14 ` Jianfeng Tan
2016-12-26 7:53 ` Yuanhan Liu
2016-12-23 7:14 ` [PATCH v2 7/7] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
6 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-23 7:14 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
When used with vhost kernel backend, we can offload at both directions.
- From vhost kernel to virtio_user, the offload is enabled so that
DPDK app can trust the flow is checksum-correct; and if DPDK app
sends it through another port, the checksum needs to be
recalculated or offloaded. It also applies to TSO.
- From virtio_user to vhost_kernel, the offload is enabled so that
kernel can trust the flow is L4-checksum-correct, no need to verify
it; if kernel will consume it, DPDK app should make sure the
l3-checksum is correctly set.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 61 ++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 2 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 8984c5c..fb3c454 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -91,6 +91,13 @@ struct vhost_memory_kernel {
#define IFF_ATTACH_QUEUE 0x0200
#define IFF_DETACH_QUEUE 0x0400
+/* Features for GSO (TUNSETOFFLOAD). */
+#define TUN_F_CSUM 0x01 /* You can hand me unchecksummed packets. */
+#define TUN_F_TSO4 0x02 /* I can handle TSO for IPv4 packets */
+#define TUN_F_TSO6 0x04 /* I can handle TSO for IPv6 packets */
+#define TUN_F_TSO_ECN 0x08 /* I can handle TSO with ECN bits. */
+#define TUN_F_UFO 0x10 /* I can handle UFO packets */
+
/* Constants */
#define TUN_DEF_SNDBUF (1ull << 20)
#define PATH_NET_TUN "/dev/net/tun"
@@ -173,6 +180,28 @@ prepare_vhost_memory_kernel(void)
return vm;
}
+/* with below features, vhost kernel does not need to do the checksum and TSO,
+ * these info will be passed to virtio_user through virtio net header.
+ */
+static const uint64_t guest_offloads_mask =
+ (1ULL << VIRTIO_NET_F_GUEST_CSUM) |
+ (1ULL << VIRTIO_NET_F_GUEST_TSO4) |
+ (1ULL << VIRTIO_NET_F_GUEST_TSO6) |
+ (1ULL << VIRTIO_NET_F_GUEST_ECN) |
+ (1ULL << VIRTIO_NET_F_GUEST_UFO);
+
+/* with below features, when flows from virtio_user to vhost kernel
+ * (1) if flows goes up through the kernel networking stack, it does not need
+ * to verify checksum, which can save CPU cycles;
+ * (2) if flows goes through a Linux bridge and outside from an interface
+ * (kernel driver), checksum and TSO will be done by GSO in kernel or even
+ * offloaded into real physical device.
+ */
+static const uint64_t host_offloads_mask =
+ (1ULL << VIRTIO_NET_F_CSUM) |
+ (1ULL << VIRTIO_NET_F_HOST_TSO4) |
+ (1ULL << VIRTIO_NET_F_HOST_TSO6);
+
static int
vhost_kernel_ioctl(struct virtio_user_dev *dev,
enum vhost_user_request req,
@@ -191,10 +220,15 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
arg = (void *)vm;
}
- /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
- if (req_kernel == VHOST_SET_FEATURES)
+ if (req_kernel == VHOST_SET_FEATURES) {
+ /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
*(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+ /* VHOST kernel does not know about below flags */
+ *(uint64_t *)arg &= ~guest_offloads_mask;
+ *(uint64_t *)arg &= ~host_offloads_mask;
+ }
+
for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
if (dev->vhostfds[i] < 0)
continue;
@@ -204,6 +238,15 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
break;
}
+ if (!ret && req_kernel == VHOST_GET_FEATURES) {
+ /* with tap as the backend, all these features are supported
+ * but not claimed by vhost-net, so we add them back when
+ * reporting to upper layer.
+ */
+ *((uint64_t *)arg) |= guest_offloads_mask;
+ *((uint64_t *)arg) |= host_offloads_mask;
+ }
+
if (vm)
free(vm);
@@ -271,6 +314,12 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
int hdr_size;
int vhostfd;
int tapfd;
+ unsigned int offload =
+ TUN_F_CSUM |
+ TUN_F_TSO4 |
+ TUN_F_TSO6 |
+ TUN_F_TSO_ECN |
+ TUN_F_UFO;
vhostfd = dev->vhostfds[pair_idx];
@@ -345,6 +394,14 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
goto error;
}
+ /* TODO: before set the offload capabilities, we'd better (1) check
+ * negotiated features to see if necessary to offload; (2) query tap
+ * to see if it supports the offload capabilities.
+ */
+ if (ioctl(tapfd, TUNSETOFFLOAD, offload) != 0)
+ PMD_DRV_LOG(ERR, "TUNSETOFFLOAD ioctl() failed: %s",
+ strerror(errno));
+
if (vhost_kernel_set_backend(vhostfd, tapfd) < 0)
goto error;
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v2 7/7] net/virtio_user: enable multiqueue with vhost kernel
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
` (5 preceding siblings ...)
2016-12-23 7:14 ` [PATCH v2 6/7] net/virtio_user: enable offloading Jianfeng Tan
@ 2016-12-23 7:14 ` Jianfeng Tan
6 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2016-12-23 7:14 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
With vhost kernel, to enable multiqueue, we need backend device
in kernel support multiqueue feature. Specifically, with tap
as the backend, as linux/Documentation/networking/tuntap.txt shows,
we check if tap supports IFF_MULTI_QUEUE feature.
And for vhost kernel, each queue pair has a vhost fd, and with a tap
fd binding this vhost fd. All tap fds are set with the same tap
interface name.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 67 +++++++++++++++++++++---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 1 +
2 files changed, 62 insertions(+), 6 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index fb3c454..67b382e 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -203,6 +203,29 @@ static const uint64_t host_offloads_mask =
(1ULL << VIRTIO_NET_F_HOST_TSO6);
static int
+tap_supporte_mq(void)
+{
+ int tapfd;
+ unsigned int tap_features;
+
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ close(tapfd);
+ return -1;
+ }
+
+ close(tapfd);
+ return tap_features & IFF_MULTI_QUEUE;
+}
+
+static int
vhost_kernel_ioctl(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg)
@@ -210,6 +233,8 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
int i, ret = -1;
uint64_t req_kernel;
struct vhost_memory_kernel *vm = NULL;
+ int vhostfd;
+ unsigned int queue_sel;
req_kernel = vhost_req_user_to_kernel[req];
@@ -229,13 +254,33 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
*(uint64_t *)arg &= ~host_offloads_mask;
}
- for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
- if (dev->vhostfds[i] < 0)
- continue;
+ switch (req_kernel) {
+ case VHOST_SET_VRING_NUM:
+ case VHOST_SET_VRING_ADDR:
+ case VHOST_SET_VRING_BASE:
+ case VHOST_GET_VRING_BASE:
+ case VHOST_SET_VRING_KICK:
+ case VHOST_SET_VRING_CALL:
+ queue_sel = *(unsigned int *)arg;
+ vhostfd = dev->vhostfds[queue_sel / 2];
+ *(unsigned int *)arg = queue_sel % 2;
+ PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
+ vhostfd, *(unsigned int *)arg);
+ break;
+ default:
+ vhostfd = -1;
+ }
+ if (vhostfd == -1) {
+ for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
+ if (dev->vhostfds[i] < 0)
+ continue;
- ret = ioctl(dev->vhostfds[i], req_kernel, arg);
- if (ret < 0)
- break;
+ ret = ioctl(dev->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+ } else {
+ ret = ioctl(vhostfd, req_kernel, arg);
}
if (!ret && req_kernel == VHOST_GET_FEATURES) {
@@ -245,6 +290,12 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
*/
*((uint64_t *)arg) |= guest_offloads_mask;
*((uint64_t *)arg) |= host_offloads_mask;
+
+ /* vhost_kernel will not declare this feature, but it does
+ * support multi-queue.
+ */
+ if (tap_supporte_mq())
+ *(uint64_t *)arg |= (1ull << VIRTIO_NET_F_MQ);
}
if (vm)
@@ -320,6 +371,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
TUN_F_TSO6 |
TUN_F_TSO_ECN |
TUN_F_UFO;
+ int req_mq = (dev->max_queue_pairs > 1);
vhostfd = dev->vhostfds[pair_idx];
@@ -373,6 +425,9 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
goto error;
}
+ if (req_mq)
+ ifr.ifr_flags |= IFF_MULTI_QUEUE;
+
if (dev->ifname)
strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
else
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index c718b85..241f5bb 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -93,6 +93,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.num = vring->num;
dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
+ state.index = queue_sel;
state.num = 0; /* no reservation */
dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification
2016-12-23 7:14 ` [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification Jianfeng Tan
@ 2016-12-26 6:27 ` Yuanhan Liu
2016-12-26 6:55 ` Tan, Jianfeng
0 siblings, 1 reply; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-26 6:27 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Dec 23, 2016 at 07:14:21AM +0000, Jianfeng Tan wrote:
> In driver probe phase, we obtain device information; and then virtio
> driver will initialize device and stores info, like negotiated
> features, in vhost_user layer; finally, vhost_user gets DRIVER_OK
> notification from virtio driver, and then sync with backend device.
>
> Previously, DRIVER_OK could be sent twice: 1. when ether layer invokes
> eth_device_init to initialize device; 2. when user configures it with
> different configuration from that of previous.
Yes, but that's wrong. Now only 1) will be taken.
> Since we can only depend on DRIVER_OK notification to sync with backend
> device, we postpone it to virtio_dev_start when everything is settled.
I don't quite understand what you were going to fix here; you don't
state it in the commit log after all. Normally, when everything is
negotiated (when DRIVER_OK is set), we should not set it again, unless
a reset has been happened.
If you look at QEMU, qemu will simply ignore it once vhost has already
started.
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 3/7] net/virtio_user: move vhost user specific code
2016-12-23 7:14 ` [PATCH v2 3/7] net/virtio_user: move vhost user specific code Jianfeng Tan
@ 2016-12-26 6:28 ` Yuanhan Liu
2016-12-26 6:58 ` Tan, Jianfeng
0 siblings, 1 reply; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-26 6:28 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Dec 23, 2016 at 07:14:22AM +0000, Jianfeng Tan wrote:
> To support vhost kernel as the backend of net_virtio_user in coming
> patches, we move vhost_user specific structs and macros into
> vhost_user.c, and only keep common definitions in vhost.h.
Good.
> Besides, remove VHOST_USER_MQ feature check, it will be added back
> in following multiqueue patch.
Why then?
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 4/7] net/virtio_user: abstract virtio user backend ops
2016-12-23 7:14 ` [PATCH v2 4/7] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
@ 2016-12-26 6:41 ` Yuanhan Liu
0 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-26 6:41 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Dec 23, 2016 at 07:14:23AM +0000, Jianfeng Tan wrote:
> +typedef int (*vhost_setup_t)(struct virtio_user_dev *dev);
> +typedef int (*vhost_send_request_t)(struct virtio_user_dev *dev,
> + enum vhost_user_request req,
> + void *arg);
> +typedef int (*vhost_enable_qp_t)(struct virtio_user_dev *dev,
> + uint16_t pair_idx,
> + int enable);
I will not bother to define new types if they are just gonna be used once.
No strong perference here, just a suggestion.
...
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index 28fc788..503a496 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -37,9 +37,15 @@
> #include <limits.h>
> #include "../virtio_pci.h"
> #include "../virtio_ring.h"
> +#include "vhost.h"
>
> struct virtio_user_dev {
> + /* for vhost_user backend */
> int vhostfd;
> +
> + /* for vhost_kernel backend */
> +
> + /* for both vhost_user and vhost_kernel */
By far (till this patch), vhost_kernel is not introduced at all, that I
will not mention it here. That said, I will leave this change to the patch
that actually enables vhost-kernel.
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification
2016-12-26 6:27 ` Yuanhan Liu
@ 2016-12-26 6:55 ` Tan, Jianfeng
2016-12-26 8:02 ` Yuanhan Liu
0 siblings, 1 reply; 73+ messages in thread
From: Tan, Jianfeng @ 2016-12-26 6:55 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: dev, Yigit, Ferruh, Liang, Cunming
> -----Original Message-----
> From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
> Sent: Monday, December 26, 2016 2:27 PM
> To: Tan, Jianfeng
> Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
> Subject: Re: [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification
>
> On Fri, Dec 23, 2016 at 07:14:21AM +0000, Jianfeng Tan wrote:
> > In driver probe phase, we obtain device information; and then virtio
> > driver will initialize device and stores info, like negotiated
> > features, in vhost_user layer; finally, vhost_user gets DRIVER_OK
> > notification from virtio driver, and then sync with backend device.
> >
> > Previously, DRIVER_OK could be sent twice: 1. when ether layer invokes
> > eth_device_init to initialize device; 2. when user configures it with
> > different configuration from that of previous.
>
> Yes, but that's wrong. Now only 1) will be taken.
>From code in virtio_dev_configure() as below, if hw_ip_checksum or enable_lro is configured, the device will be reset and re-initialized.
if (rxmode->hw_ip_checksum)
req_features |= (1ULL << VIRTIO_NET_F_GUEST_CSUM);
if (rxmode->enable_lro)
req_features |=
(1ULL << VIRTIO_NET_F_GUEST_TSO4) |
(1ULL << VIRTIO_NET_F_GUEST_TSO6);
/* if request features changed, reinit the device */
if (req_features != hw->req_guest_features) {
ret = virtio_init_device(dev, req_features);
if (ret < 0)
return ret;
}
>
> > Since we can only depend on DRIVER_OK notification to sync with backend
> > device, we postpone it to virtio_dev_start when everything is settled.
>
> I don't quite understand what you were going to fix here; you don't
> state it in the commit log after all. Normally, when everything is
> negotiated (when DRIVER_OK is set), we should not set it again, unless
> a reset has been happened.
I agree that DRIVER_OK should be set only if everything is settled. Under the case of hw_ip_checksum or enable_lro, a reset will be sent and the device will be re-initialized.
So my point is that since the configuration might be changed in dev_configure(), why not just send DRIVER_OK after everything is done.
>
> If you look at QEMU, qemu will simply ignore it once vhost has already
> started.
It does not apply here. The configuration is changed, and it cannot be ignored.
Thanks,
Jianfeng
>
> --yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 3/7] net/virtio_user: move vhost user specific code
2016-12-26 6:28 ` Yuanhan Liu
@ 2016-12-26 6:58 ` Tan, Jianfeng
2016-12-26 7:57 ` Yuanhan Liu
0 siblings, 1 reply; 73+ messages in thread
From: Tan, Jianfeng @ 2016-12-26 6:58 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: dev, Yigit, Ferruh, Liang, Cunming
> -----Original Message-----
> From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
> Sent: Monday, December 26, 2016 2:28 PM
> To: Tan, Jianfeng
> Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
> Subject: Re: [PATCH v2 3/7] net/virtio_user: move vhost user specific code
>
> On Fri, Dec 23, 2016 at 07:14:22AM +0000, Jianfeng Tan wrote:
> > To support vhost kernel as the backend of net_virtio_user in coming
> > patches, we move vhost_user specific structs and macros into
> > vhost_user.c, and only keep common definitions in vhost.h.
>
> Good.
>
> > Besides, remove VHOST_USER_MQ feature check, it will be added back
> > in following multiqueue patch.
>
> Why then?
Only vhost user recognizes this feature bit, and vhost kernel does not. My intension is to put those vhost user specific code inside vhost user. And in fact, I forget to add it back. I should add it back in this patch.
Thanks,
Jianfeng
>
> --yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2016-12-23 7:14 ` [PATCH v2 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
@ 2016-12-26 7:44 ` Yuanhan Liu
2017-01-04 7:22 ` Tan, Jianfeng
2017-01-09 4:39 ` Jason Wang
1 sibling, 1 reply; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-26 7:44 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Dec 23, 2016 at 07:14:24AM +0000, Jianfeng Tan wrote:
> + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
^^^^^^^^^
I think it should be 2016.
> +/* By default, vhost kernel module allows 64 regions, but DPDK allows
> + * 256 segments. As a relief, below function merges those virtually
> + * adjacent memsegs into one region.
> + */
> +static struct vhost_memory_kernel *
> +prepare_vhost_memory_kernel(void)
> +{
> + uint32_t i, j, k = 0;
> + struct rte_memseg *seg;
> + struct vhost_memory_region *mr;
> + struct vhost_memory_kernel *vm;
> +
> + vm = malloc(sizeof(struct vhost_memory_kernel) +
> + VHOST_KERNEL_MAX_REGIONS *
> + sizeof(struct vhost_memory_region));
> +
> + for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
> + seg = &rte_eal_get_configuration()->mem_config->memseg[i];
> + if (!seg->addr)
> + break;
> +
> + int new_region = 1;
> +
> + for (j = 0; j < k; ++j) {
> + mr = &vm->regions[j];
> +
> + if (mr->userspace_addr + mr->memory_size ==
> + (uint64_t)seg->addr) {
> + mr->memory_size += seg->len;
> + new_region = 0;
> + break;
> + }
> +
> + if ((uint64_t)seg->addr + seg->len ==
> + mr->userspace_addr) {
> + mr->guest_phys_addr = (uint64_t)seg->addr;
Be careful, such cast would break 32 bit OS build.
> +static int
> +vhost_kernel_ioctl(struct virtio_user_dev *dev,
> + enum vhost_user_request req,
> + void *arg)
> +{
> + int i, ret = -1;
> + uint64_t req_kernel;
> + struct vhost_memory_kernel *vm = NULL;
> +
> + req_kernel = vhost_req_user_to_kernel[req];
> +
> + if (req_kernel == VHOST_SET_MEM_TABLE) {
> + vm = prepare_vhost_memory_kernel();
> + if (!vm)
> + return -1;
> + arg = (void *)vm;
> + }
> +
> + /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
Because this feature need the vhost IOTLB support from the device
emulation. Patches for QEMU hasn't been merged yet, but it has been
there for a while.
Since we don't have the support yet, for sure it won't work. But
I'm wondering why you have to disable it explicitly?
> + if (req_kernel == VHOST_SET_FEATURES)
> + *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
> +
> + for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
> + if (dev->vhostfds[i] < 0)
> + continue;
> +
> + ret = ioctl(dev->vhostfds[i], req_kernel, arg);
> + if (ret < 0)
> + break;
> + }
> +
> + if (vm)
> + free(vm);
> +
> + return ret;
> +}
> +
> +/**
> + * Set up environment to talk with a vhost kernel backend.
> + *
> + * @return
> + * - (-1) if fail to set up;
> + * - (>=0) if successful.
> + */
> +static int
> +vhost_kernel_setup(struct virtio_user_dev *dev)
> +{
> + int vhostfd;
> + uint32_t q;
Why not simply use 'i'?
> +static int
> +vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
> + uint16_t pair_idx,
> + int enable)
> +{
> + unsigned int features;
Better to rename it to "tap_features" or something? When I first saw
such name, I thought it was about vhost features.
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 6/7] net/virtio_user: enable offloading
2016-12-23 7:14 ` [PATCH v2 6/7] net/virtio_user: enable offloading Jianfeng Tan
@ 2016-12-26 7:53 ` Yuanhan Liu
0 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-26 7:53 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Dec 23, 2016 at 07:14:25AM +0000, Jianfeng Tan wrote:
> When used with vhost kernel backend, we can offload at both directions.
> - From vhost kernel to virtio_user, the offload is enabled so that
> DPDK app can trust the flow is checksum-correct; and if DPDK app
> sends it through another port, the checksum needs to be
> recalculated or offloaded. It also applies to TSO.
> - From virtio_user to vhost_kernel, the offload is enabled so that
> kernel can trust the flow is L4-checksum-correct, no need to verify
> it; if kernel will consume it, DPDK app should make sure the
> l3-checksum is correctly set.
>
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> ---
> drivers/net/virtio/virtio_user/vhost_kernel.c | 61 ++++++++++++++++++++++++++-
> 1 file changed, 59 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
> index 8984c5c..fb3c454 100644
> --- a/drivers/net/virtio/virtio_user/vhost_kernel.c
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -91,6 +91,13 @@ struct vhost_memory_kernel {
> #define IFF_ATTACH_QUEUE 0x0200
> #define IFF_DETACH_QUEUE 0x0400
>
> +/* Features for GSO (TUNSETOFFLOAD). */
> +#define TUN_F_CSUM 0x01 /* You can hand me unchecksummed packets. */
> +#define TUN_F_TSO4 0x02 /* I can handle TSO for IPv4 packets */
> +#define TUN_F_TSO6 0x04 /* I can handle TSO for IPv6 packets */
> +#define TUN_F_TSO_ECN 0x08 /* I can handle TSO with ECN bits. */
> +#define TUN_F_UFO 0x10 /* I can handle UFO packets */
> +
> /* Constants */
> #define TUN_DEF_SNDBUF (1ull << 20)
> #define PATH_NET_TUN "/dev/net/tun"
> @@ -173,6 +180,28 @@ prepare_vhost_memory_kernel(void)
> return vm;
> }
>
> +/* with below features, vhost kernel does not need to do the checksum and TSO,
> + * these info will be passed to virtio_user through virtio net header.
> + */
> +static const uint64_t guest_offloads_mask =
The typical way is to define it as macro?
> + (1ULL << VIRTIO_NET_F_GUEST_CSUM) |
> + (1ULL << VIRTIO_NET_F_GUEST_TSO4) |
> + (1ULL << VIRTIO_NET_F_GUEST_TSO6) |
> + (1ULL << VIRTIO_NET_F_GUEST_ECN) |
> + (1ULL << VIRTIO_NET_F_GUEST_UFO);
> +
> @@ -271,6 +314,12 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
> int hdr_size;
> int vhostfd;
> int tapfd;
> + unsigned int offload =
> + TUN_F_CSUM |
> + TUN_F_TSO4 |
> + TUN_F_TSO6 |
> + TUN_F_TSO_ECN |
> + TUN_F_UFO;
Same here?
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 3/7] net/virtio_user: move vhost user specific code
2016-12-26 6:58 ` Tan, Jianfeng
@ 2016-12-26 7:57 ` Yuanhan Liu
2016-12-29 9:40 ` Tan, Jianfeng
0 siblings, 1 reply; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-26 7:57 UTC (permalink / raw)
To: Tan, Jianfeng; +Cc: dev, Yigit, Ferruh, Liang, Cunming
On Mon, Dec 26, 2016 at 06:58:58AM +0000, Tan, Jianfeng wrote:
>
>
> > -----Original Message-----
> > From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
> > Sent: Monday, December 26, 2016 2:28 PM
> > To: Tan, Jianfeng
> > Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
> > Subject: Re: [PATCH v2 3/7] net/virtio_user: move vhost user specific code
> >
> > On Fri, Dec 23, 2016 at 07:14:22AM +0000, Jianfeng Tan wrote:
> > > To support vhost kernel as the backend of net_virtio_user in coming
> > > patches, we move vhost_user specific structs and macros into
> > > vhost_user.c, and only keep common definitions in vhost.h.
> >
> > Good.
> >
> > > Besides, remove VHOST_USER_MQ feature check, it will be added back
> > > in following multiqueue patch.
> >
> > Why then?
>
> Only vhost user recognizes this feature bit, and vhost kernel does not. My intension is to put those vhost user specific code inside vhost user.
That's okay. But why you have to remove it and then add it back?
--yliu
> And in fact, I forget to add it back. I should add it back in this patch.
>
> Thanks,
> Jianfeng
> >
> > --yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification
2016-12-26 6:55 ` Tan, Jianfeng
@ 2016-12-26 8:02 ` Yuanhan Liu
2016-12-26 8:26 ` Tan, Jianfeng
0 siblings, 1 reply; 73+ messages in thread
From: Yuanhan Liu @ 2016-12-26 8:02 UTC (permalink / raw)
To: Tan, Jianfeng; +Cc: dev, Yigit, Ferruh, Liang, Cunming
On Mon, Dec 26, 2016 at 06:55:16AM +0000, Tan, Jianfeng wrote:
>
>
> > -----Original Message-----
> > From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
> > Sent: Monday, December 26, 2016 2:27 PM
> > To: Tan, Jianfeng
> > Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
> > Subject: Re: [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification
> >
> > On Fri, Dec 23, 2016 at 07:14:21AM +0000, Jianfeng Tan wrote:
> > > In driver probe phase, we obtain device information; and then virtio
> > > driver will initialize device and stores info, like negotiated
> > > features, in vhost_user layer; finally, vhost_user gets DRIVER_OK
> > > notification from virtio driver, and then sync with backend device.
> > >
> > > Previously, DRIVER_OK could be sent twice: 1. when ether layer invokes
> > > eth_device_init to initialize device; 2. when user configures it with
> > > different configuration from that of previous.
> >
> > Yes, but that's wrong. Now only 1) will be taken.
>
> >From code in virtio_dev_configure() as below, if hw_ip_checksum or enable_lro is configured, the device will be reset and re-initialized.
> if (rxmode->hw_ip_checksum)
> req_features |= (1ULL << VIRTIO_NET_F_GUEST_CSUM);
> if (rxmode->enable_lro)
> req_features |=
> (1ULL << VIRTIO_NET_F_GUEST_TSO4) |
> (1ULL << VIRTIO_NET_F_GUEST_TSO6);
>
> /* if request features changed, reinit the device */
> if (req_features != hw->req_guest_features) {
> ret = virtio_init_device(dev, req_features);
> if (ret < 0)
> return ret;
> }
Yes, I know. But virtio_init_device() will call vtpci_reinit_complete()
at the end, doesn't it?
> > > Since we can only depend on DRIVER_OK notification to sync with backend
> > > device, we postpone it to virtio_dev_start when everything is settled.
> >
> > I don't quite understand what you were going to fix here; you don't
> > state it in the commit log after all. Normally, when everything is
> > negotiated (when DRIVER_OK is set), we should not set it again, unless
> > a reset has been happened.
>
> I agree that DRIVER_OK should be set only if everything is settled. Under the case of hw_ip_checksum or enable_lro, a reset will be sent and the device will be re-initialized.
>
> So my point is that since the configuration might be changed in dev_configure(), why not just send DRIVER_OK after everything is done.
>
> >
> > If you look at QEMU, qemu will simply ignore it once vhost has already
> > started.
>
> It does not apply here. The configuration is changed, and it cannot be ignored.
Why it doesn't apply here? What makes virtio_user special? If it works
to QEMU and it doesn't to virtio_user, it basically means the virtio_user
is broken ...
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification
2016-12-26 8:02 ` Yuanhan Liu
@ 2016-12-26 8:26 ` Tan, Jianfeng
0 siblings, 0 replies; 73+ messages in thread
From: Tan, Jianfeng @ 2016-12-26 8:26 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: dev, Yigit, Ferruh, Liang, Cunming
As discussed offline, this will lead to many DRIVER OK among dev_stop()/dev_start(). I'll remove this patch.
Thanks,
Jianfeng
> -----Original Message-----
> From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
> Sent: Monday, December 26, 2016 4:02 PM
> To: Tan, Jianfeng
> Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
> Subject: Re: [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification
>
> On Mon, Dec 26, 2016 at 06:55:16AM +0000, Tan, Jianfeng wrote:
> >
> >
> > > -----Original Message-----
> > > From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
> > > Sent: Monday, December 26, 2016 2:27 PM
> > > To: Tan, Jianfeng
> > > Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
> > > Subject: Re: [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK
> notification
> > >
> > > On Fri, Dec 23, 2016 at 07:14:21AM +0000, Jianfeng Tan wrote:
> > > > In driver probe phase, we obtain device information; and then virtio
> > > > driver will initialize device and stores info, like negotiated
> > > > features, in vhost_user layer; finally, vhost_user gets DRIVER_OK
> > > > notification from virtio driver, and then sync with backend device.
> > > >
> > > > Previously, DRIVER_OK could be sent twice: 1. when ether layer
> invokes
> > > > eth_device_init to initialize device; 2. when user configures it with
> > > > different configuration from that of previous.
> > >
> > > Yes, but that's wrong. Now only 1) will be taken.
> >
> > >From code in virtio_dev_configure() as below, if hw_ip_checksum or
> enable_lro is configured, the device will be reset and re-initialized.
> > if (rxmode->hw_ip_checksum)
> > req_features |= (1ULL << VIRTIO_NET_F_GUEST_CSUM);
> > if (rxmode->enable_lro)
> > req_features |=
> > (1ULL << VIRTIO_NET_F_GUEST_TSO4) |
> > (1ULL << VIRTIO_NET_F_GUEST_TSO6);
> >
> > /* if request features changed, reinit the device */
> > if (req_features != hw->req_guest_features) {
> > ret = virtio_init_device(dev, req_features);
> > if (ret < 0)
> > return ret;
> > }
>
> Yes, I know. But virtio_init_device() will call vtpci_reinit_complete()
> at the end, doesn't it?
>
> > > > Since we can only depend on DRIVER_OK notification to sync with
> backend
> > > > device, we postpone it to virtio_dev_start when everything is settled.
> > >
> > > I don't quite understand what you were going to fix here; you don't
> > > state it in the commit log after all. Normally, when everything is
> > > negotiated (when DRIVER_OK is set), we should not set it again, unless
> > > a reset has been happened.
> >
> > I agree that DRIVER_OK should be set only if everything is settled. Under
> the case of hw_ip_checksum or enable_lro, a reset will be sent and the
> device will be re-initialized.
> >
> > So my point is that since the configuration might be changed in
> dev_configure(), why not just send DRIVER_OK after everything is done.
> >
> > >
> > > If you look at QEMU, qemu will simply ignore it once vhost has already
> > > started.
> >
> > It does not apply here. The configuration is changed, and it cannot be
> ignored.
>
> Why it doesn't apply here? What makes virtio_user special? If it works
> to QEMU and it doesn't to virtio_user, it basically means the virtio_user
> is broken ...
>
> --yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 3/7] net/virtio_user: move vhost user specific code
2016-12-26 7:57 ` Yuanhan Liu
@ 2016-12-29 9:40 ` Tan, Jianfeng
0 siblings, 0 replies; 73+ messages in thread
From: Tan, Jianfeng @ 2016-12-29 9:40 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: dev, Yigit, Ferruh, Liang, Cunming
Hi Yuanhan,
On 12/26/2016 3:57 PM, Yuanhan Liu wrote:
> On Mon, Dec 26, 2016 at 06:58:58AM +0000, Tan, Jianfeng wrote:
>>
>>> -----Original Message-----
>>> From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
>>> Sent: Monday, December 26, 2016 2:28 PM
>>> To: Tan, Jianfeng
>>> Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
>>> Subject: Re: [PATCH v2 3/7] net/virtio_user: move vhost user specific code
>>>
>>> On Fri, Dec 23, 2016 at 07:14:22AM +0000, Jianfeng Tan wrote:
>>>> To support vhost kernel as the backend of net_virtio_user in coming
>>>> patches, we move vhost_user specific structs and macros into
>>>> vhost_user.c, and only keep common definitions in vhost.h.
>>> Good.
>>>
>>>> Besides, remove VHOST_USER_MQ feature check, it will be added back
>>>> in following multiqueue patch.
>>> Why then?
>> Only vhost user recognizes this feature bit, and vhost kernel does not. My intension is to put those vhost user specific code inside vhost user.
> That's okay. But why you have to remove it and then add it back?
After second thought , I agree that it's necessary to add it back.
Currently, our vhost does not implement the semantics of
VHOST_USER_F_PROTOCOL_FEATURES.
Besides, all other comments from you on this patch series will be
addressed on next version. Thank you very much for those suggestions!
Thanks,
Jianfeng
>
> --yliu
>
>> And in fact, I forget to add it back. I should add it back in this patch.
>>
>> Thanks,
>> Jianfeng
>>> --yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v3 0/7] virtio_user as an alternative exception path
2016-12-02 14:31 [PATCH 0/3] virtio_user as an alternative exception path Jianfeng Tan
` (4 preceding siblings ...)
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
@ 2017-01-04 3:59 ` Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 1/7] net/virtio_user: fix wrongly set features Jianfeng Tan
` (7 more replies)
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
6 siblings, 8 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-04 3:59 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
v3:
- Drop the patch to postpone driver ok sending patch, superseded it
with a bug fix to disable all virtqueues and re-init the device.
(you might wonder why not just send reset owner msg. Under my test,
it causes spinlock deadlock problem when killing the program).
- Avoid compiling error on 32-bit system for pointer convert.
- Fix a bug in patch "abstract virtio user backend ops", vhostfd is
not properly assigned.
- Fix a "MQ cannot be used" bug in v2, which is related to strip
some feature bits that vhost kernel does not recognize.
- Update release note.
v2: (Lots of them are from yuanhan's comment)
- Add offloding feature.
- Add multiqueue support.
- Add a new patch to postpone the sending of driver ok notification.
- Put fix patch ahead of the whole patch series.
- Split original 0001 patch into 0003 and 0004 patches.
- Remove the original vhost_internal design, just add those into
struct virtio_user_dev for simplicity.
- Reword "control" to "send_request".
- Reword "host_features" to "device_features".
In v16.07, we upstreamed a virtual device, virtio_user (with vhost-user
as the backend). The path to go with a vhost-kernel backend has been
dropped for bad performance comparing to vhost-user and code simplicity.
But after a second thought, virtio_user + vhost-kernel is a good
candidate as an exceptional path, such as KNI, which exchanges packets
with kernel networking stack.
- maintenance: vhost-net (kernel) is upstreamed and extensively used
kernel module. We don't need any out-of-tree module like KNI.
- performance: as with KNI, this solution would use one or more
kthreads to send/receive packets from user space DPDK applications,
which has little impact on user space polling thread (except that
it might enter into kernel space to wake up those kthreads if
necessary).
- features: vhost-net is born to be a networking solution, which has
lots of networking related featuers, like multi queue, tso, multi-seg
mbuf, etc.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Jianfeng Tan (7):
net/virtio_user: fix wrongly set features
net/virtio_user: fix not properly reset device
net/virtio_user: move vhost user specific code
net/virtio_user: abstract virtio user backend ops
net/virtio_user: add vhost kernel support
net/virtio_user: enable offloading
net/virtio_user: enable multiqueue with vhost kernel
doc/guides/rel_notes/release_17_02.rst | 20 +
drivers/net/virtio/Makefile | 1 +
drivers/net/virtio/virtio_user/vhost.h | 51 +--
drivers/net/virtio/virtio_user/vhost_kernel.c | 487 +++++++++++++++++++++++
drivers/net/virtio/virtio_user/vhost_user.c | 97 +++--
drivers/net/virtio/virtio_user/virtio_user_dev.c | 138 ++++---
drivers/net/virtio/virtio_user/virtio_user_dev.h | 16 +-
drivers/net/virtio/virtio_user_ethdev.c | 19 +-
8 files changed, 705 insertions(+), 124 deletions(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
--
2.7.4
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v3 1/7] net/virtio_user: fix wrongly set features
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
@ 2017-01-04 3:59 ` Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 2/7] net/virtio_user: fix not properly reset device Jianfeng Tan
` (6 subsequent siblings)
7 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-04 3:59 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan, stable
Before the commit 86d59b21468a ("net/virtio: support LRO"), features
in virtio PMD, is decided and properly set at device initialization
and will not be changed. But afterward, features could be changed in
virtio_dev_configure(), and will be re-negotiated if it's changed.
In virtio_user, device features is obtained at driver probe phase
only once, but we did not store it. So the added feature bits in
re-negotiation will fail.
To fix it, we store it down, and will be used to feature negotiation
either at device initialization phase or device configure phase.
Fixes: e9efa4d93821 ("net/virtio-user: add new virtual PCI driver")
CC: stable@dpdk.org
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 34 +++++++++++-------------
drivers/net/virtio/virtio_user/virtio_user_dev.h | 5 +++-
drivers/net/virtio/virtio_user_ethdev.c | 4 +--
3 files changed, 22 insertions(+), 21 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index e239e0e..0d7e17b 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -148,12 +148,13 @@ virtio_user_start_device(struct virtio_user_dev *dev)
/* Step 1: set features
* Make sure VHOST_USER_F_PROTOCOL_FEATURES is added if mq is enabled,
- * and VIRTIO_NET_F_MAC is stripped.
+ * VIRTIO_NET_F_MAC and VIRTIO_NET_F_CTRL_VQ is stripped.
*/
features = dev->features;
if (dev->max_queue_pairs > 1)
features |= VHOST_USER_MQ;
features &= ~(1ull << VIRTIO_NET_F_MAC);
+ features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
if (ret < 0)
goto error;
@@ -228,29 +229,26 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
}
if (vhost_user_sock(dev->vhostfd, VHOST_USER_GET_FEATURES,
- &dev->features) < 0) {
+ &dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
}
if (dev->mac_specified)
- dev->features |= (1ull << VIRTIO_NET_F_MAC);
+ dev->device_features |= (1ull << VIRTIO_NET_F_MAC);
- if (!cq) {
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
- /* Also disable features depends on VIRTIO_NET_F_CTRL_VQ */
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_RX);
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_VLAN);
- dev->features &= ~(1ull << VIRTIO_NET_F_GUEST_ANNOUNCE);
- dev->features &= ~(1ull << VIRTIO_NET_F_MQ);
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
- } else {
- /* vhost user backend does not need to know ctrl-q, so
- * actually we need add this bit into features. However,
- * DPDK vhost-user does send features with this bit, so we
- * check it instead of OR it for now.
+ if (cq) {
+ /* device does not really need to know anything about CQ,
+ * so if necessary, we just claim to support CQ
*/
- if (!(dev->features & (1ull << VIRTIO_NET_F_CTRL_VQ)))
- PMD_INIT_LOG(INFO, "vhost does not support ctrl-q");
+ dev->device_features |= (1ull << VIRTIO_NET_F_CTRL_VQ);
+ } else {
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
+ /* Also disable features depends on VIRTIO_NET_F_CTRL_VQ */
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_RX);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_VLAN);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_GUEST_ANNOUNCE);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_MQ);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
}
if (dev->max_queue_pairs > 1) {
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 33690b5..28fc788 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -46,7 +46,10 @@ struct virtio_user_dev {
uint32_t max_queue_pairs;
uint32_t queue_pairs;
uint32_t queue_size;
- uint64_t features;
+ uint64_t features; /* the negotiated features with driver,
+ * and will be sync with device
+ */
+ uint64_t device_features; /* supported features by device */
uint8_t status;
uint8_t mac_addr[ETHER_ADDR_LEN];
char path[PATH_MAX];
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 8cb983c..4a5a227 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -117,7 +117,7 @@ virtio_user_get_features(struct virtio_hw *hw)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- return dev->features;
+ return dev->device_features;
}
static void
@@ -125,7 +125,7 @@ virtio_user_set_features(struct virtio_hw *hw, uint64_t features)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- dev->features = features;
+ dev->features = features & dev->device_features;
}
static uint8_t
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v3 2/7] net/virtio_user: fix not properly reset device
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 1/7] net/virtio_user: fix wrongly set features Jianfeng Tan
@ 2017-01-04 3:59 ` Jianfeng Tan
2017-01-04 5:46 ` Yuanhan Liu
2017-01-04 3:59 ` [PATCH v3 3/7] net/virtio_user: move vhost user specific code Jianfeng Tan
` (5 subsequent siblings)
7 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-04 3:59 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan, stable
virtio_user is not properly reset when users call vtpci_reset(),
as it ignores VIRTIO_CONFIG_STATUS_RESET status in
virtio_user_set_status().
This might lead to initialization failure as it starts to re-init
the device before sending RESET messege to backend. Besides, previous
callfds and kickfds are not closed.
To fix it, we add support to disable virtqueues when it's set to
DRIVER OK status, and re-init fields in struct virtio_user_dev.
Fixes: e9efa4d93821 ("net/virtio-user: add new virtual PCI driver")
Fixes: 37a7eb2ae816 ("net/virtio-user: add device emulation layer")
CC: stable@dpdk.org
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 26 ++++++++++++++++--------
drivers/net/virtio/virtio_user_ethdev.c | 15 ++++++++------
2 files changed, 27 insertions(+), 14 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 0d7e17b..a38398b 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -182,7 +182,17 @@ virtio_user_start_device(struct virtio_user_dev *dev)
int virtio_user_stop_device(struct virtio_user_dev *dev)
{
- return vhost_user_sock(dev->vhostfd, VHOST_USER_RESET_OWNER, NULL);
+ uint32_t i;
+
+ for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
+ close(dev->callfds[i]);
+ close(dev->kickfds[i]);
+ }
+
+ for (i = 0; i < dev->max_queue_pairs; ++i)
+ vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+
+ return 0;
}
static inline void
@@ -210,6 +220,8 @@ int
virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
int cq, int queue_size, const char *mac)
{
+ uint32_t i;
+
snprintf(dev->path, PATH_MAX, "%s", path);
dev->max_queue_pairs = queues;
dev->queue_pairs = 1; /* mq disabled by default */
@@ -218,6 +230,11 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
parse_mac(dev, mac);
dev->vhostfd = -1;
+ for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
+ dev->kickfds[i] = -1;
+ dev->callfds[i] = -1;
+ }
+
dev->vhostfd = vhost_user_setup(dev->path);
if (dev->vhostfd < 0) {
PMD_INIT_LOG(ERR, "backend set up fails");
@@ -264,13 +281,6 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
void
virtio_user_dev_uninit(struct virtio_user_dev *dev)
{
- uint32_t i;
-
- for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
- close(dev->callfds[i]);
- close(dev->kickfds[i]);
- }
-
close(dev->vhostfd);
}
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 4a5a227..93f5b01 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -87,21 +87,24 @@ virtio_user_write_dev_config(struct virtio_hw *hw, size_t offset,
}
static void
-virtio_user_set_status(struct virtio_hw *hw, uint8_t status)
+virtio_user_reset(struct virtio_hw *hw)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- if (status & VIRTIO_CONFIG_STATUS_DRIVER_OK)
- virtio_user_start_device(dev);
- dev->status = status;
+ if (dev->status & VIRTIO_CONFIG_STATUS_DRIVER_OK)
+ virtio_user_stop_device(dev);
}
static void
-virtio_user_reset(struct virtio_hw *hw)
+virtio_user_set_status(struct virtio_hw *hw, uint8_t status)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- virtio_user_stop_device(dev);
+ if (status & VIRTIO_CONFIG_STATUS_DRIVER_OK)
+ virtio_user_start_device(dev);
+ else if (status == VIRTIO_CONFIG_STATUS_RESET)
+ virtio_user_reset(hw);
+ dev->status = status;
}
static uint8_t
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v3 3/7] net/virtio_user: move vhost user specific code
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 1/7] net/virtio_user: fix wrongly set features Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 2/7] net/virtio_user: fix not properly reset device Jianfeng Tan
@ 2017-01-04 3:59 ` Jianfeng Tan
2017-01-04 6:02 ` Yuanhan Liu
2017-01-04 3:59 ` [PATCH v3 4/7] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
` (4 subsequent siblings)
7 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-04 3:59 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
To support vhost kernel as the backend of net_virtio_user in coming
patches, we move vhost_user specific structs and macros into
vhost_user.c, and only keep common definitions in vhost.h.
Besides, remove VHOST_USER_MQ feature check.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost.h | 36 ------------------------
drivers/net/virtio/virtio_user/vhost_user.c | 32 +++++++++++++++++++++
drivers/net/virtio/virtio_user/virtio_user_dev.c | 9 ------
3 files changed, 32 insertions(+), 45 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 7adb55f..e54ac35 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -42,8 +42,6 @@
#include "../virtio_logs.h"
#include "../virtqueue.h"
-#define VHOST_MEMORY_MAX_NREGIONS 8
-
struct vhost_vring_state {
unsigned int index;
unsigned int num;
@@ -105,40 +103,6 @@ struct vhost_memory_region {
uint64_t mmap_offset;
};
-struct vhost_memory {
- uint32_t nregions;
- uint32_t padding;
- struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
-};
-
-struct vhost_user_msg {
- enum vhost_user_request request;
-
-#define VHOST_USER_VERSION_MASK 0x3
-#define VHOST_USER_REPLY_MASK (0x1 << 2)
- uint32_t flags;
- uint32_t size; /* the following payload size */
- union {
-#define VHOST_USER_VRING_IDX_MASK 0xff
-#define VHOST_USER_VRING_NOFD_MASK (0x1 << 8)
- uint64_t u64;
- struct vhost_vring_state state;
- struct vhost_vring_addr addr;
- struct vhost_memory memory;
- } payload;
- int fds[VHOST_MEMORY_MAX_NREGIONS];
-} __attribute((packed));
-
-#define VHOST_USER_HDR_SIZE offsetof(struct vhost_user_msg, payload.u64)
-#define VHOST_USER_PAYLOAD_SIZE \
- (sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
-
-/* The version of the protocol we support */
-#define VHOST_USER_VERSION 0x1
-
-#define VHOST_USER_F_PROTOCOL_FEATURES 30
-#define VHOST_USER_MQ (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)
-
int vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg);
int vhost_user_setup(const char *path);
int vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable);
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 082e821..295ce16 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -42,6 +42,38 @@
#include "vhost.h"
+/* The version of the protocol we support */
+#define VHOST_USER_VERSION 0x1
+
+#define VHOST_MEMORY_MAX_NREGIONS 8
+struct vhost_memory {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
+};
+
+struct vhost_user_msg {
+ enum vhost_user_request request;
+
+#define VHOST_USER_VERSION_MASK 0x3
+#define VHOST_USER_REPLY_MASK (0x1 << 2)
+ uint32_t flags;
+ uint32_t size; /* the following payload size */
+ union {
+#define VHOST_USER_VRING_IDX_MASK 0xff
+#define VHOST_USER_VRING_NOFD_MASK (0x1 << 8)
+ uint64_t u64;
+ struct vhost_vring_state state;
+ struct vhost_vring_addr addr;
+ struct vhost_memory memory;
+ } payload;
+ int fds[VHOST_MEMORY_MAX_NREGIONS];
+} __attribute((packed));
+
+#define VHOST_USER_HDR_SIZE offsetof(struct vhost_user_msg, payload.u64)
+#define VHOST_USER_PAYLOAD_SIZE \
+ (sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
+
static int
vhost_user_write(int fd, void *buf, int len, int *fds, int fd_num)
{
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index a38398b..8dd563a 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -151,8 +151,6 @@ virtio_user_start_device(struct virtio_user_dev *dev)
* VIRTIO_NET_F_MAC and VIRTIO_NET_F_CTRL_VQ is stripped.
*/
features = dev->features;
- if (dev->max_queue_pairs > 1)
- features |= VHOST_USER_MQ;
features &= ~(1ull << VIRTIO_NET_F_MAC);
features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
@@ -268,13 +266,6 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
}
- if (dev->max_queue_pairs > 1) {
- if (!(dev->features & VHOST_USER_MQ)) {
- PMD_INIT_LOG(ERR, "MQ not supported by the backend");
- return -1;
- }
- }
-
return 0;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v3 4/7] net/virtio_user: abstract virtio user backend ops
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
` (2 preceding siblings ...)
2017-01-04 3:59 ` [PATCH v3 3/7] net/virtio_user: move vhost user specific code Jianfeng Tan
@ 2017-01-04 3:59 ` Jianfeng Tan
2017-01-04 6:11 ` Yuanhan Liu
2017-01-04 3:59 ` [PATCH v3 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
` (3 subsequent siblings)
7 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-04 3:59 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
Add a struct virtio_user_backend_ops to abstract three kinds of backend
operations:
- setup, create the unix socket connection;
- send_request, sync messages with backend;
- enable_qp, enable some queue pair.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost.h | 17 +++++-
drivers/net/virtio/virtio_user/vhost_user.c | 65 +++++++++++---------
drivers/net/virtio/virtio_user/virtio_user_dev.c | 77 +++++++++++++++---------
drivers/net/virtio/virtio_user/virtio_user_dev.h | 5 ++
4 files changed, 106 insertions(+), 58 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index e54ac35..515e4fc 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -96,6 +96,8 @@ enum vhost_user_request {
VHOST_USER_MAX
};
+const char * const vhost_msg_strings[VHOST_USER_MAX];
+
struct vhost_memory_region {
uint64_t guest_phys_addr;
uint64_t memory_size; /* bytes */
@@ -103,8 +105,17 @@ struct vhost_memory_region {
uint64_t mmap_offset;
};
-int vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg);
-int vhost_user_setup(const char *path);
-int vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable);
+struct virtio_user_dev;
+
+struct virtio_user_backend_ops {
+ int (*setup)(struct virtio_user_dev *dev);
+ int (*send_request)(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg);
+ int (*enable_qp)(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable);
+};
+struct virtio_user_backend_ops ops_user;
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 295ce16..a9ca10f 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -41,6 +41,7 @@
#include <errno.h>
#include "vhost.h"
+#include "virtio_user_dev.h"
/* The version of the protocol we support */
#define VHOST_USER_VERSION 0x1
@@ -255,24 +256,26 @@ prepare_vhost_memory_user(struct vhost_user_msg *msg, int fds[])
static struct vhost_user_msg m;
-static const char * const vhost_msg_strings[] = {
- [VHOST_USER_SET_OWNER] = "VHOST_USER_SET_OWNER",
- [VHOST_USER_RESET_OWNER] = "VHOST_USER_RESET_OWNER",
- [VHOST_USER_SET_FEATURES] = "VHOST_USER_SET_FEATURES",
- [VHOST_USER_GET_FEATURES] = "VHOST_USER_GET_FEATURES",
- [VHOST_USER_SET_VRING_CALL] = "VHOST_USER_SET_VRING_CALL",
- [VHOST_USER_SET_VRING_NUM] = "VHOST_USER_SET_VRING_NUM",
- [VHOST_USER_SET_VRING_BASE] = "VHOST_USER_SET_VRING_BASE",
- [VHOST_USER_GET_VRING_BASE] = "VHOST_USER_GET_VRING_BASE",
- [VHOST_USER_SET_VRING_ADDR] = "VHOST_USER_SET_VRING_ADDR",
- [VHOST_USER_SET_VRING_KICK] = "VHOST_USER_SET_VRING_KICK",
- [VHOST_USER_SET_MEM_TABLE] = "VHOST_USER_SET_MEM_TABLE",
- [VHOST_USER_SET_VRING_ENABLE] = "VHOST_USER_SET_VRING_ENABLE",
+const char * const vhost_msg_strings[] = {
+ [VHOST_USER_SET_OWNER] = "VHOST_SET_OWNER",
+ [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
+ [VHOST_USER_SET_FEATURES] = "VHOST_SET_FEATURES",
+ [VHOST_USER_GET_FEATURES] = "VHOST_GET_FEATURES",
+ [VHOST_USER_SET_VRING_CALL] = "VHOST_SET_VRING_CALL",
+ [VHOST_USER_SET_VRING_NUM] = "VHOST_SET_VRING_NUM",
+ [VHOST_USER_SET_VRING_BASE] = "VHOST_SET_VRING_BASE",
+ [VHOST_USER_GET_VRING_BASE] = "VHOST_GET_VRING_BASE",
+ [VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
+ [VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
+ [VHOST_USER_SET_MEM_TABLE] = "VHOST_SET_MEM_TABLE",
+ [VHOST_USER_SET_VRING_ENABLE] = "VHOST_SET_VRING_ENABLE",
NULL,
};
-int
-vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
+static int
+vhost_user_sock(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg)
{
struct vhost_user_msg msg;
struct vhost_vring_file *file = 0;
@@ -280,9 +283,9 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
int fds[VHOST_MEMORY_MAX_NREGIONS];
int fd_num = 0;
int i, len;
+ int vhostfd = dev->vhostfd;
RTE_SET_USED(m);
- RTE_SET_USED(vhost_msg_strings);
PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
@@ -403,15 +406,13 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
/**
* Set up environment to talk with a vhost user backend.
- * @param path
- * - The path to vhost user unix socket file.
*
* @return
- * - (-1) if fail to set up;
- * - (>=0) if successful, and it is the fd to vhostfd.
+ * - (-1) if fail;
+ * - (0) if succeed.
*/
-int
-vhost_user_setup(const char *path)
+static int
+vhost_user_setup(struct virtio_user_dev *dev)
{
int fd;
int flag;
@@ -429,18 +430,21 @@ vhost_user_setup(const char *path)
memset(&un, 0, sizeof(un));
un.sun_family = AF_UNIX;
- snprintf(un.sun_path, sizeof(un.sun_path), "%s", path);
+ snprintf(un.sun_path, sizeof(un.sun_path), "%s", dev->path);
if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno));
close(fd);
return -1;
}
- return fd;
+ dev->vhostfd = fd;
+ return 0;
}
-int
-vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
+static int
+vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable)
{
int i;
@@ -450,10 +454,15 @@ vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
.num = enable,
};
- if (vhost_user_sock(vhostfd,
- VHOST_USER_SET_VRING_ENABLE, &state))
+ if (vhost_user_sock(dev, VHOST_USER_SET_VRING_ENABLE, &state))
return -1;
}
return 0;
}
+
+struct virtio_user_backend_ops ops_user = {
+ .setup = vhost_user_setup,
+ .send_request = vhost_user_sock,
+ .enable_qp = vhost_user_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 8dd563a..32039a1 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -39,6 +39,9 @@
#include <sys/mman.h>
#include <unistd.h>
#include <sys/eventfd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
#include "vhost.h"
#include "virtio_user_dev.h"
@@ -64,7 +67,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = callfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_CALL, &file);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_CALL, &file);
dev->callfds[queue_sel] = callfd;
return 0;
@@ -88,12 +91,12 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.index = queue_sel;
state.num = vring->num;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_NUM, &state);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
state.num = 0; /* no reservation */
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_BASE, &state);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_ADDR, &addr);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
/* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
* lastly because vhost depends on this msg to judge if
@@ -106,7 +109,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = kickfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_KICK, &file);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_KICK, &file);
dev->kickfds[queue_sel] = kickfd;
return 0;
@@ -146,20 +149,19 @@ virtio_user_start_device(struct virtio_user_dev *dev)
if (virtio_user_queue_setup(dev, virtio_user_create_queue) < 0)
goto error;
- /* Step 1: set features
- * Make sure VHOST_USER_F_PROTOCOL_FEATURES is added if mq is enabled,
- * VIRTIO_NET_F_MAC and VIRTIO_NET_F_CTRL_VQ is stripped.
- */
+ /* Step 1: set features */
features = dev->features;
+ /* Strip VIRTIO_NET_F_MAC, as MAC address is handled in vdev init */
features &= ~(1ull << VIRTIO_NET_F_MAC);
+ /* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
+ ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
if (ret < 0)
goto error;
PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
/* Step 2: share memory regions */
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_MEM_TABLE, NULL);
+ ret = dev->ops->send_request(dev, VHOST_USER_SET_MEM_TABLE, NULL);
if (ret < 0)
goto error;
@@ -170,7 +172,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
/* Step 4: enable queues
* we enable the 1st queue pair by default.
*/
- vhost_user_enable_queue_pair(dev->vhostfd, 0, 1);
+ dev->ops->enable_qp(dev, 0, 1);
return 0;
error:
@@ -188,7 +190,7 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
}
for (i = 0; i < dev->max_queue_pairs; ++i)
- vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+ dev->ops->enable_qp(dev, i, 0);
return 0;
}
@@ -214,36 +216,57 @@ parse_mac(struct virtio_user_dev *dev, const char *mac)
}
}
+static int
+is_vhost_user_by_type(const char *path)
+{
+ struct stat sb;
+
+ if (stat(path, &sb) == -1)
+ return 0;
+
+ return S_ISSOCK(sb.st_mode);
+}
+
+static int
+virtio_user_dev_setup(struct virtio_user_dev *dev)
+{
+ uint32_t i;
+
+ dev->vhostfd = -1;
+ for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
+ dev->kickfds[i] = -1;
+ dev->callfds[i] = -1;
+ }
+
+ if (is_vhost_user_by_type(dev->path)) {
+ dev->ops = &ops_user;
+ return dev->ops->setup(dev);
+ }
+
+ return -1;
+}
+
int
virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
int cq, int queue_size, const char *mac)
{
- uint32_t i;
-
snprintf(dev->path, PATH_MAX, "%s", path);
dev->max_queue_pairs = queues;
dev->queue_pairs = 1; /* mq disabled by default */
dev->queue_size = queue_size;
dev->mac_specified = 0;
parse_mac(dev, mac);
- dev->vhostfd = -1;
-
- for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
- dev->kickfds[i] = -1;
- dev->callfds[i] = -1;
- }
- dev->vhostfd = vhost_user_setup(dev->path);
- if (dev->vhostfd < 0) {
+ if (virtio_user_dev_setup(dev) < 0) {
PMD_INIT_LOG(ERR, "backend set up fails");
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_SET_OWNER, NULL) < 0) {
+ if (dev->ops->send_request(dev, VHOST_USER_SET_OWNER, NULL) < 0) {
PMD_INIT_LOG(ERR, "set_owner fails: %s", strerror(errno));
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_GET_FEATURES,
+ if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
&dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
@@ -288,9 +311,9 @@ virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs)
}
for (i = 0; i < q_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 1);
+ ret |= dev->ops->enable_qp(dev, i, 1);
for (i = q_pairs; i < dev->max_queue_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+ ret |= dev->ops->enable_qp(dev, i, 0);
dev->queue_pairs = q_pairs;
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 28fc788..9f2f82e 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -37,9 +37,13 @@
#include <limits.h>
#include "../virtio_pci.h"
#include "../virtio_ring.h"
+#include "vhost.h"
struct virtio_user_dev {
+ /* for vhost_user backend */
int vhostfd;
+
+ /* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int kickfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int mac_specified;
@@ -54,6 +58,7 @@ struct virtio_user_dev {
uint8_t mac_addr[ETHER_ADDR_LEN];
char path[PATH_MAX];
struct vring vrings[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
+ struct virtio_user_backend_ops *ops;
};
int virtio_user_start_device(struct virtio_user_dev *dev);
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v3 5/7] net/virtio_user: add vhost kernel support
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
` (3 preceding siblings ...)
2017-01-04 3:59 ` [PATCH v3 4/7] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
@ 2017-01-04 3:59 ` Jianfeng Tan
2017-01-04 6:13 ` Yuanhan Liu
2017-01-04 3:59 ` [PATCH v3 6/7] net/virtio_user: enable offloading Jianfeng Tan
` (2 subsequent siblings)
7 siblings, 1 reply; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-04 3:59 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
This patch add support vhost kernel as the backend for virtio_user.
Three main hook functions are added:
- vhost_kernel_setup() to open char device, each vq pair needs one
vhostfd;
- vhost_kernel_ioctl() to communicate control messages with vhost
kernel module;
- vhost_kernel_enable_queue_pair() to open tap device and set it
as the backend of corresonding vhost fd (that is to say, vq pair).
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
doc/guides/rel_notes/release_17_02.rst | 20 ++
drivers/net/virtio/Makefile | 1 +
drivers/net/virtio/virtio_user/vhost.h | 2 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 373 +++++++++++++++++++++++
drivers/net/virtio/virtio_user/virtio_user_dev.c | 21 +-
drivers/net/virtio/virtio_user/virtio_user_dev.h | 6 +
6 files changed, 420 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
diff --git a/doc/guides/rel_notes/release_17_02.rst b/doc/guides/rel_notes/release_17_02.rst
index 180af82..7354df5 100644
--- a/doc/guides/rel_notes/release_17_02.rst
+++ b/doc/guides/rel_notes/release_17_02.rst
@@ -52,6 +52,26 @@ New Features
See the :ref:`Generic flow API <Generic_flow_API>` documentation for more
information.
+* **virtio_user with vhost-kernel as another exceptional path.**
+
+ Previously, we upstreamed a virtual device, virtio_user with vhost-user
+ as the backend, as a way for IPC (Inter-Process Communication) and user
+ space container networking.
+
+ Virtio_user with vhost-kernel as the backend is a solution for exceptional
+ path, such as KNI, which exchanges packets with kernel networking stack.
+ This solution is very promising in:
+
+ * maintenance: vhost and vhost-net (kernel) is upstreamed and extensively
+ used kernel module.
+ * features: vhost-net is born to be a networking solution, which has
+ lots of networking related featuers, like multi queue, tso, multi-seg
+ mbuf, etc.
+ * performance: similar to KNI, this solution would use one or more
+ kthreads to send/receive packets from user space DPDK applications,
+ which has little impact on user space polling thread (except that
+ it might enter into kernel space to wake up those kthreads if
+ necessary).
Resolved Issues
---------------
diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile
index 97972a6..faeffb2 100644
--- a/drivers/net/virtio/Makefile
+++ b/drivers/net/virtio/Makefile
@@ -60,6 +60,7 @@ endif
ifeq ($(CONFIG_RTE_VIRTIO_USER),y)
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_user.c
+SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_kernel.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/virtio_user_dev.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user_ethdev.c
endif
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 515e4fc..5c983bd 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -118,4 +118,6 @@ struct virtio_user_backend_ops {
};
struct virtio_user_backend_ops ops_user;
+struct virtio_user_backend_ops ops_kernel;
+
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
new file mode 100644
index 0000000..1e7cdef
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -0,0 +1,373 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <net/if.h>
+#include <string.h>
+#include <errno.h>
+
+#include <rte_memory.h>
+#include <rte_eal_memconfig.h>
+
+#include "vhost.h"
+#include "virtio_user_dev.h"
+
+struct vhost_memory_kernel {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[0];
+};
+
+/* vhost kernel ioctls */
+#define VHOST_VIRTIO 0xAF
+#define VHOST_GET_FEATURES _IOR(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_FEATURES _IOW(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
+#define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
+#define VHOST_SET_MEM_TABLE _IOW(VHOST_VIRTIO, 0x03, struct vhost_memory_kernel)
+#define VHOST_SET_LOG_BASE _IOW(VHOST_VIRTIO, 0x04, __u64)
+#define VHOST_SET_LOG_FD _IOW(VHOST_VIRTIO, 0x07, int)
+#define VHOST_SET_VRING_NUM _IOW(VHOST_VIRTIO, 0x10, struct vhost_vring_state)
+#define VHOST_SET_VRING_ADDR _IOW(VHOST_VIRTIO, 0x11, struct vhost_vring_addr)
+#define VHOST_SET_VRING_BASE _IOW(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_GET_VRING_BASE _IOWR(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_SET_VRING_KICK _IOW(VHOST_VIRTIO, 0x20, struct vhost_vring_file)
+#define VHOST_SET_VRING_CALL _IOW(VHOST_VIRTIO, 0x21, struct vhost_vring_file)
+#define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
+#define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct vhost_vring_file)
+
+/* TUN ioctls */
+#define TUNSETIFF _IOW('T', 202, int)
+#define TUNGETFEATURES _IOR('T', 207, unsigned int)
+#define TUNSETOFFLOAD _IOW('T', 208, unsigned int)
+#define TUNGETIFF _IOR('T', 210, unsigned int)
+#define TUNSETSNDBUF _IOW('T', 212, int)
+#define TUNGETVNETHDRSZ _IOR('T', 215, int)
+#define TUNSETVNETHDRSZ _IOW('T', 216, int)
+#define TUNSETQUEUE _IOW('T', 217, int)
+#define TUNSETVNETLE _IOW('T', 220, int)
+#define TUNSETVNETBE _IOW('T', 222, int)
+
+/* TUNSETIFF ifr flags */
+#define IFF_TAP 0x0002
+#define IFF_NO_PI 0x1000
+#define IFF_ONE_QUEUE 0x2000
+#define IFF_VNET_HDR 0x4000
+#define IFF_MULTI_QUEUE 0x0100
+#define IFF_ATTACH_QUEUE 0x0200
+#define IFF_DETACH_QUEUE 0x0400
+
+/* Constants */
+#define TUN_DEF_SNDBUF (1ull << 20)
+#define PATH_NET_TUN "/dev/net/tun"
+#define VHOST_KERNEL_MAX_REGIONS 64
+
+static uint64_t vhost_req_user_to_kernel[] = {
+ [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
+ [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
+ [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
+ [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
+ [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
+ [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
+ [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
+ [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
+ [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
+ [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
+ [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
+};
+
+/* By default, vhost kernel module allows 64 regions, but DPDK allows
+ * 256 segments. As a relief, below function merges those virtually
+ * adjacent memsegs into one region.
+ */
+static struct vhost_memory_kernel *
+prepare_vhost_memory_kernel(void)
+{
+ uint32_t i, j, k = 0;
+ struct rte_memseg *seg;
+ struct vhost_memory_region *mr;
+ struct vhost_memory_kernel *vm;
+
+ vm = malloc(sizeof(struct vhost_memory_kernel) +
+ VHOST_KERNEL_MAX_REGIONS *
+ sizeof(struct vhost_memory_region));
+
+ for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
+ seg = &rte_eal_get_configuration()->mem_config->memseg[i];
+ if (!seg->addr)
+ break;
+
+ int new_region = 1;
+
+ for (j = 0; j < k; ++j) {
+ mr = &vm->regions[j];
+
+ if (mr->userspace_addr + mr->memory_size ==
+ (uint64_t)(uintptr_t)seg->addr) {
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+
+ if ((uint64_t)(uintptr_t)seg->addr + seg->len ==
+ mr->userspace_addr) {
+ mr->guest_phys_addr =
+ (uint64_t)(uintptr_t)seg->addr;
+ mr->userspace_addr =
+ (uint64_t)(uintptr_t)seg->addr;
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+ }
+
+ if (new_region == 0)
+ continue;
+
+ mr = &vm->regions[k++];
+ /* use vaddr here! */
+ mr->guest_phys_addr = (uint64_t)(uintptr_t)seg->addr;
+ mr->userspace_addr = (uint64_t)(uintptr_t)seg->addr;
+ mr->memory_size = seg->len;
+ mr->mmap_offset = 0;
+
+ if (k >= VHOST_KERNEL_MAX_REGIONS) {
+ free(vm);
+ return NULL;
+ }
+ }
+
+ vm->nregions = k;
+ vm->padding = 0;
+ return vm;
+}
+
+static int
+vhost_kernel_ioctl(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg)
+{
+ int i, ret = -1;
+ uint64_t req_kernel;
+ struct vhost_memory_kernel *vm = NULL;
+
+ PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
+
+ req_kernel = vhost_req_user_to_kernel[req];
+
+ if (req_kernel == VHOST_SET_MEM_TABLE) {
+ vm = prepare_vhost_memory_kernel();
+ if (!vm)
+ return -1;
+ arg = (void *)vm;
+ }
+
+ /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
+ if (req_kernel == VHOST_SET_FEATURES)
+ *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+
+ for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
+ if (dev->vhostfds[i] < 0)
+ continue;
+
+ ret = ioctl(dev->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+
+ if (vm)
+ free(vm);
+
+ if (ret < 0)
+ PMD_DRV_LOG(ERR, "%s failed: %s",
+ vhost_msg_strings[req], strerror(errno));
+
+ return ret;
+}
+
+/**
+ * Set up environment to talk with a vhost kernel backend.
+ *
+ * @return
+ * - (-1) if fail to set up;
+ * - (>=0) if successful.
+ */
+static int
+vhost_kernel_setup(struct virtio_user_dev *dev)
+{
+ int vhostfd;
+ uint32_t i;
+
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ vhostfd = open(dev->path, O_RDWR);
+ if (vhostfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s, %s",
+ dev->path, strerror(errno));
+ return -1;
+ }
+
+ dev->vhostfds[i] = vhostfd;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_set_backend(int vhostfd, int tapfd)
+{
+ struct vhost_vring_file f;
+
+ f.fd = tapfd;
+ f.index = 0;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ f.index = 1;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable)
+{
+ unsigned int tap_features;
+ int sndbuf = TUN_DEF_SNDBUF;
+ struct ifreq ifr;
+ int hdr_size;
+ int vhostfd;
+ int tapfd;
+
+ vhostfd = dev->vhostfds[pair_idx];
+
+ if (!enable) {
+ if (dev->tapfds[pair_idx]) {
+ close(dev->tapfds[pair_idx]);
+ dev->tapfds[pair_idx] = -1;
+ }
+ return vhost_kernel_set_backend(vhostfd, -1);
+ } else if (dev->tapfds[pair_idx] >= 0) {
+ return 0;
+ }
+
+ if ((dev->features & (1ULL << VIRTIO_NET_F_MRG_RXBUF)) ||
+ (dev->features & (1ULL << VIRTIO_F_VERSION_1)))
+ hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
+ else
+ hdr_size = sizeof(struct virtio_net_hdr);
+
+ /* TODO:
+ * 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
+ * 2. get number of memory regions from vhost module parameter
+ * max_mem_regions, supported in newer version linux kernel
+ */
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ /* Construct ifr */
+ memset(&ifr, 0, sizeof(ifr));
+ ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
+
+ if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ goto error;
+ }
+ if (tap_features & IFF_ONE_QUEUE)
+ ifr.ifr_flags |= IFF_ONE_QUEUE;
+
+ /* Let tap instead of vhost-net handle vnet header, as the latter does
+ * not support offloading. And in this case, we should not set feature
+ * bit VHOST_NET_F_VIRTIO_NET_HDR.
+ */
+ if (tap_features & IFF_VNET_HDR) {
+ ifr.ifr_flags |= IFF_VNET_HDR;
+ } else {
+ PMD_DRV_LOG(ERR, "TAP does not support IFF_VNET_HDR");
+ goto error;
+ }
+
+ if (dev->ifname)
+ strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
+ else
+ strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
+ if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
+ PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ fcntl(tapfd, F_SETFL, O_NONBLOCK);
+
+ if (ioctl(tapfd, TUNSETVNETHDRSZ, &hdr_size) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETVNETHDRSZ failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (ioctl(tapfd, TUNSETSNDBUF, &sndbuf) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETSNDBUF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (vhost_kernel_set_backend(vhostfd, tapfd) < 0)
+ goto error;
+
+ dev->tapfds[pair_idx] = tapfd;
+ if (!dev->ifname)
+ dev->ifname = strdup(ifr.ifr_name);
+
+ return 0;
+error:
+ return -1;
+}
+
+struct virtio_user_backend_ops ops_kernel = {
+ .setup = vhost_kernel_setup,
+ .send_request = vhost_kernel_ioctl,
+ .enable_qp = vhost_kernel_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 32039a1..c40b77e 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -192,6 +192,9 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
for (i = 0; i < dev->max_queue_pairs; ++i)
dev->ops->enable_qp(dev, i, 0);
+ free(dev->ifname);
+ dev->ifname = NULL;
+
return 0;
}
@@ -230,7 +233,7 @@ is_vhost_user_by_type(const char *path)
static int
virtio_user_dev_setup(struct virtio_user_dev *dev)
{
- uint32_t i;
+ uint32_t i, q;
dev->vhostfd = -1;
for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
@@ -238,12 +241,18 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
dev->callfds[i] = -1;
}
+ for (q = 0; q < VHOST_KERNEL_MAX_QUEUES; ++q) {
+ dev->vhostfds[q] = -1;
+ dev->tapfds[q] = -1;
+ }
+
if (is_vhost_user_by_type(dev->path)) {
dev->ops = &ops_user;
- return dev->ops->setup(dev);
+ } else {
+ dev->ops = &ops_kernel;
}
- return -1;
+ return dev->ops->setup(dev);
}
int
@@ -295,7 +304,13 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
void
virtio_user_dev_uninit(struct virtio_user_dev *dev)
{
+ uint32_t i;
+
+ virtio_user_stop_device(dev);
+
close(dev->vhostfd);
+ for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i)
+ close(dev->vhostfds[i]);
}
static uint8_t
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 9f2f82e..148b2e6 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -43,6 +43,12 @@ struct virtio_user_dev {
/* for vhost_user backend */
int vhostfd;
+ /* for vhost_kernel backend */
+ char *ifname;
+#define VHOST_KERNEL_MAX_QUEUES 8
+ int vhostfds[VHOST_KERNEL_MAX_QUEUES];
+ int tapfds[VHOST_KERNEL_MAX_QUEUES];
+
/* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int kickfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v3 6/7] net/virtio_user: enable offloading
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
` (4 preceding siblings ...)
2017-01-04 3:59 ` [PATCH v3 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
@ 2017-01-04 3:59 ` Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 7/7] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
2017-01-09 14:06 ` [PATCH v3 0/7] virtio_user as an alternative exception path Bruce Richardson
7 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-04 3:59 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
When used with vhost kernel backend, we can offload at both directions.
- From vhost kernel to virtio_user, the offload is enabled so that
DPDK app can trust the flow is checksum-correct; and if DPDK app
sends it through another port, the checksum needs to be
recalculated or offloaded. It also applies to TSO.
- From virtio_user to vhost_kernel, the offload is enabled so that
kernel can trust the flow is L4-checksum-correct, no need to verify
it; if kernel will consume it, DPDK app should make sure the
l3-checksum is correctly set.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 61 ++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 2 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 1e7cdef..bdb4af2 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -91,6 +91,13 @@ struct vhost_memory_kernel {
#define IFF_ATTACH_QUEUE 0x0200
#define IFF_DETACH_QUEUE 0x0400
+/* Features for GSO (TUNSETOFFLOAD). */
+#define TUN_F_CSUM 0x01 /* You can hand me unchecksummed packets. */
+#define TUN_F_TSO4 0x02 /* I can handle TSO for IPv4 packets */
+#define TUN_F_TSO6 0x04 /* I can handle TSO for IPv6 packets */
+#define TUN_F_TSO_ECN 0x08 /* I can handle TSO with ECN bits. */
+#define TUN_F_UFO 0x10 /* I can handle UFO packets */
+
/* Constants */
#define TUN_DEF_SNDBUF (1ull << 20)
#define PATH_NET_TUN "/dev/net/tun"
@@ -176,6 +183,28 @@ prepare_vhost_memory_kernel(void)
return vm;
}
+/* with below features, vhost kernel does not need to do the checksum and TSO,
+ * these info will be passed to virtio_user through virtio net header.
+ */
+#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
+ ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
+ (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
+ (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
+ (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
+ (1ULL << VIRTIO_NET_F_GUEST_UFO))
+
+/* with below features, when flows from virtio_user to vhost kernel
+ * (1) if flows goes up through the kernel networking stack, it does not need
+ * to verify checksum, which can save CPU cycles;
+ * (2) if flows goes through a Linux bridge and outside from an interface
+ * (kernel driver), checksum and TSO will be done by GSO in kernel or even
+ * offloaded into real physical device.
+ */
+#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
+ ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
+ (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
+ (1ULL << VIRTIO_NET_F_CSUM))
+
static int
vhost_kernel_ioctl(struct virtio_user_dev *dev,
enum vhost_user_request req,
@@ -196,10 +225,15 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
arg = (void *)vm;
}
- /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
- if (req_kernel == VHOST_SET_FEATURES)
+ if (req_kernel == VHOST_SET_FEATURES) {
+ /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
*(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+ /* VHOST kernel does not know about below flags */
+ *(uint64_t *)arg &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
+ *(uint64_t *)arg &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
+ }
+
for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
if (dev->vhostfds[i] < 0)
continue;
@@ -209,6 +243,15 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
break;
}
+ if (!ret && req_kernel == VHOST_GET_FEATURES) {
+ /* with tap as the backend, all these features are supported
+ * but not claimed by vhost-net, so we add them back when
+ * reporting to upper layer.
+ */
+ *((uint64_t *)arg) |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
+ *((uint64_t *)arg) |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
+ }
+
if (vm)
free(vm);
@@ -280,6 +323,12 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
int hdr_size;
int vhostfd;
int tapfd;
+ unsigned int offload =
+ TUN_F_CSUM |
+ TUN_F_TSO4 |
+ TUN_F_TSO6 |
+ TUN_F_TSO_ECN |
+ TUN_F_UFO;
vhostfd = dev->vhostfds[pair_idx];
@@ -354,6 +403,14 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
goto error;
}
+ /* TODO: before set the offload capabilities, we'd better (1) check
+ * negotiated features to see if necessary to offload; (2) query tap
+ * to see if it supports the offload capabilities.
+ */
+ if (ioctl(tapfd, TUNSETOFFLOAD, offload) != 0)
+ PMD_DRV_LOG(ERR, "TUNSETOFFLOAD ioctl() failed: %s",
+ strerror(errno));
+
if (vhost_kernel_set_backend(vhostfd, tapfd) < 0)
goto error;
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v3 7/7] net/virtio_user: enable multiqueue with vhost kernel
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
` (5 preceding siblings ...)
2017-01-04 3:59 ` [PATCH v3 6/7] net/virtio_user: enable offloading Jianfeng Tan
@ 2017-01-04 3:59 ` Jianfeng Tan
2017-01-09 14:06 ` [PATCH v3 0/7] virtio_user as an alternative exception path Bruce Richardson
7 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-04 3:59 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
With vhost kernel, to enable multiqueue, we need backend device
in kernel support multiqueue feature. Specifically, with tap
as the backend, as linux/Documentation/networking/tuntap.txt shows,
we check if tap supports IFF_MULTI_QUEUE feature.
And for vhost kernel, each queue pair has a vhost fd, and with a tap
fd binding this vhost fd. All tap fds are set with the same tap
interface name.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 69 +++++++++++++++++++++---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 1 +
2 files changed, 64 insertions(+), 6 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index bdb4af2..023bdf8 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -206,6 +206,29 @@ prepare_vhost_memory_kernel(void)
(1ULL << VIRTIO_NET_F_CSUM))
static int
+tap_supporte_mq(void)
+{
+ int tapfd;
+ unsigned int tap_features;
+
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ close(tapfd);
+ return -1;
+ }
+
+ close(tapfd);
+ return tap_features & IFF_MULTI_QUEUE;
+}
+
+static int
vhost_kernel_ioctl(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg)
@@ -213,6 +236,8 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
int i, ret = -1;
uint64_t req_kernel;
struct vhost_memory_kernel *vm = NULL;
+ int vhostfd;
+ unsigned int queue_sel;
PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
@@ -232,15 +257,37 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
/* VHOST kernel does not know about below flags */
*(uint64_t *)arg &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
*(uint64_t *)arg &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
+
+ *(uint64_t *)arg &= ~(1ULL << VIRTIO_NET_F_MQ);
}
- for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
- if (dev->vhostfds[i] < 0)
- continue;
+ switch (req_kernel) {
+ case VHOST_SET_VRING_NUM:
+ case VHOST_SET_VRING_ADDR:
+ case VHOST_SET_VRING_BASE:
+ case VHOST_GET_VRING_BASE:
+ case VHOST_SET_VRING_KICK:
+ case VHOST_SET_VRING_CALL:
+ queue_sel = *(unsigned int *)arg;
+ vhostfd = dev->vhostfds[queue_sel / 2];
+ *(unsigned int *)arg = queue_sel % 2;
+ PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
+ vhostfd, *(unsigned int *)arg);
+ break;
+ default:
+ vhostfd = -1;
+ }
+ if (vhostfd == -1) {
+ for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
+ if (dev->vhostfds[i] < 0)
+ continue;
- ret = ioctl(dev->vhostfds[i], req_kernel, arg);
- if (ret < 0)
- break;
+ ret = ioctl(dev->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+ } else {
+ ret = ioctl(vhostfd, req_kernel, arg);
}
if (!ret && req_kernel == VHOST_GET_FEATURES) {
@@ -250,6 +297,12 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
*/
*((uint64_t *)arg) |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
*((uint64_t *)arg) |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
+
+ /* vhost_kernel will not declare this feature, but it does
+ * support multi-queue.
+ */
+ if (tap_supporte_mq())
+ *(uint64_t *)arg |= (1ull << VIRTIO_NET_F_MQ);
}
if (vm)
@@ -329,6 +382,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
TUN_F_TSO6 |
TUN_F_TSO_ECN |
TUN_F_UFO;
+ int req_mq = (dev->max_queue_pairs > 1);
vhostfd = dev->vhostfds[pair_idx];
@@ -382,6 +436,9 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
goto error;
}
+ if (req_mq)
+ ifr.ifr_flags |= IFF_MULTI_QUEUE;
+
if (dev->ifname)
strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
else
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index c40b77e..2d9d989 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -93,6 +93,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.num = vring->num;
dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
+ state.index = queue_sel;
state.num = 0; /* no reservation */
dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: [PATCH v3 2/7] net/virtio_user: fix not properly reset device
2017-01-04 3:59 ` [PATCH v3 2/7] net/virtio_user: fix not properly reset device Jianfeng Tan
@ 2017-01-04 5:46 ` Yuanhan Liu
0 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-04 5:46 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang, stable
On Wed, Jan 04, 2017 at 03:59:21AM +0000, Jianfeng Tan wrote:
> virtio_user is not properly reset when users call vtpci_reset(),
> as it ignores VIRTIO_CONFIG_STATUS_RESET status in
> virtio_user_set_status().
>
> This might lead to initialization failure as it starts to re-init
> the device before sending RESET messege to backend. Besides, previous
> callfds and kickfds are not closed.
>
> To fix it, we add support to disable virtqueues when it's set to
> DRIVER OK status, and re-init fields in struct virtio_user_dev.
>
> Fixes: e9efa4d93821 ("net/virtio-user: add new virtual PCI driver")
> Fixes: 37a7eb2ae816 ("net/virtio-user: add device emulation layer")
>
> CC: stable@dpdk.org
>
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Note that, typically, there should be no empty line between 'Cc' and SoB.
> ---
> drivers/net/virtio/virtio_user/virtio_user_dev.c | 26 ++++++++++++++++--------
> drivers/net/virtio/virtio_user_ethdev.c | 15 ++++++++------
> 2 files changed, 27 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index 0d7e17b..a38398b 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -182,7 +182,17 @@ virtio_user_start_device(struct virtio_user_dev *dev)
>
> int virtio_user_stop_device(struct virtio_user_dev *dev)
The name doesn't seem to be well named: "dev_stop" comes to my firstly
when I saw that :/
Rename it to "xxx_reset_device"?
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v3 3/7] net/virtio_user: move vhost user specific code
2017-01-04 3:59 ` [PATCH v3 3/7] net/virtio_user: move vhost user specific code Jianfeng Tan
@ 2017-01-04 6:02 ` Yuanhan Liu
2017-01-04 6:46 ` Tan, Jianfeng
0 siblings, 1 reply; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-04 6:02 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Wed, Jan 04, 2017 at 03:59:22AM +0000, Jianfeng Tan wrote:
> To support vhost kernel as the backend of net_virtio_user in coming
> patches, we move vhost_user specific structs and macros into
> vhost_user.c, and only keep common definitions in vhost.h.
>
> Besides, remove VHOST_USER_MQ feature check.
Again, I have to ask, why? You don't only remove the check, also, you
removed this feature setting, which seems to break the MQ support?
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v3 4/7] net/virtio_user: abstract virtio user backend ops
2017-01-04 3:59 ` [PATCH v3 4/7] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
@ 2017-01-04 6:11 ` Yuanhan Liu
0 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-04 6:11 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Wed, Jan 04, 2017 at 03:59:23AM +0000, Jianfeng Tan wrote:
> +struct virtio_user_backend_ops ops_user;
Better to qualify it with "extern const" ...
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v3 5/7] net/virtio_user: add vhost kernel support
2017-01-04 3:59 ` [PATCH v3 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
@ 2017-01-04 6:13 ` Yuanhan Liu
0 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-04 6:13 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Wed, Jan 04, 2017 at 03:59:24AM +0000, Jianfeng Tan wrote:
> +static int
> +vhost_kernel_ioctl(struct virtio_user_dev *dev,
> + enum vhost_user_request req,
> + void *arg)
> +{
> + int i, ret = -1;
> + uint64_t req_kernel;
> + struct vhost_memory_kernel *vm = NULL;
> +
> + PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
> +
> + req_kernel = vhost_req_user_to_kernel[req];
> +
> + if (req_kernel == VHOST_SET_MEM_TABLE) {
> + vm = prepare_vhost_memory_kernel();
> + if (!vm)
> + return -1;
> + arg = (void *)vm;
> + }
> +
> + /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
> + if (req_kernel == VHOST_SET_FEATURES)
> + *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
You missed my comments in last version?
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v3 3/7] net/virtio_user: move vhost user specific code
2017-01-04 6:02 ` Yuanhan Liu
@ 2017-01-04 6:46 ` Tan, Jianfeng
2017-01-04 7:08 ` Yuanhan Liu
0 siblings, 1 reply; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-04 6:46 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: dev, Yigit, Ferruh, Liang, Cunming
Hi Yuanhan,
> -----Original Message-----
> From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
> Sent: Wednesday, January 4, 2017 2:03 PM
> To: Tan, Jianfeng
> Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
> Subject: Re: [PATCH v3 3/7] net/virtio_user: move vhost user specific code
>
> On Wed, Jan 04, 2017 at 03:59:22AM +0000, Jianfeng Tan wrote:
> > To support vhost kernel as the backend of net_virtio_user in coming
> > patches, we move vhost_user specific structs and macros into
> > vhost_user.c, and only keep common definitions in vhost.h.
> >
> > Besides, remove VHOST_USER_MQ feature check.
>
> Again, I have to ask, why? You don't only remove the check, also, you
> removed this feature setting, which seems to break the MQ support?
I have answered it here:
http://dpdk.org/ml/archives/dev/2016-December/053520.html
To be more clear, VHOST_USER_MQ is a not-well-defined macro: #define VHOST_USER_MQ (1ULL << VHOST_USER_F_PROTOCOL_FEATURES),
which is a feature bit in vhost user protocol.
According to QEMU/ docs/specs/vhost-user.txt, "If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, the ring is initialized in an enabled state. "
But our DPDK vhost library does not take care of this feature bit. Just make this as default: the ring is initialized in an disabled state. And our virtio_user with vhost-user does send VHOST_USER_SET_VRING_ENABLE to enable each queue pair.
So I think it's not necessary to add it back.
How do you think?
Thanks,
Jianfeng
>
> --yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v3 3/7] net/virtio_user: move vhost user specific code
2017-01-04 6:46 ` Tan, Jianfeng
@ 2017-01-04 7:08 ` Yuanhan Liu
0 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-04 7:08 UTC (permalink / raw)
To: Tan, Jianfeng; +Cc: dev, Yigit, Ferruh, Liang, Cunming
On Wed, Jan 04, 2017 at 06:46:34AM +0000, Tan, Jianfeng wrote:
> Hi Yuanhan,
>
> > -----Original Message-----
> > From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com]
> > Sent: Wednesday, January 4, 2017 2:03 PM
> > To: Tan, Jianfeng
> > Cc: dev@dpdk.org; Yigit, Ferruh; Liang, Cunming
> > Subject: Re: [PATCH v3 3/7] net/virtio_user: move vhost user specific code
> >
> > On Wed, Jan 04, 2017 at 03:59:22AM +0000, Jianfeng Tan wrote:
> > > To support vhost kernel as the backend of net_virtio_user in coming
> > > patches, we move vhost_user specific structs and macros into
> > > vhost_user.c, and only keep common definitions in vhost.h.
> > >
> > > Besides, remove VHOST_USER_MQ feature check.
> >
> > Again, I have to ask, why? You don't only remove the check, also, you
> > removed this feature setting, which seems to break the MQ support?
>
> I have answered it here:
> http://dpdk.org/ml/archives/dev/2016-December/053520.html
I thought we have made some agreements :/
>
> To be more clear, VHOST_USER_MQ is a not-well-defined macro: #define VHOST_USER_MQ (1ULL << VHOST_USER_F_PROTOCOL_FEATURES),
> which is a feature bit in vhost user protocol.
Yes, it's again named wrongly.
> According to QEMU/ docs/specs/vhost-user.txt, "If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, the ring is initialized in an enabled state. "
>
> But our DPDK vhost library does not take care of this feature bit.
> Just make this as default: the ring is initialized in an disabled state. And our virtio_user with vhost-user does send VHOST_USER_SET_VRING_ENABLE to enable each queue pair.
VHOST_USER_F_PROTOCOL_FEATURES is a fundamental feature for quite many
vhost-user extended features, including the MQ. If it's not set, the MQ
should not work.
It may still work in your case, becase you made an assumtion that the
vhost backend supports the MQ feature (which is true in nowadays, as
the feature has been there for a quite while). However, that's not an
assumtion you can take while adding the vhost-user MQ support at that
time. And such feature bit (including the PROTOCOL_F_MQ) makes sure
that we will not try to enable MQ with and older vhost backend that
doesn't have the support.
Put simply, this feature is needed, and as the feature name states,
it's needed only for vhost-user.
--yliu
>
> So I think it's not necessary to add it back.
>
> How do you think?
>
> Thanks,
> Jianfeng
>
> >
> > --yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2016-12-26 7:44 ` Yuanhan Liu
@ 2017-01-04 7:22 ` Tan, Jianfeng
2017-01-04 7:46 ` Yuanhan Liu
2017-01-09 4:51 ` Jason Wang
0 siblings, 2 replies; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-04 7:22 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: dev, ferruh.yigit, cunming.liang
Sorry, I forget to reply this comment.
On 12/26/2016 3:44 PM, Yuanhan Liu wrote:
> [...]
>> +
>> + /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
> Because this feature need the vhost IOTLB support from the device
> emulation. Patches for QEMU hasn't been merged yet, but it has been
> there for a while.
Yes. And it's in need of help from QEMU.
>
> Since we don't have the support yet, for sure it won't work. But
> I'm wondering why you have to disable it explicitly?
Here we do not have QEMU. Frontend driver talks with vhost-net through
virtio_user_dev. And both ends claim to support VIRTIO_F_IOMMU_PLATFORM.
So this feature bit will be negotiated if we don't explicitly disable
it. In my previous test, it fails in vhost_init_device_iotlb() of vhost
kernel module. My guess is that, for this feature, without the help of
QEMU, vhost kernel module cannot work independently.
Thanks,
Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-04 7:22 ` Tan, Jianfeng
@ 2017-01-04 7:46 ` Yuanhan Liu
2017-01-09 4:51 ` Jason Wang
1 sibling, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-04 7:46 UTC (permalink / raw)
To: Tan, Jianfeng; +Cc: dev, ferruh.yigit, cunming.liang
On Wed, Jan 04, 2017 at 03:22:08PM +0800, Tan, Jianfeng wrote:
>
> Sorry, I forget to reply this comment.
>
> On 12/26/2016 3:44 PM, Yuanhan Liu wrote:
> >[...]
> >>+
> >>+ /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
> >Because this feature need the vhost IOTLB support from the device
> >emulation. Patches for QEMU hasn't been merged yet, but it has been
> >there for a while.
>
> Yes. And it's in need of help from QEMU.
>
> >
> >Since we don't have the support yet, for sure it won't work. But
> >I'm wondering why you have to disable it explicitly?
>
> Here we do not have QEMU. Frontend driver talks with vhost-net through
> virtio_user_dev. And both ends claim to support VIRTIO_F_IOMMU_PLATFORM. So
> this feature bit will be negotiated if we don't explicitly disable it. In my
> previous test, it fails in vhost_init_device_iotlb() of vhost kernel module.
> My guess is that, for this feature, without the help of QEMU, vhost kernel
> module cannot work independently.
The negotiation is a work between the driver, the device and the vhost
backend. If the device doesn't claim to support it, how could it be
nenogitated.
Think that way, does old QEMU (that doesn't support this feature) disable this
feature explicitly? No. Then why do we have to do that?
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2016-12-23 7:14 ` [PATCH v2 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
2016-12-26 7:44 ` Yuanhan Liu
@ 2017-01-09 4:39 ` Jason Wang
2017-01-10 6:11 ` Tan, Jianfeng
2017-01-11 2:30 ` Tan, Jianfeng
1 sibling, 2 replies; 73+ messages in thread
From: Jason Wang @ 2017-01-09 4:39 UTC (permalink / raw)
To: Jianfeng Tan, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
On 2016年12月23日 15:14, Jianfeng Tan wrote:
> This patch add support vhost kernel as the backend for virtio_user.
> Three main hook functions are added:
> - vhost_kernel_setup() to open char device, each vq pair needs one
> vhostfd;
> - vhost_kernel_ioctl() to communicate control messages with vhost
> kernel module;
> - vhost_kernel_enable_queue_pair() to open tap device and set it
> as the backend of corresonding vhost fd (that is to say, vq pair).
>
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> ---
> drivers/net/virtio/Makefile | 1 +
> drivers/net/virtio/virtio_user/vhost.h | 2 +
> drivers/net/virtio/virtio_user/vhost_kernel.c | 364 +++++++++++++++++++++++
> drivers/net/virtio/virtio_user/virtio_user_dev.c | 21 +-
> drivers/net/virtio/virtio_user/virtio_user_dev.h | 4 +
> 5 files changed, 388 insertions(+), 4 deletions(-)
> create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
>
> diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile
> index 97972a6..faeffb2 100644
> --- a/drivers/net/virtio/Makefile
> +++ b/drivers/net/virtio/Makefile
> @@ -60,6 +60,7 @@ endif
>
> ifeq ($(CONFIG_RTE_VIRTIO_USER),y)
> SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_user.c
> +SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_kernel.c
> SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/virtio_user_dev.c
> SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user_ethdev.c
> endif
> diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
> index bd67133..ffab13a 100644
> --- a/drivers/net/virtio/virtio_user/vhost.h
> +++ b/drivers/net/virtio/virtio_user/vhost.h
> @@ -120,4 +120,6 @@ struct virtio_user_backend_ops {
> };
>
> struct virtio_user_backend_ops ops_user;
> +struct virtio_user_backend_ops ops_kernel;
> +
> #endif
> diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
> new file mode 100644
> index 0000000..8984c5c
> --- /dev/null
> +++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
> @@ -0,0 +1,364 @@
> +/*-
> + * BSD LICENSE
> + *
> + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
> + * All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + * notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + * notice, this list of conditions and the following disclaimer in
> + * the documentation and/or other materials provided with the
> + * distribution.
> + * * Neither the name of Intel Corporation nor the names of its
> + * contributors may be used to endorse or promote products derived
> + * from this software without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#include <unistd.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <sys/ioctl.h>
> +#include <net/if.h>
> +#include <string.h>
> +#include <errno.h>
> +
> +#include <rte_memory.h>
> +#include <rte_eal_memconfig.h>
> +
> +#include "vhost.h"
> +#include "virtio_user_dev.h"
> +
> +struct vhost_memory_kernel {
> + uint32_t nregions;
> + uint32_t padding;
> + struct vhost_memory_region regions[0];
> +};
> +
> +/* vhost kernel ioctls */
> +#define VHOST_VIRTIO 0xAF
> +#define VHOST_GET_FEATURES _IOR(VHOST_VIRTIO, 0x00, __u64)
> +#define VHOST_SET_FEATURES _IOW(VHOST_VIRTIO, 0x00, __u64)
> +#define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
> +#define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
> +#define VHOST_SET_MEM_TABLE _IOW(VHOST_VIRTIO, 0x03, struct vhost_memory_kernel)
> +#define VHOST_SET_LOG_BASE _IOW(VHOST_VIRTIO, 0x04, __u64)
> +#define VHOST_SET_LOG_FD _IOW(VHOST_VIRTIO, 0x07, int)
> +#define VHOST_SET_VRING_NUM _IOW(VHOST_VIRTIO, 0x10, struct vhost_vring_state)
> +#define VHOST_SET_VRING_ADDR _IOW(VHOST_VIRTIO, 0x11, struct vhost_vring_addr)
> +#define VHOST_SET_VRING_BASE _IOW(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
> +#define VHOST_GET_VRING_BASE _IOWR(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
> +#define VHOST_SET_VRING_KICK _IOW(VHOST_VIRTIO, 0x20, struct vhost_vring_file)
> +#define VHOST_SET_VRING_CALL _IOW(VHOST_VIRTIO, 0x21, struct vhost_vring_file)
> +#define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
> +#define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct vhost_vring_file)
> +
> +/* TUN ioctls */
> +#define TUNSETIFF _IOW('T', 202, int)
> +#define TUNGETFEATURES _IOR('T', 207, unsigned int)
> +#define TUNSETOFFLOAD _IOW('T', 208, unsigned int)
> +#define TUNGETIFF _IOR('T', 210, unsigned int)
> +#define TUNSETSNDBUF _IOW('T', 212, int)
> +#define TUNGETVNETHDRSZ _IOR('T', 215, int)
> +#define TUNSETVNETHDRSZ _IOW('T', 216, int)
> +#define TUNSETQUEUE _IOW('T', 217, int)
> +#define TUNSETVNETLE _IOW('T', 220, int)
> +#define TUNSETVNETBE _IOW('T', 222, int)
> +
> +/* TUNSETIFF ifr flags */
> +#define IFF_TAP 0x0002
> +#define IFF_NO_PI 0x1000
> +#define IFF_ONE_QUEUE 0x2000
> +#define IFF_VNET_HDR 0x4000
> +#define IFF_MULTI_QUEUE 0x0100
> +#define IFF_ATTACH_QUEUE 0x0200
> +#define IFF_DETACH_QUEUE 0x0400
Do we really want to duplicate those things which has been exposed by
uapi here?
> +
> +/* Constants */
> +#define TUN_DEF_SNDBUF (1ull << 20)
> +#define PATH_NET_TUN "/dev/net/tun"
> +#define VHOST_KERNEL_MAX_REGIONS 64
Unfortunate not a constant any more since c9ce42f72fd0 vhost: add
max_mem_regions module parameter.
> +
> +static uint64_t vhost_req_user_to_kernel[] = {
> + [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
> + [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
> + [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
> + [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
> + [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
> + [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
> + [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
> + [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
> + [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
> + [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
> + [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
> +};
> +
> +/* By default, vhost kernel module allows 64 regions, but DPDK allows
> + * 256 segments. As a relief, below function merges those virtually
> + * adjacent memsegs into one region.
> + */
> +static struct vhost_memory_kernel *
> +prepare_vhost_memory_kernel(void)
> +{
> + uint32_t i, j, k = 0;
> + struct rte_memseg *seg;
> + struct vhost_memory_region *mr;
> + struct vhost_memory_kernel *vm;
> +
> + vm = malloc(sizeof(struct vhost_memory_kernel) +
> + VHOST_KERNEL_MAX_REGIONS *
> + sizeof(struct vhost_memory_region));
> +
> + for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
> + seg = &rte_eal_get_configuration()->mem_config->memseg[i];
> + if (!seg->addr)
> + break;
If we're sure the number of regions is less than 64(or the module
parameter read from /sys), can we avoid the iteration here?
> +
> + int new_region = 1;
> +
> + for (j = 0; j < k; ++j) {
> + mr = &vm->regions[j];
> +
> + if (mr->userspace_addr + mr->memory_size ==
> + (uint64_t)seg->addr) {
> + mr->memory_size += seg->len;
> + new_region = 0;
> + break;
> + }
> +
> + if ((uint64_t)seg->addr + seg->len ==
> + mr->userspace_addr) {
> + mr->guest_phys_addr = (uint64_t)seg->addr;
> + mr->userspace_addr = (uint64_t)seg->addr;
> + mr->memory_size += seg->len;
> + new_region = 0;
> + break;
> + }
> + }
> +
> + if (new_region == 0)
> + continue;
> +
> + mr = &vm->regions[k++];
> + mr->guest_phys_addr = (uint64_t)seg->addr; /* use vaddr here! */
> + mr->userspace_addr = (uint64_t)seg->addr;
> + mr->memory_size = seg->len;
> + mr->mmap_offset = 0;
> +
> + if (k >= VHOST_KERNEL_MAX_REGIONS) {
> + free(vm);
> + return NULL;
> + }
> + }
> +
> + vm->nregions = k;
> + vm->padding = 0;
> + return vm;
> +}
> +
> +static int
> +vhost_kernel_ioctl(struct virtio_user_dev *dev,
> + enum vhost_user_request req,
> + void *arg)
> +{
> + int i, ret = -1;
> + uint64_t req_kernel;
> + struct vhost_memory_kernel *vm = NULL;
> +
> + req_kernel = vhost_req_user_to_kernel[req];
> +
> + if (req_kernel == VHOST_SET_MEM_TABLE) {
> + vm = prepare_vhost_memory_kernel();
> + if (!vm)
> + return -1;
> + arg = (void *)vm;
> + }
> +
> + /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
I think the reason is when VIRTIO_F_IOMMU_PLATFORM is negotiated, all
address should be iova instead of gpa.
> + if (req_kernel == VHOST_SET_FEATURES)
> + *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
> +
> + for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
> + if (dev->vhostfds[i] < 0)
> + continue;
> +
> + ret = ioctl(dev->vhostfds[i], req_kernel, arg);
> + if (ret < 0)
> + break;
> + }
> +
> + if (vm)
> + free(vm);
> +
> + return ret;
> +}
> +
> +/**
> + * Set up environment to talk with a vhost kernel backend.
> + *
> + * @return
> + * - (-1) if fail to set up;
> + * - (>=0) if successful.
> + */
> +static int
> +vhost_kernel_setup(struct virtio_user_dev *dev)
> +{
> + int vhostfd;
> + uint32_t q;
> +
> + for (q = 0; q < dev->max_queue_pairs; ++q) {
> + vhostfd = open(dev->path, O_RDWR);
> + if (vhostfd < 0) {
> + PMD_DRV_LOG(ERR, "fail to open %s, %s",
> + dev->path, strerror(errno));
> + return -1;
> + }
> +
> + dev->vhostfds[q] = vhostfd;
> + }
> +
> + return 0;
> +}
> +
> +static int
> +vhost_kernel_set_backend(int vhostfd, int tapfd)
> +{
> + struct vhost_vring_file f;
> +
> + f.fd = tapfd;
> + f.index = 0;
> + if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
> + PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
> + strerror(errno));
> + return -1;
> + }
> +
> + f.index = 1;
> + if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
> + PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
> + strerror(errno));
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +static int
> +vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
> + uint16_t pair_idx,
> + int enable)
> +{
> + unsigned int features;
> + int sndbuf = TUN_DEF_SNDBUF;
> + struct ifreq ifr;
> + int hdr_size;
> + int vhostfd;
> + int tapfd;
> +
> + vhostfd = dev->vhostfds[pair_idx];
> +
> + if (!enable) {
> + if (dev->tapfds[pair_idx]) {
> + close(dev->tapfds[pair_idx]);
> + dev->tapfds[pair_idx] = -1;
> + }
> + return vhost_kernel_set_backend(vhostfd, -1);
If this is used to for thing like ethtool -L in guest, we should use
TUNSETQUEUE here.
> + } else if (dev->tapfds[pair_idx] >= 0) {
> + return 0;
> + }
> +
> + if ((dev->features & (1ULL << VIRTIO_NET_F_MRG_RXBUF)) ||
> + (dev->features & (1ULL << VIRTIO_F_VERSION_1)))
> + hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
> + else
> + hdr_size = sizeof(struct virtio_net_hdr);
> +
> + /* TODO:
> + * 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
> + * 2. get number of memory regions from vhost module parameter
> + * max_mem_regions, supported in newer version linux kernel
> + */
> + tapfd = open(PATH_NET_TUN, O_RDWR);
> + if (tapfd < 0) {
> + PMD_DRV_LOG(ERR, "fail to open %s: %s",
> + PATH_NET_TUN, strerror(errno));
> + return -1;
> + }
> +
> + /* Construct ifr */
> + memset(&ifr, 0, sizeof(ifr));
> + ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
> +
> + if (ioctl(tapfd, TUNGETFEATURES, &features) == -1) {
> + PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
> + goto error;
> + }
> + if (features & IFF_ONE_QUEUE)
> + ifr.ifr_flags |= IFF_ONE_QUEUE;
> +
> + /* Let tap instead of vhost-net handle vnet header, as the latter does
> + * not support offloading. And in this case, we should not set feature
> + * bit VHOST_NET_F_VIRTIO_NET_HDR.
> + */
> + if (features & IFF_VNET_HDR) {
> + ifr.ifr_flags |= IFF_VNET_HDR;
> + } else {
> + PMD_DRV_LOG(ERR, "TAP does not support IFF_VNET_HDR");
> + goto error;
> + }
> +
> + if (dev->ifname)
> + strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
> + else
> + strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
> + if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
> + PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
> + goto error;
> + }
This requires CAP_NET_ADMIN, so we should really consider to accept a
pre-created fd here.
> +
> + fcntl(tapfd, F_SETFL, O_NONBLOCK);
> +
> + if (ioctl(tapfd, TUNSETVNETHDRSZ, &hdr_size) < 0) {
> + PMD_DRV_LOG(ERR, "TUNSETVNETHDRSZ failed: %s", strerror(errno));
> + goto error;
> + }
> +
> + if (ioctl(tapfd, TUNSETSNDBUF, &sndbuf) < 0) {
> + PMD_DRV_LOG(ERR, "TUNSETSNDBUF failed: %s", strerror(errno));
> + goto error;
> + }
Let's use INT_MAX as default here to survive from evil consumer here.
> +
> + if (vhost_kernel_set_backend(vhostfd, tapfd) < 0)
> + goto error;
> +
> + dev->tapfds[pair_idx] = tapfd;
> + if (!dev->ifname)
> + dev->ifname = strdup(ifr.ifr_name);
> +
> + return 0;
> +error:
> + return -1;
> +}
> +
> +struct virtio_user_backend_ops ops_kernel = {
> + .setup = vhost_kernel_setup,
> + .send_request = vhost_kernel_ioctl,
> + .enable_qp = vhost_kernel_enable_queue_pair
> +};
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> index a818c29..c718b85 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
> @@ -219,7 +219,7 @@ is_vhost_user_by_type(const char *path)
> static int
> virtio_user_dev_setup(struct virtio_user_dev *dev)
> {
> - uint32_t i;
> + uint32_t i, q;
>
> dev->vhostfd = -1;
> for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
> @@ -227,12 +227,18 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
> dev->callfds[i] = -1;
> }
>
> + for (q = 0; q < VHOST_KERNEL_MAX_QUEUES; ++q) {
> + dev->vhostfds[q] = -1;
> + dev->tapfds[q] = -1;
> + }
> +
> if (is_vhost_user_by_type(dev->path)) {
> dev->ops = &ops_user;
> - return dev->ops->setup(dev);
> + } else {
> + dev->ops = &ops_kernel;
> }
>
> - return -1;
> + return dev->ops->setup(dev);
> }
>
> int
> @@ -284,7 +290,9 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
> void
> virtio_user_dev_uninit(struct virtio_user_dev *dev)
> {
> - uint32_t i;
> + uint32_t i, q;
> +
> + dev->ops->send_request(dev, VHOST_USER_RESET_OWNER, NULL);
>
> for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
> close(dev->callfds[i]);
> @@ -292,6 +300,11 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev)
> }
>
> close(dev->vhostfd);
> +
> + for (q = 0; q < VHOST_KERNEL_MAX_QUEUES; ++q) {
> + close(dev->vhostfds[q]);
> + close(dev->tapfds[q]);
> + }
> }
>
> static uint8_t
> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> index 503a496..148b2e6 100644
> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
> @@ -44,6 +44,10 @@ struct virtio_user_dev {
> int vhostfd;
>
> /* for vhost_kernel backend */
> + char *ifname;
> +#define VHOST_KERNEL_MAX_QUEUES 8
> + int vhostfds[VHOST_KERNEL_MAX_QUEUES];
> + int tapfds[VHOST_KERNEL_MAX_QUEUES];
>
> /* for both vhost_user and vhost_kernel */
> int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-04 7:22 ` Tan, Jianfeng
2017-01-04 7:46 ` Yuanhan Liu
@ 2017-01-09 4:51 ` Jason Wang
1 sibling, 0 replies; 73+ messages in thread
From: Jason Wang @ 2017-01-09 4:51 UTC (permalink / raw)
To: Tan, Jianfeng, Yuanhan Liu; +Cc: dev, ferruh.yigit, cunming.liang
On 2017年01月04日 15:22, Tan, Jianfeng wrote:
>
> Sorry, I forget to reply this comment.
>
> On 12/26/2016 3:44 PM, Yuanhan Liu wrote:
>> [...]
>>> +
>>> + /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
>> Because this feature need the vhost IOTLB support from the device
>> emulation. Patches for QEMU hasn't been merged yet, but it has been
>> there for a while.
>
> Yes. And it's in need of help from QEMU.
>
>>
>> Since we don't have the support yet, for sure it won't work. But
>> I'm wondering why you have to disable it explicitly?
>
> Here we do not have QEMU. Frontend driver talks with vhost-net through
> virtio_user_dev. And both ends claim to support
> VIRTIO_F_IOMMU_PLATFORM. So this feature bit will be negotiated if we
> don't explicitly disable it. In my previous test, it fails in
> vhost_init_device_iotlb() of vhost kernel module.
Interesting, vhost_init_device_iotlb() only fail when OOM. Do you meet it?
> My guess is that, for this feature, without the help of QEMU, vhost
> kernel module cannot work independently.
Technically it can if your userspace supports device IOTLB APIs too.
Or if you're using static mappings, and preset all mappings through
VHOST_IOTLB_UPADATE to make sure no IOTLB misses.
Thanks
>
> Thanks,
> Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v3 0/7] virtio_user as an alternative exception path
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
` (6 preceding siblings ...)
2017-01-04 3:59 ` [PATCH v3 7/7] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
@ 2017-01-09 14:06 ` Bruce Richardson
2017-01-10 8:46 ` Tan, Jianfeng
7 siblings, 1 reply; 73+ messages in thread
From: Bruce Richardson @ 2017-01-09 14:06 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, yuanhan.liu, ferruh.yigit, cunming.liang
On Wed, Jan 04, 2017 at 03:59:19AM +0000, Jianfeng Tan wrote:
> v3:
> - Drop the patch to postpone driver ok sending patch, superseded it
> with a bug fix to disable all virtqueues and re-init the device.
> (you might wonder why not just send reset owner msg. Under my test,
> it causes spinlock deadlock problem when killing the program).
> - Avoid compiling error on 32-bit system for pointer convert.
> - Fix a bug in patch "abstract virtio user backend ops", vhostfd is
> not properly assigned.
> - Fix a "MQ cannot be used" bug in v2, which is related to strip
> some feature bits that vhost kernel does not recognize.
> - Update release note.
>
> v2: (Lots of them are from yuanhan's comment)
> - Add offloding feature.
> - Add multiqueue support.
> - Add a new patch to postpone the sending of driver ok notification.
> - Put fix patch ahead of the whole patch series.
> - Split original 0001 patch into 0003 and 0004 patches.
> - Remove the original vhost_internal design, just add those into
> struct virtio_user_dev for simplicity.
> - Reword "control" to "send_request".
> - Reword "host_features" to "device_features".
>
> In v16.07, we upstreamed a virtual device, virtio_user (with vhost-user
> as the backend). The path to go with a vhost-kernel backend has been
> dropped for bad performance comparing to vhost-user and code simplicity.
>
> But after a second thought, virtio_user + vhost-kernel is a good
> candidate as an exceptional path, such as KNI, which exchanges packets
> with kernel networking stack.
> - maintenance: vhost-net (kernel) is upstreamed and extensively used
> kernel module. We don't need any out-of-tree module like KNI.
> - performance: as with KNI, this solution would use one or more
> kthreads to send/receive packets from user space DPDK applications,
> which has little impact on user space polling thread (except that
> it might enter into kernel space to wake up those kthreads if
> necessary).
> - features: vhost-net is born to be a networking solution, which has
> lots of networking related featuers, like multi queue, tso, multi-seg
> mbuf, etc.
>
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
>
Sounds great. However, I think we'll need a how-to doc for this to help
people get it up and running as a KNI replacement for packets to/from
the kernel. Any plans to draw up such a doc? It would be good to include
it in this patchset.
/Bruce
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-09 4:39 ` Jason Wang
@ 2017-01-10 6:11 ` Tan, Jianfeng
2017-01-10 8:46 ` Thomas Monjalon
2017-01-11 2:42 ` Jason Wang
2017-01-11 2:30 ` Tan, Jianfeng
1 sibling, 2 replies; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-10 6:11 UTC (permalink / raw)
To: Jason Wang, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
Hi Jason,
On 1/9/2017 12:39 PM, Jason Wang wrote:
>
>
> On 2016年12月23日 15:14, Jianfeng Tan wrote:
>> This patch add support vhost kernel as the backend for virtio_user.
>> Three main hook functions are added:
>> - vhost_kernel_setup() to open char device, each vq pair needs one
>> vhostfd;
>> - vhost_kernel_ioctl() to communicate control messages with vhost
>> kernel module;
>> - vhost_kernel_enable_queue_pair() to open tap device and set it
>> as the backend of corresonding vhost fd (that is to say, vq pair).
>>
>> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
>> ---
>>
[...]
>> +/* TUNSETIFF ifr flags */
>> +#define IFF_TAP 0x0002
>> +#define IFF_NO_PI 0x1000
>> +#define IFF_ONE_QUEUE 0x2000
>> +#define IFF_VNET_HDR 0x4000
>> +#define IFF_MULTI_QUEUE 0x0100
>> +#define IFF_ATTACH_QUEUE 0x0200
>> +#define IFF_DETACH_QUEUE 0x0400
>
> Do we really want to duplicate those things which has been exposed by
> uapi here?
You mean those defined in <linux/if_tun.h>? Redefine those common
macros, or include standard header file, with respective pros and cons.
DPDK prefers the redefinition way as far as I understand, doesn't it?
>
>> +
>> +/* Constants */
>> +#define TUN_DEF_SNDBUF (1ull << 20)
>> +#define PATH_NET_TUN "/dev/net/tun"
>> +#define VHOST_KERNEL_MAX_REGIONS 64
>
> Unfortunate not a constant any more since c9ce42f72fd0 vhost: add
> max_mem_regions module parameter.
Yes, I was considering to ignore this in the initial release. But it's
not a big effort, I'll try to fix it in latest version.
>
>> +
>> +static uint64_t vhost_req_user_to_kernel[] = {
>> + [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
>> + [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
>> + [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
>> + [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
>> + [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
>> + [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
>> + [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
>> + [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
>> + [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
>> + [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
>> + [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
>> +};
>> +
>> +/* By default, vhost kernel module allows 64 regions, but DPDK allows
>> + * 256 segments. As a relief, below function merges those virtually
>> + * adjacent memsegs into one region.
>> + */
>> +static struct vhost_memory_kernel *
>> +prepare_vhost_memory_kernel(void)
>> +{
>> + uint32_t i, j, k = 0;
>> + struct rte_memseg *seg;
>> + struct vhost_memory_region *mr;
>> + struct vhost_memory_kernel *vm;
>> +
>> + vm = malloc(sizeof(struct vhost_memory_kernel) +
>> + VHOST_KERNEL_MAX_REGIONS *
>> + sizeof(struct vhost_memory_region));
>> +
>> + for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
>> + seg = &rte_eal_get_configuration()->mem_config->memseg[i];
>> + if (!seg->addr)
>> + break;
>
> If we're sure the number of regions is less than 64(or the module
> parameter read from /sys), can we avoid the iteration here?
The "if" statement under the "for" statement can save us from all
RTE_MAX_MEMSEG iteration.
>
>> +
>> + int new_region = 1;
>> +
>> + for (j = 0; j < k; ++j) {
>> + mr = &vm->regions[j];
>> +
>> + if (mr->userspace_addr + mr->memory_size ==
>> + (uint64_t)seg->addr) {
>> + mr->memory_size += seg->len;
>> + new_region = 0;
>> + break;
>> + }
>> +
>> + if ((uint64_t)seg->addr + seg->len ==
>> + mr->userspace_addr) {
>> + mr->guest_phys_addr = (uint64_t)seg->addr;
>> + mr->userspace_addr = (uint64_t)seg->addr;
>> + mr->memory_size += seg->len;
>> + new_region = 0;
>> + break;
>> + }
>> + }
>> +
>> + if (new_region == 0)
>> + continue;
>> +
>> + mr = &vm->regions[k++];
>> + mr->guest_phys_addr = (uint64_t)seg->addr; /* use vaddr
>> here! */
>> + mr->userspace_addr = (uint64_t)seg->addr;
>> + mr->memory_size = seg->len;
>> + mr->mmap_offset = 0;
>> +
>> + if (k >= VHOST_KERNEL_MAX_REGIONS) {
>> + free(vm);
>> + return NULL;
>> + }
>> + }
>> +
>> + vm->nregions = k;
>> + vm->padding = 0;
>> + return vm;
>> +}
>> +
>> +static int
>> +vhost_kernel_ioctl(struct virtio_user_dev *dev,
>> + enum vhost_user_request req,
>> + void *arg)
>> +{
>> + int i, ret = -1;
>> + uint64_t req_kernel;
>> + struct vhost_memory_kernel *vm = NULL;
>> +
>> + req_kernel = vhost_req_user_to_kernel[req];
>> +
>> + if (req_kernel == VHOST_SET_MEM_TABLE) {
>> + vm = prepare_vhost_memory_kernel();
>> + if (!vm)
>> + return -1;
>> + arg = (void *)vm;
>> + }
>> +
>> + /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
>
> I think the reason is when VIRTIO_F_IOMMU_PLATFORM is negotiated, all
> address should be iova instead of gpa.
>
Yes, I agree. As we don't have to do memory protection in a single
process, so it's completely useless here, right?
>> + if (req_kernel == VHOST_SET_FEATURES)
>> + *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
>> +
>> + for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
>> + if (dev->vhostfds[i] < 0)
>> + continue;
>> +
[...]
>> + if (!enable) {
>> + if (dev->tapfds[pair_idx]) {
>> + close(dev->tapfds[pair_idx]);
>> + dev->tapfds[pair_idx] = -1;
>> + }
>> + return vhost_kernel_set_backend(vhostfd, -1);
>
> If this is used to for thing like ethtool -L in guest, we should use
> TUNSETQUEUE here.
Oops, I was missing this ioctl operation. Let me fix it in next version.
>
>> + } else if (dev->tapfds[pair_idx] >= 0) {
>> + return 0;
>> + }
>> +
>> + if ((dev->features & (1ULL << VIRTIO_NET_F_MRG_RXBUF)) ||
>> + (dev->features & (1ULL << VIRTIO_F_VERSION_1)))
>> + hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
>> + else
>> + hdr_size = sizeof(struct virtio_net_hdr);
>> +
>> + /* TODO:
>> + * 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
>> + * 2. get number of memory regions from vhost module parameter
>> + * max_mem_regions, supported in newer version linux kernel
>> + */
>> + tapfd = open(PATH_NET_TUN, O_RDWR);
>> + if (tapfd < 0) {
>> + PMD_DRV_LOG(ERR, "fail to open %s: %s",
>> + PATH_NET_TUN, strerror(errno));
>> + return -1;
>> + }
>> +
>> + /* Construct ifr */
>> + memset(&ifr, 0, sizeof(ifr));
>> + ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
>> +
>> + if (ioctl(tapfd, TUNGETFEATURES, &features) == -1) {
>> + PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
>> + goto error;
>> + }
>> + if (features & IFF_ONE_QUEUE)
>> + ifr.ifr_flags |= IFF_ONE_QUEUE;
>> +
>> + /* Let tap instead of vhost-net handle vnet header, as the
>> latter does
>> + * not support offloading. And in this case, we should not set
>> feature
>> + * bit VHOST_NET_F_VIRTIO_NET_HDR.
>> + */
>> + if (features & IFF_VNET_HDR) {
>> + ifr.ifr_flags |= IFF_VNET_HDR;
>> + } else {
>> + PMD_DRV_LOG(ERR, "TAP does not support IFF_VNET_HDR");
>> + goto error;
>> + }
>> +
>> + if (dev->ifname)
>> + strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
>> + else
>> + strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
>> + if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
>> + PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
>> + goto error;
>> + }
>
> This requires CAP_NET_ADMIN, so we should really consider to accept a
> pre-created fd here.
It sounds like a future work for me. So far, all DPDK apps are running
in privileged mode (including CAP_NET_ADMIN?).
>
>> +
>> + fcntl(tapfd, F_SETFL, O_NONBLOCK);
>> +
>> + if (ioctl(tapfd, TUNSETVNETHDRSZ, &hdr_size) < 0) {
>> + PMD_DRV_LOG(ERR, "TUNSETVNETHDRSZ failed: %s",
>> strerror(errno));
>> + goto error;
>> + }
>> +
>> + if (ioctl(tapfd, TUNSETSNDBUF, &sndbuf) < 0) {
>> + PMD_DRV_LOG(ERR, "TUNSETSNDBUF failed: %s", strerror(errno));
>> + goto error;
>> + }
>
> Let's use INT_MAX as default here to survive from evil consumer here.
Oh yes, I will fix it.
Thanks,
Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v3 0/7] virtio_user as an alternative exception path
2017-01-09 14:06 ` [PATCH v3 0/7] virtio_user as an alternative exception path Bruce Richardson
@ 2017-01-10 8:46 ` Tan, Jianfeng
0 siblings, 0 replies; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-10 8:46 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev, yuanhan.liu, ferruh.yigit, cunming.liang
On 1/9/2017 10:06 PM, Bruce Richardson wrote:
> On Wed, Jan 04, 2017 at 03:59:19AM +0000, Jianfeng Tan wrote:
>> v3:
>> - Drop the patch to postpone driver ok sending patch, superseded it
>> with a bug fix to disable all virtqueues and re-init the device.
>> (you might wonder why not just send reset owner msg. Under my test,
>> it causes spinlock deadlock problem when killing the program).
>> - Avoid compiling error on 32-bit system for pointer convert.
>> - Fix a bug in patch "abstract virtio user backend ops", vhostfd is
>> not properly assigned.
>> - Fix a "MQ cannot be used" bug in v2, which is related to strip
>> some feature bits that vhost kernel does not recognize.
>> - Update release note.
>>
>> v2: (Lots of them are from yuanhan's comment)
>> - Add offloding feature.
>> - Add multiqueue support.
>> - Add a new patch to postpone the sending of driver ok notification.
>> - Put fix patch ahead of the whole patch series.
>> - Split original 0001 patch into 0003 and 0004 patches.
>> - Remove the original vhost_internal design, just add those into
>> struct virtio_user_dev for simplicity.
>> - Reword "control" to "send_request".
>> - Reword "host_features" to "device_features".
>>
>> In v16.07, we upstreamed a virtual device, virtio_user (with vhost-user
>> as the backend). The path to go with a vhost-kernel backend has been
>> dropped for bad performance comparing to vhost-user and code simplicity.
>>
>> But after a second thought, virtio_user + vhost-kernel is a good
>> candidate as an exceptional path, such as KNI, which exchanges packets
>> with kernel networking stack.
>> - maintenance: vhost-net (kernel) is upstreamed and extensively used
>> kernel module. We don't need any out-of-tree module like KNI.
>> - performance: as with KNI, this solution would use one or more
>> kthreads to send/receive packets from user space DPDK applications,
>> which has little impact on user space polling thread (except that
>> it might enter into kernel space to wake up those kthreads if
>> necessary).
>> - features: vhost-net is born to be a networking solution, which has
>> lots of networking related featuers, like multi queue, tso, multi-seg
>> mbuf, etc.
>>
>> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
>>
> Sounds great. However, I think we'll need a how-to doc for this to help
> people get it up and running as a KNI replacement for packets to/from
> the kernel. Any plans to draw up such a doc? It would be good to include
> it in this patchset.
I was planning to do that in separate patchset. But I agree with you
that it's better to do that in this patchset.
Thanks,
Jianfeng
>
> /Bruce
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-10 6:11 ` Tan, Jianfeng
@ 2017-01-10 8:46 ` Thomas Monjalon
2017-01-10 9:11 ` Tan, Jianfeng
2017-01-11 2:42 ` Jason Wang
1 sibling, 1 reply; 73+ messages in thread
From: Thomas Monjalon @ 2017-01-10 8:46 UTC (permalink / raw)
To: Tan, Jianfeng; +Cc: dev, Jason Wang, yuanhan.liu, ferruh.yigit, cunming.liang
2017-01-10 14:11, Tan, Jianfeng:
> On 1/9/2017 12:39 PM, Jason Wang wrote:
> > On 2016年12月23日 15:14, Jianfeng Tan wrote:
> [...]
> >> +/* TUNSETIFF ifr flags */
> >> +#define IFF_TAP 0x0002
> >> +#define IFF_NO_PI 0x1000
> >> +#define IFF_ONE_QUEUE 0x2000
> >> +#define IFF_VNET_HDR 0x4000
> >> +#define IFF_MULTI_QUEUE 0x0100
> >> +#define IFF_ATTACH_QUEUE 0x0200
> >> +#define IFF_DETACH_QUEUE 0x0400
> >
> > Do we really want to duplicate those things which has been exposed by
> > uapi here?
>
> You mean those defined in <linux/if_tun.h>? Redefine those common
> macros, or include standard header file, with respective pros and cons.
> DPDK prefers the redefinition way as far as I understand, doesn't it?
What is the benefit of copying defines instead of including?
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-10 8:46 ` Thomas Monjalon
@ 2017-01-10 9:11 ` Tan, Jianfeng
0 siblings, 0 replies; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-10 9:11 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: dev, Jason Wang, yuanhan.liu, ferruh.yigit, cunming.liang
On 1/10/2017 4:46 PM, Thomas Monjalon wrote:
> 2017-01-10 14:11, Tan, Jianfeng:
>> On 1/9/2017 12:39 PM, Jason Wang wrote:
>>> On 2016年12月23日 15:14, Jianfeng Tan wrote:
>> [...]
>>>> +/* TUNSETIFF ifr flags */
>>>> +#define IFF_TAP 0x0002
>>>> +#define IFF_NO_PI 0x1000
>>>> +#define IFF_ONE_QUEUE 0x2000
>>>> +#define IFF_VNET_HDR 0x4000
>>>> +#define IFF_MULTI_QUEUE 0x0100
>>>> +#define IFF_ATTACH_QUEUE 0x0200
>>>> +#define IFF_DETACH_QUEUE 0x0400
>>> Do we really want to duplicate those things which has been exposed by
>>> uapi here?
>> You mean those defined in <linux/if_tun.h>? Redefine those common
>> macros, or include standard header file, with respective pros and cons.
>> DPDK prefers the redefinition way as far as I understand, doesn't it?
> What is the benefit of copying defines instead of including?
(1) Avoid compiling errors that old version missed some of macros.
(2) Better for portability (does not apply here).
I might take it for granted a little bit as I see some very popular open
source projects behave like this way, like QEMU, HAProxy, etc.
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-09 4:39 ` Jason Wang
2017-01-10 6:11 ` Tan, Jianfeng
@ 2017-01-11 2:30 ` Tan, Jianfeng
2017-01-11 2:45 ` Jason Wang
1 sibling, 1 reply; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-11 2:30 UTC (permalink / raw)
To: Jason Wang, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
Hi Jason,
On 1/9/2017 12:39 PM, Jason Wang wrote:
>> + if (!enable) {
>> + if (dev->tapfds[pair_idx]) {
>> + close(dev->tapfds[pair_idx]);
>> + dev->tapfds[pair_idx] = -1;
>> + }
>> + return vhost_kernel_set_backend(vhostfd, -1);
>
> If this is used to for thing like ethtool -L in guest, we should use
> TUNSETQUEUE here.
To make it clear, why we need to ioctl(..., TUNSETQUEUE, ...) here.
According to Linux/Documentation/networking/tuntap.txt,
"A new ioctl(TUNSETQUEUE) were introduced to enable or disable a
queue. When
calling it with IFF_DETACH_QUEUE flag, the queue were disabled. And
when
calling it with IFF_ATTACH_QUEUE flag, the queue were enabled. The
queue were
enabled by default after it was created through TUNSETIFF."
As it's enabled by default, do you still see the necessity to call it
explicitly?
Thanks,
Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-10 6:11 ` Tan, Jianfeng
2017-01-10 8:46 ` Thomas Monjalon
@ 2017-01-11 2:42 ` Jason Wang
2017-01-11 3:13 ` Tan, Jianfeng
1 sibling, 1 reply; 73+ messages in thread
From: Jason Wang @ 2017-01-11 2:42 UTC (permalink / raw)
To: Tan, Jianfeng, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
On 2017年01月10日 14:11, Tan, Jianfeng wrote:
>
> Hi Jason,
>
>
> On 1/9/2017 12:39 PM, Jason Wang wrote:
>>
>>
>> On 2016年12月23日 15:14, Jianfeng Tan wrote:
>>> This patch add support vhost kernel as the backend for virtio_user.
>>> Three main hook functions are added:
>>> - vhost_kernel_setup() to open char device, each vq pair needs one
>>> vhostfd;
>>> - vhost_kernel_ioctl() to communicate control messages with vhost
>>> kernel module;
>>> - vhost_kernel_enable_queue_pair() to open tap device and set it
>>> as the backend of corresonding vhost fd (that is to say, vq pair).
>>>
>>> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
>>> ---
>>>
> [...]
>>> +/* TUNSETIFF ifr flags */
>>> +#define IFF_TAP 0x0002
>>> +#define IFF_NO_PI 0x1000
>>> +#define IFF_ONE_QUEUE 0x2000
>>> +#define IFF_VNET_HDR 0x4000
>>> +#define IFF_MULTI_QUEUE 0x0100
>>> +#define IFF_ATTACH_QUEUE 0x0200
>>> +#define IFF_DETACH_QUEUE 0x0400
>>
>> Do we really want to duplicate those things which has been exposed by
>> uapi here?
>
> You mean those defined in <linux/if_tun.h>? Redefine those common
> macros, or include standard header file, with respective pros and
> cons. DPDK prefers the redefinition way as far as I understand,
> doesn't it?
>
Well, if you really want to do this, you may want to use an independent
file. Then you can sync it with linux headers with a bash script.
>>
>>> +
>>> +/* Constants */
>>> +#define TUN_DEF_SNDBUF (1ull << 20)
>>> +#define PATH_NET_TUN "/dev/net/tun"
>>> +#define VHOST_KERNEL_MAX_REGIONS 64
>>
>> Unfortunate not a constant any more since c9ce42f72fd0 vhost: add
>> max_mem_regions module parameter.
>
> Yes, I was considering to ignore this in the initial release. But it's
> not a big effort, I'll try to fix it in latest version.
>
>>
>>> +
>>> +static uint64_t vhost_req_user_to_kernel[] = {
>>> + [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
>>> + [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
>>> + [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
>>> + [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
>>> + [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
>>> + [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
>>> + [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
>>> + [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
>>> + [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
>>> + [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
>>> + [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
>>> +};
>>> +
>>> +/* By default, vhost kernel module allows 64 regions, but DPDK allows
>>> + * 256 segments. As a relief, below function merges those virtually
>>> + * adjacent memsegs into one region.
>>> + */
>>> +static struct vhost_memory_kernel *
>>> +prepare_vhost_memory_kernel(void)
>>> +{
>>> + uint32_t i, j, k = 0;
>>> + struct rte_memseg *seg;
>>> + struct vhost_memory_region *mr;
>>> + struct vhost_memory_kernel *vm;
>>> +
>>> + vm = malloc(sizeof(struct vhost_memory_kernel) +
>>> + VHOST_KERNEL_MAX_REGIONS *
>>> + sizeof(struct vhost_memory_region));
>>> +
>>> + for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
>>> + seg = &rte_eal_get_configuration()->mem_config->memseg[i];
>>> + if (!seg->addr)
>>> + break;
>>
>> If we're sure the number of regions is less than 64(or the module
>> parameter read from /sys), can we avoid the iteration here?
>
> The "if" statement under the "for" statement can save us from all
> RTE_MAX_MEMSEG iteration.
But if we know the number of regions is short than the limit, there's
even no need for this?
>
>>
>>> +
>>> + int new_region = 1;
>>> +
>>> + for (j = 0; j < k; ++j) {
>>> + mr = &vm->regions[j];
>>> +
>>> + if (mr->userspace_addr + mr->memory_size ==
>>> + (uint64_t)seg->addr) {
>>> + mr->memory_size += seg->len;
>>> + new_region = 0;
>>> + break;
>>> + }
>>> +
>>> + if ((uint64_t)seg->addr + seg->len ==
>>> + mr->userspace_addr) {
>>> + mr->guest_phys_addr = (uint64_t)seg->addr;
>>> + mr->userspace_addr = (uint64_t)seg->addr;
>>> + mr->memory_size += seg->len;
>>> + new_region = 0;
>>> + break;
>>> + }
>>> + }
>>> +
>>> + if (new_region == 0)
>>> + continue;
>>> +
>>> + mr = &vm->regions[k++];
>>> + mr->guest_phys_addr = (uint64_t)seg->addr; /* use vaddr
>>> here! */
>>> + mr->userspace_addr = (uint64_t)seg->addr;
>>> + mr->memory_size = seg->len;
>>> + mr->mmap_offset = 0;
>>> +
>>> + if (k >= VHOST_KERNEL_MAX_REGIONS) {
>>> + free(vm);
>>> + return NULL;
>>> + }
>>> + }
>>> +
>>> + vm->nregions = k;
>>> + vm->padding = 0;
>>> + return vm;
>>> +}
>>> +
>>> +static int
>>> +vhost_kernel_ioctl(struct virtio_user_dev *dev,
>>> + enum vhost_user_request req,
>>> + void *arg)
>>> +{
>>> + int i, ret = -1;
>>> + uint64_t req_kernel;
>>> + struct vhost_memory_kernel *vm = NULL;
>>> +
>>> + req_kernel = vhost_req_user_to_kernel[req];
>>> +
>>> + if (req_kernel == VHOST_SET_MEM_TABLE) {
>>> + vm = prepare_vhost_memory_kernel();
>>> + if (!vm)
>>> + return -1;
>>> + arg = (void *)vm;
>>> + }
>>> +
>>> + /* Does not work when VIRTIO_F_IOMMU_PLATFORM now, why? */
>>
>> I think the reason is when VIRTIO_F_IOMMU_PLATFORM is negotiated, all
>> address should be iova instead of gpa.
>>
>
> Yes, I agree. As we don't have to do memory protection in a single
> process, so it's completely useless here, right?
Yes if there's no IOMMU concept in this case.
>
>>> + if (req_kernel == VHOST_SET_FEATURES)
>>> + *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
>>> +
>>> + for (i = 0; i < VHOST_KERNEL_MAX_QUEUES; ++i) {
>>> + if (dev->vhostfds[i] < 0)
>>> + continue;
>>> +
> [...]
>>> + if (!enable) {
>>> + if (dev->tapfds[pair_idx]) {
>>> + close(dev->tapfds[pair_idx]);
>>> + dev->tapfds[pair_idx] = -1;
>>> + }
>>> + return vhost_kernel_set_backend(vhostfd, -1);
>>
>> If this is used to for thing like ethtool -L in guest, we should use
>> TUNSETQUEUE here.
>
> Oops, I was missing this ioctl operation. Let me fix it in next version.
>
>>
>>> + } else if (dev->tapfds[pair_idx] >= 0) {
>>> + return 0;
>>> + }
>>> +
>>> + if ((dev->features & (1ULL << VIRTIO_NET_F_MRG_RXBUF)) ||
>>> + (dev->features & (1ULL << VIRTIO_F_VERSION_1)))
>>> + hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
>>> + else
>>> + hdr_size = sizeof(struct virtio_net_hdr);
>>> +
>>> + /* TODO:
>>> + * 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
>>> + * 2. get number of memory regions from vhost module parameter
>>> + * max_mem_regions, supported in newer version linux kernel
>>> + */
>>> + tapfd = open(PATH_NET_TUN, O_RDWR);
>>> + if (tapfd < 0) {
>>> + PMD_DRV_LOG(ERR, "fail to open %s: %s",
>>> + PATH_NET_TUN, strerror(errno));
>>> + return -1;
>>> + }
>>> +
>>> + /* Construct ifr */
>>> + memset(&ifr, 0, sizeof(ifr));
>>> + ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
>>> +
>>> + if (ioctl(tapfd, TUNGETFEATURES, &features) == -1) {
>>> + PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s",
>>> strerror(errno));
>>> + goto error;
>>> + }
>>> + if (features & IFF_ONE_QUEUE)
>>> + ifr.ifr_flags |= IFF_ONE_QUEUE;
>>> +
>>> + /* Let tap instead of vhost-net handle vnet header, as the
>>> latter does
>>> + * not support offloading. And in this case, we should not set
>>> feature
>>> + * bit VHOST_NET_F_VIRTIO_NET_HDR.
>>> + */
>>> + if (features & IFF_VNET_HDR) {
>>> + ifr.ifr_flags |= IFF_VNET_HDR;
>>> + } else {
>>> + PMD_DRV_LOG(ERR, "TAP does not support IFF_VNET_HDR");
>>> + goto error;
>>> + }
>>> +
>>> + if (dev->ifname)
>>> + strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
>>> + else
>>> + strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
>>> + if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
>>> + PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
>>> + goto error;
>>> + }
>>
>> This requires CAP_NET_ADMIN, so we should really consider to accept a
>> pre-created fd here.
>
> It sounds like a future work for me. So far, all DPDK apps are running
> in privileged mode (including CAP_NET_ADMIN?).
>
That's not safe. Accepting a per-created fd can solve this, and can even
support macvtap.
>>
>>> +
>>> + fcntl(tapfd, F_SETFL, O_NONBLOCK);
>>> +
>>> + if (ioctl(tapfd, TUNSETVNETHDRSZ, &hdr_size) < 0) {
>>> + PMD_DRV_LOG(ERR, "TUNSETVNETHDRSZ failed: %s",
>>> strerror(errno));
>>> + goto error;
>>> + }
>>> +
>>> + if (ioctl(tapfd, TUNSETSNDBUF, &sndbuf) < 0) {
>>> + PMD_DRV_LOG(ERR, "TUNSETSNDBUF failed: %s", strerror(errno));
>>> + goto error;
>>> + }
>>
>> Let's use INT_MAX as default here to survive from evil consumer here.
>
> Oh yes, I will fix it.
>
> Thanks,
> Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-11 2:30 ` Tan, Jianfeng
@ 2017-01-11 2:45 ` Jason Wang
0 siblings, 0 replies; 73+ messages in thread
From: Jason Wang @ 2017-01-11 2:45 UTC (permalink / raw)
To: Tan, Jianfeng, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
On 2017年01月11日 10:30, Tan, Jianfeng wrote:
>
> Hi Jason,
>
>
> On 1/9/2017 12:39 PM, Jason Wang wrote:
>>> + if (!enable) {
>>> + if (dev->tapfds[pair_idx]) {
>>> + close(dev->tapfds[pair_idx]);
>>> + dev->tapfds[pair_idx] = -1;
>>> + }
>>> + return vhost_kernel_set_backend(vhostfd, -1);
>>
>> If this is used to for thing like ethtool -L in guest, we should use
>> TUNSETQUEUE here.
>
> To make it clear, why we need to ioctl(..., TUNSETQUEUE, ...) here.
> According to Linux/Documentation/networking/tuntap.txt,
> "A new ioctl(TUNSETQUEUE) were introduced to enable or disable a
> queue. When
> calling it with IFF_DETACH_QUEUE flag, the queue were disabled.
> And when
> calling it with IFF_ATTACH_QUEUE flag, the queue were enabled. The
> queue were
> enabled by default after it was created through TUNSETIFF."
>
> As it's enabled by default, do you still see the necessity to call it
> explicitly?
If you want to keep it enabled, no need. But if you want to disable one
specific queue (which I believe is the case of !enable?), you need to
call it.
Thanks
>
> Thanks,
> Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-11 2:42 ` Jason Wang
@ 2017-01-11 3:13 ` Tan, Jianfeng
2017-01-11 3:23 ` Jason Wang
0 siblings, 1 reply; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-11 3:13 UTC (permalink / raw)
To: Jason Wang, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
On 1/11/2017 10:42 AM, Jason Wang wrote:
>
>
> On 2017年01月10日 14:11, Tan, Jianfeng wrote:
>>
>> Hi Jason,
>>
>>
>> On 1/9/2017 12:39 PM, Jason Wang wrote:
>>>
>>>
>>> On 2016年12月23日 15:14, Jianfeng Tan wrote:
>>>> This patch add support vhost kernel as the backend for virtio_user.
>>>> Three main hook functions are added:
>>>> - vhost_kernel_setup() to open char device, each vq pair needs one
>>>> vhostfd;
>>>> - vhost_kernel_ioctl() to communicate control messages with vhost
>>>> kernel module;
>>>> - vhost_kernel_enable_queue_pair() to open tap device and set it
>>>> as the backend of corresonding vhost fd (that is to say, vq
>>>> pair).
>>>>
>>>> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
>>>> ---
>>>>
>> [...]
>>>> +/* TUNSETIFF ifr flags */
>>>> +#define IFF_TAP 0x0002
>>>> +#define IFF_NO_PI 0x1000
>>>> +#define IFF_ONE_QUEUE 0x2000
>>>> +#define IFF_VNET_HDR 0x4000
>>>> +#define IFF_MULTI_QUEUE 0x0100
>>>> +#define IFF_ATTACH_QUEUE 0x0200
>>>> +#define IFF_DETACH_QUEUE 0x0400
>>>
>>> Do we really want to duplicate those things which has been exposed
>>> by uapi here?
>>
>> You mean those defined in <linux/if_tun.h>? Redefine those common
>> macros, or include standard header file, with respective pros and
>> cons. DPDK prefers the redefinition way as far as I understand,
>> doesn't it?
>>
>
> Well, if you really want to do this, you may want to use an
> independent file. Then you can sync it with linux headers with a bash
> script.
Agreed!
[...]
>>>> +static struct vhost_memory_kernel *
>>>> +prepare_vhost_memory_kernel(void)
>>>> +{
>>>> + uint32_t i, j, k = 0;
>>>> + struct rte_memseg *seg;
>>>> + struct vhost_memory_region *mr;
>>>> + struct vhost_memory_kernel *vm;
>>>> +
>>>> + vm = malloc(sizeof(struct vhost_memory_kernel) +
>>>> + VHOST_KERNEL_MAX_REGIONS *
>>>> + sizeof(struct vhost_memory_region));
>>>> +
>>>> + for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
>>>> + seg = &rte_eal_get_configuration()->mem_config->memseg[i];
>>>> + if (!seg->addr)
>>>> + break;
>>>
>>> If we're sure the number of regions is less than 64(or the module
>>> parameter read from /sys), can we avoid the iteration here?
>>
>> The "if" statement under the "for" statement can save us from all
>> RTE_MAX_MEMSEG iteration.
>
> But if we know the number of regions is short than the limit, there's
> even no need for this?
The problem is: we don't have a variable to know how many segments are
there in DPDK. Anyway, we need to report error in the situation that #
of segments > # of regions. Or can you give a more specific example?
[...]
>>>> + if (dev->ifname)
>>>> + strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
>>>> + else
>>>> + strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
>>>> + if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
>>>> + PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
>>>> + goto error;
>>>> + }
>>>
>>> This requires CAP_NET_ADMIN, so we should really consider to accept
>>> a pre-created fd here.
>>
>> It sounds like a future work for me. So far, all DPDK apps are
>> running in privileged mode (including CAP_NET_ADMIN?).
>>
>
> That's not safe.
Yes.
> Accepting a per-created fd can solve this, and can even support macvtap
So you are proposing a way to specify the virtio_user parameter like,
--vdev=virtio_user,tapfd=fd1,xxx, right? As replied in your another
comment, it's a great way to go. But I'm afraid we have no time to
implement that in this cycle.
Thanks,
Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-11 3:13 ` Tan, Jianfeng
@ 2017-01-11 3:23 ` Jason Wang
2017-01-12 9:40 ` Tan, Jianfeng
0 siblings, 1 reply; 73+ messages in thread
From: Jason Wang @ 2017-01-11 3:23 UTC (permalink / raw)
To: Tan, Jianfeng, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
On 2017年01月11日 11:13, Tan, Jianfeng wrote:
>
>
> On 1/11/2017 10:42 AM, Jason Wang wrote:
>>
>>
>> On 2017年01月10日 14:11, Tan, Jianfeng wrote:
>>>
>>> Hi Jason,
>>>
>>>
>>> On 1/9/2017 12:39 PM, Jason Wang wrote:
>>>>
>>>>
>>>> On 2016年12月23日 15:14, Jianfeng Tan wrote:
>>>>> This patch add support vhost kernel as the backend for virtio_user.
>>>>> Three main hook functions are added:
>>>>> - vhost_kernel_setup() to open char device, each vq pair needs one
>>>>> vhostfd;
>>>>> - vhost_kernel_ioctl() to communicate control messages with vhost
>>>>> kernel module;
>>>>> - vhost_kernel_enable_queue_pair() to open tap device and set it
>>>>> as the backend of corresonding vhost fd (that is to say, vq
>>>>> pair).
>>>>>
>>>>> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
>>>>> ---
>>>>>
>>> [...]
>>>>> +/* TUNSETIFF ifr flags */
>>>>> +#define IFF_TAP 0x0002
>>>>> +#define IFF_NO_PI 0x1000
>>>>> +#define IFF_ONE_QUEUE 0x2000
>>>>> +#define IFF_VNET_HDR 0x4000
>>>>> +#define IFF_MULTI_QUEUE 0x0100
>>>>> +#define IFF_ATTACH_QUEUE 0x0200
>>>>> +#define IFF_DETACH_QUEUE 0x0400
>>>>
>>>> Do we really want to duplicate those things which has been exposed
>>>> by uapi here?
>>>
>>> You mean those defined in <linux/if_tun.h>? Redefine those common
>>> macros, or include standard header file, with respective pros and
>>> cons. DPDK prefers the redefinition way as far as I understand,
>>> doesn't it?
>>>
>>
>> Well, if you really want to do this, you may want to use an
>> independent file. Then you can sync it with linux headers with a bash
>> script.
>
> Agreed!
>
> [...]
>>>>> +static struct vhost_memory_kernel *
>>>>> +prepare_vhost_memory_kernel(void)
>>>>> +{
>>>>> + uint32_t i, j, k = 0;
>>>>> + struct rte_memseg *seg;
>>>>> + struct vhost_memory_region *mr;
>>>>> + struct vhost_memory_kernel *vm;
>>>>> +
>>>>> + vm = malloc(sizeof(struct vhost_memory_kernel) +
>>>>> + VHOST_KERNEL_MAX_REGIONS *
>>>>> + sizeof(struct vhost_memory_region));
>>>>> +
>>>>> + for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
>>>>> + seg = &rte_eal_get_configuration()->mem_config->memseg[i];
>>>>> + if (!seg->addr)
>>>>> + break;
>>>>
>>>> If we're sure the number of regions is less than 64(or the module
>>>> parameter read from /sys), can we avoid the iteration here?
>>>
>>> The "if" statement under the "for" statement can save us from all
>>> RTE_MAX_MEMSEG iteration.
>>
>> But if we know the number of regions is short than the limit, there's
>> even no need for this?
>
> The problem is: we don't have a variable to know how many segments are
> there in DPDK. Anyway, we need to report error in the situation that #
> of segments > # of regions. Or can you give a more specific example?
>
> [...]
If we don't know #segments, I'm fine with current code.
>>>>> + if (dev->ifname)
>>>>> + strncpy(ifr.ifr_name, dev->ifname, IFNAMSIZ);
>>>>> + else
>>>>> + strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
>>>>> + if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
>>>>> + PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
>>>>> + goto error;
>>>>> + }
>>>>
>>>> This requires CAP_NET_ADMIN, so we should really consider to accept
>>>> a pre-created fd here.
>>>
>>> It sounds like a future work for me. So far, all DPDK apps are
>>> running in privileged mode (including CAP_NET_ADMIN?).
>>>
>>
>> That's not safe.
>
> Yes.
>
>> Accepting a per-created fd can solve this, and can even support macvtap
>
> So you are proposing a way to specify the virtio_user parameter like,
> --vdev=virtio_user,tapfd=fd1,xxx, right? As replied in your another
> comment, it's a great way to go. But I'm afraid we have no time to
> implement that in this cycle.
>
> Thanks,
> Jianfeng
Ok, just want to show its advantages. It can be added on top.
And two more suggestions:
- better to split tap support out of vhost file
- kernel support more than 8 queues on recent kernel, so there's no need
to limit it to 8. When running on old kernel, TUNSETIFF will fail which
can give user a hint to reduce #queues.
Thanks
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-11 3:23 ` Jason Wang
@ 2017-01-12 9:40 ` Tan, Jianfeng
2017-01-13 2:27 ` Jason Wang
0 siblings, 1 reply; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-12 9:40 UTC (permalink / raw)
To: Jason Wang, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
Hi Jason,
> Ok, just want to show its advantages. It can be added on top.
>
> And two more suggestions:
> - better to split tap support out of vhost file
Good suggestion! Will do that in next version.
> - kernel support more than 8 queues on recent kernel, so there's no
> need to limit it to 8. When running on old kernel, TUNSETIFF will fail
> which can give user a hint to reduce #queues.
>
After a try, even we remove such restriction in vhost_kernel backend, we
are still restricted by VIRTIO_MAX_VIRTQUEUES (8) defined in virtio PMD.
Thanks for the suggestions!
Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v2 5/7] net/virtio_user: add vhost kernel support
2017-01-12 9:40 ` Tan, Jianfeng
@ 2017-01-13 2:27 ` Jason Wang
0 siblings, 0 replies; 73+ messages in thread
From: Jason Wang @ 2017-01-13 2:27 UTC (permalink / raw)
To: Tan, Jianfeng, dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang
On 2017年01月12日 17:40, Tan, Jianfeng wrote:
> Hi Jason,
>
>> Ok, just want to show its advantages. It can be added on top.
>>
>> And two more suggestions:
>> - better to split tap support out of vhost file
>
> Good suggestion! Will do that in next version.
>
>> - kernel support more than 8 queues on recent kernel, so there's no
>> need to limit it to 8. When running on old kernel, TUNSETIFF will
>> fail which can give user a hint to reduce #queues.
>>
>
> After a try, even we remove such restriction in vhost_kernel backend,
> we are still restricted by VIRTIO_MAX_VIRTQUEUES (8) defined in virtio
> PMD.
I think we'd better remove this limitation in the future.
Thanks
>
> Thanks for the suggestions!
>
> Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v4 0/8] virtio_user as an alternative exception path
2016-12-02 14:31 [PATCH 0/3] virtio_user as an alternative exception path Jianfeng Tan
` (5 preceding siblings ...)
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 1/8] net/virtio_user: fix wrongly get/set features Jianfeng Tan
` (8 more replies)
6 siblings, 9 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
v4:
- Fix a clang compiling error by removing "NULL" line in the definition
of vhost_msg_strings. This error does not show up when it's defined
as a static variable, so not necessary to fix it in stable branch.
- Query kernel to get how many regions are supported, default 64 regions.
- Set TUNSETSNDBUF to INT_MAX.
- When get_features, unmask those backend-specific feature bits.
- Remove VHOST_KERNEL_MAX_QUEUES (8) restriction, but due to another
restriction by VIRTIO_MAX_VIRTQUEUES (8), we still cannot configure
more than 8 queues.
- Add a howto document.
v3:
- Drop the patch to postpone driver ok sending patch, superseded it
with a bug fix to disable all virtqueues and re-init the device.
(you might wonder why not just send reset owner msg. Under my test,
it causes spinlock deadlock problem when killing the program).
- Avoid compiling error on 32-bit system for pointer convert.
- Fix a bug in patch "abstract virtio user backend ops", vhostfd is
not properly assigned.
- Fix a "MQ cannot be used" bug in v2, which is related to strip
some feature bits that vhost kernel does not recognize.
- Update release note.
v2: (Lots of them are from yuanhan's comment)
- Add offloding feature.
- Add multiqueue support.
- Add a new patch to postpone the sending of driver ok notification.
- Put fix patch ahead of the whole patch series.
- Split original 0001 patch into 0003 and 0004 patches.
- Remove the original vhost_internal design, just add those into
struct virtio_user_dev for simplicity.
- Reword "control" to "send_request".
- Reword "host_features" to "device_features".
In v16.07, we upstreamed a virtual device, virtio_user (with vhost-user
as the backend). The path to go with a vhost-kernel backend has been
dropped for bad performance comparing to vhost-user and code simplicity.
But after a second thought, virtio_user + vhost-kernel is a good
candidate as an exceptional path, such as KNI, which exchanges packets
with kernel networking stack.
- maintenance: vhost-net (kernel) is upstreamed and extensively used
kernel module. We don't need any out-of-tree module like KNI.
- performance: as with KNI, this solution would use one or more
kthreads to send/receive packets from user space DPDK applications,
which has little impact on user space polling thread (except that
it might enter into kernel space to wake up those kthreads if
necessary).
- features: vhost-net is born to be a networking solution, which has
lots of networking related featuers, like multi queue, tso, multi-seg
mbuf, etc.
How to test:
Here is a simple test case for:
_______testpmd_______
iperf-c/iperf-s -- tap0 -- [virtio_user0 -- ixgbe]
|
|
|
iperf-s/iperf-c -- ixgbe (kernel driver, eth3)
Step 1: Remove MAC addres setting in testpmd csum fwd engine.
diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c
index 57e6ae2..225c7a8 100644
--- a/app/test-pmd/csumonly.c
+++ b/app/test-pmd/csumonly.c
@@ -706,10 +706,12 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)
* and inner headers */
eth_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *);
+#if 0
ether_addr_copy(&peer_eth_addrs[fs->peer_addr],
ð_hdr->d_addr);
ether_addr_copy(&ports[fs->tx_port].eth_addr,
ð_hdr->s_addr);
+#endif
parse_ethernet(eth_hdr, &info);
l3_hdr = (char *)eth_hdr + info.l2_len;
Step 2: bind one ixgbe with igb_uio, and start testpmd
$(testpmd) -c 0xc -n 4 \
--vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 \
-- -i --txqflags=0x0 --disable-hw-vlan --enable-lro \
--enable-rx-cksum --rxd=1024 --txd=1024
To test multiqueue, start testpmd like this:
$(testpmd) -c 0xc -n 4 \
--vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 \
-- -i --txqflags=0x0 --disable-hw-vlan --enable-lro --enable-rx-cksum \
--txq=2 --rxq=2 --rxd=1024 --txd=1024
Step 3: start testpmd: (port 0 is ixgbe, port 1 is virtio_user)
# set fwd csum
# csum set ip hw 0
# csum set tcp hw 0
# csum set ip sw 1
# csum set tcp hw 1
# tso set 1448 0
# tso set 1448 1
# start
Step 4: start the other end with below script
$ ip netns add ns1
$ ip link set eth3 netns ns1
$ ip netns exec ns1 ifconfig eth3 1.1.1.2/24 up
$ ip netns exec ns1 taskset 0xf0 iperf3 -s -i 1
Use below command if you change the position of iperf-c and iperf-s.
$ ip netns exec ns1 numactl -N 0 iperf3 -c 1.1.1.3 -i 1 -t 30
Step 5: up the tap0 and configure ip
$ ifconfig tap0 1.1.1.3/24 up
Step 6: start test
$ iperf3 -c 1.1.1.2 -i 1 -t 30
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Jianfeng Tan (8):
net/virtio_user: fix wrongly get/set features
net/virtio_user: fix not properly reset device
net/virtio_user: move vhost user specific code
net/virtio_user: abstract virtio user backend ops
net/virtio_user: add vhost kernel support
net/virtio_user: enable offloading
net/virtio_user: enable multiqueue with vhost kernel
doc: add guide to use virtio_user as exceptional path
.../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes
doc/guides/prog_guide/index.rst | 1 +
.../prog_guide/virtio_user_as_exceptional_path.rst | 104 ++++++
doc/guides/rel_notes/release_17_02.rst | 20 +
drivers/net/virtio/Makefile | 2 +
drivers/net/virtio/virtio_ethdev.h | 5 +
drivers/net/virtio/virtio_user/vhost.h | 51 +--
drivers/net/virtio/virtio_user/vhost_kernel.c | 401 +++++++++++++++++++++
drivers/net/virtio/virtio_user/vhost_kernel_tap.c | 133 +++++++
drivers/net/virtio/virtio_user/vhost_kernel_tap.h | 67 ++++
drivers/net/virtio/virtio_user/vhost_user.c | 98 +++--
drivers/net/virtio/virtio_user/virtio_user_dev.c | 153 +++++---
drivers/net/virtio/virtio_user/virtio_user_dev.h | 15 +-
drivers/net/virtio/virtio_user_ethdev.c | 20 +-
14 files changed, 945 insertions(+), 125 deletions(-)
create mode 100644 doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
create mode 100644 doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel_tap.c
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel_tap.h
--
2.7.4
^ permalink raw reply [flat|nested] 73+ messages in thread
* [PATCH v4 1/8] net/virtio_user: fix wrongly get/set features
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 2/8] net/virtio_user: fix not properly reset device Jianfeng Tan
` (7 subsequent siblings)
8 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan, stable
Before the commit 86d59b21468a ("net/virtio: support LRO"), features
in virtio PMD, is decided and properly set at device initialization
and will not be changed. But afterward, features could be changed in
virtio_dev_configure(), and will be re-negotiated if it's changed.
In virtio_user, device features is obtained at driver probe phase
only once, but we did not store it. So the added feature bits in
re-negotiation will fail.
To fix it, we store it down, and will be used to feature negotiation
either at device initialization phase or device configure phase.
Fixes: e9efa4d93821 ("net/virtio-user: add new virtual PCI driver")
CC: stable@dpdk.org
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_ethdev.h | 5 ++++
drivers/net/virtio/virtio_user/virtio_user_dev.c | 34 +++++++++++-------------
drivers/net/virtio/virtio_user/virtio_user_dev.h | 5 +++-
drivers/net/virtio/virtio_user_ethdev.c | 5 ++--
4 files changed, 28 insertions(+), 21 deletions(-)
diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h
index 27d9a19..4feccf9 100644
--- a/drivers/net/virtio/virtio_ethdev.h
+++ b/drivers/net/virtio/virtio_ethdev.h
@@ -70,6 +70,11 @@
1ULL << VIRTIO_F_VERSION_1 | \
1ULL << VIRTIO_F_IOMMU_PLATFORM)
+#define VIRTIO_PMD_SUPPORTED_GUEST_FEATURES \
+ (VIRTIO_PMD_DEFAULT_GUEST_FEATURES | \
+ 1u << VIRTIO_NET_F_GUEST_CSUM | \
+ 1u << VIRTIO_NET_F_GUEST_TSO4 | \
+ 1u << VIRTIO_NET_F_GUEST_TSO6)
/*
* CQ function prototype
*/
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index e239e0e..0d7e17b 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -148,12 +148,13 @@ virtio_user_start_device(struct virtio_user_dev *dev)
/* Step 1: set features
* Make sure VHOST_USER_F_PROTOCOL_FEATURES is added if mq is enabled,
- * and VIRTIO_NET_F_MAC is stripped.
+ * VIRTIO_NET_F_MAC and VIRTIO_NET_F_CTRL_VQ is stripped.
*/
features = dev->features;
if (dev->max_queue_pairs > 1)
features |= VHOST_USER_MQ;
features &= ~(1ull << VIRTIO_NET_F_MAC);
+ features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
if (ret < 0)
goto error;
@@ -228,29 +229,26 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
}
if (vhost_user_sock(dev->vhostfd, VHOST_USER_GET_FEATURES,
- &dev->features) < 0) {
+ &dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
}
if (dev->mac_specified)
- dev->features |= (1ull << VIRTIO_NET_F_MAC);
+ dev->device_features |= (1ull << VIRTIO_NET_F_MAC);
- if (!cq) {
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
- /* Also disable features depends on VIRTIO_NET_F_CTRL_VQ */
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_RX);
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_VLAN);
- dev->features &= ~(1ull << VIRTIO_NET_F_GUEST_ANNOUNCE);
- dev->features &= ~(1ull << VIRTIO_NET_F_MQ);
- dev->features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
- } else {
- /* vhost user backend does not need to know ctrl-q, so
- * actually we need add this bit into features. However,
- * DPDK vhost-user does send features with this bit, so we
- * check it instead of OR it for now.
+ if (cq) {
+ /* device does not really need to know anything about CQ,
+ * so if necessary, we just claim to support CQ
*/
- if (!(dev->features & (1ull << VIRTIO_NET_F_CTRL_VQ)))
- PMD_INIT_LOG(INFO, "vhost does not support ctrl-q");
+ dev->device_features |= (1ull << VIRTIO_NET_F_CTRL_VQ);
+ } else {
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
+ /* Also disable features depends on VIRTIO_NET_F_CTRL_VQ */
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_RX);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_VLAN);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_GUEST_ANNOUNCE);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_MQ);
+ dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
}
if (dev->max_queue_pairs > 1) {
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 33690b5..28fc788 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -46,7 +46,10 @@ struct virtio_user_dev {
uint32_t max_queue_pairs;
uint32_t queue_pairs;
uint32_t queue_size;
- uint64_t features;
+ uint64_t features; /* the negotiated features with driver,
+ * and will be sync with device
+ */
+ uint64_t device_features; /* supported features by device */
uint8_t status;
uint8_t mac_addr[ETHER_ADDR_LEN];
char path[PATH_MAX];
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 8cb983c..6c16d2d 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -117,7 +117,8 @@ virtio_user_get_features(struct virtio_hw *hw)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- return dev->features;
+ /* unmask feature bits defined in vhost user protocol */
+ return dev->device_features & VIRTIO_PMD_SUPPORTED_GUEST_FEATURES;
}
static void
@@ -125,7 +126,7 @@ virtio_user_set_features(struct virtio_hw *hw, uint64_t features)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- dev->features = features;
+ dev->features = features & dev->device_features;
}
static uint8_t
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v4 2/8] net/virtio_user: fix not properly reset device
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 1/8] net/virtio_user: fix wrongly get/set features Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 3/8] net/virtio_user: move vhost user specific code Jianfeng Tan
` (6 subsequent siblings)
8 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan, stable
virtio_user is not properly reset when users call vtpci_reset(),
as it ignores VIRTIO_CONFIG_STATUS_RESET status in
virtio_user_set_status().
This might lead to initialization failure as it starts to re-init
the device before sending RESET messege to backend. Besides, previous
callfds and kickfds are not closed.
To fix it, we add support to disable virtqueues when it's set to
DRIVER OK status, and re-init fields in struct virtio_user_dev.
Fixes: e9efa4d93821 ("net/virtio-user: add new virtual PCI driver")
Fixes: 37a7eb2ae816 ("net/virtio-user: add device emulation layer")
CC: stable@dpdk.org
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 26 ++++++++++++++++--------
drivers/net/virtio/virtio_user_ethdev.c | 15 ++++++++------
2 files changed, 27 insertions(+), 14 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 0d7e17b..a38398b 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -182,7 +182,17 @@ virtio_user_start_device(struct virtio_user_dev *dev)
int virtio_user_stop_device(struct virtio_user_dev *dev)
{
- return vhost_user_sock(dev->vhostfd, VHOST_USER_RESET_OWNER, NULL);
+ uint32_t i;
+
+ for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
+ close(dev->callfds[i]);
+ close(dev->kickfds[i]);
+ }
+
+ for (i = 0; i < dev->max_queue_pairs; ++i)
+ vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+
+ return 0;
}
static inline void
@@ -210,6 +220,8 @@ int
virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
int cq, int queue_size, const char *mac)
{
+ uint32_t i;
+
snprintf(dev->path, PATH_MAX, "%s", path);
dev->max_queue_pairs = queues;
dev->queue_pairs = 1; /* mq disabled by default */
@@ -218,6 +230,11 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
parse_mac(dev, mac);
dev->vhostfd = -1;
+ for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
+ dev->kickfds[i] = -1;
+ dev->callfds[i] = -1;
+ }
+
dev->vhostfd = vhost_user_setup(dev->path);
if (dev->vhostfd < 0) {
PMD_INIT_LOG(ERR, "backend set up fails");
@@ -264,13 +281,6 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
void
virtio_user_dev_uninit(struct virtio_user_dev *dev)
{
- uint32_t i;
-
- for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
- close(dev->callfds[i]);
- close(dev->kickfds[i]);
- }
-
close(dev->vhostfd);
}
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 6c16d2d..298232a 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -87,21 +87,24 @@ virtio_user_write_dev_config(struct virtio_hw *hw, size_t offset,
}
static void
-virtio_user_set_status(struct virtio_hw *hw, uint8_t status)
+virtio_user_reset(struct virtio_hw *hw)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- if (status & VIRTIO_CONFIG_STATUS_DRIVER_OK)
- virtio_user_start_device(dev);
- dev->status = status;
+ if (dev->status & VIRTIO_CONFIG_STATUS_DRIVER_OK)
+ virtio_user_stop_device(dev);
}
static void
-virtio_user_reset(struct virtio_hw *hw)
+virtio_user_set_status(struct virtio_hw *hw, uint8_t status)
{
struct virtio_user_dev *dev = virtio_user_get_dev(hw);
- virtio_user_stop_device(dev);
+ if (status & VIRTIO_CONFIG_STATUS_DRIVER_OK)
+ virtio_user_start_device(dev);
+ else if (status == VIRTIO_CONFIG_STATUS_RESET)
+ virtio_user_reset(hw);
+ dev->status = status;
}
static uint8_t
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v4 3/8] net/virtio_user: move vhost user specific code
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 1/8] net/virtio_user: fix wrongly get/set features Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 2/8] net/virtio_user: fix not properly reset device Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 4/8] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
` (5 subsequent siblings)
8 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
To support vhost kernel as the backend of net_virtio_user in coming
patches, we move vhost_user specific structs and macros into
vhost_user.c, and only keep common definitions in vhost.h.
Besides, remove VHOST_USER_MQ feature check.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost.h | 36 ------------------------
drivers/net/virtio/virtio_user/vhost_user.c | 32 +++++++++++++++++++++
drivers/net/virtio/virtio_user/virtio_user_dev.c | 9 ------
3 files changed, 32 insertions(+), 45 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 7adb55f..e54ac35 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -42,8 +42,6 @@
#include "../virtio_logs.h"
#include "../virtqueue.h"
-#define VHOST_MEMORY_MAX_NREGIONS 8
-
struct vhost_vring_state {
unsigned int index;
unsigned int num;
@@ -105,40 +103,6 @@ struct vhost_memory_region {
uint64_t mmap_offset;
};
-struct vhost_memory {
- uint32_t nregions;
- uint32_t padding;
- struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
-};
-
-struct vhost_user_msg {
- enum vhost_user_request request;
-
-#define VHOST_USER_VERSION_MASK 0x3
-#define VHOST_USER_REPLY_MASK (0x1 << 2)
- uint32_t flags;
- uint32_t size; /* the following payload size */
- union {
-#define VHOST_USER_VRING_IDX_MASK 0xff
-#define VHOST_USER_VRING_NOFD_MASK (0x1 << 8)
- uint64_t u64;
- struct vhost_vring_state state;
- struct vhost_vring_addr addr;
- struct vhost_memory memory;
- } payload;
- int fds[VHOST_MEMORY_MAX_NREGIONS];
-} __attribute((packed));
-
-#define VHOST_USER_HDR_SIZE offsetof(struct vhost_user_msg, payload.u64)
-#define VHOST_USER_PAYLOAD_SIZE \
- (sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
-
-/* The version of the protocol we support */
-#define VHOST_USER_VERSION 0x1
-
-#define VHOST_USER_F_PROTOCOL_FEATURES 30
-#define VHOST_USER_MQ (1ULL << VHOST_USER_F_PROTOCOL_FEATURES)
-
int vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg);
int vhost_user_setup(const char *path);
int vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable);
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 082e821..295ce16 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -42,6 +42,38 @@
#include "vhost.h"
+/* The version of the protocol we support */
+#define VHOST_USER_VERSION 0x1
+
+#define VHOST_MEMORY_MAX_NREGIONS 8
+struct vhost_memory {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[VHOST_MEMORY_MAX_NREGIONS];
+};
+
+struct vhost_user_msg {
+ enum vhost_user_request request;
+
+#define VHOST_USER_VERSION_MASK 0x3
+#define VHOST_USER_REPLY_MASK (0x1 << 2)
+ uint32_t flags;
+ uint32_t size; /* the following payload size */
+ union {
+#define VHOST_USER_VRING_IDX_MASK 0xff
+#define VHOST_USER_VRING_NOFD_MASK (0x1 << 8)
+ uint64_t u64;
+ struct vhost_vring_state state;
+ struct vhost_vring_addr addr;
+ struct vhost_memory memory;
+ } payload;
+ int fds[VHOST_MEMORY_MAX_NREGIONS];
+} __attribute((packed));
+
+#define VHOST_USER_HDR_SIZE offsetof(struct vhost_user_msg, payload.u64)
+#define VHOST_USER_PAYLOAD_SIZE \
+ (sizeof(struct vhost_user_msg) - VHOST_USER_HDR_SIZE)
+
static int
vhost_user_write(int fd, void *buf, int len, int *fds, int fd_num)
{
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index a38398b..8dd563a 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -151,8 +151,6 @@ virtio_user_start_device(struct virtio_user_dev *dev)
* VIRTIO_NET_F_MAC and VIRTIO_NET_F_CTRL_VQ is stripped.
*/
features = dev->features;
- if (dev->max_queue_pairs > 1)
- features |= VHOST_USER_MQ;
features &= ~(1ull << VIRTIO_NET_F_MAC);
features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
@@ -268,13 +266,6 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
dev->device_features &= ~(1ull << VIRTIO_NET_F_CTRL_MAC_ADDR);
}
- if (dev->max_queue_pairs > 1) {
- if (!(dev->features & VHOST_USER_MQ)) {
- PMD_INIT_LOG(ERR, "MQ not supported by the backend");
- return -1;
- }
- }
-
return 0;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v4 4/8] net/virtio_user: abstract virtio user backend ops
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
` (2 preceding siblings ...)
2017-01-13 12:18 ` [PATCH v4 3/8] net/virtio_user: move vhost user specific code Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 5/8] net/virtio_user: add vhost kernel support Jianfeng Tan
` (4 subsequent siblings)
8 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
Add a struct virtio_user_backend_ops to abstract three kinds of backend
operations:
- setup, create the unix socket connection;
- send_request, sync messages with backend;
- enable_qp, enable some queue pair.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost.h | 17 +++++-
drivers/net/virtio/virtio_user/vhost_user.c | 66 +++++++++++---------
drivers/net/virtio/virtio_user/virtio_user_dev.c | 77 +++++++++++++++---------
drivers/net/virtio/virtio_user/virtio_user_dev.h | 5 ++
4 files changed, 106 insertions(+), 59 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index e54ac35..515e4fc 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -96,6 +96,8 @@ enum vhost_user_request {
VHOST_USER_MAX
};
+const char * const vhost_msg_strings[VHOST_USER_MAX];
+
struct vhost_memory_region {
uint64_t guest_phys_addr;
uint64_t memory_size; /* bytes */
@@ -103,8 +105,17 @@ struct vhost_memory_region {
uint64_t mmap_offset;
};
-int vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg);
-int vhost_user_setup(const char *path);
-int vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable);
+struct virtio_user_dev;
+
+struct virtio_user_backend_ops {
+ int (*setup)(struct virtio_user_dev *dev);
+ int (*send_request)(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg);
+ int (*enable_qp)(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable);
+};
+struct virtio_user_backend_ops ops_user;
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 295ce16..4ad7b21 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -41,6 +41,7 @@
#include <errno.h>
#include "vhost.h"
+#include "virtio_user_dev.h"
/* The version of the protocol we support */
#define VHOST_USER_VERSION 0x1
@@ -255,24 +256,25 @@ prepare_vhost_memory_user(struct vhost_user_msg *msg, int fds[])
static struct vhost_user_msg m;
-static const char * const vhost_msg_strings[] = {
- [VHOST_USER_SET_OWNER] = "VHOST_USER_SET_OWNER",
- [VHOST_USER_RESET_OWNER] = "VHOST_USER_RESET_OWNER",
- [VHOST_USER_SET_FEATURES] = "VHOST_USER_SET_FEATURES",
- [VHOST_USER_GET_FEATURES] = "VHOST_USER_GET_FEATURES",
- [VHOST_USER_SET_VRING_CALL] = "VHOST_USER_SET_VRING_CALL",
- [VHOST_USER_SET_VRING_NUM] = "VHOST_USER_SET_VRING_NUM",
- [VHOST_USER_SET_VRING_BASE] = "VHOST_USER_SET_VRING_BASE",
- [VHOST_USER_GET_VRING_BASE] = "VHOST_USER_GET_VRING_BASE",
- [VHOST_USER_SET_VRING_ADDR] = "VHOST_USER_SET_VRING_ADDR",
- [VHOST_USER_SET_VRING_KICK] = "VHOST_USER_SET_VRING_KICK",
- [VHOST_USER_SET_MEM_TABLE] = "VHOST_USER_SET_MEM_TABLE",
- [VHOST_USER_SET_VRING_ENABLE] = "VHOST_USER_SET_VRING_ENABLE",
- NULL,
+const char * const vhost_msg_strings[] = {
+ [VHOST_USER_SET_OWNER] = "VHOST_SET_OWNER",
+ [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
+ [VHOST_USER_SET_FEATURES] = "VHOST_SET_FEATURES",
+ [VHOST_USER_GET_FEATURES] = "VHOST_GET_FEATURES",
+ [VHOST_USER_SET_VRING_CALL] = "VHOST_SET_VRING_CALL",
+ [VHOST_USER_SET_VRING_NUM] = "VHOST_SET_VRING_NUM",
+ [VHOST_USER_SET_VRING_BASE] = "VHOST_SET_VRING_BASE",
+ [VHOST_USER_GET_VRING_BASE] = "VHOST_GET_VRING_BASE",
+ [VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
+ [VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
+ [VHOST_USER_SET_MEM_TABLE] = "VHOST_SET_MEM_TABLE",
+ [VHOST_USER_SET_VRING_ENABLE] = "VHOST_SET_VRING_ENABLE",
};
-int
-vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
+static int
+vhost_user_sock(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg)
{
struct vhost_user_msg msg;
struct vhost_vring_file *file = 0;
@@ -280,9 +282,9 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
int fds[VHOST_MEMORY_MAX_NREGIONS];
int fd_num = 0;
int i, len;
+ int vhostfd = dev->vhostfd;
RTE_SET_USED(m);
- RTE_SET_USED(vhost_msg_strings);
PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
@@ -403,15 +405,13 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
/**
* Set up environment to talk with a vhost user backend.
- * @param path
- * - The path to vhost user unix socket file.
*
* @return
- * - (-1) if fail to set up;
- * - (>=0) if successful, and it is the fd to vhostfd.
+ * - (-1) if fail;
+ * - (0) if succeed.
*/
-int
-vhost_user_setup(const char *path)
+static int
+vhost_user_setup(struct virtio_user_dev *dev)
{
int fd;
int flag;
@@ -429,18 +429,21 @@ vhost_user_setup(const char *path)
memset(&un, 0, sizeof(un));
un.sun_family = AF_UNIX;
- snprintf(un.sun_path, sizeof(un.sun_path), "%s", path);
+ snprintf(un.sun_path, sizeof(un.sun_path), "%s", dev->path);
if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno));
close(fd);
return -1;
}
- return fd;
+ dev->vhostfd = fd;
+ return 0;
}
-int
-vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
+static int
+vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable)
{
int i;
@@ -450,10 +453,15 @@ vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
.num = enable,
};
- if (vhost_user_sock(vhostfd,
- VHOST_USER_SET_VRING_ENABLE, &state))
+ if (vhost_user_sock(dev, VHOST_USER_SET_VRING_ENABLE, &state))
return -1;
}
return 0;
}
+
+struct virtio_user_backend_ops ops_user = {
+ .setup = vhost_user_setup,
+ .send_request = vhost_user_sock,
+ .enable_qp = vhost_user_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 8dd563a..32039a1 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -39,6 +39,9 @@
#include <sys/mman.h>
#include <unistd.h>
#include <sys/eventfd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
#include "vhost.h"
#include "virtio_user_dev.h"
@@ -64,7 +67,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = callfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_CALL, &file);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_CALL, &file);
dev->callfds[queue_sel] = callfd;
return 0;
@@ -88,12 +91,12 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.index = queue_sel;
state.num = vring->num;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_NUM, &state);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
state.num = 0; /* no reservation */
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_BASE, &state);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_ADDR, &addr);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
/* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
* lastly because vhost depends on this msg to judge if
@@ -106,7 +109,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = kickfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_KICK, &file);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_KICK, &file);
dev->kickfds[queue_sel] = kickfd;
return 0;
@@ -146,20 +149,19 @@ virtio_user_start_device(struct virtio_user_dev *dev)
if (virtio_user_queue_setup(dev, virtio_user_create_queue) < 0)
goto error;
- /* Step 1: set features
- * Make sure VHOST_USER_F_PROTOCOL_FEATURES is added if mq is enabled,
- * VIRTIO_NET_F_MAC and VIRTIO_NET_F_CTRL_VQ is stripped.
- */
+ /* Step 1: set features */
features = dev->features;
+ /* Strip VIRTIO_NET_F_MAC, as MAC address is handled in vdev init */
features &= ~(1ull << VIRTIO_NET_F_MAC);
+ /* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
+ ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
if (ret < 0)
goto error;
PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
/* Step 2: share memory regions */
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_MEM_TABLE, NULL);
+ ret = dev->ops->send_request(dev, VHOST_USER_SET_MEM_TABLE, NULL);
if (ret < 0)
goto error;
@@ -170,7 +172,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
/* Step 4: enable queues
* we enable the 1st queue pair by default.
*/
- vhost_user_enable_queue_pair(dev->vhostfd, 0, 1);
+ dev->ops->enable_qp(dev, 0, 1);
return 0;
error:
@@ -188,7 +190,7 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
}
for (i = 0; i < dev->max_queue_pairs; ++i)
- vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+ dev->ops->enable_qp(dev, i, 0);
return 0;
}
@@ -214,36 +216,57 @@ parse_mac(struct virtio_user_dev *dev, const char *mac)
}
}
+static int
+is_vhost_user_by_type(const char *path)
+{
+ struct stat sb;
+
+ if (stat(path, &sb) == -1)
+ return 0;
+
+ return S_ISSOCK(sb.st_mode);
+}
+
+static int
+virtio_user_dev_setup(struct virtio_user_dev *dev)
+{
+ uint32_t i;
+
+ dev->vhostfd = -1;
+ for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
+ dev->kickfds[i] = -1;
+ dev->callfds[i] = -1;
+ }
+
+ if (is_vhost_user_by_type(dev->path)) {
+ dev->ops = &ops_user;
+ return dev->ops->setup(dev);
+ }
+
+ return -1;
+}
+
int
virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
int cq, int queue_size, const char *mac)
{
- uint32_t i;
-
snprintf(dev->path, PATH_MAX, "%s", path);
dev->max_queue_pairs = queues;
dev->queue_pairs = 1; /* mq disabled by default */
dev->queue_size = queue_size;
dev->mac_specified = 0;
parse_mac(dev, mac);
- dev->vhostfd = -1;
-
- for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
- dev->kickfds[i] = -1;
- dev->callfds[i] = -1;
- }
- dev->vhostfd = vhost_user_setup(dev->path);
- if (dev->vhostfd < 0) {
+ if (virtio_user_dev_setup(dev) < 0) {
PMD_INIT_LOG(ERR, "backend set up fails");
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_SET_OWNER, NULL) < 0) {
+ if (dev->ops->send_request(dev, VHOST_USER_SET_OWNER, NULL) < 0) {
PMD_INIT_LOG(ERR, "set_owner fails: %s", strerror(errno));
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_GET_FEATURES,
+ if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
&dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
@@ -288,9 +311,9 @@ virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs)
}
for (i = 0; i < q_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 1);
+ ret |= dev->ops->enable_qp(dev, i, 1);
for (i = q_pairs; i < dev->max_queue_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+ ret |= dev->ops->enable_qp(dev, i, 0);
dev->queue_pairs = q_pairs;
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 28fc788..9f2f82e 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -37,9 +37,13 @@
#include <limits.h>
#include "../virtio_pci.h"
#include "../virtio_ring.h"
+#include "vhost.h"
struct virtio_user_dev {
+ /* for vhost_user backend */
int vhostfd;
+
+ /* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int kickfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int mac_specified;
@@ -54,6 +58,7 @@ struct virtio_user_dev {
uint8_t mac_addr[ETHER_ADDR_LEN];
char path[PATH_MAX];
struct vring vrings[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
+ struct virtio_user_backend_ops *ops;
};
int virtio_user_start_device(struct virtio_user_dev *dev);
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v4 5/8] net/virtio_user: add vhost kernel support
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
` (3 preceding siblings ...)
2017-01-13 12:18 ` [PATCH v4 4/8] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 6/8] net/virtio_user: enable offloading Jianfeng Tan
` (3 subsequent siblings)
8 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
This patch add support vhost kernel as the backend for virtio_user.
Three main hook functions are added:
- vhost_kernel_setup() to open char device, each vq pair needs one
vhostfd;
- vhost_kernel_ioctl() to communicate control messages with vhost
kernel module;
- vhost_kernel_enable_queue_pair() to open tap device and set it
as the backend of corresonding vhost fd (that is to say, vq pair).
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
doc/guides/rel_notes/release_17_02.rst | 20 ++
drivers/net/virtio/Makefile | 2 +
drivers/net/virtio/virtio_user/vhost.h | 2 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 311 ++++++++++++++++++++++
drivers/net/virtio/virtio_user/vhost_kernel_tap.c | 116 ++++++++
drivers/net/virtio/virtio_user/vhost_kernel_tap.h | 60 +++++
drivers/net/virtio/virtio_user/virtio_user_dev.c | 36 ++-
drivers/net/virtio/virtio_user/virtio_user_dev.h | 5 +
8 files changed, 549 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel_tap.c
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel_tap.h
diff --git a/doc/guides/rel_notes/release_17_02.rst b/doc/guides/rel_notes/release_17_02.rst
index 180af82..7354df5 100644
--- a/doc/guides/rel_notes/release_17_02.rst
+++ b/doc/guides/rel_notes/release_17_02.rst
@@ -52,6 +52,26 @@ New Features
See the :ref:`Generic flow API <Generic_flow_API>` documentation for more
information.
+* **virtio_user with vhost-kernel as another exceptional path.**
+
+ Previously, we upstreamed a virtual device, virtio_user with vhost-user
+ as the backend, as a way for IPC (Inter-Process Communication) and user
+ space container networking.
+
+ Virtio_user with vhost-kernel as the backend is a solution for exceptional
+ path, such as KNI, which exchanges packets with kernel networking stack.
+ This solution is very promising in:
+
+ * maintenance: vhost and vhost-net (kernel) is upstreamed and extensively
+ used kernel module.
+ * features: vhost-net is born to be a networking solution, which has
+ lots of networking related featuers, like multi queue, tso, multi-seg
+ mbuf, etc.
+ * performance: similar to KNI, this solution would use one or more
+ kthreads to send/receive packets from user space DPDK applications,
+ which has little impact on user space polling thread (except that
+ it might enter into kernel space to wake up those kthreads if
+ necessary).
Resolved Issues
---------------
diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile
index 97972a6..8b5b5d6 100644
--- a/drivers/net/virtio/Makefile
+++ b/drivers/net/virtio/Makefile
@@ -60,6 +60,8 @@ endif
ifeq ($(CONFIG_RTE_VIRTIO_USER),y)
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_user.c
+SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_kernel.c
+SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_kernel_tap.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/virtio_user_dev.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user_ethdev.c
endif
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 515e4fc..5c983bd 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -118,4 +118,6 @@ struct virtio_user_backend_ops {
};
struct virtio_user_backend_ops ops_user;
+struct virtio_user_backend_ops ops_kernel;
+
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
new file mode 100644
index 0000000..9301f96
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -0,0 +1,311 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+
+#include <rte_memory.h>
+#include <rte_eal_memconfig.h>
+
+#include "vhost.h"
+#include "virtio_user_dev.h"
+#include "vhost_kernel_tap.h"
+
+struct vhost_memory_kernel {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[0];
+};
+
+/* vhost kernel ioctls */
+#define VHOST_VIRTIO 0xAF
+#define VHOST_GET_FEATURES _IOR(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_FEATURES _IOW(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
+#define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
+#define VHOST_SET_MEM_TABLE _IOW(VHOST_VIRTIO, 0x03, struct vhost_memory_kernel)
+#define VHOST_SET_LOG_BASE _IOW(VHOST_VIRTIO, 0x04, __u64)
+#define VHOST_SET_LOG_FD _IOW(VHOST_VIRTIO, 0x07, int)
+#define VHOST_SET_VRING_NUM _IOW(VHOST_VIRTIO, 0x10, struct vhost_vring_state)
+#define VHOST_SET_VRING_ADDR _IOW(VHOST_VIRTIO, 0x11, struct vhost_vring_addr)
+#define VHOST_SET_VRING_BASE _IOW(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_GET_VRING_BASE _IOWR(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_SET_VRING_KICK _IOW(VHOST_VIRTIO, 0x20, struct vhost_vring_file)
+#define VHOST_SET_VRING_CALL _IOW(VHOST_VIRTIO, 0x21, struct vhost_vring_file)
+#define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
+#define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct vhost_vring_file)
+
+static uint64_t max_regions = 64;
+
+static void
+get_vhost_kernel_max_regions(void)
+{
+ int fd;
+ char buf[20] = {'\0'};
+
+ fd = open("/sys/module/vhost/parameters/max_mem_regions", O_RDONLY);
+ if (fd < 0)
+ return;
+
+ if (read(fd, buf, 20-1) > 0)
+ max_regions = strtoull(buf, NULL, 10);
+
+ close(fd);
+}
+
+static uint64_t vhost_req_user_to_kernel[] = {
+ [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
+ [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
+ [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
+ [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
+ [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
+ [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
+ [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
+ [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
+ [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
+ [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
+ [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
+};
+
+/* By default, vhost kernel module allows 64 regions, but DPDK allows
+ * 256 segments. As a relief, below function merges those virtually
+ * adjacent memsegs into one region.
+ */
+static struct vhost_memory_kernel *
+prepare_vhost_memory_kernel(void)
+{
+ uint32_t i, j, k = 0;
+ struct rte_memseg *seg;
+ struct vhost_memory_region *mr;
+ struct vhost_memory_kernel *vm;
+
+ vm = malloc(sizeof(struct vhost_memory_kernel) +
+ max_regions *
+ sizeof(struct vhost_memory_region));
+
+ for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
+ seg = &rte_eal_get_configuration()->mem_config->memseg[i];
+ if (!seg->addr)
+ break;
+
+ int new_region = 1;
+
+ for (j = 0; j < k; ++j) {
+ mr = &vm->regions[j];
+
+ if (mr->userspace_addr + mr->memory_size ==
+ (uint64_t)(uintptr_t)seg->addr) {
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+
+ if ((uint64_t)(uintptr_t)seg->addr + seg->len ==
+ mr->userspace_addr) {
+ mr->guest_phys_addr =
+ (uint64_t)(uintptr_t)seg->addr;
+ mr->userspace_addr =
+ (uint64_t)(uintptr_t)seg->addr;
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+ }
+
+ if (new_region == 0)
+ continue;
+
+ mr = &vm->regions[k++];
+ /* use vaddr here! */
+ mr->guest_phys_addr = (uint64_t)(uintptr_t)seg->addr;
+ mr->userspace_addr = (uint64_t)(uintptr_t)seg->addr;
+ mr->memory_size = seg->len;
+ mr->mmap_offset = 0;
+
+ if (k >= max_regions) {
+ free(vm);
+ return NULL;
+ }
+ }
+
+ vm->nregions = k;
+ vm->padding = 0;
+ return vm;
+}
+
+static int
+vhost_kernel_ioctl(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg)
+{
+ int ret = -1;
+ unsigned int i;
+ uint64_t req_kernel;
+ struct vhost_memory_kernel *vm = NULL;
+
+ PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
+
+ req_kernel = vhost_req_user_to_kernel[req];
+
+ if (req_kernel == VHOST_SET_MEM_TABLE) {
+ vm = prepare_vhost_memory_kernel();
+ if (!vm)
+ return -1;
+ arg = (void *)vm;
+ }
+
+ /* We don't need memory protection here */
+ if (req_kernel == VHOST_SET_FEATURES)
+ *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ if (dev->vhostfds[i] < 0)
+ continue;
+
+ ret = ioctl(dev->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+
+ if (vm)
+ free(vm);
+
+ if (ret < 0)
+ PMD_DRV_LOG(ERR, "%s failed: %s",
+ vhost_msg_strings[req], strerror(errno));
+
+ return ret;
+}
+
+/**
+ * Set up environment to talk with a vhost kernel backend.
+ *
+ * @return
+ * - (-1) if fail to set up;
+ * - (>=0) if successful.
+ */
+static int
+vhost_kernel_setup(struct virtio_user_dev *dev)
+{
+ int vhostfd;
+ uint32_t i;
+
+ get_vhost_kernel_max_regions();
+
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ vhostfd = open(dev->path, O_RDWR);
+ if (vhostfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s, %s",
+ dev->path, strerror(errno));
+ return -1;
+ }
+
+ dev->vhostfds[i] = vhostfd;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_set_backend(int vhostfd, int tapfd)
+{
+ struct vhost_vring_file f;
+
+ f.fd = tapfd;
+ f.index = 0;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ f.index = 1;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable)
+{
+ int hdr_size;
+ int vhostfd;
+ int tapfd;
+
+ vhostfd = dev->vhostfds[pair_idx];
+
+ if (!enable) {
+ if (dev->tapfds[pair_idx]) {
+ close(dev->tapfds[pair_idx]);
+ dev->tapfds[pair_idx] = -1;
+ }
+ return vhost_kernel_set_backend(vhostfd, -1);
+ } else if (dev->tapfds[pair_idx] >= 0) {
+ return 0;
+ }
+
+ if ((dev->features & (1ULL << VIRTIO_NET_F_MRG_RXBUF)) ||
+ (dev->features & (1ULL << VIRTIO_F_VERSION_1)))
+ hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
+ else
+ hdr_size = sizeof(struct virtio_net_hdr);
+
+ tapfd = vhost_kernel_open_tap(&dev->ifname, hdr_size);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open tap for vhost kernel");
+ return -1;
+ }
+
+ if (vhost_kernel_set_backend(vhostfd, tapfd) < 0) {
+ PMD_DRV_LOG(ERR, "fail to set backend for vhost kernel");
+ close(tapfd);
+ return -1;
+ }
+
+ dev->tapfds[pair_idx] = tapfd;
+ return 0;
+}
+
+struct virtio_user_backend_ops ops_kernel = {
+ .setup = vhost_kernel_setup,
+ .send_request = vhost_kernel_ioctl,
+ .enable_qp = vhost_kernel_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
new file mode 100644
index 0000000..797713b
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
@@ -0,0 +1,116 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <net/if.h>
+#include <errno.h>
+#include <string.h>
+#include <limits.h>
+
+#include "vhost_kernel_tap.h"
+#include "../virtio_logs.h"
+
+int
+vhost_kernel_open_tap(char **p_ifname, int hdr_size)
+{
+ unsigned int tap_features;
+ int sndbuf = INT_MAX;
+ struct ifreq ifr;
+ int tapfd;
+
+ /* TODO:
+ * 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
+ * 2. get number of memory regions from vhost module parameter
+ * max_mem_regions, supported in newer version linux kernel
+ */
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ /* Construct ifr */
+ memset(&ifr, 0, sizeof(ifr));
+ ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
+
+ if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ goto error;
+ }
+ if (tap_features & IFF_ONE_QUEUE)
+ ifr.ifr_flags |= IFF_ONE_QUEUE;
+
+ /* Let tap instead of vhost-net handle vnet header, as the latter does
+ * not support offloading. And in this case, we should not set feature
+ * bit VHOST_NET_F_VIRTIO_NET_HDR.
+ */
+ if (tap_features & IFF_VNET_HDR) {
+ ifr.ifr_flags |= IFF_VNET_HDR;
+ } else {
+ PMD_DRV_LOG(ERR, "TAP does not support IFF_VNET_HDR");
+ goto error;
+ }
+
+ if (*p_ifname)
+ strncpy(ifr.ifr_name, *p_ifname, IFNAMSIZ);
+ else
+ strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
+ if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
+ PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ fcntl(tapfd, F_SETFL, O_NONBLOCK);
+
+ if (ioctl(tapfd, TUNSETVNETHDRSZ, &hdr_size) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETVNETHDRSZ failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (ioctl(tapfd, TUNSETSNDBUF, &sndbuf) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETSNDBUF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (!(*p_ifname))
+ *p_ifname = strdup(ifr.ifr_name);
+
+ return tapfd;
+error:
+ close(tapfd);
+ return -1;
+}
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
new file mode 100644
index 0000000..1fc58ff
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
@@ -0,0 +1,60 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/ioctl.h>
+
+/* TUN ioctls */
+#define TUNSETIFF _IOW('T', 202, int)
+#define TUNGETFEATURES _IOR('T', 207, unsigned int)
+#define TUNSETOFFLOAD _IOW('T', 208, unsigned int)
+#define TUNGETIFF _IOR('T', 210, unsigned int)
+#define TUNSETSNDBUF _IOW('T', 212, int)
+#define TUNGETVNETHDRSZ _IOR('T', 215, int)
+#define TUNSETVNETHDRSZ _IOW('T', 216, int)
+#define TUNSETQUEUE _IOW('T', 217, int)
+#define TUNSETVNETLE _IOW('T', 220, int)
+#define TUNSETVNETBE _IOW('T', 222, int)
+
+/* TUNSETIFF ifr flags */
+#define IFF_TAP 0x0002
+#define IFF_NO_PI 0x1000
+#define IFF_ONE_QUEUE 0x2000
+#define IFF_VNET_HDR 0x4000
+#define IFF_MULTI_QUEUE 0x0100
+#define IFF_ATTACH_QUEUE 0x0200
+#define IFF_DETACH_QUEUE 0x0400
+
+/* Constants */
+#define PATH_NET_TUN "/dev/net/tun"
+
+int vhost_kernel_open_tap(char **p_ifname, int hdr_size);
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 32039a1..ae0824c 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -192,6 +192,9 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
for (i = 0; i < dev->max_queue_pairs; ++i)
dev->ops->enable_qp(dev, i, 0);
+ free(dev->ifname);
+ dev->ifname = NULL;
+
return 0;
}
@@ -230,7 +233,7 @@ is_vhost_user_by_type(const char *path)
static int
virtio_user_dev_setup(struct virtio_user_dev *dev)
{
- uint32_t i;
+ uint32_t i, q;
dev->vhostfd = -1;
for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
@@ -238,12 +241,28 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
dev->callfds[i] = -1;
}
+ dev->vhostfds = NULL;
+ dev->tapfds = NULL;
+
if (is_vhost_user_by_type(dev->path)) {
dev->ops = &ops_user;
- return dev->ops->setup(dev);
+ } else {
+ dev->ops = &ops_kernel;
+
+ dev->vhostfds = malloc(dev->max_queue_pairs * sizeof(int));
+ dev->tapfds = malloc(dev->max_queue_pairs * sizeof(int));
+ if (!dev->vhostfds || !dev->tapfds) {
+ PMD_INIT_LOG(ERR, "Failed to malloc");
+ return -1;
+ }
+
+ for (q = 0; q < dev->max_queue_pairs; ++q) {
+ dev->vhostfds[q] = -1;
+ dev->tapfds[q] = -1;
+ }
}
- return -1;
+ return dev->ops->setup(dev);
}
int
@@ -295,7 +314,18 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
void
virtio_user_dev_uninit(struct virtio_user_dev *dev)
{
+ uint32_t i;
+
+ virtio_user_stop_device(dev);
+
close(dev->vhostfd);
+
+ if (dev->vhostfds) {
+ for (i = 0; i < dev->max_queue_pairs; ++i)
+ close(dev->vhostfds[i]);
+ free(dev->vhostfds);
+ free(dev->tapfds);
+ }
}
static uint8_t
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 9f2f82e..0d39f40 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -43,6 +43,11 @@ struct virtio_user_dev {
/* for vhost_user backend */
int vhostfd;
+ /* for vhost_kernel backend */
+ char *ifname;
+ int *vhostfds;
+ int *tapfds;
+
/* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int kickfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v4 6/8] net/virtio_user: enable offloading
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
` (4 preceding siblings ...)
2017-01-13 12:18 ` [PATCH v4 5/8] net/virtio_user: add vhost kernel support Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 7/8] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
` (2 subsequent siblings)
8 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
When used with vhost kernel backend, we can offload at both directions.
- From vhost kernel to virtio_user, the offload is enabled so that
DPDK app can trust the flow is checksum-correct; and if DPDK app
sends it through another port, the checksum needs to be
recalculated or offloaded. It also applies to TSO.
- From virtio_user to vhost_kernel, the offload is enabled so that
kernel can trust the flow is L4-checksum-correct, no need to verify
it; if kernel will consume it, DPDK app should make sure the
l3-checksum is correctly set.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 40 +++++++++++++++++++++--
drivers/net/virtio/virtio_user/vhost_kernel_tap.c | 14 ++++++++
drivers/net/virtio/virtio_user/vhost_kernel_tap.h | 7 ++++
3 files changed, 59 insertions(+), 2 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 9301f96..a5d3e1e 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -165,6 +165,28 @@ prepare_vhost_memory_kernel(void)
return vm;
}
+/* with below features, vhost kernel does not need to do the checksum and TSO,
+ * these info will be passed to virtio_user through virtio net header.
+ */
+#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
+ ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
+ (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
+ (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
+ (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
+ (1ULL << VIRTIO_NET_F_GUEST_UFO))
+
+/* with below features, when flows from virtio_user to vhost kernel
+ * (1) if flows goes up through the kernel networking stack, it does not need
+ * to verify checksum, which can save CPU cycles;
+ * (2) if flows goes through a Linux bridge and outside from an interface
+ * (kernel driver), checksum and TSO will be done by GSO in kernel or even
+ * offloaded into real physical device.
+ */
+#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
+ ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
+ (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
+ (1ULL << VIRTIO_NET_F_CSUM))
+
static int
vhost_kernel_ioctl(struct virtio_user_dev *dev,
enum vhost_user_request req,
@@ -186,10 +208,15 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
arg = (void *)vm;
}
- /* We don't need memory protection here */
- if (req_kernel == VHOST_SET_FEATURES)
+ if (req_kernel == VHOST_SET_FEATURES) {
+ /* We don't need memory protection here */
*(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+ /* VHOST kernel does not know about below flags */
+ *(uint64_t *)arg &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
+ *(uint64_t *)arg &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
+ }
+
for (i = 0; i < dev->max_queue_pairs; ++i) {
if (dev->vhostfds[i] < 0)
continue;
@@ -199,6 +226,15 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
break;
}
+ if (!ret && req_kernel == VHOST_GET_FEATURES) {
+ /* with tap as the backend, all these features are supported
+ * but not claimed by vhost-net, so we add them back when
+ * reporting to upper layer.
+ */
+ *((uint64_t *)arg) |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
+ *((uint64_t *)arg) |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
+ }
+
if (vm)
free(vm);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
index 797713b..cdb5c3c 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
@@ -50,6 +50,12 @@ vhost_kernel_open_tap(char **p_ifname, int hdr_size)
int sndbuf = INT_MAX;
struct ifreq ifr;
int tapfd;
+ unsigned int offload =
+ TUN_F_CSUM |
+ TUN_F_TSO4 |
+ TUN_F_TSO6 |
+ TUN_F_TSO_ECN |
+ TUN_F_UFO;
/* TODO:
* 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
@@ -106,6 +112,14 @@ vhost_kernel_open_tap(char **p_ifname, int hdr_size)
goto error;
}
+ /* TODO: before set the offload capabilities, we'd better (1) check
+ * negotiated features to see if necessary to offload; (2) query tap
+ * to see if it supports the offload capabilities.
+ */
+ if (ioctl(tapfd, TUNSETOFFLOAD, offload) != 0)
+ PMD_DRV_LOG(ERR, "TUNSETOFFLOAD ioctl() failed: %s",
+ strerror(errno));
+
if (!(*p_ifname))
*p_ifname = strdup(ifr.ifr_name);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
index 1fc58ff..8535116 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
@@ -54,6 +54,13 @@
#define IFF_ATTACH_QUEUE 0x0200
#define IFF_DETACH_QUEUE 0x0400
+/* Features for GSO (TUNSETOFFLOAD). */
+#define TUN_F_CSUM 0x01 /* You can hand me unchecksummed packets. */
+#define TUN_F_TSO4 0x02 /* I can handle TSO for IPv4 packets */
+#define TUN_F_TSO6 0x04 /* I can handle TSO for IPv6 packets */
+#define TUN_F_TSO_ECN 0x08 /* I can handle TSO with ECN bits. */
+#define TUN_F_UFO 0x10 /* I can handle UFO packets */
+
/* Constants */
#define PATH_NET_TUN "/dev/net/tun"
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v4 7/8] net/virtio_user: enable multiqueue with vhost kernel
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
` (5 preceding siblings ...)
2017-01-13 12:18 ` [PATCH v4 6/8] net/virtio_user: enable offloading Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path Jianfeng Tan
2017-01-16 15:05 ` [PATCH v4 0/8] virtio_user as an alternative exception path Yuanhan Liu
8 siblings, 0 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
With vhost kernel, to enable multiqueue, we need backend device
in kernel support multiqueue feature. Specifically, with tap
as the backend, as linux/Documentation/networking/tuntap.txt shows,
we check if tap supports IFF_MULTI_QUEUE feature.
And for vhost kernel, each queue pair has a vhost fd, and with a tap
fd binding this vhost fd. All tap fds are set with the same tap
interface name.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 68 ++++++++++++++++++++---
drivers/net/virtio/virtio_user/vhost_kernel_tap.c | 5 +-
drivers/net/virtio/virtio_user/vhost_kernel_tap.h | 2 +-
drivers/net/virtio/virtio_user/virtio_user_dev.c | 1 +
4 files changed, 67 insertions(+), 9 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index a5d3e1e..0e45dd6 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -188,6 +188,29 @@ prepare_vhost_memory_kernel(void)
(1ULL << VIRTIO_NET_F_CSUM))
static int
+tap_supporte_mq(void)
+{
+ int tapfd;
+ unsigned int tap_features;
+
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ close(tapfd);
+ return -1;
+ }
+
+ close(tapfd);
+ return tap_features & IFF_MULTI_QUEUE;
+}
+
+static int
vhost_kernel_ioctl(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg)
@@ -196,6 +219,8 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
unsigned int i;
uint64_t req_kernel;
struct vhost_memory_kernel *vm = NULL;
+ int vhostfd;
+ unsigned int queue_sel;
PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
@@ -215,15 +240,37 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
/* VHOST kernel does not know about below flags */
*(uint64_t *)arg &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
*(uint64_t *)arg &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
+
+ *(uint64_t *)arg &= ~(1ULL << VIRTIO_NET_F_MQ);
}
- for (i = 0; i < dev->max_queue_pairs; ++i) {
- if (dev->vhostfds[i] < 0)
- continue;
+ switch (req_kernel) {
+ case VHOST_SET_VRING_NUM:
+ case VHOST_SET_VRING_ADDR:
+ case VHOST_SET_VRING_BASE:
+ case VHOST_GET_VRING_BASE:
+ case VHOST_SET_VRING_KICK:
+ case VHOST_SET_VRING_CALL:
+ queue_sel = *(unsigned int *)arg;
+ vhostfd = dev->vhostfds[queue_sel / 2];
+ *(unsigned int *)arg = queue_sel % 2;
+ PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
+ vhostfd, *(unsigned int *)arg);
+ break;
+ default:
+ vhostfd = -1;
+ }
+ if (vhostfd == -1) {
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ if (dev->vhostfds[i] < 0)
+ continue;
- ret = ioctl(dev->vhostfds[i], req_kernel, arg);
- if (ret < 0)
- break;
+ ret = ioctl(dev->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+ } else {
+ ret = ioctl(vhostfd, req_kernel, arg);
}
if (!ret && req_kernel == VHOST_GET_FEATURES) {
@@ -233,6 +280,12 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
*/
*((uint64_t *)arg) |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
*((uint64_t *)arg) |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
+
+ /* vhost_kernel will not declare this feature, but it does
+ * support multi-queue.
+ */
+ if (tap_supporte_mq())
+ *(uint64_t *)arg |= (1ull << VIRTIO_NET_F_MQ);
}
if (vm)
@@ -305,6 +358,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
int hdr_size;
int vhostfd;
int tapfd;
+ int req_mq = (dev->max_queue_pairs > 1);
vhostfd = dev->vhostfds[pair_idx];
@@ -324,7 +378,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
else
hdr_size = sizeof(struct virtio_net_hdr);
- tapfd = vhost_kernel_open_tap(&dev->ifname, hdr_size);
+ tapfd = vhost_kernel_open_tap(&dev->ifname, hdr_size, req_mq);
if (tapfd < 0) {
PMD_DRV_LOG(ERR, "fail to open tap for vhost kernel");
return -1;
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
index cdb5c3c..f585de8 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
@@ -44,7 +44,7 @@
#include "../virtio_logs.h"
int
-vhost_kernel_open_tap(char **p_ifname, int hdr_size)
+vhost_kernel_open_tap(char **p_ifname, int hdr_size, int req_mq)
{
unsigned int tap_features;
int sndbuf = INT_MAX;
@@ -91,6 +91,9 @@ vhost_kernel_open_tap(char **p_ifname, int hdr_size)
goto error;
}
+ if (req_mq)
+ ifr.ifr_flags |= IFF_MULTI_QUEUE;
+
if (*p_ifname)
strncpy(ifr.ifr_name, *p_ifname, IFNAMSIZ);
else
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
index 8535116..eae340c 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
@@ -64,4 +64,4 @@
/* Constants */
#define PATH_NET_TUN "/dev/net/tun"
-int vhost_kernel_open_tap(char **p_ifname, int hdr_size);
+int vhost_kernel_open_tap(char **p_ifname, int hdr_size, int req_mq);
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index ae0824c..6617bc8 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -93,6 +93,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.num = vring->num;
dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
+ state.index = queue_sel;
state.num = 0; /* no reservation */
dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
` (6 preceding siblings ...)
2017-01-13 12:18 ` [PATCH v4 7/8] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
@ 2017-01-13 12:18 ` Jianfeng Tan
2017-01-16 6:00 ` Yuanhan Liu
2017-01-22 0:46 ` Aws Ismail
2017-01-16 15:05 ` [PATCH v4 0/8] virtio_user as an alternative exception path Yuanhan Liu
8 siblings, 2 replies; 73+ messages in thread
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
.../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes
doc/guides/prog_guide/index.rst | 1 +
.../prog_guide/virtio_user_as_exceptional_path.rst | 104 +++++++++++++++++++++
3 files changed, 105 insertions(+)
create mode 100644 doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
create mode 100644 doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
diff --git a/doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png b/doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
new file mode 100644
index 0000000000000000000000000000000000000000..9fd45e13eefc2ef39ac2c389f407d60ab4144c59
GIT binary patch
literal 38600
zcmeFZdpy(qA2&W76`>TJkPbu+9mM6hkRnDo=QL5tVO(L(+mv$(Q4t~M97nNb*gBA;
znuUfnD`#ee88+K~AL_cUuIqcae~0`2J?=m5KN9o#ybrJA^Ywha-xAK7pWU)~_ht|X
zw8g~uv?U0%&Hw~j(=50F_(Yg<KNk3JorgX|9|WpN7Fxcv9{73F@5UDcKp@GjEB^%E
z$<2X5pa)(ir}eEvTo_!Tt}AvIM>*vi-TI8vT#u~x5|-A<j&eQlZMUTfTP;0M!!@i~
zh<W|d+IRu|7y8=U4T4(<Pc%gmBE!ry)U5q$OE);WyBk<L2nnCdcUpg2!_vgqvfeKB
zoTb8=N&cMA!U=lm#Q-WR)4QsP#YVnayi?d8JY6?}d*3t?(O<W~+YJI8*K+Z{dpY?s
z_Hx`hV7(xa>k&q(l2h%zHLE{W-)EUd$ev#N^M`@`f-z!0zOOqRz4}dL?_;qa&t8o(
z5M2EzW6NgQ)o)zY{}(*;+u^)FCcKo~EUC{rw>lRH^QkS>`=G<RUU^N^9}D{N`TxYP
zk@bMIMiklnAG?F7-d}#N2$MPzTF{)cAihm_RH@DFzQ=?ph#8dH?D9IN0z76cl*WqK
zd=GfxMRdfueDTSqcGO9VY|UuTocROS-_WT&yw6-wpKZ!(oDTLL=v99BPP5F&m2T%j
z7uD)JR5NNNC;+@GH}v@Q##uWhzKsRnt1L!z&oavF)jHP@zqB!4KVez2`eKCsse<))
z6`r4RgSnJsl%rAXFNk*3%DWlsxb2mYbxc(poXOawwH4b+ha%^dkMHa++D07>lNYJ>
zT&CZ=vYC1EeP6>z?}rknnDq`~D{mg%$_^D*-`YW3ev*$(p=2M}%{5hgSA*T}slTpA
zjfMMqVm)GNtm9PztX#qxcphYSI&0xAhthhI9lGOi`>peAiCwOpi{y>%+k3K==#BT!
z($2Y4%48m_xRC4qK#*V5(xRg{Q#FUz#Mkodz_vbC;<FCjI^T0T2+8)*;`$u$F2{d_
z7sa@T<0bB`EXnM2H&%L#PluW?g4qJ4^oXWH52#IPd-s^L2pifoIMkA=B7-Y~lFQql
zZi<(HG5Parq4Vm&h$9Kekpx6|49{FfhBlv1@22$CNjZ#>_!G{O2q8U9Y&QW>>es-e
z>%qd|;BiM7Q~3gfHhW?IWFDJ8qt}JqGZtFZ&Itc_*f~~O+?jWB8&#Ce5387e{w9}E
zJ!O|UajbLhTzYvs*}a(a#Mjw*SVvVoAv;$u%ba+`+nY*RQ$AMjMnrSYdFy)AMwYf-
zgr1$RGa-8D_R~AaM>(<53{jjJqczM0zU0N|vfD@}GM$*CSII%F-u^{6b&RSJc02GP
zo9%|~?z2Vs(EJoWskfrtKQGMJDb+#C`}$5pb0_ED_s>3|SYxZI{Id@L;&MDPFC8i+
zOo_HCh^o_D3z1qJKmF-}-42)`B>^vDJ+G<tyi?!DYY7KCv+*YTgjA%Wb8?j`GAUW)
zT!Xn;T#v2PHOd@20#c(I)<WW5)VM(6UQMn;<co`Ii@+~>fEB6fS}7LFo$|g7w2$ia
zZj;A3XORzWNbvEY@^aaaiICL32auA{+jVsx4(g6@PIuD_%(b}Qo_i5@yeJyAgkHM6
zGgHTVE{rjKjWlP)(JuqHwhU?^u;kBZ?uGHXClCT@6s~N1xGCtG{((B`^;$-sv^yA!
zQ75tLG?BZhabae-cZeilt3Jv{HrSK)NeesXU2@hs`s~WZz?2ekCr-J&j$n6!kCc=#
z#eE*_!m>MgM}>&IiW=@>QKsv3*>=Y`*eXX#9=LOi&sme8Qc@|4u~}kO9qGU7jGih9
z>2Nx^32Sge&u(J}GMCe^J5ZOEZp>-YMYl(vOCQcflQ3S|fl^v2$?{IP<ol3Ef!UB{
z=)7{!`#>2_%U0!_?cKX38hZ}n84n<%$?aOGLs-4MQPeHG`4X?F)rm+CaYG|Yr%u{(
zvU2=*n!9!Vm?9e>%N%v?hg?qB#yH<N9szSPy_8Y{gOyLS*km{&N1~(xvfDXl9(9LD
z;JrtpN!*U0n`kbXkhFAV%sXQqLfb$^hdGl*`-qe9l3Tv%Dy`Cfq+HL*c_sL5IJov$
zmnIM(tM5nLIJNPd*5TPGe6fIwc-G--f%Uk?GJUUzxL9+`va!MzX%CVBw7a<r^(1dg
zM4c`!sZ5}LN;KTu+422(N?C=nR-1jCOfB8j9!u<GUCTOYBW{rO``FiBIc?k)tsA$}
zJpws{+)u?JkBva#NoVBO+R!`P=^Gc>c6Q$7yv8K+pvlnP&VJNrdrllQy{g1_O?m8!
z*VOfTbS@1-;mGb9@;0vDEopB@yo8jFuHU2%nOvoK{gMOO=45C$>^q{Ql+ZFt(8Ciw
z#OA3keB6}K_P)3Y_LVPwecSz*@Ymq8oTn-!xA0Xx*E`5y-3_cSI7BgrCK<=%!)ew{
z8^(z93%m^Pyuyzuqd@L@Vf#too{UZ}-<vHQe_y13`DytEWRA$E3Ms8q<zai`2}RBY
zYw5U6r6Rv`2jLy|8r8NqovicTjuXQ5mG9(eaPeX-SieI75C8@$VYOGT)k0txtEIv2
z+AR*u>edOKpO*O<N)?e5)K_>)$k=9b#+h8Oyv|a7hfS;=uyI%kvA8qf0;P1cU8}-k
z?6hu6k~_88iC9#Y@FYhFxsA#?*-Pl}p2sel-@zH2+JnGqAf3t0>+dV#(32rv6a8}H
z({_y4koAxvF1czG{`>hl8ep#>cYMRM<_|L#c=a}ZX}j2afk@hOj)!)Tkor0=B}RYz
z;Xz51yHrJ8QOm9@SL-cLxCd=16t$OKFKcyOI)gZY=RJv~nJFSomG5}|rEACGi`?#@
zeN*GSx>4_1!B22a9y-M)3J(5_Y;uYWUrb0Tp1}AMa^M)F@tIGfwmZ(BdnB3FjOq61
z>|JM>Y+;AD7|Rw@2ofeUusq*<Tms^j>S0?1FCq6Dc?Yv^qM?ZgKi7#7bYo`N-^icB
zeO>SdM#X!U;?x%#HaAqA9ruT8GndQ4xI2Gt<@$|NRtt7a%%p0RmaC7VW#QPVzP_}~
zLbeIK6ZZNY?J4yNCot8USdQxs8H05TRC8VAH`BkT%~nNS8D5AIc~7pt-)CW&$-7xK
z?jILRN{G^hTGdrza^P5_lRGR{baSV!+aqmUuP&%+E_0M~+U63GrrR;UlRa#=!(n2K
zT|uyc<tjtd1{qrs7`r9%d>J`j4j+;o_8}?pG5!1cb&y-A56;1k)>Nd@_0@272;BQr
z;O<rGChV*q_M*q{#mm`>xW){AJE|KxsKd8=rV`vRHuNGVJ<~e9$Ci);=X;HAY-H1_
zj5bh%ybZHnS?y?Q&`-}3FV4t)T+yI<pML~ve#Q-nLeJUtsjo=<P2a?`wg^(i#K$}>
z<SXa-%mJYfVN<~hB4f?gxmaIW$VF6VzSDaGrqWR?e!kab7)mBnuOf1k55T3K!N-uW
z0EY3*dcmgh!!2kJD$MgZb2V(#`G^(PP=067=wCD+Pol#R8$6Ilvoe<{jz(o0@@n9G
z%W-+3(x7`iwqV&QhZo-?jv01dnH>ew^NG-;y2GyR3CU4>@88q0cbg+fN8o1ud!NZ2
z0#b!LY`*t0{<|&w>{%(4iXG(1{N4q04O^?LSd?Bv4mg|JIqJKnAwWNU#~g)L@_f9q
zq3Mkand9^9DcwFIJ&Mp^&Kbik`v-=**MGJuY5uX+ic_xZ*K>t%TdJ4@=c$~dJk?pv
zZK9vx)i8p$HpnUX#h9O;Io0_n6Qjt-FX^pcQzY(1kpytg?^&;6A(I^5a1Vcnt_vFC
z(kFo}Q`<5L(=FX(rn0i$G_M7@xnR{oc91wixn`8S!)`v9SCd&hIG(t@e%k_8J-D{Q
zJ)2>}gCnT>8%bmZ=cNAo-^S^=l?BsOS9mUED3PqWP-!=?>W&WT3am5(uzi<_JDD=e
zsIT9~ulr2<)IXxE!&9X@%irUv-0Ip#LZ%-nC74kt=}h~f#~h+X7rY)41<$f)B4p}!
zD5%ZwYIB2?ZCfg?6;LwV=q3<X{~GwvJFQ@)dk=u+dItcq<vg~@7Q=oM%G}McvWbB}
z3)}8zOefO&>#R%KS&DpI|L8}JD=i<Niw#g}&1c%54Hs7F^A;-ueCsa!Sn4^gt#tzD
zbrhIu&q#g8a!v0-i_O&nt9vzO_EGtD5I!M6rJ#LKSNTy-fcqhgO>gChChc`43Q>3S
zM)CNR42ND~p2Pl*{Sc0XvYI$A`?GQtpA0!!+_xNUDV_=5eNp`)MO({q*--|!eJb$g
zbYe<hi4~odOTDKh^V#G*h5(cGkb+?WCxbH#>9br_q~k4=?sPzKHuUU7re!eoZP5?q
z-w*uizP7Xr_6qs9D5_U|?OCvXhsjql9Ivi&yedh{R>`=@xu6-N$Z4Mw%bh?6@9tS=
z%g@UEo#XhK({FTX{r%@p3h32A%-(d|UXJ(g-TwX+CFZ4bZk_f5?>;Rkn;Xu_T<yJ|
zTU+9yO&?qOmUy%@pjk3M4p(rB^H>G-78_rxPAOlehvkZHWVOoS)5_>|1-A&^$GdF_
zIeJFfT8BOYRbgahVbtk8#x_)iZyT$%u0LUW=Xi)OsGt-W1{*(pE>>sEU?YRC_-!i9
zR1yZ_Z(sj{?`=i=`FojN<NGK`+Gy2wF1=b=9<FgFDdA!k5mY@d9QuC6`_B-?Idp<V
zV=UVz+G%oUrB(He(EwbYffp-iSwplKiR(+YJv!<5#i!gi{65N=jBHfl$nAhQ^~~p%
zUx0t`eogCr&?mt^8=IE|O$Y#LL)RlqYxKKaU4*vS3U}1jA9;C8UQ5Bt>dDx<4S6v6
zqv}UpJYKm6AGL-^#P-`lqj$~*?Dx^OY9)(4hi`lKF=!&dC$*y8m>TNVT1u>B+YWVD
zLqvRZ>At3W409#Nh`C3clkVm;giSGZ<CY6`cxJ-1X{%g)Qo3)HGNH^h!;#8TggV({
z$LK<NrTYsdlqVPH_><=7`<UBhWJN+w-e!nl?keVOmy6ECy=)Yjm_(Ok7E`!f*h{+b
zQ+T9Zt6UdsdT+Hs63^(V5LF%{WwZ7}9=6*;I(E4nO3hY9?m0;}vy32BeyE(G3r})6
zvnOphnCW9yTD!{VClF*(pvz-0pG_XF5<!g`bb4Qs0^fyD1un&8mc6<pg@~9iUq3kg
zaU#Iuv?PS$ji&FWj#T+6kYJdbRmBq-eRXEj@#8pX9g!0>LBF~KNRa)~1#~a=mn2!@
zswG^BdZLcHLB8>bdaklG>&rOk;{)oI>e8s=9MdrBECXEw<A1u!NCN+cPT;pgyDwiF
zgU$Ov9Tu8Kv@Vusjg71K=s_Tp%gw}8cB1bNzDrQ{Y3nU`?!f~3>9^DJi7Kis=(C;k
z<yv+7iWr?&tQq7yq7ZYG*xp55tDtqP4xi7j)T#=bnxI>CYd%?a{A4+LSM_F)!t-mi
zE^?qQxO}r0?_&av9E<VyK{M=3(_ht9(P;JdcE<C;&+C+nk1(=$RXZRPB3gsVxe`XW
z#+-3vE!=Z!y%#|w^e03Y>*Ct9cfL%Msrbx2L=shpzvvFsh!T0TJY+3@xAbMMh#cb>
zOzxIeNL_1iQ~L`)L+a&APhd89!5WbUQ-fs3?wL=UZdV?dPj|@5pda@M4VzM9@I(Cb
ziJqCliv(*rzCye8o-~w()COnuZx|*Qcv=aQOwfeHQ)HWkMp<CLC8PN$UGFeN#~f9<
zyfiapLesPL0ujXxHX~P#M(drSoNr<0oDr%oy%Opm=JO1~n<_27qTW{1)Q7Fm#bpO+
z#HgT4&ley%s;;n><?u4zFY`*zm%^%HOb&LOaT2KuOO$2fE>X}V@(rB3+E1t*SUt4S
z0=7grHST&&{+>->J<I-%xjkI07-iGFDTik#->+rQT>A`nBbP6RD>ixhNuWYk?_?W#
zP+kub)ecl7z5B49ZqP%>hIkcXU(pQeTZLkL@48d`@<~Hch9*u#S2Zw{4dr50$QI_z
zdYGKRJHnka1?`{QzbzD1on5G>rQ@7Ge^y69k{0gxCekz{vJa0<FT2B=ATe7`c7@*1
zS$5uxq0NU-YRK*dqY1n(U~^(y+5Y>|8vGDOgM%80e8jm}8b#A!(rg9>G9<(+2;<YW
z7URe3P|dKeR-2kROA5&@iP7D<L~At)W@xZ>KPC+p)8n!AWxws{d=;mvevQL~pH0TP
zoM0|sq-hIqM4uif*jd)Pg-(KfLilms<nuqnn91E^>`a%9v07(=q7IYFPYfXw1eb0+
z7TE`{wlUiH04et69*z=u9~#^V-F&}_CJnW#H6h(7)w9_605;Tbdu6R%c1UHrWMEl7
z#op#g^`?Y6eac>o@AQ054ZYB1;|xo3sCew)sg3M+ePwhXL$bJqUS_^gBF0GSWOL|^
zh=Es*R?I~WDH&ZF;$^iWu0qb4lf}v^-k%MbnkQ&7sV>lzrx-dtoPsZ<d?`LUHI8KX
zPcq=|p;3KKR=BFO1!ze&j>>|&#{Y<#<INW~R%OCBoOaNu8Kr|Nd|wo&ok5r_+rbk>
zg5ZyPt^$zys-s_)e!jT@V+Mi+x7I?~jhHmA`vH#kC1h&1ApD@81sYV2bj!gkZ1M~i
z9_p&}i)zh0aVQq4ou7f-vazX!;BJ>8O6i|X5^CBqipN#!av;{s(9-#9%;)3KEcz`Z
z+&@?Ht!c7%cfAj(p%j-x)OrYkjLdsDLxbn*%qSTSIcHEMqlhfBk+SjFQVu665{|Va
zBEl=C-TO-B>(E|Vbzr!{&cYsVcP}ZnRx$UpOfYK8gg=CIWlVEux<9_4y^CC1Mjn*Y
zT#03%%f)LA8V)`8zV7Cw4s(R(!x)|?##zB4<Ix9<Hg;p)%VIlOSIFAoxKAeWv>dqn
zA^1qw#^N_gcbbgkT9=C3s|T~)w@q-qbZ{?8Ib-dFPO!AB=0o^>MZ0cJa||U9alIpX
zDBmpPN>~ByG3AF{6Ter5%~^YDPf*Fd{<f7*K?R>2M=QmSH$#;LUg#N~rR`p21q(hc
zv^vUOj>pD>GBgm*HRPR$rWIx>@<OjNot<|Bl#<o&c{L^lFSB{e$|y~q0;lB<1B6)Q
z7iN}ug;+jcbmij$lAl!drd!DJwV<Km;2S-3VVdvC_^#GGx}?Qt8PKV#Jn!q2h;EgY
zr$Kx4!OYqH3COv)e*Id*AM|AP{bdbd&qc*m>I_s}_kWnyRZ;z$FE2Wh(tpsSy|OF1
zS}9)}4hl&s$NzZPr--`^DoFa{hv$L@vOmadALakfL{B_A)O*AvTRvLMKv353Y^$v5
zeGn)GGI4Wl1|qdf6ZkM%%u;dVA;6&1a6KS7ICAB|=zAg{Q19<~`q{s`0CV0OYL(pv
z0zFMoJ??bvYs?&=-Ny&PB6n9tin#mDV6EySN#L<VOV{Jqfg*hd5``sqhkR=UK04f%
zY<LhDTO4}%dB+Q(QJ`aR$^^N0zW``o*5dxxF&kWdQw1J7H#;b{37D~2a+iwQ_gA5S
z2+pU<#2JG?Z$}<S3vDTj{&mg|(g}pOllMV#kM{gBtN8<`10<pMdTkfCy@(b7xQ?F_
zfe!B%KK0ier?y@Kt?>aG2K85KJcv1c5ffIq4!Sr(-e--GpuIXFzt4AG#p~jPK|3>7
z7XZ2?xK_-yvyoY(I=^=4jn3RdqEq8yi>k|HU!8XiiqGq|?4v)$4<>_=^;Jizp1aLY
zdb#_y_BVywfh&t!w(%dOnQN7H&w(DRhEp#B`+ppN6eOSyA+NbfOp?-;@C&C<Y)!p9
zsPoyOBM<A=CmzEVls?WgN_qBC>xd4o%>sGcNDS1{<4sfqHem$p3}CaPdCPM|^lgd#
zOqI<jJx6oBrJ-R*-M;U1PD1Wt!Vpdk9oAj`pnaZBT4T^nIsfKM5-vYh!Y?qJWlxQT
zPn{Bog!j__HXweB@@c2A@uIV4Qzu)c!+l1v)c^t>$WK(!a?^EqZ`|9==(rgtp*MLw
z%uWzkP*JaU$6pJ=Nr*KjBPQL?Zr@fL{}^1}StpiXd-NOmuA-bzU(gtFV83xURS9(T
zV7QdSUn0m|bMqFMZvg&UJ4HGFSt0r`U;SYhmf-U#XpAub`aENSZ4cU+*d$R2*xUKj
zJ0n3^#eM?%Ek^6OlgWy734{K{HyW{{3_#TLov+oeXxADS$l$#QokT4KlSB>nOEfRw
zg39x+H;3?_b9?QUY^jxQ1dAOpECwariL3>+a`MuBvA35_`fDHl+r{pg+s-Q=!tLdv
zdD>T~2Jz-dMkxC}wg%E8lAS@%KFA7y1t_pX<#{UiteFfjPZ^BNwUrX$H2I=QZZXLD
zYOcx-<qhMb4YgN@d%}PTjJQDe?5EPJ*4+H0ec0zBTat6nS|D!=AEFvIddg>#(Lp>m
zW0X<_ZEaBDHf~2rnr4krPkLn=#P{DB#6DACYbw~Z93)DEnPHKdx7b2}asM#mN9)&R
zn^B%Jr=ssYkL-wrR>__GCKaH~Oj_5FcsdQ|enpTSB*DK?(YFC+xBOshlhA3(5ZJ>=
zh72gEy&6Ty-g*7Px%7e<321W%q9?F|QDcbSm~nu9pd2u^?OQkOdr>?i&fdG6y{D9z
z{hjW8!n1o*b?&4s!nZYpa*1RRjW;UV&RyQ>h?UT^%te124_YKT%=i_RYDtu~PDmYw
zK-X5ZDDojXeR$kx&tR;J=+U#E!M1>ZuXtr7B&{LxE4Wkk<go>MCl-7y@4ya(>APN!
zTq)^tl8JveMdr;HkvkIlOvbUIF=uJaR8<-Z)f!V&D&{uM{)Xo5{ES$vfW}>YG7kzE
z;yeIC;!i<Ot+`1U+-I{hUPusD;3rVp5Oo<4a)a$TL}EgJ?<Kzg(i*eQ?0pI|qqk}m
zqQ@xKmcgxEq&0V_!+Eeu#mn~{fOX<gCM)qLq|GQ*H<g+ywHj8cJ^dxcm~(r5LQ9G7
z#vBAmSlu!IOu08%)lx;bWO=?R@M&a+!3_yvA5GY1S$hA>z`~Vojq!Vb4R|%Z8G0G$
zK4$#x3(ArU7SN}Ca%=Cz;+vP*6aBjf2_toPj$vP;Tub}!pAQ~AC&Ip?hQaH_Nb-YH
ziw)u>LO&N7DRCZ2awqTKv!n}*yk(OC`@Q@O`YL9F4G=kfGQX@}#}~+P=!jpMA6D9V
zyIzX@c{|@WEiX~_Ge*y7?J$kkX)Q?74q#V{s#SUglmfE5%=O;!Ykcofs}~Z)?@P&a
zvM3V!1aOhQ>$0M~nJRbl@J-EAr$~HOUNCMEW?VyC;DSCC4_d8w(#2f+^}j}cb;!qr
z0*STEC>8ic_L8{5bphFQz>Ab47O`!@Ho`S%7Q<1Hgl6Bt%3dW*a~|%&5HhC(cdhu$
zMLrNFen}+2P-|}H+SAFz;=;{>nm+G>np?!0o}CstqB$JqeVhL~so21njDB{fmwFtM
zyY={rn>^*!ZT>Z-_+$dk0tG!`S9dCGH(RMYe3O<Kdw0im4;J*94x$T=(7LLyZj
z$6#$aX%mz5i*)}?Ux4~|h%FB_`W|l>k4bT&Y?U-*&eRVrvYna&is<CgD#ySXntuUD
zqo@m5!Ymi))1FeFF^U!1(g<YIEV;7+m&jGlmv&`#VJk8qD$zszB3$)d2+l2C6^~=h
zQw}7z*s78v%8%_E)K<(VqM#j*;Jg~G97E016pB8teSdcv`nOEnqlVZl$hM6ZQhB~l
z2inQJl%8cTxXWLgIkf>QJkGiF_)NNYaNwhe*|u!Gd)=rFw~8RLr#tJoJW4Rr(W`6`
zel2#@H2qd=T@L^D_q-dDAB9{yE38g+m|*c+Uh>xXk=m}MK@_9!^@$*kB%=15t8#n0
zhqm~p+DtjNw*JIvEV*w$NDUGr#<y?EDjYvy5lCaB#eMrqSkLpwz>4RSZT>#?;-)6^
z4lB|othxG0)`Q!>v7f7-+iozxas=`ekU<CH-Qw|EN>r7%EiyXx-$WW;E7Qg4ar&I^
zV4&D;9@SbfK?WTo$=#c@Xucg`dQ0!4WIFSiWoCbwTe8%u^~eBu_E+n<liPt7S61vo
zJUCO~Es$5K;M01(MU0OTlL3rTTS>`5%+gs%_T2R01fUI7$RYCI=fkd9J*sXMZIX6P
z%IE!EiN&S@`g|?4_=W5cI&_@;Fr>=r)HQcS$~SO|jXcX?Z<{`Np<qHqvK@$Hh-Qta
z6;^^5a0Di;bbq4J^v5yFv6|5>tK%Zq{%RaAQfeRPwMZ%Z+t>DNBh(cthcjl^Z!QQf
z7;{!bI)}-(YSG+&_}M?4BT}J;TxES@f62cySddvlQ_uxLf#UWbaU$US&o}|Hazi_B
zH;5en=_iwS7ZcyRY;t{7FYwP_JyNPalP?M2#b*a2H?0)|Wo!#)<cqB_OP@yrnq37*
z|KT$ZuFq7pDR4gR(rN%)1%CeykNpYvehdbROkMu%^dDLOPy89V{}pH@#Q%VvtLX`t
z{s-#(5%~WEoU1_Tze2$OH&dp3m`P1M7=EzfkoFHr{ISqe>J0~Z`wP}@eGwf1oFw@J
z9DhJ=lH6^*>EEK>a}5Kxj!4L72d08mHvu60(}^$`wa%M7;nmQak6HT*vMyru)V$73
zTS4$`_uP|pgWz>kp9>8FfB9?Tr)AvA7^xaoVs*QIG5y1^_wco#>N~|juzAKR@c#p@
z7u~7%1=t5tS5+mx>DP#oUwe$a1@uBav#+ijYxk?=-%}sK34nUeST$hSD@RiP3>S%?
z-Z*K1K$L4<WLuicuQ>ifUj}^x$aN2d(KkUk@+;^+w<mxKw6qIKogES3E7ivzA=b##
zy=o6A5~fz?UG9hewVe2$vYCyCoUWZe^hokw_TPJ8M)F}3kmCai{zU@vrd4;x)?dvX
zQUd$~RQQkIuVT}G3TMTTQ~oKJ70_n+50R|8!#}05V!p=z5X7p5EBr&Dt4{Fuy|`*O
zeLbduczr}VoG$M&(?gjGYbjXJ^_?HIMGlcTIVwbL=B?H*tQ#lrSBbBg9;%e<a~kc$
zDbC&2FJa$i_4k~^a?VPJx631kYOpSR6zW62yB)Y{2kSniU;T@oo|m&#FSO__{<&IR
zZsfyF|J-GDlMAnh;<8*o3*#8@IJ4c!4kVLuz3D5-_4Bpe$%ms4o)`4AAeMF^*Ibno
zKK1vVzhISRJ+U*PZpl9!z4wlRarjeJ;Cz+uKZS<LGHXW3;TsBWK_;z|OT_%uHp?=A
zAot*3l;fk3m4baZL~-;i)~P8Gd;0J)^t_<JmmfM5hAm7{&psN2gnko(J`Wkoi<215
zMImPwN*{#`BqT;{6s`LFT#$%ZXrMK*O`hRKYpJo%r68ZF8VfWOe?Y+^^xZ(TxVgPk
zIHRt-2<}ctzTVSc&E8b)96@`y2JnPxZr_(3ZfRi%r->6r0|`y2#v+Et-Lb+Rutlf!
z(dv21;Gjv}j#JCWl_BlEiqVU#2|Y7dL(AuG57apOV7q92-N3y=da*Vz&%Ucfzg#0E
z<_qBxO>O%z%l7ZE4w{DY+Kd}VL8~@lm}Mo~T=D@&rfTr&>D!O{Z?}wujB-A1Uu2z&
zWRORuqHzuS_n@(eBx9W!XFbA5)vXZA+;e62R#?68M)RHkNrQ;HFI5Z57Qf1nJy31#
z=`krXp_7%<mEjdi;TOM_nL5vuf0zkrarZ^QoMj6=cj2TQIxfP*W5nJ&?~GhA{~LM3
zjdPD-H{72X7<Z~aLa=(<TQ6o#iR~)SxI~|i+2KlA#Q(M)h&Ha$w&J8AyK#0xlbKIi
zutHBWZ%1+ycW{<wI8*r9f@RhIU3|p)l0i;*v$JHmjB0N*4eVEkM`&Gua<8>*-jAT|
zcBhonEBoZXg(hr5=5h~#MLGMCIWwGRo~RN4g)fx4-R5&=k0k`vI>+iQ0M<;Iwu1>B
zJ@C{0V%19w;_KNH<&v#?1K!VS!dp`CoU7RkYuoDj*>^|mzyg@$oWTbQ-+`Dp=PK^^
z@R6-~CS9%!e>5eNspxoine*D{Mxg8)>pDXomF;_TZ;TlDCAy&?A0m4>n_EQ5>Jxp#
zAojt`c7)vX1KK)LHCsqHe<gQQDb;%8r)Qr!J~dpE@u%Y4WlvLsD6quAB<(6zC}QDG
zWkI_~n*H5k`shQDRRN;?_^QzKF7FSs)_*D2u5|%IExq(*sNreG1)H7KKqt{C!+xU1
zIR_0Jvl0viG{mjPq_N5vHV9Vwa`;8T(mof841B$ydy53${s^;HOWL*~p)bRIm!sET
zwa5<*egNvi<!sr~237ybtg7JPp23F6J+8NyI#{zCKs3%s#+lSCH?n;19IZh=Z+Q9E
zJxLK$h_r3#8%sQHd(^(}S&$}Tvn+0TSZg<iRI3<WQBSCqvWXGlAkr6aws>XNwNkov
zc(Ya9O&RF`y>$O~m!I?2i{<;&myV9~XxRFA&V|Z+*>Y2pguk-tMn`2X8Q?M>an?kt
z>J(OJ*Gy||!ps-;6|`d#nwC-i+p6f4ScCXNe@=uW<;^s)ON(A8oeS(v@h6;J%}+;b
z>m>~)`thmWG@z1%>8aIK9bW6RN0+Q@^%Y1>BSzDpyPL-SQ>iRpF&n$b-z~-ayNG3l
zQ-TR?(+~W%keY)bOoixcr2Pwy?qQSYckZcS=8};1g)+>s*5erKU-PQ#wE)%V9&E$d
z+#nV`)TaS^;J$tr{<vg79^%yxUAlZJWP-<0>1jNP;8V<pJ&xb<yGKoGBYe80Byos(
zJ*AWVN%qursJ903i~{xCt*LGla+i8HbC)dQK~2#g{Mu?lG<;?fM(Jv5DewktZ%8Hj
zNf~kGZJl35kLAJxNs7GYX3H1b7x5=nO$h0;&D--t-qeX2QKQ6IZ9$w_M)@r<&CRmx
z6L!L-K6CalVpy;@meePSeVIcGoa7D;!0Vi2>tyS0EARNp;H~E8ox5XFCW1!4XkaR=
zIb4mN@AvC7glNIVF95{wp14)~i$rkL;qZ@I;^D)Is-;)P<|Y?rnjKK`MDT#f*6@hN
z3a1#cD=`nPv$}vO54BfDgr{h^p};H*txQ|GG~k6QYAGYu9XE9xbN=swMI;Kkkb&?r
z*oYftQe(^fyxQvh`XVGuvP7-dV`6gVemd>tv@RTBBpe|mtC+L1WE-|s^1#4bVzQ#d
zkW%zvsM@W>W9||L)>+((SS0lAlXpj?1L)r6m<x3mW2dlPLB6GbaFZ+P=|#ukXf?t~
zxmNrI%oe0BJomyHpA)*HKkdfz<OO~NCBZYvApYrqzRpG8XHNXZGVP20k_Pwup3yKh
zLb7FEC|j86tt#}y5}Y#Dt^68EtG%tf^=|_0`ywO@m$A%j5KCn%Iy-KJK1{49&GoOE
z%P*DrDmoK}O~r_X`%b_V>V;(Aq3%}^_7t|`L6{pk3qQ91D|iHO9$=3u(P}&P#~zOE
zahAoIaZoG9Sokxr{Ojl+W5hT<DP`Cq;R}K@&?3(AC>YF{y;0f`K>H8tNs19e$;TH@
z7@mte9DUfaDz-uYsNLs{YOAjG3)TBWoFhO5C$ujU=jXMn3^;@F(r(19DgyjyC1d_l
zu&-jr*pX@TYiYOyPESmVif4fQnU4tZXN+%00*=|?r&mMLKhdsXmZNN;X^k7;I!M^G
zsmpBY=j4r;JJZ^KtR-rjirSNHAO97!{#60}D{M_WL^fuwd1m4NVBOLGA21xytO07B
z|ILBs|JOzK?)}$n_-n25K?R`suj#Kl`t8d<p#Q1wZV64lMVy*FYGRIDlAI8|TI|a~
zoq4*2bCuAHc@0o&t7+wzVrAb>-1!>iG%mawM&MK1=QoO9p?qEY@16@gj{or-zi1nW
z>dB5ucnbS2VD_8p-;pXvOjvgp?Liw4SBBy5IhdO=Mx0L7F+XmDLFh$p8vSP#b5mAm
z2l6Snc~--j%CZuC=@dQpk3!4!YDA5pyZE$9+DzG%h-X9iZemsX-Xj-y%CdSk^l{v>
zR_UmYv;VdE6Q-2m(l3<UupEA)1b+@PO7U4;-W**HC=reO&Gfsjf=+C;s2FTC`DA_h
zNoI#*|ACq{QP49^zg0KWF9t8c2ll_7M)L@Y@LcEPT`bP4_pMZZYpLR1SgLh{fH>e9
zNxCe1l=(KG6LiqMc%Rpa$X(YmU$mO+*o9MhjpF)d85rjdlTD~x(mm^Am)|m`KfQtQ
zG1=Bxt1knT`<kCGTe-}a*;Q6yvXs3vB9$dAo>~R(XcnJ;{<fp`_TrqX+DtT6)jfU~
zK>VA;QF<ngyys$SOjaxGcKUR(1!YZ#{c?~!Nc<zfcmH4nT-%pS0w>Hx2W2EJ1}I7g
zR5moXe^2AeK7h8MHEh-$zh3iN(loXoiNczc-GLLSYM;$zUi{KK>T%=E9aDqTx-3uj
zPCXR{=OxozEaw}X+R>j_n%8AtOKUS(<+%O#^2Kh#w*E40;!?xjiK*E0$fpeX3Fimj
z<T=-0J}whW@m{itxSKbw$*cMNCJzfO`b_WE4eqqpM*MBjiSUQbb~}XoR1V+^_N%ND
zZd$&lLM+A2xsNdxh~*B*?f{Lub)6fJ{?akS*z44zaak{DTk>bp7?}3&!OG3zY+M^M
zY1l3yQ5^1fWTs^UUvu9fEV#}Yvi<?YoxxnO0)P)$?WIL3kIFA*hU}k~|DM@(xsiT+
zKy*6{N6(v{EpIjwhByhE=*qe9Qz+Bq+`g(}1lH3>W)bM8{n^EP@nFu6aV-sab^3Uw
z97nS+;}g6-v&;E_p)Ua%G*Ny%i$yKgJY2(d{oAa&;%V${ql4SBZredHSr2BUO}eA;
zo=?$iCBnJJ{#-{c%4VWp>jYqP!^N0(aPa1s6y0qU=?_17ryor}{D@M5i1i>;%z!7`
z2X;|>h+BLnr^d2PRFBzw-ys(AksGb2au|@JV(I82CeXQGj6$n*hAmKwf4k^^=QO3k
zBA>kTMNd6`cPb!`d*51RQ-@ET<jKkoy&YrlGNk#vQm{lRP^`F%o#`JWjHNwT_HxfP
z6a`>l$~LZfv6xRTJ;2i!EEOqkl{6R-{cZhumk@|DtTjpM5;Z#IxF)N|-A=c9@2#BZ
zdmrxI%DF%h<*$8lH-ewy8n%pW@?n!dt_cuxu`*rp%6q1s+jTc7<Q}_~Lyzx6EXshA
zhB$HSI=7*6HpYm#MtnP2w?Zkr|B%7`fX))LQ~_9=P=`Ak&&_fTZ+gxE+4#IYXmB&S
zbfwUD^?ftx2T&v#S*7zDG?%v8-J<taMgw(NtSk`Vdjd{eMXkV=FJ_g5CDys(?vTd^
zY2bsPNaQPTuN6`uBkLH40uVDJSG~9&uy)5Q->f~f!~^=yOmSevtFPYfr2(AIQ4K4)
zAHFHo^Q83Qwgu5mpvSr2GaXh4jFGGFc`GGf{X+-T9OJ%O`OGO3JPPR7a-+#zE40T^
z4R;oHbyHpJEI;(+pW9gj8k27<ez5xLu4jbR{S<$uMT8&f^wK$>IxVvebPD{7Jbvt_
zjyFO%^3ZnhMk*j+5Qx3$f4L|+CVy@F{P!(A?o~;c)&R~=nEj@)3UI#sd+9}9<s4#4
z0d(MMa`;b5=un!{U`heq^VOX9pLCho!i~kRwB7%kNt}B2oYoNx^y^ns&5C*bJ8@o<
zSlGiB0ooI*m010M(8`Aaq#+Gx4g9F3{)2KKU;N@XrVaG(R{_+2vJtgJ&KAT*pz;6T
zg>{{fHw~N^_`ff#^xE)+sp@RMkk@-`!Hg*$XBw(AtYC5R(eJM|C3>Tqf`Bfuf8|C>
z+s7>szzxC$A&40a#n*9f@R#d};ubpBuYdpgvfJXu@jnc5S<W-;(kteQYy3?s>28DD
zC?E9(({RVkKh9SS0=;-qdeu3KNuCNlMm6HwKwg}*1NX8n#$w`@0Y1b!9hT9-W~|UO
zqIsV9nh!BzZ1%_;Wu+zqTnm||)^zj|d&RQ<rV7){*HW@Dxw8+7G1;qJigW1j{qo$3
zR@T|1w!6dz8h5JuSxJkZ;p$$t8;HmdaUJB=Z`ezlSKVn@E>G+!_g{uBxt7-Yo4RIM
z)^D{6T|Md(?(F5F=M4Wov&Z)!JxW%OzOq^AfP2QBl8wgW+afq6Kurfk2*1=~NjbCJ
zX_-_xXn3DzN%rsT@e=-AH{pinEe<3LAGIVn+JeKgyU{9h?S$p_L@1{Y>iFe)tJF6S
zxPL{sxq9gB&a3`~Re1J9%dytIoxk0vopYlSwsR8fsi%t>LvQUlSq~Z#d{Ea`%);=o
zm;sb_83SG|ApqCc*c6k(80Ewr9QHm*j2Jl~1Anv2W10U^XYM_BmX^pEWrJL=4j`?%
zgO6H#YqXD((8kZ+M=Rl%hgDTq&H%(Tj;Q1;G>$4G=j~AN>l_A_EODA$74D146U(3Q
zjZ2n4OJ6SPVWlY-_@dz1dlhJJRET@B7@|K+WxK%`J?lLy?&?2e8<Gd|UlBxw#Hqkj
zRhomhPpI6b>6*>L%UcBk{LfpGgOB=sYaDwgsV^d>RlB&9%U>Gyt}+tZPnbPIc)HhY
zVsPN5aW0%l!fZE~kf!rDb4&?L({}zeJKfEZN8n=lL7E9|x|){wY&!1{&=f2$0#b$U
zy3Ml8c}Ov=ZlweQVk_<7b^{EcoNaYVkva@d3r_KKklWo{eyq-jecS6p+YOIugdfDy
zew>+cUflmSN>uOp^JzRs0?c^^4sZ5G7nSXMI)yW>X){YdX4D-}Cbw>eIJi(kf1%BL
zXhE58p1gj&&&=1dugb2Qd%s8S2*#P+1DxSzarkBBRSrB<H?9kpcHjy{=RRS(x1%x;
zmZt8(iF^#Arc5j~3`w5zpki7PYQClv6Fmg?8hxrgZxQpy>erP@N+hB#ovTUO+L}9Q
zlPy1QAW178J`vQ~vw@^nsT^q^H1(z4QJRx{$U%75+{ctFq}Y^KOEjP;t0hjREt{H5
zT)&(HWCe%E*Y*H~C~#mPK%X~=T25{wyP7(Q&CvU;oi`F~(xQuV>Bxju5p{*J<8M4O
z3$5>j4g^Y})hfw7P`mOsS226{*m;I$y}6#F3r$X!Nj}L)#<Y^lGbeo&sph7DqfzML
zJS-z|rk+&WuRa<=!(c-zJ)`i*=V<bWrA!tvB|>|J1X*#IoAAxDFz9*a>8c5x>#a|`
z<BCk>TS?1!AH~(}EPb;w*BqYo&>7stpXenbSpgQ>j9Qc|kvn=Y`^3lJ2CuM}6r;Dl
zYe^MNyFNQ44-UN)%B4eT3g~buT>_0@cuRtLgJAq(jSQ!zCDDbMKji)U-Xrlnj5nE(
z?qTyzT*CG4p_fSC(mjP(PgCl_7W9$sGu{N<;`UQ`-=;Uo4P}_l2kzo0C?o1=lHLRw
zSY4$&44K<@H+LQ6o2YZ&kNpnm@Uz(YDCH1hc)$3AUAgke>{9hzG*(%DOm39gJnDni
zXk$KFMlTj)MiW7qH8jSK6w?1x>_LdC*&D)6Yu(R;$B)7$O56EwvRerqy8I=gXtG82
zB2V|KO|t$83M#V;lqF{Hc4RaTITj~Dn^mT5b0KP6qzD93*X2zFBXvI2E~~pgDw|(S
zFzGC8YInawODgVyd3YT!V0i8(D|kntd?!3yDDI9eeWtR7{TX!IdcgW(#2n43Qj*Bj
z>@ddICzkb%7lEObrG`*&gz%)t%=ON}@2TmHh4aqp^`2QuD+Iwxq%q0TzlqW}Y5E#6
zBXsdgD6X+%7k|z17@^VP8bry8i~B;hR1ZEvvd4oT1@(E_O@FDMBM$MwiuIN`9C>#m
zGU3}2q!%iga_24o86))bvz+;f5?zL;>OSghSBW0Lgoj>8--}wF5o$lHH+%Q<()T!U
zW6v`W;((zHyJKLsT@||g*^m2qU<W$HX!c~!D2Kr3Nb(s#G-%;*ImscqWGuy}Ox;~*
zkEk`*hFrz9iAhn9;=_exA>x(W9nG_@-<fsKDpOM~FUw}l;Wn9duj_adVMCwN)-2Pi
z$}*;SuI8faJdJrHF!)JB=WFZiI6MP3D$I%_KWAVdFc<HM4zegV*os5Iw5@PHf8;3}
zc%v52Jn8w|q9m@^eEMY5i~P(fjrIf5Gx5`@n_jBEZRRF*MN(#O19rVRbXg&l51D18
zwi-(-FFm+PnrFo4Gxi<4-&!?WsX{wrY{Y~^0^IIRU%UG~GmHu+!*98G@JIZ3S7*{u
zOX-PVZYS6o50RVSU9p6p=HQ1Zm-OaKEtGjv(>&iAKN|jk3(<<E_{=1811ZSakB!h_
zjUAW1wtMhNS6G?c8F*hsdwCyTWjRAJhdDk=?@MLVpFV}5GjFr9_u6ocLc~3_R|A#o
zM}9Gl6Hx!5{zCT~cwZH>@iyi2a9yzog8My)a@>k?OwK#)&&Us80<VdKHWi(%*(Fs6
z$Tg$g@W4U<BalRIM6k&yE;)Bzmo}$6-B%yrF*|3F!(*19jHQD!huTup4kf2Yj0!G6
zH<c#jr0da)j3Kqv^f?Q061kxm0M5Xn#9}Kwz&TX)fMbSAyk#OkVx|?f+?q&k965Qm
ziHHBXUez8MKY`*;^AXoucGxNDu}R4)Lh=sAK#XG=7*iMXKb&mxnmvCKZ|rIH$hfkk
zU8_+gZMlp=&-VrQO<Pi2-5a{*CXglmkR)}_ZPrGA8Ut22x@+EsC@n*@SUw$|y*TGj
z<%$)cWt^Xra=mdMRpZ<kq(1`oieJ-%*3g}1=E_{2_l+HY@!}Ey|7)wS<A$K7$nW1o
zQHw`%zDLV9`i*|fURb=b*qOts1*g*ZGN~TMOihb>IyV4!|3<b}_6cnB0CReWrKH*2
zF(~F)RBgz7IaJCem_JY8krQ*-rMV7`bJ^2QP7yOLi3?5Bh{Sf42A?HNrUi=K+3sBn
zT_~0`u<qWO|6yH$*KEfXOA;11veZZ#4y9MU_Rf4ere#TZHe>C@*MJyv%#t!6G<gR8
zsdvj5G4I#785ogt)+TA##}SIX;>hlYcc%NbuXN6eyB8s!n6GMmM`1uOBjD9&n11FS
zpqApBJ++!e9|Z4PA{<1A^t_g4^~vje+cZt94;ZEHIWI_RuvIE0yxHcKz|WN7w}9bx
zAlV0tN&6ua9}9l_c5;52&zp8UOY6_~OC;TVE3Y?`aOH*}LH7(g&JQ`4mA*bDxQXtZ
z=-d#b+|TJ>P(dyPqk@-0_XC#{M!)?2dJC^EVp~i~*Ukb)bjd^}?TS*z%9hu36v3RO
z9b4qYZBR~65e#gBr@<ADS$Y=EyMQ?KHb<LPSfczBKr}DWFk+urqD3~zL=xJp>I#Wm
z&cQLVaIhmU`yQ`!U4X)pV3O&Q@tLs>5|aWq51JCZm<%9{Y^4bg)FTB_g8?2o`#Lq<
z1bKQ|EObCTH>6FW<k2j0DR><FMn~nHNqJ#^fkpfq6ebEWVmL9kPmr`oOeT$ayeri6
zD^>u6Fp}jrb#ISf<Mo>0>HcBLW#EwCs?J-cmp@J`3CR#<_7G~8vB>zc{!RIQ*(C4r
zQN4xwVTv1*Jx*>KHxd(1KH&3}qE_`JQz!7Db%mo~%`vm|P2rt|`#)dNF6%iy(O2p&
zPvELQ@Cq}xp?sE-WOF+`vP?f=99ALbBR&Li%ns<9VC>pRXLlf)`sEtc$8f8>{%RiR
z6iRD)9H{JMA6KauJvQAZy#2N=+V^Ob8ejohi)2Ey*Nws6)X{S;2LUA}9s)mSV<pbz
z>?gH8;8l0|xeVt%j}VtV8G5gMlS-?&W!l-Q*<@|G#^U|}qHblt0yS853P-PMsl@i<
zp7YrB^z}(AV9as(rOfinfweM=F9rYr!>?He&WE+%<~h+s=CCLZHc=HhlbA^HBOtC%
zH!ZNqvu)cAmVYyx_Vo-`Mz0D&?ap;;uTZ-1dkSxTS72|FqK&&U@y&JiP7VrTPTg7I
zTTeGn@_BV~1d3?*#QK{onz2m`9YanFa4<Q^%PvW;Y>fWlv$LHF&i8Nb@G&t@HK<v>
zmhYVIRSEA>A-)86a7Xf^P|DZFpq=yPqRPM@GZ=yze{>kmHO-MD(Y(K%me15$<ezD~
z9y7NkT5U+iNb^C8#1i{(YoKumL^p8kJY8q;YgFP!=@8e8+ezMeE&=mVR$x|5vNCVT
zf_HhB^7d;2r)dTSE!&hc$H*i7!8AnsC<ktlu*F{)wxPi$ElwrvR`Ic<GKU<$fX?W$
zc6UfR<B8=0*~vntg!6SM&B8?w@2O;?p)js>X=czELg7B!RPiAb;3}~15I`nuS!qcz
zO6d!4B!ve8w9d-dFy~(fLJN`oaOe7qSttL74sLi)MB+R05-i3R7hs$Bu-4>1{IovM
zst50ABoA`v=pa0zlSvbf7PVjVg5$5v7=p!5)3ge5iFtk+9sA=Y4hH6b|9&%cr>yr7
zO7`-d2<O&lvNiWn5@u;OozS@WDIH(Cx0X(oWy7C`QMu&=cKLAJlZnyDvC;T3j~k0?
z;`Ll|ONa~foNeG!z9%#<{MbE!k8s0FGiSYC?GRn2)V2Yt-%gk+uDP2#Sr^)Phhynp
zixGbiLaV(vT|5I*1<KK^N24e!==+O;7x0YIfI3fnKAUyH5X#?(hasVhp(FO_tr+Bo
zUjpIfWubH3?v)PXRyKkpNfmBjM7YyLi|o{U1T=o+9gC)@HAXgw;thY1JY74MC$r{>
zUkvc~M<|z=3<dU1<(p#%Pn?w9sR%U{l--m0J+sWuVtku&{Iv^))50W!I0+qJ+<5UY
z1<E{enz{R-V&vEya|ubc>I^y$E0Ys-mfHb)^BbgnE{cf@lzOp{y|*`{%yapw&+X46
zqUY6l=OD)^Q=+*T*cUbAo)rgoshT4M%AQ`TH+3#!yVCT$#^`a~%Fh^emjv-rf2xyW
z5bs5itu3Iq(>KfZed|*SD3k1fY-LzTHGAjj3+)L@#8PGR{T`Bt1J%Uwjx}uu4zk0K
z+#CxKaGe79G&_g))!K%&JN>`DxG_wX-OUviYm%ea9aD+CW}OR9hF}?*#87iOJ+yl$
zuf1^pS=r{E!GQ|x`nHprUfS1dXKb@>La5E<4Eyt)SNCJumN6?Vj1@bF@4nr@Y(lgy
zvp6Amb?IlH!r7w>)K<jOU+CI-vuyE1O9$bx{ivs6bVk{gBzR(%dIGETDZ$I@`s*n%
z(#<$z|4qJ+2ufZMIJCdG5unzN9Pzq#*Kp$<Rxhwx6!&5o$c<gG=hVMYI=RoPY#>aa
z<ZezB+V8;po=dtkon0ch1{4=V?t<IjFie!%tsK$Zk_?oZ?5aJd!v9J5<(}9qI}89L
zlbW0HJB*ZACtQ7*m2Q$IEaM<kW<VeaAR9T~jgoN#ap~SvQ0AR~P>#rthokQS(-&^J
zC6m$Myp87f<IwcKr>T}7T@X|t0vzT4h-D-0ib<LM7gprPzwlna4EX<;iTaNe?Ej(d
zR*B}nv1$Js2}|VMnD!s7;r~QnKkk(V*xv9%K$Gvxe<Ra7CNH<0+{QS;Ip|@sV(R~6
zUOf}4AzSG5gB1O@)ID%7rvjxbAHDsYXQ{fFK`e3mX;{wRS#jszU#{oPKN7Rtw{io>
zcC|y%`5xfgo`rN3NZ((UdUyut=gn>9?~AtHxY{yUA1QmD`RJ-Nf3a0Hhuu{ohdQec
zcHgc`-dd-<WD^&iQm}TFvtI)mp_fCs6UU)T@9ZS7{bBThP=FH{cz=TE6tVbP#wB8@
z9V$ZD=nq`wY9luuV++U*jX51$Z7SFafaZ#fM3h4!#!aCPja-HwR6<`3IONeT87CsO
zxkYTB{OWD8A4hs4LuZXr09<(N%NDn#jyS#3$JRxCU%el;dg%p}A&=00Zz!r;8}jwW
zMhQumkZ)U!#6GmUu%5Oc7N}UIdt>ejvHK){9dW6qL0>xJ`^y#vmWB_R3djw&mOUM?
zQa_1E>pWM=I4`KL^6TN`xq{%3v25wE*48TLhE!m5(d8GSN>k4VlZG8HKR>dresSfd
z+s`w{?-L1ohE|B=2-*Sy`c%-ZesLgC5;!6V4vyKk=htO~dWLJn4^KWLy5JG94F)NR
zf88(xy@&{G+SB~XS@!LZD|0_iFuUs2#^<#ND|^+}|C?V**M{U72`gXy0wBsnjizx(
zq70WL<GgZL&3XB|DF6FF3xF0^_4M@nj34g<;UZ8+qAxCUfC5qy%%1}jLMW7e%;I|w
zy=Tj1&QqK9p+7I@4IcUtsDq3_;v4pXB11a{V(`jm2qFtr4O0$I!Blvn1#&hmQ4}#|
zn7BaTa}xn_RW}iUt};#%2FR&Vr0+}?VJRz-#%ZECE|2H(#s|=RHd+$<#lL;v#f`P?
zL`l-k&DAPDt`Yvy>j#0}M>Q#*nV#rCH*;S};;R=ITLGHuHF)`T3&4u_DXY{ErM8H2
zJ4BU*WC`De2-P9e;j{*&=m=(kjB^BUE*JVGuoIFPKHZcK{Jk}epA-Cf6x&sLz~C)#
zXG}5|uyzR<-uJll^~tMsWjbMbxZU8bWB=@f7C%1APnGvol;?WA+s010s(vK1<%0&T
zF~W39j2LtHpZ6h;irNBKf!<8g2&n6WcQjOl_M&DaQ8lZq&I+N^2oU2g5(X<w^l<im
zHfAT+CuOB2ah+a?RiW)RlwF7Tm&=pyqa;+ACk$#+^44!@Ix-Zy$JZXoi$mJR<JXVO
zqfy+aVzVqP0TC+ycC~<Zyarrs!z$(U#1&Zz>dXA&lI5GAK|W%)?Bz(%F=eM~0)|4n
zMD-TGKHcd*njNeH<_t~)q>4l?z*R_sm(sxXO3_Y%B%^lXdJ>j6mc0)^+Y7Ttf*Dbk
z3O65%{*dUOH$UA?AzuY$xoZpUfg3Es^#H5<fVQu|FKuZ>|1B1LriH4IEPt4=oDL8;
z%q%y6n`N4d0cddU@Xg1n|6wp|l0od5#N9z2Z(&#}Mk%6k9OCC7VGzXVtBauA?my-h
zeA9kww1FV|A2&yjVY=2OiG2c}5^p+jT4=AsSO-z49;mJiWrqe5RbBcI<>w0J%T#&g
z3auIX`gbk5`IK(jD_e+)lne8SLu6O=FXaVCJ__y+1#IF_y3^C;NinXESd(`&w)tng
z`s?=VA2LwO6T4Zn8XW$*f3nlj>;Ee4%LAcK|9~f5qRo`ht?8f?I>ib#6mpd_p{dlS
zqS=Zj=cuTVBXUGchiDMF@0n^7RyoQorj;VsxHZi2e!epdO26Iref?E@zt8u1KKK0y
zXbOIle4rhN?hDDxJ7q3df%k<6AO5cgt1~GM=7QNhuY@;x9NY3EHbcT+EX0n?wShKZ
z>)xrovl9@&31)XB=5FwKu;dp&-cj-DhPYxJ)c*};FUD?}^{dB{u|+Sh&xML$#P=)_
zxFzp_5IVzIXk1vG9B(U`{PBuwnwEkUM5uN_bGteAC^$otM5-Cm`F2}&=hd|y10v;s
z!Z*!4JSZXAd-*Lqg*j*S&miT>TPRwcfTkZI&)Qs&QGjFa6T&RH%@?Nu$xo#I^ax<j
z2#*%RNZ?$0Q_vTi99s_wzC1S3STfBXkgBVbd@=x#h;u>0)l2;ScnM>6HC_39$TJ|f
zJI4nrn9|QzPTTm@x5<O+&*su8P=E^pO9zYLa<Djd6{xKOP+uCDS#=X?Wa3zsg4y3<
zBK%}LK03BWF2)8#PIIb*f-evM{~ttK3@evE@ctC0WoMEjh_(4|xXs##<|(Q*6jj7(
zaZq}Oo&iX_&0J7v_~Ru$DRR3O-uz63h=%3^At(l0G+2W(z)aTP{J)CI+PZw%gfkb%
zc4xOnA;Q(g7}MgX-2DK3t#gcE3U%ZHJMVOcbOxZHSAhS$6P^L;Hy3zfi@t>7+~6fh
z^~R#FC5*i^(TZcuM<s-?fEE0(f^(o81Zj7vlm7>MUFM%WK6B3VMBgr$+N@r{2f_xA
zW1D})f`Kv~opq_FfeinVWFO~~;Y4@LWLh+{a;O4YELCa+6!K_sqCapNg5<Z{PY5SI
zRZb&)fHX&kx|AQGQy`><7(4nvWIret{)vCv<na-d<YMeW9^?GH<N6-Y0E_e4r!EXu
znE@Mr8UyfnKO=v?a{zWPe*rr?IOvjJy|~{}kO8SJP&avP`{beO>gBIQqHh+u4|40r
zL3YA|2Y<ETf8EJ=8_+mYrychUm^pXB3KPNXq55+YnT?m1d!kRaj?oIgHHc483Tf!R
zC-SamfVT&$ErL7)M!R$Ij3us{1Fmz~wdgBi{pY{v47$I5hVBw~KB#VpVbf|CN?>7Z
z>uF%_h@Y9e9_WATG=c#}4lm&?&h0H4r>bcl-&OFDX?<F@`$}O%2uF{ldip@3kM6EL
z%gNXxrPlkYpxa#stALCIrwX^2kcgwb#Lbk8oU31kTo~QTp&WgcM7!=Q9aM|mSKdiW
zh*%D)`Hxt2JKX8|t$YH~<MJYl{3V0r`sM67_RB7bJOxHpMs|ml8r8Bi6i*{m(k~|Y
z*y7z`Uh=W|Tvp<X5WTG;jV_HBod>9{^!HCUdDvV`<UZ+w;ab&5ca5ePcEJk8J(yYM
zTw>-cf?ZV~M<kDiFdb`lh+%hRSMSS!V}g@nCLy9Sa1aDrh{btOv^;Ub?bAXK5#e0F
z7%O;2s2q^dP35ppVNTMU@{6%W57nW3d9ullkjK|mLYQ@$?h3G-p$5DGh^zeJqoDcz
zo+2!>7FCF>tP)w)I83LgF-GHzRp=qR*rXCt2a#1axPA9P>oShxs*%CWVI1RSK_h3-
zETp7<I7h;>u+g~=zj@f#ZPRU*u60Mk5i_fSXFHA@Q|g&Zx7g324m6yrK9hMqRW&y8
znUijJ{(wpezAn3!#W};ESALFrH0s7Fz+1_E{)9W4Zt2J7L{i<{`g&FE{F8|KWYh%r
z@lU^Lq3oQe@>ZP8Vp{B@pbMe~Zynyp4^Y{yzt2-Bxpq^7KG0*o=se+SWks=LV~$}(
zah_}A`M=Gm_BPqwm6tBX>g3AiygNYE&*~mdRwr_@-w3>Zr*_%Y#z?G@aKutu1@~d-
zdrXwF{dA}fzSR_bY#?cvT=+V<*-Jm=GtES=kELNwU-n$P@1+5=xUW>inYj|rB9s(U
z-cZE$`z#*X?kl!AuKLSkf^_ZY+v2X>qn~v`D>%aga)Pq+qeFr>+10O%O++{QXy%j+
zB<J9s^*rWOxvo+(5S7?$i|g20Jyf@Ki_=@9(b%rum20_7fVhf48{i*_fI>r0rN&@y
zeN&UT=L<Dyst<!s1<Tww=I&7XYtHSM%=5LgaMD#|n%T3GNBhu?QjZ(%g}?47vz6Rn
zMIt(DcJD}64$i^eqBv+35YpO`-*?qyW=Cge67jQzRaWKrRel~gePqYBBfGsNGfR7A
z#UA*A@&f10N*t^D{iTSZ;LFuq!8U=b;GE2y*U4pFPn|aDRjs2L+%<{Fd^jNM(Pp6p
z?YA2nODnA}Mw1)K#voR1Jx=@niP&vqQkksmoMuz;wg13C*FMu(AIg|LREyT{AwrXw
z_T;6!mNmtziq97>KzbSxqsOwiHd55-v_|{zvdDuCjdOc7Y7KW54~w3%+F#Z3G1dU{
znwI_k$*1yk#pn9%^;W?!ji7zxpZE-7&SF_xPTST>S6ek4_cQyAFehmgSHAI5G->7m
z!hoBWx<>er!w;V8Rj<;skC$S&Jxai*_E6&wkg&H%&tsbw7zLk7K0SE(j-02r9Qho<
zq&{!wel^-Uwv|c3ZzCTo>G93Ny7e(V{X6~eK;*Bv+FfojR*uT-dkp66m2tmfyaN|e
z<yr5OMHW=8!7`P7#RS_aD$!>H>C%7OQB|T1FdD*h;?7#MzaJPPQIPMoLlcvlFPri%
z5_36WaKY-)`i6{(vK5CYJt<to{RXx*^lJrr1~_+T-vk?Q8aE`-KB4CUv3D5nTwc3!
zEt$sPQc`wT<_Bmbs@)ZnuzlljL->d*tuj4V#sS*rou717pU`h?FtX(g{3E=tV+GA{
zM!gLff-e2<CSW<<r0;`hfG4Rg7DerL;=MOL6&htvki%O>y;7P?J`v4=p&a1I)2$sY
zm%N$|9Zzc{7Y=weUNAJN)Pm_5k1CQ@4dDp!h1=4TSnP_=aV3`0PN%gFe_d9XX*`K>
zCx-MOwwLdn%2s!qtuJ?o%>e;g@AmYxpE9;OTY0Z^UrcwRw}Un9kH-Zbm2{(INJQVP
zJDZlD<I>=kudFd_&CF6Al@4~`V(S+M!Sc1g->EK6O~K!&D9$_^?r{#j5$?S)L>1~^
z#qZ~oZ6|w+T_5Q7uB4B?$%qFEbCA$JYhhH8t7^}X8N4jjpw=6y*6~N=XA2oFm!CV@
z9}ZkjAPy7wT>is(#5lg2GTApnWjVNSoY+<L)Qz5K*4IVprE{a|8{BhTgmpq|PY2jD
z-iVL7zn7C_xBDC+utyl0%u>H(qO(pzo5i2a_-0TrWF0YV(?HBFh)9H5UIbHe{;QwO
zp{(Rb1d-ZV_1*$@^rN+RvA2lI2BDe6!*{UXJk$omC>3k2oiVcPD}3BxVV)NDc;MMi
zc56;k+!}2wdun@s+O5z2Ydz7rE2p=$vbI|9oAcqa00oZVn;)s6Ci3`Qt7)5cXX^I2
zkb9ekGQ?Q;53K>Ujj~7FwNw<`8hrepNe^}@u}ewc7x){)6t(1oN_=KYqDFzKJZAok
zPklm6(2rrgc&~4LLxFM4ua|hDWtY`vKDD{}xI5l{kYGCNjKz{RQJ6L}G6a1A>J7&V
zgv4l74<Q+zwkL2T@_dlC*fo1jTZRk^-?(y1PD0ONrB!)5=QO+#!iend*<*)kc{x`y
z8hy=n8tZPk@(_mXFxx{h0%n)kTz%Q?eem~tHOl+a{(Mk@y{^|2>~_nBj8~}`bk3nZ
zvl<Bz-}H8aY)EwXoL<jGvWM$3>P~xa$bY=fEoz^?r|X|_?>RN6+9@?6;Z?5Z2v;|-
zIF${KV41Xok}p-J?~X%X{Ghi>wwp5A4J1^8V{Zd%aLjX<0*=~>T=6um@`38#NP3$*
z4%-{zNnVFeMdHK0EpOb9yXAEzB7Mspr^J1shtlnG@Zo+Wc9Vp0oXwplXU)L?bGX2B
z3XvcK{CC4N{<i|gitbllcy7jew<!GwgRS8io?`hC>XJ#4<@ZKht8(HV;akfK8&<+Z
zU&!S&w%X2L*(iDBZFxpp<ECY@x@|SzGwecP#@@k)WmUO*ZF3F%pLxrv(^jSCJ#G;t
zF6us5A!1}#U&xa+e9Lp1;KhXbIn_@%dFcc}IUyo0_efFCj0t1(hZPj0T}@m|=Pgy`
zvZe)Xq#EG*N?0DSy?4yyUvMC+{J{Z5c|_9OdFSP3x}Vh6>KYxQb3<gZ-b&SMmfyw=
zzrPuG&nAqVJqlA>_4&R$V!c7&@^?Tigy$eWc6qAS=LS>Mg_j@sq<E$06GYaK!0A(L
z5!73SQo*;hoT`_(p^N=-=tr~BV3qSGl})wEdw@BvyasWy%M;!5mFq`L2`)@y2XprT
zH;laD#igb`4sJM769S<qMl0zT9yMa%7f9gpHhYW_)DH$e5~NGp3lXD<Xgc2;Uw#3j
zL6zGj=h0Tv4&c$B3Z8-)e;$tWj~1BbZMg;Re;K;26MF#e<bMcxm{0X=B<8+2PF4*R
zDhE$+YS<%qs%LZowIZZ_?X(81&&*HHjU}~b+x!eW%N3}bPe!zO3SpFnnEEZ!Zq3rb
z>O8Y=b;Q&_H)8NNqZBt+X@ct2g7`>oCt@msjrL@MQZH*W_w5&60~sKM1hW)T3v>qW
zOjVn#u_i-ANB_L_sV+Z%|BuP&i>$#KqjBn-s;&4qc<AYy1d$UT2!Y6shfeacn0+NM
zTmdP#_8Fy9Nnn!n{u~$NBg+>;GY&OYn$w2|_*j{6^7!*PxR%s(M;>pPMfjJ$*F$E4
zj&&tR_wz$Q{1Jahq;O(8U<V_0#Nt?pjI02b^5A*VQ*538495vNm=ZpjAchNL=IIv2
zX(TQm{VCX@{NW}@EFkVO)Y4CcRCr$KR1wFAS%|y`q&Q?jp8wC)A=vSL8a?Ney74kN
zqaRP39d-QDE*+sRjf|Nv5{I?|;n1L9y$eHir{(o`_r5NGh^(1f-Ta6gYP}Xhei;zx
z-N8H0yS==*Gdx^HT*GP+7GwLkL^}DYi7|>?vbPly!ZaQdESBdIzqc?3Y8i?7n>}_9
zXHTa7sdE;QKw*~R)K5TuMS{}fc?zQec@6cc!bYw%<^lTsO3wf$S7^uzVK%&*>PRbQ
ze;l8NFxWB-y%jiq0|$eig+9T_NW0h5;Y{~e`Wm)2s<q~hVdkm^DUP!aR8qvLu(NYS
z+A}r{x^TV&iT~b_X~!MP!6byb);kajMvs0->m*P|8%u~C84Z`}SNLS`_OZ!hh2o!|
zo?>rBbvT^|4%BDwG#MK}gm^4IK}HR^zuR843$i;^#$wSptS^nT*n|VpT_z<N%4kDs
z=9d_u9T%MKlzMY=oBlaiv#Q~;Rdoc7`Sm2`@w@)?)dx)kzook21d~swhq}gv51BO#
zF<M8kw=&|-*12*;l*JK222}nSTL^MfDE<|N0<PcA#gEPULj<*)zP`fCXSq6ipGfs6
zWLMYXSk~m$bn{rfN353YGb!EA`fGEbFU{VfrY;IY!jH-fD$A!E_{vhd33m0ZBzi;D
zSH*-)T<rEiU=N=k3`SmTQji?9$&JN)vlO!#%HoCzpB8c`-{fXK@%N(rzK~iFbKqm5
zg6Kt~UCI7+mtPZpf1RMxJ5uOI5nGYd`QeI{!)SbW=_+~EqYj>-_s8LBle`wR38$kB
zLMqa(z7AtE|G45|>9wZpJK<V2Vv@E3IFmhxdSB=2TwODG*y6O&*61#$oHWe>LRLbm
zN{1@$N%!+_hJQXif~z5!RSeZxKeKXZ)m*^U8RDMkyHi!G<$cWImuuJwzBwl}Bb1jm
zV{hsCeI7K{)?;=pn5|cJv0ndEZiY1b`TJy2(#6gXaja7%8LURB3|nV8Fs&g3epfu-
z573TGKIHPm{C(0z>_l`#k*S#I2>3kuenz6iQ=$YbVk5oPSny1)LCr>wG&@pFx>d&k
zE*FBLCNktXU%-Zds{W9l(gSj(2W^|r1|5&I?@%Sn*u!wJGZqgH+O0ZQ;xGB(vWivx
z<K{K5YRk-eFT@+AQxetiAkAWAOavivcPYdZ?fMSfjgM2U5uferHn}n1%d-qqUVM6n
zOdIoaP~Dt<AXIsUVZH6ml%dxh%SIWdf7+xP#8;bSQgc=$(MJ;LLUxbAb3x?LfJ5Eb
zR8AxfA#qwu!LiD<Vh6(t%1UFjDxL&9^bGiZ!72*x?<+h%N@ag$W?b90>VDSN_=clU
z0Jb%QnMmE9(FqL=r;u#v%3i4a30ns9YWol4=6;4o{B!_s%cCUP6iQQ1&BcbRN}PZp
zn|-3A-w7EXuQc10_4?yl7u>v?(dF15LA$^CaGEPy_-N0yqh$uqXhI8|^5U9GjnfVE
zKCn|9qP?v%O5<R<*~>~u;GY>;);^hIx9^Bttk$YjcE)9shRMQL@iHDuM)TF?m=GSh
zKmmZo?r+)#zc(j`SEfWr4Vf5r^>&nN6XODohewf|miuRxj<`bU*dy;J-@2nRMoau0
z?^kXG^O!u+Zbr@NoLessct%3wkQcFULxXp)Fh6E8;h>IM3f_z0nk6L?*z;7@Pyyte
z{lS^%hTH>ue>89Wv06E#*E8xT>4l%O9|bdZ3t_&wEt+csd8Y%syi;9g;CG+f;u_F$
z?8`GZ#+N!EhCRw`mVoT5B&$txqsMCHG4(K(3!kFOGvGi_MnHf(#;3VzTK{4gOt9Yc
zD%JwB3cUHbQ@sVg4&Nz;g>*I8AOEUL5sL<LH)GKqFvg;syTQYD-Cch3e;Enzf%6o`
zAN)Y=^aji%bvPxT%{Kxjp<{B|gKx7<!o&FR;whHRGq7x=iG9+LJk^oQhHU3pws3ko
z2zgo$?*v1Y#&U80kGlrtU4tX4z%slj^)Hr5K=-|v33V-b{iP5!+tq0Qp}9Mkew1s*
zBh@*JvE{r3#V?Rp;2E&|eAZRaB4Y&+N*gA!YJ7vp6r8Tf$=4tAz|+BU>c~NEI+ss6
zhk)0ox+h%EX_JYdV~Ism3a@m5A1|BgmH|sNjZ|mK31gsc2*XeDO!2iQXo9Z?xSrD&
zVGNYL)Ii=Bij(f}V}4Uoo)$-H%LNK2lI4)N7^2gtOhCn0xxv&=LH~iQJ3pbtjr*eX
z7BIUg1u%Zis-%rGvsR7*Zl-@0jAf5OU%Njc`B8f8PEQFrH@e3qh=<R8ZpF9%xZB<%
zy6M_sl&!=R14|Iew?A(LRwv|rUJ1^mJiA$bsyFN|1w$dbLS#IUehYX4lF)oFfmiSp
z4inTeAU!ro_s&8J5o*u#nH~W;`T=n1v0Q{k@Og+YZAEC@HznsigS%3M7?xX_)kvU$
zv1xg+c>P24#lL>rwOJU63cWy@X@Low#%$%OGLg4gaS*UjISeEWaTIAl>gTb%-TGdM
zr7%Gq=7jGN6~er@4Ek=|<{llJ1NhS)#<?7&2b1R-d?&>0ID5~48V>>6OyJa%5;+U-
zSX72BLCTa)4fxIGL7I<~iDd1+eDN9I2{h#r`Ggb5J8TTk`vc6V41Y|`VtkE2lnMw*
zpm?5A;WMi+4+G|ymhy+{093}8P)SH|a@eDw3xt1geR$hC2Y3e}hqZYA_m6AwdGE(<
zfrt0xheR`qi(U)h{x891z;Q_N$b=H&i(B4@A%L2g(;f%n{r;s=e#&mbQT{h{WT8{=
zG*$2%@A8xae{4LDGAZbb&@CY)jLC$HQzlUeKeIcN`<xpD<l5tt7I+5aI}Zb2jH6)c
z_#Xi-1*ky^;7f=Fgc1&pOQTgVOaERL^t;Z-oXox0nOq2`Mw*p_sM71K!N~U3a=kGh
zQ+!YmbJFc08Y{#d48H4t;q`@BKOs*v_G_rMpVA*-Qgo$#2!wZx<d6v#sEGA2+wi$7
zMgQXMmW6?uIus*e%rXWiXdOqR@3+94+7wf;JYfE&7glhK@M0|T?>+FwA1{S4v<*<?
zK{4mu7Vu)|F<Rj5H7G%mgWs-_uD}*uSj4+N0O99j;t(XWSZ4yd^|q=wvPuRhqGRH4
zBSe88q~9asNw6yX`Z)_&BNei-$TN$PN8tbP;|@b%-wN=q&pilI#9R?@58ctUr=9^Y
z1pnJW75D<kgSBqRU&n6M^&a2#i<U{_@lcW66G*`3`?54JEz8#<2Yd0RetpF}ALyS(
zl`lm;OfiKSKqi=$ibb*>E7o$a%~g2s<|Xg3#c{@Sap3vQhv8i~Q-G!x@@;otF^Rol
zd8os}W9KDl>BFv3s2C}^Yy5XWz@bn<5|f_fzFX|$vs(8kD!~)kF-Uf3N@kpq^>Bgi
ztYM`-JOasMp9}V(rQ}i70P!$SkVvtSm+PyJ2s*wGiW*qNfM6hiuNN*E4w68fgYELb
zwZ-wmLXTo$_jLI19}mgs6m>4bY}2v?U$xABaRk1eiUv^pC5B}`V=9AN^%*v(aKLib
z@jiiL5NdEYhe{26AQU@s4)b13nDlD7v7qB^5c~jmn@Jx6y$yHg=1qE#i&EXAla>Vx
z_H{yeA@MAx#hE(nj9U*{2*9QA!$TL_$+2udaV)en-z5b?%@^!%IsE~A)V&GWJ#Xcv
zRbKMfpb32sls>qd-x9`11CMPyt!@9HeQu&hz+V;YUPCD0M`E}(v^wdEbMsO5AXvg?
zZ}MQS)waU4bO03u1TQdVMf;;K+7DbN!W3V)ubE4^hXR2{4;Qap4}LTb`r!lgcK@^1
zBO1o>3L${TLaE7+k%7u!=0=YP9$PAA{>dGRf_eZE+!vcc1>3yR5aCX6LA|)2clbmS
z5#~w>RfxPD_0Xts4sE#*xRKm|S6jQ(nxwyoNAt&aT;6+BkA+%-Ojp?=ZLL>NHNQxL
zr}JA57vHVy+QrEY?GQN#I%JD*ux9|$1I*dW)`M7BvT)zo_PH;<!(<6}rKRWIv+av9
zUWtTu&wwjbE16_kWQ^LLoyfkRAjR>`_Nd<YBZ7B0Y;R@+*jm;SEmf5w?!7FyW#!Vg
z>gYxZtm^8D8GUz+yE4htFR_Z_2kJbLA3ic;3o@7Gm>e_O$|wcBt@?CO%fU?A1`m6v
zKMR`;0@K$j!OTbv^#+;mGVqV|cU3zY`yblHim%z^vie&M0ZwjbRhC8OsU12I<mlSH
zE1PI0CYzJt9GTe?ol!+IKS8#@zopeZOm5y+bvmrFD>JS&U1?P+X!@|(I6<Yoi-2xT
zfF`dCN*qIJ!bBSZRL4^p_t#{n<ysM~vEFD3y63_$=R$L?XtRL(KIw0io@C9w(0ESG
zYl3e=L-z{L02&UN8x&$;QvN(oGjL>nP!21`-+cJWTkt}TiKjh(ID3I>WF@vJ^UREO
zG9D|aBVcWv6qBRy#_XVS&W0Nodih|iq9hHvjR+dX1rqDpr3R!ABz*c5l-g)QN<I`@
zl5xbaAV9S+9-7BD8aeBfuSTf$6QodkfJ|u_BTBHOgk0YElDFu$v{+AQwBiRr?S(qF
zCsrEVXEo0KTWwc5vvTE9%JWas-7ronsezM7){8U9tzoV3jj=P)>+&W_CaJNRB6^n3
zpVs5!zPUVAUX?n5341;!(!>nHYwp!%+*?w(Z#FSh-G!juXkpeo&$Wu)d}43ApRa0H
z=^;U~^HHl&*Ic97tHdO0GDh_npYNQtZoG7QOcQKbmfMyX<A%BoeVZto5`(N<-8MbT
z4O(I=Zj)Y&&PyX*i4|I82RG(rHIss5#0Ts$iEbua*F1^KPtFQB?%(ykJIA4lmDsMh
zR0y+>PgRd{$EoUptQy&>^*kD<bw)<eSnoNj%&6e1gVX@Km0avZyFA)TsKdLNJM5bu
zTrY1WWacLMy5p>c;5Vn;e{eSlA{6O3U~9^HAj1LZRm3DWq+qRXA7SU<%wyapotW#v
zKI=w&*B~ReFSSUgtBYgfy8paIjFMSUDJBx<R<pyL-q4MVTF9s}Ze&zIS8G~g9P2B|
zES;J2d8^L28v1)<EOP*aS;GjustmD0CPC)iWo;KKv~aFqKswv;+-*|Wxj3me6u7_S
zvow=DSfkh0A=RZ9G)U<>+S-Nb&8So<%cM!1Z5Gh3TDjDu%hX}?Sy^uwgfe+@q;mp5
zI~wF4(pRV?S4T@_Ds1(qN8OTsLv9__&VM6;%F)eh&~Izdtsc$D{pJ^Rx02%LgMq@B
z68{D<PE)3y5#cRVA&`|%wMQv~JI$4swD1ic_n&j=`-w!zDbxyNDwF#k7pV6x(RvBH
z(zJKa=<Z{Fw_B%&KSYiQGh-!ooy<4Lb=~^N4zrFT{6x=?;A?NC-@uAtrWoNFXJqca
z<;t>fuoUFL98e2{K#Hl;aq~GmNeBaJBC>}gzsp^VGuCr!*sZoRd8pu4CA~vvy-CoJ
zT+pCHLzghdj<#x1=5VVi2M6UUr8~qfG_57e*i?!skF4;oLu>+U1h(k$I=C#OMjtHY
zH*#;|$&#qgFtmy{p@_YgnIWum)^LSRexrubuH!?Z^q5@9di}PJsG#@hjU5z~b?MJ2
z&wA<M(e-wAi492(qp{-vw>hqk>j!^^c85sXz>xByD0F<L=nI@YF7b6-Nb;r<EeWTN
zM%yLN6a`gO>x9d_)LLp}pvD>?1#{TUR1&Kb=YmXXDCwh#oWW?gU4BQ7jT$-6U}=^q
ze!i`C$3>&>vTKvf9(jc;hsQ-ex0~zwLk3o>p-k-MEW|Ae5!*)TCL6f6XOiGHpSkHh
z6;jHC?%kSEvG^wy%IX_#JdjwvJ7Jt9xkHs+mRB<@7wV!wmYq8@L)~YoNGS7hq%wg`
zB|p-<k4yro#*b#;Ka>e8{W0jW=_%@5^2@)*wFj@0YW_y{ZV@4j1>)QVhY+r5bza)L
zijh^Q(ye{^{)IPo<mQ`Y+ewVK{a<=Ve%m%upHzT`R*#>Lf_XMNBR&iTglw7uSB-3=
z9C}348p?v16=Z6djN4&-fR{2<=w?a(Z7sRhF<QfCgG;2G)4b~@Jju?3G}rj4J#Zwe
zT;NTOPTGUs!^OqWy3|?v0pML~yilVbN<JB8Ef;8O84Kp&VtqEhI_6+89`rwdq1j7X
z4r$97`8M#82#qDNFvZD7RA2G<g|76PM7JC(1=r=S{Ao#Cqbb7>w7Y^=)3@Y)XG*Jc
zQ?Sgn&X3ubIMDdipVYE7R*PcPOw|v?3Ra8HiEp@uEwV?DrH@Rj;_vo3w5?+w5j=M9
zP3aG__G7RC2Tdy_uu<P%OCQ}#B__6P3=CR#SyK@2yI21B^Bma?9=gc*wvN}xC@!uQ
z_Us8HeBXq9UPoKQkSpH4p40AS;L^-#^;vyni6it#vDN!>iE@sC8uZl_%Gv{di&Qs@
z<$Jtr&v<~|2L7A-Gf`v|7w<$N`6~O>dW@;FziJc*t>-i&BNcS?VMI*BGc+;|hB33R
zUqUNtJ%hIHh+>6+x42;%S|CAozG>!c&wy`WTiZq=)4#aZ>hr43L?~<t)6`*;Ta1f^
zk!CO4(|rL5aq0NGw5-Kp)J7#1$=+?7`{e9D9{AgP#U~$d#a&ou01UC(v*cVX`<@0(
z;L;GnGWpvyvvk=wwRI-+ITcthaow~(Up1~SS~GbQD$uwIw*@?aknoG@eu8K1Y0uqO
zeIE<e*uAZiFa(7Y%j7IwBpdMOF2NMxnbpbWGai>jD-Pa|!xka;0A*D5-Po0Pe6Zni
zsTzm*IOL%tGL79yBR}gmZiZ~v%_sF|%wvxQ)JqPfUST{L((JZd;hoW{E#lm{jXGRp
z7NspnE_0;y<Z(JSii{3*q?hex9H%+8miW1THF&a0QxF5K7y`D+M6;saMXc48^pLi>
zu~8e9ton?-#I>@P&#LpB*Ctg9{$Pr%opg)B!eLSRm1cZN!hr}c!iQ~}$**ptC`%fa
zdHKvyhXK<eNgI>ZVh%!e*x%4>HF`&0=>&Q|T&NasVU6`HR6}7mn3Q{1(u-J5ma;co
zk~GvLG-!+Zp8@O<00@be>y>|bw#bCw_MJp+RiV#okTCe0Wwo$(+iqfwdiw=8rSQr|
zw5eiG!xvJhlYU~R?{`)&GM_hXdy}LY&ULUENY|X5*u}mwKGWWms`OUk_xA1ubuF21
zTv`S)*UdeF+$FJ5a;XujR`s&kEw`23(9|rxl$962G3kq3l&{lrC=k!Zh>&QtUd2#i
zKxN}<897~Jz3O19(c?t-hvb~xmz65jLNm!3l$Q1K+8Oc$%<Fp7Oto<D?N&oKW#bcB
z5x|l@lI+2jJ-3l7<xcfq6)mkQlN5L!nU>YN#jB~e*MB{2T}9?An?o8ttDQ27wl3<P
zv)A<aiQJ9#x_)+njrKS9H%9&z6W8<Jc%61~v#zD>Q<?h8rH|0>{~#W==or#8TKKxw
zO3I`l6fV;FNMx<_At`phEs)>c>{-&b`ZjtSiRmxub~iv<+Tp0dwn9?wXo@BUi@!4$
z*}-wRKL{FxB%KJi_S0uQ6uh=~vvfwd-g|p9t*Vn&I~%Le@!<)(^qP(0q=u}zqqnFp
z#vY)HJR$sLZ1<NGjCNj`id=nyYH5YLP{o(Xy4ehH(ghz;A;OgKN&_EMsTW{(0=Lug
z{D|w@RgyH>LuR&Qqd|ow7Ev+pB9}UhzVKd@k*CO3vKqcAt9<^k{MUpa;~BRQhYW>!
z=(%7M3=Q~8V$X(at8=$hH0&@Tc<vC}028Hi@*?+DsmRMoRW@2}HH9hs#Ds`SHqViM
zk^E}zZ1j8?-2nx?vw<J~FvRV=lk|Q^FWpH~D?-3qUdA|myV;%9!|b*rOX46g3twuZ
zTP0CO6A1ZVoALu^*X4A&w#L&PzqKD+8w8)#=(WPwd5dOCu{&I(;HvO+Up><7W=Z9i
z#zofM{tzaF72MfPjBw|n^-|?0b=DLPe;R+)?U4)8HYeSt7xdLqloNy9=Ph==UR*e?
ziTzmN8M=8`GZgPzmu#(h3Sv>np;=F)%1-8{{9akK4h~?#%wDF4H!0rPYv$Id7sSaa
zY>(VbZZ&|)#e{sj`WCgq==LyPFk+-%^eeS~)Sx)zwPa1z!~IgnA7zr6k%D$oZ<VCk
zao_rPyNvGWZCp!P{fSs(e6sE}WSd6UntXcRTBGuY>*%A9B`Q6_b+@y#5*O^;HcvEP
z&y>)rv3dA4j`3)pRb2NnvNOA!pff*N&1Gw}oF0B7H)MbsXqA3Tp11SfAIu9@E6<)>
zB;^#LEB&_2vGmdry^xz>-y`A#_C%DSlLymWn}BBCa_<8gI$Xav0fHuQ<_BtO$Ju$h
z4~j4Iw?6li7I%y$1S(pj-;#2NPNuByEp%{rdPh5~ZH6LuK<2wCS4GZ*&6oW>E8JF-
zzYCs|vL|({XC8%`^HusSaigeB%F3~|mOK|2TWd%MR3K_GJV0W+;^@mDa|{B$pLUhp
zt}(XWocQtexcUUO;e|n%nyjczpr4!h_4RCXTA}?rFIw-);N-&ZpK}ld&NrRGH%uSW
z!9{l`Z6O|zDqpvfWvZ*gOr`jL3pbwe`h@@ZZW6|KgAAFRfqH-fxU|sJt{NkOOH6Os
zh-%C^!`Wybl{aH3elEm=6K|9x%3Xlz!4TbcRqEUBf%5<S!e@BlW{}K|`N5SV@qtd1
zf>odWJGcaVcs4IMz-!k>?U~{BiYY-HZp7@mKQz*meCml*r&Z87^Njln+A74S(bc%%
z%6{3QB}7ZJo6GKbyp(ZYJ25j7!Vhqpg<9ymv@atcj@Bq=Bvg9jpSvhq67oTc+$!>4
zzU`8(`#=_VrMknZFeEtqIYBS+RNck;b%V@@fk%iYhGm(x*8c-nNh1x?H!S-HYKHt|
zH|;!554S8y`I4sH&QM>U&W?+Y!vAl)=}MJ{l)2FThbO&l`~@0#F|d0vYLjuU%fIb_
zgRBux9j(f@E;*oAn=4h5Vyk3=#G%y<%Oqvo^2wJSeAN=;)qQne9TVew9$*TgP_w_!
z>nE|_1z}bDzDstjv@ku}Oxu|qa;k*<H*{-Y|I_=Lrv&U6<z7jhlU_d)RM~&?z*&}S
z^RIF4@&w>6=`yg%8G-xv{{;8zi;!J=GGN!t!C?=}p{JL)Z~r%}jTuX*7cYRI*Zz_|
zkB!r3ByuyCEB9g|6lzHru-Rd$@=N}lp%CMT$De*5jH&+;G?}pZQ7_bs{e7ZeRce^2
zb%jvL99O=@nCWEH)u*jO<Fh4S;6PY&EgVR)WR_F6BixdMHT1mku(>AUY^XsyL)`iH
z80yBuXCT>@7aN4JPLAIAaFq6M1b-z(Q&4*@yi=<MT&BipFUHi6hI0Om%c3dm0T2R&
z_qdW$G7$JnxXj6lUYlYlI+R$Pz8U7=MMtR_8+Ifen6%5L4PMeN9sz4F-9Z!}_nSAj
zUCsCp`~i<c0NNmo^QoV<yzd^J)jN5QaZo<A8evHUE@m}cNjUxn5jUCsnd9^ijnY5R
zFZViT6rf({-X4pe2_DjlKQIjch3h2e14TFWT~2Y&&*7clgaU&AJP{!LfLjR80^CSj
zaB`wHC37WNe@{NiG%}5Qkni(SNw&Cur)TcNpQPW0x(+RF4*W^HLOd3JK)|cq+f$~(
zZm^h!nE1z<x$SC{Gjx8Tk#&9)HV{ca{*x-Cu}We~RoSJtr(uAZDEHq|bskBa^?_;$
z9*-FUpwBCI4Nj3vN>_T0X@Pu%netU~qzTau_mzbq*D(L{=r^8DP(~<+SpRQw8QCa=
z{3Yyox`4)h`vFVoNvIi|BvPjl$iQ}gz21kGs|WoI@N`RYefI&^FbzMAa?J-wCM|~~
z#-9~)cnXvbn{)Z<Ow1$Lnv=bVH7!R%WMgI6slQK^nE-1AUCUYgGg*+_oQkSwtadI<
zuz7ke?mHfjsJb5{3Ib43iM?4iGr2mRiqN_LZg$~hJr&2bC_vbH<M44-s^nIS&aGTK
z3%G9!WFveOctU>k<Rl^es);A#y{X&Rj`7qaS@YwTX{v*_14L9uf+u6qF3^d6Q2QAb
zhQ$Bki{}CNa?X|6|95XWpAyhpAZ=o^M20+QwtmoM%qaO0`tJ->gR-K9e)~bm{)AH`
z%ZzeA4q2mdJX0}<ZN)JSMXaz7JIMmK#tf4mpv@tYu85P1uFjl`Z6*!6>efUHtsWD`
zKLE(ZOZR~`GPZM)U5C)*f&<9M4}#jjZH8}_BJu@U>MfdUn9V@FxB<7~?F&He2i&qn
z?Re8*uhE#9jU1QP;(C~lw78;Bh#fjy#J5Aor2su--i|uIDdLb*g4-QotIai8_>a9u
z9?L@PNA9mc&eA3f)^T~o0Y}&Ohc{D89Z5K5s}7F(-GJy;|CCWiXWc7MCiackFRyvq
zevm6~+zFJgQP;`E{VV!QNsRu2eek3W!$8uN-E5VVyS6Sv)qLey?uUC4&1kGQl`W1T
zjBivNbEHbSutYL4=)4(;`TeSo>CI)_r`|U1OZrGO+^~!}K$B<F;t77CYlSh+W_pNH
zftwMb@{w5&uR@lQ`-?5>aDtAl1{_wiS<>&WY0P>K({23Tc16fJn<XN@l^K6J8Zuda
z@%oaoRBkd4mYBa$&LfXyefo9?!KH03f;RYZ{|6f1oQdrhO7gm_PkX#pOC`p-Rkv=%
z$08{@__?uI0^2?6iMRL~s&;w57IXV3q5ZV$s(yv>Z)?G~ui`i%r{tKa+X44=R3o*H
z{xBT-FX+P=TSHU=yXqrcT!m*%egvjca6jRWfdUV8GndcXOcToaJK!auQ7;}qDX_Yv
z@*hh#nc9r6zGVF42cfWq@%K(#u5M#lCyfk*cY7RO0X2Yz%(aI+YCU)^8Vbe8@7OtP
p8NNZ|b_9YUlm6#hDPPaAIq4$}3IaQp97A@ay-!~w5qs>~{{d1>&&2=$
literal 0
HcmV?d00001
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index ed7f770..008a43b 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -55,6 +55,7 @@ Programmer's Guide
pdump_lib
multi_proc_support
kernel_nic_interface
+ virtio_user_as_exception_path
thread_safety_dpdk_functions
qos_framework
power_man
diff --git a/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst b/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
new file mode 100644
index 0000000..5cfeb10
--- /dev/null
+++ b/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
@@ -0,0 +1,104 @@
+.. BSD LICENSE
+ Copyright(c) 2016 Intel Corporation. All rights reserved.
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+.. _virtio_user:
+
+Virtio_user as Exceptional Path
+===============================
+
+The virtual device, virtio_user, was originally introduced with vhost-user backend, as a high performance solution for IPC (Inter-Process Communication) and user space container networking.
+
+Virtio_user with vhost-kernel backend is a solution for exceptional path, such as KNI which exchanges packets with kernel networking stack. This solution is very promising in:
+
+* Maintenance: vhost and vhost-net (kernel) is upstreamed and extensively used kernel module.
+
+* Features: vhost-net is born to be a networking solution, which has lots of networking related featuers, like multi queue, tso, multi-seg mbuf, etc.
+
+* Performance: similar to KNI, this solution would use one or more kthreads to send/receive packets from user space DPDK applications, which has little impact on user space polling thread (except that it might enter into kernel space to wake up those kthreads if necessary).
+
+The overview of an application using virtio_user as exceptional path is shown in :numref:`figure_virtio_user_as_exceptional_path`.
+
+.. _figure_virtio_user_as_exceptional_path:
+
+.. figure:: img/virtio_user_as_exceptional_path.*
+
+ Overview of a DPDK app using virtio_user as excpetional path
+
+
+Sample Usage
+------------
+
+As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen before compiling the kernel and those kernel modules should be inserted.
+
+#. Compile the DPDK and bind a physical NIC (for communicating with outside) to igb_uio/uio_pci_generic/vfio-pci.
+
+#. Run testpmd:
+
+ .. code-block:: console
+
+ examples/kni/build/app/kni -c -0xf0 -n 4 -- -p 0x3 -P --config="(0,4,6),(1,5,7)"
+ $(testpmd) -c 0xc -n 4 --vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 -- -i --txqflags=0x0 --disable-hw-vlan --enable-lro --crc-strip --enable-rx-cksum --rxd=1024 --txd=1024
+
+ This command runs testpmd with two ports, one physical NIC to communicate with outside, and one virtio_user to communicate with kernel.
+
+* ``--enable-lro`` This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can be transmitted DPDK application and further TSOed by physical NIC.
+
+* ``--enable-rx-cksum`` This is used to negotiate VIRTIO_NET_F_GUEST_CSUM so that packets from kernel can be deemed as valid Rx checksumed.
+
+* ``queue_size`` of virtio_user, 256 by default. To avoid shortage of descriptors, we can increase it to 1024.
+
+* ``queues`` of virtio_user to specify multi-queues. Each qeueue will be served by a kthread:
+
+ .. code-block:: console
+
+ $(testpmd) -c 0xc -n 4 --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 -- -i --txqflags=0x0 --disable-hw-vlan --enable-lro --crc-strip --enable-rx-cksum --txq=2 --rxq=2 --rxd=1024 --txd=1024
+
+#. Start testpmd:
+
+ .. code-block:: console
+
+ (testpmd) start
+
+#. Configure IP address and start tap:
+
+ .. code-block:: console
+
+ ifconfig tap0 1.1.1.1/24 up
+
+.. note::
+
+ The tap device will be named tap0, tap1, etc, by kernel.
+
+Then, all traffic from physical NIC can be forwarded into kernel stack, and all traffic on the tap0 can be sent out from physical NIC.
+
+Limitations
+-----------
+
+This solution is only available on Linux systems.
--
2.7.4
^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
2017-01-13 12:18 ` [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path Jianfeng Tan
@ 2017-01-16 6:00 ` Yuanhan Liu
2017-01-16 6:04 ` Yuanhan Liu
2017-01-16 9:44 ` Thomas Monjalon
2017-01-22 0:46 ` Aws Ismail
1 sibling, 2 replies; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-16 6:00 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Jan 13, 2017 at 12:18:41PM +0000, Jianfeng Tan wrote:
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> ---
> .../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes
> doc/guides/prog_guide/index.rst | 1 +
> .../prog_guide/virtio_user_as_exceptional_path.rst | 104 +++++++++++++++++++++
> 3 files changed, 105 insertions(+)
> create mode 100644 doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
> create mode 100644 doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
Doc build fails:
$ make doc-guides-html
sphinx processing guides-html...
/home/yliu/dpdk/doc/guides/prog_guide/index.rst:34: WARNING: toctree contains reference to nonexisting document u'prog_guide/virtio_user_as_exception_path'
/home/yliu/dpdk/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst:: WARNING: document isn't included in any toctree
> +Virtio_user as Exceptional Path
> +===============================
> +
> +The virtual device, virtio_user, was originally introduced with vhost-user backend, as a high performance solution for IPC (Inter-Process Communication) and user space container networking.
Doc should also be well-wrapped, say not execeed 80 chars.
> +Sample Usage
> +------------
> +
> +As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen before compiling the kernel and those kernel modules should be inserted.
> +
> +#. Compile the DPDK and bind a physical NIC (for communicating with outside) to igb_uio/uio_pci_generic/vfio-pci.
> +
> +#. Run testpmd:
> +
> + .. code-block:: console
> +
> + examples/kni/build/app/kni -c -0xf0 -n 4 -- -p 0x3 -P --config="(0,4,6),(1,5,7)"
^^^
KNI?
Besides, it's more like a howto doc instead of prog guide to me, that
it's better to put it to doc/guides/howto?
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
2017-01-16 6:00 ` Yuanhan Liu
@ 2017-01-16 6:04 ` Yuanhan Liu
2017-01-16 9:44 ` Thomas Monjalon
1 sibling, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-16 6:04 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang, Thomas Monjalon
>
> Besides, it's more like a howto doc instead of prog guide to me, that
> it's better to put it to doc/guides/howto?
Cc Thomas.
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
2017-01-16 6:00 ` Yuanhan Liu
2017-01-16 6:04 ` Yuanhan Liu
@ 2017-01-16 9:44 ` Thomas Monjalon
2017-01-16 9:49 ` Yuanhan Liu
1 sibling, 1 reply; 73+ messages in thread
From: Thomas Monjalon @ 2017-01-16 9:44 UTC (permalink / raw)
To: Yuanhan Liu, Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
2017-01-16 14:00, Yuanhan Liu:
> On Fri, Jan 13, 2017 at 12:18:41PM +0000, Jianfeng Tan wrote:
> > Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> > ---
> > .../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes
PNG is a binary format.
This repository targets hosting of sources.
Please try SVG.
> > doc/guides/prog_guide/index.rst | 1 +
> Besides, it's more like a howto doc instead of prog guide to me, that
> it's better to put it to doc/guides/howto?
Sure, the prog guide is for API usage.
You can consider the howto section.
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
2017-01-16 9:44 ` Thomas Monjalon
@ 2017-01-16 9:49 ` Yuanhan Liu
2017-01-16 14:45 ` Thomas Monjalon
0 siblings, 1 reply; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-16 9:49 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: Jianfeng Tan, dev, ferruh.yigit, cunming.liang
On Mon, Jan 16, 2017 at 10:44:04AM +0100, Thomas Monjalon wrote:
> 2017-01-16 14:00, Yuanhan Liu:
> > On Fri, Jan 13, 2017 at 12:18:41PM +0000, Jianfeng Tan wrote:
> > > Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> > > ---
> > > .../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes
>
> PNG is a binary format.
> This repository targets hosting of sources.
> Please try SVG.
Agreed. I think I saw the trend. But I don't think Jianfeng has the
time now. Maybe it's better if I apply all of them (except the doc)
now and let him dram the SVG later?
--yliu
>
> > > doc/guides/prog_guide/index.rst | 1 +
>
> > Besides, it's more like a howto doc instead of prog guide to me, that
> > it's better to put it to doc/guides/howto?
>
> Sure, the prog guide is for API usage.
> You can consider the howto section.
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
2017-01-16 9:49 ` Yuanhan Liu
@ 2017-01-16 14:45 ` Thomas Monjalon
0 siblings, 0 replies; 73+ messages in thread
From: Thomas Monjalon @ 2017-01-16 14:45 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: Jianfeng Tan, dev, ferruh.yigit, cunming.liang
2017-01-16 17:49, Yuanhan Liu:
> On Mon, Jan 16, 2017 at 10:44:04AM +0100, Thomas Monjalon wrote:
> > 2017-01-16 14:00, Yuanhan Liu:
> > > On Fri, Jan 13, 2017 at 12:18:41PM +0000, Jianfeng Tan wrote:
> > > > Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> > > > ---
> > > > .../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes
> >
> > PNG is a binary format.
> > This repository targets hosting of sources.
> > Please try SVG.
>
> Agreed. I think I saw the trend. But I don't think Jianfeng has the
> time now. Maybe it's better if I apply all of them (except the doc)
> now and let him dram the SVG later?
Yes you can
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v4 0/8] virtio_user as an alternative exception path
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
` (7 preceding siblings ...)
2017-01-13 12:18 ` [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path Jianfeng Tan
@ 2017-01-16 15:05 ` Yuanhan Liu
8 siblings, 0 replies; 73+ messages in thread
From: Yuanhan Liu @ 2017-01-16 15:05 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: dev, ferruh.yigit, cunming.liang
On Fri, Jan 13, 2017 at 12:18:33PM +0000, Jianfeng Tan wrote:
> v4:
> - Fix a clang compiling error by removing "NULL" line in the definition
> of vhost_msg_strings. This error does not show up when it's defined
> as a static variable, so not necessary to fix it in stable branch.
> - Query kernel to get how many regions are supported, default 64 regions.
> - Set TUNSETSNDBUF to INT_MAX.
> - When get_features, unmask those backend-specific feature bits.
> - Remove VHOST_KERNEL_MAX_QUEUES (8) restriction, but due to another
> restriction by VIRTIO_MAX_VIRTQUEUES (8), we still cannot configure
> more than 8 queues.
> - Add a howto document.
Series (except the doc patch) applied to dpdk-next-virtio.
Thanks.
--yliu
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
2017-01-13 12:18 ` [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path Jianfeng Tan
2017-01-16 6:00 ` Yuanhan Liu
@ 2017-01-22 0:46 ` Aws Ismail
2017-01-22 7:16 ` Tan, Jianfeng
1 sibling, 1 reply; 73+ messages in thread
From: Aws Ismail @ 2017-01-22 0:46 UTC (permalink / raw)
To: Jianfeng Tan; +Cc: DPDK, cunming.liang, yuanhan.liu, Ferruh Yigit
On Jan 13, 2017 9:42 AM, "Jianfeng Tan" <jianfeng.tan@intel.com> wrote:
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
.../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes
doc/guides/prog_guide/index.rst | 1 +
.../prog_guide/virtio_user_as_exceptional_path.rst | 104
+++++++++++++++++++++
3 files changed, 105 insertions(+)
create mode 100644 doc/guides/prog_guide/img/virtio_user_as_excpetional_
path.png
create mode 100644 doc/guides/prog_guide/virtio_
user_as_exceptional_path.rst
diff --git a/doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
b/doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
new file mode 100644
index 0000000000000000000000000000000000000000..
9fd45e13eefc2ef39ac2c389f407d60ab4144c59
GIT binary patch
literal 38600
zcmeFZdpy(qA2&W76`>TJkPbu+9mM6hkRnDo=QL5tVO(L(+mv$(Q4t~M97nNb*gBA;
znuUfnD`#ee88+K~AL_cUuIqcae~0`2J?=m5KN9o#ybrJA^Ywha-xAK7pWU)~_ht|X
zw8g~uv?U0%&Hw~j(=50F_(Yg<KNk3JorgX|9|WpN7Fxcv9{73F@5UDcKp@GjEB^%E
z$<2X5pa)(ir}eEvTo_!Tt}AvIM>*vi-TI8vT#u~x5|-A<j&eQlZMUTfTP;0M!!@i~
zh<W|d+IRu|7y8=U4T4(<Pc%gmBE!ry)U5q$OE);WyBk<L2nnCdcUpg2!_vgqvfeKB
zoTb8=N&cMA!U=lm#Q-WR)4QsP#YVnayi?d8JY6?}d*3t?(O<W~+YJI8*K+Z{dpY?s
z_Hx`hV7(xa>k&q(l2h%zHLE{W-)EUd$ev#N^M`@`f-z!0zOOqRz4}dL?_;qa&t8o(
z5M2EzW6NgQ)o)zY{}(*;+u^)FCcKo~EUC{rw>lRH^QkS>`=G<RUU^N^9}D{N`TxYP
zk@bMIMiklnAG?F7-d}#N2$MPzTF{)cAihm_RH@DFzQ=?ph#8dH?D9IN0z76cl*WqK
zd=GfxMRdfueDTSqcGO9VY|UuTocROS-_WT&yw6-wpKZ!(oDTLL=v99BPP5F&m2T%j
z7uD)JR5NNNC;+@GH}v@Q##uWhzKsRnt1L!z&oavF)jHP@zqB!4KVez2`eKCsse<))
z6`r4RgSnJsl%rAXFNk*3%DWlsxb2mYbxc(poXOawwH4b+ha%^dkMHa++D07>lNYJ>
zT&CZ=vYC1EeP6>z?}rknnDq`~D{mg%$_^D*-`YW3ev*$(p=2M}%{5hgSA*T}slTpA
zjfMMqVm)GNtm9PztX#qxcphYSI&0xAhthhI9lGOi`>peAiCwOpi{y>%+k3K==#BT!
z($2Y4%48m_xRC4qK#*V5(xRg{Q#FUz#Mkodz_vbC;<FCjI^T0T2+8)*;`$u$F2{d_
z7sa@T<0bB`EXnM2H&%L#PluW?g4qJ4^oXWH52#IPd-s^L2pifoIMkA=B7-Y~lFQql
zZi<(HG5Parq4Vm&h$9Kekpx6|49{FfhBlv1@22$CNjZ#>_!G{O2q8U9Y&QW>>es-e
z>%qd|;BiM7Q~3gfHhW?IWFDJ8qt}JqGZtFZ&Itc_*f~~O+?jWB8&#Ce5387e{w9}E
zJ!O|UajbLhTzYvs*}a(a#Mjw*SVvVoAv;$u%ba+`+nY*RQ$AMjMnrSYdFy)AMwYf-
zgr1$RGa-8D_R~AaM>(<53{jjJqczM0zU0N|vfD@}GM$*CSII%F-u^{6b&RSJc02GP
zo9%|~?z2Vs(EJoWskfrtKQGMJDb+#C`}$5pb0_ED_s>3|SYxZI{Id@L;&MDPFC8i+
zOo_HCh^o_D3z1qJKmF-}-42)`B>^vDJ+G<tyi?!DYY7KCv+*YTgjA%Wb8?j`GAUW)
zT!Xn;T#v2PHOd@20#c(I)<WW5)VM(6UQMn;<co`Ii@+~>fEB6fS}7LFo$|g7w2$ia
zZj;A3XORzWNbvEY@^aaaiICL32auA{+jVsx4(g6@PIuD_%(b}Qo_i5@yeJyAgkHM6
zGgHTVE{rjKjWlP)(JuqHwhU?^u;kBZ?uGHXClCT@6s~N1xGCtG{((B`^;$-sv^yA!
zQ75tLG?BZhabae-cZeilt3Jv{HrSK)NeesXU2@hs`s~WZz?2ekCr-J&j$n6!kCc=#
z#eE*_!m>MgM}>&IiW=@>QKsv3*>=Y`*eXX#9=LOi&sme8Qc@|4u~}kO9qGU7jGih9
z>2Nx^32Sge&u(J}GMCe^J5ZOEZp>-YMYl(vOCQcflQ3S|fl^v2$?{IP<ol3Ef!UB{
z=)7{!`#>2_%U0!_?cKX38hZ}n84n<%$?aOGLs-4MQPeHG`4X?F)rm+CaYG|Yr%u{(
zvU2=*n!9!Vm?9e>%N%v?hg?qB#yH<N9szSPy_8Y{gOyLS*km{&N1~(xvfDXl9(9LD
z;JrtpN!*U0n`kbXkhFAV%sXQqLfb$^hdGl*`-qe9l3Tv%Dy`Cfq+HL*c_sL5IJov$
zmnIM(tM5nLIJNPd*5TPGe6fIwc-G--f%Uk?GJUUzxL9+`va!MzX%CVBw7a<r^(1dg
zM4c`!sZ5}LN;KTu+422(N?C=nR-1jCOfB8j9!u<GUCTOYBW{rO``FiBIc?k)tsA$}
zJpws{+)u?JkBva#NoVBO+R!`P=^Gc>c6Q$7yv8K+pvlnP&VJNrdrllQy{g1_O?m8!
z*VOfTbS@1-;mGb9@;0vDEopB@yo8jFuHU2%nOvoK{gMOO=45C$>^q{Ql+ZFt(8Ciw
z#OA3keB6}K_P)3Y_LVPwecSz*@Ymq8oTn-!xA0Xx*E`5y-3_cSI7BgrCK<=%!)ew{
z8^(z93%m^Pyuyzuqd@L@Vf#too{UZ}-<vHQe_y13`DytEWRA$E3Ms8q<zai`2}RBY
zYw5U6r6Rv`2jLy|8r8NqovicTjuXQ5mG9(eaPeX-SieI75C8@$VYOGT)k0txtEIv2
z+AR*u>edOKpO*O<N)?e5)K_>)$k=9b#+h8Oyv|a7hfS;=uyI%kvA8qf0;P1cU8}-k
z?6hu6k~_88iC9#Y@FYhFxsA#?*-Pl}p2sel-@zH2+JnGqAf3t0>+dV#(32rv6a8}H
z({_y4koAxvF1czG{`>hl8ep#>cYMRM<_|L#c=a}ZX}j2afk@hOj)!)Tkor0=B}RYz
z;Xz51yHrJ8QOm9@SL-cLxCd=16t$OKFKcyOI)gZY=RJv~nJFSomG5}|rEACGi`?#@
zeN*GSx>4_1!B22a9y-M)3J(5_Y;uYWUrb0Tp1}AMa^M)F@tIGfwmZ(BdnB3FjOq61
z>|JM>Y+;AD7|Rw@2ofeUusq*<Tms^j>S0?1FCq6Dc?Yv^qM?ZgKi7#7bYo`N-^icB
zeO>SdM#X!U;?x%#HaAqA9ruT8GndQ4xI2Gt<@$|NRtt7a%%p0RmaC7VW#QPVzP_}~
zLbeIK6ZZNY?J4yNCot8USdQxs8H05TRC8VAH`BkT%~nNS8D5AIc~7pt-)CW&$-7xK
z?jILRN{G^hTGdrza^P5_lRGR{baSV!+aqmUuP&%+E_0M~+U63GrrR;UlRa#=!(n2K
zT|uyc<tjtd1{qrs7`r9%d>J`j4j+;o_8}?pG5!1cb&y-A56;1k)>Nd@_0@272;BQr
z;O<rGChV*q_M*q{#mm`>xW){AJE|KxsKd8=rV`vRHuNGVJ<~e9$Ci);=X;HAY-H1_
zj5bh%ybZHnS?y?Q&`-}3FV4t)T+yI<pML~ve#Q-nLeJUtsjo=<P2a?`wg^(i#K$}>
z<SXa-%mJYfVN<~hB4f?gxmaIW$VF6VzSDaGrqWR?e!kab7)mBnuOf1k55T3K!N-uW
z0EY3*dcmgh!!2kJD$MgZb2V(#`G^(PP=067=wCD+Pol#R8$6Ilvoe<{jz(o0@@n9G
z%W-+3(x7`iwqV&QhZo-?jv01dnH>ew^NG-;y2GyR3CU4>@88q0cbg+fN8o1ud!NZ2
z0#b!LY`*t0{<|&w>{%(4iXG(1{N4q04O^?LSd?Bv4mg|JIqJKnAwWNU#~g)L@_f9q
zq3Mkand9^9DcwFIJ&Mp^&Kbik`v-=**MGJuY5uX+ic_xZ*K>t%TdJ4@=c$~dJk?pv
zZK9vx)i8p$HpnUX#h9O;Io0_n6Qjt-FX^pcQzY(1kpytg?^&;6A(I^5a1Vcnt_vFC
z(kFo}Q`<5L(=FX(rn0i$G_M7@xnR{oc91wixn`8S!)`v9SCd&hIG(t@e%k_8J-D{Q
zJ)2>}gCnT>8%bmZ=cNAo-^S^=l?BsOS9mUED3PqWP-!=?>W&WT3am5(uzi<_JDD=e
zsIT9~ulr2<)IXxE!&9X@%irUv-0Ip#LZ%-nC74kt=}h~f#~h+X7rY)41<$f)B4p}!
zD5%ZwYIB2?ZCfg?6;LwV=q3<X{~GwvJFQ@)dk=u+dItcq<vg~@7Q=oM%G}McvWbB}
z3)}8zOefO&>#R%KS&DpI|L8}JD=i<Niw#g}&1c%54Hs7F^A;-ueCsa!Sn4^gt#tzD
zbrhIu&q#g8a!v0-i_O&nt9vzO_EGtD5I!M6rJ#LKSNTy-fcqhgO>gChChc`43Q>3S
zM)CNR42ND~p2Pl*{Sc0XvYI$A`?GQtpA0!!+_xNUDV_=5eNp`)MO({q*--|!eJb$g
zbYe<hi4~odOTDKh^V#G*h5(cGkb+?WCxbH#>9br_q~k4=?sPzKHuUU7re!eoZP5?q
z-w*uizP7Xr_6qs9D5_U|?OCvXhsjql9Ivi&yedh{R>`=@xu6-N$Z4Mw%bh?6@9tS=
z%g@UEo#XhK({FTX{r%@p3h32A%-(d|UXJ(g-TwX+CFZ4bZk_f5?>;Rkn;Xu_T<yJ|
zTU+9yO&?qOmUy%@pjk3M4p(rB^H>G-78_rxPAOlehvkZHWVOoS)5_>|1-A&^$GdF_
zIeJFfT8BOYRbgahVbtk8#x_)iZyT$%u0LUW=Xi)OsGt-W1{*(pE>>sEU?YRC_-!i9
zR1yZ_Z(sj{?`=i=`FojN<NGK`+Gy2wF1=b=9<FgFDdA!k5mY@d9QuC6`_B-?Idp<V
zV=UVz+G%oUrB(He(EwbYffp-iSwplKiR(+YJv!<5#i!gi{65N=jBHfl$nAhQ^~~p%
zUx0t`eogCr&?mt^8=IE|O$Y#LL)RlqYxKKaU4*vS3U}1jA9;C8UQ5Bt>dDx<4S6v6
zqv}UpJYKm6AGL-^#P-`lqj$~*?Dx^OY9)(4hi`lKF=!&dC$*y8m>TNVT1u>B+YWVD
zLqvRZ>At3W409#Nh`C3clkVm;giSGZ<CY6`cxJ-1X{%g)Qo3)HGNH^h!;#8TggV({
z$LK<NrTYsdlqVPH_><=7`<UBhWJN+w-e!nl?keVOmy6ECy=)Yjm_(Ok7E`!f*h{+b
zQ+T9Zt6UdsdT+Hs63^(V5LF%{WwZ7}9=6*;I(E4nO3hY9?m0;}vy32BeyE(G3r})6
zvnOphnCW9yTD!{VClF*(pvz-0pG_XF5<!g`bb4Qs0^fyD1un&8mc6<pg@~9iUq3kg
zaU#Iuv?PS$ji&FWj#T+6kYJdbRmBq-eRXEj@#8pX9g!0>LBF~KNRa)~1#~a=mn2!@
zswG^BdZLcHLB8>bdaklG>&rOk;{)oI>e8s=9MdrBECXEw<A1u!NCN+cPT;pgyDwiF
zgU$Ov9Tu8Kv@Vusjg71K=s_Tp%gw}8cB1bNzDrQ{Y3nU`?!f~3>9^DJi7Kis=(C;k
z<yv+7iWr?&tQq7yq7ZYG*xp55tDtqP4xi7j)T#=bnxI>CYd%?a{A4+LSM_F)!t-mi
zE^?qQxO}r0?_&av9E<VyK{M=3(_ht9(P;JdcE<C;&+C+nk1(=$RXZRPB3gsVxe`XW
z#+-3vE!=Z!y%#|w^e03Y>*Ct9cfL%Msrbx2L=shpzvvFsh!T0TJY+3@xAbMMh#cb>
zOzxIeNL_1iQ~L`)L+a&APhd89!5WbUQ-fs3?wL=UZdV?dPj|@5pda@M4VzM9@I(Cb
ziJqCliv(*rzCye8o-~w()COnuZx|*Qcv=aQOwfeHQ)HWkMp<CLC8PN$UGFeN#~f9<
zyfiapLesPL0ujXxHX~P#M(drSoNr<0oDr%oy%Opm=JO1~n<_27qTW{1)Q7Fm#bpO+
z#HgT4&ley%s;;n><?u4zFY`*zm%^%HOb&LOaT2KuOO$2fE>X}V@(rB3+E1t*SUt4S
z0=7grHST&&{+>->J<I-%xjkI07-iGFDTik#->+rQT>A`nBbP6RD>ixhNuWYk?_?W#
zP+kub)ecl7z5B49ZqP%>hIkcXU(pQeTZLkL@48d`@<~Hch9*u#S2Zw{4dr50$QI_z
zdYGKRJHnka1?`{QzbzD1on5G>rQ@7Ge^y69k{0gxCekz{vJa0<FT2B=ATe7`c7@*1
zS$5uxq0NU-YRK*dqY1n(U~^(y+5Y>|8vGDOgM%80e8jm}8b#A!(rg9>G9<(+2;<YW
z7URe3P|dKeR-2kROA5&@iP7D<L~At)W@xZ>KPC+p)8n!AWxws{d=;mvevQL~pH0TP
zoM0|sq-hIqM4uif*jd)Pg-(KfLilms<nuqnn91E^>`a%9v07(=q7IYFPYfXw1eb0+
z7TE`{wlUiH04et69*z=u9~#^V-F&}_CJnW#H6h(7)w9_605;Tbdu6R%c1UHrWMEl7
z#op#g^`?Y6eac>o@AQ054ZYB1;|xo3sCew)sg3M+ePwhXL$bJqUS_^gBF0GSWOL|^
zh=Es*R?I~WDH&ZF;$^iWu0qb4lf}v^-k%MbnkQ&7sV>lzrx-dtoPsZ<d?`LUHI8KX
zPcq=|p;3KKR=BFO1!ze&j>>|&#{Y<#<INW~R%OCBoOaNu8Kr|Nd|wo&ok5r_+rbk>
zg5ZyPt^$zys-s_)e!jT@V+Mi+x7I?~jhHmA`vH#kC1h&1ApD@81sYV2bj!gkZ1M~i
z9_p&}i)zh0aVQq4ou7f-vazX!;BJ>8O6i|X5^CBqipN#!av;{s(9-#9%;)3KEcz`Z
z+&@?Ht!c7%cfAj(p%j-x)OrYkjLdsDLxbn*%qSTSIcHEMqlhfBk+SjFQVu665{|Va
zBEl=C-TO-B>(E|Vbzr!{&cYsVcP}ZnRx$UpOfYK8gg=CIWlVEux<9_4y^CC1Mjn*Y
zT#03%%f)LA8V)`8zV7Cw4s(R(!x)|?##zB4<Ix9<Hg;p)%VIlOSIFAoxKAeWv>dqn
zA^1qw#^N_gcbbgkT9=C3s|T~)w@q-qbZ{?8Ib-dFPO!AB=0o^>MZ0cJa||U9alIpX
zDBmpPN>~ByG3AF{6Ter5%~^YDPf*Fd{<f7*K?R>2M=QmSH$#;LUg#N~rR`p21q(hc
zv^vUOj>pD>GBgm*HRPR$rWIx>@<OjNot<|Bl#<o&c{L^lFSB{e$|y~q0;lB<1B6)Q
z7iN}ug;+jcbmij$lAl!drd!DJwV<Km;2S-3VVdvC_^#GGx}?Qt8PKV#Jn!q2h;EgY
zr$Kx4!OYqH3COv)e*Id*AM|AP{bdbd&qc*m>I_s}_kWnyRZ;z$FE2Wh(tpsSy|OF1
zS}9)}4hl&s$NzZPr--`^DoFa{hv$L@vOmadALakfL{B_A)O*AvTRvLMKv353Y^$v5
zeGn)GGI4Wl1|qdf6ZkM%%u;dVA;6&1a6KS7ICAB|=zAg{Q19<~`q{s`0CV0OYL(pv
z0zFMoJ??bvYs?&=-Ny&PB6n9tin#mDV6EySN#L<VOV{Jqfg*hd5``sqhkR=UK04f%
zY<LhDTO4}%dB+Q(QJ`aR$^^N0zW``o*5dxxF&kWdQw1J7H#;b{37D~2a+iwQ_gA5S
z2+pU<#2JG?Z$}<S3vDTj{&mg|(g}pOllMV#kM{gBtN8<`10<pMdTkfCy@(b7xQ?F_
zfe!B%KK0ier?y@Kt?>aG2K85KJcv1c5ffIq4!Sr(-e--GpuIXFzt4AG#p~jPK|3>7
z7XZ2?xK_-yvyoY(I=^=4jn3RdqEq8yi>k|HU!8XiiqGq|?4v)$4<>_=^;Jizp1aLY
zdb#_y_BVywfh&t!w(%dOnQN7H&w(DRhEp#B`+ppN6eOSyA+NbfOp?-;@C&C<Y)!p9
zsPoyOBM<A=CmzEVls?WgN_qBC>xd4o%>sGcNDS1{<4sfqHem$p3}CaPdCPM|^lgd#
zOqI<jJx6oBrJ-R*-M;U1PD1Wt!Vpdk9oAj`pnaZBT4T^nIsfKM5-vYh!Y?qJWlxQT
zPn{Bog!j__HXweB@@c2A@uIV4Qzu)c!+l1v)c^t>$WK(!a?^EqZ`|9==(rgtp*MLw
z%uWzkP*JaU$6pJ=Nr*KjBPQL?Zr@fL{}^1}StpiXd-NOmuA-bzU(gtFV83xURS9(T
zV7QdSUn0m|bMqFMZvg&UJ4HGFSt0r`U;SYhmf-U#XpAub`aENSZ4cU+*d$R2*xUKj
zJ0n3^#eM?%Ek^6OlgWy734{K{HyW{{3_#TLov+oeXxADS$l$#QokT4KlSB>nOEfRw
zg39x+H;3?_b9?QUY^jxQ1dAOpECwariL3>+a`MuBvA35_`fDHl+r{pg+s-Q=!tLdv
zdD>T~2Jz-dMkxC}wg%E8lAS@%KFA7y1t_pX<#{UiteFfjPZ^BNwUrX$H2I=QZZXLD
zYOcx-<qhMb4YgN@d%}PTjJQDe?5EPJ*4+H0ec0zBTat6nS|D!=AEFvIddg>#(Lp>m
zW0X<_ZEaBDHf~2rnr4krPkLn=#P{DB#6DACYbw~Z93)DEnPHKdx7b2}asM#mN9)&R
zn^B%Jr=ssYkL-wrR>__GCKaH~Oj_5FcsdQ|enpTSB*DK?(YFC+xBOshlhA3(5ZJ>=
zh72gEy&6Ty-g*7Px%7e<321W%q9?F|QDcbSm~nu9pd2u^?OQkOdr>?i&fdG6y{D9z
z{hjW8!n1o*b?&4s!nZYpa*1RRjW;UV&RyQ>h?UT^%te124_YKT%=i_RYDtu~PDmYw
zK-X5ZDDojXeR$kx&tR;J=+U#E!M1>ZuXtr7B&{LxE4Wkk<go>MCl-7y@4ya(>APN!
zTq)^tl8JveMdr;HkvkIlOvbUIF=uJaR8<-Z)f!V&D&{uM{)Xo5{ES$vfW}>YG7kzE
z;yeIC;!i<Ot+`1U+-I{hUPusD;3rVp5Oo<4a)a$TL}EgJ?<Kzg(i*eQ?0pI|qqk}m
zqQ@xKmcgxEq&0V_!+Eeu#mn~{fOX<gCM)qLq|GQ*H<g+ywHj8cJ^dxcm~(r5LQ9G7
z#vBAmSlu!IOu08%)lx;bWO=?R@M&a+!3_yvA5GY1S$hA>z`~Vojq!Vb4R|%Z8G0G$
zK4$#x3(ArU7SN}Ca%=Cz;+vP*6aBjf2_toPj$vP;Tub}!pAQ~AC&Ip?hQaH_Nb-YH
ziw)u>LO&N7DRCZ2awqTKv!n}*yk(OC`@Q@O`YL9F4G=kfGQX@}#}~+P=!jpMA6D9V
zyIzX@c{|@WEiX~_Ge*y7?J$kkX)Q?74q#V{s#SUglmfE5%=O;!Ykcofs}~Z)?@P&a
zvM3V!1aOhQ>$0M~nJRbl@J-EAr$~HOUNCMEW?VyC;DSCC4_d8w(#2f+^}j}cb;!qr
z0*STEC>8ic_L8{5bphFQz>Ab47O`!@Ho`S%7Q<1Hgl6Bt%3dW*a~|%&5HhC(cdhu$
zMLrNFen}+2P-|}H+SAFz;=;{>nm+G>np?!0o}CstqB$JqeVhL~so21njDB{fmwFtM
zyY={rn>^*!ZT>Z-_+$dk0tG!`S9dCGH(RMYe3O<Kdw0im4;J*94x$T=(7LLyZj
z$6#$aX%mz5i*)}?Ux4~|h%FB_`W|l>k4bT&Y?U-*&eRVrvYna&is<CgD#ySXntuUD
zqo@m5!Ymi))1FeFF^U!1(g<YIEV;7+m&jGlmv&`#VJk8qD$zszB3$)d2+l2C6^~=h
zQw}7z*s78v%8%_E)K<(VqM#j*;Jg~G97E016pB8teSdcv`nOEnqlVZl$hM6ZQhB~l
z2inQJl%8cTxXWLgIkf>QJkGiF_)NNYaNwhe*|u!Gd)=rFw~8RLr#tJoJW4Rr(W`6`
zel2#@H2qd=T@L^D_q-dDAB9{yE38g+m|*c+Uh>xXk=m}MK@_9!^@$*kB%=15t8#n0
zhqm~p+DtjNw*JIvEV*w$NDUGr#<y?EDjYvy5lCaB#eMrqSkLpwz>4RSZT>#?;-)6^
z4lB|othxG0)`Q!>v7f7-+iozxas=`ekU<CH-Qw|EN>r7%EiyXx-$WW;E7Qg4ar&I^
zV4&D;9@SbfK?WTo$=#c@Xucg`dQ0!4WIFSiWoCbwTe8%u^~eBu_E+n<liPt7S61vo
zJUCO~Es$5K;M01(MU0OTlL3rTTS>`5%+gs%_T2R01fUI7$RYCI=fkd9J*sXMZIX6P
z%IE!EiN&S@`g|?4_=W5cI&_@;Fr>=r)HQcS$~SO|jXcX?Z<{`Np<qHqvK@$Hh-Qta
z6;^^5a0Di;bbq4J^v5yFv6|5>tK%Zq{%RaAQfeRPwMZ%Z+t>DNBh(cthcjl^Z!QQf
z7;{!bI)}-(YSG+&_}M?4BT}J;TxES@f62cySddvlQ_uxLf#UWbaU$US&o}|Hazi_B
zH;5en=_iwS7ZcyRY;t{7FYwP_JyNPalP?M2#b*a2H?0)|Wo!#)<cqB_OP@yrnq37*
z|KT$ZuFq7pDR4gR(rN%)1%CeykNpYvehdbROkMu%^dDLOPy89V{}pH@#Q%VvtLX`t
z{s-#(5%~WEoU1_Tze2$OH&dp3m`P1M7=EzfkoFHr{ISqe>J0~Z`wP}@eGwf1oFw@J
z9DhJ=lH6^*>EEK>a}5Kxj!4L72d08mHvu60(}^$`wa%M7;nmQak6HT*vMyru)V$73
zTS4$`_uP|pgWz>kp9>8FfB9?Tr)AvA7^xaoVs*QIG5y1^_wco#>N~|juzAKR@c#p@
z7u~7%1=t5tS5+mx>DP#oUwe$a1@uBav#+ijYxk?=-%}sK34nUeST$hSD@RiP3>S%?
z-Z*K1K$L4<WLuicuQ>ifUj}^x$aN2d(KkUk@+;^+w<mxKw6qIKogES3E7ivzA=b##
zy=o6A5~fz?UG9hewVe2$vYCyCoUWZe^hokw_TPJ8M)F}3kmCai{zU@vrd4;x)?dvX
zQUd$~RQQkIuVT}G3TMTTQ~oKJ70_n+50R|8!#}05V!p=z5X7p5EBr&Dt4{Fuy|`*O
zeLbduczr}VoG$M&(?gjGYbjXJ^_?HIMGlcTIVwbL=B?H*tQ#lrSBbBg9;%e<a~kc$
zDbC&2FJa$i_4k~^a?VPJx631kYOpSR6zW62yB)Y{2kSniU;T@oo|m&#FSO__{<&IR
zZsfyF|J-GDlMAnh;<8*o3*#8@IJ4c!4kVLuz3D5-_4Bpe$%ms4o)`4AAeMF^*Ibno
zKK1vVzhISRJ+U*PZpl9!z4wlRarjeJ;Cz+uKZS<LGHXW3;TsBWK_;z|OT_%uHp?=A
zAot*3l;fk3m4baZL~-;i)~P8Gd;0J)^t_<JmmfM5hAm7{&psN2gnko(J`Wkoi<215
zMImPwN*{#`BqT;{6s`LFT#$%ZXrMK*O`hRKYpJo%r68ZF8VfWOe?Y+^^xZ(TxVgPk
zIHRt-2<}ctzTVSc&E8b)96@`y2JnPxZr_(3ZfRi%r->6r0|`y2#v+Et-Lb+Rutlf!
z(dv21;Gjv}j#JCWl_BlEiqVU#2|Y7dL(AuG57apOV7q92-N3y=da*Vz&%Ucfzg#0E
z<_qBxO>O%z%l7ZE4w{DY+Kd}VL8~@lm}Mo~T=D@&rfTr&>D!O{Z?}wujB-A1Uu2z&
zWRORuqHzuS_n@(eBx9W!XFbA5)vXZA+;e62R#?68M)RHkNrQ;HFI5Z57Qf1nJy31#
z=`krXp_7%<mEjdi;TOM_nL5vuf0zkrarZ^QoMj6=cj2TQIxfP*W5nJ&?~GhA{~LM3
zjdPD-H{72X7<Z~aLa=(<TQ6o#iR~)SxI~|i+2KlA#Q(M)h&Ha$w&J8AyK#0xlbKIi
zutHBWZ%1+ycW{<wI8*r9f@RhIU3|p)l0i;*v$JHmjB0N*4eVEkM`&Gua<8>*-jAT|
zcBhonEBoZXg(hr5=5h~#MLGMCIWwGRo~RN4g)fx4-R5&=k0k`vI>+iQ0M<;Iwu1>B
zJ@C{0V%19w;_KNH<&v#?1K!VS!dp`CoU7RkYuoDj*>^|mzyg@$oWTbQ-+`Dp=PK^^
z@R6-~CS9%!e>5eNspxoine*D{Mxg8)>pDXomF;_TZ;TlDCAy&?A0m4>n_EQ5>Jxp#
zAojt`c7)vX1KK)LHCsqHe<gQQDb;%8r)Qr!J~dpE@u%Y4WlvLsD6quAB<(6zC}QDG
zWkI_~n*H5k`shQDRRN;?_^QzKF7FSs)_*D2u5|%IExq(*sNreG1)H7KKqt{C!+xU1
zIR_0Jvl0viG{mjPq_N5vHV9Vwa`;8T(mof841B$ydy53${s^;HOWL*~p)bRIm!sET
zwa5<*egNvi<!sr~237ybtg7JPp23F6J+8NyI#{zCKs3%s#+lSCH?n;19IZh=Z+Q9E
zJxLK$h_r3#8%sQHd(^(}S&$}Tvn+0TSZg<iRI3<WQBSCqvWXGlAkr6aws>XNwNkov
zc(Ya9O&RF`y>$O~m!I?2i{<;&myV9~XxRFA&V|Z+*>Y2pguk-tMn`2X8Q?M>an?kt
z>J(OJ*Gy||!ps-;6|`d#nwC-i+p6f4ScCXNe@=uW<;^s)ON(A8oeS(v@h6;J%}+;b
z>m>~)`thmWG@z1%>8aIK9bW6RN0+Q@^%Y1>BSzDpyPL-SQ>iRpF&n$b-z~-ayNG3l
zQ-TR?(+~W%keY)bOoixcr2Pwy?qQSYckZcS=8};1g)+>s*5erKU-PQ#wE)%V9&E$d
z+#nV`)TaS^;J$tr{<vg79^%yxUAlZJWP-<0>1jNP;8V<pJ&xb<yGKoGBYe80Byos(
zJ*AWVN%qursJ903i~{xCt*LGla+i8HbC)dQK~2#g{Mu?lG<;?fM(Jv5DewktZ%8Hj
zNf~kGZJl35kLAJxNs7GYX3H1b7x5=nO$h0;&D--t-qeX2QKQ6IZ9$w_M)@r<&CRmx
z6L!L-K6CalVpy;@meePSeVIcGoa7D;!0Vi2>tyS0EARNp;H~E8ox5XFCW1!4XkaR=
zIb4mN@AvC7glNIVF95{wp14)~i$rkL;qZ@I;^D)Is-;)P<|Y?rnjKK`MDT#f*6@hN
z3a1#cD=`nPv$}vO54BfDgr{h^p};H*txQ|GG~k6QYAGYu9XE9xbN=swMI;Kkkb&?r
z*oYftQe(^fyxQvh`XVGuvP7-dV`6gVemd>tv@RTBBpe|mtC+L1WE-|s^1#4bVzQ#d
zkW%zvsM@W>W9||L)>+((SS0lAlXpj?1L)r6m<x3mW2dlPLB6GbaFZ+P=|#ukXf?t~
zxmNrI%oe0BJomyHpA)*HKkdfz<OO~NCBZYvApYrqzRpG8XHNXZGVP20k_Pwup3yKh
zLb7FEC|j86tt#}y5}Y#Dt^68EtG%tf^=|_0`ywO@m$A%j5KCn%Iy-KJK1{49&GoOE
z%P*DrDmoK}O~r_X`%b_V>V;(Aq3%}^_7t|`L6{pk3qQ91D|iHO9$=3u(P}&P#~zOE
zahAoIaZoG9Sokxr{Ojl+W5hT<DP`Cq;R}K@&?3(AC>YF{y;0f`K>H8tNs19e$;TH@
z7@mte9DUfaDz-uYsNLs{YOAjG3)TBWoFhO5C$ujU=jXMn3^;@F(r(19DgyjyC1d_l
zu&-jr*pX@TYiYOyPESmVif4fQnU4tZXN+%00*=|?r&mMLKhdsXmZNN;X^k7;I!M^G
zsmpBY=j4r;JJZ^KtR-rjirSNHAO97!{#60}D{M_WL^fuwd1m4NVBOLGA21xytO07B
z|ILBs|JOzK?)}$n_-n25K?R`suj#Kl`t8d<p#Q1wZV64lMVy*FYGRIDlAI8|TI|a~
zoq4*2bCuAHc@0o&t7+wzVrAb>-1!>iG%mawM&MK1=QoO9p?qEY@16@gj{or-zi1nW
z>dB5ucnbS2VD_8p-;pXvOjvgp?Liw4SBBy5IhdO=Mx0L7F+XmDLFh$p8vSP#b5mAm
z2l6Snc~--j%CZuC=@dQpk3!4!YDA5pyZE$9+DzG%h-X9iZemsX-Xj-y%CdSk^l{v>
zR_UmYv;VdE6Q-2m(l3<UupEA)1b+@PO7U4;-W**HC=reO&Gfsjf=+C;s2FTC`DA_h
zNoI#*|ACq{QP49^zg0KWF9t8c2ll_7M)L@Y@LcEPT`bP4_pMZZYpLR1SgLh{fH>e9
zNxCe1l=(KG6LiqMc%Rpa$X(YmU$mO+*o9MhjpF)d85rjdlTD~x(mm^Am)|m`KfQtQ
zG1=Bxt1knT`<kCGTe-}a*;Q6yvXs3vB9$dAo>~R(XcnJ;{<fp`_TrqX+DtT6)jfU~
zK>VA;QF<ngyys$SOjaxGcKUR(1!YZ#{c?~!Nc<zfcmH4nT-%pS0w>Hx2W2EJ1}I7g
zR5moXe^2AeK7h8MHEh-$zh3iN(loXoiNczc-GLLSYM;$zUi{KK>T%=E9aDqTx-3uj
zPCXR{=OxozEaw}X+R>j_n%8AtOKUS(<+%O#^2Kh#w*E40;!?xjiK*E0$fpeX3Fimj
z<T=-0J}whW@m{itxSKbw$*cMNCJzfO`b_WE4eqqpM*MBjiSUQbb~}XoR1V+^_N%ND
zZd$&lLM+A2xsNdxh~*B*?f{Lub)6fJ{?akS*z44zaak{DTk>bp7?}3&!OG3zY+M^M
zY1l3yQ5^1fWTs^UUvu9fEV#}Yvi<?YoxxnO0)P)$?WIL3kIFA*hU}k~|DM@(xsiT+
zKy*6{N6(v{EpIjwhByhE=*qe9Qz+Bq+`g(}1lH3>W)bM8{n^EP@nFu6aV-sab^3Uw
z97nS+;}g6-v&;E_p)Ua%G*Ny%i$yKgJY2(d{oAa&;%V${ql4SBZredHSr2BUO}eA;
zo=?$iCBnJJ{#-{c%4VWp>jYqP!^N0(aPa1s6y0qU=?_17ryor}{D@M5i1i>;%z!7`
z2X;|>h+BLnr^d2PRFBzw-ys(AksGb2au|@JV(I82CeXQGj6$n*hAmKwf4k^^=QO3k
zBA>kTMNd6`cPb!`d*51RQ-@ET<jKkoy&YrlGNk#vQm{lRP^`F%o#`JWjHNwT_HxfP
z6a`>l$~LZfv6xRTJ;2i!EEOqkl{6R-{cZhumk@|DtTjpM5;Z#IxF)N|-A=c9@2#BZ
zdmrxI%DF%h<*$8lH-ewy8n%pW@?n!dt_cuxu`*rp%6q1s+jTc7<Q}_~Lyzx6EXshA
zhB$HSI=7*6HpYm#MtnP2w?Zkr|B%7`fX))LQ~_9=P=`Ak&&_fTZ+gxE+4#IYXmB&S
zbfwUD^?ftx2T&v#S*7zDG?%v8-J<taMgw(NtSk`Vdjd{eMXkV=FJ_g5CDys(?vTd^
zY2bsPNaQPTuN6`uBkLH40uVDJSG~9&uy)5Q->f~f!~^=yOmSevtFPYfr2(AIQ4K4)
zAHFHo^Q83Qwgu5mpvSr2GaXh4jFGGFc`GGf{X+-T9OJ%O`OGO3JPPR7a-+#zE40T^
z4R;oHbyHpJEI;(+pW9gj8k27<ez5xLu4jbR{S<$uMT8&f^wK$>IxVvebPD{7Jbvt_
zjyFO%^3ZnhMk*j+5Qx3$f4L|+CVy@F{P!(A?o~;c)&R~=nEj@)3UI#sd+9}9<s4#4
z0d(MMa`;b5=un!{U`heq^VOX9pLCho!i~kRwB7%kNt}B2oYoNx^y^ns&5C*bJ8@o<
zSlGiB0ooI*m010M(8`Aaq#+Gx4g9F3{)2KKU;N@XrVaG(R{_+2vJtgJ&KAT*pz;6T
zg>{{fHw~N^_`ff#^xE)+sp@RMkk@-`!Hg*$XBw(AtYC5R(eJM|C3>Tqf`Bfuf8|C>
z+s7>szzxC$A&40a#n*9f@R#d};ubpBuYdpgvfJXu@jnc5S<W-;(kteQYy3?s>28DD
zC?E9(({RVkKh9SS0=;-qdeu3KNuCNlMm6HwKwg}*1NX8n#$w`@0Y1b!9hT9-W~|UO
zqIsV9nh!BzZ1%_;Wu+zqTnm||)^zj|d&RQ<rV7){*HW@Dxw8+7G1;qJigW1j{qo$3
zR@T|1w!6dz8h5JuSxJkZ;p$$t8;HmdaUJB=Z`ezlSKVn@E>G+!_g{uBxt7-Yo4RIM
z)^D{6T|Md(?(F5F=M4Wov&Z)!JxW%OzOq^AfP2QBl8wgW+afq6Kurfk2*1=~NjbCJ
zX_-_xXn3DzN%rsT@e=-AH{pinEe<3LAGIVn+JeKgyU{9h?S$p_L@1{Y>iFe)tJF6S
zxPL{sxq9gB&a3`~Re1J9%dytIoxk0vopYlSwsR8fsi%t>LvQUlSq~Z#d{Ea`%);=o
zm;sb_83SG|ApqCc*c6k(80Ewr9QHm*j2Jl~1Anv2W10U^XYM_BmX^pEWrJL=4j`?%
zgO6H#YqXD((8kZ+M=Rl%hgDTq&H%(Tj;Q1;G>$4G=j~AN>l_A_EODA$74D146U(3Q
zjZ2n4OJ6SPVWlY-_@dz1dlhJJRET@B7@|K+WxK%`J?lLy?&?2e8<Gd|UlBxw#Hqkj
zRhomhPpI6b>6*>L%UcBk{LfpGgOB=sYaDwgsV^d>RlB&9%U>Gyt}+tZPnbPIc)HhY
zVsPN5aW0%l!fZE~kf!rDb4&?L({}zeJKfEZN8n=lL7E9|x|){wY&!1{&=f2$0#b$U
zy3Ml8c}Ov=ZlweQVk_<7b^{EcoNaYVkva@d3r_KKklWo{eyq-jecS6p+YOIugdfDy
zew>+cUflmSN>uOp^JzRs0?c^^4sZ5G7nSXMI)yW>X){YdX4D-}Cbw>eIJi(kf1%BL
zXhE58p1gj&&&=1dugb2Qd%s8S2*#P+1DxSzarkBBRSrB<H?9kpcHjy{=RRS(x1%x;
zmZt8(iF^#Arc5j~3`w5zpki7PYQClv6Fmg?8hxrgZxQpy>erP@N+hB#ovTUO+L}9Q
zlPy1QAW178J`vQ~vw@^nsT^q^H1(z4QJRx{$U%75+{ctFq}Y^KOEjP;t0hjREt{H5
zT)&(HWCe%E*Y*H~C~#mPK%X~=T25{wyP7(Q&CvU;oi`F~(xQuV>Bxju5p{*J<8M4O
z3$5>j4g^Y})hfw7P`mOsS226{*m;I$y}6#F3r$X!Nj}L)#<Y^lGbeo&sph7DqfzML
zJS-z|rk+&WuRa<=!(c-zJ)`i*=V<bWrA!tvB|>|J1X*#IoAAxDFz9*a>8c5x>#a|`
z<BCk>TS?1!AH~(}EPb;w*BqYo&>7stpXenbSpgQ>j9Qc|kvn=Y`^3lJ2CuM}6r;Dl
zYe^MNyFNQ44-UN)%B4eT3g~buT>_0@cuRtLgJAq(jSQ!zCDDbMKji)U-Xrlnj5nE(
z?qTyzT*CG4p_fSC(mjP(PgCl_7W9$sGu{N<;`UQ`-=;Uo4P}_l2kzo0C?o1=lHLRw
zSY4$&44K<@H+LQ6o2YZ&kNpnm@Uz(YDCH1hc)$3AUAgke>{9hzG*(%DOm39gJnDni
zXk$KFMlTj)MiW7qH8jSK6w?1x>_LdC*&D)6Yu(R;$B)7$O56EwvRerqy8I=gXtG82
zB2V|KO|t$83M#V;lqF{Hc4RaTITj~Dn^mT5b0KP6qzD93*X2zFBXvI2E~~pgDw|(S
zFzGC8YInawODgVyd3YT!V0i8(D|kntd?!3yDDI9eeWtR7{TX!IdcgW(#2n43Qj*Bj
z>@ddICzkb%7lEObrG`*&gz%)t%=ON}@2TmHh4aqp^`2QuD+Iwxq%q0TzlqW}Y5E#6
zBXsdgD6X+%7k|z17@^VP8bry8i~B;hR1ZEvvd4oT1@(E_O@FDMBM$MwiuIN`9C>#m
zGU3}2q!%iga_24o86))bvz+;f5?zL;>OSghSBW0Lgoj>8--}wF5o$lHH+%Q<()T!U
zW6v`W;((zHyJKLsT@||g*^m2qU<W$HX!c~!D2Kr3Nb(s#G-%;*ImscqWGuy}Ox;~*
zkEk`*hFrz9iAhn9;=_exA>x(W9nG_@-<fsKDpOM~FUw}l;Wn9duj_adVMCwN)-2Pi
z$}*;SuI8faJdJrHF!)JB=WFZiI6MP3D$I%_KWAVdFc<HM4zegV*os5Iw5@PHf8;3}
zc%v52Jn8w|q9m@^eEMY5i~P(fjrIf5Gx5`@n_jBEZRRF*MN(#O19rVRbXg&l51D18
zwi-(-FFm+PnrFo4Gxi<4-&!?WsX{wrY{Y~^0^IIRU%UG~GmHu+!*98G@JIZ3S7*{u
zOX-PVZYS6o50RVSU9p6p=HQ1Zm-OaKEtGjv(>&iAKN|jk3(<<E_{=1811ZSakB!h_
zjUAW1wtMhNS6G?c8F*hsdwCyTWjRAJhdDk=?@MLVpFV}5GjFr9_u6ocLc~3_R|A#o
zM}9Gl6Hx!5{zCT~cwZH>@iyi2a9yzog8My)a@>k?OwK#)&&Us80<VdKHWi(%*(Fs6
z$Tg$g@W4U<BalRIM6k&yE;)Bzmo}$6-B%yrF*|3F!(*19jHQD!huTup4kf2Yj0!G6
zH<c#jr0da)j3Kqv^f?Q061kxm0M5Xn#9}Kwz&TX)fMbSAyk#OkVx|?f+?q&k965Qm
ziHHBXUez8MKY`*;^AXoucGxNDu}R4)Lh=sAK#XG=7*iMXKb&mxnmvCKZ|rIH$hfkk
zU8_+gZMlp=&-VrQO<Pi2-5a{*CXglmkR)}_ZPrGA8Ut22x@+EsC@n*@SUw$|y*TGj
z<%$)cWt^Xra=mdMRpZ<kq(1`oieJ-%*3g}1=E_{2_l+HY@!}Ey|7)wS<A$K7$nW1o
zQHw`%zDLV9`i*|fURb=b*qOts1*g*ZGN~TMOihb>IyV4!|3<b}_6cnB0CReWrKH*2
zF(~F)RBgz7IaJCem_JY8krQ*-rMV7`bJ^2QP7yOLi3?5Bh{Sf42A?HNrUi=K+3sBn
zT_~0`u<qWO|6yH$*KEfXOA;11veZZ#4y9MU_Rf4ere#TZHe>C@*MJyv%#t!6G<gR8
zsdvj5G4I#785ogt)+TA##}SIX;>hlYcc%NbuXN6eyB8s!n6GMmM`1uOBjD9&n11FS
zpqApBJ++!e9|Z4PA{<1A^t_g4^~vje+cZt94;ZEHIWI_RuvIE0yxHcKz|WN7w}9bx
zAlV0tN&6ua9}9l_c5;52&zp8UOY6_~OC;TVE3Y?`aOH*}LH7(g&JQ`4mA*bDxQXtZ
z=-d#b+|TJ>P(dyPqk@-0_XC#{M!)?2dJC^EVp~i~*Ukb)bjd^}?TS*z%9hu36v3RO
z9b4qYZBR~65e#gBr@<ADS$Y=EyMQ?KHb<LPSfczBKr}DWFk+urqD3~zL=xJp>I#Wm
z&cQLVaIhmU`yQ`!U4X)pV3O&Q@tLs>5|aWq51JCZm<%9{Y^4bg)FTB_g8?2o`#Lq<
z1bKQ|EObCTH>6FW<k2j0DR><FMn~nHNqJ#^fkpfq6ebEWVmL9kPmr`oOeT$ayeri6
zD^>u6Fp}jrb#ISf<Mo>0>HcBLW#EwCs?J-cmp@J`3CR#<_7G~8vB>zc{!RIQ*(C4r
zQN4xwVTv1*Jx*>KHxd(1KH&3}qE_`JQz!7Db%mo~%`vm|P2rt|`#)dNF6%iy(O2p&
zPvELQ@Cq}xp?sE-WOF+`vP?f=99ALbBR&Li%ns<9VC>pRXLlf)`sEtc$8f8>{%RiR
z6iRD)9H{JMA6KauJvQAZy#2N=+V^Ob8ejohi)2Ey*Nws6)X{S;2LUA}9s)mSV<pbz
z>?gH8;8l0|xeVt%j}VtV8G5gMlS-?&W!l-Q*<@|G#^U|}qHblt0yS853P-PMsl@i<
zp7YrB^z}(AV9as(rOfinfweM=F9rYr!>?He&WE+%<~h+s=CCLZHc=HhlbA^HBOtC%
zH!ZNqvu)cAmVYyx_Vo-`Mz0D&?ap;;uTZ-1dkSxTS72|FqK&&U@y&JiP7VrTPTg7I
zTTeGn@_BV~1d3?*#QK{onz2m`9YanFa4<Q^%PvW;Y>fWlv$LHF&i8Nb@G&t@HK<v>
zmhYVIRSEA>A-)86a7Xf^P|DZFpq=yPqRPM@GZ=yze{>kmHO-MD(Y(K%me15$<ezD~
z9y7NkT5U+iNb^C8#1i{(YoKumL^p8kJY8q;YgFP!=@8e8+ezMeE&=mVR$x|5vNCVT
zf_HhB^7d;2r)dTSE!&hc$H*i7!8AnsC<ktlu*F{)wxPi$ElwrvR`Ic<GKU<$fX?W$
zc6UfR<B8=0*~vntg!6SM&B8?w@2O;?p)js>X=czELg7B!RPiAb;3}~15I`nuS!qcz
zO6d!4B!ve8w9d-dFy~(fLJN`oaOe7qSttL74sLi)MB+R05-i3R7hs$Bu-4>1{IovM
zst50ABoA`v=pa0zlSvbf7PVjVg5$5v7=p!5)3ge5iFtk+9sA=Y4hH6b|9&%cr>yr7
zO7`-d2<O&lvNiWn5@u;OozS@WDIH(Cx0X(oWy7C`QMu&=cKLAJlZnyDvC;T3j~k0?
z;`Ll|ONa~foNeG!z9%#<{MbE!k8s0FGiSYC?GRn2)V2Yt-%gk+uDP2#Sr^)Phhynp
zixGbiLaV(vT|5I*1<KK^N24e!==+O;7x0YIfI3fnKAUyH5X#?(hasVhp(FO_tr+Bo
zUjpIfWubH3?v)PXRyKkpNfmBjM7YyLi|o{U1T=o+9gC)@HAXgw;thY1JY74MC$r{>
zUkvc~M<|z=3<dU1<(p#%Pn?w9sR%U{l--m0J+sWuVtku&{Iv^))50W!I0+qJ+<5UY
z1<E{enz{R-V&vEya|ubc>I^y$E0Ys-mfHb)^BbgnE{cf@lzOp{y|*`{%yapw&+X46
zqUY6l=OD)^Q=+*T*cUbAo)rgoshT4M%AQ`TH+3#!yVCT$#^`a~%Fh^emjv-rf2xyW
z5bs5itu3Iq(>KfZed|*SD3k1fY-LzTHGAjj3+)L@#8PGR{T`Bt1J%Uwjx}uu4zk0K
z+#CxKaGe79G&_g))!K%&JN>`DxG_wX-OUviYm%ea9aD+CW}OR9hF}?*#87iOJ+yl$
zuf1^pS=r{E!GQ|x`nHprUfS1dXKb@>La5E<4Eyt)SNCJumN6?Vj1@bF@4nr@Y(lgy
zvp6Amb?IlH!r7w>)K<jOU+CI-vuyE1O9$bx{ivs6bVk{gBzR(%dIGETDZ$I@`s*n%
z(#<$z|4qJ+2ufZMIJCdG5unzN9Pzq#*Kp$<Rxhwx6!&5o$c<gG=hVMYI=RoPY#>aa
z<ZezB+V8;po=dtkon0ch1{4=V?t<IjFie!%tsK$Zk_?oZ?5aJd!v9J5<(}9qI}89L
zlbW0HJB*ZACtQ7*m2Q$IEaM<kW<VeaAR9T~jgoN#ap~SvQ0AR~P>#rthokQS(-&^J
zC6m$Myp87f<IwcKr>T}7T@X|t0vzT4h-D-0ib<LM7gprPzwlna4EX<;iTaNe?Ej(d
zR*B}nv1$Js2}|VMnD!s7;r~QnKkk(V*xv9%K$Gvxe<Ra7CNH<0+{QS;Ip|@sV(R~6
zUOf}4AzSG5gB1O@)ID%7rvjxbAHDsYXQ{fFK`e3mX;{wRS#jszU#{oPKN7Rtw{io>
zcC|y%`5xfgo`rN3NZ((UdUyut=gn>9?~AtHxY{yUA1QmD`RJ-Nf3a0Hhuu{ohdQec
zcHgc`-dd-<WD^&iQm}TFvtI)mp_fCs6UU)T@9ZS7{bBThP=FH{cz=TE6tVbP#wB8@
z9V$ZD=nq`wY9luuV++U*jX51$Z7SFafaZ#fM3h4!#!aCPja-HwR6<`3IONeT87CsO
zxkYTB{OWD8A4hs4LuZXr09<(N%NDn#jyS#3$JRxCU%el;dg%p}A&=00Zz!r;8}jwW
zMhQumkZ)U!#6GmUu%5Oc7N}UIdt>ejvHK){9dW6qL0>xJ`^y#vmWB_R3djw&mOUM?
zQa_1E>pWM=I4`KL^6TN`xq{%3v25wE*48TLhE!m5(d8GSN>k4VlZG8HKR>dresSfd
z+s`w{?-L1ohE|B=2-*Sy`c%-ZesLgC5;!6V4vyKk=htO~dWLJn4^KWLy5JG94F)NR
zf88(xy@&{G+SB~XS@!LZD|0_iFuUs2#^<#ND|^+}|C?V**M{U72`gXy0wBsnjizx(
zq70WL<GgZL&3XB|DF6FF3xF0^_4M@nj34g<;UZ8+qAxCUfC5qy%%1}jLMW7e%;I|w
zy=Tj1&QqK9p+7I@4IcUtsDq3_;v4pXB11a{V(`jm2qFtr4O0$I!Blvn1#&hmQ4}#|
zn7BaTa}xn_RW}iUt};#%2FR&Vr0+}?VJRz-#%ZECE|2H(#s|=RHd+$<#lL;v#f`P?
zL`l-k&DAPDt`Yvy>j#0}M>Q#*nV#rCH*;S};;R=ITLGHuHF)`T3&4u_DXY{ErM8H2
zJ4BU*WC`De2-P9e;j{*&=m=(kjB^BUE*JVGuoIFPKHZcK{Jk}epA-Cf6x&sLz~C)#
zXG}5|uyzR<-uJll^~tMsWjbMbxZU8bWB=@f7C%1APnGvol;?WA+s010s(vK1<%0&T
zF~W39j2LtHpZ6h;irNBKf!<8g2&n6WcQjOl_M&DaQ8lZq&I+N^2oU2g5(X<w^l<im
zHfAT+CuOB2ah+a?RiW)RlwF7Tm&=pyqa;+ACk$#+^44!@Ix-Zy$JZXoi$mJR<JXVO
zqfy+aVzVqP0TC+ycC~<Zyarrs!z$(U#1&Zz>dXA&lI5GAK|W%)?Bz(%F=eM~0)|4n
zMD-TGKHcd*njNeH<_t~)q>4l?z*R_sm(sxXO3_Y%B%^lXdJ>j6mc0)^+Y7Ttf*Dbk
z3O65%{*dUOH$UA?AzuY$xoZpUfg3Es^#H5<fVQu|FKuZ>|1B1LriH4IEPt4=oDL8;
z%q%y6n`N4d0cddU@Xg1n|6wp|l0od5#N9z2Z(&#}Mk%6k9OCC7VGzXVtBauA?my-h
zeA9kww1FV|A2&yjVY=2OiG2c}5^p+jT4=AsSO-z49;mJiWrqe5RbBcI<>w0J%T#&g
z3auIX`gbk5`IK(jD_e+)lne8SLu6O=FXaVCJ__y+1#IF_y3^C;NinXESd(`&w)tng
z`s?=VA2LwO6T4Zn8XW$*f3nlj>;Ee4%LAcK|9~f5qRo`ht?8f?I>ib#6mpd_p{dlS
zqS=Zj=cuTVBXUGchiDMF@0n^7RyoQorj;VsxHZi2e!epdO26Iref?E@zt8u1KKK0y
zXbOIle4rhN?hDDxJ7q3df%k<6AO5cgt1~GM=7QNhuY@;x9NY3EHbcT+EX0n?wShKZ
z>)xrovl9@&31)XB=5FwKu;dp&-cj-DhPYxJ)c*};FUD?}^{dB{u|+Sh&xML$#P=)_
zxFzp_5IVzIXk1vG9B(U`{PBuwnwEkUM5uN_bGteAC^$otM5-Cm`F2}&=hd|y10v;s
z!Z*!4JSZXAd-*Lqg*j*S&miT>TPRwcfTkZI&)Qs&QGjFa6T&RH%@?Nu$xo#I^ax<j
z2#*%RNZ?$0Q_vTi99s_wzC1S3STfBXkgBVbd@=x#h;u>0)l2;ScnM>6HC_39$TJ|f
zJI4nrn9|QzPTTm@x5<O+&*su8P=E^pO9zYLa<Djd6{xKOP+uCDS#=X?Wa3zsg4y3<
zBK%}LK03BWF2)8#PIIb*f-evM{~ttK3@evE@ctC0WoMEjh_(4|xXs##<|(Q*6jj7(
zaZq}Oo&iX_&0J7v_~Ru$DRR3O-uz63h=%3^At(l0G+2W(z)aTP{J)CI+PZw%gfkb%
zc4xOnA;Q(g7}MgX-2DK3t#gcE3U%ZHJMVOcbOxZHSAhS$6P^L;Hy3zfi@t>7+~6fh
z^~R#FC5*i^(TZcuM<s-?fEE0(f^(o81Zj7vlm7>MUFM%WK6B3VMBgr$+N@r{2f_xA
zW1D})f`Kv~opq_FfeinVWFO~~;Y4@LWLh+{a;O4YELCa+6!K_sqCapNg5<Z{PY5SI
zRZb&)fHX&kx|AQGQy`><7(4nvWIret{)vCv<na-d<YMeW9^?GH<N6-Y0E_e4r!EXu
znE@Mr8UyfnKO=v?a{zWPe*rr?IOvjJy|~{}kO8SJP&avP`{beO>gBIQqHh+u4|40r
zL3YA|2Y<ETf8EJ=8_+mYrychUm^pXB3KPNXq55+YnT?m1d!kRaj?oIgHHc483Tf!R
zC-SamfVT&$ErL7)M!R$Ij3us{1Fmz~wdgBi{pY{v47$I5hVBw~KB#VpVbf|CN?>7Z
z>uF%_h@Y9e9_WATG=c#}4lm&?&h0H4r>bcl-&OFDX?<F@`$}O%2uF{ldip@3kM6EL
z%gNXxrPlkYpxa#stALCIrwX^2kcgwb#Lbk8oU31kTo~QTp&WgcM7!=Q9aM|mSKdiW
zh*%D)`Hxt2JKX8|t$YH~<MJYl{3V0r`sM67_RB7bJOxHpMs|ml8r8Bi6i*{m(k~|Y
z*y7z`Uh=W|Tvp<X5WTG;jV_HBod>9{^!HCUdDvV`<UZ+w;ab&5ca5ePcEJk8J(yYM
zTw>-cf?ZV~M<kDiFdb`lh+%hRSMSS!V}g@nCLy9Sa1aDrh{btOv^;Ub?bAXK5#e0F
z7%O;2s2q^dP35ppVNTMU@{6%W57nW3d9ullkjK|mLYQ@$?h3G-p$5DGh^zeJqoDcz
zo+2!>7FCF>tP)w)I83LgF-GHzRp=qR*rXCt2a#1axPA9P>oShxs*%CWVI1RSK_h3-
zETp7<I7h;>u+g~=zj@f#ZPRU*u60Mk5i_fSXFHA@Q|g&Zx7g324m6yrK9hMqRW&y8
znUijJ{(wpezAn3!#W};ESALFrH0s7Fz+1_E{)9W4Zt2J7L{i<{`g&FE{F8|KWYh%r
z@lU^Lq3oQe@>ZP8Vp{B@pbMe~Zynyp4^Y{yzt2-Bxpq^7KG0*o=se+SWks=LV~$}(
zah_}A`M=Gm_BPqwm6tBX>g3AiygNYE&*~mdRwr_@-w3>Zr*_%Y#z?G@aKutu1@~d-
zdrXwF{dA}fzSR_bY#?cvT=+V<*-Jm=GtES=kELNwU-n$P@1+5=xUW>inYj|rB9s(U
z-cZE$`z#*X?kl!AuKLSkf^_ZY+v2X>qn~v`D>%aga)Pq+qeFr>+10O%O++{QXy%j+
zB<J9s^*rWOxvo+(5S7?$i|g20Jyf@Ki_=@9(b%rum20_7fVhf48{i*_fI>r0rN&@y
zeN&UT=L<Dyst<!s1<Tww=I&7XYtHSM%=5LgaMD#|n%T3GNBhu?QjZ(%g}?47vz6Rn
zMIt(DcJD}64$i^eqBv+35YpO`-*?qyW=Cge67jQzRaWKrRel~gePqYBBfGsNGfR7A
z#UA*A@&f10N*t^D{iTSZ;LFuq!8U=b;GE2y*U4pFPn|aDRjs2L+%<{Fd^jNM(Pp6p
z?YA2nODnA}Mw1)K#voR1Jx=@niP&vqQkksmoMuz;wg13C*FMu(AIg|LREyT{AwrXw
z_T;6!mNmtziq97>KzbSxqsOwiHd55-v_|{zvdDuCjdOc7Y7KW54~w3%+F#Z3G1dU{
znwI_k$*1yk#pn9%^;W?!ji7zxpZE-7&SF_xPTST>S6ek4_cQyAFehmgSHAI5G->7m
z!hoBWx<>er!w;V8Rj<;skC$S&Jxai*_E6&wkg&H%&tsbw7zLk7K0SE(j-02r9Qho<
zq&{!wel^-Uwv|c3ZzCTo>G93Ny7e(V{X6~eK;*Bv+FfojR*uT-dkp66m2tmfyaN|e
z<yr5OMHW=8!7`P7#RS_aD$!>H>C%7OQB|T1FdD*h;?7#MzaJPPQIPMoLlcvlFPri%
z5_36WaKY-)`i6{(vK5CYJt<to{RXx*^lJrr1~_+T-vk?Q8aE`-KB4CUv3D5nTwc3!
zEt$sPQc`wT<_Bmbs@)ZnuzlljL->d*tuj4V#sS*rou717pU`h?FtX(g{3E=tV+GA{
zM!gLff-e2<CSW<<r0;`hfG4Rg7DerL;=MOL6&htvki%O>y;7P?J`v4=p&a1I)2$sY
zm%N$|9Zzc{7Y=weUNAJN)Pm_5k1CQ@4dDp!h1=4TSnP_=aV3`0PN%gFe_d9XX*`K>
zCx-MOwwLdn%2s!qtuJ?o%>e;g@AmYxpE9;OTY0Z^UrcwRw}Un9kH-Zbm2{(INJQVP
zJDZlD<I>=kudFd_&CF6Al@4~`V(S+M!Sc1g->EK6O~K!&D9$_^?r{#j5$?S)L>1~^
z#qZ~oZ6|w+T_5Q7uB4B?$%qFEbCA$JYhhH8t7^}X8N4jjpw=6y*6~N=XA2oFm!CV@
z9}ZkjAPy7wT>is(#5lg2GTApnWjVNSoY+<L)Qz5K*4IVprE{a|8{BhTgmpq|PY2jD
z-iVL7zn7C_xBDC+utyl0%u>H(qO(pzo5i2a_-0TrWF0YV(?HBFh)9H5UIbHe{;QwO
zp{(Rb1d-ZV_1*$@^rN+RvA2lI2BDe6!*{UXJk$omC>3k2oiVcPD}3BxVV)NDc;MMi
zc56;k+!}2wdun@s+O5z2Ydz7rE2p=$vbI|9oAcqa00oZVn;)s6Ci3`Qt7)5cXX^I2
zkb9ekGQ?Q;53K>Ujj~7FwNw<`8hrepNe^}@u}ewc7x){)6t(1oN_=KYqDFzKJZAok
zPklm6(2rrgc&~4LLxFM4ua|hDWtY`vKDD{}xI5l{kYGCNjKz{RQJ6L}G6a1A>J7&V
zgv4l74<Q+zwkL2T@_dlC*fo1jTZRk^-?(y1PD0ONrB!)5=QO+#!iend*<*)kc{x`y
z8hy=n8tZPk@(_mXFxx{h0%n)kTz%Q?eem~tHOl+a{(Mk@y{^|2>~_nBj8~}`bk3nZ
zvl<Bz-}H8aY)EwXoL<jGvWM$3>P~xa$bY=fEoz^?r|X|_?>RN6+9@?6;Z?5Z2v;|-
zIF${KV41Xok}p-J?~X%X{Ghi>wwp5A4J1^8V{Zd%aLjX<0*=~>T=6um@`38#NP3$*
z4%-{zNnVFeMdHK0EpOb9yXAEzB7Mspr^J1shtlnG@Zo+Wc9Vp0oXwplXU)L?bGX2B
z3XvcK{CC4N{<i|gitbllcy7jew<!GwgRS8io?`hC>XJ#4<@ZKht8(HV;akfK8&<+Z
zU&!S&w%X2L*(iDBZFxpp<ECY@x@|SzGwecP#@@k)WmUO*ZF3F%pLxrv(^jSCJ#G;t
zF6us5A!1}#U&xa+e9Lp1;KhXbIn_@%dFcc}IUyo0_efFCj0t1(hZPj0T}@m|=Pgy`
zvZe)Xq#EG*N?0DSy?4yyUvMC+{J{Z5c|_9OdFSP3x}Vh6>KYxQb3<gZ-b&SMmfyw=
zzrPuG&nAqVJqlA>_4&R$V!c7&@^?Tigy$eWc6qAS=LS>Mg_j@sq<E$06GYaK!0A(L
z5!73SQo*;hoT`_(p^N=-=tr~BV3qSGl})wEdw@BvyasWy%M;!5mFq`L2`)@y2XprT
zH;laD#igb`4sJM769S<qMl0zT9yMa%7f9gpHhYW_)DH$e5~NGp3lXD<Xgc2;Uw#3j
zL6zGj=h0Tv4&c$B3Z8-)e;$tWj~1BbZMg;Re;K;26MF#e<bMcxm{0X=B<8+2PF4*R
zDhE$+YS<%qs%LZowIZZ_?X(81&&*HHjU}~b+x!eW%N3}bPe!zO3SpFnnEEZ!Zq3rb
z>O8Y=b;Q&_H)8NNqZBt+X@ct2g7`>oCt@msjrL@MQZH*W_w5&60~sKM1hW)T3v>qW
zOjVn#u_i-ANB_L_sV+Z%|BuP&i>$#KqjBn-s;&4qc<AYy1d$UT2!Y6shfeacn0+NM
zTmdP#_8Fy9Nnn!n{u~$NBg+>;GY&OYn$w2|_*j{6^7!*PxR%s(M;>pPMfjJ$*F$E4
zj&&tR_wz$Q{1Jahq;O(8U<V_0#Nt?pjI02b^5A*VQ*538495vNm=ZpjAchNL=IIv2
zX(TQm{VCX@{NW}@EFkVO)Y4CcRCr$KR1wFAS%|y`q&Q?jp8wC)A=vSL8a?Ney74kN
zqaRP39d-QDE*+sRjf|Nv5{I?|;n1L9y$eHir{(o`_r5NGh^(1f-Ta6gYP}Xhei;zx
z-N8H0yS==*Gdx^HT*GP+7GwLkL^}DYi7|>?vbPly!ZaQdESBdIzqc?3Y8i?7n>}_9
zXHTa7sdE;QKw*~R)K5TuMS{}fc?zQec@6cc!bYw%<^lTsO3wf$S7^uzVK%&*>PRbQ
ze;l8NFxWB-y%jiq0|$eig+9T_NW0h5;Y{~e`Wm)2s<q~hVdkm^DUP!aR8qvLu(NYS
z+A}r{x^TV&iT~b_X~!MP!6byb);kajMvs0->m*P|8%u~C84Z`}SNLS`_OZ!hh2o!|
zo?>rBbvT^|4%BDwG#MK}gm^4IK}HR^zuR843$i;^#$wSptS^nT*n|VpT_z<N%4kDs
z=9d_u9T%MKlzMY=oBlaiv#Q~;Rdoc7`Sm2`@w@)?)dx)kzook21d~swhq}gv51BO#
zF<M8kw=&|-*12*;l*JK222}nSTL^MfDE<|N0<PcA#gEPULj<*)zP`fCXSq6ipGfs6
zWLMYXSk~m$bn{rfN353YGb!EA`fGEbFU{VfrY;IY!jH-fD$A!E_{vhd33m0ZBzi;D
zSH*-)T<rEiU=N=k3`SmTQji?9$&JN)vlO!#%HoCzpB8c`-{fXK@%N(rzK~iFbKqm5
zg6Kt~UCI7+mtPZpf1RMxJ5uOI5nGYd`QeI{!)SbW=_+~EqYj>-_s8LBle`wR38$kB
zLMqa(z7AtE|G45|>9wZpJK<V2Vv@E3IFmhxdSB=2TwODG*y6O&*61#$oHWe>LRLbm
zN{1@$N%!+_hJQXif~z5!RSeZxKeKXZ)m*^U8RDMkyHi!G<$cWImuuJwzBwl}Bb1jm
zV{hsCeI7K{)?;=pn5|cJv0ndEZiY1b`TJy2(#6gXaja7%8LURB3|nV8Fs&g3epfu-
z573TGKIHPm{C(0z>_l`#k*S#I2>3kuenz6iQ=$YbVk5oPSny1)LCr>wG&@pFx>d&k
zE*FBLCNktXU%-Zds{W9l(gSj(2W^|r1|5&I?@%Sn*u!wJGZqgH+O0ZQ;xGB(vWivx
z<K{K5YRk-eFT@+AQxetiAkAWAOavivcPYdZ?fMSfjgM2U5uferHn}n1%d-qqUVM6n
zOdIoaP~Dt<AXIsUVZH6ml%dxh%SIWdf7+xP#8;bSQgc=$(MJ;LLUxbAb3x?LfJ5Eb
zR8AxfA#qwu!LiD<Vh6(t%1UFjDxL&9^bGiZ!72*x?<+h%N@ag$W?b90>VDSN_=clU
z0Jb%QnMmE9(FqL=r;u#v%3i4a30ns9YWol4=6;4o{B!_s%cCUP6iQQ1&BcbRN}PZp
zn|-3A-w7EXuQc10_4?yl7u>v?(dF15LA$^CaGEPy_-N0yqh$uqXhI8|^5U9GjnfVE
zKCn|9qP?v%O5<R<*~>~u;GY>;);^hIx9^Bttk$YjcE)9shRMQL@iHDuM)TF?m=GSh
zKmmZo?r+)#zc(j`SEfWr4Vf5r^>&nN6XODohewf|miuRxj<`bU*dy;J-@2nRMoau0
z?^kXG^O!u+Zbr@NoLessct%3wkQcFULxXp)Fh6E8;h>IM3f_z0nk6L?*z;7@Pyyte
z{lS^%hTH>ue>89Wv06E#*E8xT>4l%O9|bdZ3t_&wEt+csd8Y%syi;9g;CG+f;u_F$
z?8`GZ#+N!EhCRw`mVoT5B&$txqsMCHG4(K(3!kFOGvGi_MnHf(#;3VzTK{4gOt9Yc
zD%JwB3cUHbQ@sVg4&Nz;g>*I8AOEUL5sL<LH)GKqFvg;syTQYD-Cch3e;Enzf%6o`
zAN)Y=^aji%bvPxT%{Kxjp<{B|gKx7<!o&FR;whHRGq7x=iG9+LJk^oQhHU3pws3ko
z2zgo$?*v1Y#&U80kGlrtU4tX4z%slj^)Hr5K=-|v33V-b{iP5!+tq0Qp}9Mkew1s*
zBh@*JvE{r3#V?Rp;2E&|eAZRaB4Y&+N*gA!YJ7vp6r8Tf$=4tAz|+BU>c~NEI+ss6
zhk)0ox+h%EX_JYdV~Ism3a@m5A1|BgmH|sNjZ|mK31gsc2*XeDO!2iQXo9Z?xSrD&
zVGNYL)Ii=Bij(f}V}4Uoo)$-H%LNK2lI4)N7^2gtOhCn0xxv&=LH~iQJ3pbtjr*eX
z7BIUg1u%Zis-%rGvsR7*Zl-@0jAf5OU%Njc`B8f8PEQFrH@e3qh=<R8ZpF9%xZB<%
zy6M_sl&!=R14|Iew?A(LRwv|rUJ1^mJiA$bsyFN|1w$dbLS#IUehYX4lF)oFfmiSp
z4inTeAU!ro_s&8J5o*u#nH~W;`T=n1v0Q{k@Og+YZAEC@HznsigS%3M7?xX_)kvU$
zv1xg+c>P24#lL>rwOJU63cWy@X@Low#%$%OGLg4gaS*UjISeEWaTIAl>gTb%-TGdM
zr7%Gq=7jGN6~er@4Ek=|<{llJ1NhS)#<?7&2b1R-d?&>0ID5~48V>>6OyJa%5;+U-
zSX72BLCTa)4fxIGL7I<~iDd1+eDN9I2{h#r`Ggb5J8TTk`vc6V41Y|`VtkE2lnMw*
zpm?5A;WMi+4+G|ymhy+{093}8P)SH|a@eDw3xt1geR$hC2Y3e}hqZYA_m6AwdGE(<
zfrt0xheR`qi(U)h{x891z;Q_N$b=H&i(B4@A%L2g(;f%n{r;s=e#&mbQT{h{WT8{=
zG*$2%@A8xae{4LDGAZbb&@CY)jLC$HQzlUeKeIcN`<xpD<l5tt7I+5aI}Zb2jH6)c
z_#Xi-1*ky^;7f=Fgc1&pOQTgVOaERL^t;Z-oXox0nOq2`Mw*p_sM71K!N~U3a=kGh
zQ+!YmbJFc08Y{#d48H4t;q`@BKOs*v_G_rMpVA*-Qgo$#2!wZx<d6v#sEGA2+wi$7
zMgQXMmW6?uIus*e%rXWiXdOqR@3+94+7wf;JYfE&7glhK@M0|T?>+FwA1{S4v<*<?
zK{4mu7Vu)|F<Rj5H7G%mgWs-_uD}*uSj4+N0O99j;t(XWSZ4yd^|q=wvPuRhqGRH4
zBSe88q~9asNw6yX`Z)_&BNei-$TN$PN8tbP;|@b%-wN=q&pilI#9R?@58ctUr=9^Y
z1pnJW75D<kgSBqRU&n6M^&a2#i<U{_@lcW66G*`3`?54JEz8#<2Yd0RetpF}ALyS(
zl`lm;OfiKSKqi=$ibb*>E7o$a%~g2s<|Xg3#c{@Sap3vQhv8i~Q-G!x@@;otF^Rol
zd8os}W9KDl>BFv3s2C}^Yy5XWz@bn<5|f_fzFX|$vs(8kD!~)kF-Uf3N@kpq^>Bgi
ztYM`-JOasMp9}V(rQ}i70P!$SkVvtSm+PyJ2s*wGiW*qNfM6hiuNN*E4w68fgYELb
zwZ-wmLXTo$_jLI19}mgs6m>4bY}2v?U$xABaRk1eiUv^pC5B}`V=9AN^%*v(aKLib
z@jiiL5NdEYhe{26AQU@s4)b13nDlD7v7qB^5c~jmn@Jx6y$yHg=1qE#i&EXAla>Vx
z_H{yeA@MAx#hE(nj9U*{2*9QA!$TL_$+2udaV)en-z5b?%@^!%IsE~A)V&GWJ#Xcv
zRbKMfpb32sls>qd-x9`11CMPyt!@9HeQu&hz+V;YUPCD0M`E}(v^wdEbMsO5AXvg?
zZ}MQS)waU4bO03u1TQdVMf;;K+7DbN!W3V)ubE4^hXR2{4;Qap4}LTb`r!lgcK@^1
zBO1o>3L${TLaE7+k%7u!=0=YP9$PAA{>dGRf_eZE+!vcc1>3yR5aCX6LA|)2clbmS
z5#~w>RfxPD_0Xts4sE#*xRKm|S6jQ(nxwyoNAt&aT;6+BkA+%-Ojp?=ZLL>NHNQxL
zr}JA57vHVy+QrEY?GQN#I%JD*ux9|$1I*dW)`M7BvT)zo_PH;<!(<6}rKRWIv+av9
zUWtTu&wwjbE16_kWQ^LLoyfkRAjR>`_Nd<YBZ7B0Y;R@+*jm;SEmf5w?!7FyW#!Vg
z>gYxZtm^8D8GUz+yE4htFR_Z_2kJbLA3ic;3o@7Gm>e_O$|wcBt@?CO%fU?A1`m6v
zKMR`;0@K$j!OTbv^#+;mGVqV|cU3zY`yblHim%z^vie&M0ZwjbRhC8OsU12I<mlSH
zE1PI0CYzJt9GTe?ol!+IKS8#@zopeZOm5y+bvmrFD>JS&U1?P+X!@|(I6<Yoi-2xT
zfF`dCN*qIJ!bBSZRL4^p_t#{n<ysM~vEFD3y63_$=R$L?XtRL(KIw0io@C9w(0ESG
zYl3e=L-z{L02&UN8x&$;QvN(oGjL>nP!21`-+cJWTkt}TiKjh(ID3I>WF@vJ^UREO
zG9D|aBVcWv6qBRy#_XVS&W0Nodih|iq9hHvjR+dX1rqDpr3R!ABz*c5l-g)QN<I`@
zl5xbaAV9S+9-7BD8aeBfuSTf$6QodkfJ|u_BTBHOgk0YElDFu$v{+AQwBiRr?S(qF
zCsrEVXEo0KTWwc5vvTE9%JWas-7ronsezM7){8U9tzoV3jj=P)>+&W_CaJNRB6^n3
zpVs5!zPUVAUX?n5341;!(!>nHYwp!%+*?w(Z#FSh-G!juXkpeo&$Wu)d}43ApRa0H
z=^;U~^HHl&*Ic97tHdO0GDh_npYNQtZoG7QOcQKbmfMyX<A%BoeVZto5`(N<-8MbT
z4O(I=Zj)Y&&PyX*i4|I82RG(rHIss5#0Ts$iEbua*F1^KPtFQB?%(ykJIA4lmDsMh
zR0y+>PgRd{$EoUptQy&>^*kD<bw)<eSnoNj%&6e1gVX@Km0avZyFA)TsKdLNJM5bu
zTrY1WWacLMy5p>c;5Vn;e{eSlA{6O3U~9^HAj1LZRm3DWq+qRXA7SU<%wyapotW#v
zKI=w&*B~ReFSSUgtBYgfy8paIjFMSUDJBx<R<pyL-q4MVTF9s}Ze&zIS8G~g9P2B|
zES;J2d8^L28v1)<EOP*aS;GjustmD0CPC)iWo;KKv~aFqKswv;+-*|Wxj3me6u7_S
zvow=DSfkh0A=RZ9G)U<>+S-Nb&8So<%cM!1Z5Gh3TDjDu%hX}?Sy^uwgfe+@q;mp5
zI~wF4(pRV?S4T@_Ds1(qN8OTsLv9__&VM6;%F)eh&~Izdtsc$D{pJ^Rx02%LgMq@B
z68{D<PE)3y5#cRVA&`|%wMQv~JI$4swD1ic_n&j=`-w!zDbxyNDwF#k7pV6x(RvBH
z(zJKa=<Z{Fw_B%&KSYiQGh-!ooy<4Lb=~^N4zrFT{6x=?;A?NC-@uAtrWoNFXJqca
z<;t>fuoUFL98e2{K#Hl;aq~GmNeBaJBC>}gzsp^VGuCr!*sZoRd8pu4CA~vvy-CoJ
zT+pCHLzghdj<#x1=5VVi2M6UUr8~qfG_57e*i?!skF4;oLu>+U1h(k$I=C#OMjtHY
zH*#;|$&#qgFtmy{p@_YgnIWum)^LSRexrubuH!?Z^q5@9di}PJsG#@hjU5z~b?MJ2
z&wA<M(e-wAi492(qp{-vw>hqk>j!^^c85sXz>xByD0F<L=nI@YF7b6-Nb;r<EeWTN
zM%yLN6a`gO>x9d_)LLp}pvD>?1#{TUR1&Kb=YmXXDCwh#oWW?gU4BQ7jT$-6U}=^q
ze!i`C$3>&>vTKvf9(jc;hsQ-ex0~zwLk3o>p-k-MEW|Ae5!*)TCL6f6XOiGHpSkHh
z6;jHC?%kSEvG^wy%IX_#JdjwvJ7Jt9xkHs+mRB<@7wV!wmYq8@L)~YoNGS7hq%wg`
zB|p-<k4yro#*b#;Ka>e8{W0jW=_%@5^2@)*wFj@0YW_y{ZV@4j1>)QVhY+r5bza)L
zijh^Q(ye{^{)IPo<mQ`Y+ewVK{a<=Ve%m%upHzT`R*#>Lf_XMNBR&iTglw7uSB-3=
z9C}348p?v16=Z6djN4&-fR{2<=w?a(Z7sRhF<QfCgG;2G)4b~@Jju?3G}rj4J#Zwe
zT;NTOPTGUs!^OqWy3|?v0pML~yilVbN<JB8Ef;8O84Kp&VtqEhI_6+89`rwdq1j7X
z4r$97`8M#82#qDNFvZD7RA2G<g|76PM7JC(1=r=S{Ao#Cqbb7>w7Y^=)3@Y)XG*Jc
zQ?Sgn&X3ubIMDdipVYE7R*PcPOw|v?3Ra8HiEp@uEwV?DrH@Rj;_vo3w5?+w5j=M9
zP3aG__G7RC2Tdy_uu<P%OCQ}#B__6P3=CR#SyK@2yI21B^Bma?9=gc*wvN}xC@!uQ
z_Us8HeBXq9UPoKQkSpH4p40AS;L^-#^;vyni6it#vDN!>iE@sC8uZl_%Gv{di&Qs@
z<$Jtr&v<~|2L7A-Gf`v|7w<$N`6~O>dW@;FziJc*t>-i&BNcS?VMI*BGc+;|hB33R
zUqUNtJ%hIHh+>6+x42;%S|CAozG>!c&wy`WTiZq=)4#aZ>hr43L?~<t)6`*;Ta1f^
zk!CO4(|rL5aq0NGw5-Kp)J7#1$=+?7`{e9D9{AgP#U~$d#a&ou01UC(v*cVX`<@0(
z;L;GnGWpvyvvk=wwRI-+ITcthaow~(Up1~SS~GbQD$uwIw*@?aknoG@eu8K1Y0uqO
zeIE<e*uAZiFa(7Y%j7IwBpdMOF2NMxnbpbWGai>jD-Pa|!xka;0A*D5-Po0Pe6Zni
zsTzm*IOL%tGL79yBR}gmZiZ~v%_sF|%wvxQ)JqPfUST{L((JZd;hoW{E#lm{jXGRp
z7NspnE_0;y<Z(JSii{3*q?hex9H%+8miW1THF&a0QxF5K7y`D+M6;saMXc48^pLi>
zu~8e9ton?-#I>@P&#LpB*Ctg9{$Pr%opg)B!eLSRm1cZN!hr}c!iQ~}$**ptC`%fa
zdHKvyhXK<eNgI>ZVh%!e*x%4>HF`&0=>&Q|T&NasVU6`HR6}7mn3Q{1(u-J5ma;co
zk~GvLG-!+Zp8@O<00@be>y>|bw#bCw_MJp+RiV#okTCe0Wwo$(+iqfwdiw=8rSQr|
zw5eiG!xvJhlYU~R?{`)&GM_hXdy}LY&ULUENY|X5*u}mwKGWWms`OUk_xA1ubuF21
zTv`S)*UdeF+$FJ5a;XujR`s&kEw`23(9|rxl$962G3kq3l&{lrC=k!Zh>&QtUd2#i
zKxN}<897~Jz3O19(c?t-hvb~xmz65jLNm!3l$Q1K+8Oc$%<Fp7Oto<D?N&oKW#bcB
z5x|l@lI+2jJ-3l7<xcfq6)mkQlN5L!nU>YN#jB~e*MB{2T}9?An?o8ttDQ27wl3<P
zv)A<aiQJ9#x_)+njrKS9H%9&z6W8<Jc%61~v#zD>Q<?h8rH|0>{~#W==or#8TKKxw
zO3I`l6fV;FNMx<_At`phEs)>c>{-&b`ZjtSiRmxub~iv<+Tp0dwn9?wXo@BUi@!4$
z*}-wRKL{FxB%KJi_S0uQ6uh=~vvfwd-g|p9t*Vn&I~%Le@!<)(^qP(0q=u}zqqnFp
z#vY)HJR$sLZ1<NGjCNj`id=nyYH5YLP{o(Xy4ehH(ghz;A;OgKN&_EMsTW{(0=Lug
z{D|w@RgyH>LuR&Qqd|ow7Ev+pB9}UhzVKd@k*CO3vKqcAt9<^k{MUpa;~BRQhYW>!
z=(%7M3=Q~8V$X(at8=$hH0&@Tc<vC}028Hi@*?+DsmRMoRW@2}HH9hs#Ds`SHqViM
zk^E}zZ1j8?-2nx?vw<J~FvRV=lk|Q^FWpH~D?-3qUdA|myV;%9!|b*rOX46g3twuZ
zTP0CO6A1ZVoALu^*X4A&w#L&PzqKD+8w8)#=(WPwd5dOCu{&I(;HvO+Up><7W=Z9i
z#zofM{tzaF72MfPjBw|n^-|?0b=DLPe;R+)?U4)8HYeSt7xdLqloNy9=Ph==UR*e?
ziTzmN8M=8`GZgPzmu#(h3Sv>np;=F)%1-8{{9akK4h~?#%wDF4H!0rPYv$Id7sSaa
zY>(VbZZ&|)#e{sj`WCgq==LyPFk+-%^eeS~)Sx)zwPa1z!~IgnA7zr6k%D$oZ<VCk
zao_rPyNvGWZCp!P{fSs(e6sE}WSd6UntXcRTBGuY>*%A9B`Q6_b+@y#5*O^;HcvEP
z&y>)rv3dA4j`3)pRb2NnvNOA!pff*N&1Gw}oF0B7H)MbsXqA3Tp11SfAIu9@E6<)>
zB;^#LEB&_2vGmdry^xz>-y`A#_C%DSlLymWn}BBCa_<8gI$Xav0fHuQ<_BtO$Ju$h
z4~j4Iw?6li7I%y$1S(pj-;#2NPNuByEp%{rdPh5~ZH6LuK<2wCS4GZ*&6oW>E8JF-
zzYCs|vL|({XC8%`^HusSaigeB%F3~|mOK|2TWd%MR3K_GJV0W+;^@mDa|{B$pLUhp
zt}(XWocQtexcUUO;e|n%nyjczpr4!h_4RCXTA}?rFIw-);N-&ZpK}ld&NrRGH%uSW
z!9{l`Z6O|zDqpvfWvZ*gOr`jL3pbwe`h@@ZZW6|KgAAFRfqH-fxU|sJt{NkOOH6Os
zh-%C^!`Wybl{aH3elEm=6K|9x%3Xlz!4TbcRqEUBf%5<S!e@BlW{}K|`N5SV@qtd1
zf>odWJGcaVcs4IMz-!k>?U~{BiYY-HZp7@mKQz*meCml*r&Z87^Njln+A74S(bc%%
z%6{3QB}7ZJo6GKbyp(ZYJ25j7!Vhqpg<9ymv@atcj@Bq=Bvg9jpSvhq67oTc+$!>4
zzU`8(`#=_VrMknZFeEtqIYBS+RNck;b%V@@fk%iYhGm(x*8c-nNh1x?H!S-HYKHt|
zH|;!554S8y`I4sH&QM>U&W?+Y!vAl)=}MJ{l)2FThbO&l`~@0#F|d0vYLjuU%fIb_
zgRBux9j(f@E;*oAn=4h5Vyk3=#G%y<%Oqvo^2wJSeAN=;)qQne9TVew9$*TgP_w_!
z>nE|_1z}bDzDstjv@ku}Oxu|qa;k*<H*{-Y|I_=Lrv&U6<z7jhlU_d)RM~&?z*&}S
z^RIF4@&w>6=`yg%8G-xv{{;8zi;!J=GGN!t!C?=}p{JL)Z~r%}jTuX*7cYRI*Zz_|
zkB!r3ByuyCEB9g|6lzHru-Rd$@=N}lp%CMT$De*5jH&+;G?}pZQ7_bs{e7ZeRce^2
zb%jvL99O=@nCWEH)u*jO<Fh4S;6PY&EgVR)WR_F6BixdMHT1mku(>AUY^XsyL)`iH
z80yBuXCT>@7aN4JPLAIAaFq6M1b-z(Q&4*@yi=<MT&BipFUHi6hI0Om%c3dm0T2R&
z_qdW$G7$JnxXj6lUYlYlI+R$Pz8U7=MMtR_8+Ifen6%5L4PMeN9sz4F-9Z!}_nSAj
zUCsCp`~i<c0NNmo^QoV<yzd^J)jN5QaZo<A8evHUE@m}cNjUxn5jUCsnd9^ijnY5R
zFZViT6rf({-X4pe2_DjlKQIjch3h2e14TFWT~2Y&&*7clgaU&AJP{!LfLjR80^CSj
zaB`wHC37WNe@{NiG%}5Qkni(SNw&Cur)TcNpQPW0x(+RF4*W^HLOd3JK)|cq+f$~(
zZm^h!nE1z<x$SC{Gjx8Tk#&9)HV{ca{*x-Cu}We~RoSJtr(uAZDEHq|bskBa^?_;$
z9*-FUpwBCI4Nj3vN>_T0X@Pu%netU~qzTau_mzbq*D(L{=r^8DP(~<+SpRQw8QCa=
z{3Yyox`4)h`vFVoNvIi|BvPjl$iQ}gz21kGs|WoI@N`RYefI&^FbzMAa?J-wCM|~~
z#-9~)cnXvbn{)Z<Ow1$Lnv=bVH7!R%WMgI6slQK^nE-1AUCUYgGg*+_oQkSwtadI<
zuz7ke?mHfjsJb5{3Ib43iM?4iGr2mRiqN_LZg$~hJr&2bC_vbH<M44-s^nIS&aGTK
z3%G9!WFveOctU>k<Rl^es);A#y{X&Rj`7qaS@YwTX{v*_14L9uf+u6qF3^d6Q2QAb
zhQ$Bki{}CNa?X|6|95XWpAyhpAZ=o^M20+QwtmoM%qaO0`tJ->gR-K9e)~bm{)AH`
z%ZzeA4q2mdJX0}<ZN)JSMXaz7JIMmK#tf4mpv@tYu85P1uFjl`Z6*!6>efUHtsWD`
zKLE(ZOZR~`GPZM)U5C)*f&<9M4}#jjZH8}_BJu@U>MfdUn9V@FxB<7~?F&He2i&qn
z?Re8*uhE#9jU1QP;(C~lw78;Bh#fjy#J5Aor2su--i|uIDdLb*g4-QotIai8_>a9u
z9?L@PNA9mc&eA3f)^T~o0Y}&Ohc{D89Z5K5s}7F(-GJy;|CCWiXWc7MCiackFRyvq
zevm6~+zFJgQP;`E{VV!QNsRu2eek3W!$8uN-E5VVyS6Sv)qLey?uUC4&1kGQl`W1T
zjBivNbEHbSutYL4=)4(;`TeSo>CI)_r`|U1OZrGO+^~!}K$B<F;t77CYlSh+W_pNH
zftwMb@{w5&uR@lQ`-?5>aDtAl1{_wiS<>&WY0P>K({23Tc16fJn<XN@l^K6J8Zuda
z@%oaoRBkd4mYBa$&LfXyefo9?!KH03f;RYZ{|6f1oQdrhO7gm_PkX#pOC`p-Rkv=%
z$08{@__?uI0^2?6iMRL~s&;w57IXV3q5ZV$s(yv>Z)?G~ui`i%r{tKa+X44=R3o*H
z{xBT-FX+P=TSHU=yXqrcT!m*%egvjca6jRWfdUV8GndcXOcToaJK!auQ7;}qDX_Yv
z@*hh#nc9r6zGVF42cfWq@%K(#u5M#lCyfk*cY7RO0X2Yz%(aI+YCU)^8Vbe8@7OtP
p8NNZ|b_9YUlm6#hDPPaAIq4$}3IaQp97A@ay-!~w5qs>~{{d1>&&2=$
literal 0
HcmV?d00001
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.
rst
index ed7f770..008a43b 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -55,6 +55,7 @@ Programmer's Guide
pdump_lib
multi_proc_support
kernel_nic_interface
+ virtio_user_as_exception_path
thread_safety_dpdk_functions
qos_framework
power_man
diff --git a/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
b/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
new file mode 100644
index 0000000..5cfeb10
--- /dev/null
+++ b/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
@@ -0,0 +1,104 @@
+.. BSD LICENSE
+ Copyright(c) 2016 Intel Corporation. All rights reserved.
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+.. _virtio_user:
+
+Virtio_user as Exceptional Path
+===============================
+
+The virtual device, virtio_user, was originally introduced with vhost-user
backend, as a high performance solution for IPC (Inter-Process
Communication) and user space container networking.
+
+Virtio_user with vhost-kernel backend is a solution for exceptional path,
such as KNI which exchanges packets with kernel networking stack. This
solution is very promising in:
+
+* Maintenance: vhost and vhost-net (kernel) is upstreamed and
extensively used kernel module.
+
+* Features: vhost-net is born to be a networking solution, which has
lots of networking related featuers, like multi queue, tso, multi-seg mbuf,
etc.
+
+* Performance: similar to KNI, this solution would use one or more
kthreads to send/receive packets from user space DPDK applications, which
has little impact on user space polling thread (except that it might enter
into kernel space to wake up those kthreads if necessary).
+
+The overview of an application using virtio_user as exceptional path is
shown in :numref:`figure_virtio_user_as_exceptional_path`.
+
+.. _figure_virtio_user_as_exceptional_path:
+
+.. figure:: img/virtio_user_as_exceptional_path.*
+
+ Overview of a DPDK app using virtio_user as excpetional path
+
+
+Sample Usage
+------------
+
+As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen
before compiling the kernel and those kernel modules should be inserted.
+
+#. Compile the DPDK and bind a physical NIC (for communicating with
outside) to igb_uio/uio_pci_generic/vfio-pci.
+
+#. Run testpmd:
+
+ .. code-block:: console
+
+ examples/kni/build/app/kni -c -0xf0 -n 4 -- -p 0x3 -P
--config="(0,4,6),(1,5,7)"
Is this correct? The section says "run testpmd" but the first thing you are
doing here is run the KNI example app.
Aws.
^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
2017-01-22 0:46 ` Aws Ismail
@ 2017-01-22 7:16 ` Tan, Jianfeng
0 siblings, 0 replies; 73+ messages in thread
From: Tan, Jianfeng @ 2017-01-22 7:16 UTC (permalink / raw)
To: Aws Ismail; +Cc: DPDK, cunming.liang, yuanhan.liu, Ferruh Yigit
On 1/22/2017 8:46 AM, Aws Ismail wrote:
>
>
> On Jan 13, 2017 9:42 AM, "Jianfeng Tan" <jianfeng.tan@intel.com
> <mailto:jianfeng.tan@intel.com>> wrote:
>
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com
> <mailto:jianfeng.tan@intel.com>>
> ---
> .../img/virtio_user_as_excpetional_path.png | Bin 0 ->
> 38600 bytes
> doc/guides/prog_guide/index.rst | 1 +
> .../prog_guide/virtio_user_as_exceptional_path.rst | 104
> +++++++++++++++++++++
> 3 files changed, 105 insertions(+)
> create mode 100644
> doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
> create mode 100644
> doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
>
> [...]
> +Sample Usage
> +------------
> +
> +As a prerequisite, the vhost/vhost-net kernel CONFIG should be
> chosen before compiling the kernel and those kernel modules should
> be inserted.
> +
> +#. Compile the DPDK and bind a physical NIC (for communicating
> with outside) to igb_uio/uio_pci_generic/vfio-pci.
> +
> +#. Run testpmd:
> +
> + .. code-block:: console
> +
> + examples/kni/build/app/kni -c -0xf0 -n 4 -- -p 0x3 -P
> --config="(0,4,6),(1,5,7)"
>
>
> Is this correct? The section says "run testpmd" but the first thing
> you are doing here is run the KNI example app.
Oops, forget to delete this line.
Thanks,
Jianfeng
^ permalink raw reply [flat|nested] 73+ messages in thread
end of thread, other threads:[~2017-01-22 7:17 UTC | newest]
Thread overview: 73+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-02 14:31 [PATCH 0/3] virtio_user as an alternative exception path Jianfeng Tan
2016-12-02 14:31 ` [PATCH 1/3] net/virtio_user: add vhost layer Jianfeng Tan
2016-12-08 7:21 ` Yuanhan Liu
2016-12-02 14:31 ` [PATCH 2/3] net/virtio_user: add vhost kernel support Jianfeng Tan
2016-12-02 14:31 ` [PATCH 3/3] net/virtio_user: fix wrongly set features Jianfeng Tan
2016-12-08 8:32 ` Yuanhan Liu
2016-12-02 14:44 ` [PATCH 0/3] virtio_user as an alternative exception path Thomas Monjalon
2016-12-23 7:14 ` [PATCH v2 0/7] " Jianfeng Tan
2016-12-23 7:14 ` [PATCH v2 1/7] net/virtio_user: fix wrongly set features Jianfeng Tan
2016-12-23 7:14 ` [PATCH v2 2/7] net/virtio_user: postpone DRIVER OK notification Jianfeng Tan
2016-12-26 6:27 ` Yuanhan Liu
2016-12-26 6:55 ` Tan, Jianfeng
2016-12-26 8:02 ` Yuanhan Liu
2016-12-26 8:26 ` Tan, Jianfeng
2016-12-23 7:14 ` [PATCH v2 3/7] net/virtio_user: move vhost user specific code Jianfeng Tan
2016-12-26 6:28 ` Yuanhan Liu
2016-12-26 6:58 ` Tan, Jianfeng
2016-12-26 7:57 ` Yuanhan Liu
2016-12-29 9:40 ` Tan, Jianfeng
2016-12-23 7:14 ` [PATCH v2 4/7] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
2016-12-26 6:41 ` Yuanhan Liu
2016-12-23 7:14 ` [PATCH v2 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
2016-12-26 7:44 ` Yuanhan Liu
2017-01-04 7:22 ` Tan, Jianfeng
2017-01-04 7:46 ` Yuanhan Liu
2017-01-09 4:51 ` Jason Wang
2017-01-09 4:39 ` Jason Wang
2017-01-10 6:11 ` Tan, Jianfeng
2017-01-10 8:46 ` Thomas Monjalon
2017-01-10 9:11 ` Tan, Jianfeng
2017-01-11 2:42 ` Jason Wang
2017-01-11 3:13 ` Tan, Jianfeng
2017-01-11 3:23 ` Jason Wang
2017-01-12 9:40 ` Tan, Jianfeng
2017-01-13 2:27 ` Jason Wang
2017-01-11 2:30 ` Tan, Jianfeng
2017-01-11 2:45 ` Jason Wang
2016-12-23 7:14 ` [PATCH v2 6/7] net/virtio_user: enable offloading Jianfeng Tan
2016-12-26 7:53 ` Yuanhan Liu
2016-12-23 7:14 ` [PATCH v2 7/7] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 0/7] virtio_user as an alternative exception path Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 1/7] net/virtio_user: fix wrongly set features Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 2/7] net/virtio_user: fix not properly reset device Jianfeng Tan
2017-01-04 5:46 ` Yuanhan Liu
2017-01-04 3:59 ` [PATCH v3 3/7] net/virtio_user: move vhost user specific code Jianfeng Tan
2017-01-04 6:02 ` Yuanhan Liu
2017-01-04 6:46 ` Tan, Jianfeng
2017-01-04 7:08 ` Yuanhan Liu
2017-01-04 3:59 ` [PATCH v3 4/7] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
2017-01-04 6:11 ` Yuanhan Liu
2017-01-04 3:59 ` [PATCH v3 5/7] net/virtio_user: add vhost kernel support Jianfeng Tan
2017-01-04 6:13 ` Yuanhan Liu
2017-01-04 3:59 ` [PATCH v3 6/7] net/virtio_user: enable offloading Jianfeng Tan
2017-01-04 3:59 ` [PATCH v3 7/7] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
2017-01-09 14:06 ` [PATCH v3 0/7] virtio_user as an alternative exception path Bruce Richardson
2017-01-10 8:46 ` Tan, Jianfeng
2017-01-13 12:18 ` [PATCH v4 0/8] " Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 1/8] net/virtio_user: fix wrongly get/set features Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 2/8] net/virtio_user: fix not properly reset device Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 3/8] net/virtio_user: move vhost user specific code Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 4/8] net/virtio_user: abstract virtio user backend ops Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 5/8] net/virtio_user: add vhost kernel support Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 6/8] net/virtio_user: enable offloading Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 7/8] net/virtio_user: enable multiqueue with vhost kernel Jianfeng Tan
2017-01-13 12:18 ` [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path Jianfeng Tan
2017-01-16 6:00 ` Yuanhan Liu
2017-01-16 6:04 ` Yuanhan Liu
2017-01-16 9:44 ` Thomas Monjalon
2017-01-16 9:49 ` Yuanhan Liu
2017-01-16 14:45 ` Thomas Monjalon
2017-01-22 0:46 ` Aws Ismail
2017-01-22 7:16 ` Tan, Jianfeng
2017-01-16 15:05 ` [PATCH v4 0/8] virtio_user as an alternative exception path Yuanhan Liu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.