[PATCH v3 00/10] lib/librte_vhost: introduce new vhost user crypto backend support

[PATCH v3 00/10] lib/librte_vhost: introduce new vhost user crypto backend support

[Fan Zhang]

This patchset adds crypto backend suppport to vhost_user library,
including a proof-of-concept sample application. The implementation
follows the virtio-crypto specification and have been tested
with qemu 2.11.50 (with several patches applied, detailed later)
with Fedora 24 running in the frontend.

The vhost_crypto library acts as a "bridge" method that translate
the virtio-crypto crypto requests to DPDK crypto operations, so it
is purely software implementation. However it does require the user
to provide the DPDK Cryptodev ID so it knows how to handle the
virtio-crypto session creation and deletion mesages.

Currently the implementation supports AES-CBC-128 and HMAC-SHA1
cipher only/chaining modes and does not support sessionless mode
yet. The guest can use standard virtio-crypto driver to set up
session and sends encryption/decryption requests to backend. The
vhost-crypto sample application provided in this patchset will
do the actual crypto work.

To make this patchset working, a few tweaks need to be done:

In the host:
1. Download the qemu source code.

2. Configure and compile your qemu with vhost-crypto option enabled.

3. Apply this patchset to latest DPDK code and recompile DPDK.

4. Compile and run vhost-crypto sample application.

./examples/vhost_crypto/build/vhost-crypto -l 11,12,13 -w 0000:86:01.0 \
 --socket-mem 2048,2048

Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
not available. The sample application requires 2 lcores: 1 master and 1 worker.
The application will create a UNIX socket file /tmp/vhost_crypto1.socket. It
is possible to use other flags like --zero-copy and --guest-polling.

5. Start your qemu application. Here is my command:

qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
-smp 2 -m 1G -hda ~/path-to-your/image.qcow \
-object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
-mem-prealloc -numa node,memdev=mem -chardev \
socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
-object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
-device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0 

6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
default. You shall see the following logs in your demsg:

[   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
[   17.611083] virtio_crypto: module verification failed: signature and/or ...
[   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
[   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
[   18.376100] virtio_crypto virtio0: Accelerator is ready

The virtio_crypto driver in the guest is now up and running.

7. The rest steps can be as same as the Testing section in
https://wiki.qemu.org/Features/VirtioCrypto

8. It is possible to use DPDK Virtio Crypto PMD
(https://dpdk.org/dev/patchwork/patch/36471/) in the guest to work with
this patchset to achieve optimal performance. 

v3:
- Changed external vhost backend private data and message handling
- Added experimental tag to rte_vhost_crypto_set_zero_copy()

v2:
- Moved vhost_crypto_data_req data from crypto op to source mbuf.
- Removed ZERO-COPY flag from config option and make it run-timely changeable.
- Guest-polling mode possible.
- Simplified vring descriptor access procedure.
- Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.


Fan Zhang (10):
  lib/librte_vhost: add external backend support
  lib/librte_vhost: add virtio-crypto user message structure
  lib/librte_vhost: add session message handler
  lib/librte_vhost: add request handler
  lib/librte_vhost: add head file
  lib/librte_vhost: add public function implementation
  lib/librte_vhost: update version map
  lib/librte_vhost: update makefile
  examples/vhost_crypto: add vhost crypto sample application
  doc: update prog guide and sample app guide

 doc/guides/prog_guide/vhost_lib.rst       |   21 +
 doc/guides/sample_app_ug/index.rst        |    1 +
 doc/guides/sample_app_ug/vhost_crypto.rst |   84 ++
 examples/vhost_crypto/Makefile            |   32 +
 examples/vhost_crypto/main.c              |  598 ++++++++++++++
 lib/librte_vhost/Makefile                 |    6 +-
 lib/librte_vhost/meson.build              |    8 +-
 lib/librte_vhost/rte_vhost.h              |   45 +-
 lib/librte_vhost/rte_vhost_crypto.h       |  125 +++
 lib/librte_vhost/rte_vhost_version.map    |   11 +
 lib/librte_vhost/vhost.c                  |   23 +-
 lib/librte_vhost/vhost.h                  |    8 +-
 lib/librte_vhost/vhost_crypto.c           | 1269 +++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c             |   29 +-
 lib/librte_vhost/vhost_user.h             |   36 +-
 15 files changed, 2279 insertions(+), 17 deletions(-)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
 create mode 100644 lib/librte_vhost/vhost_crypto.c

-- 
2.7.4

[PATCH v3 01/10] lib/librte_vhost: add external backend support

[Fan Zhang]

This patch adds external backend support to vhost library. The patch provides
new APIs for the external backend to register private data, plus pre and post
vhost-user message handlers.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost.h  | 45 ++++++++++++++++++++++++++++++++++++++++++-
 lib/librte_vhost/vhost.c      | 23 +++++++++++++++++++++-
 lib/librte_vhost/vhost.h      |  8 ++++++--
 lib/librte_vhost/vhost_user.c | 29 +++++++++++++++++++++++++---
 4 files changed, 98 insertions(+), 7 deletions(-)

diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h
index d332069..591b731 100644
--- a/lib/librte_vhost/rte_vhost.h
+++ b/lib/librte_vhost/rte_vhost.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2017 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _RTE_VHOST_H_
@@ -88,6 +88,33 @@ struct vhost_device_ops {
 };
 
 /**
+ * function prototype for external virtio device to handler device specific
+ * vhost user messages
+ *
+ * @param extern_data
+ *  private data for external backend
+ * @param msg
+ *  Message pointer
+ * @param payload
+ *  Message payload
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*rte_vhost_msg_handler)(int vid, void *msg,
+		uint32_t *require_reply);
+
+/**
+ * pre and post vhost user message handlers
+ */
+struct rte_vhost_user_dev_extern_ops {
+	rte_vhost_msg_handler pre_vhost_user_msg_handler;
+	rte_vhost_msg_handler post_vhost_user_msg_handler;
+};
+
+/**
  * Convert guest physical address to host virtual address
  *
  * @param mem
@@ -434,6 +461,22 @@ int rte_vhost_vring_call(int vid, uint16_t vring_idx);
  */
 uint32_t rte_vhost_rx_queue_count(int vid, uint16_t qid);
 
+/**
+ * register external vhost backend
+ *
+ * @param vid
+ *  vhost device ID
+ * @param extern_data
+ *  private data for external backend
+ * @param ops
+ *  ops that process external vhost user messages
+ * @return
+ *  0 on success, -1 on failure
+ */
+int
+rte_vhost_user_register_extern_backend(int vid, void *extern_data,
+		struct rte_vhost_user_dev_extern_ops *ops);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
index a407067..0932537 100644
--- a/lib/librte_vhost/vhost.c
+++ b/lib/librte_vhost/vhost.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #include <linux/vhost.h>
@@ -627,3 +627,24 @@ rte_vhost_rx_queue_count(int vid, uint16_t qid)
 
 	return *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx;
 }
+
+int
+rte_vhost_user_register_extern_backend(int vid, void *extern_data,
+		struct rte_vhost_user_dev_extern_ops *ops)
+{
+	struct virtio_net *dev;
+
+	dev = get_device(vid);
+	if (dev == NULL)
+		return -1;
+
+	dev->extern_data = extern_data;
+	if (ops) {
+		dev->extern_ops.pre_vhost_user_msg_handler =
+				ops->pre_vhost_user_msg_handler;
+		dev->extern_ops.post_vhost_user_msg_handler =
+				ops->post_vhost_user_msg_handler;
+	}
+
+	return 0;
+}
diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
index d947bc9..6aaa46c 100644
--- a/lib/librte_vhost/vhost.h
+++ b/lib/librte_vhost/vhost.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_CDEV_H_
@@ -241,8 +241,12 @@ struct virtio_net {
 	struct guest_page       *guest_pages;
 
 	int			slave_req_fd;
-} __rte_cache_aligned;
 
+	/* private data for external virtio device */
+	void			*extern_data;
+	/* pre and post vhost user message handlers for externel backend */
+	struct rte_vhost_user_dev_extern_ops extern_ops;
+} __rte_cache_aligned;
 
 #define VHOST_LOG_PAGE	4096
 
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 90ed211..c064cb3 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #include <stdint.h>
@@ -50,6 +50,8 @@ static const char *vhost_message_str[VHOST_USER_MAX] = {
 	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
 	[VHOST_USER_SET_SLAVE_REQ_FD]  = "VHOST_USER_SET_SLAVE_REQ_FD",
 	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
+	[VHOST_USER_CRYPTO_CREATE_SESS] = "VHOST_USER_CRYPTO_CREATE_SESS",
+	[VHOST_USER_CRYPTO_CLOSE_SESS] = "VHOST_USER_CRYPTO_CLOSE_SESS",
 };
 
 static uint64_t
@@ -1379,6 +1381,18 @@ vhost_user_msg_handler(int vid, int fd)
 
 	}
 
+	if (dev->extern_ops.pre_vhost_user_msg_handler) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.pre_vhost_user_msg_handler)(dev->vid,
+				(void *)&msg, &need_reply);
+		if (ret < 0)
+			goto skip_to_reply;
+
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
+	}
+
 	switch (msg.request.master) {
 	case VHOST_USER_GET_FEATURES:
 		msg.payload.u64 = vhost_user_get_features(dev);
@@ -1477,11 +1491,20 @@ vhost_user_msg_handler(int vid, int fd)
 		break;
 
 	default:
-		ret = -1;
-		break;
+		if (dev->extern_ops.post_vhost_user_msg_handler) {
+			uint32_t need_reply;
 
+			ret = (*dev->extern_ops.post_vhost_user_msg_handler)(
+					dev->vid, (void *)&msg, &need_reply);
+
+			if (need_reply)
+				send_vhost_reply(fd, &msg);
+		} else
+			ret = -1;
+		break;
 	}
 
+skip_to_reply:
 	if (unlock_required)
 		vhost_user_unlock_all_queue_pairs(dev);
 
-- 
2.7.4

[PATCH v3 02/10] lib/librte_vhost: add virtio-crypto user message structure

[Fan Zhang]

This patch adds virtio-crypto spec user message structure to
vhost_user.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_user.h | 36 ++++++++++++++++++++++++++++++++----
 1 file changed, 32 insertions(+), 4 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h
index d4bd604..48cdb24 100644
--- a/lib/librte_vhost/vhost_user.h
+++ b/lib/librte_vhost/vhost_user.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_USER_H
@@ -20,13 +20,15 @@
 #define VHOST_USER_PROTOCOL_F_REPLY_ACK	3
 #define VHOST_USER_PROTOCOL_F_NET_MTU 4
 #define VHOST_USER_PROTOCOL_F_SLAVE_REQ 5
+#define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7
 
 #define VHOST_USER_PROTOCOL_FEATURES	((1ULL << VHOST_USER_PROTOCOL_F_MQ) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) |\
 					 (1ULL << VHOST_USER_PROTOCOL_F_RARP) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_NET_MTU) | \
-					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ))
+					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ) | \
+					 (1ULL << VHOST_USER_PROTOCOL_F_CRYPTO_SESSION))
 
 typedef enum VhostUserRequest {
 	VHOST_USER_NONE = 0,
@@ -52,7 +54,9 @@ typedef enum VhostUserRequest {
 	VHOST_USER_NET_SET_MTU = 20,
 	VHOST_USER_SET_SLAVE_REQ_FD = 21,
 	VHOST_USER_IOTLB_MSG = 22,
-	VHOST_USER_MAX
+	VHOST_USER_CRYPTO_CREATE_SESS = 26,
+	VHOST_USER_CRYPTO_CLOSE_SESS = 27,
+	VHOST_USER_MAX = 28
 } VhostUserRequest;
 
 typedef enum VhostUserSlaveRequest {
@@ -79,6 +83,30 @@ typedef struct VhostUserLog {
 	uint64_t mmap_offset;
 } VhostUserLog;
 
+/* Comply with Cryptodev-Linux */
+#define VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH	512
+#define VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH	64
+
+/* Same structure as vhost-user backend session info */
+typedef struct VhostUserCryptoSessionParam {
+	int64_t session_id;
+	uint32_t op_code;
+	uint32_t cipher_algo;
+	uint32_t cipher_key_len;
+	uint32_t hash_algo;
+	uint32_t digest_len;
+	uint32_t auth_key_len;
+	uint32_t aad_len;
+	uint8_t op_type;
+	uint8_t dir;
+	uint8_t hash_mode;
+	uint8_t chaining_dir;
+	uint8_t *ciphe_key;
+	uint8_t *auth_key;
+	uint8_t cipher_key_buf[VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH];
+	uint8_t auth_key_buf[VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH];
+} VhostUserCryptoSessionParam;
+
 typedef struct VhostUserMsg {
 	union {
 		VhostUserRequest master;
@@ -99,6 +127,7 @@ typedef struct VhostUserMsg {
 		VhostUserMemory memory;
 		VhostUserLog    log;
 		struct vhost_iotlb_msg iotlb;
+		VhostUserCryptoSessionParam crypto_session;
 	} payload;
 	int fds[VHOST_MEMORY_MAX_NREGIONS];
 } __attribute((packed)) VhostUserMsg;
@@ -108,7 +137,6 @@ typedef struct VhostUserMsg {
 /* The version of the protocol we support */
 #define VHOST_USER_VERSION    0x1
 
-
 /* vhost_user.c */
 int vhost_user_msg_handler(int vid, int fd);
 int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm);
-- 
2.7.4

[PATCH v3 03/10] lib/librte_vhost: add session message handler

[Fan Zhang]

This patch adds session message handler to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 428 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 428 insertions(+)
 create mode 100644 lib/librte_vhost/vhost_crypto.c

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
new file mode 100644
index 0000000..c639b20
--- /dev/null
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include <stdbool.h>
+
+#include <rte_malloc.h>
+#include <rte_jhash.h>
+#include <rte_mbuf.h>
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#include <rte_hexdump.h>
+#endif
+#include "vhost.h"
+#include "vhost_user.h"
+#include "rte_vhost_crypto.h"
+
+#define NB_MEMPOOL_OBJS			(1024)
+#define NB_CRYPTO_DESCRIPTORS		(1024)
+#define NB_CACHE_OBJS			(128)
+
+#define SESSION_MAP_ENTRIES		(1024) /**< Max nb sessions per vdev */
+#define MAX_KEY_SIZE			(32)
+#define VHOST_CRYPTO_MAX_IV_LEN		(16)
+#define MAX_COUNT_DOWN_TIMES		(100)
+
+#define INHDR_LEN		(sizeof(struct virtio_crypto_inhdr))
+#define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
+				sizeof(struct rte_crypto_sym_op))
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+
+#define VC_LOG_DBG(fmt, args...)				\
+	RTE_LOG(DEBUG, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#else
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_DBG(fmt, args...)
+#endif
+
+#define VIRTIO_CRYPTO_FEATURES ((1 << VIRTIO_F_NOTIFY_ON_EMPTY) |	\
+		(1 << VIRTIO_RING_F_INDIRECT_DESC) |			\
+		(1 << VIRTIO_RING_F_EVENT_IDX) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_CIPHER) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_HASH) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_MAC) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_AEAD) |			\
+		(1 << VIRTIO_NET_F_CTRL_VQ))
+
+
+#define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
+
+/* Macro to get the buffer at the end of rte_crypto_op */
+#define REQ_OP_OFFSET		(IV_OFFSET + VHOST_CRYPTO_MAX_IV_LEN)
+
+/**
+ * 1-to-1 mapping between RTE_CRYPTO_*ALGO* and VIRTIO_CRYPTO_*ALGO*, for
+ * algorithms not supported by RTE_CRYPTODEV, the -VIRTIO_CRYPTO_NOTSUPP is
+ * returned.
+ */
+static int cipher_algo_transform[] = {
+		RTE_CRYPTO_CIPHER_NULL,
+		RTE_CRYPTO_CIPHER_ARC4,
+		RTE_CRYPTO_CIPHER_AES_ECB,
+		RTE_CRYPTO_CIPHER_AES_CBC,
+		RTE_CRYPTO_CIPHER_AES_CTR,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_CIPHER_DES_ECB */
+		RTE_CRYPTO_CIPHER_DES_CBC,
+		RTE_CRYPTO_CIPHER_3DES_ECB,
+		RTE_CRYPTO_CIPHER_3DES_CBC,
+		RTE_CRYPTO_CIPHER_3DES_CTR,
+		RTE_CRYPTO_CIPHER_KASUMI_F8,
+		RTE_CRYPTO_CIPHER_SNOW3G_UEA2,
+		RTE_CRYPTO_CIPHER_AES_F8,
+		RTE_CRYPTO_CIPHER_AES_XTS,
+		RTE_CRYPTO_CIPHER_ZUC_EEA3
+};
+
+/**
+ * VIRTIO_CRYTPO_AUTH_* indexes are not sequential, the gaps are filled with
+ * -VIRTIO_CRYPTO_BADMSG errors.
+ */
+static int auth_algo_transform[] = {
+		RTE_CRYPTO_AUTH_NULL,
+		RTE_CRYPTO_AUTH_MD5_HMAC,
+		RTE_CRYPTO_AUTH_SHA1_HMAC,
+		RTE_CRYPTO_AUTH_SHA224_HMAC,
+		RTE_CRYPTO_AUTH_SHA256_HMAC,
+		RTE_CRYPTO_AUTH_SHA384_HMAC,
+		RTE_CRYPTO_AUTH_SHA512_HMAC,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_CMAC_3DES */
+		RTE_CRYPTO_AUTH_AES_CMAC,
+		RTE_CRYPTO_AUTH_KASUMI_F9,
+		RTE_CRYPTO_AUTH_SNOW3G_UIA2,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		RTE_CRYPTO_AUTH_AES_GMAC,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_GMAC_TWOFISH */
+		RTE_CRYPTO_AUTH_AES_CBC_MAC,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9 */
+		RTE_CRYPTO_AUTH_AES_XCBC_MAC
+};
+
+static int cipher_op_transform[] = {
+		-VIRTIO_CRYPTO_BADMSG, /* meaningless */
+		RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+		RTE_CRYPTO_CIPHER_OP_DECRYPT
+};
+
+static int iv_lens[] = {
+		-1, /* Invalid input */
+		0, /* RTE_CRYPTO_CIPHER_NULL */
+		8, /* RTE_CRYPTO_CIPHER_3DES_CBC */
+		8, /* RTE_CRYPTO_CIPHER_3DES_CTR */
+		8, /* RTE_CRYPTO_CIPHER_3DES_ECB */
+		16, /* RTE_CRYPTO_CIPHER_AES_CBC */
+		/* TODO: add common algos */
+};
+
+/**
+ * vhost_crypto struct is used to maintain a number of virtio_cryptos and
+ * one DPDK crypto device that deals with all crypto workloads. It is declared
+ * here and defined in vhost_crypto.c
+ */
+struct vhost_crypto {
+	/** Used to lookup DPDK Cryptodev Session based on VIRTIO crypto
+	 *  session ID.
+	 */
+	struct rte_hash *session_map;
+	struct rte_mempool *mbuf_pool;
+	struct rte_mempool *sess_pool;
+
+	/** DPDK cryptodev ID */
+	uint8_t cid;
+	uint16_t nb_qps;
+
+	uint64_t last_session_id;
+
+	uint64_t cache_session_id;
+	struct rte_cryptodev_sym_session *cache_session;
+	/** socket id for the device */
+	int socket_id;
+
+	struct virtio_net *dev;
+
+	uint8_t option;
+} __rte_cache_aligned;
+
+struct vhost_crypto_data_req {
+	struct vring_desc *head;
+	struct rte_vhost_memory *mem;
+	struct virtio_crypto_inhdr *inhdr;
+
+	uint16_t desc_idx;
+	uint32_t len;
+	struct vhost_virtqueue *vq;
+
+	uint8_t zero_copy;
+
+	int vid;
+
+	struct vring_desc *wb_desc;
+	uint16_t wb_len;
+};
+
+static int
+transform_cipher_param(struct rte_crypto_sym_xform *xform,
+		VhostUserCryptoSessionParam *param)
+{
+	int ret;
+
+	ret = cipher_algo_transform[param->cipher_algo];
+	if (unlikely(ret < 0))
+		return ret;
+
+	xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform->cipher.algo = ret;
+	xform->cipher.key.length = param->cipher_key_len;
+	if (xform->cipher.key.length > 0)
+		xform->cipher.key.data = param->cipher_key_buf;
+	ret = cipher_op_transform[param->dir];
+	if (unlikely(ret < 0)) {
+		VC_LOG_DBG("Bad operation type");
+		return ret;
+	}
+	xform->cipher.op = ret;
+	ret = iv_lens[xform->cipher.algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform->cipher.iv.length = ret;
+	xform->cipher.iv.offset = IV_OFFSET;
+	return 0;
+}
+
+static int
+transform_chain_param(struct rte_crypto_sym_xform *xforms,
+		VhostUserCryptoSessionParam *param)
+{
+	struct rte_crypto_sym_xform *xform_cipher, *xform_auth;
+	int ret;
+
+	switch (param->chaining_dir) {
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER:
+		xform_auth = xforms;
+		xform_cipher = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+		break;
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH:
+		xform_cipher = xforms;
+		xform_auth = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		break;
+	default:
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	/* cipher */
+	ret = cipher_algo_transform[param->cipher_algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform_cipher->cipher.algo = ret;
+	xform_cipher->cipher.key.length = param->cipher_key_len;
+	xform_cipher->cipher.key.data = param->cipher_key_buf;
+	ret = iv_lens[xform_cipher->cipher.algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->cipher.iv.length = ret;
+	xform_cipher->cipher.iv.offset = IV_OFFSET;
+
+	/* auth */
+	xform_auth->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+	ret = auth_algo_transform[param->hash_algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_auth->auth.algo = ret;
+	xform_auth->auth.digest_length = param->digest_len;
+	xform_auth->auth.key.length = param->auth_key_len;
+	xform_auth->auth.key.data = param->auth_key_buf;
+
+	return 0;
+}
+
+static void
+vhost_crypto_create_sess(struct vhost_crypto *vcrypto,
+		VhostUserCryptoSessionParam *sess_param)
+{
+	struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0};
+	struct rte_cryptodev_sym_session *session;
+	int ret;
+
+	switch (sess_param->op_type) {
+	case VIRTIO_CRYPTO_SYM_OP_NONE:
+	case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+		ret = transform_cipher_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session msg (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+		break;
+	case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+		if (unlikely(sess_param->hash_mode !=
+				VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH)) {
+			sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+			VC_LOG_ERR("Error transform session message (%i)",
+					-VIRTIO_CRYPTO_NOTSUPP);
+			return;
+		}
+
+		xform1.next = &xform2;
+
+		ret = transform_chain_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session message (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+
+		break;
+	default:
+		VC_LOG_ERR("Algorithm not yet supported");
+		sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+		return;
+	}
+
+	session = rte_cryptodev_sym_session_create(vcrypto->sess_pool);
+	if (!session) {
+		VC_LOG_ERR("Failed to create session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1,
+			vcrypto->sess_pool) < 0) {
+		VC_LOG_ERR("Failed to initialize session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	/* insert hash to map */
+	if (rte_hash_add_key_data(vcrypto->session_map,
+			&vcrypto->last_session_id, session) < 0) {
+		VC_LOG_ERR("Failed to insert session to hash table");
+
+		if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0)
+			VC_LOG_ERR("Failed to clear session");
+		else {
+			if (rte_cryptodev_sym_session_free(session) < 0)
+				VC_LOG_ERR("Failed to free session");
+		}
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) created.",
+			vcrypto->last_session_id, session);
+
+	sess_param->session_id = vcrypto->last_session_id;
+	vcrypto->last_session_id++;
+}
+
+static int
+vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id)
+{
+	struct rte_cryptodev_sym_session *session;
+	uint64_t sess_id = session_id;
+	int ret;
+
+	ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id,
+			(void **)&session);
+
+	if (unlikely(ret < 0)) {
+		VC_LOG_ERR("Failed to delete session (key %lu).", session_id);
+		return -VIRTIO_CRYPTO_INVSESS;
+	}
+
+	if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_cryptodev_sym_session_free(session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_hash_del_key(vcrypto->session_map, &sess_id) < 0) {
+		VC_LOG_DBG("Failed to delete session from hash table.");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) deleted.", sess_id,
+			session);
+
+	return 0;
+}
+
+static int
+vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+	VhostUserMsg *vmsg = msg;
+	int ret = 0;
+
+	if (dev == NULL || require_reply == NULL) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (vcrypto == NULL) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	*require_reply = 0;
+
+	if (vmsg->request.master == VHOST_USER_CRYPTO_CREATE_SESS) {
+		vhost_crypto_create_sess(vcrypto,
+				&vmsg->payload.crypto_session);
+		*require_reply = 1;
+	} else if (vmsg->request.master == VHOST_USER_CRYPTO_CLOSE_SESS)
+		ret = vhost_crypto_close_sess(vcrypto, vmsg->payload.u64);
+	else
+		ret = -EINVAL;
+
+	return ret;
+}
-- 
2.7.4

[PATCH v3 04/10] lib/librte_vhost: add request handler

[Fan Zhang]

This patch adds the implementation that parses virtio crypto request
to dpdk crypto operation.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 593 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 593 insertions(+)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index c639b20..1d85829 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -426,3 +426,596 @@ vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
 
 	return ret;
 }
+
+static __rte_always_inline struct vring_desc *
+find_write_desc(struct vring_desc *head, struct vring_desc *desc)
+{
+	if (desc->flags & VRING_DESC_F_WRITE)
+		return desc;
+
+	while (desc->flags & VRING_DESC_F_NEXT) {
+		desc = &head[desc->next];
+		if (desc->flags & VRING_DESC_F_WRITE)
+			return desc;
+	}
+
+	return NULL;
+}
+
+static struct virtio_crypto_inhdr *
+reach_inhdr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc *desc)
+{
+	while (desc->flags & VRING_DESC_F_NEXT)
+		desc = &head[desc->next];
+
+	return GPA_TO_VVA(struct virtio_crypto_inhdr *, mem, desc->addr);
+}
+
+static __rte_always_inline int
+move_desc(struct vring_desc *head, struct vring_desc **cur_desc,
+		uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	left -= desc->len;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		left -= desc->len;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+	return 0;
+}
+
+static int
+copy_data(void *dst_data, struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	uint32_t to_copy;
+	uint8_t *data = dst_data;
+	uint8_t *src;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	to_copy = RTE_MIN(desc->len, (uint32_t)left);
+	src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy((uint8_t *)data, src, to_copy);
+	left -= to_copy;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_copy = RTE_MIN(desc->len, (uint32_t)left);
+		src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(data + size - left, src, to_copy);
+		left -= to_copy;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+
+	return 0;
+}
+
+static __rte_always_inline void *
+get_data_ptr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	void *data;
+
+	data = GPA_TO_VVA(void *, mem, (*cur_desc)->addr);
+	if (unlikely(!data)) {
+		VC_LOG_ERR("Failed to get object");
+		return NULL;
+	}
+
+	if (unlikely(move_desc(head, cur_desc, size) < 0))
+		return NULL;
+
+	return data;
+}
+
+static int
+write_back_data(struct rte_crypto_op *op, struct vhost_crypto_data_req *vc_req)
+{
+	struct rte_mbuf *mbuf = op->sym->m_dst;
+	struct vring_desc *head = vc_req->head;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct vring_desc *desc = vc_req->wb_desc;
+	int left = vc_req->wb_len;
+	uint32_t to_write;
+	uint8_t *src_data = mbuf->buf_addr, *dst;
+
+	rte_prefetch0(&head[desc->next]);
+	to_write = RTE_MIN(desc->len, (uint32_t)left);
+	dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy(dst, src_data, to_write);
+	left -= to_write;
+	src_data += to_write;
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	printf("desc addr %llu len %u:", desc->addr, desc->len);
+	rte_hexdump(stdout, "", dst, to_write);
+#endif
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_write = RTE_MIN(desc->len, (uint32_t)left);
+		dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(dst, src_data, to_write);
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+		printf("desc addr %llu len %u:", desc->addr, desc->len);
+		rte_hexdump(stdout, "DST:", dst, to_write);
+#endif
+		left -= to_write;
+		src_data += to_write;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	return 0;
+}
+
+static uint8_t
+prepare_sym_cipher_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_cipher_data_req *cipher,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			cipher->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, cipher->para.iv_len);
+#endif
+
+	m_src->data_len = cipher->para.src_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				cipher->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 ||
+				m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		if (unlikely(cipher->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, cipher->para.src_data_len))
+				< 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			cipher->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, cipher->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		m_dst->data_len = cipher->para.dst_data_len;
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = cipher->para.dst_data_len;
+		break;
+	}
+
+	/* src data */
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = 0;
+	op->sym->cipher.data.length = cipher->para.src_data_len;
+
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+	vc_req->len = cipher->para.dst_data_len + INHDR_LEN;
+
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+static uint8_t
+prepare_sym_chain_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_alg_chain_data_req *chain,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint32_t digest_offset;
+	void *digest_addr;
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			chain->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, chain->para.iv_len);
+#endif
+	m_src->data_len = chain->para.src_data_len;
+	m_dst->data_len = chain->para.dst_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				chain->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 || m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		if (unlikely(chain->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, chain->para.src_data_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			chain->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.phys_addr = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.hash_result_len);
+		op->sym->auth.digest.data = get_data_ptr(head, mem, &desc,
+				chain->para.hash_result_len);
+		if (unlikely(op->sym->auth.digest.phys_addr == 0)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		digest_offset = m_dst->data_len;
+		digest_addr = rte_pktmbuf_mtod_offset(m_dst, void *,
+				digest_offset);
+
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = m_dst->data_len + chain->para.hash_result_len;
+
+		if (unlikely(move_desc(head, &desc,
+				chain->para.dst_data_len) < 0)) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		if (unlikely(copy_data(digest_addr, head, mem, &desc,
+				chain->para.hash_result_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.data = digest_addr;
+		op->sym->auth.digest.phys_addr = rte_pktmbuf_iova_offset(m_dst,
+				digest_offset);
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "Digest:", op->sym->auth.digest.data,
+			chain->para.hash_result_len);
+#endif
+
+	/* record inhdr */
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = chain->para.cipher_start_src_offset;
+	op->sym->cipher.data.length = chain->para.src_data_len -
+			chain->para.cipher_start_src_offset;
+
+	op->sym->auth.data.offset = chain->para.hash_start_src_offset;
+	op->sym->auth.data.length = chain->para.len_to_hash;
+
+	vc_req->len = chain->para.dst_data_len + chain->para.hash_result_len +
+			INHDR_LEN;
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+/**
+ * Process on descriptor
+ */
+static __rte_always_inline int
+vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
+		struct vhost_virtqueue *vq, struct rte_crypto_op *op,
+		struct vring_desc *head, uint16_t desc_idx,
+		struct rte_vhost_memory *mem)
+{
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(op->sym->m_src,
+			sizeof(struct rte_mbuf));
+	struct rte_cryptodev_sym_session *session;
+	struct virtio_crypto_op_data_req *req;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vring_desc *desc = NULL;
+	uint64_t session_id;
+	int err = 0;
+
+	vc_req->desc_idx = desc_idx;
+
+	if (likely(head->flags & VRING_DESC_F_INDIRECT)) {
+		head = GPA_TO_VVA(struct vring_desc *, mem, head->addr);
+		if (unlikely(!head))
+			return 0;
+		desc_idx = 0;
+	}
+
+	desc = head;
+
+	vc_req->mem = mem;
+	vc_req->head = head;
+	vc_req->vq = vq;
+
+	vc_req->zero_copy = vcrypto->option;
+
+	req = get_data_ptr(head, mem, &desc, sizeof(*req));
+	if (unlikely(req == NULL)) {
+		err = VIRTIO_CRYPTO_ERR;
+		VC_LOG_ERR("Invalid descriptor");
+		goto error_exit;
+	}
+
+	switch (req->header.opcode) {
+	case VIRTIO_CRYPTO_CIPHER_ENCRYPT:
+	case VIRTIO_CRYPTO_CIPHER_DECRYPT:
+		session_id = req->header.session_id;
+
+		/* one branch to avoid unnecessary table lookup */
+		if (vcrypto->cache_session_id != session_id) {
+			err = rte_hash_lookup_data(vcrypto->session_map,
+					&session_id, (void **)&session);
+			if (unlikely(err < 0)) {
+				err = VIRTIO_CRYPTO_ERR;
+				VC_LOG_ERR("Failed to retrieve session id %lu",
+						session_id);
+				goto error_exit;
+			}
+
+			vcrypto->cache_session = session;
+			vcrypto->cache_session_id = session_id;
+		}
+
+		session = vcrypto->cache_session;
+
+		err = rte_crypto_op_attach_sym_session(op, session);
+		if (unlikely(err < 0)) {
+			err = VIRTIO_CRYPTO_ERR;
+			VC_LOG_ERR("Failed to attach session to op");
+			goto error_exit;
+		}
+
+		switch (req->u.sym_req.op_type) {
+		case VIRTIO_CRYPTO_SYM_OP_NONE:
+			err = VIRTIO_CRYPTO_NOTSUPP;
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+			err = prepare_sym_cipher_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.cipher, desc);
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+			err = prepare_sym_chain_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.chain, desc);
+			break;
+		}
+		if (unlikely(err != 0)) {
+			VC_LOG_ERR("Failed to process sym request");
+			goto error_exit;
+		}
+		break;
+	default:
+		VC_LOG_ERR("Unsupported symmetric crypto request type %u",
+				req->header.opcode);
+		goto error_exit;
+	}
+
+	return 0;
+
+error_exit:
+
+	inhdr = reach_inhdr(head, mem, desc);
+	if (likely(inhdr != NULL))
+		inhdr->status = (uint8_t)err;
+
+	return -1;
+}
+
+static __rte_always_inline struct vhost_virtqueue *
+vhost_crypto_finalize_one_request(struct rte_crypto_op *op,
+		struct vhost_virtqueue *old_vq)
+{
+	struct rte_mbuf *m_src = op->sym->m_src;
+	struct rte_mbuf *m_dst = op->sym->m_dst;
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(m_src,
+			sizeof(struct rte_mbuf));
+	uint16_t desc_idx;
+	int ret = 0;
+
+	if (unlikely(!vc_req)) {
+		VC_LOG_ERR("Failed to retrieve vc_req");
+		return NULL;
+	}
+
+	if (old_vq && (vc_req->vq != old_vq))
+		return vc_req->vq;
+
+	desc_idx = vc_req->desc_idx;
+
+	if (unlikely(op->status != RTE_CRYPTO_OP_STATUS_SUCCESS))
+		vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+	else {
+		if (vc_req->zero_copy == 0) {
+			ret = write_back_data(op, vc_req);
+			if (unlikely(ret != 0))
+				vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+		}
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "DST:", rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *),
+			m_dst->data_len);
+#endif
+
+	vc_req->vq->used->ring[desc_idx].id = desc_idx;
+	vc_req->vq->used->ring[desc_idx].len = vc_req->len;
+
+	rte_mempool_put(m_dst->pool, (void *)m_dst);
+	rte_mempool_put(m_src->pool, (void *)m_src);
+
+	return vc_req->vq;
+}
+
+static __rte_always_inline uint16_t
+vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfd)
+{
+	uint16_t processed = 1;
+	struct vhost_virtqueue *vq, *tmp_vq;
+
+	if (unlikely(nb_ops == 0))
+		return 0;
+
+	vq = vhost_crypto_finalize_one_request(ops[0], NULL);
+	if (unlikely(vq == NULL))
+		return 0;
+	tmp_vq = vq;
+
+	while ((processed < nb_ops)) {
+		tmp_vq = vhost_crypto_finalize_one_request(ops[processed],
+				tmp_vq);
+
+		if (unlikely(vq != tmp_vq))
+			break;
+
+		processed++;
+	}
+
+	*callfd = vq->callfd;
+
+	*(volatile uint16_t *)&vq->used->idx += processed;
+
+	return processed;
+}
-- 
2.7.4

[PATCH v3 05/10] lib/librte_vhost: add head file

[Fan Zhang]

This patch adds public head file API for vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h | 125 ++++++++++++++++++++++++++++++++++++
 1 file changed, 125 insertions(+)
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
new file mode 100644
index 0000000..339a939
--- /dev/null
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -0,0 +1,125 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#ifndef _VHOST_CRYPTO_H_
+#define _VHOST_CRYPTO_H_
+
+#include <linux/virtio_crypto.h>
+#include <rte_cryptodev.h>
+#include <rte_hash.h>
+#include <rte_pause.h>
+#include "rte_vhost.h"
+
+#ifndef MAX_DATA_QUEUES
+#define MAX_DATA_QUEUES	(1)
+#endif
+
+#define VIRTIO_CRYPTO_CTRL_QUEUE	(0)
+#define VIRTIO_CRYPTO_MAX_NUM_DEVS	(64)
+#define VIRTIO_CRYPTO_MAX_NUM_BURST_VQS	(64)
+
+/** Feature bits */
+#define VIRTIO_CRYPTO_F_CIPHER_SESSION_MODE	(1)
+#define VIRTIO_CRYPTO_F_HASH_SESSION_MODE	(2)
+#define VIRTIO_CRYPTO_F_MAC_SESSION_MODE	(3)
+#define VIRTIO_CRYPTO_F_AEAD_SESSION_MODE	(4)
+
+#define VHOST_CRYPTO_MBUF_POOL_SIZE		(8192)
+#define VHOST_CRYPTO_MAX_BURST_SIZE		(64)
+
+enum rte_vhost_crypto_zero_copy {
+	RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE = 0,
+	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE,
+	RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS
+};
+
+/**
+ *  Create Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param cryptodev_id
+ *  The identifier of DPDK Cryptodev, the same cryptodev_id can be assigned to
+ *  multiple Vhost-crypto devices.
+ * @param sess_pool
+ *  The pointer to the created cryptodev session pool with the private data size
+ *  matches the target DPDK Cryptodev.
+ * @param socket_id
+ *  NUMA Socket ID to allocate resources on. *
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise
+ */
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id);
+
+/**
+ *  Free the Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise.
+ */
+int
+rte_vhost_crypto_free(int vid);
+
+/**
+ *  Enable or disable zero copy feature
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param option
+ *  Flag of zero copy feature.
+ * @return
+ *  0 if completed successfully.
+ *  Negative integer if otherwise.
+ */
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option);
+
+/**
+ * Fetch a number of vring descriptors from virt-queue and translate to DPDK
+ * crypto operations. After this function is executed, the user can enqueue
+ * the processed ops to the target cryptodev.
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param qid
+ *  Virtio queue index.
+ * @param ops
+ *  The address of an array of pointers to *rte_crypto_op* structures that must
+ *  be large enough to store *nb_ops* pointers in it.
+ * @param nb_ops
+ *  The maximum number of operations to be fetched and translated.
+ * @return
+ *  The number of fetched and processed vhost crypto request operations.
+ */
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops);
+/**
+ * Finalize the dequeued crypto ops. After the translated crypto ops are
+ * dequeued from the cryptodev, this function shall be called to write the
+ * processed data back to the vring descriptor (if no-copy is turned off).
+ *
+ * @param ops
+ *  The address of an array of *rte_crypto_op* structure that was dequeued
+ *  from cryptodev.
+ * @param nb_ops
+ *  The number of operations contained in the array.
+ * @callfds
+ *  The callfd number(s) contained in this burst
+ * @nb_callfds
+ *  The number of call_fd numbers exist in the callfds
+ * @return
+ *  The number of ops processed.
+ */
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds);
+
+#endif /**< _VHOST_CRYPTO_H_ */
-- 
2.7.4

[PATCH v3 06/10] lib/librte_vhost: add public function implementation

[Fan Zhang]

This patch adds public API implementation to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 248 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 248 insertions(+)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index 1d85829..f64261a 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -1019,3 +1019,251 @@ vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
 
 	return processed;
 }
+
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct rte_vhost_user_dev_extern_ops ops;
+	struct rte_hash_parameters params = {0};
+	struct vhost_crypto *vcrypto;
+	char name[128];
+	int ret;
+
+	if (vid >= VIRTIO_CRYPTO_MAX_NUM_DEVS || !dev) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	ret = rte_vhost_driver_set_features(dev->ifname,
+			VIRTIO_CRYPTO_FEATURES);
+	if (ret < 0) {
+		VC_LOG_ERR("Error setting features");
+		return -1;
+	}
+
+	vcrypto = rte_zmalloc_socket(NULL, sizeof(*vcrypto),
+			RTE_CACHE_LINE_SIZE, socket_id);
+	if (!vcrypto) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	vcrypto->sess_pool = sess_pool;
+	vcrypto->cid = cryptodev_id;
+	vcrypto->cache_session_id = UINT64_MAX;
+	vcrypto->last_session_id = 1;
+	vcrypto->dev = dev;
+	vcrypto->option = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	snprintf(name, 127, "HASH_VHOST_CRYPT_%u", (uint32_t)vid);
+	params.name = name;
+	params.entries = SESSION_MAP_ENTRIES;
+	params.hash_func = rte_jhash;
+	params.key_len = sizeof(uint64_t);
+	params.socket_id = socket_id;
+	vcrypto->session_map = rte_hash_create(&params);
+	if (!vcrypto->session_map) {
+		VC_LOG_ERR("Failed to creath session map");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "MBUF_POOL_VM_%u", (uint32_t)vid);
+	vcrypto->mbuf_pool = rte_pktmbuf_pool_create(name,
+			VHOST_CRYPTO_MBUF_POOL_SIZE, 512,
+			sizeof(struct vhost_crypto_data_req),
+			RTE_MBUF_DEFAULT_DATAROOM * 2 + RTE_PKTMBUF_HEADROOM,
+			rte_socket_id());
+	if (!vcrypto->mbuf_pool) {
+		VC_LOG_ERR("Failed to creath mbuf pool");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	ops.pre_vhost_user_msg_handler = NULL;
+	ops.post_vhost_user_msg_handler = vhost_crypto_msg_post_handler;
+
+	if (rte_vhost_user_register_extern_backend(dev->vid, (void *)vcrypto,
+			&ops) < 0) {
+		VC_LOG_ERR("Failed to register device");
+		goto error_exit;
+	}
+
+	return 0;
+
+error_exit:
+	if (vcrypto->session_map)
+		rte_hash_free(vcrypto->session_map);
+	if (vcrypto->mbuf_pool)
+		rte_mempool_free(vcrypto->mbuf_pool);
+
+	rte_free(vcrypto);
+
+	return ret;
+}
+
+int
+rte_vhost_crypto_free(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	rte_hash_free(vcrypto->session_map);
+	rte_mempool_free(vcrypto->mbuf_pool);
+	rte_free(vcrypto);
+
+	dev->extern_data = NULL;
+	dev->extern_ops.post_vhost_user_msg_handler = NULL;
+	dev->extern_ops.pre_vhost_user_msg_handler = NULL;
+
+	return 0;
+}
+
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	if (unlikely(option < 0 || option >=
+			RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS)) {
+		VC_LOG_ERR("Invalid option %i", option);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	if (vcrypto->option == (uint8_t)option)
+		return 0;
+
+	if (!(rte_mempool_full(vcrypto->mbuf_pool))) {
+		VC_LOG_ERR("Cannot update zero copy as mempool is not full");
+		return -EINVAL;
+	}
+
+	vcrypto->option = (uint8_t)option;
+
+	return 0;
+}
+
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+	struct rte_mbuf *mbufs[VHOST_CRYPTO_MAX_BURST_SIZE * 2];
+	struct virtio_net *dev = get_device(vid);
+	struct rte_vhost_memory *mem;
+	struct vhost_crypto *vcrypto;
+	struct vhost_virtqueue *vq;
+	uint16_t avail_idx;
+	uint16_t start_idx;
+	uint16_t required;
+	uint16_t count;
+	uint16_t i;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vq = dev->virtqueue[qid];
+	mem = dev->mem;
+
+	avail_idx = *((volatile uint16_t *)&vq->avail->idx);
+	start_idx = vq->last_used_idx;
+	count = avail_idx - start_idx;
+	count = RTE_MIN(count, VHOST_CRYPTO_MAX_BURST_SIZE);
+	count = RTE_MIN(count, nb_ops);
+
+	if (unlikely(count == 0))
+		return 0;
+
+	/* for zero copy, we need 2 empty mbufs for src and dst, otherwise
+	 * we need only 1 mbuf as src and dst
+	 */
+	required = count * 2;
+	if (unlikely(rte_mempool_get_bulk(vcrypto->mbuf_pool, (void **)mbufs,
+			required) < 0)) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	for (i = 0; i < count; i++) {
+		uint16_t used_idx = (start_idx + i) & (vq->size - 1);
+		uint16_t desc_idx = vq->avail->ring[used_idx];
+		struct vring_desc *head = &vq->desc[desc_idx];
+		struct rte_crypto_op *op = ops[i];
+
+		op->sym->m_src = mbufs[i * 2];
+		op->sym->m_dst = mbufs[i * 2 + 1];
+		op->sym->m_src->data_off = 0;
+		op->sym->m_dst->data_off = 0;
+
+		if (unlikely(vhost_crypto_process_one_req(vcrypto, vq, op, head,
+				desc_idx, mem)) < 0)
+			break;
+	}
+
+	vq->last_used_idx += i;
+
+	return i;
+}
+
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds)
+{
+	struct rte_crypto_op **tmp_ops = ops;
+	uint16_t count = 0, left = nb_ops;
+	int callfd;
+	uint16_t idx = 0;
+
+	while (left) {
+		count = vhost_crypto_complete_one_vm_requests(tmp_ops, left,
+				&callfd);
+		if (unlikely(count == 0))
+			break;
+
+		tmp_ops = &tmp_ops[count];
+		left -= count;
+
+		callfds[idx++] = callfd;
+
+		if (unlikely(idx >= VIRTIO_CRYPTO_MAX_NUM_BURST_VQS)) {
+			VC_LOG_ERR("Too many vqs");
+			break;
+		}
+	}
+
+	*nb_callfds = idx;
+
+	return nb_ops - left;
+}
-- 
2.7.4

[PATCH v3 07/10] lib/librte_vhost: update version map

[Fan Zhang]

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_version.map | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map
index df01031..935eebd 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -59,3 +59,14 @@ DPDK_18.02 {
 	rte_vhost_vring_call;
 
 } DPDK_17.08;
+
+DPDK_18.05 {
+	global:
+
+	rte_vhost_user_register_extern_backend;
+	rte_vhost_crypto_create;
+	rte_vhost_crypto_free;
+	rte_vhost_crypto_fetch_requests;
+	rte_vhost_crypto_finalize_requests;
+
+} DPDK_18.02;
-- 
2.7.4

[PATCH v3 08/10] lib/librte_vhost: update makefile

[Fan Zhang]

This patch updates the Makefile of vhost library to enable vhost crypto
compiling.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/Makefile    | 6 ++++--
 lib/librte_vhost/meson.build | 8 ++++----
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 5d6c6ab..95a6a93 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -23,8 +23,10 @@ LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev -lrte_net
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_VHOST) := fd_man.c iotlb.c socket.c vhost.c \
 					vhost_user.c virtio_net.c
+ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV), y)
+SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto.c
+endif
 
 # install includes
-SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h
-
+SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h rte_vhost_crypto.h
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_vhost/meson.build b/lib/librte_vhost/meson.build
index 9e8c0e7..36f1e27 100644
--- a/lib/librte_vhost/meson.build
+++ b/lib/librte_vhost/meson.build
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
+# Copyright(c) 2017-2018 Intel Corporation
 
 if host_machine.system() != 'linux'
 	build = false
@@ -10,6 +10,6 @@ endif
 version = 4
 allow_experimental_apis = true
 sources = files('fd_man.c', 'iotlb.c', 'socket.c', 'vhost.c', 'vhost_user.c',
-		'virtio_net.c')
-headers = files('rte_vhost.h')
-deps += ['ethdev']
+		'virtio_net.c', 'virtio_crypto.c')
+headers = files('rte_vhost.h', 'rte_vhost_crypto.h')
+deps += ['ethdev', 'cryptodev']
-- 
2.7.4

[PATCH v3 09/10] examples/vhost_crypto: add vhost crypto sample application

[Fan Zhang]

This patch adds vhost_crypto sample application to DPDK.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 examples/vhost_crypto/Makefile |  32 +++
 examples/vhost_crypto/main.c   | 598 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 630 insertions(+)
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c

diff --git a/examples/vhost_crypto/Makefile b/examples/vhost_crypto/Makefile
new file mode 100644
index 0000000..1bb65e8
--- /dev/null
+++ b/examples/vhost_crypto/Makefile
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vhost-crypto
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
diff --git a/examples/vhost_crypto/main.c b/examples/vhost_crypto/main.c
new file mode 100644
index 0000000..d5d1525
--- /dev/null
+++ b/examples/vhost_crypto/main.c
@@ -0,0 +1,598 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <stdbool.h>
+#include <assert.h>
+#include <getopt.h>
+
+#include <rte_malloc.h>
+#include <rte_cycles.h>
+#include <rte_vhost.h>
+#include <rte_cryptodev.h>
+#include <rte_vhost_crypto.h>
+
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define NB_VIRTIO_QUEUES		(1)
+#define MAX_PKT_BURST			(64)
+#define MAX_IV_LEN			(32)
+#define NB_MEMPOOL_OBJS			(8192)
+#define NB_CRYPTO_DESCRIPTORS		(4096)
+#define NB_CACHE_OBJS			(128)
+#define SESSION_MAP_ENTRIES		(1024)
+#define REFRESH_TIME_SEC		(3)
+
+#define MAX_NB_SOCKETS			(32)
+#define DEF_SOCKET_FILE			"/tmp/vhost_crypto1.socket"
+
+struct vhost_crypto_options {
+	char *socket_files[MAX_NB_SOCKETS];
+	uint32_t nb_sockets;
+	uint8_t cid;
+	uint16_t qid;
+	uint32_t zero_copy;
+	uint32_t guest_polling;
+} options;
+
+struct vhost_crypto_info {
+	int vids[MAX_NB_SOCKETS];
+	struct rte_mempool *sess_pool;
+	struct rte_mempool *cop_pool;
+	uint32_t lcore_id;
+	uint8_t cid;
+	uint32_t qid;
+	uint32_t nb_vids;
+	volatile uint32_t initialized[MAX_NB_SOCKETS];
+
+} info;
+
+#define SOCKET_FILE_KEYWORD	"socket-file"
+#define CRYPTODEV_ID_KEYWORD	"cdev-id"
+#define CRYPTODEV_QUEUE_KEYWORD	"cdev-queue-id"
+#define ZERO_COPY_KEYWORD	"zero-copy"
+#define POLLING_KEYWORD		"guest-polling"
+
+uint64_t vhost_cycles[2], last_v_cycles[2];
+uint64_t outpkt_amount;
+
+/** support *SOCKET_FILE_PATH:CRYPTODEV_ID* format */
+static int
+parse_socket_arg(char *arg)
+{
+	uint32_t nb_sockets = options.nb_sockets;
+	size_t len = strlen(arg);
+
+	if (nb_sockets >= MAX_NB_SOCKETS) {
+		RTE_LOG(ERR, USER1, "Too many socket files!\n");
+		return -ENOMEM;
+	}
+
+	options.socket_files[nb_sockets] = rte_malloc(NULL, len, 0);
+	if (!options.socket_files[nb_sockets]) {
+		RTE_LOG(ERR, USER1, "Insufficient memory\n");
+		return -ENOMEM;
+	}
+
+	rte_memcpy(options.socket_files[nb_sockets], arg, len);
+
+	options.nb_sockets++;
+
+	return 0;
+}
+
+static int
+parse_cryptodev_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if ((pm == '\0') || (end == NULL) || (*end != '\0')) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	if (pm > rte_cryptodev_count()) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.cid = (uint8_t)pm;
+
+	return 0;
+}
+
+static int
+parse_cdev_queue_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if (pm == UINT64_MAX) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev Queue ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.qid = (uint16_t)pm;
+
+	return 0;
+}
+
+static void
+vhost_crypto_usage(const char *prgname)
+{
+	printf("%s [EAL options] --\n"
+		"  --%s SOCKET-FILE-PATH\n"
+		"  --%s CRYPTODEV_ID: crypto device id\n"
+		"  --%s CDEV_QUEUE_ID: crypto device queue id\n"
+		"  --%s: zero copy\n"
+		"  --%s: guest polling\n",
+		prgname, SOCKET_FILE_KEYWORD, CRYPTODEV_ID_KEYWORD,
+		CRYPTODEV_QUEUE_KEYWORD, ZERO_COPY_KEYWORD, POLLING_KEYWORD);
+}
+
+static int
+vhost_crypto_parse_args(int argc, char **argv)
+{
+	int opt, ret;
+	char *prgname = argv[0];
+	char **argvopt;
+	int option_index;
+	struct option lgopts[] = {
+			{SOCKET_FILE_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_ID_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_QUEUE_KEYWORD, required_argument, 0, 0},
+			{ZERO_COPY_KEYWORD, no_argument, 0, 0},
+			{POLLING_KEYWORD, no_argument, 0, 0},
+			{NULL, 0, 0, 0}
+	};
+
+	options.cid = 0;
+	options.qid = 0;
+	options.nb_sockets = 0;
+	options.guest_polling = 0;
+	options.zero_copy = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, "s:",
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		case 0:
+			if (strcmp(lgopts[option_index].name,
+					SOCKET_FILE_KEYWORD) == 0) {
+				ret = parse_socket_arg(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_ID_KEYWORD) == 0) {
+				ret = parse_cryptodev_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_QUEUE_KEYWORD) == 0) {
+				ret = parse_cdev_queue_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					ZERO_COPY_KEYWORD) == 0) {
+				options.zero_copy =
+					RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE;
+			} else if (strcmp(lgopts[option_index].name,
+					POLLING_KEYWORD) == 0) {
+				options.guest_polling = 1;
+			} else {
+				vhost_crypto_usage(prgname);
+				return -EINVAL;
+			}
+			break;
+		default:
+			return -1;
+		}
+	}
+
+	if (options.nb_sockets == 0) {
+		options.socket_files[0] = strdup(DEF_SOCKET_FILE);
+		options.nb_sockets = 1;
+		RTE_LOG(INFO, USER1,
+				"VHOST-CRYPTO: use default socket file %s\n",
+				DEF_SOCKET_FILE);
+	}
+
+	return 0;
+}
+
+static int
+new_device(int vid)
+{
+	char path[PATH_MAX];
+	uint32_t idx, i;
+	int ret;
+
+	ret = rte_vhost_get_ifname(vid, path, PATH_MAX);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot find matched socket\n");
+		return ret;
+	}
+
+	for (idx = 0; idx < options.nb_sockets; idx++) {
+		if (strcmp(path, options.socket_files[idx]) == 0)
+			break;
+	}
+
+	if (idx == options.nb_sockets) {
+		RTE_LOG(ERR, USER1, "Cannot find recorded socket\n");
+		return -ENOENT;
+	}
+
+	for (i = 0; i < 2; i++) {
+		vhost_cycles[i] = 0;
+		last_v_cycles[i] = 0;
+	}
+
+	ret = rte_vhost_crypto_create(vid, info.cid, info.sess_pool,
+			rte_lcore_to_socket_id(info.lcore_id));
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot create vhost crypto\n");
+		return ret;
+	}
+
+	ret = rte_vhost_crypto_set_zero_copy(vid, options.zero_copy);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot %s zero copy feature\n",
+				options.zero_copy == 1 ? "enable" : "disable");
+		return ret;
+	}
+
+	info.vids[idx] = vid;
+	info.initialized[idx] = 1;
+
+	rte_wmb();
+
+	RTE_LOG(INFO, USER1, "New Vhost-crypto Device %s, Device ID %d\n", path,
+			vid);
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	uint32_t i;
+
+	for (i = 0; i < info.nb_vids; i++) {
+		if (vid == info.vids[i])
+			break;
+	}
+
+	if (i == info.nb_vids) {
+		RTE_LOG(ERR, USER1, "Cannot find socket file from list\n");
+		return;
+	}
+
+	info.initialized[i] = 0;
+
+	rte_wmb();
+
+	rte_vhost_crypto_free(vid);
+
+	RTE_LOG(INFO, USER1, "Vhost Crypto Device %i Removed\n", vid);
+}
+
+static const struct vhost_device_ops virtio_crypto_device_ops = {
+	.new_device =  new_device,
+	.destroy_device = destroy_device,
+};
+
+__attribute__((unused))
+static void clrscr(void)
+{
+	system("@cls||clear");
+}
+
+static int
+vhost_crypto_stats_worker(__rte_unused void *arg)
+{
+	uint64_t t_start, t_end, hz = rte_get_tsc_hz();
+	uint64_t threshold = hz * REFRESH_TIME_SEC;
+	double throughput;
+	double diff1, diff2;
+	uint64_t last_outpkt = 0, pkt_diff;
+	uint32_t i;
+
+	RTE_LOG(INFO, USER1, "VHOST-CRYPTO: start stats lcore\n");
+
+	t_start = rte_rdtsc_precise();
+	while (1) {
+		uint64_t interval;
+		uint32_t timeout;
+
+		t_end = rte_rdtsc_precise();
+		interval = t_end - t_start;
+		timeout = (interval > threshold) ? 1 : 0;
+		if (timeout) {
+			t_start = t_end;
+			pkt_diff = outpkt_amount - last_outpkt;
+			last_outpkt = outpkt_amount;
+
+			if (unlikely((pkt_diff) == 0))
+				continue;
+
+			throughput = ((double)pkt_diff / interval) * hz;
+
+			diff1 = vhost_cycles[1] - last_v_cycles[1];
+			diff2 = vhost_cycles[0] - last_v_cycles[0];
+
+			for (i = 0; i < 2; i++)
+				last_v_cycles[i] = vhost_cycles[i];
+
+			printf("%5s%5s%12s%12s\n", "cid", "qid", "reqs/sec",
+					"CPU");
+			printf("%5u%5u%12.1f%12.3f\n", info.cid, info.qid,
+					throughput,
+					(diff1 + diff2) ? (diff1 * 100 /
+						(diff1 + diff2)) : 0);
+			rte_cryptodev_stats_reset(info.cid);
+		}
+	}
+
+	return 0;
+}
+
+static int
+vhost_crypto_worker(__rte_unused void *arg)
+{
+	struct rte_crypto_op *ops[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	struct rte_crypto_op *ops_deq[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	uint32_t nb_inflight_ops = 0;
+	uint16_t nb_callfds;
+	int callfds[VIRTIO_CRYPTO_MAX_NUM_BURST_VQS];
+	uint32_t lcore_id = rte_lcore_id();
+	uint32_t burst_size = MAX_PKT_BURST;
+	uint32_t i, j, k;
+	uint32_t to_fetch, fetched;
+	uint64_t t_start, t_end, interval;
+
+	int ret = 0;
+
+	RTE_LOG(INFO, USER1, "Processing on Core %u started\n", lcore_id);
+
+	for (i = 0; i < NB_VIRTIO_QUEUES; i++) {
+		if (rte_crypto_op_bulk_alloc(info.cop_pool,
+				RTE_CRYPTO_OP_TYPE_SYMMETRIC, ops[i],
+				burst_size) < burst_size) {
+			RTE_LOG(ERR, USER1, "Failed to alloc cops\n");
+			ret = -1;
+			goto exit;
+		}
+	}
+
+	while (1) {
+		for (i = 0; i < info.nb_vids; i++) {
+			if (unlikely(info.initialized[i] == 0))
+				continue;
+
+			for (j = 0; j < NB_VIRTIO_QUEUES; j++) {
+				t_start = rte_rdtsc_precise();
+
+				to_fetch = RTE_MIN(burst_size,
+						(NB_CRYPTO_DESCRIPTORS -
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_fetch_requests(
+						info.vids[i], j, ops[j],
+						to_fetch);
+				nb_inflight_ops += rte_cryptodev_enqueue_burst(
+						info.cid, info.qid, ops[j],
+						fetched);
+				if (unlikely(rte_crypto_op_bulk_alloc(
+						info.cop_pool,
+						RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+						ops[j], fetched) < fetched)) {
+					RTE_LOG(ERR, USER1, "Failed realloc\n");
+					return -1;
+				}
+				t_end = rte_rdtsc_precise();
+				interval = t_end - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+
+				t_start = t_end;
+				fetched = rte_cryptodev_dequeue_burst(
+						info.cid, info.qid,
+						ops_deq[j], RTE_MIN(burst_size,
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_finalize_requests(
+						ops_deq[j], fetched, callfds,
+						&nb_callfds);
+
+				nb_inflight_ops -= fetched;
+				outpkt_amount += fetched;
+
+				if (!options.guest_polling) {
+					for (k = 0; k < nb_callfds; k++)
+						eventfd_write(callfds[k],
+								(eventfd_t)1);
+				}
+
+				rte_mempool_put_bulk(info.cop_pool,
+						(void **)ops_deq[j], fetched);
+				interval = rte_rdtsc_precise() - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+			}
+		}
+	}
+exit:
+	return ret;
+}
+
+
+static void
+unregister_drivers(int socket_num)
+{
+	int ret;
+
+	ret = rte_vhost_driver_unregister(options.socket_files[socket_num]);
+	if (ret != 0)
+		RTE_LOG(ERR, USER1,
+			"Fail to unregister vhost driver for %s.\n",
+			options.socket_files[socket_num]);
+}
+
+int
+main(int argc, char *argv[])
+{
+	struct rte_cryptodev_qp_conf qp_conf = {NB_CRYPTO_DESCRIPTORS};
+	struct rte_cryptodev_config config;
+	struct rte_cryptodev_info dev_info;
+	uint32_t cryptodev_id;
+	uint32_t worker_lcore;
+	uint32_t stats_lcore;
+	char name[128];
+	uint32_t i = 0;
+	int ret;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		return -1;
+	argc -= ret;
+	argv += ret;
+
+	ret = vhost_crypto_parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Failed to parse arguments!\n");
+
+	info.cid = options.cid;
+	info.qid = options.qid;
+
+	worker_lcore = rte_get_next_lcore(0, 1, 0);
+	stats_lcore = rte_get_next_lcore(worker_lcore, 1, 0);
+	if (worker_lcore == RTE_MAX_LCORE || stats_lcore == RTE_MAX_LCORE)
+		rte_exit(EXIT_FAILURE, "Not enough lcore\n");
+
+	cryptodev_id = info.cid;
+	rte_cryptodev_info_get(cryptodev_id, &dev_info);
+	if (dev_info.max_nb_queue_pairs < info.qid + 1) {
+		RTE_LOG(ERR, USER1, "Number of queues cannot over %u",
+				dev_info.max_nb_queue_pairs);
+		goto error_exit;
+	}
+
+	config.nb_queue_pairs = dev_info.max_nb_queue_pairs;
+	config.socket_id = rte_lcore_to_socket_id(worker_lcore);
+
+	ret = rte_cryptodev_configure(cryptodev_id, &config);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to configure cryptodev %u",
+				cryptodev_id);
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "SESS_POOL_%u", worker_lcore);
+	info.sess_pool = rte_mempool_create(name, SESSION_MAP_ENTRIES,
+			rte_cryptodev_get_private_session_size(
+			cryptodev_id), 64, 0, NULL, NULL, NULL, NULL,
+			rte_lcore_to_socket_id(worker_lcore), 0);
+	if (!info.sess_pool) {
+		RTE_LOG(ERR, USER1, "Failed to create mempool");
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "COPPOOL_%u", worker_lcore);
+	info.cop_pool = rte_crypto_op_pool_create(name,
+			RTE_CRYPTO_OP_TYPE_SYMMETRIC, NB_MEMPOOL_OBJS,
+			NB_CACHE_OBJS, 0, rte_lcore_to_socket_id(worker_lcore));
+
+	if (!info.cop_pool) {
+		RTE_LOG(ERR, USER1, "Lcore %u failed to create crypto pool",
+				worker_lcore);
+		ret = -1;
+		goto error_exit;
+	}
+
+	info.nb_vids = options.nb_sockets;
+	for (i = 0; i < MAX_NB_SOCKETS; i++)
+		info.vids[i] = -1;
+
+	for (i = 0; i < dev_info.max_nb_queue_pairs; i++) {
+		ret = rte_cryptodev_queue_pair_setup(cryptodev_id, i,
+				&qp_conf, rte_lcore_to_socket_id(worker_lcore),
+				info.sess_pool);
+		if (ret < 0) {
+			RTE_LOG(ERR, USER1, "Failed to configure qp %u\n",
+					info.cid);
+			goto error_exit;
+		}
+	}
+
+	ret = rte_cryptodev_start(cryptodev_id);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to start cryptodev %u\n", info.cid);
+		goto error_exit;
+	}
+
+	info.cid = cryptodev_id;
+	info.lcore_id = worker_lcore;
+
+	if (rte_eal_remote_launch(vhost_crypto_worker, NULL, worker_lcore)
+			< 0) {
+		RTE_LOG(ERR, USER1, "Failed to start worker lcore");
+		goto error_exit;
+	}
+	if (rte_eal_remote_launch(vhost_crypto_stats_worker, NULL, stats_lcore)
+			< 0) {
+		RTE_LOG(ERR, USER1, "Failed to start stats lcore");
+		goto error_exit;
+	}
+
+
+	for (i = 0; i < options.nb_sockets; i++) {
+		if (rte_vhost_driver_register(options.socket_files[i],
+				RTE_VHOST_USER_DEQUEUE_ZERO_COPY) < 0) {
+			RTE_LOG(ERR, USER1, "socket %s already exists\n",
+					options.socket_files[i]);
+			goto error_exit;
+		}
+
+		rte_vhost_driver_callback_register(options.socket_files[i],
+				&virtio_crypto_device_ops);
+
+		if (rte_vhost_driver_start(options.socket_files[i]) < 0) {
+			RTE_LOG(ERR, USER1, "failed to start vhost driver.\n");
+			goto error_exit;
+		}
+	}
+
+	RTE_LCORE_FOREACH(worker_lcore)
+		rte_eal_wait_lcore(worker_lcore);
+
+	rte_mempool_free(info.sess_pool);
+	rte_mempool_free(info.cop_pool);
+
+	return 0;
+
+error_exit:
+	for (i = 0; i < options.nb_sockets; i++)
+		unregister_drivers(i);
+
+	rte_mempool_free(info.cop_pool);
+	rte_mempool_free(info.sess_pool);
+
+	return -1;
+}
-- 
2.7.4

[PATCH v3 10/10] doc: update prog guide and sample app guide

[Fan Zhang]

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst       | 21 ++++++++
 doc/guides/sample_app_ug/index.rst        |  1 +
 doc/guides/sample_app_ug/vhost_crypto.rst | 84 +++++++++++++++++++++++++++++++
 3 files changed, 106 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 18227b6..9d6680c 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -160,6 +160,27 @@ The following is an overview of some key Vhost API functions:
 
   Receives (dequeues) ``count`` packets from guest, and stored them at ``pkts``.
 
+* ``rte_vhost_crypto_create(vid, cryptodev_id, sess_mempool, socket_id)``
+
+  As an extension of new_device(), this function adds virtio-crypto workload
+  acceleration capability to the device. All crypto workload is processed by
+  DPDK cryptodev with the device ID of ``cryptodev_id``.
+
+* ``rte_vhost_crypto_fetch_requests(vid, queue_id, ops, nb_ops)``
+
+  Receives (dequeues) ``nb_ops`` virtio-crypto requests from guest, parses
+  them to DPDK Crypto Operations, and fills the ``ops`` with parsing results.
+
+* ``rte_vhost_crypto_finalize_requests(queue_id, ops, nb_ops)``
+
+  After the ``ops`` are dequeued from Cryptodev, finalizes the jobs and
+  notifies the guest(s).
+
+* ``rte_vhost_user_register_extern_backend(vid, extern_data, ops)``
+
+  This function register private data, and pre and post vhost-user message
+  handlers.
+
 Vhost-user Implementations
 --------------------------
 
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index e87afda..57e8354 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -44,6 +44,7 @@ Sample Applications User Guides
     vmdq_dcb_forwarding
     vhost
     vhost_scsi
+    vhost_crypto
     netmap_compatibility
     ip_pipeline
     test_pipeline
diff --git a/doc/guides/sample_app_ug/vhost_crypto.rst b/doc/guides/sample_app_ug/vhost_crypto.rst
new file mode 100644
index 0000000..8e6cb2a
--- /dev/null
+++ b/doc/guides/sample_app_ug/vhost_crypto.rst
@@ -0,0 +1,84 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2017-2018 Intel Corporation.
+
+
+Vhost_Crypto Sample Application
+===============================
+
+The vhost_crypto sample application implemented a simple Crypto device,
+which used as the  backend of Qemu vhost-user-crypto device. Similar with
+vhost-user-net and vhost-user-scsi device, the sample application used
+domain socket to communicate with Qemu, and the virtio ring was processed
+by vhost_crypto sample application.
+
+Testing steps
+-------------
+
+This section shows the steps how to start a VM with the crypto device as
+fast data path for critical application.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``examples`` sub-directory.
+
+Start the vhost_crypto example
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+    ./vhost_crypto [EAL options] -- [--socket-file PATH]
+        [--cdev-id ID] [--cdev-queue-id ID] [--zero-copy] [--guest-polling]
+
+where,
+
+* socket-file PATH: the path of UNIX socket file to be created, multiple
+  instances of this config item is supported. Upon absence of this item,
+  the default socket-file `/tmp/vhost_crypto1.socket` is used.
+
+* cdev-id ID: the target DPDK Cryptodev's ID to process the actual crypto
+  workload. Upon absence of this item the default value of `0` will be used.
+  For details of DPDK Cryptodev, please refer to DPDK Cryptodev Library
+  Programmers' Guide.
+
+* cdev-queue-id ID: the target DPDK Cryptodev's queue ID to process the
+  actual crypto workload. Upon absence of this item the default value of `0`
+  will be used. For details of DPDK Cryptodev, please refer to DPDK Cryptodev
+  Library Programmers' Guide.
+
+* zero-copy: the presence of this item means the ZERO-COPY feature will be
+  enabled. Otherwise it is disabled. PLEASE NOTE the ZERO-COPY feature is still
+  in experimental stage and may cause the problem like segmentation fault. If
+  the user wants to use LKCF in the guest, this feature shall be turned off.
+
+* guest-polling: the presence of this item means the application assumes the
+  guest works in polling mode, thus will NOT notify the guest completion of
+  processing.
+
+The application requires that crypto devices capable of performing
+the specified crypto operation are available on application initialization.
+This means that HW crypto device/s must be bound to a DPDK driver or
+a SW crypto device/s (virtual crypto PMD) must be created (using --vdev).
+
+.. _vhost_crypto_app_run_vm:
+
+Start the VM
+~~~~~~~~~~~~
+
+.. code-block:: console
+
+    qemu-system-x86_64 -machine accel=kvm \
+        -m $mem -object memory-backend-file,id=mem,size=$mem,\
+        mem-path=/dev/hugepages,share=on -numa node,memdev=mem \
+        -drive file=os.img,if=none,id=disk \
+        -device ide-hd,drive=disk,bootindex=0 \
+        -chardev socket,id={chardev_id},path={PATH} \
+        -object cryptodev-vhost-user,id={obj_id},chardev={chardev_id} \
+        -device virtio-crypto-pci,id={dev_id},cryptodev={obj_id} \
+        ...
+
+.. note::
+    You must check whether your Qemu can support "vhost-user-crypto" or not.
+
-- 
2.7.4

Re: [PATCH v3 01/10] lib/librte_vhost: add external backend support

[Tan, Jianfeng]

It's interesting that we add some new APIs to be used by the 
lib/librte_vhost/ itself. I can understand as we planned to not put 
vhost crypto into the lib.

As vhost crypto is not a real "external backend", we could ask opinion 
of a real external backend if these are really necessary. pre and post 
message handlers would be OK. But do we really need register private 
data from external backend? @Changpeng @Pawel @Dariusz @Tomasz.

BTW, external backend sounds a little exclusive :-), does extended 
backend sound better?


On 3/26/2018 5:51 PM, Fan Zhang wrote:
> This patch adds external backend support to vhost library. The patch provides
> new APIs for the external backend to register private data, plus pre and post
> vhost-user message handlers.
>
> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> ---
>   lib/librte_vhost/rte_vhost.h  | 45 ++++++++++++++++++++++++++++++++++++++++++-
>   lib/librte_vhost/vhost.c      | 23 +++++++++++++++++++++-
>   lib/librte_vhost/vhost.h      |  8 ++++++--
>   lib/librte_vhost/vhost_user.c | 29 +++++++++++++++++++++++++---
>   4 files changed, 98 insertions(+), 7 deletions(-)
>
> diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h
> index d332069..591b731 100644
> --- a/lib/librte_vhost/rte_vhost.h
> +++ b/lib/librte_vhost/rte_vhost.h
> @@ -1,5 +1,5 @@
>   /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2017 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>    */
>   
>   #ifndef _RTE_VHOST_H_
> @@ -88,6 +88,33 @@ struct vhost_device_ops {
>   };
>   
>   /**
> + * function prototype for external virtio device to handler device specific

handler -> handle

> + * vhost user messages
> + *
> + * @param extern_data
> + *  private data for external backend

There is not such parameter in below function type.

> + * @param msg
> + *  Message pointer
> + * @param payload
> + *  Message payload

Ditto.

> + * @param require_reply
> + *  If the handler requires sending a reply, this varaible shall be written 1,
> + *  otherwise 0
> + * @return
> + *  0 on success, -1 on failure
> + */
> +typedef int (*rte_vhost_msg_handler)(int vid, void *msg,
> +		uint32_t *require_reply);
> +
> +/**
> + * pre and post vhost user message handlers
> + */
> +struct rte_vhost_user_dev_extern_ops {

Considering the original vhost_device_ops, does vhost_user_extern_ops 
sound better?

> +	rte_vhost_msg_handler pre_vhost_user_msg_handler;
> +	rte_vhost_msg_handler post_vhost_user_msg_handler;
> +};
> +
> +/**
>    * Convert guest physical address to host virtual address
>    *
>    * @param mem
> @@ -434,6 +461,22 @@ int rte_vhost_vring_call(int vid, uint16_t vring_idx);
>    */
>   uint32_t rte_vhost_rx_queue_count(int vid, uint16_t qid);
>   
> +/**
> + * register external vhost backend
> + *
> + * @param vid
> + *  vhost device ID
> + * @param extern_data
> + *  private data for external backend
> + * @param ops
> + *  ops that process external vhost user messages
> + * @return
> + *  0 on success, -1 on failure
> + */
> +int
> +rte_vhost_user_register_extern_backend(int vid, void *extern_data,
> +		struct rte_vhost_user_dev_extern_ops *ops);

Considering the original rte_vhost_driver_callback_register, does 
rte_vhost_message_handler_register sound better?

For extern_data, as mentioned in the head, let's discuss if it's 
necessary to be registered through API.

> +
>   #ifdef __cplusplus
>   }
>   #endif
> diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
> index a407067..0932537 100644
> --- a/lib/librte_vhost/vhost.c
> +++ b/lib/librte_vhost/vhost.c
> @@ -1,5 +1,5 @@
>   /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2016 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>    */
>   
>   #include <linux/vhost.h>
> @@ -627,3 +627,24 @@ rte_vhost_rx_queue_count(int vid, uint16_t qid)
>   
>   	return *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx;
>   }
> +
> +int
> +rte_vhost_user_register_extern_backend(int vid, void *extern_data,
> +		struct rte_vhost_user_dev_extern_ops *ops)
> +{
> +	struct virtio_net *dev;

Do we want to rename this internal structure to something like 
vhost_dev, if it contains not only information for net?

> +
> +	dev = get_device(vid);
> +	if (dev == NULL)
> +		return -1;
> +
> +	dev->extern_data = extern_data;
> +	if (ops) {
> +		dev->extern_ops.pre_vhost_user_msg_handler =
> +				ops->pre_vhost_user_msg_handler;
> +		dev->extern_ops.post_vhost_user_msg_handler =
> +				ops->post_vhost_user_msg_handler;
> +	}
> +
> +	return 0;
> +}
> diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
> index d947bc9..6aaa46c 100644
> --- a/lib/librte_vhost/vhost.h
> +++ b/lib/librte_vhost/vhost.h
> @@ -1,5 +1,5 @@
>   /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2014 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>    */
>   
>   #ifndef _VHOST_NET_CDEV_H_
> @@ -241,8 +241,12 @@ struct virtio_net {
>   	struct guest_page       *guest_pages;
>   
>   	int			slave_req_fd;
> -} __rte_cache_aligned;
>   
> +	/* private data for external virtio device */
> +	void			*extern_data;
> +	/* pre and post vhost user message handlers for externel backend */
> +	struct rte_vhost_user_dev_extern_ops extern_ops;
> +} __rte_cache_aligned;
>   
>   #define VHOST_LOG_PAGE	4096
>   
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 90ed211..c064cb3 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -1,5 +1,5 @@
>   /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2016 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>    */
>   
>   #include <stdint.h>
> @@ -50,6 +50,8 @@ static const char *vhost_message_str[VHOST_USER_MAX] = {
>   	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
>   	[VHOST_USER_SET_SLAVE_REQ_FD]  = "VHOST_USER_SET_SLAVE_REQ_FD",
>   	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
> +	[VHOST_USER_CRYPTO_CREATE_SESS] = "VHOST_USER_CRYPTO_CREATE_SESS",
> +	[VHOST_USER_CRYPTO_CLOSE_SESS] = "VHOST_USER_CRYPTO_CLOSE_SESS",

Please leave this patch device agnostic. Put these into crypto related 
patches.

>   };
>   
>   static uint64_t
> @@ -1379,6 +1381,18 @@ vhost_user_msg_handler(int vid, int fd)
>   
>   	}
>   
> +	if (dev->extern_ops.pre_vhost_user_msg_handler) {
> +		uint32_t need_reply;
> +
> +		ret = (*dev->extern_ops.pre_vhost_user_msg_handler)(dev->vid,

We have a variable vid, why use dev->vid?

> +				(void *)&msg, &need_reply);
> +		if (ret < 0)
> +			goto skip_to_reply;
> +
> +		if (need_reply)
> +			send_vhost_reply(fd, &msg);

Do we have case that, if device handles that, we don't need to common 
handle and post handle below? In other words, how to handle overlapping 
of message handle?

> +	}
> +
>   	switch (msg.request.master) {
>   	case VHOST_USER_GET_FEATURES:
>   		msg.payload.u64 = vhost_user_get_features(dev);
> @@ -1477,11 +1491,20 @@ vhost_user_msg_handler(int vid, int fd)
>   		break;
>   
>   	default:
> -		ret = -1;
> -		break;
> +		if (dev->extern_ops.post_vhost_user_msg_handler) {

Do we allow overlapping of common and post handle?

> +			uint32_t need_reply;
>   
> +			ret = (*dev->extern_ops.post_vhost_user_msg_handler)(
> +					dev->vid, (void *)&msg, &need_reply);
> +
> +			if (need_reply)
> +				send_vhost_reply(fd, &msg);
> +		} else
> +			ret = -1;
> +		break;
>   	}
>   
> +skip_to_reply:
>   	if (unlock_required)
>   		vhost_user_unlock_all_queue_pairs(dev);
>   

Re: [PATCH v3 01/10] lib/librte_vhost: add external backend support

[Liu, Changpeng]

> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Thursday, March 29, 2018 10:11 AM
> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org
> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Liu, Changpeng
> <changpeng.liu@intel.com>; Wodkowski, PawelX
> <pawelx.wodkowski@intel.com>; Stojaczyk, DariuszX
> <dariuszx.stojaczyk@intel.com>; Kulasek, TomaszX <tomaszx.kulasek@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v3 01/10] lib/librte_vhost: add external backend
> support
> 
> 
> It's interesting that we add some new APIs to be used by the
> lib/librte_vhost/ itself. I can understand as we planned to not put
> vhost crypto into the lib.
> 
> As vhost crypto is not a real "external backend", we could ask opinion
> of a real external backend if these are really necessary. pre and post
> message handlers would be OK. But do we really need register private
> data from external backend? @Changpeng @Pawel @Dariusz @Tomasz.
For now I'm not sure whether we need a private data structure. But why
put post_vhost_user_msg_handler into default section ?
> 
> BTW, external backend sounds a little exclusive :-), does extended
> backend sound better?
> 
> 
> On 3/26/2018 5:51 PM, Fan Zhang wrote:
> > This patch adds external backend support to vhost library. The patch provides
> > new APIs for the external backend to register private data, plus pre and post
> > vhost-user message handlers.
> >
> > Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> > ---
> >   lib/librte_vhost/rte_vhost.h  | 45
> ++++++++++++++++++++++++++++++++++++++++++-
> >   lib/librte_vhost/vhost.c      | 23 +++++++++++++++++++++-
> >   lib/librte_vhost/vhost.h      |  8 ++++++--
> >   lib/librte_vhost/vhost_user.c | 29 +++++++++++++++++++++++++---
> >   4 files changed, 98 insertions(+), 7 deletions(-)
> >
> > diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h
> > index d332069..591b731 100644
> > --- a/lib/librte_vhost/rte_vhost.h
> > +++ b/lib/librte_vhost/rte_vhost.h
> > @@ -1,5 +1,5 @@
> >   /* SPDX-License-Identifier: BSD-3-Clause
> > - * Copyright(c) 2010-2017 Intel Corporation
> > + * Copyright(c) 2010-2018 Intel Corporation
> >    */
> >
> >   #ifndef _RTE_VHOST_H_
> > @@ -88,6 +88,33 @@ struct vhost_device_ops {
> >   };
> >
> >   /**
> > + * function prototype for external virtio device to handler device specific
> 
> handler -> handle
> 
> > + * vhost user messages
> > + *
> > + * @param extern_data
> > + *  private data for external backend
> 
> There is not such parameter in below function type.
> 
> > + * @param msg
> > + *  Message pointer
> > + * @param payload
> > + *  Message payload
> 
> Ditto.
> 
> > + * @param require_reply
> > + *  If the handler requires sending a reply, this varaible shall be written 1,
> > + *  otherwise 0
> > + * @return
> > + *  0 on success, -1 on failure
> > + */
> > +typedef int (*rte_vhost_msg_handler)(int vid, void *msg,
> > +		uint32_t *require_reply);
> > +
> > +/**
> > + * pre and post vhost user message handlers
> > + */
> > +struct rte_vhost_user_dev_extern_ops {
> 
> Considering the original vhost_device_ops, does vhost_user_extern_ops
> sound better?
> 
> > +	rte_vhost_msg_handler pre_vhost_user_msg_handler;
> > +	rte_vhost_msg_handler post_vhost_user_msg_handler;
> > +};
> > +
> > +/**
> >    * Convert guest physical address to host virtual address
> >    *
> >    * @param mem
> > @@ -434,6 +461,22 @@ int rte_vhost_vring_call(int vid, uint16_t vring_idx);
> >    */
> >   uint32_t rte_vhost_rx_queue_count(int vid, uint16_t qid);
> >
> > +/**
> > + * register external vhost backend
> > + *
> > + * @param vid
> > + *  vhost device ID
> > + * @param extern_data
> > + *  private data for external backend
> > + * @param ops
> > + *  ops that process external vhost user messages
> > + * @return
> > + *  0 on success, -1 on failure
> > + */
> > +int
> > +rte_vhost_user_register_extern_backend(int vid, void *extern_data,
> > +		struct rte_vhost_user_dev_extern_ops *ops);
> 
> Considering the original rte_vhost_driver_callback_register, does
> rte_vhost_message_handler_register sound better?
> 
> For extern_data, as mentioned in the head, let's discuss if it's
> necessary to be registered through API.
> 
> > +
> >   #ifdef __cplusplus
> >   }
> >   #endif
> > diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
> > index a407067..0932537 100644
> > --- a/lib/librte_vhost/vhost.c
> > +++ b/lib/librte_vhost/vhost.c
> > @@ -1,5 +1,5 @@
> >   /* SPDX-License-Identifier: BSD-3-Clause
> > - * Copyright(c) 2010-2016 Intel Corporation
> > + * Copyright(c) 2010-2018 Intel Corporation
> >    */
> >
> >   #include <linux/vhost.h>
> > @@ -627,3 +627,24 @@ rte_vhost_rx_queue_count(int vid, uint16_t qid)
> >
> >   	return *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx;
> >   }
> > +
> > +int
> > +rte_vhost_user_register_extern_backend(int vid, void *extern_data,
> > +		struct rte_vhost_user_dev_extern_ops *ops)
> > +{
> > +	struct virtio_net *dev;
> 
> Do we want to rename this internal structure to something like
> vhost_dev, if it contains not only information for net?
> 
> > +
> > +	dev = get_device(vid);
> > +	if (dev == NULL)
> > +		return -1;
> > +
> > +	dev->extern_data = extern_data;
> > +	if (ops) {
> > +		dev->extern_ops.pre_vhost_user_msg_handler =
> > +				ops->pre_vhost_user_msg_handler;
> > +		dev->extern_ops.post_vhost_user_msg_handler =
> > +				ops->post_vhost_user_msg_handler;
> > +	}
> > +
> > +	return 0;
> > +}
> > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
> > index d947bc9..6aaa46c 100644
> > --- a/lib/librte_vhost/vhost.h
> > +++ b/lib/librte_vhost/vhost.h
> > @@ -1,5 +1,5 @@
> >   /* SPDX-License-Identifier: BSD-3-Clause
> > - * Copyright(c) 2010-2014 Intel Corporation
> > + * Copyright(c) 2010-2018 Intel Corporation
> >    */
> >
> >   #ifndef _VHOST_NET_CDEV_H_
> > @@ -241,8 +241,12 @@ struct virtio_net {
> >   	struct guest_page       *guest_pages;
> >
> >   	int			slave_req_fd;
> > -} __rte_cache_aligned;
> >
> > +	/* private data for external virtio device */
> > +	void			*extern_data;
> > +	/* pre and post vhost user message handlers for externel backend */
> > +	struct rte_vhost_user_dev_extern_ops extern_ops;
> > +} __rte_cache_aligned;
> >
> >   #define VHOST_LOG_PAGE	4096
> >
> > diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> > index 90ed211..c064cb3 100644
> > --- a/lib/librte_vhost/vhost_user.c
> > +++ b/lib/librte_vhost/vhost_user.c
> > @@ -1,5 +1,5 @@
> >   /* SPDX-License-Identifier: BSD-3-Clause
> > - * Copyright(c) 2010-2016 Intel Corporation
> > + * Copyright(c) 2010-2018 Intel Corporation
> >    */
> >
> >   #include <stdint.h>
> > @@ -50,6 +50,8 @@ static const char *vhost_message_str[VHOST_USER_MAX]
> = {
> >   	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
> >   	[VHOST_USER_SET_SLAVE_REQ_FD]  =
> "VHOST_USER_SET_SLAVE_REQ_FD",
> >   	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
> > +	[VHOST_USER_CRYPTO_CREATE_SESS] =
> "VHOST_USER_CRYPTO_CREATE_SESS",
> > +	[VHOST_USER_CRYPTO_CLOSE_SESS] =
> "VHOST_USER_CRYPTO_CLOSE_SESS",
> 
> Please leave this patch device agnostic. Put these into crypto related
> patches.
> 
> >   };
> >
> >   static uint64_t
> > @@ -1379,6 +1381,18 @@ vhost_user_msg_handler(int vid, int fd)
> >
> >   	}
> >
> > +	if (dev->extern_ops.pre_vhost_user_msg_handler) {
> > +		uint32_t need_reply;
> > +
> > +		ret = (*dev->extern_ops.pre_vhost_user_msg_handler)(dev->vid,
> 
> We have a variable vid, why use dev->vid?
> 
> > +				(void *)&msg, &need_reply);
> > +		if (ret < 0)
> > +			goto skip_to_reply;
> > +
> > +		if (need_reply)
> > +			send_vhost_reply(fd, &msg);
> 
> Do we have case that, if device handles that, we don't need to common
> handle and post handle below? In other words, how to handle overlapping
> of message handle?
> 
> > +	}
> > +
> >   	switch (msg.request.master) {
> >   	case VHOST_USER_GET_FEATURES:
> >   		msg.payload.u64 = vhost_user_get_features(dev);
> > @@ -1477,11 +1491,20 @@ vhost_user_msg_handler(int vid, int fd)
> >   		break;
> >
> >   	default:
> > -		ret = -1;
> > -		break;
> > +		if (dev->extern_ops.post_vhost_user_msg_handler) {
> 
> Do we allow overlapping of common and post handle?
> 
> > +			uint32_t need_reply;
> >
> > +			ret = (*dev->extern_ops.post_vhost_user_msg_handler)(
> > +					dev->vid, (void *)&msg, &need_reply);
> > +
> > +			if (need_reply)
> > +				send_vhost_reply(fd, &msg);
> > +		} else
> > +			ret = -1;
> > +		break;
> >   	}
> >
> > +skip_to_reply:
> >   	if (unlock_required)
> >   		vhost_user_unlock_all_queue_pairs(dev);
> >

[PATCH v4 0/8] vhost: intdroduce vhost user crypto backend

[Fan Zhang]

This patchset adds crypto backend suppport to vhost library
including a proof-of-concept sample application. The implementation
follows the virtio-crypto specification and have been tested
with qemu 2.11.50 (with several patches applied, detailed later)
with Fedora 24 running in the frontend.

The vhost_crypto library acts as a "bridge" method that translate
the virtio-crypto crypto requests to DPDK crypto operations, so it
is purely software implementation. However it does require the user
to provide the DPDK Cryptodev ID so it knows how to handle the
virtio-crypto session creation and deletion mesages.

Currently the implementation supports AES-CBC-128 and HMAC-SHA1
cipher only/chaining modes and does not support sessionless mode
yet. The guest can use standard virtio-crypto driver to set up
session and sends encryption/decryption requests to backend. The
vhost-crypto sample application provided in this patchset will
do the actual crypto work.

The following steps are involved to enable vhost-crypto support.

In the host:
1. Download the qemu source code.

2. Recompile your qemu with vhost-crypto option enabled.

3. Apply this patchset to latest DPDK code and recompile DPDK.

4. Compile and run vhost-crypto sample application.

./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
 --socket-mem 2048,2048

Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
not available. The sample application requires 2 lcores: 1 master and 1
worker. The application will create a UNIX socket file
/tmp/vhost_crypto1.socket.

5. Start your qemu application. Here is my command:

qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
-smp 2 -m 1G -hda ~/path-to-your/image.qcow \
-object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
-mem-prealloc -numa node,memdev=mem -chardev \
socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
-object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
-device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0

6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
default. You shall see the following logs in your demsg:

[   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
[   17.611083] virtio_crypto: module verification failed: signature and/or ...
[   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
[   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
[   18.376100] virtio_crypto virtio0: Accelerator is ready

The virtio_crypto driver in the guest is now up and running.

7. The rest steps can be as same as the Testing section in
https://wiki.qemu.org/Features/VirtioCrypto

8. It is possible to use DPDK Virtio Crypto PMD
(https://dpdk.org/dev/patchwork/patch/36471/) in the guest to work with
this patchset to achieve optimal performance.

v4:
- Changed external vhost backend ops register API.
- Fixed a bug.

v3:
- Changed external vhost backend private data and message handling
- Added experimental tag to rte_vhost_crypto_set_zero_copy()

v2:
- Moved vhost_crypto_data_req data from crypto op to source mbuf.
- Removed ZERO-COPY flag from config option and make it run-timely changeable.
- Guest-polling mode possible.
- Simplified vring descriptor access procedure.
- Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.

Fan Zhang (8):
  lib/librte_vhost: add external backend support
  lib/librte_vhost: add virtio-crypto user message structure
  lib/librte_vhost: add session message handler
  lib/librte_vhost: add request handler
  lib/librte_vhost: add public function implementation
  lib/librte_vhost: update makefile
  examples/vhost_crypto: add vhost crypto sample application
  doc: update for vhost crypto support

 doc/guides/prog_guide/vhost_lib.rst       |   21 +
 doc/guides/rel_notes/release_18_05.rst    |   10 +
 doc/guides/sample_app_ug/index.rst        |    1 +
 doc/guides/sample_app_ug/vhost_crypto.rst |   84 ++
 examples/vhost_crypto/Makefile            |   32 +
 examples/vhost_crypto/main.c              |  541 ++++++++++++
 examples/vhost_crypto/meson.build         |   14 +
 lib/librte_vhost/Makefile                 |    6 +-
 lib/librte_vhost/meson.build              |    8 +-
 lib/librte_vhost/rte_vhost.h              |   64 +-
 lib/librte_vhost/rte_vhost_crypto.h       |  125 +++
 lib/librte_vhost/rte_vhost_version.map    |   11 +
 lib/librte_vhost/vhost.c                  |   17 +-
 lib/librte_vhost/vhost.h                  |    8 +-
 lib/librte_vhost/vhost_crypto.c           | 1285 +++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c             |   33 +-
 lib/librte_vhost/vhost_user.h             |   36 +-
 17 files changed, 2281 insertions(+), 15 deletions(-)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 examples/vhost_crypto/meson.build
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
 create mode 100644 lib/librte_vhost/vhost_crypto.c

-- 
2.7.4

[PATCH v4 1/8] lib/librte_vhost: add external backend support

[Fan Zhang]

This patch adds external backend support to vhost library. The patch provides
new APIs for the external backend to register pre and post vhost-user message
handlers.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost.h           | 64 +++++++++++++++++++++++++++++++++-
 lib/librte_vhost/rte_vhost_version.map |  6 ++++
 lib/librte_vhost/vhost.c               | 17 ++++++++-
 lib/librte_vhost/vhost.h               |  8 +++--
 lib/librte_vhost/vhost_user.c          | 33 +++++++++++++++++-
 5 files changed, 123 insertions(+), 5 deletions(-)

diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h
index d332069..b902c44 100644
--- a/lib/librte_vhost/rte_vhost.h
+++ b/lib/librte_vhost/rte_vhost.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2017 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _RTE_VHOST_H_
@@ -88,6 +88,55 @@ struct vhost_device_ops {
 };
 
 /**
+ * function prototype for the vhost backend to handler specific vhost user
+ * messages prior to the master message handling
+ *
+ * @param vid
+ *  vhost device id
+ * @param msg
+ *  Message pointer.
+ * @param payload
+ *  Message payload.
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0.
+ * @param skip_master
+ *  If the handler requires skipping the master message handling, this variable
+ *  shall be written 1, otherwise 0.
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*rte_vhost_msg_pre_handle)(int vid, void *msg,
+		uint32_t *require_reply, uint32_t *skip_master);
+
+/**
+ * function prototype for the vhost backend to handler specific vhost user
+ * messages after the master message handling is done
+ *
+ * @param vid
+ *  vhost device id
+ * @param msg
+ *  Message pointer.
+ * @param payload
+ *  Message payload.
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0.
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
+		uint32_t *require_reply);
+
+/**
+ * pre and post vhost user message handlers
+ */
+struct vhost_user_extern_ops {
+	rte_vhost_msg_pre_handle pre_msg_handle;
+	rte_vhost_msg_post_handle post_msg_handle;
+};
+
+/**
  * Convert guest physical address to host virtual address
  *
  * @param mem
@@ -434,6 +483,19 @@ int rte_vhost_vring_call(int vid, uint16_t vring_idx);
  */
 uint32_t rte_vhost_rx_queue_count(int vid, uint16_t qid);
 
+/**
+ * register external vhost backend
+ *
+ * @param vid
+ *  vhost device ID
+ * @param ops
+ *  ops that process external vhost user messages
+ * @return
+ *  0 on success, -1 on failure
+ */
+int
+rte_vhost_user_register_extern_ops(int vid, struct vhost_user_extern_ops *ops);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map
index df01031..91bf9f0 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -59,3 +59,9 @@ DPDK_18.02 {
 	rte_vhost_vring_call;
 
 } DPDK_17.08;
+
+DPDK_18.05 {
+	global:
+
+	rte_vhost_user_register_extern_ops;
+} DPDK_18.02;
diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
index a407067..80af341 100644
--- a/lib/librte_vhost/vhost.c
+++ b/lib/librte_vhost/vhost.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #include <linux/vhost.h>
@@ -627,3 +627,18 @@ rte_vhost_rx_queue_count(int vid, uint16_t qid)
 
 	return *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx;
 }
+
+int
+rte_vhost_user_register_extern_ops(int vid, struct vhost_user_extern_ops *ops)
+{
+	struct virtio_net *dev;
+
+	dev = get_device(vid);
+	if (dev == NULL)
+		return -1;
+
+	if (ops)
+		rte_memcpy(&dev->extern_ops, ops, sizeof(*ops));
+
+	return 0;
+}
diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
index d947bc9..2072b88 100644
--- a/lib/librte_vhost/vhost.h
+++ b/lib/librte_vhost/vhost.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_CDEV_H_
@@ -241,8 +241,12 @@ struct virtio_net {
 	struct guest_page       *guest_pages;
 
 	int			slave_req_fd;
-} __rte_cache_aligned;
 
+	/* private data for external virtio device */
+	void			*extern_data;
+	/* pre and post vhost user message handlers for externel backend */
+	struct vhost_user_extern_ops extern_ops;
+} __rte_cache_aligned;
 
 #define VHOST_LOG_PAGE	4096
 
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 90ed211..ede8a5e 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #include <stdint.h>
@@ -50,6 +50,8 @@ static const char *vhost_message_str[VHOST_USER_MAX] = {
 	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
 	[VHOST_USER_SET_SLAVE_REQ_FD]  = "VHOST_USER_SET_SLAVE_REQ_FD",
 	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
+	[VHOST_USER_CRYPTO_CREATE_SESS] = "VHOST_USER_CRYPTO_CREATE_SESS",
+	[VHOST_USER_CRYPTO_CLOSE_SESS] = "VHOST_USER_CRYPTO_CLOSE_SESS",
 };
 
 static uint64_t
@@ -1302,6 +1304,7 @@ vhost_user_msg_handler(int vid, int fd)
 	struct VhostUserMsg msg;
 	int ret;
 	int unlock_required = 0;
+	uint32_t skip_master = 0;
 
 	dev = get_device(vid);
 	if (dev == NULL)
@@ -1379,6 +1382,21 @@ vhost_user_msg_handler(int vid, int fd)
 
 	}
 
+	if (dev->extern_ops.pre_msg_handle) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.pre_msg_handle)(dev->vid,
+				(void *)&msg, &need_reply, &skip_master);
+		if (ret < 0)
+			goto skip_to_reply;
+
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
+	}
+
+	if (skip_master)
+		goto skip_to_post_handle;
+
 	switch (msg.request.master) {
 	case VHOST_USER_GET_FEATURES:
 		msg.payload.u64 = vhost_user_get_features(dev);
@@ -1479,9 +1497,22 @@ vhost_user_msg_handler(int vid, int fd)
 	default:
 		ret = -1;
 		break;
+	}
+
+skip_to_post_handle:
+	if (dev->extern_ops.post_msg_handle) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.post_msg_handle)(
+				dev->vid, (void *)&msg, &need_reply);
+		if (ret < 0)
+			goto skip_to_reply;
 
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
 	}
 
+skip_to_reply:
 	if (unlock_required)
 		vhost_user_unlock_all_queue_pairs(dev);
 
-- 
2.7.4

[PATCH v4 2/8] lib/librte_vhost: add virtio-crypto user message structure

[Fan Zhang]

This patch adds virtio-crypto spec user message structure to
vhost_user.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_user.h | 36 ++++++++++++++++++++++++++++++++----
 1 file changed, 32 insertions(+), 4 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h
index d4bd604..48cdb24 100644
--- a/lib/librte_vhost/vhost_user.h
+++ b/lib/librte_vhost/vhost_user.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_USER_H
@@ -20,13 +20,15 @@
 #define VHOST_USER_PROTOCOL_F_REPLY_ACK	3
 #define VHOST_USER_PROTOCOL_F_NET_MTU 4
 #define VHOST_USER_PROTOCOL_F_SLAVE_REQ 5
+#define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7
 
 #define VHOST_USER_PROTOCOL_FEATURES	((1ULL << VHOST_USER_PROTOCOL_F_MQ) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) |\
 					 (1ULL << VHOST_USER_PROTOCOL_F_RARP) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_NET_MTU) | \
-					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ))
+					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ) | \
+					 (1ULL << VHOST_USER_PROTOCOL_F_CRYPTO_SESSION))
 
 typedef enum VhostUserRequest {
 	VHOST_USER_NONE = 0,
@@ -52,7 +54,9 @@ typedef enum VhostUserRequest {
 	VHOST_USER_NET_SET_MTU = 20,
 	VHOST_USER_SET_SLAVE_REQ_FD = 21,
 	VHOST_USER_IOTLB_MSG = 22,
-	VHOST_USER_MAX
+	VHOST_USER_CRYPTO_CREATE_SESS = 26,
+	VHOST_USER_CRYPTO_CLOSE_SESS = 27,
+	VHOST_USER_MAX = 28
 } VhostUserRequest;
 
 typedef enum VhostUserSlaveRequest {
@@ -79,6 +83,30 @@ typedef struct VhostUserLog {
 	uint64_t mmap_offset;
 } VhostUserLog;
 
+/* Comply with Cryptodev-Linux */
+#define VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH	512
+#define VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH	64
+
+/* Same structure as vhost-user backend session info */
+typedef struct VhostUserCryptoSessionParam {
+	int64_t session_id;
+	uint32_t op_code;
+	uint32_t cipher_algo;
+	uint32_t cipher_key_len;
+	uint32_t hash_algo;
+	uint32_t digest_len;
+	uint32_t auth_key_len;
+	uint32_t aad_len;
+	uint8_t op_type;
+	uint8_t dir;
+	uint8_t hash_mode;
+	uint8_t chaining_dir;
+	uint8_t *ciphe_key;
+	uint8_t *auth_key;
+	uint8_t cipher_key_buf[VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH];
+	uint8_t auth_key_buf[VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH];
+} VhostUserCryptoSessionParam;
+
 typedef struct VhostUserMsg {
 	union {
 		VhostUserRequest master;
@@ -99,6 +127,7 @@ typedef struct VhostUserMsg {
 		VhostUserMemory memory;
 		VhostUserLog    log;
 		struct vhost_iotlb_msg iotlb;
+		VhostUserCryptoSessionParam crypto_session;
 	} payload;
 	int fds[VHOST_MEMORY_MAX_NREGIONS];
 } __attribute((packed)) VhostUserMsg;
@@ -108,7 +137,6 @@ typedef struct VhostUserMsg {
 /* The version of the protocol we support */
 #define VHOST_USER_VERSION    0x1
 
-
 /* vhost_user.c */
 int vhost_user_msg_handler(int vid, int fd);
 int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm);
-- 
2.7.4

[PATCH v4 3/8] lib/librte_vhost: add session message handler

[Fan Zhang]

This patch adds session message handler to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 428 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 428 insertions(+)
 create mode 100644 lib/librte_vhost/vhost_crypto.c

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
new file mode 100644
index 0000000..c639b20
--- /dev/null
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include <stdbool.h>
+
+#include <rte_malloc.h>
+#include <rte_jhash.h>
+#include <rte_mbuf.h>
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#include <rte_hexdump.h>
+#endif
+#include "vhost.h"
+#include "vhost_user.h"
+#include "rte_vhost_crypto.h"
+
+#define NB_MEMPOOL_OBJS			(1024)
+#define NB_CRYPTO_DESCRIPTORS		(1024)
+#define NB_CACHE_OBJS			(128)
+
+#define SESSION_MAP_ENTRIES		(1024) /**< Max nb sessions per vdev */
+#define MAX_KEY_SIZE			(32)
+#define VHOST_CRYPTO_MAX_IV_LEN		(16)
+#define MAX_COUNT_DOWN_TIMES		(100)
+
+#define INHDR_LEN		(sizeof(struct virtio_crypto_inhdr))
+#define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
+				sizeof(struct rte_crypto_sym_op))
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+
+#define VC_LOG_DBG(fmt, args...)				\
+	RTE_LOG(DEBUG, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#else
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_DBG(fmt, args...)
+#endif
+
+#define VIRTIO_CRYPTO_FEATURES ((1 << VIRTIO_F_NOTIFY_ON_EMPTY) |	\
+		(1 << VIRTIO_RING_F_INDIRECT_DESC) |			\
+		(1 << VIRTIO_RING_F_EVENT_IDX) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_CIPHER) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_HASH) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_MAC) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_AEAD) |			\
+		(1 << VIRTIO_NET_F_CTRL_VQ))
+
+
+#define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
+
+/* Macro to get the buffer at the end of rte_crypto_op */
+#define REQ_OP_OFFSET		(IV_OFFSET + VHOST_CRYPTO_MAX_IV_LEN)
+
+/**
+ * 1-to-1 mapping between RTE_CRYPTO_*ALGO* and VIRTIO_CRYPTO_*ALGO*, for
+ * algorithms not supported by RTE_CRYPTODEV, the -VIRTIO_CRYPTO_NOTSUPP is
+ * returned.
+ */
+static int cipher_algo_transform[] = {
+		RTE_CRYPTO_CIPHER_NULL,
+		RTE_CRYPTO_CIPHER_ARC4,
+		RTE_CRYPTO_CIPHER_AES_ECB,
+		RTE_CRYPTO_CIPHER_AES_CBC,
+		RTE_CRYPTO_CIPHER_AES_CTR,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_CIPHER_DES_ECB */
+		RTE_CRYPTO_CIPHER_DES_CBC,
+		RTE_CRYPTO_CIPHER_3DES_ECB,
+		RTE_CRYPTO_CIPHER_3DES_CBC,
+		RTE_CRYPTO_CIPHER_3DES_CTR,
+		RTE_CRYPTO_CIPHER_KASUMI_F8,
+		RTE_CRYPTO_CIPHER_SNOW3G_UEA2,
+		RTE_CRYPTO_CIPHER_AES_F8,
+		RTE_CRYPTO_CIPHER_AES_XTS,
+		RTE_CRYPTO_CIPHER_ZUC_EEA3
+};
+
+/**
+ * VIRTIO_CRYTPO_AUTH_* indexes are not sequential, the gaps are filled with
+ * -VIRTIO_CRYPTO_BADMSG errors.
+ */
+static int auth_algo_transform[] = {
+		RTE_CRYPTO_AUTH_NULL,
+		RTE_CRYPTO_AUTH_MD5_HMAC,
+		RTE_CRYPTO_AUTH_SHA1_HMAC,
+		RTE_CRYPTO_AUTH_SHA224_HMAC,
+		RTE_CRYPTO_AUTH_SHA256_HMAC,
+		RTE_CRYPTO_AUTH_SHA384_HMAC,
+		RTE_CRYPTO_AUTH_SHA512_HMAC,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_CMAC_3DES */
+		RTE_CRYPTO_AUTH_AES_CMAC,
+		RTE_CRYPTO_AUTH_KASUMI_F9,
+		RTE_CRYPTO_AUTH_SNOW3G_UIA2,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		-VIRTIO_CRYPTO_BADMSG,
+		RTE_CRYPTO_AUTH_AES_GMAC,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_GMAC_TWOFISH */
+		RTE_CRYPTO_AUTH_AES_CBC_MAC,
+		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9 */
+		RTE_CRYPTO_AUTH_AES_XCBC_MAC
+};
+
+static int cipher_op_transform[] = {
+		-VIRTIO_CRYPTO_BADMSG, /* meaningless */
+		RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+		RTE_CRYPTO_CIPHER_OP_DECRYPT
+};
+
+static int iv_lens[] = {
+		-1, /* Invalid input */
+		0, /* RTE_CRYPTO_CIPHER_NULL */
+		8, /* RTE_CRYPTO_CIPHER_3DES_CBC */
+		8, /* RTE_CRYPTO_CIPHER_3DES_CTR */
+		8, /* RTE_CRYPTO_CIPHER_3DES_ECB */
+		16, /* RTE_CRYPTO_CIPHER_AES_CBC */
+		/* TODO: add common algos */
+};
+
+/**
+ * vhost_crypto struct is used to maintain a number of virtio_cryptos and
+ * one DPDK crypto device that deals with all crypto workloads. It is declared
+ * here and defined in vhost_crypto.c
+ */
+struct vhost_crypto {
+	/** Used to lookup DPDK Cryptodev Session based on VIRTIO crypto
+	 *  session ID.
+	 */
+	struct rte_hash *session_map;
+	struct rte_mempool *mbuf_pool;
+	struct rte_mempool *sess_pool;
+
+	/** DPDK cryptodev ID */
+	uint8_t cid;
+	uint16_t nb_qps;
+
+	uint64_t last_session_id;
+
+	uint64_t cache_session_id;
+	struct rte_cryptodev_sym_session *cache_session;
+	/** socket id for the device */
+	int socket_id;
+
+	struct virtio_net *dev;
+
+	uint8_t option;
+} __rte_cache_aligned;
+
+struct vhost_crypto_data_req {
+	struct vring_desc *head;
+	struct rte_vhost_memory *mem;
+	struct virtio_crypto_inhdr *inhdr;
+
+	uint16_t desc_idx;
+	uint32_t len;
+	struct vhost_virtqueue *vq;
+
+	uint8_t zero_copy;
+
+	int vid;
+
+	struct vring_desc *wb_desc;
+	uint16_t wb_len;
+};
+
+static int
+transform_cipher_param(struct rte_crypto_sym_xform *xform,
+		VhostUserCryptoSessionParam *param)
+{
+	int ret;
+
+	ret = cipher_algo_transform[param->cipher_algo];
+	if (unlikely(ret < 0))
+		return ret;
+
+	xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform->cipher.algo = ret;
+	xform->cipher.key.length = param->cipher_key_len;
+	if (xform->cipher.key.length > 0)
+		xform->cipher.key.data = param->cipher_key_buf;
+	ret = cipher_op_transform[param->dir];
+	if (unlikely(ret < 0)) {
+		VC_LOG_DBG("Bad operation type");
+		return ret;
+	}
+	xform->cipher.op = ret;
+	ret = iv_lens[xform->cipher.algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform->cipher.iv.length = ret;
+	xform->cipher.iv.offset = IV_OFFSET;
+	return 0;
+}
+
+static int
+transform_chain_param(struct rte_crypto_sym_xform *xforms,
+		VhostUserCryptoSessionParam *param)
+{
+	struct rte_crypto_sym_xform *xform_cipher, *xform_auth;
+	int ret;
+
+	switch (param->chaining_dir) {
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER:
+		xform_auth = xforms;
+		xform_cipher = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+		break;
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH:
+		xform_cipher = xforms;
+		xform_auth = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		break;
+	default:
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	/* cipher */
+	ret = cipher_algo_transform[param->cipher_algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform_cipher->cipher.algo = ret;
+	xform_cipher->cipher.key.length = param->cipher_key_len;
+	xform_cipher->cipher.key.data = param->cipher_key_buf;
+	ret = iv_lens[xform_cipher->cipher.algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->cipher.iv.length = ret;
+	xform_cipher->cipher.iv.offset = IV_OFFSET;
+
+	/* auth */
+	xform_auth->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+	ret = auth_algo_transform[param->hash_algo];
+	if (unlikely(ret < 0))
+		return ret;
+	xform_auth->auth.algo = ret;
+	xform_auth->auth.digest_length = param->digest_len;
+	xform_auth->auth.key.length = param->auth_key_len;
+	xform_auth->auth.key.data = param->auth_key_buf;
+
+	return 0;
+}
+
+static void
+vhost_crypto_create_sess(struct vhost_crypto *vcrypto,
+		VhostUserCryptoSessionParam *sess_param)
+{
+	struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0};
+	struct rte_cryptodev_sym_session *session;
+	int ret;
+
+	switch (sess_param->op_type) {
+	case VIRTIO_CRYPTO_SYM_OP_NONE:
+	case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+		ret = transform_cipher_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session msg (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+		break;
+	case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+		if (unlikely(sess_param->hash_mode !=
+				VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH)) {
+			sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+			VC_LOG_ERR("Error transform session message (%i)",
+					-VIRTIO_CRYPTO_NOTSUPP);
+			return;
+		}
+
+		xform1.next = &xform2;
+
+		ret = transform_chain_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session message (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+
+		break;
+	default:
+		VC_LOG_ERR("Algorithm not yet supported");
+		sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+		return;
+	}
+
+	session = rte_cryptodev_sym_session_create(vcrypto->sess_pool);
+	if (!session) {
+		VC_LOG_ERR("Failed to create session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1,
+			vcrypto->sess_pool) < 0) {
+		VC_LOG_ERR("Failed to initialize session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	/* insert hash to map */
+	if (rte_hash_add_key_data(vcrypto->session_map,
+			&vcrypto->last_session_id, session) < 0) {
+		VC_LOG_ERR("Failed to insert session to hash table");
+
+		if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0)
+			VC_LOG_ERR("Failed to clear session");
+		else {
+			if (rte_cryptodev_sym_session_free(session) < 0)
+				VC_LOG_ERR("Failed to free session");
+		}
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) created.",
+			vcrypto->last_session_id, session);
+
+	sess_param->session_id = vcrypto->last_session_id;
+	vcrypto->last_session_id++;
+}
+
+static int
+vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id)
+{
+	struct rte_cryptodev_sym_session *session;
+	uint64_t sess_id = session_id;
+	int ret;
+
+	ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id,
+			(void **)&session);
+
+	if (unlikely(ret < 0)) {
+		VC_LOG_ERR("Failed to delete session (key %lu).", session_id);
+		return -VIRTIO_CRYPTO_INVSESS;
+	}
+
+	if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_cryptodev_sym_session_free(session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_hash_del_key(vcrypto->session_map, &sess_id) < 0) {
+		VC_LOG_DBG("Failed to delete session from hash table.");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) deleted.", sess_id,
+			session);
+
+	return 0;
+}
+
+static int
+vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+	VhostUserMsg *vmsg = msg;
+	int ret = 0;
+
+	if (dev == NULL || require_reply == NULL) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (vcrypto == NULL) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	*require_reply = 0;
+
+	if (vmsg->request.master == VHOST_USER_CRYPTO_CREATE_SESS) {
+		vhost_crypto_create_sess(vcrypto,
+				&vmsg->payload.crypto_session);
+		*require_reply = 1;
+	} else if (vmsg->request.master == VHOST_USER_CRYPTO_CLOSE_SESS)
+		ret = vhost_crypto_close_sess(vcrypto, vmsg->payload.u64);
+	else
+		ret = -EINVAL;
+
+	return ret;
+}
-- 
2.7.4

[PATCH v4 4/8] lib/librte_vhost: add request handler

[Fan Zhang]

This patch adds the implementation that parses virtio crypto request
to dpdk crypto operation.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 609 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 609 insertions(+)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index c639b20..8fdb33f 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -426,3 +426,612 @@ vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
 
 	return ret;
 }
+
+static __rte_always_inline struct vring_desc *
+find_write_desc(struct vring_desc *head, struct vring_desc *desc)
+{
+	if (desc->flags & VRING_DESC_F_WRITE)
+		return desc;
+
+	while (desc->flags & VRING_DESC_F_NEXT) {
+		desc = &head[desc->next];
+		if (desc->flags & VRING_DESC_F_WRITE)
+			return desc;
+	}
+
+	return NULL;
+}
+
+static struct virtio_crypto_inhdr *
+reach_inhdr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc *desc)
+{
+	while (desc->flags & VRING_DESC_F_NEXT)
+		desc = &head[desc->next];
+
+	return GPA_TO_VVA(struct virtio_crypto_inhdr *, mem, desc->addr);
+}
+
+static __rte_always_inline int
+move_desc(struct vring_desc *head, struct vring_desc **cur_desc,
+		uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	left -= desc->len;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		left -= desc->len;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+	return 0;
+}
+
+static int
+copy_data(void *dst_data, struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	uint32_t to_copy;
+	uint8_t *data = dst_data;
+	uint8_t *src;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	to_copy = RTE_MIN(desc->len, (uint32_t)left);
+	src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy((uint8_t *)data, src, to_copy);
+	left -= to_copy;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_copy = RTE_MIN(desc->len, (uint32_t)left);
+		src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(data + size - left, src, to_copy);
+		left -= to_copy;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+
+	return 0;
+}
+
+static __rte_always_inline void *
+get_data_ptr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	void *data;
+
+	data = GPA_TO_VVA(void *, mem, (*cur_desc)->addr);
+	if (unlikely(!data)) {
+		VC_LOG_ERR("Failed to get object");
+		return NULL;
+	}
+
+	if (unlikely(move_desc(head, cur_desc, size) < 0))
+		return NULL;
+
+	return data;
+}
+
+static int
+write_back_data(struct rte_crypto_op *op, struct vhost_crypto_data_req *vc_req)
+{
+	struct rte_mbuf *mbuf = op->sym->m_dst;
+	struct vring_desc *head = vc_req->head;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct vring_desc *desc = vc_req->wb_desc;
+	int left = vc_req->wb_len;
+	uint32_t to_write;
+	uint8_t *src_data = mbuf->buf_addr, *dst;
+
+	rte_prefetch0(&head[desc->next]);
+	to_write = RTE_MIN(desc->len, (uint32_t)left);
+	dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy(dst, src_data, to_write);
+	left -= to_write;
+	src_data += to_write;
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	printf("desc addr %llu len %u:", desc->addr, desc->len);
+	rte_hexdump(stdout, "", dst, to_write);
+#endif
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_write = RTE_MIN(desc->len, (uint32_t)left);
+		dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(dst, src_data, to_write);
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+		printf("desc addr %llu len %u:", desc->addr, desc->len);
+		rte_hexdump(stdout, "DST:", dst, to_write);
+#endif
+		left -= to_write;
+		src_data += to_write;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	return 0;
+}
+
+static uint8_t
+prepare_sym_cipher_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_cipher_data_req *cipher,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			cipher->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, cipher->para.iv_len);
+#endif
+
+	m_src->data_len = cipher->para.src_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				cipher->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 ||
+				m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		if (unlikely(cipher->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, cipher->para.src_data_len))
+				< 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			cipher->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, cipher->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		m_dst->data_len = cipher->para.dst_data_len;
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = cipher->para.dst_data_len;
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+	/* src data */
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = 0;
+	op->sym->cipher.data.length = cipher->para.src_data_len;
+
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+	vc_req->len = cipher->para.dst_data_len + INHDR_LEN;
+
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+static uint8_t
+prepare_sym_chain_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_alg_chain_data_req *chain,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint32_t digest_offset;
+	void *digest_addr;
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			chain->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, chain->para.iv_len);
+#endif
+	m_src->data_len = chain->para.src_data_len;
+	m_dst->data_len = chain->para.dst_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				chain->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 || m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		if (unlikely(chain->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, chain->para.src_data_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			chain->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.phys_addr = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.hash_result_len);
+		op->sym->auth.digest.data = get_data_ptr(head, mem, &desc,
+				chain->para.hash_result_len);
+		if (unlikely(op->sym->auth.digest.phys_addr == 0)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		digest_offset = m_dst->data_len;
+		digest_addr = rte_pktmbuf_mtod_offset(m_dst, void *,
+				digest_offset);
+
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = m_dst->data_len + chain->para.hash_result_len;
+
+		if (unlikely(move_desc(head, &desc,
+				chain->para.dst_data_len) < 0)) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		if (unlikely(copy_data(digest_addr, head, mem, &desc,
+				chain->para.hash_result_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.data = digest_addr;
+		op->sym->auth.digest.phys_addr = rte_pktmbuf_iova_offset(m_dst,
+				digest_offset);
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "Digest:", op->sym->auth.digest.data,
+			chain->para.hash_result_len);
+#endif
+
+	/* record inhdr */
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = chain->para.cipher_start_src_offset;
+	op->sym->cipher.data.length = chain->para.src_data_len -
+			chain->para.cipher_start_src_offset;
+
+	op->sym->auth.data.offset = chain->para.hash_start_src_offset;
+	op->sym->auth.data.length = chain->para.len_to_hash;
+
+	vc_req->len = chain->para.dst_data_len + chain->para.hash_result_len +
+			INHDR_LEN;
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+/**
+ * Process on descriptor
+ */
+static __rte_always_inline int
+vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
+		struct vhost_virtqueue *vq, struct rte_crypto_op *op,
+		struct vring_desc *head, uint16_t desc_idx,
+		struct rte_vhost_memory *mem)
+{
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(op->sym->m_src,
+			sizeof(struct rte_mbuf));
+	struct rte_cryptodev_sym_session *session;
+	struct virtio_crypto_op_data_req *req;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vring_desc *desc = NULL;
+	uint64_t session_id;
+	int err = 0;
+
+	vc_req->desc_idx = desc_idx;
+
+	if (likely(head->flags & VRING_DESC_F_INDIRECT)) {
+		head = GPA_TO_VVA(struct vring_desc *, mem, head->addr);
+		if (unlikely(!head))
+			return 0;
+		desc_idx = 0;
+	}
+
+	desc = head;
+
+	vc_req->mem = mem;
+	vc_req->head = head;
+	vc_req->vq = vq;
+
+	vc_req->zero_copy = vcrypto->option;
+
+	req = get_data_ptr(head, mem, &desc, sizeof(*req));
+	if (unlikely(req == NULL)) {
+		err = VIRTIO_CRYPTO_ERR;
+		VC_LOG_ERR("Invalid descriptor");
+		goto error_exit;
+	}
+
+	switch (req->header.opcode) {
+	case VIRTIO_CRYPTO_CIPHER_ENCRYPT:
+	case VIRTIO_CRYPTO_CIPHER_DECRYPT:
+		session_id = req->header.session_id;
+
+		/* one branch to avoid unnecessary table lookup */
+		if (vcrypto->cache_session_id != session_id) {
+			err = rte_hash_lookup_data(vcrypto->session_map,
+					&session_id, (void **)&session);
+			if (unlikely(err < 0)) {
+				err = VIRTIO_CRYPTO_ERR;
+				VC_LOG_ERR("Failed to retrieve session id %lu",
+						session_id);
+				goto error_exit;
+			}
+
+			vcrypto->cache_session = session;
+			vcrypto->cache_session_id = session_id;
+		}
+
+		session = vcrypto->cache_session;
+
+		err = rte_crypto_op_attach_sym_session(op, session);
+		if (unlikely(err < 0)) {
+			err = VIRTIO_CRYPTO_ERR;
+			VC_LOG_ERR("Failed to attach session to op");
+			goto error_exit;
+		}
+
+		switch (req->u.sym_req.op_type) {
+		case VIRTIO_CRYPTO_SYM_OP_NONE:
+			err = VIRTIO_CRYPTO_NOTSUPP;
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+			err = prepare_sym_cipher_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.cipher, desc);
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+			err = prepare_sym_chain_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.chain, desc);
+			break;
+		}
+		if (unlikely(err != 0)) {
+			VC_LOG_ERR("Failed to process sym request");
+			goto error_exit;
+		}
+		break;
+	default:
+		VC_LOG_ERR("Unsupported symmetric crypto request type %u",
+				req->header.opcode);
+		goto error_exit;
+	}
+
+	return 0;
+
+error_exit:
+
+	inhdr = reach_inhdr(head, mem, desc);
+	if (likely(inhdr != NULL))
+		inhdr->status = (uint8_t)err;
+
+	return -1;
+}
+
+static __rte_always_inline struct vhost_virtqueue *
+vhost_crypto_finalize_one_request(struct rte_crypto_op *op,
+		struct vhost_virtqueue *old_vq)
+{
+	struct rte_mbuf *m_src = op->sym->m_src;
+	struct rte_mbuf *m_dst = op->sym->m_dst;
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(m_src,
+			sizeof(struct rte_mbuf));
+	uint16_t desc_idx;
+	int ret = 0;
+
+	if (unlikely(!vc_req)) {
+		VC_LOG_ERR("Failed to retrieve vc_req");
+		return NULL;
+	}
+
+	if (old_vq && (vc_req->vq != old_vq))
+		return vc_req->vq;
+
+	desc_idx = vc_req->desc_idx;
+
+	if (unlikely(op->status != RTE_CRYPTO_OP_STATUS_SUCCESS))
+		vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+	else {
+		if (vc_req->zero_copy == 0) {
+			ret = write_back_data(op, vc_req);
+			if (unlikely(ret != 0))
+				vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+		}
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "DST:", rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *),
+			m_dst->data_len);
+#endif
+
+	vc_req->vq->used->ring[desc_idx].id = desc_idx;
+	vc_req->vq->used->ring[desc_idx].len = vc_req->len;
+
+	rte_mempool_put(m_dst->pool, (void *)m_dst);
+	rte_mempool_put(m_src->pool, (void *)m_src);
+
+	return vc_req->vq;
+}
+
+static __rte_always_inline uint16_t
+vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfd)
+{
+	uint16_t processed = 1;
+	struct vhost_virtqueue *vq, *tmp_vq;
+
+	if (unlikely(nb_ops == 0))
+		return 0;
+
+	vq = vhost_crypto_finalize_one_request(ops[0], NULL);
+	if (unlikely(vq == NULL))
+		return 0;
+	tmp_vq = vq;
+
+	while ((processed < nb_ops)) {
+		tmp_vq = vhost_crypto_finalize_one_request(ops[processed],
+				tmp_vq);
+
+		if (unlikely(vq != tmp_vq))
+			break;
+
+		processed++;
+	}
+
+	*callfd = vq->callfd;
+
+	*(volatile uint16_t *)&vq->used->idx += processed;
+
+	return processed;
+}
-- 
2.7.4

[PATCH v4 5/8] lib/librte_vhost: add public function implementation

[Fan Zhang]

This patch adds public API implementation to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h    | 125 +++++++++++++++++
 lib/librte_vhost/rte_vhost_version.map |   5 +
 lib/librte_vhost/vhost_crypto.c        | 248 +++++++++++++++++++++++++++++++++
 3 files changed, 378 insertions(+)
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
new file mode 100644
index 0000000..339a939
--- /dev/null
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -0,0 +1,125 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#ifndef _VHOST_CRYPTO_H_
+#define _VHOST_CRYPTO_H_
+
+#include <linux/virtio_crypto.h>
+#include <rte_cryptodev.h>
+#include <rte_hash.h>
+#include <rte_pause.h>
+#include "rte_vhost.h"
+
+#ifndef MAX_DATA_QUEUES
+#define MAX_DATA_QUEUES	(1)
+#endif
+
+#define VIRTIO_CRYPTO_CTRL_QUEUE	(0)
+#define VIRTIO_CRYPTO_MAX_NUM_DEVS	(64)
+#define VIRTIO_CRYPTO_MAX_NUM_BURST_VQS	(64)
+
+/** Feature bits */
+#define VIRTIO_CRYPTO_F_CIPHER_SESSION_MODE	(1)
+#define VIRTIO_CRYPTO_F_HASH_SESSION_MODE	(2)
+#define VIRTIO_CRYPTO_F_MAC_SESSION_MODE	(3)
+#define VIRTIO_CRYPTO_F_AEAD_SESSION_MODE	(4)
+
+#define VHOST_CRYPTO_MBUF_POOL_SIZE		(8192)
+#define VHOST_CRYPTO_MAX_BURST_SIZE		(64)
+
+enum rte_vhost_crypto_zero_copy {
+	RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE = 0,
+	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE,
+	RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS
+};
+
+/**
+ *  Create Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param cryptodev_id
+ *  The identifier of DPDK Cryptodev, the same cryptodev_id can be assigned to
+ *  multiple Vhost-crypto devices.
+ * @param sess_pool
+ *  The pointer to the created cryptodev session pool with the private data size
+ *  matches the target DPDK Cryptodev.
+ * @param socket_id
+ *  NUMA Socket ID to allocate resources on. *
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise
+ */
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id);
+
+/**
+ *  Free the Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise.
+ */
+int
+rte_vhost_crypto_free(int vid);
+
+/**
+ *  Enable or disable zero copy feature
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param option
+ *  Flag of zero copy feature.
+ * @return
+ *  0 if completed successfully.
+ *  Negative integer if otherwise.
+ */
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option);
+
+/**
+ * Fetch a number of vring descriptors from virt-queue and translate to DPDK
+ * crypto operations. After this function is executed, the user can enqueue
+ * the processed ops to the target cryptodev.
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param qid
+ *  Virtio queue index.
+ * @param ops
+ *  The address of an array of pointers to *rte_crypto_op* structures that must
+ *  be large enough to store *nb_ops* pointers in it.
+ * @param nb_ops
+ *  The maximum number of operations to be fetched and translated.
+ * @return
+ *  The number of fetched and processed vhost crypto request operations.
+ */
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops);
+/**
+ * Finalize the dequeued crypto ops. After the translated crypto ops are
+ * dequeued from the cryptodev, this function shall be called to write the
+ * processed data back to the vring descriptor (if no-copy is turned off).
+ *
+ * @param ops
+ *  The address of an array of *rte_crypto_op* structure that was dequeued
+ *  from cryptodev.
+ * @param nb_ops
+ *  The number of operations contained in the array.
+ * @callfds
+ *  The callfd number(s) contained in this burst
+ * @nb_callfds
+ *  The number of call_fd numbers exist in the callfds
+ * @return
+ *  The number of ops processed.
+ */
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds);
+
+#endif /**< _VHOST_CRYPTO_H_ */
diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map
index 91bf9f0..5bec669 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -64,4 +64,9 @@ DPDK_18.05 {
 	global:
 
 	rte_vhost_user_register_extern_ops;
+	rte_vhost_crypto_create;
+	rte_vhost_crypto_free;
+	rte_vhost_crypto_fetch_requests;
+	rte_vhost_crypto_finalize_requests;
+
 } DPDK_18.02;
diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index 8fdb33f..59c6ad0 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -1035,3 +1035,251 @@ vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
 
 	return processed;
 }
+
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct rte_hash_parameters params = {0};
+	struct vhost_user_extern_ops ops;
+	struct vhost_crypto *vcrypto;
+	char name[128];
+	int ret;
+
+	if (vid >= VIRTIO_CRYPTO_MAX_NUM_DEVS || !dev) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	ret = rte_vhost_driver_set_features(dev->ifname,
+			VIRTIO_CRYPTO_FEATURES);
+	if (ret < 0) {
+		VC_LOG_ERR("Error setting features");
+		return -1;
+	}
+
+	vcrypto = rte_zmalloc_socket(NULL, sizeof(*vcrypto),
+			RTE_CACHE_LINE_SIZE, socket_id);
+	if (!vcrypto) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	vcrypto->sess_pool = sess_pool;
+	vcrypto->cid = cryptodev_id;
+	vcrypto->cache_session_id = UINT64_MAX;
+	vcrypto->last_session_id = 1;
+	vcrypto->dev = dev;
+	vcrypto->option = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	snprintf(name, 127, "HASH_VHOST_CRYPT_%u", (uint32_t)vid);
+	params.name = name;
+	params.entries = SESSION_MAP_ENTRIES;
+	params.hash_func = rte_jhash;
+	params.key_len = sizeof(uint64_t);
+	params.socket_id = socket_id;
+	vcrypto->session_map = rte_hash_create(&params);
+	if (!vcrypto->session_map) {
+		VC_LOG_ERR("Failed to creath session map");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "MBUF_POOL_VM_%u", (uint32_t)vid);
+	vcrypto->mbuf_pool = rte_pktmbuf_pool_create(name,
+			VHOST_CRYPTO_MBUF_POOL_SIZE, 512,
+			sizeof(struct vhost_crypto_data_req),
+			RTE_MBUF_DEFAULT_DATAROOM * 2 + RTE_PKTMBUF_HEADROOM,
+			rte_socket_id());
+	if (!vcrypto->mbuf_pool) {
+		VC_LOG_ERR("Failed to creath mbuf pool");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	ops.pre_msg_handle = NULL;
+	ops.post_msg_handle = vhost_crypto_msg_post_handler;
+
+	dev->extern_data = vcrypto;
+	if (rte_vhost_user_register_extern_ops(dev->vid, &ops) < 0) {
+		VC_LOG_ERR("Failed to register device");
+		goto error_exit;
+	}
+
+	return 0;
+
+error_exit:
+	if (vcrypto->session_map)
+		rte_hash_free(vcrypto->session_map);
+	if (vcrypto->mbuf_pool)
+		rte_mempool_free(vcrypto->mbuf_pool);
+
+	rte_free(vcrypto);
+
+	return ret;
+}
+
+int
+rte_vhost_crypto_free(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	rte_hash_free(vcrypto->session_map);
+	rte_mempool_free(vcrypto->mbuf_pool);
+	rte_free(vcrypto);
+
+	dev->extern_data = NULL;
+	dev->extern_ops.pre_msg_handle = NULL;
+	dev->extern_ops.post_msg_handle = NULL;
+
+	return 0;
+}
+
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	if (unlikely(option < 0 || option >=
+			RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS)) {
+		VC_LOG_ERR("Invalid option %i", option);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	if (vcrypto->option == (uint8_t)option)
+		return 0;
+
+	if (!(rte_mempool_full(vcrypto->mbuf_pool))) {
+		VC_LOG_ERR("Cannot update zero copy as mempool is not full");
+		return -EINVAL;
+	}
+
+	vcrypto->option = (uint8_t)option;
+
+	return 0;
+}
+
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+	struct rte_mbuf *mbufs[VHOST_CRYPTO_MAX_BURST_SIZE * 2];
+	struct virtio_net *dev = get_device(vid);
+	struct rte_vhost_memory *mem;
+	struct vhost_crypto *vcrypto;
+	struct vhost_virtqueue *vq;
+	uint16_t avail_idx;
+	uint16_t start_idx;
+	uint16_t required;
+	uint16_t count;
+	uint16_t i;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vq = dev->virtqueue[qid];
+	mem = dev->mem;
+
+	avail_idx = *((volatile uint16_t *)&vq->avail->idx);
+	start_idx = vq->last_used_idx;
+	count = avail_idx - start_idx;
+	count = RTE_MIN(count, VHOST_CRYPTO_MAX_BURST_SIZE);
+	count = RTE_MIN(count, nb_ops);
+
+	if (unlikely(count == 0))
+		return 0;
+
+	/* for zero copy, we need 2 empty mbufs for src and dst, otherwise
+	 * we need only 1 mbuf as src and dst
+	 */
+	required = count * 2;
+	if (unlikely(rte_mempool_get_bulk(vcrypto->mbuf_pool, (void **)mbufs,
+			required) < 0)) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	for (i = 0; i < count; i++) {
+		uint16_t used_idx = (start_idx + i) & (vq->size - 1);
+		uint16_t desc_idx = vq->avail->ring[used_idx];
+		struct vring_desc *head = &vq->desc[desc_idx];
+		struct rte_crypto_op *op = ops[i];
+
+		op->sym->m_src = mbufs[i * 2];
+		op->sym->m_dst = mbufs[i * 2 + 1];
+		op->sym->m_src->data_off = 0;
+		op->sym->m_dst->data_off = 0;
+
+		if (unlikely(vhost_crypto_process_one_req(vcrypto, vq, op, head,
+				desc_idx, mem)) < 0)
+			break;
+	}
+
+	vq->last_used_idx += i;
+
+	return i;
+}
+
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds)
+{
+	struct rte_crypto_op **tmp_ops = ops;
+	uint16_t count = 0, left = nb_ops;
+	int callfd;
+	uint16_t idx = 0;
+
+	while (left) {
+		count = vhost_crypto_complete_one_vm_requests(tmp_ops, left,
+				&callfd);
+		if (unlikely(count == 0))
+			break;
+
+		tmp_ops = &tmp_ops[count];
+		left -= count;
+
+		callfds[idx++] = callfd;
+
+		if (unlikely(idx >= VIRTIO_CRYPTO_MAX_NUM_BURST_VQS)) {
+			VC_LOG_ERR("Too many vqs");
+			break;
+		}
+	}
+
+	*nb_callfds = idx;
+
+	return nb_ops - left;
+}
-- 
2.7.4

[PATCH v4 6/8] lib/librte_vhost: update makefile

[Fan Zhang]

This patch updates the Makefile of vhost library to enable vhost crypto
compiling.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/Makefile    | 6 ++++--
 lib/librte_vhost/meson.build | 8 ++++----
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 5d6c6ab..95a6a93 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -23,8 +23,10 @@ LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev -lrte_net
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_VHOST) := fd_man.c iotlb.c socket.c vhost.c \
 					vhost_user.c virtio_net.c
+ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV), y)
+SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto.c
+endif
 
 # install includes
-SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h
-
+SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h rte_vhost_crypto.h
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_vhost/meson.build b/lib/librte_vhost/meson.build
index 9e8c0e7..36f1e27 100644
--- a/lib/librte_vhost/meson.build
+++ b/lib/librte_vhost/meson.build
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
+# Copyright(c) 2017-2018 Intel Corporation
 
 if host_machine.system() != 'linux'
 	build = false
@@ -10,6 +10,6 @@ endif
 version = 4
 allow_experimental_apis = true
 sources = files('fd_man.c', 'iotlb.c', 'socket.c', 'vhost.c', 'vhost_user.c',
-		'virtio_net.c')
-headers = files('rte_vhost.h')
-deps += ['ethdev']
+		'virtio_net.c', 'virtio_crypto.c')
+headers = files('rte_vhost.h', 'rte_vhost_crypto.h')
+deps += ['ethdev', 'cryptodev']
-- 
2.7.4

[PATCH v4 7/8] examples/vhost_crypto: add vhost crypto sample application

[Fan Zhang]

This patch adds vhost_crypto sample application to DPDK.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 examples/vhost_crypto/Makefile    |  32 +++
 examples/vhost_crypto/main.c      | 541 ++++++++++++++++++++++++++++++++++++++
 examples/vhost_crypto/meson.build |  14 +
 3 files changed, 587 insertions(+)
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 examples/vhost_crypto/meson.build

diff --git a/examples/vhost_crypto/Makefile b/examples/vhost_crypto/Makefile
new file mode 100644
index 0000000..1bb65e8
--- /dev/null
+++ b/examples/vhost_crypto/Makefile
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vhost-crypto
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
diff --git a/examples/vhost_crypto/main.c b/examples/vhost_crypto/main.c
new file mode 100644
index 0000000..bc86724
--- /dev/null
+++ b/examples/vhost_crypto/main.c
@@ -0,0 +1,541 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <stdbool.h>
+#include <assert.h>
+#include <getopt.h>
+
+#include <rte_malloc.h>
+#include <rte_cycles.h>
+#include <rte_vhost.h>
+#include <rte_cryptodev.h>
+#include <rte_vhost_crypto.h>
+
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define NB_VIRTIO_QUEUES		(1)
+#define MAX_PKT_BURST			(64)
+#define MAX_IV_LEN			(32)
+#define NB_MEMPOOL_OBJS			(8192)
+#define NB_CRYPTO_DESCRIPTORS		(4096)
+#define NB_CACHE_OBJS			(128)
+#define SESSION_MAP_ENTRIES		(1024)
+#define REFRESH_TIME_SEC		(3)
+
+#define MAX_NB_SOCKETS			(32)
+#define DEF_SOCKET_FILE			"/tmp/vhost_crypto1.socket"
+
+struct vhost_crypto_options {
+	char *socket_files[MAX_NB_SOCKETS];
+	uint32_t nb_sockets;
+	uint8_t cid;
+	uint16_t qid;
+	uint32_t zero_copy;
+	uint32_t guest_polling;
+} options;
+
+struct vhost_crypto_info {
+	int vids[MAX_NB_SOCKETS];
+	struct rte_mempool *sess_pool;
+	struct rte_mempool *cop_pool;
+	uint32_t lcore_id;
+	uint8_t cid;
+	uint32_t qid;
+	uint32_t nb_vids;
+	volatile uint32_t initialized[MAX_NB_SOCKETS];
+
+} info;
+
+#define SOCKET_FILE_KEYWORD	"socket-file"
+#define CRYPTODEV_ID_KEYWORD	"cdev-id"
+#define CRYPTODEV_QUEUE_KEYWORD	"cdev-queue-id"
+#define ZERO_COPY_KEYWORD	"zero-copy"
+#define POLLING_KEYWORD		"guest-polling"
+
+uint64_t vhost_cycles[2], last_v_cycles[2];
+uint64_t outpkt_amount;
+
+/** support *SOCKET_FILE_PATH:CRYPTODEV_ID* format */
+static int
+parse_socket_arg(char *arg)
+{
+	uint32_t nb_sockets = options.nb_sockets;
+	size_t len = strlen(arg);
+
+	if (nb_sockets >= MAX_NB_SOCKETS) {
+		RTE_LOG(ERR, USER1, "Too many socket files!\n");
+		return -ENOMEM;
+	}
+
+	options.socket_files[nb_sockets] = rte_malloc(NULL, len, 0);
+	if (!options.socket_files[nb_sockets]) {
+		RTE_LOG(ERR, USER1, "Insufficient memory\n");
+		return -ENOMEM;
+	}
+
+	rte_memcpy(options.socket_files[nb_sockets], arg, len);
+
+	options.nb_sockets++;
+
+	return 0;
+}
+
+static int
+parse_cryptodev_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if ((pm == '\0') || (end == NULL) || (*end != '\0')) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	if (pm > rte_cryptodev_count()) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.cid = (uint8_t)pm;
+
+	return 0;
+}
+
+static int
+parse_cdev_queue_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if (pm == UINT64_MAX) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev Queue ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.qid = (uint16_t)pm;
+
+	return 0;
+}
+
+static void
+vhost_crypto_usage(const char *prgname)
+{
+	printf("%s [EAL options] --\n"
+		"  --%s SOCKET-FILE-PATH\n"
+		"  --%s CRYPTODEV_ID: crypto device id\n"
+		"  --%s CDEV_QUEUE_ID: crypto device queue id\n"
+		"  --%s: zero copy\n"
+		"  --%s: guest polling\n",
+		prgname, SOCKET_FILE_KEYWORD, CRYPTODEV_ID_KEYWORD,
+		CRYPTODEV_QUEUE_KEYWORD, ZERO_COPY_KEYWORD, POLLING_KEYWORD);
+}
+
+static int
+vhost_crypto_parse_args(int argc, char **argv)
+{
+	int opt, ret;
+	char *prgname = argv[0];
+	char **argvopt;
+	int option_index;
+	struct option lgopts[] = {
+			{SOCKET_FILE_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_ID_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_QUEUE_KEYWORD, required_argument, 0, 0},
+			{ZERO_COPY_KEYWORD, no_argument, 0, 0},
+			{POLLING_KEYWORD, no_argument, 0, 0},
+			{NULL, 0, 0, 0}
+	};
+
+	options.cid = 0;
+	options.qid = 0;
+	options.nb_sockets = 0;
+	options.guest_polling = 0;
+	options.zero_copy = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, "s:",
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		case 0:
+			if (strcmp(lgopts[option_index].name,
+					SOCKET_FILE_KEYWORD) == 0) {
+				ret = parse_socket_arg(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_ID_KEYWORD) == 0) {
+				ret = parse_cryptodev_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_QUEUE_KEYWORD) == 0) {
+				ret = parse_cdev_queue_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					ZERO_COPY_KEYWORD) == 0) {
+				options.zero_copy =
+					RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE;
+			} else if (strcmp(lgopts[option_index].name,
+					POLLING_KEYWORD) == 0) {
+				options.guest_polling = 1;
+			} else {
+				vhost_crypto_usage(prgname);
+				return -EINVAL;
+			}
+			break;
+		default:
+			return -1;
+		}
+	}
+
+	if (options.nb_sockets == 0) {
+		options.socket_files[0] = strdup(DEF_SOCKET_FILE);
+		options.nb_sockets = 1;
+		RTE_LOG(INFO, USER1,
+				"VHOST-CRYPTO: use default socket file %s\n",
+				DEF_SOCKET_FILE);
+	}
+
+	return 0;
+}
+
+static int
+new_device(int vid)
+{
+	char path[PATH_MAX];
+	uint32_t idx, i;
+	int ret;
+
+	ret = rte_vhost_get_ifname(vid, path, PATH_MAX);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot find matched socket\n");
+		return ret;
+	}
+
+	for (idx = 0; idx < options.nb_sockets; idx++) {
+		if (strcmp(path, options.socket_files[idx]) == 0)
+			break;
+	}
+
+	if (idx == options.nb_sockets) {
+		RTE_LOG(ERR, USER1, "Cannot find recorded socket\n");
+		return -ENOENT;
+	}
+
+	for (i = 0; i < 2; i++) {
+		vhost_cycles[i] = 0;
+		last_v_cycles[i] = 0;
+	}
+
+	ret = rte_vhost_crypto_create(vid, info.cid, info.sess_pool,
+			rte_lcore_to_socket_id(info.lcore_id));
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot create vhost crypto\n");
+		return ret;
+	}
+
+	ret = rte_vhost_crypto_set_zero_copy(vid, options.zero_copy);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot %s zero copy feature\n",
+				options.zero_copy == 1 ? "enable" : "disable");
+		return ret;
+	}
+
+	info.vids[idx] = vid;
+	info.initialized[idx] = 1;
+
+	rte_wmb();
+
+	RTE_LOG(INFO, USER1, "New Vhost-crypto Device %s, Device ID %d\n", path,
+			vid);
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	uint32_t i;
+
+	for (i = 0; i < info.nb_vids; i++) {
+		if (vid == info.vids[i])
+			break;
+	}
+
+	if (i == info.nb_vids) {
+		RTE_LOG(ERR, USER1, "Cannot find socket file from list\n");
+		return;
+	}
+
+	info.initialized[i] = 0;
+
+	rte_wmb();
+
+	rte_vhost_crypto_free(vid);
+
+	RTE_LOG(INFO, USER1, "Vhost Crypto Device %i Removed\n", vid);
+}
+
+static const struct vhost_device_ops virtio_crypto_device_ops = {
+	.new_device =  new_device,
+	.destroy_device = destroy_device,
+};
+
+__attribute__((unused))
+static void clrscr(void)
+{
+	system("@cls||clear");
+}
+
+static int
+vhost_crypto_worker(__rte_unused void *arg)
+{
+	struct rte_crypto_op *ops[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	struct rte_crypto_op *ops_deq[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	uint32_t nb_inflight_ops = 0;
+	uint16_t nb_callfds;
+	int callfds[VIRTIO_CRYPTO_MAX_NUM_BURST_VQS];
+	uint32_t lcore_id = rte_lcore_id();
+	uint32_t burst_size = MAX_PKT_BURST;
+	uint32_t i, j, k;
+	uint32_t to_fetch, fetched;
+	uint64_t t_start, t_end, interval;
+
+	int ret = 0;
+
+	RTE_LOG(INFO, USER1, "Processing on Core %u started\n", lcore_id);
+
+	for (i = 0; i < NB_VIRTIO_QUEUES; i++) {
+		if (rte_crypto_op_bulk_alloc(info.cop_pool,
+				RTE_CRYPTO_OP_TYPE_SYMMETRIC, ops[i],
+				burst_size) < burst_size) {
+			RTE_LOG(ERR, USER1, "Failed to alloc cops\n");
+			ret = -1;
+			goto exit;
+		}
+	}
+
+	while (1) {
+		for (i = 0; i < info.nb_vids; i++) {
+			if (unlikely(info.initialized[i] == 0))
+				continue;
+
+			for (j = 0; j < NB_VIRTIO_QUEUES; j++) {
+				t_start = rte_rdtsc_precise();
+
+				to_fetch = RTE_MIN(burst_size,
+						(NB_CRYPTO_DESCRIPTORS -
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_fetch_requests(
+						info.vids[i], j, ops[j],
+						to_fetch);
+				nb_inflight_ops += rte_cryptodev_enqueue_burst(
+						info.cid, info.qid, ops[j],
+						fetched);
+				if (unlikely(rte_crypto_op_bulk_alloc(
+						info.cop_pool,
+						RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+						ops[j], fetched) < fetched)) {
+					RTE_LOG(ERR, USER1, "Failed realloc\n");
+					return -1;
+				}
+				t_end = rte_rdtsc_precise();
+				interval = t_end - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+
+				t_start = t_end;
+				fetched = rte_cryptodev_dequeue_burst(
+						info.cid, info.qid,
+						ops_deq[j], RTE_MIN(burst_size,
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_finalize_requests(
+						ops_deq[j], fetched, callfds,
+						&nb_callfds);
+
+				nb_inflight_ops -= fetched;
+				outpkt_amount += fetched;
+
+				if (!options.guest_polling) {
+					for (k = 0; k < nb_callfds; k++)
+						eventfd_write(callfds[k],
+								(eventfd_t)1);
+				}
+
+				rte_mempool_put_bulk(info.cop_pool,
+						(void **)ops_deq[j], fetched);
+				interval = rte_rdtsc_precise() - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+			}
+		}
+	}
+exit:
+	return ret;
+}
+
+
+static void
+unregister_drivers(int socket_num)
+{
+	int ret;
+
+	ret = rte_vhost_driver_unregister(options.socket_files[socket_num]);
+	if (ret != 0)
+		RTE_LOG(ERR, USER1,
+			"Fail to unregister vhost driver for %s.\n",
+			options.socket_files[socket_num]);
+}
+
+int
+main(int argc, char *argv[])
+{
+	struct rte_cryptodev_qp_conf qp_conf = {NB_CRYPTO_DESCRIPTORS};
+	struct rte_cryptodev_config config;
+	struct rte_cryptodev_info dev_info;
+	uint32_t cryptodev_id;
+	uint32_t worker_lcore;
+	char name[128];
+	uint32_t i = 0;
+	int ret;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		return -1;
+	argc -= ret;
+	argv += ret;
+
+	ret = vhost_crypto_parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Failed to parse arguments!\n");
+
+	info.cid = options.cid;
+	info.qid = options.qid;
+
+	worker_lcore = rte_get_next_lcore(0, 1, 0);
+	if (worker_lcore == RTE_MAX_LCORE)
+		rte_exit(EXIT_FAILURE, "Not enough lcore\n");
+
+	cryptodev_id = info.cid;
+	rte_cryptodev_info_get(cryptodev_id, &dev_info);
+	if (dev_info.max_nb_queue_pairs < info.qid + 1) {
+		RTE_LOG(ERR, USER1, "Number of queues cannot over %u",
+				dev_info.max_nb_queue_pairs);
+		goto error_exit;
+	}
+
+	config.nb_queue_pairs = dev_info.max_nb_queue_pairs;
+	config.socket_id = rte_lcore_to_socket_id(worker_lcore);
+
+	ret = rte_cryptodev_configure(cryptodev_id, &config);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to configure cryptodev %u",
+				cryptodev_id);
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "SESS_POOL_%u", worker_lcore);
+	info.sess_pool = rte_mempool_create(name, SESSION_MAP_ENTRIES,
+			rte_cryptodev_get_private_session_size(
+			cryptodev_id), 64, 0, NULL, NULL, NULL, NULL,
+			rte_lcore_to_socket_id(worker_lcore), 0);
+	if (!info.sess_pool) {
+		RTE_LOG(ERR, USER1, "Failed to create mempool");
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "COPPOOL_%u", worker_lcore);
+	info.cop_pool = rte_crypto_op_pool_create(name,
+			RTE_CRYPTO_OP_TYPE_SYMMETRIC, NB_MEMPOOL_OBJS,
+			NB_CACHE_OBJS, 0, rte_lcore_to_socket_id(worker_lcore));
+
+	if (!info.cop_pool) {
+		RTE_LOG(ERR, USER1, "Lcore %u failed to create crypto pool",
+				worker_lcore);
+		ret = -1;
+		goto error_exit;
+	}
+
+	info.nb_vids = options.nb_sockets;
+	for (i = 0; i < MAX_NB_SOCKETS; i++)
+		info.vids[i] = -1;
+
+	for (i = 0; i < dev_info.max_nb_queue_pairs; i++) {
+		ret = rte_cryptodev_queue_pair_setup(cryptodev_id, i,
+				&qp_conf, rte_lcore_to_socket_id(worker_lcore),
+				info.sess_pool);
+		if (ret < 0) {
+			RTE_LOG(ERR, USER1, "Failed to configure qp %u\n",
+					info.cid);
+			goto error_exit;
+		}
+	}
+
+	ret = rte_cryptodev_start(cryptodev_id);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to start cryptodev %u\n", info.cid);
+		goto error_exit;
+	}
+
+	info.cid = cryptodev_id;
+	info.lcore_id = worker_lcore;
+
+	if (rte_eal_remote_launch(vhost_crypto_worker, NULL, worker_lcore)
+			< 0) {
+		RTE_LOG(ERR, USER1, "Failed to start worker lcore");
+		goto error_exit;
+	}
+
+	for (i = 0; i < options.nb_sockets; i++) {
+		if (rte_vhost_driver_register(options.socket_files[i],
+				RTE_VHOST_USER_DEQUEUE_ZERO_COPY) < 0) {
+			RTE_LOG(ERR, USER1, "socket %s already exists\n",
+					options.socket_files[i]);
+			goto error_exit;
+		}
+
+		rte_vhost_driver_callback_register(options.socket_files[i],
+				&virtio_crypto_device_ops);
+
+		if (rte_vhost_driver_start(options.socket_files[i]) < 0) {
+			RTE_LOG(ERR, USER1, "failed to start vhost driver.\n");
+			goto error_exit;
+		}
+	}
+
+	RTE_LCORE_FOREACH(worker_lcore)
+		rte_eal_wait_lcore(worker_lcore);
+
+	rte_mempool_free(info.sess_pool);
+	rte_mempool_free(info.cop_pool);
+
+	return 0;
+
+error_exit:
+	for (i = 0; i < options.nb_sockets; i++)
+		unregister_drivers(i);
+
+	rte_mempool_free(info.cop_pool);
+	rte_mempool_free(info.sess_pool);
+
+	return -1;
+}
diff --git a/examples/vhost_crypto/meson.build b/examples/vhost_crypto/meson.build
new file mode 100644
index 0000000..0f4876f
--- /dev/null
+++ b/examples/vhost_crypto/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+allow_experimental_apis = true
+deps += ['vhost', 'cryptodev']
+cflags += ['-D_GNU_SOURCE','-D_FILE_OFFSET_BITS=64']
+sources = files(
+	'main.c'
+)
-- 
2.7.4

[PATCH v4 8/8] doc: update for vhost crypto support

[Fan Zhang]

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst       | 21 ++++++++
 doc/guides/rel_notes/release_18_05.rst    | 10 ++++
 doc/guides/sample_app_ug/index.rst        |  1 +
 doc/guides/sample_app_ug/vhost_crypto.rst | 84 +++++++++++++++++++++++++++++++
 4 files changed, 116 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 18227b6..9d6680c 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -160,6 +160,27 @@ The following is an overview of some key Vhost API functions:
 
   Receives (dequeues) ``count`` packets from guest, and stored them at ``pkts``.
 
+* ``rte_vhost_crypto_create(vid, cryptodev_id, sess_mempool, socket_id)``
+
+  As an extension of new_device(), this function adds virtio-crypto workload
+  acceleration capability to the device. All crypto workload is processed by
+  DPDK cryptodev with the device ID of ``cryptodev_id``.
+
+* ``rte_vhost_crypto_fetch_requests(vid, queue_id, ops, nb_ops)``
+
+  Receives (dequeues) ``nb_ops`` virtio-crypto requests from guest, parses
+  them to DPDK Crypto Operations, and fills the ``ops`` with parsing results.
+
+* ``rte_vhost_crypto_finalize_requests(queue_id, ops, nb_ops)``
+
+  After the ``ops`` are dequeued from Cryptodev, finalizes the jobs and
+  notifies the guest(s).
+
+* ``rte_vhost_user_register_extern_backend(vid, extern_data, ops)``
+
+  This function register private data, and pre and post vhost-user message
+  handlers.
+
 Vhost-user Implementations
 --------------------------
 
diff --git a/doc/guides/rel_notes/release_18_05.rst b/doc/guides/rel_notes/release_18_05.rst
index 3923dc2..3a0587d 100644
--- a/doc/guides/rel_notes/release_18_05.rst
+++ b/doc/guides/rel_notes/release_18_05.rst
@@ -41,6 +41,16 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **Added external backend support to vhost library
+
+  A new API is introduced for user to register the private data and vhost-user
+  message handlers to vhost library.
+
+* **Added crypto workload support to vhost library
+
+  New APIs are introduced in vhost library to enable virtio crypto support
+  including session creation/deletion handling and translating virito-crypto
+  request into DPDK crypto operations. A sample application is also introduced.
 
 API Changes
 -----------
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index e87afda..57e8354 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -44,6 +44,7 @@ Sample Applications User Guides
     vmdq_dcb_forwarding
     vhost
     vhost_scsi
+    vhost_crypto
     netmap_compatibility
     ip_pipeline
     test_pipeline
diff --git a/doc/guides/sample_app_ug/vhost_crypto.rst b/doc/guides/sample_app_ug/vhost_crypto.rst
new file mode 100644
index 0000000..8e6cb2a
--- /dev/null
+++ b/doc/guides/sample_app_ug/vhost_crypto.rst
@@ -0,0 +1,84 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2017-2018 Intel Corporation.
+
+
+Vhost_Crypto Sample Application
+===============================
+
+The vhost_crypto sample application implemented a simple Crypto device,
+which used as the  backend of Qemu vhost-user-crypto device. Similar with
+vhost-user-net and vhost-user-scsi device, the sample application used
+domain socket to communicate with Qemu, and the virtio ring was processed
+by vhost_crypto sample application.
+
+Testing steps
+-------------
+
+This section shows the steps how to start a VM with the crypto device as
+fast data path for critical application.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``examples`` sub-directory.
+
+Start the vhost_crypto example
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+    ./vhost_crypto [EAL options] -- [--socket-file PATH]
+        [--cdev-id ID] [--cdev-queue-id ID] [--zero-copy] [--guest-polling]
+
+where,
+
+* socket-file PATH: the path of UNIX socket file to be created, multiple
+  instances of this config item is supported. Upon absence of this item,
+  the default socket-file `/tmp/vhost_crypto1.socket` is used.
+
+* cdev-id ID: the target DPDK Cryptodev's ID to process the actual crypto
+  workload. Upon absence of this item the default value of `0` will be used.
+  For details of DPDK Cryptodev, please refer to DPDK Cryptodev Library
+  Programmers' Guide.
+
+* cdev-queue-id ID: the target DPDK Cryptodev's queue ID to process the
+  actual crypto workload. Upon absence of this item the default value of `0`
+  will be used. For details of DPDK Cryptodev, please refer to DPDK Cryptodev
+  Library Programmers' Guide.
+
+* zero-copy: the presence of this item means the ZERO-COPY feature will be
+  enabled. Otherwise it is disabled. PLEASE NOTE the ZERO-COPY feature is still
+  in experimental stage and may cause the problem like segmentation fault. If
+  the user wants to use LKCF in the guest, this feature shall be turned off.
+
+* guest-polling: the presence of this item means the application assumes the
+  guest works in polling mode, thus will NOT notify the guest completion of
+  processing.
+
+The application requires that crypto devices capable of performing
+the specified crypto operation are available on application initialization.
+This means that HW crypto device/s must be bound to a DPDK driver or
+a SW crypto device/s (virtual crypto PMD) must be created (using --vdev).
+
+.. _vhost_crypto_app_run_vm:
+
+Start the VM
+~~~~~~~~~~~~
+
+.. code-block:: console
+
+    qemu-system-x86_64 -machine accel=kvm \
+        -m $mem -object memory-backend-file,id=mem,size=$mem,\
+        mem-path=/dev/hugepages,share=on -numa node,memdev=mem \
+        -drive file=os.img,if=none,id=disk \
+        -device ide-hd,drive=disk,bootindex=0 \
+        -chardev socket,id={chardev_id},path={PATH} \
+        -object cryptodev-vhost-user,id={obj_id},chardev={chardev_id} \
+        -device virtio-crypto-pci,id={dev_id},cryptodev={obj_id} \
+        ...
+
+.. note::
+    You must check whether your Qemu can support "vhost-user-crypto" or not.
+
-- 
2.7.4

Re: [PATCH v4 1/8] lib/librte_vhost: add external backend support

[Wodkowski, PawelX]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Fan Zhang
> Sent: Thursday, March 29, 2018 2:53 PM
> To: dev@dpdk.org
> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan, Jianfeng
> <jianfeng.tan@intel.com>
> Subject: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external backend
> support
> 
> This patch adds external backend support to vhost library. The patch provides
> new APIs for the external backend to register pre and post vhost-user
> message
> handlers.
> 
> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> ---
>  lib/librte_vhost/rte_vhost.h           | 64
> +++++++++++++++++++++++++++++++++-
>  lib/librte_vhost/rte_vhost_version.map |  6 ++++
>  lib/librte_vhost/vhost.c               | 17 ++++++++-
>  lib/librte_vhost/vhost.h               |  8 +++--
>  lib/librte_vhost/vhost_user.c          | 33 +++++++++++++++++-
>  5 files changed, 123 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h
> index d332069..b902c44 100644
> --- a/lib/librte_vhost/rte_vhost.h
> +++ b/lib/librte_vhost/rte_vhost.h
> @@ -1,5 +1,5 @@
>  /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2017 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>   */
> 
>  #ifndef _RTE_VHOST_H_
> @@ -88,6 +88,55 @@ struct vhost_device_ops {
>  };
> 
>  /**
> + * function prototype for the vhost backend to handler specific vhost user
> + * messages prior to the master message handling
> + *
> + * @param vid
> + *  vhost device id
> + * @param msg
> + *  Message pointer.
> + * @param payload
> + *  Message payload.

No payload parameter.

> + * @param require_reply
> + *  If the handler requires sending a reply, this varaible shall be written 1,
> + *  otherwise 0.
> + * @param skip_master
> + *  If the handler requires skipping the master message handling, this
> variable
> + *  shall be written 1, otherwise 0.
> + * @return
> + *  0 on success, -1 on failure
> + */
> +typedef int (*rte_vhost_msg_pre_handle)(int vid, void *msg,
> +		uint32_t *require_reply, uint32_t *skip_master);
> +
> +/**
> + * function prototype for the vhost backend to handler specific vhost user
> + * messages after the master message handling is done
> + *
> + * @param vid
> + *  vhost device id
> + * @param msg
> + *  Message pointer.
> + * @param payload
> + *  Message payload.

No payload parameter :)

> + * @param require_reply
> + *  If the handler requires sending a reply, this varaible shall be written 1,
> + *  otherwise 0.
> + * @return
> + *  0 on success, -1 on failure
> + */
> +typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
> +		uint32_t *require_reply);
> +

What mean 'Message pointer' Is this const for us? Is this payload? Making msg 'void *' is not a
way to go here. Those pre and post handlers need to see exactly the same
structures like vhost_user.c file. Otherwise we can get into troubles when ABI
changes.

Also you can easily merge pre and post handlers into one handler with one
Parameter describing what phase of message processing we are now.

> +/**
> + * pre and post vhost user message handlers
> + */
> +struct vhost_user_extern_ops {
> +	rte_vhost_msg_pre_handle pre_msg_handle;
> +	rte_vhost_msg_post_handle post_msg_handle;
> +};
> +
> +/**
>   * Convert guest physical address to host virtual address
>   *
>   * @param mem
> @@ -434,6 +483,19 @@ int rte_vhost_vring_call(int vid, uint16_t vring_idx);
>   */
>  uint32_t rte_vhost_rx_queue_count(int vid, uint16_t qid);
> 
> +/**
> + * register external vhost backend
> + *
> + * @param vid
> + *  vhost device ID
> + * @param ops
> + *  ops that process external vhost user messages
> + * @return
> + *  0 on success, -1 on failure
> + */
> +int
> +rte_vhost_user_register_extern_ops(int vid, struct
> vhost_user_extern_ops *ops);
> +
>  #ifdef __cplusplus
>  }
>  #endif
> diff --git a/lib/librte_vhost/rte_vhost_version.map
> b/lib/librte_vhost/rte_vhost_version.map
> index df01031..91bf9f0 100644
> --- a/lib/librte_vhost/rte_vhost_version.map
> +++ b/lib/librte_vhost/rte_vhost_version.map
> @@ -59,3 +59,9 @@ DPDK_18.02 {
>  	rte_vhost_vring_call;
> 
>  } DPDK_17.08;
> +
> +DPDK_18.05 {
> +	global:
> +
> +	rte_vhost_user_register_extern_ops;
> +} DPDK_18.02;
> diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
> index a407067..80af341 100644
> --- a/lib/librte_vhost/vhost.c
> +++ b/lib/librte_vhost/vhost.c
> @@ -1,5 +1,5 @@
>  /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2016 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>   */
> 
>  #include <linux/vhost.h>
> @@ -627,3 +627,18 @@ rte_vhost_rx_queue_count(int vid, uint16_t qid)
> 
>  	return *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx;
>  }
> +
> +int
> +rte_vhost_user_register_extern_ops(int vid, struct
> vhost_user_extern_ops *ops)
> +{
> +	struct virtio_net *dev;
> +
> +	dev = get_device(vid);
> +	if (dev == NULL)
> +		return -1;
> +
> +	if (ops)
> +		rte_memcpy(&dev->extern_ops, ops, sizeof(*ops));
> +
> +	return 0;
> +}

Why we need this new "register" API? Why can't you use one of the 
(struct vhost_device_ops).reserved[0] field to put this callback there?
I think this is right time to utilize this field.

Can you do something similar to 
http://dpdk.org/ml/archives/dev/2018-March/094213.html ?

> diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
> index d947bc9..2072b88 100644
> --- a/lib/librte_vhost/vhost.h
> +++ b/lib/librte_vhost/vhost.h
> @@ -1,5 +1,5 @@
>  /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2014 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>   */
> 
>  #ifndef _VHOST_NET_CDEV_H_
> @@ -241,8 +241,12 @@ struct virtio_net {
>  	struct guest_page       *guest_pages;
> 
>  	int			slave_req_fd;
> -} __rte_cache_aligned;
> 
> +	/* private data for external virtio device */
> +	void			*extern_data;
> +	/* pre and post vhost user message handlers for externel backend */
> +	struct vhost_user_extern_ops extern_ops;
> +} __rte_cache_aligned;
> 
>  #define VHOST_LOG_PAGE	4096
> 
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 90ed211..ede8a5e 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -1,5 +1,5 @@
>  /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2016 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>   */
> 
>  #include <stdint.h>
> @@ -50,6 +50,8 @@ static const char
> *vhost_message_str[VHOST_USER_MAX] = {
>  	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
>  	[VHOST_USER_SET_SLAVE_REQ_FD]  =
> "VHOST_USER_SET_SLAVE_REQ_FD",
>  	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
> +	[VHOST_USER_CRYPTO_CREATE_SESS] =
> "VHOST_USER_CRYPTO_CREATE_SESS",
> +	[VHOST_USER_CRYPTO_CLOSE_SESS] =
> "VHOST_USER_CRYPTO_CLOSE_SESS",
>  };
> 
>  static uint64_t
> @@ -1302,6 +1304,7 @@ vhost_user_msg_handler(int vid, int fd)
>  	struct VhostUserMsg msg;
>  	int ret;
>  	int unlock_required = 0;
> +	uint32_t skip_master = 0;
> 
>  	dev = get_device(vid);
>  	if (dev == NULL)
> @@ -1379,6 +1382,21 @@ vhost_user_msg_handler(int vid, int fd)
> 
>  	}
> 
> +	if (dev->extern_ops.pre_msg_handle) {
> +		uint32_t need_reply;
> +
> +		ret = (*dev->extern_ops.pre_msg_handle)(dev->vid,
> +				(void *)&msg, &need_reply, &skip_master);
> +		if (ret < 0)
> +			goto skip_to_reply;
> +
> +		if (need_reply)
> +			send_vhost_reply(fd, &msg);
> +	}
> +
> +	if (skip_master)
> +		goto skip_to_post_handle;

This can be moved inside above  if () { } 

> +
>  	switch (msg.request.master) {
>  	case VHOST_USER_GET_FEATURES:
>  		msg.payload.u64 = vhost_user_get_features(dev);
> @@ -1479,9 +1497,22 @@ vhost_user_msg_handler(int vid, int fd)
>  	default:
>  		ret = -1;
>  		break;
> +	}
> +
> +skip_to_post_handle:
> +	if (dev->extern_ops.post_msg_handle) {
> +		uint32_t need_reply;
> +
> +		ret = (*dev->extern_ops.post_msg_handle)(
> +				dev->vid, (void *)&msg, &need_reply);
> +		if (ret < 0)
> +			goto skip_to_reply;
> 
> +		if (need_reply)
> +			send_vhost_reply(fd, &msg);
>  	}
> 
> +skip_to_reply:
>  	if (unlock_required)
>  		vhost_user_unlock_all_queue_pairs(dev);
> 
> --
> 2.7.4

Overall, I think, this direction where we need to go.

Pawel

Re: [PATCH v4 3/8] lib/librte_vhost: add session message handler

[Tan, Jianfeng]

For the title prefix, "vhost" is just fine.


On 3/29/2018 8:52 PM, Fan Zhang wrote:
> This patch adds session message handler to vhost crypto.
>
> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> ---
>   lib/librte_vhost/vhost_crypto.c | 428 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 428 insertions(+)
>   create mode 100644 lib/librte_vhost/vhost_crypto.c
>
> diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
> new file mode 100644
> index 0000000..c639b20
> --- /dev/null
> +++ b/lib/librte_vhost/vhost_crypto.c
> @@ -0,0 +1,428 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2017-2018 Intel Corporation
> + */
> +
> +#include <stdbool.h>

No bool variable below?

> +
> +#include <rte_malloc.h>
> +#include <rte_jhash.h>
> +#include <rte_mbuf.h>
> +#ifdef RTE_LIBRTE_VHOST_DEBUG

No need to add the ifdef.

> +#include <rte_hexdump.h>

Actually I don't see why we need this header file.

> +#endif
> +#include "vhost.h"
> +#include "vhost_user.h"
> +#include "rte_vhost_crypto.h"
> +
> +#define NB_MEMPOOL_OBJS			(1024)
> +#define NB_CRYPTO_DESCRIPTORS		(1024)
> +#define NB_CACHE_OBJS			(128)
> +
> +#define SESSION_MAP_ENTRIES		(1024) /**< Max nb sessions per vdev */
> +#define MAX_KEY_SIZE			(32)
> +#define VHOST_CRYPTO_MAX_IV_LEN		(16)
> +#define MAX_COUNT_DOWN_TIMES		(100)
> +
> +#define INHDR_LEN		(sizeof(struct virtio_crypto_inhdr))
> +#define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
> +				sizeof(struct rte_crypto_sym_op))

Some of above macros are not used in lib which we shall delete.

> +
> +#ifdef RTE_LIBRTE_VHOST_DEBUG
> +#define VC_LOG_ERR(fmt, args...)				\
> +	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
> +		"Vhost-Crypto",	__func__, __LINE__, ## args)
> +#define VC_LOG_INFO(fmt, args...)				\
> +	RTE_LOG(INFO, USER1, "[%s] %s() line %u: " fmt "\n",	\
> +		"Vhost-Crypto",	__func__, __LINE__, ## args)
> +
> +#define VC_LOG_DBG(fmt, args...)				\
> +	RTE_LOG(DEBUG, USER1, "[%s] %s() line %u: " fmt "\n",	\
> +		"Vhost-Crypto",	__func__, __LINE__, ## args)
> +#else
> +#define VC_LOG_ERR(fmt, args...)				\
> +	RTE_LOG(ERR, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
> +#define VC_LOG_INFO(fmt, args...)				\
> +	RTE_LOG(INFO, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
> +#define VC_LOG_DBG(fmt, args...)
> +#endif
> +
> +#define VIRTIO_CRYPTO_FEATURES ((1 << VIRTIO_F_NOTIFY_ON_EMPTY) |	\
> +		(1 << VIRTIO_RING_F_INDIRECT_DESC) |			\
> +		(1 << VIRTIO_RING_F_EVENT_IDX) |			\
> +		(1 << VIRTIO_CRYPTO_SERVICE_CIPHER) |			\
> +		(1 << VIRTIO_CRYPTO_SERVICE_HASH) |			\
> +		(1 << VIRTIO_CRYPTO_SERVICE_MAC) |			\
> +		(1 << VIRTIO_CRYPTO_SERVICE_AEAD) |			\
> +		(1 << VIRTIO_NET_F_CTRL_VQ))

Just wonder if the above are required features or supported features?

And if it's only used by 5/8 patch, shall we move the definition there?  
Let this patch to focus on "session message handler".

> +
> +
> +#define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
> +
> +/* Macro to get the buffer at the end of rte_crypto_op */
> +#define REQ_OP_OFFSET		(IV_OFFSET + VHOST_CRYPTO_MAX_IV_LEN)

Another unused macro?

> +
> +/**
> + * 1-to-1 mapping between RTE_CRYPTO_*ALGO* and VIRTIO_CRYPTO_*ALGO*, for
> + * algorithms not supported by RTE_CRYPTODEV, the -VIRTIO_CRYPTO_NOTSUPP is
> + * returned.
> + */
> +static int cipher_algo_transform[] = {

s/transform/map?

> +		RTE_CRYPTO_CIPHER_NULL,
> +		RTE_CRYPTO_CIPHER_ARC4,
> +		RTE_CRYPTO_CIPHER_AES_ECB,
> +		RTE_CRYPTO_CIPHER_AES_CBC,
> +		RTE_CRYPTO_CIPHER_AES_CTR,
> +		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_CIPHER_DES_ECB */
> +		RTE_CRYPTO_CIPHER_DES_CBC,
> +		RTE_CRYPTO_CIPHER_3DES_ECB,
> +		RTE_CRYPTO_CIPHER_3DES_CBC,
> +		RTE_CRYPTO_CIPHER_3DES_CTR,
> +		RTE_CRYPTO_CIPHER_KASUMI_F8,
> +		RTE_CRYPTO_CIPHER_SNOW3G_UEA2,
> +		RTE_CRYPTO_CIPHER_AES_F8,
> +		RTE_CRYPTO_CIPHER_AES_XTS,
> +		RTE_CRYPTO_CIPHER_ZUC_EEA3
> +};

How do we check if an input overflows the array?

> +
> +/**
> + * VIRTIO_CRYTPO_AUTH_* indexes are not sequential, the gaps are filled with
> + * -VIRTIO_CRYPTO_BADMSG errors.
> + */
> +static int auth_algo_transform[] = {
> +		RTE_CRYPTO_AUTH_NULL,
> +		RTE_CRYPTO_AUTH_MD5_HMAC,
> +		RTE_CRYPTO_AUTH_SHA1_HMAC,
> +		RTE_CRYPTO_AUTH_SHA224_HMAC,
> +		RTE_CRYPTO_AUTH_SHA256_HMAC,
> +		RTE_CRYPTO_AUTH_SHA384_HMAC,
> +		RTE_CRYPTO_AUTH_SHA512_HMAC,
> +		-VIRTIO_CRYPTO_BADMSG,

Where is this macro defined?

> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_CMAC_3DES */
> +		RTE_CRYPTO_AUTH_AES_CMAC,
> +		RTE_CRYPTO_AUTH_KASUMI_F9,
> +		RTE_CRYPTO_AUTH_SNOW3G_UIA2,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		-VIRTIO_CRYPTO_BADMSG,
> +		RTE_CRYPTO_AUTH_AES_GMAC,
> +		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_GMAC_TWOFISH */
> +		RTE_CRYPTO_AUTH_AES_CBC_MAC,
> +		-VIRTIO_CRYPTO_NOTSUPP, /* VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9 */
> +		RTE_CRYPTO_AUTH_AES_XCBC_MAC
> +};

Ditto.

> +
> +static int cipher_op_transform[] = {
> +		-VIRTIO_CRYPTO_BADMSG, /* meaningless */
> +		RTE_CRYPTO_CIPHER_OP_ENCRYPT,
> +		RTE_CRYPTO_CIPHER_OP_DECRYPT
> +};

Ditto.

> +
> +static int iv_lens[] = {
> +		-1, /* Invalid input */
> +		0, /* RTE_CRYPTO_CIPHER_NULL */
> +		8, /* RTE_CRYPTO_CIPHER_3DES_CBC */
> +		8, /* RTE_CRYPTO_CIPHER_3DES_CTR */
> +		8, /* RTE_CRYPTO_CIPHER_3DES_ECB */
> +		16, /* RTE_CRYPTO_CIPHER_AES_CBC */
> +		/* TODO: add common algos */
> +};

Ditto.

> +
> +/**
> + * vhost_crypto struct is used to maintain a number of virtio_cryptos and
> + * one DPDK crypto device that deals with all crypto workloads. It is declared
> + * here and defined in vhost_crypto.c

The last note is not valid now.

> + */
> +struct vhost_crypto {
> +	/** Used to lookup DPDK Cryptodev Session based on VIRTIO crypto
> +	 *  session ID.
> +	 */
> +	struct rte_hash *session_map;
> +	struct rte_mempool *mbuf_pool;
> +	struct rte_mempool *sess_pool;
> +
> +	/** DPDK cryptodev ID */
> +	uint8_t cid;
> +	uint16_t nb_qps;
> +
> +	uint64_t last_session_id;
> +
> +	uint64_t cache_session_id;
> +	struct rte_cryptodev_sym_session *cache_session;
> +	/** socket id for the device */
> +	int socket_id;
> +
> +	struct virtio_net *dev;
> +
> +	uint8_t option;
> +} __rte_cache_aligned;
> +
> +struct vhost_crypto_data_req {
> +	struct vring_desc *head;
> +	struct rte_vhost_memory *mem;
> +	struct virtio_crypto_inhdr *inhdr;
> +
> +	uint16_t desc_idx;
> +	uint32_t len;
> +	struct vhost_virtqueue *vq;
> +
> +	uint8_t zero_copy;
> +
> +	int vid;
> +
> +	struct vring_desc *wb_desc;
> +	uint16_t wb_len;
> +};

You might want to adjust the sequence of the fields so that we will not 
have some holes.

> +
> +static int
> +transform_cipher_param(struct rte_crypto_sym_xform *xform,
> +		VhostUserCryptoSessionParam *param)
> +{
> +	int ret;
> +
> +	ret = cipher_algo_transform[param->cipher_algo];
> +	if (unlikely(ret < 0))
> +		return ret;
> +
> +	xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
> +	xform->cipher.algo = ret;
> +	xform->cipher.key.length = param->cipher_key_len;
> +	if (xform->cipher.key.length > 0)
> +		xform->cipher.key.data = param->cipher_key_buf;
> +	ret = cipher_op_transform[param->dir];
> +	if (unlikely(ret < 0)) {
> +		VC_LOG_DBG("Bad operation type");
> +		return ret;
> +	}
> +	xform->cipher.op = ret;
> +	ret = iv_lens[xform->cipher.algo];
> +	if (unlikely(ret < 0))
> +		return ret;
> +	xform->cipher.iv.length = ret;
> +	xform->cipher.iv.offset = IV_OFFSET;
> +	return 0;
> +}
> +
> +static int
> +transform_chain_param(struct rte_crypto_sym_xform *xforms,
> +		VhostUserCryptoSessionParam *param)
> +{
> +	struct rte_crypto_sym_xform *xform_cipher, *xform_auth;
> +	int ret;
> +
> +	switch (param->chaining_dir) {
> +	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER:
> +		xform_auth = xforms;
> +		xform_cipher = xforms->next;
> +		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
> +		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
> +		break;
> +	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH:
> +		xform_cipher = xforms;
> +		xform_auth = xforms->next;
> +		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
> +		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
> +		break;
> +	default:
> +		return -VIRTIO_CRYPTO_BADMSG;
> +	}
> +
> +	/* cipher */
> +	ret = cipher_algo_transform[param->cipher_algo];
> +	if (unlikely(ret < 0))
> +		return ret;
> +	xform_cipher->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
> +	xform_cipher->cipher.algo = ret;
> +	xform_cipher->cipher.key.length = param->cipher_key_len;
> +	xform_cipher->cipher.key.data = param->cipher_key_buf;
> +	ret = iv_lens[xform_cipher->cipher.algo];
> +	if (unlikely(ret < 0))
> +		return ret;
> +	xform_cipher->cipher.iv.length = ret;
> +	xform_cipher->cipher.iv.offset = IV_OFFSET;
> +
> +	/* auth */
> +	xform_auth->type = RTE_CRYPTO_SYM_XFORM_AUTH;
> +	ret = auth_algo_transform[param->hash_algo];
> +	if (unlikely(ret < 0))
> +		return ret;
> +	xform_auth->auth.algo = ret;
> +	xform_auth->auth.digest_length = param->digest_len;
> +	xform_auth->auth.key.length = param->auth_key_len;
> +	xform_auth->auth.key.data = param->auth_key_buf;
> +
> +	return 0;
> +}
> +
> +static void
> +vhost_crypto_create_sess(struct vhost_crypto *vcrypto,
> +		VhostUserCryptoSessionParam *sess_param)
> +{
> +	struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0};
> +	struct rte_cryptodev_sym_session *session;
> +	int ret;
> +
> +	switch (sess_param->op_type) {
> +	case VIRTIO_CRYPTO_SYM_OP_NONE:
> +	case VIRTIO_CRYPTO_SYM_OP_CIPHER:
> +		ret = transform_cipher_param(&xform1, sess_param);
> +		if (unlikely(ret)) {
> +			VC_LOG_ERR("Error transform session msg (%i)", ret);
> +			sess_param->session_id = ret;
> +			return;
> +		}
> +		break;
> +	case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
> +		if (unlikely(sess_param->hash_mode !=
> +				VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH)) {
> +			sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
> +			VC_LOG_ERR("Error transform session message (%i)",
> +					-VIRTIO_CRYPTO_NOTSUPP);
> +			return;
> +		}
> +
> +		xform1.next = &xform2;
> +
> +		ret = transform_chain_param(&xform1, sess_param);
> +		if (unlikely(ret)) {
> +			VC_LOG_ERR("Error transform session message (%i)", ret);
> +			sess_param->session_id = ret;
> +			return;
> +		}
> +
> +		break;
> +	default:
> +		VC_LOG_ERR("Algorithm not yet supported");
> +		sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
> +		return;
> +	}
> +
> +	session = rte_cryptodev_sym_session_create(vcrypto->sess_pool);
> +	if (!session) {
> +		VC_LOG_ERR("Failed to create session");
> +		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
> +		return;
> +	}
> +
> +	if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1,
> +			vcrypto->sess_pool) < 0) {
> +		VC_LOG_ERR("Failed to initialize session");
> +		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
> +		return;
> +	}
> +
> +	/* insert hash to map */
> +	if (rte_hash_add_key_data(vcrypto->session_map,
> +			&vcrypto->last_session_id, session) < 0) {
> +		VC_LOG_ERR("Failed to insert session to hash table");
> +
> +		if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0)
> +			VC_LOG_ERR("Failed to clear session");
> +		else {
> +			if (rte_cryptodev_sym_session_free(session) < 0)
> +				VC_LOG_ERR("Failed to free session");
> +		}
> +		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
> +		return;
> +	}
> +
> +	VC_LOG_DBG("Session (key %lu, session %p) created.",
> +			vcrypto->last_session_id, session);
> +
> +	sess_param->session_id = vcrypto->last_session_id;
> +	vcrypto->last_session_id++;
> +}
> +
> +static int
> +vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id)
> +{
> +	struct rte_cryptodev_sym_session *session;
> +	uint64_t sess_id = session_id;
> +	int ret;
> +
> +	ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id,
> +			(void **)&session);
> +
> +	if (unlikely(ret < 0)) {
> +		VC_LOG_ERR("Failed to delete session (key %lu).", session_id);
> +		return -VIRTIO_CRYPTO_INVSESS;
> +	}
> +
> +	if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) {
> +		VC_LOG_DBG("Failed to delete session");
> +		return -VIRTIO_CRYPTO_ERR;
> +	}
> +
> +	if (rte_cryptodev_sym_session_free(session) < 0) {
> +		VC_LOG_DBG("Failed to delete session");
> +		return -VIRTIO_CRYPTO_ERR;
> +	}
> +
> +	if (rte_hash_del_key(vcrypto->session_map, &sess_id) < 0) {
> +		VC_LOG_DBG("Failed to delete session from hash table.");
> +		return -VIRTIO_CRYPTO_ERR;
> +	}
> +
> +	VC_LOG_DBG("Session (key %lu, session %p) deleted.", sess_id,
> +			session);
> +
> +	return 0;
> +}
> +
> +static int
> +vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
> +{
> +	struct virtio_net *dev = get_device(vid);
> +	struct vhost_crypto *vcrypto;
> +	VhostUserMsg *vmsg = msg;
> +	int ret = 0;
> +
> +	if (dev == NULL || require_reply == NULL) {
> +		VC_LOG_ERR("Invalid vid %i", vid);
> +		return -EINVAL;
> +	}
> +
> +	vcrypto = dev->extern_data;
> +	if (vcrypto == NULL) {
> +		VC_LOG_ERR("Cannot find required data, is it initialized?");
> +		return -ENOENT;
> +	}
> +
> +	*require_reply = 0;
> +
> +	if (vmsg->request.master == VHOST_USER_CRYPTO_CREATE_SESS) {
> +		vhost_crypto_create_sess(vcrypto,
> +				&vmsg->payload.crypto_session);
> +		*require_reply = 1;
> +	} else if (vmsg->request.master == VHOST_USER_CRYPTO_CLOSE_SESS)
> +		ret = vhost_crypto_close_sess(vcrypto, vmsg->payload.u64);
> +	else
> +		ret = -EINVAL;
> +
> +	return ret;
> +}

Re: [PATCH v4 1/8] lib/librte_vhost: add external backend support

[Zhang, Roy Fan]

Hi Pawel,

> -----Original Message-----
> From: Wodkowski, PawelX
> Sent: Thursday, March 29, 2018 2:48 PM
> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org
> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan, Jianfeng
> <jianfeng.tan@intel.com>
> Subject: RE: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
> backend support
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Fan Zhang
> > Sent: Thursday, March 29, 2018 2:53 PM
> > To: dev@dpdk.org
> > Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan,
> Jianfeng
> > <jianfeng.tan@intel.com>
> > Subject: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
> > backend support
> >
> > This patch adds external backend support to vhost library. The patch
> > provides new APIs for the external backend to register pre and post
> > vhost-user message handlers.
> >
> > Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> > ---
> >  lib/librte_vhost/rte_vhost.h           | 64
> > +++++++++++++++++++++++++++++++++-
> >  lib/librte_vhost/rte_vhost_version.map |  6 ++++
> >  lib/librte_vhost/vhost.c               | 17 ++++++++-
> >  lib/librte_vhost/vhost.h               |  8 +++--
> >  lib/librte_vhost/vhost_user.c          | 33 +++++++++++++++++-
> >  5 files changed, 123 insertions(+), 5 deletions(-)
> >
> > diff --git a/lib/librte_vhost/rte_vhost.h
> > b/lib/librte_vhost/rte_vhost.h index d332069..b902c44 100644
> > --- a/lib/librte_vhost/rte_vhost.h
> > +++ b/lib/librte_vhost/rte_vhost.h
> > @@ -1,5 +1,5 @@
> >  /* SPDX-License-Identifier: BSD-3-Clause
> > - * Copyright(c) 2010-2017 Intel Corporation
> > + * Copyright(c) 2010-2018 Intel Corporation
> >   */
> >
> >  #ifndef _RTE_VHOST_H_
> > @@ -88,6 +88,55 @@ struct vhost_device_ops {  };
> >
> >  /**
> > + * function prototype for the vhost backend to handler specific vhost
> > + user
> > + * messages prior to the master message handling
> > + *
> > + * @param vid
> > + *  vhost device id
> > + * @param msg
> > + *  Message pointer.
> > + * @param payload
> > + *  Message payload.
> 
> No payload parameter.
Sorry about that. I will fix the comment.

> 
> > + * @param require_reply
> > + *  If the handler requires sending a reply, this varaible shall be
> > + written 1,
> > + *  otherwise 0.
> > + * @param skip_master
> > + *  If the handler requires skipping the master message handling,
> > + this
> > variable
> > + *  shall be written 1, otherwise 0.
> > + * @return
> > + *  0 on success, -1 on failure
> > + */
> > +typedef int (*rte_vhost_msg_pre_handle)(int vid, void *msg,
> > +		uint32_t *require_reply, uint32_t *skip_master);
> > +
> > +/**
> > + * function prototype for the vhost backend to handler specific vhost
> > +user
> > + * messages after the master message handling is done
> > + *
> > + * @param vid
> > + *  vhost device id
> > + * @param msg
> > + *  Message pointer.
> > + * @param payload
> > + *  Message payload.
> 
> No payload parameter :)
> 

Same here

> > + * @param require_reply
> > + *  If the handler requires sending a reply, this varaible shall be
> > +written 1,
> > + *  otherwise 0.
> > + * @return
> > + *  0 on success, -1 on failure
> > + */
> > +typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
> > +		uint32_t *require_reply);
> > +
> 
> What mean 'Message pointer' Is this const for us? Is this payload? Making
> msg 'void *' is not a way to go here. Those pre and post handlers need to see
> exactly the same structures like vhost_user.c file. Otherwise we can get into
> troubles when ABI changes.

It is the pointer to the vhost_user message. It cannot be const as the backend
may change the payload. 

> 
> Also you can easily merge pre and post handlers into one handler with one
> Parameter describing what phase of message processing we are now.
> 

No I don't think so. To do so it will be quite unclear in the future as we are
using one function to do two totally different things. 

> > +/**
> > + * pre and post vhost user message handlers  */ struct
> > +vhost_user_extern_ops {
> > +	rte_vhost_msg_pre_handle pre_msg_handle;
> > +	rte_vhost_msg_post_handle post_msg_handle; };
> > +
> > +/**
> >   * Convert guest physical address to host virtual address
> >   *
> >   * @param mem
> > @@ -434,6 +483,19 @@ int rte_vhost_vring_call(int vid, uint16_t vring_idx);
> >   */
> >  uint32_t rte_vhost_rx_queue_count(int vid, uint16_t qid);
> >
> > +/**
> > + * register external vhost backend
> > + *
> > + * @param vid
> > + *  vhost device ID
> > + * @param ops
> > + *  ops that process external vhost user messages
> > + * @return
> > + *  0 on success, -1 on failure
> > + */
> > +int
> > +rte_vhost_user_register_extern_ops(int vid, struct
> > vhost_user_extern_ops *ops);
> > +
> >  #ifdef __cplusplus
> >  }
> >  #endif
> > diff --git a/lib/librte_vhost/rte_vhost_version.map
> > b/lib/librte_vhost/rte_vhost_version.map
> > index df01031..91bf9f0 100644
> > --- a/lib/librte_vhost/rte_vhost_version.map
> > +++ b/lib/librte_vhost/rte_vhost_version.map
> > @@ -59,3 +59,9 @@ DPDK_18.02 {
> >  	rte_vhost_vring_call;
> >
> >  } DPDK_17.08;
> > +
> > +DPDK_18.05 {
> > +	global:
> > +
> > +	rte_vhost_user_register_extern_ops;
> > +} DPDK_18.02;
> > diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c index
> > a407067..80af341 100644
> > --- a/lib/librte_vhost/vhost.c
> > +++ b/lib/librte_vhost/vhost.c
> > @@ -1,5 +1,5 @@
> >  /* SPDX-License-Identifier: BSD-3-Clause
> > - * Copyright(c) 2010-2016 Intel Corporation
> > + * Copyright(c) 2010-2018 Intel Corporation
> >   */
> >
> >  #include <linux/vhost.h>
> > @@ -627,3 +627,18 @@ rte_vhost_rx_queue_count(int vid, uint16_t qid)
> >
> >  	return *((volatile uint16_t *)&vq->avail->idx) - vq->last_avail_idx;
> > }
> > +
> > +int
> > +rte_vhost_user_register_extern_ops(int vid, struct
> > vhost_user_extern_ops *ops)
> > +{
> > +	struct virtio_net *dev;
> > +
> > +	dev = get_device(vid);
> > +	if (dev == NULL)
> > +		return -1;
> > +
> > +	if (ops)
> > +		rte_memcpy(&dev->extern_ops, ops, sizeof(*ops));
> > +
> > +	return 0;
> > +}
> 
> Why we need this new "register" API? Why can't you use one of the (struct
> vhost_device_ops).reserved[0] field to put this callback there?
> I think this is right time to utilize this field.
> 

The patch here is a more generic and intuitive way for external backend to
register the handlers to process the vhost user message only recognized by it.
Please read Maxime's comments in v2 version of this patch.
http://dpdk.org/ml/archives/dev/2018-March/093408.html.
As we discussed we need 2 different handlers for external vhost user device
to handle device specifc vhost user messages. A public API is needed. 

> Can you do something similar to
> http://dpdk.org/ml/archives/dev/2018-March/094213.html ?

The patch content here causes the least damage to the existing library. Plus
the patch you mentioned won't help with the pre and post handlers problem - 
or it would consume all of two remaining reserved fields in vhost_user_ops
structure for pre and post handlers, respectively.

> 
> > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h index
> > d947bc9..2072b88 100644
> > --- a/lib/librte_vhost/vhost.h
> > +++ b/lib/librte_vhost/vhost.h
> > @@ -1,5 +1,5 @@
> >  /* SPDX-License-Identifier: BSD-3-Clause
> > - * Copyright(c) 2010-2014 Intel Corporation
> > + * Copyright(c) 2010-2018 Intel Corporation
> >   */
> >
> >  #ifndef _VHOST_NET_CDEV_H_
> > @@ -241,8 +241,12 @@ struct virtio_net {
> >  	struct guest_page       *guest_pages;
> >
> >  	int			slave_req_fd;
> > -} __rte_cache_aligned;
> >
> > +	/* private data for external virtio device */
> > +	void			*extern_data;
> > +	/* pre and post vhost user message handlers for externel backend */
> > +	struct vhost_user_extern_ops extern_ops; } __rte_cache_aligned;
> >
> >  #define VHOST_LOG_PAGE	4096
> >
> > diff --git a/lib/librte_vhost/vhost_user.c
> > b/lib/librte_vhost/vhost_user.c index 90ed211..ede8a5e 100644
> > --- a/lib/librte_vhost/vhost_user.c
> > +++ b/lib/librte_vhost/vhost_user.c
> > @@ -1,5 +1,5 @@
> >  /* SPDX-License-Identifier: BSD-3-Clause
> > - * Copyright(c) 2010-2016 Intel Corporation
> > + * Copyright(c) 2010-2018 Intel Corporation
> >   */
> >
> >  #include <stdint.h>
> > @@ -50,6 +50,8 @@ static const char
> > *vhost_message_str[VHOST_USER_MAX] = {
> >  	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
> >  	[VHOST_USER_SET_SLAVE_REQ_FD]  =
> > "VHOST_USER_SET_SLAVE_REQ_FD",
> >  	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
> > +	[VHOST_USER_CRYPTO_CREATE_SESS] =
> > "VHOST_USER_CRYPTO_CREATE_SESS",
> > +	[VHOST_USER_CRYPTO_CLOSE_SESS] =
> > "VHOST_USER_CRYPTO_CLOSE_SESS",
> >  };
> >
> >  static uint64_t
> > @@ -1302,6 +1304,7 @@ vhost_user_msg_handler(int vid, int fd)
> >  	struct VhostUserMsg msg;
> >  	int ret;
> >  	int unlock_required = 0;
> > +	uint32_t skip_master = 0;
> >
> >  	dev = get_device(vid);
> >  	if (dev == NULL)
> > @@ -1379,6 +1382,21 @@ vhost_user_msg_handler(int vid, int fd)
> >
> >  	}
> >
> > +	if (dev->extern_ops.pre_msg_handle) {
> > +		uint32_t need_reply;
> > +
> > +		ret = (*dev->extern_ops.pre_msg_handle)(dev->vid,
> > +				(void *)&msg, &need_reply, &skip_master);
> > +		if (ret < 0)
> > +			goto skip_to_reply;
> > +
> > +		if (need_reply)
> > +			send_vhost_reply(fd, &msg);
> > +	}
> > +
> > +	if (skip_master)
> > +		goto skip_to_post_handle;
> 
> This can be moved inside above  if () { }

Yes, you are right.

> 
> > +
> >  	switch (msg.request.master) {
> >  	case VHOST_USER_GET_FEATURES:
> >  		msg.payload.u64 = vhost_user_get_features(dev); @@ -
> 1479,9 +1497,22
> > @@ vhost_user_msg_handler(int vid, int fd)
> >  	default:
> >  		ret = -1;
> >  		break;
> > +	}
> > +
> > +skip_to_post_handle:
> > +	if (dev->extern_ops.post_msg_handle) {
> > +		uint32_t need_reply;
> > +
> > +		ret = (*dev->extern_ops.post_msg_handle)(
> > +				dev->vid, (void *)&msg, &need_reply);
> > +		if (ret < 0)
> > +			goto skip_to_reply;
> >
> > +		if (need_reply)
> > +			send_vhost_reply(fd, &msg);
> >  	}
> >
> > +skip_to_reply:
> >  	if (unlock_required)
> >  		vhost_user_unlock_all_queue_pairs(dev);
> >
> > --
> > 2.7.4
> 
> Overall, I think, this direction where we need to go.
> 
> Pawel

Regards,
Fan

Re: [PATCH v4 1/8] lib/librte_vhost: add external backend support

[Maxime Coquelin]

Hi Pawel, Fan,

On 04/01/2018 09:53 PM, Zhang, Roy Fan wrote:
> Hi Pawel,
> 
>> -----Original Message-----
>> From: Wodkowski, PawelX
>> Sent: Thursday, March 29, 2018 2:48 PM
>> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org
>> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan, Jianfeng
>> <jianfeng.tan@intel.com>
>> Subject: RE: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
>> backend support
>>
>>> -----Original Message-----
>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Fan Zhang
>>> Sent: Thursday, March 29, 2018 2:53 PM
>>> To: dev@dpdk.org
>>> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan,
>> Jianfeng
>>> <jianfeng.tan@intel.com>
>>> Subject: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
>>> backend support
>>>
>>> This patch adds external backend support to vhost library. The patch
>>> provides new APIs for the external backend to register pre and post
>>> vhost-user message handlers.
>>>
>>> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
>>> ---
>>>   lib/librte_vhost/rte_vhost.h           | 64
>>> +++++++++++++++++++++++++++++++++-
>>>   lib/librte_vhost/rte_vhost_version.map |  6 ++++
>>>   lib/librte_vhost/vhost.c               | 17 ++++++++-
>>>   lib/librte_vhost/vhost.h               |  8 +++--
>>>   lib/librte_vhost/vhost_user.c          | 33 +++++++++++++++++-
>>>   5 files changed, 123 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/lib/librte_vhost/rte_vhost.h
>>> b/lib/librte_vhost/rte_vhost.h index d332069..b902c44 100644
>>> --- a/lib/librte_vhost/rte_vhost.h
>>> +++ b/lib/librte_vhost/rte_vhost.h
>>> @@ -1,5 +1,5 @@

<snip/>

> 
>>> + * @param require_reply
>>> + *  If the handler requires sending a reply, this varaible shall be
>>> +written 1,
>>> + *  otherwise 0.
>>> + * @return
>>> + *  0 on success, -1 on failure
>>> + */
>>> +typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
>>> +		uint32_t *require_reply);
>>> +
>>
>> What mean 'Message pointer' Is this const for us? Is this payload? Making
>> msg 'void *' is not a way to go here. Those pre and post handlers need to see
>> exactly the same structures like vhost_user.c file. Otherwise we can get into
>> troubles when ABI changes.
> 
> It is the pointer to the vhost_user message. It cannot be const as the backend
> may change the payload.
> 
>>
>> Also you can easily merge pre and post handlers into one handler with one
>> Parameter describing what phase of message processing we are now.
>>
> 
> No I don't think so. To do so it will be quite unclear in the future as we are
> using one function to do two totally different things.

Time is running out for v18.05 integration deadline (April 6th), and we 
haven't reached a consensus.

Except this API point, I think vhost-crypto is at the right level.
Since vhost-crypto lives in librte_vhost, I propose Fan cooks an
intermediate solution that does not need API change.

Doing this, we postpone the API change to v18.08, so we have time to
discuss what the right API should be. Once agreed, vhost-crypto moves to
the new API.

Pawel, Jianfeng, Fan, is it fine for you?

Thanks,
Maxime

Re: [PATCH v4 1/8] lib/librte_vhost: add external backend support

[Zhang, Roy Fan]

Hi Maxime,

No problem. I will work on that.
Pawel, Jianfeng, if you guys have other concerns or suggestions, please give me a shout.

Thanks a lot guys, for the review and help!

Regards,
Fan

> -----Original Message-----
> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> Sent: Tuesday, April 3, 2018 2:45 PM
> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; Wodkowski, PawelX
> <pawelx.wodkowski@intel.com>; dev@dpdk.org
> Cc: jianjay.zhou@huawei.com; Tan, Jianfeng <jianfeng.tan@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
> backend support
> 
> Hi Pawel, Fan,
> 
> On 04/01/2018 09:53 PM, Zhang, Roy Fan wrote:
> > Hi Pawel,
> >
> >> -----Original Message-----
> >> From: Wodkowski, PawelX
> >> Sent: Thursday, March 29, 2018 2:48 PM
> >> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org
> >> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan,
> >> Jianfeng <jianfeng.tan@intel.com>
> >> Subject: RE: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
> >> backend support
> >>
> >>> -----Original Message-----
> >>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Fan Zhang
> >>> Sent: Thursday, March 29, 2018 2:53 PM
> >>> To: dev@dpdk.org
> >>> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan,
> >> Jianfeng
> >>> <jianfeng.tan@intel.com>
> >>> Subject: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
> >>> backend support
> >>>
> >>> This patch adds external backend support to vhost library. The patch
> >>> provides new APIs for the external backend to register pre and post
> >>> vhost-user message handlers.
> >>>
> >>> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> >>> ---
> >>>   lib/librte_vhost/rte_vhost.h           | 64
> >>> +++++++++++++++++++++++++++++++++-
> >>>   lib/librte_vhost/rte_vhost_version.map |  6 ++++
> >>>   lib/librte_vhost/vhost.c               | 17 ++++++++-
> >>>   lib/librte_vhost/vhost.h               |  8 +++--
> >>>   lib/librte_vhost/vhost_user.c          | 33 +++++++++++++++++-
> >>>   5 files changed, 123 insertions(+), 5 deletions(-)
> >>>
> >>> diff --git a/lib/librte_vhost/rte_vhost.h
> >>> b/lib/librte_vhost/rte_vhost.h index d332069..b902c44 100644
> >>> --- a/lib/librte_vhost/rte_vhost.h
> >>> +++ b/lib/librte_vhost/rte_vhost.h
> >>> @@ -1,5 +1,5 @@
> 
> <snip/>
> 
> >
> >>> + * @param require_reply
> >>> + *  If the handler requires sending a reply, this varaible shall be
> >>> +written 1,
> >>> + *  otherwise 0.
> >>> + * @return
> >>> + *  0 on success, -1 on failure
> >>> + */
> >>> +typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
> >>> +		uint32_t *require_reply);
> >>> +
> >>
> >> What mean 'Message pointer' Is this const for us? Is this payload?
> >> Making msg 'void *' is not a way to go here. Those pre and post
> >> handlers need to see exactly the same structures like vhost_user.c
> >> file. Otherwise we can get into troubles when ABI changes.
> >
> > It is the pointer to the vhost_user message. It cannot be const as the
> > backend may change the payload.
> >
> >>
> >> Also you can easily merge pre and post handlers into one handler with
> >> one Parameter describing what phase of message processing we are now.
> >>
> >
> > No I don't think so. To do so it will be quite unclear in the future
> > as we are using one function to do two totally different things.
> 
> Time is running out for v18.05 integration deadline (April 6th), and we haven't
> reached a consensus.
> 
> Except this API point, I think vhost-crypto is at the right level.
> Since vhost-crypto lives in librte_vhost, I propose Fan cooks an intermediate
> solution that does not need API change.
> 
> Doing this, we postpone the API change to v18.08, so we have time to discuss
> what the right API should be. Once agreed, vhost-crypto moves to the new
> API.
> 
> Pawel, Jianfeng, Fan, is it fine for you?
> 
> Thanks,
> Maxime

Re: [PATCH v4 1/8] lib/librte_vhost: add external backend support

[Tan, Jianfeng]

On 4/3/2018 9:44 PM, Maxime Coquelin wrote:
> Hi Pawel, Fan,
>
> On 04/01/2018 09:53 PM, Zhang, Roy Fan wrote:
>> Hi Pawel,
>>
>>> -----Original Message-----
>>> From: Wodkowski, PawelX
>>> Sent: Thursday, March 29, 2018 2:48 PM
>>> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org
>>> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan, Jianfeng
>>> <jianfeng.tan@intel.com>
>>> Subject: RE: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
>>> backend support
>>>
>>>> -----Original Message-----
>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Fan Zhang
>>>> Sent: Thursday, March 29, 2018 2:53 PM
>>>> To: dev@dpdk.org
>>>> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan,
>>> Jianfeng
>>>> <jianfeng.tan@intel.com>
>>>> Subject: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
>>>> backend support
>>>>
>>>> This patch adds external backend support to vhost library. The patch
>>>> provides new APIs for the external backend to register pre and post
>>>> vhost-user message handlers.
>>>>
>>>> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
>>>> ---
>>>>   lib/librte_vhost/rte_vhost.h           | 64
>>>> +++++++++++++++++++++++++++++++++-
>>>>   lib/librte_vhost/rte_vhost_version.map |  6 ++++
>>>>   lib/librte_vhost/vhost.c               | 17 ++++++++-
>>>>   lib/librte_vhost/vhost.h               |  8 +++--
>>>>   lib/librte_vhost/vhost_user.c          | 33 +++++++++++++++++-
>>>>   5 files changed, 123 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/lib/librte_vhost/rte_vhost.h
>>>> b/lib/librte_vhost/rte_vhost.h index d332069..b902c44 100644
>>>> --- a/lib/librte_vhost/rte_vhost.h
>>>> +++ b/lib/librte_vhost/rte_vhost.h
>>>> @@ -1,5 +1,5 @@
>
> <snip/>
>
>>
>>>> + * @param require_reply
>>>> + *  If the handler requires sending a reply, this varaible shall be
>>>> +written 1,
>>>> + *  otherwise 0.
>>>> + * @return
>>>> + *  0 on success, -1 on failure
>>>> + */
>>>> +typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
>>>> +        uint32_t *require_reply);
>>>> +
>>>
>>> What mean 'Message pointer' Is this const for us? Is this payload? 
>>> Making
>>> msg 'void *' is not a way to go here. Those pre and post handlers 
>>> need to see
>>> exactly the same structures like vhost_user.c file. Otherwise we can 
>>> get into
>>> troubles when ABI changes.
>>
>> It is the pointer to the vhost_user message. It cannot be const as 
>> the backend
>> may change the payload.
>>
>>>
>>> Also you can easily merge pre and post handlers into one handler 
>>> with one
>>> Parameter describing what phase of message processing we are now.
>>>
>>
>> No I don't think so. To do so it will be quite unclear in the future 
>> as we are
>> using one function to do two totally different things.
>
> Time is running out for v18.05 integration deadline (April 6th), and 
> we haven't reached a consensus.
>
> Except this API point, I think vhost-crypto is at the right level.
> Since vhost-crypto lives in librte_vhost, I propose Fan cooks an
> intermediate solution that does not need API change.
>
> Doing this, we postpone the API change to v18.08, so we have time to
> discuss what the right API should be. Once agreed, vhost-crypto moves to
> the new API.
>
> Pawel, Jianfeng, Fan, is it fine for you?

+1. This can avoid blocking this patch set, and give more time for 
discussing new APIs and external structs.

Thanks,
Jianfeng

>
> Thanks,
> Maxime

Re: [PATCH v4 1/8] lib/librte_vhost: add external backend support

[Wodkowski, PawelX]

> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Tuesday, April 3, 2018 4:43 PM
> To: Maxime Coquelin <maxime.coquelin@redhat.com>; Zhang, Roy Fan
> <roy.fan.zhang@intel.com>; Wodkowski, PawelX
> <pawelx.wodkowski@intel.com>; dev@dpdk.org
> Cc: jianjay.zhou@huawei.com
> Subject: Re: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
> backend support
> 
> 
> 
> On 4/3/2018 9:44 PM, Maxime Coquelin wrote:
> > Hi Pawel, Fan,
> >
> > On 04/01/2018 09:53 PM, Zhang, Roy Fan wrote:
> >> Hi Pawel,
> >>
> >>> -----Original Message-----
> >>> From: Wodkowski, PawelX
> >>> Sent: Thursday, March 29, 2018 2:48 PM
> >>> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org
> >>> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan,
> Jianfeng
> >>> <jianfeng.tan@intel.com>
> >>> Subject: RE: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
> >>> backend support
> >>>
> >>>> -----Original Message-----
> >>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Fan Zhang
> >>>> Sent: Thursday, March 29, 2018 2:53 PM
> >>>> To: dev@dpdk.org
> >>>> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com; Tan,
> >>> Jianfeng
> >>>> <jianfeng.tan@intel.com>
> >>>> Subject: [dpdk-dev] [PATCH v4 1/8] lib/librte_vhost: add external
> >>>> backend support
> >>>>
> >>>> This patch adds external backend support to vhost library. The patch
> >>>> provides new APIs for the external backend to register pre and post
> >>>> vhost-user message handlers.
> >>>>
> >>>> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> >>>> ---
> >>>>   lib/librte_vhost/rte_vhost.h           | 64
> >>>> +++++++++++++++++++++++++++++++++-
> >>>>   lib/librte_vhost/rte_vhost_version.map |  6 ++++
> >>>>   lib/librte_vhost/vhost.c               | 17 ++++++++-
> >>>>   lib/librte_vhost/vhost.h               |  8 +++--
> >>>>   lib/librte_vhost/vhost_user.c          | 33 +++++++++++++++++-
> >>>>   5 files changed, 123 insertions(+), 5 deletions(-)
> >>>>
> >>>> diff --git a/lib/librte_vhost/rte_vhost.h
> >>>> b/lib/librte_vhost/rte_vhost.h index d332069..b902c44 100644
> >>>> --- a/lib/librte_vhost/rte_vhost.h
> >>>> +++ b/lib/librte_vhost/rte_vhost.h
> >>>> @@ -1,5 +1,5 @@
> >
> > <snip/>
> >
> >>
> >>>> + * @param require_reply
> >>>> + *  If the handler requires sending a reply, this varaible shall be
> >>>> +written 1,
> >>>> + *  otherwise 0.
> >>>> + * @return
> >>>> + *  0 on success, -1 on failure
> >>>> + */
> >>>> +typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
> >>>> +        uint32_t *require_reply);
> >>>> +
> >>>
> >>> What mean 'Message pointer' Is this const for us? Is this payload?
> >>> Making
> >>> msg 'void *' is not a way to go here. Those pre and post handlers
> >>> need to see
> >>> exactly the same structures like vhost_user.c file. Otherwise we can
> >>> get into
> >>> troubles when ABI changes.
> >>
> >> It is the pointer to the vhost_user message. It cannot be const as
> >> the backend
> >> may change the payload.
> >>
> >>>
> >>> Also you can easily merge pre and post handlers into one handler
> >>> with one
> >>> Parameter describing what phase of message processing we are now.
> >>>
> >>
> >> No I don't think so. To do so it will be quite unclear in the future
> >> as we are
> >> using one function to do two totally different things.
> >
> > Time is running out for v18.05 integration deadline (April 6th), and
> > we haven't reached a consensus.
> >
> > Except this API point, I think vhost-crypto is at the right level.
> > Since vhost-crypto lives in librte_vhost, I propose Fan cooks an
> > intermediate solution that does not need API change.
> >
> > Doing this, we postpone the API change to v18.08, so we have time to
> > discuss what the right API should be. Once agreed, vhost-crypto moves to
> > the new API.
> >
> > Pawel, Jianfeng, Fan, is it fine for you?
> 
> +1. This can avoid blocking this patch set, and give more time for
> discussing new APIs and external structs.
> 
> Thanks,
> Jianfeng
> 
> >
> > Thanks,
> > Maxime

Fine for me too.
Pawel

Re: [PATCH v4 3/8] lib/librte_vhost: add session message handler

[Zhang, Roy Fan]

Hi Jianfeng,

Thanks for the review.

> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Thursday, March 29, 2018 4:03 PM
> To: Zhang, Roy Fan <roy.fan.zhang@intel.com>; dev@dpdk.org
> Cc: maxime.coquelin@redhat.com; jianjay.zhou@huawei.com
> Subject: Re: [PATCH v4 3/8] lib/librte_vhost: add session message handler
> 
> For the title prefix, "vhost" is just fine.
> 
> 
> On 3/29/2018 8:52 PM, Fan Zhang wrote:
> > This patch adds session message handler to vhost crypto.
> >
> > Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> > ---
> >   lib/librte_vhost/vhost_crypto.c | 428
> ++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 428 insertions(+)
> >   create mode 100644 lib/librte_vhost/vhost_crypto.c
> >
> > diff --git a/lib/librte_vhost/vhost_crypto.c
> > b/lib/librte_vhost/vhost_crypto.c new file mode 100644 index
> > 0000000..c639b20
> > --- /dev/null
> > +++ b/lib/librte_vhost/vhost_crypto.c
> > @@ -0,0 +1,428 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2017-2018 Intel Corporation  */
> > +
> > +#include <stdbool.h>
> 
> No bool variable below?
> 

I found a compile error when not adding this header.
"
In file included from /root/dpdk/lib/librte_vhost/vhost_crypto.c:12:0:
/root/dpdk/lib/librte_vhost/vhost.h:366:44: error: unknown type name 'bool'
 void vhost_set_builtin_virtio_net(int vid, bool enable);
"

> > +
> > +#include <rte_malloc.h>
> > +#include <rte_jhash.h>
> > +#include <rte_mbuf.h>
> > +#ifdef RTE_LIBRTE_VHOST_DEBUG
> 
> No need to add the ifdef.
> 
> > +#include <rte_hexdump.h>
> 
> Actually I don't see why we need this header file.

This header help dump the content of request message for debugging reason. I can remove it.

> 
> > +#endif
> > +#include "vhost.h"
> > +#include "vhost_user.h"
> > +#include "rte_vhost_crypto.h"
> > +
> > +#define NB_MEMPOOL_OBJS			(1024)
> > +#define NB_CRYPTO_DESCRIPTORS		(1024)
> > +#define NB_CACHE_OBJS			(128)
> > +
> > +#define SESSION_MAP_ENTRIES		(1024) /**< Max nb sessions
> per vdev */
> > +#define MAX_KEY_SIZE			(32)
> > +#define VHOST_CRYPTO_MAX_IV_LEN		(16)
> > +#define MAX_COUNT_DOWN_TIMES		(100)
> > +
> > +#define INHDR_LEN		(sizeof(struct virtio_crypto_inhdr))
> > +#define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
> > +				sizeof(struct rte_crypto_sym_op))
> 
> Some of above macros are not used in lib which we shall delete.

Most are used in 4th patch in this patchset. Do you think I should move  them
To it then?

> 
> > +
> > +#ifdef RTE_LIBRTE_VHOST_DEBUG
> > +#define VC_LOG_ERR(fmt, args...)				\
> > +	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
> > +		"Vhost-Crypto",	__func__, __LINE__, ## args)
> > +#define VC_LOG_INFO(fmt, args...)				\
> > +	RTE_LOG(INFO, USER1, "[%s] %s() line %u: " fmt "\n",	\
> > +		"Vhost-Crypto",	__func__, __LINE__, ## args)
> > +
> > +#define VC_LOG_DBG(fmt, args...)				\
> > +	RTE_LOG(DEBUG, USER1, "[%s] %s() line %u: " fmt "\n",	\
> > +		"Vhost-Crypto",	__func__, __LINE__, ## args)
> > +#else
> > +#define VC_LOG_ERR(fmt, args...)				\
> > +	RTE_LOG(ERR, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
> > +#define VC_LOG_INFO(fmt, args...)				\
> > +	RTE_LOG(INFO, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
> #define
> > +VC_LOG_DBG(fmt, args...) #endif
> > +
> > +#define VIRTIO_CRYPTO_FEATURES ((1 << VIRTIO_F_NOTIFY_ON_EMPTY)
> |	\
> > +		(1 << VIRTIO_RING_F_INDIRECT_DESC) |
> 	\
> > +		(1 << VIRTIO_RING_F_EVENT_IDX) |			\
> > +		(1 << VIRTIO_CRYPTO_SERVICE_CIPHER) |
> 	\
> > +		(1 << VIRTIO_CRYPTO_SERVICE_HASH) |
> 	\
> > +		(1 << VIRTIO_CRYPTO_SERVICE_MAC) |
> 	\
> > +		(1 << VIRTIO_CRYPTO_SERVICE_AEAD) |
> 	\
> > +		(1 << VIRTIO_NET_F_CTRL_VQ))
> 
> Just wonder if the above are required features or supported features?

Some are supported, but not all. I will fix it.

> 
> And if it's only used by 5/8 patch, shall we move the definition there?
> Let this patch to focus on "session message handler".

Ok.

> 
> > +
> > +
> > +#define GPA_TO_VVA(t, m, a)
> 	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
> > +
> > +/* Macro to get the buffer at the end of rte_crypto_op */
> > +#define REQ_OP_OFFSET		(IV_OFFSET +
> VHOST_CRYPTO_MAX_IV_LEN)
> 
> Another unused macro?

I will move it to 4/8.

> 
> > +
> > +/**
> > + * 1-to-1 mapping between RTE_CRYPTO_*ALGO* and
> VIRTIO_CRYPTO_*ALGO*,
> > +for
> > + * algorithms not supported by RTE_CRYPTODEV, the
> > +-VIRTIO_CRYPTO_NOTSUPP is
> > + * returned.
> > + */
> > +static int cipher_algo_transform[] = {
> 
> s/transform/map?
> 
> > +		RTE_CRYPTO_CIPHER_NULL,
> > +		RTE_CRYPTO_CIPHER_ARC4,
> > +		RTE_CRYPTO_CIPHER_AES_ECB,
> > +		RTE_CRYPTO_CIPHER_AES_CBC,
> > +		RTE_CRYPTO_CIPHER_AES_CTR,
> > +		-VIRTIO_CRYPTO_NOTSUPP, /*
> VIRTIO_CRYPTO_CIPHER_DES_ECB */
> > +		RTE_CRYPTO_CIPHER_DES_CBC,
> > +		RTE_CRYPTO_CIPHER_3DES_ECB,
> > +		RTE_CRYPTO_CIPHER_3DES_CBC,
> > +		RTE_CRYPTO_CIPHER_3DES_CTR,
> > +		RTE_CRYPTO_CIPHER_KASUMI_F8,
> > +		RTE_CRYPTO_CIPHER_SNOW3G_UEA2,
> > +		RTE_CRYPTO_CIPHER_AES_F8,
> > +		RTE_CRYPTO_CIPHER_AES_XTS,
> > +		RTE_CRYPTO_CIPHER_ZUC_EEA3
> > +};
> 
> How do we check if an input overflows the array?
I will change them to functions.

> 
> > +
> > +/**
> > + * VIRTIO_CRYTPO_AUTH_* indexes are not sequential, the gaps are
> > +filled with
> > + * -VIRTIO_CRYPTO_BADMSG errors.
> > + */
> > +static int auth_algo_transform[] = {
> > +		RTE_CRYPTO_AUTH_NULL,
> > +		RTE_CRYPTO_AUTH_MD5_HMAC,
> > +		RTE_CRYPTO_AUTH_SHA1_HMAC,
> > +		RTE_CRYPTO_AUTH_SHA224_HMAC,
> > +		RTE_CRYPTO_AUTH_SHA256_HMAC,
> > +		RTE_CRYPTO_AUTH_SHA384_HMAC,
> > +		RTE_CRYPTO_AUTH_SHA512_HMAC,
> > +		-VIRTIO_CRYPTO_BADMSG,
> 
> Where is this macro defined?

They are defined in linux/virtio_crypto.h. I will move them to this file.

> 
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_NOTSUPP, /*
> VIRTIO_CRYPTO_MAC_CMAC_3DES */
> > +		RTE_CRYPTO_AUTH_AES_CMAC,
> > +		RTE_CRYPTO_AUTH_KASUMI_F9,
> > +		RTE_CRYPTO_AUTH_SNOW3G_UIA2,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		-VIRTIO_CRYPTO_BADMSG,
> > +		RTE_CRYPTO_AUTH_AES_GMAC,
> > +		-VIRTIO_CRYPTO_NOTSUPP, /*
> VIRTIO_CRYPTO_MAC_GMAC_TWOFISH */
> > +		RTE_CRYPTO_AUTH_AES_CBC_MAC,
> > +		-VIRTIO_CRYPTO_NOTSUPP, /*
> VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9 */
> > +		RTE_CRYPTO_AUTH_AES_XCBC_MAC
> > +};
> 
> Ditto.
> 
> > +
> > +static int cipher_op_transform[] = {
> > +		-VIRTIO_CRYPTO_BADMSG, /* meaningless */
> > +		RTE_CRYPTO_CIPHER_OP_ENCRYPT,
> > +		RTE_CRYPTO_CIPHER_OP_DECRYPT
> > +};
> 
> Ditto.
> 
> > +
> > +static int iv_lens[] = {
> > +		-1, /* Invalid input */
> > +		0, /* RTE_CRYPTO_CIPHER_NULL */
> > +		8, /* RTE_CRYPTO_CIPHER_3DES_CBC */
> > +		8, /* RTE_CRYPTO_CIPHER_3DES_CTR */
> > +		8, /* RTE_CRYPTO_CIPHER_3DES_ECB */
> > +		16, /* RTE_CRYPTO_CIPHER_AES_CBC */
> > +		/* TODO: add common algos */
> > +};
> 
> Ditto.
> 
> > +
> > +/**
> > + * vhost_crypto struct is used to maintain a number of virtio_cryptos
> > +and
> > + * one DPDK crypto device that deals with all crypto workloads. It is
> > +declared
> > + * here and defined in vhost_crypto.c
> 
> The last note is not valid now.
> 
> > + */
> > +struct vhost_crypto {
> > +	/** Used to lookup DPDK Cryptodev Session based on VIRTIO crypto
> > +	 *  session ID.
> > +	 */
> > +	struct rte_hash *session_map;
> > +	struct rte_mempool *mbuf_pool;
> > +	struct rte_mempool *sess_pool;
> > +
> > +	/** DPDK cryptodev ID */
> > +	uint8_t cid;
> > +	uint16_t nb_qps;
> > +
> > +	uint64_t last_session_id;
> > +
> > +	uint64_t cache_session_id;
> > +	struct rte_cryptodev_sym_session *cache_session;
> > +	/** socket id for the device */
> > +	int socket_id;
> > +
> > +	struct virtio_net *dev;
> > +
> > +	uint8_t option;
> > +} __rte_cache_aligned;
> > +
> > +struct vhost_crypto_data_req {
> > +	struct vring_desc *head;
> > +	struct rte_vhost_memory *mem;
> > +	struct virtio_crypto_inhdr *inhdr;
> > +
> > +	uint16_t desc_idx;
> > +	uint32_t len;
> > +	struct vhost_virtqueue *vq;
> > +
> > +	uint8_t zero_copy;
> > +
> > +	int vid;
> > +
> > +	struct vring_desc *wb_desc;
> > +	uint16_t wb_len;
> > +};
> 
> You might want to adjust the sequence of the fields so that we will not have
> some holes.

Ok.

> 
> > +
> > +static int
> > +transform_cipher_param(struct rte_crypto_sym_xform *xform,
> > +		VhostUserCryptoSessionParam *param) {
> > +	int ret;
> > +
> > +	ret = cipher_algo_transform[param->cipher_algo];
> > +	if (unlikely(ret < 0))
> > +		return ret;
> > +
> > +	xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
> > +	xform->cipher.algo = ret;
> > +	xform->cipher.key.length = param->cipher_key_len;
> > +	if (xform->cipher.key.length > 0)
> > +		xform->cipher.key.data = param->cipher_key_buf;
> > +	ret = cipher_op_transform[param->dir];
> > +	if (unlikely(ret < 0)) {
> > +		VC_LOG_DBG("Bad operation type");
> > +		return ret;
> > +	}
> > +	xform->cipher.op = ret;
> > +	ret = iv_lens[xform->cipher.algo];
> > +	if (unlikely(ret < 0))
> > +		return ret;
> > +	xform->cipher.iv.length = ret;
> > +	xform->cipher.iv.offset = IV_OFFSET;
> > +	return 0;
> > +}
> > +
> > +static int
> > +transform_chain_param(struct rte_crypto_sym_xform *xforms,
> > +		VhostUserCryptoSessionParam *param) {
> > +	struct rte_crypto_sym_xform *xform_cipher, *xform_auth;
> > +	int ret;
> > +
> > +	switch (param->chaining_dir) {
> > +	case
> VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER:
> > +		xform_auth = xforms;
> > +		xform_cipher = xforms->next;
> > +		xform_cipher->cipher.op =
> RTE_CRYPTO_CIPHER_OP_DECRYPT;
> > +		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
> > +		break;
> > +	case
> VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH:
> > +		xform_cipher = xforms;
> > +		xform_auth = xforms->next;
> > +		xform_cipher->cipher.op =
> RTE_CRYPTO_CIPHER_OP_ENCRYPT;
> > +		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
> > +		break;
> > +	default:
> > +		return -VIRTIO_CRYPTO_BADMSG;
> > +	}
> > +
> > +	/* cipher */
> > +	ret = cipher_algo_transform[param->cipher_algo];
> > +	if (unlikely(ret < 0))
> > +		return ret;
> > +	xform_cipher->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
> > +	xform_cipher->cipher.algo = ret;
> > +	xform_cipher->cipher.key.length = param->cipher_key_len;
> > +	xform_cipher->cipher.key.data = param->cipher_key_buf;
> > +	ret = iv_lens[xform_cipher->cipher.algo];
> > +	if (unlikely(ret < 0))
> > +		return ret;
> > +	xform_cipher->cipher.iv.length = ret;
> > +	xform_cipher->cipher.iv.offset = IV_OFFSET;
> > +
> > +	/* auth */
> > +	xform_auth->type = RTE_CRYPTO_SYM_XFORM_AUTH;
> > +	ret = auth_algo_transform[param->hash_algo];
> > +	if (unlikely(ret < 0))
> > +		return ret;
> > +	xform_auth->auth.algo = ret;
> > +	xform_auth->auth.digest_length = param->digest_len;
> > +	xform_auth->auth.key.length = param->auth_key_len;
> > +	xform_auth->auth.key.data = param->auth_key_buf;
> > +
> > +	return 0;
> > +}
> > +
> > +static void
> > +vhost_crypto_create_sess(struct vhost_crypto *vcrypto,
> > +		VhostUserCryptoSessionParam *sess_param) {
> > +	struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0};
> > +	struct rte_cryptodev_sym_session *session;
> > +	int ret;
> > +
> > +	switch (sess_param->op_type) {
> > +	case VIRTIO_CRYPTO_SYM_OP_NONE:
> > +	case VIRTIO_CRYPTO_SYM_OP_CIPHER:
> > +		ret = transform_cipher_param(&xform1, sess_param);
> > +		if (unlikely(ret)) {
> > +			VC_LOG_ERR("Error transform session msg (%i)", ret);
> > +			sess_param->session_id = ret;
> > +			return;
> > +		}
> > +		break;
> > +	case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
> > +		if (unlikely(sess_param->hash_mode !=
> > +				VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH))
> {
> > +			sess_param->session_id = -
> VIRTIO_CRYPTO_NOTSUPP;
> > +			VC_LOG_ERR("Error transform session message (%i)",
> > +					-VIRTIO_CRYPTO_NOTSUPP);
> > +			return;
> > +		}
> > +
> > +		xform1.next = &xform2;
> > +
> > +		ret = transform_chain_param(&xform1, sess_param);
> > +		if (unlikely(ret)) {
> > +			VC_LOG_ERR("Error transform session message (%i)",
> ret);
> > +			sess_param->session_id = ret;
> > +			return;
> > +		}
> > +
> > +		break;
> > +	default:
> > +		VC_LOG_ERR("Algorithm not yet supported");
> > +		sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
> > +		return;
> > +	}
> > +
> > +	session = rte_cryptodev_sym_session_create(vcrypto->sess_pool);
> > +	if (!session) {
> > +		VC_LOG_ERR("Failed to create session");
> > +		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
> > +		return;
> > +	}
> > +
> > +	if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1,
> > +			vcrypto->sess_pool) < 0) {
> > +		VC_LOG_ERR("Failed to initialize session");
> > +		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
> > +		return;
> > +	}
> > +
> > +	/* insert hash to map */
> > +	if (rte_hash_add_key_data(vcrypto->session_map,
> > +			&vcrypto->last_session_id, session) < 0) {
> > +		VC_LOG_ERR("Failed to insert session to hash table");
> > +
> > +		if (rte_cryptodev_sym_session_clear(vcrypto->cid, session)
> < 0)
> > +			VC_LOG_ERR("Failed to clear session");
> > +		else {
> > +			if (rte_cryptodev_sym_session_free(session) < 0)
> > +				VC_LOG_ERR("Failed to free session");
> > +		}
> > +		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
> > +		return;
> > +	}
> > +
> > +	VC_LOG_DBG("Session (key %lu, session %p) created.",
> > +			vcrypto->last_session_id, session);
> > +
> > +	sess_param->session_id = vcrypto->last_session_id;
> > +	vcrypto->last_session_id++;
> > +}
> > +
> > +static int
> > +vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t
> > +session_id) {
> > +	struct rte_cryptodev_sym_session *session;
> > +	uint64_t sess_id = session_id;
> > +	int ret;
> > +
> > +	ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id,
> > +			(void **)&session);
> > +
> > +	if (unlikely(ret < 0)) {
> > +		VC_LOG_ERR("Failed to delete session (key %lu).",
> session_id);
> > +		return -VIRTIO_CRYPTO_INVSESS;
> > +	}
> > +
> > +	if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) {
> > +		VC_LOG_DBG("Failed to delete session");
> > +		return -VIRTIO_CRYPTO_ERR;
> > +	}
> > +
> > +	if (rte_cryptodev_sym_session_free(session) < 0) {
> > +		VC_LOG_DBG("Failed to delete session");
> > +		return -VIRTIO_CRYPTO_ERR;
> > +	}
> > +
> > +	if (rte_hash_del_key(vcrypto->session_map, &sess_id) < 0) {
> > +		VC_LOG_DBG("Failed to delete session from hash table.");
> > +		return -VIRTIO_CRYPTO_ERR;
> > +	}
> > +
> > +	VC_LOG_DBG("Session (key %lu, session %p) deleted.", sess_id,
> > +			session);
> > +
> > +	return 0;
> > +}
> > +
> > +static int
> > +vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t
> > +*require_reply) {
> > +	struct virtio_net *dev = get_device(vid);
> > +	struct vhost_crypto *vcrypto;
> > +	VhostUserMsg *vmsg = msg;
> > +	int ret = 0;
> > +
> > +	if (dev == NULL || require_reply == NULL) {
> > +		VC_LOG_ERR("Invalid vid %i", vid);
> > +		return -EINVAL;
> > +	}
> > +
> > +	vcrypto = dev->extern_data;
> > +	if (vcrypto == NULL) {
> > +		VC_LOG_ERR("Cannot find required data, is it initialized?");
> > +		return -ENOENT;
> > +	}
> > +
> > +	*require_reply = 0;
> > +
> > +	if (vmsg->request.master == VHOST_USER_CRYPTO_CREATE_SESS) {
> > +		vhost_crypto_create_sess(vcrypto,
> > +				&vmsg->payload.crypto_session);
> > +		*require_reply = 1;
> > +	} else if (vmsg->request.master ==
> VHOST_USER_CRYPTO_CLOSE_SESS)
> > +		ret = vhost_crypto_close_sess(vcrypto, vmsg->payload.u64);
> > +	else
> > +		ret = -EINVAL;
> > +
> > +	return ret;
> > +}

[PATCH v5 0/8] vhost: introduce vhost crypto backend

[Fan Zhang]

This patchset adds crypto backend suppport to vhost library
including a proof-of-concept sample application. The implementation
follows the virtio-crypto specification and have been tested
with qemu 2.11.50 (with several patches applied, detailed later)
with Fedora 24 running in the frontend.

The vhost_crypto library acts as a "bridge" method that translate
the virtio-crypto crypto requests to DPDK crypto operations, so it
is purely software implementation. However it does require the user
to provide the DPDK Cryptodev ID so it knows how to handle the
virtio-crypto session creation and deletion mesages.

Currently the implementation supports AES-CBC-128 and HMAC-SHA1
cipher only/chaining modes and does not support sessionless mode
yet. The guest can use standard virtio-crypto driver to set up
session and sends encryption/decryption requests to backend. The
vhost-crypto sample application provided in this patchset will
do the actual crypto work.

The following steps are involved to enable vhost-crypto support.

In the host:
1. Download the qemu source code.

2. Recompile your qemu with vhost-crypto option enabled.

3. Apply this patchset to latest DPDK code and recompile DPDK.

4. Compile and run vhost-crypto sample application.

./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
 --socket-mem 2048,2048

Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
not available. The sample application requires 2 lcores: 1 master and 1
worker. The application will create a UNIX socket file
/tmp/vhost_crypto1.socket.

5. Start your qemu application. Here is my command:

qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
-smp 2 -m 1G -hda ~/path-to-your/image.qcow \
-object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
-mem-prealloc -numa node,memdev=mem -chardev \
socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
-object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
-device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0

6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
default. You shall see the following logs in your demsg:

[   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
[   17.611083] virtio_crypto: module verification failed: signature and/or ...
[   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
[   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
[   18.376100] virtio_crypto virtio0: Accelerator is ready

The virtio_crypto driver in the guest is now up and running.

7. The rest steps can be as same as the Testing section in
https://wiki.qemu.org/Features/VirtioCrypto

8. It is possible to use DPDK Virtio Crypto PMD
(https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work with
this patchset to achieve optimal performance.

v5:
- removed external ops register API.
- patch cleaned.

v4:
- Changed external vhost backend ops register API.
- Fixed a bug.

v3:
- Changed external vhost backend private data and message handling
- Added experimental tag to rte_vhost_crypto_set_zero_copy()

v2:
- Moved vhost_crypto_data_req data from crypto op to source mbuf.
- Removed ZERO-COPY flag from config option and make it run-timely changeable.
- Guest-polling mode possible.
- Simplified vring descriptor access procedure.
- Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.


Fan Zhang (8):
  lib/librte_vhost: add external backend support
  lib/librte_vhost: add virtio-crypto user message structure
  lib/librte_vhost: add session message handler
  lib/librte_vhost: add request handler
  lib/librte_vhost: add public function implementation
  lib/librte_vhost: update makefile
  examples/vhost_crypto: add vhost crypto sample application
  doc: update for vhost crypto support

 doc/guides/prog_guide/vhost_lib.rst       |   25 +
 doc/guides/rel_notes/release_18_05.rst    |    5 +
 doc/guides/sample_app_ug/index.rst        |    1 +
 doc/guides/sample_app_ug/vhost_crypto.rst |   83 ++
 examples/vhost_crypto/Makefile            |   32 +
 examples/vhost_crypto/main.c              |  541 ++++++++++++
 examples/vhost_crypto/meson.build         |   14 +
 lib/librte_vhost/Makefile                 |    6 +-
 lib/librte_vhost/meson.build              |    8 +-
 lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
 lib/librte_vhost/rte_vhost_version.map    |   11 +
 lib/librte_vhost/vhost.c                  |    2 +-
 lib/librte_vhost/vhost.h                  |   53 +-
 lib/librte_vhost/vhost_crypto.c           | 1315 +++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c             |   33 +-
 lib/librte_vhost/vhost_user.h             |   35 +-
 16 files changed, 2260 insertions(+), 13 deletions(-)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 examples/vhost_crypto/meson.build
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
 create mode 100644 lib/librte_vhost/vhost_crypto.c

-- 
2.13.6

[PATCH v5 1/8] lib/librte_vhost: add external backend support

[Fan Zhang]

This patch adds external backend support to vhost library. The patch provides
new function prototypes for the external backend to register pre and post
vhost-user message handlers.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost.c      |  2 +-
 lib/librte_vhost/vhost.h      | 53 +++++++++++++++++++++++++++++++++++++++++--
 lib/librte_vhost/vhost_user.c | 31 ++++++++++++++++++++++++-
 3 files changed, 82 insertions(+), 4 deletions(-)

diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
index f6f12a03b..99f8b01fb 100644
--- a/lib/librte_vhost/vhost.c
+++ b/lib/librte_vhost/vhost.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2017 Intel Corporation
  */
 
 #include <linux/vhost.h>
diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
index 4981e6d0c..f28164849 100644
--- a/lib/librte_vhost/vhost.h
+++ b/lib/librte_vhost/vhost.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_CDEV_H_
@@ -211,6 +211,51 @@ struct guest_page {
 };
 
 /**
+ * function prototype for the vhost backend to handler specific vhost user
+ * messages prior to the master message handling
+ *
+ * @param vid
+ *  vhost device id
+ * @param msg
+ *  Message pointer.
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0.
+ * @param skip_master
+ *  If the handler requires skipping the master message handling, this variable
+ *  shall be written 1, otherwise 0.
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*rte_vhost_msg_pre_handle)(int vid, void *msg,
+		uint32_t *require_reply, uint32_t *skip_master);
+
+/**
+ * function prototype for the vhost backend to handler specific vhost user
+ * messages after the master message handling is done
+ *
+ * @param vid
+ *  vhost device id
+ * @param msg
+ *  Message pointer.
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0.
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
+		uint32_t *require_reply);
+
+/**
+ * pre and post vhost user message handlers
+ */
+struct vhost_user_extern_ops {
+	rte_vhost_msg_pre_handle pre_msg_handle;
+	rte_vhost_msg_post_handle post_msg_handle;
+};
+
+/**
  * Device structure contains all configuration information relating
  * to the device.
  */
@@ -242,8 +287,12 @@ struct virtio_net {
 	struct guest_page       *guest_pages;
 
 	int			slave_req_fd;
-} __rte_cache_aligned;
 
+	/* private data for external virtio device */
+	void			*extern_data;
+	/* pre and post vhost user message handlers for externel backend */
+	struct vhost_user_extern_ops extern_ops;
+} __rte_cache_aligned;
 
 #define VHOST_LOG_PAGE	4096
 
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 66e1b82a5..a435cce17 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 /* Security model
@@ -1379,6 +1379,7 @@ vhost_user_msg_handler(int vid, int fd)
 	struct VhostUserMsg msg;
 	int ret;
 	int unlock_required = 0;
+	uint32_t skip_master = 0;
 
 	dev = get_device(vid);
 	if (dev == NULL)
@@ -1456,6 +1457,21 @@ vhost_user_msg_handler(int vid, int fd)
 
 	}
 
+	if (dev->extern_ops.pre_msg_handle) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.pre_msg_handle)(dev->vid,
+				(void *)&msg, &need_reply, &skip_master);
+		if (ret < 0)
+			goto skip_to_reply;
+
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
+
+		if (skip_master)
+			goto skip_to_post_handle;
+	}
+
 	switch (msg.request.master) {
 	case VHOST_USER_GET_FEATURES:
 		msg.payload.u64 = vhost_user_get_features(dev);
@@ -1556,9 +1572,22 @@ vhost_user_msg_handler(int vid, int fd)
 	default:
 		ret = -1;
 		break;
+	}
+
+skip_to_post_handle:
+	if (dev->extern_ops.post_msg_handle) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.post_msg_handle)(
+				dev->vid, (void *)&msg, &need_reply);
+		if (ret < 0)
+			goto skip_to_reply;
 
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
 	}
 
+skip_to_reply:
 	if (unlock_required)
 		vhost_user_unlock_all_queue_pairs(dev);
 
-- 
2.13.6

[PATCH v5 2/8] lib/librte_vhost: add virtio-crypto user message structure

[Fan Zhang]

This patch adds virtio-crypto spec user message structure to
vhost_user.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_user.h | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h
index 0fafbe6e0..33d64fc82 100644
--- a/lib/librte_vhost/vhost_user.h
+++ b/lib/librte_vhost/vhost_user.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_USER_H
@@ -20,13 +20,15 @@
 #define VHOST_USER_PROTOCOL_F_REPLY_ACK	3
 #define VHOST_USER_PROTOCOL_F_NET_MTU 4
 #define VHOST_USER_PROTOCOL_F_SLAVE_REQ 5
+#define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7
 
 #define VHOST_USER_PROTOCOL_FEATURES	((1ULL << VHOST_USER_PROTOCOL_F_MQ) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) |\
 					 (1ULL << VHOST_USER_PROTOCOL_F_RARP) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_NET_MTU) | \
-					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ))
+					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ) | \
+					 (1ULL << VHOST_USER_PROTOCOL_F_CRYPTO_SESSION))
 
 typedef enum VhostUserRequest {
 	VHOST_USER_NONE = 0,
@@ -52,7 +54,9 @@ typedef enum VhostUserRequest {
 	VHOST_USER_NET_SET_MTU = 20,
 	VHOST_USER_SET_SLAVE_REQ_FD = 21,
 	VHOST_USER_IOTLB_MSG = 22,
-	VHOST_USER_MAX
+	VHOST_USER_CRYPTO_CREATE_SESS = 26,
+	VHOST_USER_CRYPTO_CLOSE_SESS = 27,
+	VHOST_USER_MAX = 28
 } VhostUserRequest;
 
 typedef enum VhostUserSlaveRequest {
@@ -79,6 +83,30 @@ typedef struct VhostUserLog {
 	uint64_t mmap_offset;
 } VhostUserLog;
 
+/* Comply with Cryptodev-Linux */
+#define VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH	512
+#define VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH	64
+
+/* Same structure as vhost-user backend session info */
+typedef struct VhostUserCryptoSessionParam {
+	int64_t session_id;
+	uint32_t op_code;
+	uint32_t cipher_algo;
+	uint32_t cipher_key_len;
+	uint32_t hash_algo;
+	uint32_t digest_len;
+	uint32_t auth_key_len;
+	uint32_t aad_len;
+	uint8_t op_type;
+	uint8_t dir;
+	uint8_t hash_mode;
+	uint8_t chaining_dir;
+	uint8_t *ciphe_key;
+	uint8_t *auth_key;
+	uint8_t cipher_key_buf[VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH];
+	uint8_t auth_key_buf[VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH];
+} VhostUserCryptoSessionParam;
+
 typedef struct VhostUserMsg {
 	union {
 		uint32_t master; /* a VhostUserRequest value */
@@ -99,6 +127,7 @@ typedef struct VhostUserMsg {
 		VhostUserMemory memory;
 		VhostUserLog    log;
 		struct vhost_iotlb_msg iotlb;
+		VhostUserCryptoSessionParam crypto_session;
 	} payload;
 	int fds[VHOST_MEMORY_MAX_NREGIONS];
 } __attribute((packed)) VhostUserMsg;
-- 
2.13.6

[PATCH v5 3/8] lib/librte_vhost: add session message handler

[Fan Zhang]

This patch adds session message handler to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 435 ++++++++++++++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c   |   2 +
 2 files changed, 437 insertions(+)
 create mode 100644 lib/librte_vhost/vhost_crypto.c

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
new file mode 100644
index 000000000..34587c27e
--- /dev/null
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -0,0 +1,435 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+#include <linux/virtio_crypto.h>
+
+#include <rte_malloc.h>
+#include <rte_hash.h>
+#include <rte_mbuf.h>
+#include <rte_cryptodev.h>
+
+#include "vhost.h"
+#include "vhost_user.h"
+
+#define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
+				sizeof(struct rte_crypto_sym_op))
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+
+#define VC_LOG_DBG(fmt, args...)				\
+	RTE_LOG(DEBUG, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#else
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_DBG(fmt, args...)
+#endif
+
+static int
+cipher_algo_transform(uint32_t virtio_cipher_algo)
+{
+	int ret;
+
+	switch (virtio_cipher_algo) {
+	case VIRTIO_CRYPTO_CIPHER_AES_CBC:
+		ret = RTE_CRYPTO_CIPHER_AES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_CTR:
+		ret = RTE_CRYPTO_CIPHER_AES_CTR;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_DES_ECB:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_DES_CBC:
+		ret = RTE_CRYPTO_CIPHER_DES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_ECB:
+		ret = RTE_CRYPTO_CIPHER_3DES_ECB;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_CBC:
+		ret = RTE_CRYPTO_CIPHER_3DES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_CTR:
+		ret = RTE_CRYPTO_CIPHER_3DES_CTR;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_KASUMI_F8:
+		ret = RTE_CRYPTO_CIPHER_KASUMI_F8;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_SNOW3G_UEA2:
+		ret = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_F8:
+		ret = RTE_CRYPTO_CIPHER_AES_F8;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_XTS:
+		ret = RTE_CRYPTO_CIPHER_AES_XTS;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_ZUC_EEA3:
+		ret = RTE_CRYPTO_CIPHER_ZUC_EEA3;
+		break;
+	default:
+		ret = -VIRTIO_CRYPTO_BADMSG;
+		break;
+	}
+
+	return ret;
+}
+
+static int
+auth_algo_transform(uint32_t virtio_auth_algo)
+{
+	int ret;
+
+	switch (virtio_auth_algo) {
+
+	case VIRTIO_CRYPTO_NO_MAC:
+		ret = RTE_CRYPTO_AUTH_NULL;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_MD5:
+		ret = RTE_CRYPTO_AUTH_MD5_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA1:
+		ret = RTE_CRYPTO_AUTH_SHA1_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_224:
+		ret = RTE_CRYPTO_AUTH_SHA224_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_256:
+		ret = RTE_CRYPTO_AUTH_SHA256_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_384:
+		ret = RTE_CRYPTO_AUTH_SHA384_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_512:
+		ret = RTE_CRYPTO_AUTH_SHA512_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_CMAC_3DES:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_CMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_CMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_KASUMI_F9:
+		ret = RTE_CRYPTO_AUTH_KASUMI_F9;
+		break;
+	case VIRTIO_CRYPTO_MAC_SNOW3G_UIA2:
+		ret = RTE_CRYPTO_AUTH_SNOW3G_UIA2;
+		break;
+	case VIRTIO_CRYPTO_MAC_GMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_GMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_GMAC_TWOFISH:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_CBCMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_CBC_MAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_XCBC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_XCBC_MAC;
+		break;
+	default:
+		ret = -VIRTIO_CRYPTO_BADMSG;
+		break;
+	}
+
+	return ret;
+}
+
+static int get_iv_len(enum rte_crypto_cipher_algorithm algo)
+{
+	int len;
+
+	switch (algo) {
+	case RTE_CRYPTO_CIPHER_3DES_CBC:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_3DES_CTR:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_3DES_ECB:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_AES_CBC:
+		len = 16;
+		break;
+
+	/* TODO: add common algos */
+
+	default:
+		len = -1;
+		break;
+	}
+
+	return len;
+}
+
+/**
+ * vhost_crypto struct is used to maintain a number of virtio_cryptos and
+ * one DPDK crypto device that deals with all crypto workloads. It is declared
+ * here and defined in vhost_crypto.c
+ */
+struct vhost_crypto {
+	/** Used to lookup DPDK Cryptodev Session based on VIRTIO crypto
+	 *  session ID.
+	 */
+	struct rte_hash *session_map;
+	struct rte_mempool *mbuf_pool;
+	struct rte_mempool *sess_pool;
+
+	/** DPDK cryptodev ID */
+	uint8_t cid;
+	uint16_t nb_qps;
+
+	uint64_t last_session_id;
+
+	uint64_t cache_session_id;
+	struct rte_cryptodev_sym_session *cache_session;
+	/** socket id for the device */
+	int socket_id;
+
+	struct virtio_net *dev;
+
+	uint8_t option;
+} __rte_cache_aligned;
+
+static int
+transform_cipher_param(struct rte_crypto_sym_xform *xform,
+		VhostUserCryptoSessionParam *param)
+{
+	int ret;
+
+	ret = cipher_algo_transform(param->cipher_algo);
+	if (unlikely(ret < 0))
+		return ret;
+
+	xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform->cipher.algo = (uint32_t)ret;
+	xform->cipher.key.length = param->cipher_key_len;
+	if (xform->cipher.key.length > 0)
+		xform->cipher.key.data = param->cipher_key_buf;
+	if (param->dir == VIRTIO_CRYPTO_OP_ENCRYPT)
+		xform->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+	else if (param->dir == VIRTIO_CRYPTO_OP_DECRYPT)
+		xform->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+	else {
+		VC_LOG_DBG("Bad operation type");
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	ret = get_iv_len(xform->cipher.algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform->cipher.iv.length = (uint16_t)ret;
+	xform->cipher.iv.offset = IV_OFFSET;
+	return 0;
+}
+
+static int
+transform_chain_param(struct rte_crypto_sym_xform *xforms,
+		VhostUserCryptoSessionParam *param)
+{
+	struct rte_crypto_sym_xform *xform_cipher, *xform_auth;
+	int ret;
+
+	switch (param->chaining_dir) {
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER:
+		xform_auth = xforms;
+		xform_cipher = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+		break;
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH:
+		xform_cipher = xforms;
+		xform_auth = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		break;
+	default:
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	/* cipher */
+	ret = cipher_algo_transform(param->cipher_algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform_cipher->cipher.algo = (uint32_t)ret;
+	xform_cipher->cipher.key.length = param->cipher_key_len;
+	xform_cipher->cipher.key.data = param->cipher_key_buf;
+	ret = get_iv_len(xform_cipher->cipher.algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->cipher.iv.length = (uint16_t)ret;
+	xform_cipher->cipher.iv.offset = IV_OFFSET;
+
+	/* auth */
+	xform_auth->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+	ret = auth_algo_transform(param->hash_algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_auth->auth.algo = (uint32_t)ret;
+	xform_auth->auth.digest_length = param->digest_len;
+	xform_auth->auth.key.length = param->auth_key_len;
+	xform_auth->auth.key.data = param->auth_key_buf;
+
+	return 0;
+}
+
+static void
+vhost_crypto_create_sess(struct vhost_crypto *vcrypto,
+		VhostUserCryptoSessionParam *sess_param)
+{
+	struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0};
+	struct rte_cryptodev_sym_session *session;
+	int ret;
+
+	switch (sess_param->op_type) {
+	case VIRTIO_CRYPTO_SYM_OP_NONE:
+	case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+		ret = transform_cipher_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session msg (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+		break;
+	case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+		if (unlikely(sess_param->hash_mode !=
+				VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH)) {
+			sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+			VC_LOG_ERR("Error transform session message (%i)",
+					-VIRTIO_CRYPTO_NOTSUPP);
+			return;
+		}
+
+		xform1.next = &xform2;
+
+		ret = transform_chain_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session message (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+
+		break;
+	default:
+		VC_LOG_ERR("Algorithm not yet supported");
+		sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+		return;
+	}
+
+	session = rte_cryptodev_sym_session_create(vcrypto->sess_pool);
+	if (!session) {
+		VC_LOG_ERR("Failed to create session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1,
+			vcrypto->sess_pool) < 0) {
+		VC_LOG_ERR("Failed to initialize session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	/* insert hash to map */
+	if (rte_hash_add_key_data(vcrypto->session_map,
+			&vcrypto->last_session_id, session) < 0) {
+		VC_LOG_ERR("Failed to insert session to hash table");
+
+		if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0)
+			VC_LOG_ERR("Failed to clear session");
+		else {
+			if (rte_cryptodev_sym_session_free(session) < 0)
+				VC_LOG_ERR("Failed to free session");
+		}
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) created.",
+			vcrypto->last_session_id, session);
+
+	sess_param->session_id = vcrypto->last_session_id;
+	vcrypto->last_session_id++;
+}
+
+static int
+vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id)
+{
+	struct rte_cryptodev_sym_session *session;
+	uint64_t sess_id = session_id;
+	int ret;
+
+	ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id,
+			(void **)&session);
+
+	if (unlikely(ret < 0)) {
+		VC_LOG_ERR("Failed to delete session (key %lu).", session_id);
+		return -VIRTIO_CRYPTO_INVSESS;
+	}
+
+	if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_cryptodev_sym_session_free(session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_hash_del_key(vcrypto->session_map, &sess_id) < 0) {
+		VC_LOG_DBG("Failed to delete session from hash table.");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) deleted.", sess_id,
+			session);
+
+	return 0;
+}
+
+static int
+vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+	VhostUserMsg *vmsg = msg;
+	int ret = 0;
+
+	if (dev == NULL || require_reply == NULL) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (vcrypto == NULL) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	*require_reply = 0;
+
+	if (vmsg->request.master == VHOST_USER_CRYPTO_CREATE_SESS) {
+		vhost_crypto_create_sess(vcrypto,
+				&vmsg->payload.crypto_session);
+		*require_reply = 1;
+	} else if (vmsg->request.master == VHOST_USER_CRYPTO_CLOSE_SESS)
+		ret = vhost_crypto_close_sess(vcrypto, vmsg->payload.u64);
+	else
+		ret = -EINVAL;
+
+	return ret;
+}
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index a435cce17..3117c6330 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -67,6 +67,8 @@ static const char *vhost_message_str[VHOST_USER_MAX] = {
 	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
 	[VHOST_USER_SET_SLAVE_REQ_FD]  = "VHOST_USER_SET_SLAVE_REQ_FD",
 	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
+	[VHOST_USER_CRYPTO_CREATE_SESS] = "VHOST_USER_CRYPTO_CREATE_SESS",
+	[VHOST_USER_CRYPTO_CLOSE_SESS] = "VHOST_USER_CRYPTO_CLOSE_SESS",
 };
 
 static uint64_t
-- 
2.13.6

[PATCH v5 4/8] lib/librte_vhost: add request handler

[Fan Zhang]

This patch adds the implementation that parses virtio crypto request
to dpdk crypto operation.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h |  14 +
 lib/librte_vhost/vhost_crypto.c     | 625 ++++++++++++++++++++++++++++++++++++
 2 files changed, 639 insertions(+)
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
new file mode 100644
index 000000000..b6be9d87d
--- /dev/null
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#ifndef _VHOST_CRYPTO_H_
+#define _VHOST_CRYPTO_H_
+
+enum rte_vhost_crypto_zero_copy {
+	RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE = 0,
+	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE,
+	RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS
+};
+
+#endif /**< _VHOST_CRYPTO_H_ */
diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index 34587c27e..67f1189ec 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -8,9 +8,11 @@
 #include <rte_mbuf.h>
 #include <rte_cryptodev.h>
 
+#include "rte_vhost_crypto.h"
 #include "vhost.h"
 #include "vhost_user.h"
 
+#define INHDR_LEN		(sizeof(struct virtio_crypto_inhdr))
 #define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
 				sizeof(struct rte_crypto_sym_op))
 
@@ -33,6 +35,8 @@
 #define VC_LOG_DBG(fmt, args...)
 #endif
 
+#define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
+
 static int
 cipher_algo_transform(uint32_t virtio_cipher_algo)
 {
@@ -203,6 +207,18 @@ struct vhost_crypto {
 	uint8_t option;
 } __rte_cache_aligned;
 
+struct vhost_crypto_data_req {
+	struct vring_desc *head;
+	struct rte_vhost_memory *mem;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vhost_virtqueue *vq;
+	struct vring_desc *wb_desc;
+	uint16_t wb_len;
+	uint16_t desc_idx;
+	uint16_t len;
+	uint16_t zero_copy;
+};
+
 static int
 transform_cipher_param(struct rte_crypto_sym_xform *xform,
 		VhostUserCryptoSessionParam *param)
@@ -433,3 +449,612 @@ vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
 
 	return ret;
 }
+
+static __rte_always_inline struct vring_desc *
+find_write_desc(struct vring_desc *head, struct vring_desc *desc)
+{
+	if (desc->flags & VRING_DESC_F_WRITE)
+		return desc;
+
+	while (desc->flags & VRING_DESC_F_NEXT) {
+		desc = &head[desc->next];
+		if (desc->flags & VRING_DESC_F_WRITE)
+			return desc;
+	}
+
+	return NULL;
+}
+
+static struct virtio_crypto_inhdr *
+reach_inhdr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc *desc)
+{
+	while (desc->flags & VRING_DESC_F_NEXT)
+		desc = &head[desc->next];
+
+	return GPA_TO_VVA(struct virtio_crypto_inhdr *, mem, desc->addr);
+}
+
+static __rte_always_inline int
+move_desc(struct vring_desc *head, struct vring_desc **cur_desc,
+		uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	left -= desc->len;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		left -= desc->len;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+	return 0;
+}
+
+static int
+copy_data(void *dst_data, struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	uint32_t to_copy;
+	uint8_t *data = dst_data;
+	uint8_t *src;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	to_copy = RTE_MIN(desc->len, (uint32_t)left);
+	src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy((uint8_t *)data, src, to_copy);
+	left -= to_copy;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_copy = RTE_MIN(desc->len, (uint32_t)left);
+		src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(data + size - left, src, to_copy);
+		left -= to_copy;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+
+	return 0;
+}
+
+static __rte_always_inline void *
+get_data_ptr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	void *data;
+
+	data = GPA_TO_VVA(void *, mem, (*cur_desc)->addr);
+	if (unlikely(!data)) {
+		VC_LOG_ERR("Failed to get object");
+		return NULL;
+	}
+
+	if (unlikely(move_desc(head, cur_desc, size) < 0))
+		return NULL;
+
+	return data;
+}
+
+static int
+write_back_data(struct rte_crypto_op *op, struct vhost_crypto_data_req *vc_req)
+{
+	struct rte_mbuf *mbuf = op->sym->m_dst;
+	struct vring_desc *head = vc_req->head;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct vring_desc *desc = vc_req->wb_desc;
+	int left = vc_req->wb_len;
+	uint32_t to_write;
+	uint8_t *src_data = mbuf->buf_addr, *dst;
+
+	rte_prefetch0(&head[desc->next]);
+	to_write = RTE_MIN(desc->len, (uint32_t)left);
+	dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy(dst, src_data, to_write);
+	left -= to_write;
+	src_data += to_write;
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	printf("desc addr %llu len %u:", desc->addr, desc->len);
+	rte_hexdump(stdout, "", dst, to_write);
+#endif
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_write = RTE_MIN(desc->len, (uint32_t)left);
+		dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(dst, src_data, to_write);
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+		printf("desc addr %llu len %u:", desc->addr, desc->len);
+		rte_hexdump(stdout, "DST:", dst, to_write);
+#endif
+		left -= to_write;
+		src_data += to_write;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	return 0;
+}
+
+static uint8_t
+prepare_sym_cipher_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_cipher_data_req *cipher,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			cipher->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, cipher->para.iv_len);
+#endif
+
+	m_src->data_len = cipher->para.src_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				cipher->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 ||
+				m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		if (unlikely(cipher->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, cipher->para.src_data_len))
+				< 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			cipher->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, cipher->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		m_dst->data_len = cipher->para.dst_data_len;
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = cipher->para.dst_data_len;
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+	/* src data */
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = 0;
+	op->sym->cipher.data.length = cipher->para.src_data_len;
+
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+	vc_req->len = cipher->para.dst_data_len + INHDR_LEN;
+
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+static uint8_t
+prepare_sym_chain_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_alg_chain_data_req *chain,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint32_t digest_offset;
+	void *digest_addr;
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			chain->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, chain->para.iv_len);
+#endif
+	m_src->data_len = chain->para.src_data_len;
+	m_dst->data_len = chain->para.dst_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				chain->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 || m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		if (unlikely(chain->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, chain->para.src_data_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			chain->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.phys_addr = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.hash_result_len);
+		op->sym->auth.digest.data = get_data_ptr(head, mem, &desc,
+				chain->para.hash_result_len);
+		if (unlikely(op->sym->auth.digest.phys_addr == 0)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		digest_offset = m_dst->data_len;
+		digest_addr = rte_pktmbuf_mtod_offset(m_dst, void *,
+				digest_offset);
+
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = m_dst->data_len + chain->para.hash_result_len;
+
+		if (unlikely(move_desc(head, &desc,
+				chain->para.dst_data_len) < 0)) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		if (unlikely(copy_data(digest_addr, head, mem, &desc,
+				chain->para.hash_result_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.data = digest_addr;
+		op->sym->auth.digest.phys_addr = rte_pktmbuf_iova_offset(m_dst,
+				digest_offset);
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "Digest:", op->sym->auth.digest.data,
+			chain->para.hash_result_len);
+#endif
+
+	/* record inhdr */
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = chain->para.cipher_start_src_offset;
+	op->sym->cipher.data.length = chain->para.src_data_len -
+			chain->para.cipher_start_src_offset;
+
+	op->sym->auth.data.offset = chain->para.hash_start_src_offset;
+	op->sym->auth.data.length = chain->para.len_to_hash;
+
+	vc_req->len = chain->para.dst_data_len + chain->para.hash_result_len +
+			INHDR_LEN;
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+/**
+ * Process on descriptor
+ */
+static __rte_always_inline int
+vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
+		struct vhost_virtqueue *vq, struct rte_crypto_op *op,
+		struct vring_desc *head, uint16_t desc_idx,
+		struct rte_vhost_memory *mem)
+{
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(op->sym->m_src,
+			sizeof(struct rte_mbuf));
+	struct rte_cryptodev_sym_session *session;
+	struct virtio_crypto_op_data_req *req;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vring_desc *desc = NULL;
+	uint64_t session_id;
+	int err = 0;
+
+	vc_req->desc_idx = desc_idx;
+
+	if (likely(head->flags & VRING_DESC_F_INDIRECT)) {
+		head = GPA_TO_VVA(struct vring_desc *, mem, head->addr);
+		if (unlikely(!head))
+			return 0;
+		desc_idx = 0;
+	}
+
+	desc = head;
+
+	vc_req->mem = mem;
+	vc_req->head = head;
+	vc_req->vq = vq;
+
+	vc_req->zero_copy = vcrypto->option;
+
+	req = get_data_ptr(head, mem, &desc, sizeof(*req));
+	if (unlikely(req == NULL)) {
+		err = VIRTIO_CRYPTO_ERR;
+		VC_LOG_ERR("Invalid descriptor");
+		goto error_exit;
+	}
+
+	switch (req->header.opcode) {
+	case VIRTIO_CRYPTO_CIPHER_ENCRYPT:
+	case VIRTIO_CRYPTO_CIPHER_DECRYPT:
+		session_id = req->header.session_id;
+
+		/* one branch to avoid unnecessary table lookup */
+		if (vcrypto->cache_session_id != session_id) {
+			err = rte_hash_lookup_data(vcrypto->session_map,
+					&session_id, (void **)&session);
+			if (unlikely(err < 0)) {
+				err = VIRTIO_CRYPTO_ERR;
+				VC_LOG_ERR("Failed to retrieve session id %lu",
+						session_id);
+				goto error_exit;
+			}
+
+			vcrypto->cache_session = session;
+			vcrypto->cache_session_id = session_id;
+		}
+
+		session = vcrypto->cache_session;
+
+		err = rte_crypto_op_attach_sym_session(op, session);
+		if (unlikely(err < 0)) {
+			err = VIRTIO_CRYPTO_ERR;
+			VC_LOG_ERR("Failed to attach session to op");
+			goto error_exit;
+		}
+
+		switch (req->u.sym_req.op_type) {
+		case VIRTIO_CRYPTO_SYM_OP_NONE:
+			err = VIRTIO_CRYPTO_NOTSUPP;
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+			err = prepare_sym_cipher_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.cipher, desc);
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+			err = prepare_sym_chain_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.chain, desc);
+			break;
+		}
+		if (unlikely(err != 0)) {
+			VC_LOG_ERR("Failed to process sym request");
+			goto error_exit;
+		}
+		break;
+	default:
+		VC_LOG_ERR("Unsupported symmetric crypto request type %u",
+				req->header.opcode);
+		goto error_exit;
+	}
+
+	return 0;
+
+error_exit:
+
+	inhdr = reach_inhdr(head, mem, desc);
+	if (likely(inhdr != NULL))
+		inhdr->status = (uint8_t)err;
+
+	return -1;
+}
+
+static __rte_always_inline struct vhost_virtqueue *
+vhost_crypto_finalize_one_request(struct rte_crypto_op *op,
+		struct vhost_virtqueue *old_vq)
+{
+	struct rte_mbuf *m_src = op->sym->m_src;
+	struct rte_mbuf *m_dst = op->sym->m_dst;
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(m_src,
+			sizeof(struct rte_mbuf));
+	uint16_t desc_idx;
+	int ret = 0;
+
+	if (unlikely(!vc_req)) {
+		VC_LOG_ERR("Failed to retrieve vc_req");
+		return NULL;
+	}
+
+	if (old_vq && (vc_req->vq != old_vq))
+		return vc_req->vq;
+
+	desc_idx = vc_req->desc_idx;
+
+	if (unlikely(op->status != RTE_CRYPTO_OP_STATUS_SUCCESS))
+		vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+	else {
+		if (vc_req->zero_copy == 0) {
+			ret = write_back_data(op, vc_req);
+			if (unlikely(ret != 0))
+				vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+		}
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "DST:", rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *),
+			m_dst->data_len);
+#endif
+
+	vc_req->vq->used->ring[desc_idx].id = desc_idx;
+	vc_req->vq->used->ring[desc_idx].len = vc_req->len;
+
+	rte_mempool_put(m_dst->pool, (void *)m_dst);
+	rte_mempool_put(m_src->pool, (void *)m_src);
+
+	return vc_req->vq;
+}
+
+static __rte_always_inline uint16_t
+vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfd)
+{
+	uint16_t processed = 1;
+	struct vhost_virtqueue *vq, *tmp_vq;
+
+	if (unlikely(nb_ops == 0))
+		return 0;
+
+	vq = vhost_crypto_finalize_one_request(ops[0], NULL);
+	if (unlikely(vq == NULL))
+		return 0;
+	tmp_vq = vq;
+
+	while ((processed < nb_ops)) {
+		tmp_vq = vhost_crypto_finalize_one_request(ops[processed],
+				tmp_vq);
+
+		if (unlikely(vq != tmp_vq))
+			break;
+
+		processed++;
+	}
+
+	*callfd = vq->callfd;
+
+	*(volatile uint16_t *)&vq->used->idx += processed;
+
+	return processed;
+}
-- 
2.13.6

[PATCH v5 5/8] lib/librte_vhost: add public function implementation

[Fan Zhang]

This patch adds public API implementation to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h    |  97 ++++++++++++-
 lib/librte_vhost/rte_vhost_version.map |  11 ++
 lib/librte_vhost/vhost_crypto.c        | 255 +++++++++++++++++++++++++++++++++
 3 files changed, 362 insertions(+), 1 deletion(-)

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
index b6be9d87d..acdc7f0fe 100644
--- a/lib/librte_vhost/rte_vhost_crypto.h
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -5,10 +5,105 @@
 #ifndef _VHOST_CRYPTO_H_
 #define _VHOST_CRYPTO_H_
 
+#define VHOST_CRYPTO_MBUF_POOL_SIZE		(8192)
+#define VHOST_CRYPTO_MAX_BURST_SIZE		(64)
+#define VHOST_CRYPTO_SESSION_MAP_ENTRIES	(1024) /**< Max nb sessions */
+/** max nb virtual queues in a burst for finalizing*/
+#define VIRTIO_CRYPTO_MAX_NUM_BURST_VQS		(64)
+
 enum rte_vhost_crypto_zero_copy {
 	RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE = 0,
-	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE,
+	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE = 1,
 	RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS
 };
 
+/**
+ *  Create Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param cryptodev_id
+ *  The identifier of DPDK Cryptodev, the same cryptodev_id can be assigned to
+ *  multiple Vhost-crypto devices.
+ * @param sess_pool
+ *  The pointer to the created cryptodev session pool with the private data size
+ *  matches the target DPDK Cryptodev.
+ * @param socket_id
+ *  NUMA Socket ID to allocate resources on. *
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise
+ */
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id);
+
+/**
+ *  Free the Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise.
+ */
+int
+rte_vhost_crypto_free(int vid);
+
+/**
+ *  Enable or disable zero copy feature
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param option
+ *  Flag of zero copy feature.
+ * @return
+ *  0 if completed successfully.
+ *  Negative integer if otherwise.
+ */
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option);
+
+/**
+ * Fetch a number of vring descriptors from virt-queue and translate to DPDK
+ * crypto operations. After this function is executed, the user can enqueue
+ * the processed ops to the target cryptodev.
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param qid
+ *  Virtio queue index.
+ * @param ops
+ *  The address of an array of pointers to *rte_crypto_op* structures that must
+ *  be large enough to store *nb_ops* pointers in it.
+ * @param nb_ops
+ *  The maximum number of operations to be fetched and translated.
+ * @return
+ *  The number of fetched and processed vhost crypto request operations.
+ */
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops);
+/**
+ * Finalize the dequeued crypto ops. After the translated crypto ops are
+ * dequeued from the cryptodev, this function shall be called to write the
+ * processed data back to the vring descriptor (if no-copy is turned off).
+ *
+ * @param ops
+ *  The address of an array of *rte_crypto_op* structure that was dequeued
+ *  from cryptodev.
+ * @param nb_ops
+ *  The number of operations contained in the array.
+ * @callfds
+ *  The callfd number(s) contained in this burst, this shall be an array with
+ *  no less than VIRTIO_CRYPTO_MAX_NUM_BURST_VQS elements.
+ * @nb_callfds
+ *  The number of call_fd numbers exist in the callfds.
+ * @return
+ *  The number of ops processed.
+ */
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds);
+
 #endif /**< _VHOST_CRYPTO_H_ */
diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map
index df0103129..65807488a 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -59,3 +59,14 @@ DPDK_18.02 {
 	rte_vhost_vring_call;
 
 } DPDK_17.08;
+
+DPDK_18.05 {
+	global:
+
+	rte_vhost_crypto_create;
+	rte_vhost_crypto_free;
+	rte_vhost_crypto_fetch_requests;
+	rte_vhost_crypto_finalize_requests;
+	rte_vhost_crypto_set_zero_copy;
+
+} DPDK_18.02;
diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index 67f1189ec..aac2eb1df 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -5,6 +5,7 @@
 
 #include <rte_malloc.h>
 #include <rte_hash.h>
+#include <rte_jhash.h>
 #include <rte_mbuf.h>
 #include <rte_cryptodev.h>
 
@@ -35,6 +36,13 @@
 #define VC_LOG_DBG(fmt, args...)
 #endif
 
+#define VIRTIO_CRYPTO_FEATURES ((1 << VIRTIO_F_NOTIFY_ON_EMPTY) |	\
+		(1 << VIRTIO_RING_F_INDIRECT_DESC) |			\
+		(1 << VIRTIO_RING_F_EVENT_IDX) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_CIPHER) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_MAC) |			\
+		(1 << VIRTIO_NET_F_CTRL_VQ))
+
 #define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
 
 static int
@@ -1058,3 +1066,250 @@ vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
 
 	return processed;
 }
+
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct rte_hash_parameters params = {0};
+	struct vhost_crypto *vcrypto;
+	char name[128];
+	int ret;
+
+	if (!dev) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	ret = rte_vhost_driver_set_features(dev->ifname,
+			VIRTIO_CRYPTO_FEATURES);
+	if (ret < 0) {
+		VC_LOG_ERR("Error setting features");
+		return -1;
+	}
+
+	vcrypto = rte_zmalloc_socket(NULL, sizeof(*vcrypto),
+			RTE_CACHE_LINE_SIZE, socket_id);
+	if (!vcrypto) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	vcrypto->sess_pool = sess_pool;
+	vcrypto->cid = cryptodev_id;
+	vcrypto->cache_session_id = UINT64_MAX;
+	vcrypto->last_session_id = 1;
+	vcrypto->dev = dev;
+	vcrypto->option = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	snprintf(name, 127, "HASH_VHOST_CRYPT_%u", (uint32_t)vid);
+	params.name = name;
+	params.entries = VHOST_CRYPTO_SESSION_MAP_ENTRIES;
+	params.hash_func = rte_jhash;
+	params.key_len = sizeof(uint64_t);
+	params.socket_id = socket_id;
+	vcrypto->session_map = rte_hash_create(&params);
+	if (!vcrypto->session_map) {
+		VC_LOG_ERR("Failed to creath session map");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "MBUF_POOL_VM_%u", (uint32_t)vid);
+	vcrypto->mbuf_pool = rte_pktmbuf_pool_create(name,
+			VHOST_CRYPTO_MBUF_POOL_SIZE, 512,
+			sizeof(struct vhost_crypto_data_req),
+			RTE_MBUF_DEFAULT_DATAROOM * 2 + RTE_PKTMBUF_HEADROOM,
+			rte_socket_id());
+	if (!vcrypto->mbuf_pool) {
+		VC_LOG_ERR("Failed to creath mbuf pool");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	dev->extern_data = vcrypto;
+	dev->extern_ops.pre_msg_handle = NULL;
+	dev->extern_ops.post_msg_handle = vhost_crypto_msg_post_handler;
+
+	return 0;
+
+error_exit:
+	if (vcrypto->session_map)
+		rte_hash_free(vcrypto->session_map);
+	if (vcrypto->mbuf_pool)
+		rte_mempool_free(vcrypto->mbuf_pool);
+
+	rte_free(vcrypto);
+
+	return ret;
+}
+
+int
+rte_vhost_crypto_free(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	rte_hash_free(vcrypto->session_map);
+	rte_mempool_free(vcrypto->mbuf_pool);
+	rte_free(vcrypto);
+
+	dev->extern_data = NULL;
+	dev->extern_ops.pre_msg_handle = NULL;
+	dev->extern_ops.post_msg_handle = NULL;
+
+	return 0;
+}
+
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	if (unlikely(option < 0 || option >=
+			RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS)) {
+		VC_LOG_ERR("Invalid option %i", option);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	if (vcrypto->option == (uint8_t)option)
+		return 0;
+
+	if (!(rte_mempool_full(vcrypto->mbuf_pool))) {
+		VC_LOG_ERR("Cannot update zero copy as mempool is not full");
+		return -EINVAL;
+	}
+
+	vcrypto->option = (uint8_t)option;
+
+	return 0;
+}
+
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+	struct rte_mbuf *mbufs[VHOST_CRYPTO_MAX_BURST_SIZE * 2];
+	struct virtio_net *dev = get_device(vid);
+	struct rte_vhost_memory *mem;
+	struct vhost_crypto *vcrypto;
+	struct vhost_virtqueue *vq;
+	uint16_t avail_idx;
+	uint16_t start_idx;
+	uint16_t required;
+	uint16_t count;
+	uint16_t i;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	if (unlikely(qid >= VHOST_MAX_QUEUE_PAIRS)) {
+		VC_LOG_ERR("Invalid qid %u", qid);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vq = dev->virtqueue[qid];
+	mem = dev->mem;
+
+	avail_idx = *((volatile uint16_t *)&vq->avail->idx);
+	start_idx = vq->last_used_idx;
+	count = avail_idx - start_idx;
+	count = RTE_MIN(count, VHOST_CRYPTO_MAX_BURST_SIZE);
+	count = RTE_MIN(count, nb_ops);
+
+	if (unlikely(count == 0))
+		return 0;
+
+	/* for zero copy, we need 2 empty mbufs for src and dst, otherwise
+	 * we need only 1 mbuf as src and dst
+	 */
+	required = count * 2;
+	if (unlikely(rte_mempool_get_bulk(vcrypto->mbuf_pool, (void **)mbufs,
+			required) < 0)) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	for (i = 0; i < count; i++) {
+		uint16_t used_idx = (start_idx + i) & (vq->size - 1);
+		uint16_t desc_idx = vq->avail->ring[used_idx];
+		struct vring_desc *head = &vq->desc[desc_idx];
+		struct rte_crypto_op *op = ops[i];
+
+		op->sym->m_src = mbufs[i * 2];
+		op->sym->m_dst = mbufs[i * 2 + 1];
+		op->sym->m_src->data_off = 0;
+		op->sym->m_dst->data_off = 0;
+
+		if (unlikely(vhost_crypto_process_one_req(vcrypto, vq, op, head,
+				desc_idx, mem)) < 0)
+			break;
+	}
+
+	vq->last_used_idx += i;
+
+	return i;
+}
+
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds)
+{
+	struct rte_crypto_op **tmp_ops = ops;
+	uint16_t count = 0, left = nb_ops;
+	int callfd;
+	uint16_t idx = 0;
+
+	while (left) {
+		count = vhost_crypto_complete_one_vm_requests(tmp_ops, left,
+				&callfd);
+		if (unlikely(count == 0))
+			break;
+
+		tmp_ops = &tmp_ops[count];
+		left -= count;
+
+		callfds[idx++] = callfd;
+
+		if (unlikely(idx >= VIRTIO_CRYPTO_MAX_NUM_BURST_VQS)) {
+			VC_LOG_ERR("Too many vqs");
+			break;
+		}
+	}
+
+	*nb_callfds = idx;
+
+	return nb_ops - left;
+}
-- 
2.13.6

[PATCH v5 6/8] lib/librte_vhost: update makefile

[Fan Zhang]

This patch updates the Makefile of vhost library to enable vhost crypto
compiling.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/Makefile    | 6 ++++--
 lib/librte_vhost/meson.build | 8 ++++----
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 5d6c6abae..95a6a9363 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -23,8 +23,10 @@ LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev -lrte_net
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_VHOST) := fd_man.c iotlb.c socket.c vhost.c \
 					vhost_user.c virtio_net.c
+ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV), y)
+SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto.c
+endif
 
 # install includes
-SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h
-
+SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h rte_vhost_crypto.h
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_vhost/meson.build b/lib/librte_vhost/meson.build
index 9e8c0e766..36f1e2754 100644
--- a/lib/librte_vhost/meson.build
+++ b/lib/librte_vhost/meson.build
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
+# Copyright(c) 2017-2018 Intel Corporation
 
 if host_machine.system() != 'linux'
 	build = false
@@ -10,6 +10,6 @@ endif
 version = 4
 allow_experimental_apis = true
 sources = files('fd_man.c', 'iotlb.c', 'socket.c', 'vhost.c', 'vhost_user.c',
-		'virtio_net.c')
-headers = files('rte_vhost.h')
-deps += ['ethdev']
+		'virtio_net.c', 'virtio_crypto.c')
+headers = files('rte_vhost.h', 'rte_vhost_crypto.h')
+deps += ['ethdev', 'cryptodev']
-- 
2.13.6

[PATCH v5 7/8] examples/vhost_crypto: add vhost crypto sample application

[Fan Zhang]

This patch adds vhost_crypto sample application to DPDK.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 examples/vhost_crypto/Makefile    |  32 +++
 examples/vhost_crypto/main.c      | 541 ++++++++++++++++++++++++++++++++++++++
 examples/vhost_crypto/meson.build |  14 +
 3 files changed, 587 insertions(+)
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 examples/vhost_crypto/meson.build

diff --git a/examples/vhost_crypto/Makefile b/examples/vhost_crypto/Makefile
new file mode 100644
index 000000000..1bb65e8fd
--- /dev/null
+++ b/examples/vhost_crypto/Makefile
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vhost-crypto
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
diff --git a/examples/vhost_crypto/main.c b/examples/vhost_crypto/main.c
new file mode 100644
index 000000000..bc867240d
--- /dev/null
+++ b/examples/vhost_crypto/main.c
@@ -0,0 +1,541 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <stdbool.h>
+#include <assert.h>
+#include <getopt.h>
+
+#include <rte_malloc.h>
+#include <rte_cycles.h>
+#include <rte_vhost.h>
+#include <rte_cryptodev.h>
+#include <rte_vhost_crypto.h>
+
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define NB_VIRTIO_QUEUES		(1)
+#define MAX_PKT_BURST			(64)
+#define MAX_IV_LEN			(32)
+#define NB_MEMPOOL_OBJS			(8192)
+#define NB_CRYPTO_DESCRIPTORS		(4096)
+#define NB_CACHE_OBJS			(128)
+#define SESSION_MAP_ENTRIES		(1024)
+#define REFRESH_TIME_SEC		(3)
+
+#define MAX_NB_SOCKETS			(32)
+#define DEF_SOCKET_FILE			"/tmp/vhost_crypto1.socket"
+
+struct vhost_crypto_options {
+	char *socket_files[MAX_NB_SOCKETS];
+	uint32_t nb_sockets;
+	uint8_t cid;
+	uint16_t qid;
+	uint32_t zero_copy;
+	uint32_t guest_polling;
+} options;
+
+struct vhost_crypto_info {
+	int vids[MAX_NB_SOCKETS];
+	struct rte_mempool *sess_pool;
+	struct rte_mempool *cop_pool;
+	uint32_t lcore_id;
+	uint8_t cid;
+	uint32_t qid;
+	uint32_t nb_vids;
+	volatile uint32_t initialized[MAX_NB_SOCKETS];
+
+} info;
+
+#define SOCKET_FILE_KEYWORD	"socket-file"
+#define CRYPTODEV_ID_KEYWORD	"cdev-id"
+#define CRYPTODEV_QUEUE_KEYWORD	"cdev-queue-id"
+#define ZERO_COPY_KEYWORD	"zero-copy"
+#define POLLING_KEYWORD		"guest-polling"
+
+uint64_t vhost_cycles[2], last_v_cycles[2];
+uint64_t outpkt_amount;
+
+/** support *SOCKET_FILE_PATH:CRYPTODEV_ID* format */
+static int
+parse_socket_arg(char *arg)
+{
+	uint32_t nb_sockets = options.nb_sockets;
+	size_t len = strlen(arg);
+
+	if (nb_sockets >= MAX_NB_SOCKETS) {
+		RTE_LOG(ERR, USER1, "Too many socket files!\n");
+		return -ENOMEM;
+	}
+
+	options.socket_files[nb_sockets] = rte_malloc(NULL, len, 0);
+	if (!options.socket_files[nb_sockets]) {
+		RTE_LOG(ERR, USER1, "Insufficient memory\n");
+		return -ENOMEM;
+	}
+
+	rte_memcpy(options.socket_files[nb_sockets], arg, len);
+
+	options.nb_sockets++;
+
+	return 0;
+}
+
+static int
+parse_cryptodev_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if ((pm == '\0') || (end == NULL) || (*end != '\0')) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	if (pm > rte_cryptodev_count()) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.cid = (uint8_t)pm;
+
+	return 0;
+}
+
+static int
+parse_cdev_queue_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if (pm == UINT64_MAX) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev Queue ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.qid = (uint16_t)pm;
+
+	return 0;
+}
+
+static void
+vhost_crypto_usage(const char *prgname)
+{
+	printf("%s [EAL options] --\n"
+		"  --%s SOCKET-FILE-PATH\n"
+		"  --%s CRYPTODEV_ID: crypto device id\n"
+		"  --%s CDEV_QUEUE_ID: crypto device queue id\n"
+		"  --%s: zero copy\n"
+		"  --%s: guest polling\n",
+		prgname, SOCKET_FILE_KEYWORD, CRYPTODEV_ID_KEYWORD,
+		CRYPTODEV_QUEUE_KEYWORD, ZERO_COPY_KEYWORD, POLLING_KEYWORD);
+}
+
+static int
+vhost_crypto_parse_args(int argc, char **argv)
+{
+	int opt, ret;
+	char *prgname = argv[0];
+	char **argvopt;
+	int option_index;
+	struct option lgopts[] = {
+			{SOCKET_FILE_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_ID_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_QUEUE_KEYWORD, required_argument, 0, 0},
+			{ZERO_COPY_KEYWORD, no_argument, 0, 0},
+			{POLLING_KEYWORD, no_argument, 0, 0},
+			{NULL, 0, 0, 0}
+	};
+
+	options.cid = 0;
+	options.qid = 0;
+	options.nb_sockets = 0;
+	options.guest_polling = 0;
+	options.zero_copy = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, "s:",
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		case 0:
+			if (strcmp(lgopts[option_index].name,
+					SOCKET_FILE_KEYWORD) == 0) {
+				ret = parse_socket_arg(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_ID_KEYWORD) == 0) {
+				ret = parse_cryptodev_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_QUEUE_KEYWORD) == 0) {
+				ret = parse_cdev_queue_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					ZERO_COPY_KEYWORD) == 0) {
+				options.zero_copy =
+					RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE;
+			} else if (strcmp(lgopts[option_index].name,
+					POLLING_KEYWORD) == 0) {
+				options.guest_polling = 1;
+			} else {
+				vhost_crypto_usage(prgname);
+				return -EINVAL;
+			}
+			break;
+		default:
+			return -1;
+		}
+	}
+
+	if (options.nb_sockets == 0) {
+		options.socket_files[0] = strdup(DEF_SOCKET_FILE);
+		options.nb_sockets = 1;
+		RTE_LOG(INFO, USER1,
+				"VHOST-CRYPTO: use default socket file %s\n",
+				DEF_SOCKET_FILE);
+	}
+
+	return 0;
+}
+
+static int
+new_device(int vid)
+{
+	char path[PATH_MAX];
+	uint32_t idx, i;
+	int ret;
+
+	ret = rte_vhost_get_ifname(vid, path, PATH_MAX);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot find matched socket\n");
+		return ret;
+	}
+
+	for (idx = 0; idx < options.nb_sockets; idx++) {
+		if (strcmp(path, options.socket_files[idx]) == 0)
+			break;
+	}
+
+	if (idx == options.nb_sockets) {
+		RTE_LOG(ERR, USER1, "Cannot find recorded socket\n");
+		return -ENOENT;
+	}
+
+	for (i = 0; i < 2; i++) {
+		vhost_cycles[i] = 0;
+		last_v_cycles[i] = 0;
+	}
+
+	ret = rte_vhost_crypto_create(vid, info.cid, info.sess_pool,
+			rte_lcore_to_socket_id(info.lcore_id));
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot create vhost crypto\n");
+		return ret;
+	}
+
+	ret = rte_vhost_crypto_set_zero_copy(vid, options.zero_copy);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot %s zero copy feature\n",
+				options.zero_copy == 1 ? "enable" : "disable");
+		return ret;
+	}
+
+	info.vids[idx] = vid;
+	info.initialized[idx] = 1;
+
+	rte_wmb();
+
+	RTE_LOG(INFO, USER1, "New Vhost-crypto Device %s, Device ID %d\n", path,
+			vid);
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	uint32_t i;
+
+	for (i = 0; i < info.nb_vids; i++) {
+		if (vid == info.vids[i])
+			break;
+	}
+
+	if (i == info.nb_vids) {
+		RTE_LOG(ERR, USER1, "Cannot find socket file from list\n");
+		return;
+	}
+
+	info.initialized[i] = 0;
+
+	rte_wmb();
+
+	rte_vhost_crypto_free(vid);
+
+	RTE_LOG(INFO, USER1, "Vhost Crypto Device %i Removed\n", vid);
+}
+
+static const struct vhost_device_ops virtio_crypto_device_ops = {
+	.new_device =  new_device,
+	.destroy_device = destroy_device,
+};
+
+__attribute__((unused))
+static void clrscr(void)
+{
+	system("@cls||clear");
+}
+
+static int
+vhost_crypto_worker(__rte_unused void *arg)
+{
+	struct rte_crypto_op *ops[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	struct rte_crypto_op *ops_deq[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	uint32_t nb_inflight_ops = 0;
+	uint16_t nb_callfds;
+	int callfds[VIRTIO_CRYPTO_MAX_NUM_BURST_VQS];
+	uint32_t lcore_id = rte_lcore_id();
+	uint32_t burst_size = MAX_PKT_BURST;
+	uint32_t i, j, k;
+	uint32_t to_fetch, fetched;
+	uint64_t t_start, t_end, interval;
+
+	int ret = 0;
+
+	RTE_LOG(INFO, USER1, "Processing on Core %u started\n", lcore_id);
+
+	for (i = 0; i < NB_VIRTIO_QUEUES; i++) {
+		if (rte_crypto_op_bulk_alloc(info.cop_pool,
+				RTE_CRYPTO_OP_TYPE_SYMMETRIC, ops[i],
+				burst_size) < burst_size) {
+			RTE_LOG(ERR, USER1, "Failed to alloc cops\n");
+			ret = -1;
+			goto exit;
+		}
+	}
+
+	while (1) {
+		for (i = 0; i < info.nb_vids; i++) {
+			if (unlikely(info.initialized[i] == 0))
+				continue;
+
+			for (j = 0; j < NB_VIRTIO_QUEUES; j++) {
+				t_start = rte_rdtsc_precise();
+
+				to_fetch = RTE_MIN(burst_size,
+						(NB_CRYPTO_DESCRIPTORS -
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_fetch_requests(
+						info.vids[i], j, ops[j],
+						to_fetch);
+				nb_inflight_ops += rte_cryptodev_enqueue_burst(
+						info.cid, info.qid, ops[j],
+						fetched);
+				if (unlikely(rte_crypto_op_bulk_alloc(
+						info.cop_pool,
+						RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+						ops[j], fetched) < fetched)) {
+					RTE_LOG(ERR, USER1, "Failed realloc\n");
+					return -1;
+				}
+				t_end = rte_rdtsc_precise();
+				interval = t_end - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+
+				t_start = t_end;
+				fetched = rte_cryptodev_dequeue_burst(
+						info.cid, info.qid,
+						ops_deq[j], RTE_MIN(burst_size,
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_finalize_requests(
+						ops_deq[j], fetched, callfds,
+						&nb_callfds);
+
+				nb_inflight_ops -= fetched;
+				outpkt_amount += fetched;
+
+				if (!options.guest_polling) {
+					for (k = 0; k < nb_callfds; k++)
+						eventfd_write(callfds[k],
+								(eventfd_t)1);
+				}
+
+				rte_mempool_put_bulk(info.cop_pool,
+						(void **)ops_deq[j], fetched);
+				interval = rte_rdtsc_precise() - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+			}
+		}
+	}
+exit:
+	return ret;
+}
+
+
+static void
+unregister_drivers(int socket_num)
+{
+	int ret;
+
+	ret = rte_vhost_driver_unregister(options.socket_files[socket_num]);
+	if (ret != 0)
+		RTE_LOG(ERR, USER1,
+			"Fail to unregister vhost driver for %s.\n",
+			options.socket_files[socket_num]);
+}
+
+int
+main(int argc, char *argv[])
+{
+	struct rte_cryptodev_qp_conf qp_conf = {NB_CRYPTO_DESCRIPTORS};
+	struct rte_cryptodev_config config;
+	struct rte_cryptodev_info dev_info;
+	uint32_t cryptodev_id;
+	uint32_t worker_lcore;
+	char name[128];
+	uint32_t i = 0;
+	int ret;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		return -1;
+	argc -= ret;
+	argv += ret;
+
+	ret = vhost_crypto_parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Failed to parse arguments!\n");
+
+	info.cid = options.cid;
+	info.qid = options.qid;
+
+	worker_lcore = rte_get_next_lcore(0, 1, 0);
+	if (worker_lcore == RTE_MAX_LCORE)
+		rte_exit(EXIT_FAILURE, "Not enough lcore\n");
+
+	cryptodev_id = info.cid;
+	rte_cryptodev_info_get(cryptodev_id, &dev_info);
+	if (dev_info.max_nb_queue_pairs < info.qid + 1) {
+		RTE_LOG(ERR, USER1, "Number of queues cannot over %u",
+				dev_info.max_nb_queue_pairs);
+		goto error_exit;
+	}
+
+	config.nb_queue_pairs = dev_info.max_nb_queue_pairs;
+	config.socket_id = rte_lcore_to_socket_id(worker_lcore);
+
+	ret = rte_cryptodev_configure(cryptodev_id, &config);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to configure cryptodev %u",
+				cryptodev_id);
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "SESS_POOL_%u", worker_lcore);
+	info.sess_pool = rte_mempool_create(name, SESSION_MAP_ENTRIES,
+			rte_cryptodev_get_private_session_size(
+			cryptodev_id), 64, 0, NULL, NULL, NULL, NULL,
+			rte_lcore_to_socket_id(worker_lcore), 0);
+	if (!info.sess_pool) {
+		RTE_LOG(ERR, USER1, "Failed to create mempool");
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "COPPOOL_%u", worker_lcore);
+	info.cop_pool = rte_crypto_op_pool_create(name,
+			RTE_CRYPTO_OP_TYPE_SYMMETRIC, NB_MEMPOOL_OBJS,
+			NB_CACHE_OBJS, 0, rte_lcore_to_socket_id(worker_lcore));
+
+	if (!info.cop_pool) {
+		RTE_LOG(ERR, USER1, "Lcore %u failed to create crypto pool",
+				worker_lcore);
+		ret = -1;
+		goto error_exit;
+	}
+
+	info.nb_vids = options.nb_sockets;
+	for (i = 0; i < MAX_NB_SOCKETS; i++)
+		info.vids[i] = -1;
+
+	for (i = 0; i < dev_info.max_nb_queue_pairs; i++) {
+		ret = rte_cryptodev_queue_pair_setup(cryptodev_id, i,
+				&qp_conf, rte_lcore_to_socket_id(worker_lcore),
+				info.sess_pool);
+		if (ret < 0) {
+			RTE_LOG(ERR, USER1, "Failed to configure qp %u\n",
+					info.cid);
+			goto error_exit;
+		}
+	}
+
+	ret = rte_cryptodev_start(cryptodev_id);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to start cryptodev %u\n", info.cid);
+		goto error_exit;
+	}
+
+	info.cid = cryptodev_id;
+	info.lcore_id = worker_lcore;
+
+	if (rte_eal_remote_launch(vhost_crypto_worker, NULL, worker_lcore)
+			< 0) {
+		RTE_LOG(ERR, USER1, "Failed to start worker lcore");
+		goto error_exit;
+	}
+
+	for (i = 0; i < options.nb_sockets; i++) {
+		if (rte_vhost_driver_register(options.socket_files[i],
+				RTE_VHOST_USER_DEQUEUE_ZERO_COPY) < 0) {
+			RTE_LOG(ERR, USER1, "socket %s already exists\n",
+					options.socket_files[i]);
+			goto error_exit;
+		}
+
+		rte_vhost_driver_callback_register(options.socket_files[i],
+				&virtio_crypto_device_ops);
+
+		if (rte_vhost_driver_start(options.socket_files[i]) < 0) {
+			RTE_LOG(ERR, USER1, "failed to start vhost driver.\n");
+			goto error_exit;
+		}
+	}
+
+	RTE_LCORE_FOREACH(worker_lcore)
+		rte_eal_wait_lcore(worker_lcore);
+
+	rte_mempool_free(info.sess_pool);
+	rte_mempool_free(info.cop_pool);
+
+	return 0;
+
+error_exit:
+	for (i = 0; i < options.nb_sockets; i++)
+		unregister_drivers(i);
+
+	rte_mempool_free(info.cop_pool);
+	rte_mempool_free(info.sess_pool);
+
+	return -1;
+}
diff --git a/examples/vhost_crypto/meson.build b/examples/vhost_crypto/meson.build
new file mode 100644
index 000000000..0f4876f06
--- /dev/null
+++ b/examples/vhost_crypto/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+allow_experimental_apis = true
+deps += ['vhost', 'cryptodev']
+cflags += ['-D_GNU_SOURCE','-D_FILE_OFFSET_BITS=64']
+sources = files(
+	'main.c'
+)
-- 
2.13.6

[PATCH v5 8/8] doc: update for vhost crypto support

[Fan Zhang]

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst       | 25 ++++++++++
 doc/guides/rel_notes/release_18_05.rst    |  5 ++
 doc/guides/sample_app_ug/index.rst        |  1 +
 doc/guides/sample_app_ug/vhost_crypto.rst | 83 +++++++++++++++++++++++++++++++
 4 files changed, 114 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index f47473609..04ab1eeb4 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -160,6 +160,31 @@ The following is an overview of some key Vhost API functions:
 
   Receives (dequeues) ``count`` packets from guest, and stored them at ``pkts``.
 
+* ``rte_vhost_crypto_create(vid, cryptodev_id, sess_mempool, socket_id)``
+
+  As an extension of new_device(), this function adds virtio-crypto workload
+  acceleration capability to the device. All crypto workload is processed by
+  DPDK cryptodev with the device ID of ``cryptodev_id``.
+
+* ``rte_vhost_crypto_free(vid)``
+
+  Frees the memory and vhost-user message handlers created in
+  rte_vhost_crypto_create().
+
+* ``rte_vhost_crypto_fetch_requests(vid, queue_id, ops, nb_ops)``
+
+  Receives (dequeues) ``nb_ops`` virtio-crypto requests from guest, parses
+  them to DPDK Crypto Operations, and fills the ``ops`` with parsing results.
+
+* ``rte_vhost_crypto_finalize_requests(queue_id, ops, nb_ops)``
+
+  After the ``ops`` are dequeued from Cryptodev, finalizes the jobs and
+  notifies the guest(s).
+
+* ``rte_vhost_crypto_set_zero_copy(vid, option)``
+
+  Enable or disable zero copy feature of the vhost crypto backend.
+
 Vhost-user Implementations
 --------------------------
 
diff --git a/doc/guides/rel_notes/release_18_05.rst b/doc/guides/rel_notes/release_18_05.rst
index 9cc77f893..6451f8ea0 100644
--- a/doc/guides/rel_notes/release_18_05.rst
+++ b/doc/guides/rel_notes/release_18_05.rst
@@ -58,6 +58,11 @@ New Features
   * Added support for NVGRE, VXLAN and GENEVE filters in flow API.
   * Added support for DROP action in flow API.
 
+* **Added crypto workload support to vhost library
+
+  New APIs are introduced in vhost library to enable virtio crypto support
+  including session creation/deletion handling and translating virito-crypto
+  request into DPDK crypto operations. A sample application is also introduced.
 
 API Changes
 -----------
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index e87afda80..57e8354d4 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -44,6 +44,7 @@ Sample Applications User Guides
     vmdq_dcb_forwarding
     vhost
     vhost_scsi
+    vhost_crypto
     netmap_compatibility
     ip_pipeline
     test_pipeline
diff --git a/doc/guides/sample_app_ug/vhost_crypto.rst b/doc/guides/sample_app_ug/vhost_crypto.rst
new file mode 100644
index 000000000..316988738
--- /dev/null
+++ b/doc/guides/sample_app_ug/vhost_crypto.rst
@@ -0,0 +1,83 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2017-2018 Intel Corporation.
+
+Vhost_Crypto Sample Application
+===============================
+
+The vhost_crypto sample application implemented a simple Crypto device,
+which used as the  backend of Qemu vhost-user-crypto device. Similar with
+vhost-user-net and vhost-user-scsi device, the sample application used
+domain socket to communicate with Qemu, and the virtio ring was processed
+by vhost_crypto sample application.
+
+Testing steps
+-------------
+
+This section shows the steps how to start a VM with the crypto device as
+fast data path for critical application.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``examples`` sub-directory.
+
+Start the vhost_crypto example
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+    ./vhost_crypto [EAL options] -- [--socket-file PATH]
+        [--cdev-id ID] [--cdev-queue-id ID] [--zero-copy] [--guest-polling]
+
+where,
+
+* socket-file PATH: the path of UNIX socket file to be created, multiple
+  instances of this config item is supported. Upon absence of this item,
+  the default socket-file `/tmp/vhost_crypto1.socket` is used.
+
+* cdev-id ID: the target DPDK Cryptodev's ID to process the actual crypto
+  workload. Upon absence of this item the default value of `0` will be used.
+  For details of DPDK Cryptodev, please refer to DPDK Cryptodev Library
+  Programmers' Guide.
+
+* cdev-queue-id ID: the target DPDK Cryptodev's queue ID to process the
+  actual crypto workload. Upon absence of this item the default value of `0`
+  will be used. For details of DPDK Cryptodev, please refer to DPDK Cryptodev
+  Library Programmers' Guide.
+
+* zero-copy: the presence of this item means the ZERO-COPY feature will be
+  enabled. Otherwise it is disabled. PLEASE NOTE the ZERO-COPY feature is still
+  in experimental stage and may cause the problem like segmentation fault. If
+  the user wants to use LKCF in the guest, this feature shall be turned off.
+
+* guest-polling: the presence of this item means the application assumes the
+  guest works in polling mode, thus will NOT notify the guest completion of
+  processing.
+
+The application requires that crypto devices capable of performing
+the specified crypto operation are available on application initialization.
+This means that HW crypto device/s must be bound to a DPDK driver or
+a SW crypto device/s (virtual crypto PMD) must be created (using --vdev).
+
+.. _vhost_crypto_app_run_vm:
+
+Start the VM
+~~~~~~~~~~~~
+
+.. code-block:: console
+
+    qemu-system-x86_64 -machine accel=kvm \
+        -m $mem -object memory-backend-file,id=mem,size=$mem,\
+        mem-path=/dev/hugepages,share=on -numa node,memdev=mem \
+        -drive file=os.img,if=none,id=disk \
+        -device ide-hd,drive=disk,bootindex=0 \
+        -chardev socket,id={chardev_id},path={PATH} \
+        -object cryptodev-vhost-user,id={obj_id},chardev={chardev_id} \
+        -device virtio-crypto-pci,id={dev_id},cryptodev={obj_id} \
+        ...
+
+.. note::
+    You must check whether your Qemu can support "vhost-user-crypto" or not.
+
-- 
2.13.6

Re: [PATCH v5 1/8] lib/librte_vhost: add external backend support

[Maxime Coquelin]

Hi,

I think the title should change, as it makes to think it could be used 
by external backends that doesn't live in the librte_vhost directory.

One more comment below, otherwise that looks good to me.

Thanks,
Maxime

On 04/04/2018 12:08 PM, Fan Zhang wrote:
> This patch adds external backend support to vhost library. The patch provides
> new function prototypes for the external backend to register pre and post
> vhost-user message handlers.
> 
> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> ---
>   lib/librte_vhost/vhost.c      |  2 +-
>   lib/librte_vhost/vhost.h      | 53 +++++++++++++++++++++++++++++++++++++++++--
>   lib/librte_vhost/vhost_user.c | 31 ++++++++++++++++++++++++-
>   3 files changed, 82 insertions(+), 4 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
> index f6f12a03b..99f8b01fb 100644
> --- a/lib/librte_vhost/vhost.c
> +++ b/lib/librte_vhost/vhost.c
> @@ -1,5 +1,5 @@
>   /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2016 Intel Corporation
> + * Copyright(c) 2010-2017 Intel Corporation
>    */
>   
>   #include <linux/vhost.h>
> diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
> index 4981e6d0c..f28164849 100644
> --- a/lib/librte_vhost/vhost.h
> +++ b/lib/librte_vhost/vhost.h
> @@ -1,5 +1,5 @@
>   /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2014 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>    */
>   
>   #ifndef _VHOST_NET_CDEV_H_
> @@ -211,6 +211,51 @@ struct guest_page {
>   };
>   
>   /**
> + * function prototype for the vhost backend to handler specific vhost user
> + * messages prior to the master message handling
> + *
> + * @param vid
> + *  vhost device id
> + * @param msg
> + *  Message pointer.
> + * @param require_reply
> + *  If the handler requires sending a reply, this varaible shall be written 1,
> + *  otherwise 0.
> + * @param skip_master
> + *  If the handler requires skipping the master message handling, this variable
> + *  shall be written 1, otherwise 0.
> + * @return
> + *  0 on success, -1 on failure
> + */
> +typedef int (*rte_vhost_msg_pre_handle)(int vid, void *msg,
> +		uint32_t *require_reply, uint32_t *skip_master);
Also, please remove the rte_ prefix here and evrywhere else as it is
purely vhost lib internal for now.

> +
> +/**
> + * function prototype for the vhost backend to handler specific vhost user
> + * messages after the master message handling is done
> + *
> + * @param vid
> + *  vhost device id
> + * @param msg
> + *  Message pointer.
> + * @param require_reply
> + *  If the handler requires sending a reply, this varaible shall be written 1,
> + *  otherwise 0.
> + * @return
> + *  0 on success, -1 on failure
> + */
> +typedef int (*rte_vhost_msg_post_handle)(int vid, void *msg,
> +		uint32_t *require_reply);
> +
> +/**
> + * pre and post vhost user message handlers
> + */
> +struct vhost_user_extern_ops {
> +	rte_vhost_msg_pre_handle pre_msg_handle;
> +	rte_vhost_msg_post_handle post_msg_handle;
> +};
> +
> +/**
>    * Device structure contains all configuration information relating
>    * to the device.
>    */
> @@ -242,8 +287,12 @@ struct virtio_net {
>   	struct guest_page       *guest_pages;
>   
>   	int			slave_req_fd;
> -} __rte_cache_aligned;
>   
> +	/* private data for external virtio device */
> +	void			*extern_data;
> +	/* pre and post vhost user message handlers for externel backend */
> +	struct vhost_user_extern_ops extern_ops;
> +} __rte_cache_aligned;
>   
>   #define VHOST_LOG_PAGE	4096
>   
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 66e1b82a5..a435cce17 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -1,5 +1,5 @@
>   /* SPDX-License-Identifier: BSD-3-Clause
> - * Copyright(c) 2010-2016 Intel Corporation
> + * Copyright(c) 2010-2018 Intel Corporation
>    */
>   
>   /* Security model
> @@ -1379,6 +1379,7 @@ vhost_user_msg_handler(int vid, int fd)
>   	struct VhostUserMsg msg;
>   	int ret;
>   	int unlock_required = 0;
> +	uint32_t skip_master = 0;
>   
>   	dev = get_device(vid);
>   	if (dev == NULL)
> @@ -1456,6 +1457,21 @@ vhost_user_msg_handler(int vid, int fd)
>   
>   	}
>   
> +	if (dev->extern_ops.pre_msg_handle) {
> +		uint32_t need_reply;
> +
> +		ret = (*dev->extern_ops.pre_msg_handle)(dev->vid,
> +				(void *)&msg, &need_reply, &skip_master);
> +		if (ret < 0)
> +			goto skip_to_reply;
> +
> +		if (need_reply)
> +			send_vhost_reply(fd, &msg);
> +
> +		if (skip_master)
> +			goto skip_to_post_handle;
> +	}
> +
>   	switch (msg.request.master) {
>   	case VHOST_USER_GET_FEATURES:
>   		msg.payload.u64 = vhost_user_get_features(dev);
> @@ -1556,9 +1572,22 @@ vhost_user_msg_handler(int vid, int fd)
>   	default:
>   		ret = -1;
>   		break;
> +	}
> +
> +skip_to_post_handle:
> +	if (dev->extern_ops.post_msg_handle) {
> +		uint32_t need_reply;
> +
> +		ret = (*dev->extern_ops.post_msg_handle)(
> +				dev->vid, (void *)&msg, &need_reply);
> +		if (ret < 0)
> +			goto skip_to_reply;
>   
> +		if (need_reply)
> +			send_vhost_reply(fd, &msg);
>   	}
>   
> +skip_to_reply:
>   	if (unlock_required)
>   		vhost_user_unlock_all_queue_pairs(dev);
>   
> 

[PATCH v6 0/8] vhost: introduce vhost crypto backend

[Fan Zhang]

This patchset adds crypto backend suppport to vhost library
including a proof-of-concept sample application. The implementation
follows the virtio-crypto specification and have been tested
with qemu 2.11.50 (with several patches applied, detailed later)
with Fedora 24 running in the frontend.

The vhost_crypto library acts as a "bridge" method that translate
the virtio-crypto crypto requests to DPDK crypto operations, so it
is purely software implementation. However it does require the user
to provide the DPDK Cryptodev ID so it knows how to handle the
virtio-crypto session creation and deletion mesages.

Currently the implementation supports AES-CBC-128 and HMAC-SHA1
cipher only/chaining modes and does not support sessionless mode
yet. The guest can use standard virtio-crypto driver to set up
session and sends encryption/decryption requests to backend. The
vhost-crypto sample application provided in this patchset will
do the actual crypto work.

The following steps are involved to enable vhost-crypto support.

In the host:
1. Download the qemu source code.

2. Recompile your qemu with vhost-crypto option enabled.

3. Apply this patchset to latest DPDK code and recompile DPDK.

4. Compile and run vhost-crypto sample application.

./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
 --socket-mem 2048,2048

Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
not available. The sample application requires 2 lcores: 1 master and 1
worker. The application will create a UNIX socket file
/tmp/vhost_crypto1.socket.

5. Start your qemu application. Here is my command:

qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
-smp 2 -m 1G -hda ~/path-to-your/image.qcow \
-object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
-mem-prealloc -numa node,memdev=mem -chardev \
socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
-object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
-device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0

6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
default. You shall see the following logs in your demsg:

[   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
[   17.611083] virtio_crypto: module verification failed: signature and/or ...
[   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
[   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
[   18.376100] virtio_crypto virtio0: Accelerator is ready

The virtio_crypto driver in the guest is now up and running.

7. The rest steps can be as same as the Testing section in
https://wiki.qemu.org/Features/VirtioCrypto

8. It is possible to use DPDK Virtio Crypto PMD
(https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work with
this patchset to achieve optimal performance.

v6:
- Changed commit message
- removed rte prefix in handler prototype

v5:
- removed external ops register API.
- patch cleaned.

v4:
- Changed external vhost backend ops register API.
- Fixed a bug.

v3:
- Changed external vhost backend private data and message handling
- Added experimental tag to rte_vhost_crypto_set_zero_copy()

v2:
- Moved vhost_crypto_data_req data from crypto op to source mbuf.
- Removed ZERO-COPY flag from config option and make it run-timely changeable.
- Guest-polling mode possible.
- Simplified vring descriptor access procedure.
- Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.

Fan Zhang (8):
  lib/librte_vhost: add vhost user message handlers
  lib/librte_vhost: add virtio-crypto user message structure
  lib/librte_vhost: add session message handler
  lib/librte_vhost: add request handler
  lib/librte_vhost: add public function implementation
  lib/librte_vhost: update makefile
  examples/vhost_crypto: add vhost crypto sample application
  doc: update for vhost crypto support

 doc/guides/prog_guide/vhost_lib.rst       |   25 +
 doc/guides/rel_notes/release_18_05.rst    |    5 +
 doc/guides/sample_app_ug/index.rst        |    1 +
 doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
 examples/vhost_crypto/Makefile            |   32 +
 examples/vhost_crypto/main.c              |  541 ++++++++++++
 examples/vhost_crypto/meson.build         |   14 +
 lib/librte_vhost/Makefile                 |    6 +-
 lib/librte_vhost/meson.build              |    8 +-
 lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
 lib/librte_vhost/rte_vhost_version.map    |   11 +
 lib/librte_vhost/vhost.c                  |    2 +-
 lib/librte_vhost/vhost.h                  |   53 +-
 lib/librte_vhost/vhost_crypto.c           | 1312 +++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c             |   33 +-
 lib/librte_vhost/vhost_user.h             |   35 +-
 16 files changed, 2256 insertions(+), 13 deletions(-)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 examples/vhost_crypto/meson.build
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
 create mode 100644 lib/librte_vhost/vhost_crypto.c

-- 
2.13.6

[PATCH v6 1/8] lib/librte_vhost: add vhost user message handlers

[Fan Zhang]

Previously, vhost library lacks the support to the vhost backend
other than net such as adding private data or registering vhost-user
message handlers. This patch fills the gap by adding data pointer and
vhost-user pre and post message handlers to vhost library.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost.c      |  2 +-
 lib/librte_vhost/vhost.h      | 53 +++++++++++++++++++++++++++++++++++++++++--
 lib/librte_vhost/vhost_user.c | 31 ++++++++++++++++++++++++-
 3 files changed, 82 insertions(+), 4 deletions(-)

diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
index f6f12a03b..99f8b01fb 100644
--- a/lib/librte_vhost/vhost.c
+++ b/lib/librte_vhost/vhost.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2017 Intel Corporation
  */
 
 #include <linux/vhost.h>
diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
index 4981e6d0c..fb2f050eb 100644
--- a/lib/librte_vhost/vhost.h
+++ b/lib/librte_vhost/vhost.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_CDEV_H_
@@ -211,6 +211,51 @@ struct guest_page {
 };
 
 /**
+ * function prototype for the vhost backend to handler specific vhost user
+ * messages prior to the master message handling
+ *
+ * @param vid
+ *  vhost device id
+ * @param msg
+ *  Message pointer.
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0.
+ * @param skip_master
+ *  If the handler requires skipping the master message handling, this variable
+ *  shall be written 1, otherwise 0.
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*vhost_msg_pre_handle)(int vid, void *msg,
+		uint32_t *require_reply, uint32_t *skip_master);
+
+/**
+ * function prototype for the vhost backend to handler specific vhost user
+ * messages after the master message handling is done
+ *
+ * @param vid
+ *  vhost device id
+ * @param msg
+ *  Message pointer.
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0.
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*vhost_msg_post_handle)(int vid, void *msg,
+		uint32_t *require_reply);
+
+/**
+ * pre and post vhost user message handlers
+ */
+struct vhost_user_extern_ops {
+	vhost_msg_pre_handle pre_msg_handle;
+	vhost_msg_post_handle post_msg_handle;
+};
+
+/**
  * Device structure contains all configuration information relating
  * to the device.
  */
@@ -242,8 +287,12 @@ struct virtio_net {
 	struct guest_page       *guest_pages;
 
 	int			slave_req_fd;
-} __rte_cache_aligned;
 
+	/* private data for external virtio device */
+	void			*extern_data;
+	/* pre and post vhost user message handlers for externel backend */
+	struct vhost_user_extern_ops extern_ops;
+} __rte_cache_aligned;
 
 #define VHOST_LOG_PAGE	4096
 
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 66e1b82a5..a435cce17 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 /* Security model
@@ -1379,6 +1379,7 @@ vhost_user_msg_handler(int vid, int fd)
 	struct VhostUserMsg msg;
 	int ret;
 	int unlock_required = 0;
+	uint32_t skip_master = 0;
 
 	dev = get_device(vid);
 	if (dev == NULL)
@@ -1456,6 +1457,21 @@ vhost_user_msg_handler(int vid, int fd)
 
 	}
 
+	if (dev->extern_ops.pre_msg_handle) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.pre_msg_handle)(dev->vid,
+				(void *)&msg, &need_reply, &skip_master);
+		if (ret < 0)
+			goto skip_to_reply;
+
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
+
+		if (skip_master)
+			goto skip_to_post_handle;
+	}
+
 	switch (msg.request.master) {
 	case VHOST_USER_GET_FEATURES:
 		msg.payload.u64 = vhost_user_get_features(dev);
@@ -1556,9 +1572,22 @@ vhost_user_msg_handler(int vid, int fd)
 	default:
 		ret = -1;
 		break;
+	}
+
+skip_to_post_handle:
+	if (dev->extern_ops.post_msg_handle) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.post_msg_handle)(
+				dev->vid, (void *)&msg, &need_reply);
+		if (ret < 0)
+			goto skip_to_reply;
 
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
 	}
 
+skip_to_reply:
 	if (unlock_required)
 		vhost_user_unlock_all_queue_pairs(dev);
 
-- 
2.13.6

[PATCH v6 2/8] lib/librte_vhost: add virtio-crypto user message structure

[Fan Zhang]

This patch adds virtio-crypto spec user message structure to
vhost_user.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_user.h | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h
index 0fafbe6e0..33d64fc82 100644
--- a/lib/librte_vhost/vhost_user.h
+++ b/lib/librte_vhost/vhost_user.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_USER_H
@@ -20,13 +20,15 @@
 #define VHOST_USER_PROTOCOL_F_REPLY_ACK	3
 #define VHOST_USER_PROTOCOL_F_NET_MTU 4
 #define VHOST_USER_PROTOCOL_F_SLAVE_REQ 5
+#define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7
 
 #define VHOST_USER_PROTOCOL_FEATURES	((1ULL << VHOST_USER_PROTOCOL_F_MQ) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) |\
 					 (1ULL << VHOST_USER_PROTOCOL_F_RARP) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_NET_MTU) | \
-					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ))
+					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ) | \
+					 (1ULL << VHOST_USER_PROTOCOL_F_CRYPTO_SESSION))
 
 typedef enum VhostUserRequest {
 	VHOST_USER_NONE = 0,
@@ -52,7 +54,9 @@ typedef enum VhostUserRequest {
 	VHOST_USER_NET_SET_MTU = 20,
 	VHOST_USER_SET_SLAVE_REQ_FD = 21,
 	VHOST_USER_IOTLB_MSG = 22,
-	VHOST_USER_MAX
+	VHOST_USER_CRYPTO_CREATE_SESS = 26,
+	VHOST_USER_CRYPTO_CLOSE_SESS = 27,
+	VHOST_USER_MAX = 28
 } VhostUserRequest;
 
 typedef enum VhostUserSlaveRequest {
@@ -79,6 +83,30 @@ typedef struct VhostUserLog {
 	uint64_t mmap_offset;
 } VhostUserLog;
 
+/* Comply with Cryptodev-Linux */
+#define VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH	512
+#define VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH	64
+
+/* Same structure as vhost-user backend session info */
+typedef struct VhostUserCryptoSessionParam {
+	int64_t session_id;
+	uint32_t op_code;
+	uint32_t cipher_algo;
+	uint32_t cipher_key_len;
+	uint32_t hash_algo;
+	uint32_t digest_len;
+	uint32_t auth_key_len;
+	uint32_t aad_len;
+	uint8_t op_type;
+	uint8_t dir;
+	uint8_t hash_mode;
+	uint8_t chaining_dir;
+	uint8_t *ciphe_key;
+	uint8_t *auth_key;
+	uint8_t cipher_key_buf[VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH];
+	uint8_t auth_key_buf[VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH];
+} VhostUserCryptoSessionParam;
+
 typedef struct VhostUserMsg {
 	union {
 		uint32_t master; /* a VhostUserRequest value */
@@ -99,6 +127,7 @@ typedef struct VhostUserMsg {
 		VhostUserMemory memory;
 		VhostUserLog    log;
 		struct vhost_iotlb_msg iotlb;
+		VhostUserCryptoSessionParam crypto_session;
 	} payload;
 	int fds[VHOST_MEMORY_MAX_NREGIONS];
 } __attribute((packed)) VhostUserMsg;
-- 
2.13.6

[PATCH v6 3/8] lib/librte_vhost: add session message handler

[Fan Zhang]

This patch adds session message handler to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 435 ++++++++++++++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c   |   2 +
 2 files changed, 437 insertions(+)
 create mode 100644 lib/librte_vhost/vhost_crypto.c

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
new file mode 100644
index 000000000..34587c27e
--- /dev/null
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -0,0 +1,435 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+#include <linux/virtio_crypto.h>
+
+#include <rte_malloc.h>
+#include <rte_hash.h>
+#include <rte_mbuf.h>
+#include <rte_cryptodev.h>
+
+#include "vhost.h"
+#include "vhost_user.h"
+
+#define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
+				sizeof(struct rte_crypto_sym_op))
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+
+#define VC_LOG_DBG(fmt, args...)				\
+	RTE_LOG(DEBUG, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#else
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_DBG(fmt, args...)
+#endif
+
+static int
+cipher_algo_transform(uint32_t virtio_cipher_algo)
+{
+	int ret;
+
+	switch (virtio_cipher_algo) {
+	case VIRTIO_CRYPTO_CIPHER_AES_CBC:
+		ret = RTE_CRYPTO_CIPHER_AES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_CTR:
+		ret = RTE_CRYPTO_CIPHER_AES_CTR;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_DES_ECB:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_DES_CBC:
+		ret = RTE_CRYPTO_CIPHER_DES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_ECB:
+		ret = RTE_CRYPTO_CIPHER_3DES_ECB;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_CBC:
+		ret = RTE_CRYPTO_CIPHER_3DES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_CTR:
+		ret = RTE_CRYPTO_CIPHER_3DES_CTR;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_KASUMI_F8:
+		ret = RTE_CRYPTO_CIPHER_KASUMI_F8;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_SNOW3G_UEA2:
+		ret = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_F8:
+		ret = RTE_CRYPTO_CIPHER_AES_F8;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_XTS:
+		ret = RTE_CRYPTO_CIPHER_AES_XTS;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_ZUC_EEA3:
+		ret = RTE_CRYPTO_CIPHER_ZUC_EEA3;
+		break;
+	default:
+		ret = -VIRTIO_CRYPTO_BADMSG;
+		break;
+	}
+
+	return ret;
+}
+
+static int
+auth_algo_transform(uint32_t virtio_auth_algo)
+{
+	int ret;
+
+	switch (virtio_auth_algo) {
+
+	case VIRTIO_CRYPTO_NO_MAC:
+		ret = RTE_CRYPTO_AUTH_NULL;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_MD5:
+		ret = RTE_CRYPTO_AUTH_MD5_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA1:
+		ret = RTE_CRYPTO_AUTH_SHA1_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_224:
+		ret = RTE_CRYPTO_AUTH_SHA224_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_256:
+		ret = RTE_CRYPTO_AUTH_SHA256_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_384:
+		ret = RTE_CRYPTO_AUTH_SHA384_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_512:
+		ret = RTE_CRYPTO_AUTH_SHA512_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_CMAC_3DES:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_CMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_CMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_KASUMI_F9:
+		ret = RTE_CRYPTO_AUTH_KASUMI_F9;
+		break;
+	case VIRTIO_CRYPTO_MAC_SNOW3G_UIA2:
+		ret = RTE_CRYPTO_AUTH_SNOW3G_UIA2;
+		break;
+	case VIRTIO_CRYPTO_MAC_GMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_GMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_GMAC_TWOFISH:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_CBCMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_CBC_MAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_XCBC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_XCBC_MAC;
+		break;
+	default:
+		ret = -VIRTIO_CRYPTO_BADMSG;
+		break;
+	}
+
+	return ret;
+}
+
+static int get_iv_len(enum rte_crypto_cipher_algorithm algo)
+{
+	int len;
+
+	switch (algo) {
+	case RTE_CRYPTO_CIPHER_3DES_CBC:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_3DES_CTR:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_3DES_ECB:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_AES_CBC:
+		len = 16;
+		break;
+
+	/* TODO: add common algos */
+
+	default:
+		len = -1;
+		break;
+	}
+
+	return len;
+}
+
+/**
+ * vhost_crypto struct is used to maintain a number of virtio_cryptos and
+ * one DPDK crypto device that deals with all crypto workloads. It is declared
+ * here and defined in vhost_crypto.c
+ */
+struct vhost_crypto {
+	/** Used to lookup DPDK Cryptodev Session based on VIRTIO crypto
+	 *  session ID.
+	 */
+	struct rte_hash *session_map;
+	struct rte_mempool *mbuf_pool;
+	struct rte_mempool *sess_pool;
+
+	/** DPDK cryptodev ID */
+	uint8_t cid;
+	uint16_t nb_qps;
+
+	uint64_t last_session_id;
+
+	uint64_t cache_session_id;
+	struct rte_cryptodev_sym_session *cache_session;
+	/** socket id for the device */
+	int socket_id;
+
+	struct virtio_net *dev;
+
+	uint8_t option;
+} __rte_cache_aligned;
+
+static int
+transform_cipher_param(struct rte_crypto_sym_xform *xform,
+		VhostUserCryptoSessionParam *param)
+{
+	int ret;
+
+	ret = cipher_algo_transform(param->cipher_algo);
+	if (unlikely(ret < 0))
+		return ret;
+
+	xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform->cipher.algo = (uint32_t)ret;
+	xform->cipher.key.length = param->cipher_key_len;
+	if (xform->cipher.key.length > 0)
+		xform->cipher.key.data = param->cipher_key_buf;
+	if (param->dir == VIRTIO_CRYPTO_OP_ENCRYPT)
+		xform->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+	else if (param->dir == VIRTIO_CRYPTO_OP_DECRYPT)
+		xform->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+	else {
+		VC_LOG_DBG("Bad operation type");
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	ret = get_iv_len(xform->cipher.algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform->cipher.iv.length = (uint16_t)ret;
+	xform->cipher.iv.offset = IV_OFFSET;
+	return 0;
+}
+
+static int
+transform_chain_param(struct rte_crypto_sym_xform *xforms,
+		VhostUserCryptoSessionParam *param)
+{
+	struct rte_crypto_sym_xform *xform_cipher, *xform_auth;
+	int ret;
+
+	switch (param->chaining_dir) {
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER:
+		xform_auth = xforms;
+		xform_cipher = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+		break;
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH:
+		xform_cipher = xforms;
+		xform_auth = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		break;
+	default:
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	/* cipher */
+	ret = cipher_algo_transform(param->cipher_algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform_cipher->cipher.algo = (uint32_t)ret;
+	xform_cipher->cipher.key.length = param->cipher_key_len;
+	xform_cipher->cipher.key.data = param->cipher_key_buf;
+	ret = get_iv_len(xform_cipher->cipher.algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->cipher.iv.length = (uint16_t)ret;
+	xform_cipher->cipher.iv.offset = IV_OFFSET;
+
+	/* auth */
+	xform_auth->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+	ret = auth_algo_transform(param->hash_algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_auth->auth.algo = (uint32_t)ret;
+	xform_auth->auth.digest_length = param->digest_len;
+	xform_auth->auth.key.length = param->auth_key_len;
+	xform_auth->auth.key.data = param->auth_key_buf;
+
+	return 0;
+}
+
+static void
+vhost_crypto_create_sess(struct vhost_crypto *vcrypto,
+		VhostUserCryptoSessionParam *sess_param)
+{
+	struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0};
+	struct rte_cryptodev_sym_session *session;
+	int ret;
+
+	switch (sess_param->op_type) {
+	case VIRTIO_CRYPTO_SYM_OP_NONE:
+	case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+		ret = transform_cipher_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session msg (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+		break;
+	case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+		if (unlikely(sess_param->hash_mode !=
+				VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH)) {
+			sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+			VC_LOG_ERR("Error transform session message (%i)",
+					-VIRTIO_CRYPTO_NOTSUPP);
+			return;
+		}
+
+		xform1.next = &xform2;
+
+		ret = transform_chain_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session message (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+
+		break;
+	default:
+		VC_LOG_ERR("Algorithm not yet supported");
+		sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+		return;
+	}
+
+	session = rte_cryptodev_sym_session_create(vcrypto->sess_pool);
+	if (!session) {
+		VC_LOG_ERR("Failed to create session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1,
+			vcrypto->sess_pool) < 0) {
+		VC_LOG_ERR("Failed to initialize session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	/* insert hash to map */
+	if (rte_hash_add_key_data(vcrypto->session_map,
+			&vcrypto->last_session_id, session) < 0) {
+		VC_LOG_ERR("Failed to insert session to hash table");
+
+		if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0)
+			VC_LOG_ERR("Failed to clear session");
+		else {
+			if (rte_cryptodev_sym_session_free(session) < 0)
+				VC_LOG_ERR("Failed to free session");
+		}
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) created.",
+			vcrypto->last_session_id, session);
+
+	sess_param->session_id = vcrypto->last_session_id;
+	vcrypto->last_session_id++;
+}
+
+static int
+vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id)
+{
+	struct rte_cryptodev_sym_session *session;
+	uint64_t sess_id = session_id;
+	int ret;
+
+	ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id,
+			(void **)&session);
+
+	if (unlikely(ret < 0)) {
+		VC_LOG_ERR("Failed to delete session (key %lu).", session_id);
+		return -VIRTIO_CRYPTO_INVSESS;
+	}
+
+	if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_cryptodev_sym_session_free(session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_hash_del_key(vcrypto->session_map, &sess_id) < 0) {
+		VC_LOG_DBG("Failed to delete session from hash table.");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) deleted.", sess_id,
+			session);
+
+	return 0;
+}
+
+static int
+vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+	VhostUserMsg *vmsg = msg;
+	int ret = 0;
+
+	if (dev == NULL || require_reply == NULL) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (vcrypto == NULL) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	*require_reply = 0;
+
+	if (vmsg->request.master == VHOST_USER_CRYPTO_CREATE_SESS) {
+		vhost_crypto_create_sess(vcrypto,
+				&vmsg->payload.crypto_session);
+		*require_reply = 1;
+	} else if (vmsg->request.master == VHOST_USER_CRYPTO_CLOSE_SESS)
+		ret = vhost_crypto_close_sess(vcrypto, vmsg->payload.u64);
+	else
+		ret = -EINVAL;
+
+	return ret;
+}
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index a435cce17..3117c6330 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -67,6 +67,8 @@ static const char *vhost_message_str[VHOST_USER_MAX] = {
 	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
 	[VHOST_USER_SET_SLAVE_REQ_FD]  = "VHOST_USER_SET_SLAVE_REQ_FD",
 	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
+	[VHOST_USER_CRYPTO_CREATE_SESS] = "VHOST_USER_CRYPTO_CREATE_SESS",
+	[VHOST_USER_CRYPTO_CLOSE_SESS] = "VHOST_USER_CRYPTO_CLOSE_SESS",
 };
 
 static uint64_t
-- 
2.13.6

[PATCH v6 4/8] lib/librte_vhost: add request handler

[Fan Zhang]

This patch adds the implementation that parses virtio crypto request
to dpdk crypto operation.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h |  14 +
 lib/librte_vhost/vhost_crypto.c     | 622 ++++++++++++++++++++++++++++++++++++
 2 files changed, 636 insertions(+)
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
new file mode 100644
index 000000000..b6be9d87d
--- /dev/null
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#ifndef _VHOST_CRYPTO_H_
+#define _VHOST_CRYPTO_H_
+
+enum rte_vhost_crypto_zero_copy {
+	RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE = 0,
+	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE,
+	RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS
+};
+
+#endif /**< _VHOST_CRYPTO_H_ */
diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index 34587c27e..5608e94ad 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -8,9 +8,11 @@
 #include <rte_mbuf.h>
 #include <rte_cryptodev.h>
 
+#include "rte_vhost_crypto.h"
 #include "vhost.h"
 #include "vhost_user.h"
 
+#define INHDR_LEN		(sizeof(struct virtio_crypto_inhdr))
 #define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
 				sizeof(struct rte_crypto_sym_op))
 
@@ -33,6 +35,8 @@
 #define VC_LOG_DBG(fmt, args...)
 #endif
 
+#define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
+
 static int
 cipher_algo_transform(uint32_t virtio_cipher_algo)
 {
@@ -203,6 +207,18 @@ struct vhost_crypto {
 	uint8_t option;
 } __rte_cache_aligned;
 
+struct vhost_crypto_data_req {
+	struct vring_desc *head;
+	struct rte_vhost_memory *mem;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vhost_virtqueue *vq;
+	struct vring_desc *wb_desc;
+	uint16_t wb_len;
+	uint16_t desc_idx;
+	uint16_t len;
+	uint16_t zero_copy;
+};
+
 static int
 transform_cipher_param(struct rte_crypto_sym_xform *xform,
 		VhostUserCryptoSessionParam *param)
@@ -433,3 +449,609 @@ vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
 
 	return ret;
 }
+
+static __rte_always_inline struct vring_desc *
+find_write_desc(struct vring_desc *head, struct vring_desc *desc)
+{
+	if (desc->flags & VRING_DESC_F_WRITE)
+		return desc;
+
+	while (desc->flags & VRING_DESC_F_NEXT) {
+		desc = &head[desc->next];
+		if (desc->flags & VRING_DESC_F_WRITE)
+			return desc;
+	}
+
+	return NULL;
+}
+
+static struct virtio_crypto_inhdr *
+reach_inhdr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc *desc)
+{
+	while (desc->flags & VRING_DESC_F_NEXT)
+		desc = &head[desc->next];
+
+	return GPA_TO_VVA(struct virtio_crypto_inhdr *, mem, desc->addr);
+}
+
+static __rte_always_inline int
+move_desc(struct vring_desc *head, struct vring_desc **cur_desc,
+		uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	left -= desc->len;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		left -= desc->len;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+	return 0;
+}
+
+static int
+copy_data(void *dst_data, struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	uint32_t to_copy;
+	uint8_t *data = dst_data;
+	uint8_t *src;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	to_copy = RTE_MIN(desc->len, (uint32_t)left);
+	src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy((uint8_t *)data, src, to_copy);
+	left -= to_copy;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_copy = RTE_MIN(desc->len, (uint32_t)left);
+		src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(data + size - left, src, to_copy);
+		left -= to_copy;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+
+	return 0;
+}
+
+static __rte_always_inline void *
+get_data_ptr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	void *data;
+
+	data = GPA_TO_VVA(void *, mem, (*cur_desc)->addr);
+	if (unlikely(!data)) {
+		VC_LOG_ERR("Failed to get object");
+		return NULL;
+	}
+
+	if (unlikely(move_desc(head, cur_desc, size) < 0))
+		return NULL;
+
+	return data;
+}
+
+static int
+write_back_data(struct rte_crypto_op *op, struct vhost_crypto_data_req *vc_req)
+{
+	struct rte_mbuf *mbuf = op->sym->m_dst;
+	struct vring_desc *head = vc_req->head;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct vring_desc *desc = vc_req->wb_desc;
+	int left = vc_req->wb_len;
+	uint32_t to_write;
+	uint8_t *src_data = mbuf->buf_addr, *dst;
+
+	rte_prefetch0(&head[desc->next]);
+	to_write = RTE_MIN(desc->len, (uint32_t)left);
+	dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy(dst, src_data, to_write);
+	left -= to_write;
+	src_data += to_write;
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	printf("desc addr %llu len %u:", desc->addr, desc->len);
+	rte_hexdump(stdout, "", dst, to_write);
+#endif
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_write = RTE_MIN(desc->len, (uint32_t)left);
+		dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(dst, src_data, to_write);
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+		printf("desc addr %llu len %u:", desc->addr, desc->len);
+		rte_hexdump(stdout, "DST:", dst, to_write);
+#endif
+		left -= to_write;
+		src_data += to_write;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	return 0;
+}
+
+static uint8_t
+prepare_sym_cipher_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_cipher_data_req *cipher,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			cipher->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, cipher->para.iv_len);
+#endif
+
+	m_src->data_len = cipher->para.src_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				cipher->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 ||
+				m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		if (unlikely(cipher->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, cipher->para.src_data_len))
+				< 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			cipher->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, cipher->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		m_dst->data_len = cipher->para.dst_data_len;
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = cipher->para.dst_data_len;
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	/* src data */
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = 0;
+	op->sym->cipher.data.length = cipher->para.src_data_len;
+
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+	vc_req->len = cipher->para.dst_data_len + INHDR_LEN;
+
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+static uint8_t
+prepare_sym_chain_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_alg_chain_data_req *chain,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint32_t digest_offset;
+	void *digest_addr;
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			chain->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, chain->para.iv_len);
+#endif
+	m_src->data_len = chain->para.src_data_len;
+	m_dst->data_len = chain->para.dst_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				chain->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 || m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		if (unlikely(chain->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, chain->para.src_data_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			chain->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.phys_addr = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.hash_result_len);
+		op->sym->auth.digest.data = get_data_ptr(head, mem, &desc,
+				chain->para.hash_result_len);
+		if (unlikely(op->sym->auth.digest.phys_addr == 0)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		digest_offset = m_dst->data_len;
+		digest_addr = rte_pktmbuf_mtod_offset(m_dst, void *,
+				digest_offset);
+
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = m_dst->data_len + chain->para.hash_result_len;
+
+		if (unlikely(move_desc(head, &desc,
+				chain->para.dst_data_len) < 0)) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		if (unlikely(copy_data(digest_addr, head, mem, &desc,
+				chain->para.hash_result_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.data = digest_addr;
+		op->sym->auth.digest.phys_addr = rte_pktmbuf_iova_offset(m_dst,
+				digest_offset);
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "Digest:", op->sym->auth.digest.data,
+			chain->para.hash_result_len);
+#endif
+
+	/* record inhdr */
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = chain->para.cipher_start_src_offset;
+	op->sym->cipher.data.length = chain->para.src_data_len -
+			chain->para.cipher_start_src_offset;
+
+	op->sym->auth.data.offset = chain->para.hash_start_src_offset;
+	op->sym->auth.data.length = chain->para.len_to_hash;
+
+	vc_req->len = chain->para.dst_data_len + chain->para.hash_result_len +
+			INHDR_LEN;
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+/**
+ * Process on descriptor
+ */
+static __rte_always_inline int
+vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
+		struct vhost_virtqueue *vq, struct rte_crypto_op *op,
+		struct vring_desc *head, uint16_t desc_idx,
+		struct rte_vhost_memory *mem)
+{
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(op->sym->m_src,
+			sizeof(struct rte_mbuf));
+	struct rte_cryptodev_sym_session *session;
+	struct virtio_crypto_op_data_req *req;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vring_desc *desc = NULL;
+	uint64_t session_id;
+	int err = 0;
+
+	vc_req->desc_idx = desc_idx;
+
+	if (likely(head->flags & VRING_DESC_F_INDIRECT)) {
+		head = GPA_TO_VVA(struct vring_desc *, mem, head->addr);
+		if (unlikely(!head))
+			return 0;
+		desc_idx = 0;
+	}
+
+	desc = head;
+
+	vc_req->mem = mem;
+	vc_req->head = head;
+	vc_req->vq = vq;
+
+	vc_req->zero_copy = vcrypto->option;
+
+	req = get_data_ptr(head, mem, &desc, sizeof(*req));
+	if (unlikely(req == NULL)) {
+		err = VIRTIO_CRYPTO_ERR;
+		VC_LOG_ERR("Invalid descriptor");
+		goto error_exit;
+	}
+
+	switch (req->header.opcode) {
+	case VIRTIO_CRYPTO_CIPHER_ENCRYPT:
+	case VIRTIO_CRYPTO_CIPHER_DECRYPT:
+		session_id = req->header.session_id;
+
+		/* one branch to avoid unnecessary table lookup */
+		if (vcrypto->cache_session_id != session_id) {
+			err = rte_hash_lookup_data(vcrypto->session_map,
+					&session_id, (void **)&session);
+			if (unlikely(err < 0)) {
+				err = VIRTIO_CRYPTO_ERR;
+				VC_LOG_ERR("Failed to retrieve session id %lu",
+						session_id);
+				goto error_exit;
+			}
+
+			vcrypto->cache_session = session;
+			vcrypto->cache_session_id = session_id;
+		}
+
+		session = vcrypto->cache_session;
+
+		err = rte_crypto_op_attach_sym_session(op, session);
+		if (unlikely(err < 0)) {
+			err = VIRTIO_CRYPTO_ERR;
+			VC_LOG_ERR("Failed to attach session to op");
+			goto error_exit;
+		}
+
+		switch (req->u.sym_req.op_type) {
+		case VIRTIO_CRYPTO_SYM_OP_NONE:
+			err = VIRTIO_CRYPTO_NOTSUPP;
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+			err = prepare_sym_cipher_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.cipher, desc);
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+			err = prepare_sym_chain_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.chain, desc);
+			break;
+		}
+		if (unlikely(err != 0)) {
+			VC_LOG_ERR("Failed to process sym request");
+			goto error_exit;
+		}
+		break;
+	default:
+		VC_LOG_ERR("Unsupported symmetric crypto request type %u",
+				req->header.opcode);
+		goto error_exit;
+	}
+
+	return 0;
+
+error_exit:
+
+	inhdr = reach_inhdr(head, mem, desc);
+	if (likely(inhdr != NULL))
+		inhdr->status = (uint8_t)err;
+
+	return -1;
+}
+
+static __rte_always_inline struct vhost_virtqueue *
+vhost_crypto_finalize_one_request(struct rte_crypto_op *op,
+		struct vhost_virtqueue *old_vq)
+{
+	struct rte_mbuf *m_src = op->sym->m_src;
+	struct rte_mbuf *m_dst = op->sym->m_dst;
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(m_src,
+			sizeof(struct rte_mbuf));
+	uint16_t desc_idx;
+	int ret = 0;
+
+	if (unlikely(!vc_req)) {
+		VC_LOG_ERR("Failed to retrieve vc_req");
+		return NULL;
+	}
+
+	if (old_vq && (vc_req->vq != old_vq))
+		return vc_req->vq;
+
+	desc_idx = vc_req->desc_idx;
+
+	if (unlikely(op->status != RTE_CRYPTO_OP_STATUS_SUCCESS))
+		vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+	else {
+		if (vc_req->zero_copy == 0) {
+			ret = write_back_data(op, vc_req);
+			if (unlikely(ret != 0))
+				vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+		}
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "DST:", rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *),
+			m_dst->data_len);
+#endif
+
+	vc_req->vq->used->ring[desc_idx].id = desc_idx;
+	vc_req->vq->used->ring[desc_idx].len = vc_req->len;
+
+	rte_mempool_put(m_dst->pool, (void *)m_dst);
+	rte_mempool_put(m_src->pool, (void *)m_src);
+
+	return vc_req->vq;
+}
+
+static __rte_always_inline uint16_t
+vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfd)
+{
+	uint16_t processed = 1;
+	struct vhost_virtqueue *vq, *tmp_vq;
+
+	if (unlikely(nb_ops == 0))
+		return 0;
+
+	vq = vhost_crypto_finalize_one_request(ops[0], NULL);
+	if (unlikely(vq == NULL))
+		return 0;
+	tmp_vq = vq;
+
+	while ((processed < nb_ops)) {
+		tmp_vq = vhost_crypto_finalize_one_request(ops[processed],
+				tmp_vq);
+
+		if (unlikely(vq != tmp_vq))
+			break;
+
+		processed++;
+	}
+
+	*callfd = vq->callfd;
+
+	*(volatile uint16_t *)&vq->used->idx += processed;
+
+	return processed;
+}
-- 
2.13.6

[PATCH v6 5/8] lib/librte_vhost: add public function implementation

[Fan Zhang]

This patch adds public API implementation to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h    |  97 ++++++++++++-
 lib/librte_vhost/rte_vhost_version.map |  11 ++
 lib/librte_vhost/vhost_crypto.c        | 255 +++++++++++++++++++++++++++++++++
 3 files changed, 362 insertions(+), 1 deletion(-)

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
index b6be9d87d..acdc7f0fe 100644
--- a/lib/librte_vhost/rte_vhost_crypto.h
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -5,10 +5,105 @@
 #ifndef _VHOST_CRYPTO_H_
 #define _VHOST_CRYPTO_H_
 
+#define VHOST_CRYPTO_MBUF_POOL_SIZE		(8192)
+#define VHOST_CRYPTO_MAX_BURST_SIZE		(64)
+#define VHOST_CRYPTO_SESSION_MAP_ENTRIES	(1024) /**< Max nb sessions */
+/** max nb virtual queues in a burst for finalizing*/
+#define VIRTIO_CRYPTO_MAX_NUM_BURST_VQS		(64)
+
 enum rte_vhost_crypto_zero_copy {
 	RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE = 0,
-	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE,
+	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE = 1,
 	RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS
 };
 
+/**
+ *  Create Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param cryptodev_id
+ *  The identifier of DPDK Cryptodev, the same cryptodev_id can be assigned to
+ *  multiple Vhost-crypto devices.
+ * @param sess_pool
+ *  The pointer to the created cryptodev session pool with the private data size
+ *  matches the target DPDK Cryptodev.
+ * @param socket_id
+ *  NUMA Socket ID to allocate resources on. *
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise
+ */
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id);
+
+/**
+ *  Free the Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise.
+ */
+int
+rte_vhost_crypto_free(int vid);
+
+/**
+ *  Enable or disable zero copy feature
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param option
+ *  Flag of zero copy feature.
+ * @return
+ *  0 if completed successfully.
+ *  Negative integer if otherwise.
+ */
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option);
+
+/**
+ * Fetch a number of vring descriptors from virt-queue and translate to DPDK
+ * crypto operations. After this function is executed, the user can enqueue
+ * the processed ops to the target cryptodev.
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param qid
+ *  Virtio queue index.
+ * @param ops
+ *  The address of an array of pointers to *rte_crypto_op* structures that must
+ *  be large enough to store *nb_ops* pointers in it.
+ * @param nb_ops
+ *  The maximum number of operations to be fetched and translated.
+ * @return
+ *  The number of fetched and processed vhost crypto request operations.
+ */
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops);
+/**
+ * Finalize the dequeued crypto ops. After the translated crypto ops are
+ * dequeued from the cryptodev, this function shall be called to write the
+ * processed data back to the vring descriptor (if no-copy is turned off).
+ *
+ * @param ops
+ *  The address of an array of *rte_crypto_op* structure that was dequeued
+ *  from cryptodev.
+ * @param nb_ops
+ *  The number of operations contained in the array.
+ * @callfds
+ *  The callfd number(s) contained in this burst, this shall be an array with
+ *  no less than VIRTIO_CRYPTO_MAX_NUM_BURST_VQS elements.
+ * @nb_callfds
+ *  The number of call_fd numbers exist in the callfds.
+ * @return
+ *  The number of ops processed.
+ */
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds);
+
 #endif /**< _VHOST_CRYPTO_H_ */
diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map
index df0103129..65807488a 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -59,3 +59,14 @@ DPDK_18.02 {
 	rte_vhost_vring_call;
 
 } DPDK_17.08;
+
+DPDK_18.05 {
+	global:
+
+	rte_vhost_crypto_create;
+	rte_vhost_crypto_free;
+	rte_vhost_crypto_fetch_requests;
+	rte_vhost_crypto_finalize_requests;
+	rte_vhost_crypto_set_zero_copy;
+
+} DPDK_18.02;
diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index 5608e94ad..f00420a89 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -5,6 +5,7 @@
 
 #include <rte_malloc.h>
 #include <rte_hash.h>
+#include <rte_jhash.h>
 #include <rte_mbuf.h>
 #include <rte_cryptodev.h>
 
@@ -35,6 +36,13 @@
 #define VC_LOG_DBG(fmt, args...)
 #endif
 
+#define VIRTIO_CRYPTO_FEATURES ((1 << VIRTIO_F_NOTIFY_ON_EMPTY) |	\
+		(1 << VIRTIO_RING_F_INDIRECT_DESC) |			\
+		(1 << VIRTIO_RING_F_EVENT_IDX) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_CIPHER) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_MAC) |			\
+		(1 << VIRTIO_NET_F_CTRL_VQ))
+
 #define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
 
 static int
@@ -1055,3 +1063,250 @@ vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
 
 	return processed;
 }
+
+int
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct rte_hash_parameters params = {0};
+	struct vhost_crypto *vcrypto;
+	char name[128];
+	int ret;
+
+	if (!dev) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	ret = rte_vhost_driver_set_features(dev->ifname,
+			VIRTIO_CRYPTO_FEATURES);
+	if (ret < 0) {
+		VC_LOG_ERR("Error setting features");
+		return -1;
+	}
+
+	vcrypto = rte_zmalloc_socket(NULL, sizeof(*vcrypto),
+			RTE_CACHE_LINE_SIZE, socket_id);
+	if (!vcrypto) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	vcrypto->sess_pool = sess_pool;
+	vcrypto->cid = cryptodev_id;
+	vcrypto->cache_session_id = UINT64_MAX;
+	vcrypto->last_session_id = 1;
+	vcrypto->dev = dev;
+	vcrypto->option = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	snprintf(name, 127, "HASH_VHOST_CRYPT_%u", (uint32_t)vid);
+	params.name = name;
+	params.entries = VHOST_CRYPTO_SESSION_MAP_ENTRIES;
+	params.hash_func = rte_jhash;
+	params.key_len = sizeof(uint64_t);
+	params.socket_id = socket_id;
+	vcrypto->session_map = rte_hash_create(&params);
+	if (!vcrypto->session_map) {
+		VC_LOG_ERR("Failed to creath session map");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "MBUF_POOL_VM_%u", (uint32_t)vid);
+	vcrypto->mbuf_pool = rte_pktmbuf_pool_create(name,
+			VHOST_CRYPTO_MBUF_POOL_SIZE, 512,
+			sizeof(struct vhost_crypto_data_req),
+			RTE_MBUF_DEFAULT_DATAROOM * 2 + RTE_PKTMBUF_HEADROOM,
+			rte_socket_id());
+	if (!vcrypto->mbuf_pool) {
+		VC_LOG_ERR("Failed to creath mbuf pool");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	dev->extern_data = vcrypto;
+	dev->extern_ops.pre_msg_handle = NULL;
+	dev->extern_ops.post_msg_handle = vhost_crypto_msg_post_handler;
+
+	return 0;
+
+error_exit:
+	if (vcrypto->session_map)
+		rte_hash_free(vcrypto->session_map);
+	if (vcrypto->mbuf_pool)
+		rte_mempool_free(vcrypto->mbuf_pool);
+
+	rte_free(vcrypto);
+
+	return ret;
+}
+
+int
+rte_vhost_crypto_free(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	rte_hash_free(vcrypto->session_map);
+	rte_mempool_free(vcrypto->mbuf_pool);
+	rte_free(vcrypto);
+
+	dev->extern_data = NULL;
+	dev->extern_ops.pre_msg_handle = NULL;
+	dev->extern_ops.post_msg_handle = NULL;
+
+	return 0;
+}
+
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	if (unlikely(option < 0 || option >=
+			RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS)) {
+		VC_LOG_ERR("Invalid option %i", option);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	if (vcrypto->option == (uint8_t)option)
+		return 0;
+
+	if (!(rte_mempool_full(vcrypto->mbuf_pool))) {
+		VC_LOG_ERR("Cannot update zero copy as mempool is not full");
+		return -EINVAL;
+	}
+
+	vcrypto->option = (uint8_t)option;
+
+	return 0;
+}
+
+uint16_t
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+	struct rte_mbuf *mbufs[VHOST_CRYPTO_MAX_BURST_SIZE * 2];
+	struct virtio_net *dev = get_device(vid);
+	struct rte_vhost_memory *mem;
+	struct vhost_crypto *vcrypto;
+	struct vhost_virtqueue *vq;
+	uint16_t avail_idx;
+	uint16_t start_idx;
+	uint16_t required;
+	uint16_t count;
+	uint16_t i;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	if (unlikely(qid >= VHOST_MAX_QUEUE_PAIRS)) {
+		VC_LOG_ERR("Invalid qid %u", qid);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vq = dev->virtqueue[qid];
+	mem = dev->mem;
+
+	avail_idx = *((volatile uint16_t *)&vq->avail->idx);
+	start_idx = vq->last_used_idx;
+	count = avail_idx - start_idx;
+	count = RTE_MIN(count, VHOST_CRYPTO_MAX_BURST_SIZE);
+	count = RTE_MIN(count, nb_ops);
+
+	if (unlikely(count == 0))
+		return 0;
+
+	/* for zero copy, we need 2 empty mbufs for src and dst, otherwise
+	 * we need only 1 mbuf as src and dst
+	 */
+	required = count * 2;
+	if (unlikely(rte_mempool_get_bulk(vcrypto->mbuf_pool, (void **)mbufs,
+			required) < 0)) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	for (i = 0; i < count; i++) {
+		uint16_t used_idx = (start_idx + i) & (vq->size - 1);
+		uint16_t desc_idx = vq->avail->ring[used_idx];
+		struct vring_desc *head = &vq->desc[desc_idx];
+		struct rte_crypto_op *op = ops[i];
+
+		op->sym->m_src = mbufs[i * 2];
+		op->sym->m_dst = mbufs[i * 2 + 1];
+		op->sym->m_src->data_off = 0;
+		op->sym->m_dst->data_off = 0;
+
+		if (unlikely(vhost_crypto_process_one_req(vcrypto, vq, op, head,
+				desc_idx, mem)) < 0)
+			break;
+	}
+
+	vq->last_used_idx += i;
+
+	return i;
+}
+
+uint16_t
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds)
+{
+	struct rte_crypto_op **tmp_ops = ops;
+	uint16_t count = 0, left = nb_ops;
+	int callfd;
+	uint16_t idx = 0;
+
+	while (left) {
+		count = vhost_crypto_complete_one_vm_requests(tmp_ops, left,
+				&callfd);
+		if (unlikely(count == 0))
+			break;
+
+		tmp_ops = &tmp_ops[count];
+		left -= count;
+
+		callfds[idx++] = callfd;
+
+		if (unlikely(idx >= VIRTIO_CRYPTO_MAX_NUM_BURST_VQS)) {
+			VC_LOG_ERR("Too many vqs");
+			break;
+		}
+	}
+
+	*nb_callfds = idx;
+
+	return nb_ops - left;
+}
-- 
2.13.6

[PATCH v6 6/8] lib/librte_vhost: update makefile

[Fan Zhang]

This patch updates the Makefile of vhost library to enable vhost crypto
compiling.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/Makefile    | 6 ++++--
 lib/librte_vhost/meson.build | 8 ++++----
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 5d6c6abae..95a6a9363 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -23,8 +23,10 @@ LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev -lrte_net
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_VHOST) := fd_man.c iotlb.c socket.c vhost.c \
 					vhost_user.c virtio_net.c
+ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV), y)
+SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto.c
+endif
 
 # install includes
-SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h
-
+SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h rte_vhost_crypto.h
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_vhost/meson.build b/lib/librte_vhost/meson.build
index 9e8c0e766..36f1e2754 100644
--- a/lib/librte_vhost/meson.build
+++ b/lib/librte_vhost/meson.build
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
+# Copyright(c) 2017-2018 Intel Corporation
 
 if host_machine.system() != 'linux'
 	build = false
@@ -10,6 +10,6 @@ endif
 version = 4
 allow_experimental_apis = true
 sources = files('fd_man.c', 'iotlb.c', 'socket.c', 'vhost.c', 'vhost_user.c',
-		'virtio_net.c')
-headers = files('rte_vhost.h')
-deps += ['ethdev']
+		'virtio_net.c', 'virtio_crypto.c')
+headers = files('rte_vhost.h', 'rte_vhost_crypto.h')
+deps += ['ethdev', 'cryptodev']
-- 
2.13.6

[PATCH v6 7/8] examples/vhost_crypto: add vhost crypto sample application

[Fan Zhang]

This patch adds vhost_crypto sample application to DPDK.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 examples/vhost_crypto/Makefile    |  32 +++
 examples/vhost_crypto/main.c      | 541 ++++++++++++++++++++++++++++++++++++++
 examples/vhost_crypto/meson.build |  14 +
 3 files changed, 587 insertions(+)
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 examples/vhost_crypto/meson.build

diff --git a/examples/vhost_crypto/Makefile b/examples/vhost_crypto/Makefile
new file mode 100644
index 000000000..1bb65e8fd
--- /dev/null
+++ b/examples/vhost_crypto/Makefile
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vhost-crypto
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
diff --git a/examples/vhost_crypto/main.c b/examples/vhost_crypto/main.c
new file mode 100644
index 000000000..bc867240d
--- /dev/null
+++ b/examples/vhost_crypto/main.c
@@ -0,0 +1,541 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <stdbool.h>
+#include <assert.h>
+#include <getopt.h>
+
+#include <rte_malloc.h>
+#include <rte_cycles.h>
+#include <rte_vhost.h>
+#include <rte_cryptodev.h>
+#include <rte_vhost_crypto.h>
+
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define NB_VIRTIO_QUEUES		(1)
+#define MAX_PKT_BURST			(64)
+#define MAX_IV_LEN			(32)
+#define NB_MEMPOOL_OBJS			(8192)
+#define NB_CRYPTO_DESCRIPTORS		(4096)
+#define NB_CACHE_OBJS			(128)
+#define SESSION_MAP_ENTRIES		(1024)
+#define REFRESH_TIME_SEC		(3)
+
+#define MAX_NB_SOCKETS			(32)
+#define DEF_SOCKET_FILE			"/tmp/vhost_crypto1.socket"
+
+struct vhost_crypto_options {
+	char *socket_files[MAX_NB_SOCKETS];
+	uint32_t nb_sockets;
+	uint8_t cid;
+	uint16_t qid;
+	uint32_t zero_copy;
+	uint32_t guest_polling;
+} options;
+
+struct vhost_crypto_info {
+	int vids[MAX_NB_SOCKETS];
+	struct rte_mempool *sess_pool;
+	struct rte_mempool *cop_pool;
+	uint32_t lcore_id;
+	uint8_t cid;
+	uint32_t qid;
+	uint32_t nb_vids;
+	volatile uint32_t initialized[MAX_NB_SOCKETS];
+
+} info;
+
+#define SOCKET_FILE_KEYWORD	"socket-file"
+#define CRYPTODEV_ID_KEYWORD	"cdev-id"
+#define CRYPTODEV_QUEUE_KEYWORD	"cdev-queue-id"
+#define ZERO_COPY_KEYWORD	"zero-copy"
+#define POLLING_KEYWORD		"guest-polling"
+
+uint64_t vhost_cycles[2], last_v_cycles[2];
+uint64_t outpkt_amount;
+
+/** support *SOCKET_FILE_PATH:CRYPTODEV_ID* format */
+static int
+parse_socket_arg(char *arg)
+{
+	uint32_t nb_sockets = options.nb_sockets;
+	size_t len = strlen(arg);
+
+	if (nb_sockets >= MAX_NB_SOCKETS) {
+		RTE_LOG(ERR, USER1, "Too many socket files!\n");
+		return -ENOMEM;
+	}
+
+	options.socket_files[nb_sockets] = rte_malloc(NULL, len, 0);
+	if (!options.socket_files[nb_sockets]) {
+		RTE_LOG(ERR, USER1, "Insufficient memory\n");
+		return -ENOMEM;
+	}
+
+	rte_memcpy(options.socket_files[nb_sockets], arg, len);
+
+	options.nb_sockets++;
+
+	return 0;
+}
+
+static int
+parse_cryptodev_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if ((pm == '\0') || (end == NULL) || (*end != '\0')) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	if (pm > rte_cryptodev_count()) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.cid = (uint8_t)pm;
+
+	return 0;
+}
+
+static int
+parse_cdev_queue_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if (pm == UINT64_MAX) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev Queue ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.qid = (uint16_t)pm;
+
+	return 0;
+}
+
+static void
+vhost_crypto_usage(const char *prgname)
+{
+	printf("%s [EAL options] --\n"
+		"  --%s SOCKET-FILE-PATH\n"
+		"  --%s CRYPTODEV_ID: crypto device id\n"
+		"  --%s CDEV_QUEUE_ID: crypto device queue id\n"
+		"  --%s: zero copy\n"
+		"  --%s: guest polling\n",
+		prgname, SOCKET_FILE_KEYWORD, CRYPTODEV_ID_KEYWORD,
+		CRYPTODEV_QUEUE_KEYWORD, ZERO_COPY_KEYWORD, POLLING_KEYWORD);
+}
+
+static int
+vhost_crypto_parse_args(int argc, char **argv)
+{
+	int opt, ret;
+	char *prgname = argv[0];
+	char **argvopt;
+	int option_index;
+	struct option lgopts[] = {
+			{SOCKET_FILE_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_ID_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_QUEUE_KEYWORD, required_argument, 0, 0},
+			{ZERO_COPY_KEYWORD, no_argument, 0, 0},
+			{POLLING_KEYWORD, no_argument, 0, 0},
+			{NULL, 0, 0, 0}
+	};
+
+	options.cid = 0;
+	options.qid = 0;
+	options.nb_sockets = 0;
+	options.guest_polling = 0;
+	options.zero_copy = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, "s:",
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		case 0:
+			if (strcmp(lgopts[option_index].name,
+					SOCKET_FILE_KEYWORD) == 0) {
+				ret = parse_socket_arg(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_ID_KEYWORD) == 0) {
+				ret = parse_cryptodev_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_QUEUE_KEYWORD) == 0) {
+				ret = parse_cdev_queue_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					ZERO_COPY_KEYWORD) == 0) {
+				options.zero_copy =
+					RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE;
+			} else if (strcmp(lgopts[option_index].name,
+					POLLING_KEYWORD) == 0) {
+				options.guest_polling = 1;
+			} else {
+				vhost_crypto_usage(prgname);
+				return -EINVAL;
+			}
+			break;
+		default:
+			return -1;
+		}
+	}
+
+	if (options.nb_sockets == 0) {
+		options.socket_files[0] = strdup(DEF_SOCKET_FILE);
+		options.nb_sockets = 1;
+		RTE_LOG(INFO, USER1,
+				"VHOST-CRYPTO: use default socket file %s\n",
+				DEF_SOCKET_FILE);
+	}
+
+	return 0;
+}
+
+static int
+new_device(int vid)
+{
+	char path[PATH_MAX];
+	uint32_t idx, i;
+	int ret;
+
+	ret = rte_vhost_get_ifname(vid, path, PATH_MAX);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot find matched socket\n");
+		return ret;
+	}
+
+	for (idx = 0; idx < options.nb_sockets; idx++) {
+		if (strcmp(path, options.socket_files[idx]) == 0)
+			break;
+	}
+
+	if (idx == options.nb_sockets) {
+		RTE_LOG(ERR, USER1, "Cannot find recorded socket\n");
+		return -ENOENT;
+	}
+
+	for (i = 0; i < 2; i++) {
+		vhost_cycles[i] = 0;
+		last_v_cycles[i] = 0;
+	}
+
+	ret = rte_vhost_crypto_create(vid, info.cid, info.sess_pool,
+			rte_lcore_to_socket_id(info.lcore_id));
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot create vhost crypto\n");
+		return ret;
+	}
+
+	ret = rte_vhost_crypto_set_zero_copy(vid, options.zero_copy);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot %s zero copy feature\n",
+				options.zero_copy == 1 ? "enable" : "disable");
+		return ret;
+	}
+
+	info.vids[idx] = vid;
+	info.initialized[idx] = 1;
+
+	rte_wmb();
+
+	RTE_LOG(INFO, USER1, "New Vhost-crypto Device %s, Device ID %d\n", path,
+			vid);
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	uint32_t i;
+
+	for (i = 0; i < info.nb_vids; i++) {
+		if (vid == info.vids[i])
+			break;
+	}
+
+	if (i == info.nb_vids) {
+		RTE_LOG(ERR, USER1, "Cannot find socket file from list\n");
+		return;
+	}
+
+	info.initialized[i] = 0;
+
+	rte_wmb();
+
+	rte_vhost_crypto_free(vid);
+
+	RTE_LOG(INFO, USER1, "Vhost Crypto Device %i Removed\n", vid);
+}
+
+static const struct vhost_device_ops virtio_crypto_device_ops = {
+	.new_device =  new_device,
+	.destroy_device = destroy_device,
+};
+
+__attribute__((unused))
+static void clrscr(void)
+{
+	system("@cls||clear");
+}
+
+static int
+vhost_crypto_worker(__rte_unused void *arg)
+{
+	struct rte_crypto_op *ops[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	struct rte_crypto_op *ops_deq[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	uint32_t nb_inflight_ops = 0;
+	uint16_t nb_callfds;
+	int callfds[VIRTIO_CRYPTO_MAX_NUM_BURST_VQS];
+	uint32_t lcore_id = rte_lcore_id();
+	uint32_t burst_size = MAX_PKT_BURST;
+	uint32_t i, j, k;
+	uint32_t to_fetch, fetched;
+	uint64_t t_start, t_end, interval;
+
+	int ret = 0;
+
+	RTE_LOG(INFO, USER1, "Processing on Core %u started\n", lcore_id);
+
+	for (i = 0; i < NB_VIRTIO_QUEUES; i++) {
+		if (rte_crypto_op_bulk_alloc(info.cop_pool,
+				RTE_CRYPTO_OP_TYPE_SYMMETRIC, ops[i],
+				burst_size) < burst_size) {
+			RTE_LOG(ERR, USER1, "Failed to alloc cops\n");
+			ret = -1;
+			goto exit;
+		}
+	}
+
+	while (1) {
+		for (i = 0; i < info.nb_vids; i++) {
+			if (unlikely(info.initialized[i] == 0))
+				continue;
+
+			for (j = 0; j < NB_VIRTIO_QUEUES; j++) {
+				t_start = rte_rdtsc_precise();
+
+				to_fetch = RTE_MIN(burst_size,
+						(NB_CRYPTO_DESCRIPTORS -
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_fetch_requests(
+						info.vids[i], j, ops[j],
+						to_fetch);
+				nb_inflight_ops += rte_cryptodev_enqueue_burst(
+						info.cid, info.qid, ops[j],
+						fetched);
+				if (unlikely(rte_crypto_op_bulk_alloc(
+						info.cop_pool,
+						RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+						ops[j], fetched) < fetched)) {
+					RTE_LOG(ERR, USER1, "Failed realloc\n");
+					return -1;
+				}
+				t_end = rte_rdtsc_precise();
+				interval = t_end - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+
+				t_start = t_end;
+				fetched = rte_cryptodev_dequeue_burst(
+						info.cid, info.qid,
+						ops_deq[j], RTE_MIN(burst_size,
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_finalize_requests(
+						ops_deq[j], fetched, callfds,
+						&nb_callfds);
+
+				nb_inflight_ops -= fetched;
+				outpkt_amount += fetched;
+
+				if (!options.guest_polling) {
+					for (k = 0; k < nb_callfds; k++)
+						eventfd_write(callfds[k],
+								(eventfd_t)1);
+				}
+
+				rte_mempool_put_bulk(info.cop_pool,
+						(void **)ops_deq[j], fetched);
+				interval = rte_rdtsc_precise() - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+			}
+		}
+	}
+exit:
+	return ret;
+}
+
+
+static void
+unregister_drivers(int socket_num)
+{
+	int ret;
+
+	ret = rte_vhost_driver_unregister(options.socket_files[socket_num]);
+	if (ret != 0)
+		RTE_LOG(ERR, USER1,
+			"Fail to unregister vhost driver for %s.\n",
+			options.socket_files[socket_num]);
+}
+
+int
+main(int argc, char *argv[])
+{
+	struct rte_cryptodev_qp_conf qp_conf = {NB_CRYPTO_DESCRIPTORS};
+	struct rte_cryptodev_config config;
+	struct rte_cryptodev_info dev_info;
+	uint32_t cryptodev_id;
+	uint32_t worker_lcore;
+	char name[128];
+	uint32_t i = 0;
+	int ret;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		return -1;
+	argc -= ret;
+	argv += ret;
+
+	ret = vhost_crypto_parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Failed to parse arguments!\n");
+
+	info.cid = options.cid;
+	info.qid = options.qid;
+
+	worker_lcore = rte_get_next_lcore(0, 1, 0);
+	if (worker_lcore == RTE_MAX_LCORE)
+		rte_exit(EXIT_FAILURE, "Not enough lcore\n");
+
+	cryptodev_id = info.cid;
+	rte_cryptodev_info_get(cryptodev_id, &dev_info);
+	if (dev_info.max_nb_queue_pairs < info.qid + 1) {
+		RTE_LOG(ERR, USER1, "Number of queues cannot over %u",
+				dev_info.max_nb_queue_pairs);
+		goto error_exit;
+	}
+
+	config.nb_queue_pairs = dev_info.max_nb_queue_pairs;
+	config.socket_id = rte_lcore_to_socket_id(worker_lcore);
+
+	ret = rte_cryptodev_configure(cryptodev_id, &config);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to configure cryptodev %u",
+				cryptodev_id);
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "SESS_POOL_%u", worker_lcore);
+	info.sess_pool = rte_mempool_create(name, SESSION_MAP_ENTRIES,
+			rte_cryptodev_get_private_session_size(
+			cryptodev_id), 64, 0, NULL, NULL, NULL, NULL,
+			rte_lcore_to_socket_id(worker_lcore), 0);
+	if (!info.sess_pool) {
+		RTE_LOG(ERR, USER1, "Failed to create mempool");
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "COPPOOL_%u", worker_lcore);
+	info.cop_pool = rte_crypto_op_pool_create(name,
+			RTE_CRYPTO_OP_TYPE_SYMMETRIC, NB_MEMPOOL_OBJS,
+			NB_CACHE_OBJS, 0, rte_lcore_to_socket_id(worker_lcore));
+
+	if (!info.cop_pool) {
+		RTE_LOG(ERR, USER1, "Lcore %u failed to create crypto pool",
+				worker_lcore);
+		ret = -1;
+		goto error_exit;
+	}
+
+	info.nb_vids = options.nb_sockets;
+	for (i = 0; i < MAX_NB_SOCKETS; i++)
+		info.vids[i] = -1;
+
+	for (i = 0; i < dev_info.max_nb_queue_pairs; i++) {
+		ret = rte_cryptodev_queue_pair_setup(cryptodev_id, i,
+				&qp_conf, rte_lcore_to_socket_id(worker_lcore),
+				info.sess_pool);
+		if (ret < 0) {
+			RTE_LOG(ERR, USER1, "Failed to configure qp %u\n",
+					info.cid);
+			goto error_exit;
+		}
+	}
+
+	ret = rte_cryptodev_start(cryptodev_id);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to start cryptodev %u\n", info.cid);
+		goto error_exit;
+	}
+
+	info.cid = cryptodev_id;
+	info.lcore_id = worker_lcore;
+
+	if (rte_eal_remote_launch(vhost_crypto_worker, NULL, worker_lcore)
+			< 0) {
+		RTE_LOG(ERR, USER1, "Failed to start worker lcore");
+		goto error_exit;
+	}
+
+	for (i = 0; i < options.nb_sockets; i++) {
+		if (rte_vhost_driver_register(options.socket_files[i],
+				RTE_VHOST_USER_DEQUEUE_ZERO_COPY) < 0) {
+			RTE_LOG(ERR, USER1, "socket %s already exists\n",
+					options.socket_files[i]);
+			goto error_exit;
+		}
+
+		rte_vhost_driver_callback_register(options.socket_files[i],
+				&virtio_crypto_device_ops);
+
+		if (rte_vhost_driver_start(options.socket_files[i]) < 0) {
+			RTE_LOG(ERR, USER1, "failed to start vhost driver.\n");
+			goto error_exit;
+		}
+	}
+
+	RTE_LCORE_FOREACH(worker_lcore)
+		rte_eal_wait_lcore(worker_lcore);
+
+	rte_mempool_free(info.sess_pool);
+	rte_mempool_free(info.cop_pool);
+
+	return 0;
+
+error_exit:
+	for (i = 0; i < options.nb_sockets; i++)
+		unregister_drivers(i);
+
+	rte_mempool_free(info.cop_pool);
+	rte_mempool_free(info.sess_pool);
+
+	return -1;
+}
diff --git a/examples/vhost_crypto/meson.build b/examples/vhost_crypto/meson.build
new file mode 100644
index 000000000..0f4876f06
--- /dev/null
+++ b/examples/vhost_crypto/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+allow_experimental_apis = true
+deps += ['vhost', 'cryptodev']
+cflags += ['-D_GNU_SOURCE','-D_FILE_OFFSET_BITS=64']
+sources = files(
+	'main.c'
+)
-- 
2.13.6

[PATCH v6 8/8] doc: update for vhost crypto support

[Fan Zhang]

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst       | 25 ++++++++++
 doc/guides/rel_notes/release_18_05.rst    |  5 ++
 doc/guides/sample_app_ug/index.rst        |  1 +
 doc/guides/sample_app_ug/vhost_crypto.rst | 82 +++++++++++++++++++++++++++++++
 4 files changed, 113 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index f47473609..04ab1eeb4 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -160,6 +160,31 @@ The following is an overview of some key Vhost API functions:
 
   Receives (dequeues) ``count`` packets from guest, and stored them at ``pkts``.
 
+* ``rte_vhost_crypto_create(vid, cryptodev_id, sess_mempool, socket_id)``
+
+  As an extension of new_device(), this function adds virtio-crypto workload
+  acceleration capability to the device. All crypto workload is processed by
+  DPDK cryptodev with the device ID of ``cryptodev_id``.
+
+* ``rte_vhost_crypto_free(vid)``
+
+  Frees the memory and vhost-user message handlers created in
+  rte_vhost_crypto_create().
+
+* ``rte_vhost_crypto_fetch_requests(vid, queue_id, ops, nb_ops)``
+
+  Receives (dequeues) ``nb_ops`` virtio-crypto requests from guest, parses
+  them to DPDK Crypto Operations, and fills the ``ops`` with parsing results.
+
+* ``rte_vhost_crypto_finalize_requests(queue_id, ops, nb_ops)``
+
+  After the ``ops`` are dequeued from Cryptodev, finalizes the jobs and
+  notifies the guest(s).
+
+* ``rte_vhost_crypto_set_zero_copy(vid, option)``
+
+  Enable or disable zero copy feature of the vhost crypto backend.
+
 Vhost-user Implementations
 --------------------------
 
diff --git a/doc/guides/rel_notes/release_18_05.rst b/doc/guides/rel_notes/release_18_05.rst
index 9cc77f893..6451f8ea0 100644
--- a/doc/guides/rel_notes/release_18_05.rst
+++ b/doc/guides/rel_notes/release_18_05.rst
@@ -58,6 +58,11 @@ New Features
   * Added support for NVGRE, VXLAN and GENEVE filters in flow API.
   * Added support for DROP action in flow API.
 
+* **Added crypto workload support to vhost library
+
+  New APIs are introduced in vhost library to enable virtio crypto support
+  including session creation/deletion handling and translating virito-crypto
+  request into DPDK crypto operations. A sample application is also introduced.
 
 API Changes
 -----------
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index e87afda80..57e8354d4 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -44,6 +44,7 @@ Sample Applications User Guides
     vmdq_dcb_forwarding
     vhost
     vhost_scsi
+    vhost_crypto
     netmap_compatibility
     ip_pipeline
     test_pipeline
diff --git a/doc/guides/sample_app_ug/vhost_crypto.rst b/doc/guides/sample_app_ug/vhost_crypto.rst
new file mode 100644
index 000000000..65c86a534
--- /dev/null
+++ b/doc/guides/sample_app_ug/vhost_crypto.rst
@@ -0,0 +1,82 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2017-2018 Intel Corporation.
+
+Vhost_Crypto Sample Application
+===============================
+
+The vhost_crypto sample application implemented a simple Crypto device,
+which used as the  backend of Qemu vhost-user-crypto device. Similar with
+vhost-user-net and vhost-user-scsi device, the sample application used
+domain socket to communicate with Qemu, and the virtio ring was processed
+by vhost_crypto sample application.
+
+Testing steps
+-------------
+
+This section shows the steps how to start a VM with the crypto device as
+fast data path for critical application.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``examples`` sub-directory.
+
+Start the vhost_crypto example
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+    ./vhost_crypto [EAL options] -- [--socket-file PATH]
+        [--cdev-id ID] [--cdev-queue-id ID] [--zero-copy] [--guest-polling]
+
+where,
+
+* socket-file PATH: the path of UNIX socket file to be created, multiple
+  instances of this config item is supported. Upon absence of this item,
+  the default socket-file `/tmp/vhost_crypto1.socket` is used.
+
+* cdev-id ID: the target DPDK Cryptodev's ID to process the actual crypto
+  workload. Upon absence of this item the default value of `0` will be used.
+  For details of DPDK Cryptodev, please refer to DPDK Cryptodev Library
+  Programmers' Guide.
+
+* cdev-queue-id ID: the target DPDK Cryptodev's queue ID to process the
+  actual crypto workload. Upon absence of this item the default value of `0`
+  will be used. For details of DPDK Cryptodev, please refer to DPDK Cryptodev
+  Library Programmers' Guide.
+
+* zero-copy: the presence of this item means the ZERO-COPY feature will be
+  enabled. Otherwise it is disabled. PLEASE NOTE the ZERO-COPY feature is still
+  in experimental stage and may cause the problem like segmentation fault. If
+  the user wants to use LKCF in the guest, this feature shall be turned off.
+
+* guest-polling: the presence of this item means the application assumes the
+  guest works in polling mode, thus will NOT notify the guest completion of
+  processing.
+
+The application requires that crypto devices capable of performing
+the specified crypto operation are available on application initialization.
+This means that HW crypto device/s must be bound to a DPDK driver or
+a SW crypto device/s (virtual crypto PMD) must be created (using --vdev).
+
+.. _vhost_crypto_app_run_vm:
+
+Start the VM
+~~~~~~~~~~~~
+
+.. code-block:: console
+
+    qemu-system-x86_64 -machine accel=kvm \
+        -m $mem -object memory-backend-file,id=mem,size=$mem,\
+        mem-path=/dev/hugepages,share=on -numa node,memdev=mem \
+        -drive file=os.img,if=none,id=disk \
+        -device ide-hd,drive=disk,bootindex=0 \
+        -chardev socket,id={chardev_id},path={PATH} \
+        -object cryptodev-vhost-user,id={obj_id},chardev={chardev_id} \
+        -device virtio-crypto-pci,id={dev_id},cryptodev={obj_id} \
+        ...
+
+.. note::
+    You must check whether your Qemu can support "vhost-user-crypto" or not.
-- 
2.13.6

Re: [PATCH v6 0/8] vhost: introduce vhost crypto backend

[Maxime Coquelin]

Hi Fan,

On 04/04/2018 04:24 PM, Fan Zhang wrote:
> This patchset adds crypto backend suppport to vhost library
> including a proof-of-concept sample application. The implementation
> follows the virtio-crypto specification and have been tested
> with qemu 2.11.50 (with several patches applied, detailed later)
> with Fedora 24 running in the frontend.
> 
> The vhost_crypto library acts as a "bridge" method that translate
> the virtio-crypto crypto requests to DPDK crypto operations, so it
> is purely software implementation. However it does require the user
> to provide the DPDK Cryptodev ID so it knows how to handle the
> virtio-crypto session creation and deletion mesages.
> 
> Currently the implementation supports AES-CBC-128 and HMAC-SHA1
> cipher only/chaining modes and does not support sessionless mode
> yet. The guest can use standard virtio-crypto driver to set up
> session and sends encryption/decryption requests to backend. The
> vhost-crypto sample application provided in this patchset will
> do the actual crypto work.
> 
> The following steps are involved to enable vhost-crypto support.
> 
> In the host:
> 1. Download the qemu source code.
> 
> 2. Recompile your qemu with vhost-crypto option enabled.
> 
> 3. Apply this patchset to latest DPDK code and recompile DPDK.
> 
> 4. Compile and run vhost-crypto sample application.
> 
> ./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
>   --socket-mem 2048,2048
> 
> Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
> not available. The sample application requires 2 lcores: 1 master and 1
> worker. The application will create a UNIX socket file
> /tmp/vhost_crypto1.socket.
> 
> 5. Start your qemu application. Here is my command:
> 
> qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
> -smp 2 -m 1G -hda ~/path-to-your/image.qcow \
> -object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
> -mem-prealloc -numa node,memdev=mem -chardev \
> socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
> -object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
> -device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0
> 
> 6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
> default. You shall see the following logs in your demsg:
> 
> [   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
> [   17.611083] virtio_crypto: module verification failed: signature and/or ...
> [   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
> [   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
> [   18.376100] virtio_crypto virtio0: Accelerator is ready
> 
> The virtio_crypto driver in the guest is now up and running.
> 
> 7. The rest steps can be as same as the Testing section in
> https://wiki.qemu.org/Features/VirtioCrypto
> 
> 8. It is possible to use DPDK Virtio Crypto PMD
> (https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work with
> this patchset to achieve optimal performance.
> 
> v6:
> - Changed commit message
> - removed rte prefix in handler prototype
> 
> v5:
> - removed external ops register API.
> - patch cleaned.
> 
> v4:
> - Changed external vhost backend ops register API.
> - Fixed a bug.
> 
> v3:
> - Changed external vhost backend private data and message handling
> - Added experimental tag to rte_vhost_crypto_set_zero_copy()
> 
> v2:
> - Moved vhost_crypto_data_req data from crypto op to source mbuf.
> - Removed ZERO-COPY flag from config option and make it run-timely changeable.
> - Guest-polling mode possible.
> - Simplified vring descriptor access procedure.
> - Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.
> 
> Fan Zhang (8):
>    lib/librte_vhost: add vhost user message handlers
>    lib/librte_vhost: add virtio-crypto user message structure
>    lib/librte_vhost: add session message handler
>    lib/librte_vhost: add request handler
>    lib/librte_vhost: add public function implementation
>    lib/librte_vhost: update makefile
>    examples/vhost_crypto: add vhost crypto sample application
>    doc: update for vhost crypto support
> 
>   doc/guides/prog_guide/vhost_lib.rst       |   25 +
>   doc/guides/rel_notes/release_18_05.rst    |    5 +
>   doc/guides/sample_app_ug/index.rst        |    1 +
>   doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
>   examples/vhost_crypto/Makefile            |   32 +
>   examples/vhost_crypto/main.c              |  541 ++++++++++++
>   examples/vhost_crypto/meson.build         |   14 +
>   lib/librte_vhost/Makefile                 |    6 +-
>   lib/librte_vhost/meson.build              |    8 +-
>   lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
>   lib/librte_vhost/rte_vhost_version.map    |   11 +
>   lib/librte_vhost/vhost.c                  |    2 +-
>   lib/librte_vhost/vhost.h                  |   53 +-
>   lib/librte_vhost/vhost_crypto.c           | 1312 +++++++++++++++++++++++++++++
>   lib/librte_vhost/vhost_user.c             |   33 +-
>   lib/librte_vhost/vhost_user.h             |   35 +-
>   16 files changed, 2256 insertions(+), 13 deletions(-)
>   create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
>   create mode 100644 examples/vhost_crypto/Makefile
>   create mode 100644 examples/vhost_crypto/main.c
>   create mode 100644 examples/vhost_crypto/meson.build
>   create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
>   create mode 100644 lib/librte_vhost/vhost_crypto.c
> 

For the series:
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Ideally, I would like to have it reviewed/acked by people having
knowledge of crypto.
Jianjay, is the series good for you?

Thanks,
Maxime

Re: [PATCH v6 8/8] doc: update for vhost crypto support

[Zhoujian (jay)]

Hi Fan,

> -----Original Message-----
> From: Fan Zhang [mailto:roy.fan.zhang@intel.com]
> Sent: Wednesday, April 04, 2018 10:25 PM
> To: dev@dpdk.org
> Cc: Zhoujian (jay) <jianjay.zhou@huawei.com>; roy.fan.zhang@intel.com;
> maxime.coquelin@redhat.com; jianfeng.tan@intel.com;
> pawelx.wodkowski@intel.com
> Subject: [PATCH v6 8/8] doc: update for vhost crypto support
> 
> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> ---
>  doc/guides/prog_guide/vhost_lib.rst       | 25 ++++++++++
>  doc/guides/rel_notes/release_18_05.rst    |  5 ++
>  doc/guides/sample_app_ug/index.rst        |  1 +
>  doc/guides/sample_app_ug/vhost_crypto.rst | 82
> +++++++++++++++++++++++++++++++
>  4 files changed, 113 insertions(+)
>  create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
> 
> diff --git a/doc/guides/prog_guide/vhost_lib.rst
> b/doc/guides/prog_guide/vhost_lib.rst
> index f47473609..04ab1eeb4 100644
> --- a/doc/guides/prog_guide/vhost_lib.rst
> +++ b/doc/guides/prog_guide/vhost_lib.rst
> @@ -160,6 +160,31 @@ The following is an overview of some key Vhost API
> functions:
> 
>    Receives (dequeues) ``count`` packets from guest, and stored them at
> ``pkts``.
> 
> +* ``rte_vhost_crypto_create(vid, cryptodev_id, sess_mempool,
> +socket_id)``
> +
> +  As an extension of new_device(), this function adds virtio-crypto
> + workload  acceleration capability to the device. All crypto workload
> + is processed by  DPDK cryptodev with the device ID of ``cryptodev_id``.
> +
> +* ``rte_vhost_crypto_free(vid)``
> +
> +  Frees the memory and vhost-user message handlers created in
> + rte_vhost_crypto_create().
> +
> +* ``rte_vhost_crypto_fetch_requests(vid, queue_id, ops, nb_ops)``
> +
> +  Receives (dequeues) ``nb_ops`` virtio-crypto requests from guest,
> + parses  them to DPDK Crypto Operations, and fills the ``ops`` with parsing
> results.
> +
> +* ``rte_vhost_crypto_finalize_requests(queue_id, ops, nb_ops)``
> +
> +  After the ``ops`` are dequeued from Cryptodev, finalizes the jobs and
> + notifies the guest(s).
> +
> +* ``rte_vhost_crypto_set_zero_copy(vid, option)``
> +
> +  Enable or disable zero copy feature of the vhost crypto backend.
> +
>  Vhost-user Implementations
>  --------------------------
> 
> diff --git a/doc/guides/rel_notes/release_18_05.rst
> b/doc/guides/rel_notes/release_18_05.rst
> index 9cc77f893..6451f8ea0 100644
> --- a/doc/guides/rel_notes/release_18_05.rst
> +++ b/doc/guides/rel_notes/release_18_05.rst
> @@ -58,6 +58,11 @@ New Features
>    * Added support for NVGRE, VXLAN and GENEVE filters in flow API.
>    * Added support for DROP action in flow API.
> 
> +* **Added crypto workload support to vhost library

Please add the "**" end-string at the end of the line above.

Regards,
Jay

> +
> +  New APIs are introduced in vhost library to enable virtio crypto
> + support  including session creation/deletion handling and translating
> + virito-crypto  request into DPDK crypto operations. A sample application is
> also introduced.
> 
>  API Changes
>  -----------
> diff --git a/doc/guides/sample_app_ug/index.rst
> b/doc/guides/sample_app_ug/index.rst
> index e87afda80..57e8354d4 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -44,6 +44,7 @@ Sample Applications User Guides
>      vmdq_dcb_forwarding
>      vhost
>      vhost_scsi
> +    vhost_crypto
>      netmap_compatibility
>      ip_pipeline
>      test_pipeline
> diff --git a/doc/guides/sample_app_ug/vhost_crypto.rst
> b/doc/guides/sample_app_ug/vhost_crypto.rst
> new file mode 100644
> index 000000000..65c86a534
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/vhost_crypto.rst
> @@ -0,0 +1,82 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2017-2018 Intel Corporation.
> +
> +Vhost_Crypto Sample Application
> +===============================
> +
> +The vhost_crypto sample application implemented a simple Crypto device,
> +which used as the  backend of Qemu vhost-user-crypto device. Similar
> +with vhost-user-net and vhost-user-scsi device, the sample application
> +used domain socket to communicate with Qemu, and the virtio ring was
> +processed by vhost_crypto sample application.
> +
> +Testing steps
> +-------------
> +
> +This section shows the steps how to start a VM with the crypto device
> +as fast data path for critical application.
> +
> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``examples`` sub-directory.
> +
> +Start the vhost_crypto example
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +.. code-block:: console
> +
> +    ./vhost_crypto [EAL options] -- [--socket-file PATH]
> +        [--cdev-id ID] [--cdev-queue-id ID] [--zero-copy]
> + [--guest-polling]
> +
> +where,
> +
> +* socket-file PATH: the path of UNIX socket file to be created,
> +multiple
> +  instances of this config item is supported. Upon absence of this
> +item,
> +  the default socket-file `/tmp/vhost_crypto1.socket` is used.
> +
> +* cdev-id ID: the target DPDK Cryptodev's ID to process the actual
> +crypto
> +  workload. Upon absence of this item the default value of `0` will be used.
> +  For details of DPDK Cryptodev, please refer to DPDK Cryptodev Library
> +  Programmers' Guide.
> +
> +* cdev-queue-id ID: the target DPDK Cryptodev's queue ID to process the
> +  actual crypto workload. Upon absence of this item the default value
> +of `0`
> +  will be used. For details of DPDK Cryptodev, please refer to DPDK
> +Cryptodev
> +  Library Programmers' Guide.
> +
> +* zero-copy: the presence of this item means the ZERO-COPY feature will
> +be
> +  enabled. Otherwise it is disabled. PLEASE NOTE the ZERO-COPY feature
> +is still
> +  in experimental stage and may cause the problem like segmentation
> +fault. If
> +  the user wants to use LKCF in the guest, this feature shall be turned off.
> +
> +* guest-polling: the presence of this item means the application
> +assumes the
> +  guest works in polling mode, thus will NOT notify the guest
> +completion of
> +  processing.
> +
> +The application requires that crypto devices capable of performing the
> +specified crypto operation are available on application initialization.
> +This means that HW crypto device/s must be bound to a DPDK driver or a
> +SW crypto device/s (virtual crypto PMD) must be created (using --vdev).
> +
> +.. _vhost_crypto_app_run_vm:
> +
> +Start the VM
> +~~~~~~~~~~~~
> +
> +.. code-block:: console
> +
> +    qemu-system-x86_64 -machine accel=kvm \
> +        -m $mem -object memory-backend-file,id=mem,size=$mem,\
> +        mem-path=/dev/hugepages,share=on -numa node,memdev=mem \
> +        -drive file=os.img,if=none,id=disk \
> +        -device ide-hd,drive=disk,bootindex=0 \
> +        -chardev socket,id={chardev_id},path={PATH} \
> +        -object cryptodev-vhost-user,id={obj_id},chardev={chardev_id} \
> +        -device virtio-crypto-pci,id={dev_id},cryptodev={obj_id} \
> +        ...
> +
> +.. note::
> +    You must check whether your Qemu can support "vhost-user-crypto" or not.
> --
> 2.13.6

Re: [PATCH v6 0/8] vhost: introduce vhost crypto backend

[Zhoujian (jay)]

Hi Maxime,

> -----Original Message-----
> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> Sent: Wednesday, April 04, 2018 11:37 PM
> To: Fan Zhang <roy.fan.zhang@intel.com>; dev@dpdk.org; Zhoujian (jay)
> <jianjay.zhou@huawei.com>
> Cc: jianfeng.tan@intel.com; pawelx.wodkowski@intel.com
> Subject: Re: [PATCH v6 0/8] vhost: introduce vhost crypto backend
> 
> Hi Fan,
> 
> On 04/04/2018 04:24 PM, Fan Zhang wrote:
> > This patchset adds crypto backend suppport to vhost library including
> > a proof-of-concept sample application. The implementation follows the
> > virtio-crypto specification and have been tested with qemu 2.11.50
> > (with several patches applied, detailed later) with Fedora 24 running
> > in the frontend.
> >
> > The vhost_crypto library acts as a "bridge" method that translate the
> > virtio-crypto crypto requests to DPDK crypto operations, so it is
> > purely software implementation. However it does require the user to
> > provide the DPDK Cryptodev ID so it knows how to handle the
> > virtio-crypto session creation and deletion mesages.
> >
> > Currently the implementation supports AES-CBC-128 and HMAC-SHA1 cipher
> > only/chaining modes and does not support sessionless mode yet. The
> > guest can use standard virtio-crypto driver to set up session and
> > sends encryption/decryption requests to backend. The vhost-crypto
> > sample application provided in this patchset will do the actual crypto
> > work.
> >
> > The following steps are involved to enable vhost-crypto support.
> >
> > In the host:
> > 1. Download the qemu source code.
> >
> > 2. Recompile your qemu with vhost-crypto option enabled.
> >
> > 3. Apply this patchset to latest DPDK code and recompile DPDK.
> >
> > 4. Compile and run vhost-crypto sample application.
> >
> > ./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
> >   --socket-mem 2048,2048
> >
> > Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it
> > is not available. The sample application requires 2 lcores: 1 master
> > and 1 worker. The application will create a UNIX socket file
> > /tmp/vhost_crypto1.socket.
> >
> > 5. Start your qemu application. Here is my command:
> >
> > qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
> > -smp 2 -m 1G -hda ~/path-to-your/image.qcow \ -object
> > memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
> > -mem-prealloc -numa node,memdev=mem -chardev \
> > socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \ -object
> > cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \ -device
> > virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0
> >
> > 6. Once guest is booted. The Linux virtio_crypto kernel module is
> > loaded by default. You shall see the following logs in your demsg:
> >
> > [   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
> > [   17.611083] virtio_crypto: module verification failed: signature
> and/or ...
> > [   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
> > [   17.612156] virtio_crypto virtio0: will run requests pump with
> realtime ...
> > [   18.376100] virtio_crypto virtio0: Accelerator is ready
> >
> > The virtio_crypto driver in the guest is now up and running.
> >
> > 7. The rest steps can be as same as the Testing section in
> > https://wiki.qemu.org/Features/VirtioCrypto
> >
> > 8. It is possible to use DPDK Virtio Crypto PMD
> > (https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work
> > with this patchset to achieve optimal performance.
> >
> > v6:
> > - Changed commit message
> > - removed rte prefix in handler prototype
> >
> > v5:
> > - removed external ops register API.
> > - patch cleaned.
> >
> > v4:
> > - Changed external vhost backend ops register API.
> > - Fixed a bug.
> >
> > v3:
> > - Changed external vhost backend private data and message handling
> > - Added experimental tag to rte_vhost_crypto_set_zero_copy()
> >
> > v2:
> > - Moved vhost_crypto_data_req data from crypto op to source mbuf.
> > - Removed ZERO-COPY flag from config option and make it run-timely
> changeable.
> > - Guest-polling mode possible.
> > - Simplified vring descriptor access procedure.
> > - Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.
> >
> > Fan Zhang (8):
> >    lib/librte_vhost: add vhost user message handlers
> >    lib/librte_vhost: add virtio-crypto user message structure
> >    lib/librte_vhost: add session message handler
> >    lib/librte_vhost: add request handler
> >    lib/librte_vhost: add public function implementation
> >    lib/librte_vhost: update makefile
> >    examples/vhost_crypto: add vhost crypto sample application
> >    doc: update for vhost crypto support
> >
> >   doc/guides/prog_guide/vhost_lib.rst       |   25 +
> >   doc/guides/rel_notes/release_18_05.rst    |    5 +
> >   doc/guides/sample_app_ug/index.rst        |    1 +
> >   doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
> >   examples/vhost_crypto/Makefile            |   32 +
> >   examples/vhost_crypto/main.c              |  541 ++++++++++++
> >   examples/vhost_crypto/meson.build         |   14 +
> >   lib/librte_vhost/Makefile                 |    6 +-
> >   lib/librte_vhost/meson.build              |    8 +-
> >   lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
> >   lib/librte_vhost/rte_vhost_version.map    |   11 +
> >   lib/librte_vhost/vhost.c                  |    2 +-
> >   lib/librte_vhost/vhost.h                  |   53 +-
> >   lib/librte_vhost/vhost_crypto.c           | 1312
> +++++++++++++++++++++++++++++
> >   lib/librte_vhost/vhost_user.c             |   33 +-
> >   lib/librte_vhost/vhost_user.h             |   35 +-
> >   16 files changed, 2256 insertions(+), 13 deletions(-)
> >   create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
> >   create mode 100644 examples/vhost_crypto/Makefile
> >   create mode 100644 examples/vhost_crypto/main.c
> >   create mode 100644 examples/vhost_crypto/meson.build
> >   create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
> >   create mode 100644 lib/librte_vhost/vhost_crypto.c
> >
> 
> For the series:
> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> 
> Ideally, I would like to have it reviewed/acked by people having knowledge of
> crypto.
> Jianjay, is the series good for you?

With the typo of 8/8 fixed, this series looks good to me, so:
Acked-by: Jay Zhou <jianjay.zhou@huawei.com>

> 
> Thanks,
> Maxime

Re: [PATCH v6 0/8] vhost: introduce vhost crypto backend

[Maxime Coquelin]

On 04/04/2018 06:50 PM, Zhoujian (jay) wrote:
> Hi Maxime,
> 
>> -----Original Message-----
>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
>> Sent: Wednesday, April 04, 2018 11:37 PM
>> To: Fan Zhang <roy.fan.zhang@intel.com>; dev@dpdk.org; Zhoujian (jay)
>> <jianjay.zhou@huawei.com>
>> Cc: jianfeng.tan@intel.com; pawelx.wodkowski@intel.com
>> Subject: Re: [PATCH v6 0/8] vhost: introduce vhost crypto backend
>>
>> Hi Fan,
>>
>> On 04/04/2018 04:24 PM, Fan Zhang wrote:
>>> This patchset adds crypto backend suppport to vhost library including
>>> a proof-of-concept sample application. The implementation follows the
>>> virtio-crypto specification and have been tested with qemu 2.11.50
>>> (with several patches applied, detailed later) with Fedora 24 running
>>> in the frontend.
>>>
>>> The vhost_crypto library acts as a "bridge" method that translate the
>>> virtio-crypto crypto requests to DPDK crypto operations, so it is
>>> purely software implementation. However it does require the user to
>>> provide the DPDK Cryptodev ID so it knows how to handle the
>>> virtio-crypto session creation and deletion mesages.
>>>
>>> Currently the implementation supports AES-CBC-128 and HMAC-SHA1 cipher
>>> only/chaining modes and does not support sessionless mode yet. The
>>> guest can use standard virtio-crypto driver to set up session and
>>> sends encryption/decryption requests to backend. The vhost-crypto
>>> sample application provided in this patchset will do the actual crypto
>>> work.
>>>
>>> The following steps are involved to enable vhost-crypto support.
>>>
>>> In the host:
>>> 1. Download the qemu source code.
>>>
>>> 2. Recompile your qemu with vhost-crypto option enabled.
>>>
>>> 3. Apply this patchset to latest DPDK code and recompile DPDK.
>>>
>>> 4. Compile and run vhost-crypto sample application.
>>>
>>> ./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
>>>    --socket-mem 2048,2048
>>>
>>> Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it
>>> is not available. The sample application requires 2 lcores: 1 master
>>> and 1 worker. The application will create a UNIX socket file
>>> /tmp/vhost_crypto1.socket.
>>>
>>> 5. Start your qemu application. Here is my command:
>>>
>>> qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
>>> -smp 2 -m 1G -hda ~/path-to-your/image.qcow \ -object
>>> memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
>>> -mem-prealloc -numa node,memdev=mem -chardev \
>>> socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \ -object
>>> cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \ -device
>>> virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0
>>>
>>> 6. Once guest is booted. The Linux virtio_crypto kernel module is
>>> loaded by default. You shall see the following logs in your demsg:
>>>
>>> [   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
>>> [   17.611083] virtio_crypto: module verification failed: signature
>> and/or ...
>>> [   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
>>> [   17.612156] virtio_crypto virtio0: will run requests pump with
>> realtime ...
>>> [   18.376100] virtio_crypto virtio0: Accelerator is ready
>>>
>>> The virtio_crypto driver in the guest is now up and running.
>>>
>>> 7. The rest steps can be as same as the Testing section in
>>> https://wiki.qemu.org/Features/VirtioCrypto
>>>
>>> 8. It is possible to use DPDK Virtio Crypto PMD
>>> (https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work
>>> with this patchset to achieve optimal performance.
>>>
>>> v6:
>>> - Changed commit message
>>> - removed rte prefix in handler prototype
>>>
>>> v5:
>>> - removed external ops register API.
>>> - patch cleaned.
>>>
>>> v4:
>>> - Changed external vhost backend ops register API.
>>> - Fixed a bug.
>>>
>>> v3:
>>> - Changed external vhost backend private data and message handling
>>> - Added experimental tag to rte_vhost_crypto_set_zero_copy()
>>>
>>> v2:
>>> - Moved vhost_crypto_data_req data from crypto op to source mbuf.
>>> - Removed ZERO-COPY flag from config option and make it run-timely
>> changeable.
>>> - Guest-polling mode possible.
>>> - Simplified vring descriptor access procedure.
>>> - Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.
>>>
>>> Fan Zhang (8):
>>>     lib/librte_vhost: add vhost user message handlers
>>>     lib/librte_vhost: add virtio-crypto user message structure
>>>     lib/librte_vhost: add session message handler
>>>     lib/librte_vhost: add request handler
>>>     lib/librte_vhost: add public function implementation
>>>     lib/librte_vhost: update makefile
>>>     examples/vhost_crypto: add vhost crypto sample application
>>>     doc: update for vhost crypto support
>>>
>>>    doc/guides/prog_guide/vhost_lib.rst       |   25 +
>>>    doc/guides/rel_notes/release_18_05.rst    |    5 +
>>>    doc/guides/sample_app_ug/index.rst        |    1 +
>>>    doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
>>>    examples/vhost_crypto/Makefile            |   32 +
>>>    examples/vhost_crypto/main.c              |  541 ++++++++++++
>>>    examples/vhost_crypto/meson.build         |   14 +
>>>    lib/librte_vhost/Makefile                 |    6 +-
>>>    lib/librte_vhost/meson.build              |    8 +-
>>>    lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
>>>    lib/librte_vhost/rte_vhost_version.map    |   11 +
>>>    lib/librte_vhost/vhost.c                  |    2 +-
>>>    lib/librte_vhost/vhost.h                  |   53 +-
>>>    lib/librte_vhost/vhost_crypto.c           | 1312
>> +++++++++++++++++++++++++++++
>>>    lib/librte_vhost/vhost_user.c             |   33 +-
>>>    lib/librte_vhost/vhost_user.h             |   35 +-
>>>    16 files changed, 2256 insertions(+), 13 deletions(-)
>>>    create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
>>>    create mode 100644 examples/vhost_crypto/Makefile
>>>    create mode 100644 examples/vhost_crypto/main.c
>>>    create mode 100644 examples/vhost_crypto/meson.build
>>>    create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
>>>    create mode 100644 lib/librte_vhost/vhost_crypto.c
>>>
>>
>> For the series:
>> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>>
>> Ideally, I would like to have it reviewed/acked by people having knowledge of
>> crypto.
>> Jianjay, is the series good for you?
> 
> With the typo of 8/8 fixed, this series looks good to me, so:
> Acked-by: Jay Zhou <jianjay.zhou@huawei.com>

Thanks Jay. can you point me to the typo you are referring to?

I can handle the change while applying.

Cheers,
Maxime

>>
>> Thanks,
>> Maxime
> 

Re: [PATCH v6 4/8] lib/librte_vhost: add request handler

[Maxime Coquelin]

On 04/04/2018 04:25 PM, Fan Zhang wrote:
> This patch adds the implementation that parses virtio crypto request
> to dpdk crypto operation.
> 
> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> ---
>   lib/librte_vhost/rte_vhost_crypto.h |  14 +
>   lib/librte_vhost/vhost_crypto.c     | 622 ++++++++++++++++++++++++++++++++++++
>   2 files changed, 636 insertions(+)
>   create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
> 
...
> diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
> index 34587c27e..5608e94ad 100644
> --- a/lib/librte_vhost/vhost_crypto.c
> +++ b/lib/librte_vhost/vhost_crypto.c
> @@ -8,9 +8,11 @@
...
> +static uint8_t
> +prepare_sym_cipher_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
> +		struct vhost_crypto_data_req *vc_req,
> +		struct virtio_crypto_cipher_data_req *cipher,
> +		struct vring_desc *cur_desc)
> +{
> +	struct vring_desc *head = vc_req->head;
> +	struct vring_desc *desc = cur_desc;
> +	struct rte_vhost_memory *mem = vc_req->mem;
> +	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
> +	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
> +	uint8_t ret = 0;
> +
> +	/* prepare */
> +	/* iv */
> +	if (unlikely(copy_data(iv_data, head, mem, &desc,
> +			cipher->para.iv_len) < 0)) {
> +		ret = VIRTIO_CRYPTO_BADMSG;
> +		goto error_exit;
> +	}
> +
> +#ifdef RTE_LIBRTE_VHOST_DEBUG
> +	rte_hexdump(stdout, "IV:", iv_data, cipher->para.iv_len);
> +#endif
> +
> +	m_src->data_len = cipher->para.src_data_len;
> +
> +	switch (vcrypto->option) {
> +	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
> +		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
> +				cipher->para.src_data_len);
> +		m_src->buf_addr = get_data_ptr(head, mem, &desc,
> +				cipher->para.src_data_len);
> +		if (unlikely(m_src->buf_iova == 0 ||
> +				m_src->buf_addr == NULL)) {
> +			VC_LOG_ERR("zero_copy may fail due to cross page data");
> +			ret = VIRTIO_CRYPTO_ERR;
> +			goto error_exit;
> +		}
> +		break;
> +	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
> +		if (unlikely(cipher->para.src_data_len >
> +				RTE_MBUF_DEFAULT_BUF_SIZE)) {
> +			VC_LOG_ERR("Not enough space to do data copy");
> +			ret = VIRTIO_CRYPTO_ERR;
> +			goto error_exit;
> +		}
> +		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
> +				mem, &desc, cipher->para.src_data_len))
> +				< 0) {
> +			ret = VIRTIO_CRYPTO_BADMSG;
> +			goto error_exit;
> +		}
> +		break;
> +	default:
> +		ret = VIRTIO_CRYPTO_BADMSG;
> +		goto error_exit;
> +		break;

Note: I removed the break to make checkpatch happy.

Re: [PATCH v6 0/8] vhost: introduce vhost crypto backend

[Maxime Coquelin]

On 04/04/2018 04:24 PM, Fan Zhang wrote:
> This patchset adds crypto backend suppport to vhost library
> including a proof-of-concept sample application. The implementation
> follows the virtio-crypto specification and have been tested
> with qemu 2.11.50 (with several patches applied, detailed later)
> with Fedora 24 running in the frontend.
> 
> The vhost_crypto library acts as a "bridge" method that translate
> the virtio-crypto crypto requests to DPDK crypto operations, so it
> is purely software implementation. However it does require the user
> to provide the DPDK Cryptodev ID so it knows how to handle the
> virtio-crypto session creation and deletion mesages.
> 
> Currently the implementation supports AES-CBC-128 and HMAC-SHA1
> cipher only/chaining modes and does not support sessionless mode
> yet. The guest can use standard virtio-crypto driver to set up
> session and sends encryption/decryption requests to backend. The
> vhost-crypto sample application provided in this patchset will
> do the actual crypto work.
> 
> The following steps are involved to enable vhost-crypto support.
> 
> In the host:
> 1. Download the qemu source code.
> 
> 2. Recompile your qemu with vhost-crypto option enabled.
> 
> 3. Apply this patchset to latest DPDK code and recompile DPDK.
> 
> 4. Compile and run vhost-crypto sample application.
> 
> ./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
>   --socket-mem 2048,2048
> 
> Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
> not available. The sample application requires 2 lcores: 1 master and 1
> worker. The application will create a UNIX socket file
> /tmp/vhost_crypto1.socket.
> 
> 5. Start your qemu application. Here is my command:
> 
> qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
> -smp 2 -m 1G -hda ~/path-to-your/image.qcow \
> -object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
> -mem-prealloc -numa node,memdev=mem -chardev \
> socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
> -object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
> -device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0
> 
> 6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
> default. You shall see the following logs in your demsg:
> 
> [   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
> [   17.611083] virtio_crypto: module verification failed: signature and/or ...
> [   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
> [   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
> [   18.376100] virtio_crypto virtio0: Accelerator is ready
> 
> The virtio_crypto driver in the guest is now up and running.
> 
> 7. The rest steps can be as same as the Testing section in
> https://wiki.qemu.org/Features/VirtioCrypto
> 
> 8. It is possible to use DPDK Virtio Crypto PMD
> (https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work with
> this patchset to achieve optimal performance.
> 
> v6:
> - Changed commit message
> - removed rte prefix in handler prototype
> 
> v5:
> - removed external ops register API.
> - patch cleaned.
> 
> v4:
> - Changed external vhost backend ops register API.
> - Fixed a bug.
> 
> v3:
> - Changed external vhost backend private data and message handling
> - Added experimental tag to rte_vhost_crypto_set_zero_copy()
> 
> v2:
> - Moved vhost_crypto_data_req data from crypto op to source mbuf.
> - Removed ZERO-COPY flag from config option and make it run-timely changeable.
> - Guest-polling mode possible.
> - Simplified vring descriptor access procedure.
> - Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.
> 
> Fan Zhang (8):
>    lib/librte_vhost: add vhost user message handlers
>    lib/librte_vhost: add virtio-crypto user message structure
>    lib/librte_vhost: add session message handler
>    lib/librte_vhost: add request handler
>    lib/librte_vhost: add public function implementation
>    lib/librte_vhost: update makefile
>    examples/vhost_crypto: add vhost crypto sample application
>    doc: update for vhost crypto support
> 
>   doc/guides/prog_guide/vhost_lib.rst       |   25 +
>   doc/guides/rel_notes/release_18_05.rst    |    5 +
>   doc/guides/sample_app_ug/index.rst        |    1 +
>   doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
>   examples/vhost_crypto/Makefile            |   32 +
>   examples/vhost_crypto/main.c              |  541 ++++++++++++
>   examples/vhost_crypto/meson.build         |   14 +
>   lib/librte_vhost/Makefile                 |    6 +-
>   lib/librte_vhost/meson.build              |    8 +-
>   lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
>   lib/librte_vhost/rte_vhost_version.map    |   11 +
>   lib/librte_vhost/vhost.c                  |    2 +-
>   lib/librte_vhost/vhost.h                  |   53 +-
>   lib/librte_vhost/vhost_crypto.c           | 1312 +++++++++++++++++++++++++++++
>   lib/librte_vhost/vhost_user.c             |   33 +-
>   lib/librte_vhost/vhost_user.h             |   35 +-
>   16 files changed, 2256 insertions(+), 13 deletions(-)
>   create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
>   create mode 100644 examples/vhost_crypto/Makefile
>   create mode 100644 examples/vhost_crypto/main.c
>   create mode 100644 examples/vhost_crypto/meson.build
>   create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
>   create mode 100644 lib/librte_vhost/vhost_crypto.c
> 

Applied to dpdk-next-virtio/master.

Fan, I had quite a few conflicts to solve due to vDPA series.

Can you please a trial of vhost crypto usecase and confirm this is all
good?

Thanks,
Maxime

Re: [PATCH v6 0/8] vhost: introduce vhost crypto backend

[Maxime Coquelin]

On 04/05/2018 10:26 AM, Maxime Coquelin wrote:
> 
> 
> On 04/04/2018 04:24 PM, Fan Zhang wrote:
>> This patchset adds crypto backend suppport to vhost library
>> including a proof-of-concept sample application. The implementation
>> follows the virtio-crypto specification and have been tested
>> with qemu 2.11.50 (with several patches applied, detailed later)
>> with Fedora 24 running in the frontend.
>>
>> The vhost_crypto library acts as a "bridge" method that translate
>> the virtio-crypto crypto requests to DPDK crypto operations, so it
>> is purely software implementation. However it does require the user
>> to provide the DPDK Cryptodev ID so it knows how to handle the
>> virtio-crypto session creation and deletion mesages.
>>
>> Currently the implementation supports AES-CBC-128 and HMAC-SHA1
>> cipher only/chaining modes and does not support sessionless mode
>> yet. The guest can use standard virtio-crypto driver to set up
>> session and sends encryption/decryption requests to backend. The
>> vhost-crypto sample application provided in this patchset will
>> do the actual crypto work.
>>
>> The following steps are involved to enable vhost-crypto support.
>>
>> In the host:
>> 1. Download the qemu source code.
>>
>> 2. Recompile your qemu with vhost-crypto option enabled.
>>
>> 3. Apply this patchset to latest DPDK code and recompile DPDK.
>>
>> 4. Compile and run vhost-crypto sample application.
>>
>> ./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
>>   --socket-mem 2048,2048
>>
>> Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
>> not available. The sample application requires 2 lcores: 1 master and 1
>> worker. The application will create a UNIX socket file
>> /tmp/vhost_crypto1.socket.
>>
>> 5. Start your qemu application. Here is my command:
>>
>> qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
>> -smp 2 -m 1G -hda ~/path-to-your/image.qcow \
>> -object 
>> memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
>> -mem-prealloc -numa node,memdev=mem -chardev \
>> socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
>> -object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
>> -device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0
>>
>> 6. Once guest is booted. The Linux virtio_crypto kernel module is 
>> loaded by
>> default. You shall see the following logs in your demsg:
>>
>> [   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
>> [   17.611083] virtio_crypto: module verification failed: signature 
>> and/or ...
>> [   17.611723] virtio_crypto virtio0: max_queues: 1, 
>> max_cipher_key_len: ...
>> [   17.612156] virtio_crypto virtio0: will run requests pump with 
>> realtime ...
>> [   18.376100] virtio_crypto virtio0: Accelerator is ready
>>
>> The virtio_crypto driver in the guest is now up and running.
>>
>> 7. The rest steps can be as same as the Testing section in
>> https://wiki.qemu.org/Features/VirtioCrypto
>>
>> 8. It is possible to use DPDK Virtio Crypto PMD
>> (https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work with
>> this patchset to achieve optimal performance.
>>
>> v6:
>> - Changed commit message
>> - removed rte prefix in handler prototype
>>
>> v5:
>> - removed external ops register API.
>> - patch cleaned.
>>
>> v4:
>> - Changed external vhost backend ops register API.
>> - Fixed a bug.
>>
>> v3:
>> - Changed external vhost backend private data and message handling
>> - Added experimental tag to rte_vhost_crypto_set_zero_copy()
>>
>> v2:
>> - Moved vhost_crypto_data_req data from crypto op to source mbuf.
>> - Removed ZERO-COPY flag from config option and make it run-timely 
>> changeable.
>> - Guest-polling mode possible.
>> - Simplified vring descriptor access procedure.
>> - Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.
>>
>> Fan Zhang (8):
>>    lib/librte_vhost: add vhost user message handlers
>>    lib/librte_vhost: add virtio-crypto user message structure
>>    lib/librte_vhost: add session message handler
>>    lib/librte_vhost: add request handler
>>    lib/librte_vhost: add public function implementation
>>    lib/librte_vhost: update makefile
>>    examples/vhost_crypto: add vhost crypto sample application
>>    doc: update for vhost crypto support
>>
>>   doc/guides/prog_guide/vhost_lib.rst       |   25 +
>>   doc/guides/rel_notes/release_18_05.rst    |    5 +
>>   doc/guides/sample_app_ug/index.rst        |    1 +
>>   doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
>>   examples/vhost_crypto/Makefile            |   32 +
>>   examples/vhost_crypto/main.c              |  541 ++++++++++++
>>   examples/vhost_crypto/meson.build         |   14 +
>>   lib/librte_vhost/Makefile                 |    6 +-
>>   lib/librte_vhost/meson.build              |    8 +-
>>   lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
>>   lib/librte_vhost/rte_vhost_version.map    |   11 +
>>   lib/librte_vhost/vhost.c                  |    2 +-
>>   lib/librte_vhost/vhost.h                  |   53 +-
>>   lib/librte_vhost/vhost_crypto.c           | 1312 
>> +++++++++++++++++++++++++++++
>>   lib/librte_vhost/vhost_user.c             |   33 +-
>>   lib/librte_vhost/vhost_user.h             |   35 +-
>>   16 files changed, 2256 insertions(+), 13 deletions(-)
>>   create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
>>   create mode 100644 examples/vhost_crypto/Makefile
>>   create mode 100644 examples/vhost_crypto/main.c
>>   create mode 100644 examples/vhost_crypto/meson.build
>>   create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
>>   create mode 100644 lib/librte_vhost/vhost_crypto.c
>>
> 
> Applied to dpdk-next-virtio/master.

Sorry, but I had to revert because it fails to build with older
Kernels:
fatal error: linux/virtio_crypto.h: No such file or directory

Please fix it, fix the typo, and take the opportunity to rebase the
series on top of dpdk-next-virtio/master.

Note that you were also missing _rte_experimental in front of
some of the new APIs.

Thanks in advance,
Maxime
> Fan, I had quite a few conflicts to solve due to vDPA series.
> 
> Can you please a trial of vhost crypto usecase and confirm this is all
> good?
> 
> Thanks,
> Maxime

[PATCH v7 0/8] vhost: introduce vhost crypto backend

[Fan Zhang]

This patchset depends on the following patch and must be applied first:

"lib/librte_vhost: add virtio_crypto.h header file"
(https://dpdk.org/dev/patchwork/patch/37308/)

This patchset adds crypto backend suppport to vhost library
including a proof-of-concept sample application. The implementation
follows the virtio-crypto specification and have been tested
with qemu 2.11.50 (with several patches applied, detailed later)
with Fedora 24 running in the frontend.

The vhost_crypto library acts as a "bridge" method that translate
the virtio-crypto crypto requests to DPDK crypto operations, so it
is purely software implementation. However it does require the user
to provide the DPDK Cryptodev ID so it knows how to handle the
virtio-crypto session creation and deletion mesages.

Currently the implementation supports AES-CBC-128 and HMAC-SHA1
cipher only/chaining modes and does not support sessionless mode
yet. The guest can use standard virtio-crypto driver to set up
session and sends encryption/decryption requests to backend. The
vhost-crypto sample application provided in this patchset will
do the actual crypto work.

The following steps are involved to enable vhost-crypto support.

In the host:
1. Download the qemu source code.

2. Recompile your qemu with vhost-crypto option enabled.

3. Apply this patchset to latest DPDK code and recompile DPDK.

4. Compile and run vhost-crypto sample application.

./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
 --socket-mem 2048,2048

Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
not available. The sample application requires 2 lcores: 1 master and 1
worker. The application will create a UNIX socket file
/tmp/vhost_crypto1.socket.

5. Start your qemu application. Here is my command:

qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
-smp 2 -m 1G -hda ~/path-to-your/image.qcow \
-object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
-mem-prealloc -numa node,memdev=mem -chardev \
socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
-object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
-device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0

6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
default. You shall see the following logs in your demsg:

[   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
[   17.611083] virtio_crypto: module verification failed: signature and/or ...
[   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
[   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
[   18.376100] virtio_crypto virtio0: Accelerator is ready

The virtio_crypto driver in the guest is now up and running.

7. The rest steps can be as same as the Testing section in
https://wiki.qemu.org/Features/VirtioCrypto

8. It is possible to use DPDK Virtio Crypto PMD
(https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work with
this patchset to achieve optimal performance.

v7:
- Rebased on top of dpdk-next-virtio
- Removed dependency of linux virtio crypto header
- Fixed a typo

v6:
- Changed commit message
- removed rte prefix in handler prototype

v5:
- removed external ops register API.
- patch cleaned.

v4:
- Changed external vhost backend ops register API.
- Fixed a bug.

v3:
- Changed external vhost backend private data and message handling
- Added experimental tag to rte_vhost_crypto_set_zero_copy()

v2:
- Moved vhost_crypto_data_req data from crypto op to source mbuf.
- Removed ZERO-COPY flag from config option and make it run-timely changeable.
- Guest-polling mode possible.
- Simplified vring descriptor access procedure.
- Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.

Fan Zhang (8):
  lib/librte_vhost: add vhost user message handlers
  lib/librte_vhost: add virtio-crypto user message structure
  lib/librte_vhost: add session message handler
  lib/librte_vhost: add request handler
  lib/librte_vhost: add public function implementation
  lib/librte_vhost: update makefile
  examples/vhost_crypto: add vhost crypto sample application
  doc: update for vhost crypto support

 doc/guides/prog_guide/vhost_lib.rst       |   25 +
 doc/guides/rel_notes/release_18_05.rst    |    5 +
 doc/guides/sample_app_ug/index.rst        |    1 +
 doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
 examples/vhost_crypto/Makefile            |   32 +
 examples/vhost_crypto/main.c              |  541 ++++++++++++
 examples/vhost_crypto/meson.build         |   14 +
 lib/librte_vhost/Makefile                 |    5 +-
 lib/librte_vhost/meson.build              |    8 +-
 lib/librte_vhost/rte_vhost.h              |    4 +
 lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
 lib/librte_vhost/rte_vhost_version.map    |    6 +
 lib/librte_vhost/vhost.c                  |    2 +-
 lib/librte_vhost/vhost.h                  |   53 +-
 lib/librte_vhost/vhost_crypto.c           | 1311 +++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c             |   33 +-
 lib/librte_vhost/vhost_user.h             |   34 +-
 17 files changed, 2252 insertions(+), 13 deletions(-)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 examples/vhost_crypto/meson.build
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
 create mode 100644 lib/librte_vhost/vhost_crypto.c

-- 
2.13.6

[PATCH v7 1/8] lib/librte_vhost: add vhost user message handlers

[Fan Zhang]

Previously, vhost library lacks the support to the vhost backend
other than net such as adding private data or registering vhost-user
message handlers. This patch fills the gap by adding data pointer and
vhost-user pre and post message handlers to vhost library.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost.c      |  2 +-
 lib/librte_vhost/vhost.h      | 53 +++++++++++++++++++++++++++++++++++++++++--
 lib/librte_vhost/vhost_user.c | 31 ++++++++++++++++++++++++-
 3 files changed, 82 insertions(+), 4 deletions(-)

diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
index 1f17cdd75..8ad9ea8c6 100644
--- a/lib/librte_vhost/vhost.c
+++ b/lib/librte_vhost/vhost.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2017 Intel Corporation
  */
 
 #include <linux/vhost.h>
diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
index c14a90529..c9b64461d 100644
--- a/lib/librte_vhost/vhost.h
+++ b/lib/librte_vhost/vhost.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_CDEV_H_
@@ -212,6 +212,51 @@ struct guest_page {
 };
 
 /**
+ * function prototype for the vhost backend to handler specific vhost user
+ * messages prior to the master message handling
+ *
+ * @param vid
+ *  vhost device id
+ * @param msg
+ *  Message pointer.
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0.
+ * @param skip_master
+ *  If the handler requires skipping the master message handling, this variable
+ *  shall be written 1, otherwise 0.
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*vhost_msg_pre_handle)(int vid, void *msg,
+		uint32_t *require_reply, uint32_t *skip_master);
+
+/**
+ * function prototype for the vhost backend to handler specific vhost user
+ * messages after the master message handling is done
+ *
+ * @param vid
+ *  vhost device id
+ * @param msg
+ *  Message pointer.
+ * @param require_reply
+ *  If the handler requires sending a reply, this varaible shall be written 1,
+ *  otherwise 0.
+ * @return
+ *  0 on success, -1 on failure
+ */
+typedef int (*vhost_msg_post_handle)(int vid, void *msg,
+		uint32_t *require_reply);
+
+/**
+ * pre and post vhost user message handlers
+ */
+struct vhost_user_extern_ops {
+	vhost_msg_pre_handle pre_msg_handle;
+	vhost_msg_post_handle post_msg_handle;
+};
+
+/**
  * Device structure contains all configuration information relating
  * to the device.
  */
@@ -249,8 +294,12 @@ struct virtio_net {
 	 * It's set to -1 for the default software implementation.
 	 */
 	int			vdpa_dev_id;
-} __rte_cache_aligned;
 
+	/* private data for virtio device */
+	void			*extern_data;
+	/* pre and post vhost user message handlers for the device */
+	struct vhost_user_extern_ops extern_ops;
+} __rte_cache_aligned;
 
 #define VHOST_LOG_PAGE	4096
 
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 157cf2f60..ecc1a3eea 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2016 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 /* Security model
@@ -1429,6 +1429,7 @@ vhost_user_msg_handler(int vid, int fd)
 	int did = -1;
 	int ret;
 	int unlock_required = 0;
+	uint32_t skip_master = 0;
 
 	dev = get_device(vid);
 	if (dev == NULL)
@@ -1506,6 +1507,21 @@ vhost_user_msg_handler(int vid, int fd)
 
 	}
 
+	if (dev->extern_ops.pre_msg_handle) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.pre_msg_handle)(dev->vid,
+				(void *)&msg, &need_reply, &skip_master);
+		if (ret < 0)
+			goto skip_to_reply;
+
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
+
+		if (skip_master)
+			goto skip_to_post_handle;
+	}
+
 	switch (msg.request.master) {
 	case VHOST_USER_GET_FEATURES:
 		msg.payload.u64 = vhost_user_get_features(dev);
@@ -1606,9 +1622,22 @@ vhost_user_msg_handler(int vid, int fd)
 	default:
 		ret = -1;
 		break;
+	}
+
+skip_to_post_handle:
+	if (dev->extern_ops.post_msg_handle) {
+		uint32_t need_reply;
+
+		ret = (*dev->extern_ops.post_msg_handle)(
+				dev->vid, (void *)&msg, &need_reply);
+		if (ret < 0)
+			goto skip_to_reply;
 
+		if (need_reply)
+			send_vhost_reply(fd, &msg);
 	}
 
+skip_to_reply:
 	if (unlock_required)
 		vhost_user_unlock_all_queue_pairs(dev);
 
-- 
2.13.6

[PATCH v7 2/8] lib/librte_vhost: add virtio-crypto user message structure

[Fan Zhang]

This patch adds virtio-crypto spec user message structure to
vhost_user.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost.h  |  4 ++++
 lib/librte_vhost/vhost_user.h | 34 +++++++++++++++++++++++++++++++---
 2 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h
index 807da8c4b..e4e8824c9 100644
--- a/lib/librte_vhost/rte_vhost.h
+++ b/lib/librte_vhost/rte_vhost.h
@@ -54,6 +54,10 @@ extern "C" {
 #define VHOST_USER_PROTOCOL_F_SLAVE_REQ	5
 #endif
 
+#ifndef VHOST_USER_PROTOCOL_F_CRYPTO_SESSION
+#define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7
+#endif
+
 /** Indicate whether protocol features negotiation is supported. */
 #ifndef VHOST_USER_F_PROTOCOL_FEATURES
 #define VHOST_USER_F_PROTOCOL_FEATURES	30
diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h
index 97afabc47..1ad5cf467 100644
--- a/lib/librte_vhost/vhost_user.h
+++ b/lib/librte_vhost/vhost_user.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2010-2018 Intel Corporation
  */
 
 #ifndef _VHOST_NET_USER_H
@@ -19,7 +19,8 @@
 					 (1ULL << VHOST_USER_PROTOCOL_F_RARP) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \
 					 (1ULL << VHOST_USER_PROTOCOL_F_NET_MTU) | \
-					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ))
+					 (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ) | \
+					 (1ULL << VHOST_USER_PROTOCOL_F_CRYPTO_SESSION))
 
 typedef enum VhostUserRequest {
 	VHOST_USER_NONE = 0,
@@ -45,7 +46,9 @@ typedef enum VhostUserRequest {
 	VHOST_USER_NET_SET_MTU = 20,
 	VHOST_USER_SET_SLAVE_REQ_FD = 21,
 	VHOST_USER_IOTLB_MSG = 22,
-	VHOST_USER_MAX
+	VHOST_USER_CRYPTO_CREATE_SESS = 26,
+	VHOST_USER_CRYPTO_CLOSE_SESS = 27,
+	VHOST_USER_MAX = 28
 } VhostUserRequest;
 
 typedef enum VhostUserSlaveRequest {
@@ -72,6 +75,30 @@ typedef struct VhostUserLog {
 	uint64_t mmap_offset;
 } VhostUserLog;
 
+/* Comply with Cryptodev-Linux */
+#define VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH	512
+#define VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH	64
+
+/* Same structure as vhost-user backend session info */
+typedef struct VhostUserCryptoSessionParam {
+	int64_t session_id;
+	uint32_t op_code;
+	uint32_t cipher_algo;
+	uint32_t cipher_key_len;
+	uint32_t hash_algo;
+	uint32_t digest_len;
+	uint32_t auth_key_len;
+	uint32_t aad_len;
+	uint8_t op_type;
+	uint8_t dir;
+	uint8_t hash_mode;
+	uint8_t chaining_dir;
+	uint8_t *ciphe_key;
+	uint8_t *auth_key;
+	uint8_t cipher_key_buf[VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH];
+	uint8_t auth_key_buf[VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH];
+} VhostUserCryptoSessionParam;
+
 typedef struct VhostUserMsg {
 	union {
 		uint32_t master; /* a VhostUserRequest value */
@@ -92,6 +119,7 @@ typedef struct VhostUserMsg {
 		VhostUserMemory memory;
 		VhostUserLog    log;
 		struct vhost_iotlb_msg iotlb;
+		VhostUserCryptoSessionParam crypto_session;
 	} payload;
 	int fds[VHOST_MEMORY_MAX_NREGIONS];
 } __attribute((packed)) VhostUserMsg;
-- 
2.13.6

[PATCH v7 3/8] lib/librte_vhost: add session message handler

[Fan Zhang]

This patch adds session message handler to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/vhost_crypto.c | 434 ++++++++++++++++++++++++++++++++++++++++
 lib/librte_vhost/vhost_user.c   |   2 +
 2 files changed, 436 insertions(+)
 create mode 100644 lib/librte_vhost/vhost_crypto.c

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
new file mode 100644
index 000000000..d1ff21eb7
--- /dev/null
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -0,0 +1,434 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+#include <rte_malloc.h>
+#include <rte_hash.h>
+#include <rte_mbuf.h>
+#include <rte_cryptodev.h>
+
+#include "vhost.h"
+#include "vhost_user.h"
+#include "virtio_crypto.h"
+
+#define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
+				sizeof(struct rte_crypto_sym_op))
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+
+#define VC_LOG_DBG(fmt, args...)				\
+	RTE_LOG(DEBUG, USER1, "[%s] %s() line %u: " fmt "\n",	\
+		"Vhost-Crypto",	__func__, __LINE__, ## args)
+#else
+#define VC_LOG_ERR(fmt, args...)				\
+	RTE_LOG(ERR, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_INFO(fmt, args...)				\
+	RTE_LOG(INFO, USER1, "[VHOST-Crypto]: " fmt "\n", ## args)
+#define VC_LOG_DBG(fmt, args...)
+#endif
+
+static int
+cipher_algo_transform(uint32_t virtio_cipher_algo)
+{
+	int ret;
+
+	switch (virtio_cipher_algo) {
+	case VIRTIO_CRYPTO_CIPHER_AES_CBC:
+		ret = RTE_CRYPTO_CIPHER_AES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_CTR:
+		ret = RTE_CRYPTO_CIPHER_AES_CTR;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_DES_ECB:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_DES_CBC:
+		ret = RTE_CRYPTO_CIPHER_DES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_ECB:
+		ret = RTE_CRYPTO_CIPHER_3DES_ECB;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_CBC:
+		ret = RTE_CRYPTO_CIPHER_3DES_CBC;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_3DES_CTR:
+		ret = RTE_CRYPTO_CIPHER_3DES_CTR;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_KASUMI_F8:
+		ret = RTE_CRYPTO_CIPHER_KASUMI_F8;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_SNOW3G_UEA2:
+		ret = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_F8:
+		ret = RTE_CRYPTO_CIPHER_AES_F8;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_AES_XTS:
+		ret = RTE_CRYPTO_CIPHER_AES_XTS;
+		break;
+	case VIRTIO_CRYPTO_CIPHER_ZUC_EEA3:
+		ret = RTE_CRYPTO_CIPHER_ZUC_EEA3;
+		break;
+	default:
+		ret = -VIRTIO_CRYPTO_BADMSG;
+		break;
+	}
+
+	return ret;
+}
+
+static int
+auth_algo_transform(uint32_t virtio_auth_algo)
+{
+	int ret;
+
+	switch (virtio_auth_algo) {
+
+	case VIRTIO_CRYPTO_NO_MAC:
+		ret = RTE_CRYPTO_AUTH_NULL;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_MD5:
+		ret = RTE_CRYPTO_AUTH_MD5_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA1:
+		ret = RTE_CRYPTO_AUTH_SHA1_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_224:
+		ret = RTE_CRYPTO_AUTH_SHA224_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_256:
+		ret = RTE_CRYPTO_AUTH_SHA256_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_384:
+		ret = RTE_CRYPTO_AUTH_SHA384_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_HMAC_SHA_512:
+		ret = RTE_CRYPTO_AUTH_SHA512_HMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_CMAC_3DES:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_CMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_CMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_KASUMI_F9:
+		ret = RTE_CRYPTO_AUTH_KASUMI_F9;
+		break;
+	case VIRTIO_CRYPTO_MAC_SNOW3G_UIA2:
+		ret = RTE_CRYPTO_AUTH_SNOW3G_UIA2;
+		break;
+	case VIRTIO_CRYPTO_MAC_GMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_GMAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_GMAC_TWOFISH:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_CBCMAC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_CBC_MAC;
+		break;
+	case VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9:
+		ret = -VIRTIO_CRYPTO_NOTSUPP;
+		break;
+	case VIRTIO_CRYPTO_MAC_XCBC_AES:
+		ret = RTE_CRYPTO_AUTH_AES_XCBC_MAC;
+		break;
+	default:
+		ret = -VIRTIO_CRYPTO_BADMSG;
+		break;
+	}
+
+	return ret;
+}
+
+static int get_iv_len(enum rte_crypto_cipher_algorithm algo)
+{
+	int len;
+
+	switch (algo) {
+	case RTE_CRYPTO_CIPHER_3DES_CBC:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_3DES_CTR:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_3DES_ECB:
+		len = 8;
+		break;
+	case RTE_CRYPTO_CIPHER_AES_CBC:
+		len = 16;
+		break;
+
+	/* TODO: add common algos */
+
+	default:
+		len = -1;
+		break;
+	}
+
+	return len;
+}
+
+/**
+ * vhost_crypto struct is used to maintain a number of virtio_cryptos and
+ * one DPDK crypto device that deals with all crypto workloads. It is declared
+ * here and defined in vhost_crypto.c
+ */
+struct vhost_crypto {
+	/** Used to lookup DPDK Cryptodev Session based on VIRTIO crypto
+	 *  session ID.
+	 */
+	struct rte_hash *session_map;
+	struct rte_mempool *mbuf_pool;
+	struct rte_mempool *sess_pool;
+
+	/** DPDK cryptodev ID */
+	uint8_t cid;
+	uint16_t nb_qps;
+
+	uint64_t last_session_id;
+
+	uint64_t cache_session_id;
+	struct rte_cryptodev_sym_session *cache_session;
+	/** socket id for the device */
+	int socket_id;
+
+	struct virtio_net *dev;
+
+	uint8_t option;
+} __rte_cache_aligned;
+
+static int
+transform_cipher_param(struct rte_crypto_sym_xform *xform,
+		VhostUserCryptoSessionParam *param)
+{
+	int ret;
+
+	ret = cipher_algo_transform(param->cipher_algo);
+	if (unlikely(ret < 0))
+		return ret;
+
+	xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform->cipher.algo = (uint32_t)ret;
+	xform->cipher.key.length = param->cipher_key_len;
+	if (xform->cipher.key.length > 0)
+		xform->cipher.key.data = param->cipher_key_buf;
+	if (param->dir == VIRTIO_CRYPTO_OP_ENCRYPT)
+		xform->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+	else if (param->dir == VIRTIO_CRYPTO_OP_DECRYPT)
+		xform->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+	else {
+		VC_LOG_DBG("Bad operation type");
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	ret = get_iv_len(xform->cipher.algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform->cipher.iv.length = (uint16_t)ret;
+	xform->cipher.iv.offset = IV_OFFSET;
+	return 0;
+}
+
+static int
+transform_chain_param(struct rte_crypto_sym_xform *xforms,
+		VhostUserCryptoSessionParam *param)
+{
+	struct rte_crypto_sym_xform *xform_cipher, *xform_auth;
+	int ret;
+
+	switch (param->chaining_dir) {
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER:
+		xform_auth = xforms;
+		xform_cipher = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+		break;
+	case VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH:
+		xform_cipher = xforms;
+		xform_auth = xforms->next;
+		xform_cipher->cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		xform_auth->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		break;
+	default:
+		return -VIRTIO_CRYPTO_BADMSG;
+	}
+
+	/* cipher */
+	ret = cipher_algo_transform(param->cipher_algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+	xform_cipher->cipher.algo = (uint32_t)ret;
+	xform_cipher->cipher.key.length = param->cipher_key_len;
+	xform_cipher->cipher.key.data = param->cipher_key_buf;
+	ret = get_iv_len(xform_cipher->cipher.algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_cipher->cipher.iv.length = (uint16_t)ret;
+	xform_cipher->cipher.iv.offset = IV_OFFSET;
+
+	/* auth */
+	xform_auth->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+	ret = auth_algo_transform(param->hash_algo);
+	if (unlikely(ret < 0))
+		return ret;
+	xform_auth->auth.algo = (uint32_t)ret;
+	xform_auth->auth.digest_length = param->digest_len;
+	xform_auth->auth.key.length = param->auth_key_len;
+	xform_auth->auth.key.data = param->auth_key_buf;
+
+	return 0;
+}
+
+static void
+vhost_crypto_create_sess(struct vhost_crypto *vcrypto,
+		VhostUserCryptoSessionParam *sess_param)
+{
+	struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0};
+	struct rte_cryptodev_sym_session *session;
+	int ret;
+
+	switch (sess_param->op_type) {
+	case VIRTIO_CRYPTO_SYM_OP_NONE:
+	case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+		ret = transform_cipher_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session msg (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+		break;
+	case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+		if (unlikely(sess_param->hash_mode !=
+				VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH)) {
+			sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+			VC_LOG_ERR("Error transform session message (%i)",
+					-VIRTIO_CRYPTO_NOTSUPP);
+			return;
+		}
+
+		xform1.next = &xform2;
+
+		ret = transform_chain_param(&xform1, sess_param);
+		if (unlikely(ret)) {
+			VC_LOG_ERR("Error transform session message (%i)", ret);
+			sess_param->session_id = ret;
+			return;
+		}
+
+		break;
+	default:
+		VC_LOG_ERR("Algorithm not yet supported");
+		sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP;
+		return;
+	}
+
+	session = rte_cryptodev_sym_session_create(vcrypto->sess_pool);
+	if (!session) {
+		VC_LOG_ERR("Failed to create session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1,
+			vcrypto->sess_pool) < 0) {
+		VC_LOG_ERR("Failed to initialize session");
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	/* insert hash to map */
+	if (rte_hash_add_key_data(vcrypto->session_map,
+			&vcrypto->last_session_id, session) < 0) {
+		VC_LOG_ERR("Failed to insert session to hash table");
+
+		if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0)
+			VC_LOG_ERR("Failed to clear session");
+		else {
+			if (rte_cryptodev_sym_session_free(session) < 0)
+				VC_LOG_ERR("Failed to free session");
+		}
+		sess_param->session_id = -VIRTIO_CRYPTO_ERR;
+		return;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) created.",
+			vcrypto->last_session_id, session);
+
+	sess_param->session_id = vcrypto->last_session_id;
+	vcrypto->last_session_id++;
+}
+
+static int
+vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id)
+{
+	struct rte_cryptodev_sym_session *session;
+	uint64_t sess_id = session_id;
+	int ret;
+
+	ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id,
+			(void **)&session);
+
+	if (unlikely(ret < 0)) {
+		VC_LOG_ERR("Failed to delete session (key %lu).", session_id);
+		return -VIRTIO_CRYPTO_INVSESS;
+	}
+
+	if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_cryptodev_sym_session_free(session) < 0) {
+		VC_LOG_DBG("Failed to delete session");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	if (rte_hash_del_key(vcrypto->session_map, &sess_id) < 0) {
+		VC_LOG_DBG("Failed to delete session from hash table.");
+		return -VIRTIO_CRYPTO_ERR;
+	}
+
+	VC_LOG_DBG("Session (key %lu, session %p) deleted.", sess_id,
+			session);
+
+	return 0;
+}
+
+static int
+vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+	VhostUserMsg *vmsg = msg;
+	int ret = 0;
+
+	if (dev == NULL || require_reply == NULL) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (vcrypto == NULL) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	*require_reply = 0;
+
+	if (vmsg->request.master == VHOST_USER_CRYPTO_CREATE_SESS) {
+		vhost_crypto_create_sess(vcrypto,
+				&vmsg->payload.crypto_session);
+		*require_reply = 1;
+	} else if (vmsg->request.master == VHOST_USER_CRYPTO_CLOSE_SESS)
+		ret = vhost_crypto_close_sess(vcrypto, vmsg->payload.u64);
+	else
+		ret = -EINVAL;
+
+	return ret;
+}
diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index ecc1a3eea..a3dccf67b 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -67,6 +67,8 @@ static const char *vhost_message_str[VHOST_USER_MAX] = {
 	[VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU",
 	[VHOST_USER_SET_SLAVE_REQ_FD]  = "VHOST_USER_SET_SLAVE_REQ_FD",
 	[VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG",
+	[VHOST_USER_CRYPTO_CREATE_SESS] = "VHOST_USER_CRYPTO_CREATE_SESS",
+	[VHOST_USER_CRYPTO_CLOSE_SESS] = "VHOST_USER_CRYPTO_CLOSE_SESS",
 };
 
 static uint64_t
-- 
2.13.6

[PATCH v7 4/8] lib/librte_vhost: add request handler

[Fan Zhang]

This patch adds the implementation that parses virtio crypto request
to dpdk crypto operation.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h |  14 +
 lib/librte_vhost/vhost_crypto.c     | 622 ++++++++++++++++++++++++++++++++++++
 2 files changed, 636 insertions(+)
 create mode 100644 lib/librte_vhost/rte_vhost_crypto.h

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
new file mode 100644
index 000000000..b6be9d87d
--- /dev/null
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#ifndef _VHOST_CRYPTO_H_
+#define _VHOST_CRYPTO_H_
+
+enum rte_vhost_crypto_zero_copy {
+	RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE = 0,
+	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE,
+	RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS
+};
+
+#endif /**< _VHOST_CRYPTO_H_ */
diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index d1ff21eb7..066af7d70 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -6,10 +6,12 @@
 #include <rte_mbuf.h>
 #include <rte_cryptodev.h>
 
+#include "rte_vhost_crypto.h"
 #include "vhost.h"
 #include "vhost_user.h"
 #include "virtio_crypto.h"
 
+#define INHDR_LEN		(sizeof(struct virtio_crypto_inhdr))
 #define IV_OFFSET		(sizeof(struct rte_crypto_op) + \
 				sizeof(struct rte_crypto_sym_op))
 
@@ -32,6 +34,8 @@
 #define VC_LOG_DBG(fmt, args...)
 #endif
 
+#define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
+
 static int
 cipher_algo_transform(uint32_t virtio_cipher_algo)
 {
@@ -202,6 +206,18 @@ struct vhost_crypto {
 	uint8_t option;
 } __rte_cache_aligned;
 
+struct vhost_crypto_data_req {
+	struct vring_desc *head;
+	struct rte_vhost_memory *mem;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vhost_virtqueue *vq;
+	struct vring_desc *wb_desc;
+	uint16_t wb_len;
+	uint16_t desc_idx;
+	uint16_t len;
+	uint16_t zero_copy;
+};
+
 static int
 transform_cipher_param(struct rte_crypto_sym_xform *xform,
 		VhostUserCryptoSessionParam *param)
@@ -432,3 +448,609 @@ vhost_crypto_msg_post_handler(int vid, void *msg, uint32_t *require_reply)
 
 	return ret;
 }
+
+static __rte_always_inline struct vring_desc *
+find_write_desc(struct vring_desc *head, struct vring_desc *desc)
+{
+	if (desc->flags & VRING_DESC_F_WRITE)
+		return desc;
+
+	while (desc->flags & VRING_DESC_F_NEXT) {
+		desc = &head[desc->next];
+		if (desc->flags & VRING_DESC_F_WRITE)
+			return desc;
+	}
+
+	return NULL;
+}
+
+static struct virtio_crypto_inhdr *
+reach_inhdr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc *desc)
+{
+	while (desc->flags & VRING_DESC_F_NEXT)
+		desc = &head[desc->next];
+
+	return GPA_TO_VVA(struct virtio_crypto_inhdr *, mem, desc->addr);
+}
+
+static __rte_always_inline int
+move_desc(struct vring_desc *head, struct vring_desc **cur_desc,
+		uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	left -= desc->len;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		left -= desc->len;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+	return 0;
+}
+
+static int
+copy_data(void *dst_data, struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	struct vring_desc *desc = *cur_desc;
+	uint32_t to_copy;
+	uint8_t *data = dst_data;
+	uint8_t *src;
+	int left = size;
+
+	rte_prefetch0(&head[desc->next]);
+	to_copy = RTE_MIN(desc->len, (uint32_t)left);
+	src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy((uint8_t *)data, src, to_copy);
+	left -= to_copy;
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_copy = RTE_MIN(desc->len, (uint32_t)left);
+		src = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(data + size - left, src, to_copy);
+		left -= to_copy;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	*cur_desc = &head[desc->next];
+
+	return 0;
+}
+
+static __rte_always_inline void *
+get_data_ptr(struct vring_desc *head, struct rte_vhost_memory *mem,
+		struct vring_desc **cur_desc, uint32_t size)
+{
+	void *data;
+
+	data = GPA_TO_VVA(void *, mem, (*cur_desc)->addr);
+	if (unlikely(!data)) {
+		VC_LOG_ERR("Failed to get object");
+		return NULL;
+	}
+
+	if (unlikely(move_desc(head, cur_desc, size) < 0))
+		return NULL;
+
+	return data;
+}
+
+static int
+write_back_data(struct rte_crypto_op *op, struct vhost_crypto_data_req *vc_req)
+{
+	struct rte_mbuf *mbuf = op->sym->m_dst;
+	struct vring_desc *head = vc_req->head;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct vring_desc *desc = vc_req->wb_desc;
+	int left = vc_req->wb_len;
+	uint32_t to_write;
+	uint8_t *src_data = mbuf->buf_addr, *dst;
+
+	rte_prefetch0(&head[desc->next]);
+	to_write = RTE_MIN(desc->len, (uint32_t)left);
+	dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+	rte_memcpy(dst, src_data, to_write);
+	left -= to_write;
+	src_data += to_write;
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	printf("desc addr %llu len %u:", desc->addr, desc->len);
+	rte_hexdump(stdout, "", dst, to_write);
+#endif
+
+	while ((desc->flags & VRING_DESC_F_NEXT) && left > 0) {
+		desc = &head[desc->next];
+		rte_prefetch0(&head[desc->next]);
+		to_write = RTE_MIN(desc->len, (uint32_t)left);
+		dst = GPA_TO_VVA(uint8_t *, mem, desc->addr);
+		rte_memcpy(dst, src_data, to_write);
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+		printf("desc addr %llu len %u:", desc->addr, desc->len);
+		rte_hexdump(stdout, "DST:", dst, to_write);
+#endif
+		left -= to_write;
+		src_data += to_write;
+	}
+
+	if (unlikely(left < 0)) {
+		VC_LOG_ERR("Incorrect virtio descriptor");
+		return -1;
+	}
+
+	return 0;
+}
+
+static uint8_t
+prepare_sym_cipher_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_cipher_data_req *cipher,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			cipher->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, cipher->para.iv_len);
+#endif
+
+	m_src->data_len = cipher->para.src_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				cipher->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 ||
+				m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		if (unlikely(cipher->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, cipher->para.src_data_len))
+				< 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+		break;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			cipher->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, cipher->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				cipher->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		m_dst->data_len = cipher->para.dst_data_len;
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = cipher->para.dst_data_len;
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	/* src data */
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = 0;
+	op->sym->cipher.data.length = cipher->para.src_data_len;
+
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+	vc_req->len = cipher->para.dst_data_len + INHDR_LEN;
+
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+static uint8_t
+prepare_sym_chain_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op,
+		struct vhost_crypto_data_req *vc_req,
+		struct virtio_crypto_alg_chain_data_req *chain,
+		struct vring_desc *cur_desc)
+{
+	struct vring_desc *head = vc_req->head;
+	struct vring_desc *desc = cur_desc;
+	struct rte_vhost_memory *mem = vc_req->mem;
+	struct rte_mbuf *m_src = op->sym->m_src, *m_dst = op->sym->m_dst;
+	uint8_t *iv_data = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET);
+	uint32_t digest_offset;
+	void *digest_addr;
+	uint8_t ret = 0;
+
+	/* prepare */
+	/* iv */
+	if (unlikely(copy_data(iv_data, head, mem, &desc,
+			chain->para.iv_len) < 0)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "IV:", iv_data, chain->para.iv_len);
+#endif
+	m_src->data_len = chain->para.src_data_len;
+	m_dst->data_len = chain->para.dst_data_len;
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_src->buf_iova = gpa_to_hpa(vcrypto->dev, desc->addr,
+				chain->para.src_data_len);
+		m_src->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.src_data_len);
+		if (unlikely(m_src->buf_iova == 0 || m_src->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		if (unlikely(chain->para.src_data_len >
+				RTE_MBUF_DEFAULT_BUF_SIZE)) {
+			VC_LOG_ERR("Not enough space to do data copy");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		if (unlikely(copy_data(rte_pktmbuf_mtod(m_src, uint8_t *), head,
+				mem, &desc, chain->para.src_data_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "SRC:", rte_pktmbuf_mtod(m_src, void *),
+			chain->para.src_data_len);
+#endif
+
+	/* dst */
+	desc = find_write_desc(head, desc);
+	if (unlikely(!desc)) {
+		VC_LOG_ERR("Cannot find write location");
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	switch (vcrypto->option) {
+	case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE:
+		m_dst->buf_iova = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.dst_data_len);
+		m_dst->buf_addr = get_data_ptr(head, mem, &desc,
+				chain->para.dst_data_len);
+		if (unlikely(m_dst->buf_iova == 0 || m_dst->buf_addr == NULL)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.phys_addr = gpa_to_hpa(vcrypto->dev,
+				desc->addr, chain->para.hash_result_len);
+		op->sym->auth.digest.data = get_data_ptr(head, mem, &desc,
+				chain->para.hash_result_len);
+		if (unlikely(op->sym->auth.digest.phys_addr == 0)) {
+			VC_LOG_ERR("zero_copy may fail due to cross page data");
+			ret = VIRTIO_CRYPTO_ERR;
+			goto error_exit;
+		}
+		break;
+	case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE:
+		digest_offset = m_dst->data_len;
+		digest_addr = rte_pktmbuf_mtod_offset(m_dst, void *,
+				digest_offset);
+
+		vc_req->wb_desc = desc;
+		vc_req->wb_len = m_dst->data_len + chain->para.hash_result_len;
+
+		if (unlikely(move_desc(head, &desc,
+				chain->para.dst_data_len) < 0)) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		if (unlikely(copy_data(digest_addr, head, mem, &desc,
+				chain->para.hash_result_len)) < 0) {
+			ret = VIRTIO_CRYPTO_BADMSG;
+			goto error_exit;
+		}
+
+		op->sym->auth.digest.data = digest_addr;
+		op->sym->auth.digest.phys_addr = rte_pktmbuf_iova_offset(m_dst,
+				digest_offset);
+		break;
+	default:
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "Digest:", op->sym->auth.digest.data,
+			chain->para.hash_result_len);
+#endif
+
+	/* record inhdr */
+	vc_req->inhdr = get_data_ptr(head, mem, &desc, INHDR_LEN);
+	if (unlikely(vc_req->inhdr == NULL)) {
+		ret = VIRTIO_CRYPTO_BADMSG;
+		goto error_exit;
+	}
+
+	vc_req->inhdr->status = VIRTIO_CRYPTO_OK;
+
+	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
+	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
+
+	op->sym->cipher.data.offset = chain->para.cipher_start_src_offset;
+	op->sym->cipher.data.length = chain->para.src_data_len -
+			chain->para.cipher_start_src_offset;
+
+	op->sym->auth.data.offset = chain->para.hash_start_src_offset;
+	op->sym->auth.data.length = chain->para.len_to_hash;
+
+	vc_req->len = chain->para.dst_data_len + chain->para.hash_result_len +
+			INHDR_LEN;
+	return 0;
+
+error_exit:
+	vc_req->len = INHDR_LEN;
+	return ret;
+}
+
+/**
+ * Process on descriptor
+ */
+static __rte_always_inline int
+vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
+		struct vhost_virtqueue *vq, struct rte_crypto_op *op,
+		struct vring_desc *head, uint16_t desc_idx,
+		struct rte_vhost_memory *mem)
+{
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(op->sym->m_src,
+			sizeof(struct rte_mbuf));
+	struct rte_cryptodev_sym_session *session;
+	struct virtio_crypto_op_data_req *req;
+	struct virtio_crypto_inhdr *inhdr;
+	struct vring_desc *desc = NULL;
+	uint64_t session_id;
+	int err = 0;
+
+	vc_req->desc_idx = desc_idx;
+
+	if (likely(head->flags & VRING_DESC_F_INDIRECT)) {
+		head = GPA_TO_VVA(struct vring_desc *, mem, head->addr);
+		if (unlikely(!head))
+			return 0;
+		desc_idx = 0;
+	}
+
+	desc = head;
+
+	vc_req->mem = mem;
+	vc_req->head = head;
+	vc_req->vq = vq;
+
+	vc_req->zero_copy = vcrypto->option;
+
+	req = get_data_ptr(head, mem, &desc, sizeof(*req));
+	if (unlikely(req == NULL)) {
+		err = VIRTIO_CRYPTO_ERR;
+		VC_LOG_ERR("Invalid descriptor");
+		goto error_exit;
+	}
+
+	switch (req->header.opcode) {
+	case VIRTIO_CRYPTO_CIPHER_ENCRYPT:
+	case VIRTIO_CRYPTO_CIPHER_DECRYPT:
+		session_id = req->header.session_id;
+
+		/* one branch to avoid unnecessary table lookup */
+		if (vcrypto->cache_session_id != session_id) {
+			err = rte_hash_lookup_data(vcrypto->session_map,
+					&session_id, (void **)&session);
+			if (unlikely(err < 0)) {
+				err = VIRTIO_CRYPTO_ERR;
+				VC_LOG_ERR("Failed to retrieve session id %lu",
+						session_id);
+				goto error_exit;
+			}
+
+			vcrypto->cache_session = session;
+			vcrypto->cache_session_id = session_id;
+		}
+
+		session = vcrypto->cache_session;
+
+		err = rte_crypto_op_attach_sym_session(op, session);
+		if (unlikely(err < 0)) {
+			err = VIRTIO_CRYPTO_ERR;
+			VC_LOG_ERR("Failed to attach session to op");
+			goto error_exit;
+		}
+
+		switch (req->u.sym_req.op_type) {
+		case VIRTIO_CRYPTO_SYM_OP_NONE:
+			err = VIRTIO_CRYPTO_NOTSUPP;
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_CIPHER:
+			err = prepare_sym_cipher_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.cipher, desc);
+			break;
+		case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING:
+			err = prepare_sym_chain_op(vcrypto, op, vc_req,
+					&req->u.sym_req.u.chain, desc);
+			break;
+		}
+		if (unlikely(err != 0)) {
+			VC_LOG_ERR("Failed to process sym request");
+			goto error_exit;
+		}
+		break;
+	default:
+		VC_LOG_ERR("Unsupported symmetric crypto request type %u",
+				req->header.opcode);
+		goto error_exit;
+	}
+
+	return 0;
+
+error_exit:
+
+	inhdr = reach_inhdr(head, mem, desc);
+	if (likely(inhdr != NULL))
+		inhdr->status = (uint8_t)err;
+
+	return -1;
+}
+
+static __rte_always_inline struct vhost_virtqueue *
+vhost_crypto_finalize_one_request(struct rte_crypto_op *op,
+		struct vhost_virtqueue *old_vq)
+{
+	struct rte_mbuf *m_src = op->sym->m_src;
+	struct rte_mbuf *m_dst = op->sym->m_dst;
+	struct vhost_crypto_data_req *vc_req = RTE_PTR_ADD(m_src,
+			sizeof(struct rte_mbuf));
+	uint16_t desc_idx;
+	int ret = 0;
+
+	if (unlikely(!vc_req)) {
+		VC_LOG_ERR("Failed to retrieve vc_req");
+		return NULL;
+	}
+
+	if (old_vq && (vc_req->vq != old_vq))
+		return vc_req->vq;
+
+	desc_idx = vc_req->desc_idx;
+
+	if (unlikely(op->status != RTE_CRYPTO_OP_STATUS_SUCCESS))
+		vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+	else {
+		if (vc_req->zero_copy == 0) {
+			ret = write_back_data(op, vc_req);
+			if (unlikely(ret != 0))
+				vc_req->inhdr->status = VIRTIO_CRYPTO_ERR;
+		}
+	}
+
+#ifdef RTE_LIBRTE_VHOST_DEBUG
+	rte_hexdump(stdout, "DST:", rte_pktmbuf_mtod(op->sym->m_dst, uint8_t *),
+			m_dst->data_len);
+#endif
+
+	vc_req->vq->used->ring[desc_idx].id = desc_idx;
+	vc_req->vq->used->ring[desc_idx].len = vc_req->len;
+
+	rte_mempool_put(m_dst->pool, (void *)m_dst);
+	rte_mempool_put(m_src->pool, (void *)m_src);
+
+	return vc_req->vq;
+}
+
+static __rte_always_inline uint16_t
+vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfd)
+{
+	uint16_t processed = 1;
+	struct vhost_virtqueue *vq, *tmp_vq;
+
+	if (unlikely(nb_ops == 0))
+		return 0;
+
+	vq = vhost_crypto_finalize_one_request(ops[0], NULL);
+	if (unlikely(vq == NULL))
+		return 0;
+	tmp_vq = vq;
+
+	while ((processed < nb_ops)) {
+		tmp_vq = vhost_crypto_finalize_one_request(ops[processed],
+				tmp_vq);
+
+		if (unlikely(vq != tmp_vq))
+			break;
+
+		processed++;
+	}
+
+	*callfd = vq->callfd;
+
+	*(volatile uint16_t *)&vq->used->idx += processed;
+
+	return processed;
+}
-- 
2.13.6

[PATCH v7 5/8] lib/librte_vhost: add public function implementation

[Fan Zhang]

This patch adds public API implementation to vhost crypto.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/rte_vhost_crypto.h    |  97 ++++++++++++-
 lib/librte_vhost/rte_vhost_version.map |   6 +
 lib/librte_vhost/vhost_crypto.c        | 255 +++++++++++++++++++++++++++++++++
 3 files changed, 357 insertions(+), 1 deletion(-)

diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h
index b6be9d87d..f9fbc0548 100644
--- a/lib/librte_vhost/rte_vhost_crypto.h
+++ b/lib/librte_vhost/rte_vhost_crypto.h
@@ -5,10 +5,105 @@
 #ifndef _VHOST_CRYPTO_H_
 #define _VHOST_CRYPTO_H_
 
+#define VHOST_CRYPTO_MBUF_POOL_SIZE		(8192)
+#define VHOST_CRYPTO_MAX_BURST_SIZE		(64)
+#define VHOST_CRYPTO_SESSION_MAP_ENTRIES	(1024) /**< Max nb sessions */
+/** max nb virtual queues in a burst for finalizing*/
+#define VIRTIO_CRYPTO_MAX_NUM_BURST_VQS		(64)
+
 enum rte_vhost_crypto_zero_copy {
 	RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE = 0,
-	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE,
+	RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE = 1,
 	RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS
 };
 
+/**
+ *  Create Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param cryptodev_id
+ *  The identifier of DPDK Cryptodev, the same cryptodev_id can be assigned to
+ *  multiple Vhost-crypto devices.
+ * @param sess_pool
+ *  The pointer to the created cryptodev session pool with the private data size
+ *  matches the target DPDK Cryptodev.
+ * @param socket_id
+ *  NUMA Socket ID to allocate resources on. *
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise
+ */
+int __rte_experimental
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id);
+
+/**
+ *  Free the Vhost-crypto instance
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @return
+ *  0 if the Vhost Crypto Instance is created successfully.
+ *  Negative integer if otherwise.
+ */
+int __rte_experimental
+rte_vhost_crypto_free(int vid);
+
+/**
+ *  Enable or disable zero copy feature
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param option
+ *  Flag of zero copy feature.
+ * @return
+ *  0 if completed successfully.
+ *  Negative integer if otherwise.
+ */
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option);
+
+/**
+ * Fetch a number of vring descriptors from virt-queue and translate to DPDK
+ * crypto operations. After this function is executed, the user can enqueue
+ * the processed ops to the target cryptodev.
+ *
+ * @param vid
+ *  The identifier of the vhost device.
+ * @param qid
+ *  Virtio queue index.
+ * @param ops
+ *  The address of an array of pointers to *rte_crypto_op* structures that must
+ *  be large enough to store *nb_ops* pointers in it.
+ * @param nb_ops
+ *  The maximum number of operations to be fetched and translated.
+ * @return
+ *  The number of fetched and processed vhost crypto request operations.
+ */
+uint16_t __rte_experimental
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops);
+/**
+ * Finalize the dequeued crypto ops. After the translated crypto ops are
+ * dequeued from the cryptodev, this function shall be called to write the
+ * processed data back to the vring descriptor (if no-copy is turned off).
+ *
+ * @param ops
+ *  The address of an array of *rte_crypto_op* structure that was dequeued
+ *  from cryptodev.
+ * @param nb_ops
+ *  The number of operations contained in the array.
+ * @callfds
+ *  The callfd number(s) contained in this burst, this shall be an array with
+ *  no less than VIRTIO_CRYPTO_MAX_NUM_BURST_VQS elements.
+ * @nb_callfds
+ *  The number of call_fd numbers exist in the callfds.
+ * @return
+ *  The number of ops processed.
+ */
+uint16_t __rte_experimental
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds);
+
 #endif /**< _VHOST_CRYPTO_H_ */
diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map
index 9d3049342..b9d338077 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -74,4 +74,10 @@ EXPERIMENTAL {
 	rte_vhost_get_log_base;
 	rte_vhost_get_vring_base;
 	rte_vhost_set_vring_base;
+	rte_vhost_crypto_create;
+	rte_vhost_crypto_free;
+	rte_vhost_crypto_fetch_requests;
+	rte_vhost_crypto_finalize_requests;
+	rte_vhost_crypto_set_zero_copy;
+
 } DPDK_18.02;
diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index 066af7d70..bd45e1466 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -3,6 +3,7 @@
  */
 #include <rte_malloc.h>
 #include <rte_hash.h>
+#include <rte_jhash.h>
 #include <rte_mbuf.h>
 #include <rte_cryptodev.h>
 
@@ -34,6 +35,13 @@
 #define VC_LOG_DBG(fmt, args...)
 #endif
 
+#define VIRTIO_CRYPTO_FEATURES ((1 << VIRTIO_F_NOTIFY_ON_EMPTY) |	\
+		(1 << VIRTIO_RING_F_INDIRECT_DESC) |			\
+		(1 << VIRTIO_RING_F_EVENT_IDX) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_CIPHER) |			\
+		(1 << VIRTIO_CRYPTO_SERVICE_MAC) |			\
+		(1 << VIRTIO_NET_F_CTRL_VQ))
+
 #define GPA_TO_VVA(t, m, a)	((t)(uintptr_t)rte_vhost_gpa_to_vva(m, a))
 
 static int
@@ -1054,3 +1062,250 @@ vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops,
 
 	return processed;
 }
+
+int __rte_experimental
+rte_vhost_crypto_create(int vid, uint8_t cryptodev_id,
+		struct rte_mempool *sess_pool, int socket_id)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct rte_hash_parameters params = {0};
+	struct vhost_crypto *vcrypto;
+	char name[128];
+	int ret;
+
+	if (!dev) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	ret = rte_vhost_driver_set_features(dev->ifname,
+			VIRTIO_CRYPTO_FEATURES);
+	if (ret < 0) {
+		VC_LOG_ERR("Error setting features");
+		return -1;
+	}
+
+	vcrypto = rte_zmalloc_socket(NULL, sizeof(*vcrypto),
+			RTE_CACHE_LINE_SIZE, socket_id);
+	if (!vcrypto) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	vcrypto->sess_pool = sess_pool;
+	vcrypto->cid = cryptodev_id;
+	vcrypto->cache_session_id = UINT64_MAX;
+	vcrypto->last_session_id = 1;
+	vcrypto->dev = dev;
+	vcrypto->option = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	snprintf(name, 127, "HASH_VHOST_CRYPT_%u", (uint32_t)vid);
+	params.name = name;
+	params.entries = VHOST_CRYPTO_SESSION_MAP_ENTRIES;
+	params.hash_func = rte_jhash;
+	params.key_len = sizeof(uint64_t);
+	params.socket_id = socket_id;
+	vcrypto->session_map = rte_hash_create(&params);
+	if (!vcrypto->session_map) {
+		VC_LOG_ERR("Failed to creath session map");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "MBUF_POOL_VM_%u", (uint32_t)vid);
+	vcrypto->mbuf_pool = rte_pktmbuf_pool_create(name,
+			VHOST_CRYPTO_MBUF_POOL_SIZE, 512,
+			sizeof(struct vhost_crypto_data_req),
+			RTE_MBUF_DEFAULT_DATAROOM * 2 + RTE_PKTMBUF_HEADROOM,
+			rte_socket_id());
+	if (!vcrypto->mbuf_pool) {
+		VC_LOG_ERR("Failed to creath mbuf pool");
+		ret = -ENOMEM;
+		goto error_exit;
+	}
+
+	dev->extern_data = vcrypto;
+	dev->extern_ops.pre_msg_handle = NULL;
+	dev->extern_ops.post_msg_handle = vhost_crypto_msg_post_handler;
+
+	return 0;
+
+error_exit:
+	if (vcrypto->session_map)
+		rte_hash_free(vcrypto->session_map);
+	if (vcrypto->mbuf_pool)
+		rte_mempool_free(vcrypto->mbuf_pool);
+
+	rte_free(vcrypto);
+
+	return ret;
+}
+
+int __rte_experimental
+rte_vhost_crypto_free(int vid)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	vcrypto = dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	rte_hash_free(vcrypto->session_map);
+	rte_mempool_free(vcrypto->mbuf_pool);
+	rte_free(vcrypto);
+
+	dev->extern_data = NULL;
+	dev->extern_ops.pre_msg_handle = NULL;
+	dev->extern_ops.post_msg_handle = NULL;
+
+	return 0;
+}
+
+int __rte_experimental
+rte_vhost_crypto_set_zero_copy(int vid, enum rte_vhost_crypto_zero_copy option)
+{
+	struct virtio_net *dev = get_device(vid);
+	struct vhost_crypto *vcrypto;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	if (unlikely(option < 0 || option >=
+			RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS)) {
+		VC_LOG_ERR("Invalid option %i", option);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	if (vcrypto->option == (uint8_t)option)
+		return 0;
+
+	if (!(rte_mempool_full(vcrypto->mbuf_pool))) {
+		VC_LOG_ERR("Cannot update zero copy as mempool is not full");
+		return -EINVAL;
+	}
+
+	vcrypto->option = (uint8_t)option;
+
+	return 0;
+}
+
+uint16_t __rte_experimental
+rte_vhost_crypto_fetch_requests(int vid, uint32_t qid,
+		struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+	struct rte_mbuf *mbufs[VHOST_CRYPTO_MAX_BURST_SIZE * 2];
+	struct virtio_net *dev = get_device(vid);
+	struct rte_vhost_memory *mem;
+	struct vhost_crypto *vcrypto;
+	struct vhost_virtqueue *vq;
+	uint16_t avail_idx;
+	uint16_t start_idx;
+	uint16_t required;
+	uint16_t count;
+	uint16_t i;
+
+	if (unlikely(dev == NULL)) {
+		VC_LOG_ERR("Invalid vid %i", vid);
+		return -EINVAL;
+	}
+
+	if (unlikely(qid >= VHOST_MAX_QUEUE_PAIRS)) {
+		VC_LOG_ERR("Invalid qid %u", qid);
+		return -EINVAL;
+	}
+
+	vcrypto = (struct vhost_crypto *)dev->extern_data;
+	if (unlikely(vcrypto == NULL)) {
+		VC_LOG_ERR("Cannot find required data, is it initialized?");
+		return -ENOENT;
+	}
+
+	vq = dev->virtqueue[qid];
+	mem = dev->mem;
+
+	avail_idx = *((volatile uint16_t *)&vq->avail->idx);
+	start_idx = vq->last_used_idx;
+	count = avail_idx - start_idx;
+	count = RTE_MIN(count, VHOST_CRYPTO_MAX_BURST_SIZE);
+	count = RTE_MIN(count, nb_ops);
+
+	if (unlikely(count == 0))
+		return 0;
+
+	/* for zero copy, we need 2 empty mbufs for src and dst, otherwise
+	 * we need only 1 mbuf as src and dst
+	 */
+	required = count * 2;
+	if (unlikely(rte_mempool_get_bulk(vcrypto->mbuf_pool, (void **)mbufs,
+			required) < 0)) {
+		VC_LOG_ERR("Insufficient memory");
+		return -ENOMEM;
+	}
+
+	for (i = 0; i < count; i++) {
+		uint16_t used_idx = (start_idx + i) & (vq->size - 1);
+		uint16_t desc_idx = vq->avail->ring[used_idx];
+		struct vring_desc *head = &vq->desc[desc_idx];
+		struct rte_crypto_op *op = ops[i];
+
+		op->sym->m_src = mbufs[i * 2];
+		op->sym->m_dst = mbufs[i * 2 + 1];
+		op->sym->m_src->data_off = 0;
+		op->sym->m_dst->data_off = 0;
+
+		if (unlikely(vhost_crypto_process_one_req(vcrypto, vq, op, head,
+				desc_idx, mem)) < 0)
+			break;
+	}
+
+	vq->last_used_idx += i;
+
+	return i;
+}
+
+uint16_t __rte_experimental
+rte_vhost_crypto_finalize_requests(struct rte_crypto_op **ops,
+		uint16_t nb_ops, int *callfds, uint16_t *nb_callfds)
+{
+	struct rte_crypto_op **tmp_ops = ops;
+	uint16_t count = 0, left = nb_ops;
+	int callfd;
+	uint16_t idx = 0;
+
+	while (left) {
+		count = vhost_crypto_complete_one_vm_requests(tmp_ops, left,
+				&callfd);
+		if (unlikely(count == 0))
+			break;
+
+		tmp_ops = &tmp_ops[count];
+		left -= count;
+
+		callfds[idx++] = callfd;
+
+		if (unlikely(idx >= VIRTIO_CRYPTO_MAX_NUM_BURST_VQS)) {
+			VC_LOG_ERR("Too many vqs");
+			break;
+		}
+	}
+
+	*nb_callfds = idx;
+
+	return nb_ops - left;
+}
-- 
2.13.6

[PATCH v7 6/8] lib/librte_vhost: update makefile

[Fan Zhang]

This patch updates the Makefile of vhost library to enable vhost crypto
compiling.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 lib/librte_vhost/Makefile    | 5 +++--
 lib/librte_vhost/meson.build | 8 ++++----
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile
index 37044ac03..2cc65f95e 100644
--- a/lib/librte_vhost/Makefile
+++ b/lib/librte_vhost/Makefile
@@ -22,9 +22,10 @@ LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev -lrte_net
 
 # all source are stored in SRCS-y
 SRCS-$(CONFIG_RTE_LIBRTE_VHOST) := fd_man.c iotlb.c socket.c vhost.c \
-					vhost_user.c virtio_net.c vdpa.c
+					vhost_user.c virtio_net.c vdpa.c vhost_crypto.c
 
 # install includes
-SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h rte_vdpa.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h rte_vdpa.h \
+					rte_vhost_crypto.h
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_vhost/meson.build b/lib/librte_vhost/meson.build
index 9e8c0e766..36f1e2754 100644
--- a/lib/librte_vhost/meson.build
+++ b/lib/librte_vhost/meson.build
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
+# Copyright(c) 2017-2018 Intel Corporation
 
 if host_machine.system() != 'linux'
 	build = false
@@ -10,6 +10,6 @@ endif
 version = 4
 allow_experimental_apis = true
 sources = files('fd_man.c', 'iotlb.c', 'socket.c', 'vhost.c', 'vhost_user.c',
-		'virtio_net.c')
-headers = files('rte_vhost.h')
-deps += ['ethdev']
+		'virtio_net.c', 'virtio_crypto.c')
+headers = files('rte_vhost.h', 'rte_vhost_crypto.h')
+deps += ['ethdev', 'cryptodev']
-- 
2.13.6

[PATCH v7 7/8] examples/vhost_crypto: add vhost crypto sample application

[Fan Zhang]

This patch adds vhost_crypto sample application to DPDK.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 examples/vhost_crypto/Makefile    |  32 +++
 examples/vhost_crypto/main.c      | 541 ++++++++++++++++++++++++++++++++++++++
 examples/vhost_crypto/meson.build |  14 +
 3 files changed, 587 insertions(+)
 create mode 100644 examples/vhost_crypto/Makefile
 create mode 100644 examples/vhost_crypto/main.c
 create mode 100644 examples/vhost_crypto/meson.build

diff --git a/examples/vhost_crypto/Makefile b/examples/vhost_crypto/Makefile
new file mode 100644
index 000000000..1bb65e8fd
--- /dev/null
+++ b/examples/vhost_crypto/Makefile
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vhost-crypto
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
diff --git a/examples/vhost_crypto/main.c b/examples/vhost_crypto/main.c
new file mode 100644
index 000000000..bc867240d
--- /dev/null
+++ b/examples/vhost_crypto/main.c
@@ -0,0 +1,541 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <stdbool.h>
+#include <assert.h>
+#include <getopt.h>
+
+#include <rte_malloc.h>
+#include <rte_cycles.h>
+#include <rte_vhost.h>
+#include <rte_cryptodev.h>
+#include <rte_vhost_crypto.h>
+
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define NB_VIRTIO_QUEUES		(1)
+#define MAX_PKT_BURST			(64)
+#define MAX_IV_LEN			(32)
+#define NB_MEMPOOL_OBJS			(8192)
+#define NB_CRYPTO_DESCRIPTORS		(4096)
+#define NB_CACHE_OBJS			(128)
+#define SESSION_MAP_ENTRIES		(1024)
+#define REFRESH_TIME_SEC		(3)
+
+#define MAX_NB_SOCKETS			(32)
+#define DEF_SOCKET_FILE			"/tmp/vhost_crypto1.socket"
+
+struct vhost_crypto_options {
+	char *socket_files[MAX_NB_SOCKETS];
+	uint32_t nb_sockets;
+	uint8_t cid;
+	uint16_t qid;
+	uint32_t zero_copy;
+	uint32_t guest_polling;
+} options;
+
+struct vhost_crypto_info {
+	int vids[MAX_NB_SOCKETS];
+	struct rte_mempool *sess_pool;
+	struct rte_mempool *cop_pool;
+	uint32_t lcore_id;
+	uint8_t cid;
+	uint32_t qid;
+	uint32_t nb_vids;
+	volatile uint32_t initialized[MAX_NB_SOCKETS];
+
+} info;
+
+#define SOCKET_FILE_KEYWORD	"socket-file"
+#define CRYPTODEV_ID_KEYWORD	"cdev-id"
+#define CRYPTODEV_QUEUE_KEYWORD	"cdev-queue-id"
+#define ZERO_COPY_KEYWORD	"zero-copy"
+#define POLLING_KEYWORD		"guest-polling"
+
+uint64_t vhost_cycles[2], last_v_cycles[2];
+uint64_t outpkt_amount;
+
+/** support *SOCKET_FILE_PATH:CRYPTODEV_ID* format */
+static int
+parse_socket_arg(char *arg)
+{
+	uint32_t nb_sockets = options.nb_sockets;
+	size_t len = strlen(arg);
+
+	if (nb_sockets >= MAX_NB_SOCKETS) {
+		RTE_LOG(ERR, USER1, "Too many socket files!\n");
+		return -ENOMEM;
+	}
+
+	options.socket_files[nb_sockets] = rte_malloc(NULL, len, 0);
+	if (!options.socket_files[nb_sockets]) {
+		RTE_LOG(ERR, USER1, "Insufficient memory\n");
+		return -ENOMEM;
+	}
+
+	rte_memcpy(options.socket_files[nb_sockets], arg, len);
+
+	options.nb_sockets++;
+
+	return 0;
+}
+
+static int
+parse_cryptodev_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if ((pm == '\0') || (end == NULL) || (*end != '\0')) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	if (pm > rte_cryptodev_count()) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.cid = (uint8_t)pm;
+
+	return 0;
+}
+
+static int
+parse_cdev_queue_id(const char *q_arg)
+{
+	char *end = NULL;
+	uint64_t pm;
+
+	/* parse decimal string */
+	pm = strtoul(q_arg, &end, 10);
+	if (pm == UINT64_MAX) {
+		RTE_LOG(ERR, USER1, "Invalid Cryptodev Queue ID %s\n", q_arg);
+		return -1;
+	}
+
+	options.qid = (uint16_t)pm;
+
+	return 0;
+}
+
+static void
+vhost_crypto_usage(const char *prgname)
+{
+	printf("%s [EAL options] --\n"
+		"  --%s SOCKET-FILE-PATH\n"
+		"  --%s CRYPTODEV_ID: crypto device id\n"
+		"  --%s CDEV_QUEUE_ID: crypto device queue id\n"
+		"  --%s: zero copy\n"
+		"  --%s: guest polling\n",
+		prgname, SOCKET_FILE_KEYWORD, CRYPTODEV_ID_KEYWORD,
+		CRYPTODEV_QUEUE_KEYWORD, ZERO_COPY_KEYWORD, POLLING_KEYWORD);
+}
+
+static int
+vhost_crypto_parse_args(int argc, char **argv)
+{
+	int opt, ret;
+	char *prgname = argv[0];
+	char **argvopt;
+	int option_index;
+	struct option lgopts[] = {
+			{SOCKET_FILE_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_ID_KEYWORD, required_argument, 0, 0},
+			{CRYPTODEV_QUEUE_KEYWORD, required_argument, 0, 0},
+			{ZERO_COPY_KEYWORD, no_argument, 0, 0},
+			{POLLING_KEYWORD, no_argument, 0, 0},
+			{NULL, 0, 0, 0}
+	};
+
+	options.cid = 0;
+	options.qid = 0;
+	options.nb_sockets = 0;
+	options.guest_polling = 0;
+	options.zero_copy = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE;
+
+	argvopt = argv;
+
+	while ((opt = getopt_long(argc, argvopt, "s:",
+				  lgopts, &option_index)) != EOF) {
+
+		switch (opt) {
+		case 0:
+			if (strcmp(lgopts[option_index].name,
+					SOCKET_FILE_KEYWORD) == 0) {
+				ret = parse_socket_arg(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_ID_KEYWORD) == 0) {
+				ret = parse_cryptodev_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					CRYPTODEV_QUEUE_KEYWORD) == 0) {
+				ret = parse_cdev_queue_id(optarg);
+				if (ret < 0) {
+					vhost_crypto_usage(prgname);
+					return ret;
+				}
+			} else if (strcmp(lgopts[option_index].name,
+					ZERO_COPY_KEYWORD) == 0) {
+				options.zero_copy =
+					RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE;
+			} else if (strcmp(lgopts[option_index].name,
+					POLLING_KEYWORD) == 0) {
+				options.guest_polling = 1;
+			} else {
+				vhost_crypto_usage(prgname);
+				return -EINVAL;
+			}
+			break;
+		default:
+			return -1;
+		}
+	}
+
+	if (options.nb_sockets == 0) {
+		options.socket_files[0] = strdup(DEF_SOCKET_FILE);
+		options.nb_sockets = 1;
+		RTE_LOG(INFO, USER1,
+				"VHOST-CRYPTO: use default socket file %s\n",
+				DEF_SOCKET_FILE);
+	}
+
+	return 0;
+}
+
+static int
+new_device(int vid)
+{
+	char path[PATH_MAX];
+	uint32_t idx, i;
+	int ret;
+
+	ret = rte_vhost_get_ifname(vid, path, PATH_MAX);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot find matched socket\n");
+		return ret;
+	}
+
+	for (idx = 0; idx < options.nb_sockets; idx++) {
+		if (strcmp(path, options.socket_files[idx]) == 0)
+			break;
+	}
+
+	if (idx == options.nb_sockets) {
+		RTE_LOG(ERR, USER1, "Cannot find recorded socket\n");
+		return -ENOENT;
+	}
+
+	for (i = 0; i < 2; i++) {
+		vhost_cycles[i] = 0;
+		last_v_cycles[i] = 0;
+	}
+
+	ret = rte_vhost_crypto_create(vid, info.cid, info.sess_pool,
+			rte_lcore_to_socket_id(info.lcore_id));
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot create vhost crypto\n");
+		return ret;
+	}
+
+	ret = rte_vhost_crypto_set_zero_copy(vid, options.zero_copy);
+	if (ret) {
+		RTE_LOG(ERR, USER1, "Cannot %s zero copy feature\n",
+				options.zero_copy == 1 ? "enable" : "disable");
+		return ret;
+	}
+
+	info.vids[idx] = vid;
+	info.initialized[idx] = 1;
+
+	rte_wmb();
+
+	RTE_LOG(INFO, USER1, "New Vhost-crypto Device %s, Device ID %d\n", path,
+			vid);
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	uint32_t i;
+
+	for (i = 0; i < info.nb_vids; i++) {
+		if (vid == info.vids[i])
+			break;
+	}
+
+	if (i == info.nb_vids) {
+		RTE_LOG(ERR, USER1, "Cannot find socket file from list\n");
+		return;
+	}
+
+	info.initialized[i] = 0;
+
+	rte_wmb();
+
+	rte_vhost_crypto_free(vid);
+
+	RTE_LOG(INFO, USER1, "Vhost Crypto Device %i Removed\n", vid);
+}
+
+static const struct vhost_device_ops virtio_crypto_device_ops = {
+	.new_device =  new_device,
+	.destroy_device = destroy_device,
+};
+
+__attribute__((unused))
+static void clrscr(void)
+{
+	system("@cls||clear");
+}
+
+static int
+vhost_crypto_worker(__rte_unused void *arg)
+{
+	struct rte_crypto_op *ops[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	struct rte_crypto_op *ops_deq[NB_VIRTIO_QUEUES][MAX_PKT_BURST + 1];
+	uint32_t nb_inflight_ops = 0;
+	uint16_t nb_callfds;
+	int callfds[VIRTIO_CRYPTO_MAX_NUM_BURST_VQS];
+	uint32_t lcore_id = rte_lcore_id();
+	uint32_t burst_size = MAX_PKT_BURST;
+	uint32_t i, j, k;
+	uint32_t to_fetch, fetched;
+	uint64_t t_start, t_end, interval;
+
+	int ret = 0;
+
+	RTE_LOG(INFO, USER1, "Processing on Core %u started\n", lcore_id);
+
+	for (i = 0; i < NB_VIRTIO_QUEUES; i++) {
+		if (rte_crypto_op_bulk_alloc(info.cop_pool,
+				RTE_CRYPTO_OP_TYPE_SYMMETRIC, ops[i],
+				burst_size) < burst_size) {
+			RTE_LOG(ERR, USER1, "Failed to alloc cops\n");
+			ret = -1;
+			goto exit;
+		}
+	}
+
+	while (1) {
+		for (i = 0; i < info.nb_vids; i++) {
+			if (unlikely(info.initialized[i] == 0))
+				continue;
+
+			for (j = 0; j < NB_VIRTIO_QUEUES; j++) {
+				t_start = rte_rdtsc_precise();
+
+				to_fetch = RTE_MIN(burst_size,
+						(NB_CRYPTO_DESCRIPTORS -
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_fetch_requests(
+						info.vids[i], j, ops[j],
+						to_fetch);
+				nb_inflight_ops += rte_cryptodev_enqueue_burst(
+						info.cid, info.qid, ops[j],
+						fetched);
+				if (unlikely(rte_crypto_op_bulk_alloc(
+						info.cop_pool,
+						RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+						ops[j], fetched) < fetched)) {
+					RTE_LOG(ERR, USER1, "Failed realloc\n");
+					return -1;
+				}
+				t_end = rte_rdtsc_precise();
+				interval = t_end - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+
+				t_start = t_end;
+				fetched = rte_cryptodev_dequeue_burst(
+						info.cid, info.qid,
+						ops_deq[j], RTE_MIN(burst_size,
+						nb_inflight_ops));
+				fetched = rte_vhost_crypto_finalize_requests(
+						ops_deq[j], fetched, callfds,
+						&nb_callfds);
+
+				nb_inflight_ops -= fetched;
+				outpkt_amount += fetched;
+
+				if (!options.guest_polling) {
+					for (k = 0; k < nb_callfds; k++)
+						eventfd_write(callfds[k],
+								(eventfd_t)1);
+				}
+
+				rte_mempool_put_bulk(info.cop_pool,
+						(void **)ops_deq[j], fetched);
+				interval = rte_rdtsc_precise() - t_start;
+
+				vhost_cycles[fetched > 0] += interval;
+			}
+		}
+	}
+exit:
+	return ret;
+}
+
+
+static void
+unregister_drivers(int socket_num)
+{
+	int ret;
+
+	ret = rte_vhost_driver_unregister(options.socket_files[socket_num]);
+	if (ret != 0)
+		RTE_LOG(ERR, USER1,
+			"Fail to unregister vhost driver for %s.\n",
+			options.socket_files[socket_num]);
+}
+
+int
+main(int argc, char *argv[])
+{
+	struct rte_cryptodev_qp_conf qp_conf = {NB_CRYPTO_DESCRIPTORS};
+	struct rte_cryptodev_config config;
+	struct rte_cryptodev_info dev_info;
+	uint32_t cryptodev_id;
+	uint32_t worker_lcore;
+	char name[128];
+	uint32_t i = 0;
+	int ret;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		return -1;
+	argc -= ret;
+	argv += ret;
+
+	ret = vhost_crypto_parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "Failed to parse arguments!\n");
+
+	info.cid = options.cid;
+	info.qid = options.qid;
+
+	worker_lcore = rte_get_next_lcore(0, 1, 0);
+	if (worker_lcore == RTE_MAX_LCORE)
+		rte_exit(EXIT_FAILURE, "Not enough lcore\n");
+
+	cryptodev_id = info.cid;
+	rte_cryptodev_info_get(cryptodev_id, &dev_info);
+	if (dev_info.max_nb_queue_pairs < info.qid + 1) {
+		RTE_LOG(ERR, USER1, "Number of queues cannot over %u",
+				dev_info.max_nb_queue_pairs);
+		goto error_exit;
+	}
+
+	config.nb_queue_pairs = dev_info.max_nb_queue_pairs;
+	config.socket_id = rte_lcore_to_socket_id(worker_lcore);
+
+	ret = rte_cryptodev_configure(cryptodev_id, &config);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to configure cryptodev %u",
+				cryptodev_id);
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "SESS_POOL_%u", worker_lcore);
+	info.sess_pool = rte_mempool_create(name, SESSION_MAP_ENTRIES,
+			rte_cryptodev_get_private_session_size(
+			cryptodev_id), 64, 0, NULL, NULL, NULL, NULL,
+			rte_lcore_to_socket_id(worker_lcore), 0);
+	if (!info.sess_pool) {
+		RTE_LOG(ERR, USER1, "Failed to create mempool");
+		goto error_exit;
+	}
+
+	snprintf(name, 127, "COPPOOL_%u", worker_lcore);
+	info.cop_pool = rte_crypto_op_pool_create(name,
+			RTE_CRYPTO_OP_TYPE_SYMMETRIC, NB_MEMPOOL_OBJS,
+			NB_CACHE_OBJS, 0, rte_lcore_to_socket_id(worker_lcore));
+
+	if (!info.cop_pool) {
+		RTE_LOG(ERR, USER1, "Lcore %u failed to create crypto pool",
+				worker_lcore);
+		ret = -1;
+		goto error_exit;
+	}
+
+	info.nb_vids = options.nb_sockets;
+	for (i = 0; i < MAX_NB_SOCKETS; i++)
+		info.vids[i] = -1;
+
+	for (i = 0; i < dev_info.max_nb_queue_pairs; i++) {
+		ret = rte_cryptodev_queue_pair_setup(cryptodev_id, i,
+				&qp_conf, rte_lcore_to_socket_id(worker_lcore),
+				info.sess_pool);
+		if (ret < 0) {
+			RTE_LOG(ERR, USER1, "Failed to configure qp %u\n",
+					info.cid);
+			goto error_exit;
+		}
+	}
+
+	ret = rte_cryptodev_start(cryptodev_id);
+	if (ret < 0) {
+		RTE_LOG(ERR, USER1, "Failed to start cryptodev %u\n", info.cid);
+		goto error_exit;
+	}
+
+	info.cid = cryptodev_id;
+	info.lcore_id = worker_lcore;
+
+	if (rte_eal_remote_launch(vhost_crypto_worker, NULL, worker_lcore)
+			< 0) {
+		RTE_LOG(ERR, USER1, "Failed to start worker lcore");
+		goto error_exit;
+	}
+
+	for (i = 0; i < options.nb_sockets; i++) {
+		if (rte_vhost_driver_register(options.socket_files[i],
+				RTE_VHOST_USER_DEQUEUE_ZERO_COPY) < 0) {
+			RTE_LOG(ERR, USER1, "socket %s already exists\n",
+					options.socket_files[i]);
+			goto error_exit;
+		}
+
+		rte_vhost_driver_callback_register(options.socket_files[i],
+				&virtio_crypto_device_ops);
+
+		if (rte_vhost_driver_start(options.socket_files[i]) < 0) {
+			RTE_LOG(ERR, USER1, "failed to start vhost driver.\n");
+			goto error_exit;
+		}
+	}
+
+	RTE_LCORE_FOREACH(worker_lcore)
+		rte_eal_wait_lcore(worker_lcore);
+
+	rte_mempool_free(info.sess_pool);
+	rte_mempool_free(info.cop_pool);
+
+	return 0;
+
+error_exit:
+	for (i = 0; i < options.nb_sockets; i++)
+		unregister_drivers(i);
+
+	rte_mempool_free(info.cop_pool);
+	rte_mempool_free(info.sess_pool);
+
+	return -1;
+}
diff --git a/examples/vhost_crypto/meson.build b/examples/vhost_crypto/meson.build
new file mode 100644
index 000000000..0f4876f06
--- /dev/null
+++ b/examples/vhost_crypto/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017-2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+allow_experimental_apis = true
+deps += ['vhost', 'cryptodev']
+cflags += ['-D_GNU_SOURCE','-D_FILE_OFFSET_BITS=64']
+sources = files(
+	'main.c'
+)
-- 
2.13.6

[PATCH v7 8/8] doc: update for vhost crypto support

[Fan Zhang]

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst       | 25 ++++++++++
 doc/guides/rel_notes/release_18_05.rst    |  5 ++
 doc/guides/sample_app_ug/index.rst        |  1 +
 doc/guides/sample_app_ug/vhost_crypto.rst | 82 +++++++++++++++++++++++++++++++
 4 files changed, 113 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index f47473609..04ab1eeb4 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -160,6 +160,31 @@ The following is an overview of some key Vhost API functions:
 
   Receives (dequeues) ``count`` packets from guest, and stored them at ``pkts``.
 
+* ``rte_vhost_crypto_create(vid, cryptodev_id, sess_mempool, socket_id)``
+
+  As an extension of new_device(), this function adds virtio-crypto workload
+  acceleration capability to the device. All crypto workload is processed by
+  DPDK cryptodev with the device ID of ``cryptodev_id``.
+
+* ``rte_vhost_crypto_free(vid)``
+
+  Frees the memory and vhost-user message handlers created in
+  rte_vhost_crypto_create().
+
+* ``rte_vhost_crypto_fetch_requests(vid, queue_id, ops, nb_ops)``
+
+  Receives (dequeues) ``nb_ops`` virtio-crypto requests from guest, parses
+  them to DPDK Crypto Operations, and fills the ``ops`` with parsing results.
+
+* ``rte_vhost_crypto_finalize_requests(queue_id, ops, nb_ops)``
+
+  After the ``ops`` are dequeued from Cryptodev, finalizes the jobs and
+  notifies the guest(s).
+
+* ``rte_vhost_crypto_set_zero_copy(vid, option)``
+
+  Enable or disable zero copy feature of the vhost crypto backend.
+
 Vhost-user Implementations
 --------------------------
 
diff --git a/doc/guides/rel_notes/release_18_05.rst b/doc/guides/rel_notes/release_18_05.rst
index 9cc77f893..73894b2e3 100644
--- a/doc/guides/rel_notes/release_18_05.rst
+++ b/doc/guides/rel_notes/release_18_05.rst
@@ -58,6 +58,11 @@ New Features
   * Added support for NVGRE, VXLAN and GENEVE filters in flow API.
   * Added support for DROP action in flow API.
 
+* **Added crypto workload support to vhost library.**
+
+  New APIs are introduced in vhost library to enable virtio crypto support
+  including session creation/deletion handling and translating virito-crypto
+  request into DPDK crypto operations. A sample application is also introduced.
 
 API Changes
 -----------
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index e87afda80..57e8354d4 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -44,6 +44,7 @@ Sample Applications User Guides
     vmdq_dcb_forwarding
     vhost
     vhost_scsi
+    vhost_crypto
     netmap_compatibility
     ip_pipeline
     test_pipeline
diff --git a/doc/guides/sample_app_ug/vhost_crypto.rst b/doc/guides/sample_app_ug/vhost_crypto.rst
new file mode 100644
index 000000000..65c86a534
--- /dev/null
+++ b/doc/guides/sample_app_ug/vhost_crypto.rst
@@ -0,0 +1,82 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2017-2018 Intel Corporation.
+
+Vhost_Crypto Sample Application
+===============================
+
+The vhost_crypto sample application implemented a simple Crypto device,
+which used as the  backend of Qemu vhost-user-crypto device. Similar with
+vhost-user-net and vhost-user-scsi device, the sample application used
+domain socket to communicate with Qemu, and the virtio ring was processed
+by vhost_crypto sample application.
+
+Testing steps
+-------------
+
+This section shows the steps how to start a VM with the crypto device as
+fast data path for critical application.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``examples`` sub-directory.
+
+Start the vhost_crypto example
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+    ./vhost_crypto [EAL options] -- [--socket-file PATH]
+        [--cdev-id ID] [--cdev-queue-id ID] [--zero-copy] [--guest-polling]
+
+where,
+
+* socket-file PATH: the path of UNIX socket file to be created, multiple
+  instances of this config item is supported. Upon absence of this item,
+  the default socket-file `/tmp/vhost_crypto1.socket` is used.
+
+* cdev-id ID: the target DPDK Cryptodev's ID to process the actual crypto
+  workload. Upon absence of this item the default value of `0` will be used.
+  For details of DPDK Cryptodev, please refer to DPDK Cryptodev Library
+  Programmers' Guide.
+
+* cdev-queue-id ID: the target DPDK Cryptodev's queue ID to process the
+  actual crypto workload. Upon absence of this item the default value of `0`
+  will be used. For details of DPDK Cryptodev, please refer to DPDK Cryptodev
+  Library Programmers' Guide.
+
+* zero-copy: the presence of this item means the ZERO-COPY feature will be
+  enabled. Otherwise it is disabled. PLEASE NOTE the ZERO-COPY feature is still
+  in experimental stage and may cause the problem like segmentation fault. If
+  the user wants to use LKCF in the guest, this feature shall be turned off.
+
+* guest-polling: the presence of this item means the application assumes the
+  guest works in polling mode, thus will NOT notify the guest completion of
+  processing.
+
+The application requires that crypto devices capable of performing
+the specified crypto operation are available on application initialization.
+This means that HW crypto device/s must be bound to a DPDK driver or
+a SW crypto device/s (virtual crypto PMD) must be created (using --vdev).
+
+.. _vhost_crypto_app_run_vm:
+
+Start the VM
+~~~~~~~~~~~~
+
+.. code-block:: console
+
+    qemu-system-x86_64 -machine accel=kvm \
+        -m $mem -object memory-backend-file,id=mem,size=$mem,\
+        mem-path=/dev/hugepages,share=on -numa node,memdev=mem \
+        -drive file=os.img,if=none,id=disk \
+        -device ide-hd,drive=disk,bootindex=0 \
+        -chardev socket,id={chardev_id},path={PATH} \
+        -object cryptodev-vhost-user,id={obj_id},chardev={chardev_id} \
+        -device virtio-crypto-pci,id={dev_id},cryptodev={obj_id} \
+        ...
+
+.. note::
+    You must check whether your Qemu can support "vhost-user-crypto" or not.
-- 
2.13.6

Re: [PATCH v7 0/8] vhost: introduce vhost crypto backend

[Maxime Coquelin]

On 04/05/2018 06:01 PM, Fan Zhang wrote:
> This patchset depends on the following patch and must be applied first:
> 
> "lib/librte_vhost: add virtio_crypto.h header file"
> (https://dpdk.org/dev/patchwork/patch/37308/)
> 
> This patchset adds crypto backend suppport to vhost library
> including a proof-of-concept sample application. The implementation
> follows the virtio-crypto specification and have been tested
> with qemu 2.11.50 (with several patches applied, detailed later)
> with Fedora 24 running in the frontend.
> 
> The vhost_crypto library acts as a "bridge" method that translate
> the virtio-crypto crypto requests to DPDK crypto operations, so it
> is purely software implementation. However it does require the user
> to provide the DPDK Cryptodev ID so it knows how to handle the
> virtio-crypto session creation and deletion mesages.
> 
> Currently the implementation supports AES-CBC-128 and HMAC-SHA1
> cipher only/chaining modes and does not support sessionless mode
> yet. The guest can use standard virtio-crypto driver to set up
> session and sends encryption/decryption requests to backend. The
> vhost-crypto sample application provided in this patchset will
> do the actual crypto work.
> 
> The following steps are involved to enable vhost-crypto support.
> 
> In the host:
> 1. Download the qemu source code.
> 
> 2. Recompile your qemu with vhost-crypto option enabled.
> 
> 3. Apply this patchset to latest DPDK code and recompile DPDK.
> 
> 4. Compile and run vhost-crypto sample application.
> 
> ./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
>   --socket-mem 2048,2048
> 
> Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
> not available. The sample application requires 2 lcores: 1 master and 1
> worker. The application will create a UNIX socket file
> /tmp/vhost_crypto1.socket.
> 
> 5. Start your qemu application. Here is my command:
> 
> qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
> -smp 2 -m 1G -hda ~/path-to-your/image.qcow \
> -object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
> -mem-prealloc -numa node,memdev=mem -chardev \
> socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
> -object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
> -device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0
> 
> 6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
> default. You shall see the following logs in your demsg:
> 
> [   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
> [   17.611083] virtio_crypto: module verification failed: signature and/or ...
> [   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
> [   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
> [   18.376100] virtio_crypto virtio0: Accelerator is ready
> 
> The virtio_crypto driver in the guest is now up and running.
> 
> 7. The rest steps can be as same as the Testing section in
> https://wiki.qemu.org/Features/VirtioCrypto
> 
> 8. It is possible to use DPDK Virtio Crypto PMD
> (https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work with
> this patchset to achieve optimal performance.
> 
> v7:
> - Rebased on top of dpdk-next-virtio
> - Removed dependency of linux virtio crypto header
> - Fixed a typo
> 
> v6:
> - Changed commit message
> - removed rte prefix in handler prototype
> 
> v5:
> - removed external ops register API.
> - patch cleaned.
> 
> v4:
> - Changed external vhost backend ops register API.
> - Fixed a bug.
> 
> v3:
> - Changed external vhost backend private data and message handling
> - Added experimental tag to rte_vhost_crypto_set_zero_copy()
> 
> v2:
> - Moved vhost_crypto_data_req data from crypto op to source mbuf.
> - Removed ZERO-COPY flag from config option and make it run-timely changeable.
> - Guest-polling mode possible.
> - Simplified vring descriptor access procedure.
> - Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.
> 
> Fan Zhang (8):
>    lib/librte_vhost: add vhost user message handlers
>    lib/librte_vhost: add virtio-crypto user message structure
>    lib/librte_vhost: add session message handler
>    lib/librte_vhost: add request handler
>    lib/librte_vhost: add public function implementation
>    lib/librte_vhost: update makefile
>    examples/vhost_crypto: add vhost crypto sample application
>    doc: update for vhost crypto support
> 
>   doc/guides/prog_guide/vhost_lib.rst       |   25 +
>   doc/guides/rel_notes/release_18_05.rst    |    5 +
>   doc/guides/sample_app_ug/index.rst        |    1 +
>   doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
>   examples/vhost_crypto/Makefile            |   32 +
>   examples/vhost_crypto/main.c              |  541 ++++++++++++
>   examples/vhost_crypto/meson.build         |   14 +
>   lib/librte_vhost/Makefile                 |    5 +-
>   lib/librte_vhost/meson.build              |    8 +-
>   lib/librte_vhost/rte_vhost.h              |    4 +
>   lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
>   lib/librte_vhost/rte_vhost_version.map    |    6 +
>   lib/librte_vhost/vhost.c                  |    2 +-
>   lib/librte_vhost/vhost.h                  |   53 +-
>   lib/librte_vhost/vhost_crypto.c           | 1311 +++++++++++++++++++++++++++++
>   lib/librte_vhost/vhost_user.c             |   33 +-
>   lib/librte_vhost/vhost_user.h             |   34 +-
>   17 files changed, 2252 insertions(+), 13 deletions(-)
>   create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
>   create mode 100644 examples/vhost_crypto/Makefile
>   create mode 100644 examples/vhost_crypto/main.c
>   create mode 100644 examples/vhost_crypto/meson.build
>   create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
>   create mode 100644 lib/librte_vhost/vhost_crypto.c
> 

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks!
Maxime

Re: [PATCH v7 0/8] vhost: introduce vhost crypto backend

[Maxime Coquelin]

On 04/05/2018 06:01 PM, Fan Zhang wrote:
> This patchset depends on the following patch and must be applied first:
> 
> "lib/librte_vhost: add virtio_crypto.h header file"
> (https://dpdk.org/dev/patchwork/patch/37308/)
> 
> This patchset adds crypto backend suppport to vhost library
> including a proof-of-concept sample application. The implementation
> follows the virtio-crypto specification and have been tested
> with qemu 2.11.50 (with several patches applied, detailed later)
> with Fedora 24 running in the frontend.
> 
> The vhost_crypto library acts as a "bridge" method that translate
> the virtio-crypto crypto requests to DPDK crypto operations, so it
> is purely software implementation. However it does require the user
> to provide the DPDK Cryptodev ID so it knows how to handle the
> virtio-crypto session creation and deletion mesages.
> 
> Currently the implementation supports AES-CBC-128 and HMAC-SHA1
> cipher only/chaining modes and does not support sessionless mode
> yet. The guest can use standard virtio-crypto driver to set up
> session and sends encryption/decryption requests to backend. The
> vhost-crypto sample application provided in this patchset will
> do the actual crypto work.
> 
> The following steps are involved to enable vhost-crypto support.
> 
> In the host:
> 1. Download the qemu source code.
> 
> 2. Recompile your qemu with vhost-crypto option enabled.
> 
> 3. Apply this patchset to latest DPDK code and recompile DPDK.
> 
> 4. Compile and run vhost-crypto sample application.
> 
> ./examples/vhost_crypto/build/vhost-crypto -l 11,12 -w 0000:86:01.0 \
>   --socket-mem 2048,2048
> 
> Where 0000:86:01.0 is the QAT PCI address. You may use AES-NI-MB if it is
> not available. The sample application requires 2 lcores: 1 master and 1
> worker. The application will create a UNIX socket file
> /tmp/vhost_crypto1.socket.
> 
> 5. Start your qemu application. Here is my command:
> 
> qemu/x86_64-softmmu/qemu-system-x86_64 -machine accel=kvm -cpu host \
> -smp 2 -m 1G -hda ~/path-to-your/image.qcow \
> -object memory-backend-file,id=mem,size=1G,mem-path=/dev/hugepages,share=on \
> -mem-prealloc -numa node,memdev=mem -chardev \
> socket,id=charcrypto0,path=/tmp/vhost_crypto1.socket \
> -object cryptodev-vhost-user,id=cryptodev0,chardev=charcrypto0 \
> -device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0
> 
> 6. Once guest is booted. The Linux virtio_crypto kernel module is loaded by
> default. You shall see the following logs in your demsg:
> 
> [   17.611044] virtio_crypto: loading out-of-tree module taints kernel.
> [   17.611083] virtio_crypto: module verification failed: signature and/or ...
> [   17.611723] virtio_crypto virtio0: max_queues: 1, max_cipher_key_len: ...
> [   17.612156] virtio_crypto virtio0: will run requests pump with realtime ...
> [   18.376100] virtio_crypto virtio0: Accelerator is ready
> 
> The virtio_crypto driver in the guest is now up and running.
> 
> 7. The rest steps can be as same as the Testing section in
> https://wiki.qemu.org/Features/VirtioCrypto
> 
> 8. It is possible to use DPDK Virtio Crypto PMD
> (https://dpdk.org/dev/patchwork/patch/36921/) in the guest to work with
> this patchset to achieve optimal performance.
> 
> v7:
> - Rebased on top of dpdk-next-virtio
> - Removed dependency of linux virtio crypto header
> - Fixed a typo
> 
> v6:
> - Changed commit message
> - removed rte prefix in handler prototype
> 
> v5:
> - removed external ops register API.
> - patch cleaned.
> 
> v4:
> - Changed external vhost backend ops register API.
> - Fixed a bug.
> 
> v3:
> - Changed external vhost backend private data and message handling
> - Added experimental tag to rte_vhost_crypto_set_zero_copy()
> 
> v2:
> - Moved vhost_crypto_data_req data from crypto op to source mbuf.
> - Removed ZERO-COPY flag from config option and make it run-timely changeable.
> - Guest-polling mode possible.
> - Simplified vring descriptor access procedure.
> - Work with both LKCF and DPDK Virtio-Crypto PMD guest drivers.
> 
> Fan Zhang (8):
>    lib/librte_vhost: add vhost user message handlers
>    lib/librte_vhost: add virtio-crypto user message structure
>    lib/librte_vhost: add session message handler
>    lib/librte_vhost: add request handler
>    lib/librte_vhost: add public function implementation
>    lib/librte_vhost: update makefile
>    examples/vhost_crypto: add vhost crypto sample application
>    doc: update for vhost crypto support
> 
>   doc/guides/prog_guide/vhost_lib.rst       |   25 +
>   doc/guides/rel_notes/release_18_05.rst    |    5 +
>   doc/guides/sample_app_ug/index.rst        |    1 +
>   doc/guides/sample_app_ug/vhost_crypto.rst |   82 ++
>   examples/vhost_crypto/Makefile            |   32 +
>   examples/vhost_crypto/main.c              |  541 ++++++++++++
>   examples/vhost_crypto/meson.build         |   14 +
>   lib/librte_vhost/Makefile                 |    5 +-
>   lib/librte_vhost/meson.build              |    8 +-
>   lib/librte_vhost/rte_vhost.h              |    4 +
>   lib/librte_vhost/rte_vhost_crypto.h       |  109 +++
>   lib/librte_vhost/rte_vhost_version.map    |    6 +
>   lib/librte_vhost/vhost.c                  |    2 +-
>   lib/librte_vhost/vhost.h                  |   53 +-
>   lib/librte_vhost/vhost_crypto.c           | 1311 +++++++++++++++++++++++++++++
>   lib/librte_vhost/vhost_user.c             |   33 +-
>   lib/librte_vhost/vhost_user.h             |   34 +-
>   17 files changed, 2252 insertions(+), 13 deletions(-)
>   create mode 100644 doc/guides/sample_app_ug/vhost_crypto.rst
>   create mode 100644 examples/vhost_crypto/Makefile
>   create mode 100644 examples/vhost_crypto/main.c
>   create mode 100644 examples/vhost_crypto/meson.build
>   create mode 100644 lib/librte_vhost/rte_vhost_crypto.h
>   create mode 100644 lib/librte_vhost/vhost_crypto.c
> 


Applied to dpdk-next-virtio/master.

Thanks,
Maxime

Re: [PATCH v7 7/8] examples/vhost_crypto: add vhost crypto sample application

[Thomas Monjalon]

Hi,

05/04/2018 18:01, Fan Zhang:
> This patch adds vhost_crypto sample application to DPDK.
> 
> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> ---
>  examples/vhost_crypto/Makefile    |  32 +++
>  examples/vhost_crypto/main.c      | 541 ++++++++++++++++++++++++++++++++++++++
>  examples/vhost_crypto/meson.build |  14 +
>  3 files changed, 587 insertions(+)

There are 3 misses:

- not in examples/Makefile
- not in MAINTAINERS
- no documentation in doc/guides/sample_app_ug/

It won't be accepted in master as-is.

Re: [PATCH v7 7/8] examples/vhost_crypto: add vhost crypto sample application

[Thomas Monjalon]

15/04/2018 16:34, Thomas Monjalon:
> Hi,
> 
> 05/04/2018 18:01, Fan Zhang:
> > This patch adds vhost_crypto sample application to DPDK.
> > 
> > Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
> > ---
> >  examples/vhost_crypto/Makefile    |  32 +++
> >  examples/vhost_crypto/main.c      | 541 ++++++++++++++++++++++++++++++++++++++
> >  examples/vhost_crypto/meson.build |  14 +
> >  3 files changed, 587 insertions(+)
> 
> There are 3 misses:
> 
> - not in examples/Makefile
> - not in MAINTAINERS
> - no documentation in doc/guides/sample_app_ug/
> 
> It won't be accepted in master as-is.

The doc is (curiously) in the next patch. I will move it here.
I will also fix the Makefile and the MAINTAINERS file:

ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
endif

Vhost-user
M: Maxime Coquelin <maxime.coquelin@redhat.com>
M: Jianfeng Tan <jianfeng.tan@intel.com>
T: git://dpdk.org/next/dpdk-next-virtio
F: lib/librte_vhost/
[...]
F: examples/vhost_crypto/


Ferruh, Maxime, I hope you agree with this last-minute fix.

Re: [PATCH v7 7/8] examples/vhost_crypto: add vhost crypto sample application

[Maxime Coquelin]

On 04/15/2018 07:35 PM, Thomas Monjalon wrote:
> 15/04/2018 16:34, Thomas Monjalon:
>> Hi,
>>
>> 05/04/2018 18:01, Fan Zhang:
>>> This patch adds vhost_crypto sample application to DPDK.
>>>
>>> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
>>> ---
>>>   examples/vhost_crypto/Makefile    |  32 +++
>>>   examples/vhost_crypto/main.c      | 541 ++++++++++++++++++++++++++++++++++++++
>>>   examples/vhost_crypto/meson.build |  14 +
>>>   3 files changed, 587 insertions(+)
>>
>> There are 3 misses:
>>
>> - not in examples/Makefile
>> - not in MAINTAINERS
>> - no documentation in doc/guides/sample_app_ug/
>>
>> It won't be accepted in master as-is.
> 
> The doc is (curiously) in the next patch. I will move it here.
> I will also fix the Makefile and the MAINTAINERS file:
> 
> ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
> DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
> endif
> 
> Vhost-user
> M: Maxime Coquelin <maxime.coquelin@redhat.com>
> M: Jianfeng Tan <jianfeng.tan@intel.com>
> T: git://dpdk.org/next/dpdk-next-virtio
> F: lib/librte_vhost/
> [...]
> F: examples/vhost_crypto/
> 
> 
> Ferruh, Maxime, I hope you agree with this last-minute fix.
> 
> 

Agreed. thanks.

Maxime