* [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD
@ 2020-03-16 13:21 Konstantin Ananyev
2020-03-16 13:21 ` [dpdk-dev] [PATCH 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
` (3 more replies)
0 siblings, 4 replies; 18+ messages in thread
From: Konstantin Ananyev @ 2020-03-16 13:21 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
Add support for CPU crypto API into aesni_mb PMD
and extend cryptodev UT to cover this new functionality.
Konstantin Ananyev (2):
crypto/aesni_mb: support CPU crypto
test/crypto: add CPU crypto mode for AESNI MB
app/test/test_cryptodev.c | 146 +++++--
doc/guides/cryptodevs/aesni_mb.rst | 3 +
doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
.../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 384 ++++++++++++++++--
.../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
6 files changed, 476 insertions(+), 65 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH 1/2] crypto/aesni_mb: support CPU crypto
2020-03-16 13:21 [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Konstantin Ananyev
@ 2020-03-16 13:21 ` Konstantin Ananyev
2020-03-16 13:21 ` [dpdk-dev] [PATCH 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
` (2 subsequent siblings)
3 siblings, 0 replies; 18+ messages in thread
From: Konstantin Ananyev @ 2020-03-16 13:21 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
Add support for CPU crypto mode by introducing required handler.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
---
doc/guides/cryptodevs/aesni_mb.rst | 3 +
doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
.../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 384 ++++++++++++++++--
.../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
5 files changed, 361 insertions(+), 34 deletions(-)
diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst
index 465d20d88..02a0a499d 100644
--- a/doc/guides/cryptodevs/aesni_mb.rst
+++ b/doc/guides/cryptodevs/aesni_mb.rst
@@ -12,6 +12,9 @@ support for utilizing Intel multi buffer library, see the white paper
The AES-NI MB PMD has current only been tested on Fedora 21 64-bit with gcc.
+The AES-NI GCM PMD supports synchronous mode of operation with
+``rte_cryptodev_sym_cpu_crypto_process`` function call.
+
Features
--------
diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini
index ee6a07460..6cc4c82e8 100644
--- a/doc/guides/cryptodevs/features/aesni_mb.ini
+++ b/doc/guides/cryptodevs/features/aesni_mb.ini
@@ -12,6 +12,7 @@ CPU AVX2 = Y
CPU AVX512 = Y
CPU AESNI = Y
OOP LB In LB Out = Y
+CPU crypto = Y
;
; Supported crypto algorithms of the 'aesni_mb' crypto driver.
diff --git a/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
index 3456693c2..ce3ea9add 100644
--- a/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
@@ -275,6 +275,9 @@ aesni_mb_set_session_parameters(const MB_MGR *mb_mgr,
/** device specific operations function pointer structure */
extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;
-
+extern uint32_t
+aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs,
+ struct rte_crypto_sym_vec *vec);
#endif /* _AESNI_MB_PMD_PRIVATE_H_ */
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 33f416745..2febd6278 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -11,6 +11,7 @@
#include <rte_bus_vdev.h>
#include <rte_malloc.h>
#include <rte_cpuflags.h>
+#include <rte_per_lcore.h>
#include "aesni_mb_pmd_private.h"
@@ -19,6 +20,12 @@
#define HMAC_MAX_BLOCK_SIZE 128
static uint8_t cryptodev_driver_id;
+/*
+ * Needed to support CPU-CRYPTO API (rte_cryptodev_sym_cpu_crypto_process),
+ * as we still use JOB based API even for synchronous processing.
+ */
+static RTE_DEFINE_PER_LCORE(MB_MGR *, sync_mb_mgr);
+
typedef void (*hash_one_block_t)(const void *data, void *digest);
typedef void (*aes_keyexp_t)(const void *key, void *enc_exp_keys, void *dec_exp_keys);
@@ -808,6 +815,119 @@ auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session,
(UINT64_MAX - u_src + u_dst + 1);
}
+static inline void
+set_cpu_mb_job_params(JOB_AES_HMAC *job, struct aesni_mb_session *session,
+ union rte_crypto_sym_ofs sofs, void *buf, uint32_t len,
+ void *iv, void *aad, void *digest, void *udata)
+{
+ /* Set crypto operation */
+ job->chain_order = session->chain_order;
+
+ /* Set cipher parameters */
+ job->cipher_direction = session->cipher.direction;
+ job->cipher_mode = session->cipher.mode;
+
+ job->aes_key_len_in_bytes = session->cipher.key_length_in_bytes;
+
+ /* Set authentication parameters */
+ job->hash_alg = session->auth.algo;
+ job->iv = iv;
+
+ switch (job->hash_alg) {
+ case AES_XCBC:
+ job->u.XCBC._k1_expanded = session->auth.xcbc.k1_expanded;
+ job->u.XCBC._k2 = session->auth.xcbc.k2;
+ job->u.XCBC._k3 = session->auth.xcbc.k3;
+
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ break;
+
+ case AES_CCM:
+ job->u.CCM.aad = (uint8_t *)aad + 18;
+ job->u.CCM.aad_len_in_bytes = session->aead.aad_len;
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ job->iv++;
+ break;
+
+ case AES_CMAC:
+ job->u.CMAC._key_expanded = session->auth.cmac.expkey;
+ job->u.CMAC._skey1 = session->auth.cmac.skey1;
+ job->u.CMAC._skey2 = session->auth.cmac.skey2;
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ break;
+
+ case AES_GMAC:
+ if (session->cipher.mode == GCM) {
+ job->u.GCM.aad = aad;
+ job->u.GCM.aad_len_in_bytes = session->aead.aad_len;
+ } else {
+ /* For GMAC */
+ job->u.GCM.aad = buf;
+ job->u.GCM.aad_len_in_bytes = len;
+ job->cipher_mode = GCM;
+ }
+ job->aes_enc_key_expanded = &session->cipher.gcm_key;
+ job->aes_dec_key_expanded = &session->cipher.gcm_key;
+ break;
+
+ default:
+ job->u.HMAC._hashed_auth_key_xor_ipad =
+ session->auth.pads.inner;
+ job->u.HMAC._hashed_auth_key_xor_opad =
+ session->auth.pads.outer;
+
+ if (job->cipher_mode == DES3) {
+ job->aes_enc_key_expanded =
+ session->cipher.exp_3des_keys.ks_ptr;
+ job->aes_dec_key_expanded =
+ session->cipher.exp_3des_keys.ks_ptr;
+ } else {
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ }
+ }
+
+ /*
+ * Multi-buffer library current only support returning a truncated
+ * digest length as specified in the relevant IPsec RFCs
+ */
+
+ /* Set digest location and length */
+ job->auth_tag_output = digest;
+ job->auth_tag_output_len_in_bytes = session->auth.gen_digest_len;
+
+ /* Set IV parameters */
+ job->iv_len_in_bytes = session->iv.length;
+
+ /* Data Parameters */
+ job->src = buf;
+ job->dst = (uint8_t *)buf + sofs.ofs.cipher.head;
+ job->cipher_start_src_offset_in_bytes = sofs.ofs.cipher.head;
+ job->hash_start_src_offset_in_bytes = sofs.ofs.auth.head;
+ if (job->hash_alg == AES_GMAC && session->cipher.mode != GCM) {
+ job->msg_len_to_hash_in_bytes = 0;
+ job->msg_len_to_cipher_in_bytes = 0;
+ } else {
+ job->msg_len_to_hash_in_bytes = len - sofs.ofs.auth.head -
+ sofs.ofs.auth.tail;
+ job->msg_len_to_cipher_in_bytes = len - sofs.ofs.cipher.head -
+ sofs.ofs.cipher.tail;
+ }
+
+ job->user_data = udata;
+}
+
/**
* Process a crypto operation and complete a JOB_AES_HMAC job structure for
* submission to the multi buffer library for processing.
@@ -1100,6 +1220,15 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)
return op;
}
+static inline void
+post_process_mb_sync_job(JOB_AES_HMAC *job)
+{
+ uint32_t *st;
+
+ st = job->user_data;
+ st[0] = (job->status == STS_COMPLETED) ? 0 : EBADMSG;
+}
+
/**
* Process a completed JOB_AES_HMAC job and keep processing jobs until
* get_completed_job return NULL
@@ -1136,6 +1265,26 @@ handle_completed_jobs(struct aesni_mb_qp *qp, JOB_AES_HMAC *job,
return processed_jobs;
}
+static inline uint32_t
+handle_completed_sync_jobs(JOB_AES_HMAC *job, MB_MGR *mb_mgr)
+{
+ uint32_t i;
+
+ for (i = 0; job != NULL; i++, job = IMB_GET_COMPLETED_JOB(mb_mgr))
+ post_process_mb_sync_job(job);
+
+ return i;
+}
+
+static inline uint32_t
+flush_mb_sync_mgr(MB_MGR *mb_mgr)
+{
+ JOB_AES_HMAC *job;
+
+ job = IMB_FLUSH_JOB(mb_mgr);
+ return handle_completed_sync_jobs(job, mb_mgr);
+}
+
static inline uint16_t
flush_mb_mgr(struct aesni_mb_qp *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
@@ -1239,8 +1388,199 @@ aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
return processed_jobs;
}
+static MB_MGR *
+alloc_init_mb_mgr(enum aesni_mb_vector_mode vector_mode)
+{
+ MB_MGR *mb_mgr = alloc_mb_mgr(0);
+ if (mb_mgr == NULL)
+ return NULL;
+
+ switch (vector_mode) {
+ case RTE_AESNI_MB_SSE:
+ init_mb_mgr_sse(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX:
+ init_mb_mgr_avx(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX2:
+ init_mb_mgr_avx2(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX512:
+ init_mb_mgr_avx512(mb_mgr);
+ break;
+ default:
+ AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", vector_mode);
+ free_mb_mgr(mb_mgr);
+ return NULL;
+ }
+
+ return mb_mgr;
+}
+
+static inline void
+aesni_mb_fill_error_code(struct rte_crypto_sym_vec *vec, int32_t err)
+{
+ uint32_t i;
+
+ for (i = 0; i != vec->num; ++i)
+ vec->status[i] = err;
+}
+
+static inline int
+check_crypto_sgl(union rte_crypto_sym_ofs so, const struct rte_crypto_sgl *sgl)
+{
+ /* no multi-seg support with current AESNI-MB PMD */
+ if (sgl->num != 1)
+ return ENOTSUP;
+ else if (so.ofs.cipher.head + so.ofs.cipher.tail > sgl->vec[0].len)
+ return EINVAL;
+ return 0;
+}
+
+static inline JOB_AES_HMAC *
+submit_sync_job(MB_MGR *mb_mgr)
+{
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+ return IMB_SUBMIT_JOB(mb_mgr);
+#else
+ return IMB_SUBMIT_JOB_NOCHECK(mb_mgr);
+#endif
+}
+
+static inline uint32_t
+generate_sync_dgst(struct rte_crypto_sym_vec *vec,
+ const uint8_t dgst[][DIGEST_LENGTH_MAX], uint32_t len)
+{
+ uint32_t i, k;
+
+ for (i = 0, k = 0; i != vec->num; i++) {
+ if (vec->status[i] == 0) {
+ memcpy(vec->digest[i], dgst[i], len);
+ k++;
+ }
+ }
+
+ return k;
+}
+
+static inline uint32_t
+verify_sync_dgst(struct rte_crypto_sym_vec *vec,
+ const uint8_t dgst[][DIGEST_LENGTH_MAX], uint32_t len)
+{
+ uint32_t i, k;
+
+ for (i = 0, k = 0; i != vec->num; i++) {
+ if (vec->status[i] == 0) {
+ if (memcmp(vec->digest[i], dgst[i], len) != 0)
+ vec->status[i] = EBADMSG;
+ else
+ k++;
+ }
+ }
+
+ return k;
+}
+
+uint32_t
+aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs,
+ struct rte_crypto_sym_vec *vec)
+{
+ int32_t ret;
+ uint32_t i, j, k, len;
+ void *buf;
+ JOB_AES_HMAC *job;
+ MB_MGR *mb_mgr;
+ struct aesni_mb_private *priv;
+ struct aesni_mb_session *s;
+ uint8_t tmp_dgst[vec->num][DIGEST_LENGTH_MAX];
+
+ s = get_sym_session_private_data(sess, dev->driver_id);
+ if (s == NULL) {
+ aesni_mb_fill_error_code(vec, EINVAL);
+ return 0;
+ }
+
+ /* get per-thread MB MGR, create one if needed */
+ mb_mgr = RTE_PER_LCORE(sync_mb_mgr);
+ if (mb_mgr == NULL) {
+
+ priv = dev->data->dev_private;
+ mb_mgr = alloc_init_mb_mgr(priv->vector_mode);
+ if (mb_mgr == NULL) {
+ aesni_mb_fill_error_code(vec, ENOMEM);
+ return 0;
+ }
+ RTE_PER_LCORE(sync_mb_mgr) = mb_mgr;
+ }
+
+ for (i = 0, j = 0, k = 0; i != vec->num; i++) {
+
+
+ ret = check_crypto_sgl(sofs, vec->sgl + i);
+ if (ret != 0) {
+ vec->status[i] = ret;
+ continue;
+ }
+
+ buf = vec->sgl[i].vec[0].base;
+ len = vec->sgl[i].vec[0].len;
+
+ job = IMB_GET_NEXT_JOB(mb_mgr);
+ if (job == NULL) {
+ k += flush_mb_sync_mgr(mb_mgr);
+ job = IMB_GET_NEXT_JOB(mb_mgr);
+ RTE_ASSERT(job != NULL);
+ }
+
+ /* Submit job for processing */
+ set_cpu_mb_job_params(job, s, sofs, buf, len,
+ vec->iv[i], vec->aad[i], tmp_dgst[i],
+ &vec->status[i]);
+ job = submit_sync_job(mb_mgr);
+ j++;
+
+ /* handle completed jobs */
+ k += handle_completed_sync_jobs(job, mb_mgr);
+ }
+
+ /* flush remaining jobs */
+ while (k != j)
+ k += flush_mb_sync_mgr(mb_mgr);
+
+ /* finish processing for successful jobs: check/update digest */
+ if (k != 0) {
+ if (s->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY)
+ k = verify_sync_dgst(vec, tmp_dgst,
+ s->auth.req_digest_len);
+ else
+ k = generate_sync_dgst(vec, tmp_dgst,
+ s->auth.req_digest_len);
+ }
+
+ return k;
+}
+
static int cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev);
+static uint64_t
+vec_mode_to_flags(enum aesni_mb_vector_mode mode)
+{
+ switch (mode) {
+ case RTE_AESNI_MB_SSE:
+ return RTE_CRYPTODEV_FF_CPU_SSE;
+ case RTE_AESNI_MB_AVX:
+ return RTE_CRYPTODEV_FF_CPU_AVX;
+ case RTE_AESNI_MB_AVX2:
+ return RTE_CRYPTODEV_FF_CPU_AVX2;
+ case RTE_AESNI_MB_AVX512:
+ return RTE_CRYPTODEV_FF_CPU_AVX512;
+ default:
+ AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", mode);
+ return 0;
+ }
+}
+
static int
cryptodev_aesni_mb_create(const char *name,
struct rte_vdev_device *vdev,
@@ -1276,7 +1616,8 @@ cryptodev_aesni_mb_create(const char *name,
dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
- RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+ RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+ RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO;
/* Check CPU for support for AES instruction set */
if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES))
@@ -1284,30 +1625,12 @@ cryptodev_aesni_mb_create(const char *name,
else
AESNI_MB_LOG(WARNING, "AES instructions not supported by CPU");
- mb_mgr = alloc_mb_mgr(0);
- if (mb_mgr == NULL)
- return -ENOMEM;
+ dev->feature_flags |= vec_mode_to_flags(vector_mode);
- switch (vector_mode) {
- case RTE_AESNI_MB_SSE:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_SSE;
- init_mb_mgr_sse(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX;
- init_mb_mgr_avx(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX2:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX2;
- init_mb_mgr_avx2(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX512:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX512;
- init_mb_mgr_avx512(mb_mgr);
- break;
- default:
- AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", vector_mode);
- goto error_exit;
+ mb_mgr = alloc_init_mb_mgr(vector_mode);
+ if (mb_mgr == NULL) {
+ rte_cryptodev_pmd_destroy(dev);
+ return -ENOMEM;
}
/* Set vector instructions mode supported */
@@ -1319,16 +1642,7 @@ cryptodev_aesni_mb_create(const char *name,
AESNI_MB_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
imb_get_version_str());
-
return 0;
-
-error_exit:
- if (mb_mgr)
- free_mb_mgr(mb_mgr);
-
- rte_cryptodev_pmd_destroy(dev);
-
- return -1;
}
static int
@@ -1377,6 +1691,10 @@ cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev)
internals = cryptodev->data->dev_private;
free_mb_mgr(internals->mb_mgr);
+ if (RTE_PER_LCORE(sync_mb_mgr)) {
+ free_mb_mgr(RTE_PER_LCORE(sync_mb_mgr));
+ RTE_PER_LCORE(sync_mb_mgr) = NULL;
+ }
return rte_cryptodev_pmd_destroy(cryptodev);
}
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
index d8609ad11..e8371ea69 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
@@ -805,6 +805,8 @@ struct rte_cryptodev_ops aesni_mb_pmd_ops = {
.queue_pair_release = aesni_mb_pmd_qp_release,
.queue_pair_count = aesni_mb_pmd_qp_count,
+ .sym_cpu_process = aesni_mb_cpu_crypto_process_bulk,
+
.sym_session_get_size = aesni_mb_pmd_sym_session_get_size,
.sym_session_configure = aesni_mb_pmd_sym_session_configure,
.sym_session_clear = aesni_mb_pmd_sym_session_clear
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH 2/2] test/crypto: add CPU crypto mode for AESNI MB
2020-03-16 13:21 [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Konstantin Ananyev
2020-03-16 13:21 ` [dpdk-dev] [PATCH 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
@ 2020-03-16 13:21 ` Konstantin Ananyev
2020-04-01 14:18 ` [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 " Konstantin Ananyev
3 siblings, 0 replies; 18+ messages in thread
From: Konstantin Ananyev @ 2020-03-16 13:21 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
This patch adds ability to run unit tests in cpu crypto mode
for AESNI MB cryptodev.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
---
app/test/test_cryptodev.c | 146 ++++++++++++++++++++++++++++++--------
1 file changed, 115 insertions(+), 31 deletions(-)
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 7b1ef5c86..7235b701b 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -143,7 +143,7 @@ ceil_byte_length(uint32_t num_bits)
}
static void
-process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
+process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
{
int32_t n, st;
void *iv;
@@ -155,8 +155,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
sop = op->sym;
- n = rte_crypto_mbuf_to_vec(sop->m_src, sop->auth.data.offset,
- sop->auth.data.length, vec, RTE_DIM(vec));
+ n = rte_crypto_mbuf_to_vec(sop->m_src, sop->aead.data.offset,
+ sop->aead.data.length, vec, RTE_DIM(vec));
if (n < 0 || n != sop->m_src->nb_segs) {
op->status = RTE_CRYPTO_OP_STATUS_ERROR;
@@ -168,8 +168,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
symvec.sgl = &sgl;
iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
symvec.iv = &iv;
- symvec.aad = NULL;
- symvec.digest = (void **)&sop->auth.digest.data;
+ symvec.aad = (void **)&sop->aead.aad.data;
+ symvec.digest = (void **)&sop->aead.digest.data;
symvec.status = &st;
symvec.num = 1;
@@ -184,9 +184,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
}
-
static void
-process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
+process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
{
int32_t n, st;
void *iv;
@@ -198,8 +197,8 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
sop = op->sym;
- n = rte_crypto_mbuf_to_vec(sop->m_src, sop->aead.data.offset,
- sop->aead.data.length, vec, RTE_DIM(vec));
+ n = rte_crypto_mbuf_to_vec(sop->m_src, sop->auth.data.offset,
+ sop->auth.data.length, vec, RTE_DIM(vec));
if (n < 0 || n != sop->m_src->nb_segs) {
op->status = RTE_CRYPTO_OP_STATUS_ERROR;
@@ -212,11 +211,14 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
symvec.iv = &iv;
symvec.aad = (void **)&sop->aead.aad.data;
- symvec.digest = (void **)&sop->aead.digest.data;
+ symvec.digest = (void **)&sop->auth.digest.data;
symvec.status = &st;
symvec.num = 1;
ofs.raw = 0;
+ ofs.ofs.cipher.head = sop->cipher.data.offset - sop->auth.data.offset;
+ ofs.ofs.cipher.tail = (sop->auth.data.offset + sop->auth.data.length) -
+ (sop->cipher.data.offset + sop->cipher.data.length);
n = rte_cryptodev_sym_cpu_crypto_process(dev_id, sop->session, ofs,
&symvec);
@@ -1538,8 +1540,14 @@ test_AES_CBC_HMAC_SHA1_encrypt_digest(void)
sym_op->cipher.data.length = QUOTE_512_BYTES;
/* Process crypto operation */
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -1689,8 +1697,14 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
sym_op->cipher.data.length = QUOTE_512_BYTES;
/* Process crypto operation */
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -2637,8 +2651,13 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
- ut_params->op);
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+
ut_params->obuf = ut_params->op->sym->m_src;
TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
ut_params->digest = rte_pktmbuf_mtod(ut_params->obuf, uint8_t *)
@@ -8715,6 +8734,9 @@ test_stats(void)
struct rte_cryptodev *dev;
cryptodev_stats_get_t temp_pfn;
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ return -ENOTSUP;
+
/* Verify the capabilities */
struct rte_cryptodev_sym_capability_idx cap_idx;
cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -8886,8 +8908,14 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
if (MD5_HMAC_create_op(ut_params, test_case, &plaintext))
return TEST_FAILED;
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -8938,8 +8966,14 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
if (MD5_HMAC_create_op(ut_params, test_case, &plaintext))
return TEST_FAILED;
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"HMAC_MD5 crypto op processing failed");
@@ -9852,7 +9886,8 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
ut_params->op->sym->m_src = ut_params->ibuf;
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
else
TEST_ASSERT_NOT_NULL(
process_crypto_request(ts_params->valid_devs[0],
@@ -9968,7 +10003,8 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
ut_params->op->sym->m_src = ut_params->ibuf;
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
else
TEST_ASSERT_NOT_NULL(
process_crypto_request(ts_params->valid_devs[0],
@@ -10528,10 +10564,17 @@ test_authentication_verify_fail_when_data_corruption(
else
tag_corruption(plaintext, reference->plaintext.len);
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
-
- TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "authentication not failed");
+ } else {
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ }
return 0;
}
@@ -10593,7 +10636,8 @@ test_authentication_verify_GMAC_fail_when_corruption(
tag_corruption(plaintext, reference->aad.len);
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
RTE_CRYPTO_OP_STATUS_SUCCESS,
"authentication not failed");
@@ -10666,10 +10710,17 @@ test_authenticated_decryption_fail_when_corruption(
else
tag_corruption(ciphertext, reference->ciphertext.len);
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
-
- TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "authentication not failed");
+ } else {
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ }
return 0;
}
@@ -10757,8 +10808,12 @@ test_authenticated_encryt_with_esn(
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
+ else
+ ut_params->op = process_crypto_request(
+ ts_params->valid_devs[0], ut_params->op);
TEST_ASSERT_NOT_NULL(ut_params->op, "no crypto operation returned");
@@ -10873,7 +10928,11 @@ test_authenticated_decrypt_with_esn(
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
ut_params->op);
TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
@@ -13377,6 +13436,29 @@ test_cryptodev_aesni_mb(void /*argv __rte_unused, int argc __rte_unused*/)
return unit_test_suite_runner(&cryptodev_testsuite);
}
+static int
+test_cryptodev_cpu_aesni_mb(void)
+{
+ int32_t rc;
+ enum rte_security_session_action_type at;
+
+ gbl_driver_id = rte_cryptodev_driver_id_get(
+ RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+ if (gbl_driver_id == -1) {
+ RTE_LOG(ERR, USER1, "AESNI MB PMD must be loaded. Check if "
+ "CONFIG_RTE_LIBRTE_PMD_AESNI_MB is enabled "
+ "in config file to run this testsuite.\n");
+ return TEST_SKIPPED;
+ }
+
+ at = gbl_action_type;
+ gbl_action_type = RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
+ rc = unit_test_suite_runner(&cryptodev_testsuite);
+ gbl_action_type = at;
+ return rc;
+}
+
static int
test_cryptodev_openssl(void)
{
@@ -13668,6 +13750,8 @@ test_cryptodev_nitrox(void)
REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+ test_cryptodev_cpu_aesni_mb);
REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD
2020-03-16 13:21 [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Konstantin Ananyev
2020-03-16 13:21 ` [dpdk-dev] [PATCH 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
2020-03-16 13:21 ` [dpdk-dev] [PATCH 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
@ 2020-04-01 14:18 ` Akhil Goyal
2020-04-01 15:09 ` Ananyev, Konstantin
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 " Konstantin Ananyev
3 siblings, 1 reply; 18+ messages in thread
From: Akhil Goyal @ 2020-04-01 14:18 UTC (permalink / raw)
To: Konstantin Ananyev, dev; +Cc: pablo.de.lara.guarch, declan.doherty
Hi Konstantin,
It looks there are some failures in meson build.
Could you please fix them and send a v2?
Thanks,
Akhil
> -----Original Message-----
> From: Konstantin Ananyev <konstantin.ananyev@intel.com>
> Sent: Monday, March 16, 2020 6:51 PM
> To: dev@dpdk.org
> Cc: Akhil Goyal <akhil.goyal@nxp.com>; pablo.de.lara.guarch@intel.com;
> declan.doherty@intel.com; Konstantin Ananyev
> <konstantin.ananyev@intel.com>
> Subject: [PATCH 0/2] support CPU crypto for AESNI MB PMD
>
> Add support for CPU crypto API into aesni_mb PMD
> and extend cryptodev UT to cover this new functionality.
>
> Konstantin Ananyev (2):
> crypto/aesni_mb: support CPU crypto
> test/crypto: add CPU crypto mode for AESNI MB
>
> app/test/test_cryptodev.c | 146 +++++--
> doc/guides/cryptodevs/aesni_mb.rst | 3 +
> doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
> .../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
> drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 384 ++++++++++++++++--
> .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
> 6 files changed, 476 insertions(+), 65 deletions(-)
>
> --
> 2.17.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD
2020-04-01 14:18 ` [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
@ 2020-04-01 15:09 ` Ananyev, Konstantin
0 siblings, 0 replies; 18+ messages in thread
From: Ananyev, Konstantin @ 2020-04-01 15:09 UTC (permalink / raw)
To: Akhil Goyal, dev; +Cc: De Lara Guarch, Pablo, Doherty, Declan
Hi Akhil,
>
> Hi Konstantin,
>
> It looks there are some failures in meson build.
> Could you please fix them and send a v2?
Will do.
>
> Thanks,
> Akhil
>
> > -----Original Message-----
> > From: Konstantin Ananyev <konstantin.ananyev@intel.com>
> > Sent: Monday, March 16, 2020 6:51 PM
> > To: dev@dpdk.org
> > Cc: Akhil Goyal <akhil.goyal@nxp.com>; pablo.de.lara.guarch@intel.com;
> > declan.doherty@intel.com; Konstantin Ananyev
> > <konstantin.ananyev@intel.com>
> > Subject: [PATCH 0/2] support CPU crypto for AESNI MB PMD
> >
> > Add support for CPU crypto API into aesni_mb PMD
> > and extend cryptodev UT to cover this new functionality.
> >
> > Konstantin Ananyev (2):
> > crypto/aesni_mb: support CPU crypto
> > test/crypto: add CPU crypto mode for AESNI MB
> >
> > app/test/test_cryptodev.c | 146 +++++--
> > doc/guides/cryptodevs/aesni_mb.rst | 3 +
> > doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
> > .../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
> > drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 384 ++++++++++++++++--
> > .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
> > 6 files changed, 476 insertions(+), 65 deletions(-)
> >
> > --
> > 2.17.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH v2 0/2] support CPU crypto for AESNI MB PMD
2020-03-16 13:21 [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Konstantin Ananyev
` (2 preceding siblings ...)
2020-04-01 14:18 ` [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
@ 2020-04-03 15:55 ` Konstantin Ananyev
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
` (3 more replies)
3 siblings, 4 replies; 18+ messages in thread
From: Konstantin Ananyev @ 2020-04-03 15:55 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
v2 - v1
Fix gcc 4.8 compilation issue
Add support for CPU crypto API into aesni_mb PMD
and extend cryptodev UT to cover this new functionality.
Konstantin Ananyev (2):
crypto/aesni_mb: support CPU crypto
test/crypto: add CPU crypto mode for AESNI MB
app/test/test_cryptodev.c | 146 +++++--
doc/guides/cryptodevs/aesni_mb.rst | 3 +
doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
.../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 386 ++++++++++++++++--
.../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
6 files changed, 478 insertions(+), 65 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 " Konstantin Ananyev
@ 2020-04-03 15:55 ` Konstantin Ananyev
2020-04-15 9:08 ` De Lara Guarch, Pablo
2020-04-15 9:14 ` De Lara Guarch, Pablo
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
` (2 subsequent siblings)
3 siblings, 2 replies; 18+ messages in thread
From: Konstantin Ananyev @ 2020-04-03 15:55 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
Add support for CPU crypto mode by introducing required handler.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
---
doc/guides/cryptodevs/aesni_mb.rst | 3 +
doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
.../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 386 ++++++++++++++++--
.../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
5 files changed, 363 insertions(+), 34 deletions(-)
diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst
index 465d20d88..02a0a499d 100644
--- a/doc/guides/cryptodevs/aesni_mb.rst
+++ b/doc/guides/cryptodevs/aesni_mb.rst
@@ -12,6 +12,9 @@ support for utilizing Intel multi buffer library, see the white paper
The AES-NI MB PMD has current only been tested on Fedora 21 64-bit with gcc.
+The AES-NI GCM PMD supports synchronous mode of operation with
+``rte_cryptodev_sym_cpu_crypto_process`` function call.
+
Features
--------
diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini
index ee6a07460..6cc4c82e8 100644
--- a/doc/guides/cryptodevs/features/aesni_mb.ini
+++ b/doc/guides/cryptodevs/features/aesni_mb.ini
@@ -12,6 +12,7 @@ CPU AVX2 = Y
CPU AVX512 = Y
CPU AESNI = Y
OOP LB In LB Out = Y
+CPU crypto = Y
;
; Supported crypto algorithms of the 'aesni_mb' crypto driver.
diff --git a/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
index 3456693c2..ce3ea9add 100644
--- a/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
@@ -275,6 +275,9 @@ aesni_mb_set_session_parameters(const MB_MGR *mb_mgr,
/** device specific operations function pointer structure */
extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;
-
+extern uint32_t
+aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs,
+ struct rte_crypto_sym_vec *vec);
#endif /* _AESNI_MB_PMD_PRIVATE_H_ */
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 33f416745..91567f98e 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -11,6 +11,7 @@
#include <rte_bus_vdev.h>
#include <rte_malloc.h>
#include <rte_cpuflags.h>
+#include <rte_per_lcore.h>
#include "aesni_mb_pmd_private.h"
@@ -19,6 +20,12 @@
#define HMAC_MAX_BLOCK_SIZE 128
static uint8_t cryptodev_driver_id;
+/*
+ * Needed to support CPU-CRYPTO API (rte_cryptodev_sym_cpu_crypto_process),
+ * as we still use JOB based API even for synchronous processing.
+ */
+static RTE_DEFINE_PER_LCORE(MB_MGR *, sync_mb_mgr);
+
typedef void (*hash_one_block_t)(const void *data, void *digest);
typedef void (*aes_keyexp_t)(const void *key, void *enc_exp_keys, void *dec_exp_keys);
@@ -808,6 +815,119 @@ auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session,
(UINT64_MAX - u_src + u_dst + 1);
}
+static inline void
+set_cpu_mb_job_params(JOB_AES_HMAC *job, struct aesni_mb_session *session,
+ union rte_crypto_sym_ofs sofs, void *buf, uint32_t len,
+ void *iv, void *aad, void *digest, void *udata)
+{
+ /* Set crypto operation */
+ job->chain_order = session->chain_order;
+
+ /* Set cipher parameters */
+ job->cipher_direction = session->cipher.direction;
+ job->cipher_mode = session->cipher.mode;
+
+ job->aes_key_len_in_bytes = session->cipher.key_length_in_bytes;
+
+ /* Set authentication parameters */
+ job->hash_alg = session->auth.algo;
+ job->iv = iv;
+
+ switch (job->hash_alg) {
+ case AES_XCBC:
+ job->u.XCBC._k1_expanded = session->auth.xcbc.k1_expanded;
+ job->u.XCBC._k2 = session->auth.xcbc.k2;
+ job->u.XCBC._k3 = session->auth.xcbc.k3;
+
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ break;
+
+ case AES_CCM:
+ job->u.CCM.aad = (uint8_t *)aad + 18;
+ job->u.CCM.aad_len_in_bytes = session->aead.aad_len;
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ job->iv++;
+ break;
+
+ case AES_CMAC:
+ job->u.CMAC._key_expanded = session->auth.cmac.expkey;
+ job->u.CMAC._skey1 = session->auth.cmac.skey1;
+ job->u.CMAC._skey2 = session->auth.cmac.skey2;
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ break;
+
+ case AES_GMAC:
+ if (session->cipher.mode == GCM) {
+ job->u.GCM.aad = aad;
+ job->u.GCM.aad_len_in_bytes = session->aead.aad_len;
+ } else {
+ /* For GMAC */
+ job->u.GCM.aad = buf;
+ job->u.GCM.aad_len_in_bytes = len;
+ job->cipher_mode = GCM;
+ }
+ job->aes_enc_key_expanded = &session->cipher.gcm_key;
+ job->aes_dec_key_expanded = &session->cipher.gcm_key;
+ break;
+
+ default:
+ job->u.HMAC._hashed_auth_key_xor_ipad =
+ session->auth.pads.inner;
+ job->u.HMAC._hashed_auth_key_xor_opad =
+ session->auth.pads.outer;
+
+ if (job->cipher_mode == DES3) {
+ job->aes_enc_key_expanded =
+ session->cipher.exp_3des_keys.ks_ptr;
+ job->aes_dec_key_expanded =
+ session->cipher.exp_3des_keys.ks_ptr;
+ } else {
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ }
+ }
+
+ /*
+ * Multi-buffer library current only support returning a truncated
+ * digest length as specified in the relevant IPsec RFCs
+ */
+
+ /* Set digest location and length */
+ job->auth_tag_output = digest;
+ job->auth_tag_output_len_in_bytes = session->auth.gen_digest_len;
+
+ /* Set IV parameters */
+ job->iv_len_in_bytes = session->iv.length;
+
+ /* Data Parameters */
+ job->src = buf;
+ job->dst = (uint8_t *)buf + sofs.ofs.cipher.head;
+ job->cipher_start_src_offset_in_bytes = sofs.ofs.cipher.head;
+ job->hash_start_src_offset_in_bytes = sofs.ofs.auth.head;
+ if (job->hash_alg == AES_GMAC && session->cipher.mode != GCM) {
+ job->msg_len_to_hash_in_bytes = 0;
+ job->msg_len_to_cipher_in_bytes = 0;
+ } else {
+ job->msg_len_to_hash_in_bytes = len - sofs.ofs.auth.head -
+ sofs.ofs.auth.tail;
+ job->msg_len_to_cipher_in_bytes = len - sofs.ofs.cipher.head -
+ sofs.ofs.cipher.tail;
+ }
+
+ job->user_data = udata;
+}
+
/**
* Process a crypto operation and complete a JOB_AES_HMAC job structure for
* submission to the multi buffer library for processing.
@@ -1100,6 +1220,15 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)
return op;
}
+static inline void
+post_process_mb_sync_job(JOB_AES_HMAC *job)
+{
+ uint32_t *st;
+
+ st = job->user_data;
+ st[0] = (job->status == STS_COMPLETED) ? 0 : EBADMSG;
+}
+
/**
* Process a completed JOB_AES_HMAC job and keep processing jobs until
* get_completed_job return NULL
@@ -1136,6 +1265,26 @@ handle_completed_jobs(struct aesni_mb_qp *qp, JOB_AES_HMAC *job,
return processed_jobs;
}
+static inline uint32_t
+handle_completed_sync_jobs(JOB_AES_HMAC *job, MB_MGR *mb_mgr)
+{
+ uint32_t i;
+
+ for (i = 0; job != NULL; i++, job = IMB_GET_COMPLETED_JOB(mb_mgr))
+ post_process_mb_sync_job(job);
+
+ return i;
+}
+
+static inline uint32_t
+flush_mb_sync_mgr(MB_MGR *mb_mgr)
+{
+ JOB_AES_HMAC *job;
+
+ job = IMB_FLUSH_JOB(mb_mgr);
+ return handle_completed_sync_jobs(job, mb_mgr);
+}
+
static inline uint16_t
flush_mb_mgr(struct aesni_mb_qp *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
@@ -1239,8 +1388,201 @@ aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
return processed_jobs;
}
+static MB_MGR *
+alloc_init_mb_mgr(enum aesni_mb_vector_mode vector_mode)
+{
+ MB_MGR *mb_mgr = alloc_mb_mgr(0);
+ if (mb_mgr == NULL)
+ return NULL;
+
+ switch (vector_mode) {
+ case RTE_AESNI_MB_SSE:
+ init_mb_mgr_sse(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX:
+ init_mb_mgr_avx(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX2:
+ init_mb_mgr_avx2(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX512:
+ init_mb_mgr_avx512(mb_mgr);
+ break;
+ default:
+ AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", vector_mode);
+ free_mb_mgr(mb_mgr);
+ return NULL;
+ }
+
+ return mb_mgr;
+}
+
+static inline void
+aesni_mb_fill_error_code(struct rte_crypto_sym_vec *vec, int32_t err)
+{
+ uint32_t i;
+
+ for (i = 0; i != vec->num; ++i)
+ vec->status[i] = err;
+}
+
+static inline int
+check_crypto_sgl(union rte_crypto_sym_ofs so, const struct rte_crypto_sgl *sgl)
+{
+ /* no multi-seg support with current AESNI-MB PMD */
+ if (sgl->num != 1)
+ return ENOTSUP;
+ else if (so.ofs.cipher.head + so.ofs.cipher.tail > sgl->vec[0].len)
+ return EINVAL;
+ return 0;
+}
+
+static inline JOB_AES_HMAC *
+submit_sync_job(MB_MGR *mb_mgr)
+{
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+ return IMB_SUBMIT_JOB(mb_mgr);
+#else
+ return IMB_SUBMIT_JOB_NOCHECK(mb_mgr);
+#endif
+}
+
+static inline uint32_t
+generate_sync_dgst(struct rte_crypto_sym_vec *vec,
+ const uint8_t dgst[][DIGEST_LENGTH_MAX], uint32_t len)
+{
+ uint32_t i, k;
+
+ for (i = 0, k = 0; i != vec->num; i++) {
+ if (vec->status[i] == 0) {
+ memcpy(vec->digest[i], dgst[i], len);
+ k++;
+ }
+ }
+
+ return k;
+}
+
+static inline uint32_t
+verify_sync_dgst(struct rte_crypto_sym_vec *vec,
+ const uint8_t dgst[][DIGEST_LENGTH_MAX], uint32_t len)
+{
+ uint32_t i, k;
+
+ for (i = 0, k = 0; i != vec->num; i++) {
+ if (vec->status[i] == 0) {
+ if (memcmp(vec->digest[i], dgst[i], len) != 0)
+ vec->status[i] = EBADMSG;
+ else
+ k++;
+ }
+ }
+
+ return k;
+}
+
+uint32_t
+aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs,
+ struct rte_crypto_sym_vec *vec)
+{
+ int32_t ret;
+ uint32_t i, j, k, len;
+ void *buf;
+ JOB_AES_HMAC *job;
+ MB_MGR *mb_mgr;
+ struct aesni_mb_private *priv;
+ struct aesni_mb_session *s;
+ uint8_t tmp_dgst[vec->num][DIGEST_LENGTH_MAX];
+
+ s = get_sym_session_private_data(sess, dev->driver_id);
+ if (s == NULL) {
+ aesni_mb_fill_error_code(vec, EINVAL);
+ return 0;
+ }
+
+ /* get per-thread MB MGR, create one if needed */
+ mb_mgr = RTE_PER_LCORE(sync_mb_mgr);
+ if (mb_mgr == NULL) {
+
+ priv = dev->data->dev_private;
+ mb_mgr = alloc_init_mb_mgr(priv->vector_mode);
+ if (mb_mgr == NULL) {
+ aesni_mb_fill_error_code(vec, ENOMEM);
+ return 0;
+ }
+ RTE_PER_LCORE(sync_mb_mgr) = mb_mgr;
+ }
+
+ for (i = 0, j = 0, k = 0; i != vec->num; i++) {
+
+
+ ret = check_crypto_sgl(sofs, vec->sgl + i);
+ if (ret != 0) {
+ vec->status[i] = ret;
+ continue;
+ }
+
+ buf = vec->sgl[i].vec[0].base;
+ len = vec->sgl[i].vec[0].len;
+
+ job = IMB_GET_NEXT_JOB(mb_mgr);
+ if (job == NULL) {
+ k += flush_mb_sync_mgr(mb_mgr);
+ job = IMB_GET_NEXT_JOB(mb_mgr);
+ RTE_ASSERT(job != NULL);
+ }
+
+ /* Submit job for processing */
+ set_cpu_mb_job_params(job, s, sofs, buf, len,
+ vec->iv[i], vec->aad[i], tmp_dgst[i],
+ &vec->status[i]);
+ job = submit_sync_job(mb_mgr);
+ j++;
+
+ /* handle completed jobs */
+ k += handle_completed_sync_jobs(job, mb_mgr);
+ }
+
+ /* flush remaining jobs */
+ while (k != j)
+ k += flush_mb_sync_mgr(mb_mgr);
+
+ /* finish processing for successful jobs: check/update digest */
+ if (k != 0) {
+ if (s->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY)
+ k = verify_sync_dgst(vec,
+ (const uint8_t (*)[DIGEST_LENGTH_MAX])tmp_dgst,
+ s->auth.req_digest_len);
+ else
+ k = generate_sync_dgst(vec,
+ (const uint8_t (*)[DIGEST_LENGTH_MAX])tmp_dgst,
+ s->auth.req_digest_len);
+ }
+
+ return k;
+}
+
static int cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev);
+static uint64_t
+vec_mode_to_flags(enum aesni_mb_vector_mode mode)
+{
+ switch (mode) {
+ case RTE_AESNI_MB_SSE:
+ return RTE_CRYPTODEV_FF_CPU_SSE;
+ case RTE_AESNI_MB_AVX:
+ return RTE_CRYPTODEV_FF_CPU_AVX;
+ case RTE_AESNI_MB_AVX2:
+ return RTE_CRYPTODEV_FF_CPU_AVX2;
+ case RTE_AESNI_MB_AVX512:
+ return RTE_CRYPTODEV_FF_CPU_AVX512;
+ default:
+ AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", mode);
+ return 0;
+ }
+}
+
static int
cryptodev_aesni_mb_create(const char *name,
struct rte_vdev_device *vdev,
@@ -1276,7 +1618,8 @@ cryptodev_aesni_mb_create(const char *name,
dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
- RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+ RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+ RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO;
/* Check CPU for support for AES instruction set */
if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES))
@@ -1284,30 +1627,12 @@ cryptodev_aesni_mb_create(const char *name,
else
AESNI_MB_LOG(WARNING, "AES instructions not supported by CPU");
- mb_mgr = alloc_mb_mgr(0);
- if (mb_mgr == NULL)
- return -ENOMEM;
+ dev->feature_flags |= vec_mode_to_flags(vector_mode);
- switch (vector_mode) {
- case RTE_AESNI_MB_SSE:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_SSE;
- init_mb_mgr_sse(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX;
- init_mb_mgr_avx(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX2:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX2;
- init_mb_mgr_avx2(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX512:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX512;
- init_mb_mgr_avx512(mb_mgr);
- break;
- default:
- AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", vector_mode);
- goto error_exit;
+ mb_mgr = alloc_init_mb_mgr(vector_mode);
+ if (mb_mgr == NULL) {
+ rte_cryptodev_pmd_destroy(dev);
+ return -ENOMEM;
}
/* Set vector instructions mode supported */
@@ -1319,16 +1644,7 @@ cryptodev_aesni_mb_create(const char *name,
AESNI_MB_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
imb_get_version_str());
-
return 0;
-
-error_exit:
- if (mb_mgr)
- free_mb_mgr(mb_mgr);
-
- rte_cryptodev_pmd_destroy(dev);
-
- return -1;
}
static int
@@ -1377,6 +1693,10 @@ cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev)
internals = cryptodev->data->dev_private;
free_mb_mgr(internals->mb_mgr);
+ if (RTE_PER_LCORE(sync_mb_mgr)) {
+ free_mb_mgr(RTE_PER_LCORE(sync_mb_mgr));
+ RTE_PER_LCORE(sync_mb_mgr) = NULL;
+ }
return rte_cryptodev_pmd_destroy(cryptodev);
}
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
index d8609ad11..e8371ea69 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
@@ -805,6 +805,8 @@ struct rte_cryptodev_ops aesni_mb_pmd_ops = {
.queue_pair_release = aesni_mb_pmd_qp_release,
.queue_pair_count = aesni_mb_pmd_qp_count,
+ .sym_cpu_process = aesni_mb_cpu_crypto_process_bulk,
+
.sym_session_get_size = aesni_mb_pmd_sym_session_get_size,
.sym_session_configure = aesni_mb_pmd_sym_session_configure,
.sym_session_clear = aesni_mb_pmd_sym_session_clear
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH v2 2/2] test/crypto: add CPU crypto mode for AESNI MB
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 " Konstantin Ananyev
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
@ 2020-04-03 15:55 ` Konstantin Ananyev
2020-04-15 9:13 ` De Lara Guarch, Pablo
2020-04-03 16:26 ` [dpdk-dev] [PATCH v2 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 " Konstantin Ananyev
3 siblings, 1 reply; 18+ messages in thread
From: Konstantin Ananyev @ 2020-04-03 15:55 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
This patch adds ability to run unit tests in cpu crypto mode
for AESNI MB cryptodev.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
---
app/test/test_cryptodev.c | 146 ++++++++++++++++++++++++++++++--------
1 file changed, 115 insertions(+), 31 deletions(-)
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 7b1ef5c86..7235b701b 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -143,7 +143,7 @@ ceil_byte_length(uint32_t num_bits)
}
static void
-process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
+process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
{
int32_t n, st;
void *iv;
@@ -155,8 +155,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
sop = op->sym;
- n = rte_crypto_mbuf_to_vec(sop->m_src, sop->auth.data.offset,
- sop->auth.data.length, vec, RTE_DIM(vec));
+ n = rte_crypto_mbuf_to_vec(sop->m_src, sop->aead.data.offset,
+ sop->aead.data.length, vec, RTE_DIM(vec));
if (n < 0 || n != sop->m_src->nb_segs) {
op->status = RTE_CRYPTO_OP_STATUS_ERROR;
@@ -168,8 +168,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
symvec.sgl = &sgl;
iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
symvec.iv = &iv;
- symvec.aad = NULL;
- symvec.digest = (void **)&sop->auth.digest.data;
+ symvec.aad = (void **)&sop->aead.aad.data;
+ symvec.digest = (void **)&sop->aead.digest.data;
symvec.status = &st;
symvec.num = 1;
@@ -184,9 +184,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
}
-
static void
-process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
+process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
{
int32_t n, st;
void *iv;
@@ -198,8 +197,8 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
sop = op->sym;
- n = rte_crypto_mbuf_to_vec(sop->m_src, sop->aead.data.offset,
- sop->aead.data.length, vec, RTE_DIM(vec));
+ n = rte_crypto_mbuf_to_vec(sop->m_src, sop->auth.data.offset,
+ sop->auth.data.length, vec, RTE_DIM(vec));
if (n < 0 || n != sop->m_src->nb_segs) {
op->status = RTE_CRYPTO_OP_STATUS_ERROR;
@@ -212,11 +211,14 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
symvec.iv = &iv;
symvec.aad = (void **)&sop->aead.aad.data;
- symvec.digest = (void **)&sop->aead.digest.data;
+ symvec.digest = (void **)&sop->auth.digest.data;
symvec.status = &st;
symvec.num = 1;
ofs.raw = 0;
+ ofs.ofs.cipher.head = sop->cipher.data.offset - sop->auth.data.offset;
+ ofs.ofs.cipher.tail = (sop->auth.data.offset + sop->auth.data.length) -
+ (sop->cipher.data.offset + sop->cipher.data.length);
n = rte_cryptodev_sym_cpu_crypto_process(dev_id, sop->session, ofs,
&symvec);
@@ -1538,8 +1540,14 @@ test_AES_CBC_HMAC_SHA1_encrypt_digest(void)
sym_op->cipher.data.length = QUOTE_512_BYTES;
/* Process crypto operation */
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -1689,8 +1697,14 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
sym_op->cipher.data.length = QUOTE_512_BYTES;
/* Process crypto operation */
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -2637,8 +2651,13 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
- ut_params->op);
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+
ut_params->obuf = ut_params->op->sym->m_src;
TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
ut_params->digest = rte_pktmbuf_mtod(ut_params->obuf, uint8_t *)
@@ -8715,6 +8734,9 @@ test_stats(void)
struct rte_cryptodev *dev;
cryptodev_stats_get_t temp_pfn;
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ return -ENOTSUP;
+
/* Verify the capabilities */
struct rte_cryptodev_sym_capability_idx cap_idx;
cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -8886,8 +8908,14 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
if (MD5_HMAC_create_op(ut_params, test_case, &plaintext))
return TEST_FAILED;
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -8938,8 +8966,14 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
if (MD5_HMAC_create_op(ut_params, test_case, &plaintext))
return TEST_FAILED;
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"HMAC_MD5 crypto op processing failed");
@@ -9852,7 +9886,8 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
ut_params->op->sym->m_src = ut_params->ibuf;
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
else
TEST_ASSERT_NOT_NULL(
process_crypto_request(ts_params->valid_devs[0],
@@ -9968,7 +10003,8 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
ut_params->op->sym->m_src = ut_params->ibuf;
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
else
TEST_ASSERT_NOT_NULL(
process_crypto_request(ts_params->valid_devs[0],
@@ -10528,10 +10564,17 @@ test_authentication_verify_fail_when_data_corruption(
else
tag_corruption(plaintext, reference->plaintext.len);
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
-
- TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "authentication not failed");
+ } else {
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ }
return 0;
}
@@ -10593,7 +10636,8 @@ test_authentication_verify_GMAC_fail_when_corruption(
tag_corruption(plaintext, reference->aad.len);
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
RTE_CRYPTO_OP_STATUS_SUCCESS,
"authentication not failed");
@@ -10666,10 +10710,17 @@ test_authenticated_decryption_fail_when_corruption(
else
tag_corruption(ciphertext, reference->ciphertext.len);
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
-
- TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "authentication not failed");
+ } else {
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ }
return 0;
}
@@ -10757,8 +10808,12 @@ test_authenticated_encryt_with_esn(
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
+ else
+ ut_params->op = process_crypto_request(
+ ts_params->valid_devs[0], ut_params->op);
TEST_ASSERT_NOT_NULL(ut_params->op, "no crypto operation returned");
@@ -10873,7 +10928,11 @@ test_authenticated_decrypt_with_esn(
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
ut_params->op);
TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
@@ -13377,6 +13436,29 @@ test_cryptodev_aesni_mb(void /*argv __rte_unused, int argc __rte_unused*/)
return unit_test_suite_runner(&cryptodev_testsuite);
}
+static int
+test_cryptodev_cpu_aesni_mb(void)
+{
+ int32_t rc;
+ enum rte_security_session_action_type at;
+
+ gbl_driver_id = rte_cryptodev_driver_id_get(
+ RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+ if (gbl_driver_id == -1) {
+ RTE_LOG(ERR, USER1, "AESNI MB PMD must be loaded. Check if "
+ "CONFIG_RTE_LIBRTE_PMD_AESNI_MB is enabled "
+ "in config file to run this testsuite.\n");
+ return TEST_SKIPPED;
+ }
+
+ at = gbl_action_type;
+ gbl_action_type = RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
+ rc = unit_test_suite_runner(&cryptodev_testsuite);
+ gbl_action_type = at;
+ return rc;
+}
+
static int
test_cryptodev_openssl(void)
{
@@ -13668,6 +13750,8 @@ test_cryptodev_nitrox(void)
REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+ test_cryptodev_cpu_aesni_mb);
REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/2] support CPU crypto for AESNI MB PMD
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 " Konstantin Ananyev
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
@ 2020-04-03 16:26 ` Akhil Goyal
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 " Konstantin Ananyev
3 siblings, 0 replies; 18+ messages in thread
From: Akhil Goyal @ 2020-04-03 16:26 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: declan.doherty, Konstantin Ananyev
Hi Pablo,
Could you please ack this patchset if no issues?
>
> v2 - v1
> Fix gcc 4.8 compilation issue
>
> Add support for CPU crypto API into aesni_mb PMD
> and extend cryptodev UT to cover this new functionality.
>
> Konstantin Ananyev (2):
> crypto/aesni_mb: support CPU crypto
> test/crypto: add CPU crypto mode for AESNI MB
>
> app/test/test_cryptodev.c | 146 +++++--
> doc/guides/cryptodevs/aesni_mb.rst | 3 +
> doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
> .../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
> drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 386 ++++++++++++++++--
> .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
> 6 files changed, 478 insertions(+), 65 deletions(-)
>
> --
> 2.17.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
@ 2020-04-15 9:08 ` De Lara Guarch, Pablo
2020-04-15 9:14 ` De Lara Guarch, Pablo
1 sibling, 0 replies; 18+ messages in thread
From: De Lara Guarch, Pablo @ 2020-04-15 9:08 UTC (permalink / raw)
To: Ananyev, Konstantin, dev; +Cc: akhil.goyal, Doherty, Declan
Hi Konstantin,
Just a small comment.
> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Friday, April 3, 2020 4:56 PM
> To: dev@dpdk.org
> Cc: akhil.goyal@nxp.com; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Doherty, Declan
> <declan.doherty@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Subject: [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto
>
> Add support for CPU crypto mode by introducing required handler.
>
> Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> ---
> doc/guides/cryptodevs/aesni_mb.rst | 3 +
> doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
> .../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
> drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 386 ++++++++++++++++--
> .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
> 5 files changed, 363 insertions(+), 34 deletions(-)
>
> diff --git a/doc/guides/cryptodevs/aesni_mb.rst
> b/doc/guides/cryptodevs/aesni_mb.rst
> index 465d20d88..02a0a499d 100644
> --- a/doc/guides/cryptodevs/aesni_mb.rst
> +++ b/doc/guides/cryptodevs/aesni_mb.rst
> @@ -12,6 +12,9 @@ support for utilizing Intel multi buffer library, see the white
> paper
>
> The AES-NI MB PMD has current only been tested on Fedora 21 64-bit with gcc.
>
> +The AES-NI GCM PMD supports synchronous mode of operation with
> +``rte_cryptodev_sym_cpu_crypto_process`` function call.
This should be AES-NI MB PMD. Also, this patch needs rebasing.
Apart from this, rest looks good:
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/crypto: add CPU crypto mode for AESNI MB
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
@ 2020-04-15 9:13 ` De Lara Guarch, Pablo
2020-04-15 18:25 ` Akhil Goyal
0 siblings, 1 reply; 18+ messages in thread
From: De Lara Guarch, Pablo @ 2020-04-15 9:13 UTC (permalink / raw)
To: Ananyev, Konstantin, dev; +Cc: akhil.goyal, Doherty, Declan
> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Friday, April 3, 2020 4:56 PM
> To: dev@dpdk.org
> Cc: akhil.goyal@nxp.com; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Doherty, Declan
> <declan.doherty@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Subject: [PATCH v2 2/2] test/crypto: add CPU crypto mode for AESNI MB
>
> This patch adds ability to run unit tests in cpu crypto mode for AESNI MB
> cryptodev.
>
> Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> ---
> app/test/test_cryptodev.c | 146 ++++++++++++++++++++++++++++++--------
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
2020-04-15 9:08 ` De Lara Guarch, Pablo
@ 2020-04-15 9:14 ` De Lara Guarch, Pablo
2020-04-15 9:52 ` Ananyev, Konstantin
1 sibling, 1 reply; 18+ messages in thread
From: De Lara Guarch, Pablo @ 2020-04-15 9:14 UTC (permalink / raw)
To: Ananyev, Konstantin, dev; +Cc: akhil.goyal, Doherty, Declan
> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Friday, April 3, 2020 4:56 PM
> To: dev@dpdk.org
> Cc: akhil.goyal@nxp.com; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Doherty, Declan
> <declan.doherty@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Subject: [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto
>
> Add support for CPU crypto mode by introducing required handler.
>
> Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> ---
> doc/guides/cryptodevs/aesni_mb.rst | 3 +
> doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
> .../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
> drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 386 ++++++++++++++++--
> .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
One last thing. Could you update "Release Notes"?
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto
2020-04-15 9:14 ` De Lara Guarch, Pablo
@ 2020-04-15 9:52 ` Ananyev, Konstantin
0 siblings, 0 replies; 18+ messages in thread
From: Ananyev, Konstantin @ 2020-04-15 9:52 UTC (permalink / raw)
To: De Lara Guarch, Pablo, dev; +Cc: akhil.goyal, Doherty, Declan
Hi Pablo,
> >
> > Add support for CPU crypto mode by introducing required handler.
> >
> > Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> > ---
> > doc/guides/cryptodevs/aesni_mb.rst | 3 +
> > doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
> > .../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
> > drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 386 ++++++++++++++++--
> > .../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
>
> One last thing. Could you update "Release Notes"?
Will do.
Thanks
Konstantin
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/crypto: add CPU crypto mode for AESNI MB
2020-04-15 9:13 ` De Lara Guarch, Pablo
@ 2020-04-15 18:25 ` Akhil Goyal
0 siblings, 0 replies; 18+ messages in thread
From: Akhil Goyal @ 2020-04-15 18:25 UTC (permalink / raw)
To: De Lara Guarch, Pablo, Ananyev, Konstantin, dev; +Cc: Doherty, Declan
> >
> > This patch adds ability to run unit tests in cpu crypto mode for AESNI MB
> > cryptodev.
> >
> > Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> > ---
> > app/test/test_cryptodev.c | 146 ++++++++++++++++++++++++++++++--------
>
> Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH v3 0/2] support CPU crypto for AESNI MB PMD
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 " Konstantin Ananyev
` (2 preceding siblings ...)
2020-04-03 16:26 ` [dpdk-dev] [PATCH v2 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
@ 2020-04-17 15:03 ` Konstantin Ananyev
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
` (2 more replies)
3 siblings, 3 replies; 18+ messages in thread
From: Konstantin Ananyev @ 2020-04-17 15:03 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
v3 - v2
Address Pablo comments:
- Fix doc typo
- Update release notes
v2 - v1
Fix gcc 4.8 compilation issue
Add support for CPU crypto API into aesni_mb PMD
and extend cryptodev UT to cover this new functionality.
Konstantin Ananyev (2):
crypto/aesni_mb: support CPU crypto
test/crypto: add CPU crypto mode for AESNI MB
app/test/test_cryptodev.c | 146 +++++--
doc/guides/cryptodevs/aesni_mb.rst | 3 +
doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
doc/guides/rel_notes/release_20_05.rst | 4 +
.../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 386 ++++++++++++++++--
.../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
7 files changed, 482 insertions(+), 65 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH v3 1/2] crypto/aesni_mb: support CPU crypto
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 " Konstantin Ananyev
@ 2020-04-17 15:03 ` Konstantin Ananyev
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
2020-04-17 20:57 ` [dpdk-dev] [PATCH v3 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
2 siblings, 0 replies; 18+ messages in thread
From: Konstantin Ananyev @ 2020-04-17 15:03 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
Add support for CPU crypto mode by introducing required handler.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
doc/guides/cryptodevs/aesni_mb.rst | 3 +
doc/guides/cryptodevs/features/aesni_mb.ini | 1 +
doc/guides/rel_notes/release_20_05.rst | 4 +
.../crypto/aesni_mb/aesni_mb_pmd_private.h | 5 +-
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 386 ++++++++++++++++--
.../crypto/aesni_mb/rte_aesni_mb_pmd_ops.c | 2 +
6 files changed, 367 insertions(+), 34 deletions(-)
diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst
index 465d20d88..baa11fac4 100644
--- a/doc/guides/cryptodevs/aesni_mb.rst
+++ b/doc/guides/cryptodevs/aesni_mb.rst
@@ -12,6 +12,9 @@ support for utilizing Intel multi buffer library, see the white paper
The AES-NI MB PMD has current only been tested on Fedora 21 64-bit with gcc.
+The AES-NI MB PMD supports synchronous mode of operation with
+``rte_cryptodev_sym_cpu_crypto_process`` function call.
+
Features
--------
diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini
index ee6a07460..6cc4c82e8 100644
--- a/doc/guides/cryptodevs/features/aesni_mb.ini
+++ b/doc/guides/cryptodevs/features/aesni_mb.ini
@@ -12,6 +12,7 @@ CPU AVX2 = Y
CPU AVX512 = Y
CPU AESNI = Y
OOP LB In LB Out = Y
+CPU crypto = Y
;
; Supported crypto algorithms of the 'aesni_mb' crypto driver.
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index 184967844..360e3ddb8 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -81,6 +81,10 @@ New Features
by making use of the event device capabilities. The event mode currently supports
only inline IPsec protocol offload.
+* **Updated the aesni_mb cryptodev PMD.**
+
+ Add support for synchronous Crypto burst API.
+
Removed Items
-------------
diff --git a/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h b/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
index 3456693c2..ce3ea9add 100644
--- a/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
+++ b/drivers/crypto/aesni_mb/aesni_mb_pmd_private.h
@@ -275,6 +275,9 @@ aesni_mb_set_session_parameters(const MB_MGR *mb_mgr,
/** device specific operations function pointer structure */
extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;
-
+extern uint32_t
+aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs,
+ struct rte_crypto_sym_vec *vec);
#endif /* _AESNI_MB_PMD_PRIVATE_H_ */
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 33f416745..91567f98e 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -11,6 +11,7 @@
#include <rte_bus_vdev.h>
#include <rte_malloc.h>
#include <rte_cpuflags.h>
+#include <rte_per_lcore.h>
#include "aesni_mb_pmd_private.h"
@@ -19,6 +20,12 @@
#define HMAC_MAX_BLOCK_SIZE 128
static uint8_t cryptodev_driver_id;
+/*
+ * Needed to support CPU-CRYPTO API (rte_cryptodev_sym_cpu_crypto_process),
+ * as we still use JOB based API even for synchronous processing.
+ */
+static RTE_DEFINE_PER_LCORE(MB_MGR *, sync_mb_mgr);
+
typedef void (*hash_one_block_t)(const void *data, void *digest);
typedef void (*aes_keyexp_t)(const void *key, void *enc_exp_keys, void *dec_exp_keys);
@@ -808,6 +815,119 @@ auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session,
(UINT64_MAX - u_src + u_dst + 1);
}
+static inline void
+set_cpu_mb_job_params(JOB_AES_HMAC *job, struct aesni_mb_session *session,
+ union rte_crypto_sym_ofs sofs, void *buf, uint32_t len,
+ void *iv, void *aad, void *digest, void *udata)
+{
+ /* Set crypto operation */
+ job->chain_order = session->chain_order;
+
+ /* Set cipher parameters */
+ job->cipher_direction = session->cipher.direction;
+ job->cipher_mode = session->cipher.mode;
+
+ job->aes_key_len_in_bytes = session->cipher.key_length_in_bytes;
+
+ /* Set authentication parameters */
+ job->hash_alg = session->auth.algo;
+ job->iv = iv;
+
+ switch (job->hash_alg) {
+ case AES_XCBC:
+ job->u.XCBC._k1_expanded = session->auth.xcbc.k1_expanded;
+ job->u.XCBC._k2 = session->auth.xcbc.k2;
+ job->u.XCBC._k3 = session->auth.xcbc.k3;
+
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ break;
+
+ case AES_CCM:
+ job->u.CCM.aad = (uint8_t *)aad + 18;
+ job->u.CCM.aad_len_in_bytes = session->aead.aad_len;
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ job->iv++;
+ break;
+
+ case AES_CMAC:
+ job->u.CMAC._key_expanded = session->auth.cmac.expkey;
+ job->u.CMAC._skey1 = session->auth.cmac.skey1;
+ job->u.CMAC._skey2 = session->auth.cmac.skey2;
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ break;
+
+ case AES_GMAC:
+ if (session->cipher.mode == GCM) {
+ job->u.GCM.aad = aad;
+ job->u.GCM.aad_len_in_bytes = session->aead.aad_len;
+ } else {
+ /* For GMAC */
+ job->u.GCM.aad = buf;
+ job->u.GCM.aad_len_in_bytes = len;
+ job->cipher_mode = GCM;
+ }
+ job->aes_enc_key_expanded = &session->cipher.gcm_key;
+ job->aes_dec_key_expanded = &session->cipher.gcm_key;
+ break;
+
+ default:
+ job->u.HMAC._hashed_auth_key_xor_ipad =
+ session->auth.pads.inner;
+ job->u.HMAC._hashed_auth_key_xor_opad =
+ session->auth.pads.outer;
+
+ if (job->cipher_mode == DES3) {
+ job->aes_enc_key_expanded =
+ session->cipher.exp_3des_keys.ks_ptr;
+ job->aes_dec_key_expanded =
+ session->cipher.exp_3des_keys.ks_ptr;
+ } else {
+ job->aes_enc_key_expanded =
+ session->cipher.expanded_aes_keys.encode;
+ job->aes_dec_key_expanded =
+ session->cipher.expanded_aes_keys.decode;
+ }
+ }
+
+ /*
+ * Multi-buffer library current only support returning a truncated
+ * digest length as specified in the relevant IPsec RFCs
+ */
+
+ /* Set digest location and length */
+ job->auth_tag_output = digest;
+ job->auth_tag_output_len_in_bytes = session->auth.gen_digest_len;
+
+ /* Set IV parameters */
+ job->iv_len_in_bytes = session->iv.length;
+
+ /* Data Parameters */
+ job->src = buf;
+ job->dst = (uint8_t *)buf + sofs.ofs.cipher.head;
+ job->cipher_start_src_offset_in_bytes = sofs.ofs.cipher.head;
+ job->hash_start_src_offset_in_bytes = sofs.ofs.auth.head;
+ if (job->hash_alg == AES_GMAC && session->cipher.mode != GCM) {
+ job->msg_len_to_hash_in_bytes = 0;
+ job->msg_len_to_cipher_in_bytes = 0;
+ } else {
+ job->msg_len_to_hash_in_bytes = len - sofs.ofs.auth.head -
+ sofs.ofs.auth.tail;
+ job->msg_len_to_cipher_in_bytes = len - sofs.ofs.cipher.head -
+ sofs.ofs.cipher.tail;
+ }
+
+ job->user_data = udata;
+}
+
/**
* Process a crypto operation and complete a JOB_AES_HMAC job structure for
* submission to the multi buffer library for processing.
@@ -1100,6 +1220,15 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)
return op;
}
+static inline void
+post_process_mb_sync_job(JOB_AES_HMAC *job)
+{
+ uint32_t *st;
+
+ st = job->user_data;
+ st[0] = (job->status == STS_COMPLETED) ? 0 : EBADMSG;
+}
+
/**
* Process a completed JOB_AES_HMAC job and keep processing jobs until
* get_completed_job return NULL
@@ -1136,6 +1265,26 @@ handle_completed_jobs(struct aesni_mb_qp *qp, JOB_AES_HMAC *job,
return processed_jobs;
}
+static inline uint32_t
+handle_completed_sync_jobs(JOB_AES_HMAC *job, MB_MGR *mb_mgr)
+{
+ uint32_t i;
+
+ for (i = 0; job != NULL; i++, job = IMB_GET_COMPLETED_JOB(mb_mgr))
+ post_process_mb_sync_job(job);
+
+ return i;
+}
+
+static inline uint32_t
+flush_mb_sync_mgr(MB_MGR *mb_mgr)
+{
+ JOB_AES_HMAC *job;
+
+ job = IMB_FLUSH_JOB(mb_mgr);
+ return handle_completed_sync_jobs(job, mb_mgr);
+}
+
static inline uint16_t
flush_mb_mgr(struct aesni_mb_qp *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
@@ -1239,8 +1388,201 @@ aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
return processed_jobs;
}
+static MB_MGR *
+alloc_init_mb_mgr(enum aesni_mb_vector_mode vector_mode)
+{
+ MB_MGR *mb_mgr = alloc_mb_mgr(0);
+ if (mb_mgr == NULL)
+ return NULL;
+
+ switch (vector_mode) {
+ case RTE_AESNI_MB_SSE:
+ init_mb_mgr_sse(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX:
+ init_mb_mgr_avx(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX2:
+ init_mb_mgr_avx2(mb_mgr);
+ break;
+ case RTE_AESNI_MB_AVX512:
+ init_mb_mgr_avx512(mb_mgr);
+ break;
+ default:
+ AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", vector_mode);
+ free_mb_mgr(mb_mgr);
+ return NULL;
+ }
+
+ return mb_mgr;
+}
+
+static inline void
+aesni_mb_fill_error_code(struct rte_crypto_sym_vec *vec, int32_t err)
+{
+ uint32_t i;
+
+ for (i = 0; i != vec->num; ++i)
+ vec->status[i] = err;
+}
+
+static inline int
+check_crypto_sgl(union rte_crypto_sym_ofs so, const struct rte_crypto_sgl *sgl)
+{
+ /* no multi-seg support with current AESNI-MB PMD */
+ if (sgl->num != 1)
+ return ENOTSUP;
+ else if (so.ofs.cipher.head + so.ofs.cipher.tail > sgl->vec[0].len)
+ return EINVAL;
+ return 0;
+}
+
+static inline JOB_AES_HMAC *
+submit_sync_job(MB_MGR *mb_mgr)
+{
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+ return IMB_SUBMIT_JOB(mb_mgr);
+#else
+ return IMB_SUBMIT_JOB_NOCHECK(mb_mgr);
+#endif
+}
+
+static inline uint32_t
+generate_sync_dgst(struct rte_crypto_sym_vec *vec,
+ const uint8_t dgst[][DIGEST_LENGTH_MAX], uint32_t len)
+{
+ uint32_t i, k;
+
+ for (i = 0, k = 0; i != vec->num; i++) {
+ if (vec->status[i] == 0) {
+ memcpy(vec->digest[i], dgst[i], len);
+ k++;
+ }
+ }
+
+ return k;
+}
+
+static inline uint32_t
+verify_sync_dgst(struct rte_crypto_sym_vec *vec,
+ const uint8_t dgst[][DIGEST_LENGTH_MAX], uint32_t len)
+{
+ uint32_t i, k;
+
+ for (i = 0, k = 0; i != vec->num; i++) {
+ if (vec->status[i] == 0) {
+ if (memcmp(vec->digest[i], dgst[i], len) != 0)
+ vec->status[i] = EBADMSG;
+ else
+ k++;
+ }
+ }
+
+ return k;
+}
+
+uint32_t
+aesni_mb_cpu_crypto_process_bulk(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs,
+ struct rte_crypto_sym_vec *vec)
+{
+ int32_t ret;
+ uint32_t i, j, k, len;
+ void *buf;
+ JOB_AES_HMAC *job;
+ MB_MGR *mb_mgr;
+ struct aesni_mb_private *priv;
+ struct aesni_mb_session *s;
+ uint8_t tmp_dgst[vec->num][DIGEST_LENGTH_MAX];
+
+ s = get_sym_session_private_data(sess, dev->driver_id);
+ if (s == NULL) {
+ aesni_mb_fill_error_code(vec, EINVAL);
+ return 0;
+ }
+
+ /* get per-thread MB MGR, create one if needed */
+ mb_mgr = RTE_PER_LCORE(sync_mb_mgr);
+ if (mb_mgr == NULL) {
+
+ priv = dev->data->dev_private;
+ mb_mgr = alloc_init_mb_mgr(priv->vector_mode);
+ if (mb_mgr == NULL) {
+ aesni_mb_fill_error_code(vec, ENOMEM);
+ return 0;
+ }
+ RTE_PER_LCORE(sync_mb_mgr) = mb_mgr;
+ }
+
+ for (i = 0, j = 0, k = 0; i != vec->num; i++) {
+
+
+ ret = check_crypto_sgl(sofs, vec->sgl + i);
+ if (ret != 0) {
+ vec->status[i] = ret;
+ continue;
+ }
+
+ buf = vec->sgl[i].vec[0].base;
+ len = vec->sgl[i].vec[0].len;
+
+ job = IMB_GET_NEXT_JOB(mb_mgr);
+ if (job == NULL) {
+ k += flush_mb_sync_mgr(mb_mgr);
+ job = IMB_GET_NEXT_JOB(mb_mgr);
+ RTE_ASSERT(job != NULL);
+ }
+
+ /* Submit job for processing */
+ set_cpu_mb_job_params(job, s, sofs, buf, len,
+ vec->iv[i], vec->aad[i], tmp_dgst[i],
+ &vec->status[i]);
+ job = submit_sync_job(mb_mgr);
+ j++;
+
+ /* handle completed jobs */
+ k += handle_completed_sync_jobs(job, mb_mgr);
+ }
+
+ /* flush remaining jobs */
+ while (k != j)
+ k += flush_mb_sync_mgr(mb_mgr);
+
+ /* finish processing for successful jobs: check/update digest */
+ if (k != 0) {
+ if (s->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY)
+ k = verify_sync_dgst(vec,
+ (const uint8_t (*)[DIGEST_LENGTH_MAX])tmp_dgst,
+ s->auth.req_digest_len);
+ else
+ k = generate_sync_dgst(vec,
+ (const uint8_t (*)[DIGEST_LENGTH_MAX])tmp_dgst,
+ s->auth.req_digest_len);
+ }
+
+ return k;
+}
+
static int cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev);
+static uint64_t
+vec_mode_to_flags(enum aesni_mb_vector_mode mode)
+{
+ switch (mode) {
+ case RTE_AESNI_MB_SSE:
+ return RTE_CRYPTODEV_FF_CPU_SSE;
+ case RTE_AESNI_MB_AVX:
+ return RTE_CRYPTODEV_FF_CPU_AVX;
+ case RTE_AESNI_MB_AVX2:
+ return RTE_CRYPTODEV_FF_CPU_AVX2;
+ case RTE_AESNI_MB_AVX512:
+ return RTE_CRYPTODEV_FF_CPU_AVX512;
+ default:
+ AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", mode);
+ return 0;
+ }
+}
+
static int
cryptodev_aesni_mb_create(const char *name,
struct rte_vdev_device *vdev,
@@ -1276,7 +1618,8 @@ cryptodev_aesni_mb_create(const char *name,
dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
- RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+ RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+ RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO;
/* Check CPU for support for AES instruction set */
if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES))
@@ -1284,30 +1627,12 @@ cryptodev_aesni_mb_create(const char *name,
else
AESNI_MB_LOG(WARNING, "AES instructions not supported by CPU");
- mb_mgr = alloc_mb_mgr(0);
- if (mb_mgr == NULL)
- return -ENOMEM;
+ dev->feature_flags |= vec_mode_to_flags(vector_mode);
- switch (vector_mode) {
- case RTE_AESNI_MB_SSE:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_SSE;
- init_mb_mgr_sse(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX;
- init_mb_mgr_avx(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX2:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX2;
- init_mb_mgr_avx2(mb_mgr);
- break;
- case RTE_AESNI_MB_AVX512:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX512;
- init_mb_mgr_avx512(mb_mgr);
- break;
- default:
- AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", vector_mode);
- goto error_exit;
+ mb_mgr = alloc_init_mb_mgr(vector_mode);
+ if (mb_mgr == NULL) {
+ rte_cryptodev_pmd_destroy(dev);
+ return -ENOMEM;
}
/* Set vector instructions mode supported */
@@ -1319,16 +1644,7 @@ cryptodev_aesni_mb_create(const char *name,
AESNI_MB_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
imb_get_version_str());
-
return 0;
-
-error_exit:
- if (mb_mgr)
- free_mb_mgr(mb_mgr);
-
- rte_cryptodev_pmd_destroy(dev);
-
- return -1;
}
static int
@@ -1377,6 +1693,10 @@ cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev)
internals = cryptodev->data->dev_private;
free_mb_mgr(internals->mb_mgr);
+ if (RTE_PER_LCORE(sync_mb_mgr)) {
+ free_mb_mgr(RTE_PER_LCORE(sync_mb_mgr));
+ RTE_PER_LCORE(sync_mb_mgr) = NULL;
+ }
return rte_cryptodev_pmd_destroy(cryptodev);
}
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
index 845661174..b972ab434 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
@@ -797,6 +797,8 @@ struct rte_cryptodev_ops aesni_mb_pmd_ops = {
.queue_pair_setup = aesni_mb_pmd_qp_setup,
.queue_pair_release = aesni_mb_pmd_qp_release,
+ .sym_cpu_process = aesni_mb_cpu_crypto_process_bulk,
+
.sym_session_get_size = aesni_mb_pmd_sym_session_get_size,
.sym_session_configure = aesni_mb_pmd_sym_session_configure,
.sym_session_clear = aesni_mb_pmd_sym_session_clear
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH v3 2/2] test/crypto: add CPU crypto mode for AESNI MB
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 " Konstantin Ananyev
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
@ 2020-04-17 15:03 ` Konstantin Ananyev
2020-04-17 20:57 ` [dpdk-dev] [PATCH v3 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
2 siblings, 0 replies; 18+ messages in thread
From: Konstantin Ananyev @ 2020-04-17 15:03 UTC (permalink / raw)
To: dev; +Cc: akhil.goyal, pablo.de.lara.guarch, declan.doherty, Konstantin Ananyev
This patch adds ability to run unit tests in cpu crypto mode
for AESNI MB cryptodev.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
---
app/test/test_cryptodev.c | 146 ++++++++++++++++++++++++++++++--------
1 file changed, 115 insertions(+), 31 deletions(-)
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 3df63eb71..cdb840879 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -143,7 +143,7 @@ ceil_byte_length(uint32_t num_bits)
}
static void
-process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
+process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
{
int32_t n, st;
void *iv;
@@ -155,8 +155,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
sop = op->sym;
- n = rte_crypto_mbuf_to_vec(sop->m_src, sop->auth.data.offset,
- sop->auth.data.length, vec, RTE_DIM(vec));
+ n = rte_crypto_mbuf_to_vec(sop->m_src, sop->aead.data.offset,
+ sop->aead.data.length, vec, RTE_DIM(vec));
if (n < 0 || n != sop->m_src->nb_segs) {
op->status = RTE_CRYPTO_OP_STATUS_ERROR;
@@ -168,8 +168,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
symvec.sgl = &sgl;
iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
symvec.iv = &iv;
- symvec.aad = NULL;
- symvec.digest = (void **)&sop->auth.digest.data;
+ symvec.aad = (void **)&sop->aead.aad.data;
+ symvec.digest = (void **)&sop->aead.digest.data;
symvec.status = &st;
symvec.num = 1;
@@ -184,9 +184,8 @@ process_cpu_gmac_op(uint8_t dev_id, struct rte_crypto_op *op)
op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
}
-
static void
-process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
+process_cpu_crypt_auth_op(uint8_t dev_id, struct rte_crypto_op *op)
{
int32_t n, st;
void *iv;
@@ -198,8 +197,8 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
sop = op->sym;
- n = rte_crypto_mbuf_to_vec(sop->m_src, sop->aead.data.offset,
- sop->aead.data.length, vec, RTE_DIM(vec));
+ n = rte_crypto_mbuf_to_vec(sop->m_src, sop->auth.data.offset,
+ sop->auth.data.length, vec, RTE_DIM(vec));
if (n < 0 || n != sop->m_src->nb_segs) {
op->status = RTE_CRYPTO_OP_STATUS_ERROR;
@@ -212,11 +211,14 @@ process_cpu_aead_op(uint8_t dev_id, struct rte_crypto_op *op)
iv = rte_crypto_op_ctod_offset(op, void *, IV_OFFSET);
symvec.iv = &iv;
symvec.aad = (void **)&sop->aead.aad.data;
- symvec.digest = (void **)&sop->aead.digest.data;
+ symvec.digest = (void **)&sop->auth.digest.data;
symvec.status = &st;
symvec.num = 1;
ofs.raw = 0;
+ ofs.ofs.cipher.head = sop->cipher.data.offset - sop->auth.data.offset;
+ ofs.ofs.cipher.tail = (sop->auth.data.offset + sop->auth.data.length) -
+ (sop->cipher.data.offset + sop->cipher.data.length);
n = rte_cryptodev_sym_cpu_crypto_process(dev_id, sop->session, ofs,
&symvec);
@@ -1538,8 +1540,14 @@ test_AES_CBC_HMAC_SHA1_encrypt_digest(void)
sym_op->cipher.data.length = QUOTE_512_BYTES;
/* Process crypto operation */
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -1689,8 +1697,14 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
sym_op->cipher.data.length = QUOTE_512_BYTES;
/* Process crypto operation */
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -2637,8 +2651,13 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
- ut_params->op);
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+
ut_params->obuf = ut_params->op->sym->m_src;
TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
ut_params->digest = rte_pktmbuf_mtod(ut_params->obuf, uint8_t *)
@@ -8733,6 +8752,9 @@ test_stats(void)
struct rte_cryptodev *dev;
cryptodev_stats_get_t temp_pfn;
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ return -ENOTSUP;
+
/* Verify the capabilities */
struct rte_cryptodev_sym_capability_idx cap_idx;
cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -8904,8 +8926,14 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
if (MD5_HMAC_create_op(ut_params, test_case, &plaintext))
return TEST_FAILED;
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
@@ -8956,8 +8984,14 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
if (MD5_HMAC_create_op(ut_params, test_case, &plaintext))
return TEST_FAILED;
- TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
- ut_params->op), "failed to process sym crypto op");
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ TEST_ASSERT_NOT_NULL(
+ process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op),
+ "failed to process sym crypto op");
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"HMAC_MD5 crypto op processing failed");
@@ -9870,7 +9904,8 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
ut_params->op->sym->m_src = ut_params->ibuf;
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
else
TEST_ASSERT_NOT_NULL(
process_crypto_request(ts_params->valid_devs[0],
@@ -9986,7 +10021,8 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
ut_params->op->sym->m_src = ut_params->ibuf;
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
else
TEST_ASSERT_NOT_NULL(
process_crypto_request(ts_params->valid_devs[0],
@@ -10546,10 +10582,17 @@ test_authentication_verify_fail_when_data_corruption(
else
tag_corruption(plaintext, reference->plaintext.len);
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
-
- TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "authentication not failed");
+ } else {
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ }
return 0;
}
@@ -10611,7 +10654,8 @@ test_authentication_verify_GMAC_fail_when_corruption(
tag_corruption(plaintext, reference->aad.len);
if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
- process_cpu_gmac_op(ts_params->valid_devs[0], ut_params->op);
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
RTE_CRYPTO_OP_STATUS_SUCCESS,
"authentication not failed");
@@ -10684,10 +10728,17 @@ test_authenticated_decryption_fail_when_corruption(
else
tag_corruption(ciphertext, reference->ciphertext.len);
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
-
- TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ TEST_ASSERT_NOT_EQUAL(ut_params->op->status,
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "authentication not failed");
+ } else {
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NULL(ut_params->op, "authentication not failed");
+ }
return 0;
}
@@ -10775,8 +10826,12 @@ test_authenticated_encryt_with_esn(
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
ut_params->op);
+ else
+ ut_params->op = process_crypto_request(
+ ts_params->valid_devs[0], ut_params->op);
TEST_ASSERT_NOT_NULL(ut_params->op, "no crypto operation returned");
@@ -10891,7 +10946,11 @@ test_authenticated_decrypt_with_esn(
if (retval < 0)
return retval;
- ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
+ process_cpu_crypt_auth_op(ts_params->valid_devs[0],
+ ut_params->op);
+ else
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
ut_params->op);
TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
@@ -13400,6 +13459,29 @@ test_cryptodev_aesni_mb(void /*argv __rte_unused, int argc __rte_unused*/)
return unit_test_suite_runner(&cryptodev_testsuite);
}
+static int
+test_cryptodev_cpu_aesni_mb(void)
+{
+ int32_t rc;
+ enum rte_security_session_action_type at;
+
+ gbl_driver_id = rte_cryptodev_driver_id_get(
+ RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+ if (gbl_driver_id == -1) {
+ RTE_LOG(ERR, USER1, "AESNI MB PMD must be loaded. Check if "
+ "CONFIG_RTE_LIBRTE_PMD_AESNI_MB is enabled "
+ "in config file to run this testsuite.\n");
+ return TEST_SKIPPED;
+ }
+
+ at = gbl_action_type;
+ gbl_action_type = RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
+ rc = unit_test_suite_runner(&cryptodev_testsuite);
+ gbl_action_type = at;
+ return rc;
+}
+
static int
test_cryptodev_openssl(void)
{
@@ -13691,6 +13773,8 @@ test_cryptodev_nitrox(void)
REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
+REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_mb_autotest,
+ test_cryptodev_cpu_aesni_mb);
REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
REGISTER_TEST_COMMAND(cryptodev_aesni_gcm_autotest, test_cryptodev_aesni_gcm);
REGISTER_TEST_COMMAND(cryptodev_cpu_aesni_gcm_autotest,
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/2] support CPU crypto for AESNI MB PMD
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 " Konstantin Ananyev
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
@ 2020-04-17 20:57 ` Akhil Goyal
2 siblings, 0 replies; 18+ messages in thread
From: Akhil Goyal @ 2020-04-17 20:57 UTC (permalink / raw)
To: Konstantin Ananyev, dev; +Cc: pablo.de.lara.guarch, declan.doherty
>
> v3 - v2
> Address Pablo comments:
> - Fix doc typo
> - Update release notes
>
> v2 - v1
> Fix gcc 4.8 compilation issue
>
> Add support for CPU crypto API into aesni_mb PMD
> and extend cryptodev UT to cover this new functionality.
>
Applied to dpdk-next-crypto
Thanks.
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2020-04-17 20:57 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-16 13:21 [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Konstantin Ananyev
2020-03-16 13:21 ` [dpdk-dev] [PATCH 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
2020-03-16 13:21 ` [dpdk-dev] [PATCH 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
2020-04-01 14:18 ` [dpdk-dev] [PATCH 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
2020-04-01 15:09 ` Ananyev, Konstantin
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 " Konstantin Ananyev
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
2020-04-15 9:08 ` De Lara Guarch, Pablo
2020-04-15 9:14 ` De Lara Guarch, Pablo
2020-04-15 9:52 ` Ananyev, Konstantin
2020-04-03 15:55 ` [dpdk-dev] [PATCH v2 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
2020-04-15 9:13 ` De Lara Guarch, Pablo
2020-04-15 18:25 ` Akhil Goyal
2020-04-03 16:26 ` [dpdk-dev] [PATCH v2 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 " Konstantin Ananyev
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 1/2] crypto/aesni_mb: support CPU crypto Konstantin Ananyev
2020-04-17 15:03 ` [dpdk-dev] [PATCH v3 2/2] test/crypto: add CPU crypto mode for AESNI MB Konstantin Ananyev
2020-04-17 20:57 ` [dpdk-dev] [PATCH v3 0/2] support CPU crypto for AESNI MB PMD Akhil Goyal
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.