* [PATCH 00/30] crypto/qat: refactor to support multiple service
@ 2018-04-06 18:51 Fiona Trahe
2018-04-06 18:51 ` [PATCH 01/30] crypto/qat: use SPDX license Fiona Trahe
` (63 more replies)
0 siblings, 64 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
The QAT crypto PMD was peppered with references to rte_cryptodev artefacts.
Also the pci device it presented to the API layer was
entirely owned by the one cryptodev device instance created by the probe.
This patchset refactors the PMD so one pci device can present out multiple
different device instances to implement different APIs, specifically
cryptodev and compressdev instances.
The code is refactored into
- common code which handles the pci device discovery, configuration,
queue-pair setup, stats gathering and data-path enqueue and dequeue.
- service-specific code for symmetric crypto, which is a thin layer
implementing the cryptodev API and using the common code.
- place-holder files for service-specific code to support asymmetric
crypto and compression services in future.
Fiona Trahe (29):
crypto/qat: use SPDX license
crypto/qat: add qat common header
crypto/qat: add qat device files
crypto/qat: remove unused includes
crypto/qat: add symmetric session file
crypto/qat: change filename crypto to sym
crypto/qat: rename fns for consistency
crypto/qat: renamed sym-specific structs
crypto/qat: make enqueue function generic
crypto/qat: make dequeue function generic
crypto/qat: move generic qp fn to qp file
crypto/qat: separate sym-specific from generic qp setup
crypto/qat: move sym-specific qp code to sym file
crypto/qat: remove dependencies on cryptodev from common
crypto/qat: move defines from sym to qp header file
crypto/qat: create data structures to support different generations
crypto/qat: rename sgl related objects
crypto/qat: move sgl related element to appropriate files
crypto/qat: add QAT PCI device struct
crypto/qat: separate the name used for PCI reg from crypto name
crypto/qat: move to using new device structure
crypto/qat: use common stats structures
crypto/qat: rename functions which depend on cryptodev
crypto/qat: move code into appropriate files
crypto/qat: add lock around csr access and change logic
crypto/qat: remove incorrect usage of bundle number
crypto/qat: cleanups
crypto/qat: create appropriately named device for registration
crypto/qat: add MAX PCI DEVICES flag to config file
Tomasz Jozwiak (1):
crypto/qat: add performance improvement into qat crypto dev.
config/common_base | 4 +
config/rte_config.h | 2 +
drivers/crypto/qat/Makefile | 12 +-
drivers/crypto/qat/meson.build | 11 +-
.../qat/qat_adf/adf_transport_access_macros.h | 56 +-
drivers/crypto/qat/qat_adf/icp_qat_fw.h | 47 +-
drivers/crypto/qat/qat_adf/icp_qat_fw_la.h | 47 +-
drivers/crypto/qat/qat_adf/icp_qat_hw.h | 47 +-
drivers/crypto/qat/qat_adf/qat_algs.h | 72 +-
drivers/crypto/qat/qat_adf/qat_algs_build_desc.c | 1059 ------------
drivers/crypto/qat/qat_asym_pmd.c | 17 +
drivers/crypto/qat/qat_asym_pmd.h | 15 +
drivers/crypto/qat/qat_common.c | 107 ++
drivers/crypto/qat/qat_common.h | 76 +
drivers/crypto/qat/qat_comp_pmd.c | 18 +
drivers/crypto/qat/qat_comp_pmd.h | 29 +
drivers/crypto/qat/qat_crypto.c | 1696 -------------------
drivers/crypto/qat/qat_crypto.h | 150 --
drivers/crypto/qat/qat_crypto_capabilities.h | 557 -------
drivers/crypto/qat/qat_device.c | 241 +++
drivers/crypto/qat/qat_device.h | 78 +
drivers/crypto/qat/qat_logs.h | 2 +-
drivers/crypto/qat/qat_qp.c | 464 ++++--
drivers/crypto/qat/qat_qp.h | 106 ++
drivers/crypto/qat/qat_sym.c | 568 +++++++
drivers/crypto/qat/qat_sym.h | 152 ++
drivers/crypto/qat/qat_sym_capabilities.h | 557 +++++++
drivers/crypto/qat/qat_sym_pmd.c | 335 ++++
drivers/crypto/qat/qat_sym_pmd.h | 39 +
drivers/crypto/qat/qat_sym_session.c | 1723 ++++++++++++++++++++
drivers/crypto/qat/qat_sym_session.h | 143 ++
drivers/crypto/qat/rte_qat_cryptodev.c | 180 --
32 files changed, 4570 insertions(+), 4040 deletions(-)
delete mode 100644 drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
create mode 100644 drivers/crypto/qat/qat_asym_pmd.c
create mode 100644 drivers/crypto/qat/qat_asym_pmd.h
create mode 100644 drivers/crypto/qat/qat_common.c
create mode 100644 drivers/crypto/qat/qat_common.h
create mode 100644 drivers/crypto/qat/qat_comp_pmd.c
create mode 100644 drivers/crypto/qat/qat_comp_pmd.h
delete mode 100644 drivers/crypto/qat/qat_crypto.c
delete mode 100644 drivers/crypto/qat/qat_crypto.h
delete mode 100644 drivers/crypto/qat/qat_crypto_capabilities.h
create mode 100644 drivers/crypto/qat/qat_device.c
create mode 100644 drivers/crypto/qat/qat_device.h
create mode 100644 drivers/crypto/qat/qat_qp.h
create mode 100644 drivers/crypto/qat/qat_sym.c
create mode 100644 drivers/crypto/qat/qat_sym.h
create mode 100644 drivers/crypto/qat/qat_sym_capabilities.h
create mode 100644 drivers/crypto/qat/qat_sym_pmd.c
create mode 100644 drivers/crypto/qat/qat_sym_pmd.h
create mode 100644 drivers/crypto/qat/qat_sym_session.c
create mode 100644 drivers/crypto/qat/qat_sym_session.h
delete mode 100644 drivers/crypto/qat/rte_qat_cryptodev.c
^ permalink raw reply [flat|nested] 109+ messages in thread
* [PATCH 01/30] crypto/qat: use SPDX license
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-18 8:03 ` De Lara Guarch, Pablo
2018-04-06 18:51 ` [PATCH 02/30] crypto/qat: add qat common header Fiona Trahe
` (62 subsequent siblings)
63 siblings, 1 reply; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Use SPDK license for dual-licensed files
and update license date in all files
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 2 +-
drivers/crypto/qat/meson.build | 2 +-
.../qat/qat_adf/adf_transport_access_macros.h | 47 +--------------------
drivers/crypto/qat/qat_adf/icp_qat_fw.h | 47 +--------------------
drivers/crypto/qat/qat_adf/icp_qat_fw_la.h | 47 +--------------------
drivers/crypto/qat/qat_adf/icp_qat_hw.h | 47 +--------------------
drivers/crypto/qat/qat_adf/qat_algs.h | 47 +--------------------
drivers/crypto/qat/qat_adf/qat_algs_build_desc.c | 48 +---------------------
drivers/crypto/qat/qat_crypto.c | 2 +-
drivers/crypto/qat/qat_crypto.h | 2 +-
drivers/crypto/qat/qat_crypto_capabilities.h | 2 +-
drivers/crypto/qat/qat_logs.h | 2 +-
drivers/crypto/qat/qat_qp.c | 2 +-
drivers/crypto/qat/rte_qat_cryptodev.c | 2 +-
14 files changed, 20 insertions(+), 279 deletions(-)
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 260912dca..07266a5e9 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2015 Intel Corporation
+# Copyright(c) 2015-2018 Intel Corporation
include $(RTE_SDK)/mk/rte.vars.mk
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 7b904630b..006cd6557 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
+# Copyright(c) 2017-2018 Intel Corporation
dep = dependency('libcrypto', required: false)
if not dep.found()
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index 4f8f3d130..bfdbc979f 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -1,48 +1,5 @@
-/*
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- * Copyright(c) 2015 Intel Corporation.
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * Contact Information:
- * qat-linux@intel.com
- *
- * BSD LICENSE
- * Copyright(c) 2015 Intel Corporation.
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef ADF_TRANSPORT_ACCESS_MACROS_H
#define ADF_TRANSPORT_ACCESS_MACROS_H
diff --git a/drivers/crypto/qat/qat_adf/icp_qat_fw.h b/drivers/crypto/qat/qat_adf/icp_qat_fw.h
index 5de34d554..ae39b7f11 100644
--- a/drivers/crypto/qat/qat_adf/icp_qat_fw.h
+++ b/drivers/crypto/qat/qat_adf/icp_qat_fw.h
@@ -1,48 +1,5 @@
-/*
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- * Copyright(c) 2015 Intel Corporation.
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * Contact Information:
- * qat-linux@intel.com
- *
- * BSD LICENSE
- * Copyright(c) 2015 Intel Corporation.
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _ICP_QAT_FW_H_
#define _ICP_QAT_FW_H_
diff --git a/drivers/crypto/qat/qat_adf/icp_qat_fw_la.h b/drivers/crypto/qat/qat_adf/icp_qat_fw_la.h
index fbf2b839e..c33bc3fe7 100644
--- a/drivers/crypto/qat/qat_adf/icp_qat_fw_la.h
+++ b/drivers/crypto/qat/qat_adf/icp_qat_fw_la.h
@@ -1,48 +1,5 @@
-/*
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- * Copyright(c) 2015 Intel Corporation.
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * Contact Information:
- * qat-linux@intel.com
- *
- * BSD LICENSE
- * Copyright(c) 2015 Intel Corporation.
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _ICP_QAT_FW_LA_H_
#define _ICP_QAT_FW_LA_H_
diff --git a/drivers/crypto/qat/qat_adf/icp_qat_hw.h b/drivers/crypto/qat/qat_adf/icp_qat_hw.h
index d03688c7a..56e3cf798 100644
--- a/drivers/crypto/qat/qat_adf/icp_qat_hw.h
+++ b/drivers/crypto/qat/qat_adf/icp_qat_hw.h
@@ -1,48 +1,5 @@
-/*
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- * Copyright(c) 2015 Intel Corporation.
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * Contact Information:
- * qat-linux@intel.com
- *
- * BSD LICENSE
- * Copyright(c) 2015 Intel Corporation.
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _ICP_QAT_HW_H_
#define _ICP_QAT_HW_H_
diff --git a/drivers/crypto/qat/qat_adf/qat_algs.h b/drivers/crypto/qat/qat_adf/qat_algs.h
index 802ba95d1..88bd5f00e 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs.h
+++ b/drivers/crypto/qat/qat_adf/qat_algs.h
@@ -1,48 +1,5 @@
-/*
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- * Copyright(c) 2015-2016 Intel Corporation.
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * Contact Information:
- * qat-linux@intel.com
- *
- * BSD LICENSE
- * Copyright(c) 2015-2017 Intel Corporation.
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _ICP_QAT_ALGS_H_
#define _ICP_QAT_ALGS_H_
diff --git a/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c b/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
index 26f854c21..c87ed40fe 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
+++ b/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
@@ -1,50 +1,6 @@
-/*
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- * Copyright(c) 2015-2016 Intel Corporation.
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * Contact Information:
- * qat-linux@intel.com
- *
- * BSD LICENSE
- * Copyright(c) 2015-2017 Intel Corporation.
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * * Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
+ * Copyright(c) 2015-2018 Intel Corporation
*/
-
#include <rte_memcpy.h>
#include <rte_common.h>
#include <rte_spinlock.h>
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index 4afe159d2..d9ce2a136 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2015-2017 Intel Corporation
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#include <stdio.h>
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index c182af2e5..281a142b9 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2015-2016 Intel Corporation
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _QAT_CRYPTO_H_
diff --git a/drivers/crypto/qat/qat_crypto_capabilities.h b/drivers/crypto/qat/qat_crypto_capabilities.h
index 37a6b7cbd..001c32c5d 100644
--- a/drivers/crypto/qat/qat_crypto_capabilities.h
+++ b/drivers/crypto/qat/qat_crypto_capabilities.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2017 Intel Corporation
+ * Copyright(c) 2017-2018 Intel Corporation
*/
#ifndef _QAT_CRYPTO_CAPABILITIES_H_
diff --git a/drivers/crypto/qat/qat_logs.h b/drivers/crypto/qat/qat_logs.h
index 565089a41..c9144bf61 100644
--- a/drivers/crypto/qat/qat_logs.h
+++ b/drivers/crypto/qat/qat_logs.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2015 Intel Corporation
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _QAT_LOGS_H_
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 87b9ce0bb..ba6df7f6c 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2010-2015 Intel Corporation
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#include <rte_common.h>
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index d2a451e7d..c8da07af6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2015-2017 Intel Corporation
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#include <rte_bus_pci.h>
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 02/30] crypto/qat: add qat common header
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
2018-04-06 18:51 ` [PATCH 01/30] crypto/qat: use SPDX license Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 03/30] crypto/qat: add qat device files Fiona Trahe
` (61 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
This commit adds qat_common.h header file.
Following objects were moved to it:
qat_algs.h =>. qat_common.h
- struct qat_alg_buf
- struct qat_alg_buf_list
- struct qat_crypto_op_cookie
- QAT_SGL_MAX_NUMBER
qat_crypto.h => qat_common.h
- CRYPTODEV_NAME_QAT_SYM_PMD
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_adf/qat_algs.h | 25 -------------------
drivers/crypto/qat/qat_common.h | 47 +++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_crypto.h | 9 +------
3 files changed, 48 insertions(+), 33 deletions(-)
create mode 100644 drivers/crypto/qat/qat_common.h
diff --git a/drivers/crypto/qat/qat_adf/qat_algs.h b/drivers/crypto/qat/qat_adf/qat_algs.h
index 88bd5f00e..6c49c6529 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs.h
+++ b/drivers/crypto/qat/qat_adf/qat_algs.h
@@ -32,12 +32,6 @@
ICP_QAT_HW_CIPHER_KEY_CONVERT, \
ICP_QAT_HW_CIPHER_DECRYPT)
-struct qat_alg_buf {
- uint32_t len;
- uint32_t resrvd;
- uint64_t addr;
-} __rte_packed;
-
enum qat_crypto_proto_flag {
QAT_CRYPTO_PROTO_FLAG_NONE = 0,
QAT_CRYPTO_PROTO_FLAG_CCM = 1,
@@ -46,25 +40,6 @@ enum qat_crypto_proto_flag {
QAT_CRYPTO_PROTO_FLAG_ZUC = 4
};
-/*
- * Maximum number of SGL entries
- */
-#define QAT_SGL_MAX_NUMBER 16
-
-struct qat_alg_buf_list {
- uint64_t resrvd;
- uint32_t num_bufs;
- uint32_t num_mapped_bufs;
- struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
-} __rte_packed __rte_cache_aligned;
-
-struct qat_crypto_op_cookie {
- struct qat_alg_buf_list qat_sgl_list_src;
- struct qat_alg_buf_list qat_sgl_list_dst;
- rte_iova_t qat_sgl_src_phys_addr;
- rte_iova_t qat_sgl_dst_phys_addr;
-};
-
/* Common content descriptor */
struct qat_alg_cd {
struct icp_qat_hw_cipher_algo_blk cipher;
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
new file mode 100644
index 000000000..293b6f700
--- /dev/null
+++ b/drivers/crypto/qat/qat_common.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_COMMON_H_
+#define _QAT_COMMON_H_
+
+#include <stdint.h>
+
+/**< Intel(R) QAT Symmetric Crypto PMD device name */
+#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+
+/*
+ * Maximum number of SGL entries
+ */
+#define QAT_SGL_MAX_NUMBER 16
+
+/* Intel(R) QuickAssist Technology device generation is enumerated
+ * from one according to the generation of the device
+ */
+
+enum qat_device_gen {
+ QAT_GEN1 = 1,
+ QAT_GEN2,
+};
+
+/**< Common struct for scatter-gather list operations */
+struct qat_alg_buf {
+ uint32_t len;
+ uint32_t resrvd;
+ uint64_t addr;
+} __rte_packed;
+
+struct qat_alg_buf_list {
+ uint64_t resrvd;
+ uint32_t num_bufs;
+ uint32_t num_mapped_bufs;
+ struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
+} __rte_packed __rte_cache_aligned;
+
+struct qat_crypto_op_cookie {
+ struct qat_alg_buf_list qat_sgl_list_src;
+ struct qat_alg_buf_list qat_sgl_list_dst;
+ phys_addr_t qat_sgl_src_phys_addr;
+ phys_addr_t qat_sgl_dst_phys_addr;
+};
+
+#endif /* _QAT_QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 281a142b9..5190d25e5 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -8,11 +8,9 @@
#include <rte_cryptodev_pmd.h>
#include <rte_memzone.h>
+#include "qat_common.h"
#include "qat_crypto_capabilities.h"
-#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
-/**< Intel QAT Symmetric Crypto PMD device name */
-
/*
* This macro rounds up a number to a be a multiple of
* the alignment when the alignment is a power of 2
@@ -30,11 +28,6 @@
struct qat_session;
-enum qat_device_gen {
- QAT_GEN1 = 1,
- QAT_GEN2,
-};
-
/**
* Structure associated with each queue.
*/
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 03/30] crypto/qat: add qat device files
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
2018-04-06 18:51 ` [PATCH 01/30] crypto/qat: use SPDX license Fiona Trahe
2018-04-06 18:51 ` [PATCH 02/30] crypto/qat: add qat common header Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 04/30] crypto/qat: remove unused includes Fiona Trahe
` (60 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
This commit adds new qat_device file.
Following objects were moved:
qat_crypto.h => qat_device.h
- struct qat_pmd_private
- uint8_t cryptodev_qat_driver_id
- int qat_crypto_sym_qp_release (EXTERN)
- int qat_dev_config()
- int qat_dev_start()
- void qat_dev_stop()
- int qat_dev_close()
- void qat_dev_info_get()
qat_crypto.c => qat_device.c
- int qat_dev_config()
- int qat_dev_start()
- void qat_dev_stop()
- int qat_dev_close()
- void qat_dev_info_get()
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 1 +
drivers/crypto/qat/meson.build | 3 ++-
drivers/crypto/qat/qat_crypto.c | 51 ------------------------------------
drivers/crypto/qat/qat_crypto.h | 22 +---------------
drivers/crypto/qat/qat_device.c | 57 +++++++++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_device.h | 36 ++++++++++++++++++++++++++
6 files changed, 97 insertions(+), 73 deletions(-)
create mode 100644 drivers/crypto/qat/qat_device.c
create mode 100644 drivers/crypto/qat/qat_device.h
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 07266a5e9..6bdd11679 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -22,6 +22,7 @@ LDLIBS += -lrte_pci -lrte_bus_pci
# library source files
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_adf/qat_algs_build_desc.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 006cd6557..51630e31b 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -7,7 +7,8 @@ if not dep.found()
endif
sources = files('qat_crypto.c', 'qat_qp.c',
'qat_adf/qat_algs_build_desc.c',
- 'rte_qat_cryptodev.c')
+ 'rte_qat_cryptodev.c',
+ 'qat_device.c')
includes += include_directories('qat_adf')
deps += ['bus_pci']
ext_deps += dep
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index d9ce2a136..928a50475 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -1609,57 +1609,6 @@ static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
return data - mult;
}
-int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
- __rte_unused struct rte_cryptodev_config *config)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
-}
-
-int qat_dev_close(struct rte_cryptodev *dev)
-{
- int i, ret;
-
- PMD_INIT_FUNC_TRACE();
-
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_crypto_sym_qp_release(dev, i);
- if (ret < 0)
- return ret;
- }
-
- return 0;
-}
-
-void qat_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info)
-{
- struct qat_pmd_private *internals = dev->data->dev_private;
-
- PMD_INIT_FUNC_TRACE();
- if (info != NULL) {
- info->max_nb_queue_pairs =
- ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV;
- info->feature_flags = dev->feature_flags;
- info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = internals->max_nb_sessions;
- info->driver_id = cryptodev_qat_driver_id;
- info->pci_dev = RTE_DEV_TO_PCI(dev->device);
- }
-}
-
void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
{
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 5190d25e5..46f03ccde 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -9,6 +9,7 @@
#include <rte_memzone.h>
#include "qat_common.h"
+#include "qat_device.h"
#include "qat_crypto_capabilities.h"
/*
@@ -64,27 +65,6 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
} __rte_cache_aligned;
-/** private data structure for each QAT device */
-struct qat_pmd_private {
- unsigned max_nb_queue_pairs;
- /**< Max number of queue pairs supported by device */
- unsigned max_nb_sessions;
- /**< Max number of sessions supported by device */
- enum qat_device_gen qat_dev_gen;
- /**< QAT device generation */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
-};
-
-extern uint8_t cryptodev_qat_driver_id;
-
-int qat_dev_config(struct rte_cryptodev *dev,
- struct rte_cryptodev_config *config);
-int qat_dev_start(struct rte_cryptodev *dev);
-void qat_dev_stop(struct rte_cryptodev *dev);
-int qat_dev_close(struct rte_cryptodev *dev);
-void qat_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info);
-
void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
new file mode 100644
index 000000000..c2bf9b7a7
--- /dev/null
+++ b/drivers/crypto/qat/qat_device.c
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_device.h"
+#include "adf_transport_access_macros.h"
+
+int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused struct rte_cryptodev_config *config)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+}
+
+int qat_dev_close(struct rte_cryptodev *dev)
+{
+ int i, ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ ret = qat_crypto_sym_qp_release(dev, i);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+void qat_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info)
+{
+ struct qat_pmd_private *internals = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+ if (info != NULL) {
+ info->max_nb_queue_pairs =
+ ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV;
+ info->feature_flags = dev->feature_flags;
+ info->capabilities = internals->qat_dev_capabilities;
+ info->sym.max_nb_sessions = internals->max_nb_sessions;
+ info->driver_id = cryptodev_qat_driver_id;
+ info->pci_dev = RTE_DEV_TO_PCI(dev->device);
+ }
+}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
new file mode 100644
index 000000000..5c48fdb93
--- /dev/null
+++ b/drivers/crypto/qat/qat_device.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_DEVICE_H_
+#define _QAT_DEVICE_H_
+
+#include <rte_cryptodev_pmd.h>
+#include <rte_bus_pci.h>
+#include "qat_common.h"
+#include "qat_logs.h"
+
+extern uint8_t cryptodev_qat_driver_id;
+
+extern int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+ uint16_t queue_pair_id);
+
+/** private data structure for each QAT device */
+struct qat_pmd_private {
+ unsigned int max_nb_queue_pairs;
+ /**< Max number of queue pairs supported by device */
+ unsigned int max_nb_sessions;
+ /**< Max number of sessions supported by device */
+ enum qat_device_gen qat_dev_gen;
+ /**< QAT device generation */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+};
+
+int qat_dev_config(struct rte_cryptodev *dev,
+ struct rte_cryptodev_config *config);
+int qat_dev_start(struct rte_cryptodev *dev);
+void qat_dev_stop(struct rte_cryptodev *dev);
+int qat_dev_close(struct rte_cryptodev *dev);
+void qat_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info);
+
+#endif /* _QAT_DEVICE_H_ */
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 04/30] crypto/qat: remove unused includes
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (2 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 03/30] crypto/qat: add qat device files Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 05/30] crypto/qat: add symmetric session file Fiona Trahe
` (59 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
This commit removes unused includes from qat_crypto.c
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_crypto.c | 22 ----------------------
1 file changed, 22 deletions(-)
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index 928a50475..7f2c2c86b 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -2,30 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
-#include <stdio.h>
-#include <stdlib.h>
-#include <strings.h>
-#include <string.h>
-#include <inttypes.h>
-#include <errno.h>
-#include <sys/queue.h>
-#include <stdarg.h>
-
-#include <rte_common.h>
-#include <rte_log.h>
-#include <rte_debug.h>
-#include <rte_memory.h>
-#include <rte_tailq.h>
-#include <rte_malloc.h>
-#include <rte_launch.h>
-#include <rte_eal.h>
-#include <rte_per_lcore.h>
-#include <rte_lcore.h>
-#include <rte_branch_prediction.h>
#include <rte_mempool.h>
#include <rte_mbuf.h>
-#include <rte_string_fns.h>
-#include <rte_spinlock.h>
#include <rte_hexdump.h>
#include <rte_crypto_sym.h>
#include <rte_byteorder.h>
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 05/30] crypto/qat: add symmetric session file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (3 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 04/30] crypto/qat: remove unused includes Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 06/30] crypto/qat: change filename crypto to sym Fiona Trahe
` (58 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
This commit adds qat_sym_session.c/h files and moves objects
from qat_algs_build_desc and qat_algs.h
Following objects were moved:
qat_adf/qat_algs_build_desc.c => qat_sym_session.c
- all objects -
qat_adf/qat_algs.h => qat_sym_session.h
- enum qat_crypto_proto_flag
- struct qat_alg_cd
- struct qat_session
- int qat_get_inter_state_size()
- int qat_alg_aead_session_create_content_desc_cipher()
- int qat_alg_aead_session_create_content_desc_auth()
- void qat_alg_init_common_hdr()
- int qat_alg_validate_aes_key()
- int qat_alg_validate_aes_docsisbpi_key()
- int qat_alg_validate_snow3g_key()
- int qat_alg_validate_kasumi_key()
- int qat_alg_validate_3des_key()
- int qat_alg_validate_des_key()
- int qat_cipher_get_block_size()
- int qat_alg_validate_zuc_key()
-- all macros
qat_crypto.h => qat_sym_session.h
int qat_crypto_sym_configure_session()
int qat_crypto_set_session_parameters()
int qat_crypto_sym_configure_session_aead()
int qat_crypto_sym_configure_session_cipher()
int qat_crypto_sym_configure_session_auth()
int qat_alg_aead_session_create_content_desc_cipher()
int qat_alg_aead_session_create_content_desc_auth()
static struct rte_crypto_auth_xform qat_get_auth_xform()
static struct rte_crypto_cipher_xform qat_get_cipher_xform()
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 2 +-
drivers/crypto/qat/meson.build | 2 +-
drivers/crypto/qat/qat_crypto.c | 704 +-------------------
drivers/crypto/qat/qat_crypto.h | 36 -
drivers/crypto/qat/qat_qp.c | 1 -
.../qat_algs_build_desc.c => qat_sym_session.c} | 726 ++++++++++++++++++++-
drivers/crypto/qat/qat_sym_session.h | 142 ++++
drivers/crypto/qat/rte_qat_cryptodev.c | 1 +
8 files changed, 863 insertions(+), 751 deletions(-)
rename drivers/crypto/qat/{qat_adf/qat_algs_build_desc.c => qat_sym_session.c} (61%)
create mode 100644 drivers/crypto/qat/qat_sym_session.h
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 6bdd11679..c63c1515e 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -24,7 +24,7 @@ LDLIBS += -lrte_pci -lrte_bus_pci
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_adf/qat_algs_build_desc.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
# export include files
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 51630e31b..be4282a83 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -6,7 +6,7 @@ if not dep.found()
build = false
endif
sources = files('qat_crypto.c', 'qat_qp.c',
- 'qat_adf/qat_algs_build_desc.c',
+ 'qat_sym_session.c',
'rte_qat_cryptodev.c',
'qat_device.c')
includes += include_directories('qat_adf')
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index 7f2c2c86b..96a1b78f0 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -13,7 +13,7 @@
#include <openssl/evp.h>
#include "qat_logs.h"
-#include "qat_algs.h"
+#include "qat_sym_session.h"
#include "qat_crypto.h"
#include "adf_transport_access_macros.h"
@@ -23,46 +23,6 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-static int
-qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
- struct qat_pmd_private *internals) {
- int i = 0;
- const struct rte_cryptodev_capabilities *capability;
-
- while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
- RTE_CRYPTO_OP_TYPE_UNDEFINED) {
- if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
- continue;
-
- if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_CIPHER)
- continue;
-
- if (capability->sym.cipher.algo == algo)
- return 1;
- }
- return 0;
-}
-
-static int
-qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
- struct qat_pmd_private *internals) {
- int i = 0;
- const struct rte_cryptodev_capabilities *capability;
-
- while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
- RTE_CRYPTO_OP_TYPE_UNDEFINED) {
- if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
- continue;
-
- if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_AUTH)
- continue;
-
- if (capability->sym.auth.algo == algo)
- return 1;
- }
- return 0;
-}
-
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -124,49 +84,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
/** Creates a context in either AES or DES in ECB mode
* Depends on openssl libcrypto
*/
-static int
-bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
- enum rte_crypto_cipher_operation direction __rte_unused,
- uint8_t *key, void **ctx)
-{
- const EVP_CIPHER *algo = NULL;
- int ret;
- *ctx = EVP_CIPHER_CTX_new();
-
- if (*ctx == NULL) {
- ret = -ENOMEM;
- goto ctx_init_err;
- }
-
- if (cryptodev_algo == RTE_CRYPTO_CIPHER_DES_DOCSISBPI)
- algo = EVP_des_ecb();
- else
- algo = EVP_aes_128_ecb();
-
- /* IV will be ECB encrypted whether direction is encrypt or decrypt*/
- if (EVP_EncryptInit_ex(*ctx, algo, NULL, key, 0) != 1) {
- ret = -EINVAL;
- goto ctx_init_err;
- }
-
- return 0;
-
-ctx_init_err:
- if (*ctx != NULL)
- EVP_CIPHER_CTX_free(*ctx);
- return ret;
-}
-
-/** Frees a context previously created
- * Depends on openssl libcrypto
- */
-static void
-bpi_cipher_ctx_free(void *bpi_ctx)
-{
- if (bpi_ctx != NULL)
- EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
-}
-
static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
@@ -174,625 +91,6 @@ static inline int
qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
-void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
- struct rte_cryptodev_sym_session *sess)
-{
- PMD_INIT_FUNC_TRACE();
- uint8_t index = dev->driver_id;
- void *sess_priv = get_session_private_data(sess, index);
- struct qat_session *s = (struct qat_session *)sess_priv;
-
- if (sess_priv) {
- if (s->bpi_ctx)
- bpi_cipher_ctx_free(s->bpi_ctx);
- memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
- struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
- set_session_private_data(sess, index, NULL);
- rte_mempool_put(sess_mp, sess_priv);
- }
-}
-
-static int
-qat_get_cmd_id(const struct rte_crypto_sym_xform *xform)
-{
- /* Cipher Only */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
- return ICP_QAT_FW_LA_CMD_CIPHER;
-
- /* Authentication Only */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
- return ICP_QAT_FW_LA_CMD_AUTH;
-
- /* AEAD */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- /* AES-GCM and AES-CCM works with different direction
- * GCM first encrypts and generate hash where AES-CCM
- * first generate hash and encrypts. Similar relation
- * applies to decryption.
- */
- if (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT)
- if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- else
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- else
- if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- else
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- }
-
- if (xform->next == NULL)
- return -1;
-
- /* Cipher then Authenticate */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
- xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH)
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
-
- /* Authenticate then Cipher */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
- xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
-
- return -1;
-}
-
-static struct rte_crypto_auth_xform *
-qat_get_auth_xform(struct rte_crypto_sym_xform *xform)
-{
- do {
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH)
- return &xform->auth;
-
- xform = xform->next;
- } while (xform);
-
- return NULL;
-}
-
-static struct rte_crypto_cipher_xform *
-qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
-{
- do {
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
- return &xform->cipher;
-
- xform = xform->next;
- } while (xform);
-
- return NULL;
-}
-
-int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct qat_pmd_private *internals = dev->data->dev_private;
- struct rte_crypto_cipher_xform *cipher_xform = NULL;
- int ret;
-
- /* Get cipher xform from crypto xform chain */
- cipher_xform = qat_get_cipher_xform(xform);
-
- session->cipher_iv.offset = cipher_xform->iv.offset;
- session->cipher_iv.length = cipher_xform->iv.length;
-
- switch (cipher_xform->algo) {
- case RTE_CRYPTO_CIPHER_AES_CBC:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_AES_CTR:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- break;
- case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
- if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_NULL:
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_KASUMI_F8:
- if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_CBC:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_DES_CBC:
- if (qat_alg_validate_des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_CTR:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- break;
- case RTE_CRYPTO_CIPHER_DES_DOCSISBPI:
- ret = bpi_cipher_ctx_init(
- cipher_xform->algo,
- cipher_xform->op,
- cipher_xform->key.data,
- &session->bpi_ctx);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
- goto error_out;
- }
- if (qat_alg_validate_des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_AES_DOCSISBPI:
- ret = bpi_cipher_ctx_init(
- cipher_xform->algo,
- cipher_xform->op,
- cipher_xform->key.data,
- &session->bpi_ctx);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
- goto error_out;
- }
- if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_ZUC_EEA3:
- if (!qat_is_cipher_alg_supported(
- cipher_xform->algo, internals)) {
- PMD_DRV_LOG(ERR, "%s not supported on this device",
- rte_crypto_cipher_algorithm_strings
- [cipher_xform->algo]);
- ret = -ENOTSUP;
- goto error_out;
- }
- if (qat_alg_validate_zuc_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_ECB:
- case RTE_CRYPTO_CIPHER_AES_ECB:
- case RTE_CRYPTO_CIPHER_AES_F8:
- case RTE_CRYPTO_CIPHER_AES_XTS:
- case RTE_CRYPTO_CIPHER_ARC4:
- PMD_DRV_LOG(ERR, "Crypto QAT PMD: Unsupported Cipher alg %u",
- cipher_xform->algo);
- ret = -ENOTSUP;
- goto error_out;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined Cipher specified %u\n",
- cipher_xform->algo);
- ret = -EINVAL;
- goto error_out;
- }
-
- if (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- else
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- cipher_xform->key.data,
- cipher_xform->key.length)) {
- ret = -EINVAL;
- goto error_out;
- }
-
- return 0;
-
-error_out:
- if (session->bpi_ctx) {
- bpi_cipher_ctx_free(session->bpi_ctx);
- session->bpi_ctx = NULL;
- }
- return ret;
-}
-
-int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct rte_cryptodev_sym_session *sess,
- struct rte_mempool *mempool)
-{
- void *sess_private_data;
- int ret;
-
- if (rte_mempool_get(mempool, &sess_private_data)) {
- CDEV_LOG_ERR(
- "Couldn't get object from session mempool");
- return -ENOMEM;
- }
-
- ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "Crypto QAT PMD: failed to configure "
- "session parameters");
-
- /* Return session to mempool */
- rte_mempool_put(mempool, sess_private_data);
- return ret;
- }
-
- set_session_private_data(sess, dev->driver_id,
- sess_private_data);
-
- return 0;
-}
-
-int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform, void *session_private)
-{
- struct qat_session *session = session_private;
- int ret;
-
- int qat_cmd_id;
- PMD_INIT_FUNC_TRACE();
-
- /* Set context descriptor physical address */
- session->cd_paddr = rte_mempool_virt2iova(session) +
- offsetof(struct qat_session, cd);
-
- session->min_qat_dev_gen = QAT_GEN1;
-
- /* Get requested QAT command id */
- qat_cmd_id = qat_get_cmd_id(xform);
- if (qat_cmd_id < 0 || qat_cmd_id >= ICP_QAT_FW_LA_CMD_DELIMITER) {
- PMD_DRV_LOG(ERR, "Unsupported xform chain requested");
- return -ENOTSUP;
- }
- session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
- switch (session->qat_cmd) {
- case ICP_QAT_FW_LA_CMD_CIPHER:
- ret = qat_crypto_sym_configure_session_cipher(dev, xform, session);
- if (ret < 0)
- return ret;
- break;
- case ICP_QAT_FW_LA_CMD_AUTH:
- ret = qat_crypto_sym_configure_session_auth(dev, xform, session);
- if (ret < 0)
- return ret;
- break;
- case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
- session);
- if (ret < 0)
- return ret;
- } else {
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
- if (ret < 0)
- return ret;
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
- if (ret < 0)
- return ret;
- }
- break;
- case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
- session);
- if (ret < 0)
- return ret;
- } else {
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
- if (ret < 0)
- return ret;
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
- if (ret < 0)
- return ret;
- }
- break;
- case ICP_QAT_FW_LA_CMD_TRNG_GET_RANDOM:
- case ICP_QAT_FW_LA_CMD_TRNG_TEST:
- case ICP_QAT_FW_LA_CMD_SSL3_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_TLS_V1_1_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_TLS_V1_2_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_MGF1:
- case ICP_QAT_FW_LA_CMD_AUTH_PRE_COMP:
- case ICP_QAT_FW_LA_CMD_CIPHER_PRE_COMP:
- case ICP_QAT_FW_LA_CMD_DELIMITER:
- PMD_DRV_LOG(ERR, "Unsupported Service %u",
- session->qat_cmd);
- return -ENOTSUP;
- default:
- PMD_DRV_LOG(ERR, "Unsupported Service %u",
- session->qat_cmd);
- return -ENOTSUP;
- }
-
- return 0;
-}
-
-int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct rte_crypto_auth_xform *auth_xform = NULL;
- struct qat_pmd_private *internals = dev->data->dev_private;
- auth_xform = qat_get_auth_xform(xform);
- uint8_t *key_data = auth_xform->key.data;
- uint8_t key_length = auth_xform->key.length;
-
- switch (auth_xform->algo) {
- case RTE_CRYPTO_AUTH_SHA1_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA1;
- break;
- case RTE_CRYPTO_AUTH_SHA224_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA224;
- break;
- case RTE_CRYPTO_AUTH_SHA256_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA256;
- break;
- case RTE_CRYPTO_AUTH_SHA384_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA384;
- break;
- case RTE_CRYPTO_AUTH_SHA512_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA512;
- break;
- case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
- break;
- case RTE_CRYPTO_AUTH_AES_GMAC:
- if (qat_alg_validate_aes_key(auth_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
-
- break;
- case RTE_CRYPTO_AUTH_SNOW3G_UIA2:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SNOW_3G_UIA2;
- break;
- case RTE_CRYPTO_AUTH_MD5_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_MD5;
- break;
- case RTE_CRYPTO_AUTH_NULL:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_NULL;
- break;
- case RTE_CRYPTO_AUTH_KASUMI_F9:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_KASUMI_F9;
- break;
- case RTE_CRYPTO_AUTH_ZUC_EIA3:
- if (!qat_is_auth_alg_supported(auth_xform->algo, internals)) {
- PMD_DRV_LOG(ERR, "%s not supported on this device",
- rte_crypto_auth_algorithm_strings
- [auth_xform->algo]);
- return -ENOTSUP;
- }
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_ZUC_3G_128_EIA3;
- break;
- case RTE_CRYPTO_AUTH_SHA1:
- case RTE_CRYPTO_AUTH_SHA256:
- case RTE_CRYPTO_AUTH_SHA512:
- case RTE_CRYPTO_AUTH_SHA224:
- case RTE_CRYPTO_AUTH_SHA384:
- case RTE_CRYPTO_AUTH_MD5:
- case RTE_CRYPTO_AUTH_AES_CMAC:
- case RTE_CRYPTO_AUTH_AES_CBC_MAC:
- PMD_DRV_LOG(ERR, "Crypto: Unsupported hash alg %u",
- auth_xform->algo);
- return -ENOTSUP;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined Hash algo %u specified",
- auth_xform->algo);
- return -EINVAL;
- }
-
- session->auth_iv.offset = auth_xform->iv.offset;
- session->auth_iv.length = auth_xform->iv.length;
-
- if (auth_xform->algo == RTE_CRYPTO_AUTH_AES_GMAC) {
- if (auth_xform->op == RTE_CRYPTO_AUTH_OP_GENERATE) {
- session->qat_cmd = ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- /*
- * It needs to create cipher desc content first,
- * then authentication
- */
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- auth_xform->key.data,
- auth_xform->key.length))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
- } else {
- session->qat_cmd = ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- /*
- * It needs to create authentication desc content first,
- * then cipher
- */
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- auth_xform->key.data,
- auth_xform->key.length))
- return -EINVAL;
- }
- /* Restore to authentication only only */
- session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
- } else {
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
- }
-
- session->digest_length = auth_xform->digest_length;
- return 0;
-}
-
-int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct rte_crypto_aead_xform *aead_xform = &xform->aead;
- enum rte_crypto_auth_operation crypto_operation;
-
- /*
- * Store AEAD IV parameters as cipher IV,
- * to avoid unnecessary memory usage
- */
- session->cipher_iv.offset = xform->aead.iv.offset;
- session->cipher_iv.length = xform->aead.iv.length;
-
- switch (aead_xform->algo) {
- case RTE_CRYPTO_AEAD_AES_GCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
- break;
- case RTE_CRYPTO_AEAD_AES_CCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC;
- break;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined AEAD specified %u\n",
- aead_xform->algo);
- return -EINVAL;
- }
-
- if ((aead_xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT &&
- aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM) ||
- (aead_xform->op == RTE_CRYPTO_AEAD_OP_DECRYPT &&
- aead_xform->algo == RTE_CRYPTO_AEAD_AES_CCM)) {
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- /*
- * It needs to create cipher desc content first,
- * then authentication
- */
-
- crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
- RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- aead_xform->key.data,
- aead_xform->key.length))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- aead_xform->key.data,
- aead_xform->key.length,
- aead_xform->aad_length,
- aead_xform->digest_length,
- crypto_operation))
- return -EINVAL;
- } else {
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- /*
- * It needs to create authentication desc content first,
- * then cipher
- */
-
- crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
- RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- aead_xform->key.data,
- aead_xform->key.length,
- aead_xform->aad_length,
- aead_xform->digest_length,
- crypto_operation))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- aead_xform->key.data,
- aead_xform->key.length))
- return -EINVAL;
- }
-
- session->digest_length = aead_xform->digest_length;
- return 0;
-}
-
-unsigned qat_crypto_sym_get_session_private_size(
- struct rte_cryptodev *dev __rte_unused)
-{
- return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
-}
-
static inline uint32_t
qat_bpicipher_preprocess(struct qat_session *ctx,
struct rte_crypto_op *op)
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 46f03ccde..e3873171c 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -75,42 +75,6 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-int
-qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
- unsigned nb_objs, unsigned obj_cache_size, int socket_id);
-
-extern unsigned
-qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
-
-extern int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct rte_cryptodev_sym_session *sess,
- struct rte_mempool *mempool);
-
-
-int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform, void *session_private);
-
-int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-
-extern void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
- struct rte_cryptodev_sym_session *session);
extern uint16_t
qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index ba6df7f6c..e42acd877 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -14,7 +14,6 @@
#include "qat_logs.h"
#include "qat_crypto.h"
-#include "qat_algs.h"
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
diff --git a/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c b/drivers/crypto/qat/qat_sym_session.c
similarity index 61%
rename from drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
rename to drivers/crypto/qat/qat_sym_session.c
index c87ed40fe..a67e22919 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -9,13 +9,722 @@
#include <rte_malloc.h>
#include <rte_crypto_sym.h>
-#include "../qat_logs.h"
+#include "qat_logs.h"
+#include "qat_device.h"
#include <openssl/sha.h> /* Needed to calculate pre-compute values */
#include <openssl/aes.h> /* Needed to calculate pre-compute values */
#include <openssl/md5.h> /* Needed to calculate pre-compute values */
+#include <openssl/evp.h>
-#include "qat_algs.h"
+#include "qat_sym_session.h"
+
+/** Frees a context previously created
+ * Depends on openssl libcrypto
+ */
+static void
+bpi_cipher_ctx_free(void *bpi_ctx)
+{
+ if (bpi_ctx != NULL)
+ EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
+}
+
+static int
+bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
+ enum rte_crypto_cipher_operation direction __rte_unused,
+ uint8_t *key, void **ctx)
+{
+ const EVP_CIPHER *algo = NULL;
+ int ret;
+ *ctx = EVP_CIPHER_CTX_new();
+
+ if (*ctx == NULL) {
+ ret = -ENOMEM;
+ goto ctx_init_err;
+ }
+
+ if (cryptodev_algo == RTE_CRYPTO_CIPHER_DES_DOCSISBPI)
+ algo = EVP_des_ecb();
+ else
+ algo = EVP_aes_128_ecb();
+
+ /* IV will be ECB encrypted whether direction is encrypt or decrypt*/
+ if (EVP_EncryptInit_ex(*ctx, algo, NULL, key, 0) != 1) {
+ ret = -EINVAL;
+ goto ctx_init_err;
+ }
+
+ return 0;
+
+ctx_init_err:
+ if (*ctx != NULL)
+ EVP_CIPHER_CTX_free(*ctx);
+ return ret;
+}
+
+static int
+qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
+ struct qat_pmd_private *internals) {
+ int i = 0;
+ const struct rte_cryptodev_capabilities *capability;
+
+ while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
+ RTE_CRYPTO_OP_TYPE_UNDEFINED) {
+ if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
+ continue;
+
+ if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_CIPHER)
+ continue;
+
+ if (capability->sym.cipher.algo == algo)
+ return 1;
+ }
+ return 0;
+}
+
+static int
+qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
+ struct qat_pmd_private *internals) {
+ int i = 0;
+ const struct rte_cryptodev_capabilities *capability;
+
+ while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
+ RTE_CRYPTO_OP_TYPE_UNDEFINED) {
+ if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
+ continue;
+
+ if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_AUTH)
+ continue;
+
+ if (capability->sym.auth.algo == algo)
+ return 1;
+ }
+ return 0;
+}
+
+void
+qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess)
+{
+ PMD_INIT_FUNC_TRACE();
+ uint8_t index = dev->driver_id;
+ void *sess_priv = get_session_private_data(sess, index);
+ struct qat_session *s = (struct qat_session *)sess_priv;
+
+ if (sess_priv) {
+ if (s->bpi_ctx)
+ bpi_cipher_ctx_free(s->bpi_ctx);
+ memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
+ struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
+
+ set_session_private_data(sess, index, NULL);
+ rte_mempool_put(sess_mp, sess_priv);
+ }
+}
+
+static int
+qat_get_cmd_id(const struct rte_crypto_sym_xform *xform)
+{
+ /* Cipher Only */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
+ return ICP_QAT_FW_LA_CMD_CIPHER;
+
+ /* Authentication Only */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
+ return ICP_QAT_FW_LA_CMD_AUTH;
+
+ /* AEAD */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ /* AES-GCM and AES-CCM works with different direction
+ * GCM first encrypts and generate hash where AES-CCM
+ * first generate hash and encrypts. Similar relation
+ * applies to decryption.
+ */
+ if (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT)
+ if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ else
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ else
+ if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ else
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ }
+
+ if (xform->next == NULL)
+ return -1;
+
+ /* Cipher then Authenticate */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
+ xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH)
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+
+ /* Authenticate then Cipher */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
+ xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+
+ return -1;
+}
+
+static struct rte_crypto_auth_xform *
+qat_get_auth_xform(struct rte_crypto_sym_xform *xform)
+{
+ do {
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH)
+ return &xform->auth;
+
+ xform = xform->next;
+ } while (xform);
+
+ return NULL;
+}
+
+static struct rte_crypto_cipher_xform *
+qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
+{
+ do {
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
+ return &xform->cipher;
+
+ xform = xform->next;
+ } while (xform);
+
+ return NULL;
+}
+
+int
+qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct qat_pmd_private *internals = dev->data->dev_private;
+ struct rte_crypto_cipher_xform *cipher_xform = NULL;
+ int ret;
+
+ /* Get cipher xform from crypto xform chain */
+ cipher_xform = qat_get_cipher_xform(xform);
+
+ session->cipher_iv.offset = cipher_xform->iv.offset;
+ session->cipher_iv.length = cipher_xform->iv.length;
+
+ switch (cipher_xform->algo) {
+ case RTE_CRYPTO_CIPHER_AES_CBC:
+ if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_AES_CTR:
+ if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
+ if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_NULL:
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_KASUMI_F8:
+ if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_CBC:
+ if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_DES_CBC:
+ if (qat_alg_validate_des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_CTR:
+ if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_DES_DOCSISBPI:
+ ret = bpi_cipher_ctx_init(
+ cipher_xform->algo,
+ cipher_xform->op,
+ cipher_xform->key.data,
+ &session->bpi_ctx);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
+ goto error_out;
+ }
+ if (qat_alg_validate_des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_AES_DOCSISBPI:
+ ret = bpi_cipher_ctx_init(
+ cipher_xform->algo,
+ cipher_xform->op,
+ cipher_xform->key.data,
+ &session->bpi_ctx);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
+ goto error_out;
+ }
+ if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_ZUC_EEA3:
+ if (!qat_is_cipher_alg_supported(
+ cipher_xform->algo, internals)) {
+ PMD_DRV_LOG(ERR, "%s not supported on this device",
+ rte_crypto_cipher_algorithm_strings
+ [cipher_xform->algo]);
+ ret = -ENOTSUP;
+ goto error_out;
+ }
+ if (qat_alg_validate_zuc_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_ECB:
+ case RTE_CRYPTO_CIPHER_AES_ECB:
+ case RTE_CRYPTO_CIPHER_AES_F8:
+ case RTE_CRYPTO_CIPHER_AES_XTS:
+ case RTE_CRYPTO_CIPHER_ARC4:
+ PMD_DRV_LOG(ERR, "Crypto QAT PMD: Unsupported Cipher alg %u",
+ cipher_xform->algo);
+ ret = -ENOTSUP;
+ goto error_out;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined Cipher specified %u\n",
+ cipher_xform->algo);
+ ret = -EINVAL;
+ goto error_out;
+ }
+
+ if (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ else
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ cipher_xform->key.data,
+ cipher_xform->key.length)) {
+ ret = -EINVAL;
+ goto error_out;
+ }
+
+ return 0;
+
+error_out:
+ if (session->bpi_ctx) {
+ bpi_cipher_ctx_free(session->bpi_ctx);
+ session->bpi_ctx = NULL;
+ }
+ return ret;
+}
+
+int
+qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct rte_cryptodev_sym_session *sess,
+ struct rte_mempool *mempool)
+{
+ void *sess_private_data;
+ int ret;
+
+ if (rte_mempool_get(mempool, &sess_private_data)) {
+ CDEV_LOG_ERR(
+ "Couldn't get object from session mempool");
+ return -ENOMEM;
+ }
+
+ ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR,
+ "Crypto QAT PMD: failed to configure session parameters");
+
+ /* Return session to mempool */
+ rte_mempool_put(mempool, sess_private_data);
+ return ret;
+ }
+
+ set_session_private_data(sess, dev->driver_id,
+ sess_private_data);
+
+ return 0;
+}
+
+int
+qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform, void *session_private)
+{
+ struct qat_session *session = session_private;
+ int ret;
+ int qat_cmd_id;
+
+ PMD_INIT_FUNC_TRACE();
+
+ /* Set context descriptor physical address */
+ session->cd_paddr = rte_mempool_virt2iova(session) +
+ offsetof(struct qat_session, cd);
+
+ session->min_qat_dev_gen = QAT_GEN1;
+
+ /* Get requested QAT command id */
+ qat_cmd_id = qat_get_cmd_id(xform);
+ if (qat_cmd_id < 0 || qat_cmd_id >= ICP_QAT_FW_LA_CMD_DELIMITER) {
+ PMD_DRV_LOG(ERR, "Unsupported xform chain requested");
+ return -ENOTSUP;
+ }
+ session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
+ switch (session->qat_cmd) {
+ case ICP_QAT_FW_LA_CMD_CIPHER:
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ break;
+ case ICP_QAT_FW_LA_CMD_AUTH:
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ break;
+ case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ ret = qat_crypto_sym_configure_session_aead(xform,
+ session);
+ if (ret < 0)
+ return ret;
+ } else {
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ }
+ break;
+ case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ ret = qat_crypto_sym_configure_session_aead(xform,
+ session);
+ if (ret < 0)
+ return ret;
+ } else {
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ }
+ break;
+ case ICP_QAT_FW_LA_CMD_TRNG_GET_RANDOM:
+ case ICP_QAT_FW_LA_CMD_TRNG_TEST:
+ case ICP_QAT_FW_LA_CMD_SSL3_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_TLS_V1_1_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_TLS_V1_2_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_MGF1:
+ case ICP_QAT_FW_LA_CMD_AUTH_PRE_COMP:
+ case ICP_QAT_FW_LA_CMD_CIPHER_PRE_COMP:
+ case ICP_QAT_FW_LA_CMD_DELIMITER:
+ PMD_DRV_LOG(ERR, "Unsupported Service %u",
+ session->qat_cmd);
+ return -ENOTSUP;
+ default:
+ PMD_DRV_LOG(ERR, "Unsupported Service %u",
+ session->qat_cmd);
+ return -ENOTSUP;
+ }
+
+ return 0;
+}
+
+int
+qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
+ struct qat_pmd_private *internals = dev->data->dev_private;
+ uint8_t *key_data = auth_xform->key.data;
+ uint8_t key_length = auth_xform->key.length;
+
+ switch (auth_xform->algo) {
+ case RTE_CRYPTO_AUTH_SHA1_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA1;
+ break;
+ case RTE_CRYPTO_AUTH_SHA224_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA224;
+ break;
+ case RTE_CRYPTO_AUTH_SHA256_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA256;
+ break;
+ case RTE_CRYPTO_AUTH_SHA384_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA384;
+ break;
+ case RTE_CRYPTO_AUTH_SHA512_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA512;
+ break;
+ case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
+ break;
+ case RTE_CRYPTO_AUTH_AES_GMAC:
+ if (qat_alg_validate_aes_key(auth_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
+
+ break;
+ case RTE_CRYPTO_AUTH_SNOW3G_UIA2:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SNOW_3G_UIA2;
+ break;
+ case RTE_CRYPTO_AUTH_MD5_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_MD5;
+ break;
+ case RTE_CRYPTO_AUTH_NULL:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_NULL;
+ break;
+ case RTE_CRYPTO_AUTH_KASUMI_F9:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_KASUMI_F9;
+ break;
+ case RTE_CRYPTO_AUTH_ZUC_EIA3:
+ if (!qat_is_auth_alg_supported(auth_xform->algo, internals)) {
+ PMD_DRV_LOG(ERR, "%s not supported on this device",
+ rte_crypto_auth_algorithm_strings
+ [auth_xform->algo]);
+ return -ENOTSUP;
+ }
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_ZUC_3G_128_EIA3;
+ break;
+ case RTE_CRYPTO_AUTH_SHA1:
+ case RTE_CRYPTO_AUTH_SHA256:
+ case RTE_CRYPTO_AUTH_SHA512:
+ case RTE_CRYPTO_AUTH_SHA224:
+ case RTE_CRYPTO_AUTH_SHA384:
+ case RTE_CRYPTO_AUTH_MD5:
+ case RTE_CRYPTO_AUTH_AES_CMAC:
+ case RTE_CRYPTO_AUTH_AES_CBC_MAC:
+ PMD_DRV_LOG(ERR, "Crypto: Unsupported hash alg %u",
+ auth_xform->algo);
+ return -ENOTSUP;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined Hash algo %u specified",
+ auth_xform->algo);
+ return -EINVAL;
+ }
+
+ session->auth_iv.offset = auth_xform->iv.offset;
+ session->auth_iv.length = auth_xform->iv.length;
+
+ if (auth_xform->algo == RTE_CRYPTO_AUTH_AES_GMAC) {
+ if (auth_xform->op == RTE_CRYPTO_AUTH_OP_GENERATE) {
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ /*
+ * It needs to create cipher desc content first,
+ * then authentication
+ */
+ if (qat_alg_aead_session_create_content_desc_cipher(
+ session,
+ auth_xform->key.data,
+ auth_xform->key.length))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_auth(
+ session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+ } else {
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+ /*
+ * It needs to create authentication desc content first,
+ * then cipher
+ */
+ if (qat_alg_aead_session_create_content_desc_auth(
+ session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(
+ session,
+ auth_xform->key.data,
+ auth_xform->key.length))
+ return -EINVAL;
+ }
+ /* Restore to authentication only only */
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
+ } else {
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+ }
+
+ session->digest_length = auth_xform->digest_length;
+ return 0;
+}
+
+int
+qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct rte_crypto_aead_xform *aead_xform = &xform->aead;
+ enum rte_crypto_auth_operation crypto_operation;
+
+ /*
+ * Store AEAD IV parameters as cipher IV,
+ * to avoid unnecessary memory usage
+ */
+ session->cipher_iv.offset = xform->aead.iv.offset;
+ session->cipher_iv.length = xform->aead.iv.length;
+
+ switch (aead_xform->algo) {
+ case RTE_CRYPTO_AEAD_AES_GCM:
+ if (qat_alg_validate_aes_key(aead_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
+ break;
+ case RTE_CRYPTO_AEAD_AES_CCM:
+ if (qat_alg_validate_aes_key(aead_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC;
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined AEAD specified %u\n",
+ aead_xform->algo);
+ return -EINVAL;
+ }
+
+ if ((aead_xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT &&
+ aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM) ||
+ (aead_xform->op == RTE_CRYPTO_AEAD_OP_DECRYPT &&
+ aead_xform->algo == RTE_CRYPTO_AEAD_AES_CCM)) {
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ /*
+ * It needs to create cipher desc content first,
+ * then authentication
+ */
+ crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
+ RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ aead_xform->key.data,
+ aead_xform->key.length))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ aead_xform->key.data,
+ aead_xform->key.length,
+ aead_xform->aad_length,
+ aead_xform->digest_length,
+ crypto_operation))
+ return -EINVAL;
+ } else {
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+ /*
+ * It needs to create authentication desc content first,
+ * then cipher
+ */
+
+ crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
+ RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
+
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ aead_xform->key.data,
+ aead_xform->key.length,
+ aead_xform->aad_length,
+ aead_xform->digest_length,
+ crypto_operation))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ aead_xform->key.data,
+ aead_xform->key.length))
+ return -EINVAL;
+ }
+
+ session->digest_length = aead_xform->digest_length;
+ return 0;
+}
+
+unsigned int qat_crypto_sym_get_session_private_size(
+ struct rte_cryptodev *dev __rte_unused)
+{
+ return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
+}
/* returns block size in bytes per cipher algo */
int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg)
@@ -77,12 +786,12 @@ static int qat_hash_get_state1_size(enum icp_qat_hw_auth_algo qat_hash_alg)
case ICP_QAT_HW_AUTH_ALGO_KASUMI_F9:
return QAT_HW_ROUND_UP(ICP_QAT_HW_KASUMI_F9_STATE1_SZ,
QAT_HW_DEFAULT_ALIGNMENT);
- case ICP_QAT_HW_AUTH_ALGO_NULL:
- return QAT_HW_ROUND_UP(ICP_QAT_HW_NULL_STATE1_SZ,
- QAT_HW_DEFAULT_ALIGNMENT);
case ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC:
return QAT_HW_ROUND_UP(ICP_QAT_HW_AES_CBC_MAC_STATE1_SZ,
QAT_HW_DEFAULT_ALIGNMENT);
+ case ICP_QAT_HW_AUTH_ALGO_NULL:
+ return QAT_HW_ROUND_UP(ICP_QAT_HW_NULL_STATE1_SZ,
+ QAT_HW_DEFAULT_ALIGNMENT);
case ICP_QAT_HW_AUTH_ALGO_DELIMITER:
/* return maximum state1 size in this case */
return QAT_HW_ROUND_UP(ICP_QAT_HW_SHA512_STATE1_SZ,
@@ -854,14 +1563,13 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
if (aad_length > 0) {
aad_length += ICP_QAT_HW_CCM_AAD_B0_LEN +
- ICP_QAT_HW_CCM_AAD_LEN_INFO;
+ ICP_QAT_HW_CCM_AAD_LEN_INFO;
auth_param->u2.aad_sz =
- RTE_ALIGN_CEIL(aad_length,
- ICP_QAT_HW_CCM_AAD_ALIGNMENT);
+ RTE_ALIGN_CEIL(aad_length,
+ ICP_QAT_HW_CCM_AAD_ALIGNMENT);
} else {
auth_param->u2.aad_sz = ICP_QAT_HW_CCM_AAD_B0_LEN;
}
-
cdesc->aad_len = aad_length;
hash->auth_counter.counter = 0;
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
new file mode 100644
index 000000000..f90b1821d
--- /dev/null
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -0,0 +1,142 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_SYM_SESSION_H_
+#define _QAT_SYM_SESSION_H_
+
+#include <rte_crypto.h>
+#include <rte_cryptodev_pmd.h>
+
+#include "qat_common.h"
+#include "icp_qat_hw.h"
+#include "icp_qat_fw.h"
+#include "icp_qat_fw_la.h"
+
+/*
+ * Key Modifier (KM) value used in KASUMI algorithm in F9 mode to XOR
+ * Integrity Key (IK)
+ */
+#define KASUMI_F9_KEY_MODIFIER_4_BYTES 0xAAAAAAAA
+
+#define KASUMI_F8_KEY_MODIFIER_4_BYTES 0x55555555
+
+/* 3DES key sizes */
+#define QAT_3DES_KEY_SZ_OPT1 24 /* Keys are independent */
+#define QAT_3DES_KEY_SZ_OPT2 16 /* K3=K1 */
+
+#define QAT_AES_HW_CONFIG_CBC_ENC(alg) \
+ ICP_QAT_HW_CIPHER_CONFIG_BUILD(ICP_QAT_HW_CIPHER_CBC_MODE, alg, \
+ ICP_QAT_HW_CIPHER_NO_CONVERT, \
+ ICP_QAT_HW_CIPHER_ENCRYPT)
+
+#define QAT_AES_HW_CONFIG_CBC_DEC(alg) \
+ ICP_QAT_HW_CIPHER_CONFIG_BUILD(ICP_QAT_HW_CIPHER_CBC_MODE, alg, \
+ ICP_QAT_HW_CIPHER_KEY_CONVERT, \
+ ICP_QAT_HW_CIPHER_DECRYPT)
+
+enum qat_crypto_proto_flag {
+ QAT_CRYPTO_PROTO_FLAG_NONE = 0,
+ QAT_CRYPTO_PROTO_FLAG_CCM = 1,
+ QAT_CRYPTO_PROTO_FLAG_GCM = 2,
+ QAT_CRYPTO_PROTO_FLAG_SNOW3G = 3,
+ QAT_CRYPTO_PROTO_FLAG_ZUC = 4
+};
+
+/* Common content descriptor */
+struct qat_alg_cd {
+ struct icp_qat_hw_cipher_algo_blk cipher;
+ struct icp_qat_hw_auth_algo_blk hash;
+} __rte_packed __rte_cache_aligned;
+
+struct qat_session {
+ enum icp_qat_fw_la_cmd_id qat_cmd;
+ enum icp_qat_hw_cipher_algo qat_cipher_alg;
+ enum icp_qat_hw_cipher_dir qat_dir;
+ enum icp_qat_hw_cipher_mode qat_mode;
+ enum icp_qat_hw_auth_algo qat_hash_alg;
+ enum icp_qat_hw_auth_op auth_op;
+ void *bpi_ctx;
+ struct qat_alg_cd cd;
+ uint8_t *cd_cur_ptr;
+ phys_addr_t cd_paddr;
+ struct icp_qat_fw_la_bulk_req fw_req;
+ uint8_t aad_len;
+ struct qat_crypto_instance *inst;
+ struct {
+ uint16_t offset;
+ uint16_t length;
+ } cipher_iv;
+ struct {
+ uint16_t offset;
+ uint16_t length;
+ } auth_iv;
+ uint16_t digest_length;
+ rte_spinlock_t lock; /* protects this struct */
+ enum qat_device_gen min_qat_dev_gen;
+};
+
+int
+qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct rte_cryptodev_sym_session *sess,
+ struct rte_mempool *mempool);
+
+int
+qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform, void *session_private);
+
+int
+qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
+
+int
+qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
+
+int
+qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
+
+int
+qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cd,
+ uint8_t *enckey,
+ uint32_t enckeylen);
+
+int
+qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+ uint8_t *authkey,
+ uint32_t authkeylen,
+ uint32_t aad_length,
+ uint32_t digestsize,
+ unsigned int operation);
+
+int
+qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
+ unsigned int nb_objs, unsigned int obj_cache_size, int socket_id);
+
+void
+qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *session);
+
+unsigned int
+qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
+
+int qat_get_inter_state_size(enum icp_qat_hw_auth_algo qat_hash_alg);
+
+
+void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
+ enum qat_crypto_proto_flag proto_flags);
+
+int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_aes_docsisbpi_key(int key_len,
+ enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
+int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+
+#endif /* _QAT_SYM_SESSION_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index c8da07af6..e425eb43f 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -10,6 +10,7 @@
#include <rte_cryptodev_pmd.h>
#include "qat_crypto.h"
+#include "qat_sym_session.h"
#include "qat_logs.h"
uint8_t cryptodev_qat_driver_id;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 06/30] crypto/qat: change filename crypto to sym
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (4 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 05/30] crypto/qat: add symmetric session file Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 07/30] crypto/qat: rename fns for consistency Fiona Trahe
` (57 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
This commit renames qat_crypto.c/h to qat_sym.c/h
And makes a few whitespace changes to resolve line-length
issues.
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 2 +-
drivers/crypto/qat/meson.build | 2 +-
drivers/crypto/qat/qat_qp.c | 2 +-
drivers/crypto/qat/{qat_crypto.c => qat_sym.c} | 60 +++++++++++++-------------
drivers/crypto/qat/{qat_crypto.h => qat_sym.h} | 0
drivers/crypto/qat/rte_qat_cryptodev.c | 2 +-
6 files changed, 33 insertions(+), 35 deletions(-)
rename drivers/crypto/qat/{qat_crypto.c => qat_sym.c} (95%)
rename drivers/crypto/qat/{qat_crypto.h => qat_sym.h} (100%)
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index c63c1515e..8cb802b9d 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -21,7 +21,7 @@ LDLIBS += -lrte_cryptodev
LDLIBS += -lrte_pci -lrte_bus_pci
# library source files
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index be4282a83..e596006da 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -5,7 +5,7 @@ dep = dependency('libcrypto', required: false)
if not dep.found()
build = false
endif
-sources = files('qat_crypto.c', 'qat_qp.c',
+sources = files('qat_sym.c', 'qat_qp.c',
'qat_sym_session.c',
'rte_qat_cryptodev.c',
'qat_device.c')
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index e42acd877..927c78111 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,7 +13,7 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_sym.c
similarity index 95%
rename from drivers/crypto/qat/qat_crypto.c
rename to drivers/crypto/qat/qat_sym.c
index 96a1b78f0..f5d542ae3 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -6,15 +6,14 @@
#include <rte_mbuf.h>
#include <rte_hexdump.h>
#include <rte_crypto_sym.h>
-#include <rte_byteorder.h>
-#include <rte_pci.h>
#include <rte_bus_pci.h>
+#include <rte_byteorder.h>
#include <openssl/evp.h>
#include "qat_logs.h"
#include "qat_sym_session.h"
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
#define BYTE_LENGTH 8
@@ -500,8 +499,6 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
-
-
qat_req = (struct icp_qat_fw_la_bulk_req *)out_msg;
rte_mov128((uint8_t *)qat_req, (const uint8_t *)&(ctx->fw_req));
qat_req->comn_mid.opaque_data = (uint64_t)(uintptr_t)op;
@@ -512,11 +509,11 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
ctx->qat_cmd == ICP_QAT_FW_LA_CMD_CIPHER_HASH) {
/* AES-GCM or AES-CCM */
if (ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_128 ||
- ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_64 ||
- (ctx->qat_cipher_alg == ICP_QAT_HW_CIPHER_ALGO_AES128
- && ctx->qat_mode == ICP_QAT_HW_CIPHER_CTR_MODE
- && ctx->qat_hash_alg ==
- ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC)) {
+ ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_64 ||
+ (ctx->qat_cipher_alg == ICP_QAT_HW_CIPHER_ALGO_AES128
+ && ctx->qat_mode == ICP_QAT_HW_CIPHER_CTR_MODE
+ && ctx->qat_hash_alg ==
+ ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC)) {
do_aead = 1;
} else {
do_auth = 1;
@@ -642,7 +639,6 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
qat_req->comn_hdr.serv_specif_flags,
ICP_QAT_FW_LA_GCM_IV_LEN_12_OCTETS);
}
-
set_cipher_iv(ctx->cipher_iv.length,
ctx->cipher_iv.offset,
cipher_param, op, qat_req);
@@ -650,15 +646,14 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
} else if (ctx->qat_hash_alg ==
ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC) {
- /* In case of AES-CCM this may point to user selected memory
- * or iv offset in cypto_op
+ /* In case of AES-CCM this may point to user selected
+ * memory or iv offset in cypto_op
*/
uint8_t *aad_data = op->sym->aead.aad.data;
/* This is true AAD length, it not includes 18 bytes of
* preceding data
*/
uint8_t aad_ccm_real_len = 0;
-
uint8_t aad_len_field_sz = 0;
uint32_t msg_len_be =
rte_bswap32(op->sym->aead.data.length);
@@ -670,33 +665,36 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
ICP_QAT_HW_CCM_AAD_LEN_INFO;
} else {
/*
- * aad_len not greater than 18, so no actual aad data,
- * then use IV after op for B0 block
+ * aad_len not greater than 18, so no actual aad
+ * data, then use IV after op for B0 block
*/
- aad_data = rte_crypto_op_ctod_offset(op, uint8_t *,
+ aad_data = rte_crypto_op_ctod_offset(op,
+ uint8_t *,
ctx->cipher_iv.offset);
aad_phys_addr_aead =
rte_crypto_op_ctophys_offset(op,
- ctx->cipher_iv.offset);
+ ctx->cipher_iv.offset);
}
- uint8_t q = ICP_QAT_HW_CCM_NQ_CONST - ctx->cipher_iv.length;
+ uint8_t q = ICP_QAT_HW_CCM_NQ_CONST -
+ ctx->cipher_iv.length;
- aad_data[0] = ICP_QAT_HW_CCM_BUILD_B0_FLAGS(aad_len_field_sz,
+ aad_data[0] = ICP_QAT_HW_CCM_BUILD_B0_FLAGS(
+ aad_len_field_sz,
ctx->digest_length, q);
if (q > ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE) {
memcpy(aad_data + ctx->cipher_iv.length +
- ICP_QAT_HW_CCM_NONCE_OFFSET
- + (q - ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE),
- (uint8_t *)&msg_len_be,
- ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE);
+ ICP_QAT_HW_CCM_NONCE_OFFSET +
+ (q - ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE),
+ (uint8_t *)&msg_len_be,
+ ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE);
} else {
memcpy(aad_data + ctx->cipher_iv.length +
- ICP_QAT_HW_CCM_NONCE_OFFSET,
- (uint8_t *)&msg_len_be
- + (ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE
- - q), q);
+ ICP_QAT_HW_CCM_NONCE_OFFSET,
+ (uint8_t *)&msg_len_be
+ + (ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE
+ - q), q);
}
if (aad_len_field_sz > 0) {
@@ -709,10 +707,10 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
uint8_t pad_idx = 0;
pad_len = ICP_QAT_HW_CCM_AAD_B0_LEN -
- ((aad_ccm_real_len + aad_len_field_sz) %
- ICP_QAT_HW_CCM_AAD_B0_LEN);
+ ((aad_ccm_real_len + aad_len_field_sz) %
+ ICP_QAT_HW_CCM_AAD_B0_LEN);
pad_idx = ICP_QAT_HW_CCM_AAD_B0_LEN +
- aad_ccm_real_len + aad_len_field_sz;
+ aad_ccm_real_len + aad_len_field_sz;
memset(&aad_data[pad_idx],
0, pad_len);
}
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_sym.h
similarity index 100%
rename from drivers/crypto/qat/qat_crypto.h
rename to drivers/crypto/qat/qat_sym.h
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index e425eb43f..45f8a253b 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -9,7 +9,7 @@
#include <rte_pci.h>
#include <rte_cryptodev_pmd.h>
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "qat_sym_session.h"
#include "qat_logs.h"
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 07/30] crypto/qat: rename fns for consistency
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (5 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 06/30] crypto/qat: change filename crypto to sym Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 08/30] crypto/qat: renamed sym-specific structs Fiona Trahe
` (56 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Rename fn names to shorten them, i.e.
qat_crypto_sym_xxx to qat_sym_xxx
_content_desc_ to _cd_
Renaming symmetric crypto specific with consistent names:
qat_crypto_set_session_parameters->qat_sym_set_session_parameters
qat_write_hw_desc_entry()->qat_sym_build_request()
qat_alg_xxx ->qat_sym_xxx
qat_sym_xxx_session_yyy()->qat_sym_session_xxx_yyy()
Removed unused prototypes:
qat_get_inter_state_size()
qat_pmd_session_mempool_create()
Removed 2 unnecessary extern declarations
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 2 +-
drivers/crypto/qat/qat_device.h | 2 +-
drivers/crypto/qat/qat_qp.c | 6 +-
drivers/crypto/qat/qat_sym.c | 14 ++--
drivers/crypto/qat/qat_sym.h | 22 +++---
drivers/crypto/qat/qat_sym_session.c | 131 ++++++++++++++++-----------------
drivers/crypto/qat/qat_sym_session.h | 53 ++++++-------
drivers/crypto/qat/rte_qat_cryptodev.c | 18 ++---
8 files changed, 123 insertions(+), 125 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index c2bf9b7a7..ac6bd1af6 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -30,7 +30,7 @@ int qat_dev_close(struct rte_cryptodev *dev)
PMD_INIT_FUNC_TRACE();
for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_crypto_sym_qp_release(dev, i);
+ ret = qat_sym_qp_release(dev, i);
if (ret < 0)
return ret;
}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 5c48fdb93..2cb8e7612 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,7 +11,7 @@
extern uint8_t cryptodev_qat_driver_id;
-extern int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
/** private data structure for each QAT device */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 927c78111..7e4dc93c9 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -98,7 +98,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
memzone_flags, queue_size);
}
-int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
const struct rte_cryptodev_qp_conf *qp_conf,
int socket_id, struct rte_mempool *session_pool __rte_unused)
{
@@ -112,7 +112,7 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
/* If qp is already in use free ring memory and qp metadata. */
if (dev->data->queue_pairs[queue_pair_id] != NULL) {
- ret = qat_crypto_sym_qp_release(dev, queue_pair_id);
+ ret = qat_sym_qp_release(dev, queue_pair_id);
if (ret < 0)
return ret;
}
@@ -228,7 +228,7 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-int qat_crypto_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index f5d542ae3..ae521c2b1 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -87,7 +87,7 @@ static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
static inline int
-qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
+qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
static inline uint32_t
@@ -210,7 +210,7 @@ txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
}
uint16_t
-qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
register struct qat_queue *queue;
@@ -242,7 +242,7 @@ qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
while (nb_ops_sent != nb_ops_possible) {
- ret = qat_write_hw_desc_entry(*cur_op, base_addr + tail,
+ ret = qat_sym_build_request(*cur_op, base_addr + tail,
tmp_qp->op_cookies[tail / queue->msg_size], tmp_qp);
if (ret != 0) {
tmp_qp->stats.enqueue_err_count++;
@@ -299,7 +299,7 @@ void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
}
uint16_t
-qat_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
struct qat_queue *rx_queue, *tx_queue;
@@ -456,7 +456,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
}
static inline int
-qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
+qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp)
{
int ret = 0;
@@ -883,7 +883,7 @@ static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
return data - mult;
}
-void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
+void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
{
int i;
@@ -907,7 +907,7 @@ void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
}
}
-void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev)
+void qat_sym_stats_reset(struct rte_cryptodev *dev)
{
int i;
struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index e3873171c..811ba8619 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -2,8 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
-#ifndef _QAT_CRYPTO_H_
-#define _QAT_CRYPTO_H_
+#ifndef _QAT_SYM_H_
+#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
#include <rte_memzone.h>
@@ -65,23 +65,23 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
} __rte_cache_aligned;
-void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
+void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
-void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev);
+void qat_sym_stats_reset(struct rte_cryptodev *dev);
-int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
const struct rte_cryptodev_qp_conf *rx_conf, int socket_id,
struct rte_mempool *session_pool);
-int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-extern uint16_t
-qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops);
-extern uint16_t
-qat_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops);
-#endif /* _QAT_CRYPTO_H_ */
+#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index a67e22919..4977b0a11 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -103,7 +103,7 @@ qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
}
void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+qat_sym_session_clear(struct rte_cryptodev *dev,
struct rte_cryptodev_sym_session *sess)
{
PMD_INIT_FUNC_TRACE();
@@ -114,7 +114,7 @@ qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
if (sess_priv) {
if (s->bpi_ctx)
bpi_cipher_ctx_free(s->bpi_ctx);
- memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
+ memset(s, 0, qat_sym_session_get_private_size(dev));
struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
set_session_private_data(sess, index, NULL);
@@ -195,7 +195,7 @@ qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
}
int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
@@ -211,7 +211,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
switch (cipher_xform->algo) {
case RTE_CRYPTO_CIPHER_AES_CBC:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
ret = -EINVAL;
@@ -220,7 +220,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_AES_CTR:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
ret = -EINVAL;
@@ -229,7 +229,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
break;
case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
- if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
+ if (qat_sym_validate_snow3g_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
ret = -EINVAL;
@@ -241,7 +241,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
break;
case RTE_CRYPTO_CIPHER_KASUMI_F8:
- if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
+ if (qat_sym_validate_kasumi_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
ret = -EINVAL;
@@ -250,7 +250,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
break;
case RTE_CRYPTO_CIPHER_3DES_CBC:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ if (qat_sym_validate_3des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
ret = -EINVAL;
@@ -259,7 +259,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_DES_CBC:
- if (qat_alg_validate_des_key(cipher_xform->key.length,
+ if (qat_sym_validate_des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
ret = -EINVAL;
@@ -268,7 +268,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_3DES_CTR:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ if (qat_sym_validate_3des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
ret = -EINVAL;
@@ -286,7 +286,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
goto error_out;
}
- if (qat_alg_validate_des_key(cipher_xform->key.length,
+ if (qat_sym_validate_des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
ret = -EINVAL;
@@ -304,7 +304,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
goto error_out;
}
- if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_docsisbpi_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
ret = -EINVAL;
@@ -321,7 +321,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
ret = -ENOTSUP;
goto error_out;
}
- if (qat_alg_validate_zuc_key(cipher_xform->key.length,
+ if (qat_sym_validate_zuc_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
ret = -EINVAL;
@@ -350,7 +350,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
else
session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
cipher_xform->key.data,
cipher_xform->key.length)) {
ret = -EINVAL;
@@ -368,7 +368,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+qat_sym_session_configure(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct rte_cryptodev_sym_session *sess,
struct rte_mempool *mempool)
@@ -382,7 +382,7 @@ qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
return -ENOMEM;
}
- ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
+ ret = qat_sym_session_set_parameters(dev, xform, sess_private_data);
if (ret != 0) {
PMD_DRV_LOG(ERR,
"Crypto QAT PMD: failed to configure session parameters");
@@ -399,7 +399,7 @@ qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
}
int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private)
{
struct qat_session *session = session_private;
@@ -423,29 +423,27 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
switch (session->qat_cmd) {
case ICP_QAT_FW_LA_CMD_CIPHER:
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
+ ret = qat_sym_session_configure_cipher(dev, xform, session);
if (ret < 0)
return ret;
break;
case ICP_QAT_FW_LA_CMD_AUTH:
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
+ ret = qat_sym_session_configure_auth(dev, xform, session);
if (ret < 0)
return ret;
break;
case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
+ ret = qat_sym_session_configure_aead(xform,
session);
if (ret < 0)
return ret;
} else {
- ret = qat_crypto_sym_configure_session_cipher(dev,
+ ret = qat_sym_session_configure_cipher(dev,
xform, session);
if (ret < 0)
return ret;
- ret = qat_crypto_sym_configure_session_auth(dev,
+ ret = qat_sym_session_configure_auth(dev,
xform, session);
if (ret < 0)
return ret;
@@ -453,16 +451,16 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
break;
case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
+ ret = qat_sym_session_configure_aead(xform,
session);
if (ret < 0)
return ret;
} else {
- ret = qat_crypto_sym_configure_session_auth(dev,
+ ret = qat_sym_session_configure_auth(dev,
xform, session);
if (ret < 0)
return ret;
- ret = qat_crypto_sym_configure_session_cipher(dev,
+ ret = qat_sym_session_configure_cipher(dev,
xform, session);
if (ret < 0)
return ret;
@@ -490,7 +488,7 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
@@ -519,7 +517,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
break;
case RTE_CRYPTO_AUTH_AES_GMAC:
- if (qat_alg_validate_aes_key(auth_xform->key.length,
+ if (qat_sym_validate_aes_key(auth_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -577,14 +575,13 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
* It needs to create cipher desc content first,
* then authentication
*/
- if (qat_alg_aead_session_create_content_desc_cipher(
- session,
+
+ if (qat_sym_session_aead_create_cd_cipher(session,
auth_xform->key.data,
auth_xform->key.length))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_auth(
- session,
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -598,8 +595,8 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
* It needs to create authentication desc content first,
* then cipher
*/
- if (qat_alg_aead_session_create_content_desc_auth(
- session,
+
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -607,8 +604,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
auth_xform->op))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_cipher(
- session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
auth_xform->key.data,
auth_xform->key.length))
return -EINVAL;
@@ -616,7 +612,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
/* Restore to authentication only only */
session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
} else {
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -630,7 +626,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
struct rte_crypto_aead_xform *aead_xform = &xform->aead;
@@ -645,7 +641,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
switch (aead_xform->algo) {
case RTE_CRYPTO_AEAD_AES_GCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
+ if (qat_sym_validate_aes_key(aead_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -654,7 +650,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
break;
case RTE_CRYPTO_AEAD_AES_CCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
+ if (qat_sym_validate_aes_key(aead_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -680,12 +676,12 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
aead_xform->key.data,
aead_xform->key.length))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
aead_xform->key.data,
aead_xform->key.length,
aead_xform->aad_length,
@@ -702,7 +698,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
aead_xform->key.data,
aead_xform->key.length,
aead_xform->aad_length,
@@ -710,7 +706,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
aead_xform->key.data,
aead_xform->key.length))
return -EINVAL;
@@ -720,7 +716,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
return 0;
}
-unsigned int qat_crypto_sym_get_session_private_size(
+unsigned int qat_sym_session_get_private_size(
struct rte_cryptodev *dev __rte_unused)
{
return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
@@ -994,7 +990,7 @@ static int partial_hash_compute(enum icp_qat_hw_auth_algo hash_alg,
#define HMAC_OPAD_VALUE 0x5c
#define HASH_XCBC_PRECOMP_KEY_NUM 3
-static int qat_alg_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
+static int qat_sym_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
const uint8_t *auth_key,
uint16_t auth_keylen,
uint8_t *p_state_buf,
@@ -1124,7 +1120,8 @@ static int qat_alg_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
return 0;
}
-void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
+static void
+qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
enum qat_crypto_proto_flag proto_flags)
{
PMD_INIT_FUNC_TRACE();
@@ -1192,7 +1189,7 @@ qat_get_crypto_proto_flag(uint16_t flags)
return qat_proto_flag;
}
-int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
uint8_t *cipherkey,
uint32_t cipherkeylen)
{
@@ -1296,7 +1293,7 @@ int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
cipher_cd_ctrl->cipher_cfg_offset = cipher_offset >> 3;
header->service_cmd_id = cdesc->qat_cmd;
- qat_alg_init_common_hdr(header, qat_proto_flag);
+ qat_sym_session_init_common_hdr(header, qat_proto_flag);
cipher = (struct icp_qat_hw_cipher_algo_blk *)cdesc->cd_cur_ptr;
cipher->cipher_config.val =
@@ -1340,7 +1337,7 @@ int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
return 0;
}
-int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -1429,7 +1426,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
*/
switch (cdesc->qat_hash_alg) {
case ICP_QAT_HW_AUTH_ALGO_SHA1:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA1,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA1,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1437,7 +1434,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = RTE_ALIGN_CEIL(ICP_QAT_HW_SHA1_STATE2_SZ, 8);
break;
case ICP_QAT_HW_AUTH_ALGO_SHA224:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA224,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA224,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1445,7 +1442,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA224_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA256:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA256,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA256,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1453,7 +1450,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA256_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA384:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA384,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA384,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1461,7 +1458,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA384_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA512:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA512,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA512,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1470,7 +1467,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
break;
case ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC:
state1_size = ICP_QAT_HW_AES_XCBC_MAC_STATE1_SZ;
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC,
authkey, authkeylen, cdesc->cd_cur_ptr + state1_size,
&state2_size)) {
PMD_DRV_LOG(ERR, "(XCBC)precompute failed");
@@ -1481,7 +1478,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
case ICP_QAT_HW_AUTH_ALGO_GALOIS_64:
qat_proto_flag = QAT_CRYPTO_PROTO_FLAG_GCM;
state1_size = ICP_QAT_HW_GALOIS_128_STATE1_SZ;
- if (qat_alg_do_precomputes(cdesc->qat_hash_alg,
+ if (qat_sym_do_precomputes(cdesc->qat_hash_alg,
authkey, authkeylen, cdesc->cd_cur_ptr + state1_size,
&state2_size)) {
PMD_DRV_LOG(ERR, "(GCM)precompute failed");
@@ -1541,7 +1538,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
break;
case ICP_QAT_HW_AUTH_ALGO_MD5:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_MD5,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_MD5,
authkey, authkeylen, cdesc->cd_cur_ptr,
&state1_size)) {
PMD_DRV_LOG(ERR, "(MD5)precompute failed");
@@ -1604,7 +1601,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
}
/* Request template setup */
- qat_alg_init_common_hdr(header, qat_proto_flag);
+ qat_sym_session_init_common_hdr(header, qat_proto_flag);
header->service_cmd_id = cdesc->qat_cmd;
/* Auth CD config setup */
@@ -1630,7 +1627,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
return 0;
}
-int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_AES_128_KEY_SZ:
@@ -1648,7 +1645,7 @@ int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_aes_docsisbpi_key(int key_len,
+int qat_sym_validate_aes_docsisbpi_key(int key_len,
enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
@@ -1661,7 +1658,7 @@ int qat_alg_validate_aes_docsisbpi_key(int key_len,
return 0;
}
-int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_SNOW_3G_UEA2_KEY_SZ:
@@ -1673,7 +1670,7 @@ int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_KASUMI_KEY_SZ:
@@ -1685,7 +1682,7 @@ int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_DES_KEY_SZ:
@@ -1697,7 +1694,7 @@ int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case QAT_3DES_KEY_SZ_OPT1:
@@ -1710,7 +1707,7 @@ int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_ZUC_3G_EEA3_KEY_SZ:
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index f90b1821d..493036ebf 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -76,67 +76,68 @@ struct qat_session {
};
int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+qat_sym_session_configure(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct rte_cryptodev_sym_session *sess,
struct rte_mempool *mempool);
int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private);
int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cd,
+qat_sym_session_aead_create_cd_cipher(struct qat_session *cd,
uint8_t *enckey,
uint32_t enckeylen);
int
-qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
uint32_t digestsize,
unsigned int operation);
-int
-qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
- unsigned int nb_objs, unsigned int obj_cache_size, int socket_id);
-
void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+qat_sym_session_clear(struct rte_cryptodev *dev,
struct rte_cryptodev_sym_session *session);
unsigned int
-qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
-
-int qat_get_inter_state_size(enum icp_qat_hw_auth_algo qat_hash_alg);
-
+qat_sym_session_get_private_size(struct rte_cryptodev *dev);
-void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
+void
+qat_sym_sesssion_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
enum qat_crypto_proto_flag proto_flags);
-
-int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_aes_docsisbpi_key(int key_len,
+int
+qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_aes_docsisbpi_key(int key_len,
enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
-int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
+int
+qat_sym_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
#endif /* _QAT_SYM_SESSION_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 45f8a253b..42011dcd9 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -35,18 +35,18 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
.dev_close = qat_dev_close,
.dev_infos_get = qat_dev_info_get,
- .stats_get = qat_crypto_sym_stats_get,
- .stats_reset = qat_crypto_sym_stats_reset,
- .queue_pair_setup = qat_crypto_sym_qp_setup,
- .queue_pair_release = qat_crypto_sym_qp_release,
+ .stats_get = qat_sym_stats_get,
+ .stats_reset = qat_sym_stats_reset,
+ .queue_pair_setup = qat_sym_qp_setup,
+ .queue_pair_release = qat_sym_qp_release,
.queue_pair_start = NULL,
.queue_pair_stop = NULL,
.queue_pair_count = NULL,
/* Crypto related operations */
- .session_get_size = qat_crypto_sym_get_session_private_size,
- .session_configure = qat_crypto_sym_configure_session,
- .session_clear = qat_crypto_sym_clear_session
+ .session_get_size = qat_sym_session_get_private_size,
+ .session_configure = qat_sym_session_configure,
+ .session_clear = qat_sym_session_clear
};
/*
@@ -86,8 +86,8 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
cryptodev->driver_id = cryptodev_qat_driver_id;
cryptodev->dev_ops = &crypto_qat_ops;
- cryptodev->enqueue_burst = qat_pmd_enqueue_op_burst;
- cryptodev->dequeue_burst = qat_pmd_dequeue_op_burst;
+ cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
+ cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
RTE_CRYPTODEV_FF_HW_ACCELERATED |
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 08/30] crypto/qat: renamed sym-specific structs
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (6 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 07/30] crypto/qat: rename fns for consistency Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 09/30] crypto/qat: make enqueue function generic Fiona Trahe
` (55 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
qat_session -> qat_sym_session
qat_crypto_proto_flag -> qat_sym_proto_flag
qat_alg_cd -> qat_sym_cd
qat_crypto_op_cookie -> qat_sym_op_cookie
qat_gen1_capabilities -> qat_gen1_sym_capabilities
qat_gen2_capabilities -> qat_gen2_sym_capabilities
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 4 ++--
drivers/crypto/qat/qat_qp.c | 8 ++++----
drivers/crypto/qat/qat_sym.c | 19 ++++++++++---------
drivers/crypto/qat/qat_sym.h | 2 +-
drivers/crypto/qat/qat_sym_session.c | 28 ++++++++++++++--------------
drivers/crypto/qat/qat_sym_session.h | 20 ++++++++++----------
drivers/crypto/qat/rte_qat_cryptodev.c | 8 ++++----
7 files changed, 45 insertions(+), 44 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 293b6f700..7802e96f9 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -37,11 +37,11 @@ struct qat_alg_buf_list {
struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
-struct qat_crypto_op_cookie {
+struct qat_sym_op_cookie {
struct qat_alg_buf_list qat_sgl_list_src;
struct qat_alg_buf_list qat_sgl_list_dst;
phys_addr_t qat_sgl_src_phys_addr;
phys_addr_t qat_sgl_dst_phys_addr;
};
-#endif /* _QAT_QAT_COMMON_H_ */
+#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 7e4dc93c9..990cde242 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -184,7 +184,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
qp->nb_descriptors,
- sizeof(struct qat_crypto_op_cookie), 64, 0,
+ sizeof(struct qat_sym_op_cookie), 64, 0,
NULL, NULL, NULL, NULL, socket_id,
0);
if (!qp->op_cookie_pool) {
@@ -199,17 +199,17 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
goto create_err;
}
- struct qat_crypto_op_cookie *sql_cookie =
+ struct qat_sym_op_cookie *sql_cookie =
qp->op_cookies[i];
sql_cookie->qat_sgl_src_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_crypto_op_cookie,
+ offsetof(struct qat_sym_op_cookie,
qat_sgl_list_src);
sql_cookie->qat_sgl_dst_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_crypto_op_cookie,
+ offsetof(struct qat_sym_op_cookie,
qat_sgl_list_dst);
}
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index ae521c2b1..2dfdc9cce 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -88,10 +88,10 @@ adf_modulo(uint32_t data, uint32_t shift);
static inline int
qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
+ struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp);
static inline uint32_t
-qat_bpicipher_preprocess(struct qat_session *ctx,
+qat_bpicipher_preprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
{
int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
@@ -146,7 +146,7 @@ qat_bpicipher_preprocess(struct qat_session *ctx,
}
static inline uint32_t
-qat_bpicipher_postprocess(struct qat_session *ctx,
+qat_bpicipher_postprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
{
int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
@@ -329,10 +329,11 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
resp_msg->comn_hdr.comn_status)) {
rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
} else {
- struct qat_session *sess = (struct qat_session *)
+ struct qat_sym_session *sess =
+ (struct qat_sym_session *)
get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
if (sess->bpi_ctx)
qat_bpicipher_postprocess(sess, rx_op);
@@ -457,10 +458,10 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
static inline int
qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp)
+ struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp)
{
int ret = 0;
- struct qat_session *ctx;
+ struct qat_sym_session *ctx;
struct icp_qat_fw_la_cipher_req_params *cipher_param;
struct icp_qat_fw_la_auth_req_params *auth_param;
register struct icp_qat_fw_la_bulk_req *qat_req;
@@ -485,7 +486,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
- ctx = (struct qat_session *)get_session_private_data(
+ ctx = (struct qat_sym_session *)get_session_private_data(
op->sym->session, cryptodev_qat_driver_id);
if (unlikely(ctx == NULL)) {
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 811ba8619..18c77ea11 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,7 +27,7 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
-struct qat_session;
+struct qat_sym_session;
/**
* Structure associated with each queue.
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index 4977b0a11..d71d7fcd1 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -109,7 +109,7 @@ qat_sym_session_clear(struct rte_cryptodev *dev,
PMD_INIT_FUNC_TRACE();
uint8_t index = dev->driver_id;
void *sess_priv = get_session_private_data(sess, index);
- struct qat_session *s = (struct qat_session *)sess_priv;
+ struct qat_sym_session *s = (struct qat_sym_session *)sess_priv;
if (sess_priv) {
if (s->bpi_ctx)
@@ -197,7 +197,7 @@ qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
int
qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct qat_pmd_private *internals = dev->data->dev_private;
struct rte_crypto_cipher_xform *cipher_xform = NULL;
@@ -402,7 +402,7 @@ int
qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private)
{
- struct qat_session *session = session_private;
+ struct qat_sym_session *session = session_private;
int ret;
int qat_cmd_id;
@@ -410,7 +410,7 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
/* Set context descriptor physical address */
session->cd_paddr = rte_mempool_virt2iova(session) +
- offsetof(struct qat_session, cd);
+ offsetof(struct qat_sym_session, cd);
session->min_qat_dev_gen = QAT_GEN1;
@@ -490,7 +490,7 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
int
qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
struct qat_pmd_private *internals = dev->data->dev_private;
@@ -627,7 +627,7 @@ qat_sym_session_configure_auth(struct rte_cryptodev *dev,
int
qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct rte_crypto_aead_xform *aead_xform = &xform->aead;
enum rte_crypto_auth_operation crypto_operation;
@@ -719,7 +719,7 @@ qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
unsigned int qat_sym_session_get_private_size(
struct rte_cryptodev *dev __rte_unused)
{
- return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
+ return RTE_ALIGN_CEIL(sizeof(struct qat_sym_session), 8);
}
/* returns block size in bytes per cipher algo */
@@ -1122,7 +1122,7 @@ static int qat_sym_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
static void
qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
- enum qat_crypto_proto_flag proto_flags)
+ enum qat_sym_proto_flag proto_flags)
{
PMD_INIT_FUNC_TRACE();
header->hdr_flags =
@@ -1170,11 +1170,11 @@ qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
* and set its protocol flag in both cipher and auth part of content
* descriptor building function
*/
-static enum qat_crypto_proto_flag
+static enum qat_sym_proto_flag
qat_get_crypto_proto_flag(uint16_t flags)
{
int proto = ICP_QAT_FW_LA_PROTO_GET(flags);
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
switch (proto) {
@@ -1189,7 +1189,7 @@ qat_get_crypto_proto_flag(uint16_t flags)
return qat_proto_flag;
}
-int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_cipher(struct qat_sym_session *cdesc,
uint8_t *cipherkey,
uint32_t cipherkeylen)
{
@@ -1201,7 +1201,7 @@ int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
struct icp_qat_fw_cipher_cd_ctrl_hdr *cipher_cd_ctrl = ptr;
struct icp_qat_fw_auth_cd_ctrl_hdr *hash_cd_ctrl = ptr;
enum icp_qat_hw_cipher_convert key_convert;
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
uint32_t total_key_size;
uint16_t cipher_offset, cd_size;
@@ -1337,7 +1337,7 @@ int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
return 0;
}
-int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_auth(struct qat_sym_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -1361,7 +1361,7 @@ int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint32_t *aad_len = NULL;
uint32_t wordIndex = 0;
uint32_t *pTempKey;
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
PMD_INIT_FUNC_TRACE();
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index 493036ebf..d3d27ff46 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -34,7 +34,7 @@
ICP_QAT_HW_CIPHER_KEY_CONVERT, \
ICP_QAT_HW_CIPHER_DECRYPT)
-enum qat_crypto_proto_flag {
+enum qat_sym_proto_flag {
QAT_CRYPTO_PROTO_FLAG_NONE = 0,
QAT_CRYPTO_PROTO_FLAG_CCM = 1,
QAT_CRYPTO_PROTO_FLAG_GCM = 2,
@@ -43,12 +43,12 @@ enum qat_crypto_proto_flag {
};
/* Common content descriptor */
-struct qat_alg_cd {
+struct qat_sym_cd {
struct icp_qat_hw_cipher_algo_blk cipher;
struct icp_qat_hw_auth_algo_blk hash;
} __rte_packed __rte_cache_aligned;
-struct qat_session {
+struct qat_sym_session {
enum icp_qat_fw_la_cmd_id qat_cmd;
enum icp_qat_hw_cipher_algo qat_cipher_alg;
enum icp_qat_hw_cipher_dir qat_dir;
@@ -56,7 +56,7 @@ struct qat_session {
enum icp_qat_hw_auth_algo qat_hash_alg;
enum icp_qat_hw_auth_op auth_op;
void *bpi_ctx;
- struct qat_alg_cd cd;
+ struct qat_sym_cd cd;
uint8_t *cd_cur_ptr;
phys_addr_t cd_paddr;
struct icp_qat_fw_la_bulk_req fw_req;
@@ -87,25 +87,25 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
int
qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
-qat_sym_session_aead_create_cd_cipher(struct qat_session *cd,
+qat_sym_session_aead_create_cd_cipher(struct qat_sym_session *cd,
uint8_t *enckey,
uint32_t enckeylen);
int
-qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
+qat_sym_session_aead_create_cd_auth(struct qat_sym_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -121,7 +121,7 @@ qat_sym_session_get_private_size(struct rte_cryptodev *dev);
void
qat_sym_sesssion_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
- enum qat_crypto_proto_flag proto_flags);
+ enum qat_sym_proto_flag proto_flags);
int
qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
int
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 42011dcd9..05cb5ef40 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -15,12 +15,12 @@
uint8_t cryptodev_qat_driver_id;
-static const struct rte_cryptodev_capabilities qat_gen1_capabilities[] = {
+static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
QAT_BASE_GEN1_SYM_CAPABILITIES,
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
};
-static const struct rte_cryptodev_capabilities qat_gen2_capabilities[] = {
+static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
QAT_BASE_GEN1_SYM_CAPABILITIES,
QAT_EXTRA_GEN2_SYM_CAPABILITIES,
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
@@ -99,13 +99,13 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
switch (pci_dev->id.device_id) {
case 0x0443:
internals->qat_dev_gen = QAT_GEN1;
- internals->qat_dev_capabilities = qat_gen1_capabilities;
+ internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
break;
case 0x37c9:
case 0x19e3:
case 0x6f55:
internals->qat_dev_gen = QAT_GEN2;
- internals->qat_dev_capabilities = qat_gen2_capabilities;
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
break;
default:
PMD_DRV_LOG(ERR,
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 09/30] crypto/qat: make enqueue function generic
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (7 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 08/30] crypto/qat: renamed sym-specific structs Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 10/30] crypto/qat: make dequeue " Fiona Trahe
` (54 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Queue-handling code in enqueue is made generic, so it can
be used by other services in future. This is done by
- Removing all sym-specific refs in input params - replace with void ptrs.
- Wrapping this generic enqueue with the sym-specific enqueue
called through the API.
- Setting a fn ptr for build_request in qp on qp creation
- Passing void * params to this, in the service-specific implementation
qat_sym_build_request cast back to sym structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 1 +
drivers/crypto/qat/qat_sym.c | 46 +++++++++++++++++++++++++-------------------
drivers/crypto/qat/qat_sym.h | 11 +++++++++++
3 files changed, 38 insertions(+), 20 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 990cde242..74db1069c 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -216,6 +216,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
struct qat_pmd_private *internals
= dev->data->dev_private;
qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->build_request = qat_sym_build_request;
dev->data->queue_pairs[queue_pair_id] = qp;
return 0;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 2dfdc9cce..4e404749a 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -86,10 +86,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
-static inline int
-qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp);
-
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
@@ -209,14 +205,12 @@ txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
q->csr_tail = q->tail;
}
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
+static uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
{
register struct qat_queue *queue;
struct qat_qp *tmp_qp = (struct qat_qp *)qp;
register uint32_t nb_ops_sent = 0;
- register struct rte_crypto_op **cur_op = ops;
register int ret;
uint16_t nb_ops_possible = nb_ops;
register uint8_t *base_addr;
@@ -242,8 +236,9 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
while (nb_ops_sent != nb_ops_possible) {
- ret = qat_sym_build_request(*cur_op, base_addr + tail,
- tmp_qp->op_cookies[tail / queue->msg_size], tmp_qp);
+ ret = tmp_qp->build_request(*ops, base_addr + tail,
+ tmp_qp->op_cookies[tail / queue->msg_size],
+ tmp_qp->qat_dev_gen);
if (ret != 0) {
tmp_qp->stats.enqueue_err_count++;
/*
@@ -257,8 +252,8 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ ops++;
nb_ops_sent++;
- cur_op++;
}
kick_tail:
queue->tail = tail;
@@ -299,6 +294,13 @@ void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
}
uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
+}
+
+uint16_t
qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
@@ -456,9 +458,10 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-static inline int
-qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp)
+
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen)
{
int ret = 0;
struct qat_sym_session *ctx;
@@ -471,6 +474,9 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
uint32_t min_ofs = 0;
uint64_t src_buf_start = 0, dst_buf_start = 0;
uint8_t do_sgl = 0;
+ struct rte_crypto_op *op = (struct rte_crypto_op *)in_op;
+ struct qat_sym_op_cookie *cookie =
+ (struct qat_sym_op_cookie *)op_cookie;
#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_TX
if (unlikely(op->type != RTE_CRYPTO_OP_TYPE_SYMMETRIC)) {
@@ -494,7 +500,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
- if (unlikely(ctx->min_qat_dev_gen > qp->qat_dev_gen)) {
+ if (unlikely(ctx->min_qat_dev_gen > qat_dev_gen)) {
PMD_DRV_LOG(ERR, "Session alg not supported on this device gen");
op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION;
return -EINVAL;
@@ -807,7 +813,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
ICP_QAT_FW_COMN_PTR_TYPE_SET(qat_req->comn_hdr.comn_req_flags,
QAT_COMN_PTR_TYPE_SGL);
ret = qat_sgl_fill_array(op->sym->m_src, src_buf_start,
- &qat_op_cookie->qat_sgl_list_src,
+ &cookie->qat_sgl_list_src,
qat_req->comn_mid.src_length);
if (ret) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot fill sgl array");
@@ -817,11 +823,11 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
if (likely(op->sym->m_dst == NULL))
qat_req->comn_mid.dest_data_addr =
qat_req->comn_mid.src_data_addr =
- qat_op_cookie->qat_sgl_src_phys_addr;
+ cookie->qat_sgl_src_phys_addr;
else {
ret = qat_sgl_fill_array(op->sym->m_dst,
dst_buf_start,
- &qat_op_cookie->qat_sgl_list_dst,
+ &cookie->qat_sgl_list_dst,
qat_req->comn_mid.dst_length);
if (ret) {
@@ -831,9 +837,9 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
}
qat_req->comn_mid.src_data_addr =
- qat_op_cookie->qat_sgl_src_phys_addr;
+ cookie->qat_sgl_src_phys_addr;
qat_req->comn_mid.dest_data_addr =
- qat_op_cookie->qat_sgl_dst_phys_addr;
+ cookie->qat_sgl_dst_phys_addr;
}
} else {
qat_req->comn_mid.src_data_addr = src_buf_start;
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 18c77ea11..b1ddb6e93 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,6 +27,11 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
+typedef int (*build_request_t)(void *op,
+ uint8_t *req, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Build a request from an op. */
+
struct qat_sym_session;
/**
@@ -63,8 +68,14 @@ struct qat_qp {
void **op_cookies;
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
+ build_request_t build_request;
} __rte_cache_aligned;
+
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen);
+
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 10/30] crypto/qat: make dequeue function generic
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (8 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 09/30] crypto/qat: make enqueue function generic Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 11/30] crypto/qat: move generic qp fn to qp file Fiona Trahe
` (53 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Queue-handling code in dequeue is made generic, so it can
be used by other services in future. This is done by
- Removing all sym-specific refs in input params - replace with void ptrs.
- Wrapping this generic dequeue with the sym-specific dequeue
called through the API.
- extracting the sym-specific response processing into a new fn.
- Setting a fn ptr for process_response in qp on qp creation
- Passing void * params to this, in the service-specific implementation
qat_sym_process_response cast back to sym structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 1 +
drivers/crypto/qat/qat_sym.c | 101 ++++++++++++++++++++++++++-----------------
drivers/crypto/qat/qat_sym.h | 10 +++++
3 files changed, 72 insertions(+), 40 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 74db1069c..a9b558563 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -217,6 +217,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
= dev->data->dev_private;
qp->qat_dev_gen = internals->qat_dev_gen;
qp->build_request = qat_sym_build_request;
+ qp->process_response = qat_sym_process_response;
dev->data->queue_pairs[queue_pair_id] = qp;
return 0;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 4e404749a..2bae913a1 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -300,70 +300,91 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
}
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen)
+{
+
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
+
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
+
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
+
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+ }
+ *op = (void *)rx_op;
+
+ return 0;
+}
+
+static uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
{
struct qat_queue *rx_queue, *tx_queue;
struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- uint32_t msg_counter = 0;
- struct rte_crypto_op *rx_op;
- struct icp_qat_fw_comn_resp *resp_msg;
uint32_t head;
+ uint32_t resp_counter = 0;
+ uint8_t *resp_msg;
rx_queue = &(tmp_qp->rx_q);
tx_queue = &(tmp_qp->tx_q);
head = rx_queue->head;
- resp_msg = (struct icp_qat_fw_comn_resp *)
- ((uint8_t *)rx_queue->base_addr + head);
+ resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
- msg_counter != nb_ops) {
- rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess =
- (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
+ resp_counter != nb_ops) {
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
+ tmp_qp->process_response(ops, resp_msg,
+ tmp_qp->op_cookies[head / rx_queue->msg_size],
+ tmp_qp->qat_dev_gen);
head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
- resp_msg = (struct icp_qat_fw_comn_resp *)
- ((uint8_t *)rx_queue->base_addr + head);
- *ops = rx_op;
+
+ resp_msg = (uint8_t *)rx_queue->base_addr + head;
ops++;
- msg_counter++;
+ resp_counter++;
}
- if (msg_counter > 0) {
+ if (resp_counter > 0) {
rx_queue->head = head;
- tmp_qp->stats.dequeued_count += msg_counter;
- rx_queue->nb_processed_responses += msg_counter;
- tmp_qp->inflights16 -= msg_counter;
+ tmp_qp->stats.dequeued_count += resp_counter;
+ rx_queue->nb_processed_responses += resp_counter;
+ tmp_qp->inflights16 -= resp_counter;
if (rx_queue->nb_processed_responses > QAT_CSR_HEAD_WRITE_THRESH)
rxq_free_desc(tmp_qp, rx_queue);
}
/* also check if tail needs to be advanced */
if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
- tx_queue->tail != tx_queue->csr_tail) {
+ tx_queue->tail != tx_queue->csr_tail) {
txq_write_tail(tmp_qp, tx_queue);
}
- return msg_counter;
+ return resp_counter;
+}
+
+uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
static inline int
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index b1ddb6e93..279d3a3ae 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -32,6 +32,11 @@ typedef int (*build_request_t)(void *op,
enum qat_device_gen qat_dev_gen);
/**< Build a request from an op. */
+typedef int (*process_response_t)(void **ops,
+ uint8_t *resp, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Process a response descriptor and return the associated op. */
+
struct qat_sym_session;
/**
@@ -69,6 +74,7 @@ struct qat_qp {
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
+ process_response_t process_response;
} __rte_cache_aligned;
@@ -76,6 +82,10 @@ int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie, enum qat_device_gen qat_dev_gen);
+
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 11/30] crypto/qat: move generic qp fn to qp file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (9 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 10/30] crypto/qat: make dequeue " Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 12/30] crypto/qat: separate sym-specific from generic qp setup Fiona Trahe
` (52 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Move the generic enqueue and dequeue fns from
the qat_sym.c file to the qat_qp.c file
Move generic qp structs to a new qat_qp.h file
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 152 +++++++++++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_qp.h | 63 ++++++++++++++++++
drivers/crypto/qat/qat_sym.c | 149 +-----------------------------------------
drivers/crypto/qat/qat_sym.h | 49 --------------
4 files changed, 216 insertions(+), 197 deletions(-)
create mode 100644 drivers/crypto/qat/qat_qp.h
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index a9b558563..527603300 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,7 +13,9 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
+#include "qat_qp.h"
#include "qat_sym.h"
+
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
@@ -469,3 +471,153 @@ static void adf_configure_queues(struct qat_qp *qp)
WRITE_CSR_RING_CONFIG(qp->mmap_bar_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_config);
}
+
+
+static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
+{
+ uint32_t div = data >> shift;
+ uint32_t mult = div << shift;
+
+ return data - mult;
+}
+
+static inline void
+txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
+ WRITE_CSR_RING_TAIL(qp->mmap_bar_addr, q->hw_bundle_number,
+ q->hw_queue_number, q->tail);
+ q->nb_pending_requests = 0;
+ q->csr_tail = q->tail;
+}
+
+static inline
+void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
+{
+ uint32_t old_head, new_head;
+ uint32_t max_head;
+
+ old_head = q->csr_head;
+ new_head = q->head;
+ max_head = qp->nb_descriptors * q->msg_size;
+
+ /* write out free descriptors */
+ void *cur_desc = (uint8_t *)q->base_addr + old_head;
+
+ if (new_head < old_head) {
+ memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, max_head - old_head);
+ memset(q->base_addr, ADF_RING_EMPTY_SIG_BYTE, new_head);
+ } else {
+ memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, new_head - old_head);
+ }
+ q->nb_processed_responses = 0;
+ q->csr_head = new_head;
+
+ /* write current head to CSR */
+ WRITE_CSR_RING_HEAD(qp->mmap_bar_addr, q->hw_bundle_number,
+ q->hw_queue_number, new_head);
+}
+
+uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
+{
+ register struct qat_queue *queue;
+ struct qat_qp *tmp_qp = (struct qat_qp *)qp;
+ register uint32_t nb_ops_sent = 0;
+ register int ret;
+ uint16_t nb_ops_possible = nb_ops;
+ register uint8_t *base_addr;
+ register uint32_t tail;
+ int overflow;
+
+ if (unlikely(nb_ops == 0))
+ return 0;
+
+ /* read params used a lot in main loop into registers */
+ queue = &(tmp_qp->tx_q);
+ base_addr = (uint8_t *)queue->base_addr;
+ tail = queue->tail;
+
+ /* Find how many can actually fit on the ring */
+ tmp_qp->inflights16 += nb_ops;
+ overflow = tmp_qp->inflights16 - queue->max_inflights;
+ if (overflow > 0) {
+ tmp_qp->inflights16 -= overflow;
+ nb_ops_possible = nb_ops - overflow;
+ if (nb_ops_possible == 0)
+ return 0;
+ }
+
+ while (nb_ops_sent != nb_ops_possible) {
+ ret = tmp_qp->build_request(*ops, base_addr + tail,
+ tmp_qp->op_cookies[tail / queue->msg_size],
+ tmp_qp->qat_dev_gen);
+ if (ret != 0) {
+ tmp_qp->stats.enqueue_err_count++;
+ /*
+ * This message cannot be enqueued,
+ * decrease number of ops that wasn't sent
+ */
+ tmp_qp->inflights16 -= nb_ops_possible - nb_ops_sent;
+ if (nb_ops_sent == 0)
+ return 0;
+ goto kick_tail;
+ }
+
+ tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ ops++;
+ nb_ops_sent++;
+ }
+kick_tail:
+ queue->tail = tail;
+ tmp_qp->stats.enqueued_count += nb_ops_sent;
+ queue->nb_pending_requests += nb_ops_sent;
+ if (tmp_qp->inflights16 < QAT_CSR_TAIL_FORCE_WRITE_THRESH ||
+ queue->nb_pending_requests > QAT_CSR_TAIL_WRITE_THRESH) {
+ txq_write_tail(tmp_qp, queue);
+ }
+ return nb_ops_sent;
+}
+
+uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
+{
+ struct qat_queue *rx_queue, *tx_queue;
+ struct qat_qp *tmp_qp = (struct qat_qp *)qp;
+ uint32_t head;
+ uint32_t resp_counter = 0;
+ uint8_t *resp_msg;
+
+ rx_queue = &(tmp_qp->rx_q);
+ tx_queue = &(tmp_qp->tx_q);
+ head = rx_queue->head;
+ resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
+
+ while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
+ resp_counter != nb_ops) {
+
+ tmp_qp->process_response(ops, resp_msg,
+ tmp_qp->op_cookies[head / rx_queue->msg_size],
+ tmp_qp->qat_dev_gen);
+
+ head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
+
+ resp_msg = (uint8_t *)rx_queue->base_addr + head;
+ ops++;
+ resp_counter++;
+ }
+ if (resp_counter > 0) {
+ rx_queue->head = head;
+ tmp_qp->stats.dequeued_count += resp_counter;
+ rx_queue->nb_processed_responses += resp_counter;
+ tmp_qp->inflights16 -= resp_counter;
+
+ if (rx_queue->nb_processed_responses >
+ QAT_CSR_HEAD_WRITE_THRESH)
+ rxq_free_desc(tmp_qp, rx_queue);
+ }
+ /* also check if tail needs to be advanced */
+ if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
+ tx_queue->tail != tx_queue->csr_tail) {
+ txq_write_tail(tmp_qp, tx_queue);
+ }
+ return resp_counter;
+}
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
new file mode 100644
index 000000000..87d55c5f2
--- /dev/null
+++ b/drivers/crypto/qat/qat_qp.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_QP_H_
+#define _QAT_QP_H_
+
+#include "qat_common.h"
+
+typedef int (*build_request_t)(void *op,
+ uint8_t *req, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Build a request from an op. */
+
+typedef int (*process_response_t)(void **ops,
+ uint8_t *resp, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Process a response descriptor and return the associated op. */
+
+/**
+ * Structure associated with each queue.
+ */
+struct qat_queue {
+ char memz_name[RTE_MEMZONE_NAMESIZE];
+ void *base_addr; /* Base address */
+ rte_iova_t base_phys_addr; /* Queue physical address */
+ uint32_t head; /* Shadow copy of the head */
+ uint32_t tail; /* Shadow copy of the tail */
+ uint32_t modulo;
+ uint32_t msg_size;
+ uint16_t max_inflights;
+ uint32_t queue_size;
+ uint8_t hw_bundle_number;
+ uint8_t hw_queue_number;
+ /* HW queue aka ring offset on bundle */
+ uint32_t csr_head; /* last written head value */
+ uint32_t csr_tail; /* last written tail value */
+ uint16_t nb_processed_responses;
+ /* number of responses processed since last CSR head write */
+ uint16_t nb_pending_requests;
+ /* number of requests pending since last CSR tail write */
+};
+
+struct qat_qp {
+ void *mmap_bar_addr;
+ uint16_t inflights16;
+ struct qat_queue tx_q;
+ struct qat_queue rx_q;
+ struct rte_cryptodev_stats stats;
+ struct rte_mempool *op_cookie_pool;
+ void **op_cookies;
+ uint32_t nb_descriptors;
+ enum qat_device_gen qat_dev_gen;
+ build_request_t build_request;
+ process_response_t process_response;
+} __rte_cache_aligned;
+
+uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+
+uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+
+#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 2bae913a1..ab8ce2c96 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -14,6 +14,7 @@
#include "qat_logs.h"
#include "qat_sym_session.h"
#include "qat_sym.h"
+#include "qat_qp.h"
#include "adf_transport_access_macros.h"
#define BYTE_LENGTH 8
@@ -83,8 +84,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
/** Creates a context in either AES or DES in ECB mode
* Depends on openssl libcrypto
*/
-static inline uint32_t
-adf_modulo(uint32_t data, uint32_t shift);
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
@@ -197,102 +196,6 @@ qat_bpicipher_postprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-static inline void
-txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
- WRITE_CSR_RING_TAIL(qp->mmap_bar_addr, q->hw_bundle_number,
- q->hw_queue_number, q->tail);
- q->nb_pending_requests = 0;
- q->csr_tail = q->tail;
-}
-
-static uint16_t
-qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
-{
- register struct qat_queue *queue;
- struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- register uint32_t nb_ops_sent = 0;
- register int ret;
- uint16_t nb_ops_possible = nb_ops;
- register uint8_t *base_addr;
- register uint32_t tail;
- int overflow;
-
- if (unlikely(nb_ops == 0))
- return 0;
-
- /* read params used a lot in main loop into registers */
- queue = &(tmp_qp->tx_q);
- base_addr = (uint8_t *)queue->base_addr;
- tail = queue->tail;
-
- /* Find how many can actually fit on the ring */
- tmp_qp->inflights16 += nb_ops;
- overflow = tmp_qp->inflights16 - queue->max_inflights;
- if (overflow > 0) {
- tmp_qp->inflights16 -= overflow;
- nb_ops_possible = nb_ops - overflow;
- if (nb_ops_possible == 0)
- return 0;
- }
-
- while (nb_ops_sent != nb_ops_possible) {
- ret = tmp_qp->build_request(*ops, base_addr + tail,
- tmp_qp->op_cookies[tail / queue->msg_size],
- tmp_qp->qat_dev_gen);
- if (ret != 0) {
- tmp_qp->stats.enqueue_err_count++;
- /*
- * This message cannot be enqueued,
- * decrease number of ops that wasn't sent
- */
- tmp_qp->inflights16 -= nb_ops_possible - nb_ops_sent;
- if (nb_ops_sent == 0)
- return 0;
- goto kick_tail;
- }
-
- tail = adf_modulo(tail + queue->msg_size, queue->modulo);
- ops++;
- nb_ops_sent++;
- }
-kick_tail:
- queue->tail = tail;
- tmp_qp->stats.enqueued_count += nb_ops_sent;
- queue->nb_pending_requests += nb_ops_sent;
- if (tmp_qp->inflights16 < QAT_CSR_TAIL_FORCE_WRITE_THRESH ||
- queue->nb_pending_requests > QAT_CSR_TAIL_WRITE_THRESH) {
- txq_write_tail(tmp_qp, queue);
- }
- return nb_ops_sent;
-}
-
-static inline
-void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
-{
- uint32_t old_head, new_head;
- uint32_t max_head;
-
- old_head = q->csr_head;
- new_head = q->head;
- max_head = qp->nb_descriptors * q->msg_size;
-
- /* write out free descriptors */
- void *cur_desc = (uint8_t *)q->base_addr + old_head;
-
- if (new_head < old_head) {
- memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, max_head - old_head);
- memset(q->base_addr, ADF_RING_EMPTY_SIG_BYTE, new_head);
- } else {
- memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, new_head - old_head);
- }
- q->nb_processed_responses = 0;
- q->csr_head = new_head;
-
- /* write current head to CSR */
- WRITE_CSR_RING_HEAD(qp->mmap_bar_addr, q->hw_bundle_number,
- q->hw_queue_number, new_head);
-}
-
uint16_t
qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
@@ -336,49 +239,6 @@ qat_sym_process_response(void **op, uint8_t *resp,
return 0;
}
-static uint16_t
-qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
-{
- struct qat_queue *rx_queue, *tx_queue;
- struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- uint32_t head;
- uint32_t resp_counter = 0;
- uint8_t *resp_msg;
-
- rx_queue = &(tmp_qp->rx_q);
- tx_queue = &(tmp_qp->tx_q);
- head = rx_queue->head;
- resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
-
- while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
- resp_counter != nb_ops) {
-
- tmp_qp->process_response(ops, resp_msg,
- tmp_qp->op_cookies[head / rx_queue->msg_size],
- tmp_qp->qat_dev_gen);
-
- head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
-
- resp_msg = (uint8_t *)rx_queue->base_addr + head;
- ops++;
- resp_counter++;
- }
- if (resp_counter > 0) {
- rx_queue->head = head;
- tmp_qp->stats.dequeued_count += resp_counter;
- rx_queue->nb_processed_responses += resp_counter;
- tmp_qp->inflights16 -= resp_counter;
-
- if (rx_queue->nb_processed_responses > QAT_CSR_HEAD_WRITE_THRESH)
- rxq_free_desc(tmp_qp, rx_queue);
- }
- /* also check if tail needs to be advanced */
- if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
- tx_queue->tail != tx_queue->csr_tail) {
- txq_write_tail(tmp_qp, tx_queue);
- }
- return resp_counter;
-}
uint16_t
qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
@@ -903,13 +763,6 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
return 0;
}
-static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
-{
- uint32_t div = data >> shift;
- uint32_t mult = div << shift;
-
- return data - mult;
-}
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 279d3a3ae..39574eeb6 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,57 +27,8 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
-typedef int (*build_request_t)(void *op,
- uint8_t *req, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
-/**< Build a request from an op. */
-
-typedef int (*process_response_t)(void **ops,
- uint8_t *resp, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
-/**< Process a response descriptor and return the associated op. */
-
struct qat_sym_session;
-/**
- * Structure associated with each queue.
- */
-struct qat_queue {
- char memz_name[RTE_MEMZONE_NAMESIZE];
- void *base_addr; /* Base address */
- rte_iova_t base_phys_addr; /* Queue physical address */
- uint32_t head; /* Shadow copy of the head */
- uint32_t tail; /* Shadow copy of the tail */
- uint32_t modulo;
- uint32_t msg_size;
- uint16_t max_inflights;
- uint32_t queue_size;
- uint8_t hw_bundle_number;
- uint8_t hw_queue_number;
- /* HW queue aka ring offset on bundle */
- uint32_t csr_head; /* last written head value */
- uint32_t csr_tail; /* last written tail value */
- uint16_t nb_processed_responses;
- /* number of responses processed since last CSR head write */
- uint16_t nb_pending_requests;
- /* number of requests pending since last CSR tail write */
-};
-
-struct qat_qp {
- void *mmap_bar_addr;
- uint16_t inflights16;
- struct qat_queue tx_q;
- struct qat_queue rx_q;
- struct rte_cryptodev_stats stats;
- struct rte_mempool *op_cookie_pool;
- void **op_cookies;
- uint32_t nb_descriptors;
- enum qat_device_gen qat_dev_gen;
- build_request_t build_request;
- process_response_t process_response;
-} __rte_cache_aligned;
-
-
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 12/30] crypto/qat: separate sym-specific from generic qp setup
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (10 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 11/30] crypto/qat: move generic qp fn to qp file Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 13/30] crypto/qat: move sym-specific qp code to sym file Fiona Trahe
` (51 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Extracted all sym-specific code from qp setup fns, leaving
generic qat_qp_setup fn and helper fns. Created a new
meta-data struct qat_qp_config to hold all the data needed
to create a qp, filled this out in the sym-specific code
and passed to the generic qp_setup fn.
No need now for rx and tx queue_create fns, one generic
queue_create fn replaces these.
Included the service name (e.g. "sym") in the qp memzone
and cookie pool names.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../qat/qat_adf/adf_transport_access_macros.h | 2 +
drivers/crypto/qat/qat_qp.c | 221 +++++++++++----------
drivers/crypto/qat/qat_qp.h | 17 ++
3 files changed, 137 insertions(+), 103 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index bfdbc979f..8b88b69de 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -52,6 +52,8 @@
#define ADF_NUM_BUNDLES_PER_DEV 1
#define ADF_NUM_SYM_QPS_PER_BUNDLE 2
+#define ADF_RING_DIR_TX 0
+#define ADF_RING_DIR_RX 1
/* Valid internal msg size values */
#define ADF_MSG_SIZE_32 0x01
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 527603300..caec4ea4c 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -18,8 +18,8 @@
#include "adf_transport_access_macros.h"
-#define ADF_MAX_SYM_DESC 4096
-#define ADF_MIN_SYM_DESC 128
+#define ADF_MAX_DESC 4096
+#define ADF_MIN_DESC 128
#define ADF_SYM_TX_RING_DESC_SIZE 128
#define ADF_SYM_RX_RING_DESC_SIZE 32
#define ADF_SYM_TX_QUEUE_STARTOFF 2
@@ -34,16 +34,9 @@
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
-static int qat_tx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t id, uint32_t nb_desc,
- int socket_id);
-static int qat_rx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t id, uint32_t nb_desc,
- int socket_id);
static void qat_queue_delete(struct qat_queue *queue);
static int qat_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint32_t nb_desc, uint8_t desc_size,
- int socket_id);
+ struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
static void adf_configure_queues(struct qat_qp *queue);
@@ -100,29 +93,18 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
memzone_flags, queue_size);
}
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
+static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+ struct qat_qp_config *qat_qp_conf)
{
struct qat_qp *qp;
struct rte_pci_device *pci_dev;
- int ret;
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
- PMD_INIT_FUNC_TRACE();
-
- /* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[queue_pair_id] != NULL) {
- ret = qat_sym_qp_release(dev, queue_pair_id);
- if (ret < 0)
- return ret;
- }
-
- if ((qp_conf->nb_descriptors > ADF_MAX_SYM_DESC) ||
- (qp_conf->nb_descriptors < ADF_MIN_SYM_DESC)) {
+ if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
+ (qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
PMD_DRV_LOG(ERR, "Can't create qp for %u descriptors",
- qp_conf->nb_descriptors);
+ qat_qp_conf->nb_descriptors);
return -EINVAL;
}
@@ -134,13 +116,6 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EINVAL;
}
- if (queue_pair_id >=
- (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device",
- queue_pair_id);
- return -EINVAL;
- }
/* Allocate the queue pair data structure. */
qp = rte_zmalloc("qat PMD qp metadata",
sizeof(*qp), RTE_CACHE_LINE_SIZE);
@@ -148,9 +123,9 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
PMD_DRV_LOG(ERR, "Failed to alloc mem for qp struct");
return -ENOMEM;
}
- qp->nb_descriptors = qp_conf->nb_descriptors;
+ qp->nb_descriptors = qat_qp_conf->nb_descriptors;
qp->op_cookies = rte_zmalloc("qat PMD op cookie pointer",
- qp_conf->nb_descriptors * sizeof(*qp->op_cookies),
+ qat_qp_conf->nb_descriptors * sizeof(*qp->op_cookies),
RTE_CACHE_LINE_SIZE);
if (qp->op_cookies == NULL) {
PMD_DRV_LOG(ERR, "Failed to alloc mem for cookie");
@@ -161,15 +136,15 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qp->mmap_bar_addr = pci_dev->mem_resource[0].addr;
qp->inflights16 = 0;
- if (qat_tx_queue_create(dev, &(qp->tx_q),
- queue_pair_id, qp_conf->nb_descriptors, socket_id) != 0) {
+ if (qat_queue_create(dev, &(qp->tx_q), qat_qp_conf,
+ ADF_RING_DIR_TX) != 0) {
PMD_INIT_LOG(ERR, "Tx queue create failed "
"queue_pair_id=%u", queue_pair_id);
goto create_err;
}
- if (qat_rx_queue_create(dev, &(qp->rx_q),
- queue_pair_id, qp_conf->nb_descriptors, socket_id) != 0) {
+ if (qat_queue_create(dev, &(qp->rx_q), qat_qp_conf,
+ ADF_RING_DIR_RX) != 0) {
PMD_DRV_LOG(ERR, "Rx queue create failed "
"queue_pair_id=%hu", queue_pair_id);
qat_queue_delete(&(qp->tx_q));
@@ -178,16 +153,17 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
adf_configure_queues(qp);
adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
- snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_qp_op_%d_%hu",
- pci_dev->driver->driver.name, dev->data->dev_id,
- queue_pair_id);
+
+ snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
+ pci_dev->driver->driver.name, qat_qp_conf->service_str,
+ dev->data->dev_id, queue_pair_id);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
qp->nb_descriptors,
- sizeof(struct qat_sym_op_cookie), 64, 0,
- NULL, NULL, NULL, NULL, socket_id,
+ qat_qp_conf->cookie_size, 64, 0,
+ NULL, NULL, NULL, NULL, qat_qp_conf->socket_id,
0);
if (!qp->op_cookie_pool) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot create"
@@ -200,6 +176,67 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
PMD_DRV_LOG(ERR, "QAT PMD Cannot get op_cookie");
goto create_err;
}
+ }
+
+ struct qat_pmd_private *internals
+ = dev->data->dev_private;
+ qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->build_request = qat_qp_conf->build_request;
+ qp->process_response = qat_qp_conf->process_response;
+
+ dev->data->queue_pairs[queue_pair_id] = qp;
+ return 0;
+
+create_err:
+ rte_free(qp);
+ return -EFAULT;
+}
+
+
+
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (dev->data->queue_pairs[qp_id] != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+
+ qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
+ qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_TX_QUEUE_STARTOFF;
+ qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_RX_QUEUE_STARTOFF;
+ qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
+ qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
struct qat_sym_op_cookie *sql_cookie =
qp->op_cookies[i];
@@ -215,24 +252,11 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qat_sgl_list_dst);
}
- struct qat_pmd_private *internals
- = dev->data->dev_private;
- qp->qat_dev_gen = internals->qat_dev_gen;
- qp->build_request = qat_sym_build_request;
- qp->process_response = qat_sym_process_response;
-
- dev->data->queue_pairs[queue_pair_id] = qp;
- return 0;
+ return ret;
-create_err:
- if (qp->op_cookie_pool)
- rte_mempool_free(qp->op_cookie_pool);
- rte_free(qp->op_cookies);
- rte_free(qp);
- return -EFAULT;
}
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
@@ -266,38 +290,13 @@ int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
return 0;
}
-static int qat_tx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t qp_id,
- uint32_t nb_desc, int socket_id)
-{
- PMD_INIT_FUNC_TRACE();
- queue->hw_bundle_number = qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE;
- queue->hw_queue_number = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- PMD_DRV_LOG(DEBUG, "TX ring for %u msgs: qp_id %d, bundle %u, ring %u",
- nb_desc, qp_id, queue->hw_bundle_number,
- queue->hw_queue_number);
-
- return qat_queue_create(dev, queue, nb_desc,
- ADF_SYM_TX_RING_DESC_SIZE, socket_id);
-}
-static int qat_rx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t qp_id, uint32_t nb_desc,
- int socket_id)
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
- PMD_INIT_FUNC_TRACE();
- queue->hw_bundle_number = qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE;
- queue->hw_queue_number = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
-
- PMD_DRV_LOG(DEBUG, "RX ring for %u msgs: qp id %d, bundle %u, ring %u",
- nb_desc, qp_id, queue->hw_bundle_number,
- queue->hw_queue_number);
- return qat_queue_create(dev, queue, nb_desc,
- ADF_SYM_RX_RING_DESC_SIZE, socket_id);
+ return qat_qp_release(dev, queue_pair_id);
}
+
static void qat_queue_delete(struct qat_queue *queue)
{
const struct rte_memzone *mz;
@@ -323,15 +322,21 @@ static void qat_queue_delete(struct qat_queue *queue)
static int
qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
- uint32_t nb_desc, uint8_t desc_size, int socket_id)
+ struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
void *io_addr;
const struct rte_memzone *qp_mz;
- uint32_t queue_size_bytes = nb_desc*desc_size;
struct rte_pci_device *pci_dev;
+ int ret = 0;
+ uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
+ qp_conf->tx_msg_size : qp_conf->rx_msg_size);
+ uint32_t queue_size_bytes = (qp_conf->nb_descriptors)*(desc_size);
+
+ queue->hw_bundle_number = qp_conf->hw_bundle_num;
+ queue->hw_queue_number = (dir == ADF_RING_DIR_TX ?
+ qp_conf->tx_ring_num : qp_conf->rx_ring_num);
- PMD_INIT_FUNC_TRACE();
if (desc_size > ADF_MSG_SIZE_TO_BYTES(ADF_MAX_MSG_SIZE)) {
PMD_DRV_LOG(ERR, "Invalid descriptor size %d", desc_size);
return -EINVAL;
@@ -342,11 +347,13 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
/*
* Allocate a memzone for the queue - create a unique name.
*/
- snprintf(queue->memz_name, sizeof(queue->memz_name), "%s_%s_%d_%d_%d",
- pci_dev->driver->driver.name, "qp_mem", dev->data->dev_id,
+ snprintf(queue->memz_name, sizeof(queue->memz_name),
+ "%s_%s_%s_%d_%d_%d",
+ pci_dev->driver->driver.name, qp_conf->service_str,
+ "qp_mem", dev->data->dev_id,
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
- socket_id);
+ qp_conf->socket_id);
if (qp_mz == NULL) {
PMD_DRV_LOG(ERR, "Failed to allocate ring memzone");
return -ENOMEM;
@@ -359,27 +366,31 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
PMD_DRV_LOG(ERR, "Invalid alignment on queue create "
" 0x%"PRIx64"\n",
queue->base_phys_addr);
- return -EFAULT;
+ ret = -EFAULT;
+ goto queue_create_err;
}
- if (adf_verify_queue_size(desc_size, nb_desc, &(queue->queue_size))
- != 0) {
+ if (adf_verify_queue_size(desc_size, qp_conf->nb_descriptors,
+ &(queue->queue_size)) != 0) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
- return -EINVAL;
+ ret = -EINVAL;
+ goto queue_create_err;
}
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
- PMD_DRV_LOG(DEBUG, "RING size in CSR: %u, in bytes %u, nb msgs %u,"
- " msg_size %u, max_inflights %u modulo %u",
- queue->queue_size, queue_size_bytes,
- nb_desc, desc_size, queue->max_inflights,
- queue->modulo);
+ PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
+ " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ queue->memz_name,
+ queue->queue_size, queue_size_bytes,
+ qp_conf->nb_descriptors, desc_size,
+ queue->max_inflights, queue->modulo);
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
- return -EINVAL;
+ ret = -EINVAL;
+ goto queue_create_err;
}
queue->head = 0;
queue->tail = 0;
@@ -398,6 +409,10 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
WRITE_CSR_RING_BASE(io_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_base);
return 0;
+
+queue_create_err:
+ rte_memzone_free(qp_mz);
+ return ret;
}
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 87d55c5f2..edebb8773 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -17,6 +17,23 @@ typedef int (*process_response_t)(void **ops,
/**< Process a response descriptor and return the associated op. */
/**
+ * Structure with data needed for creation of queue pair.
+ */
+struct qat_qp_config {
+ uint8_t hw_bundle_num;
+ uint8_t tx_ring_num;
+ uint8_t rx_ring_num;
+ uint16_t tx_msg_size;
+ uint16_t rx_msg_size;
+ uint32_t nb_descriptors;
+ uint32_t cookie_size;
+ int socket_id;
+ build_request_t build_request;
+ process_response_t process_response;
+ const char *service_str;
+};
+
+/**
* Structure associated with each queue.
*/
struct qat_queue {
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 13/30] crypto/qat: move sym-specific qp code to sym file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (11 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 12/30] crypto/qat: separate sym-specific from generic qp setup Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 14/30] crypto/qat: remove dependencies on cryptodev from common Fiona Trahe
` (50 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Move sym qp setup code from qat_qp.c to qat_sym.c
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 78 ++------------------------------------------
drivers/crypto/qat/qat_qp.h | 6 ++++
drivers/crypto/qat/qat_sym.c | 75 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 84 insertions(+), 75 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index caec4ea4c..947090580 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -20,11 +20,7 @@
#define ADF_MAX_DESC 4096
#define ADF_MIN_DESC 128
-#define ADF_SYM_TX_RING_DESC_SIZE 128
-#define ADF_SYM_RX_RING_DESC_SIZE 32
-#define ADF_SYM_TX_QUEUE_STARTOFF 2
-/* Offset from bundle start to 1st Sym Tx queue */
-#define ADF_SYM_RX_QUEUE_STARTOFF 10
+
#define ADF_ARB_REG_SLOT 0x1000
#define ADF_ARB_RINGSRVARBEN_OFFSET 0x19C
@@ -93,7 +89,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
memzone_flags, queue_size);
}
-static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
{
struct qat_qp *qp;
@@ -192,71 +188,7 @@ static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
-{
- struct qat_qp *qp;
- int ret = 0;
- uint32_t i;
- struct qat_qp_config qat_qp_conf;
-
- /* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[qp_id] != NULL) {
- ret = qat_sym_qp_release(dev, qp_id);
- if (ret < 0)
- return ret;
- }
- if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
- return -EINVAL;
- }
-
-
- qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
- qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
- qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
- qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
- qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
- qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
- qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
- qat_qp_conf.socket_id = socket_id;
- qat_qp_conf.service_str = "sym";
-
- ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
- if (ret != 0)
- return ret;
-
- qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
-
- for (i = 0; i < qp->nb_descriptors; i++) {
-
- struct qat_sym_op_cookie *sql_cookie =
- qp->op_cookies[i];
-
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_src);
-
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_dst);
- }
-
- return ret;
-
-}
-
-static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
@@ -291,10 +223,6 @@ static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
}
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
-{
- return qat_qp_release(dev, queue_pair_id);
-}
static void qat_queue_delete(struct qat_queue *queue)
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index edebb8773..0cdf37f61 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -77,4 +77,10 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
uint16_t
qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+int
+qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id);
+
+int
+qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+ struct qat_qp_config *qat_qp_conf);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index ab8ce2c96..4f86f2258 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -23,6 +23,12 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
+#define ADF_SYM_TX_RING_DESC_SIZE 128
+#define ADF_SYM_RX_RING_DESC_SIZE 32
+#define ADF_SYM_TX_QUEUE_STARTOFF 2
+/* Offset from bundle start to 1st Sym Tx queue */
+#define ADF_SYM_RX_QUEUE_STARTOFF 10
+
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -798,3 +804,72 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
}
+
+
+
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+{
+ return qat_qp_release(dev, queue_pair_id);
+}
+
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (dev->data->queue_pairs[qp_id] != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+
+ qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
+ qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_TX_QUEUE_STARTOFF;
+ qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_RX_QUEUE_STARTOFF;
+ qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
+ qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
+
+ struct qat_sym_op_cookie *sql_cookie =
+ qp->op_cookies[i];
+
+ sql_cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_list_src);
+
+ sql_cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_list_dst);
+ }
+
+ return ret;
+
+}
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 14/30] crypto/qat: remove dependencies on cryptodev from common
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (12 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 13/30] crypto/qat: move sym-specific qp code to sym file Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 15/30] crypto/qat: move defines from sym to qp header file Fiona Trahe
` (49 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Remove dependence on rte_cryptodev from common qp code
to facilitate being used by other device types in future.
Transferred required data into qat-specific structures.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.h | 11 +++++-
drivers/crypto/qat/qat_qp.c | 67 +++++++++++++++++++---------------
drivers/crypto/qat/qat_qp.h | 8 +++-
drivers/crypto/qat/qat_sym.c | 15 +++++---
drivers/crypto/qat/rte_qat_cryptodev.c | 4 +-
5 files changed, 66 insertions(+), 39 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 2cb8e7612..64706abae 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -14,7 +14,11 @@ extern uint8_t cryptodev_qat_driver_id;
extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-/** private data structure for each QAT device */
+/** private data structure for each QAT device.
+ * In this context a QAT device is a device offering only one service,
+ * so there can be more than 1 device on a pci_dev (VF),
+ * one for symmetric crypto, one for compression
+ */
struct qat_pmd_private {
unsigned int max_nb_queue_pairs;
/**< Max number of queue pairs supported by device */
@@ -23,6 +27,11 @@ struct qat_pmd_private {
enum qat_device_gen qat_dev_gen;
/**< QAT device generation */
const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device capabilities */
+ struct rte_pci_device *pci_dev;
+ /**< PCI information. */
+ uint8_t dev_id;
+ /**< Device ID for this instance */
};
int qat_dev_config(struct rte_cryptodev *dev,
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 947090580..e3c476bbd 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -6,7 +6,6 @@
#include <rte_dev.h>
#include <rte_malloc.h>
#include <rte_memzone.h>
-#include <rte_cryptodev_pmd.h>
#include <rte_pci.h>
#include <rte_bus_pci.h>
#include <rte_atomic.h>
@@ -31,7 +30,7 @@
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
-static int qat_queue_create(struct rte_cryptodev *dev,
+static int qat_queue_create(struct qat_pmd_private *qat_dev,
struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
@@ -89,14 +88,19 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
memzone_flags, queue_size);
}
-int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_qp_setup(struct qat_pmd_private *qat_dev,
+ struct qat_qp **qp_addr,
+ uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
{
struct qat_qp *qp;
- struct rte_pci_device *pci_dev;
+ struct rte_pci_device *pci_dev = qat_dev->pci_dev;
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
+ PMD_DRV_LOG(DEBUG, "Setup qp %u on device %d gen %d",
+ queue_pair_id, qat_dev->dev_id, qat_dev->qat_dev_gen);
+
if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
(qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
PMD_DRV_LOG(ERR, "Can't create qp for %u descriptors",
@@ -104,8 +108,6 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EINVAL;
}
- pci_dev = RTE_DEV_TO_PCI(dev->device);
-
if (pci_dev->mem_resource[0].addr == NULL) {
PMD_DRV_LOG(ERR, "Could not find VF config space "
"(UIO driver attached?).");
@@ -132,14 +134,14 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qp->mmap_bar_addr = pci_dev->mem_resource[0].addr;
qp->inflights16 = 0;
- if (qat_queue_create(dev, &(qp->tx_q), qat_qp_conf,
+ if (qat_queue_create(qat_dev, &(qp->tx_q), qat_qp_conf,
ADF_RING_DIR_TX) != 0) {
PMD_INIT_LOG(ERR, "Tx queue create failed "
"queue_pair_id=%u", queue_pair_id);
goto create_err;
}
- if (qat_queue_create(dev, &(qp->rx_q), qat_qp_conf,
+ if (qat_queue_create(qat_dev, &(qp->rx_q), qat_qp_conf,
ADF_RING_DIR_RX) != 0) {
PMD_DRV_LOG(ERR, "Rx queue create failed "
"queue_pair_id=%hu", queue_pair_id);
@@ -152,7 +154,7 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
pci_dev->driver->driver.name, qat_qp_conf->service_str,
- dev->data->dev_id, queue_pair_id);
+ qat_dev->dev_id, queue_pair_id);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
@@ -174,13 +176,15 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
}
}
- struct qat_pmd_private *internals
- = dev->data->dev_private;
- qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->qat_dev_gen = qat_dev->qat_dev_gen;
qp->build_request = qat_qp_conf->build_request;
qp->process_response = qat_qp_conf->process_response;
+ qp->qat_dev = qat_dev;
+
+ PMD_DRV_LOG(DEBUG, "QP setup complete: id: %d, cookiepool: %s",
+ queue_pair_id, op_cookie_pool_name);
- dev->data->queue_pairs[queue_pair_id] = qp;
+ *qp_addr = qp;
return 0;
create_err:
@@ -188,10 +192,9 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_qp_release(struct qat_qp **qp_addr)
{
- struct qat_qp *qp =
- (struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
+ struct qat_qp *qp = *qp_addr;
uint32_t i;
PMD_INIT_FUNC_TRACE();
@@ -200,6 +203,9 @@ int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
return 0;
}
+ PMD_DRV_LOG(DEBUG, "Free qp %u on device %d",
+ qp->queue_pair_id, qp->qat_dev->dev_id);
+
/* Don't free memory if there are still responses to be processed */
if (qp->inflights16 == 0) {
qat_queue_delete(&(qp->tx_q));
@@ -218,13 +224,11 @@ int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
rte_free(qp->op_cookies);
rte_free(qp);
- dev->data->queue_pairs[queue_pair_id] = NULL;
+ *qp_addr = NULL;
return 0;
}
-
-
static void qat_queue_delete(struct qat_queue *queue)
{
const struct rte_memzone *mz;
@@ -234,6 +238,9 @@ static void qat_queue_delete(struct qat_queue *queue)
PMD_DRV_LOG(DEBUG, "Invalid queue");
return;
}
+ PMD_DRV_LOG(DEBUG, "Free ring %d, memzone: %s",
+ queue->hw_queue_number, queue->memz_name);
+
mz = rte_memzone_lookup(queue->memz_name);
if (mz != NULL) {
/* Write an unused pattern to the queue memory. */
@@ -249,13 +256,13 @@ static void qat_queue_delete(struct qat_queue *queue)
}
static int
-qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
+qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
void *io_addr;
const struct rte_memzone *qp_mz;
- struct rte_pci_device *pci_dev;
+ struct rte_pci_device *pci_dev = qat_dev->pci_dev;
int ret = 0;
uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
qp_conf->tx_msg_size : qp_conf->rx_msg_size);
@@ -270,15 +277,13 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
return -EINVAL;
}
- pci_dev = RTE_DEV_TO_PCI(dev->device);
-
/*
* Allocate a memzone for the queue - create a unique name.
*/
snprintf(queue->memz_name, sizeof(queue->memz_name),
"%s_%s_%s_%d_%d_%d",
pci_dev->driver->driver.name, qp_conf->service_str,
- "qp_mem", dev->data->dev_id,
+ "qp_mem", qat_dev->dev_id,
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
qp_conf->socket_id);
@@ -308,12 +313,6 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
- PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
- " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
- queue->memz_name,
- queue->queue_size, queue_size_bytes,
- qp_conf->nb_descriptors, desc_size,
- queue->max_inflights, queue->modulo);
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
@@ -336,6 +335,14 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
WRITE_CSR_RING_BASE(io_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_base);
+
+ PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
+ " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ queue->memz_name,
+ queue->queue_size, queue_size_bytes,
+ qp_conf->nb_descriptors, desc_size,
+ queue->max_inflights, queue->modulo);
+
return 0;
queue_create_err:
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 0cdf37f61..7bd8fdcec 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,6 +5,7 @@
#define _QAT_QP_H_
#include "qat_common.h"
+#include "qat_device.h"
typedef int (*build_request_t)(void *op,
uint8_t *req, void *op_cookie,
@@ -69,6 +70,8 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
process_response_t process_response;
+ struct qat_pmd_private *qat_dev;
+ /**< qat device this qp is on */
} __rte_cache_aligned;
uint16_t
@@ -78,9 +81,10 @@ uint16_t
qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
int
-qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id);
+qat_qp_release(struct qat_qp **qp_addr);
int
-qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+qat_qp_setup(struct qat_pmd_private *qat_dev,
+ struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 4f86f2258..8ab95ac43 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -809,7 +809,10 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
- return qat_qp_release(dev, queue_pair_id);
+ PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
+ queue_pair_id, dev->data->dev_id);
+ return qat_qp_release((struct qat_qp **)
+ &(dev->data->queue_pairs[queue_pair_id]));
}
int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
@@ -820,9 +823,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
int ret = 0;
uint32_t i;
struct qat_qp_config qat_qp_conf;
+ struct qat_qp **qp_addr =
+ (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
+ struct qat_pmd_private *qat_private = dev->data->dev_private;
/* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[qp_id] != NULL) {
+ if (*qp_addr != NULL) {
ret = qat_sym_qp_release(dev, qp_id);
if (ret < 0)
return ret;
@@ -833,7 +839,6 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
return -EINVAL;
}
-
qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
ADF_SYM_TX_QUEUE_STARTOFF;
@@ -848,11 +853,11 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.socket_id = socket_id;
qat_qp_conf.service_str = "sym";
- ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ ret = qat_qp_setup(qat_private, qp_addr, qp_id, &qat_qp_conf);
if (ret != 0)
return ret;
- qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+ qp = (struct qat_qp *)*qp_addr;
for (i = 0; i < qp->nb_descriptors; i++) {
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 05cb5ef40..fe19b18b6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -96,7 +96,9 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
internals = cryptodev->data->dev_private;
internals->max_nb_sessions = init_params->max_nb_sessions;
- switch (pci_dev->id.device_id) {
+ internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
+ internals->dev_id = cryptodev->data->dev_id;
+ switch (internals->pci_dev->id.device_id) {
case 0x0443:
internals->qat_dev_gen = QAT_GEN1;
internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 15/30] crypto/qat: move defines from sym to qp header file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (13 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 14/30] crypto/qat: remove dependencies on cryptodev from common Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 16/30] crypto/qat: create data structures to support different generations Fiona Trahe
` (48 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Move defines related to coalescing from sym header file to qp header
file as these will be common for all services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.h | 7 +++++++
drivers/crypto/qat/qat_sym.h | 7 -------
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 7bd8fdcec..8cf072c55 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -7,6 +7,13 @@
#include "qat_common.h"
#include "qat_device.h"
+#define QAT_CSR_HEAD_WRITE_THRESH 32U
+/* number of requests to accumulate before writing head CSR */
+#define QAT_CSR_TAIL_WRITE_THRESH 32U
+/* number of requests to accumulate before writing tail CSR */
+#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
+/* number of inflights below which no tail write coalescing should occur */
+
typedef int (*build_request_t)(void *op,
uint8_t *req, void *op_cookie,
enum qat_device_gen qat_dev_gen);
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 39574eeb6..b92ec72de 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -20,13 +20,6 @@
(((num) + (align) - 1) & ~((align) - 1))
#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
-#define QAT_CSR_HEAD_WRITE_THRESH 32U
-/* number of requests to accumulate before writing head CSR */
-#define QAT_CSR_TAIL_WRITE_THRESH 32U
-/* number of requests to accumulate before writing tail CSR */
-#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
-/* number of inflights below which no tail write coalescing should occur */
-
struct qat_sym_session;
int
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 16/30] crypto/qat: create data structures to support different generations
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (14 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 15/30] crypto/qat: move defines from sym to qp header file Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:51 ` [PATCH 17/30] crypto/qat: rename sgl related objects Fiona Trahe
` (47 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Create data structures to support different generations
of qat hardware supplying services through different queue pairs.
- Add two new structs qat_gen_hw_data and qat_qp_hw_dat
- Add a qat_service_type enum
An array of qat_qp_hw_data elements is
initialised with constants, these are arranged so that the qp_id used
on the API can be used as an index to pick up the qp data to use.
The constants are common to current generations,
new arrays will be added for future generations.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../qat/qat_adf/adf_transport_access_macros.h | 6 +-
drivers/crypto/qat/qat_common.h | 8 ++
drivers/crypto/qat/qat_device.c | 21 ++++-
drivers/crypto/qat/qat_device.h | 9 +++
drivers/crypto/qat/qat_qp.c | 94 ++++++++++++++++++++--
drivers/crypto/qat/qat_qp.h | 18 ++++-
drivers/crypto/qat/qat_sym.c | 27 +++----
7 files changed, 154 insertions(+), 29 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index 8b88b69de..2136d54ab 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -51,7 +51,8 @@
#define ADF_DEFAULT_RING_SIZE ADF_RING_SIZE_16K
#define ADF_NUM_BUNDLES_PER_DEV 1
-#define ADF_NUM_SYM_QPS_PER_BUNDLE 2
+/* Maximum number of qps for any service type */
+#define ADF_MAX_QPS_PER_BUNDLE 4
#define ADF_RING_DIR_TX 0
#define ADF_RING_DIR_RX 1
@@ -132,4 +133,5 @@ do { \
#define WRITE_CSR_INT_FLAG_AND_COL(csr_base_addr, bank, value) \
ADF_CSR_WR(csr_base_addr, (ADF_RING_BUNDLE_SIZE * bank) + \
ADF_RING_CSR_INT_FLAG_AND_COL, value)
-#endif
+
+#endif /*ADF_TRANSPORT_ACCESS_MACROS_H */
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 7802e96f9..c3e7bd9a7 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -23,6 +23,14 @@ enum qat_device_gen {
QAT_GEN2,
};
+enum qat_service_type {
+ QAT_SERVICE_ASYMMETRIC = 0,
+ QAT_SERVICE_SYMMETRIC,
+ QAT_SERVICE_COMPRESSION,
+ QAT_SERVICE_INVALID
+};
+#define QAT_MAX_SERVICES (QAT_SERVICE_INVALID)
+
/**< Common struct for scatter-gather list operations */
struct qat_alg_buf {
uint32_t len;
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index ac6bd1af6..cdf4f7058 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -4,6 +4,21 @@
#include "qat_device.h"
#include "adf_transport_access_macros.h"
+#include "qat_qp.h"
+
+/* Hardware device information per generation */
+__extension__
+struct qat_gen_hw_data qp_gen_config[] = {
+ [QAT_GEN1] = {
+ .dev_gen = QAT_GEN1,
+ .qp_hw_data = qat_gen1_qps,
+ },
+ [QAT_GEN2] = {
+ .dev_gen = QAT_GEN2,
+ .qp_hw_data = qat_gen1_qps,
+ /* gen2 has same ring layout as gen1 */
+ },
+};
int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
@@ -42,12 +57,14 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info)
{
struct qat_pmd_private *internals = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[internals->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
if (info != NULL) {
info->max_nb_queue_pairs =
- ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV;
+ qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
info->feature_flags = dev->feature_flags;
info->capabilities = internals->qat_dev_capabilities;
info->sym.max_nb_sessions = internals->max_nb_sessions;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 64706abae..0983e3c2e 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -8,6 +8,8 @@
#include <rte_bus_pci.h>
#include "qat_common.h"
#include "qat_logs.h"
+#include "adf_transport_access_macros.h"
+#include "qat_qp.h"
extern uint8_t cryptodev_qat_driver_id;
@@ -34,6 +36,13 @@ struct qat_pmd_private {
/**< Device ID for this instance */
};
+struct qat_gen_hw_data {
+ enum qat_device_gen dev_gen;
+ const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
+};
+
+extern struct qat_gen_hw_data qp_gen_config[];
+
int qat_dev_config(struct rte_cryptodev *dev,
struct rte_cryptodev_config *config);
int qat_dev_start(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index e3c476bbd..a59735344 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,10 +13,10 @@
#include "qat_logs.h"
#include "qat_qp.h"
-#include "qat_sym.h"
-
+#include "qat_device.h"
#include "adf_transport_access_macros.h"
+
#define ADF_MAX_DESC 4096
#define ADF_MIN_DESC 128
@@ -27,6 +27,78 @@
ADF_CSR_WR(csr_addr, ADF_ARB_RINGSRVARBEN_OFFSET + \
(ADF_ARB_REG_SLOT * index), value)
+__extension__
+const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
+ [ADF_MAX_QPS_PER_BUNDLE] = {
+ /* queue pairs which provide an asymmetric crypto service */
+ [QAT_SERVICE_ASYMMETRIC] = {
+ {
+ .service_type = QAT_SERVICE_ASYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 0,
+ .rx_ring_num = 8,
+ .tx_msg_size = 64,
+ .rx_msg_size = 32,
+
+ }, {
+ .service_type = QAT_SERVICE_ASYMMETRIC,
+ .tx_ring_num = 1,
+ .rx_ring_num = 9,
+ .tx_msg_size = 64,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ },
+ /* queue pairs which provide a symmetric crypto service */
+ [QAT_SERVICE_SYMMETRIC] = {
+ {
+ .service_type = QAT_SERVICE_SYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 2,
+ .rx_ring_num = 10,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ },
+ {
+ .service_type = QAT_SERVICE_SYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 3,
+ .rx_ring_num = 11,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ },
+ /* queue pairs which provide a compression service */
+ [QAT_SERVICE_COMPRESSION] = {
+ {
+ .service_type = QAT_SERVICE_COMPRESSION,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 6,
+ .rx_ring_num = 14,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_COMPRESSION,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 7,
+ .rx_ring_num = 15,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ }
+};
+
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
@@ -38,6 +110,18 @@ static void adf_configure_queues(struct qat_qp *queue);
static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr);
static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr);
+
+int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
+ enum qat_service_type service)
+{
+ int i, count;
+
+ for (i = 0, count = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++)
+ if (qp_hw_data[i].service_type == service)
+ count++;
+ return count * ADF_NUM_BUNDLES_PER_DEV;
+}
+
static const struct rte_memzone *
queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
int socket_id)
@@ -265,12 +349,12 @@ qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
struct rte_pci_device *pci_dev = qat_dev->pci_dev;
int ret = 0;
uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
- qp_conf->tx_msg_size : qp_conf->rx_msg_size);
+ qp_conf->hw->tx_msg_size : qp_conf->hw->rx_msg_size);
uint32_t queue_size_bytes = (qp_conf->nb_descriptors)*(desc_size);
- queue->hw_bundle_number = qp_conf->hw_bundle_num;
+ queue->hw_bundle_number = qp_conf->hw->hw_bundle_num;
queue->hw_queue_number = (dir == ADF_RING_DIR_TX ?
- qp_conf->tx_ring_num : qp_conf->rx_ring_num);
+ qp_conf->hw->tx_ring_num : qp_conf->hw->rx_ring_num);
if (desc_size > ADF_MSG_SIZE_TO_BYTES(ADF_MAX_MSG_SIZE)) {
PMD_DRV_LOG(ERR, "Invalid descriptor size %d", desc_size);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 8cf072c55..f808e16a5 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,7 +5,8 @@
#define _QAT_QP_H_
#include "qat_common.h"
-#include "qat_device.h"
+#include <rte_cryptodev_pmd.h>
+#include "adf_transport_access_macros.h"
#define QAT_CSR_HEAD_WRITE_THRESH 32U
/* number of requests to accumulate before writing head CSR */
@@ -27,12 +28,19 @@ typedef int (*process_response_t)(void **ops,
/**
* Structure with data needed for creation of queue pair.
*/
-struct qat_qp_config {
+struct qat_qp_hw_data {
+ enum qat_service_type service_type;
uint8_t hw_bundle_num;
uint8_t tx_ring_num;
uint8_t rx_ring_num;
uint16_t tx_msg_size;
uint16_t rx_msg_size;
+};
+/**
+ * Structure with data needed for creation of queue pair.
+ */
+struct qat_qp_config {
+ const struct qat_qp_hw_data *hw;
uint32_t nb_descriptors;
uint32_t cookie_size;
int socket_id;
@@ -81,6 +89,8 @@ struct qat_qp {
/**< qat device this qp is on */
} __rte_cache_aligned;
+extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_PER_BUNDLE];
+
uint16_t
qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
@@ -94,4 +104,8 @@ int
qat_qp_setup(struct qat_pmd_private *qat_dev,
struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
+
+int
+qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
+ enum qat_service_type service);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 8ab95ac43..e448dc859 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -16,6 +16,7 @@
#include "qat_sym.h"
#include "qat_qp.h"
#include "adf_transport_access_macros.h"
+#include "qat_device.h"
#define BYTE_LENGTH 8
/* bpi is only used for partial blocks of DES and AES
@@ -23,12 +24,6 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-#define ADF_SYM_TX_RING_DESC_SIZE 128
-#define ADF_SYM_RX_RING_DESC_SIZE 32
-#define ADF_SYM_TX_QUEUE_STARTOFF 2
-/* Offset from bundle start to 1st Sym Tx queue */
-#define ADF_SYM_RX_QUEUE_STARTOFF 10
-
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -805,12 +800,11 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
}
-
-
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
queue_pair_id, dev->data->dev_id);
+
return qat_qp_release((struct qat_qp **)
&(dev->data->queue_pairs[queue_pair_id]));
}
@@ -823,9 +817,14 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
int ret = 0;
uint32_t i;
struct qat_qp_config qat_qp_conf;
+
struct qat_qp **qp_addr =
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
struct qat_pmd_private *qat_private = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[qat_private->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+ const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
/* If qp is already in use free ring memory and qp metadata. */
if (*qp_addr != NULL) {
@@ -833,19 +832,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
if (ret < 0)
return ret;
}
- if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
+ if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
return -EINVAL;
}
- qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
- qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
- qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
- qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.hw = qp_hw_data;
qat_qp_conf.build_request = qat_sym_build_request;
qat_qp_conf.process_response = qat_sym_process_response;
qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
@@ -876,5 +868,4 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
}
return ret;
-
}
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 17/30] crypto/qat: rename sgl related objects
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (15 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 16/30] crypto/qat: create data structures to support different generations Fiona Trahe
@ 2018-04-06 18:51 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 18/30] crypto/qat: move sgl related element to appropriate files Fiona Trahe
` (46 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:51 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Change SGL (Scatter-Gather List) related structs and member names
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
---
drivers/crypto/qat/qat_common.h | 10 +++++-----
drivers/crypto/qat/qat_sym.c | 28 ++++++++++++++--------------
2 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index c3e7bd9a7..193639550 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -32,22 +32,22 @@ enum qat_service_type {
#define QAT_MAX_SERVICES (QAT_SERVICE_INVALID)
/**< Common struct for scatter-gather list operations */
-struct qat_alg_buf {
+struct qat_flat_buf {
uint32_t len;
uint32_t resrvd;
uint64_t addr;
} __rte_packed;
-struct qat_alg_buf_list {
+struct qat_sgl {
uint64_t resrvd;
uint32_t num_bufs;
uint32_t num_mapped_bufs;
- struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
+ struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
struct qat_sym_op_cookie {
- struct qat_alg_buf_list qat_sgl_list_src;
- struct qat_alg_buf_list qat_sgl_list_dst;
+ struct qat_sgl qat_sgl_src;
+ struct qat_sgl qat_sgl_dst;
phys_addr_t qat_sgl_src_phys_addr;
phys_addr_t qat_sgl_dst_phys_addr;
};
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index e448dc859..a9beff064 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -250,20 +250,20 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
static inline int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
- struct qat_alg_buf_list *list, uint32_t data_len)
+ struct qat_sgl *list, uint32_t data_len)
{
int nr = 1;
uint32_t buf_len = rte_pktmbuf_iova(buf) -
buff_start + rte_pktmbuf_data_len(buf);
- list->bufers[0].addr = buff_start;
- list->bufers[0].resrvd = 0;
- list->bufers[0].len = buf_len;
+ list->buffers[0].addr = buff_start;
+ list->buffers[0].resrvd = 0;
+ list->buffers[0].len = buf_len;
if (data_len <= buf_len) {
list->num_bufs = nr;
- list->bufers[0].len = data_len;
+ list->buffers[0].len = data_len;
return 0;
}
@@ -276,15 +276,15 @@ qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
return -EINVAL;
}
- list->bufers[nr].len = rte_pktmbuf_data_len(buf);
- list->bufers[nr].resrvd = 0;
- list->bufers[nr].addr = rte_pktmbuf_iova(buf);
+ list->buffers[nr].len = rte_pktmbuf_data_len(buf);
+ list->buffers[nr].resrvd = 0;
+ list->buffers[nr].addr = rte_pktmbuf_iova(buf);
- buf_len += list->bufers[nr].len;
+ buf_len += list->buffers[nr].len;
buf = buf->next;
if (buf_len > data_len) {
- list->bufers[nr].len -=
+ list->buffers[nr].len -=
buf_len - data_len;
buf = NULL;
}
@@ -695,7 +695,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
ICP_QAT_FW_COMN_PTR_TYPE_SET(qat_req->comn_hdr.comn_req_flags,
QAT_COMN_PTR_TYPE_SGL);
ret = qat_sgl_fill_array(op->sym->m_src, src_buf_start,
- &cookie->qat_sgl_list_src,
+ &cookie->qat_sgl_src,
qat_req->comn_mid.src_length);
if (ret) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot fill sgl array");
@@ -709,7 +709,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
else {
ret = qat_sgl_fill_array(op->sym->m_dst,
dst_buf_start,
- &cookie->qat_sgl_list_dst,
+ &cookie->qat_sgl_dst,
qat_req->comn_mid.dst_length);
if (ret) {
@@ -859,12 +859,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
sql_cookie->qat_sgl_src_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_src);
+ qat_sgl_src);
sql_cookie->qat_sgl_dst_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_dst);
+ qat_sgl_dst);
}
return ret;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 18/30] crypto/qat: move sgl related element to appropriate files
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (16 preceding siblings ...)
2018-04-06 18:51 ` [PATCH 17/30] crypto/qat: rename sgl related objects Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 19/30] crypto/qat: add QAT PCI device struct Fiona Trahe
` (45 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Move SGL (Scatter-Gather List) related functions to common file
Move qat_sym_op_cookie struct to sym header file
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 1 +
drivers/crypto/qat/meson.build | 1 +
drivers/crypto/qat/qat_common.c | 53 +++++++++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_common.h | 11 ++++-----
drivers/crypto/qat/qat_sym.c | 47 ------------------------------------
drivers/crypto/qat/qat_sym.h | 7 ++++++
6 files changed, 67 insertions(+), 53 deletions(-)
create mode 100644 drivers/crypto/qat/qat_common.c
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 8cb802b9d..902c47ff4 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_common.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
# export include files
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index e596006da..12910c377 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -7,6 +7,7 @@ if not dep.found()
endif
sources = files('qat_sym.c', 'qat_qp.c',
'qat_sym_session.c',
+ 'qat_common.c',
'rte_qat_cryptodev.c',
'qat_device.c')
includes += include_directories('qat_adf')
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
new file mode 100644
index 000000000..a8865904f
--- /dev/null
+++ b/drivers/crypto/qat/qat_common.c
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_common.h"
+#include "qat_logs.h"
+
+int
+qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
+ struct qat_sgl *list, uint32_t data_len)
+{
+ int nr = 1;
+
+ uint32_t buf_len = rte_pktmbuf_iova(buf) -
+ buf_start + rte_pktmbuf_data_len(buf);
+
+ list->buffers[0].addr = buf_start;
+ list->buffers[0].resrvd = 0;
+ list->buffers[0].len = buf_len;
+
+ if (data_len <= buf_len) {
+ list->num_bufs = nr;
+ list->buffers[0].len = data_len;
+ return 0;
+ }
+
+ buf = buf->next;
+ while (buf) {
+ if (unlikely(nr == QAT_SGL_MAX_NUMBER)) {
+ PMD_DRV_LOG(ERR,
+ "QAT PMD exceeded size of QAT SGL entry(%u)",
+ QAT_SGL_MAX_NUMBER);
+ return -EINVAL;
+ }
+
+ list->buffers[nr].len = rte_pktmbuf_data_len(buf);
+ list->buffers[nr].resrvd = 0;
+ list->buffers[nr].addr = rte_pktmbuf_iova(buf);
+
+ buf_len += list->buffers[nr].len;
+ buf = buf->next;
+
+ if (buf_len > data_len) {
+ list->buffers[nr].len -=
+ buf_len - data_len;
+ buf = NULL;
+ }
+ ++nr;
+ }
+ list->num_bufs = nr;
+
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 193639550..77ffc8f72 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -6,6 +6,8 @@
#include <stdint.h>
+#include <rte_mbuf.h>
+
/**< Intel(R) QAT Symmetric Crypto PMD device name */
#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
@@ -45,11 +47,8 @@ struct qat_sgl {
struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
-struct qat_sym_op_cookie {
- struct qat_sgl qat_sgl_src;
- struct qat_sgl qat_sgl_dst;
- phys_addr_t qat_sgl_src_phys_addr;
- phys_addr_t qat_sgl_dst_phys_addr;
-};
+int
+qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
+ struct qat_sgl *list, uint32_t data_len);
#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index a9beff064..b74dfa634 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -248,53 +248,6 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
-static inline int
-qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
- struct qat_sgl *list, uint32_t data_len)
-{
- int nr = 1;
-
- uint32_t buf_len = rte_pktmbuf_iova(buf) -
- buff_start + rte_pktmbuf_data_len(buf);
-
- list->buffers[0].addr = buff_start;
- list->buffers[0].resrvd = 0;
- list->buffers[0].len = buf_len;
-
- if (data_len <= buf_len) {
- list->num_bufs = nr;
- list->buffers[0].len = data_len;
- return 0;
- }
-
- buf = buf->next;
- while (buf) {
- if (unlikely(nr == QAT_SGL_MAX_NUMBER)) {
- PMD_DRV_LOG(ERR, "QAT PMD exceeded size of QAT SGL"
- " entry(%u)",
- QAT_SGL_MAX_NUMBER);
- return -EINVAL;
- }
-
- list->buffers[nr].len = rte_pktmbuf_data_len(buf);
- list->buffers[nr].resrvd = 0;
- list->buffers[nr].addr = rte_pktmbuf_iova(buf);
-
- buf_len += list->buffers[nr].len;
- buf = buf->next;
-
- if (buf_len > data_len) {
- list->buffers[nr].len -=
- buf_len - data_len;
- buf = NULL;
- }
- ++nr;
- }
- list->num_bufs = nr;
-
- return 0;
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index b92ec72de..37bec3ce3 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -22,6 +22,13 @@
struct qat_sym_session;
+struct qat_sym_op_cookie {
+ struct qat_sgl qat_sgl_src;
+ struct qat_sgl qat_sgl_dst;
+ phys_addr_t qat_sgl_src_phys_addr;
+ phys_addr_t qat_sgl_dst_phys_addr;
+};
+
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 19/30] crypto/qat: add QAT PCI device struct
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (17 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 18/30] crypto/qat: move sgl related element to appropriate files Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 20/30] crypto/qat: separate the name used for PCI reg from crypto name Fiona Trahe
` (44 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
- Added struct qat_pci_device to use internally in QAT PMD
to avoid dependencies on rte_cryptodev or rte_compressdev
- Added a global array of these
- Restructured probe/release to separate QAT common init/clear
from crypto pmd create/destroy.
- In QAT common part allocated a qat_pci_device and populated it
- Removed meaningless check in probe for RTE_PROC_PRIMARY
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 127 ++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_device.h | 60 ++++++++++++++-
drivers/crypto/qat/rte_qat_cryptodev.c | 136 ++++++++++++++++++++++++---------
3 files changed, 282 insertions(+), 41 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index cdf4f7058..75af1e8bc 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -20,6 +20,10 @@ struct qat_gen_hw_data qp_gen_config[] = {
},
};
+
+static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
+static int qat_nb_pci_devices;
+
int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
{
@@ -72,3 +76,126 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
info->pci_dev = RTE_DEV_TO_PCI(dev->device);
}
}
+
+
+static struct qat_pci_device *
+qat_pci_get_dev(uint8_t dev_id)
+{
+ return &qat_pci_devices[dev_id];
+}
+static struct qat_pci_device *
+qat_pci_get_named_dev(const char *name)
+{
+ struct qat_pci_device *dev;
+ unsigned int i;
+
+ if (name == NULL)
+ return NULL;
+
+ for (i = 0; i < QAT_MAX_PCI_DEVICES; i++) {
+ dev = &qat_pci_devices[i];
+
+ if ((dev->attached == QAT_ATTACHED) &&
+ (strcmp(dev->name, name) == 0))
+ return dev;
+ }
+
+ return NULL;
+}
+
+static uint8_t
+qat_pci_find_free_device_index(void)
+{
+ uint8_t dev_id;
+
+ for (dev_id = 0; dev_id < QAT_MAX_PCI_DEVICES; dev_id++) {
+ if (qat_pci_devices[dev_id].attached == QAT_DETACHED)
+ break;
+ }
+ return dev_id;
+}
+
+struct qat_pci_device *
+qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev)
+{
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+
+ return qat_pci_get_named_dev(name);
+}
+
+struct qat_pci_device *
+qat_pci_device_allocate(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_dev;
+ uint8_t qat_dev_id;
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+
+ if (qat_pci_get_named_dev(name) != NULL) {
+ PMD_DRV_LOG(ERR, "QAT device with name %s already allocated!",
+ name);
+ return NULL;
+ }
+
+ qat_dev_id = qat_pci_find_free_device_index();
+ if (qat_dev_id == QAT_MAX_PCI_DEVICES) {
+ PMD_DRV_LOG(ERR, "Reached maximum number of QAT devices");
+ return NULL;
+ }
+
+ qat_dev = qat_pci_get_dev(qat_dev_id);
+ snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
+ qat_dev->pci_dev = pci_dev;
+ switch (qat_dev->pci_dev->id.device_id) {
+ case 0x0443:
+ qat_dev->qat_dev_gen = QAT_GEN1;
+ break;
+ case 0x37c9:
+ case 0x19e3:
+ case 0x6f55:
+ qat_dev->qat_dev_gen = QAT_GEN2;
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "Invalid dev_id, can't determine generation");
+ return NULL;
+ }
+
+ rte_spinlock_init(&qat_dev->arb_csr_lock);
+
+ qat_dev->attached = QAT_ATTACHED;
+
+ qat_nb_pci_devices++;
+
+ PMD_DRV_LOG(DEBUG, "QAT device %d allocated, total QATs %d",
+ qat_dev_id, qat_nb_pci_devices);
+
+ return qat_dev;
+}
+
+int
+qat_pci_device_release(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_dev;
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ if (pci_dev == NULL)
+ return -EINVAL;
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ qat_dev = qat_pci_get_named_dev(name);
+ if (qat_dev != NULL) {
+
+ /* Check that there are no service devs still on pci device */
+ if (qat_dev->sym_dev != NULL)
+ return -EBUSY;
+
+ qat_dev->attached = QAT_DETACHED;
+ qat_nb_pci_devices--;
+ }
+ PMD_DRV_LOG(DEBUG, "QAT device %s released, total QATs %d",
+ name, qat_nb_pci_devices);
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 0983e3c2e..d83ad632c 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,15 +11,58 @@
#include "adf_transport_access_macros.h"
#include "qat_qp.h"
+
+#define QAT_DETACHED (0)
+#define QAT_ATTACHED (1)
+
+#define QAT_MAX_PCI_DEVICES 48
+#define QAT_DEV_NAME_MAX_LEN 64
+
+
extern uint8_t cryptodev_qat_driver_id;
extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-/** private data structure for each QAT device.
- * In this context a QAT device is a device offering only one service,
- * so there can be more than 1 device on a pci_dev (VF),
- * one for symmetric crypto, one for compression
+/*
+ * This struct holds all the data about a QAT pci device
+ * including data about all services it supports.
+ * It contains
+ * - hw_data
+ * - config data
+ * - runtime data
+ */
+struct qat_pci_device {
+
+ /* data used by all services */
+ char name[QAT_DEV_NAME_MAX_LEN];
+ /**< Name of qat pci device */
+ struct rte_pci_device *pci_dev;
+ /**< PCI information. */
+ enum qat_device_gen qat_dev_gen;
+ /**< QAT device generation */
+ rte_spinlock_t arb_csr_lock;
+ /* protects accesses to the arbiter CSR */
+ __extension__
+ uint8_t attached : 1;
+ /**< Flag indicating the device is attached */
+
+ /* data relating to symmetric crypto service */
+ struct qat_pmd_private *sym_dev;
+ /**< link back to cryptodev private data */
+ unsigned int max_nb_sym_queue_pairs;
+ /**< Max number of queue pairs supported by device */
+
+ /* data relating to compression service */
+
+ /* data relating to asymmetric crypto service */
+
+};
+
+/** private data structure for a QAT device.
+ * This QAT device is a device offering only symmetric crypto service,
+ * there can be one of these on each qat_pci_device (VF),
+ * in future there may also be private data structures for other services.
*/
struct qat_pmd_private {
unsigned int max_nb_queue_pairs;
@@ -34,6 +77,8 @@ struct qat_pmd_private {
/**< PCI information. */
uint8_t dev_id;
/**< Device ID for this instance */
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
};
struct qat_gen_hw_data {
@@ -51,4 +96,11 @@ int qat_dev_close(struct rte_cryptodev *dev);
void qat_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
+struct qat_pci_device *
+qat_pci_device_allocate(struct rte_pci_device *pci_dev);
+int
+qat_pci_device_release(struct rte_pci_device *pci_dev);
+struct qat_pci_device *
+qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev);
+
#endif /* _QAT_DEVICE_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index fe19b18b6..bb1381297 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -69,17 +69,28 @@ static const struct rte_pci_id pci_id_qat_map[] = {
{.device_id = 0},
};
+
+
static int
-crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
- struct rte_cryptodev_pmd_init_params *init_params)
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
{
+ struct rte_cryptodev_pmd_init_params init_params = {
+ .name = "",
+ .socket_id = qat_pci_dev->pci_dev->device.numa_node,
+ .private_data_size = sizeof(struct qat_pmd_private),
+ .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
+ };
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct rte_cryptodev *cryptodev;
struct qat_pmd_private *internals;
- PMD_INIT_FUNC_TRACE();
+ snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
+ qat_pci_dev->name, "sym");
+ PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+
+ cryptodev = rte_cryptodev_pmd_create(name,
+ &qat_pci_dev->pci_dev->device, &init_params);
- cryptodev = rte_cryptodev_pmd_create(name, &pci_dev->device,
- init_params);
if (cryptodev == NULL)
return -ENODEV;
@@ -95,7 +106,10 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
internals = cryptodev->data->dev_private;
- internals->max_nb_sessions = init_params->max_nb_sessions;
+ internals->qat_dev = qat_pci_dev;
+ qat_pci_dev->sym_dev = internals;
+
+ internals->max_nb_sessions = init_params.max_nb_sessions;
internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
internals->dev_id = cryptodev->data->dev_id;
switch (internals->pci_dev->id.device_id) {
@@ -111,68 +125,116 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
break;
default:
PMD_DRV_LOG(ERR,
- "Invalid dev_id, can't determine capabilities");
+ "Invalid dev_id, can't determine capabilities");
break;
}
- /*
- * For secondary processes, we don't initialise any further as primary
- * has already done this work. Only check we don't need a different
- * RX function
- */
- if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
- PMD_DRV_LOG(DEBUG, "Device already initialised by primary process");
return 0;
- }
+}
+
+static int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev *cryptodev;
+
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+ if (qat_pci_dev->sym_dev == NULL)
+ return 0;
+
+ /* free crypto device */
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->dev_id);
+ rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_dev = NULL;
return 0;
}
-static int crypto_qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+static int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+static int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+static int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+static int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
struct rte_pci_device *pci_dev)
{
- struct rte_cryptodev_pmd_init_params init_params = {
- .name = "",
- .socket_id = pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_pmd_private),
- .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
- };
- char name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ int ret = 0;
+ struct qat_pci_device *qat_pci_dev;
PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
pci_dev->addr.bus,
pci_dev->addr.devid,
pci_dev->addr.function);
- rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ qat_pci_dev = qat_pci_device_allocate(pci_dev);
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+
+ ret = qat_sym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_comp_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_asym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ return 0;
+
+error_out:
+ qat_asym_dev_destroy(qat_pci_dev);
+ qat_comp_dev_destroy(qat_pci_dev);
+ qat_sym_dev_destroy(qat_pci_dev);
+ qat_pci_device_release(pci_dev);
+ return ret;
- return crypto_qat_create(name, pci_dev, &init_params);
}
-static int crypto_qat_pci_remove(struct rte_pci_device *pci_dev)
+static int qat_pci_remove(struct rte_pci_device *pci_dev)
{
- struct rte_cryptodev *cryptodev;
- char cryptodev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct qat_pci_device *qat_pci_dev;
if (pci_dev == NULL)
return -EINVAL;
- rte_pci_device_name(&pci_dev->addr, cryptodev_name,
- sizeof(cryptodev_name));
+ qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
+ if (qat_pci_dev == NULL)
+ return 0;
- cryptodev = rte_cryptodev_pmd_get_named_dev(cryptodev_name);
- if (cryptodev == NULL)
- return -ENODEV;
+ qat_asym_dev_destroy(qat_pci_dev);
+ qat_comp_dev_destroy(qat_pci_dev);
+ qat_sym_dev_destroy(qat_pci_dev);
+ return qat_pci_device_release(pci_dev);
- /* free crypto device */
- return rte_cryptodev_pmd_destroy(cryptodev);
}
static struct rte_pci_driver rte_qat_pmd = {
.id_table = pci_id_qat_map,
.drv_flags = RTE_PCI_DRV_NEED_MAPPING,
- .probe = crypto_qat_pci_probe,
- .remove = crypto_qat_pci_remove
+ .probe = qat_pci_probe,
+ .remove = qat_pci_remove
};
static struct cryptodev_driver qat_crypto_drv;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 20/30] crypto/qat: separate the name used for PCI reg from crypto name
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (18 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 19/30] crypto/qat: add QAT PCI device struct Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 21/30] crypto/qat: move to using new device structure Fiona Trahe
` (43 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
The QAT PMD used to register with PCI using the name "crypto_qat".
Keep this name for the driver registered with cryptodev
and use a more generic name "qat" for the PCI registration.
This paves the way for the PCI device to host other services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 2 ++
drivers/crypto/qat/rte_qat_cryptodev.c | 25 ++++++++++++++++++-------
2 files changed, 20 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 77ffc8f72..63e5569ea 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -11,6 +11,8 @@
/**< Intel(R) QAT Symmetric Crypto PMD device name */
#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+/**< Intel(R) QAT device name for PCI registration */
+#define QAT_PCI_NAME qat
/*
* Maximum number of SGL entries
*/
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index bb1381297..acc0b924a 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -230,16 +230,27 @@ static int qat_pci_remove(struct rte_pci_device *pci_dev)
}
+
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+static struct cryptodev_driver qat_crypto_drv;
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+ cryptodev_qat_driver_id);
+
static struct rte_pci_driver rte_qat_pmd = {
.id_table = pci_id_qat_map,
.drv_flags = RTE_PCI_DRV_NEED_MAPPING,
.probe = qat_pci_probe,
.remove = qat_pci_remove
};
-
-static struct cryptodev_driver qat_crypto_drv;
-
-RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_QAT_SYM_PMD, rte_qat_pmd);
-RTE_PMD_REGISTER_PCI_TABLE(CRYPTODEV_NAME_QAT_SYM_PMD, pci_id_qat_map);
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, rte_qat_pmd.driver,
- cryptodev_qat_driver_id);
+RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 21/30] crypto/qat: move to using new device structure
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (19 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 20/30] crypto/qat: separate the name used for PCI reg from crypto name Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 22/30] crypto/qat: use common stats structures Fiona Trahe
` (42 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Struct qat_pmd_private held the data needed by cryptodev, common code
now gets most data from struct qat_pci_device instead. qat_pmd_private
is trimmed to hold only sym crypto data and renamed qat_sym_private
to reflect its usage.
Specifically
- remove max_nb_queue_pairs from qat_pmd_private, get from qp_hw_data
- remove max_nb_sesssions from qat_pmd_private as not needed.
- remove qat_gen from qat_pmd_private, get from qat_pci_device instead.
- use qat_pci_device throughout common code instead of qat_pmd_private
- rename qat_pmd_private to qat_sym_dev_private - this now holds only
sym-specific data for the cryptodev API
- extend pci device name to <bdf>_qat for clarity, was just <bdf>
- update qp mem and cookiepool names to reflect the appropriate device,
service and qp.
- rename qat_dev_info_get() to qat_sym_dev_info_get() as mostly sym,
not enough common info to warrant a generic fn.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 17 +++++++++-------
drivers/crypto/qat/qat_device.h | 37 ++++++++++++++--------------------
drivers/crypto/qat/qat_qp.c | 28 +++++++++++++------------
drivers/crypto/qat/qat_qp.h | 4 ++--
drivers/crypto/qat/qat_sym.c | 11 +++++-----
drivers/crypto/qat/qat_sym.h | 8 --------
drivers/crypto/qat/qat_sym_session.c | 8 ++++----
drivers/crypto/qat/rte_qat_cryptodev.c | 32 ++++++++++++++---------------
8 files changed, 66 insertions(+), 79 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 75af1e8bc..8ad3162e1 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -57,12 +57,12 @@ int qat_dev_close(struct rte_cryptodev *dev)
return 0;
}
-void qat_dev_info_get(struct rte_cryptodev *dev,
+void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info)
{
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev_gen]
+ qp_gen_config[internals->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
@@ -71,7 +71,7 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
info->feature_flags = dev->feature_flags;
info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = internals->max_nb_sessions;
+ info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
info->driver_id = cryptodev_qat_driver_id;
info->pci_dev = RTE_DEV_TO_PCI(dev->device);
}
@@ -83,6 +83,7 @@ qat_pci_get_dev(uint8_t dev_id)
{
return &qat_pci_devices[dev_id];
}
+
static struct qat_pci_device *
qat_pci_get_named_dev(const char *name)
{
@@ -133,7 +134,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
char name[QAT_DEV_NAME_MAX_LEN];
rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
-
+ snprintf(name+strlen(name), QAT_DEV_NAME_MAX_LEN-strlen(name), "_qat");
if (qat_pci_get_named_dev(name) != NULL) {
PMD_DRV_LOG(ERR, "QAT device with name %s already allocated!",
name);
@@ -148,6 +149,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
qat_dev = qat_pci_get_dev(qat_dev_id);
snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
+ qat_dev->qat_dev_id = qat_dev_id;
qat_dev->pci_dev = pci_dev;
switch (qat_dev->pci_dev->id.device_id) {
case 0x0443:
@@ -169,8 +171,8 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
qat_nb_pci_devices++;
- PMD_DRV_LOG(DEBUG, "QAT device %d allocated, total QATs %d",
- qat_dev_id, qat_nb_pci_devices);
+ PMD_DRV_LOG(DEBUG, "QAT device %d allocated, name %s, total QATs %d",
+ qat_dev->qat_dev_id, qat_dev->name, qat_nb_pci_devices);
return qat_dev;
}
@@ -185,6 +187,7 @@ qat_pci_device_release(struct rte_pci_device *pci_dev)
return -EINVAL;
rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ snprintf(name+strlen(name), QAT_DEV_NAME_MAX_LEN-strlen(name), "_qat");
qat_dev = qat_pci_get_named_dev(name);
if (qat_dev != NULL) {
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index d83ad632c..855bf6c1c 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -32,30 +32,31 @@ extern int qat_sym_qp_release(struct rte_cryptodev *dev,
* - config data
* - runtime data
*/
+struct qat_sym_dev_private;
struct qat_pci_device {
- /* data used by all services */
+ /* Data used by all services */
char name[QAT_DEV_NAME_MAX_LEN];
/**< Name of qat pci device */
+ uint8_t qat_dev_id;
+ /**< Device instance for this qat pci device */
struct rte_pci_device *pci_dev;
/**< PCI information. */
enum qat_device_gen qat_dev_gen;
/**< QAT device generation */
rte_spinlock_t arb_csr_lock;
- /* protects accesses to the arbiter CSR */
+ /**< lock to protect accesses to the arbiter CSR */
__extension__
uint8_t attached : 1;
/**< Flag indicating the device is attached */
- /* data relating to symmetric crypto service */
- struct qat_pmd_private *sym_dev;
+ /* Data relating to symmetric crypto service */
+ struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
- unsigned int max_nb_sym_queue_pairs;
- /**< Max number of queue pairs supported by device */
- /* data relating to compression service */
+ /* Data relating to compression service */
- /* data relating to asymmetric crypto service */
+ /* Data relating to asymmetric crypto service */
};
@@ -64,21 +65,13 @@ struct qat_pci_device {
* there can be one of these on each qat_pci_device (VF),
* in future there may also be private data structures for other services.
*/
-struct qat_pmd_private {
- unsigned int max_nb_queue_pairs;
- /**< Max number of queue pairs supported by device */
- unsigned int max_nb_sessions;
- /**< Max number of sessions supported by device */
- enum qat_device_gen qat_dev_gen;
- /**< QAT device generation */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
- /* QAT device capabilities */
- struct rte_pci_device *pci_dev;
- /**< PCI information. */
- uint8_t dev_id;
- /**< Device ID for this instance */
+struct qat_sym_dev_private {
struct qat_pci_device *qat_dev;
/**< The qat pci device hosting the service */
+ uint8_t sym_dev_id;
+ /**< Device instance for this rte_cryptodev */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device symmetric crypto capabilities */
};
struct qat_gen_hw_data {
@@ -93,7 +86,7 @@ int qat_dev_config(struct rte_cryptodev *dev,
int qat_dev_start(struct rte_cryptodev *dev);
void qat_dev_stop(struct rte_cryptodev *dev);
int qat_dev_close(struct rte_cryptodev *dev);
-void qat_dev_info_get(struct rte_cryptodev *dev,
+void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
struct qat_pci_device *
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index a59735344..4ad4a5326 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -102,7 +102,7 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
-static int qat_queue_create(struct qat_pmd_private *qat_dev,
+static int qat_queue_create(struct qat_pci_device *qat_dev,
struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
@@ -172,7 +172,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
memzone_flags, queue_size);
}
-int qat_qp_setup(struct qat_pmd_private *qat_dev,
+int qat_qp_setup(struct qat_pci_device *qat_dev,
struct qat_qp **qp_addr,
uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
@@ -182,8 +182,8 @@ int qat_qp_setup(struct qat_pmd_private *qat_dev,
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
- PMD_DRV_LOG(DEBUG, "Setup qp %u on device %d gen %d",
- queue_pair_id, qat_dev->dev_id, qat_dev->qat_dev_gen);
+ PMD_DRV_LOG(DEBUG, "Setup qp %u on qat pci device %d gen %d",
+ queue_pair_id, qat_dev->qat_dev_id, qat_dev->qat_dev_gen);
if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
(qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
@@ -236,10 +236,12 @@ int qat_qp_setup(struct qat_pmd_private *qat_dev,
adf_configure_queues(qp);
adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
- snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
- pci_dev->driver->driver.name, qat_qp_conf->service_str,
- qat_dev->dev_id, queue_pair_id);
+ snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE,
+ "%s%d_cookies_%s_qp%hu",
+ pci_dev->driver->driver.name, qat_dev->qat_dev_id,
+ qat_qp_conf->service_str, queue_pair_id);
+ PMD_DRV_LOG(DEBUG, "cookiepool: %s", op_cookie_pool_name);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
@@ -287,8 +289,8 @@ int qat_qp_release(struct qat_qp **qp_addr)
return 0;
}
- PMD_DRV_LOG(DEBUG, "Free qp %u on device %d",
- qp->queue_pair_id, qp->qat_dev->dev_id);
+ PMD_DRV_LOG(DEBUG, "Free qp %u on qat_pci device %d",
+ qp->queue_pair_id, qp->qat_dev->qat_dev_id);
/* Don't free memory if there are still responses to be processed */
if (qp->inflights16 == 0) {
@@ -340,7 +342,7 @@ static void qat_queue_delete(struct qat_queue *queue)
}
static int
-qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
+qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
@@ -365,9 +367,9 @@ qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
* Allocate a memzone for the queue - create a unique name.
*/
snprintf(queue->memz_name, sizeof(queue->memz_name),
- "%s_%s_%s_%d_%d_%d",
- pci_dev->driver->driver.name, qp_conf->service_str,
- "qp_mem", qat_dev->dev_id,
+ "%s_%d_%s_%s_%d_%d",
+ pci_dev->driver->driver.name, qat_dev->qat_dev_id,
+ qp_conf->service_str, "qp_mem",
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
qp_conf->socket_id);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index f808e16a5..d482d5732 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -85,7 +85,7 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
process_response_t process_response;
- struct qat_pmd_private *qat_dev;
+ struct qat_pci_device *qat_dev;
/**< qat device this qp is on */
} __rte_cache_aligned;
@@ -101,7 +101,7 @@ int
qat_qp_release(struct qat_qp **qp_addr);
int
-qat_qp_setup(struct qat_pmd_private *qat_dev,
+qat_qp_setup(struct qat_pci_device *qat_dev,
struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index b74dfa634..e77fbe4c4 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -204,7 +204,7 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
}
-int
+static int
qat_sym_process_response(void **op, uint8_t *resp,
__rte_unused void *op_cookie,
__rte_unused enum qat_device_gen qat_dev_gen)
@@ -293,8 +293,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-
-int
+static int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen)
{
@@ -773,9 +772,9 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
struct qat_qp **qp_addr =
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
- struct qat_pmd_private *qat_private = dev->data->dev_private;
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev_gen]
+ qp_gen_config[qat_private->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
@@ -798,7 +797,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.socket_id = socket_id;
qat_qp_conf.service_str = "sym";
- ret = qat_qp_setup(qat_private, qp_addr, qp_id, &qat_qp_conf);
+ ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
if (ret != 0)
return ret;
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 37bec3ce3..78b40e378 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -29,14 +29,6 @@ struct qat_sym_op_cookie {
phys_addr_t qat_sgl_dst_phys_addr;
};
-int
-qat_sym_build_request(void *in_op, uint8_t *out_msg,
- void *op_cookie, enum qat_device_gen qat_dev_gen);
-
-int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie, enum qat_device_gen qat_dev_gen);
-
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index d71d7fcd1..d8a5d883f 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -64,7 +64,7 @@ bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
static int
qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
- struct qat_pmd_private *internals) {
+ struct qat_sym_dev_private *internals) {
int i = 0;
const struct rte_cryptodev_capabilities *capability;
@@ -84,7 +84,7 @@ qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
static int
qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
- struct qat_pmd_private *internals) {
+ struct qat_sym_dev_private *internals) {
int i = 0;
const struct rte_cryptodev_capabilities *capability;
@@ -199,7 +199,7 @@ qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_sym_session *session)
{
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
struct rte_crypto_cipher_xform *cipher_xform = NULL;
int ret;
@@ -493,7 +493,7 @@ qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct qat_sym_session *session)
{
struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
uint8_t *key_data = auth_xform->key.data;
uint8_t key_length = auth_xform->key.length;
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index acc0b924a..6d265d5e3 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -33,7 +33,7 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
.dev_start = qat_dev_start,
.dev_stop = qat_dev_stop,
.dev_close = qat_dev_close,
- .dev_infos_get = qat_dev_info_get,
+ .dev_infos_get = qat_sym_dev_info_get,
.stats_get = qat_sym_stats_get,
.stats_reset = qat_sym_stats_reset,
@@ -77,12 +77,12 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
struct rte_cryptodev_pmd_init_params init_params = {
.name = "",
.socket_id = qat_pci_dev->pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_pmd_private),
+ .private_data_size = sizeof(struct qat_sym_dev_private),
.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
};
char name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct rte_cryptodev *cryptodev;
- struct qat_pmd_private *internals;
+ struct qat_sym_dev_private *internals;
snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
qat_pci_dev->name, "sym");
@@ -109,27 +109,25 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
internals->qat_dev = qat_pci_dev;
qat_pci_dev->sym_dev = internals;
- internals->max_nb_sessions = init_params.max_nb_sessions;
- internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
- internals->dev_id = cryptodev->data->dev_id;
- switch (internals->pci_dev->id.device_id) {
- case 0x0443:
- internals->qat_dev_gen = QAT_GEN1;
+ internals->sym_dev_id = cryptodev->data->dev_id;
+ switch (qat_pci_dev->qat_dev_gen) {
+ case QAT_GEN1:
internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
break;
- case 0x37c9:
- case 0x19e3:
- case 0x6f55:
- internals->qat_dev_gen = QAT_GEN2;
+ case QAT_GEN2:
internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
break;
default:
- PMD_DRV_LOG(ERR,
- "Invalid dev_id, can't determine capabilities");
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ PMD_DRV_LOG(DEBUG,
+ "QAT gen %d capabilities unknown, default to GEN2",
+ qat_pci_dev->qat_dev_gen);
break;
}
- return 0;
+ PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
+ name, internals->sym_dev_id);
+ return 0;
}
static int
@@ -143,7 +141,7 @@ qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
return 0;
/* free crypto device */
- cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->dev_id);
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
rte_cryptodev_pmd_destroy(cryptodev);
qat_pci_dev->sym_dev = NULL;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 22/30] crypto/qat: use common stats structures
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (20 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 21/30] crypto/qat: move to using new device structure Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 23/30] crypto/qat: rename functions which depend on cryptodev Fiona Trahe
` (41 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Split qat_sym_stats_get/reset into 2 functions, a wrapper function calling
a new qat_stats_get/reset function which can be called per service.
Remove cryptodev stats struct from qat_qp, replace with qat_common_stats.
Add links for qat_qp into qat_pci_device using an array per service to
avoid need for a lock and so qp_id for the service can be used as index.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 13 +++++++
drivers/crypto/qat/qat_device.c | 1 +
drivers/crypto/qat/qat_device.h | 3 ++
drivers/crypto/qat/qat_qp.h | 2 +-
drivers/crypto/qat/qat_sym.c | 86 ++++++++++++++++++++++++++++++++++-------
5 files changed, 91 insertions(+), 14 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 63e5569ea..fcf5c4c09 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -49,6 +49,19 @@ struct qat_sgl {
struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
+/** Common, i.e. not service-specific, statistics */
+struct qat_common_stats {
+ uint64_t enqueued_count;
+ /**< Count of all operations enqueued */
+ uint64_t dequeued_count;
+ /**< Count of all operations dequeued */
+
+ uint64_t enqueue_err_count;
+ /**< Total error count on operations enqueued */
+ uint64_t dequeue_err_count;
+ /**< Total error count on operations dequeued */
+};
+
int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
struct qat_sgl *list, uint32_t data_len);
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 8ad3162e1..4dfbc84c8 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -148,6 +148,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
}
qat_dev = qat_pci_get_dev(qat_dev_id);
+ memset(qat_dev, 0, sizeof(*qat_dev));
snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
qat_dev->qat_dev_id = qat_dev_id;
qat_dev->pci_dev = pci_dev;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 855bf6c1c..d1615e2a7 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -50,6 +50,9 @@ struct qat_pci_device {
uint8_t attached : 1;
/**< Flag indicating the device is attached */
+ struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_PER_BUNDLE];
+ /**< links to qps set up for each service, index same as on API */
+
/* Data relating to symmetric crypto service */
struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index d482d5732..49d9f29d3 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -78,7 +78,7 @@ struct qat_qp {
uint16_t inflights16;
struct qat_queue tx_q;
struct qat_queue rx_q;
- struct rte_cryptodev_stats stats;
+ struct qat_common_stats stats;
struct rte_mempool *op_cookie_pool;
void **op_cookies;
uint32_t nb_descriptors;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index e77fbe4c4..8007e25d6 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -717,20 +717,24 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
}
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats)
+static void qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service)
{
int i;
- struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
+ struct qat_qp **qp;
- PMD_INIT_FUNC_TRACE();
- if (stats == NULL) {
- PMD_DRV_LOG(ERR, "invalid stats ptr NULL");
+ if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
+ stats, dev, service);
return;
}
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Uninitialised queue pair");
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
continue;
}
@@ -741,22 +745,74 @@ void qat_sym_stats_get(struct rte_cryptodev *dev,
}
}
-void qat_sym_stats_reset(struct rte_cryptodev *dev)
+void qat_sym_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ struct qat_common_stats qat_stats = {0};
+ struct qat_sym_dev_private *qat_priv;
+
+ if (stats == NULL || dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
+ stats->enqueued_count = qat_stats.enqueued_count;
+ stats->dequeued_count = qat_stats.dequeued_count;
+ stats->enqueue_err_count = qat_stats.enqueue_err_count;
+ stats->dequeue_err_count = qat_stats.dequeue_err_count;
+}
+
+static void qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service)
{
int i;
- struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
+ struct qat_qp **qp;
- PMD_INIT_FUNC_TRACE();
- for (i = 0; i < dev->data->nb_queue_pairs; i++)
+ if (dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
+ dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
- PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
+ }
+
+ PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+}
+
+void qat_sym_stats_reset(struct rte_cryptodev *dev)
+{
+ struct qat_sym_dev_private *qat_priv;
+
+ if (dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
+
}
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+
PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
queue_pair_id, dev->data->dev_id);
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
+ = NULL;
+
return qat_qp_release((struct qat_qp **)
&(dev->data->queue_pairs[queue_pair_id]));
}
@@ -801,6 +857,10 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
if (ret != 0)
return ret;
+ /* store a link to the qp in the qat_pci_device */
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
+ = *qp_addr;
+
qp = (struct qat_qp *)*qp_addr;
for (i = 0; i < qp->nb_descriptors; i++) {
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 23/30] crypto/qat: rename functions which depend on cryptodev
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (21 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 22/30] crypto/qat: use common stats structures Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 24/30] crypto/qat: move code into appropriate files Fiona Trahe
` (40 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Rename all device functions which depend on cryptodev structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 8 ++++----
drivers/crypto/qat/qat_device.h | 8 ++++----
drivers/crypto/qat/rte_qat_cryptodev.c | 8 ++++----
3 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 4dfbc84c8..8b2ac5a5f 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -24,25 +24,25 @@ struct qat_gen_hw_data qp_gen_config[] = {
static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
-int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
+int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
{
PMD_INIT_FUNC_TRACE();
return 0;
}
-int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
+int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
{
PMD_INIT_FUNC_TRACE();
return 0;
}
-void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
+void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
{
PMD_INIT_FUNC_TRACE();
}
-int qat_dev_close(struct rte_cryptodev *dev)
+int qat_sym_dev_close(struct rte_cryptodev *dev)
{
int i, ret;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index d1615e2a7..5424a9a94 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -84,11 +84,11 @@ struct qat_gen_hw_data {
extern struct qat_gen_hw_data qp_gen_config[];
-int qat_dev_config(struct rte_cryptodev *dev,
+int qat_sym_dev_config(struct rte_cryptodev *dev,
struct rte_cryptodev_config *config);
-int qat_dev_start(struct rte_cryptodev *dev);
-void qat_dev_stop(struct rte_cryptodev *dev);
-int qat_dev_close(struct rte_cryptodev *dev);
+int qat_sym_dev_start(struct rte_cryptodev *dev);
+void qat_sym_dev_stop(struct rte_cryptodev *dev);
+int qat_sym_dev_close(struct rte_cryptodev *dev);
void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 6d265d5e3..adad0ebe6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -29,10 +29,10 @@ static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
static struct rte_cryptodev_ops crypto_qat_ops = {
/* Device related operations */
- .dev_configure = qat_dev_config,
- .dev_start = qat_dev_start,
- .dev_stop = qat_dev_stop,
- .dev_close = qat_dev_close,
+ .dev_configure = qat_sym_dev_config,
+ .dev_start = qat_sym_dev_start,
+ .dev_stop = qat_sym_dev_stop,
+ .dev_close = qat_sym_dev_close,
.dev_infos_get = qat_sym_dev_info_get,
.stats_get = qat_sym_stats_get,
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 24/30] crypto/qat: move code into appropriate files
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (22 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 23/30] crypto/qat: rename functions which depend on cryptodev Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 25/30] crypto/qat: add lock around csr access and change logic Fiona Trahe
` (39 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Move all code into appropriate files, no actual code changes. Specifically:
- Rename rte_qat_cryptodev.c to qat_sym_pmd.c
- Create qat_sym_pmd.h and populate with fn prototypes for qat_sym_pmd.c
- Create qat_comp_pmd.c/.h and populate with placeholder functions
- Create qat_asym_pmd.c/.h and populate with placeholder functions
- Rename qat_crypto_capabilities.h to qat_sym_capabilities.h
- Move CRYPTODEV_NAME_QAT_SYM_PMD from qat_common.h to qat_sym_pmd.h
- Move qat_sym_dev_private from qat_device.h to qat_sym_pmd.h
- Move prototype for qat_sym_dev_info_get frm qat_device.h 2 qat_sym_pmd.h
- Move all qat_device.c sym dev_ops fns to qat_sym_pmd.c file
- Move all qat_sym.c dev_ops fns to qat_sym_pmd.c file
- Remove unused header file #includes from all files.
- Move pci_id_qat_map, probe/release/register from
rte_qat_cryptodev.c to qat_device.c
- Moved stray comment for bpi_cipher_ctx_init() from qat_sym.c
to qat_sym_session.c
- Changed all files to use SPDX license header
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 4 +-
drivers/crypto/qat/meson.build | 11 +-
drivers/crypto/qat/qat_asym_pmd.c | 17 ++
drivers/crypto/qat/qat_asym_pmd.h | 15 +
drivers/crypto/qat/qat_common.c | 54 ++++
drivers/crypto/qat/qat_common.h | 15 +-
drivers/crypto/qat/qat_comp_pmd.c | 18 ++
drivers/crypto/qat/qat_comp_pmd.h | 29 ++
drivers/crypto/qat/qat_device.c | 140 +++++----
drivers/crypto/qat/qat_device.h | 31 +-
drivers/crypto/qat/qat_qp.c | 2 +-
drivers/crypto/qat/qat_qp.h | 9 +-
drivers/crypto/qat/qat_sym.c | 246 ++--------------
drivers/crypto/qat/qat_sym.h | 28 +-
...rypto_capabilities.h => qat_sym_capabilities.h} | 6 +-
drivers/crypto/qat/qat_sym_pmd.c | 326 +++++++++++++++++++++
drivers/crypto/qat/qat_sym_pmd.h | 39 +++
drivers/crypto/qat/qat_sym_session.c | 17 +-
drivers/crypto/qat/qat_sym_session.h | 2 +-
drivers/crypto/qat/rte_qat_cryptodev.c | 254 ----------------
20 files changed, 664 insertions(+), 599 deletions(-)
create mode 100644 drivers/crypto/qat/qat_asym_pmd.c
create mode 100644 drivers/crypto/qat/qat_asym_pmd.h
create mode 100644 drivers/crypto/qat/qat_comp_pmd.c
create mode 100644 drivers/crypto/qat/qat_comp_pmd.h
rename drivers/crypto/qat/{qat_crypto_capabilities.h => qat_sym_capabilities.h} (99%)
create mode 100644 drivers/crypto/qat/qat_sym_pmd.c
create mode 100644 drivers/crypto/qat/qat_sym_pmd.h
delete mode 100644 drivers/crypto/qat/rte_qat_cryptodev.c
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 902c47ff4..d467683fd 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -26,7 +26,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_common.c
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_asym_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_comp_pmd.c
# export include files
SYMLINK-y-include +=
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 12910c377..e22e08fba 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -5,11 +5,12 @@ dep = dependency('libcrypto', required: false)
if not dep.found()
build = false
endif
-sources = files('qat_sym.c', 'qat_qp.c',
- 'qat_sym_session.c',
- 'qat_common.c',
- 'rte_qat_cryptodev.c',
- 'qat_device.c')
+sources = files('qat_common.c',
+ 'qat_qp.c',
+ 'qat_device.c',
+ 'qat_sym_pmd.c', 'qat_sym.c', 'qat_sym_session.c',
+ 'qat_asym_pmd.c',
+ 'qat_comp_pmd.c')
includes += include_directories('qat_adf')
deps += ['bus_pci']
ext_deps += dep
diff --git a/drivers/crypto/qat/qat_asym_pmd.c b/drivers/crypto/qat/qat_asym_pmd.c
new file mode 100644
index 000000000..8d36300ac
--- /dev/null
+++ b/drivers/crypto/qat/qat_asym_pmd.c
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_asym_pmd.h"
+
+int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_asym_pmd.h b/drivers/crypto/qat/qat_asym_pmd.h
new file mode 100644
index 000000000..0465e0300
--- /dev/null
+++ b/drivers/crypto/qat/qat_asym_pmd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _QAT_ASYM_PMD_H_
+#define _QAT_ASYM_PMD_H_
+
+#include "qat_device.h"
+
+int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_ASYM_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index a8865904f..f1759ea76 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -3,6 +3,7 @@
*/
#include "qat_common.h"
+#include "qat_device.h"
#include "qat_logs.h"
int
@@ -51,3 +52,56 @@ qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
return 0;
}
+
+void qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service)
+{
+ int i;
+ struct qat_qp **qp;
+
+ if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
+ stats, dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
+
+ stats->enqueued_count += qp[i]->stats.enqueued_count;
+ stats->dequeued_count += qp[i]->stats.dequeued_count;
+ stats->enqueue_err_count += qp[i]->stats.enqueue_err_count;
+ stats->dequeue_err_count += qp[i]->stats.dequeue_err_count;
+ }
+}
+
+void qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service)
+{
+ int i;
+ struct qat_qp **qp;
+
+ if (dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
+ dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
+ memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
+ }
+
+ PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+}
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index fcf5c4c09..8ecebe954 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -8,9 +8,6 @@
#include <rte_mbuf.h>
-/**< Intel(R) QAT Symmetric Crypto PMD device name */
-#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
-
/**< Intel(R) QAT device name for PCI registration */
#define QAT_PCI_NAME qat
/*
@@ -21,10 +18,9 @@
/* Intel(R) QuickAssist Technology device generation is enumerated
* from one according to the generation of the device
*/
-
enum qat_device_gen {
QAT_GEN1 = 1,
- QAT_GEN2,
+ QAT_GEN2
};
enum qat_service_type {
@@ -62,8 +58,17 @@ struct qat_common_stats {
/**< Total error count on operations dequeued */
};
+struct qat_pci_device;
+
int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
struct qat_sgl *list, uint32_t data_len);
+void
+qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service);
+void
+qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service);
#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_comp_pmd.c b/drivers/crypto/qat/qat_comp_pmd.c
new file mode 100644
index 000000000..547b3db49
--- /dev/null
+++ b/drivers/crypto/qat/qat_comp_pmd.c
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_comp_pmd.h"
+
+
+int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_comp_pmd.h b/drivers/crypto/qat/qat_comp_pmd.h
new file mode 100644
index 000000000..cc31246c3
--- /dev/null
+++ b/drivers/crypto/qat/qat_comp_pmd.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _QAT_COMP_PMD_H_
+#define _QAT_COMP_PMD_H_
+
+#include "qat_device.h"
+
+
+/**< Intel(R) QAT Compression PMD device name */
+#define COMPRESSDEV_NAME_QAT_PMD comp_qat
+
+
+/** private data structure for a QAT compression device.
+ * This QAT device is a device offering only a compression service,
+ * there can be one of these on each qat_pci_device (VF).
+ */
+struct qat_comp_dev_private {
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
+};
+
+int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_COMP_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 8b2ac5a5f..d1e103efd 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -4,7 +4,9 @@
#include "qat_device.h"
#include "adf_transport_access_macros.h"
-#include "qat_qp.h"
+#include "qat_sym_pmd.h"
+#include "qat_asym_pmd.h"
+#include "qat_comp_pmd.h"
/* Hardware device information per generation */
__extension__
@@ -24,58 +26,25 @@ struct qat_gen_hw_data qp_gen_config[] = {
static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
-int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
- __rte_unused struct rte_cryptodev_config *config)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
-}
-
-int qat_sym_dev_close(struct rte_cryptodev *dev)
-{
- int i, ret;
-
- PMD_INIT_FUNC_TRACE();
-
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_sym_qp_release(dev, i);
- if (ret < 0)
- return ret;
- }
-
- return 0;
-}
+/*
+ * The set of PCI devices this driver supports
+ */
-void qat_sym_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info)
-{
- struct qat_sym_dev_private *internals = dev->data->dev_private;
- const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev->qat_dev_gen]
- .qp_hw_data[QAT_SERVICE_SYMMETRIC];
-
- PMD_INIT_FUNC_TRACE();
- if (info != NULL) {
- info->max_nb_queue_pairs =
- qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
- info->feature_flags = dev->feature_flags;
- info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
- info->driver_id = cryptodev_qat_driver_id;
- info->pci_dev = RTE_DEV_TO_PCI(dev->device);
- }
-}
+static const struct rte_pci_id pci_id_qat_map[] = {
+ {
+ RTE_PCI_DEVICE(0x8086, 0x0443),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x37c9),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x19e3),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x6f55),
+ },
+ {.device_id = 0},
+};
static struct qat_pci_device *
@@ -203,3 +172,70 @@ qat_pci_device_release(struct rte_pci_device *pci_dev)
name, qat_nb_pci_devices);
return 0;
}
+
+
+
+static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ int ret = 0;
+ struct qat_pci_device *qat_pci_dev;
+
+ PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
+ pci_dev->addr.bus,
+ pci_dev->addr.devid,
+ pci_dev->addr.function);
+
+ qat_pci_dev = qat_pci_device_allocate(pci_dev);
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+
+ ret = qat_sym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_comp_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_asym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ return 0;
+
+error_out:
+ qat_asym_dev_destroy(qat_pci_dev);
+ qat_comp_dev_destroy(qat_pci_dev);
+ qat_sym_dev_destroy(qat_pci_dev);
+ qat_pci_device_release(pci_dev);
+ return ret;
+
+}
+
+static int qat_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_pci_dev;
+
+ if (pci_dev == NULL)
+ return -EINVAL;
+
+ qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
+ if (qat_pci_dev == NULL)
+ return 0;
+
+ qat_asym_dev_destroy(qat_pci_dev);
+ qat_comp_dev_destroy(qat_pci_dev);
+ qat_sym_dev_destroy(qat_pci_dev);
+ return qat_pci_device_release(pci_dev);
+
+}
+
+static struct rte_pci_driver rte_qat_pmd = {
+ .id_table = pci_id_qat_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = qat_pci_probe,
+ .remove = qat_pci_remove
+};
+RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 5424a9a94..fd1819354 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -4,8 +4,8 @@
#ifndef _QAT_DEVICE_H_
#define _QAT_DEVICE_H_
-#include <rte_cryptodev_pmd.h>
#include <rte_bus_pci.h>
+
#include "qat_common.h"
#include "qat_logs.h"
#include "adf_transport_access_macros.h"
@@ -18,12 +18,6 @@
#define QAT_MAX_PCI_DEVICES 48
#define QAT_DEV_NAME_MAX_LEN 64
-
-extern uint8_t cryptodev_qat_driver_id;
-
-extern int qat_sym_qp_release(struct rte_cryptodev *dev,
- uint16_t queue_pair_id);
-
/*
* This struct holds all the data about a QAT pci device
* including data about all services it supports.
@@ -63,20 +57,6 @@ struct qat_pci_device {
};
-/** private data structure for a QAT device.
- * This QAT device is a device offering only symmetric crypto service,
- * there can be one of these on each qat_pci_device (VF),
- * in future there may also be private data structures for other services.
- */
-struct qat_sym_dev_private {
- struct qat_pci_device *qat_dev;
- /**< The qat pci device hosting the service */
- uint8_t sym_dev_id;
- /**< Device instance for this rte_cryptodev */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
- /* QAT device symmetric crypto capabilities */
-};
-
struct qat_gen_hw_data {
enum qat_device_gen dev_gen;
const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
@@ -84,14 +64,6 @@ struct qat_gen_hw_data {
extern struct qat_gen_hw_data qp_gen_config[];
-int qat_sym_dev_config(struct rte_cryptodev *dev,
- struct rte_cryptodev_config *config);
-int qat_sym_dev_start(struct rte_cryptodev *dev);
-void qat_sym_dev_stop(struct rte_cryptodev *dev);
-int qat_sym_dev_close(struct rte_cryptodev *dev);
-void qat_sym_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info);
-
struct qat_pci_device *
qat_pci_device_allocate(struct rte_pci_device *pci_dev);
int
@@ -99,4 +71,5 @@ qat_pci_device_release(struct rte_pci_device *pci_dev);
struct qat_pci_device *
qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev);
+
#endif /* _QAT_DEVICE_H_ */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 4ad4a5326..cf89d8440 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -12,8 +12,8 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
-#include "qat_qp.h"
#include "qat_device.h"
+#include "qat_qp.h"
#include "adf_transport_access_macros.h"
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 49d9f29d3..73888b805 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,9 +5,10 @@
#define _QAT_QP_H_
#include "qat_common.h"
-#include <rte_cryptodev_pmd.h>
#include "adf_transport_access_macros.h"
+struct qat_pci_device;
+
#define QAT_CSR_HEAD_WRITE_THRESH 32U
/* number of requests to accumulate before writing head CSR */
#define QAT_CSR_TAIL_WRITE_THRESH 32U
@@ -76,9 +77,9 @@ struct qat_queue {
struct qat_qp {
void *mmap_bar_addr;
uint16_t inflights16;
- struct qat_queue tx_q;
- struct qat_queue rx_q;
- struct qat_common_stats stats;
+ struct qat_queue tx_q;
+ struct qat_queue rx_q;
+ struct qat_common_stats stats;
struct rte_mempool *op_cookie_pool;
void **op_cookies;
uint32_t nb_descriptors;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 8007e25d6..15244d113 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -2,6 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
+#include <openssl/evp.h>
+
#include <rte_mempool.h>
#include <rte_mbuf.h>
#include <rte_hexdump.h>
@@ -9,14 +11,10 @@
#include <rte_bus_pci.h>
#include <rte_byteorder.h>
-#include <openssl/evp.h>
-
#include "qat_logs.h"
#include "qat_sym_session.h"
#include "qat_sym.h"
-#include "qat_qp.h"
-#include "adf_transport_access_macros.h"
-#include "qat_device.h"
+#include "qat_sym_pmd.h"
#define BYTE_LENGTH 8
/* bpi is only used for partial blocks of DES and AES
@@ -82,9 +80,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
return -EINVAL;
}
-/** Creates a context in either AES or DES in ECB mode
- * Depends on openssl libcrypto
- */
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
@@ -197,57 +192,6 @@ qat_bpicipher_postprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
-{
- return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
-}
-
-static int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen)
-{
-
- struct icp_qat_fw_comn_resp *resp_msg =
- (struct icp_qat_fw_comn_resp *)resp;
- struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
-
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
-
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess = (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
-
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
- *op = (void *)rx_op;
-
- return 0;
-}
-
-
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
-{
- return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
@@ -293,7 +237,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-static int
+int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen)
{
@@ -716,168 +660,38 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
return 0;
}
-
-static void qat_stats_get(struct qat_pci_device *dev,
- struct qat_common_stats *stats,
- enum qat_service_type service)
-{
- int i;
- struct qat_qp **qp;
-
- if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
- PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
- stats, dev, service);
- return;
- }
-
- qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
- if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
- service, i);
- continue;
- }
-
- stats->enqueued_count += qp[i]->stats.enqueued_count;
- stats->dequeued_count += qp[i]->stats.dequeued_count;
- stats->enqueue_err_count += qp[i]->stats.enqueue_err_count;
- stats->dequeue_err_count += qp[i]->stats.dequeue_err_count;
- }
-}
-
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats)
-{
- struct qat_common_stats qat_stats = {0};
- struct qat_sym_dev_private *qat_priv;
-
- if (stats == NULL || dev == NULL) {
- PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
- return;
- }
- qat_priv = dev->data->dev_private;
-
- qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
- stats->enqueued_count = qat_stats.enqueued_count;
- stats->dequeued_count = qat_stats.dequeued_count;
- stats->enqueue_err_count = qat_stats.enqueue_err_count;
- stats->dequeue_err_count = qat_stats.dequeue_err_count;
-}
-
-static void qat_stats_reset(struct qat_pci_device *dev,
- enum qat_service_type service)
-{
- int i;
- struct qat_qp **qp;
-
- if (dev == NULL || service >= QAT_SERVICE_INVALID) {
- PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
- dev, service);
- return;
- }
-
- qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
- if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
- service, i);
- continue;
- }
- memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
- }
-
- PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
-}
-
-void qat_sym_stats_reset(struct rte_cryptodev *dev)
-{
- struct qat_sym_dev_private *qat_priv;
-
- if (dev == NULL) {
- PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
- return;
- }
- qat_priv = dev->data->dev_private;
-
- qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
-
-}
-
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen)
{
- struct qat_sym_dev_private *qat_private = dev->data->dev_private;
- PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
- queue_pair_id, dev->data->dev_id);
-
- qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
- = NULL;
-
- return qat_qp_release((struct qat_qp **)
- &(dev->data->queue_pairs[queue_pair_id]));
-}
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
-{
- struct qat_qp *qp;
- int ret = 0;
- uint32_t i;
- struct qat_qp_config qat_qp_conf;
-
- struct qat_qp **qp_addr =
- (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
- struct qat_sym_dev_private *qat_private = dev->data->dev_private;
- const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev->qat_dev_gen]
- .qp_hw_data[QAT_SERVICE_SYMMETRIC];
- const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
-
- /* If qp is already in use free ring memory and qp metadata. */
- if (*qp_addr != NULL) {
- ret = qat_sym_qp_release(dev, qp_id);
- if (ret < 0)
- return ret;
- }
- if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
- return -EINVAL;
- }
-
- qat_qp_conf.hw = qp_hw_data;
- qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
- qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
- qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
- qat_qp_conf.socket_id = socket_id;
- qat_qp_conf.service_str = "sym";
-
- ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
- if (ret != 0)
- return ret;
-
- /* store a link to the qp in the qat_pci_device */
- qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
- = *qp_addr;
-
- qp = (struct qat_qp *)*qp_addr;
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
- for (i = 0; i < qp->nb_descriptors; i++) {
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
- struct qat_sym_op_cookie *sql_cookie =
- qp->op_cookies[i];
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_src);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_dst);
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
}
+ *op = (void *)rx_op;
- return ret;
+ return 0;
}
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 78b40e378..d887dc126 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -6,11 +6,8 @@
#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
-#include <rte_memzone.h>
#include "qat_common.h"
-#include "qat_device.h"
-#include "qat_crypto_capabilities.h"
/*
* This macro rounds up a number to a be a multiple of
@@ -29,23 +26,12 @@ struct qat_sym_op_cookie {
phys_addr_t qat_sgl_dst_phys_addr;
};
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats);
-void qat_sym_stats_reset(struct rte_cryptodev *dev);
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
- const struct rte_cryptodev_qp_conf *rx_conf, int socket_id,
- struct rte_mempool *session_pool);
-int qat_sym_qp_release(struct rte_cryptodev *dev,
- uint16_t queue_pair_id);
-
-
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops);
-
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops);
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen);
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen);
#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_crypto_capabilities.h b/drivers/crypto/qat/qat_sym_capabilities.h
similarity index 99%
rename from drivers/crypto/qat/qat_crypto_capabilities.h
rename to drivers/crypto/qat/qat_sym_capabilities.h
index 001c32c5d..d10a95ecb 100644
--- a/drivers/crypto/qat/qat_crypto_capabilities.h
+++ b/drivers/crypto/qat/qat_sym_capabilities.h
@@ -2,8 +2,8 @@
* Copyright(c) 2017-2018 Intel Corporation
*/
-#ifndef _QAT_CRYPTO_CAPABILITIES_H_
-#define _QAT_CRYPTO_CAPABILITIES_H_
+#ifndef _QAT_SYM_CAPABILITIES_H_
+#define _QAT_SYM_CAPABILITIES_H_
#define QAT_BASE_GEN1_SYM_CAPABILITIES \
{ /* SHA1 HMAC */ \
@@ -554,4 +554,4 @@
}, } \
}
-#endif /* _QAT_CRYPTO_CAPABILITIES_H_ */
+#endif /* _QAT_SYM_CAPABILITIES_H_ */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
new file mode 100644
index 000000000..aa71b4641
--- /dev/null
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -0,0 +1,326 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2015-2018 Intel Corporation
+ */
+
+#include <rte_bus_pci.h>
+#include <rte_common.h>
+#include <rte_dev.h>
+#include <rte_malloc.h>
+#include <rte_pci.h>
+#include <rte_cryptodev_pmd.h>
+
+#include "qat_logs.h"
+#include "qat_sym.h"
+#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
+
+uint8_t cryptodev_qat_driver_id;
+
+static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
+ QAT_BASE_GEN1_SYM_CAPABILITIES,
+ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
+};
+
+static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
+ QAT_BASE_GEN1_SYM_CAPABILITIES,
+ QAT_EXTRA_GEN2_SYM_CAPABILITIES,
+ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
+};
+
+static int qat_sym_qp_release(struct rte_cryptodev *dev,
+ uint16_t queue_pair_id);
+
+static int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused struct rte_cryptodev_config *config)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+static int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+static void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+}
+
+static int qat_sym_dev_close(struct rte_cryptodev *dev)
+{
+ int i, ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ ret = qat_sym_qp_release(dev, i);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static void qat_sym_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info)
+{
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[internals->qat_dev->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+
+ PMD_INIT_FUNC_TRACE();
+ if (info != NULL) {
+ info->max_nb_queue_pairs =
+ qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
+ info->feature_flags = dev->feature_flags;
+ info->capabilities = internals->qat_dev_capabilities;
+ info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
+ info->driver_id = cryptodev_qat_driver_id;
+ info->pci_dev = RTE_DEV_TO_PCI(dev->device);
+ }
+}
+
+static void qat_sym_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ struct qat_common_stats qat_stats = {0};
+ struct qat_sym_dev_private *qat_priv;
+
+ if (stats == NULL || dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
+ stats->enqueued_count = qat_stats.enqueued_count;
+ stats->dequeued_count = qat_stats.dequeued_count;
+ stats->enqueue_err_count = qat_stats.enqueue_err_count;
+ stats->dequeue_err_count = qat_stats.dequeue_err_count;
+}
+
+static void qat_sym_stats_reset(struct rte_cryptodev *dev)
+{
+ struct qat_sym_dev_private *qat_priv;
+
+ if (dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
+
+}
+
+static int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+{
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+
+ PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
+ queue_pair_id, dev->data->dev_id);
+
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
+ = NULL;
+
+ return qat_qp_release((struct qat_qp **)
+ &(dev->data->queue_pairs[queue_pair_id]));
+}
+
+static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ struct qat_qp **qp_addr =
+ (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[qat_private->qat_dev->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+ const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (*qp_addr != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+ qat_qp_conf.hw = qp_hw_data;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ /* store a link to the qp in the qat_pci_device */
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
+ = *qp_addr;
+
+ qp = (struct qat_qp *)*qp_addr;
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
+
+ struct qat_sym_op_cookie *sql_cookie =
+ qp->op_cookies[i];
+
+ sql_cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_src);
+
+ sql_cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_dst);
+ }
+
+ return ret;
+}
+
+static struct rte_cryptodev_ops crypto_qat_ops = {
+
+ /* Device related operations */
+ .dev_configure = qat_sym_dev_config,
+ .dev_start = qat_sym_dev_start,
+ .dev_stop = qat_sym_dev_stop,
+ .dev_close = qat_sym_dev_close,
+ .dev_infos_get = qat_sym_dev_info_get,
+
+ .stats_get = qat_sym_stats_get,
+ .stats_reset = qat_sym_stats_reset,
+ .queue_pair_setup = qat_sym_qp_setup,
+ .queue_pair_release = qat_sym_qp_release,
+ .queue_pair_start = NULL,
+ .queue_pair_stop = NULL,
+ .queue_pair_count = NULL,
+
+ /* Crypto related operations */
+ .session_get_size = qat_sym_session_get_private_size,
+ .session_configure = qat_sym_session_configure,
+ .session_clear = qat_sym_session_clear
+};
+
+static uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
+}
+
+static uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
+}
+
+int
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev_pmd_init_params init_params = {
+ .name = "",
+ .socket_id = qat_pci_dev->pci_dev->device.numa_node,
+ .private_data_size = sizeof(struct qat_sym_dev_private),
+ .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
+ };
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct rte_cryptodev *cryptodev;
+ struct qat_sym_dev_private *internals;
+
+ snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
+ qat_pci_dev->name, "sym");
+ PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+
+ cryptodev = rte_cryptodev_pmd_create(name,
+ &qat_pci_dev->pci_dev->device, &init_params);
+
+ if (cryptodev == NULL)
+ return -ENODEV;
+
+ cryptodev->driver_id = cryptodev_qat_driver_id;
+ cryptodev->dev_ops = &crypto_qat_ops;
+
+ cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
+ cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
+
+ cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
+ RTE_CRYPTODEV_FF_HW_ACCELERATED |
+ RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
+ RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
+
+ internals = cryptodev->data->dev_private;
+ internals->qat_dev = qat_pci_dev;
+ qat_pci_dev->sym_dev = internals;
+
+ internals->sym_dev_id = cryptodev->data->dev_id;
+ switch (qat_pci_dev->qat_dev_gen) {
+ case QAT_GEN1:
+ internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
+ break;
+ case QAT_GEN2:
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ break;
+ default:
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ PMD_DRV_LOG(DEBUG,
+ "QAT gen %d capabilities unknown, default to GEN2",
+ qat_pci_dev->qat_dev_gen);
+ break;
+ }
+
+ PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
+ name, internals->sym_dev_id);
+ return 0;
+}
+
+int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev *cryptodev;
+
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+ if (qat_pci_dev->sym_dev == NULL)
+ return 0;
+
+ /* free crypto device */
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
+ rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_dev = NULL;
+
+ return 0;
+}
+
+
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+static struct cryptodev_driver qat_crypto_drv;
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+ cryptodev_qat_driver_id);
diff --git a/drivers/crypto/qat/qat_sym_pmd.h b/drivers/crypto/qat/qat_sym_pmd.h
new file mode 100644
index 000000000..efa3b0775
--- /dev/null
+++ b/drivers/crypto/qat/qat_sym_pmd.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2015-2018 Intel Corporation
+ */
+
+#ifndef _QAT_SYM_PMD_H_
+#define _QAT_SYM_PMD_H_
+
+#include <rte_cryptodev.h>
+
+#include "qat_sym_capabilities.h"
+#include "qat_device.h"
+
+
+/**< Intel(R) QAT Symmetric Crypto PMD device name */
+#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+
+extern uint8_t cryptodev_qat_driver_id;
+
+/** private data structure for a QAT device.
+ * This QAT device is a device offering only symmetric crypto service,
+ * there can be one of these on each qat_pci_device (VF),
+ * in future there may also be private data structures for other services.
+ */
+struct qat_sym_dev_private {
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
+ uint8_t sym_dev_id;
+ /**< Device instance for this rte_cryptodev */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device symmetric crypto capabilities */
+};
+
+
+int
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_SYM_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index d8a5d883f..a7093fa4b 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -1,6 +1,12 @@
/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
* Copyright(c) 2015-2018 Intel Corporation
*/
+
+#include <openssl/sha.h> /* Needed to calculate pre-compute values */
+#include <openssl/aes.h> /* Needed to calculate pre-compute values */
+#include <openssl/md5.h> /* Needed to calculate pre-compute values */
+#include <openssl/evp.h> /* Needed for bpi runt block processing */
+
#include <rte_memcpy.h>
#include <rte_common.h>
#include <rte_spinlock.h>
@@ -10,14 +16,8 @@
#include <rte_crypto_sym.h>
#include "qat_logs.h"
-#include "qat_device.h"
-
-#include <openssl/sha.h> /* Needed to calculate pre-compute values */
-#include <openssl/aes.h> /* Needed to calculate pre-compute values */
-#include <openssl/md5.h> /* Needed to calculate pre-compute values */
-#include <openssl/evp.h>
-
#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
/** Frees a context previously created
* Depends on openssl libcrypto
@@ -29,6 +29,9 @@ bpi_cipher_ctx_free(void *bpi_ctx)
EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
}
+/** Creates a context in either AES or DES in ECB mode
+ * Depends on openssl libcrypto
+ */
static int
bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
enum rte_crypto_cipher_operation direction __rte_unused,
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index d3d27ff46..18d77e990 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _QAT_SYM_SESSION_H_
#define _QAT_SYM_SESSION_H_
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
deleted file mode 100644
index adad0ebe6..000000000
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ /dev/null
@@ -1,254 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2015-2018 Intel Corporation
- */
-
-#include <rte_bus_pci.h>
-#include <rte_common.h>
-#include <rte_dev.h>
-#include <rte_malloc.h>
-#include <rte_pci.h>
-#include <rte_cryptodev_pmd.h>
-
-#include "qat_sym.h"
-#include "qat_sym_session.h"
-#include "qat_logs.h"
-
-uint8_t cryptodev_qat_driver_id;
-
-static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
- QAT_BASE_GEN1_SYM_CAPABILITIES,
- RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
-};
-
-static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
- QAT_BASE_GEN1_SYM_CAPABILITIES,
- QAT_EXTRA_GEN2_SYM_CAPABILITIES,
- RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
-};
-
-static struct rte_cryptodev_ops crypto_qat_ops = {
-
- /* Device related operations */
- .dev_configure = qat_sym_dev_config,
- .dev_start = qat_sym_dev_start,
- .dev_stop = qat_sym_dev_stop,
- .dev_close = qat_sym_dev_close,
- .dev_infos_get = qat_sym_dev_info_get,
-
- .stats_get = qat_sym_stats_get,
- .stats_reset = qat_sym_stats_reset,
- .queue_pair_setup = qat_sym_qp_setup,
- .queue_pair_release = qat_sym_qp_release,
- .queue_pair_start = NULL,
- .queue_pair_stop = NULL,
- .queue_pair_count = NULL,
-
- /* Crypto related operations */
- .session_get_size = qat_sym_session_get_private_size,
- .session_configure = qat_sym_session_configure,
- .session_clear = qat_sym_session_clear
-};
-
-/*
- * The set of PCI devices this driver supports
- */
-
-static const struct rte_pci_id pci_id_qat_map[] = {
- {
- RTE_PCI_DEVICE(0x8086, 0x0443),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x37c9),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x19e3),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x6f55),
- },
- {.device_id = 0},
-};
-
-
-
-static int
-qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
-{
- struct rte_cryptodev_pmd_init_params init_params = {
- .name = "",
- .socket_id = qat_pci_dev->pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_sym_dev_private),
- .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
- };
- char name[RTE_CRYPTODEV_NAME_MAX_LEN];
- struct rte_cryptodev *cryptodev;
- struct qat_sym_dev_private *internals;
-
- snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
- qat_pci_dev->name, "sym");
- PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
-
- cryptodev = rte_cryptodev_pmd_create(name,
- &qat_pci_dev->pci_dev->device, &init_params);
-
- if (cryptodev == NULL)
- return -ENODEV;
-
- cryptodev->driver_id = cryptodev_qat_driver_id;
- cryptodev->dev_ops = &crypto_qat_ops;
-
- cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
- cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
-
- cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
- RTE_CRYPTODEV_FF_HW_ACCELERATED |
- RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
- RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
-
- internals = cryptodev->data->dev_private;
- internals->qat_dev = qat_pci_dev;
- qat_pci_dev->sym_dev = internals;
-
- internals->sym_dev_id = cryptodev->data->dev_id;
- switch (qat_pci_dev->qat_dev_gen) {
- case QAT_GEN1:
- internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
- break;
- case QAT_GEN2:
- internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
- break;
- default:
- internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
- PMD_DRV_LOG(DEBUG,
- "QAT gen %d capabilities unknown, default to GEN2",
- qat_pci_dev->qat_dev_gen);
- break;
- }
-
- PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
- name, internals->sym_dev_id);
- return 0;
-}
-
-static int
-qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
-{
- struct rte_cryptodev *cryptodev;
-
- if (qat_pci_dev == NULL)
- return -ENODEV;
- if (qat_pci_dev->sym_dev == NULL)
- return 0;
-
- /* free crypto device */
- cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
- rte_cryptodev_pmd_destroy(cryptodev);
- qat_pci_dev->sym_dev = NULL;
-
- return 0;
-}
-
-static int
-qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
- struct rte_pci_device *pci_dev)
-{
- int ret = 0;
- struct qat_pci_device *qat_pci_dev;
-
- PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
- pci_dev->addr.bus,
- pci_dev->addr.devid,
- pci_dev->addr.function);
-
- qat_pci_dev = qat_pci_device_allocate(pci_dev);
- if (qat_pci_dev == NULL)
- return -ENODEV;
-
- ret = qat_sym_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- ret = qat_comp_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- ret = qat_asym_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- return 0;
-
-error_out:
- qat_asym_dev_destroy(qat_pci_dev);
- qat_comp_dev_destroy(qat_pci_dev);
- qat_sym_dev_destroy(qat_pci_dev);
- qat_pci_device_release(pci_dev);
- return ret;
-
-}
-
-static int qat_pci_remove(struct rte_pci_device *pci_dev)
-{
- struct qat_pci_device *qat_pci_dev;
-
- if (pci_dev == NULL)
- return -EINVAL;
-
- qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
- if (qat_pci_dev == NULL)
- return 0;
-
- qat_asym_dev_destroy(qat_pci_dev);
- qat_comp_dev_destroy(qat_pci_dev);
- qat_sym_dev_destroy(qat_pci_dev);
- return qat_pci_device_release(pci_dev);
-
-}
-
-
-/* An rte_driver is needed in the registration of both the device and the driver
- * with cryptodev.
- * The actual qat pci's rte_driver can't be used as its name represents
- * the whole pci device with all services. Think of this as a holder for a name
- * for the crypto part of the pci device.
- */
-static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
-static struct rte_driver cryptodev_qat_sym_driver = {
- .name = qat_sym_drv_name,
- .alias = qat_sym_drv_name
-};
-static struct cryptodev_driver qat_crypto_drv;
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
- cryptodev_qat_driver_id);
-
-static struct rte_pci_driver rte_qat_pmd = {
- .id_table = pci_id_qat_map,
- .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
- .probe = qat_pci_probe,
- .remove = qat_pci_remove
-};
-RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
-RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 25/30] crypto/qat: add lock around csr access and change logic
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (23 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 24/30] crypto/qat: move code into appropriate files Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 26/30] crypto/qat: remove incorrect usage of bundle number Fiona Trahe
` (38 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Add lock around accesses to the arbiter CSR
and use & instead of ^ as ^ not safe if
arb_disable called when already disabled.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 26 +++++++++++++++++++-------
1 file changed, 19 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index cf89d8440..2e12f7c82 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -107,8 +107,10 @@ static int qat_queue_create(struct qat_pci_device *qat_dev,
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
static void adf_configure_queues(struct qat_qp *queue);
-static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr);
-static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr);
+static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock);
+static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock);
int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
@@ -234,7 +236,8 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
}
adf_configure_queues(qp);
- adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
+ adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr,
+ &qat_dev->arb_csr_lock);
snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE,
"%s%d_cookies_%s_qp%hu",
@@ -300,7 +303,8 @@ int qat_qp_release(struct qat_qp **qp_addr)
return -EAGAIN;
}
- adf_queue_arb_disable(&(qp->tx_q), qp->mmap_bar_addr);
+ adf_queue_arb_disable(&(qp->tx_q), qp->mmap_bar_addr,
+ &qp->qat_dev->arb_csr_lock);
for (i = 0; i < qp->nb_descriptors; i++)
rte_mempool_put(qp->op_cookie_pool, qp->op_cookies[i]);
@@ -461,7 +465,8 @@ static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
return -EINVAL;
}
-static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr)
+static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock)
{
uint32_t arb_csr_offset = ADF_ARB_RINGSRVARBEN_OFFSET +
(ADF_ARB_REG_SLOT *
@@ -469,12 +474,16 @@ static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr)
uint32_t value;
PMD_INIT_FUNC_TRACE();
+
+ rte_spinlock_lock(lock);
value = ADF_CSR_RD(base_addr, arb_csr_offset);
value |= (0x01 << txq->hw_queue_number);
ADF_CSR_WR(base_addr, arb_csr_offset, value);
+ rte_spinlock_unlock(lock);
}
-static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr)
+static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock)
{
uint32_t arb_csr_offset = ADF_ARB_RINGSRVARBEN_OFFSET +
(ADF_ARB_REG_SLOT *
@@ -482,9 +491,12 @@ static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr)
uint32_t value;
PMD_INIT_FUNC_TRACE();
+
+ rte_spinlock_lock(lock);
value = ADF_CSR_RD(base_addr, arb_csr_offset);
- value ^= (0x01 << txq->hw_queue_number);
+ value &= ~(0x01 << txq->hw_queue_number);
ADF_CSR_WR(base_addr, arb_csr_offset, value);
+ rte_spinlock_unlock(lock);
}
static void adf_configure_queues(struct qat_qp *qp)
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 26/30] crypto/qat: remove incorrect usage of bundle number
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (24 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 25/30] crypto/qat: add lock around csr access and change logic Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 27/30] crypto/qat: cleanups Fiona Trahe
` (37 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
As bundle_num is included in qat_gen1_qps static array
there shouldn't be a multiplier used in qat_qps_per_service()
Then removed ADF_NUM_BUNDLES_PER_DEV as no longer used.
Also renamed ADF_MAX_QPS_PER_BUNDLE to ADF_MAX_QPS_ON_ANY_SERVICE
and reduced from 4 to 2 which is enough for all current devices.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../crypto/qat/qat_adf/adf_transport_access_macros.h | 5 ++---
drivers/crypto/qat/qat_common.c | 4 ++--
drivers/crypto/qat/qat_device.h | 4 ++--
drivers/crypto/qat/qat_qp.c | 19 ++++---------------
drivers/crypto/qat/qat_qp.h | 2 +-
5 files changed, 11 insertions(+), 23 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index 2136d54ab..1eef5513f 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -50,9 +50,8 @@
#define ADF_MAX_RING_SIZE ADF_RING_SIZE_4M
#define ADF_DEFAULT_RING_SIZE ADF_RING_SIZE_16K
-#define ADF_NUM_BUNDLES_PER_DEV 1
-/* Maximum number of qps for any service type */
-#define ADF_MAX_QPS_PER_BUNDLE 4
+/* Maximum number of qps on a device for any service type */
+#define ADF_MAX_QPS_ON_ANY_SERVICE 2
#define ADF_RING_DIR_TX 0
#define ADF_RING_DIR_RX 1
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index f1759ea76..5d6779757 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -67,7 +67,7 @@ void qat_stats_get(struct qat_pci_device *dev,
}
qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ for (i = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++) {
if (qp[i] == NULL) {
PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
service, i);
@@ -94,7 +94,7 @@ void qat_stats_reset(struct qat_pci_device *dev,
}
qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ for (i = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++) {
if (qp[i] == NULL) {
PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
service, i);
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index fd1819354..fd20a0147 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -44,7 +44,7 @@ struct qat_pci_device {
uint8_t attached : 1;
/**< Flag indicating the device is attached */
- struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_PER_BUNDLE];
+ struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_ON_ANY_SERVICE];
/**< links to qps set up for each service, index same as on API */
/* Data relating to symmetric crypto service */
@@ -59,7 +59,7 @@ struct qat_pci_device {
struct qat_gen_hw_data {
enum qat_device_gen dev_gen;
- const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
+ const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_ON_ANY_SERVICE];
};
extern struct qat_gen_hw_data qp_gen_config[];
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 2e12f7c82..8fbdf85b8 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -29,7 +29,7 @@
__extension__
const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
- [ADF_MAX_QPS_PER_BUNDLE] = {
+ [ADF_MAX_QPS_ON_ANY_SERVICE] = {
/* queue pairs which provide an asymmetric crypto service */
[QAT_SERVICE_ASYMMETRIC] = {
{
@@ -42,14 +42,11 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
}, {
.service_type = QAT_SERVICE_ASYMMETRIC,
+ .hw_bundle_num = 0,
.tx_ring_num = 1,
.rx_ring_num = 9,
.tx_msg_size = 64,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
},
/* queue pairs which provide a symmetric crypto service */
@@ -69,10 +66,6 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
.rx_ring_num = 11,
.tx_msg_size = 128,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
},
/* queue pairs which provide a compression service */
@@ -91,10 +84,6 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
.rx_ring_num = 15,
.tx_msg_size = 128,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
}
};
@@ -118,10 +107,10 @@ int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
{
int i, count;
- for (i = 0, count = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++)
+ for (i = 0, count = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++)
if (qp_hw_data[i].service_type == service)
count++;
- return count * ADF_NUM_BUNDLES_PER_DEV;
+ return count;
}
static const struct rte_memzone *
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 73888b805..6f07bd67c 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -90,7 +90,7 @@ struct qat_qp {
/**< qat device this qp is on */
} __rte_cache_aligned;
-extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_PER_BUNDLE];
+extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_ON_ANY_SERVICE];
uint16_t
qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 27/30] crypto/qat: cleanups
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (25 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 26/30] crypto/qat: remove incorrect usage of bundle number Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 28/30] crypto/qat: create appropriately named device for registration Fiona Trahe
` (36 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
- renamed sgl_cookie to cookie
- renamed qp_gen_config to qat_gen_config as it is intended
to hold more than just qp data.
- removed unused macro - ALIGN_POW2
- moved 64_BYTE_ALIGN to qat_common.h
- removed crypto reference in common debug msg
- free cookie pool on qp creation error
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/qat_common.c | 2 +-
drivers/crypto/qat/qat_common.h | 2 ++
drivers/crypto/qat/qat_device.c | 2 +-
drivers/crypto/qat/qat_device.h | 2 +-
drivers/crypto/qat/qat_qp.c | 3 +++
drivers/crypto/qat/qat_sym.h | 8 --------
drivers/crypto/qat/qat_sym_pmd.c | 14 +++++++-------
7 files changed, 15 insertions(+), 18 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index 5d6779757..c10c1421a 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -103,5 +103,5 @@ void qat_stats_reset(struct qat_pci_device *dev,
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
}
- PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+ PMD_DRV_LOG(DEBUG, "QAT: %d stats cleared", service);
}
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 8ecebe954..db85d5482 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -15,6 +15,8 @@
*/
#define QAT_SGL_MAX_NUMBER 16
+#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
+
/* Intel(R) QuickAssist Technology device generation is enumerated
* from one according to the generation of the device
*/
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index d1e103efd..27f08874e 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -10,7 +10,7 @@
/* Hardware device information per generation */
__extension__
-struct qat_gen_hw_data qp_gen_config[] = {
+struct qat_gen_hw_data qat_gen_config[] = {
[QAT_GEN1] = {
.dev_gen = QAT_GEN1,
.qp_hw_data = qat_gen1_qps,
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index fd20a0147..4201a1c71 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -62,7 +62,7 @@ struct qat_gen_hw_data {
const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_ON_ANY_SERVICE];
};
-extern struct qat_gen_hw_data qp_gen_config[];
+extern struct qat_gen_hw_data qat_gen_config[];
struct qat_pci_device *
qat_pci_device_allocate(struct rte_pci_device *pci_dev);
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 8fbdf85b8..06ffac47b 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -266,6 +266,9 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
return 0;
create_err:
+ if (qp->op_cookie_pool)
+ rte_mempool_free(qp->op_cookie_pool);
+ rte_free(qp->op_cookies);
rte_free(qp);
return -EFAULT;
}
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index d887dc126..bdb672be4 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -9,14 +9,6 @@
#include "qat_common.h"
-/*
- * This macro rounds up a number to a be a multiple of
- * the alignment when the alignment is a power of 2
- */
-#define ALIGN_POW2_ROUNDUP(num, align) \
- (((num) + (align) - 1) & ~((align) - 1))
-#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
-
struct qat_sym_session;
struct qat_sym_op_cookie {
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index aa71b4641..e6760b8f8 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -68,7 +68,7 @@ static void qat_sym_dev_info_get(struct rte_cryptodev *dev,
{
struct qat_sym_dev_private *internals = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev->qat_dev_gen]
+ qat_gen_config[internals->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
@@ -143,7 +143,7 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
struct qat_sym_dev_private *qat_private = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev->qat_dev_gen]
+ qat_gen_config[qat_private->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
@@ -178,16 +178,16 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
for (i = 0; i < qp->nb_descriptors; i++) {
- struct qat_sym_op_cookie *sql_cookie =
+ struct qat_sym_op_cookie *cookie =
qp->op_cookies[i];
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
+ cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(cookie) +
offsetof(struct qat_sym_op_cookie,
qat_sgl_src);
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
+ cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(cookie) +
offsetof(struct qat_sym_op_cookie,
qat_sgl_dst);
}
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 28/30] crypto/qat: create appropriately named device for registration
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (26 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 27/30] crypto/qat: cleanups Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 29/30] crypto/qat: add MAX PCI DEVICES flag to config file Fiona Trahe
` (35 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
For every QAT pci device probed, populate a local rte_device
containing an rte_driver. The rte_driver was created in a previous
patch to provide a crypto-specific driver name: "crypto_qat".
This was previously only used for driver registration, now it's also
used in device creation.
This allows applications to find devices driven by "crypto_qat".
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.h | 5 +++++
drivers/crypto/qat/qat_sym_pmd.c | 38 ++++++++++++++++++++++++--------------
2 files changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 4201a1c71..3df6520c3 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -50,6 +50,11 @@ struct qat_pci_device {
/* Data relating to symmetric crypto service */
struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
+ struct rte_device sym_rte_dev;
+ /**< This represents the crypto subset of this pci device.
+ * Register with this rather than with the one in
+ * pci_dev so that its driver can have a crypto-specific name
+ */
/* Data relating to compression service */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index e6760b8f8..28e579b77 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -232,6 +232,18 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static const struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+
int
qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
{
@@ -249,12 +261,19 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
qat_pci_dev->name, "sym");
PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+ /* Populate subset device to use in cryptodev device creation */
+ qat_pci_dev->sym_rte_dev.driver = &cryptodev_qat_sym_driver;
+ qat_pci_dev->sym_rte_dev.numa_node =
+ qat_pci_dev->pci_dev->device.numa_node;
+ qat_pci_dev->sym_rte_dev.devargs = NULL;
+
cryptodev = rte_cryptodev_pmd_create(name,
- &qat_pci_dev->pci_dev->device, &init_params);
+ &(qat_pci_dev->sym_rte_dev), &init_params);
if (cryptodev == NULL)
return -ENODEV;
+ qat_pci_dev->sym_rte_dev.name = cryptodev->data->name;
cryptodev->driver_id = cryptodev_qat_driver_id;
cryptodev->dev_ops = &crypto_qat_ops;
@@ -287,7 +306,7 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
}
PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
- name, internals->sym_dev_id);
+ cryptodev->data->name, internals->sym_dev_id);
return 0;
}
@@ -304,23 +323,14 @@ qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
/* free crypto device */
cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_rte_dev.name = NULL;
qat_pci_dev->sym_dev = NULL;
return 0;
}
-/* An rte_driver is needed in the registration of both the device and the driver
- * with cryptodev.
- * The actual qat pci's rte_driver can't be used as its name represents
- * the whole pci device with all services. Think of this as a holder for a name
- * for the crypto part of the pci device.
- */
-static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
-static struct rte_driver cryptodev_qat_sym_driver = {
- .name = qat_sym_drv_name,
- .alias = qat_sym_drv_name
-};
static struct cryptodev_driver qat_crypto_drv;
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv,
+ cryptodev_qat_sym_driver,
cryptodev_qat_driver_id);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 29/30] crypto/qat: add MAX PCI DEVICES flag to config file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (27 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 28/30] crypto/qat: create appropriately named device for registration Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-06 18:52 ` [PATCH 30/30] crypto/qat: add performance improvement into qat crypto dev Fiona Trahe
` (34 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
Added CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES to
build config files.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
config/common_base | 4 ++++
config/rte_config.h | 2 ++
drivers/crypto/qat/qat_device.c | 8 ++++----
drivers/crypto/qat/qat_device.h | 2 --
4 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/config/common_base b/config/common_base
index ad4706267..47659bd01 100644
--- a/config/common_base
+++ b/config/common_base
@@ -478,6 +478,10 @@ CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n
# on a single QuickAssist device.
#
CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=2048
+#
+# Max. number of QuickAssist devices, which can be detected and attached
+#
+CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48
#
# Compile PMD for AESNI backed device
diff --git a/config/rte_config.h b/config/rte_config.h
index db6ceb6cd..1b2261a26 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -77,6 +77,8 @@
*/
/* QuickAssist device */
#define RTE_QAT_PMD_MAX_NB_SESSIONS 2048
+/* Max. number of QuickAssist devices which can be attached */
+#define RTE_PMD_QAT_MAX_PCI_DEVICES 48
/* DPAA2_SEC */
#define RTE_DPAA2_SEC_PMD_MAX_NB_SESSIONS 2048
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 27f08874e..94073a600 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -23,7 +23,7 @@ struct qat_gen_hw_data qat_gen_config[] = {
};
-static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
+static struct qat_pci_device qat_pci_devices[RTE_PMD_QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
/*
@@ -62,7 +62,7 @@ qat_pci_get_named_dev(const char *name)
if (name == NULL)
return NULL;
- for (i = 0; i < QAT_MAX_PCI_DEVICES; i++) {
+ for (i = 0; i < RTE_PMD_QAT_MAX_PCI_DEVICES; i++) {
dev = &qat_pci_devices[i];
if ((dev->attached == QAT_ATTACHED) &&
@@ -78,7 +78,7 @@ qat_pci_find_free_device_index(void)
{
uint8_t dev_id;
- for (dev_id = 0; dev_id < QAT_MAX_PCI_DEVICES; dev_id++) {
+ for (dev_id = 0; dev_id < RTE_PMD_QAT_MAX_PCI_DEVICES; dev_id++) {
if (qat_pci_devices[dev_id].attached == QAT_DETACHED)
break;
}
@@ -111,7 +111,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
}
qat_dev_id = qat_pci_find_free_device_index();
- if (qat_dev_id == QAT_MAX_PCI_DEVICES) {
+ if (qat_dev_id == RTE_PMD_QAT_MAX_PCI_DEVICES) {
PMD_DRV_LOG(ERR, "Reached maximum number of QAT devices");
return NULL;
}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 3df6520c3..e18c8a706 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,11 +11,9 @@
#include "adf_transport_access_macros.h"
#include "qat_qp.h"
-
#define QAT_DETACHED (0)
#define QAT_ATTACHED (1)
-#define QAT_MAX_PCI_DEVICES 48
#define QAT_DEV_NAME_MAX_LEN 64
/*
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH 30/30] crypto/qat: add performance improvement into qat crypto dev.
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (28 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 29/30] crypto/qat: add MAX PCI DEVICES flag to config file Fiona Trahe
@ 2018-04-06 18:52 ` Fiona Trahe
2018-04-11 12:41 ` [PATCH 00/30] crypto/qat: refactor to support multiple service De Lara Guarch, Pablo
` (33 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-04-06 18:52 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch; +Cc: fiona.trahe, tomaszx.jozwiak
From: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
- removed unused args. from process_response function
- changed adf_modulo function
- changed qat_sym_process_response into static inline version
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 29 +++++----
drivers/crypto/qat/qat_qp.h | 10 +--
drivers/crypto/qat/qat_sym.c | 129 --------------------------------------
drivers/crypto/qat/qat_sym.h | 131 +++++++++++++++++++++++++++++++++++++--
drivers/crypto/qat/qat_sym_pmd.c | 1 -
5 files changed, 143 insertions(+), 157 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 06ffac47b..44a24926b 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -14,6 +14,7 @@
#include "qat_logs.h"
#include "qat_device.h"
#include "qat_qp.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
@@ -256,7 +257,7 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
qp->qat_dev_gen = qat_dev->qat_dev_gen;
qp->build_request = qat_qp_conf->build_request;
- qp->process_response = qat_qp_conf->process_response;
+ qp->service_type = qat_qp_conf->hw->service_type;
qp->qat_dev = qat_dev;
PMD_DRV_LOG(DEBUG, "QP setup complete: id: %d, cookiepool: %s",
@@ -394,7 +395,7 @@ qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
- queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
+ queue->modulo_mask = (1 << ADF_RING_SIZE_MODULO(queue->queue_size)) - 1;
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
@@ -419,11 +420,11 @@ qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
queue->hw_queue_number, queue_base);
PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
- " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ " nb msgs %u, msg_size %u, max_inflights %u modulo mask %u",
queue->memz_name,
queue->queue_size, queue_size_bytes,
qp_conf->nb_descriptors, desc_size,
- queue->max_inflights, queue->modulo);
+ queue->max_inflights, queue->modulo_mask);
return 0;
@@ -512,13 +513,9 @@ static void adf_configure_queues(struct qat_qp *qp)
queue->hw_queue_number, queue_config);
}
-
-static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
+static inline uint32_t adf_modulo(uint32_t data, uint32_t modulo_mask)
{
- uint32_t div = data >> shift;
- uint32_t mult = div << shift;
-
- return data - mult;
+ return data & modulo_mask;
}
static inline void
@@ -602,7 +599,7 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
goto kick_tail;
}
- tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ tail = adf_modulo(tail + queue->msg_size, queue->modulo_mask);
ops++;
nb_ops_sent++;
}
@@ -634,11 +631,13 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
resp_counter != nb_ops) {
- tmp_qp->process_response(ops, resp_msg,
- tmp_qp->op_cookies[head / rx_queue->msg_size],
- tmp_qp->qat_dev_gen);
+ if (tmp_qp->service_type == QAT_SERVICE_SYMMETRIC)
+ qat_sym_process_response(ops, resp_msg);
+ /* add qat_asym_process_response here */
+ /* add qat_comp_process_response here */
- head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
+ head = adf_modulo(head + rx_queue->msg_size,
+ rx_queue->modulo_mask);
resp_msg = (uint8_t *)rx_queue->base_addr + head;
ops++;
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 6f07bd67c..59db945e7 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -21,11 +21,6 @@ typedef int (*build_request_t)(void *op,
enum qat_device_gen qat_dev_gen);
/**< Build a request from an op. */
-typedef int (*process_response_t)(void **ops,
- uint8_t *resp, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
-/**< Process a response descriptor and return the associated op. */
-
/**
* Structure with data needed for creation of queue pair.
*/
@@ -46,7 +41,6 @@ struct qat_qp_config {
uint32_t cookie_size;
int socket_id;
build_request_t build_request;
- process_response_t process_response;
const char *service_str;
};
@@ -59,7 +53,7 @@ struct qat_queue {
rte_iova_t base_phys_addr; /* Queue physical address */
uint32_t head; /* Shadow copy of the head */
uint32_t tail; /* Shadow copy of the tail */
- uint32_t modulo;
+ uint32_t modulo_mask;
uint32_t msg_size;
uint16_t max_inflights;
uint32_t queue_size;
@@ -85,7 +79,7 @@ struct qat_qp {
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
- process_response_t process_response;
+ enum qat_service_type service_type;
struct qat_pci_device *qat_dev;
/**< qat device this qp is on */
} __rte_cache_aligned;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 15244d113..887d4ebcd 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -12,44 +12,7 @@
#include <rte_byteorder.h>
#include "qat_logs.h"
-#include "qat_sym_session.h"
#include "qat_sym.h"
-#include "qat_sym_pmd.h"
-
-#define BYTE_LENGTH 8
-/* bpi is only used for partial blocks of DES and AES
- * so AES block len can be assumed as max len for iv, src and dst
- */
-#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-
-/** Encrypt a single partial block
- * Depends on openssl libcrypto
- * Uses ECB+XOR to do CFB encryption, same result, more performant
- */
-static inline int
-bpi_cipher_encrypt(uint8_t *src, uint8_t *dst,
- uint8_t *iv, int ivlen, int srclen,
- void *bpi_ctx)
-{
- EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX *)bpi_ctx;
- int encrypted_ivlen;
- uint8_t encrypted_iv[BPI_MAX_ENCR_IV_LEN];
- uint8_t *encr = encrypted_iv;
-
- /* ECB method: encrypt the IV, then XOR this with plaintext */
- if (EVP_EncryptUpdate(ctx, encrypted_iv, &encrypted_ivlen, iv, ivlen)
- <= 0)
- goto cipher_encrypt_err;
-
- for (; srclen != 0; --srclen, ++dst, ++src, ++encr)
- *dst = *src ^ *encr;
-
- return 0;
-
-cipher_encrypt_err:
- PMD_DRV_LOG(ERR, "libcrypto ECB cipher encrypt failed");
- return -EINVAL;
-}
/** Decrypt a single partial block
* Depends on openssl libcrypto
@@ -136,62 +99,6 @@ qat_bpicipher_preprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-static inline uint32_t
-qat_bpicipher_postprocess(struct qat_sym_session *ctx,
- struct rte_crypto_op *op)
-{
- int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
- struct rte_crypto_sym_op *sym_op = op->sym;
- uint8_t last_block_len = block_len > 0 ?
- sym_op->cipher.data.length % block_len : 0;
-
- if (last_block_len > 0 &&
- ctx->qat_dir == ICP_QAT_HW_CIPHER_ENCRYPT) {
-
- /* Encrypt last block */
- uint8_t *last_block, *dst, *iv;
- uint32_t last_block_offset;
-
- last_block_offset = sym_op->cipher.data.offset +
- sym_op->cipher.data.length - last_block_len;
- last_block = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_src,
- uint8_t *, last_block_offset);
-
- if (unlikely(sym_op->m_dst != NULL))
- /* out-of-place operation (OOP) */
- dst = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_dst,
- uint8_t *, last_block_offset);
- else
- dst = last_block;
-
- if (last_block_len < sym_op->cipher.data.length)
- /* use previous block ciphertext as IV */
- iv = dst - block_len;
- else
- /* runt block, i.e. less than one full block */
- iv = rte_crypto_op_ctod_offset(op, uint8_t *,
- ctx->cipher_iv.offset);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "BPI: src before post-process:", last_block,
- last_block_len);
- if (sym_op->m_dst != NULL)
- rte_hexdump(stdout, "BPI: dst before post-process:",
- dst, last_block_len);
-#endif
- bpi_cipher_encrypt(last_block, dst, iv, block_len,
- last_block_len, ctx->bpi_ctx);
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "BPI: src after post-process:", last_block,
- last_block_len);
- if (sym_op->m_dst != NULL)
- rte_hexdump(stdout, "BPI: dst after post-process:", dst,
- last_block_len);
-#endif
- }
- return sym_op->cipher.data.length - last_block_len;
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
@@ -659,39 +566,3 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
#endif
return 0;
}
-
-int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen)
-{
-
- struct icp_qat_fw_comn_resp *resp_msg =
- (struct icp_qat_fw_comn_resp *)resp;
- struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
-
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
-
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess = (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
-
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
- *op = (void *)rx_op;
-
- return 0;
-}
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index bdb672be4..2eaba931e 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -6,8 +6,17 @@
#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
+#include <openssl/evp.h>
#include "qat_common.h"
+#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
+
+#define BYTE_LENGTH 8
+/* bpi is only used for partial blocks of DES and AES
+ * so AES block len can be assumed as max len for iv, src and dst
+ */
+#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
struct qat_sym_session;
@@ -21,9 +30,123 @@ struct qat_sym_op_cookie {
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
-int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen);
+
+/** Encrypt a single partial block
+ * Depends on openssl libcrypto
+ * Uses ECB+XOR to do CFB encryption, same result, more performant
+ */
+static inline int
+bpi_cipher_encrypt(uint8_t *src, uint8_t *dst,
+ uint8_t *iv, int ivlen, int srclen,
+ void *bpi_ctx)
+{
+ EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX *)bpi_ctx;
+ int encrypted_ivlen;
+ uint8_t encrypted_iv[BPI_MAX_ENCR_IV_LEN];
+ uint8_t *encr = encrypted_iv;
+
+ /* ECB method: encrypt the IV, then XOR this with plaintext */
+ if (EVP_EncryptUpdate(ctx, encrypted_iv, &encrypted_ivlen, iv, ivlen)
+ <= 0)
+ goto cipher_encrypt_err;
+
+ for (; srclen != 0; --srclen, ++dst, ++src, ++encr)
+ *dst = *src ^ *encr;
+
+ return 0;
+
+cipher_encrypt_err:
+ PMD_DRV_LOG(ERR, "libcrypto ECB cipher encrypt failed");
+ return -EINVAL;
+}
+
+static inline uint32_t
+qat_bpicipher_postprocess(struct qat_sym_session *ctx,
+ struct rte_crypto_op *op)
+{
+ int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
+ struct rte_crypto_sym_op *sym_op = op->sym;
+ uint8_t last_block_len = block_len > 0 ?
+ sym_op->cipher.data.length % block_len : 0;
+
+ if (last_block_len > 0 &&
+ ctx->qat_dir == ICP_QAT_HW_CIPHER_ENCRYPT) {
+
+ /* Encrypt last block */
+ uint8_t *last_block, *dst, *iv;
+ uint32_t last_block_offset;
+
+ last_block_offset = sym_op->cipher.data.offset +
+ sym_op->cipher.data.length - last_block_len;
+ last_block = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_src,
+ uint8_t *, last_block_offset);
+
+ if (unlikely(sym_op->m_dst != NULL))
+ /* out-of-place operation (OOP) */
+ dst = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_dst,
+ uint8_t *, last_block_offset);
+ else
+ dst = last_block;
+
+ if (last_block_len < sym_op->cipher.data.length)
+ /* use previous block ciphertext as IV */
+ iv = dst - block_len;
+ else
+ /* runt block, i.e. less than one full block */
+ iv = rte_crypto_op_ctod_offset(op, uint8_t *,
+ ctx->cipher_iv.offset);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "BPI: src before post-process:", last_block,
+ last_block_len);
+ if (sym_op->m_dst != NULL)
+ rte_hexdump(stdout, "BPI: dst before post-process:",
+ dst, last_block_len);
+#endif
+ bpi_cipher_encrypt(last_block, dst, iv, block_len,
+ last_block_len, ctx->bpi_ctx);
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "BPI: src after post-process:", last_block,
+ last_block_len);
+ if (sym_op->m_dst != NULL)
+ rte_hexdump(stdout, "BPI: dst after post-process:", dst,
+ last_block_len);
+#endif
+ }
+ return sym_op->cipher.data.length - last_block_len;
+}
+
+static inline void
+qat_sym_process_response(void **op, uint8_t *resp)
+{
+
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
+
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
+
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
+
+
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+ }
+ *op = (void *)rx_op;
+}
#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index 28e579b77..6b39b32f8 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -160,7 +160,6 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.hw = qp_hw_data;
qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
qat_qp_conf.socket_id = socket_id;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [PATCH 00/30] crypto/qat: refactor to support multiple service
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (29 preceding siblings ...)
2018-04-06 18:52 ` [PATCH 30/30] crypto/qat: add performance improvement into qat crypto dev Fiona Trahe
@ 2018-04-11 12:41 ` De Lara Guarch, Pablo
2018-05-11 11:13 ` [PATCH v2 00/31] crypto/qat: refactor to support multiple Fiona Trahe
` (32 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: De Lara Guarch, Pablo @ 2018-04-11 12:41 UTC (permalink / raw)
To: Trahe, Fiona, dev; +Cc: Jozwiak, TomaszX
Hi Fiona,
> -----Original Message-----
> From: Trahe, Fiona
> Sent: Friday, April 6, 2018 7:52 PM
> To: dev@dpdk.org; De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: Trahe, Fiona <fiona.trahe@intel.com>; Jozwiak, TomaszX
> <tomaszx.jozwiak@intel.com>
> Subject: [PATCH 00/30] crypto/qat: refactor to support multiple service
>
> The QAT crypto PMD was peppered with references to rte_cryptodev artefacts.
> Also the pci device it presented to the API layer was entirely owned by the one
> cryptodev device instance created by the probe.
> This patchset refactors the PMD so one pci device can present out multiple
> different device instances to implement different APIs, specifically cryptodev
> and compressdev instances.
> The code is refactored into
> - common code which handles the pci device discovery, configuration,
> queue-pair setup, stats gathering and data-path enqueue and dequeue.
> - service-specific code for symmetric crypto, which is a thin layer
> implementing the cryptodev API and using the common code.
> - place-holder files for service-specific code to support asymmetric
> crypto and compression services in future.
Since these patches arrived late in the merge window, I will defer them to 18.08,
targeting an early merge in that release.
Still, we can merge the first one, as we are trying to move all license headers to the
new SPDX format.
Thanks,
Pablo
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH 01/30] crypto/qat: use SPDX license
2018-04-06 18:51 ` [PATCH 01/30] crypto/qat: use SPDX license Fiona Trahe
@ 2018-04-18 8:03 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 109+ messages in thread
From: De Lara Guarch, Pablo @ 2018-04-18 8:03 UTC (permalink / raw)
To: Trahe, Fiona, dev; +Cc: Jozwiak, TomaszX
> -----Original Message-----
> From: Trahe, Fiona
> Sent: Friday, April 6, 2018 7:52 PM
> To: dev@dpdk.org; De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: Trahe, Fiona <fiona.trahe@intel.com>; Jozwiak, TomaszX
> <tomaszx.jozwiak@intel.com>
> Subject: [PATCH 01/30] crypto/qat: use SPDX license
>
> Use SPDK license for dual-licensed files and update license date in all files
>
> Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Applied this single patch to dpdk-next-crypto.
Thanks,
Pablo
^ permalink raw reply [flat|nested] 109+ messages in thread
* [PATCH v2 00/31] crypto/qat: refactor to support multiple
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (30 preceding siblings ...)
2018-04-11 12:41 ` [PATCH 00/30] crypto/qat: refactor to support multiple service De Lara Guarch, Pablo
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 01/31] crypto/qat: add qat common header Fiona Trahe
` (31 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Note this patchset is targeting 18.08 release
The QAT crypto PMD was peppered with references to rte_cryptodev
artefacts.
Also the pci device it presented to the API layer was
entirely owned by the one cryptodev device instance created by the
probe.
This patchset refactors the PMD so one pci device can present out
multiple different device instances to implement different APIs,
specifically cryptodev and compressdev instances.
The code is refactored into
- common code which handles the pci device discovery, configuration,
queue-pair setup, stats gathering and data-path enqueue and dequeue.
- service-specific code for symmetric crypto, which is a thin layer
implementing the cryptodev API and using the common code.
- place-holder files for service-specific code to support asymmetric
crypto and compression services in future.
The code is all still in the crypto/qat directory as it is functionally
unchanged, still only supporting a symmetric crypto service. The
code will move to drivers/common, /crypto and /compress in
a future patch when this is needed for a compression PMD.
Changes for v2:
- fixed some bugs compiled in when debug enabled
- added patch to remove MAX_NB_SESSIONS from config file
- removed SPDX license patch as already applied
- documented device name formats
Fiona Trahe (29):
crypto/qat: add qat common header
crypto/qat: add qat device files
crypto/qat: remove unused includes
crypto/qat: add symmetric session file
crypto/qat: change filename crypto to sym
crypto/qat: rename fns for consistency
crypto/qat: renamed sym-specific structs
crypto/qat: make enqueue function generic
crypto/qat: make dequeue function generic
crypto/qat: move generic qp fn to qp file
crypto/qat: separate sym-specific from generic qp setup
crypto/qat: move sym-specific qp code to sym file
crypto/qat: remove dependencies on cryptodev from common
crypto/qat: move defines from sym to qp header file
crypto/qat: create data structures to support different generations
crypto/qat: rename sgl related objects
crypto/qat: move sgl related element to appropriate files
crypto/qat: add QAT PCI device struct
crypto/qat: separate the name used for PCI reg from crypto name
crypto/qat: move to using new device structure
crypto/qat: use common stats structures
crypto/qat: rename functions which depend on cryptodev
crypto/qat: move code into appropriate files
crypto/qat: add lock around csr access and change logic
crypto/qat: remove incorrect usage of bundle number
crypto/qat: cleanups
crypto/qat: create appropriately named device for registration
crypto/qat: add MAX PCI DEVICES flag to config file
doc/qat: specify QAT driver and device name formats
Tomasz Jozwiak (2):
crypto/qat: add performance improvement into qat crypto dev.
crypto/qat: remove CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS
config/common_base | 6 +-
config/rte_config.h | 7 +-
doc/guides/cryptodevs/qat.rst | 12 +
drivers/crypto/qat/Makefile | 10 +-
drivers/crypto/qat/meson.build | 9 +-
.../qat/qat_adf/adf_transport_access_macros.h | 9 +-
drivers/crypto/qat/qat_adf/qat_algs.h | 126 --
drivers/crypto/qat/qat_asym_pmd.c | 17 +
drivers/crypto/qat/qat_asym_pmd.h | 15 +
drivers/crypto/qat/qat_common.c | 107 ++
drivers/crypto/qat/qat_common.h | 76 +
drivers/crypto/qat/qat_comp_pmd.c | 18 +
drivers/crypto/qat/qat_comp_pmd.h | 29 +
drivers/crypto/qat/qat_crypto.c | 1696 --------------------
drivers/crypto/qat/qat_crypto.h | 150 --
drivers/crypto/qat/qat_device.c | 242 +++
drivers/crypto/qat/qat_device.h | 78 +
drivers/crypto/qat/qat_qp.c | 463 ++++--
drivers/crypto/qat/qat_qp.h | 106 ++
drivers/crypto/qat/qat_sym.c | 568 +++++++
drivers/crypto/qat/qat_sym.h | 154 ++
...rypto_capabilities.h => qat_sym_capabilities.h} | 6 +-
drivers/crypto/qat/qat_sym_pmd.c | 335 ++++
drivers/crypto/qat/qat_sym_pmd.h | 40 +
.../qat_algs_build_desc.c => qat_sym_session.c} | 786 ++++++++-
drivers/crypto/qat/qat_sym_session.h | 143 ++
drivers/crypto/qat/rte_qat_cryptodev.c | 180 ---
27 files changed, 3042 insertions(+), 2346 deletions(-)
delete mode 100644 drivers/crypto/qat/qat_adf/qat_algs.h
create mode 100644 drivers/crypto/qat/qat_asym_pmd.c
create mode 100644 drivers/crypto/qat/qat_asym_pmd.h
create mode 100644 drivers/crypto/qat/qat_common.c
create mode 100644 drivers/crypto/qat/qat_common.h
create mode 100644 drivers/crypto/qat/qat_comp_pmd.c
create mode 100644 drivers/crypto/qat/qat_comp_pmd.h
delete mode 100644 drivers/crypto/qat/qat_crypto.c
delete mode 100644 drivers/crypto/qat/qat_crypto.h
create mode 100644 drivers/crypto/qat/qat_device.c
create mode 100644 drivers/crypto/qat/qat_device.h
create mode 100644 drivers/crypto/qat/qat_qp.h
create mode 100644 drivers/crypto/qat/qat_sym.c
create mode 100644 drivers/crypto/qat/qat_sym.h
rename drivers/crypto/qat/{qat_crypto_capabilities.h => qat_sym_capabilities.h} (99%)
create mode 100644 drivers/crypto/qat/qat_sym_pmd.c
create mode 100644 drivers/crypto/qat/qat_sym_pmd.h
rename drivers/crypto/qat/{qat_adf/qat_algs_build_desc.c => qat_sym_session.c} (59%)
create mode 100644 drivers/crypto/qat/qat_sym_session.h
delete mode 100644 drivers/crypto/qat/rte_qat_cryptodev.c
--
2.13.6
^ permalink raw reply [flat|nested] 109+ messages in thread
* [PATCH v2 01/31] crypto/qat: add qat common header
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (31 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 00/31] crypto/qat: refactor to support multiple Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 02/31] crypto/qat: add qat device files Fiona Trahe
` (30 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev
Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak, ArkadiuszX Kusztal
This commit adds qat_common.h header file.
Following objects were moved to it:
qat_algs.h =>. qat_common.h
- struct qat_alg_buf
- struct qat_alg_buf_list
- struct qat_crypto_op_cookie
- QAT_SGL_MAX_NUMBER
qat_crypto.h => qat_common.h
- CRYPTODEV_NAME_QAT_SYM_PMD
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_adf/qat_algs.h | 25 -------------------
drivers/crypto/qat/qat_common.h | 47 +++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_crypto.h | 9 +------
3 files changed, 48 insertions(+), 33 deletions(-)
create mode 100644 drivers/crypto/qat/qat_common.h
diff --git a/drivers/crypto/qat/qat_adf/qat_algs.h b/drivers/crypto/qat/qat_adf/qat_algs.h
index 88bd5f00e..6c49c6529 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs.h
+++ b/drivers/crypto/qat/qat_adf/qat_algs.h
@@ -32,12 +32,6 @@
ICP_QAT_HW_CIPHER_KEY_CONVERT, \
ICP_QAT_HW_CIPHER_DECRYPT)
-struct qat_alg_buf {
- uint32_t len;
- uint32_t resrvd;
- uint64_t addr;
-} __rte_packed;
-
enum qat_crypto_proto_flag {
QAT_CRYPTO_PROTO_FLAG_NONE = 0,
QAT_CRYPTO_PROTO_FLAG_CCM = 1,
@@ -46,25 +40,6 @@ enum qat_crypto_proto_flag {
QAT_CRYPTO_PROTO_FLAG_ZUC = 4
};
-/*
- * Maximum number of SGL entries
- */
-#define QAT_SGL_MAX_NUMBER 16
-
-struct qat_alg_buf_list {
- uint64_t resrvd;
- uint32_t num_bufs;
- uint32_t num_mapped_bufs;
- struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
-} __rte_packed __rte_cache_aligned;
-
-struct qat_crypto_op_cookie {
- struct qat_alg_buf_list qat_sgl_list_src;
- struct qat_alg_buf_list qat_sgl_list_dst;
- rte_iova_t qat_sgl_src_phys_addr;
- rte_iova_t qat_sgl_dst_phys_addr;
-};
-
/* Common content descriptor */
struct qat_alg_cd {
struct icp_qat_hw_cipher_algo_blk cipher;
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
new file mode 100644
index 000000000..293b6f700
--- /dev/null
+++ b/drivers/crypto/qat/qat_common.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_COMMON_H_
+#define _QAT_COMMON_H_
+
+#include <stdint.h>
+
+/**< Intel(R) QAT Symmetric Crypto PMD device name */
+#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+
+/*
+ * Maximum number of SGL entries
+ */
+#define QAT_SGL_MAX_NUMBER 16
+
+/* Intel(R) QuickAssist Technology device generation is enumerated
+ * from one according to the generation of the device
+ */
+
+enum qat_device_gen {
+ QAT_GEN1 = 1,
+ QAT_GEN2,
+};
+
+/**< Common struct for scatter-gather list operations */
+struct qat_alg_buf {
+ uint32_t len;
+ uint32_t resrvd;
+ uint64_t addr;
+} __rte_packed;
+
+struct qat_alg_buf_list {
+ uint64_t resrvd;
+ uint32_t num_bufs;
+ uint32_t num_mapped_bufs;
+ struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
+} __rte_packed __rte_cache_aligned;
+
+struct qat_crypto_op_cookie {
+ struct qat_alg_buf_list qat_sgl_list_src;
+ struct qat_alg_buf_list qat_sgl_list_dst;
+ phys_addr_t qat_sgl_src_phys_addr;
+ phys_addr_t qat_sgl_dst_phys_addr;
+};
+
+#endif /* _QAT_QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 281a142b9..5190d25e5 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -8,11 +8,9 @@
#include <rte_cryptodev_pmd.h>
#include <rte_memzone.h>
+#include "qat_common.h"
#include "qat_crypto_capabilities.h"
-#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
-/**< Intel QAT Symmetric Crypto PMD device name */
-
/*
* This macro rounds up a number to a be a multiple of
* the alignment when the alignment is a power of 2
@@ -30,11 +28,6 @@
struct qat_session;
-enum qat_device_gen {
- QAT_GEN1 = 1,
- QAT_GEN2,
-};
-
/**
* Structure associated with each queue.
*/
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 02/31] crypto/qat: add qat device files
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (32 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 01/31] crypto/qat: add qat common header Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 03/31] crypto/qat: remove unused includes Fiona Trahe
` (29 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev
Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak, ArkadiuszX Kusztal
This commit adds new qat_device file.
Following objects were moved:
qat_crypto.h => qat_device.h
- struct qat_pmd_private
- uint8_t cryptodev_qat_driver_id
- int qat_crypto_sym_qp_release (EXTERN)
- int qat_dev_config()
- int qat_dev_start()
- void qat_dev_stop()
- int qat_dev_close()
- void qat_dev_info_get()
qat_crypto.c => qat_device.c
- int qat_dev_config()
- int qat_dev_start()
- void qat_dev_stop()
- int qat_dev_close()
- void qat_dev_info_get()
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 1 +
drivers/crypto/qat/meson.build | 3 ++-
drivers/crypto/qat/qat_crypto.c | 51 ------------------------------------
drivers/crypto/qat/qat_crypto.h | 22 +---------------
drivers/crypto/qat/qat_device.c | 57 +++++++++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_device.h | 36 ++++++++++++++++++++++++++
6 files changed, 97 insertions(+), 73 deletions(-)
create mode 100644 drivers/crypto/qat/qat_device.c
create mode 100644 drivers/crypto/qat/qat_device.h
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 07266a5e9..6bdd11679 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -22,6 +22,7 @@ LDLIBS += -lrte_pci -lrte_bus_pci
# library source files
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_adf/qat_algs_build_desc.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 006cd6557..51630e31b 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -7,7 +7,8 @@ if not dep.found()
endif
sources = files('qat_crypto.c', 'qat_qp.c',
'qat_adf/qat_algs_build_desc.c',
- 'rte_qat_cryptodev.c')
+ 'rte_qat_cryptodev.c',
+ 'qat_device.c')
includes += include_directories('qat_adf')
deps += ['bus_pci']
ext_deps += dep
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index d9ce2a136..928a50475 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -1609,57 +1609,6 @@ static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
return data - mult;
}
-int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
- __rte_unused struct rte_cryptodev_config *config)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
-}
-
-int qat_dev_close(struct rte_cryptodev *dev)
-{
- int i, ret;
-
- PMD_INIT_FUNC_TRACE();
-
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_crypto_sym_qp_release(dev, i);
- if (ret < 0)
- return ret;
- }
-
- return 0;
-}
-
-void qat_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info)
-{
- struct qat_pmd_private *internals = dev->data->dev_private;
-
- PMD_INIT_FUNC_TRACE();
- if (info != NULL) {
- info->max_nb_queue_pairs =
- ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV;
- info->feature_flags = dev->feature_flags;
- info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = internals->max_nb_sessions;
- info->driver_id = cryptodev_qat_driver_id;
- info->pci_dev = RTE_DEV_TO_PCI(dev->device);
- }
-}
-
void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
{
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 5190d25e5..46f03ccde 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -9,6 +9,7 @@
#include <rte_memzone.h>
#include "qat_common.h"
+#include "qat_device.h"
#include "qat_crypto_capabilities.h"
/*
@@ -64,27 +65,6 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
} __rte_cache_aligned;
-/** private data structure for each QAT device */
-struct qat_pmd_private {
- unsigned max_nb_queue_pairs;
- /**< Max number of queue pairs supported by device */
- unsigned max_nb_sessions;
- /**< Max number of sessions supported by device */
- enum qat_device_gen qat_dev_gen;
- /**< QAT device generation */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
-};
-
-extern uint8_t cryptodev_qat_driver_id;
-
-int qat_dev_config(struct rte_cryptodev *dev,
- struct rte_cryptodev_config *config);
-int qat_dev_start(struct rte_cryptodev *dev);
-void qat_dev_stop(struct rte_cryptodev *dev);
-int qat_dev_close(struct rte_cryptodev *dev);
-void qat_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info);
-
void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
new file mode 100644
index 000000000..c2bf9b7a7
--- /dev/null
+++ b/drivers/crypto/qat/qat_device.c
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_device.h"
+#include "adf_transport_access_macros.h"
+
+int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused struct rte_cryptodev_config *config)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+}
+
+int qat_dev_close(struct rte_cryptodev *dev)
+{
+ int i, ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ ret = qat_crypto_sym_qp_release(dev, i);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+void qat_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info)
+{
+ struct qat_pmd_private *internals = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+ if (info != NULL) {
+ info->max_nb_queue_pairs =
+ ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV;
+ info->feature_flags = dev->feature_flags;
+ info->capabilities = internals->qat_dev_capabilities;
+ info->sym.max_nb_sessions = internals->max_nb_sessions;
+ info->driver_id = cryptodev_qat_driver_id;
+ info->pci_dev = RTE_DEV_TO_PCI(dev->device);
+ }
+}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
new file mode 100644
index 000000000..5c48fdb93
--- /dev/null
+++ b/drivers/crypto/qat/qat_device.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_DEVICE_H_
+#define _QAT_DEVICE_H_
+
+#include <rte_cryptodev_pmd.h>
+#include <rte_bus_pci.h>
+#include "qat_common.h"
+#include "qat_logs.h"
+
+extern uint8_t cryptodev_qat_driver_id;
+
+extern int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+ uint16_t queue_pair_id);
+
+/** private data structure for each QAT device */
+struct qat_pmd_private {
+ unsigned int max_nb_queue_pairs;
+ /**< Max number of queue pairs supported by device */
+ unsigned int max_nb_sessions;
+ /**< Max number of sessions supported by device */
+ enum qat_device_gen qat_dev_gen;
+ /**< QAT device generation */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+};
+
+int qat_dev_config(struct rte_cryptodev *dev,
+ struct rte_cryptodev_config *config);
+int qat_dev_start(struct rte_cryptodev *dev);
+void qat_dev_stop(struct rte_cryptodev *dev);
+int qat_dev_close(struct rte_cryptodev *dev);
+void qat_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info);
+
+#endif /* _QAT_DEVICE_H_ */
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 03/31] crypto/qat: remove unused includes
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (33 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 02/31] crypto/qat: add qat device files Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-06-11 22:20 ` De Lara Guarch, Pablo
2018-05-11 11:13 ` [PATCH v2 04/31] crypto/qat: add symmetric session file Fiona Trahe
` (28 subsequent siblings)
63 siblings, 1 reply; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev
Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak, ArkadiuszX Kusztal
This commit removes unused includes from qat_crypto.c
and removes unused file qat_algs.h
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_adf/qat_algs.h | 101 ----------------------------------
drivers/crypto/qat/qat_crypto.c | 22 --------
2 files changed, 123 deletions(-)
delete mode 100644 drivers/crypto/qat/qat_adf/qat_algs.h
diff --git a/drivers/crypto/qat/qat_adf/qat_algs.h b/drivers/crypto/qat/qat_adf/qat_algs.h
deleted file mode 100644
index 6c49c6529..000000000
--- a/drivers/crypto/qat/qat_adf/qat_algs.h
+++ /dev/null
@@ -1,101 +0,0 @@
-/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
- * Copyright(c) 2015-2018 Intel Corporation
- */
-#ifndef _ICP_QAT_ALGS_H_
-#define _ICP_QAT_ALGS_H_
-#include <rte_memory.h>
-#include <rte_crypto.h>
-#include "icp_qat_hw.h"
-#include "icp_qat_fw.h"
-#include "icp_qat_fw_la.h"
-#include "../qat_crypto.h"
-
-/*
- * Key Modifier (KM) value used in KASUMI algorithm in F9 mode to XOR
- * Integrity Key (IK)
- */
-#define KASUMI_F9_KEY_MODIFIER_4_BYTES 0xAAAAAAAA
-
-#define KASUMI_F8_KEY_MODIFIER_4_BYTES 0x55555555
-
-/* 3DES key sizes */
-#define QAT_3DES_KEY_SZ_OPT1 24 /* Keys are independent */
-#define QAT_3DES_KEY_SZ_OPT2 16 /* K3=K1 */
-
-#define QAT_AES_HW_CONFIG_CBC_ENC(alg) \
- ICP_QAT_HW_CIPHER_CONFIG_BUILD(ICP_QAT_HW_CIPHER_CBC_MODE, alg, \
- ICP_QAT_HW_CIPHER_NO_CONVERT, \
- ICP_QAT_HW_CIPHER_ENCRYPT)
-
-#define QAT_AES_HW_CONFIG_CBC_DEC(alg) \
- ICP_QAT_HW_CIPHER_CONFIG_BUILD(ICP_QAT_HW_CIPHER_CBC_MODE, alg, \
- ICP_QAT_HW_CIPHER_KEY_CONVERT, \
- ICP_QAT_HW_CIPHER_DECRYPT)
-
-enum qat_crypto_proto_flag {
- QAT_CRYPTO_PROTO_FLAG_NONE = 0,
- QAT_CRYPTO_PROTO_FLAG_CCM = 1,
- QAT_CRYPTO_PROTO_FLAG_GCM = 2,
- QAT_CRYPTO_PROTO_FLAG_SNOW3G = 3,
- QAT_CRYPTO_PROTO_FLAG_ZUC = 4
-};
-
-/* Common content descriptor */
-struct qat_alg_cd {
- struct icp_qat_hw_cipher_algo_blk cipher;
- struct icp_qat_hw_auth_algo_blk hash;
-} __rte_packed __rte_cache_aligned;
-
-struct qat_session {
- enum icp_qat_fw_la_cmd_id qat_cmd;
- enum icp_qat_hw_cipher_algo qat_cipher_alg;
- enum icp_qat_hw_cipher_dir qat_dir;
- enum icp_qat_hw_cipher_mode qat_mode;
- enum icp_qat_hw_auth_algo qat_hash_alg;
- enum icp_qat_hw_auth_op auth_op;
- void *bpi_ctx;
- struct qat_alg_cd cd;
- uint8_t *cd_cur_ptr;
- rte_iova_t cd_paddr;
- struct icp_qat_fw_la_bulk_req fw_req;
- uint8_t aad_len;
- struct qat_crypto_instance *inst;
- struct {
- uint16_t offset;
- uint16_t length;
- } cipher_iv;
- struct {
- uint16_t offset;
- uint16_t length;
- } auth_iv;
- uint16_t digest_length;
- rte_spinlock_t lock; /* protects this struct */
- enum qat_device_gen min_qat_dev_gen;
-};
-
-int qat_get_inter_state_size(enum icp_qat_hw_auth_algo qat_hash_alg);
-
-int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cd,
- uint8_t *enckey,
- uint32_t enckeylen);
-
-int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
- uint8_t *authkey,
- uint32_t authkeylen,
- uint32_t aad_length,
- uint32_t digestsize,
- unsigned int operation);
-
-void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
- enum qat_crypto_proto_flag proto_flags);
-
-int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_aes_docsisbpi_key(int key_len,
- enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
-int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-#endif
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index 928a50475..7f2c2c86b 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -2,30 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
-#include <stdio.h>
-#include <stdlib.h>
-#include <strings.h>
-#include <string.h>
-#include <inttypes.h>
-#include <errno.h>
-#include <sys/queue.h>
-#include <stdarg.h>
-
-#include <rte_common.h>
-#include <rte_log.h>
-#include <rte_debug.h>
-#include <rte_memory.h>
-#include <rte_tailq.h>
-#include <rte_malloc.h>
-#include <rte_launch.h>
-#include <rte_eal.h>
-#include <rte_per_lcore.h>
-#include <rte_lcore.h>
-#include <rte_branch_prediction.h>
#include <rte_mempool.h>
#include <rte_mbuf.h>
-#include <rte_string_fns.h>
-#include <rte_spinlock.h>
#include <rte_hexdump.h>
#include <rte_crypto_sym.h>
#include <rte_byteorder.h>
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 04/31] crypto/qat: add symmetric session file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (34 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 03/31] crypto/qat: remove unused includes Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 05/31] crypto/qat: change filename crypto to sym Fiona Trahe
` (27 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev
Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak, ArkadiuszX Kusztal
This commit adds qat_sym_session.c/h files and moves objects
from qat_algs_build_desc and qat_algs.h
Following objects were moved:
qat_adf/qat_algs_build_desc.c => qat_sym_session.c
- all objects -
qat_adf/qat_algs.h => qat_sym_session.h
- enum qat_crypto_proto_flag
- struct qat_alg_cd
- struct qat_session
- int qat_get_inter_state_size()
- int qat_alg_aead_session_create_content_desc_cipher()
- int qat_alg_aead_session_create_content_desc_auth()
- void qat_alg_init_common_hdr()
- int qat_alg_validate_aes_key()
- int qat_alg_validate_aes_docsisbpi_key()
- int qat_alg_validate_snow3g_key()
- int qat_alg_validate_kasumi_key()
- int qat_alg_validate_3des_key()
- int qat_alg_validate_des_key()
- int qat_cipher_get_block_size()
- int qat_alg_validate_zuc_key()
-- all macros
qat_crypto.h => qat_sym_session.h
int qat_crypto_sym_configure_session()
int qat_crypto_set_session_parameters()
int qat_crypto_sym_configure_session_aead()
int qat_crypto_sym_configure_session_cipher()
int qat_crypto_sym_configure_session_auth()
int qat_alg_aead_session_create_content_desc_cipher()
int qat_alg_aead_session_create_content_desc_auth()
static struct rte_crypto_auth_xform qat_get_auth_xform()
static struct rte_crypto_cipher_xform qat_get_cipher_xform()
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 2 +-
drivers/crypto/qat/meson.build | 2 +-
drivers/crypto/qat/qat_crypto.c | 704 +-------------------
drivers/crypto/qat/qat_crypto.h | 36 -
drivers/crypto/qat/qat_qp.c | 1 -
.../qat_algs_build_desc.c => qat_sym_session.c} | 728 ++++++++++++++++++++-
drivers/crypto/qat/qat_sym_session.h | 142 ++++
drivers/crypto/qat/rte_qat_cryptodev.c | 1 +
8 files changed, 865 insertions(+), 751 deletions(-)
rename drivers/crypto/qat/{qat_adf/qat_algs_build_desc.c => qat_sym_session.c} (61%)
create mode 100644 drivers/crypto/qat/qat_sym_session.h
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 6bdd11679..c63c1515e 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -24,7 +24,7 @@ LDLIBS += -lrte_pci -lrte_bus_pci
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_adf/qat_algs_build_desc.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
# export include files
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 51630e31b..be4282a83 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -6,7 +6,7 @@ if not dep.found()
build = false
endif
sources = files('qat_crypto.c', 'qat_qp.c',
- 'qat_adf/qat_algs_build_desc.c',
+ 'qat_sym_session.c',
'rte_qat_cryptodev.c',
'qat_device.c')
includes += include_directories('qat_adf')
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index 7f2c2c86b..96a1b78f0 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -13,7 +13,7 @@
#include <openssl/evp.h>
#include "qat_logs.h"
-#include "qat_algs.h"
+#include "qat_sym_session.h"
#include "qat_crypto.h"
#include "adf_transport_access_macros.h"
@@ -23,46 +23,6 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-static int
-qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
- struct qat_pmd_private *internals) {
- int i = 0;
- const struct rte_cryptodev_capabilities *capability;
-
- while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
- RTE_CRYPTO_OP_TYPE_UNDEFINED) {
- if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
- continue;
-
- if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_CIPHER)
- continue;
-
- if (capability->sym.cipher.algo == algo)
- return 1;
- }
- return 0;
-}
-
-static int
-qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
- struct qat_pmd_private *internals) {
- int i = 0;
- const struct rte_cryptodev_capabilities *capability;
-
- while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
- RTE_CRYPTO_OP_TYPE_UNDEFINED) {
- if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
- continue;
-
- if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_AUTH)
- continue;
-
- if (capability->sym.auth.algo == algo)
- return 1;
- }
- return 0;
-}
-
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -124,49 +84,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
/** Creates a context in either AES or DES in ECB mode
* Depends on openssl libcrypto
*/
-static int
-bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
- enum rte_crypto_cipher_operation direction __rte_unused,
- uint8_t *key, void **ctx)
-{
- const EVP_CIPHER *algo = NULL;
- int ret;
- *ctx = EVP_CIPHER_CTX_new();
-
- if (*ctx == NULL) {
- ret = -ENOMEM;
- goto ctx_init_err;
- }
-
- if (cryptodev_algo == RTE_CRYPTO_CIPHER_DES_DOCSISBPI)
- algo = EVP_des_ecb();
- else
- algo = EVP_aes_128_ecb();
-
- /* IV will be ECB encrypted whether direction is encrypt or decrypt*/
- if (EVP_EncryptInit_ex(*ctx, algo, NULL, key, 0) != 1) {
- ret = -EINVAL;
- goto ctx_init_err;
- }
-
- return 0;
-
-ctx_init_err:
- if (*ctx != NULL)
- EVP_CIPHER_CTX_free(*ctx);
- return ret;
-}
-
-/** Frees a context previously created
- * Depends on openssl libcrypto
- */
-static void
-bpi_cipher_ctx_free(void *bpi_ctx)
-{
- if (bpi_ctx != NULL)
- EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
-}
-
static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
@@ -174,625 +91,6 @@ static inline int
qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
-void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
- struct rte_cryptodev_sym_session *sess)
-{
- PMD_INIT_FUNC_TRACE();
- uint8_t index = dev->driver_id;
- void *sess_priv = get_session_private_data(sess, index);
- struct qat_session *s = (struct qat_session *)sess_priv;
-
- if (sess_priv) {
- if (s->bpi_ctx)
- bpi_cipher_ctx_free(s->bpi_ctx);
- memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
- struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
- set_session_private_data(sess, index, NULL);
- rte_mempool_put(sess_mp, sess_priv);
- }
-}
-
-static int
-qat_get_cmd_id(const struct rte_crypto_sym_xform *xform)
-{
- /* Cipher Only */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
- return ICP_QAT_FW_LA_CMD_CIPHER;
-
- /* Authentication Only */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
- return ICP_QAT_FW_LA_CMD_AUTH;
-
- /* AEAD */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- /* AES-GCM and AES-CCM works with different direction
- * GCM first encrypts and generate hash where AES-CCM
- * first generate hash and encrypts. Similar relation
- * applies to decryption.
- */
- if (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT)
- if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- else
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- else
- if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- else
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- }
-
- if (xform->next == NULL)
- return -1;
-
- /* Cipher then Authenticate */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
- xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH)
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
-
- /* Authenticate then Cipher */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
- xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
-
- return -1;
-}
-
-static struct rte_crypto_auth_xform *
-qat_get_auth_xform(struct rte_crypto_sym_xform *xform)
-{
- do {
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH)
- return &xform->auth;
-
- xform = xform->next;
- } while (xform);
-
- return NULL;
-}
-
-static struct rte_crypto_cipher_xform *
-qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
-{
- do {
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
- return &xform->cipher;
-
- xform = xform->next;
- } while (xform);
-
- return NULL;
-}
-
-int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct qat_pmd_private *internals = dev->data->dev_private;
- struct rte_crypto_cipher_xform *cipher_xform = NULL;
- int ret;
-
- /* Get cipher xform from crypto xform chain */
- cipher_xform = qat_get_cipher_xform(xform);
-
- session->cipher_iv.offset = cipher_xform->iv.offset;
- session->cipher_iv.length = cipher_xform->iv.length;
-
- switch (cipher_xform->algo) {
- case RTE_CRYPTO_CIPHER_AES_CBC:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_AES_CTR:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- break;
- case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
- if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_NULL:
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_KASUMI_F8:
- if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_CBC:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_DES_CBC:
- if (qat_alg_validate_des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_CTR:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- break;
- case RTE_CRYPTO_CIPHER_DES_DOCSISBPI:
- ret = bpi_cipher_ctx_init(
- cipher_xform->algo,
- cipher_xform->op,
- cipher_xform->key.data,
- &session->bpi_ctx);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
- goto error_out;
- }
- if (qat_alg_validate_des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_AES_DOCSISBPI:
- ret = bpi_cipher_ctx_init(
- cipher_xform->algo,
- cipher_xform->op,
- cipher_xform->key.data,
- &session->bpi_ctx);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
- goto error_out;
- }
- if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_ZUC_EEA3:
- if (!qat_is_cipher_alg_supported(
- cipher_xform->algo, internals)) {
- PMD_DRV_LOG(ERR, "%s not supported on this device",
- rte_crypto_cipher_algorithm_strings
- [cipher_xform->algo]);
- ret = -ENOTSUP;
- goto error_out;
- }
- if (qat_alg_validate_zuc_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_ECB:
- case RTE_CRYPTO_CIPHER_AES_ECB:
- case RTE_CRYPTO_CIPHER_AES_F8:
- case RTE_CRYPTO_CIPHER_AES_XTS:
- case RTE_CRYPTO_CIPHER_ARC4:
- PMD_DRV_LOG(ERR, "Crypto QAT PMD: Unsupported Cipher alg %u",
- cipher_xform->algo);
- ret = -ENOTSUP;
- goto error_out;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined Cipher specified %u\n",
- cipher_xform->algo);
- ret = -EINVAL;
- goto error_out;
- }
-
- if (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- else
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- cipher_xform->key.data,
- cipher_xform->key.length)) {
- ret = -EINVAL;
- goto error_out;
- }
-
- return 0;
-
-error_out:
- if (session->bpi_ctx) {
- bpi_cipher_ctx_free(session->bpi_ctx);
- session->bpi_ctx = NULL;
- }
- return ret;
-}
-
-int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct rte_cryptodev_sym_session *sess,
- struct rte_mempool *mempool)
-{
- void *sess_private_data;
- int ret;
-
- if (rte_mempool_get(mempool, &sess_private_data)) {
- CDEV_LOG_ERR(
- "Couldn't get object from session mempool");
- return -ENOMEM;
- }
-
- ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "Crypto QAT PMD: failed to configure "
- "session parameters");
-
- /* Return session to mempool */
- rte_mempool_put(mempool, sess_private_data);
- return ret;
- }
-
- set_session_private_data(sess, dev->driver_id,
- sess_private_data);
-
- return 0;
-}
-
-int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform, void *session_private)
-{
- struct qat_session *session = session_private;
- int ret;
-
- int qat_cmd_id;
- PMD_INIT_FUNC_TRACE();
-
- /* Set context descriptor physical address */
- session->cd_paddr = rte_mempool_virt2iova(session) +
- offsetof(struct qat_session, cd);
-
- session->min_qat_dev_gen = QAT_GEN1;
-
- /* Get requested QAT command id */
- qat_cmd_id = qat_get_cmd_id(xform);
- if (qat_cmd_id < 0 || qat_cmd_id >= ICP_QAT_FW_LA_CMD_DELIMITER) {
- PMD_DRV_LOG(ERR, "Unsupported xform chain requested");
- return -ENOTSUP;
- }
- session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
- switch (session->qat_cmd) {
- case ICP_QAT_FW_LA_CMD_CIPHER:
- ret = qat_crypto_sym_configure_session_cipher(dev, xform, session);
- if (ret < 0)
- return ret;
- break;
- case ICP_QAT_FW_LA_CMD_AUTH:
- ret = qat_crypto_sym_configure_session_auth(dev, xform, session);
- if (ret < 0)
- return ret;
- break;
- case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
- session);
- if (ret < 0)
- return ret;
- } else {
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
- if (ret < 0)
- return ret;
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
- if (ret < 0)
- return ret;
- }
- break;
- case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
- session);
- if (ret < 0)
- return ret;
- } else {
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
- if (ret < 0)
- return ret;
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
- if (ret < 0)
- return ret;
- }
- break;
- case ICP_QAT_FW_LA_CMD_TRNG_GET_RANDOM:
- case ICP_QAT_FW_LA_CMD_TRNG_TEST:
- case ICP_QAT_FW_LA_CMD_SSL3_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_TLS_V1_1_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_TLS_V1_2_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_MGF1:
- case ICP_QAT_FW_LA_CMD_AUTH_PRE_COMP:
- case ICP_QAT_FW_LA_CMD_CIPHER_PRE_COMP:
- case ICP_QAT_FW_LA_CMD_DELIMITER:
- PMD_DRV_LOG(ERR, "Unsupported Service %u",
- session->qat_cmd);
- return -ENOTSUP;
- default:
- PMD_DRV_LOG(ERR, "Unsupported Service %u",
- session->qat_cmd);
- return -ENOTSUP;
- }
-
- return 0;
-}
-
-int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct rte_crypto_auth_xform *auth_xform = NULL;
- struct qat_pmd_private *internals = dev->data->dev_private;
- auth_xform = qat_get_auth_xform(xform);
- uint8_t *key_data = auth_xform->key.data;
- uint8_t key_length = auth_xform->key.length;
-
- switch (auth_xform->algo) {
- case RTE_CRYPTO_AUTH_SHA1_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA1;
- break;
- case RTE_CRYPTO_AUTH_SHA224_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA224;
- break;
- case RTE_CRYPTO_AUTH_SHA256_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA256;
- break;
- case RTE_CRYPTO_AUTH_SHA384_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA384;
- break;
- case RTE_CRYPTO_AUTH_SHA512_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA512;
- break;
- case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
- break;
- case RTE_CRYPTO_AUTH_AES_GMAC:
- if (qat_alg_validate_aes_key(auth_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
-
- break;
- case RTE_CRYPTO_AUTH_SNOW3G_UIA2:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SNOW_3G_UIA2;
- break;
- case RTE_CRYPTO_AUTH_MD5_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_MD5;
- break;
- case RTE_CRYPTO_AUTH_NULL:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_NULL;
- break;
- case RTE_CRYPTO_AUTH_KASUMI_F9:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_KASUMI_F9;
- break;
- case RTE_CRYPTO_AUTH_ZUC_EIA3:
- if (!qat_is_auth_alg_supported(auth_xform->algo, internals)) {
- PMD_DRV_LOG(ERR, "%s not supported on this device",
- rte_crypto_auth_algorithm_strings
- [auth_xform->algo]);
- return -ENOTSUP;
- }
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_ZUC_3G_128_EIA3;
- break;
- case RTE_CRYPTO_AUTH_SHA1:
- case RTE_CRYPTO_AUTH_SHA256:
- case RTE_CRYPTO_AUTH_SHA512:
- case RTE_CRYPTO_AUTH_SHA224:
- case RTE_CRYPTO_AUTH_SHA384:
- case RTE_CRYPTO_AUTH_MD5:
- case RTE_CRYPTO_AUTH_AES_CMAC:
- case RTE_CRYPTO_AUTH_AES_CBC_MAC:
- PMD_DRV_LOG(ERR, "Crypto: Unsupported hash alg %u",
- auth_xform->algo);
- return -ENOTSUP;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined Hash algo %u specified",
- auth_xform->algo);
- return -EINVAL;
- }
-
- session->auth_iv.offset = auth_xform->iv.offset;
- session->auth_iv.length = auth_xform->iv.length;
-
- if (auth_xform->algo == RTE_CRYPTO_AUTH_AES_GMAC) {
- if (auth_xform->op == RTE_CRYPTO_AUTH_OP_GENERATE) {
- session->qat_cmd = ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- /*
- * It needs to create cipher desc content first,
- * then authentication
- */
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- auth_xform->key.data,
- auth_xform->key.length))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
- } else {
- session->qat_cmd = ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- /*
- * It needs to create authentication desc content first,
- * then cipher
- */
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- auth_xform->key.data,
- auth_xform->key.length))
- return -EINVAL;
- }
- /* Restore to authentication only only */
- session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
- } else {
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
- }
-
- session->digest_length = auth_xform->digest_length;
- return 0;
-}
-
-int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct rte_crypto_aead_xform *aead_xform = &xform->aead;
- enum rte_crypto_auth_operation crypto_operation;
-
- /*
- * Store AEAD IV parameters as cipher IV,
- * to avoid unnecessary memory usage
- */
- session->cipher_iv.offset = xform->aead.iv.offset;
- session->cipher_iv.length = xform->aead.iv.length;
-
- switch (aead_xform->algo) {
- case RTE_CRYPTO_AEAD_AES_GCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
- break;
- case RTE_CRYPTO_AEAD_AES_CCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC;
- break;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined AEAD specified %u\n",
- aead_xform->algo);
- return -EINVAL;
- }
-
- if ((aead_xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT &&
- aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM) ||
- (aead_xform->op == RTE_CRYPTO_AEAD_OP_DECRYPT &&
- aead_xform->algo == RTE_CRYPTO_AEAD_AES_CCM)) {
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- /*
- * It needs to create cipher desc content first,
- * then authentication
- */
-
- crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
- RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- aead_xform->key.data,
- aead_xform->key.length))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- aead_xform->key.data,
- aead_xform->key.length,
- aead_xform->aad_length,
- aead_xform->digest_length,
- crypto_operation))
- return -EINVAL;
- } else {
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- /*
- * It needs to create authentication desc content first,
- * then cipher
- */
-
- crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
- RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- aead_xform->key.data,
- aead_xform->key.length,
- aead_xform->aad_length,
- aead_xform->digest_length,
- crypto_operation))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- aead_xform->key.data,
- aead_xform->key.length))
- return -EINVAL;
- }
-
- session->digest_length = aead_xform->digest_length;
- return 0;
-}
-
-unsigned qat_crypto_sym_get_session_private_size(
- struct rte_cryptodev *dev __rte_unused)
-{
- return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
-}
-
static inline uint32_t
qat_bpicipher_preprocess(struct qat_session *ctx,
struct rte_crypto_op *op)
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 46f03ccde..e3873171c 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -75,42 +75,6 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-int
-qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
- unsigned nb_objs, unsigned obj_cache_size, int socket_id);
-
-extern unsigned
-qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
-
-extern int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct rte_cryptodev_sym_session *sess,
- struct rte_mempool *mempool);
-
-
-int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform, void *session_private);
-
-int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-
-extern void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
- struct rte_cryptodev_sym_session *session);
extern uint16_t
qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 7fea10c76..ee3b30a36 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -14,7 +14,6 @@
#include "qat_logs.h"
#include "qat_crypto.h"
-#include "qat_algs.h"
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
diff --git a/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c b/drivers/crypto/qat/qat_sym_session.c
similarity index 61%
rename from drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
rename to drivers/crypto/qat/qat_sym_session.c
index c87ed40fe..5caac5a6f 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -9,13 +9,724 @@
#include <rte_malloc.h>
#include <rte_crypto_sym.h>
-#include "../qat_logs.h"
+#include "qat_logs.h"
+#include "qat_device.h"
#include <openssl/sha.h> /* Needed to calculate pre-compute values */
#include <openssl/aes.h> /* Needed to calculate pre-compute values */
#include <openssl/md5.h> /* Needed to calculate pre-compute values */
+#include <openssl/evp.h>
-#include "qat_algs.h"
+#include "qat_sym_session.h"
+
+/** Frees a context previously created
+ * Depends on openssl libcrypto
+ */
+static void
+bpi_cipher_ctx_free(void *bpi_ctx)
+{
+ if (bpi_ctx != NULL)
+ EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
+}
+
+static int
+bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
+ enum rte_crypto_cipher_operation direction __rte_unused,
+ uint8_t *key, void **ctx)
+{
+ const EVP_CIPHER *algo = NULL;
+ int ret;
+ *ctx = EVP_CIPHER_CTX_new();
+
+ if (*ctx == NULL) {
+ ret = -ENOMEM;
+ goto ctx_init_err;
+ }
+
+ if (cryptodev_algo == RTE_CRYPTO_CIPHER_DES_DOCSISBPI)
+ algo = EVP_des_ecb();
+ else
+ algo = EVP_aes_128_ecb();
+
+ /* IV will be ECB encrypted whether direction is encrypt or decrypt*/
+ if (EVP_EncryptInit_ex(*ctx, algo, NULL, key, 0) != 1) {
+ ret = -EINVAL;
+ goto ctx_init_err;
+ }
+
+ return 0;
+
+ctx_init_err:
+ if (*ctx != NULL)
+ EVP_CIPHER_CTX_free(*ctx);
+ return ret;
+}
+
+static int
+qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
+ struct qat_pmd_private *internals)
+{
+ int i = 0;
+ const struct rte_cryptodev_capabilities *capability;
+
+ while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
+ RTE_CRYPTO_OP_TYPE_UNDEFINED) {
+ if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
+ continue;
+
+ if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_CIPHER)
+ continue;
+
+ if (capability->sym.cipher.algo == algo)
+ return 1;
+ }
+ return 0;
+}
+
+static int
+qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
+ struct qat_pmd_private *internals)
+{
+ int i = 0;
+ const struct rte_cryptodev_capabilities *capability;
+
+ while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
+ RTE_CRYPTO_OP_TYPE_UNDEFINED) {
+ if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
+ continue;
+
+ if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_AUTH)
+ continue;
+
+ if (capability->sym.auth.algo == algo)
+ return 1;
+ }
+ return 0;
+}
+
+void
+qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess)
+{
+ PMD_INIT_FUNC_TRACE();
+ uint8_t index = dev->driver_id;
+ void *sess_priv = get_session_private_data(sess, index);
+ struct qat_session *s = (struct qat_session *)sess_priv;
+
+ if (sess_priv) {
+ if (s->bpi_ctx)
+ bpi_cipher_ctx_free(s->bpi_ctx);
+ memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
+ struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
+
+ set_session_private_data(sess, index, NULL);
+ rte_mempool_put(sess_mp, sess_priv);
+ }
+}
+
+static int
+qat_get_cmd_id(const struct rte_crypto_sym_xform *xform)
+{
+ /* Cipher Only */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
+ return ICP_QAT_FW_LA_CMD_CIPHER;
+
+ /* Authentication Only */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
+ return ICP_QAT_FW_LA_CMD_AUTH;
+
+ /* AEAD */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ /* AES-GCM and AES-CCM works with different direction
+ * GCM first encrypts and generate hash where AES-CCM
+ * first generate hash and encrypts. Similar relation
+ * applies to decryption.
+ */
+ if (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT)
+ if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ else
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ else
+ if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ else
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ }
+
+ if (xform->next == NULL)
+ return -1;
+
+ /* Cipher then Authenticate */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
+ xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH)
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+
+ /* Authenticate then Cipher */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
+ xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+
+ return -1;
+}
+
+static struct rte_crypto_auth_xform *
+qat_get_auth_xform(struct rte_crypto_sym_xform *xform)
+{
+ do {
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH)
+ return &xform->auth;
+
+ xform = xform->next;
+ } while (xform);
+
+ return NULL;
+}
+
+static struct rte_crypto_cipher_xform *
+qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
+{
+ do {
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
+ return &xform->cipher;
+
+ xform = xform->next;
+ } while (xform);
+
+ return NULL;
+}
+
+int
+qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct qat_pmd_private *internals = dev->data->dev_private;
+ struct rte_crypto_cipher_xform *cipher_xform = NULL;
+ int ret;
+
+ /* Get cipher xform from crypto xform chain */
+ cipher_xform = qat_get_cipher_xform(xform);
+
+ session->cipher_iv.offset = cipher_xform->iv.offset;
+ session->cipher_iv.length = cipher_xform->iv.length;
+
+ switch (cipher_xform->algo) {
+ case RTE_CRYPTO_CIPHER_AES_CBC:
+ if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_AES_CTR:
+ if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
+ if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_NULL:
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_KASUMI_F8:
+ if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_CBC:
+ if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_DES_CBC:
+ if (qat_alg_validate_des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_CTR:
+ if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_DES_DOCSISBPI:
+ ret = bpi_cipher_ctx_init(
+ cipher_xform->algo,
+ cipher_xform->op,
+ cipher_xform->key.data,
+ &session->bpi_ctx);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
+ goto error_out;
+ }
+ if (qat_alg_validate_des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_AES_DOCSISBPI:
+ ret = bpi_cipher_ctx_init(
+ cipher_xform->algo,
+ cipher_xform->op,
+ cipher_xform->key.data,
+ &session->bpi_ctx);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
+ goto error_out;
+ }
+ if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_ZUC_EEA3:
+ if (!qat_is_cipher_alg_supported(
+ cipher_xform->algo, internals)) {
+ PMD_DRV_LOG(ERR, "%s not supported on this device",
+ rte_crypto_cipher_algorithm_strings
+ [cipher_xform->algo]);
+ ret = -ENOTSUP;
+ goto error_out;
+ }
+ if (qat_alg_validate_zuc_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_ECB:
+ case RTE_CRYPTO_CIPHER_AES_ECB:
+ case RTE_CRYPTO_CIPHER_AES_F8:
+ case RTE_CRYPTO_CIPHER_AES_XTS:
+ case RTE_CRYPTO_CIPHER_ARC4:
+ PMD_DRV_LOG(ERR, "Crypto QAT PMD: Unsupported Cipher alg %u",
+ cipher_xform->algo);
+ ret = -ENOTSUP;
+ goto error_out;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined Cipher specified %u\n",
+ cipher_xform->algo);
+ ret = -EINVAL;
+ goto error_out;
+ }
+
+ if (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ else
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ cipher_xform->key.data,
+ cipher_xform->key.length)) {
+ ret = -EINVAL;
+ goto error_out;
+ }
+
+ return 0;
+
+error_out:
+ if (session->bpi_ctx) {
+ bpi_cipher_ctx_free(session->bpi_ctx);
+ session->bpi_ctx = NULL;
+ }
+ return ret;
+}
+
+int
+qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct rte_cryptodev_sym_session *sess,
+ struct rte_mempool *mempool)
+{
+ void *sess_private_data;
+ int ret;
+
+ if (rte_mempool_get(mempool, &sess_private_data)) {
+ CDEV_LOG_ERR(
+ "Couldn't get object from session mempool");
+ return -ENOMEM;
+ }
+
+ ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR,
+ "Crypto QAT PMD: failed to configure session parameters");
+
+ /* Return session to mempool */
+ rte_mempool_put(mempool, sess_private_data);
+ return ret;
+ }
+
+ set_session_private_data(sess, dev->driver_id,
+ sess_private_data);
+
+ return 0;
+}
+
+int
+qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform, void *session_private)
+{
+ struct qat_session *session = session_private;
+ int ret;
+ int qat_cmd_id;
+
+ PMD_INIT_FUNC_TRACE();
+
+ /* Set context descriptor physical address */
+ session->cd_paddr = rte_mempool_virt2iova(session) +
+ offsetof(struct qat_session, cd);
+
+ session->min_qat_dev_gen = QAT_GEN1;
+
+ /* Get requested QAT command id */
+ qat_cmd_id = qat_get_cmd_id(xform);
+ if (qat_cmd_id < 0 || qat_cmd_id >= ICP_QAT_FW_LA_CMD_DELIMITER) {
+ PMD_DRV_LOG(ERR, "Unsupported xform chain requested");
+ return -ENOTSUP;
+ }
+ session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
+ switch (session->qat_cmd) {
+ case ICP_QAT_FW_LA_CMD_CIPHER:
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ break;
+ case ICP_QAT_FW_LA_CMD_AUTH:
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ break;
+ case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ ret = qat_crypto_sym_configure_session_aead(xform,
+ session);
+ if (ret < 0)
+ return ret;
+ } else {
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ }
+ break;
+ case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ ret = qat_crypto_sym_configure_session_aead(xform,
+ session);
+ if (ret < 0)
+ return ret;
+ } else {
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ }
+ break;
+ case ICP_QAT_FW_LA_CMD_TRNG_GET_RANDOM:
+ case ICP_QAT_FW_LA_CMD_TRNG_TEST:
+ case ICP_QAT_FW_LA_CMD_SSL3_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_TLS_V1_1_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_TLS_V1_2_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_MGF1:
+ case ICP_QAT_FW_LA_CMD_AUTH_PRE_COMP:
+ case ICP_QAT_FW_LA_CMD_CIPHER_PRE_COMP:
+ case ICP_QAT_FW_LA_CMD_DELIMITER:
+ PMD_DRV_LOG(ERR, "Unsupported Service %u",
+ session->qat_cmd);
+ return -ENOTSUP;
+ default:
+ PMD_DRV_LOG(ERR, "Unsupported Service %u",
+ session->qat_cmd);
+ return -ENOTSUP;
+ }
+
+ return 0;
+}
+
+int
+qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
+ struct qat_pmd_private *internals = dev->data->dev_private;
+ uint8_t *key_data = auth_xform->key.data;
+ uint8_t key_length = auth_xform->key.length;
+
+ switch (auth_xform->algo) {
+ case RTE_CRYPTO_AUTH_SHA1_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA1;
+ break;
+ case RTE_CRYPTO_AUTH_SHA224_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA224;
+ break;
+ case RTE_CRYPTO_AUTH_SHA256_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA256;
+ break;
+ case RTE_CRYPTO_AUTH_SHA384_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA384;
+ break;
+ case RTE_CRYPTO_AUTH_SHA512_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA512;
+ break;
+ case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
+ break;
+ case RTE_CRYPTO_AUTH_AES_GMAC:
+ if (qat_alg_validate_aes_key(auth_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
+
+ break;
+ case RTE_CRYPTO_AUTH_SNOW3G_UIA2:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SNOW_3G_UIA2;
+ break;
+ case RTE_CRYPTO_AUTH_MD5_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_MD5;
+ break;
+ case RTE_CRYPTO_AUTH_NULL:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_NULL;
+ break;
+ case RTE_CRYPTO_AUTH_KASUMI_F9:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_KASUMI_F9;
+ break;
+ case RTE_CRYPTO_AUTH_ZUC_EIA3:
+ if (!qat_is_auth_alg_supported(auth_xform->algo, internals)) {
+ PMD_DRV_LOG(ERR, "%s not supported on this device",
+ rte_crypto_auth_algorithm_strings
+ [auth_xform->algo]);
+ return -ENOTSUP;
+ }
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_ZUC_3G_128_EIA3;
+ break;
+ case RTE_CRYPTO_AUTH_SHA1:
+ case RTE_CRYPTO_AUTH_SHA256:
+ case RTE_CRYPTO_AUTH_SHA512:
+ case RTE_CRYPTO_AUTH_SHA224:
+ case RTE_CRYPTO_AUTH_SHA384:
+ case RTE_CRYPTO_AUTH_MD5:
+ case RTE_CRYPTO_AUTH_AES_CMAC:
+ case RTE_CRYPTO_AUTH_AES_CBC_MAC:
+ PMD_DRV_LOG(ERR, "Crypto: Unsupported hash alg %u",
+ auth_xform->algo);
+ return -ENOTSUP;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined Hash algo %u specified",
+ auth_xform->algo);
+ return -EINVAL;
+ }
+
+ session->auth_iv.offset = auth_xform->iv.offset;
+ session->auth_iv.length = auth_xform->iv.length;
+
+ if (auth_xform->algo == RTE_CRYPTO_AUTH_AES_GMAC) {
+ if (auth_xform->op == RTE_CRYPTO_AUTH_OP_GENERATE) {
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ /*
+ * It needs to create cipher desc content first,
+ * then authentication
+ */
+ if (qat_alg_aead_session_create_content_desc_cipher(
+ session,
+ auth_xform->key.data,
+ auth_xform->key.length))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_auth(
+ session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+ } else {
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+ /*
+ * It needs to create authentication desc content first,
+ * then cipher
+ */
+ if (qat_alg_aead_session_create_content_desc_auth(
+ session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(
+ session,
+ auth_xform->key.data,
+ auth_xform->key.length))
+ return -EINVAL;
+ }
+ /* Restore to authentication only only */
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
+ } else {
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+ }
+
+ session->digest_length = auth_xform->digest_length;
+ return 0;
+}
+
+int
+qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct rte_crypto_aead_xform *aead_xform = &xform->aead;
+ enum rte_crypto_auth_operation crypto_operation;
+
+ /*
+ * Store AEAD IV parameters as cipher IV,
+ * to avoid unnecessary memory usage
+ */
+ session->cipher_iv.offset = xform->aead.iv.offset;
+ session->cipher_iv.length = xform->aead.iv.length;
+
+ switch (aead_xform->algo) {
+ case RTE_CRYPTO_AEAD_AES_GCM:
+ if (qat_alg_validate_aes_key(aead_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
+ break;
+ case RTE_CRYPTO_AEAD_AES_CCM:
+ if (qat_alg_validate_aes_key(aead_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC;
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined AEAD specified %u\n",
+ aead_xform->algo);
+ return -EINVAL;
+ }
+
+ if ((aead_xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT &&
+ aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM) ||
+ (aead_xform->op == RTE_CRYPTO_AEAD_OP_DECRYPT &&
+ aead_xform->algo == RTE_CRYPTO_AEAD_AES_CCM)) {
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ /*
+ * It needs to create cipher desc content first,
+ * then authentication
+ */
+ crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
+ RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ aead_xform->key.data,
+ aead_xform->key.length))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ aead_xform->key.data,
+ aead_xform->key.length,
+ aead_xform->aad_length,
+ aead_xform->digest_length,
+ crypto_operation))
+ return -EINVAL;
+ } else {
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+ /*
+ * It needs to create authentication desc content first,
+ * then cipher
+ */
+
+ crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
+ RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
+
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ aead_xform->key.data,
+ aead_xform->key.length,
+ aead_xform->aad_length,
+ aead_xform->digest_length,
+ crypto_operation))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ aead_xform->key.data,
+ aead_xform->key.length))
+ return -EINVAL;
+ }
+
+ session->digest_length = aead_xform->digest_length;
+ return 0;
+}
+
+unsigned int qat_crypto_sym_get_session_private_size(
+ struct rte_cryptodev *dev __rte_unused)
+{
+ return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
+}
/* returns block size in bytes per cipher algo */
int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg)
@@ -77,12 +788,12 @@ static int qat_hash_get_state1_size(enum icp_qat_hw_auth_algo qat_hash_alg)
case ICP_QAT_HW_AUTH_ALGO_KASUMI_F9:
return QAT_HW_ROUND_UP(ICP_QAT_HW_KASUMI_F9_STATE1_SZ,
QAT_HW_DEFAULT_ALIGNMENT);
- case ICP_QAT_HW_AUTH_ALGO_NULL:
- return QAT_HW_ROUND_UP(ICP_QAT_HW_NULL_STATE1_SZ,
- QAT_HW_DEFAULT_ALIGNMENT);
case ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC:
return QAT_HW_ROUND_UP(ICP_QAT_HW_AES_CBC_MAC_STATE1_SZ,
QAT_HW_DEFAULT_ALIGNMENT);
+ case ICP_QAT_HW_AUTH_ALGO_NULL:
+ return QAT_HW_ROUND_UP(ICP_QAT_HW_NULL_STATE1_SZ,
+ QAT_HW_DEFAULT_ALIGNMENT);
case ICP_QAT_HW_AUTH_ALGO_DELIMITER:
/* return maximum state1 size in this case */
return QAT_HW_ROUND_UP(ICP_QAT_HW_SHA512_STATE1_SZ,
@@ -854,14 +1565,13 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
if (aad_length > 0) {
aad_length += ICP_QAT_HW_CCM_AAD_B0_LEN +
- ICP_QAT_HW_CCM_AAD_LEN_INFO;
+ ICP_QAT_HW_CCM_AAD_LEN_INFO;
auth_param->u2.aad_sz =
- RTE_ALIGN_CEIL(aad_length,
- ICP_QAT_HW_CCM_AAD_ALIGNMENT);
+ RTE_ALIGN_CEIL(aad_length,
+ ICP_QAT_HW_CCM_AAD_ALIGNMENT);
} else {
auth_param->u2.aad_sz = ICP_QAT_HW_CCM_AAD_B0_LEN;
}
-
cdesc->aad_len = aad_length;
hash->auth_counter.counter = 0;
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
new file mode 100644
index 000000000..f90b1821d
--- /dev/null
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -0,0 +1,142 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_SYM_SESSION_H_
+#define _QAT_SYM_SESSION_H_
+
+#include <rte_crypto.h>
+#include <rte_cryptodev_pmd.h>
+
+#include "qat_common.h"
+#include "icp_qat_hw.h"
+#include "icp_qat_fw.h"
+#include "icp_qat_fw_la.h"
+
+/*
+ * Key Modifier (KM) value used in KASUMI algorithm in F9 mode to XOR
+ * Integrity Key (IK)
+ */
+#define KASUMI_F9_KEY_MODIFIER_4_BYTES 0xAAAAAAAA
+
+#define KASUMI_F8_KEY_MODIFIER_4_BYTES 0x55555555
+
+/* 3DES key sizes */
+#define QAT_3DES_KEY_SZ_OPT1 24 /* Keys are independent */
+#define QAT_3DES_KEY_SZ_OPT2 16 /* K3=K1 */
+
+#define QAT_AES_HW_CONFIG_CBC_ENC(alg) \
+ ICP_QAT_HW_CIPHER_CONFIG_BUILD(ICP_QAT_HW_CIPHER_CBC_MODE, alg, \
+ ICP_QAT_HW_CIPHER_NO_CONVERT, \
+ ICP_QAT_HW_CIPHER_ENCRYPT)
+
+#define QAT_AES_HW_CONFIG_CBC_DEC(alg) \
+ ICP_QAT_HW_CIPHER_CONFIG_BUILD(ICP_QAT_HW_CIPHER_CBC_MODE, alg, \
+ ICP_QAT_HW_CIPHER_KEY_CONVERT, \
+ ICP_QAT_HW_CIPHER_DECRYPT)
+
+enum qat_crypto_proto_flag {
+ QAT_CRYPTO_PROTO_FLAG_NONE = 0,
+ QAT_CRYPTO_PROTO_FLAG_CCM = 1,
+ QAT_CRYPTO_PROTO_FLAG_GCM = 2,
+ QAT_CRYPTO_PROTO_FLAG_SNOW3G = 3,
+ QAT_CRYPTO_PROTO_FLAG_ZUC = 4
+};
+
+/* Common content descriptor */
+struct qat_alg_cd {
+ struct icp_qat_hw_cipher_algo_blk cipher;
+ struct icp_qat_hw_auth_algo_blk hash;
+} __rte_packed __rte_cache_aligned;
+
+struct qat_session {
+ enum icp_qat_fw_la_cmd_id qat_cmd;
+ enum icp_qat_hw_cipher_algo qat_cipher_alg;
+ enum icp_qat_hw_cipher_dir qat_dir;
+ enum icp_qat_hw_cipher_mode qat_mode;
+ enum icp_qat_hw_auth_algo qat_hash_alg;
+ enum icp_qat_hw_auth_op auth_op;
+ void *bpi_ctx;
+ struct qat_alg_cd cd;
+ uint8_t *cd_cur_ptr;
+ phys_addr_t cd_paddr;
+ struct icp_qat_fw_la_bulk_req fw_req;
+ uint8_t aad_len;
+ struct qat_crypto_instance *inst;
+ struct {
+ uint16_t offset;
+ uint16_t length;
+ } cipher_iv;
+ struct {
+ uint16_t offset;
+ uint16_t length;
+ } auth_iv;
+ uint16_t digest_length;
+ rte_spinlock_t lock; /* protects this struct */
+ enum qat_device_gen min_qat_dev_gen;
+};
+
+int
+qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct rte_cryptodev_sym_session *sess,
+ struct rte_mempool *mempool);
+
+int
+qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform, void *session_private);
+
+int
+qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
+
+int
+qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
+
+int
+qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
+
+int
+qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cd,
+ uint8_t *enckey,
+ uint32_t enckeylen);
+
+int
+qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+ uint8_t *authkey,
+ uint32_t authkeylen,
+ uint32_t aad_length,
+ uint32_t digestsize,
+ unsigned int operation);
+
+int
+qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
+ unsigned int nb_objs, unsigned int obj_cache_size, int socket_id);
+
+void
+qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *session);
+
+unsigned int
+qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
+
+int qat_get_inter_state_size(enum icp_qat_hw_auth_algo qat_hash_alg);
+
+
+void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
+ enum qat_crypto_proto_flag proto_flags);
+
+int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_aes_docsisbpi_key(int key_len,
+ enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
+int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+
+#endif /* _QAT_SYM_SESSION_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index c8da07af6..e425eb43f 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -10,6 +10,7 @@
#include <rte_cryptodev_pmd.h>
#include "qat_crypto.h"
+#include "qat_sym_session.h"
#include "qat_logs.h"
uint8_t cryptodev_qat_driver_id;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 05/31] crypto/qat: change filename crypto to sym
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (35 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 04/31] crypto/qat: add symmetric session file Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 06/31] crypto/qat: rename fns for consistency Fiona Trahe
` (26 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev
Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak, ArkadiuszX Kusztal
This commit renames qat_crypto.c/h to qat_sym.c/h
And makes a few whitespace changes to resolve line-length
issues.
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 2 +-
drivers/crypto/qat/meson.build | 2 +-
drivers/crypto/qat/qat_qp.c | 2 +-
drivers/crypto/qat/{qat_crypto.c => qat_sym.c} | 60 +++++++++++++-------------
drivers/crypto/qat/{qat_crypto.h => qat_sym.h} | 0
drivers/crypto/qat/rte_qat_cryptodev.c | 2 +-
6 files changed, 33 insertions(+), 35 deletions(-)
rename drivers/crypto/qat/{qat_crypto.c => qat_sym.c} (95%)
rename drivers/crypto/qat/{qat_crypto.h => qat_sym.h} (100%)
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index c63c1515e..8cb802b9d 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -21,7 +21,7 @@ LDLIBS += -lrte_cryptodev
LDLIBS += -lrte_pci -lrte_bus_pci
# library source files
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index be4282a83..e596006da 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -5,7 +5,7 @@ dep = dependency('libcrypto', required: false)
if not dep.found()
build = false
endif
-sources = files('qat_crypto.c', 'qat_qp.c',
+sources = files('qat_sym.c', 'qat_qp.c',
'qat_sym_session.c',
'rte_qat_cryptodev.c',
'qat_device.c')
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index ee3b30a36..794a8d7c9 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,7 +13,7 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_sym.c
similarity index 95%
rename from drivers/crypto/qat/qat_crypto.c
rename to drivers/crypto/qat/qat_sym.c
index 96a1b78f0..f5d542ae3 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -6,15 +6,14 @@
#include <rte_mbuf.h>
#include <rte_hexdump.h>
#include <rte_crypto_sym.h>
-#include <rte_byteorder.h>
-#include <rte_pci.h>
#include <rte_bus_pci.h>
+#include <rte_byteorder.h>
#include <openssl/evp.h>
#include "qat_logs.h"
#include "qat_sym_session.h"
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
#define BYTE_LENGTH 8
@@ -500,8 +499,6 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
-
-
qat_req = (struct icp_qat_fw_la_bulk_req *)out_msg;
rte_mov128((uint8_t *)qat_req, (const uint8_t *)&(ctx->fw_req));
qat_req->comn_mid.opaque_data = (uint64_t)(uintptr_t)op;
@@ -512,11 +509,11 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
ctx->qat_cmd == ICP_QAT_FW_LA_CMD_CIPHER_HASH) {
/* AES-GCM or AES-CCM */
if (ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_128 ||
- ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_64 ||
- (ctx->qat_cipher_alg == ICP_QAT_HW_CIPHER_ALGO_AES128
- && ctx->qat_mode == ICP_QAT_HW_CIPHER_CTR_MODE
- && ctx->qat_hash_alg ==
- ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC)) {
+ ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_64 ||
+ (ctx->qat_cipher_alg == ICP_QAT_HW_CIPHER_ALGO_AES128
+ && ctx->qat_mode == ICP_QAT_HW_CIPHER_CTR_MODE
+ && ctx->qat_hash_alg ==
+ ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC)) {
do_aead = 1;
} else {
do_auth = 1;
@@ -642,7 +639,6 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
qat_req->comn_hdr.serv_specif_flags,
ICP_QAT_FW_LA_GCM_IV_LEN_12_OCTETS);
}
-
set_cipher_iv(ctx->cipher_iv.length,
ctx->cipher_iv.offset,
cipher_param, op, qat_req);
@@ -650,15 +646,14 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
} else if (ctx->qat_hash_alg ==
ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC) {
- /* In case of AES-CCM this may point to user selected memory
- * or iv offset in cypto_op
+ /* In case of AES-CCM this may point to user selected
+ * memory or iv offset in cypto_op
*/
uint8_t *aad_data = op->sym->aead.aad.data;
/* This is true AAD length, it not includes 18 bytes of
* preceding data
*/
uint8_t aad_ccm_real_len = 0;
-
uint8_t aad_len_field_sz = 0;
uint32_t msg_len_be =
rte_bswap32(op->sym->aead.data.length);
@@ -670,33 +665,36 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
ICP_QAT_HW_CCM_AAD_LEN_INFO;
} else {
/*
- * aad_len not greater than 18, so no actual aad data,
- * then use IV after op for B0 block
+ * aad_len not greater than 18, so no actual aad
+ * data, then use IV after op for B0 block
*/
- aad_data = rte_crypto_op_ctod_offset(op, uint8_t *,
+ aad_data = rte_crypto_op_ctod_offset(op,
+ uint8_t *,
ctx->cipher_iv.offset);
aad_phys_addr_aead =
rte_crypto_op_ctophys_offset(op,
- ctx->cipher_iv.offset);
+ ctx->cipher_iv.offset);
}
- uint8_t q = ICP_QAT_HW_CCM_NQ_CONST - ctx->cipher_iv.length;
+ uint8_t q = ICP_QAT_HW_CCM_NQ_CONST -
+ ctx->cipher_iv.length;
- aad_data[0] = ICP_QAT_HW_CCM_BUILD_B0_FLAGS(aad_len_field_sz,
+ aad_data[0] = ICP_QAT_HW_CCM_BUILD_B0_FLAGS(
+ aad_len_field_sz,
ctx->digest_length, q);
if (q > ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE) {
memcpy(aad_data + ctx->cipher_iv.length +
- ICP_QAT_HW_CCM_NONCE_OFFSET
- + (q - ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE),
- (uint8_t *)&msg_len_be,
- ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE);
+ ICP_QAT_HW_CCM_NONCE_OFFSET +
+ (q - ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE),
+ (uint8_t *)&msg_len_be,
+ ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE);
} else {
memcpy(aad_data + ctx->cipher_iv.length +
- ICP_QAT_HW_CCM_NONCE_OFFSET,
- (uint8_t *)&msg_len_be
- + (ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE
- - q), q);
+ ICP_QAT_HW_CCM_NONCE_OFFSET,
+ (uint8_t *)&msg_len_be
+ + (ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE
+ - q), q);
}
if (aad_len_field_sz > 0) {
@@ -709,10 +707,10 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
uint8_t pad_idx = 0;
pad_len = ICP_QAT_HW_CCM_AAD_B0_LEN -
- ((aad_ccm_real_len + aad_len_field_sz) %
- ICP_QAT_HW_CCM_AAD_B0_LEN);
+ ((aad_ccm_real_len + aad_len_field_sz) %
+ ICP_QAT_HW_CCM_AAD_B0_LEN);
pad_idx = ICP_QAT_HW_CCM_AAD_B0_LEN +
- aad_ccm_real_len + aad_len_field_sz;
+ aad_ccm_real_len + aad_len_field_sz;
memset(&aad_data[pad_idx],
0, pad_len);
}
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_sym.h
similarity index 100%
rename from drivers/crypto/qat/qat_crypto.h
rename to drivers/crypto/qat/qat_sym.h
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index e425eb43f..45f8a253b 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -9,7 +9,7 @@
#include <rte_pci.h>
#include <rte_cryptodev_pmd.h>
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "qat_sym_session.h"
#include "qat_logs.h"
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 06/31] crypto/qat: rename fns for consistency
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (36 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 05/31] crypto/qat: change filename crypto to sym Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 07/31] crypto/qat: renamed sym-specific structs Fiona Trahe
` (25 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Rename fn names to shorten them, i.e.
qat_crypto_sym_xxx to qat_sym_xxx
_content_desc_ to _cd_
Renaming symmetric crypto specific with consistent names:
qat_crypto_set_session_parameters->qat_sym_set_session_parameters
qat_write_hw_desc_entry()->qat_sym_build_request()
qat_alg_xxx ->qat_sym_xxx
qat_sym_xxx_session_yyy()->qat_sym_session_xxx_yyy()
Removed unused prototypes:
qat_get_inter_state_size()
qat_pmd_session_mempool_create()
Removed 2 unnecessary extern declarations
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 2 +-
drivers/crypto/qat/qat_device.h | 2 +-
drivers/crypto/qat/qat_qp.c | 6 +-
drivers/crypto/qat/qat_sym.c | 14 ++--
drivers/crypto/qat/qat_sym.h | 22 +++---
drivers/crypto/qat/qat_sym_session.c | 131 ++++++++++++++++-----------------
drivers/crypto/qat/qat_sym_session.h | 53 ++++++-------
drivers/crypto/qat/rte_qat_cryptodev.c | 18 ++---
8 files changed, 123 insertions(+), 125 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index c2bf9b7a7..ac6bd1af6 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -30,7 +30,7 @@ int qat_dev_close(struct rte_cryptodev *dev)
PMD_INIT_FUNC_TRACE();
for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_crypto_sym_qp_release(dev, i);
+ ret = qat_sym_qp_release(dev, i);
if (ret < 0)
return ret;
}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 5c48fdb93..2cb8e7612 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,7 +11,7 @@
extern uint8_t cryptodev_qat_driver_id;
-extern int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
/** private data structure for each QAT device */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 794a8d7c9..23a9d5f01 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -79,7 +79,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
const struct rte_cryptodev_qp_conf *qp_conf,
int socket_id, struct rte_mempool *session_pool __rte_unused)
{
@@ -93,7 +93,7 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
/* If qp is already in use free ring memory and qp metadata. */
if (dev->data->queue_pairs[queue_pair_id] != NULL) {
- ret = qat_crypto_sym_qp_release(dev, queue_pair_id);
+ ret = qat_sym_qp_release(dev, queue_pair_id);
if (ret < 0)
return ret;
}
@@ -209,7 +209,7 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-int qat_crypto_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index f5d542ae3..ae521c2b1 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -87,7 +87,7 @@ static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
static inline int
-qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
+qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
static inline uint32_t
@@ -210,7 +210,7 @@ txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
}
uint16_t
-qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
register struct qat_queue *queue;
@@ -242,7 +242,7 @@ qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
while (nb_ops_sent != nb_ops_possible) {
- ret = qat_write_hw_desc_entry(*cur_op, base_addr + tail,
+ ret = qat_sym_build_request(*cur_op, base_addr + tail,
tmp_qp->op_cookies[tail / queue->msg_size], tmp_qp);
if (ret != 0) {
tmp_qp->stats.enqueue_err_count++;
@@ -299,7 +299,7 @@ void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
}
uint16_t
-qat_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
struct qat_queue *rx_queue, *tx_queue;
@@ -456,7 +456,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
}
static inline int
-qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
+qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp)
{
int ret = 0;
@@ -883,7 +883,7 @@ static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
return data - mult;
}
-void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
+void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
{
int i;
@@ -907,7 +907,7 @@ void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
}
}
-void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev)
+void qat_sym_stats_reset(struct rte_cryptodev *dev)
{
int i;
struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index e3873171c..811ba8619 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -2,8 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
-#ifndef _QAT_CRYPTO_H_
-#define _QAT_CRYPTO_H_
+#ifndef _QAT_SYM_H_
+#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
#include <rte_memzone.h>
@@ -65,23 +65,23 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
} __rte_cache_aligned;
-void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
+void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
-void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev);
+void qat_sym_stats_reset(struct rte_cryptodev *dev);
-int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
const struct rte_cryptodev_qp_conf *rx_conf, int socket_id,
struct rte_mempool *session_pool);
-int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-extern uint16_t
-qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops);
-extern uint16_t
-qat_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops);
-#endif /* _QAT_CRYPTO_H_ */
+#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index 5caac5a6f..f598d1461 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -105,7 +105,7 @@ qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
}
void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+qat_sym_session_clear(struct rte_cryptodev *dev,
struct rte_cryptodev_sym_session *sess)
{
PMD_INIT_FUNC_TRACE();
@@ -116,7 +116,7 @@ qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
if (sess_priv) {
if (s->bpi_ctx)
bpi_cipher_ctx_free(s->bpi_ctx);
- memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
+ memset(s, 0, qat_sym_session_get_private_size(dev));
struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
set_session_private_data(sess, index, NULL);
@@ -197,7 +197,7 @@ qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
}
int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
@@ -213,7 +213,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
switch (cipher_xform->algo) {
case RTE_CRYPTO_CIPHER_AES_CBC:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
ret = -EINVAL;
@@ -222,7 +222,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_AES_CTR:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
ret = -EINVAL;
@@ -231,7 +231,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
break;
case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
- if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
+ if (qat_sym_validate_snow3g_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
ret = -EINVAL;
@@ -243,7 +243,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
break;
case RTE_CRYPTO_CIPHER_KASUMI_F8:
- if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
+ if (qat_sym_validate_kasumi_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
ret = -EINVAL;
@@ -252,7 +252,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
break;
case RTE_CRYPTO_CIPHER_3DES_CBC:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ if (qat_sym_validate_3des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
ret = -EINVAL;
@@ -261,7 +261,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_DES_CBC:
- if (qat_alg_validate_des_key(cipher_xform->key.length,
+ if (qat_sym_validate_des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
ret = -EINVAL;
@@ -270,7 +270,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_3DES_CTR:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ if (qat_sym_validate_3des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
ret = -EINVAL;
@@ -288,7 +288,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
goto error_out;
}
- if (qat_alg_validate_des_key(cipher_xform->key.length,
+ if (qat_sym_validate_des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
ret = -EINVAL;
@@ -306,7 +306,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
goto error_out;
}
- if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_docsisbpi_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
ret = -EINVAL;
@@ -323,7 +323,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
ret = -ENOTSUP;
goto error_out;
}
- if (qat_alg_validate_zuc_key(cipher_xform->key.length,
+ if (qat_sym_validate_zuc_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
ret = -EINVAL;
@@ -352,7 +352,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
else
session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
cipher_xform->key.data,
cipher_xform->key.length)) {
ret = -EINVAL;
@@ -370,7 +370,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+qat_sym_session_configure(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct rte_cryptodev_sym_session *sess,
struct rte_mempool *mempool)
@@ -384,7 +384,7 @@ qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
return -ENOMEM;
}
- ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
+ ret = qat_sym_session_set_parameters(dev, xform, sess_private_data);
if (ret != 0) {
PMD_DRV_LOG(ERR,
"Crypto QAT PMD: failed to configure session parameters");
@@ -401,7 +401,7 @@ qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
}
int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private)
{
struct qat_session *session = session_private;
@@ -425,29 +425,27 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
switch (session->qat_cmd) {
case ICP_QAT_FW_LA_CMD_CIPHER:
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
+ ret = qat_sym_session_configure_cipher(dev, xform, session);
if (ret < 0)
return ret;
break;
case ICP_QAT_FW_LA_CMD_AUTH:
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
+ ret = qat_sym_session_configure_auth(dev, xform, session);
if (ret < 0)
return ret;
break;
case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
+ ret = qat_sym_session_configure_aead(xform,
session);
if (ret < 0)
return ret;
} else {
- ret = qat_crypto_sym_configure_session_cipher(dev,
+ ret = qat_sym_session_configure_cipher(dev,
xform, session);
if (ret < 0)
return ret;
- ret = qat_crypto_sym_configure_session_auth(dev,
+ ret = qat_sym_session_configure_auth(dev,
xform, session);
if (ret < 0)
return ret;
@@ -455,16 +453,16 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
break;
case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
+ ret = qat_sym_session_configure_aead(xform,
session);
if (ret < 0)
return ret;
} else {
- ret = qat_crypto_sym_configure_session_auth(dev,
+ ret = qat_sym_session_configure_auth(dev,
xform, session);
if (ret < 0)
return ret;
- ret = qat_crypto_sym_configure_session_cipher(dev,
+ ret = qat_sym_session_configure_cipher(dev,
xform, session);
if (ret < 0)
return ret;
@@ -492,7 +490,7 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
@@ -521,7 +519,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
break;
case RTE_CRYPTO_AUTH_AES_GMAC:
- if (qat_alg_validate_aes_key(auth_xform->key.length,
+ if (qat_sym_validate_aes_key(auth_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -579,14 +577,13 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
* It needs to create cipher desc content first,
* then authentication
*/
- if (qat_alg_aead_session_create_content_desc_cipher(
- session,
+
+ if (qat_sym_session_aead_create_cd_cipher(session,
auth_xform->key.data,
auth_xform->key.length))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_auth(
- session,
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -600,8 +597,8 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
* It needs to create authentication desc content first,
* then cipher
*/
- if (qat_alg_aead_session_create_content_desc_auth(
- session,
+
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -609,8 +606,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
auth_xform->op))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_cipher(
- session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
auth_xform->key.data,
auth_xform->key.length))
return -EINVAL;
@@ -618,7 +614,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
/* Restore to authentication only only */
session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
} else {
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -632,7 +628,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
struct rte_crypto_aead_xform *aead_xform = &xform->aead;
@@ -647,7 +643,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
switch (aead_xform->algo) {
case RTE_CRYPTO_AEAD_AES_GCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
+ if (qat_sym_validate_aes_key(aead_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -656,7 +652,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
break;
case RTE_CRYPTO_AEAD_AES_CCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
+ if (qat_sym_validate_aes_key(aead_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -682,12 +678,12 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
aead_xform->key.data,
aead_xform->key.length))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
aead_xform->key.data,
aead_xform->key.length,
aead_xform->aad_length,
@@ -704,7 +700,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
aead_xform->key.data,
aead_xform->key.length,
aead_xform->aad_length,
@@ -712,7 +708,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
aead_xform->key.data,
aead_xform->key.length))
return -EINVAL;
@@ -722,7 +718,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
return 0;
}
-unsigned int qat_crypto_sym_get_session_private_size(
+unsigned int qat_sym_session_get_private_size(
struct rte_cryptodev *dev __rte_unused)
{
return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
@@ -996,7 +992,7 @@ static int partial_hash_compute(enum icp_qat_hw_auth_algo hash_alg,
#define HMAC_OPAD_VALUE 0x5c
#define HASH_XCBC_PRECOMP_KEY_NUM 3
-static int qat_alg_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
+static int qat_sym_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
const uint8_t *auth_key,
uint16_t auth_keylen,
uint8_t *p_state_buf,
@@ -1126,7 +1122,8 @@ static int qat_alg_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
return 0;
}
-void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
+static void
+qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
enum qat_crypto_proto_flag proto_flags)
{
PMD_INIT_FUNC_TRACE();
@@ -1194,7 +1191,7 @@ qat_get_crypto_proto_flag(uint16_t flags)
return qat_proto_flag;
}
-int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
uint8_t *cipherkey,
uint32_t cipherkeylen)
{
@@ -1298,7 +1295,7 @@ int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
cipher_cd_ctrl->cipher_cfg_offset = cipher_offset >> 3;
header->service_cmd_id = cdesc->qat_cmd;
- qat_alg_init_common_hdr(header, qat_proto_flag);
+ qat_sym_session_init_common_hdr(header, qat_proto_flag);
cipher = (struct icp_qat_hw_cipher_algo_blk *)cdesc->cd_cur_ptr;
cipher->cipher_config.val =
@@ -1342,7 +1339,7 @@ int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
return 0;
}
-int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -1431,7 +1428,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
*/
switch (cdesc->qat_hash_alg) {
case ICP_QAT_HW_AUTH_ALGO_SHA1:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA1,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA1,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1439,7 +1436,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = RTE_ALIGN_CEIL(ICP_QAT_HW_SHA1_STATE2_SZ, 8);
break;
case ICP_QAT_HW_AUTH_ALGO_SHA224:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA224,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA224,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1447,7 +1444,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA224_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA256:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA256,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA256,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1455,7 +1452,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA256_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA384:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA384,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA384,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1463,7 +1460,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA384_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA512:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA512,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA512,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1472,7 +1469,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
break;
case ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC:
state1_size = ICP_QAT_HW_AES_XCBC_MAC_STATE1_SZ;
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC,
authkey, authkeylen, cdesc->cd_cur_ptr + state1_size,
&state2_size)) {
PMD_DRV_LOG(ERR, "(XCBC)precompute failed");
@@ -1483,7 +1480,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
case ICP_QAT_HW_AUTH_ALGO_GALOIS_64:
qat_proto_flag = QAT_CRYPTO_PROTO_FLAG_GCM;
state1_size = ICP_QAT_HW_GALOIS_128_STATE1_SZ;
- if (qat_alg_do_precomputes(cdesc->qat_hash_alg,
+ if (qat_sym_do_precomputes(cdesc->qat_hash_alg,
authkey, authkeylen, cdesc->cd_cur_ptr + state1_size,
&state2_size)) {
PMD_DRV_LOG(ERR, "(GCM)precompute failed");
@@ -1543,7 +1540,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
break;
case ICP_QAT_HW_AUTH_ALGO_MD5:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_MD5,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_MD5,
authkey, authkeylen, cdesc->cd_cur_ptr,
&state1_size)) {
PMD_DRV_LOG(ERR, "(MD5)precompute failed");
@@ -1606,7 +1603,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
}
/* Request template setup */
- qat_alg_init_common_hdr(header, qat_proto_flag);
+ qat_sym_session_init_common_hdr(header, qat_proto_flag);
header->service_cmd_id = cdesc->qat_cmd;
/* Auth CD config setup */
@@ -1632,7 +1629,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
return 0;
}
-int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_AES_128_KEY_SZ:
@@ -1650,7 +1647,7 @@ int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_aes_docsisbpi_key(int key_len,
+int qat_sym_validate_aes_docsisbpi_key(int key_len,
enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
@@ -1663,7 +1660,7 @@ int qat_alg_validate_aes_docsisbpi_key(int key_len,
return 0;
}
-int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_SNOW_3G_UEA2_KEY_SZ:
@@ -1675,7 +1672,7 @@ int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_KASUMI_KEY_SZ:
@@ -1687,7 +1684,7 @@ int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_DES_KEY_SZ:
@@ -1699,7 +1696,7 @@ int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case QAT_3DES_KEY_SZ_OPT1:
@@ -1712,7 +1709,7 @@ int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_ZUC_3G_EEA3_KEY_SZ:
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index f90b1821d..493036ebf 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -76,67 +76,68 @@ struct qat_session {
};
int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+qat_sym_session_configure(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct rte_cryptodev_sym_session *sess,
struct rte_mempool *mempool);
int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private);
int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cd,
+qat_sym_session_aead_create_cd_cipher(struct qat_session *cd,
uint8_t *enckey,
uint32_t enckeylen);
int
-qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
uint32_t digestsize,
unsigned int operation);
-int
-qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
- unsigned int nb_objs, unsigned int obj_cache_size, int socket_id);
-
void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+qat_sym_session_clear(struct rte_cryptodev *dev,
struct rte_cryptodev_sym_session *session);
unsigned int
-qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
-
-int qat_get_inter_state_size(enum icp_qat_hw_auth_algo qat_hash_alg);
-
+qat_sym_session_get_private_size(struct rte_cryptodev *dev);
-void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
+void
+qat_sym_sesssion_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
enum qat_crypto_proto_flag proto_flags);
-
-int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_aes_docsisbpi_key(int key_len,
+int
+qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_aes_docsisbpi_key(int key_len,
enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
-int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
+int
+qat_sym_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
#endif /* _QAT_SYM_SESSION_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 45f8a253b..42011dcd9 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -35,18 +35,18 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
.dev_close = qat_dev_close,
.dev_infos_get = qat_dev_info_get,
- .stats_get = qat_crypto_sym_stats_get,
- .stats_reset = qat_crypto_sym_stats_reset,
- .queue_pair_setup = qat_crypto_sym_qp_setup,
- .queue_pair_release = qat_crypto_sym_qp_release,
+ .stats_get = qat_sym_stats_get,
+ .stats_reset = qat_sym_stats_reset,
+ .queue_pair_setup = qat_sym_qp_setup,
+ .queue_pair_release = qat_sym_qp_release,
.queue_pair_start = NULL,
.queue_pair_stop = NULL,
.queue_pair_count = NULL,
/* Crypto related operations */
- .session_get_size = qat_crypto_sym_get_session_private_size,
- .session_configure = qat_crypto_sym_configure_session,
- .session_clear = qat_crypto_sym_clear_session
+ .session_get_size = qat_sym_session_get_private_size,
+ .session_configure = qat_sym_session_configure,
+ .session_clear = qat_sym_session_clear
};
/*
@@ -86,8 +86,8 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
cryptodev->driver_id = cryptodev_qat_driver_id;
cryptodev->dev_ops = &crypto_qat_ops;
- cryptodev->enqueue_burst = qat_pmd_enqueue_op_burst;
- cryptodev->dequeue_burst = qat_pmd_dequeue_op_burst;
+ cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
+ cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
RTE_CRYPTODEV_FF_HW_ACCELERATED |
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 07/31] crypto/qat: renamed sym-specific structs
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (37 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 06/31] crypto/qat: rename fns for consistency Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 08/31] crypto/qat: make enqueue function generic Fiona Trahe
` (24 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
qat_session -> qat_sym_session
qat_crypto_proto_flag -> qat_sym_proto_flag
qat_alg_cd -> qat_sym_cd
qat_crypto_op_cookie -> qat_sym_op_cookie
qat_gen1_capabilities -> qat_gen1_sym_capabilities
qat_gen2_capabilities -> qat_gen2_sym_capabilities
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 4 ++--
drivers/crypto/qat/qat_qp.c | 8 ++++----
drivers/crypto/qat/qat_sym.c | 19 ++++++++++---------
drivers/crypto/qat/qat_sym.h | 2 +-
drivers/crypto/qat/qat_sym_session.c | 28 ++++++++++++++--------------
drivers/crypto/qat/qat_sym_session.h | 20 ++++++++++----------
drivers/crypto/qat/rte_qat_cryptodev.c | 8 ++++----
7 files changed, 45 insertions(+), 44 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 293b6f700..7802e96f9 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -37,11 +37,11 @@ struct qat_alg_buf_list {
struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
-struct qat_crypto_op_cookie {
+struct qat_sym_op_cookie {
struct qat_alg_buf_list qat_sgl_list_src;
struct qat_alg_buf_list qat_sgl_list_dst;
phys_addr_t qat_sgl_src_phys_addr;
phys_addr_t qat_sgl_dst_phys_addr;
};
-#endif /* _QAT_QAT_COMMON_H_ */
+#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 23a9d5f01..fb9c2a7ef 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -165,7 +165,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
qp->nb_descriptors,
- sizeof(struct qat_crypto_op_cookie), 64, 0,
+ sizeof(struct qat_sym_op_cookie), 64, 0,
NULL, NULL, NULL, NULL, socket_id,
0);
if (!qp->op_cookie_pool) {
@@ -180,17 +180,17 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
goto create_err;
}
- struct qat_crypto_op_cookie *sql_cookie =
+ struct qat_sym_op_cookie *sql_cookie =
qp->op_cookies[i];
sql_cookie->qat_sgl_src_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_crypto_op_cookie,
+ offsetof(struct qat_sym_op_cookie,
qat_sgl_list_src);
sql_cookie->qat_sgl_dst_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_crypto_op_cookie,
+ offsetof(struct qat_sym_op_cookie,
qat_sgl_list_dst);
}
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index ae521c2b1..2dfdc9cce 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -88,10 +88,10 @@ adf_modulo(uint32_t data, uint32_t shift);
static inline int
qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
+ struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp);
static inline uint32_t
-qat_bpicipher_preprocess(struct qat_session *ctx,
+qat_bpicipher_preprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
{
int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
@@ -146,7 +146,7 @@ qat_bpicipher_preprocess(struct qat_session *ctx,
}
static inline uint32_t
-qat_bpicipher_postprocess(struct qat_session *ctx,
+qat_bpicipher_postprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
{
int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
@@ -329,10 +329,11 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
resp_msg->comn_hdr.comn_status)) {
rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
} else {
- struct qat_session *sess = (struct qat_session *)
+ struct qat_sym_session *sess =
+ (struct qat_sym_session *)
get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
if (sess->bpi_ctx)
qat_bpicipher_postprocess(sess, rx_op);
@@ -457,10 +458,10 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
static inline int
qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp)
+ struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp)
{
int ret = 0;
- struct qat_session *ctx;
+ struct qat_sym_session *ctx;
struct icp_qat_fw_la_cipher_req_params *cipher_param;
struct icp_qat_fw_la_auth_req_params *auth_param;
register struct icp_qat_fw_la_bulk_req *qat_req;
@@ -485,7 +486,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
- ctx = (struct qat_session *)get_session_private_data(
+ ctx = (struct qat_sym_session *)get_session_private_data(
op->sym->session, cryptodev_qat_driver_id);
if (unlikely(ctx == NULL)) {
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 811ba8619..18c77ea11 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,7 +27,7 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
-struct qat_session;
+struct qat_sym_session;
/**
* Structure associated with each queue.
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index f598d1461..a08e93037 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -111,7 +111,7 @@ qat_sym_session_clear(struct rte_cryptodev *dev,
PMD_INIT_FUNC_TRACE();
uint8_t index = dev->driver_id;
void *sess_priv = get_session_private_data(sess, index);
- struct qat_session *s = (struct qat_session *)sess_priv;
+ struct qat_sym_session *s = (struct qat_sym_session *)sess_priv;
if (sess_priv) {
if (s->bpi_ctx)
@@ -199,7 +199,7 @@ qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
int
qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct qat_pmd_private *internals = dev->data->dev_private;
struct rte_crypto_cipher_xform *cipher_xform = NULL;
@@ -404,7 +404,7 @@ int
qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private)
{
- struct qat_session *session = session_private;
+ struct qat_sym_session *session = session_private;
int ret;
int qat_cmd_id;
@@ -412,7 +412,7 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
/* Set context descriptor physical address */
session->cd_paddr = rte_mempool_virt2iova(session) +
- offsetof(struct qat_session, cd);
+ offsetof(struct qat_sym_session, cd);
session->min_qat_dev_gen = QAT_GEN1;
@@ -492,7 +492,7 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
int
qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
struct qat_pmd_private *internals = dev->data->dev_private;
@@ -629,7 +629,7 @@ qat_sym_session_configure_auth(struct rte_cryptodev *dev,
int
qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct rte_crypto_aead_xform *aead_xform = &xform->aead;
enum rte_crypto_auth_operation crypto_operation;
@@ -721,7 +721,7 @@ qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
unsigned int qat_sym_session_get_private_size(
struct rte_cryptodev *dev __rte_unused)
{
- return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
+ return RTE_ALIGN_CEIL(sizeof(struct qat_sym_session), 8);
}
/* returns block size in bytes per cipher algo */
@@ -1124,7 +1124,7 @@ static int qat_sym_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
static void
qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
- enum qat_crypto_proto_flag proto_flags)
+ enum qat_sym_proto_flag proto_flags)
{
PMD_INIT_FUNC_TRACE();
header->hdr_flags =
@@ -1172,11 +1172,11 @@ qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
* and set its protocol flag in both cipher and auth part of content
* descriptor building function
*/
-static enum qat_crypto_proto_flag
+static enum qat_sym_proto_flag
qat_get_crypto_proto_flag(uint16_t flags)
{
int proto = ICP_QAT_FW_LA_PROTO_GET(flags);
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
switch (proto) {
@@ -1191,7 +1191,7 @@ qat_get_crypto_proto_flag(uint16_t flags)
return qat_proto_flag;
}
-int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_cipher(struct qat_sym_session *cdesc,
uint8_t *cipherkey,
uint32_t cipherkeylen)
{
@@ -1203,7 +1203,7 @@ int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
struct icp_qat_fw_cipher_cd_ctrl_hdr *cipher_cd_ctrl = ptr;
struct icp_qat_fw_auth_cd_ctrl_hdr *hash_cd_ctrl = ptr;
enum icp_qat_hw_cipher_convert key_convert;
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
uint32_t total_key_size;
uint16_t cipher_offset, cd_size;
@@ -1339,7 +1339,7 @@ int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
return 0;
}
-int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_auth(struct qat_sym_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -1363,7 +1363,7 @@ int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint32_t *aad_len = NULL;
uint32_t wordIndex = 0;
uint32_t *pTempKey;
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
PMD_INIT_FUNC_TRACE();
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index 493036ebf..d3d27ff46 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -34,7 +34,7 @@
ICP_QAT_HW_CIPHER_KEY_CONVERT, \
ICP_QAT_HW_CIPHER_DECRYPT)
-enum qat_crypto_proto_flag {
+enum qat_sym_proto_flag {
QAT_CRYPTO_PROTO_FLAG_NONE = 0,
QAT_CRYPTO_PROTO_FLAG_CCM = 1,
QAT_CRYPTO_PROTO_FLAG_GCM = 2,
@@ -43,12 +43,12 @@ enum qat_crypto_proto_flag {
};
/* Common content descriptor */
-struct qat_alg_cd {
+struct qat_sym_cd {
struct icp_qat_hw_cipher_algo_blk cipher;
struct icp_qat_hw_auth_algo_blk hash;
} __rte_packed __rte_cache_aligned;
-struct qat_session {
+struct qat_sym_session {
enum icp_qat_fw_la_cmd_id qat_cmd;
enum icp_qat_hw_cipher_algo qat_cipher_alg;
enum icp_qat_hw_cipher_dir qat_dir;
@@ -56,7 +56,7 @@ struct qat_session {
enum icp_qat_hw_auth_algo qat_hash_alg;
enum icp_qat_hw_auth_op auth_op;
void *bpi_ctx;
- struct qat_alg_cd cd;
+ struct qat_sym_cd cd;
uint8_t *cd_cur_ptr;
phys_addr_t cd_paddr;
struct icp_qat_fw_la_bulk_req fw_req;
@@ -87,25 +87,25 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
int
qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
-qat_sym_session_aead_create_cd_cipher(struct qat_session *cd,
+qat_sym_session_aead_create_cd_cipher(struct qat_sym_session *cd,
uint8_t *enckey,
uint32_t enckeylen);
int
-qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
+qat_sym_session_aead_create_cd_auth(struct qat_sym_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -121,7 +121,7 @@ qat_sym_session_get_private_size(struct rte_cryptodev *dev);
void
qat_sym_sesssion_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
- enum qat_crypto_proto_flag proto_flags);
+ enum qat_sym_proto_flag proto_flags);
int
qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
int
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 42011dcd9..05cb5ef40 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -15,12 +15,12 @@
uint8_t cryptodev_qat_driver_id;
-static const struct rte_cryptodev_capabilities qat_gen1_capabilities[] = {
+static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
QAT_BASE_GEN1_SYM_CAPABILITIES,
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
};
-static const struct rte_cryptodev_capabilities qat_gen2_capabilities[] = {
+static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
QAT_BASE_GEN1_SYM_CAPABILITIES,
QAT_EXTRA_GEN2_SYM_CAPABILITIES,
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
@@ -99,13 +99,13 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
switch (pci_dev->id.device_id) {
case 0x0443:
internals->qat_dev_gen = QAT_GEN1;
- internals->qat_dev_capabilities = qat_gen1_capabilities;
+ internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
break;
case 0x37c9:
case 0x19e3:
case 0x6f55:
internals->qat_dev_gen = QAT_GEN2;
- internals->qat_dev_capabilities = qat_gen2_capabilities;
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
break;
default:
PMD_DRV_LOG(ERR,
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 08/31] crypto/qat: make enqueue function generic
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (38 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 07/31] crypto/qat: renamed sym-specific structs Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 09/31] crypto/qat: make dequeue " Fiona Trahe
` (23 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Queue-handling code in enqueue is made generic, so it can
be used by other services in future. This is done by
- Removing all sym-specific refs in input params - replace with void ptrs.
- Wrapping this generic enqueue with the sym-specific enqueue
called through the API.
- Setting a fn ptr for build_request in qp on qp creation
- Passing void * params to this, in the service-specific implementation
qat_sym_build_request cast back to sym structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 1 +
drivers/crypto/qat/qat_sym.c | 46 +++++++++++++++++++++++++-------------------
drivers/crypto/qat/qat_sym.h | 11 +++++++++++
3 files changed, 38 insertions(+), 20 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index fb9c2a7ef..d7d79f1af 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -197,6 +197,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
struct qat_pmd_private *internals
= dev->data->dev_private;
qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->build_request = qat_sym_build_request;
dev->data->queue_pairs[queue_pair_id] = qp;
return 0;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 2dfdc9cce..4e404749a 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -86,10 +86,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
-static inline int
-qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp);
-
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
@@ -209,14 +205,12 @@ txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
q->csr_tail = q->tail;
}
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
+static uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
{
register struct qat_queue *queue;
struct qat_qp *tmp_qp = (struct qat_qp *)qp;
register uint32_t nb_ops_sent = 0;
- register struct rte_crypto_op **cur_op = ops;
register int ret;
uint16_t nb_ops_possible = nb_ops;
register uint8_t *base_addr;
@@ -242,8 +236,9 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
while (nb_ops_sent != nb_ops_possible) {
- ret = qat_sym_build_request(*cur_op, base_addr + tail,
- tmp_qp->op_cookies[tail / queue->msg_size], tmp_qp);
+ ret = tmp_qp->build_request(*ops, base_addr + tail,
+ tmp_qp->op_cookies[tail / queue->msg_size],
+ tmp_qp->qat_dev_gen);
if (ret != 0) {
tmp_qp->stats.enqueue_err_count++;
/*
@@ -257,8 +252,8 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ ops++;
nb_ops_sent++;
- cur_op++;
}
kick_tail:
queue->tail = tail;
@@ -299,6 +294,13 @@ void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
}
uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
+}
+
+uint16_t
qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
@@ -456,9 +458,10 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-static inline int
-qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp)
+
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen)
{
int ret = 0;
struct qat_sym_session *ctx;
@@ -471,6 +474,9 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
uint32_t min_ofs = 0;
uint64_t src_buf_start = 0, dst_buf_start = 0;
uint8_t do_sgl = 0;
+ struct rte_crypto_op *op = (struct rte_crypto_op *)in_op;
+ struct qat_sym_op_cookie *cookie =
+ (struct qat_sym_op_cookie *)op_cookie;
#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_TX
if (unlikely(op->type != RTE_CRYPTO_OP_TYPE_SYMMETRIC)) {
@@ -494,7 +500,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
- if (unlikely(ctx->min_qat_dev_gen > qp->qat_dev_gen)) {
+ if (unlikely(ctx->min_qat_dev_gen > qat_dev_gen)) {
PMD_DRV_LOG(ERR, "Session alg not supported on this device gen");
op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION;
return -EINVAL;
@@ -807,7 +813,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
ICP_QAT_FW_COMN_PTR_TYPE_SET(qat_req->comn_hdr.comn_req_flags,
QAT_COMN_PTR_TYPE_SGL);
ret = qat_sgl_fill_array(op->sym->m_src, src_buf_start,
- &qat_op_cookie->qat_sgl_list_src,
+ &cookie->qat_sgl_list_src,
qat_req->comn_mid.src_length);
if (ret) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot fill sgl array");
@@ -817,11 +823,11 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
if (likely(op->sym->m_dst == NULL))
qat_req->comn_mid.dest_data_addr =
qat_req->comn_mid.src_data_addr =
- qat_op_cookie->qat_sgl_src_phys_addr;
+ cookie->qat_sgl_src_phys_addr;
else {
ret = qat_sgl_fill_array(op->sym->m_dst,
dst_buf_start,
- &qat_op_cookie->qat_sgl_list_dst,
+ &cookie->qat_sgl_list_dst,
qat_req->comn_mid.dst_length);
if (ret) {
@@ -831,9 +837,9 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
}
qat_req->comn_mid.src_data_addr =
- qat_op_cookie->qat_sgl_src_phys_addr;
+ cookie->qat_sgl_src_phys_addr;
qat_req->comn_mid.dest_data_addr =
- qat_op_cookie->qat_sgl_dst_phys_addr;
+ cookie->qat_sgl_dst_phys_addr;
}
} else {
qat_req->comn_mid.src_data_addr = src_buf_start;
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 18c77ea11..b1ddb6e93 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,6 +27,11 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
+typedef int (*build_request_t)(void *op,
+ uint8_t *req, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Build a request from an op. */
+
struct qat_sym_session;
/**
@@ -63,8 +68,14 @@ struct qat_qp {
void **op_cookies;
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
+ build_request_t build_request;
} __rte_cache_aligned;
+
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen);
+
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 09/31] crypto/qat: make dequeue function generic
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (39 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 08/31] crypto/qat: make enqueue function generic Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 10/31] crypto/qat: move generic qp fn to qp file Fiona Trahe
` (22 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Queue-handling code in dequeue is made generic, so it can
be used by other services in future. This is done by
- Removing all sym-specific refs in input params - replace with void ptrs.
- Wrapping this generic dequeue with the sym-specific dequeue
called through the API.
- extracting the sym-specific response processing into a new fn.
- Setting a fn ptr for process_response in qp on qp creation
- Passing void * params to this, in the service-specific implementation
qat_sym_process_response cast back to sym structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 1 +
drivers/crypto/qat/qat_sym.c | 101 ++++++++++++++++++++++++++-----------------
drivers/crypto/qat/qat_sym.h | 10 +++++
3 files changed, 72 insertions(+), 40 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index d7d79f1af..bae6cf114 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -198,6 +198,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
= dev->data->dev_private;
qp->qat_dev_gen = internals->qat_dev_gen;
qp->build_request = qat_sym_build_request;
+ qp->process_response = qat_sym_process_response;
dev->data->queue_pairs[queue_pair_id] = qp;
return 0;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 4e404749a..2bae913a1 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -300,70 +300,91 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
}
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen)
+{
+
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
+
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
+
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
+
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+ }
+ *op = (void *)rx_op;
+
+ return 0;
+}
+
+static uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
{
struct qat_queue *rx_queue, *tx_queue;
struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- uint32_t msg_counter = 0;
- struct rte_crypto_op *rx_op;
- struct icp_qat_fw_comn_resp *resp_msg;
uint32_t head;
+ uint32_t resp_counter = 0;
+ uint8_t *resp_msg;
rx_queue = &(tmp_qp->rx_q);
tx_queue = &(tmp_qp->tx_q);
head = rx_queue->head;
- resp_msg = (struct icp_qat_fw_comn_resp *)
- ((uint8_t *)rx_queue->base_addr + head);
+ resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
- msg_counter != nb_ops) {
- rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess =
- (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
+ resp_counter != nb_ops) {
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
+ tmp_qp->process_response(ops, resp_msg,
+ tmp_qp->op_cookies[head / rx_queue->msg_size],
+ tmp_qp->qat_dev_gen);
head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
- resp_msg = (struct icp_qat_fw_comn_resp *)
- ((uint8_t *)rx_queue->base_addr + head);
- *ops = rx_op;
+
+ resp_msg = (uint8_t *)rx_queue->base_addr + head;
ops++;
- msg_counter++;
+ resp_counter++;
}
- if (msg_counter > 0) {
+ if (resp_counter > 0) {
rx_queue->head = head;
- tmp_qp->stats.dequeued_count += msg_counter;
- rx_queue->nb_processed_responses += msg_counter;
- tmp_qp->inflights16 -= msg_counter;
+ tmp_qp->stats.dequeued_count += resp_counter;
+ rx_queue->nb_processed_responses += resp_counter;
+ tmp_qp->inflights16 -= resp_counter;
if (rx_queue->nb_processed_responses > QAT_CSR_HEAD_WRITE_THRESH)
rxq_free_desc(tmp_qp, rx_queue);
}
/* also check if tail needs to be advanced */
if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
- tx_queue->tail != tx_queue->csr_tail) {
+ tx_queue->tail != tx_queue->csr_tail) {
txq_write_tail(tmp_qp, tx_queue);
}
- return msg_counter;
+ return resp_counter;
+}
+
+uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
static inline int
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index b1ddb6e93..279d3a3ae 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -32,6 +32,11 @@ typedef int (*build_request_t)(void *op,
enum qat_device_gen qat_dev_gen);
/**< Build a request from an op. */
+typedef int (*process_response_t)(void **ops,
+ uint8_t *resp, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Process a response descriptor and return the associated op. */
+
struct qat_sym_session;
/**
@@ -69,6 +74,7 @@ struct qat_qp {
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
+ process_response_t process_response;
} __rte_cache_aligned;
@@ -76,6 +82,10 @@ int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie, enum qat_device_gen qat_dev_gen);
+
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 10/31] crypto/qat: move generic qp fn to qp file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (40 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 09/31] crypto/qat: make dequeue " Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 11/31] crypto/qat: separate sym-specific from generic qp setup Fiona Trahe
` (21 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Move the generic enqueue and dequeue fns from
the qat_sym.c file to the qat_qp.c file
Move generic qp structs to a new qat_qp.h file
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 152 +++++++++++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_qp.h | 63 ++++++++++++++++++
drivers/crypto/qat/qat_sym.c | 149 +-----------------------------------------
drivers/crypto/qat/qat_sym.h | 49 --------------
4 files changed, 216 insertions(+), 197 deletions(-)
create mode 100644 drivers/crypto/qat/qat_qp.h
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index bae6cf114..56ea10242 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,7 +13,9 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
+#include "qat_qp.h"
#include "qat_sym.h"
+
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
@@ -450,3 +452,153 @@ static void adf_configure_queues(struct qat_qp *qp)
WRITE_CSR_RING_CONFIG(qp->mmap_bar_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_config);
}
+
+
+static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
+{
+ uint32_t div = data >> shift;
+ uint32_t mult = div << shift;
+
+ return data - mult;
+}
+
+static inline void
+txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
+ WRITE_CSR_RING_TAIL(qp->mmap_bar_addr, q->hw_bundle_number,
+ q->hw_queue_number, q->tail);
+ q->nb_pending_requests = 0;
+ q->csr_tail = q->tail;
+}
+
+static inline
+void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
+{
+ uint32_t old_head, new_head;
+ uint32_t max_head;
+
+ old_head = q->csr_head;
+ new_head = q->head;
+ max_head = qp->nb_descriptors * q->msg_size;
+
+ /* write out free descriptors */
+ void *cur_desc = (uint8_t *)q->base_addr + old_head;
+
+ if (new_head < old_head) {
+ memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, max_head - old_head);
+ memset(q->base_addr, ADF_RING_EMPTY_SIG_BYTE, new_head);
+ } else {
+ memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, new_head - old_head);
+ }
+ q->nb_processed_responses = 0;
+ q->csr_head = new_head;
+
+ /* write current head to CSR */
+ WRITE_CSR_RING_HEAD(qp->mmap_bar_addr, q->hw_bundle_number,
+ q->hw_queue_number, new_head);
+}
+
+uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
+{
+ register struct qat_queue *queue;
+ struct qat_qp *tmp_qp = (struct qat_qp *)qp;
+ register uint32_t nb_ops_sent = 0;
+ register int ret;
+ uint16_t nb_ops_possible = nb_ops;
+ register uint8_t *base_addr;
+ register uint32_t tail;
+ int overflow;
+
+ if (unlikely(nb_ops == 0))
+ return 0;
+
+ /* read params used a lot in main loop into registers */
+ queue = &(tmp_qp->tx_q);
+ base_addr = (uint8_t *)queue->base_addr;
+ tail = queue->tail;
+
+ /* Find how many can actually fit on the ring */
+ tmp_qp->inflights16 += nb_ops;
+ overflow = tmp_qp->inflights16 - queue->max_inflights;
+ if (overflow > 0) {
+ tmp_qp->inflights16 -= overflow;
+ nb_ops_possible = nb_ops - overflow;
+ if (nb_ops_possible == 0)
+ return 0;
+ }
+
+ while (nb_ops_sent != nb_ops_possible) {
+ ret = tmp_qp->build_request(*ops, base_addr + tail,
+ tmp_qp->op_cookies[tail / queue->msg_size],
+ tmp_qp->qat_dev_gen);
+ if (ret != 0) {
+ tmp_qp->stats.enqueue_err_count++;
+ /*
+ * This message cannot be enqueued,
+ * decrease number of ops that wasn't sent
+ */
+ tmp_qp->inflights16 -= nb_ops_possible - nb_ops_sent;
+ if (nb_ops_sent == 0)
+ return 0;
+ goto kick_tail;
+ }
+
+ tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ ops++;
+ nb_ops_sent++;
+ }
+kick_tail:
+ queue->tail = tail;
+ tmp_qp->stats.enqueued_count += nb_ops_sent;
+ queue->nb_pending_requests += nb_ops_sent;
+ if (tmp_qp->inflights16 < QAT_CSR_TAIL_FORCE_WRITE_THRESH ||
+ queue->nb_pending_requests > QAT_CSR_TAIL_WRITE_THRESH) {
+ txq_write_tail(tmp_qp, queue);
+ }
+ return nb_ops_sent;
+}
+
+uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
+{
+ struct qat_queue *rx_queue, *tx_queue;
+ struct qat_qp *tmp_qp = (struct qat_qp *)qp;
+ uint32_t head;
+ uint32_t resp_counter = 0;
+ uint8_t *resp_msg;
+
+ rx_queue = &(tmp_qp->rx_q);
+ tx_queue = &(tmp_qp->tx_q);
+ head = rx_queue->head;
+ resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
+
+ while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
+ resp_counter != nb_ops) {
+
+ tmp_qp->process_response(ops, resp_msg,
+ tmp_qp->op_cookies[head / rx_queue->msg_size],
+ tmp_qp->qat_dev_gen);
+
+ head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
+
+ resp_msg = (uint8_t *)rx_queue->base_addr + head;
+ ops++;
+ resp_counter++;
+ }
+ if (resp_counter > 0) {
+ rx_queue->head = head;
+ tmp_qp->stats.dequeued_count += resp_counter;
+ rx_queue->nb_processed_responses += resp_counter;
+ tmp_qp->inflights16 -= resp_counter;
+
+ if (rx_queue->nb_processed_responses >
+ QAT_CSR_HEAD_WRITE_THRESH)
+ rxq_free_desc(tmp_qp, rx_queue);
+ }
+ /* also check if tail needs to be advanced */
+ if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
+ tx_queue->tail != tx_queue->csr_tail) {
+ txq_write_tail(tmp_qp, tx_queue);
+ }
+ return resp_counter;
+}
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
new file mode 100644
index 000000000..87d55c5f2
--- /dev/null
+++ b/drivers/crypto/qat/qat_qp.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_QP_H_
+#define _QAT_QP_H_
+
+#include "qat_common.h"
+
+typedef int (*build_request_t)(void *op,
+ uint8_t *req, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Build a request from an op. */
+
+typedef int (*process_response_t)(void **ops,
+ uint8_t *resp, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Process a response descriptor and return the associated op. */
+
+/**
+ * Structure associated with each queue.
+ */
+struct qat_queue {
+ char memz_name[RTE_MEMZONE_NAMESIZE];
+ void *base_addr; /* Base address */
+ rte_iova_t base_phys_addr; /* Queue physical address */
+ uint32_t head; /* Shadow copy of the head */
+ uint32_t tail; /* Shadow copy of the tail */
+ uint32_t modulo;
+ uint32_t msg_size;
+ uint16_t max_inflights;
+ uint32_t queue_size;
+ uint8_t hw_bundle_number;
+ uint8_t hw_queue_number;
+ /* HW queue aka ring offset on bundle */
+ uint32_t csr_head; /* last written head value */
+ uint32_t csr_tail; /* last written tail value */
+ uint16_t nb_processed_responses;
+ /* number of responses processed since last CSR head write */
+ uint16_t nb_pending_requests;
+ /* number of requests pending since last CSR tail write */
+};
+
+struct qat_qp {
+ void *mmap_bar_addr;
+ uint16_t inflights16;
+ struct qat_queue tx_q;
+ struct qat_queue rx_q;
+ struct rte_cryptodev_stats stats;
+ struct rte_mempool *op_cookie_pool;
+ void **op_cookies;
+ uint32_t nb_descriptors;
+ enum qat_device_gen qat_dev_gen;
+ build_request_t build_request;
+ process_response_t process_response;
+} __rte_cache_aligned;
+
+uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+
+uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+
+#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 2bae913a1..ab8ce2c96 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -14,6 +14,7 @@
#include "qat_logs.h"
#include "qat_sym_session.h"
#include "qat_sym.h"
+#include "qat_qp.h"
#include "adf_transport_access_macros.h"
#define BYTE_LENGTH 8
@@ -83,8 +84,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
/** Creates a context in either AES or DES in ECB mode
* Depends on openssl libcrypto
*/
-static inline uint32_t
-adf_modulo(uint32_t data, uint32_t shift);
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
@@ -197,102 +196,6 @@ qat_bpicipher_postprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-static inline void
-txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
- WRITE_CSR_RING_TAIL(qp->mmap_bar_addr, q->hw_bundle_number,
- q->hw_queue_number, q->tail);
- q->nb_pending_requests = 0;
- q->csr_tail = q->tail;
-}
-
-static uint16_t
-qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
-{
- register struct qat_queue *queue;
- struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- register uint32_t nb_ops_sent = 0;
- register int ret;
- uint16_t nb_ops_possible = nb_ops;
- register uint8_t *base_addr;
- register uint32_t tail;
- int overflow;
-
- if (unlikely(nb_ops == 0))
- return 0;
-
- /* read params used a lot in main loop into registers */
- queue = &(tmp_qp->tx_q);
- base_addr = (uint8_t *)queue->base_addr;
- tail = queue->tail;
-
- /* Find how many can actually fit on the ring */
- tmp_qp->inflights16 += nb_ops;
- overflow = tmp_qp->inflights16 - queue->max_inflights;
- if (overflow > 0) {
- tmp_qp->inflights16 -= overflow;
- nb_ops_possible = nb_ops - overflow;
- if (nb_ops_possible == 0)
- return 0;
- }
-
- while (nb_ops_sent != nb_ops_possible) {
- ret = tmp_qp->build_request(*ops, base_addr + tail,
- tmp_qp->op_cookies[tail / queue->msg_size],
- tmp_qp->qat_dev_gen);
- if (ret != 0) {
- tmp_qp->stats.enqueue_err_count++;
- /*
- * This message cannot be enqueued,
- * decrease number of ops that wasn't sent
- */
- tmp_qp->inflights16 -= nb_ops_possible - nb_ops_sent;
- if (nb_ops_sent == 0)
- return 0;
- goto kick_tail;
- }
-
- tail = adf_modulo(tail + queue->msg_size, queue->modulo);
- ops++;
- nb_ops_sent++;
- }
-kick_tail:
- queue->tail = tail;
- tmp_qp->stats.enqueued_count += nb_ops_sent;
- queue->nb_pending_requests += nb_ops_sent;
- if (tmp_qp->inflights16 < QAT_CSR_TAIL_FORCE_WRITE_THRESH ||
- queue->nb_pending_requests > QAT_CSR_TAIL_WRITE_THRESH) {
- txq_write_tail(tmp_qp, queue);
- }
- return nb_ops_sent;
-}
-
-static inline
-void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
-{
- uint32_t old_head, new_head;
- uint32_t max_head;
-
- old_head = q->csr_head;
- new_head = q->head;
- max_head = qp->nb_descriptors * q->msg_size;
-
- /* write out free descriptors */
- void *cur_desc = (uint8_t *)q->base_addr + old_head;
-
- if (new_head < old_head) {
- memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, max_head - old_head);
- memset(q->base_addr, ADF_RING_EMPTY_SIG_BYTE, new_head);
- } else {
- memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, new_head - old_head);
- }
- q->nb_processed_responses = 0;
- q->csr_head = new_head;
-
- /* write current head to CSR */
- WRITE_CSR_RING_HEAD(qp->mmap_bar_addr, q->hw_bundle_number,
- q->hw_queue_number, new_head);
-}
-
uint16_t
qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
@@ -336,49 +239,6 @@ qat_sym_process_response(void **op, uint8_t *resp,
return 0;
}
-static uint16_t
-qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
-{
- struct qat_queue *rx_queue, *tx_queue;
- struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- uint32_t head;
- uint32_t resp_counter = 0;
- uint8_t *resp_msg;
-
- rx_queue = &(tmp_qp->rx_q);
- tx_queue = &(tmp_qp->tx_q);
- head = rx_queue->head;
- resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
-
- while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
- resp_counter != nb_ops) {
-
- tmp_qp->process_response(ops, resp_msg,
- tmp_qp->op_cookies[head / rx_queue->msg_size],
- tmp_qp->qat_dev_gen);
-
- head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
-
- resp_msg = (uint8_t *)rx_queue->base_addr + head;
- ops++;
- resp_counter++;
- }
- if (resp_counter > 0) {
- rx_queue->head = head;
- tmp_qp->stats.dequeued_count += resp_counter;
- rx_queue->nb_processed_responses += resp_counter;
- tmp_qp->inflights16 -= resp_counter;
-
- if (rx_queue->nb_processed_responses > QAT_CSR_HEAD_WRITE_THRESH)
- rxq_free_desc(tmp_qp, rx_queue);
- }
- /* also check if tail needs to be advanced */
- if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
- tx_queue->tail != tx_queue->csr_tail) {
- txq_write_tail(tmp_qp, tx_queue);
- }
- return resp_counter;
-}
uint16_t
qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
@@ -903,13 +763,6 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
return 0;
}
-static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
-{
- uint32_t div = data >> shift;
- uint32_t mult = div << shift;
-
- return data - mult;
-}
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 279d3a3ae..39574eeb6 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,57 +27,8 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
-typedef int (*build_request_t)(void *op,
- uint8_t *req, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
-/**< Build a request from an op. */
-
-typedef int (*process_response_t)(void **ops,
- uint8_t *resp, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
-/**< Process a response descriptor and return the associated op. */
-
struct qat_sym_session;
-/**
- * Structure associated with each queue.
- */
-struct qat_queue {
- char memz_name[RTE_MEMZONE_NAMESIZE];
- void *base_addr; /* Base address */
- rte_iova_t base_phys_addr; /* Queue physical address */
- uint32_t head; /* Shadow copy of the head */
- uint32_t tail; /* Shadow copy of the tail */
- uint32_t modulo;
- uint32_t msg_size;
- uint16_t max_inflights;
- uint32_t queue_size;
- uint8_t hw_bundle_number;
- uint8_t hw_queue_number;
- /* HW queue aka ring offset on bundle */
- uint32_t csr_head; /* last written head value */
- uint32_t csr_tail; /* last written tail value */
- uint16_t nb_processed_responses;
- /* number of responses processed since last CSR head write */
- uint16_t nb_pending_requests;
- /* number of requests pending since last CSR tail write */
-};
-
-struct qat_qp {
- void *mmap_bar_addr;
- uint16_t inflights16;
- struct qat_queue tx_q;
- struct qat_queue rx_q;
- struct rte_cryptodev_stats stats;
- struct rte_mempool *op_cookie_pool;
- void **op_cookies;
- uint32_t nb_descriptors;
- enum qat_device_gen qat_dev_gen;
- build_request_t build_request;
- process_response_t process_response;
-} __rte_cache_aligned;
-
-
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 11/31] crypto/qat: separate sym-specific from generic qp setup
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (41 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 10/31] crypto/qat: move generic qp fn to qp file Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 12/31] crypto/qat: move sym-specific qp code to sym file Fiona Trahe
` (20 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Extracted all sym-specific code from qp setup fns, leaving
generic qat_qp_setup fn and helper fns. Created a new
meta-data struct qat_qp_config to hold all the data needed
to create a qp, filled this out in the sym-specific code
and passed to the generic qp_setup fn.
No need now for rx and tx queue_create fns, one generic
queue_create fn replaces these.
Included the service name (e.g. "sym") in the qp memzone
and cookie pool names.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../qat/qat_adf/adf_transport_access_macros.h | 2 +
drivers/crypto/qat/qat_qp.c | 220 +++++++++++----------
drivers/crypto/qat/qat_qp.h | 17 ++
3 files changed, 137 insertions(+), 102 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index bfdbc979f..8b88b69de 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -52,6 +52,8 @@
#define ADF_NUM_BUNDLES_PER_DEV 1
#define ADF_NUM_SYM_QPS_PER_BUNDLE 2
+#define ADF_RING_DIR_TX 0
+#define ADF_RING_DIR_RX 1
/* Valid internal msg size values */
#define ADF_MSG_SIZE_32 0x01
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 56ea10242..5a543f6cb 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -18,8 +18,8 @@
#include "adf_transport_access_macros.h"
-#define ADF_MAX_SYM_DESC 4096
-#define ADF_MIN_SYM_DESC 128
+#define ADF_MAX_DESC 4096
+#define ADF_MIN_DESC 128
#define ADF_SYM_TX_RING_DESC_SIZE 128
#define ADF_SYM_RX_RING_DESC_SIZE 32
#define ADF_SYM_TX_QUEUE_STARTOFF 2
@@ -34,16 +34,9 @@
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
-static int qat_tx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t id, uint32_t nb_desc,
- int socket_id);
-static int qat_rx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t id, uint32_t nb_desc,
- int socket_id);
static void qat_queue_delete(struct qat_queue *queue);
static int qat_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint32_t nb_desc, uint8_t desc_size,
- int socket_id);
+ struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
static void adf_configure_queues(struct qat_qp *queue);
@@ -81,29 +74,19 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
+static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+ struct qat_qp_config *qat_qp_conf)
{
struct qat_qp *qp;
struct rte_pci_device *pci_dev;
- int ret;
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
- PMD_INIT_FUNC_TRACE();
- /* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[queue_pair_id] != NULL) {
- ret = qat_sym_qp_release(dev, queue_pair_id);
- if (ret < 0)
- return ret;
- }
-
- if ((qp_conf->nb_descriptors > ADF_MAX_SYM_DESC) ||
- (qp_conf->nb_descriptors < ADF_MIN_SYM_DESC)) {
+ if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
+ (qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
PMD_DRV_LOG(ERR, "Can't create qp for %u descriptors",
- qp_conf->nb_descriptors);
+ qat_qp_conf->nb_descriptors);
return -EINVAL;
}
@@ -115,13 +98,6 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EINVAL;
}
- if (queue_pair_id >=
- (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device",
- queue_pair_id);
- return -EINVAL;
- }
/* Allocate the queue pair data structure. */
qp = rte_zmalloc("qat PMD qp metadata",
sizeof(*qp), RTE_CACHE_LINE_SIZE);
@@ -129,9 +105,9 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
PMD_DRV_LOG(ERR, "Failed to alloc mem for qp struct");
return -ENOMEM;
}
- qp->nb_descriptors = qp_conf->nb_descriptors;
+ qp->nb_descriptors = qat_qp_conf->nb_descriptors;
qp->op_cookies = rte_zmalloc("qat PMD op cookie pointer",
- qp_conf->nb_descriptors * sizeof(*qp->op_cookies),
+ qat_qp_conf->nb_descriptors * sizeof(*qp->op_cookies),
RTE_CACHE_LINE_SIZE);
if (qp->op_cookies == NULL) {
PMD_DRV_LOG(ERR, "Failed to alloc mem for cookie");
@@ -142,15 +118,15 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qp->mmap_bar_addr = pci_dev->mem_resource[0].addr;
qp->inflights16 = 0;
- if (qat_tx_queue_create(dev, &(qp->tx_q),
- queue_pair_id, qp_conf->nb_descriptors, socket_id) != 0) {
+ if (qat_queue_create(dev, &(qp->tx_q), qat_qp_conf,
+ ADF_RING_DIR_TX) != 0) {
PMD_INIT_LOG(ERR, "Tx queue create failed "
"queue_pair_id=%u", queue_pair_id);
goto create_err;
}
- if (qat_rx_queue_create(dev, &(qp->rx_q),
- queue_pair_id, qp_conf->nb_descriptors, socket_id) != 0) {
+ if (qat_queue_create(dev, &(qp->rx_q), qat_qp_conf,
+ ADF_RING_DIR_RX) != 0) {
PMD_DRV_LOG(ERR, "Rx queue create failed "
"queue_pair_id=%hu", queue_pair_id);
qat_queue_delete(&(qp->tx_q));
@@ -159,16 +135,17 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
adf_configure_queues(qp);
adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
- snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_qp_op_%d_%hu",
- pci_dev->driver->driver.name, dev->data->dev_id,
- queue_pair_id);
+
+ snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
+ pci_dev->driver->driver.name, qat_qp_conf->service_str,
+ dev->data->dev_id, queue_pair_id);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
qp->nb_descriptors,
- sizeof(struct qat_sym_op_cookie), 64, 0,
- NULL, NULL, NULL, NULL, socket_id,
+ qat_qp_conf->cookie_size, 64, 0,
+ NULL, NULL, NULL, NULL, qat_qp_conf->socket_id,
0);
if (!qp->op_cookie_pool) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot create"
@@ -181,6 +158,67 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
PMD_DRV_LOG(ERR, "QAT PMD Cannot get op_cookie");
goto create_err;
}
+ }
+
+ struct qat_pmd_private *internals
+ = dev->data->dev_private;
+ qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->build_request = qat_qp_conf->build_request;
+ qp->process_response = qat_qp_conf->process_response;
+
+ dev->data->queue_pairs[queue_pair_id] = qp;
+ return 0;
+
+create_err:
+ rte_free(qp);
+ return -EFAULT;
+}
+
+
+
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (dev->data->queue_pairs[qp_id] != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+
+ qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
+ qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_TX_QUEUE_STARTOFF;
+ qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_RX_QUEUE_STARTOFF;
+ qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
+ qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
struct qat_sym_op_cookie *sql_cookie =
qp->op_cookies[i];
@@ -196,24 +234,11 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qat_sgl_list_dst);
}
- struct qat_pmd_private *internals
- = dev->data->dev_private;
- qp->qat_dev_gen = internals->qat_dev_gen;
- qp->build_request = qat_sym_build_request;
- qp->process_response = qat_sym_process_response;
+ return ret;
- dev->data->queue_pairs[queue_pair_id] = qp;
- return 0;
-
-create_err:
- if (qp->op_cookie_pool)
- rte_mempool_free(qp->op_cookie_pool);
- rte_free(qp->op_cookies);
- rte_free(qp);
- return -EFAULT;
}
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
@@ -247,38 +272,13 @@ int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
return 0;
}
-static int qat_tx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t qp_id,
- uint32_t nb_desc, int socket_id)
-{
- PMD_INIT_FUNC_TRACE();
- queue->hw_bundle_number = qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE;
- queue->hw_queue_number = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- PMD_DRV_LOG(DEBUG, "TX ring for %u msgs: qp_id %d, bundle %u, ring %u",
- nb_desc, qp_id, queue->hw_bundle_number,
- queue->hw_queue_number);
-
- return qat_queue_create(dev, queue, nb_desc,
- ADF_SYM_TX_RING_DESC_SIZE, socket_id);
-}
-static int qat_rx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t qp_id, uint32_t nb_desc,
- int socket_id)
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
- PMD_INIT_FUNC_TRACE();
- queue->hw_bundle_number = qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE;
- queue->hw_queue_number = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
-
- PMD_DRV_LOG(DEBUG, "RX ring for %u msgs: qp id %d, bundle %u, ring %u",
- nb_desc, qp_id, queue->hw_bundle_number,
- queue->hw_queue_number);
- return qat_queue_create(dev, queue, nb_desc,
- ADF_SYM_RX_RING_DESC_SIZE, socket_id);
+ return qat_qp_release(dev, queue_pair_id);
}
+
static void qat_queue_delete(struct qat_queue *queue)
{
const struct rte_memzone *mz;
@@ -304,15 +304,21 @@ static void qat_queue_delete(struct qat_queue *queue)
static int
qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
- uint32_t nb_desc, uint8_t desc_size, int socket_id)
+ struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
void *io_addr;
const struct rte_memzone *qp_mz;
- uint32_t queue_size_bytes = nb_desc*desc_size;
struct rte_pci_device *pci_dev;
+ int ret = 0;
+ uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
+ qp_conf->tx_msg_size : qp_conf->rx_msg_size);
+ uint32_t queue_size_bytes = (qp_conf->nb_descriptors)*(desc_size);
+
+ queue->hw_bundle_number = qp_conf->hw_bundle_num;
+ queue->hw_queue_number = (dir == ADF_RING_DIR_TX ?
+ qp_conf->tx_ring_num : qp_conf->rx_ring_num);
- PMD_INIT_FUNC_TRACE();
if (desc_size > ADF_MSG_SIZE_TO_BYTES(ADF_MAX_MSG_SIZE)) {
PMD_DRV_LOG(ERR, "Invalid descriptor size %d", desc_size);
return -EINVAL;
@@ -323,11 +329,13 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
/*
* Allocate a memzone for the queue - create a unique name.
*/
- snprintf(queue->memz_name, sizeof(queue->memz_name), "%s_%s_%d_%d_%d",
- pci_dev->driver->driver.name, "qp_mem", dev->data->dev_id,
+ snprintf(queue->memz_name, sizeof(queue->memz_name),
+ "%s_%s_%s_%d_%d_%d",
+ pci_dev->driver->driver.name, qp_conf->service_str,
+ "qp_mem", dev->data->dev_id,
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
- socket_id);
+ qp_conf->socket_id);
if (qp_mz == NULL) {
PMD_DRV_LOG(ERR, "Failed to allocate ring memzone");
return -ENOMEM;
@@ -340,27 +348,31 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
PMD_DRV_LOG(ERR, "Invalid alignment on queue create "
" 0x%"PRIx64"\n",
queue->base_phys_addr);
- return -EFAULT;
+ ret = -EFAULT;
+ goto queue_create_err;
}
- if (adf_verify_queue_size(desc_size, nb_desc, &(queue->queue_size))
- != 0) {
+ if (adf_verify_queue_size(desc_size, qp_conf->nb_descriptors,
+ &(queue->queue_size)) != 0) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
- return -EINVAL;
+ ret = -EINVAL;
+ goto queue_create_err;
}
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
- PMD_DRV_LOG(DEBUG, "RING size in CSR: %u, in bytes %u, nb msgs %u,"
- " msg_size %u, max_inflights %u modulo %u",
- queue->queue_size, queue_size_bytes,
- nb_desc, desc_size, queue->max_inflights,
- queue->modulo);
+ PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
+ " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ queue->memz_name,
+ queue->queue_size, queue_size_bytes,
+ qp_conf->nb_descriptors, desc_size,
+ queue->max_inflights, queue->modulo);
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
- return -EINVAL;
+ ret = -EINVAL;
+ goto queue_create_err;
}
queue->head = 0;
queue->tail = 0;
@@ -379,6 +391,10 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
WRITE_CSR_RING_BASE(io_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_base);
return 0;
+
+queue_create_err:
+ rte_memzone_free(qp_mz);
+ return ret;
}
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 87d55c5f2..edebb8773 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -17,6 +17,23 @@ typedef int (*process_response_t)(void **ops,
/**< Process a response descriptor and return the associated op. */
/**
+ * Structure with data needed for creation of queue pair.
+ */
+struct qat_qp_config {
+ uint8_t hw_bundle_num;
+ uint8_t tx_ring_num;
+ uint8_t rx_ring_num;
+ uint16_t tx_msg_size;
+ uint16_t rx_msg_size;
+ uint32_t nb_descriptors;
+ uint32_t cookie_size;
+ int socket_id;
+ build_request_t build_request;
+ process_response_t process_response;
+ const char *service_str;
+};
+
+/**
* Structure associated with each queue.
*/
struct qat_queue {
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 12/31] crypto/qat: move sym-specific qp code to sym file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (42 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 11/31] crypto/qat: separate sym-specific from generic qp setup Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 13/31] crypto/qat: remove dependencies on cryptodev from common Fiona Trahe
` (19 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Move sym qp setup code from qat_qp.c to qat_sym.c
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 78 ++------------------------------------------
drivers/crypto/qat/qat_qp.h | 6 ++++
drivers/crypto/qat/qat_sym.c | 75 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 84 insertions(+), 75 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 5a543f6cb..d1d2be34f 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -20,11 +20,7 @@
#define ADF_MAX_DESC 4096
#define ADF_MIN_DESC 128
-#define ADF_SYM_TX_RING_DESC_SIZE 128
-#define ADF_SYM_RX_RING_DESC_SIZE 32
-#define ADF_SYM_TX_QUEUE_STARTOFF 2
-/* Offset from bundle start to 1st Sym Tx queue */
-#define ADF_SYM_RX_QUEUE_STARTOFF 10
+
#define ADF_ARB_REG_SLOT 0x1000
#define ADF_ARB_RINGSRVARBEN_OFFSET 0x19C
@@ -74,7 +70,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
{
struct qat_qp *qp;
@@ -174,71 +170,7 @@ static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
-{
- struct qat_qp *qp;
- int ret = 0;
- uint32_t i;
- struct qat_qp_config qat_qp_conf;
-
- /* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[qp_id] != NULL) {
- ret = qat_sym_qp_release(dev, qp_id);
- if (ret < 0)
- return ret;
- }
- if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
- return -EINVAL;
- }
-
-
- qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
- qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
- qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
- qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
- qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
- qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
- qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
- qat_qp_conf.socket_id = socket_id;
- qat_qp_conf.service_str = "sym";
-
- ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
- if (ret != 0)
- return ret;
-
- qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
-
- for (i = 0; i < qp->nb_descriptors; i++) {
-
- struct qat_sym_op_cookie *sql_cookie =
- qp->op_cookies[i];
-
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_src);
-
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_dst);
- }
-
- return ret;
-
-}
-
-static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
@@ -273,10 +205,6 @@ static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
}
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
-{
- return qat_qp_release(dev, queue_pair_id);
-}
static void qat_queue_delete(struct qat_queue *queue)
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index edebb8773..0cdf37f61 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -77,4 +77,10 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
uint16_t
qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+int
+qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id);
+
+int
+qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+ struct qat_qp_config *qat_qp_conf);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index ab8ce2c96..4f86f2258 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -23,6 +23,12 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
+#define ADF_SYM_TX_RING_DESC_SIZE 128
+#define ADF_SYM_RX_RING_DESC_SIZE 32
+#define ADF_SYM_TX_QUEUE_STARTOFF 2
+/* Offset from bundle start to 1st Sym Tx queue */
+#define ADF_SYM_RX_QUEUE_STARTOFF 10
+
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -798,3 +804,72 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
}
+
+
+
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+{
+ return qat_qp_release(dev, queue_pair_id);
+}
+
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (dev->data->queue_pairs[qp_id] != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+
+ qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
+ qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_TX_QUEUE_STARTOFF;
+ qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_RX_QUEUE_STARTOFF;
+ qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
+ qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
+
+ struct qat_sym_op_cookie *sql_cookie =
+ qp->op_cookies[i];
+
+ sql_cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_list_src);
+
+ sql_cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_list_dst);
+ }
+
+ return ret;
+
+}
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 13/31] crypto/qat: remove dependencies on cryptodev from common
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (43 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 12/31] crypto/qat: move sym-specific qp code to sym file Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 14/31] crypto/qat: move defines from sym to qp header file Fiona Trahe
` (18 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Remove dependence on rte_cryptodev from common qp code
to facilitate being used by other device types in future.
Transferred required data into qat-specific structures.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.h | 11 +++++-
drivers/crypto/qat/qat_qp.c | 67 +++++++++++++++++++---------------
drivers/crypto/qat/qat_qp.h | 8 +++-
drivers/crypto/qat/qat_sym.c | 15 +++++---
drivers/crypto/qat/rte_qat_cryptodev.c | 4 +-
5 files changed, 66 insertions(+), 39 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 2cb8e7612..64706abae 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -14,7 +14,11 @@ extern uint8_t cryptodev_qat_driver_id;
extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-/** private data structure for each QAT device */
+/** private data structure for each QAT device.
+ * In this context a QAT device is a device offering only one service,
+ * so there can be more than 1 device on a pci_dev (VF),
+ * one for symmetric crypto, one for compression
+ */
struct qat_pmd_private {
unsigned int max_nb_queue_pairs;
/**< Max number of queue pairs supported by device */
@@ -23,6 +27,11 @@ struct qat_pmd_private {
enum qat_device_gen qat_dev_gen;
/**< QAT device generation */
const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device capabilities */
+ struct rte_pci_device *pci_dev;
+ /**< PCI information. */
+ uint8_t dev_id;
+ /**< Device ID for this instance */
};
int qat_dev_config(struct rte_cryptodev *dev,
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index d1d2be34f..b831ab420 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -6,7 +6,6 @@
#include <rte_dev.h>
#include <rte_malloc.h>
#include <rte_memzone.h>
-#include <rte_cryptodev_pmd.h>
#include <rte_pci.h>
#include <rte_bus_pci.h>
#include <rte_atomic.h>
@@ -31,7 +30,7 @@
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
-static int qat_queue_create(struct rte_cryptodev *dev,
+static int qat_queue_create(struct qat_pmd_private *qat_dev,
struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
@@ -70,14 +69,19 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_qp_setup(struct qat_pmd_private *qat_dev,
+ struct qat_qp **qp_addr,
+ uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
+
{
struct qat_qp *qp;
- struct rte_pci_device *pci_dev;
+ struct rte_pci_device *pci_dev = qat_dev->pci_dev;
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
+ PMD_DRV_LOG(DEBUG, "Setup qp %u on device %d gen %d",
+ queue_pair_id, qat_dev->dev_id, qat_dev->qat_dev_gen);
if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
(qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
@@ -86,8 +90,6 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EINVAL;
}
- pci_dev = RTE_DEV_TO_PCI(dev->device);
-
if (pci_dev->mem_resource[0].addr == NULL) {
PMD_DRV_LOG(ERR, "Could not find VF config space "
"(UIO driver attached?).");
@@ -114,14 +116,14 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qp->mmap_bar_addr = pci_dev->mem_resource[0].addr;
qp->inflights16 = 0;
- if (qat_queue_create(dev, &(qp->tx_q), qat_qp_conf,
+ if (qat_queue_create(qat_dev, &(qp->tx_q), qat_qp_conf,
ADF_RING_DIR_TX) != 0) {
PMD_INIT_LOG(ERR, "Tx queue create failed "
"queue_pair_id=%u", queue_pair_id);
goto create_err;
}
- if (qat_queue_create(dev, &(qp->rx_q), qat_qp_conf,
+ if (qat_queue_create(qat_dev, &(qp->rx_q), qat_qp_conf,
ADF_RING_DIR_RX) != 0) {
PMD_DRV_LOG(ERR, "Rx queue create failed "
"queue_pair_id=%hu", queue_pair_id);
@@ -134,7 +136,7 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
pci_dev->driver->driver.name, qat_qp_conf->service_str,
- dev->data->dev_id, queue_pair_id);
+ qat_dev->dev_id, queue_pair_id);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
@@ -156,13 +158,15 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
}
}
- struct qat_pmd_private *internals
- = dev->data->dev_private;
- qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->qat_dev_gen = qat_dev->qat_dev_gen;
qp->build_request = qat_qp_conf->build_request;
qp->process_response = qat_qp_conf->process_response;
+ qp->qat_dev = qat_dev;
+
+ PMD_DRV_LOG(DEBUG, "QP setup complete: id: %d, cookiepool: %s",
+ queue_pair_id, op_cookie_pool_name);
- dev->data->queue_pairs[queue_pair_id] = qp;
+ *qp_addr = qp;
return 0;
create_err:
@@ -170,10 +174,9 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_qp_release(struct qat_qp **qp_addr)
{
- struct qat_qp *qp =
- (struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
+ struct qat_qp *qp = *qp_addr;
uint32_t i;
PMD_INIT_FUNC_TRACE();
@@ -182,6 +185,9 @@ int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
return 0;
}
+ PMD_DRV_LOG(DEBUG, "Free qp on qat_pci device %d",
+ qp->qat_dev->dev_id);
+
/* Don't free memory if there are still responses to be processed */
if (qp->inflights16 == 0) {
qat_queue_delete(&(qp->tx_q));
@@ -200,13 +206,11 @@ int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
rte_free(qp->op_cookies);
rte_free(qp);
- dev->data->queue_pairs[queue_pair_id] = NULL;
+ *qp_addr = NULL;
return 0;
}
-
-
static void qat_queue_delete(struct qat_queue *queue)
{
const struct rte_memzone *mz;
@@ -216,6 +220,9 @@ static void qat_queue_delete(struct qat_queue *queue)
PMD_DRV_LOG(DEBUG, "Invalid queue");
return;
}
+ PMD_DRV_LOG(DEBUG, "Free ring %d, memzone: %s",
+ queue->hw_queue_number, queue->memz_name);
+
mz = rte_memzone_lookup(queue->memz_name);
if (mz != NULL) {
/* Write an unused pattern to the queue memory. */
@@ -231,13 +238,13 @@ static void qat_queue_delete(struct qat_queue *queue)
}
static int
-qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
+qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
void *io_addr;
const struct rte_memzone *qp_mz;
- struct rte_pci_device *pci_dev;
+ struct rte_pci_device *pci_dev = qat_dev->pci_dev;
int ret = 0;
uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
qp_conf->tx_msg_size : qp_conf->rx_msg_size);
@@ -252,15 +259,13 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
return -EINVAL;
}
- pci_dev = RTE_DEV_TO_PCI(dev->device);
-
/*
* Allocate a memzone for the queue - create a unique name.
*/
snprintf(queue->memz_name, sizeof(queue->memz_name),
"%s_%s_%s_%d_%d_%d",
pci_dev->driver->driver.name, qp_conf->service_str,
- "qp_mem", dev->data->dev_id,
+ "qp_mem", qat_dev->dev_id,
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
qp_conf->socket_id);
@@ -290,12 +295,6 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
- PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
- " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
- queue->memz_name,
- queue->queue_size, queue_size_bytes,
- qp_conf->nb_descriptors, desc_size,
- queue->max_inflights, queue->modulo);
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
@@ -318,6 +317,14 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
WRITE_CSR_RING_BASE(io_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_base);
+
+ PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
+ " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ queue->memz_name,
+ queue->queue_size, queue_size_bytes,
+ qp_conf->nb_descriptors, desc_size,
+ queue->max_inflights, queue->modulo);
+
return 0;
queue_create_err:
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 0cdf37f61..7bd8fdcec 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,6 +5,7 @@
#define _QAT_QP_H_
#include "qat_common.h"
+#include "qat_device.h"
typedef int (*build_request_t)(void *op,
uint8_t *req, void *op_cookie,
@@ -69,6 +70,8 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
process_response_t process_response;
+ struct qat_pmd_private *qat_dev;
+ /**< qat device this qp is on */
} __rte_cache_aligned;
uint16_t
@@ -78,9 +81,10 @@ uint16_t
qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
int
-qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id);
+qat_qp_release(struct qat_qp **qp_addr);
int
-qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+qat_qp_setup(struct qat_pmd_private *qat_dev,
+ struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 4f86f2258..8ab95ac43 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -809,7 +809,10 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
- return qat_qp_release(dev, queue_pair_id);
+ PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
+ queue_pair_id, dev->data->dev_id);
+ return qat_qp_release((struct qat_qp **)
+ &(dev->data->queue_pairs[queue_pair_id]));
}
int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
@@ -820,9 +823,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
int ret = 0;
uint32_t i;
struct qat_qp_config qat_qp_conf;
+ struct qat_qp **qp_addr =
+ (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
+ struct qat_pmd_private *qat_private = dev->data->dev_private;
/* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[qp_id] != NULL) {
+ if (*qp_addr != NULL) {
ret = qat_sym_qp_release(dev, qp_id);
if (ret < 0)
return ret;
@@ -833,7 +839,6 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
return -EINVAL;
}
-
qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
ADF_SYM_TX_QUEUE_STARTOFF;
@@ -848,11 +853,11 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.socket_id = socket_id;
qat_qp_conf.service_str = "sym";
- ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ ret = qat_qp_setup(qat_private, qp_addr, qp_id, &qat_qp_conf);
if (ret != 0)
return ret;
- qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+ qp = (struct qat_qp *)*qp_addr;
for (i = 0; i < qp->nb_descriptors; i++) {
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 05cb5ef40..fe19b18b6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -96,7 +96,9 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
internals = cryptodev->data->dev_private;
internals->max_nb_sessions = init_params->max_nb_sessions;
- switch (pci_dev->id.device_id) {
+ internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
+ internals->dev_id = cryptodev->data->dev_id;
+ switch (internals->pci_dev->id.device_id) {
case 0x0443:
internals->qat_dev_gen = QAT_GEN1;
internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 14/31] crypto/qat: move defines from sym to qp header file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (44 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 13/31] crypto/qat: remove dependencies on cryptodev from common Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 15/31] crypto/qat: create data structures to support different generations Fiona Trahe
` (17 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Move defines related to coalescing from sym header file to qp header
file as these will be common for all services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.h | 7 +++++++
drivers/crypto/qat/qat_sym.h | 7 -------
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 7bd8fdcec..8cf072c55 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -7,6 +7,13 @@
#include "qat_common.h"
#include "qat_device.h"
+#define QAT_CSR_HEAD_WRITE_THRESH 32U
+/* number of requests to accumulate before writing head CSR */
+#define QAT_CSR_TAIL_WRITE_THRESH 32U
+/* number of requests to accumulate before writing tail CSR */
+#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
+/* number of inflights below which no tail write coalescing should occur */
+
typedef int (*build_request_t)(void *op,
uint8_t *req, void *op_cookie,
enum qat_device_gen qat_dev_gen);
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 39574eeb6..b92ec72de 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -20,13 +20,6 @@
(((num) + (align) - 1) & ~((align) - 1))
#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
-#define QAT_CSR_HEAD_WRITE_THRESH 32U
-/* number of requests to accumulate before writing head CSR */
-#define QAT_CSR_TAIL_WRITE_THRESH 32U
-/* number of requests to accumulate before writing tail CSR */
-#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
-/* number of inflights below which no tail write coalescing should occur */
-
struct qat_sym_session;
int
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 15/31] crypto/qat: create data structures to support different generations
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (45 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 14/31] crypto/qat: move defines from sym to qp header file Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 16/31] crypto/qat: rename sgl related objects Fiona Trahe
` (16 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Create data structures to support different generations
of qat hardware supplying services through different queue pairs.
- Add two new structs qat_gen_hw_data and qat_qp_hw_dat
- Add a qat_service_type enum
An array of qat_qp_hw_data elements is
initialised with constants, these are arranged so that the qp_id used
on the API can be used as an index to pick up the qp data to use.
The constants are common to current generations,
new arrays will be added for future generations.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../qat/qat_adf/adf_transport_access_macros.h | 6 +-
drivers/crypto/qat/qat_common.h | 8 ++
drivers/crypto/qat/qat_device.c | 21 ++++-
drivers/crypto/qat/qat_device.h | 9 +++
drivers/crypto/qat/qat_qp.c | 94 ++++++++++++++++++++--
drivers/crypto/qat/qat_qp.h | 18 ++++-
drivers/crypto/qat/qat_sym.c | 27 +++----
7 files changed, 154 insertions(+), 29 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index 8b88b69de..2136d54ab 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -51,7 +51,8 @@
#define ADF_DEFAULT_RING_SIZE ADF_RING_SIZE_16K
#define ADF_NUM_BUNDLES_PER_DEV 1
-#define ADF_NUM_SYM_QPS_PER_BUNDLE 2
+/* Maximum number of qps for any service type */
+#define ADF_MAX_QPS_PER_BUNDLE 4
#define ADF_RING_DIR_TX 0
#define ADF_RING_DIR_RX 1
@@ -132,4 +133,5 @@ do { \
#define WRITE_CSR_INT_FLAG_AND_COL(csr_base_addr, bank, value) \
ADF_CSR_WR(csr_base_addr, (ADF_RING_BUNDLE_SIZE * bank) + \
ADF_RING_CSR_INT_FLAG_AND_COL, value)
-#endif
+
+#endif /*ADF_TRANSPORT_ACCESS_MACROS_H */
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 7802e96f9..c3e7bd9a7 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -23,6 +23,14 @@ enum qat_device_gen {
QAT_GEN2,
};
+enum qat_service_type {
+ QAT_SERVICE_ASYMMETRIC = 0,
+ QAT_SERVICE_SYMMETRIC,
+ QAT_SERVICE_COMPRESSION,
+ QAT_SERVICE_INVALID
+};
+#define QAT_MAX_SERVICES (QAT_SERVICE_INVALID)
+
/**< Common struct for scatter-gather list operations */
struct qat_alg_buf {
uint32_t len;
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index ac6bd1af6..cdf4f7058 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -4,6 +4,21 @@
#include "qat_device.h"
#include "adf_transport_access_macros.h"
+#include "qat_qp.h"
+
+/* Hardware device information per generation */
+__extension__
+struct qat_gen_hw_data qp_gen_config[] = {
+ [QAT_GEN1] = {
+ .dev_gen = QAT_GEN1,
+ .qp_hw_data = qat_gen1_qps,
+ },
+ [QAT_GEN2] = {
+ .dev_gen = QAT_GEN2,
+ .qp_hw_data = qat_gen1_qps,
+ /* gen2 has same ring layout as gen1 */
+ },
+};
int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
@@ -42,12 +57,14 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info)
{
struct qat_pmd_private *internals = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[internals->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
if (info != NULL) {
info->max_nb_queue_pairs =
- ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV;
+ qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
info->feature_flags = dev->feature_flags;
info->capabilities = internals->qat_dev_capabilities;
info->sym.max_nb_sessions = internals->max_nb_sessions;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 64706abae..0983e3c2e 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -8,6 +8,8 @@
#include <rte_bus_pci.h>
#include "qat_common.h"
#include "qat_logs.h"
+#include "adf_transport_access_macros.h"
+#include "qat_qp.h"
extern uint8_t cryptodev_qat_driver_id;
@@ -34,6 +36,13 @@ struct qat_pmd_private {
/**< Device ID for this instance */
};
+struct qat_gen_hw_data {
+ enum qat_device_gen dev_gen;
+ const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
+};
+
+extern struct qat_gen_hw_data qp_gen_config[];
+
int qat_dev_config(struct rte_cryptodev *dev,
struct rte_cryptodev_config *config);
int qat_dev_start(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index b831ab420..656645e4c 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,10 +13,10 @@
#include "qat_logs.h"
#include "qat_qp.h"
-#include "qat_sym.h"
-
+#include "qat_device.h"
#include "adf_transport_access_macros.h"
+
#define ADF_MAX_DESC 4096
#define ADF_MIN_DESC 128
@@ -27,6 +27,78 @@
ADF_CSR_WR(csr_addr, ADF_ARB_RINGSRVARBEN_OFFSET + \
(ADF_ARB_REG_SLOT * index), value)
+__extension__
+const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
+ [ADF_MAX_QPS_PER_BUNDLE] = {
+ /* queue pairs which provide an asymmetric crypto service */
+ [QAT_SERVICE_ASYMMETRIC] = {
+ {
+ .service_type = QAT_SERVICE_ASYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 0,
+ .rx_ring_num = 8,
+ .tx_msg_size = 64,
+ .rx_msg_size = 32,
+
+ }, {
+ .service_type = QAT_SERVICE_ASYMMETRIC,
+ .tx_ring_num = 1,
+ .rx_ring_num = 9,
+ .tx_msg_size = 64,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ },
+ /* queue pairs which provide a symmetric crypto service */
+ [QAT_SERVICE_SYMMETRIC] = {
+ {
+ .service_type = QAT_SERVICE_SYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 2,
+ .rx_ring_num = 10,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ },
+ {
+ .service_type = QAT_SERVICE_SYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 3,
+ .rx_ring_num = 11,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ },
+ /* queue pairs which provide a compression service */
+ [QAT_SERVICE_COMPRESSION] = {
+ {
+ .service_type = QAT_SERVICE_COMPRESSION,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 6,
+ .rx_ring_num = 14,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_COMPRESSION,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 7,
+ .rx_ring_num = 15,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ }
+};
+
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
@@ -38,6 +110,18 @@ static void adf_configure_queues(struct qat_qp *queue);
static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr);
static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr);
+
+int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
+ enum qat_service_type service)
+{
+ int i, count;
+
+ for (i = 0, count = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++)
+ if (qp_hw_data[i].service_type == service)
+ count++;
+ return count * ADF_NUM_BUNDLES_PER_DEV;
+}
+
static const struct rte_memzone *
queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
int socket_id)
@@ -247,12 +331,12 @@ qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
struct rte_pci_device *pci_dev = qat_dev->pci_dev;
int ret = 0;
uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
- qp_conf->tx_msg_size : qp_conf->rx_msg_size);
+ qp_conf->hw->tx_msg_size : qp_conf->hw->rx_msg_size);
uint32_t queue_size_bytes = (qp_conf->nb_descriptors)*(desc_size);
- queue->hw_bundle_number = qp_conf->hw_bundle_num;
+ queue->hw_bundle_number = qp_conf->hw->hw_bundle_num;
queue->hw_queue_number = (dir == ADF_RING_DIR_TX ?
- qp_conf->tx_ring_num : qp_conf->rx_ring_num);
+ qp_conf->hw->tx_ring_num : qp_conf->hw->rx_ring_num);
if (desc_size > ADF_MSG_SIZE_TO_BYTES(ADF_MAX_MSG_SIZE)) {
PMD_DRV_LOG(ERR, "Invalid descriptor size %d", desc_size);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 8cf072c55..f808e16a5 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,7 +5,8 @@
#define _QAT_QP_H_
#include "qat_common.h"
-#include "qat_device.h"
+#include <rte_cryptodev_pmd.h>
+#include "adf_transport_access_macros.h"
#define QAT_CSR_HEAD_WRITE_THRESH 32U
/* number of requests to accumulate before writing head CSR */
@@ -27,12 +28,19 @@ typedef int (*process_response_t)(void **ops,
/**
* Structure with data needed for creation of queue pair.
*/
-struct qat_qp_config {
+struct qat_qp_hw_data {
+ enum qat_service_type service_type;
uint8_t hw_bundle_num;
uint8_t tx_ring_num;
uint8_t rx_ring_num;
uint16_t tx_msg_size;
uint16_t rx_msg_size;
+};
+/**
+ * Structure with data needed for creation of queue pair.
+ */
+struct qat_qp_config {
+ const struct qat_qp_hw_data *hw;
uint32_t nb_descriptors;
uint32_t cookie_size;
int socket_id;
@@ -81,6 +89,8 @@ struct qat_qp {
/**< qat device this qp is on */
} __rte_cache_aligned;
+extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_PER_BUNDLE];
+
uint16_t
qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
@@ -94,4 +104,8 @@ int
qat_qp_setup(struct qat_pmd_private *qat_dev,
struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
+
+int
+qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
+ enum qat_service_type service);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 8ab95ac43..e448dc859 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -16,6 +16,7 @@
#include "qat_sym.h"
#include "qat_qp.h"
#include "adf_transport_access_macros.h"
+#include "qat_device.h"
#define BYTE_LENGTH 8
/* bpi is only used for partial blocks of DES and AES
@@ -23,12 +24,6 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-#define ADF_SYM_TX_RING_DESC_SIZE 128
-#define ADF_SYM_RX_RING_DESC_SIZE 32
-#define ADF_SYM_TX_QUEUE_STARTOFF 2
-/* Offset from bundle start to 1st Sym Tx queue */
-#define ADF_SYM_RX_QUEUE_STARTOFF 10
-
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -805,12 +800,11 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
}
-
-
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
queue_pair_id, dev->data->dev_id);
+
return qat_qp_release((struct qat_qp **)
&(dev->data->queue_pairs[queue_pair_id]));
}
@@ -823,9 +817,14 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
int ret = 0;
uint32_t i;
struct qat_qp_config qat_qp_conf;
+
struct qat_qp **qp_addr =
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
struct qat_pmd_private *qat_private = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[qat_private->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+ const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
/* If qp is already in use free ring memory and qp metadata. */
if (*qp_addr != NULL) {
@@ -833,19 +832,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
if (ret < 0)
return ret;
}
- if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
+ if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
return -EINVAL;
}
- qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
- qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
- qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
- qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.hw = qp_hw_data;
qat_qp_conf.build_request = qat_sym_build_request;
qat_qp_conf.process_response = qat_sym_process_response;
qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
@@ -876,5 +868,4 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
}
return ret;
-
}
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 16/31] crypto/qat: rename sgl related objects
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (46 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 15/31] crypto/qat: create data structures to support different generations Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 17/31] crypto/qat: move sgl related element to appropriate files Fiona Trahe
` (15 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev
Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak, ArkadiuszX Kusztal
Change SGL (Scatter-Gather List) related structs and member names
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
---
drivers/crypto/qat/qat_common.h | 10 +++++-----
drivers/crypto/qat/qat_sym.c | 28 ++++++++++++++--------------
2 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index c3e7bd9a7..193639550 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -32,22 +32,22 @@ enum qat_service_type {
#define QAT_MAX_SERVICES (QAT_SERVICE_INVALID)
/**< Common struct for scatter-gather list operations */
-struct qat_alg_buf {
+struct qat_flat_buf {
uint32_t len;
uint32_t resrvd;
uint64_t addr;
} __rte_packed;
-struct qat_alg_buf_list {
+struct qat_sgl {
uint64_t resrvd;
uint32_t num_bufs;
uint32_t num_mapped_bufs;
- struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
+ struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
struct qat_sym_op_cookie {
- struct qat_alg_buf_list qat_sgl_list_src;
- struct qat_alg_buf_list qat_sgl_list_dst;
+ struct qat_sgl qat_sgl_src;
+ struct qat_sgl qat_sgl_dst;
phys_addr_t qat_sgl_src_phys_addr;
phys_addr_t qat_sgl_dst_phys_addr;
};
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index e448dc859..a9beff064 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -250,20 +250,20 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
static inline int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
- struct qat_alg_buf_list *list, uint32_t data_len)
+ struct qat_sgl *list, uint32_t data_len)
{
int nr = 1;
uint32_t buf_len = rte_pktmbuf_iova(buf) -
buff_start + rte_pktmbuf_data_len(buf);
- list->bufers[0].addr = buff_start;
- list->bufers[0].resrvd = 0;
- list->bufers[0].len = buf_len;
+ list->buffers[0].addr = buff_start;
+ list->buffers[0].resrvd = 0;
+ list->buffers[0].len = buf_len;
if (data_len <= buf_len) {
list->num_bufs = nr;
- list->bufers[0].len = data_len;
+ list->buffers[0].len = data_len;
return 0;
}
@@ -276,15 +276,15 @@ qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
return -EINVAL;
}
- list->bufers[nr].len = rte_pktmbuf_data_len(buf);
- list->bufers[nr].resrvd = 0;
- list->bufers[nr].addr = rte_pktmbuf_iova(buf);
+ list->buffers[nr].len = rte_pktmbuf_data_len(buf);
+ list->buffers[nr].resrvd = 0;
+ list->buffers[nr].addr = rte_pktmbuf_iova(buf);
- buf_len += list->bufers[nr].len;
+ buf_len += list->buffers[nr].len;
buf = buf->next;
if (buf_len > data_len) {
- list->bufers[nr].len -=
+ list->buffers[nr].len -=
buf_len - data_len;
buf = NULL;
}
@@ -695,7 +695,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
ICP_QAT_FW_COMN_PTR_TYPE_SET(qat_req->comn_hdr.comn_req_flags,
QAT_COMN_PTR_TYPE_SGL);
ret = qat_sgl_fill_array(op->sym->m_src, src_buf_start,
- &cookie->qat_sgl_list_src,
+ &cookie->qat_sgl_src,
qat_req->comn_mid.src_length);
if (ret) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot fill sgl array");
@@ -709,7 +709,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
else {
ret = qat_sgl_fill_array(op->sym->m_dst,
dst_buf_start,
- &cookie->qat_sgl_list_dst,
+ &cookie->qat_sgl_dst,
qat_req->comn_mid.dst_length);
if (ret) {
@@ -859,12 +859,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
sql_cookie->qat_sgl_src_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_src);
+ qat_sgl_src);
sql_cookie->qat_sgl_dst_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_dst);
+ qat_sgl_dst);
}
return ret;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 17/31] crypto/qat: move sgl related element to appropriate files
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (47 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 16/31] crypto/qat: rename sgl related objects Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 18/31] crypto/qat: add QAT PCI device struct Fiona Trahe
` (14 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev
Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak, ArkadiuszX Kusztal
Move SGL (Scatter-Gather List) related functions to common file
Move qat_sym_op_cookie struct to sym header file
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 1 +
drivers/crypto/qat/meson.build | 1 +
drivers/crypto/qat/qat_common.c | 53 +++++++++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_common.h | 11 ++++-----
drivers/crypto/qat/qat_sym.c | 47 ------------------------------------
drivers/crypto/qat/qat_sym.h | 7 ++++++
6 files changed, 67 insertions(+), 53 deletions(-)
create mode 100644 drivers/crypto/qat/qat_common.c
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 8cb802b9d..902c47ff4 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_common.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
# export include files
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index e596006da..12910c377 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -7,6 +7,7 @@ if not dep.found()
endif
sources = files('qat_sym.c', 'qat_qp.c',
'qat_sym_session.c',
+ 'qat_common.c',
'rte_qat_cryptodev.c',
'qat_device.c')
includes += include_directories('qat_adf')
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
new file mode 100644
index 000000000..a8865904f
--- /dev/null
+++ b/drivers/crypto/qat/qat_common.c
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_common.h"
+#include "qat_logs.h"
+
+int
+qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
+ struct qat_sgl *list, uint32_t data_len)
+{
+ int nr = 1;
+
+ uint32_t buf_len = rte_pktmbuf_iova(buf) -
+ buf_start + rte_pktmbuf_data_len(buf);
+
+ list->buffers[0].addr = buf_start;
+ list->buffers[0].resrvd = 0;
+ list->buffers[0].len = buf_len;
+
+ if (data_len <= buf_len) {
+ list->num_bufs = nr;
+ list->buffers[0].len = data_len;
+ return 0;
+ }
+
+ buf = buf->next;
+ while (buf) {
+ if (unlikely(nr == QAT_SGL_MAX_NUMBER)) {
+ PMD_DRV_LOG(ERR,
+ "QAT PMD exceeded size of QAT SGL entry(%u)",
+ QAT_SGL_MAX_NUMBER);
+ return -EINVAL;
+ }
+
+ list->buffers[nr].len = rte_pktmbuf_data_len(buf);
+ list->buffers[nr].resrvd = 0;
+ list->buffers[nr].addr = rte_pktmbuf_iova(buf);
+
+ buf_len += list->buffers[nr].len;
+ buf = buf->next;
+
+ if (buf_len > data_len) {
+ list->buffers[nr].len -=
+ buf_len - data_len;
+ buf = NULL;
+ }
+ ++nr;
+ }
+ list->num_bufs = nr;
+
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 193639550..77ffc8f72 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -6,6 +6,8 @@
#include <stdint.h>
+#include <rte_mbuf.h>
+
/**< Intel(R) QAT Symmetric Crypto PMD device name */
#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
@@ -45,11 +47,8 @@ struct qat_sgl {
struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
-struct qat_sym_op_cookie {
- struct qat_sgl qat_sgl_src;
- struct qat_sgl qat_sgl_dst;
- phys_addr_t qat_sgl_src_phys_addr;
- phys_addr_t qat_sgl_dst_phys_addr;
-};
+int
+qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
+ struct qat_sgl *list, uint32_t data_len);
#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index a9beff064..b74dfa634 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -248,53 +248,6 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
-static inline int
-qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
- struct qat_sgl *list, uint32_t data_len)
-{
- int nr = 1;
-
- uint32_t buf_len = rte_pktmbuf_iova(buf) -
- buff_start + rte_pktmbuf_data_len(buf);
-
- list->buffers[0].addr = buff_start;
- list->buffers[0].resrvd = 0;
- list->buffers[0].len = buf_len;
-
- if (data_len <= buf_len) {
- list->num_bufs = nr;
- list->buffers[0].len = data_len;
- return 0;
- }
-
- buf = buf->next;
- while (buf) {
- if (unlikely(nr == QAT_SGL_MAX_NUMBER)) {
- PMD_DRV_LOG(ERR, "QAT PMD exceeded size of QAT SGL"
- " entry(%u)",
- QAT_SGL_MAX_NUMBER);
- return -EINVAL;
- }
-
- list->buffers[nr].len = rte_pktmbuf_data_len(buf);
- list->buffers[nr].resrvd = 0;
- list->buffers[nr].addr = rte_pktmbuf_iova(buf);
-
- buf_len += list->buffers[nr].len;
- buf = buf->next;
-
- if (buf_len > data_len) {
- list->buffers[nr].len -=
- buf_len - data_len;
- buf = NULL;
- }
- ++nr;
- }
- list->num_bufs = nr;
-
- return 0;
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index b92ec72de..37bec3ce3 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -22,6 +22,13 @@
struct qat_sym_session;
+struct qat_sym_op_cookie {
+ struct qat_sgl qat_sgl_src;
+ struct qat_sgl qat_sgl_dst;
+ phys_addr_t qat_sgl_src_phys_addr;
+ phys_addr_t qat_sgl_dst_phys_addr;
+};
+
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 18/31] crypto/qat: add QAT PCI device struct
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (48 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 17/31] crypto/qat: move sgl related element to appropriate files Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 19/31] crypto/qat: separate the name used for PCI reg from crypto name Fiona Trahe
` (13 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
- Added struct qat_pci_device to use internally in QAT PMD
to avoid dependencies on rte_cryptodev or rte_compressdev
- Added a global array of these
- Restructured probe/release to separate QAT common init/clear
from crypto pmd create/destroy.
- In QAT common part allocated a qat_pci_device and populated it
- Removed meaningless check in probe for RTE_PROC_PRIMARY
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 127 ++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_device.h | 60 +++++++++++++-
drivers/crypto/qat/rte_qat_cryptodev.c | 140 ++++++++++++++++++++++++---------
3 files changed, 286 insertions(+), 41 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index cdf4f7058..75af1e8bc 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -20,6 +20,10 @@ struct qat_gen_hw_data qp_gen_config[] = {
},
};
+
+static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
+static int qat_nb_pci_devices;
+
int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
{
@@ -72,3 +76,126 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
info->pci_dev = RTE_DEV_TO_PCI(dev->device);
}
}
+
+
+static struct qat_pci_device *
+qat_pci_get_dev(uint8_t dev_id)
+{
+ return &qat_pci_devices[dev_id];
+}
+static struct qat_pci_device *
+qat_pci_get_named_dev(const char *name)
+{
+ struct qat_pci_device *dev;
+ unsigned int i;
+
+ if (name == NULL)
+ return NULL;
+
+ for (i = 0; i < QAT_MAX_PCI_DEVICES; i++) {
+ dev = &qat_pci_devices[i];
+
+ if ((dev->attached == QAT_ATTACHED) &&
+ (strcmp(dev->name, name) == 0))
+ return dev;
+ }
+
+ return NULL;
+}
+
+static uint8_t
+qat_pci_find_free_device_index(void)
+{
+ uint8_t dev_id;
+
+ for (dev_id = 0; dev_id < QAT_MAX_PCI_DEVICES; dev_id++) {
+ if (qat_pci_devices[dev_id].attached == QAT_DETACHED)
+ break;
+ }
+ return dev_id;
+}
+
+struct qat_pci_device *
+qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev)
+{
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+
+ return qat_pci_get_named_dev(name);
+}
+
+struct qat_pci_device *
+qat_pci_device_allocate(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_dev;
+ uint8_t qat_dev_id;
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+
+ if (qat_pci_get_named_dev(name) != NULL) {
+ PMD_DRV_LOG(ERR, "QAT device with name %s already allocated!",
+ name);
+ return NULL;
+ }
+
+ qat_dev_id = qat_pci_find_free_device_index();
+ if (qat_dev_id == QAT_MAX_PCI_DEVICES) {
+ PMD_DRV_LOG(ERR, "Reached maximum number of QAT devices");
+ return NULL;
+ }
+
+ qat_dev = qat_pci_get_dev(qat_dev_id);
+ snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
+ qat_dev->pci_dev = pci_dev;
+ switch (qat_dev->pci_dev->id.device_id) {
+ case 0x0443:
+ qat_dev->qat_dev_gen = QAT_GEN1;
+ break;
+ case 0x37c9:
+ case 0x19e3:
+ case 0x6f55:
+ qat_dev->qat_dev_gen = QAT_GEN2;
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "Invalid dev_id, can't determine generation");
+ return NULL;
+ }
+
+ rte_spinlock_init(&qat_dev->arb_csr_lock);
+
+ qat_dev->attached = QAT_ATTACHED;
+
+ qat_nb_pci_devices++;
+
+ PMD_DRV_LOG(DEBUG, "QAT device %d allocated, total QATs %d",
+ qat_dev_id, qat_nb_pci_devices);
+
+ return qat_dev;
+}
+
+int
+qat_pci_device_release(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_dev;
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ if (pci_dev == NULL)
+ return -EINVAL;
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ qat_dev = qat_pci_get_named_dev(name);
+ if (qat_dev != NULL) {
+
+ /* Check that there are no service devs still on pci device */
+ if (qat_dev->sym_dev != NULL)
+ return -EBUSY;
+
+ qat_dev->attached = QAT_DETACHED;
+ qat_nb_pci_devices--;
+ }
+ PMD_DRV_LOG(DEBUG, "QAT device %s released, total QATs %d",
+ name, qat_nb_pci_devices);
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 0983e3c2e..d83ad632c 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,15 +11,58 @@
#include "adf_transport_access_macros.h"
#include "qat_qp.h"
+
+#define QAT_DETACHED (0)
+#define QAT_ATTACHED (1)
+
+#define QAT_MAX_PCI_DEVICES 48
+#define QAT_DEV_NAME_MAX_LEN 64
+
+
extern uint8_t cryptodev_qat_driver_id;
extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-/** private data structure for each QAT device.
- * In this context a QAT device is a device offering only one service,
- * so there can be more than 1 device on a pci_dev (VF),
- * one for symmetric crypto, one for compression
+/*
+ * This struct holds all the data about a QAT pci device
+ * including data about all services it supports.
+ * It contains
+ * - hw_data
+ * - config data
+ * - runtime data
+ */
+struct qat_pci_device {
+
+ /* data used by all services */
+ char name[QAT_DEV_NAME_MAX_LEN];
+ /**< Name of qat pci device */
+ struct rte_pci_device *pci_dev;
+ /**< PCI information. */
+ enum qat_device_gen qat_dev_gen;
+ /**< QAT device generation */
+ rte_spinlock_t arb_csr_lock;
+ /* protects accesses to the arbiter CSR */
+ __extension__
+ uint8_t attached : 1;
+ /**< Flag indicating the device is attached */
+
+ /* data relating to symmetric crypto service */
+ struct qat_pmd_private *sym_dev;
+ /**< link back to cryptodev private data */
+ unsigned int max_nb_sym_queue_pairs;
+ /**< Max number of queue pairs supported by device */
+
+ /* data relating to compression service */
+
+ /* data relating to asymmetric crypto service */
+
+};
+
+/** private data structure for a QAT device.
+ * This QAT device is a device offering only symmetric crypto service,
+ * there can be one of these on each qat_pci_device (VF),
+ * in future there may also be private data structures for other services.
*/
struct qat_pmd_private {
unsigned int max_nb_queue_pairs;
@@ -34,6 +77,8 @@ struct qat_pmd_private {
/**< PCI information. */
uint8_t dev_id;
/**< Device ID for this instance */
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
};
struct qat_gen_hw_data {
@@ -51,4 +96,11 @@ int qat_dev_close(struct rte_cryptodev *dev);
void qat_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
+struct qat_pci_device *
+qat_pci_device_allocate(struct rte_pci_device *pci_dev);
+int
+qat_pci_device_release(struct rte_pci_device *pci_dev);
+struct qat_pci_device *
+qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev);
+
#endif /* _QAT_DEVICE_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index fe19b18b6..6ab870cad 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -69,17 +69,28 @@ static const struct rte_pci_id pci_id_qat_map[] = {
{.device_id = 0},
};
+
+
static int
-crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
- struct rte_cryptodev_pmd_init_params *init_params)
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
{
+ struct rte_cryptodev_pmd_init_params init_params = {
+ .name = "",
+ .socket_id = qat_pci_dev->pci_dev->device.numa_node,
+ .private_data_size = sizeof(struct qat_pmd_private),
+ .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
+ };
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct rte_cryptodev *cryptodev;
struct qat_pmd_private *internals;
- PMD_INIT_FUNC_TRACE();
+ snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
+ qat_pci_dev->name, "sym");
+ PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+
+ cryptodev = rte_cryptodev_pmd_create(name,
+ &qat_pci_dev->pci_dev->device, &init_params);
- cryptodev = rte_cryptodev_pmd_create(name, &pci_dev->device,
- init_params);
if (cryptodev == NULL)
return -ENODEV;
@@ -95,7 +106,10 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
internals = cryptodev->data->dev_private;
- internals->max_nb_sessions = init_params->max_nb_sessions;
+ internals->qat_dev = qat_pci_dev;
+ qat_pci_dev->sym_dev = internals;
+
+ internals->max_nb_sessions = init_params.max_nb_sessions;
internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
internals->dev_id = cryptodev->data->dev_id;
switch (internals->pci_dev->id.device_id) {
@@ -111,68 +125,120 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
break;
default:
PMD_DRV_LOG(ERR,
- "Invalid dev_id, can't determine capabilities");
+ "Invalid dev_id, can't determine capabilities");
break;
}
- /*
- * For secondary processes, we don't initialise any further as primary
- * has already done this work. Only check we don't need a different
- * RX function
- */
- if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
- PMD_DRV_LOG(DEBUG, "Device already initialised by primary process");
return 0;
- }
+}
+
+static int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev *cryptodev;
+
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+ if (qat_pci_dev->sym_dev == NULL)
+ return 0;
+
+ /* free crypto device */
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->dev_id);
+ rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_dev = NULL;
+
+ return 0;
+}
+
+static int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+static int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+static int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
return 0;
}
-static int crypto_qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+static int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+static int
+qat_pci_dev_destroy(struct qat_pci_device *qat_pci_dev,
struct rte_pci_device *pci_dev)
{
- struct rte_cryptodev_pmd_init_params init_params = {
- .name = "",
- .socket_id = pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_pmd_private),
- .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
- };
- char name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ qat_sym_dev_destroy(qat_pci_dev);
+ qat_comp_dev_destroy(qat_pci_dev);
+ qat_asym_dev_destroy(qat_pci_dev);
+ return qat_pci_device_release(pci_dev);
+}
+
+static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ int ret = 0;
+ struct qat_pci_device *qat_pci_dev;
PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
pci_dev->addr.bus,
pci_dev->addr.devid,
pci_dev->addr.function);
- rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ qat_pci_dev = qat_pci_device_allocate(pci_dev);
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+
+ ret = qat_sym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_comp_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_asym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ return 0;
+
+error_out:
+ qat_pci_dev_destroy(qat_pci_dev, pci_dev);
+ return ret;
- return crypto_qat_create(name, pci_dev, &init_params);
}
-static int crypto_qat_pci_remove(struct rte_pci_device *pci_dev)
+static int qat_pci_remove(struct rte_pci_device *pci_dev)
{
- struct rte_cryptodev *cryptodev;
- char cryptodev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct qat_pci_device *qat_pci_dev;
if (pci_dev == NULL)
return -EINVAL;
- rte_pci_device_name(&pci_dev->addr, cryptodev_name,
- sizeof(cryptodev_name));
+ qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
+ if (qat_pci_dev == NULL)
+ return 0;
- cryptodev = rte_cryptodev_pmd_get_named_dev(cryptodev_name);
- if (cryptodev == NULL)
- return -ENODEV;
+ return qat_pci_dev_destroy(qat_pci_dev, pci_dev);
- /* free crypto device */
- return rte_cryptodev_pmd_destroy(cryptodev);
}
static struct rte_pci_driver rte_qat_pmd = {
.id_table = pci_id_qat_map,
.drv_flags = RTE_PCI_DRV_NEED_MAPPING,
- .probe = crypto_qat_pci_probe,
- .remove = crypto_qat_pci_remove
+ .probe = qat_pci_probe,
+ .remove = qat_pci_remove
};
static struct cryptodev_driver qat_crypto_drv;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 19/31] crypto/qat: separate the name used for PCI reg from crypto name
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (49 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 18/31] crypto/qat: add QAT PCI device struct Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 20/31] crypto/qat: move to using new device structure Fiona Trahe
` (12 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
The QAT PMD used to register with PCI using the name "crypto_qat".
Keep this name for the driver registered with cryptodev
and use a more generic name "qat" for the PCI registration.
This paves the way for the PCI device to host other services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 2 ++
drivers/crypto/qat/rte_qat_cryptodev.c | 25 ++++++++++++++++++-------
2 files changed, 20 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 77ffc8f72..63e5569ea 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -11,6 +11,8 @@
/**< Intel(R) QAT Symmetric Crypto PMD device name */
#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+/**< Intel(R) QAT device name for PCI registration */
+#define QAT_PCI_NAME qat
/*
* Maximum number of SGL entries
*/
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 6ab870cad..ad8a56374 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -234,16 +234,27 @@ static int qat_pci_remove(struct rte_pci_device *pci_dev)
}
+
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+static struct cryptodev_driver qat_crypto_drv;
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+ cryptodev_qat_driver_id);
+
static struct rte_pci_driver rte_qat_pmd = {
.id_table = pci_id_qat_map,
.drv_flags = RTE_PCI_DRV_NEED_MAPPING,
.probe = qat_pci_probe,
.remove = qat_pci_remove
};
-
-static struct cryptodev_driver qat_crypto_drv;
-
-RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_QAT_SYM_PMD, rte_qat_pmd);
-RTE_PMD_REGISTER_PCI_TABLE(CRYPTODEV_NAME_QAT_SYM_PMD, pci_id_qat_map);
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, rte_qat_pmd.driver,
- cryptodev_qat_driver_id);
+RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 20/31] crypto/qat: move to using new device structure
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (50 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 19/31] crypto/qat: separate the name used for PCI reg from crypto name Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 21/31] crypto/qat: use common stats structures Fiona Trahe
` (11 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Struct qat_pmd_private held the data needed by cryptodev, common code
now gets most data from struct qat_pci_device instead. qat_pmd_private
is trimmed to hold only sym crypto data and renamed qat_sym_private
to reflect its usage.
Specifically
- remove max_nb_queue_pairs from qat_pmd_private, get from qp_hw_data
- remove max_nb_sesssions from qat_pmd_private as not needed.
- remove qat_gen from qat_pmd_private, get from qat_pci_device instead.
- use qat_pci_device throughout common code instead of qat_pmd_private
- rename qat_pmd_private to qat_sym_dev_private - this now holds only
sym-specific data for the cryptodev API
- extend pci device name to <bdf>_qat for clarity, was just <bdf>
- update qp mem and cookiepool names to reflect the appropriate device,
service and qp.
- rename qat_dev_info_get() to qat_sym_dev_info_get() as mostly sym,
not enough common info to warrant a generic fn.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 17 +++++++++-------
drivers/crypto/qat/qat_device.h | 37 ++++++++++++++--------------------
drivers/crypto/qat/qat_qp.c | 26 +++++++++++++-----------
drivers/crypto/qat/qat_qp.h | 4 ++--
drivers/crypto/qat/qat_sym.c | 11 +++++-----
drivers/crypto/qat/qat_sym.h | 8 --------
drivers/crypto/qat/qat_sym_session.c | 8 ++++----
drivers/crypto/qat/rte_qat_cryptodev.c | 32 ++++++++++++++---------------
8 files changed, 65 insertions(+), 78 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 75af1e8bc..8ad3162e1 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -57,12 +57,12 @@ int qat_dev_close(struct rte_cryptodev *dev)
return 0;
}
-void qat_dev_info_get(struct rte_cryptodev *dev,
+void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info)
{
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev_gen]
+ qp_gen_config[internals->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
@@ -71,7 +71,7 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
info->feature_flags = dev->feature_flags;
info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = internals->max_nb_sessions;
+ info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
info->driver_id = cryptodev_qat_driver_id;
info->pci_dev = RTE_DEV_TO_PCI(dev->device);
}
@@ -83,6 +83,7 @@ qat_pci_get_dev(uint8_t dev_id)
{
return &qat_pci_devices[dev_id];
}
+
static struct qat_pci_device *
qat_pci_get_named_dev(const char *name)
{
@@ -133,7 +134,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
char name[QAT_DEV_NAME_MAX_LEN];
rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
-
+ snprintf(name+strlen(name), QAT_DEV_NAME_MAX_LEN-strlen(name), "_qat");
if (qat_pci_get_named_dev(name) != NULL) {
PMD_DRV_LOG(ERR, "QAT device with name %s already allocated!",
name);
@@ -148,6 +149,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
qat_dev = qat_pci_get_dev(qat_dev_id);
snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
+ qat_dev->qat_dev_id = qat_dev_id;
qat_dev->pci_dev = pci_dev;
switch (qat_dev->pci_dev->id.device_id) {
case 0x0443:
@@ -169,8 +171,8 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
qat_nb_pci_devices++;
- PMD_DRV_LOG(DEBUG, "QAT device %d allocated, total QATs %d",
- qat_dev_id, qat_nb_pci_devices);
+ PMD_DRV_LOG(DEBUG, "QAT device %d allocated, name %s, total QATs %d",
+ qat_dev->qat_dev_id, qat_dev->name, qat_nb_pci_devices);
return qat_dev;
}
@@ -185,6 +187,7 @@ qat_pci_device_release(struct rte_pci_device *pci_dev)
return -EINVAL;
rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ snprintf(name+strlen(name), QAT_DEV_NAME_MAX_LEN-strlen(name), "_qat");
qat_dev = qat_pci_get_named_dev(name);
if (qat_dev != NULL) {
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index d83ad632c..855bf6c1c 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -32,30 +32,31 @@ extern int qat_sym_qp_release(struct rte_cryptodev *dev,
* - config data
* - runtime data
*/
+struct qat_sym_dev_private;
struct qat_pci_device {
- /* data used by all services */
+ /* Data used by all services */
char name[QAT_DEV_NAME_MAX_LEN];
/**< Name of qat pci device */
+ uint8_t qat_dev_id;
+ /**< Device instance for this qat pci device */
struct rte_pci_device *pci_dev;
/**< PCI information. */
enum qat_device_gen qat_dev_gen;
/**< QAT device generation */
rte_spinlock_t arb_csr_lock;
- /* protects accesses to the arbiter CSR */
+ /**< lock to protect accesses to the arbiter CSR */
__extension__
uint8_t attached : 1;
/**< Flag indicating the device is attached */
- /* data relating to symmetric crypto service */
- struct qat_pmd_private *sym_dev;
+ /* Data relating to symmetric crypto service */
+ struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
- unsigned int max_nb_sym_queue_pairs;
- /**< Max number of queue pairs supported by device */
- /* data relating to compression service */
+ /* Data relating to compression service */
- /* data relating to asymmetric crypto service */
+ /* Data relating to asymmetric crypto service */
};
@@ -64,21 +65,13 @@ struct qat_pci_device {
* there can be one of these on each qat_pci_device (VF),
* in future there may also be private data structures for other services.
*/
-struct qat_pmd_private {
- unsigned int max_nb_queue_pairs;
- /**< Max number of queue pairs supported by device */
- unsigned int max_nb_sessions;
- /**< Max number of sessions supported by device */
- enum qat_device_gen qat_dev_gen;
- /**< QAT device generation */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
- /* QAT device capabilities */
- struct rte_pci_device *pci_dev;
- /**< PCI information. */
- uint8_t dev_id;
- /**< Device ID for this instance */
+struct qat_sym_dev_private {
struct qat_pci_device *qat_dev;
/**< The qat pci device hosting the service */
+ uint8_t sym_dev_id;
+ /**< Device instance for this rte_cryptodev */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device symmetric crypto capabilities */
};
struct qat_gen_hw_data {
@@ -93,7 +86,7 @@ int qat_dev_config(struct rte_cryptodev *dev,
int qat_dev_start(struct rte_cryptodev *dev);
void qat_dev_stop(struct rte_cryptodev *dev);
int qat_dev_close(struct rte_cryptodev *dev);
-void qat_dev_info_get(struct rte_cryptodev *dev,
+void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
struct qat_pci_device *
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 656645e4c..869140fc0 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -102,7 +102,7 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
-static int qat_queue_create(struct qat_pmd_private *qat_dev,
+static int qat_queue_create(struct qat_pci_device *qat_dev,
struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
@@ -153,7 +153,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-int qat_qp_setup(struct qat_pmd_private *qat_dev,
+int qat_qp_setup(struct qat_pci_device *qat_dev,
struct qat_qp **qp_addr,
uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
@@ -164,8 +164,8 @@ int qat_qp_setup(struct qat_pmd_private *qat_dev,
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
- PMD_DRV_LOG(DEBUG, "Setup qp %u on device %d gen %d",
- queue_pair_id, qat_dev->dev_id, qat_dev->qat_dev_gen);
+ PMD_DRV_LOG(DEBUG, "Setup qp %u on qat pci device %d gen %d",
+ queue_pair_id, qat_dev->qat_dev_id, qat_dev->qat_dev_gen);
if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
(qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
@@ -218,10 +218,12 @@ int qat_qp_setup(struct qat_pmd_private *qat_dev,
adf_configure_queues(qp);
adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
- snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
- pci_dev->driver->driver.name, qat_qp_conf->service_str,
- qat_dev->dev_id, queue_pair_id);
+ snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE,
+ "%s%d_cookies_%s_qp%hu",
+ pci_dev->driver->driver.name, qat_dev->qat_dev_id,
+ qat_qp_conf->service_str, queue_pair_id);
+ PMD_DRV_LOG(DEBUG, "cookiepool: %s", op_cookie_pool_name);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
@@ -270,7 +272,7 @@ int qat_qp_release(struct qat_qp **qp_addr)
}
PMD_DRV_LOG(DEBUG, "Free qp on qat_pci device %d",
- qp->qat_dev->dev_id);
+ qp->qat_dev->qat_dev_id);
/* Don't free memory if there are still responses to be processed */
if (qp->inflights16 == 0) {
@@ -322,7 +324,7 @@ static void qat_queue_delete(struct qat_queue *queue)
}
static int
-qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
+qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
@@ -347,9 +349,9 @@ qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
* Allocate a memzone for the queue - create a unique name.
*/
snprintf(queue->memz_name, sizeof(queue->memz_name),
- "%s_%s_%s_%d_%d_%d",
- pci_dev->driver->driver.name, qp_conf->service_str,
- "qp_mem", qat_dev->dev_id,
+ "%s_%d_%s_%s_%d_%d",
+ pci_dev->driver->driver.name, qat_dev->qat_dev_id,
+ qp_conf->service_str, "qp_mem",
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
qp_conf->socket_id);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index f808e16a5..d482d5732 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -85,7 +85,7 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
process_response_t process_response;
- struct qat_pmd_private *qat_dev;
+ struct qat_pci_device *qat_dev;
/**< qat device this qp is on */
} __rte_cache_aligned;
@@ -101,7 +101,7 @@ int
qat_qp_release(struct qat_qp **qp_addr);
int
-qat_qp_setup(struct qat_pmd_private *qat_dev,
+qat_qp_setup(struct qat_pci_device *qat_dev,
struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index b74dfa634..e77fbe4c4 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -204,7 +204,7 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
}
-int
+static int
qat_sym_process_response(void **op, uint8_t *resp,
__rte_unused void *op_cookie,
__rte_unused enum qat_device_gen qat_dev_gen)
@@ -293,8 +293,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-
-int
+static int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen)
{
@@ -773,9 +772,9 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
struct qat_qp **qp_addr =
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
- struct qat_pmd_private *qat_private = dev->data->dev_private;
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev_gen]
+ qp_gen_config[qat_private->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
@@ -798,7 +797,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.socket_id = socket_id;
qat_qp_conf.service_str = "sym";
- ret = qat_qp_setup(qat_private, qp_addr, qp_id, &qat_qp_conf);
+ ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
if (ret != 0)
return ret;
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 37bec3ce3..78b40e378 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -29,14 +29,6 @@ struct qat_sym_op_cookie {
phys_addr_t qat_sgl_dst_phys_addr;
};
-int
-qat_sym_build_request(void *in_op, uint8_t *out_msg,
- void *op_cookie, enum qat_device_gen qat_dev_gen);
-
-int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie, enum qat_device_gen qat_dev_gen);
-
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index a08e93037..68d7773a2 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -64,7 +64,7 @@ bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
static int
qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
- struct qat_pmd_private *internals)
+ struct qat_sym_dev_private *internals)
{
int i = 0;
const struct rte_cryptodev_capabilities *capability;
@@ -85,7 +85,7 @@ qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
static int
qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
- struct qat_pmd_private *internals)
+ struct qat_sym_dev_private *internals)
{
int i = 0;
const struct rte_cryptodev_capabilities *capability;
@@ -201,7 +201,7 @@ qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_sym_session *session)
{
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
struct rte_crypto_cipher_xform *cipher_xform = NULL;
int ret;
@@ -495,7 +495,7 @@ qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct qat_sym_session *session)
{
struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
uint8_t *key_data = auth_xform->key.data;
uint8_t key_length = auth_xform->key.length;
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index ad8a56374..6d807177e 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -33,7 +33,7 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
.dev_start = qat_dev_start,
.dev_stop = qat_dev_stop,
.dev_close = qat_dev_close,
- .dev_infos_get = qat_dev_info_get,
+ .dev_infos_get = qat_sym_dev_info_get,
.stats_get = qat_sym_stats_get,
.stats_reset = qat_sym_stats_reset,
@@ -77,12 +77,12 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
struct rte_cryptodev_pmd_init_params init_params = {
.name = "",
.socket_id = qat_pci_dev->pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_pmd_private),
+ .private_data_size = sizeof(struct qat_sym_dev_private),
.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
};
char name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct rte_cryptodev *cryptodev;
- struct qat_pmd_private *internals;
+ struct qat_sym_dev_private *internals;
snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
qat_pci_dev->name, "sym");
@@ -109,27 +109,25 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
internals->qat_dev = qat_pci_dev;
qat_pci_dev->sym_dev = internals;
- internals->max_nb_sessions = init_params.max_nb_sessions;
- internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
- internals->dev_id = cryptodev->data->dev_id;
- switch (internals->pci_dev->id.device_id) {
- case 0x0443:
- internals->qat_dev_gen = QAT_GEN1;
+ internals->sym_dev_id = cryptodev->data->dev_id;
+ switch (qat_pci_dev->qat_dev_gen) {
+ case QAT_GEN1:
internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
break;
- case 0x37c9:
- case 0x19e3:
- case 0x6f55:
- internals->qat_dev_gen = QAT_GEN2;
+ case QAT_GEN2:
internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
break;
default:
- PMD_DRV_LOG(ERR,
- "Invalid dev_id, can't determine capabilities");
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ PMD_DRV_LOG(DEBUG,
+ "QAT gen %d capabilities unknown, default to GEN2",
+ qat_pci_dev->qat_dev_gen);
break;
}
- return 0;
+ PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
+ name, internals->sym_dev_id);
+ return 0;
}
static int
@@ -143,7 +141,7 @@ qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
return 0;
/* free crypto device */
- cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->dev_id);
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
rte_cryptodev_pmd_destroy(cryptodev);
qat_pci_dev->sym_dev = NULL;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 21/31] crypto/qat: use common stats structures
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (51 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 20/31] crypto/qat: move to using new device structure Fiona Trahe
@ 2018-05-11 11:13 ` Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 22/31] crypto/qat: rename functions which depend on cryptodev Fiona Trahe
` (10 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:13 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Split qat_sym_stats_get/reset into 2 functions, a wrapper function calling
a new qat_stats_get/reset function which can be called per service.
Remove cryptodev stats struct from qat_qp, replace with qat_common_stats.
Add links for qat_qp into qat_pci_device using an array per service to
avoid need for a lock and so qp_id for the service can be used as index.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 13 +++++++
drivers/crypto/qat/qat_device.c | 1 +
drivers/crypto/qat/qat_device.h | 3 ++
drivers/crypto/qat/qat_qp.h | 2 +-
drivers/crypto/qat/qat_sym.c | 86 ++++++++++++++++++++++++++++++++++-------
5 files changed, 91 insertions(+), 14 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 63e5569ea..fcf5c4c09 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -49,6 +49,19 @@ struct qat_sgl {
struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
+/** Common, i.e. not service-specific, statistics */
+struct qat_common_stats {
+ uint64_t enqueued_count;
+ /**< Count of all operations enqueued */
+ uint64_t dequeued_count;
+ /**< Count of all operations dequeued */
+
+ uint64_t enqueue_err_count;
+ /**< Total error count on operations enqueued */
+ uint64_t dequeue_err_count;
+ /**< Total error count on operations dequeued */
+};
+
int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
struct qat_sgl *list, uint32_t data_len);
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 8ad3162e1..4dfbc84c8 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -148,6 +148,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
}
qat_dev = qat_pci_get_dev(qat_dev_id);
+ memset(qat_dev, 0, sizeof(*qat_dev));
snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
qat_dev->qat_dev_id = qat_dev_id;
qat_dev->pci_dev = pci_dev;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 855bf6c1c..d1615e2a7 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -50,6 +50,9 @@ struct qat_pci_device {
uint8_t attached : 1;
/**< Flag indicating the device is attached */
+ struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_PER_BUNDLE];
+ /**< links to qps set up for each service, index same as on API */
+
/* Data relating to symmetric crypto service */
struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index d482d5732..49d9f29d3 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -78,7 +78,7 @@ struct qat_qp {
uint16_t inflights16;
struct qat_queue tx_q;
struct qat_queue rx_q;
- struct rte_cryptodev_stats stats;
+ struct qat_common_stats stats;
struct rte_mempool *op_cookie_pool;
void **op_cookies;
uint32_t nb_descriptors;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index e77fbe4c4..8007e25d6 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -717,20 +717,24 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
}
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats)
+static void qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service)
{
int i;
- struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
+ struct qat_qp **qp;
- PMD_INIT_FUNC_TRACE();
- if (stats == NULL) {
- PMD_DRV_LOG(ERR, "invalid stats ptr NULL");
+ if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
+ stats, dev, service);
return;
}
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Uninitialised queue pair");
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
continue;
}
@@ -741,22 +745,74 @@ void qat_sym_stats_get(struct rte_cryptodev *dev,
}
}
-void qat_sym_stats_reset(struct rte_cryptodev *dev)
+void qat_sym_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ struct qat_common_stats qat_stats = {0};
+ struct qat_sym_dev_private *qat_priv;
+
+ if (stats == NULL || dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
+ stats->enqueued_count = qat_stats.enqueued_count;
+ stats->dequeued_count = qat_stats.dequeued_count;
+ stats->enqueue_err_count = qat_stats.enqueue_err_count;
+ stats->dequeue_err_count = qat_stats.dequeue_err_count;
+}
+
+static void qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service)
{
int i;
- struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
+ struct qat_qp **qp;
- PMD_INIT_FUNC_TRACE();
- for (i = 0; i < dev->data->nb_queue_pairs; i++)
+ if (dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
+ dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
- PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
+ }
+
+ PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+}
+
+void qat_sym_stats_reset(struct rte_cryptodev *dev)
+{
+ struct qat_sym_dev_private *qat_priv;
+
+ if (dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
+
}
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+
PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
queue_pair_id, dev->data->dev_id);
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
+ = NULL;
+
return qat_qp_release((struct qat_qp **)
&(dev->data->queue_pairs[queue_pair_id]));
}
@@ -801,6 +857,10 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
if (ret != 0)
return ret;
+ /* store a link to the qp in the qat_pci_device */
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
+ = *qp_addr;
+
qp = (struct qat_qp *)*qp_addr;
for (i = 0; i < qp->nb_descriptors; i++) {
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 22/31] crypto/qat: rename functions which depend on cryptodev
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (52 preceding siblings ...)
2018-05-11 11:13 ` [PATCH v2 21/31] crypto/qat: use common stats structures Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 23/31] crypto/qat: move code into appropriate files Fiona Trahe
` (9 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Rename all device functions which depend on cryptodev structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 8 ++++----
drivers/crypto/qat/qat_device.h | 8 ++++----
drivers/crypto/qat/rte_qat_cryptodev.c | 8 ++++----
3 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 4dfbc84c8..8b2ac5a5f 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -24,25 +24,25 @@ struct qat_gen_hw_data qp_gen_config[] = {
static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
-int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
+int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
{
PMD_INIT_FUNC_TRACE();
return 0;
}
-int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
+int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
{
PMD_INIT_FUNC_TRACE();
return 0;
}
-void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
+void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
{
PMD_INIT_FUNC_TRACE();
}
-int qat_dev_close(struct rte_cryptodev *dev)
+int qat_sym_dev_close(struct rte_cryptodev *dev)
{
int i, ret;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index d1615e2a7..5424a9a94 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -84,11 +84,11 @@ struct qat_gen_hw_data {
extern struct qat_gen_hw_data qp_gen_config[];
-int qat_dev_config(struct rte_cryptodev *dev,
+int qat_sym_dev_config(struct rte_cryptodev *dev,
struct rte_cryptodev_config *config);
-int qat_dev_start(struct rte_cryptodev *dev);
-void qat_dev_stop(struct rte_cryptodev *dev);
-int qat_dev_close(struct rte_cryptodev *dev);
+int qat_sym_dev_start(struct rte_cryptodev *dev);
+void qat_sym_dev_stop(struct rte_cryptodev *dev);
+int qat_sym_dev_close(struct rte_cryptodev *dev);
void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 6d807177e..91bb1e590 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -29,10 +29,10 @@ static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
static struct rte_cryptodev_ops crypto_qat_ops = {
/* Device related operations */
- .dev_configure = qat_dev_config,
- .dev_start = qat_dev_start,
- .dev_stop = qat_dev_stop,
- .dev_close = qat_dev_close,
+ .dev_configure = qat_sym_dev_config,
+ .dev_start = qat_sym_dev_start,
+ .dev_stop = qat_sym_dev_stop,
+ .dev_close = qat_sym_dev_close,
.dev_infos_get = qat_sym_dev_info_get,
.stats_get = qat_sym_stats_get,
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 23/31] crypto/qat: move code into appropriate files
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (53 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 22/31] crypto/qat: rename functions which depend on cryptodev Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 24/31] crypto/qat: add lock around csr access and change logic Fiona Trahe
` (8 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Move all code into appropriate files, no actual code changes. Specifically:
- Rename rte_qat_cryptodev.c to qat_sym_pmd.c
- Create qat_sym_pmd.h and populate with fn prototypes for qat_sym_pmd.c
- Create qat_comp_pmd.c/.h and populate with placeholder functions
- Create qat_asym_pmd.c/.h and populate with placeholder functions
- Rename qat_crypto_capabilities.h to qat_sym_capabilities.h
- Move CRYPTODEV_NAME_QAT_SYM_PMD from qat_common.h to qat_sym_pmd.h
- Move qat_sym_dev_private from qat_device.h to qat_sym_pmd.h
- Move prototype for qat_sym_dev_info_get frm qat_device.h 2 qat_sym_pmd.h
- Move all qat_device.c sym dev_ops fns to qat_sym_pmd.c file
- Move all qat_sym.c dev_ops fns to qat_sym_pmd.c file
- Remove unused header file #includes from all files.
- Move pci_id_qat_map, probe/release/register from
rte_qat_cryptodev.c to qat_device.c
- Moved stray comment for bpi_cipher_ctx_init() from qat_sym.c
to qat_sym_session.c
- Changed all files to use SPDX license header
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 4 +-
drivers/crypto/qat/meson.build | 11 +-
drivers/crypto/qat/qat_asym_pmd.c | 17 ++
drivers/crypto/qat/qat_asym_pmd.h | 15 +
drivers/crypto/qat/qat_common.c | 54 ++++
drivers/crypto/qat/qat_common.h | 15 +-
drivers/crypto/qat/qat_comp_pmd.c | 18 ++
drivers/crypto/qat/qat_comp_pmd.h | 29 ++
drivers/crypto/qat/qat_device.c | 141 +++++----
drivers/crypto/qat/qat_device.h | 31 +-
drivers/crypto/qat/qat_qp.c | 2 +-
drivers/crypto/qat/qat_qp.h | 9 +-
drivers/crypto/qat/qat_sym.c | 246 ++--------------
drivers/crypto/qat/qat_sym.h | 28 +-
...rypto_capabilities.h => qat_sym_capabilities.h} | 6 +-
drivers/crypto/qat/qat_sym_pmd.c | 326 +++++++++++++++++++++
drivers/crypto/qat/qat_sym_pmd.h | 39 +++
drivers/crypto/qat/qat_sym_session.c | 17 +-
drivers/crypto/qat/qat_sym_session.h | 2 +-
drivers/crypto/qat/rte_qat_cryptodev.c | 258 ----------------
20 files changed, 665 insertions(+), 603 deletions(-)
create mode 100644 drivers/crypto/qat/qat_asym_pmd.c
create mode 100644 drivers/crypto/qat/qat_asym_pmd.h
create mode 100644 drivers/crypto/qat/qat_comp_pmd.c
create mode 100644 drivers/crypto/qat/qat_comp_pmd.h
rename drivers/crypto/qat/{qat_crypto_capabilities.h => qat_sym_capabilities.h} (99%)
create mode 100644 drivers/crypto/qat/qat_sym_pmd.c
create mode 100644 drivers/crypto/qat/qat_sym_pmd.h
delete mode 100644 drivers/crypto/qat/rte_qat_cryptodev.c
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 902c47ff4..d467683fd 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -26,7 +26,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_common.c
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_asym_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_comp_pmd.c
# export include files
SYMLINK-y-include +=
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 12910c377..e22e08fba 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -5,11 +5,12 @@ dep = dependency('libcrypto', required: false)
if not dep.found()
build = false
endif
-sources = files('qat_sym.c', 'qat_qp.c',
- 'qat_sym_session.c',
- 'qat_common.c',
- 'rte_qat_cryptodev.c',
- 'qat_device.c')
+sources = files('qat_common.c',
+ 'qat_qp.c',
+ 'qat_device.c',
+ 'qat_sym_pmd.c', 'qat_sym.c', 'qat_sym_session.c',
+ 'qat_asym_pmd.c',
+ 'qat_comp_pmd.c')
includes += include_directories('qat_adf')
deps += ['bus_pci']
ext_deps += dep
diff --git a/drivers/crypto/qat/qat_asym_pmd.c b/drivers/crypto/qat/qat_asym_pmd.c
new file mode 100644
index 000000000..8d36300ac
--- /dev/null
+++ b/drivers/crypto/qat/qat_asym_pmd.c
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_asym_pmd.h"
+
+int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_asym_pmd.h b/drivers/crypto/qat/qat_asym_pmd.h
new file mode 100644
index 000000000..0465e0300
--- /dev/null
+++ b/drivers/crypto/qat/qat_asym_pmd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _QAT_ASYM_PMD_H_
+#define _QAT_ASYM_PMD_H_
+
+#include "qat_device.h"
+
+int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_ASYM_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index a8865904f..f1759ea76 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -3,6 +3,7 @@
*/
#include "qat_common.h"
+#include "qat_device.h"
#include "qat_logs.h"
int
@@ -51,3 +52,56 @@ qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
return 0;
}
+
+void qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service)
+{
+ int i;
+ struct qat_qp **qp;
+
+ if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
+ stats, dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
+
+ stats->enqueued_count += qp[i]->stats.enqueued_count;
+ stats->dequeued_count += qp[i]->stats.dequeued_count;
+ stats->enqueue_err_count += qp[i]->stats.enqueue_err_count;
+ stats->dequeue_err_count += qp[i]->stats.dequeue_err_count;
+ }
+}
+
+void qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service)
+{
+ int i;
+ struct qat_qp **qp;
+
+ if (dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
+ dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
+ memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
+ }
+
+ PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+}
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index fcf5c4c09..8ecebe954 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -8,9 +8,6 @@
#include <rte_mbuf.h>
-/**< Intel(R) QAT Symmetric Crypto PMD device name */
-#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
-
/**< Intel(R) QAT device name for PCI registration */
#define QAT_PCI_NAME qat
/*
@@ -21,10 +18,9 @@
/* Intel(R) QuickAssist Technology device generation is enumerated
* from one according to the generation of the device
*/
-
enum qat_device_gen {
QAT_GEN1 = 1,
- QAT_GEN2,
+ QAT_GEN2
};
enum qat_service_type {
@@ -62,8 +58,17 @@ struct qat_common_stats {
/**< Total error count on operations dequeued */
};
+struct qat_pci_device;
+
int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
struct qat_sgl *list, uint32_t data_len);
+void
+qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service);
+void
+qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service);
#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_comp_pmd.c b/drivers/crypto/qat/qat_comp_pmd.c
new file mode 100644
index 000000000..547b3db49
--- /dev/null
+++ b/drivers/crypto/qat/qat_comp_pmd.c
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_comp_pmd.h"
+
+
+int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_comp_pmd.h b/drivers/crypto/qat/qat_comp_pmd.h
new file mode 100644
index 000000000..cc31246c3
--- /dev/null
+++ b/drivers/crypto/qat/qat_comp_pmd.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _QAT_COMP_PMD_H_
+#define _QAT_COMP_PMD_H_
+
+#include "qat_device.h"
+
+
+/**< Intel(R) QAT Compression PMD device name */
+#define COMPRESSDEV_NAME_QAT_PMD comp_qat
+
+
+/** private data structure for a QAT compression device.
+ * This QAT device is a device offering only a compression service,
+ * there can be one of these on each qat_pci_device (VF).
+ */
+struct qat_comp_dev_private {
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
+};
+
+int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_COMP_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 8b2ac5a5f..c9d4b32ed 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -4,7 +4,9 @@
#include "qat_device.h"
#include "adf_transport_access_macros.h"
-#include "qat_qp.h"
+#include "qat_sym_pmd.h"
+#include "qat_asym_pmd.h"
+#include "qat_comp_pmd.h"
/* Hardware device information per generation */
__extension__
@@ -24,58 +26,25 @@ struct qat_gen_hw_data qp_gen_config[] = {
static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
-int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
- __rte_unused struct rte_cryptodev_config *config)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
-}
-
-int qat_sym_dev_close(struct rte_cryptodev *dev)
-{
- int i, ret;
-
- PMD_INIT_FUNC_TRACE();
-
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_sym_qp_release(dev, i);
- if (ret < 0)
- return ret;
- }
-
- return 0;
-}
+/*
+ * The set of PCI devices this driver supports
+ */
-void qat_sym_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info)
-{
- struct qat_sym_dev_private *internals = dev->data->dev_private;
- const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev->qat_dev_gen]
- .qp_hw_data[QAT_SERVICE_SYMMETRIC];
-
- PMD_INIT_FUNC_TRACE();
- if (info != NULL) {
- info->max_nb_queue_pairs =
- qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
- info->feature_flags = dev->feature_flags;
- info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
- info->driver_id = cryptodev_qat_driver_id;
- info->pci_dev = RTE_DEV_TO_PCI(dev->device);
- }
-}
+static const struct rte_pci_id pci_id_qat_map[] = {
+ {
+ RTE_PCI_DEVICE(0x8086, 0x0443),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x37c9),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x19e3),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x6f55),
+ },
+ {.device_id = 0},
+};
static struct qat_pci_device *
@@ -203,3 +172,71 @@ qat_pci_device_release(struct rte_pci_device *pci_dev)
name, qat_nb_pci_devices);
return 0;
}
+
+static int
+qat_pci_dev_destroy(struct qat_pci_device *qat_pci_dev,
+ struct rte_pci_device *pci_dev)
+{
+ qat_sym_dev_destroy(qat_pci_dev);
+ qat_comp_dev_destroy(qat_pci_dev);
+ qat_asym_dev_destroy(qat_pci_dev);
+ return qat_pci_device_release(pci_dev);
+}
+
+static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ int ret = 0;
+ struct qat_pci_device *qat_pci_dev;
+
+ PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
+ pci_dev->addr.bus,
+ pci_dev->addr.devid,
+ pci_dev->addr.function);
+
+ qat_pci_dev = qat_pci_device_allocate(pci_dev);
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+
+ ret = qat_sym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_comp_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_asym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ return 0;
+
+error_out:
+ qat_pci_dev_destroy(qat_pci_dev, pci_dev);
+ return ret;
+
+}
+
+static int qat_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_pci_dev;
+
+ if (pci_dev == NULL)
+ return -EINVAL;
+
+ qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
+ if (qat_pci_dev == NULL)
+ return 0;
+
+ return qat_pci_dev_destroy(qat_pci_dev, pci_dev);
+}
+
+static struct rte_pci_driver rte_qat_pmd = {
+ .id_table = pci_id_qat_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = qat_pci_probe,
+ .remove = qat_pci_remove
+};
+RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 5424a9a94..fd1819354 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -4,8 +4,8 @@
#ifndef _QAT_DEVICE_H_
#define _QAT_DEVICE_H_
-#include <rte_cryptodev_pmd.h>
#include <rte_bus_pci.h>
+
#include "qat_common.h"
#include "qat_logs.h"
#include "adf_transport_access_macros.h"
@@ -18,12 +18,6 @@
#define QAT_MAX_PCI_DEVICES 48
#define QAT_DEV_NAME_MAX_LEN 64
-
-extern uint8_t cryptodev_qat_driver_id;
-
-extern int qat_sym_qp_release(struct rte_cryptodev *dev,
- uint16_t queue_pair_id);
-
/*
* This struct holds all the data about a QAT pci device
* including data about all services it supports.
@@ -63,20 +57,6 @@ struct qat_pci_device {
};
-/** private data structure for a QAT device.
- * This QAT device is a device offering only symmetric crypto service,
- * there can be one of these on each qat_pci_device (VF),
- * in future there may also be private data structures for other services.
- */
-struct qat_sym_dev_private {
- struct qat_pci_device *qat_dev;
- /**< The qat pci device hosting the service */
- uint8_t sym_dev_id;
- /**< Device instance for this rte_cryptodev */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
- /* QAT device symmetric crypto capabilities */
-};
-
struct qat_gen_hw_data {
enum qat_device_gen dev_gen;
const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
@@ -84,14 +64,6 @@ struct qat_gen_hw_data {
extern struct qat_gen_hw_data qp_gen_config[];
-int qat_sym_dev_config(struct rte_cryptodev *dev,
- struct rte_cryptodev_config *config);
-int qat_sym_dev_start(struct rte_cryptodev *dev);
-void qat_sym_dev_stop(struct rte_cryptodev *dev);
-int qat_sym_dev_close(struct rte_cryptodev *dev);
-void qat_sym_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info);
-
struct qat_pci_device *
qat_pci_device_allocate(struct rte_pci_device *pci_dev);
int
@@ -99,4 +71,5 @@ qat_pci_device_release(struct rte_pci_device *pci_dev);
struct qat_pci_device *
qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev);
+
#endif /* _QAT_DEVICE_H_ */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 869140fc0..7b2dc3f90 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -12,8 +12,8 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
-#include "qat_qp.h"
#include "qat_device.h"
+#include "qat_qp.h"
#include "adf_transport_access_macros.h"
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 49d9f29d3..73888b805 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,9 +5,10 @@
#define _QAT_QP_H_
#include "qat_common.h"
-#include <rte_cryptodev_pmd.h>
#include "adf_transport_access_macros.h"
+struct qat_pci_device;
+
#define QAT_CSR_HEAD_WRITE_THRESH 32U
/* number of requests to accumulate before writing head CSR */
#define QAT_CSR_TAIL_WRITE_THRESH 32U
@@ -76,9 +77,9 @@ struct qat_queue {
struct qat_qp {
void *mmap_bar_addr;
uint16_t inflights16;
- struct qat_queue tx_q;
- struct qat_queue rx_q;
- struct qat_common_stats stats;
+ struct qat_queue tx_q;
+ struct qat_queue rx_q;
+ struct qat_common_stats stats;
struct rte_mempool *op_cookie_pool;
void **op_cookies;
uint32_t nb_descriptors;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 8007e25d6..15244d113 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -2,6 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
+#include <openssl/evp.h>
+
#include <rte_mempool.h>
#include <rte_mbuf.h>
#include <rte_hexdump.h>
@@ -9,14 +11,10 @@
#include <rte_bus_pci.h>
#include <rte_byteorder.h>
-#include <openssl/evp.h>
-
#include "qat_logs.h"
#include "qat_sym_session.h"
#include "qat_sym.h"
-#include "qat_qp.h"
-#include "adf_transport_access_macros.h"
-#include "qat_device.h"
+#include "qat_sym_pmd.h"
#define BYTE_LENGTH 8
/* bpi is only used for partial blocks of DES and AES
@@ -82,9 +80,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
return -EINVAL;
}
-/** Creates a context in either AES or DES in ECB mode
- * Depends on openssl libcrypto
- */
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
@@ -197,57 +192,6 @@ qat_bpicipher_postprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
-{
- return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
-}
-
-static int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen)
-{
-
- struct icp_qat_fw_comn_resp *resp_msg =
- (struct icp_qat_fw_comn_resp *)resp;
- struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
-
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
-
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess = (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
-
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
- *op = (void *)rx_op;
-
- return 0;
-}
-
-
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
-{
- return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
@@ -293,7 +237,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-static int
+int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen)
{
@@ -716,168 +660,38 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
return 0;
}
-
-static void qat_stats_get(struct qat_pci_device *dev,
- struct qat_common_stats *stats,
- enum qat_service_type service)
-{
- int i;
- struct qat_qp **qp;
-
- if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
- PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
- stats, dev, service);
- return;
- }
-
- qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
- if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
- service, i);
- continue;
- }
-
- stats->enqueued_count += qp[i]->stats.enqueued_count;
- stats->dequeued_count += qp[i]->stats.dequeued_count;
- stats->enqueue_err_count += qp[i]->stats.enqueue_err_count;
- stats->dequeue_err_count += qp[i]->stats.dequeue_err_count;
- }
-}
-
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats)
-{
- struct qat_common_stats qat_stats = {0};
- struct qat_sym_dev_private *qat_priv;
-
- if (stats == NULL || dev == NULL) {
- PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
- return;
- }
- qat_priv = dev->data->dev_private;
-
- qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
- stats->enqueued_count = qat_stats.enqueued_count;
- stats->dequeued_count = qat_stats.dequeued_count;
- stats->enqueue_err_count = qat_stats.enqueue_err_count;
- stats->dequeue_err_count = qat_stats.dequeue_err_count;
-}
-
-static void qat_stats_reset(struct qat_pci_device *dev,
- enum qat_service_type service)
-{
- int i;
- struct qat_qp **qp;
-
- if (dev == NULL || service >= QAT_SERVICE_INVALID) {
- PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
- dev, service);
- return;
- }
-
- qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
- if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
- service, i);
- continue;
- }
- memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
- }
-
- PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
-}
-
-void qat_sym_stats_reset(struct rte_cryptodev *dev)
-{
- struct qat_sym_dev_private *qat_priv;
-
- if (dev == NULL) {
- PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
- return;
- }
- qat_priv = dev->data->dev_private;
-
- qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
-
-}
-
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen)
{
- struct qat_sym_dev_private *qat_private = dev->data->dev_private;
- PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
- queue_pair_id, dev->data->dev_id);
-
- qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
- = NULL;
-
- return qat_qp_release((struct qat_qp **)
- &(dev->data->queue_pairs[queue_pair_id]));
-}
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
-{
- struct qat_qp *qp;
- int ret = 0;
- uint32_t i;
- struct qat_qp_config qat_qp_conf;
-
- struct qat_qp **qp_addr =
- (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
- struct qat_sym_dev_private *qat_private = dev->data->dev_private;
- const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev->qat_dev_gen]
- .qp_hw_data[QAT_SERVICE_SYMMETRIC];
- const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
-
- /* If qp is already in use free ring memory and qp metadata. */
- if (*qp_addr != NULL) {
- ret = qat_sym_qp_release(dev, qp_id);
- if (ret < 0)
- return ret;
- }
- if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
- return -EINVAL;
- }
-
- qat_qp_conf.hw = qp_hw_data;
- qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
- qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
- qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
- qat_qp_conf.socket_id = socket_id;
- qat_qp_conf.service_str = "sym";
-
- ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
- if (ret != 0)
- return ret;
-
- /* store a link to the qp in the qat_pci_device */
- qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
- = *qp_addr;
-
- qp = (struct qat_qp *)*qp_addr;
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
- for (i = 0; i < qp->nb_descriptors; i++) {
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
- struct qat_sym_op_cookie *sql_cookie =
- qp->op_cookies[i];
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_src);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_dst);
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
}
+ *op = (void *)rx_op;
- return ret;
+ return 0;
}
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 78b40e378..d887dc126 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -6,11 +6,8 @@
#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
-#include <rte_memzone.h>
#include "qat_common.h"
-#include "qat_device.h"
-#include "qat_crypto_capabilities.h"
/*
* This macro rounds up a number to a be a multiple of
@@ -29,23 +26,12 @@ struct qat_sym_op_cookie {
phys_addr_t qat_sgl_dst_phys_addr;
};
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats);
-void qat_sym_stats_reset(struct rte_cryptodev *dev);
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
- const struct rte_cryptodev_qp_conf *rx_conf, int socket_id,
- struct rte_mempool *session_pool);
-int qat_sym_qp_release(struct rte_cryptodev *dev,
- uint16_t queue_pair_id);
-
-
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops);
-
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops);
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen);
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen);
#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_crypto_capabilities.h b/drivers/crypto/qat/qat_sym_capabilities.h
similarity index 99%
rename from drivers/crypto/qat/qat_crypto_capabilities.h
rename to drivers/crypto/qat/qat_sym_capabilities.h
index 001c32c5d..d10a95ecb 100644
--- a/drivers/crypto/qat/qat_crypto_capabilities.h
+++ b/drivers/crypto/qat/qat_sym_capabilities.h
@@ -2,8 +2,8 @@
* Copyright(c) 2017-2018 Intel Corporation
*/
-#ifndef _QAT_CRYPTO_CAPABILITIES_H_
-#define _QAT_CRYPTO_CAPABILITIES_H_
+#ifndef _QAT_SYM_CAPABILITIES_H_
+#define _QAT_SYM_CAPABILITIES_H_
#define QAT_BASE_GEN1_SYM_CAPABILITIES \
{ /* SHA1 HMAC */ \
@@ -554,4 +554,4 @@
}, } \
}
-#endif /* _QAT_CRYPTO_CAPABILITIES_H_ */
+#endif /* _QAT_SYM_CAPABILITIES_H_ */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
new file mode 100644
index 000000000..aa71b4641
--- /dev/null
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -0,0 +1,326 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2015-2018 Intel Corporation
+ */
+
+#include <rte_bus_pci.h>
+#include <rte_common.h>
+#include <rte_dev.h>
+#include <rte_malloc.h>
+#include <rte_pci.h>
+#include <rte_cryptodev_pmd.h>
+
+#include "qat_logs.h"
+#include "qat_sym.h"
+#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
+
+uint8_t cryptodev_qat_driver_id;
+
+static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
+ QAT_BASE_GEN1_SYM_CAPABILITIES,
+ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
+};
+
+static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
+ QAT_BASE_GEN1_SYM_CAPABILITIES,
+ QAT_EXTRA_GEN2_SYM_CAPABILITIES,
+ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
+};
+
+static int qat_sym_qp_release(struct rte_cryptodev *dev,
+ uint16_t queue_pair_id);
+
+static int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused struct rte_cryptodev_config *config)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+static int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+static void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+}
+
+static int qat_sym_dev_close(struct rte_cryptodev *dev)
+{
+ int i, ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ ret = qat_sym_qp_release(dev, i);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static void qat_sym_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info)
+{
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[internals->qat_dev->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+
+ PMD_INIT_FUNC_TRACE();
+ if (info != NULL) {
+ info->max_nb_queue_pairs =
+ qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
+ info->feature_flags = dev->feature_flags;
+ info->capabilities = internals->qat_dev_capabilities;
+ info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
+ info->driver_id = cryptodev_qat_driver_id;
+ info->pci_dev = RTE_DEV_TO_PCI(dev->device);
+ }
+}
+
+static void qat_sym_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ struct qat_common_stats qat_stats = {0};
+ struct qat_sym_dev_private *qat_priv;
+
+ if (stats == NULL || dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
+ stats->enqueued_count = qat_stats.enqueued_count;
+ stats->dequeued_count = qat_stats.dequeued_count;
+ stats->enqueue_err_count = qat_stats.enqueue_err_count;
+ stats->dequeue_err_count = qat_stats.dequeue_err_count;
+}
+
+static void qat_sym_stats_reset(struct rte_cryptodev *dev)
+{
+ struct qat_sym_dev_private *qat_priv;
+
+ if (dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
+
+}
+
+static int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+{
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+
+ PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
+ queue_pair_id, dev->data->dev_id);
+
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
+ = NULL;
+
+ return qat_qp_release((struct qat_qp **)
+ &(dev->data->queue_pairs[queue_pair_id]));
+}
+
+static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ struct qat_qp **qp_addr =
+ (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[qat_private->qat_dev->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+ const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (*qp_addr != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+ qat_qp_conf.hw = qp_hw_data;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ /* store a link to the qp in the qat_pci_device */
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
+ = *qp_addr;
+
+ qp = (struct qat_qp *)*qp_addr;
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
+
+ struct qat_sym_op_cookie *sql_cookie =
+ qp->op_cookies[i];
+
+ sql_cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_src);
+
+ sql_cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_dst);
+ }
+
+ return ret;
+}
+
+static struct rte_cryptodev_ops crypto_qat_ops = {
+
+ /* Device related operations */
+ .dev_configure = qat_sym_dev_config,
+ .dev_start = qat_sym_dev_start,
+ .dev_stop = qat_sym_dev_stop,
+ .dev_close = qat_sym_dev_close,
+ .dev_infos_get = qat_sym_dev_info_get,
+
+ .stats_get = qat_sym_stats_get,
+ .stats_reset = qat_sym_stats_reset,
+ .queue_pair_setup = qat_sym_qp_setup,
+ .queue_pair_release = qat_sym_qp_release,
+ .queue_pair_start = NULL,
+ .queue_pair_stop = NULL,
+ .queue_pair_count = NULL,
+
+ /* Crypto related operations */
+ .session_get_size = qat_sym_session_get_private_size,
+ .session_configure = qat_sym_session_configure,
+ .session_clear = qat_sym_session_clear
+};
+
+static uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
+}
+
+static uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
+}
+
+int
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev_pmd_init_params init_params = {
+ .name = "",
+ .socket_id = qat_pci_dev->pci_dev->device.numa_node,
+ .private_data_size = sizeof(struct qat_sym_dev_private),
+ .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
+ };
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct rte_cryptodev *cryptodev;
+ struct qat_sym_dev_private *internals;
+
+ snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
+ qat_pci_dev->name, "sym");
+ PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+
+ cryptodev = rte_cryptodev_pmd_create(name,
+ &qat_pci_dev->pci_dev->device, &init_params);
+
+ if (cryptodev == NULL)
+ return -ENODEV;
+
+ cryptodev->driver_id = cryptodev_qat_driver_id;
+ cryptodev->dev_ops = &crypto_qat_ops;
+
+ cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
+ cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
+
+ cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
+ RTE_CRYPTODEV_FF_HW_ACCELERATED |
+ RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
+ RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
+
+ internals = cryptodev->data->dev_private;
+ internals->qat_dev = qat_pci_dev;
+ qat_pci_dev->sym_dev = internals;
+
+ internals->sym_dev_id = cryptodev->data->dev_id;
+ switch (qat_pci_dev->qat_dev_gen) {
+ case QAT_GEN1:
+ internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
+ break;
+ case QAT_GEN2:
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ break;
+ default:
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ PMD_DRV_LOG(DEBUG,
+ "QAT gen %d capabilities unknown, default to GEN2",
+ qat_pci_dev->qat_dev_gen);
+ break;
+ }
+
+ PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
+ name, internals->sym_dev_id);
+ return 0;
+}
+
+int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev *cryptodev;
+
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+ if (qat_pci_dev->sym_dev == NULL)
+ return 0;
+
+ /* free crypto device */
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
+ rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_dev = NULL;
+
+ return 0;
+}
+
+
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+static struct cryptodev_driver qat_crypto_drv;
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+ cryptodev_qat_driver_id);
diff --git a/drivers/crypto/qat/qat_sym_pmd.h b/drivers/crypto/qat/qat_sym_pmd.h
new file mode 100644
index 000000000..efa3b0775
--- /dev/null
+++ b/drivers/crypto/qat/qat_sym_pmd.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2015-2018 Intel Corporation
+ */
+
+#ifndef _QAT_SYM_PMD_H_
+#define _QAT_SYM_PMD_H_
+
+#include <rte_cryptodev.h>
+
+#include "qat_sym_capabilities.h"
+#include "qat_device.h"
+
+
+/**< Intel(R) QAT Symmetric Crypto PMD device name */
+#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+
+extern uint8_t cryptodev_qat_driver_id;
+
+/** private data structure for a QAT device.
+ * This QAT device is a device offering only symmetric crypto service,
+ * there can be one of these on each qat_pci_device (VF),
+ * in future there may also be private data structures for other services.
+ */
+struct qat_sym_dev_private {
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
+ uint8_t sym_dev_id;
+ /**< Device instance for this rte_cryptodev */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device symmetric crypto capabilities */
+};
+
+
+int
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_SYM_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index 68d7773a2..689596d51 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -1,6 +1,12 @@
/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
* Copyright(c) 2015-2018 Intel Corporation
*/
+
+#include <openssl/sha.h> /* Needed to calculate pre-compute values */
+#include <openssl/aes.h> /* Needed to calculate pre-compute values */
+#include <openssl/md5.h> /* Needed to calculate pre-compute values */
+#include <openssl/evp.h> /* Needed for bpi runt block processing */
+
#include <rte_memcpy.h>
#include <rte_common.h>
#include <rte_spinlock.h>
@@ -10,14 +16,8 @@
#include <rte_crypto_sym.h>
#include "qat_logs.h"
-#include "qat_device.h"
-
-#include <openssl/sha.h> /* Needed to calculate pre-compute values */
-#include <openssl/aes.h> /* Needed to calculate pre-compute values */
-#include <openssl/md5.h> /* Needed to calculate pre-compute values */
-#include <openssl/evp.h>
-
#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
/** Frees a context previously created
* Depends on openssl libcrypto
@@ -29,6 +29,9 @@ bpi_cipher_ctx_free(void *bpi_ctx)
EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
}
+/** Creates a context in either AES or DES in ECB mode
+ * Depends on openssl libcrypto
+ */
static int
bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
enum rte_crypto_cipher_operation direction __rte_unused,
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index d3d27ff46..18d77e990 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _QAT_SYM_SESSION_H_
#define _QAT_SYM_SESSION_H_
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
deleted file mode 100644
index 91bb1e590..000000000
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ /dev/null
@@ -1,258 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2015-2018 Intel Corporation
- */
-
-#include <rte_bus_pci.h>
-#include <rte_common.h>
-#include <rte_dev.h>
-#include <rte_malloc.h>
-#include <rte_pci.h>
-#include <rte_cryptodev_pmd.h>
-
-#include "qat_sym.h"
-#include "qat_sym_session.h"
-#include "qat_logs.h"
-
-uint8_t cryptodev_qat_driver_id;
-
-static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
- QAT_BASE_GEN1_SYM_CAPABILITIES,
- RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
-};
-
-static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
- QAT_BASE_GEN1_SYM_CAPABILITIES,
- QAT_EXTRA_GEN2_SYM_CAPABILITIES,
- RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
-};
-
-static struct rte_cryptodev_ops crypto_qat_ops = {
-
- /* Device related operations */
- .dev_configure = qat_sym_dev_config,
- .dev_start = qat_sym_dev_start,
- .dev_stop = qat_sym_dev_stop,
- .dev_close = qat_sym_dev_close,
- .dev_infos_get = qat_sym_dev_info_get,
-
- .stats_get = qat_sym_stats_get,
- .stats_reset = qat_sym_stats_reset,
- .queue_pair_setup = qat_sym_qp_setup,
- .queue_pair_release = qat_sym_qp_release,
- .queue_pair_start = NULL,
- .queue_pair_stop = NULL,
- .queue_pair_count = NULL,
-
- /* Crypto related operations */
- .session_get_size = qat_sym_session_get_private_size,
- .session_configure = qat_sym_session_configure,
- .session_clear = qat_sym_session_clear
-};
-
-/*
- * The set of PCI devices this driver supports
- */
-
-static const struct rte_pci_id pci_id_qat_map[] = {
- {
- RTE_PCI_DEVICE(0x8086, 0x0443),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x37c9),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x19e3),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x6f55),
- },
- {.device_id = 0},
-};
-
-
-
-static int
-qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
-{
- struct rte_cryptodev_pmd_init_params init_params = {
- .name = "",
- .socket_id = qat_pci_dev->pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_sym_dev_private),
- .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
- };
- char name[RTE_CRYPTODEV_NAME_MAX_LEN];
- struct rte_cryptodev *cryptodev;
- struct qat_sym_dev_private *internals;
-
- snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
- qat_pci_dev->name, "sym");
- PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
-
- cryptodev = rte_cryptodev_pmd_create(name,
- &qat_pci_dev->pci_dev->device, &init_params);
-
- if (cryptodev == NULL)
- return -ENODEV;
-
- cryptodev->driver_id = cryptodev_qat_driver_id;
- cryptodev->dev_ops = &crypto_qat_ops;
-
- cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
- cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
-
- cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
- RTE_CRYPTODEV_FF_HW_ACCELERATED |
- RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
- RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
-
- internals = cryptodev->data->dev_private;
- internals->qat_dev = qat_pci_dev;
- qat_pci_dev->sym_dev = internals;
-
- internals->sym_dev_id = cryptodev->data->dev_id;
- switch (qat_pci_dev->qat_dev_gen) {
- case QAT_GEN1:
- internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
- break;
- case QAT_GEN2:
- internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
- break;
- default:
- internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
- PMD_DRV_LOG(DEBUG,
- "QAT gen %d capabilities unknown, default to GEN2",
- qat_pci_dev->qat_dev_gen);
- break;
- }
-
- PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
- name, internals->sym_dev_id);
- return 0;
-}
-
-static int
-qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
-{
- struct rte_cryptodev *cryptodev;
-
- if (qat_pci_dev == NULL)
- return -ENODEV;
- if (qat_pci_dev->sym_dev == NULL)
- return 0;
-
- /* free crypto device */
- cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
- rte_cryptodev_pmd_destroy(cryptodev);
- qat_pci_dev->sym_dev = NULL;
-
- return 0;
-}
-
-static int
-qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_pci_dev_destroy(struct qat_pci_device *qat_pci_dev,
- struct rte_pci_device *pci_dev)
-{
- qat_sym_dev_destroy(qat_pci_dev);
- qat_comp_dev_destroy(qat_pci_dev);
- qat_asym_dev_destroy(qat_pci_dev);
- return qat_pci_device_release(pci_dev);
-}
-
-static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
- struct rte_pci_device *pci_dev)
-{
- int ret = 0;
- struct qat_pci_device *qat_pci_dev;
-
- PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
- pci_dev->addr.bus,
- pci_dev->addr.devid,
- pci_dev->addr.function);
-
- qat_pci_dev = qat_pci_device_allocate(pci_dev);
- if (qat_pci_dev == NULL)
- return -ENODEV;
-
- ret = qat_sym_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- ret = qat_comp_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- ret = qat_asym_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- return 0;
-
-error_out:
- qat_pci_dev_destroy(qat_pci_dev, pci_dev);
- return ret;
-
-}
-
-static int qat_pci_remove(struct rte_pci_device *pci_dev)
-{
- struct qat_pci_device *qat_pci_dev;
-
- if (pci_dev == NULL)
- return -EINVAL;
-
- qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
- if (qat_pci_dev == NULL)
- return 0;
-
- return qat_pci_dev_destroy(qat_pci_dev, pci_dev);
-
-}
-
-
-/* An rte_driver is needed in the registration of both the device and the driver
- * with cryptodev.
- * The actual qat pci's rte_driver can't be used as its name represents
- * the whole pci device with all services. Think of this as a holder for a name
- * for the crypto part of the pci device.
- */
-static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
-static struct rte_driver cryptodev_qat_sym_driver = {
- .name = qat_sym_drv_name,
- .alias = qat_sym_drv_name
-};
-static struct cryptodev_driver qat_crypto_drv;
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
- cryptodev_qat_driver_id);
-
-static struct rte_pci_driver rte_qat_pmd = {
- .id_table = pci_id_qat_map,
- .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
- .probe = qat_pci_probe,
- .remove = qat_pci_remove
-};
-RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
-RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 24/31] crypto/qat: add lock around csr access and change logic
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (54 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 23/31] crypto/qat: move code into appropriate files Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 25/31] crypto/qat: remove incorrect usage of bundle number Fiona Trahe
` (7 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Add lock around accesses to the arbiter CSR
and use & instead of ^ as ^ not safe if
arb_disable called when already disabled.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 26 +++++++++++++++++++-------
1 file changed, 19 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 7b2dc3f90..f26fd0900 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -107,8 +107,10 @@ static int qat_queue_create(struct qat_pci_device *qat_dev,
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
static void adf_configure_queues(struct qat_qp *queue);
-static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr);
-static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr);
+static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock);
+static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock);
int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
@@ -216,7 +218,8 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
}
adf_configure_queues(qp);
- adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
+ adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr,
+ &qat_dev->arb_csr_lock);
snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE,
"%s%d_cookies_%s_qp%hu",
@@ -282,7 +285,8 @@ int qat_qp_release(struct qat_qp **qp_addr)
return -EAGAIN;
}
- adf_queue_arb_disable(&(qp->tx_q), qp->mmap_bar_addr);
+ adf_queue_arb_disable(&(qp->tx_q), qp->mmap_bar_addr,
+ &qp->qat_dev->arb_csr_lock);
for (i = 0; i < qp->nb_descriptors; i++)
rte_mempool_put(qp->op_cookie_pool, qp->op_cookies[i]);
@@ -443,7 +447,8 @@ static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
return -EINVAL;
}
-static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr)
+static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock)
{
uint32_t arb_csr_offset = ADF_ARB_RINGSRVARBEN_OFFSET +
(ADF_ARB_REG_SLOT *
@@ -451,12 +456,16 @@ static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr)
uint32_t value;
PMD_INIT_FUNC_TRACE();
+
+ rte_spinlock_lock(lock);
value = ADF_CSR_RD(base_addr, arb_csr_offset);
value |= (0x01 << txq->hw_queue_number);
ADF_CSR_WR(base_addr, arb_csr_offset, value);
+ rte_spinlock_unlock(lock);
}
-static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr)
+static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock)
{
uint32_t arb_csr_offset = ADF_ARB_RINGSRVARBEN_OFFSET +
(ADF_ARB_REG_SLOT *
@@ -464,9 +473,12 @@ static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr)
uint32_t value;
PMD_INIT_FUNC_TRACE();
+
+ rte_spinlock_lock(lock);
value = ADF_CSR_RD(base_addr, arb_csr_offset);
- value ^= (0x01 << txq->hw_queue_number);
+ value &= ~(0x01 << txq->hw_queue_number);
ADF_CSR_WR(base_addr, arb_csr_offset, value);
+ rte_spinlock_unlock(lock);
}
static void adf_configure_queues(struct qat_qp *qp)
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 25/31] crypto/qat: remove incorrect usage of bundle number
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (55 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 24/31] crypto/qat: add lock around csr access and change logic Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 26/31] crypto/qat: cleanups Fiona Trahe
` (6 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
As bundle_num is included in qat_gen1_qps static array
there shouldn't be a multiplier used in qat_qps_per_service()
Then removed ADF_NUM_BUNDLES_PER_DEV as no longer used.
Also renamed ADF_MAX_QPS_PER_BUNDLE to ADF_MAX_QPS_ON_ANY_SERVICE
and reduced from 4 to 2 which is enough for all current devices.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../crypto/qat/qat_adf/adf_transport_access_macros.h | 5 ++---
drivers/crypto/qat/qat_common.c | 4 ++--
drivers/crypto/qat/qat_device.h | 4 ++--
drivers/crypto/qat/qat_qp.c | 19 ++++---------------
drivers/crypto/qat/qat_qp.h | 2 +-
5 files changed, 11 insertions(+), 23 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index 2136d54ab..1eef5513f 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -50,9 +50,8 @@
#define ADF_MAX_RING_SIZE ADF_RING_SIZE_4M
#define ADF_DEFAULT_RING_SIZE ADF_RING_SIZE_16K
-#define ADF_NUM_BUNDLES_PER_DEV 1
-/* Maximum number of qps for any service type */
-#define ADF_MAX_QPS_PER_BUNDLE 4
+/* Maximum number of qps on a device for any service type */
+#define ADF_MAX_QPS_ON_ANY_SERVICE 2
#define ADF_RING_DIR_TX 0
#define ADF_RING_DIR_RX 1
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index f1759ea76..5d6779757 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -67,7 +67,7 @@ void qat_stats_get(struct qat_pci_device *dev,
}
qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ for (i = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++) {
if (qp[i] == NULL) {
PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
service, i);
@@ -94,7 +94,7 @@ void qat_stats_reset(struct qat_pci_device *dev,
}
qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ for (i = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++) {
if (qp[i] == NULL) {
PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
service, i);
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index fd1819354..fd20a0147 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -44,7 +44,7 @@ struct qat_pci_device {
uint8_t attached : 1;
/**< Flag indicating the device is attached */
- struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_PER_BUNDLE];
+ struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_ON_ANY_SERVICE];
/**< links to qps set up for each service, index same as on API */
/* Data relating to symmetric crypto service */
@@ -59,7 +59,7 @@ struct qat_pci_device {
struct qat_gen_hw_data {
enum qat_device_gen dev_gen;
- const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
+ const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_ON_ANY_SERVICE];
};
extern struct qat_gen_hw_data qp_gen_config[];
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index f26fd0900..9938c1493 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -29,7 +29,7 @@
__extension__
const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
- [ADF_MAX_QPS_PER_BUNDLE] = {
+ [ADF_MAX_QPS_ON_ANY_SERVICE] = {
/* queue pairs which provide an asymmetric crypto service */
[QAT_SERVICE_ASYMMETRIC] = {
{
@@ -42,14 +42,11 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
}, {
.service_type = QAT_SERVICE_ASYMMETRIC,
+ .hw_bundle_num = 0,
.tx_ring_num = 1,
.rx_ring_num = 9,
.tx_msg_size = 64,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
},
/* queue pairs which provide a symmetric crypto service */
@@ -69,10 +66,6 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
.rx_ring_num = 11,
.tx_msg_size = 128,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
},
/* queue pairs which provide a compression service */
@@ -91,10 +84,6 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
.rx_ring_num = 15,
.tx_msg_size = 128,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
}
};
@@ -118,10 +107,10 @@ int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
{
int i, count;
- for (i = 0, count = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++)
+ for (i = 0, count = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++)
if (qp_hw_data[i].service_type == service)
count++;
- return count * ADF_NUM_BUNDLES_PER_DEV;
+ return count;
}
static const struct rte_memzone *
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 73888b805..6f07bd67c 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -90,7 +90,7 @@ struct qat_qp {
/**< qat device this qp is on */
} __rte_cache_aligned;
-extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_PER_BUNDLE];
+extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_ON_ANY_SERVICE];
uint16_t
qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 26/31] crypto/qat: cleanups
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (56 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 25/31] crypto/qat: remove incorrect usage of bundle number Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-06-11 22:21 ` De Lara Guarch, Pablo
2018-05-11 11:14 ` [PATCH v2 27/31] crypto/qat: create appropriately named device for registration Fiona Trahe
` (5 subsequent siblings)
63 siblings, 1 reply; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
- renamed sgl_cookie to cookie
- renamed qp_gen_config to qat_gen_config as it is intended
to hold more than just qp data.
- removed unused macro - ALIGN_POW2
- moved 64_BYTE_ALIGN to qat_common.h
- removed crypto reference in common debug msg
- free cookie pool on qp creation error
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/qat_common.c | 2 +-
drivers/crypto/qat/qat_common.h | 2 ++
drivers/crypto/qat/qat_device.c | 2 +-
drivers/crypto/qat/qat_device.h | 2 +-
drivers/crypto/qat/qat_qp.c | 3 +++
drivers/crypto/qat/qat_sym.h | 8 --------
drivers/crypto/qat/qat_sym_pmd.c | 14 +++++++-------
7 files changed, 15 insertions(+), 18 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index 5d6779757..c10c1421a 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -103,5 +103,5 @@ void qat_stats_reset(struct qat_pci_device *dev,
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
}
- PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+ PMD_DRV_LOG(DEBUG, "QAT: %d stats cleared", service);
}
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 8ecebe954..db85d5482 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -15,6 +15,8 @@
*/
#define QAT_SGL_MAX_NUMBER 16
+#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
+
/* Intel(R) QuickAssist Technology device generation is enumerated
* from one according to the generation of the device
*/
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index c9d4b32ed..a0e414905 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -10,7 +10,7 @@
/* Hardware device information per generation */
__extension__
-struct qat_gen_hw_data qp_gen_config[] = {
+struct qat_gen_hw_data qat_gen_config[] = {
[QAT_GEN1] = {
.dev_gen = QAT_GEN1,
.qp_hw_data = qat_gen1_qps,
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index fd20a0147..4201a1c71 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -62,7 +62,7 @@ struct qat_gen_hw_data {
const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_ON_ANY_SERVICE];
};
-extern struct qat_gen_hw_data qp_gen_config[];
+extern struct qat_gen_hw_data qat_gen_config[];
struct qat_pci_device *
qat_pci_device_allocate(struct rte_pci_device *pci_dev);
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 9938c1493..569eace57 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -248,6 +248,9 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
return 0;
create_err:
+ if (qp->op_cookie_pool)
+ rte_mempool_free(qp->op_cookie_pool);
+ rte_free(qp->op_cookies);
rte_free(qp);
return -EFAULT;
}
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index d887dc126..bdb672be4 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -9,14 +9,6 @@
#include "qat_common.h"
-/*
- * This macro rounds up a number to a be a multiple of
- * the alignment when the alignment is a power of 2
- */
-#define ALIGN_POW2_ROUNDUP(num, align) \
- (((num) + (align) - 1) & ~((align) - 1))
-#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
-
struct qat_sym_session;
struct qat_sym_op_cookie {
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index aa71b4641..e6760b8f8 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -68,7 +68,7 @@ static void qat_sym_dev_info_get(struct rte_cryptodev *dev,
{
struct qat_sym_dev_private *internals = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev->qat_dev_gen]
+ qat_gen_config[internals->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
@@ -143,7 +143,7 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
struct qat_sym_dev_private *qat_private = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev->qat_dev_gen]
+ qat_gen_config[qat_private->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
@@ -178,16 +178,16 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
for (i = 0; i < qp->nb_descriptors; i++) {
- struct qat_sym_op_cookie *sql_cookie =
+ struct qat_sym_op_cookie *cookie =
qp->op_cookies[i];
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
+ cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(cookie) +
offsetof(struct qat_sym_op_cookie,
qat_sgl_src);
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
+ cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(cookie) +
offsetof(struct qat_sym_op_cookie,
qat_sgl_dst);
}
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 27/31] crypto/qat: create appropriately named device for registration
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (57 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 26/31] crypto/qat: cleanups Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 28/31] crypto/qat: add MAX PCI DEVICES flag to config file Fiona Trahe
` (4 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
For every QAT pci device probed, populate a local rte_device
containing an rte_driver. The rte_driver was created in a previous
patch to provide a crypto-specific driver name: "crypto_qat".
This was previously only used for driver registration, now it's also
used in device creation.
This allows applications to find devices driven by "crypto_qat".
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.h | 5 +++++
drivers/crypto/qat/qat_sym_pmd.c | 38 ++++++++++++++++++++++++--------------
2 files changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 4201a1c71..3df6520c3 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -50,6 +50,11 @@ struct qat_pci_device {
/* Data relating to symmetric crypto service */
struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
+ struct rte_device sym_rte_dev;
+ /**< This represents the crypto subset of this pci device.
+ * Register with this rather than with the one in
+ * pci_dev so that its driver can have a crypto-specific name
+ */
/* Data relating to compression service */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index e6760b8f8..28e579b77 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -232,6 +232,18 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static const struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+
int
qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
{
@@ -249,12 +261,19 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
qat_pci_dev->name, "sym");
PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+ /* Populate subset device to use in cryptodev device creation */
+ qat_pci_dev->sym_rte_dev.driver = &cryptodev_qat_sym_driver;
+ qat_pci_dev->sym_rte_dev.numa_node =
+ qat_pci_dev->pci_dev->device.numa_node;
+ qat_pci_dev->sym_rte_dev.devargs = NULL;
+
cryptodev = rte_cryptodev_pmd_create(name,
- &qat_pci_dev->pci_dev->device, &init_params);
+ &(qat_pci_dev->sym_rte_dev), &init_params);
if (cryptodev == NULL)
return -ENODEV;
+ qat_pci_dev->sym_rte_dev.name = cryptodev->data->name;
cryptodev->driver_id = cryptodev_qat_driver_id;
cryptodev->dev_ops = &crypto_qat_ops;
@@ -287,7 +306,7 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
}
PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
- name, internals->sym_dev_id);
+ cryptodev->data->name, internals->sym_dev_id);
return 0;
}
@@ -304,23 +323,14 @@ qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
/* free crypto device */
cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_rte_dev.name = NULL;
qat_pci_dev->sym_dev = NULL;
return 0;
}
-/* An rte_driver is needed in the registration of both the device and the driver
- * with cryptodev.
- * The actual qat pci's rte_driver can't be used as its name represents
- * the whole pci device with all services. Think of this as a holder for a name
- * for the crypto part of the pci device.
- */
-static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
-static struct rte_driver cryptodev_qat_sym_driver = {
- .name = qat_sym_drv_name,
- .alias = qat_sym_drv_name
-};
static struct cryptodev_driver qat_crypto_drv;
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv,
+ cryptodev_qat_sym_driver,
cryptodev_qat_driver_id);
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 28/31] crypto/qat: add MAX PCI DEVICES flag to config file
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (58 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 27/31] crypto/qat: create appropriately named device for registration Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 29/31] crypto/qat: add performance improvement into qat crypto dev Fiona Trahe
` (3 subsequent siblings)
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Added CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES to
build config files.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
config/common_base | 5 ++++-
config/rte_config.h | 2 ++
drivers/crypto/qat/qat_device.c | 8 ++++----
drivers/crypto/qat/qat_device.h | 2 --
4 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/config/common_base b/config/common_base
index 28557ed66..3dbe95bd4 100644
--- a/config/common_base
+++ b/config/common_base
@@ -491,7 +491,10 @@ CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n
# on a single QuickAssist device.
#
CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=2048
-
+#
+# Max. number of QuickAssist devices, which can be detected and attached
+#
+CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48
#
# Compile PMD for virtio crypto devices
#
diff --git a/config/rte_config.h b/config/rte_config.h
index a1d01759e..e353d8d6d 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -91,6 +91,8 @@
*/
/* QuickAssist device */
#define RTE_QAT_PMD_MAX_NB_SESSIONS 2048
+/* Max. number of QuickAssist devices which can be attached */
+#define RTE_PMD_QAT_MAX_PCI_DEVICES 48
/* virtio crypto defines */
#define RTE_VIRTIO_CRYPTO_PMD_MAX_NB_SESSIONS 1024
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index a0e414905..0cab2e928 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -23,7 +23,7 @@ struct qat_gen_hw_data qat_gen_config[] = {
};
-static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
+static struct qat_pci_device qat_pci_devices[RTE_PMD_QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
/*
@@ -62,7 +62,7 @@ qat_pci_get_named_dev(const char *name)
if (name == NULL)
return NULL;
- for (i = 0; i < QAT_MAX_PCI_DEVICES; i++) {
+ for (i = 0; i < RTE_PMD_QAT_MAX_PCI_DEVICES; i++) {
dev = &qat_pci_devices[i];
if ((dev->attached == QAT_ATTACHED) &&
@@ -78,7 +78,7 @@ qat_pci_find_free_device_index(void)
{
uint8_t dev_id;
- for (dev_id = 0; dev_id < QAT_MAX_PCI_DEVICES; dev_id++) {
+ for (dev_id = 0; dev_id < RTE_PMD_QAT_MAX_PCI_DEVICES; dev_id++) {
if (qat_pci_devices[dev_id].attached == QAT_DETACHED)
break;
}
@@ -111,7 +111,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
}
qat_dev_id = qat_pci_find_free_device_index();
- if (qat_dev_id == QAT_MAX_PCI_DEVICES) {
+ if (qat_dev_id == RTE_PMD_QAT_MAX_PCI_DEVICES) {
PMD_DRV_LOG(ERR, "Reached maximum number of QAT devices");
return NULL;
}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 3df6520c3..e18c8a706 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,11 +11,9 @@
#include "adf_transport_access_macros.h"
#include "qat_qp.h"
-
#define QAT_DETACHED (0)
#define QAT_ATTACHED (1)
-#define QAT_MAX_PCI_DEVICES 48
#define QAT_DEV_NAME_MAX_LEN 64
/*
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 29/31] crypto/qat: add performance improvement into qat crypto dev.
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (59 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 28/31] crypto/qat: add MAX PCI DEVICES flag to config file Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-06-11 22:23 ` De Lara Guarch, Pablo
2018-05-11 11:14 ` [PATCH v2 30/31] doc/qat: specify QAT driver and device name formats Fiona Trahe
` (2 subsequent siblings)
63 siblings, 1 reply; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
From: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
- removed unused args. from process_response function
- changed adf_modulo function
- changed qat_sym_process_response into static inline version
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 29 +++++----
drivers/crypto/qat/qat_qp.h | 10 +--
drivers/crypto/qat/qat_sym.c | 129 -------------------------------------
drivers/crypto/qat/qat_sym.h | 133 +++++++++++++++++++++++++++++++++++++--
drivers/crypto/qat/qat_sym_pmd.c | 1 -
5 files changed, 145 insertions(+), 157 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 569eace57..f9d3762d7 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -14,6 +14,7 @@
#include "qat_logs.h"
#include "qat_device.h"
#include "qat_qp.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
@@ -238,7 +239,7 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
qp->qat_dev_gen = qat_dev->qat_dev_gen;
qp->build_request = qat_qp_conf->build_request;
- qp->process_response = qat_qp_conf->process_response;
+ qp->service_type = qat_qp_conf->hw->service_type;
qp->qat_dev = qat_dev;
PMD_DRV_LOG(DEBUG, "QP setup complete: id: %d, cookiepool: %s",
@@ -376,7 +377,7 @@ qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
- queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
+ queue->modulo_mask = (1 << ADF_RING_SIZE_MODULO(queue->queue_size)) - 1;
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
@@ -401,11 +402,11 @@ qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
queue->hw_queue_number, queue_base);
PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
- " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ " nb msgs %u, msg_size %u, max_inflights %u modulo mask %u",
queue->memz_name,
queue->queue_size, queue_size_bytes,
qp_conf->nb_descriptors, desc_size,
- queue->max_inflights, queue->modulo);
+ queue->max_inflights, queue->modulo_mask);
return 0;
@@ -494,13 +495,9 @@ static void adf_configure_queues(struct qat_qp *qp)
queue->hw_queue_number, queue_config);
}
-
-static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
+static inline uint32_t adf_modulo(uint32_t data, uint32_t modulo_mask)
{
- uint32_t div = data >> shift;
- uint32_t mult = div << shift;
-
- return data - mult;
+ return data & modulo_mask;
}
static inline void
@@ -584,7 +581,7 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
goto kick_tail;
}
- tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ tail = adf_modulo(tail + queue->msg_size, queue->modulo_mask);
ops++;
nb_ops_sent++;
}
@@ -616,11 +613,13 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
resp_counter != nb_ops) {
- tmp_qp->process_response(ops, resp_msg,
- tmp_qp->op_cookies[head / rx_queue->msg_size],
- tmp_qp->qat_dev_gen);
+ if (tmp_qp->service_type == QAT_SERVICE_SYMMETRIC)
+ qat_sym_process_response(ops, resp_msg);
+ /* add qat_asym_process_response here */
+ /* add qat_comp_process_response here */
- head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
+ head = adf_modulo(head + rx_queue->msg_size,
+ rx_queue->modulo_mask);
resp_msg = (uint8_t *)rx_queue->base_addr + head;
ops++;
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 6f07bd67c..59db945e7 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -21,11 +21,6 @@ typedef int (*build_request_t)(void *op,
enum qat_device_gen qat_dev_gen);
/**< Build a request from an op. */
-typedef int (*process_response_t)(void **ops,
- uint8_t *resp, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
-/**< Process a response descriptor and return the associated op. */
-
/**
* Structure with data needed for creation of queue pair.
*/
@@ -46,7 +41,6 @@ struct qat_qp_config {
uint32_t cookie_size;
int socket_id;
build_request_t build_request;
- process_response_t process_response;
const char *service_str;
};
@@ -59,7 +53,7 @@ struct qat_queue {
rte_iova_t base_phys_addr; /* Queue physical address */
uint32_t head; /* Shadow copy of the head */
uint32_t tail; /* Shadow copy of the tail */
- uint32_t modulo;
+ uint32_t modulo_mask;
uint32_t msg_size;
uint16_t max_inflights;
uint32_t queue_size;
@@ -85,7 +79,7 @@ struct qat_qp {
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
- process_response_t process_response;
+ enum qat_service_type service_type;
struct qat_pci_device *qat_dev;
/**< qat device this qp is on */
} __rte_cache_aligned;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 15244d113..887d4ebcd 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -12,44 +12,7 @@
#include <rte_byteorder.h>
#include "qat_logs.h"
-#include "qat_sym_session.h"
#include "qat_sym.h"
-#include "qat_sym_pmd.h"
-
-#define BYTE_LENGTH 8
-/* bpi is only used for partial blocks of DES and AES
- * so AES block len can be assumed as max len for iv, src and dst
- */
-#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-
-/** Encrypt a single partial block
- * Depends on openssl libcrypto
- * Uses ECB+XOR to do CFB encryption, same result, more performant
- */
-static inline int
-bpi_cipher_encrypt(uint8_t *src, uint8_t *dst,
- uint8_t *iv, int ivlen, int srclen,
- void *bpi_ctx)
-{
- EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX *)bpi_ctx;
- int encrypted_ivlen;
- uint8_t encrypted_iv[BPI_MAX_ENCR_IV_LEN];
- uint8_t *encr = encrypted_iv;
-
- /* ECB method: encrypt the IV, then XOR this with plaintext */
- if (EVP_EncryptUpdate(ctx, encrypted_iv, &encrypted_ivlen, iv, ivlen)
- <= 0)
- goto cipher_encrypt_err;
-
- for (; srclen != 0; --srclen, ++dst, ++src, ++encr)
- *dst = *src ^ *encr;
-
- return 0;
-
-cipher_encrypt_err:
- PMD_DRV_LOG(ERR, "libcrypto ECB cipher encrypt failed");
- return -EINVAL;
-}
/** Decrypt a single partial block
* Depends on openssl libcrypto
@@ -136,62 +99,6 @@ qat_bpicipher_preprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-static inline uint32_t
-qat_bpicipher_postprocess(struct qat_sym_session *ctx,
- struct rte_crypto_op *op)
-{
- int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
- struct rte_crypto_sym_op *sym_op = op->sym;
- uint8_t last_block_len = block_len > 0 ?
- sym_op->cipher.data.length % block_len : 0;
-
- if (last_block_len > 0 &&
- ctx->qat_dir == ICP_QAT_HW_CIPHER_ENCRYPT) {
-
- /* Encrypt last block */
- uint8_t *last_block, *dst, *iv;
- uint32_t last_block_offset;
-
- last_block_offset = sym_op->cipher.data.offset +
- sym_op->cipher.data.length - last_block_len;
- last_block = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_src,
- uint8_t *, last_block_offset);
-
- if (unlikely(sym_op->m_dst != NULL))
- /* out-of-place operation (OOP) */
- dst = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_dst,
- uint8_t *, last_block_offset);
- else
- dst = last_block;
-
- if (last_block_len < sym_op->cipher.data.length)
- /* use previous block ciphertext as IV */
- iv = dst - block_len;
- else
- /* runt block, i.e. less than one full block */
- iv = rte_crypto_op_ctod_offset(op, uint8_t *,
- ctx->cipher_iv.offset);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "BPI: src before post-process:", last_block,
- last_block_len);
- if (sym_op->m_dst != NULL)
- rte_hexdump(stdout, "BPI: dst before post-process:",
- dst, last_block_len);
-#endif
- bpi_cipher_encrypt(last_block, dst, iv, block_len,
- last_block_len, ctx->bpi_ctx);
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "BPI: src after post-process:", last_block,
- last_block_len);
- if (sym_op->m_dst != NULL)
- rte_hexdump(stdout, "BPI: dst after post-process:", dst,
- last_block_len);
-#endif
- }
- return sym_op->cipher.data.length - last_block_len;
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
@@ -659,39 +566,3 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
#endif
return 0;
}
-
-int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen)
-{
-
- struct icp_qat_fw_comn_resp *resp_msg =
- (struct icp_qat_fw_comn_resp *)resp;
- struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
-
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
-
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess = (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
-
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
- *op = (void *)rx_op;
-
- return 0;
-}
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index bdb672be4..ddd0f473f 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -6,8 +6,19 @@
#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
+#include <rte_hexdump.h>
+
+#include <openssl/evp.h>
#include "qat_common.h"
+#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
+
+#define BYTE_LENGTH 8
+/* bpi is only used for partial blocks of DES and AES
+ * so AES block len can be assumed as max len for iv, src and dst
+ */
+#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
struct qat_sym_session;
@@ -21,9 +32,123 @@ struct qat_sym_op_cookie {
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
-int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen);
+
+/** Encrypt a single partial block
+ * Depends on openssl libcrypto
+ * Uses ECB+XOR to do CFB encryption, same result, more performant
+ */
+static inline int
+bpi_cipher_encrypt(uint8_t *src, uint8_t *dst,
+ uint8_t *iv, int ivlen, int srclen,
+ void *bpi_ctx)
+{
+ EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX *)bpi_ctx;
+ int encrypted_ivlen;
+ uint8_t encrypted_iv[BPI_MAX_ENCR_IV_LEN];
+ uint8_t *encr = encrypted_iv;
+
+ /* ECB method: encrypt the IV, then XOR this with plaintext */
+ if (EVP_EncryptUpdate(ctx, encrypted_iv, &encrypted_ivlen, iv, ivlen)
+ <= 0)
+ goto cipher_encrypt_err;
+
+ for (; srclen != 0; --srclen, ++dst, ++src, ++encr)
+ *dst = *src ^ *encr;
+
+ return 0;
+
+cipher_encrypt_err:
+ PMD_DRV_LOG(ERR, "libcrypto ECB cipher encrypt failed");
+ return -EINVAL;
+}
+
+static inline uint32_t
+qat_bpicipher_postprocess(struct qat_sym_session *ctx,
+ struct rte_crypto_op *op)
+{
+ int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
+ struct rte_crypto_sym_op *sym_op = op->sym;
+ uint8_t last_block_len = block_len > 0 ?
+ sym_op->cipher.data.length % block_len : 0;
+
+ if (last_block_len > 0 &&
+ ctx->qat_dir == ICP_QAT_HW_CIPHER_ENCRYPT) {
+
+ /* Encrypt last block */
+ uint8_t *last_block, *dst, *iv;
+ uint32_t last_block_offset;
+
+ last_block_offset = sym_op->cipher.data.offset +
+ sym_op->cipher.data.length - last_block_len;
+ last_block = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_src,
+ uint8_t *, last_block_offset);
+
+ if (unlikely(sym_op->m_dst != NULL))
+ /* out-of-place operation (OOP) */
+ dst = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_dst,
+ uint8_t *, last_block_offset);
+ else
+ dst = last_block;
+
+ if (last_block_len < sym_op->cipher.data.length)
+ /* use previous block ciphertext as IV */
+ iv = dst - block_len;
+ else
+ /* runt block, i.e. less than one full block */
+ iv = rte_crypto_op_ctod_offset(op, uint8_t *,
+ ctx->cipher_iv.offset);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "BPI: src before post-process:", last_block,
+ last_block_len);
+ if (sym_op->m_dst != NULL)
+ rte_hexdump(stdout, "BPI: dst before post-process:",
+ dst, last_block_len);
+#endif
+ bpi_cipher_encrypt(last_block, dst, iv, block_len,
+ last_block_len, ctx->bpi_ctx);
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "BPI: src after post-process:", last_block,
+ last_block_len);
+ if (sym_op->m_dst != NULL)
+ rte_hexdump(stdout, "BPI: dst after post-process:", dst,
+ last_block_len);
+#endif
+ }
+ return sym_op->cipher.data.length - last_block_len;
+}
+
+static inline void
+qat_sym_process_response(void **op, uint8_t *resp)
+{
+
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
+
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
+
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
+
+
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+ }
+ *op = (void *)rx_op;
+}
#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index 28e579b77..6b39b32f8 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -160,7 +160,6 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.hw = qp_hw_data;
qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
qat_qp_conf.socket_id = socket_id;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 30/31] doc/qat: specify QAT driver and device name formats
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (60 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 29/31] crypto/qat: add performance improvement into qat crypto dev Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-05-14 15:38 ` Kovacevic, Marko
2018-05-11 11:14 ` [PATCH v2 31/31] crypto/qat: remove CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS Fiona Trahe
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
63 siblings, 1 reply; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
Document the driver and device naming formats.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
doc/guides/cryptodevs/qat.rst | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst
index 8c8fefaa0..67176bd81 100644
--- a/doc/guides/cryptodevs/qat.rst
+++ b/doc/guides/cryptodevs/qat.rst
@@ -359,3 +359,15 @@ length of data to authenticate (op.sym.auth.data.length) must be the length
of all the items described above, including the padding at the end.
Also, offset of data to authenticate (op.sym.auth.data.offset)
must be such that points at the start of the COUNT bytes.
+
+Device and driver naming
+----------------------------------------
+
+The qat crypto driver name is "crypto_qat".
+This name is passed to the dpdk-test-crypto-perf tool in the -devtype parameter.
+The rte_cryptodev_devices_get() can return the devices exposed by a driver.
+
+Each qat crypto device has a unique name, in format
+<pci bdf>_<service>, e.g. "0000:41:01.0_qat_sym".
+This name can be passed to rte_cryptodev_get_dev_id() to get the device_id.
+This is also the format of the slave parameter passed to the crypto scheduler.
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v2 31/31] crypto/qat: remove CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (61 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 30/31] doc/qat: specify QAT driver and device name formats Fiona Trahe
@ 2018-05-11 11:14 ` Fiona Trahe
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
63 siblings, 0 replies; 109+ messages in thread
From: Fiona Trahe @ 2018-05-11 11:14 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, fiona.trahe, tomaszx.jozwiak
From: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
This patch removes CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS
from common_base, config/rte_config.h files and
defines QAT_SYM_PMD_MAX_NB_SESSIONS inside qat_sym_pmd.h
file instead.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
---
config/common_base | 5 -----
config/rte_config.h | 5 -----
drivers/crypto/qat/qat_sym_pmd.c | 4 ++--
drivers/crypto/qat/qat_sym_pmd.h | 1 +
4 files changed, 3 insertions(+), 12 deletions(-)
diff --git a/config/common_base b/config/common_base
index 3dbe95bd4..5c114fae6 100644
--- a/config/common_base
+++ b/config/common_base
@@ -487,11 +487,6 @@ CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n
#
-# Number of sessions to create in the session memory pool
-# on a single QuickAssist device.
-#
-CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=2048
-#
# Max. number of QuickAssist devices, which can be detected and attached
#
CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48
diff --git a/config/rte_config.h b/config/rte_config.h
index e353d8d6d..0ba0ead7e 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -85,12 +85,7 @@
/****** driver defines ********/
-/*
- * Number of sessions to create in the session memory pool
- * on a single instance of crypto HW device.
- */
/* QuickAssist device */
-#define RTE_QAT_PMD_MAX_NB_SESSIONS 2048
/* Max. number of QuickAssist devices which can be attached */
#define RTE_PMD_QAT_MAX_PCI_DEVICES 48
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index 6b39b32f8..c94a8a197 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -77,7 +77,7 @@ static void qat_sym_dev_info_get(struct rte_cryptodev *dev,
qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
info->feature_flags = dev->feature_flags;
info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
+ info->sym.max_nb_sessions = QAT_SYM_PMD_MAX_NB_SESSIONS;
info->driver_id = cryptodev_qat_driver_id;
info->pci_dev = RTE_DEV_TO_PCI(dev->device);
}
@@ -250,7 +250,7 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
.name = "",
.socket_id = qat_pci_dev->pci_dev->device.numa_node,
.private_data_size = sizeof(struct qat_sym_dev_private),
- .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
+ .max_nb_sessions = QAT_SYM_PMD_MAX_NB_SESSIONS
};
char name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct rte_cryptodev *cryptodev;
diff --git a/drivers/crypto/qat/qat_sym_pmd.h b/drivers/crypto/qat/qat_sym_pmd.h
index efa3b0775..80a198741 100644
--- a/drivers/crypto/qat/qat_sym_pmd.h
+++ b/drivers/crypto/qat/qat_sym_pmd.h
@@ -13,6 +13,7 @@
/**< Intel(R) QAT Symmetric Crypto PMD device name */
#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+#define QAT_SYM_PMD_MAX_NB_SESSIONS 2048
extern uint8_t cryptodev_qat_driver_id;
--
2.13.6
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [PATCH v2 30/31] doc/qat: specify QAT driver and device name formats
2018-05-11 11:14 ` [PATCH v2 30/31] doc/qat: specify QAT driver and device name formats Fiona Trahe
@ 2018-05-14 15:38 ` Kovacevic, Marko
0 siblings, 0 replies; 109+ messages in thread
From: Kovacevic, Marko @ 2018-05-14 15:38 UTC (permalink / raw)
To: Trahe, Fiona, dev; +Cc: De Lara Guarch, Pablo, Trahe, Fiona, Jozwiak, TomaszX
> Document the driver and device naming formats.
>
> Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
> ---
> doc/guides/cryptodevs/qat.rst | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst
> index 8c8fefaa0..67176bd81 100644
> --- a/doc/guides/cryptodevs/qat.rst
> +++ b/doc/guides/cryptodevs/qat.rst
> @@ -359,3 +359,15 @@ length of data to authenticate
> (op.sym.auth.data.length) must be the length of all the items described above,
> including the padding at the end.
> Also, offset of data to authenticate (op.sym.auth.data.offset) must be such
> that points at the start of the COUNT bytes.
> +
> +Device and driver naming
> +----------------------------------------
Try to keep the underscores aligned for the heading doesn't affect the build but for consistency
> +The qat crypto driver name is "crypto_qat".
> +This name is passed to the dpdk-test-crypto-perf tool in the -devtype
> parameter.
``-devtype``
> +The rte_cryptodev_devices_get() can return the devices exposed by a
> driver.
Try putting rte_cryptodev_devices_get() like this --> ``rte_cryptodev_devices_get()``
> +Each qat crypto device has a unique name, in format <pci
> +bdf>_<service>, e.g. "0000:41:01.0_qat_sym".
And these two ``<pci> +bdf>_<service>``, e.g. ``0000:41:01.0_qat_sym``.
> +This name can be passed to rte_cryptodev_get_dev_id() to get the
> device_id.
This also ``rte_cryptodev_get_dev_id()``
> +This is also the format of the slave parameter passed to the crypto scheduler.
Marko K.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH v2 03/31] crypto/qat: remove unused includes
2018-05-11 11:13 ` [PATCH v2 03/31] crypto/qat: remove unused includes Fiona Trahe
@ 2018-06-11 22:20 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 109+ messages in thread
From: De Lara Guarch, Pablo @ 2018-06-11 22:20 UTC (permalink / raw)
To: Trahe, Fiona, dev; +Cc: Jozwiak, TomaszX, ArkadiuszX Kusztal
Hi,
> -----Original Message-----
> From: Trahe, Fiona
> Sent: Friday, May 11, 2018 12:14 PM
> To: dev@dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Trahe, Fiona
> <fiona.trahe@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>;
> ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
> Subject: [PATCH v2 03/31] crypto/qat: remove unused includes
>
> This commit removes unused includes from qat_crypto.c and removes unused
> file qat_algs.h
>
> Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
> Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
There is a compilation error on this patch.
Thanks,
Pablo
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH v2 26/31] crypto/qat: cleanups
2018-05-11 11:14 ` [PATCH v2 26/31] crypto/qat: cleanups Fiona Trahe
@ 2018-06-11 22:21 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 109+ messages in thread
From: De Lara Guarch, Pablo @ 2018-06-11 22:21 UTC (permalink / raw)
To: Trahe, Fiona, dev; +Cc: Jozwiak, TomaszX
> -----Original Message-----
> From: Trahe, Fiona
> Sent: Friday, May 11, 2018 12:14 PM
> To: dev@dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Trahe, Fiona
> <fiona.trahe@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> Subject: [PATCH v2 26/31] crypto/qat: cleanups
>
> - renamed sgl_cookie to cookie
> - renamed qp_gen_config to qat_gen_config as it is intended
> to hold more than just qp data.
> - removed unused macro - ALIGN_POW2
> - moved 64_BYTE_ALIGN to qat_common.h
> - removed crypto reference in common debug msg
> - free cookie pool on qp creation error
>
> Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
> Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
This patch can be split into multiple patches based on the commit message.
Thanks,
Pablo
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH v2 29/31] crypto/qat: add performance improvement into qat crypto dev.
2018-05-11 11:14 ` [PATCH v2 29/31] crypto/qat: add performance improvement into qat crypto dev Fiona Trahe
@ 2018-06-11 22:23 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 109+ messages in thread
From: De Lara Guarch, Pablo @ 2018-06-11 22:23 UTC (permalink / raw)
To: Trahe, Fiona, dev; +Cc: Jozwiak, TomaszX
> -----Original Message-----
> From: Trahe, Fiona
> Sent: Friday, May 11, 2018 12:14 PM
> To: dev@dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Trahe, Fiona
> <fiona.trahe@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> Subject: [PATCH v2 29/31] crypto/qat: add performance improvement into qat
> crypto dev.
>
> From: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
>
> - removed unused args. from process_response function
> - changed adf_modulo function
> - changed qat_sym_process_response into static inline version
>
> Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
> Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
This patch can be split into multiple patches, looking at the commit message.
Thanks,
Pablo
^ permalink raw reply [flat|nested] 109+ messages in thread
* [PATCH v3 00/38] crypto/qat: refactor to support multiple services
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
` (62 preceding siblings ...)
2018-05-11 11:14 ` [PATCH v2 31/31] crypto/qat: remove CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS Fiona Trahe
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 01/38] crypto/qat: add qat common header Tomasz Jozwiak
` (38 more replies)
63 siblings, 39 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Pablo de Lara <pablo.de.lara.guarch@intel.com>
The QAT crypto PMD was peppered with references to rte_cryptodev artefacts.
Also the pci device it presented to the API layer was entirely owned by the one
cryptodev device instance created by the probe.
This patchset refactors the PMD so one pci device can present out multiple different
device instances to implement different APIs, specifically cryptodev and compressdev instances.
The code is refactored into
- common code which handles the pci device discovery, configuration,
queue-pair setup, stats gathering and data-path enqueue and dequeue.
- service-specific code for symmetric crypto, which is a thin layer
implementing the cryptodev API and using the common code.
- place-holder files for service-specific code to support asymmetric
crypto and compression services in future.
The code is all still in the crypto/qat directory as it is functionally unchanged,
still only supporting a symmetric crypto service. The code will move to drivers/common,
/crypto and /compress in a future patch when this is needed for a compression PMD.
Changes for v3:
- fixed some commit messages
- fixed compilation issue in one of the patches
- split two commits into multiple commits
Changes for v2:
- fixed some bugs compiled in when debug enabled
- added patch to remove MAX_NB_SESSIONS from config file
- removed SPDX license patch as already applied
- documented device name formats
Fiona Trahe (36):
crypto/qat: add qat common header
crypto/qat: add qat device files
crypto/qat: add symmetric session file
crypto/qat: change filename crypto to sym
crypto/qat: rename fns for consistency
crypto/qat: renamed sym-specific structs
crypto/qat: make enqueue function generic
crypto/qat: make dequeue function generic
crypto/qat: move generic qp fn to qp file
crypto/qat: separate sym-specific from generic qp setup
crypto/qat: move sym-specific qp code to sym file
crypto/qat: remove dependencies on cryptodev from common
crypto/qat: move defines from sym to qp header file
crypto/qat: create structures to support various generations
crypto/qat: rename sgl related objects
crypto/qat: move sgl related element to appropriate files
crypto/qat: add QAT PCI device struct
crypto/qat: use generic driver name for PCI registration
crypto/qat: move to using new device structure
crypto/qat: use common stats structures
crypto/qat: rename functions which depend on cryptodev
crypto/qat: move code into appropriate files
crypto/qat: add lock around csr access and change logic
crypto/qat: remove incorrect usage of bundle number
crypto/qat: rename variables
crypto/qat: modify debug message
crypto/qat: free cookie pool on queue creation error
crypto/qat: remove unused macro
crypto/qat: move macro to common file
crypto/qat: register appropriately named device
crypto/qat: add max PCI devices to config file
crypto/qat: optimize adf modulo function
crypto/qat: remove unused arguments
crypto/qat: make response process function inline
crypto/qat: check for service type
doc/qat: specify QAT driver and device name formats
Tomasz Jozwiak (2):
crypto/qat: remove unused includes
crypto/qat: remove configurable max number of sessions
config/common_base | 6 +-
config/rte_config.h | 7 +-
doc/guides/cryptodevs/qat.rst | 12 +
drivers/crypto/qat/Makefile | 10 +-
drivers/crypto/qat/meson.build | 9 +-
.../qat/qat_adf/adf_transport_access_macros.h | 9 +-
drivers/crypto/qat/qat_adf/qat_algs.h | 126 --
drivers/crypto/qat/qat_asym_pmd.c | 17 +
drivers/crypto/qat/qat_asym_pmd.h | 15 +
drivers/crypto/qat/qat_common.c | 107 ++
drivers/crypto/qat/qat_common.h | 76 +
drivers/crypto/qat/qat_comp_pmd.c | 18 +
drivers/crypto/qat/qat_comp_pmd.h | 29 +
drivers/crypto/qat/qat_crypto.c | 1696 -----------------
drivers/crypto/qat/qat_crypto.h | 150 --
drivers/crypto/qat/qat_device.c | 242 +++
drivers/crypto/qat/qat_device.h | 78 +
drivers/crypto/qat/qat_qp.c | 463 +++--
drivers/crypto/qat/qat_qp.h | 106 ++
drivers/crypto/qat/qat_sym.c | 568 ++++++
drivers/crypto/qat/qat_sym.h | 154 ++
..._capabilities.h => qat_sym_capabilities.h} | 6 +-
drivers/crypto/qat/qat_sym_pmd.c | 335 ++++
drivers/crypto/qat/qat_sym_pmd.h | 40 +
...at_algs_build_desc.c => qat_sym_session.c} | 786 +++++++-
drivers/crypto/qat/qat_sym_session.h | 143 ++
drivers/crypto/qat/rte_qat_cryptodev.c | 180 --
27 files changed, 3042 insertions(+), 2346 deletions(-)
delete mode 100644 drivers/crypto/qat/qat_adf/qat_algs.h
create mode 100644 drivers/crypto/qat/qat_asym_pmd.c
create mode 100644 drivers/crypto/qat/qat_asym_pmd.h
create mode 100644 drivers/crypto/qat/qat_common.c
create mode 100644 drivers/crypto/qat/qat_common.h
create mode 100644 drivers/crypto/qat/qat_comp_pmd.c
create mode 100644 drivers/crypto/qat/qat_comp_pmd.h
delete mode 100644 drivers/crypto/qat/qat_crypto.c
delete mode 100644 drivers/crypto/qat/qat_crypto.h
create mode 100644 drivers/crypto/qat/qat_device.c
create mode 100644 drivers/crypto/qat/qat_device.h
create mode 100644 drivers/crypto/qat/qat_qp.h
create mode 100644 drivers/crypto/qat/qat_sym.c
create mode 100644 drivers/crypto/qat/qat_sym.h
rename drivers/crypto/qat/{qat_crypto_capabilities.h => qat_sym_capabilities.h} (99%)
create mode 100644 drivers/crypto/qat/qat_sym_pmd.c
create mode 100644 drivers/crypto/qat/qat_sym_pmd.h
rename drivers/crypto/qat/{qat_adf/qat_algs_build_desc.c => qat_sym_session.c} (59%)
create mode 100644 drivers/crypto/qat/qat_sym_session.h
delete mode 100644 drivers/crypto/qat/rte_qat_cryptodev.c
--
2.17.0
^ permalink raw reply [flat|nested] 109+ messages in thread
* [PATCH v3 01/38] crypto/qat: add qat common header
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 02/38] crypto/qat: add qat device files Tomasz Jozwiak
` (37 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
This commit adds qat_common.h header file.
Following objects were moved to it:
qat_algs.h =>. qat_common.h
- struct qat_alg_buf
- struct qat_alg_buf_list
- struct qat_crypto_op_cookie
- QAT_SGL_MAX_NUMBER
qat_crypto.h => qat_common.h
- CRYPTODEV_NAME_QAT_SYM_PMD
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_adf/qat_algs.h | 25 --------------
drivers/crypto/qat/qat_common.h | 47 +++++++++++++++++++++++++++
drivers/crypto/qat/qat_crypto.h | 9 +----
3 files changed, 48 insertions(+), 33 deletions(-)
create mode 100644 drivers/crypto/qat/qat_common.h
diff --git a/drivers/crypto/qat/qat_adf/qat_algs.h b/drivers/crypto/qat/qat_adf/qat_algs.h
index 88bd5f00e..6c49c6529 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs.h
+++ b/drivers/crypto/qat/qat_adf/qat_algs.h
@@ -32,12 +32,6 @@
ICP_QAT_HW_CIPHER_KEY_CONVERT, \
ICP_QAT_HW_CIPHER_DECRYPT)
-struct qat_alg_buf {
- uint32_t len;
- uint32_t resrvd;
- uint64_t addr;
-} __rte_packed;
-
enum qat_crypto_proto_flag {
QAT_CRYPTO_PROTO_FLAG_NONE = 0,
QAT_CRYPTO_PROTO_FLAG_CCM = 1,
@@ -46,25 +40,6 @@ enum qat_crypto_proto_flag {
QAT_CRYPTO_PROTO_FLAG_ZUC = 4
};
-/*
- * Maximum number of SGL entries
- */
-#define QAT_SGL_MAX_NUMBER 16
-
-struct qat_alg_buf_list {
- uint64_t resrvd;
- uint32_t num_bufs;
- uint32_t num_mapped_bufs;
- struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
-} __rte_packed __rte_cache_aligned;
-
-struct qat_crypto_op_cookie {
- struct qat_alg_buf_list qat_sgl_list_src;
- struct qat_alg_buf_list qat_sgl_list_dst;
- rte_iova_t qat_sgl_src_phys_addr;
- rte_iova_t qat_sgl_dst_phys_addr;
-};
-
/* Common content descriptor */
struct qat_alg_cd {
struct icp_qat_hw_cipher_algo_blk cipher;
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
new file mode 100644
index 000000000..293b6f700
--- /dev/null
+++ b/drivers/crypto/qat/qat_common.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_COMMON_H_
+#define _QAT_COMMON_H_
+
+#include <stdint.h>
+
+/**< Intel(R) QAT Symmetric Crypto PMD device name */
+#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+
+/*
+ * Maximum number of SGL entries
+ */
+#define QAT_SGL_MAX_NUMBER 16
+
+/* Intel(R) QuickAssist Technology device generation is enumerated
+ * from one according to the generation of the device
+ */
+
+enum qat_device_gen {
+ QAT_GEN1 = 1,
+ QAT_GEN2,
+};
+
+/**< Common struct for scatter-gather list operations */
+struct qat_alg_buf {
+ uint32_t len;
+ uint32_t resrvd;
+ uint64_t addr;
+} __rte_packed;
+
+struct qat_alg_buf_list {
+ uint64_t resrvd;
+ uint32_t num_bufs;
+ uint32_t num_mapped_bufs;
+ struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
+} __rte_packed __rte_cache_aligned;
+
+struct qat_crypto_op_cookie {
+ struct qat_alg_buf_list qat_sgl_list_src;
+ struct qat_alg_buf_list qat_sgl_list_dst;
+ phys_addr_t qat_sgl_src_phys_addr;
+ phys_addr_t qat_sgl_dst_phys_addr;
+};
+
+#endif /* _QAT_QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 281a142b9..5190d25e5 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -8,11 +8,9 @@
#include <rte_cryptodev_pmd.h>
#include <rte_memzone.h>
+#include "qat_common.h"
#include "qat_crypto_capabilities.h"
-#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
-/**< Intel QAT Symmetric Crypto PMD device name */
-
/*
* This macro rounds up a number to a be a multiple of
* the alignment when the alignment is a power of 2
@@ -30,11 +28,6 @@
struct qat_session;
-enum qat_device_gen {
- QAT_GEN1 = 1,
- QAT_GEN2,
-};
-
/**
* Structure associated with each queue.
*/
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 02/38] crypto/qat: add qat device files
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 01/38] crypto/qat: add qat common header Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 03/38] crypto/qat: remove unused includes Tomasz Jozwiak
` (36 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
This commit adds new qat_device file.
Following objects were moved:
qat_crypto.h => qat_device.h
- struct qat_pmd_private
- uint8_t cryptodev_qat_driver_id
- int qat_crypto_sym_qp_release (EXTERN)
- int qat_dev_config()
- int qat_dev_start()
- void qat_dev_stop()
- int qat_dev_close()
- void qat_dev_info_get()
qat_crypto.c => qat_device.c
- int qat_dev_config()
- int qat_dev_start()
- void qat_dev_stop()
- int qat_dev_close()
- void qat_dev_info_get()
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 1 +
drivers/crypto/qat/meson.build | 3 +-
drivers/crypto/qat/qat_crypto.c | 51 -----------------------------
drivers/crypto/qat/qat_crypto.h | 22 +------------
drivers/crypto/qat/qat_device.c | 57 +++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_device.h | 36 +++++++++++++++++++++
6 files changed, 97 insertions(+), 73 deletions(-)
create mode 100644 drivers/crypto/qat/qat_device.c
create mode 100644 drivers/crypto/qat/qat_device.h
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 07266a5e9..6bdd11679 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -22,6 +22,7 @@ LDLIBS += -lrte_pci -lrte_bus_pci
# library source files
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_adf/qat_algs_build_desc.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 006cd6557..51630e31b 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -7,7 +7,8 @@ if not dep.found()
endif
sources = files('qat_crypto.c', 'qat_qp.c',
'qat_adf/qat_algs_build_desc.c',
- 'rte_qat_cryptodev.c')
+ 'rte_qat_cryptodev.c',
+ 'qat_device.c')
includes += include_directories('qat_adf')
deps += ['bus_pci']
ext_deps += dep
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index d9ce2a136..928a50475 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -1609,57 +1609,6 @@ static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
return data - mult;
}
-int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
- __rte_unused struct rte_cryptodev_config *config)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
-}
-
-int qat_dev_close(struct rte_cryptodev *dev)
-{
- int i, ret;
-
- PMD_INIT_FUNC_TRACE();
-
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_crypto_sym_qp_release(dev, i);
- if (ret < 0)
- return ret;
- }
-
- return 0;
-}
-
-void qat_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info)
-{
- struct qat_pmd_private *internals = dev->data->dev_private;
-
- PMD_INIT_FUNC_TRACE();
- if (info != NULL) {
- info->max_nb_queue_pairs =
- ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV;
- info->feature_flags = dev->feature_flags;
- info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = internals->max_nb_sessions;
- info->driver_id = cryptodev_qat_driver_id;
- info->pci_dev = RTE_DEV_TO_PCI(dev->device);
- }
-}
-
void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
{
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 5190d25e5..46f03ccde 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -9,6 +9,7 @@
#include <rte_memzone.h>
#include "qat_common.h"
+#include "qat_device.h"
#include "qat_crypto_capabilities.h"
/*
@@ -64,27 +65,6 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
} __rte_cache_aligned;
-/** private data structure for each QAT device */
-struct qat_pmd_private {
- unsigned max_nb_queue_pairs;
- /**< Max number of queue pairs supported by device */
- unsigned max_nb_sessions;
- /**< Max number of sessions supported by device */
- enum qat_device_gen qat_dev_gen;
- /**< QAT device generation */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
-};
-
-extern uint8_t cryptodev_qat_driver_id;
-
-int qat_dev_config(struct rte_cryptodev *dev,
- struct rte_cryptodev_config *config);
-int qat_dev_start(struct rte_cryptodev *dev);
-void qat_dev_stop(struct rte_cryptodev *dev);
-int qat_dev_close(struct rte_cryptodev *dev);
-void qat_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info);
-
void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
new file mode 100644
index 000000000..c2bf9b7a7
--- /dev/null
+++ b/drivers/crypto/qat/qat_device.c
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_device.h"
+#include "adf_transport_access_macros.h"
+
+int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused struct rte_cryptodev_config *config)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+}
+
+int qat_dev_close(struct rte_cryptodev *dev)
+{
+ int i, ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ ret = qat_crypto_sym_qp_release(dev, i);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+void qat_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info)
+{
+ struct qat_pmd_private *internals = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+ if (info != NULL) {
+ info->max_nb_queue_pairs =
+ ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV;
+ info->feature_flags = dev->feature_flags;
+ info->capabilities = internals->qat_dev_capabilities;
+ info->sym.max_nb_sessions = internals->max_nb_sessions;
+ info->driver_id = cryptodev_qat_driver_id;
+ info->pci_dev = RTE_DEV_TO_PCI(dev->device);
+ }
+}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
new file mode 100644
index 000000000..5c48fdb93
--- /dev/null
+++ b/drivers/crypto/qat/qat_device.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_DEVICE_H_
+#define _QAT_DEVICE_H_
+
+#include <rte_cryptodev_pmd.h>
+#include <rte_bus_pci.h>
+#include "qat_common.h"
+#include "qat_logs.h"
+
+extern uint8_t cryptodev_qat_driver_id;
+
+extern int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+ uint16_t queue_pair_id);
+
+/** private data structure for each QAT device */
+struct qat_pmd_private {
+ unsigned int max_nb_queue_pairs;
+ /**< Max number of queue pairs supported by device */
+ unsigned int max_nb_sessions;
+ /**< Max number of sessions supported by device */
+ enum qat_device_gen qat_dev_gen;
+ /**< QAT device generation */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+};
+
+int qat_dev_config(struct rte_cryptodev *dev,
+ struct rte_cryptodev_config *config);
+int qat_dev_start(struct rte_cryptodev *dev);
+void qat_dev_stop(struct rte_cryptodev *dev);
+int qat_dev_close(struct rte_cryptodev *dev);
+void qat_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info);
+
+#endif /* _QAT_DEVICE_H_ */
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 03/38] crypto/qat: remove unused includes
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 01/38] crypto/qat: add qat common header Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 02/38] crypto/qat: add qat device files Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 04/38] crypto/qat: add symmetric session file Tomasz Jozwiak
` (35 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
This commit removes unused includes from qat_crypto.c
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/qat_crypto.c | 22 ----------------------
1 file changed, 22 deletions(-)
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index 928a50475..7f2c2c86b 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -2,30 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
-#include <stdio.h>
-#include <stdlib.h>
-#include <strings.h>
-#include <string.h>
-#include <inttypes.h>
-#include <errno.h>
-#include <sys/queue.h>
-#include <stdarg.h>
-
-#include <rte_common.h>
-#include <rte_log.h>
-#include <rte_debug.h>
-#include <rte_memory.h>
-#include <rte_tailq.h>
-#include <rte_malloc.h>
-#include <rte_launch.h>
-#include <rte_eal.h>
-#include <rte_per_lcore.h>
-#include <rte_lcore.h>
-#include <rte_branch_prediction.h>
#include <rte_mempool.h>
#include <rte_mbuf.h>
-#include <rte_string_fns.h>
-#include <rte_spinlock.h>
#include <rte_hexdump.h>
#include <rte_crypto_sym.h>
#include <rte_byteorder.h>
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 04/38] crypto/qat: add symmetric session file
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (2 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 03/38] crypto/qat: remove unused includes Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 05/38] crypto/qat: change filename crypto to sym Tomasz Jozwiak
` (34 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
This commit adds qat_sym_session.c/h files and moves objects
from qat_algs_build_desc and qat_algs.h
Following objects were moved:
qat_adf/qat_algs_build_desc.c => qat_sym_session.c
- all objects -
qat_adf/qat_algs.h => qat_sym_session.h
- enum qat_crypto_proto_flag
- struct qat_alg_cd
- struct qat_session
- int qat_get_inter_state_size()
- int qat_alg_aead_session_create_content_desc_cipher()
- int qat_alg_aead_session_create_content_desc_auth()
- void qat_alg_init_common_hdr()
- int qat_alg_validate_aes_key()
- int qat_alg_validate_aes_docsisbpi_key()
- int qat_alg_validate_snow3g_key()
- int qat_alg_validate_kasumi_key()
- int qat_alg_validate_3des_key()
- int qat_alg_validate_des_key()
- int qat_cipher_get_block_size()
- int qat_alg_validate_zuc_key()
-- all macros
qat_crypto.h => qat_sym_session.h
int qat_crypto_sym_configure_session()
int qat_crypto_set_session_parameters()
int qat_crypto_sym_configure_session_aead()
int qat_crypto_sym_configure_session_cipher()
int qat_crypto_sym_configure_session_auth()
int qat_alg_aead_session_create_content_desc_cipher()
int qat_alg_aead_session_create_content_desc_auth()
static struct rte_crypto_auth_xform qat_get_auth_xform()
static struct rte_crypto_cipher_xform qat_get_cipher_xform()
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/Makefile | 2 +-
drivers/crypto/qat/meson.build | 2 +-
drivers/crypto/qat/qat_crypto.c | 704 +----------------
drivers/crypto/qat/qat_crypto.h | 36 -
drivers/crypto/qat/qat_qp.c | 1 -
...at_algs_build_desc.c => qat_sym_session.c} | 728 +++++++++++++++++-
.../{qat_adf/qat_algs.h => qat_sym_session.h} | 63 +-
drivers/crypto/qat/rte_qat_cryptodev.c | 1 +
8 files changed, 775 insertions(+), 762 deletions(-)
rename drivers/crypto/qat/{qat_adf/qat_algs_build_desc.c => qat_sym_session.c} (61%)
rename drivers/crypto/qat/{qat_adf/qat_algs.h => qat_sym_session.h} (66%)
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 6bdd11679..c63c1515e 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -24,7 +24,7 @@ LDLIBS += -lrte_pci -lrte_bus_pci
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_adf/qat_algs_build_desc.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
# export include files
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 51630e31b..be4282a83 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -6,7 +6,7 @@ if not dep.found()
build = false
endif
sources = files('qat_crypto.c', 'qat_qp.c',
- 'qat_adf/qat_algs_build_desc.c',
+ 'qat_sym_session.c',
'rte_qat_cryptodev.c',
'qat_device.c')
includes += include_directories('qat_adf')
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_crypto.c
index 7f2c2c86b..96a1b78f0 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_crypto.c
@@ -13,7 +13,7 @@
#include <openssl/evp.h>
#include "qat_logs.h"
-#include "qat_algs.h"
+#include "qat_sym_session.h"
#include "qat_crypto.h"
#include "adf_transport_access_macros.h"
@@ -23,46 +23,6 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-static int
-qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
- struct qat_pmd_private *internals) {
- int i = 0;
- const struct rte_cryptodev_capabilities *capability;
-
- while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
- RTE_CRYPTO_OP_TYPE_UNDEFINED) {
- if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
- continue;
-
- if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_CIPHER)
- continue;
-
- if (capability->sym.cipher.algo == algo)
- return 1;
- }
- return 0;
-}
-
-static int
-qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
- struct qat_pmd_private *internals) {
- int i = 0;
- const struct rte_cryptodev_capabilities *capability;
-
- while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
- RTE_CRYPTO_OP_TYPE_UNDEFINED) {
- if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
- continue;
-
- if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_AUTH)
- continue;
-
- if (capability->sym.auth.algo == algo)
- return 1;
- }
- return 0;
-}
-
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -124,49 +84,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
/** Creates a context in either AES or DES in ECB mode
* Depends on openssl libcrypto
*/
-static int
-bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
- enum rte_crypto_cipher_operation direction __rte_unused,
- uint8_t *key, void **ctx)
-{
- const EVP_CIPHER *algo = NULL;
- int ret;
- *ctx = EVP_CIPHER_CTX_new();
-
- if (*ctx == NULL) {
- ret = -ENOMEM;
- goto ctx_init_err;
- }
-
- if (cryptodev_algo == RTE_CRYPTO_CIPHER_DES_DOCSISBPI)
- algo = EVP_des_ecb();
- else
- algo = EVP_aes_128_ecb();
-
- /* IV will be ECB encrypted whether direction is encrypt or decrypt*/
- if (EVP_EncryptInit_ex(*ctx, algo, NULL, key, 0) != 1) {
- ret = -EINVAL;
- goto ctx_init_err;
- }
-
- return 0;
-
-ctx_init_err:
- if (*ctx != NULL)
- EVP_CIPHER_CTX_free(*ctx);
- return ret;
-}
-
-/** Frees a context previously created
- * Depends on openssl libcrypto
- */
-static void
-bpi_cipher_ctx_free(void *bpi_ctx)
-{
- if (bpi_ctx != NULL)
- EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
-}
-
static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
@@ -174,625 +91,6 @@ static inline int
qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
-void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
- struct rte_cryptodev_sym_session *sess)
-{
- PMD_INIT_FUNC_TRACE();
- uint8_t index = dev->driver_id;
- void *sess_priv = get_session_private_data(sess, index);
- struct qat_session *s = (struct qat_session *)sess_priv;
-
- if (sess_priv) {
- if (s->bpi_ctx)
- bpi_cipher_ctx_free(s->bpi_ctx);
- memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
- struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
- set_session_private_data(sess, index, NULL);
- rte_mempool_put(sess_mp, sess_priv);
- }
-}
-
-static int
-qat_get_cmd_id(const struct rte_crypto_sym_xform *xform)
-{
- /* Cipher Only */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
- return ICP_QAT_FW_LA_CMD_CIPHER;
-
- /* Authentication Only */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
- return ICP_QAT_FW_LA_CMD_AUTH;
-
- /* AEAD */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- /* AES-GCM and AES-CCM works with different direction
- * GCM first encrypts and generate hash where AES-CCM
- * first generate hash and encrypts. Similar relation
- * applies to decryption.
- */
- if (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT)
- if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- else
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- else
- if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- else
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- }
-
- if (xform->next == NULL)
- return -1;
-
- /* Cipher then Authenticate */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
- xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH)
- return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
-
- /* Authenticate then Cipher */
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
- xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
- return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
-
- return -1;
-}
-
-static struct rte_crypto_auth_xform *
-qat_get_auth_xform(struct rte_crypto_sym_xform *xform)
-{
- do {
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH)
- return &xform->auth;
-
- xform = xform->next;
- } while (xform);
-
- return NULL;
-}
-
-static struct rte_crypto_cipher_xform *
-qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
-{
- do {
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
- return &xform->cipher;
-
- xform = xform->next;
- } while (xform);
-
- return NULL;
-}
-
-int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct qat_pmd_private *internals = dev->data->dev_private;
- struct rte_crypto_cipher_xform *cipher_xform = NULL;
- int ret;
-
- /* Get cipher xform from crypto xform chain */
- cipher_xform = qat_get_cipher_xform(xform);
-
- session->cipher_iv.offset = cipher_xform->iv.offset;
- session->cipher_iv.length = cipher_xform->iv.length;
-
- switch (cipher_xform->algo) {
- case RTE_CRYPTO_CIPHER_AES_CBC:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_AES_CTR:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- break;
- case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
- if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_NULL:
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_KASUMI_F8:
- if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_CBC:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_DES_CBC:
- if (qat_alg_validate_des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_CTR:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- break;
- case RTE_CRYPTO_CIPHER_DES_DOCSISBPI:
- ret = bpi_cipher_ctx_init(
- cipher_xform->algo,
- cipher_xform->op,
- cipher_xform->key.data,
- &session->bpi_ctx);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
- goto error_out;
- }
- if (qat_alg_validate_des_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_AES_DOCSISBPI:
- ret = bpi_cipher_ctx_init(
- cipher_xform->algo,
- cipher_xform->op,
- cipher_xform->key.data,
- &session->bpi_ctx);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
- goto error_out;
- }
- if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
- break;
- case RTE_CRYPTO_CIPHER_ZUC_EEA3:
- if (!qat_is_cipher_alg_supported(
- cipher_xform->algo, internals)) {
- PMD_DRV_LOG(ERR, "%s not supported on this device",
- rte_crypto_cipher_algorithm_strings
- [cipher_xform->algo]);
- ret = -ENOTSUP;
- goto error_out;
- }
- if (qat_alg_validate_zuc_key(cipher_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
- ret = -EINVAL;
- goto error_out;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
- break;
- case RTE_CRYPTO_CIPHER_3DES_ECB:
- case RTE_CRYPTO_CIPHER_AES_ECB:
- case RTE_CRYPTO_CIPHER_AES_F8:
- case RTE_CRYPTO_CIPHER_AES_XTS:
- case RTE_CRYPTO_CIPHER_ARC4:
- PMD_DRV_LOG(ERR, "Crypto QAT PMD: Unsupported Cipher alg %u",
- cipher_xform->algo);
- ret = -ENOTSUP;
- goto error_out;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined Cipher specified %u\n",
- cipher_xform->algo);
- ret = -EINVAL;
- goto error_out;
- }
-
- if (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- else
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- cipher_xform->key.data,
- cipher_xform->key.length)) {
- ret = -EINVAL;
- goto error_out;
- }
-
- return 0;
-
-error_out:
- if (session->bpi_ctx) {
- bpi_cipher_ctx_free(session->bpi_ctx);
- session->bpi_ctx = NULL;
- }
- return ret;
-}
-
-int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct rte_cryptodev_sym_session *sess,
- struct rte_mempool *mempool)
-{
- void *sess_private_data;
- int ret;
-
- if (rte_mempool_get(mempool, &sess_private_data)) {
- CDEV_LOG_ERR(
- "Couldn't get object from session mempool");
- return -ENOMEM;
- }
-
- ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
- if (ret != 0) {
- PMD_DRV_LOG(ERR, "Crypto QAT PMD: failed to configure "
- "session parameters");
-
- /* Return session to mempool */
- rte_mempool_put(mempool, sess_private_data);
- return ret;
- }
-
- set_session_private_data(sess, dev->driver_id,
- sess_private_data);
-
- return 0;
-}
-
-int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform, void *session_private)
-{
- struct qat_session *session = session_private;
- int ret;
-
- int qat_cmd_id;
- PMD_INIT_FUNC_TRACE();
-
- /* Set context descriptor physical address */
- session->cd_paddr = rte_mempool_virt2iova(session) +
- offsetof(struct qat_session, cd);
-
- session->min_qat_dev_gen = QAT_GEN1;
-
- /* Get requested QAT command id */
- qat_cmd_id = qat_get_cmd_id(xform);
- if (qat_cmd_id < 0 || qat_cmd_id >= ICP_QAT_FW_LA_CMD_DELIMITER) {
- PMD_DRV_LOG(ERR, "Unsupported xform chain requested");
- return -ENOTSUP;
- }
- session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
- switch (session->qat_cmd) {
- case ICP_QAT_FW_LA_CMD_CIPHER:
- ret = qat_crypto_sym_configure_session_cipher(dev, xform, session);
- if (ret < 0)
- return ret;
- break;
- case ICP_QAT_FW_LA_CMD_AUTH:
- ret = qat_crypto_sym_configure_session_auth(dev, xform, session);
- if (ret < 0)
- return ret;
- break;
- case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
- session);
- if (ret < 0)
- return ret;
- } else {
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
- if (ret < 0)
- return ret;
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
- if (ret < 0)
- return ret;
- }
- break;
- case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
- session);
- if (ret < 0)
- return ret;
- } else {
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
- if (ret < 0)
- return ret;
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
- if (ret < 0)
- return ret;
- }
- break;
- case ICP_QAT_FW_LA_CMD_TRNG_GET_RANDOM:
- case ICP_QAT_FW_LA_CMD_TRNG_TEST:
- case ICP_QAT_FW_LA_CMD_SSL3_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_TLS_V1_1_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_TLS_V1_2_KEY_DERIVE:
- case ICP_QAT_FW_LA_CMD_MGF1:
- case ICP_QAT_FW_LA_CMD_AUTH_PRE_COMP:
- case ICP_QAT_FW_LA_CMD_CIPHER_PRE_COMP:
- case ICP_QAT_FW_LA_CMD_DELIMITER:
- PMD_DRV_LOG(ERR, "Unsupported Service %u",
- session->qat_cmd);
- return -ENOTSUP;
- default:
- PMD_DRV_LOG(ERR, "Unsupported Service %u",
- session->qat_cmd);
- return -ENOTSUP;
- }
-
- return 0;
-}
-
-int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct rte_crypto_auth_xform *auth_xform = NULL;
- struct qat_pmd_private *internals = dev->data->dev_private;
- auth_xform = qat_get_auth_xform(xform);
- uint8_t *key_data = auth_xform->key.data;
- uint8_t key_length = auth_xform->key.length;
-
- switch (auth_xform->algo) {
- case RTE_CRYPTO_AUTH_SHA1_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA1;
- break;
- case RTE_CRYPTO_AUTH_SHA224_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA224;
- break;
- case RTE_CRYPTO_AUTH_SHA256_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA256;
- break;
- case RTE_CRYPTO_AUTH_SHA384_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA384;
- break;
- case RTE_CRYPTO_AUTH_SHA512_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA512;
- break;
- case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
- break;
- case RTE_CRYPTO_AUTH_AES_GMAC:
- if (qat_alg_validate_aes_key(auth_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
-
- break;
- case RTE_CRYPTO_AUTH_SNOW3G_UIA2:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SNOW_3G_UIA2;
- break;
- case RTE_CRYPTO_AUTH_MD5_HMAC:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_MD5;
- break;
- case RTE_CRYPTO_AUTH_NULL:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_NULL;
- break;
- case RTE_CRYPTO_AUTH_KASUMI_F9:
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_KASUMI_F9;
- break;
- case RTE_CRYPTO_AUTH_ZUC_EIA3:
- if (!qat_is_auth_alg_supported(auth_xform->algo, internals)) {
- PMD_DRV_LOG(ERR, "%s not supported on this device",
- rte_crypto_auth_algorithm_strings
- [auth_xform->algo]);
- return -ENOTSUP;
- }
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_ZUC_3G_128_EIA3;
- break;
- case RTE_CRYPTO_AUTH_SHA1:
- case RTE_CRYPTO_AUTH_SHA256:
- case RTE_CRYPTO_AUTH_SHA512:
- case RTE_CRYPTO_AUTH_SHA224:
- case RTE_CRYPTO_AUTH_SHA384:
- case RTE_CRYPTO_AUTH_MD5:
- case RTE_CRYPTO_AUTH_AES_CMAC:
- case RTE_CRYPTO_AUTH_AES_CBC_MAC:
- PMD_DRV_LOG(ERR, "Crypto: Unsupported hash alg %u",
- auth_xform->algo);
- return -ENOTSUP;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined Hash algo %u specified",
- auth_xform->algo);
- return -EINVAL;
- }
-
- session->auth_iv.offset = auth_xform->iv.offset;
- session->auth_iv.length = auth_xform->iv.length;
-
- if (auth_xform->algo == RTE_CRYPTO_AUTH_AES_GMAC) {
- if (auth_xform->op == RTE_CRYPTO_AUTH_OP_GENERATE) {
- session->qat_cmd = ICP_QAT_FW_LA_CMD_CIPHER_HASH;
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- /*
- * It needs to create cipher desc content first,
- * then authentication
- */
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- auth_xform->key.data,
- auth_xform->key.length))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
- } else {
- session->qat_cmd = ICP_QAT_FW_LA_CMD_HASH_CIPHER;
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- /*
- * It needs to create authentication desc content first,
- * then cipher
- */
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- auth_xform->key.data,
- auth_xform->key.length))
- return -EINVAL;
- }
- /* Restore to authentication only only */
- session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
- } else {
- if (qat_alg_aead_session_create_content_desc_auth(session,
- key_data,
- key_length,
- 0,
- auth_xform->digest_length,
- auth_xform->op))
- return -EINVAL;
- }
-
- session->digest_length = auth_xform->digest_length;
- return 0;
-}
-
-int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
-{
- struct rte_crypto_aead_xform *aead_xform = &xform->aead;
- enum rte_crypto_auth_operation crypto_operation;
-
- /*
- * Store AEAD IV parameters as cipher IV,
- * to avoid unnecessary memory usage
- */
- session->cipher_iv.offset = xform->aead.iv.offset;
- session->cipher_iv.length = xform->aead.iv.length;
-
- switch (aead_xform->algo) {
- case RTE_CRYPTO_AEAD_AES_GCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
- break;
- case RTE_CRYPTO_AEAD_AES_CCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
- &session->qat_cipher_alg) != 0) {
- PMD_DRV_LOG(ERR, "Invalid AES key size");
- return -EINVAL;
- }
- session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
- session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC;
- break;
- default:
- PMD_DRV_LOG(ERR, "Crypto: Undefined AEAD specified %u\n",
- aead_xform->algo);
- return -EINVAL;
- }
-
- if ((aead_xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT &&
- aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM) ||
- (aead_xform->op == RTE_CRYPTO_AEAD_OP_DECRYPT &&
- aead_xform->algo == RTE_CRYPTO_AEAD_AES_CCM)) {
- session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
- /*
- * It needs to create cipher desc content first,
- * then authentication
- */
-
- crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
- RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- aead_xform->key.data,
- aead_xform->key.length))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- aead_xform->key.data,
- aead_xform->key.length,
- aead_xform->aad_length,
- aead_xform->digest_length,
- crypto_operation))
- return -EINVAL;
- } else {
- session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- /*
- * It needs to create authentication desc content first,
- * then cipher
- */
-
- crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
- RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
-
- if (qat_alg_aead_session_create_content_desc_auth(session,
- aead_xform->key.data,
- aead_xform->key.length,
- aead_xform->aad_length,
- aead_xform->digest_length,
- crypto_operation))
- return -EINVAL;
-
- if (qat_alg_aead_session_create_content_desc_cipher(session,
- aead_xform->key.data,
- aead_xform->key.length))
- return -EINVAL;
- }
-
- session->digest_length = aead_xform->digest_length;
- return 0;
-}
-
-unsigned qat_crypto_sym_get_session_private_size(
- struct rte_cryptodev *dev __rte_unused)
-{
- return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
-}
-
static inline uint32_t
qat_bpicipher_preprocess(struct qat_session *ctx,
struct rte_crypto_op *op)
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_crypto.h
index 46f03ccde..e3873171c 100644
--- a/drivers/crypto/qat/qat_crypto.h
+++ b/drivers/crypto/qat/qat_crypto.h
@@ -75,42 +75,6 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-int
-qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
- unsigned nb_objs, unsigned obj_cache_size, int socket_id);
-
-extern unsigned
-qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
-
-extern int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct rte_cryptodev_sym_session *sess,
- struct rte_mempool *mempool);
-
-
-int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform, void *session_private);
-
-int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
- struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
-
-
-extern void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
- struct rte_cryptodev_sym_session *session);
extern uint16_t
qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 7fea10c76..ee3b30a36 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -14,7 +14,6 @@
#include "qat_logs.h"
#include "qat_crypto.h"
-#include "qat_algs.h"
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
diff --git a/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c b/drivers/crypto/qat/qat_sym_session.c
similarity index 61%
rename from drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
rename to drivers/crypto/qat/qat_sym_session.c
index c87ed40fe..5caac5a6f 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -9,13 +9,724 @@
#include <rte_malloc.h>
#include <rte_crypto_sym.h>
-#include "../qat_logs.h"
+#include "qat_logs.h"
+#include "qat_device.h"
#include <openssl/sha.h> /* Needed to calculate pre-compute values */
#include <openssl/aes.h> /* Needed to calculate pre-compute values */
#include <openssl/md5.h> /* Needed to calculate pre-compute values */
+#include <openssl/evp.h>
-#include "qat_algs.h"
+#include "qat_sym_session.h"
+
+/** Frees a context previously created
+ * Depends on openssl libcrypto
+ */
+static void
+bpi_cipher_ctx_free(void *bpi_ctx)
+{
+ if (bpi_ctx != NULL)
+ EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
+}
+
+static int
+bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
+ enum rte_crypto_cipher_operation direction __rte_unused,
+ uint8_t *key, void **ctx)
+{
+ const EVP_CIPHER *algo = NULL;
+ int ret;
+ *ctx = EVP_CIPHER_CTX_new();
+
+ if (*ctx == NULL) {
+ ret = -ENOMEM;
+ goto ctx_init_err;
+ }
+
+ if (cryptodev_algo == RTE_CRYPTO_CIPHER_DES_DOCSISBPI)
+ algo = EVP_des_ecb();
+ else
+ algo = EVP_aes_128_ecb();
+
+ /* IV will be ECB encrypted whether direction is encrypt or decrypt*/
+ if (EVP_EncryptInit_ex(*ctx, algo, NULL, key, 0) != 1) {
+ ret = -EINVAL;
+ goto ctx_init_err;
+ }
+
+ return 0;
+
+ctx_init_err:
+ if (*ctx != NULL)
+ EVP_CIPHER_CTX_free(*ctx);
+ return ret;
+}
+
+static int
+qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
+ struct qat_pmd_private *internals)
+{
+ int i = 0;
+ const struct rte_cryptodev_capabilities *capability;
+
+ while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
+ RTE_CRYPTO_OP_TYPE_UNDEFINED) {
+ if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
+ continue;
+
+ if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_CIPHER)
+ continue;
+
+ if (capability->sym.cipher.algo == algo)
+ return 1;
+ }
+ return 0;
+}
+
+static int
+qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
+ struct qat_pmd_private *internals)
+{
+ int i = 0;
+ const struct rte_cryptodev_capabilities *capability;
+
+ while ((capability = &(internals->qat_dev_capabilities[i++]))->op !=
+ RTE_CRYPTO_OP_TYPE_UNDEFINED) {
+ if (capability->op != RTE_CRYPTO_OP_TYPE_SYMMETRIC)
+ continue;
+
+ if (capability->sym.xform_type != RTE_CRYPTO_SYM_XFORM_AUTH)
+ continue;
+
+ if (capability->sym.auth.algo == algo)
+ return 1;
+ }
+ return 0;
+}
+
+void
+qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess)
+{
+ PMD_INIT_FUNC_TRACE();
+ uint8_t index = dev->driver_id;
+ void *sess_priv = get_session_private_data(sess, index);
+ struct qat_session *s = (struct qat_session *)sess_priv;
+
+ if (sess_priv) {
+ if (s->bpi_ctx)
+ bpi_cipher_ctx_free(s->bpi_ctx);
+ memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
+ struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
+
+ set_session_private_data(sess, index, NULL);
+ rte_mempool_put(sess_mp, sess_priv);
+ }
+}
+
+static int
+qat_get_cmd_id(const struct rte_crypto_sym_xform *xform)
+{
+ /* Cipher Only */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL)
+ return ICP_QAT_FW_LA_CMD_CIPHER;
+
+ /* Authentication Only */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && xform->next == NULL)
+ return ICP_QAT_FW_LA_CMD_AUTH;
+
+ /* AEAD */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ /* AES-GCM and AES-CCM works with different direction
+ * GCM first encrypts and generate hash where AES-CCM
+ * first generate hash and encrypts. Similar relation
+ * applies to decryption.
+ */
+ if (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT)
+ if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ else
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ else
+ if (xform->aead.algo == RTE_CRYPTO_AEAD_AES_GCM)
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ else
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ }
+
+ if (xform->next == NULL)
+ return -1;
+
+ /* Cipher then Authenticate */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
+ xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH)
+ return ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+
+ /* Authenticate then Cipher */
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
+ xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
+ return ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+
+ return -1;
+}
+
+static struct rte_crypto_auth_xform *
+qat_get_auth_xform(struct rte_crypto_sym_xform *xform)
+{
+ do {
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH)
+ return &xform->auth;
+
+ xform = xform->next;
+ } while (xform);
+
+ return NULL;
+}
+
+static struct rte_crypto_cipher_xform *
+qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
+{
+ do {
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
+ return &xform->cipher;
+
+ xform = xform->next;
+ } while (xform);
+
+ return NULL;
+}
+
+int
+qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct qat_pmd_private *internals = dev->data->dev_private;
+ struct rte_crypto_cipher_xform *cipher_xform = NULL;
+ int ret;
+
+ /* Get cipher xform from crypto xform chain */
+ cipher_xform = qat_get_cipher_xform(xform);
+
+ session->cipher_iv.offset = cipher_xform->iv.offset;
+ session->cipher_iv.length = cipher_xform->iv.length;
+
+ switch (cipher_xform->algo) {
+ case RTE_CRYPTO_CIPHER_AES_CBC:
+ if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_AES_CTR:
+ if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
+ if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_NULL:
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_KASUMI_F8:
+ if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_CBC:
+ if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_DES_CBC:
+ if (qat_alg_validate_des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_CTR:
+ if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_DES_DOCSISBPI:
+ ret = bpi_cipher_ctx_init(
+ cipher_xform->algo,
+ cipher_xform->op,
+ cipher_xform->key.data,
+ &session->bpi_ctx);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
+ goto error_out;
+ }
+ if (qat_alg_validate_des_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_AES_DOCSISBPI:
+ ret = bpi_cipher_ctx_init(
+ cipher_xform->algo,
+ cipher_xform->op,
+ cipher_xform->key.data,
+ &session->bpi_ctx);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
+ goto error_out;
+ }
+ if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_ZUC_EEA3:
+ if (!qat_is_cipher_alg_supported(
+ cipher_xform->algo, internals)) {
+ PMD_DRV_LOG(ERR, "%s not supported on this device",
+ rte_crypto_cipher_algorithm_strings
+ [cipher_xform->algo]);
+ ret = -ENOTSUP;
+ goto error_out;
+ }
+ if (qat_alg_validate_zuc_key(cipher_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
+ ret = -EINVAL;
+ goto error_out;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
+ break;
+ case RTE_CRYPTO_CIPHER_3DES_ECB:
+ case RTE_CRYPTO_CIPHER_AES_ECB:
+ case RTE_CRYPTO_CIPHER_AES_F8:
+ case RTE_CRYPTO_CIPHER_AES_XTS:
+ case RTE_CRYPTO_CIPHER_ARC4:
+ PMD_DRV_LOG(ERR, "Crypto QAT PMD: Unsupported Cipher alg %u",
+ cipher_xform->algo);
+ ret = -ENOTSUP;
+ goto error_out;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined Cipher specified %u\n",
+ cipher_xform->algo);
+ ret = -EINVAL;
+ goto error_out;
+ }
+
+ if (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ else
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ cipher_xform->key.data,
+ cipher_xform->key.length)) {
+ ret = -EINVAL;
+ goto error_out;
+ }
+
+ return 0;
+
+error_out:
+ if (session->bpi_ctx) {
+ bpi_cipher_ctx_free(session->bpi_ctx);
+ session->bpi_ctx = NULL;
+ }
+ return ret;
+}
+
+int
+qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct rte_cryptodev_sym_session *sess,
+ struct rte_mempool *mempool)
+{
+ void *sess_private_data;
+ int ret;
+
+ if (rte_mempool_get(mempool, &sess_private_data)) {
+ CDEV_LOG_ERR(
+ "Couldn't get object from session mempool");
+ return -ENOMEM;
+ }
+
+ ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR,
+ "Crypto QAT PMD: failed to configure session parameters");
+
+ /* Return session to mempool */
+ rte_mempool_put(mempool, sess_private_data);
+ return ret;
+ }
+
+ set_session_private_data(sess, dev->driver_id,
+ sess_private_data);
+
+ return 0;
+}
+
+int
+qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform, void *session_private)
+{
+ struct qat_session *session = session_private;
+ int ret;
+ int qat_cmd_id;
+
+ PMD_INIT_FUNC_TRACE();
+
+ /* Set context descriptor physical address */
+ session->cd_paddr = rte_mempool_virt2iova(session) +
+ offsetof(struct qat_session, cd);
+
+ session->min_qat_dev_gen = QAT_GEN1;
+
+ /* Get requested QAT command id */
+ qat_cmd_id = qat_get_cmd_id(xform);
+ if (qat_cmd_id < 0 || qat_cmd_id >= ICP_QAT_FW_LA_CMD_DELIMITER) {
+ PMD_DRV_LOG(ERR, "Unsupported xform chain requested");
+ return -ENOTSUP;
+ }
+ session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
+ switch (session->qat_cmd) {
+ case ICP_QAT_FW_LA_CMD_CIPHER:
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ break;
+ case ICP_QAT_FW_LA_CMD_AUTH:
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ break;
+ case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ ret = qat_crypto_sym_configure_session_aead(xform,
+ session);
+ if (ret < 0)
+ return ret;
+ } else {
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ }
+ break;
+ case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
+ ret = qat_crypto_sym_configure_session_aead(xform,
+ session);
+ if (ret < 0)
+ return ret;
+ } else {
+ ret = qat_crypto_sym_configure_session_auth(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ ret = qat_crypto_sym_configure_session_cipher(dev,
+ xform, session);
+ if (ret < 0)
+ return ret;
+ }
+ break;
+ case ICP_QAT_FW_LA_CMD_TRNG_GET_RANDOM:
+ case ICP_QAT_FW_LA_CMD_TRNG_TEST:
+ case ICP_QAT_FW_LA_CMD_SSL3_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_TLS_V1_1_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_TLS_V1_2_KEY_DERIVE:
+ case ICP_QAT_FW_LA_CMD_MGF1:
+ case ICP_QAT_FW_LA_CMD_AUTH_PRE_COMP:
+ case ICP_QAT_FW_LA_CMD_CIPHER_PRE_COMP:
+ case ICP_QAT_FW_LA_CMD_DELIMITER:
+ PMD_DRV_LOG(ERR, "Unsupported Service %u",
+ session->qat_cmd);
+ return -ENOTSUP;
+ default:
+ PMD_DRV_LOG(ERR, "Unsupported Service %u",
+ session->qat_cmd);
+ return -ENOTSUP;
+ }
+
+ return 0;
+}
+
+int
+qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
+ struct qat_pmd_private *internals = dev->data->dev_private;
+ uint8_t *key_data = auth_xform->key.data;
+ uint8_t key_length = auth_xform->key.length;
+
+ switch (auth_xform->algo) {
+ case RTE_CRYPTO_AUTH_SHA1_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA1;
+ break;
+ case RTE_CRYPTO_AUTH_SHA224_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA224;
+ break;
+ case RTE_CRYPTO_AUTH_SHA256_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA256;
+ break;
+ case RTE_CRYPTO_AUTH_SHA384_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA384;
+ break;
+ case RTE_CRYPTO_AUTH_SHA512_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SHA512;
+ break;
+ case RTE_CRYPTO_AUTH_AES_XCBC_MAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
+ break;
+ case RTE_CRYPTO_AUTH_AES_GMAC:
+ if (qat_alg_validate_aes_key(auth_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
+
+ break;
+ case RTE_CRYPTO_AUTH_SNOW3G_UIA2:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_SNOW_3G_UIA2;
+ break;
+ case RTE_CRYPTO_AUTH_MD5_HMAC:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_MD5;
+ break;
+ case RTE_CRYPTO_AUTH_NULL:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_NULL;
+ break;
+ case RTE_CRYPTO_AUTH_KASUMI_F9:
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_KASUMI_F9;
+ break;
+ case RTE_CRYPTO_AUTH_ZUC_EIA3:
+ if (!qat_is_auth_alg_supported(auth_xform->algo, internals)) {
+ PMD_DRV_LOG(ERR, "%s not supported on this device",
+ rte_crypto_auth_algorithm_strings
+ [auth_xform->algo]);
+ return -ENOTSUP;
+ }
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_ZUC_3G_128_EIA3;
+ break;
+ case RTE_CRYPTO_AUTH_SHA1:
+ case RTE_CRYPTO_AUTH_SHA256:
+ case RTE_CRYPTO_AUTH_SHA512:
+ case RTE_CRYPTO_AUTH_SHA224:
+ case RTE_CRYPTO_AUTH_SHA384:
+ case RTE_CRYPTO_AUTH_MD5:
+ case RTE_CRYPTO_AUTH_AES_CMAC:
+ case RTE_CRYPTO_AUTH_AES_CBC_MAC:
+ PMD_DRV_LOG(ERR, "Crypto: Unsupported hash alg %u",
+ auth_xform->algo);
+ return -ENOTSUP;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined Hash algo %u specified",
+ auth_xform->algo);
+ return -EINVAL;
+ }
+
+ session->auth_iv.offset = auth_xform->iv.offset;
+ session->auth_iv.length = auth_xform->iv.length;
+
+ if (auth_xform->algo == RTE_CRYPTO_AUTH_AES_GMAC) {
+ if (auth_xform->op == RTE_CRYPTO_AUTH_OP_GENERATE) {
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_CIPHER_HASH;
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ /*
+ * It needs to create cipher desc content first,
+ * then authentication
+ */
+ if (qat_alg_aead_session_create_content_desc_cipher(
+ session,
+ auth_xform->key.data,
+ auth_xform->key.length))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_auth(
+ session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+ } else {
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_HASH_CIPHER;
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+ /*
+ * It needs to create authentication desc content first,
+ * then cipher
+ */
+ if (qat_alg_aead_session_create_content_desc_auth(
+ session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(
+ session,
+ auth_xform->key.data,
+ auth_xform->key.length))
+ return -EINVAL;
+ }
+ /* Restore to authentication only only */
+ session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
+ } else {
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ key_data,
+ key_length,
+ 0,
+ auth_xform->digest_length,
+ auth_xform->op))
+ return -EINVAL;
+ }
+
+ session->digest_length = auth_xform->digest_length;
+ return 0;
+}
+
+int
+qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+ struct qat_session *session)
+{
+ struct rte_crypto_aead_xform *aead_xform = &xform->aead;
+ enum rte_crypto_auth_operation crypto_operation;
+
+ /*
+ * Store AEAD IV parameters as cipher IV,
+ * to avoid unnecessary memory usage
+ */
+ session->cipher_iv.offset = xform->aead.iv.offset;
+ session->cipher_iv.length = xform->aead.iv.length;
+
+ switch (aead_xform->algo) {
+ case RTE_CRYPTO_AEAD_AES_GCM:
+ if (qat_alg_validate_aes_key(aead_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
+ break;
+ case RTE_CRYPTO_AEAD_AES_CCM:
+ if (qat_alg_validate_aes_key(aead_xform->key.length,
+ &session->qat_cipher_alg) != 0) {
+ PMD_DRV_LOG(ERR, "Invalid AES key size");
+ return -EINVAL;
+ }
+ session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
+ session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC;
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "Crypto: Undefined AEAD specified %u\n",
+ aead_xform->algo);
+ return -EINVAL;
+ }
+
+ if ((aead_xform->op == RTE_CRYPTO_AEAD_OP_ENCRYPT &&
+ aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM) ||
+ (aead_xform->op == RTE_CRYPTO_AEAD_OP_DECRYPT &&
+ aead_xform->algo == RTE_CRYPTO_AEAD_AES_CCM)) {
+ session->qat_dir = ICP_QAT_HW_CIPHER_ENCRYPT;
+ /*
+ * It needs to create cipher desc content first,
+ * then authentication
+ */
+ crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
+ RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ aead_xform->key.data,
+ aead_xform->key.length))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ aead_xform->key.data,
+ aead_xform->key.length,
+ aead_xform->aad_length,
+ aead_xform->digest_length,
+ crypto_operation))
+ return -EINVAL;
+ } else {
+ session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
+ /*
+ * It needs to create authentication desc content first,
+ * then cipher
+ */
+
+ crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
+ RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
+
+ if (qat_alg_aead_session_create_content_desc_auth(session,
+ aead_xform->key.data,
+ aead_xform->key.length,
+ aead_xform->aad_length,
+ aead_xform->digest_length,
+ crypto_operation))
+ return -EINVAL;
+
+ if (qat_alg_aead_session_create_content_desc_cipher(session,
+ aead_xform->key.data,
+ aead_xform->key.length))
+ return -EINVAL;
+ }
+
+ session->digest_length = aead_xform->digest_length;
+ return 0;
+}
+
+unsigned int qat_crypto_sym_get_session_private_size(
+ struct rte_cryptodev *dev __rte_unused)
+{
+ return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
+}
/* returns block size in bytes per cipher algo */
int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg)
@@ -77,12 +788,12 @@ static int qat_hash_get_state1_size(enum icp_qat_hw_auth_algo qat_hash_alg)
case ICP_QAT_HW_AUTH_ALGO_KASUMI_F9:
return QAT_HW_ROUND_UP(ICP_QAT_HW_KASUMI_F9_STATE1_SZ,
QAT_HW_DEFAULT_ALIGNMENT);
- case ICP_QAT_HW_AUTH_ALGO_NULL:
- return QAT_HW_ROUND_UP(ICP_QAT_HW_NULL_STATE1_SZ,
- QAT_HW_DEFAULT_ALIGNMENT);
case ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC:
return QAT_HW_ROUND_UP(ICP_QAT_HW_AES_CBC_MAC_STATE1_SZ,
QAT_HW_DEFAULT_ALIGNMENT);
+ case ICP_QAT_HW_AUTH_ALGO_NULL:
+ return QAT_HW_ROUND_UP(ICP_QAT_HW_NULL_STATE1_SZ,
+ QAT_HW_DEFAULT_ALIGNMENT);
case ICP_QAT_HW_AUTH_ALGO_DELIMITER:
/* return maximum state1 size in this case */
return QAT_HW_ROUND_UP(ICP_QAT_HW_SHA512_STATE1_SZ,
@@ -854,14 +1565,13 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
if (aad_length > 0) {
aad_length += ICP_QAT_HW_CCM_AAD_B0_LEN +
- ICP_QAT_HW_CCM_AAD_LEN_INFO;
+ ICP_QAT_HW_CCM_AAD_LEN_INFO;
auth_param->u2.aad_sz =
- RTE_ALIGN_CEIL(aad_length,
- ICP_QAT_HW_CCM_AAD_ALIGNMENT);
+ RTE_ALIGN_CEIL(aad_length,
+ ICP_QAT_HW_CCM_AAD_ALIGNMENT);
} else {
auth_param->u2.aad_sz = ICP_QAT_HW_CCM_AAD_B0_LEN;
}
-
cdesc->aad_len = aad_length;
hash->auth_counter.counter = 0;
diff --git a/drivers/crypto/qat/qat_adf/qat_algs.h b/drivers/crypto/qat/qat_sym_session.h
similarity index 66%
rename from drivers/crypto/qat/qat_adf/qat_algs.h
rename to drivers/crypto/qat/qat_sym_session.h
index 6c49c6529..f90b1821d 100644
--- a/drivers/crypto/qat/qat_adf/qat_algs.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -1,14 +1,16 @@
-/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
- * Copyright(c) 2015-2018 Intel Corporation
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
*/
-#ifndef _ICP_QAT_ALGS_H_
-#define _ICP_QAT_ALGS_H_
-#include <rte_memory.h>
+#ifndef _QAT_SYM_SESSION_H_
+#define _QAT_SYM_SESSION_H_
+
#include <rte_crypto.h>
+#include <rte_cryptodev_pmd.h>
+
+#include "qat_common.h"
#include "icp_qat_hw.h"
#include "icp_qat_fw.h"
#include "icp_qat_fw_la.h"
-#include "../qat_crypto.h"
/*
* Key Modifier (KM) value used in KASUMI algorithm in F9 mode to XOR
@@ -56,7 +58,7 @@ struct qat_session {
void *bpi_ctx;
struct qat_alg_cd cd;
uint8_t *cd_cur_ptr;
- rte_iova_t cd_paddr;
+ phys_addr_t cd_paddr;
struct icp_qat_fw_la_bulk_req fw_req;
uint8_t aad_len;
struct qat_crypto_instance *inst;
@@ -73,19 +75,57 @@ struct qat_session {
enum qat_device_gen min_qat_dev_gen;
};
-int qat_get_inter_state_size(enum icp_qat_hw_auth_algo qat_hash_alg);
+int
+qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct rte_cryptodev_sym_session *sess,
+ struct rte_mempool *mempool);
+
+int
+qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform, void *session_private);
+
+int
+qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
+
+int
+qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
+
+int
+qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct qat_session *session);
-int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cd,
+int
+qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cd,
uint8_t *enckey,
uint32_t enckeylen);
-int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+int
+qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
uint32_t digestsize,
unsigned int operation);
+int
+qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
+ unsigned int nb_objs, unsigned int obj_cache_size, int socket_id);
+
+void
+qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *session);
+
+unsigned int
+qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
+
+int qat_get_inter_state_size(enum icp_qat_hw_auth_algo qat_hash_alg);
+
+
void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
enum qat_crypto_proto_flag proto_flags);
@@ -98,4 +138,5 @@ int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-#endif
+
+#endif /* _QAT_SYM_SESSION_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index c8da07af6..e425eb43f 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -10,6 +10,7 @@
#include <rte_cryptodev_pmd.h>
#include "qat_crypto.h"
+#include "qat_sym_session.h"
#include "qat_logs.h"
uint8_t cryptodev_qat_driver_id;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 05/38] crypto/qat: change filename crypto to sym
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (3 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 04/38] crypto/qat: add symmetric session file Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 06/38] crypto/qat: rename fns for consistency Tomasz Jozwiak
` (33 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
This commit renames qat_crypto.c/h to qat_sym.c/h
And makes a few whitespace changes to resolve line-length
issues.
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 2 +-
drivers/crypto/qat/meson.build | 2 +-
drivers/crypto/qat/qat_qp.c | 2 +-
.../crypto/qat/{qat_crypto.c => qat_sym.c} | 60 +++++++++----------
.../crypto/qat/{qat_crypto.h => qat_sym.h} | 0
drivers/crypto/qat/rte_qat_cryptodev.c | 2 +-
6 files changed, 33 insertions(+), 35 deletions(-)
rename drivers/crypto/qat/{qat_crypto.c => qat_sym.c} (95%)
rename drivers/crypto/qat/{qat_crypto.h => qat_sym.h} (100%)
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index c63c1515e..8cb802b9d 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -21,7 +21,7 @@ LDLIBS += -lrte_cryptodev
LDLIBS += -lrte_pci -lrte_bus_pci
# library source files
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_crypto.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index be4282a83..e596006da 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -5,7 +5,7 @@ dep = dependency('libcrypto', required: false)
if not dep.found()
build = false
endif
-sources = files('qat_crypto.c', 'qat_qp.c',
+sources = files('qat_sym.c', 'qat_qp.c',
'qat_sym_session.c',
'rte_qat_cryptodev.c',
'qat_device.c')
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index ee3b30a36..794a8d7c9 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,7 +13,7 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
diff --git a/drivers/crypto/qat/qat_crypto.c b/drivers/crypto/qat/qat_sym.c
similarity index 95%
rename from drivers/crypto/qat/qat_crypto.c
rename to drivers/crypto/qat/qat_sym.c
index 96a1b78f0..f5d542ae3 100644
--- a/drivers/crypto/qat/qat_crypto.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -6,15 +6,14 @@
#include <rte_mbuf.h>
#include <rte_hexdump.h>
#include <rte_crypto_sym.h>
-#include <rte_byteorder.h>
-#include <rte_pci.h>
#include <rte_bus_pci.h>
+#include <rte_byteorder.h>
#include <openssl/evp.h>
#include "qat_logs.h"
#include "qat_sym_session.h"
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
#define BYTE_LENGTH 8
@@ -500,8 +499,6 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
-
-
qat_req = (struct icp_qat_fw_la_bulk_req *)out_msg;
rte_mov128((uint8_t *)qat_req, (const uint8_t *)&(ctx->fw_req));
qat_req->comn_mid.opaque_data = (uint64_t)(uintptr_t)op;
@@ -512,11 +509,11 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
ctx->qat_cmd == ICP_QAT_FW_LA_CMD_CIPHER_HASH) {
/* AES-GCM or AES-CCM */
if (ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_128 ||
- ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_64 ||
- (ctx->qat_cipher_alg == ICP_QAT_HW_CIPHER_ALGO_AES128
- && ctx->qat_mode == ICP_QAT_HW_CIPHER_CTR_MODE
- && ctx->qat_hash_alg ==
- ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC)) {
+ ctx->qat_hash_alg == ICP_QAT_HW_AUTH_ALGO_GALOIS_64 ||
+ (ctx->qat_cipher_alg == ICP_QAT_HW_CIPHER_ALGO_AES128
+ && ctx->qat_mode == ICP_QAT_HW_CIPHER_CTR_MODE
+ && ctx->qat_hash_alg ==
+ ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC)) {
do_aead = 1;
} else {
do_auth = 1;
@@ -642,7 +639,6 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
qat_req->comn_hdr.serv_specif_flags,
ICP_QAT_FW_LA_GCM_IV_LEN_12_OCTETS);
}
-
set_cipher_iv(ctx->cipher_iv.length,
ctx->cipher_iv.offset,
cipher_param, op, qat_req);
@@ -650,15 +646,14 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
} else if (ctx->qat_hash_alg ==
ICP_QAT_HW_AUTH_ALGO_AES_CBC_MAC) {
- /* In case of AES-CCM this may point to user selected memory
- * or iv offset in cypto_op
+ /* In case of AES-CCM this may point to user selected
+ * memory or iv offset in cypto_op
*/
uint8_t *aad_data = op->sym->aead.aad.data;
/* This is true AAD length, it not includes 18 bytes of
* preceding data
*/
uint8_t aad_ccm_real_len = 0;
-
uint8_t aad_len_field_sz = 0;
uint32_t msg_len_be =
rte_bswap32(op->sym->aead.data.length);
@@ -670,33 +665,36 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
ICP_QAT_HW_CCM_AAD_LEN_INFO;
} else {
/*
- * aad_len not greater than 18, so no actual aad data,
- * then use IV after op for B0 block
+ * aad_len not greater than 18, so no actual aad
+ * data, then use IV after op for B0 block
*/
- aad_data = rte_crypto_op_ctod_offset(op, uint8_t *,
+ aad_data = rte_crypto_op_ctod_offset(op,
+ uint8_t *,
ctx->cipher_iv.offset);
aad_phys_addr_aead =
rte_crypto_op_ctophys_offset(op,
- ctx->cipher_iv.offset);
+ ctx->cipher_iv.offset);
}
- uint8_t q = ICP_QAT_HW_CCM_NQ_CONST - ctx->cipher_iv.length;
+ uint8_t q = ICP_QAT_HW_CCM_NQ_CONST -
+ ctx->cipher_iv.length;
- aad_data[0] = ICP_QAT_HW_CCM_BUILD_B0_FLAGS(aad_len_field_sz,
+ aad_data[0] = ICP_QAT_HW_CCM_BUILD_B0_FLAGS(
+ aad_len_field_sz,
ctx->digest_length, q);
if (q > ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE) {
memcpy(aad_data + ctx->cipher_iv.length +
- ICP_QAT_HW_CCM_NONCE_OFFSET
- + (q - ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE),
- (uint8_t *)&msg_len_be,
- ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE);
+ ICP_QAT_HW_CCM_NONCE_OFFSET +
+ (q - ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE),
+ (uint8_t *)&msg_len_be,
+ ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE);
} else {
memcpy(aad_data + ctx->cipher_iv.length +
- ICP_QAT_HW_CCM_NONCE_OFFSET,
- (uint8_t *)&msg_len_be
- + (ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE
- - q), q);
+ ICP_QAT_HW_CCM_NONCE_OFFSET,
+ (uint8_t *)&msg_len_be
+ + (ICP_QAT_HW_CCM_MSG_LEN_MAX_FIELD_SIZE
+ - q), q);
}
if (aad_len_field_sz > 0) {
@@ -709,10 +707,10 @@ qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
uint8_t pad_idx = 0;
pad_len = ICP_QAT_HW_CCM_AAD_B0_LEN -
- ((aad_ccm_real_len + aad_len_field_sz) %
- ICP_QAT_HW_CCM_AAD_B0_LEN);
+ ((aad_ccm_real_len + aad_len_field_sz) %
+ ICP_QAT_HW_CCM_AAD_B0_LEN);
pad_idx = ICP_QAT_HW_CCM_AAD_B0_LEN +
- aad_ccm_real_len + aad_len_field_sz;
+ aad_ccm_real_len + aad_len_field_sz;
memset(&aad_data[pad_idx],
0, pad_len);
}
diff --git a/drivers/crypto/qat/qat_crypto.h b/drivers/crypto/qat/qat_sym.h
similarity index 100%
rename from drivers/crypto/qat/qat_crypto.h
rename to drivers/crypto/qat/qat_sym.h
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index e425eb43f..45f8a253b 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -9,7 +9,7 @@
#include <rte_pci.h>
#include <rte_cryptodev_pmd.h>
-#include "qat_crypto.h"
+#include "qat_sym.h"
#include "qat_sym_session.h"
#include "qat_logs.h"
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 06/38] crypto/qat: rename fns for consistency
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (4 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 05/38] crypto/qat: change filename crypto to sym Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 07/38] crypto/qat: renamed sym-specific structs Tomasz Jozwiak
` (32 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Rename fn names to shorten them, i.e.
qat_crypto_sym_xxx to qat_sym_xxx
_content_desc_ to _cd_
Renaming symmetric crypto specific with consistent names:
qat_crypto_set_session_parameters->qat_sym_set_session_parameters
qat_write_hw_desc_entry()->qat_sym_build_request()
qat_alg_xxx ->qat_sym_xxx
qat_sym_xxx_session_yyy()->qat_sym_session_xxx_yyy()
Removed unused prototypes:
qat_get_inter_state_size()
qat_pmd_session_mempool_create()
Removed 2 unnecessary extern declarations
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 2 +-
drivers/crypto/qat/qat_device.h | 2 +-
drivers/crypto/qat/qat_qp.c | 6 +-
drivers/crypto/qat/qat_sym.c | 14 +--
drivers/crypto/qat/qat_sym.h | 22 ++---
drivers/crypto/qat/qat_sym_session.c | 131 ++++++++++++-------------
drivers/crypto/qat/qat_sym_session.h | 53 +++++-----
drivers/crypto/qat/rte_qat_cryptodev.c | 18 ++--
8 files changed, 123 insertions(+), 125 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index c2bf9b7a7..ac6bd1af6 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -30,7 +30,7 @@ int qat_dev_close(struct rte_cryptodev *dev)
PMD_INIT_FUNC_TRACE();
for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_crypto_sym_qp_release(dev, i);
+ ret = qat_sym_qp_release(dev, i);
if (ret < 0)
return ret;
}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 5c48fdb93..2cb8e7612 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,7 +11,7 @@
extern uint8_t cryptodev_qat_driver_id;
-extern int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
/** private data structure for each QAT device */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 794a8d7c9..23a9d5f01 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -79,7 +79,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
const struct rte_cryptodev_qp_conf *qp_conf,
int socket_id, struct rte_mempool *session_pool __rte_unused)
{
@@ -93,7 +93,7 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
/* If qp is already in use free ring memory and qp metadata. */
if (dev->data->queue_pairs[queue_pair_id] != NULL) {
- ret = qat_crypto_sym_qp_release(dev, queue_pair_id);
+ ret = qat_sym_qp_release(dev, queue_pair_id);
if (ret < 0)
return ret;
}
@@ -209,7 +209,7 @@ int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-int qat_crypto_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index f5d542ae3..ae521c2b1 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -87,7 +87,7 @@ static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
static inline int
-qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
+qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
static inline uint32_t
@@ -210,7 +210,7 @@ txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
}
uint16_t
-qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
register struct qat_queue *queue;
@@ -242,7 +242,7 @@ qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
while (nb_ops_sent != nb_ops_possible) {
- ret = qat_write_hw_desc_entry(*cur_op, base_addr + tail,
+ ret = qat_sym_build_request(*cur_op, base_addr + tail,
tmp_qp->op_cookies[tail / queue->msg_size], tmp_qp);
if (ret != 0) {
tmp_qp->stats.enqueue_err_count++;
@@ -299,7 +299,7 @@ void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
}
uint16_t
-qat_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
struct qat_queue *rx_queue, *tx_queue;
@@ -456,7 +456,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
}
static inline int
-qat_write_hw_desc_entry(struct rte_crypto_op *op, uint8_t *out_msg,
+qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp)
{
int ret = 0;
@@ -883,7 +883,7 @@ static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
return data - mult;
}
-void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
+void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
{
int i;
@@ -907,7 +907,7 @@ void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
}
}
-void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev)
+void qat_sym_stats_reset(struct rte_cryptodev *dev)
{
int i;
struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index e3873171c..811ba8619 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -2,8 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
-#ifndef _QAT_CRYPTO_H_
-#define _QAT_CRYPTO_H_
+#ifndef _QAT_SYM_H_
+#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
#include <rte_memzone.h>
@@ -65,23 +65,23 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
} __rte_cache_aligned;
-void qat_crypto_sym_stats_get(struct rte_cryptodev *dev,
+void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
-void qat_crypto_sym_stats_reset(struct rte_cryptodev *dev);
+void qat_sym_stats_reset(struct rte_cryptodev *dev);
-int qat_crypto_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
const struct rte_cryptodev_qp_conf *rx_conf, int socket_id,
struct rte_mempool *session_pool);
-int qat_crypto_sym_qp_release(struct rte_cryptodev *dev,
+int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-extern uint16_t
-qat_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops);
-extern uint16_t
-qat_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops);
-#endif /* _QAT_CRYPTO_H_ */
+#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index 5caac5a6f..f598d1461 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -105,7 +105,7 @@ qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
}
void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+qat_sym_session_clear(struct rte_cryptodev *dev,
struct rte_cryptodev_sym_session *sess)
{
PMD_INIT_FUNC_TRACE();
@@ -116,7 +116,7 @@ qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
if (sess_priv) {
if (s->bpi_ctx)
bpi_cipher_ctx_free(s->bpi_ctx);
- memset(s, 0, qat_crypto_sym_get_session_private_size(dev));
+ memset(s, 0, qat_sym_session_get_private_size(dev));
struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
set_session_private_data(sess, index, NULL);
@@ -197,7 +197,7 @@ qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
}
int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
@@ -213,7 +213,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
switch (cipher_xform->algo) {
case RTE_CRYPTO_CIPHER_AES_CBC:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
ret = -EINVAL;
@@ -222,7 +222,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_AES_CTR:
- if (qat_alg_validate_aes_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES cipher key size");
ret = -EINVAL;
@@ -231,7 +231,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CTR_MODE;
break;
case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
- if (qat_alg_validate_snow3g_key(cipher_xform->key.length,
+ if (qat_sym_validate_snow3g_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid SNOW 3G cipher key size");
ret = -EINVAL;
@@ -243,7 +243,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_ECB_MODE;
break;
case RTE_CRYPTO_CIPHER_KASUMI_F8:
- if (qat_alg_validate_kasumi_key(cipher_xform->key.length,
+ if (qat_sym_validate_kasumi_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid KASUMI cipher key size");
ret = -EINVAL;
@@ -252,7 +252,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_F8_MODE;
break;
case RTE_CRYPTO_CIPHER_3DES_CBC:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ if (qat_sym_validate_3des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
ret = -EINVAL;
@@ -261,7 +261,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_DES_CBC:
- if (qat_alg_validate_des_key(cipher_xform->key.length,
+ if (qat_sym_validate_des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
ret = -EINVAL;
@@ -270,7 +270,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
session->qat_mode = ICP_QAT_HW_CIPHER_CBC_MODE;
break;
case RTE_CRYPTO_CIPHER_3DES_CTR:
- if (qat_alg_validate_3des_key(cipher_xform->key.length,
+ if (qat_sym_validate_3des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid 3DES cipher key size");
ret = -EINVAL;
@@ -288,7 +288,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
PMD_DRV_LOG(ERR, "failed to create DES BPI ctx");
goto error_out;
}
- if (qat_alg_validate_des_key(cipher_xform->key.length,
+ if (qat_sym_validate_des_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid DES cipher key size");
ret = -EINVAL;
@@ -306,7 +306,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
PMD_DRV_LOG(ERR, "failed to create AES BPI ctx");
goto error_out;
}
- if (qat_alg_validate_aes_docsisbpi_key(cipher_xform->key.length,
+ if (qat_sym_validate_aes_docsisbpi_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES DOCSISBPI key size");
ret = -EINVAL;
@@ -323,7 +323,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
ret = -ENOTSUP;
goto error_out;
}
- if (qat_alg_validate_zuc_key(cipher_xform->key.length,
+ if (qat_sym_validate_zuc_key(cipher_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid ZUC cipher key size");
ret = -EINVAL;
@@ -352,7 +352,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
else
session->qat_dir = ICP_QAT_HW_CIPHER_DECRYPT;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
cipher_xform->key.data,
cipher_xform->key.length)) {
ret = -EINVAL;
@@ -370,7 +370,7 @@ qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+qat_sym_session_configure(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct rte_cryptodev_sym_session *sess,
struct rte_mempool *mempool)
@@ -384,7 +384,7 @@ qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
return -ENOMEM;
}
- ret = qat_crypto_set_session_parameters(dev, xform, sess_private_data);
+ ret = qat_sym_session_set_parameters(dev, xform, sess_private_data);
if (ret != 0) {
PMD_DRV_LOG(ERR,
"Crypto QAT PMD: failed to configure session parameters");
@@ -401,7 +401,7 @@ qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
}
int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private)
{
struct qat_session *session = session_private;
@@ -425,29 +425,27 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
session->qat_cmd = (enum icp_qat_fw_la_cmd_id)qat_cmd_id;
switch (session->qat_cmd) {
case ICP_QAT_FW_LA_CMD_CIPHER:
- ret = qat_crypto_sym_configure_session_cipher(dev,
- xform, session);
+ ret = qat_sym_session_configure_cipher(dev, xform, session);
if (ret < 0)
return ret;
break;
case ICP_QAT_FW_LA_CMD_AUTH:
- ret = qat_crypto_sym_configure_session_auth(dev,
- xform, session);
+ ret = qat_sym_session_configure_auth(dev, xform, session);
if (ret < 0)
return ret;
break;
case ICP_QAT_FW_LA_CMD_CIPHER_HASH:
if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
+ ret = qat_sym_session_configure_aead(xform,
session);
if (ret < 0)
return ret;
} else {
- ret = qat_crypto_sym_configure_session_cipher(dev,
+ ret = qat_sym_session_configure_cipher(dev,
xform, session);
if (ret < 0)
return ret;
- ret = qat_crypto_sym_configure_session_auth(dev,
+ ret = qat_sym_session_configure_auth(dev,
xform, session);
if (ret < 0)
return ret;
@@ -455,16 +453,16 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
break;
case ICP_QAT_FW_LA_CMD_HASH_CIPHER:
if (xform->type == RTE_CRYPTO_SYM_XFORM_AEAD) {
- ret = qat_crypto_sym_configure_session_aead(xform,
+ ret = qat_sym_session_configure_aead(xform,
session);
if (ret < 0)
return ret;
} else {
- ret = qat_crypto_sym_configure_session_auth(dev,
+ ret = qat_sym_session_configure_auth(dev,
xform, session);
if (ret < 0)
return ret;
- ret = qat_crypto_sym_configure_session_cipher(dev,
+ ret = qat_sym_session_configure_cipher(dev,
xform, session);
if (ret < 0)
return ret;
@@ -492,7 +490,7 @@ qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
@@ -521,7 +519,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC;
break;
case RTE_CRYPTO_AUTH_AES_GMAC:
- if (qat_alg_validate_aes_key(auth_xform->key.length,
+ if (qat_sym_validate_aes_key(auth_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -579,14 +577,13 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
* It needs to create cipher desc content first,
* then authentication
*/
- if (qat_alg_aead_session_create_content_desc_cipher(
- session,
+
+ if (qat_sym_session_aead_create_cd_cipher(session,
auth_xform->key.data,
auth_xform->key.length))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_auth(
- session,
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -600,8 +597,8 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
* It needs to create authentication desc content first,
* then cipher
*/
- if (qat_alg_aead_session_create_content_desc_auth(
- session,
+
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -609,8 +606,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
auth_xform->op))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_cipher(
- session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
auth_xform->key.data,
auth_xform->key.length))
return -EINVAL;
@@ -618,7 +614,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
/* Restore to authentication only only */
session->qat_cmd = ICP_QAT_FW_LA_CMD_AUTH;
} else {
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
key_data,
key_length,
0,
@@ -632,7 +628,7 @@ qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
}
int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
struct qat_session *session)
{
struct rte_crypto_aead_xform *aead_xform = &xform->aead;
@@ -647,7 +643,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
switch (aead_xform->algo) {
case RTE_CRYPTO_AEAD_AES_GCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
+ if (qat_sym_validate_aes_key(aead_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -656,7 +652,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
session->qat_hash_alg = ICP_QAT_HW_AUTH_ALGO_GALOIS_128;
break;
case RTE_CRYPTO_AEAD_AES_CCM:
- if (qat_alg_validate_aes_key(aead_xform->key.length,
+ if (qat_sym_validate_aes_key(aead_xform->key.length,
&session->qat_cipher_alg) != 0) {
PMD_DRV_LOG(ERR, "Invalid AES key size");
return -EINVAL;
@@ -682,12 +678,12 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
RTE_CRYPTO_AUTH_OP_GENERATE : RTE_CRYPTO_AUTH_OP_VERIFY;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
aead_xform->key.data,
aead_xform->key.length))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
aead_xform->key.data,
aead_xform->key.length,
aead_xform->aad_length,
@@ -704,7 +700,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation = aead_xform->algo == RTE_CRYPTO_AEAD_AES_GCM ?
RTE_CRYPTO_AUTH_OP_VERIFY : RTE_CRYPTO_AUTH_OP_GENERATE;
- if (qat_alg_aead_session_create_content_desc_auth(session,
+ if (qat_sym_session_aead_create_cd_auth(session,
aead_xform->key.data,
aead_xform->key.length,
aead_xform->aad_length,
@@ -712,7 +708,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
crypto_operation))
return -EINVAL;
- if (qat_alg_aead_session_create_content_desc_cipher(session,
+ if (qat_sym_session_aead_create_cd_cipher(session,
aead_xform->key.data,
aead_xform->key.length))
return -EINVAL;
@@ -722,7 +718,7 @@ qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
return 0;
}
-unsigned int qat_crypto_sym_get_session_private_size(
+unsigned int qat_sym_session_get_private_size(
struct rte_cryptodev *dev __rte_unused)
{
return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
@@ -996,7 +992,7 @@ static int partial_hash_compute(enum icp_qat_hw_auth_algo hash_alg,
#define HMAC_OPAD_VALUE 0x5c
#define HASH_XCBC_PRECOMP_KEY_NUM 3
-static int qat_alg_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
+static int qat_sym_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
const uint8_t *auth_key,
uint16_t auth_keylen,
uint8_t *p_state_buf,
@@ -1126,7 +1122,8 @@ static int qat_alg_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
return 0;
}
-void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
+static void
+qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
enum qat_crypto_proto_flag proto_flags)
{
PMD_INIT_FUNC_TRACE();
@@ -1194,7 +1191,7 @@ qat_get_crypto_proto_flag(uint16_t flags)
return qat_proto_flag;
}
-int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
uint8_t *cipherkey,
uint32_t cipherkeylen)
{
@@ -1298,7 +1295,7 @@ int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
cipher_cd_ctrl->cipher_cfg_offset = cipher_offset >> 3;
header->service_cmd_id = cdesc->qat_cmd;
- qat_alg_init_common_hdr(header, qat_proto_flag);
+ qat_sym_session_init_common_hdr(header, qat_proto_flag);
cipher = (struct icp_qat_hw_cipher_algo_blk *)cdesc->cd_cur_ptr;
cipher->cipher_config.val =
@@ -1342,7 +1339,7 @@ int qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cdesc,
return 0;
}
-int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -1431,7 +1428,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
*/
switch (cdesc->qat_hash_alg) {
case ICP_QAT_HW_AUTH_ALGO_SHA1:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA1,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA1,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1439,7 +1436,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = RTE_ALIGN_CEIL(ICP_QAT_HW_SHA1_STATE2_SZ, 8);
break;
case ICP_QAT_HW_AUTH_ALGO_SHA224:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA224,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA224,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1447,7 +1444,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA224_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA256:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA256,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA256,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1455,7 +1452,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA256_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA384:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA384,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA384,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1463,7 +1460,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
state2_size = ICP_QAT_HW_SHA384_STATE2_SZ;
break;
case ICP_QAT_HW_AUTH_ALGO_SHA512:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA512,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_SHA512,
authkey, authkeylen, cdesc->cd_cur_ptr, &state1_size)) {
PMD_DRV_LOG(ERR, "(SHA)precompute failed");
return -EFAULT;
@@ -1472,7 +1469,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
break;
case ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC:
state1_size = ICP_QAT_HW_AES_XCBC_MAC_STATE1_SZ;
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_AES_XCBC_MAC,
authkey, authkeylen, cdesc->cd_cur_ptr + state1_size,
&state2_size)) {
PMD_DRV_LOG(ERR, "(XCBC)precompute failed");
@@ -1483,7 +1480,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
case ICP_QAT_HW_AUTH_ALGO_GALOIS_64:
qat_proto_flag = QAT_CRYPTO_PROTO_FLAG_GCM;
state1_size = ICP_QAT_HW_GALOIS_128_STATE1_SZ;
- if (qat_alg_do_precomputes(cdesc->qat_hash_alg,
+ if (qat_sym_do_precomputes(cdesc->qat_hash_alg,
authkey, authkeylen, cdesc->cd_cur_ptr + state1_size,
&state2_size)) {
PMD_DRV_LOG(ERR, "(GCM)precompute failed");
@@ -1543,7 +1540,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
break;
case ICP_QAT_HW_AUTH_ALGO_MD5:
- if (qat_alg_do_precomputes(ICP_QAT_HW_AUTH_ALGO_MD5,
+ if (qat_sym_do_precomputes(ICP_QAT_HW_AUTH_ALGO_MD5,
authkey, authkeylen, cdesc->cd_cur_ptr,
&state1_size)) {
PMD_DRV_LOG(ERR, "(MD5)precompute failed");
@@ -1606,7 +1603,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
}
/* Request template setup */
- qat_alg_init_common_hdr(header, qat_proto_flag);
+ qat_sym_session_init_common_hdr(header, qat_proto_flag);
header->service_cmd_id = cdesc->qat_cmd;
/* Auth CD config setup */
@@ -1632,7 +1629,7 @@ int qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
return 0;
}
-int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_AES_128_KEY_SZ:
@@ -1650,7 +1647,7 @@ int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_aes_docsisbpi_key(int key_len,
+int qat_sym_validate_aes_docsisbpi_key(int key_len,
enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
@@ -1663,7 +1660,7 @@ int qat_alg_validate_aes_docsisbpi_key(int key_len,
return 0;
}
-int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_SNOW_3G_UEA2_KEY_SZ:
@@ -1675,7 +1672,7 @@ int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_KASUMI_KEY_SZ:
@@ -1687,7 +1684,7 @@ int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_DES_KEY_SZ:
@@ -1699,7 +1696,7 @@ int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case QAT_3DES_KEY_SZ_OPT1:
@@ -1712,7 +1709,7 @@ int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
return 0;
}
-int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
+int qat_sym_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg)
{
switch (key_len) {
case ICP_QAT_HW_ZUC_3G_EEA3_KEY_SZ:
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index f90b1821d..493036ebf 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -76,67 +76,68 @@ struct qat_session {
};
int
-qat_crypto_sym_configure_session(struct rte_cryptodev *dev,
+qat_sym_session_configure(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct rte_cryptodev_sym_session *sess,
struct rte_mempool *mempool);
int
-qat_crypto_set_session_parameters(struct rte_cryptodev *dev,
+qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private);
int
-qat_crypto_sym_configure_session_aead(struct rte_crypto_sym_xform *xform,
+qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_crypto_sym_configure_session_cipher(struct rte_cryptodev *dev,
+qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_crypto_sym_configure_session_auth(struct rte_cryptodev *dev,
+qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_session *session);
int
-qat_alg_aead_session_create_content_desc_cipher(struct qat_session *cd,
+qat_sym_session_aead_create_cd_cipher(struct qat_session *cd,
uint8_t *enckey,
uint32_t enckeylen);
int
-qat_alg_aead_session_create_content_desc_auth(struct qat_session *cdesc,
+qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
uint32_t digestsize,
unsigned int operation);
-int
-qat_pmd_session_mempool_create(struct rte_cryptodev *dev,
- unsigned int nb_objs, unsigned int obj_cache_size, int socket_id);
-
void
-qat_crypto_sym_clear_session(struct rte_cryptodev *dev,
+qat_sym_session_clear(struct rte_cryptodev *dev,
struct rte_cryptodev_sym_session *session);
unsigned int
-qat_crypto_sym_get_session_private_size(struct rte_cryptodev *dev);
-
-int qat_get_inter_state_size(enum icp_qat_hw_auth_algo qat_hash_alg);
-
+qat_sym_session_get_private_size(struct rte_cryptodev *dev);
-void qat_alg_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
+void
+qat_sym_sesssion_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
enum qat_crypto_proto_flag proto_flags);
-
-int qat_alg_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_aes_docsisbpi_key(int key_len,
+int
+qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_aes_docsisbpi_key(int key_len,
enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_alg_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
-int qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
-int qat_alg_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_snow3g_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_kasumi_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_3des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_sym_validate_des_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
+int
+qat_cipher_get_block_size(enum icp_qat_hw_cipher_algo qat_cipher_alg);
+int
+qat_sym_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
#endif /* _QAT_SYM_SESSION_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 45f8a253b..42011dcd9 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -35,18 +35,18 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
.dev_close = qat_dev_close,
.dev_infos_get = qat_dev_info_get,
- .stats_get = qat_crypto_sym_stats_get,
- .stats_reset = qat_crypto_sym_stats_reset,
- .queue_pair_setup = qat_crypto_sym_qp_setup,
- .queue_pair_release = qat_crypto_sym_qp_release,
+ .stats_get = qat_sym_stats_get,
+ .stats_reset = qat_sym_stats_reset,
+ .queue_pair_setup = qat_sym_qp_setup,
+ .queue_pair_release = qat_sym_qp_release,
.queue_pair_start = NULL,
.queue_pair_stop = NULL,
.queue_pair_count = NULL,
/* Crypto related operations */
- .session_get_size = qat_crypto_sym_get_session_private_size,
- .session_configure = qat_crypto_sym_configure_session,
- .session_clear = qat_crypto_sym_clear_session
+ .session_get_size = qat_sym_session_get_private_size,
+ .session_configure = qat_sym_session_configure,
+ .session_clear = qat_sym_session_clear
};
/*
@@ -86,8 +86,8 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
cryptodev->driver_id = cryptodev_qat_driver_id;
cryptodev->dev_ops = &crypto_qat_ops;
- cryptodev->enqueue_burst = qat_pmd_enqueue_op_burst;
- cryptodev->dequeue_burst = qat_pmd_dequeue_op_burst;
+ cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
+ cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
RTE_CRYPTODEV_FF_HW_ACCELERATED |
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 07/38] crypto/qat: renamed sym-specific structs
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (5 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 06/38] crypto/qat: rename fns for consistency Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 08/38] crypto/qat: make enqueue function generic Tomasz Jozwiak
` (31 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
qat_session -> qat_sym_session
qat_crypto_proto_flag -> qat_sym_proto_flag
qat_alg_cd -> qat_sym_cd
qat_crypto_op_cookie -> qat_sym_op_cookie
qat_gen1_capabilities -> qat_gen1_sym_capabilities
qat_gen2_capabilities -> qat_gen2_sym_capabilities
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 4 ++--
drivers/crypto/qat/qat_qp.c | 8 ++++----
drivers/crypto/qat/qat_sym.c | 19 ++++++++---------
drivers/crypto/qat/qat_sym.h | 2 +-
drivers/crypto/qat/qat_sym_session.c | 28 +++++++++++++-------------
drivers/crypto/qat/qat_sym_session.h | 20 +++++++++---------
drivers/crypto/qat/rte_qat_cryptodev.c | 8 ++++----
7 files changed, 45 insertions(+), 44 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 293b6f700..7802e96f9 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -37,11 +37,11 @@ struct qat_alg_buf_list {
struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
-struct qat_crypto_op_cookie {
+struct qat_sym_op_cookie {
struct qat_alg_buf_list qat_sgl_list_src;
struct qat_alg_buf_list qat_sgl_list_dst;
phys_addr_t qat_sgl_src_phys_addr;
phys_addr_t qat_sgl_dst_phys_addr;
};
-#endif /* _QAT_QAT_COMMON_H_ */
+#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 23a9d5f01..fb9c2a7ef 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -165,7 +165,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
qp->nb_descriptors,
- sizeof(struct qat_crypto_op_cookie), 64, 0,
+ sizeof(struct qat_sym_op_cookie), 64, 0,
NULL, NULL, NULL, NULL, socket_id,
0);
if (!qp->op_cookie_pool) {
@@ -180,17 +180,17 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
goto create_err;
}
- struct qat_crypto_op_cookie *sql_cookie =
+ struct qat_sym_op_cookie *sql_cookie =
qp->op_cookies[i];
sql_cookie->qat_sgl_src_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_crypto_op_cookie,
+ offsetof(struct qat_sym_op_cookie,
qat_sgl_list_src);
sql_cookie->qat_sgl_dst_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_crypto_op_cookie,
+ offsetof(struct qat_sym_op_cookie,
qat_sgl_list_dst);
}
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index ae521c2b1..2dfdc9cce 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -88,10 +88,10 @@ adf_modulo(uint32_t data, uint32_t shift);
static inline int
qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp);
+ struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp);
static inline uint32_t
-qat_bpicipher_preprocess(struct qat_session *ctx,
+qat_bpicipher_preprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
{
int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
@@ -146,7 +146,7 @@ qat_bpicipher_preprocess(struct qat_session *ctx,
}
static inline uint32_t
-qat_bpicipher_postprocess(struct qat_session *ctx,
+qat_bpicipher_postprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
{
int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
@@ -329,10 +329,11 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
resp_msg->comn_hdr.comn_status)) {
rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
} else {
- struct qat_session *sess = (struct qat_session *)
+ struct qat_sym_session *sess =
+ (struct qat_sym_session *)
get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
if (sess->bpi_ctx)
qat_bpicipher_postprocess(sess, rx_op);
@@ -457,10 +458,10 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
static inline int
qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_crypto_op_cookie *qat_op_cookie, struct qat_qp *qp)
+ struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp)
{
int ret = 0;
- struct qat_session *ctx;
+ struct qat_sym_session *ctx;
struct icp_qat_fw_la_cipher_req_params *cipher_param;
struct icp_qat_fw_la_auth_req_params *auth_param;
register struct icp_qat_fw_la_bulk_req *qat_req;
@@ -485,7 +486,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
- ctx = (struct qat_session *)get_session_private_data(
+ ctx = (struct qat_sym_session *)get_session_private_data(
op->sym->session, cryptodev_qat_driver_id);
if (unlikely(ctx == NULL)) {
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 811ba8619..18c77ea11 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,7 +27,7 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
-struct qat_session;
+struct qat_sym_session;
/**
* Structure associated with each queue.
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index f598d1461..a08e93037 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -111,7 +111,7 @@ qat_sym_session_clear(struct rte_cryptodev *dev,
PMD_INIT_FUNC_TRACE();
uint8_t index = dev->driver_id;
void *sess_priv = get_session_private_data(sess, index);
- struct qat_session *s = (struct qat_session *)sess_priv;
+ struct qat_sym_session *s = (struct qat_sym_session *)sess_priv;
if (sess_priv) {
if (s->bpi_ctx)
@@ -199,7 +199,7 @@ qat_get_cipher_xform(struct rte_crypto_sym_xform *xform)
int
qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct qat_pmd_private *internals = dev->data->dev_private;
struct rte_crypto_cipher_xform *cipher_xform = NULL;
@@ -404,7 +404,7 @@ int
qat_sym_session_set_parameters(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform, void *session_private)
{
- struct qat_session *session = session_private;
+ struct qat_sym_session *session = session_private;
int ret;
int qat_cmd_id;
@@ -412,7 +412,7 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
/* Set context descriptor physical address */
session->cd_paddr = rte_mempool_virt2iova(session) +
- offsetof(struct qat_session, cd);
+ offsetof(struct qat_sym_session, cd);
session->min_qat_dev_gen = QAT_GEN1;
@@ -492,7 +492,7 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
int
qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
struct qat_pmd_private *internals = dev->data->dev_private;
@@ -629,7 +629,7 @@ qat_sym_session_configure_auth(struct rte_cryptodev *dev,
int
qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session)
+ struct qat_sym_session *session)
{
struct rte_crypto_aead_xform *aead_xform = &xform->aead;
enum rte_crypto_auth_operation crypto_operation;
@@ -721,7 +721,7 @@ qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
unsigned int qat_sym_session_get_private_size(
struct rte_cryptodev *dev __rte_unused)
{
- return RTE_ALIGN_CEIL(sizeof(struct qat_session), 8);
+ return RTE_ALIGN_CEIL(sizeof(struct qat_sym_session), 8);
}
/* returns block size in bytes per cipher algo */
@@ -1124,7 +1124,7 @@ static int qat_sym_do_precomputes(enum icp_qat_hw_auth_algo hash_alg,
static void
qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
- enum qat_crypto_proto_flag proto_flags)
+ enum qat_sym_proto_flag proto_flags)
{
PMD_INIT_FUNC_TRACE();
header->hdr_flags =
@@ -1172,11 +1172,11 @@ qat_sym_session_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
* and set its protocol flag in both cipher and auth part of content
* descriptor building function
*/
-static enum qat_crypto_proto_flag
+static enum qat_sym_proto_flag
qat_get_crypto_proto_flag(uint16_t flags)
{
int proto = ICP_QAT_FW_LA_PROTO_GET(flags);
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
switch (proto) {
@@ -1191,7 +1191,7 @@ qat_get_crypto_proto_flag(uint16_t flags)
return qat_proto_flag;
}
-int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_cipher(struct qat_sym_session *cdesc,
uint8_t *cipherkey,
uint32_t cipherkeylen)
{
@@ -1203,7 +1203,7 @@ int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
struct icp_qat_fw_cipher_cd_ctrl_hdr *cipher_cd_ctrl = ptr;
struct icp_qat_fw_auth_cd_ctrl_hdr *hash_cd_ctrl = ptr;
enum icp_qat_hw_cipher_convert key_convert;
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
uint32_t total_key_size;
uint16_t cipher_offset, cd_size;
@@ -1339,7 +1339,7 @@ int qat_sym_session_aead_create_cd_cipher(struct qat_session *cdesc,
return 0;
}
-int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
+int qat_sym_session_aead_create_cd_auth(struct qat_sym_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -1363,7 +1363,7 @@ int qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
uint32_t *aad_len = NULL;
uint32_t wordIndex = 0;
uint32_t *pTempKey;
- enum qat_crypto_proto_flag qat_proto_flag =
+ enum qat_sym_proto_flag qat_proto_flag =
QAT_CRYPTO_PROTO_FLAG_NONE;
PMD_INIT_FUNC_TRACE();
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index 493036ebf..d3d27ff46 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -34,7 +34,7 @@
ICP_QAT_HW_CIPHER_KEY_CONVERT, \
ICP_QAT_HW_CIPHER_DECRYPT)
-enum qat_crypto_proto_flag {
+enum qat_sym_proto_flag {
QAT_CRYPTO_PROTO_FLAG_NONE = 0,
QAT_CRYPTO_PROTO_FLAG_CCM = 1,
QAT_CRYPTO_PROTO_FLAG_GCM = 2,
@@ -43,12 +43,12 @@ enum qat_crypto_proto_flag {
};
/* Common content descriptor */
-struct qat_alg_cd {
+struct qat_sym_cd {
struct icp_qat_hw_cipher_algo_blk cipher;
struct icp_qat_hw_auth_algo_blk hash;
} __rte_packed __rte_cache_aligned;
-struct qat_session {
+struct qat_sym_session {
enum icp_qat_fw_la_cmd_id qat_cmd;
enum icp_qat_hw_cipher_algo qat_cipher_alg;
enum icp_qat_hw_cipher_dir qat_dir;
@@ -56,7 +56,7 @@ struct qat_session {
enum icp_qat_hw_auth_algo qat_hash_alg;
enum icp_qat_hw_auth_op auth_op;
void *bpi_ctx;
- struct qat_alg_cd cd;
+ struct qat_sym_cd cd;
uint8_t *cd_cur_ptr;
phys_addr_t cd_paddr;
struct icp_qat_fw_la_bulk_req fw_req;
@@ -87,25 +87,25 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev,
int
qat_sym_session_configure_aead(struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
- struct qat_session *session);
+ struct qat_sym_session *session);
int
-qat_sym_session_aead_create_cd_cipher(struct qat_session *cd,
+qat_sym_session_aead_create_cd_cipher(struct qat_sym_session *cd,
uint8_t *enckey,
uint32_t enckeylen);
int
-qat_sym_session_aead_create_cd_auth(struct qat_session *cdesc,
+qat_sym_session_aead_create_cd_auth(struct qat_sym_session *cdesc,
uint8_t *authkey,
uint32_t authkeylen,
uint32_t aad_length,
@@ -121,7 +121,7 @@ qat_sym_session_get_private_size(struct rte_cryptodev *dev);
void
qat_sym_sesssion_init_common_hdr(struct icp_qat_fw_comn_req_hdr *header,
- enum qat_crypto_proto_flag proto_flags);
+ enum qat_sym_proto_flag proto_flags);
int
qat_sym_validate_aes_key(int key_len, enum icp_qat_hw_cipher_algo *alg);
int
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 42011dcd9..05cb5ef40 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -15,12 +15,12 @@
uint8_t cryptodev_qat_driver_id;
-static const struct rte_cryptodev_capabilities qat_gen1_capabilities[] = {
+static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
QAT_BASE_GEN1_SYM_CAPABILITIES,
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
};
-static const struct rte_cryptodev_capabilities qat_gen2_capabilities[] = {
+static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
QAT_BASE_GEN1_SYM_CAPABILITIES,
QAT_EXTRA_GEN2_SYM_CAPABILITIES,
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
@@ -99,13 +99,13 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
switch (pci_dev->id.device_id) {
case 0x0443:
internals->qat_dev_gen = QAT_GEN1;
- internals->qat_dev_capabilities = qat_gen1_capabilities;
+ internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
break;
case 0x37c9:
case 0x19e3:
case 0x6f55:
internals->qat_dev_gen = QAT_GEN2;
- internals->qat_dev_capabilities = qat_gen2_capabilities;
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
break;
default:
PMD_DRV_LOG(ERR,
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 08/38] crypto/qat: make enqueue function generic
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (6 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 07/38] crypto/qat: renamed sym-specific structs Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 09/38] crypto/qat: make dequeue " Tomasz Jozwiak
` (30 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Queue-handling code in enqueue is made generic, so it can
be used by other services in future. This is done by
- Removing all sym-specific refs in input params - replace with void ptrs.
- Wrapping this generic enqueue with the sym-specific enqueue
called through the API.
- Setting a fn ptr for build_request in qp on qp creation
- Passing void * params to this, in the service-specific implementation
qat_sym_build_request cast back to sym structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 1 +
drivers/crypto/qat/qat_sym.c | 46 ++++++++++++++++++++----------------
drivers/crypto/qat/qat_sym.h | 11 +++++++++
3 files changed, 38 insertions(+), 20 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index fb9c2a7ef..d7d79f1af 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -197,6 +197,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
struct qat_pmd_private *internals
= dev->data->dev_private;
qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->build_request = qat_sym_build_request;
dev->data->queue_pairs[queue_pair_id] = qp;
return 0;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 2dfdc9cce..4e404749a 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -86,10 +86,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
static inline uint32_t
adf_modulo(uint32_t data, uint32_t shift);
-static inline int
-qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp);
-
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
struct rte_crypto_op *op)
@@ -209,14 +205,12 @@ txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
q->csr_tail = q->tail;
}
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
+static uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
{
register struct qat_queue *queue;
struct qat_qp *tmp_qp = (struct qat_qp *)qp;
register uint32_t nb_ops_sent = 0;
- register struct rte_crypto_op **cur_op = ops;
register int ret;
uint16_t nb_ops_possible = nb_ops;
register uint8_t *base_addr;
@@ -242,8 +236,9 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
while (nb_ops_sent != nb_ops_possible) {
- ret = qat_sym_build_request(*cur_op, base_addr + tail,
- tmp_qp->op_cookies[tail / queue->msg_size], tmp_qp);
+ ret = tmp_qp->build_request(*ops, base_addr + tail,
+ tmp_qp->op_cookies[tail / queue->msg_size],
+ tmp_qp->qat_dev_gen);
if (ret != 0) {
tmp_qp->stats.enqueue_err_count++;
/*
@@ -257,8 +252,8 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
}
tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ ops++;
nb_ops_sent++;
- cur_op++;
}
kick_tail:
queue->tail = tail;
@@ -298,6 +293,13 @@ void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
q->hw_queue_number, new_head);
}
+uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
+}
+
uint16_t
qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
@@ -456,9 +458,10 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-static inline int
-qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
- struct qat_sym_op_cookie *qat_op_cookie, struct qat_qp *qp)
+
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen)
{
int ret = 0;
struct qat_sym_session *ctx;
@@ -471,6 +474,9 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
uint32_t min_ofs = 0;
uint64_t src_buf_start = 0, dst_buf_start = 0;
uint8_t do_sgl = 0;
+ struct rte_crypto_op *op = (struct rte_crypto_op *)in_op;
+ struct qat_sym_op_cookie *cookie =
+ (struct qat_sym_op_cookie *)op_cookie;
#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_TX
if (unlikely(op->type != RTE_CRYPTO_OP_TYPE_SYMMETRIC)) {
@@ -494,7 +500,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
return -EINVAL;
}
- if (unlikely(ctx->min_qat_dev_gen > qp->qat_dev_gen)) {
+ if (unlikely(ctx->min_qat_dev_gen > qat_dev_gen)) {
PMD_DRV_LOG(ERR, "Session alg not supported on this device gen");
op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION;
return -EINVAL;
@@ -807,7 +813,7 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
ICP_QAT_FW_COMN_PTR_TYPE_SET(qat_req->comn_hdr.comn_req_flags,
QAT_COMN_PTR_TYPE_SGL);
ret = qat_sgl_fill_array(op->sym->m_src, src_buf_start,
- &qat_op_cookie->qat_sgl_list_src,
+ &cookie->qat_sgl_list_src,
qat_req->comn_mid.src_length);
if (ret) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot fill sgl array");
@@ -817,11 +823,11 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
if (likely(op->sym->m_dst == NULL))
qat_req->comn_mid.dest_data_addr =
qat_req->comn_mid.src_data_addr =
- qat_op_cookie->qat_sgl_src_phys_addr;
+ cookie->qat_sgl_src_phys_addr;
else {
ret = qat_sgl_fill_array(op->sym->m_dst,
dst_buf_start,
- &qat_op_cookie->qat_sgl_list_dst,
+ &cookie->qat_sgl_list_dst,
qat_req->comn_mid.dst_length);
if (ret) {
@@ -831,9 +837,9 @@ qat_sym_build_request(struct rte_crypto_op *op, uint8_t *out_msg,
}
qat_req->comn_mid.src_data_addr =
- qat_op_cookie->qat_sgl_src_phys_addr;
+ cookie->qat_sgl_src_phys_addr;
qat_req->comn_mid.dest_data_addr =
- qat_op_cookie->qat_sgl_dst_phys_addr;
+ cookie->qat_sgl_dst_phys_addr;
}
} else {
qat_req->comn_mid.src_data_addr = src_buf_start;
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 18c77ea11..b1ddb6e93 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,6 +27,11 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
+typedef int (*build_request_t)(void *op,
+ uint8_t *req, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Build a request from an op. */
+
struct qat_sym_session;
/**
@@ -63,8 +68,14 @@ struct qat_qp {
void **op_cookies;
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
+ build_request_t build_request;
} __rte_cache_aligned;
+
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen);
+
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 09/38] crypto/qat: make dequeue function generic
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (7 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 08/38] crypto/qat: make enqueue function generic Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 10/38] crypto/qat: move generic qp fn to qp file Tomasz Jozwiak
` (29 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Queue-handling code in dequeue is made generic, so it can
be used by other services in future. This is done by
- Removing all sym-specific refs in input params - replace with void ptrs.
- Wrapping this generic dequeue with the sym-specific dequeue
called through the API.
- extracting the sym-specific response processing into a new fn.
- Setting a fn ptr for process_response in qp on qp creation
- Passing void * params to this, in the service-specific implementation
qat_sym_process_response cast back to sym structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 1 +
drivers/crypto/qat/qat_sym.c | 101 +++++++++++++++++++++--------------
drivers/crypto/qat/qat_sym.h | 10 ++++
3 files changed, 72 insertions(+), 40 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index d7d79f1af..bae6cf114 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -198,6 +198,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
= dev->data->dev_private;
qp->qat_dev_gen = internals->qat_dev_gen;
qp->build_request = qat_sym_build_request;
+ qp->process_response = qat_sym_process_response;
dev->data->queue_pairs[queue_pair_id] = qp;
return 0;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 4e404749a..2bae913a1 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -300,70 +300,91 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
}
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen)
+{
+
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
+
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
+
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
+
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+ }
+ *op = (void *)rx_op;
+
+ return 0;
+}
+
+static uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
{
struct qat_queue *rx_queue, *tx_queue;
struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- uint32_t msg_counter = 0;
- struct rte_crypto_op *rx_op;
- struct icp_qat_fw_comn_resp *resp_msg;
uint32_t head;
+ uint32_t resp_counter = 0;
+ uint8_t *resp_msg;
rx_queue = &(tmp_qp->rx_q);
tx_queue = &(tmp_qp->tx_q);
head = rx_queue->head;
- resp_msg = (struct icp_qat_fw_comn_resp *)
- ((uint8_t *)rx_queue->base_addr + head);
+ resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
- msg_counter != nb_ops) {
- rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess =
- (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
+ resp_counter != nb_ops) {
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
+ tmp_qp->process_response(ops, resp_msg,
+ tmp_qp->op_cookies[head / rx_queue->msg_size],
+ tmp_qp->qat_dev_gen);
head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
- resp_msg = (struct icp_qat_fw_comn_resp *)
- ((uint8_t *)rx_queue->base_addr + head);
- *ops = rx_op;
+
+ resp_msg = (uint8_t *)rx_queue->base_addr + head;
ops++;
- msg_counter++;
+ resp_counter++;
}
- if (msg_counter > 0) {
+ if (resp_counter > 0) {
rx_queue->head = head;
- tmp_qp->stats.dequeued_count += msg_counter;
- rx_queue->nb_processed_responses += msg_counter;
- tmp_qp->inflights16 -= msg_counter;
+ tmp_qp->stats.dequeued_count += resp_counter;
+ rx_queue->nb_processed_responses += resp_counter;
+ tmp_qp->inflights16 -= resp_counter;
if (rx_queue->nb_processed_responses > QAT_CSR_HEAD_WRITE_THRESH)
rxq_free_desc(tmp_qp, rx_queue);
}
/* also check if tail needs to be advanced */
if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
- tx_queue->tail != tx_queue->csr_tail) {
+ tx_queue->tail != tx_queue->csr_tail) {
txq_write_tail(tmp_qp, tx_queue);
}
- return msg_counter;
+ return resp_counter;
+}
+
+uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
static inline int
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index b1ddb6e93..279d3a3ae 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -32,6 +32,11 @@ typedef int (*build_request_t)(void *op,
enum qat_device_gen qat_dev_gen);
/**< Build a request from an op. */
+typedef int (*process_response_t)(void **ops,
+ uint8_t *resp, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Process a response descriptor and return the associated op. */
+
struct qat_sym_session;
/**
@@ -69,6 +74,7 @@ struct qat_qp {
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
+ process_response_t process_response;
} __rte_cache_aligned;
@@ -76,6 +82,10 @@ int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie, enum qat_device_gen qat_dev_gen);
+
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 10/38] crypto/qat: move generic qp fn to qp file
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (8 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 09/38] crypto/qat: make dequeue " Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 11/38] crypto/qat: separate sym-specific from generic qp setup Tomasz Jozwiak
` (28 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Move the generic enqueue and dequeue fns from
the qat_sym.c file to the qat_qp.c file
Move generic qp structs to a new qat_qp.h file
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 152 +++++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_qp.h | 63 +++++++++++++++
drivers/crypto/qat/qat_sym.c | 149 +---------------------------------
drivers/crypto/qat/qat_sym.h | 49 -----------
4 files changed, 216 insertions(+), 197 deletions(-)
create mode 100644 drivers/crypto/qat/qat_qp.h
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index bae6cf114..56ea10242 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,7 +13,9 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
+#include "qat_qp.h"
#include "qat_sym.h"
+
#include "adf_transport_access_macros.h"
#define ADF_MAX_SYM_DESC 4096
@@ -450,3 +452,153 @@ static void adf_configure_queues(struct qat_qp *qp)
WRITE_CSR_RING_CONFIG(qp->mmap_bar_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_config);
}
+
+
+static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
+{
+ uint32_t div = data >> shift;
+ uint32_t mult = div << shift;
+
+ return data - mult;
+}
+
+static inline void
+txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
+ WRITE_CSR_RING_TAIL(qp->mmap_bar_addr, q->hw_bundle_number,
+ q->hw_queue_number, q->tail);
+ q->nb_pending_requests = 0;
+ q->csr_tail = q->tail;
+}
+
+static inline
+void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
+{
+ uint32_t old_head, new_head;
+ uint32_t max_head;
+
+ old_head = q->csr_head;
+ new_head = q->head;
+ max_head = qp->nb_descriptors * q->msg_size;
+
+ /* write out free descriptors */
+ void *cur_desc = (uint8_t *)q->base_addr + old_head;
+
+ if (new_head < old_head) {
+ memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, max_head - old_head);
+ memset(q->base_addr, ADF_RING_EMPTY_SIG_BYTE, new_head);
+ } else {
+ memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, new_head - old_head);
+ }
+ q->nb_processed_responses = 0;
+ q->csr_head = new_head;
+
+ /* write current head to CSR */
+ WRITE_CSR_RING_HEAD(qp->mmap_bar_addr, q->hw_bundle_number,
+ q->hw_queue_number, new_head);
+}
+
+uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
+{
+ register struct qat_queue *queue;
+ struct qat_qp *tmp_qp = (struct qat_qp *)qp;
+ register uint32_t nb_ops_sent = 0;
+ register int ret;
+ uint16_t nb_ops_possible = nb_ops;
+ register uint8_t *base_addr;
+ register uint32_t tail;
+ int overflow;
+
+ if (unlikely(nb_ops == 0))
+ return 0;
+
+ /* read params used a lot in main loop into registers */
+ queue = &(tmp_qp->tx_q);
+ base_addr = (uint8_t *)queue->base_addr;
+ tail = queue->tail;
+
+ /* Find how many can actually fit on the ring */
+ tmp_qp->inflights16 += nb_ops;
+ overflow = tmp_qp->inflights16 - queue->max_inflights;
+ if (overflow > 0) {
+ tmp_qp->inflights16 -= overflow;
+ nb_ops_possible = nb_ops - overflow;
+ if (nb_ops_possible == 0)
+ return 0;
+ }
+
+ while (nb_ops_sent != nb_ops_possible) {
+ ret = tmp_qp->build_request(*ops, base_addr + tail,
+ tmp_qp->op_cookies[tail / queue->msg_size],
+ tmp_qp->qat_dev_gen);
+ if (ret != 0) {
+ tmp_qp->stats.enqueue_err_count++;
+ /*
+ * This message cannot be enqueued,
+ * decrease number of ops that wasn't sent
+ */
+ tmp_qp->inflights16 -= nb_ops_possible - nb_ops_sent;
+ if (nb_ops_sent == 0)
+ return 0;
+ goto kick_tail;
+ }
+
+ tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ ops++;
+ nb_ops_sent++;
+ }
+kick_tail:
+ queue->tail = tail;
+ tmp_qp->stats.enqueued_count += nb_ops_sent;
+ queue->nb_pending_requests += nb_ops_sent;
+ if (tmp_qp->inflights16 < QAT_CSR_TAIL_FORCE_WRITE_THRESH ||
+ queue->nb_pending_requests > QAT_CSR_TAIL_WRITE_THRESH) {
+ txq_write_tail(tmp_qp, queue);
+ }
+ return nb_ops_sent;
+}
+
+uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
+{
+ struct qat_queue *rx_queue, *tx_queue;
+ struct qat_qp *tmp_qp = (struct qat_qp *)qp;
+ uint32_t head;
+ uint32_t resp_counter = 0;
+ uint8_t *resp_msg;
+
+ rx_queue = &(tmp_qp->rx_q);
+ tx_queue = &(tmp_qp->tx_q);
+ head = rx_queue->head;
+ resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
+
+ while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
+ resp_counter != nb_ops) {
+
+ tmp_qp->process_response(ops, resp_msg,
+ tmp_qp->op_cookies[head / rx_queue->msg_size],
+ tmp_qp->qat_dev_gen);
+
+ head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
+
+ resp_msg = (uint8_t *)rx_queue->base_addr + head;
+ ops++;
+ resp_counter++;
+ }
+ if (resp_counter > 0) {
+ rx_queue->head = head;
+ tmp_qp->stats.dequeued_count += resp_counter;
+ rx_queue->nb_processed_responses += resp_counter;
+ tmp_qp->inflights16 -= resp_counter;
+
+ if (rx_queue->nb_processed_responses >
+ QAT_CSR_HEAD_WRITE_THRESH)
+ rxq_free_desc(tmp_qp, rx_queue);
+ }
+ /* also check if tail needs to be advanced */
+ if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
+ tx_queue->tail != tx_queue->csr_tail) {
+ txq_write_tail(tmp_qp, tx_queue);
+ }
+ return resp_counter;
+}
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
new file mode 100644
index 000000000..87d55c5f2
--- /dev/null
+++ b/drivers/crypto/qat/qat_qp.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#ifndef _QAT_QP_H_
+#define _QAT_QP_H_
+
+#include "qat_common.h"
+
+typedef int (*build_request_t)(void *op,
+ uint8_t *req, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Build a request from an op. */
+
+typedef int (*process_response_t)(void **ops,
+ uint8_t *resp, void *op_cookie,
+ enum qat_device_gen qat_dev_gen);
+/**< Process a response descriptor and return the associated op. */
+
+/**
+ * Structure associated with each queue.
+ */
+struct qat_queue {
+ char memz_name[RTE_MEMZONE_NAMESIZE];
+ void *base_addr; /* Base address */
+ rte_iova_t base_phys_addr; /* Queue physical address */
+ uint32_t head; /* Shadow copy of the head */
+ uint32_t tail; /* Shadow copy of the tail */
+ uint32_t modulo;
+ uint32_t msg_size;
+ uint16_t max_inflights;
+ uint32_t queue_size;
+ uint8_t hw_bundle_number;
+ uint8_t hw_queue_number;
+ /* HW queue aka ring offset on bundle */
+ uint32_t csr_head; /* last written head value */
+ uint32_t csr_tail; /* last written tail value */
+ uint16_t nb_processed_responses;
+ /* number of responses processed since last CSR head write */
+ uint16_t nb_pending_requests;
+ /* number of requests pending since last CSR tail write */
+};
+
+struct qat_qp {
+ void *mmap_bar_addr;
+ uint16_t inflights16;
+ struct qat_queue tx_q;
+ struct qat_queue rx_q;
+ struct rte_cryptodev_stats stats;
+ struct rte_mempool *op_cookie_pool;
+ void **op_cookies;
+ uint32_t nb_descriptors;
+ enum qat_device_gen qat_dev_gen;
+ build_request_t build_request;
+ process_response_t process_response;
+} __rte_cache_aligned;
+
+uint16_t
+qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+
+uint16_t
+qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+
+#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 2bae913a1..ab8ce2c96 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -14,6 +14,7 @@
#include "qat_logs.h"
#include "qat_sym_session.h"
#include "qat_sym.h"
+#include "qat_qp.h"
#include "adf_transport_access_macros.h"
#define BYTE_LENGTH 8
@@ -83,8 +84,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
/** Creates a context in either AES or DES in ECB mode
* Depends on openssl libcrypto
*/
-static inline uint32_t
-adf_modulo(uint32_t data, uint32_t shift);
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
@@ -197,102 +196,6 @@ qat_bpicipher_postprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-static inline void
-txq_write_tail(struct qat_qp *qp, struct qat_queue *q) {
- WRITE_CSR_RING_TAIL(qp->mmap_bar_addr, q->hw_bundle_number,
- q->hw_queue_number, q->tail);
- q->nb_pending_requests = 0;
- q->csr_tail = q->tail;
-}
-
-static uint16_t
-qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
-{
- register struct qat_queue *queue;
- struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- register uint32_t nb_ops_sent = 0;
- register int ret;
- uint16_t nb_ops_possible = nb_ops;
- register uint8_t *base_addr;
- register uint32_t tail;
- int overflow;
-
- if (unlikely(nb_ops == 0))
- return 0;
-
- /* read params used a lot in main loop into registers */
- queue = &(tmp_qp->tx_q);
- base_addr = (uint8_t *)queue->base_addr;
- tail = queue->tail;
-
- /* Find how many can actually fit on the ring */
- tmp_qp->inflights16 += nb_ops;
- overflow = tmp_qp->inflights16 - queue->max_inflights;
- if (overflow > 0) {
- tmp_qp->inflights16 -= overflow;
- nb_ops_possible = nb_ops - overflow;
- if (nb_ops_possible == 0)
- return 0;
- }
-
- while (nb_ops_sent != nb_ops_possible) {
- ret = tmp_qp->build_request(*ops, base_addr + tail,
- tmp_qp->op_cookies[tail / queue->msg_size],
- tmp_qp->qat_dev_gen);
- if (ret != 0) {
- tmp_qp->stats.enqueue_err_count++;
- /*
- * This message cannot be enqueued,
- * decrease number of ops that wasn't sent
- */
- tmp_qp->inflights16 -= nb_ops_possible - nb_ops_sent;
- if (nb_ops_sent == 0)
- return 0;
- goto kick_tail;
- }
-
- tail = adf_modulo(tail + queue->msg_size, queue->modulo);
- ops++;
- nb_ops_sent++;
- }
-kick_tail:
- queue->tail = tail;
- tmp_qp->stats.enqueued_count += nb_ops_sent;
- queue->nb_pending_requests += nb_ops_sent;
- if (tmp_qp->inflights16 < QAT_CSR_TAIL_FORCE_WRITE_THRESH ||
- queue->nb_pending_requests > QAT_CSR_TAIL_WRITE_THRESH) {
- txq_write_tail(tmp_qp, queue);
- }
- return nb_ops_sent;
-}
-
-static inline
-void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q)
-{
- uint32_t old_head, new_head;
- uint32_t max_head;
-
- old_head = q->csr_head;
- new_head = q->head;
- max_head = qp->nb_descriptors * q->msg_size;
-
- /* write out free descriptors */
- void *cur_desc = (uint8_t *)q->base_addr + old_head;
-
- if (new_head < old_head) {
- memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, max_head - old_head);
- memset(q->base_addr, ADF_RING_EMPTY_SIG_BYTE, new_head);
- } else {
- memset(cur_desc, ADF_RING_EMPTY_SIG_BYTE, new_head - old_head);
- }
- q->nb_processed_responses = 0;
- q->csr_head = new_head;
-
- /* write current head to CSR */
- WRITE_CSR_RING_HEAD(qp->mmap_bar_addr, q->hw_bundle_number,
- q->hw_queue_number, new_head);
-}
-
uint16_t
qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
uint16_t nb_ops)
@@ -336,49 +239,6 @@ qat_sym_process_response(void **op, uint8_t *resp,
return 0;
}
-static uint16_t
-qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
-{
- struct qat_queue *rx_queue, *tx_queue;
- struct qat_qp *tmp_qp = (struct qat_qp *)qp;
- uint32_t head;
- uint32_t resp_counter = 0;
- uint8_t *resp_msg;
-
- rx_queue = &(tmp_qp->rx_q);
- tx_queue = &(tmp_qp->tx_q);
- head = rx_queue->head;
- resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head;
-
- while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
- resp_counter != nb_ops) {
-
- tmp_qp->process_response(ops, resp_msg,
- tmp_qp->op_cookies[head / rx_queue->msg_size],
- tmp_qp->qat_dev_gen);
-
- head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
-
- resp_msg = (uint8_t *)rx_queue->base_addr + head;
- ops++;
- resp_counter++;
- }
- if (resp_counter > 0) {
- rx_queue->head = head;
- tmp_qp->stats.dequeued_count += resp_counter;
- rx_queue->nb_processed_responses += resp_counter;
- tmp_qp->inflights16 -= resp_counter;
-
- if (rx_queue->nb_processed_responses > QAT_CSR_HEAD_WRITE_THRESH)
- rxq_free_desc(tmp_qp, rx_queue);
- }
- /* also check if tail needs to be advanced */
- if (tmp_qp->inflights16 <= QAT_CSR_TAIL_FORCE_WRITE_THRESH &&
- tx_queue->tail != tx_queue->csr_tail) {
- txq_write_tail(tmp_qp, tx_queue);
- }
- return resp_counter;
-}
uint16_t
qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
@@ -903,13 +763,6 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
return 0;
}
-static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
-{
- uint32_t div = data >> shift;
- uint32_t mult = div << shift;
-
- return data - mult;
-}
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats)
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 279d3a3ae..39574eeb6 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -27,57 +27,8 @@
#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
/* number of inflights below which no tail write coalescing should occur */
-typedef int (*build_request_t)(void *op,
- uint8_t *req, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
-/**< Build a request from an op. */
-
-typedef int (*process_response_t)(void **ops,
- uint8_t *resp, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
-/**< Process a response descriptor and return the associated op. */
-
struct qat_sym_session;
-/**
- * Structure associated with each queue.
- */
-struct qat_queue {
- char memz_name[RTE_MEMZONE_NAMESIZE];
- void *base_addr; /* Base address */
- rte_iova_t base_phys_addr; /* Queue physical address */
- uint32_t head; /* Shadow copy of the head */
- uint32_t tail; /* Shadow copy of the tail */
- uint32_t modulo;
- uint32_t msg_size;
- uint16_t max_inflights;
- uint32_t queue_size;
- uint8_t hw_bundle_number;
- uint8_t hw_queue_number;
- /* HW queue aka ring offset on bundle */
- uint32_t csr_head; /* last written head value */
- uint32_t csr_tail; /* last written tail value */
- uint16_t nb_processed_responses;
- /* number of responses processed since last CSR head write */
- uint16_t nb_pending_requests;
- /* number of requests pending since last CSR tail write */
-};
-
-struct qat_qp {
- void *mmap_bar_addr;
- uint16_t inflights16;
- struct qat_queue tx_q;
- struct qat_queue rx_q;
- struct rte_cryptodev_stats stats;
- struct rte_mempool *op_cookie_pool;
- void **op_cookies;
- uint32_t nb_descriptors;
- enum qat_device_gen qat_dev_gen;
- build_request_t build_request;
- process_response_t process_response;
-} __rte_cache_aligned;
-
-
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 11/38] crypto/qat: separate sym-specific from generic qp setup
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (9 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 10/38] crypto/qat: move generic qp fn to qp file Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 12/38] crypto/qat: move sym-specific qp code to sym file Tomasz Jozwiak
` (27 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Extracted all sym-specific code from qp setup fns, leaving
generic qat_qp_setup fn and helper fns. Created a new
meta-data struct qat_qp_config to hold all the data needed
to create a qp, filled this out in the sym-specific code
and passed to the generic qp_setup fn.
No need now for rx and tx queue_create fns, one generic
queue_create fn replaces these.
Included the service name (e.g. "sym") in the qp memzone
and cookie pool names.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../qat/qat_adf/adf_transport_access_macros.h | 2 +
drivers/crypto/qat/qat_qp.c | 220 ++++++++++--------
drivers/crypto/qat/qat_qp.h | 17 ++
3 files changed, 137 insertions(+), 102 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index bfdbc979f..8b88b69de 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -52,6 +52,8 @@
#define ADF_NUM_BUNDLES_PER_DEV 1
#define ADF_NUM_SYM_QPS_PER_BUNDLE 2
+#define ADF_RING_DIR_TX 0
+#define ADF_RING_DIR_RX 1
/* Valid internal msg size values */
#define ADF_MSG_SIZE_32 0x01
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 56ea10242..5a543f6cb 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -18,8 +18,8 @@
#include "adf_transport_access_macros.h"
-#define ADF_MAX_SYM_DESC 4096
-#define ADF_MIN_SYM_DESC 128
+#define ADF_MAX_DESC 4096
+#define ADF_MIN_DESC 128
#define ADF_SYM_TX_RING_DESC_SIZE 128
#define ADF_SYM_RX_RING_DESC_SIZE 32
#define ADF_SYM_TX_QUEUE_STARTOFF 2
@@ -34,16 +34,9 @@
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
-static int qat_tx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t id, uint32_t nb_desc,
- int socket_id);
-static int qat_rx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t id, uint32_t nb_desc,
- int socket_id);
static void qat_queue_delete(struct qat_queue *queue);
static int qat_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint32_t nb_desc, uint8_t desc_size,
- int socket_id);
+ struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
static void adf_configure_queues(struct qat_qp *queue);
@@ -81,29 +74,19 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
+static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+ struct qat_qp_config *qat_qp_conf)
{
struct qat_qp *qp;
struct rte_pci_device *pci_dev;
- int ret;
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
- PMD_INIT_FUNC_TRACE();
- /* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[queue_pair_id] != NULL) {
- ret = qat_sym_qp_release(dev, queue_pair_id);
- if (ret < 0)
- return ret;
- }
-
- if ((qp_conf->nb_descriptors > ADF_MAX_SYM_DESC) ||
- (qp_conf->nb_descriptors < ADF_MIN_SYM_DESC)) {
+ if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
+ (qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
PMD_DRV_LOG(ERR, "Can't create qp for %u descriptors",
- qp_conf->nb_descriptors);
+ qat_qp_conf->nb_descriptors);
return -EINVAL;
}
@@ -115,13 +98,6 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EINVAL;
}
- if (queue_pair_id >=
- (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device",
- queue_pair_id);
- return -EINVAL;
- }
/* Allocate the queue pair data structure. */
qp = rte_zmalloc("qat PMD qp metadata",
sizeof(*qp), RTE_CACHE_LINE_SIZE);
@@ -129,9 +105,9 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
PMD_DRV_LOG(ERR, "Failed to alloc mem for qp struct");
return -ENOMEM;
}
- qp->nb_descriptors = qp_conf->nb_descriptors;
+ qp->nb_descriptors = qat_qp_conf->nb_descriptors;
qp->op_cookies = rte_zmalloc("qat PMD op cookie pointer",
- qp_conf->nb_descriptors * sizeof(*qp->op_cookies),
+ qat_qp_conf->nb_descriptors * sizeof(*qp->op_cookies),
RTE_CACHE_LINE_SIZE);
if (qp->op_cookies == NULL) {
PMD_DRV_LOG(ERR, "Failed to alloc mem for cookie");
@@ -142,15 +118,15 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qp->mmap_bar_addr = pci_dev->mem_resource[0].addr;
qp->inflights16 = 0;
- if (qat_tx_queue_create(dev, &(qp->tx_q),
- queue_pair_id, qp_conf->nb_descriptors, socket_id) != 0) {
+ if (qat_queue_create(dev, &(qp->tx_q), qat_qp_conf,
+ ADF_RING_DIR_TX) != 0) {
PMD_INIT_LOG(ERR, "Tx queue create failed "
"queue_pair_id=%u", queue_pair_id);
goto create_err;
}
- if (qat_rx_queue_create(dev, &(qp->rx_q),
- queue_pair_id, qp_conf->nb_descriptors, socket_id) != 0) {
+ if (qat_queue_create(dev, &(qp->rx_q), qat_qp_conf,
+ ADF_RING_DIR_RX) != 0) {
PMD_DRV_LOG(ERR, "Rx queue create failed "
"queue_pair_id=%hu", queue_pair_id);
qat_queue_delete(&(qp->tx_q));
@@ -159,16 +135,17 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
adf_configure_queues(qp);
adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
- snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_qp_op_%d_%hu",
- pci_dev->driver->driver.name, dev->data->dev_id,
- queue_pair_id);
+
+ snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
+ pci_dev->driver->driver.name, qat_qp_conf->service_str,
+ dev->data->dev_id, queue_pair_id);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
qp->nb_descriptors,
- sizeof(struct qat_sym_op_cookie), 64, 0,
- NULL, NULL, NULL, NULL, socket_id,
+ qat_qp_conf->cookie_size, 64, 0,
+ NULL, NULL, NULL, NULL, qat_qp_conf->socket_id,
0);
if (!qp->op_cookie_pool) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot create"
@@ -181,6 +158,67 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
PMD_DRV_LOG(ERR, "QAT PMD Cannot get op_cookie");
goto create_err;
}
+ }
+
+ struct qat_pmd_private *internals
+ = dev->data->dev_private;
+ qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->build_request = qat_qp_conf->build_request;
+ qp->process_response = qat_qp_conf->process_response;
+
+ dev->data->queue_pairs[queue_pair_id] = qp;
+ return 0;
+
+create_err:
+ rte_free(qp);
+ return -EFAULT;
+}
+
+
+
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (dev->data->queue_pairs[qp_id] != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+
+ qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
+ qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_TX_QUEUE_STARTOFF;
+ qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_RX_QUEUE_STARTOFF;
+ qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
+ qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
struct qat_sym_op_cookie *sql_cookie =
qp->op_cookies[i];
@@ -196,24 +234,11 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qat_sgl_list_dst);
}
- struct qat_pmd_private *internals
- = dev->data->dev_private;
- qp->qat_dev_gen = internals->qat_dev_gen;
- qp->build_request = qat_sym_build_request;
- qp->process_response = qat_sym_process_response;
+ return ret;
- dev->data->queue_pairs[queue_pair_id] = qp;
- return 0;
-
-create_err:
- if (qp->op_cookie_pool)
- rte_mempool_free(qp->op_cookie_pool);
- rte_free(qp->op_cookies);
- rte_free(qp);
- return -EFAULT;
}
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
@@ -247,38 +272,13 @@ int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
return 0;
}
-static int qat_tx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t qp_id,
- uint32_t nb_desc, int socket_id)
-{
- PMD_INIT_FUNC_TRACE();
- queue->hw_bundle_number = qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE;
- queue->hw_queue_number = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- PMD_DRV_LOG(DEBUG, "TX ring for %u msgs: qp_id %d, bundle %u, ring %u",
- nb_desc, qp_id, queue->hw_bundle_number,
- queue->hw_queue_number);
-
- return qat_queue_create(dev, queue, nb_desc,
- ADF_SYM_TX_RING_DESC_SIZE, socket_id);
-}
-static int qat_rx_queue_create(struct rte_cryptodev *dev,
- struct qat_queue *queue, uint8_t qp_id, uint32_t nb_desc,
- int socket_id)
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
- PMD_INIT_FUNC_TRACE();
- queue->hw_bundle_number = qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE;
- queue->hw_queue_number = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
-
- PMD_DRV_LOG(DEBUG, "RX ring for %u msgs: qp id %d, bundle %u, ring %u",
- nb_desc, qp_id, queue->hw_bundle_number,
- queue->hw_queue_number);
- return qat_queue_create(dev, queue, nb_desc,
- ADF_SYM_RX_RING_DESC_SIZE, socket_id);
+ return qat_qp_release(dev, queue_pair_id);
}
+
static void qat_queue_delete(struct qat_queue *queue)
{
const struct rte_memzone *mz;
@@ -304,15 +304,21 @@ static void qat_queue_delete(struct qat_queue *queue)
static int
qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
- uint32_t nb_desc, uint8_t desc_size, int socket_id)
+ struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
void *io_addr;
const struct rte_memzone *qp_mz;
- uint32_t queue_size_bytes = nb_desc*desc_size;
struct rte_pci_device *pci_dev;
+ int ret = 0;
+ uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
+ qp_conf->tx_msg_size : qp_conf->rx_msg_size);
+ uint32_t queue_size_bytes = (qp_conf->nb_descriptors)*(desc_size);
+
+ queue->hw_bundle_number = qp_conf->hw_bundle_num;
+ queue->hw_queue_number = (dir == ADF_RING_DIR_TX ?
+ qp_conf->tx_ring_num : qp_conf->rx_ring_num);
- PMD_INIT_FUNC_TRACE();
if (desc_size > ADF_MSG_SIZE_TO_BYTES(ADF_MAX_MSG_SIZE)) {
PMD_DRV_LOG(ERR, "Invalid descriptor size %d", desc_size);
return -EINVAL;
@@ -323,11 +329,13 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
/*
* Allocate a memzone for the queue - create a unique name.
*/
- snprintf(queue->memz_name, sizeof(queue->memz_name), "%s_%s_%d_%d_%d",
- pci_dev->driver->driver.name, "qp_mem", dev->data->dev_id,
+ snprintf(queue->memz_name, sizeof(queue->memz_name),
+ "%s_%s_%s_%d_%d_%d",
+ pci_dev->driver->driver.name, qp_conf->service_str,
+ "qp_mem", dev->data->dev_id,
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
- socket_id);
+ qp_conf->socket_id);
if (qp_mz == NULL) {
PMD_DRV_LOG(ERR, "Failed to allocate ring memzone");
return -ENOMEM;
@@ -340,27 +348,31 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
PMD_DRV_LOG(ERR, "Invalid alignment on queue create "
" 0x%"PRIx64"\n",
queue->base_phys_addr);
- return -EFAULT;
+ ret = -EFAULT;
+ goto queue_create_err;
}
- if (adf_verify_queue_size(desc_size, nb_desc, &(queue->queue_size))
- != 0) {
+ if (adf_verify_queue_size(desc_size, qp_conf->nb_descriptors,
+ &(queue->queue_size)) != 0) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
- return -EINVAL;
+ ret = -EINVAL;
+ goto queue_create_err;
}
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
- PMD_DRV_LOG(DEBUG, "RING size in CSR: %u, in bytes %u, nb msgs %u,"
- " msg_size %u, max_inflights %u modulo %u",
- queue->queue_size, queue_size_bytes,
- nb_desc, desc_size, queue->max_inflights,
- queue->modulo);
+ PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
+ " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ queue->memz_name,
+ queue->queue_size, queue_size_bytes,
+ qp_conf->nb_descriptors, desc_size,
+ queue->max_inflights, queue->modulo);
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
- return -EINVAL;
+ ret = -EINVAL;
+ goto queue_create_err;
}
queue->head = 0;
queue->tail = 0;
@@ -379,6 +391,10 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
WRITE_CSR_RING_BASE(io_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_base);
return 0;
+
+queue_create_err:
+ rte_memzone_free(qp_mz);
+ return ret;
}
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 87d55c5f2..edebb8773 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -16,6 +16,23 @@ typedef int (*process_response_t)(void **ops,
enum qat_device_gen qat_dev_gen);
/**< Process a response descriptor and return the associated op. */
+/**
+ * Structure with data needed for creation of queue pair.
+ */
+struct qat_qp_config {
+ uint8_t hw_bundle_num;
+ uint8_t tx_ring_num;
+ uint8_t rx_ring_num;
+ uint16_t tx_msg_size;
+ uint16_t rx_msg_size;
+ uint32_t nb_descriptors;
+ uint32_t cookie_size;
+ int socket_id;
+ build_request_t build_request;
+ process_response_t process_response;
+ const char *service_str;
+};
+
/**
* Structure associated with each queue.
*/
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 12/38] crypto/qat: move sym-specific qp code to sym file
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (10 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 11/38] crypto/qat: separate sym-specific from generic qp setup Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 13/38] crypto/qat: remove dependencies on cryptodev from common Tomasz Jozwiak
` (26 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Move sym qp setup code from qat_qp.c to qat_sym.c
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 78 ++----------------------------------
drivers/crypto/qat/qat_qp.h | 6 +++
drivers/crypto/qat/qat_sym.c | 75 ++++++++++++++++++++++++++++++++++
3 files changed, 84 insertions(+), 75 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 5a543f6cb..d1d2be34f 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -20,11 +20,7 @@
#define ADF_MAX_DESC 4096
#define ADF_MIN_DESC 128
-#define ADF_SYM_TX_RING_DESC_SIZE 128
-#define ADF_SYM_RX_RING_DESC_SIZE 32
-#define ADF_SYM_TX_QUEUE_STARTOFF 2
-/* Offset from bundle start to 1st Sym Tx queue */
-#define ADF_SYM_RX_QUEUE_STARTOFF 10
+
#define ADF_ARB_REG_SLOT 0x1000
#define ADF_ARB_RINGSRVARBEN_OFFSET 0x19C
@@ -74,7 +70,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
{
struct qat_qp *qp;
@@ -174,71 +170,7 @@ static int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
-{
- struct qat_qp *qp;
- int ret = 0;
- uint32_t i;
- struct qat_qp_config qat_qp_conf;
-
- /* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[qp_id] != NULL) {
- ret = qat_sym_qp_release(dev, qp_id);
- if (ret < 0)
- return ret;
- }
- if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
- return -EINVAL;
- }
-
-
- qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
- qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
- qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
- qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
- qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
- qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
- qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
- qat_qp_conf.socket_id = socket_id;
- qat_qp_conf.service_str = "sym";
-
- ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
- if (ret != 0)
- return ret;
-
- qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
-
- for (i = 0; i < qp->nb_descriptors; i++) {
-
- struct qat_sym_op_cookie *sql_cookie =
- qp->op_cookies[i];
-
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_src);
-
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_dst);
- }
-
- return ret;
-
-}
-
-static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
struct qat_qp *qp =
(struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
@@ -273,10 +205,6 @@ static int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
}
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
-{
- return qat_qp_release(dev, queue_pair_id);
-}
static void qat_queue_delete(struct qat_queue *queue)
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index edebb8773..0cdf37f61 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -77,4 +77,10 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
uint16_t
qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
+int
+qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id);
+
+int
+qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+ struct qat_qp_config *qat_qp_conf);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index ab8ce2c96..4f86f2258 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -23,6 +23,12 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
+#define ADF_SYM_TX_RING_DESC_SIZE 128
+#define ADF_SYM_RX_RING_DESC_SIZE 32
+#define ADF_SYM_TX_QUEUE_STARTOFF 2
+/* Offset from bundle start to 1st Sym Tx queue */
+#define ADF_SYM_RX_QUEUE_STARTOFF 10
+
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -798,3 +804,72 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
}
+
+
+
+int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+{
+ return qat_qp_release(dev, queue_pair_id);
+}
+
+int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (dev->data->queue_pairs[qp_id] != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
+ ADF_NUM_BUNDLES_PER_DEV)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+
+ qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
+ qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_TX_QUEUE_STARTOFF;
+ qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
+ ADF_SYM_RX_QUEUE_STARTOFF;
+ qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
+ qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
+
+ struct qat_sym_op_cookie *sql_cookie =
+ qp->op_cookies[i];
+
+ sql_cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_list_src);
+
+ sql_cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_list_dst);
+ }
+
+ return ret;
+
+}
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 13/38] crypto/qat: remove dependencies on cryptodev from common
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (11 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 12/38] crypto/qat: move sym-specific qp code to sym file Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 14/38] crypto/qat: move defines from sym to qp header file Tomasz Jozwiak
` (25 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Remove dependence on rte_cryptodev from common qp code
to facilitate being used by other device types in future.
Transferred required data into qat-specific structures.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.h | 11 ++++-
drivers/crypto/qat/qat_qp.c | 67 ++++++++++++++------------
drivers/crypto/qat/qat_qp.h | 8 ++-
drivers/crypto/qat/qat_sym.c | 15 ++++--
drivers/crypto/qat/rte_qat_cryptodev.c | 4 +-
5 files changed, 66 insertions(+), 39 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 2cb8e7612..64706abae 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -14,7 +14,11 @@ extern uint8_t cryptodev_qat_driver_id;
extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-/** private data structure for each QAT device */
+/** private data structure for each QAT device.
+ * In this context a QAT device is a device offering only one service,
+ * so there can be more than 1 device on a pci_dev (VF),
+ * one for symmetric crypto, one for compression
+ */
struct qat_pmd_private {
unsigned int max_nb_queue_pairs;
/**< Max number of queue pairs supported by device */
@@ -23,6 +27,11 @@ struct qat_pmd_private {
enum qat_device_gen qat_dev_gen;
/**< QAT device generation */
const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device capabilities */
+ struct rte_pci_device *pci_dev;
+ /**< PCI information. */
+ uint8_t dev_id;
+ /**< Device ID for this instance */
};
int qat_dev_config(struct rte_cryptodev *dev,
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index d1d2be34f..b831ab420 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -6,7 +6,6 @@
#include <rte_dev.h>
#include <rte_malloc.h>
#include <rte_memzone.h>
-#include <rte_cryptodev_pmd.h>
#include <rte_pci.h>
#include <rte_bus_pci.h>
#include <rte_atomic.h>
@@ -31,7 +30,7 @@
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
-static int qat_queue_create(struct rte_cryptodev *dev,
+static int qat_queue_create(struct qat_pmd_private *qat_dev,
struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
@@ -70,14 +69,19 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+int qat_qp_setup(struct qat_pmd_private *qat_dev,
+ struct qat_qp **qp_addr,
+ uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
+
{
struct qat_qp *qp;
- struct rte_pci_device *pci_dev;
+ struct rte_pci_device *pci_dev = qat_dev->pci_dev;
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
+ PMD_DRV_LOG(DEBUG, "Setup qp %u on device %d gen %d",
+ queue_pair_id, qat_dev->dev_id, qat_dev->qat_dev_gen);
if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
(qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
@@ -86,8 +90,6 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EINVAL;
}
- pci_dev = RTE_DEV_TO_PCI(dev->device);
-
if (pci_dev->mem_resource[0].addr == NULL) {
PMD_DRV_LOG(ERR, "Could not find VF config space "
"(UIO driver attached?).");
@@ -114,14 +116,14 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
qp->mmap_bar_addr = pci_dev->mem_resource[0].addr;
qp->inflights16 = 0;
- if (qat_queue_create(dev, &(qp->tx_q), qat_qp_conf,
+ if (qat_queue_create(qat_dev, &(qp->tx_q), qat_qp_conf,
ADF_RING_DIR_TX) != 0) {
PMD_INIT_LOG(ERR, "Tx queue create failed "
"queue_pair_id=%u", queue_pair_id);
goto create_err;
}
- if (qat_queue_create(dev, &(qp->rx_q), qat_qp_conf,
+ if (qat_queue_create(qat_dev, &(qp->rx_q), qat_qp_conf,
ADF_RING_DIR_RX) != 0) {
PMD_DRV_LOG(ERR, "Rx queue create failed "
"queue_pair_id=%hu", queue_pair_id);
@@ -134,7 +136,7 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
pci_dev->driver->driver.name, qat_qp_conf->service_str,
- dev->data->dev_id, queue_pair_id);
+ qat_dev->dev_id, queue_pair_id);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
@@ -156,13 +158,15 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
}
}
- struct qat_pmd_private *internals
- = dev->data->dev_private;
- qp->qat_dev_gen = internals->qat_dev_gen;
+ qp->qat_dev_gen = qat_dev->qat_dev_gen;
qp->build_request = qat_qp_conf->build_request;
qp->process_response = qat_qp_conf->process_response;
+ qp->qat_dev = qat_dev;
+
+ PMD_DRV_LOG(DEBUG, "QP setup complete: id: %d, cookiepool: %s",
+ queue_pair_id, op_cookie_pool_name);
- dev->data->queue_pairs[queue_pair_id] = qp;
+ *qp_addr = qp;
return 0;
create_err:
@@ -170,10 +174,9 @@ int qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
return -EFAULT;
}
-int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int qat_qp_release(struct qat_qp **qp_addr)
{
- struct qat_qp *qp =
- (struct qat_qp *)dev->data->queue_pairs[queue_pair_id];
+ struct qat_qp *qp = *qp_addr;
uint32_t i;
PMD_INIT_FUNC_TRACE();
@@ -182,6 +185,9 @@ int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
return 0;
}
+ PMD_DRV_LOG(DEBUG, "Free qp on qat_pci device %d",
+ qp->qat_dev->dev_id);
+
/* Don't free memory if there are still responses to be processed */
if (qp->inflights16 == 0) {
qat_queue_delete(&(qp->tx_q));
@@ -200,13 +206,11 @@ int qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
rte_free(qp->op_cookies);
rte_free(qp);
- dev->data->queue_pairs[queue_pair_id] = NULL;
+ *qp_addr = NULL;
return 0;
}
-
-
static void qat_queue_delete(struct qat_queue *queue)
{
const struct rte_memzone *mz;
@@ -216,6 +220,9 @@ static void qat_queue_delete(struct qat_queue *queue)
PMD_DRV_LOG(DEBUG, "Invalid queue");
return;
}
+ PMD_DRV_LOG(DEBUG, "Free ring %d, memzone: %s",
+ queue->hw_queue_number, queue->memz_name);
+
mz = rte_memzone_lookup(queue->memz_name);
if (mz != NULL) {
/* Write an unused pattern to the queue memory. */
@@ -231,13 +238,13 @@ static void qat_queue_delete(struct qat_queue *queue)
}
static int
-qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
+qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
void *io_addr;
const struct rte_memzone *qp_mz;
- struct rte_pci_device *pci_dev;
+ struct rte_pci_device *pci_dev = qat_dev->pci_dev;
int ret = 0;
uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
qp_conf->tx_msg_size : qp_conf->rx_msg_size);
@@ -252,15 +259,13 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
return -EINVAL;
}
- pci_dev = RTE_DEV_TO_PCI(dev->device);
-
/*
* Allocate a memzone for the queue - create a unique name.
*/
snprintf(queue->memz_name, sizeof(queue->memz_name),
"%s_%s_%s_%d_%d_%d",
pci_dev->driver->driver.name, qp_conf->service_str,
- "qp_mem", dev->data->dev_id,
+ "qp_mem", qat_dev->dev_id,
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
qp_conf->socket_id);
@@ -290,12 +295,6 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
- PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
- " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
- queue->memz_name,
- queue->queue_size, queue_size_bytes,
- qp_conf->nb_descriptors, desc_size,
- queue->max_inflights, queue->modulo);
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
@@ -318,6 +317,14 @@ qat_queue_create(struct rte_cryptodev *dev, struct qat_queue *queue,
WRITE_CSR_RING_BASE(io_addr, queue->hw_bundle_number,
queue->hw_queue_number, queue_base);
+
+ PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
+ " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ queue->memz_name,
+ queue->queue_size, queue_size_bytes,
+ qp_conf->nb_descriptors, desc_size,
+ queue->max_inflights, queue->modulo);
+
return 0;
queue_create_err:
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 0cdf37f61..7bd8fdcec 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,6 +5,7 @@
#define _QAT_QP_H_
#include "qat_common.h"
+#include "qat_device.h"
typedef int (*build_request_t)(void *op,
uint8_t *req, void *op_cookie,
@@ -69,6 +70,8 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
process_response_t process_response;
+ struct qat_pmd_private *qat_dev;
+ /**< qat device this qp is on */
} __rte_cache_aligned;
uint16_t
@@ -78,9 +81,10 @@ uint16_t
qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops);
int
-qat_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id);
+qat_qp_release(struct qat_qp **qp_addr);
int
-qat_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
+qat_qp_setup(struct qat_pmd_private *qat_dev,
+ struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 4f86f2258..8ab95ac43 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -809,7 +809,10 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
- return qat_qp_release(dev, queue_pair_id);
+ PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
+ queue_pair_id, dev->data->dev_id);
+ return qat_qp_release((struct qat_qp **)
+ &(dev->data->queue_pairs[queue_pair_id]));
}
int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
@@ -820,9 +823,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
int ret = 0;
uint32_t i;
struct qat_qp_config qat_qp_conf;
+ struct qat_qp **qp_addr =
+ (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
+ struct qat_pmd_private *qat_private = dev->data->dev_private;
/* If qp is already in use free ring memory and qp metadata. */
- if (dev->data->queue_pairs[qp_id] != NULL) {
+ if (*qp_addr != NULL) {
ret = qat_sym_qp_release(dev, qp_id);
if (ret < 0)
return ret;
@@ -833,7 +839,6 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
return -EINVAL;
}
-
qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
ADF_SYM_TX_QUEUE_STARTOFF;
@@ -848,11 +853,11 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.socket_id = socket_id;
qat_qp_conf.service_str = "sym";
- ret = qat_qp_setup(dev, qp_id, &qat_qp_conf);
+ ret = qat_qp_setup(qat_private, qp_addr, qp_id, &qat_qp_conf);
if (ret != 0)
return ret;
- qp = (struct qat_qp *)dev->data->queue_pairs[qp_id];
+ qp = (struct qat_qp *)*qp_addr;
for (i = 0; i < qp->nb_descriptors; i++) {
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 05cb5ef40..fe19b18b6 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -96,7 +96,9 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
internals = cryptodev->data->dev_private;
internals->max_nb_sessions = init_params->max_nb_sessions;
- switch (pci_dev->id.device_id) {
+ internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
+ internals->dev_id = cryptodev->data->dev_id;
+ switch (internals->pci_dev->id.device_id) {
case 0x0443:
internals->qat_dev_gen = QAT_GEN1;
internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 14/38] crypto/qat: move defines from sym to qp header file
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (12 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 13/38] crypto/qat: remove dependencies on cryptodev from common Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 15/38] crypto/qat: create structures to support various generations Tomasz Jozwiak
` (24 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Move defines related to coalescing from sym header file to qp header
file as these will be common for all services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.h | 7 +++++++
drivers/crypto/qat/qat_sym.h | 7 -------
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 7bd8fdcec..8cf072c55 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -7,6 +7,13 @@
#include "qat_common.h"
#include "qat_device.h"
+#define QAT_CSR_HEAD_WRITE_THRESH 32U
+/* number of requests to accumulate before writing head CSR */
+#define QAT_CSR_TAIL_WRITE_THRESH 32U
+/* number of requests to accumulate before writing tail CSR */
+#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
+/* number of inflights below which no tail write coalescing should occur */
+
typedef int (*build_request_t)(void *op,
uint8_t *req, void *op_cookie,
enum qat_device_gen qat_dev_gen);
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 39574eeb6..b92ec72de 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -20,13 +20,6 @@
(((num) + (align) - 1) & ~((align) - 1))
#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
-#define QAT_CSR_HEAD_WRITE_THRESH 32U
-/* number of requests to accumulate before writing head CSR */
-#define QAT_CSR_TAIL_WRITE_THRESH 32U
-/* number of requests to accumulate before writing tail CSR */
-#define QAT_CSR_TAIL_FORCE_WRITE_THRESH 256U
-/* number of inflights below which no tail write coalescing should occur */
-
struct qat_sym_session;
int
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 15/38] crypto/qat: create structures to support various generations
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (13 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 14/38] crypto/qat: move defines from sym to qp header file Tomasz Jozwiak
@ 2018-06-13 12:13 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 16/38] crypto/qat: rename sgl related objects Tomasz Jozwiak
` (23 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:13 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Create data structures to support different generations
of qat hardware supplying services through different queue pairs.
- Add two new structs qat_gen_hw_data and qat_qp_hw_dat
- Add a qat_service_type enum
An array of qat_qp_hw_data elements is
initialised with constants, these are arranged so that the qp_id used
on the API can be used as an index to pick up the qp data to use.
The constants are common to current generations,
new arrays will be added for future generations.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../qat/qat_adf/adf_transport_access_macros.h | 6 +-
drivers/crypto/qat/qat_common.h | 8 ++
drivers/crypto/qat/qat_device.c | 21 ++++-
drivers/crypto/qat/qat_device.h | 9 ++
drivers/crypto/qat/qat_qp.c | 94 ++++++++++++++++++-
drivers/crypto/qat/qat_qp.h | 18 +++-
drivers/crypto/qat/qat_sym.c | 27 ++----
7 files changed, 154 insertions(+), 29 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index 8b88b69de..2136d54ab 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -51,7 +51,8 @@
#define ADF_DEFAULT_RING_SIZE ADF_RING_SIZE_16K
#define ADF_NUM_BUNDLES_PER_DEV 1
-#define ADF_NUM_SYM_QPS_PER_BUNDLE 2
+/* Maximum number of qps for any service type */
+#define ADF_MAX_QPS_PER_BUNDLE 4
#define ADF_RING_DIR_TX 0
#define ADF_RING_DIR_RX 1
@@ -132,4 +133,5 @@ do { \
#define WRITE_CSR_INT_FLAG_AND_COL(csr_base_addr, bank, value) \
ADF_CSR_WR(csr_base_addr, (ADF_RING_BUNDLE_SIZE * bank) + \
ADF_RING_CSR_INT_FLAG_AND_COL, value)
-#endif
+
+#endif /*ADF_TRANSPORT_ACCESS_MACROS_H */
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 7802e96f9..c3e7bd9a7 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -23,6 +23,14 @@ enum qat_device_gen {
QAT_GEN2,
};
+enum qat_service_type {
+ QAT_SERVICE_ASYMMETRIC = 0,
+ QAT_SERVICE_SYMMETRIC,
+ QAT_SERVICE_COMPRESSION,
+ QAT_SERVICE_INVALID
+};
+#define QAT_MAX_SERVICES (QAT_SERVICE_INVALID)
+
/**< Common struct for scatter-gather list operations */
struct qat_alg_buf {
uint32_t len;
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index ac6bd1af6..cdf4f7058 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -4,6 +4,21 @@
#include "qat_device.h"
#include "adf_transport_access_macros.h"
+#include "qat_qp.h"
+
+/* Hardware device information per generation */
+__extension__
+struct qat_gen_hw_data qp_gen_config[] = {
+ [QAT_GEN1] = {
+ .dev_gen = QAT_GEN1,
+ .qp_hw_data = qat_gen1_qps,
+ },
+ [QAT_GEN2] = {
+ .dev_gen = QAT_GEN2,
+ .qp_hw_data = qat_gen1_qps,
+ /* gen2 has same ring layout as gen1 */
+ },
+};
int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
@@ -42,12 +57,14 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info)
{
struct qat_pmd_private *internals = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[internals->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
if (info != NULL) {
info->max_nb_queue_pairs =
- ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV;
+ qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
info->feature_flags = dev->feature_flags;
info->capabilities = internals->qat_dev_capabilities;
info->sym.max_nb_sessions = internals->max_nb_sessions;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 64706abae..0983e3c2e 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -8,6 +8,8 @@
#include <rte_bus_pci.h>
#include "qat_common.h"
#include "qat_logs.h"
+#include "adf_transport_access_macros.h"
+#include "qat_qp.h"
extern uint8_t cryptodev_qat_driver_id;
@@ -34,6 +36,13 @@ struct qat_pmd_private {
/**< Device ID for this instance */
};
+struct qat_gen_hw_data {
+ enum qat_device_gen dev_gen;
+ const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
+};
+
+extern struct qat_gen_hw_data qp_gen_config[];
+
int qat_dev_config(struct rte_cryptodev *dev,
struct rte_cryptodev_config *config);
int qat_dev_start(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index b831ab420..656645e4c 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -13,10 +13,10 @@
#include "qat_logs.h"
#include "qat_qp.h"
-#include "qat_sym.h"
-
+#include "qat_device.h"
#include "adf_transport_access_macros.h"
+
#define ADF_MAX_DESC 4096
#define ADF_MIN_DESC 128
@@ -27,6 +27,78 @@
ADF_CSR_WR(csr_addr, ADF_ARB_RINGSRVARBEN_OFFSET + \
(ADF_ARB_REG_SLOT * index), value)
+__extension__
+const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
+ [ADF_MAX_QPS_PER_BUNDLE] = {
+ /* queue pairs which provide an asymmetric crypto service */
+ [QAT_SERVICE_ASYMMETRIC] = {
+ {
+ .service_type = QAT_SERVICE_ASYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 0,
+ .rx_ring_num = 8,
+ .tx_msg_size = 64,
+ .rx_msg_size = 32,
+
+ }, {
+ .service_type = QAT_SERVICE_ASYMMETRIC,
+ .tx_ring_num = 1,
+ .rx_ring_num = 9,
+ .tx_msg_size = 64,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ },
+ /* queue pairs which provide a symmetric crypto service */
+ [QAT_SERVICE_SYMMETRIC] = {
+ {
+ .service_type = QAT_SERVICE_SYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 2,
+ .rx_ring_num = 10,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ },
+ {
+ .service_type = QAT_SERVICE_SYMMETRIC,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 3,
+ .rx_ring_num = 11,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ },
+ /* queue pairs which provide a compression service */
+ [QAT_SERVICE_COMPRESSION] = {
+ {
+ .service_type = QAT_SERVICE_COMPRESSION,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 6,
+ .rx_ring_num = 14,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_COMPRESSION,
+ .hw_bundle_num = 0,
+ .tx_ring_num = 7,
+ .rx_ring_num = 15,
+ .tx_msg_size = 128,
+ .rx_msg_size = 32,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }, {
+ .service_type = QAT_SERVICE_INVALID,
+ }
+ }
+};
+
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
@@ -38,6 +110,18 @@ static void adf_configure_queues(struct qat_qp *queue);
static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr);
static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr);
+
+int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
+ enum qat_service_type service)
+{
+ int i, count;
+
+ for (i = 0, count = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++)
+ if (qp_hw_data[i].service_type == service)
+ count++;
+ return count * ADF_NUM_BUNDLES_PER_DEV;
+}
+
static const struct rte_memzone *
queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
int socket_id)
@@ -247,12 +331,12 @@ qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
struct rte_pci_device *pci_dev = qat_dev->pci_dev;
int ret = 0;
uint16_t desc_size = (dir == ADF_RING_DIR_TX ?
- qp_conf->tx_msg_size : qp_conf->rx_msg_size);
+ qp_conf->hw->tx_msg_size : qp_conf->hw->rx_msg_size);
uint32_t queue_size_bytes = (qp_conf->nb_descriptors)*(desc_size);
- queue->hw_bundle_number = qp_conf->hw_bundle_num;
+ queue->hw_bundle_number = qp_conf->hw->hw_bundle_num;
queue->hw_queue_number = (dir == ADF_RING_DIR_TX ?
- qp_conf->tx_ring_num : qp_conf->rx_ring_num);
+ qp_conf->hw->tx_ring_num : qp_conf->hw->rx_ring_num);
if (desc_size > ADF_MSG_SIZE_TO_BYTES(ADF_MAX_MSG_SIZE)) {
PMD_DRV_LOG(ERR, "Invalid descriptor size %d", desc_size);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 8cf072c55..f808e16a5 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,7 +5,8 @@
#define _QAT_QP_H_
#include "qat_common.h"
-#include "qat_device.h"
+#include <rte_cryptodev_pmd.h>
+#include "adf_transport_access_macros.h"
#define QAT_CSR_HEAD_WRITE_THRESH 32U
/* number of requests to accumulate before writing head CSR */
@@ -27,12 +28,19 @@ typedef int (*process_response_t)(void **ops,
/**
* Structure with data needed for creation of queue pair.
*/
-struct qat_qp_config {
+struct qat_qp_hw_data {
+ enum qat_service_type service_type;
uint8_t hw_bundle_num;
uint8_t tx_ring_num;
uint8_t rx_ring_num;
uint16_t tx_msg_size;
uint16_t rx_msg_size;
+};
+/**
+ * Structure with data needed for creation of queue pair.
+ */
+struct qat_qp_config {
+ const struct qat_qp_hw_data *hw;
uint32_t nb_descriptors;
uint32_t cookie_size;
int socket_id;
@@ -81,6 +89,8 @@ struct qat_qp {
/**< qat device this qp is on */
} __rte_cache_aligned;
+extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_PER_BUNDLE];
+
uint16_t
qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
@@ -94,4 +104,8 @@ int
qat_qp_setup(struct qat_pmd_private *qat_dev,
struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
+
+int
+qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
+ enum qat_service_type service);
#endif /* _QAT_QP_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 8ab95ac43..e448dc859 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -16,6 +16,7 @@
#include "qat_sym.h"
#include "qat_qp.h"
#include "adf_transport_access_macros.h"
+#include "qat_device.h"
#define BYTE_LENGTH 8
/* bpi is only used for partial blocks of DES and AES
@@ -23,12 +24,6 @@
*/
#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-#define ADF_SYM_TX_RING_DESC_SIZE 128
-#define ADF_SYM_RX_RING_DESC_SIZE 32
-#define ADF_SYM_TX_QUEUE_STARTOFF 2
-/* Offset from bundle start to 1st Sym Tx queue */
-#define ADF_SYM_RX_QUEUE_STARTOFF 10
-
/** Encrypt a single partial block
* Depends on openssl libcrypto
* Uses ECB+XOR to do CFB encryption, same result, more performant
@@ -805,12 +800,11 @@ void qat_sym_stats_reset(struct rte_cryptodev *dev)
PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
}
-
-
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
queue_pair_id, dev->data->dev_id);
+
return qat_qp_release((struct qat_qp **)
&(dev->data->queue_pairs[queue_pair_id]));
}
@@ -823,9 +817,14 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
int ret = 0;
uint32_t i;
struct qat_qp_config qat_qp_conf;
+
struct qat_qp **qp_addr =
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
struct qat_pmd_private *qat_private = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[qat_private->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+ const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
/* If qp is already in use free ring memory and qp metadata. */
if (*qp_addr != NULL) {
@@ -833,19 +832,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
if (ret < 0)
return ret;
}
- if (qp_id >= (ADF_NUM_SYM_QPS_PER_BUNDLE *
- ADF_NUM_BUNDLES_PER_DEV)) {
+ if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
return -EINVAL;
}
- qat_qp_conf.hw_bundle_num = (qp_id/ADF_NUM_SYM_QPS_PER_BUNDLE);
- qat_qp_conf.tx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_TX_QUEUE_STARTOFF;
- qat_qp_conf.rx_ring_num = (qp_id%ADF_NUM_SYM_QPS_PER_BUNDLE) +
- ADF_SYM_RX_QUEUE_STARTOFF;
- qat_qp_conf.tx_msg_size = ADF_SYM_TX_RING_DESC_SIZE;
- qat_qp_conf.rx_msg_size = ADF_SYM_RX_RING_DESC_SIZE;
+ qat_qp_conf.hw = qp_hw_data;
qat_qp_conf.build_request = qat_sym_build_request;
qat_qp_conf.process_response = qat_sym_process_response;
qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
@@ -876,5 +868,4 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
}
return ret;
-
}
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 16/38] crypto/qat: rename sgl related objects
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (14 preceding siblings ...)
2018-06-13 12:13 ` [PATCH v3 15/38] crypto/qat: create structures to support various generations Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 17/38] crypto/qat: move sgl related element to appropriate files Tomasz Jozwiak
` (22 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Change SGL (Scatter-Gather List) related structs and member names
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
---
drivers/crypto/qat/qat_common.h | 10 +++++-----
drivers/crypto/qat/qat_sym.c | 28 ++++++++++++++--------------
2 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index c3e7bd9a7..193639550 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -32,22 +32,22 @@ enum qat_service_type {
#define QAT_MAX_SERVICES (QAT_SERVICE_INVALID)
/**< Common struct for scatter-gather list operations */
-struct qat_alg_buf {
+struct qat_flat_buf {
uint32_t len;
uint32_t resrvd;
uint64_t addr;
} __rte_packed;
-struct qat_alg_buf_list {
+struct qat_sgl {
uint64_t resrvd;
uint32_t num_bufs;
uint32_t num_mapped_bufs;
- struct qat_alg_buf bufers[QAT_SGL_MAX_NUMBER];
+ struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
struct qat_sym_op_cookie {
- struct qat_alg_buf_list qat_sgl_list_src;
- struct qat_alg_buf_list qat_sgl_list_dst;
+ struct qat_sgl qat_sgl_src;
+ struct qat_sgl qat_sgl_dst;
phys_addr_t qat_sgl_src_phys_addr;
phys_addr_t qat_sgl_dst_phys_addr;
};
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index e448dc859..a9beff064 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -250,20 +250,20 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
static inline int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
- struct qat_alg_buf_list *list, uint32_t data_len)
+ struct qat_sgl *list, uint32_t data_len)
{
int nr = 1;
uint32_t buf_len = rte_pktmbuf_iova(buf) -
buff_start + rte_pktmbuf_data_len(buf);
- list->bufers[0].addr = buff_start;
- list->bufers[0].resrvd = 0;
- list->bufers[0].len = buf_len;
+ list->buffers[0].addr = buff_start;
+ list->buffers[0].resrvd = 0;
+ list->buffers[0].len = buf_len;
if (data_len <= buf_len) {
list->num_bufs = nr;
- list->bufers[0].len = data_len;
+ list->buffers[0].len = data_len;
return 0;
}
@@ -276,15 +276,15 @@ qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
return -EINVAL;
}
- list->bufers[nr].len = rte_pktmbuf_data_len(buf);
- list->bufers[nr].resrvd = 0;
- list->bufers[nr].addr = rte_pktmbuf_iova(buf);
+ list->buffers[nr].len = rte_pktmbuf_data_len(buf);
+ list->buffers[nr].resrvd = 0;
+ list->buffers[nr].addr = rte_pktmbuf_iova(buf);
- buf_len += list->bufers[nr].len;
+ buf_len += list->buffers[nr].len;
buf = buf->next;
if (buf_len > data_len) {
- list->bufers[nr].len -=
+ list->buffers[nr].len -=
buf_len - data_len;
buf = NULL;
}
@@ -695,7 +695,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
ICP_QAT_FW_COMN_PTR_TYPE_SET(qat_req->comn_hdr.comn_req_flags,
QAT_COMN_PTR_TYPE_SGL);
ret = qat_sgl_fill_array(op->sym->m_src, src_buf_start,
- &cookie->qat_sgl_list_src,
+ &cookie->qat_sgl_src,
qat_req->comn_mid.src_length);
if (ret) {
PMD_DRV_LOG(ERR, "QAT PMD Cannot fill sgl array");
@@ -709,7 +709,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
else {
ret = qat_sgl_fill_array(op->sym->m_dst,
dst_buf_start,
- &cookie->qat_sgl_list_dst,
+ &cookie->qat_sgl_dst,
qat_req->comn_mid.dst_length);
if (ret) {
@@ -859,12 +859,12 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
sql_cookie->qat_sgl_src_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_src);
+ qat_sgl_src);
sql_cookie->qat_sgl_dst_phys_addr =
rte_mempool_virt2iova(sql_cookie) +
offsetof(struct qat_sym_op_cookie,
- qat_sgl_list_dst);
+ qat_sgl_dst);
}
return ret;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 17/38] crypto/qat: move sgl related element to appropriate files
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (15 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 16/38] crypto/qat: rename sgl related objects Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 18/38] crypto/qat: add QAT PCI device struct Tomasz Jozwiak
` (21 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Move SGL (Scatter-Gather List) related functions to common file
Move qat_sym_op_cookie struct to sym header file
Signed-off-by: ArkadiuszX Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 1 +
drivers/crypto/qat/meson.build | 1 +
drivers/crypto/qat/qat_common.c | 53 +++++++++++++++++++++++++++++++++
drivers/crypto/qat/qat_common.h | 11 ++++---
drivers/crypto/qat/qat_sym.c | 47 -----------------------------
drivers/crypto/qat/qat_sym.h | 7 +++++
6 files changed, 67 insertions(+), 53 deletions(-)
create mode 100644 drivers/crypto/qat/qat_common.c
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 8cb802b9d..902c47ff4 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_common.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
# export include files
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index e596006da..12910c377 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -7,6 +7,7 @@ if not dep.found()
endif
sources = files('qat_sym.c', 'qat_qp.c',
'qat_sym_session.c',
+ 'qat_common.c',
'rte_qat_cryptodev.c',
'qat_device.c')
includes += include_directories('qat_adf')
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
new file mode 100644
index 000000000..a8865904f
--- /dev/null
+++ b/drivers/crypto/qat/qat_common.c
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_common.h"
+#include "qat_logs.h"
+
+int
+qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
+ struct qat_sgl *list, uint32_t data_len)
+{
+ int nr = 1;
+
+ uint32_t buf_len = rte_pktmbuf_iova(buf) -
+ buf_start + rte_pktmbuf_data_len(buf);
+
+ list->buffers[0].addr = buf_start;
+ list->buffers[0].resrvd = 0;
+ list->buffers[0].len = buf_len;
+
+ if (data_len <= buf_len) {
+ list->num_bufs = nr;
+ list->buffers[0].len = data_len;
+ return 0;
+ }
+
+ buf = buf->next;
+ while (buf) {
+ if (unlikely(nr == QAT_SGL_MAX_NUMBER)) {
+ PMD_DRV_LOG(ERR,
+ "QAT PMD exceeded size of QAT SGL entry(%u)",
+ QAT_SGL_MAX_NUMBER);
+ return -EINVAL;
+ }
+
+ list->buffers[nr].len = rte_pktmbuf_data_len(buf);
+ list->buffers[nr].resrvd = 0;
+ list->buffers[nr].addr = rte_pktmbuf_iova(buf);
+
+ buf_len += list->buffers[nr].len;
+ buf = buf->next;
+
+ if (buf_len > data_len) {
+ list->buffers[nr].len -=
+ buf_len - data_len;
+ buf = NULL;
+ }
+ ++nr;
+ }
+ list->num_bufs = nr;
+
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 193639550..77ffc8f72 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -6,6 +6,8 @@
#include <stdint.h>
+#include <rte_mbuf.h>
+
/**< Intel(R) QAT Symmetric Crypto PMD device name */
#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
@@ -45,11 +47,8 @@ struct qat_sgl {
struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
-struct qat_sym_op_cookie {
- struct qat_sgl qat_sgl_src;
- struct qat_sgl qat_sgl_dst;
- phys_addr_t qat_sgl_src_phys_addr;
- phys_addr_t qat_sgl_dst_phys_addr;
-};
+int
+qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
+ struct qat_sgl *list, uint32_t data_len);
#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index a9beff064..b74dfa634 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -248,53 +248,6 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
-static inline int
-qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buff_start,
- struct qat_sgl *list, uint32_t data_len)
-{
- int nr = 1;
-
- uint32_t buf_len = rte_pktmbuf_iova(buf) -
- buff_start + rte_pktmbuf_data_len(buf);
-
- list->buffers[0].addr = buff_start;
- list->buffers[0].resrvd = 0;
- list->buffers[0].len = buf_len;
-
- if (data_len <= buf_len) {
- list->num_bufs = nr;
- list->buffers[0].len = data_len;
- return 0;
- }
-
- buf = buf->next;
- while (buf) {
- if (unlikely(nr == QAT_SGL_MAX_NUMBER)) {
- PMD_DRV_LOG(ERR, "QAT PMD exceeded size of QAT SGL"
- " entry(%u)",
- QAT_SGL_MAX_NUMBER);
- return -EINVAL;
- }
-
- list->buffers[nr].len = rte_pktmbuf_data_len(buf);
- list->buffers[nr].resrvd = 0;
- list->buffers[nr].addr = rte_pktmbuf_iova(buf);
-
- buf_len += list->buffers[nr].len;
- buf = buf->next;
-
- if (buf_len > data_len) {
- list->buffers[nr].len -=
- buf_len - data_len;
- buf = NULL;
- }
- ++nr;
- }
- list->num_bufs = nr;
-
- return 0;
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index b92ec72de..37bec3ce3 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -22,6 +22,13 @@
struct qat_sym_session;
+struct qat_sym_op_cookie {
+ struct qat_sgl qat_sgl_src;
+ struct qat_sgl qat_sgl_dst;
+ phys_addr_t qat_sgl_src_phys_addr;
+ phys_addr_t qat_sgl_dst_phys_addr;
+};
+
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 18/38] crypto/qat: add QAT PCI device struct
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (16 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 17/38] crypto/qat: move sgl related element to appropriate files Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 19/38] crypto/qat: use generic driver name for PCI registration Tomasz Jozwiak
` (20 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
- Added struct qat_pci_device to use internally in QAT PMD
to avoid dependencies on rte_cryptodev or rte_compressdev
- Added a global array of these
- Restructured probe/release to separate QAT common init/clear
from crypto pmd create/destroy.
- In QAT common part allocated a qat_pci_device and populated it
- Removed meaningless check in probe for RTE_PROC_PRIMARY
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 127 ++++++++++++++++++++++
drivers/crypto/qat/qat_device.h | 60 ++++++++++-
drivers/crypto/qat/rte_qat_cryptodev.c | 140 ++++++++++++++++++-------
3 files changed, 286 insertions(+), 41 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index cdf4f7058..75af1e8bc 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -20,6 +20,10 @@ struct qat_gen_hw_data qp_gen_config[] = {
},
};
+
+static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
+static int qat_nb_pci_devices;
+
int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
{
@@ -72,3 +76,126 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
info->pci_dev = RTE_DEV_TO_PCI(dev->device);
}
}
+
+
+static struct qat_pci_device *
+qat_pci_get_dev(uint8_t dev_id)
+{
+ return &qat_pci_devices[dev_id];
+}
+static struct qat_pci_device *
+qat_pci_get_named_dev(const char *name)
+{
+ struct qat_pci_device *dev;
+ unsigned int i;
+
+ if (name == NULL)
+ return NULL;
+
+ for (i = 0; i < QAT_MAX_PCI_DEVICES; i++) {
+ dev = &qat_pci_devices[i];
+
+ if ((dev->attached == QAT_ATTACHED) &&
+ (strcmp(dev->name, name) == 0))
+ return dev;
+ }
+
+ return NULL;
+}
+
+static uint8_t
+qat_pci_find_free_device_index(void)
+{
+ uint8_t dev_id;
+
+ for (dev_id = 0; dev_id < QAT_MAX_PCI_DEVICES; dev_id++) {
+ if (qat_pci_devices[dev_id].attached == QAT_DETACHED)
+ break;
+ }
+ return dev_id;
+}
+
+struct qat_pci_device *
+qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev)
+{
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+
+ return qat_pci_get_named_dev(name);
+}
+
+struct qat_pci_device *
+qat_pci_device_allocate(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_dev;
+ uint8_t qat_dev_id;
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+
+ if (qat_pci_get_named_dev(name) != NULL) {
+ PMD_DRV_LOG(ERR, "QAT device with name %s already allocated!",
+ name);
+ return NULL;
+ }
+
+ qat_dev_id = qat_pci_find_free_device_index();
+ if (qat_dev_id == QAT_MAX_PCI_DEVICES) {
+ PMD_DRV_LOG(ERR, "Reached maximum number of QAT devices");
+ return NULL;
+ }
+
+ qat_dev = qat_pci_get_dev(qat_dev_id);
+ snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
+ qat_dev->pci_dev = pci_dev;
+ switch (qat_dev->pci_dev->id.device_id) {
+ case 0x0443:
+ qat_dev->qat_dev_gen = QAT_GEN1;
+ break;
+ case 0x37c9:
+ case 0x19e3:
+ case 0x6f55:
+ qat_dev->qat_dev_gen = QAT_GEN2;
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "Invalid dev_id, can't determine generation");
+ return NULL;
+ }
+
+ rte_spinlock_init(&qat_dev->arb_csr_lock);
+
+ qat_dev->attached = QAT_ATTACHED;
+
+ qat_nb_pci_devices++;
+
+ PMD_DRV_LOG(DEBUG, "QAT device %d allocated, total QATs %d",
+ qat_dev_id, qat_nb_pci_devices);
+
+ return qat_dev;
+}
+
+int
+qat_pci_device_release(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_dev;
+ char name[QAT_DEV_NAME_MAX_LEN];
+
+ if (pci_dev == NULL)
+ return -EINVAL;
+
+ rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ qat_dev = qat_pci_get_named_dev(name);
+ if (qat_dev != NULL) {
+
+ /* Check that there are no service devs still on pci device */
+ if (qat_dev->sym_dev != NULL)
+ return -EBUSY;
+
+ qat_dev->attached = QAT_DETACHED;
+ qat_nb_pci_devices--;
+ }
+ PMD_DRV_LOG(DEBUG, "QAT device %s released, total QATs %d",
+ name, qat_nb_pci_devices);
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 0983e3c2e..d83ad632c 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,15 +11,58 @@
#include "adf_transport_access_macros.h"
#include "qat_qp.h"
+
+#define QAT_DETACHED (0)
+#define QAT_ATTACHED (1)
+
+#define QAT_MAX_PCI_DEVICES 48
+#define QAT_DEV_NAME_MAX_LEN 64
+
+
extern uint8_t cryptodev_qat_driver_id;
extern int qat_sym_qp_release(struct rte_cryptodev *dev,
uint16_t queue_pair_id);
-/** private data structure for each QAT device.
- * In this context a QAT device is a device offering only one service,
- * so there can be more than 1 device on a pci_dev (VF),
- * one for symmetric crypto, one for compression
+/*
+ * This struct holds all the data about a QAT pci device
+ * including data about all services it supports.
+ * It contains
+ * - hw_data
+ * - config data
+ * - runtime data
+ */
+struct qat_pci_device {
+
+ /* data used by all services */
+ char name[QAT_DEV_NAME_MAX_LEN];
+ /**< Name of qat pci device */
+ struct rte_pci_device *pci_dev;
+ /**< PCI information. */
+ enum qat_device_gen qat_dev_gen;
+ /**< QAT device generation */
+ rte_spinlock_t arb_csr_lock;
+ /* protects accesses to the arbiter CSR */
+ __extension__
+ uint8_t attached : 1;
+ /**< Flag indicating the device is attached */
+
+ /* data relating to symmetric crypto service */
+ struct qat_pmd_private *sym_dev;
+ /**< link back to cryptodev private data */
+ unsigned int max_nb_sym_queue_pairs;
+ /**< Max number of queue pairs supported by device */
+
+ /* data relating to compression service */
+
+ /* data relating to asymmetric crypto service */
+
+};
+
+/** private data structure for a QAT device.
+ * This QAT device is a device offering only symmetric crypto service,
+ * there can be one of these on each qat_pci_device (VF),
+ * in future there may also be private data structures for other services.
*/
struct qat_pmd_private {
unsigned int max_nb_queue_pairs;
@@ -34,6 +77,8 @@ struct qat_pmd_private {
/**< PCI information. */
uint8_t dev_id;
/**< Device ID for this instance */
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
};
struct qat_gen_hw_data {
@@ -51,4 +96,11 @@ int qat_dev_close(struct rte_cryptodev *dev);
void qat_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
+struct qat_pci_device *
+qat_pci_device_allocate(struct rte_pci_device *pci_dev);
+int
+qat_pci_device_release(struct rte_pci_device *pci_dev);
+struct qat_pci_device *
+qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev);
+
#endif /* _QAT_DEVICE_H_ */
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index fe19b18b6..6ab870cad 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -69,17 +69,28 @@ static const struct rte_pci_id pci_id_qat_map[] = {
{.device_id = 0},
};
+
+
static int
-crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
- struct rte_cryptodev_pmd_init_params *init_params)
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
{
+ struct rte_cryptodev_pmd_init_params init_params = {
+ .name = "",
+ .socket_id = qat_pci_dev->pci_dev->device.numa_node,
+ .private_data_size = sizeof(struct qat_pmd_private),
+ .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
+ };
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct rte_cryptodev *cryptodev;
struct qat_pmd_private *internals;
- PMD_INIT_FUNC_TRACE();
+ snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
+ qat_pci_dev->name, "sym");
+ PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+
+ cryptodev = rte_cryptodev_pmd_create(name,
+ &qat_pci_dev->pci_dev->device, &init_params);
- cryptodev = rte_cryptodev_pmd_create(name, &pci_dev->device,
- init_params);
if (cryptodev == NULL)
return -ENODEV;
@@ -95,7 +106,10 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
internals = cryptodev->data->dev_private;
- internals->max_nb_sessions = init_params->max_nb_sessions;
+ internals->qat_dev = qat_pci_dev;
+ qat_pci_dev->sym_dev = internals;
+
+ internals->max_nb_sessions = init_params.max_nb_sessions;
internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
internals->dev_id = cryptodev->data->dev_id;
switch (internals->pci_dev->id.device_id) {
@@ -111,68 +125,120 @@ crypto_qat_create(const char *name, struct rte_pci_device *pci_dev,
break;
default:
PMD_DRV_LOG(ERR,
- "Invalid dev_id, can't determine capabilities");
+ "Invalid dev_id, can't determine capabilities");
break;
}
- /*
- * For secondary processes, we don't initialise any further as primary
- * has already done this work. Only check we don't need a different
- * RX function
- */
- if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
- PMD_DRV_LOG(DEBUG, "Device already initialised by primary process");
return 0;
- }
+}
+
+static int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev *cryptodev;
+
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+ if (qat_pci_dev->sym_dev == NULL)
+ return 0;
+
+ /* free crypto device */
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->dev_id);
+ rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_dev = NULL;
+
+ return 0;
+}
+
+static int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+static int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+static int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
return 0;
}
-static int crypto_qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+static int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+static int
+qat_pci_dev_destroy(struct qat_pci_device *qat_pci_dev,
struct rte_pci_device *pci_dev)
{
- struct rte_cryptodev_pmd_init_params init_params = {
- .name = "",
- .socket_id = pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_pmd_private),
- .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
- };
- char name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ qat_sym_dev_destroy(qat_pci_dev);
+ qat_comp_dev_destroy(qat_pci_dev);
+ qat_asym_dev_destroy(qat_pci_dev);
+ return qat_pci_device_release(pci_dev);
+}
+
+static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ int ret = 0;
+ struct qat_pci_device *qat_pci_dev;
PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
pci_dev->addr.bus,
pci_dev->addr.devid,
pci_dev->addr.function);
- rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ qat_pci_dev = qat_pci_device_allocate(pci_dev);
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+
+ ret = qat_sym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_comp_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_asym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ return 0;
+
+error_out:
+ qat_pci_dev_destroy(qat_pci_dev, pci_dev);
+ return ret;
- return crypto_qat_create(name, pci_dev, &init_params);
}
-static int crypto_qat_pci_remove(struct rte_pci_device *pci_dev)
+static int qat_pci_remove(struct rte_pci_device *pci_dev)
{
- struct rte_cryptodev *cryptodev;
- char cryptodev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct qat_pci_device *qat_pci_dev;
if (pci_dev == NULL)
return -EINVAL;
- rte_pci_device_name(&pci_dev->addr, cryptodev_name,
- sizeof(cryptodev_name));
+ qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
+ if (qat_pci_dev == NULL)
+ return 0;
- cryptodev = rte_cryptodev_pmd_get_named_dev(cryptodev_name);
- if (cryptodev == NULL)
- return -ENODEV;
+ return qat_pci_dev_destroy(qat_pci_dev, pci_dev);
- /* free crypto device */
- return rte_cryptodev_pmd_destroy(cryptodev);
}
static struct rte_pci_driver rte_qat_pmd = {
.id_table = pci_id_qat_map,
.drv_flags = RTE_PCI_DRV_NEED_MAPPING,
- .probe = crypto_qat_pci_probe,
- .remove = crypto_qat_pci_remove
+ .probe = qat_pci_probe,
+ .remove = qat_pci_remove
};
static struct cryptodev_driver qat_crypto_drv;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 19/38] crypto/qat: use generic driver name for PCI registration
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (17 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 18/38] crypto/qat: add QAT PCI device struct Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 20/38] crypto/qat: move to using new device structure Tomasz Jozwiak
` (19 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
The QAT PMD used to register with PCI using the name "crypto_qat".
Keep this name for the driver registered with cryptodev
and use a more generic name "qat" for the PCI registration.
This paves the way for the PCI device to host other services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 2 ++
drivers/crypto/qat/rte_qat_cryptodev.c | 25 ++++++++++++++++++-------
2 files changed, 20 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 77ffc8f72..63e5569ea 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -11,6 +11,8 @@
/**< Intel(R) QAT Symmetric Crypto PMD device name */
#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+/**< Intel(R) QAT device name for PCI registration */
+#define QAT_PCI_NAME qat
/*
* Maximum number of SGL entries
*/
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 6ab870cad..ad8a56374 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -234,16 +234,27 @@ static int qat_pci_remove(struct rte_pci_device *pci_dev)
}
+
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+static struct cryptodev_driver qat_crypto_drv;
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+ cryptodev_qat_driver_id);
+
static struct rte_pci_driver rte_qat_pmd = {
.id_table = pci_id_qat_map,
.drv_flags = RTE_PCI_DRV_NEED_MAPPING,
.probe = qat_pci_probe,
.remove = qat_pci_remove
};
-
-static struct cryptodev_driver qat_crypto_drv;
-
-RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_QAT_SYM_PMD, rte_qat_pmd);
-RTE_PMD_REGISTER_PCI_TABLE(CRYPTODEV_NAME_QAT_SYM_PMD, pci_id_qat_map);
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, rte_qat_pmd.driver,
- cryptodev_qat_driver_id);
+RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 20/38] crypto/qat: move to using new device structure
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (18 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 19/38] crypto/qat: use generic driver name for PCI registration Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 21/38] crypto/qat: use common stats structures Tomasz Jozwiak
` (18 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Struct qat_pmd_private held the data needed by cryptodev, common code
now gets most data from struct qat_pci_device instead. qat_pmd_private
is trimmed to hold only sym crypto data and renamed qat_sym_private
to reflect its usage.
Specifically
- remove max_nb_queue_pairs from qat_pmd_private, get from qp_hw_data
- remove max_nb_sesssions from qat_pmd_private as not needed.
- remove qat_gen from qat_pmd_private, get from qat_pci_device instead.
- use qat_pci_device throughout common code instead of qat_pmd_private
- rename qat_pmd_private to qat_sym_dev_private - this now holds only
sym-specific data for the cryptodev API
- extend pci device name to <bdf>_qat for clarity, was just <bdf>
- update qp mem and cookiepool names to reflect the appropriate device,
service and qp.
- rename qat_dev_info_get() to qat_sym_dev_info_get() as mostly sym,
not enough common info to warrant a generic fn.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 17 +++++++-----
drivers/crypto/qat/qat_device.h | 37 +++++++++++---------------
drivers/crypto/qat/qat_qp.c | 26 +++++++++---------
drivers/crypto/qat/qat_qp.h | 4 +--
drivers/crypto/qat/qat_sym.c | 11 ++++----
drivers/crypto/qat/qat_sym.h | 8 ------
drivers/crypto/qat/qat_sym_session.c | 8 +++---
drivers/crypto/qat/rte_qat_cryptodev.c | 32 +++++++++++-----------
8 files changed, 65 insertions(+), 78 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 75af1e8bc..8ad3162e1 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -57,12 +57,12 @@ int qat_dev_close(struct rte_cryptodev *dev)
return 0;
}
-void qat_dev_info_get(struct rte_cryptodev *dev,
+void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info)
{
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev_gen]
+ qp_gen_config[internals->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
@@ -71,7 +71,7 @@ void qat_dev_info_get(struct rte_cryptodev *dev,
qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
info->feature_flags = dev->feature_flags;
info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = internals->max_nb_sessions;
+ info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
info->driver_id = cryptodev_qat_driver_id;
info->pci_dev = RTE_DEV_TO_PCI(dev->device);
}
@@ -83,6 +83,7 @@ qat_pci_get_dev(uint8_t dev_id)
{
return &qat_pci_devices[dev_id];
}
+
static struct qat_pci_device *
qat_pci_get_named_dev(const char *name)
{
@@ -133,7 +134,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
char name[QAT_DEV_NAME_MAX_LEN];
rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
-
+ snprintf(name+strlen(name), QAT_DEV_NAME_MAX_LEN-strlen(name), "_qat");
if (qat_pci_get_named_dev(name) != NULL) {
PMD_DRV_LOG(ERR, "QAT device with name %s already allocated!",
name);
@@ -148,6 +149,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
qat_dev = qat_pci_get_dev(qat_dev_id);
snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
+ qat_dev->qat_dev_id = qat_dev_id;
qat_dev->pci_dev = pci_dev;
switch (qat_dev->pci_dev->id.device_id) {
case 0x0443:
@@ -169,8 +171,8 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
qat_nb_pci_devices++;
- PMD_DRV_LOG(DEBUG, "QAT device %d allocated, total QATs %d",
- qat_dev_id, qat_nb_pci_devices);
+ PMD_DRV_LOG(DEBUG, "QAT device %d allocated, name %s, total QATs %d",
+ qat_dev->qat_dev_id, qat_dev->name, qat_nb_pci_devices);
return qat_dev;
}
@@ -185,6 +187,7 @@ qat_pci_device_release(struct rte_pci_device *pci_dev)
return -EINVAL;
rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+ snprintf(name+strlen(name), QAT_DEV_NAME_MAX_LEN-strlen(name), "_qat");
qat_dev = qat_pci_get_named_dev(name);
if (qat_dev != NULL) {
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index d83ad632c..855bf6c1c 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -32,30 +32,31 @@ extern int qat_sym_qp_release(struct rte_cryptodev *dev,
* - config data
* - runtime data
*/
+struct qat_sym_dev_private;
struct qat_pci_device {
- /* data used by all services */
+ /* Data used by all services */
char name[QAT_DEV_NAME_MAX_LEN];
/**< Name of qat pci device */
+ uint8_t qat_dev_id;
+ /**< Device instance for this qat pci device */
struct rte_pci_device *pci_dev;
/**< PCI information. */
enum qat_device_gen qat_dev_gen;
/**< QAT device generation */
rte_spinlock_t arb_csr_lock;
- /* protects accesses to the arbiter CSR */
+ /**< lock to protect accesses to the arbiter CSR */
__extension__
uint8_t attached : 1;
/**< Flag indicating the device is attached */
- /* data relating to symmetric crypto service */
- struct qat_pmd_private *sym_dev;
+ /* Data relating to symmetric crypto service */
+ struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
- unsigned int max_nb_sym_queue_pairs;
- /**< Max number of queue pairs supported by device */
- /* data relating to compression service */
+ /* Data relating to compression service */
- /* data relating to asymmetric crypto service */
+ /* Data relating to asymmetric crypto service */
};
@@ -64,21 +65,13 @@ struct qat_pci_device {
* there can be one of these on each qat_pci_device (VF),
* in future there may also be private data structures for other services.
*/
-struct qat_pmd_private {
- unsigned int max_nb_queue_pairs;
- /**< Max number of queue pairs supported by device */
- unsigned int max_nb_sessions;
- /**< Max number of sessions supported by device */
- enum qat_device_gen qat_dev_gen;
- /**< QAT device generation */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
- /* QAT device capabilities */
- struct rte_pci_device *pci_dev;
- /**< PCI information. */
- uint8_t dev_id;
- /**< Device ID for this instance */
+struct qat_sym_dev_private {
struct qat_pci_device *qat_dev;
/**< The qat pci device hosting the service */
+ uint8_t sym_dev_id;
+ /**< Device instance for this rte_cryptodev */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device symmetric crypto capabilities */
};
struct qat_gen_hw_data {
@@ -93,7 +86,7 @@ int qat_dev_config(struct rte_cryptodev *dev,
int qat_dev_start(struct rte_cryptodev *dev);
void qat_dev_stop(struct rte_cryptodev *dev);
int qat_dev_close(struct rte_cryptodev *dev);
-void qat_dev_info_get(struct rte_cryptodev *dev,
+void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
struct qat_pci_device *
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 656645e4c..869140fc0 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -102,7 +102,7 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
static int qat_qp_check_queue_alignment(uint64_t phys_addr,
uint32_t queue_size_bytes);
static void qat_queue_delete(struct qat_queue *queue);
-static int qat_queue_create(struct qat_pmd_private *qat_dev,
+static int qat_queue_create(struct qat_pci_device *qat_dev,
struct qat_queue *queue, struct qat_qp_config *, uint8_t dir);
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
@@ -153,7 +153,7 @@ queue_dma_zone_reserve(const char *queue_name, uint32_t queue_size,
socket_id, RTE_MEMZONE_IOVA_CONTIG, queue_size);
}
-int qat_qp_setup(struct qat_pmd_private *qat_dev,
+int qat_qp_setup(struct qat_pci_device *qat_dev,
struct qat_qp **qp_addr,
uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf)
@@ -164,8 +164,8 @@ int qat_qp_setup(struct qat_pmd_private *qat_dev,
char op_cookie_pool_name[RTE_RING_NAMESIZE];
uint32_t i;
- PMD_DRV_LOG(DEBUG, "Setup qp %u on device %d gen %d",
- queue_pair_id, qat_dev->dev_id, qat_dev->qat_dev_gen);
+ PMD_DRV_LOG(DEBUG, "Setup qp %u on qat pci device %d gen %d",
+ queue_pair_id, qat_dev->qat_dev_id, qat_dev->qat_dev_gen);
if ((qat_qp_conf->nb_descriptors > ADF_MAX_DESC) ||
(qat_qp_conf->nb_descriptors < ADF_MIN_DESC)) {
@@ -218,10 +218,12 @@ int qat_qp_setup(struct qat_pmd_private *qat_dev,
adf_configure_queues(qp);
adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
- snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE, "%s_%s_qp_op_%d_%hu",
- pci_dev->driver->driver.name, qat_qp_conf->service_str,
- qat_dev->dev_id, queue_pair_id);
+ snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE,
+ "%s%d_cookies_%s_qp%hu",
+ pci_dev->driver->driver.name, qat_dev->qat_dev_id,
+ qat_qp_conf->service_str, queue_pair_id);
+ PMD_DRV_LOG(DEBUG, "cookiepool: %s", op_cookie_pool_name);
qp->op_cookie_pool = rte_mempool_lookup(op_cookie_pool_name);
if (qp->op_cookie_pool == NULL)
qp->op_cookie_pool = rte_mempool_create(op_cookie_pool_name,
@@ -270,7 +272,7 @@ int qat_qp_release(struct qat_qp **qp_addr)
}
PMD_DRV_LOG(DEBUG, "Free qp on qat_pci device %d",
- qp->qat_dev->dev_id);
+ qp->qat_dev->qat_dev_id);
/* Don't free memory if there are still responses to be processed */
if (qp->inflights16 == 0) {
@@ -322,7 +324,7 @@ static void qat_queue_delete(struct qat_queue *queue)
}
static int
-qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
+qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
struct qat_qp_config *qp_conf, uint8_t dir)
{
uint64_t queue_base;
@@ -347,9 +349,9 @@ qat_queue_create(struct qat_pmd_private *qat_dev, struct qat_queue *queue,
* Allocate a memzone for the queue - create a unique name.
*/
snprintf(queue->memz_name, sizeof(queue->memz_name),
- "%s_%s_%s_%d_%d_%d",
- pci_dev->driver->driver.name, qp_conf->service_str,
- "qp_mem", qat_dev->dev_id,
+ "%s_%d_%s_%s_%d_%d",
+ pci_dev->driver->driver.name, qat_dev->qat_dev_id,
+ qp_conf->service_str, "qp_mem",
queue->hw_bundle_number, queue->hw_queue_number);
qp_mz = queue_dma_zone_reserve(queue->memz_name, queue_size_bytes,
qp_conf->socket_id);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index f808e16a5..d482d5732 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -85,7 +85,7 @@ struct qat_qp {
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
process_response_t process_response;
- struct qat_pmd_private *qat_dev;
+ struct qat_pci_device *qat_dev;
/**< qat device this qp is on */
} __rte_cache_aligned;
@@ -101,7 +101,7 @@ int
qat_qp_release(struct qat_qp **qp_addr);
int
-qat_qp_setup(struct qat_pmd_private *qat_dev,
+qat_qp_setup(struct qat_pci_device *qat_dev,
struct qat_qp **qp_addr, uint16_t queue_pair_id,
struct qat_qp_config *qat_qp_conf);
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index b74dfa634..e77fbe4c4 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -204,7 +204,7 @@ qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
}
-int
+static int
qat_sym_process_response(void **op, uint8_t *resp,
__rte_unused void *op_cookie,
__rte_unused enum qat_device_gen qat_dev_gen)
@@ -293,8 +293,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-
-int
+static int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen)
{
@@ -773,9 +772,9 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
struct qat_qp **qp_addr =
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
- struct qat_pmd_private *qat_private = dev->data->dev_private;
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev_gen]
+ qp_gen_config[qat_private->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
@@ -798,7 +797,7 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.socket_id = socket_id;
qat_qp_conf.service_str = "sym";
- ret = qat_qp_setup(qat_private, qp_addr, qp_id, &qat_qp_conf);
+ ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
if (ret != 0)
return ret;
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 37bec3ce3..78b40e378 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -29,14 +29,6 @@ struct qat_sym_op_cookie {
phys_addr_t qat_sgl_dst_phys_addr;
};
-int
-qat_sym_build_request(void *in_op, uint8_t *out_msg,
- void *op_cookie, enum qat_device_gen qat_dev_gen);
-
-int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie, enum qat_device_gen qat_dev_gen);
-
void qat_sym_stats_get(struct rte_cryptodev *dev,
struct rte_cryptodev_stats *stats);
void qat_sym_stats_reset(struct rte_cryptodev *dev);
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index a08e93037..68d7773a2 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -64,7 +64,7 @@ bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
static int
qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
- struct qat_pmd_private *internals)
+ struct qat_sym_dev_private *internals)
{
int i = 0;
const struct rte_cryptodev_capabilities *capability;
@@ -85,7 +85,7 @@ qat_is_cipher_alg_supported(enum rte_crypto_cipher_algorithm algo,
static int
qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo,
- struct qat_pmd_private *internals)
+ struct qat_sym_dev_private *internals)
{
int i = 0;
const struct rte_cryptodev_capabilities *capability;
@@ -201,7 +201,7 @@ qat_sym_session_configure_cipher(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
struct qat_sym_session *session)
{
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
struct rte_crypto_cipher_xform *cipher_xform = NULL;
int ret;
@@ -495,7 +495,7 @@ qat_sym_session_configure_auth(struct rte_cryptodev *dev,
struct qat_sym_session *session)
{
struct rte_crypto_auth_xform *auth_xform = qat_get_auth_xform(xform);
- struct qat_pmd_private *internals = dev->data->dev_private;
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
uint8_t *key_data = auth_xform->key.data;
uint8_t key_length = auth_xform->key.length;
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index ad8a56374..6d807177e 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -33,7 +33,7 @@ static struct rte_cryptodev_ops crypto_qat_ops = {
.dev_start = qat_dev_start,
.dev_stop = qat_dev_stop,
.dev_close = qat_dev_close,
- .dev_infos_get = qat_dev_info_get,
+ .dev_infos_get = qat_sym_dev_info_get,
.stats_get = qat_sym_stats_get,
.stats_reset = qat_sym_stats_reset,
@@ -77,12 +77,12 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
struct rte_cryptodev_pmd_init_params init_params = {
.name = "",
.socket_id = qat_pci_dev->pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_pmd_private),
+ .private_data_size = sizeof(struct qat_sym_dev_private),
.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
};
char name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct rte_cryptodev *cryptodev;
- struct qat_pmd_private *internals;
+ struct qat_sym_dev_private *internals;
snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
qat_pci_dev->name, "sym");
@@ -109,27 +109,25 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
internals->qat_dev = qat_pci_dev;
qat_pci_dev->sym_dev = internals;
- internals->max_nb_sessions = init_params.max_nb_sessions;
- internals->pci_dev = RTE_DEV_TO_PCI(cryptodev->device);
- internals->dev_id = cryptodev->data->dev_id;
- switch (internals->pci_dev->id.device_id) {
- case 0x0443:
- internals->qat_dev_gen = QAT_GEN1;
+ internals->sym_dev_id = cryptodev->data->dev_id;
+ switch (qat_pci_dev->qat_dev_gen) {
+ case QAT_GEN1:
internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
break;
- case 0x37c9:
- case 0x19e3:
- case 0x6f55:
- internals->qat_dev_gen = QAT_GEN2;
+ case QAT_GEN2:
internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
break;
default:
- PMD_DRV_LOG(ERR,
- "Invalid dev_id, can't determine capabilities");
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ PMD_DRV_LOG(DEBUG,
+ "QAT gen %d capabilities unknown, default to GEN2",
+ qat_pci_dev->qat_dev_gen);
break;
}
- return 0;
+ PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
+ name, internals->sym_dev_id);
+ return 0;
}
static int
@@ -143,7 +141,7 @@ qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
return 0;
/* free crypto device */
- cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->dev_id);
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
rte_cryptodev_pmd_destroy(cryptodev);
qat_pci_dev->sym_dev = NULL;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 21/38] crypto/qat: use common stats structures
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (19 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 20/38] crypto/qat: move to using new device structure Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 22/38] crypto/qat: rename functions which depend on cryptodev Tomasz Jozwiak
` (17 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Split qat_sym_stats_get/reset into 2 functions, a wrapper function calling
a new qat_stats_get/reset function which can be called per service.
Remove cryptodev stats struct from qat_qp, replace with qat_common_stats.
Add links for qat_qp into qat_pci_device using an array per service to
avoid need for a lock and so qp_id for the service can be used as index.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_common.h | 13 +++++
drivers/crypto/qat/qat_device.c | 1 +
drivers/crypto/qat/qat_device.h | 3 ++
drivers/crypto/qat/qat_qp.h | 2 +-
drivers/crypto/qat/qat_sym.c | 86 ++++++++++++++++++++++++++++-----
5 files changed, 91 insertions(+), 14 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 63e5569ea..fcf5c4c09 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -49,6 +49,19 @@ struct qat_sgl {
struct qat_flat_buf buffers[QAT_SGL_MAX_NUMBER];
} __rte_packed __rte_cache_aligned;
+/** Common, i.e. not service-specific, statistics */
+struct qat_common_stats {
+ uint64_t enqueued_count;
+ /**< Count of all operations enqueued */
+ uint64_t dequeued_count;
+ /**< Count of all operations dequeued */
+
+ uint64_t enqueue_err_count;
+ /**< Total error count on operations enqueued */
+ uint64_t dequeue_err_count;
+ /**< Total error count on operations dequeued */
+};
+
int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
struct qat_sgl *list, uint32_t data_len);
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 8ad3162e1..4dfbc84c8 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -148,6 +148,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
}
qat_dev = qat_pci_get_dev(qat_dev_id);
+ memset(qat_dev, 0, sizeof(*qat_dev));
snprintf(qat_dev->name, QAT_DEV_NAME_MAX_LEN, "%s", name);
qat_dev->qat_dev_id = qat_dev_id;
qat_dev->pci_dev = pci_dev;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 855bf6c1c..d1615e2a7 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -50,6 +50,9 @@ struct qat_pci_device {
uint8_t attached : 1;
/**< Flag indicating the device is attached */
+ struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_PER_BUNDLE];
+ /**< links to qps set up for each service, index same as on API */
+
/* Data relating to symmetric crypto service */
struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index d482d5732..49d9f29d3 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -78,7 +78,7 @@ struct qat_qp {
uint16_t inflights16;
struct qat_queue tx_q;
struct qat_queue rx_q;
- struct rte_cryptodev_stats stats;
+ struct qat_common_stats stats;
struct rte_mempool *op_cookie_pool;
void **op_cookies;
uint32_t nb_descriptors;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index e77fbe4c4..8007e25d6 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -717,20 +717,24 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
}
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats)
+static void qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service)
{
int i;
- struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
+ struct qat_qp **qp;
- PMD_INIT_FUNC_TRACE();
- if (stats == NULL) {
- PMD_DRV_LOG(ERR, "invalid stats ptr NULL");
+ if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
+ stats, dev, service);
return;
}
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Uninitialised queue pair");
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
continue;
}
@@ -741,22 +745,74 @@ void qat_sym_stats_get(struct rte_cryptodev *dev,
}
}
-void qat_sym_stats_reset(struct rte_cryptodev *dev)
+void qat_sym_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ struct qat_common_stats qat_stats = {0};
+ struct qat_sym_dev_private *qat_priv;
+
+ if (stats == NULL || dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
+ stats->enqueued_count = qat_stats.enqueued_count;
+ stats->dequeued_count = qat_stats.dequeued_count;
+ stats->enqueue_err_count = qat_stats.enqueue_err_count;
+ stats->dequeue_err_count = qat_stats.dequeue_err_count;
+}
+
+static void qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service)
{
int i;
- struct qat_qp **qp = (struct qat_qp **)(dev->data->queue_pairs);
+ struct qat_qp **qp;
- PMD_INIT_FUNC_TRACE();
- for (i = 0; i < dev->data->nb_queue_pairs; i++)
+ if (dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
+ dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
- PMD_DRV_LOG(DEBUG, "QAT crypto: stats cleared");
+ }
+
+ PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+}
+
+void qat_sym_stats_reset(struct rte_cryptodev *dev)
+{
+ struct qat_sym_dev_private *qat_priv;
+
+ if (dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
+
}
int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
{
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+
PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
queue_pair_id, dev->data->dev_id);
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
+ = NULL;
+
return qat_qp_release((struct qat_qp **)
&(dev->data->queue_pairs[queue_pair_id]));
}
@@ -801,6 +857,10 @@ int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
if (ret != 0)
return ret;
+ /* store a link to the qp in the qat_pci_device */
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
+ = *qp_addr;
+
qp = (struct qat_qp *)*qp_addr;
for (i = 0; i < qp->nb_descriptors; i++) {
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 22/38] crypto/qat: rename functions which depend on cryptodev
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (20 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 21/38] crypto/qat: use common stats structures Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 23/38] crypto/qat: move code into appropriate files Tomasz Jozwiak
` (16 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Rename all device functions which depend on cryptodev structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.c | 8 ++++----
drivers/crypto/qat/qat_device.h | 8 ++++----
drivers/crypto/qat/rte_qat_cryptodev.c | 8 ++++----
3 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 4dfbc84c8..8b2ac5a5f 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -24,25 +24,25 @@ struct qat_gen_hw_data qp_gen_config[] = {
static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
-int qat_dev_config(__rte_unused struct rte_cryptodev *dev,
+int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
__rte_unused struct rte_cryptodev_config *config)
{
PMD_INIT_FUNC_TRACE();
return 0;
}
-int qat_dev_start(__rte_unused struct rte_cryptodev *dev)
+int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
{
PMD_INIT_FUNC_TRACE();
return 0;
}
-void qat_dev_stop(__rte_unused struct rte_cryptodev *dev)
+void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
{
PMD_INIT_FUNC_TRACE();
}
-int qat_dev_close(struct rte_cryptodev *dev)
+int qat_sym_dev_close(struct rte_cryptodev *dev)
{
int i, ret;
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index d1615e2a7..5424a9a94 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -84,11 +84,11 @@ struct qat_gen_hw_data {
extern struct qat_gen_hw_data qp_gen_config[];
-int qat_dev_config(struct rte_cryptodev *dev,
+int qat_sym_dev_config(struct rte_cryptodev *dev,
struct rte_cryptodev_config *config);
-int qat_dev_start(struct rte_cryptodev *dev);
-void qat_dev_stop(struct rte_cryptodev *dev);
-int qat_dev_close(struct rte_cryptodev *dev);
+int qat_sym_dev_start(struct rte_cryptodev *dev);
+void qat_sym_dev_stop(struct rte_cryptodev *dev);
+int qat_sym_dev_close(struct rte_cryptodev *dev);
void qat_sym_dev_info_get(struct rte_cryptodev *dev,
struct rte_cryptodev_info *info);
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
index 6d807177e..91bb1e590 100644
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ b/drivers/crypto/qat/rte_qat_cryptodev.c
@@ -29,10 +29,10 @@ static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
static struct rte_cryptodev_ops crypto_qat_ops = {
/* Device related operations */
- .dev_configure = qat_dev_config,
- .dev_start = qat_dev_start,
- .dev_stop = qat_dev_stop,
- .dev_close = qat_dev_close,
+ .dev_configure = qat_sym_dev_config,
+ .dev_start = qat_sym_dev_start,
+ .dev_stop = qat_sym_dev_stop,
+ .dev_close = qat_sym_dev_close,
.dev_infos_get = qat_sym_dev_info_get,
.stats_get = qat_sym_stats_get,
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 23/38] crypto/qat: move code into appropriate files
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (21 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 22/38] crypto/qat: rename functions which depend on cryptodev Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 24/38] crypto/qat: add lock around csr access and change logic Tomasz Jozwiak
` (15 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Move all code into appropriate files, no actual code changes. Specifically:
- Rename rte_qat_cryptodev.c to qat_sym_pmd.c
- Create qat_sym_pmd.h and populate with fn prototypes for qat_sym_pmd.c
- Create qat_comp_pmd.c/.h and populate with placeholder functions
- Create qat_asym_pmd.c/.h and populate with placeholder functions
- Rename qat_crypto_capabilities.h to qat_sym_capabilities.h
- Move CRYPTODEV_NAME_QAT_SYM_PMD from qat_common.h to qat_sym_pmd.h
- Move qat_sym_dev_private from qat_device.h to qat_sym_pmd.h
- Move prototype for qat_sym_dev_info_get frm qat_device.h 2 qat_sym_pmd.h
- Move all qat_device.c sym dev_ops fns to qat_sym_pmd.c file
- Move all qat_sym.c dev_ops fns to qat_sym_pmd.c file
- Remove unused header file #includes from all files.
- Move pci_id_qat_map, probe/release/register from
rte_qat_cryptodev.c to qat_device.c
- Moved stray comment for bpi_cipher_ctx_init() from qat_sym.c
to qat_sym_session.c
- Changed all files to use SPDX license header
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/Makefile | 4 +-
drivers/crypto/qat/meson.build | 11 +-
drivers/crypto/qat/qat_asym_pmd.c | 17 +
drivers/crypto/qat/qat_asym_pmd.h | 15 +
drivers/crypto/qat/qat_common.c | 54 +++
drivers/crypto/qat/qat_common.h | 15 +-
drivers/crypto/qat/qat_comp_pmd.c | 18 +
drivers/crypto/qat/qat_comp_pmd.h | 29 ++
drivers/crypto/qat/qat_device.c | 141 +++++---
drivers/crypto/qat/qat_device.h | 31 +-
drivers/crypto/qat/qat_qp.c | 2 +-
drivers/crypto/qat/qat_qp.h | 9 +-
drivers/crypto/qat/qat_sym.c | 246 ++-----------
drivers/crypto/qat/qat_sym.h | 28 +-
..._capabilities.h => qat_sym_capabilities.h} | 6 +-
drivers/crypto/qat/qat_sym_pmd.c | 326 ++++++++++++++++++
drivers/crypto/qat/qat_sym_pmd.h | 39 +++
drivers/crypto/qat/qat_sym_session.c | 17 +-
drivers/crypto/qat/qat_sym_session.h | 2 +-
drivers/crypto/qat/rte_qat_cryptodev.c | 258 --------------
20 files changed, 665 insertions(+), 603 deletions(-)
create mode 100644 drivers/crypto/qat/qat_asym_pmd.c
create mode 100644 drivers/crypto/qat/qat_asym_pmd.h
create mode 100644 drivers/crypto/qat/qat_comp_pmd.c
create mode 100644 drivers/crypto/qat/qat_comp_pmd.h
rename drivers/crypto/qat/{qat_crypto_capabilities.h => qat_sym_capabilities.h} (99%)
create mode 100644 drivers/crypto/qat/qat_sym_pmd.c
create mode 100644 drivers/crypto/qat/qat_sym_pmd.h
delete mode 100644 drivers/crypto/qat/rte_qat_cryptodev.c
diff --git a/drivers/crypto/qat/Makefile b/drivers/crypto/qat/Makefile
index 902c47ff4..d467683fd 100644
--- a/drivers/crypto/qat/Makefile
+++ b/drivers/crypto/qat/Makefile
@@ -26,7 +26,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_device.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_qp.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_session.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_common.c
-SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += rte_qat_cryptodev.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_sym_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_asym_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat_comp_pmd.c
# export include files
SYMLINK-y-include +=
diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.build
index 12910c377..e22e08fba 100644
--- a/drivers/crypto/qat/meson.build
+++ b/drivers/crypto/qat/meson.build
@@ -5,11 +5,12 @@ dep = dependency('libcrypto', required: false)
if not dep.found()
build = false
endif
-sources = files('qat_sym.c', 'qat_qp.c',
- 'qat_sym_session.c',
- 'qat_common.c',
- 'rte_qat_cryptodev.c',
- 'qat_device.c')
+sources = files('qat_common.c',
+ 'qat_qp.c',
+ 'qat_device.c',
+ 'qat_sym_pmd.c', 'qat_sym.c', 'qat_sym_session.c',
+ 'qat_asym_pmd.c',
+ 'qat_comp_pmd.c')
includes += include_directories('qat_adf')
deps += ['bus_pci']
ext_deps += dep
diff --git a/drivers/crypto/qat/qat_asym_pmd.c b/drivers/crypto/qat/qat_asym_pmd.c
new file mode 100644
index 000000000..8d36300ac
--- /dev/null
+++ b/drivers/crypto/qat/qat_asym_pmd.c
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_asym_pmd.h"
+
+int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_asym_pmd.h b/drivers/crypto/qat/qat_asym_pmd.h
new file mode 100644
index 000000000..0465e0300
--- /dev/null
+++ b/drivers/crypto/qat/qat_asym_pmd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _QAT_ASYM_PMD_H_
+#define _QAT_ASYM_PMD_H_
+
+#include "qat_device.h"
+
+int
+qat_asym_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_ASYM_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index a8865904f..f1759ea76 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -3,6 +3,7 @@
*/
#include "qat_common.h"
+#include "qat_device.h"
#include "qat_logs.h"
int
@@ -51,3 +52,56 @@ qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
return 0;
}
+
+void qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service)
+{
+ int i;
+ struct qat_qp **qp;
+
+ if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
+ stats, dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
+
+ stats->enqueued_count += qp[i]->stats.enqueued_count;
+ stats->dequeued_count += qp[i]->stats.dequeued_count;
+ stats->enqueue_err_count += qp[i]->stats.enqueue_err_count;
+ stats->dequeue_err_count += qp[i]->stats.dequeue_err_count;
+ }
+}
+
+void qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service)
+{
+ int i;
+ struct qat_qp **qp;
+
+ if (dev == NULL || service >= QAT_SERVICE_INVALID) {
+ PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
+ dev, service);
+ return;
+ }
+
+ qp = dev->qps_in_use[service];
+ for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ if (qp[i] == NULL) {
+ PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
+ service, i);
+ continue;
+ }
+ memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
+ }
+
+ PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+}
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index fcf5c4c09..8ecebe954 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -8,9 +8,6 @@
#include <rte_mbuf.h>
-/**< Intel(R) QAT Symmetric Crypto PMD device name */
-#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
-
/**< Intel(R) QAT device name for PCI registration */
#define QAT_PCI_NAME qat
/*
@@ -21,10 +18,9 @@
/* Intel(R) QuickAssist Technology device generation is enumerated
* from one according to the generation of the device
*/
-
enum qat_device_gen {
QAT_GEN1 = 1,
- QAT_GEN2,
+ QAT_GEN2
};
enum qat_service_type {
@@ -62,8 +58,17 @@ struct qat_common_stats {
/**< Total error count on operations dequeued */
};
+struct qat_pci_device;
+
int
qat_sgl_fill_array(struct rte_mbuf *buf, uint64_t buf_start,
struct qat_sgl *list, uint32_t data_len);
+void
+qat_stats_get(struct qat_pci_device *dev,
+ struct qat_common_stats *stats,
+ enum qat_service_type service);
+void
+qat_stats_reset(struct qat_pci_device *dev,
+ enum qat_service_type service);
#endif /* _QAT_COMMON_H_ */
diff --git a/drivers/crypto/qat/qat_comp_pmd.c b/drivers/crypto/qat/qat_comp_pmd.c
new file mode 100644
index 000000000..547b3db49
--- /dev/null
+++ b/drivers/crypto/qat/qat_comp_pmd.c
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include "qat_comp_pmd.h"
+
+
+int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
+
+int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
+{
+ return 0;
+}
diff --git a/drivers/crypto/qat/qat_comp_pmd.h b/drivers/crypto/qat/qat_comp_pmd.h
new file mode 100644
index 000000000..cc31246c3
--- /dev/null
+++ b/drivers/crypto/qat/qat_comp_pmd.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#ifndef _QAT_COMP_PMD_H_
+#define _QAT_COMP_PMD_H_
+
+#include "qat_device.h"
+
+
+/**< Intel(R) QAT Compression PMD device name */
+#define COMPRESSDEV_NAME_QAT_PMD comp_qat
+
+
+/** private data structure for a QAT compression device.
+ * This QAT device is a device offering only a compression service,
+ * there can be one of these on each qat_pci_device (VF).
+ */
+struct qat_comp_dev_private {
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
+};
+
+int
+qat_comp_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_COMP_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index 8b2ac5a5f..c9d4b32ed 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -4,7 +4,9 @@
#include "qat_device.h"
#include "adf_transport_access_macros.h"
-#include "qat_qp.h"
+#include "qat_sym_pmd.h"
+#include "qat_asym_pmd.h"
+#include "qat_comp_pmd.h"
/* Hardware device information per generation */
__extension__
@@ -24,58 +26,25 @@ struct qat_gen_hw_data qp_gen_config[] = {
static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
-int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
- __rte_unused struct rte_cryptodev_config *config)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
- return 0;
-}
-
-void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
-{
- PMD_INIT_FUNC_TRACE();
-}
-
-int qat_sym_dev_close(struct rte_cryptodev *dev)
-{
- int i, ret;
-
- PMD_INIT_FUNC_TRACE();
-
- for (i = 0; i < dev->data->nb_queue_pairs; i++) {
- ret = qat_sym_qp_release(dev, i);
- if (ret < 0)
- return ret;
- }
-
- return 0;
-}
+/*
+ * The set of PCI devices this driver supports
+ */
-void qat_sym_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info)
-{
- struct qat_sym_dev_private *internals = dev->data->dev_private;
- const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev->qat_dev_gen]
- .qp_hw_data[QAT_SERVICE_SYMMETRIC];
-
- PMD_INIT_FUNC_TRACE();
- if (info != NULL) {
- info->max_nb_queue_pairs =
- qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
- info->feature_flags = dev->feature_flags;
- info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
- info->driver_id = cryptodev_qat_driver_id;
- info->pci_dev = RTE_DEV_TO_PCI(dev->device);
- }
-}
+static const struct rte_pci_id pci_id_qat_map[] = {
+ {
+ RTE_PCI_DEVICE(0x8086, 0x0443),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x37c9),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x19e3),
+ },
+ {
+ RTE_PCI_DEVICE(0x8086, 0x6f55),
+ },
+ {.device_id = 0},
+};
static struct qat_pci_device *
@@ -203,3 +172,71 @@ qat_pci_device_release(struct rte_pci_device *pci_dev)
name, qat_nb_pci_devices);
return 0;
}
+
+static int
+qat_pci_dev_destroy(struct qat_pci_device *qat_pci_dev,
+ struct rte_pci_device *pci_dev)
+{
+ qat_sym_dev_destroy(qat_pci_dev);
+ qat_comp_dev_destroy(qat_pci_dev);
+ qat_asym_dev_destroy(qat_pci_dev);
+ return qat_pci_device_release(pci_dev);
+}
+
+static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ int ret = 0;
+ struct qat_pci_device *qat_pci_dev;
+
+ PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
+ pci_dev->addr.bus,
+ pci_dev->addr.devid,
+ pci_dev->addr.function);
+
+ qat_pci_dev = qat_pci_device_allocate(pci_dev);
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+
+ ret = qat_sym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_comp_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ ret = qat_asym_dev_create(qat_pci_dev);
+ if (ret != 0)
+ goto error_out;
+
+ return 0;
+
+error_out:
+ qat_pci_dev_destroy(qat_pci_dev, pci_dev);
+ return ret;
+
+}
+
+static int qat_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct qat_pci_device *qat_pci_dev;
+
+ if (pci_dev == NULL)
+ return -EINVAL;
+
+ qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
+ if (qat_pci_dev == NULL)
+ return 0;
+
+ return qat_pci_dev_destroy(qat_pci_dev, pci_dev);
+}
+
+static struct rte_pci_driver rte_qat_pmd = {
+ .id_table = pci_id_qat_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = qat_pci_probe,
+ .remove = qat_pci_remove
+};
+RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 5424a9a94..fd1819354 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -4,8 +4,8 @@
#ifndef _QAT_DEVICE_H_
#define _QAT_DEVICE_H_
-#include <rte_cryptodev_pmd.h>
#include <rte_bus_pci.h>
+
#include "qat_common.h"
#include "qat_logs.h"
#include "adf_transport_access_macros.h"
@@ -18,12 +18,6 @@
#define QAT_MAX_PCI_DEVICES 48
#define QAT_DEV_NAME_MAX_LEN 64
-
-extern uint8_t cryptodev_qat_driver_id;
-
-extern int qat_sym_qp_release(struct rte_cryptodev *dev,
- uint16_t queue_pair_id);
-
/*
* This struct holds all the data about a QAT pci device
* including data about all services it supports.
@@ -63,20 +57,6 @@ struct qat_pci_device {
};
-/** private data structure for a QAT device.
- * This QAT device is a device offering only symmetric crypto service,
- * there can be one of these on each qat_pci_device (VF),
- * in future there may also be private data structures for other services.
- */
-struct qat_sym_dev_private {
- struct qat_pci_device *qat_dev;
- /**< The qat pci device hosting the service */
- uint8_t sym_dev_id;
- /**< Device instance for this rte_cryptodev */
- const struct rte_cryptodev_capabilities *qat_dev_capabilities;
- /* QAT device symmetric crypto capabilities */
-};
-
struct qat_gen_hw_data {
enum qat_device_gen dev_gen;
const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
@@ -84,14 +64,6 @@ struct qat_gen_hw_data {
extern struct qat_gen_hw_data qp_gen_config[];
-int qat_sym_dev_config(struct rte_cryptodev *dev,
- struct rte_cryptodev_config *config);
-int qat_sym_dev_start(struct rte_cryptodev *dev);
-void qat_sym_dev_stop(struct rte_cryptodev *dev);
-int qat_sym_dev_close(struct rte_cryptodev *dev);
-void qat_sym_dev_info_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_info *info);
-
struct qat_pci_device *
qat_pci_device_allocate(struct rte_pci_device *pci_dev);
int
@@ -99,4 +71,5 @@ qat_pci_device_release(struct rte_pci_device *pci_dev);
struct qat_pci_device *
qat_get_qat_dev_from_pci_dev(struct rte_pci_device *pci_dev);
+
#endif /* _QAT_DEVICE_H_ */
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 869140fc0..7b2dc3f90 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -12,8 +12,8 @@
#include <rte_prefetch.h>
#include "qat_logs.h"
-#include "qat_qp.h"
#include "qat_device.h"
+#include "qat_qp.h"
#include "adf_transport_access_macros.h"
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 49d9f29d3..73888b805 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -5,9 +5,10 @@
#define _QAT_QP_H_
#include "qat_common.h"
-#include <rte_cryptodev_pmd.h>
#include "adf_transport_access_macros.h"
+struct qat_pci_device;
+
#define QAT_CSR_HEAD_WRITE_THRESH 32U
/* number of requests to accumulate before writing head CSR */
#define QAT_CSR_TAIL_WRITE_THRESH 32U
@@ -76,9 +77,9 @@ struct qat_queue {
struct qat_qp {
void *mmap_bar_addr;
uint16_t inflights16;
- struct qat_queue tx_q;
- struct qat_queue rx_q;
- struct qat_common_stats stats;
+ struct qat_queue tx_q;
+ struct qat_queue rx_q;
+ struct qat_common_stats stats;
struct rte_mempool *op_cookie_pool;
void **op_cookies;
uint32_t nb_descriptors;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 8007e25d6..15244d113 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -2,6 +2,8 @@
* Copyright(c) 2015-2018 Intel Corporation
*/
+#include <openssl/evp.h>
+
#include <rte_mempool.h>
#include <rte_mbuf.h>
#include <rte_hexdump.h>
@@ -9,14 +11,10 @@
#include <rte_bus_pci.h>
#include <rte_byteorder.h>
-#include <openssl/evp.h>
-
#include "qat_logs.h"
#include "qat_sym_session.h"
#include "qat_sym.h"
-#include "qat_qp.h"
-#include "adf_transport_access_macros.h"
-#include "qat_device.h"
+#include "qat_sym_pmd.h"
#define BYTE_LENGTH 8
/* bpi is only used for partial blocks of DES and AES
@@ -82,9 +80,6 @@ bpi_cipher_decrypt(uint8_t *src, uint8_t *dst,
return -EINVAL;
}
-/** Creates a context in either AES or DES in ECB mode
- * Depends on openssl libcrypto
- */
static inline uint32_t
qat_bpicipher_preprocess(struct qat_sym_session *ctx,
@@ -197,57 +192,6 @@ qat_bpicipher_postprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
-{
- return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
-}
-
-static int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen)
-{
-
- struct icp_qat_fw_comn_resp *resp_msg =
- (struct icp_qat_fw_comn_resp *)resp;
- struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
-
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
-
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess = (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
-
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
- *op = (void *)rx_op;
-
- return 0;
-}
-
-
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops)
-{
- return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
@@ -293,7 +237,7 @@ set_cipher_iv_ccm(uint16_t iv_length, uint16_t iv_offset,
iv_length);
}
-static int
+int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen)
{
@@ -716,168 +660,38 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
return 0;
}
-
-static void qat_stats_get(struct qat_pci_device *dev,
- struct qat_common_stats *stats,
- enum qat_service_type service)
-{
- int i;
- struct qat_qp **qp;
-
- if (stats == NULL || dev == NULL || service >= QAT_SERVICE_INVALID) {
- PMD_DRV_LOG(ERR, "invalid param: stats %p, dev %p, service %d",
- stats, dev, service);
- return;
- }
-
- qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
- if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
- service, i);
- continue;
- }
-
- stats->enqueued_count += qp[i]->stats.enqueued_count;
- stats->dequeued_count += qp[i]->stats.dequeued_count;
- stats->enqueue_err_count += qp[i]->stats.enqueue_err_count;
- stats->dequeue_err_count += qp[i]->stats.dequeue_err_count;
- }
-}
-
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats)
-{
- struct qat_common_stats qat_stats = {0};
- struct qat_sym_dev_private *qat_priv;
-
- if (stats == NULL || dev == NULL) {
- PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
- return;
- }
- qat_priv = dev->data->dev_private;
-
- qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
- stats->enqueued_count = qat_stats.enqueued_count;
- stats->dequeued_count = qat_stats.dequeued_count;
- stats->enqueue_err_count = qat_stats.enqueue_err_count;
- stats->dequeue_err_count = qat_stats.dequeue_err_count;
-}
-
-static void qat_stats_reset(struct qat_pci_device *dev,
- enum qat_service_type service)
-{
- int i;
- struct qat_qp **qp;
-
- if (dev == NULL || service >= QAT_SERVICE_INVALID) {
- PMD_DRV_LOG(ERR, "invalid param: dev %p, service %d",
- dev, service);
- return;
- }
-
- qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
- if (qp[i] == NULL) {
- PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
- service, i);
- continue;
- }
- memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
- }
-
- PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
-}
-
-void qat_sym_stats_reset(struct rte_cryptodev *dev)
-{
- struct qat_sym_dev_private *qat_priv;
-
- if (dev == NULL) {
- PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
- return;
- }
- qat_priv = dev->data->dev_private;
-
- qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
-
-}
-
-int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen)
{
- struct qat_sym_dev_private *qat_private = dev->data->dev_private;
- PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
- queue_pair_id, dev->data->dev_id);
-
- qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
- = NULL;
-
- return qat_qp_release((struct qat_qp **)
- &(dev->data->queue_pairs[queue_pair_id]));
-}
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
- const struct rte_cryptodev_qp_conf *qp_conf,
- int socket_id, struct rte_mempool *session_pool __rte_unused)
-{
- struct qat_qp *qp;
- int ret = 0;
- uint32_t i;
- struct qat_qp_config qat_qp_conf;
-
- struct qat_qp **qp_addr =
- (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
- struct qat_sym_dev_private *qat_private = dev->data->dev_private;
- const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev->qat_dev_gen]
- .qp_hw_data[QAT_SERVICE_SYMMETRIC];
- const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
-
- /* If qp is already in use free ring memory and qp metadata. */
- if (*qp_addr != NULL) {
- ret = qat_sym_qp_release(dev, qp_id);
- if (ret < 0)
- return ret;
- }
- if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
- PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
- return -EINVAL;
- }
-
- qat_qp_conf.hw = qp_hw_data;
- qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
- qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
- qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
- qat_qp_conf.socket_id = socket_id;
- qat_qp_conf.service_str = "sym";
-
- ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
- if (ret != 0)
- return ret;
-
- /* store a link to the qp in the qat_pci_device */
- qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
- = *qp_addr;
-
- qp = (struct qat_qp *)*qp_addr;
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
- for (i = 0; i < qp->nb_descriptors; i++) {
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
- struct qat_sym_op_cookie *sql_cookie =
- qp->op_cookies[i];
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_src);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
- offsetof(struct qat_sym_op_cookie,
- qat_sgl_dst);
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
}
+ *op = (void *)rx_op;
- return ret;
+ return 0;
}
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index 78b40e378..d887dc126 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -6,11 +6,8 @@
#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
-#include <rte_memzone.h>
#include "qat_common.h"
-#include "qat_device.h"
-#include "qat_crypto_capabilities.h"
/*
* This macro rounds up a number to a be a multiple of
@@ -29,23 +26,12 @@ struct qat_sym_op_cookie {
phys_addr_t qat_sgl_dst_phys_addr;
};
-void qat_sym_stats_get(struct rte_cryptodev *dev,
- struct rte_cryptodev_stats *stats);
-void qat_sym_stats_reset(struct rte_cryptodev *dev);
-
-int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t queue_pair_id,
- const struct rte_cryptodev_qp_conf *rx_conf, int socket_id,
- struct rte_mempool *session_pool);
-int qat_sym_qp_release(struct rte_cryptodev *dev,
- uint16_t queue_pair_id);
-
-
-uint16_t
-qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops);
-
-uint16_t
-qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
- uint16_t nb_ops);
+int
+qat_sym_build_request(void *in_op, uint8_t *out_msg,
+ void *op_cookie, enum qat_device_gen qat_dev_gen);
+int
+qat_sym_process_response(void **op, uint8_t *resp,
+ __rte_unused void *op_cookie,
+ __rte_unused enum qat_device_gen qat_dev_gen);
#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_crypto_capabilities.h b/drivers/crypto/qat/qat_sym_capabilities.h
similarity index 99%
rename from drivers/crypto/qat/qat_crypto_capabilities.h
rename to drivers/crypto/qat/qat_sym_capabilities.h
index 001c32c5d..d10a95ecb 100644
--- a/drivers/crypto/qat/qat_crypto_capabilities.h
+++ b/drivers/crypto/qat/qat_sym_capabilities.h
@@ -2,8 +2,8 @@
* Copyright(c) 2017-2018 Intel Corporation
*/
-#ifndef _QAT_CRYPTO_CAPABILITIES_H_
-#define _QAT_CRYPTO_CAPABILITIES_H_
+#ifndef _QAT_SYM_CAPABILITIES_H_
+#define _QAT_SYM_CAPABILITIES_H_
#define QAT_BASE_GEN1_SYM_CAPABILITIES \
{ /* SHA1 HMAC */ \
@@ -554,4 +554,4 @@
}, } \
}
-#endif /* _QAT_CRYPTO_CAPABILITIES_H_ */
+#endif /* _QAT_SYM_CAPABILITIES_H_ */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
new file mode 100644
index 000000000..aa71b4641
--- /dev/null
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -0,0 +1,326 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2015-2018 Intel Corporation
+ */
+
+#include <rte_bus_pci.h>
+#include <rte_common.h>
+#include <rte_dev.h>
+#include <rte_malloc.h>
+#include <rte_pci.h>
+#include <rte_cryptodev_pmd.h>
+
+#include "qat_logs.h"
+#include "qat_sym.h"
+#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
+
+uint8_t cryptodev_qat_driver_id;
+
+static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
+ QAT_BASE_GEN1_SYM_CAPABILITIES,
+ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
+};
+
+static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
+ QAT_BASE_GEN1_SYM_CAPABILITIES,
+ QAT_EXTRA_GEN2_SYM_CAPABILITIES,
+ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
+};
+
+static int qat_sym_qp_release(struct rte_cryptodev *dev,
+ uint16_t queue_pair_id);
+
+static int qat_sym_dev_config(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused struct rte_cryptodev_config *config)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+static int qat_sym_dev_start(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+ return 0;
+}
+
+static void qat_sym_dev_stop(__rte_unused struct rte_cryptodev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+}
+
+static int qat_sym_dev_close(struct rte_cryptodev *dev)
+{
+ int i, ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ ret = qat_sym_qp_release(dev, i);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static void qat_sym_dev_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *info)
+{
+ struct qat_sym_dev_private *internals = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[internals->qat_dev->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+
+ PMD_INIT_FUNC_TRACE();
+ if (info != NULL) {
+ info->max_nb_queue_pairs =
+ qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
+ info->feature_flags = dev->feature_flags;
+ info->capabilities = internals->qat_dev_capabilities;
+ info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
+ info->driver_id = cryptodev_qat_driver_id;
+ info->pci_dev = RTE_DEV_TO_PCI(dev->device);
+ }
+}
+
+static void qat_sym_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ struct qat_common_stats qat_stats = {0};
+ struct qat_sym_dev_private *qat_priv;
+
+ if (stats == NULL || dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid ptr: stats %p, dev %p", stats, dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_get(qat_priv->qat_dev, &qat_stats, QAT_SERVICE_SYMMETRIC);
+ stats->enqueued_count = qat_stats.enqueued_count;
+ stats->dequeued_count = qat_stats.dequeued_count;
+ stats->enqueue_err_count = qat_stats.enqueue_err_count;
+ stats->dequeue_err_count = qat_stats.dequeue_err_count;
+}
+
+static void qat_sym_stats_reset(struct rte_cryptodev *dev)
+{
+ struct qat_sym_dev_private *qat_priv;
+
+ if (dev == NULL) {
+ PMD_DRV_LOG(ERR, "invalid cryptodev ptr %p", dev);
+ return;
+ }
+ qat_priv = dev->data->dev_private;
+
+ qat_stats_reset(qat_priv->qat_dev, QAT_SERVICE_SYMMETRIC);
+
+}
+
+static int qat_sym_qp_release(struct rte_cryptodev *dev, uint16_t queue_pair_id)
+{
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+
+ PMD_DRV_LOG(DEBUG, "Release sym qp %u on device %d",
+ queue_pair_id, dev->data->dev_id);
+
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][queue_pair_id]
+ = NULL;
+
+ return qat_qp_release((struct qat_qp **)
+ &(dev->data->queue_pairs[queue_pair_id]));
+}
+
+static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id, struct rte_mempool *session_pool __rte_unused)
+{
+ struct qat_qp *qp;
+ int ret = 0;
+ uint32_t i;
+ struct qat_qp_config qat_qp_conf;
+
+ struct qat_qp **qp_addr =
+ (struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
+ struct qat_sym_dev_private *qat_private = dev->data->dev_private;
+ const struct qat_qp_hw_data *sym_hw_qps =
+ qp_gen_config[qat_private->qat_dev->qat_dev_gen]
+ .qp_hw_data[QAT_SERVICE_SYMMETRIC];
+ const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
+
+ /* If qp is already in use free ring memory and qp metadata. */
+ if (*qp_addr != NULL) {
+ ret = qat_sym_qp_release(dev, qp_id);
+ if (ret < 0)
+ return ret;
+ }
+ if (qp_id >= qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC)) {
+ PMD_DRV_LOG(ERR, "qp_id %u invalid for this device", qp_id);
+ return -EINVAL;
+ }
+
+ qat_qp_conf.hw = qp_hw_data;
+ qat_qp_conf.build_request = qat_sym_build_request;
+ qat_qp_conf.process_response = qat_sym_process_response;
+ qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
+ qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
+ qat_qp_conf.socket_id = socket_id;
+ qat_qp_conf.service_str = "sym";
+
+ ret = qat_qp_setup(qat_private->qat_dev, qp_addr, qp_id, &qat_qp_conf);
+ if (ret != 0)
+ return ret;
+
+ /* store a link to the qp in the qat_pci_device */
+ qat_private->qat_dev->qps_in_use[QAT_SERVICE_SYMMETRIC][qp_id]
+ = *qp_addr;
+
+ qp = (struct qat_qp *)*qp_addr;
+
+ for (i = 0; i < qp->nb_descriptors; i++) {
+
+ struct qat_sym_op_cookie *sql_cookie =
+ qp->op_cookies[i];
+
+ sql_cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_src);
+
+ sql_cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(sql_cookie) +
+ offsetof(struct qat_sym_op_cookie,
+ qat_sgl_dst);
+ }
+
+ return ret;
+}
+
+static struct rte_cryptodev_ops crypto_qat_ops = {
+
+ /* Device related operations */
+ .dev_configure = qat_sym_dev_config,
+ .dev_start = qat_sym_dev_start,
+ .dev_stop = qat_sym_dev_stop,
+ .dev_close = qat_sym_dev_close,
+ .dev_infos_get = qat_sym_dev_info_get,
+
+ .stats_get = qat_sym_stats_get,
+ .stats_reset = qat_sym_stats_reset,
+ .queue_pair_setup = qat_sym_qp_setup,
+ .queue_pair_release = qat_sym_qp_release,
+ .queue_pair_start = NULL,
+ .queue_pair_stop = NULL,
+ .queue_pair_count = NULL,
+
+ /* Crypto related operations */
+ .session_get_size = qat_sym_session_get_private_size,
+ .session_configure = qat_sym_session_configure,
+ .session_clear = qat_sym_session_clear
+};
+
+static uint16_t
+qat_sym_pmd_enqueue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_enqueue_op_burst(qp, (void **)ops, nb_ops);
+}
+
+static uint16_t
+qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
+}
+
+int
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev_pmd_init_params init_params = {
+ .name = "",
+ .socket_id = qat_pci_dev->pci_dev->device.numa_node,
+ .private_data_size = sizeof(struct qat_sym_dev_private),
+ .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
+ };
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct rte_cryptodev *cryptodev;
+ struct qat_sym_dev_private *internals;
+
+ snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
+ qat_pci_dev->name, "sym");
+ PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+
+ cryptodev = rte_cryptodev_pmd_create(name,
+ &qat_pci_dev->pci_dev->device, &init_params);
+
+ if (cryptodev == NULL)
+ return -ENODEV;
+
+ cryptodev->driver_id = cryptodev_qat_driver_id;
+ cryptodev->dev_ops = &crypto_qat_ops;
+
+ cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
+ cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
+
+ cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
+ RTE_CRYPTODEV_FF_HW_ACCELERATED |
+ RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
+ RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
+
+ internals = cryptodev->data->dev_private;
+ internals->qat_dev = qat_pci_dev;
+ qat_pci_dev->sym_dev = internals;
+
+ internals->sym_dev_id = cryptodev->data->dev_id;
+ switch (qat_pci_dev->qat_dev_gen) {
+ case QAT_GEN1:
+ internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
+ break;
+ case QAT_GEN2:
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ break;
+ default:
+ internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
+ PMD_DRV_LOG(DEBUG,
+ "QAT gen %d capabilities unknown, default to GEN2",
+ qat_pci_dev->qat_dev_gen);
+ break;
+ }
+
+ PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
+ name, internals->sym_dev_id);
+ return 0;
+}
+
+int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
+{
+ struct rte_cryptodev *cryptodev;
+
+ if (qat_pci_dev == NULL)
+ return -ENODEV;
+ if (qat_pci_dev->sym_dev == NULL)
+ return 0;
+
+ /* free crypto device */
+ cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
+ rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_dev = NULL;
+
+ return 0;
+}
+
+
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+static struct cryptodev_driver qat_crypto_drv;
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+ cryptodev_qat_driver_id);
diff --git a/drivers/crypto/qat/qat_sym_pmd.h b/drivers/crypto/qat/qat_sym_pmd.h
new file mode 100644
index 000000000..efa3b0775
--- /dev/null
+++ b/drivers/crypto/qat/qat_sym_pmd.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2015-2018 Intel Corporation
+ */
+
+#ifndef _QAT_SYM_PMD_H_
+#define _QAT_SYM_PMD_H_
+
+#include <rte_cryptodev.h>
+
+#include "qat_sym_capabilities.h"
+#include "qat_device.h"
+
+
+/**< Intel(R) QAT Symmetric Crypto PMD device name */
+#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+
+extern uint8_t cryptodev_qat_driver_id;
+
+/** private data structure for a QAT device.
+ * This QAT device is a device offering only symmetric crypto service,
+ * there can be one of these on each qat_pci_device (VF),
+ * in future there may also be private data structures for other services.
+ */
+struct qat_sym_dev_private {
+ struct qat_pci_device *qat_dev;
+ /**< The qat pci device hosting the service */
+ uint8_t sym_dev_id;
+ /**< Device instance for this rte_cryptodev */
+ const struct rte_cryptodev_capabilities *qat_dev_capabilities;
+ /* QAT device symmetric crypto capabilities */
+};
+
+
+int
+qat_sym_dev_create(struct qat_pci_device *qat_pci_dev);
+
+int
+qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev);
+#endif /* _QAT_SYM_PMD_H_ */
diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c
index 68d7773a2..689596d51 100644
--- a/drivers/crypto/qat/qat_sym_session.c
+++ b/drivers/crypto/qat/qat_sym_session.c
@@ -1,6 +1,12 @@
/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
* Copyright(c) 2015-2018 Intel Corporation
*/
+
+#include <openssl/sha.h> /* Needed to calculate pre-compute values */
+#include <openssl/aes.h> /* Needed to calculate pre-compute values */
+#include <openssl/md5.h> /* Needed to calculate pre-compute values */
+#include <openssl/evp.h> /* Needed for bpi runt block processing */
+
#include <rte_memcpy.h>
#include <rte_common.h>
#include <rte_spinlock.h>
@@ -10,14 +16,8 @@
#include <rte_crypto_sym.h>
#include "qat_logs.h"
-#include "qat_device.h"
-
-#include <openssl/sha.h> /* Needed to calculate pre-compute values */
-#include <openssl/aes.h> /* Needed to calculate pre-compute values */
-#include <openssl/md5.h> /* Needed to calculate pre-compute values */
-#include <openssl/evp.h>
-
#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
/** Frees a context previously created
* Depends on openssl libcrypto
@@ -29,6 +29,9 @@ bpi_cipher_ctx_free(void *bpi_ctx)
EVP_CIPHER_CTX_free((EVP_CIPHER_CTX *)bpi_ctx);
}
+/** Creates a context in either AES or DES in ECB mode
+ * Depends on openssl libcrypto
+ */
static int
bpi_cipher_ctx_init(enum rte_crypto_cipher_algorithm cryptodev_algo,
enum rte_crypto_cipher_operation direction __rte_unused,
diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h
index d3d27ff46..18d77e990 100644
--- a/drivers/crypto/qat/qat_sym_session.h
+++ b/drivers/crypto/qat/qat_sym_session.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2015-2018 Intel Corporation
*/
#ifndef _QAT_SYM_SESSION_H_
#define _QAT_SYM_SESSION_H_
diff --git a/drivers/crypto/qat/rte_qat_cryptodev.c b/drivers/crypto/qat/rte_qat_cryptodev.c
deleted file mode 100644
index 91bb1e590..000000000
--- a/drivers/crypto/qat/rte_qat_cryptodev.c
+++ /dev/null
@@ -1,258 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2015-2018 Intel Corporation
- */
-
-#include <rte_bus_pci.h>
-#include <rte_common.h>
-#include <rte_dev.h>
-#include <rte_malloc.h>
-#include <rte_pci.h>
-#include <rte_cryptodev_pmd.h>
-
-#include "qat_sym.h"
-#include "qat_sym_session.h"
-#include "qat_logs.h"
-
-uint8_t cryptodev_qat_driver_id;
-
-static const struct rte_cryptodev_capabilities qat_gen1_sym_capabilities[] = {
- QAT_BASE_GEN1_SYM_CAPABILITIES,
- RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
-};
-
-static const struct rte_cryptodev_capabilities qat_gen2_sym_capabilities[] = {
- QAT_BASE_GEN1_SYM_CAPABILITIES,
- QAT_EXTRA_GEN2_SYM_CAPABILITIES,
- RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
-};
-
-static struct rte_cryptodev_ops crypto_qat_ops = {
-
- /* Device related operations */
- .dev_configure = qat_sym_dev_config,
- .dev_start = qat_sym_dev_start,
- .dev_stop = qat_sym_dev_stop,
- .dev_close = qat_sym_dev_close,
- .dev_infos_get = qat_sym_dev_info_get,
-
- .stats_get = qat_sym_stats_get,
- .stats_reset = qat_sym_stats_reset,
- .queue_pair_setup = qat_sym_qp_setup,
- .queue_pair_release = qat_sym_qp_release,
- .queue_pair_start = NULL,
- .queue_pair_stop = NULL,
- .queue_pair_count = NULL,
-
- /* Crypto related operations */
- .session_get_size = qat_sym_session_get_private_size,
- .session_configure = qat_sym_session_configure,
- .session_clear = qat_sym_session_clear
-};
-
-/*
- * The set of PCI devices this driver supports
- */
-
-static const struct rte_pci_id pci_id_qat_map[] = {
- {
- RTE_PCI_DEVICE(0x8086, 0x0443),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x37c9),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x19e3),
- },
- {
- RTE_PCI_DEVICE(0x8086, 0x6f55),
- },
- {.device_id = 0},
-};
-
-
-
-static int
-qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
-{
- struct rte_cryptodev_pmd_init_params init_params = {
- .name = "",
- .socket_id = qat_pci_dev->pci_dev->device.numa_node,
- .private_data_size = sizeof(struct qat_sym_dev_private),
- .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
- };
- char name[RTE_CRYPTODEV_NAME_MAX_LEN];
- struct rte_cryptodev *cryptodev;
- struct qat_sym_dev_private *internals;
-
- snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%s",
- qat_pci_dev->name, "sym");
- PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
-
- cryptodev = rte_cryptodev_pmd_create(name,
- &qat_pci_dev->pci_dev->device, &init_params);
-
- if (cryptodev == NULL)
- return -ENODEV;
-
- cryptodev->driver_id = cryptodev_qat_driver_id;
- cryptodev->dev_ops = &crypto_qat_ops;
-
- cryptodev->enqueue_burst = qat_sym_pmd_enqueue_op_burst;
- cryptodev->dequeue_burst = qat_sym_pmd_dequeue_op_burst;
-
- cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
- RTE_CRYPTODEV_FF_HW_ACCELERATED |
- RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
- RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
-
- internals = cryptodev->data->dev_private;
- internals->qat_dev = qat_pci_dev;
- qat_pci_dev->sym_dev = internals;
-
- internals->sym_dev_id = cryptodev->data->dev_id;
- switch (qat_pci_dev->qat_dev_gen) {
- case QAT_GEN1:
- internals->qat_dev_capabilities = qat_gen1_sym_capabilities;
- break;
- case QAT_GEN2:
- internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
- break;
- default:
- internals->qat_dev_capabilities = qat_gen2_sym_capabilities;
- PMD_DRV_LOG(DEBUG,
- "QAT gen %d capabilities unknown, default to GEN2",
- qat_pci_dev->qat_dev_gen);
- break;
- }
-
- PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
- name, internals->sym_dev_id);
- return 0;
-}
-
-static int
-qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
-{
- struct rte_cryptodev *cryptodev;
-
- if (qat_pci_dev == NULL)
- return -ENODEV;
- if (qat_pci_dev->sym_dev == NULL)
- return 0;
-
- /* free crypto device */
- cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
- rte_cryptodev_pmd_destroy(cryptodev);
- qat_pci_dev->sym_dev = NULL;
-
- return 0;
-}
-
-static int
-qat_comp_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_comp_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_asym_dev_create(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_asym_dev_destroy(struct qat_pci_device *qat_pci_dev __rte_unused)
-{
- return 0;
-}
-
-static int
-qat_pci_dev_destroy(struct qat_pci_device *qat_pci_dev,
- struct rte_pci_device *pci_dev)
-{
- qat_sym_dev_destroy(qat_pci_dev);
- qat_comp_dev_destroy(qat_pci_dev);
- qat_asym_dev_destroy(qat_pci_dev);
- return qat_pci_device_release(pci_dev);
-}
-
-static int qat_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
- struct rte_pci_device *pci_dev)
-{
- int ret = 0;
- struct qat_pci_device *qat_pci_dev;
-
- PMD_DRV_LOG(DEBUG, "Found QAT device at %02x:%02x.%x",
- pci_dev->addr.bus,
- pci_dev->addr.devid,
- pci_dev->addr.function);
-
- qat_pci_dev = qat_pci_device_allocate(pci_dev);
- if (qat_pci_dev == NULL)
- return -ENODEV;
-
- ret = qat_sym_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- ret = qat_comp_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- ret = qat_asym_dev_create(qat_pci_dev);
- if (ret != 0)
- goto error_out;
-
- return 0;
-
-error_out:
- qat_pci_dev_destroy(qat_pci_dev, pci_dev);
- return ret;
-
-}
-
-static int qat_pci_remove(struct rte_pci_device *pci_dev)
-{
- struct qat_pci_device *qat_pci_dev;
-
- if (pci_dev == NULL)
- return -EINVAL;
-
- qat_pci_dev = qat_get_qat_dev_from_pci_dev(pci_dev);
- if (qat_pci_dev == NULL)
- return 0;
-
- return qat_pci_dev_destroy(qat_pci_dev, pci_dev);
-
-}
-
-
-/* An rte_driver is needed in the registration of both the device and the driver
- * with cryptodev.
- * The actual qat pci's rte_driver can't be used as its name represents
- * the whole pci device with all services. Think of this as a holder for a name
- * for the crypto part of the pci device.
- */
-static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
-static struct rte_driver cryptodev_qat_sym_driver = {
- .name = qat_sym_drv_name,
- .alias = qat_sym_drv_name
-};
-static struct cryptodev_driver qat_crypto_drv;
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
- cryptodev_qat_driver_id);
-
-static struct rte_pci_driver rte_qat_pmd = {
- .id_table = pci_id_qat_map,
- .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
- .probe = qat_pci_probe,
- .remove = qat_pci_remove
-};
-RTE_PMD_REGISTER_PCI(QAT_PCI_NAME, rte_qat_pmd);
-RTE_PMD_REGISTER_PCI_TABLE(QAT_PCI_NAME, pci_id_qat_map);
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 24/38] crypto/qat: add lock around csr access and change logic
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (22 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 23/38] crypto/qat: move code into appropriate files Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 25/38] crypto/qat: remove incorrect usage of bundle number Tomasz Jozwiak
` (14 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Add lock around accesses to the arbiter CSR
and use & instead of ^ as ^ not safe if
arb_disable called when already disabled.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 26 +++++++++++++++++++-------
1 file changed, 19 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 7b2dc3f90..f26fd0900 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -107,8 +107,10 @@ static int qat_queue_create(struct qat_pci_device *qat_dev,
static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
uint32_t *queue_size_for_csr);
static void adf_configure_queues(struct qat_qp *queue);
-static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr);
-static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr);
+static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock);
+static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock);
int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
@@ -216,7 +218,8 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
}
adf_configure_queues(qp);
- adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr);
+ adf_queue_arb_enable(&qp->tx_q, qp->mmap_bar_addr,
+ &qat_dev->arb_csr_lock);
snprintf(op_cookie_pool_name, RTE_RING_NAMESIZE,
"%s%d_cookies_%s_qp%hu",
@@ -282,7 +285,8 @@ int qat_qp_release(struct qat_qp **qp_addr)
return -EAGAIN;
}
- adf_queue_arb_disable(&(qp->tx_q), qp->mmap_bar_addr);
+ adf_queue_arb_disable(&(qp->tx_q), qp->mmap_bar_addr,
+ &qp->qat_dev->arb_csr_lock);
for (i = 0; i < qp->nb_descriptors; i++)
rte_mempool_put(qp->op_cookie_pool, qp->op_cookies[i]);
@@ -443,7 +447,8 @@ static int adf_verify_queue_size(uint32_t msg_size, uint32_t msg_num,
return -EINVAL;
}
-static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr)
+static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock)
{
uint32_t arb_csr_offset = ADF_ARB_RINGSRVARBEN_OFFSET +
(ADF_ARB_REG_SLOT *
@@ -451,12 +456,16 @@ static void adf_queue_arb_enable(struct qat_queue *txq, void *base_addr)
uint32_t value;
PMD_INIT_FUNC_TRACE();
+
+ rte_spinlock_lock(lock);
value = ADF_CSR_RD(base_addr, arb_csr_offset);
value |= (0x01 << txq->hw_queue_number);
ADF_CSR_WR(base_addr, arb_csr_offset, value);
+ rte_spinlock_unlock(lock);
}
-static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr)
+static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr,
+ rte_spinlock_t *lock)
{
uint32_t arb_csr_offset = ADF_ARB_RINGSRVARBEN_OFFSET +
(ADF_ARB_REG_SLOT *
@@ -464,9 +473,12 @@ static void adf_queue_arb_disable(struct qat_queue *txq, void *base_addr)
uint32_t value;
PMD_INIT_FUNC_TRACE();
+
+ rte_spinlock_lock(lock);
value = ADF_CSR_RD(base_addr, arb_csr_offset);
- value ^= (0x01 << txq->hw_queue_number);
+ value &= ~(0x01 << txq->hw_queue_number);
ADF_CSR_WR(base_addr, arb_csr_offset, value);
+ rte_spinlock_unlock(lock);
}
static void adf_configure_queues(struct qat_qp *qp)
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 25/38] crypto/qat: remove incorrect usage of bundle number
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (23 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 24/38] crypto/qat: add lock around csr access and change logic Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 26/38] crypto/qat: rename variables Tomasz Jozwiak
` (13 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
As bundle_num is included in qat_gen1_qps static array
there shouldn't be a multiplier used in qat_qps_per_service()
Then removed ADF_NUM_BUNDLES_PER_DEV as no longer used.
Also renamed ADF_MAX_QPS_PER_BUNDLE to ADF_MAX_QPS_ON_ANY_SERVICE
and reduced from 4 to 2 which is enough for all current devices.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
.../qat/qat_adf/adf_transport_access_macros.h | 5 ++---
drivers/crypto/qat/qat_common.c | 4 ++--
drivers/crypto/qat/qat_device.h | 4 ++--
drivers/crypto/qat/qat_qp.c | 19 ++++---------------
drivers/crypto/qat/qat_qp.h | 2 +-
5 files changed, 11 insertions(+), 23 deletions(-)
diff --git a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
index 2136d54ab..1eef5513f 100644
--- a/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
+++ b/drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
@@ -50,9 +50,8 @@
#define ADF_MAX_RING_SIZE ADF_RING_SIZE_4M
#define ADF_DEFAULT_RING_SIZE ADF_RING_SIZE_16K
-#define ADF_NUM_BUNDLES_PER_DEV 1
-/* Maximum number of qps for any service type */
-#define ADF_MAX_QPS_PER_BUNDLE 4
+/* Maximum number of qps on a device for any service type */
+#define ADF_MAX_QPS_ON_ANY_SERVICE 2
#define ADF_RING_DIR_TX 0
#define ADF_RING_DIR_RX 1
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index f1759ea76..5d6779757 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -67,7 +67,7 @@ void qat_stats_get(struct qat_pci_device *dev,
}
qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ for (i = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++) {
if (qp[i] == NULL) {
PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
service, i);
@@ -94,7 +94,7 @@ void qat_stats_reset(struct qat_pci_device *dev,
}
qp = dev->qps_in_use[service];
- for (i = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++) {
+ for (i = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++) {
if (qp[i] == NULL) {
PMD_DRV_LOG(DEBUG, "Service %d Uninitialised qp %d",
service, i);
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index fd1819354..fd20a0147 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -44,7 +44,7 @@ struct qat_pci_device {
uint8_t attached : 1;
/**< Flag indicating the device is attached */
- struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_PER_BUNDLE];
+ struct qat_qp *qps_in_use[QAT_MAX_SERVICES][ADF_MAX_QPS_ON_ANY_SERVICE];
/**< links to qps set up for each service, index same as on API */
/* Data relating to symmetric crypto service */
@@ -59,7 +59,7 @@ struct qat_pci_device {
struct qat_gen_hw_data {
enum qat_device_gen dev_gen;
- const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_PER_BUNDLE];
+ const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_ON_ANY_SERVICE];
};
extern struct qat_gen_hw_data qp_gen_config[];
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index f26fd0900..9938c1493 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -29,7 +29,7 @@
__extension__
const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
- [ADF_MAX_QPS_PER_BUNDLE] = {
+ [ADF_MAX_QPS_ON_ANY_SERVICE] = {
/* queue pairs which provide an asymmetric crypto service */
[QAT_SERVICE_ASYMMETRIC] = {
{
@@ -42,14 +42,11 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
}, {
.service_type = QAT_SERVICE_ASYMMETRIC,
+ .hw_bundle_num = 0,
.tx_ring_num = 1,
.rx_ring_num = 9,
.tx_msg_size = 64,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
},
/* queue pairs which provide a symmetric crypto service */
@@ -69,10 +66,6 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
.rx_ring_num = 11,
.tx_msg_size = 128,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
},
/* queue pairs which provide a compression service */
@@ -91,10 +84,6 @@ const struct qat_qp_hw_data qat_gen1_qps[QAT_MAX_SERVICES]
.rx_ring_num = 15,
.tx_msg_size = 128,
.rx_msg_size = 32,
- }, {
- .service_type = QAT_SERVICE_INVALID,
- }, {
- .service_type = QAT_SERVICE_INVALID,
}
}
};
@@ -118,10 +107,10 @@ int qat_qps_per_service(const struct qat_qp_hw_data *qp_hw_data,
{
int i, count;
- for (i = 0, count = 0; i < ADF_MAX_QPS_PER_BUNDLE; i++)
+ for (i = 0, count = 0; i < ADF_MAX_QPS_ON_ANY_SERVICE; i++)
if (qp_hw_data[i].service_type == service)
count++;
- return count * ADF_NUM_BUNDLES_PER_DEV;
+ return count;
}
static const struct rte_memzone *
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 73888b805..6f07bd67c 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -90,7 +90,7 @@ struct qat_qp {
/**< qat device this qp is on */
} __rte_cache_aligned;
-extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_PER_BUNDLE];
+extern const struct qat_qp_hw_data qat_gen1_qps[][ADF_MAX_QPS_ON_ANY_SERVICE];
uint16_t
qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops);
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 26/38] crypto/qat: rename variables
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (24 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 25/38] crypto/qat: remove incorrect usage of bundle number Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 27/38] crypto/qat: modify debug message Tomasz Jozwiak
` (12 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Renamed sgl_cookie to cookie and qp_gen_config
to qat_gen_config, as it is intended to hold
more than just queue pair data.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/qat_device.c | 2 +-
drivers/crypto/qat/qat_device.h | 2 +-
drivers/crypto/qat/qat_sym_pmd.c | 14 +++++++-------
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index c9d4b32ed..a0e414905 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -10,7 +10,7 @@
/* Hardware device information per generation */
__extension__
-struct qat_gen_hw_data qp_gen_config[] = {
+struct qat_gen_hw_data qat_gen_config[] = {
[QAT_GEN1] = {
.dev_gen = QAT_GEN1,
.qp_hw_data = qat_gen1_qps,
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index fd20a0147..4201a1c71 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -62,7 +62,7 @@ struct qat_gen_hw_data {
const struct qat_qp_hw_data (*qp_hw_data)[ADF_MAX_QPS_ON_ANY_SERVICE];
};
-extern struct qat_gen_hw_data qp_gen_config[];
+extern struct qat_gen_hw_data qat_gen_config[];
struct qat_pci_device *
qat_pci_device_allocate(struct rte_pci_device *pci_dev);
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index aa71b4641..e6760b8f8 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -68,7 +68,7 @@ static void qat_sym_dev_info_get(struct rte_cryptodev *dev,
{
struct qat_sym_dev_private *internals = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[internals->qat_dev->qat_dev_gen]
+ qat_gen_config[internals->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
PMD_INIT_FUNC_TRACE();
@@ -143,7 +143,7 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
(struct qat_qp **)&(dev->data->queue_pairs[qp_id]);
struct qat_sym_dev_private *qat_private = dev->data->dev_private;
const struct qat_qp_hw_data *sym_hw_qps =
- qp_gen_config[qat_private->qat_dev->qat_dev_gen]
+ qat_gen_config[qat_private->qat_dev->qat_dev_gen]
.qp_hw_data[QAT_SERVICE_SYMMETRIC];
const struct qat_qp_hw_data *qp_hw_data = sym_hw_qps + qp_id;
@@ -178,16 +178,16 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
for (i = 0; i < qp->nb_descriptors; i++) {
- struct qat_sym_op_cookie *sql_cookie =
+ struct qat_sym_op_cookie *cookie =
qp->op_cookies[i];
- sql_cookie->qat_sgl_src_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
+ cookie->qat_sgl_src_phys_addr =
+ rte_mempool_virt2iova(cookie) +
offsetof(struct qat_sym_op_cookie,
qat_sgl_src);
- sql_cookie->qat_sgl_dst_phys_addr =
- rte_mempool_virt2iova(sql_cookie) +
+ cookie->qat_sgl_dst_phys_addr =
+ rte_mempool_virt2iova(cookie) +
offsetof(struct qat_sym_op_cookie,
qat_sgl_dst);
}
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 27/38] crypto/qat: modify debug message
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (25 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 26/38] crypto/qat: rename variables Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 28/38] crypto/qat: free cookie pool on queue creation error Tomasz Jozwiak
` (11 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Removed crypto reference in common debug message.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/qat_common.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/crypto/qat/qat_common.c b/drivers/crypto/qat/qat_common.c
index 5d6779757..c10c1421a 100644
--- a/drivers/crypto/qat/qat_common.c
+++ b/drivers/crypto/qat/qat_common.c
@@ -103,5 +103,5 @@ void qat_stats_reset(struct qat_pci_device *dev,
memset(&(qp[i]->stats), 0, sizeof(qp[i]->stats));
}
- PMD_DRV_LOG(DEBUG, "QAT crypto: %d stats cleared", service);
+ PMD_DRV_LOG(DEBUG, "QAT: %d stats cleared", service);
}
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 28/38] crypto/qat: free cookie pool on queue creation error
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (26 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 27/38] crypto/qat: modify debug message Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 29/38] crypto/qat: remove unused macro Tomasz Jozwiak
` (10 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/qat_qp.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 9938c1493..569eace57 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -248,6 +248,9 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
return 0;
create_err:
+ if (qp->op_cookie_pool)
+ rte_mempool_free(qp->op_cookie_pool);
+ rte_free(qp->op_cookies);
rte_free(qp);
return -EFAULT;
}
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 29/38] crypto/qat: remove unused macro
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (27 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 28/38] crypto/qat: free cookie pool on queue creation error Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 30/38] crypto/qat: move macro to common file Tomasz Jozwiak
` (9 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/qat_sym.h | 6 ------
1 file changed, 6 deletions(-)
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index d887dc126..e46448bd2 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -9,12 +9,6 @@
#include "qat_common.h"
-/*
- * This macro rounds up a number to a be a multiple of
- * the alignment when the alignment is a power of 2
- */
-#define ALIGN_POW2_ROUNDUP(num, align) \
- (((num) + (align) - 1) & ~((align) - 1))
#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
struct qat_sym_session;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 30/38] crypto/qat: move macro to common file
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (28 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 29/38] crypto/qat: remove unused macro Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 31/38] crypto/qat: register appropriately named device Tomasz Jozwiak
` (8 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
QAT_64_BTYE_ALIGN_MASK macro is not specific to
symmetric crypto, but it is common for other
future services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
drivers/crypto/qat/qat_common.h | 2 ++
drivers/crypto/qat/qat_sym.h | 2 --
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/qat/qat_common.h b/drivers/crypto/qat/qat_common.h
index 8ecebe954..db85d5482 100644
--- a/drivers/crypto/qat/qat_common.h
+++ b/drivers/crypto/qat/qat_common.h
@@ -15,6 +15,8 @@
*/
#define QAT_SGL_MAX_NUMBER 16
+#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
+
/* Intel(R) QuickAssist Technology device generation is enumerated
* from one according to the generation of the device
*/
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index e46448bd2..bdb672be4 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -9,8 +9,6 @@
#include "qat_common.h"
-#define QAT_64_BTYE_ALIGN_MASK (~0x3f)
-
struct qat_sym_session;
struct qat_sym_op_cookie {
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 31/38] crypto/qat: register appropriately named device
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (29 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 30/38] crypto/qat: move macro to common file Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 32/38] crypto/qat: add max PCI devices to config file Tomasz Jozwiak
` (7 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
For every QAT PCI device probed, populate a local rte_device
containing an rte_driver. The rte_driver was created in a previous
patch to provide a crypto-specific driver name: "crypto_qat".
This was previously only used for driver registration, now it's also
used in device creation.
This allows applications to find devices driven by "crypto_qat".
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_device.h | 5 +++++
drivers/crypto/qat/qat_sym_pmd.c | 38 ++++++++++++++++++++------------
2 files changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 4201a1c71..3df6520c3 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -50,6 +50,11 @@ struct qat_pci_device {
/* Data relating to symmetric crypto service */
struct qat_sym_dev_private *sym_dev;
/**< link back to cryptodev private data */
+ struct rte_device sym_rte_dev;
+ /**< This represents the crypto subset of this pci device.
+ * Register with this rather than with the one in
+ * pci_dev so that its driver can have a crypto-specific name
+ */
/* Data relating to compression service */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index e6760b8f8..28e579b77 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -232,6 +232,18 @@ qat_sym_pmd_dequeue_op_burst(void *qp, struct rte_crypto_op **ops,
return qat_dequeue_op_burst(qp, (void **)ops, nb_ops);
}
+/* An rte_driver is needed in the registration of both the device and the driver
+ * with cryptodev.
+ * The actual qat pci's rte_driver can't be used as its name represents
+ * the whole pci device with all services. Think of this as a holder for a name
+ * for the crypto part of the pci device.
+ */
+static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
+static const struct rte_driver cryptodev_qat_sym_driver = {
+ .name = qat_sym_drv_name,
+ .alias = qat_sym_drv_name
+};
+
int
qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
{
@@ -249,12 +261,19 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
qat_pci_dev->name, "sym");
PMD_DRV_LOG(DEBUG, "Creating QAT SYM device %s", name);
+ /* Populate subset device to use in cryptodev device creation */
+ qat_pci_dev->sym_rte_dev.driver = &cryptodev_qat_sym_driver;
+ qat_pci_dev->sym_rte_dev.numa_node =
+ qat_pci_dev->pci_dev->device.numa_node;
+ qat_pci_dev->sym_rte_dev.devargs = NULL;
+
cryptodev = rte_cryptodev_pmd_create(name,
- &qat_pci_dev->pci_dev->device, &init_params);
+ &(qat_pci_dev->sym_rte_dev), &init_params);
if (cryptodev == NULL)
return -ENODEV;
+ qat_pci_dev->sym_rte_dev.name = cryptodev->data->name;
cryptodev->driver_id = cryptodev_qat_driver_id;
cryptodev->dev_ops = &crypto_qat_ops;
@@ -287,7 +306,7 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
}
PMD_DRV_LOG(DEBUG, "Created QAT SYM device %s as cryptodev instance %d",
- name, internals->sym_dev_id);
+ cryptodev->data->name, internals->sym_dev_id);
return 0;
}
@@ -304,23 +323,14 @@ qat_sym_dev_destroy(struct qat_pci_device *qat_pci_dev)
/* free crypto device */
cryptodev = rte_cryptodev_pmd_get_dev(qat_pci_dev->sym_dev->sym_dev_id);
rte_cryptodev_pmd_destroy(cryptodev);
+ qat_pci_dev->sym_rte_dev.name = NULL;
qat_pci_dev->sym_dev = NULL;
return 0;
}
-/* An rte_driver is needed in the registration of both the device and the driver
- * with cryptodev.
- * The actual qat pci's rte_driver can't be used as its name represents
- * the whole pci device with all services. Think of this as a holder for a name
- * for the crypto part of the pci device.
- */
-static const char qat_sym_drv_name[] = RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD);
-static struct rte_driver cryptodev_qat_sym_driver = {
- .name = qat_sym_drv_name,
- .alias = qat_sym_drv_name
-};
static struct cryptodev_driver qat_crypto_drv;
-RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv, cryptodev_qat_sym_driver,
+RTE_PMD_REGISTER_CRYPTO_DRIVER(qat_crypto_drv,
+ cryptodev_qat_sym_driver,
cryptodev_qat_driver_id);
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 32/38] crypto/qat: add max PCI devices to config file
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (30 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 31/38] crypto/qat: register appropriately named device Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 33/38] crypto/qat: optimize adf modulo function Tomasz Jozwiak
` (6 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Added CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES to
build config files.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
config/common_base | 5 ++++-
config/rte_config.h | 2 ++
drivers/crypto/qat/qat_device.c | 8 ++++----
drivers/crypto/qat/qat_device.h | 2 --
4 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/config/common_base b/config/common_base
index 6b0d1cbbb..cf0741199 100644
--- a/config/common_base
+++ b/config/common_base
@@ -495,7 +495,10 @@ CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n
# on a single QuickAssist device.
#
CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=2048
-
+#
+# Max. number of QuickAssist devices, which can be detected and attached
+#
+CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48
#
# Compile PMD for virtio crypto devices
#
diff --git a/config/rte_config.h b/config/rte_config.h
index a1d01759e..e353d8d6d 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -91,6 +91,8 @@
*/
/* QuickAssist device */
#define RTE_QAT_PMD_MAX_NB_SESSIONS 2048
+/* Max. number of QuickAssist devices which can be attached */
+#define RTE_PMD_QAT_MAX_PCI_DEVICES 48
/* virtio crypto defines */
#define RTE_VIRTIO_CRYPTO_PMD_MAX_NB_SESSIONS 1024
diff --git a/drivers/crypto/qat/qat_device.c b/drivers/crypto/qat/qat_device.c
index a0e414905..0cab2e928 100644
--- a/drivers/crypto/qat/qat_device.c
+++ b/drivers/crypto/qat/qat_device.c
@@ -23,7 +23,7 @@ struct qat_gen_hw_data qat_gen_config[] = {
};
-static struct qat_pci_device qat_pci_devices[QAT_MAX_PCI_DEVICES];
+static struct qat_pci_device qat_pci_devices[RTE_PMD_QAT_MAX_PCI_DEVICES];
static int qat_nb_pci_devices;
/*
@@ -62,7 +62,7 @@ qat_pci_get_named_dev(const char *name)
if (name == NULL)
return NULL;
- for (i = 0; i < QAT_MAX_PCI_DEVICES; i++) {
+ for (i = 0; i < RTE_PMD_QAT_MAX_PCI_DEVICES; i++) {
dev = &qat_pci_devices[i];
if ((dev->attached == QAT_ATTACHED) &&
@@ -78,7 +78,7 @@ qat_pci_find_free_device_index(void)
{
uint8_t dev_id;
- for (dev_id = 0; dev_id < QAT_MAX_PCI_DEVICES; dev_id++) {
+ for (dev_id = 0; dev_id < RTE_PMD_QAT_MAX_PCI_DEVICES; dev_id++) {
if (qat_pci_devices[dev_id].attached == QAT_DETACHED)
break;
}
@@ -111,7 +111,7 @@ qat_pci_device_allocate(struct rte_pci_device *pci_dev)
}
qat_dev_id = qat_pci_find_free_device_index();
- if (qat_dev_id == QAT_MAX_PCI_DEVICES) {
+ if (qat_dev_id == RTE_PMD_QAT_MAX_PCI_DEVICES) {
PMD_DRV_LOG(ERR, "Reached maximum number of QAT devices");
return NULL;
}
diff --git a/drivers/crypto/qat/qat_device.h b/drivers/crypto/qat/qat_device.h
index 3df6520c3..e18c8a706 100644
--- a/drivers/crypto/qat/qat_device.h
+++ b/drivers/crypto/qat/qat_device.h
@@ -11,11 +11,9 @@
#include "adf_transport_access_macros.h"
#include "qat_qp.h"
-
#define QAT_DETACHED (0)
#define QAT_ATTACHED (1)
-#define QAT_MAX_PCI_DEVICES 48
#define QAT_DEV_NAME_MAX_LEN 64
/*
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 33/38] crypto/qat: optimize adf modulo function
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (31 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 32/38] crypto/qat: add max PCI devices to config file Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 34/38] crypto/qat: remove unused arguments Tomasz Jozwiak
` (5 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 19 ++++++++-----------
drivers/crypto/qat/qat_qp.h | 2 +-
2 files changed, 9 insertions(+), 12 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 569eace57..b84ba643c 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -376,7 +376,7 @@ qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
queue->max_inflights = ADF_MAX_INFLIGHTS(queue->queue_size,
ADF_BYTES_TO_MSG_SIZE(desc_size));
- queue->modulo = ADF_RING_SIZE_MODULO(queue->queue_size);
+ queue->modulo_mask = (1 << ADF_RING_SIZE_MODULO(queue->queue_size)) - 1;
if (queue->max_inflights < 2) {
PMD_DRV_LOG(ERR, "Invalid num inflights");
@@ -401,11 +401,11 @@ qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
queue->hw_queue_number, queue_base);
PMD_DRV_LOG(DEBUG, "RING: Name:%s, size in CSR: %u, in bytes %u,"
- " nb msgs %u, msg_size %u, max_inflights %u modulo %u",
+ " nb msgs %u, msg_size %u, max_inflights %u modulo mask %u",
queue->memz_name,
queue->queue_size, queue_size_bytes,
qp_conf->nb_descriptors, desc_size,
- queue->max_inflights, queue->modulo);
+ queue->max_inflights, queue->modulo_mask);
return 0;
@@ -494,13 +494,9 @@ static void adf_configure_queues(struct qat_qp *qp)
queue->hw_queue_number, queue_config);
}
-
-static inline uint32_t adf_modulo(uint32_t data, uint32_t shift)
+static inline uint32_t adf_modulo(uint32_t data, uint32_t modulo_mask)
{
- uint32_t div = data >> shift;
- uint32_t mult = div << shift;
-
- return data - mult;
+ return data & modulo_mask;
}
static inline void
@@ -584,7 +580,7 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
goto kick_tail;
}
- tail = adf_modulo(tail + queue->msg_size, queue->modulo);
+ tail = adf_modulo(tail + queue->msg_size, queue->modulo_mask);
ops++;
nb_ops_sent++;
}
@@ -620,7 +616,8 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
tmp_qp->op_cookies[head / rx_queue->msg_size],
tmp_qp->qat_dev_gen);
- head = adf_modulo(head + rx_queue->msg_size, rx_queue->modulo);
+ head = adf_modulo(head + rx_queue->msg_size,
+ rx_queue->modulo_mask);
resp_msg = (uint8_t *)rx_queue->base_addr + head;
ops++;
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 6f07bd67c..764125d59 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -59,7 +59,7 @@ struct qat_queue {
rte_iova_t base_phys_addr; /* Queue physical address */
uint32_t head; /* Shadow copy of the head */
uint32_t tail; /* Shadow copy of the tail */
- uint32_t modulo;
+ uint32_t modulo_mask;
uint32_t msg_size;
uint16_t max_inflights;
uint32_t queue_size;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 34/38] crypto/qat: remove unused arguments
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (32 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 33/38] crypto/qat: optimize adf modulo function Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 35/38] crypto/qat: make response process function inline Tomasz Jozwiak
` (4 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Process response function is only implemented
for crypto symmetric operations, which do not require
some of the arguments.
Therefore, these arguments can be removed from the
function prototype.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 4 +---
drivers/crypto/qat/qat_qp.h | 4 +---
drivers/crypto/qat/qat_sym.c | 4 +---
drivers/crypto/qat/qat_sym.h | 4 +---
4 files changed, 4 insertions(+), 12 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index b84ba643c..0fdec0da0 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -612,9 +612,7 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
resp_counter != nb_ops) {
- tmp_qp->process_response(ops, resp_msg,
- tmp_qp->op_cookies[head / rx_queue->msg_size],
- tmp_qp->qat_dev_gen);
+ tmp_qp->process_response(ops, resp_msg);
head = adf_modulo(head + rx_queue->msg_size,
rx_queue->modulo_mask);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 764125d59..eb9188410 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -21,9 +21,7 @@ typedef int (*build_request_t)(void *op,
enum qat_device_gen qat_dev_gen);
/**< Build a request from an op. */
-typedef int (*process_response_t)(void **ops,
- uint8_t *resp, void *op_cookie,
- enum qat_device_gen qat_dev_gen);
+typedef int (*process_response_t)(void **ops, uint8_t *resp);
/**< Process a response descriptor and return the associated op. */
/**
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 15244d113..f613adce6 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -661,9 +661,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
}
int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen)
+qat_sym_process_response(void **op, uint8_t *resp)
{
struct icp_qat_fw_comn_resp *resp_msg =
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index bdb672be4..dffd5f369 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -22,8 +22,6 @@ int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
int
-qat_sym_process_response(void **op, uint8_t *resp,
- __rte_unused void *op_cookie,
- __rte_unused enum qat_device_gen qat_dev_gen);
+qat_sym_process_response(void **op, uint8_t *resp);
#endif /* _QAT_SYM_H_ */
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 35/38] crypto/qat: make response process function inline
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (33 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 34/38] crypto/qat: remove unused arguments Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 36/38] crypto/qat: check for service type Tomasz Jozwiak
` (3 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Optimize the dequeue function by inlining the response
processing function, assuming only symmetric
operations are supported.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 4 +-
drivers/crypto/qat/qat_qp.h | 5 --
drivers/crypto/qat/qat_sym.c | 127 ------------------------------
drivers/crypto/qat/qat_sym.h | 131 ++++++++++++++++++++++++++++++-
drivers/crypto/qat/qat_sym_pmd.c | 1 -
5 files changed, 131 insertions(+), 137 deletions(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index 0fdec0da0..b190f2cee 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -14,6 +14,7 @@
#include "qat_logs.h"
#include "qat_device.h"
#include "qat_qp.h"
+#include "qat_sym.h"
#include "adf_transport_access_macros.h"
@@ -238,7 +239,6 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
qp->qat_dev_gen = qat_dev->qat_dev_gen;
qp->build_request = qat_qp_conf->build_request;
- qp->process_response = qat_qp_conf->process_response;
qp->qat_dev = qat_dev;
PMD_DRV_LOG(DEBUG, "QP setup complete: id: %d, cookiepool: %s",
@@ -612,7 +612,7 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
resp_counter != nb_ops) {
- tmp_qp->process_response(ops, resp_msg);
+ qat_sym_process_response(ops, resp_msg);
head = adf_modulo(head + rx_queue->msg_size,
rx_queue->modulo_mask);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index eb9188410..0b3d6d3aa 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -21,9 +21,6 @@ typedef int (*build_request_t)(void *op,
enum qat_device_gen qat_dev_gen);
/**< Build a request from an op. */
-typedef int (*process_response_t)(void **ops, uint8_t *resp);
-/**< Process a response descriptor and return the associated op. */
-
/**
* Structure with data needed for creation of queue pair.
*/
@@ -44,7 +41,6 @@ struct qat_qp_config {
uint32_t cookie_size;
int socket_id;
build_request_t build_request;
- process_response_t process_response;
const char *service_str;
};
@@ -83,7 +79,6 @@ struct qat_qp {
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
- process_response_t process_response;
struct qat_pci_device *qat_dev;
/**< qat device this qp is on */
} __rte_cache_aligned;
diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index f613adce6..887d4ebcd 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -12,44 +12,7 @@
#include <rte_byteorder.h>
#include "qat_logs.h"
-#include "qat_sym_session.h"
#include "qat_sym.h"
-#include "qat_sym_pmd.h"
-
-#define BYTE_LENGTH 8
-/* bpi is only used for partial blocks of DES and AES
- * so AES block len can be assumed as max len for iv, src and dst
- */
-#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
-
-/** Encrypt a single partial block
- * Depends on openssl libcrypto
- * Uses ECB+XOR to do CFB encryption, same result, more performant
- */
-static inline int
-bpi_cipher_encrypt(uint8_t *src, uint8_t *dst,
- uint8_t *iv, int ivlen, int srclen,
- void *bpi_ctx)
-{
- EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX *)bpi_ctx;
- int encrypted_ivlen;
- uint8_t encrypted_iv[BPI_MAX_ENCR_IV_LEN];
- uint8_t *encr = encrypted_iv;
-
- /* ECB method: encrypt the IV, then XOR this with plaintext */
- if (EVP_EncryptUpdate(ctx, encrypted_iv, &encrypted_ivlen, iv, ivlen)
- <= 0)
- goto cipher_encrypt_err;
-
- for (; srclen != 0; --srclen, ++dst, ++src, ++encr)
- *dst = *src ^ *encr;
-
- return 0;
-
-cipher_encrypt_err:
- PMD_DRV_LOG(ERR, "libcrypto ECB cipher encrypt failed");
- return -EINVAL;
-}
/** Decrypt a single partial block
* Depends on openssl libcrypto
@@ -136,62 +99,6 @@ qat_bpicipher_preprocess(struct qat_sym_session *ctx,
return sym_op->cipher.data.length - last_block_len;
}
-static inline uint32_t
-qat_bpicipher_postprocess(struct qat_sym_session *ctx,
- struct rte_crypto_op *op)
-{
- int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
- struct rte_crypto_sym_op *sym_op = op->sym;
- uint8_t last_block_len = block_len > 0 ?
- sym_op->cipher.data.length % block_len : 0;
-
- if (last_block_len > 0 &&
- ctx->qat_dir == ICP_QAT_HW_CIPHER_ENCRYPT) {
-
- /* Encrypt last block */
- uint8_t *last_block, *dst, *iv;
- uint32_t last_block_offset;
-
- last_block_offset = sym_op->cipher.data.offset +
- sym_op->cipher.data.length - last_block_len;
- last_block = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_src,
- uint8_t *, last_block_offset);
-
- if (unlikely(sym_op->m_dst != NULL))
- /* out-of-place operation (OOP) */
- dst = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_dst,
- uint8_t *, last_block_offset);
- else
- dst = last_block;
-
- if (last_block_len < sym_op->cipher.data.length)
- /* use previous block ciphertext as IV */
- iv = dst - block_len;
- else
- /* runt block, i.e. less than one full block */
- iv = rte_crypto_op_ctod_offset(op, uint8_t *,
- ctx->cipher_iv.offset);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "BPI: src before post-process:", last_block,
- last_block_len);
- if (sym_op->m_dst != NULL)
- rte_hexdump(stdout, "BPI: dst before post-process:",
- dst, last_block_len);
-#endif
- bpi_cipher_encrypt(last_block, dst, iv, block_len,
- last_block_len, ctx->bpi_ctx);
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "BPI: src after post-process:", last_block,
- last_block_len);
- if (sym_op->m_dst != NULL)
- rte_hexdump(stdout, "BPI: dst after post-process:", dst,
- last_block_len);
-#endif
- }
- return sym_op->cipher.data.length - last_block_len;
-}
-
static inline void
set_cipher_iv(uint16_t iv_length, uint16_t iv_offset,
struct icp_qat_fw_la_cipher_req_params *cipher_param,
@@ -659,37 +566,3 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
#endif
return 0;
}
-
-int
-qat_sym_process_response(void **op, uint8_t *resp)
-{
-
- struct icp_qat_fw_comn_resp *resp_msg =
- (struct icp_qat_fw_comn_resp *)resp;
- struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
- (resp_msg->opaque_data);
-
-#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
- rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
- sizeof(struct icp_qat_fw_comn_resp));
-#endif
-
- if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
- ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
- resp_msg->comn_hdr.comn_status)) {
-
- rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
- } else {
- struct qat_sym_session *sess = (struct qat_sym_session *)
- get_session_private_data(
- rx_op->sym->session,
- cryptodev_qat_driver_id);
-
- if (sess->bpi_ctx)
- qat_bpicipher_postprocess(sess, rx_op);
- rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
- }
- *op = (void *)rx_op;
-
- return 0;
-}
diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h
index dffd5f369..ddd0f473f 100644
--- a/drivers/crypto/qat/qat_sym.h
+++ b/drivers/crypto/qat/qat_sym.h
@@ -6,8 +6,19 @@
#define _QAT_SYM_H_
#include <rte_cryptodev_pmd.h>
+#include <rte_hexdump.h>
+
+#include <openssl/evp.h>
#include "qat_common.h"
+#include "qat_sym_session.h"
+#include "qat_sym_pmd.h"
+
+#define BYTE_LENGTH 8
+/* bpi is only used for partial blocks of DES and AES
+ * so AES block len can be assumed as max len for iv, src and dst
+ */
+#define BPI_MAX_ENCR_IV_LEN ICP_QAT_HW_AES_BLK_SZ
struct qat_sym_session;
@@ -21,7 +32,123 @@ struct qat_sym_op_cookie {
int
qat_sym_build_request(void *in_op, uint8_t *out_msg,
void *op_cookie, enum qat_device_gen qat_dev_gen);
-int
-qat_sym_process_response(void **op, uint8_t *resp);
+
+/** Encrypt a single partial block
+ * Depends on openssl libcrypto
+ * Uses ECB+XOR to do CFB encryption, same result, more performant
+ */
+static inline int
+bpi_cipher_encrypt(uint8_t *src, uint8_t *dst,
+ uint8_t *iv, int ivlen, int srclen,
+ void *bpi_ctx)
+{
+ EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX *)bpi_ctx;
+ int encrypted_ivlen;
+ uint8_t encrypted_iv[BPI_MAX_ENCR_IV_LEN];
+ uint8_t *encr = encrypted_iv;
+
+ /* ECB method: encrypt the IV, then XOR this with plaintext */
+ if (EVP_EncryptUpdate(ctx, encrypted_iv, &encrypted_ivlen, iv, ivlen)
+ <= 0)
+ goto cipher_encrypt_err;
+
+ for (; srclen != 0; --srclen, ++dst, ++src, ++encr)
+ *dst = *src ^ *encr;
+
+ return 0;
+
+cipher_encrypt_err:
+ PMD_DRV_LOG(ERR, "libcrypto ECB cipher encrypt failed");
+ return -EINVAL;
+}
+
+static inline uint32_t
+qat_bpicipher_postprocess(struct qat_sym_session *ctx,
+ struct rte_crypto_op *op)
+{
+ int block_len = qat_cipher_get_block_size(ctx->qat_cipher_alg);
+ struct rte_crypto_sym_op *sym_op = op->sym;
+ uint8_t last_block_len = block_len > 0 ?
+ sym_op->cipher.data.length % block_len : 0;
+
+ if (last_block_len > 0 &&
+ ctx->qat_dir == ICP_QAT_HW_CIPHER_ENCRYPT) {
+
+ /* Encrypt last block */
+ uint8_t *last_block, *dst, *iv;
+ uint32_t last_block_offset;
+
+ last_block_offset = sym_op->cipher.data.offset +
+ sym_op->cipher.data.length - last_block_len;
+ last_block = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_src,
+ uint8_t *, last_block_offset);
+
+ if (unlikely(sym_op->m_dst != NULL))
+ /* out-of-place operation (OOP) */
+ dst = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_dst,
+ uint8_t *, last_block_offset);
+ else
+ dst = last_block;
+
+ if (last_block_len < sym_op->cipher.data.length)
+ /* use previous block ciphertext as IV */
+ iv = dst - block_len;
+ else
+ /* runt block, i.e. less than one full block */
+ iv = rte_crypto_op_ctod_offset(op, uint8_t *,
+ ctx->cipher_iv.offset);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "BPI: src before post-process:", last_block,
+ last_block_len);
+ if (sym_op->m_dst != NULL)
+ rte_hexdump(stdout, "BPI: dst before post-process:",
+ dst, last_block_len);
+#endif
+ bpi_cipher_encrypt(last_block, dst, iv, block_len,
+ last_block_len, ctx->bpi_ctx);
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "BPI: src after post-process:", last_block,
+ last_block_len);
+ if (sym_op->m_dst != NULL)
+ rte_hexdump(stdout, "BPI: dst after post-process:", dst,
+ last_block_len);
+#endif
+ }
+ return sym_op->cipher.data.length - last_block_len;
+}
+
+static inline void
+qat_sym_process_response(void **op, uint8_t *resp)
+{
+
+ struct icp_qat_fw_comn_resp *resp_msg =
+ (struct icp_qat_fw_comn_resp *)resp;
+ struct rte_crypto_op *rx_op = (struct rte_crypto_op *)(uintptr_t)
+ (resp_msg->opaque_data);
+
+#ifdef RTE_LIBRTE_PMD_QAT_DEBUG_RX
+ rte_hexdump(stdout, "qat_response:", (uint8_t *)resp_msg,
+ sizeof(struct icp_qat_fw_comn_resp));
+#endif
+
+ if (ICP_QAT_FW_COMN_STATUS_FLAG_OK !=
+ ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET(
+ resp_msg->comn_hdr.comn_status)) {
+
+ rx_op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ } else {
+ struct qat_sym_session *sess = (struct qat_sym_session *)
+ get_session_private_data(
+ rx_op->sym->session,
+ cryptodev_qat_driver_id);
+
+
+ if (sess->bpi_ctx)
+ qat_bpicipher_postprocess(sess, rx_op);
+ rx_op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+ }
+ *op = (void *)rx_op;
+}
#endif /* _QAT_SYM_H_ */
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index 28e579b77..6b39b32f8 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -160,7 +160,6 @@ static int qat_sym_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
qat_qp_conf.hw = qp_hw_data;
qat_qp_conf.build_request = qat_sym_build_request;
- qat_qp_conf.process_response = qat_sym_process_response;
qat_qp_conf.cookie_size = sizeof(struct qat_sym_op_cookie);
qat_qp_conf.nb_descriptors = qp_conf->nb_descriptors;
qat_qp_conf.socket_id = socket_id;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 36/38] crypto/qat: check for service type
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (34 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 35/38] crypto/qat: make response process function inline Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 37/38] doc/qat: specify QAT driver and device name formats Tomasz Jozwiak
` (2 subsequent siblings)
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Other services, apart from symmetric crypto,
such as compression, will be added in future patches.
Therefore, the assumption that only symmetric crypto
operations are processed will not be valid anymore,
and service type needs to be checked.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
drivers/crypto/qat/qat_qp.c | 6 +++++-
drivers/crypto/qat/qat_qp.h | 1 +
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/crypto/qat/qat_qp.c b/drivers/crypto/qat/qat_qp.c
index b190f2cee..f9d3762d7 100644
--- a/drivers/crypto/qat/qat_qp.c
+++ b/drivers/crypto/qat/qat_qp.c
@@ -239,6 +239,7 @@ int qat_qp_setup(struct qat_pci_device *qat_dev,
qp->qat_dev_gen = qat_dev->qat_dev_gen;
qp->build_request = qat_qp_conf->build_request;
+ qp->service_type = qat_qp_conf->hw->service_type;
qp->qat_dev = qat_dev;
PMD_DRV_LOG(DEBUG, "QP setup complete: id: %d, cookiepool: %s",
@@ -612,7 +613,10 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG &&
resp_counter != nb_ops) {
- qat_sym_process_response(ops, resp_msg);
+ if (tmp_qp->service_type == QAT_SERVICE_SYMMETRIC)
+ qat_sym_process_response(ops, resp_msg);
+ /* add qat_asym_process_response here */
+ /* add qat_comp_process_response here */
head = adf_modulo(head + rx_queue->msg_size,
rx_queue->modulo_mask);
diff --git a/drivers/crypto/qat/qat_qp.h b/drivers/crypto/qat/qat_qp.h
index 0b3d6d3aa..59db945e7 100644
--- a/drivers/crypto/qat/qat_qp.h
+++ b/drivers/crypto/qat/qat_qp.h
@@ -79,6 +79,7 @@ struct qat_qp {
uint32_t nb_descriptors;
enum qat_device_gen qat_dev_gen;
build_request_t build_request;
+ enum qat_service_type service_type;
struct qat_pci_device *qat_dev;
/**< qat device this qp is on */
} __rte_cache_aligned;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 37/38] doc/qat: specify QAT driver and device name formats
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (35 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 36/38] crypto/qat: check for service type Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 38/38] crypto/qat: remove configurable max number of sessions Tomasz Jozwiak
2018-06-14 10:59 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services De Lara Guarch, Pablo
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
From: Fiona Trahe <fiona.trahe@intel.com>
Document the driver and device naming formats.
Changed the underscores alignment.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
---
doc/guides/cryptodevs/qat.rst | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst
index 8c8fefaa0..22a2c715d 100644
--- a/doc/guides/cryptodevs/qat.rst
+++ b/doc/guides/cryptodevs/qat.rst
@@ -359,3 +359,15 @@ length of data to authenticate (op.sym.auth.data.length) must be the length
of all the items described above, including the padding at the end.
Also, offset of data to authenticate (op.sym.auth.data.offset)
must be such that points at the start of the COUNT bytes.
+
+Device and driver naming
+------------------------
+
+The qat crypto driver name is "crypto_qat".
+This name is passed to the dpdk-test-crypto-perf tool in the -devtype parameter.
+The rte_cryptodev_devices_get() can return the devices exposed by a driver.
+
+Each qat crypto device has a unique name, in format
+<pci bdf>_<service>, e.g. "0000:41:01.0_qat_sym".
+This name can be passed to rte_cryptodev_get_dev_id() to get the device_id.
+This is also the format of the slave parameter passed to the crypto scheduler.
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH v3 38/38] crypto/qat: remove configurable max number of sessions
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (36 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 37/38] doc/qat: specify QAT driver and device name formats Tomasz Jozwiak
@ 2018-06-13 12:14 ` Tomasz Jozwiak
2018-06-14 10:59 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services De Lara Guarch, Pablo
38 siblings, 0 replies; 109+ messages in thread
From: Tomasz Jozwiak @ 2018-06-13 12:14 UTC (permalink / raw)
To: fiona.trahe, tomaszx.jozwiak, dev
This patch removes CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS
from common_base, config/rte_config.h files and
defines QAT_SYM_PMD_MAX_NB_SESSIONS inside qat_sym_pmd.h
file instead.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
---
config/common_base | 5 -----
config/rte_config.h | 5 -----
drivers/crypto/qat/qat_sym_pmd.c | 4 ++--
drivers/crypto/qat/qat_sym_pmd.h | 1 +
4 files changed, 3 insertions(+), 12 deletions(-)
diff --git a/config/common_base b/config/common_base
index cf0741199..f03f9c390 100644
--- a/config/common_base
+++ b/config/common_base
@@ -491,11 +491,6 @@ CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n
#
-# Number of sessions to create in the session memory pool
-# on a single QuickAssist device.
-#
-CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=2048
-#
# Max. number of QuickAssist devices, which can be detected and attached
#
CONFIG_RTE_PMD_QAT_MAX_PCI_DEVICES=48
diff --git a/config/rte_config.h b/config/rte_config.h
index e353d8d6d..0ba0ead7e 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -85,12 +85,7 @@
/****** driver defines ********/
-/*
- * Number of sessions to create in the session memory pool
- * on a single instance of crypto HW device.
- */
/* QuickAssist device */
-#define RTE_QAT_PMD_MAX_NB_SESSIONS 2048
/* Max. number of QuickAssist devices which can be attached */
#define RTE_PMD_QAT_MAX_PCI_DEVICES 48
diff --git a/drivers/crypto/qat/qat_sym_pmd.c b/drivers/crypto/qat/qat_sym_pmd.c
index 6b39b32f8..c94a8a197 100644
--- a/drivers/crypto/qat/qat_sym_pmd.c
+++ b/drivers/crypto/qat/qat_sym_pmd.c
@@ -77,7 +77,7 @@ static void qat_sym_dev_info_get(struct rte_cryptodev *dev,
qat_qps_per_service(sym_hw_qps, QAT_SERVICE_SYMMETRIC);
info->feature_flags = dev->feature_flags;
info->capabilities = internals->qat_dev_capabilities;
- info->sym.max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS;
+ info->sym.max_nb_sessions = QAT_SYM_PMD_MAX_NB_SESSIONS;
info->driver_id = cryptodev_qat_driver_id;
info->pci_dev = RTE_DEV_TO_PCI(dev->device);
}
@@ -250,7 +250,7 @@ qat_sym_dev_create(struct qat_pci_device *qat_pci_dev)
.name = "",
.socket_id = qat_pci_dev->pci_dev->device.numa_node,
.private_data_size = sizeof(struct qat_sym_dev_private),
- .max_nb_sessions = RTE_QAT_PMD_MAX_NB_SESSIONS
+ .max_nb_sessions = QAT_SYM_PMD_MAX_NB_SESSIONS
};
char name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct rte_cryptodev *cryptodev;
diff --git a/drivers/crypto/qat/qat_sym_pmd.h b/drivers/crypto/qat/qat_sym_pmd.h
index efa3b0775..80a198741 100644
--- a/drivers/crypto/qat/qat_sym_pmd.h
+++ b/drivers/crypto/qat/qat_sym_pmd.h
@@ -13,6 +13,7 @@
/**< Intel(R) QAT Symmetric Crypto PMD device name */
#define CRYPTODEV_NAME_QAT_SYM_PMD crypto_qat
+#define QAT_SYM_PMD_MAX_NB_SESSIONS 2048
extern uint8_t cryptodev_qat_driver_id;
--
2.17.0
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [PATCH v3 00/38] crypto/qat: refactor to support multiple services
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
` (37 preceding siblings ...)
2018-06-13 12:14 ` [PATCH v3 38/38] crypto/qat: remove configurable max number of sessions Tomasz Jozwiak
@ 2018-06-14 10:59 ` De Lara Guarch, Pablo
38 siblings, 0 replies; 109+ messages in thread
From: De Lara Guarch, Pablo @ 2018-06-14 10:59 UTC (permalink / raw)
To: Jozwiak, TomaszX, Trahe, Fiona, Jozwiak, TomaszX, dev
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tomasz Jozwiak
> Sent: Wednesday, June 13, 2018 1:14 PM
> To: Trahe, Fiona <fiona.trahe@intel.com>; Jozwiak, TomaszX
> <tomaszx.jozwiak@intel.com>; dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v3 00/38] crypto/qat: refactor to support multiple
> services
>
> From: Pablo de Lara <pablo.de.lara.guarch@intel.com>
>
> The QAT crypto PMD was peppered with references to rte_cryptodev artefacts.
> Also the pci device it presented to the API layer was entirely owned by the one
> cryptodev device instance created by the probe.
> This patchset refactors the PMD so one pci device can present out multiple
> different device instances to implement different APIs, specifically cryptodev
> and compressdev instances.
> The code is refactored into
> - common code which handles the pci device discovery, configuration,
> queue-pair setup, stats gathering and data-path enqueue and dequeue.
> - service-specific code for symmetric crypto, which is a thin layer
> implementing the cryptodev API and using the common code.
> - place-holder files for service-specific code to support asymmetric
> crypto and compression services in future.
> The code is all still in the crypto/qat directory as it is functionally unchanged, still
> only supporting a symmetric crypto service. The code will move to
> drivers/common, /crypto and /compress in a future patch when this is needed
> for a compression PMD.
>
> Changes for v3:
> - fixed some commit messages
> - fixed compilation issue in one of the patches
> - split two commits into multiple commits
>
> Changes for v2:
> - fixed some bugs compiled in when debug enabled
> - added patch to remove MAX_NB_SESSIONS from config file
> - removed SPDX license patch as already applied
> - documented device name formats
>
Applied to dpdk-next-crypto.
Thanks,
Pablo
^ permalink raw reply [flat|nested] 109+ messages in thread
end of thread, other threads:[~2018-06-14 11:00 UTC | newest]
Thread overview: 109+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-06 18:51 [PATCH 00/30] crypto/qat: refactor to support multiple service Fiona Trahe
2018-04-06 18:51 ` [PATCH 01/30] crypto/qat: use SPDX license Fiona Trahe
2018-04-18 8:03 ` De Lara Guarch, Pablo
2018-04-06 18:51 ` [PATCH 02/30] crypto/qat: add qat common header Fiona Trahe
2018-04-06 18:51 ` [PATCH 03/30] crypto/qat: add qat device files Fiona Trahe
2018-04-06 18:51 ` [PATCH 04/30] crypto/qat: remove unused includes Fiona Trahe
2018-04-06 18:51 ` [PATCH 05/30] crypto/qat: add symmetric session file Fiona Trahe
2018-04-06 18:51 ` [PATCH 06/30] crypto/qat: change filename crypto to sym Fiona Trahe
2018-04-06 18:51 ` [PATCH 07/30] crypto/qat: rename fns for consistency Fiona Trahe
2018-04-06 18:51 ` [PATCH 08/30] crypto/qat: renamed sym-specific structs Fiona Trahe
2018-04-06 18:51 ` [PATCH 09/30] crypto/qat: make enqueue function generic Fiona Trahe
2018-04-06 18:51 ` [PATCH 10/30] crypto/qat: make dequeue " Fiona Trahe
2018-04-06 18:51 ` [PATCH 11/30] crypto/qat: move generic qp fn to qp file Fiona Trahe
2018-04-06 18:51 ` [PATCH 12/30] crypto/qat: separate sym-specific from generic qp setup Fiona Trahe
2018-04-06 18:51 ` [PATCH 13/30] crypto/qat: move sym-specific qp code to sym file Fiona Trahe
2018-04-06 18:51 ` [PATCH 14/30] crypto/qat: remove dependencies on cryptodev from common Fiona Trahe
2018-04-06 18:51 ` [PATCH 15/30] crypto/qat: move defines from sym to qp header file Fiona Trahe
2018-04-06 18:51 ` [PATCH 16/30] crypto/qat: create data structures to support different generations Fiona Trahe
2018-04-06 18:51 ` [PATCH 17/30] crypto/qat: rename sgl related objects Fiona Trahe
2018-04-06 18:52 ` [PATCH 18/30] crypto/qat: move sgl related element to appropriate files Fiona Trahe
2018-04-06 18:52 ` [PATCH 19/30] crypto/qat: add QAT PCI device struct Fiona Trahe
2018-04-06 18:52 ` [PATCH 20/30] crypto/qat: separate the name used for PCI reg from crypto name Fiona Trahe
2018-04-06 18:52 ` [PATCH 21/30] crypto/qat: move to using new device structure Fiona Trahe
2018-04-06 18:52 ` [PATCH 22/30] crypto/qat: use common stats structures Fiona Trahe
2018-04-06 18:52 ` [PATCH 23/30] crypto/qat: rename functions which depend on cryptodev Fiona Trahe
2018-04-06 18:52 ` [PATCH 24/30] crypto/qat: move code into appropriate files Fiona Trahe
2018-04-06 18:52 ` [PATCH 25/30] crypto/qat: add lock around csr access and change logic Fiona Trahe
2018-04-06 18:52 ` [PATCH 26/30] crypto/qat: remove incorrect usage of bundle number Fiona Trahe
2018-04-06 18:52 ` [PATCH 27/30] crypto/qat: cleanups Fiona Trahe
2018-04-06 18:52 ` [PATCH 28/30] crypto/qat: create appropriately named device for registration Fiona Trahe
2018-04-06 18:52 ` [PATCH 29/30] crypto/qat: add MAX PCI DEVICES flag to config file Fiona Trahe
2018-04-06 18:52 ` [PATCH 30/30] crypto/qat: add performance improvement into qat crypto dev Fiona Trahe
2018-04-11 12:41 ` [PATCH 00/30] crypto/qat: refactor to support multiple service De Lara Guarch, Pablo
2018-05-11 11:13 ` [PATCH v2 00/31] crypto/qat: refactor to support multiple Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 01/31] crypto/qat: add qat common header Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 02/31] crypto/qat: add qat device files Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 03/31] crypto/qat: remove unused includes Fiona Trahe
2018-06-11 22:20 ` De Lara Guarch, Pablo
2018-05-11 11:13 ` [PATCH v2 04/31] crypto/qat: add symmetric session file Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 05/31] crypto/qat: change filename crypto to sym Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 06/31] crypto/qat: rename fns for consistency Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 07/31] crypto/qat: renamed sym-specific structs Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 08/31] crypto/qat: make enqueue function generic Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 09/31] crypto/qat: make dequeue " Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 10/31] crypto/qat: move generic qp fn to qp file Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 11/31] crypto/qat: separate sym-specific from generic qp setup Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 12/31] crypto/qat: move sym-specific qp code to sym file Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 13/31] crypto/qat: remove dependencies on cryptodev from common Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 14/31] crypto/qat: move defines from sym to qp header file Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 15/31] crypto/qat: create data structures to support different generations Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 16/31] crypto/qat: rename sgl related objects Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 17/31] crypto/qat: move sgl related element to appropriate files Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 18/31] crypto/qat: add QAT PCI device struct Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 19/31] crypto/qat: separate the name used for PCI reg from crypto name Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 20/31] crypto/qat: move to using new device structure Fiona Trahe
2018-05-11 11:13 ` [PATCH v2 21/31] crypto/qat: use common stats structures Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 22/31] crypto/qat: rename functions which depend on cryptodev Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 23/31] crypto/qat: move code into appropriate files Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 24/31] crypto/qat: add lock around csr access and change logic Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 25/31] crypto/qat: remove incorrect usage of bundle number Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 26/31] crypto/qat: cleanups Fiona Trahe
2018-06-11 22:21 ` De Lara Guarch, Pablo
2018-05-11 11:14 ` [PATCH v2 27/31] crypto/qat: create appropriately named device for registration Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 28/31] crypto/qat: add MAX PCI DEVICES flag to config file Fiona Trahe
2018-05-11 11:14 ` [PATCH v2 29/31] crypto/qat: add performance improvement into qat crypto dev Fiona Trahe
2018-06-11 22:23 ` De Lara Guarch, Pablo
2018-05-11 11:14 ` [PATCH v2 30/31] doc/qat: specify QAT driver and device name formats Fiona Trahe
2018-05-14 15:38 ` Kovacevic, Marko
2018-05-11 11:14 ` [PATCH v2 31/31] crypto/qat: remove CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS Fiona Trahe
2018-06-13 12:13 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 01/38] crypto/qat: add qat common header Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 02/38] crypto/qat: add qat device files Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 03/38] crypto/qat: remove unused includes Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 04/38] crypto/qat: add symmetric session file Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 05/38] crypto/qat: change filename crypto to sym Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 06/38] crypto/qat: rename fns for consistency Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 07/38] crypto/qat: renamed sym-specific structs Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 08/38] crypto/qat: make enqueue function generic Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 09/38] crypto/qat: make dequeue " Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 10/38] crypto/qat: move generic qp fn to qp file Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 11/38] crypto/qat: separate sym-specific from generic qp setup Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 12/38] crypto/qat: move sym-specific qp code to sym file Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 13/38] crypto/qat: remove dependencies on cryptodev from common Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 14/38] crypto/qat: move defines from sym to qp header file Tomasz Jozwiak
2018-06-13 12:13 ` [PATCH v3 15/38] crypto/qat: create structures to support various generations Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 16/38] crypto/qat: rename sgl related objects Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 17/38] crypto/qat: move sgl related element to appropriate files Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 18/38] crypto/qat: add QAT PCI device struct Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 19/38] crypto/qat: use generic driver name for PCI registration Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 20/38] crypto/qat: move to using new device structure Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 21/38] crypto/qat: use common stats structures Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 22/38] crypto/qat: rename functions which depend on cryptodev Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 23/38] crypto/qat: move code into appropriate files Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 24/38] crypto/qat: add lock around csr access and change logic Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 25/38] crypto/qat: remove incorrect usage of bundle number Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 26/38] crypto/qat: rename variables Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 27/38] crypto/qat: modify debug message Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 28/38] crypto/qat: free cookie pool on queue creation error Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 29/38] crypto/qat: remove unused macro Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 30/38] crypto/qat: move macro to common file Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 31/38] crypto/qat: register appropriately named device Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 32/38] crypto/qat: add max PCI devices to config file Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 33/38] crypto/qat: optimize adf modulo function Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 34/38] crypto/qat: remove unused arguments Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 35/38] crypto/qat: make response process function inline Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 36/38] crypto/qat: check for service type Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 37/38] doc/qat: specify QAT driver and device name formats Tomasz Jozwiak
2018-06-13 12:14 ` [PATCH v3 38/38] crypto/qat: remove configurable max number of sessions Tomasz Jozwiak
2018-06-14 10:59 ` [PATCH v3 00/38] crypto/qat: refactor to support multiple services De Lara Guarch, Pablo
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.