From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 226DEC433EF for ; Mon, 18 Oct 2021 14:42:47 +0000 (UTC) Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by mail.kernel.org (Postfix) with ESMTP id B87FB60FC3 for ; Mon, 18 Oct 2021 14:42:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B87FB60FC3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dpdk.org Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1FE6540DF5; Mon, 18 Oct 2021 16:42:46 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 2098C40141 for ; Mon, 18 Oct 2021 16:42:44 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19ICWLLU009556; Mon, 18 Oct 2021 07:42:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=FOAg/k8hvEZNTvV2Y013Lym8nHdLMZgIFoncsBxlFDU=; b=I0syAvQm2DXVikZeZdXj0CDAd2pgXTKhoFFUC53O+7IimJIEx+eIWJV0pVVR+RsAnS40 KwLxNDy2B+EJYHuqr6x1ARoHJv8XdnYgJTtScco6bfaH1EOfQIRxXeTmPekuEdPhpJNd +7GlaxCv2eUYXxyaqgGyFCKOXvi083yzd7vGXbAAwf6VcpYE1KDRs9JRa9U2UfyoG3PE GkzZxkRW+kPlQIektAhPYh92L3w8NOlaT/3vZlO0Te2yIKkI6P+rwft0bl+0A/rs5L0c 6x6FQ6z/c+S8zxlPueuuogkyJ7AzfuGRE0ZRMXMGTRiYeRvt1l7HvKRgQYO9zT0PIed9 IA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 3brt1s34qt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 18 Oct 2021 07:42:38 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 18 Oct 2021 07:42:36 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Mon, 18 Oct 2021 07:42:36 -0700 Received: from localhost.localdomain (unknown [10.28.36.185]) by maili.marvell.com (Postfix) with ESMTP id DCCA53F7041; Mon, 18 Oct 2021 07:42:28 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , , , , , , , , , , , , , , Akhil Goyal , Rebecca Troy Date: Mon, 18 Oct 2021 20:11:57 +0530 Message-ID: <20211018144201.2028022-4-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211018144201.2028022-1-gakhil@marvell.com> References: <20211011124309.4066491-1-gakhil@marvell.com> <20211018144201.2028022-1-gakhil@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: 53L_Ht3y2WDn3et3l47SCdHUhq5QrhkW X-Proofpoint-GUID: 53L_Ht3y2WDn3et3l47SCdHUhq5QrhkW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-18_06,2021-10-18_01,2020-04-07_01 Subject: [dpdk-dev] [PATCH v3 3/7] cryptodev: move inline APIs into separate structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Move fastpath inline function pointers from rte_cryptodev into a separate structure accessed via a flat array. The intension is to make rte_cryptodev and related structures private to avoid future API/ABI breakages. Signed-off-by: Akhil Goyal Tested-by: Rebecca Troy Acked-by: Fan Zhang --- lib/cryptodev/cryptodev_pmd.c | 53 +++++++++++++++++++++++++++++- lib/cryptodev/cryptodev_pmd.h | 11 +++++++ lib/cryptodev/rte_cryptodev.c | 19 +++++++++++ lib/cryptodev/rte_cryptodev_core.h | 29 ++++++++++++++++ lib/cryptodev/version.map | 5 +++ 5 files changed, 116 insertions(+), 1 deletion(-) diff --git a/lib/cryptodev/cryptodev_pmd.c b/lib/cryptodev/cryptodev_pmd.c index 44a70ecb35..fd74543682 100644 --- a/lib/cryptodev/cryptodev_pmd.c +++ b/lib/cryptodev/cryptodev_pmd.c @@ -3,7 +3,7 @@ */ #include - +#include #include #include @@ -160,3 +160,54 @@ rte_cryptodev_pmd_destroy(struct rte_cryptodev *cryptodev) return 0; } + +static uint16_t +dummy_crypto_enqueue_burst(__rte_unused void *qp, + __rte_unused struct rte_crypto_op **ops, + __rte_unused uint16_t nb_ops) +{ + CDEV_LOG_ERR( + "crypto enqueue burst requested for unconfigured device"); + rte_errno = ENOTSUP; + return 0; +} + +static uint16_t +dummy_crypto_dequeue_burst(__rte_unused void *qp, + __rte_unused struct rte_crypto_op **ops, + __rte_unused uint16_t nb_ops) +{ + CDEV_LOG_ERR( + "crypto dequeue burst requested for unconfigured device"); + rte_errno = ENOTSUP; + return 0; +} + +void +cryptodev_fp_ops_reset(struct rte_crypto_fp_ops *fp_ops) +{ + static struct rte_cryptodev_cb_rcu dummy_cb[RTE_MAX_QUEUES_PER_PORT]; + static void *dummy_data[RTE_MAX_QUEUES_PER_PORT]; + static const struct rte_crypto_fp_ops dummy = { + .enqueue_burst = dummy_crypto_enqueue_burst, + .dequeue_burst = dummy_crypto_dequeue_burst, + .qp = { + .data = dummy_data, + .enq_cb = dummy_cb, + .deq_cb = dummy_cb, + }, + }; + + *fp_ops = dummy; +} + +void +cryptodev_fp_ops_set(struct rte_crypto_fp_ops *fp_ops, + const struct rte_cryptodev *dev) +{ + fp_ops->enqueue_burst = dev->enqueue_burst; + fp_ops->dequeue_burst = dev->dequeue_burst; + fp_ops->qp.data = dev->data->queue_pairs; + fp_ops->qp.enq_cb = dev->enq_cbs; + fp_ops->qp.deq_cb = dev->deq_cbs; +} diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index 36606dd10b..a71edbb991 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -516,6 +516,17 @@ RTE_INIT(init_ ##driver_id)\ driver_id = rte_cryptodev_allocate_driver(&crypto_drv, &(drv));\ } +/* Reset crypto device fastpath APIs to dummy values. */ +__rte_internal +void +cryptodev_fp_ops_reset(struct rte_crypto_fp_ops *fp_ops); + +/* Setup crypto device fastpath APIs. */ +__rte_internal +void +cryptodev_fp_ops_set(struct rte_crypto_fp_ops *fp_ops, + const struct rte_cryptodev *dev); + static inline void * get_sym_session_private_data(const struct rte_cryptodev_sym_session *sess, uint8_t driver_id) { diff --git a/lib/cryptodev/rte_cryptodev.c b/lib/cryptodev/rte_cryptodev.c index eb86e629aa..305e013ebb 100644 --- a/lib/cryptodev/rte_cryptodev.c +++ b/lib/cryptodev/rte_cryptodev.c @@ -53,6 +53,9 @@ static struct rte_cryptodev_global cryptodev_globals = { .nb_devs = 0 }; +/* Public fastpath APIs. */ +struct rte_crypto_fp_ops rte_crypto_fp_ops[RTE_CRYPTO_MAX_DEVS]; + /* spinlock for crypto device callbacks */ static rte_spinlock_t rte_cryptodev_cb_lock = RTE_SPINLOCK_INITIALIZER; @@ -917,6 +920,8 @@ rte_cryptodev_pmd_release_device(struct rte_cryptodev *cryptodev) dev_id = cryptodev->data->dev_id; + cryptodev_fp_ops_reset(rte_crypto_fp_ops + dev_id); + /* Close device only if device operations have been set */ if (cryptodev->dev_ops) { ret = rte_cryptodev_close(dev_id); @@ -1080,6 +1085,9 @@ rte_cryptodev_start(uint8_t dev_id) } diag = (*dev->dev_ops->dev_start)(dev); + /* expose selection of PMD fast-path functions */ + cryptodev_fp_ops_set(rte_crypto_fp_ops + dev_id, dev); + rte_cryptodev_trace_start(dev_id, diag); if (diag == 0) dev->data->dev_started = 1; @@ -1109,6 +1117,9 @@ rte_cryptodev_stop(uint8_t dev_id) return; } + /* point fast-path functions to dummy ones */ + cryptodev_fp_ops_reset(rte_crypto_fp_ops + dev_id); + (*dev->dev_ops->dev_stop)(dev); rte_cryptodev_trace_stop(dev_id); dev->data->dev_started = 0; @@ -2411,3 +2422,11 @@ rte_cryptodev_allocate_driver(struct cryptodev_driver *crypto_drv, return nb_drivers++; } + +RTE_INIT(cryptodev_init_fp_ops) +{ + uint32_t i; + + for (i = 0; i != RTE_DIM(rte_crypto_fp_ops); i++) + cryptodev_fp_ops_reset(rte_crypto_fp_ops + i); +} diff --git a/lib/cryptodev/rte_cryptodev_core.h b/lib/cryptodev/rte_cryptodev_core.h index 1633e55889..e9e9a44b3c 100644 --- a/lib/cryptodev/rte_cryptodev_core.h +++ b/lib/cryptodev/rte_cryptodev_core.h @@ -25,6 +25,35 @@ typedef uint16_t (*enqueue_pkt_burst_t)(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops); /**< Enqueue packets for processing on queue pair of a device. */ +/** + * @internal + * Structure used to hold opaque pointers to internal ethdev Rx/Tx + * queues data. + * The main purpose to expose these pointers at all - allow compiler + * to fetch this data for fast-path cryptodev inline functions in advance. + */ +struct rte_cryptodev_qpdata { + /** points to array of internal queue pair data pointers. */ + void **data; + /** points to array of enqueue callback data pointers */ + struct rte_cryptodev_cb_rcu *enq_cb; + /** points to array of dequeue callback data pointers */ + struct rte_cryptodev_cb_rcu *deq_cb; +}; + +struct rte_crypto_fp_ops { + /** PMD enqueue burst function. */ + enqueue_pkt_burst_t enqueue_burst; + /** PMD dequeue burst function. */ + dequeue_pkt_burst_t dequeue_burst; + /** Internal queue pair data pointers. */ + struct rte_cryptodev_qpdata qp; + /** Reserved for future ops. */ + uintptr_t reserved[4]; +} __rte_cache_aligned; + +extern struct rte_crypto_fp_ops rte_crypto_fp_ops[RTE_CRYPTO_MAX_DEVS]; + /** * @internal * The data part, with no function pointers, associated with each device. diff --git a/lib/cryptodev/version.map b/lib/cryptodev/version.map index 43cf937e40..ed62ced221 100644 --- a/lib/cryptodev/version.map +++ b/lib/cryptodev/version.map @@ -45,6 +45,9 @@ DPDK_22 { rte_cryptodev_sym_session_init; rte_cryptodevs; + #added in 21.11 + rte_crypto_fp_ops; + local: *; }; @@ -109,6 +112,8 @@ EXPERIMENTAL { INTERNAL { global: + cryptodev_fp_ops_reset; + cryptodev_fp_ops_set; rte_cryptodev_allocate_driver; rte_cryptodev_pmd_allocate; rte_cryptodev_pmd_callback_process; -- 2.25.1