[PATCH 00/13] crypto: inside-secure - various improvements

[PATCH 00/13] crypto: inside-secure - various improvements

[Antoine Tenart]

Hi Herbert,

This series improves the newly added inside-secure driver in various
ways. There are also a few non-critical fixes.

The series is based on top of your cryptodev/master branch, as it
depends on the inside-secure driver addition ("crypto: inside-secure -
add SafeXcel EIP197 crypto engine driver").

I'd like to thank Ofer Heifetz and Igal Liberman who helped making
these changes!

Thanks,
Antoine

Antoine Tenart (10):
  crypto: inside-secure - use hmac ipad/opad constants
  crypto: inside-secure - fix the ring wr_cache offset
  crypto: inside-secure - fix incorrect DSE data cache setting
  crypto: inside-secure - update the context and request later
  crypto: inside-secure - use one queue per hw ring
  crypto: inside-secure - stop requeueing failed requests
  crypto: inside-secure - get the backlog before dequeueing the request
  crypto: inside-secure - only dequeue when needed
  crypto: inside-secure - increase the batch size
  crypto: inside-secure - use the base_end pointer in ring rollback

Igal Liberman (2):
  crypto: inside-secure - enable single WR in DSE configuration
  crypto: inside-secure - optimize DSE bufferability control

Ofer Heifetz (1):
  crypto: inside-secure - align the cipher and hash send functions

 drivers/crypto/inside-secure/safexcel.c        | 104 ++++++++++++-------------
 drivers/crypto/inside-secure/safexcel.h        |  18 +++--
 drivers/crypto/inside-secure/safexcel_cipher.c |  47 ++++++-----
 drivers/crypto/inside-secure/safexcel_hash.c   |  51 ++++++------
 drivers/crypto/inside-secure/safexcel_ring.c   |   2 +-
 5 files changed, 116 insertions(+), 106 deletions(-)

-- 
2.9.4

[PATCH 01/13] crypto: inside-secure - use hmac ipad/opad constants

[Antoine Tenart]

Replace the hmac ipad/opad values by their defined constants.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel_hash.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/inside-secure/safexcel_hash.c b/drivers/crypto/inside-secure/safexcel_hash.c
index 060ea034c9da..6eee1a502225 100644
--- a/drivers/crypto/inside-secure/safexcel_hash.c
+++ b/drivers/crypto/inside-secure/safexcel_hash.c
@@ -8,6 +8,7 @@
  * warranty of any kind, whether express or implied.
  */
 
+#include <crypto/hmac.h>
 #include <crypto/sha.h>
 #include <linux/device.h>
 #include <linux/dma-mapping.h>
@@ -774,8 +775,8 @@ static int safexcel_hmac_init_pad(struct ahash_request *areq,
 	memcpy(opad, ipad, blocksize);
 
 	for (i = 0; i < blocksize; i++) {
-		ipad[i] ^= 0x36;
-		opad[i] ^= 0x5c;
+		ipad[i] ^= HMAC_IPAD_VALUE;
+		opad[i] ^= HMAC_OPAD_VALUE;
 	}
 
 	return 0;
-- 
2.9.4

[PATCH 02/13] crypto: inside-secure - fix the ring wr_cache offset

[Antoine Tenart]

The EIP197_HIA_xDR_CFG_WR_CACHE macro was defined to use an offset of
23, which is wrong as it's actually 25. Fix this.

Reported-by: Igal Liberman <igall@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h
index c17fdd40b99f..0328a9314b90 100644
--- a/drivers/crypto/inside-secure/safexcel.h
+++ b/drivers/crypto/inside-secure/safexcel.h
@@ -99,7 +99,7 @@
 #define EIP197_HIA_xDR_WR_RES_BUF		BIT(22)
 #define EIP197_HIA_xDR_WR_CTRL_BUG		BIT(23)
 #define EIP197_HIA_xDR_WR_OWN_BUF		BIT(24)
-#define EIP197_HIA_xDR_CFG_WR_CACHE(n)		(((n) & 0x7) << 23)
+#define EIP197_HIA_xDR_CFG_WR_CACHE(n)		(((n) & 0x7) << 25)
 #define EIP197_HIA_xDR_CFG_RD_CACHE(n)		(((n) & 0x7) << 29)
 
 /* EIP197_HIA_CDR_THRESH */
-- 
2.9.4

[PATCH 03/13] crypto: inside-secure - fix incorrect DSE data cache setting

[Antoine Tenart]

Set the correct value to the DSE data cache, using WR_CACHE_3BITS
instead of RD_CACHE_3BITS. This fixes an incorrect setting and helps
improving performances.

Reported-by: Igal Liberman <igall@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c
index 5485e925e18d..99755fc1a161 100644
--- a/drivers/crypto/inside-secure/safexcel.c
+++ b/drivers/crypto/inside-secure/safexcel.c
@@ -328,7 +328,7 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv)
 	/* DMA transfer size to use */
 	val = EIP197_HIA_DSE_CFG_DIS_DEBUG;
 	val |= EIP197_HIA_DxE_CFG_MIN_DATA_SIZE(7) | EIP197_HIA_DxE_CFG_MAX_DATA_SIZE(8);
-	val |= EIP197_HIA_DxE_CFG_DATA_CACHE_CTRL(RD_CACHE_3BITS);
+	val |= EIP197_HIA_DxE_CFG_DATA_CACHE_CTRL(WR_CACHE_3BITS);
 	writel(val, priv->base + EIP197_HIA_DSE_CFG);
 
 	/* Leave the DSE threads reset state */
-- 
2.9.4

[PATCH 04/13] crypto: inside-secure - enable single WR in DSE configuration

[Antoine Tenart]

From: Igal Liberman <igall@marvell.com>

When enable_single_wr is not enabled, the DSE will only write those
parts of a result descriptor that need updating, which means a final
result descriptor will be written in 2 or 3 smaller transfers.
When enable_single_wr is enabled the DSE will combine these 2-3
updates into one large write transfer, generally improving performance.

Signed-off-by: Igal Liberman <igall@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.c | 1 +
 drivers/crypto/inside-secure/safexcel.h | 1 +
 2 files changed, 2 insertions(+)

diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c
index 99755fc1a161..658b307c6a11 100644
--- a/drivers/crypto/inside-secure/safexcel.c
+++ b/drivers/crypto/inside-secure/safexcel.c
@@ -329,6 +329,7 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv)
 	val = EIP197_HIA_DSE_CFG_DIS_DEBUG;
 	val |= EIP197_HIA_DxE_CFG_MIN_DATA_SIZE(7) | EIP197_HIA_DxE_CFG_MAX_DATA_SIZE(8);
 	val |= EIP197_HIA_DxE_CFG_DATA_CACHE_CTRL(WR_CACHE_3BITS);
+	val |= EIP197_HIA_DSE_CFG_EN_SINGLE_WR;
 	writel(val, priv->base + EIP197_HIA_DSE_CFG);
 
 	/* Leave the DSE threads reset state */
diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h
index 0328a9314b90..b8a81c568c99 100644
--- a/drivers/crypto/inside-secure/safexcel.h
+++ b/drivers/crypto/inside-secure/safexcel.h
@@ -143,6 +143,7 @@
 #define EIP197_HIA_DxE_CFG_CTRL_CACHE_CTRL(n)	(((n) & 0x7) << 20)
 #define EIP197_HIA_DxE_CFG_MAX_CTRL_SIZE(n)	((n) << 24)
 #define EIP197_HIA_DFE_CFG_DIS_DEBUG		(BIT(31) | BIT(29))
+#define EIP197_HIA_DSE_CFG_EN_SINGLE_WR		BIT(29)
 #define EIP197_HIA_DSE_CFG_DIS_DEBUG		BIT(31)
 
 /* EIP197_HIA_DFE/DSE_THR_CTRL */
-- 
2.9.4

[PATCH 05/13] crypto: inside-secure - optimize DSE bufferability control

[Antoine Tenart]

From: Igal Liberman <igall@marvell.com>

Configure the data write bufferability to always buffer packets in the
DSE. This change slightly improves performance.

Signed-off-by: Igal Liberman <igall@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.c | 1 +
 drivers/crypto/inside-secure/safexcel.h | 1 +
 2 files changed, 2 insertions(+)

diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c
index 658b307c6a11..73f4ef8d71f3 100644
--- a/drivers/crypto/inside-secure/safexcel.c
+++ b/drivers/crypto/inside-secure/safexcel.c
@@ -329,6 +329,7 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv)
 	val = EIP197_HIA_DSE_CFG_DIS_DEBUG;
 	val |= EIP197_HIA_DxE_CFG_MIN_DATA_SIZE(7) | EIP197_HIA_DxE_CFG_MAX_DATA_SIZE(8);
 	val |= EIP197_HIA_DxE_CFG_DATA_CACHE_CTRL(WR_CACHE_3BITS);
+	val |= EIP197_HIA_DSE_CFG_ALLWAYS_BUFFERABLE;
 	val |= EIP197_HIA_DSE_CFG_EN_SINGLE_WR;
 	writel(val, priv->base + EIP197_HIA_DSE_CFG);
 
diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h
index b8a81c568c99..7e3cbb9ac98e 100644
--- a/drivers/crypto/inside-secure/safexcel.h
+++ b/drivers/crypto/inside-secure/safexcel.h
@@ -139,6 +139,7 @@
 #define EIP197_HIA_DxE_CFG_MIN_DATA_SIZE(n)	((n) << 0)
 #define EIP197_HIA_DxE_CFG_DATA_CACHE_CTRL(n)	(((n) & 0x7) << 4)
 #define EIP197_HIA_DxE_CFG_MAX_DATA_SIZE(n)	((n) << 8)
+#define EIP197_HIA_DSE_CFG_ALLWAYS_BUFFERABLE	GENMASK(15, 14)
 #define EIP197_HIA_DxE_CFG_MIN_CTRL_SIZE(n)	((n) << 16)
 #define EIP197_HIA_DxE_CFG_CTRL_CACHE_CTRL(n)	(((n) & 0x7) << 20)
 #define EIP197_HIA_DxE_CFG_MAX_CTRL_SIZE(n)	((n) << 24)
-- 
2.9.4

[PATCH 06/13] crypto: inside-secure - align the cipher and hash send functions

[Antoine Tenart]

From: Ofer Heifetz <oferh@marvell.com>

The cipher and hash send functions both set the results variable, to be
used later by the caller. Align the way they're setting this variable.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel_cipher.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/inside-secure/safexcel_cipher.c b/drivers/crypto/inside-secure/safexcel_cipher.c
index 59e6081602a2..8eea4d30db31 100644
--- a/drivers/crypto/inside-secure/safexcel_cipher.c
+++ b/drivers/crypto/inside-secure/safexcel_cipher.c
@@ -269,7 +269,7 @@ static int safexcel_aes_send(struct crypto_async_request *async,
 	spin_unlock_bh(&priv->ring[ring].egress_lock);
 
 	*commands = n_cdesc;
-	*results = nr_dst;
+	*results = n_rdesc;
 	return 0;
 
 rdesc_rollback:
-- 
2.9.4

[PATCH 07/13] crypto: inside-secure - update the context and request later

[Antoine Tenart]

This move the context and request updates at the end of the cipher and
hash send() functions. This way the context and request fields are set
only when everything else was successful in the send() functions.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel_cipher.c | 7 +++----
 drivers/crypto/inside-secure/safexcel_hash.c   | 8 ++++----
 2 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/inside-secure/safexcel_cipher.c b/drivers/crypto/inside-secure/safexcel_cipher.c
index 8eea4d30db31..6037cdfc1f16 100644
--- a/drivers/crypto/inside-secure/safexcel_cipher.c
+++ b/drivers/crypto/inside-secure/safexcel_cipher.c
@@ -190,8 +190,6 @@ static int safexcel_aes_send(struct crypto_async_request *async,
 	int nr_src, nr_dst, n_cdesc = 0, n_rdesc = 0, queued = req->cryptlen;
 	int i, ret = 0;
 
-	request->req = &req->base;
-
 	if (req->src == req->dst) {
 		nr_src = dma_map_sg(priv->dev, req->src,
 				    sg_nents_for_len(req->src, req->cryptlen),
@@ -264,10 +262,11 @@ static int safexcel_aes_send(struct crypto_async_request *async,
 		n_rdesc++;
 	}
 
-	ctx->base.handle_result = safexcel_handle_result;
-
 	spin_unlock_bh(&priv->ring[ring].egress_lock);
 
+	request->req = &req->base;
+	ctx->base.handle_result = safexcel_handle_result;
+
 	*commands = n_cdesc;
 	*results = n_rdesc;
 	return 0;
diff --git a/drivers/crypto/inside-secure/safexcel_hash.c b/drivers/crypto/inside-secure/safexcel_hash.c
index 6eee1a502225..4e526372464f 100644
--- a/drivers/crypto/inside-secure/safexcel_hash.c
+++ b/drivers/crypto/inside-secure/safexcel_hash.c
@@ -198,9 +198,6 @@ static int safexcel_ahash_send(struct crypto_async_request *async, int ring,
 		len -= extra;
 	}
 
-	request->req = &areq->base;
-	ctx->base.handle_result = safexcel_handle_result;
-
 	spin_lock_bh(&priv->ring[ring].egress_lock);
 
 	/* Add a command descriptor for the cached data, if any */
@@ -291,9 +288,12 @@ static int safexcel_ahash_send(struct crypto_async_request *async, int ring,
 		goto cdesc_rollback;
 	}
 
-	req->processed += len;
 	spin_unlock_bh(&priv->ring[ring].egress_lock);
 
+	req->processed += len;
+	request->req = &areq->base;
+	ctx->base.handle_result = safexcel_handle_result;
+
 	*commands = n_cdesc;
 	*results = 1;
 	return 0;
-- 
2.9.4

[PATCH 08/13] crypto: inside-secure - use one queue per hw ring

[Antoine Tenart]

Update the inside-secure safexcel driver from using one global queue to
one queue per hw ring. This ease the request management and keep the hw
in sync with what's done in sw.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.c        | 86 ++++++++++++--------------
 drivers/crypto/inside-secure/safexcel.h        | 12 ++--
 drivers/crypto/inside-secure/safexcel_cipher.c | 38 +++++++-----
 drivers/crypto/inside-secure/safexcel_hash.c   | 38 +++++++-----
 4 files changed, 89 insertions(+), 85 deletions(-)

diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c
index 73f4ef8d71f3..8956b23803a8 100644
--- a/drivers/crypto/inside-secure/safexcel.c
+++ b/drivers/crypto/inside-secure/safexcel.c
@@ -422,20 +422,18 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv)
 	return 0;
 }
 
-void safexcel_dequeue(struct safexcel_crypto_priv *priv)
+void safexcel_dequeue(struct safexcel_crypto_priv *priv, int ring)
 {
 	struct crypto_async_request *req, *backlog;
 	struct safexcel_context *ctx;
 	struct safexcel_request *request;
-	int i, ret, n = 0, nreq[EIP197_MAX_RINGS] = {0};
-	int cdesc[EIP197_MAX_RINGS] = {0}, rdesc[EIP197_MAX_RINGS] = {0};
-	int commands, results;
+	int ret, nreq = 0, cdesc = 0, rdesc = 0, commands, results;
 
 	do {
-		spin_lock_bh(&priv->lock);
-		req = crypto_dequeue_request(&priv->queue);
-		backlog = crypto_get_backlog(&priv->queue);
-		spin_unlock_bh(&priv->lock);
+		spin_lock_bh(&priv->ring[ring].queue_lock);
+		req = crypto_dequeue_request(&priv->ring[ring].queue);
+		backlog = crypto_get_backlog(&priv->ring[ring].queue);
+		spin_unlock_bh(&priv->ring[ring].queue_lock);
 
 		if (!req)
 			goto finalize;
@@ -445,58 +443,51 @@ void safexcel_dequeue(struct safexcel_crypto_priv *priv)
 			goto requeue;
 
 		ctx = crypto_tfm_ctx(req->tfm);
-		ret = ctx->send(req, ctx->ring, request, &commands, &results);
+		ret = ctx->send(req, ring, request, &commands, &results);
 		if (ret) {
 			kfree(request);
 requeue:
-			spin_lock_bh(&priv->lock);
-			crypto_enqueue_request(&priv->queue, req);
-			spin_unlock_bh(&priv->lock);
+			spin_lock_bh(&priv->ring[ring].queue_lock);
+			crypto_enqueue_request(&priv->ring[ring].queue, req);
+			spin_unlock_bh(&priv->ring[ring].queue_lock);
 
-			priv->need_dequeue = true;
+			priv->ring[ring].need_dequeue = true;
 			continue;
 		}
 
 		if (backlog)
 			backlog->complete(backlog, -EINPROGRESS);
 
-		spin_lock_bh(&priv->ring[ctx->ring].egress_lock);
-		list_add_tail(&request->list, &priv->ring[ctx->ring].list);
-		spin_unlock_bh(&priv->ring[ctx->ring].egress_lock);
-
-		cdesc[ctx->ring] += commands;
-		rdesc[ctx->ring] += results;
+		spin_lock_bh(&priv->ring[ring].egress_lock);
+		list_add_tail(&request->list, &priv->ring[ring].list);
+		spin_unlock_bh(&priv->ring[ring].egress_lock);
 
-		nreq[ctx->ring]++;
-	} while (n++ < EIP197_MAX_BATCH_SZ);
+		cdesc += commands;
+		rdesc += results;
+	} while (nreq++ < EIP197_MAX_BATCH_SZ);
 
 finalize:
-	if (n == EIP197_MAX_BATCH_SZ)
-		priv->need_dequeue = true;
-	else if (!n)
+	if (nreq == EIP197_MAX_BATCH_SZ)
+		priv->ring[ring].need_dequeue = true;
+	else if (!nreq)
 		return;
 
-	for (i = 0; i < priv->config.rings; i++) {
-		if (!nreq[i])
-			continue;
+	spin_lock_bh(&priv->ring[ring].lock);
 
-		spin_lock_bh(&priv->ring[i].lock);
+	/* Configure when we want an interrupt */
+	writel(EIP197_HIA_RDR_THRESH_PKT_MODE |
+	       EIP197_HIA_RDR_THRESH_PROC_PKT(nreq),
+	       priv->base + EIP197_HIA_RDR(ring) + EIP197_HIA_xDR_THRESH);
 
-		/* Configure when we want an interrupt */
-		writel(EIP197_HIA_RDR_THRESH_PKT_MODE |
-		       EIP197_HIA_RDR_THRESH_PROC_PKT(nreq[i]),
-		       priv->base + EIP197_HIA_RDR(i) + EIP197_HIA_xDR_THRESH);
+	/* let the RDR know we have pending descriptors */
+	writel((rdesc * priv->config.rd_offset) << 2,
+	       priv->base + EIP197_HIA_RDR(ring) + EIP197_HIA_xDR_PREP_COUNT);
 
-		/* let the RDR know we have pending descriptors */
-		writel((rdesc[i] * priv->config.rd_offset) << 2,
-		       priv->base + EIP197_HIA_RDR(i) + EIP197_HIA_xDR_PREP_COUNT);
+	/* let the CDR know we have pending descriptors */
+	writel((cdesc * priv->config.cd_offset) << 2,
+	       priv->base + EIP197_HIA_CDR(ring) + EIP197_HIA_xDR_PREP_COUNT);
 
-		/* let the CDR know we have pending descriptors */
-		writel((cdesc[i] * priv->config.cd_offset) << 2,
-		       priv->base + EIP197_HIA_CDR(i) + EIP197_HIA_xDR_PREP_COUNT);
-
-		spin_unlock_bh(&priv->ring[i].lock);
-	}
+	spin_unlock_bh(&priv->ring[ring].lock);
 }
 
 void safexcel_free_context(struct safexcel_crypto_priv *priv,
@@ -638,9 +629,9 @@ static void safexcel_handle_result_work(struct work_struct *work)
 
 	safexcel_handle_result_descriptor(priv, data->ring);
 
-	if (priv->need_dequeue) {
-		priv->need_dequeue = false;
-		safexcel_dequeue(data->priv);
+	if (priv->ring[data->ring].need_dequeue) {
+		priv->ring[data->ring].need_dequeue = false;
+		safexcel_dequeue(data->priv, data->ring);
 	}
 }
 
@@ -864,17 +855,18 @@ static int safexcel_probe(struct platform_device *pdev)
 			goto err_clk;
 		}
 
+		crypto_init_queue(&priv->ring[i].queue,
+				  EIP197_DEFAULT_RING_SIZE);
+
 		INIT_LIST_HEAD(&priv->ring[i].list);
 		spin_lock_init(&priv->ring[i].lock);
 		spin_lock_init(&priv->ring[i].egress_lock);
+		spin_lock_init(&priv->ring[i].queue_lock);
 	}
 
 	platform_set_drvdata(pdev, priv);
 	atomic_set(&priv->ring_used, 0);
 
-	spin_lock_init(&priv->lock);
-	crypto_init_queue(&priv->queue, EIP197_DEFAULT_RING_SIZE);
-
 	ret = safexcel_hw_init(priv);
 	if (ret) {
 		dev_err(dev, "EIP h/w init failed (%d)\n", ret);
diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h
index 7e3cbb9ac98e..abe0f59d1473 100644
--- a/drivers/crypto/inside-secure/safexcel.h
+++ b/drivers/crypto/inside-secure/safexcel.h
@@ -469,11 +469,6 @@ struct safexcel_crypto_priv {
 	struct clk *clk;
 	struct safexcel_config config;
 
-	spinlock_t lock;
-	struct crypto_queue queue;
-
-	bool need_dequeue;
-
 	/* context DMA pool */
 	struct dma_pool *context_pool;
 
@@ -490,6 +485,11 @@ struct safexcel_crypto_priv {
 		/* command/result rings */
 		struct safexcel_ring cdr;
 		struct safexcel_ring rdr;
+
+		/* queue */
+		struct crypto_queue queue;
+		spinlock_t queue_lock;
+		bool need_dequeue;
 	} ring[EIP197_MAX_RINGS];
 };
 
@@ -533,7 +533,7 @@ struct safexcel_inv_result {
 	int error;
 };
 
-void safexcel_dequeue(struct safexcel_crypto_priv *priv);
+void safexcel_dequeue(struct safexcel_crypto_priv *priv, int ring);
 void safexcel_complete(struct safexcel_crypto_priv *priv, int ring);
 void safexcel_free_context(struct safexcel_crypto_priv *priv,
 				  struct crypto_async_request *req,
diff --git a/drivers/crypto/inside-secure/safexcel_cipher.c b/drivers/crypto/inside-secure/safexcel_cipher.c
index 6037cdfc1f16..d2207ac5ba19 100644
--- a/drivers/crypto/inside-secure/safexcel_cipher.c
+++ b/drivers/crypto/inside-secure/safexcel_cipher.c
@@ -339,18 +339,21 @@ static int safexcel_handle_inv_result(struct safexcel_crypto_priv *priv,
 		return ndesc;
 	}
 
+	ring = safexcel_select_ring(priv);
+	ctx->base.ring = ring;
 	ctx->base.needs_inv = false;
-	ctx->base.ring = safexcel_select_ring(priv);
 	ctx->base.send = safexcel_aes_send;
 
-	spin_lock_bh(&priv->lock);
-	enq_ret = crypto_enqueue_request(&priv->queue, async);
-	spin_unlock_bh(&priv->lock);
+	spin_lock_bh(&priv->ring[ring].queue_lock);
+	enq_ret = crypto_enqueue_request(&priv->ring[ring].queue, async);
+	spin_unlock_bh(&priv->ring[ring].queue_lock);
 
 	if (enq_ret != -EINPROGRESS)
 		*ret = enq_ret;
 
-	priv->need_dequeue = true;
+	if (!priv->ring[ring].need_dequeue)
+		safexcel_dequeue(priv, ring);
+
 	*should_complete = false;
 
 	return ndesc;
@@ -384,6 +387,7 @@ static int safexcel_cipher_exit_inv(struct crypto_tfm *tfm)
 	struct safexcel_crypto_priv *priv = ctx->priv;
 	struct skcipher_request req;
 	struct safexcel_inv_result result = { 0 };
+	int ring = ctx->base.ring;
 
 	memset(&req, 0, sizeof(struct skcipher_request));
 
@@ -397,12 +401,12 @@ static int safexcel_cipher_exit_inv(struct crypto_tfm *tfm)
 	ctx->base.exit_inv = true;
 	ctx->base.send = safexcel_cipher_send_inv;
 
-	spin_lock_bh(&priv->lock);
-	crypto_enqueue_request(&priv->queue, &req.base);
-	spin_unlock_bh(&priv->lock);
+	spin_lock_bh(&priv->ring[ring].queue_lock);
+	crypto_enqueue_request(&priv->ring[ring].queue, &req.base);
+	spin_unlock_bh(&priv->ring[ring].queue_lock);
 
-	if (!priv->need_dequeue)
-		safexcel_dequeue(priv);
+	if (!priv->ring[ring].need_dequeue)
+		safexcel_dequeue(priv, ring);
 
 	wait_for_completion_interruptible(&result.completion);
 
@@ -421,7 +425,7 @@ static int safexcel_aes(struct skcipher_request *req,
 {
 	struct safexcel_cipher_ctx *ctx = crypto_tfm_ctx(req->base.tfm);
 	struct safexcel_crypto_priv *priv = ctx->priv;
-	int ret;
+	int ret, ring;
 
 	ctx->direction = dir;
 	ctx->mode = mode;
@@ -440,12 +444,14 @@ static int safexcel_aes(struct skcipher_request *req,
 			return -ENOMEM;
 	}
 
-	spin_lock_bh(&priv->lock);
-	ret = crypto_enqueue_request(&priv->queue, &req->base);
-	spin_unlock_bh(&priv->lock);
+	ring = ctx->base.ring;
+
+	spin_lock_bh(&priv->ring[ring].queue_lock);
+	ret = crypto_enqueue_request(&priv->ring[ring].queue, &req->base);
+	spin_unlock_bh(&priv->ring[ring].queue_lock);
 
-	if (!priv->need_dequeue)
-		safexcel_dequeue(priv);
+	if (!priv->ring[ring].need_dequeue)
+		safexcel_dequeue(priv, ring);
 
 	return ret;
 }
diff --git a/drivers/crypto/inside-secure/safexcel_hash.c b/drivers/crypto/inside-secure/safexcel_hash.c
index 4e526372464f..8527a5899a2f 100644
--- a/drivers/crypto/inside-secure/safexcel_hash.c
+++ b/drivers/crypto/inside-secure/safexcel_hash.c
@@ -374,18 +374,21 @@ static int safexcel_handle_inv_result(struct safexcel_crypto_priv *priv,
 		return 1;
 	}
 
-	ctx->base.ring = safexcel_select_ring(priv);
+	ring = safexcel_select_ring(priv);
+	ctx->base.ring = ring;
 	ctx->base.needs_inv = false;
 	ctx->base.send = safexcel_ahash_send;
 
-	spin_lock_bh(&priv->lock);
-	enq_ret = crypto_enqueue_request(&priv->queue, async);
-	spin_unlock_bh(&priv->lock);
+	spin_lock_bh(&priv->ring[ring].queue_lock);
+	enq_ret = crypto_enqueue_request(&priv->ring[ring].queue, async);
+	spin_unlock_bh(&priv->ring[ring].queue_lock);
 
 	if (enq_ret != -EINPROGRESS)
 		*ret = enq_ret;
 
-	priv->need_dequeue = true;
+	if (!priv->ring[ring].need_dequeue)
+		safexcel_dequeue(priv, ring);
+
 	*should_complete = false;
 
 	return 1;
@@ -417,6 +420,7 @@ static int safexcel_ahash_exit_inv(struct crypto_tfm *tfm)
 	struct safexcel_crypto_priv *priv = ctx->priv;
 	struct ahash_request req;
 	struct safexcel_inv_result result = { 0 };
+	int ring = ctx->base.ring;
 
 	memset(&req, 0, sizeof(struct ahash_request));
 
@@ -430,12 +434,12 @@ static int safexcel_ahash_exit_inv(struct crypto_tfm *tfm)
 	ctx->base.exit_inv = true;
 	ctx->base.send = safexcel_ahash_send_inv;
 
-	spin_lock_bh(&priv->lock);
-	crypto_enqueue_request(&priv->queue, &req.base);
-	spin_unlock_bh(&priv->lock);
+	spin_lock_bh(&priv->ring[ring].queue_lock);
+	crypto_enqueue_request(&priv->ring[ring].queue, &req.base);
+	spin_unlock_bh(&priv->ring[ring].queue_lock);
 
-	if (!priv->need_dequeue)
-		safexcel_dequeue(priv);
+	if (!priv->ring[ring].need_dequeue)
+		safexcel_dequeue(priv, ring);
 
 	wait_for_completion_interruptible(&result.completion);
 
@@ -477,7 +481,7 @@ static int safexcel_ahash_enqueue(struct ahash_request *areq)
 	struct safexcel_ahash_ctx *ctx = crypto_ahash_ctx(crypto_ahash_reqtfm(areq));
 	struct safexcel_ahash_req *req = ahash_request_ctx(areq);
 	struct safexcel_crypto_priv *priv = ctx->priv;
-	int ret;
+	int ret, ring;
 
 	ctx->base.send = safexcel_ahash_send;
 
@@ -496,12 +500,14 @@ static int safexcel_ahash_enqueue(struct ahash_request *areq)
 			return -ENOMEM;
 	}
 
-	spin_lock_bh(&priv->lock);
-	ret = crypto_enqueue_request(&priv->queue, &areq->base);
-	spin_unlock_bh(&priv->lock);
+	ring = ctx->base.ring;
+
+	spin_lock_bh(&priv->ring[ring].queue_lock);
+	ret = crypto_enqueue_request(&priv->ring[ring].queue, &areq->base);
+	spin_unlock_bh(&priv->ring[ring].queue_lock);
 
-	if (!priv->need_dequeue)
-		safexcel_dequeue(priv);
+	if (!priv->ring[ring].need_dequeue)
+		safexcel_dequeue(priv, ring);
 
 	return ret;
 }
-- 
2.9.4

[PATCH 09/13] crypto: inside-secure - stop requeueing failed requests

[Antoine Tenart]

This update the dequeue function of the inside-secure safexcel driver so
that failed requests aren't requeued when they fail (for whatever
reason, which can be because the hw ring is full).

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c
index 8956b23803a8..8ae133a9e3f2 100644
--- a/drivers/crypto/inside-secure/safexcel.c
+++ b/drivers/crypto/inside-secure/safexcel.c
@@ -439,20 +439,22 @@ void safexcel_dequeue(struct safexcel_crypto_priv *priv, int ring)
 			goto finalize;
 
 		request = kzalloc(sizeof(*request), EIP197_GFP_FLAGS(*req));
-		if (!request)
-			goto requeue;
+		if (!request) {
+			spin_lock_bh(&priv->ring[ring].queue_lock);
+			crypto_enqueue_request(&priv->ring[ring].queue, req);
+			spin_unlock_bh(&priv->ring[ring].queue_lock);
+
+			priv->ring[ring].need_dequeue = true;
+			goto finalize;
+		}
 
 		ctx = crypto_tfm_ctx(req->tfm);
 		ret = ctx->send(req, ring, request, &commands, &results);
 		if (ret) {
 			kfree(request);
-requeue:
-			spin_lock_bh(&priv->ring[ring].queue_lock);
-			crypto_enqueue_request(&priv->ring[ring].queue, req);
-			spin_unlock_bh(&priv->ring[ring].queue_lock);
-
+			req->complete(req, ret);
 			priv->ring[ring].need_dequeue = true;
-			continue;
+			goto finalize;
 		}
 
 		if (backlog)
-- 
2.9.4

[PATCH 10/13] crypto: inside-secure - get the backlog before dequeueing the request

[Antoine Tenart]

Get the backlog before dequeuing the request otherwise we'll miss the
first request in line.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c
index 8ae133a9e3f2..8f195e031938 100644
--- a/drivers/crypto/inside-secure/safexcel.c
+++ b/drivers/crypto/inside-secure/safexcel.c
@@ -431,8 +431,8 @@ void safexcel_dequeue(struct safexcel_crypto_priv *priv, int ring)
 
 	do {
 		spin_lock_bh(&priv->ring[ring].queue_lock);
-		req = crypto_dequeue_request(&priv->ring[ring].queue);
 		backlog = crypto_get_backlog(&priv->ring[ring].queue);
+		req = crypto_dequeue_request(&priv->ring[ring].queue);
 		spin_unlock_bh(&priv->ring[ring].queue_lock);
 
 		if (!req)
-- 
2.9.4

[PATCH 11/13] crypto: inside-secure - only dequeue when needed

[Antoine Tenart]

This force the need_dequeue flag to be unset whenever the dequeue
function is called, to avoid calling it when it is not necessary.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c
index 8f195e031938..e7f87ac12685 100644
--- a/drivers/crypto/inside-secure/safexcel.c
+++ b/drivers/crypto/inside-secure/safexcel.c
@@ -429,6 +429,8 @@ void safexcel_dequeue(struct safexcel_crypto_priv *priv, int ring)
 	struct safexcel_request *request;
 	int ret, nreq = 0, cdesc = 0, rdesc = 0, commands, results;
 
+	priv->ring[ring].need_dequeue = false;
+
 	do {
 		spin_lock_bh(&priv->ring[ring].queue_lock);
 		backlog = crypto_get_backlog(&priv->ring[ring].queue);
@@ -631,10 +633,8 @@ static void safexcel_handle_result_work(struct work_struct *work)
 
 	safexcel_handle_result_descriptor(priv, data->ring);
 
-	if (priv->ring[data->ring].need_dequeue) {
-		priv->ring[data->ring].need_dequeue = false;
+	if (priv->ring[data->ring].need_dequeue)
 		safexcel_dequeue(data->priv, data->ring);
-	}
 }
 
 struct safexcel_ring_irq_data {
-- 
2.9.4

[PATCH 12/13] crypto: inside-secure - increase the batch size

[Antoine Tenart]

Increase the batch size to the maximum number of requests a ring can
handle at a time (its size). This is possible now that the request
queues are per hw ring. This improves performances.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h
index abe0f59d1473..304c5838c11a 100644
--- a/drivers/crypto/inside-secure/safexcel.h
+++ b/drivers/crypto/inside-secure/safexcel.h
@@ -23,7 +23,7 @@
 #define EIP197_MAX_TOKENS			5
 #define EIP197_MAX_RINGS			4
 #define EIP197_FETCH_COUNT			1
-#define EIP197_MAX_BATCH_SZ			8
+#define EIP197_MAX_BATCH_SZ			EIP197_DEFAULT_RING_SIZE
 
 #define EIP197_GFP_FLAGS(base)	((base).flags & CRYPTO_TFM_REQ_MAY_SLEEP ? \
 				 GFP_KERNEL : GFP_ATOMIC)
-- 
2.9.4

[PATCH 13/13] crypto: inside-secure - use the base_end pointer in ring rollback

[Antoine Tenart]

A base_end pointer is set and provided. Use it in the ring rollback
function to avoid using build-in defines.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
---
 drivers/crypto/inside-secure/safexcel_ring.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/inside-secure/safexcel_ring.c b/drivers/crypto/inside-secure/safexcel_ring.c
index fdbf05ae55fc..c9d2a8716b5b 100644
--- a/drivers/crypto/inside-secure/safexcel_ring.c
+++ b/drivers/crypto/inside-secure/safexcel_ring.c
@@ -84,7 +84,7 @@ void safexcel_ring_rollback_wptr(struct safexcel_crypto_priv *priv,
 		return;
 
 	if (ring->write == ring->base)
-		ring->write += (EIP197_DEFAULT_RING_SIZE - 1) * ring->offset;
+		ring->write = ring->base_end - ring->offset;
 	else
 		ring->write -= ring->offset;
 
-- 
2.9.4

Re: [PATCH 00/13] crypto: inside-secure - various improvements

[Herbert Xu]

On Thu, Jun 15, 2017 at 09:56:16AM +0200, Antoine Tenart wrote:
> Hi Herbert,
> 
> This series improves the newly added inside-secure driver in various
> ways. There are also a few non-critical fixes.
> 
> The series is based on top of your cryptodev/master branch, as it
> depends on the inside-secure driver addition ("crypto: inside-secure -
> add SafeXcel EIP197 crypto engine driver").
> 
> I'd like to thank Ofer Heifetz and Igal Liberman who helped making
> these changes!
> 
> Thanks,
> Antoine
> 
> Antoine Tenart (10):

All applied.  Thanks.
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt