* [PATCH 01/19] crypto: sun8i-ce: Fix minor style issue
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 02/19] crypto: sun8i-ce: do not allocate memory when handling requests Corentin Labbe
` (18 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
This patch remove a double blank line.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
index 35e3cadccac2..01d032e08825 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
@@ -398,7 +398,6 @@ int sun8i_ce_cipher_init(struct crypto_tfm *tfm)
sktfm->reqsize = sizeof(struct sun8i_cipher_req_ctx) +
crypto_skcipher_reqsize(op->fallback_tfm);
-
dev_info(op->ce->dev, "Fallback for %s is %s\n",
crypto_tfm_alg_driver_name(&sktfm->base),
crypto_tfm_alg_driver_name(crypto_skcipher_tfm(op->fallback_tfm)));
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 02/19] crypto: sun8i-ce: do not allocate memory when handling requests
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
2022-03-17 20:55 ` [PATCH 01/19] crypto: sun8i-ce: Fix minor style issue Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 03/19] crypto: sun4i-ss: do not allocate backup IV on requests Corentin Labbe
` (17 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
Instead of allocate memory on each requests, it is easier to
pre-allocate buffer for IV.
This made error path easier.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../allwinner/sun8i-ce/sun8i-ce-cipher.c | 29 ++++++-------------
.../crypto/allwinner/sun8i-ce/sun8i-ce-core.c | 20 ++++++++++---
drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h | 8 ++---
3 files changed, 29 insertions(+), 28 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
index 01d032e08825..0b1ce58bdeb9 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
@@ -152,23 +152,13 @@ static int sun8i_ce_cipher_prepare(struct crypto_engine *engine, void *async_req
ivsize = crypto_skcipher_ivsize(tfm);
if (areq->iv && crypto_skcipher_ivsize(tfm) > 0) {
rctx->ivlen = ivsize;
- rctx->bounce_iv = kzalloc(ivsize, GFP_KERNEL | GFP_DMA);
- if (!rctx->bounce_iv) {
- err = -ENOMEM;
- goto theend_key;
- }
if (rctx->op_dir & CE_DECRYPTION) {
- rctx->backup_iv = kzalloc(ivsize, GFP_KERNEL);
- if (!rctx->backup_iv) {
- err = -ENOMEM;
- goto theend_key;
- }
offset = areq->cryptlen - ivsize;
- scatterwalk_map_and_copy(rctx->backup_iv, areq->src,
+ scatterwalk_map_and_copy(chan->backup_iv, areq->src,
offset, ivsize, 0);
}
- memcpy(rctx->bounce_iv, areq->iv, ivsize);
- rctx->addr_iv = dma_map_single(ce->dev, rctx->bounce_iv, rctx->ivlen,
+ memcpy(chan->bounce_iv, areq->iv, ivsize);
+ rctx->addr_iv = dma_map_single(ce->dev, chan->bounce_iv, rctx->ivlen,
DMA_TO_DEVICE);
if (dma_mapping_error(ce->dev, rctx->addr_iv)) {
dev_err(ce->dev, "Cannot DMA MAP IV\n");
@@ -257,16 +247,15 @@ static int sun8i_ce_cipher_prepare(struct crypto_engine *engine, void *async_req
dma_unmap_single(ce->dev, rctx->addr_iv, rctx->ivlen, DMA_TO_DEVICE);
offset = areq->cryptlen - ivsize;
if (rctx->op_dir & CE_DECRYPTION) {
- memcpy(areq->iv, rctx->backup_iv, ivsize);
- kfree_sensitive(rctx->backup_iv);
+ memcpy(areq->iv, chan->backup_iv, ivsize);
+ memzero_explicit(chan->backup_iv, ivsize);
} else {
scatterwalk_map_and_copy(areq->iv, areq->dst, offset,
ivsize, 0);
}
- kfree(rctx->bounce_iv);
+ memzero_explicit(chan->bounce_iv, ivsize);
}
-theend_key:
dma_unmap_single(ce->dev, rctx->addr_key, op->keylen, DMA_TO_DEVICE);
theend:
@@ -322,13 +311,13 @@ static int sun8i_ce_cipher_unprepare(struct crypto_engine *engine, void *async_r
dma_unmap_single(ce->dev, rctx->addr_iv, rctx->ivlen, DMA_TO_DEVICE);
offset = areq->cryptlen - ivsize;
if (rctx->op_dir & CE_DECRYPTION) {
- memcpy(areq->iv, rctx->backup_iv, ivsize);
- kfree_sensitive(rctx->backup_iv);
+ memcpy(areq->iv, chan->backup_iv, ivsize);
+ memzero_explicit(chan->backup_iv, ivsize);
} else {
scatterwalk_map_and_copy(areq->iv, areq->dst, offset,
ivsize, 0);
}
- kfree(rctx->bounce_iv);
+ memzero_explicit(chan->bounce_iv, ivsize);
}
dma_unmap_single(ce->dev, rctx->addr_key, op->keylen, DMA_TO_DEVICE);
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
index 8be27a462784..8f902607af68 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
@@ -305,7 +305,7 @@ static struct sun8i_ce_alg_template ce_algs[] = {
.cra_priority = 400,
.cra_blocksize = AES_BLOCK_SIZE,
.cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
- CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_ASYNC |
CRYPTO_ALG_NEED_FALLBACK,
.cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx),
.cra_module = THIS_MODULE,
@@ -332,7 +332,7 @@ static struct sun8i_ce_alg_template ce_algs[] = {
.cra_priority = 400,
.cra_blocksize = AES_BLOCK_SIZE,
.cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
- CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_ASYNC |
CRYPTO_ALG_NEED_FALLBACK,
.cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx),
.cra_module = THIS_MODULE,
@@ -358,7 +358,7 @@ static struct sun8i_ce_alg_template ce_algs[] = {
.cra_priority = 400,
.cra_blocksize = DES3_EDE_BLOCK_SIZE,
.cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
- CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_ASYNC |
CRYPTO_ALG_NEED_FALLBACK,
.cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx),
.cra_module = THIS_MODULE,
@@ -385,7 +385,7 @@ static struct sun8i_ce_alg_template ce_algs[] = {
.cra_priority = 400,
.cra_blocksize = DES3_EDE_BLOCK_SIZE,
.cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
- CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_ASYNC |
CRYPTO_ALG_NEED_FALLBACK,
.cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx),
.cra_module = THIS_MODULE,
@@ -728,6 +728,18 @@ static int sun8i_ce_allocate_chanlist(struct sun8i_ce_dev *ce)
err = -ENOMEM;
goto error_engine;
}
+ ce->chanlist[i].bounce_iv = devm_kmalloc(ce->dev, AES_BLOCK_SIZE,
+ GFP_KERNEL | GFP_DMA);
+ if (!ce->chanlist[i].bounce_iv) {
+ err = -ENOMEM;
+ goto error_engine;
+ }
+ ce->chanlist[i].backup_iv = devm_kmalloc(ce->dev, AES_BLOCK_SIZE,
+ GFP_KERNEL);
+ if (!ce->chanlist[i].backup_iv) {
+ err = -ENOMEM;
+ goto error_engine;
+ }
}
return 0;
error_engine:
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h
index 719f9a730857..229b696d5a2c 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h
@@ -213,6 +213,8 @@ struct ce_task {
* @status: set to 1 by interrupt if task is done
* @t_phy: Physical address of task
* @tl: pointer to the current ce_task for this flow
+ * @backup_iv: buffer which contain the next IV to store
+ * @bounce_iv: buffer which contain the IV
* @stat_req: number of request done by this flow
*/
struct sun8i_ce_flow {
@@ -222,6 +224,8 @@ struct sun8i_ce_flow {
dma_addr_t t_phy;
int timeout;
struct ce_task *tl;
+ void *backup_iv;
+ void *bounce_iv;
#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG
unsigned long stat_req;
#endif
@@ -268,8 +272,6 @@ struct sun8i_ce_dev {
* struct sun8i_cipher_req_ctx - context for a skcipher request
* @op_dir: direction (encrypt vs decrypt) for this request
* @flow: the flow to use for this request
- * @backup_iv: buffer which contain the next IV to store
- * @bounce_iv: buffer which contain the IV
* @ivlen: size of bounce_iv
* @nr_sgs: The number of source SG (as given by dma_map_sg())
* @nr_sgd: The number of destination SG (as given by dma_map_sg())
@@ -280,8 +282,6 @@ struct sun8i_ce_dev {
struct sun8i_cipher_req_ctx {
u32 op_dir;
int flow;
- void *backup_iv;
- void *bounce_iv;
unsigned int ivlen;
int nr_sgs;
int nr_sgd;
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 03/19] crypto: sun4i-ss: do not allocate backup IV on requests
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
2022-03-17 20:55 ` [PATCH 01/19] crypto: sun8i-ce: Fix minor style issue Corentin Labbe
2022-03-17 20:55 ` [PATCH 02/19] crypto: sun8i-ce: do not allocate memory when handling requests Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 04/19] crypto: sun8i-ss: rework handling of IV Corentin Labbe
` (16 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
Instead of allocate memory on each requests, it is easier to
pre-allocate buffer for backup IV.
This made error path easier.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../allwinner/sun4i-ss/sun4i-ss-cipher.c | 22 +++++++------------
drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h | 1 +
2 files changed, 9 insertions(+), 14 deletions(-)
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c
index 8dc2a475c601..a8c784acce13 100644
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c
@@ -20,7 +20,6 @@ static int noinline_for_stack sun4i_ss_opti_poll(struct skcipher_request *areq)
unsigned int ivsize = crypto_skcipher_ivsize(tfm);
struct sun4i_cipher_req_ctx *ctx = skcipher_request_ctx(areq);
u32 mode = ctx->mode;
- void *backup_iv = NULL;
/* when activating SS, the default FIFO space is SS_RX_DEFAULT(32) */
u32 rx_cnt = SS_RX_DEFAULT;
u32 tx_cnt = 0;
@@ -47,10 +46,8 @@ static int noinline_for_stack sun4i_ss_opti_poll(struct skcipher_request *areq)
}
if (areq->iv && ivsize > 0 && mode & SS_DECRYPTION) {
- backup_iv = kzalloc(ivsize, GFP_KERNEL);
- if (!backup_iv)
- return -ENOMEM;
- scatterwalk_map_and_copy(backup_iv, areq->src, areq->cryptlen - ivsize, ivsize, 0);
+ scatterwalk_map_and_copy(ctx->backup_iv, areq->src,
+ areq->cryptlen - ivsize, ivsize, 0);
}
if (IS_ENABLED(CONFIG_CRYPTO_DEV_SUN4I_SS_DEBUG)) {
@@ -133,8 +130,8 @@ static int noinline_for_stack sun4i_ss_opti_poll(struct skcipher_request *areq)
if (areq->iv) {
if (mode & SS_DECRYPTION) {
- memcpy(areq->iv, backup_iv, ivsize);
- kfree_sensitive(backup_iv);
+ memcpy(areq->iv, ctx->backup_iv, ivsize);
+ memzero_explicit(ctx->backup_iv, ivsize);
} else {
scatterwalk_map_and_copy(areq->iv, areq->dst, areq->cryptlen - ivsize,
ivsize, 0);
@@ -198,7 +195,6 @@ static int sun4i_ss_cipher_poll(struct skcipher_request *areq)
unsigned int ileft = areq->cryptlen;
unsigned int oleft = areq->cryptlen;
unsigned int todo;
- void *backup_iv = NULL;
struct sg_mapping_iter mi, mo;
unsigned long pi = 0, po = 0; /* progress for in and out */
bool miter_err;
@@ -242,10 +238,8 @@ static int sun4i_ss_cipher_poll(struct skcipher_request *areq)
return sun4i_ss_cipher_poll_fallback(areq);
if (areq->iv && ivsize > 0 && mode & SS_DECRYPTION) {
- backup_iv = kzalloc(ivsize, GFP_KERNEL);
- if (!backup_iv)
- return -ENOMEM;
- scatterwalk_map_and_copy(backup_iv, areq->src, areq->cryptlen - ivsize, ivsize, 0);
+ scatterwalk_map_and_copy(ctx->backup_iv, areq->src,
+ areq->cryptlen - ivsize, ivsize, 0);
}
if (IS_ENABLED(CONFIG_CRYPTO_DEV_SUN4I_SS_DEBUG)) {
@@ -382,8 +376,8 @@ static int sun4i_ss_cipher_poll(struct skcipher_request *areq)
}
if (areq->iv) {
if (mode & SS_DECRYPTION) {
- memcpy(areq->iv, backup_iv, ivsize);
- kfree_sensitive(backup_iv);
+ memcpy(areq->iv, ctx->backup_iv, ivsize);
+ memzero_explicit(ctx->backup_iv, ivsize);
} else {
scatterwalk_map_and_copy(areq->iv, areq->dst, areq->cryptlen - ivsize,
ivsize, 0);
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h
index 0fee6f4e2d90..ba59c7a48825 100644
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h
+++ b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h
@@ -183,6 +183,7 @@ struct sun4i_tfm_ctx {
struct sun4i_cipher_req_ctx {
u32 mode;
+ u8 backup_iv[AES_BLOCK_SIZE];
struct skcipher_request fallback_req; // keep at the end
};
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 04/19] crypto: sun8i-ss: rework handling of IV
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (2 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 03/19] crypto: sun4i-ss: do not allocate backup IV on requests Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 05/19] crypto: sun8i-ss: handle zero sized sg Corentin Labbe
` (15 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
sun8i-ss fail handling IVs when doing decryption of multiple SGs in-place.
It should backup the last block of each SG source for using it later as
IVs.
In the same time remove allocation on requests path for storing all
IVs.
Fixes: f08fcced6d00 ("crypto: allwinner - Add sun8i-ss cryptographic offloader")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../allwinner/sun8i-ss/sun8i-ss-cipher.c | 115 ++++++++++++------
.../crypto/allwinner/sun8i-ss/sun8i-ss-core.c | 30 +++--
drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h | 14 ++-
3 files changed, 107 insertions(+), 52 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
index 554e400d41ca..70e2e6e37389 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
@@ -93,6 +93,68 @@ static int sun8i_ss_cipher_fallback(struct skcipher_request *areq)
return err;
}
+static int sun8i_ss_setup_ivs(struct skcipher_request *areq)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
+ struct sun8i_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct sun8i_ss_dev *ss = op->ss;
+ struct sun8i_cipher_req_ctx *rctx = skcipher_request_ctx(areq);
+ struct scatterlist *sg = areq->src;
+ unsigned int todo, offset;
+ unsigned int len = areq->cryptlen;
+ unsigned int ivsize = crypto_skcipher_ivsize(tfm);
+ struct sun8i_ss_flow *sf = &ss->flows[rctx->flow];
+ int i = 0;
+ u32 a;
+ int err;
+
+ rctx->ivlen = ivsize;
+ if (rctx->op_dir & SS_DECRYPTION) {
+ offset = areq->cryptlen - ivsize;
+ scatterwalk_map_and_copy(sf->biv, areq->src, offset,
+ ivsize, 0);
+ }
+
+ /* we need to copy all IVs from source in case DMA is bi-directionnal */
+ while (sg && len) {
+ if (sg_dma_len(sg) == 0) {
+ sg = sg_next(sg);
+ continue;
+ }
+ if (i == 0)
+ memcpy(sf->iv[0], areq->iv, ivsize);
+ a = dma_map_single(ss->dev, sf->iv[i], ivsize, DMA_TO_DEVICE);
+ if (dma_mapping_error(ss->dev, a)) {
+ memzero_explicit(sf->iv[i], ivsize);
+ dev_err(ss->dev, "Cannot DMA MAP IV\n");
+ err = -EFAULT;
+ goto dma_iv_error;
+ }
+ rctx->p_iv[i] = a;
+ /* we need to setup all others IVs only in the decrypt way */
+ if (rctx->op_dir & SS_ENCRYPTION)
+ return 0;
+ todo = min(len, sg_dma_len(sg));
+ len -= todo;
+ i++;
+ if (i < MAX_SG) {
+ offset = sg->length - ivsize;
+ scatterwalk_map_and_copy(sf->iv[i], sg, offset, ivsize, 0);
+ }
+ rctx->niv = i;
+ sg = sg_next(sg);
+ }
+
+ return 0;
+dma_iv_error:
+ i--;
+ while (i >= 0) {
+ dma_unmap_single(ss->dev, rctx->p_iv[i], ivsize, DMA_TO_DEVICE);
+ memzero_explicit(sf->iv[i], ivsize);
+ }
+ return err;
+}
+
static int sun8i_ss_cipher(struct skcipher_request *areq)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
@@ -101,9 +163,9 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
struct sun8i_cipher_req_ctx *rctx = skcipher_request_ctx(areq);
struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
struct sun8i_ss_alg_template *algt;
+ struct sun8i_ss_flow *sf = &ss->flows[rctx->flow];
struct scatterlist *sg;
unsigned int todo, len, offset, ivsize;
- void *backup_iv = NULL;
int nr_sgs = 0;
int nr_sgd = 0;
int err = 0;
@@ -134,30 +196,9 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
ivsize = crypto_skcipher_ivsize(tfm);
if (areq->iv && crypto_skcipher_ivsize(tfm) > 0) {
- rctx->ivlen = ivsize;
- rctx->biv = kzalloc(ivsize, GFP_KERNEL | GFP_DMA);
- if (!rctx->biv) {
- err = -ENOMEM;
+ err = sun8i_ss_setup_ivs(areq);
+ if (err)
goto theend_key;
- }
- if (rctx->op_dir & SS_DECRYPTION) {
- backup_iv = kzalloc(ivsize, GFP_KERNEL);
- if (!backup_iv) {
- err = -ENOMEM;
- goto theend_key;
- }
- offset = areq->cryptlen - ivsize;
- scatterwalk_map_and_copy(backup_iv, areq->src, offset,
- ivsize, 0);
- }
- memcpy(rctx->biv, areq->iv, ivsize);
- rctx->p_iv = dma_map_single(ss->dev, rctx->biv, rctx->ivlen,
- DMA_TO_DEVICE);
- if (dma_mapping_error(ss->dev, rctx->p_iv)) {
- dev_err(ss->dev, "Cannot DMA MAP IV\n");
- err = -ENOMEM;
- goto theend_iv;
- }
}
if (areq->src == areq->dst) {
nr_sgs = dma_map_sg(ss->dev, areq->src, sg_nents(areq->src),
@@ -243,21 +284,19 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
}
theend_iv:
- if (rctx->p_iv)
- dma_unmap_single(ss->dev, rctx->p_iv, rctx->ivlen,
- DMA_TO_DEVICE);
-
if (areq->iv && ivsize > 0) {
- if (rctx->biv) {
- offset = areq->cryptlen - ivsize;
- if (rctx->op_dir & SS_DECRYPTION) {
- memcpy(areq->iv, backup_iv, ivsize);
- kfree_sensitive(backup_iv);
- } else {
- scatterwalk_map_and_copy(areq->iv, areq->dst, offset,
- ivsize, 0);
- }
- kfree(rctx->biv);
+ for (i = 0; i < rctx->niv; i++) {
+ dma_unmap_single(ss->dev, rctx->p_iv[i], ivsize, DMA_TO_DEVICE);
+ memzero_explicit(sf->iv[i], ivsize);
+ }
+
+ offset = areq->cryptlen - ivsize;
+ if (rctx->op_dir & SS_DECRYPTION) {
+ memcpy(areq->iv, sf->biv, ivsize);
+ memzero_explicit(sf->biv, ivsize);
+ } else {
+ scatterwalk_map_and_copy(areq->iv, areq->dst, offset,
+ ivsize, 0);
}
}
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
index 319fe3279a71..657530578643 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
@@ -66,6 +66,7 @@ int sun8i_ss_run_task(struct sun8i_ss_dev *ss, struct sun8i_cipher_req_ctx *rctx
const char *name)
{
int flow = rctx->flow;
+ unsigned int ivlen = rctx->ivlen;
u32 v = SS_START;
int i;
@@ -104,15 +105,14 @@ int sun8i_ss_run_task(struct sun8i_ss_dev *ss, struct sun8i_cipher_req_ctx *rctx
mutex_lock(&ss->mlock);
writel(rctx->p_key, ss->base + SS_KEY_ADR_REG);
- if (i == 0) {
- if (rctx->p_iv)
- writel(rctx->p_iv, ss->base + SS_IV_ADR_REG);
- } else {
- if (rctx->biv) {
- if (rctx->op_dir == SS_ENCRYPTION)
- writel(rctx->t_dst[i - 1].addr + rctx->t_dst[i - 1].len * 4 - rctx->ivlen, ss->base + SS_IV_ADR_REG);
+ if (ivlen) {
+ if (rctx->op_dir == SS_ENCRYPTION) {
+ if (i == 0)
+ writel(rctx->p_iv[0], ss->base + SS_IV_ADR_REG);
else
- writel(rctx->t_src[i - 1].addr + rctx->t_src[i - 1].len * 4 - rctx->ivlen, ss->base + SS_IV_ADR_REG);
+ writel(rctx->t_dst[i - 1].addr + rctx->t_dst[i - 1].len * 4 - ivlen, ss->base + SS_IV_ADR_REG);
+ } else {
+ writel(rctx->p_iv[i], ss->base + SS_IV_ADR_REG);
}
}
@@ -464,7 +464,7 @@ static void sun8i_ss_free_flows(struct sun8i_ss_dev *ss, int i)
*/
static int allocate_flows(struct sun8i_ss_dev *ss)
{
- int i, err;
+ int i, j, err;
ss->flows = devm_kcalloc(ss->dev, MAXFLOW, sizeof(struct sun8i_ss_flow),
GFP_KERNEL);
@@ -474,6 +474,18 @@ static int allocate_flows(struct sun8i_ss_dev *ss)
for (i = 0; i < MAXFLOW; i++) {
init_completion(&ss->flows[i].complete);
+ ss->flows[i].biv = devm_kmalloc(ss->dev, AES_BLOCK_SIZE,
+ GFP_KERNEL | GFP_DMA);
+ if (!ss->flows[i].biv)
+ goto error_engine;
+
+ for (j = 0; j < MAX_SG; j++) {
+ ss->flows[i].iv[j] = devm_kmalloc(ss->dev, AES_BLOCK_SIZE,
+ GFP_KERNEL | GFP_DMA);
+ if (!ss->flows[i].iv[j])
+ goto error_engine;
+ }
+
ss->flows[i].engine = crypto_engine_alloc_init(ss->dev, true);
if (!ss->flows[i].engine) {
dev_err(ss->dev, "Cannot allocate engine\n");
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
index 28188685b910..57ada8653855 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
@@ -121,11 +121,15 @@ struct sginfo {
* @complete: completion for the current task on this flow
* @status: set to 1 by interrupt if task is done
* @stat_req: number of request done by this flow
+ * @iv: list of IV to use for each step
+ * @biv: buffer which contain the backuped IV
*/
struct sun8i_ss_flow {
struct crypto_engine *engine;
struct completion complete;
int status;
+ u8 *iv[MAX_SG];
+ u8 *biv;
#ifdef CONFIG_CRYPTO_DEV_SUN8I_SS_DEBUG
unsigned long stat_req;
#endif
@@ -164,28 +168,28 @@ struct sun8i_ss_dev {
* @t_src: list of mapped SGs with their size
* @t_dst: list of mapped SGs with their size
* @p_key: DMA address of the key
- * @p_iv: DMA address of the IV
+ * @p_iv: DMA address of the IVs
+ * @niv: Number of IVs DMA mapped
* @method: current algorithm for this request
* @op_mode: op_mode for this request
* @op_dir: direction (encrypt vs decrypt) for this request
* @flow: the flow to use for this request
- * @ivlen: size of biv
+ * @ivlen: size of IVs
* @keylen: keylen for this request
- * @biv: buffer which contain the IV
* @fallback_req: request struct for invoking the fallback skcipher TFM
*/
struct sun8i_cipher_req_ctx {
struct sginfo t_src[MAX_SG];
struct sginfo t_dst[MAX_SG];
u32 p_key;
- u32 p_iv;
+ u32 p_iv[MAX_SG];
+ int niv;
u32 method;
u32 op_mode;
u32 op_dir;
int flow;
unsigned int ivlen;
unsigned int keylen;
- void *biv;
struct skcipher_request fallback_req; // keep at the end
};
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 05/19] crypto: sun8i-ss: handle zero sized sg
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (3 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 04/19] crypto: sun8i-ss: rework handling of IV Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 06/19] crypto: sun8i-ss: remove redundant test Corentin Labbe
` (14 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
sun8i-ss does not handle well the possible zero sized sg.
Fixes: d9b45418a917 ("crypto: sun8i-ss - support hash algorithms")
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index 1a71ed49d233..ca4f280af35d 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -380,13 +380,21 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
}
len = areq->nbytes;
- for_each_sg(areq->src, sg, nr_sgs, i) {
+ sg = areq->src;
+ i = 0;
+ while (len > 0 && sg) {
+ if (sg_dma_len(sg) == 0) {
+ sg = sg_next(sg);
+ continue;
+ }
rctx->t_src[i].addr = sg_dma_address(sg);
todo = min(len, sg_dma_len(sg));
rctx->t_src[i].len = todo / 4;
len -= todo;
rctx->t_dst[i].addr = addr_res;
rctx->t_dst[i].len = digestsize / 4;
+ sg = sg_next(sg);
+ i++;
}
if (len > 0) {
dev_err(ss->dev, "remaining len %d\n", len);
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 06/19] crypto: sun8i-ss: remove redundant test
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (4 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 05/19] crypto: sun8i-ss: handle zero sized sg Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 07/19] crypto: sun8i-ss: test error before assigning Corentin Labbe
` (13 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
Some fallback tests were redundant with what sun8i_ss_hash_need_fallback() already do.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 12 +-----------
1 file changed, 1 insertion(+), 11 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index ca4f280af35d..eaa0bbaf5581 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -288,21 +288,11 @@ int sun8i_ss_hash_digest(struct ahash_request *areq)
struct sun8i_ss_alg_template *algt;
struct sun8i_ss_dev *ss;
struct crypto_engine *engine;
- struct scatterlist *sg;
- int nr_sgs, e, i;
+ int e;
if (sun8i_ss_hash_need_fallback(areq))
return sun8i_ss_hash_digest_fb(areq);
- nr_sgs = sg_nents(areq->src);
- if (nr_sgs > MAX_SG - 1)
- return sun8i_ss_hash_digest_fb(areq);
-
- for_each_sg(areq->src, sg, nr_sgs, i) {
- if (sg->length % 4 || !IS_ALIGNED(sg->offset, sizeof(u32)))
- return sun8i_ss_hash_digest_fb(areq);
- }
-
algt = container_of(alg, struct sun8i_ss_alg_template, alg.hash);
ss = algt->ss;
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 07/19] crypto: sun8i-ss: test error before assigning
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (5 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 06/19] crypto: sun8i-ss: remove redundant test Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 08/19] crypto: sun8i-ss: use sg_nents_for_len Corentin Labbe
` (12 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
The first thing we should do after dma_map_single() is to test the
result.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index eaa0bbaf5581..49e2e947b36b 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -420,15 +420,15 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
}
addr_pad = dma_map_single(ss->dev, pad, j * 4, DMA_TO_DEVICE);
- rctx->t_src[i].addr = addr_pad;
- rctx->t_src[i].len = j;
- rctx->t_dst[i].addr = addr_res;
- rctx->t_dst[i].len = digestsize / 4;
if (dma_mapping_error(ss->dev, addr_pad)) {
dev_err(ss->dev, "DMA error on padding SG\n");
err = -EINVAL;
goto theend;
}
+ rctx->t_src[i].addr = addr_pad;
+ rctx->t_src[i].len = j;
+ rctx->t_dst[i].addr = addr_res;
+ rctx->t_dst[i].len = digestsize / 4;
err = sun8i_ss_run_hash_task(ss, rctx, crypto_tfm_alg_name(areq->base.tfm));
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 08/19] crypto: sun8i-ss: use sg_nents_for_len
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (6 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 07/19] crypto: sun8i-ss: test error before assigning Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 09/19] crypto: sun8i-ss: do not allocate memory when handling hash requests Corentin Labbe
` (11 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
When testing with some large SG list, the sun8i-ss drivers always
fallback even if it can handle it.
So use sg_nents_for_len() which permits to see less SGs than needed.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../allwinner/sun8i-ss/sun8i-ss-cipher.c | 23 ++++++++-----------
1 file changed, 10 insertions(+), 13 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
index 70e2e6e37389..c4cb1ab1eeaa 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
@@ -29,7 +29,8 @@ static bool sun8i_ss_need_fallback(struct skcipher_request *areq)
if (areq->cryptlen == 0 || areq->cryptlen % 16)
return true;
- if (sg_nents(areq->src) > 8 || sg_nents(areq->dst) > 8)
+ if (sg_nents_for_len(areq->src, areq->cryptlen) > 8 ||
+ sg_nents_for_len(areq->dst, areq->cryptlen) > 8)
return true;
sg = areq->src;
@@ -169,6 +170,8 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
int nr_sgs = 0;
int nr_sgd = 0;
int err = 0;
+ int nsgs = sg_nents_for_len(areq->src, areq->cryptlen);
+ int nsgd = sg_nents_for_len(areq->dst, areq->cryptlen);
int i;
algt = container_of(alg, struct sun8i_ss_alg_template, alg.skcipher);
@@ -201,8 +204,7 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
goto theend_key;
}
if (areq->src == areq->dst) {
- nr_sgs = dma_map_sg(ss->dev, areq->src, sg_nents(areq->src),
- DMA_BIDIRECTIONAL);
+ nr_sgs = dma_map_sg(ss->dev, areq->src, nsgs, DMA_BIDIRECTIONAL);
if (nr_sgs <= 0 || nr_sgs > 8) {
dev_err(ss->dev, "Invalid sg number %d\n", nr_sgs);
err = -EINVAL;
@@ -210,15 +212,13 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
}
nr_sgd = nr_sgs;
} else {
- nr_sgs = dma_map_sg(ss->dev, areq->src, sg_nents(areq->src),
- DMA_TO_DEVICE);
+ nr_sgs = dma_map_sg(ss->dev, areq->src, nsgs, DMA_TO_DEVICE);
if (nr_sgs <= 0 || nr_sgs > 8) {
dev_err(ss->dev, "Invalid sg number %d\n", nr_sgs);
err = -EINVAL;
goto theend_iv;
}
- nr_sgd = dma_map_sg(ss->dev, areq->dst, sg_nents(areq->dst),
- DMA_FROM_DEVICE);
+ nr_sgd = dma_map_sg(ss->dev, areq->dst, nsgd, DMA_FROM_DEVICE);
if (nr_sgd <= 0 || nr_sgd > 8) {
dev_err(ss->dev, "Invalid sg number %d\n", nr_sgd);
err = -EINVAL;
@@ -274,13 +274,10 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
theend_sgs:
if (areq->src == areq->dst) {
- dma_unmap_sg(ss->dev, areq->src, sg_nents(areq->src),
- DMA_BIDIRECTIONAL);
+ dma_unmap_sg(ss->dev, areq->src, nsgs, DMA_BIDIRECTIONAL);
} else {
- dma_unmap_sg(ss->dev, areq->src, sg_nents(areq->src),
- DMA_TO_DEVICE);
- dma_unmap_sg(ss->dev, areq->dst, sg_nents(areq->dst),
- DMA_FROM_DEVICE);
+ dma_unmap_sg(ss->dev, areq->src, nsgs, DMA_TO_DEVICE);
+ dma_unmap_sg(ss->dev, areq->dst, nsgd, DMA_FROM_DEVICE);
}
theend_iv:
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 09/19] crypto: sun8i-ss: do not allocate memory when handling hash requests
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (7 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 08/19] crypto: sun8i-ss: use sg_nents_for_len Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 10/19] crypto: sun8i-ss: do not zeroize all pad Corentin Labbe
` (10 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
Instead of allocate memory on each requests, it is easier to
pre-allocate buffers.
This made error path easier.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c | 10 ++++++++++
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 15 +++------------
drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h | 4 ++++
3 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
index 657530578643..786b6f5cf300 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
@@ -486,6 +486,16 @@ static int allocate_flows(struct sun8i_ss_dev *ss)
goto error_engine;
}
+ /* the padding could be up to two block. */
+ ss->flows[i].pad = devm_kmalloc(ss->dev, SHA256_BLOCK_SIZE * 2,
+ GFP_KERNEL | GFP_DMA);
+ if (!ss->flows[i].pad)
+ goto error_engine;
+ ss->flows[i].result = devm_kmalloc(ss->dev, SHA256_DIGEST_SIZE,
+ GFP_KERNEL | GFP_DMA);
+ if (!ss->flows[i].result)
+ goto error_engine;
+
ss->flows[i].engine = crypto_engine_alloc_init(ss->dev, true);
if (!ss->flows[i].engine) {
dev_err(ss->dev, "Cannot allocate engine\n");
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index 49e2e947b36b..9582ac450d08 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -332,18 +332,11 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
if (digestsize == SHA224_DIGEST_SIZE)
digestsize = SHA256_DIGEST_SIZE;
- /* the padding could be up to two block. */
- pad = kzalloc(algt->alg.hash.halg.base.cra_blocksize * 2, GFP_KERNEL | GFP_DMA);
- if (!pad)
- return -ENOMEM;
+ result = ss->flows[rctx->flow].result;
+ pad = ss->flows[rctx->flow].pad;
+ memset(pad, 0, algt->alg.hash.halg.base.cra_blocksize * 2);
bf = (__le32 *)pad;
- result = kzalloc(digestsize, GFP_KERNEL | GFP_DMA);
- if (!result) {
- kfree(pad);
- return -ENOMEM;
- }
-
for (i = 0; i < MAX_SG; i++) {
rctx->t_dst[i].addr = 0;
rctx->t_dst[i].len = 0;
@@ -439,8 +432,6 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
memcpy(areq->result, result, algt->alg.hash.halg.digestsize);
theend:
- kfree(pad);
- kfree(result);
local_bh_disable();
crypto_finalize_hash_request(engine, breq, err);
local_bh_enable();
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
index 57ada8653855..eb82ee5345ae 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
@@ -123,6 +123,8 @@ struct sginfo {
* @stat_req: number of request done by this flow
* @iv: list of IV to use for each step
* @biv: buffer which contain the backuped IV
+ * @pad: padding buffer for hash operations
+ * @result: buffer for storing the result of hash operations
*/
struct sun8i_ss_flow {
struct crypto_engine *engine;
@@ -130,6 +132,8 @@ struct sun8i_ss_flow {
int status;
u8 *iv[MAX_SG];
u8 *biv;
+ void *pad;
+ void *result;
#ifdef CONFIG_CRYPTO_DEV_SUN8I_SS_DEBUG
unsigned long stat_req;
#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 10/19] crypto: sun8i-ss: do not zeroize all pad
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (8 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 09/19] crypto: sun8i-ss: do not allocate memory when handling hash requests Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 11/19] crypto: sun8i-ss: handle requests if last block is not modulo 64 Corentin Labbe
` (9 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
Instead of memset all pad buffer, it is faster to only put 0 where
needed.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index 9582ac450d08..53e5bfb99c93 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -319,7 +319,7 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
unsigned int len;
u64 fill, min_fill, byte_count;
void *pad, *result;
- int j, i, todo;
+ int j, i, k, todo;
__be64 *bebits;
__le64 *lebits;
dma_addr_t addr_res, addr_pad;
@@ -334,7 +334,6 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
result = ss->flows[rctx->flow].result;
pad = ss->flows[rctx->flow].pad;
- memset(pad, 0, algt->alg.hash.halg.base.cra_blocksize * 2);
bf = (__le32 *)pad;
for (i = 0; i < MAX_SG; i++) {
@@ -395,7 +394,10 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
if (fill < min_fill)
fill += 64;
+ k = j;
j += (fill - min_fill) / sizeof(u32);
+ for (; k < j; k++)
+ bf[k] = 0;
switch (algt->ss_algo_id) {
case SS_ID_HASH_MD5:
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 11/19] crypto: sun8i-ss: handle requests if last block is not modulo 64
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (9 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 10/19] crypto: sun8i-ss: do not zeroize all pad Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 12/19] crypto: sun8i-ss: rework debugging Corentin Labbe
` (8 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
The current sun8i-ss handle only requests with all SG length being
modulo 64.
But the last SG could be always handled by copying it on the pad buffer.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../crypto/allwinner/sun8i-ss/sun8i-ss-core.c | 2 +-
.../crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 35 ++++++++++++++-----
drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h | 2 ++
3 files changed, 29 insertions(+), 10 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
index 786b6f5cf300..8d31fd4968f3 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
@@ -487,7 +487,7 @@ static int allocate_flows(struct sun8i_ss_dev *ss)
}
/* the padding could be up to two block. */
- ss->flows[i].pad = devm_kmalloc(ss->dev, SHA256_BLOCK_SIZE * 2,
+ ss->flows[i].pad = devm_kmalloc(ss->dev, MAX_PAD_SIZE,
GFP_KERNEL | GFP_DMA);
if (!ss->flows[i].pad)
goto error_engine;
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index 53e5bfb99c93..1b44c1a115d6 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -14,6 +14,7 @@
#include <linux/pm_runtime.h>
#include <linux/scatterlist.h>
#include <crypto/internal/hash.h>
+#include <crypto/scatterwalk.h>
#include <crypto/sha1.h>
#include <crypto/sha2.h>
#include <crypto/md5.h>
@@ -262,6 +263,9 @@ static bool sun8i_ss_hash_need_fallback(struct ahash_request *areq)
if (areq->nbytes == 0)
return true;
+ if (areq->nbytes >= MAX_PAD_SIZE - 64)
+ return true;
+
/* we need to reserve one SG for the padding one */
if (sg_nents(areq->src) > MAX_SG - 1)
return true;
@@ -270,10 +274,13 @@ static bool sun8i_ss_hash_need_fallback(struct ahash_request *areq)
/* SS can operate hash only on full block size
* since SS support only MD5,sha1,sha224 and sha256, blocksize
* is always 64
- * TODO: handle request if last SG is not len%64
- * but this will need to copy data on a new SG of size=64
*/
- if (sg->length % 64 || !IS_ALIGNED(sg->offset, sizeof(u32)))
+ /* Only the last block could be bounced to the pad buffer */
+ if (sg->length % 64 && sg_next(sg))
+ return true;
+ if (!IS_ALIGNED(sg->offset, sizeof(u32)))
+ return true;
+ if (sg->length % 4)
return true;
sg = sg_next(sg);
}
@@ -361,6 +368,7 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
goto theend;
}
+ j = 0;
len = areq->nbytes;
sg = areq->src;
i = 0;
@@ -369,12 +377,19 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
sg = sg_next(sg);
continue;
}
- rctx->t_src[i].addr = sg_dma_address(sg);
todo = min(len, sg_dma_len(sg));
- rctx->t_src[i].len = todo / 4;
- len -= todo;
- rctx->t_dst[i].addr = addr_res;
- rctx->t_dst[i].len = digestsize / 4;
+ /* only the last SG could be with a size not modulo64 */
+ if (todo % 64 == 0) {
+ rctx->t_src[i].addr = sg_dma_address(sg);
+ rctx->t_src[i].len = todo / 4;
+ rctx->t_dst[i].addr = addr_res;
+ rctx->t_dst[i].len = digestsize / 4;
+ len -= todo;
+ } else {
+ scatterwalk_map_and_copy(bf, sg, 0, todo, 0);
+ j += todo / 4;
+ len -= todo;
+ }
sg = sg_next(sg);
i++;
}
@@ -384,8 +399,10 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
goto theend;
}
+ if (j > 0)
+ i--;
+
byte_count = areq->nbytes;
- j = 0;
bf[j++] = cpu_to_le32(0x80);
fill = 64 - (byte_count % 64);
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
index eb82ee5345ae..2e3524654aca 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
@@ -82,6 +82,8 @@
#define PRNG_DATA_SIZE (160 / 8)
#define PRNG_SEED_SIZE DIV_ROUND_UP(175, 8)
+#define MAX_PAD_SIZE 4096
+
/*
* struct ss_clock - Describe clocks used by sun8i-ss
* @name: Name of clock needed by this variant
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 12/19] crypto: sun8i-ss: rework debugging
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (10 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 11/19] crypto: sun8i-ss: handle requests if last block is not modulo 64 Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:55 ` [PATCH 13/19] crypto: sun8i-ss: Add function for handling hash padding Corentin Labbe
` (7 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
The "Fallback for xxx" message is annoying, remove it and store the
information in the debugfs.
In the same time, reports more fallback statistics.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../allwinner/sun8i-ss/sun8i-ss-cipher.c | 41 ++++++++++++++-----
.../crypto/allwinner/sun8i-ss/sun8i-ss-core.c | 21 ++++++++++
.../crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 36 ++++++++++++----
drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h | 7 +++-
4 files changed, 83 insertions(+), 22 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
index c4cb1ab1eeaa..7f1940c6cc41 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
@@ -22,35 +22,54 @@
static bool sun8i_ss_need_fallback(struct skcipher_request *areq)
{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct sun8i_ss_alg_template *algt = container_of(alg, struct sun8i_ss_alg_template, alg.skcipher);
struct scatterlist *in_sg = areq->src;
struct scatterlist *out_sg = areq->dst;
struct scatterlist *sg;
- if (areq->cryptlen == 0 || areq->cryptlen % 16)
+ if (areq->cryptlen == 0 || areq->cryptlen % 16) {
+ algt->stat_fb_len++;
return true;
+ }
if (sg_nents_for_len(areq->src, areq->cryptlen) > 8 ||
- sg_nents_for_len(areq->dst, areq->cryptlen) > 8)
+ sg_nents_for_len(areq->dst, areq->cryptlen) > 8) {
+ algt->stat_fb_sgnum++;
return true;
+ }
sg = areq->src;
while (sg) {
- if ((sg->length % 16) != 0)
+ if ((sg->length % 16) != 0) {
+ algt->stat_fb_sglen++;
return true;
- if ((sg_dma_len(sg) % 16) != 0)
+ }
+ if ((sg_dma_len(sg) % 16) != 0) {
+ algt->stat_fb_sglen++;
return true;
- if (!IS_ALIGNED(sg->offset, 16))
+ }
+ if (!IS_ALIGNED(sg->offset, 16)) {
+ algt->stat_fb_align++;
return true;
+ }
sg = sg_next(sg);
}
sg = areq->dst;
while (sg) {
- if ((sg->length % 16) != 0)
+ if ((sg->length % 16) != 0) {
+ algt->stat_fb_sglen++;
return true;
- if ((sg_dma_len(sg) % 16) != 0)
+ }
+ if ((sg_dma_len(sg) % 16) != 0) {
+ algt->stat_fb_sglen++;
return true;
- if (!IS_ALIGNED(sg->offset, 16))
+ }
+ if (!IS_ALIGNED(sg->offset, 16)) {
+ algt->stat_fb_align++;
return true;
+ }
sg = sg_next(sg);
}
@@ -385,9 +404,9 @@ int sun8i_ss_cipher_init(struct crypto_tfm *tfm)
crypto_skcipher_reqsize(op->fallback_tfm);
- dev_info(op->ss->dev, "Fallback for %s is %s\n",
- crypto_tfm_alg_driver_name(&sktfm->base),
- crypto_tfm_alg_driver_name(crypto_skcipher_tfm(op->fallback_tfm)));
+ memcpy(algt->fbname,
+ crypto_tfm_alg_driver_name(crypto_skcipher_tfm(op->fallback_tfm)),
+ CRYPTO_MAX_ALG_NAME);
op->enginectx.op.do_one_request = sun8i_ss_handle_cipher_request;
op->enginectx.op.prepare_request = NULL;
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
index 8d31fd4968f3..f09de5737e8b 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
@@ -430,6 +430,17 @@ static int sun8i_ss_debugfs_show(struct seq_file *seq, void *v)
ss_algs[i].alg.skcipher.base.cra_driver_name,
ss_algs[i].alg.skcipher.base.cra_name,
ss_algs[i].stat_req, ss_algs[i].stat_fb);
+
+ seq_printf(seq, "\tLast fallback is: %s\n",
+ ss_algs[i].fbname);
+ seq_printf(seq, "\tFallback due to length: %lu\n",
+ ss_algs[i].stat_fb_len);
+ seq_printf(seq, "\tFallback due to SG length: %lu\n",
+ ss_algs[i].stat_fb_sglen);
+ seq_printf(seq, "\tFallback due to alignment: %lu\n",
+ ss_algs[i].stat_fb_align);
+ seq_printf(seq, "\tFallback due to SG numbers: %lu\n",
+ ss_algs[i].stat_fb_sgnum);
break;
case CRYPTO_ALG_TYPE_RNG:
seq_printf(seq, "%s %s reqs=%lu tsize=%lu\n",
@@ -442,6 +453,16 @@ static int sun8i_ss_debugfs_show(struct seq_file *seq, void *v)
ss_algs[i].alg.hash.halg.base.cra_driver_name,
ss_algs[i].alg.hash.halg.base.cra_name,
ss_algs[i].stat_req, ss_algs[i].stat_fb);
+ seq_printf(seq, "\tLast fallback is: %s\n",
+ ss_algs[i].fbname);
+ seq_printf(seq, "\tFallback due to length: %lu\n",
+ ss_algs[i].stat_fb_len);
+ seq_printf(seq, "\tFallback due to SG length: %lu\n",
+ ss_algs[i].stat_fb_sglen);
+ seq_printf(seq, "\tFallback due to alignment: %lu\n",
+ ss_algs[i].stat_fb_align);
+ seq_printf(seq, "\tFallback due to SG numbers: %lu\n",
+ ss_algs[i].stat_fb_sgnum);
break;
}
}
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index 1b44c1a115d6..cb510ec21ec4 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -51,9 +51,8 @@ int sun8i_ss_hash_crainit(struct crypto_tfm *tfm)
sizeof(struct sun8i_ss_hash_reqctx) +
crypto_ahash_reqsize(op->fallback_tfm));
- dev_info(op->ss->dev, "Fallback for %s is %s\n",
- crypto_tfm_alg_driver_name(tfm),
- crypto_tfm_alg_driver_name(&op->fallback_tfm->base));
+ memcpy(algt->fbname, crypto_tfm_alg_driver_name(&op->fallback_tfm->base), CRYPTO_MAX_ALG_NAME);
+
err = pm_runtime_get_sync(op->ss->dev);
if (err < 0)
goto error_pm;
@@ -259,16 +258,29 @@ static int sun8i_ss_run_hash_task(struct sun8i_ss_dev *ss,
static bool sun8i_ss_hash_need_fallback(struct ahash_request *areq)
{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
+ struct ahash_alg *alg = __crypto_ahash_alg(tfm->base.__crt_alg);
+ struct sun8i_ss_alg_template *algt;
struct scatterlist *sg;
- if (areq->nbytes == 0)
+ algt = container_of(alg, struct sun8i_ss_alg_template, alg.hash);
+
+ if (areq->nbytes == 0) {
+ algt->stat_fb_len++;
return true;
- if (areq->nbytes >= MAX_PAD_SIZE - 64)
+ }
+
+ if (areq->nbytes >= MAX_PAD_SIZE - 64) {
+ algt->stat_fb_len++;
return true;
+ }
/* we need to reserve one SG for the padding one */
- if (sg_nents(areq->src) > MAX_SG - 1)
+ if (sg_nents(areq->src) > MAX_SG - 1) {
+ algt->stat_fb_sgnum++;
return true;
+ }
+
sg = areq->src;
while (sg) {
/* SS can operate hash only on full block size
@@ -276,12 +288,18 @@ static bool sun8i_ss_hash_need_fallback(struct ahash_request *areq)
* is always 64
*/
/* Only the last block could be bounced to the pad buffer */
- if (sg->length % 64 && sg_next(sg))
+ if (sg->length % 64 && sg_next(sg)) {
+ algt->stat_fb_sglen++;
return true;
- if (!IS_ALIGNED(sg->offset, sizeof(u32)))
+ }
+ if (!IS_ALIGNED(sg->offset, sizeof(u32))) {
+ algt->stat_fb_align++;
return true;
- if (sg->length % 4)
+ }
+ if (sg->length % 4) {
+ algt->stat_fb_sglen++;
return true;
+ }
sg = sg_next(sg);
}
return false;
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
index 2e3524654aca..b56038de333b 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
@@ -279,11 +279,14 @@ struct sun8i_ss_alg_template {
struct rng_alg rng;
struct ahash_alg hash;
} alg;
-#ifdef CONFIG_CRYPTO_DEV_SUN8I_SS_DEBUG
unsigned long stat_req;
unsigned long stat_fb;
unsigned long stat_bytes;
-#endif
+ unsigned long stat_fb_len;
+ unsigned long stat_fb_sglen;
+ unsigned long stat_fb_align;
+ unsigned long stat_fb_sgnum;
+ char fbname[CRYPTO_MAX_ALG_NAME];
};
int sun8i_ss_enqueue(struct crypto_async_request *areq, u32 type);
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 13/19] crypto: sun8i-ss: Add function for handling hash padding
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (11 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 12/19] crypto: sun8i-ss: rework debugging Corentin Labbe
@ 2022-03-17 20:55 ` Corentin Labbe
2022-03-17 20:56 ` [PATCH 14/19] crypto: sun8i-ss: add hmac(sha1) Corentin Labbe
` (6 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:55 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
Move all padding work to a dedicated function.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 87 ++++++++++++++-----
1 file changed, 65 insertions(+), 22 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index cb510ec21ec4..6adb6a0428b7 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -328,6 +328,64 @@ int sun8i_ss_hash_digest(struct ahash_request *areq)
return crypto_transfer_hash_request_to_engine(engine, areq);
}
+static u64 hash_pad(u32 *buf, unsigned int bufsize, u64 padi, u64 byte_count, bool le, int bs)
+{
+ u64 fill, min_fill, j, k;
+ __be64 *bebits;
+ __le64 *lebits;
+
+ j = padi;
+ buf[j++] = cpu_to_le32(0x80);
+
+ if (bs == 64) {
+ fill = 64 - (byte_count % 64);
+ min_fill = 2 * sizeof(u32) + sizeof(u32);
+ } else {
+ fill = 128 - (byte_count % 128);
+ min_fill = 4 * sizeof(u32) + sizeof(u32);
+ }
+
+ if (fill < min_fill)
+ fill += bs;
+
+ k = j;
+ j += (fill - min_fill) / sizeof(u32);
+ if (j * 4 > bufsize) {
+ pr_err("%s OVERFLOW %llu\n", __func__, j);
+ return 0;
+ }
+ for (; k < j; k++)
+ buf[k] = 0;
+
+ if (le) {
+ /* MD5 */
+ lebits = (__le64 *)&buf[j];
+ *lebits = cpu_to_le64(byte_count << 3);
+ j += 2;
+ } else {
+ if (bs == 64) {
+ /* sha1 sha224 sha256 */
+ bebits = (__be64 *)&buf[j];
+ *bebits = cpu_to_be64(byte_count << 3);
+ j += 2;
+ } else {
+ /* sha384 sha512*/
+ bebits = (__be64 *)&buf[j];
+ *bebits = cpu_to_be64(byte_count >> 61);
+ j += 2;
+ bebits = (__be64 *)&buf[j];
+ *bebits = cpu_to_be64(byte_count << 3);
+ j += 2;
+ }
+ }
+ if (j * 4 > bufsize) {
+ pr_err("%s OVERFLOW %llu\n", __func__, j);
+ return 0;
+ }
+
+ return j;
+}
+
/* sun8i_ss_hash_run - run an ahash request
* Send the data of the request to the SS along with an extra SG with padding
*/
@@ -342,11 +400,9 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
struct scatterlist *sg;
int nr_sgs, err, digestsize;
unsigned int len;
- u64 fill, min_fill, byte_count;
+ u64 byte_count;
void *pad, *result;
int j, i, k, todo;
- __be64 *bebits;
- __le64 *lebits;
dma_addr_t addr_res, addr_pad;
__le32 *bf;
@@ -421,33 +477,20 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
i--;
byte_count = areq->nbytes;
- bf[j++] = cpu_to_le32(0x80);
-
- fill = 64 - (byte_count % 64);
- min_fill = 3 * sizeof(u32);
-
- if (fill < min_fill)
- fill += 64;
-
- k = j;
- j += (fill - min_fill) / sizeof(u32);
- for (; k < j; k++)
- bf[k] = 0;
-
switch (algt->ss_algo_id) {
case SS_ID_HASH_MD5:
- lebits = (__le64 *)&bf[j];
- *lebits = cpu_to_le64(byte_count << 3);
- j += 2;
+ j = hash_pad(bf, 4096, j, byte_count, true, bs);
break;
case SS_ID_HASH_SHA1:
case SS_ID_HASH_SHA224:
case SS_ID_HASH_SHA256:
- bebits = (__be64 *)&bf[j];
- *bebits = cpu_to_be64(byte_count << 3);
- j += 2;
+ j = hash_pad(bf, 4096, j, byte_count, false, bs);
break;
}
+ if (!j) {
+ err = -EINVAL;
+ goto theend;
+ }
addr_pad = dma_map_single(ss->dev, pad, j * 4, DMA_TO_DEVICE);
if (dma_mapping_error(ss->dev, addr_pad)) {
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 14/19] crypto: sun8i-ss: add hmac(sha1)
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (12 preceding siblings ...)
2022-03-17 20:55 ` [PATCH 13/19] crypto: sun8i-ss: Add function for handling hash padding Corentin Labbe
@ 2022-03-17 20:56 ` Corentin Labbe
2022-03-17 20:56 ` [PATCH 15/19] crypto: sun8i-ss: do not fallback if cryptlen is less than sg length Corentin Labbe
` (5 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:56 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
Even if sun8i-ss does not handle hmac(sha1) directly, we can provide one
which use the already supported acceleration of sha1.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../crypto/allwinner/sun8i-ss/sun8i-ss-core.c | 31 +++
.../crypto/allwinner/sun8i-ss/sun8i-ss-hash.c | 200 +++++++++++++++++-
drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h | 6 +
3 files changed, 231 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
index f09de5737e8b..98593a0cff69 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
@@ -409,6 +409,37 @@ static struct sun8i_ss_alg_template ss_algs[] = {
}
}
},
+{ .type = CRYPTO_ALG_TYPE_AHASH,
+ .ss_algo_id = SS_ID_HASH_SHA1,
+ .alg.hash = {
+ .init = sun8i_ss_hash_init,
+ .update = sun8i_ss_hash_update,
+ .final = sun8i_ss_hash_final,
+ .finup = sun8i_ss_hash_finup,
+ .digest = sun8i_ss_hash_digest,
+ .export = sun8i_ss_hash_export,
+ .import = sun8i_ss_hash_import,
+ .setkey = sun8i_ss_hmac_setkey,
+ .halg = {
+ .digestsize = SHA1_DIGEST_SIZE,
+ .statesize = sizeof(struct sha1_state),
+ .base = {
+ .cra_name = "hmac(sha1)",
+ .cra_driver_name = "hmac-sha1-sun8i-ss",
+ .cra_priority = 300,
+ .cra_alignmask = 3,
+ .cra_flags = CRYPTO_ALG_TYPE_AHASH |
+ CRYPTO_ALG_ASYNC |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_blocksize = SHA1_BLOCK_SIZE,
+ .cra_ctxsize = sizeof(struct sun8i_ss_hash_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_init = sun8i_ss_hash_crainit,
+ .cra_exit = sun8i_ss_hash_craexit,
+ }
+ }
+ }
+},
#endif
};
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index 6adb6a0428b7..512cf52cd240 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -14,12 +14,99 @@
#include <linux/pm_runtime.h>
#include <linux/scatterlist.h>
#include <crypto/internal/hash.h>
+#include <crypto/hmac.h>
#include <crypto/scatterwalk.h>
#include <crypto/sha1.h>
#include <crypto/sha2.h>
#include <crypto/md5.h>
#include "sun8i-ss.h"
+static int sun8i_ss_hashkey(struct sun8i_ss_hash_tfm_ctx *tfmctx, const u8 *key,
+ unsigned int keylen)
+{
+ struct crypto_shash *xtfm;
+ struct shash_desc *sdesc;
+ size_t len;
+ int ret = 0;
+
+ xtfm = crypto_alloc_shash("sha1", 0, CRYPTO_ALG_NEED_FALLBACK);
+ if (!xtfm)
+ return -ENOMEM;
+
+ len = sizeof(*sdesc) + crypto_shash_descsize(xtfm);
+ sdesc = kmalloc(len, GFP_KERNEL);
+ if (!sdesc) {
+ ret = -ENOMEM;
+ goto err_hashkey_sdesc;
+ }
+ sdesc->tfm = xtfm;
+
+ ret = crypto_shash_init(sdesc);
+ if (ret) {
+ dev_err(tfmctx->ss->dev, "shash init error ret=%d\n", ret);
+ goto err_hashkey;
+ }
+ ret = crypto_shash_finup(sdesc, key, keylen, tfmctx->key);
+ if (ret)
+ dev_err(tfmctx->ss->dev, "shash finup error\n");
+err_hashkey:
+ kfree(sdesc);
+err_hashkey_sdesc:
+ crypto_free_shash(xtfm);
+ return ret;
+}
+
+int sun8i_ss_hmac_setkey(struct crypto_ahash *ahash, const u8 *key,
+ unsigned int keylen)
+{
+ struct sun8i_ss_hash_tfm_ctx *tfmctx = crypto_ahash_ctx(ahash);
+ struct ahash_alg *alg = __crypto_ahash_alg(ahash->base.__crt_alg);
+ struct sun8i_ss_alg_template *algt;
+ int digestsize, i;
+ int bs = crypto_ahash_blocksize(ahash);
+ int ret;
+
+ algt = container_of(alg, struct sun8i_ss_alg_template, alg.hash);
+ digestsize = algt->alg.hash.halg.digestsize;
+
+ if (keylen > bs) {
+ ret = sun8i_ss_hashkey(tfmctx, key, keylen);
+ if (ret)
+ return ret;
+ tfmctx->keylen = digestsize;
+ } else {
+ tfmctx->keylen = keylen;
+ memcpy(tfmctx->key, key, keylen);
+ }
+
+ tfmctx->ipad = kzalloc(bs, GFP_KERNEL | GFP_DMA);
+ if (!tfmctx->ipad)
+ return -ENOMEM;
+ tfmctx->opad = kzalloc(bs, GFP_KERNEL | GFP_DMA);
+ if (!tfmctx->opad) {
+ ret = -ENOMEM;
+ goto err_opad;
+ }
+
+ memset(tfmctx->key + tfmctx->keylen, 0, bs - tfmctx->keylen);
+ memcpy(tfmctx->ipad, tfmctx->key, tfmctx->keylen);
+ memcpy(tfmctx->opad, tfmctx->key, tfmctx->keylen);
+ for (i = 0; i < bs; i++) {
+ tfmctx->ipad[i] ^= HMAC_IPAD_VALUE;
+ tfmctx->opad[i] ^= HMAC_OPAD_VALUE;
+ }
+
+ ret = crypto_ahash_setkey(tfmctx->fallback_tfm, key, keylen);
+ if (!ret)
+ return 0;
+
+ memzero_explicit(tfmctx->key, keylen);
+ kfree_sensitive(tfmctx->opad);
+err_opad:
+ kfree_sensitive(tfmctx->ipad);
+ return ret;
+}
+
int sun8i_ss_hash_crainit(struct crypto_tfm *tfm)
{
struct sun8i_ss_hash_tfm_ctx *op = crypto_tfm_ctx(tfm);
@@ -67,6 +154,9 @@ void sun8i_ss_hash_craexit(struct crypto_tfm *tfm)
{
struct sun8i_ss_hash_tfm_ctx *tfmctx = crypto_tfm_ctx(tfm);
+ kfree_sensitive(tfmctx->ipad);
+ kfree_sensitive(tfmctx->opad);
+
crypto_free_ahash(tfmctx->fallback_tfm);
pm_runtime_put_sync_suspend(tfmctx->ss->dev);
}
@@ -393,18 +483,26 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
{
struct ahash_request *areq = container_of(breq, struct ahash_request, base);
struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
+ struct sun8i_ss_hash_tfm_ctx *tfmctx = crypto_ahash_ctx(tfm);
struct ahash_alg *alg = __crypto_ahash_alg(tfm->base.__crt_alg);
struct sun8i_ss_hash_reqctx *rctx = ahash_request_ctx(areq);
struct sun8i_ss_alg_template *algt;
struct sun8i_ss_dev *ss;
struct scatterlist *sg;
+ int bs = crypto_ahash_blocksize(tfm);
int nr_sgs, err, digestsize;
unsigned int len;
u64 byte_count;
void *pad, *result;
int j, i, k, todo;
- dma_addr_t addr_res, addr_pad;
+ dma_addr_t addr_res, addr_pad, addr_xpad;
__le32 *bf;
+ /* HMAC step:
+ * 0: normal hashing
+ * 1: IPAD
+ * 2: OPAD
+ */
+ int hmac = 0;
algt = container_of(alg, struct sun8i_ss_alg_template, alg.hash);
ss = algt->ss;
@@ -439,7 +537,7 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
if (dma_mapping_error(ss->dev, addr_res)) {
dev_err(ss->dev, "DMA map dest\n");
err = -EINVAL;
- goto theend;
+ goto err_dma_result;
}
j = 0;
@@ -476,7 +574,60 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
if (j > 0)
i--;
+retry:
byte_count = areq->nbytes;
+ if (tfmctx->keylen && hmac == 0) {
+ hmac = 1;
+ /* shift all SG one slot up, to free slot 0 for IPAD */
+ for (k = 6; k >= 0; k--) {
+ rctx->t_src[k + 1].addr = rctx->t_src[k].addr;
+ rctx->t_src[k + 1].len = rctx->t_src[k].len;
+ rctx->t_dst[k + 1].addr = rctx->t_dst[k].addr;
+ rctx->t_dst[k + 1].len = rctx->t_dst[k].len;
+ }
+ addr_xpad = dma_map_single(ss->dev, tfmctx->ipad, bs, DMA_TO_DEVICE);
+ if (dma_mapping_error(ss->dev, addr_xpad)) {
+ dev_err(ss->dev, "Fail to create DMA mapping of ipad\n");
+ goto err_dma_xpad;
+ }
+ rctx->t_src[0].addr = addr_xpad;
+ rctx->t_src[0].len = bs / 4;
+ rctx->t_dst[0].addr = addr_res;
+ rctx->t_dst[0].len = digestsize / 4;
+ i++;
+ byte_count = areq->nbytes + bs;
+ }
+ if (tfmctx->keylen && hmac == 2) {
+ for (i = 0; i < MAX_SG; i++) {
+ rctx->t_src[i].addr = 0;
+ rctx->t_src[i].len = 0;
+ rctx->t_dst[i].addr = 0;
+ rctx->t_dst[i].len = 0;
+ }
+
+ addr_res = dma_map_single(ss->dev, result, digestsize, DMA_FROM_DEVICE);
+ if (dma_mapping_error(ss->dev, addr_res)) {
+ dev_err(ss->dev, "Fail to create DMA mapping of result\n");
+ err = -EINVAL;
+ goto err_dma_result;
+ }
+ addr_xpad = dma_map_single(ss->dev, tfmctx->opad, bs, DMA_TO_DEVICE);
+ if (dma_mapping_error(ss->dev, addr_xpad)) {
+ dev_err(ss->dev, "Fail to create DMA mapping of opad\n");
+ goto err_dma_xpad;
+ }
+ rctx->t_src[0].addr = addr_xpad;
+ rctx->t_src[0].len = bs / 4;
+
+ memcpy(bf, result, digestsize);
+ j = digestsize / 4;
+ i = 1;
+ byte_count = digestsize + bs;
+
+ rctx->t_dst[0].addr = addr_res;
+ rctx->t_dst[0].len = digestsize / 4;
+ }
+
switch (algt->ss_algo_id) {
case SS_ID_HASH_MD5:
j = hash_pad(bf, 4096, j, byte_count, true, bs);
@@ -496,7 +647,7 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
if (dma_mapping_error(ss->dev, addr_pad)) {
dev_err(ss->dev, "DMA error on padding SG\n");
err = -EINVAL;
- goto theend;
+ goto err_dma_pad;
}
rctx->t_src[i].addr = addr_pad;
rctx->t_src[i].len = j;
@@ -505,12 +656,49 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
err = sun8i_ss_run_hash_task(ss, rctx, crypto_tfm_alg_name(areq->base.tfm));
+ /*
+ * mini helper for checking dma map/unmap
+ * flow start for hmac = 0 (and HMAC = 1)
+ * HMAC = 0
+ * MAP src
+ * MAP res
+ *
+ * retry:
+ * if hmac then hmac = 1
+ * MAP xpad (ipad)
+ * if hmac == 2
+ * MAP res
+ * MAP xpad (opad)
+ * MAP pad
+ * ACTION!
+ * UNMAP pad
+ * if hmac
+ * UNMAP xpad
+ * UNMAP res
+ * if hmac < 2
+ * UNMAP SRC
+ *
+ * if hmac = 1 then hmac = 2 goto retry
+ */
+
dma_unmap_single(ss->dev, addr_pad, j * 4, DMA_TO_DEVICE);
- dma_unmap_sg(ss->dev, areq->src, sg_nents(areq->src),
- DMA_TO_DEVICE);
+
+err_dma_pad:
+ if (hmac > 0)
+ dma_unmap_single(ss->dev, addr_xpad, bs, DMA_TO_DEVICE);
+err_dma_xpad:
dma_unmap_single(ss->dev, addr_res, digestsize, DMA_FROM_DEVICE);
+err_dma_result:
+ if (hmac < 2)
+ dma_unmap_sg(ss->dev, areq->src, sg_nents(areq->src),
+ DMA_TO_DEVICE);
+ if (hmac == 1 && !err) {
+ hmac = 2;
+ goto retry;
+ }
- memcpy(areq->result, result, algt->alg.hash.halg.digestsize);
+ if (!err)
+ memcpy(areq->result, result, algt->alg.hash.halg.digestsize);
theend:
local_bh_disable();
crypto_finalize_hash_request(engine, breq, err);
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
index b56038de333b..df6f08f6092f 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h
@@ -239,6 +239,10 @@ struct sun8i_ss_hash_tfm_ctx {
struct crypto_engine_ctx enginectx;
struct crypto_ahash *fallback_tfm;
struct sun8i_ss_dev *ss;
+ u8 *ipad;
+ u8 *opad;
+ u8 key[SHA256_BLOCK_SIZE];
+ int keylen;
};
/*
@@ -319,3 +323,5 @@ int sun8i_ss_hash_update(struct ahash_request *areq);
int sun8i_ss_hash_finup(struct ahash_request *areq);
int sun8i_ss_hash_digest(struct ahash_request *areq);
int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq);
+int sun8i_ss_hmac_setkey(struct crypto_ahash *ahash, const u8 *key,
+ unsigned int keylen);
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 15/19] crypto: sun8i-ss: do not fallback if cryptlen is less than sg length
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (13 preceding siblings ...)
2022-03-17 20:56 ` [PATCH 14/19] crypto: sun8i-ss: add hmac(sha1) Corentin Labbe
@ 2022-03-17 20:56 ` Corentin Labbe
2022-03-17 20:56 ` [PATCH 16/19] crypto: sun8i-ce: Add function for handling hash padding Corentin Labbe
` (4 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:56 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
The sg length could be more than remaining data on it.
So check the length requirement against the minimum between those two
values.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../allwinner/sun8i-ss/sun8i-ss-cipher.c | 19 +++++++++----------
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
index 7f1940c6cc41..5bb950182026 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
@@ -28,6 +28,7 @@ static bool sun8i_ss_need_fallback(struct skcipher_request *areq)
struct scatterlist *in_sg = areq->src;
struct scatterlist *out_sg = areq->dst;
struct scatterlist *sg;
+ unsigned int todo, len;
if (areq->cryptlen == 0 || areq->cryptlen % 16) {
algt->stat_fb_len++;
@@ -40,13 +41,11 @@ static bool sun8i_ss_need_fallback(struct skcipher_request *areq)
return true;
}
+ len = areq->cryptlen;
sg = areq->src;
while (sg) {
- if ((sg->length % 16) != 0) {
- algt->stat_fb_sglen++;
- return true;
- }
- if ((sg_dma_len(sg) % 16) != 0) {
+ todo = min(len, sg->length);
+ if ((todo % 16) != 0) {
algt->stat_fb_sglen++;
return true;
}
@@ -54,15 +53,14 @@ static bool sun8i_ss_need_fallback(struct skcipher_request *areq)
algt->stat_fb_align++;
return true;
}
+ len -= todo;
sg = sg_next(sg);
}
+ len = areq->cryptlen;
sg = areq->dst;
while (sg) {
- if ((sg->length % 16) != 0) {
- algt->stat_fb_sglen++;
- return true;
- }
- if ((sg_dma_len(sg) % 16) != 0) {
+ todo = min(len, sg->length);
+ if ((todo % 16) != 0) {
algt->stat_fb_sglen++;
return true;
}
@@ -70,6 +68,7 @@ static bool sun8i_ss_need_fallback(struct skcipher_request *areq)
algt->stat_fb_align++;
return true;
}
+ len -= todo;
sg = sg_next(sg);
}
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 16/19] crypto: sun8i-ce: Add function for handling hash padding
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (14 preceding siblings ...)
2022-03-17 20:56 ` [PATCH 15/19] crypto: sun8i-ss: do not fallback if cryptlen is less than sg length Corentin Labbe
@ 2022-03-17 20:56 ` Corentin Labbe
2022-03-17 20:56 ` [PATCH 17/19] crypto: sun8i-ce: use sg_nents_for_len Corentin Labbe
` (3 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:56 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
Move all padding work to a dedicated function.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../crypto/allwinner/sun8i-ce/sun8i-ce-hash.c | 95 +++++++++++++------
1 file changed, 65 insertions(+), 30 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
index 859b7522faaa..820651f0387f 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
@@ -248,6 +248,64 @@ int sun8i_ce_hash_digest(struct ahash_request *areq)
return crypto_transfer_hash_request_to_engine(engine, areq);
}
+static u64 hash_pad(u32 *buf, unsigned int bufsize, u64 padi, u64 byte_count, bool le, int bs)
+{
+ u64 fill, min_fill, j, k;
+ __be64 *bebits;
+ __le64 *lebits;
+
+ j = padi;
+ buf[j++] = cpu_to_le32(0x80);
+
+ if (bs == 64) {
+ fill = 64 - (byte_count % 64);
+ min_fill = 2 * sizeof(u32) + sizeof(u32);
+ } else {
+ fill = 128 - (byte_count % 128);
+ min_fill = 4 * sizeof(u32) + sizeof(u32);
+ }
+
+ if (fill < min_fill)
+ fill += bs;
+
+ k = j;
+ j += (fill - min_fill) / sizeof(u32);
+ if (j * 4 > bufsize) {
+ pr_err("%s OVERFLOW %llu\n", __func__, j);
+ return 0;
+ }
+ for (; k < j; k++)
+ buf[k] = 0;
+
+ if (le) {
+ /* MD5 */
+ lebits = (__le64 *)&buf[j];
+ *lebits = cpu_to_le64(byte_count << 3);
+ j += 2;
+ } else {
+ if (bs == 64) {
+ /* sha1 sha224 sha256 */
+ bebits = (__be64 *)&buf[j];
+ *bebits = cpu_to_be64(byte_count << 3);
+ j += 2;
+ } else {
+ /* sha384 sha512*/
+ bebits = (__be64 *)&buf[j];
+ *bebits = cpu_to_be64(byte_count >> 61);
+ j += 2;
+ bebits = (__be64 *)&buf[j];
+ *bebits = cpu_to_be64(byte_count << 3);
+ j += 2;
+ }
+ }
+ if (j * 4 > bufsize) {
+ pr_err("%s OVERFLOW %llu\n", __func__, j);
+ return 0;
+ }
+
+ return j;
+}
+
int sun8i_ce_hash_run(struct crypto_engine *engine, void *breq)
{
struct ahash_request *areq = container_of(breq, struct ahash_request, base);
@@ -266,10 +324,6 @@ int sun8i_ce_hash_run(struct crypto_engine *engine, void *breq)
__le32 *bf;
void *buf = NULL;
int j, i, todo;
- int nbw = 0;
- u64 fill, min_fill;
- __be64 *bebits;
- __le64 *lebits;
void *result = NULL;
u64 bs;
int digestsize;
@@ -348,44 +402,25 @@ int sun8i_ce_hash_run(struct crypto_engine *engine, void *breq)
byte_count = areq->nbytes;
j = 0;
- bf[j++] = cpu_to_le32(0x80);
-
- if (bs == 64) {
- fill = 64 - (byte_count % 64);
- min_fill = 2 * sizeof(u32) + (nbw ? 0 : sizeof(u32));
- } else {
- fill = 128 - (byte_count % 128);
- min_fill = 4 * sizeof(u32) + (nbw ? 0 : sizeof(u32));
- }
-
- if (fill < min_fill)
- fill += bs;
-
- j += (fill - min_fill) / sizeof(u32);
switch (algt->ce_algo_id) {
case CE_ID_HASH_MD5:
- lebits = (__le64 *)&bf[j];
- *lebits = cpu_to_le64(byte_count << 3);
- j += 2;
+ j = hash_pad(bf, 2 * bs, j, byte_count, true, bs);
break;
case CE_ID_HASH_SHA1:
case CE_ID_HASH_SHA224:
case CE_ID_HASH_SHA256:
- bebits = (__be64 *)&bf[j];
- *bebits = cpu_to_be64(byte_count << 3);
- j += 2;
+ j = hash_pad(bf, 2 * bs, j, byte_count, false, bs);
break;
case CE_ID_HASH_SHA384:
case CE_ID_HASH_SHA512:
- bebits = (__be64 *)&bf[j];
- *bebits = cpu_to_be64(byte_count >> 61);
- j += 2;
- bebits = (__be64 *)&bf[j];
- *bebits = cpu_to_be64(byte_count << 3);
- j += 2;
+ j = hash_pad(bf, 2 * bs, j, byte_count, false, bs);
break;
}
+ if (!j) {
+ err = -EINVAL;
+ goto theend;
+ }
addr_pad = dma_map_single(ce->dev, buf, j * 4, DMA_TO_DEVICE);
cet->t_src[i].addr = cpu_to_le32(addr_pad);
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 17/19] crypto: sun8i-ce: use sg_nents_for_len
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (15 preceding siblings ...)
2022-03-17 20:56 ` [PATCH 16/19] crypto: sun8i-ce: Add function for handling hash padding Corentin Labbe
@ 2022-03-17 20:56 ` Corentin Labbe
2022-03-17 20:56 ` [PATCH 18/19] crypto: sun8i-ce: rework debugging Corentin Labbe
` (2 subsequent siblings)
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:56 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
When testing with some large SG list, the sun8i-ce drivers always
fallback even if it can handle it.
So use sg_nents_for_len() which permits to see less SGs than needed.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../allwinner/sun8i-ce/sun8i-ce-cipher.c | 23 ++++++++-----------
.../crypto/allwinner/sun8i-ce/sun8i-ce-hash.c | 10 ++++----
2 files changed, 15 insertions(+), 18 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
index 0b1ce58bdeb9..35ab71d3a82d 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
@@ -26,7 +26,8 @@ static int sun8i_ce_cipher_need_fallback(struct skcipher_request *areq)
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
struct scatterlist *sg;
- if (sg_nents(areq->src) > MAX_SG || sg_nents(areq->dst) > MAX_SG)
+ if (sg_nents_for_len(areq->src, areq->cryptlen) > MAX_SG ||
+ sg_nents_for_len(areq->dst, areq->cryptlen) > MAX_SG)
return true;
if (areq->cryptlen < crypto_skcipher_ivsize(tfm))
@@ -94,6 +95,8 @@ static int sun8i_ce_cipher_prepare(struct crypto_engine *engine, void *async_req
int nr_sgs = 0;
int nr_sgd = 0;
int err = 0;
+ int ns = sg_nents_for_len(areq->src, areq->cryptlen);
+ int nd = sg_nents_for_len(areq->dst, areq->cryptlen);
algt = container_of(alg, struct sun8i_ce_alg_template, alg.skcipher);
@@ -169,8 +172,7 @@ static int sun8i_ce_cipher_prepare(struct crypto_engine *engine, void *async_req
}
if (areq->src == areq->dst) {
- nr_sgs = dma_map_sg(ce->dev, areq->src, sg_nents(areq->src),
- DMA_BIDIRECTIONAL);
+ nr_sgs = dma_map_sg(ce->dev, areq->src, ns, DMA_BIDIRECTIONAL);
if (nr_sgs <= 0 || nr_sgs > MAX_SG) {
dev_err(ce->dev, "Invalid sg number %d\n", nr_sgs);
err = -EINVAL;
@@ -178,15 +180,13 @@ static int sun8i_ce_cipher_prepare(struct crypto_engine *engine, void *async_req
}
nr_sgd = nr_sgs;
} else {
- nr_sgs = dma_map_sg(ce->dev, areq->src, sg_nents(areq->src),
- DMA_TO_DEVICE);
+ nr_sgs = dma_map_sg(ce->dev, areq->src, ns, DMA_TO_DEVICE);
if (nr_sgs <= 0 || nr_sgs > MAX_SG) {
dev_err(ce->dev, "Invalid sg number %d\n", nr_sgs);
err = -EINVAL;
goto theend_iv;
}
- nr_sgd = dma_map_sg(ce->dev, areq->dst, sg_nents(areq->dst),
- DMA_FROM_DEVICE);
+ nr_sgd = dma_map_sg(ce->dev, areq->dst, nd, DMA_FROM_DEVICE);
if (nr_sgd <= 0 || nr_sgd > MAX_SG) {
dev_err(ce->dev, "Invalid sg number %d\n", nr_sgd);
err = -EINVAL;
@@ -231,14 +231,11 @@ static int sun8i_ce_cipher_prepare(struct crypto_engine *engine, void *async_req
theend_sgs:
if (areq->src == areq->dst) {
- dma_unmap_sg(ce->dev, areq->src, sg_nents(areq->src),
- DMA_BIDIRECTIONAL);
+ dma_unmap_sg(ce->dev, areq->src, ns, DMA_BIDIRECTIONAL);
} else {
if (nr_sgs > 0)
- dma_unmap_sg(ce->dev, areq->src, sg_nents(areq->src),
- DMA_TO_DEVICE);
- dma_unmap_sg(ce->dev, areq->dst, sg_nents(areq->dst),
- DMA_FROM_DEVICE);
+ dma_unmap_sg(ce->dev, areq->src, ns, DMA_TO_DEVICE);
+ dma_unmap_sg(ce->dev, areq->dst, nd, DMA_FROM_DEVICE);
}
theend_iv:
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
index 820651f0387f..17f6885dbe35 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
@@ -204,7 +204,7 @@ static bool sun8i_ce_hash_need_fallback(struct ahash_request *areq)
if (areq->nbytes == 0)
return true;
/* we need to reserve one SG for padding one */
- if (sg_nents(areq->src) > MAX_SG - 1)
+ if (sg_nents_for_len(areq->src, areq->nbytes) > MAX_SG - 1)
return true;
sg = areq->src;
while (sg) {
@@ -229,7 +229,7 @@ int sun8i_ce_hash_digest(struct ahash_request *areq)
if (sun8i_ce_hash_need_fallback(areq))
return sun8i_ce_hash_digest_fb(areq);
- nr_sgs = sg_nents(areq->src);
+ nr_sgs = sg_nents_for_len(areq->src, areq->nbytes);
if (nr_sgs > MAX_SG - 1)
return sun8i_ce_hash_digest_fb(areq);
@@ -328,6 +328,7 @@ int sun8i_ce_hash_run(struct crypto_engine *engine, void *breq)
u64 bs;
int digestsize;
dma_addr_t addr_res, addr_pad;
+ int ns = sg_nents_for_len(areq->src, areq->nbytes);
algt = container_of(alg, struct sun8i_ce_alg_template, alg.hash);
ce = algt->ce;
@@ -372,7 +373,7 @@ int sun8i_ce_hash_run(struct crypto_engine *engine, void *breq)
cet->t_sym_ctl = 0;
cet->t_asym_ctl = 0;
- nr_sgs = dma_map_sg(ce->dev, areq->src, sg_nents(areq->src), DMA_TO_DEVICE);
+ nr_sgs = dma_map_sg(ce->dev, areq->src, ns, DMA_TO_DEVICE);
if (nr_sgs <= 0 || nr_sgs > MAX_SG) {
dev_err(ce->dev, "Invalid sg number %d\n", nr_sgs);
err = -EINVAL;
@@ -441,8 +442,7 @@ int sun8i_ce_hash_run(struct crypto_engine *engine, void *breq)
err = sun8i_ce_run_task(ce, flow, crypto_tfm_alg_name(areq->base.tfm));
dma_unmap_single(ce->dev, addr_pad, j * 4, DMA_TO_DEVICE);
- dma_unmap_sg(ce->dev, areq->src, sg_nents(areq->src),
- DMA_TO_DEVICE);
+ dma_unmap_sg(ce->dev, areq->src, ns, DMA_TO_DEVICE);
dma_unmap_single(ce->dev, addr_res, digestsize, DMA_FROM_DEVICE);
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 18/19] crypto: sun8i-ce: rework debugging
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (16 preceding siblings ...)
2022-03-17 20:56 ` [PATCH 17/19] crypto: sun8i-ce: use sg_nents_for_len Corentin Labbe
@ 2022-03-17 20:56 ` Corentin Labbe
2022-03-17 20:56 ` [PATCH 19/19] crypto: sun8i-ce: do not fallback if cryptlen is less than sg length Corentin Labbe
2022-04-08 8:24 ` [PATCH 00/19] crypto: allwinner: lots of fixes Herbert Xu
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:56 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
The "Fallback for xxx" message is annoying, remove it and store the
information in the debugfs.
Let's add more precise fallback stats and display it better.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
.../allwinner/sun8i-ce/sun8i-ce-cipher.c | 43 +++++++++++++++----
.../crypto/allwinner/sun8i-ce/sun8i-ce-core.c | 34 +++++++++++++--
.../crypto/allwinner/sun8i-ce/sun8i-ce-hash.c | 27 +++++++++---
drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h | 11 ++++-
4 files changed, 96 insertions(+), 19 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
index 35ab71d3a82d..315a62e424d6 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
@@ -25,27 +25,54 @@ static int sun8i_ce_cipher_need_fallback(struct skcipher_request *areq)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
struct scatterlist *sg;
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct sun8i_ce_alg_template *algt;
+
+ algt = container_of(alg, struct sun8i_ce_alg_template, alg.skcipher);
if (sg_nents_for_len(areq->src, areq->cryptlen) > MAX_SG ||
- sg_nents_for_len(areq->dst, areq->cryptlen) > MAX_SG)
+ sg_nents_for_len(areq->dst, areq->cryptlen) > MAX_SG) {
+ algt->stat_fb_maxsg++;
return true;
+ }
+
+ if (areq->cryptlen < crypto_skcipher_ivsize(tfm)) {
+ algt->stat_fb_leniv++;
+ return true;
+ }
- if (areq->cryptlen < crypto_skcipher_ivsize(tfm))
+ if (areq->cryptlen == 0) {
+ algt->stat_fb_len0++;
return true;
+ }
- if (areq->cryptlen == 0 || areq->cryptlen % 16)
+ if (areq->cryptlen % 16) {
+ algt->stat_fb_mod16++;
return true;
+ }
sg = areq->src;
while (sg) {
- if (sg->length % 4 || !IS_ALIGNED(sg->offset, sizeof(u32)))
+ if (!IS_ALIGNED(sg->offset, sizeof(u32))) {
+ algt->stat_fb_srcali++;
+ return true;
+ }
+ if (sg->length % 4) {
+ algt->stat_fb_srclen++;
return true;
+ }
sg = sg_next(sg);
}
sg = areq->dst;
while (sg) {
- if (sg->length % 4 || !IS_ALIGNED(sg->offset, sizeof(u32)))
+ if (!IS_ALIGNED(sg->offset, sizeof(u32))) {
+ algt->stat_fb_dstali++;
+ return true;
+ }
+ if (sg->length % 4) {
+ algt->stat_fb_dstlen++;
return true;
+ }
sg = sg_next(sg);
}
return false;
@@ -384,9 +411,9 @@ int sun8i_ce_cipher_init(struct crypto_tfm *tfm)
sktfm->reqsize = sizeof(struct sun8i_cipher_req_ctx) +
crypto_skcipher_reqsize(op->fallback_tfm);
- dev_info(op->ce->dev, "Fallback for %s is %s\n",
- crypto_tfm_alg_driver_name(&sktfm->base),
- crypto_tfm_alg_driver_name(crypto_skcipher_tfm(op->fallback_tfm)));
+ memcpy(algt->fbname,
+ crypto_tfm_alg_driver_name(crypto_skcipher_tfm(op->fallback_tfm)),
+ CRYPTO_MAX_ALG_NAME);
op->enginectx.op.do_one_request = sun8i_ce_cipher_run;
op->enginectx.op.prepare_request = sun8i_ce_cipher_prepare;
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
index 8f902607af68..7f608e1e6bde 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
@@ -644,19 +644,47 @@ static int sun8i_ce_debugfs_show(struct seq_file *seq, void *v)
continue;
switch (ce_algs[i].type) {
case CRYPTO_ALG_TYPE_SKCIPHER:
- seq_printf(seq, "%s %s %lu %lu\n",
+ seq_printf(seq, "%s %s reqs=%lu fallback=%lu\n",
ce_algs[i].alg.skcipher.base.cra_driver_name,
ce_algs[i].alg.skcipher.base.cra_name,
ce_algs[i].stat_req, ce_algs[i].stat_fb);
+ seq_printf(seq, "\tLast fallback is: %s\n",
+ ce_algs[i].fbname);
+ seq_printf(seq, "\tFallback due to 0 length: %lu\n",
+ ce_algs[i].stat_fb_len0);
+ seq_printf(seq, "\tFallback due to length !mod16: %lu\n",
+ ce_algs[i].stat_fb_mod16);
+ seq_printf(seq, "\tFallback due to length < IV: %lu\n",
+ ce_algs[i].stat_fb_leniv);
+ seq_printf(seq, "\tFallback due to source alignment: %lu\n",
+ ce_algs[i].stat_fb_srcali);
+ seq_printf(seq, "\tFallback due to dest alignment: %lu\n",
+ ce_algs[i].stat_fb_dstali);
+ seq_printf(seq, "\tFallback due to source length: %lu\n",
+ ce_algs[i].stat_fb_srclen);
+ seq_printf(seq, "\tFallback due to dest length: %lu\n",
+ ce_algs[i].stat_fb_dstlen);
+ seq_printf(seq, "\tFallback due to SG numbers: %lu\n",
+ ce_algs[i].stat_fb_maxsg);
break;
case CRYPTO_ALG_TYPE_AHASH:
- seq_printf(seq, "%s %s %lu %lu\n",
+ seq_printf(seq, "%s %s reqs=%lu fallback=%lu\n",
ce_algs[i].alg.hash.halg.base.cra_driver_name,
ce_algs[i].alg.hash.halg.base.cra_name,
ce_algs[i].stat_req, ce_algs[i].stat_fb);
+ seq_printf(seq, "\tLast fallback is: %s\n",
+ ce_algs[i].fbname);
+ seq_printf(seq, "\tFallback due to 0 length: %lu\n",
+ ce_algs[i].stat_fb_len0);
+ seq_printf(seq, "\tFallback due to length: %lu\n",
+ ce_algs[i].stat_fb_srclen);
+ seq_printf(seq, "\tFallback due to alignment: %lu\n",
+ ce_algs[i].stat_fb_srcali);
+ seq_printf(seq, "\tFallback due to SG numbers: %lu\n",
+ ce_algs[i].stat_fb_maxsg);
break;
case CRYPTO_ALG_TYPE_RNG:
- seq_printf(seq, "%s %s %lu %lu\n",
+ seq_printf(seq, "%s %s reqs=%lu bytes=%lu\n",
ce_algs[i].alg.rng.base.cra_driver_name,
ce_algs[i].alg.rng.base.cra_name,
ce_algs[i].stat_req, ce_algs[i].stat_bytes);
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
index 17f6885dbe35..0568fffc4dd3 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
@@ -50,9 +50,9 @@ int sun8i_ce_hash_crainit(struct crypto_tfm *tfm)
sizeof(struct sun8i_ce_hash_reqctx) +
crypto_ahash_reqsize(op->fallback_tfm));
- dev_info(op->ce->dev, "Fallback for %s is %s\n",
- crypto_tfm_alg_driver_name(tfm),
- crypto_tfm_alg_driver_name(&op->fallback_tfm->base));
+ memcpy(algt->fbname, crypto_tfm_alg_driver_name(&op->fallback_tfm->base),
+ CRYPTO_MAX_ALG_NAME);
+
err = pm_runtime_get_sync(op->ce->dev);
if (err < 0)
goto error_pm;
@@ -199,17 +199,32 @@ static int sun8i_ce_hash_digest_fb(struct ahash_request *areq)
static bool sun8i_ce_hash_need_fallback(struct ahash_request *areq)
{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
+ struct ahash_alg *alg = __crypto_ahash_alg(tfm->base.__crt_alg);
+ struct sun8i_ce_alg_template *algt;
struct scatterlist *sg;
- if (areq->nbytes == 0)
+ algt = container_of(alg, struct sun8i_ce_alg_template, alg.hash);
+
+ if (areq->nbytes == 0) {
+ algt->stat_fb_len0++;
return true;
+ }
/* we need to reserve one SG for padding one */
- if (sg_nents_for_len(areq->src, areq->nbytes) > MAX_SG - 1)
+ if (sg_nents_for_len(areq->src, areq->nbytes) > MAX_SG - 1) {
+ algt->stat_fb_maxsg++;
return true;
+ }
sg = areq->src;
while (sg) {
- if (sg->length % 4 || !IS_ALIGNED(sg->offset, sizeof(u32)))
+ if (sg->length % 4) {
+ algt->stat_fb_srclen++;
return true;
+ }
+ if (!IS_ALIGNED(sg->offset, sizeof(u32))) {
+ algt->stat_fb_srcali++;
+ return true;
+ }
sg = sg_next(sg);
}
return false;
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h
index 229b696d5a2c..30a6405b2051 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h
@@ -393,11 +393,18 @@ struct sun8i_ce_alg_template {
struct rng_alg rng;
struct akcipher_alg rsa;
} alg;
-#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG
unsigned long stat_req;
unsigned long stat_fb;
unsigned long stat_bytes;
-#endif
+ unsigned long stat_fb_maxsg;
+ unsigned long stat_fb_leniv;
+ unsigned long stat_fb_len0;
+ unsigned long stat_fb_mod16;
+ unsigned long stat_fb_srcali;
+ unsigned long stat_fb_srclen;
+ unsigned long stat_fb_dstali;
+ unsigned long stat_fb_dstlen;
+ char fbname[CRYPTO_MAX_ALG_NAME];
};
int sun8i_ce_enqueue(struct crypto_async_request *areq, u32 type);
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 19/19] crypto: sun8i-ce: do not fallback if cryptlen is less than sg length
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (17 preceding siblings ...)
2022-03-17 20:56 ` [PATCH 18/19] crypto: sun8i-ce: rework debugging Corentin Labbe
@ 2022-03-17 20:56 ` Corentin Labbe
2022-04-08 8:24 ` [PATCH 00/19] crypto: allwinner: lots of fixes Herbert Xu
19 siblings, 0 replies; 24+ messages in thread
From: Corentin Labbe @ 2022-03-17 20:56 UTC (permalink / raw)
To: herbert, jernej.skrabec, samuel, wens
Cc: linux-arm-kernel, linux-crypto, linux-kernel, linux-sunxi,
Corentin Labbe
The sg length could be more than remaining data on it.
So check the length requirement against the minimum between those two
values.
Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
---
drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
index 315a62e424d6..74b4e910a38d 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
@@ -27,6 +27,7 @@ static int sun8i_ce_cipher_need_fallback(struct skcipher_request *areq)
struct scatterlist *sg;
struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
struct sun8i_ce_alg_template *algt;
+ unsigned int todo, len;
algt = container_of(alg, struct sun8i_ce_alg_template, alg.skcipher);
@@ -51,28 +52,35 @@ static int sun8i_ce_cipher_need_fallback(struct skcipher_request *areq)
return true;
}
+ len = areq->cryptlen;
sg = areq->src;
while (sg) {
if (!IS_ALIGNED(sg->offset, sizeof(u32))) {
algt->stat_fb_srcali++;
return true;
}
- if (sg->length % 4) {
+ todo = min(len, sg->length);
+ if (todo % 4) {
algt->stat_fb_srclen++;
return true;
}
+ len -= todo;
sg = sg_next(sg);
}
+
+ len = areq->cryptlen;
sg = areq->dst;
while (sg) {
if (!IS_ALIGNED(sg->offset, sizeof(u32))) {
algt->stat_fb_dstali++;
return true;
}
- if (sg->length % 4) {
+ todo = min(len, sg->length);
+ if (todo % 4) {
algt->stat_fb_dstlen++;
return true;
}
+ len -= todo;
sg = sg_next(sg);
}
return false;
--
2.34.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH 00/19] crypto: allwinner: lots of fixes
2022-03-17 20:55 [PATCH 00/19] crypto: allwinner: lots of fixes Corentin Labbe
` (18 preceding siblings ...)
2022-03-17 20:56 ` [PATCH 19/19] crypto: sun8i-ce: do not fallback if cryptlen is less than sg length Corentin Labbe
@ 2022-04-08 8:24 ` Herbert Xu
2022-04-11 7:37 ` LABBE Corentin
19 siblings, 1 reply; 24+ messages in thread
From: Herbert Xu @ 2022-04-08 8:24 UTC (permalink / raw)
To: Corentin Labbe
Cc: jernej.skrabec, samuel, wens, linux-arm-kernel, linux-crypto,
linux-kernel, linux-sunxi, clabbe
Corentin Labbe <clabbe@baylibre.com> wrote:
> Hello
>
> This series is all fixes which I found on allwinner crypto drivers.
This doesn't compile cleanly with sparse. Please fix and resubmit.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH 00/19] crypto: allwinner: lots of fixes
2022-04-08 8:24 ` [PATCH 00/19] crypto: allwinner: lots of fixes Herbert Xu
@ 2022-04-11 7:37 ` LABBE Corentin
2022-04-11 7:40 ` Herbert Xu
0 siblings, 1 reply; 24+ messages in thread
From: LABBE Corentin @ 2022-04-11 7:37 UTC (permalink / raw)
To: Herbert Xu
Cc: jernej.skrabec, samuel, wens, linux-arm-kernel, linux-crypto,
linux-kernel, linux-sunxi
Le Fri, Apr 08, 2022 at 04:24:20PM +0800, Herbert Xu a écrit :
> Corentin Labbe <clabbe@baylibre.com> wrote:
> > Hello
> >
> > This series is all fixes which I found on allwinner crypto drivers.
>
> This doesn't compile cleanly with sparse. Please fix and resubmit.
>
> Thanks,
Hello
Coul you give me more details ?
I do not have any sparse error.
Regards
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 00/19] crypto: allwinner: lots of fixes
2022-04-11 7:37 ` LABBE Corentin
@ 2022-04-11 7:40 ` Herbert Xu
2022-04-11 8:37 ` LABBE Corentin
0 siblings, 1 reply; 24+ messages in thread
From: Herbert Xu @ 2022-04-11 7:40 UTC (permalink / raw)
To: LABBE Corentin
Cc: jernej.skrabec, samuel, wens, linux-arm-kernel, linux-crypto,
linux-kernel, linux-sunxi
On Mon, Apr 11, 2022 at 09:37:22AM +0200, LABBE Corentin wrote:
>
> Coul you give me more details ?
> I do not have any sparse error.
Did you compile with C=1? Anyway, for a start hash_pad is broken as
it tries to store an le32 value into a u32.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 00/19] crypto: allwinner: lots of fixes
2022-04-11 7:40 ` Herbert Xu
@ 2022-04-11 8:37 ` LABBE Corentin
0 siblings, 0 replies; 24+ messages in thread
From: LABBE Corentin @ 2022-04-11 8:37 UTC (permalink / raw)
To: Herbert Xu
Cc: jernej.skrabec, samuel, wens, linux-arm-kernel, linux-crypto,
linux-kernel, linux-sunxi
Le Mon, Apr 11, 2022 at 03:40:28PM +0800, Herbert Xu a écrit :
> On Mon, Apr 11, 2022 at 09:37:22AM +0200, LABBE Corentin wrote:
> >
> > Coul you give me more details ?
> > I do not have any sparse error.
>
> Did you compile with C=1? Anyway, for a start hash_pad is broken as
> it tries to store an le32 value into a u32.
Yes I compile with both W=1 and C=1.
But rigth, hash_pad takes a u32 but the origin variable bf is __le32, so perhaps it is why sparse dont see a problem.
Anyway I will fix hash_pad declaration.
Thanks
^ permalink raw reply [flat|nested] 24+ messages in thread