From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
To: linux-crypto@vger.kernel.org
Cc: herbert@gondor.apana.org.au, ebiggers@google.com,
Ard Biesheuvel <ard.biesheuvel@linaro.org>
Subject: [PATCH v4 21/32] crypto: arm/aes-neonbs - provide a synchronous version of ctr(aes)
Date: Tue, 2 Jul 2019 21:41:39 +0200 [thread overview]
Message-ID: <20190702194150.10405-22-ard.biesheuvel@linaro.org> (raw)
In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org>
AES in CTR mode is used by modes such as GCM and CCM, which are often
used in contexts where only synchronous ciphers are permitted. So
provide a synchronous version of ctr(aes) based on the existing code.
This requires a non-SIMD fallback to deal with invocations occurring
from a context where SIMD instructions may not be used. We have a
helper for this now in the AES library, so wire that up.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/crypto/aes-neonbs-glue.c | 65 ++++++++++++++++++++
1 file changed, 65 insertions(+)
diff --git a/arch/arm/crypto/aes-neonbs-glue.c b/arch/arm/crypto/aes-neonbs-glue.c
index f43c9365b6a9..6eecdbb7e9b6 100644
--- a/arch/arm/crypto/aes-neonbs-glue.c
+++ b/arch/arm/crypto/aes-neonbs-glue.c
@@ -9,8 +9,10 @@
*/
#include <asm/neon.h>
+#include <asm/simd.h>
#include <crypto/aes.h>
#include <crypto/cbc.h>
+#include <crypto/ctr.h>
#include <crypto/internal/simd.h>
#include <crypto/internal/skcipher.h>
#include <crypto/xts.h>
@@ -57,6 +59,11 @@ struct aesbs_xts_ctx {
struct crypto_cipher *tweak_tfm;
};
+struct aesbs_ctr_ctx {
+ struct aesbs_ctx key; /* must be first member */
+ struct crypto_aes_ctx fallback;
+};
+
static int aesbs_setkey(struct crypto_skcipher *tfm, const u8 *in_key,
unsigned int key_len)
{
@@ -192,6 +199,25 @@ static void cbc_exit(struct crypto_tfm *tfm)
crypto_free_cipher(ctx->enc_tfm);
}
+static int aesbs_ctr_setkey_sync(struct crypto_skcipher *tfm, const u8 *in_key,
+ unsigned int key_len)
+{
+ struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm);
+ int err;
+
+ err = aes_expandkey(&ctx->fallback, in_key, key_len);
+ if (err)
+ return err;
+
+ ctx->key.rounds = 6 + key_len / 4;
+
+ kernel_neon_begin();
+ aesbs_convert_key(ctx->key.rk, ctx->fallback.key_enc, ctx->key.rounds);
+ kernel_neon_end();
+
+ return 0;
+}
+
static int ctr_encrypt(struct skcipher_request *req)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
@@ -234,6 +260,29 @@ static int ctr_encrypt(struct skcipher_request *req)
return err;
}
+static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst)
+{
+ struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm);
+ unsigned long flags;
+
+ /*
+ * Temporarily disable interrupts to avoid races where
+ * cachelines are evicted when the CPU is interrupted
+ * to do something else.
+ */
+ local_irq_save(flags);
+ aes_encrypt(&ctx->fallback, dst, src);
+ local_irq_restore(flags);
+}
+
+static int ctr_encrypt_sync(struct skcipher_request *req)
+{
+ if (!crypto_simd_usable())
+ return crypto_ctr_encrypt_walk(req, ctr_encrypt_one);
+
+ return ctr_encrypt(req);
+}
+
static int aesbs_xts_setkey(struct crypto_skcipher *tfm, const u8 *in_key,
unsigned int key_len)
{
@@ -361,6 +410,22 @@ static struct skcipher_alg aes_algs[] = { {
.setkey = aesbs_setkey,
.encrypt = ctr_encrypt,
.decrypt = ctr_encrypt,
+}, {
+ .base.cra_name = "ctr(aes)",
+ .base.cra_driver_name = "ctr-aes-neonbs-sync",
+ .base.cra_priority = 250 - 1,
+ .base.cra_blocksize = 1,
+ .base.cra_ctxsize = sizeof(struct aesbs_ctr_ctx),
+ .base.cra_module = THIS_MODULE,
+
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .chunksize = AES_BLOCK_SIZE,
+ .walksize = 8 * AES_BLOCK_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = aesbs_ctr_setkey_sync,
+ .encrypt = ctr_encrypt_sync,
+ .decrypt = ctr_encrypt_sync,
}, {
.base.cra_name = "__xts(aes)",
.base.cra_driver_name = "__xts-aes-neonbs",
--
2.17.1
next prev parent reply other threads:[~2019-07-02 19:42 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-02 19:41 [PATCH v4 00/32] crypto: AES cleanup Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 01/32] crypto: arm/aes-ce - cosmetic/whitespace cleanup Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 02/32] crypto: aes - rename local routines to prevent future clashes Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 03/32] crypto: aes/fixed-time - align key schedule with other implementations Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 04/32] crypto: aes - create AES library based on the fixed time AES code Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 05/32] crypto: x86/aes-ni - switch to generic for fallback and key routines Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 06/32] crypto: x86/aes - drop scalar assembler implementations Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 07/32] crypto: padlock/aes - switch to library version of key expansion routine Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 08/32] crypto: cesa/aes " Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 09/32] crypto: safexcel/aes " Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 10/32] crypto: arm64/ghash - switch to AES library Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 11/32] crypto: arm/aes-neonbs - switch to library version of key expansion routine Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 12/32] crypto: arm64/aes-ccm - switch to AES library Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 13/32] crypto: arm64/aes-neonbs - switch to library version of key expansion routine Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 14/32] crypto: arm64/aes-ce " Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 15/32] crypto: generic/aes - drop key expansion routine in favor of library version Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 16/32] crypto: ctr - add helper for performing a CTR encryption walk Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 17/32] crypto: aes - move sync ctr(aes) to AES library and generic helper Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 18/32] crypto: arm64/aes-ce-cipher - use AES library as fallback Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 19/32] crypto: aes/arm - use native endiannes for key schedule Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 20/32] crypto: arm/aes-ce - provide a synchronous version of ctr(aes) Ard Biesheuvel
2019-07-02 19:41 ` Ard Biesheuvel [this message]
2019-07-02 19:41 ` [PATCH v4 22/32] crypto: arm/ghash - provide a synchronous version Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 23/32] bluetooth: switch to AES library Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 24/32] crypto: amcc/aes - switch to AES library for GCM key derivation Ard Biesheuvel
2019-10-27 10:08 ` Christian Lamparter
2019-07-02 19:41 ` [PATCH v4 25/32] crypto: ccp - move to AES library for CMAC " Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 26/32] crypto: chelsio/aes - replace AES cipher calls with library calls Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 27/32] crypto: aes/generic - unexport last-round AES tables Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 28/32] crypto: lib/aes - export sbox and inverse sbox Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 29/32] crypto: arm64/aes-neon - switch to shared AES Sboxes Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 30/32] crypto: arm/aes-cipher - switch to shared AES inverse Sbox Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 31/32] crypto: arm64/aes-cipher " Ard Biesheuvel
2019-07-02 19:41 ` [PATCH v4 32/32] crypto: arm/aes-scalar - unexport en/decryption routines Ard Biesheuvel
2019-07-26 12:31 ` [PATCH v4 00/32] crypto: AES cleanup Herbert Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190702194150.10405-22-ard.biesheuvel@linaro.org \
--to=ard.biesheuvel@linaro.org \
--cc=ebiggers@google.com \
--cc=herbert@gondor.apana.org.au \
--cc=linux-crypto@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).