From: Eric Biggers <ebiggers@kernel.org> To: linux-riscv@lists.infradead.org, Palmer Dabbelt <palmer@dabbelt.com> Cc: linux-crypto@vger.kernel.org, "Jerry Shih" <jerry.shih@sifive.com>, "Christoph Müllner" <christoph.muellner@vrull.eu>, "Heiko Stuebner" <heiko@sntech.de>, "Phoebe Chen" <phoebe.chen@sifive.com>, "Andy Chiu" <andy.chiu@sifive.com> Subject: [PATCH riscv/for-next] crypto: riscv - parallelize AES-CBC decryption Date: Wed, 7 Feb 2024 22:08:51 -0800 [thread overview] Message-ID: <20240208060851.154129-1-ebiggers@kernel.org> (raw) From: Eric Biggers <ebiggers@google.com> Since CBC decryption is parallelizable, make the RISC-V implementation of AES-CBC decryption process multiple blocks at a time, instead of processing the blocks one by one. This should improve performance. Signed-off-by: Eric Biggers <ebiggers@google.com> --- arch/riscv/crypto/aes-riscv64-zvkned.S | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/arch/riscv/crypto/aes-riscv64-zvkned.S b/arch/riscv/crypto/aes-riscv64-zvkned.S index 78d4e1186c074..43541aad6386c 100644 --- a/arch/riscv/crypto/aes-riscv64-zvkned.S +++ b/arch/riscv/crypto/aes-riscv64-zvkned.S @@ -132,33 +132,39 @@ SYM_FUNC_END(aes_ecb_decrypt_zvkned) addi INP, INP, 16 addi OUTP, OUTP, 16 addi LEN, LEN, -16 bnez LEN, 1b vse32.v v16, (IVP) // Store next IV ret .endm .macro aes_cbc_decrypt keylen + srli LEN, LEN, 2 // Convert LEN from bytes to words vle32.v v16, (IVP) // Load IV 1: - vle32.v v17, (INP) // Load ciphertext block - vmv.v.v v18, v17 // Save ciphertext block - aes_decrypt v17, \keylen // Decrypt - vxor.vv v17, v17, v16 // XOR with IV or prev ciphertext block - vse32.v v17, (OUTP) // Store plaintext block - vmv.v.v v16, v18 // Next "IV" is prev ciphertext block - addi INP, INP, 16 - addi OUTP, OUTP, 16 - addi LEN, LEN, -16 + vsetvli t0, LEN, e32, m4, ta, ma + vle32.v v20, (INP) // Load ciphertext blocks + vslideup.vi v16, v20, 4 // Setup prev ciphertext blocks + addi t1, t0, -4 + vslidedown.vx v24, v20, t1 // Save last ciphertext block + aes_decrypt v20, \keylen // Decrypt the blocks + vxor.vv v20, v20, v16 // XOR with prev ciphertext blocks + vse32.v v20, (OUTP) // Store plaintext blocks + vmv.v.v v16, v24 // Next "IV" is last ciphertext block + slli t1, t0, 2 // Words to bytes + add INP, INP, t1 + add OUTP, OUTP, t1 + sub LEN, LEN, t0 bnez LEN, 1b + vsetivli zero, 4, e32, m1, ta, ma vse32.v v16, (IVP) // Store next IV ret .endm // void aes_cbc_encrypt_zvkned(const struct crypto_aes_ctx *key, // const u8 *in, u8 *out, size_t len, u8 iv[16]); // // |len| must be nonzero and a multiple of 16 (AES_BLOCK_SIZE). SYM_FUNC_START(aes_cbc_encrypt_zvkned) aes_begin KEYP, 128f, 192f base-commit: cb4ede926134a65bc3bf90ed58dace8451d7e759 -- 2.43.0
WARNING: multiple messages have this Message-ID (diff)
From: Eric Biggers <ebiggers@kernel.org> To: linux-riscv@lists.infradead.org, Palmer Dabbelt <palmer@dabbelt.com> Cc: linux-crypto@vger.kernel.org, "Jerry Shih" <jerry.shih@sifive.com>, "Christoph Müllner" <christoph.muellner@vrull.eu>, "Heiko Stuebner" <heiko@sntech.de>, "Phoebe Chen" <phoebe.chen@sifive.com>, "Andy Chiu" <andy.chiu@sifive.com> Subject: [PATCH riscv/for-next] crypto: riscv - parallelize AES-CBC decryption Date: Wed, 7 Feb 2024 22:08:51 -0800 [thread overview] Message-ID: <20240208060851.154129-1-ebiggers@kernel.org> (raw) From: Eric Biggers <ebiggers@google.com> Since CBC decryption is parallelizable, make the RISC-V implementation of AES-CBC decryption process multiple blocks at a time, instead of processing the blocks one by one. This should improve performance. Signed-off-by: Eric Biggers <ebiggers@google.com> --- arch/riscv/crypto/aes-riscv64-zvkned.S | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/arch/riscv/crypto/aes-riscv64-zvkned.S b/arch/riscv/crypto/aes-riscv64-zvkned.S index 78d4e1186c074..43541aad6386c 100644 --- a/arch/riscv/crypto/aes-riscv64-zvkned.S +++ b/arch/riscv/crypto/aes-riscv64-zvkned.S @@ -132,33 +132,39 @@ SYM_FUNC_END(aes_ecb_decrypt_zvkned) addi INP, INP, 16 addi OUTP, OUTP, 16 addi LEN, LEN, -16 bnez LEN, 1b vse32.v v16, (IVP) // Store next IV ret .endm .macro aes_cbc_decrypt keylen + srli LEN, LEN, 2 // Convert LEN from bytes to words vle32.v v16, (IVP) // Load IV 1: - vle32.v v17, (INP) // Load ciphertext block - vmv.v.v v18, v17 // Save ciphertext block - aes_decrypt v17, \keylen // Decrypt - vxor.vv v17, v17, v16 // XOR with IV or prev ciphertext block - vse32.v v17, (OUTP) // Store plaintext block - vmv.v.v v16, v18 // Next "IV" is prev ciphertext block - addi INP, INP, 16 - addi OUTP, OUTP, 16 - addi LEN, LEN, -16 + vsetvli t0, LEN, e32, m4, ta, ma + vle32.v v20, (INP) // Load ciphertext blocks + vslideup.vi v16, v20, 4 // Setup prev ciphertext blocks + addi t1, t0, -4 + vslidedown.vx v24, v20, t1 // Save last ciphertext block + aes_decrypt v20, \keylen // Decrypt the blocks + vxor.vv v20, v20, v16 // XOR with prev ciphertext blocks + vse32.v v20, (OUTP) // Store plaintext blocks + vmv.v.v v16, v24 // Next "IV" is last ciphertext block + slli t1, t0, 2 // Words to bytes + add INP, INP, t1 + add OUTP, OUTP, t1 + sub LEN, LEN, t0 bnez LEN, 1b + vsetivli zero, 4, e32, m1, ta, ma vse32.v v16, (IVP) // Store next IV ret .endm // void aes_cbc_encrypt_zvkned(const struct crypto_aes_ctx *key, // const u8 *in, u8 *out, size_t len, u8 iv[16]); // // |len| must be nonzero and a multiple of 16 (AES_BLOCK_SIZE). SYM_FUNC_START(aes_cbc_encrypt_zvkned) aes_begin KEYP, 128f, 192f base-commit: cb4ede926134a65bc3bf90ed58dace8451d7e759 -- 2.43.0 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next reply other threads:[~2024-02-08 6:10 UTC|newest] Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-02-08 6:08 Eric Biggers [this message] 2024-02-08 6:08 ` [PATCH riscv/for-next] crypto: riscv - parallelize AES-CBC decryption Eric Biggers 2024-02-10 15:25 ` Jerry Shih 2024-02-10 15:25 ` Jerry Shih 2024-02-10 18:12 ` Eric Biggers 2024-02-10 18:12 ` Eric Biggers 2024-02-26 1:40 ` Jerry Shih 2024-02-26 1:40 ` Jerry Shih 2024-03-20 1:48 ` Palmer Dabbelt 2024-03-20 1:48 ` Palmer Dabbelt 2024-03-20 20:50 ` patchwork-bot+linux-riscv 2024-03-20 20:50 ` patchwork-bot+linux-riscv
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20240208060851.154129-1-ebiggers@kernel.org \ --to=ebiggers@kernel.org \ --cc=andy.chiu@sifive.com \ --cc=christoph.muellner@vrull.eu \ --cc=heiko@sntech.de \ --cc=jerry.shih@sifive.com \ --cc=linux-crypto@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=palmer@dabbelt.com \ --cc=phoebe.chen@sifive.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.