From: Eric Biggers <ebiggers@kernel.org>
To: Ard Biesheuvel <ardb@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org,
Megha Dey <megha.dey@intel.com>,
Herbert Xu <herbert@gondor.apana.org.au>
Subject: Re: [PATCH 0/2] crypto: x86/aes-ni-xts - recover and improve performance
Date: Fri, 25 Dec 2020 11:14:35 -0800 [thread overview]
Message-ID: <X+Y6G0pHja1C61s9@sol.localdomain> (raw)
In-Reply-To: <20201222160629.22268-1-ardb@kernel.org>
On Tue, Dec 22, 2020 at 05:06:27PM +0100, Ard Biesheuvel wrote:
> The AES-NI implementation of XTS was impacted significantly by the retpoline
> changes, which is due to the fact that both its asm helper and the chaining
> mode glue library use indirect calls for processing small quantitities of
> data
>
> So let's fix this, by:
> - creating a minimal, backportable fix that recovers most of the performance,
> by reducing the number of indirect calls substantially;
> - for future releases, rewrite the XTS implementation completely, and replace
> the glue helper with a core asm routine that is more flexible, making the C
> code wrapper much more straight-forward.
>
> This results in a substantial performance improvement: around ~2x for 1k and
> 4k blocks, and more than 3x for ~1k blocks that require ciphertext stealing
> (benchmarked using tcrypt using 1420 byte blocks - full results below)
>
> It also allows us to enable the same driver for i386.
>
> Cc: Megha Dey <megha.dey@intel.com>
> Cc: Eric Biggers <ebiggers@kernel.org>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>
>
> Ard Biesheuvel (2):
> crypto: x86/aes-ni-xts - use direct calls to and 4-way stride
> crypto: x86/aes-ni-xts - rewrite and drop indirections via glue helper
>
> arch/x86/crypto/aesni-intel_asm.S | 353 ++++++++++++++++----
> arch/x86/crypto/aesni-intel_glue.c | 230 +++++++------
> 2 files changed, 412 insertions(+), 171 deletions(-)
>
> --
> 2.17.1
>
> Benchmarked using tcrypt on a Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz.
Thanks for doing this! I didn't realize that there was such a big performance
regression here. Getting rid of these indirect calls looks like the right
approach; this all seems to have been written for a world where indirect calls
are much faster...
I did some quick benchmarks on Zen ("AMD Ryzen Threadripper 1950X 16-Core
Processor") with CONFIG_RETPOLINE=y and confirmed the speedup on 4096-byte
blocks is around 2x there too. (It's over 2x for AES-128-XTS and AES-192-XTS,
and a bit under 2x for AES-256-XTS. And most of the speedup comes from the
first patch.) Also, the extra self-tests are passing.
So feel free to add:
Tested-by: Eric Biggers <ebiggers@google.com> # x86_64
Note that this patch series didn't apply cleanly, as it seems to depend on some
other patches you've sent out recently. So I actually tested your
"for-kernelci" branch instead of applying these directly.
- Eric
next prev parent reply other threads:[~2020-12-25 19:15 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-22 16:06 [PATCH 0/2] crypto: x86/aes-ni-xts - recover and improve performance Ard Biesheuvel
2020-12-22 16:06 ` [PATCH 1/2] crypto: x86/aes-ni-xts - use direct calls to and 4-way stride Ard Biesheuvel
2020-12-22 16:06 ` [PATCH 2/2] crypto: x86/aes-ni-xts - rewrite and drop indirections via glue helper Ard Biesheuvel
2020-12-25 19:14 ` Eric Biggers [this message]
2020-12-29 10:52 ` [PATCH 0/2] crypto: x86/aes-ni-xts - recover and improve performance Ard Biesheuvel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=X+Y6G0pHja1C61s9@sol.localdomain \
--to=ebiggers@kernel.org \
--cc=ardb@kernel.org \
--cc=herbert@gondor.apana.org.au \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=megha.dey@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).