All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/8] crypto: avoid DMA padding for request structures
@ 2022-04-06 14:27 Ard Biesheuvel
  2022-04-06 14:27 ` [PATCH 1/8] crypto: add flag for algos that need DMA aligned context buffers Ard Biesheuvel
                   ` (7 more replies)
  0 siblings, 8 replies; 13+ messages in thread
From: Ard Biesheuvel @ 2022-04-06 14:27 UTC (permalink / raw)
  To: linux-crypto
  Cc: herbert, keescook, Ard Biesheuvel, Catalin Marinas, Will Deacon,
	Marc Zyngier, Arnd Bergmann, Eric Biggers, Gilad Ben-Yossef,
	Corentin Labbe, Saravana Kannan

This series is related to Catalin's work [0] on reducing the memory
overhead of supporting non-coherent DMA, which requires memory buffers
to be rounded up to avoid corruption by the cache invalidation that is
needed for inbound DMA.

In the crypto subsystem, every skcipher, aead or ahash request requires
a request struct to be allocated, and these are usually backed by
kmalloc(). Such request buffers are sized dynamically, based on the
requirements of the implementation of the algorithm, and the surplus is
made available to the driver via an opaque context pointer.

Since some drivers may perform inbound non-coherent DMA into that
buffer, it must not share any cachelines with adjacent allocations, and
this is why the context pointer is rounded up to ARCH_KMALLOC_MINALIGN,
which takes the minimum DMA alignment into account on architectures
where this is needed.

This means that, even when using crypto drivers that don't do DMA to
begin with (which includes synchronous skciphers, aeads and ahashes
based on CPU instructions), or that do only coherent DMA are forced to
perform this padding and alignment, which may affect the memory
footprint substantially: on arm64, the compile time minimum DMA
alignment is 128 bytes.

So instead, require drivers to set a new flag
CRYPTO_ALG_NEED_DMA_ALIGNMENT if it performs DMA into the context
buffers, and only take DMA alignment into account if the flag is set.

Initially, we set this flag for all asynchronous accelerator drivers in
drivers/crypto, which simply preserves the status quo for these systems
once the subsequent patches get rid of this overhead. Future patches
could be applied to drivers that don't actually need it, or only need it
when running in non-coherent mode.

Note that the new approach proposed here still uses the compile time
value for DMA alignment, but this can be updated to use the runtime
values (which is usually lower and therefore less wasteful) after
Catalin's changes land.

[0] https://lore.kernel.org/linux-arm-kernel/20220405135758.774016-1-catalin.marinas@arm.com/

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Eric Biggers <ebiggers@kernel.org>,
Cc: Gilad Ben-Yossef <gilad@benyossef.com>
Cc: Corentin Labbe <clabbe@baylibre.com>
Cc: Saravana Kannan <saravanak@google.com>

Ard Biesheuvel (8):
  crypto: add flag for algos that need DMA aligned context buffers
  crypto: safexcel - take request size after setting TFM
  crypto: drivers - set CRYPTO_ALG_NEED_DMA_ALIGNMENT where needed
  crypto: drivers - avoid setting skcipher TFM reqsize directly
  crypto: skcipher - avoid rounding up request size to DMA alignment
  crypto: aead - avoid DMA alignment for request structures unless
    needed
  crypto: ahash - avoid DMA alignment for request structures unless
    needed
  crypto: safexcel - reduce alignment of stack buffer

 drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c |  6 +-
 drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c   | 20 +++----
 drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c |  6 +-
 drivers/crypto/amcc/crypto4xx_core.c                |  8 +++
 drivers/crypto/amlogic/amlogic-gxl-cipher.c         |  5 +-
 drivers/crypto/amlogic/amlogic-gxl-core.c           |  2 +
 drivers/crypto/atmel-aes.c                          |  2 +-
 drivers/crypto/atmel-sha.c                          |  2 +-
 drivers/crypto/atmel-tdes.c                         |  2 +-
 drivers/crypto/axis/artpec6_crypto.c                |  8 +++
 drivers/crypto/bcm/cipher.c                         | 23 +++++++-
 drivers/crypto/caam/caamalg.c                       |  4 +-
 drivers/crypto/caam/caamalg_qi.c                    |  2 +
 drivers/crypto/caam/caamalg_qi2.c                   |  4 +-
 drivers/crypto/caam/caamhash.c                      |  3 +-
 drivers/crypto/cavium/cpt/cptvf_algs.c              |  6 ++
 drivers/crypto/cavium/nitrox/nitrox_aead.c          |  6 +-
 drivers/crypto/cavium/nitrox/nitrox_skcipher.c      | 24 +++++---
 drivers/crypto/ccree/cc_aead.c                      |  3 +
 drivers/crypto/ccree/cc_cipher.c                    |  3 +
 drivers/crypto/ccree/cc_hash.c                      |  6 ++
 drivers/crypto/chelsio/chcr_algo.c                  |  5 +-
 drivers/crypto/gemini/sl3516-ce-cipher.c            |  5 +-
 drivers/crypto/gemini/sl3516-ce-core.c              |  1 +
 drivers/crypto/hifn_795x.c                          |  4 +-
 drivers/crypto/hisilicon/sec/sec_algs.c             |  8 +++
 drivers/crypto/hisilicon/sec2/sec_crypto.c          |  2 +
 drivers/crypto/inside-secure/safexcel.h             | 17 +++---
 drivers/crypto/inside-secure/safexcel_cipher.c      | 55 ++++++++++++++++--
 drivers/crypto/inside-secure/safexcel_hash.c        | 26 +++++++++
 drivers/crypto/ixp4xx_crypto.c                      |  2 +
 drivers/crypto/keembay/keembay-ocs-aes-core.c       | 12 ++++
 drivers/crypto/keembay/keembay-ocs-hcu-core.c       | 30 ++++++----
 drivers/crypto/marvell/cesa/cipher.c                |  6 ++
 drivers/crypto/marvell/cesa/hash.c                  |  6 ++
 drivers/crypto/marvell/octeontx/otx_cptvf_algs.c    | 60 +++++++++++++++-----
 drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.c  | 52 ++++++++++++-----
 drivers/crypto/mxs-dcp.c                            |  8 ++-
 drivers/crypto/n2_core.c                            |  1 +
 drivers/crypto/omap-aes.c                           |  5 ++
 drivers/crypto/omap-des.c                           |  4 ++
 drivers/crypto/omap-sham.c                          | 12 ++++
 drivers/crypto/qce/aead.c                           |  1 +
 drivers/crypto/qce/sha.c                            |  3 +-
 drivers/crypto/qce/skcipher.c                       |  1 +
 drivers/crypto/rockchip/rk3288_crypto_ahash.c       |  3 +
 drivers/crypto/rockchip/rk3288_crypto_skcipher.c    | 18 ++++--
 drivers/crypto/s5p-sss.c                            |  6 ++
 drivers/crypto/sa2ul.c                              |  9 +++
 drivers/crypto/sahara.c                             | 14 +++--
 drivers/crypto/stm32/stm32-cryp.c                   | 27 ++++++---
 drivers/crypto/stm32/stm32-hash.c                   |  8 +++
 drivers/crypto/talitos.c                            | 40 +++++++++++++
 drivers/crypto/ux500/cryp/cryp_core.c               | 21 ++++---
 drivers/crypto/ux500/hash/hash_core.c               | 12 ++--
 drivers/crypto/xilinx/zynqmp-aes-gcm.c              |  1 +
 include/crypto/aead.h                               |  2 +-
 include/crypto/hash.h                               |  5 +-
 include/crypto/internal/aead.h                      | 13 ++++-
 include/crypto/internal/hash.h                      | 10 +++-
 include/crypto/internal/skcipher.h                  | 13 ++++-
 include/crypto/skcipher.h                           |  8 +--
 include/linux/crypto.h                              | 21 +++++++
 63 files changed, 568 insertions(+), 134 deletions(-)

-- 
2.30.2


^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2022-04-07  8:33 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-06 14:27 [PATCH 0/8] crypto: avoid DMA padding for request structures Ard Biesheuvel
2022-04-06 14:27 ` [PATCH 1/8] crypto: add flag for algos that need DMA aligned context buffers Ard Biesheuvel
2022-04-06 14:27 ` [PATCH 2/8] crypto: safexcel - take request size after setting TFM Ard Biesheuvel
2022-04-07  4:32   ` Herbert Xu
2022-04-07  8:32     ` Ard Biesheuvel
2022-04-07  8:33       ` Ard Biesheuvel
2022-04-06 14:27 ` [PATCH 3/8] crypto: drivers - set CRYPTO_ALG_NEED_DMA_ALIGNMENT where needed Ard Biesheuvel
2022-04-06 14:27 ` [PATCH 4/8] crypto: drivers - avoid setting skcipher TFM reqsize directly Ard Biesheuvel
2022-04-06 14:27 ` [PATCH 5/8] crypto: skcipher - avoid rounding up request size to DMA alignment Ard Biesheuvel
2022-04-06 14:27 ` [PATCH 6/8] crypto: aead - avoid DMA alignment for request structures unless needed Ard Biesheuvel
2022-04-06 14:27 ` [PATCH 7/8] crypto: ahash " Ard Biesheuvel
2022-04-06 20:05   ` kernel test robot
2022-04-06 14:27 ` [PATCH 8/8] crypto: safexcel - reduce alignment of stack buffer Ard Biesheuvel

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.