linux-crypto.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 0/7] crypto: x86 - fix RCU stalls
@ 2022-10-06 22:31 Robert Elliott
  2022-10-06 22:31 ` [RFC PATCH 1/7] rcu: correct CONFIG_EXT_RCU_CPU_STALL_TIMEOUT descriptions Robert Elliott
                   ` (7 more replies)
  0 siblings, 8 replies; 126+ messages in thread
From: Robert Elliott @ 2022-10-06 22:31 UTC (permalink / raw)
  To: herbert, davem, tim.c.chen, linux-crypto, linux-kernel; +Cc: Robert Elliott

This series attempts to fix the RCU stalls triggered
by the x86 crypto drivers discussed in
https://lore.kernel.org/all/MW5PR84MB18426EBBA3303770A8BC0BDFAB759@MW5PR84MB1842.NAMPRD84.PROD.OUTLOOK.COM/

The following x86 drivers are enforcing a 4 KiB limit today,
using either the SZ_4K macro or a direct reference to 4096 bytes:
    blake-2s, chacha, nhpoly1305, poly1305, polyval
I've included a patch to make them use the same macro name.

These are not currently limited, so I've included patches for
them:
   sha*, crc*, sm3, ghash

I originally encountered some RCU stalls with tcrypt in aesni:
    tcrypt: testing encryption speed of sync skcipher cts(cbc(aes)) using cts(cbc(aes-aesni))
    tcrypt: testing encryption speed of sync skcipher cfb(aes) using cfb(aes-aesni)
    tcrypt: testing decryption speed of sync skcipher cfb(aes) using cfb(aes-aesni)
but I don't see any problems in the source code. So, no patch
is proposed for that driver yet.

With various errors inserted, all the drivers failed self-tests or
hung boot, so the changes seem functionally correct. I haven't done
comprehensive tests of different data sizes and alignments, so
please consider this an RFC.

I added some counters (not posted) to the drivers to observe
their behavior. During boot, the finup function is actually
called much more often than update - 1500 calls for 2 GiB via
finup vs. 23 KiB via update. The patch breaks that into half
a million 4 KiB chunks.

/sys/module/sha512_ssse3/parameters/rob_call_finup:1541
/sys/module/sha512_ssse3/parameters/rob_call_finup_fpu:469325
/sys/module/sha512_ssse3/parameters/rob_call_update:174
/sys/module/sha512_ssse3/parameters/rob_call_update_fpu:32
/sys/module/sha512_ssse3/parameters/rob_len_finup:2123048456
/sys/module/sha512_ssse3/parameters/rob_len_update:24120


Robert Elliott (7):
  rcu: correct CONFIG_EXT_RCU_CPU_STALL_TIMEOUT descriptions
  crypto: x86/sha - limit FPU preemption
  crypto: x86/crc - limit FPU preemption
  crypto: x86/sm3 - limit FPU preemption
  crypto: x86/ghash - restructure FPU context saving
  crypto: x86/ghash - limit FPU preemption
  crypto: x86 - use common macro for FPU limit

 Documentation/RCU/stallwarn.rst            |  9 +++---
 arch/x86/crypto/blake2s-glue.c             |  7 +++--
 arch/x86/crypto/chacha_glue.c              |  4 ++-
 arch/x86/crypto/crc32-pclmul_glue.c        | 18 ++++++++---
 arch/x86/crypto/crc32c-intel_glue.c        | 32 ++++++++++++++++----
 arch/x86/crypto/crct10dif-pclmul_glue.c    | 32 ++++++++++++++++----
 arch/x86/crypto/ghash-clmulni-intel_glue.c | 32 +++++++++++++++-----
 arch/x86/crypto/nhpoly1305-avx2-glue.c     |  3 +-
 arch/x86/crypto/nhpoly1305-sse2-glue.c     |  4 ++-
 arch/x86/crypto/poly1305_glue.c            | 25 +++++++++-------
 arch/x86/crypto/polyval-clmulni_glue.c     |  5 ++--
 arch/x86/crypto/sha1_ssse3_glue.c          | 34 +++++++++++++++++----
 arch/x86/crypto/sha256_ssse3_glue.c        | 35 ++++++++++++++++++----
 arch/x86/crypto/sha512_ssse3_glue.c        | 35 ++++++++++++++++++----
 arch/x86/crypto/sm3_avx_glue.c             | 28 +++++++++++++----
 kernel/rcu/Kconfig.debug                   |  2 +-
 16 files changed, 237 insertions(+), 68 deletions(-)

-- 
2.37.3


^ permalink raw reply	[flat|nested] 126+ messages in thread

end of thread, other threads:[~2022-12-16 22:12 UTC | newest]

Thread overview: 126+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-10-06 22:31 [RFC PATCH 0/7] crypto: x86 - fix RCU stalls Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 1/7] rcu: correct CONFIG_EXT_RCU_CPU_STALL_TIMEOUT descriptions Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 2/7] crypto: x86/sha - limit FPU preemption Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 3/7] crypto: x86/crc " Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 4/7] crypto: x86/sm3 " Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 5/7] crypto: x86/ghash - restructure FPU context saving Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 6/7] crypto: x86/ghash - limit FPU preemption Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 7/7] crypto: x86 - use common macro for FPU limit Robert Elliott
2022-10-12 21:59 ` [PATCH v2 00/19] crypto: x86 - fix RCU stalls Robert Elliott
2022-10-12 21:59   ` [PATCH v2 01/19] crypto: tcrypt - test crc32 Robert Elliott
2022-10-12 21:59   ` [PATCH v2 02/19] crypto: tcrypt - test nhpoly1305 Robert Elliott
2022-10-12 21:59   ` [PATCH v2 03/19] crypto: tcrypt - reschedule during cycles speed tests Robert Elliott
2022-10-12 21:59   ` [PATCH v2 04/19] crypto: x86/sha - limit FPU preemption Robert Elliott
2022-10-13  0:41     ` Jason A. Donenfeld
2022-10-13 21:50       ` Elliott, Robert (Servers)
2022-10-14 11:01       ` David Laight
2022-10-13  5:57     ` Eric Biggers
2022-10-13  6:04       ` Herbert Xu
2022-10-13  6:08         ` Eric Biggers
2022-10-13  7:50           ` Herbert Xu
2022-10-13 22:41       ` :Re: " Elliott, Robert (Servers)
2022-10-12 21:59   ` [PATCH v2 05/19] crypto: x86/crc " Robert Elliott
2022-10-13  2:00     ` Herbert Xu
2022-10-13 22:34       ` Elliott, Robert (Servers)
2022-10-14  4:02     ` David Laight
2022-10-24  2:03     ` kernel test robot
2022-10-12 21:59   ` [PATCH v2 06/19] crypto: x86/sm3 " Robert Elliott
2022-10-12 21:59   ` [PATCH v2 07/19] crypto: x86/ghash - restructure FPU context saving Robert Elliott
2022-10-12 21:59   ` [PATCH v2 08/19] crypto: x86/ghash - limit FPU preemption Robert Elliott
2022-10-13  6:03     ` Eric Biggers
2022-10-13 22:52       ` Elliott, Robert (Servers)
2022-10-12 21:59   ` [PATCH v2 09/19] crypto: x86 - use common macro for FPU limit Robert Elliott
2022-10-13  0:35     ` Jason A. Donenfeld
2022-10-13 21:48       ` Elliott, Robert (Servers)
2022-10-14  1:26         ` Jason A. Donenfeld
2022-10-18  0:06           ` Elliott, Robert (Servers)
2022-10-12 21:59   ` [PATCH v2 10/19] crypto: x86/sha1, sha256 - load based on CPU features Robert Elliott
2022-10-12 21:59   ` [PATCH v2 11/19] crypto: x86/crc " Robert Elliott
2022-10-12 21:59   ` [PATCH v2 12/19] crypto: x86/sm3 " Robert Elliott
2022-10-12 21:59   ` [PATCH v2 13/19] crypto: x86/ghash " Robert Elliott
2022-10-12 21:59   ` [PATCH v2 14/19] crypto: x86 " Robert Elliott
2022-10-14 14:26     ` Elliott, Robert (Servers)
2022-10-12 21:59   ` [PATCH v2 15/19] crypto: x86 - add pr_fmt to all modules Robert Elliott
2022-10-12 21:59   ` [PATCH v2 16/19] crypto: x86 - print CPU optimized loaded messages Robert Elliott
2022-10-13  0:40     ` Jason A. Donenfeld
2022-10-13 13:47     ` kernel test robot
2022-10-13 13:48     ` kernel test robot
2022-10-12 21:59   ` [PATCH v2 17/19] crypto: x86 - standardize suboptimal prints Robert Elliott
2022-10-13  0:38     ` Jason A. Donenfeld
2022-10-12 21:59   ` [PATCH v2 18/19] crypto: x86 - standardize not loaded prints Robert Elliott
2022-10-13  0:42     ` Jason A. Donenfeld
2022-10-13 22:20       ` Elliott, Robert (Servers)
2022-11-10 22:06         ` Elliott, Robert (Servers)
2022-10-12 21:59   ` [PATCH v2 19/19] crypto: x86/sha - register only the best function Robert Elliott
2022-10-13  6:07     ` Eric Biggers
2022-10-13  7:52       ` Herbert Xu
2022-10-13 22:59         ` Elliott, Robert (Servers)
2022-10-14  8:22           ` Herbert Xu
2022-11-01 21:34   ` [PATCH v2 00/19] crypto: x86 - fix RCU stalls Elliott, Robert (Servers)
2022-11-03  4:27   ` [PATCH v3 00/17] crypt: " Robert Elliott
2022-11-03  4:27     ` [PATCH v3 01/17] crypto: tcrypt - test crc32 Robert Elliott
2022-11-03  4:27     ` [PATCH v3 02/17] crypto: tcrypt - test nhpoly1305 Robert Elliott
2022-11-03  4:27     ` [PATCH v3 03/17] crypto: tcrypt - reschedule during cycles speed tests Robert Elliott
2022-11-03  4:27     ` [PATCH v3 04/17] crypto: x86/sha - limit FPU preemption Robert Elliott
2022-11-03  4:27     ` [PATCH v3 05/17] crypto: x86/crc " Robert Elliott
2022-11-03  4:27     ` [PATCH v3 06/17] crypto: x86/sm3 " Robert Elliott
2022-11-03  4:27     ` [PATCH v3 07/17] crypto: x86/ghash - use u8 rather than char Robert Elliott
2022-11-03  4:27     ` [PATCH v3 08/17] crypto: x86/ghash - restructure FPU context saving Robert Elliott
2022-11-03  4:27     ` [PATCH v3 09/17] crypto: x86/ghash - limit FPU preemption Robert Elliott
2022-11-03  4:27     ` [PATCH v3 10/17] crypto: x86/*poly* " Robert Elliott
2022-11-03  4:27     ` [PATCH v3 11/17] crypto: x86/sha - register all variations Robert Elliott
2022-11-03  9:26       ` kernel test robot
2022-11-03  4:27     ` [PATCH v3 12/17] crypto: x86/sha - minimize time in FPU context Robert Elliott
2022-11-03  4:27     ` [PATCH v3 13/17] crypto: x86/sha1, sha256 - load based on CPU features Robert Elliott
2022-11-03  4:27     ` [PATCH v3 14/17] crypto: x86/crc " Robert Elliott
2022-11-03  4:27     ` [PATCH v3 15/17] crypto: x86/sm3 " Robert Elliott
2022-11-03  4:27     ` [PATCH v3 16/17] crypto: x86/ghash,polyval " Robert Elliott
2022-11-03  4:27     ` [PATCH v3 17/17] crypto: x86/nhpoly1305, poly1305 " Robert Elliott
2022-11-16  4:13     ` [PATCH v4 00/24] crypto: fix RCU stalls Robert Elliott
2022-11-16  4:13       ` [PATCH v4 01/24] crypto: tcrypt - test crc32 Robert Elliott
2022-11-16  4:13       ` [PATCH v4 02/24] crypto: tcrypt - test nhpoly1305 Robert Elliott
2022-11-16  4:13       ` [PATCH v4 03/24] crypto: tcrypt - reschedule during cycles speed tests Robert Elliott
2022-11-16  4:13       ` [PATCH v4 04/24] crypto: x86/sha - limit FPU preemption Robert Elliott
2022-11-16  4:13       ` [PATCH v4 05/24] crypto: x86/crc " Robert Elliott
2022-11-16  4:13       ` [PATCH v4 06/24] crypto: x86/sm3 " Robert Elliott
2022-11-16  4:13       ` [PATCH v4 07/24] crypto: x86/ghash - use u8 rather than char Robert Elliott
2022-11-16  4:13       ` [PATCH v4 08/24] crypto: x86/ghash - restructure FPU context saving Robert Elliott
2022-11-16  4:13       ` [PATCH v4 09/24] crypto: x86/ghash - limit FPU preemption Robert Elliott
2022-11-16  4:13       ` [PATCH v4 10/24] crypto: x86/poly " Robert Elliott
2022-11-16 11:13         ` Jason A. Donenfeld
2022-11-22  5:06           ` Elliott, Robert (Servers)
2022-11-22  9:07             ` David Laight
2022-11-25  8:40           ` Herbert Xu
2022-11-25  8:59             ` Ard Biesheuvel
2022-11-25  9:03               ` Herbert Xu
2022-11-28 16:57                 ` Elliott, Robert (Servers)
2022-11-28 18:48                   ` Elliott, Robert (Servers)
2022-12-02  6:21             ` Elliott, Robert (Servers)
2022-12-02  9:25               ` Herbert Xu
2022-12-02 16:15                 ` Elliott, Robert (Servers)
2022-12-06  4:27                   ` Herbert Xu
2022-12-06 14:03                     ` Peter Lafreniere
2022-12-06 14:44                       ` David Laight
2022-12-06 23:06               ` Peter Lafreniere
2022-12-10  0:34                 ` Elliott, Robert (Servers)
2022-12-16 22:12                   ` Elliott, Robert (Servers)
2022-11-16  4:13       ` [PATCH v4 11/24] crypto: x86/aegis " Robert Elliott
2022-11-16  4:13       ` [PATCH v4 12/24] crypto: x86/sha - register all variations Robert Elliott
2022-11-16  4:13       ` [PATCH v4 13/24] crypto: x86/sha - minimize time in FPU context Robert Elliott
2022-11-16  4:13       ` [PATCH v4 14/24] crypto: x86/sha - load based on CPU features Robert Elliott
2022-11-16  4:13       ` [PATCH v4 15/24] crypto: x86/crc " Robert Elliott
2022-11-16  4:13       ` [PATCH v4 16/24] crypto: x86/sm3 " Robert Elliott
2022-11-16  4:13       ` [PATCH v4 17/24] crypto: x86/poly " Robert Elliott
2022-11-16 11:19         ` Jason A. Donenfeld
2022-11-16  4:13       ` [PATCH v4 18/24] crypto: x86/ghash " Robert Elliott
2022-11-16  4:13       ` [PATCH v4 19/24] crypto: x86/aesni - avoid type conversions Robert Elliott
2022-11-16  4:13       ` [PATCH v4 20/24] crypto: x86/ciphers - load based on CPU features Robert Elliott
2022-11-16 11:30         ` Jason A. Donenfeld
2022-11-16  4:13       ` [PATCH v4 21/24] crypto: x86 - report used CPU features via module parameters Robert Elliott
2022-11-16 11:26         ` Jason A. Donenfeld
2022-11-16  4:13       ` [PATCH v4 22/24] crypto: x86 - report missing " Robert Elliott
2022-11-16  4:13       ` [PATCH v4 23/24] crypto: x86 - report suboptimal CPUs " Robert Elliott
2022-11-16  4:13       ` [PATCH v4 24/24] crypto: x86 - standarize module descriptions Robert Elliott
2022-11-17  3:58       ` [PATCH v4 00/24] crypto: fix RCU stalls Herbert Xu
2022-11-17 15:13         ` Elliott, Robert (Servers)
2022-11-17 15:15           ` Jason A. Donenfeld

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).