All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC 00/10] kmod: stress test driver, few fixes and enhancements
@ 2016-12-08 18:47 Luis R. Rodriguez
  2016-12-08 18:47 ` [RFC 01/10] kmod: add test driver to stress test the module loader Luis R. Rodriguez
                   ` (10 more replies)
  0 siblings, 11 replies; 65+ messages in thread
From: Luis R. Rodriguez @ 2016-12-08 18:47 UTC (permalink / raw)
  To: shuah, jeyu, rusty, ebiederm, dmitry.torokhov, acme, corbet
  Cc: martin.wilck, mmarek, pmladek, hare, rwright, jeffm, DSterba,
	fdmanana, neilb, rgoldwyn, subashab, xypron.glpk, keescook,
	atomlin, mbenes, paulmck, dan.j.williams, jpoimboe, davem, mingo,
	akpm, torvalds, linux-kselftest, linux-doc, linux-kernel,
	Luis R. Rodriguez

Upon running into an old kmod v19 issue with mount (get_fs_type()) a few of us
hunted for the cause of the issue. Although the issue ended up being a
userspace issue, a stress test driver was written to help reproduce the issue,
and along the way a few other fixes and sanity checks were implemented.

I've taken the time to generalize the stress test driver as a kselftest driver
with a 9 test cases. The last two test cases reveal an existing issue which
is not yet addressed upstream, even if you have kmod v19 present. A fix is
proposed in the last patch. Orignally we had discarded this patch as too
complex due to the alias handling, but upon further analysis of test cases
and memory pressure issues, it seems worth considering. Other than the
last patch I don't think much of the other patches are controversial, but
sending as RFC first just in case.

If its not clear, an end goal here is to make module loading a bit more
deterministic with stronger sanity checks and stress tests. Please note,
the stress test diver requires 4 GiB of RAM to run all tests without running
out of memory. A lot of this has to do with the memory requirements needed
for a dynamic test for multiple threads, but note that the final memory
pressure and OOMs actually don't come from this allocation, but instead
from many finit_module() calls, this consumes quite a bit of memory, specially
if you have a lot of dependencies which also need to be loaded prior to
your needed module -- as is the case for filesystem drivers.

These patches are available on my linux-next git-tree on my branch
20161208-kmod-test-driver-try2 [0], which is based on linux-next tag
next-20161208. Patches are also available based on v4.9-rc8 [1] for
those looking for a bit more stable tree given x86_64 on linux-next is
hosed at the moment.

Since kmod.c doesn't seem to get much love, and since I've been digging
quite a bit into it for other users (firmware) I suppose I could volunteer
myself to maintain this code as well, unless there are oppositions to this.

[0] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git/log/?h=20161208-kmod-test-driver-try2
[1] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux.git/log/?h=20161208-kmod-test-driver

Luis R. Rodriguez (10):
  kmod: add test driver to stress test the module loader
  module: fix memory leak on early load_module() failures
  kmod: add dynamic max concurrent thread count
  kmod: provide wrappers for kmod_concurrent inc/dec
  kmod: return -EBUSY if modprobe limit is reached
  kmod: provide sanity check on kmod_concurrent access
  kmod: use simplified rate limit printk
  sysctl: add support for unsigned int properly
  kmod: add helpers for getting kmod count and limit
  kmod: add a sanity check on module loading

 Documentation/admin-guide/kernel-parameters.txt |    7 +
 include/linux/kmod.h                            |    9 +
 include/linux/sysctl.h                          |    3 +
 init/Kconfig                                    |   23 +
 init/main.c                                     |    1 +
 kernel/kmod.c                                   |  244 ++++-
 kernel/module.c                                 |   12 +-
 kernel/sysctl.c                                 |  198 +++-
 lib/Kconfig.debug                               |   25 +
 lib/Makefile                                    |    1 +
 lib/test_kmod.c                                 | 1248 +++++++++++++++++++++++
 tools/testing/selftests/kmod/Makefile           |   11 +
 tools/testing/selftests/kmod/config             |    7 +
 tools/testing/selftests/kmod/kmod.sh            |  448 ++++++++
 14 files changed, 2199 insertions(+), 38 deletions(-)
 create mode 100644 lib/test_kmod.c
 create mode 100644 tools/testing/selftests/kmod/Makefile
 create mode 100644 tools/testing/selftests/kmod/config
 create mode 100755 tools/testing/selftests/kmod/kmod.sh

-- 
2.10.1

^ permalink raw reply	[flat|nested] 65+ messages in thread

end of thread, other threads:[~2017-05-18  2:37 UTC | newest]

Thread overview: 65+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-08 18:47 [RFC 00/10] kmod: stress test driver, few fixes and enhancements Luis R. Rodriguez
2016-12-08 18:47 ` [RFC 01/10] kmod: add test driver to stress test the module loader Luis R. Rodriguez
2016-12-08 20:24   ` Kees Cook
2016-12-13 21:10     ` Luis R. Rodriguez
2016-12-16  7:41       ` Luis R. Rodriguez
2016-12-08 19:48 ` [RFC 02/10] module: fix memory leak on early load_module() failures Luis R. Rodriguez
2016-12-08 20:30   ` Kees Cook
2016-12-08 21:10     ` Luis R. Rodriguez
2016-12-08 21:17       ` Kees Cook
2016-12-09 17:06   ` Miroslav Benes
2016-12-16  8:51     ` Luis R. Rodriguez
2016-12-15 18:46   ` Aaron Tomlin
2016-12-08 19:48 ` [RFC 03/10] kmod: add dynamic max concurrent thread count Luis R. Rodriguez
2016-12-08 20:28   ` Kees Cook
2016-12-08 21:00     ` Luis R. Rodriguez
2016-12-14 15:38   ` Petr Mladek
2016-12-16  8:39     ` Luis R. Rodriguez
2017-01-10 19:24       ` Luis R. Rodriguez
2016-12-08 19:48 ` [RFC 04/10] kmod: provide wrappers for kmod_concurrent inc/dec Luis R. Rodriguez
2016-12-08 20:29   ` Kees Cook
2016-12-08 21:08     ` Luis R. Rodriguez
2016-12-15 12:46       ` Petr Mladek
2016-12-16  8:05         ` Luis R. Rodriguez
2016-12-22  4:48           ` Jessica Yu
2017-01-06 20:54             ` Luis R. Rodriguez
2017-01-10 18:57           ` [RFC 04/10] " Luis R. Rodriguez
2017-01-11 20:08             ` Luis R. Rodriguez
2017-05-16 18:02               ` Luis R. Rodriguez
2017-05-18  2:37                 ` Luis R. Rodriguez
2016-12-22  5:07   ` Jessica Yu
2017-01-10 20:28     ` Luis R. Rodriguez
2016-12-08 19:48 ` [RFC 05/10] kmod: return -EBUSY if modprobe limit is reached Luis R. Rodriguez
2016-12-08 19:48 ` [RFC 06/10] kmod: provide sanity check on kmod_concurrent access Luis R. Rodriguez
2016-12-14 16:08   ` Petr Mladek
2016-12-14 17:12     ` Luis R. Rodriguez
2016-12-15 12:57   ` Petr Mladek
2017-01-10 20:00     ` Luis R. Rodriguez
2016-12-08 19:49 ` [RFC 07/10] kmod: use simplified rate limit printk Luis R. Rodriguez
2016-12-14 16:23   ` Petr Mladek
2016-12-14 16:41     ` Joe Perches
2016-12-16  8:44     ` Luis R. Rodriguez
2016-12-08 19:49 ` [RFC 08/10] sysctl: add support for unsigned int properly Luis R. Rodriguez
2016-12-08 19:49 ` [RFC 09/10] kmod: add helpers for getting kmod count and limit Luis R. Rodriguez
2016-12-15 16:56   ` Petr Mladek
2016-12-16  7:57     ` Luis R. Rodriguez
2017-01-11 18:27       ` Luis R. Rodriguez
2016-12-08 19:49 ` [RFC 10/10] kmod: add a sanity check on module loading Luis R. Rodriguez
2016-12-09 20:03   ` Martin Wilck
2016-12-09 20:56     ` Linus Torvalds
2016-12-15 18:08       ` Luis R. Rodriguez
2016-12-15  0:27   ` Rusty Russell
2016-12-16  8:31     ` Luis R. Rodriguez
2016-12-17  3:54       ` Rusty Russell
     [not found]         ` <CAB=NE6VvuA9a6hf6yoopGfUxVJQM5HyV5bNzUdsEtUV0UhbG-g@mail.gmail.com>
2016-12-20  0:53           ` Rusty Russell
2016-12-20 18:52             ` Luis R. Rodriguez
2016-12-21  2:21               ` Rusty Russell
2016-12-21 13:08                 ` Luis R. Rodriguez
2017-01-03  0:04                   ` Rusty Russell
2017-01-06 20:36                     ` Luis R. Rodriguez
2017-01-06 21:53                       ` Jessica Yu
2017-01-09 20:27                         ` Luis R. Rodriguez
     [not found]                       ` <87bmvgax51.fsf@rustcorp.com.au>
2017-01-09 19:56                         ` [RFC 10/10] " Luis R. Rodriguez
2017-01-06 21:03                     ` Jessica Yu
2017-01-04  2:47   ` Jessica Yu
2017-01-11 19:10 ` [RFC 00/10] kmod: stress test driver, few fixes and enhancements Luis R. Rodriguez

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.