LKML Archive on lore.kernel.org
 help / color / Atom feed
From: "Luis R. Rodriguez" <mcgrof@kernel.org>
To: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Cc: Tom Gundersen <teg@jklm.no>, Filipe Manana <fdmanana@suse.com>,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	rgoldwyn@suse.com, hare <hare@suse.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	linux-kselftest@vger.kernel.org,
	Andrew Morton <akpm@linux-foundation.org>,
	Dan Williams <dan.j.williams@intel.com>,
	Aaron Tomlin <atomlin@redhat.com>,
	rwright@hpe.com, Heinrich Schuchardt <xypron.glpk@gmx.de>,
	Michal Marek <mmarek@suse.com>,
	martin.wilck@suse.com, Rusty Russell <rusty@rustcorp.com.au>,
	Jeff Mahoney <jeffm@suse.com>, Ingo Molnar <mingo@redhat.com>,
	Petr Mladek <pmladek@suse.com>,
	Guenter Roeck <linux@roeck-us.net>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	shuah@kernel.org, DSterba@suse.com,
	Kees Cook <keescook@chromium.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	Arnaldo Carvalho de Melo <acme@redhat.com>,
	Miroslav Benes <mbenes@suse.cz>, NeilBrown <neilb@suse.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	David Miller <davem@davemloft.net>, Jessica Yu <jeyu@redhat.com>,
	Subash Abhinov Kasiviswanathan <subashab@codeaurora.org>
Subject: Re: [PATCH 1/6] kmod: add dynamic max concurrent thread count
Date: Thu, 25 May 2017 14:38:39 -0700
Message-ID: <CAB=NE6Xr4bm1FfUZ0+ruegn0W63dVOYbS71SJG94r4z+iACTOQ@mail.gmail.com> (raw)
In-Reply-To: <CAB=NE6V-2hhTmOj4bxQBYSwGknZkauPvvJ0vbN4AQ-OMhkPfTA@mail.gmail.com>

On Thu, May 25, 2017 at 12:01 PM, Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> On Thu, May 25, 2017 at 11:26 AM, Dmitry Torokhov
> <dmitry.torokhov@gmail.com> wrote:
>> On Thu, May 25, 2017 at 08:06:03PM +0200, Luis R. Rodriguez wrote:
>>> On Thu, May 25, 2017 at 10:38:40AM -0700, Luis R. Rodriguez wrote:
>>> > On Thu, May 25, 2017 at 10:30 AM, Dmitry Torokhov
>>> > > There is
>>> > > no slippery slope for systems to move away, no need to backport
>>> > > anything. We seem to agree that a better solution is possible (throttle
>>> > > number of concurrently running modprobes without killing requesters),
>>> > > and with that solution the band-aid will no longer be needed.
>>> > >
>>> > > So please implement and post the proper fix for the issue.
>>> >
>>> > Alright, will do away with this patch and just go for the jugular of the issue.
>>>
>>> I gave this some more thought, even if we go with the throttling right away in
>>> practice you'll end up with a dmesg notice of a throttle kicking in once you *do*
>>
>> So remove it. The warning was meaningful when we rejected requests, now
>> it is not.
>
> Great.
>
>>> reach this. We are forcing only 50 concurrent threads and making this a static
>>> limit with no good reason than 2.3.38 days evaluation from 16 years ago (2000).
>>> If we throttle we are going to throttle with a 2.3.38 days limit. And you
>>> advocate that ?
>>
>> Yes. Can you give me reason why slamming the system with more than 50
>> modprobes is a good idea in 4.12 days? Does the increased limit
>> decreases boot time? By how much?
>
> If in practice we are not hitting the limit the point is moot, and
> when we do I agree we can re-evaluate. With my stress test driver on a
> test case we can push as hard as bringing out the OOM killer even if
> we throttle, fun.

Alright, I don't see these OOMs anymore *after* I actually nuked that
patch which incremented the kmod limit. The reason we can OOM is
finit_module() can consume gobs of memory, the current value then
seems fair, and work well for my tests provided we do use the proper
throttle. Will respin.

  Luis

  reply index

Thread overview: 69+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-05-19  3:24 [PATCH 0/6] kmod: few simple enhancements Luis R. Rodriguez
2017-05-19  3:24 ` [PATCH 1/6] kmod: add dynamic max concurrent thread count Luis R. Rodriguez
2017-05-19 20:44   ` Dmitry Torokhov
     [not found]     ` <CAB=NE6XGL24O+JfTNUG0HO4obhDc-v+HyL0SCrQELiZrj2-qNw@mail.gmail.com>
     [not found]       ` <CAB=NE6Wa4Nemh80yaCCwbjrNRLPD+GJMncg12APg9Vq63AWVng@mail.gmail.com>
     [not found]         ` <CAB=NE6Vc6RDAytn2Pkv2V58HFo8ncR0eOHZ3===kbZ2NF78ubg@mail.gmail.com>
     [not found]           ` <CAB=NE6Vqmx=y6muenpuQKynTP=pGWMF8tzoCA0BXD6d63q9wPg@mail.gmail.com>
2017-05-19 21:58             ` Dmitry Torokhov
2017-05-25 16:22               ` Luis R. Rodriguez
2017-05-25 16:38                 ` Dmitry Torokhov
2017-05-25 16:50                   ` Luis R. Rodriguez
2017-05-25 17:30                     ` Dmitry Torokhov
2017-05-25 17:38                       ` Luis R. Rodriguez
2017-05-25 18:06                         ` Luis R. Rodriguez
2017-05-25 18:26                           ` Dmitry Torokhov
2017-05-25 19:01                             ` Luis R. Rodriguez
2017-05-25 21:38                               ` Luis R. Rodriguez [this message]
2017-05-19  3:24 ` [PATCH 2/6] module: use list_for_each_entry_rcu() on find_module_all() Luis R. Rodriguez
2017-05-23 14:46   ` Miroslav Benes
2017-05-19  3:24 ` [PATCH 3/6] kmod: provide wrappers for kmod_concurrent inc/dec Luis R. Rodriguez
2017-05-19  3:24 ` [PATCH 4/6] kmod: return -EBUSY if modprobe limit is reached Luis R. Rodriguez
2017-05-19  3:24 ` [PATCH 5/6] kmod: preempt on kmod_umh_threads_get() Luis R. Rodriguez
2017-05-19 22:27   ` Dmitry Torokhov
2017-05-25  0:14     ` Luis R. Rodriguez
2017-05-25  0:45       ` Dmitry Torokhov
2017-05-25  1:00         ` Luis R. Rodriguez
2017-05-25  2:27           ` Dmitry Torokhov
2017-05-25 11:19             ` Petr Mladek
2017-05-25 15:38               ` Luis R. Rodriguez
2017-05-25 16:42               ` Dmitry Torokhov
2017-05-25 15:18             ` Jessica Yu
2017-05-19  3:24 ` [PATCH 6/6] kmod: use simplified rate limit printk Luis R. Rodriguez
2017-05-19 22:23   ` Dmitry Torokhov
2017-05-23  9:00     ` Petr Mladek
2017-05-26  0:16 ` [PATCH v2 0/5] kmod: help make deterministic Luis R. Rodriguez
2017-05-26  0:16   ` [PATCH v2 1/5] module: use list_for_each_entry_rcu() on find_module_all() Luis R. Rodriguez
2017-05-26  0:16   ` [PATCH v2 2/5] kmod: reduce atomic operations on kmod_concurrent Luis R. Rodriguez
2017-05-26  1:11     ` Dmitry Torokhov
2017-05-26 20:03       ` Luis R. Rodriguez
2017-05-26  0:16   ` [PATCH v2 3/5] kmod: add test driver to stress test the module loader Luis R. Rodriguez
2017-05-26  0:16   ` [PATCH v2 4/5] kmod: add helpers for getting kmod limit Luis R. Rodriguez
2017-05-26  0:56     ` Dmitry Torokhov
2017-05-26 20:27       ` Luis R. Rodriguez
2017-05-26  0:16   ` [PATCH v2 5/5] kmod: throttle kmod thread limit Luis R. Rodriguez
2017-05-26 21:12   ` [PATCH v3 0/4] kmod: help make deterministic Luis R. Rodriguez
2017-05-26 21:12     ` [PATCH v3 1/4] module: use list_for_each_entry_rcu() on find_module_all() Luis R. Rodriguez
2017-05-26 21:12     ` [PATCH v3 2/4] kmod: reduce atomic operations on kmod_concurrent and simplify Luis R. Rodriguez
2017-06-23 19:19       ` [PATCH v4 " Luis R. Rodriguez
2017-06-26 11:36       ` [PATCH v3 " Petr Mladek
2017-05-26 21:12     ` [PATCH v3 3/4] kmod: add test driver to stress test the module loader Luis R. Rodriguez
2017-05-26 21:12     ` [PATCH v3 4/4] kmod: throttle kmod thread limit Luis R. Rodriguez
2017-06-22 15:19       ` Petr Mladek
2017-06-23 16:16         ` Luis R. Rodriguez
2017-06-23 17:56           ` Luis R. Rodriguez
2017-06-23 19:16             ` Luis R. Rodriguez
2017-06-26 10:03               ` Petr Mladek
2017-06-26  9:55           ` Petr Mladek
2017-06-23 19:20       ` [PATCH v4 " Luis R. Rodriguez
2017-06-26 11:38         ` Petr Mladek
2017-06-28 22:11           ` Luis R. Rodriguez
2017-06-20 20:56     ` [PATCH v3 0/4] kmod: help make deterministic Luis R. Rodriguez
2017-06-21  0:23       ` Kees Cook
2017-06-26 21:37         ` Jessica Yu
2017-06-26 22:44           ` Luis R. Rodriguez
2017-06-27  0:27             ` Luis R. Rodriguez
2017-06-27  8:13               ` Petr Mladek
2017-06-27 10:04                 ` Jessica Yu
2017-06-27 15:26             ` Jessica Yu
2017-06-28  0:49               ` Luis R. Rodriguez
2017-06-28 22:31     ` [PATCH v4 0/3] " Luis R. Rodriguez
2017-06-28 22:31       ` [PATCH v4 1/3] MAINTAINERS: give kmod some maintainer love Luis R. Rodriguez
2017-06-28 22:31       ` [PATCH v4 2/3] kmod: add test driver to stress test the module loader Luis R. Rodriguez
2017-06-28 22:31       ` [PATCH v4 3/3] kmod: throttle kmod thread limit Luis R. Rodriguez

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAB=NE6Xr4bm1FfUZ0+ruegn0W63dVOYbS71SJG94r4z+iACTOQ@mail.gmail.com' \
    --to=mcgrof@kernel.org \
    --cc=DSterba@suse.com \
    --cc=acme@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=atomlin@redhat.com \
    --cc=corbet@lwn.net \
    --cc=dan.j.williams@intel.com \
    --cc=davem@davemloft.net \
    --cc=dmitry.torokhov@gmail.com \
    --cc=ebiederm@xmission.com \
    --cc=fdmanana@suse.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hare@suse.com \
    --cc=jeffm@suse.com \
    --cc=jeyu@redhat.com \
    --cc=jpoimboe@redhat.com \
    --cc=keescook@chromium.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux@roeck-us.net \
    --cc=martin.wilck@suse.com \
    --cc=mbenes@suse.cz \
    --cc=mingo@redhat.com \
    --cc=mmarek@suse.com \
    --cc=neilb@suse.com \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=pmladek@suse.com \
    --cc=rgoldwyn@suse.com \
    --cc=rusty@rustcorp.com.au \
    --cc=rwright@hpe.com \
    --cc=shuah@kernel.org \
    --cc=subashab@codeaurora.org \
    --cc=teg@jklm.no \
    --cc=torvalds@linux-foundation.org \
    --cc=xypron.glpk@gmx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git