Re: [PATCH 1/6] kmod: add dynamic max concurrent thread count

From: "Luis R. Rodriguez" <mcgrof@kernel.org>
To: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Cc: Tom Gundersen <teg@jklm.no>, Filipe Manana <fdmanana@suse.com>,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	rgoldwyn@suse.com, hare <hare@suse.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	linux-kselftest@vger.kernel.org,
	Andrew Morton <akpm@linux-foundation.org>,
	Dan Williams <dan.j.williams@intel.com>,
	Aaron Tomlin <atomlin@redhat.com>,
	rwright@hpe.com, Heinrich Schuchardt <xypron.glpk@gmx.de>,
	Michal Marek <mmarek@suse.com>,
	martin.wilck@suse.com, Rusty Russell <rusty@rustcorp.com.au>,
	Jeff Mahoney <jeffm@suse.com>, Ingo Molnar <mingo@redhat.com>,
	Petr Mladek <pmladek@suse.com>,
	Guenter Roeck <linux@roeck-us.net>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	shuah@kernel.org, DSterba@suse.com,
	Kees Cook <keescook@chromium.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	Arnaldo Carvalho de Melo <acme@redhat.com>,
	Miroslav Benes <mbenes@suse.cz>, NeilBrown <neilb@suse.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	David Miller <davem@davemloft.net>, Jessica Yu <jeyu@redhat.com>,
	Subash Abhinov Kasiviswanathan <subashab@codeaurora.org>
Subject: Re: [PATCH 1/6] kmod: add dynamic max concurrent thread count
Date: Thu, 25 May 2017 14:38:39 -0700	[thread overview]
Message-ID: <CAB=NE6Xr4bm1FfUZ0+ruegn0W63dVOYbS71SJG94r4z+iACTOQ@mail.gmail.com> (raw)
In-Reply-To: <CAB=NE6V-2hhTmOj4bxQBYSwGknZkauPvvJ0vbN4AQ-OMhkPfTA@mail.gmail.com>

On Thu, May 25, 2017 at 12:01 PM, Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> On Thu, May 25, 2017 at 11:26 AM, Dmitry Torokhov
> <dmitry.torokhov@gmail.com> wrote:
>> On Thu, May 25, 2017 at 08:06:03PM +0200, Luis R. Rodriguez wrote:
>>> On Thu, May 25, 2017 at 10:38:40AM -0700, Luis R. Rodriguez wrote:
>>> > On Thu, May 25, 2017 at 10:30 AM, Dmitry Torokhov
>>> > > There is
>>> > > no slippery slope for systems to move away, no need to backport
>>> > > anything. We seem to agree that a better solution is possible (throttle
>>> > > number of concurrently running modprobes without killing requesters),
>>> > > and with that solution the band-aid will no longer be needed.
>>> > >
>>> > > So please implement and post the proper fix for the issue.
>>> >
>>> > Alright, will do away with this patch and just go for the jugular of the issue.
>>>
>>> I gave this some more thought, even if we go with the throttling right away in
>>> practice you'll end up with a dmesg notice of a throttle kicking in once you *do*
>>
>> So remove it. The warning was meaningful when we rejected requests, now
>> it is not.
>
> Great.
>
>>> reach this. We are forcing only 50 concurrent threads and making this a static
>>> limit with no good reason than 2.3.38 days evaluation from 16 years ago (2000).
>>> If we throttle we are going to throttle with a 2.3.38 days limit. And you
>>> advocate that ?
>>
>> Yes. Can you give me reason why slamming the system with more than 50
>> modprobes is a good idea in 4.12 days? Does the increased limit
>> decreases boot time? By how much?
>
> If in practice we are not hitting the limit the point is moot, and
> when we do I agree we can re-evaluate. With my stress test driver on a
> test case we can push as hard as bringing out the OOM killer even if
> we throttle, fun.

Alright, I don't see these OOMs anymore *after* I actually nuked that
patch which incremented the kmod limit. The reason we can OOM is
finit_module() can consume gobs of memory, the current value then
seems fair, and work well for my tests provided we do use the proper
throttle. Will respin.

  Luis