linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Tim Hockin <thockin@hockin.org>
To: Tejun Heo <tj@kernel.org>
Cc: "Austin S Hemmelgarn" <ahferroin7@gmail.com>,
	"Aleksa Sarai" <cyphar@cyphar.com>,
	"Li Zefan" <lizefan@huawei.com>, mingo <mingo@redhat.com>,
	"Peter Zijlstra" <peterz@infradead.org>, richard <richard@nod.at>,
	"Frédéric Weisbecker" <fweisbec@gmail.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Cgroups <cgroups@vger.kernel.org>
Subject: Re: [PATCH RFC 0/2] add nproc cgroup subsystem
Date: Fri, 27 Feb 2015 13:45:09 -0800	[thread overview]
Message-ID: <CAAAKZwvuR2X7EEZz+UWRCGYajaJQ+VzXYfDEq6EQnssv9+6FJQ@mail.gmail.com> (raw)
In-Reply-To: <20150227174503.GM3964@htj.duckdns.org>

On Fri, Feb 27, 2015 at 9:45 AM, Tejun Heo <tj@kernel.org> wrote:
> On Fri, Feb 27, 2015 at 09:25:10AM -0800, Tim Hockin wrote:
>> > In general, I'm pretty strongly against adding controllers for things
>> > which aren't fundamental resources in the system.  What's next?  Open
>> > files?  Pipe buffer?  Number of flocks?  Number of session leaders or
>> > program groups?
>>
>> Yes to some or all of those.  We do exactly this internally and it has
>> greatly added to the stability of our overall container management
>> system.  and while you have been telling everyone to wait for kmemcg,
>> we have had an extra 3+ years of stability.
>
> Yeah, good job.  I totally get why kernel part of memory consumption
> needs protection.  I'm not arguing against that at all.

You keep shifting the focus to be about memory, but that's not what
people are asking for.  You're letting the desire for a perfect
solution (which is years late) block good solutions that exist NOW.

>> > If you want to prevent a certain class of jobs from exhausting a given
>> > resource, protecting that resource is the obvious thing to do.
>>
>> I don't follow your argument - isn't this exactly what this patch set
>> is doing - protecting resources?
>
> If you have proper protection over kernel memory consumption, this is
> completely covered because memory is the fundamental resource here.
> Controlling distribution of those fundamental resources is what
> cgroups are primarily about.

You say that's what cgroups are about, but it's not at all obvious
that you are right.  What users, admins, systems people want is
building blocks that are usable and make sense.  Limiting kernel
memory is NOT the logical building block, here.  It's not something
people can reason about or quantify easily.  if you need to implement
the interfaces in terms of memory, go nuts, but making users think
liek that is just not right.

>> > Wasn't it like a year ago?  Yeah, it's taking longer than everybody
>> > hoped but seriously kmemcg reclaimer just got merged and also did the
>> > new memcg interface which will tie kmemcg and memcg together.
>>
>> By my email it was almost 2 years ago, and that was the second or
>> third incarnation of this patch.
>
> Again, I agree this is taking a while.  Memory people had to retool
> the whole reclamation path to make this work, which is the pattern
> being repeated across the different controllers - we're refactoring a
> lot of infrastructure code so that resource control can integrate with
> the regular operation of the kernel, which BTW is what we should have
> been doing from the beginning.
>
> If your complaint is that this is taking too long, I hear you, and
> there's a certain amount of validity in arguing that upstreaming a
> temporary measure is the better trade-off, but the rationale for nproc
> (or nfds, or virtual memory, whatever) has been pretty weak otherwise.

At least 3 or 4 people have INDEPENDENTLY decided this is what is
causing them pain and tried to fix it and invested the time to send a
patch says that it is actually a thing.  There exists a problem that
you are disallowing to be fixed.  Do you recognize that users are
experiencing pain?  Why do you hate your users? :)

> And as for the different incarnations of this patchset.  Reposting the
> same stuff repeatedly doesn't really change anything.  Why would it?

Because reasonable people might survey the ecosystem and say "humm,
things have changed over the years - isolation has become a pretty
serious topic".  or maybe they hope that you'll finally agree that
fixing the problem NOW is worthwhile, even if the solution is
imperfect, and that a more perfect solution will arrive.

>> >> Something like this is long overdue, IMO, and is still more
>> >> appropriate and obvious than kmemcg anyway.
>> >
>> > Thanks for chiming in again but if you aren't bringing out anything
>> > new to the table (I don't remember you doing that last time either),
>> > I'm not sure why the decision would be different this time.
>>
>> I'm just vocalizing my support for this idea in defense of practical
>> solutions that work NOW instead of "engineering ideals" that never
>> actually arrive.
>>
>> As containers take the server world by storm, stuff like this gets
>> more and more important.
>
> Again, protection of kernel side memory consumption is important.
> There's no question about that.  As for the never-arriving part, well,
> it is arriving.  If you still can't believe, just take a look at the
> code.

Are you willing to put a drop-dead date on it?  If we don't have
kmemcg working well enough to _actually_ bound PID usage and FD usage
by, say, June 1st, will you then accept a patch to this effect?  If
the answer is no, then I have zero faith that it's coming any time
soon - I heard this 2 years ago.  I believed you then.

I see further downthread that you said you'll think about it.  Thank
you.  Just because our use cases are not normal does not mean we're
not valid :)

Tim

  parent reply	other threads:[~2015-02-27 21:45 UTC|newest]

Thread overview: 65+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-02-23  3:08 [PATCH RFC 0/2] add nproc cgroup subsystem Aleksa Sarai
2015-02-23  3:08 ` [PATCH RFC 1/2] cgroups: allow a cgroup subsystem to reject a fork Aleksa Sarai
2015-02-23 14:49   ` Peter Zijlstra
2015-02-23  3:08 ` [PATCH RFC 2/2] cgroups: add an nproc subsystem Aleksa Sarai
2015-02-27  4:17 ` [RFC PATCH v2 0/2] add nproc cgroup subsystem Aleksa Sarai
2015-02-27  4:17   ` [PATCH v2 1/2] cgroups: allow a cgroup subsystem to reject a fork Aleksa Sarai
2015-03-09  3:06     ` Tejun Heo
     [not found]       ` <CAOviyaip7Faz98YWzGoTaXGYVb72sfD+ZL4Xa89reU9+=43jFA@mail.gmail.com>
     [not found]         ` <20150309065902.GP13283@htj.duckdns.org>
2015-03-10  8:19           ` Aleksa Sarai
2015-03-10 12:47             ` Tejun Heo
2015-03-10 14:51               ` Aleksa Sarai
2015-03-10 15:17                 ` Tejun Heo
2015-03-11  5:16                   ` Aleksa Sarai
2015-03-11 11:46                     ` Tejun Heo
2015-03-11 23:47           ` Aleksa Sarai
2015-03-12  1:25             ` Tejun Heo
2015-02-27  4:17   ` [PATCH v2 2/2] cgroups: add an nproc subsystem Aleksa Sarai
2015-03-02 15:22     ` Tejun Heo
2015-03-09  1:49       ` Zefan Li
2015-03-09  2:34         ` Tejun Heo
2015-02-27 11:49 ` [PATCH RFC 0/2] add nproc cgroup subsystem Tejun Heo
2015-02-27 13:46   ` Richard Weinberger
2015-02-27 13:52     ` Tejun Heo
2015-02-27 16:42   ` Austin S Hemmelgarn
2015-02-27 17:06     ` Tejun Heo
2015-02-27 17:25       ` Tim Hockin
2015-02-27 17:45         ` Tejun Heo
2015-02-27 17:56           ` Tejun Heo
2015-02-27 21:45           ` Tim Hockin [this message]
2015-02-27 21:49             ` Tejun Heo
     [not found]               ` <CAAAKZwsCc8BtFx58KMFpRTohU81oCBeGVOPGMJrjJt9q5upKfQ@mail.gmail.com>
2015-02-28 16:57                 ` Tejun Heo
2015-02-28 22:26                   ` Tim Hockin
2015-02-28 22:50                     ` Tejun Heo
2015-03-01  4:46                       ` Tim Hockin
2015-02-28 23:11                     ` Johannes Weiner
2015-02-27 18:49       ` Austin S Hemmelgarn
2015-02-27 19:35         ` Tejun Heo
2015-02-28  9:26         ` Aleksa Sarai
2015-02-28 11:59           ` Tejun Heo
     [not found]             ` <CAAAKZws45c3PhFQMGrm_K+OZV+KOyGV9sXTakHcTfNP1kHxzOQ@mail.gmail.com>
2015-02-28 16:43               ` Tejun Heo
2015-03-02 13:13                 ` Austin S Hemmelgarn
2015-03-02 13:31                   ` Aleksa Sarai
2015-03-02 13:54                     ` Tejun Heo
2015-03-02 13:49                   ` Tejun Heo
2015-02-27 17:12     ` Tim Hockin
2015-02-27 17:15       ` Tejun Heo
2015-03-04 20:23 ` [PATCH v3 0/2] cgroup: add pids subsystem Aleksa Sarai
2015-03-04 20:23   ` [PATCH v3 1/2] cgroups: allow a cgroup subsystem to reject a fork Aleksa Sarai
2015-03-04 20:23   ` [PATCH v3 2/2] cgroups: add a pids subsystem Aleksa Sarai
2015-03-05  8:39     ` Aleksa Sarai
2015-03-05 14:37     ` Marian Marinov
2015-03-06  1:45 ` [PATCH v4 0/2] cgroup: add " Aleksa Sarai
2015-03-06  1:45   ` [PATCH v4 1/2] cgroups: allow a cgroup subsystem to reject a fork Aleksa Sarai
2015-03-06  1:45   ` [PATCH v4 2/2] cgroups: add a pids subsystem Aleksa Sarai
2015-03-09  3:34     ` Tejun Heo
2015-03-09  3:39       ` Tejun Heo
2015-03-09 18:58       ` Austin S Hemmelgarn
2015-03-09 19:51         ` Tejun Heo
2015-03-10  8:10         ` Aleksa Sarai
2015-03-10 11:32           ` Austin S Hemmelgarn
2015-03-10 12:31             ` Aleksa Sarai
2015-03-11 15:13               ` Austin S Hemmelgarn
2015-03-12  2:28                 ` Aleksa Sarai
2015-03-12 15:35                   ` Austin S Hemmelgarn
2015-03-12  3:47                 ` Tejun Heo
2015-03-09  3:08   ` [PATCH v4 0/2] cgroup: add " Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAAAKZwvuR2X7EEZz+UWRCGYajaJQ+VzXYfDEq6EQnssv9+6FJQ@mail.gmail.com \
    --to=thockin@hockin.org \
    --cc=ahferroin7@gmail.com \
    --cc=cgroups@vger.kernel.org \
    --cc=cyphar@cyphar.com \
    --cc=fweisbec@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizefan@huawei.com \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=richard@nod.at \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).