Re: [PATCH v1] proc: Implement /proc/self/meminfo

From: ebiederm@xmission.com (Eric W. Biederman)
To: "Enrico Weigelt\, metux IT consult" <lkml@metux.net>
Cc: Chris Down <chris@chrisdown.name>,
	 legion@kernel.org,  LKML <linux-kernel@vger.kernel.org>,
	 Linux Containers <containers@lists.linux.dev>,
	 Linux Containers <containers@lists.linux-foundation.org>,
	 Linux FS Devel <linux-fsdevel@vger.kernel.org>,
	 linux-mm@kvack.org,  Andrew Morton <akpm@linux-foundation.org>,
	 Christian Brauner <christian.brauner@ubuntu.com>,
	 Johannes Weiner <hannes@cmpxchg.org>,
	 Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH v1] proc: Implement /proc/self/meminfo
Date: Wed, 09 Jun 2021 14:14:16 -0500	[thread overview]
Message-ID: <87k0n2am0n.fsf@disp2133> (raw)
In-Reply-To: <b8c86081-503c-3671-2ea3-dd3a0950ce25@metux.net> (Enrico Weigelt's message of "Wed, 9 Jun 2021 10:16:25 +0200")

"Enrico Weigelt, metux IT consult" <lkml@metux.net> writes:

> On 03.06.21 13:33, Chris Down wrote:
>
> Hi folks,
>
>
>> Putting stuff in /proc to get around the problem of "some other metric I need
>> might not be exported to a container" is not a very compelling argument. If
>> they want it, then export it to the container...
>>
>> Ultimately, if they're going to have to add support for a new
>> /proc/self/meminfo file anyway, these use cases should just do it properly
>> through the already supported APIs.
>
> It's even a bit more complex ...
>
> /proc/meminfo always tells what the *machine* has available, not what a
> process can eat up. That has been this way even long before cgroups.
> (eg. ulimits).
>
> Even if you want a container look more like a VM - /proc/meminfo showing
> what the container (instead of the machine) has available - just looking
> at the calling task's cgroup is also wrong. Because there're cgroups
> outside containers (that really shouldn't be affected) and there're even
> other cgroups inside the container (that further restrict below the
> container's limits).
>
> BTW: applications trying to autotune themselves by looking at
> /proc/meminfo are broken-by-design anyways. This never has been a valid
> metric on how much memory invididual processes can or should eat.

Which brings us to the problem.

Using /proc/meminfo is not valid unless your application can know it has
the machine to itself.  Something that is becoming increasing less
common.

Unless something has changed in the last couple of years, reading values
out of the cgroup filesystem is both difficult (v1 and v2 have some
gratuitous differences) and is actively discouraged.

So what should applications do?

Alex has found applications that are trying to do something with
meminfo, and the fields that those applications care about.  I don't see
anyone making the case that specifically what the applications are
trying to do is buggy.

Alex's suggest is to have a /proc/self/meminfo that has the information
that applications want, which would be something that would be easy
to switch applications to.  The patch to userspace at that point is
as simple as 3 lines of code.  I can imagine people take that patch into
their userspace programs.

The simple fact that people are using /proc/meminfo when it doesn't make
sense for anything except system monitoring tools is a pretty solid bug
report on the existing linux apis.

So how do people suggest these applications get the information they
need?

Alex perhaps I missed it, but I didn't see a lot of description on why
the individual applications are using meminfo.  Can you provide a bit
more detail?  At least for a several of them, so we can see the trends
and say yes this kind of information makes sense to provide to
applications.  I think that would help move forward the discussion about
the best way to provide that information to userspace.

Eric