Re: [PATCH] proc/meminfo: add MemKernel counter

[Konstantin Khlebnikov <khlebnikov@yandex-team.ru>]

On 22.05.2019 18:01, Vlastimil Babka wrote:
> On 5/22/19 4:40 PM, Konstantin Khlebnikov wrote:
>> Some kinds of kernel allocations are not accounted or not show in meminfo.
>> For example vmalloc allocations are tracked but overall size is not shown
> 
> I think Roman's vmalloc patch [1] is on its way?
> 
>> for performance reasons. There is no information about network buffers.
> 
> xfs buffers can also occupy a lot, from my experience
> 
>> In most cases detailed statistics is not required. At first place we need
>> information about overall kernel memory usage regardless of its structure.
>>
>> This patch estimates kernel memory usage by subtracting known sizes of
>> free, anonymous, hugetlb and caches from total memory size: MemKernel =
>> MemTotal - MemFree - Buffers - Cached - SwapCached - AnonPages - Hugetlb.
>>
>> Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
> 
> I've tried this once in [2]. The name was Unaccounted and one of the objections
> was that people would get worried. Yours is a bit better, perhaps MemKernMisc
> would be even more descriptive? Michal Hocko worried about maintainability, that
> we forget something, but I don't think that's a big issue.

I've started with Misc/Unaccounted too
https://lore.kernel.org/lkml/155792098821.1536.17069603544573830315.stgit@buzz/

But this version simply shows all kernel memory.

> 
> Vlastimil
> 
> [1] https://lore.kernel.org/linux-mm/20190514235111.2817276-2-guro@fb.com/T/#u
> [2] https://lore.kernel.org/linux-mm/20161020121149.9935-1-vbabka@suse.cz/T/#u
> 
>> ---
>>   Documentation/filesystems/proc.txt |    5 +++++
>>   fs/proc/meminfo.c                  |   20 +++++++++++++++-----
>>   2 files changed, 20 insertions(+), 5 deletions(-)
>>
>> diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
>> index 66cad5c86171..a0ab7f273ea0 100644
>> --- a/Documentation/filesystems/proc.txt
>> +++ b/Documentation/filesystems/proc.txt
>> @@ -860,6 +860,7 @@ varies by architecture and compile options.  The following is from a
>>   
>>   MemTotal:     16344972 kB
>>   MemFree:      13634064 kB
>> +MemKernel:      862600 kB
>>   MemAvailable: 14836172 kB
>>   Buffers:          3656 kB
>>   Cached:        1195708 kB
>> @@ -908,6 +909,10 @@ MemAvailable: An estimate of how much memory is available for starting new
>>                 page cache to function well, and that not all reclaimable
>>                 slab will be reclaimable, due to items being in use. The
>>                 impact of those factors will vary from system to system.
>> +   MemKernel: The sum of all kinds of kernel memory allocations: Slab,
>> +              Vmalloc, Percpu, KernelStack, PageTables, socket buffers,
>> +              and some other untracked allocations. Does not include
>> +              MemFree, Buffers, Cached, SwapCached, AnonPages, Hugetlb.
>>        Buffers: Relatively temporary storage for raw disk blocks
>>                 shouldn't get tremendously large (20MB or so)
>>         Cached: in-memory cache for files read from the disk (the
>> diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
>> index 568d90e17c17..b27d56dd619a 100644
>> --- a/fs/proc/meminfo.c
>> +++ b/fs/proc/meminfo.c
>> @@ -39,17 +39,27 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
>>   	long available;
>>   	unsigned long pages[NR_LRU_LISTS];
>>   	unsigned long sreclaimable, sunreclaim;
>> +	unsigned long anon_pages, file_pages, swap_cached;
>> +	long kernel_pages;
>>   	int lru;
>>   
>>   	si_meminfo(&i);
>>   	si_swapinfo(&i);
>>   	committed = percpu_counter_read_positive(&vm_committed_as);
>>   
>> -	cached = global_node_page_state(NR_FILE_PAGES) -
>> -			total_swapcache_pages() - i.bufferram;
>> +	anon_pages = global_node_page_state(NR_ANON_MAPPED);
>> +	file_pages = global_node_page_state(NR_FILE_PAGES);
>> +	swap_cached = total_swapcache_pages();
>> +
>> +	cached = file_pages - swap_cached - i.bufferram;
>>   	if (cached < 0)
>>   		cached = 0;
>>   
>> +	kernel_pages = i.totalram - i.freeram - anon_pages - file_pages -
>> +		       hugetlb_total_pages();
>> +	if (kernel_pages < 0)
>> +		kernel_pages = 0;
>> +
>>   	for (lru = LRU_BASE; lru < NR_LRU_LISTS; lru++)
>>   		pages[lru] = global_node_page_state(NR_LRU_BASE + lru);
>>   
>> @@ -60,9 +70,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
>>   	show_val_kb(m, "MemTotal:       ", i.totalram);
>>   	show_val_kb(m, "MemFree:        ", i.freeram);
>>   	show_val_kb(m, "MemAvailable:   ", available);
>> +	show_val_kb(m, "MemKernel:      ", kernel_pages);
>>   	show_val_kb(m, "Buffers:        ", i.bufferram);
>>   	show_val_kb(m, "Cached:         ", cached);
>> -	show_val_kb(m, "SwapCached:     ", total_swapcache_pages());
>> +	show_val_kb(m, "SwapCached:     ", swap_cached);
>>   	show_val_kb(m, "Active:         ", pages[LRU_ACTIVE_ANON] +
>>   					   pages[LRU_ACTIVE_FILE]);
>>   	show_val_kb(m, "Inactive:       ", pages[LRU_INACTIVE_ANON] +
>> @@ -92,8 +103,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
>>   		    global_node_page_state(NR_FILE_DIRTY));
>>   	show_val_kb(m, "Writeback:      ",
>>   		    global_node_page_state(NR_WRITEBACK));
>> -	show_val_kb(m, "AnonPages:      ",
>> -		    global_node_page_state(NR_ANON_MAPPED));
>> +	show_val_kb(m, "AnonPages:      ", anon_pages);
>>   	show_val_kb(m, "Mapped:         ",
>>   		    global_node_page_state(NR_FILE_MAPPED));
>>   	show_val_kb(m, "Shmem:          ", i.sharedram);
>>
>