Estimating total memory usage in any Linux system

From: Pintu Agarwal <pintu.ping@gmail.com>
To: Kernelnewbies <kernelnewbies@kernelnewbies.org>,
	linux-mm <linux-mm@kvack.org>
Subject: Estimating total memory usage in any Linux system
Date: Mon, 18 May 2020 23:27:34 +0530	[thread overview]
Message-ID: <CAOuPNLio8oY4=yVeuW_E1Cvu_waaKMU=1OyZXvv6CKFKwc01QQ@mail.gmail.com> (raw)

Hi,

I was just trying to manually calculate the total memory available in
a system to match up the MemTotal shown in /proc/meminfo
I know "free -m" command will give the vary high level usage, but I
wanted to breakdown the MemTotal numbers to see usage in each
category.

I am looking for a common formula that should match on almost all systems.
So, I came up with below fields from meminfo to closely match the total.
~Total = {MemFree + Active + Inactive + Shmem + Mapped + Slab +
KernelStack + PageTables + (init freed) + (Boot Diff) + (low
watermarks)}

But I am not sure if these fields are correct at least for arm32 system.
In some boards I could able to match up these figures, but some not.

So I have some doubts:
* Is "Mapped" (NR_MAPPED_PAGES) already part of Active pages ?
* Is Kernel allocation (alloc_pages, etc.) accounted somewhere ? Is it
part of Anon pages ?
* What about vmalloc / ioremap pages ? Do we also need to consider VmallocUsed ?
==> In my opinion vmalloc internally used kmalloc and page tables, so
I guess it should be already part of Slab or PageTables ??

Below are some data taken from a small embedded arm32 device with 512MB RAM:

Kernel Boot Logs:
Memory: 99244K/254976K available (10720K kernel code, 1336K rwdata,
4792K rodata, 400K init, 1602K bss, 135252K reserved, 20480K
cma-reserved)
Total = 99244 + 400 (init) + 20480 (cma) =  120124
Boot Diff = 121204 - 120124 => 1080 kB

MemTotal: 121204 kB
MemFree: 2540 kB
MemAvailable: 51924 kB
Buffers: 0 kB
Cached: 47492 kB
SwapCached: 0 kB
Active: 42604 kB
Inactive: 24308 kB
Active(anon): 19476 kB
Inactive(anon): 92 kB
Active(file): 23128 kB
Inactive(file): 24216 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 8 kB
Writeback: 0 kB
AnonPages: 19452 kB
Mapped: 12644 kB
Shmem: 148 kB
Slab: 27560 kB
SReclaimable: 7224 kB
SUnreclaim: 20336 kB
KernelStack: 3104 kB
PageTables: 1896 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 60600 kB
Committed_AS: 1576740 kB
VmallocTotal: 777216 kB
VmallocUsed: 53696 kB
VmallocChunk: 689148 kB
--------------------

These are my calculations from the above fields.
---------------------
MemFree 2540
Active 42604
Inactive 24308
Mapped 12644
Slab 27560
KernelStack 3104
PageTables 1896
Shmem 148
init freed 400
Diff from boot total 1080
min_free_kbytes 1384
user_reserve_kbytes 3537
My Total ==> 121205  (add all the above)

As you can see "My Total" closely matches with the MemTotal from /proc/meminfo

But on some systems it does not match.
So, I wanted to understand if there is something wrong in my
calculation, or there is something wrong in the system itself (like
leaks etc,)

Note: I don't want to use any tools or utilize for these, but I wanted
to manually understand it.

I am also trying to explore every vm counter that populates these fields.

But if you have any other opinion about it please let me know.

Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies