Prevent inode/dentry trashing?

Prevent inode/dentry trashing?

[Philipp Hahn]

Hello,

Similar to 
<https://unix.stackexchange.com/questions/202586/limit-the-inode-cache-used-by-a-command> 
I would like to prevent certain programs from trashing the inode/dentry 
cache, which is a shared resource for all processes:

- For example the nightly <man:updatedb(8)> used <man:find(1) > to 
recursively walk the complete file system. As long as `d_name` and the 
`d_type` information from <man:readdir(3)> is enough this only pollutes 
the dentry cache.

- Similar our backup software, but this also needs to <man:stat(2)> each 
path to get the `mtime`, which additionally pollutes the inode cache.

Both examples only walk the tree once (per day). In my case the caches 
do not fit into memory completely, so the second process does not even 
benefit from the first process filling the cache as that data is already 
replaced again.

The trashed caches affect all other processes running in parallel or the 
first processes started each morning.

Is it possible to prevent inode/dentry trashing for example by limiting 
the cache per process(-group)?
Something like MADV_DONTNEED from <man:madvise(2)> for IO would be nice.

An external knob to limit the cache usage per process(-group) would be 
nice, but even a hint for an API for such kind of programs to prevent 
trashing would help me.

Thank you in advance.
Philipp

Re: Prevent inode/dentry trashing?

[Michal Koutný]

[-- Attachment #1: Type: text/plain, Size: 714 bytes --]

Hello Phillip.

On Mon, Jun 07, 2021 at 02:39:35PM +0200, Philipp Hahn <pmhahn+lkml@pmhahn.de> wrote:
> The trashed caches affect all other processes running in parallel or the
> first processes started each morning.
> 
> Is it possible to prevent inode/dentry trashing for example by limiting the
> cache per process(-group)?

Yes. Unless you have disabled it with CONFIG_MEMCG_KMEM or
cgroup.memory=nokmem, dentries and inodes are charged to respective
cgroups. And you can limit overall memory of a cgroup, see
memory.{max,high} attributes. (You suggest this inode/dentry consumption
is dominant enough to affect other jobs, so the limit would keep it
constrained as you intend).

HTH,
Michal

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Prevent inode/dentry trashing?

[Enrico Weigelt, metux IT consult]

On 25.06.21 17:55, Michal Koutný wrote:
>> Is it possible to prevent inode/dentry trashing for example by limiting the
>> cache per process(-group)?
> 
> Yes. Unless you have disabled it with CONFIG_MEMCG_KMEM or
> cgroup.memory=nokmem, dentries and inodes are charged to respective
> cgroups. And you can limit overall memory of a cgroup, see
> memory.{max,high} attributes. (You suggest this inode/dentry consumption
> is dominant enough to affect other jobs, so the limit would keep it
> constrained as you intend).

Could you please tell a bit more how this really works ?
(maybe some pointers to the code)

I'm curios what happens if those cache objects are used by different
cgroups - are they accounted to multiple times (once per cgroup) ?
What happens when one cgroup using some cache object reaching its limit,
wile another one does not ?


--mtx

-- 
---
Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
GPG/PGP-Schlüssel zu.
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287

Re: Prevent inode/dentry trashing?

[Michal Koutný]

[-- Attachment #1: Type: text/plain, Size: 1228 bytes --]

On Mon, Jun 28, 2021 at 11:40:39AM +0200, "Enrico Weigelt, metux IT consult" <lkml@metux.net> wrote:
> Could you please tell a bit more how this really works ?
> (maybe some pointers to the code)

When cgroup's consumption is about to cross the configured limit,
reclaim is started
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/memcontrol.c?id=62fb9874f5da54fdb243003b386128037319b219#n2579

that may evict old entries
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/vmscan.c?id=62fb9874f5da54fdb243003b386128037319b219#n2852

and if there's still no success freeing some space the dentry allocation
can fail
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/slab.h?id=62fb9874f5da54fdb243003b386128037319b219#n277

(This describes just one code path, the subject isn't always a dentry.)

> I'm curios what happens if those cache objects are used by different
> cgroups - are they accounted to multiple times (once per cgroup) ?
> What happens when one cgroup using some cache object reaching its limit,
> wile another one does not ?

That's explained here
https://www.kernel.org/doc/html/v5.13/admin-guide/cgroup-v2.html#memory-ownership

Michal

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]