From: Hyeonggon Yoo <42.hyeyoo@gmail.com>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: Damien Le Moal <damien.lemoal@opensource.wdc.com>,
Conor Dooley <conor@kernel.org>,
Pasha Tatashin <pasha.tatashin@soleen.com>,
Christoph Lameter <cl@linux.com>,
David Rientjes <rientjes@google.com>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Pekka Enberg <penberg@kernel.org>,
Matthew Wilcox <willy@infradead.org>,
Roman Gushchin <roman.gushchin@linux.dev>,
Linus Torvalds <torvalds@linux-foundation.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Rustam Kovhaev <rkovhaev@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Josh Triplett <josh@joshtriplett.org>,
Arnd Bergmann <arnd@arndb.de>,
Russell King <linux@armlinux.org.uk>,
Alexander Shiyan <shc_work@mail.ru>,
Aaro Koskinen <aaro.koskinen@iki.fi>,
Janusz Krzysztofik <jmkrzyszt@gmail.com>,
Tony Lindgren <tony@atomide.com>,
Yoshinori Sato <ysato@users.sourceforge.jp>,
Rich Felker <dalias@libc.org>, Jonas Bonn <jonas@southpole.se>,
Stefan Kristiansson <stefan.kristiansson@saunalahti.fi>,
Stafford Horne <shorne@gmail.com>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
openrisc@lists.librecores.org, linux-riscv@lists.infradead.org,
linux-sh@vger.kernel.org,
Geert Uytterhoeven <geert@linux-m68k.org>,
Conor.Dooley@microchip.com, Paul Cercueil <paul@crapouillou.net>
Subject: Re: Deprecating and removing SLOB
Date: Mon, 14 Nov 2022 20:50:03 +0900 [thread overview]
Message-ID: <Y3Ira4A7jNF+5l2d@hyeyoo> (raw)
In-Reply-To: <44da078c-b630-a249-bf50-67df83cd8347@suse.cz>
On Mon, Nov 14, 2022 at 10:36:31AM +0100, Vlastimil Babka wrote:
> On 11/14/22 06:48, Damien Le Moal wrote:
> > On 11/14/22 10:55, Damien Le Moal wrote:
> >> On 11/12/22 05:46, Conor Dooley wrote:
> >>> On Fri, Nov 11, 2022 at 11:33:30AM +0100, Vlastimil Babka wrote:
> >>>> On 11/8/22 22:44, Pasha Tatashin wrote:
> >>>>> On Tue, Nov 8, 2022 at 10:55 AM Vlastimil Babka <vbabka@suse.cz> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> as we all know, we currently have three slab allocators. As we discussed
> >>>>>> at LPC [1], it is my hope that one of these allocators has a future, and
> >>>>>> two of them do not.
> >>>>>>
> >>>>>> The unsurprising reasons include code maintenance burden, other features
> >>>>>> compatible with only a subset of allocators (or more effort spent on the
> >>>>>> features), blocking API improvements (more on that below), and my
> >>>>>> inability to pronounce SLAB and SLUB in a properly distinguishable way,
> >>>>>> without resorting to spelling out the letters.
> >>>>>>
> >>>>>> I think (but may be proven wrong) that SLOB is the easier target of the
> >>>>>> two to be removed, so I'd like to focus on it first.
> >>>>>>
> >>>>>> I believe SLOB can be removed because:
> >>>>>>
> >>>>>> - AFAIK nobody really uses it? It strives for minimal memory footprint
> >>>>>> by putting all objects together, which has its CPU performance costs
> >>>>>> (locking, lack of percpu caching, searching for free space...). I'm not
> >>>>>> aware of any "tiny linux" deployment that opts for this. For example,
> >>>>>> OpenWRT seems to use SLUB and the devices these days have e.g. 128MB
> >>>>>> RAM, not up to 16 MB anymore. I've heard anecdotes that the performance
> >>>>>> SLOB impact is too much for those who tried. Googling for
> >>>>>> "CONFIG_SLOB=y" yielded nothing useful.
> >>>>>
> >>>>> I am all for removing SLOB.
> >>>>>
> >>>>> There are some devices with configs where SLOB is enabled by default.
> >>>>> Perhaps, the owners/maintainers of those devices/configs should be
> >>>>> included into this thread:
> >>>>>
> >>>>> tatashin@soleen:~/x/linux$ git grep SLOB=y
> >>>
> >>>>> arch/riscv/configs/nommu_k210_defconfig:CONFIG_SLOB=y
> >>>>> arch/riscv/configs/nommu_k210_sdcard_defconfig:CONFIG_SLOB=y
> >>>>> arch/riscv/configs/nommu_virt_defconfig:CONFIG_SLOB=y
> >>>
> >>>>
> >>>> Turns out that since SLOB depends on EXPERT, many of those lack it so
> >>>> running make defconfig ends up with SLUB anyway, unless I miss something.
> >>>> Only a subset has both SLOB and EXPERT:
> >>>>
> >>>>> git grep CONFIG_EXPERT `git grep -l "CONFIG_SLOB=y"`
> >>>
> >>>> arch/riscv/configs/nommu_virt_defconfig:CONFIG_EXPERT=y
> >>>
> >>> I suppose there's not really a concern with the virt defconfig, but I
> >>> did check the output of `make nommu_k210_defconfig" and despite not
> >>> having expert it seems to end up CONFIG_SLOB=y in the generated .config.
> >>>
> >>> I do have a board with a k210 so I checked with s/SLOB/SLUB and it still
> >>> boots etc, but I have no workloads or w/e to run on it.
> >>
> >> I sent a patch to change the k210 defconfig to using SLUB. However...
>
> Thanks!
>
> >> The current default config using SLOB gives about 630 free memory pages
> >> after boot (cat /proc/vmstat). Switching to SLUB, this is down to about
> >> 400 free memory pages (CONFIG_SLUB_CPU_PARTIAL is off).
>
> Thanks for the testing! How much RAM does the system have btw? I found 8MB
> somewhere, is that correct?
> So 230 pages that's a ~920 kB difference. Last time we saw less dramatic
> difference [1]. But that was looking at Slab pages, not free pages.
IIRC overhead of s->min_partial (between 5 and 10) was pretty big because SLUB
caches at most (s->min_partial) * (nr of caches) * (size of slab) bytes of
unused memory.
Passing slub_max_order=0 also may help a little bit.
> The extra overhead could be also in percpu allocations, code etc.
SLUB do not use large amount of percpu allocator I think, less than
30kB on such a small machine.
Maybe also it would help reducing code size to disable CONFIG_MEMCG and CONFIG_TRACING,
CONFIG_SLUB_DEBUG and CONFIG_SYSFS.
I started from tinyconfig and enabled only necessary configs when testing in [1]
(it's a bit laborious cuz pure tinyconfig does not even boot...).
> >> This is with a buildroot kernel 5.19 build including a shell and sd-card
> >> boot. With SLUB, I get clean boots and a shell prompt as expected. But I
> >> definitely see more errors with shell commands failing due to allocation
> >> failures for the shell process fork. So as far as the K210 is concerned,
> >> switching to SLUB is not ideal.
> >>
> >> I would not want to hold on kernel mm improvements because of this toy
> >> k210 though, so I am not going to prevent SLOB deprecation. I just wish
> >> SLUB itself used less memory :)
> >
> > Did further tests with kernel 6.0.1:
> > * SLOB: 630 free pages after boot, shell working (occasional shell fork
> > failure happen though)
> > * SLAB: getting memory allocation for order 7 failures on boot already
> > (init process). Shell barely working (high frequency of shell command fork
> > failures)
> > * SLUB: getting memory allocation for order 7 failures on boot. I do get a
> > shell prompt but cannot run any shell command that involves forking a new
> > process.
> >
> > So if we want to keep the k210 support functional with a shell, we need
> > slob. If we reduce that board support to only one application started as
> > the init process, then I guess anything is OK.
>
> In [1] it was possible to save some more memory with more tuning. Some of
> that required boot parameters and other code changes. In another reply [2] I
> considered adding something like SLUB_TINY to take care of all that, so
> looks like it would make sense to proceed with that.
>
> [1]
> https://lore.kernel.org/all/Yg9xSWEaTZLA+hYt@ip-172-31-19-208.ap-northeast-1.compute.internal/
> [2] https://lore.kernel.org/all/eebc9dc8-6a45-c099-61da-230d06cb3157@suse.cz/
--
Thanks,
Hyeonggon
next prev parent reply other threads:[~2022-11-14 11:54 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-08 15:55 Deprecating and removing SLOB Vlastimil Babka
2022-11-08 18:18 ` Christophe Leroy
2022-11-08 19:17 ` Andrew Morton
2022-11-08 18:46 ` Roman Gushchin
2022-11-08 20:13 ` Yosry Ahmed
2022-11-09 9:09 ` Vlastimil Babka
2022-11-08 21:44 ` Pasha Tatashin
2022-11-09 9:00 ` Vlastimil Babka
2022-11-09 15:50 ` Aaro Koskinen
2022-11-09 16:45 ` Geert Uytterhoeven
2022-11-09 17:45 ` Mike Rapoport
2022-11-09 21:16 ` Janusz Krzysztofik
2022-11-09 17:57 ` Conor.Dooley
2022-11-09 23:00 ` Damien Le Moal
2022-11-11 10:25 ` Vlastimil Babka
2022-11-12 1:40 ` Damien Le Moal
2022-11-11 10:33 ` Vlastimil Babka
2022-11-11 20:46 ` Conor Dooley
2022-11-12 1:40 ` Damien Le Moal
2022-11-14 1:55 ` Damien Le Moal
2022-11-14 5:48 ` Damien Le Moal
2022-11-14 9:36 ` Vlastimil Babka
2022-11-14 11:35 ` Damien Le Moal
2022-11-14 14:47 ` Hyeonggon Yoo
2022-11-15 4:24 ` Damien Le Moal
2022-11-15 4:28 ` Damien Le Moal
2022-11-16 7:57 ` Matthew Wilcox
2022-11-16 8:02 ` Damien Le Moal
2022-11-16 17:51 ` Vlastimil Babka
2022-11-17 0:22 ` Damien Le Moal
2022-11-21 4:30 ` Damien Le Moal
2022-11-21 17:02 ` Vlastimil Babka
2022-11-14 11:50 ` Hyeonggon Yoo [this message]
2022-11-09 20:56 Paul Cercueil
2022-11-09 21:39 ` Linus Torvalds
2022-11-09 23:48 ` Aaro Koskinen
2022-11-09 23:51 ` Aaro Koskinen
2022-11-10 4:40 ` Theodore Ts'o
2022-11-10 7:31 ` Vlastimil Babka
2022-11-10 7:54 ` Feng Tang
2022-11-10 16:20 ` Matthew Wilcox
2022-11-11 9:37 ` David Laight
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y3Ira4A7jNF+5l2d@hyeyoo \
--to=42.hyeyoo@gmail.com \
--cc=Conor.Dooley@microchip.com \
--cc=aaro.koskinen@iki.fi \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=catalin.marinas@arm.com \
--cc=cl@linux.com \
--cc=conor@kernel.org \
--cc=dalias@libc.org \
--cc=damien.lemoal@opensource.wdc.com \
--cc=geert@linux-m68k.org \
--cc=iamjoonsoo.kim@lge.com \
--cc=jmkrzyszt@gmail.com \
--cc=jonas@southpole.se \
--cc=josh@joshtriplett.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux-sh@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=openrisc@lists.librecores.org \
--cc=pasha.tatashin@soleen.com \
--cc=paul@crapouillou.net \
--cc=penberg@kernel.org \
--cc=rientjes@google.com \
--cc=rkovhaev@gmail.com \
--cc=roman.gushchin@linux.dev \
--cc=shc_work@mail.ru \
--cc=shorne@gmail.com \
--cc=stefan.kristiansson@saunalahti.fi \
--cc=tony@atomide.com \
--cc=torvalds@linux-foundation.org \
--cc=vbabka@suse.cz \
--cc=willy@infradead.org \
--cc=ysato@users.sourceforge.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).