* [QUESTION] is SLAB considered legacy and deprecated? @ 2021-09-27 9:03 Hyeonggon Yoo 2021-09-27 17:03 ` Vlastimil Babka 0 siblings, 1 reply; 18+ messages in thread From: Hyeonggon Yoo @ 2021-09-27 9:03 UTC (permalink / raw) To: linux-mm Cc: 42.hyeyoo, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, Vlastimil Babka, linux-kernel Hello there, I've been working on adding 'lockless cache' on sl[au]b for a while. But what it actually does is actually adding 'queuing' on slub. So there is a fundamental question coming into my mind: 'is SLAB considered legacy and deprecated?' It seems there are little development on SLAB and people think that SLAB is legacy and deprecated, so CONFIG_SLUB is used by default. But I think both has pros and cons for their own: SLAB: more temporal locality (cache friendly) but high usage of memory, and less spatial locality (TLB misses) than SLUB. SLUB: less temporal locality (less cache friendly) than SLAB but more spatial locality (TLB hit), and low usage of memory and good debugging feature. Why do people say SLAB is deprecated/legacy? Thanks, Hyeonggon ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-09-27 9:03 [QUESTION] is SLAB considered legacy and deprecated? Hyeonggon Yoo @ 2021-09-27 17:03 ` Vlastimil Babka 2021-09-28 11:12 ` Hyeonggon Yoo 2021-10-01 0:39 ` Queueing is outside of SLUB nowdays Hyeonggon Yoo 0 siblings, 2 replies; 18+ messages in thread From: Vlastimil Babka @ 2021-09-27 17:03 UTC (permalink / raw) To: Hyeonggon Yoo, linux-mm Cc: Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On 9/27/21 11:03, Hyeonggon Yoo wrote: > Hello there, > > I've been working on adding 'lockless cache' on sl[au]b for a while. > But what it actually does is actually adding 'queuing' on slub. Yeah, I pointed out those threads from 2011 that called it exactly that... was there any conclusion why that was not ultimately merged? > So there is a fundamental question coming into my mind: > 'is SLAB considered legacy and deprecated?' To some extend, but not yet in a sense where we would have a deadline to get rid of it. In some contexts it's still being preferred, AFAIK. But it's okay if new optional features require just SLUB - for example PREEMPT_RT. > It seems there are little development on SLAB and people think that > SLAB is legacy and deprecated, so CONFIG_SLUB is used by default. > > But I think both has pros and cons for their own: > SLAB: more temporal locality (cache friendly) > but high usage of memory, and less spatial locality (TLB misses) than SLUB. > > SLUB: less temporal locality (less cache friendly) than SLAB > but more spatial locality (TLB hit), and low usage of memory > and good debugging feature. I'm not so sure about the usage of memory, SLUB can easily use more I believe. Instead of caching some arrays of objects it will have one or more private slabs per cpu, and the slabs are larger-order pages. > Why do people say SLAB is deprecated/legacy? Do they? > Thanks, > Hyeonggon > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-09-27 17:03 ` Vlastimil Babka @ 2021-09-28 11:12 ` Hyeonggon Yoo 2021-09-29 10:50 ` Hyeonggon Yoo 2021-10-01 14:07 ` Vlastimil Babka 2021-10-01 0:39 ` Queueing is outside of SLUB nowdays Hyeonggon Yoo 1 sibling, 2 replies; 18+ messages in thread From: Hyeonggon Yoo @ 2021-09-28 11:12 UTC (permalink / raw) To: Vlastimil Babka Cc: linux-mm, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On Mon, Sep 27, 2021 at 07:03:19PM +0200, Vlastimil Babka wrote: > On 9/27/21 11:03, Hyeonggon Yoo wrote: > > Hello there, > > > > I've been working on adding 'lockless cache' on sl[au]b for a while. > > But what it actually does is actually adding 'queuing' on slub. > > Yeah, I pointed out those threads from 2011 that called it exactly that... > was there any conclusion why that was not ultimately merged? > It's best to ask Christoph, as he suggested to add queuing on slub. Well, after reading old threads, I concluded that it was not merged because Christoph stopped developing this. To make sure that he stopped developing this, I searched his name and sorted by date. [1] Then I see Christoph's last response on SLUB + Queueing was: Christopher Lameter Wrote: David Rientjes Wrote: > Overall, the results are _much_ better than the vanilla slub allocator > that I frequently saw ~20% regressions with the TCP_RR netperf benchmark > on a couple of my machines with larger cpu counts. However, there still > is a significant performance degradation compared to slab. It seems that the memory leak is still present. This likely skews the results. Thought I had it fixed. Thanks. It was not merged on 2.6.37-rc1 at that time [2] And after some time he worked on improving SLUB itself [3], [4], [5], not SLUB with queuing. (anyway his work resulted in lots of improvement on slub without queuing) And in [4], Christoph Lameter wrote: Well here is another result of my obsession with SLAB allocators. There must be some way to get an allocator done that is faster without queueing and I hope that we are now there (maybe only almost...). Any help with cleaning up the rough edges would be appreciated. So I guess he concluded that adding queuing on SLUB was not good idea (again, it's best to ask him) > > So there is a fundamental question coming into my mind: > > 'is SLAB considered legacy and deprecated?' > > To some extend, but not yet in a sense where we would have a deadline to get > rid of it. What makes you to say 'To some extent'? That's what I'm curious about :) > In some contexts it's still being preferred, AFAIK. In what context is SLAB or SLUB is preferred? And what is the core reason that SLUB is used by default? > > It seems there are little development on SLAB and people think that > > SLAB is legacy and deprecated, so CONFIG_SLUB is used by default. > > > > But I think both has pros and cons for their own: > > SLAB: more temporal locality (cache friendly) > > but high usage of memory, and less spatial locality (TLB misses) than SLUB. > > > > SLUB: less temporal locality (less cache friendly) than SLAB > > but more spatial locality (TLB hit), and low usage of memory > > and good debugging feature. > > I'm not so sure about the usage of memory, SLUB can easily use more I > believe. Instead of caching some arrays of objects it will have one or more > private slabs per cpu, and the slabs are larger-order pages. > SLAB uses much memory than slab, when there are *lots* of NUMA nodes. because kmem_cache_node exist per node and it keeps alien cache (used for caching freeing object from remote node), so it keeps MAX_NUMANODES ^ 2 of array_cache per slab. that is why I said SLAB uses much memory than SLUB. to quote Christoph Lameter's (who made SLUB and current maintainer) presentation in 2014: See link [6] for this p21: "SLAB queuing can get intensive memory usage going grows exponentially by NUMA node" But I'm not sure if SLAB or SLUB is good on UMA architecture. > > Why do people say SLAB is deprecated/legacy? > > Do they? > Yes I think they do. One example is presentation of David Rientjes (one of maintainers) in 2011: See link [7] p5: "SLAB is deprecated, very little development" p5: "Many distributions still ship with CONFIG_SLAB even though it is not the kernel default" Anyway, I wanted to discuss the current status of SLAB. To me, it seems that people do not work on SLAB lately and it seems some maintainers (David, Christoph) think it's better to move from SLAB to SLUB. one of them is Christoph himself. As he made SLUB because he don't agree on design of SLAB. [8] Thanks, Hyeonggon. Searching his name and sort by date [1] https://lore.kernel.org/lkml/?q=Christoph+Lameter&o=10800 [GIT PULL] SLAB updates for 2.6.37-rc1 by Pekka Enberg [2] https://lore.kernel.org/lkml/alpine.DEB.2.00.1010242005280.4447@tiger/#r His work on lockless fastpath of slub (2011-02), [3] https://lore.kernel.org/lkml/20110225173850.486326452@linux.com/ His work on lockless fastpath of slub (2011-06) [4] https://lore.kernel.org/lkml/20110601172543.437240675@linux.com/ His work on per cpu partial lists [5] https://lore.kernel.org/lkml/20110809211221.831975979@linux.com/ Christoph's presentation on slab [6] - https://events.static.linuxfound.org/sites/events/files/slides/slaballocators.pdf David's presentation on slab [7] - https://www.socallinuxexpo.org/scale9x-media/scalemedia/scale/scale9x-media/simple_cfp/presentations/16_30-DavidRientjes-Status_of_the_Linux_Slab_Allocators.pdf [8] SLUB: The unqueued slab allocator V6 https://lwn.net/Articles/229096/ ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-09-28 11:12 ` Hyeonggon Yoo @ 2021-09-29 10:50 ` Hyeonggon Yoo 2021-10-01 14:07 ` Vlastimil Babka 1 sibling, 0 replies; 18+ messages in thread From: Hyeonggon Yoo @ 2021-09-29 10:50 UTC (permalink / raw) To: Vlastimil Babka Cc: linux-mm, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On Tue, Sep 28, 2021 at 11:12:36AM +0000, Hyeonggon Yoo wrote: > SLAB uses much memory than slab, not slab, slub. > when there are *lots* of NUMA nodes. > because kmem_cache_node exist per node and it keeps alien cache > (used for caching freeing object from remote node), > so it keeps MAX_NUMANODES ^ 2 of array_cache per slab. not "per slab". it is per cache (kmem_cache). it was my mistake. but that is still too much for systems with high number of numa nodes. batching freeing object from remote node is needed, but slab uses too much memory for that. for each node, it allocates alien cache for every other nodes. so it's using MAX_NUMNODES ^ 2 alien_cache. > that is why I said SLAB uses much memory than SLUB. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-09-28 11:12 ` Hyeonggon Yoo 2021-09-29 10:50 ` Hyeonggon Yoo @ 2021-10-01 14:07 ` Vlastimil Babka 2021-10-03 5:59 ` Hyeonggon Yoo 1 sibling, 1 reply; 18+ messages in thread From: Vlastimil Babka @ 2021-10-01 14:07 UTC (permalink / raw) To: Hyeonggon Yoo Cc: linux-mm, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On 9/28/21 13:12, Hyeonggon Yoo wrote: >>> So there is a fundamental question coming into my mind: >>> 'is SLAB considered legacy and deprecated?' >> >> To some extend, but not yet in a sense where we would have a deadline to get >> rid of it. > > What makes you to say 'To some extent'? > That's what I'm curious about :) "To some extent" because SLUB is default and if some new stuff was added that only worked with SLUB and not SLAB, there wouldn't be major objections expected. >> In some contexts it's still being preferred, AFAIK. > > In what context is SLAB or SLUB is preferred? I don't know the full answer. With our distro we have used SLAB, and switched to SLUB after verifying that there are no major regressions. Better debugging features were perhaps the major reason. > And what is the core reason that SLUB is used by default? The usual reason in MM, nobody objected :) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a0acd82080768 >>> It seems there are little development on SLAB and people think that >>> SLAB is legacy and deprecated, so CONFIG_SLUB is used by default. >>> >>> But I think both has pros and cons for their own: >>> SLAB: more temporal locality (cache friendly) >>> but high usage of memory, and less spatial locality (TLB misses) than SLUB. >>> >>> SLUB: less temporal locality (less cache friendly) than SLAB >>> but more spatial locality (TLB hit), and low usage of memory >>> and good debugging feature. >> >> I'm not so sure about the usage of memory, SLUB can easily use more I >> believe. Instead of caching some arrays of objects it will have one or more >> private slabs per cpu, and the slabs are larger-order pages. >> > > SLAB uses much memory than slab, when there are *lots* of NUMA nodes. > because kmem_cache_node exist per node and it keeps alien cache > (used for caching freeing object from remote node), > so it keeps MAX_NUMANODES ^ 2 of array_cache per slab. > that is why I said SLAB uses much memory than SLUB. Maybe, theoretically. But SLUB has per-cpu partial lists caching whole slab pages, not just arrays of objects. Number of cpus will generally grow with number of NUMA nodes. > to quote Christoph Lameter's (who made SLUB and current maintainer) presentation in 2014: > See link [6] for this > p21: "SLAB queuing can get intensive memory usage going grows > exponentially by NUMA node" > > But I'm not sure if SLAB or SLUB is good on UMA architecture. > >>> Why do people say SLAB is deprecated/legacy? >> >> Do they? >> > > Yes I think they do. > > One example is presentation of David Rientjes (one of maintainers) in 2011: > See link [7] > p5: "SLAB is deprecated, very little development" > p5: "Many distributions still ship with CONFIG_SLAB even > though it is not the kernel default" > > Anyway, I wanted to discuss the current status of SLAB. > > To me, it seems that people do not work on SLAB lately > and it seems some maintainers (David, Christoph) think > it's better to move from SLAB to SLUB. one of them is Christoph himself. > As he made SLUB because he don't agree on design of SLAB. [8] Right. Here's the discussion from the last time SLAB removal was proposed and rejected, feel free to evaluate if the points there are still valid today: https://lore.kernel.org/all/20190410024714.26607-1-tobin@kernel.org/ > Thanks, > Hyeonggon. > > Searching his name and sort by date > [1] https://lore.kernel.org/lkml/?q=Christoph+Lameter&o=10800 > > [GIT PULL] SLAB updates for 2.6.37-rc1 by Pekka Enberg > [2] https://lore.kernel.org/lkml/alpine.DEB.2.00.1010242005280.4447@tiger/#r > > His work on lockless fastpath of slub (2011-02), > [3] https://lore.kernel.org/lkml/20110225173850.486326452@linux.com/ > > His work on lockless fastpath of slub (2011-06) > [4] https://lore.kernel.org/lkml/20110601172543.437240675@linux.com/ > > His work on per cpu partial lists > [5] https://lore.kernel.org/lkml/20110809211221.831975979@linux.com/ > > Christoph's presentation on slab > [6] - https://events.static.linuxfound.org/sites/events/files/slides/slaballocators.pdf > > David's presentation on slab > [7] - https://www.socallinuxexpo.org/scale9x-media/scalemedia/scale/scale9x-media/simple_cfp/presentations/16_30-DavidRientjes-Status_of_the_Linux_Slab_Allocators.pdf > > [8] SLUB: The unqueued slab allocator V6 > https://lwn.net/Articles/229096/ > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-10-01 14:07 ` Vlastimil Babka @ 2021-10-03 5:59 ` Hyeonggon Yoo 2021-10-04 1:25 ` David Rientjes 2021-10-04 11:34 ` Vlastimil Babka 0 siblings, 2 replies; 18+ messages in thread From: Hyeonggon Yoo @ 2021-10-03 5:59 UTC (permalink / raw) To: Vlastimil Babka Cc: linux-mm, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel Hello Vlastimil! I'm happy to discuss with you. I hope this discussion to make us think about slab allocator. On Fri, Oct 01, 2021 at 04:07:56PM +0200, Vlastimil Babka wrote: > >> In some contexts it's still being preferred, AFAIK. > > > > In what context is SLAB or SLUB is preferred? > > I don't know the full answer. With our distro we have used SLAB, and > switched to SLUB after verifying that there are no major regressions. > Better debugging features were perhaps the major reason. > > > And what is the core reason that SLUB is used by default? > > The usual reason in MM, nobody objected :) > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a0acd82080768 > What was the decision based on ? performance measurements or anything else? 'Because nobody objected' is not a good reason to make a decision. >>>> So there is a fundamental question coming into my mind: >>>> 'is SLAB considered legacy and deprecated?' >>> >>> To some extend, but not yet in a sense where we would have a deadline to get >>> rid of it. >>> >> What makes you to say 'To some extent'? >> That's what I'm curious about :) > "To some extent" because SLUB is default and if some new stuff was added > that only worked with SLUB and not SLAB, there wouldn't be major > objections expected. You said new features are added to only SLUB and there are no objections expected, But what makes you to do so? You are not saying why. what you are saying is just only result. What is the mind behind maintaining SLUB and neglecting SLAB? is SLUB really next-generation slab allocator that can ultimately replace SLAB? if so, why not just drop SLAB? if not, why not maintain SLAB? > >>> It seems there are little development on SLAB and people think that > >>> SLAB is legacy and deprecated, so CONFIG_SLUB is used by default. > >>> > >>> But I think both has pros and cons for their own: > >>> SLAB: more temporal locality (cache friendly) > >>> but high usage of memory, and less spatial locality (TLB misses) than SLUB. > >>> > >>> SLUB: less temporal locality (less cache friendly) than SLAB > >>> but more spatial locality (TLB hit), and low usage of memory > >>> and good debugging feature. > >> > >> I'm not so sure about the usage of memory, SLUB can easily use more I > >> believe. Instead of caching some arrays of objects it will have one or more > >> private slabs per cpu, and the slabs are larger-order pages. > >> > > > > SLAB uses much memory than slab, when there are *lots* of NUMA nodes. > > because kmem_cache_node exist per node and it keeps alien cache > > (used for caching freeing object from remote node), > > so it keeps MAX_NUMANODES ^ 2 of array_cache per slab. > > that is why I said SLAB uses much memory than SLUB. > > Maybe, theoretically. But SLUB has per-cpu partial lists caching whole > slab pages, not just arrays of objects. Number of cpus will generally > grow with number of NUMA nodes. > Sorry, it was my mistake. I mean SLAB uses much memory than SLUB when there are lots of NUMA nodes. in SLAB, there is alien_cache to batch freeing objects from remote node. and alien cache has space complexity of MAX_NUMNODES ^ 2. That's why I said SLAB uses much memory. > > to quote Christoph Lameter's (who made SLUB and current maintainer) presentation in 2014: > > See link [6] for this > > p21: "SLAB queuing can get intensive memory usage going grows > > exponentially by NUMA node" > > > > But I'm not sure if SLAB or SLUB is good on UMA architecture. > > And that is one of reasons why Christoph made SLUB [8] > >>> Why do people say SLAB is deprecated/legacy? > >> > >> Do they? > >> > > > > Yes I think they do. > > > > One example is presentation of David Rientjes (one of maintainers) in 2011: > > See link [7] > > p5: "SLAB is deprecated, very little development" > > p5: "Many distributions still ship with CONFIG_SLAB even > > though it is not the kernel default" > > > > Anyway, I wanted to discuss the current status of SLAB. > > > > To me, it seems that people do not work on SLAB lately > > and it seems some maintainers (David, Christoph) think > > it's better to move from SLAB to SLUB. one of them is Christoph himself. > > As he made SLUB because he don't agree on design of SLAB. [8] > > Right. > Here's the discussion from the last time SLAB removal was proposed and > rejected, feel free to evaluate if the points there are still valid today: > > https://lore.kernel.org/all/20190410024714.26607-1-tobin@kernel.org/ > I think the points are still valid because on some workloads SLAB works better. especially when alloc/frees are intensive, SLUB tends to become bottleneck. If we can't drop SLAB, it should be at least maintained :( But it has been neglected for a long time, which makes people not to use SLAB. Nobody likes to use a subsystem that isn't maintained. Anyway, I'm curious about share of SLAB and SLUB and on what situations SLAB or SLUB is preferred. that information would help maintain both. Thanks, Hyeonggon > > Thanks, > > Hyeonggon. > > > > Searching his name and sort by date > > [1] https://lore.kernel.org/lkml/?q=Christoph+Lameter&o=10800 > > > > [GIT PULL] SLAB updates for 2.6.37-rc1 by Pekka Enberg > > [2] https://lore.kernel.org/lkml/alpine.DEB.2.00.1010242005280.4447@tiger/#r > > > > His work on lockless fastpath of slub (2011-02), > > [3] https://lore.kernel.org/lkml/20110225173850.486326452@linux.com/ > > > > His work on lockless fastpath of slub (2011-06) > > [4] https://lore.kernel.org/lkml/20110601172543.437240675@linux.com/ > > > > His work on per cpu partial lists > > [5] https://lore.kernel.org/lkml/20110809211221.831975979@linux.com/ > > > > Christoph's presentation on slab > > [6] - https://events.static.linuxfound.org/sites/events/files/slides/slaballocators.pdf > > > > David's presentation on slab > > [7] - https://www.socallinuxexpo.org/scale9x-media/scalemedia/scale/scale9x-media/simple_cfp/presentations/16_30-DavidRientjes-Status_of_the_Linux_Slab_Allocators.pdf > > > > [8] SLUB: The unqueued slab allocator V6 > > https://lwn.net/Articles/229096/ > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-10-03 5:59 ` Hyeonggon Yoo @ 2021-10-04 1:25 ` David Rientjes 2021-10-04 11:34 ` Vlastimil Babka 1 sibling, 0 replies; 18+ messages in thread From: David Rientjes @ 2021-10-04 1:25 UTC (permalink / raw) To: Hyeonggon Yoo Cc: Vlastimil Babka, linux-mm, Christoph Lameter, Pekka Enberg, Joonsoo Kim, Andrew Morton, linux-kernel On Sun, 3 Oct 2021, Hyeonggon Yoo wrote: > I think the points are still valid because on some workloads SLAB works > better. especially when alloc/frees are intensive, SLUB tends to become > bottleneck. > > If we can't drop SLAB, it should be at least maintained :( > But it has been neglected for a long time, which makes people not to > use SLAB. Nobody likes to use a subsystem that isn't maintained. > > Anyway, I'm curious about share of SLAB and SLUB and on what situations > SLAB or SLUB is preferred. that information would help maintain both. > Thanks for raising this, the discussion is always useful. Both allocators have their pros and cons. I would disagree that SLAB isn't currently maintained, I think it's actively maintained. Google actually uses it for its production kernel although we're investigating the performance results that we can obtain from SLUB not that we have per-object memcg accounting. There have been workloads, as you mention, that perform better with SLAB even though SLUB can make up for some of its degradation by throwing more memory at the problem (like per-cpu partial slabs). I think the general guidance is that changes for both allocators can still be merged upstream if they show a significant win (improved performnace, maintaining performance while reducing memory footprint, code hygiene, etc) and there's no specific policy that we cannot make changes to mm/slab.c. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? @ 2021-10-04 1:25 ` David Rientjes 0 siblings, 0 replies; 18+ messages in thread From: David Rientjes @ 2021-10-04 1:25 UTC (permalink / raw) To: Hyeonggon Yoo Cc: Vlastimil Babka, linux-mm, Christoph Lameter, Pekka Enberg, Joonsoo Kim, Andrew Morton, linux-kernel On Sun, 3 Oct 2021, Hyeonggon Yoo wrote: > I think the points are still valid because on some workloads SLAB works > better. especially when alloc/frees are intensive, SLUB tends to become > bottleneck. > > If we can't drop SLAB, it should be at least maintained :( > But it has been neglected for a long time, which makes people not to > use SLAB. Nobody likes to use a subsystem that isn't maintained. > > Anyway, I'm curious about share of SLAB and SLUB and on what situations > SLAB or SLUB is preferred. that information would help maintain both. > Thanks for raising this, the discussion is always useful. Both allocators have their pros and cons. I would disagree that SLAB isn't currently maintained, I think it's actively maintained. Google actually uses it for its production kernel although we're investigating the performance results that we can obtain from SLUB not that we have per-object memcg accounting. There have been workloads, as you mention, that perform better with SLAB even though SLUB can make up for some of its degradation by throwing more memory at the problem (like per-cpu partial slabs). I think the general guidance is that changes for both allocators can still be merged upstream if they show a significant win (improved performnace, maintaining performance while reducing memory footprint, code hygiene, etc) and there's no specific policy that we cannot make changes to mm/slab.c. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-10-04 1:25 ` David Rientjes (?) @ 2021-10-04 6:01 ` Hyeonggon Yoo 2021-10-04 11:39 ` Vlastimil Babka -1 siblings, 1 reply; 18+ messages in thread From: Hyeonggon Yoo @ 2021-10-04 6:01 UTC (permalink / raw) To: David Rientjes Cc: Vlastimil Babka, linux-mm, Christoph Lameter, Pekka Enberg, Joonsoo Kim, Andrew Morton, linux-kernel On Sun, Oct 03, 2021 at 06:25:29PM -0700, David Rientjes wrote: > On Sun, 3 Oct 2021, Hyeonggon Yoo wrote: > > > I think the points are still valid because on some workloads SLAB works > > better. especially when alloc/frees are intensive, SLUB tends to become > > bottleneck. > > > > If we can't drop SLAB, it should be at least maintained :( > > But it has been neglected for a long time, which makes people not to > > use SLAB. Nobody likes to use a subsystem that isn't maintained. > > > > Anyway, I'm curious about share of SLAB and SLUB and on what situations > > SLAB or SLUB is preferred. that information would help maintain both. > > > > Thanks for raising this, the discussion is always useful. Both allocators > have their pros and cons. > Thanks for kind words, I was curious and wanted to help improve SLAB > I would disagree that SLAB isn't currently maintained, I think it's > actively maintained. I thought it was not actively maintained because most of patches were fixups and cleanups for years and as Vlastimil said, new features are only added to SLUB. development was focused on SLUB. > I think the general guidance is that changes for both allocators can still > be merged upstream if they show a significant win (improved performnace, > maintaining performance while reducing memory footprint, code hygiene, > etc) and there's no specific policy that we cannot make changes to > mm/slab.c. Good. I see things to improve in SLAB and want to improve it. I will appreciate if you review them. Thanks, Hyeonggon ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-10-04 6:01 ` Hyeonggon Yoo @ 2021-10-04 11:39 ` Vlastimil Babka 2021-10-05 13:31 ` Hyeonggon Yoo 0 siblings, 1 reply; 18+ messages in thread From: Vlastimil Babka @ 2021-10-04 11:39 UTC (permalink / raw) To: Hyeonggon Yoo, David Rientjes Cc: linux-mm, Christoph Lameter, Pekka Enberg, Joonsoo Kim, Andrew Morton, linux-kernel On 10/4/21 08:01, Hyeonggon Yoo wrote: > On Sun, Oct 03, 2021 at 06:25:29PM -0700, David Rientjes wrote: >> I would disagree that SLAB isn't currently maintained, I think it's >> actively maintained. > > I thought it was not actively maintained because most of patches were > fixups and cleanups for years and as Vlastimil said, new features are Fixups and cleanups still count as "actively maintained". The opposite case would be "nobody uses it because it was broken for years since commit X and we only noticed now". > only added to SLUB. development was focused on SLUB. > >> I think the general guidance is that changes for both allocators can still >> be merged upstream if they show a significant win (improved performnace, >> maintaining performance while reducing memory footprint, code hygiene, >> etc) and there's no specific policy that we cannot make changes to >> mm/slab.c. > > Good. > > I see things to improve in SLAB and want to improve it. > I will appreciate if you review them. It would be great if your motivation started with "I prefer SLAB over SLUB because X and Y but I need to improve Z", not just a theoretical concern. > Thanks, > Hyeonggon > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-10-04 11:39 ` Vlastimil Babka @ 2021-10-05 13:31 ` Hyeonggon Yoo 0 siblings, 0 replies; 18+ messages in thread From: Hyeonggon Yoo @ 2021-10-05 13:31 UTC (permalink / raw) To: Vlastimil Babka Cc: David Rientjes, linux-mm, Christoph Lameter, Pekka Enberg, Joonsoo Kim, Andrew Morton, linux-kernel On Mon, Oct 04, 2021 at 01:39:46PM +0200, Vlastimil Babka wrote: > On 10/4/21 08:01, Hyeonggon Yoo wrote: > > On Sun, Oct 03, 2021 at 06:25:29PM -0700, David Rientjes wrote: > >> I would disagree that SLAB isn't currently maintained, I think it's > >> actively maintained. > > > > I thought it was not actively maintained because most of patches were > > fixups and cleanups for years and as Vlastimil said, new features are > > Fixups and cleanups still count as "actively maintained". The opposite > case would be "nobody uses it because it was broken for years since > commit X and we only noticed now". > Yup, there seems I was differently using meaning of "actively maintained". > > only added to SLUB. development was focused on SLUB. > > > >> I think the general guidance is that changes for both allocators can still > >> be merged upstream if they show a significant win (improved performnace, > >> maintaining performance while reducing memory footprint, code hygiene, > >> etc) and there's no specific policy that we cannot make changes to > >> mm/slab.c. > > > > Good. > > > > I see things to improve in SLAB and want to improve it. > > I will appreciate if you review them. > > It would be great if your motivation started with "I prefer SLAB over > SLUB because X and Y but I need to improve Z", not just a theoretical > concern. > Thank you for advice. by making dumb patches I realized that , yeah, just a theoretical concern does not help. I should have more understanding on internals of slab allocators and on how their characteristics affect performance depending on situation. and most importantly I should have actual evidence of performance measurement. I'm sorry and Thank you for thinking about and answering my (somewhat dumb) questions. But I'm happy that I'm learning a lot from your feedback. > > Thanks, > > Hyeonggon > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-10-03 5:59 ` Hyeonggon Yoo 2021-10-04 1:25 ` David Rientjes @ 2021-10-04 11:34 ` Vlastimil Babka 2021-10-04 14:22 ` Hyeonggon Yoo 1 sibling, 1 reply; 18+ messages in thread From: Vlastimil Babka @ 2021-10-04 11:34 UTC (permalink / raw) To: Hyeonggon Yoo Cc: linux-mm, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On 10/3/21 07:59, Hyeonggon Yoo wrote: > Hello Vlastimil! I'm happy to discuss with you. > I hope this discussion to make us think about slab allocator. Yeah, it's useful, thanks for asking! > On Fri, Oct 01, 2021 at 04:07:56PM +0200, Vlastimil Babka wrote: >>>> In some contexts it's still being preferred, AFAIK. >>> >>> In what context is SLAB or SLUB is preferred? >> >> I don't know the full answer. With our distro we have used SLAB, and >> switched to SLUB after verifying that there are no major regressions. >> Better debugging features were perhaps the major reason. >> > >>> And what is the core reason that SLUB is used by default? >> >> The usual reason in MM, nobody objected :) >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a0acd82080768 >> > > What was the decision based on ? performance measurements or anything I haven't been around back then, so don't know. Also tried to find this particular patch (and possible replies) in the linux-mm lore archive, and didn't succeed. Might have been event sent off-list by mistake. > else? 'Because nobody objected' is not a good reason to make a decision. :) >>>>> So there is a fundamental question coming into my mind: >>>>> 'is SLAB considered legacy and deprecated?' >>>> >>>> To some extend, but not yet in a sense where we would have a deadline to get >>>> rid of it. >>>> >>> What makes you to say 'To some extent'? >>> That's what I'm curious about :) > >> "To some extent" because SLUB is default and if some new stuff was added >> that only worked with SLUB and not SLAB, there wouldn't be major >> objections expected. > > You said new features are added to only SLUB and there are no > objections expected, But what makes you to do so? > > You are not saying why. what you are saying is just only result. > What is the mind behind maintaining SLUB and neglecting SLAB? David explained it well. I'll add it's a question of motivation, people generally add features to the subsystem they prefer, and most prefer SLUB these days, and in that case we don't require the same feature to be added to SLAB (or even SLOB) as well. But if someone wants to, we also don't stop them - but to some extent. If someone suddenly came up with "SLUB has this nice debugging and SLAB not, so I will reimplement it there" we would be questioning hard if the code churn is really needed. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [QUESTION] is SLAB considered legacy and deprecated? 2021-10-04 11:34 ` Vlastimil Babka @ 2021-10-04 14:22 ` Hyeonggon Yoo 0 siblings, 0 replies; 18+ messages in thread From: Hyeonggon Yoo @ 2021-10-04 14:22 UTC (permalink / raw) To: Vlastimil Babka Cc: linux-mm, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On Mon, Oct 04, 2021 at 01:34:17PM +0200, Vlastimil Babka wrote: > On 10/3/21 07:59, Hyeonggon Yoo wrote: > > Hello Vlastimil! I'm happy to discuss with you. > > I hope this discussion to make us think about slab allocator. > > Yeah, it's useful, thanks for asking! > Me too, Thanks for answering. > > On Fri, Oct 01, 2021 at 04:07:56PM +0200, Vlastimil Babka wrote: > >>>> In some contexts it's still being preferred, AFAIK. > >>> > >>> In what context is SLAB or SLUB is preferred? > >> > >> I don't know the full answer. With our distro we have used SLAB, and > >> switched to SLUB after verifying that there are no major regressions. > >> Better debugging features were perhaps the major reason. > >> > > > >>> And what is the core reason that SLUB is used by default? > >> > >> The usual reason in MM, nobody objected :) > >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a0acd82080768 > >> > > > > What was the decision based on ? performance measurements or anything > > I haven't been around back then, so don't know. Also tried to find this > particular patch (and possible replies) in the linux-mm lore archive, > and didn't succeed. Might have been event sent off-list by mistake. > I can't find it in the list too. Maybe it was not sent to list. > >>>>> So there is a fundamental question coming into my mind: > >>>>> 'is SLAB considered legacy and deprecated?' > >>>> > >>>> To some extend, but not yet in a sense where we would have a deadline to get > >>>> rid of it. > >>>> > >>> What makes you to say 'To some extent'? > >>> That's what I'm curious about :) > > > >> "To some extent" because SLUB is default and if some new stuff was added > >> that only worked with SLUB and not SLAB, there wouldn't be major > >> objections expected. > > > > You said new features are added to only SLUB and there are no > > objections expected, But what makes you to do so? > > > > You are not saying why. what you are saying is just only result. > > What is the mind behind maintaining SLUB and neglecting SLAB? > > David explained it well. I'll add it's a question of motivation, people > generally add features to the subsystem they prefer, and most prefer > SLUB these days, and in that case we don't require the same feature to > be added to SLAB (or even SLOB) as well. But if someone wants to, we > also don't stop them - but to some extent. If someone suddenly came up > with "SLUB has this nice debugging and SLAB not, so I will reimplement > it there" we would be questioning hard if the code churn is really needed. > I got your point. no need to take unnecessary maintenance cost. Thanks, Hyeonggon ^ permalink raw reply [flat|nested] 18+ messages in thread
* Queueing is outside of SLUB nowdays 2021-09-27 17:03 ` Vlastimil Babka 2021-09-28 11:12 ` Hyeonggon Yoo @ 2021-10-01 0:39 ` Hyeonggon Yoo 2021-10-04 14:56 ` Christoph Lameter 1 sibling, 1 reply; 18+ messages in thread From: Hyeonggon Yoo @ 2021-10-01 0:39 UTC (permalink / raw) To: Vlastimil Babka Cc: linux-mm, Christoph Lameter, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On Mon, Sep 27, 2021 at 07:03:19PM +0200, Vlastimil Babka wrote: > On 9/27/21 11:03, Hyeonggon Yoo wrote: > > Hello there, > > > > I've been working on adding 'lockless cache' on sl[au]b for a while. > > But what it actually does is actually adding 'queuing' on slub. > > Yeah, I pointed out those threads from 2011 that called it exactly that... > was there any conclusion why that was not ultimately merged? Looking at other layers, they implemented queuing layer outside of SLUB. See commit 795bb1c00dd ("net: bulk free infrastructure for NAPI context, use napi_consume_skb") for example. They made skb cache because SLUB is not suitable for intensive alloc/free. And because the queue is outside of slab, it can go lockless depending on it's context. (But it's not easy to do so in slab because slab is general purpose allocator.) So current approach on place where slab's performance is critical is implementing queuing layer on top of slab. Then new question arising: - Is that proper way to solve fundamental problem? - why not use SLAB if they need queuing? - how does this approach work on SLAB? - If there are no problems with queuing outside of slab, why does SLAB exist? can we just drop SLAB? I think we need to think about questions above. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Queueing is outside of SLUB nowdays 2021-10-01 0:39 ` Queueing is outside of SLUB nowdays Hyeonggon Yoo @ 2021-10-04 14:56 ` Christoph Lameter 0 siblings, 0 replies; 18+ messages in thread From: Christoph Lameter @ 2021-10-04 14:56 UTC (permalink / raw) To: Hyeonggon Yoo Cc: Vlastimil Babka, linux-mm, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On Fri, 1 Oct 2021, Hyeonggon Yoo wrote: > Looking at other layers, they implemented queuing layer outside of SLUB. > See commit 795bb1c00dd ("net: bulk free infrastructure for NAPI context, > use napi_consume_skb") for example. They made skb cache because SLUB is > not suitable for intensive alloc/free. > > And because the queue is outside of slab, it can go lockless > depending on it's context. (But it's not easy to do so in slab because > slab is general purpose allocator.) The queuing within in SLUB/SLAB is lockless. > So current approach on place where slab's performance is critical > is implementing queuing layer on top of slab. If you have to use object specific characteristics to optimize then yes you can optimize further. However, the slab allocators implement each their own form of queuing that is generic. > Then new question arising: > - Is that proper way to solve fundamental problem? There is a problem? > - why not use SLAB if they need queuing? SLAB is LIFO queuing whereas SLUB uses spatial considerations and queues within a page before going outside. Slab requires disabling interrupts, SLUB is optimized to rely on per cpu atomics and there are numerous other differences. > - how does this approach work on SLAB? SLAB has a lockless layer that is only requiring disabling interrupts. It provides a generic queuing layer as well. See my talk on Slab allocators awhile back. https://www.youtube.com/watch?v=h0VMLXavx30 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Queueing is outside of SLUB nowdays @ 2021-10-04 14:56 ` Christoph Lameter 0 siblings, 0 replies; 18+ messages in thread From: Christoph Lameter @ 2021-10-04 14:56 UTC (permalink / raw) To: Hyeonggon Yoo Cc: Vlastimil Babka, linux-mm, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On Fri, 1 Oct 2021, Hyeonggon Yoo wrote: > Looking at other layers, they implemented queuing layer outside of SLUB. > See commit 795bb1c00dd ("net: bulk free infrastructure for NAPI context, > use napi_consume_skb") for example. They made skb cache because SLUB is > not suitable for intensive alloc/free. > > And because the queue is outside of slab, it can go lockless > depending on it's context. (But it's not easy to do so in slab because > slab is general purpose allocator.) The queuing within in SLUB/SLAB is lockless. > So current approach on place where slab's performance is critical > is implementing queuing layer on top of slab. If you have to use object specific characteristics to optimize then yes you can optimize further. However, the slab allocators implement each their own form of queuing that is generic. > Then new question arising: > - Is that proper way to solve fundamental problem? There is a problem? > - why not use SLAB if they need queuing? SLAB is LIFO queuing whereas SLUB uses spatial considerations and queues within a page before going outside. Slab requires disabling interrupts, SLUB is optimized to rely on per cpu atomics and there are numerous other differences. > - how does this approach work on SLAB? SLAB has a lockless layer that is only requiring disabling interrupts. It provides a generic queuing layer as well. See my talk on Slab allocators awhile back. https://www.youtube.com/watch?v=h0VMLXavx30 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Queueing is outside of SLUB nowdays 2021-10-04 14:56 ` Christoph Lameter (?) @ 2021-10-05 8:19 ` Vlastimil Babka 2021-10-05 14:16 ` Hyeonggon Yoo -1 siblings, 1 reply; 18+ messages in thread From: Vlastimil Babka @ 2021-10-05 8:19 UTC (permalink / raw) To: Christoph Lameter, Hyeonggon Yoo Cc: linux-mm, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On 10/4/21 16:56, Christoph Lameter wrote: > On Fri, 1 Oct 2021, Hyeonggon Yoo wrote: > >> Looking at other layers, they implemented queuing layer outside of SLUB. >> See commit 795bb1c00dd ("net: bulk free infrastructure for NAPI context, >> use napi_consume_skb") for example. They made skb cache because SLUB is >> not suitable for intensive alloc/free. >> >> And because the queue is outside of slab, it can go lockless >> depending on it's context. (But it's not easy to do so in slab because >> slab is general purpose allocator.) > > The queuing within in SLUB/SLAB is lockless. > >> So current approach on place where slab's performance is critical >> is implementing queuing layer on top of slab. > > If you have to use object specific characteristics to optimize then yes > you can optimize further. However, the slab allocators implement each > their own form of queuing that is generic. > >> Then new question arising: >> - Is that proper way to solve fundamental problem? > > There is a problem? If someone benefits from implementing a caching layer on top of SL*B, it probably indicates a problem. >> - why not use SLAB if they need queuing? > > SLAB is LIFO queuing whereas SLUB uses spatial considerations and queues > within a page before going outside. IIUC SLUB queueing works well for allocation (we just consume a per-cpu freelist that nobody else can touch) but freeing uses the corresponding page's freelist so the atomics are more expensive. In both cases the linked freelists might be also worse for cache locality than an array of pointers. So perhaps some workload still benefit from a array-based cache on top of SLUB and it would be great if they didn't have to implement own solutions? > Slab requires disabling interrupts, > SLUB is optimized to rely on per cpu atomics and there are numerous other > differences. > >> - how does this approach work on SLAB? > > SLAB has a lockless layer that is only requiring disabling interrupts. It > provides a generic queuing layer as well. > > See my talk on Slab allocators awhile back. > > https://www.youtube.com/watch?v=h0VMLXavx30 > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Queueing is outside of SLUB nowdays 2021-10-05 8:19 ` Vlastimil Babka @ 2021-10-05 14:16 ` Hyeonggon Yoo 0 siblings, 0 replies; 18+ messages in thread From: Hyeonggon Yoo @ 2021-10-05 14:16 UTC (permalink / raw) To: Vlastimil Babka Cc: Christoph Lameter, linux-mm, Pekka Enberg, David Rientjes, Joonsoo Kim, Andrew Morton, linux-kernel On Tue, Oct 05, 2021 at 10:19:32AM +0200, Vlastimil Babka wrote: > On 10/4/21 16:56, Christoph Lameter wrote: > > On Fri, 1 Oct 2021, Hyeonggon Yoo wrote: > > > >> Looking at other layers, they implemented queuing layer outside of SLUB. > >> See commit 795bb1c00dd ("net: bulk free infrastructure for NAPI context, > >> use napi_consume_skb") for example. They made skb cache because SLUB is > >> not suitable for intensive alloc/free. > >> > >> And because the queue is outside of slab, it can go lockless > >> depending on it's context. (But it's not easy to do so in slab because > >> slab is general purpose allocator.) > > > > The queuing within in SLUB/SLAB is lockless. > > Oh, yes. both SLAB/SLUB has lockless queueing. I misused word 'lockless'. it's lockless and also without disabling interrupt. > >> So current approach on place where slab's performance is critical > >> is implementing queuing layer on top of slab. > > > > If you have to use object specific characteristics to optimize then yes > > you can optimize further. However, the slab allocators implement each > > their own form of queuing that is generic. > > > >> Then new question arising: > >> - Is that proper way to solve fundamental problem? > > > > There is a problem? > > If someone benefits from implementing a caching layer on top of SL*B, it > probably indicates a problem. > Before I say something, I want to ask why Christoph stopped implementing SLUB+Q at that time. And Yeah, I think there are some problems. If objects are manged outside of slab allocator and most of alloc/frees are done outside of slab, it's waste of memory. To say the extreme case (even if it's not common situation), how does implementing queueing layer on SLAB, on system with high NUMA nodes makes sense? it's wasting lots of memory. and objects are treated as 'allocated' even if it's actually being not used in queue outside slab. so the memory is unreclaimable. I think that, If objects are mostly allocated / freed outside of slab allocator, it does not need to be on top of slab allocator. and implementing same queueing layers on similar situation is duplication of code and increased maintenance cost. So what I tried was generalizing optimizations that are done in some layers (block and networks). But that was not an easy task, though. ============================================================================= And what recently I was surprised was: I asked Jens (who recently made bio caching layer on slab) "I think it would have better performance if you run benchmarks with SLAB?". because there is lots of allocations (Millions of allocations per second), it's likely to cache-friendly characteristics of SLAB would have result in performance. but the response was "I would be surprised if SLAB was better, SLAB is considered legacy and everybody uses SLUB." and the solution was too SLUB-specific way. (implementing queuing layer) I'll say it's too SLUB-specific because SLAB's cache utilization functionalities weren't even considered. That's why I started this thread at first. > >> - why not use SLAB if they need queuing? > > > > SLAB is LIFO queuing whereas SLUB uses spatial considerations and queues > > within a page before going outside. > > IIUC SLUB queueing works well for allocation (we just consume a per-cpu > freelist that nobody else can touch) but freeing uses the corresponding > page's freelist so the atomics are more expensive. In both cases the linked > freelists might be also worse for cache locality than an array of pointers. > So perhaps some workload still benefit from a array-based cache on top of > SLUB and it would be great if they didn't have to implement own solutions? > I wonder if page-based policy will work well with queueing. What to do if the page is full and we must take new page to satisfy a request? and what If the queue is mixed with objects of different pages? That might be somewhat losing spatial locality of SLUB. So if what you need is queueing, I think SLAB might be better than adding queueing on SLUB because adding queueing on SLUB might result in losing its own characteristics. It seems it's really difficult to consider all situations in a single memory allocator... T.T. > > Slab requires disabling interrupts, > > SLUB is optimized to rely on per cpu atomics and there are numerous other > > differences. > > > >> - how does this approach work on SLAB? > > > > SLAB has a lockless layer that is only requiring disabling interrupts. It > > provides a generic queuing layer as well. > > > > See my talk on Slab allocators awhile back. > > > > https://www.youtube.com/watch?v=h0VMLXavx30 Thank you for sharing that! I have read the presentation before but didn't know that there was a video too! It's very useful and I became more familiar with them. > > > If I wrongly understand something, please tell me. I'm so excited to talk about this topic. Thanks, Hyeonggon. ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2021-10-05 14:16 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-09-27 9:03 [QUESTION] is SLAB considered legacy and deprecated? Hyeonggon Yoo 2021-09-27 17:03 ` Vlastimil Babka 2021-09-28 11:12 ` Hyeonggon Yoo 2021-09-29 10:50 ` Hyeonggon Yoo 2021-10-01 14:07 ` Vlastimil Babka 2021-10-03 5:59 ` Hyeonggon Yoo 2021-10-04 1:25 ` David Rientjes 2021-10-04 1:25 ` David Rientjes 2021-10-04 6:01 ` Hyeonggon Yoo 2021-10-04 11:39 ` Vlastimil Babka 2021-10-05 13:31 ` Hyeonggon Yoo 2021-10-04 11:34 ` Vlastimil Babka 2021-10-04 14:22 ` Hyeonggon Yoo 2021-10-01 0:39 ` Queueing is outside of SLUB nowdays Hyeonggon Yoo 2021-10-04 14:56 ` Christoph Lameter 2021-10-04 14:56 ` Christoph Lameter 2021-10-05 8:19 ` Vlastimil Babka 2021-10-05 14:16 ` Hyeonggon Yoo
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.