* Re: 3.14.65: Memory leak when slub_debug is enabled
[not found] <CAA4-JFLOmeYrWOEO_d2ALPgf0cWhC_fv1Gisz5fyH3uY1ogV1g@mail.gmail.com>
@ 2016-03-29 23:35 ` Christoph Lameter
2016-03-30 8:58 ` Jesper Dangaard Brouer
1 sibling, 0 replies; 4+ messages in thread
From: Christoph Lameter @ 2016-03-29 23:35 UTC (permalink / raw)
To: Ajay Patel; +Cc: linux-kernel, penberg, brouer, rientjes, iamjoonsoo.kim
On Tue, 29 Mar 2016, Ajay Patel wrote:
> We have custom board with Marvell Armada dual core ARMV7.
> The driver uses buffers from kmalloc-8192 slab heavily.
> When slub_debug is enabled, the kmalloc-8192 active slabs are
> increasing. The slub stats shows cmpxchg_double_fail and objects_partial
> are increasing too. Eventually system panics on oom.
Hmmm... I thought we fall back to pass through to the page allocator for
order 1 requests? Why is it going through the regular allocator paths?
> Following patch fixes the issue.
Wonder how that could be? Does the __cmpxchg_double work correctly on ARM?
> Has anybody encountered this issue?
> Is this right fix?
Looks like something is screwing around with the page flags because an
order 1 page is a compound page? Can you ensure that order 1 allocs are
using page allocator fallback. See kmalloc_large().
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: 3.14.65: Memory leak when slub_debug is enabled
[not found] <CAA4-JFLOmeYrWOEO_d2ALPgf0cWhC_fv1Gisz5fyH3uY1ogV1g@mail.gmail.com>
@ 2016-03-30 8:58 ` Jesper Dangaard Brouer
2016-03-30 8:58 ` Jesper Dangaard Brouer
1 sibling, 0 replies; 4+ messages in thread
From: Jesper Dangaard Brouer @ 2016-03-30 8:58 UTC (permalink / raw)
To: Ajay Patel
Cc: linux-kernel, cl, penberg, rientjes, iamjoonsoo.kim, brouer, linux-mm
Hi Ajay,
Could you please provide info on kernel .config settings via commands:
grep HAVE_ALIGNED_STRUCT_PAGE .config
grep CONFIG_HAVE_CMPXCHG_DOUBLE .config
You can try to further debug your problem by defining SLUB_DEBUG_CMPXCHG
manually in mm/slub.c to get some verbose output on the cmpxchg failures.
Is the "Marvell Armada dual core ARMV7" a 32-bit CPU?
--Jesper
On Tue, 29 Mar 2016 15:32:26 -0700 Ajay Patel <patela@gmail.com> wrote:
> We have custom board with Marvell Armada dual core ARMV7.
> The driver uses buffers from kmalloc-8192 slab heavily.
> When slub_debug is enabled, the kmalloc-8192 active slabs are
> increasing. The slub stats shows cmpxchg_double_fail and objects_partial
> are increasing too. Eventually system panics on oom.
>
> Following patch fixes the issue.
> Has anybody encountered this issue?
> Is this right fix?
>
> I am not in mailing list please cc me.
>
> Thanks
> Ajay
>
>
> --- slub.c.orig Tue Mar 29 11:54:42 2016
> +++ slub.c Tue Mar 29 15:08:30 2016
> @@ -1562,9 +1562,12 @@
> void *freelist;
> unsigned long counters;
> struct page new;
> + int retry_count = 0;
> +#define RETRY_COUNT 10
>
> lockdep_assert_held(&n->list_lock);
>
> +again:
> /*
> * Zap the freelist and set the frozen bit.
> * The old freelist is the list of objects for the
> @@ -1587,8 +1590,13 @@
> if (!__cmpxchg_double_slab(s, page,
> freelist, counters,
> new.freelist, new.counters,
> - "acquire_slab"))
> + "acquire_slab")) {
> + if (retry_count++ < RETRY_COUNT) {
> + new.frozen = 0;
> + goto again;
> + }
> return NULL;
> + }
>
> remove_partial(n, page);
> WARN_ON(!freelist);
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: 3.14.65: Memory leak when slub_debug is enabled
@ 2016-03-30 8:58 ` Jesper Dangaard Brouer
0 siblings, 0 replies; 4+ messages in thread
From: Jesper Dangaard Brouer @ 2016-03-30 8:58 UTC (permalink / raw)
To: Ajay Patel
Cc: linux-kernel, cl, penberg, rientjes, iamjoonsoo.kim, brouer, linux-mm
Hi Ajay,
Could you please provide info on kernel .config settings via commands:
grep HAVE_ALIGNED_STRUCT_PAGE .config
grep CONFIG_HAVE_CMPXCHG_DOUBLE .config
You can try to further debug your problem by defining SLUB_DEBUG_CMPXCHG
manually in mm/slub.c to get some verbose output on the cmpxchg failures.
Is the "Marvell Armada dual core ARMV7" a 32-bit CPU?
--Jesper
On Tue, 29 Mar 2016 15:32:26 -0700 Ajay Patel <patela@gmail.com> wrote:
> We have custom board with Marvell Armada dual core ARMV7.
> The driver uses buffers from kmalloc-8192 slab heavily.
> When slub_debug is enabled, the kmalloc-8192 active slabs are
> increasing. The slub stats shows cmpxchg_double_fail and objects_partial
> are increasing too. Eventually system panics on oom.
>
> Following patch fixes the issue.
> Has anybody encountered this issue?
> Is this right fix?
>
> I am not in mailing list please cc me.
>
> Thanks
> Ajay
>
>
> --- slub.c.orig Tue Mar 29 11:54:42 2016
> +++ slub.c Tue Mar 29 15:08:30 2016
> @@ -1562,9 +1562,12 @@
> void *freelist;
> unsigned long counters;
> struct page new;
> + int retry_count = 0;
> +#define RETRY_COUNT 10
>
> lockdep_assert_held(&n->list_lock);
>
> +again:
> /*
> * Zap the freelist and set the frozen bit.
> * The old freelist is the list of objects for the
> @@ -1587,8 +1590,13 @@
> if (!__cmpxchg_double_slab(s, page,
> freelist, counters,
> new.freelist, new.counters,
> - "acquire_slab"))
> + "acquire_slab")) {
> + if (retry_count++ < RETRY_COUNT) {
> + new.frozen = 0;
> + goto again;
> + }
> return NULL;
> + }
>
> remove_partial(n, page);
> WARN_ON(!freelist);
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: 3.14.65: Memory leak when slub_debug is enabled
2016-03-30 8:58 ` Jesper Dangaard Brouer
(?)
@ 2016-03-30 20:50 ` Ajay Patel
-1 siblings, 0 replies; 4+ messages in thread
From: Ajay Patel @ 2016-03-30 20:50 UTC (permalink / raw)
To: Jesper Dangaard Brouer
Cc: linux-kernel, cl, penberg, rientjes, iamjoonsoo.kim, linux-mm
[-- Attachment #1: Type: text/plain, Size: 18753 bytes --]
Jesper/Christop
Thanks for getting back.
This is Marvel Armada dual core ARMV7 and it is 32bit CPU.
The problem is NOT seen if maxcpus=1 used in command line.
The HAVE_ALIGNED_STRUCT_PAGE and CONFIG_HAVE_CMPXCHG_DOUBLE is NOT defined
for the board where problem is seen.
The problem is profoundly seen on kmalloc-8192 slab.
The slab size and object size is displayed below.
Also the object partial is growing and active slabs are growing.
It seems one core is trying to allocate the buffers while other core is
freeing the buffer and causing this.
I have to add some debug to confirm the theory.
I also turned the SLUB_DEBUG_CMPXCHG and it is flooding the console.
Some of those messages pasted below.
Let me know if you need more info.
Thanks
Ajay
================== CPU info ============================
:/# cat /proc/cpuinfo
processor : 0
model name : ARMv7 Processor rev 1 (v7l)
Features : swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls
vfpd32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x4
CPU part : 0xc09
CPU revision : 1
processor : 1
model name : ARMv7 Processor rev 1 (v7l)
Features : swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls
vfpd32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x4
CPU part : 0xc09
CPU revision : 1
Hardware : Marvell Armada 380/385/390/398
:/#
#:/sys/kernel/slab/kmalloc-8192# cat slab_size
8384
#:/sys/kernel/slab/kmalloc-8192# cat object_size
8192
=====================Initial state of counters =====================
:/sys/kernel/slab/kmalloc-8192# cat objects_partial
2
:/sys/kernel/slab/kmalloc-8192# cat /proc/slabinfo | grep 8192
kmalloc-8192 32786 32790 8384 3 8 : tunables 0 0 0 :
slabdata 10930 10930 0
===================== counters after some time ============
#:/sys/kernel/slab/kmalloc-8192# cat /proc/slabinfo | grep 8192
kmalloc-8192 32789 44712 8384 3 8 : tunables 0 0 0 :
slabdata 14904 14904 0
#:/sys/kernel/slab/kmalloc-8192# cat objects_partial
15006
======================= Debug messages ======================
[03/30/2016 13:19:21.9400] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:22.0700] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab names_cache: cmpxchg double redo
[03/30/2016 13:19:22.2100] __slab_free names_cache: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:22.3400] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:22.5000] __slab_free filp: cmpxchg double redo unfreezing
slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:22.6300] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:22.7900] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free kmalloc-2048: cmpxchg double redo
[03/30/2016 13:19:22.9500] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.0900] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.2300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:23.3600] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.4900] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:23.6500] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.7900] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.9200] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:24.0500] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.2300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.3600] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.6300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:24.7600] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.9000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.0300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:25.1700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.3000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.4300] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.5600] unfreezing slab kmalloc-8192: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.6900] acquire_slab skbuff_head_cache: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.8300] unfreezing slab kmalloc-8192: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.9600] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.0900] unfreezing slab kmalloc-8192: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.2300] unfreezing slab kmalloc-8192: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.3600] __slab_free kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-256: cmpxchg double redo
[03/30/2016 13:19:26.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.6900] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.8200] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.9400] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.0700] unfreezing slab skbuff_head_cache: cmpxchg
double redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:27.2100] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.3400] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:27.4800] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.6100] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.7400] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.8700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:28.0000] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:28.1300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:28.2600] acquire_slab kmalloc-4096: cmpxchg double redo
__slab_free kmalloc-4096: cmpxchg double redo
[03/30/2016 13:19:28.4300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:28.5700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:28.7000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:28.8400] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:28.9700] __slab_free kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:29.0900] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:29.2300] acquire_slab skbuff_head_cache: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:29.3600] unfreezing slab kmalloc-8192: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:29.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:29.6300] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:29.7700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:29.9000] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.0300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.1700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.3000] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.4400] __slab_free kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.5700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:30.7000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:30.8300] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.9800] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.1100] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:31.2400] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.3700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.6300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:31.7700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.9000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.0300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:32.1600] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.2900] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:32.4300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.5600] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:32.7000] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.8300] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.9700] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:33.1200] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:33.2600] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free kmalloc-64: cmpxchg double redo
[03/30/2016 13:19:33.5900] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:33.7300] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:33.8700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.0000] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:34.2300] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.3700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.6300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:34.7600] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.9000] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.0300] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:35.1700] acquire_slab skbuff_head_cache: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.3000] __slab_free kmalloc-8192: cmpxchg double redo
unfreezing slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:35.4400] unfreezing slab kmalloc-8192: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.5700] __slab_free kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.7000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:35.8400] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.9600] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.0900] unfreezing slab kmalloc-8192: cmpxchg double
redo acquire_slab kmalloc-4096: cmpxchg double redo
[03/30/2016 13:19:36.2300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.3500] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:36.4900] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.6200] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.7500] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.8800] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.0200] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.1500] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.2900] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:37.4200] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.5500] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:37.7500] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.8900] acquire_slab kmalloc-8192: cmpxchg double redo ac
On Wed, Mar 30, 2016 at 1:58 AM, Jesper Dangaard Brouer <brouer@redhat.com>
wrote:
>
> Hi Ajay,
>
> Could you please provide info on kernel .config settings via commands:
>
> grep HAVE_ALIGNED_STRUCT_PAGE .config
> grep CONFIG_HAVE_CMPXCHG_DOUBLE .config
>
> You can try to further debug your problem by defining SLUB_DEBUG_CMPXCHG
> manually in mm/slub.c to get some verbose output on the cmpxchg failures.
>
> Is the "Marvell Armada dual core ARMV7" a 32-bit CPU?
>
> --Jesper
>
> On Tue, 29 Mar 2016 15:32:26 -0700 Ajay Patel <patela@gmail.com> wrote:
>
> > We have custom board with Marvell Armada dual core ARMV7.
> > The driver uses buffers from kmalloc-8192 slab heavily.
> > When slub_debug is enabled, the kmalloc-8192 active slabs are
> > increasing. The slub stats shows cmpxchg_double_fail and objects_partial
> > are increasing too. Eventually system panics on oom.
> >
> > Following patch fixes the issue.
> > Has anybody encountered this issue?
> > Is this right fix?
> >
> > I am not in mailing list please cc me.
> >
> > Thanks
> > Ajay
> >
> >
> > --- slub.c.orig Tue Mar 29 11:54:42 2016
> > +++ slub.c Tue Mar 29 15:08:30 2016
> > @@ -1562,9 +1562,12 @@
> > void *freelist;
> > unsigned long counters;
> > struct page new;
> > + int retry_count = 0;
> > +#define RETRY_COUNT 10
> >
> > lockdep_assert_held(&n->list_lock);
> >
> > +again:
> > /*
> > * Zap the freelist and set the frozen bit.
> > * The old freelist is the list of objects for the
> > @@ -1587,8 +1590,13 @@
> > if (!__cmpxchg_double_slab(s, page,
> > freelist, counters,
> > new.freelist, new.counters,
> > - "acquire_slab"))
> > + "acquire_slab")) {
> > + if (retry_count++ < RETRY_COUNT) {
> > + new.frozen = 0;
> > + goto again;
> > + }
> > return NULL;
> > + }
> >
> > remove_partial(n, page);
> > WARN_ON(!freelist);
>
>
>
> --
> Best regards,
> Jesper Dangaard Brouer
> MSc.CS, Principal Kernel Engineer at Red Hat
> Author of http://www.iptv-analyzer.org
> LinkedIn: http://www.linkedin.com/in/brouer
>
[-- Attachment #2: Type: text/html, Size: 21198 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2016-03-30 20:50 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CAA4-JFLOmeYrWOEO_d2ALPgf0cWhC_fv1Gisz5fyH3uY1ogV1g@mail.gmail.com>
2016-03-29 23:35 ` 3.14.65: Memory leak when slub_debug is enabled Christoph Lameter
2016-03-30 8:58 ` Jesper Dangaard Brouer
2016-03-30 8:58 ` Jesper Dangaard Brouer
2016-03-30 20:50 ` Ajay Patel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.