* DMA: atmel_serial: Opening and closing the serial device repeatedly causes kmalloc-32 slab leak
[not found] <22061488.b0eNpyQjWt@linux-7rm0>
@ 2018-11-27 9:51 ` richard.genoud
2018-11-27 9:58 ` Alexandre Belloni
2018-11-27 9:55 ` richard.genoud
1 sibling, 1 reply; 4+ messages in thread
From: richard.genoud @ 2018-11-27 9:51 UTC (permalink / raw)
To: Ludovic Desroches, Nicolas Ferre, Maxime Ripard, Vinod Koul,
Alexandre Belloni
Cc: linux-arm-kernel, dmaengine, Linux Kernel, linux-serial, Mario Forner
[re-sending the bug report to the lists]
Le 16/11/2018 à 17:04, Mario Forner a écrit :
> Problem:
> When I open and close the serial device /dev/ttyS4 in a loop
> the amount of kmalloc-32 slabs increases slowly but steadily without limit.
>
> The serial device is configured in acme-aria.dts to use DMA.
>
> If DMA is disabled, the amount of kmalloc32-slabs remains constant
> over several hours, just fluctuating slightly.
>
> The serial device is accessed by atmel_serial.c which is evident from
> the drivers kernel log output.
>
> The bug was noticed on a device which had been running over several weeks and
> has accumulated ~86MB of unrelaimable kmalloc-32 slabs by now. Example:
>
> root@master1083:~# slabtop -o | head -10
> Active / Total Objects (% used) : 2704880 / 2716124 (99.6%)
> Active / Total Slabs (% used) : 23150 / 23150 (100.0%)
> Active / Total Caches (% used) : 57 / 76 (75.0%)
> Active / Total Size (% used) : 88893.62K / 89964.32K (98.8%)
> Minimum / Average / Maximum Object : 0.02K / 0.03K / 4096.00K
>
> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
> 2674804 2673840 99% 0.03K 21571 124 86284K kmalloc-32
> 7200 7104 98% 0.08K 144 50 576K kernfs_node_cache
> 6930 5370 77% 0.13K 231 30 924K dentry
>
> root@master1083:~# uptime
> 15:12:55 up 93 days, 16:54, 1 user, load average: 1.59, 2.31, 2.42
>
> Keywords: atmel, serial, kernel, leak, memory, dma, slab, kmalloc
>
> Kernel information:
>
> Kernel version:
> Linux version 4.2.6 (mario@linux-7rm0) (gcc version 4.9.3 (crosstool-NG ) ) #115 Fri Nov 16 11:05:22 CET 2018
> Linux version 4.9.124 (mario@linux-7rm0) (gcc version 4.9.3 (crosstool-NG ) ) #16 Fri Nov 16 13:54:25 CET 2018
>
> Most recent Kernel version which did not have the bug: unknown
>
> How to reproduce the bug:
>
> The kmalloc-32 slab count should be monitored every few minutes with "slabtop -o".
> The leak can be triggered by running the following script, given DMA has
> been enabled for ttyS4:
>
> #!/usr/bin/env python
> import serial
> import time
>
> while True:
> try:
> with serial.Serial(port = '/dev/ttyS4'):
> pass
> except Exception as e:
> print e
> finally:
> time.sleep(0.5)
> # end script
>
> Environment:
> Software:
> Linux master 4.2.6 #114 Fri Nov 16 10:14:30 CET 2018 armv5tejl GNU/Linux
>
> Binutils 2.25
> Util-linux 2.25.2
> Mount 2.25.2
> Module-init-tools 18
> E2fsprogs 1.42.12
> Linux C Library 2.19
> Dynamic linker (ldd) 2.19
> Linux C++ Library 6.0.20
> Procps 3.3.9
> Net-tools 1.60
> Sh-utils 8.23
> Udev 215
> Modules Loaded iptable_nat option usb_wwan usbserial
>
> Processor information:
> # cat /proc/cpuinfo
> processor : 0
> model name : ARM926EJ-S rev 5 (v5l)
> BogoMIPS : 198.76
> Features : swp half thumb fastmult edsp java
> CPU implementer : 0x41
> CPU architecture: 5TEJ
> CPU variant : 0x0
> CPU part : 0x926
> CPU revision : 5
>
> Hardware : Atmel AT91SAM9
> Revision : 0000
> Serial : 0000000000000000
>
> Module information:
> # cat /proc/modules
> iptable_nat 1720 0 - Live 0xbf0a5000
> option 28780 0 - Live 0xbf03c000
> usb_wwan 6876 1 option, Live 0xbf024000
> usbserial 23392 2 option,usb_wwan, Live 0xbf000000
>
> Loaded drivers:
> # cat /proc/ioports
> # cat /proc/iomem
> 00300000-00307fff : 300000.sram
> 00600000-006fffff : /ahb/ohci@00600000
> 00700000-007fffff : /ahb/ehci@00700000
> 20000000-2fffffff : System RAM
> 20008000-20577287 : Kernel code
> 205a8000-205f02b7 : Kernel data
> f0000000-f00000ff : /ahb/apb/spi@f0000000
> f8008000-f80080ff : /ahb/apb/timer@f8008000
> f800c000-f800c0ff : /ahb/apb/timer@f800c000
> f8010000-f80100ff : /ahb/apb/i2c@f8010000
> f8014000-f80140ff : /ahb/apb/i2c@f8014000
> f801c000-f801c1ff : atmel_serial
> f8020000-f80201ff : atmel_serial
> f8024000-f80241ff : atmel_serial
> f8028000-f80281ff : atmel_serial
> f802c000-f802c0ff : /ahb/apb/ethernet@f802c000
> f8034000-f80342ff : /ahb/apb/pwm@f8034000
> f804c000-f804c0ff : /ahb/apb/adc@f804c000
> ffffec00-ffffedff : at_hdmac
> ffffee00-ffffefff : at_hdmac
> fffff200-fffff3ff : atmel_serial
> fffff400-fffff5ff : /ahb/apb/pinctrl@fffff400/gpio@fffff400
> fffff600-fffff7ff : /ahb/apb/pinctrl@fffff400/gpio@fffff600
> fffff800-fffff9ff : /ahb/apb/pinctrl@fffff400/gpio@fffff800
> fffffa00-fffffbff : /ahb/apb/pinctrl@fffff400/gpio@fffffa00
> fffffe10-fffffe1f : /ahb/apb/shdwc@fffffe10
>
> Other Information:
> The serial device ttyS4 is configuered inside .dts file by
>
> usart3: serial@f8028000 {
> status = "okay";
> compatible = "atmel,at91sam9260-usart";
> reg = <0xf8028000 0x200>;
> interrupts = <8 IRQ_TYPE_LEVEL_HIGH 5>;
> pinctrl-names = "default";
> atmel,use-dma-tx;
> atmel,use-dma-rx;
> pinctrl-0 = <&pinctrl_usart3
> &pinctrl_usart3_rts~
> &pinctrl_usart3_cts>;
> linux,rs485-enabled-at-boot-time;
> rs485-rts-delay = <0 0>;
> };
>
> Other notes:
> I tried the SLUB allocator with slub_debug=U but opening serial device then
> resulted in a Kernel-Oops:
> at BUGON(ents < 0);
> in function: dma_map_sg_attrs
> in file: /include/asm-generic/dma-mapping-common.h
> So I could not gather information that way. I don't know if this is related.
> SLUB works if User Tracking is disabled.
>
> Enabling kmemleak did not detect any leaks.
>
> I also tried kernel version 4.19.1 and it exhibits the same behaviour, but I
> could not configure it to run reliably on the target system yet.
>
Hi all,
I reproduced the memory leak on my board (at91sam9g35-cm) with a 4.20-rc3.
It triggered an OOM after a couple of hours running a code like this:
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
int main(int argc, char **argv)
{
int fd;
do {
fd = open("/dev/ttyS1", O_RDONLY);
close(fd);
} while (true);
return 0;
}
As Mario pointed out, this only happens when atmel,use-dma-{r,t}x are
used in the device-tree.
Adding:
CONFIG_DEBUG_SLAB=y
CONFIG_DEBUG_SLAB_LEAK=y
Doesn't show anything suspect in /proc/slab_allocators
From what I found until now, it's something done in :
dma_request_slave_channel();
that leaks kmalloc-32
Mabe I missed something, but it seems that everything DMA related is
deallocated in atmel_release_{tx,rx}_dma().
Is this ringing a bell ?
The oom-killer, slabinfo, meminfo and slab_allocator traces follow.
NB: the -dirty flag is there beacause I removed some log from atmel-serial:
--- a/drivers/tty/serial/atmel_serial.c
+++ b/drivers/tty/serial/atmel_serial.c
@@ -1056,8 +1056,6 @@ static int atmel_prepare_tx_dma(struct uart_port *port)
atmel_port->chan_tx = dma_request_slave_channel(mfd_dev, "tx");
if (atmel_port->chan_tx == NULL)
goto chan_err;
- dev_info(port->dev, "using %s for tx DMA transfers\n",
- dma_chan_name(atmel_port->chan_tx));
spin_lock_init(&atmel_port->lock_tx);
sg_init_table(&atmel_port->sg_tx, 1);
@@ -1239,8 +1237,6 @@ static int atmel_prepare_rx_dma(struct uart_port *port)
atmel_port->chan_rx = dma_request_slave_channel(mfd_dev, "rx");
if (atmel_port->chan_rx == NULL)
goto chan_err;
- dev_info(port->dev, "using %s for rx DMA transfers\n",
- dma_chan_name(atmel_port->chan_rx));
spin_lock_init(&atmel_port->lock_rx);
sg_init_table(&atmel_port->sg_rx, 1);
OOM-killer trace :
[ 5220.560000] test_open invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0
[ 5220.580000] CPU: 0 PID: 1062 Comm: test_open Not tainted 4.20.0-rc3-dirty #31
[ 5220.580000] Hardware name: Atmel AT91SAM9
[ 5220.590000] Backtrace:
[ 5220.590000] [<c000e020>] (dump_backtrace) from [<c000e2b8>] (show_stack+0x20/0x24)
[ 5220.600000] r6:00000009 r5:c0649bb5 r4:c72a1e18 r3:02b88502
[ 5220.600000] [<c000e298>] (show_stack) from [<c0521114>] (dump_stack+0x20/0x28)
[ 5220.610000] [<c05210f4>] (dump_stack) from [<c0097280>] (dump_header+0x6c/0x1c4)
[ 5220.620000] [<c0097214>] (dump_header) from [<c0096454>] (oom_kill_process+0x6c/0x3cc)
[ 5220.620000] r10:00000001 r8:c070e788 r7:c72a1e18 r6:00000009 r5:c0649bb5 r4:c7b64ea0
[ 5220.630000] [<c00963e8>] (oom_kill_process) from [<c009712c>] (out_of_memory+0x388/0x41c)
[ 5220.640000] r10:00000001 r9:00000000 r8:c070e788 r7:0000000b r6:c070e8e4 r5:c070e788
[ 5220.650000] r4:c72a1e18
[ 5220.650000] [<c0096da4>] (out_of_memory) from [<c009b1d8>] (__alloc_pages_nodemask+0x8b4/0xc48)
[ 5220.660000] r8:00000011 r7:00000000 r6:00000000 r5:00000000 r4:006200ca
[ 5220.670000] [<c009a924>] (__alloc_pages_nodemask) from [<c0092bd4>] (filemap_fault+0x384/0x54c)
[ 5220.680000] r10:c609b480 r9:00000000 r8:00000004 r7:00000000 r6:c72a1ed0 r5:c77dbb5c
[ 5220.680000] r4:006200ca
[ 5220.690000] [<c0092850>] (filemap_fault) from [<c00babe8>] (__do_fault+0x28/0x9c)
[ 5220.690000] r10:c7375a20 r9:c7375a58 r8:80000005 r7:000000b1 r6:b6efe000 r5:c72a1ed0
[ 5220.700000] r4:00000000
[ 5220.700000] [<c00babc0>] (__do_fault) from [<c00bd9b0>] (handle_mm_fault+0x3e4/0x97c)
[ 5220.710000] r5:000000c0 r4:00000000
[ 5220.720000] [<c00bd5cc>] (handle_mm_fault) from [<c000f774>] (do_page_fault+0x138/0x2ac)
[ 5220.720000] r7:00000054 r6:b6efef9c r5:c72a1fb0 r4:c7a300e0
[ 5220.730000] [<c000f63c>] (do_page_fault) from [<c000f99c>] (do_translation_fault+0x2c/0xb4)
[ 5220.740000] r10:b6f9e000 r9:00000000 r8:00053177 r7:80000005 r6:b6efef9c r5:c070f91c
[ 5220.750000] r4:00000005
[ 5220.750000] [<c000f970>] (do_translation_fault) from [<c000fb38>] (do_PrefetchAbort+0x44/0x94)
[ 5220.760000] r7:c72a1fb0 r6:b6efef9c r5:c070f91c r4:00000005
[ 5220.760000] [<c000faf4>] (do_PrefetchAbort) from [<c0009f08>] (ret_from_exception+0x0/0x18)
[ 5220.770000] Exception stack(0xc72a1fb0 to 0xc72a1ff8)
[ 5220.780000] 1fa0: 00000000 00000000 beb25d54 beb25e71
[ 5220.780000] 1fc0: 00000000 00000000 000082f4 00000006 00000000 00000000 b6f9e000 beb25bf4
[ 5220.790000] 1fe0: 00000000 beb25bdc 0000847c b6efef9c 40000010 ffffffff
[ 5220.800000] r7:0005317f r6:ffffffff r5:40000010 r4:b6efef9c
[ 5220.800000] Mem-Info:
[ 5220.810000] active_anon:1659 inactive_anon:47 isolated_anon:0
[ 5220.810000] active_file:2 inactive_file:4 isolated_file:0
[ 5220.810000] unevictable:0 dirty:0 writeback:0 unstable:0
[ 5220.810000] slab_reclaimable:1098 slab_unreclaimable:26200
[ 5220.810000] mapped:3 shmem:60 pagetables:125 bounce:0
[ 5220.810000] free:344 free_pcp:45 free_cma:0
[ 5220.840000] Node 0 active_anon:6636kB inactive_anon:188kB active_file:8kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:12kB dirty:0kB writeback:0kB shmem:240kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
[ 5220.860000] Normal free:1376kB min:1396kB low:1744kB high:2092kB active_anon:6636kB inactive_anon:188kB active_file:8kB inactive_file:16kB unevictable:0kB writepending:0kB present:131072kB managed:122316kB mlocked:0kB kernel_stack:448kB pagetables:500kB bounce:0kB free_pcp:180kB local_pcp:180kB free_cma:0kB
[ 5220.890000] lowmem_reserve[]: 0 0
[ 5220.890000] Normal: 0*4kB 54*8kB (U) 29*16kB (U) 11*32kB (U) 2*64kB (U) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1376kB
[ 5220.900000] 66 total pagecache pages
[ 5220.910000] 32768 pages RAM
[ 5220.910000] 0 pages HighMem/MovableOnly
[ 5220.910000] 2189 pages reserved
[ 5220.920000] Unreclaimable slab info:
[ 5220.920000] Name Used Total
[ 5220.930000] ubi_wl_entry_slab 92KB 93KB
[ 5220.930000] bridge_fdb_cache 0KB 3KB
[ 5220.940000] fib6_nodes 0KB 3KB
[ 5220.940000] ip6_dst_cache 0KB 3KB
[ 5220.950000] RAWv6 4KB 7KB
[ 5220.950000] UDPv6 1KB 7KB
[ 5220.960000] TCPv6 3KB 6KB
[ 5220.960000] sd_ext_cdb 0KB 3KB
[ 5220.970000] sgpool-128 4KB 4KB
[ 5220.970000] sgpool-64 2KB 4KB
[ 5220.980000] sgpool-32 1KB 4KB
[ 5220.980000] sgpool-16 0KB 4KB
[ 5220.990000] sgpool-8 0KB 4KB
[ 5220.990000] nfs_commit_data 1KB 3KB
[ 5221.000000] nfs_write_data 18KB 19KB
[ 5221.000000] bio-1 0KB 3KB
[ 5221.010000] rpc_buffers 16KB 16KB
[ 5221.010000] rpc_tasks 1KB 4KB
[ 5221.020000] UNIX 14KB 15KB
[ 5221.030000] tcp_bind_bucket 0KB 3KB
[ 5221.030000] ip_fib_trie 0KB 3KB
[ 5221.040000] ip_fib_alias 0KB 3KB
[ 5221.040000] ip_dst_cache 1KB 4KB
[ 5221.050000] RAW 1KB 3KB
[ 5221.050000] UDP 2KB 7KB
[ 5221.060000] TCP 6KB 6KB
[ 5221.060000] eventpoll_pwq 0KB 3KB
[ 5221.070000] eventpoll_epi 1KB 3KB
[ 5221.070000] inotify_inode_mark 1KB 3KB
[ 5221.080000] request_queue 18KB 22KB
[ 5221.080000] blkdev_ioc 0KB 3KB
[ 5221.090000] bio-0 4KB 7KB
[ 5221.090000] biovec-max 90KB 90KB
[ 5221.100000] uid_cache 0KB 3KB
[ 5221.100000] dmaengine-unmap-2 0KB 3KB
[ 5221.110000] skbuff_head_cache 5KB 7KB
[ 5221.110000] configfs_dir_cache 0KB 3KB
[ 5221.120000] file_lock_ctx 0KB 3KB
[ 5221.130000] fsnotify_mark_connector 0KB 3KB
[ 5221.130000] shmem_inode_cache 205KB 210KB
[ 5221.140000] proc_dir_entry 30KB 30KB
[ 5221.140000] pde_opener 0KB 3KB
[ 5221.150000] seq_file 0KB 3KB
[ 5221.150000] kernfs_node_cache 672KB 673KB
[ 5221.160000] mnt_cache 5KB 7KB
[ 5221.160000] filp 52KB 58KB
[ 5221.170000] names_cache 24KB 24KB
[ 5221.170000] key_jar 0KB 4KB
[ 5221.180000] nsproxy 0KB 3KB
[ 5221.180000] vm_area_struct 92KB 103KB
[ 5221.190000] mm_struct 12KB 14KB
[ 5221.190000] fs_cache 0KB 3KB
[ 5221.200000] files_cache 5KB 11KB
[ 5221.200000] signal_cache 33KB 46KB
[ 5221.210000] sighand_cache 71KB 80KB
[ 5221.210000] task_struct 57KB 86KB
[ 5221.220000] cred_jar 13KB 20KB
[ 5221.220000] anon_vma_chain 32KB 46KB
[ 5221.230000] anon_vma 29KB 39KB
[ 5221.240000] pid 4KB 7KB
[ 5221.240000] debug_objects_cache 48KB 50KB
[ 5221.250000] trace_event_file 85KB 86KB
[ 5221.250000] ftrace_event_field 161KB 163KB
[ 5221.260000] pool_workqueue 1KB 4KB
[ 5221.260000] kmalloc-256k 256KB 256KB
[ 5221.270000] kmalloc-64k 64KB 64KB
[ 5221.270000] kmalloc-32k 32KB 32KB
[ 5221.280000] kmalloc-16k 32KB 32KB
[ 5221.280000] kmalloc-8k 128KB 128KB
[ 5221.290000] kmalloc-4k 104KB 104KB
[ 5221.290000] kmalloc-2k 102KB 108KB
[ 5221.300000] kmalloc-1k 191KB 192KB
[ 5221.300000] kmalloc-512 154KB 168KB
[ 5221.310000] kmalloc-256 64KB 80KB
[ 5221.310000] kmalloc-192 78KB 78KB
[ 5221.320000] kmalloc-128 22KB 24KB
[ 5221.320000] kmalloc-96 60KB 98KB
[ 5221.330000] kmalloc-32 97766KB 97770KB
[ 5221.340000] kmalloc-64 173KB 176KB
[ 5221.340000] kmem_cache 31KB 35KB
[ 5221.350000] Tasks state (memory values in pages):
[ 5221.350000] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 5221.360000] [ 617] 0 617 458 30 8192 0 0 bootlogd
[ 5221.370000] [ 653] 0 653 776 11 8192 0 0 syslogd
[ 5221.380000] [ 656] 0 656 776 12 6144 0 0 klogd
[ 5221.380000] [ 676] 0 676 1578 299 10240 0 0 conf-manager
[ 5221.390000] [ 679] 0 679 484 27 6144 0 0 conf-dispatcher
[ 5221.400000] [ 688] 0 688 432 17 8192 0 0 keypad
[ 5221.410000] [ 700] 0 700 904 79 8192 0 -1000 udevd
[ 5221.420000] [ 717] 81 717 1074 49 8192 0 0 dbus-daemon
[ 5221.430000] [ 747] 0 747 776 11 8192 0 0 respawn.sh
[ 5221.440000] [ 749] 0 749 1493 69 10240 0 0 connmand
[ 5221.440000] [ 751] 99 751 581 27 6144 0 0 dnsmasq
[ 5221.450000] [ 755] 0 755 572 21 6144 0 0 dropbear
[ 5221.460000] [ 759] 0 759 1083 75 10240 0 0 lighttpd
[ 5221.470000] [ 761] 0 761 4393 291 20480 0 0 php-cgi
[ 5221.480000] [ 763] 0 763 4393 291 22528 0 0 php-cgi
[ 5221.490000] [ 768] 0 768 777 39 6144 0 0 lns_networking.
[ 5221.500000] [ 839] 0 839 455 23 6144 0 0 inotifywait
[ 5221.500000] [ 897] 0 897 776 9 6144 0 0 ntpd
[ 5221.510000] [ 905] 0 905 1404 22 8192 0 0 crond
[ 5221.520000] [ 922] 0 922 484 21 8192 0 0 event-monitor
[ 5221.530000] [ 941] 0 941 1011 29 8192 0 0 sh
[ 5221.540000] [ 1062] 0 1062 383 11 6144 0 0 test_open
[ 5221.550000] [ 1063] 0 1063 592 31 8192 0 0 dropbear
[ 5221.550000] [ 1064] 0 1064 1011 30 8192 0 0 sh
[ 5221.560000] [ 5315] 0 5315 776 15 6144 0 0 check_online.sh
[ 5221.570000] [ 5316] 0 5316 776 15 4096 0 0 check_online.sh
[ 5221.580000] [ 5317] 0 5317 1159 53 10240 0 0 socat
[ 5221.590000] [ 5318] 0 5318 1016 37 10240 0 0 xsltproc
[ 5221.600000] [ 5319] 0 5319 777 9 8192 0 0 uniq
[ 5221.610000] Out of memory: Kill process 676 (conf-manager) score 9 or sacrifice child
[ 5221.610000] Killed process 676 (conf-manager) total-vm:6312kB, anon-rss:1196kB, file-rss:0kB, shmem-rss:0kB
full stabinfo:
slabinfo - version: 2.1 (statistics)
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail> : globalstat <listallocs> <maxobjs> <grown> <reaped> <error> <maxfreeable> <nodeallocs> <remotefrees> <alienoverflow> : cpustat <allochit> <allocmiss> <freehit> <freemiss>
ubi_wl_entry_slab 1964 1992 48 83 1 : tunables 32 16 0 : slabdata 24 24 0 : globalstat 1973 1973 24 0 0 0 0 0 0 : cpustat 1822 142 0 0
ubifs_inode_slab 66 189 432 9 1 : tunables 32 16 0 : slabdata 21 21 0 : globalstat 92965 477 53 0 0 0 0 0 0 : cpustat 354826 5836 355569 5041
bridge_fdb_cache 2 63 64 63 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 48 17 1 0 0 0 0 0 0 : cpustat 1 3 2 0
nf-frags 0 0 152 26 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
xfrm6_tunnel_spi 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ip6-frags 0 0 152 26 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fib6_nodes 6 63 64 63 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 32 17 1 0 0 0 0 0 0 : cpustat 4 2 0 0
ip6_dst_cache 3 25 160 25 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 640 19 1 0 0 0 0 0 0 : cpustat 32 40 69 0
PINGv6 0 0 832 9 2 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
RAWv6 5 9 832 9 2 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 13 9 1 0 0 0 0 0 0 : cpustat 4 2 1 0
UDPLITEv6 0 0 832 9 2 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
UDPv6 1 9 832 9 2 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 17 9 1 0 0 0 0 0 0 : cpustat 4 2 5 0
tw_sock_TCPv6 0 0 200 20 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
request_sock_TCPv6 0 0 256 16 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
TCPv6 1 4 1760 4 2 : tunables 24 12 0 : slabdata 1 1 0 : globalstat 4 4 1 0 0 0 0 0 0 : cpustat 1 1 1 0
nf_conntrack_expect 0 0 152 26 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
nf_conntrack 0 0 192 21 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
sd_ext_cdb 2 71 56 71 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-128 2 2 2048 2 1 : tunables 24 12 0 : slabdata 1 1 0 : globalstat 2 2 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-64 2 4 1024 4 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 4 4 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-32 2 8 512 8 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 8 8 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-16 2 16 256 16 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-8 2 32 128 32 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 1 1 0 0
nfs_direct_cache 0 0 208 19 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
nfs_commit_data 4 9 448 9 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 9 9 1 0 0 0 0 0 0 : cpustat 3 1 0 0
nfs_write_data 32 35 576 7 1 : tunables 32 16 0 : slabdata 5 5 0 : globalstat 35 35 5 0 0 0 0 0 0 : cpustat 27 5 0 0
nfs_read_data 0 0 576 7 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
nfs_inode_cache 0 0 608 6 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
nfs_page 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fat_inode_cache 0 0 424 9 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fat_cache 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_transaction_s 0 0 160 25 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_inode 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_journal_handle 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_journal_head 0 0 88 46 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_revoke_table_s 0 0 40 99 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_revoke_record_s 0 0 32 124 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_inode_cache 0 0 624 6 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_free_data 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_allocation_context 0 0 128 32 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_prealloc_space 0 0 88 46 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_system_zone 0 0 56 71 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_io_end 0 0 72 56 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_pending_reservation 0 0 40 99 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_extent_status 0 0 56 71 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
mbcache 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kioctx 0 0 224 18 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
aio_kiocb 0 0 96 42 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fanotify_event_info 0 0 56 71 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fsnotify_mark 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
dnotify_mark 0 0 72 56 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
dnotify_struct 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
dio 0 0 360 11 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
bio-1 4 25 160 25 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 3 1 0 0
fasync_cache 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
posix_timers_cache 0 0 192 21 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
rpc_inode_cache 0 0 352 11 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
rpc_buffers 8 8 2048 2 1 : tunables 24 12 0 : slabdata 4 4 0 : globalstat 8 8 4 0 0 0 0 0 0 : cpustat 4 4 0 0
rpc_tasks 8 32 128 32 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 7 1 0 0
UNIX 7 28 576 7 1 : tunables 32 16 0 : slabdata 4 4 0 : globalstat 65532 35 88 84 0 0 0 0 0 : cpustat 20071 4224 24288 0
ip4-frags 0 0 160 25 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
UDP-Lite 0 0 704 11 2 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
tcp_bind_bucket 1 63 64 63 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 4 1 4 0
inet_peer_cache 0 0 128 32 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
secpath_cache 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
xfrm_dst_cache 0 0 224 18 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
xfrm_state 0 0 576 7 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ip_fib_trie 12 71 56 71 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 32 19 1 0 0 0 0 0 0 : cpustat 11 2 1 0
ip_fib_alias 15 71 56 71 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 32 20 1 0 0 0 0 0 0 : cpustat 14 2 1 0
ip_dst_cache 10 32 128 32 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 1824 24 1 0 0 0 0 0 0 : cpustat 116 114 222 0
PING 0 0 672 6 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
RAW 2 6 672 6 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 10 6 1 0 0 0 0 0 0 : cpustat 12 2 12 0
UDP 0 0 704 11 2 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 994 11 2 2 0 0 0 0 0 : cpustat 580 123 703 0
tw_sock_TCP 0 0 200 20 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
request_sock_TCP 0 0 256 16 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 16 16 1 1 0 0 0 0 0 : cpustat 0 1 1 0
TCP 1 4 1664 4 2 : tunables 24 12 0 : slabdata 1 1 0 : globalstat 5 4 1 0 0 0 0 0 0 : cpustat 11 2 12 0
eventpoll_pwq 5 63 64 63 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 1392 48 1 0 0 0 0 0 0 : cpustat 61 87 142 1
eventpoll_epi 5 42 96 42 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 1386 42 1 0 0 0 0 0 0 : cpustat 61 87 143 0
inotify_inode_mark 15 56 72 56 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 48 39 1 0 0 0 0 0 0 : cpustat 21 3 9 0
scsi_data_buffer 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
request_queue 12 15 1536 5 2 : tunables 24 12 0 : slabdata 3 3 0 : globalstat 15 15 3 0 0 0 0 0 0 : cpustat 9 3 0 0
blkdev_requests 0 0 216 18 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
blkdev_ioc 1 51 80 51 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
bio-0 26 50 160 25 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 41 41 2 0 0 0 0 0 0 : cpustat 27 3 4 0
biovec-max 30 30 3072 2 2 : tunables 24 12 0 : slabdata 15 15 0 : globalstat 30 30 15 0 0 0 0 0 0 : cpustat 15 15 0 0
biovec-128 0 0 1536 5 2 : tunables 24 12 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
biovec-64 0 0 768 5 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
biovec-16 0 0 192 21 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
uid_cache 0 0 96 42 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 16 16 1 1 0 0 0 0 0 : cpustat 1 1 2 0
dmaengine-unmap-2 1 124 32 124 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
sock_inode_cache 26 66 352 11 1 : tunables 32 16 0 : slabdata 6 6 0 : globalstat 55763 77 7 0 0 0 0 0 0 : cpustat 7743 3489 11206 0
skbuff_fclone_cache 0 0 384 10 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 5630 10 552 552 0 0 0 0 0 : cpustat 122 563 685 0
skbuff_head_cache 12 42 192 21 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 56541 121 6 1 0 0 0 0 0 : cpustat 24027 3924 27932 9
configfs_dir_cache 1 51 80 51 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
file_lock_cache 0 0 144 28 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 1696 18 103 103 0 0 0 0 0 : cpustat 1338 106 1444 0
file_lock_ctx 1 83 48 83 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
fsnotify_mark_connector 15 99 40 99 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 32 32 1 0 0 0 0 0 0 : cpustat 20 2 7 0
shmem_inode_cache 545 560 384 10 1 : tunables 32 16 0 : slabdata 56 56 0 : globalstat 573 560 56 0 0 0 0 0 0 : cpustat 533 58 46 0
proc_dir_entry 203 208 152 26 1 : tunables 32 16 0 : slabdata 8 8 0 : globalstat 232 208 8 0 0 0 0 0 0 : cpustat 1655734 19 1655550 0
pde_opener 16 83 48 83 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 3392 16 208 207 0 0 0 0 0 : cpustat 379 212 590 0
proc_inode_cache 19 20 384 10 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 65673 330 58 24 0 2 0 0 0 : cpustat 17629 9256 26850 29
seq_file 16 39 104 39 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 3440 19 4 3 0 0 0 0 0 : cpustat 1822 215 2036 0
sigqueue 0 0 72 56 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 496 25 28 28 0 0 0 0 0 : cpustat 22 31 53 0
bdev_cache 13 18 448 9 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 26 18 2 0 0 0 0 0 0 : cpustat 14 3 4 0
kernfs_node_cache 6152 6156 112 36 1 : tunables 32 16 0 : slabdata 171 171 0 : globalstat 6164 6152 171 0 0 0 0 0 0 : cpustat 6046 512 406 0
mnt_cache 23 36 224 18 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 34 34 2 0 0 0 0 0 0 : cpustat 20 3 0 0
filp 94 325 160 25 1 : tunables 32 16 0 : slabdata 13 13 0 : globalstat 56563 366 16 1 0 1 0 0 0 : cpustat 2140672 3542 2141060 3073
inode_cache 7258 7260 344 11 1 : tunables 32 16 0 : slabdata 660 660 0 : globalstat 9668 8316 828 77 0 0 0 0 0 : cpustat 9994 928 3600 64
dentry 7885 8800 160 25 1 : tunables 32 16 0 : slabdata 352 352 0 : globalstat 209061 9941 398 0 0 1 0 0 0 : cpustat 579132 13240 572829 11669
names_cache 1 1 4096 1 1 : tunables 24 12 0 : slabdata 1 1 0 : globalstat 1216 11 965 964 0 0 0 0 0 : cpustat 2313916 1211 2315127 0
key_jar 3 32 128 32 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 2 1 0 0
buffer_head 0 0 88 46 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 16 16 1 1 0 0 0 0 0 : cpustat 3 1 4 0
nsproxy 1 71 56 71 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
vm_area_struct 196 646 120 34 1 : tunables 32 16 0 : slabdata 19 19 0 : globalstat 493665 1326 41 1 0 1 0 0 0 : cpustat 1030662 30889 1031088 30293
mm_struct 25 36 416 9 1 : tunables 32 16 0 : slabdata 4 4 0 : globalstat 19979 45 5 1 0 0 0 0 0 : cpustat 44416 1297 45703 0
fs_cache 18 124 32 124 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 1904 112 1 0 0 0 0 0 0 : cpustat 23929 119 24029 9
files_cache 18 63 192 21 1 : tunables 32 16 0 : slabdata 3 3 0 : globalstat 1913 105 5 0 0 0 0 0 0 : cpustat 23925 123 24028 10
signal_cache 45 72 608 6 1 : tunables 32 16 0 : slabdata 12 12 0 : globalstat 15230 120 20 1 0 0 0 0 0 : cpustat 23152 965 24070 10
sighand_cache 43 57 1312 3 1 : tunables 24 12 0 : slabdata 19 19 0 : globalstat 1401 117 48 8 0 6 0 0 0 : cpustat 23947 170 24066 14
task_struct 43 84 1056 7 2 : tunables 24 12 0 : slabdata 12 12 0 : globalstat 11531 119 18 0 0 1 0 0 0 : cpustat 23150 969 24064 18
cred_jar 70 160 128 32 1 : tunables 32 16 0 : slabdata 5 5 0 : globalstat 20529 144 5 0 0 0 0 0 0 : cpustat 56391 1284 57604 15
anon_vma_chain 161 781 56 71 1 : tunables 32 16 0 : slabdata 11 11 0 : globalstat 139671 1381 21 1 0 1 0 0 0 : cpustat 472282 8741 473000 7891
anon_vma 125 567 64 63 1 : tunables 32 16 0 : slabdata 9 9 0 : globalstat 68139 741 12 0 0 0 0 0 0 : cpustat 329676 4260 330198 3626
pid 50 126 64 63 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 15054 126 2 0 0 0 0 0 0 : cpustat 23178 941 24066 11
debug_objects_cache 1024 1079 48 83 1 : tunables 32 16 0 : slabdata 13 13 0 : globalstat 1028 1028 13 0 0 0 0 0 0 : cpustat 950 74 0 0
trace_event_file 1219 1232 72 56 1 : tunables 32 16 0 : slabdata 22 22 0 : globalstat 1219 1219 22 0 0 0 0 0 0 : cpustat 0 1219 0 0
ftrace_event_field 2953 2982 56 71 1 : tunables 32 16 0 : slabdata 42 42 0 : globalstat 2959 2959 42 0 0 0 0 0 0 : cpustat 2745 208 0 0
pool_workqueue 4 16 256 16 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 2 2 0 0
radix_tree_node 193 300 328 12 1 : tunables 32 16 0 : slabdata 25 25 0 : globalstat 60882 396 33 0 0 0 0 0 0 : cpustat 234553 3841 234990 3211
kmalloc-rcl-4M 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-2M 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-1M 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-512k 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-256k 0 0 262144 1 64 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-128k 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-64k 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-32k 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-16k 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-8k 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-4k 0 0 4096 1 1 : tunables 24 12 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-2k 0 0 2048 2 1 : tunables 24 12 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-1k 0 0 1024 4 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-512 0 0 512 8 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-256 0 0 256 16 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-192 0 0 192 21 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-128 0 0 128 32 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-96 0 0 96 42 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-64 1 64 64 64 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
kmalloc-rcl-32 0 0 32 124 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-4M 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-2M 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-1M 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-512k 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-256k 1 1 262144 1 64 : tunables 1 1 0 : slabdata 1 1 0 : globalstat 1 1 1 0 0 0 0 0 0 : cpustat 0 1 0 0
kmalloc-128k 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-64k 1 1 65536 1 16 : tunables 8 4 0 : slabdata 1 1 0 : globalstat 1 1 1 0 0 0 0 0 0 : cpustat 0 1 0 0
kmalloc-32k 1 1 32768 1 8 : tunables 8 4 0 : slabdata 1 1 0 : globalstat 1 1 1 0 0 0 0 0 0 : cpustat 0 1 0 0
kmalloc-16k 2 2 16384 1 4 : tunables 8 4 0 : slabdata 2 2 0 : globalstat 222 4 212 210 0 0 0 0 0 : cpustat 7107 220 7325 0
kmalloc-8k 9 9 8192 1 2 : tunables 8 4 0 : slabdata 9 9 0 : globalstat 503 20 155 146 0 4 0 0 0 : cpustat 22043876 253 22044039 82
kmalloc-4k 27 27 4096 1 1 : tunables 24 12 0 : slabdata 27 27 0 : globalstat 504 135 399 255 0 25 0 0 0 : cpustat 3415 419 3789 18
kmalloc-2k 54 54 2048 2 1 : tunables 24 12 0 : slabdata 27 27 0 : globalstat 2022 68 36 9 0 0 0 0 0 : cpustat 858 667 1474 0
kmalloc-1k 151 172 1024 4 1 : tunables 32 16 0 : slabdata 43 43 0 : globalstat 1056 196 54 11 0 3 0 0 0 : cpustat 453 153 453 2
kmalloc-512 302 320 512 8 1 : tunables 32 16 0 : slabdata 40 40 0 : globalstat 7933 400 51 3 0 2 0 0 0 : cpustat 2220216 547 2220469 8
kmalloc-256 293 304 256 16 1 : tunables 32 16 0 : slabdata 19 19 0 : globalstat 12361098 1616 53263 284 0 2 0 0 0 : cpustat 19771823 772713 19771803 772442
kmalloc-192 410 420 192 21 1 : tunables 32 16 0 : slabdata 20 20 0 : globalstat 3763 417 20 0 0 0 0 0 0 : cpustat 20526116 416 20526138 0
kmalloc-128 288 288 128 32 1 : tunables 32 16 0 : slabdata 9 9 0 : globalstat 11689 288 13 4 0 0 0 0 0 : cpustat 21015 1676 22409 0
kmalloc-96 666 966 96 42 1 : tunables 32 16 0 : slabdata 23 23 0 : globalstat 138190 1241 30 0 0 0 0 0 0 : cpustat 264861 8696 269873 3026
kmalloc-32 3317614 3317620 32 124 1 : tunables 32 16 0 : slabdata 26755 26755 0 : globalstat 3329656 3317661 26779 17 0 1 0 0 0 : cpustat 11621275 215178 8518750 104
kmalloc-64 3094 3136 64 64 1 : tunables 32 16 0 : slabdata 49 49 0 : globalstat 1643930 3440 20618 0 0 1 0 0 0 : cpustat 46206626 102866 46203878 102535
kmem_cache 169 189 192 21 1 : tunables 32 16 0 : slabdata 9 9 0 : globalstat 184 184 9 0 0 0 0 0 0 : cpustat 114 57 2 0
# cat /proc/meminfo
MemTotal: 122316 kB
MemFree: 1804 kB
MemAvailable: 2468 kB
Buffers: 0 kB
Cached: 716 kB
SwapCached: 0 kB
Active: 1148 kB
Inactive: 404 kB
Active(anon): 932 kB
Inactive(anon): 188 kB
Active(file): 216 kB
Inactive(file): 216 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 884 kB
Mapped: 472 kB
Shmem: 236 kB
KReclaimable: 4284 kB
Slab: 115048 kB
SReclaimable: 4284 kB
SUnreclaim: 110764 kB
KernelStack: 296 kB
PageTables: 136 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 61156 kB
Committed_AS: 5640 kB
VmallocTotal: 901120 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
Percpu: 64 kB
# cat /proc/slab_allocators
ubi_wl_entry_slab: 3 erase_aeb+0x34/0xe0
ubi_wl_entry_slab: 805 ubi_wl_init+0x148/0x4e4
ubi_wl_entry_slab: 1155 ubi_wl_init+0x218/0x4e4
ubi_wl_entry_slab: 1 ubi_scan_fastmap+0x6f8/0x8fc
ubifs_inode_slab: 50 ubifs_alloc_inode+0x24/0x60
sd_ext_cdb: 2 mempool_alloc_slab+0x24/0x28
ip_fib_trie: 12 fib_insert_alias+0x40/0x298
ip_fib_alias: 15 fib_table_insert+0x33c/0x49c
eventpoll_pwq: 5 ep_ptable_queue_proc+0x38/0xb4
inotify_inode_mark: 15 sys_inotify_add_watch+0x12c/0x2bc
request_queue: 12 blk_alloc_queue_node+0x34/0x280
blkdev_ioc: 1 create_task_io_context+0x28/0xe0
configfs_dir_cache: 1 configfs_new_dirent+0x30/0xb4
file_lock_ctx: 1 locks_get_lock_context+0x48/0x10c
fsnotify_mark_connector: 15 fsnotify_add_mark_locked+0xbc/0x2b0
shmem_inode_cache: 545 shmem_alloc_inode+0x24/0x38
proc_dir_entry: 202 __proc_create+0x160/0x22c
proc_dir_entry: 1 proc_net_ns_init+0x28/0xd0
pde_opener: 1 proc_reg_open+0x7c/0x140
proc_inode_cache: 6 proc_alloc_inode+0x24/0x5c
seq_file: 1 seq_open+0x44/0xa4
kernfs_node_cache: 6152 __kernfs_new_node+0x48/0x174
inode_cache: 7258 alloc_inode+0x40/0xa8
dentry: 7872 __d_alloc+0x2c/0x1b4
nsproxy: 1 create_new_namespaces+0x34/0x144
vm_area_struct: 70 vm_area_alloc+0x28/0x68
vm_area_struct: 100 vm_area_dup+0x28/0x58
anon_vma_chain: 41 __anon_vma_prepare+0x28/0x124
anon_vma_chain: 51 anon_vma_clone+0x3c/0x168
anon_vma_chain: 40 anon_vma_fork+0x88/0x144
anon_vma: 72 __anon_vma_prepare+0x50/0x124
anon_vma: 40 anon_vma_fork+0x60/0x144
debug_objects_cache: 1024 debug_objects_mem_init+0x8c/0x248
trace_event_file: 1219 trace_create_new_event+0x24/0x6c
ftrace_event_field: 2953 __trace_define_field+0x30/0x98
radix_tree_node: 11 __radix_tree_preload.constprop.4+0x28/0x80
radix_tree_node: 125 radix_tree_node_alloc.constprop.5+0x54/0xd8
radix_tree_node: 11 radix_tree_node_alloc.constprop.5+0x8c/0xd8
radix_tree_node: 45 xas_alloc+0x40/0xb0
^ permalink raw reply [flat|nested] 4+ messages in thread* DMA: atmel_serial: Opening and closing the serial device repeatedly causes kmalloc-32 slab leak
[not found] <22061488.b0eNpyQjWt@linux-7rm0>
2018-11-27 9:51 ` DMA: atmel_serial: Opening and closing the serial device repeatedly causes kmalloc-32 slab leak richard.genoud
@ 2018-11-27 9:55 ` richard.genoud
1 sibling, 0 replies; 4+ messages in thread
From: richard.genoud @ 2018-11-27 9:55 UTC (permalink / raw)
To: Ludovic Desroches, Nicolas Ferre, Maxime Ripard, Vinod Koul,
Alexandre Belloni
Cc: linux-arm-kernel, dmaengine, Linux Kernel, linux-serial, Mario Forner
[re-sending with Vinod's correct email address. Sorry for the noise. ]
[re-sending the bug report to the lists]
Le 16/11/2018 à 17:04, Mario Forner a écrit :
> Problem:
> When I open and close the serial device /dev/ttyS4 in a loop
> the amount of kmalloc-32 slabs increases slowly but steadily without limit.
>
> The serial device is configured in acme-aria.dts to use DMA.
>
> If DMA is disabled, the amount of kmalloc32-slabs remains constant
> over several hours, just fluctuating slightly.
>
> The serial device is accessed by atmel_serial.c which is evident from
> the drivers kernel log output.
>
> The bug was noticed on a device which had been running over several weeks and
> has accumulated ~86MB of unrelaimable kmalloc-32 slabs by now. Example:
>
> root@master1083:~# slabtop -o | head -10
> Active / Total Objects (% used) : 2704880 / 2716124 (99.6%)
> Active / Total Slabs (% used) : 23150 / 23150 (100.0%)
> Active / Total Caches (% used) : 57 / 76 (75.0%)
> Active / Total Size (% used) : 88893.62K / 89964.32K (98.8%)
> Minimum / Average / Maximum Object : 0.02K / 0.03K / 4096.00K
>
> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
> 2674804 2673840 99% 0.03K 21571 124 86284K kmalloc-32
> 7200 7104 98% 0.08K 144 50 576K kernfs_node_cache
> 6930 5370 77% 0.13K 231 30 924K dentry
>
> root@master1083:~# uptime
> 15:12:55 up 93 days, 16:54, 1 user, load average: 1.59, 2.31, 2.42
>
> Keywords: atmel, serial, kernel, leak, memory, dma, slab, kmalloc
>
> Kernel information:
>
> Kernel version:
> Linux version 4.2.6 (mario@linux-7rm0) (gcc version 4.9.3 (crosstool-NG ) ) #115 Fri Nov 16 11:05:22 CET 2018
> Linux version 4.9.124 (mario@linux-7rm0) (gcc version 4.9.3 (crosstool-NG ) ) #16 Fri Nov 16 13:54:25 CET 2018
>
> Most recent Kernel version which did not have the bug: unknown
>
> How to reproduce the bug:
>
> The kmalloc-32 slab count should be monitored every few minutes with "slabtop -o".
> The leak can be triggered by running the following script, given DMA has
> been enabled for ttyS4:
>
> #!/usr/bin/env python
> import serial
> import time
>
> while True:
> try:
> with serial.Serial(port = '/dev/ttyS4'):
> pass
> except Exception as e:
> print e
> finally:
> time.sleep(0.5)
> # end script
>
> Environment:
> Software:
> Linux master 4.2.6 #114 Fri Nov 16 10:14:30 CET 2018 armv5tejl GNU/Linux
>
> Binutils 2.25
> Util-linux 2.25.2
> Mount 2.25.2
> Module-init-tools 18
> E2fsprogs 1.42.12
> Linux C Library 2.19
> Dynamic linker (ldd) 2.19
> Linux C++ Library 6.0.20
> Procps 3.3.9
> Net-tools 1.60
> Sh-utils 8.23
> Udev 215
> Modules Loaded iptable_nat option usb_wwan usbserial
>
> Processor information:
> # cat /proc/cpuinfo
> processor : 0
> model name : ARM926EJ-S rev 5 (v5l)
> BogoMIPS : 198.76
> Features : swp half thumb fastmult edsp java
> CPU implementer : 0x41
> CPU architecture: 5TEJ
> CPU variant : 0x0
> CPU part : 0x926
> CPU revision : 5
>
> Hardware : Atmel AT91SAM9
> Revision : 0000
> Serial : 0000000000000000
>
> Module information:
> # cat /proc/modules
> iptable_nat 1720 0 - Live 0xbf0a5000
> option 28780 0 - Live 0xbf03c000
> usb_wwan 6876 1 option, Live 0xbf024000
> usbserial 23392 2 option,usb_wwan, Live 0xbf000000
>
> Loaded drivers:
> # cat /proc/ioports
> # cat /proc/iomem
> 00300000-00307fff : 300000.sram
> 00600000-006fffff : /ahb/ohci@00600000
> 00700000-007fffff : /ahb/ehci@00700000
> 20000000-2fffffff : System RAM
> 20008000-20577287 : Kernel code
> 205a8000-205f02b7 : Kernel data
> f0000000-f00000ff : /ahb/apb/spi@f0000000
> f8008000-f80080ff : /ahb/apb/timer@f8008000
> f800c000-f800c0ff : /ahb/apb/timer@f800c000
> f8010000-f80100ff : /ahb/apb/i2c@f8010000
> f8014000-f80140ff : /ahb/apb/i2c@f8014000
> f801c000-f801c1ff : atmel_serial
> f8020000-f80201ff : atmel_serial
> f8024000-f80241ff : atmel_serial
> f8028000-f80281ff : atmel_serial
> f802c000-f802c0ff : /ahb/apb/ethernet@f802c000
> f8034000-f80342ff : /ahb/apb/pwm@f8034000
> f804c000-f804c0ff : /ahb/apb/adc@f804c000
> ffffec00-ffffedff : at_hdmac
> ffffee00-ffffefff : at_hdmac
> fffff200-fffff3ff : atmel_serial
> fffff400-fffff5ff : /ahb/apb/pinctrl@fffff400/gpio@fffff400
> fffff600-fffff7ff : /ahb/apb/pinctrl@fffff400/gpio@fffff600
> fffff800-fffff9ff : /ahb/apb/pinctrl@fffff400/gpio@fffff800
> fffffa00-fffffbff : /ahb/apb/pinctrl@fffff400/gpio@fffffa00
> fffffe10-fffffe1f : /ahb/apb/shdwc@fffffe10
>
> Other Information:
> The serial device ttyS4 is configuered inside .dts file by
>
> usart3: serial@f8028000 {
> status = "okay";
> compatible = "atmel,at91sam9260-usart";
> reg = <0xf8028000 0x200>;
> interrupts = <8 IRQ_TYPE_LEVEL_HIGH 5>;
> pinctrl-names = "default";
> atmel,use-dma-tx;
> atmel,use-dma-rx;
> pinctrl-0 = <&pinctrl_usart3
> &pinctrl_usart3_rts~
> &pinctrl_usart3_cts>;
> linux,rs485-enabled-at-boot-time;
> rs485-rts-delay = <0 0>;
> };
>
> Other notes:
> I tried the SLUB allocator with slub_debug=U but opening serial device then
> resulted in a Kernel-Oops:
> at BUGON(ents < 0);
> in function: dma_map_sg_attrs
> in file: /include/asm-generic/dma-mapping-common.h
> So I could not gather information that way. I don't know if this is related.
> SLUB works if User Tracking is disabled.
>
> Enabling kmemleak did not detect any leaks.
>
> I also tried kernel version 4.19.1 and it exhibits the same behaviour, but I
> could not configure it to run reliably on the target system yet.
>
Hi all,
I reproduced the memory leak on my board (at91sam9g35-cm) with a 4.20-rc3.
It triggered an OOM after a couple of hours running a code like this:
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
int main(int argc, char **argv)
{
int fd;
do {
fd = open("/dev/ttyS1", O_RDONLY);
close(fd);
} while (true);
return 0;
}
As Mario pointed out, this only happens when atmel,use-dma-{r,t}x are
used in the device-tree.
Adding:
CONFIG_DEBUG_SLAB=y
CONFIG_DEBUG_SLAB_LEAK=y
Doesn't show anything suspect in /proc/slab_allocators
From what I found until now, it's something done in :
dma_request_slave_channel();
that leaks kmalloc-32
Mabe I missed something, but it seems that everything DMA related is
deallocated in atmel_release_{tx,rx}_dma().
Is this ringing a bell ?
The oom-killer, slabinfo, meminfo and slab_allocator traces follow.
NB: the -dirty flag is there beacause I removed some log from atmel-serial:
--- a/drivers/tty/serial/atmel_serial.c
+++ b/drivers/tty/serial/atmel_serial.c
@@ -1056,8 +1056,6 @@ static int atmel_prepare_tx_dma(struct uart_port *port)
atmel_port->chan_tx = dma_request_slave_channel(mfd_dev, "tx");
if (atmel_port->chan_tx == NULL)
goto chan_err;
- dev_info(port->dev, "using %s for tx DMA transfers\n",
- dma_chan_name(atmel_port->chan_tx));
spin_lock_init(&atmel_port->lock_tx);
sg_init_table(&atmel_port->sg_tx, 1);
@@ -1239,8 +1237,6 @@ static int atmel_prepare_rx_dma(struct uart_port *port)
atmel_port->chan_rx = dma_request_slave_channel(mfd_dev, "rx");
if (atmel_port->chan_rx == NULL)
goto chan_err;
- dev_info(port->dev, "using %s for rx DMA transfers\n",
- dma_chan_name(atmel_port->chan_rx));
spin_lock_init(&atmel_port->lock_rx);
sg_init_table(&atmel_port->sg_rx, 1);
OOM-killer trace :
[ 5220.560000] test_open invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0
[ 5220.580000] CPU: 0 PID: 1062 Comm: test_open Not tainted 4.20.0-rc3-dirty #31
[ 5220.580000] Hardware name: Atmel AT91SAM9
[ 5220.590000] Backtrace: [ 5220.590000] [<c000e020>] (dump_backtrace) from [<c000e2b8>] (show_stack+0x20/0x24)
[ 5220.600000] r6:00000009 r5:c0649bb5 r4:c72a1e18 r3:02b88502
[ 5220.600000] [<c000e298>] (show_stack) from [<c0521114>] (dump_stack+0x20/0x28)
[ 5220.610000] [<c05210f4>] (dump_stack) from [<c0097280>] (dump_header+0x6c/0x1c4)
[ 5220.620000] [<c0097214>] (dump_header) from [<c0096454>] (oom_kill_process+0x6c/0x3cc)
[ 5220.620000] r10:00000001 r8:c070e788 r7:c72a1e18 r6:00000009 r5:c0649bb5 r4:c7b64ea0
[ 5220.630000] [<c00963e8>] (oom_kill_process) from [<c009712c>] (out_of_memory+0x388/0x41c)
[ 5220.640000] r10:00000001 r9:00000000 r8:c070e788 r7:0000000b r6:c070e8e4 r5:c070e788
[ 5220.650000] r4:c72a1e18
[ 5220.650000] [<c0096da4>] (out_of_memory) from [<c009b1d8>] (__alloc_pages_nodemask+0x8b4/0xc48)
[ 5220.660000] r8:00000011 r7:00000000 r6:00000000 r5:00000000 r4:006200ca
[ 5220.670000] [<c009a924>] (__alloc_pages_nodemask) from [<c0092bd4>] (filemap_fault+0x384/0x54c)
[ 5220.680000] r10:c609b480 r9:00000000 r8:00000004 r7:00000000 r6:c72a1ed0 r5:c77dbb5c
[ 5220.680000] r4:006200ca
[ 5220.690000] [<c0092850>] (filemap_fault) from [<c00babe8>] (__do_fault+0x28/0x9c)
[ 5220.690000] r10:c7375a20 r9:c7375a58 r8:80000005 r7:000000b1 r6:b6efe000 r5:c72a1ed0
[ 5220.700000] r4:00000000
[ 5220.700000] [<c00babc0>] (__do_fault) from [<c00bd9b0>] (handle_mm_fault+0x3e4/0x97c)
[ 5220.710000] r5:000000c0 r4:00000000
[ 5220.720000] [<c00bd5cc>] (handle_mm_fault) from [<c000f774>] (do_page_fault+0x138/0x2ac)
[ 5220.720000] r7:00000054 r6:b6efef9c r5:c72a1fb0 r4:c7a300e0
[ 5220.730000] [<c000f63c>] (do_page_fault) from [<c000f99c>] (do_translation_fault+0x2c/0xb4)
[ 5220.740000] r10:b6f9e000 r9:00000000 r8:00053177 r7:80000005 r6:b6efef9c r5:c070f91c
[ 5220.750000] r4:00000005
[ 5220.750000] [<c000f970>] (do_translation_fault) from [<c000fb38>] (do_PrefetchAbort+0x44/0x94)
[ 5220.760000] r7:c72a1fb0 r6:b6efef9c r5:c070f91c r4:00000005
[ 5220.760000] [<c000faf4>] (do_PrefetchAbort) from [<c0009f08>] (ret_from_exception+0x0/0x18)
[ 5220.770000] Exception stack(0xc72a1fb0 to 0xc72a1ff8)
[ 5220.780000] 1fa0: 00000000 00000000 beb25d54 beb25e71
[ 5220.780000] 1fc0: 00000000 00000000 000082f4 00000006 00000000 00000000 b6f9e000 beb25bf4
[ 5220.790000] 1fe0: 00000000 beb25bdc 0000847c b6efef9c 40000010 ffffffff
[ 5220.800000] r7:0005317f r6:ffffffff r5:40000010 r4:b6efef9c
[ 5220.800000] Mem-Info:
[ 5220.810000] active_anon:1659 inactive_anon:47 isolated_anon:0
[ 5220.810000] active_file:2 inactive_file:4 isolated_file:0
[ 5220.810000] unevictable:0 dirty:0 writeback:0 unstable:0
[ 5220.810000] slab_reclaimable:1098 slab_unreclaimable:26200
[ 5220.810000] mapped:3 shmem:60 pagetables:125 bounce:0
[ 5220.810000] free:344 free_pcp:45 free_cma:0
[ 5220.840000] Node 0 active_anon:6636kB inactive_anon:188kB active_file:8kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:12kB dirty:0kB writeback:0kB shmem:240kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
[ 5220.860000] Normal free:1376kB min:1396kB low:1744kB high:2092kB active_anon:6636kB inactive_anon:188kB active_file:8kB inactive_file:16kB unevictable:0kB writepending:0kB present:131072kB managed:122316kB mlocked:0kB kernel_stack:448kB pagetables:500kB bounce:0kB free_pcp:180kB local_pcp:180kB free_cma:0kB
[ 5220.890000] lowmem_reserve[]: 0 0
[ 5220.890000] Normal: 0*4kB 54*8kB (U) 29*16kB (U) 11*32kB (U) 2*64kB (U) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1376kB
[ 5220.900000] 66 total pagecache pages
[ 5220.910000] 32768 pages RAM
[ 5220.910000] 0 pages HighMem/MovableOnly
[ 5220.910000] 2189 pages reserved
[ 5220.920000] Unreclaimable slab info:
[ 5220.920000] Name Used Total
[ 5220.930000] ubi_wl_entry_slab 92KB 93KB
[ 5220.930000] bridge_fdb_cache 0KB 3KB
[ 5220.940000] fib6_nodes 0KB 3KB
[ 5220.940000] ip6_dst_cache 0KB 3KB
[ 5220.950000] RAWv6 4KB 7KB
[ 5220.950000] UDPv6 1KB 7KB
[ 5220.960000] TCPv6 3KB 6KB
[ 5220.960000] sd_ext_cdb 0KB 3KB
[ 5220.970000] sgpool-128 4KB 4KB
[ 5220.970000] sgpool-64 2KB 4KB
[ 5220.980000] sgpool-32 1KB 4KB
[ 5220.980000] sgpool-16 0KB 4KB
[ 5220.990000] sgpool-8 0KB 4KB
[ 5220.990000] nfs_commit_data 1KB 3KB
[ 5221.000000] nfs_write_data 18KB 19KB
[ 5221.000000] bio-1 0KB 3KB
[ 5221.010000] rpc_buffers 16KB 16KB
[ 5221.010000] rpc_tasks 1KB 4KB
[ 5221.020000] UNIX 14KB 15KB
[ 5221.030000] tcp_bind_bucket 0KB 3KB
[ 5221.030000] ip_fib_trie 0KB 3KB
[ 5221.040000] ip_fib_alias 0KB 3KB
[ 5221.040000] ip_dst_cache 1KB 4KB
[ 5221.050000] RAW 1KB 3KB
[ 5221.050000] UDP 2KB 7KB
[ 5221.060000] TCP 6KB 6KB
[ 5221.060000] eventpoll_pwq 0KB 3KB
[ 5221.070000] eventpoll_epi 1KB 3KB
[ 5221.070000] inotify_inode_mark 1KB 3KB
[ 5221.080000] request_queue 18KB 22KB
[ 5221.080000] blkdev_ioc 0KB 3KB
[ 5221.090000] bio-0 4KB 7KB
[ 5221.090000] biovec-max 90KB 90KB
[ 5221.100000] uid_cache 0KB 3KB
[ 5221.100000] dmaengine-unmap-2 0KB 3KB
[ 5221.110000] skbuff_head_cache 5KB 7KB
[ 5221.110000] configfs_dir_cache 0KB 3KB
[ 5221.120000] file_lock_ctx 0KB 3KB
[ 5221.130000] fsnotify_mark_connector 0KB 3KB
[ 5221.130000] shmem_inode_cache 205KB 210KB
[ 5221.140000] proc_dir_entry 30KB 30KB
[ 5221.140000] pde_opener 0KB 3KB
[ 5221.150000] seq_file 0KB 3KB
[ 5221.150000] kernfs_node_cache 672KB 673KB
[ 5221.160000] mnt_cache 5KB 7KB
[ 5221.160000] filp 52KB 58KB
[ 5221.170000] names_cache 24KB 24KB
[ 5221.170000] key_jar 0KB 4KB
[ 5221.180000] nsproxy 0KB 3KB
[ 5221.180000] vm_area_struct 92KB 103KB
[ 5221.190000] mm_struct 12KB 14KB
[ 5221.190000] fs_cache 0KB 3KB
[ 5221.200000] files_cache 5KB 11KB
[ 5221.200000] signal_cache 33KB 46KB
[ 5221.210000] sighand_cache 71KB 80KB
[ 5221.210000] task_struct 57KB 86KB
[ 5221.220000] cred_jar 13KB 20KB
[ 5221.220000] anon_vma_chain 32KB 46KB
[ 5221.230000] anon_vma 29KB 39KB
[ 5221.240000] pid 4KB 7KB
[ 5221.240000] debug_objects_cache 48KB 50KB
[ 5221.250000] trace_event_file 85KB 86KB
[ 5221.250000] ftrace_event_field 161KB 163KB
[ 5221.260000] pool_workqueue 1KB 4KB
[ 5221.260000] kmalloc-256k 256KB 256KB
[ 5221.270000] kmalloc-64k 64KB 64KB
[ 5221.270000] kmalloc-32k 32KB 32KB
[ 5221.280000] kmalloc-16k 32KB 32KB
[ 5221.280000] kmalloc-8k 128KB 128KB
[ 5221.290000] kmalloc-4k 104KB 104KB
[ 5221.290000] kmalloc-2k 102KB 108KB
[ 5221.300000] kmalloc-1k 191KB 192KB
[ 5221.300000] kmalloc-512 154KB 168KB
[ 5221.310000] kmalloc-256 64KB 80KB
[ 5221.310000] kmalloc-192 78KB 78KB
[ 5221.320000] kmalloc-128 22KB 24KB
[ 5221.320000] kmalloc-96 60KB 98KB
[ 5221.330000] kmalloc-32 97766KB 97770KB
[ 5221.340000] kmalloc-64 173KB 176KB
[ 5221.340000] kmem_cache 31KB 35KB
[ 5221.350000] Tasks state (memory values in pages):
[ 5221.350000] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 5221.360000] [ 617] 0 617 458 30 8192 0 0 bootlogd
[ 5221.370000] [ 653] 0 653 776 11 8192 0 0 syslogd
[ 5221.380000] [ 656] 0 656 776 12 6144 0 0 klogd
[ 5221.380000] [ 676] 0 676 1578 299 10240 0 0 conf-manager
[ 5221.390000] [ 679] 0 679 484 27 6144 0 0 conf-dispatcher
[ 5221.400000] [ 688] 0 688 432 17 8192 0 0 keypad
[ 5221.410000] [ 700] 0 700 904 79 8192 0 -1000 udevd
[ 5221.420000] [ 717] 81 717 1074 49 8192 0 0 dbus-daemon
[ 5221.430000] [ 747] 0 747 776 11 8192 0 0 respawn.sh
[ 5221.440000] [ 749] 0 749 1493 69 10240 0 0 connmand
[ 5221.440000] [ 751] 99 751 581 27 6144 0 0 dnsmasq
[ 5221.450000] [ 755] 0 755 572 21 6144 0 0 dropbear
[ 5221.460000] [ 759] 0 759 1083 75 10240 0 0 lighttpd
[ 5221.470000] [ 761] 0 761 4393 291 20480 0 0 php-cgi
[ 5221.480000] [ 763] 0 763 4393 291 22528 0 0 php-cgi
[ 5221.490000] [ 768] 0 768 777 39 6144 0 0 lns_networking.
[ 5221.500000] [ 839] 0 839 455 23 6144 0 0 inotifywait
[ 5221.500000] [ 897] 0 897 776 9 6144 0 0 ntpd
[ 5221.510000] [ 905] 0 905 1404 22 8192 0 0 crond
[ 5221.520000] [ 922] 0 922 484 21 8192 0 0 event-monitor
[ 5221.530000] [ 941] 0 941 1011 29 8192 0 0 sh
[ 5221.540000] [ 1062] 0 1062 383 11 6144 0 0 test_open
[ 5221.550000] [ 1063] 0 1063 592 31 8192 0 0 dropbear
[ 5221.550000] [ 1064] 0 1064 1011 30 8192 0 0 sh
[ 5221.560000] [ 5315] 0 5315 776 15 6144 0 0 check_online.sh
[ 5221.570000] [ 5316] 0 5316 776 15 4096 0 0 check_online.sh
[ 5221.580000] [ 5317] 0 5317 1159 53 10240 0 0 socat
[ 5221.590000] [ 5318] 0 5318 1016 37 10240 0 0 xsltproc
[ 5221.600000] [ 5319] 0 5319 777 9 8192 0 0 uniq
[ 5221.610000] Out of memory: Kill process 676 (conf-manager) score 9 or sacrifice child
[ 5221.610000] Killed process 676 (conf-manager) total-vm:6312kB, anon-rss:1196kB, file-rss:0kB, shmem-rss:0kB
full stabinfo:
slabinfo - version: 2.1 (statistics)
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail> : globalstat <listallocs> <maxobjs> <grown> <reaped> <error> <maxfreeable> <nodeallocs> <remotefrees> <alienoverflow> : cpustat <allochit> <allocmiss> <freehit> <freemiss>
ubi_wl_entry_slab 1964 1992 48 83 1 : tunables 32 16 0 : slabdata 24 24 0 : globalstat 1973 1973 24 0 0 0 0 0 0 : cpustat 1822 142 0 0
ubifs_inode_slab 66 189 432 9 1 : tunables 32 16 0 : slabdata 21 21 0 : globalstat 92965 477 53 0 0 0 0 0 0 : cpustat 354826 5836 355569 5041
bridge_fdb_cache 2 63 64 63 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 48 17 1 0 0 0 0 0 0 : cpustat 1 3 2 0
nf-frags 0 0 152 26 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
xfrm6_tunnel_spi 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ip6-frags 0 0 152 26 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fib6_nodes 6 63 64 63 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 32 17 1 0 0 0 0 0 0 : cpustat 4 2 0 0
ip6_dst_cache 3 25 160 25 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 640 19 1 0 0 0 0 0 0 : cpustat 32 40 69 0
PINGv6 0 0 832 9 2 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
RAWv6 5 9 832 9 2 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 13 9 1 0 0 0 0 0 0 : cpustat 4 2 1 0
UDPLITEv6 0 0 832 9 2 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
UDPv6 1 9 832 9 2 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 17 9 1 0 0 0 0 0 0 : cpustat 4 2 5 0
tw_sock_TCPv6 0 0 200 20 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
request_sock_TCPv6 0 0 256 16 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
TCPv6 1 4 1760 4 2 : tunables 24 12 0 : slabdata 1 1 0 : globalstat 4 4 1 0 0 0 0 0 0 : cpustat 1 1 1 0
nf_conntrack_expect 0 0 152 26 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
nf_conntrack 0 0 192 21 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
sd_ext_cdb 2 71 56 71 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-128 2 2 2048 2 1 : tunables 24 12 0 : slabdata 1 1 0 : globalstat 2 2 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-64 2 4 1024 4 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 4 4 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-32 2 8 512 8 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 8 8 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-16 2 16 256 16 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 1 1 0 0
sgpool-8 2 32 128 32 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 1 1 0 0
nfs_direct_cache 0 0 208 19 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
nfs_commit_data 4 9 448 9 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 9 9 1 0 0 0 0 0 0 : cpustat 3 1 0 0
nfs_write_data 32 35 576 7 1 : tunables 32 16 0 : slabdata 5 5 0 : globalstat 35 35 5 0 0 0 0 0 0 : cpustat 27 5 0 0
nfs_read_data 0 0 576 7 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
nfs_inode_cache 0 0 608 6 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
nfs_page 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fat_inode_cache 0 0 424 9 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fat_cache 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_transaction_s 0 0 160 25 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_inode 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_journal_handle 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_journal_head 0 0 88 46 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_revoke_table_s 0 0 40 99 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
jbd2_revoke_record_s 0 0 32 124 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_inode_cache 0 0 624 6 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_free_data 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_allocation_context 0 0 128 32 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_prealloc_space 0 0 88 46 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_system_zone 0 0 56 71 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_io_end 0 0 72 56 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_pending_reservation 0 0 40 99 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ext4_extent_status 0 0 56 71 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
mbcache 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kioctx 0 0 224 18 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
aio_kiocb 0 0 96 42 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fanotify_event_info 0 0 56 71 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
fsnotify_mark 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
dnotify_mark 0 0 72 56 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
dnotify_struct 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
dio 0 0 360 11 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
bio-1 4 25 160 25 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 3 1 0 0
fasync_cache 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
posix_timers_cache 0 0 192 21 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
rpc_inode_cache 0 0 352 11 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
rpc_buffers 8 8 2048 2 1 : tunables 24 12 0 : slabdata 4 4 0 : globalstat 8 8 4 0 0 0 0 0 0 : cpustat 4 4 0 0
rpc_tasks 8 32 128 32 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 7 1 0 0
UNIX 7 28 576 7 1 : tunables 32 16 0 : slabdata 4 4 0 : globalstat 65532 35 88 84 0 0 0 0 0 : cpustat 20071 4224 24288 0
ip4-frags 0 0 160 25 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
UDP-Lite 0 0 704 11 2 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
tcp_bind_bucket 1 63 64 63 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 4 1 4 0
inet_peer_cache 0 0 128 32 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
secpath_cache 0 0 64 63 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
xfrm_dst_cache 0 0 224 18 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
xfrm_state 0 0 576 7 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
ip_fib_trie 12 71 56 71 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 32 19 1 0 0 0 0 0 0 : cpustat 11 2 1 0
ip_fib_alias 15 71 56 71 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 32 20 1 0 0 0 0 0 0 : cpustat 14 2 1 0
ip_dst_cache 10 32 128 32 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 1824 24 1 0 0 0 0 0 0 : cpustat 116 114 222 0
PING 0 0 672 6 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
RAW 2 6 672 6 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 10 6 1 0 0 0 0 0 0 : cpustat 12 2 12 0
UDP 0 0 704 11 2 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 994 11 2 2 0 0 0 0 0 : cpustat 580 123 703 0
tw_sock_TCP 0 0 200 20 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
request_sock_TCP 0 0 256 16 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 16 16 1 1 0 0 0 0 0 : cpustat 0 1 1 0
TCP 1 4 1664 4 2 : tunables 24 12 0 : slabdata 1 1 0 : globalstat 5 4 1 0 0 0 0 0 0 : cpustat 11 2 12 0
eventpoll_pwq 5 63 64 63 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 1392 48 1 0 0 0 0 0 0 : cpustat 61 87 142 1
eventpoll_epi 5 42 96 42 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 1386 42 1 0 0 0 0 0 0 : cpustat 61 87 143 0
inotify_inode_mark 15 56 72 56 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 48 39 1 0 0 0 0 0 0 : cpustat 21 3 9 0
scsi_data_buffer 0 0 48 83 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
request_queue 12 15 1536 5 2 : tunables 24 12 0 : slabdata 3 3 0 : globalstat 15 15 3 0 0 0 0 0 0 : cpustat 9 3 0 0
blkdev_requests 0 0 216 18 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
blkdev_ioc 1 51 80 51 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
bio-0 26 50 160 25 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 41 41 2 0 0 0 0 0 0 : cpustat 27 3 4 0
biovec-max 30 30 3072 2 2 : tunables 24 12 0 : slabdata 15 15 0 : globalstat 30 30 15 0 0 0 0 0 0 : cpustat 15 15 0 0
biovec-128 0 0 1536 5 2 : tunables 24 12 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
biovec-64 0 0 768 5 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
biovec-16 0 0 192 21 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
uid_cache 0 0 96 42 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 16 16 1 1 0 0 0 0 0 : cpustat 1 1 2 0
dmaengine-unmap-2 1 124 32 124 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
sock_inode_cache 26 66 352 11 1 : tunables 32 16 0 : slabdata 6 6 0 : globalstat 55763 77 7 0 0 0 0 0 0 : cpustat 7743 3489 11206 0
skbuff_fclone_cache 0 0 384 10 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 5630 10 552 552 0 0 0 0 0 : cpustat 122 563 685 0
skbuff_head_cache 12 42 192 21 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 56541 121 6 1 0 0 0 0 0 : cpustat 24027 3924 27932 9
configfs_dir_cache 1 51 80 51 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
file_lock_cache 0 0 144 28 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 1696 18 103 103 0 0 0 0 0 : cpustat 1338 106 1444 0
file_lock_ctx 1 83 48 83 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
fsnotify_mark_connector 15 99 40 99 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 32 32 1 0 0 0 0 0 0 : cpustat 20 2 7 0
shmem_inode_cache 545 560 384 10 1 : tunables 32 16 0 : slabdata 56 56 0 : globalstat 573 560 56 0 0 0 0 0 0 : cpustat 533 58 46 0
proc_dir_entry 203 208 152 26 1 : tunables 32 16 0 : slabdata 8 8 0 : globalstat 232 208 8 0 0 0 0 0 0 : cpustat 1655734 19 1655550 0
pde_opener 16 83 48 83 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 3392 16 208 207 0 0 0 0 0 : cpustat 379 212 590 0
proc_inode_cache 19 20 384 10 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 65673 330 58 24 0 2 0 0 0 : cpustat 17629 9256 26850 29
seq_file 16 39 104 39 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 3440 19 4 3 0 0 0 0 0 : cpustat 1822 215 2036 0
sigqueue 0 0 72 56 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 496 25 28 28 0 0 0 0 0 : cpustat 22 31 53 0
bdev_cache 13 18 448 9 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 26 18 2 0 0 0 0 0 0 : cpustat 14 3 4 0
kernfs_node_cache 6152 6156 112 36 1 : tunables 32 16 0 : slabdata 171 171 0 : globalstat 6164 6152 171 0 0 0 0 0 0 : cpustat 6046 512 406 0
mnt_cache 23 36 224 18 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 34 34 2 0 0 0 0 0 0 : cpustat 20 3 0 0
filp 94 325 160 25 1 : tunables 32 16 0 : slabdata 13 13 0 : globalstat 56563 366 16 1 0 1 0 0 0 : cpustat 2140672 3542 2141060 3073
inode_cache 7258 7260 344 11 1 : tunables 32 16 0 : slabdata 660 660 0 : globalstat 9668 8316 828 77 0 0 0 0 0 : cpustat 9994 928 3600 64
dentry 7885 8800 160 25 1 : tunables 32 16 0 : slabdata 352 352 0 : globalstat 209061 9941 398 0 0 1 0 0 0 : cpustat 579132 13240 572829 11669
names_cache 1 1 4096 1 1 : tunables 24 12 0 : slabdata 1 1 0 : globalstat 1216 11 965 964 0 0 0 0 0 : cpustat 2313916 1211 2315127 0
key_jar 3 32 128 32 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 2 1 0 0
buffer_head 0 0 88 46 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 16 16 1 1 0 0 0 0 0 : cpustat 3 1 4 0
nsproxy 1 71 56 71 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
vm_area_struct 196 646 120 34 1 : tunables 32 16 0 : slabdata 19 19 0 : globalstat 493665 1326 41 1 0 1 0 0 0 : cpustat 1030662 30889 1031088 30293
mm_struct 25 36 416 9 1 : tunables 32 16 0 : slabdata 4 4 0 : globalstat 19979 45 5 1 0 0 0 0 0 : cpustat 44416 1297 45703 0
fs_cache 18 124 32 124 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 1904 112 1 0 0 0 0 0 0 : cpustat 23929 119 24029 9
files_cache 18 63 192 21 1 : tunables 32 16 0 : slabdata 3 3 0 : globalstat 1913 105 5 0 0 0 0 0 0 : cpustat 23925 123 24028 10
signal_cache 45 72 608 6 1 : tunables 32 16 0 : slabdata 12 12 0 : globalstat 15230 120 20 1 0 0 0 0 0 : cpustat 23152 965 24070 10
sighand_cache 43 57 1312 3 1 : tunables 24 12 0 : slabdata 19 19 0 : globalstat 1401 117 48 8 0 6 0 0 0 : cpustat 23947 170 24066 14
task_struct 43 84 1056 7 2 : tunables 24 12 0 : slabdata 12 12 0 : globalstat 11531 119 18 0 0 1 0 0 0 : cpustat 23150 969 24064 18
cred_jar 70 160 128 32 1 : tunables 32 16 0 : slabdata 5 5 0 : globalstat 20529 144 5 0 0 0 0 0 0 : cpustat 56391 1284 57604 15
anon_vma_chain 161 781 56 71 1 : tunables 32 16 0 : slabdata 11 11 0 : globalstat 139671 1381 21 1 0 1 0 0 0 : cpustat 472282 8741 473000 7891
anon_vma 125 567 64 63 1 : tunables 32 16 0 : slabdata 9 9 0 : globalstat 68139 741 12 0 0 0 0 0 0 : cpustat 329676 4260 330198 3626
pid 50 126 64 63 1 : tunables 32 16 0 : slabdata 2 2 0 : globalstat 15054 126 2 0 0 0 0 0 0 : cpustat 23178 941 24066 11
debug_objects_cache 1024 1079 48 83 1 : tunables 32 16 0 : slabdata 13 13 0 : globalstat 1028 1028 13 0 0 0 0 0 0 : cpustat 950 74 0 0
trace_event_file 1219 1232 72 56 1 : tunables 32 16 0 : slabdata 22 22 0 : globalstat 1219 1219 22 0 0 0 0 0 0 : cpustat 0 1219 0 0
ftrace_event_field 2953 2982 56 71 1 : tunables 32 16 0 : slabdata 42 42 0 : globalstat 2959 2959 42 0 0 0 0 0 0 : cpustat 2745 208 0 0
pool_workqueue 4 16 256 16 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 2 2 0 0
radix_tree_node 193 300 328 12 1 : tunables 32 16 0 : slabdata 25 25 0 : globalstat 60882 396 33 0 0 0 0 0 0 : cpustat 234553 3841 234990 3211
kmalloc-rcl-4M 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-2M 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-1M 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-512k 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-256k 0 0 262144 1 64 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-128k 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-64k 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-32k 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-16k 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-8k 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-4k 0 0 4096 1 1 : tunables 24 12 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-2k 0 0 2048 2 1 : tunables 24 12 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-1k 0 0 1024 4 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-512 0 0 512 8 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-256 0 0 256 16 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-192 0 0 192 21 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-128 0 0 128 32 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-96 0 0 96 42 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-rcl-64 1 64 64 64 1 : tunables 32 16 0 : slabdata 1 1 0 : globalstat 16 16 1 0 0 0 0 0 0 : cpustat 0 1 0 0
kmalloc-rcl-32 0 0 32 124 1 : tunables 32 16 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-4M 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-2M 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-1M 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-512k 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-256k 1 1 262144 1 64 : tunables 1 1 0 : slabdata 1 1 0 : globalstat 1 1 1 0 0 0 0 0 0 : cpustat 0 1 0 0
kmalloc-128k 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0 : globalstat 0 0 0 0 0 0 0 0 0 : cpustat 0 0 0 0
kmalloc-64k 1 1 65536 1 16 : tunables 8 4 0 : slabdata 1 1 0 : globalstat 1 1 1 0 0 0 0 0 0 : cpustat 0 1 0 0
kmalloc-32k 1 1 32768 1 8 : tunables 8 4 0 : slabdata 1 1 0 : globalstat 1 1 1 0 0 0 0 0 0 : cpustat 0 1 0 0
kmalloc-16k 2 2 16384 1 4 : tunables 8 4 0 : slabdata 2 2 0 : globalstat 222 4 212 210 0 0 0 0 0 : cpustat 7107 220 7325 0
kmalloc-8k 9 9 8192 1 2 : tunables 8 4 0 : slabdata 9 9 0 : globalstat 503 20 155 146 0 4 0 0 0 : cpustat 22043876 253 22044039 82
kmalloc-4k 27 27 4096 1 1 : tunables 24 12 0 : slabdata 27 27 0 : globalstat 504 135 399 255 0 25 0 0 0 : cpustat 3415 419 3789 18
kmalloc-2k 54 54 2048 2 1 : tunables 24 12 0 : slabdata 27 27 0 : globalstat 2022 68 36 9 0 0 0 0 0 : cpustat 858 667 1474 0
kmalloc-1k 151 172 1024 4 1 : tunables 32 16 0 : slabdata 43 43 0 : globalstat 1056 196 54 11 0 3 0 0 0 : cpustat 453 153 453 2
kmalloc-512 302 320 512 8 1 : tunables 32 16 0 : slabdata 40 40 0 : globalstat 7933 400 51 3 0 2 0 0 0 : cpustat 2220216 547 2220469 8
kmalloc-256 293 304 256 16 1 : tunables 32 16 0 : slabdata 19 19 0 : globalstat 12361098 1616 53263 284 0 2 0 0 0 : cpustat 19771823 772713 19771803 772442
kmalloc-192 410 420 192 21 1 : tunables 32 16 0 : slabdata 20 20 0 : globalstat 3763 417 20 0 0 0 0 0 0 : cpustat 20526116 416 20526138 0
kmalloc-128 288 288 128 32 1 : tunables 32 16 0 : slabdata 9 9 0 : globalstat 11689 288 13 4 0 0 0 0 0 : cpustat 21015 1676 22409 0
kmalloc-96 666 966 96 42 1 : tunables 32 16 0 : slabdata 23 23 0 : globalstat 138190 1241 30 0 0 0 0 0 0 : cpustat 264861 8696 269873 3026
kmalloc-32 3317614 3317620 32 124 1 : tunables 32 16 0 : slabdata 26755 26755 0 : globalstat 3329656 3317661 26779 17 0 1 0 0 0 : cpustat 11621275 215178 8518750 104
kmalloc-64 3094 3136 64 64 1 : tunables 32 16 0 : slabdata 49 49 0 : globalstat 1643930 3440 20618 0 0 1 0 0 0 : cpustat 46206626 102866 46203878 102535
kmem_cache 169 189 192 21 1 : tunables 32 16 0 : slabdata 9 9 0 : globalstat 184 184 9 0 0 0 0 0 0 : cpustat 114 57 2 0
# cat /proc/meminfo
MemTotal: 122316 kB
MemFree: 1804 kB
MemAvailable: 2468 kB
Buffers: 0 kB
Cached: 716 kB
SwapCached: 0 kB
Active: 1148 kB
Inactive: 404 kB
Active(anon): 932 kB
Inactive(anon): 188 kB
Active(file): 216 kB
Inactive(file): 216 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 884 kB
Mapped: 472 kB
Shmem: 236 kB
KReclaimable: 4284 kB
Slab: 115048 kB
SReclaimable: 4284 kB
SUnreclaim: 110764 kB
KernelStack: 296 kB
PageTables: 136 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 61156 kB
Committed_AS: 5640 kB
VmallocTotal: 901120 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
Percpu: 64 kB
# cat /proc/slab_allocators ubi_wl_entry_slab: 3 erase_aeb+0x34/0xe0
ubi_wl_entry_slab: 805 ubi_wl_init+0x148/0x4e4
ubi_wl_entry_slab: 1155 ubi_wl_init+0x218/0x4e4
ubi_wl_entry_slab: 1 ubi_scan_fastmap+0x6f8/0x8fc
ubifs_inode_slab: 50 ubifs_alloc_inode+0x24/0x60
sd_ext_cdb: 2 mempool_alloc_slab+0x24/0x28
ip_fib_trie: 12 fib_insert_alias+0x40/0x298
ip_fib_alias: 15 fib_table_insert+0x33c/0x49c
eventpoll_pwq: 5 ep_ptable_queue_proc+0x38/0xb4
inotify_inode_mark: 15 sys_inotify_add_watch+0x12c/0x2bc
request_queue: 12 blk_alloc_queue_node+0x34/0x280
blkdev_ioc: 1 create_task_io_context+0x28/0xe0
configfs_dir_cache: 1 configfs_new_dirent+0x30/0xb4
file_lock_ctx: 1 locks_get_lock_context+0x48/0x10c
fsnotify_mark_connector: 15 fsnotify_add_mark_locked+0xbc/0x2b0
shmem_inode_cache: 545 shmem_alloc_inode+0x24/0x38
proc_dir_entry: 202 __proc_create+0x160/0x22c
proc_dir_entry: 1 proc_net_ns_init+0x28/0xd0
pde_opener: 1 proc_reg_open+0x7c/0x140
proc_inode_cache: 6 proc_alloc_inode+0x24/0x5c
seq_file: 1 seq_open+0x44/0xa4
kernfs_node_cache: 6152 __kernfs_new_node+0x48/0x174
inode_cache: 7258 alloc_inode+0x40/0xa8
dentry: 7872 __d_alloc+0x2c/0x1b4
nsproxy: 1 create_new_namespaces+0x34/0x144
vm_area_struct: 70 vm_area_alloc+0x28/0x68
vm_area_struct: 100 vm_area_dup+0x28/0x58
anon_vma_chain: 41 __anon_vma_prepare+0x28/0x124
anon_vma_chain: 51 anon_vma_clone+0x3c/0x168
anon_vma_chain: 40 anon_vma_fork+0x88/0x144
anon_vma: 72 __anon_vma_prepare+0x50/0x124
anon_vma: 40 anon_vma_fork+0x60/0x144
debug_objects_cache: 1024 debug_objects_mem_init+0x8c/0x248
trace_event_file: 1219 trace_create_new_event+0x24/0x6c
ftrace_event_field: 2953 __trace_define_field+0x30/0x98
radix_tree_node: 11 __radix_tree_preload.constprop.4+0x28/0x80
radix_tree_node: 125 radix_tree_node_alloc.constprop.5+0x54/0xd8
radix_tree_node: 11 radix_tree_node_alloc.constprop.5+0x8c/0xd8
radix_tree_node: 45 xas_alloc+0x40/0xb0
^ permalink raw reply [flat|nested] 4+ messages in thread