Re: CRASH: General protection fault in z3fold

[Vlastimil Babka <vbabka@suse.cz>]

+CC Vitaly

On 9/5/19 6:05 AM, Agustín DallʼAlba wrote:
> Hello,
> 
> I've been experiencing crashes using zswap with the z3fold allocator. The
> crashes happen once I'm using more memory than fits in the zpool. I can
> reliably reproduce the crash on v5.3-rc7 with these steps:
> 
> 1. Enable swap. My swap is located on a small 16 GB SSD that I dedicate to
>    this purpose, encrypted with dm-crypt. I didn't test other backing devices.
> 2. echo z3fold > /sys/module/zswap/parameters/zpool
>    echo lz4 > /sys/module/zswap/parameters/compressor
>    echo 1 > /sys/module/zswap/parameters/enabled
> 3. `watch -d grep -r . /sys/kernel/debug/zswap/` to monitor the statistics.
>    The most important parameter is pool_limit_hit.
> 4. Use a lot of memory. I spawn several `stress -m 1 --vm-hang 0&`.
> 5. At some point, pool_limit_hit starts increasing. Soon after TRACE 1
>    (at the bottom of the email) appears in dmesg.
> 6. Now at least 1 process is in an unkillable state and I must reset my CPU.
> 
> Additional notes:
>  - I can't reproduce the issue with zbud or zsmalloc. For now I'm using
>    zsmalloc as a workaround.
>  - Sometimes TRACE 2 and/or TRACE 3 appear in dmesg after or before
>    the protection fault. 
>  - If I run swapoff after the fault it seems to only move a handful of
>    pages per second so I haven't left it running long enough to see if
>    it ever finishes.
> 
> I first noticed this bug on 5.2.11 but I noticed there had been several
> commits to z3fold in 5.3 so I compiled rc7 using the Arch Linux config file.
> 
> I think this isn't a regression: A few months ago I tried zswap and ended up
> disabling it because it crashed my machine, but I didn't have time to figure
> out why back then. But that might have been a different bug. I'm willing
> to test older versions, of course.
> 
> I'm sorry if this has already been reported, I've seen a few similar reports
> but not one exactly like this.
> 
> Kind regards,
> 
> Agustín.
> 
> 
> Additional information:
> 
> /proc/version
> Linux version 5.3.0-rc7-1-ARCH (agustin@hostname) (gcc version 9.1.0 (GCC)) #1 SMP PREEMPT Wed Sep 4 17:23:45 -03 2019
> 
> /proc/iomem
> 00000000-00000fff : Reserved
> 00001000-00057fff : System RAM
> 00058000-00058fff : Reserved
> 00059000-0008bfff : System RAM
> 0008c000-0009ffff : Reserved
> 000a0000-000bffff : PCI Bus 0000:00
> 000c0000-000cfdff : Video ROM
> 000d0000-000d3fff : pnp 00:00
> 000d4000-000d7fff : pnp 00:00
> 000d8000-000dbfff : pnp 00:00
> 000dc000-000dffff : pnp 00:00
> 000e0000-000fffff : Reserved
>   000f0000-000fffff : System ROM
> 00100000-c006d017 : System RAM
>   66600000-67200e20 : Kernel code
>   67200e21-6794a3ff : Kernel data
>   67e87000-681fffff : Kernel bss
> c006d018-c007d657 : System RAM
> c007d658-c007e017 : System RAM
> c007e018-c008e057 : System RAM
> c008e058-cbd01fff : System RAM
> cbd02000-ccbfdfff : Reserved
> ccbfe000-ccd7dfff : ACPI Non-volatile Storage
> ccd7e000-ccdfdfff : ACPI Tables
> ccdfe000-ccdfefff : System RAM
> ccdff000-ccdfffff : MSFT0101:00
>   ccdff000-ccdff02f : MSFT0101:00
>   ccdff080-ccdfffff : MSFT0101:00
> cce00000-cdffffff : RAM buffer
> ce000000-cfffffff : Reserved
>   ce000000-cfffffff : Graphics Stolen Memory
> d0000000-febfffff : PCI Bus 0000:00
>   d0000000-d000ffff : pnp 00:05
>   d0010000-d001ffff : pnp 00:05
>   e0000000-efffffff : 0000:00:02.0
>   f0000000-f0ffffff : 0000:00:02.0
>   f1000000-f10fffff : PCI Bus 0000:03
>     f1000000-f1001fff : 0000:03:00.0
>       f1000000-f1001fff : iwlwifi
>   f1100000-f11fffff : PCI Bus 0000:02
>     f1100000-f1100fff : 0000:02:00.0
>       f1100000-f1100fff : rtsx_pci
>   f1200000-f121ffff : 0000:00:19.0
>     f1200000-f121ffff : e1000e
>   f1220000-f122ffff : 0000:00:14.0
>     f1220000-f122ffff : xhci-hcd
>   f1230000-f1233fff : 0000:00:03.0
>     f1230000-f1233fff : ICH HD audio
>   f1234000-f1237fff : 0000:00:1b.0
>     f1234000-f1237fff : ICH HD audio
>   f1238000-f12380ff : 0000:00:1f.3
>   f1239000-f123901f : 0000:00:16.0
>     f1239000-f123901f : mei_me
>   f123b000-f123bfff : 0000:00:1f.6
>     f123b000-f123bfff : Intel PCH thermal driver
>   f123c000-f123c7ff : 0000:00:1f.2
>     f123c000-f123c7ff : ahci
>   f123d000-f123d3ff : 0000:00:1d.0
>     f123d000-f123d3ff : ehci_hcd
>   f123e000-f123efff : 0000:00:19.0
>     f123e000-f123efff : e1000e
>   f8000000-fbffffff : PCI MMCONFIG 0000 [bus 00-3f]
>     f80f8000-f80f8fff : Reserved
> fec00000-fec003ff : IOAPIC 0
> fed00000-fed003ff : HPET 0
>   fed00000-fed003ff : PNP0103:00
> fed10000-fed17fff : pnp 00:01
> fed18000-fed18fff : pnp 00:01
> fed19000-fed19fff : pnp 00:01
> fed1c000-fed1ffff : Reserved
>   fed1c000-fed1ffff : pnp 00:01
>     fed1f410-fed1f414 : iTCO_wdt.0.auto
>       fed1f410-fed1f414 : iTCO_wdt.0.auto
>     fed1f800-fed1f9ff : intel-spi
>       fed1f800-fed1f9ff : intel-spi
> fed40000-fed4bfff : PCI Bus 0000:00
>   fed45000-fed4bfff : pnp 00:01
> fed70000-fed70fff : MSFT0101:00
>   fed70000-fed70fff : MSFT0101:00
> fed90000-fed90fff : dmar0
> fed91000-fed91fff : dmar1
> fee00000-fee00fff : Local APIC
> 100000000-12dffffff : System RAM
> 12e000000-12fffffff : RAM buffer
> 
> /proc/config.gz
> (attached)
> 
> ========================================================
> TRACE 1: WARNING z3fold.c:428 + general protection fault
> ========================================================
> 
> ------------[ cut here ]------------
> WARNING: CPU: 2 PID: 144 at mm/z3fold.c:428 handle_to_buddy.cold+0xc/0x13
> Modules linked in: ccm bnep fuse snd_hda_codec_hdmi uvcvideo joydev mousedev rmi_smbus rmi_core videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common btusb videodev btrtl btbcm btintel bluetooth mc msr intel_rapl_msr intel_rapl_common ecdh_generic ecc x86_pkg_temp_thermal crc16 i915 intel_powerclamp coretemp kvm_intel lz4 lz4_compress iwlmvm i2c_algo_bit drm_kms_helper mac80211 kvm drm ofpart snd_hda_codec_realtek cmdlinepart libarc4 nls_iso8859_1 snd_hda_codec_generic intel_spi_platform nls_cp437 intel_spi vfat spi_nor snd_hda_intel wmi_bmof mei_wdt iwlwifi mei_hdcp irqbypass intel_gtt fat iTCO_wdt intel_cstate snd_hda_codec agpgart mtd syscopyarea tpm_crb intel_uncore iTCO_vendor_support snd_hda_core tpm_tis rtsx_pci_ms psmouse snd_hwdep tpm_tis_core thinkpad_acpi input_leds cfg80211 sysfillrect pcspkr intel_rapl_perf nvram sysimgblt memstick ledtrig_audio tpm snd_pcm wmi battery rfkill ac fb_sys_fops mei_me rng_core snd_timer evdev e1000e intel_pch_thermal
>  mac_hid mei snd lpc_ich i2c_i801 soundcore crypto_user ip_tables x_tables btrfs libcrc32c crc32c_generic xor raid6_pq dm_crypt dm_mod sd_mod crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel rtsx_pci_sdmmc serio_raw mmc_core atkbd libps2 aesni_intel ahci aes_x86_64 crypto_simd libahci xhci_pci cryptd glue_helper xhci_hcd libata scsi_mod ehci_pci ehci_hcd rtsx_pci i8042 serio
> CPU: 2 PID: 144 Comm: kswapd0 Not tainted 5.3.0-rc7-1-ARCH #1
> Hardware name: LENOVO 20BXCTO1WW/20BXCTO1WW, BIOS JBET63WW (1.27 ) 11/10/2016
> RIP: 0010:handle_to_buddy.cold+0xc/0x13
> Code: 44 24 10 85 c0 0f 85 61 ff ff ff 4c 8b 44 24 18 4d 85 c0 0f 85 f8 fe ff ff e9 f7 fd ff ff 48 c7 c7 80 c7 ea 94 e8 42 c2 e5 ff <0f> 0b e9 6f de ff ff 48 c7 c7 80 c7 ea 94 e8 2f c2 e5 ff 0f 0b 48
> RSP: 0018:ffffa7ab0023f830 EFLAGS: 00010246
> RAX: 0000000000000024 RBX: ffff93a5fe157001 RCX: 0000000000000000
> RDX: 0000000000000000 RSI: ffff93a669a97708 RDI: 00000000ffffffff
> RBP: ffff93a5fe157000 R08: 0000000000000532 R09: 0000000000000004
> R10: 0000000000000000 R11: 0000000000000001 R12: ffff93a5fe157001
> R13: ffff93a5fe157010 R14: ffff93a64a505848 R15: ffff93a64a505840
> FS:  0000000000000000(0000) GS:ffff93a669a80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007fb7ee08a928 CR3: 000000005b20a001 CR4: 00000000003606e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  z3fold_zpool_map+0x76/0x100
>  zswap_writeback_entry+0x5b/0x3b0
>  z3fold_zpool_shrink+0x28b/0x4a0
>  zswap_frontswap_store+0x12a/0x6a0
>  __frontswap_store+0xa6/0xf0
>  swap_writepage+0x39/0x70
>  pageout.isra.0+0x12b/0x360
>  shrink_page_list+0x74b/0xb80
>  shrink_inactive_list+0x24f/0x410
>  shrink_node_memcg+0x258/0x7b0
>  shrink_node+0xe8/0x4f0
>  balance_pgdat+0x2e3/0x530
>  kswapd+0x200/0x3f0
>  ? wait_woken+0x70/0x70
>  kthread+0xfb/0x130
>  ? balance_pgdat+0x530/0x530
>  ? kthread_park+0x80/0x80
>  ret_from_fork+0x35/0x40
> ---[ end trace c799dc3361263fe2 ]---
> general protection fault: 0000 [#1] PREEMPT SMP PTI
> CPU: 2 PID: 144 Comm: kswapd0 Tainted: G        W         5.3.0-rc7-1-ARCH #1
> Hardware name: LENOVO 20BXCTO1WW/20BXCTO1WW, BIOS JBET63WW (1.27 ) 11/10/2016
> RIP: 0010:handle_to_buddy+0x20/0x30
> Code: 8b 47 38 48 c1 e0 0c c3 66 90 0f 1f 44 00 00 53 48 89 fb 83 e7 01 0f 85 7e 21 00 00 48 8b 03 5b 48 89 c2 48 81 e2 00 f0 ff ff <0f> b6 52 52 29 d0 83 e0 03 c3 66 0f 1f 44 00 00 0f 1f 44 00 00 55
> RSP: 0018:ffffa7ab0023f838 EFLAGS: 00010206
> RAX: 00ffff93a5fe1570 RBX: ffffd09502f855c0 RCX: 0000000000000000
> RDX: 00ffff93a5fe1000 RSI: ffff93a669a97708 RDI: 00000000ffffffff
> RBP: ffff93a5fe157000 R08: 0000000000000532 R09: 0000000000000004
> R10: 0000000000000000 R11: 0000000000000001 R12: ffff93a5fe157001
> R13: ffff93a5fe157010 R14: ffff93a64a505848 R15: ffff93a64a505840
> FS:  0000000000000000(0000) GS:ffff93a669a80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007fb7ee08a928 CR3: 000000005b20a001 CR4: 00000000003606e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  z3fold_zpool_map+0x76/0x100
>  zswap_writeback_entry+0x5b/0x3b0
>  z3fold_zpool_shrink+0x28b/0x4a0
>  zswap_frontswap_store+0x12a/0x6a0
>  __frontswap_store+0xa6/0xf0
>  swap_writepage+0x39/0x70
>  pageout.isra.0+0x12b/0x360
>  shrink_page_list+0x74b/0xb80
>  shrink_inactive_list+0x24f/0x410
>  shrink_node_memcg+0x258/0x7b0
>  shrink_node+0xe8/0x4f0
>  balance_pgdat+0x2e3/0x530
>  kswapd+0x200/0x3f0
>  ? wait_woken+0x70/0x70
>  kthread+0xfb/0x130
>  ? balance_pgdat+0x530/0x530
>  ? kthread_park+0x80/0x80
>  ret_from_fork+0x35/0x40
> Modules linked in: ccm bnep fuse snd_hda_codec_hdmi uvcvideo joydev mousedev rmi_smbus rmi_core videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common btusb videodev btrtl btbcm btintel bluetooth mc msr intel_rapl_msr intel_rapl_common ecdh_generic ecc x86_pkg_temp_thermal crc16 i915 intel_powerclamp coretemp kvm_intel lz4 lz4_compress iwlmvm i2c_algo_bit drm_kms_helper mac80211 kvm drm ofpart snd_hda_codec_realtek cmdlinepart libarc4 nls_iso8859_1 snd_hda_codec_generic intel_spi_platform nls_cp437 intel_spi vfat spi_nor snd_hda_intel wmi_bmof mei_wdt iwlwifi mei_hdcp irqbypass intel_gtt fat iTCO_wdt intel_cstate snd_hda_codec agpgart mtd syscopyarea tpm_crb intel_uncore iTCO_vendor_support snd_hda_core tpm_tis rtsx_pci_ms psmouse snd_hwdep tpm_tis_core thinkpad_acpi input_leds cfg80211 sysfillrect pcspkr intel_rapl_perf nvram sysimgblt memstick ledtrig_audio tpm snd_pcm wmi battery rfkill ac fb_sys_fops mei_me rng_core snd_timer evdev e1000e intel_pch_thermal
>  mac_hid mei snd lpc_ich i2c_i801 soundcore crypto_user ip_tables x_tables btrfs libcrc32c crc32c_generic xor raid6_pq dm_crypt dm_mod sd_mod crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel rtsx_pci_sdmmc serio_raw mmc_core atkbd libps2 aesni_intel ahci aes_x86_64 crypto_simd libahci xhci_pci cryptd glue_helper xhci_hcd libata scsi_mod ehci_pci ehci_hcd rtsx_pci i8042 serio
> ---[ end trace c799dc3361263fe3 ]---
> RIP: 0010:handle_to_buddy+0x20/0x30
> Code: 8b 47 38 48 c1 e0 0c c3 66 90 0f 1f 44 00 00 53 48 89 fb 83 e7 01 0f 85 7e 21 00 00 48 8b 03 5b 48 89 c2 48 81 e2 00 f0 ff ff <0f> b6 52 52 29 d0 83 e0 03 c3 66 0f 1f 44 00 00 0f 1f 44 00 00 55
> RSP: 0018:ffffa7ab0023f838 EFLAGS: 00010206
> RAX: 00ffff93a5fe1570 RBX: ffffd09502f855c0 RCX: 0000000000000000
> RDX: 00ffff93a5fe1000 RSI: ffff93a669a97708 RDI: 00000000ffffffff
> RBP: ffff93a5fe157000 R08: 0000000000000532 R09: 0000000000000004
> R10: 0000000000000000 R11: 0000000000000001 R12: ffff93a5fe157001
> R13: ffff93a5fe157010 R14: ffff93a64a505848 R15: ffff93a64a505840
> FS:  0000000000000000(0000) GS:ffff93a669a80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007fb7ee08a928 CR3: 000000005b20a001 CR4: 00000000003606e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> note: kswapd0[144] exited with preempt_count 1
> 
> 
> =====================================
> TRACE 2: z3fold_zpool_destroy blocked
> =====================================
> 
> INFO: task kworker/2:3:335 blocked for more than 122 seconds.
>       Not tainted 5.3.0-rc7-1-ARCH #1
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> kworker/2:3     D    0   335      2 0x80004080
> Workqueue: events __zswap_pool_release
> Call Trace:
>  ? __schedule+0x27f/0x6d0
>  schedule+0x43/0xd0
>  z3fold_zpool_destroy+0xe9/0x130
>  ? wait_woken+0x70/0x70
>  zpool_destroy_pool+0x5c/0x90
>  __zswap_pool_release+0x6a/0xb0
>  process_one_work+0x1d1/0x3a0
>  worker_thread+0x4a/0x3d0
>  kthread+0xfb/0x130
>  ? process_one_work+0x3a0/0x3a0
>  ? kthread_park+0x80/0x80
>  ret_from_fork+0x35/0x40
> 
> 
> ================================
> TRACE 3: page allocation failure (probably unrelated)
> ================================
> gnome-shell: page allocation failure: order:0, mode:0x400d0(__GFP_IO|__GFP_FS|__GFP_COMP|__GFP_RECLAIMABLE), nodemask=(null),cpuset=/,mems_allowed=0
> CPU: 2 PID: 1022 Comm: gnome-shell Not tainted 5.3.0-rc7-1-ARCH #1
> Hardware name: LENOVO 20BXCTO1WW/20BXCTO1WW, BIOS JBET63WW (1.27 ) 11/10/2016
> Call Trace:
>  dump_stack+0x5c/0x80
>  warn_alloc.cold+0x78/0xf8
>  __alloc_pages_nodemask+0x107d/0x10b0
>  new_slab+0x29a/0xbe0
>  ___slab_alloc+0x44c/0x5d0
>  ? xas_nomem+0x49/0x70
>  ? xas_alloc+0x9b/0xc0
>  ? kmem_cache_alloc+0x16f/0x210
>  ? xas_nomem+0x49/0x70
>  __slab_alloc.isra.0+0x52/0x70
>  ? xas_nomem+0x49/0x70
>  kmem_cache_alloc+0x1e3/0x210
>  xas_nomem+0x49/0x70
>  add_to_swap_cache+0x264/0x320
>  __read_swap_cache_async+0x112/0x220
>  swap_cluster_readahead+0x1e2/0x320
>  shmem_swapin+0x74/0xc0
>  shmem_swapin_page+0x51c/0x770
>  ? find_get_entry+0x101/0x160
>  shmem_getpage_gfp.isra.0+0x3dd/0x8c0
>  shmem_read_mapping_page_gfp+0x48/0x80
>  shmem_get_pages+0x21d/0x5b0 [i915]
>  __i915_gem_object_get_pages+0x54/0x60 [i915]
>  __i915_vma_do_pin+0x294/0x450 [i915]
>  eb_lookup_vmas+0x7ce/0xb10 [i915]
>  i915_gem_do_execbuffer+0x60f/0x12f0 [i915]
>  ? __alloc_pages_nodemask+0x1c4/0x10b0
>  i915_gem_execbuffer2_ioctl+0x1d3/0x3c0 [i915]
>  ? put_swap_page+0x102/0x2e0
>  ? i915_gem_execbuffer_ioctl+0x2d0/0x2d0 [i915]
>  drm_ioctl_kernel+0xb8/0x100 [drm]
>  drm_ioctl+0x23d/0x3d0 [drm]
>  ? i915_gem_execbuffer_ioctl+0x2d0/0x2d0 [i915]
>  do_vfs_ioctl+0x43d/0x6c0
>  ? syscall_trace_enter+0x1f2/0x2e0
>  ksys_ioctl+0x5e/0x90
>  __x64_sys_ioctl+0x16/0x20
>  do_syscall_64+0x5f/0x1c0
>  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> RIP: 0033:0x7f500ca5421b
> Code: 0f 1e fa 48 8b 05 75 8c 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 45 8c 0c 00 f7 d8 64 89 01 48
> RSP: 002b:00007ffdeab46518 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> RAX: ffffffffffffffda RBX: 00007ffdeab46560 RCX: 00007f500ca5421b
> RDX: 00007ffdeab46560 RSI: 0000000040406469 RDI: 000000000000000b
> RBP: 0000000040406469 R08: 000055ea80aa8cd0 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 000055ea80aab240
> R13: 000000000000000b R14: ffffffffffffffff R15: 00007f4ffe539730
> Mem-Info:
> active_anon:505791 inactive_anon:137312 isolated_anon:53
>  active_file:7074 inactive_file:8072 isolated_file:32
>  unevictable:2125 dirty:56 writeback:0 unstable:0
>  slab_reclaimable:14348 slab_unreclaimable:31867
>  mapped:5631 shmem:10740 pagetables:5907 bounce:0
>  free:45505 free_pcp:908 free_cma:0
> Node 0 active_anon:2023164kB inactive_anon:549248kB active_file:28296kB inactive_file:32288kB unevictable:8500kB isolated(anon):212kB isolated(file):128kB mapped:22524kB dirty:224kB writeback:0kB shmem:42960kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
> Node 0 DMA free:15456kB min:272kB low:340kB high:408kB active_anon:160kB inactive_anon:68kB active_file:0kB inactive_file:16kB unevictable:0kB writepending:0kB present:15912kB managed:15872kB mlocked:0kB kernel_stack:0kB pagetables:20kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
> lowmem_reserve[]: 0 3137 3801 3801 3801
> Node 0 DMA32 free:160740kB min:157952kB low:171840kB high:185728kB active_anon:1861456kB inactive_anon:523796kB active_file:27964kB inactive_file:32576kB unevictable:8240kB writepending:224kB present:3322892kB managed:3234336kB mlocked:48kB kernel_stack:7092kB pagetables:19796kB bounce:0kB free_pcp:3176kB local_pcp:120kB free_cma:0kB
> lowmem_reserve[]: 0 0 663 663 663
> Node 0 Normal free:5824kB min:38188kB low:41124kB high:44060kB active_anon:162152kB inactive_anon:25216kB active_file:68kB inactive_file:276kB unevictable:260kB writepending:0kB present:753664kB managed:679748kB mlocked:16kB kernel_stack:3692kB pagetables:3812kB bounce:0kB free_pcp:296kB local_pcp:0kB free_cma:0kB
> lowmem_reserve[]: 0 0 0 0 0
> Node 0 DMA: 17*4kB (UM) 7*8kB (UM) 1*16kB (U) 2*32kB (UM) 3*64kB (UM) 2*128kB (UM) 0*256kB 1*512kB (M) 2*1024kB (UM) 0*2048kB 3*4096kB (ME) = 15500kB
> Node 0 DMA32: 1155*4kB (UM) 8387*8kB (UM) 2986*16kB (UME) 1229*32kB (MEH) 0*64kB 0*128kB 1*256kB (H) 1*512kB (H) 1*1024kB (H) 0*2048kB 0*4096kB = 160612kB
> Node 0 Normal: 1439*4kB (UMH) 20*8kB (UMH) 2*16kB (UH) 3*32kB (H) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 6044kB
> Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
> Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
> 122450 total pagecache pages
> 96549 pages in swap cache
> Swap cache stats: add 14474738, delete 14376799, find 7531326/8981020
> Free swap  = 13216084kB
> Total swap = 15638612kB
> 1023117 pages RAM
> 0 pages HighMem/MovableOnly
> 40628 pages reserved
> 0 pages hwpoisoned
> SLUB: Unable to allocate memory on node -1, gfp=0xc0(__GFP_IO|__GFP_FS)
>   cache: radix_tree_node, object size: 576, buffer size: 584, default order: 2, min order: 0
>   node 0: slabs: 410, objs: 11144, free: 63
>