* [x86? mm? fs? 4.15-rc6] Random oopses by simple write under memory pressure.
@ 2018-01-05 14:45 Tetsuo Handa
2018-01-09 10:39 ` [mm? 4.15-rc7] " Tetsuo Handa
0 siblings, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-05 14:45 UTC (permalink / raw)
To: x86, linux-mm
Using current linux.git (e1915c8195b38393), I get various oopses shown below.
Note that I'm testing x86_32 kernel using x86_64 CPUs. Note that *pde = 00000000 in most cases.
----------------------------------------
localhost login: [ 64.800994] BUG: unable to handle kernel paging request at 1b4150b6
[ 64.803072] IP: lock_page_memcg+0x3c/0x80
[ 64.804408] *pde = 00000000
[ 64.805352] Oops: 0000 [#1] SMP
[ 64.806418] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp vmw_balloon vmw_vmci pcspkr sg i2c_piix4 shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
[ 64.823138] CPU: 4 PID: 3928 Comm: a.out Not tainted 4.15.0-rc6+ #258
[ 64.824992] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 64.828027] EIP: lock_page_memcg+0x3c/0x80
[ 64.829193] EFLAGS: 00010202 CPU: 4
[ 64.830193] EAX: f33fa6e0 EBX: 1b414f46 ECX: 00000011 EDX: 00000000
[ 64.831968] ESI: 00000010 EDI: f33fa6e0 EBP: ec3e3a84 ESP: ec3e3a78
[ 64.833733] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 64.835533] CR0: 80050033 CR2: 1b4150b6 CR3: 2ba99000 CR4: 000406d0
[ 64.837416] Call Trace:
[ 64.838140] page_remove_rmap+0x92/0x280
[ 64.839276] try_to_unmap_one+0x202/0x530
[ 64.840423] rmap_walk_file+0xf0/0x1e0
[ 64.841497] rmap_walk+0x32/0x60
[ 64.842448] try_to_unmap+0x4d/0xd0
[ 64.843463] ? page_remove_rmap+0x280/0x280
[ 64.844654] ? page_not_mapped+0x10/0x10
[ 64.845781] ? page_get_anon_vma+0x80/0x80
[ 64.847085] shrink_page_list+0x3e2/0xe40
[ 64.848235] shrink_inactive_list+0x1b2/0x440
[ 64.849476] shrink_node_memcg+0x34a/0x770
[ 64.850684] shrink_node+0xbb/0x2e0
[ 64.852554] do_try_to_free_pages+0xb2/0x300
[ 64.854599] try_to_free_pages+0x20b/0x330
----------------------------------------
----------------------------------------
localhost login: [ 205.084518] BUG: unable to handle kernel NULL pointer dereference at 00000104
[ 205.087492] IP: free_pcppages_bulk+0x8e/0x300
[ 205.088554] WARNING: CPU: 5 PID: 1817 at fs/xfs/xfs_aops.c:1468 xfs_vm_set_page_dirty+0x125/0x210 [xfs]
[ 205.088555] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg vmw_balloon pcspkr i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw e1000 mptspi scsi_transport_spi ata_piix mptscsih libata mptbase
[ 205.088582] CPU: 5 PID: 1817 Comm: a.out Not tainted 4.15.0-rc6+ #258
[ 205.088583] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 205.088604] EIP: xfs_vm_set_page_dirty+0x125/0x210 [xfs]
[ 205.088605] EFLAGS: 00010046 CPU: 5
[ 205.088606] EAX: b6000011 EBX: f247dd58 ECX: f247dd58 EDX: f247dd4c
[ 205.088607] ESI: 00000246 EDI: f4b7b8c8 EBP: f36b9a8c ESP: f36b9a68
[ 205.088609] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 205.088610] CR0: 80050033 CR2: 38e5e008 CR3: 32881000 CR4: 000406d0
[ 205.088689] Call Trace:
[ 205.088696] set_page_dirty+0x3d/0x90
[ 205.088699] try_to_unmap_one+0x36b/0x530
[ 205.088702] rmap_walk_file+0xf0/0x1e0
[ 205.088704] rmap_walk+0x32/0x60
[ 205.088706] try_to_unmap+0x4d/0xd0
[ 205.088708] ? page_remove_rmap+0x280/0x280
[ 205.088709] ? page_not_mapped+0x10/0x10
[ 205.088711] ? page_get_anon_vma+0x80/0x80
[ 205.088714] shrink_page_list+0x3e2/0xe40
[ 205.088717] ? clockevents_program_event+0xa5/0x160
[ 205.088721] shrink_inactive_list+0x1b2/0x440
[ 205.088723] shrink_node_memcg+0x34a/0x770
[ 205.088726] shrink_node+0xbb/0x2e0
[ 205.088728] do_try_to_free_pages+0xb2/0x300
[ 205.088730] try_to_free_pages+0x20b/0x330
[ 205.088733] __alloc_pages_slowpath+0x2fb/0x6d9
[ 205.088736] ? __pagevec_lru_add_fn+0xdb/0x190
[ 205.088738] __alloc_pages_nodemask+0x17a/0x190
[ 205.088742] do_anonymous_page+0xab/0x490
[ 205.088744] handle_mm_fault+0x531/0x8b0
[ 205.088746] ? pick_next_task_fair+0xe1/0x490
[ 205.088749] ? sched_clock_cpu+0x13/0x120
[ 205.088752] ? __do_page_fault+0x4e0/0x4e0
[ 205.088754] __do_page_fault+0x1ef/0x4e0
[ 205.088756] ? __do_page_fault+0x4e0/0x4e0
[ 205.088758] do_page_fault+0x1a/0x20
[ 205.088762] common_exception+0x6f/0x76
[ 205.088763] EIP: 0x80484be
[ 205.088764] EFLAGS: 00010202 CPU: 5
[ 205.088765] EAX: 026a6000 EBX: 80000000 ECX: 3a597008 EDX: 00000000
[ 205.088766] ESI: 7ff00000 EDI: 00000000 EBP: bf89aa48 ESP: bf89aa20
[ 205.088767] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 205.088768] Code: e4 8d 58 0c 89 d8 e8 5b b1 c2 c8 89 c6 8b 45 e8 8b 50 04 85 d2 74 5b 8b 40 14 a8 01 0f 85 ce 00 00 00 8b 45 e8 8b 00 a8 08 75 7b <0f> ff 8b 7d e8 8b 55 e4 89 f8 e8 8c 71 6f c8 8b 47 14 a8 01 0f
[ 205.088792] ---[ end trace ffb022c0c4b1f1da ]---
[ 205.245444] *pde = 00000000
[ 205.247303] Oops: 0002 [#1] SMP
[ 205.249245] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg vmw_balloon pcspkr i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw e1000 mptspi scsi_transport_spi ata_piix mptscsih libata mptbase
[ 205.278257] CPU: 3 PID: 363 Comm: kworker/3:2 Tainted: G W 4.15.0-rc6+ #258
[ 205.282307] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 205.288393] Workqueue: mm_percpu_wq drain_local_pages_wq
[ 205.291486] EIP: free_pcppages_bulk+0x8e/0x300
[ 205.294272] EFLAGS: 00010093 CPU: 3
[ 205.296725] EAX: 00000200 EBX: f4b7b8dc ECX: c18dbc80 EDX: 00000100
[ 205.300201] ESI: f4b7b8c8 EDI: c18db9e0 EBP: f6429ef4 ESP: f6429ebc
[ 205.303655] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 205.306816] CR0: 80050033 CR2: 00000104 CR3: 2876b000 CR4: 000406d0
[ 205.310373] Call Trace:
[ 205.312512] drain_pages_zone+0x31/0x50
[ 205.315176] drain_pages+0x35/0x50
[ 205.317650] drain_local_pages+0x25/0x30
[ 205.320316] drain_local_pages_wq+0xa/0x10
[ 205.323025] process_one_work+0xe7/0x240
[ 205.325684] worker_thread+0x182/0x360
[ 205.328275] ? __wake_up_locked+0x22/0x30
[ 205.330995] kthread+0xd1/0x100
[ 205.333395] ? rescuer_thread+0x2d0/0x2d0
[ 205.336087] ? kthread_associate_blkcg+0x80/0x80
[ 205.338997] ret_from_fork+0x19/0x24
[ 205.341530] Code: 40 0c 89 75 e0 39 c6 74 d4 8b 45 dc 83 f8 03 0f 44 45 d4 89 45 dc 8d b4 26 00 00 00 00 8b 45 e0 8b 58 04 8b 43 04 8d 73 ec 8b 13 <89> 42 04 89 10 8b 43 f4 c7 03 00 01 00 00 c7 43 04 00 02 00 00
[ 205.352013] EIP: free_pcppages_bulk+0x8e/0x300 SS:ESP: 0068:f6429ebc
[ 205.355540] CR2: 0000000000000104
[ 205.358007] ---[ end trace ffb022c0c4b1f1db ]---
[ 205.358008] BUG: unable to handle kernel paging request at c10746c4
[ 205.358015] IP: _raw_spin_lock_irqsave+0x1c/0x40
[ 205.358015] *pde = 010001e1
[ 205.358018] Oops: 0003 [#2] SMP
[ 205.358018] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg vmw_balloon pcspkr i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw e1000 mptspi scsi_transport_spi ata_piix mptscsih libata mptbase
[ 205.358051] CPU: 5 PID: 1817 Comm: a.out Tainted: G D W 4.15.0-rc6+ #258
[ 205.358051] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 205.358054] EIP: _raw_spin_lock_irqsave+0x1c/0x40
[ 205.358054] EFLAGS: 00010046 CPU: 5
[ 205.358055] EAX: 00000000 EBX: 00000001 ECX: c10746c4 EDX: 00000000
[ 205.358057] ESI: 00000206 EDI: f31afe80 EBP: f36b9a70 ESP: f36b9a68
[ 205.358058] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 205.358059] CR0: 80050033 CR2: c10746c4 CR3: 32881000 CR4: 000406d0
[ 205.358165] Call Trace:
[ 205.358169] ? dequeue_entity+0xd0/0x290
[ 205.358171] ? dequeue_entity+0x244/0x290
[ 205.358175] lock_page_memcg+0x25/0x80
[ 205.358178] page_remove_rmap+0x92/0x280
[ 205.358180] try_to_unmap_one+0x202/0x530
[ 205.358182] rmap_walk_file+0xf0/0x1e0
[ 205.358184] rmap_walk+0x32/0x60
[ 205.358186] try_to_unmap+0x4d/0xd0
[ 205.358188] ? page_remove_rmap+0x280/0x280
[ 205.358189] ? page_not_mapped+0x10/0x10
[ 205.358191] ? page_get_anon_vma+0x80/0x80
[ 205.358195] shrink_page_list+0x3e2/0xe40
[ 205.358198] ? clockevents_program_event+0xa5/0x160
[ 205.358201] shrink_inactive_list+0x1b2/0x440
[ 205.358204] shrink_node_memcg+0x34a/0x770
[ 205.358206] shrink_node+0xbb/0x2e0
[ 205.358208] do_try_to_free_pages+0xb2/0x300
[ 205.358210] try_to_free_pages+0x20b/0x330
[ 205.358213] __alloc_pages_slowpath+0x2fb/0x6d9
[ 205.358215] ? __pagevec_lru_add_fn+0xdb/0x190
[ 205.358218] __alloc_pages_nodemask+0x17a/0x190
[ 205.358221] do_anonymous_page+0xab/0x490
[ 205.358223] handle_mm_fault+0x531/0x8b0
[ 205.358224] ? pick_next_task_fair+0xe1/0x490
[ 205.358226] ? sched_clock_cpu+0x13/0x120
[ 205.358229] ? __do_page_fault+0x4e0/0x4e0
[ 205.358231] __do_page_fault+0x1ef/0x4e0
[ 205.358233] ? __do_page_fault+0x4e0/0x4e0
[ 205.358234] do_page_fault+0x1a/0x20
[ 205.358237] common_exception+0x6f/0x76
[ 205.358238] EIP: 0x80484be
[ 205.358239] EFLAGS: 00010202 CPU: 5
[ 205.358240] EAX: 026a6000 EBX: 80000000 ECX: 3a597008 EDX: 00000000
[ 205.358241] ESI: 7ff00000 EDI: 00000000 EBP: bf89aa48 ESP: bf89aa20
[ 205.358242] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 205.358243] Code: eb f2 8d b6 00 00 00 00 8d bc 27 00 00 00 00 55 89 c1 89 e5 56 53 9c 58 66 66 66 90 89 c6 fa 66 66 90 66 90 31 c0 bb 01 00 00 00 <f0> 0f b1 19 85 c0 75 06 89 f0 5b 5e 5d c3 89 c2 89 c8 e8 5d 1b
[ 205.358268] EIP: _raw_spin_lock_irqsave+0x1c/0x40 SS:ESP: 0068:f36b9a68
[ 205.358269] CR2: 00000000c10746c4
[ 205.358271] ---[ end trace ffb022c0c4b1f1dc ]---
----------------------------------------
----------------------------------------
localhost login: [ 67.347704] BUG: unable to handle kernel NULL pointer dereference at 00000180
[ 67.350154] IP: lock_page_memcg+0x3c/0x80
[ 67.351621] *pde = 00000000
[ 67.352731] Oops: 0000 [#1] SMP
[ 67.353916] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter i2c_piix4 coretemp vmw_balloon pcspkr shpchp sg vmw_vmci ip_tables xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi e1000 scsi_transport_spi ata_piix mptscsih libata mptbase
[ 67.373491] CPU: 2 PID: 1052 Comm: a.out Not tainted 4.15.0-rc6+ #258
[ 67.376019] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 67.380424] EIP: lock_page_memcg+0x3c/0x80
[ 67.382368] EFLAGS: 00010202 CPU: 2
[ 67.384037] EAX: f3537288 EBX: 00000010 ECX: 00000222 EDX: 00000000
[ 67.386589] ESI: 0000021e EDI: f3537288 EBP: f29438e4 ESP: f29438d8
[ 67.389089] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 67.391425] CR0: 80050033 CR2: 00000180 CR3: 32fcd000 CR4: 000406d0
[ 67.394103] Call Trace:
[ 67.395533] page_remove_rmap+0x92/0x280
[ 67.397470] try_to_unmap_one+0x202/0x530
[ 67.399499] rmap_walk_file+0xf0/0x1e0
[ 67.401413] rmap_walk+0x32/0x60
[ 67.403160] try_to_unmap+0x4d/0xd0
[ 67.404984] ? page_remove_rmap+0x280/0x280
[ 67.407122] ? page_not_mapped+0x10/0x10
[ 67.409115] ? page_get_anon_vma+0x80/0x80
[ 67.411170] shrink_page_list+0x3e2/0xe40
[ 67.413214] shrink_inactive_list+0x1b2/0x440
[ 67.415471] shrink_node_memcg+0x34a/0x770
[ 67.417562] shrink_node+0xbb/0x2e0
[ 67.419462] do_try_to_free_pages+0xb2/0x300
[ 67.421628] try_to_free_pages+0x20b/0x330
[ 67.423783] ? __queue_work+0xe6/0x270
[ 67.425803] __alloc_pages_slowpath+0x2fb/0x6d9
[ 67.428044] __alloc_pages_nodemask+0x17a/0x190
[ 67.430293] pagecache_get_page+0x53/0x200
[ 67.432417] ? common_exception+0x6f/0x76
[ 67.434476] grab_cache_page_write_begin+0x20/0x40
[ 67.436758] iomap_write_begin.constprop.17+0x6d/0xf0
[ 67.439118] ? iov_iter_fault_in_readable+0x7b/0xd0
[ 67.441397] iomap_write_actor+0xb4/0x1a0
[ 67.443392] ? iomap_write_begin.constprop.17+0xf0/0xf0
[ 67.445735] iomap_apply+0xd8/0x1b0
[ 67.447574] ? iomap_write_begin.constprop.17+0xf0/0xf0
[ 67.449710] iomap_file_buffered_write+0x83/0xc0
[ 67.451670] ? iomap_write_begin.constprop.17+0xf0/0xf0
[ 67.453780] xfs_file_buffered_aio_write+0xa3/0x200 [xfs]
[ 67.455983] xfs_file_write_iter+0x77/0x150 [xfs]
[ 67.457919] __vfs_write+0xef/0x150
[ 67.459511] vfs_write+0x96/0x190
[ 67.461042] SyS_write+0x44/0xa0
[ 67.462737] do_fast_syscall_32+0x87/0x179
[ 67.464471] entry_SYSENTER_32+0x4e/0x7c
[ 67.466141] EIP: 0xb7fd2da1
[ 67.467484] EFLAGS: 00000246 CPU: 2
[ 67.469020] EAX: ffffffda EBX: 00000003 ECX: 0804a060 EDX: 00001000
[ 67.471392] ESI: 00000009 EDI: 00000000 EBP: bfc76ff8 ESP: bfc76fb8
[ 67.473702] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 67.475803] Code: db 75 28 eb 5c 66 90 8d b3 74 01 00 00 89 f0 e8 3b 2a 4e 00 8b 4f 20 39 d9 74 2c 89 c2 89 f0 e8 db 26 4e 00 8b 5f 20 85 db 74 36 <8b> 83 70 01 00 00 85 c0 7f d2 89 d9 5b 89 c8 5e 5f 5d c3 90 31
[ 67.483059] EIP: lock_page_memcg+0x3c/0x80 SS:ESP: 0068:f29438d8
[ 67.485404] CR2: 0000000000000180
[ 67.487139] ---[ end trace 8260c8de0ee96afd ]---
----------------------------------------
----------------------------------------
[ 47.171548] BUG: unable to handle kernel paging request at 32eac034
[ 47.174531] IP: page_remove_rmap+0x19a/0x2e0
[ 47.176259] *pde = 00000000
[ 47.177373] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
[ 47.179335] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp vmw_balloon pcspkr sg vmw_vmci i2c_piix4 shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
[ 47.202163] CPU: 6 PID: 2947 Comm: a.out Not tainted 4.15.0-rc6+ #259
[ 47.204828] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 47.210263] EIP: page_remove_rmap+0x19a/0x2e0
[ 47.212606] EFLAGS: 00010286 CPU: 6
[ 47.214687] EAX: 00000000 EBX: f342cbe0 ECX: ffffffe2 EDX: 0000000d
[ 47.217816] ESI: ffffffff EDI: f4a29240 EBP: cc4e1a6c ESP: cc4e1a64
[ 47.220462] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 47.222837] CR0: 80050033 CR2: 32eac034 CR3: 2bb4a000 CR4: 000406d0
[ 47.225461] Call Trace:
[ 47.226954] try_to_unmap_one+0x20e/0x590
[ 47.228920] rmap_walk_file+0x13c/0x250
[ 47.230802] rmap_walk+0x32/0x60
[ 47.232945] try_to_unmap+0x4d/0x100
[ 47.235331] ? page_remove_rmap+0x2e0/0x2e0
[ 47.237999] ? page_not_mapped+0x10/0x10
[ 47.240660] ? page_get_anon_vma+0x80/0x80
[ 47.243251] shrink_page_list+0x3a2/0x1010
[ 47.245916] shrink_inactive_list+0x1b2/0x440
[ 47.248042] shrink_node_memcg+0x34a/0x770
[ 47.250084] ? __queue_work+0xf2/0x290
[ 47.252020] shrink_node+0xbb/0x2e0
[ 47.253940] do_try_to_free_pages+0xba/0x320
[ 47.256063] try_to_free_pages+0x11d/0x340
[ 47.258152] ? __switch_to+0xa2/0x220
[ 47.260102] __alloc_pages_slowpath+0x303/0x6e2
[ 47.262314] ? page_cache_tree_insert+0xb0/0xb0
[ 47.264506] __alloc_pages_nodemask+0x1a7/0x1c0
[ 47.266713] wp_page_copy+0x5f/0x650
[ 47.268627] ? reuse_swap_page+0x64/0x200
[ 47.270586] do_wp_page+0x82/0x450
[ 47.272541] ? filemap_fdatawait_keep_errors+0x50/0x50
[ 47.274707] handle_mm_fault+0x522/0x8d0
[ 47.276547] __do_page_fault+0x1ef/0x4e0
[ 47.278328] ? __do_page_fault+0x4e0/0x4e0
[ 47.280160] do_page_fault+0x1a/0x20
[ 47.281798] common_exception+0x6f/0x76
[ 47.283490] EIP: 0xb7f86025
[ 47.284869] EFLAGS: 00010246 CPU: 6
[ 47.286405] EAX: b7e91930 EBX: b7f99fbc ECX: 0804a00c EDX: 00000000
[ 47.289057] ESI: b7f983d0 EDI: b7db39cc EBP: bfe325b0 ESP: bfe32568
[ 47.291615] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 47.293891] Code: ff ff ff 89 f1 ba 0d 00 00 00 b8 20 f8 8e c1 e8 3d 37 fe ff 66 66 66 66 90 8b 43 20 85 c0 74 1f 66 66 66 66 90 8b 80 80 01 00 00 <64> 01 70 34 8b 43 20 8b 80 fc 01 00 00 8b 40 44 64 01 70 34 8b
[ 47.301428] EIP: page_remove_rmap+0x19a/0x2e0 SS:ESP: 0068:cc4e1a64
[ 47.303992] CR2: 0000000032eac034
[ 47.305695] ---[ end trace 1887d451e7998639 ]---
[ 47.310724] BUG: unable to handle kernel paging request at 5a5a5a04
[ 47.313277] IP: blk_flush_plug_list+0x77/0x210
[ 47.315196] *pde = 00000000
[ 47.316683] Oops: 0002 [#2] SMP DEBUG_PAGEALLOC
[ 47.318630] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp vmw_balloon pcspkr sg vmw_vmci i2c_piix4 shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
[ 47.339036] CPU: 0 PID: 2947 Comm: a.out Tainted: G D 4.15.0-rc6+ #259
[ 47.342045] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 47.346781] EIP: blk_flush_plug_list+0x77/0x210
[ 47.348958] EFLAGS: 00010297 CPU: 0
[ 47.350841] EAX: cc4e1e40 EBX: cc4e1e40 ECX: eeed0ad7 EDX: 5a5a5a00
[ 47.353501] ESI: 00000001 EDI: cc4e1cbc EBP: cc4e1e58 ESP: cc4e1e2c
[ 47.356178] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 47.358636] CR0: 80050033 CR2: 5a5a5a04 CR3: 019d0000 CR4: 000406d0
[ 47.361376] Call Trace:
[ 47.363000] schedule+0x52/0x70
[ 47.364817] rwsem_down_write_failed+0x185/0x290
[ 47.367133] call_rwsem_down_write_failed+0x9/0xc
[ 47.369496] down_write+0x20/0x30
[ 47.371376] unlink_file_vma+0x25/0x40
[ 47.373370] free_pgtables+0x33/0xf0
[ 47.375323] ? unmap_vmas+0x3b/0x50
[ 47.377312] exit_mmap+0x92/0x140
[ 47.379162] mmput+0x4f/0xf0
[ 47.380827] do_exit+0x22c/0xa10
[ 47.382738] ? __do_page_fault+0x4e0/0x4e0
[ 47.384775] rewind_stack_do_exit+0x11/0x13
[ 47.386850] EIP: 0xb7f86025
[ 47.388512] EFLAGS: 00010246 CPU: 0
[ 47.390300] EAX: b7e91930 EBX: b7f99fbc ECX: 0804a00c EDX: 00000000
[ 47.392883] ESI: b7f983d0 EDI: b7db39cc EBP: bfe325b0 ESP: bfe32568
[ 47.395478] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 47.397799] Code: 45 e8 8b 57 14 89 4d e8 89 59 04 89 02 89 50 04 8b 45 dc 89 47 10 89 47 14 8d 76 00 8b 45 e8 39 c3 74 c9 8b 4d e8 8b 11 8b 41 04 <89> 42 04 89 10 89 f2 89 c8 c7 01 00 01 00 00 c7 41 04 00 02 00
[ 47.405344] EIP: blk_flush_plug_list+0x77/0x210 SS:ESP: 0068:cc4e1e2c
[ 47.407978] CR2: 000000005a5a5a04
[ 47.409744] ---[ end trace 1887d451e799863a ]---
----------------------------------------
----------------------------------------
[ 121.578226] BUG: unable to handle kernel paging request at 330fa01a
[ 121.581057] IP: page_remove_rmap+0x14/0x2e0
[ 121.583085] *pde = 00000000
[ 121.584784] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 121.586923] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp vmw_balloon pcspkr vmw_vmci sg i2c_piix4 shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 ata_piix mptbase libata
[ 121.607308] CPU: 0 PID: 402 Comm: systemd-journal Not tainted 4.15.0-rc6+ #259
[ 121.610170] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 121.615142] EIP: page_remove_rmap+0x14/0x2e0
[ 121.617238] EFLAGS: 00010202 CPU: 0
[ 121.619085] EAX: 330fa016 EBX: f3246808 ECX: 00001534 EDX: 00000000
[ 121.621615] ESI: 00001533 EDI: f4af1d30 EBP: f2bb7a34 ESP: f2bb7a2c
[ 121.624079] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 121.626348] CR0: 80050033 CR2: 330fa01a CR3: 32b46000 CR4: 000406d0
[ 121.629207] Call Trace:
[ 121.630896] try_to_unmap_one+0x20e/0x590
[ 121.632921] rmap_walk_file+0x13c/0x250
[ 121.634703] rmap_walk+0x32/0x60
[ 121.636312] try_to_unmap+0x4d/0x100
[ 121.637987] ? page_remove_rmap+0x2e0/0x2e0
[ 121.639813] ? page_not_mapped+0x10/0x10
[ 121.641569] ? page_get_anon_vma+0x80/0x80
[ 121.643473] shrink_page_list+0x3a2/0x1010
[ 121.645371] ? wake_up_worker+0x1c/0x20
[ 121.647311] shrink_inactive_list+0x1b2/0x440
[ 121.649337] shrink_node_memcg+0x34a/0x770
[ 121.651141] shrink_node+0xbb/0x2e0
[ 121.652797] do_try_to_free_pages+0xba/0x320
[ 121.654649] try_to_free_pages+0x11d/0x340
[ 121.656427] ? schedule_timeout+0x136/0x200
[ 121.658204] __alloc_pages_slowpath+0x303/0x6e2
[ 121.660145] __alloc_pages_nodemask+0x1a7/0x1c0
[ 121.662107] filemap_fault+0x3ca/0x610
[ 121.663953] ? radix_tree_next_chunk+0xf1/0x2c0
[ 121.665860] ? _cond_resched+0x12/0x30
[ 121.667507] ? down_read+0xb/0x30
[ 121.669004] ? filemap_fdatawait_keep_errors+0x50/0x50
[ 121.670954] __xfs_filemap_fault.isra.16+0x2d/0xb0 [xfs]
[ 121.672910] ? filemap_fdatawait_keep_errors+0x50/0x50
[ 121.674819] xfs_filemap_fault+0xa/0x10 [xfs]
[ 121.676503] __do_fault+0x11/0x60
[ 121.678115] handle_mm_fault+0x6dd/0x8d0
[ 121.679850] ? ep_poll+0x1b2/0x340
[ 121.681347] __do_page_fault+0x1ef/0x4e0
[ 121.682889] ? __do_page_fault+0x4e0/0x4e0
[ 121.684422] do_page_fault+0x1a/0x20
[ 121.685827] common_exception+0x6f/0x76
[ 121.687285] EIP: 0xb7d89ad8
[ 121.688492] EFLAGS: 00010246 CPU: 0
[ 121.689858] EAX: 00000001 EBX: 00000007 ECX: bfd3e800 EDX: 00000012
[ 121.691915] ESI: ffffffff EDI: 00000012 EBP: bfd3e9b8 ESP: bfd3e7e4
[ 121.694042] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 121.696023] Code: c4 fe ff ff ba 78 55 7f c1 89 d8 e8 47 e5 fe ff 0f 0b 83 e8 01 eb 8e 55 89 e5 56 53 89 c3 8b 40 14 a8 01 0f 85 be 01 00 00 89 d8 <f6> 40 04 01 74 5e 84 d2 0f 85 9e 00 00 00 f0 83 43 0c ff 78 07
[ 121.702493] EIP: page_remove_rmap+0x14/0x2e0 SS:ESP: 0068:f2bb7a2c
[ 121.704587] CR2: 00000000330fa01a
[ 121.706032] ---[ end trace 6878785dacdbec93 ]---
----------------------------------------
----------------------------------------
localhost login: [ 87.199039] BUG: unable to handle kernel paging request at 3304501a
[ 87.200976] IP: page_remove_rmap+0x14/0x2e0
[ 87.202311] *pde = 00000000
[ 87.203268] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 87.204740] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp pcspkr vmw_balloon sg vmw_vmci shpchp i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi mptspi e1000 ata_piix scsi_transport_spi mptscsih mptbase libata serio_raw
[ 87.221708] CPU: 0 PID: 32 Comm: kswapd0 Not tainted 4.15.0-rc6+ #259
[ 87.223963] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 87.227825] EIP: page_remove_rmap+0x14/0x2e0
[ 87.229475] EFLAGS: 00210202 CPU: 0
[ 87.230981] EAX: 33045016 EBX: f30afe40 ECX: 00000eca EDX: 00000000
[ 87.233204] ESI: 00000e7e EDI: f494dd08 EBP: f35ebc4c ESP: f35ebc44
[ 87.235533] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 87.237674] CR0: 80050033 CR2: 3304501a CR3: 019d0000 CR4: 000406d0
[ 87.240187] Call Trace:
[ 87.241491] try_to_unmap_one+0x20e/0x590
[ 87.243246] rmap_walk_file+0x13c/0x250
[ 87.244921] rmap_walk+0x32/0x60
[ 87.246500] try_to_unmap+0x4d/0x100
[ 87.248134] ? page_remove_rmap+0x2e0/0x2e0
[ 87.249932] ? page_not_mapped+0x10/0x10
[ 87.251897] ? page_get_anon_vma+0x80/0x80
[ 87.253806] shrink_page_list+0x3a2/0x1010
[ 87.255700] shrink_inactive_list+0x1b2/0x440
[ 87.257671] shrink_node_memcg+0x34a/0x770
[ 87.259527] shrink_node+0xbb/0x2e0
[ 87.261369] kswapd+0x23f/0x5c0
[ 87.263047] kthread+0xd1/0x100
[ 87.264812] ? mem_cgroup_shrink_node+0xa0/0xa0
[ 87.267011] ? kthread_associate_blkcg+0x80/0x80
[ 87.269229] ret_from_fork+0x19/0x24
[ 87.271061] Code: c4 fe ff ff ba 78 55 7f c1 89 d8 e8 47 e5 fe ff 0f 0b 83 e8 01 eb 8e 55 89 e5 56 53 89 c3 8b 40 14 a8 01 0f 85 be 01 00 00 89 d8 <f6> 40 04 01 74 5e 84 d2 0f 85 9e 00 00 00 3e 83 43 0c ff 78 07
[ 87.278478] EIP: page_remove_rmap+0x14/0x2e0 SS:ESP: 0068:f35ebc44
[ 87.280872] CR2: 000000003304501a
[ 87.282602] ---[ end trace dd23f463fd05949e ]---
----------------------------------------
----------------------------------------
localhost login: [ 44.046356] a.out invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0
[ 44.050525] CPU: 7 PID: 2807 Comm: a.out Not tainted 4.15.0-rc6+ #260
[ 44.052755] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 44.057764] Call Trace:
[ 44.059257] dump_stack+0x58/0x76
[ 44.061118] dump_header+0x66/0x261
[ 44.063095] ? ___ratelimit+0x83/0xf0
[ 44.065194] oom_kill_process+0x1e5/0x3e0
[ 44.067397] ? has_capability_noaudit+0x1f/0x30
[ 44.069842] ? oom_badness+0xd0/0x130
[ 44.071974] ? oom_evaluate_task+0xa2/0xe0
[ 44.074257] out_of_memory+0xf2/0x290
[ 44.076144] __alloc_pages_slowpath+0x5c1/0x6e2
[ 44.078336] __alloc_pages_nodemask+0x1a7/0x1c0
[ 44.080369] do_anonymous_page+0xab/0x4f0
[ 44.082067] handle_mm_fault+0x531/0x8d0
[ 44.083738] ? __phys_addr+0x32/0x70
[ 44.085469] ? load_new_mm_cr3+0x6a/0x90
[ 44.087200] __do_page_fault+0x1ef/0x4e0
[ 44.089274] ? __do_page_fault+0x4e0/0x4e0
[ 44.091028] do_page_fault+0x1a/0x20
[ 44.092647] common_exception+0x6f/0x76
[ 44.094341] EIP: 0x80484be
[ 44.096293] EFLAGS: 00010202 CPU: 7
[ 44.098275] EAX: 00a2d000 EBX: 80000000 ECX: 38902008 EDX: 00000000
[ 44.101035] ESI: 7ff00000 EDI: 00000000 EBP: bfa039f8 ESP: bfa039d0
[ 44.103192] page:5a5a0697 count:-1055023618 mapcount:-1055030029 mapping:26f4be11 index:0xc11d7c83
[ 44.103196] flags: 0xc10528fe(waiters|error|referenced|uptodate|dirty|lru|active|reserved|private_2|mappedtodisk|swapbacked)
[ 44.103200] raw: c10528fe c114fff7 c11d7c83 c11d84f2 c11d9dfe c11daa34 c11daaa0 c13e65df
[ 44.103201] raw: c13e4a1c c13e4c62
[ 44.103202] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) <= 0)
[ 44.103203] page->mem_cgroup:35401b27
[ 44.103208] ------------[ cut here ]------------
[ 44.103209] kernel BUG at ./include/linux/mm.h:844!
[ 44.103214] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 44.103214] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg i2c_piix4 pcspkr vmw_balloon vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
[ 44.103246] CPU: 4 PID: 1954 Comm: a.out Not tainted 4.15.0-rc6+ #260
[ 44.103247] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 44.103253] EIP: isolate_lru_page+0x234/0x260
[ 44.103254] EFLAGS: 00010086 CPU: 4
[ 44.103255] EAX: 00000019 EBX: c18efb40 ECX: c18c4b5c EDX: 00000002
[ 44.103256] ESI: f3179560 EDI: e910244c EBP: ee30fa80 ESP: ee30fa64
[ 44.103257] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 44.103258] CR0: 80050033 CR2: 3bf77008 CR3: 2e72c000 CR4: 000406d0
[ 44.103332] Call Trace:
[ 44.103338] clear_page_mlock+0x71/0xc0
[ 44.103340] page_remove_rmap+0x2a9/0x2e0
[ 44.103341] try_to_unmap_one+0x20e/0x590
[ 44.103344] rmap_walk_file+0x13c/0x250
[ 44.103346] rmap_walk+0x32/0x60
[ 44.103347] try_to_unmap+0x4d/0x100
[ 44.103349] ? page_remove_rmap+0x2e0/0x2e0
[ 44.103351] ? page_not_mapped+0x10/0x10
[ 44.103352] ? page_get_anon_vma+0x80/0x80
[ 44.103354] shrink_page_list+0x3a2/0x1010
[ 44.103357] shrink_inactive_list+0x1b2/0x440
[ 44.103360] shrink_node_memcg+0x34a/0x770
[ 44.103363] shrink_node+0xbb/0x2e0
[ 44.103365] do_try_to_free_pages+0xba/0x320
[ 44.103367] try_to_free_pages+0x11d/0x340
[ 44.103371] __alloc_pages_slowpath+0x303/0x6e2
[ 44.103373] ? release_pages+0x239/0x330
[ 44.103376] __alloc_pages_nodemask+0x1a7/0x1c0
[ 44.103378] do_anonymous_page+0xab/0x4f0
[ 44.103380] handle_mm_fault+0x531/0x8d0
[ 44.103383] ? __schedule+0x173/0x5b0
[ 44.103387] __do_page_fault+0x1ef/0x4e0
[ 44.103389] ? __do_page_fault+0x4e0/0x4e0
[ 44.103390] do_page_fault+0x1a/0x20
[ 44.103393] common_exception+0x6f/0x76
[ 44.103394] EIP: 0x80484be
[ 44.103395] EFLAGS: 00010202 CPU: 4
[ 44.103396] EAX: 00d66000 EBX: 80000000 ECX: 38b97008 EDX: 00000000
[ 44.103397] ESI: 7ff00000 EDI: 00000000 EBP: bfcd2948 ESP: bfcd2920
[ 44.103398] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 44.103399] Code: fe ff ff 83 e8 01 e9 6b fe ff ff ba 80 44 7f c1 89 f0 e8 90 66 01 00 0f 0b 83 e8 01 e9 ac fe ff ff ba 3c a5 7d c1 e8 7c 66 01 00 <0f> 0b 83 e8 01 e9 b4 fe ff ff 89 f0 e8 4c 3a 00 00 83 e8 01 e9
[ 44.103425] EIP: isolate_lru_page+0x234/0x260 SS:ESP: 0068:ee30fa64
[ 44.103427] ---[ end trace e79ca6d793be8d54 ]---
[ 44.300720] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 44.303579] Mem-Info:
[ 44.305289] active_anon:690488 inactive_anon:2081 isolated_anon:0
[ 44.305289] active_file:508 inactive_file:529 isolated_file:27
[ 44.305289] unevictable:0 dirty:0 writeback:662 unstable:0
[ 44.305289] slab_reclaimable:1410 slab_unreclaimable:20537
[ 44.305289] mapped:459 shmem:2151 pagetables:11526 bounce:0
[ 44.305289] free:30998 free_pcp:59 free_cma:0
[ 44.322423] Node 0 active_anon:2761952kB inactive_anon:8324kB active_file:2032kB inactive_file:2116kB unevictable:0kB isolated(anon):0kB isolated(file):108kB mapped:1836kB dirty:0kB writeback:2648kB shmem:8604kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 323584kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[ 44.336597] DMA free:12708kB min:788kB low:984kB high:1180kB active_anon:3192kB inactive_anon:0kB active_file:0kB inactive_file:4kB unevictable:0kB writepending:0kB present:15992kB managed:15916kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 44.349286] lowmem_reserve[]: 0 812 2996 2996
[ 44.352127] Normal free:110912kB min:41308kB low:51632kB high:61956kB active_anon:547592kB inactive_anon:0kB active_file:300kB inactive_file:236kB unevictable:0kB writepending:352kB present:892920kB managed:831668kB mlocked:0kB kernel_stack:23088kB pagetables:46104kB bounce:0kB free_pcp:108kB local_pcp:0kB free_cma:0kB
[ 44.367461] lowmem_reserve[]: 0 0 17471 17471
[ 44.370437] HighMem free:372kB min:512kB low:28280kB high:56048kB active_anon:2210712kB inactive_anon:8324kB active_file:1312kB inactive_file:1724kB unevictable:0kB writepending:1176kB present:2236360kB managed:2236360kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:128kB local_pcp:0kB free_cma:0kB
[ 44.386249] lowmem_reserve[]: 0 0 0 0
[ 44.389137] DMA: 13*4kB (UM) 2*8kB (UM) 2*16kB (UM) 0*32kB 3*64kB (UM) 1*128kB (U) 2*256kB (UM) 1*512kB (M) 1*1024kB (U) 1*2048kB (U) 2*4096kB (M) = 12708kB
[ 44.397316] Normal: 377*4kB (UM) 382*8kB (UME) 274*16kB (UE) 95*32kB (UME) 39*64kB (UME) 14*128kB (UME) 12*256kB (UME) 3*512kB (ME) 0*1024kB 2*2048kB (ME) 21*4096kB (M) = 110996kB
[ 44.406379] HighMem: 0*4kB 1*8kB (M) 3*16kB (M) 2*32kB (M) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 120kB
[ 44.413805] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=4096kB
[ 44.421438] 3325 total pagecache pages
[ 44.424507] 0 pages in swap cache
[ 44.427416] Swap cache stats: add 0, delete 0, find 0/0
[ 44.431158] Free swap = 0kB
[ 44.433919] Total swap = 0kB
[ 44.436623] 786318 pages RAM
[ 44.439339] 559090 pages HighMem/MovableOnly
[ 44.442496] 15332 pages reserved
[ 44.445283] 0 pages cma reserved
[ 44.448070] Out of memory: Kill process 962 (a.out) score 32 or sacrifice child
[ 44.452335] Killed process 1167 (a.out) total-vm:2108kB, anon-rss:60kB, file-rss:0kB, shmem-rss:0kB
----------------------------------------
So far I can reproduce this problem when running a guest on an i7-2630QM (8 core) host
but not on an i5-4440S (4 core) host. Thus, maybe this is an architecture specific problem.
----------------------------------------
1c1
< Linux version 4.15.0-rc6+ (root@localhost.localdomain) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)) #258 SMP Thu Jan 4 16:19:44 JST 2018
---
> Linux version 4.15.0-rc6+ (root@localhost.localdomain) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)) #195 SMP Fri Jan 5 21:10:49 JST 2018
26c26
< vmware: TSC freq read from hypervisor : 1995.458 MHz
---
> vmware: TSC freq read from hypervisor : 2793.542 MHz
28c28
< vmware: using sched offset of 5372044450 ns
---
> vmware: using sched offset of 57103384027 ns
36c36
< RAMDISK: [mem 0x35fb5000-0x36fd2fff]
---
> RAMDISK: [mem 0x35fda000-0x36fe4fff]
39c39
< ACPI: XSDT 0x00000000BFEF00CB 00005C (v01 INTEL 440BX 06040000 VMW 01324272)
---
> ACPI: XSDT 0x00000000BFEF02EF 00005C (v01 INTEL 440BX 06040000 VMW 01324272)
41c41
< ACPI: DSDT 0x00000000BFEF0405 00EA6E (v01 PTLTD Custom 06040000 MSFT 03000001)
---
> ACPI: DSDT 0x00000000BFEF05B1 00E8C2 (v01 PTLTD Custom 06040000 MSFT 03000001)
44,49c44,49
< ACPI: BOOT 0x00000000BFEF03DD 000028 (v01 PTLTD $SBFTBL$ 06040000 LTP 00000001)
< ACPI: APIC 0x00000000BFEF032B 0000B2 (v01 PTLTD ? APIC 06040000 LTP 00000000)
< ACPI: MCFG 0x00000000BFEF02EF 00003C (v01 PTLTD $PCITBL$ 06040000 LTP 00000001)
< ACPI: SRAT 0x00000000BFEF01C7 000128 (v02 VMWARE MEMPLUG 06040000 VMW 00000001)
< ACPI: HPET 0x00000000BFEF018F 000038 (v01 VMWARE VMW HPET 06040000 VMW 00000001)
< ACPI: WAET 0x00000000BFEF0167 000028 (v01 VMWARE VMW WAET 06040000 VMW 00000001)
---
> ACPI: BOOT 0x00000000BFEF0589 000028 (v01 PTLTD $SBFTBL$ 06040000 LTP 00000001)
> ACPI: APIC 0x00000000BFEF050F 00007A (v01 PTLTD ? APIC 06040000 LTP 00000000)
> ACPI: MCFG 0x00000000BFEF04D3 00003C (v01 PTLTD $PCITBL$ 06040000 LTP 00000001)
> ACPI: SRAT 0x00000000BFEF03EB 0000E8 (v02 VMWARE MEMPLUG 06040000 VMW 00000001)
> ACPI: HPET 0x00000000BFEF03B3 000038 (v01 VMWARE VMW HPET 06040000 VMW 00000001)
> ACPI: WAET 0x00000000BFEF038B 000028 (v01 VMWARE VMW WAET 06040000 VMW 00000001)
54d53
< crashkernel: memory value expected
72,75d70
< ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
< ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
< ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])
< ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])
79c74
< smpboot: Allowing 8 CPUs, 0 hotplug CPUs
---
> smpboot: Allowing 4 CPUs, 0 hotplug CPUs
87c82
< setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
---
> setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:4 nr_node_ids:1
90c85
< Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-rc6+ root=UUID=dcfdc514-ddcb-4f0c-bddf-e229701a66e5 ro crashkernel=auto sysrq_always_enabled console=ttyS0,115200n8 console=tty0 LANG=en_US.UTF-8
---
> Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-rc6+ root=UUID=c6605616-f00a-4630-97c1-847f8e0c850f ro security=none sysrq_always_enabled console=ttyS0,115200n8 console=tty0
97c92
< Memory: 3085040K/3145272K available (6360K kernel code, 391K rwdata, 2408K rodata, 640K init, 1280K bss, 60232K reserved, 0K cma-reserved, 2236360K highmem)
---
> Memory: 3085528K/3145272K available (6385K kernel code, 388K rwdata, 2416K rodata, 644K init, 1280K bss, 59744K reserved, 0K cma-reserved, 2236360K highmem)
104,106c99,101
< .init : 0xc1907000 - 0xc19a7000 ( 640 kB)
< .data : 0xc1636011 - 0xc18f6f60 (2819 kB)
< .text : 0xc1000000 - 0xc1636011 (6360 kB)
---
> .init : 0xc190f000 - 0xc19b0000 ( 644 kB)
> .data : 0xc163c611 - 0xc18ff060 (2826 kB)
> .text : 0xc1000000 - 0xc163c611 (6385 kB)
108c103
< SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
---
> SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
110c105,107
< NR_IRQS: 2304, nr_irqs: 488, preallocated irqs: 16
---
> RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
> RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
> NR_IRQS: 2304, nr_irqs: 456, preallocated irqs: 16
119,120c116,117
< tsc: Detected 1995.458 MHz processor
< Calibrating delay loop (skipped) preset value.. 3990.91 BogoMIPS (lpj=1995458)
---
> tsc: Detected 2793.542 MHz processor
> Calibrating delay loop (skipped) preset value.. 5587.08 BogoMIPS (lpj=2793542)
129,130c126,127
< Last level iTLB entries: 4KB 512, 2MB 8, 4MB 8
< Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32, 1GB 0
---
> Last level iTLB entries: 4KB 1024, 2MB 1024, 4MB 1024
> Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 1024, 1GB 4
132,133c129,130
< smpboot: CPU0: Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz (family: 0x6, model: 0x2a, stepping: 0x7)
< Performance Events: SandyBridge events, core PMU driver.
---
> smpboot: CPU0: Intel(R) Core(TM) i5-4440S CPU @ 2.80GHz (family: 0x6, model: 0x3c, stepping: 0x3)
> Performance Events: Haswell events, core PMU driver.
143c140
< ... generic registers: 4
---
> ... generic registers: 8
147c144
< ... event mask: 000000000000000f
---
> ... event mask: 00000000000000ff
166,188c163,165
< #4
< Initializing CPU#4
< Disabled fast string operations
< mce: CPU supports 0 MCE banks
< smpboot: CPU 4 Converting physical 8 to logical package 4
< #5
< Initializing CPU#5
< Disabled fast string operations
< mce: CPU supports 0 MCE banks
< smpboot: CPU 5 Converting physical 10 to logical package 5
< #6
< Initializing CPU#6
< Disabled fast string operations
< mce: CPU supports 0 MCE banks
< smpboot: CPU 6 Converting physical 12 to logical package 6
< #7
< Initializing CPU#7
< Disabled fast string operations
< mce: CPU supports 0 MCE banks
< smpboot: CPU 7 Converting physical 14 to logical package 7
< smp: Brought up 1 node, 8 CPUs
< smpboot: Max logical packages: 8
< smpboot: Total of 8 processors activated (31927.32 BogoMIPS)
---
> smp: Brought up 1 node, 4 CPUs
> smpboot: Max logical packages: 4
> smpboot: Total of 4 processors activated (22348.33 BogoMIPS)
192c169
< futex hash table entries: 2048 (order: 4, 65536 bytes)
---
> futex hash table entries: 1024 (order: 3, 32768 bytes)
----------------------------------------
Reproducer is simple. Put the system under memory pressure
while writing to a file on XFS filesystem.
----------------------------------------
#define _FILE_OFFSET_BITS 64
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <fcntl.h>
static void run(void)
{
unsigned long long size;
char *buf = NULL;
unsigned long long i;
for (i = 0; i < 24; i++) {
if (fork() == 0) {
static char buf[4096];
int fd;
snprintf(buf, sizeof(buf), "file.%u", getpid());
fd = open(buf, O_CREAT | O_WRONLY | O_APPEND, 0600);
while (write(fd, buf, sizeof(buf)) == sizeof(buf));
close(fd);
_exit(0);
}
}
for (size = 1048576; size < 512ULL * (1 << 30); size += 1048576) {
char *cp = realloc(buf, size);
if (!cp) {
size -= 1048576;
break;
}
buf = cp;
}
for (i = 0; i < size; i += 4096)
buf[i] = 0;
_exit(0);
}
int main(int argc, char *argv[])
{
if (argc != 1)
run();
else
while (1)
if (fork() == 0)
execlp(argv[0], argv[0], "", NULL);
return 0;
}
----------------------------------------
Do you have any clue?
(Since I need to use i7-2630QM for testing, I won't be able to test
until next Tuesday, sorry.)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm? 4.15-rc7] Random oopses by simple write under memory pressure.
2018-01-05 14:45 [x86? mm? fs? 4.15-rc6] Random oopses by simple write under memory pressure Tetsuo Handa
@ 2018-01-09 10:39 ` Tetsuo Handa
2018-01-10 11:49 ` [mm? 4.15-rc7] Random oopses " Tetsuo Handa
0 siblings, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-09 10:39 UTC (permalink / raw)
To: linux-mm
I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
Does anyone know what is happening?
As of v4.8:
** 1516 printk messages dropped ** [ 131.524698] BUG: unable to handle kernel paging request at 0158de94
** 2263 printk messages dropped ** [ 131.529186] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp ppdev vmw_balloon pcspkr shpchp sg vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
** 11660 printk messages dropped ** [ 131.563480] Call Trace:
** 1274 printk messages dropped ** [ 131.567313] [<c55de022>] schedule+0x52/0x70
** 1554 printk messages dropped ** [ 131.572008] ------------[ cut here ]------------
** 1669 printk messages dropped ** [ 131.577027] [<c55de022>] schedule+0x52/0x70
** 1185 printk messages dropped ** [ 131.580606] IP: [<c523a834>] blk_flush_plug_list+0x54/0x210
** 1256 printk messages dropped ** [ 131.584396] ESI: 00000001 EDI: e658de3c EBP: e658df58 ESP: e658df2c
** 1719 printk messages dropped ** [ 131.589577] ESI: 00000001 EDI: e658de3c EBP: e658df58 ESP: e658df2c
** 1898 printk messages dropped ** [ 131.595393] Call Trace:
** 952 printk messages dropped ** [ 131.598851] ---[ end trace 27af91326c31342e ]---
** 1522 printk messages dropped ** [ 131.603609] BUG: unable to handle kernel paging request at 0158de94
** 2167 printk messages dropped ** [ 131.610234] Call Trace:
** 1184 printk messages dropped ** [ 131.613877] Fixing recursive fault but reboot is needed!
** 1524 printk messages dropped ** [ 131.618480] [<c55de022>] schedule+0x52/0x70
** 1379 printk messages dropped ** [ 131.622669] EIP: [<c523a834>] blk_flush_plug_list+0x54/0x210 SS:ESP 0068:e658df2c
** 2273 printk messages dropped ** [ 131.629554] Call Trace:
** 1143 printk messages dropped ** [ 131.633005] IP: [<c523a834>] blk_flush_plug_list+0x54/0x210
** 1537 printk messages dropped ** [ 131.637673] Call Trace:
** 1170 printk messages dropped ** [ 131.641177] IP: [<c523a834>] blk_flush_plug_list+0x54/0x210
** 1476 printk messages dropped ** [ 131.645634] 00000009 e658e000 eb265a40 e658df64 c55de022 00000009 e658dfac c50545b0
** 2235 printk messages dropped ** [ 131.652386] 00000009 e658e000 eb265a40 e658df64 c55de022 00000009 e658dfac c50545b0
** 2282 printk messages dropped ** [ 131.659252] [<c50545b0>] do_exit+0x9d0/0xa30
** 1341 printk messages dropped ** [ 131.663305] CPU: 7 PID: 7783 Comm: a.out Tainted: G D W 4.8.0 #1
** 2146 printk messages dropped ** [ 131.669755] ------------[ cut here ]------------
** 1362 printk messages dropped ** [ 131.673851] CR0: 80050033 CR2: 0158de94 CR3: 33233000 CR4: 000406d0
** 2052 printk messages dropped ** [ 131.680023] *pde = 00000000
** 1165 printk messages dropped ** [ 131.683703] task: eb265a40 task.stack: e658c000
** 1326 printk messages dropped ** [ 131.687678] IP: [<c523a834>] blk_flush_plug_list+0x54/0x210
** 1436 printk messages dropped ** [ 131.692003] [<c55de022>] schedule+0x52/0x70
** 1287 printk messages dropped ** [ 131.695871] c576539c 00000000 00000000 eb265a40 eb265dfc 00000001 eb265c98 eb265a40
** 2133 printk messages dropped ** [ 131.702282] IP: [<c523a834>] blk_flush_plug_list+0x54/0x210
** 1435 printk messages dropped ** [ 131.706608] Call Trace:
** 1118 printk messages dropped ** [ 131.709977] [<c55de022>] schedule+0x52/0x70
** 1295 printk messages dropped ** [ 131.713889] ---[ end trace 27af91326c313b16 ]---
** 1332 printk messages dropped ** [ 131.717899] ---[ end trace 27af91326c313b54 ]---
** 1321 printk messages dropped ** [ 131.721862] Stack:
** 1029 printk messages dropped ** [ 131.724942] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
** 1381 printk messages dropped ** [ 131.729095] Call Trace:
** 1057 printk messages dropped ** [ 131.732279] IP: [<c523a834>] blk_flush_plug_list+0x54/0x210
** 1392 printk messages dropped ** [ 131.736474] Call Trace:
** 1074 printk messages dropped ** [ 131.739687] Call Trace:
** 1036 printk messages dropped ** [ 131.742845] EIP: [<c523a834>] blk_flush_plug_list+0x54/0x210 SS:ESP 0068:e658df2c
** 2017 printk messages dropped ** [ 131.748890] [<c50545b0>] do_exit+0x9d0/0xa30
** 1273 printk messages dropped ** [ 131.752731] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp ppdev vmw_balloon pcspkr shpchp sg vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
** 10488 printk messages dropped ** [ 131.784217] IP: [<c523a834>] blk_flush_plug_list+0x54/0x210
** 1465 printk messages dropped ** [ 131.788624] CPU: 7 PID: 7783 Comm: a.out Tainted: G D W 4.8.0 #1
** 2119 printk messages dropped ** [ 131.794935] Call Trace:
** 1155 printk messages dropped ** [ 131.798375] CR0: 80050033 CR2: 0158de94 CR3: 33233000 CR4: 000406d0
** 2051 printk messages dropped ** [ 131.804506] IP: [<c523a834>] blk_flush_plug_list+0x54/0x210
** 1457 printk messages dropped ** [ 131.808872] [<c55e2291>] rewind_stack_do_exit+0x11/0x13
** 1459 printk messages dropped ** [ 131.813214] [<c5050d35>] warn_slowpath_null+0x25/0x30
** 1418 printk messages dropped ** [ 131.817447] [<c5050d35>] warn_slowpath_null+0x25/0x30
** 1405 printk messages dropped ** [ 131.821640] ---[ end trace 27af91326c31419e ]---
** 1332 printk messages dropped ** [ 131.825618] ---[ end trace 27af91326c3141dc ]---
As of v4.11:
[ 159.748100] BUG: unable to handle kernel paging request at f6f48ca4
[ 159.748106] IP: page_remove_rmap+0x7/0x2c0
[ 159.748107] *pde = 3732c067
[ 159.748108] *pte = 36f48062
[ 159.748109]
[ 159.748111] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 159.748112] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg ppdev vmw_balloon pcspkr vmw_vmci shpchp parport_pc i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
[ 159.748149] CPU: 7 PID: 74 Comm: kswapd0 Not tainted 4.11.0 #266
[ 159.748150] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 159.748151] task: f3822840 task.stack: f3766000
[ 159.748153] EIP: page_remove_rmap+0x7/0x2c0
[ 159.748153] EFLAGS: 00010246 CPU: 7
[ 159.748155] EAX: f6f48c90 EBX: f6f48c90 ECX: 00000000 EDX: 00000000
[ 159.748156] ESI: e8e78e00 EDI: 000002a2 EBP: f3767c30 ESP: f3767c28
[ 159.748157] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 159.748158] CR0: 80050033 CR2: f6f48ca4 CR3: 24c10000 CR4: 000406d0
[ 159.748239] Call Trace:
[ 159.748255] try_to_unmap_one+0x206/0x4f0
[ 159.748261] rmap_walk_file+0x13c/0x270
[ 159.748262] rmap_walk+0x32/0x60
[ 159.748264] try_to_unmap+0xad/0x150
[ 159.748265] ? page_remove_rmap+0x2c0/0x2c0
[ 159.748267] ? page_not_mapped+0x10/0x10
[ 159.748268] ? page_get_anon_vma+0x90/0x90
[ 159.748271] shrink_page_list+0x37a/0xd10
[ 159.748274] shrink_inactive_list+0x173/0x370
[ 159.748277] shrink_node_memcg+0x572/0x7d0
[ 159.748279] ? __list_lru_count_one.isra.7+0x14/0x40
[ 159.748282] shrink_node+0xb3/0x2c0
[ 159.748284] kswapd+0x287/0x5b0
[ 159.748287] kthread+0xd7/0x110
[ 159.748289] ? mem_cgroup_shrink_node+0xa0/0xa0
[ 159.748291] ? kthread_park+0x70/0x70
[ 159.748294] ret_from_fork+0x21/0x2c
[ 159.748295] Code: ff ff ba 78 50 7a c1 89 d8 e8 a6 f8 fe ff 0f 0b 83 e8 01 e9 66 ff ff ff 8d b6 00 00 00 00 8d bf 00 00 00 00 55 89 e5 56 53 89 c3 <8b> 40 14 a8 01 0f 85 a4 01 00 00 89 d8 f6 40 04 01 74 5e 84 d2
[ 159.748321] EIP: page_remove_rmap+0x7/0x2c0 SS:ESP: 0068:f3767c28
[ 159.748321] CR2: 00000000f6f48ca4
[ 159.748324] ---[ end trace e46bc6bd14f9eb43 ]---
As of ad6b67041a45497261617d7a28b15159b202cb5a:
[ 79.478257] BUG: unable to handle kernel paging request at f6df5974
[ 79.478263] IP: page_remove_rmap+0x7/0x2c0
[ 79.478264] *pde = 3732c067
[ 79.478265] *pte = 36df5062
[ 79.478265]
[ 79.478268] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 79.478269] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg ppdev vmw_balloon pcspkr vmw_vmci parport_pc shpchp i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
[ 79.478303] CPU: 5 PID: 978 Comm: a.out Not tainted 4.11.0+ #275
[ 79.478304] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 79.478306] task: f15d9e40 task.stack: f0cc8000
[ 79.478307] EIP: page_remove_rmap+0x7/0x2c0
[ 79.478308] EFLAGS: 00010246 CPU: 5
[ 79.478309] EAX: f6df5960 EBX: f6df5960 ECX: 00000000 EDX: 00000000
[ 79.478310] ESI: f70d3218 EDI: 0000000f EBP: f0cc9ab8 ESP: f0cc9ab0
[ 79.478311] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 79.478313] CR0: 80050033 CR2: f6df5974 CR3: 30b18000 CR4: 000406d0
[ 79.478420] Call Trace:
[ 79.478424] try_to_unmap_one+0x200/0x540
[ 79.478426] rmap_walk_file+0x13c/0x270
[ 79.478428] rmap_walk+0x32/0x60
[ 79.478429] try_to_unmap+0x9d/0x120
[ 79.478431] ? page_remove_rmap+0x2c0/0x2c0
[ 79.478432] ? page_not_mapped+0x10/0x10
[ 79.478434] ? page_get_anon_vma+0x80/0x80
[ 79.478437] shrink_page_list+0x38d/0xdd0
[ 79.478440] shrink_inactive_list+0x173/0x360
[ 79.478443] shrink_node_memcg+0x33a/0x720
[ 79.478446] shrink_node+0xbb/0x2e0
[ 79.478449] do_try_to_free_pages+0xb2/0x2a0
[ 79.478451] try_to_free_pages+0xf9/0x330
[ 79.478455] ? schedule_timeout+0x142/0x200
[ 79.478458] __alloc_pages_slowpath+0x343/0x6f8
[ 79.478460] __alloc_pages_nodemask+0x1aa/0x1c0
[ 79.478464] handle_mm_fault+0x5c9/0xdb0
** 49 printk messages dropped ** [ 79.572015] Code: 55 d4 88 55 db 89 5d e8 89 5d ec 89 45 dc 90 8b 47 10 39 45 dc 74 52 8b 47 10 39 45 dc 74 20 8b 4f 10 8b 45 e8 8b 57 14 89 4d e8 <89> 59 04 89 02 89 50 04 8b 45 dc 89 47 10 89 47 14 8d 76 00 8b
As of 192d7232569ab61ded40c8be691b12832bc6bcd1:
[ 192.152510] BUG: Bad page state in process a.out pfn:18566
[ 192.152513] page:f72997f0 count:0 mapcount:8 mapping:f118f5a4 index:0x0
[ 192.152516] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
[ 192.152520] raw: 19010019 f118f5a4 00000000 00000007 00000000 f7299804 f7299804 00000000
[ 192.152521] raw: 00000000 00000000
[ 192.152521] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 192.152522] bad because of flags: 0x1(locked)
[ 192.152523] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp pcspkr sg vmw_balloon ppdev shpchp parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi ata_piix mptscsih e1000 mptbase libata
[ 192.152561] CPU: 0 PID: 9717 Comm: a.out Not tainted 4.11.0+ #276
[ 192.152562] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 192.152563] Call Trace:
[ 192.152572] dump_stack+0x58/0x76
[ 192.152577] bad_page+0xbf/0x130
[ 192.152581] free_pages_check_bad+0x5b/0x5e
[ 192.152583] free_hot_cold_page+0x211/0x250
[ 192.152586] __put_page+0x30/0x40
[ 192.152588] try_to_unmap_one+0x375/0x550
[ 192.152590] rmap_walk_file+0x13c/0x270
[ 192.152592] rmap_walk+0x32/0x60
[ 192.152593] try_to_unmap+0x9d/0x120
[ 192.152595] ? page_remove_rmap+0x2c0/0x2c0
[ 192.152596] ? page_not_mapped+0x10/0x10
[ 192.152598] ? page_get_anon_vma+0x80/0x80
[ 192.152601] shrink_page_list+0x376/0xdf0
[ 192.152604] shrink_inactive_list+0x173/0x360
[ 192.152607] shrink_node_memcg+0x33a/0x720
[ 192.152609] shrink_node+0xbb/0x2e0
[ 192.152612] do_try_to_free_pages+0xb2/0x2a0
[ 192.152615] try_to_free_pages+0xf9/0x330
[ 192.152619] ? schedule_timeout+0x142/0x200
[ 192.152622] __alloc_pages_slowpath+0x343/0x6f8
[ 192.152624] __alloc_pages_nodemask+0x1aa/0x1c0
[ 192.152627] handle_mm_fault+0x5c9/0xdb0
[ 192.152629] ? filemap_fdatawait+0x50/0x50
[ 192.152633] __do_page_fault+0x19c/0x460
[ 192.152635] ? __do_page_fault+0x460/0x460
[ 192.152637] do_page_fault+0x1a/0x20
[ 192.152639] common_exception+0x6c/0x72
[ 192.152641] EIP: 0x804858f
[ 192.152642] EFLAGS: 00010202 CPU: 0
[ 192.152643] EAX: 02d8b000 EBX: 37689008 ECX: 3a414008 EDX: 00000000
[ 192.152644] ESI: 7ff00000 EDI: 00000000 EBP: bfd2b608 ESP: bfd2b5d0
[ 192.152645] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 192.152647] Disabling lock debugging due to kernel taint
[ 192.152652] page:f72997f0 count:0 mapcount:0 mapping:f118f5a4 index:0x0
[ 192.152654] flags: 0x19030078(uptodate|dirty|lru|active|mappedtodisk|reclaim)
[ 192.152656] raw: 19030078 f118f5a4 00000000 ffffffff 00000000 f400c2d8 f400c2d8 00000000
[ 192.152657] raw: 00000000 00000000
[ 192.152658] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0)
[ 192.152668] ------------[ cut here ]------------
[ 192.152669] kernel BUG at ./include/linux/mm.h:466!
[ 192.152671] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 192.152672] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp pcspkr sg vmw_balloon ppdev shpchp parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi ata_piix mptscsih e1000 mptbase libata
[ 192.152697] CPU: 0 PID: 9717 Comm: a.out Tainted: G B 4.11.0+ #276
[ 192.152698] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 192.152699] task: def05a40 task.stack: dde1a000
[ 192.152701] EIP: put_page_testzero.part.46+0xd/0xf
[ 192.152702] EFLAGS: 00010092 CPU: 0
[ 192.152703] EAX: 00000000 EBX: f400c2c0 ECX: c1870e80 EDX: 00000002
[ 192.152704] ESI: f7299804 EDI: 00000003 EBP: dde1bbec ESP: dde1bbec
[ 192.152705] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 192.152706] CR0: 80050033 CR2: b7505dec CR3: 2c682000 CR4: 000406d0
[ 192.152793] Call Trace:
[ 192.152797] putback_inactive_pages+0x384/0x3c0
[ 192.152800] shrink_inactive_list+0x193/0x360
[ 192.152802] shrink_node_memcg+0x33a/0x720
[ 192.152805] shrink_node+0xbb/0x2e0
[ 192.152808] do_try_to_free_pages+0xb2/0x2a0
[ 192.152810] try_to_free_pages+0xf9/0x330
[ 192.152813] ? schedule_timeout+0x142/0x200
[ 192.152815] __alloc_pages_slowpath+0x343/0x6f8
[ 192.152817] __alloc_pages_nodemask+0x1aa/0x1c0
[ 192.152819] handle_mm_fault+0x5c9/0xdb0
[ 192.152821] ? filemap_fdatawait+0x50/0x50
[ 192.152824] __do_page_fault+0x19c/0x460
[ 192.152826] ? __do_page_fault+0x460/0x460
[ 192.152827] do_page_fault+0x1a/0x20
[ 192.152829] common_exception+0x6c/0x72
[ 192.152830] EIP: 0x804858f
[ 192.152830] EFLAGS: 00010202 CPU: 0
[ 192.152831] EAX: 02d8b000 EBX: 37689008 ECX: 3a414008 EDX: 00000000
[ 192.152832] ESI: 7ff00000 EDI: 00000000 EBP: bfd2b608 ESP: bfd2b5d0
[ 192.152834] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 192.152835] Code: c1 89 e5 e8 5e dd fb ff 0f 0b 55 89 e5 0f 0b 55 ba 70 82 7a c1 89 e5 e8 4a dd fb ff 0f 0b 55 ba 20 e0 78 c1 89 e5 e8 3b dd fb ff <0f> 0b 55 ba 88 70 7a c1 89 e5 e8 2c dd fb ff 0f 0b 55 ba 6c 81
[ 192.152860] EIP: put_page_testzero.part.46+0xd/0xf SS:ESP: 0068:dde1bbec
[ 192.152862] ---[ end trace fc4e4fddd132aaf1 ]---
As of 22ffb33f4620b502799877d4186502bfe20621ea:
[ 77.872133] BUG: Bad page state in process a.out pfn:1873a
[ 77.872136] page:f729e110 count:0 mapcount:6 mapping:f1187224 index:0x0
[ 77.872138] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
[ 77.872141] raw: 19010019 f1187224 00000000 00000005 00000000 f729e124 f729e124 00000000
[ 77.872141] raw: 00000000 00000000
[ 77.872142] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 77.872142] bad because of flags: 0x1(locked)
[ 77.872143] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp ppdev pcspkr vmw_balloon sg vmw_vmci shpchp parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix e1000 mptbase libata
[ 77.872170] CPU: 6 PID: 1538 Comm: a.out Not tainted 4.11.0+ #277
[ 77.872171] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 77.872172] Call Trace:
[ 77.872179] dump_stack+0x58/0x76
[ 77.872183] bad_page+0xbf/0x130
[ 77.872185] free_pages_check_bad+0x5b/0x5e
[ 77.872186] free_hot_cold_page+0x211/0x250
[ 77.872188] __put_page+0x30/0x40
[ 77.872190] try_to_unmap_one+0x375/0x550
[ 77.872191] rmap_walk_file+0x13c/0x270
[ 77.872192] rmap_walk+0x32/0x60
[ 77.872193] try_to_unmap+0x9d/0x120
[ 77.872195] ? page_remove_rmap+0x2c0/0x2c0
[ 77.872195] ? page_not_mapped+0x10/0x10
[ 77.872196] ? page_get_anon_vma+0x80/0x80
[ 77.872198] shrink_page_list+0x376/0xdf0
[ 77.872201] shrink_inactive_list+0x173/0x360
[ 77.872203] shrink_node_memcg+0x33a/0x720
[ 77.872205] shrink_node+0xbb/0x2e0
[ 77.872207] do_try_to_free_pages+0xb2/0x2a0
[ 77.872209] try_to_free_pages+0xf9/0x330
[ 77.872211] __alloc_pages_slowpath+0x343/0x6f8
[ 77.872212] __alloc_pages_nodemask+0x1aa/0x1c0
[ 77.872214] pagecache_get_page+0x56/0x250
[ 77.872217] ? common_exception+0x6c/0x72
[ 77.872218] grab_cache_page_write_begin+0x20/0x40
[ 77.872221] iomap_write_begin+0x6b/0xf0
[ 77.872223] ? iov_iter_fault_in_readable+0x7b/0xd0
[ 77.872224] iomap_write_actor+0xcc/0x1c0
[ 77.872226] ? iomap_write_end+0x70/0x70
[ 77.872228] iomap_apply+0x117/0x1a0
[ 77.872230] iomap_file_buffered_write+0x83/0xc0
[ 77.872231] ? iomap_write_end+0x70/0x70
[ 77.872250] xfs_file_buffered_aio_write+0x93/0x1c0 [xfs]
[ 77.872264] xfs_file_write_iter+0x77/0x150 [xfs]
[ 77.872266] ? __debug_check_no_obj_freed+0xed/0x1b0
[ 77.872268] __vfs_write+0xdf/0x140
[ 77.872269] vfs_write+0x96/0x190
[ 77.872270] SyS_write+0x44/0xa0
[ 77.872272] do_fast_syscall_32+0x86/0x130
[ 77.872275] entry_SYSENTER_32+0x4e/0x7c
[ 77.872276] EIP: 0xb76fdda1
[ 77.872276] EFLAGS: 00000246 CPU: 6
[ 77.872277] EAX: ffffffda EBX: 00000003 ECX: 0804a060 EDX: 00001000
[ 77.872278] ESI: bf863384 EDI: 00000000 EBP: bf8632e8 ESP: bf863298
[ 77.872279] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 77.872280] Disabling lock debugging due to kernel taint
[ 77.872284] page:f729e110 count:0 mapcount:0 mapping:f1187224 index:0x0
[ 77.872285] flags: 0x19030078(uptodate|dirty|lru|active|mappedtodisk|reclaim)
[ 77.872287] raw: 19030078 f1187224 00000000 ffffffff 00000000 f400c2d8 f400c2d8 00000000
[ 77.872287] raw: 00000000 00000000
[ 77.872288] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0)
[ 77.872293] ------------[ cut here ]------------
[ 77.872294] kernel BUG at ./include/linux/mm.h:466!
[ 77.872307] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 77.872307] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp ppdev pcspkr vmw_balloon sg vmw_vmci shpchp parport_pc parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix e1000 mptbase libata
[ 77.872324] CPU: 6 PID: 1538 Comm: a.out Tainted: G B 4.11.0+ #277
[ 77.872324] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 77.872325] task: ef500a40 task.stack: ef562000
[ 77.872327] EIP: put_page_testzero.part.46+0xd/0xf
[ 77.872327] EFLAGS: 00010082 CPU: 6
[ 77.872328] EAX: 00000000 EBX: f400c2c0 ECX: c1870e80 EDX: 00000002
[ 77.872329] ESI: f729e124 EDI: 00000003 EBP: ef563a24 ESP: ef563a24
[ 77.872329] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 77.872330] CR0: 80050033 CR2: 0804b05f CR3: 2fbc5000 CR4: 000406d0
[ 77.872389] Call Trace:
[ 77.872392] putback_inactive_pages+0x384/0x3c0
[ 77.872394] shrink_inactive_list+0x193/0x360
[ 77.872396] shrink_node_memcg+0x33a/0x720
[ 77.872398] shrink_node+0xbb/0x2e0
[ 77.872400] do_try_to_free_pages+0xb2/0x2a0
[ 77.872401] try_to_free_pages+0xf9/0x330
[ 77.872403] __alloc_pages_slowpath+0x343/0x6f8
[ 77.872405] __alloc_pages_nodemask+0x1aa/0x1c0
[ 77.872407] pagecache_get_page+0x56/0x250
[ 77.872408] ? common_exception+0x6c/0x72
[ 77.872409] grab_cache_page_write_begin+0x20/0x40
[ 77.872411] iomap_write_begin+0x6b/0xf0
[ 77.872412] ? iov_iter_fault_in_readable+0x7b/0xd0
[ 77.872413] iomap_write_actor+0xcc/0x1c0
[ 77.872415] ? iomap_write_end+0x70/0x70
[ 77.872416] iomap_apply+0x117/0x1a0
[ 77.872418] iomap_file_buffered_write+0x83/0xc0
[ 77.872420] ? iomap_write_end+0x70/0x70
[ 77.872433] xfs_file_buffered_aio_write+0x93/0x1c0 [xfs]
[ 77.872445] xfs_file_write_iter+0x77/0x150 [xfs]
[ 77.872446] ? __debug_check_no_obj_freed+0xed/0x1b0
[ 77.872447] __vfs_write+0xdf/0x140
[ 77.872448] vfs_write+0x96/0x190
[ 77.872449] SyS_write+0x44/0xa0
[ 77.872451] do_fast_syscall_32+0x86/0x130
[ 77.872453] entry_SYSENTER_32+0x4e/0x7c
[ 77.872453] EIP: 0xb76fdda1
[ 77.872454] EFLAGS: 00000246 CPU: 6
[ 77.872454] EAX: ffffffda EBX: 00000003 ECX: 0804a060 EDX: 00001000
[ 77.872455] ESI: bf863384 EDI: 00000000 EBP: bf8632e8 ESP: bf863298
[ 77.872456] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 77.872456] Code: c1 89 e5 e8 9e dd fb ff 0f 0b 55 89 e5 0f 0b 55 ba 70 82 7a c1 89 e5 e8 8a dd fb ff 0f 0b 55 ba 20 e0 78 c1 89 e5 e8 7b dd fb ff <0f> 0b 55 ba 88 70 7a c1 89 e5 e8 6c dd fb ff 0f 0b 55 ba 6c 81
[ 77.872472] EIP: put_page_testzero.part.46+0xd/0xf SS:ESP: 0068:ef563a24
[ 77.872474] ---[ end trace fac67653a2c4994f ]---
As of 18863d3a3f593f47b075b9f53ebf9228dc76cf72:
[ 188.992549] BUG: Bad page state in process a.out pfn:197ea
[ 188.992551] page:f72c7c90 count:0 mapcount:12 mapping:f11b8ca4 index:0x0
[ 188.992554] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
[ 188.992557] raw: 19010019 f11b8ca4 00000000 0000000b 00000000 f72c7ca4 f72c7ca4 00000000
[ 188.992557] raw: 00000000 00000000
[ 188.992558] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 188.992559] bad because of flags: 0x1(locked)
[ 188.992559] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp pcspkr sg ppdev vmw_balloon parport_pc parport i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi ata_piix e1000 mptscsih libata mptbase
[ 188.992624] CPU: 3 PID: 5140 Comm: a.out Not tainted 4.11.0+ #278
[ 188.992625] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 188.992625] Call Trace:
[ 188.992632] dump_stack+0x58/0x76
[ 188.992636] bad_page+0xbf/0x130
[ 188.992639] free_pages_check_bad+0x5b/0x5e
[ 188.992640] free_hot_cold_page+0x211/0x250
[ 188.992642] __put_page+0x30/0x40
[ 188.992643] try_to_unmap_one+0x375/0x550
[ 188.992645] rmap_walk_file+0x13c/0x270
[ 188.992646] rmap_walk+0x32/0x60
[ 188.992647] try_to_unmap+0x9d/0x120
[ 188.992648] ? page_remove_rmap+0x2c0/0x2c0
[ 188.992649] ? page_not_mapped+0x10/0x10
[ 188.992650] ? page_get_anon_vma+0x80/0x80
[ 188.992652] shrink_page_list+0x376/0xdf0
[ 188.992654] shrink_inactive_list+0x173/0x360
[ 188.992656] shrink_node_memcg+0x33a/0x720
[ 188.992659] shrink_node+0xbb/0x2e0
[ 188.992661] do_try_to_free_pages+0xb2/0x2a0
[ 188.992662] try_to_free_pages+0xf9/0x330
[ 188.992666] ? schedule_timeout+0x142/0x200
[ 188.992668] __alloc_pages_slowpath+0x343/0x6f8
[ 188.992669] __alloc_pages_nodemask+0x1aa/0x1c0
[ 188.992671] handle_mm_fault+0x5c9/0xdb0
[ 188.992673] ? pick_next_task_fair+0x431/0x530
[ 188.992676] __do_page_fault+0x19c/0x460
[ 188.992677] ? __do_page_fault+0x460/0x460
[ 188.992678] do_page_fault+0x1a/0x20
[ 188.992680] common_exception+0x6c/0x72
[ 188.992681] EIP: 0x804858f
[ 188.992681] EFLAGS: 00010202 CPU: 3
[ 188.992682] EAX: 0094c000 EBX: 37706008 ECX: 38052008 EDX: 00000000
[ 188.992683] ESI: 7ff00000 EDI: 00000000 EBP: bfeac158 ESP: bfeac120
[ 188.992684] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 188.992685] Disabling lock debugging due to kernel taint
[ 188.992689] page:f72c7c90 count:0 mapcount:0 mapping:f11b8ca4 index:0x0
[ 188.992691] flags: 0x19030078(uptodate|dirty|lru|active|mappedtodisk|reclaim)
[ 188.992692] raw: 19030078 f11b8ca4 00000000 ffffffff 00000000 f400c2d8 f400c2d8 00000000
[ 188.992693] raw: 00000000 00000000
[ 188.992693] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0)
[ 188.992698] ------------[ cut here ]------------
[ 188.992699] kernel BUG at ./include/linux/mm.h:466!
[ 188.992700] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 188.992701] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp pcspkr sg ppdev vmw_balloon parport_pc parport i2c_piix4 vmw_vmci shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi ata_piix e1000 mptscsih libata mptbase
[ 188.992716] CPU: 3 PID: 5140 Comm: a.out Tainted: G B 4.11.0+ #278
[ 188.992716] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 188.992717] task: e4929440 task.stack: e49ee000
[ 188.992719] EIP: put_page_testzero.part.46+0xd/0xf
[ 188.992719] EFLAGS: 00010092 CPU: 3
[ 188.992720] EAX: 00000000 EBX: f400c2c0 ECX: c1870e80 EDX: 00000002
[ 188.992720] ESI: f72c7ca4 EDI: 00000003 EBP: e49efbec ESP: e49efbec
[ 188.992721] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 188.992722] CR0: 80050033 CR2: 386b6008 CR3: 24a5d000 CR4: 000406d0
[ 188.992781] Call Trace:
[ 188.992784] putback_inactive_pages+0x384/0x3c0
[ 188.992786] shrink_inactive_list+0x193/0x360
[ 188.992788] shrink_node_memcg+0x33a/0x720
[ 188.992790] shrink_node+0xbb/0x2e0
[ 188.992791] do_try_to_free_pages+0xb2/0x2a0
[ 188.992793] try_to_free_pages+0xf9/0x330
[ 188.992795] ? schedule_timeout+0x142/0x200
[ 188.992797] __alloc_pages_slowpath+0x343/0x6f8
[ 188.992798] __alloc_pages_nodemask+0x1aa/0x1c0
[ 188.992800] handle_mm_fault+0x5c9/0xdb0
[ 188.992801] ? pick_next_task_fair+0x431/0x530
[ 188.992803] __do_page_fault+0x19c/0x460
[ 188.992804] ? __do_page_fault+0x460/0x460
[ 188.992805] do_page_fault+0x1a/0x20
[ 188.992806] common_exception+0x6c/0x72
[ 188.992807] EIP: 0x804858f
[ 188.992807] EFLAGS: 00010202 CPU: 3
[ 188.992808] EAX: 0094c000 EBX: 37706008 ECX: 38052008 EDX: 00000000
[ 188.992809] ESI: 7ff00000 EDI: 00000000 EBP: bfeac158 ESP: bfeac120
[ 188.992809] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 188.992810] Code: c1 89 e5 e8 9e dd fb ff 0f 0b 55 89 e5 0f 0b 55 ba 70 82 7a c1 89 e5 e8 8a dd fb ff 0f 0b 55 ba 20 e0 78 c1 89 e5 e8 7b dd fb ff <0f> 0b 55 ba 88 70 7a c1 89 e5 e8 6c dd fb ff 0f 0b 55 ba 6c 81
[ 188.992826] EIP: put_page_testzero.part.46+0xd/0xf SS:ESP: 0068:e49efbec
[ 188.992827] ---[ end trace 16134a8679cd7c8c ]---
As of c24f386c60b2269d532a23e70939ed8ce55d7005:
[ 56.841712] BUG: unable to handle kernel paging request at 32eba010
[ 56.841718] IP: page_remove_rmap+0x190/0x2c0
[ 56.841719] *pde = 00000000
[ 56.841720]
[ 56.841722] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
[ 56.841723] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp ppdev vmw_balloon pcspkr sg shpchp i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix e1000 mptbase libata
** 361 printk messages dropped ** [ 56.843217] BUG: unable to handle kernel paging request at 00737d5c
** 2283 printk messages dropped ** [ 56.849961] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
** 1563 printk messages dropped ** [ 56.854562] BUG: unable to handle kernel paging request at 00737d5c
** 2110 printk messages dropped ** [ 56.860720] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp ppdev vmw_balloon pcspkr sg shpchp i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix e1000 mptbase libata
** 11037 printk messages dropped ** [ 56.885507] EFLAGS: 00010202 CPU: 2
** 1332 printk messages dropped ** [ 56.889324] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:e0737f30
** 1975 printk messages dropped ** [ 56.895432] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:e0737f30
** 2201 printk messages dropped ** [ 56.901915] EIP: 0x804858f
** 1357 printk messages dropped ** [ 56.905721] ---[ end trace 66e5871b6d0ec353 ]---
As of d44d363f65780f2ac2ec672164555af54896d40d:
[ 157.814036] BUG: unable to handle kernel paging request at f6c8f9a4
[ 157.814041] IP: page_remove_rmap+0x7/0x2c0
[ 157.814042] *pde = 3732c067
[ 157.814043] *pte = 36c8f062
[ 157.814044]
[ 157.814045] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 157.814046] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg ppdev vmw_balloon pcspkr shpchp vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix e1000 mptbase libata
[ 157.814074] CPU: 6 PID: 3979 Comm: a.out Not tainted 4.11.0+ #280
[ 157.814074] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 157.814075] task: e8456440 task.stack: e8502000
[ 157.814077] EIP: page_remove_rmap+0x7/0x2c0
[ 157.814077] EFLAGS: 00010246 CPU: 6
[ 157.814078] EAX: f6c8f990 EBX: f6c8f990 ECX: 00000001 EDX: 00000000
[ 157.814079] ESI: f71511b8 EDI: 000006d9 EBP: e85038e0 ESP: e85038d8
[ 157.814080] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 157.814081] CR0: 80050033 CR2: f6c8f9a4 CR3: 284b0000 CR4: 000406d0
[ 157.814138] Call Trace:
[ 157.814141] try_to_unmap_one+0x200/0x500
[ 157.814143] rmap_walk_file+0x13c/0x270
[ 157.814144] rmap_walk+0x32/0x60
[ 157.814145] try_to_unmap+0xb5/0x150
[ 157.814146] ? page_remove_rmap+0x2c0/0x2c0
[ 157.814147] ? page_not_mapped+0x10/0x10
[ 157.814148] ? page_get_anon_vma+0x80/0x80
[ 157.814151] shrink_page_list+0x368/0xce0
[ 157.814153] shrink_inactive_list+0x173/0x360
[ 157.814155] shrink_node_memcg+0x33a/0x720
[ 157.814157] shrink_node+0xb3/0x2e0
[ 157.814159] do_try_to_free_pages+0xb2/0x290
[ 157.814161] try_to_free_pages+0xea/0x310
[ 157.814164] __alloc_pages_slowpath+0x340/0x6f5
[ 157.814184] ? xfs_trans_free+0x59/0x60 [xfs]
[ 157.814185] __alloc_pages_nodemask+0x19e/0x1b0
[ 157.814188] pagecache_get_page+0x56/0x250
[ 157.814203] ? xfs_iunlock+0xac/0x160 [xfs]
[ 157.814204] grab_cache_page_write_begin+0x20/0x40
[ 157.814207] iomap_write_begin+0x6b/0xf0
[ 157.814208] iomap_write_actor+0xcc/0x1c0
[ 157.814210] ? iomap_write_end+0x70/0x70
[ 157.814211] iomap_apply+0x117/0x1a0
[ 157.814213] iomap_file_buffered_write+0x83/0xc0
[ 157.814214] ? iomap_write_end+0x70/0x70
[ 157.814229] xfs_file_buffered_aio_write+0x93/0x1c0 [xfs]
[ 157.814230] ? filemap_map_pages+0x34b/0x3c0
[ 157.814243] xfs_file_write_iter+0x77/0x150 [xfs]
[ 157.814255] ? xfs_filemap_fault+0x36/0x40 [xfs]
[ 157.814256] __vfs_write+0xdf/0x140
[ 157.814258] vfs_write+0x96/0x190
[ 157.814259] SyS_write+0x44/0xa0
[ 157.814261] do_fast_syscall_32+0x86/0x130
[ 157.814263] entry_SYSENTER_32+0x4e/0x7c
[ 157.814264] EIP: 0xb7751da1
[ 157.814265] EFLAGS: 00000246 CPU: 6
[ 157.814265] EAX: ffffffda EBX: 00000003 ECX: 0804a060 EDX: 00001000
[ 157.814266] ESI: bf989bc4 EDI: 00000000 EBP: bf989b28 ESP: bf989ad8
[ 157.814267] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 157.814268] Code: 00 83 e8 01 e9 dc fe ff ff ba b8 73 7a c1 89 d8 e8 af fa fe ff 0f 0b 83 e8 01 eb 8e 90 8d b4 26 00 00 00 00 55 89 e5 56 53 89 c3 <8b> 40 14 a8 01 0f 85 a4 01 00 00 89 d8 f6 40 04 01 74 5e 84 d2
[ 157.814283] EIP: page_remove_rmap+0x7/0x2c0 SS:ESP: 0068:e85038d8
[ 157.814284] CR2: 00000000f6c8f9a4
[ 157.814286] ---[ end trace a941a5b48c8262c4 ]---
As of a128ca71fb29ed4444b80f38a0148b468826e19b:
[ 54.943695] BUG: unable to handle kernel paging request at c1147f4b
[ 54.943704] IP: _raw_spin_lock_irqsave+0x1c/0x40
[ 54.943705] *pde = 01adc063
[ 54.943706] *pte = 01147161
[ 54.943707]
[ 54.943709] Oops: 0003 [#1] SMP DEBUG_PAGEALLOC
[ 54.943710] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg ppdev pcspkr vmw_balloon vmw_vmci i2c_piix4 parport_pc parport shpchp ip_tables xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
** 4557 printk messages dropped ** [ 54.959191] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2357 printk messages dropped ** [ 54.966297] ---[ end trace 738da708534ed3b1 ]---
** 1530 printk messages dropped ** [ 54.970986] EFLAGS: 00010202 CPU: 3
** 1356 printk messages dropped ** [ 54.975010] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:e0e35f30
** 2246 printk messages dropped ** [ 54.981496] EAX: e0e35d04 EBX: 00000009 ECX: e0cfc640 EDX: c1074c33
** 2251 printk messages dropped ** [ 54.987786] IP: blk_flush_plug_list+0x54/0x210
** 1526 printk messages dropped ** [ 54.992041] EIP: 0x804858f
** 1285 printk messages dropped ** [ 54.995936] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2452 printk messages dropped ** [ 55.003072] Call Trace:
** 1290 printk messages dropped ** [ 55.006821] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2456 printk messages dropped ** [ 55.014067] rewind_stack_do_exit+0x11/0x13
** 1514 printk messages dropped ** [ 55.018507] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:e0e35f30
** 2336 printk messages dropped ** [ 55.025302] ESI: 7ff00000 EDI: 00000000 EBP: bf9bdce8 ESP: bf9bdcb0
** 2296 printk messages dropped ** [ 55.032024] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2457 printk messages dropped ** [ 55.039223] EAX: 00218000 EBX: 37701008 ECX: 37919008 EDX: 00000000
** 2339 printk messages dropped ** [ 55.046204] EAX: 00218000 EBX: 37701008 ECX: 37919008 EDX: 00000000
** 2377 printk messages dropped ** [ 55.053003] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2573 printk messages dropped ** [ 55.060515] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2631 printk messages dropped ** [ 55.068308] ESI: e0e36000 EDI: e0cfc640 EBP: e0e35fac ESP: e0e35f70
** 2483 printk messages dropped ** [ 55.075574] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:e0e35f30
** 2476 printk messages dropped ** [ 55.082895] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
** 3092 printk messages dropped ** [ 55.092077] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2703 printk messages dropped ** [ 55.100031] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2677 printk messages dropped ** [ 55.107930] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2704 printk messages dropped ** [ 55.115891] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
** 3133 printk messages dropped ** [ 55.124900] EFLAGS: 00010202 CPU: 3
** 1518 printk messages dropped ** [ 55.129366] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2721 printk messages dropped ** [ 55.137262] ---[ end trace 738da708534edc79 ]---
[ 54.943695] BUG: unable to handle kernel paging request at c1147f4b
[ 54.943704] IP: _raw_spin_lock_irqsave+0x1c/0x40
[ 54.943705] *pde = 01adc063
[ 54.943706] *pte = 01147161
[ 54.943707]
[ 54.943709] Oops: 0003 [#1] SMP DEBUG_PAGEALLOC
[ 54.943710] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp sg ppdev pcspkr vmw_balloon vmw_vmci i2c_piix4 parport_pc parport shpchp ip_tables xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
** 4557 printk messages dropped ** [ 54.959191] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2357 printk messages dropped ** [ 54.966297] ---[ end trace 738da708534ed3b1 ]---
** 1530 printk messages dropped ** [ 54.970986] EFLAGS: 00010202 CPU: 3
** 1356 printk messages dropped ** [ 54.975010] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:e0e35f30
** 2246 printk messages dropped ** [ 54.981496] EAX: e0e35d04 EBX: 00000009 ECX: e0cfc640 EDX: c1074c33
** 2251 printk messages dropped ** [ 54.987786] IP: blk_flush_plug_list+0x54/0x210
** 1526 printk messages dropped ** [ 54.992041] EIP: 0x804858f
** 1285 printk messages dropped ** [ 54.995936] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2452 printk messages dropped ** [ 55.003072] Call Trace:
** 1290 printk messages dropped ** [ 55.006821] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2456 printk messages dropped ** [ 55.014067] rewind_stack_do_exit+0x11/0x13
** 1514 printk messages dropped ** [ 55.018507] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:e0e35f30
** 2336 printk messages dropped ** [ 55.025302] ESI: 7ff00000 EDI: 00000000 EBP: bf9bdce8 ESP: bf9bdcb0
** 2296 printk messages dropped ** [ 55.032024] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2457 printk messages dropped ** [ 55.039223] EAX: 00218000 EBX: 37701008 ECX: 37919008 EDX: 00000000
** 2339 printk messages dropped ** [ 55.046204] EAX: 00218000 EBX: 37701008 ECX: 37919008 EDX: 00000000
** 2377 printk messages dropped ** [ 55.053003] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2573 printk messages dropped ** [ 55.060515] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2631 printk messages dropped ** [ 55.068308] ESI: e0e36000 EDI: e0cfc640 EBP: e0e35fac ESP: e0e35f70
** 2483 printk messages dropped ** [ 55.075574] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:e0e35f30
** 2476 printk messages dropped ** [ 55.082895] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
** 3092 printk messages dropped ** [ 55.092077] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2703 printk messages dropped ** [ 55.100031] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2677 printk messages dropped ** [ 55.107930] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2704 printk messages dropped ** [ 55.115891] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
** 3133 printk messages dropped ** [ 55.124900] EFLAGS: 00010202 CPU: 3
** 1518 printk messages dropped ** [ 55.129366] CPU: 3 PID: 6387 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2721 printk messages dropped ** [ 55.137262] ---[ end trace 738da708534edc79 ]---
As of a128ca71fb29ed4444b80f38a0148b468826e19b:
[ 65.347528] BUG: unable to handle kernel paging request at 330de01a
[ 65.347534] IP: page_remove_rmap+0x14/0x2c0
[ 65.347535] *pde = 00000000
[ 65.347536]
[ 65.347538] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 65.347539] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter coretemp pcspkr vmw_balloon sg ppdev shpchp vmw_vmci parport_pc i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi ata_piix mptscsih e1000 mptbase libata
** 3288 printk messages dropped ** [ 65.362121] CR0: 80050033 CR2: 008b7d5c CR3: 2e67c000 CR4: 000406d0
** 2299 printk messages dropped ** [ 65.368742] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:ee8b7f30
** 2345 printk messages dropped ** [ 65.375469] CPU: 1 PID: 1883 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2487 printk messages dropped ** [ 65.382562] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
** 1300 printk messages dropped ** [ 65.386360] Code: 55 d4 88 55 db 89 5d e8 89 5d ec 89 45 dc 90 8b 47 10 39 45 dc 74 52 8b 47 10 39 45 dc 74 20 8b 4f 10 8b 45 e8 8b 57 14 89 4d e8 <89> 59 04 89 02 89 50 04 8b 45 dc 89 47 10 89 47 14 8d 76 00 8b
** 4576 printk messages dropped ** [ 65.400878] EIP: blk_flush_plug_list+0x54/0x210 SS:ESP: 0068:ee8b7f30
** 2499 printk messages dropped ** [ 65.408166] WARNING: CPU: 1 PID: 1883 at kernel/exit.c:785 do_exit+0x3a/0xa30
** 2627 printk messages dropped ** [ 65.415842] CPU: 1 PID: 1883 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2669 printk messages dropped ** [ 65.424749] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
** 1749 printk messages dropped ** [ 65.429863] CPU: 1 PID: 1883 Comm: a.out Tainted: G D W 4.11.0+ #281
** 2640 printk messages dropped ** [ 65.437531] ESI: 7ff00000 EDI: 00000000 EBP: bfbb8f38 ESP: bfbb8f00
** 2503 printk messages dropped ** [ 65.444809]
** 1242 printk messages dropped ** [ 65.448427] ---[ end trace 2b10bcdd7b092bf2 ]---
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-09 10:39 ` [mm? 4.15-rc7] " Tetsuo Handa
@ 2018-01-10 11:49 ` Tetsuo Handa
2018-01-10 12:45 ` Michal Hocko
0 siblings, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-10 11:49 UTC (permalink / raw)
To: linux-mm; +Cc: mhocko
Tetsuo Handa wrote:
> I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
> So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
> Does anyone know what is happening?
I simplified the reproducer and succeeded to reproduce this bug with both
i7-2630QM (8 core) and i5-4440S (4 core). Thus, I think that this bug is
not architecture specific.
---------- reproducer start ----------
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char *argv[])
{
if (argc != 1) {
unsigned long long size;
char *buf = NULL;
unsigned long long i;
for (size = 1048576; size < 512ULL * (1 << 30); size += 1048576) {
char *cp = realloc(buf, size);
if (!cp) {
size -= 1048576;
break;
}
buf = cp;
}
for (i = 0; i < size; i += 4096)
buf[i] = 0;
_exit(0);
} else
while (1)
if (fork() == 0)
execlp(argv[0], argv[0], "", NULL);
return 0;
}
---------- reproducer end ----------
This bug occurs immediately after a process terminated
(not limited to being killed by the OOM-killer).
----------
[ 100.838906] Out of memory: Kill process 891 (a.out) score 14 or sacrifice child
[ 100.841891] Killed process 891 (a.out) total-vm:2099260kB, anon-rss:47312kB, file-rss:8kB, shmem-rss:0kB
[ 100.847395] BUG: Bad page state in process a.out pfn:3e6a9
[ 100.849515] page:f5d19a68 count:0 mapcount:9 mapping:f3d57c8c index:0x0
[ 100.851773] flags: 0x3e010019(locked|uptodate|dirty|mappedtodisk)
[ 100.853950] raw: 3e010019 f3d57c8c 00000000 00000008 00000000 00000100 00000200 00000000
[ 100.856738] raw: 00000000 00000000
[ 100.858510] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 100.860880] bad because of flags: 0x1(locked)
[ 100.862685] Modules linked in: xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix mptbase libata
[ 100.867413] CPU: 0 PID: 919 Comm: a.out Not tainted 4.15.0-rc7+ #196
[ 100.869521] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
[ 100.874256] Call Trace:
[ 100.875664] dump_stack+0x58/0x76
[ 100.878332] bad_page+0xc4/0x140
[ 100.879911] free_pages_check_bad+0x5b/0x5e
[ 100.882825] free_unref_page+0x149/0x160
[ 100.884498] __put_page+0x2e/0x40
[ 100.886089] try_to_unmap_one+0x37d/0x590
[ 100.887839] rmap_walk_file+0x13c/0x250
[ 100.890229] rmap_walk+0x32/0x60
[ 100.893452] try_to_unmap+0x4d/0x100
[ 100.895316] ? page_remove_rmap+0x2e0/0x2e0
[ 100.897142] ? page_not_mapped+0x10/0x10
[ 100.898815] ? page_get_anon_vma+0x80/0x80
[ 100.900539] shrink_page_list+0x3a2/0x1000
[ 100.902247] shrink_inactive_list+0x1b2/0x440
[ 100.904431] shrink_node_memcg+0x34a/0x770
[ 100.906126] shrink_node+0xbb/0x2e0
[ 100.910555] do_try_to_free_pages+0xba/0x320
[ 100.913526] try_to_free_pages+0x11d/0x330
[ 100.915234] __alloc_pages_slowpath+0x303/0x6d9
[ 100.916941] ? __accumulate_pelt_segments+0x32/0x50
[ 100.918991] __alloc_pages_nodemask+0x16d/0x180
[ 100.921866] do_anonymous_page+0xab/0x4f0
[ 100.926874] handle_mm_fault+0x531/0x8d0
[ 100.929028] ? __phys_addr+0x32/0x70
[ 100.930829] ? load_new_mm_cr3+0x6a/0x90
[ 100.932898] __do_page_fault+0x1ea/0x4d0
[ 100.939138] ? __do_page_fault+0x4d0/0x4d0
[ 100.944536] do_page_fault+0x1a/0x20
[ 100.948193] common_exception+0x6f/0x76
[ 100.951431] EIP: 0x8048437
[ 100.952710] EFLAGS: 00010202 CPU: 0
[ 100.954576] EAX: 01e7c000 EBX: 7ff00000 ECX: 39d6a008 EDX: 00000000
[ 100.957048] ESI: 7ff00000 EDI: 00000000 EBP: bfc49318 ESP: bfc492e0
[ 100.960794] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 100.965552] Disabling lock debugging due to kernel taint
[ 100.967496] page:f5d19a68 count:0 mapcount:0 mapping:f3d57c8c index:0x0
[ 100.969872] flags: 0x3e030078(uptodate|dirty|lru|active|mappedtodisk|reclaim)
[ 100.976635] raw: 3e030078 f3d57c8c 00000000 ffffffff 00000000 f400f680 f400f680 00000000
[ 100.979529] raw: 00000000 00000000
[ 100.981347] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0)
[ 100.983838] ------------[ cut here ]------------
[ 100.986268] kernel BUG at ./include/linux/mm.h:483!
[ 100.990886] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 100.993122] Modules linked in: xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix mptbase libata
[ 100.997779] CPU: 0 PID: 919 Comm: a.out Tainted: G B 4.15.0-rc7+ #196
[ 101.000477] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
[ 101.004621] EIP: put_page_testzero.part.53+0xd/0xf
[ 101.006692] EFLAGS: 00010086 CPU: 0
[ 101.008524] EAX: 00000000 EBX: f400f668 ECX: c19dd988 EDX: 00000086
[ 101.011584] ESI: f5d19a7c EDI: 00000003 EBP: f2e15bcc ESP: f2e15bcc
[ 101.013808] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 101.015759] CR0: 80050033 CR2: 38a1e008 CR3: 3197e000 CR4: 001406d0
[ 101.017934] Call Trace:
[ 101.019235] putback_inactive_pages+0x384/0x3c0
[ 101.021417] shrink_inactive_list+0x205/0x440
[ 101.023592] shrink_node_memcg+0x34a/0x770
[ 101.025453] shrink_node+0xbb/0x2e0
[ 101.027097] do_try_to_free_pages+0xba/0x320
[ 101.028729] try_to_free_pages+0x11d/0x330
[ 101.030419] __alloc_pages_slowpath+0x303/0x6d9
[ 101.032194] ? __accumulate_pelt_segments+0x32/0x50
[ 101.034161] __alloc_pages_nodemask+0x16d/0x180
[ 101.035866] do_anonymous_page+0xab/0x4f0
[ 101.037464] handle_mm_fault+0x531/0x8d0
[ 101.039042] ? __phys_addr+0x32/0x70
[ 101.041023] ? load_new_mm_cr3+0x6a/0x90
[ 101.042784] __do_page_fault+0x1ea/0x4d0
[ 101.044805] ? __do_page_fault+0x4d0/0x4d0
[ 101.046503] do_page_fault+0x1a/0x20
[ 101.048046] common_exception+0x6f/0x76
[ 101.049637] EIP: 0x8048437
[ 101.051098] EFLAGS: 00010202 CPU: 0
[ 101.054732] EAX: 01e7c000 EBX: 7ff00000 ECX: 39d6a008 EDX: 00000000
[ 101.058435] ESI: 7ff00000 EDI: 00000000 EBP: bfc49318 ESP: bfc492e0
[ 101.062077] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 101.064576] Code: 55 ba 24 1a 7f c1 89 e5 e8 b2 2c 01 00 0f 0b 55 ba ec 2f 7f c1 89 e5 e8 a3 2c 01 00 0f 0b 55 ba a8 7b 7d c1 89 e5 e8 94 2c 01 00 <0f> 0b 55 ba b0 2e 7f c1 89 e5 e8 85 2c 01 00 0f 0b 90 90 90 55
[ 101.074916] EIP: put_page_testzero.part.53+0xd/0xf SS:ESP: 0068:f2e15bcc
[ 101.078189] ---[ end trace d34ac0a26b29ce39 ]---
----------
----------
[ 30.998639] Out of memory: Kill process 330 (a.out) score 22 or sacrifice child
[ 31.001106] Killed process 330 (a.out) total-vm:2099260kB, anon-rss:72524kB, file-rss:8kB, shmem-rss:0kB
[ 31.042032] BUG: unable to handle kernel paging request at 3304001a
[ 31.044552] IP: page_remove_rmap+0x14/0x2e0
[ 31.046248] *pde = 00000000
[ 31.047722] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 31.049977] Modules linked in: xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix mptbase libata
[ 31.054476] CPU: 0 PID: 474 Comm: a.out Not tainted 4.15.0-rc7+ #196
[ 31.056620] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
[ 31.060414] EIP: page_remove_rmap+0x14/0x2e0
[ 31.062110] EFLAGS: 00010202 CPU: 0
[ 31.063896] EAX: 33040016 EBX: f3096c20 ECX: 0000015a EDX: 00000000
[ 31.066797] ESI: 00000159 EDI: f48b92f8 EBP: f10cda9c ESP: f10cda94
[ 31.069081] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 31.071035] CR0: 80050033 CR2: 3304001a CR3: 31100000 CR4: 001406d0
[ 31.073261] Call Trace:
[ 31.074534] try_to_unmap_one+0x20e/0x590
[ 31.076124] rmap_walk_file+0x13c/0x250
[ 31.077652] rmap_walk+0x32/0x60
[ 31.079000] try_to_unmap+0x4d/0x100
[ 31.080381] ? page_remove_rmap+0x2e0/0x2e0
[ 31.082397] ? page_not_mapped+0x10/0x10
[ 31.084272] ? page_get_anon_vma+0x80/0x80
[ 31.085852] shrink_page_list+0x3a2/0x1000
[ 31.087260] shrink_inactive_list+0x1b2/0x440
[ 31.088728] shrink_node_memcg+0x34a/0x770
[ 31.090099] shrink_node+0xbb/0x2e0
[ 31.091376] do_try_to_free_pages+0xba/0x320
[ 31.092908] try_to_free_pages+0x11d/0x330
[ 31.094272] ? __wake_up+0x1a/0x20
[ 31.095515] __alloc_pages_slowpath+0x303/0x6d9
[ 31.097094] ? debug_object_activate+0x126/0x1d0
[ 31.099222] ? ktime_get+0x47/0xf0
[ 31.100605] __alloc_pages_nodemask+0x16d/0x180
[ 31.102142] do_anonymous_page+0xab/0x4f0
[ 31.103546] handle_mm_fault+0x531/0x8d0
[ 31.104844] ? pick_next_task_fair+0xe1/0x490
[ 31.106201] ? irq_exit+0x45/0xb0
[ 31.107822] ? smp_apic_timer_interrupt+0x4b/0x80
[ 31.109289] __do_page_fault+0x1ea/0x4d0
[ 31.110631] ? __do_page_fault+0x4d0/0x4d0
[ 31.112277] do_page_fault+0x1a/0x20
[ 31.113990] common_exception+0x6f/0x76
[ 31.115879] EIP: 0x8048437
[ 31.117195] EFLAGS: 00010202 CPU: 0
[ 31.118702] EAX: 00c4f000 EBX: 7ff00000 ECX: 38acc008 EDX: 00000000
[ 31.120624] ESI: 7ff00000 EDI: 00000000 EBP: bf937a08 ESP: bf9379d0
[ 31.122365] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 31.124045] Code: c4 fe ff ff ba ec 32 7f c1 89 d8 e8 97 e5 fe ff 0f 0b 83 e8 01 eb 8e 55 89 e5 56 53 89 c3 8b 40 14 a8 01 0f 85 be 01 00 00 89 d8 <f6> 40 04 01 74 5e 84 d2 0f 85 9e 00 00 00 3e 83 43 0c ff 78 07
[ 31.129606] EIP: page_remove_rmap+0x14/0x2e0 SS:ESP: 0068:f10cda94
[ 31.131409] CR2: 000000003304001a
[ 31.133754] ---[ end trace bee2e936b4fd74fc ]---
----------
I can hit this bug with CONFIG_SMP=y CONFIG_PREEMPT_NONE=y nosmp init=/bin/bash
but so far I haven't hit this bug with CONFIG_SMP=n .
Config is at http://I-love.SAKURA.ne.jp/tmp/config-4.15-rc7 .
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-10 11:49 ` [mm? 4.15-rc7] Random oopses " Tetsuo Handa
@ 2018-01-10 12:45 ` Michal Hocko
2018-01-10 13:37 ` Tetsuo Handa
0 siblings, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2018-01-10 12:45 UTC (permalink / raw)
To: Tetsuo Handa; +Cc: linux-mm
On Wed 10-01-18 20:49:56, Tetsuo Handa wrote:
> Tetsuo Handa wrote:
> > I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
> > So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
> > Does anyone know what is happening?
>
> I simplified the reproducer and succeeded to reproduce this bug with both
> i7-2630QM (8 core) and i5-4440S (4 core). Thus, I think that this bug is
> not architecture specific.
Can you see the same with 64b kernel?
It smells like a ref count imbalance and premature page free to me. Can
you try to bisect this?
[...]
> [ 100.847395] BUG: Bad page state in process a.out pfn:3e6a9
> [ 100.849515] page:f5d19a68 count:0 mapcount:9 mapping:f3d57c8c index:0x0
> [ 100.851773] flags: 0x3e010019(locked|uptodate|dirty|mappedtodisk)
> [ 100.853950] raw: 3e010019 f3d57c8c 00000000 00000008 00000000 00000100 00000200 00000000
> [ 100.856738] raw: 00000000 00000000
> [ 100.858510] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
> [ 100.860880] bad because of flags: 0x1(locked)
> [ 100.862685] Modules linked in: xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix mptbase libata
> [ 100.867413] CPU: 0 PID: 919 Comm: a.out Not tainted 4.15.0-rc7+ #196
> [ 100.869521] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
> [ 100.874256] Call Trace:
> [ 100.875664] dump_stack+0x58/0x76
> [ 100.878332] bad_page+0xc4/0x140
> [ 100.879911] free_pages_check_bad+0x5b/0x5e
> [ 100.882825] free_unref_page+0x149/0x160
> [ 100.884498] __put_page+0x2e/0x40
> [ 100.886089] try_to_unmap_one+0x37d/0x590
> [ 100.887839] rmap_walk_file+0x13c/0x250
> [ 100.890229] rmap_walk+0x32/0x60
> [ 100.893452] try_to_unmap+0x4d/0x100
> [ 100.895316] ? page_remove_rmap+0x2e0/0x2e0
> [ 100.897142] ? page_not_mapped+0x10/0x10
> [ 100.898815] ? page_get_anon_vma+0x80/0x80
> [ 100.900539] shrink_page_list+0x3a2/0x1000
> [ 100.902247] shrink_inactive_list+0x1b2/0x440
> [ 100.904431] shrink_node_memcg+0x34a/0x770
> [ 100.906126] shrink_node+0xbb/0x2e0
> [ 100.910555] do_try_to_free_pages+0xba/0x320
> [ 100.913526] try_to_free_pages+0x11d/0x330
> [ 100.915234] __alloc_pages_slowpath+0x303/0x6d9
> [ 100.916941] ? __accumulate_pelt_segments+0x32/0x50
> [ 100.918991] __alloc_pages_nodemask+0x16d/0x180
> [ 100.921866] do_anonymous_page+0xab/0x4f0
> [ 100.926874] handle_mm_fault+0x531/0x8d0
> [ 100.929028] ? __phys_addr+0x32/0x70
> [ 100.930829] ? load_new_mm_cr3+0x6a/0x90
> [ 100.932898] __do_page_fault+0x1ea/0x4d0
> [ 100.939138] ? __do_page_fault+0x4d0/0x4d0
> [ 100.944536] do_page_fault+0x1a/0x20
> [ 100.948193] common_exception+0x6f/0x76
> [ 100.951431] EIP: 0x8048437
> [ 100.952710] EFLAGS: 00010202 CPU: 0
> [ 100.954576] EAX: 01e7c000 EBX: 7ff00000 ECX: 39d6a008 EDX: 00000000
> [ 100.957048] ESI: 7ff00000 EDI: 00000000 EBP: bfc49318 ESP: bfc492e0
> [ 100.960794] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
> [ 100.965552] Disabling lock debugging due to kernel taint
> [ 100.967496] page:f5d19a68 count:0 mapcount:0 mapping:f3d57c8c index:0x0
> [ 100.969872] flags: 0x3e030078(uptodate|dirty|lru|active|mappedtodisk|reclaim)
> [ 100.976635] raw: 3e030078 f3d57c8c 00000000 ffffffff 00000000 f400f680 f400f680 00000000
> [ 100.979529] raw: 00000000 00000000
> [ 100.981347] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0)
> [ 100.983838] ------------[ cut here ]------------
> [ 100.986268] kernel BUG at ./include/linux/mm.h:483!
> [ 100.990886] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
> [ 100.993122] Modules linked in: xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix mptbase libata
> [ 100.997779] CPU: 0 PID: 919 Comm: a.out Tainted: G B 4.15.0-rc7+ #196
> [ 101.000477] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
> [ 101.004621] EIP: put_page_testzero.part.53+0xd/0xf
> [ 101.006692] EFLAGS: 00010086 CPU: 0
> [ 101.008524] EAX: 00000000 EBX: f400f668 ECX: c19dd988 EDX: 00000086
> [ 101.011584] ESI: f5d19a7c EDI: 00000003 EBP: f2e15bcc ESP: f2e15bcc
> [ 101.013808] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
> [ 101.015759] CR0: 80050033 CR2: 38a1e008 CR3: 3197e000 CR4: 001406d0
> [ 101.017934] Call Trace:
> [ 101.019235] putback_inactive_pages+0x384/0x3c0
> [ 101.021417] shrink_inactive_list+0x205/0x440
> [ 101.023592] shrink_node_memcg+0x34a/0x770
> [ 101.025453] shrink_node+0xbb/0x2e0
> [ 101.027097] do_try_to_free_pages+0xba/0x320
> [ 101.028729] try_to_free_pages+0x11d/0x330
> [ 101.030419] __alloc_pages_slowpath+0x303/0x6d9
> [ 101.032194] ? __accumulate_pelt_segments+0x32/0x50
> [ 101.034161] __alloc_pages_nodemask+0x16d/0x180
> [ 101.035866] do_anonymous_page+0xab/0x4f0
> [ 101.037464] handle_mm_fault+0x531/0x8d0
> [ 101.039042] ? __phys_addr+0x32/0x70
> [ 101.041023] ? load_new_mm_cr3+0x6a/0x90
> [ 101.042784] __do_page_fault+0x1ea/0x4d0
> [ 101.044805] ? __do_page_fault+0x4d0/0x4d0
> [ 101.046503] do_page_fault+0x1a/0x20
> [ 101.048046] common_exception+0x6f/0x76
> [ 101.049637] EIP: 0x8048437
> [ 101.051098] EFLAGS: 00010202 CPU: 0
> [ 101.054732] EAX: 01e7c000 EBX: 7ff00000 ECX: 39d6a008 EDX: 00000000
> [ 101.058435] ESI: 7ff00000 EDI: 00000000 EBP: bfc49318 ESP: bfc492e0
> [ 101.062077] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
> [ 101.064576] Code: 55 ba 24 1a 7f c1 89 e5 e8 b2 2c 01 00 0f 0b 55 ba ec 2f 7f c1 89 e5 e8 a3 2c 01 00 0f 0b 55 ba a8 7b 7d c1 89 e5 e8 94 2c 01 00 <0f> 0b 55 ba b0 2e 7f c1 89 e5 e8 85 2c 01 00 0f 0b 90 90 90 55
> [ 101.074916] EIP: put_page_testzero.part.53+0xd/0xf SS:ESP: 0068:f2e15bcc
> [ 101.078189] ---[ end trace d34ac0a26b29ce39 ]---
> ----------
>
> ----------
> [ 30.998639] Out of memory: Kill process 330 (a.out) score 22 or sacrifice child
> [ 31.001106] Killed process 330 (a.out) total-vm:2099260kB, anon-rss:72524kB, file-rss:8kB, shmem-rss:0kB
> [ 31.042032] BUG: unable to handle kernel paging request at 3304001a
> [ 31.044552] IP: page_remove_rmap+0x14/0x2e0
> [ 31.046248] *pde = 00000000
> [ 31.047722] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
> [ 31.049977] Modules linked in: xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix mptbase libata
> [ 31.054476] CPU: 0 PID: 474 Comm: a.out Not tainted 4.15.0-rc7+ #196
> [ 31.056620] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
> [ 31.060414] EIP: page_remove_rmap+0x14/0x2e0
> [ 31.062110] EFLAGS: 00010202 CPU: 0
> [ 31.063896] EAX: 33040016 EBX: f3096c20 ECX: 0000015a EDX: 00000000
> [ 31.066797] ESI: 00000159 EDI: f48b92f8 EBP: f10cda9c ESP: f10cda94
> [ 31.069081] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
> [ 31.071035] CR0: 80050033 CR2: 3304001a CR3: 31100000 CR4: 001406d0
> [ 31.073261] Call Trace:
> [ 31.074534] try_to_unmap_one+0x20e/0x590
> [ 31.076124] rmap_walk_file+0x13c/0x250
> [ 31.077652] rmap_walk+0x32/0x60
> [ 31.079000] try_to_unmap+0x4d/0x100
> [ 31.080381] ? page_remove_rmap+0x2e0/0x2e0
> [ 31.082397] ? page_not_mapped+0x10/0x10
> [ 31.084272] ? page_get_anon_vma+0x80/0x80
> [ 31.085852] shrink_page_list+0x3a2/0x1000
> [ 31.087260] shrink_inactive_list+0x1b2/0x440
> [ 31.088728] shrink_node_memcg+0x34a/0x770
> [ 31.090099] shrink_node+0xbb/0x2e0
> [ 31.091376] do_try_to_free_pages+0xba/0x320
> [ 31.092908] try_to_free_pages+0x11d/0x330
> [ 31.094272] ? __wake_up+0x1a/0x20
> [ 31.095515] __alloc_pages_slowpath+0x303/0x6d9
> [ 31.097094] ? debug_object_activate+0x126/0x1d0
> [ 31.099222] ? ktime_get+0x47/0xf0
> [ 31.100605] __alloc_pages_nodemask+0x16d/0x180
> [ 31.102142] do_anonymous_page+0xab/0x4f0
> [ 31.103546] handle_mm_fault+0x531/0x8d0
> [ 31.104844] ? pick_next_task_fair+0xe1/0x490
> [ 31.106201] ? irq_exit+0x45/0xb0
> [ 31.107822] ? smp_apic_timer_interrupt+0x4b/0x80
> [ 31.109289] __do_page_fault+0x1ea/0x4d0
> [ 31.110631] ? __do_page_fault+0x4d0/0x4d0
> [ 31.112277] do_page_fault+0x1a/0x20
> [ 31.113990] common_exception+0x6f/0x76
> [ 31.115879] EIP: 0x8048437
> [ 31.117195] EFLAGS: 00010202 CPU: 0
> [ 31.118702] EAX: 00c4f000 EBX: 7ff00000 ECX: 38acc008 EDX: 00000000
> [ 31.120624] ESI: 7ff00000 EDI: 00000000 EBP: bf937a08 ESP: bf9379d0
> [ 31.122365] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
> [ 31.124045] Code: c4 fe ff ff ba ec 32 7f c1 89 d8 e8 97 e5 fe ff 0f 0b 83 e8 01 eb 8e 55 89 e5 56 53 89 c3 8b 40 14 a8 01 0f 85 be 01 00 00 89 d8 <f6> 40 04 01 74 5e 84 d2 0f 85 9e 00 00 00 3e 83 43 0c ff 78 07
> [ 31.129606] EIP: page_remove_rmap+0x14/0x2e0 SS:ESP: 0068:f10cda94
> [ 31.131409] CR2: 000000003304001a
> [ 31.133754] ---[ end trace bee2e936b4fd74fc ]---
> ----------
>
> I can hit this bug with CONFIG_SMP=y CONFIG_PREEMPT_NONE=y nosmp init=/bin/bash
> but so far I haven't hit this bug with CONFIG_SMP=n .
> Config is at http://I-love.SAKURA.ne.jp/tmp/config-4.15-rc7 .
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-10 12:45 ` Michal Hocko
@ 2018-01-10 13:37 ` Tetsuo Handa
2018-01-11 13:57 ` Michal Hocko
0 siblings, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-10 13:37 UTC (permalink / raw)
To: mhocko; +Cc: linux-mm
Michal Hocko wrote:
> On Wed 10-01-18 20:49:56, Tetsuo Handa wrote:
> > Tetsuo Handa wrote:
> > > I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
> > > So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
> > > Does anyone know what is happening?
> >
> > I simplified the reproducer and succeeded to reproduce this bug with both
> > i7-2630QM (8 core) and i5-4440S (4 core). Thus, I think that this bug is
> > not architecture specific.
>
> Can you see the same with 64b kernel?
No. I can hit this bug with only x86_32 kernels.
But if the cause is not specific to 32b, this might be silent memory corruption.
> It smells like a ref count imbalance and premature page free to me. Can
> you try to bisect this?
Too difficult to bisect, but at least I can hit this bug with 4.8+ kernels.
The XXX in "count:XXX mapcount:XXX mapping:XXX index:XXX" are rather random
as if they are overwritten.
[ 44.103192] page:5a5a0697 count:-1055023618 mapcount:-1055030029 mapping:26f4be11 index:0xc11d7c83
[ 44.103196] flags: 0xc10528fe(waiters|error|referenced|uptodate|dirty|lru|active|reserved|private_2|mappedtodisk|swapbacked)
[ 44.103200] raw: c10528fe c114fff7 c11d7c83 c11d84f2 c11d9dfe c11daa34 c11daaa0 c13e65df
[ 44.103201] raw: c13e4a1c c13e4c62
[ 44.103202] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) <= 0)
[ 44.103203] page->mem_cgroup:35401b27
[ 192.152510] BUG: Bad page state in process a.out pfn:18566
[ 192.152513] page:f72997f0 count:0 mapcount:8 mapping:f118f5a4 index:0x0
[ 192.152516] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
[ 192.152520] raw: 19010019 f118f5a4 00000000 00000007 00000000 f7299804 f7299804 00000000
[ 192.152521] raw: 00000000 00000000
[ 192.152521] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 192.152522] bad because of flags: 0x1(locked)
[ 77.872133] BUG: Bad page state in process a.out pfn:1873a
[ 77.872136] page:f729e110 count:0 mapcount:6 mapping:f1187224 index:0x0
[ 77.872138] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
[ 77.872141] raw: 19010019 f1187224 00000000 00000005 00000000 f729e124 f729e124 00000000
[ 77.872141] raw: 00000000 00000000
[ 77.872142] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 77.872142] bad because of flags: 0x1(locked)
[ 188.992549] BUG: Bad page state in process a.out pfn:197ea
[ 188.992551] page:f72c7c90 count:0 mapcount:12 mapping:f11b8ca4 index:0x0
[ 188.992554] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
[ 188.992557] raw: 19010019 f11b8ca4 00000000 0000000b 00000000 f72c7ca4 f72c7ca4 00000000
[ 188.992557] raw: 00000000 00000000
[ 188.992558] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 188.992559] bad because of flags: 0x1(locked)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-10 13:37 ` Tetsuo Handa
@ 2018-01-11 13:57 ` Michal Hocko
2018-01-11 14:11 ` Tetsuo Handa
0 siblings, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2018-01-11 13:57 UTC (permalink / raw)
To: Tetsuo Handa; +Cc: linux-mm
On Wed 10-01-18 22:37:52, Tetsuo Handa wrote:
> Michal Hocko wrote:
> > On Wed 10-01-18 20:49:56, Tetsuo Handa wrote:
> > > Tetsuo Handa wrote:
> > > > I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
> > > > So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
> > > > Does anyone know what is happening?
> > >
> > > I simplified the reproducer and succeeded to reproduce this bug with both
> > > i7-2630QM (8 core) and i5-4440S (4 core). Thus, I think that this bug is
> > > not architecture specific.
> >
> > Can you see the same with 64b kernel?
>
> No. I can hit this bug with only x86_32 kernels.
> But if the cause is not specific to 32b, this might be silent memory corruption.
>
> > It smells like a ref count imbalance and premature page free to me. Can
> > you try to bisect this?
>
> Too difficult to bisect, but at least I can hit this bug with 4.8+ kernels.
>
> The XXX in "count:XXX mapcount:XXX mapping:XXX index:XXX" are rather random
> as if they are overwritten.
>
> [ 44.103192] page:5a5a0697 count:-1055023618 mapcount:-1055030029 mapping:26f4be11 index:0xc11d7c83
Yes, this looks like somebody is clobbering the page. I've seen one with
refcount 0 so I though this would be a ref count issue. But the one
below looks definitely like a memory corruption. A nasty one to debug :/
All of those seem to be file pages. So maybe try to use a different FS.
> [ 44.103196] flags: 0xc10528fe(waiters|error|referenced|uptodate|dirty|lru|active|reserved|private_2|mappedtodisk|swapbacked)
> [ 44.103200] raw: c10528fe c114fff7 c11d7c83 c11d84f2 c11d9dfe c11daa34 c11daaa0 c13e65df
> [ 44.103201] raw: c13e4a1c c13e4c62
> [ 44.103202] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) <= 0)
> [ 44.103203] page->mem_cgroup:35401b27
>
> [ 192.152510] BUG: Bad page state in process a.out pfn:18566
> [ 192.152513] page:f72997f0 count:0 mapcount:8 mapping:f118f5a4 index:0x0
> [ 192.152516] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
> [ 192.152520] raw: 19010019 f118f5a4 00000000 00000007 00000000 f7299804 f7299804 00000000
> [ 192.152521] raw: 00000000 00000000
> [ 192.152521] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
> [ 192.152522] bad because of flags: 0x1(locked)
>
> [ 77.872133] BUG: Bad page state in process a.out pfn:1873a
> [ 77.872136] page:f729e110 count:0 mapcount:6 mapping:f1187224 index:0x0
> [ 77.872138] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
> [ 77.872141] raw: 19010019 f1187224 00000000 00000005 00000000 f729e124 f729e124 00000000
> [ 77.872141] raw: 00000000 00000000
> [ 77.872142] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
> [ 77.872142] bad because of flags: 0x1(locked)
>
> [ 188.992549] BUG: Bad page state in process a.out pfn:197ea
> [ 188.992551] page:f72c7c90 count:0 mapcount:12 mapping:f11b8ca4 index:0x0
> [ 188.992554] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
> [ 188.992557] raw: 19010019 f11b8ca4 00000000 0000000b 00000000 f72c7ca4 f72c7ca4 00000000
> [ 188.992557] raw: 00000000 00000000
> [ 188.992558] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
> [ 188.992559] bad because of flags: 0x1(locked)
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-11 13:57 ` Michal Hocko
@ 2018-01-11 14:11 ` Tetsuo Handa
2018-01-11 14:21 ` Michal Hocko
2018-01-11 18:11 ` [mm? 4.15-rc7] " Linus Torvalds
0 siblings, 2 replies; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-11 14:11 UTC (permalink / raw)
To: mhocko; +Cc: linux-mm, x86, linux-fsdevel
Michal Hocko wrote:
> On Wed 10-01-18 22:37:52, Tetsuo Handa wrote:
> > Michal Hocko wrote:
> > > On Wed 10-01-18 20:49:56, Tetsuo Handa wrote:
> > > > Tetsuo Handa wrote:
> > > > > I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
> > > > > So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
> > > > > Does anyone know what is happening?
> > > >
> > > > I simplified the reproducer and succeeded to reproduce this bug with both
> > > > i7-2630QM (8 core) and i5-4440S (4 core). Thus, I think that this bug is
> > > > not architecture specific.
> > >
> > > Can you see the same with 64b kernel?
> >
> > No. I can hit this bug with only x86_32 kernels.
> > But if the cause is not specific to 32b, this might be silent memory corruption.
> >
> > > It smells like a ref count imbalance and premature page free to me. Can
> > > you try to bisect this?
> >
> > Too difficult to bisect, but at least I can hit this bug with 4.8+ kernels.
The bug in 4.8 kernel might be different from the bug in 4.15-rc7 kernel.
4.15-rc7 kernel hits the bug so trivially.
----------------------------------------
[ 201.838763] Out of memory: Kill process 1131 (a.out) score 20 or sacrifice child
[ 201.841780] Killed process 1131 (a.out) total-vm:2099264kB, anon-rss:66060kB, file-rss:8kB, shmem-rss:0kB
[ 201.891026] ------------[ cut here ]------------
[ 201.893343] kernel BUG at ./include/linux/swap.h:276!
[ 201.895973] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 201.898707] Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp vmw_balloon ppdev pcspkr sg vmw_vmci i2c_piix4 shpchp parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
[ 201.924427] CPU: 0 PID: 3495 Comm: a.out Not tainted 4.8.0 #2
[ 201.927017] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 201.932381] task: e47d3240 task.stack: ec032000
[ 201.934796] EIP: 0060:[<c80f557e>] EFLAGS: 00010046 CPU: 0
[ 201.937460] EIP is at page_cache_tree_insert+0xbe/0xc0
[ 201.939766] EAX: f41663d8 EBX: f41f8930 ECX: 00000004 EDX: f41663f0
[ 201.942672] ESI: f5243b38 EDI: 00000000 EBP: ec033d24 ESP: ec033d04
[ 201.945483] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 201.948112] CR0: 80050033 CR2: b77792c9 CR3: 275d1000 CR4: 000406d0
[ 201.950866] Stack:
[ 201.952302] ec033d0c ec033d10 f41663d8 f41663f0 ceaaa77b f5243b38 00000000 f41f8930
[ 201.955563] ec033d54 c80f5ef6 ec033d40 00000000 ec033d64 00000000 f41f8930 f44192e8
[ 201.958862] ceaaa77b f5243b38 0242134a f5243b38 ec033d78 c80f61dd 0242134a ec033d64
[ 201.962307] Call Trace:
[ 201.964005] [<c80f5ef6>] __add_to_page_cache_locked+0xb6/0x1e0
[ 201.966584] [<c80f61dd>] add_to_page_cache_lru+0x4d/0xf0
[ 201.968890] [<c8198f6b>] mpage_readpages+0xcb/0x170
[ 201.971104] [<c80ff2e3>] ? __alloc_pages_nodemask+0x243/0x950
[ 201.973560] [<f8ae8050>] ? __xfs_get_blocks+0x690/0x690 [xfs]
[ 201.975997] [<c826713e>] ? __radix_tree_lookup+0x6e/0xd0
[ 201.978531] [<f8ae7070>] ? xfs_vm_bmap+0x60/0x60 [xfs]
[ 201.981085] [<f8ae7089>] xfs_vm_readpages+0x19/0x20 [xfs]
[ 201.983488] [<f8ae8050>] ? __xfs_get_blocks+0x690/0x690 [xfs]
[ 201.985931] [<c8103b10>] __do_page_cache_readahead+0x1a0/0x260
[ 201.988386] [<c80f959d>] filemap_fault+0x31d/0x540
[ 201.990536] [<c8266135>] ? radix_tree_next_chunk+0xe5/0x2e0
[ 201.992916] [<f8af1dfb>] xfs_filemap_fault+0x2b/0x40 [xfs]
[ 201.995335] [<c811fca5>] __do_fault+0x65/0xe0
[ 201.997574] [<c80f6a20>] ? find_lock_entry+0xb0/0xb0
[ 201.999932] [<c8124a30>] handle_mm_fault+0x810/0xa30
[ 202.002104] [<c80f6a20>] ? find_lock_entry+0xb0/0xb0
[ 202.004240] [<c8044519>] __do_page_fault+0x199/0x470
[ 202.006396] [<c80447f0>] ? __do_page_fault+0x470/0x470
[ 202.008554] [<c804480a>] do_page_fault+0x1a/0x20
[ 202.010537] [<c85e1727>] error_code+0x67/0x6c
[ 202.012611] Code: 49 7f 02 00 31 c0 8b 5d f0 65 33 1d 14 00 00 00 75 12 83 c4 14 5b 5e 5f 5d c3 8d 76 00 b8 ef ff ff ff eb e2 e8 b4 b3 f5 ff 0f 0b <0f> 0b 55 89 e5 83 ec 28 65 8b 0d 14 00 00 00 89 4d fc 31 c9 c7
[ 202.022555] EIP: [<c80f557e>] page_cache_tree_insert+0xbe/0xc0 SS:ESP 0068:ec033d04
[ 202.025375] ---[ end trace 0a3a1155cbc67d7f ]---
----------------------------------------
> >
> > The XXX in "count:XXX mapcount:XXX mapping:XXX index:XXX" are rather random
> > as if they are overwritten.
> >
> > [ 44.103192] page:5a5a0697 count:-1055023618 mapcount:-1055030029 mapping:26f4be11 index:0xc11d7c83
>
> Yes, this looks like somebody is clobbering the page. I've seen one with
> refcount 0 so I though this would be a ref count issue. But the one
> below looks definitely like a memory corruption. A nasty one to debug :/
>
> All of those seem to be file pages. So maybe try to use a different FS.
Maybe that's the next thing I should try.
>
> > [ 44.103196] flags: 0xc10528fe(waiters|error|referenced|uptodate|dirty|lru|active|reserved|private_2|mappedtodisk|swapbacked)
> > [ 44.103200] raw: c10528fe c114fff7 c11d7c83 c11d84f2 c11d9dfe c11daa34 c11daaa0 c13e65df
> > [ 44.103201] raw: c13e4a1c c13e4c62
> > [ 44.103202] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) <= 0)
> > [ 44.103203] page->mem_cgroup:35401b27
> >
> > [ 192.152510] BUG: Bad page state in process a.out pfn:18566
> > [ 192.152513] page:f72997f0 count:0 mapcount:8 mapping:f118f5a4 index:0x0
> > [ 192.152516] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
> > [ 192.152520] raw: 19010019 f118f5a4 00000000 00000007 00000000 f7299804 f7299804 00000000
> > [ 192.152521] raw: 00000000 00000000
> > [ 192.152521] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
> > [ 192.152522] bad because of flags: 0x1(locked)
> >
> > [ 77.872133] BUG: Bad page state in process a.out pfn:1873a
> > [ 77.872136] page:f729e110 count:0 mapcount:6 mapping:f1187224 index:0x0
> > [ 77.872138] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
> > [ 77.872141] raw: 19010019 f1187224 00000000 00000005 00000000 f729e124 f729e124 00000000
> > [ 77.872141] raw: 00000000 00000000
> > [ 77.872142] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
> > [ 77.872142] bad because of flags: 0x1(locked)
> >
> > [ 188.992549] BUG: Bad page state in process a.out pfn:197ea
> > [ 188.992551] page:f72c7c90 count:0 mapcount:12 mapping:f11b8ca4 index:0x0
> > [ 188.992554] flags: 0x19010019(locked|uptodate|dirty|mappedtodisk)
> > [ 188.992557] raw: 19010019 f11b8ca4 00000000 0000000b 00000000 f72c7ca4 f72c7ca4 00000000
> > [ 188.992557] raw: 00000000 00000000
> > [ 188.992558] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
> > [ 188.992559] bad because of flags: 0x1(locked)
>
I retested with some debug printk() patch.
Complete log is at http://I-love.SAKURA.ne.jp/tmp/serial-4.15-rc7.zip
----------------------------------------
diff --git a/kernel/exit.c b/kernel/exit.c
index 995453d..baa7cea 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -765,6 +765,7 @@ void __noreturn do_exit(long code)
struct task_struct *tsk = current;
int group_dead;
+ pr_info("exit(%s/%u): started do_exit()\n", tsk->comm, tsk->pid);
profile_task_exit(tsk);
kcov_task_exit(tsk);
@@ -808,7 +809,9 @@ void __noreturn do_exit(long code)
schedule();
}
+ pr_info("exit(%s/%u): started exit_signals()\n", tsk->comm, tsk->pid);
exit_signals(tsk); /* sets PF_EXITING */
+ pr_info("exit(%s/%u): ended exit_signals()\n", tsk->comm, tsk->pid);
/*
* Ensure that all new tsk->pi_lock acquisitions must observe
* PF_EXITING. Serializes against futex.c:attach_to_pi_owner().
@@ -829,6 +832,7 @@ void __noreturn do_exit(long code)
}
/* sync mm's RSS info before statistics gathering */
+ pr_info("exit(%s/%u): started sync_mm_rss()\n", tsk->comm, tsk->pid);
if (tsk->mm)
sync_mm_rss(tsk->mm);
acct_update_integrals(tsk);
@@ -849,8 +853,10 @@ void __noreturn do_exit(long code)
tsk->exit_code = code;
taskstats_exit(tsk, group_dead);
+ pr_info("exit(%s/%u): started exit_mm()\n", tsk->comm, tsk->pid);
exit_mm();
-
+ pr_info("exit(%s/%u): ended exit_mm()\n", tsk->comm, tsk->pid);
+
if (group_dead)
acct_process();
trace_sched_process_exit(tsk);
@@ -919,6 +925,7 @@ void __noreturn do_exit(long code)
exit_tasks_rcu_finish();
lockdep_free_task(tsk);
+ pr_info("exit(%s/%u): ended do_exit()\n", tsk->comm, tsk->pid);
do_task_dead();
}
EXPORT_SYMBOL_GPL(do_exit);
diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 29f8555..8bd3f25 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -505,6 +505,7 @@ static bool __oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm)
* # no TIF_MEMDIE task selects new victim
* unmap_page_range # frees some memory
*/
+ pr_info("oom_reaper: started reaping\n");
mutex_lock(&oom_lock);
if (!down_read_trylock(&mm->mmap_sem)) {
----------------------------------------
----------------------------------------
[ 38.988178] Out of memory: Kill process 354 (b.out) score 7 or sacrifice child
[ 38.991145] Killed process 354 (b.out) total-vm:2099260kB, anon-rss:23288kB, file-rss:8kB, shmem-rss:0kB
[ 38.996277] oom_reaper: started reaping
[ 38.999033] BUG: unable to handle kernel paging request at c130d86d
[ 39.001802] IP: _raw_spin_lock_irqsave+0x1c/0x40
[ 39.004069] *pde = 01f88063 *pte = 0130d161
[ 39.006250] Oops: 0003 [#1] SMP DEBUG_PAGEALLOC
[ 39.008779] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi e1000 scsi_transport_spi mptscsih ata_piix mptbase libata
[ 39.014942] CPU: 0 PID: 586 Comm: b.out Tainted: G W 4.15.0-rc7+ #303
[ 39.018014] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 39.022885] EIP: _raw_spin_lock_irqsave+0x1c/0x40
[ 39.025233] EFLAGS: 00010046 CPU: 0
[ 39.027220] EAX: 00000000 EBX: 00000001 ECX: c130d86d EDX: 00000000
[ 39.030090] ESI: 00000202 EDI: f3102100 EBP: ef6dda78 ESP: ef6dda70
[ 39.032714] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 39.035214] CR0: 80050033 CR2: c130d86d CR3: 2f726000 CR4: 000406d0
[ 39.037889] Call Trace:
[ 39.039470] ? acpi_ps_parse_aml+0x85/0x25c
[ 39.041542] ? acpi_ps_parse_aml+0x1f9/0x25c
[ 39.043562] lock_page_memcg+0x25/0x80
[ 39.045421] page_remove_rmap+0x87/0x2e0
[ 39.047315] try_to_unmap_one+0x20e/0x590
[ 39.049198] rmap_walk_file+0x13c/0x250
[ 39.051012] rmap_walk+0x32/0x60
[ 39.052619] try_to_unmap+0x4d/0x100
[ 39.054316] ? page_remove_rmap+0x2e0/0x2e0
[ 39.056196] ? page_not_mapped+0x10/0x10
[ 39.058001] ? page_get_anon_vma+0x80/0x80
[ 39.059849] shrink_page_list+0x3a2/0x1000
[ 39.061678] shrink_inactive_list+0x1b2/0x440
[ 39.063539] shrink_node_memcg+0x34a/0x770
[ 39.065297] shrink_node+0xbb/0x2e0
[ 39.066920] do_try_to_free_pages+0xba/0x320
[ 39.068752] try_to_free_pages+0x11d/0x330
[ 39.070499] ? __wake_up+0x1a/0x20
[ 39.072084] __alloc_pages_slowpath+0x303/0x6d9
[ 39.073910] ? __accumulate_pelt_segments+0x32/0x50
[ 39.075932] __alloc_pages_nodemask+0x16d/0x180
[ 39.077809] do_anonymous_page+0xab/0x4f0
[ 39.079551] handle_mm_fault+0x531/0x8d0
[ 39.081219] ? __phys_addr+0x32/0x70
[ 39.082797] ? load_new_mm_cr3+0x6a/0x90
[ 39.084422] __do_page_fault+0x1ea/0x4d0
[ 39.086034] ? __do_page_fault+0x4d0/0x4d0
[ 39.087666] do_page_fault+0x1a/0x20
[ 39.089184] common_exception+0x6f/0x76
[ 39.090708] EIP: 0x8048437
[ 39.091964] EFLAGS: 00010202 CPU: 0
[ 39.093435] EAX: 0018d000 EBX: 7ff00000 ECX: 38007008 EDX: 00000000
[ 39.095587] ESI: 7ff00000 EDI: 00000000 EBP: bf9f5e98 ESP: bf9f5e60
[ 39.097735] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 39.099687] Code: eb f2 8d b6 00 00 00 00 8d bc 27 00 00 00 00 55 89 c1 89 e5 56 53 9c 58 66 66 66 90 89 c6 fa 66 66 90 66 90 31 c0 bb 01 00 00 00 <3e> 0f b1 19 85 c0 75 06 89 f0 5b 5e 5d c3 89 c2 89 c8 e8 ad 4b
[ 39.106164] EIP: _raw_spin_lock_irqsave+0x1c/0x40 SS:ESP: 0068:ef6dda70
[ 39.108487] CR2: 00000000c130d86d
[ 39.109981] ---[ end trace c89b8f16688d25d1 ]---
[ 39.111763] exit(b.out/586): started do_exit()
(...snipped...)
[ 39.167242] exit(b.out/586): started exit_signals()
[ 39.169364] exit(b.out/586): ended exit_signals()
[ 39.171473] exit(b.out/586): started sync_mm_rss()
[ 39.173549] exit(b.out/586): started exit_mm()
[ 39.181703] exit(b.out/354): started do_exit()
[ 39.183740] exit(b.out/354): started exit_signals()
[ 39.185798] exit(b.out/354): ended exit_signals()
[ 39.187822] exit(b.out/354): started sync_mm_rss()
[ 39.189813] exit(b.out/354): started exit_mm()
----------------------------------------
cccccccc sounds like a poison.
----------------------------------------
[ 64.763544] Out of memory: Kill process 645 (b.out) score 17 or sacrifice child
[ 64.766900] Killed process 645 (b.out) total-vm:2099260kB, anon-rss:56292kB, file-rss:8kB, shmem-rss:0kB
[ 64.772670] oom_reaper: started reaping
[ 64.786642] exit(b.out/645): started do_exit()
[ 64.789145] exit(b.out/645): started exit_signals()
[ 64.791624] exit(b.out/645): ended exit_signals()
[ 64.794173] exit(b.out/645): started sync_mm_rss()
[ 64.796730] exit(b.out/645): started exit_mm()
[ 64.822147] exit(b.out/859): ended exit_mm()
[ 64.834061] exit(b.out/859): ended do_exit()
[ 64.920253] exit(b.out/645): ended exit_mm()
[ 64.950084] exit(b.out/645): ended do_exit()
[ 64.980290] WARNING: CPU: 0 PID: 696 at arch/x86/mm/tlb.c:576 flush_tlb_mm_range+0xfe/0x110
[ 64.983665] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi e1000 scsi_transport_spi mptscsih mptbase ata_piix libata
[ 64.989842] CPU: 0 PID: 696 Comm: b.out Tainted: G W 4.15.0-rc7+ #303
[ 64.993148] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 64.998050] EIP: flush_tlb_mm_range+0xfe/0x110
[ 65.000282] EFLAGS: 00010046 CPU: 0
[ 65.002171] EAX: 00000046 EBX: f051ac40 ECX: 00000000 EDX: 00000000
[ 65.005030] ESI: f051adc0 EDI: 74002000 EBP: f15778d8 ESP: f15778ac
[ 65.007793] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 65.010390] CR0: 80050033 CR2: 740010bf CR3: 2fee1000 CR4: 000406d0
[ 65.013060] Call Trace:
[ 65.014621] ptep_clear_flush+0x48/0x60
[ 65.016480] wp_page_copy+0x23d/0x650
[ 65.018430] do_wp_page+0x82/0x440
[ 65.020263] handle_mm_fault+0x522/0x8d0
[ 65.022251] __do_page_fault+0x1ea/0x4d0
[ 65.024281] ? __do_page_fault+0x4d0/0x4d0
[ 65.026208] do_page_fault+0x1a/0x20
[ 65.028043] common_exception+0x6f/0x76
[ 65.029925] EIP: mem_cgroup_page_lruvec+0x30/0x40
[ 65.032134] EFLAGS: 00010082 CPU: 0
[ 65.033953] EAX: 7400107f EBX: c18ede80 ECX: c18ee540 EDX: c18ed840
[ 65.036746] ESI: f317a8f8 EDI: f48a4650 EBP: f1577a5c ESP: f1577a5c
[ 65.039399] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 65.041890] isolate_lru_page+0x70/0x260
[ 65.043917] clear_page_mlock+0x71/0xc0
[ 65.045619] page_remove_rmap+0x2a9/0x2e0
[ 65.047665] try_to_unmap_one+0x20e/0x590
[ 65.049707] rmap_walk_file+0x13c/0x250
[ 65.051596] rmap_walk+0x32/0x60
[ 65.053326] try_to_unmap+0x4d/0x100
[ 65.055154] ? page_remove_rmap+0x2e0/0x2e0
[ 65.057403] ? page_not_mapped+0x10/0x10
[ 65.059331] ? page_get_anon_vma+0x80/0x80
[ 65.061324] shrink_page_list+0x3a2/0x1000
[ 65.063292] shrink_inactive_list+0x1b2/0x440
[ 65.065356] shrink_node_memcg+0x34a/0x770
[ 65.067326] shrink_node+0xbb/0x2e0
[ 65.069117] do_try_to_free_pages+0xba/0x320
[ 65.071107] try_to_free_pages+0x11d/0x330
[ 65.072968] ? __wake_up+0x1a/0x20
[ 65.074681] __alloc_pages_slowpath+0x303/0x6d9
[ 65.076685] __alloc_pages_nodemask+0x16d/0x180
[ 65.078661] do_anonymous_page+0xab/0x4f0
[ 65.080485] handle_mm_fault+0x531/0x8d0
[ 65.082211] ? pick_next_task_fair+0xe1/0x490
[ 65.084092] ? irq_exit+0x45/0xb0
[ 65.085644] ? smp_apic_timer_interrupt+0x4b/0x80
[ 65.087729] __do_page_fault+0x1ea/0x4d0
[ 65.089568] ? __do_page_fault+0x4d0/0x4d0
[ 65.091463] do_page_fault+0x1a/0x20
[ 65.093168] common_exception+0x6f/0x76
[ 65.094898] EIP: 0x8048437
[ 65.096248] EFLAGS: 00010202 CPU: 0
[ 65.097863] EAX: 03667000 EBX: 7ff00000 ECX: 3b557008 EDX: 00000000
[ 65.100286] ESI: 7ff00000 EDI: 00000000 EBP: bf8212e8 ESP: bf8212b0
[ 65.102815] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 65.105050] Code: 90 8d 74 26 00 9c 58 66 66 66 90 f6 c4 02 74 1b fa 66 66 90 66 90 8d 45 dc e8 4f fb ff ff fb 66 66 90 66 90 eb 81 e8 12 57 00 00 <0f> ff eb e1 8d b4 26 00 00 00 00 8d bc 27 00 00 00 00 55 b9 01
[ 65.112209] ---[ end trace c89b8f16688d25d1 ]---
[ 65.114373] page:f317a8f8 count:-216553848 mapcount:-217582360 mapping:0100060f index:0x36414550
[ 65.117522] flags: 0xf2c75628(uptodate|lru|owner_priv_1|arch_1|private|writeback|mappedtodisk|reclaim|swapbacked)
[ 65.122013] raw: f2c75628 0100060f 36414550 f307f4e7 f317a688 f317c688 cccccccc f317a828
[ 65.125340] raw: c1308381 c1150c19
[ 65.127185] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) <= 0)
[ 65.130063] page->mem_cgroup:c1308381
[ 65.131938] There is not page extension available.
[ 65.134188] ------------[ cut here ]------------
[ 65.136372] kernel BUG at ./include/linux/mm.h:844!
[ 65.138762] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 65.141176] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi e1000 scsi_transport_spi mptscsih mptbase ata_piix libata
[ 65.147105] CPU: 0 PID: 696 Comm: b.out Tainted: G W 4.15.0-rc7+ #303
[ 65.150162] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 65.155193] EIP: isolate_lru_page+0x234/0x260
[ 65.157420] EFLAGS: 00010096 CPU: 0
[ 65.159268] EAX: 00000000 EBX: c18ede80 ECX: c1a2e988 EDX: 00000082
[ 65.162161] ESI: f317a8f8 EDI: 7400107f EBP: f1577a80 ESP: f1577a64
[ 65.165020] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 65.167742] CR0: 80050033 CR2: 740010bf CR3: 2fee1000 CR4: 000406d0
[ 65.170715] Call Trace:
[ 65.172504] clear_page_mlock+0x71/0xc0
[ 65.174751] page_remove_rmap+0x2a9/0x2e0
[ 65.176915] try_to_unmap_one+0x20e/0x590
[ 65.179087] rmap_walk_file+0x13c/0x250
[ 65.181198] rmap_walk+0x32/0x60
[ 65.183229] try_to_unmap+0x4d/0x100
[ 65.185211] ? page_remove_rmap+0x2e0/0x2e0
[ 65.187407] ? page_not_mapped+0x10/0x10
[ 65.189516] ? page_get_anon_vma+0x80/0x80
[ 65.191681] shrink_page_list+0x3a2/0x1000
[ 65.193901] shrink_inactive_list+0x1b2/0x440
[ 65.196125] shrink_node_memcg+0x34a/0x770
[ 65.198331] shrink_node+0xbb/0x2e0
[ 65.200235] do_try_to_free_pages+0xba/0x320
[ 65.202464] try_to_free_pages+0x11d/0x330
[ 65.204527] ? __wake_up+0x1a/0x20
[ 65.206434] __alloc_pages_slowpath+0x303/0x6d9
[ 65.208569] __alloc_pages_nodemask+0x16d/0x180
[ 65.210620] do_anonymous_page+0xab/0x4f0
[ 65.212610] handle_mm_fault+0x531/0x8d0
[ 65.214611] ? pick_next_task_fair+0xe1/0x490
[ 65.216640] ? irq_exit+0x45/0xb0
[ 65.218250] ? smp_apic_timer_interrupt+0x4b/0x80
[ 65.220496] __do_page_fault+0x1ea/0x4d0
[ 65.222465] ? __do_page_fault+0x4d0/0x4d0
[ 65.224285] do_page_fault+0x1a/0x20
[ 65.226110] common_exception+0x6f/0x76
[ 65.227912] EIP: 0x8048437
[ 65.229483] EFLAGS: 00010202 CPU: 0
[ 65.231535] EAX: 03667000 EBX: 7ff00000 ECX: 3b557008 EDX: 00000000
[ 65.234106] ESI: 7ff00000 EDI: 00000000 EBP: bf8212e8 ESP: bf8212b0
[ 65.236517] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 65.238687] Code: fe ff ff 83 e8 01 e9 6b fe ff ff ba 9c 23 7f c1 89 f0 e8 00 67 01 00 0f 0b 83 e8 01 e9 ac fe ff ff ba c4 83 7d c1 e8 ec 66 01 00 <0f> 0b 83 e8 01 e9 b4 fe ff ff 89 f0 e8 1c 3a 00 00 83 e8 01 e9
[ 65.245838] EIP: isolate_lru_page+0x234/0x260 SS:ESP: 0068:f1577a64
[ 65.248344] ---[ end trace c89b8f16688d25d2 ]---
[ 65.250488] exit(b.out/696): started do_exit()
----------------------------------------
This bug can occur before the OOM killer is invoked.
Thus, at least this is not a race with the OOM reaper.
----------------------------------------
[ 92.247160] exit(b.out/1893): started do_exit()
[ 92.249129] exit(b.out/1893): started exit_signals()
[ 92.250896] exit(b.out/1893): ended exit_signals()
[ 92.252621] exit(b.out/1893): started sync_mm_rss()
[ 92.254339] exit(b.out/1893): started exit_mm()
[ 92.397291] exit(b.out/1894): started do_exit()
[ 92.398984] exit(b.out/1894): started exit_signals()
[ 92.400974] exit(b.out/1894): ended exit_signals()
[ 92.402792] exit(b.out/1894): started sync_mm_rss()
[ 92.404595] exit(b.out/1894): started exit_mm()
[ 92.407241] exit(b.out/1830): started do_exit()
[ 92.409055] exit(b.out/1830): started exit_signals()
[ 92.410958] exit(b.out/1830): ended exit_signals()
[ 92.412649] exit(b.out/1830): started sync_mm_rss()
[ 92.414442] exit(b.out/1830): started exit_mm()
[ 92.623678] BUG: unable to handle kernel paging request at 89c08510
[ 92.625824] IP: debug_object_deactivate.part.4+0x5b/0x110
[ 92.628103] *pde = 00000000
[ 92.629531] Thread overran stack, or stack corrupted
[ 92.631420] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 92.633204] Modules linked in: xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih ata_piix e1000 mptbase libata
[ 92.638390] CPU: 0 PID: 32 Comm: kswapd0 Tainted: G W 4.15.0-rc7+ #303
[ 92.641071] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 92.645127] EIP: debug_object_deactivate.part.4+0x5b/0x110
[ 92.647368] EFLAGS: 00010006 CPU: 0
[ 92.649021] EAX: 89c08500 EBX: f7430240 ECX: 00000003 EDX: 00000005
[ 92.651385] ESI: c1b435b4 EDI: 00000426 EBP: f29034e8 ESP: f29034b8
[ 92.653810] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 92.655974] CR0: 80050033 CR2: 89c08510 CR3: 01a1f000 CR4: 000406d0
[ 92.658504] Call Trace:
[ 92.659865] Code: e8 bb 81 3b 00 89 c1 8b 04 fd 80 14 b4 c1 85 c0 0f 84 ba 00 00 00 3b 58 10 74 65 ba 01 00 00 00 eb 0e 8d b6 00 00 00 00 83 c2 01 <3b> 58 10 74 50 8b 00 85 c0 75 f2 39 15 c4 67 90 c1 7c 3a 31 c0
[ 92.667155] EIP: debug_object_deactivate.part.4+0x5b/0x110 SS:ESP: 0068:f29034b8
[ 92.669965] CR2: 0000000089c08510
[ 92.671657] ---[ end trace c89b8f16688d25d1 ]---
[ 92.673717] Kernel panic - not syncing: Fatal exception in interrupt
[ 92.676240] Kernel Offset: 0x0 from 0xc1000000 (relocation range: 0xc0000000-0xf7ffdfff)
[ 92.679311] Rebooting in 5 seconds..
[ 97.758198] ACPI MEMORY or I/O RESET_REG.
----------------------------------------
----------------------------------------
[ 13.672160] Out of memory: Kill process 395 (b.out) score 8 or sacrifice child
[ 13.708418] Killed process 395 (b.out) total-vm:2099260kB, anon-rss:27776kB, file-rss:28kB, shmem-rss:0kB
[ 13.713193] oom_reaper: started reaping
[ 13.716851] exit(b.out/395): started do_exit()
[ 13.719041] exit(b.out/395): started exit_signals()
[ 13.721397] exit(b.out/395): ended exit_signals()
[ 13.756917] exit(b.out/395): started sync_mm_rss()
[ 13.759208] exit(b.out/395): started exit_mm()
[ 13.814381] exit(b.out/599): ended exit_mm()
[ 13.821208] exit(b.out/601): ended exit_mm()
[ 13.863622] exit(b.out/599): ended do_exit()
[ 14.018589] BUG: unable to handle kernel paging request at 32d0a01a
[ 14.021835] IP: page_remove_rmap+0x17/0x280
[ 14.057429] *pde = 00000000
[ 14.059233] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 14.061456] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi e1000 mptspi scsi_transport_spi mptscsih ata_piix mptbase libata serio_raw
[ 14.067501] CPU: 0 PID: 30 Comm: kswapd0 Tainted: G W 4.15.0-rc7+ #304
[ 14.070580] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 14.108859] EIP: page_remove_rmap+0x17/0x280
[ 14.110748] EFLAGS: 00010202 CPU: 0
[ 14.112417] EAX: 32d0a016 EBX: f2d5dfa8 ECX: 00000010 EDX: 00000000
[ 14.114868] ESI: 0000000f EDI: f4931d60 EBP: f2c53c48 ESP: f2c53c3c
[ 14.117266] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 14.119394] CR0: 80050033 CR2: 32d0a01a CR3: 01a06000 CR4: 000406d0
[ 14.121795] Call Trace:
[ 14.156545] try_to_unmap_one+0x206/0x540
[ 14.158371] rmap_walk_file+0xf0/0x1e0
[ 14.160080] rmap_walk+0x32/0x60
[ 14.161606] try_to_unmap+0x4d/0xd0
[ 14.163176] ? page_remove_rmap+0x280/0x280
[ 14.164931] ? page_not_mapped+0x10/0x10
[ 14.166600] ? page_get_anon_vma+0x80/0x80
[ 14.168351] shrink_page_list+0x3e2/0xe80
[ 14.170176] shrink_inactive_list+0x1b2/0x440
[ 14.172137] shrink_node_memcg+0x34a/0x770
[ 14.207197] shrink_node+0xbb/0x2e0
[ 14.208761] kswapd+0x23f/0x5b0
[ 14.210220] kthread+0xd1/0x100
[ 14.211637] ? mem_cgroup_shrink_node+0xa0/0xa0
[ 14.213514] ? kthread_associate_blkcg+0x80/0x80
[ 14.215311] ret_from_fork+0x19/0x24
[ 14.216836] Code: ff ff 83 e8 01 e9 4f ff ff ff 8d 76 00 8d bc 27 00 00 00 00 55 89 e5 56 53 89 c3 83 ec 04 8b 40 14 a8 01 0f 85 18 02 00 00 89 d8 <f6> 40 04 01 74 6b 84 d2 0f 85 5b 01 00 00 3e 83 43 0c ff 78 0c
[ 14.257635] EIP: page_remove_rmap+0x17/0x280 SS:ESP: 0068:f2c53c3c
[ 14.260467] CR2: 0000000032d0a01a
[ 14.262535] ---[ end trace c89b8f16688d25d1 ]---
[ 14.264713] exit(kswapd0/30): started do_exit()
----------------------------------------
"Bad rss-counter state" messages can appear before the OOM killer is invoked.
----------------------------------------
[ 17.057515] exit(b.out/661): started do_exit()
[ 17.059220] exit(b.out/661): started exit_signals()
[ 17.060974] exit(b.out/661): ended exit_signals()
[ 17.062663] exit(b.out/661): started sync_mm_rss()
[ 17.064455] exit(b.out/661): started exit_mm()
[ 17.311701] exit(b.out/664): started do_exit()
[ 17.313880] exit(b.out/664): started exit_signals()
[ 17.315714] exit(b.out/664): ended exit_signals()
[ 17.317471] exit(b.out/664): started sync_mm_rss()
[ 17.352606] exit(b.out/664): started exit_mm()
[ 17.807363] exit(b.out/661): ended exit_mm()
[ 17.810459] exit(b.out/659): ended exit_mm()
[ 17.819504] exit(b.out/659): ended do_exit()
[ 17.855075] exit(b.out/661): ended do_exit()
[ 17.863491] BUG: Bad rss-counter state mm:3a10ce4a idx:0 val:1
[ 17.865642] BUG: Bad rss-counter state mm:3a10ce4a idx:1 val:-1
[ 17.867663] exit(b.out/664): ended exit_mm()
[ 17.917000] exit(b.out/664): ended do_exit()
[ 17.961639] BUG: unable to handle kernel paging request at 0e200f90
[ 17.963906] IP: lock_page_memcg+0x3c/0x80
[ 17.965528] *pde = 00000000
[ 17.966811] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 17.968543] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi e1000 mptspi scsi_transport_spi mptscsih ata_piix mptbase libata serio_raw
[ 18.006806] CPU: 0 PID: 367 Comm: b.out Tainted: G W 4.15.0-rc7+ #304
[ 18.009450] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 18.013417] EIP: lock_page_memcg+0x3c/0x80
[ 18.015138] EFLAGS: 00010202 CPU: 0
[ 18.016746] EAX: f2db0b90 EBX: 0e200e20 ECX: 00000011 EDX: 00000000
[ 18.052561] ESI: 00000010 EDI: f2db0b90 EBP: f2c2fab4 ESP: f2c2faa8
[ 18.054954] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 18.057013] CR0: 80050033 CR2: 0e200f90 CR3: 3298f000 CR4: 000406d0
[ 18.059343] Call Trace:
[ 18.060655] page_remove_rmap+0x92/0x280
[ 18.062360] try_to_unmap_one+0x206/0x540
[ 18.064109] rmap_walk_file+0xf0/0x1e0
[ 18.065839] rmap_walk+0x32/0x60
[ 18.067475] try_to_unmap+0x4d/0xd0
[ 18.102537] ? page_remove_rmap+0x280/0x280
[ 18.104313] ? page_not_mapped+0x10/0x10
[ 18.106001] ? page_get_anon_vma+0x80/0x80
[ 18.107680] shrink_page_list+0x3e2/0xe80
[ 18.109338] shrink_inactive_list+0x1b2/0x440
[ 18.111074] shrink_node_memcg+0x34a/0x770
[ 18.112743] shrink_node+0xbb/0x2e0
[ 18.114255] do_try_to_free_pages+0xb2/0x300
[ 18.116057] try_to_free_pages+0x20b/0x330
[ 18.117777] __alloc_pages_slowpath+0x2fb/0x6d3
[ 18.152982] ? hrtimer_interrupt+0x9e/0x170
[ 18.154768] ? irq_exit+0x45/0xb0
[ 18.156268] ? smp_apic_timer_interrupt+0x4b/0x80
[ 18.158126] ? __pagevec_lru_add_fn+0xdb/0x190
[ 18.159906] __alloc_pages_nodemask+0x14a/0x170
[ 18.161797] handle_mm_fault+0x5a5/0xcd0
[ 18.163515] ? pick_next_task_fair+0xe1/0x490
[ 18.165358] __do_page_fault+0x1ea/0x4d0
[ 18.167043] ? __do_page_fault+0x4d0/0x4d0
[ 18.202081] do_page_fault+0x1a/0x20
[ 18.203600] common_exception+0x6f/0x76
[ 18.205187] EIP: 0x8048437
[ 18.206504] EFLAGS: 00010202 CPU: 0
[ 18.207974] EAX: 00229000 EBX: 7ff00000 ECX: 3811f008 EDX: 00000000
[ 18.210139] ESI: 7ff00000 EDI: 00000000 EBP: bffedf28 ESP: bffedef0
[ 18.212290] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 18.214217] Code: db 75 28 eb 5c 66 90 8d b3 74 01 00 00 89 f0 e8 3b 37 4e 00 8b 4f 20 39 d9 74 2c 89 c2 89 f0 e8 db 33 4e 00 8b 5f 20 85 db 74 36 <8b> 83 70 01 00 00 85 c0 7f d2 89 d9 5b 89 c8 5e 5f 5d c3 90 31
[ 18.254093] EIP: lock_page_memcg+0x3c/0x80 SS:ESP: 0068:f2c2faa8
[ 18.256263] CR2: 000000000e200f90
[ 18.257948] ---[ end trace c89b8f16688d25d1 ]---
[ 18.259817] exit(b.out/367): started do_exit()
----------------------------------------
page_has_buffers() assertion can fail when the OOM killer is invoked.
----------------------------------------
[ 41.581473] Out of memory: Kill process 456 (b.out) score 18 or sacrifice child
[ 41.584500] Killed process 456 (b.out) total-vm:2099260kB, anon-rss:59724kB, file-rss:8kB, shmem-rss:0kB
[ 41.589503] oom_reaper: started reaping
[ 41.600128] XFS: Assertion failed: page_has_buffers(page), file: fs/xfs/xfs_aops.c, line: 1060
[ 41.603529] WARNING: CPU: 0 PID: 45 at fs/xfs/xfs_message.c:105 asswarn+0x26/0x30 [xfs]
[ 41.606787] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod serio_raw ata_piix e1000 mptspi scsi_transport_spi mptscsih mptbase libata
[ 41.612889] CPU: 0 PID: 45 Comm: kworker/u2:1 Tainted: G W 4.15.0-rc7+ #304
[ 41.616116] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 41.621016] Workqueue: writeback wb_workfn (flush-8:0)
[ 41.623407] EIP: asswarn+0x26/0x30 [xfs]
[ 41.625445] EFLAGS: 00010246 CPU: 0
[ 41.627382] EAX: 00000000 EBX: f48a5950 ECX: 00000021 EDX: f2d13bac
[ 41.630143] ESI: 00000000 EDI: f48a5950 EBP: f2d13c28 ESP: f2d13c14
[ 41.632803] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 41.635197] CR0: 80050033 CR2: 38211008 CR3: 2fed6000 CR4: 000406d0
[ 41.637854] Call Trace:
[ 41.639428] xfs_do_writepage+0x635/0x670 [xfs]
[ 41.641581] ? rmap_walk_file+0x133/0x1e0
[ 41.643527] ? rmap_walk+0x32/0x60
[ 41.645311] ? clear_page_dirty_for_io+0x156/0x1d0
[ 41.647457] ? xfs_vm_set_page_dirty+0x210/0x210 [xfs]
[ 41.649746] write_cache_pages+0x1ea/0x3f0
[ 41.651713] ? xfs_vm_set_page_dirty+0x210/0x210 [xfs]
[ 41.653878] ? __enqueue_entity+0x63/0x70
[ 41.655733] xfs_vm_writepages+0x48/0x80 [xfs]
[ 41.657677] do_writepages+0x1a/0x70
[ 41.659361] __writeback_single_inode+0x27/0x160
[ 41.661377] writeback_sb_inodes+0x1bd/0x320
[ 41.663310] __writeback_inodes_wb+0x74/0xb0
[ 41.665216] wb_writeback+0x169/0x190
[ 41.666971] wb_workfn+0x22c/0x2f0
[ 41.668618] ? __switch_to+0xa2/0x220
[ 41.670352] process_one_work+0xe7/0x240
[ 41.672834] worker_thread+0x31/0x360
[ 41.674704] kthread+0xd1/0x100
[ 41.676313] ? rescuer_thread+0x2d0/0x2d0
[ 41.678103] ? kthread_associate_blkcg+0x80/0x80
[ 41.680057] ret_from_fork+0x19/0x24
[ 41.681712] Code: 90 90 90 90 90 55 89 e5 83 ec 14 89 4c 24 10 89 54 24 0c 89 44 24 08 c7 44 24 04 9c 7b a4 f8 c7 04 24 00 00 00 00 e8 ba fd ff ff <0f> ff c9 c3 8d b6 00 00 00 00 55 89 e5 83 ec 14 89 4c 24 10 89
[ 41.688711] ---[ end trace c89b8f16688d25d1 ]---
[ 41.690645] ------------[ cut here ]------------
[ 41.692546] kernel BUG at fs/xfs/xfs_aops.c:911!
[ 41.694446] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 41.696522] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod serio_raw ata_piix e1000 mptspi scsi_transport_spi mptscsih mptbase libata
[ 41.701631] CPU: 0 PID: 45 Comm: kworker/u2:1 Tainted: G W 4.15.0-rc7+ #304
[ 41.704495] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 41.708742] Workqueue: writeback wb_workfn (flush-8:0)
[ 41.710865] EIP: xfs_do_writepage+0x641/0x670 [xfs]
[ 41.712874] EFLAGS: 00010246 CPU: 0
[ 41.714510] EAX: 00001000 EBX: f48a5950 ECX: 0000000c EDX: be01006d
[ 41.716924] ESI: 00000001 EDI: f48a5950 EBP: f2d13cb0 ESP: f2d13c30
[ 41.719308] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 41.721487] CR0: 80050033 CR2: 38211008 CR3: 2fed6000 CR4: 000406d0
[ 41.723922] Call Trace:
[ 41.725325] ? rmap_walk_file+0x133/0x1e0
[ 41.727146] ? clear_page_dirty_for_io+0x156/0x1d0
[ 41.729206] ? xfs_vm_set_page_dirty+0x210/0x210 [xfs]
[ 41.731314] write_cache_pages+0x1ea/0x3f0
[ 41.733159] ? xfs_vm_set_page_dirty+0x210/0x210 [xfs]
[ 41.735275] ? __enqueue_entity+0x63/0x70
[ 41.737120] xfs_vm_writepages+0x48/0x80 [xfs]
[ 41.739048] do_writepages+0x1a/0x70
[ 41.740788] __writeback_single_inode+0x27/0x160
[ 41.742769] writeback_sb_inodes+0x1bd/0x320
[ 41.744655] __writeback_inodes_wb+0x74/0xb0
[ 41.746621] wb_writeback+0x169/0x190
[ 41.748374] wb_workfn+0x22c/0x2f0
[ 41.750029] ? __switch_to+0xa2/0x220
[ 41.751776] process_one_work+0xe7/0x240
[ 41.753592] worker_thread+0x31/0x360
[ 41.755297] kthread+0xd1/0x100
[ 41.756889] ? rescuer_thread+0x2d0/0x2d0
[ 41.758682] ? kthread_associate_blkcg+0x80/0x80
[ 41.760624] ret_from_fork+0x19/0x24
[ 41.762274] Code: 4a fb ff ff 0f ff e9 71 fb ff ff b9 24 04 00 00 ba 01 db a3 f8 b8 f1 db a3 f8 e8 7b e7 01 00 e9 f9 f9 ff ff f3 90 e9 1d fa ff ff <0f> 0b 0f ff 8d 74 26 00 8d bc 27 00 00 00 00 e9 3d fb ff ff ba
[ 41.769434] EIP: xfs_do_writepage+0x641/0x670 [xfs] SS:ESP: 0068:f2d13c30
[ 41.771966] ---[ end trace c89b8f16688d25d2 ]---
[ 41.774051] exit(kworker/u2:1/45): started do_exit()
----------------------------------------
xfs_vm_set_page_dirty() hits warning after the OOM killer is invoked.
----------------------------------------
[ 35.662606] Out of memory: Kill process 383 (b.out) score 18 or sacrifice child
[ 35.665550] Killed process 383 (b.out) total-vm:2099260kB, anon-rss:58668kB, file-rss:8kB, shmem-rss:0kB
[ 35.683131] WARNING: CPU: 0 PID: 830 at fs/xfs/xfs_aops.c:1468 xfs_vm_set_page_dirty+0x125/0x210 [xfs]
[ 35.687796] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi scsi_transport_spi ata_piix e1000 libata mptscsih mptbase
[ 35.693698] CPU: 0 PID: 830 Comm: b.out Tainted: G W 4.15.0-rc7+ #305
[ 35.696759] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 35.701695] EIP: xfs_vm_set_page_dirty+0x125/0x210 [xfs]
[ 35.704206] EFLAGS: 00010046 CPU: 0
[ 35.706137] EAX: be000010 EBX: f3c81d58 ECX: f3c81d58 EDX: f3c81d4c
[ 35.708790] ESI: 00000246 EDI: f4895528 EBP: ee215abc ESP: ee215a98
[ 35.711394] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 35.713772] CR0: 80050033 CR2: 38572008 CR3: 2e2a6000 CR4: 000406d0
[ 35.716429] Call Trace:
[ 35.718094] set_page_dirty+0x3d/0x90
[ 35.720160] try_to_unmap_one+0x373/0x540
[ 35.722204] rmap_walk_file+0xf0/0x1e0
[ 35.724125] rmap_walk+0x32/0x60
[ 35.725845] try_to_unmap+0x4d/0xd0
[ 35.727618] ? page_remove_rmap+0x280/0x280
[ 35.729584] ? page_not_mapped+0x10/0x10
[ 35.731418] ? page_get_anon_vma+0x80/0x80
[ 35.733271] shrink_page_list+0x3e2/0xe80
[ 35.735153] shrink_inactive_list+0x1b2/0x440
[ 35.737132] shrink_node_memcg+0x34a/0x770
[ 35.738987] shrink_node+0xbb/0x2e0
[ 35.740626] do_try_to_free_pages+0xb2/0x300
[ 35.742459] try_to_free_pages+0x20b/0x330
[ 35.744248] ? schedule_timeout+0x10c/0x1d0
[ 35.746055] __alloc_pages_slowpath+0x2fb/0x6d3
[ 35.747980] __alloc_pages_nodemask+0x14a/0x170
[ 35.749913] handle_mm_fault+0x5a5/0xcd0
[ 35.751695] ? pick_next_task_fair+0xe1/0x490
[ 35.753582] ? sched_clock_cpu+0x13/0x120
[ 35.755387] __do_page_fault+0x1ea/0x4d0
[ 35.757106] ? __do_page_fault+0x4d0/0x4d0
[ 35.758861] do_page_fault+0x1a/0x20
[ 35.760468] common_exception+0x6f/0x76
[ 35.762612] EIP: 0x8048437
[ 35.764088] EFLAGS: 00010202 CPU: 0
[ 35.765737] EAX: 00096000 EBX: 7ff00000 ECX: 37f10008 EDX: 00000000
[ 35.768028] ESI: 7ff00000 EDI: 00000000 EBP: bf96e078 ESP: bf96e040
[ 35.770370] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 35.772393] Code: e4 8d 58 0c 89 d8 e8 8b c3 c4 c8 89 c6 8b 45 e8 8b 50 04 85 d2 74 5b 8b 40 14 a8 01 0f 85 ce 00 00 00 8b 45 e8 8b 00 a8 08 75 7b <0f> ff 8b 7d e8 8b 55 e4 89 f8 e8 4c 76 71 c8 8b 47 14 a8 01 0f
[ 35.779201] ---[ end trace c89b8f16688d25d1 ]---
[ 35.781296] BUG: unable to handle kernel paging request at 06000170
[ 35.783805] IP: lock_page_memcg+0x3c/0x80
[ 35.785616] *pde = 00000000
[ 35.787117] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 35.789110] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi serio_raw mptspi scsi_transport_spi ata_piix e1000 libata mptscsih mptbase
[ 35.794654] CPU: 0 PID: 830 Comm: b.out Tainted: G W 4.15.0-rc7+ #305
[ 35.797605] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 35.802203] EIP: lock_page_memcg+0x3c/0x80
[ 35.804174] EFLAGS: 00010206 CPU: 0
[ 35.805960] EAX: f2dc6850 EBX: 06000000 ECX: 00000012 EDX: 00000000
[ 35.808433] ESI: 00000010 EDI: f2dc6850 EBP: ee215ab4 ESP: ee215aa8
[ 35.810887] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 35.813095] CR0: 80050033 CR2: 06000170 CR3: 2e2a6000 CR4: 000406d0
[ 35.815719] Call Trace:
[ 35.817167] page_remove_rmap+0x92/0x280
[ 35.819046] try_to_unmap_one+0x206/0x540
[ 35.820965] rmap_walk_file+0xf0/0x1e0
[ 35.822785] rmap_walk+0x32/0x60
[ 35.824457] try_to_unmap+0x4d/0xd0
[ 35.826209] ? page_remove_rmap+0x280/0x280
[ 35.828203] ? page_not_mapped+0x10/0x10
[ 35.830025] ? page_get_anon_vma+0x80/0x80
[ 35.831952] shrink_page_list+0x3e2/0xe80
[ 35.833800] shrink_inactive_list+0x1b2/0x440
[ 35.835761] shrink_node_memcg+0x34a/0x770
[ 35.837679] shrink_node+0xbb/0x2e0
[ 35.839432] do_try_to_free_pages+0xb2/0x300
[ 35.841411] try_to_free_pages+0x20b/0x330
[ 35.843356] ? schedule_timeout+0x10c/0x1d0
[ 35.845301] __alloc_pages_slowpath+0x2fb/0x6d3
[ 35.847302] __alloc_pages_nodemask+0x14a/0x170
[ 35.849279] handle_mm_fault+0x5a5/0xcd0
[ 35.851066] ? pick_next_task_fair+0xe1/0x490
[ 35.852981] ? sched_clock_cpu+0x13/0x120
[ 35.854801] __do_page_fault+0x1ea/0x4d0
[ 35.856515] ? __do_page_fault+0x4d0/0x4d0
[ 35.858326] do_page_fault+0x1a/0x20
[ 35.860004] common_exception+0x6f/0x76
[ 35.861705] EIP: 0x8048437
[ 35.863114] EFLAGS: 00010202 CPU: 0
[ 35.864687] EAX: 00096000 EBX: 7ff00000 ECX: 37f10008 EDX: 00000000
[ 35.867027] ESI: 7ff00000 EDI: 00000000 EBP: bf96e078 ESP: bf96e040
[ 35.869348] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 35.871349] Code: db 75 28 eb 5c 66 90 8d b3 74 01 00 00 89 f0 e8 3b 37 4e 00 8b 4f 20 39 d9 74 2c 89 c2 89 f0 e8 db 33 4e 00 8b 5f 20 85 db 74 36 <8b> 83 70 01 00 00 85 c0 7f d2 89 d9 5b 89 c8 5e 5f 5d c3 90 31
[ 35.879102] EIP: lock_page_memcg+0x3c/0x80 SS:ESP: 0068:ee215aa8
[ 35.881510] CR2: 0000000006000170
[ 35.883126] ---[ end trace c89b8f16688d25d2 ]---
[ 35.885122] exit(b.out/830): started do_exit()
----------------------------------------
But xfs_vm_set_page_dirty() hits warning before the OOM killer is invoked.
----------------------------------------
[ 40.752823] exit(b.out/687): started do_exit()
[ 40.754594] exit(b.out/687): started exit_signals()
[ 40.756434] exit(b.out/687): ended exit_signals()
[ 40.758185] exit(b.out/687): started sync_mm_rss()
[ 40.759974] exit(b.out/687): started exit_mm()
[ 41.063943] exit(b.out/687): ended exit_mm()
[ 41.065842] exit(b.out/687): ended do_exit()
[ 41.787284] WARNING: CPU: 0 PID: 326 at fs/xfs/xfs_aops.c:1468 xfs_vm_set_page_dirty+0x125/0x210 [xfs]
[ 41.791206] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi mptspi scsi_transport_spi mptscsih ata_piix e1000 mptbase libata serio_raw
[ 41.796701] CPU: 0 PID: 326 Comm: b.out Tainted: G W 4.15.0-rc7+ #305
[ 41.799700] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 41.804302] EIP: xfs_vm_set_page_dirty+0x125/0x210 [xfs]
[ 41.806609] EFLAGS: 00010046 CPU: 0
[ 41.808432] EAX: be000010 EBX: f3cb1d58 ECX: f3cb1d58 EDX: f3cb1d4c
[ 41.811049] ESI: 00000246 EDI: f49320d0 EBP: f2c0f9cc ESP: f2c0f9a8
[ 41.813671] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 41.816002] CR0: 80050033 CR2: b7e6a49a CR3: 324bb000 CR4: 000406d0
[ 41.818649] Call Trace:
[ 41.820063] set_page_dirty+0x3d/0x90
[ 41.821917] try_to_unmap_one+0x373/0x540
[ 41.823891] rmap_walk_file+0xf0/0x1e0
[ 41.825867] rmap_walk+0x32/0x60
[ 41.827672] try_to_unmap+0x4d/0xd0
[ 41.829456] ? page_remove_rmap+0x280/0x280
[ 41.831445] ? page_not_mapped+0x10/0x10
[ 41.833336] ? page_get_anon_vma+0x80/0x80
[ 41.835250] shrink_page_list+0x3e2/0xe80
[ 41.837169] shrink_inactive_list+0x1b2/0x440
[ 41.839157] shrink_node_memcg+0x34a/0x770
[ 41.841086] shrink_node+0xbb/0x2e0
[ 41.842878] do_try_to_free_pages+0xb2/0x300
[ 41.844841] try_to_free_pages+0x20b/0x330
[ 41.846807] __alloc_pages_slowpath+0x2fb/0x6d3
[ 41.848871] ? _lookup_address_cpa.isra.24+0x25/0x30
[ 41.851042] ? __change_page_attr+0x1e4/0x6f0
[ 41.853083] __alloc_pages_nodemask+0x14a/0x170
[ 41.855173] __do_page_cache_readahead+0xd6/0x210
[ 41.857315] ? pagecache_get_page+0x1f/0x1f0
[ 41.859360] filemap_fault+0x234/0x570
[ 41.861222] ? filemap_map_pages+0x12f/0x340
[ 41.863299] ? filemap_fdatawait_keep_errors+0x50/0x50
[ 41.865582] __xfs_filemap_fault.isra.16+0x2d/0xb0 [xfs]
[ 41.867887] ? filemap_fdatawait_keep_errors+0x50/0x50
[ 41.870156] xfs_filemap_fault+0xa/0x10 [xfs]
[ 41.872141] __do_fault+0x11/0x30
[ 41.873825] handle_mm_fault+0x8f0/0xcd0
[ 41.875632] __do_page_fault+0x1ea/0x4d0
[ 41.877426] ? __do_page_fault+0x4d0/0x4d0
[ 41.879264] do_page_fault+0x1a/0x20
[ 41.880907] common_exception+0x6f/0x76
[ 41.882638] EIP: 0xb7e6a49a
[ 41.884016] EFLAGS: 00010246 CPU: 0
[ 41.885639] EAX: fffffff4 EBX: 01200011 ECX: 00000000 EDX: 00000000
[ 41.888093] ESI: 00000000 EDI: b7db0728 EBP: bf884358 ESP: bf884310
[ 41.890744] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 41.893088] Code: e4 8d 58 0c 89 d8 e8 8b c3 c4 c8 89 c6 8b 45 e8 8b 50 04 85 d2 74 5b 8b 40 14 a8 01 0f 85 ce 00 00 00 8b 45 e8 8b 00 a8 08 75 7b <0f> ff 8b 7d e8 8b 55 e4 89 f8 e8 4c 76 71 c8 8b 47 14 a8 01 0f
[ 41.900752] ---[ end trace c89b8f16688d25d1 ]---
----------------------------------------
----------------------------------------
[ 41.378852] Out of memory: Kill process 360 (b.out) score 19 or sacrifice child
[ 41.382162] Killed process 360 (b.out) total-vm:2099260kB, anon-rss:60512kB, file-rss:8kB, shmem-rss:0kB
[ 41.392549] BUG: unable to handle kernel NULL pointer dereference at 00000004
[ 41.395775] IP: page_remove_rmap+0x17/0x280
[ 41.398033] *pde = 00000000
[ 41.399860] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 41.402116] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod serio_raw e1000 ata_piix mptspi scsi_transport_spi mptscsih mptbase libata
[ 41.408340] CPU: 0 PID: 326 Comm: b.out Tainted: G W 4.15.0-rc7+ #305
[ 41.411632] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 41.416729] EIP: page_remove_rmap+0x17/0x280
[ 41.418931] EFLAGS: 00010246 CPU: 0
[ 41.420865] EAX: 00000000 EBX: f2db8d40 ECX: 0000000e EDX: 00000000
[ 41.423567] ESI: 0000000e EDI: f4930a50 EBP: f2fa79d8 ESP: f2fa79cc
[ 41.426422] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 41.429016] CR0: 80050033 CR2: 00000004 CR3: 328ac000 CR4: 000406d0
[ 41.431752] Call Trace:
[ 41.433338] try_to_unmap_one+0x206/0x540
[ 41.435351] rmap_walk_file+0xf0/0x1e0
[ 41.437332] rmap_walk+0x32/0x60
[ 41.439218] try_to_unmap+0x4d/0xd0
[ 41.441141] ? page_remove_rmap+0x280/0x280
[ 41.443105] ? page_not_mapped+0x10/0x10
[ 41.444980] ? page_get_anon_vma+0x80/0x80
[ 41.447040] shrink_page_list+0x3e2/0xe80
[ 41.449005] shrink_inactive_list+0x1b2/0x440
[ 41.450909] shrink_node_memcg+0x34a/0x770
[ 41.452774] shrink_node+0xbb/0x2e0
[ 41.454570] do_try_to_free_pages+0xb2/0x300
[ 41.456589] try_to_free_pages+0x20b/0x330
[ 41.458479] __alloc_pages_slowpath+0x2fb/0x6d3
[ 41.460487] ? ktime_get+0x47/0xf0
[ 41.462192] ? lapic_next_deadline+0x24/0x30
[ 41.464117] __alloc_pages_nodemask+0x14a/0x170
[ 41.466131] __do_page_cache_readahead+0xd6/0x210
[ 41.468184] ? pagecache_get_page+0x1f/0x1f0
[ 41.470089] ? apic_timer_interrupt+0x3c/0x44
[ 41.471997] filemap_fault+0x234/0x570
[ 41.473717] ? radix_tree_next_chunk+0xf1/0x2c0
[ 41.475630] ? filemap_map_pages+0x12f/0x340
[ 41.477534] ? filemap_fdatawait_keep_errors+0x50/0x50
[ 41.479768] __xfs_filemap_fault.isra.16+0x2d/0xb0 [xfs]
[ 41.482018] ? filemap_fdatawait_keep_errors+0x50/0x50
[ 41.484138] xfs_filemap_fault+0xa/0x10 [xfs]
[ 41.485958] __do_fault+0x11/0x30
[ 41.487476] handle_mm_fault+0x8f0/0xcd0
[ 41.489108] __do_page_fault+0x1ea/0x4d0
[ 41.490739] ? __do_page_fault+0x4d0/0x4d0
[ 41.492408] do_page_fault+0x1a/0x20
[ 41.493989] common_exception+0x6f/0x76
[ 41.495687] EIP: 0xb7e8449a
[ 41.497077] EFLAGS: 00010246 CPU: 0
[ 41.498552] EAX: fffffff4 EBX: 01200011 ECX: 00000000 EDX: 00000000
[ 41.500739] ESI: 00000000 EDI: b7dca728 EBP: bfc79c58 ESP: bfc79c10
[ 41.502997] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 41.504987] Code: ff ff 83 e8 01 e9 4f ff ff ff 8d 76 00 8d bc 27 00 00 00 00 55 89 e5 56 53 89 c3 83 ec 04 8b 40 14 a8 01 0f 85 18 02 00 00 89 d8 <f6> 40 04 01 74 6b 84 d2 0f 85 5b 01 00 00 3e 83 43 0c ff 78 0c
[ 41.511758] EIP: page_remove_rmap+0x17/0x280 SS:ESP: 0068:f2fa79cc
[ 41.514034] CR2: 0000000000000004
[ 41.515779] ---[ end trace c89b8f16688d25d1 ]---
[ 41.517708] exit(b.out/326): started do_exit()
----------------------------------------
----------------------------------------
[ 35.986299] Out of memory: Kill process 335 (b.out) score 18 or sacrifice child
[ 35.988781] Killed process 335 (b.out) total-vm:2099260kB, anon-rss:57344kB, file-rss:8kB, shmem-rss:0kB
[ 35.994786] BUG: unable to handle kernel NULL pointer dereference at 0000073c
[ 35.996860] IP: page_remove_rmap+0x17/0x280
[ 35.998175] *pde = 00000000
[ 35.999075] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 36.000472] Modules linked in: xfs libcrc32c sd_mod sr_mod cdrom ata_generic pata_acpi e1000 mptspi scsi_transport_spi mptscsih ata_piix mptbase libata serio_raw
[ 36.004711] CPU: 0 PID: 480 Comm: b.out Tainted: G W 4.15.0-rc7+ #306
[ 36.006905] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 36.009964] EIP: page_remove_rmap+0x17/0x280
[ 36.011210] EFLAGS: 00010202 CPU: 0
[ 36.012225] EAX: 00000738 EBX: f2d84040 ECX: 00000010 EDX: 00000000
[ 36.014071] ESI: 0000000f EDI: f4932030 EBP: f159bac8 ESP: f159babc
[ 36.016011] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 36.017657] CR0: 80050033 CR2: 0000073c CR3: 3153b000 CR4: 000406d0
[ 36.019569] Call Trace:
[ 36.020440] try_to_unmap_one+0x206/0x540
[ 36.021879] rmap_walk_file+0xf0/0x1e0
[ 36.023126] rmap_walk+0x32/0x60
[ 36.024169] try_to_unmap+0x4d/0xd0
[ 36.025245] ? page_remove_rmap+0x280/0x280
[ 36.026551] ? page_not_mapped+0x10/0x10
[ 36.027775] ? page_get_anon_vma+0x80/0x80
[ 36.029077] shrink_page_list+0x3e2/0xe80
[ 36.030341] shrink_inactive_list+0x1b2/0x440
[ 36.031763] shrink_node_memcg+0x34a/0x770
[ 36.033047] shrink_node+0xbb/0x2e0
[ 36.034152] do_try_to_free_pages+0xb2/0x300
[ 36.035567] try_to_free_pages+0x20b/0x330
[ 36.036853] ? schedule_timeout+0x10c/0x1d0
[ 36.038141] __alloc_pages_slowpath+0x2fb/0x6d3
[ 36.039521] __alloc_pages_nodemask+0x14a/0x170
[ 36.040897] handle_mm_fault+0x5a5/0xcd0
[ 36.042036] ? pick_next_task_fair+0xe1/0x490
[ 36.043376] ? sched_clock_cpu+0x13/0x120
[ 36.044600] __do_page_fault+0x1ea/0x4d0
[ 36.045760] ? __do_page_fault+0x4d0/0x4d0
[ 36.046964] do_page_fault+0x1a/0x20
[ 36.048043] common_exception+0x6f/0x76
[ 36.049280] EIP: 0x8048437
[ 36.050135] EFLAGS: 00010202 CPU: 0
[ 36.051219] EAX: 00615000 EBX: 7ff00000 ECX: 38420008 EDX: 00000000
[ 36.053080] ESI: 7ff00000 EDI: 00000000 EBP: bf883de8 ESP: bf883db0
[ 36.054876] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 36.056455] Code: ff ff 83 e8 01 e9 4f ff ff ff 8d 76 00 8d bc 27 00 00 00 00 55 89 e5 56 53 89 c3 83 ec 04 8b 40 14 a8 01 0f 85 18 02 00 00 89 d8 <f6> 40 04 01 74 6b 84 d2 0f 85 5b 01 00 00 3e 83 43 0c ff 78 0c
[ 36.062043] EIP: page_remove_rmap+0x17/0x280 SS:ESP: 0068:f159babc
[ 36.063847] CR2: 000000000000073c
[ 36.064945] ---[ end trace c89b8f16688d25d1 ]---
[ 36.066379] exit(b.out/480): started do_exit()
----------------------------------------
----------------------------------------
[ 56.306933] Out of memory: Kill process 619 (b.out) score 15 or sacrifice child
[ 56.309123] Killed process 619 (b.out) total-vm:2099260kB, anon-rss:48104kB, file-rss:8kB, shmem-rss:0kB
[ 56.313607] BUG: unable to handle kernel NULL pointer dereference at 00000174
[ 56.315623] IP: lock_page_memcg+0x3c/0x80
[ 56.316758] *pde = 00000000
[ 56.317582] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 56.318858] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod e1000 ata_piix mptspi scsi_transport_spi mptscsih mptbase libata serio_raw
[ 56.323133] CPU: 0 PID: 782 Comm: b.out Tainted: G W 4.15.0-rc7+ #306
[ 56.325410] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 56.328521] EIP: lock_page_memcg+0x3c/0x80
[ 56.329767] EFLAGS: 00010202 CPU: 0
[ 56.330836] EAX: f2e97900 EBX: 00000004 ECX: 00000050 EDX: 00000000
[ 56.332733] ESI: 0000004f EDI: f2e97900 EBP: f2365ab4 ESP: f2365aa8
[ 56.334483] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 56.335984] CR0: 80050033 CR2: 00000174 CR3: 306f3000 CR4: 000406d0
[ 56.337949] Call Trace:
[ 56.338755] page_remove_rmap+0x92/0x280
[ 56.339969] try_to_unmap_one+0x206/0x540
[ 56.341199] rmap_walk_file+0xf0/0x1e0
[ 56.342359] rmap_walk+0x32/0x60
[ 56.343366] try_to_unmap+0x4d/0xd0
[ 56.344444] ? page_remove_rmap+0x280/0x280
[ 56.345716] ? page_not_mapped+0x10/0x10
[ 56.346893] ? page_get_anon_vma+0x80/0x80
[ 56.348070] shrink_page_list+0x3e2/0xe80
[ 56.349206] ? depot_save_stack+0x122/0x3c0
[ 56.350387] shrink_inactive_list+0x1b2/0x440
[ 56.351616] shrink_node_memcg+0x34a/0x770
[ 56.352776] shrink_node+0xbb/0x2e0
[ 56.353844] do_try_to_free_pages+0xb2/0x300
[ 56.355226] try_to_free_pages+0x20b/0x330
[ 56.356486] __alloc_pages_slowpath+0x2fb/0x6d3
[ 56.357875] ? __accumulate_pelt_segments+0x32/0x50
[ 56.359332] ? update_load_avg+0xa30/0xa70
[ 56.360504] __alloc_pages_nodemask+0x14a/0x170
[ 56.361782] handle_mm_fault+0x5a5/0xcd0
[ 56.362868] ? pick_next_task_fair+0xe1/0x490
[ 56.364113] ? sched_clock_cpu+0x13/0x120
[ 56.365240] __do_page_fault+0x1ea/0x4d0
[ 56.366352] ? __do_page_fault+0x4d0/0x4d0
[ 56.367483] do_page_fault+0x1a/0x20
[ 56.368540] common_exception+0x6f/0x76
[ 56.369694] EIP: 0x8048437
[ 56.370525] EFLAGS: 00010202 CPU: 0
[ 56.371650] EAX: 0030b000 EBX: 7ff00000 ECX: 38206008 EDX: 00000000
[ 56.373812] ESI: 7ff00000 EDI: 00000000 EBP: bfdb7328 ESP: bfdb72f0
[ 56.375702] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 56.377290] Code: db 75 28 eb 5c 66 90 8d b3 74 01 00 00 89 f0 e8 cb 25 46 00 8b 4f 20 39 d9 74 2c 89 c2 89 f0 e8 6b 22 46 00 8b 5f 20 85 db 74 36 <8b> 83 70 01 00 00 85 c0 7f d2 89 d9 5b 89 c8 5e 5f 5d c3 90 31
[ 56.382432] EIP: lock_page_memcg+0x3c/0x80 SS:ESP: 0068:f2365aa8
[ 56.384169] CR2: 0000000000000174
[ 56.385191] ---[ end trace c89b8f16688d25d1 ]---
[ 56.386495] exit(b.out/782): started do_exit()
----------------------------------------
Overall, memory corruption is strongly suspected.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 59+ messages in thread* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-11 14:11 ` Tetsuo Handa
@ 2018-01-11 14:21 ` Michal Hocko
2018-01-11 14:37 ` Tetsuo Handa
2018-01-12 1:31 ` [mm " Tetsuo Handa
2018-01-11 18:11 ` [mm? 4.15-rc7] " Linus Torvalds
1 sibling, 2 replies; 59+ messages in thread
From: Michal Hocko @ 2018-01-11 14:21 UTC (permalink / raw)
To: Tetsuo Handa; +Cc: linux-mm, x86, linux-fsdevel
On Thu 11-01-18 23:11:12, Tetsuo Handa wrote:
> Michal Hocko wrote:
> > On Wed 10-01-18 22:37:52, Tetsuo Handa wrote:
> > > Michal Hocko wrote:
> > > > On Wed 10-01-18 20:49:56, Tetsuo Handa wrote:
> > > > > Tetsuo Handa wrote:
> > > > > > I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
> > > > > > So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
> > > > > > Does anyone know what is happening?
> > > > >
> > > > > I simplified the reproducer and succeeded to reproduce this bug with both
> > > > > i7-2630QM (8 core) and i5-4440S (4 core). Thus, I think that this bug is
> > > > > not architecture specific.
> > > >
> > > > Can you see the same with 64b kernel?
> > >
> > > No. I can hit this bug with only x86_32 kernels.
> > > But if the cause is not specific to 32b, this might be silent memory corruption.
> > >
> > > > It smells like a ref count imbalance and premature page free to me. Can
> > > > you try to bisect this?
> > >
> > > Too difficult to bisect, but at least I can hit this bug with 4.8+ kernels.
>
> The bug in 4.8 kernel might be different from the bug in 4.15-rc7 kernel.
> 4.15-rc7 kernel hits the bug so trivially.
Maybe you want to disable the oom reaper to reduce chances of some issue
there.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-11 14:21 ` Michal Hocko
@ 2018-01-11 14:37 ` Tetsuo Handa
2018-01-12 1:31 ` [mm " Tetsuo Handa
1 sibling, 0 replies; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-11 14:37 UTC (permalink / raw)
To: mhocko; +Cc: linux-mm, x86, linux-fsdevel
Michal Hocko wrote:
> On Thu 11-01-18 23:11:12, Tetsuo Handa wrote:
> > Michal Hocko wrote:
> > > On Wed 10-01-18 22:37:52, Tetsuo Handa wrote:
> > > > Michal Hocko wrote:
> > > > > On Wed 10-01-18 20:49:56, Tetsuo Handa wrote:
> > > > > > Tetsuo Handa wrote:
> > > > > > > I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
> > > > > > > So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
> > > > > > > Does anyone know what is happening?
> > > > > >
> > > > > > I simplified the reproducer and succeeded to reproduce this bug with both
> > > > > > i7-2630QM (8 core) and i5-4440S (4 core). Thus, I think that this bug is
> > > > > > not architecture specific.
> > > > >
> > > > > Can you see the same with 64b kernel?
> > > >
> > > > No. I can hit this bug with only x86_32 kernels.
> > > > But if the cause is not specific to 32b, this might be silent memory corruption.
> > > >
> > > > > It smells like a ref count imbalance and premature page free to me. Can
> > > > > you try to bisect this?
> > > >
> > > > Too difficult to bisect, but at least I can hit this bug with 4.8+ kernels.
> >
> > The bug in 4.8 kernel might be different from the bug in 4.15-rc7 kernel.
> > 4.15-rc7 kernel hits the bug so trivially.
>
> Maybe you want to disable the oom reaper to reduce chances of some issue
> there.
I already tried it for 4.15-rc7.
The bug can occur before the OOM killer is invoked for the first time after boot.
The OOM killer and the OOM reaper are not the culprit.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc7] Random oopses under memory pressure.
2018-01-11 14:21 ` Michal Hocko
2018-01-11 14:37 ` Tetsuo Handa
@ 2018-01-12 1:31 ` Tetsuo Handa
2018-01-12 1:42 ` Linus Torvalds
1 sibling, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-12 1:31 UTC (permalink / raw)
To: Michal Hocko; +Cc: torvalds, linux-mm, x86, linux-fsdevel
Tetsuo Handa wrote:
> Michal Hocko wrote:
> > On Wed 10-01-18 22:37:52, Tetsuo Handa wrote:
> > > Michal Hocko wrote:
> > > > On Wed 10-01-18 20:49:56, Tetsuo Handa wrote:
> > > > > Tetsuo Handa wrote:
> > > > > > I can hit this bug with Linux 4.11 and 4.8. (i.e. at least all 4.8+ have this bug.)
> > > > > > So far I haven't hit this bug with Linux 4.8-rc3 and 4.7.
> > > > > > Does anyone know what is happening?
> > > > >
> > > > > I simplified the reproducer and succeeded to reproduce this bug with both
> > > > > i7-2630QM (8 core) and i5-4440S (4 core). Thus, I think that this bug is
> > > > > not architecture specific.
> > > >
> > > > Can you see the same with 64b kernel?
> > >
> > > No. I can hit this bug with only x86_32 kernels.
> > > But if the cause is not specific to 32b, this might be silent memory corruption.
> > >
> > > > It smells like a ref count imbalance and premature page free to me. Can
> > > > you try to bisect this?
> > >
> > > Too difficult to bisect, but at least I can hit this bug with 4.8+ kernels.
>
> The bug in 4.8 kernel might be different from the bug in 4.15-rc7 kernel.
> 4.15-rc7 kernel hits the bug so trivially.
CONFIG_HIGHMEM4G=y or CONFIG_HIGHMEM64G=y x86_32 kernel hits this bug.
[ 0.000000] 2184MB HIGHMEM available.
[ 0.000000] 887MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 377fe000
[ 0.000000] low ram: 0 - 377fe000
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] Normal [mem 0x0000000001000000-0x00000000377fdfff]
[ 0.000000] HighMem [mem 0x00000000377fe000-0x00000000bfffffff]
[ 0.000000] 4230MB HIGHMEM available.
[ 0.000000] 889MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 379fe000
[ 0.000000] low ram: 0 - 379fe000
[ 0.000000] crashkernel: memory value expected
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] Normal [mem 0x0000000001000000-0x00000000379fdfff]
[ 0.000000] HighMem [mem 0x00000000379fe000-0x000000013fffffff]
CONFIG_NOHIGHMEM=y x86_32 kernel does not hit this bug.
[ 0.000000] Warning only 891MB will be used.
[ 0.000000] Use a HIGHMEM enabled kernel.
[ 0.000000] 891MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 37bfe000
[ 0.000000] low ram: 0 - 37bfe000
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] Normal [mem 0x0000000001000000-0x0000000037bfdfff]
x86_64 kernel does not show HighMem line.
[ 0.000000] Reserving 256MB of memory at 448MB for crashkernel (System RAM: 4095MB)
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.000000] Normal [mem 0x0000000100000000-0x000000013fffffff]
Thus, I suspect that somewhere is confusing HighMem pages and !HighMem pages.
[ 2.845150] systemd-journald[128]: Received SIGTERM from PID 1 (systemd).
[ 2.859834] bash: 31 output lines suppressed due to ratelimiting
[ 8.234776] random: crng init done
[ 58.935340] WARNING: CPU: 0 PID: 720 at fs/xfs/xfs_aops.c:1468 xfs_vm_set_page_dirty+0x12d/0x210 [xfs]
[ 58.939192] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic sd_mod pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi drm crc32c_intel scsi_transport_spi serio_raw ata_piix mptscsih libata e1000 mptbase i2c_core
[ 58.948541] CPU: 0 PID: 720 Comm: b.out Not tainted 4.15.0-rc7+ #309
[ 58.951330] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 58.956375] EIP: xfs_vm_set_page_dirty+0x12d/0x210 [xfs]
[ 58.958909] EFLAGS: 00010046 CPU: 0
[ 58.960846] EAX: 13000010 EBX: f3bd61d8 ECX: f3bd61d8 EDX: f3bd61cc
[ 58.963663] ESI: 00000246 EDI: f4503f68 EBP: eed8d9ec ESP: eed8d9c4
[ 58.966515] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 58.969065] CR0: 80050033 CR2: b7f07700 CR3: 33453a80 CR4: 000406f0
[ 58.971948] Call Trace:
[ 58.973593] set_page_dirty+0x42/0xa0
[ 58.975606] try_to_unmap_one+0x417/0x6c0
[ 58.977768] rmap_walk_file+0xf0/0x1e0
[ 58.979878] rmap_walk+0x37/0x60
[ 58.981781] try_to_unmap+0x52/0xd0
[ 58.983645] ? page_remove_rmap+0x270/0x270
[ 58.985938] ? page_not_mapped+0x20/0x20
[ 58.988044] ? page_get_anon_vma+0x80/0x80
[ 58.990222] shrink_page_list+0x3f6/0xef0
[ 58.992275] shrink_inactive_list+0x1c2/0x530
[ 58.994528] ? check_preempt_wakeup+0x181/0x230
[ 58.996757] shrink_node_memcg+0x352/0x770
[ 58.998710] shrink_node+0xc3/0x2f0
[ 59.000481] do_try_to_free_pages+0xc9/0x320
[ 59.002494] try_to_free_pages+0x163/0x420
[ 59.004472] __alloc_pages_slowpath+0x280/0x8e4
[ 59.006613] ? release_pages+0x13a/0x340
[ 59.008508] ? __accumulate_pelt_segments+0x37/0x50
[ 59.010835] __alloc_pages_nodemask+0x23b/0x260
[ 59.012955] do_anonymous_page+0xef/0x5a0
[ 59.014879] ? set_next_entity+0x96/0x280
[ 59.016772] handle_mm_fault+0x888/0xa50
[ 59.018665] __do_page_fault+0x1e3/0x4d0
[ 59.020609] ? __do_page_fault+0x4d0/0x4d0
[ 59.022481] do_page_fault+0x29/0xe0
[ 59.024165] ? __do_page_fault+0x4d0/0x4d0
[ 59.025992] ? __do_page_fault+0x4d0/0x4d0
[ 59.027808] common_exception+0x6f/0x76
[ 59.029566] EIP: 0x8048437
[ 59.031034] EFLAGS: 00010202 CPU: 0
[ 59.032699] EAX: 004c5000 EBX: 7ff00000 ECX: 382cd008 EDX: 00000000
[ 59.035109] ESI: 7ff00000 EDI: 00000000 EBP: bfbc6198 ESP: bfbc6160
[ 59.037516] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 59.039825] Code: e4 8d 58 0c 89 d8 e8 23 c6 8c c8 89 c6 8b 45 e8 8b 50 04 85 d2 74 5b 8b 40 14 a8 01 0f 85 d3 00 00 00 8b 45 e8 8b 00 a8 08 75 7b <0f> ff 8b 7d e8 8b 55 e4 89 f8 e8 94 0a 38 c8 8b 47 14 a8 01 0f
[ 59.046971] ---[ end trace a3882c406e0b0b93 ]---
[ 59.061588] b.out invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0
[ 59.066371] b.out cpuset=/ mems_allowed=0
[ 59.068273] CPU: 0 PID: 798 Comm: b.out Tainted: G W 4.15.0-rc7+ #309
[ 59.071107] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 59.075456] Call Trace:
[ 59.076963] dump_stack+0x58/0x78
[ 59.078653] dump_header+0x70/0x271
[ 59.080371] ? ___ratelimit+0x83/0xf0
[ 59.082086] oom_kill_process+0x1f5/0x3f0
[ 59.083881] ? has_capability_noaudit+0x1a/0x30
[ 59.086106] ? oom_badness+0xe1/0x140
[ 59.087885] out_of_memory+0xe6/0x280
[ 59.089653] __alloc_pages_slowpath+0x647/0x8e4
[ 59.091643] __alloc_pages_nodemask+0x23b/0x260
[ 59.093741] do_anonymous_page+0xef/0x5a0
[ 59.095767] ? set_next_entity+0x96/0x280
[ 59.097770] handle_mm_fault+0x888/0xa50
[ 59.099640] __do_page_fault+0x1e3/0x4d0
[ 59.101440] ? __do_page_fault+0x4d0/0x4d0
[ 59.103344] do_page_fault+0x29/0xe0
[ 59.105011] ? __do_page_fault+0x4d0/0x4d0
[ 59.106821] ? __do_page_fault+0x4d0/0x4d0
[ 59.108635] common_exception+0x6f/0x76
[ 59.110432] EIP: 0x8048437
[ 59.111920] EFLAGS: 00010202 CPU: 0
[ 59.113598] EAX: 003e5000 EBX: 7ff00000 ECX: 382a3008 EDX: 00000000
[ 59.116181] ESI: 7ff00000 EDI: 00000000 EBP: bf8ccc08 ESP: bf8ccbd0
[ 59.118630] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 59.120851] ? move_vma+0x90/0x280
[ 59.122535] Mem-Info:
[ 59.123848] active_anon:957369 inactive_anon:2048 isolated_anon:0
[ 59.123848] active_file:8 inactive_file:32 isolated_file:34
[ 59.123848] unevictable:0 dirty:0 writeback:0 unstable:0
[ 59.123848] slab_reclaimable:1481 slab_unreclaimable:2749
[ 59.123848] mapped:0 shmem:2050 pagetables:24289 bounce:0
[ 59.123848] free:39933 free_pcp:63 free_cma:0
[ 59.137797] Node 0 active_anon:3829476kB inactive_anon:8192kB active_file:32kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):136kB mapped:0kB dirty:0kB writeback:0kB shmem:8200kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
[ 59.150272] DMA free:15912kB min:788kB low:984kB high:1180kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15916kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 59.164549] lowmem_reserve[]: 0 803 4009 4009
[ 59.166791] Normal free:143380kB min:40868kB low:51084kB high:61300kB active_anon:568644kB inactive_anon:0kB active_file:20kB inactive_file:44kB unevictable:0kB writepending:0kB present:894968kB managed:840896kB mlocked:0kB kernel_stack:4760kB pagetables:97156kB bounce:0kB free_pcp:120kB local_pcp:120kB free_cma:0kB
[ 59.179026] lowmem_reserve[]: 0 0 25647 25647
[ 59.181335] HighMem free:440kB min:512kB low:41276kB high:82040kB active_anon:3260828kB inactive_anon:8192kB active_file:12kB inactive_file:24kB unevictable:0kB writepending:0kB present:3282888kB managed:3282888kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:132kB local_pcp:132kB free_cma:0kB
[ 59.193942] lowmem_reserve[]: 0 0 0 0
[ 59.196116] DMA: 0*4kB 1*8kB (U) 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15912kB
[ 59.202192] Normal: 159*4kB (UE) 91*8kB (UE) 36*16kB (UME) 18*32kB (UE) 9*64kB (UE) 6*128kB (UME) 3*256kB (UE) 3*512kB (UME) 2*1024kB (ME) 2*2048kB (UE) 32*4096kB (M) = 143380kB
[ 59.209381] HighMem: 6*4kB (UM) 4*8kB (UM) 4*16kB (UM) 4*32kB (UM) 1*64kB (M) 1*128kB (U) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 440kB
[ 59.215678] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[ 59.220196] 2125 total pagecache pages
[ 59.222587] 0 pages in swap cache
[ 59.224777] Swap cache stats: add 0, delete 0, find 0/0
[ 59.227634] Free swap = 0kB
[ 59.229594] Total swap = 0kB
[ 59.231533] 1048462 pages RAM
[ 59.233527] 820722 pages HighMem/MovableOnly
[ 59.235870] 13537 pages reserved
[ 59.237905] 0 pages cma reserved
[ 59.239980] Out of memory: Kill process 388 (b.out) score 3 or sacrifice child
[ 59.243277] Killed process 388 (b.out) total-vm:2099260kB, anon-rss:16688kB, file-rss:12kB, shmem-rss:0kB
[ 66.736837] ------------[ cut here ]------------
[ 66.739227] list_add corruption. next->prev should be prev (cf64836e), but was 6673909a. (next=b675dd8b).
[ 66.743934] WARNING: CPU: 0 PID: 720 at lib/list_debug.c:25 __list_add_valid+0x3f/0x90
[ 66.747137] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic sd_mod pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi drm crc32c_intel scsi_transport_spi serio_raw ata_piix mptscsih libata e1000 mptbase i2c_core
[ 66.756725] CPU: 0 PID: 720 Comm: b.out Tainted: G W 4.15.0-rc7+ #309
[ 66.759984] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 66.764940] EIP: __list_add_valid+0x3f/0x90
[ 66.767046] EFLAGS: 00010096 CPU: 0
[ 66.768984] EAX: 0000005d EBX: 00000200 ECX: c1b3bce8 EDX: 00000086
[ 66.771626] ESI: f4503f7c EDI: f4503f7c EBP: eed8db40 ESP: eed8db2c
[ 66.774286] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 66.776701] CR0: 80050033 CR2: 08048437 CR3: 33453a80 CR4: 000406f0
[ 66.779404] Call Trace:
[ 66.781033] putback_inactive_pages+0x13f/0x3b0
[ 66.783198] shrink_inactive_list+0x215/0x530
[ 66.785300] ? check_preempt_wakeup+0x181/0x230
[ 66.787465] shrink_node_memcg+0x352/0x770
[ 66.789431] shrink_node+0xc3/0x2f0
[ 66.791192] do_try_to_free_pages+0xc9/0x320
[ 66.793156] try_to_free_pages+0x163/0x420
[ 66.795057] __alloc_pages_slowpath+0x280/0x8e4
[ 66.797072] ? release_pages+0x13a/0x340
[ 66.798908] ? __accumulate_pelt_segments+0x37/0x50
[ 66.800993] __alloc_pages_nodemask+0x23b/0x260
[ 66.802991] do_anonymous_page+0xef/0x5a0
[ 66.804855] ? set_next_entity+0x96/0x280
[ 66.806716] handle_mm_fault+0x888/0xa50
[ 66.808573] __do_page_fault+0x1e3/0x4d0
[ 66.810423] ? __do_page_fault+0x4d0/0x4d0
[ 66.812325] do_page_fault+0x29/0xe0
[ 66.814069] ? __do_page_fault+0x4d0/0x4d0
[ 66.815902] ? __do_page_fault+0x4d0/0x4d0
[ 66.817738] common_exception+0x6f/0x76
[ 66.819512] EIP: 0x8048437
[ 66.820931] EFLAGS: 00010202 CPU: 0
[ 66.822529] EAX: 004c5000 EBX: 7ff00000 ECX: 382cd008 EDX: 00000000
[ 66.824891] ESI: 7ff00000 EDI: 00000000 EBP: bfbc6198 ESP: bfbc6160
[ 66.827199] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 66.829285] Code: 39 c8 74 2d 39 c3 74 29 b8 01 00 00 00 83 c4 10 5b 5d c3 89 4c 24 0c 89 5c 24 08 89 54 24 04 c7 04 24 70 aa 8d c1 e8 b1 e5 d2 ff <0f> ff 31 c0 eb dc 89 4c 24 0c 89 5c 24 08 89 44 24 04 c7 04 24
[ 66.836182] ---[ end trace a3882c406e0b0b94 ]---
[ 66.844058] ------------[ cut here ]------------
[ 66.845963] list_add corruption. next->prev should be prev (cf64836e), but was 6673909a. (next=b675dd8b).
[ 66.849814] WARNING: CPU: 0 PID: 30 at lib/list_debug.c:25 __list_add_valid+0x3f/0x90
[ 66.852599] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic sd_mod pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi drm crc32c_intel scsi_transport_spi serio_raw ata_piix mptscsih libata e1000 mptbase i2c_core
[ 66.861186] CPU: 0 PID: 30 Comm: kswapd0 Tainted: G W 4.15.0-rc7+ #309
[ 66.864078] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 66.868375] EIP: __list_add_valid+0x3f/0x90
[ 66.870277] EFLAGS: 00010096 CPU: 0
[ 66.872003] EAX: 0000005d EBX: 00000200 ECX: c1b3bce8 EDX: 00000082
[ 66.874461] ESI: f4503f7c EDI: f55738bc EBP: f352dd50 ESP: f352dd3c
[ 66.876985] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 66.879233] CR0: 80050033 CR2: b7d9f2a0 CR3: 33560d20 CR4: 000406f0
[ 66.881768] Call Trace:
[ 66.883233] putback_inactive_pages+0x13f/0x3b0
[ 66.885256] shrink_inactive_list+0x215/0x530
[ 66.887211] shrink_node_memcg+0x352/0x770
[ 66.889120] ? _cond_resched+0x17/0x30
[ 66.890928] shrink_node+0xc3/0x2f0
[ 66.892748] kswapd+0x257/0x670
[ 66.894395] kthread+0xdb/0x110
[ 66.896078] ? mem_cgroup_shrink_node+0x160/0x160
[ 66.898157] ? kthread_worker_fn+0x160/0x160
[ 66.900153] ret_from_fork+0x19/0x24
[ 66.901902] Code: 39 c8 74 2d 39 c3 74 29 b8 01 00 00 00 83 c4 10 5b 5d c3 89 4c 24 0c 89 5c 24 08 89 54 24 04 c7 04 24 70 aa 8d c1 e8 b1 e5 d2 ff <0f> ff 31 c0 eb dc 89 4c 24 0c 89 5c 24 08 89 44 24 04 c7 04 24
[ 66.909496] ---[ end trace a3882c406e0b0b95 ]---
(...snipped...)
[ 67.964739] ------------[ cut here ]------------
[ 67.966833] list_add corruption. next->prev should be prev (cf64836e), but was 6673909a. (next=b675dd8b).
[ 67.971133] WARNING: CPU: 0 PID: 30 at lib/list_debug.c:25 __list_add_valid+0x3f/0x90
[ 67.974100] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic sd_mod pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi drm crc32c_intel scsi_transport_spi serio_raw ata_piix mptscsih libata e1000 mptbase i2c_core
[ 67.982900] CPU: 0 PID: 30 Comm: kswapd0 Tainted: G W 4.15.0-rc7+ #309
[ 67.985849] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 67.990519] EIP: __list_add_valid+0x3f/0x90
[ 67.992533] EFLAGS: 00010096 CPU: 0
[ 67.994334] EAX: 0000005d EBX: 00000200 ECX: c1b3bce8 EDX: 00000082
[ 67.996837] ESI: f4503f7c EDI: f5573b14 EBP: f352dd50 ESP: f352dd3c
[ 67.999430] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 68.001757] CR0: 80050033 CR2: b7d9f2a0 CR3: 33436e20 CR4: 000406f0
[ 68.004304] Call Trace:
[ 68.005785] putback_inactive_pages+0x13f/0x3b0
[ 68.007812] shrink_inactive_list+0x215/0x530
[ 68.009804] shrink_node_memcg+0x352/0x770
[ 68.011771] ? _cond_resched+0x17/0x30
[ 68.013593] shrink_node+0xc3/0x2f0
[ 68.015379] kswapd+0x257/0x670
[ 68.017047] kthread+0xdb/0x110
[ 68.018674] ? mem_cgroup_shrink_node+0x160/0x160
[ 68.020700] ? kthread_worker_fn+0x160/0x160
[ 68.022667] ret_from_fork+0x19/0x24
[ 68.024436] Code: 39 c8 74 2d 39 c3 74 29 b8 01 00 00 00 83 c4 10 5b 5d c3 89 4c 24 0c 89 5c 24 08 89 54 24 04 c7 04 24 70 aa 8d c1 e8 b1 e5 d2 ff <0f> ff 31 c0 eb dc 89 4c 24 0c 89 5c 24 08 89 44 24 04 c7 04 24
[ 68.031947] ---[ end trace a3882c406e0b0ba3 ]---
[ 68.034203] ------------[ cut here ]------------
[ 68.036504] list_del corruption, b675dd8b->next is LIST_POISON1 (490f1932)
[ 68.039405] WARNING: CPU: 0 PID: 30 at lib/list_debug.c:47 __list_del_entry_valid+0x47/0xa0
[ 68.042497] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic sd_mod pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi drm crc32c_intel scsi_transport_spi serio_raw ata_piix mptscsih libata e1000 mptbase i2c_core
[ 68.051350] CPU: 0 PID: 30 Comm: kswapd0 Tainted: G W 4.15.0-rc7+ #309
[ 68.054343] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 68.059006] EIP: __list_del_entry_valid+0x47/0xa0
[ 68.061197] EFLAGS: 00010086 CPU: 0
[ 68.063002] EAX: 0000003e EBX: f4503f7c ECX: c1b3bce8 EDX: 00000082
[ 68.065590] ESI: f3c60c18 EDI: f352dd90 EBP: f352dd4c ESP: f352dd40
[ 68.068098] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 68.070375] CR0: 80050033 CR2: b7d9f2a0 CR3: 33436e20 CR4: 000406f0
[ 68.072858] Call Trace:
[ 68.074311] isolate_lru_pages.isra.65+0x210/0x360
[ 68.076482] ? putback_inactive_pages+0x13f/0x3b0
[ 68.078583] ? apic_timer_interrupt+0x3c/0x44
[ 68.080537] shrink_active_list+0xb1/0x3a0
[ 68.082502] ? inactive_list_is_low.isra.64+0x12b/0x1f0
[ 68.084688] shrink_node_memcg+0x3a2/0x770
[ 68.086600] shrink_node+0xc3/0x2f0
[ 68.088372] kswapd+0x257/0x670
[ 68.090007] kthread+0xdb/0x110
[ 68.091642] ? mem_cgroup_shrink_node+0x160/0x160
[ 68.093779] ? kthread_worker_fn+0x160/0x160
[ 68.095686] ret_from_fork+0x19/0x24
[ 68.097479] Code: 09 39 c8 75 46 8b 52 04 39 d0 75 25 b8 01 00 00 00 c9 c3 89 44 24 04 c7 44 24 08 00 01 00 00 c7 04 24 40 ab 8d c1 e8 19 e5 d2 ff <0f> ff 31 c0 c9 c3 89 54 24 08 89 44 24 04 c7 04 24 e4 ab 8d c1
[ 68.104937] ---[ end trace a3882c406e0b0ba4 ]---
[ 68.107030] ------------[ cut here ]------------
[ 68.109187] kernel BUG at mm/vmscan.c:1560!
[ 68.111147] invalid opcode: 0000 [#1] SMP
[ 68.112978] Modules linked in: xfs libcrc32c sr_mod cdrom ata_generic sd_mod pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi drm crc32c_intel scsi_transport_spi serio_raw ata_piix mptscsih libata e1000 mptbase i2c_core
[ 68.121527] CPU: 0 PID: 30 Comm: kswapd0 Tainted: G W 4.15.0-rc7+ #309
[ 68.124410] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 68.129258] EIP: isolate_lru_pages.isra.65+0x248/0x360
[ 68.132206] EFLAGS: 00010082 CPU: 0
[ 68.134103] EAX: ffffffea EBX: f4503f7c ECX: f352df68 EDX: 00000000
[ 68.136776] ESI: f3c60c18 EDI: 00000001 EBP: f352ddc8 ESP: f352dd54
[ 68.139404] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 68.141700] CR0: 80050033 CR2: b7d9f2a0 CR3: 33436e20 CR4: 000406f0
[ 68.144202] Call Trace:
[ 68.145656] ? putback_inactive_pages+0x13f/0x3b0
[ 68.147714] ? apic_timer_interrupt+0x3c/0x44
[ 68.149628] shrink_active_list+0xb1/0x3a0
[ 68.151576] ? inactive_list_is_low.isra.64+0x12b/0x1f0
[ 68.153749] shrink_node_memcg+0x3a2/0x770
[ 68.155605] shrink_node+0xc3/0x2f0
[ 68.157303] kswapd+0x257/0x670
[ 68.158884] kthread+0xdb/0x110
[ 68.160500] ? mem_cgroup_shrink_node+0x160/0x160
[ 68.162491] ? kthread_worker_fn+0x160/0x160
[ 68.164366] ret_from_fork+0x19/0x24
[ 68.166050] Code: 10 8b 45 b0 8b 38 89 c2 89 d8 89 f9 e8 02 36 1b 00 84 c0 0f 84 d5 fe ff ff 8b 45 b0 89 5f 04 89 3b 89 43 04 89 18 e9 c3 fe ff ff <0f> 0b 8d b6 00 00 00 00 89 d8 e8 69 36 1b 00 84 c0 74 0a 8b 13
[ 68.173408] EIP: isolate_lru_pages.isra.65+0x248/0x360 SS:ESP: 0068:f352dd54
[ 68.176105] ---[ end trace a3882c406e0b0ba5 ]---
[ 68.178248] Kernel panic - not syncing: Fatal exception
[ 68.180517] Kernel Offset: 0x0 from 0xc1000000 (relocation range: 0xc0000000-0xf81fdfff)
[ 68.183605] Rebooting in 5 seconds..
[ 73.262601] ACPI MEMORY or I/O RESET_REG.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc7] Random oopses under memory pressure.
2018-01-12 1:31 ` [mm " Tetsuo Handa
@ 2018-01-12 1:42 ` Linus Torvalds
2018-01-12 11:22 ` Tetsuo Handa
0 siblings, 1 reply; 59+ messages in thread
From: Linus Torvalds @ 2018-01-12 1:42 UTC (permalink / raw)
To: Tetsuo Handa
Cc: Michal Hocko, linux-mm, the arch/x86 maintainers, linux-fsdevel
On Thu, Jan 11, 2018 at 5:31 PM, Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
>
> Thus, I suspect that somewhere is confusing HighMem pages and !HighMem pages.
Hmm. I can't even imagine how you'd do that.
Sure, if you take page_address() to get a kmap'ed linear address, and
then feed that linear address back to virt_to_page(), you'd certainly
get a crazy page. But that would be insane.. I don't see how you'd do
that.
Hmm. Do you have CONFIG_DEBUG_VIRTUAL enabled? That should catch at
least the above case, it should enable a debugging version of
__virt_to_phys() and use it.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc7] Random oopses under memory pressure.
2018-01-12 1:42 ` Linus Torvalds
@ 2018-01-12 11:22 ` Tetsuo Handa
2018-01-14 11:54 ` Tetsuo Handa
0 siblings, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-12 11:22 UTC (permalink / raw)
To: torvalds, mhocko; +Cc: linux-mm, x86, linux-fsdevel
Tetsuo Handa wrote:
> Michal Hocko wrote:
> > All of those seem to be file pages. So maybe try to use a different FS.
>
> Maybe that's the next thing I should try.
xfs versus ext4 => Both triggers the bug
Linus Torvalds wrote:
> On Thu, Jan 11, 2018 at 5:31 PM, Tetsuo Handa
> <penguin-kernel@i-love.sakura.ne.jp> wrote:
> >
> > Thus, I suspect that somewhere is confusing HighMem pages and !HighMem pages.
>
> Hmm. I can't even imagine how you'd do that.
>
> Sure, if you take page_address() to get a kmap'ed linear address, and
> then feed that linear address back to virt_to_page(), you'd certainly
> get a crazy page. But that would be insane.. I don't see how you'd do
> that.
>
> Hmm. Do you have CONFIG_DEBUG_VIRTUAL enabled? That should catch at
> least the above case, it should enable a debugging version of
> __virt_to_phys() and use it.
>
CONFIG_DEBUG_VIRTUAL=y versus CONFIG_DEBUG_VIRTUAL=n => No difference
No mem= parameter versus mem=768M => Cannot trigger if mem=768M (i.e. not using HighMem)
CONFIG_SLUB=y versus CONFIG_SLAB=y => Both triggers the bug
CONFIG_DEBUG_PAGEALLOC=y versus CONFIG_DEBUG_PAGEALLOC=n => Cannot trigger if CONFIG_DEBUG_PAGEALLOC=n
I don't know whether there is a bug in CONFIG_DEBUG_PAGEALLOC=y code.
Config is at http://I-love.SAKURA.ne.jp/tmp/config-4.15-rc7-min .
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc7] Random oopses under memory pressure.
2018-01-12 11:22 ` Tetsuo Handa
@ 2018-01-14 11:54 ` Tetsuo Handa
2018-01-15 23:05 ` Linus Torvalds
0 siblings, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-14 11:54 UTC (permalink / raw)
To: linux-kernel, linux-mm; +Cc: x86, linux-fsdevel, torvalds, mhocko
This memory corruption bug occurs even on CONFIG_SMP=n CONFIG_PREEMPT_NONE=y
kernel. This bug highly depends on timing and thus too difficult to bisect.
This bug seems to exist at least since Linux 4.8 (judging from the traces, though
the cause might be different). None of debugging configuration gives me a clue.
So far only CONFIG_HIGHMEM=y CONFIG_DEBUG_PAGEALLOC=y kernel (with RAM enough to
use HighMem: zone) seems to hit this bug, but it might be just by chance caused
by timings. Thus, there is no evidence that 64bit kernels are not affected by
this bug. But I can't narrow down any more. Thus, I call for developers who can
narrow down / identify where the memory corruption bug is.
Depends on block device?
Can't reproduce if executing reproducer from initramfs.
But maybe this is due to timing dependent.
Depends on dynamic link?
Can't reproduce if reproducer is statically linked.
But maybe this is due to timing dependent.
Depends on hypervisor?
Can reproduce on both VMware and QEMU.
Depends on filesystem?
Can reproduce on xfs, ext4 and cramfs.
Below is complete procedure for reproducing this bug using minimal kernel
config tuned for QEMU running on x86_64 CentOS 7.4 host.
(1) Create a disk image with minimal contents.
qemu-img create -f raw /mnt/disk1.img 128M
mkfs.xfs /mnt/disk1.img
mount -o loop /mnt/disk1.img /mnt/
gcc -Wall -O3 -m32 -o /mnt/init -x c - << "EOF"
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mount.h>
#include <sys/wait.h>
int main(int argc, char *argv[])
{
int fd;
mount("/proc", "/proc", "proc", 0, NULL);
fd = open("/proc/sys/vm/oom_dump_tasks", O_WRONLY);
write(fd, "0\n", 2);
close(fd);
while (1) {
write(1, "Starting a.out\n", 15);
if (fork() == 0) {
execlp("/a.out", "/a.out", NULL);
write(1, "Failed\n", 7);
_exit(0);
}
wait(NULL);
}
return 0;
}
EOF
gcc -Wall -O3 -m32 -o /mnt/a.out -x c - << "EOF"
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char *argv[])
{
if (argc != 1) {
unsigned long long size;
char *buf = NULL;
unsigned long long i;
for (size = 1048576; size < 512ULL * (1 << 30); size += 1048576) {
char *cp = realloc(buf, size);
if (!cp) {
size -= 1048576;
break;
}
buf = cp;
}
for (i = 0; i < size; i += 4096)
buf[i] = 0;
_exit(0);
} else
while (1)
if (fork() == 0)
execlp(argv[0], argv[0], "", NULL);
return 0;
}
EOF
mkdir /mnt/lib /mnt/proc /mnt/dev
cp -pL /lib/libc.so.6 /lib/ld-linux.so.2 /mnt/lib/
umount -d /mnt/
(2) Build a 32bit kernel and boot it with the disk image.
cd /path/to/linux.git
wget -O .config http://I-love.SAKURA.ne.jp/tmp/config-4.15-rc7-min-qemu
make -s
/usr/libexec/qemu-kvm -no-kvm -machine pc -cpu kvm32 -smp 1 -m 2048 --no-reboot --kernel arch/x86/boot/bzImage --nographic --append "ro console=ttyS0,115200n8 root=/dev/vda init=/init" -drive file=/mnt/disk1.img,if=virtio
An example result is shown below.
[ 0.000000] Linux version 4.15.0-rc7+ (root@ccsecurity) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)) #220 Sun Jan 14 20:36:37 JST 2018
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007fffbfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000007fffc000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] Notice: NX (Execute Disable) protection missing in CPU!
[ 0.000000] random: fast init done
[ 0.000000] SMBIOS 2.4 present.
[ 0.000000] DMI: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 0.000000] e820: last_pfn = 0x7fffc max_arch_pfn = 0x100000
[ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WC UC- UC
[ 0.000000] found SMP MP-table at [mem 0x000f7300-0x000f730f] mapped at [(ptrval)]
[ 0.000000] Scanning 1 areas for low memory corruption
[ 0.000000] ACPI: Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 0x00000000000F7160 000014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 0x000000007FFFFA9B 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 0x000000007FFFF177 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 0x000000007FFFE040 001137 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 0x000000007FFFE000 000040
[ 0.000000] ACPI: SSDT 0x000000007FFFF1EB 000838 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: APIC 0x000000007FFFFA23 000078 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] 1159MB HIGHMEM available.
[ 0.000000] 887MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 377fe000
[ 0.000000] low ram: 0 - 377fe000
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] Normal [mem 0x0000000001000000-0x00000000377fdfff]
[ 0.000000] HighMem [mem 0x00000000377fe000-0x000000007fffbfff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.000000] node 0: [mem 0x0000000000100000-0x000000007fffbfff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000007fffbfff]
[ 0.000000] Reserved but unavailable: 98 pages
[ 0.000000] Using APIC driver default
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] IOAPIC[0]: apic_id 0 already used, trying 1
[ 0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] e820: [mem 0x80000000-0xfffbffff] available for PCI devices
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 521966
[ 0.000000] Kernel command line: ro console=ttyS0,115200n8 root=/dev/vda init=/init
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Initializing CPU#0
[ 0.000000] Initializing HighMem for node 0 (000377fe:0007fffc)
[ 0.000000] Initializing Movable for node 0 (00000000:00000000)
[ 0.000000] Memory: 2064228K/2096744K available (3330K kernel code, 256K rwdata, 916K rodata, 380K init, 5308K bss, 32516K reserved, 0K cma-reserved, 1187832K highmem)
[ 0.000000] virtual kernel memory layout:
[ 0.000000] fixmap : 0xfffa4000 - 0xfffff000 ( 364 kB)
[ 0.000000] cpu_entry : 0xffc00000 - 0xffc28000 ( 160 kB)
[ 0.000000] pkmap : 0xff800000 - 0xffc00000 (4096 kB)
[ 0.000000] vmalloc : 0xf7ffe000 - 0xff7fe000 ( 120 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf77fe000 ( 887 MB)
[ 0.000000] .init : 0xc1474000 - 0xc14d3000 ( 380 kB)
[ 0.000000] .data : 0xc1340b0a - 0xc14693a0 (1186 kB)
[ 0.000000] .text : 0xc1000000 - 0xc1340b0a (3330 kB)
[ 0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok.
[ 0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS: 2304, nr_irqs: 256, preallocated irqs: 16
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [ttyS0] enabled
[ 0.000000] ACPI: Core revision 20170831
[ 0.000000] ACPI: 2 ACPI AML tables successfully acquired and loaded
[ 0.002000] APIC: Switch to symmetric I/O mode setup
[ 0.002000] Enabling APIC mode: Flat. Using 1 I/O APICs
[ 0.003000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.008000] tsc: Fast TSC calibration using PIT
[ 0.009000] tsc: Detected 2793.513 MHz processor
[ 0.010000] Calibrating delay loop (skipped), value calculated using timer frequency.. 5587.02 BogoMIPS (lpj=2793513)
[ 0.010000] pid_max: default: 32768 minimum: 301
[ 0.011243] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.011590] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.019489] mce: CPU supports 10 MCE banks
[ 0.020000] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.020000] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 0.020000] CPU: Intel Common 32-bit KVM processor (family: 0xf, model: 0x6, stepping: 0x1)
[ 0.029000] Performance Events: unsupported Netburst CPU model 6 no PMU driver, software events only.
[ 0.032000] APIC calibration not consistent with PM-Timer: 193ms instead of 100ms
[ 0.032000] APIC delta adjusted to PM-Timer: 6250001 (12069151)
[ 0.035000] devtmpfs: initialized
[ 0.037160] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 0.037567] futex hash table entries: 256 (order: -1, 3072 bytes)
[ 0.042266] cpuidle: using governor menu
[ 0.043000] ACPI: bus type PCI registered
[ 0.044933] PCI: PCI BIOS revision 2.10 entry at 0xfd54b, last bus=0
[ 0.045000] PCI: Using configuration type 1 for base access
[ 0.051898] HugeTLB registered 4.00 MiB page size, pre-allocated 0 pages
[ 0.054220] ACPI: Added _OSI(Module Device)
[ 0.054416] ACPI: Added _OSI(Processor Device)
[ 0.054553] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.054696] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.071000] ACPI: Interpreter enabled
[ 0.071000] ACPI: (supports S0 S5)
[ 0.071000] ACPI: Using IOAPIC for interrupt routing
[ 0.071516] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.073000] ACPI: Enabled 16 GPEs in block 00 to 0F
[ 0.094000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.094000] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
[ 0.094128] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 0.095318] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 0.099000] PCI host bridge to bus 0000:00
[ 0.099211] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.099484] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.099616] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 0.099731] pci_bus 0000:00: root bus resource [mem 0x80000000-0xfebfffff window]
[ 0.099926] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.106000] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 0.106078] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 0.106325] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 0.106486] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 0.108636] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 0.108866] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 0.127454] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[ 0.128000] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[ 0.128508] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[ 0.129000] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[ 0.129574] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
[ 0.134000] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[ 0.134000] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[ 0.134068] pci 0000:00:02.0: vgaarb: bridge control possible
[ 0.134233] vgaarb: loaded
[ 0.135514] SCSI subsystem initialized
[ 0.136526] PCI: Using ACPI for IRQ routing
[ 0.137000] clocksource: Switched to clocksource refined-jiffies
[ 0.138266] ACPI: Failed to create genetlink family for ACPI event
[ 0.139337] pnp: PnP ACPI init
[ 0.144998] pnp: PnP ACPI: found 5 devices
[ 0.172994] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.173217] clocksource: Switched to clocksource acpi_pm
[ 0.173994] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 0.173994] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 0.173994] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 0.173994] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 0.183369] Scanning for low memory corruption every 60 seconds
[ 0.189392] workingset: timestamp_bits=14 max_order=19 bucket_order=5
[ 0.204712] zbud: loaded
[ 0.210911] SGI XFS with ACLs, security attributes, no debug enabled
[ 0.220200] bounce: pool size: 64 pages
[ 0.220200] io scheduler noop registered (default)
[ 0.229848] atomic64_test: passed for i586+ platform with CX8 and with SSE
[ 0.236151] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.237081] ACPI: Power Button [PWRF]
[ 0.248193] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
[ 0.248193] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
[ 0.254507] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.277797] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 0.293374] Non-volatile memory driver v1.3
[ 0.337929] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[ 0.342766] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.344004] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.347349] Using IPI Shortcut mode
[ 0.347732] sched_clock: Marking stable (347235515, 0)->(709645618, -362410103)
[ 0.350608] page_owner is disabled
[ 0.375161] XFS (vda): Mounting V5 Filesystem
[ 0.434199] XFS (vda): Ending clean mount
[ 0.466036] VFS: Mounted root (xfs filesystem) readonly on device 254:0.
[ 0.468423] devtmpfs: mounted
[ 0.469817] debug: unmapping init [mem 0xc1474000-0xc14d2fff]
[ 0.471908] Write protecting the kernel text: 3332k
[ 0.472225] Write protecting the kernel read-only data: 928k
Starting a.out
[ 1.184498] tsc: Refined TSC clocksource calibration: 2793.541 MHz
[ 1.184788] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x28446b3c7b0, max_idle_ns: 440795331399 ns
[ 2.262012] clocksource: Switched to clocksource tsc
/a.out: error while loading shared libraries: cannot create cache for search path: Cannot allocate memory
/a.out: error while loading shared libraries: libc.so.6: failed to map segment from shared object: Cannot allocate memory
/a.out: error while loading shared libraries: libc.so.6: failed to map segment from shared object: Cannot allocate memory
cannot allocate TLS data structures for initial thread/a.out: error while loading shared libraries: cannot create cache for search path: Cannot allocate memory
/a.out: error while loading shared libraries: libc.so.6: failed to map segment from shared object: Cannot allocate memory
/a.out: error while loading shared libraries: libc.so.6: failed to map segment from shared object: Cannot allocate memory
/a.out: error while loading shared libraries: libc.so.6: failed to map segment from shared object: Cannot allocate memory
[ 40.328889] a.out invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0
[ 40.329204] CPU: 0 PID: 144 Comm: a.out Not tainted 4.15.0-rc7+ #220
[ 40.329371] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 40.329676] Call Trace:
[ 40.330675] dump_stack+0x16/0x24
[ 40.330901] dump_header+0x64/0x211
[ 40.330991] ? irq_exit+0x2a/0x90
[ 40.331052] ? smp_apic_timer_interrupt+0x45/0x80
[ 40.331122] oom_kill_process+0x1ec/0x400
[ 40.331183] ? has_capability_noaudit+0x1f/0x30
[ 40.331248] ? oom_badness+0xc6/0x140
[ 40.331365] ? oom_evaluate_task+0xa2/0xe0
[ 40.331561] out_of_memory+0xe0/0x270
[ 40.331625] __alloc_pages_nodemask+0x6a3/0x830
[ 40.331697] handle_mm_fault+0xa5e/0xdd0
[ 40.331768] ? slow_virt_to_phys+0x2b/0x90
[ 40.331886] __do_page_fault+0x194/0x420
[ 40.331969] ? vmalloc_sync_all+0x150/0x150
[ 40.332040] do_page_fault+0xb/0xd
[ 40.332093] common_exception+0x6d/0x72
[ 40.332599] EIP: 0x8048437
[ 40.332948] EFLAGS: 00000202 CPU: 0
[ 40.333046] EAX: 007a3000 EBX: 7ff00000 ECX: 3862e008 EDX: 00000000
[ 40.333479] ESI: 7ff00000 EDI: 00000000 EBP: bfd4b908 ESP: bfd4b8d0
[ 40.333643] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 40.333995] Mem-Info:
[ 40.334767] active_anon:483450 inactive_anon:0 isolated_anon:0
[ 40.334767] active_file:0 inactive_file:10 isolated_file:55
[ 40.334767] unevictable:0 dirty:0 writeback:0 unstable:0
[ 40.334767] slab_reclaimable:26 slab_unreclaimable:919
[ 40.334767] mapped:40 shmem:0 pagetables:8018 bounce:0
[ 40.334767] free:22250 free_pcp:190 free_cma:0
[ 40.335681] Node 0 active_anon:1933800kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):220kB mapped:160kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[ 40.336482] DMA free:8788kB min:788kB low:984kB high:1180kB active_anon:7128kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15916kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 40.336893] lowmem_reserve[]: 0 840 2000 2000
[ 40.337028] Normal free:79804kB min:42744kB low:53428kB high:64112kB active_anon:739736kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:892920kB managed:860480kB mlocked:0kB kernel_stack:2400kB pagetables:32072kB bounce:0kB free_pcp:640kB local_pcp:640kB free_cma:0kB
[ 40.337567] lowmem_reserve[]: 0 0 9279 9279
[ 40.337666] HighMem free:408kB min:512kB low:15260kB high:30008kB active_anon:1186936kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB writepending:0kB present:1187832kB managed:1187832kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:120kB local_pcp:120kB free_cma:0kB
[ 40.338051] lowmem_reserve[]: 0 0 0 0
[ 40.338159] DMA: 1*4kB (U) 2*8kB (UM) 0*16kB 2*32kB (UM) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 2*1024kB (UM) 1*2048kB (U) 1*4096kB (M) = 8788kB
[ 40.338781] Normal: 11*4kB (U) 14*8kB (UE) 6*16kB (UME) 2*32kB (UM) 0*64kB 1*128kB (E) 2*256kB (ME) 2*512kB (ME) 2*1024kB (ME) 1*2048kB (E) 18*4096kB (UM) = 79804kB
[ 40.339083] HighMem: 2*4kB (U) 2*8kB (UM) 2*16kB (U) 1*32kB (M) 1*64kB (M) 0*128kB 1*256kB (M) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 408kB
[ 40.339563] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=4096kB
[ 40.339724] 39 total pagecache pages
[ 40.339945] 524186 pages RAM
[ 40.340032] 296958 pages HighMem/MovableOnly
[ 40.340380] 8129 pages reserved
[ 40.340526] Out of memory: Kill process 41 (a.out) score 12 or sacrifice child
[ 40.341733] Killed process 41 (a.out) total-vm:2099260kB, anon-rss:26664kB, file-rss:64kB, shmem-rss:0kB
[ 40.351554] clocksource: timekeeping watchdog on CPU0: Marking clocksource 'tsc' as unstable because the skew is too large:
[ 40.351854] clocksource: 'acpi_pm' wd_now: ea5383 wd_last: 1125e2 mask: ffffff
[ 40.352083] clocksource: 'tsc' cs_now: 1b2db1039e cs_last: 158b342a09 mask: ffffffffffffffff
[ 40.352476] tsc: Marking TSC unstable due to clocksource watchdog
[ 40.353017] TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'.
[ 40.353325] sched_clock: Marking unstable (40352968436, 26691)<-(40715409249, -362410103)
[ 40.392298] clocksource: Switched to clocksource acpi_pm
[ 40.683413] a.out invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0
[ 40.683680] CPU: 0 PID: 131 Comm: a.out Not tainted 4.15.0-rc7+ #220
[ 40.683766] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 40.683842] Call Trace:
[ 40.683901] dump_stack+0x16/0x24
[ 40.683959] dump_header+0x64/0x211
[ 40.684016] ? irq_exit+0x2a/0x90
[ 40.684219] ? smp_apic_timer_interrupt+0x45/0x80
[ 40.684356] oom_kill_process+0x1ec/0x400
[ 40.684520] ? has_capability_noaudit+0x1f/0x30
[ 40.684694] ? oom_badness+0xc6/0x140
[ 40.684752] ? oom_evaluate_task+0x14/0xe0
[ 40.684813] out_of_memory+0xe0/0x270
[ 40.684871] __alloc_pages_nodemask+0x6a3/0x830
[ 40.684942] handle_mm_fault+0xa5e/0xdd0
[ 40.685002] ? slow_virt_to_phys+0x2b/0x90
[ 40.685410] __do_page_fault+0x194/0x420
[ 40.685486] ? vmalloc_sync_all+0x150/0x150
[ 40.685692] do_page_fault+0xb/0xd
[ 40.685777] common_exception+0x6d/0x72
[ 40.685848] EIP: 0x8048437
[ 40.685910] EFLAGS: 00000202 CPU: 0
[ 40.685981] EAX: 009f9000 EBX: 7ff00000 ECX: 38876008 EDX: 00000000
[ 40.686533] ESI: 7ff00000 EDI: 00000000 EBP: bfc00b38 ESP: bfc00b00
[ 40.686715] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 40.686831] Mem-Info:
[ 40.686919] active_anon:483543 inactive_anon:0 isolated_anon:0
[ 40.686919] active_file:31 inactive_file:25 isolated_file:35
[ 40.686919] unevictable:0 dirty:0 writeback:0 unstable:0
[ 40.686919] slab_reclaimable:26 slab_unreclaimable:919
[ 40.686919] mapped:83 shmem:0 pagetables:7986 bounce:0
[ 40.686919] free:22233 free_pcp:120 free_cma:0
[ 40.687885] Node 0 active_anon:1934172kB inactive_anon:0kB active_file:124kB inactive_file:100kB unevictable:0kB isolated(anon):0kB isolated(file):140kB mapped:332kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[ 40.689342] DMA free:8788kB min:788kB low:984kB high:1180kB active_anon:7128kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15916kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 40.689842] lowmem_reserve[]: 0 840 2000 2000
[ 40.689972] Normal free:79744kB min:42744kB low:53428kB high:64112kB active_anon:740068kB inactive_anon:0kB active_file:88kB inactive_file:28kB unevictable:0kB writepending:0kB present:892920kB managed:860480kB mlocked:0kB kernel_stack:2408kB pagetables:31944kB bounce:0kB free_pcp:296kB local_pcp:296kB free_cma:0kB
[ 40.690726] lowmem_reserve[]: 0 0 9279 9279
[ 40.690828] HighMem free:400kB min:512kB low:15260kB high:30008kB active_anon:1186976kB inactive_anon:0kB active_file:36kB inactive_file:72kB unevictable:0kB writepending:0kB present:1187832kB managed:1187832kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:184kB local_pcp:184kB free_cma:0kB
[ 40.691461] lowmem_reserve[]: 0 0 0 0
[ 40.691579] DMA: 1*4kB (U) 2*8kB (UM) 0*16kB 2*32kB (UM) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 2*1024kB (UM) 1*2048kB (U) 1*4096kB (M) = 8788kB
[ 40.691832] Normal: 42*4kB (UM) 35*8kB (UME) 12*16kB (UE) 6*32kB (UM) 1*64kB (M) 2*128kB (ME) 1*256kB (E) 1*512kB (E) 2*1024kB (ME) 1*2048kB (E) 18*4096kB (UM) = 79744kB
[ 40.692432] HighMem: 2*4kB (U) 1*8kB (U) 2*16kB (U) 1*32kB (M) 1*64kB (M) 0*128kB 1*256kB (M) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 400kB
[ 40.692714] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=4096kB
[ 40.692831] 82 total pagecache pages
[ 40.692887] 524186 pages RAM
[ 40.692971] 296958 pages HighMem/MovableOnly
[ 40.693033] 8129 pages reserved
[ 40.693528] Out of memory: Kill process 50 (a.out) score 10 or sacrifice child
[ 40.693705] Killed process 50 (a.out) total-vm:2099260kB, anon-rss:22284kB, file-rss:40kB, shmem-rss:0kB
[ 41.251822] a.out invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0
[ 41.252049] CPU: 0 PID: 217 Comm: a.out Not tainted 4.15.0-rc7+ #220
[ 41.252139] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 41.252215] Call Trace:
[ 41.252274] dump_stack+0x16/0x24
[ 41.252331] dump_header+0x64/0x211
[ 41.252387] ? irq_exit+0x2a/0x90
[ 41.252500] ? smp_apic_timer_interrupt+0x45/0x80
[ 41.253033] oom_kill_process+0x1ec/0x400
[ 41.253128] ? has_capability_noaudit+0x1f/0x30
[ 41.253220] ? oom_badness+0xc6/0x140
[ 41.253299] ? oom_evaluate_task+0x14/0xe0
[ 41.253374] out_of_memory+0xe0/0x270
[ 41.253489] __alloc_pages_nodemask+0x6a3/0x830
[ 41.253747] handle_mm_fault+0xa5e/0xdd0
[ 41.253825] ? slow_virt_to_phys+0x2b/0x90
[ 41.253897] __do_page_fault+0x194/0x420
[ 41.253985] ? vmalloc_sync_all+0x150/0x150
[ 41.254055] do_page_fault+0xb/0xd
[ 41.254116] common_exception+0x6d/0x72
[ 41.254181] EIP: 0x8048437
[ 41.254228] EFLAGS: 00000202 CPU: 0
[ 41.254357] EAX: 00196000 EBX: 7ff00000 ECX: 37fe4008 EDX: 00000000
[ 41.254764] ESI: 7ff00000 EDI: 00000000 EBP: bfd59aa8 ESP: bfd59a70
[ 41.254917] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 41.255018] Mem-Info:
[ 41.255121] active_anon:483725 inactive_anon:0 isolated_anon:0
[ 41.255121] active_file:0 inactive_file:0 isolated_file:1
[ 41.255121] unevictable:0 dirty:0 writeback:0 unstable:0
[ 41.255121] slab_reclaimable:26 slab_unreclaimable:919
[ 41.255121] mapped:2 shmem:0 pagetables:7936 bounce:0
[ 41.255121] free:22269 free_pcp:60 free_cma:0
[ 41.256137] Node 0 active_anon:1934900kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):4kB mapped:8kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[ 41.256637] DMA free:8788kB min:788kB low:984kB high:1180kB active_anon:7128kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15916kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 41.257064] lowmem_reserve[]: 0 840 2000 2000
[ 41.257346] Normal free:79808kB min:42744kB low:53428kB high:64112kB active_anon:740620kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:892920kB managed:860480kB mlocked:0kB kernel_stack:2344kB pagetables:31744kB bounce:0kB free_pcp:120kB local_pcp:120kB free_cma:0kB
[ 41.257816] lowmem_reserve[]: 0 0 9279 9279
[ 41.257934] HighMem free:480kB min:512kB low:15260kB high:30008kB active_anon:1187152kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:1187832kB managed:1187832kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:120kB local_pcp:120kB free_cma:0kB
[ 41.258204] lowmem_reserve[]: 0 0 0 0
[ 41.258204] DMA: 1*4kB (U) 2*8kB (UM) 0*16kB 2*32kB (UM) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 2*1024kB (UM) 1*2048kB (U) 1*4096kB (M) = 8788kB
[ 41.259035] Normal: 98*4kB (UM) 43*8kB (UE) 16*16kB (UME) 5*32kB (U) 1*64kB (U) 2*128kB (ME) 2*256kB (ME) 2*512kB (ME) 1*1024kB (E) 1*2048kB (E) 18*4096kB (UM) = 79808kB
[ 41.259312] HighMem: 2*4kB (U) 1*8kB (U) 3*16kB (UM) 1*32kB (M) 0*64kB 1*128kB (M) 1*256kB (M) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 480kB
[ 41.259782] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=4096kB
[ 41.259903] 1 total pagecache pages
[ 41.259975] 524186 pages RAM
[ 41.260022] 296958 pages HighMem/MovableOnly
[ 41.260084] 8129 pages reserved
[ 41.260137] Out of memory: Kill process 53 (a.out) score 10 or sacrifice child
[ 41.260259] Killed process 53 (a.out) total-vm:2099260kB, anon-rss:21276kB, file-rss:8kB, shmem-rss:0kB
[ 42.305105] a.out invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0
[ 42.305302] CPU: 0 PID: 224 Comm: a.out Not tainted 4.15.0-rc7+ #220
[ 42.305388] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 42.305667] Call Trace:
[ 42.305732] dump_stack+0x16/0x24
[ 42.305790] dump_header+0x64/0x211
[ 42.305847] ? get_page_from_freelist+0x106/0xb30
[ 42.305932] oom_kill_process+0x1ec/0x400
[ 42.305995] ? has_capability_noaudit+0x1f/0x30
[ 42.306060] ? oom_badness+0xc6/0x140
[ 42.306115] ? oom_evaluate_task+0x14/0xe0
[ 42.306175] out_of_memory+0xe0/0x270
[ 42.306231] __alloc_pages_nodemask+0x6a3/0x830
[ 42.306302] handle_mm_fault+0xa5e/0xdd0
[ 42.306361] ? slow_virt_to_phys+0x2b/0x90
[ 42.306485] __do_page_fault+0x194/0x420
[ 42.306725] ? vmalloc_sync_all+0x150/0x150
[ 42.306795] do_page_fault+0xb/0xd
[ 42.306870] common_exception+0x6d/0x72
[ 42.306961] EIP: 0x8048437
[ 42.307003] EFLAGS: 00000202 CPU: 0
[ 42.307053] EAX: 0009c000 EBX: 7e600000 ECX: 37f31008 EDX: 00000000
[ 42.307133] ESI: 7e600000 EDI: 00000000 EBP: bf8660b8 ESP: bf866080
[ 42.307215] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 42.307287] Mem-Info:
[ 42.307391] active_anon:483682 inactive_anon:0 isolated_anon:0
[ 42.307391] active_file:0 inactive_file:3 isolated_file:17
[ 42.307391] unevictable:0 dirty:1 writeback:0 unstable:0
[ 42.307391] slab_reclaimable:26 slab_unreclaimable:919
[ 42.307391] mapped:17 shmem:0 pagetables:7904 bounce:0
[ 42.307391] free:22274 free_pcp:125 free_cma:0
[ 42.308066] Node 0 active_anon:1934728kB inactive_anon:0kB active_file:0kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):68kB mapped:68kB dirty:4kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[ 42.308390] DMA free:8788kB min:788kB low:984kB high:1180kB active_anon:7128kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15916kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 42.308810] lowmem_reserve[]: 0 840 2000 2000
[ 42.308948] Normal free:79832kB min:42744kB low:53428kB high:64112kB active_anon:740776kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:892920kB managed:860480kB mlocked:0kB kernel_stack:2328kB pagetables:31616kB bounce:0kB free_pcp:124kB local_pcp:124kB free_cma:0kB
[ 42.309038] lowmem_reserve[]: 0 0 9279 9279
[ 42.309823] HighMem free:476kB min:512kB low:15260kB high:30008kB active_anon:1186824kB inactive_anon:0kB active_file:0kB inactive_file:12kB unevictable:0kB writepending:4kB present:1187832kB managed:1187832kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:376kB local_pcp:376kB free_cma:0kB
[ 42.310231] lowmem_reserve[]: 0 0 0 0
[ 42.310338] DMA: 1*4kB (U) 2*8kB (UM) 0*16kB 2*32kB (UM) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 2*1024kB (UM) 1*2048kB (U) 1*4096kB (M) = 8788kB
[ 42.310838] Normal: 104*4kB (U) 47*8kB (UME) 18*16kB (UME) 7*32kB (U) 1*64kB (U) 1*128kB (E) 2*256kB (ME) 2*512kB (ME) 1*1024kB (E) 1*2048kB (E) 18*4096kB (UM) = 79832kB
[ 42.311455] HighMem: 3*4kB (UM) 2*8kB (UM) 2*16kB (U) 1*32kB (M) 0*64kB 1*128kB (M) 1*256kB (M) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 476kB
[ 42.311775] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=4096kB
[ 42.311935] 20 total pagecache pages
[ 42.312011] 524186 pages RAM
[ 42.312090] 296958 pages HighMem/MovableOnly
[ 42.312319] 8129 pages reserved
[ 42.312413] Out of memory: Kill process 36 (a.out) score 9 or sacrifice child
[ 42.312583] Killed process 36 (a.out) total-vm:2099260kB, anon-rss:21080kB, file-rss:4kB, shmem-rss:0kB
[ 42.366107] BUG: unable to handle kernel NULL pointer dereference at 00000200
[ 42.366346] IP: mpage_readpages+0x8b/0x160
[ 42.366431] *pde = 00000000
[ 42.366599] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 42.366599] CPU: 0 PID: 308 Comm: a.out Not tainted 4.15.0-rc7+ #220
[ 42.366599] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 42.366599] EIP: mpage_readpages+0x8b/0x160
[ 42.366599] EFLAGS: 00000206 CPU: 0
[ 42.366599] EAX: d85afe28 EBX: 00000200 ECX: d85afe30 EDX: 00000100
[ 42.366599] ESI: 00000011 EDI: 000001ec EBP: d85afdf0 ESP: d85afd74
[ 42.366599] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068
[ 42.366599] CR0: 80050033 CR2: 00000200 CR3: 185a0000 CR4: 000006d0
[ 42.366599] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 42.366599] DR6: 00000000 DR7: 00000000
[ 42.366599] Call Trace:
[ 42.366599] ? xfs_setfilesize_trans_alloc.isra.29+0x80/0x80
[ 42.366599] ? xfs_setfilesize_trans_alloc.isra.29+0x80/0x80
[ 42.366599] ? __radix_tree_lookup+0x6e/0xd0
[ 42.366599] ? xfs_map_at_offset+0x30/0x30
[ 42.366599] xfs_vm_readpages+0x19/0x20
[ 42.366599] ? xfs_setfilesize_trans_alloc.isra.29+0x80/0x80
[ 42.366599] __do_page_cache_readahead+0x158/0x1f0
[ 42.366599] filemap_fault+0x299/0x510
[ 42.366599] ? page_add_file_rmap+0xfb/0x150
[ 42.366599] ? unlock_page+0x30/0x30
[ 42.366599] ? filemap_map_pages+0x265/0x2e0
[ 42.366599] __xfs_filemap_fault.isra.18+0x2d/0xb0
[ 42.366599] xfs_filemap_fault+0xa/0x10
[ 42.366599] __do_fault+0x11/0x30
[ 42.366599] ? unlock_page+0x30/0x30
[ 42.366599] handle_mm_fault+0x7d4/0xdd0
[ 42.366599] __do_page_fault+0x194/0x420
[ 42.366599] ? vmalloc_sync_all+0x150/0x150
[ 42.366599] do_page_fault+0xb/0xd
[ 42.366599] common_exception+0x6d/0x72
[ 42.366599] EIP: 0xb7efd1e0
[ 42.366599] EFLAGS: 00000246 CPU: 0
[ 42.366599] EAX: b7e42790 EBX: b7f85000 ECX: bfe5d02c EDX: bfe5d030
[ 42.366599] ESI: 00000000 EDI: b7f85404 EBP: bfe5d058 ESP: bfe5cffc
[ 42.366599] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 42.366599] Code: 00 00 eb 1a 8d 74 26 00 8b 03 a8 01 0f 85 cc 00 00 00 89 f8 ff 48 10 74 77 83 ee 01 74 7c 8b 45 a4 8b 58 04 8d 7b ec 8d 74 26 00 <8b> 13 8b 43 04 89 42 04 89 10 8b 45 a8 8b 4b f4 8b 55 a0 c7 03
[ 42.366599] EIP: mpage_readpages+0x8b/0x160 SS:ESP: 0068:d85afd74
[ 42.366599] CR2: 0000000000000200
[ 42.371210] ---[ end trace c88beaeceebe4c9b ]---
[ 42.371357] Kernel panic - not syncing: Fatal exception
[ 42.371508] Kernel Offset: disabled
[ 42.371617] Rebooting in 1 seconds..
Another example is shown below.
[ 31.067600] BUG: Bad page state in process a.out pfn:7fe84
[ 31.067817] page:f63884a0 count:0 mapcount:2 mapping:f5818f94 index:0x1
[ 31.067955] flags: 0x7e000000()
[ 31.068198] raw: 7e000000 f5818f94 00000001 00000001 00000000 00000100 00000200 00000000
[ 31.068198] raw: 00000000 00000000
[ 31.068198] page dumped because: non-NULL mapping
[ 31.068198] CPU: 0 PID: 47 Comm: a.out Not tainted 4.15.0-rc7+ #220
[ 31.068198] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 31.068198] Call Trace:
[ 31.068198] dump_stack+0x16/0x24
[ 31.068198] bad_page+0x106/0x122
[ 31.068198] free_pages_check_bad+0x5b/0x5e
[ 31.068198] free_pcppages_bulk+0x3c4/0x3d0
[ 31.068198] free_unref_page_commit.isra.117+0x6c/0x80
[ 31.068198] free_unref_page_list+0xd9/0x110
[ 31.068198] release_pages+0x208/0x2c0
[ 31.068198] tlb_flush_mmu_free+0x2f/0x50
[ 31.068198] arch_tlb_finish_mmu+0x30/0x60
[ 31.068198] tlb_finish_mmu+0x24/0x40
[ 31.068198] exit_mmap+0x99/0x130
[ 31.068198] mmput+0x36/0xb0
[ 31.068198] do_exit+0x199/0x8d0
[ 31.068198] ? recalc_sigpending+0x11/0x40
[ 31.068198] ? __alloc_pages_nodemask+0x1d8/0x830
[ 31.068198] do_group_exit+0x2a/0x70
[ 31.068198] get_signal+0x121/0x460
[ 31.068198] ? ktime_get+0x47/0xf0
[ 31.068198] do_signal+0x24/0x590
[ 31.068198] ? native_sched_clock+0x35/0x100
[ 31.068198] ? __phys_addr+0x32/0x70
[ 31.068198] ? __schedule+0x126/0x36b
[ 31.068198] ? __do_page_fault+0x1b0/0x420
[ 31.068198] exit_to_usermode_loop+0x32/0x5e
[ 31.068198] ? vmalloc_sync_all+0x150/0x150
[ 31.068198] prepare_exit_to_usermode+0x43/0x70
[ 31.068198] ? vmalloc_sync_all+0x150/0x150
[ 31.068198] resume_userspace+0x8/0xd
[ 31.068198] EIP: 0x8048437
[ 31.068198] EFLAGS: 00000202 CPU: 0
[ 31.068198] EAX: 01659000 EBX: 7ff00000 ECX: 394f7008 EDX: 00000000
[ 31.068198] ESI: 7ff00000 EDI: 00000000 EBP: bff1d698 ESP: bff1d660
[ 31.068198] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 31.068198] Disabling lock debugging due to kernel taint
Yet another example is shown below.
[ 28.211801] BUG: unable to handle kernel paging request at 1a8a306a
[ 28.212034] IP: page_remove_rmap+0x17/0x280
[ 28.212269] *pde = 00000000
[ 28.212447] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 28.212614] CPU: 0 PID: 54 Comm: a.out Not tainted 4.15.0-rc7+ #220
[ 28.212767] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 28.212901] EIP: page_remove_rmap+0x17/0x280
[ 28.212981] EFLAGS: 00000206 CPU: 0
[ 28.213061] EAX: 1a8a3066 EBX: f52c2210 ECX: 00000051 EDX: 00000000
[ 28.213203] ESI: 148da045 EDI: f63884c8 EBP: f5ea1b10 ESP: f5ea1b04
[ 28.213410] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068
[ 28.213562] CR0: 80050033 CR2: 1a8a306a CR3: 35404000 CR4: 000006d0
[ 28.213762] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 28.213942] DR6: 00000000 DR7: 00000000
[ 28.214082] Call Trace:
[ 28.214228] try_to_unmap_one+0x1ca/0x410
[ 28.214341] rmap_walk_file+0xf0/0x1e0
[ 28.214433] rmap_walk+0x32/0x60
[ 28.214524] try_to_unmap+0x4d/0xd0
[ 28.214608] ? page_remove_rmap+0x280/0x280
[ 28.214715] ? page_not_mapped+0x10/0x10
[ 28.214816] ? page_get_anon_vma+0x80/0x80
[ 28.214918] shrink_page_list+0x293/0xcd0
[ 28.215037] shrink_inactive_list+0x1a3/0x450
[ 28.215137] shrink_node_memcg+0x2b0/0x350
[ 28.215263] shrink_node+0xbb/0x2c0
[ 28.215370] do_try_to_free_pages+0x92/0x2c0
[ 28.215480] try_to_free_pages+0x1fb/0x320
[ 28.215589] __alloc_pages_nodemask+0x351/0x830
[ 28.215689] ? tick_program_event+0x3a/0x80
[ 28.215853] ? SyS_readahead+0xa0/0xa0
[ 28.215957] ? mem_cgroup_commit_charge+0x6e/0xb0
[ 28.216068] ? page_add_new_anon_rmap+0x6c/0xa0
[ 28.216202] handle_mm_fault+0xa5e/0xdd0
[ 28.216304] __do_page_fault+0x194/0x420
[ 28.216396] ? vmalloc_sync_all+0x150/0x150
[ 28.216498] do_page_fault+0xb/0xd
[ 28.216591] common_exception+0x6d/0x72
[ 28.216697] EIP: 0x8048437
[ 28.216766] EFLAGS: 00000202 CPU: 0
[ 28.216843] EAX: 01267000 EBX: 7ff00000 ECX: 3910e008 EDX: 00000000
[ 28.216981] ESI: 7ff00000 EDI: 00000000 EBP: bfc0cf08 ESP: bfc0ced0
[ 28.217098] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 28.217199] Code: b1 83 e8 01 e9 55 ff ff ff 83 e8 01 e9 37 ff ff ff 8d 76 00 55 89 e5 56 53 89 c3 83 ec 04 8b 40 14 a8 01 0f 85 10 02 00 00 89 d8 <f6> 40 04 01 74 6b 84 d2 0f 85 5b 01 00 00 83 43 0c ff 78 0d 83
[ 28.217849] EIP: page_remove_rmap+0x17/0x280 SS:ESP: 0068:f5ea1b04
[ 28.217973] CR2: 000000001a8a306a
[ 28.218377] ---[ end trace 118e4d2be9e69f8a ]---
[ 28.218661] Kernel panic - not syncing: Fatal exception
[ 28.218808] Kernel Offset: disabled
[ 28.218931] Rebooting in 1 seconds..
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc7] Random oopses under memory pressure.
2018-01-14 11:54 ` Tetsuo Handa
@ 2018-01-15 23:05 ` Linus Torvalds
2018-01-16 1:15 ` [mm 4.15-rc8] " Tetsuo Handa
0 siblings, 1 reply; 59+ messages in thread
From: Linus Torvalds @ 2018-01-15 23:05 UTC (permalink / raw)
To: Tetsuo Handa
Cc: Linux Kernel Mailing List, linux-mm, the arch/x86 maintainers,
linux-fsdevel, Michal Hocko
On Sun, Jan 14, 2018 at 3:54 AM, Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
> This memory corruption bug occurs even on CONFIG_SMP=n CONFIG_PREEMPT_NONE=y
> kernel. This bug highly depends on timing and thus too difficult to bisect.
> This bug seems to exist at least since Linux 4.8 (judging from the traces, though
> the cause might be different). None of debugging configuration gives me a clue.
> So far only CONFIG_HIGHMEM=y CONFIG_DEBUG_PAGEALLOC=y kernel (with RAM enough to
> use HighMem: zone) seems to hit this bug, but it might be just by chance caused
> by timings. Thus, there is no evidence that 64bit kernels are not affected by
> this bug. But I can't narrow down any more. Thus, I call for developers who can
> narrow down / identify where the memory corruption bug is.
Hmm.
I guess I'm still hung up on the "it does not look like a valid
'struct page *'" thing.
Can you reproduce this with CONFIG_FLATMEM=y instead of CONFIG_SPARSEMEM?
Because if you can, I think we can easily add a few more pfn and
'struct page' validation debug statements. With SPARSEMEM, it gets
pretty complicated because the whole struct page setup is much more
complex.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-15 23:05 ` Linus Torvalds
@ 2018-01-16 1:15 ` Tetsuo Handa
2018-01-16 2:14 ` Linus Torvalds
0 siblings, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-16 1:15 UTC (permalink / raw)
To: Linus Torvalds
Cc: Linux Kernel Mailing List, linux-mm, the arch/x86 maintainers,
linux-fsdevel, Michal Hocko
Linus Torvalds wrote:
> On Sun, Jan 14, 2018 at 3:54 AM, Tetsuo Handa
> <penguin-kernel@i-love.sakura.ne.jp> wrote:
> > This memory corruption bug occurs even on CONFIG_SMP=n CONFIG_PREEMPT_NONE=y
> > kernel. This bug highly depends on timing and thus too difficult to bisect.
> > This bug seems to exist at least since Linux 4.8 (judging from the traces, though
> > the cause might be different). None of debugging configuration gives me a clue.
> > So far only CONFIG_HIGHMEM=y CONFIG_DEBUG_PAGEALLOC=y kernel (with RAM enough to
> > use HighMem: zone) seems to hit this bug, but it might be just by chance caused
> > by timings. Thus, there is no evidence that 64bit kernels are not affected by
> > this bug. But I can't narrow down any more. Thus, I call for developers who can
> > narrow down / identify where the memory corruption bug is.
>
> Hmm.
>
> I guess I'm still hung up on the "it does not look like a valid
> 'struct page *'" thing.
>
> Can you reproduce this with CONFIG_FLATMEM=y instead of CONFIG_SPARSEMEM?
>
> Because if you can, I think we can easily add a few more pfn and
> 'struct page' validation debug statements. With SPARSEMEM, it gets
> pretty complicated because the whole struct page setup is much more
> complex.
I can't reproduce this with CONFIG_FLATMEM=y . But I'm not sure whether
we are hitting a bug in CONFIG_SPARSEMEM=y code, for the bug is highly
timing dependent.
----------
# diff .config.old .config
372a373
> CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
462d462
< CONFIG_NEED_NODE_MEMMAP_SIZE=y
468,471c468,471
< # CONFIG_FLATMEM_MANUAL is not set
< CONFIG_SPARSEMEM_MANUAL=y
< CONFIG_SPARSEMEM=y
< CONFIG_HAVE_MEMORY_PRESENT=y
---
> CONFIG_FLATMEM_MANUAL=y
> # CONFIG_SPARSEMEM_MANUAL is not set
> CONFIG_FLATMEM=y
> CONFIG_FLAT_NODE_MEM_MAP=y
478d477
< # CONFIG_MEMORY_HOTPLUG is not set
486a486,487
> CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
> # CONFIG_MEMORY_FAILURE is not set
----------
----------
[ 0.000000] Linux version 4.15.0-rc8 (root@localhost.localdomain) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)) #381 Tue Jan 16 09:38:22 JST 2018
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007fffbfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000007fffc000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] Notice: NX (Execute Disable) protection missing in CPU!
[ 0.000000] random: fast init done
[ 0.000000] SMBIOS 2.4 present.
[ 0.000000] DMI: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 0.000000] e820: last_pfn = 0x7fffc max_arch_pfn = 0x100000
[ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WC UC- UC
[ 0.000000] found SMP MP-table at [mem 0x000f7300-0x000f730f] mapped at [(ptrval)]
[ 0.000000] Scanning 1 areas for low memory corruption
[ 0.000000] ACPI: Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 0x00000000000F7160 000014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 0x000000007FFFFA9B 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 0x000000007FFFF177 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 0x000000007FFFE040 001137 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 0x000000007FFFE000 000040
[ 0.000000] ACPI: SSDT 0x000000007FFFF1EB 000838 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: APIC 0x000000007FFFFA23 000078 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] 1159MB HIGHMEM available.
[ 0.000000] 887MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 377fe000
[ 0.000000] low ram: 0 - 377fe000
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] Normal [mem 0x0000000001000000-0x00000000377fdfff]
[ 0.000000] HighMem [mem 0x00000000377fe000-0x000000007fffbfff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.000000] node 0: [mem 0x0000000000100000-0x000000007fffbfff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000007fffbfff]
[ 0.000000] Reserved but unavailable: 98 pages
[ 0.000000] Using APIC driver default
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] IOAPIC[0]: apic_id 0 already used, trying 1
[ 0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] e820: [mem 0x80000000-0xfffbffff] available for PCI devices
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 521966
[ 0.000000] Kernel command line: ro console=ttyS0,115200n8 root=/dev/vda init=/init
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Initializing CPU#0
[ 0.000000] Initializing HighMem for node 0 (000377fe:0007fffc)
[ 0.000000] Initializing Movable for node 0 (00000000:00000000)
[ 0.000000] Memory: 2064192K/2096744K available (3353K kernel code, 256K rwdata, 924K rodata, 376K init, 5308K bss, 32552K reserved, 0K cma-reserved, 1187832K highmem)
[ 0.000000] virtual kernel memory layout:
[ 0.000000] fixmap : 0xfffa4000 - 0xfffff000 ( 364 kB)
[ 0.000000] cpu_entry : 0xffc00000 - 0xffc28000 ( 160 kB)
[ 0.000000] pkmap : 0xff800000 - 0xffc00000 (4096 kB)
[ 0.000000] vmalloc : 0xf7ffe000 - 0xff7fe000 ( 120 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf77fe000 ( 887 MB)
[ 0.000000] .init : 0xc147e000 - 0xc14dc000 ( 376 kB)
[ 0.000000] .data : 0xc13466c8 - 0xc14713e0 (1195 kB)
[ 0.000000] .text : 0xc1000000 - 0xc13466c8 (3353 kB)
[ 0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok.
[ 0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS: 2304, nr_irqs: 256, preallocated irqs: 16
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [ttyS0] enabled
[ 0.000000] ACPI: Core revision 20170831
[ 0.000000] ACPI: 2 ACPI AML tables successfully acquired and loaded
[ 0.001000] APIC: Switch to symmetric I/O mode setup
[ 0.002000] Enabling APIC mode: Flat. Using 1 I/O APICs
[ 0.002000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.007000] tsc: Fast TSC calibration using PIT
[ 0.008000] tsc: Detected 1995.399 MHz processor
[ 0.008000] Calibrating delay loop (skipped), value calculated using timer frequency.. 3990.79 BogoMIPS (lpj=1995399)
[ 0.008000] pid_max: default: 32768 minimum: 301
[ 0.009788] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.010000] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.017000] mce: CPU supports 10 MCE banks
[ 0.017478] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.017644] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 0.017792] CPU: Intel Common 32-bit KVM processor (family: 0xf, model: 0x6, stepping: 0x1)
[ 0.018000] Spectre V2 mitigation: Vulnerable: Minimal generic ASM retpoline
[ 0.024000] Performance Events: unsupported Netburst CPU model 6 no PMU driver, software events only.
[ 0.027000] APIC calibration not consistent with PM-Timer: 205ms instead of 100ms
[ 0.027000] APIC delta adjusted to PM-Timer: 6250051 (12854429)
[ 0.029058] devtmpfs: initialized
[ 0.032000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 0.032000] futex hash table entries: 256 (order: -1, 3072 bytes)
[ 0.035992] cpuidle: using governor menu
[ 0.037000] ACPI: bus type PCI registered
[ 0.039000] PCI: PCI BIOS revision 2.10 entry at 0xfd54b, last bus=0
[ 0.039000] PCI: Using configuration type 1 for base access
[ 0.048000] HugeTLB registered 4.00 MiB page size, pre-allocated 0 pages
[ 0.050000] ACPI: Added _OSI(Module Device)
[ 0.050000] ACPI: Added _OSI(Processor Device)
[ 0.050000] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.050000] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.071000] ACPI: Interpreter enabled
[ 0.071000] ACPI: (supports S0 S5)
[ 0.071000] ACPI: Using IOAPIC for interrupt routing
[ 0.071409] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.073000] ACPI: Enabled 16 GPEs in block 00 to 0F
[ 0.100000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.100000] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
[ 0.100314] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 0.100785] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 0.102000] PCI host bridge to bus 0000:00
[ 0.102156] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.102378] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.102602] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 0.102784] pci_bus 0000:00: root bus resource [mem 0x80000000-0xfebfffff window]
[ 0.103000] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.108493] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 0.108716] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 0.108905] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 0.109000] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 0.110000] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 0.110000] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 0.129000] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[ 0.130000] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[ 0.130000] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[ 0.131000] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[ 0.131000] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
[ 0.135000] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[ 0.135000] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[ 0.135000] pci 0000:00:02.0: vgaarb: bridge control possible
[ 0.135000] vgaarb: loaded
[ 0.136000] SCSI subsystem initialized
[ 0.136735] PCI: Using ACPI for IRQ routing
[ 0.137652] clocksource: Switched to clocksource refined-jiffies
[ 0.138999] ACPI: Failed to create genetlink family for ACPI event
[ 0.138999] pnp: PnP ACPI init
[ 0.142999] pnp: PnP ACPI: found 5 devices
[ 0.167133] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.167414] clocksource: Switched to clocksource acpi_pm
[ 0.167995] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 0.167995] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 0.167995] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 0.170867] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 0.179490] Scanning for low memory corruption every 60 seconds
[ 0.185037] workingset: timestamp_bits=14 max_order=19 bucket_order=5
[ 0.209598] zbud: loaded
[ 0.214606] SGI XFS with ACLs, security attributes, no debug enabled
[ 0.226422] bounce: pool size: 64 pages
[ 0.226745] io scheduler noop registered (default)
[ 0.239185] atomic64_test: passed for i586+ platform with CX8 and with SSE
[ 0.245995] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.246746] ACPI: Power Button [PWRF]
[ 0.259029] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
[ 0.259249] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
[ 0.263351] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.286776] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 0.293723] Non-volatile memory driver v1.3
[ 0.330628] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[ 0.335343] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.335644] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.338091] Using IPI Shortcut mode
[ 0.338091] sched_clock: Marking stable (337091080, 0)->(767486575, -430395495)
[ 0.341662] page_owner is disabled
[ 0.373592] XFS (vda): Mounting V5 Filesystem
[ 0.440300] XFS (vda): Ending clean mount
[ 0.463216] VFS: Mounted root (xfs filesystem) readonly on device 254:0.
[ 0.465860] devtmpfs: mounted
[ 0.466592] debug: unmapping init [mem 0xc147e000-0xc14dbfff]
[ 0.468584] Write protecting the kernel text: 3356k
[ 0.469119] Write protecting the kernel read-only data: 936k
Starting a.out
[ 1.184517] tsc: Refined TSC clocksource calibration: 1995.468 MHz
[ 1.184906] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x3986e9f5ad2, max_idle_ns: 881590531620 ns
[ 2.259952] clocksource: Switched to clocksource tsc
----------
I dont know why but selecting CONFIG_FLATMEM=y seems to avoid a different bug
where bootup of qemu randomly fails at
----------
[ 0.000000] Linux version 4.15.0-rc8 (root@localhost.localdomain) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)) #380 Tue Jan 16 09:25:52 JST 2018
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007fffbfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000007fffc000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] Notice: NX (Execute Disable) protection missing in CPU!
[ 0.000000] random: fast init done
[ 0.000000] SMBIOS 2.4 present.
[ 0.000000] DMI: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 0.000000] e820: last_pfn = 0x7fffc max_arch_pfn = 0x100000
[ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WC UC- UC
[ 0.000000] found SMP MP-table at [mem 0x000f7300-0x000f730f] mapped at [(ptrval)]
[ 0.000000] Scanning 1 areas for low memory corruption
[ 0.000000] ACPI: Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 0x00000000000F7160 000014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 0x000000007FFFFA9B 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 0x000000007FFFF177 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 0x000000007FFFE040 001137 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 0x000000007FFFE000 000040
[ 0.000000] ACPI: SSDT 0x000000007FFFF1EB 000838 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: APIC 0x000000007FFFFA23 000078 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] 1159MB HIGHMEM available.
[ 0.000000] 887MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 377fe000
[ 0.000000] low ram: 0 - 377fe000
[ 0.000000] tsc: Unable to calibrate against PIT
[ 0.000000] tsc: No reference (HPET/PMTIMER) available
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] Normal [mem 0x0000000001000000-0x00000000377fdfff]
[ 0.000000] HighMem [mem 0x00000000377fe000-0x000000007fffbfff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.000000] node 0: [mem 0x0000000000100000-0x000000007fffbfff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000007fffbfff]
[ 0.000000] Reserved but unavailable: 98 pages
[ 0.000000] Using APIC driver default
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] IOAPIC[0]: apic_id 0 already used, trying 1
[ 0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] e820: [mem 0x80000000-0xfffbffff] available for PCI devices
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 521966
[ 0.000000] Kernel command line: ro console=ttyS0,115200n8 root=/dev/vda init=/init
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Initializing CPU#0
[ 0.000000] Initializing HighMem for node 0 (000377fe:0007fffc)
[ 0.000000] Initializing Movable for node 0 (00000000:00000000)
[ 0.000000] Memory: 2064188K/2096744K available (3357K kernel code, 252K rwdata, 924K rodata, 380K init, 5308K bss, 32556K reserved, 0K cma-reserved, 1187832K highmem)
[ 0.000000] virtual kernel memory layout:
[ 0.000000] fixmap : 0xfffa4000 - 0xfffff000 ( 364 kB)
[ 0.000000] cpu_entry : 0xffc00000 - 0xffc28000 ( 160 kB)
[ 0.000000] pkmap : 0xff800000 - 0xffc00000 (4096 kB)
[ 0.000000] vmalloc : 0xf7ffe000 - 0xff7fe000 ( 120 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf77fe000 ( 887 MB)
[ 0.000000] .init : 0xc147e000 - 0xc14dd000 ( 380 kB)
[ 0.000000] .data : 0xc13476a8 - 0xc14713e0 (1191 kB)
[ 0.000000] .text : 0xc1000000 - 0xc13476a8 (3357 kB)
[ 0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok.
[ 0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS: 2304, nr_irqs: 256, preallocated irqs: 16
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [ttyS0] enabled
[ 0.000000] ACPI: Core revision 20170831
[ 0.000000] ACPI: 2 ACPI AML tables successfully acquired and loaded
[ 0.000000] APIC: Switch to symmetric I/O mode setup
[ 0.000000] Enabling APIC mode: Flat. Using 1 I/O APICs
[ 0.000000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.000000] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
[ 0.000000] ...trying to set up timer (IRQ0) through the 8259A ...
[ 0.000000] ..... (found apic 0 pin 2) ...
[ 0.000000] ....... failed.
[ 0.000000] ...trying to set up timer as Virtual Wire IRQ...
[ 0.000000] ..... failed.
[ 0.000000] ...trying to set up timer as ExtINT IRQ...
[ 0.000000] ..... failed :(.
[ 0.000000] Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option.
[ 0.000000]
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.15.0-rc8 #380
[ 0.000000] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 0.000000] Call Trace:
[ 0.000000] dump_stack+0x16/0x24
[ 0.000000] panic+0x87/0x1a2
[ 0.000000] ? vprintk_func+0x23/0x60
[ 0.000000] setup_IO_APIC+0x4dd/0x6f8
[ 0.000000] ? clear_IO_APIC+0x29/0x50
[ 0.000000] ? lapic_get_maxlvt+0x2e/0x40
[ 0.000000] ? end_local_APIC_setup+0x9d/0x130
[ 0.000000] apic_bsp_setup+0x70/0x75
[ 0.000000] apic_intr_mode_init+0x10c/0x10e
[ 0.000000] ? hpet_time_init+0x1e/0x20
[ 0.000000] x86_late_time_init+0xf/0x16
[ 0.000000] start_kernel+0x2ef/0x360
[ 0.000000] ? set_init_arg+0x52/0x52
[ 0.000000] i386_start_kernel+0x13d/0x140
[ 0.000000] startup_32_smp+0x164/0x168
[ 0.000000] Rebooting in 1 seconds..
----------
or hangs at
----------
[ 0.000000] Linux version 4.15.0-rc8 (root@localhost.localdomain) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)) #380 Tue Jan 16 09:25:52 JST 2018
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007fffbfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000007fffc000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] Notice: NX (Execute Disable) protection missing in CPU!
[ 0.000000] random: fast init done
[ 0.000000] SMBIOS 2.4 present.
[ 0.000000] DMI: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 0.000000] e820: last_pfn = 0x7fffc max_arch_pfn = 0x100000
[ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WC UC- UC
[ 0.000000] found SMP MP-table at [mem 0x000f7300-0x000f730f] mapped at [(ptrval)]
[ 0.000000] Scanning 1 areas for low memory corruption
[ 0.000000] ACPI: Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 0x00000000000F7160 000014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 0x000000007FFFFA9B 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 0x000000007FFFF177 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 0x000000007FFFE040 001137 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 0x000000007FFFE000 000040
[ 0.000000] ACPI: SSDT 0x000000007FFFF1EB 000838 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: APIC 0x000000007FFFFA23 000078 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] 1159MB HIGHMEM available.
[ 0.000000] 887MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 377fe000
[ 0.000000] low ram: 0 - 377fe000
[ 0.000000] tsc: Unable to calibrate against PIT
[ 0.000000] tsc: No reference (HPET/PMTIMER) available
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] Normal [mem 0x0000000001000000-0x00000000377fdfff]
[ 0.000000] HighMem [mem 0x00000000377fe000-0x000000007fffbfff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.000000] node 0: [mem 0x0000000000100000-0x000000007fffbfff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000007fffbfff]
[ 0.000000] Reserved but unavailable: 98 pages
[ 0.000000] Using APIC driver default
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] IOAPIC[0]: apic_id 0 already used, trying 1
[ 0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] e820: [mem 0x80000000-0xfffbffff] available for PCI devices
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 521966
[ 0.000000] Kernel command line: ro console=ttyS0,115200n8 root=/dev/vda init=/init
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Initializing CPU#0
[ 0.000000] Initializing HighMem for node 0 (000377fe:0007fffc)
[ 0.000000] Initializing Movable for node 0 (00000000:00000000)
[ 0.000000] Memory: 2064188K/2096744K available (3357K kernel code, 252K rwdata, 924K rodata, 380K init, 5308K bss, 32556K reserved, 0K cma-reserved, 1187832K highmem)
[ 0.000000] virtual kernel memory layout:
[ 0.000000] fixmap : 0xfffa4000 - 0xfffff000 ( 364 kB)
[ 0.000000] cpu_entry : 0xffc00000 - 0xffc28000 ( 160 kB)
[ 0.000000] pkmap : 0xff800000 - 0xffc00000 (4096 kB)
[ 0.000000] vmalloc : 0xf7ffe000 - 0xff7fe000 ( 120 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf77fe000 ( 887 MB)
[ 0.000000] .init : 0xc147e000 - 0xc14dd000 ( 380 kB)
[ 0.000000] .data : 0xc13476a8 - 0xc14713e0 (1191 kB)
[ 0.000000] .text : 0xc1000000 - 0xc13476a8 (3357 kB)
[ 0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok.
[ 0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS: 2304, nr_irqs: 256, preallocated irqs: 16
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [ttyS0] enabled
[ 0.000000] ACPI: Core revision 20170831
[ 0.000000] ACPI: 2 ACPI AML tables successfully acquired and loaded
[ 0.001000] APIC: Switch to symmetric I/O mode setup
[ 0.001000] Enabling APIC mode: Flat. Using 1 I/O APICs
[ 0.001000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.003000] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
[ 0.003000] ...trying to set up timer (IRQ0) through the 8259A ...
[ 0.003000] ..... (found apic 0 pin 2) ...
[ 0.005000] ....... failed.
[ 0.005000] ...trying to set up timer as Virtual Wire IRQ...
[ 13.120000] random: crng init done
----------
which has been bothering me when testing using qemu. Note that the 13.120000 part is
incorrect. There is no delay between "...trying to set up timer as Virtual Wire IRQ..."
line and "random: crng init done" line.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-16 1:15 ` [mm 4.15-rc8] " Tetsuo Handa
@ 2018-01-16 2:14 ` Linus Torvalds
2018-01-16 8:06 ` Dave Hansen
2018-01-16 17:33 ` Tetsuo Handa
0 siblings, 2 replies; 59+ messages in thread
From: Linus Torvalds @ 2018-01-16 2:14 UTC (permalink / raw)
To: Tetsuo Handa, Dave Hansen
Cc: Linux Kernel Mailing List, linux-mm, the arch/x86 maintainers,
linux-fsdevel, Michal Hocko
On Mon, Jan 15, 2018 at 5:15 PM, Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
>
> I can't reproduce this with CONFIG_FLATMEM=y . But I'm not sure whether
> we are hitting a bug in CONFIG_SPARSEMEM=y code, for the bug is highly
> timing dependent.
Hmm. Maybe. But sparsemem really also generates *much* more complex
code particularly for the pfn_to_page() case.
It also has much less testing. For example, on x86-64 we do use
sparsemem, but we use the VMEMMAP version of sparsemem: the version
that does *not* play really odd and complex games with that whole
pfn_to_page().
I've always felt like sparsemem was really damn complicated. The
whole "section_mem_map" encoding is really subtle and odd.
And considering that we're getting what appears to be a invalid page,
in one of the more complicated sequences that very much does that
whole pfn_to_page(), I really wonder.
I wonder if somebody could add some VM_BUG_ON() checks to the
non-vmemmap case of sparsemem in include/asm-generic/memory_model.h.
Because this:
#define __pfn_to_page(pfn) \
({ unsigned long __pfn = (pfn); \
struct mem_section *__sec = __pfn_to_section(__pfn); \
__section_mem_map_addr(__sec) + __pfn; \
})
is really subtle, and if we have some case where we pass in an
out-of-range pfn, or some case where we get the section wrong (because
the pfn is between sections or whatever due to some subtle setup bug),
things will really go sideways.
The reason I was hoping you could do this for FLATMEM is that it's
much easier to verify the pfn range in that case. The sparsemem cases
really makes it much nastier.
That said, all of that code is really old. Most of it goes back to
-05/06 or so. But since you seem to be able to reproduce at least back
to 4.8, I guess this bug does back years too.
But I'm adding Dave Hansen explicitly to the cc, in case he has any
ideas. Not because I blame him, but he's touched the sparsemem code
fairly recently, so maybe he'd have some idea on adding sanity
checking to the sparsemem version of pfn_to_page().
> I dont know why but selecting CONFIG_FLATMEM=y seems to avoid a different bug
> where bootup of qemu randomly fails at
Hmm. That looks very different indeed. But if CONFIG_SPARSEMEM
(presumably together with HIGHMEM) has some odd off-by-one corner case
or similar, who knows *what* issues it could trigger.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-16 2:14 ` Linus Torvalds
@ 2018-01-16 8:06 ` Dave Hansen
2018-01-16 8:37 ` Ingo Molnar
2018-01-16 19:30 ` Linus Torvalds
2018-01-16 17:33 ` Tetsuo Handa
1 sibling, 2 replies; 59+ messages in thread
From: Dave Hansen @ 2018-01-16 8:06 UTC (permalink / raw)
To: Linus Torvalds, Tetsuo Handa
Cc: Linux Kernel Mailing List, linux-mm, the arch/x86 maintainers,
linux-fsdevel, Michal Hocko
On 01/15/2018 06:14 PM, Linus Torvalds wrote:
> But I'm adding Dave Hansen explicitly to the cc, in case he has any
> ideas. Not because I blame him, but he's touched the sparsemem code
> fairly recently, so maybe he'd have some idea on adding sanity
> checking to the sparsemem version of pfn_to_page().
I swear I haven't touched it lately!
I'm not sure I'd go after pfn_to_page(). *Maybe* if we were close to
the places where we've done a pfn_to_page(), but I'm not seeing those.
These, for instance (from the January 5th post) have sane (~500MB) PFNs
and all BUG_ON() because of seeing the page being locked at free:
[ 192.152510] BUG: Bad page state in process a.out pfn:18566
[ 77.872133] BUG: Bad page state in process a.out pfn:1873a
[ 188.992549] BUG: Bad page state in process a.out pfn:197ea
and the page in all those cases came off a list, not out of a pte or
something that would need pfn_to_page(). The page fault path leading up
to the "EIP is at page_cache_tree_insert+0xbe/0xc0" probably doesn't
have a pfn_to_page() anywhere in there at all.
Did anyone else notice the
[ 31.068198] ? vmalloc_sync_all+0x150/0x150
present in a bunch of the stack traces? That should be pretty uncommon.
Is it just part of the normal do_page_fault() stack and the stack
dumper picks up on it?
A few things from earlier in this thread:
> [ 44.103192] page:5a5a0697 count:-1055023618 mapcount:-1055030029 mapping:26f4be11 index:0xc11d7c83
> [ 44.103196] flags: 0xc10528fe(waiters|error|referenced|uptodate|dirty|lru|active|reserved|private_2|mappedtodisk|swapbacked)
> [ 44.103200] raw: c10528fe c114fff7 c11d7c83 c11d84f2 c11d9dfe c11daa34 c11daaa0 c13e65df
> [ 44.103201] raw: c13e4a1c c13e4c62
> [ 44.103202] page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) <= 0)
> [ 44.103203] page->mem_cgroup:35401b27
Isn't that 'page:' a non-aligned address in userspace? It's also weird
that you start dumping out kernel-looking addresses that came from
userspace addresses. Which VM_SPLIT option are you running with, btw?
I'm still pretty stumped, though.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-16 8:06 ` Dave Hansen
@ 2018-01-16 8:37 ` Ingo Molnar
2018-01-16 19:30 ` Linus Torvalds
1 sibling, 0 replies; 59+ messages in thread
From: Ingo Molnar @ 2018-01-16 8:37 UTC (permalink / raw)
To: Dave Hansen
Cc: Linus Torvalds, Tetsuo Handa, Linux Kernel Mailing List,
linux-mm, the arch/x86 maintainers, linux-fsdevel, Michal Hocko
* Dave Hansen <dave.hansen@linux.intel.com> wrote:
> Did anyone else notice the
>
> [ 31.068198] ? vmalloc_sync_all+0x150/0x150
>
> present in a bunch of the stack traces? That should be pretty uncommon.
I thikn that's pretty unusual:
> Is it just part of the normal do_page_fault() stack and the stack
> dumper picks up on it?
No, it should only be called by register_die_notifier(), which is not part of the
regular stack dump, AFAICS.
Thanks,
Ingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-16 8:06 ` Dave Hansen
2018-01-16 8:37 ` Ingo Molnar
@ 2018-01-16 19:30 ` Linus Torvalds
1 sibling, 0 replies; 59+ messages in thread
From: Linus Torvalds @ 2018-01-16 19:30 UTC (permalink / raw)
To: Dave Hansen
Cc: Tetsuo Handa, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers, linux-fsdevel, Michal Hocko
On Tue, Jan 16, 2018 at 12:06 AM, Dave Hansen
<dave.hansen@linux.intel.com> wrote:
> On 01/15/2018 06:14 PM, Linus Torvalds wrote:
>> But I'm adding Dave Hansen explicitly to the cc, in case he has any
>> ideas. Not because I blame him, but he's touched the sparsemem code
>> fairly recently, so maybe he'd have some idea on adding sanity
>> checking to the sparsemem version of pfn_to_page().
>
> I swear I haven't touched it lately!
Heh. I did
git blame -C mm/sparse.c | grep 2017
and your name shows up at the beginning a lot because of commit
c4e1be9ec113 ("mm, sparsemem: break out of loops early").
And Michal Hocko (who shows up even more) was already on the cc.
> I'm not sure I'd go after pfn_to_page(). *Maybe* if we were close to
> the places where we've done a pfn_to_page(), but I'm not seeing those.
Fair enough. I just wanted to add debugging, looked at Tetsuo's
config, and went "no way am I adding debugging to the sparsemem case
because it's so confusing".
That said, I also started looking at "kmap_to_page()". That's
something that is *really* different with HIGHMEM, and while most of
the users are in random drivers that do crazy things, I do note that
one of the users is in mm/swap.c.
That thing goes back to commit 5a178119b0fb ("mm: add support for
direct_IO to highmem pages") and was only used for swap_writepage(),
if I read this right.
That swap_writepage() use of kmap()'ed patches was removed some time
later in commit 62a8067a7f35 ("bio_vec-backed iov_iter"), but the
crazy kmap_to_page() thing remained.
I see nothing actively wrong in there, but it really feels like a
"that is all bogus" thing to me.
> Did anyone else notice the
>
> [ 31.068198] ? vmalloc_sync_all+0x150/0x150
>
> present in a bunch of the stack traces? That should be pretty uncommon.
No, didn't notice that. And yes, vmalloc_sync_all() might be interesting.
> Is it just part of the normal do_page_fault() stack and the stack
> dumper picks up on it?
I don't think so. It should *not* happen normally. The fact that it
shows up in the trace means it happened that time.
It doesn't seem HIGHMEM-related, though. Or maybe the highmem signal
is bogus too, and it's just that Tetsuo's reproducer needs magical
timing.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-16 2:14 ` Linus Torvalds
2018-01-16 8:06 ` Dave Hansen
@ 2018-01-16 17:33 ` Tetsuo Handa
2018-01-16 19:34 ` Linus Torvalds
1 sibling, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-16 17:33 UTC (permalink / raw)
To: torvalds, dave.hansen, mingo
Cc: linux-kernel, linux-mm, x86, linux-fsdevel, mhocko
Linus Torvalds wrote:
> On Mon, Jan 15, 2018 at 5:15 PM, Tetsuo Handa
> <penguin-kernel@i-love.sakura.ne.jp> wrote:
> >
> > I can't reproduce this with CONFIG_FLATMEM=y . But I'm not sure whether
> > we are hitting a bug in CONFIG_SPARSEMEM=y code, for the bug is highly
> > timing dependent.
>
> Hmm. Maybe. But sparsemem really also generates *much* more complex
> code particularly for the pfn_to_page() case.
Since I got a faster reproducer, I tried full bisection between 4.11 and 4.12-rc1.
But I have no idea why bisection arrives at c0332694903a37cf.
----------
gcc -Wall -O3 -m32 -o /mnt/a.out -x c - << "EOF"
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char *argv[])
{
if (argc != 1) {
unsigned long long size;
char *buf = NULL;
unsigned long long i;
for (size = 1048576; size < 512ULL * (1 << 30); size += 1048576) {
char *cp = realloc(buf, size);
if (!cp) {
size -= 1048576;
break;
}
buf = cp;
}
for (i = 0; i < size; i += 4096)
buf[i] = 0;
_exit(0);
} else
while (1) {
if (fork() == 0) {
execlp(argv[0], argv[0], "", NULL);
_exit(0);
}
sleep(1);
}
return 0;
}
EOF
----------
----------
# bad: [2ea659a9ef488125eb46da6eb571de5eae5c43f6] Linux 4.12-rc1
# good: [a351e9b9fc24e982ec2f0e76379a49826036da12] Linux 4.11
git bisect start 'HEAD' 'v4.11'
# good: [221656e7c4ce342b99c31eca96c1cbb6d1dce45f] Merge tag 'sound-4.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound
git bisect good 221656e7c4ce342b99c31eca96c1cbb6d1dce45f
# good: [c6a677c6f37bb7abc85ba7e3465e82b9f7eb1d91] Merge tag 'staging-4.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging
git bisect good c6a677c6f37bb7abc85ba7e3465e82b9f7eb1d91
# bad: [0ff4c01b279a590a2826ade9321ad8c7ca5a1b6c] Merge tag 'armsoc-arm64' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
git bisect bad 0ff4c01b279a590a2826ade9321ad8c7ca5a1b6c
# bad: [8f3207c7eab9d885cc64c778416537034a7d9c5b] Merge tag 'tty-4.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
git bisect bad 8f3207c7eab9d885cc64c778416537034a7d9c5b
# bad: [9c6ee01ed5bb1ee489d580eaa60d7eb5a8ede336] Merge branch 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm
git bisect bad 9c6ee01ed5bb1ee489d580eaa60d7eb5a8ede336
# bad: [fe7a719b30dfdb4d55680461954b99b257ebe671] Merge branch 'for-next' of git://git.samba.org/sfrench/cifs-2.6
git bisect bad fe7a719b30dfdb4d55680461954b99b257ebe671
# good: [d557d1b58b3546bab2c5bc2d624c5709840e6b10] refcount: change EXPORT_SYMBOL markings
git bisect good d557d1b58b3546bab2c5bc2d624c5709840e6b10
# good: [8f720d9f892e0e223dab8400f03130bc208c72e7] xfs: publish UUID in struct super_block
git bisect good 8f720d9f892e0e223dab8400f03130bc208c72e7
# bad: [d173a25165c124442182f6b21d0c2ec381a0eebe] blk-mq: move debugfs declarations to a separate header file
git bisect bad d173a25165c124442182f6b21d0c2ec381a0eebe
# bad: [2719aa217e0d025dbfce74ac777815776ccec072] blk-mq: don't use sync workqueue flushing from drivers
git bisect bad 2719aa217e0d025dbfce74ac777815776ccec072
# bad: [9f2779bff2f178496fb00b89797734ee245d2c93] blk-mq-sched: remove hack that bypasses scheduler for reserved requests
git bisect bad 9f2779bff2f178496fb00b89797734ee245d2c93
# bad: [8afdd94c74e416de74a8ee61d79e4bf93466420b] mtip32xx: kill atomic argument to mtip_quiesce_io()
git bisect bad 8afdd94c74e416de74a8ee61d79e4bf93466420b
# bad: [0f6422a2c57c6afcf66ead903dc3fa6641184aa4] mtip32xx: get rid of 'atomic' argument to mtip_exec_internal_command()
git bisect bad 0f6422a2c57c6afcf66ead903dc3fa6641184aa4
# bad: [c0332694903a37cf8ecdc9102d5c9e09cf8643d0] block: Remove elevator_change()
git bisect bad c0332694903a37cf8ecdc9102d5c9e09cf8643d0
# first bad commit: [c0332694903a37cf8ecdc9102d5c9e09cf8643d0] block: Remove elevator_change()
----------
I tried different routes from mm/sparse.c , but I feel I can't find the culprit.
----------
# bad: [7660a6fddcbae344de8583aa4092071312f110c3] mm: allow slab_nomerge to be set at build time
# good: [60a7a88dbb9fc9adcca78a10a3ecf36966b5a45c] mm/sparse: refine usemap_size() a little
git bisect start '7660a6fddcbae344de8583aa4092071312f110c3' '60a7a88dbb9fc9adcca78a10a3ecf36966b5a45c'
# bad: [9786e34e0a6055dbd1b46e16dfa791ac2b3da289] Merge tag 'for-linus-20170510' of git://git.infradead.org/linux-mtd
git bisect bad 9786e34e0a6055dbd1b46e16dfa791ac2b3da289
# good: [1062ae4982cabbf60f89b4e069fbb7def7edc8f7] Merge tag 'drm-forgot-about-tegra-for-v4.12-rc1' of git://people.freedesktop.org/~airlied/linux
git bisect good 1062ae4982cabbf60f89b4e069fbb7def7edc8f7
# bad: [bf5f89463f5b3109a72ed13ca62b57e90213387d] Merge branch 'akpm' (patches from Andrew)
git bisect bad bf5f89463f5b3109a72ed13ca62b57e90213387d
# good: [2b0d92b265324cfe42839a23cb46677bb0112c2c] staging: ks7010: remove cast from netdev_priv()
git bisect good 2b0d92b265324cfe42839a23cb46677bb0112c2c
# bad: [d484467c860dab3e17893d23b2238e1f581460fa] Merge tag 'xfs-4.12-merge-7' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux
git bisect bad d484467c860dab3e17893d23b2238e1f581460fa
# good: [e8357cdec3d1b6b42566ce3bc960e5e10c2b3787] [media] media: stk1160: Add Kconfig help on snd-usb-audio requirement
git bisect good e8357cdec3d1b6b42566ce3bc960e5e10c2b3787
# good: [53ef7d0e208fa38c3f63d287e0c3ab174f1e1235] Merge tag 'libnvdimm-for-4.12' of git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm
git bisect good 53ef7d0e208fa38c3f63d287e0c3ab174f1e1235
# good: [044f1daaaaf7c86bc4fcf433848b7baae236946b] Merge branch 'for-linus' of git://git.kernel.dk/linux-block
git bisect good 044f1daaaaf7c86bc4fcf433848b7baae236946b
# good: [bf8eadbacb24e321c99bbdd901589942712810d1] xfs: remove xfs_bmap_remap_alloc
git bisect good bf8eadbacb24e321c99bbdd901589942712810d1
# good: [e2a641922a3592b5ea226624d5abeb13eb49622c] xfs: corruption needs to respect endianess too!
git bisect good e2a641922a3592b5ea226624d5abeb13eb49622c
# good: [3c3781951c9a155a56e5eed567349118374cc315] xfs: Allow user to kill fstrim process
git bisect good 3c3781951c9a155a56e5eed567349118374cc315
# good: [ae2c4ac2dd39b23a87ddb14ceddc3f2872c6aef5] xfs: update ag iterator to support wait on new inodes
git bisect good ae2c4ac2dd39b23a87ddb14ceddc3f2872c6aef5
# good: [fe0be23e68200573de027de9b8cc2b27e7fce35e] xfs: reserve enough blocks to handle btree splits when remapping
git bisect good fe0be23e68200573de027de9b8cc2b27e7fce35e
# good: [161f55efba5ddccc690139fae9373cafc3447a97] xfs: fix use-after-free in xfs_finish_page_writeback
git bisect good 161f55efba5ddccc690139fae9373cafc3447a97
# first bad commit: [d484467c860dab3e17893d23b2238e1f581460fa] Merge tag 'xfs-4.12-merge-7' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux
# bad: [7660a6fddcbae344de8583aa4092071312f110c3] mm: allow slab_nomerge to be set at build time
# good: [60a7a88dbb9fc9adcca78a10a3ecf36966b5a45c] mm/sparse: refine usemap_size() a little
git bisect start 'HEAD' '60a7a88dbb9fc9adcca78a10a3ecf36966b5a45c' 'mm/'
# bad: [499118e966f1d2150bd66647c8932343c4e9a0b8] mm: introduce memalloc_noreclaim_{save,restore}
git bisect bad 499118e966f1d2150bd66647c8932343c4e9a0b8
# good: [5e82cd120382ad7bbcc82298e34a034538b4384c] kasan: introduce helper functions for determining bug type
git bisect good 5e82cd120382ad7bbcc82298e34a034538b4384c
# bad: [f25ba6dccc3bfe7e1524f4498a171be038507c45] mm, compaction: reorder fields in struct compact_control
git bisect bad f25ba6dccc3bfe7e1524f4498a171be038507c45
# good: [b19385993623c1a18a686b6b271cd24d5aa96f52] kasan: separate report parts by empty lines
git bisect good b19385993623c1a18a686b6b271cd24d5aa96f52
# good: [ab182e67ec99ea0c8d7435a32a4a1ed9bb02559a] Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
git bisect good ab182e67ec99ea0c8d7435a32a4a1ed9bb02559a
# good: [e4231bcda72daef497af45e195a33daa0f9357d0] cma: Introduce cma_for_each_area
git bisect good e4231bcda72daef497af45e195a33daa0f9357d0
# good: [c6a677c6f37bb7abc85ba7e3465e82b9f7eb1d91] Merge tag 'staging-4.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging
git bisect good c6a677c6f37bb7abc85ba7e3465e82b9f7eb1d91
# first bad commit: [f25ba6dccc3bfe7e1524f4498a171be038507c45] mm, compaction: reorder fields in struct compact_control
----------
>
> It also has much less testing. For example, on x86-64 we do use
> sparsemem, but we use the VMEMMAP version of sparsemem: the version
> that does *not* play really odd and complex games with that whole
> pfn_to_page().
>
> I've always felt like sparsemem was really damn complicated. The
> whole "section_mem_map" encoding is really subtle and odd.
>
> And considering that we're getting what appears to be a invalid page,
> in one of the more complicated sequences that very much does that
> whole pfn_to_page(), I really wonder.
>
> I wonder if somebody could add some VM_BUG_ON() checks to the
> non-vmemmap case of sparsemem in include/asm-generic/memory_model.h.
>
> Because this:
>
> #define __pfn_to_page(pfn) \
> ({ unsigned long __pfn = (pfn); \
> struct mem_section *__sec = __pfn_to_section(__pfn); \
> __section_mem_map_addr(__sec) + __pfn; \
> })
>
> is really subtle, and if we have some case where we pass in an
> out-of-range pfn, or some case where we get the section wrong (because
> the pfn is between sections or whatever due to some subtle setup bug),
> things will really go sideways.
>
> The reason I was hoping you could do this for FLATMEM is that it's
> much easier to verify the pfn range in that case. The sparsemem cases
> really makes it much nastier.
>
> That said, all of that code is really old. Most of it goes back to
> -05/06 or so. But since you seem to be able to reproduce at least back
> to 4.8, I guess this bug does back years too.
I feel that the bug in 4.8 is a different one, though the reproducer is the same.
>
> But I'm adding Dave Hansen explicitly to the cc, in case he has any
> ideas. Not because I blame him, but he's touched the sparsemem code
> fairly recently, so maybe he'd have some idea on adding sanity
> checking to the sparsemem version of pfn_to_page().
>
> > I dont know why but selecting CONFIG_FLATMEM=y seems to avoid a different bug
> > where bootup of qemu randomly fails at
>
> Hmm. That looks very different indeed. But if CONFIG_SPARSEMEM
> (presumably together with HIGHMEM) has some odd off-by-one corner case
> or similar, who knows *what* issues it could trigger.
It turned out that CONFIG_FLATMEM was irrelevant. I just did not hit it.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-16 17:33 ` Tetsuo Handa
@ 2018-01-16 19:34 ` Linus Torvalds
2018-01-17 11:08 ` Tetsuo Handa
0 siblings, 1 reply; 59+ messages in thread
From: Linus Torvalds @ 2018-01-16 19:34 UTC (permalink / raw)
To: Tetsuo Handa
Cc: Dave Hansen, Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers, linux-fsdevel, Michal Hocko
On Tue, Jan 16, 2018 at 9:33 AM, Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
>
> Since I got a faster reproducer, I tried full bisection between 4.11 and 4.12-rc1.
> But I have no idea why bisection arrives at c0332694903a37cf.
I don't think your reproducer is 100% reliable.
And bisection is great because it's very aggressive and optimal when
it comes to testing. But that also implies that if *any* of the
good/bad choices were incorrect, then the end result is pure garbage
and isn't even *close* to the right commit.
> It turned out that CONFIG_FLATMEM was irrelevant. I just did not hit it.
So have you actually been able to see the problem with FLATMEM, or is
this based on the bisect? Because I really think the bisect is pretty
much guaranteed to be wrong.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-16 19:34 ` Linus Torvalds
@ 2018-01-17 11:08 ` Tetsuo Handa
2018-01-17 21:39 ` Linus Torvalds
2018-01-18 8:12 ` Tetsuo Handa
0 siblings, 2 replies; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-17 11:08 UTC (permalink / raw)
To: torvalds, kirill.shutemov, akpm, hannes, iamjoonsoo.kim, mgorman,
tony.luck, vbabka, mhocko
Cc: dave.hansen, mingo, linux-kernel, linux-mm, x86
Linus Torvalds wrote:
> > It turned out that CONFIG_FLATMEM was irrelevant. I just did not hit it.
>
> So have you actually been able to see the problem with FLATMEM, or is
> this based on the bisect? Because I really think the bisect is pretty
> much guaranteed to be wrong.
Oops, this "it" is "a different bug where bootup of qemu randomly hangs at
[ 0.001000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.003000] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
[ 0.003000] ...trying to set up timer (IRQ0) through the 8259A ...
[ 0.003000] ..... (found apic 0 pin 2) ...
[ 0.005000] ....... failed.
[ 0.005000] ...trying to set up timer as Virtual Wire IRQ...
[ 13.120000] random: crng init done
". This bug occurs with both CONFIG_FLATMEM=y or CONFIG_SPARSEMEM=y .
> On Tue, Jan 16, 2018 at 9:33 AM, Tetsuo Handa
> <penguin-kernel@i-love.sakura.ne.jp> wrote:
> >
> > Since I got a faster reproducer, I tried full bisection between 4.11 and 4.12-rc1.
> > But I have no idea why bisection arrives at c0332694903a37cf.
>
> I don't think your reproducer is 100% reliable.
>
> And bisection is great because it's very aggressive and optimal when
> it comes to testing. But that also implies that if *any* of the
> good/bad choices were incorrect, then the end result is pure garbage
> and isn't even *close* to the right commit.
OK. I missed the mark. I overlooked that 4.11 already has this problem.
----------
[ 40.272926] BUG: unable to handle kernel paging request at f6d74b44
[ 40.272934] IP: page_remove_rmap+0x7/0x2c0
[ 40.272935] *pde = 3732c067
[ 40.272936] *pte = 36d74062
[ 40.272936]
[ 40.272938] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 40.272939] Modules linked in: xfs libcrc32c sr_mod cdrom sd_mod ata_generic pata_acpi serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase ata_piix libata
[ 40.272952] CPU: 6 PID: 719 Comm: b.out Not tainted 4.11.0 #266
[ 40.272952] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 40.272953] task: ef4c1e40 task.stack: ef578000
[ 40.272955] EIP: page_remove_rmap+0x7/0x2c0
[ 40.272956] EFLAGS: 00010246 CPU: 6
[ 40.272957] EAX: f6d74b30 EBX: f6d74b30 ECX: 00000000 EDX: 00000000
[ 40.272958] ESI: ef7d9640 EDI: 00000093 EBP: ef579a78 ESP: ef579a70
[ 40.272959] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 40.272961] CR0: 80050033 CR2: f6d74b44 CR3: 2f4a8000 CR4: 000406d0
[ 40.273046] Call Trace:
[ 40.273050] try_to_unmap_one+0x206/0x4f0
[ 40.273055] rmap_walk_file+0x13c/0x270
[ 40.273057] rmap_walk+0x32/0x60
[ 40.273058] try_to_unmap+0xad/0x150
[ 40.273060] ? page_remove_rmap+0x2c0/0x2c0
[ 40.273062] ? page_not_mapped+0x10/0x10
[ 40.273063] ? page_get_anon_vma+0x90/0x90
[ 40.273066] shrink_page_list+0x37a/0xd10
[ 40.273069] shrink_inactive_list+0x173/0x370
[ 40.273072] shrink_node_memcg+0x572/0x7d0
[ 40.273074] ? shrink_slab+0x1a0/0x2e0
[ 40.273077] shrink_node+0xb3/0x2c0
[ 40.273079] do_try_to_free_pages+0xb2/0x2b0
[ 40.273081] try_to_free_pages+0x1a4/0x2f0
[ 40.273085] ? schedule_timeout+0x142/0x200
[ 40.273088] __alloc_pages_slowpath+0x360/0x7e6
[ 40.273091] __alloc_pages_nodemask+0x1a4/0x1b0
[ 40.273093] do_anonymous_page+0xcb/0x500
[ 40.273120] ? xfs_filemap_fault+0x36/0x40 [xfs]
[ 40.273122] handle_mm_fault+0x52f/0x990
[ 40.273125] __do_page_fault+0x19c/0x460
[ 40.273127] ? __do_page_fault+0x460/0x460
[ 40.273129] do_page_fault+0x1a/0x20
[ 40.273131] common_exception+0x6c/0x72
[ 40.273133] EIP: 0x8048437
[ 40.273133] EFLAGS: 00010202 CPU: 6
[ 40.273134] EAX: 001f8000 EBX: 7ff00000 ECX: 37803008 EDX: 00000000
[ 40.273135] ESI: 7ff00000 EDI: 00000000 EBP: bfeffa68 ESP: bfeffa30
[ 40.273137] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 40.273137] Code: ff ff ba 78 50 7a c1 89 d8 e8 a6 f8 fe ff 0f 0b 83 e8 01 e9 66 ff ff ff 8d b6 00 00 00 00 8d bf 00 00 00 00 55 89 e5 56 53 89 c3 <8b> 40 14 a8 01 0f 85 a4 01 00 00 89 d8 f6 40 04 01 74 5e 84 d2
[ 40.273161] EIP: page_remove_rmap+0x7/0x2c0 SS:ESP: 0068:ef579a70
[ 40.273162] CR2: 00000000f6d74b44
[ 40.273164] ---[ end trace 902077077bed43fd ]---
----------
I needed to bisect between 4.10 and 4.11, and I got plausible culprit.
----------
# bad: [a351e9b9fc24e982ec2f0e76379a49826036da12] Linux 4.11
# good: [c470abd4fde40ea6a0846a2beab642a578c0b8cd] Linux 4.10
# good: [69973b830859bc6529a7a0468ba0d80ee5117826] Linux 4.9
# good: [c8d2bc9bc39ebea8437fd974fdbc21847bb897a3] Linux 4.8
# good: [523d939ef98fd712632d93a5a2b588e477a7565e] Linux 4.7
# good: [2dcd0af568b0cf583645c8a317dd12e344b1c72a] Linux 4.6
# good: [b562e44f507e863c6792946e4e1b1449fbbac85d] Linux 4.5
# good: [afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc] Linux 4.4
# good: [6a13feb9c82803e2b815eca72fa7a9f5561d7861] Linux 4.3
# good: [64291f7db5bd8150a74ad2036f1037e6a0428df2] Linux 4.2
# good: [b953c0d234bc72e8489d3bf51a276c5c4ec85345] Linux 4.1
# good: [39a8804455fb23f09157341d3ba7db6d7ae6ee76] Linux 4.0
git bisect start 'v4.11' 'v4.10' 'v4.9' 'v4.8' 'v4.7' 'v4.6' 'v4.5' 'v4.4' 'v4.3' 'v4.2' 'v4.1' 'v4.0' 'mm/'
# good: [1b096e5ae9f7181c770d59c6895f23a76c63adee] z3fold: extend compaction function
git bisect good 1b096e5ae9f7181c770d59c6895f23a76c63adee
# good: [8703e8a465b1e9cadc3680b4b1248f5987e54518] sched/headers: Prepare for new header dependencies before moving code to <linux/sched/user.h>
git bisect good 8703e8a465b1e9cadc3680b4b1248f5987e54518
# bad: [8fe3ccaed080a7804bc459c42e0419253973be92] Merge branch 'akpm' (patches from Andrew)
git bisect bad 8fe3ccaed080a7804bc459c42e0419253973be92
# good: [1827adb11ad26b2290dc9fe2aaf54976b2439865] Merge branch 'WIP.sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 1827adb11ad26b2290dc9fe2aaf54976b2439865
# bad: [8346242a7e32c4c93316684ad8f45473932586b9] rmap: fix NULL-pointer dereference on THP munlocking
git bisect bad 8346242a7e32c4c93316684ad8f45473932586b9
# bad: [ce9311cf95ad8baf044a014738d76973d93b739a] mm/vmstats: add thp_split_pud event for clarity
git bisect bad ce9311cf95ad8baf044a014738d76973d93b739a
# good: [590dce2d4934fb909b112cd80c80486362337744] Merge branch 'rebased-statx' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
git bisect good 590dce2d4934fb909b112cd80c80486362337744
# bad: [b4fb8f66f1ae2e167d06c12d018025a8d4d3ba7e] mm, page_alloc: Add missing check for memory holes
git bisect bad b4fb8f66f1ae2e167d06c12d018025a8d4d3ba7e
# first bad commit: [b4fb8f66f1ae2e167d06c12d018025a8d4d3ba7e] mm, page_alloc: Add missing check for memory holes
----------
I haven't completed bisecting between b4fb8f66f1ae2e16 and c470abd4fde40ea6, but
b4fb8f66f1ae2e16 ("mm, page_alloc: Add missing check for memory holes") and
13ad59df67f19788 ("mm, page_alloc: avoid page_to_pfn() when merging buddies")
are talking about memory holes, which matches the situation that I'm trivially
hitting the bug if CONFIG_SPARSEMEM=y .
----------
# bad: [b4fb8f66f1ae2e167d06c12d018025a8d4d3ba7e] mm, page_alloc: Add missing check for memory holes
# good: [590dce2d4934fb909b112cd80c80486362337744] Merge branch 'rebased-statx' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
# good: [1827adb11ad26b2290dc9fe2aaf54976b2439865] Merge branch 'WIP.sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
# good: [8703e8a465b1e9cadc3680b4b1248f5987e54518] sched/headers: Prepare for new header dependencies before moving code to <linux/sched/user.h>
# good: [1b096e5ae9f7181c770d59c6895f23a76c63adee] z3fold: extend compaction function
# good: [c470abd4fde40ea6a0846a2beab642a578c0b8cd] Linux 4.10
# good: [69973b830859bc6529a7a0468ba0d80ee5117826] Linux 4.9
# good: [c8d2bc9bc39ebea8437fd974fdbc21847bb897a3] Linux 4.8
# good: [523d939ef98fd712632d93a5a2b588e477a7565e] Linux 4.7
# good: [2dcd0af568b0cf583645c8a317dd12e344b1c72a] Linux 4.6
# good: [b562e44f507e863c6792946e4e1b1449fbbac85d] Linux 4.5
# good: [afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc] Linux 4.4
# good: [6a13feb9c82803e2b815eca72fa7a9f5561d7861] Linux 4.3
# good: [64291f7db5bd8150a74ad2036f1037e6a0428df2] Linux 4.2
# good: [b953c0d234bc72e8489d3bf51a276c5c4ec85345] Linux 4.1
# good: [39a8804455fb23f09157341d3ba7db6d7ae6ee76] Linux 4.0
# bad: [8a9172356f747dc3734cc8043a44bbe158f44749] Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
# bad: [04bb94b13c02e9dbc92d622cddf88937127bd7ed] overlayfs: remove now unnecessary header file include
# bad: [bd0f9b356d00aa241ced36fb075a07041c28d3b8] sched/headers: fix up header file dependency on <linux/sched/signal.h>
# bad: [ec3b93ae0bf4742e9cbb40e1964129926c1464e0] Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
----------
----------
[ 30.997908] a.out invoked oom-killer: gfp_mask=0x14280ca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), nodemask=(null), order=0, oom_score_adj=0
[ 31.002850] CPU: 0 PID: 274 Comm: a.out Not tainted 4.11.0-rc1-00068-g8a91723 #341
[ 31.005963] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 31.010320] Call Trace:
[ 31.011391] dump_stack+0x58/0x73
[ 31.012774] dump_header+0x92/0x202
[ 31.014319] ? ___ratelimit+0x83/0xf0
[ 31.015854] oom_kill_process+0x1e5/0x3b0
[ 31.017682] ? has_capability_noaudit+0x1f/0x30
[ 31.019629] ? oom_badness+0xcf/0x130
[ 31.021182] ? oom_evaluate_task+0x14/0xe0
[ 31.022909] out_of_memory+0xe1/0x280
[ 31.024460] __alloc_pages_nodemask+0x70d/0x9d0
[ 31.026347] ? vma_gap_callbacks_rotate+0x14/0x20
[ 31.028292] handle_mm_fault+0x599/0xd90
[ 31.029926] __do_page_fault+0x197/0x450
[ 31.031480] ? __do_page_fault+0x450/0x450
[ 31.032999] do_page_fault+0x1a/0x20
[ 31.034639] common_exception+0x6c/0x72
[ 31.036266] EIP: 0x804858f
[ 31.037430] EFLAGS: 00010202 CPU: 0
[ 31.038921] EAX: 007bc000 EBX: 376aa008 ECX: 37e66008 EDX: 00000000
[ 31.041530] ESI: 7ff00000 EDI: 00000000 EBP: bfcc7368 ESP: bfcc7330
[ 31.044118] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
[ 31.046387] Mem-Info:
[ 31.047394] active_anon:656090 inactive_anon:0 isolated_anon:0
[ 31.047394] active_file:0 inactive_file:0 isolated_file:3
[ 31.047394] unevictable:0 dirty:0 writeback:0 unstable:0
[ 31.047394] slab_reclaimable:173 slab_unreclaimable:20805
[ 31.047394] mapped:4 shmem:0 pagetables:38151 bounce:0
[ 31.047394] free:31109 free_pcp:171 free_cma:0
[ 31.060114] Node 0 active_anon:2624360kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):12kB mapped:16kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB pages_scanned:151 all_unreclaimable? yes
[ 31.071561] DMA free:12720kB min:788kB low:984kB high:1180kB active_anon:3196kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15916kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 31.080790] BUG: unable to handle kernel paging request at bc25106a
[ 31.080799] IP: page_remove_rmap+0x17/0x260
[ 31.080800] *pde = 00000000
[ 31.080801]
[ 31.080803] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 31.080804] Modules linked in:
[ 31.080807] CPU: 1 PID: 61 Comm: kswapd0 Not tainted 4.11.0-rc1-00068-g8a91723 #341
[ 31.080808] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 31.080809] task: f517e300 task.stack: f5346000
[ 31.080812] EIP: page_remove_rmap+0x17/0x260
[ 31.080812] EFLAGS: 00010286 CPU: 1
[ 31.080814] EAX: bc251066 EBX: f3f2dbf0 ECX: 01de4080 EDX: 00000000
[ 31.080815] ESI: f3f2dbf0 EDI: f5970080 EBP: f5347c24 ESP: f5347c18
[ 31.080816] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 31.080817] CR0: 80050033 CR2: bc25106a CR3: 0170f000 CR4: 000406d0
[ 31.080897] Call Trace:
[ 31.080902] try_to_unmap_one+0x210/0x4b0
[ 31.080905] rmap_walk_file+0xf0/0x200
[ 31.080907] rmap_walk+0x32/0x60
[ 31.080909] try_to_unmap+0x95/0x120
[ 31.080910] ? page_remove_rmap+0x260/0x260
[ 31.080912] ? page_not_mapped+0x10/0x10
[ 31.080914] ? page_get_anon_vma+0x90/0x90
[ 31.080916] shrink_page_list+0x387/0xbf0
[ 31.080919] shrink_inactive_list+0x173/0x370
[ 31.080921] shrink_node_memcg+0x572/0x7d0
[ 31.080924] ? __list_lru_count_one.isra.7+0x14/0x40
[ 31.080927] ? tick_program_event+0x3b/0x70
[ 31.080929] shrink_node+0xb3/0x2c0
[ 31.080931] kswapd+0x27f/0x5a0
[ 31.080935] kthread+0xd1/0x100
[ 31.080936] ? mem_cgroup_shrink_node+0xa0/0xa0
[ 31.080938] ? kthread_park+0x70/0x70
[ 31.080942] ret_from_fork+0x21/0x2c
[ 31.080943] Code: 83 e8 01 90 e9 7a ff ff ff 83 e8 01 e9 60 ff ff ff 8d 76 00 55 89 e5 56 53 89 c3 83 ec 04 8b 40 14 a8 01 0f 85 00 02 00 00 89 d8 <f6> 40 04 01 74 6b 84 d2 0f 85 3b 01 00 00 f0 83 43 0c ff 78 0c
[ 31.080981] EIP: page_remove_rmap+0x17/0x260 SS:ESP: 0068:f5347c18
[ 31.080981] CR2: 00000000bc25106a
[ 31.080983] ---[ end trace 53061eeda268128d ]---
[ 31.159021] lowmem_reserve[]: 0 827 3011 3011
[ 31.160883] Normal free:111096kB min:42088kB low:52608kB high:63128kB active_anon:385820kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:892920kB managed:847288kB mlocked:0kB slab_reclaimable:692kB slab_unreclaimable:83220kB kernel_stack:74800kB pagetables:152604kB bounce:0kB free_pcp:680kB local_pcp:120kB free_cma:0kB
[ 31.173443] lowmem_reserve[]: 0 0 17471 17471
[ 31.175185] HighMem free:620kB min:512kB low:28284kB high:56056kB active_anon:2233384kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:2236360kB managed:2236360kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 31.186860] lowmem_reserve[]: 0 0 0 0
[ 31.188445] DMA: 2*4kB (UM) 1*8kB (U) 0*16kB 1*32kB (U) 2*64kB (U) 2*128kB (UM) 2*256kB (UM) 1*512kB (M) 1*1024kB (U) 1*2048kB (U) 2*4096kB (M) = 12720kB
[ 31.194038] Normal: 41*4kB (UME) 114*8kB (UME) 58*16kB (UE) 23*32kB (UME) 22*64kB (UME) 11*128kB (UME) 6*256kB (UM) 2*512kB (UE) 1*1024kB (U) 1*2048kB (E) 24*4096kB (M) = 109492kB
[ 31.200324] HighMem: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
[ 31.204414] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=4096kB
[ 31.207586] 3 total pagecache pages
[ 31.209085] 0 pages in swap cache
[ 31.210527] Swap cache stats: add 0, delete 0, find 0/0
[ 31.212706] Free swap = 0kB
[ 31.213952] Total swap = 0kB
[ 31.215203] 786318 pages RAM
[ 31.216439] 559090 pages HighMem/MovableOnly
[ 31.218281] 11427 pages reserved
[ 31.219486] 0 pages cma reserved
[ 31.220862] Out of memory: Kill process 129 (a.out) score 5 or sacrifice child
[ 31.223843] Killed process 180 (a.out) total-vm:2108kB, anon-rss:56kB, file-rss:0kB, shmem-rss:0kB
[ 31.231024] oom_reaper: reaped process 180 (a.out), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
----------
Thus, I call for an attention by speculative execution. ;-)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-17 11:08 ` Tetsuo Handa
@ 2018-01-17 21:39 ` Linus Torvalds
2018-01-17 21:51 ` Linus Torvalds
2018-01-17 22:00 ` Dave Hansen
2018-01-18 8:12 ` Tetsuo Handa
1 sibling, 2 replies; 59+ messages in thread
From: Linus Torvalds @ 2018-01-17 21:39 UTC (permalink / raw)
To: Tetsuo Handa
Cc: Kirill A. Shutemov, Andrew Morton, Johannes Weiner, Joonsoo Kim,
Mel Gorman, Tony Luck, Vlastimil Babka, Michal Hocko,
Dave Hansen, Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Wed, Jan 17, 2018 at 3:08 AM, Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
>
> I needed to bisect between 4.10 and 4.11, and I got plausible culprit.
> [...]
> git bisect bad b4fb8f66f1ae2e167d06c12d018025a8d4d3ba7e
> # first bad commit: [b4fb8f66f1ae2e167d06c12d018025a8d4d3ba7e] mm, page_alloc: Add missing check for memory holes
Ok, that is indeed much more likely, and very much matches the whole
"this problem only happens with sparsemem" issue.
In fact, the whole
pfn_valid_within(buddy_pfn)
test looks very odd. Maybe the pfn of the buddy is valid, but it's not
in the same zone? Then we'd combine the two pages in two different
zones into one combined page.
Maybe that's why HIGHMEM matters? The low DMA zone is obviously
aligned in the whole PAGE_ORDER range. But the highmem zone might not
be. I used to know the highmem code, but I've happily forgotten
everything. But I think we end up deciding on some random non-aligned
number in the 900MB range as being the limit between the regular zone
and the HIGHMEM zone.
So maybe something like this to test the theory?
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 76c9688b6a0a..f919a5548943 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -756,6 +756,8 @@ static inline void rmv_page_order(struct page *page)
static inline int page_is_buddy(struct page *page, struct page *buddy,
unsigned int order)
{
+ if (WARN_ON_ONCE(page_zone(page) != page_zone(buddy)))
+ return 0;
if (page_is_guard(buddy) && page_order(buddy) == order) {
if (page_zone_id(page) != page_zone_id(buddy))
return 0;
I don't know. Does that warning trigger for you?
The above is completely untested. It might not compile. If it compiles
it might not work. And even if it "works", it might not matter,
because perhaps the boundary between regular memory and HIGHMEM is
already sufficiently aligned.
Comments?
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-17 21:39 ` Linus Torvalds
@ 2018-01-17 21:51 ` Linus Torvalds
2018-01-17 22:04 ` Dave Hansen
2018-01-17 22:00 ` Dave Hansen
1 sibling, 1 reply; 59+ messages in thread
From: Linus Torvalds @ 2018-01-17 21:51 UTC (permalink / raw)
To: Tetsuo Handa
Cc: Kirill A. Shutemov, Andrew Morton, Johannes Weiner, Joonsoo Kim,
Mel Gorman, Tony Luck, Vlastimil Babka, Michal Hocko,
Dave Hansen, Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Wed, Jan 17, 2018 at 1:39 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> In fact, the whole
>
> pfn_valid_within(buddy_pfn)
>
> test looks very odd. Maybe the pfn of the buddy is valid, but it's not
> in the same zone? Then we'd combine the two pages in two different
> zones into one combined page.
It might also be the same allocation zone, but if the pfn's are in
different sparsemem sections that would also be problematic.
But I hope/assume that all sparsemem sections are always aligned to
(PAGE_SIZE << MAXORDER).
In contrast, the ZONE_HIGHMEM limit really does seems to be
potentially not aligned to anything, ie
arch/x86/include/asm/pgtable_32_types.h:
#define MAXMEM (VMALLOC_END - PAGE_OFFSET - __VMALLOC_RESERVE)
which I have no idea what the alignment is, but VMALLOC_END at least
does not seem to have any MAXORDER alignment.
So it really does look like the zone for two page orders that would
otherwise be buddies might actually be different.
Interesting if this really is the case. Because afaik, if that
WARN_ON_ONCE actually triggers, it does seem like this bug could go
back pretty much forever.
In fact, it seems to be such a fundamental bug that I suspect I'm
entirely wrong, and full of shit. So it's an interesting and not
_obviously_ incorrect theory, but I suspect I must be missing
something.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-17 21:51 ` Linus Torvalds
@ 2018-01-17 22:04 ` Dave Hansen
0 siblings, 0 replies; 59+ messages in thread
From: Dave Hansen @ 2018-01-17 22:04 UTC (permalink / raw)
To: Linus Torvalds, Tetsuo Handa
Cc: Kirill A. Shutemov, Andrew Morton, Johannes Weiner, Joonsoo Kim,
Mel Gorman, Tony Luck, Vlastimil Babka, Michal Hocko,
Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On 01/17/2018 01:51 PM, Linus Torvalds wrote:
> In fact, it seems to be such a fundamental bug that I suspect I'm
> entirely wrong, and full of shit. So it's an interesting and not
> _obviously_ incorrect theory, but I suspect I must be missing
> something.
I'll just note that a few of the pfns I decoded were smack in the middle
of the zone, not near either the high or low end of ZONE_NORMAL where we
would expect this cross-zone stuff to happen.
But I guess we could get similar wonkiness where 'struct page' is
screwed up in so many different ways if during buddy joining you do:
list_del(&buddy->lru);
and 'buddy' is off in another zone for which you do not hold the
spinlock. If we are somehow missing some locking, or double-allocating
a page, something like this would help:
static inline void rmv_page_order(struct page *page)
{
+ WARN_ON_ONCE(!PageBuddy(page));
__ClearPageBuddy(page);
set_page_private(page, 0);
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-17 21:39 ` Linus Torvalds
2018-01-17 21:51 ` Linus Torvalds
@ 2018-01-17 22:00 ` Dave Hansen
2018-01-17 22:15 ` Linus Torvalds
1 sibling, 1 reply; 59+ messages in thread
From: Dave Hansen @ 2018-01-17 22:00 UTC (permalink / raw)
To: Linus Torvalds, Tetsuo Handa
Cc: Kirill A. Shutemov, Andrew Morton, Johannes Weiner, Joonsoo Kim,
Mel Gorman, Tony Luck, Vlastimil Babka, Michal Hocko,
Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On 01/17/2018 01:39 PM, Linus Torvalds wrote:
>
> So maybe something like this to test the theory?
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 76c9688b6a0a..f919a5548943 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -756,6 +756,8 @@ static inline void rmv_page_order(struct page *page)
> static inline int page_is_buddy(struct page *page, struct page *buddy,
> unsigned int order)
> {
> + if (WARN_ON_ONCE(page_zone(page) != page_zone(buddy)))
> + return 0;
> if (page_is_guard(buddy) && page_order(buddy) == order) {
> if (page_zone_id(page) != page_zone_id(buddy))
> return 0;
I thought that page_zone_id() stuff was there to prevent this kind of
cross-zone stuff from happening.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-17 22:00 ` Dave Hansen
@ 2018-01-17 22:15 ` Linus Torvalds
0 siblings, 0 replies; 59+ messages in thread
From: Linus Torvalds @ 2018-01-17 22:15 UTC (permalink / raw)
To: Dave Hansen
Cc: Tetsuo Handa, Kirill A. Shutemov, Andrew Morton, Johannes Weiner,
Joonsoo Kim, Mel Gorman, Tony Luck, Vlastimil Babka,
Michal Hocko, Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Wed, Jan 17, 2018 at 2:00 PM, Dave Hansen
<dave.hansen@linux.intel.com> wrote:
>
> I thought that page_zone_id() stuff was there to prevent this kind of
> cross-zone stuff from happening.
Ahh, that was the part I missed. Yeah looks like that checks things
properly. Although the mask generation is *so* confusing that I
stopped following it and will just take your word for it ;)
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-17 11:08 ` Tetsuo Handa
2018-01-17 21:39 ` Linus Torvalds
@ 2018-01-18 8:12 ` Tetsuo Handa
2018-01-18 12:25 ` Kirill A. Shutemov
1 sibling, 1 reply; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-18 8:12 UTC (permalink / raw)
To: torvalds, kirill.shutemov, akpm, hannes, iamjoonsoo.kim, mgorman,
tony.luck, vbabka, mhocko, aarcange, hillf.zj, hughd, oleg,
peterz, riel, srikar, vdavydov.dev
Cc: dave.hansen, mingo, linux-kernel, linux-mm, x86
Tetsuo Handa wrote:
> OK. I missed the mark. I overlooked that 4.11 already has this problem.
>
> I needed to bisect between 4.10 and 4.11, and I got plausible culprit.
>
> I haven't completed bisecting between b4fb8f66f1ae2e16 and c470abd4fde40ea6, but
> b4fb8f66f1ae2e16 ("mm, page_alloc: Add missing check for memory holes") and
> 13ad59df67f19788 ("mm, page_alloc: avoid page_to_pfn() when merging buddies")
> are talking about memory holes, which matches the situation that I'm trivially
> hitting the bug if CONFIG_SPARSEMEM=y .
>
> Thus, I call for an attention by speculative execution. ;-)
Speculative execution failed. I was confused by jiffies precision bug.
The final culprit is c7ab0d2fdc840266 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()").
----------------------------------------
[ 103.132986] BUG: unable to handle kernel paging request at b6c00171
[ 103.132996] IP: lock_page_memcg+0x3a/0x70
[ 103.132997] *pde = 00000000
[ 103.132997]
[ 103.132999] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 103.133000] Modules linked in: pcspkr shpchp serio_raw
[ 103.133006] CPU: 3 PID: 62 Comm: kswapd0 Not tainted 4.10.0-09628-gc7ab0d2-dirty #359
[ 103.133007] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[ 103.133008] task: f4559b00 task.stack: f2c74000
[ 103.133011] EIP: lock_page_memcg+0x3a/0x70
[ 103.133012] EFLAGS: 00010282 CPU: 3
[ 103.133013] EAX: f3108060 EBX: b6c00001 ECX: 01c8e5a0 EDX: 00000000
[ 103.133014] ESI: f4afe5a0 EDI: f3108060 EBP: f2c75c10 ESP: f2c75c04
[ 103.133015] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 103.133017] CR0: 80050033 CR2: b6c00171 CR3: 0170a000 CR4: 000406d0
[ 103.133098] Call Trace:
[ 103.133104] page_remove_rmap+0x92/0x260
[ 103.133106] try_to_unmap_one+0x210/0x4b0
[ 103.133108] rmap_walk_file+0xf0/0x200
[ 103.133111] rmap_walk+0x32/0x60
[ 103.133112] try_to_unmap+0x95/0x120
[ 103.133114] ? page_remove_rmap+0x260/0x260
[ 103.133116] ? page_not_mapped+0x10/0x10
[ 103.133118] ? page_get_anon_vma+0x90/0x90
[ 103.133120] shrink_page_list+0x3af/0xc40
[ 103.133123] shrink_inactive_list+0x173/0x370
[ 103.133125] shrink_node_memcg+0x572/0x7d0
[ 103.133128] ? __list_lru_count_one.isra.5+0x14/0x40
[ 103.133130] shrink_node+0xb3/0x2c0
[ 103.133132] kswapd+0x27f/0x5a0
[ 103.133137] kthread+0xd1/0x100
[ 103.133139] ? mem_cgroup_shrink_node+0xa0/0xa0
[ 103.133140] ? kthread_park+0x70/0x70
[ 103.133144] ret_from_fork+0x21/0x2c
[ 103.133145] Code: 20 85 db 75 26 eb 2e 66 90 8d b3 74 01 00 00 89 f0 e8 9b 5a 37 00 3b 5f 20 74 26 89 c2 89 f0 e8 3d 57 37 00 8b 5f 20 85 db 74 0a <8b> 83 70 01 00 00 85 c0 7f d4 5b 5e 5f 5d f3 c3 8d b6 00 00 00
[ 103.133171] EIP: lock_page_memcg+0x3a/0x70 SS:ESP: 0068:f2c75c04
[ 103.133172] CR2: 00000000b6c00171
[ 103.133175] ---[ end trace fa59c5a5ab752d7a ]---
----------------------------------------
----------------------------------------
# bad: [86292b33d4b79ee03e2f43ea0381ef85f077c760] Merge branch 'akpm' (patches from Andrew)
# good: [13ad59df67f19788f6c22985b1a33e466eceb643] mm, page_alloc: avoid page_to_pfn() when merging buddies
git bisect start '86292b33d4b79ee03e2f43ea0381ef85f077c760' '13ad59df67f19788' 'mm/'
# good: [0262d9c845ec349edf93f69688a5129c36cc2232] memblock: embed memblock type name within struct memblock_type
git bisect good 0262d9c845ec349edf93f69688a5129c36cc2232
# good: [0262d9c845ec349edf93f69688a5129c36cc2232] memblock: embed memblock type name within struct memblock_type
git bisect good 0262d9c845ec349edf93f69688a5129c36cc2232
# bad: [897ab3e0c49e24b62e2d54d165c7afec6bbca65b] userfaultfd: non-cooperative: add event for memory unmaps
git bisect bad 897ab3e0c49e24b62e2d54d165c7afec6bbca65b
# good: [c791ace1e747371658237f0d30234fef56c39669] mm: replace FAULT_FLAG_SIZE with parameter to huge_fault
git bisect good c791ace1e747371658237f0d30234fef56c39669
# good: [8eaedede825a02dbe2420b9c9be9b5b2d7515496] mm: fix handling PTE-mapped THPs in page_referenced()
git bisect good 8eaedede825a02dbe2420b9c9be9b5b2d7515496
# bad: [36eaff3364e8cd35552a77ee426a8170f4f5fde9] mm, ksm: convert write_protect_page() to use page_vma_mapped_walk()
git bisect bad 36eaff3364e8cd35552a77ee426a8170f4f5fde9
# good: [a8fa41ad2f6f7ca08edd1afcf8149ae5a4dcf654] mm, rmap: check all VMAs that PTE-mapped THP can be part of
git bisect good a8fa41ad2f6f7ca08edd1afcf8149ae5a4dcf654
# bad: [c7ab0d2fdc840266b39db94538f74207ec2afbf6] mm: convert try_to_unmap_one() to use page_vma_mapped_walk()
git bisect bad c7ab0d2fdc840266b39db94538f74207ec2afbf6
# good: [f27176cfc363d395eea8dc5c4a26e5d6d7d65eaf] mm: convert page_mkclean_one() to use page_vma_mapped_walk()
git bisect good f27176cfc363d395eea8dc5c4a26e5d6d7d65eaf
# first bad commit: [c7ab0d2fdc840266b39db94538f74207ec2afbf6] mm: convert try_to_unmap_one() to use page_vma_mapped_walk()
bad 4.10.0-10531-g86292b3-dirty # 86292b33d4b79ee0 ("Merge branch 'akpm' (patches from Andrew)")
bad 4.10.0-09635-g897ab3e-dirty
bad 4.10.0-09629-g36eaff3-dirty
bad 4.10.0-09628-gc7ab0d2-dirty
good 4.10.0-09627-gf27176c-dirty
good 4.10.0-09626-ga8fa41ad-dirty
good 4.10.0-09624-g8eaeded-dirty
good 4.10.0-09611-gc791ace-dirty
good 4.10.0-09588-g0262d9c-dirty
good 4.10.0-06032-g13ad59d-dirty # 13ad59df67f19788 ("mm, page_alloc: avoid page_to_pfn() when merging buddies")
----------------------------------------
The "-dirty" part is due to testing with below patch (jiffies fix,
warn_alloc() lockup avoidance and disabling the OOM reaper).
----------------------------------------
--- a/kernel/time/jiffies.c
+++ b/kernel/time/jiffies.c
@@ -125,7 +125,7 @@ int register_refined_jiffies(long cycles_per_second)
shift_hz += cycles_per_tick/2;
do_div(shift_hz, cycles_per_tick);
/* Calculate nsec_per_tick using shift_hz */
- nsec_per_tick = (u64)TICK_NSEC << 8;
+ nsec_per_tick = (u64)NSEC_PER_SEC << 8;
nsec_per_tick += (u32)shift_hz/2;
do_div(nsec_per_tick, (u32)shift_hz);
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -553,7 +553,7 @@ static void oom_reap_task(struct task_struct *tsk)
struct mm_struct *mm = tsk->signal->oom_mm;
/* Retry the down_read_trylock(mmap_sem) a few times */
- while (attempts++ < MAX_OOM_REAP_RETRIES && !__oom_reap_task_mm(tsk, mm))
+ //while (attempts++ < MAX_OOM_REAP_RETRIES && !__oom_reap_task_mm(tsk, mm))
schedule_timeout_idle(HZ/10);
if (attempts <= MAX_OOM_REAP_RETRIES)
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3765,7 +3765,7 @@ bool gfp_pfmemalloc_allowed(gfp_t gfp_mask)
goto nopage;
/* Make sure we know about allocations which stall for too long */
- if (time_after(jiffies, alloc_start + stall_timeout)) {
+ if (0 && time_after(jiffies, alloc_start + stall_timeout)) {
warn_alloc(gfp_mask, ac->nodemask,
"page allocation stalls for %ums, order:%u",
jiffies_to_msecs(jiffies-alloc_start), order);
----------------------------------------
Michal Hocko wrote:
> Yes, this looks like somebody is clobbering the page. I've seen one with
> refcount 0 so I though this would be a ref count issue. But the one
> below looks definitely like a memory corruption. A nasty one to debug :/
Yes, I guess that c7ab0d2fdc840266 is somewhere doing wrong ref count check.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 8:12 ` Tetsuo Handa
@ 2018-01-18 12:25 ` Kirill A. Shutemov
2018-01-18 13:12 ` Kirill A. Shutemov
2018-01-18 14:45 ` Dave Hansen
0 siblings, 2 replies; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-18 12:25 UTC (permalink / raw)
To: Tetsuo Handa
Cc: torvalds, kirill.shutemov, akpm, hannes, iamjoonsoo.kim, mgorman,
tony.luck, vbabka, mhocko, aarcange, hillf.zj, hughd, oleg,
peterz, riel, srikar, vdavydov.dev, dave.hansen, mingo,
linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 05:12:45PM +0900, Tetsuo Handa wrote:
> Tetsuo Handa wrote:
> > OK. I missed the mark. I overlooked that 4.11 already has this problem.
> >
> > I needed to bisect between 4.10 and 4.11, and I got plausible culprit.
> >
> > I haven't completed bisecting between b4fb8f66f1ae2e16 and c470abd4fde40ea6, but
> > b4fb8f66f1ae2e16 ("mm, page_alloc: Add missing check for memory holes") and
> > 13ad59df67f19788 ("mm, page_alloc: avoid page_to_pfn() when merging buddies")
> > are talking about memory holes, which matches the situation that I'm trivially
> > hitting the bug if CONFIG_SPARSEMEM=y .
> >
> > Thus, I call for an attention by speculative execution. ;-)
>
> Speculative execution failed. I was confused by jiffies precision bug.
> The final culprit is c7ab0d2fdc840266 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()").
I think I've tracked it down. check_pte() in mm/page_vma_mapped.c doesn't
work as intended.
I've added instrumentation below to prove it.
The BUG() triggers with following output:
[ 10.084024] diff: -858690919
[ 10.084258] hpage_nr_pages: 1
[ 10.084386] check1: 0
[ 10.084478] check2: 0
Basically, pte_page(*pvmw->pte) is below pvmw->page, but
(pte_page(*pvmw->pte) < pvmw->page) doesn't catch it.
Well, I can see how C lawyer can argue that you can only compare pointers
of the same memory object which is not the case here. But this is kinda
insane.
Any suggestions how to rewrite it in a way that compiler would
understand?
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index d22b84310f6d..57b4397f1ea5 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -70,6 +70,14 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
}
if (pte_page(*pvmw->pte) < pvmw->page)
return false;
+
+ if (pte_page(*pvmw->pte) - pvmw->page) {
+ printk("diff: %d\n", pte_page(*pvmw->pte) - pvmw->page);
+ printk("hpage_nr_pages: %d\n", hpage_nr_pages(pvmw->page));
+ printk("check1: %d\n", pte_page(*pvmw->pte) - pvmw->page < 0);
+ printk("check2: %d\n", pte_page(*pvmw->pte) - pvmw->page >= hpage_nr_pages(pvmw->page));
+ BUG();
+ }
}
return true;
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 12:25 ` Kirill A. Shutemov
@ 2018-01-18 13:12 ` Kirill A. Shutemov
2018-01-18 14:34 ` Kirill A. Shutemov
2018-01-18 14:38 ` Dave Hansen
2018-01-18 14:45 ` Dave Hansen
1 sibling, 2 replies; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-18 13:12 UTC (permalink / raw)
To: Tetsuo Handa
Cc: torvalds, kirill.shutemov, akpm, hannes, iamjoonsoo.kim, mgorman,
tony.luck, vbabka, mhocko, aarcange, hillf.zj, hughd, oleg,
peterz, riel, srikar, vdavydov.dev, dave.hansen, mingo,
linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 03:25:50PM +0300, Kirill A. Shutemov wrote:
> On Thu, Jan 18, 2018 at 05:12:45PM +0900, Tetsuo Handa wrote:
> > Tetsuo Handa wrote:
> > > OK. I missed the mark. I overlooked that 4.11 already has this problem.
> > >
> > > I needed to bisect between 4.10 and 4.11, and I got plausible culprit.
> > >
> > > I haven't completed bisecting between b4fb8f66f1ae2e16 and c470abd4fde40ea6, but
> > > b4fb8f66f1ae2e16 ("mm, page_alloc: Add missing check for memory holes") and
> > > 13ad59df67f19788 ("mm, page_alloc: avoid page_to_pfn() when merging buddies")
> > > are talking about memory holes, which matches the situation that I'm trivially
> > > hitting the bug if CONFIG_SPARSEMEM=y .
> > >
> > > Thus, I call for an attention by speculative execution. ;-)
> >
> > Speculative execution failed. I was confused by jiffies precision bug.
> > The final culprit is c7ab0d2fdc840266 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()").
>
> I think I've tracked it down. check_pte() in mm/page_vma_mapped.c doesn't
> work as intended.
>
> I've added instrumentation below to prove it.
>
> The BUG() triggers with following output:
>
> [ 10.084024] diff: -858690919
> [ 10.084258] hpage_nr_pages: 1
> [ 10.084386] check1: 0
> [ 10.084478] check2: 0
>
> Basically, pte_page(*pvmw->pte) is below pvmw->page, but
> (pte_page(*pvmw->pte) < pvmw->page) doesn't catch it.
>
> Well, I can see how C lawyer can argue that you can only compare pointers
> of the same memory object which is not the case here. But this is kinda
> insane.
>
> Any suggestions how to rewrite it in a way that compiler would
> understand?
The patch below makes the crash go away for me.
But this is situation is scary. So we cannot compare arbitrary pointers in
kernel?
Don't we rely on this for lock ordering in some cases? Like in
mutex_lock_double()?
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index d22b84310f6d..1f0f512fd127 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -51,6 +51,8 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
WARN_ON_ONCE(1);
#endif
} else {
+ unsigned long ptr1, ptr2;
+
if (is_swap_pte(*pvmw->pte)) {
swp_entry_t entry;
@@ -63,12 +65,14 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
if (!pte_present(*pvmw->pte))
return false;
- /* THP can be referenced by any subpage */
- if (pte_page(*pvmw->pte) - pvmw->page >=
- hpage_nr_pages(pvmw->page)) {
+ ptr1 = (unsigned long)pte_page(*pvmw->pte);
+ ptr2 = (unsigned long)pvmw->page;
+
+ if (ptr1 < ptr2)
return false;
- }
- if (pte_page(*pvmw->pte) < pvmw->page)
+
+ /* THP can be referenced by any subpage */
+ if (ptr1 - ptr2 >= hpage_nr_pages(pvmw->page))
return false;
}
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 13:12 ` Kirill A. Shutemov
@ 2018-01-18 14:34 ` Kirill A. Shutemov
2018-01-18 14:38 ` Dave Hansen
1 sibling, 0 replies; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-18 14:34 UTC (permalink / raw)
To: Tetsuo Handa
Cc: torvalds, kirill.shutemov, akpm, hannes, iamjoonsoo.kim, mgorman,
tony.luck, vbabka, mhocko, aarcange, hillf.zj, hughd, oleg,
peterz, riel, srikar, vdavydov.dev, dave.hansen, mingo,
linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 04:12:10PM +0300, Kirill A. Shutemov wrote:
> On Thu, Jan 18, 2018 at 03:25:50PM +0300, Kirill A. Shutemov wrote:
> > On Thu, Jan 18, 2018 at 05:12:45PM +0900, Tetsuo Handa wrote:
> > > Tetsuo Handa wrote:
> > > > OK. I missed the mark. I overlooked that 4.11 already has this problem.
> > > >
> > > > I needed to bisect between 4.10 and 4.11, and I got plausible culprit.
> > > >
> > > > I haven't completed bisecting between b4fb8f66f1ae2e16 and c470abd4fde40ea6, but
> > > > b4fb8f66f1ae2e16 ("mm, page_alloc: Add missing check for memory holes") and
> > > > 13ad59df67f19788 ("mm, page_alloc: avoid page_to_pfn() when merging buddies")
> > > > are talking about memory holes, which matches the situation that I'm trivially
> > > > hitting the bug if CONFIG_SPARSEMEM=y .
> > > >
> > > > Thus, I call for an attention by speculative execution. ;-)
> > >
> > > Speculative execution failed. I was confused by jiffies precision bug.
> > > The final culprit is c7ab0d2fdc840266 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()").
> >
> > I think I've tracked it down. check_pte() in mm/page_vma_mapped.c doesn't
> > work as intended.
> >
> > I've added instrumentation below to prove it.
> >
> > The BUG() triggers with following output:
> >
> > [ 10.084024] diff: -858690919
> > [ 10.084258] hpage_nr_pages: 1
> > [ 10.084386] check1: 0
> > [ 10.084478] check2: 0
> >
> > Basically, pte_page(*pvmw->pte) is below pvmw->page, but
> > (pte_page(*pvmw->pte) < pvmw->page) doesn't catch it.
> >
> > Well, I can see how C lawyer can argue that you can only compare pointers
> > of the same memory object which is not the case here. But this is kinda
> > insane.
> >
> > Any suggestions how to rewrite it in a way that compiler would
> > understand?
>
> The patch below makes the crash go away for me.
>
> But this is situation is scary. So we cannot compare arbitrary pointers in
> kernel?
>
> Don't we rely on this for lock ordering in some cases? Like in
> mutex_lock_double()?
>
> diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
> index d22b84310f6d..1f0f512fd127 100644
> --- a/mm/page_vma_mapped.c
> +++ b/mm/page_vma_mapped.c
> @@ -51,6 +51,8 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
> WARN_ON_ONCE(1);
> #endif
> } else {
> + unsigned long ptr1, ptr2;
> +
> if (is_swap_pte(*pvmw->pte)) {
> swp_entry_t entry;
>
> @@ -63,12 +65,14 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
> if (!pte_present(*pvmw->pte))
> return false;
>
> - /* THP can be referenced by any subpage */
> - if (pte_page(*pvmw->pte) - pvmw->page >=
> - hpage_nr_pages(pvmw->page)) {
> + ptr1 = (unsigned long)pte_page(*pvmw->pte);
> + ptr2 = (unsigned long)pvmw->page;
> +
> + if (ptr1 < ptr2)
> return false;
> - }
> - if (pte_page(*pvmw->pte) < pvmw->page)
> +
> + /* THP can be referenced by any subpage */
> + if (ptr1 - ptr2 >= hpage_nr_pages(pvmw->page))
Arghhh.. It has to be
if (ptr1 - ptr2 >= hpage_nr_pages(pvmw->page) * sizeof(*pvmw->page))
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 13:12 ` Kirill A. Shutemov
2018-01-18 14:34 ` Kirill A. Shutemov
@ 2018-01-18 14:38 ` Dave Hansen
2018-01-18 14:45 ` Kirill A. Shutemov
2018-01-18 16:58 ` Linus Torvalds
1 sibling, 2 replies; 59+ messages in thread
From: Dave Hansen @ 2018-01-18 14:38 UTC (permalink / raw)
To: Kirill A. Shutemov, Tetsuo Handa
Cc: torvalds, kirill.shutemov, akpm, hannes, iamjoonsoo.kim, mgorman,
tony.luck, vbabka, mhocko, aarcange, hillf.zj, hughd, oleg,
peterz, riel, srikar, vdavydov.dev, mingo, linux-kernel,
linux-mm, x86
On 01/18/2018 05:12 AM, Kirill A. Shutemov wrote:
> - if (pte_page(*pvmw->pte) - pvmw->page >=
> - hpage_nr_pages(pvmw->page)) {
Is ->pte guaranteed to map a page which is within the same section as
pvmw->page? Otherwise, with sparsemem (non-vmemmap), the pointer
arithmetic won't work.
WARN_ON_ONCE(page_to_section(pvmw->page) !=
page_to_section(pte_page(*pvmw->pte));
Will detect that.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 14:38 ` Dave Hansen
@ 2018-01-18 14:45 ` Kirill A. Shutemov
2018-01-18 14:51 ` Dave Hansen
2018-01-18 16:58 ` Linus Torvalds
1 sibling, 1 reply; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-18 14:45 UTC (permalink / raw)
To: Dave Hansen
Cc: Tetsuo Handa, torvalds, kirill.shutemov, akpm, hannes,
iamjoonsoo.kim, mgorman, tony.luck, vbabka, mhocko, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 06:38:10AM -0800, Dave Hansen wrote:
> On 01/18/2018 05:12 AM, Kirill A. Shutemov wrote:
> > - if (pte_page(*pvmw->pte) - pvmw->page >=
> > - hpage_nr_pages(pvmw->page)) {
>
> Is ->pte guaranteed to map a page which is within the same section as
> pvmw->page? Otherwise, with sparsemem (non-vmemmap), the pointer
> arithmetic won't work.
No, it's not guaranteed. It can be arbitrary page.
The arithmetic won't work because they are different "memory objects"?
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 14:45 ` Kirill A. Shutemov
@ 2018-01-18 14:51 ` Dave Hansen
0 siblings, 0 replies; 59+ messages in thread
From: Dave Hansen @ 2018-01-18 14:51 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Tetsuo Handa, torvalds, kirill.shutemov, akpm, hannes,
iamjoonsoo.kim, mgorman, tony.luck, vbabka, mhocko, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On 01/18/2018 06:45 AM, Kirill A. Shutemov wrote:
> On Thu, Jan 18, 2018 at 06:38:10AM -0800, Dave Hansen wrote:
>> On 01/18/2018 05:12 AM, Kirill A. Shutemov wrote:
>>> - if (pte_page(*pvmw->pte) - pvmw->page >=
>>> - hpage_nr_pages(pvmw->page)) {
>> Is ->pte guaranteed to map a page which is within the same section as
>> pvmw->page? Otherwise, with sparsemem (non-vmemmap), the pointer
>> arithmetic won't work.
> No, it's not guaranteed. It can be arbitrary page.
>
> The arithmetic won't work because they are different "memory objects"?
No, because sections' mem_map[]s can be allocated non-contiguously.
Section 1's might be a lower virtual address than Section 0's.
They're allocated not unlike this:
mem_section[0]->section_mem_map = kmalloc(SECTION_SIZE);
mem_section[1]->section_mem_map = kmalloc(SECTION_SIZE);
...
The first pfn in section 1 and the last pfn in section 0 are adjacent
PFNs, but their 'struct page' might have virtual addresses that are TB
apart.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 14:38 ` Dave Hansen
2018-01-18 14:45 ` Kirill A. Shutemov
@ 2018-01-18 16:58 ` Linus Torvalds
1 sibling, 0 replies; 59+ messages in thread
From: Linus Torvalds @ 2018-01-18 16:58 UTC (permalink / raw)
To: Dave Hansen
Cc: Kirill A. Shutemov, Tetsuo Handa, Kirill A. Shutemov,
Andrew Morton, Johannes Weiner, Joonsoo Kim, Mel Gorman,
Tony Luck, Vlastimil Babka, Michal Hocko, Andrea Arcangeli,
hillf.zj, Hugh Dickins, Oleg Nesterov, Peter Zijlstra,
Rik van Riel, Srikar Dronamraju, Vladimir Davydov, Ingo Molnar,
Linux Kernel Mailing List, linux-mm, the arch/x86 maintainers
On Thu, Jan 18, 2018 at 6:38 AM, Dave Hansen
<dave.hansen@linux.intel.com> wrote:
> On 01/18/2018 05:12 AM, Kirill A. Shutemov wrote:
>> - if (pte_page(*pvmw->pte) - pvmw->page >=
>> - hpage_nr_pages(pvmw->page)) {
>
> Is ->pte guaranteed to map a page which is within the same section as
> pvmw->page? Otherwise, with sparsemem (non-vmemmap), the pointer
> arithmetic won't work.
Lovely.
Finally a reason for this bug that actually seems to make sense.
Thanks guys.
Tetsuo - does Kirill's latest patch fix this for you? The one with
Subject: [PATCH] mm, page_vma_mapped: Fix pointer arithmetics in check_pte()
in the body of the email? I'm really hoping it does, since this seems
to make sense.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 12:25 ` Kirill A. Shutemov
2018-01-18 13:12 ` Kirill A. Shutemov
@ 2018-01-18 14:45 ` Dave Hansen
2018-01-18 14:58 ` Andrea Arcangeli
2018-01-18 15:40 ` Kirill A. Shutemov
1 sibling, 2 replies; 59+ messages in thread
From: Dave Hansen @ 2018-01-18 14:45 UTC (permalink / raw)
To: Kirill A. Shutemov, Tetsuo Handa
Cc: torvalds, kirill.shutemov, akpm, hannes, iamjoonsoo.kim, mgorman,
tony.luck, vbabka, mhocko, aarcange, hillf.zj, hughd, oleg,
peterz, riel, srikar, vdavydov.dev, mingo, linux-kernel,
linux-mm, x86
On 01/18/2018 04:25 AM, Kirill A. Shutemov wrote:
> [ 10.084024] diff: -858690919
> [ 10.084258] hpage_nr_pages: 1
> [ 10.084386] check1: 0
> [ 10.084478] check2: 0
...
> diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
> index d22b84310f6d..57b4397f1ea5 100644
> --- a/mm/page_vma_mapped.c
> +++ b/mm/page_vma_mapped.c
> @@ -70,6 +70,14 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
> }
> if (pte_page(*pvmw->pte) < pvmw->page)
> return false;
> +
> + if (pte_page(*pvmw->pte) - pvmw->page) {
> + printk("diff: %d\n", pte_page(*pvmw->pte) - pvmw->page);
> + printk("hpage_nr_pages: %d\n", hpage_nr_pages(pvmw->page));
> + printk("check1: %d\n", pte_page(*pvmw->pte) - pvmw->page < 0);
> + printk("check2: %d\n", pte_page(*pvmw->pte) - pvmw->page >= hpage_nr_pages(pvmw->page));
> + BUG();
> + }
This says that pte_page(*pvmw->pte) and pvmw->page are roughly 4GB away
from each other (858690919*4=0xccba559c0). That's not the compiler
being wonky, it just means that the virtual addresses of the memory
sections are that far apart.
This won't happen when you have vmemmap or flatmem because the mem_map[]
is virtually contiguous and pointer arithmetic just works against all
'struct page' pointers. But with classic sparsemem, it doesn't.
You need to make sure that the PFNs are in the same section before you
can do the math that you want to do here.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 14:45 ` Dave Hansen
@ 2018-01-18 14:58 ` Andrea Arcangeli
2018-01-18 16:56 ` Kirill A. Shutemov
2018-01-18 15:40 ` Kirill A. Shutemov
1 sibling, 1 reply; 59+ messages in thread
From: Andrea Arcangeli @ 2018-01-18 14:58 UTC (permalink / raw)
To: Dave Hansen
Cc: Kirill A. Shutemov, Tetsuo Handa, torvalds, kirill.shutemov,
akpm, hannes, iamjoonsoo.kim, mgorman, tony.luck, vbabka, mhocko,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 06:45:00AM -0800, Dave Hansen wrote:
> On 01/18/2018 04:25 AM, Kirill A. Shutemov wrote:
> > [ 10.084024] diff: -858690919
> > [ 10.084258] hpage_nr_pages: 1
> > [ 10.084386] check1: 0
> > [ 10.084478] check2: 0
> ...
> > diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
> > index d22b84310f6d..57b4397f1ea5 100644
> > --- a/mm/page_vma_mapped.c
> > +++ b/mm/page_vma_mapped.c
> > @@ -70,6 +70,14 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
> > }
> > if (pte_page(*pvmw->pte) < pvmw->page)
> > return false;
> > +
> > + if (pte_page(*pvmw->pte) - pvmw->page) {
> > + printk("diff: %d\n", pte_page(*pvmw->pte) - pvmw->page);
> > + printk("hpage_nr_pages: %d\n", hpage_nr_pages(pvmw->page));
> > + printk("check1: %d\n", pte_page(*pvmw->pte) - pvmw->page < 0);
> > + printk("check2: %d\n", pte_page(*pvmw->pte) - pvmw->page >= hpage_nr_pages(pvmw->page));
> > + BUG();
> > + }
>
> This says that pte_page(*pvmw->pte) and pvmw->page are roughly 4GB away
> from each other (858690919*4=0xccba559c0). That's not the compiler
> being wonky, it just means that the virtual addresses of the memory
> sections are that far apart.
>
> This won't happen when you have vmemmap or flatmem because the mem_map[]
> is virtually contiguous and pointer arithmetic just works against all
> 'struct page' pointers. But with classic sparsemem, it doesn't.
>
> You need to make sure that the PFNs are in the same section before you
> can do the math that you want to do here.
Isn't it simply that pvmw->page isn't a page or pte_page(*pvmw->pte)
isn't a page?
The distance cannot matter, MMU isn't involved, this is pure 64bit
aritmetics, 1giga 1 terabyte, 48bits 5level pagetables are meaningless
in this comparison.
#include <stdio.h>
int main()
{
volatile long i;
struct x { char a[4000000000]; };
for (i = 0; i < 4000000000*3; i += 4000000000) {
printf("%ld\n", ((struct x *)0)-((((struct x *)i))));
}
printf("xxxx\n");
for (i = 0; i < 4000000000; i += 1) {
if (i==4)
i = 4000000000;
printf("%ld\n", ((struct x *)0)-((((struct x *)i))));
}
return 0;
}
You need to add two debug checks on "pte_page(*pvmw->pte) % 64" and
same for pvmw->page to find out the one of the two that isn't a page.
If both are real pages there's a bug that allocates page structs not
naturally aligned.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 14:58 ` Andrea Arcangeli
@ 2018-01-18 16:56 ` Kirill A. Shutemov
2018-01-18 17:26 ` Luck, Tony
2018-01-18 17:26 ` Linus Torvalds
0 siblings, 2 replies; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-18 16:56 UTC (permalink / raw)
To: Andrea Arcangeli
Cc: Dave Hansen, Tetsuo Handa, torvalds, kirill.shutemov, akpm,
hannes, iamjoonsoo.kim, mgorman, tony.luck, vbabka, mhocko,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 03:58:30PM +0100, Andrea Arcangeli wrote:
> On Thu, Jan 18, 2018 at 06:45:00AM -0800, Dave Hansen wrote:
> > On 01/18/2018 04:25 AM, Kirill A. Shutemov wrote:
> > > [ 10.084024] diff: -858690919
> > > [ 10.084258] hpage_nr_pages: 1
> > > [ 10.084386] check1: 0
> > > [ 10.084478] check2: 0
> > ...
> > > diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
> > > index d22b84310f6d..57b4397f1ea5 100644
> > > --- a/mm/page_vma_mapped.c
> > > +++ b/mm/page_vma_mapped.c
> > > @@ -70,6 +70,14 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
> > > }
> > > if (pte_page(*pvmw->pte) < pvmw->page)
> > > return false;
> > > +
> > > + if (pte_page(*pvmw->pte) - pvmw->page) {
> > > + printk("diff: %d\n", pte_page(*pvmw->pte) - pvmw->page);
> > > + printk("hpage_nr_pages: %d\n", hpage_nr_pages(pvmw->page));
> > > + printk("check1: %d\n", pte_page(*pvmw->pte) - pvmw->page < 0);
> > > + printk("check2: %d\n", pte_page(*pvmw->pte) - pvmw->page >= hpage_nr_pages(pvmw->page));
> > > + BUG();
> > > + }
> >
> > This says that pte_page(*pvmw->pte) and pvmw->page are roughly 4GB away
> > from each other (858690919*4=0xccba559c0). That's not the compiler
> > being wonky, it just means that the virtual addresses of the memory
> > sections are that far apart.
> >
> > This won't happen when you have vmemmap or flatmem because the mem_map[]
> > is virtually contiguous and pointer arithmetic just works against all
> > 'struct page' pointers. But with classic sparsemem, it doesn't.
> >
> > You need to make sure that the PFNs are in the same section before you
> > can do the math that you want to do here.
>
> Isn't it simply that pvmw->page isn't a page or pte_page(*pvmw->pte)
> isn't a page?
>
> The distance cannot matter, MMU isn't involved, this is pure 64bit
> aritmetics, 1giga 1 terabyte, 48bits 5level pagetables are meaningless
> in this comparison.
Note, it's 32-bit.
>
> #include <stdio.h>
>
> int main()
> {
> volatile long i;
> struct x { char a[4000000000]; };
> for (i = 0; i < 4000000000*3; i += 4000000000) {
> printf("%ld\n", ((struct x *)0)-((((struct x *)i))));
> }
> printf("xxxx\n");
> for (i = 0; i < 4000000000; i += 1) {
> if (i==4)
> i = 4000000000;
> printf("%ld\n", ((struct x *)0)-((((struct x *)i))));
> }
> return 0;
> }
>
> You need to add two debug checks on "pte_page(*pvmw->pte) % 64" and
> same for pvmw->page to find out the one of the two that isn't a page.
>
> If both are real pages there's a bug that allocates page structs not
> naturally aligned.
Both are real page. But why do you expect pages to be 64-byte alinged?
Both are aligned to 64-bit as they suppose to be IIUC.
I can't say I fully grasp how 'diff' got this value and how it leads to both
checks being false.
[ 10.209657] page:f6e4cc38 count:8 mapcount:6 mapping:f5818f94 index:0x0
[ 10.209989] flags: 0x3501004d(locked|referenced|uptodate|active|mappedtodisk)
[ 10.210496] raw: 3501004d f5818f94 00000000 00000005 00000008 00000100 00000200 00000000
[ 10.210742] raw: f5c06600 00000000
[ 10.210863] page dumped because: pvmw->page
[ 10.210992] page->mem_cgroup:f5c06600
[ 10.211192] page:f74749d8 count:1 mapcount:1 mapping:f54612d1 index:0x0
[ 10.211381] flags: 0x15040068(uptodate|lru|active|swapbacked)
[ 10.211530] raw: 15040068 f54612d1 00000000 00000000 00000001 f74749c4 f75b9014 00000000
[ 10.211729] raw: f5c06600 00000000
[ 10.211806] page dumped because: pte_page(*pvmw->pte)
[ 10.211920] page->mem_cgroup:f5c06600
[ 10.212079] diff: -858832092
[ 10.212184] hpage_nr_pages: 1
[ 10.212284] check1: 0
[ 10.212384] check2: 0
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* RE: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 16:56 ` Kirill A. Shutemov
@ 2018-01-18 17:26 ` Luck, Tony
2018-01-18 17:28 ` Linus Torvalds
2018-01-18 17:26 ` Linus Torvalds
1 sibling, 1 reply; 59+ messages in thread
From: Luck, Tony @ 2018-01-18 17:26 UTC (permalink / raw)
To: Kirill A. Shutemov, Andrea Arcangeli
Cc: Dave Hansen, Tetsuo Handa, torvalds, kirill.shutemov, akpm,
hannes, iamjoonsoo.kim, mgorman, vbabka, mhocko, hillf.zj, hughd,
oleg, peterz, riel, srikar, vdavydov.dev, mingo, linux-kernel,
linux-mm, x86
> Both are real page. But why do you expect pages to be 64-byte alinged?
> Both are aligned to 64-bit as they suppose to be IIUC.
On a 64-bit kernel sizeof struct page == 64 (after much work by people to
trim out excess stuff). So I thought we made sure to align the base address
of blocks of "struct page" so that every one neatly fits into one cache line.
-Tony
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 17:26 ` Luck, Tony
@ 2018-01-18 17:28 ` Linus Torvalds
0 siblings, 0 replies; 59+ messages in thread
From: Linus Torvalds @ 2018-01-18 17:28 UTC (permalink / raw)
To: Luck, Tony
Cc: Kirill A. Shutemov, Andrea Arcangeli, Dave Hansen, Tetsuo Handa,
kirill.shutemov, akpm, hannes, iamjoonsoo.kim, mgorman, vbabka,
mhocko, hillf.zj, hughd, oleg, peterz, riel, srikar,
vdavydov.dev, mingo, linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 9:26 AM, Luck, Tony <tony.luck@intel.com> wrote:
>> Both are real page. But why do you expect pages to be 64-byte alinged?
>> Both are aligned to 64-bit as they suppose to be IIUC.
>
> On a 64-bit kernel sizeof struct page == 64 (after much work by people to
> trim out excess stuff). So I thought we made sure to align the base address
> of blocks of "struct page" so that every one neatly fits into one cache line.
The bug happens on 32-bit, and a 'struct page' is not 64-byte aligned
there at all.
See my other email about the explanation of why "page1 - page2"
doesn't work when they aren't mutually aligned to the actual size of
the 'struct page'.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 16:56 ` Kirill A. Shutemov
2018-01-18 17:26 ` Luck, Tony
@ 2018-01-18 17:26 ` Linus Torvalds
2018-01-18 23:49 ` Kirill A. Shutemov
1 sibling, 1 reply; 59+ messages in thread
From: Linus Torvalds @ 2018-01-18 17:26 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Andrea Arcangeli, Dave Hansen, Tetsuo Handa, Kirill A. Shutemov,
Andrew Morton, Johannes Weiner, Joonsoo Kim, Mel Gorman,
Tony Luck, Vlastimil Babka, Michal Hocko, hillf.zj, Hugh Dickins,
Oleg Nesterov, Peter Zijlstra, Rik van Riel, Srikar Dronamraju,
Vladimir Davydov, Ingo Molnar, Linux Kernel Mailing List,
linux-mm, the arch/x86 maintainers
On Thu, Jan 18, 2018 at 8:56 AM, Kirill A. Shutemov
<kirill@shutemov.name> wrote:
>
> I can't say I fully grasp how 'diff' got this value and how it leads to both
> checks being false.
I think the problem is that page difference when they are in different sections.
When you do
pte_page(*pvmw->pte) - pvmw->page
then the compiler takes the pointer difference, and then divides by
the size of "struct page" to get an index.
But - and this is important - it does so knowing that the division it
does will have no modulus: the two 'struct page *' pointers are really
in the same array, and they really are 'n*sizeof(struct page)' apart
for some 'n'.
That means that the compiler can optimize the division. In fact, for
this case, gcc will generate
subl %ebx, %eax
sarl $3, %eax
imull $-858993459, %eax, %eax
because 'struct page' is 40 bytes in size, and that magic sequence
happens to divide by 40 (first divide by 8, then that magical "imull"
will divide by 5 *IFF* the thing is evenly divisible by 5 (and not too
big - but the shift guarantees that).
Basically, it's a magic trick, because real divides are very
expensive, but you can fake them more quickly if you can limit the
input domain.
But what does it mean if the two "struct page *" are not in the same
array, and the two arrays were allocated not aligned exactly 40 bytes
away, but some random number of pages away?
You get *COMPLETE*GARBAGE* when you do the above optimized divide.
Suddenly the divide had a modulus (because the base of the two arrays
weren't 40-byte aligned), and the "trick" doesn't work.
So that's why you can't do pointer diffs between two arrays. Not
because you can't subtract the two pointers, but because the
*division* part of the C pointer diff rules leads to issues.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 17:26 ` Linus Torvalds
@ 2018-01-18 23:49 ` Kirill A. Shutemov
2018-01-19 12:55 ` Matthew Wilcox
0 siblings, 1 reply; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-18 23:49 UTC (permalink / raw)
To: Linus Torvalds, Peter Zijlstra
Cc: Andrea Arcangeli, Dave Hansen, Tetsuo Handa, Kirill A. Shutemov,
Andrew Morton, Johannes Weiner, Joonsoo Kim, Mel Gorman,
Tony Luck, Vlastimil Babka, Michal Hocko, hillf.zj, Hugh Dickins,
Oleg Nesterov, Rik van Riel, Srikar Dronamraju, Vladimir Davydov,
Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Thu, Jan 18, 2018 at 09:26:25AM -0800, Linus Torvalds wrote:
> On Thu, Jan 18, 2018 at 8:56 AM, Kirill A. Shutemov
> <kirill@shutemov.name> wrote:
> >
> > I can't say I fully grasp how 'diff' got this value and how it leads to both
> > checks being false.
>
> I think the problem is that page difference when they are in different sections.
>
> When you do
>
> pte_page(*pvmw->pte) - pvmw->page
>
> then the compiler takes the pointer difference, and then divides by
> the size of "struct page" to get an index.
>
> But - and this is important - it does so knowing that the division it
> does will have no modulus: the two 'struct page *' pointers are really
> in the same array, and they really are 'n*sizeof(struct page)' apart
> for some 'n'.
>
> That means that the compiler can optimize the division. In fact, for
> this case, gcc will generate
>
> subl %ebx, %eax
> sarl $3, %eax
> imull $-858993459, %eax, %eax
>
> because 'struct page' is 40 bytes in size, and that magic sequence
> happens to divide by 40 (first divide by 8, then that magical "imull"
> will divide by 5 *IFF* the thing is evenly divisible by 5 (and not too
> big - but the shift guarantees that).
>
> Basically, it's a magic trick, because real divides are very
> expensive, but you can fake them more quickly if you can limit the
> input domain.
>
> But what does it mean if the two "struct page *" are not in the same
> array, and the two arrays were allocated not aligned exactly 40 bytes
> away, but some random number of pages away?
>
> You get *COMPLETE*GARBAGE* when you do the above optimized divide.
> Suddenly the divide had a modulus (because the base of the two arrays
> weren't 40-byte aligned), and the "trick" doesn't work.
>
> So that's why you can't do pointer diffs between two arrays. Not
> because you can't subtract the two pointers, but because the
> *division* part of the C pointer diff rules leads to issues.
Thanks a lot for the explanation!
I wounder if this may be a problem in other places?
For instance, perf uses address of a mutex to determinate the lock
ordering. See mutex_lock_double(). The mutex is embedded into struct
perf_event_context, which is allocated with kzalloc() so I don't see how
we can presume that alignment is consistent between them.
I don't think it's the only example in kernel. Are we just lucky?
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 23:49 ` Kirill A. Shutemov
@ 2018-01-19 12:55 ` Matthew Wilcox
2018-01-19 18:42 ` Linus Torvalds
0 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox @ 2018-01-19 12:55 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Linus Torvalds, Peter Zijlstra, Andrea Arcangeli, Dave Hansen,
Tetsuo Handa, Kirill A. Shutemov, Andrew Morton, Johannes Weiner,
Joonsoo Kim, Mel Gorman, Tony Luck, Vlastimil Babka,
Michal Hocko, hillf.zj, Hugh Dickins, Oleg Nesterov,
Rik van Riel, Srikar Dronamraju, Vladimir Davydov, Ingo Molnar,
Linux Kernel Mailing List, linux-mm, the arch/x86 maintainers
On Fri, Jan 19, 2018 at 02:49:55AM +0300, Kirill A. Shutemov wrote:
> > So that's why you can't do pointer diffs between two arrays. Not
> > because you can't subtract the two pointers, but because the
> > *division* part of the C pointer diff rules leads to issues.
>
> Thanks a lot for the explanation!
>
> I wounder if this may be a problem in other places?
>
> For instance, perf uses address of a mutex to determinate the lock
> ordering. See mutex_lock_double(). The mutex is embedded into struct
> perf_event_context, which is allocated with kzalloc() so I don't see how
> we can presume that alignment is consistent between them.
>
> I don't think it's the only example in kernel. Are we just lucky?
If you're just *comparing* the addresses of two objects, GCC doesn't
care what the size of the object is. ie there's a difference between
'if (b < a)' and 'if ((a - b) < n)'.
But yes, if you go by the strict wording of the standard:
When two pointers are compared, the result depends on the relative
locations in the address space of the objects pointed to. [...] In
all other cases, the behavior is undefined
http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf
So really we should be casting 'b' and 'a' to uintptr_t to be fully
compliant with the spec.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-19 12:55 ` Matthew Wilcox
@ 2018-01-19 18:42 ` Linus Torvalds
2018-01-19 22:12 ` Al Viro
0 siblings, 1 reply; 59+ messages in thread
From: Linus Torvalds @ 2018-01-19 18:42 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Kirill A. Shutemov, Peter Zijlstra, Andrea Arcangeli,
Dave Hansen, Tetsuo Handa, Kirill A. Shutemov, Andrew Morton,
Johannes Weiner, Joonsoo Kim, Mel Gorman, Tony Luck,
Vlastimil Babka, Michal Hocko, hillf.zj, Hugh Dickins,
Oleg Nesterov, Rik van Riel, Srikar Dronamraju, Vladimir Davydov,
Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Fri, Jan 19, 2018 at 4:55 AM, Matthew Wilcox <willy@infradead.org> wrote:
>
> So really we should be casting 'b' and 'a' to uintptr_t to be fully
> compliant with the spec.
That's an unnecessary technicality.
Any compiler that doesn't get pointer inequality testing right is not
worth even worrying about. We wouldn't want to use such a compiler,
because it's intentionally generating garbage just to f*ck with us.
Why would you go along with that?
So the only real issue is that pointer subtraction case.
I actually asked (long long ago) for an optinal compiler warning for
"pointer subtraction with non-power-of-2 sizes". Not because of it
being undefined, but simply because it's expensive. The
divide->multiply thing doesn't always work, and a real divide is
really quite expensive on many architectures.
We *should* be careful about it. I guess sparse could be made to warn,
but I'm afraid that we have so many of these things that a warning
isn't reasonable.
And most of the time, a pointer difference really is inside the same
array. The operation doesn't tend to make sense otherwise.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-19 18:42 ` Linus Torvalds
@ 2018-01-19 22:12 ` Al Viro
2018-01-19 22:53 ` Linus Torvalds
0 siblings, 1 reply; 59+ messages in thread
From: Al Viro @ 2018-01-19 22:12 UTC (permalink / raw)
To: Linus Torvalds
Cc: Matthew Wilcox, Kirill A. Shutemov, Peter Zijlstra,
Andrea Arcangeli, Dave Hansen, Tetsuo Handa, Kirill A. Shutemov,
Andrew Morton, Johannes Weiner, Joonsoo Kim, Mel Gorman,
Tony Luck, Vlastimil Babka, Michal Hocko, hillf.zj, Hugh Dickins,
Oleg Nesterov, Rik van Riel, Srikar Dronamraju, Vladimir Davydov,
Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Fri, Jan 19, 2018 at 10:42:18AM -0800, Linus Torvalds wrote:
> On Fri, Jan 19, 2018 at 4:55 AM, Matthew Wilcox <willy@infradead.org> wrote:
> >
> > So really we should be casting 'b' and 'a' to uintptr_t to be fully
> > compliant with the spec.
>
> That's an unnecessary technicality.
>
> Any compiler that doesn't get pointer inequality testing right is not
> worth even worrying about. We wouldn't want to use such a compiler,
> because it's intentionally generating garbage just to f*ck with us.
> Why would you go along with that?
>
> So the only real issue is that pointer subtraction case.
>
> I actually asked (long long ago) for an optinal compiler warning for
> "pointer subtraction with non-power-of-2 sizes". Not because of it
> being undefined, but simply because it's expensive. The
> divide->multiply thing doesn't always work, and a real divide is
> really quite expensive on many architectures.
>
> We *should* be careful about it. I guess sparse could be made to warn,
> but I'm afraid that we have so many of these things that a warning
> isn't reasonable.
You mean like -Wptr-subtraction-blows?
FWIW, allmodconfig on amd64 with C=2 CF=-Wptr-subtraction-blows is not too large
The tail (alphabetically sorted) is
lib/dynamic_debug.c:1013:9: warning: potentially expensive pointer subtraction
lib/extable.c:70:28: warning: potentially expensive pointer subtraction
mm/memory_hotplug.c:1530:13: warning: potentially expensive pointer subtraction
mm/memory_hotplug.c:734:13: warning: potentially expensive pointer subtraction
mm/memory_hotplug.c:831:41: warning: potentially expensive pointer subtraction
mm/page_owner.c:607:38: warning: potentially expensive pointer subtraction
mm/vmstat.c:1334:38: warning: potentially expensive pointer subtraction
net/core/net-sysfs.c:1040:19: warning: potentially expensive pointer subtraction
net/ipv4/ipmr.c:3026:32: warning: potentially expensive pointer subtraction
net/ipv6/ip6mr.c:458:32: warning: potentially expensive pointer subtraction
net/mac80211/tx.c:1307:41: warning: potentially expensive pointer subtraction
net/mac80211/tx.c:1351:44: warning: potentially expensive pointer subtraction
net/rds/ib_recv.c:345:49: warning: potentially expensive pointer subtraction
net/rds/ib_recv.c:861:38: warning: potentially expensive pointer subtraction
net/sched/cls_tcindex.c:622:43: warning: potentially expensive pointer subtraction
net/sched/sch_cbs.c:302:35: warning: potentially expensive pointer subtraction
net/sched/sch_hhf.c:367:23: warning: potentially expensive pointer subtraction
net/sched/sch_hhf.c:434:38: warning: potentially expensive pointer subtraction
net/sunrpc/svc_xprt.c:1377:43: warning: potentially expensive pointer subtraction
sound/pci/hda/hda_generic.c:301:20: warning: potentially expensive pointer subtraction
IOW it's not terribly noisy. Might be an interesting idea to teach sparse to
print the type in question... Aha - with
--- a/evaluate.c
+++ b/evaluate.c
@@ -848,7 +848,8 @@ static struct symbol *evaluate_ptr_sub(struct expression *expr)
if (value & (value-1)) {
if (Wptr_subtraction_blows)
- warning(expr->pos, "potentially expensive pointer subtraction");
+ warning(expr->pos, "[%s] potentially expensive pointer subtraction",
+ show_typename(lbase));
}
sub->op = '-';
we get things like
drivers/gpu/drm/i915/i915_gem_execbuffer.c:435:17: warning: [struct drm_i915_gem_exec_object2] potentially expensive pointer subtraction
OK, the top sources of that warning are:
91 struct cpufreq_frequency_table
36 struct Indirect (actually, that conflates ext2/ext4/minix/sysv)
21 struct ips_scb
18 struct runlist_element
13 struct zone
13 struct vring
11 struct usbhsh_device
10 struct xpc_partition
9 struct skge_element
9 struct lock_class
9 struct hstate
7 struct nvme_rdma_queue
7 struct iso_context
6 struct i915_power_well
6 struct hpet_dev
6 struct ext4_extent
6 struct esas2r_target
5 struct iio_chan_spec
5 struct hwspinlock
4 struct myri10ge_slice_state
4 struct ext4_extent_idx
everything beyond that is 3 instances or less...
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-19 22:12 ` Al Viro
@ 2018-01-19 22:53 ` Linus Torvalds
2018-01-20 2:02 ` Al Viro
0 siblings, 1 reply; 59+ messages in thread
From: Linus Torvalds @ 2018-01-19 22:53 UTC (permalink / raw)
To: Al Viro
Cc: Matthew Wilcox, Kirill A. Shutemov, Peter Zijlstra,
Andrea Arcangeli, Dave Hansen, Tetsuo Handa, Kirill A. Shutemov,
Andrew Morton, Johannes Weiner, Joonsoo Kim, Mel Gorman,
Tony Luck, Vlastimil Babka, Michal Hocko, hillf.zj, Hugh Dickins,
Oleg Nesterov, Rik van Riel, Srikar Dronamraju, Vladimir Davydov,
Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Fri, Jan 19, 2018 at 2:12 PM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Fri, Jan 19, 2018 at 10:42:18AM -0800, Linus Torvalds wrote:
>>
>> We *should* be careful about it. I guess sparse could be made to warn,
>> but I'm afraid that we have so many of these things that a warning
>> isn't reasonable.
>
> You mean like -Wptr-subtraction-blows?
Heh. Apparently I already did that trivial warning back in 2005. I'd
forgotten about it.
> FWIW, allmodconfig on amd64 with C=2 CF=-Wptr-subtraction-blows is not too large
>
> IOW it's not terribly noisy. Might be an interesting idea to teach sparse to
> print the type in question... Aha - with
>
> --- a/evaluate.c
> +++ b/evaluate.c
> @@ -848,7 +848,8 @@ static struct symbol *evaluate_ptr_sub(struct expression *expr)
>
> if (value & (value-1)) {
> if (Wptr_subtraction_blows)
> - warning(expr->pos, "potentially expensive pointer subtraction");
> + warning(expr->pos, "[%s] potentially expensive pointer subtraction",
> + show_typename(lbase));
> }
>
> sub->op = '-';
>
> we get things like
> drivers/gpu/drm/i915/i915_gem_execbuffer.c:435:17: warning: [struct drm_i915_gem_exec_object2] potentially expensive pointer subtraction
It would probably be good to add the size too, just to explain why
it's potentially expensive.
That said, apparently we do have hundreds of them, with just
cpufreq_frequency_table having a ton. Maybe some are hidden in macros
and removing one removes a lot.
The real problem is that sometimes the subtraction is simply the right
thing to do, and there's no sane way to say "yeah, this is one of
those cases you shouldn't warn about".
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-19 22:53 ` Linus Torvalds
@ 2018-01-20 2:02 ` Al Viro
2018-01-20 5:24 ` Al Viro
0 siblings, 1 reply; 59+ messages in thread
From: Al Viro @ 2018-01-20 2:02 UTC (permalink / raw)
To: Linus Torvalds
Cc: Matthew Wilcox, Kirill A. Shutemov, Peter Zijlstra,
Andrea Arcangeli, Dave Hansen, Tetsuo Handa, Kirill A. Shutemov,
Andrew Morton, Johannes Weiner, Joonsoo Kim, Mel Gorman,
Tony Luck, Vlastimil Babka, Michal Hocko, hillf.zj, Hugh Dickins,
Oleg Nesterov, Rik van Riel, Srikar Dronamraju, Vladimir Davydov,
Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Fri, Jan 19, 2018 at 02:53:25PM -0800, Linus Torvalds wrote:
> It would probably be good to add the size too, just to explain why
> it's potentially expensive.
>
> That said, apparently we do have hundreds of them, with just
> cpufreq_frequency_table having a ton. Maybe some are hidden in macros
> and removing one removes a lot.
cpufreq_table_find_index_...(), mostly.
> The real problem is that sometimes the subtraction is simply the right
> thing to do, and there's no sane way to say "yeah, this is one of
> those cases you shouldn't warn about".
FWIW, the sizes of the most common ones are
91 sizeof struct cpufreq_frequency_table = 12
Almost all of those come from
cpufreq_for_each_valid_entry(pos, table)
if (....)
return pos - table;
and I wonder if we would be better off with something like
#define cpufreq_for_each_valid_entry(pos, table, idx) \
for (pos = table, idx = 0; pos->frequency != CPUFREQ_TABLE_END; pos++, idx++) \
if (pos->frequency == CPUFREQ_ENTRY_INVALID) \
continue; \
else
so that those loops would become
cpufreq_for_each_valid_entry(pos, table, idx)
if (....)
return idx;
36 sizeof struct Indirect = 24
21 sizeof struct ips_scb = 216
18 sizeof struct runlist_element = 24
13 sizeof struct zone = 1728
Some are from
#define zone_idx(zone) ((zone) - (zone)->zone_pgdat->node_zones)
but there's
static inline int zone_id(const struct zone *zone)
{
struct pglist_data *pgdat = zone->zone_pgdat;
return zone - pgdat->node_zones;
}
and a couple of places where we have
for (zone = node_zones; zone - node_zones < MAX_NR_ZONES; ++zone) {
Those bloody well ought to be
for (zone = node_zones, end = node_zones + MAX_NR_ZONES; zone < end; zone++) {
13 sizeof struct vring = 40
11 sizeof struct usbhsh_device = 24
10 sizeof struct xpc_partition = 888
9 sizeof struct skge_element = 40
9 sizeof struct lock_class = 400
9 sizeof struct hstate = 29872
That little horror comes from get_hstate_idx() and hstate_index(). Code generated
for division is
movabsq $-5542915600080909725, %rax
sarq $4, %rdi
imulq %rdi, %rax
7 sizeof struct nvme_rdma_queue = 312
7 sizeof struct iso_context = 208
6 sizeof struct i915_power_well = 48
6 sizeof struct hpet_dev = 168
6 sizeof struct ext4_extent = 12
6 sizeof struct esas2r_target = 120
5 sizeof struct iio_chan_spec = 152
5 sizeof struct hwspinlock = 96
4 sizeof struct myri10ge_slice_state = 704
4 sizeof struct ext4_extent_idx = 12
Another interesting-looking one is struct vhost_net_virtqueue (18080 bytes)
Note that those sizes are rather sensitive to lockdep, spinlock debugging, etc.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-20 2:02 ` Al Viro
@ 2018-01-20 5:24 ` Al Viro
2018-01-20 9:38 ` Luc Van Oostenryck
0 siblings, 1 reply; 59+ messages in thread
From: Al Viro @ 2018-01-20 5:24 UTC (permalink / raw)
To: Linus Torvalds
Cc: Matthew Wilcox, Kirill A. Shutemov, Peter Zijlstra,
Andrea Arcangeli, Dave Hansen, Tetsuo Handa, Kirill A. Shutemov,
Andrew Morton, Johannes Weiner, Joonsoo Kim, Mel Gorman,
Tony Luck, Vlastimil Babka, Michal Hocko, hillf.zj, Hugh Dickins,
Oleg Nesterov, Rik van Riel, Srikar Dronamraju, Vladimir Davydov,
Ingo Molnar, Linux Kernel Mailing List, linux-mm,
the arch/x86 maintainers
On Sat, Jan 20, 2018 at 02:02:37AM +0000, Al Viro wrote:
> Note that those sizes are rather sensitive to lockdep, spinlock debugging, etc.
That they certainly are: on one of the testing .config I'm using it gave this:
1104 sizeof struct page = 56
81 sizeof struct cpufreq_frequency_table = 12
32 sizeof struct Indirect = 24
7 sizeof struct zone = 1400
7 sizeof struct hstate = 152
6 sizeof struct lock_class = 336
6 sizeof struct hpet_dev = 152
6 sizeof struct ext4_extent = 12
4 sizeof struct ext4_extent_idx = 12
3 sizeof struct mbox_chan = 456
2 sizeof struct strip_zone = 24
2 sizeof struct kobj_attribute = 48
2 sizeof struct kernel_param = 40
2 sizeof struct exception_table_entry = 12
1 sizeof struct vif_device = 104
1 sizeof struct unixware_slice = 12
1 sizeof struct svc_pool = 152
1 sizeof struct srcu_node = 152
1 sizeof struct r5worker_group = 56
1 sizeof struct pebs_record_core = 144
1 sizeof struct netdev_queue = 384
1 sizeof struct mirror = 40
1 sizeof struct mif_device = 56
1 sizeof struct e1000_tx_ring = 48
1 sizeof struct dx_frame = 24
1 sizeof struct bts_record = 24
1 sizeof struct ata_device = 2560
1 sizeof struct acpi_processor_cx = 52
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-20 5:24 ` Al Viro
@ 2018-01-20 9:38 ` Luc Van Oostenryck
0 siblings, 0 replies; 59+ messages in thread
From: Luc Van Oostenryck @ 2018-01-20 9:38 UTC (permalink / raw)
To: Al Viro
Cc: Linus Torvalds, Matthew Wilcox, Kirill A. Shutemov,
Peter Zijlstra, Andrea Arcangeli, Dave Hansen, Tetsuo Handa,
Kirill A. Shutemov, Andrew Morton, Johannes Weiner, Joonsoo Kim,
Mel Gorman, Tony Luck, Vlastimil Babka, Michal Hocko, hillf.zj,
Hugh Dickins, Oleg Nesterov, Rik van Riel, Srikar Dronamraju,
Vladimir Davydov, Ingo Molnar, Linux Kernel Mailing List,
linux-mm, the arch/x86 maintainers, sparse mailing list
On Sat, Jan 20, 2018 at 05:24:32AM +0000, Al Viro wrote:
> On Sat, Jan 20, 2018 at 02:02:37AM +0000, Al Viro wrote:
>
> > Note that those sizes are rather sensitive to lockdep, spinlock debugging, etc.
>
> That they certainly are: on one of the testing .config I'm using it gave this:
> 1104 sizeof struct page = 56
Yes, I get this already with defconfig.
It's a problem with sparse which ignore the alignment attribute
(in fact all 'trailing' attributes in type declarations).
I'm looking to fix it.
-- Luc Van Oostenryck
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 14:45 ` Dave Hansen
2018-01-18 14:58 ` Andrea Arcangeli
@ 2018-01-18 15:40 ` Kirill A. Shutemov
2018-01-18 17:22 ` Michal Hocko
2018-01-19 2:01 ` Tetsuo Handa
1 sibling, 2 replies; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-18 15:40 UTC (permalink / raw)
To: Dave Hansen
Cc: Tetsuo Handa, torvalds, kirill.shutemov, akpm, hannes,
iamjoonsoo.kim, mgorman, tony.luck, vbabka, mhocko, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 06:45:00AM -0800, Dave Hansen wrote:
> On 01/18/2018 04:25 AM, Kirill A. Shutemov wrote:
> > [ 10.084024] diff: -858690919
> > [ 10.084258] hpage_nr_pages: 1
> > [ 10.084386] check1: 0
> > [ 10.084478] check2: 0
> ...
> > diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
> > index d22b84310f6d..57b4397f1ea5 100644
> > --- a/mm/page_vma_mapped.c
> > +++ b/mm/page_vma_mapped.c
> > @@ -70,6 +70,14 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
> > }
> > if (pte_page(*pvmw->pte) < pvmw->page)
> > return false;
> > +
> > + if (pte_page(*pvmw->pte) - pvmw->page) {
> > + printk("diff: %d\n", pte_page(*pvmw->pte) - pvmw->page);
> > + printk("hpage_nr_pages: %d\n", hpage_nr_pages(pvmw->page));
> > + printk("check1: %d\n", pte_page(*pvmw->pte) - pvmw->page < 0);
> > + printk("check2: %d\n", pte_page(*pvmw->pte) - pvmw->page >= hpage_nr_pages(pvmw->page));
> > + BUG();
> > + }
>
> This says that pte_page(*pvmw->pte) and pvmw->page are roughly 4GB away
> from each other (858690919*4=0xccba559c0). That's not the compiler
> being wonky, it just means that the virtual addresses of the memory
> sections are that far apart.
>
> This won't happen when you have vmemmap or flatmem because the mem_map[]
> is virtually contiguous and pointer arithmetic just works against all
> 'struct page' pointers. But with classic sparsemem, it doesn't.
>
> You need to make sure that the PFNs are in the same section before you
> can do the math that you want to do here.
Something like this?
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 15:40 ` Kirill A. Shutemov
@ 2018-01-18 17:22 ` Michal Hocko
2018-01-19 10:02 ` Kirill A. Shutemov
2018-01-19 2:01 ` Tetsuo Handa
1 sibling, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2018-01-18 17:22 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Dave Hansen, Tetsuo Handa, torvalds, kirill.shutemov, akpm,
hannes, iamjoonsoo.kim, mgorman, tony.luck, vbabka, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Thu 18-01-18 18:40:26, Kirill A. Shutemov wrote:
[...]
> + /*
> + * Make sure that pages are in the same section before doing pointer
> + * arithmetics.
> + */
> + if (page_to_section(pvmw->page) != page_to_section(page))
> + return false;
OK, THPs shouldn't cross memory sections AFAIK. My brain is meltdown
these days so this might be a completely stupid question. But why don't
you simply compare pfns? This would be just simpler, no?
> +
> + if (page < pvmw->page)
> + return false;
> +
> + /* THP can be referenced by any subpage */
> + if (page - pvmw->page >= hpage_nr_pages(pvmw->page))
> + return false;
> +
> return true;
> }
>
> --
> Kirill A. Shutemov
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 17:22 ` Michal Hocko
@ 2018-01-19 10:02 ` Kirill A. Shutemov
2018-01-19 10:33 ` Michal Hocko
0 siblings, 1 reply; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-19 10:02 UTC (permalink / raw)
To: Michal Hocko
Cc: Dave Hansen, Tetsuo Handa, torvalds, kirill.shutemov, akpm,
hannes, iamjoonsoo.kim, mgorman, tony.luck, vbabka, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Thu, Jan 18, 2018 at 06:22:13PM +0100, Michal Hocko wrote:
> On Thu 18-01-18 18:40:26, Kirill A. Shutemov wrote:
> [...]
> > + /*
> > + * Make sure that pages are in the same section before doing pointer
> > + * arithmetics.
> > + */
> > + if (page_to_section(pvmw->page) != page_to_section(page))
> > + return false;
>
> OK, THPs shouldn't cross memory sections AFAIK. My brain is meltdown
> these days so this might be a completely stupid question. But why don't
> you simply compare pfns? This would be just simpler, no?
In original code, we already had pvmw->page around and I thought it would
be easier to get page for the pte intead of looking for pfn for both
sides.
We these changes it's no longer the case.
Do you care enough to send a patch? :)
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-19 10:02 ` Kirill A. Shutemov
@ 2018-01-19 10:33 ` Michal Hocko
2018-01-19 11:49 ` Kirill A. Shutemov
0 siblings, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2018-01-19 10:33 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Dave Hansen, Tetsuo Handa, torvalds, kirill.shutemov, akpm,
hannes, iamjoonsoo.kim, mgorman, tony.luck, vbabka, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Fri 19-01-18 13:02:59, Kirill A. Shutemov wrote:
> On Thu, Jan 18, 2018 at 06:22:13PM +0100, Michal Hocko wrote:
> > On Thu 18-01-18 18:40:26, Kirill A. Shutemov wrote:
> > [...]
> > > + /*
> > > + * Make sure that pages are in the same section before doing pointer
> > > + * arithmetics.
> > > + */
> > > + if (page_to_section(pvmw->page) != page_to_section(page))
> > > + return false;
> >
> > OK, THPs shouldn't cross memory sections AFAIK. My brain is meltdown
> > these days so this might be a completely stupid question. But why don't
> > you simply compare pfns? This would be just simpler, no?
>
> In original code, we already had pvmw->page around and I thought it would
> be easier to get page for the pte intead of looking for pfn for both
> sides.
>
> We these changes it's no longer the case.
>
> Do you care enough to send a patch? :)
Well, memory sections are sparsemem concept IIRC. Unless I've missed
something page_to_section is quarded by SECTION_IN_PAGE_FLAGS and that
is conditional to CONFIG_SPARSEMEM. THP is a generic code so using it
there is wrong unless I miss some subtle detail here.
Comparing pfn should be generic enough.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-19 10:33 ` Michal Hocko
@ 2018-01-19 11:49 ` Kirill A. Shutemov
2018-01-19 12:07 ` Michal Hocko
0 siblings, 1 reply; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-19 11:49 UTC (permalink / raw)
To: Michal Hocko
Cc: Dave Hansen, Tetsuo Handa, torvalds, kirill.shutemov, akpm,
hannes, iamjoonsoo.kim, mgorman, tony.luck, vbabka, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Fri, Jan 19, 2018 at 11:33:42AM +0100, Michal Hocko wrote:
> On Fri 19-01-18 13:02:59, Kirill A. Shutemov wrote:
> > On Thu, Jan 18, 2018 at 06:22:13PM +0100, Michal Hocko wrote:
> > > On Thu 18-01-18 18:40:26, Kirill A. Shutemov wrote:
> > > [...]
> > > > + /*
> > > > + * Make sure that pages are in the same section before doing pointer
> > > > + * arithmetics.
> > > > + */
> > > > + if (page_to_section(pvmw->page) != page_to_section(page))
> > > > + return false;
> > >
> > > OK, THPs shouldn't cross memory sections AFAIK. My brain is meltdown
> > > these days so this might be a completely stupid question. But why don't
> > > you simply compare pfns? This would be just simpler, no?
> >
> > In original code, we already had pvmw->page around and I thought it would
> > be easier to get page for the pte intead of looking for pfn for both
> > sides.
> >
> > We these changes it's no longer the case.
> >
> > Do you care enough to send a patch? :)
>
> Well, memory sections are sparsemem concept IIRC. Unless I've missed
> something page_to_section is quarded by SECTION_IN_PAGE_FLAGS and that
> is conditional to CONFIG_SPARSEMEM. THP is a generic code so using it
> there is wrong unless I miss some subtle detail here.
>
> Comparing pfn should be generic enough.
Good point.
What about something like this?
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-19 11:49 ` Kirill A. Shutemov
@ 2018-01-19 12:07 ` Michal Hocko
2018-01-19 12:30 ` Kirill A. Shutemov
0 siblings, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2018-01-19 12:07 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Dave Hansen, Tetsuo Handa, torvalds, kirill.shutemov, akpm,
hannes, iamjoonsoo.kim, mgorman, tony.luck, vbabka, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Fri 19-01-18 14:49:17, Kirill A. Shutemov wrote:
> On Fri, Jan 19, 2018 at 11:33:42AM +0100, Michal Hocko wrote:
> > On Fri 19-01-18 13:02:59, Kirill A. Shutemov wrote:
> > > On Thu, Jan 18, 2018 at 06:22:13PM +0100, Michal Hocko wrote:
> > > > On Thu 18-01-18 18:40:26, Kirill A. Shutemov wrote:
> > > > [...]
> > > > > + /*
> > > > > + * Make sure that pages are in the same section before doing pointer
> > > > > + * arithmetics.
> > > > > + */
> > > > > + if (page_to_section(pvmw->page) != page_to_section(page))
> > > > > + return false;
> > > >
> > > > OK, THPs shouldn't cross memory sections AFAIK. My brain is meltdown
> > > > these days so this might be a completely stupid question. But why don't
> > > > you simply compare pfns? This would be just simpler, no?
> > >
> > > In original code, we already had pvmw->page around and I thought it would
> > > be easier to get page for the pte intead of looking for pfn for both
> > > sides.
> > >
> > > We these changes it's no longer the case.
> > >
> > > Do you care enough to send a patch? :)
> >
> > Well, memory sections are sparsemem concept IIRC. Unless I've missed
> > something page_to_section is quarded by SECTION_IN_PAGE_FLAGS and that
> > is conditional to CONFIG_SPARSEMEM. THP is a generic code so using it
> > there is wrong unless I miss some subtle detail here.
> >
> > Comparing pfn should be generic enough.
>
> Good point.
>
> What about something like this?
>
> >From 861f68c555b87fd6c0ccc3428ace91b7e185b73a Mon Sep 17 00:00:00 2001
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Date: Thu, 18 Jan 2018 18:24:07 +0300
> Subject: [PATCH] mm, page_vma_mapped: Drop faulty pointer arithmetics in
> check_pte()
>
> Tetsuo reported random crashes under memory pressure on 32-bit x86
> system and tracked down to change that introduced
> page_vma_mapped_walk().
>
> The root cause of the issue is the faulty pointer math in check_pte().
> As ->pte may point to an arbitrary page we have to check that they are
> belong to the section before doing math. Otherwise it may lead to weird
> results.
>
> It wasn't noticed until now as mem_map[] is virtually contiguous on flatmem or
> vmemmap sparsemem. Pointer arithmetic just works against all 'struct page'
> pointers. But with classic sparsemem, it doesn't.
it doesn't because each section memap is allocated separately and so
consecutive pfns crossing two sections might have struct pages at
completely unrelated addresses.
> Let's restructure code a bit and replace pointer arithmetic with
> operations on pfns.
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Reported-by: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
> Fixes: ace71a19cec5 ("mm: introduce page_vma_mapped_walk()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
The patch makes sense but there is one more thing to fix here.
[...]
> static bool check_pte(struct page_vma_mapped_walk *pvmw)
> {
> + unsigned long pfn;
> +
> if (pvmw->flags & PVMW_MIGRATION) {
> #ifdef CONFIG_MIGRATION
> swp_entry_t entry;
> @@ -41,37 +61,34 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
>
> if (!is_migration_entry(entry))
> return false;
> - if (migration_entry_to_page(entry) - pvmw->page >=
> - hpage_nr_pages(pvmw->page)) {
> - return false;
> - }
> - if (migration_entry_to_page(entry) < pvmw->page)
> - return false;
> +
> + pfn = migration_entry_to_pfn(entry);
> #else
> WARN_ON_ONCE(1);
> #endif
> - } else {
now you allow to pass through with uninitialized pfn. We used to return
true in that case so we should probably keep it in this WARN_ON_ONCE
case. Please note that I haven't studied this particular case and the
ifdef is definitely not an act of art but that is a separate topic.
> - if (is_swap_pte(*pvmw->pte)) {
> - swp_entry_t entry;
> + } else if (is_swap_pte(*pvmw->pte)) {
> + swp_entry_t entry;
>
> - entry = pte_to_swp_entry(*pvmw->pte);
> - if (is_device_private_entry(entry) &&
> - device_private_entry_to_page(entry) == pvmw->page)
> - return true;
> - }
> + /* Handle un-addressable ZONE_DEVICE memory */
> + entry = pte_to_swp_entry(*pvmw->pte);
> + if (!is_device_private_entry(entry))
> + return false;
>
> + pfn = device_private_entry_to_pfn(entry);
> + } else {
> if (!pte_present(*pvmw->pte))
> return false;
>
> - /* THP can be referenced by any subpage */
> - if (pte_page(*pvmw->pte) - pvmw->page >=
> - hpage_nr_pages(pvmw->page)) {
> - return false;
> - }
> - if (pte_page(*pvmw->pte) < pvmw->page)
> - return false;
> + pfn = pte_pfn(*pvmw->pte);
> }
>
> + if (pfn < page_to_pfn(pvmw->page))
> + return false;
> +
> + /* THP can be referenced by any subpage */
> + if (pfn - page_to_pfn(pvmw->page) >= hpage_nr_pages(pvmw->page))
> + return false;
> +
> return true;
> }
>
> --
> Kirill A. Shutemov
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-19 12:07 ` Michal Hocko
@ 2018-01-19 12:30 ` Kirill A. Shutemov
0 siblings, 0 replies; 59+ messages in thread
From: Kirill A. Shutemov @ 2018-01-19 12:30 UTC (permalink / raw)
To: Michal Hocko
Cc: Kirill A. Shutemov, Dave Hansen, Tetsuo Handa, torvalds, akpm,
hannes, iamjoonsoo.kim, mgorman, tony.luck, vbabka, aarcange,
hillf.zj, hughd, oleg, peterz, riel, srikar, vdavydov.dev, mingo,
linux-kernel, linux-mm, x86
On Fri, Jan 19, 2018 at 12:07:47PM +0000, Michal Hocko wrote:
> > >From 861f68c555b87fd6c0ccc3428ace91b7e185b73a Mon Sep 17 00:00:00 2001
> > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> > Date: Thu, 18 Jan 2018 18:24:07 +0300
> > Subject: [PATCH] mm, page_vma_mapped: Drop faulty pointer arithmetics in
> > check_pte()
> >
> > Tetsuo reported random crashes under memory pressure on 32-bit x86
> > system and tracked down to change that introduced
> > page_vma_mapped_walk().
> >
> > The root cause of the issue is the faulty pointer math in check_pte().
> > As ->pte may point to an arbitrary page we have to check that they are
> > belong to the section before doing math. Otherwise it may lead to weird
> > results.
> >
> > It wasn't noticed until now as mem_map[] is virtually contiguous on flatmem or
> > vmemmap sparsemem. Pointer arithmetic just works against all 'struct page'
> > pointers. But with classic sparsemem, it doesn't.
>
> it doesn't because each section memap is allocated separately and so
> consecutive pfns crossing two sections might have struct pages at
> completely unrelated addresses.
Okay, I'll amend it.
> > Let's restructure code a bit and replace pointer arithmetic with
> > operations on pfns.
> >
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > Reported-by: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
> > Fixes: ace71a19cec5 ("mm: introduce page_vma_mapped_walk()")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>
> The patch makes sense but there is one more thing to fix here.
>
> [...]
> > static bool check_pte(struct page_vma_mapped_walk *pvmw)
> > {
> > + unsigned long pfn;
> > +
> > if (pvmw->flags & PVMW_MIGRATION) {
> > #ifdef CONFIG_MIGRATION
> > swp_entry_t entry;
> > @@ -41,37 +61,34 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
> >
> > if (!is_migration_entry(entry))
> > return false;
> > - if (migration_entry_to_page(entry) - pvmw->page >=
> > - hpage_nr_pages(pvmw->page)) {
> > - return false;
> > - }
> > - if (migration_entry_to_page(entry) < pvmw->page)
> > - return false;
> > +
> > + pfn = migration_entry_to_pfn(entry);
> > #else
> > WARN_ON_ONCE(1);
> > #endif
> > - } else {
>
> now you allow to pass through with uninitialized pfn. We used to return
> true in that case so we should probably keep it in this WARN_ON_ONCE
> case. Please note that I haven't studied this particular case and the
> ifdef is definitely not an act of art but that is a separate topic.
Good catch. Thanks.
I think returning true here is wrong as we don't validate in any way what
is mapped there. I'll put "return false;".
And I take a look if we can drop the #ifdef.
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm 4.15-rc8] Random oopses under memory pressure.
2018-01-18 15:40 ` Kirill A. Shutemov
2018-01-18 17:22 ` Michal Hocko
@ 2018-01-19 2:01 ` Tetsuo Handa
1 sibling, 0 replies; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-19 2:01 UTC (permalink / raw)
To: Kirill A. Shutemov, torvalds
Cc: Dave Hansen, kirill.shutemov, akpm, hannes, iamjoonsoo.kim,
mgorman, tony.luck, vbabka, mhocko, aarcange, hillf.zj, hughd,
oleg, peterz, riel, srikar, vdavydov.dev, mingo, linux-kernel,
linux-mm, x86
Kirill A. Shutemov wrote:
> Something like this?
>
>
> From 251e124630da82482e8b320c73162ce89af04d5d Mon Sep 17 00:00:00 2001
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Date: Thu, 18 Jan 2018 18:24:07 +0300
> Subject: [PATCH] mm, page_vma_mapped: Fix pointer arithmetics in check_pte()
>
> Tetsuo reported random crashes under memory pressure on 32-bit x86
> system and tracked down to change that introduced
> page_vma_mapped_walk().
>
> The root cause of the issue is the faulty pointer math in check_pte().
> As ->pte may point to an arbitrary page we have to check that they are
> belong to the section before doing math. Otherwise it may lead to weird
> results.
>
> It wasn't noticed until now as mem_map[] is virtually contiguous on flatmem or
> vmemmap sparsemem. Pointer arithmetic just works against all 'struct page'
> pointers. But with classic sparsemem, it doesn't.
>
> Let's restructure code a bit and add necessary check.
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Reported-by: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
> Fixes: ace71a19cec5 ("mm: introduce page_vma_mapped_walk()")
> Cc: stable@vger.kernel.org
This patch solves the problem. Thank you.
Tested-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
> ---
> mm/page_vma_mapped.c | 66 +++++++++++++++++++++++++++++++++++-----------------
> 1 file changed, 45 insertions(+), 21 deletions(-)
>
> diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
> index d22b84310f6d..de195dcdfbd8 100644
> --- a/mm/page_vma_mapped.c
> +++ b/mm/page_vma_mapped.c
> @@ -30,8 +30,28 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw)
> return true;
> }
>
> +/**
> + * check_pte - check if @pvmw->page is mapped at the @pvmw->pte
> + *
> + * page_vma_mapped_walk() found a place where @pvmw->page is *potentially*
> + * mapped. check_pte() has to validate this.
> + *
> + * @pvmw->pte may point to empty PTE, swap PTE or PTE pointing to arbitrary
> + * page.
> + *
> + * If PVMW_MIGRATION flag is set, returns true if @pvmw->pte contains migration
> + * entry that points to @pvmw->page or any subpage in case of THP.
> + *
> + * If PVMW_MIGRATION flag is not set, returns true if @pvmw->pte points to
> + * @pvmw->page or any subpage in case of THP.
> + *
> + * Otherwise, return false.
> + *
> + */
> static bool check_pte(struct page_vma_mapped_walk *pvmw)
> {
> + struct page *page;
> +
> if (pvmw->flags & PVMW_MIGRATION) {
> #ifdef CONFIG_MIGRATION
> swp_entry_t entry;
> @@ -41,37 +61,41 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw)
>
> if (!is_migration_entry(entry))
> return false;
> - if (migration_entry_to_page(entry) - pvmw->page >=
> - hpage_nr_pages(pvmw->page)) {
> - return false;
> - }
> - if (migration_entry_to_page(entry) < pvmw->page)
> - return false;
> +
> + page = migration_entry_to_page(entry);
> #else
> WARN_ON_ONCE(1);
> #endif
> - } else {
> - if (is_swap_pte(*pvmw->pte)) {
> - swp_entry_t entry;
> + } else if (is_swap_pte(*pvmw->pte)) {
> + swp_entry_t entry;
>
> - entry = pte_to_swp_entry(*pvmw->pte);
> - if (is_device_private_entry(entry) &&
> - device_private_entry_to_page(entry) == pvmw->page)
> - return true;
> - }
> + /* Handle un-addressable ZONE_DEVICE memory */
> + entry = pte_to_swp_entry(*pvmw->pte);
> + if (!is_device_private_entry(entry))
> + return false;
>
> + page = device_private_entry_to_page(entry);
> + } else {
> if (!pte_present(*pvmw->pte))
> return false;
>
> - /* THP can be referenced by any subpage */
> - if (pte_page(*pvmw->pte) - pvmw->page >=
> - hpage_nr_pages(pvmw->page)) {
> - return false;
> - }
> - if (pte_page(*pvmw->pte) < pvmw->page)
> - return false;
> + page = pte_page(*pvmw->pte);
> }
>
> + /*
> + * Make sure that pages are in the same section before doing pointer
> + * arithmetics.
> + */
> + if (page_to_section(pvmw->page) != page_to_section(page))
> + return false;
> +
> + if (page < pvmw->page)
> + return false;
> +
> + /* THP can be referenced by any subpage */
> + if (page - pvmw->page >= hpage_nr_pages(pvmw->page))
> + return false;
> +
> return true;
> }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-11 14:11 ` Tetsuo Handa
2018-01-11 14:21 ` Michal Hocko
@ 2018-01-11 18:11 ` Linus Torvalds
2018-01-11 20:59 ` Tetsuo Handa
1 sibling, 1 reply; 59+ messages in thread
From: Linus Torvalds @ 2018-01-11 18:11 UTC (permalink / raw)
To: Tetsuo Handa
Cc: Michal Hocko, linux-mm, the arch/x86 maintainers, linux-fsdevel
On Thu, Jan 11, 2018 at 6:11 AM, Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
>
> I retested with some debug printk() patch.
Could you perhaps enable KASAN too?
> [ 38.988178] Out of memory: Kill process 354 (b.out) score 7 or sacrifice child
> [ 38.991145] Killed process 354 (b.out) total-vm:2099260kB, anon-rss:23288kB, file-rss:8kB, shmem-rss:0kB
> [ 38.996277] oom_reaper: started reaping
> [ 38.999033] BUG: unable to handle kernel paging request at c130d86d
> [ 39.001802] IP: _raw_spin_lock_irqsave+0x1c/0x40
The "Code:" line shows the whole function in this case:
0: 55 push %ebp
1: 89 c1 mov %eax,%ecx
3: 89 e5 mov %esp,%ebp
5: 56 push %esi
6: 53 push %ebx
7: 9c pushf
8: 58 pop %eax
9: 66 66 66 90 nop
d: 89 c6 mov %eax,%esi
f: fa cli
10: 66 66 90 nop
13: 66 90 nop
15: 31 c0 xor %eax,%eax
17: bb 01 00 00 00 mov $0x1,%ebx
1c: 3e 0f b1 19 cmpxchg %ebx,%ds:*(%ecx)
<-- trapping instruction
20: 85 c0 test %eax,%eax
22: 75 06 jne 0x2a
24: 89 f0 mov %esi,%eax
26: 5b pop %ebx
27: 5e pop %esi
28: 5d pop %ebp
29: c3 ret
although it isn't all that interesting since it's just
"_raw_spin_lock_irqsave". The odd "nop" instructions are because of
paravirtualization support leaving room for rewriting the eflags
operations.
Anyway, %ecx is garbage - it *should* be "&memcg->move_lock",
apparently. The caller does:
again:
memcg = page->mem_cgroup;
if (unlikely(!memcg))
return NULL;
if (atomic_read(&memcg->moving_account) <= 0)
return memcg;
spin_lock_irqsave(&memcg->move_lock, flags);
if (memcg != page->mem_cgroup) {
spin_unlock_irqrestore(&memcg->move_lock, flags);
goto again;
}
What's a bit odd is how the access to "memcg->move_lock" seems to
trap, but we did that atomic_read() from memcg->moving_account ok.
The reason seems to be that this is actually a valid kernel pointer,
but it's read-protected:
> [ 39.004069] *pde = 01f88063 *pte = 0130d161
> [ 39.006250] Oops: 0003 [#1] SMP DEBUG_PAGEALLOC
That "0003" means that it was a protection fault on a write. The
"*pte" thing agrees. It's the normal 1:1 mapping of the physical page
0130d000 (which matches the virtual address c130d86d), but it's
presumably a kernel code pointer and this RO.
So presumably "page->mem_cgroup" was just a random pointer. Which
probably means that "page" itself is not actually a page pointer, sinc
eI assume there was no memory hotplug going on here?
> [ 39.022885] EIP: _raw_spin_lock_irqsave+0x1c/0x40
> [ 39.037889] Call Trace:
> [ 39.043562] lock_page_memcg+0x25/0x80
> [ 39.045421] page_remove_rmap+0x87/0x2e0
> [ 39.047315] try_to_unmap_one+0x20e/0x590
> [ 39.049198] rmap_walk_file+0x13c/0x250
> [ 39.051012] rmap_walk+0x32/0x60
> [ 39.052619] try_to_unmap+0x4d/0x100
> [ 39.059849] shrink_page_list+0x3a2/0x1000
> [ 39.061678] shrink_inactive_list+0x1b2/0x440
> [ 39.063539] shrink_node_memcg+0x34a/0x770
> [ 39.065297] shrink_node+0xbb/0x2e0
> [ 39.066920] do_try_to_free_pages+0xba/0x320
> [ 39.068752] try_to_free_pages+0x11d/0x330
> [ 39.072084] __alloc_pages_slowpath+0x303/0x6d9
> [ 39.075932] __alloc_pages_nodemask+0x16d/0x180
> [ 39.077809] do_anonymous_page+0xab/0x4f0
> [ 39.079551] handle_mm_fault+0x531/0x8d0
> [ 39.084422] __do_page_fault+0x1ea/0x4d0
> [ 39.087666] do_page_fault+0x1a/0x20
> [ 39.089184] common_exception+0x6f/0x76
Looks like the page pointer came from shrink_inactive_list() doing
isolate_lru_pages().
Scary. It all seems to just mean that the page LRU queues are corrupted.
Most (all?) of your other oopses seem to have somewhat similar
patterns: shrink_inactive_list() -> rmap_walk_file() -> oops due to
garbage.
> Overall, memory corruption is strongly suspected.
Yeah, this very much looks like some internal VM memory corruption.
Which is why I'm wondering if enabling KASAN might help find the
actual access that causes the corruption. Or at least an _earlier_
access that is closer to it than these that all seem to be fairly far
removed from where it actually all started..
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [mm? 4.15-rc7] Random oopses under memory pressure.
2018-01-11 18:11 ` [mm? 4.15-rc7] " Linus Torvalds
@ 2018-01-11 20:59 ` Tetsuo Handa
0 siblings, 0 replies; 59+ messages in thread
From: Tetsuo Handa @ 2018-01-11 20:59 UTC (permalink / raw)
To: torvalds; +Cc: mhocko, linux-mm, x86, linux-fsdevel
Linus Torvalds wrote:
> On Thu, Jan 11, 2018 at 6:11 AM, Tetsuo Handa
> <penguin-kernel@i-love.sakura.ne.jp> wrote:
> >
> > I retested with some debug printk() patch.
>
> Could you perhaps enable KASAN too?
Unfortunately, KASAN is not available for x86_32 kernels. Thus, I'm stuck.
> So presumably "page->mem_cgroup" was just a random pointer. Which
> probably means that "page" itself is not actually a page pointer, sinc
> e I assume there was no memory hotplug going on here?
Nothing special. No memory hotplug etc.
> Most (all?) of your other oopses seem to have somewhat similar
> patterns: shrink_inactive_list() -> rmap_walk_file() -> oops due to
> garbage.
Yes. In most cases, problems are detected by that sequence.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 59+ messages in thread
end of thread, other threads:[~2018-01-20 9:38 UTC | newest]
Thread overview: 59+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-05 14:45 [x86? mm? fs? 4.15-rc6] Random oopses by simple write under memory pressure Tetsuo Handa
2018-01-09 10:39 ` [mm? 4.15-rc7] " Tetsuo Handa
2018-01-10 11:49 ` [mm? 4.15-rc7] Random oopses " Tetsuo Handa
2018-01-10 12:45 ` Michal Hocko
2018-01-10 13:37 ` Tetsuo Handa
2018-01-11 13:57 ` Michal Hocko
2018-01-11 14:11 ` Tetsuo Handa
2018-01-11 14:21 ` Michal Hocko
2018-01-11 14:37 ` Tetsuo Handa
2018-01-12 1:31 ` [mm " Tetsuo Handa
2018-01-12 1:42 ` Linus Torvalds
2018-01-12 11:22 ` Tetsuo Handa
2018-01-14 11:54 ` Tetsuo Handa
2018-01-15 23:05 ` Linus Torvalds
2018-01-16 1:15 ` [mm 4.15-rc8] " Tetsuo Handa
2018-01-16 2:14 ` Linus Torvalds
2018-01-16 8:06 ` Dave Hansen
2018-01-16 8:37 ` Ingo Molnar
2018-01-16 19:30 ` Linus Torvalds
2018-01-16 17:33 ` Tetsuo Handa
2018-01-16 19:34 ` Linus Torvalds
2018-01-17 11:08 ` Tetsuo Handa
2018-01-17 21:39 ` Linus Torvalds
2018-01-17 21:51 ` Linus Torvalds
2018-01-17 22:04 ` Dave Hansen
2018-01-17 22:00 ` Dave Hansen
2018-01-17 22:15 ` Linus Torvalds
2018-01-18 8:12 ` Tetsuo Handa
2018-01-18 12:25 ` Kirill A. Shutemov
2018-01-18 13:12 ` Kirill A. Shutemov
2018-01-18 14:34 ` Kirill A. Shutemov
2018-01-18 14:38 ` Dave Hansen
2018-01-18 14:45 ` Kirill A. Shutemov
2018-01-18 14:51 ` Dave Hansen
2018-01-18 16:58 ` Linus Torvalds
2018-01-18 14:45 ` Dave Hansen
2018-01-18 14:58 ` Andrea Arcangeli
2018-01-18 16:56 ` Kirill A. Shutemov
2018-01-18 17:26 ` Luck, Tony
2018-01-18 17:28 ` Linus Torvalds
2018-01-18 17:26 ` Linus Torvalds
2018-01-18 23:49 ` Kirill A. Shutemov
2018-01-19 12:55 ` Matthew Wilcox
2018-01-19 18:42 ` Linus Torvalds
2018-01-19 22:12 ` Al Viro
2018-01-19 22:53 ` Linus Torvalds
2018-01-20 2:02 ` Al Viro
2018-01-20 5:24 ` Al Viro
2018-01-20 9:38 ` Luc Van Oostenryck
2018-01-18 15:40 ` Kirill A. Shutemov
2018-01-18 17:22 ` Michal Hocko
2018-01-19 10:02 ` Kirill A. Shutemov
2018-01-19 10:33 ` Michal Hocko
2018-01-19 11:49 ` Kirill A. Shutemov
2018-01-19 12:07 ` Michal Hocko
2018-01-19 12:30 ` Kirill A. Shutemov
2018-01-19 2:01 ` Tetsuo Handa
2018-01-11 18:11 ` [mm? 4.15-rc7] " Linus Torvalds
2018-01-11 20:59 ` Tetsuo Handa
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).