* 5.8.0rc3 page allocation failure: order:0, mode:0x40810
@ 2020-07-04 7:21 Chris Murphy
2020-07-04 8:02 ` Pekka Enberg
2020-07-04 11:37 ` Matthew Wilcox
0 siblings, 2 replies; 6+ messages in thread
From: Chris Murphy @ 2020-07-04 7:21 UTC (permalink / raw)
To: Linux Memory Management List
Hi,
This seems to be a one off - I'm not even sure what was happening at
the time. Possibly running a qemu-kvm VM.
5.8.0-0.rc3.1.fc33.x86_64+debug
i7-2820QM
MemTotal: 8021304 kB
$ swapon
NAME TYPE SIZE USED PRIO
/dev/sda5 partition 10.4G 0B -2
/dev/zram0 partition 3.8G 9M 100
$ zramctl
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo-rle 3.8G 7.8M 2.3M 4.5M 8 [SWAP]
[126841.385050] swapper/2: page allocation failure: order:0,
mode:0x40810(GFP_NOWAIT|__GFP_COMP|__GFP_RECLAIMABLE),
nodemask=(null),cpuset=/,mems_allowed=0
[126841.385059] CPU: 2 PID: 0 Comm: swapper/2 Not tainted
5.8.0-0.rc3.1.fc33.x86_64+debug #1
[126841.385061] Hardware name: Apple Inc.
MacBookPro8,2/Mac-94245A3940C91C80, BIOS
MBP81.88Z.0050.B00.1804101331 04/10/18
[126841.385062] Call Trace:
[126841.385064] <IRQ>
[126841.385071] dump_stack+0x92/0xc8
[126841.385076] warn_alloc.cold+0x75/0xd9
[126841.385083] __alloc_pages_slowpath.constprop.0+0x10ce/0x1110
[126841.385090] ? lock_acquire+0xb0/0x400
[126841.385095] __alloc_pages_nodemask+0x3c4/0x450
[126841.385099] alloc_slab_page+0x345/0x790
[126841.385103] allocate_slab+0x473/0x570
[126841.385106] ___slab_alloc+0x4bb/0x7d0
[126841.385109] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
[126841.385115] ? __module_address.part.0+0xe/0xe0
[126841.385117] ? __module_address.part.0+0xe/0xe0
[126841.385120] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
[126841.385122] ? __slab_alloc+0x46/0x90
[126841.385123] __slab_alloc+0x46/0x90
[126841.385126] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
[126841.385128] kmem_cache_alloc+0x2c8/0x310
[126841.385131] radix_tree_node_alloc.constprop.0+0x44/0xe0
[126841.385134] radix_tree_insert+0xd1/0x1b0
[126841.385139] add_dma_entry+0xb4/0x1a0
[126841.385145] dma_map_single_attrs.constprop.0+0xcb/0x150 [tg3]
[126841.385149] tg3_alloc_rx_data+0xaa/0x190 [tg3]
[126841.385153] tg3_poll_work+0x1d7/0x10b0 [tg3]
[126841.385160] tg3_poll_msix+0x39/0x1c0 [tg3]
[126841.385164] net_rx_action+0x150/0x4f0
[126841.385170] __do_softirq+0xdb/0x4a2
[126841.385175] asm_call_on_stack+0xf/0x20
[126841.385176] </IRQ>
[126841.385179] do_softirq_own_stack+0x80/0x90
[126841.385182] irq_exit_rcu+0xe0/0x120
[126841.385185] common_interrupt+0xaa/0x210
[126841.385188] asm_common_interrupt+0x1e/0x40
[126841.385191] RIP: 0010:cpuidle_enter_state+0xb8/0x440
[126841.385194] Code: 69 6b e8 eb 04 7b ff 49 89 c7 66 66 66 66 90 31
ff e8 bc 27 7b ff 80 7c 24 0f 00 0f 85 0d 02 00 00 e8 3c 3c 8b ff fb
66 66 90 <66> 66 90 45 85 e4 0f 88 14 02 00 00 49 63 d4 4c 2b 7c 24 10
48 8d
[126841.385196] RSP: 0018:ffffab2f0009fe80 EFLAGS: 00000246
[126841.385198] RAX: 0000000000000000 RBX: ffffcb2eff201400 RCX:
000000000000001f
[126841.385199] RDX: 0000000000000000 RSI: 0000000037a6deab RDI:
ffffffff949749b4
[126841.385201] RBP: ffffffff95c18e80 R08: 0000735c8fb2993f R09:
0000000000000008
[126841.385202] R10: 0000000000000000 R11: 0000000000000016 R12:
0000000000000002
[126841.385204] R13: ffffffff95c18e80 R14: 0000000000000002 R15:
0000735c8fb2993f
[126841.385208] ? cpuidle_enter_state+0xb4/0x440
[126841.385213] cpuidle_enter+0x29/0x40
[126841.385217] do_idle+0x1f6/0x2e0
[126841.385220] cpu_startup_entry+0x19/0x20
[126841.385224] start_secondary+0x15c/0x190
[126841.385227] secondary_startup_64+0xb6/0xc0
[126841.385233] Mem-Info:
[126841.385237] active_anon:141209 inactive_anon:596767 isolated_anon:0
active_file:122875 inactive_file:993681 isolated_file:9
unevictable:2594 dirty:110075 writeback:0
slab_reclaimable:42804 slab_unreclaimable:46586
mapped:65915 shmem:676937 pagetables:2719 bounce:0
free:25946 free_pcp:2811 free_cma:0
[126841.385241] Node 0 active_anon:564836kB inactive_anon:2387068kB
active_file:491500kB inactive_file:3974724kB unevictable:10376kB
isolated(anon):0kB isolated(file):36kB mapped:263660kB dirty:440300kB
writeback:0kB shmem:2707748kB shmem_thp: 0kB shmem_pmdmapped: 0kB
anon_thp: 0kB writeback_tmp:0kB all_unreclaimable? no
[126841.385242] Node 0 DMA free:14320kB min:128kB low:160kB high:192kB
reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB
active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB
present:15988kB managed:15360kB mlocked:0kB kernel_stack:0kB
pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[126841.385246] lowmem_reserve[]: 0 2048 7776 7776 7776
[126841.385249] Node 0 DMA32 free:40352kB min:17772kB low:22212kB
high:26652kB reserved_highatomic:0KB active_anon:0kB
inactive_anon:18908kB active_file:0kB inactive_file:2063284kB
unevictable:0kB writepending:96256kB present:2255864kB
managed:2140908kB mlocked:0kB kernel_stack:0kB pagetables:0kB
bounce:0kB free_pcp:4188kB local_pcp:36kB free_cma:0kB
[126841.385253] lowmem_reserve[]: 0 0 5727 5727 5727
[126841.385255] Node 0 Normal free:49112kB min:49680kB low:62100kB
high:74520kB reserved_highatomic:0KB active_anon:564748kB
inactive_anon:2368400kB active_file:491756kB inactive_file:1911480kB
unevictable:10700kB writepending:343392kB present:6027264kB
managed:5865036kB mlocked:1800kB kernel_stack:6768kB
pagetables:10876kB bounce:0kB free_pcp:6896kB local_pcp:328kB
free_cma:0kB
[126841.385259] lowmem_reserve[]: 0 0 0 0 0
[126841.385261] Node 0 DMA: 0*4kB 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB
(U) 1*128kB (U) 1*256kB (U) 1*512kB (U) 1*1024kB (U) 2*2048kB (UM)
2*4096kB (M) = 14320kB
[126841.385269] Node 0 DMA32: 1*4kB (E) 3*8kB (UME) 2*16kB (UE) 3*32kB
(UME) 7*64kB (E) 3*128kB (UE) 6*256kB (ME) 5*512kB (ME) 2*1024kB (UM)
2*2048kB (UM) 7*4096kB (M) = 39900kB
[126841.385277] Node 0 Normal: 366*4kB (UME) 413*8kB (UME) 76*16kB
(UME) 18*32kB (UME) 20*64kB (UME) 16*128kB (UME) 10*256kB (UME)
8*512kB (ME) 5*1024kB (UE) 2*2048kB (UM) 6*4096kB (M) = 50336kB
[126841.385289] Node 0 hugepages_total=0 hugepages_free=0
hugepages_surp=0 hugepages_size=2048kB
[126841.385290] 1793872 total pagecache pages
[126841.385294] 45 pages in swap cache
[126841.385296] Swap cache stats: add 14529, delete 14484, find 3855/4431
[126841.385297] Free swap = 14869404kB
[126841.385299] Total swap = 14892188kB
[126841.385300] 2074779 pages RAM
[126841.385301] 0 pages HighMem/MovableOnly
[126841.385302] 69453 pages reserved
[126841.385304] 0 pages cma reserved
[126841.385305] 0 pages hwpoisoned
[126841.385307] SLUB: Unable to allocate memory on node -1,
gfp=0x800(GFP_NOWAIT)
[126841.385309] cache: radix_tree_node, object size: 576, buffer
size: 584, default order: 2, min order: 0
[126841.385311] node 0: slabs: 392, objs: 10976, free: 0
[126841.385312] DMA-API: cacheline tracking ENOMEM, dma-debug disabled
Entire dmesg:
https://bugzilla.redhat.com/attachment.cgi?id=1699909
Downstream bug (same as above)
https://bugzilla.redhat.com/show_bug.cgi?id=1853795
--
Chris Murphy
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 5.8.0rc3 page allocation failure: order:0, mode:0x40810
2020-07-04 7:21 5.8.0rc3 page allocation failure: order:0, mode:0x40810 Chris Murphy
@ 2020-07-04 8:02 ` Pekka Enberg
2020-07-04 11:37 ` Matthew Wilcox
1 sibling, 0 replies; 6+ messages in thread
From: Pekka Enberg @ 2020-07-04 8:02 UTC (permalink / raw)
To: Chris Murphy; +Cc: Linux Memory Management List
On Sat, Jul 4, 2020 at 10:21 AM Chris Murphy <lists@colorremedies.com> wrote:
> This seems to be a one off - I'm not even sure what was happening at
> the time. Possibly running a qemu-kvm VM.
The network stack is doing RX polling in the tg3 driver, but a page
allocation initiated by the slab allocator failed. Specifically, an
insertion to a radix tree by dma_map_single_attrs() ran out of slab
objects, and is therefore calling into the page allocator.
I don't see what's wrong here, though. The page allocation does not
seem to have any unreasonable constraints and there's memory available
in the normal zone too (where the page allocation is going AFAICT).
> 5.8.0-0.rc3.1.fc33.x86_64+debug
> i7-2820QM
> MemTotal: 8021304 kB
> $ swapon
> NAME TYPE SIZE USED PRIO
> /dev/sda5 partition 10.4G 0B -2
> /dev/zram0 partition 3.8G 9M 100
> $ zramctl
> NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
> /dev/zram0 lzo-rle 3.8G 7.8M 2.3M 4.5M 8 [SWAP]
>
>
>
>
> [126841.385050] swapper/2: page allocation failure: order:0,
> mode:0x40810(GFP_NOWAIT|__GFP_COMP|__GFP_RECLAIMABLE),
> nodemask=(null),cpuset=/,mems_allowed=0
> [126841.385059] CPU: 2 PID: 0 Comm: swapper/2 Not tainted
> 5.8.0-0.rc3.1.fc33.x86_64+debug #1
> [126841.385061] Hardware name: Apple Inc.
> MacBookPro8,2/Mac-94245A3940C91C80, BIOS
> MBP81.88Z.0050.B00.1804101331 04/10/18
> [126841.385062] Call Trace:
> [126841.385064] <IRQ>
> [126841.385071] dump_stack+0x92/0xc8
> [126841.385076] warn_alloc.cold+0x75/0xd9
> [126841.385083] __alloc_pages_slowpath.constprop.0+0x10ce/0x1110
> [126841.385090] ? lock_acquire+0xb0/0x400
> [126841.385095] __alloc_pages_nodemask+0x3c4/0x450
> [126841.385099] alloc_slab_page+0x345/0x790
> [126841.385103] allocate_slab+0x473/0x570
> [126841.385106] ___slab_alloc+0x4bb/0x7d0
> [126841.385109] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
> [126841.385115] ? __module_address.part.0+0xe/0xe0
> [126841.385117] ? __module_address.part.0+0xe/0xe0
> [126841.385120] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
> [126841.385122] ? __slab_alloc+0x46/0x90
> [126841.385123] __slab_alloc+0x46/0x90
> [126841.385126] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
> [126841.385128] kmem_cache_alloc+0x2c8/0x310
> [126841.385131] radix_tree_node_alloc.constprop.0+0x44/0xe0
> [126841.385134] radix_tree_insert+0xd1/0x1b0
> [126841.385139] add_dma_entry+0xb4/0x1a0
> [126841.385145] dma_map_single_attrs.constprop.0+0xcb/0x150 [tg3]
> [126841.385149] tg3_alloc_rx_data+0xaa/0x190 [tg3]
> [126841.385153] tg3_poll_work+0x1d7/0x10b0 [tg3]
> [126841.385160] tg3_poll_msix+0x39/0x1c0 [tg3]
> [126841.385164] net_rx_action+0x150/0x4f0
> [126841.385170] __do_softirq+0xdb/0x4a2
> [126841.385175] asm_call_on_stack+0xf/0x20
> [126841.385176] </IRQ>
> [126841.385179] do_softirq_own_stack+0x80/0x90
> [126841.385182] irq_exit_rcu+0xe0/0x120
> [126841.385185] common_interrupt+0xaa/0x210
> [126841.385188] asm_common_interrupt+0x1e/0x40
> [126841.385191] RIP: 0010:cpuidle_enter_state+0xb8/0x440
> [126841.385194] Code: 69 6b e8 eb 04 7b ff 49 89 c7 66 66 66 66 90 31
> ff e8 bc 27 7b ff 80 7c 24 0f 00 0f 85 0d 02 00 00 e8 3c 3c 8b ff fb
> 66 66 90 <66> 66 90 45 85 e4 0f 88 14 02 00 00 49 63 d4 4c 2b 7c 24 10
> 48 8d
> [126841.385196] RSP: 0018:ffffab2f0009fe80 EFLAGS: 00000246
> [126841.385198] RAX: 0000000000000000 RBX: ffffcb2eff201400 RCX:
> 000000000000001f
> [126841.385199] RDX: 0000000000000000 RSI: 0000000037a6deab RDI:
> ffffffff949749b4
> [126841.385201] RBP: ffffffff95c18e80 R08: 0000735c8fb2993f R09:
> 0000000000000008
> [126841.385202] R10: 0000000000000000 R11: 0000000000000016 R12:
> 0000000000000002
> [126841.385204] R13: ffffffff95c18e80 R14: 0000000000000002 R15:
> 0000735c8fb2993f
> [126841.385208] ? cpuidle_enter_state+0xb4/0x440
> [126841.385213] cpuidle_enter+0x29/0x40
> [126841.385217] do_idle+0x1f6/0x2e0
> [126841.385220] cpu_startup_entry+0x19/0x20
> [126841.385224] start_secondary+0x15c/0x190
> [126841.385227] secondary_startup_64+0xb6/0xc0
> [126841.385233] Mem-Info:
> [126841.385237] active_anon:141209 inactive_anon:596767 isolated_anon:0
> active_file:122875 inactive_file:993681 isolated_file:9
> unevictable:2594 dirty:110075 writeback:0
> slab_reclaimable:42804 slab_unreclaimable:46586
> mapped:65915 shmem:676937 pagetables:2719 bounce:0
> free:25946 free_pcp:2811 free_cma:0
> [126841.385241] Node 0 active_anon:564836kB inactive_anon:2387068kB
> active_file:491500kB inactive_file:3974724kB unevictable:10376kB
> isolated(anon):0kB isolated(file):36kB mapped:263660kB dirty:440300kB
> writeback:0kB shmem:2707748kB shmem_thp: 0kB shmem_pmdmapped: 0kB
> anon_thp: 0kB writeback_tmp:0kB all_unreclaimable? no
> [126841.385242] Node 0 DMA free:14320kB min:128kB low:160kB high:192kB
> reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB
> active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB
> present:15988kB managed:15360kB mlocked:0kB kernel_stack:0kB
> pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
> [126841.385246] lowmem_reserve[]: 0 2048 7776 7776 7776
> [126841.385249] Node 0 DMA32 free:40352kB min:17772kB low:22212kB
> high:26652kB reserved_highatomic:0KB active_anon:0kB
> inactive_anon:18908kB active_file:0kB inactive_file:2063284kB
> unevictable:0kB writepending:96256kB present:2255864kB
> managed:2140908kB mlocked:0kB kernel_stack:0kB pagetables:0kB
> bounce:0kB free_pcp:4188kB local_pcp:36kB free_cma:0kB
> [126841.385253] lowmem_reserve[]: 0 0 5727 5727 5727
> [126841.385255] Node 0 Normal free:49112kB min:49680kB low:62100kB
> high:74520kB reserved_highatomic:0KB active_anon:564748kB
> inactive_anon:2368400kB active_file:491756kB inactive_file:1911480kB
> unevictable:10700kB writepending:343392kB present:6027264kB
> managed:5865036kB mlocked:1800kB kernel_stack:6768kB
> pagetables:10876kB bounce:0kB free_pcp:6896kB local_pcp:328kB
> free_cma:0kB
> [126841.385259] lowmem_reserve[]: 0 0 0 0 0
> [126841.385261] Node 0 DMA: 0*4kB 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB
> (U) 1*128kB (U) 1*256kB (U) 1*512kB (U) 1*1024kB (U) 2*2048kB (UM)
> 2*4096kB (M) = 14320kB
> [126841.385269] Node 0 DMA32: 1*4kB (E) 3*8kB (UME) 2*16kB (UE) 3*32kB
> (UME) 7*64kB (E) 3*128kB (UE) 6*256kB (ME) 5*512kB (ME) 2*1024kB (UM)
> 2*2048kB (UM) 7*4096kB (M) = 39900kB
> [126841.385277] Node 0 Normal: 366*4kB (UME) 413*8kB (UME) 76*16kB
> (UME) 18*32kB (UME) 20*64kB (UME) 16*128kB (UME) 10*256kB (UME)
> 8*512kB (ME) 5*1024kB (UE) 2*2048kB (UM) 6*4096kB (M) = 50336kB
> [126841.385289] Node 0 hugepages_total=0 hugepages_free=0
> hugepages_surp=0 hugepages_size=2048kB
> [126841.385290] 1793872 total pagecache pages
> [126841.385294] 45 pages in swap cache
> [126841.385296] Swap cache stats: add 14529, delete 14484, find 3855/4431
> [126841.385297] Free swap = 14869404kB
> [126841.385299] Total swap = 14892188kB
> [126841.385300] 2074779 pages RAM
> [126841.385301] 0 pages HighMem/MovableOnly
> [126841.385302] 69453 pages reserved
> [126841.385304] 0 pages cma reserved
> [126841.385305] 0 pages hwpoisoned
> [126841.385307] SLUB: Unable to allocate memory on node -1,
> gfp=0x800(GFP_NOWAIT)
> [126841.385309] cache: radix_tree_node, object size: 576, buffer
> size: 584, default order: 2, min order: 0
> [126841.385311] node 0: slabs: 392, objs: 10976, free: 0
> [126841.385312] DMA-API: cacheline tracking ENOMEM, dma-debug disabled
>
>
> Entire dmesg:
> https://bugzilla.redhat.com/attachment.cgi?id=1699909
> Downstream bug (same as above)
> https://bugzilla.redhat.com/show_bug.cgi?id=1853795
>
>
>
>
> --
> Chris Murphy
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 5.8.0rc3 page allocation failure: order:0, mode:0x40810
2020-07-04 7:21 5.8.0rc3 page allocation failure: order:0, mode:0x40810 Chris Murphy
2020-07-04 8:02 ` Pekka Enberg
@ 2020-07-04 11:37 ` Matthew Wilcox
2020-07-04 11:49 ` Pekka Enberg
1 sibling, 1 reply; 6+ messages in thread
From: Matthew Wilcox @ 2020-07-04 11:37 UTC (permalink / raw)
To: Chris Murphy; +Cc: Linux Memory Management List, Dan Williams
On Sat, Jul 04, 2020 at 01:21:05AM -0600, Chris Murphy wrote:
> Hi,
>
> This seems to be a one off - I'm not even sure what was happening at
> the time. Possibly running a qemu-kvm VM.
>
> 5.8.0-0.rc3.1.fc33.x86_64+debug
> i7-2820QM
> MemTotal: 8021304 kB
> $ swapon
> NAME TYPE SIZE USED PRIO
> /dev/sda5 partition 10.4G 0B -2
> /dev/zram0 partition 3.8G 9M 100
> $ zramctl
> NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
> /dev/zram0 lzo-rle 3.8G 7.8M 2.3M 4.5M 8 [SWAP]
>
>
>
>
> [126841.385050] swapper/2: page allocation failure: order:0,
> mode:0x40810(GFP_NOWAIT|__GFP_COMP|__GFP_RECLAIMABLE),
This is one for Dan; he decided to use GFP_NOWAIT in commit
0abdd7a81b7e3fd781d7fabcca49501852bba17e
> nodemask=(null),cpuset=/,mems_allowed=0
> [126841.385059] CPU: 2 PID: 0 Comm: swapper/2 Not tainted
> 5.8.0-0.rc3.1.fc33.x86_64+debug #1
> [126841.385061] Hardware name: Apple Inc.
> MacBookPro8,2/Mac-94245A3940C91C80, BIOS
> MBP81.88Z.0050.B00.1804101331 04/10/18
> [126841.385062] Call Trace:
> [126841.385064] <IRQ>
> [126841.385071] dump_stack+0x92/0xc8
> [126841.385076] warn_alloc.cold+0x75/0xd9
> [126841.385083] __alloc_pages_slowpath.constprop.0+0x10ce/0x1110
> [126841.385090] ? lock_acquire+0xb0/0x400
> [126841.385095] __alloc_pages_nodemask+0x3c4/0x450
> [126841.385099] alloc_slab_page+0x345/0x790
> [126841.385103] allocate_slab+0x473/0x570
> [126841.385106] ___slab_alloc+0x4bb/0x7d0
> [126841.385109] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
> [126841.385115] ? __module_address.part.0+0xe/0xe0
> [126841.385117] ? __module_address.part.0+0xe/0xe0
> [126841.385120] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
> [126841.385122] ? __slab_alloc+0x46/0x90
> [126841.385123] __slab_alloc+0x46/0x90
> [126841.385126] ? radix_tree_node_alloc.constprop.0+0x44/0xe0
> [126841.385128] kmem_cache_alloc+0x2c8/0x310
> [126841.385131] radix_tree_node_alloc.constprop.0+0x44/0xe0
> [126841.385134] radix_tree_insert+0xd1/0x1b0
> [126841.385139] add_dma_entry+0xb4/0x1a0
> [126841.385145] dma_map_single_attrs.constprop.0+0xcb/0x150 [tg3]
> [126841.385149] tg3_alloc_rx_data+0xaa/0x190 [tg3]
> [126841.385153] tg3_poll_work+0x1d7/0x10b0 [tg3]
> [126841.385160] tg3_poll_msix+0x39/0x1c0 [tg3]
> [126841.385164] net_rx_action+0x150/0x4f0
> [126841.385170] __do_softirq+0xdb/0x4a2
> [126841.385175] asm_call_on_stack+0xf/0x20
> [126841.385176] </IRQ>
> [126841.385179] do_softirq_own_stack+0x80/0x90
> [126841.385182] irq_exit_rcu+0xe0/0x120
> [126841.385185] common_interrupt+0xaa/0x210
> [126841.385188] asm_common_interrupt+0x1e/0x40
> [126841.385191] RIP: 0010:cpuidle_enter_state+0xb8/0x440
> [126841.385194] Code: 69 6b e8 eb 04 7b ff 49 89 c7 66 66 66 66 90 31
> ff e8 bc 27 7b ff 80 7c 24 0f 00 0f 85 0d 02 00 00 e8 3c 3c 8b ff fb
> 66 66 90 <66> 66 90 45 85 e4 0f 88 14 02 00 00 49 63 d4 4c 2b 7c 24 10
> 48 8d
> [126841.385196] RSP: 0018:ffffab2f0009fe80 EFLAGS: 00000246
> [126841.385198] RAX: 0000000000000000 RBX: ffffcb2eff201400 RCX:
> 000000000000001f
> [126841.385199] RDX: 0000000000000000 RSI: 0000000037a6deab RDI:
> ffffffff949749b4
> [126841.385201] RBP: ffffffff95c18e80 R08: 0000735c8fb2993f R09:
> 0000000000000008
> [126841.385202] R10: 0000000000000000 R11: 0000000000000016 R12:
> 0000000000000002
> [126841.385204] R13: ffffffff95c18e80 R14: 0000000000000002 R15:
> 0000735c8fb2993f
> [126841.385208] ? cpuidle_enter_state+0xb4/0x440
> [126841.385213] cpuidle_enter+0x29/0x40
> [126841.385217] do_idle+0x1f6/0x2e0
> [126841.385220] cpu_startup_entry+0x19/0x20
> [126841.385224] start_secondary+0x15c/0x190
> [126841.385227] secondary_startup_64+0xb6/0xc0
> [126841.385233] Mem-Info:
> [126841.385237] active_anon:141209 inactive_anon:596767 isolated_anon:0
> active_file:122875 inactive_file:993681 isolated_file:9
> unevictable:2594 dirty:110075 writeback:0
> slab_reclaimable:42804 slab_unreclaimable:46586
> mapped:65915 shmem:676937 pagetables:2719 bounce:0
> free:25946 free_pcp:2811 free_cma:0
> [126841.385241] Node 0 active_anon:564836kB inactive_anon:2387068kB
> active_file:491500kB inactive_file:3974724kB unevictable:10376kB
> isolated(anon):0kB isolated(file):36kB mapped:263660kB dirty:440300kB
> writeback:0kB shmem:2707748kB shmem_thp: 0kB shmem_pmdmapped: 0kB
> anon_thp: 0kB writeback_tmp:0kB all_unreclaimable? no
> [126841.385242] Node 0 DMA free:14320kB min:128kB low:160kB high:192kB
> reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB
> active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB
> present:15988kB managed:15360kB mlocked:0kB kernel_stack:0kB
> pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
> [126841.385246] lowmem_reserve[]: 0 2048 7776 7776 7776
> [126841.385249] Node 0 DMA32 free:40352kB min:17772kB low:22212kB
> high:26652kB reserved_highatomic:0KB active_anon:0kB
> inactive_anon:18908kB active_file:0kB inactive_file:2063284kB
> unevictable:0kB writepending:96256kB present:2255864kB
> managed:2140908kB mlocked:0kB kernel_stack:0kB pagetables:0kB
> bounce:0kB free_pcp:4188kB local_pcp:36kB free_cma:0kB
> [126841.385253] lowmem_reserve[]: 0 0 5727 5727 5727
> [126841.385255] Node 0 Normal free:49112kB min:49680kB low:62100kB
> high:74520kB reserved_highatomic:0KB active_anon:564748kB
> inactive_anon:2368400kB active_file:491756kB inactive_file:1911480kB
> unevictable:10700kB writepending:343392kB present:6027264kB
> managed:5865036kB mlocked:1800kB kernel_stack:6768kB
> pagetables:10876kB bounce:0kB free_pcp:6896kB local_pcp:328kB
> free_cma:0kB
> [126841.385259] lowmem_reserve[]: 0 0 0 0 0
> [126841.385261] Node 0 DMA: 0*4kB 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB
> (U) 1*128kB (U) 1*256kB (U) 1*512kB (U) 1*1024kB (U) 2*2048kB (UM)
> 2*4096kB (M) = 14320kB
> [126841.385269] Node 0 DMA32: 1*4kB (E) 3*8kB (UME) 2*16kB (UE) 3*32kB
> (UME) 7*64kB (E) 3*128kB (UE) 6*256kB (ME) 5*512kB (ME) 2*1024kB (UM)
> 2*2048kB (UM) 7*4096kB (M) = 39900kB
> [126841.385277] Node 0 Normal: 366*4kB (UME) 413*8kB (UME) 76*16kB
> (UME) 18*32kB (UME) 20*64kB (UME) 16*128kB (UME) 10*256kB (UME)
> 8*512kB (ME) 5*1024kB (UE) 2*2048kB (UM) 6*4096kB (M) = 50336kB
> [126841.385289] Node 0 hugepages_total=0 hugepages_free=0
> hugepages_surp=0 hugepages_size=2048kB
> [126841.385290] 1793872 total pagecache pages
> [126841.385294] 45 pages in swap cache
> [126841.385296] Swap cache stats: add 14529, delete 14484, find 3855/4431
> [126841.385297] Free swap = 14869404kB
> [126841.385299] Total swap = 14892188kB
> [126841.385300] 2074779 pages RAM
> [126841.385301] 0 pages HighMem/MovableOnly
> [126841.385302] 69453 pages reserved
> [126841.385304] 0 pages cma reserved
> [126841.385305] 0 pages hwpoisoned
> [126841.385307] SLUB: Unable to allocate memory on node -1,
> gfp=0x800(GFP_NOWAIT)
> [126841.385309] cache: radix_tree_node, object size: 576, buffer
> size: 584, default order: 2, min order: 0
> [126841.385311] node 0: slabs: 392, objs: 10976, free: 0
> [126841.385312] DMA-API: cacheline tracking ENOMEM, dma-debug disabled
>
>
> Entire dmesg:
> https://bugzilla.redhat.com/attachment.cgi?id=1699909
> Downstream bug (same as above)
> https://bugzilla.redhat.com/show_bug.cgi?id=1853795
>
>
>
>
> --
> Chris Murphy
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 5.8.0rc3 page allocation failure: order:0, mode:0x40810
2020-07-04 11:37 ` Matthew Wilcox
@ 2020-07-04 11:49 ` Pekka Enberg
2020-07-04 12:01 ` Matthew Wilcox
0 siblings, 1 reply; 6+ messages in thread
From: Pekka Enberg @ 2020-07-04 11:49 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: Chris Murphy, Linux Memory Management List, Dan Williams
Hi Matthew,
On Sat, Jul 4, 2020 at 2:37 PM Matthew Wilcox <willy@infradead.org> wrote:
>
> On Sat, Jul 04, 2020 at 01:21:05AM -0600, Chris Murphy wrote:
> > Hi,
> >
> > This seems to be a one off - I'm not even sure what was happening at
> > the time. Possibly running a qemu-kvm VM.
> >
> > 5.8.0-0.rc3.1.fc33.x86_64+debug
> > i7-2820QM
> > MemTotal: 8021304 kB
> > $ swapon
> > NAME TYPE SIZE USED PRIO
> > /dev/sda5 partition 10.4G 0B -2
> > /dev/zram0 partition 3.8G 9M 100
> > $ zramctl
> > NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
> > /dev/zram0 lzo-rle 3.8G 7.8M 2.3M 4.5M 8 [SWAP]
> >
> >
> >
> >
> > [126841.385050] swapper/2: page allocation failure: order:0,
> > mode:0x40810(GFP_NOWAIT|__GFP_COMP|__GFP_RECLAIMABLE),
>
> This is one for Dan; he decided to use GFP_NOWAIT in commit
> 0abdd7a81b7e3fd781d7fabcca49501852bba17e
Why is that a problem? Unless I am missing something, there's plenty
of free pages available for the slab allocation:
> > [126841.385277] Node 0 Normal: 366*4kB (UME) 413*8kB (UME) 76*16kB
> > (UME) 18*32kB (UME) 20*64kB (UME) 16*128kB (UME) 10*256kB (UME)
> > 8*512kB (ME) 5*1024kB (UE) 2*2048kB (UM) 6*4096kB (M) = 50336kB
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 5.8.0rc3 page allocation failure: order:0, mode:0x40810
2020-07-04 11:49 ` Pekka Enberg
@ 2020-07-04 12:01 ` Matthew Wilcox
2020-07-04 18:33 ` Pekka Enberg
0 siblings, 1 reply; 6+ messages in thread
From: Matthew Wilcox @ 2020-07-04 12:01 UTC (permalink / raw)
To: Pekka Enberg
Cc: Chris Murphy, Linux Memory Management List, Dan Williams, Mike Kravetz
On Sat, Jul 04, 2020 at 02:49:32PM +0300, Pekka Enberg wrote:
> Hi Matthew,
>
> On Sat, Jul 4, 2020 at 2:37 PM Matthew Wilcox <willy@infradead.org> wrote:
> >
> > On Sat, Jul 04, 2020 at 01:21:05AM -0600, Chris Murphy wrote:
> > > Hi,
> > >
> > > This seems to be a one off - I'm not even sure what was happening at
> > > the time. Possibly running a qemu-kvm VM.
> > >
> > > 5.8.0-0.rc3.1.fc33.x86_64+debug
> > > i7-2820QM
> > > MemTotal: 8021304 kB
> > > $ swapon
> > > NAME TYPE SIZE USED PRIO
> > > /dev/sda5 partition 10.4G 0B -2
> > > /dev/zram0 partition 3.8G 9M 100
> > > $ zramctl
> > > NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
> > > /dev/zram0 lzo-rle 3.8G 7.8M 2.3M 4.5M 8 [SWAP]
> > >
> > >
> > >
> > >
> > > [126841.385050] swapper/2: page allocation failure: order:0,
> > > mode:0x40810(GFP_NOWAIT|__GFP_COMP|__GFP_RECLAIMABLE),
> >
> > This is one for Dan; he decided to use GFP_NOWAIT in commit
> > 0abdd7a81b7e3fd781d7fabcca49501852bba17e
>
> Why is that a problem? Unless I am missing something, there's plenty
> of free pages available for the slab allocation:
>
> > > [126841.385277] Node 0 Normal: 366*4kB (UME) 413*8kB (UME) 76*16kB
> > > (UME) 18*32kB (UME) 20*64kB (UME) 16*128kB (UME) 10*256kB (UME)
> > > 8*512kB (ME) 5*1024kB (UE) 2*2048kB (UM) 6*4096kB (M) = 50336kB
One of the things we (mostly Mike Kravetz) found recently when debugging
a memory allocation stall issue is that this message does not capture
the state which actually caused the message to be printed. In the
intervening time between the memory allocation failure being detected
and this message being printed, kswapd may have made forward progress.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 5.8.0rc3 page allocation failure: order:0, mode:0x40810
2020-07-04 12:01 ` Matthew Wilcox
@ 2020-07-04 18:33 ` Pekka Enberg
0 siblings, 0 replies; 6+ messages in thread
From: Pekka Enberg @ 2020-07-04 18:33 UTC (permalink / raw)
To: Matthew Wilcox
Cc: Chris Murphy, Linux Memory Management List, Dan Williams, Mike Kravetz
Hi Matthew,
On Sat, Jul 4, 2020 at 3:01 PM Matthew Wilcox <willy@infradead.org> wrote:
> One of the things we (mostly Mike Kravetz) found recently when debugging
> a memory allocation stall issue is that this message does not capture
> the state which actually caused the message to be printed. In the
> intervening time between the memory allocation failure being detected
> and this message being printed, kswapd may have made forward progress.
Oh, right, makes sense. Worth fixing, of course, but probably not so easy to do.
- Pekka
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-07-04 18:34 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-04 7:21 5.8.0rc3 page allocation failure: order:0, mode:0x40810 Chris Murphy
2020-07-04 8:02 ` Pekka Enberg
2020-07-04 11:37 ` Matthew Wilcox
2020-07-04 11:49 ` Pekka Enberg
2020-07-04 12:01 ` Matthew Wilcox
2020-07-04 18:33 ` Pekka Enberg
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.