* page allocator bug in 3.16?
@ 2014-09-25 18:55 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 18:55 UTC (permalink / raw)
To: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom
Cc: Andrew Morton, Linus Torvalds, Linux kernel, linux-mm,
Ingo Molnar, Hugh Dickens, dri-devel
After several days uptime with a 3.16 kernel (generally running
Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
desktop workspaces) I've been seeing some really extreme slowdowns.
Mostly the slowdowns are associated with gpu-related tasks, like
opening new emacs windows, switching workspaces, laughing at internet
gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
it right away -- 3.15 is the first time suspend has worked reliably.
This week I started looking into what the slowdown was and discovered
it's happening during dma allocation through swiotlb (the cpus can do
intel iommu but I don't use it because it's not the default for most users).
I'm still working on a bisection but each step takes 8+ hours to
validate and even then I'm no longer sure I still have the 'bad'
commit in the bisection. [edit: yup, I started over]
I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
fragmentation [1].
Besides Mel's page allocator changes in 3.16, another suspect commit is:
commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
Author: Shaohua Li <shli@kernel.org>
Date: Tue Apr 8 15:58:09 2014 +0800
x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
Specifically, this statement:
It could cause incorrect page aging and the (mistaken) reclaim of
hot pages, but the chance of that should be relatively low.
I'm wondering if this could cause worse-case behavior with TTM? I'm
testing a revert of this on mainline 3.16-final now, with no results yet.
Thoughts?
Regards,
Peter Hurley
[1]
SysRq : Show Memory
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 18
CPU 1: hi: 186, btch: 31 usd: 82
CPU 2: hi: 186, btch: 31 usd: 46
CPU 3: hi: 186, btch: 31 usd: 30
CPU 4: hi: 186, btch: 31 usd: 18
CPU 5: hi: 186, btch: 31 usd: 43
CPU 6: hi: 186, btch: 31 usd: 157
CPU 7: hi: 186, btch: 31 usd: 26
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 25
CPU 1: hi: 186, btch: 31 usd: 33
CPU 2: hi: 186, btch: 31 usd: 28
CPU 3: hi: 186, btch: 31 usd: 46
CPU 4: hi: 186, btch: 31 usd: 23
CPU 5: hi: 186, btch: 31 usd: 8
CPU 6: hi: 186, btch: 31 usd: 112
CPU 7: hi: 186, btch: 31 usd: 18
active_anon:382833 inactive_anon:12103 isolated_anon:0
active_file:1156997 inactive_file:733988 isolated_file:0
unevictable:15 dirty:35833 writeback:0 unstable:0
free:129383 slab_reclaimable:95038 slab_unreclaimable:11095
mapped:81924 shmem:12509 pagetables:9039 bounce:0
free_cma:0
Node 0 DMA free:15860kB min:104kB low:128kB high:156kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15960kB managed:15876kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2974 9980 9980
Node 0 DMA32 free:166712kB min:20108kB low:25132kB high:30160kB active_anon:475548kB inactive_anon:15204kB active_file:1368716kB inactive_file:865832kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3127336kB managed:3048188kB mlocked:0kB dirty:38228kB writeback:0kB mapped:94340kB shmem:15436kB slab_reclaimable:116424kB slab_unreclaimable:12756kB kernel_stack:2512kB pagetables:11532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 7006 7006
Node 0 Normal free:334960kB min:47368kB low:59208kB high:71052kB active_anon:1055784kB inactive_anon:33208kB active_file:3259272kB inactive_file:2070120kB unevictable:60kB isolated(anon):0kB isolated(file):0kB present:7340032kB managed:7174484kB mlocked:60kB dirty:105104kB writeback:0kB mapped:233356kB shmem:34600kB slab_reclaimable:263728kB slab_unreclaimable:31608kB kernel_stack:7344kB pagetables:24624kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB (U) 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15860kB
Node 0 DMA32: 209*4kB (UEM) 394*8kB (UEM) 303*16kB (UEM) 60*32kB (UEM) 314*64kB (UEM) 117*128kB (UEM) 9*256kB (EM) 3*512kB (UEM) 2*1024kB (EM) 2*2048kB (UM) 27*4096kB (MR) = 166404kB
Node 0 Normal: 17*4kB (UE) 460*8kB (UEM) 747*16kB (UM) 130*32kB (UEM) 521*64kB (UM) 184*128kB (UEM) 70*256kB (UM) 22*512kB (UM) 11*1024kB (UM) 2*2048kB (EM) 52*4096kB (MR) = 334292kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
1903443 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 10996456kB
Total swap = 10996456kB
2620832 pages RAM
0 pages HighMem/MovableOnly
41387 pages reserved
0 pages hwpoisoned
^ permalink raw reply [flat|nested] 42+ messages in thread
* page allocator bug in 3.16?
@ 2014-09-25 18:55 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 18:55 UTC (permalink / raw)
To: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom
Cc: Andrew Morton, Linus Torvalds, Linux kernel, linux-mm,
Ingo Molnar, Hugh Dickens, dri-devel
After several days uptime with a 3.16 kernel (generally running
Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
desktop workspaces) I've been seeing some really extreme slowdowns.
Mostly the slowdowns are associated with gpu-related tasks, like
opening new emacs windows, switching workspaces, laughing at internet
gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
it right away -- 3.15 is the first time suspend has worked reliably.
This week I started looking into what the slowdown was and discovered
it's happening during dma allocation through swiotlb (the cpus can do
intel iommu but I don't use it because it's not the default for most users).
I'm still working on a bisection but each step takes 8+ hours to
validate and even then I'm no longer sure I still have the 'bad'
commit in the bisection. [edit: yup, I started over]
I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
fragmentation [1].
Besides Mel's page allocator changes in 3.16, another suspect commit is:
commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
Author: Shaohua Li <shli@kernel.org>
Date: Tue Apr 8 15:58:09 2014 +0800
x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
Specifically, this statement:
It could cause incorrect page aging and the (mistaken) reclaim of
hot pages, but the chance of that should be relatively low.
I'm wondering if this could cause worse-case behavior with TTM? I'm
testing a revert of this on mainline 3.16-final now, with no results yet.
Thoughts?
Regards,
Peter Hurley
[1]
SysRq : Show Memory
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 18
CPU 1: hi: 186, btch: 31 usd: 82
CPU 2: hi: 186, btch: 31 usd: 46
CPU 3: hi: 186, btch: 31 usd: 30
CPU 4: hi: 186, btch: 31 usd: 18
CPU 5: hi: 186, btch: 31 usd: 43
CPU 6: hi: 186, btch: 31 usd: 157
CPU 7: hi: 186, btch: 31 usd: 26
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 25
CPU 1: hi: 186, btch: 31 usd: 33
CPU 2: hi: 186, btch: 31 usd: 28
CPU 3: hi: 186, btch: 31 usd: 46
CPU 4: hi: 186, btch: 31 usd: 23
CPU 5: hi: 186, btch: 31 usd: 8
CPU 6: hi: 186, btch: 31 usd: 112
CPU 7: hi: 186, btch: 31 usd: 18
active_anon:382833 inactive_anon:12103 isolated_anon:0
active_file:1156997 inactive_file:733988 isolated_file:0
unevictable:15 dirty:35833 writeback:0 unstable:0
free:129383 slab_reclaimable:95038 slab_unreclaimable:11095
mapped:81924 shmem:12509 pagetables:9039 bounce:0
free_cma:0
Node 0 DMA free:15860kB min:104kB low:128kB high:156kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15960kB managed:15876kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2974 9980 9980
Node 0 DMA32 free:166712kB min:20108kB low:25132kB high:30160kB active_anon:475548kB inactive_anon:15204kB active_file:1368716kB inactive_file:865832kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3127336kB managed:3048188kB mlocked:0kB dirty:38228kB writeback:0kB mapped:94340kB shmem:15436kB slab_reclaimable:116424kB slab_unreclaimable:12756kB kernel_stack:2512kB pagetables:11532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 7006 7006
Node 0 Normal free:334960kB min:47368kB low:59208kB high:71052kB active_anon:1055784kB inactive_anon:33208kB active_file:3259272kB inactive_file:2070120kB unevictable:60kB isolated(anon):0kB isolated(file):0kB present:7340032kB managed:7174484kB mlocked:60kB dirty:105104kB writeback:0kB mapped:233356kB shmem:34600kB slab_reclaimable:263728kB slab_unreclaimable:31608kB kernel_stack:7344kB pagetables:24624kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB (U) 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15860kB
Node 0 DMA32: 209*4kB (UEM) 394*8kB (UEM) 303*16kB (UEM) 60*32kB (UEM) 314*64kB (UEM) 117*128kB (UEM) 9*256kB (EM) 3*512kB (UEM) 2*1024kB (EM) 2*2048kB (UM) 27*4096kB (MR) = 166404kB
Node 0 Normal: 17*4kB (UE) 460*8kB (UEM) 747*16kB (UM) 130*32kB (UEM) 521*64kB (UM) 184*128kB (UEM) 70*256kB (UM) 22*512kB (UM) 11*1024kB (UM) 2*2048kB (EM) 52*4096kB (MR) = 334292kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
1903443 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 10996456kB
Total swap = 10996456kB
2620832 pages RAM
0 pages HighMem/MovableOnly
41387 pages reserved
0 pages hwpoisoned
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* page allocator bug in 3.16?
@ 2014-09-25 18:55 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 18:55 UTC (permalink / raw)
To: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom
Cc: Hugh Dickens, Linux kernel, dri-devel, linux-mm, Andrew Morton,
Linus Torvalds, Ingo Molnar
After several days uptime with a 3.16 kernel (generally running
Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
desktop workspaces) I've been seeing some really extreme slowdowns.
Mostly the slowdowns are associated with gpu-related tasks, like
opening new emacs windows, switching workspaces, laughing at internet
gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
it right away -- 3.15 is the first time suspend has worked reliably.
This week I started looking into what the slowdown was and discovered
it's happening during dma allocation through swiotlb (the cpus can do
intel iommu but I don't use it because it's not the default for most users).
I'm still working on a bisection but each step takes 8+ hours to
validate and even then I'm no longer sure I still have the 'bad'
commit in the bisection. [edit: yup, I started over]
I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
fragmentation [1].
Besides Mel's page allocator changes in 3.16, another suspect commit is:
commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
Author: Shaohua Li <shli@kernel.org>
Date: Tue Apr 8 15:58:09 2014 +0800
x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
Specifically, this statement:
It could cause incorrect page aging and the (mistaken) reclaim of
hot pages, but the chance of that should be relatively low.
I'm wondering if this could cause worse-case behavior with TTM? I'm
testing a revert of this on mainline 3.16-final now, with no results yet.
Thoughts?
Regards,
Peter Hurley
[1]
SysRq : Show Memory
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 18
CPU 1: hi: 186, btch: 31 usd: 82
CPU 2: hi: 186, btch: 31 usd: 46
CPU 3: hi: 186, btch: 31 usd: 30
CPU 4: hi: 186, btch: 31 usd: 18
CPU 5: hi: 186, btch: 31 usd: 43
CPU 6: hi: 186, btch: 31 usd: 157
CPU 7: hi: 186, btch: 31 usd: 26
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 25
CPU 1: hi: 186, btch: 31 usd: 33
CPU 2: hi: 186, btch: 31 usd: 28
CPU 3: hi: 186, btch: 31 usd: 46
CPU 4: hi: 186, btch: 31 usd: 23
CPU 5: hi: 186, btch: 31 usd: 8
CPU 6: hi: 186, btch: 31 usd: 112
CPU 7: hi: 186, btch: 31 usd: 18
active_anon:382833 inactive_anon:12103 isolated_anon:0
active_file:1156997 inactive_file:733988 isolated_file:0
unevictable:15 dirty:35833 writeback:0 unstable:0
free:129383 slab_reclaimable:95038 slab_unreclaimable:11095
mapped:81924 shmem:12509 pagetables:9039 bounce:0
free_cma:0
Node 0 DMA free:15860kB min:104kB low:128kB high:156kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15960kB managed:15876kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2974 9980 9980
Node 0 DMA32 free:166712kB min:20108kB low:25132kB high:30160kB active_anon:475548kB inactive_anon:15204kB active_file:1368716kB inactive_file:865832kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3127336kB managed:3048188kB mlocked:0kB dirty:38228kB writeback:0kB mapped:94340kB shmem:15436kB slab_reclaimable:116424kB slab_unreclaimable:12756kB kernel_stack:2512kB pagetables:11532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 7006 7006
Node 0 Normal free:334960kB min:47368kB low:59208kB high:71052kB active_anon:1055784kB inactive_anon:33208kB active_file:3259272kB inactive_file:2070120kB unevictable:60kB isolated(anon):0kB isolated(file):0kB present:7340032kB managed:7174484kB mlocked:60kB dirty:105104kB writeback:0kB mapped:233356kB shmem:34600kB slab_reclaimable:263728kB slab_unreclaimable:31608kB kernel_stack:7344kB pagetables:24624kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB (U) 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15860kB
Node 0 DMA32: 209*4kB (UEM) 394*8kB (UEM) 303*16kB (UEM) 60*32kB (UEM) 314*64kB (UEM) 117*128kB (UEM) 9*256kB (EM) 3*512kB (UEM) 2*1024kB (EM) 2*2048kB (UM) 27*4096kB (MR) = 166404kB
Node 0 Normal: 17*4kB (UE) 460*8kB (UEM) 747*16kB (UM) 130*32kB (UEM) 521*64kB (UM) 184*128kB (UEM) 70*256kB (UM) 22*512kB (UM) 11*1024kB (UM) 2*2048kB (EM) 52*4096kB (MR) = 334292kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
1903443 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 10996456kB
Total swap = 10996456kB
2620832 pages RAM
0 pages HighMem/MovableOnly
41387 pages reserved
0 pages hwpoisoned
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-25 18:55 ` Peter Hurley
(?)
@ 2014-09-25 19:34 ` Maarten Lankhorst
-1 siblings, 0 replies; 42+ messages in thread
From: Maarten Lankhorst @ 2014-09-25 19:34 UTC (permalink / raw)
To: Peter Hurley, Mel Gorman, Shaohua Li, Rik van Riel, Thomas Hellstrom
Cc: Andrew Morton, Linus Torvalds, Linux kernel, linux-mm,
Ingo Molnar, Hugh Dickens, dri-devel
Hey,
On 25-09-14 20:55, Peter Hurley wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>
> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
> fragmentation [1].
>
> Besides Mel's page allocator changes in 3.16, another suspect commit is:
Maybe related, but I've been seeing page corruption in nouveau as well, with 3.15.9:
http://paste.debian.net/122800/
I think it might be an even older bug because I've been using nouveau on my desktop and it hasn't been stable for the past few releases. I'm also lazy with updating kernel, still do it from time to time.
The lookup and nvapeek warnings/crashes are not important btw, I was testing some nouveau things.
The linker trap probably is. After the second BUG Xorg was no longer able to recover.
But this was after various suspend/resume cycles, although I suspect I've hit some corruption on radeon too (on a somewhat more recent kernel) when I fiddle with vgaswitcheroo, ending up with a real massive amount of spam there, etc.
Unfortunately I haven't been able to find out what caused it yet, nor am I sure what debug options I should set in the kernel to debug this.
~Maarten
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-25 19:34 ` Maarten Lankhorst
0 siblings, 0 replies; 42+ messages in thread
From: Maarten Lankhorst @ 2014-09-25 19:34 UTC (permalink / raw)
To: Peter Hurley, Mel Gorman, Shaohua Li, Rik van Riel, Thomas Hellstrom
Cc: Andrew Morton, Linus Torvalds, Linux kernel, linux-mm,
Ingo Molnar, Hugh Dickens, dri-devel
Hey,
On 25-09-14 20:55, Peter Hurley wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>
> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
> fragmentation [1].
>
> Besides Mel's page allocator changes in 3.16, another suspect commit is:
Maybe related, but I've been seeing page corruption in nouveau as well, with 3.15.9:
http://paste.debian.net/122800/
I think it might be an even older bug because I've been using nouveau on my desktop and it hasn't been stable for the past few releases. I'm also lazy with updating kernel, still do it from time to time.
The lookup and nvapeek warnings/crashes are not important btw, I was testing some nouveau things.
The linker trap probably is. After the second BUG Xorg was no longer able to recover.
But this was after various suspend/resume cycles, although I suspect I've hit some corruption on radeon too (on a somewhat more recent kernel) when I fiddle with vgaswitcheroo, ending up with a real massive amount of spam there, etc.
Unfortunately I haven't been able to find out what caused it yet, nor am I sure what debug options I should set in the kernel to debug this.
~Maarten
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-25 19:34 ` Maarten Lankhorst
0 siblings, 0 replies; 42+ messages in thread
From: Maarten Lankhorst @ 2014-09-25 19:34 UTC (permalink / raw)
To: Peter Hurley, Mel Gorman, Shaohua Li, Rik van Riel, Thomas Hellstrom
Cc: Hugh Dickens, Linux kernel, dri-devel, linux-mm, Andrew Morton,
Linus Torvalds, Ingo Molnar
Hey,
On 25-09-14 20:55, Peter Hurley wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>
> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
> fragmentation [1].
>
> Besides Mel's page allocator changes in 3.16, another suspect commit is:
Maybe related, but I've been seeing page corruption in nouveau as well, with 3.15.9:
http://paste.debian.net/122800/
I think it might be an even older bug because I've been using nouveau on my desktop and it hasn't been stable for the past few releases. I'm also lazy with updating kernel, still do it from time to time.
The lookup and nvapeek warnings/crashes are not important btw, I was testing some nouveau things.
The linker trap probably is. After the second BUG Xorg was no longer able to recover.
But this was after various suspend/resume cycles, although I suspect I've hit some corruption on radeon too (on a somewhat more recent kernel) when I fiddle with vgaswitcheroo, ending up with a real massive amount of spam there, etc.
Unfortunately I haven't been able to find out what caused it yet, nor am I sure what debug options I should set in the kernel to debug this.
~Maarten
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-25 18:55 ` Peter Hurley
@ 2014-09-25 19:35 ` Chuck Ebbert
-1 siblings, 0 replies; 42+ messages in thread
From: Chuck Ebbert @ 2014-09-25 19:35 UTC (permalink / raw)
To: Peter Hurley
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On Thu, 25 Sep 2014 14:55:02 -0400
Peter Hurley <peter@hurleysoftware.com> wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
There are six ttm patches queued for 3.16.4:
drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-25 19:35 ` Chuck Ebbert
0 siblings, 0 replies; 42+ messages in thread
From: Chuck Ebbert @ 2014-09-25 19:35 UTC (permalink / raw)
To: Peter Hurley
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On Thu, 25 Sep 2014 14:55:02 -0400
Peter Hurley <peter@hurleysoftware.com> wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
There are six ttm patches queued for 3.16.4:
drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-25 18:55 ` Peter Hurley
@ 2014-09-25 20:33 ` Alex Deucher
-1 siblings, 0 replies; 42+ messages in thread
From: Alex Deucher @ 2014-09-25 20:33 UTC (permalink / raw)
To: Peter Hurley
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom, Hugh Dickens, Linux kernel, dri-devel,
linux-mm, Andrew Morton, Linus Torvalds, Ingo Molnar
On Thu, Sep 25, 2014 at 2:55 PM, Peter Hurley <peter@hurleysoftware.com> wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>
> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
> fragmentation [1].
>
> Besides Mel's page allocator changes in 3.16, another suspect commit is:
>
> commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
> Author: Shaohua Li <shli@kernel.org>
> Date: Tue Apr 8 15:58:09 2014 +0800
>
> x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
>
> Specifically, this statement:
>
> It could cause incorrect page aging and the (mistaken) reclaim of
> hot pages, but the chance of that should be relatively low.
>
> I'm wondering if this could cause worse-case behavior with TTM? I'm
> testing a revert of this on mainline 3.16-final now, with no results yet.
>
> Thoughts?
You may also be seeing this:
https://lkml.org/lkml/2014/8/8/445
Alex
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-25 20:33 ` Alex Deucher
0 siblings, 0 replies; 42+ messages in thread
From: Alex Deucher @ 2014-09-25 20:33 UTC (permalink / raw)
To: Peter Hurley
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom, Hugh Dickens, Linux kernel, dri-devel,
linux-mm, Andrew Morton, Linus Torvalds, Ingo Molnar
On Thu, Sep 25, 2014 at 2:55 PM, Peter Hurley <peter@hurleysoftware.com> wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>
> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
> fragmentation [1].
>
> Besides Mel's page allocator changes in 3.16, another suspect commit is:
>
> commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
> Author: Shaohua Li <shli@kernel.org>
> Date: Tue Apr 8 15:58:09 2014 +0800
>
> x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
>
> Specifically, this statement:
>
> It could cause incorrect page aging and the (mistaken) reclaim of
> hot pages, but the chance of that should be relatively low.
>
> I'm wondering if this could cause worse-case behavior with TTM? I'm
> testing a revert of this on mainline 3.16-final now, with no results yet.
>
> Thoughts?
You may also be seeing this:
https://lkml.org/lkml/2014/8/8/445
Alex
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-25 18:55 ` Peter Hurley
(?)
@ 2014-09-25 20:36 ` Peter Hurley
-1 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 20:36 UTC (permalink / raw)
To: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom
Cc: Andrew Morton, Linus Torvalds, Linux kernel, linux-mm,
Ingo Molnar, Hugh Dickens, dri-devel
On 09/25/2014 02:55 PM, Peter Hurley wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>
> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
> fragmentation [1].
It's swapping, which is crazy because there's 7+GB of file cache [1] which
should be dropped before swapping.
The alloc_contig_range() failure precedes the swapping but not immediately
(44 mins. earlier).
How I reproduce this is to simply do a full distro kernel build.
Skipping the TLB flush is not the problem; the results below are from
3.16-final with that commit reverted.
The slowdown is really obvious because workspace switching redraw takes
multiple seconds to complete (all-cpu perf record of that below [2])
Regards,
Peter Hurley
[1]
SysRq : Show Memory
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 71
CPU 1: hi: 186, btch: 31 usd: 166
CPU 2: hi: 186, btch: 31 usd: 183
CPU 3: hi: 186, btch: 31 usd: 109
CPU 4: hi: 186, btch: 31 usd: 106
CPU 5: hi: 186, btch: 31 usd: 161
CPU 6: hi: 186, btch: 31 usd: 120
CPU 7: hi: 186, btch: 31 usd: 54
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 159
CPU 1: hi: 186, btch: 31 usd: 66
CPU 2: hi: 186, btch: 31 usd: 178
CPU 3: hi: 186, btch: 31 usd: 173
CPU 4: hi: 186, btch: 31 usd: 91
CPU 5: hi: 186, btch: 31 usd: 57
CPU 6: hi: 186, btch: 31 usd: 58
CPU 7: hi: 186, btch: 31 usd: 158
active_anon:170368 inactive_anon:173964 isolated_anon:0
active_file:982209 inactive_file:973911 isolated_file:0
unevictable:15 dirty:15 writeback:1 unstable:0
free:96067 slab_reclaimable:107401 slab_unreclaimable:12572
mapped:58271 shmem:10857 pagetables:9898 bounce:0
free_cma:18
Node 0 DMA free:15860kB min:104kB low:128kB high:156kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15960kB managed:15876kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2974 9980 9980
Node 0 DMA32 free:117740kB min:20108kB low:25132kB high:30160kB active_anon:205232kB inactive_anon:196308kB active_file:1186764kB inactive_file:1173760kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3127336kB managed:3048212kB mlocked:0kB dirty:24kB writeback:4kB mapped:71600kB shmem:8776kB slab_reclaimable:129132kB slab_unreclaimable:13468kB kernel_stack:2864kB pagetables:11536kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 7006 7006
Node 0 Normal free:250668kB min:47368kB low:59208kB high:71052kB active_anon:476240kB inactive_anon:499548kB active_file:2742072kB inactive_file:2721884kB unevictable:60kB isolated(anon):0kB isolated(file):0kB present:7340032kB managed:7174484kB mlocked:60kB dirty:36kB writeback:0kB mapped:161484kB shmem:34652kB slab_reclaimable:300472kB slab_unreclaimable:36804kB kernel_stack:7232kB pagetables:28056kB unstable:0kB bounce:0kB free_cma:72kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB (U) 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15860kB
Node 0 DMA32: 4099*4kB (UEM) 4372*8kB (UEM) 668*16kB (UEM) 294*32kB (UEM) 47*64kB (UEM) 24*128kB (UEM) 19*256kB (UM) 5*512kB (UM) 0*1024kB 6*2048kB (M) 5*4096kB (M) = 117740kB
Node 0 Normal: 22224*4kB (UEMC) 8120*8kB (UEMC) 1594*16kB (UEMC) 301*32kB (UEMC) 154*64kB (UMC) 106*128kB (UEMC) 86*256kB (UMC) 13*512kB (UEMC) 3*1024kB (M) 3*2048kB (M) 1*4096kB (R) = 254400kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
1967616 total pagecache pages
637 pages in swap cache
Swap cache stats: add 9593, delete 8956, find 2162/2929
Free swap = 10966116kB
Total swap = 10996456kB
2620832 pages RAM
0 pages HighMem/MovableOnly
41387 pages reserved
0 pages hwpoisoned
[2] 'perf record -a -g' while workspace switching
Samples: 27K of event 'cycles', Event count (approx.): 15793770075
+ 87.11% 0.00% Xorg [kernel.kallsyms] [k] tracesys
+ 86.81% 0.00% Xorg [unknown] [k] 0x00007fca2df34e77
+ 86.77% 0.00% Xorg [kernel.kallsyms] [k] sys_ioctl
+ 86.77% 0.00% Xorg [kernel.kallsyms] [k] do_vfs_ioctl
+ 86.77% 0.00% Xorg [nouveau] [k] nouveau_drm_ioctl
+ 86.77% 0.00% Xorg [drm] [k] drm_ioctl
+ 86.66% 0.00% Xorg [nouveau] [k] nouveau_gem_ioctl_new
+ 86.50% 0.00% Xorg [nouveau] [k] nouveau_gem_new
+ 86.49% 0.00% Xorg [nouveau] [k] nouveau_bo_new
+ 86.48% 0.00% Xorg [ttm] [k] ttm_bo_init
+ 86.47% 0.00% Xorg [ttm] [k] ttm_bo_validate
+ 86.46% 0.00% Xorg [ttm] [k] ttm_bo_handle_move_mem
+ 86.45% 0.00% Xorg [ttm] [k] ttm_tt_bind
+ 86.45% 0.00% Xorg [nouveau] [k] nouveau_ttm_tt_populate
+ 86.45% 0.00% Xorg [ttm] [k] ttm_dma_populate
+ 86.43% 0.01% Xorg [ttm] [k] ttm_dma_pool_alloc_new_pages
+ 86.42% 0.00% Xorg [kernel.kallsyms] [k] x86_swiotlb_alloc_coherent
+ 86.37% 0.00% Xorg [kernel.kallsyms] [k] dma_generic_alloc_coherent
+ 86.19% 0.00% Xorg [unknown] [k] 0x0000000000c00000
+ 85.82% 0.31% Xorg [kernel.kallsyms] [k] dma_alloc_from_contiguous
+ 84.21% 1.05% Xorg [kernel.kallsyms] [k] alloc_contig_range
+ 46.56% 46.56% Xorg [kernel.kallsyms] [k] move_freepages
+ 46.53% 0.29% Xorg [kernel.kallsyms] [k] move_freepages_block
+ 39.78% 0.13% Xorg [kernel.kallsyms] [k] start_isolate_page_range
+ 39.22% 0.40% Xorg [kernel.kallsyms] [k] set_migratetype_isolate
+ 27.26% 0.17% Xorg [kernel.kallsyms] [k] undo_isolate_page_range
+ 26.62% 0.33% Xorg [kernel.kallsyms] [k] unset_migratetype_isolate
+ 15.54% 7.93% Xorg [kernel.kallsyms] [k] drain_all_pages
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-25 20:36 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 20:36 UTC (permalink / raw)
To: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom
Cc: Andrew Morton, Linus Torvalds, Linux kernel, linux-mm,
Ingo Molnar, Hugh Dickens, dri-devel
On 09/25/2014 02:55 PM, Peter Hurley wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>
> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
> fragmentation [1].
It's swapping, which is crazy because there's 7+GB of file cache [1] which
should be dropped before swapping.
The alloc_contig_range() failure precedes the swapping but not immediately
(44 mins. earlier).
How I reproduce this is to simply do a full distro kernel build.
Skipping the TLB flush is not the problem; the results below are from
3.16-final with that commit reverted.
The slowdown is really obvious because workspace switching redraw takes
multiple seconds to complete (all-cpu perf record of that below [2])
Regards,
Peter Hurley
[1]
SysRq : Show Memory
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 71
CPU 1: hi: 186, btch: 31 usd: 166
CPU 2: hi: 186, btch: 31 usd: 183
CPU 3: hi: 186, btch: 31 usd: 109
CPU 4: hi: 186, btch: 31 usd: 106
CPU 5: hi: 186, btch: 31 usd: 161
CPU 6: hi: 186, btch: 31 usd: 120
CPU 7: hi: 186, btch: 31 usd: 54
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 159
CPU 1: hi: 186, btch: 31 usd: 66
CPU 2: hi: 186, btch: 31 usd: 178
CPU 3: hi: 186, btch: 31 usd: 173
CPU 4: hi: 186, btch: 31 usd: 91
CPU 5: hi: 186, btch: 31 usd: 57
CPU 6: hi: 186, btch: 31 usd: 58
CPU 7: hi: 186, btch: 31 usd: 158
active_anon:170368 inactive_anon:173964 isolated_anon:0
active_file:982209 inactive_file:973911 isolated_file:0
unevictable:15 dirty:15 writeback:1 unstable:0
free:96067 slab_reclaimable:107401 slab_unreclaimable:12572
mapped:58271 shmem:10857 pagetables:9898 bounce:0
free_cma:18
Node 0 DMA free:15860kB min:104kB low:128kB high:156kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15960kB managed:15876kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2974 9980 9980
Node 0 DMA32 free:117740kB min:20108kB low:25132kB high:30160kB active_anon:205232kB inactive_anon:196308kB active_file:1186764kB inactive_file:1173760kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3127336kB managed:3048212kB mlocked:0kB dirty:24kB writeback:4kB mapped:71600kB shmem:8776kB slab_reclaimable:129132kB slab_unreclaimable:13468kB kernel_stack:2864kB pagetables:11536kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 7006 7006
Node 0 Normal free:250668kB min:47368kB low:59208kB high:71052kB active_anon:476240kB inactive_anon:499548kB active_file:2742072kB inactive_file:2721884kB unevictable:60kB isolated(anon):0kB isolated(file):0kB present:7340032kB managed:7174484kB mlocked:60kB dirty:36kB writeback:0kB mapped:161484kB shmem:34652kB slab_reclaimable:300472kB slab_unreclaimable:36804kB kernel_stack:7232kB pagetables:28056kB unstable:0kB bounce:0kB free_cma:72kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB (U) 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15860kB
Node 0 DMA32: 4099*4kB (UEM) 4372*8kB (UEM) 668*16kB (UEM) 294*32kB (UEM) 47*64kB (UEM) 24*128kB (UEM) 19*256kB (UM) 5*512kB (UM) 0*1024kB 6*2048kB (M) 5*4096kB (M) = 117740kB
Node 0 Normal: 22224*4kB (UEMC) 8120*8kB (UEMC) 1594*16kB (UEMC) 301*32kB (UEMC) 154*64kB (UMC) 106*128kB (UEMC) 86*256kB (UMC) 13*512kB (UEMC) 3*1024kB (M) 3*2048kB (M) 1*4096kB (R) = 254400kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
1967616 total pagecache pages
637 pages in swap cache
Swap cache stats: add 9593, delete 8956, find 2162/2929
Free swap = 10966116kB
Total swap = 10996456kB
2620832 pages RAM
0 pages HighMem/MovableOnly
41387 pages reserved
0 pages hwpoisoned
[2] 'perf record -a -g' while workspace switching
Samples: 27K of event 'cycles', Event count (approx.): 15793770075
+ 87.11% 0.00% Xorg [kernel.kallsyms] [k] tracesys
+ 86.81% 0.00% Xorg [unknown] [k] 0x00007fca2df34e77
+ 86.77% 0.00% Xorg [kernel.kallsyms] [k] sys_ioctl
+ 86.77% 0.00% Xorg [kernel.kallsyms] [k] do_vfs_ioctl
+ 86.77% 0.00% Xorg [nouveau] [k] nouveau_drm_ioctl
+ 86.77% 0.00% Xorg [drm] [k] drm_ioctl
+ 86.66% 0.00% Xorg [nouveau] [k] nouveau_gem_ioctl_new
+ 86.50% 0.00% Xorg [nouveau] [k] nouveau_gem_new
+ 86.49% 0.00% Xorg [nouveau] [k] nouveau_bo_new
+ 86.48% 0.00% Xorg [ttm] [k] ttm_bo_init
+ 86.47% 0.00% Xorg [ttm] [k] ttm_bo_validate
+ 86.46% 0.00% Xorg [ttm] [k] ttm_bo_handle_move_mem
+ 86.45% 0.00% Xorg [ttm] [k] ttm_tt_bind
+ 86.45% 0.00% Xorg [nouveau] [k] nouveau_ttm_tt_populate
+ 86.45% 0.00% Xorg [ttm] [k] ttm_dma_populate
+ 86.43% 0.01% Xorg [ttm] [k] ttm_dma_pool_alloc_new_pages
+ 86.42% 0.00% Xorg [kernel.kallsyms] [k] x86_swiotlb_alloc_coherent
+ 86.37% 0.00% Xorg [kernel.kallsyms] [k] dma_generic_alloc_coherent
+ 86.19% 0.00% Xorg [unknown] [k] 0x0000000000c00000
+ 85.82% 0.31% Xorg [kernel.kallsyms] [k] dma_alloc_from_contiguous
+ 84.21% 1.05% Xorg [kernel.kallsyms] [k] alloc_contig_range
+ 46.56% 46.56% Xorg [kernel.kallsyms] [k] move_freepages
+ 46.53% 0.29% Xorg [kernel.kallsyms] [k] move_freepages_block
+ 39.78% 0.13% Xorg [kernel.kallsyms] [k] start_isolate_page_range
+ 39.22% 0.40% Xorg [kernel.kallsyms] [k] set_migratetype_isolate
+ 27.26% 0.17% Xorg [kernel.kallsyms] [k] undo_isolate_page_range
+ 26.62% 0.33% Xorg [kernel.kallsyms] [k] unset_migratetype_isolate
+ 15.54% 7.93% Xorg [kernel.kallsyms] [k] drain_all_pages
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-25 20:36 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 20:36 UTC (permalink / raw)
To: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom
Cc: Hugh Dickens, Linux kernel, dri-devel, linux-mm, Andrew Morton,
Linus Torvalds, Ingo Molnar
On 09/25/2014 02:55 PM, Peter Hurley wrote:
> After several days uptime with a 3.16 kernel (generally running
> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
> desktop workspaces) I've been seeing some really extreme slowdowns.
>
> Mostly the slowdowns are associated with gpu-related tasks, like
> opening new emacs windows, switching workspaces, laughing at internet
> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
> it right away -- 3.15 is the first time suspend has worked reliably.
>
> This week I started looking into what the slowdown was and discovered
> it's happening during dma allocation through swiotlb (the cpus can do
> intel iommu but I don't use it because it's not the default for most users).
>
> I'm still working on a bisection but each step takes 8+ hours to
> validate and even then I'm no longer sure I still have the 'bad'
> commit in the bisection. [edit: yup, I started over]
>
> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>
> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
> fragmentation [1].
It's swapping, which is crazy because there's 7+GB of file cache [1] which
should be dropped before swapping.
The alloc_contig_range() failure precedes the swapping but not immediately
(44 mins. earlier).
How I reproduce this is to simply do a full distro kernel build.
Skipping the TLB flush is not the problem; the results below are from
3.16-final with that commit reverted.
The slowdown is really obvious because workspace switching redraw takes
multiple seconds to complete (all-cpu perf record of that below [2])
Regards,
Peter Hurley
[1]
SysRq : Show Memory
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 71
CPU 1: hi: 186, btch: 31 usd: 166
CPU 2: hi: 186, btch: 31 usd: 183
CPU 3: hi: 186, btch: 31 usd: 109
CPU 4: hi: 186, btch: 31 usd: 106
CPU 5: hi: 186, btch: 31 usd: 161
CPU 6: hi: 186, btch: 31 usd: 120
CPU 7: hi: 186, btch: 31 usd: 54
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 159
CPU 1: hi: 186, btch: 31 usd: 66
CPU 2: hi: 186, btch: 31 usd: 178
CPU 3: hi: 186, btch: 31 usd: 173
CPU 4: hi: 186, btch: 31 usd: 91
CPU 5: hi: 186, btch: 31 usd: 57
CPU 6: hi: 186, btch: 31 usd: 58
CPU 7: hi: 186, btch: 31 usd: 158
active_anon:170368 inactive_anon:173964 isolated_anon:0
active_file:982209 inactive_file:973911 isolated_file:0
unevictable:15 dirty:15 writeback:1 unstable:0
free:96067 slab_reclaimable:107401 slab_unreclaimable:12572
mapped:58271 shmem:10857 pagetables:9898 bounce:0
free_cma:18
Node 0 DMA free:15860kB min:104kB low:128kB high:156kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15960kB managed:15876kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2974 9980 9980
Node 0 DMA32 free:117740kB min:20108kB low:25132kB high:30160kB active_anon:205232kB inactive_anon:196308kB active_file:1186764kB inactive_file:1173760kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3127336kB managed:3048212kB mlocked:0kB dirty:24kB writeback:4kB mapped:71600kB shmem:8776kB slab_reclaimable:129132kB slab_unreclaimable:13468kB kernel_stack:2864kB pagetables:11536kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 7006 7006
Node 0 Normal free:250668kB min:47368kB low:59208kB high:71052kB active_anon:476240kB inactive_anon:499548kB active_file:2742072kB inactive_file:2721884kB unevictable:60kB isolated(anon):0kB isolated(file):0kB present:7340032kB managed:7174484kB mlocked:60kB dirty:36kB writeback:0kB mapped:161484kB shmem:34652kB slab_reclaimable:300472kB slab_unreclaimable:36804kB kernel_stack:7232kB pagetables:28056kB unstable:0kB bounce:0kB free_cma:72kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB (U) 0*8kB 1*16kB (U) 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15860kB
Node 0 DMA32: 4099*4kB (UEM) 4372*8kB (UEM) 668*16kB (UEM) 294*32kB (UEM) 47*64kB (UEM) 24*128kB (UEM) 19*256kB (UM) 5*512kB (UM) 0*1024kB 6*2048kB (M) 5*4096kB (M) = 117740kB
Node 0 Normal: 22224*4kB (UEMC) 8120*8kB (UEMC) 1594*16kB (UEMC) 301*32kB (UEMC) 154*64kB (UMC) 106*128kB (UEMC) 86*256kB (UMC) 13*512kB (UEMC) 3*1024kB (M) 3*2048kB (M) 1*4096kB (R) = 254400kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
1967616 total pagecache pages
637 pages in swap cache
Swap cache stats: add 9593, delete 8956, find 2162/2929
Free swap = 10966116kB
Total swap = 10996456kB
2620832 pages RAM
0 pages HighMem/MovableOnly
41387 pages reserved
0 pages hwpoisoned
[2] 'perf record -a -g' while workspace switching
Samples: 27K of event 'cycles', Event count (approx.): 15793770075
+ 87.11% 0.00% Xorg [kernel.kallsyms] [k] tracesys
+ 86.81% 0.00% Xorg [unknown] [k] 0x00007fca2df34e77
+ 86.77% 0.00% Xorg [kernel.kallsyms] [k] sys_ioctl
+ 86.77% 0.00% Xorg [kernel.kallsyms] [k] do_vfs_ioctl
+ 86.77% 0.00% Xorg [nouveau] [k] nouveau_drm_ioctl
+ 86.77% 0.00% Xorg [drm] [k] drm_ioctl
+ 86.66% 0.00% Xorg [nouveau] [k] nouveau_gem_ioctl_new
+ 86.50% 0.00% Xorg [nouveau] [k] nouveau_gem_new
+ 86.49% 0.00% Xorg [nouveau] [k] nouveau_bo_new
+ 86.48% 0.00% Xorg [ttm] [k] ttm_bo_init
+ 86.47% 0.00% Xorg [ttm] [k] ttm_bo_validate
+ 86.46% 0.00% Xorg [ttm] [k] ttm_bo_handle_move_mem
+ 86.45% 0.00% Xorg [ttm] [k] ttm_tt_bind
+ 86.45% 0.00% Xorg [nouveau] [k] nouveau_ttm_tt_populate
+ 86.45% 0.00% Xorg [ttm] [k] ttm_dma_populate
+ 86.43% 0.01% Xorg [ttm] [k] ttm_dma_pool_alloc_new_pages
+ 86.42% 0.00% Xorg [kernel.kallsyms] [k] x86_swiotlb_alloc_coherent
+ 86.37% 0.00% Xorg [kernel.kallsyms] [k] dma_generic_alloc_coherent
+ 86.19% 0.00% Xorg [unknown] [k] 0x0000000000c00000
+ 85.82% 0.31% Xorg [kernel.kallsyms] [k] dma_alloc_from_contiguous
+ 84.21% 1.05% Xorg [kernel.kallsyms] [k] alloc_contig_range
+ 46.56% 46.56% Xorg [kernel.kallsyms] [k] move_freepages
+ 46.53% 0.29% Xorg [kernel.kallsyms] [k] move_freepages_block
+ 39.78% 0.13% Xorg [kernel.kallsyms] [k] start_isolate_page_range
+ 39.22% 0.40% Xorg [kernel.kallsyms] [k] set_migratetype_isolate
+ 27.26% 0.17% Xorg [kernel.kallsyms] [k] undo_isolate_page_range
+ 26.62% 0.33% Xorg [kernel.kallsyms] [k] unset_migratetype_isolate
+ 15.54% 7.93% Xorg [kernel.kallsyms] [k] drain_all_pages
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-25 20:33 ` Alex Deucher
@ 2014-09-25 21:10 ` Peter Hurley
-1 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 21:10 UTC (permalink / raw)
To: Alex Deucher
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom, Hugh Dickens, Linux kernel, dri-devel,
linux-mm, Andrew Morton, Linus Torvalds, Ingo Molnar
On 09/25/2014 04:33 PM, Alex Deucher wrote:
> On Thu, Sep 25, 2014 at 2:55 PM, Peter Hurley <peter@hurleysoftware.com> wrote:
>> After several days uptime with a 3.16 kernel (generally running
>> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
>> desktop workspaces) I've been seeing some really extreme slowdowns.
>>
>> Mostly the slowdowns are associated with gpu-related tasks, like
>> opening new emacs windows, switching workspaces, laughing at internet
>> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
>> it right away -- 3.15 is the first time suspend has worked reliably.
>>
>> This week I started looking into what the slowdown was and discovered
>> it's happening during dma allocation through swiotlb (the cpus can do
>> intel iommu but I don't use it because it's not the default for most users).
>>
>> I'm still working on a bisection but each step takes 8+ hours to
>> validate and even then I'm no longer sure I still have the 'bad'
>> commit in the bisection. [edit: yup, I started over]
>>
>> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
>> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>>
>> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
>> fragmentation [1].
>>
>> Besides Mel's page allocator changes in 3.16, another suspect commit is:
>>
>> commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
>> Author: Shaohua Li <shli@kernel.org>
>> Date: Tue Apr 8 15:58:09 2014 +0800
>>
>> x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
>>
>> Specifically, this statement:
>>
>> It could cause incorrect page aging and the (mistaken) reclaim of
>> hot pages, but the chance of that should be relatively low.
>>
>> I'm wondering if this could cause worse-case behavior with TTM? I'm
>> testing a revert of this on mainline 3.16-final now, with no results yet.
>>
>> Thoughts?
>
> You may also be seeing this:
> https://lkml.org/lkml/2014/8/8/445
Thanks Alex. That is indeed the problem.
Still reading the email thread to find out where the patches
are that fix this. Although it doesn't make much sense to me
that nouveau sets up a 1GB GART and then uses TTM which is
trying to shove all the DMA through a 16MB CMA window
(which turns out to be the base Ubuntu config).
Regards,
Peter Hurley
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-25 21:10 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 21:10 UTC (permalink / raw)
To: Alex Deucher
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom, Hugh Dickens, Linux kernel, dri-devel,
linux-mm, Andrew Morton, Linus Torvalds, Ingo Molnar
On 09/25/2014 04:33 PM, Alex Deucher wrote:
> On Thu, Sep 25, 2014 at 2:55 PM, Peter Hurley <peter@hurleysoftware.com> wrote:
>> After several days uptime with a 3.16 kernel (generally running
>> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
>> desktop workspaces) I've been seeing some really extreme slowdowns.
>>
>> Mostly the slowdowns are associated with gpu-related tasks, like
>> opening new emacs windows, switching workspaces, laughing at internet
>> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
>> it right away -- 3.15 is the first time suspend has worked reliably.
>>
>> This week I started looking into what the slowdown was and discovered
>> it's happening during dma allocation through swiotlb (the cpus can do
>> intel iommu but I don't use it because it's not the default for most users).
>>
>> I'm still working on a bisection but each step takes 8+ hours to
>> validate and even then I'm no longer sure I still have the 'bad'
>> commit in the bisection. [edit: yup, I started over]
>>
>> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
>> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>>
>> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
>> fragmentation [1].
>>
>> Besides Mel's page allocator changes in 3.16, another suspect commit is:
>>
>> commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
>> Author: Shaohua Li <shli@kernel.org>
>> Date: Tue Apr 8 15:58:09 2014 +0800
>>
>> x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
>>
>> Specifically, this statement:
>>
>> It could cause incorrect page aging and the (mistaken) reclaim of
>> hot pages, but the chance of that should be relatively low.
>>
>> I'm wondering if this could cause worse-case behavior with TTM? I'm
>> testing a revert of this on mainline 3.16-final now, with no results yet.
>>
>> Thoughts?
>
> You may also be seeing this:
> https://lkml.org/lkml/2014/8/8/445
Thanks Alex. That is indeed the problem.
Still reading the email thread to find out where the patches
are that fix this. Although it doesn't make much sense to me
that nouveau sets up a 1GB GART and then uses TTM which is
trying to shove all the DMA through a 16MB CMA window
(which turns out to be the base Ubuntu config).
Regards,
Peter Hurley
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-25 19:35 ` Chuck Ebbert
@ 2014-09-25 23:52 ` Peter Hurley
-1 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 23:52 UTC (permalink / raw)
To: Chuck Ebbert
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
> There are six ttm patches queued for 3.16.4:
>
> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
Thanks for info, Chuck.
Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
which is the root problem.
Regards,
Peter Hurley
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-25 23:52 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-25 23:52 UTC (permalink / raw)
To: Chuck Ebbert
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Maarten Lankhorst,
Thomas Hellstrom, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
> There are six ttm patches queued for 3.16.4:
>
> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
Thanks for info, Chuck.
Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
which is the root problem.
Regards,
Peter Hurley
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-25 23:52 ` Peter Hurley
(?)
@ 2014-09-26 7:15 ` Thomas Hellstrom
-1 siblings, 0 replies; 42+ messages in thread
From: Thomas Hellstrom @ 2014-09-26 7:15 UTC (permalink / raw)
To: Peter Hurley
Cc: Chuck Ebbert, Mel Gorman, Shaohua Li, Rik van Riel,
Maarten Lankhorst, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On 09/26/2014 01:52 AM, Peter Hurley wrote:
> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>> There are six ttm patches queued for 3.16.4:
>>
>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
> Thanks for info, Chuck.
>
> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
> which is the root problem.
>
> Regards,
> Peter Hurley
The problem is not really in TTM but in CMA, There was a guy offering to
fix this in the CMA code but I guess he didn't probably because he
didn't receive any feedback.
/Thomas
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 7:15 ` Thomas Hellstrom
0 siblings, 0 replies; 42+ messages in thread
From: Thomas Hellstrom @ 2014-09-26 7:15 UTC (permalink / raw)
To: Peter Hurley
Cc: Chuck Ebbert, Mel Gorman, Shaohua Li, Rik van Riel,
Maarten Lankhorst, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On 09/26/2014 01:52 AM, Peter Hurley wrote:
> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>> There are six ttm patches queued for 3.16.4:
>>
>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
> Thanks for info, Chuck.
>
> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
> which is the root problem.
>
> Regards,
> Peter Hurley
The problem is not really in TTM but in CMA, There was a guy offering to
fix this in the CMA code but I guess he didn't probably because he
didn't receive any feedback.
/Thomas
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 7:15 ` Thomas Hellstrom
0 siblings, 0 replies; 42+ messages in thread
From: Thomas Hellstrom @ 2014-09-26 7:15 UTC (permalink / raw)
To: Peter Hurley
Cc: Rik van Riel, Linus Torvalds, Hugh Dickens, Chuck Ebbert,
dri-devel, Linux kernel, linux-mm, Mel Gorman, Andrew Morton,
Shaohua Li, Ingo Molnar
On 09/26/2014 01:52 AM, Peter Hurley wrote:
> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>> There are six ttm patches queued for 3.16.4:
>>
>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
> Thanks for info, Chuck.
>
> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
> which is the root problem.
>
> Regards,
> Peter Hurley
The problem is not really in TTM but in CMA, There was a guy offering to
fix this in the CMA code but I guess he didn't probably because he
didn't receive any feedback.
/Thomas
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 7:15 ` Thomas Hellstrom
@ 2014-09-26 10:40 ` Chuck Ebbert
-1 siblings, 0 replies; 42+ messages in thread
From: Chuck Ebbert @ 2014-09-26 10:40 UTC (permalink / raw)
To: Thomas Hellstrom
Cc: Peter Hurley, Mel Gorman, Shaohua Li, Rik van Riel,
Maarten Lankhorst, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On Fri, 26 Sep 2014 09:15:57 +0200
Thomas Hellstrom <thellstrom@vmware.com> wrote:
> On 09/26/2014 01:52 AM, Peter Hurley wrote:
> > On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
> >> There are six ttm patches queued for 3.16.4:
> >>
> >> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
> >> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
> >> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
> >> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
> >> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
> >> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
> > Thanks for info, Chuck.
> >
> > Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
> > which is the root problem.
> >
> > Regards,
> > Peter Hurley
>
> The problem is not really in TTM but in CMA, There was a guy offering to
> fix this in the CMA code but I guess he didn't probably because he
> didn't receive any feedback.
>
Yeah, the "solution" to this problem seems to be "don't enable CMA on
x86". Maybe it should even be disabled in the config system.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 10:40 ` Chuck Ebbert
0 siblings, 0 replies; 42+ messages in thread
From: Chuck Ebbert @ 2014-09-26 10:40 UTC (permalink / raw)
To: Thomas Hellstrom
Cc: Peter Hurley, Mel Gorman, Shaohua Li, Rik van Riel,
Maarten Lankhorst, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On Fri, 26 Sep 2014 09:15:57 +0200
Thomas Hellstrom <thellstrom@vmware.com> wrote:
> On 09/26/2014 01:52 AM, Peter Hurley wrote:
> > On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
> >> There are six ttm patches queued for 3.16.4:
> >>
> >> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
> >> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
> >> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
> >> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
> >> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
> >> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
> > Thanks for info, Chuck.
> >
> > Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
> > which is the root problem.
> >
> > Regards,
> > Peter Hurley
>
> The problem is not really in TTM but in CMA, There was a guy offering to
> fix this in the CMA code but I guess he didn't probably because he
> didn't receive any feedback.
>
Yeah, the "solution" to this problem seems to be "don't enable CMA on
x86". Maybe it should even be disabled in the config system.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 10:40 ` Chuck Ebbert
@ 2014-09-26 10:45 ` Thomas Hellstrom
-1 siblings, 0 replies; 42+ messages in thread
From: Thomas Hellstrom @ 2014-09-26 10:45 UTC (permalink / raw)
To: Chuck Ebbert
Cc: Peter Hurley, Mel Gorman, Shaohua Li, Rik van Riel,
Maarten Lankhorst, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
> On Fri, 26 Sep 2014 09:15:57 +0200
> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>
>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>> There are six ttm patches queued for 3.16.4:
>>>>
>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>> Thanks for info, Chuck.
>>>
>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>> which is the root problem.
>>>
>>> Regards,
>>> Peter Hurley
>> The problem is not really in TTM but in CMA, There was a guy offering to
>> fix this in the CMA code but I guess he didn't probably because he
>> didn't receive any feedback.
>>
> Yeah, the "solution" to this problem seems to be "don't enable CMA on
> x86". Maybe it should even be disabled in the config system.
Or, as previously suggested, don't use CMA for order 0 (single page)
allocations....
/Thomas
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 10:45 ` Thomas Hellstrom
0 siblings, 0 replies; 42+ messages in thread
From: Thomas Hellstrom @ 2014-09-26 10:45 UTC (permalink / raw)
To: Chuck Ebbert
Cc: Peter Hurley, Mel Gorman, Shaohua Li, Rik van Riel,
Maarten Lankhorst, Andrew Morton, Linus Torvalds, Linux kernel,
linux-mm, Ingo Molnar, Hugh Dickens, dri-devel
On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
> On Fri, 26 Sep 2014 09:15:57 +0200
> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>
>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>> There are six ttm patches queued for 3.16.4:
>>>>
>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>> Thanks for info, Chuck.
>>>
>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>> which is the root problem.
>>>
>>> Regards,
>>> Peter Hurley
>> The problem is not really in TTM but in CMA, There was a guy offering to
>> fix this in the CMA code but I guess he didn't probably because he
>> didn't receive any feedback.
>>
> Yeah, the "solution" to this problem seems to be "don't enable CMA on
> x86". Maybe it should even be disabled in the config system.
Or, as previously suggested, don't use CMA for order 0 (single page)
allocations....
/Thomas
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 10:45 ` Thomas Hellstrom
(?)
@ 2014-09-26 12:28 ` Rob Clark
-1 siblings, 0 replies; 42+ messages in thread
From: Rob Clark @ 2014-09-26 12:28 UTC (permalink / raw)
To: Thomas Hellstrom
Cc: Chuck Ebbert, Rik van Riel, Peter Hurley, Linus Torvalds,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Mel Gorman,
Andrew Morton, Shaohua Li, Ingo Molnar
On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>> On Fri, 26 Sep 2014 09:15:57 +0200
>> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>
>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>> There are six ttm patches queued for 3.16.4:
>>>>>
>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>> Thanks for info, Chuck.
>>>>
>>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>>> which is the root problem.
>>>>
>>>> Regards,
>>>> Peter Hurley
>>> The problem is not really in TTM but in CMA, There was a guy offering to
>>> fix this in the CMA code but I guess he didn't probably because he
>>> didn't receive any feedback.
>>>
>> Yeah, the "solution" to this problem seems to be "don't enable CMA on
>> x86". Maybe it should even be disabled in the config system.
> Or, as previously suggested, don't use CMA for order 0 (single page)
> allocations....
On devices that actually need CMA pools to arrange for memory to be in
certain ranges, I think you probably do want to have order 0 pages
come from the CMA pool.
Seems like disabling CMA on x86 (where it should be unneeded) is the
better way, IMO
BR,
-R
> /Thomas
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 12:28 ` Rob Clark
0 siblings, 0 replies; 42+ messages in thread
From: Rob Clark @ 2014-09-26 12:28 UTC (permalink / raw)
To: Thomas Hellstrom
Cc: Chuck Ebbert, Rik van Riel, Peter Hurley, Linus Torvalds,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Mel Gorman,
Andrew Morton, Shaohua Li, Ingo Molnar
On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>> On Fri, 26 Sep 2014 09:15:57 +0200
>> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>
>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>> There are six ttm patches queued for 3.16.4:
>>>>>
>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>> Thanks for info, Chuck.
>>>>
>>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>>> which is the root problem.
>>>>
>>>> Regards,
>>>> Peter Hurley
>>> The problem is not really in TTM but in CMA, There was a guy offering to
>>> fix this in the CMA code but I guess he didn't probably because he
>>> didn't receive any feedback.
>>>
>> Yeah, the "solution" to this problem seems to be "don't enable CMA on
>> x86". Maybe it should even be disabled in the config system.
> Or, as previously suggested, don't use CMA for order 0 (single page)
> allocations....
On devices that actually need CMA pools to arrange for memory to be in
certain ranges, I think you probably do want to have order 0 pages
come from the CMA pool.
Seems like disabling CMA on x86 (where it should be unneeded) is the
better way, IMO
BR,
-R
> /Thomas
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 12:28 ` Rob Clark
0 siblings, 0 replies; 42+ messages in thread
From: Rob Clark @ 2014-09-26 12:28 UTC (permalink / raw)
To: Thomas Hellstrom
Cc: Chuck Ebbert, Peter Hurley, Shaohua Li, Hugh Dickens,
Rik van Riel, dri-devel, Linux kernel, linux-mm, Mel Gorman,
Andrew Morton, Linus Torvalds, Ingo Molnar
On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>> On Fri, 26 Sep 2014 09:15:57 +0200
>> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>
>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>> There are six ttm patches queued for 3.16.4:
>>>>>
>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>> Thanks for info, Chuck.
>>>>
>>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>>> which is the root problem.
>>>>
>>>> Regards,
>>>> Peter Hurley
>>> The problem is not really in TTM but in CMA, There was a guy offering to
>>> fix this in the CMA code but I guess he didn't probably because he
>>> didn't receive any feedback.
>>>
>> Yeah, the "solution" to this problem seems to be "don't enable CMA on
>> x86". Maybe it should even be disabled in the config system.
> Or, as previously suggested, don't use CMA for order 0 (single page)
> allocations....
On devices that actually need CMA pools to arrange for memory to be in
certain ranges, I think you probably do want to have order 0 pages
come from the CMA pool.
Seems like disabling CMA on x86 (where it should be unneeded) is the
better way, IMO
BR,
-R
> /Thomas
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 12:28 ` Rob Clark
@ 2014-09-26 12:34 ` Thomas Hellstrom
-1 siblings, 0 replies; 42+ messages in thread
From: Thomas Hellstrom @ 2014-09-26 12:34 UTC (permalink / raw)
To: Rob Clark
Cc: Chuck Ebbert, Rik van Riel, Peter Hurley, Linus Torvalds,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Mel Gorman,
Andrew Morton, Shaohua Li, Ingo Molnar
On 09/26/2014 02:28 PM, Rob Clark wrote:
> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>> On Fri, 26 Sep 2014 09:15:57 +0200
>>> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>>
>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>
>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>> Thanks for info, Chuck.
>>>>>
>>>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>>>> which is the root problem.
>>>>>
>>>>> Regards,
>>>>> Peter Hurley
>>>> The problem is not really in TTM but in CMA, There was a guy offering to
>>>> fix this in the CMA code but I guess he didn't probably because he
>>>> didn't receive any feedback.
>>>>
>>> Yeah, the "solution" to this problem seems to be "don't enable CMA on
>>> x86". Maybe it should even be disabled in the config system.
>> Or, as previously suggested, don't use CMA for order 0 (single page)
>> allocations....
> On devices that actually need CMA pools to arrange for memory to be in
> certain ranges, I think you probably do want to have order 0 pages
> come from the CMA pool.
But can the DMA subsystem or more specifically dma_alloc_coherent()
really guarantee such things? Isn't it better for such devices to use
CMA directly?
/Thomas
>
> Seems like disabling CMA on x86 (where it should be unneeded) is the
> better way, IMO
>
> BR,
> -R
>
>
>> /Thomas
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://urldefense.proofpoint.com/v1/url?u=http://lists.freedesktop.org/mailman/listinfo/dri-devel&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=l5Ago9ekmVFZ3c4M6eauqrJWGwjf6fTb%2BP3CxbBFkVM%3D%0A&m=Uz7JXDXYXp4RlLs7G6qxMQlhOOT0trW3l78xpKg6Ass%3D%0A&s=50d6b7b3bfd093c93a228437a3d4414e49b4de817657c49c35154a115a5c2188
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 12:34 ` Thomas Hellstrom
0 siblings, 0 replies; 42+ messages in thread
From: Thomas Hellstrom @ 2014-09-26 12:34 UTC (permalink / raw)
To: Rob Clark
Cc: Chuck Ebbert, Rik van Riel, Peter Hurley, Linus Torvalds,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Mel Gorman,
Andrew Morton, Shaohua Li, Ingo Molnar
On 09/26/2014 02:28 PM, Rob Clark wrote:
> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>> On Fri, 26 Sep 2014 09:15:57 +0200
>>> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>>
>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>
>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>> Thanks for info, Chuck.
>>>>>
>>>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>>>> which is the root problem.
>>>>>
>>>>> Regards,
>>>>> Peter Hurley
>>>> The problem is not really in TTM but in CMA, There was a guy offering to
>>>> fix this in the CMA code but I guess he didn't probably because he
>>>> didn't receive any feedback.
>>>>
>>> Yeah, the "solution" to this problem seems to be "don't enable CMA on
>>> x86". Maybe it should even be disabled in the config system.
>> Or, as previously suggested, don't use CMA for order 0 (single page)
>> allocations....
> On devices that actually need CMA pools to arrange for memory to be in
> certain ranges, I think you probably do want to have order 0 pages
> come from the CMA pool.
But can the DMA subsystem or more specifically dma_alloc_coherent()
really guarantee such things? Isn't it better for such devices to use
CMA directly?
/Thomas
>
> Seems like disabling CMA on x86 (where it should be unneeded) is the
> better way, IMO
>
> BR,
> -R
>
>
>> /Thomas
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://urldefense.proofpoint.com/v1/url?u=http://lists.freedesktop.org/mailman/listinfo/dri-devel&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=l5Ago9ekmVFZ3c4M6eauqrJWGwjf6fTb%2BP3CxbBFkVM%3D%0A&m=Uz7JXDXYXp4RlLs7G6qxMQlhOOT0trW3l78xpKg6Ass%3D%0A&s=50d6b7b3bfd093c93a228437a3d4414e49b4de817657c49c35154a115a5c2188
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 12:28 ` Rob Clark
@ 2014-09-26 12:40 ` Rik van Riel
-1 siblings, 0 replies; 42+ messages in thread
From: Rik van Riel @ 2014-09-26 12:40 UTC (permalink / raw)
To: Rob Clark, Thomas Hellstrom
Cc: Chuck Ebbert, Peter Hurley, Linus Torvalds, Hugh Dickens,
Linux kernel, dri-devel, linux-mm, Mel Gorman, Andrew Morton,
Shaohua Li, Ingo Molnar
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 09/26/2014 08:28 AM, Rob Clark wrote:
> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
> <thellstrom@vmware.com> wrote:
>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>> <thellstrom@vmware.com> wrote:
>>>
>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>
>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>
>>>>>>
drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>
>>>>>>
drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>
>>>>>>
Thanks for info, Chuck.
>>>>>
>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>> CMA dma allocation, which is the root problem.
>>>>>
>>>>> Regards, Peter Hurley
>>>> The problem is not really in TTM but in CMA, There was a guy
>>>> offering to fix this in the CMA code but I guess he didn't
>>>> probably because he didn't receive any feedback.
>>>>
>>> Yeah, the "solution" to this problem seems to be "don't enable
>>> CMA on x86". Maybe it should even be disabled in the config
>>> system.
>> Or, as previously suggested, don't use CMA for order 0 (single
>> page) allocations....
>
> On devices that actually need CMA pools to arrange for memory to be
> in certain ranges, I think you probably do want to have order 0
> pages come from the CMA pool.
>
> Seems like disabling CMA on x86 (where it should be unneeded) is
> the better way, IMO
CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
pages.
There may also be people with devices that do not scatter-gather, and
need a large physically contiguous buffer, though there should be
relatively few of those on x86.
I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
through the normal allocator on x86, and only invoking CMA for larger
allocations.
- --
All rights reversed
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBAgAGBQJUJV6lAAoJEM553pKExN6DjrQH/1N19Hp5FrJr+a3GpQDh6ouc
YSBChxe+wG1h3OmcFGAG69tOK9XPw0oaV77ohwLxnvSv6BQZyi2CUIJvUdgSaOOx
8XYnn8VIIlMn4IKYmraAhSWT/gm3FkyDW7tckEdLV0NsrKeUcavCRHcLXxh41OBw
XJboZyS3XvwF+scAwjHpWPxby1Byi0lZJizTAzI3xdlyVaM5Lio1xLvOW2MHY7dR
h/ai8mfAAdQvnaHsFLoypBM/xYJqaUVU8IyCzhOeO86dUMy2xhD4vm/f9vSLuOju
4VYf7POuziNo1q2vJ8YcrThsAjB0Oiu9B5nDar471G3l1xN1zHQVw/RAnpNF9Kk=
=iZlM
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 12:40 ` Rik van Riel
0 siblings, 0 replies; 42+ messages in thread
From: Rik van Riel @ 2014-09-26 12:40 UTC (permalink / raw)
To: Rob Clark, Thomas Hellstrom
Cc: Chuck Ebbert, Peter Hurley, Linus Torvalds, Hugh Dickens,
Linux kernel, dri-devel, linux-mm, Mel Gorman, Andrew Morton,
Shaohua Li, Ingo Molnar
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 09/26/2014 08:28 AM, Rob Clark wrote:
> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
> <thellstrom@vmware.com> wrote:
>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>> <thellstrom@vmware.com> wrote:
>>>
>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>
>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>
>>>>>>
drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>
>>>>>>
drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>
>>>>>>
Thanks for info, Chuck.
>>>>>
>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>> CMA dma allocation, which is the root problem.
>>>>>
>>>>> Regards, Peter Hurley
>>>> The problem is not really in TTM but in CMA, There was a guy
>>>> offering to fix this in the CMA code but I guess he didn't
>>>> probably because he didn't receive any feedback.
>>>>
>>> Yeah, the "solution" to this problem seems to be "don't enable
>>> CMA on x86". Maybe it should even be disabled in the config
>>> system.
>> Or, as previously suggested, don't use CMA for order 0 (single
>> page) allocations....
>
> On devices that actually need CMA pools to arrange for memory to be
> in certain ranges, I think you probably do want to have order 0
> pages come from the CMA pool.
>
> Seems like disabling CMA on x86 (where it should be unneeded) is
> the better way, IMO
CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
pages.
There may also be people with devices that do not scatter-gather, and
need a large physically contiguous buffer, though there should be
relatively few of those on x86.
I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
through the normal allocator on x86, and only invoking CMA for larger
allocations.
- --
All rights reversed
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBAgAGBQJUJV6lAAoJEM553pKExN6DjrQH/1N19Hp5FrJr+a3GpQDh6ouc
YSBChxe+wG1h3OmcFGAG69tOK9XPw0oaV77ohwLxnvSv6BQZyi2CUIJvUdgSaOOx
8XYnn8VIIlMn4IKYmraAhSWT/gm3FkyDW7tckEdLV0NsrKeUcavCRHcLXxh41OBw
XJboZyS3XvwF+scAwjHpWPxby1Byi0lZJizTAzI3xdlyVaM5Lio1xLvOW2MHY7dR
h/ai8mfAAdQvnaHsFLoypBM/xYJqaUVU8IyCzhOeO86dUMy2xhD4vm/f9vSLuOju
4VYf7POuziNo1q2vJ8YcrThsAjB0Oiu9B5nDar471G3l1xN1zHQVw/RAnpNF9Kk=
=iZlM
-----END PGP SIGNATURE-----
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 12:34 ` Thomas Hellstrom
@ 2014-09-26 13:40 ` Rob Clark
-1 siblings, 0 replies; 42+ messages in thread
From: Rob Clark @ 2014-09-26 13:40 UTC (permalink / raw)
To: Thomas Hellstrom
Cc: Chuck Ebbert, Rik van Riel, Peter Hurley, Linus Torvalds,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Mel Gorman,
Andrew Morton, Shaohua Li, Ingo Molnar
On Fri, Sep 26, 2014 at 8:34 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
> On 09/26/2014 02:28 PM, Rob Clark wrote:
>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>> On Fri, 26 Sep 2014 09:15:57 +0200
>>>> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>>>
>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>
>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>> Thanks for info, Chuck.
>>>>>>
>>>>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>>>>> which is the root problem.
>>>>>>
>>>>>> Regards,
>>>>>> Peter Hurley
>>>>> The problem is not really in TTM but in CMA, There was a guy offering to
>>>>> fix this in the CMA code but I guess he didn't probably because he
>>>>> didn't receive any feedback.
>>>>>
>>>> Yeah, the "solution" to this problem seems to be "don't enable CMA on
>>>> x86". Maybe it should even be disabled in the config system.
>>> Or, as previously suggested, don't use CMA for order 0 (single page)
>>> allocations....
>> On devices that actually need CMA pools to arrange for memory to be in
>> certain ranges, I think you probably do want to have order 0 pages
>> come from the CMA pool.
>
> But can the DMA subsystem or more specifically dma_alloc_coherent()
> really guarantee such things? Isn't it better for such devices to use
> CMA directly?
Well, I was thinking more specifically about a use-case that was
mentioned several times during the early CMA discussions, about video
decoders/encoders which needed Y and UV split across memory banks to
achieve sufficient bandwidth. I assume they must use CMA directly for
this (since they'd need multiple pools per device), but not really
100% sure about that.
So perhaps, yeah, if you shunt order 0 allocations away from CMA at
the DMA layer, maybe it is ok. If there actually is a valid use-case
for CMA on sane hardware, then maybe this is the better way, and let
the insane hw folks hack around it.
(plus, well, the use-case I was mentioning isn't really about order 0
allocations anyway)
BR,
-R
> /Thomas
>
>
>>
>> Seems like disabling CMA on x86 (where it should be unneeded) is the
>> better way, IMO
>>
>> BR,
>> -R
>>
>>
>>> /Thomas
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://urldefense.proofpoint.com/v1/url?u=http://lists.freedesktop.org/mailman/listinfo/dri-devel&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=l5Ago9ekmVFZ3c4M6eauqrJWGwjf6fTb%2BP3CxbBFkVM%3D%0A&m=Uz7JXDXYXp4RlLs7G6qxMQlhOOT0trW3l78xpKg6Ass%3D%0A&s=50d6b7b3bfd093c93a228437a3d4414e49b4de817657c49c35154a115a5c2188
>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 13:40 ` Rob Clark
0 siblings, 0 replies; 42+ messages in thread
From: Rob Clark @ 2014-09-26 13:40 UTC (permalink / raw)
To: Thomas Hellstrom
Cc: Chuck Ebbert, Rik van Riel, Peter Hurley, Linus Torvalds,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Mel Gorman,
Andrew Morton, Shaohua Li, Ingo Molnar
On Fri, Sep 26, 2014 at 8:34 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
> On 09/26/2014 02:28 PM, Rob Clark wrote:
>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>> On Fri, 26 Sep 2014 09:15:57 +0200
>>>> Thomas Hellstrom <thellstrom@vmware.com> wrote:
>>>>
>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>
>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>> Thanks for info, Chuck.
>>>>>>
>>>>>> Unfortunately, none of these fix TTM dma allocation doing CMA dma allocation,
>>>>>> which is the root problem.
>>>>>>
>>>>>> Regards,
>>>>>> Peter Hurley
>>>>> The problem is not really in TTM but in CMA, There was a guy offering to
>>>>> fix this in the CMA code but I guess he didn't probably because he
>>>>> didn't receive any feedback.
>>>>>
>>>> Yeah, the "solution" to this problem seems to be "don't enable CMA on
>>>> x86". Maybe it should even be disabled in the config system.
>>> Or, as previously suggested, don't use CMA for order 0 (single page)
>>> allocations....
>> On devices that actually need CMA pools to arrange for memory to be in
>> certain ranges, I think you probably do want to have order 0 pages
>> come from the CMA pool.
>
> But can the DMA subsystem or more specifically dma_alloc_coherent()
> really guarantee such things? Isn't it better for such devices to use
> CMA directly?
Well, I was thinking more specifically about a use-case that was
mentioned several times during the early CMA discussions, about video
decoders/encoders which needed Y and UV split across memory banks to
achieve sufficient bandwidth. I assume they must use CMA directly for
this (since they'd need multiple pools per device), but not really
100% sure about that.
So perhaps, yeah, if you shunt order 0 allocations away from CMA at
the DMA layer, maybe it is ok. If there actually is a valid use-case
for CMA on sane hardware, then maybe this is the better way, and let
the insane hw folks hack around it.
(plus, well, the use-case I was mentioning isn't really about order 0
allocations anyway)
BR,
-R
> /Thomas
>
>
>>
>> Seems like disabling CMA on x86 (where it should be unneeded) is the
>> better way, IMO
>>
>> BR,
>> -R
>>
>>
>>> /Thomas
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://urldefense.proofpoint.com/v1/url?u=http://lists.freedesktop.org/mailman/listinfo/dri-devel&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=l5Ago9ekmVFZ3c4M6eauqrJWGwjf6fTb%2BP3CxbBFkVM%3D%0A&m=Uz7JXDXYXp4RlLs7G6qxMQlhOOT0trW3l78xpKg6Ass%3D%0A&s=50d6b7b3bfd093c93a228437a3d4414e49b4de817657c49c35154a115a5c2188
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 12:40 ` Rik van Riel
(?)
@ 2014-09-26 14:10 ` Peter Hurley
-1 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-26 14:10 UTC (permalink / raw)
To: Rik van Riel, Rob Clark, Thomas Hellstrom
Cc: Chuck Ebbert, Linus Torvalds, Hugh Dickens, Linux kernel,
dri-devel, linux-mm, Mel Gorman, Andrew Morton, Shaohua Li,
Ingo Molnar, Leann Ogasawara, Marek Szyprowski, Kyungmin Park,
Arnd Bergmann
[ +cc Leann Ogasawara, Marek Szyprowski, Kyungmin Park, Arnd Bergmann ]
On 09/26/2014 08:40 AM, Rik van Riel wrote:
> On 09/26/2014 08:28 AM, Rob Clark wrote:
>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
>> <thellstrom@vmware.com> wrote:
>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>>> <thellstrom@vmware.com> wrote:
>>>>
>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>
>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>
>>>>>>>
> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>
>>>>>>>
> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>>
>>>>>>>
> Thanks for info, Chuck.
>>>>>>
>>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>>> CMA dma allocation, which is the root problem.
>>>>>>
>>>>>> Regards, Peter Hurley
>>>>> The problem is not really in TTM but in CMA, There was a guy
>>>>> offering to fix this in the CMA code but I guess he didn't
>>>>> probably because he didn't receive any feedback.
>>>>>
>>>> Yeah, the "solution" to this problem seems to be "don't enable
>>>> CMA on x86". Maybe it should even be disabled in the config
>>>> system.
>>> Or, as previously suggested, don't use CMA for order 0 (single
>>> page) allocations....
>>
>> On devices that actually need CMA pools to arrange for memory to be
>> in certain ranges, I think you probably do want to have order 0
>> pages come from the CMA pool.
>>
>> Seems like disabling CMA on x86 (where it should be unneeded) is
>> the better way, IMO
>
> CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
> pages.
>
> There may also be people with devices that do not scatter-gather, and
> need a large physically contiguous buffer, though there should be
> relatively few of those on x86.
>
> I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
> through the normal allocator on x86, and only invoking CMA for larger
> allocations.
The code that uses CMA to satisfy DMA allocations on x86 is
specific to the x86 arch and was added in 2011 as a means of _testing_
CMA in KVM:
commit 0a2b9a6ea93650b8a00f9fd5ee8fdd25671e2df6
Author: Marek Szyprowski <m.szyprowski@samsung.com>
Date: Thu Dec 29 13:09:51 2011 +0100
X86: integrate CMA with DMA-mapping subsystem
This patch adds support for CMA to dma-mapping subsystem for x86
architecture that uses common pci-dma/pci-nommu implementation. This
allows to test CMA on KVM/QEMU and a lot of common x86 boxes.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
CC: Michal Nazarewicz <mina86@mina86.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
(no x86 maintainer acks?).
Unfortunately, this code is enabled whenever CMA is enabled, rather
than as a separate test configuration.
So, while enabling CMA may have other purposes on x86, using it for
x86 swiotlb and nommu dma allocations is not one of the them.
And Ubuntu should not be enabling CONFIG_DMA_CMA for their i386
and amd64 configurations, as this is trying to drive _all_ dma mapping
allocations through a _very_ small window (which is killing GPU
performance).
Regards,
Peter Hurley
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 14:10 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-26 14:10 UTC (permalink / raw)
To: Rik van Riel, Rob Clark, Thomas Hellstrom
Cc: Chuck Ebbert, Linus Torvalds, Hugh Dickens, Linux kernel,
dri-devel, linux-mm, Mel Gorman, Andrew Morton, Shaohua Li,
Ingo Molnar, Leann Ogasawara, Marek Szyprowski, Kyungmin Park,
Arnd Bergmann
[ +cc Leann Ogasawara, Marek Szyprowski, Kyungmin Park, Arnd Bergmann ]
On 09/26/2014 08:40 AM, Rik van Riel wrote:
> On 09/26/2014 08:28 AM, Rob Clark wrote:
>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
>> <thellstrom@vmware.com> wrote:
>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>>> <thellstrom@vmware.com> wrote:
>>>>
>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>
>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>
>>>>>>>
> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>
>>>>>>>
> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>>
>>>>>>>
> Thanks for info, Chuck.
>>>>>>
>>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>>> CMA dma allocation, which is the root problem.
>>>>>>
>>>>>> Regards, Peter Hurley
>>>>> The problem is not really in TTM but in CMA, There was a guy
>>>>> offering to fix this in the CMA code but I guess he didn't
>>>>> probably because he didn't receive any feedback.
>>>>>
>>>> Yeah, the "solution" to this problem seems to be "don't enable
>>>> CMA on x86". Maybe it should even be disabled in the config
>>>> system.
>>> Or, as previously suggested, don't use CMA for order 0 (single
>>> page) allocations....
>>
>> On devices that actually need CMA pools to arrange for memory to be
>> in certain ranges, I think you probably do want to have order 0
>> pages come from the CMA pool.
>>
>> Seems like disabling CMA on x86 (where it should be unneeded) is
>> the better way, IMO
>
> CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
> pages.
>
> There may also be people with devices that do not scatter-gather, and
> need a large physically contiguous buffer, though there should be
> relatively few of those on x86.
>
> I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
> through the normal allocator on x86, and only invoking CMA for larger
> allocations.
The code that uses CMA to satisfy DMA allocations on x86 is
specific to the x86 arch and was added in 2011 as a means of _testing_
CMA in KVM:
commit 0a2b9a6ea93650b8a00f9fd5ee8fdd25671e2df6
Author: Marek Szyprowski <m.szyprowski@samsung.com>
Date: Thu Dec 29 13:09:51 2011 +0100
X86: integrate CMA with DMA-mapping subsystem
This patch adds support for CMA to dma-mapping subsystem for x86
architecture that uses common pci-dma/pci-nommu implementation. This
allows to test CMA on KVM/QEMU and a lot of common x86 boxes.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
CC: Michal Nazarewicz <mina86@mina86.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
(no x86 maintainer acks?).
Unfortunately, this code is enabled whenever CMA is enabled, rather
than as a separate test configuration.
So, while enabling CMA may have other purposes on x86, using it for
x86 swiotlb and nommu dma allocations is not one of the them.
And Ubuntu should not be enabling CONFIG_DMA_CMA for their i386
and amd64 configurations, as this is trying to drive _all_ dma mapping
allocations through a _very_ small window (which is killing GPU
performance).
Regards,
Peter Hurley
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 14:10 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-26 14:10 UTC (permalink / raw)
To: Rik van Riel, Rob Clark, Thomas Hellstrom
Cc: Chuck Ebbert, Arnd Bergmann, Leann Ogasawara, Shaohua Li,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Kyungmin Park,
Mel Gorman, Andrew Morton, Linus Torvalds, Ingo Molnar,
Marek Szyprowski
[ +cc Leann Ogasawara, Marek Szyprowski, Kyungmin Park, Arnd Bergmann ]
On 09/26/2014 08:40 AM, Rik van Riel wrote:
> On 09/26/2014 08:28 AM, Rob Clark wrote:
>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
>> <thellstrom@vmware.com> wrote:
>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>>> <thellstrom@vmware.com> wrote:
>>>>
>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>
>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>
>>>>>>>
> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>
>>>>>>>
> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>>
>>>>>>>
> Thanks for info, Chuck.
>>>>>>
>>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>>> CMA dma allocation, which is the root problem.
>>>>>>
>>>>>> Regards, Peter Hurley
>>>>> The problem is not really in TTM but in CMA, There was a guy
>>>>> offering to fix this in the CMA code but I guess he didn't
>>>>> probably because he didn't receive any feedback.
>>>>>
>>>> Yeah, the "solution" to this problem seems to be "don't enable
>>>> CMA on x86". Maybe it should even be disabled in the config
>>>> system.
>>> Or, as previously suggested, don't use CMA for order 0 (single
>>> page) allocations....
>>
>> On devices that actually need CMA pools to arrange for memory to be
>> in certain ranges, I think you probably do want to have order 0
>> pages come from the CMA pool.
>>
>> Seems like disabling CMA on x86 (where it should be unneeded) is
>> the better way, IMO
>
> CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
> pages.
>
> There may also be people with devices that do not scatter-gather, and
> need a large physically contiguous buffer, though there should be
> relatively few of those on x86.
>
> I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
> through the normal allocator on x86, and only invoking CMA for larger
> allocations.
The code that uses CMA to satisfy DMA allocations on x86 is
specific to the x86 arch and was added in 2011 as a means of _testing_
CMA in KVM:
commit 0a2b9a6ea93650b8a00f9fd5ee8fdd25671e2df6
Author: Marek Szyprowski <m.szyprowski@samsung.com>
Date: Thu Dec 29 13:09:51 2011 +0100
X86: integrate CMA with DMA-mapping subsystem
This patch adds support for CMA to dma-mapping subsystem for x86
architecture that uses common pci-dma/pci-nommu implementation. This
allows to test CMA on KVM/QEMU and a lot of common x86 boxes.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
CC: Michal Nazarewicz <mina86@mina86.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
(no x86 maintainer acks?).
Unfortunately, this code is enabled whenever CMA is enabled, rather
than as a separate test configuration.
So, while enabling CMA may have other purposes on x86, using it for
x86 swiotlb and nommu dma allocations is not one of the them.
And Ubuntu should not be enabling CONFIG_DMA_CMA for their i386
and amd64 configurations, as this is trying to drive _all_ dma mapping
allocations through a _very_ small window (which is killing GPU
performance).
Regards,
Peter Hurley
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 14:10 ` Peter Hurley
@ 2014-09-26 15:12 ` Leann Ogasawara
-1 siblings, 0 replies; 42+ messages in thread
From: Leann Ogasawara @ 2014-09-26 15:12 UTC (permalink / raw)
To: Peter Hurley
Cc: Rik van Riel, Rob Clark, Thomas Hellstrom, Chuck Ebbert,
Linus Torvalds, Hugh Dickens, Linux kernel, dri-devel, linux-mm,
Mel Gorman, Andrew Morton, Shaohua Li, Ingo Molnar,
Marek Szyprowski, Kyungmin Park, Arnd Bergmann
On Fri, Sep 26, 2014 at 7:10 AM, Peter Hurley <peter@hurleysoftware.com> wrote:
> [ +cc Leann Ogasawara, Marek Szyprowski, Kyungmin Park, Arnd Bergmann ]
>
> On 09/26/2014 08:40 AM, Rik van Riel wrote:
>> On 09/26/2014 08:28 AM, Rob Clark wrote:
>>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
>>> <thellstrom@vmware.com> wrote:
>>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>>>> <thellstrom@vmware.com> wrote:
>>>>>
>>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>>
>>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>>
>>>>>>>>
>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>>
>>>>>>>>
>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>>>
>>>>>>>>
>> Thanks for info, Chuck.
>>>>>>>
>>>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>>>> CMA dma allocation, which is the root problem.
>>>>>>>
>>>>>>> Regards, Peter Hurley
>>>>>> The problem is not really in TTM but in CMA, There was a guy
>>>>>> offering to fix this in the CMA code but I guess he didn't
>>>>>> probably because he didn't receive any feedback.
>>>>>>
>>>>> Yeah, the "solution" to this problem seems to be "don't enable
>>>>> CMA on x86". Maybe it should even be disabled in the config
>>>>> system.
>>>> Or, as previously suggested, don't use CMA for order 0 (single
>>>> page) allocations....
>>>
>>> On devices that actually need CMA pools to arrange for memory to be
>>> in certain ranges, I think you probably do want to have order 0
>>> pages come from the CMA pool.
>>>
>>> Seems like disabling CMA on x86 (where it should be unneeded) is
>>> the better way, IMO
>>
>> CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
>> pages.
>>
>> There may also be people with devices that do not scatter-gather, and
>> need a large physically contiguous buffer, though there should be
>> relatively few of those on x86.
>>
>> I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
>> through the normal allocator on x86, and only invoking CMA for larger
>> allocations.
>
> The code that uses CMA to satisfy DMA allocations on x86 is
> specific to the x86 arch and was added in 2011 as a means of _testing_
> CMA in KVM:
>
> commit 0a2b9a6ea93650b8a00f9fd5ee8fdd25671e2df6
> Author: Marek Szyprowski <m.szyprowski@samsung.com>
> Date: Thu Dec 29 13:09:51 2011 +0100
>
> X86: integrate CMA with DMA-mapping subsystem
>
> This patch adds support for CMA to dma-mapping subsystem for x86
> architecture that uses common pci-dma/pci-nommu implementation. This
> allows to test CMA on KVM/QEMU and a lot of common x86 boxes.
>
> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> CC: Michal Nazarewicz <mina86@mina86.com>
> Acked-by: Arnd Bergmann <arnd@arndb.de>
>
> (no x86 maintainer acks?).
>
> Unfortunately, this code is enabled whenever CMA is enabled, rather
> than as a separate test configuration.
>
> So, while enabling CMA may have other purposes on x86, using it for
> x86 swiotlb and nommu dma allocations is not one of the them.
>
> And Ubuntu should not be enabling CONFIG_DMA_CMA for their i386
> and amd64 configurations, as this is trying to drive _all_ dma mapping
> allocations through a _very_ small window (which is killing GPU
> performance).
Thanks for the note Peter. We do have this disabled for our upcoming
Ubuntu 14.10 release. It is however still enabled in the previous 14.04
release. We have been tracking this in
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1362261 but users
able to reproduce performance impacts in 14.10 were unable to reproduce
in 14.04 which is why we hadn't yet disabled it there.
Thanks,
Leann
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-26 15:12 ` Leann Ogasawara
0 siblings, 0 replies; 42+ messages in thread
From: Leann Ogasawara @ 2014-09-26 15:12 UTC (permalink / raw)
To: Peter Hurley
Cc: Rik van Riel, Rob Clark, Thomas Hellstrom, Chuck Ebbert,
Linus Torvalds, Hugh Dickens, Linux kernel, dri-devel, linux-mm,
Mel Gorman, Andrew Morton, Shaohua Li, Ingo Molnar,
Marek Szyprowski, Kyungmin Park, Arnd Bergmann
On Fri, Sep 26, 2014 at 7:10 AM, Peter Hurley <peter@hurleysoftware.com> wrote:
> [ +cc Leann Ogasawara, Marek Szyprowski, Kyungmin Park, Arnd Bergmann ]
>
> On 09/26/2014 08:40 AM, Rik van Riel wrote:
>> On 09/26/2014 08:28 AM, Rob Clark wrote:
>>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
>>> <thellstrom@vmware.com> wrote:
>>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>>>> <thellstrom@vmware.com> wrote:
>>>>>
>>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>>
>>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>>
>>>>>>>>
>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>>
>>>>>>>>
>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>>>
>>>>>>>>
>> Thanks for info, Chuck.
>>>>>>>
>>>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>>>> CMA dma allocation, which is the root problem.
>>>>>>>
>>>>>>> Regards, Peter Hurley
>>>>>> The problem is not really in TTM but in CMA, There was a guy
>>>>>> offering to fix this in the CMA code but I guess he didn't
>>>>>> probably because he didn't receive any feedback.
>>>>>>
>>>>> Yeah, the "solution" to this problem seems to be "don't enable
>>>>> CMA on x86". Maybe it should even be disabled in the config
>>>>> system.
>>>> Or, as previously suggested, don't use CMA for order 0 (single
>>>> page) allocations....
>>>
>>> On devices that actually need CMA pools to arrange for memory to be
>>> in certain ranges, I think you probably do want to have order 0
>>> pages come from the CMA pool.
>>>
>>> Seems like disabling CMA on x86 (where it should be unneeded) is
>>> the better way, IMO
>>
>> CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
>> pages.
>>
>> There may also be people with devices that do not scatter-gather, and
>> need a large physically contiguous buffer, though there should be
>> relatively few of those on x86.
>>
>> I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
>> through the normal allocator on x86, and only invoking CMA for larger
>> allocations.
>
> The code that uses CMA to satisfy DMA allocations on x86 is
> specific to the x86 arch and was added in 2011 as a means of _testing_
> CMA in KVM:
>
> commit 0a2b9a6ea93650b8a00f9fd5ee8fdd25671e2df6
> Author: Marek Szyprowski <m.szyprowski@samsung.com>
> Date: Thu Dec 29 13:09:51 2011 +0100
>
> X86: integrate CMA with DMA-mapping subsystem
>
> This patch adds support for CMA to dma-mapping subsystem for x86
> architecture that uses common pci-dma/pci-nommu implementation. This
> allows to test CMA on KVM/QEMU and a lot of common x86 boxes.
>
> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
> CC: Michal Nazarewicz <mina86@mina86.com>
> Acked-by: Arnd Bergmann <arnd@arndb.de>
>
> (no x86 maintainer acks?).
>
> Unfortunately, this code is enabled whenever CMA is enabled, rather
> than as a separate test configuration.
>
> So, while enabling CMA may have other purposes on x86, using it for
> x86 swiotlb and nommu dma allocations is not one of the them.
>
> And Ubuntu should not be enabling CONFIG_DMA_CMA for their i386
> and amd64 configurations, as this is trying to drive _all_ dma mapping
> allocations through a _very_ small window (which is killing GPU
> performance).
Thanks for the note Peter. We do have this disabled for our upcoming
Ubuntu 14.10 release. It is however still enabled in the previous 14.04
release. We have been tracking this in
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1362261 but users
able to reproduce performance impacts in 14.10 were unable to reproduce
in 14.04 which is why we hadn't yet disabled it there.
Thanks,
Leann
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-26 15:12 ` Leann Ogasawara
@ 2014-09-27 14:15 ` Peter Hurley
-1 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-27 14:15 UTC (permalink / raw)
To: Leann Ogasawara, Andrew Morton
Cc: Rik van Riel, Rob Clark, Thomas Hellstrom, Chuck Ebbert,
Linus Torvalds, Hugh Dickens, Linux kernel, dri-devel, linux-mm,
Mel Gorman, Shaohua Li, Ingo Molnar, Marek Szyprowski,
Kyungmin Park, Arnd Bergmann, Akinobu Mita
On 09/26/2014 11:12 AM, Leann Ogasawara wrote:
> On Fri, Sep 26, 2014 at 7:10 AM, Peter Hurley <peter@hurleysoftware.com> wrote:
>> [ +cc Leann Ogasawara, Marek Szyprowski, Kyungmin Park, Arnd Bergmann ]
>>
>> On 09/26/2014 08:40 AM, Rik van Riel wrote:
>>> On 09/26/2014 08:28 AM, Rob Clark wrote:
>>>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
>>>> <thellstrom@vmware.com> wrote:
>>>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>>>>> <thellstrom@vmware.com> wrote:
>>>>>>
>>>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>>>
>>>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>>>
>>>>>>>>>
>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>>>
>>>>>>>>>
>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>>>>
>>>>>>>>>
>>> Thanks for info, Chuck.
>>>>>>>>
>>>>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>>>>> CMA dma allocation, which is the root problem.
>>>>>>>>
>>>>>>>> Regards, Peter Hurley
>>>>>>> The problem is not really in TTM but in CMA, There was a guy
>>>>>>> offering to fix this in the CMA code but I guess he didn't
>>>>>>> probably because he didn't receive any feedback.
>>>>>>>
>>>>>> Yeah, the "solution" to this problem seems to be "don't enable
>>>>>> CMA on x86". Maybe it should even be disabled in the config
>>>>>> system.
>>>>> Or, as previously suggested, don't use CMA for order 0 (single
>>>>> page) allocations....
>>>>
>>>> On devices that actually need CMA pools to arrange for memory to be
>>>> in certain ranges, I think you probably do want to have order 0
>>>> pages come from the CMA pool.
>>>>
>>>> Seems like disabling CMA on x86 (where it should be unneeded) is
>>>> the better way, IMO
>>>
>>> CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
>>> pages.
>>>
>>> There may also be people with devices that do not scatter-gather, and
>>> need a large physically contiguous buffer, though there should be
>>> relatively few of those on x86.
>>>
>>> I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
>>> through the normal allocator on x86, and only invoking CMA for larger
>>> allocations.
>>
>> The code that uses CMA to satisfy DMA allocations on x86 is
>> specific to the x86 arch and was added in 2011 as a means of _testing_
>> CMA in KVM:
>>
>> commit 0a2b9a6ea93650b8a00f9fd5ee8fdd25671e2df6
>> Author: Marek Szyprowski <m.szyprowski@samsung.com>
>> Date: Thu Dec 29 13:09:51 2011 +0100
>>
>> X86: integrate CMA with DMA-mapping subsystem
>>
>> This patch adds support for CMA to dma-mapping subsystem for x86
>> architecture that uses common pci-dma/pci-nommu implementation. This
>> allows to test CMA on KVM/QEMU and a lot of common x86 boxes.
>>
>> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
>> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
>> CC: Michal Nazarewicz <mina86@mina86.com>
>> Acked-by: Arnd Bergmann <arnd@arndb.de>
>>
>> (no x86 maintainer acks?).
>>
>> Unfortunately, this code is enabled whenever CMA is enabled, rather
>> than as a separate test configuration.
>>
>> So, while enabling CMA may have other purposes on x86, using it for
>> x86 swiotlb and nommu dma allocations is not one of the them.
>>
>> And Ubuntu should not be enabling CONFIG_DMA_CMA for their i386
>> and amd64 configurations, as this is trying to drive _all_ dma mapping
>> allocations through a _very_ small window (which is killing GPU
>> performance).
>
> Thanks for the note Peter. We do have this disabled for our upcoming
> Ubuntu 14.10 release. It is however still enabled in the previous 14.04
> release. We have been tracking this in
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1362261 but users
> able to reproduce performance impacts in 14.10 were unable to reproduce
> in 14.04 which is why we hadn't yet disabled it there.
Leann,
Thanks for that important clue.
The missing piece specific to 3.16+ is these patches which impact every
iommu config:
Akinobu Mita (5):
x86: make dma_alloc_coherent() return zeroed memory if CMA is enabled
x86: enable DMA CMA with swiotlb
intel-iommu: integrate DMA CMA
memblock: introduce memblock_alloc_range()
cma: add placement specifier for "cma=" kernel parameter
These patches take the pre-existing nommu CMA test configuration and
hook it up to all the x86 iommus, effectively reducing 10GB of DMA-able
memory to 64MB, and hooks it all up to an allocator that's not nearly as
effective as the page allocator.
All to enable DMA allocation below 4GB which is already supported with the
GFP_DMA32 flag to dma_alloc_coherent().
Regards,
Peter Hurley
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-09-27 14:15 ` Peter Hurley
0 siblings, 0 replies; 42+ messages in thread
From: Peter Hurley @ 2014-09-27 14:15 UTC (permalink / raw)
To: Leann Ogasawara, Andrew Morton
Cc: Rik van Riel, Rob Clark, Thomas Hellstrom, Chuck Ebbert,
Linus Torvalds, Hugh Dickens, Linux kernel, dri-devel, linux-mm,
Mel Gorman, Shaohua Li, Ingo Molnar, Marek Szyprowski,
Kyungmin Park, Arnd Bergmann, Akinobu Mita
On 09/26/2014 11:12 AM, Leann Ogasawara wrote:
> On Fri, Sep 26, 2014 at 7:10 AM, Peter Hurley <peter@hurleysoftware.com> wrote:
>> [ +cc Leann Ogasawara, Marek Szyprowski, Kyungmin Park, Arnd Bergmann ]
>>
>> On 09/26/2014 08:40 AM, Rik van Riel wrote:
>>> On 09/26/2014 08:28 AM, Rob Clark wrote:
>>>> On Fri, Sep 26, 2014 at 6:45 AM, Thomas Hellstrom
>>>> <thellstrom@vmware.com> wrote:
>>>>> On 09/26/2014 12:40 PM, Chuck Ebbert wrote:
>>>>>> On Fri, 26 Sep 2014 09:15:57 +0200 Thomas Hellstrom
>>>>>> <thellstrom@vmware.com> wrote:
>>>>>>
>>>>>>> On 09/26/2014 01:52 AM, Peter Hurley wrote:
>>>>>>>> On 09/25/2014 03:35 PM, Chuck Ebbert wrote:
>>>>>>>>> There are six ttm patches queued for 3.16.4:
>>>>>>>>>
>>>>>>>>> drm-ttm-choose-a-pool-to-shrink-correctly-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>>>
>>>>>>>>>
>>> drm-ttm-fix-handling-of-ttm_pl_flag_topdown-v2.patch
>>>>>>>>> drm-ttm-fix-possible-division-by-0-in-ttm_dma_pool_shrink_scan.patch
>>>>>>>>>
>>>>>>>>>
>>> drm-ttm-fix-possible-stack-overflow-by-recursive-shrinker-calls.patch
>>>>>>>>> drm-ttm-pass-gfp-flags-in-order-to-avoid-deadlock.patch
>>>>>>>>> drm-ttm-use-mutex_trylock-to-avoid-deadlock-inside-shrinker-functions.patch
>>>>>>>>
>>>>>>>>>
>>> Thanks for info, Chuck.
>>>>>>>>
>>>>>>>> Unfortunately, none of these fix TTM dma allocation doing
>>>>>>>> CMA dma allocation, which is the root problem.
>>>>>>>>
>>>>>>>> Regards, Peter Hurley
>>>>>>> The problem is not really in TTM but in CMA, There was a guy
>>>>>>> offering to fix this in the CMA code but I guess he didn't
>>>>>>> probably because he didn't receive any feedback.
>>>>>>>
>>>>>> Yeah, the "solution" to this problem seems to be "don't enable
>>>>>> CMA on x86". Maybe it should even be disabled in the config
>>>>>> system.
>>>>> Or, as previously suggested, don't use CMA for order 0 (single
>>>>> page) allocations....
>>>>
>>>> On devices that actually need CMA pools to arrange for memory to be
>>>> in certain ranges, I think you probably do want to have order 0
>>>> pages come from the CMA pool.
>>>>
>>>> Seems like disabling CMA on x86 (where it should be unneeded) is
>>>> the better way, IMO
>>>
>>> CMA has its uses on x86. For example, CMA is used to allocate 1GB huge
>>> pages.
>>>
>>> There may also be people with devices that do not scatter-gather, and
>>> need a large physically contiguous buffer, though there should be
>>> relatively few of those on x86.
>>>
>>> I suspect it makes most sense to do DMA allocations up to PAGE_ORDER
>>> through the normal allocator on x86, and only invoking CMA for larger
>>> allocations.
>>
>> The code that uses CMA to satisfy DMA allocations on x86 is
>> specific to the x86 arch and was added in 2011 as a means of _testing_
>> CMA in KVM:
>>
>> commit 0a2b9a6ea93650b8a00f9fd5ee8fdd25671e2df6
>> Author: Marek Szyprowski <m.szyprowski@samsung.com>
>> Date: Thu Dec 29 13:09:51 2011 +0100
>>
>> X86: integrate CMA with DMA-mapping subsystem
>>
>> This patch adds support for CMA to dma-mapping subsystem for x86
>> architecture that uses common pci-dma/pci-nommu implementation. This
>> allows to test CMA on KVM/QEMU and a lot of common x86 boxes.
>>
>> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
>> Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
>> CC: Michal Nazarewicz <mina86@mina86.com>
>> Acked-by: Arnd Bergmann <arnd@arndb.de>
>>
>> (no x86 maintainer acks?).
>>
>> Unfortunately, this code is enabled whenever CMA is enabled, rather
>> than as a separate test configuration.
>>
>> So, while enabling CMA may have other purposes on x86, using it for
>> x86 swiotlb and nommu dma allocations is not one of the them.
>>
>> And Ubuntu should not be enabling CONFIG_DMA_CMA for their i386
>> and amd64 configurations, as this is trying to drive _all_ dma mapping
>> allocations through a _very_ small window (which is killing GPU
>> performance).
>
> Thanks for the note Peter. We do have this disabled for our upcoming
> Ubuntu 14.10 release. It is however still enabled in the previous 14.04
> release. We have been tracking this in
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1362261 but users
> able to reproduce performance impacts in 14.10 were unable to reproduce
> in 14.04 which is why we hadn't yet disabled it there.
Leann,
Thanks for that important clue.
The missing piece specific to 3.16+ is these patches which impact every
iommu config:
Akinobu Mita (5):
x86: make dma_alloc_coherent() return zeroed memory if CMA is enabled
x86: enable DMA CMA with swiotlb
intel-iommu: integrate DMA CMA
memblock: introduce memblock_alloc_range()
cma: add placement specifier for "cma=" kernel parameter
These patches take the pre-existing nommu CMA test configuration and
hook it up to all the x86 iommus, effectively reducing 10GB of DMA-able
memory to 64MB, and hooks it all up to an allocator that's not nearly as
effective as the page allocator.
All to enable DMA allocation below 4GB which is already supported with the
GFP_DMA32 flag to dma_alloc_coherent().
Regards,
Peter Hurley
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
2014-09-25 21:10 ` Peter Hurley
@ 2014-10-01 9:26 ` Maarten Lankhorst
-1 siblings, 0 replies; 42+ messages in thread
From: Maarten Lankhorst @ 2014-10-01 9:26 UTC (permalink / raw)
To: Peter Hurley, Alex Deucher
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Thomas Hellstrom,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Andrew Morton,
Linus Torvalds, Ingo Molnar
Op 25-09-14 om 23:10 schreef Peter Hurley:
> On 09/25/2014 04:33 PM, Alex Deucher wrote:
>> On Thu, Sep 25, 2014 at 2:55 PM, Peter Hurley <peter@hurleysoftware.com> wrote:
>>> After several days uptime with a 3.16 kernel (generally running
>>> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
>>> desktop workspaces) I've been seeing some really extreme slowdowns.
>>>
>>> Mostly the slowdowns are associated with gpu-related tasks, like
>>> opening new emacs windows, switching workspaces, laughing at internet
>>> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
>>> it right away -- 3.15 is the first time suspend has worked reliably.
>>>
>>> This week I started looking into what the slowdown was and discovered
>>> it's happening during dma allocation through swiotlb (the cpus can do
>>> intel iommu but I don't use it because it's not the default for most users).
>>>
>>> I'm still working on a bisection but each step takes 8+ hours to
>>> validate and even then I'm no longer sure I still have the 'bad'
>>> commit in the bisection. [edit: yup, I started over]
>>>
>>> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
>>> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>>>
>>> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
>>> fragmentation [1].
>>>
>>> Besides Mel's page allocator changes in 3.16, another suspect commit is:
>>>
>>> commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
>>> Author: Shaohua Li <shli@kernel.org>
>>> Date: Tue Apr 8 15:58:09 2014 +0800
>>>
>>> x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
>>>
>>> Specifically, this statement:
>>>
>>> It could cause incorrect page aging and the (mistaken) reclaim of
>>> hot pages, but the chance of that should be relatively low.
>>>
>>> I'm wondering if this could cause worse-case behavior with TTM? I'm
>>> testing a revert of this on mainline 3.16-final now, with no results yet.
>>>
>>> Thoughts?
>> You may also be seeing this:
>> https://lkml.org/lkml/2014/8/8/445
> Thanks Alex. That is indeed the problem.
>
> Still reading the email thread to find out where the patches
> are that fix this. Although it doesn't make much sense to me
> that nouveau sets up a 1GB GART and then uses TTM which is
> trying to shove all the DMA through a 16MB CMA window
> (which turns out to be the base Ubuntu config).
>
> Regards,
> Peter Hurley
>
>
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1362261
CMA's already disabled on x86 in most recent ubuntu kernels. :-)
~Maarten
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: page allocator bug in 3.16?
@ 2014-10-01 9:26 ` Maarten Lankhorst
0 siblings, 0 replies; 42+ messages in thread
From: Maarten Lankhorst @ 2014-10-01 9:26 UTC (permalink / raw)
To: Peter Hurley, Alex Deucher
Cc: Mel Gorman, Shaohua Li, Rik van Riel, Thomas Hellstrom,
Hugh Dickens, Linux kernel, dri-devel, linux-mm, Andrew Morton,
Linus Torvalds, Ingo Molnar
Op 25-09-14 om 23:10 schreef Peter Hurley:
> On 09/25/2014 04:33 PM, Alex Deucher wrote:
>> On Thu, Sep 25, 2014 at 2:55 PM, Peter Hurley <peter@hurleysoftware.com> wrote:
>>> After several days uptime with a 3.16 kernel (generally running
>>> Thunderbird, emacs, kernel builds, several Chrome tabs on multiple
>>> desktop workspaces) I've been seeing some really extreme slowdowns.
>>>
>>> Mostly the slowdowns are associated with gpu-related tasks, like
>>> opening new emacs windows, switching workspaces, laughing at internet
>>> gifs, etc. Because this x86_64 desktop is nouveau-based, I didn't pursue
>>> it right away -- 3.15 is the first time suspend has worked reliably.
>>>
>>> This week I started looking into what the slowdown was and discovered
>>> it's happening during dma allocation through swiotlb (the cpus can do
>>> intel iommu but I don't use it because it's not the default for most users).
>>>
>>> I'm still working on a bisection but each step takes 8+ hours to
>>> validate and even then I'm no longer sure I still have the 'bad'
>>> commit in the bisection. [edit: yup, I started over]
>>>
>>> I just discovered a smattering of these in my logs and only on 3.16-rc+ kernels:
>>> Sep 25 07:57:59 thor kernel: [28786.001300] alloc_contig_range test_pages_isolated(2bf560, 2bf562) failed
>>>
>>> This dual-Xeon box has 10GB and sysrq Show Memory isn't showing heavy
>>> fragmentation [1].
>>>
>>> Besides Mel's page allocator changes in 3.16, another suspect commit is:
>>>
>>> commit b13b1d2d8692b437203de7a404c6b809d2cc4d99
>>> Author: Shaohua Li <shli@kernel.org>
>>> Date: Tue Apr 8 15:58:09 2014 +0800
>>>
>>> x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB
>>>
>>> Specifically, this statement:
>>>
>>> It could cause incorrect page aging and the (mistaken) reclaim of
>>> hot pages, but the chance of that should be relatively low.
>>>
>>> I'm wondering if this could cause worse-case behavior with TTM? I'm
>>> testing a revert of this on mainline 3.16-final now, with no results yet.
>>>
>>> Thoughts?
>> You may also be seeing this:
>> https://lkml.org/lkml/2014/8/8/445
> Thanks Alex. That is indeed the problem.
>
> Still reading the email thread to find out where the patches
> are that fix this. Although it doesn't make much sense to me
> that nouveau sets up a 1GB GART and then uses TTM which is
> trying to shove all the DMA through a 16MB CMA window
> (which turns out to be the base Ubuntu config).
>
> Regards,
> Peter Hurley
>
>
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1362261
CMA's already disabled on x86 in most recent ubuntu kernels. :-)
~Maarten
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 42+ messages in thread
end of thread, other threads:[~2014-10-01 9:26 UTC | newest]
Thread overview: 42+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-25 18:55 page allocator bug in 3.16? Peter Hurley
2014-09-25 18:55 ` Peter Hurley
2014-09-25 18:55 ` Peter Hurley
2014-09-25 19:34 ` Maarten Lankhorst
2014-09-25 19:34 ` Maarten Lankhorst
2014-09-25 19:34 ` Maarten Lankhorst
2014-09-25 19:35 ` Chuck Ebbert
2014-09-25 19:35 ` Chuck Ebbert
2014-09-25 23:52 ` Peter Hurley
2014-09-25 23:52 ` Peter Hurley
2014-09-26 7:15 ` Thomas Hellstrom
2014-09-26 7:15 ` Thomas Hellstrom
2014-09-26 7:15 ` Thomas Hellstrom
2014-09-26 10:40 ` Chuck Ebbert
2014-09-26 10:40 ` Chuck Ebbert
2014-09-26 10:45 ` Thomas Hellstrom
2014-09-26 10:45 ` Thomas Hellstrom
2014-09-26 12:28 ` Rob Clark
2014-09-26 12:28 ` Rob Clark
2014-09-26 12:28 ` Rob Clark
2014-09-26 12:34 ` Thomas Hellstrom
2014-09-26 12:34 ` Thomas Hellstrom
2014-09-26 13:40 ` Rob Clark
2014-09-26 13:40 ` Rob Clark
2014-09-26 12:40 ` Rik van Riel
2014-09-26 12:40 ` Rik van Riel
2014-09-26 14:10 ` Peter Hurley
2014-09-26 14:10 ` Peter Hurley
2014-09-26 14:10 ` Peter Hurley
2014-09-26 15:12 ` Leann Ogasawara
2014-09-26 15:12 ` Leann Ogasawara
2014-09-27 14:15 ` Peter Hurley
2014-09-27 14:15 ` Peter Hurley
2014-09-25 20:33 ` Alex Deucher
2014-09-25 20:33 ` Alex Deucher
2014-09-25 21:10 ` Peter Hurley
2014-09-25 21:10 ` Peter Hurley
2014-10-01 9:26 ` Maarten Lankhorst
2014-10-01 9:26 ` Maarten Lankhorst
2014-09-25 20:36 ` Peter Hurley
2014-09-25 20:36 ` Peter Hurley
2014-09-25 20:36 ` Peter Hurley
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.