All of lore.kernel.org
 help / color / mirror / Atom feed
* khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y
@ 2019-10-29 20:15 Ville Syrjälä
  2019-10-29 21:02 ` Andrew Morton
  2019-10-29 23:15 ` khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y Jason Gunthorpe
  0 siblings, 2 replies; 6+ messages in thread
From: Ville Syrjälä @ 2019-10-29 20:15 UTC (permalink / raw)
  To: linux-kernel
  Cc: Andrew Morton, Kirill A. Shutemov, Thomas Gleixner, Ingo Molnar,
	Borislav Petkov, H. Peter Anvin, linux-mm

Hi,

I got some khugepaged spew on a 32bit x86:

[  217.490026] BUG: sleeping function called from invalid context at include/linux/mmu_notifier.h:346
[  217.492826] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 25, name: khugepaged
[  217.495589] INFO: lockdep is turned off.
[  217.498371] CPU: 1 PID: 25 Comm: khugepaged Not tainted 5.4.0-rc5-elk+ #206
[  217.501233] Hardware name: System manufacturer P5Q-EM/P5Q-EM, BIOS 2203    07/08/2009
[  217.501697] Call Trace:
[  217.501697]  dump_stack+0x66/0x8e
[  217.501697]  ___might_sleep.cold.96+0x95/0xa6
[  217.501697]  __might_sleep+0x2e/0x80
[  217.501697]  collapse_huge_page.isra.51+0x5ac/0x1360
[  217.501697]  ? __alloc_pages_nodemask+0xec/0xf80
[  217.501697]  ? __alloc_pages_nodemask+0x191/0xf80
[  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
[  217.501697]  khugepaged+0x9a9/0x20f0
[  217.501697]  ? _raw_spin_unlock+0x21/0x30
[  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
[  217.501697]  ? wait_woken+0xa0/0xa0
[  217.501697]  kthread+0xf5/0x110
[  217.501697]  ? collapse_pte_mapped_thp+0x3b0/0x3b0
[  217.501697]  ? kthread_create_worker_on_cpu+0x20/0x20
[  217.501697]  ret_from_fork+0x2e/0x38

Looks like it's due to CONFIG_HIGHPTE=y pte_offset_map()->kmap_atomic() vs.
mmu_notifier_invalidate_range_start().

My naive idea would be to just reorder those things, but not sure
if there's some magic ordering constraint here. At least the machine
still boots when I do it :)

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 0a1b4b484ac5..f05d27b7183d 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1028,12 +1028,13 @@ static void collapse_huge_page(struct mm_struct *mm,
 
 	anon_vma_lock_write(vma->anon_vma);
 
-	pte = pte_offset_map(pmd, address);
-	pte_ptl = pte_lockptr(mm, pmd);
-
 	mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm,
 				address, address + HPAGE_PMD_SIZE);
 	mmu_notifier_invalidate_range_start(&range);
+
+	pte = pte_offset_map(pmd, address);
+	pte_ptl = pte_lockptr(mm, pmd);
+
 	pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */
 	/*
 	 * After this gup_fast can't run anymore. This also removes

-- 
Ville Syrjälä
Intel

^ permalink raw reply related	[flat|nested] 6+ messages in thread
* Re: khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y
  2019-10-29 20:15 khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y Ville Syrjälä
@ 2019-10-29 21:02 ` Andrew Morton
  2019-10-29 21:25   ` [PATCH] mm/khugepaged: Fix might_sleep() warn with CONFIG_HIGHPTE=y Ville Syrjala
  2019-10-29 23:15 ` khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y Jason Gunthorpe
  1 sibling, 1 reply; 6+ messages in thread
From: Andrew Morton @ 2019-10-29 21:02 UTC (permalink / raw)
  To: Ville Syrjälä
  Cc: linux-kernel, Kirill A. Shutemov, Thomas Gleixner, Ingo Molnar,
	Borislav Petkov, H. Peter Anvin, linux-mm,
	Jérôme Glisse, Ralph Campbell, Ira Weiny,
	Jason Gunthorpe, Daniel Vetter, Andrea Arcangeli

On Tue, 29 Oct 2019 22:15:13 +0200 Ville Syrjälä <ville.syrjala@linux.intel.com> wrote:

> Hi,
> 
> I got some khugepaged spew on a 32bit x86:
> 
> [  217.490026] BUG: sleeping function called from invalid context at include/linux/mmu_notifier.h:346
> [  217.492826] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 25, name: khugepaged
> [  217.495589] INFO: lockdep is turned off.
> [  217.498371] CPU: 1 PID: 25 Comm: khugepaged Not tainted 5.4.0-rc5-elk+ #206
> [  217.501233] Hardware name: System manufacturer P5Q-EM/P5Q-EM, BIOS 2203    07/08/2009
> [  217.501697] Call Trace:
> [  217.501697]  dump_stack+0x66/0x8e
> [  217.501697]  ___might_sleep.cold.96+0x95/0xa6
> [  217.501697]  __might_sleep+0x2e/0x80
> [  217.501697]  collapse_huge_page.isra.51+0x5ac/0x1360
> [  217.501697]  ? __alloc_pages_nodemask+0xec/0xf80
> [  217.501697]  ? __alloc_pages_nodemask+0x191/0xf80
> [  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
> [  217.501697]  khugepaged+0x9a9/0x20f0
> [  217.501697]  ? _raw_spin_unlock+0x21/0x30
> [  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
> [  217.501697]  ? wait_woken+0xa0/0xa0
> [  217.501697]  kthread+0xf5/0x110
> [  217.501697]  ? collapse_pte_mapped_thp+0x3b0/0x3b0
> [  217.501697]  ? kthread_create_worker_on_cpu+0x20/0x20
> [  217.501697]  ret_from_fork+0x2e/0x38
> 
> Looks like it's due to CONFIG_HIGHPTE=y pte_offset_map()->kmap_atomic() vs.
> mmu_notifier_invalidate_range_start().
> 
> My naive idea would be to just reorder those things, but not sure
> if there's some magic ordering constraint here. At least the machine
> still boots when I do it :)
> 
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index 0a1b4b484ac5..f05d27b7183d 100644
> --- a/mm/khugepaged.c
> +++ b/mm/khugepaged.c
> @@ -1028,12 +1028,13 @@ static void collapse_huge_page(struct mm_struct *mm,
>  
>  	anon_vma_lock_write(vma->anon_vma);
>  
> -	pte = pte_offset_map(pmd, address);
> -	pte_ptl = pte_lockptr(mm, pmd);
> -
>  	mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm,
>  				address, address + HPAGE_PMD_SIZE);
>  	mmu_notifier_invalidate_range_start(&range);
> +
> +	pte = pte_offset_map(pmd, address);
> +	pte_ptl = pte_lockptr(mm, pmd);
> +
>  	pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */
>  	/*
>  	 * After this gup_fast can't run anymore. This also removes
> 

Looks good to me.  Can you resend it with a signoff please?

^ permalink raw reply	[flat|nested] 6+ messages in thread
* [PATCH] mm/khugepaged: Fix might_sleep() warn with CONFIG_HIGHPTE=y
  2019-10-29 21:02 ` Andrew Morton
@ 2019-10-29 21:25   ` Ville Syrjala
  2019-10-29 21:43     ` Andrew Morton
  2019-10-30  7:08     ` Kirill A. Shutemov
  0 siblings, 2 replies; 6+ messages in thread
From: Ville Syrjala @ 2019-10-29 21:25 UTC (permalink / raw)
  To: linux-kernel
  Cc: Andrew Morton, Kirill A. Shutemov, Thomas Gleixner, Ingo Molnar,
	Borislav Petkov, H. Peter Anvin, linux-mm,
	Ville Syrjälä

From: Ville Syrjälä <ville.syrjala@linux.intel.com>

I got some khugepaged spew on a 32bit x86:

[  217.490026] BUG: sleeping function called from invalid context at include/linux/mmu_notifier.h:346
[  217.492826] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 25, name: khugepaged
[  217.495589] INFO: lockdep is turned off.
[  217.498371] CPU: 1 PID: 25 Comm: khugepaged Not tainted 5.4.0-rc5-elk+ #206
[  217.501233] Hardware name: System manufacturer P5Q-EM/P5Q-EM, BIOS 2203    07/08/2009
[  217.501697] Call Trace:
[  217.501697]  dump_stack+0x66/0x8e
[  217.501697]  ___might_sleep.cold.96+0x95/0xa6
[  217.501697]  __might_sleep+0x2e/0x80
[  217.501697]  collapse_huge_page.isra.51+0x5ac/0x1360
[  217.501697]  ? __alloc_pages_nodemask+0xec/0xf80
[  217.501697]  ? __alloc_pages_nodemask+0x191/0xf80
[  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
[  217.501697]  khugepaged+0x9a9/0x20f0
[  217.501697]  ? _raw_spin_unlock+0x21/0x30
[  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
[  217.501697]  ? wait_woken+0xa0/0xa0
[  217.501697]  kthread+0xf5/0x110
[  217.501697]  ? collapse_pte_mapped_thp+0x3b0/0x3b0
[  217.501697]  ? kthread_create_worker_on_cpu+0x20/0x20
[  217.501697]  ret_from_fork+0x2e/0x38

Looks like it's due to CONFIG_HIGHPTE=y pte_offset_map()->kmap_atomic()
vs. mmu_notifier_invalidate_range_start(). Let's do the naive approach
and just reorder the two operations.

Cc: linux-kernel@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: linux-mm@kvack.org
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
 mm/khugepaged.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 0a1b4b484ac5..f05d27b7183d 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1028,12 +1028,13 @@ static void collapse_huge_page(struct mm_struct *mm,
 
 	anon_vma_lock_write(vma->anon_vma);
 
-	pte = pte_offset_map(pmd, address);
-	pte_ptl = pte_lockptr(mm, pmd);
-
 	mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm,
 				address, address + HPAGE_PMD_SIZE);
 	mmu_notifier_invalidate_range_start(&range);
+
+	pte = pte_offset_map(pmd, address);
+	pte_ptl = pte_lockptr(mm, pmd);
+
 	pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */
 	/*
 	 * After this gup_fast can't run anymore. This also removes
-- 
2.23.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread
* Re: [PATCH] mm/khugepaged: Fix might_sleep() warn with CONFIG_HIGHPTE=y
  2019-10-29 21:25   ` [PATCH] mm/khugepaged: Fix might_sleep() warn with CONFIG_HIGHPTE=y Ville Syrjala
@ 2019-10-29 21:43     ` Andrew Morton
  2019-10-30  7:08     ` Kirill A. Shutemov
  1 sibling, 0 replies; 6+ messages in thread
From: Andrew Morton @ 2019-10-29 21:43 UTC (permalink / raw)
  To: Ville Syrjala
  Cc: linux-kernel, Kirill A. Shutemov, Thomas Gleixner, Ingo Molnar,
	Borislav Petkov, H. Peter Anvin, linux-mm,
	Jérôme Glisse, Ralph Campbell, Ira Weiny,
	Jason Gunthorpe, Daniel Vetter, Andrea Arcangeli

On Tue, 29 Oct 2019 23:25:13 +0200 Ville Syrjala <ville.syrjala@linux.intel.com> wrote:

> From: Ville Syrjälä <ville.syrjala@linux.intel.com>
> 
> I got some khugepaged spew on a 32bit x86:
> 
> [  217.490026] BUG: sleeping function called from invalid context at include/linux/mmu_notifier.h:346
> [  217.492826] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 25, name: khugepaged
> [  217.495589] INFO: lockdep is turned off.
> [  217.498371] CPU: 1 PID: 25 Comm: khugepaged Not tainted 5.4.0-rc5-elk+ #206
> [  217.501233] Hardware name: System manufacturer P5Q-EM/P5Q-EM, BIOS 2203    07/08/2009
> [  217.501697] Call Trace:
> [  217.501697]  dump_stack+0x66/0x8e
> [  217.501697]  ___might_sleep.cold.96+0x95/0xa6
> [  217.501697]  __might_sleep+0x2e/0x80
> [  217.501697]  collapse_huge_page.isra.51+0x5ac/0x1360
> [  217.501697]  ? __alloc_pages_nodemask+0xec/0xf80
> [  217.501697]  ? __alloc_pages_nodemask+0x191/0xf80
> [  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
> [  217.501697]  khugepaged+0x9a9/0x20f0
> [  217.501697]  ? _raw_spin_unlock+0x21/0x30
> [  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
> [  217.501697]  ? wait_woken+0xa0/0xa0
> [  217.501697]  kthread+0xf5/0x110
> [  217.501697]  ? collapse_pte_mapped_thp+0x3b0/0x3b0
> [  217.501697]  ? kthread_create_worker_on_cpu+0x20/0x20
> [  217.501697]  ret_from_fork+0x2e/0x38
> 
> Looks like it's due to CONFIG_HIGHPTE=y pte_offset_map()->kmap_atomic()
> vs. mmu_notifier_invalidate_range_start(). Let's do the naive approach
> and just reorder the two operations.

Thanks.

I'll add 
Fixes: 810e24e009cf71 ("mm/mmu_notifiers: annotate with might_sleep()")
Cc: <stable@vger.kernel.org>

although 810e24e009cf71 merely exposed the bug.

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y
  2019-10-29 20:15 khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y Ville Syrjälä
  2019-10-29 21:02 ` Andrew Morton
@ 2019-10-29 23:15 ` Jason Gunthorpe
  1 sibling, 0 replies; 6+ messages in thread
From: Jason Gunthorpe @ 2019-10-29 23:15 UTC (permalink / raw)
  To: Ville Syrjälä
  Cc: linux-kernel, Andrew Morton, Kirill A. Shutemov, Thomas Gleixner,
	Ingo Molnar, Borislav Petkov, H. Peter Anvin, linux-mm

On Tue, Oct 29, 2019 at 10:15:13PM +0200, Ville Syrjälä wrote:
> Hi,
> 
> I got some khugepaged spew on a 32bit x86:
> 
> [  217.490026] BUG: sleeping function called from invalid context at include/linux/mmu_notifier.h:346
> [  217.492826] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 25, name: khugepaged
> [  217.495589] INFO: lockdep is turned off.
> [  217.498371] CPU: 1 PID: 25 Comm: khugepaged Not tainted 5.4.0-rc5-elk+ #206
> [  217.501233] Hardware name: System manufacturer P5Q-EM/P5Q-EM, BIOS 2203    07/08/2009
> [  217.501697] Call Trace:
> [  217.501697]  dump_stack+0x66/0x8e
> [  217.501697]  ___might_sleep.cold.96+0x95/0xa6
> [  217.501697]  __might_sleep+0x2e/0x80
> [  217.501697]  collapse_huge_page.isra.51+0x5ac/0x1360
> [  217.501697]  ? __alloc_pages_nodemask+0xec/0xf80
> [  217.501697]  ? __alloc_pages_nodemask+0x191/0xf80
> [  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
> [  217.501697]  khugepaged+0x9a9/0x20f0
> [  217.501697]  ? _raw_spin_unlock+0x21/0x30
> [  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
> [  217.501697]  ? wait_woken+0xa0/0xa0
> [  217.501697]  kthread+0xf5/0x110
> [  217.501697]  ? collapse_pte_mapped_thp+0x3b0/0x3b0
> [  217.501697]  ? kthread_create_worker_on_cpu+0x20/0x20
> [  217.501697]  ret_from_fork+0x2e/0x38
> 
> Looks like it's due to CONFIG_HIGHPTE=y pte_offset_map()->kmap_atomic() vs.
> mmu_notifier_invalidate_range_start().
> 
> My naive idea would be to just reorder those things, but not sure
> if there's some magic ordering constraint here. At least the machine
> still boots when I do it :)
> 
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index 0a1b4b484ac5..f05d27b7183d 100644
> +++ b/mm/khugepaged.c
> @@ -1028,12 +1028,13 @@ static void collapse_huge_page(struct mm_struct *mm,
>  
>  	anon_vma_lock_write(vma->anon_vma);
>  
> -	pte = pte_offset_map(pmd, address);
> -	pte_ptl = pte_lockptr(mm, pmd);
> -
>  	mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm,
>  				address, address + HPAGE_PMD_SIZE);
>  	mmu_notifier_invalidate_range_start(&range);
> +
> +	pte = pte_offset_map(pmd, address);
> +	pte_ptl = pte_lockptr(mm, pmd);
> +

Since pte and pte_ptl don't leak into invalidate_range_start this
seems reasonable to me..

Good catch with the new debugging!

Jason

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [PATCH] mm/khugepaged: Fix might_sleep() warn with CONFIG_HIGHPTE=y
  2019-10-29 21:25   ` [PATCH] mm/khugepaged: Fix might_sleep() warn with CONFIG_HIGHPTE=y Ville Syrjala
  2019-10-29 21:43     ` Andrew Morton
@ 2019-10-30  7:08     ` Kirill A. Shutemov
  1 sibling, 0 replies; 6+ messages in thread
From: Kirill A. Shutemov @ 2019-10-30  7:08 UTC (permalink / raw)
  To: Ville Syrjala
  Cc: linux-kernel, Andrew Morton, Kirill A. Shutemov, Thomas Gleixner,
	Ingo Molnar, Borislav Petkov, H. Peter Anvin, linux-mm

On Tue, Oct 29, 2019 at 11:25:13PM +0200, Ville Syrjala wrote:
> From: Ville Syrjälä <ville.syrjala@linux.intel.com>
> 
> I got some khugepaged spew on a 32bit x86:
> 
> [  217.490026] BUG: sleeping function called from invalid context at include/linux/mmu_notifier.h:346
> [  217.492826] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 25, name: khugepaged
> [  217.495589] INFO: lockdep is turned off.
> [  217.498371] CPU: 1 PID: 25 Comm: khugepaged Not tainted 5.4.0-rc5-elk+ #206
> [  217.501233] Hardware name: System manufacturer P5Q-EM/P5Q-EM, BIOS 2203    07/08/2009
> [  217.501697] Call Trace:
> [  217.501697]  dump_stack+0x66/0x8e
> [  217.501697]  ___might_sleep.cold.96+0x95/0xa6
> [  217.501697]  __might_sleep+0x2e/0x80
> [  217.501697]  collapse_huge_page.isra.51+0x5ac/0x1360
> [  217.501697]  ? __alloc_pages_nodemask+0xec/0xf80
> [  217.501697]  ? __alloc_pages_nodemask+0x191/0xf80
> [  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
> [  217.501697]  khugepaged+0x9a9/0x20f0
> [  217.501697]  ? _raw_spin_unlock+0x21/0x30
> [  217.501697]  ? trace_hardirqs_on+0x4a/0xf0
> [  217.501697]  ? wait_woken+0xa0/0xa0
> [  217.501697]  kthread+0xf5/0x110
> [  217.501697]  ? collapse_pte_mapped_thp+0x3b0/0x3b0
> [  217.501697]  ? kthread_create_worker_on_cpu+0x20/0x20
> [  217.501697]  ret_from_fork+0x2e/0x38
> 
> Looks like it's due to CONFIG_HIGHPTE=y pte_offset_map()->kmap_atomic()
> vs. mmu_notifier_invalidate_range_start(). Let's do the naive approach
> and just reorder the two operations.
> 
> Cc: linux-kernel@vger.kernel.org
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: linux-mm@kvack.org
> Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>

Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>

-- 
 Kirill A. Shutemov

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-10-30  7:08 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-29 20:15 khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y Ville Syrjälä
2019-10-29 21:02 ` Andrew Morton
2019-10-29 21:25   ` [PATCH] mm/khugepaged: Fix might_sleep() warn with CONFIG_HIGHPTE=y Ville Syrjala
2019-10-29 21:43     ` Andrew Morton
2019-10-30  7:08     ` Kirill A. Shutemov
2019-10-29 23:15 ` khugepaged might_sleep() warn due to CONFIG_HIGHPTE=y Jason Gunthorpe

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.