[PATCH v3.18 v4.4] arm64: Add trace_hardirqs_off annotation in ret_to_user

[PATCH v3.18 v4.4] arm64: Add trace_hardirqs_off annotation in ret_to_user

[Guenter Roeck]

From: Catalin Marinas <catalin.marinas@arm.com>

commit db3899a6477a4dccd26cbfb7f408b6be2cc068e0 upstream.

When a kernel is built with CONFIG_TRACE_IRQFLAGS the following warning
is produced when entering userspace for the first time:

  WARNING: at /work/Linux/linux-2.6-aarch64/kernel/locking/lockdep.c:3519
  Modules linked in:
  CPU: 1 PID: 1 Comm: systemd Not tainted 4.4.0-rc3+ #639
  Hardware name: Juno (DT)
  task: ffffffc9768a0000 ti: ffffffc9768a8000 task.ti: ffffffc9768a8000
  PC is at check_flags.part.22+0x19c/0x1a8
  LR is at check_flags.part.22+0x19c/0x1a8
  pc : [<ffffffc0000fba6c>] lr : [<ffffffc0000fba6c>] pstate: 600001c5
  sp : ffffffc9768abe10
  x29: ffffffc9768abe10 x28: ffffffc9768a8000
  x27: 0000000000000000 x26: 0000000000000001
  x25: 00000000000000a6 x24: ffffffc00064be6c
  x23: ffffffc0009f249e x22: ffffffc9768a0000
  x21: ffffffc97fea5480 x20: 00000000000001c0
  x19: ffffffc00169a000 x18: 0000005558cc7b58
  x17: 0000007fb78e3180 x16: 0000005558d2e238
  x15: ffffffffffffffff x14: 0ffffffffffffffd
  x13: 0000000000000008 x12: 0101010101010101
  x11: 7f7f7f7f7f7f7f7f x10: fefefefefefeff63
  x9 : 7f7f7f7f7f7f7f7f x8 : 6e655f7371726964
  x7 : 0000000000000001 x6 : ffffffc0001079c4
  x5 : 0000000000000000 x4 : 0000000000000001
  x3 : ffffffc001698438 x2 : 0000000000000000
  x1 : ffffffc9768a0000 x0 : 000000000000002e
  Call trace:
  [<ffffffc0000fba6c>] check_flags.part.22+0x19c/0x1a8
  [<ffffffc0000fc440>] lock_is_held+0x80/0x98
  [<ffffffc00064bafc>] __schedule+0x404/0x730
  [<ffffffc00064be6c>] schedule+0x44/0xb8
  [<ffffffc000085bb0>] ret_to_user+0x0/0x24
  possible reason: unannotated irqs-off.
  irq event stamp: 502169
  hardirqs last  enabled at (502169): [<ffffffc000085a98>] el0_irq_naked+0x1c/0x24
  hardirqs last disabled at (502167): [<ffffffc0000bb3bc>] __do_softirq+0x17c/0x298
  softirqs last  enabled at (502168): [<ffffffc0000bb43c>] __do_softirq+0x1fc/0x298
  softirqs last disabled at (502143): [<ffffffc0000bb830>] irq_exit+0xa0/0xf0

This happens because we disable interrupts in ret_to_user before calling
schedule() in work_resched. This patch adds the necessary
trace_hardirqs_off annotation.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Mark Rutland <mark.rutland@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
Without this patch and CONFIG_TRACE_IRQFLAGS enabled, the mentioned warning
is seen in v3.18.y and v4.4.y. The patch applies cleanly to both releases.
I confirmed that the problem is gone after applying it.

 arch/arm64/kernel/entry.S | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 7ed3d75f6304..e5b25389c48f 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -634,6 +634,9 @@ work_pending:
 	bl	do_notify_resume
 	b	ret_to_user
 work_resched:
+#ifdef CONFIG_TRACE_IRQFLAGS
+	bl	trace_hardirqs_off		// the IRQs are off here, inform the tracing code
+#endif
 	bl	schedule
 
 /*
-- 
2.7.4

Re: [PATCH v3.18 v4.4] arm64: Add trace_hardirqs_off annotation in ret_to_user

[Greg Kroah-Hartman]

On Tue, Sep 25, 2018 at 05:55:47PM -0700, Guenter Roeck wrote:
> From: Catalin Marinas <catalin.marinas@arm.com>
> 
> commit db3899a6477a4dccd26cbfb7f408b6be2cc068e0 upstream.
> 
> When a kernel is built with CONFIG_TRACE_IRQFLAGS the following warning
> is produced when entering userspace for the first time:
> 
>   WARNING: at /work/Linux/linux-2.6-aarch64/kernel/locking/lockdep.c:3519
>   Modules linked in:
>   CPU: 1 PID: 1 Comm: systemd Not tainted 4.4.0-rc3+ #639
>   Hardware name: Juno (DT)
>   task: ffffffc9768a0000 ti: ffffffc9768a8000 task.ti: ffffffc9768a8000
>   PC is at check_flags.part.22+0x19c/0x1a8
>   LR is at check_flags.part.22+0x19c/0x1a8
>   pc : [<ffffffc0000fba6c>] lr : [<ffffffc0000fba6c>] pstate: 600001c5
>   sp : ffffffc9768abe10
>   x29: ffffffc9768abe10 x28: ffffffc9768a8000
>   x27: 0000000000000000 x26: 0000000000000001
>   x25: 00000000000000a6 x24: ffffffc00064be6c
>   x23: ffffffc0009f249e x22: ffffffc9768a0000
>   x21: ffffffc97fea5480 x20: 00000000000001c0
>   x19: ffffffc00169a000 x18: 0000005558cc7b58
>   x17: 0000007fb78e3180 x16: 0000005558d2e238
>   x15: ffffffffffffffff x14: 0ffffffffffffffd
>   x13: 0000000000000008 x12: 0101010101010101
>   x11: 7f7f7f7f7f7f7f7f x10: fefefefefefeff63
>   x9 : 7f7f7f7f7f7f7f7f x8 : 6e655f7371726964
>   x7 : 0000000000000001 x6 : ffffffc0001079c4
>   x5 : 0000000000000000 x4 : 0000000000000001
>   x3 : ffffffc001698438 x2 : 0000000000000000
>   x1 : ffffffc9768a0000 x0 : 000000000000002e
>   Call trace:
>   [<ffffffc0000fba6c>] check_flags.part.22+0x19c/0x1a8
>   [<ffffffc0000fc440>] lock_is_held+0x80/0x98
>   [<ffffffc00064bafc>] __schedule+0x404/0x730
>   [<ffffffc00064be6c>] schedule+0x44/0xb8
>   [<ffffffc000085bb0>] ret_to_user+0x0/0x24
>   possible reason: unannotated irqs-off.
>   irq event stamp: 502169
>   hardirqs last  enabled at (502169): [<ffffffc000085a98>] el0_irq_naked+0x1c/0x24
>   hardirqs last disabled at (502167): [<ffffffc0000bb3bc>] __do_softirq+0x17c/0x298
>   softirqs last  enabled at (502168): [<ffffffc0000bb43c>] __do_softirq+0x1fc/0x298
>   softirqs last disabled at (502143): [<ffffffc0000bb830>] irq_exit+0xa0/0xf0
> 
> This happens because we disable interrupts in ret_to_user before calling
> schedule() in work_resched. This patch adds the necessary
> trace_hardirqs_off annotation.
> 
> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
> Reported-by: Mark Rutland <mark.rutland@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Will Deacon <will.deacon@arm.com>
> ---
> Without this patch and CONFIG_TRACE_IRQFLAGS enabled, the mentioned warning
> is seen in v3.18.y and v4.4.y. The patch applies cleanly to both releases.
> I confirmed that the problem is gone after applying it.

Now applied, thanks.

greg k-h