Re: [PATCH v3 0/6] Tracing vs CR2

From: Eiichi Tsukata <devel@etsukata.com>
To: Vegard Nossum <vegard.nossum@oracle.com>,
	Peter Zijlstra <peterz@infradead.org>,
	tglx@linutronix.de, bp@alien8.de, mingo@kernel.org,
	rostedt@goodmis.org, luto@kernel.org,
	torvalds@linux-foundation.org, linux_lkml_grp@oracle.com
Cc: hpa@zytor.com, dave.hansen@linux.intel.com, jgross@suse.com,
	linux-kernel@vger.kernel.org, zhe.he@windriver.com,
	joel@joelfernandes.org
Subject: Re: [PATCH v3 0/6] Tracing vs CR2
Date: Wed, 17 Jul 2019 18:37:00 +0900	[thread overview]
Message-ID: <d82854b2-d2a4-5b83-b4a4-796db0fd401b@etsukata.com> (raw)
In-Reply-To: <97cdd0af-95cc-2583-dc19-129b20809110@oracle.com>

On 2019/07/17 6:51, Vegard Nossum wrote:
> 
...
> 
> Got a different one:
> 
> WARNING: CPU: 0 PID: 2150 at arch/x86/kernel/traps.c:791 do_debug+0xfe/0x240
> CPU: 0 PID: 2150 Comm: init Not tainted 5.2.0+ #124
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
> RIP: 0010:do_debug+0xfe/0x240
...

Hello Vegard

I found a way to reproduce #DB WARNING by setting hardware watchpoint to
the address arch_stack_walk_user() will touch.

[Steps to Reproduce #DB WARNING]

poc.s:

```
        .global _start

        .text
_start:
        # exit(0)
        mov $60, %rax
        xor %rdi, %rdi
        syscall
```

build:

  # gcc -g -c poc.s; ld -o poc poc.o

setup ftrace:

  # echo 1 > options/userstacktrace
  # echo 1 > events/preemptirq/irq_disable/enable

exec gdb:(set hardware watch point to $rbp)

  [18:28:48 root@vm loops]# gdb ./poc
  GNU gdb (GDB) Fedora 8.3-6.fc30
  Copyright (C) 2019 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.
  Type "show copying" and "show warranty" for details.
  This GDB was configured as "x86_64-redhat-linux-gnu".
  Type "show configuration" for configuration details.
  For bug reporting instructions, please see:
  <http://www.gnu.org/software/gdb/bugs/>.
  Find the GDB manual and other documentation resources online at:
      <http://www.gnu.org/software/gdb/documentation/>.

  For help, type "help".
  Type "apropos word" to search for commands related to "word"...
  Reading symbols from ./poc...
  (gdb) l
  1               .global _start
  2
  3               .text
  4       _start:
  5               # exit(0)
  6               mov $60, %rax
  7               xor %rdi, %rdi
  8               syscall
  (gdb) b 6
  Breakpoint 1 at 0x401000: file poc.s, line 6.
  (gdb) start
  Function "main" not defined.
  Make breakpoint pending on future shared library load? (y or [n]) n
  Starting program: /root/tmp/loops/poc

  Breakpoint 1, _start () at poc.s:6
  6               mov $60, %rax
  (gdb) set $rbp = $rsp
  (gdb) p $rbp
  $1 = (void *) 0x7fffffffe4b0
  (gdb) rwatch *0x7fffffffe4b0
  Hardware read watchpoint 2: *0x7fffffffe4b0
  (gdb) c
  Continuing.
  [Inferior 1 (process 2744) exited normally]

dmesg:

[  564.646159][ T2744] WARNING: CPU: 0 PID: 2744 at arch/x86/kernel/traps.c:791 do_debug+0x220/0x490
[  564.648581][ T2744] Modules linked in:
[  564.649530][ T2744] CPU: 0 PID: 2744 Comm: poc Tainted: G        W         5.2.0+ #77
[  564.651121][ T2744] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-2.fc30 04/01/2014
[  564.653569][ T2744] RIP: 0010:do_debug+0x220/0x490
[  564.654847][ T2744] Code: 00 48 8b 95 60 ff ff ff 48 b8 00 00 00 00 00 fc ff df 48 c1 ea 03 80 3c 02 00 0f 85 03 02 00 00 41 f6 87 88 00 00 00 03 75 60 <0f> 0b 4c 89 f2 49 81 e5 ff bf ff ff 48 b8 00 00 00 00 00 fc ff df
[  564.659905][ T2744] RSP: 0000:fffffe0000014e98 EFLAGS: 00010046
[  564.661500][ T2744] RAX: dffffc0000000000 RBX: 1fffffc0000029d8 RCX: 1ffff1100f81c2d3
[  564.663531][ T2744] RDX: 1fffffc0000029fc RSI: 0000000000000000 RDI: ffffffff85c19f00
[  564.665553][ T2744] RBP: fffffe0000014f48 R08: fffffe0000014fe8 R09: ffff88807c0e08a0
[  564.667637][ T2744] R10: 0000000000000001 R11: 1ffff1100d1042ba R12: ffff88807c0e0000
[  564.669700][ T2744] R13: 0000000000004001 R14: ffff88807c0e1698 R15: fffffe0000014f58
[  564.671768][ T2744] FS:  0000000000000000(0000) GS:ffff888068800000(0000) knlGS:0000000000000000
[  564.674032][ T2744] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  564.675752][ T2744] CR2: 0000000000000001 CR3: 000000005fe08002 CR4: 0000000000160ef0
[  564.677570][ T2744] DR0: 00007fffffffe4b0 DR1: 0000000000000000 DR2: 0000000000000000
[  564.679686][ T2744] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 00000000000f0602
[  564.681788][ T2744] Call Trace:
[  564.682700][ T2744]  <#DB>
[  564.683492][ T2744]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  564.684954][ T2744]  ? do_int3+0x1f0/0x1f0
[  564.686074][ T2744]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  564.687512][ T2744]  debug+0x2d/0x70
[  564.688456][ T2744] RIP: 0010:arch_stack_walk_user+0x7d/0xf2
[  564.689899][ T2744] Code: 00 0f 85 8d 00 00 00 49 8b 87 d8 16 00 00 48 83 e8 10 49 39 c6 77 32 41 83 87 e8 15 00 00 01 0f 1f 00 0f ae e8 31 c0 49 8b 0e <85> c0 75 6d 49 8b 76 08 0f 1f 00 85 c0 74 1f 65 48 8b 04 25 00 ef
[  564.694763][ T2744] RSP: 0000:ffff888061fb7c48 EFLAGS: 00000046
[  564.696316][ T2744] RAX: 0000000000000000 RBX: ffff88807c0e0000 RCX: 0000000000000001
[  564.698342][ T2744] RDX: 1ffff1100ba08e93 RSI: 0000000000401009 RDI: ffff888061fb7cbc
[  564.700323][ T2744] RBP: ffff888061fb7c80 R08: 1ffff1100ba08e93 R09: ffff88805d04749c
[  564.702337][ T2744] R10: ffffed100ba08e9b R11: ffff88805d0474db R12: ffff888061fb7cb0
[  564.704359][ T2744] R13: ffff888061fb7f58 R14: 00007fffffffe4b0 R15: ffff88807c0e0000
[  564.706413][ T2744]  </#DB>
[  564.707182][ T2744]  ? stack_trace_save+0xc0/0xc0
[  564.708447][ T2744]  stack_trace_save_user+0x138/0x160
[  564.709752][ T2744]  ? stack_trace_save_tsk_reliable+0x210/0x210
[  564.711235][ T2744]  ? kasan_check_read+0x11/0x20
[  564.712358][ T2744]  trace_buffer_unlock_commit_regs+0x208/0x360
[  564.713871][ T2744]  trace_event_buffer_commit+0x1a0/0x790
[  564.715278][ T2744]  ? trace_event_buffer_reserve+0x163/0x240
[  564.716742][ T2744]  trace_event_raw_event_preemptirq_template+0x156/0x200
[  564.718431][ T2744]  ? perf_trace_preemptirq_template+0x490/0x490
[  564.719761][ T2744]  ? rcu_irq_enter_irqson+0x23/0x30
[  564.721064][ T2744]  ? trace_hardirqs_off+0x28/0x180
[  564.722337][ T2744]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  564.723729][ T2744]  ? debug+0x49/0x70
[  564.724703][ T2744]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  564.726113][ T2744]  ? perf_trace_preemptirq_template+0x490/0x490
[  564.727721][ T2744]  trace_hardirqs_off_caller+0x106/0x170
[  564.729154][ T2744]  ? debug+0x44/0x70
[  564.730108][ T2744]  trace_hardirqs_off_thunk+0x1a/0x1c
[  564.731456][ T2744]  debug+0x49/0x70
[  564.732278][ T2744] RIP: 0033:0x401009
[  564.733155][ T2744] Code: Bad RIP value.
[  564.734019][ T2744] RSP: 002b:00007fffffffe4b0 EFLAGS: 00000302
[  564.735366][ T2744] RAX: 000000000000003c RBX: 0000000000000000 RCX: 0000000000000000
[  564.737290][ T2744] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[  564.739314][ T2744] RBP: 00007fffffffe4b0 R08: 0000000000000000 R09: 0000000000000000
[  564.741338][ T2744] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[  564.743311][ T2744] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[  564.745463][ T2744] irq event stamp: 3340
[  564.746554][ T2744] hardirqs last  enabled at (3339): [<ffffffff82a04c12>] trace_hardirqs_on_thunk+0x1a/0x1c
[  564.748558][ T2744] hardirqs last disabled at (3340): [<ffffffff82d28b32>] rcu_irq_enter_irqson+0x12/0x30
[  564.750479][ T2744] softirqs last  enabled at (3330): [<ffffffff85a00634>] __do_softirq+0x634/0x9f1
[  564.752737][ T2744] softirqs last disabled at (3319): [<ffffffff82b77920>] irq_exit+0x150/0x180
[  564.754933][ T2744] ---[ end trace 67c6e66ff6ba5cd0 ]---