All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] powerpc/perf: add missing put_cpu_var in power_pmu_event_init
@ 2015-03-24 12:33 ` Jan Stancek
  0 siblings, 0 replies; 5+ messages in thread
From: Jan Stancek @ 2015-03-24 12:33 UTC (permalink / raw)
  To: linuxppc-dev
  Cc: benh, paulus, mpe, anton, cl, tj, joel, linux-kernel, jolsa, jstancek

One path in power_pmu_event_init() calls get_cpu_var(), but is
missing matching call to put_cpu_var(), which causes preemption
imbalance and crash in user-space:

  Page fault in user mode with in_atomic() = 1 mm = c000001fefa5a280
  NIP = 3fff9bf2cae0  MSR = 900000014280f032
  Oops: Weird page fault, sig: 11 [#23]
  SMP NR_CPUS=2048 NUMA PowerNV
  Modules linked in: <snip>
  CPU: 43 PID: 10285 Comm: a.out Tainted: G      D         4.0.0-rc5+ #1
  task: c000001fe82c9200 ti: c000001fe835c000 task.ti: c000001fe835c000
  NIP: 00003fff9bf2cae0 LR: 00003fff9bee4898 CTR: 00003fff9bf2cae0
  REGS: c000001fe835fea0 TRAP: 0401   Tainted: G      D          (4.0.0-rc5+)
  MSR: 900000014280f032 <SF,HV,VEC,VSX,EE,PR,FP,ME,IR,DR,RI>  CR: 22000028  XER: 00000000
  CFAR: 00003fff9bee4894 SOFTE: 1
   GPR00: 00003fff9bee494c 00003fffe01c2ee0 00003fff9c084410 0000000010020068
   GPR04: 0000000000000000 0000000000000002 0000000000000008 0000000000000001
   GPR08: 0000000000000001 00003fff9c074a30 00003fff9bf2cae0 00003fff9bf2cd70
   GPR12: 0000000052000022 00003fff9c10b700
  NIP [00003fff9bf2cae0] 0x3fff9bf2cae0
  LR [00003fff9bee4898] 0x3fff9bee4898
  Call Trace:
  ---[ end trace 5d3d952b5d4185d4 ]---

  BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:41
  in_atomic(): 1, irqs_disabled(): 0, pid: 10285, name: a.out
  INFO: lockdep is turned off.
  CPU: 43 PID: 10285 Comm: a.out Tainted: G      D         4.0.0-rc5+ #1
  Call Trace:
  [c000001fe835f990] [c00000000089c014] .dump_stack+0x98/0xd4 (unreliable)
  [c000001fe835fa10] [c0000000000e4138] .___might_sleep+0x1d8/0x2e0
  [c000001fe835faa0] [c000000000888da8] .down_read+0x38/0x110
  [c000001fe835fb30] [c0000000000bf2f4] .exit_signals+0x24/0x160
  [c000001fe835fbc0] [c0000000000abde0] .do_exit+0xd0/0xe70
  [c000001fe835fcb0] [c00000000001f4c4] .die+0x304/0x450
  [c000001fe835fd60] [c00000000088e1f4] .do_page_fault+0x2d4/0x900
  [c000001fe835fe30] [c000000000008664] handle_page_fault+0x10/0x30
  note: a.out[10285] exited with preempt_count 1

Reproducer:
  #include <stdio.h>
  #include <unistd.h>
  #include <syscall.h>
  #include <sys/types.h>
  #include <sys/stat.h>
  #include <linux/perf_event.h>
  #include <linux/hw_breakpoint.h>

  static struct perf_event_attr event = {
          .type = PERF_TYPE_RAW,
          .size = sizeof(struct perf_event_attr),
          .sample_type = PERF_SAMPLE_BRANCH_STACK,
          .branch_sample_type = PERF_SAMPLE_BRANCH_ANY_RETURN,
  };

  int main()
  {
          syscall(__NR_perf_event_open, &event, 0, -1, -1, 0);
  }

Signed-off-by: Jan Stancek <jstancek@redhat.com>
---
 arch/powerpc/perf/core-book3s.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c
index 7c4f669..b101c0b 100644
--- a/arch/powerpc/perf/core-book3s.c
+++ b/arch/powerpc/perf/core-book3s.c
@@ -1832,8 +1832,10 @@ static int power_pmu_event_init(struct perf_event *event)
 		cpuhw->bhrb_filter = ppmu->bhrb_filter_map(
 					event->attr.branch_sample_type);
 
-		if(cpuhw->bhrb_filter == -1)
+		if (cpuhw->bhrb_filter == -1) {
+			put_cpu_var(cpu_hw_events);
 			return -EOPNOTSUPP;
+		}
 	}
 
 	put_cpu_var(cpu_hw_events);
-- 
1.7.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH] powerpc/perf: add missing put_cpu_var in power_pmu_event_init
@ 2015-03-24 12:33 ` Jan Stancek
  0 siblings, 0 replies; 5+ messages in thread
From: Jan Stancek @ 2015-03-24 12:33 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: linux-kernel, paulus, anton, tj, cl, jolsa, jstancek, joel

One path in power_pmu_event_init() calls get_cpu_var(), but is
missing matching call to put_cpu_var(), which causes preemption
imbalance and crash in user-space:

  Page fault in user mode with in_atomic() = 1 mm = c000001fefa5a280
  NIP = 3fff9bf2cae0  MSR = 900000014280f032
  Oops: Weird page fault, sig: 11 [#23]
  SMP NR_CPUS=2048 NUMA PowerNV
  Modules linked in: <snip>
  CPU: 43 PID: 10285 Comm: a.out Tainted: G      D         4.0.0-rc5+ #1
  task: c000001fe82c9200 ti: c000001fe835c000 task.ti: c000001fe835c000
  NIP: 00003fff9bf2cae0 LR: 00003fff9bee4898 CTR: 00003fff9bf2cae0
  REGS: c000001fe835fea0 TRAP: 0401   Tainted: G      D          (4.0.0-rc5+)
  MSR: 900000014280f032 <SF,HV,VEC,VSX,EE,PR,FP,ME,IR,DR,RI>  CR: 22000028  XER: 00000000
  CFAR: 00003fff9bee4894 SOFTE: 1
   GPR00: 00003fff9bee494c 00003fffe01c2ee0 00003fff9c084410 0000000010020068
   GPR04: 0000000000000000 0000000000000002 0000000000000008 0000000000000001
   GPR08: 0000000000000001 00003fff9c074a30 00003fff9bf2cae0 00003fff9bf2cd70
   GPR12: 0000000052000022 00003fff9c10b700
  NIP [00003fff9bf2cae0] 0x3fff9bf2cae0
  LR [00003fff9bee4898] 0x3fff9bee4898
  Call Trace:
  ---[ end trace 5d3d952b5d4185d4 ]---

  BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:41
  in_atomic(): 1, irqs_disabled(): 0, pid: 10285, name: a.out
  INFO: lockdep is turned off.
  CPU: 43 PID: 10285 Comm: a.out Tainted: G      D         4.0.0-rc5+ #1
  Call Trace:
  [c000001fe835f990] [c00000000089c014] .dump_stack+0x98/0xd4 (unreliable)
  [c000001fe835fa10] [c0000000000e4138] .___might_sleep+0x1d8/0x2e0
  [c000001fe835faa0] [c000000000888da8] .down_read+0x38/0x110
  [c000001fe835fb30] [c0000000000bf2f4] .exit_signals+0x24/0x160
  [c000001fe835fbc0] [c0000000000abde0] .do_exit+0xd0/0xe70
  [c000001fe835fcb0] [c00000000001f4c4] .die+0x304/0x450
  [c000001fe835fd60] [c00000000088e1f4] .do_page_fault+0x2d4/0x900
  [c000001fe835fe30] [c000000000008664] handle_page_fault+0x10/0x30
  note: a.out[10285] exited with preempt_count 1

Reproducer:
  #include <stdio.h>
  #include <unistd.h>
  #include <syscall.h>
  #include <sys/types.h>
  #include <sys/stat.h>
  #include <linux/perf_event.h>
  #include <linux/hw_breakpoint.h>

  static struct perf_event_attr event = {
          .type = PERF_TYPE_RAW,
          .size = sizeof(struct perf_event_attr),
          .sample_type = PERF_SAMPLE_BRANCH_STACK,
          .branch_sample_type = PERF_SAMPLE_BRANCH_ANY_RETURN,
  };

  int main()
  {
          syscall(__NR_perf_event_open, &event, 0, -1, -1, 0);
  }

Signed-off-by: Jan Stancek <jstancek@redhat.com>
---
 arch/powerpc/perf/core-book3s.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c
index 7c4f669..b101c0b 100644
--- a/arch/powerpc/perf/core-book3s.c
+++ b/arch/powerpc/perf/core-book3s.c
@@ -1832,8 +1832,10 @@ static int power_pmu_event_init(struct perf_event *event)
 		cpuhw->bhrb_filter = ppmu->bhrb_filter_map(
 					event->attr.branch_sample_type);
 
-		if(cpuhw->bhrb_filter == -1)
+		if (cpuhw->bhrb_filter == -1) {
+			put_cpu_var(cpu_hw_events);
 			return -EOPNOTSUPP;
+		}
 	}
 
 	put_cpu_var(cpu_hw_events);
-- 
1.7.1

^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: powerpc/perf: add missing put_cpu_var in power_pmu_event_init
  2015-03-24 12:33 ` Jan Stancek
  (?)
@ 2015-03-25  5:25 ` Michael Ellerman
  2015-03-25  9:29     ` Jan Stancek
  -1 siblings, 1 reply; 5+ messages in thread
From: Michael Ellerman @ 2015-03-25  5:25 UTC (permalink / raw)
  To: Jan Stancek, linuxppc-dev
  Cc: linux-kernel, paulus, anton, tj, cl, jolsa, jstancek, joel

On Tue, 2015-24-03 at 12:33:22 UTC, Jan Stancek wrote:
> One path in power_pmu_event_init() calls get_cpu_var(), but is
> missing matching call to put_cpu_var(), which causes preemption
> imbalance and crash in user-space:
> 
>   Page fault in user mode with in_atomic() = 1 mm = c000001fefa5a280
>   NIP = 3fff9bf2cae0  MSR = 900000014280f032
>   Oops: Weird page fault, sig: 11 [#23]

<snip>

Thanks. But I don't see this. I guess you have CONFIG_PREEMPT enabled?

cheers

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: powerpc/perf: add missing put_cpu_var in power_pmu_event_init
  2015-03-25  5:25 ` Michael Ellerman
@ 2015-03-25  9:29     ` Jan Stancek
  0 siblings, 0 replies; 5+ messages in thread
From: Jan Stancek @ 2015-03-25  9:29 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: linuxppc-dev, linux-kernel, paulus, anton, tj, cl, jolsa, joel



----- Original Message -----
> From: "Michael Ellerman" <mpe@ellerman.id.au>
> To: "Jan Stancek" <jstancek@redhat.com>, linuxppc-dev@lists.ozlabs.org
> Cc: linux-kernel@vger.kernel.org, paulus@samba.org, anton@samba.org, tj@kernel.org, cl@linux.com, jolsa@redhat.com,
> jstancek@redhat.com, joel@jms.id.au
> Sent: Wednesday, 25 March, 2015 6:25:09 AM
> Subject: Re: powerpc/perf: add missing put_cpu_var in power_pmu_event_init
> 
> On Tue, 2015-24-03 at 12:33:22 UTC, Jan Stancek wrote:
> > One path in power_pmu_event_init() calls get_cpu_var(), but is
> > missing matching call to put_cpu_var(), which causes preemption
> > imbalance and crash in user-space:
> > 
> >   Page fault in user mode with in_atomic() = 1 mm = c000001fefa5a280
> >   NIP = 3fff9bf2cae0  MSR = 900000014280f032
> >   Oops: Weird page fault, sig: 11 [#23]
> 
> <snip>
> 
> Thanks. But I don't see this. I guess you have CONFIG_PREEMPT enabled?

Hi,

CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
CONFIG_PREEMPT_COUNT=y

but I think the difference comes from:
  CONFIG_DEBUG_ATOMIC_SLEEP=y

I did following:
- took the default config from RHEL7.1 kernel
- ran 'make oldnoconfig'.
- reproducer didn't trigger anything
- then I added "CONFIG_DEBUG_ATOMIC_SLEEP=y"
- this time reproducer triggered a panic (3 out of 3 attempts)

Here's config from panic-ing kernel: http://fpaste.org/202543/

[  133.957305] Page fault in user mode with in_atomic() = 1 mm = c000000005fc7e80
[  133.957399] NIP = 3fff9be0cae0  MSR = 900000014280f032
[  133.957405] Oops: Weird page fault, sig: 11 [#1]
[  133.957409] SMP NR_CPUS=2048 NUMA PowerNV
[  133.957414] Modules linked in: ses enclosure shpchp uio_pdrv_genirq powernv_rng uio xfs libcrc32c sr_mod sd_mod cdrom ipr libata tg3 ptp pps_core dm_mirror dm_region_hash dm_log dm_mod
[  133.957638] CPU: 16 PID: 6035 Comm: a.out Not tainted 4.0.0-rc5+ #4
[  133.957693] task: c000000fea44b640 ti: c000000fea5e4000 task.ti: c000000fea5e4000
[  133.957759] NIP: 00003fff9be0cae0 LR: 00003fff9bdc4898 CTR: 00003fff9be0cae0
[  133.957825] REGS: c000000fea5e7ea0 TRAP: 0401   Not tainted  (4.0.0-rc5+)
[  133.957880] MSR: 900000014280f032 <SF,HV,VEC,VSX,EE,PR,FP,ME,IR,DR,RI>  CR: 22000028  XER: 00000000
[  133.958079] CFAR: 00003fff9bdc4894 SOFTE: 1 
GPR00: 00003fff9bdc494c 00003ffff1fef3e0 00003fff9bf64410 0000000010020068 
GPR04: 0000000000000000 0000000000000002 0000000000000008 0000000000000001 
GPR08: 0000000000000001 00003fff9bf54a30 00003fff9be0cae0 00003fff9be0cd70 
GPR12: 0000000052000022 00003fff9bfeb700 
[  133.958485] NIP [00003fff9be0cae0] 0x3fff9be0cae0
[  133.958530] LR [00003fff9bdc4898] 0x3fff9bdc4898
[  133.958574] Call Trace:
[  133.958597] ---[ end trace 56ec543903422cd9 ]---
[  133.958642] 
[  135.958709] Kernel panic - not syncing: Fatal exception
[  135.958863] Rebooting in 10 seconds..
[  145.970348] BUG: sleeping function called from invalid context at kernel/irq/manage.c:104
[  145.970453] in_atomic(): 1, irqs_disabled(): 1, pid: 6035, name: a.out
[  145.970515] CPU: 16 PID: 6035 Comm: a.out Tainted: G      D         4.0.0-rc5+ #4
[  145.970588] Call Trace:
[  145.970618] [c000000fea5e76d0] [c0000000007c2090] .dump_stack+0x98/0xd4 (unreliable)
[  145.970707] [c000000fea5e7750] [c0000000000d5fe4] .___might_sleep+0x124/0x170
[  145.970782] [c000000fea5e77c0] [c000000000112860] .synchronize_irq+0x40/0xe0
[  145.970857] [c000000fea5e7880] [c000000000112fa8] .__free_irq+0xf8/0x2b0
[  145.970931] [c000000fea5e7920] [c000000000113258] .free_irq+0x78/0x100
[  145.971007] [c000000fea5e79b0] [c000000000067ae8] .opal_shutdown+0x88/0x120
[  145.971081] [c000000fea5e7a40] [c000000000063e88] .pnv_shutdown+0x18/0x30
[  145.971157] [c000000fea5e7ab0] [c000000000020c98] .machine_shutdown+0x38/0x50
[  145.971231] [c000000fea5e7b20] [c000000000020d24] .machine_restart+0x14/0x70
[  145.971307] [c000000fea5e7ba0] [c0000000000cdc10] .emergency_restart+0x20/0x40
[  145.971393] [c000000fea5e7c10] [c0000000007bb0a4] .panic+0x224/0x2a4
[  145.971468] [c000000fea5e7cb0] [c00000000001e1fc] .die+0x43c/0x450
[  145.971543] [c000000fea5e7d60] [c0000000007b62c4] .do_page_fault+0x2d4/0x8f0
[  145.971618] [c000000fea5e7e30] [c000000000008664] handle_page_fault+0x10/0x30

Regards,
Jan

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: powerpc/perf: add missing put_cpu_var in power_pmu_event_init
@ 2015-03-25  9:29     ` Jan Stancek
  0 siblings, 0 replies; 5+ messages in thread
From: Jan Stancek @ 2015-03-25  9:29 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: jolsa, linux-kernel, paulus, anton, tj, cl, linuxppc-dev, joel



----- Original Message -----
> From: "Michael Ellerman" <mpe@ellerman.id.au>
> To: "Jan Stancek" <jstancek@redhat.com>, linuxppc-dev@lists.ozlabs.org
> Cc: linux-kernel@vger.kernel.org, paulus@samba.org, anton@samba.org, tj@kernel.org, cl@linux.com, jolsa@redhat.com,
> jstancek@redhat.com, joel@jms.id.au
> Sent: Wednesday, 25 March, 2015 6:25:09 AM
> Subject: Re: powerpc/perf: add missing put_cpu_var in power_pmu_event_init
> 
> On Tue, 2015-24-03 at 12:33:22 UTC, Jan Stancek wrote:
> > One path in power_pmu_event_init() calls get_cpu_var(), but is
> > missing matching call to put_cpu_var(), which causes preemption
> > imbalance and crash in user-space:
> > 
> >   Page fault in user mode with in_atomic() = 1 mm = c000001fefa5a280
> >   NIP = 3fff9bf2cae0  MSR = 900000014280f032
> >   Oops: Weird page fault, sig: 11 [#23]
> 
> <snip>
> 
> Thanks. But I don't see this. I guess you have CONFIG_PREEMPT enabled?

Hi,

CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
CONFIG_PREEMPT_COUNT=y

but I think the difference comes from:
  CONFIG_DEBUG_ATOMIC_SLEEP=y

I did following:
- took the default config from RHEL7.1 kernel
- ran 'make oldnoconfig'.
- reproducer didn't trigger anything
- then I added "CONFIG_DEBUG_ATOMIC_SLEEP=y"
- this time reproducer triggered a panic (3 out of 3 attempts)

Here's config from panic-ing kernel: http://fpaste.org/202543/

[  133.957305] Page fault in user mode with in_atomic() = 1 mm = c000000005fc7e80
[  133.957399] NIP = 3fff9be0cae0  MSR = 900000014280f032
[  133.957405] Oops: Weird page fault, sig: 11 [#1]
[  133.957409] SMP NR_CPUS=2048 NUMA PowerNV
[  133.957414] Modules linked in: ses enclosure shpchp uio_pdrv_genirq powernv_rng uio xfs libcrc32c sr_mod sd_mod cdrom ipr libata tg3 ptp pps_core dm_mirror dm_region_hash dm_log dm_mod
[  133.957638] CPU: 16 PID: 6035 Comm: a.out Not tainted 4.0.0-rc5+ #4
[  133.957693] task: c000000fea44b640 ti: c000000fea5e4000 task.ti: c000000fea5e4000
[  133.957759] NIP: 00003fff9be0cae0 LR: 00003fff9bdc4898 CTR: 00003fff9be0cae0
[  133.957825] REGS: c000000fea5e7ea0 TRAP: 0401   Not tainted  (4.0.0-rc5+)
[  133.957880] MSR: 900000014280f032 <SF,HV,VEC,VSX,EE,PR,FP,ME,IR,DR,RI>  CR: 22000028  XER: 00000000
[  133.958079] CFAR: 00003fff9bdc4894 SOFTE: 1 
GPR00: 00003fff9bdc494c 00003ffff1fef3e0 00003fff9bf64410 0000000010020068 
GPR04: 0000000000000000 0000000000000002 0000000000000008 0000000000000001 
GPR08: 0000000000000001 00003fff9bf54a30 00003fff9be0cae0 00003fff9be0cd70 
GPR12: 0000000052000022 00003fff9bfeb700 
[  133.958485] NIP [00003fff9be0cae0] 0x3fff9be0cae0
[  133.958530] LR [00003fff9bdc4898] 0x3fff9bdc4898
[  133.958574] Call Trace:
[  133.958597] ---[ end trace 56ec543903422cd9 ]---
[  133.958642] 
[  135.958709] Kernel panic - not syncing: Fatal exception
[  135.958863] Rebooting in 10 seconds..
[  145.970348] BUG: sleeping function called from invalid context at kernel/irq/manage.c:104
[  145.970453] in_atomic(): 1, irqs_disabled(): 1, pid: 6035, name: a.out
[  145.970515] CPU: 16 PID: 6035 Comm: a.out Tainted: G      D         4.0.0-rc5+ #4
[  145.970588] Call Trace:
[  145.970618] [c000000fea5e76d0] [c0000000007c2090] .dump_stack+0x98/0xd4 (unreliable)
[  145.970707] [c000000fea5e7750] [c0000000000d5fe4] .___might_sleep+0x124/0x170
[  145.970782] [c000000fea5e77c0] [c000000000112860] .synchronize_irq+0x40/0xe0
[  145.970857] [c000000fea5e7880] [c000000000112fa8] .__free_irq+0xf8/0x2b0
[  145.970931] [c000000fea5e7920] [c000000000113258] .free_irq+0x78/0x100
[  145.971007] [c000000fea5e79b0] [c000000000067ae8] .opal_shutdown+0x88/0x120
[  145.971081] [c000000fea5e7a40] [c000000000063e88] .pnv_shutdown+0x18/0x30
[  145.971157] [c000000fea5e7ab0] [c000000000020c98] .machine_shutdown+0x38/0x50
[  145.971231] [c000000fea5e7b20] [c000000000020d24] .machine_restart+0x14/0x70
[  145.971307] [c000000fea5e7ba0] [c0000000000cdc10] .emergency_restart+0x20/0x40
[  145.971393] [c000000fea5e7c10] [c0000000007bb0a4] .panic+0x224/0x2a4
[  145.971468] [c000000fea5e7cb0] [c00000000001e1fc] .die+0x43c/0x450
[  145.971543] [c000000fea5e7d60] [c0000000007b62c4] .do_page_fault+0x2d4/0x8f0
[  145.971618] [c000000fea5e7e30] [c000000000008664] handle_page_fault+0x10/0x30

Regards,
Jan

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2015-03-25 10:21 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-03-24 12:33 [PATCH] powerpc/perf: add missing put_cpu_var in power_pmu_event_init Jan Stancek
2015-03-24 12:33 ` Jan Stancek
2015-03-25  5:25 ` Michael Ellerman
2015-03-25  9:29   ` Jan Stancek
2015-03-25  9:29     ` Jan Stancek

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.