[PATCH V2 0/4] ARM64: Fix irq generation between breakpoint and step exception

[PATCH V2 0/4] ARM64: Fix irq generation between breakpoint and step exception

[Pratyush Anand]

v1 was here http://marc.info/?l=linux-arm-kernel&m=149910958418708&w=2

v1 -> v2:
- patch 1 of v1 has been modified to patch 1-3 of v2.
- Introduced a new event attribute step_needed and implemented
  hw_breakpoint_needs_single_step() (patch 1)
- Replaced usage of is_default_overflow_handler() with
  hw_breakpoint_needs_single_step(). (patch 2)
- Modified sample test to set set step_needed bit field (patch 3)

samples/hw_breakpoint/data_breakpoint.c passes with x86_64 but fails with
ARM64. Even though it has been NAKed previously on upstream [1, 2], I have
tried to come up with patches which can resolve it for ARM64 as well.

I noticed that even perf step exception can go into an infinite loop if CPU
receives an interrupt while executing breakpoint/watchpoint handler. So,
event though we are not concerned about above test, we will have to find a
solution for the perf issue.

This patchset attempts to resolve both the issue. Please review.

[1] http://marc.info/?l=linux-arm-kernel&m=149580777524910&w=2
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2016-April/425266.html

Pratyush Anand (4):
  hw_breakpoint: Add step_needed event attribute
  arm64: use hw_breakpoint_needs_single_step() to decide if step is
    needed
  hw-breakpoint: sample test: set step_needed bit field
  arm64: disable irq between breakpoint and step exception

 arch/arm64/kernel/debug-monitors.c      |  3 +++
 arch/arm64/kernel/hw_breakpoint.c       | 10 +++++-----
 arch/arm64/mm/fault.c                   | 22 ++++++++++++++++++----
 include/linux/hw_breakpoint.h           |  6 ++++++
 include/uapi/linux/perf_event.h         |  3 ++-
 kernel/events/core.c                    |  2 ++
 samples/hw_breakpoint/data_breakpoint.c |  1 +
 tools/include/uapi/linux/perf_event.h   |  3 ++-
 8 files changed, 39 insertions(+), 11 deletions(-)

-- 
2.9.3

[PATCH V2 0/4] ARM64: Fix irq generation between breakpoint and step exception

[Pratyush Anand]

v1 was here http://marc.info/?l=linux-arm-kernel&m=149910958418708&w=2

v1 -> v2:
- patch 1 of v1 has been modified to patch 1-3 of v2.
- Introduced a new event attribute step_needed and implemented
  hw_breakpoint_needs_single_step() (patch 1)
- Replaced usage of is_default_overflow_handler() with
  hw_breakpoint_needs_single_step(). (patch 2)
- Modified sample test to set set step_needed bit field (patch 3)

samples/hw_breakpoint/data_breakpoint.c passes with x86_64 but fails with
ARM64. Even though it has been NAKed previously on upstream [1, 2], I have
tried to come up with patches which can resolve it for ARM64 as well.

I noticed that even perf step exception can go into an infinite loop if CPU
receives an interrupt while executing breakpoint/watchpoint handler. So,
event though we are not concerned about above test, we will have to find a
solution for the perf issue.

This patchset attempts to resolve both the issue. Please review.

[1] http://marc.info/?l=linux-arm-kernel&m=149580777524910&w=2
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2016-April/425266.html

Pratyush Anand (4):
  hw_breakpoint: Add step_needed event attribute
  arm64: use hw_breakpoint_needs_single_step() to decide if step is
    needed
  hw-breakpoint: sample test: set step_needed bit field
  arm64: disable irq between breakpoint and step exception

 arch/arm64/kernel/debug-monitors.c      |  3 +++
 arch/arm64/kernel/hw_breakpoint.c       | 10 +++++-----
 arch/arm64/mm/fault.c                   | 22 ++++++++++++++++++----
 include/linux/hw_breakpoint.h           |  6 ++++++
 include/uapi/linux/perf_event.h         |  3 ++-
 kernel/events/core.c                    |  2 ++
 samples/hw_breakpoint/data_breakpoint.c |  1 +
 tools/include/uapi/linux/perf_event.h   |  3 ++-
 8 files changed, 39 insertions(+), 11 deletions(-)

-- 
2.9.3

[PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Pratyush Anand]

Architecture like ARM64 currently allows to use default hw breakpoint
single step handler only to perf. However, some other users like few
systemtap tests or kernel test in
samples/hw_breakpoint/data_breakpoint.c can also work with default step
handler implementation. At the same time, some other like GDB/ptrace may
implement their own step handler.

Therefore, this patch introduces a new perf_event_attr bit field, so
that arch specific code(specially on arm64) can make a decision to
enable single stepping.

Any architecture which is not using this field will not have any
side effect.

Signed-off-by: Pratyush Anand <panand@redhat.com>
---
 include/linux/hw_breakpoint.h         | 6 ++++++
 include/uapi/linux/perf_event.h       | 3 ++-
 kernel/events/core.c                  | 2 ++
 tools/include/uapi/linux/perf_event.h | 3 ++-
 4 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
index 0464c85e63fd..6173ae048cbc 100644
--- a/include/linux/hw_breakpoint.h
+++ b/include/linux/hw_breakpoint.h
@@ -38,6 +38,12 @@ static inline int hw_breakpoint_type(struct perf_event *bp)
 	return bp->attr.bp_type;
 }
 
+static inline bool
+hw_breakpoint_needs_single_step(struct perf_event *bp)
+{
+	return bp->attr.step_needed;
+}
+
 static inline unsigned long hw_breakpoint_len(struct perf_event *bp)
 {
 	return bp->attr.bp_len;
diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
index b1c0b187acfe..00935808de0d 100644
--- a/include/uapi/linux/perf_event.h
+++ b/include/uapi/linux/perf_event.h
@@ -345,7 +345,8 @@ struct perf_event_attr {
 				context_switch :  1, /* context switch data */
 				write_backward :  1, /* Write ring buffer from end to beginning */
 				namespaces     :  1, /* include namespaces data */
-				__reserved_1   : 35;
+				step_needed    :  1, /* Use arch step handler */
+				__reserved_1   : 34;
 
 	union {
 		__u32		wakeup_events;	  /* wakeup every n events */
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 6c4e523dc1e2..220e26941475 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -9444,9 +9444,11 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	} else if (is_write_backward(event)){
 		event->overflow_handler = perf_event_output_backward;
 		event->overflow_handler_context = NULL;
+		event->attr.step_needed = 1;
 	} else {
 		event->overflow_handler = perf_event_output_forward;
 		event->overflow_handler_context = NULL;
+		event->attr.step_needed = 1;
 	}
 
 	perf_event__state_init(event);
diff --git a/tools/include/uapi/linux/perf_event.h b/tools/include/uapi/linux/perf_event.h
index b1c0b187acfe..00935808de0d 100644
--- a/tools/include/uapi/linux/perf_event.h
+++ b/tools/include/uapi/linux/perf_event.h
@@ -345,7 +345,8 @@ struct perf_event_attr {
 				context_switch :  1, /* context switch data */
 				write_backward :  1, /* Write ring buffer from end to beginning */
 				namespaces     :  1, /* include namespaces data */
-				__reserved_1   : 35;
+				step_needed    :  1, /* Use arch step handler */
+				__reserved_1   : 34;
 
 	union {
 		__u32		wakeup_events;	  /* wakeup every n events */
-- 
2.9.3

[PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Pratyush Anand]

Architecture like ARM64 currently allows to use default hw breakpoint
single step handler only to perf. However, some other users like few
systemtap tests or kernel test in
samples/hw_breakpoint/data_breakpoint.c can also work with default step
handler implementation. At the same time, some other like GDB/ptrace may
implement their own step handler.

Therefore, this patch introduces a new perf_event_attr bit field, so
that arch specific code(specially on arm64) can make a decision to
enable single stepping.

Any architecture which is not using this field will not have any
side effect.

Signed-off-by: Pratyush Anand <panand@redhat.com>
---
 include/linux/hw_breakpoint.h         | 6 ++++++
 include/uapi/linux/perf_event.h       | 3 ++-
 kernel/events/core.c                  | 2 ++
 tools/include/uapi/linux/perf_event.h | 3 ++-
 4 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
index 0464c85e63fd..6173ae048cbc 100644
--- a/include/linux/hw_breakpoint.h
+++ b/include/linux/hw_breakpoint.h
@@ -38,6 +38,12 @@ static inline int hw_breakpoint_type(struct perf_event *bp)
 	return bp->attr.bp_type;
 }
 
+static inline bool
+hw_breakpoint_needs_single_step(struct perf_event *bp)
+{
+	return bp->attr.step_needed;
+}
+
 static inline unsigned long hw_breakpoint_len(struct perf_event *bp)
 {
 	return bp->attr.bp_len;
diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
index b1c0b187acfe..00935808de0d 100644
--- a/include/uapi/linux/perf_event.h
+++ b/include/uapi/linux/perf_event.h
@@ -345,7 +345,8 @@ struct perf_event_attr {
 				context_switch :  1, /* context switch data */
 				write_backward :  1, /* Write ring buffer from end to beginning */
 				namespaces     :  1, /* include namespaces data */
-				__reserved_1   : 35;
+				step_needed    :  1, /* Use arch step handler */
+				__reserved_1   : 34;
 
 	union {
 		__u32		wakeup_events;	  /* wakeup every n events */
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 6c4e523dc1e2..220e26941475 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -9444,9 +9444,11 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	} else if (is_write_backward(event)){
 		event->overflow_handler = perf_event_output_backward;
 		event->overflow_handler_context = NULL;
+		event->attr.step_needed = 1;
 	} else {
 		event->overflow_handler = perf_event_output_forward;
 		event->overflow_handler_context = NULL;
+		event->attr.step_needed = 1;
 	}
 
 	perf_event__state_init(event);
diff --git a/tools/include/uapi/linux/perf_event.h b/tools/include/uapi/linux/perf_event.h
index b1c0b187acfe..00935808de0d 100644
--- a/tools/include/uapi/linux/perf_event.h
+++ b/tools/include/uapi/linux/perf_event.h
@@ -345,7 +345,8 @@ struct perf_event_attr {
 				context_switch :  1, /* context switch data */
 				write_backward :  1, /* Write ring buffer from end to beginning */
 				namespaces     :  1, /* include namespaces data */
-				__reserved_1   : 35;
+				step_needed    :  1, /* Use arch step handler */
+				__reserved_1   : 34;
 
 	union {
 		__u32		wakeup_events;	  /* wakeup every n events */
-- 
2.9.3

[PATCH V2 2/4] arm64: use hw_breakpoint_needs_single_step() to decide if step is needed

[Pratyush Anand]

Currently we use is_default_overflow_handler() to decide whether a
"step" will be needed or not. However, is_default_overflow_handler() is
true only for perf implementation. There can be some custom kernel
module tests like samples/hw_breakpoint/data_breakpoint.c which can
rely on default step handler.

hw_breakpoint_needs_single_step() will be true if any hw_breakpoint user
wants to use default step handler and sets step_needed in attribute.

Signed-off-by: Pratyush Anand <panand@redhat.com>
---
 arch/arm64/kernel/hw_breakpoint.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c
index 749f81779420..9a73f85ab9ad 100644
--- a/arch/arm64/kernel/hw_breakpoint.c
+++ b/arch/arm64/kernel/hw_breakpoint.c
@@ -661,7 +661,7 @@ static int breakpoint_handler(unsigned long unused, unsigned int esr,
 		perf_bp_event(bp, regs);
 
 		/* Do we need to handle the stepping? */
-		if (is_default_overflow_handler(bp))
+		if (hw_breakpoint_needs_single_step(bp))
 			step = 1;
 unlock:
 		rcu_read_unlock();
@@ -789,7 +789,7 @@ static int watchpoint_handler(unsigned long addr, unsigned int esr,
 		perf_bp_event(wp, regs);
 
 		/* Do we need to handle the stepping? */
-		if (is_default_overflow_handler(wp))
+		if (hw_breakpoint_needs_single_step(wp))
 			step = 1;
 	}
 	if (min_dist > 0 && min_dist != -1) {
@@ -800,7 +800,7 @@ static int watchpoint_handler(unsigned long addr, unsigned int esr,
 		perf_bp_event(wp, regs);
 
 		/* Do we need to handle the stepping? */
-		if (is_default_overflow_handler(wp))
+		if (hw_breakpoint_needs_single_step(wp))
 			step = 1;
 	}
 	rcu_read_unlock();
-- 
2.9.3

[PATCH V2 2/4] arm64: use hw_breakpoint_needs_single_step() to decide if step is needed

[Pratyush Anand]

Currently we use is_default_overflow_handler() to decide whether a
"step" will be needed or not. However, is_default_overflow_handler() is
true only for perf implementation. There can be some custom kernel
module tests like samples/hw_breakpoint/data_breakpoint.c which can
rely on default step handler.

hw_breakpoint_needs_single_step() will be true if any hw_breakpoint user
wants to use default step handler and sets step_needed in attribute.

Signed-off-by: Pratyush Anand <panand@redhat.com>
---
 arch/arm64/kernel/hw_breakpoint.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c
index 749f81779420..9a73f85ab9ad 100644
--- a/arch/arm64/kernel/hw_breakpoint.c
+++ b/arch/arm64/kernel/hw_breakpoint.c
@@ -661,7 +661,7 @@ static int breakpoint_handler(unsigned long unused, unsigned int esr,
 		perf_bp_event(bp, regs);
 
 		/* Do we need to handle the stepping? */
-		if (is_default_overflow_handler(bp))
+		if (hw_breakpoint_needs_single_step(bp))
 			step = 1;
 unlock:
 		rcu_read_unlock();
@@ -789,7 +789,7 @@ static int watchpoint_handler(unsigned long addr, unsigned int esr,
 		perf_bp_event(wp, regs);
 
 		/* Do we need to handle the stepping? */
-		if (is_default_overflow_handler(wp))
+		if (hw_breakpoint_needs_single_step(wp))
 			step = 1;
 	}
 	if (min_dist > 0 && min_dist != -1) {
@@ -800,7 +800,7 @@ static int watchpoint_handler(unsigned long addr, unsigned int esr,
 		perf_bp_event(wp, regs);
 
 		/* Do we need to handle the stepping? */
-		if (is_default_overflow_handler(wp))
+		if (hw_breakpoint_needs_single_step(wp))
 			step = 1;
 	}
 	rcu_read_unlock();
-- 
2.9.3

[PATCH V2 3/4] hw-breakpoint: sample test: set step_needed bit field

[Pratyush Anand]

arch like ARM64 expects 'step_needed == true'  in order to use default
single step handler. Therefore, set the bit field in the test case.
Other arch will not have any affect as they do not use it so far.

Signed-off-by: Pratyush Anand <panand@redhat.com>
---
 samples/hw_breakpoint/data_breakpoint.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/samples/hw_breakpoint/data_breakpoint.c b/samples/hw_breakpoint/data_breakpoint.c
index ef7f32291852..5a1919d01800 100644
--- a/samples/hw_breakpoint/data_breakpoint.c
+++ b/samples/hw_breakpoint/data_breakpoint.c
@@ -59,6 +59,7 @@ static int __init hw_break_module_init(void)
 	attr.bp_addr = kallsyms_lookup_name(ksym_name);
 	attr.bp_len = HW_BREAKPOINT_LEN_4;
 	attr.bp_type = HW_BREAKPOINT_W | HW_BREAKPOINT_R;
+	attr.step_needed = true;
 
 	sample_hbp = register_wide_hw_breakpoint(&attr, sample_hbp_handler, NULL);
 	if (IS_ERR((void __force *)sample_hbp)) {
-- 
2.9.3

[PATCH V2 3/4] hw-breakpoint: sample test: set step_needed bit field

[Pratyush Anand]

arch like ARM64 expects 'step_needed == true'  in order to use default
single step handler. Therefore, set the bit field in the test case.
Other arch will not have any affect as they do not use it so far.

Signed-off-by: Pratyush Anand <panand@redhat.com>
---
 samples/hw_breakpoint/data_breakpoint.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/samples/hw_breakpoint/data_breakpoint.c b/samples/hw_breakpoint/data_breakpoint.c
index ef7f32291852..5a1919d01800 100644
--- a/samples/hw_breakpoint/data_breakpoint.c
+++ b/samples/hw_breakpoint/data_breakpoint.c
@@ -59,6 +59,7 @@ static int __init hw_break_module_init(void)
 	attr.bp_addr = kallsyms_lookup_name(ksym_name);
 	attr.bp_len = HW_BREAKPOINT_LEN_4;
 	attr.bp_type = HW_BREAKPOINT_W | HW_BREAKPOINT_R;
+	attr.step_needed = true;
 
 	sample_hbp = register_wide_hw_breakpoint(&attr, sample_hbp_handler, NULL);
 	if (IS_ERR((void __force *)sample_hbp)) {
-- 
2.9.3

[PATCH V2 4/4] arm64: disable irq between breakpoint and step exception

[Pratyush Anand]

If an interrupt is generated between breakpoint and step handler then
step handler can not get correct step address. This situation can easily
be invoked by samples/hw_breakpoint/data_breakpoint.c. It can also be
reproduced if we insert any printk() statement or dump_stack() in perf
overflow_handler. So, it seems that perf is working fine just luckily.
If the CPU which is handling perf breakpoint handler receives any
interrupt then, perf step handler will not execute sanely.

This patch improves do_debug_exception() handling, which enforces now,
that exception handler function:
- should return 0 for any software breakpoint and hw
breakpoint/watchpoint handler if it does not expect a single step stage
- should return 1 if it expects single step.
- A single step handler should always return 0.
- All handler should return a -ve error in any other case.

Now, we can know in do_debug_exception() that whether a step exception
will be followed or not. If there will a step exception then disable
irq. Re-enable it after single step handling.

Signed-off-by: Pratyush Anand <panand@redhat.com>
---
 arch/arm64/kernel/debug-monitors.c |  3 +++
 arch/arm64/kernel/hw_breakpoint.c  |  4 ++--
 arch/arm64/mm/fault.c              | 22 ++++++++++++++++++----
 3 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c
index d618e25c3de1..16f29f853b54 100644
--- a/arch/arm64/kernel/debug-monitors.c
+++ b/arch/arm64/kernel/debug-monitors.c
@@ -325,6 +325,9 @@ static int brk_handler(unsigned long addr, unsigned int esr,
 		return -EFAULT;
 	}
 
+	if (kernel_active_single_step() || test_thread_flag(TIF_SINGLESTEP))
+		return 1;
+
 	return 0;
 }
 NOKPROBE_SYMBOL(brk_handler);
diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c
index 9a73f85ab9ad..d39b8039c70e 100644
--- a/arch/arm64/kernel/hw_breakpoint.c
+++ b/arch/arm64/kernel/hw_breakpoint.c
@@ -697,7 +697,7 @@ static int breakpoint_handler(unsigned long unused, unsigned int esr,
 		}
 	}
 
-	return 0;
+	return 1;
 }
 NOKPROBE_SYMBOL(breakpoint_handler);
 
@@ -840,7 +840,7 @@ static int watchpoint_handler(unsigned long addr, unsigned int esr,
 		}
 	}
 
-	return 0;
+	return 1;
 }
 NOKPROBE_SYMBOL(watchpoint_handler);
 
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 37b95dff0b07..ce5290dacba3 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -653,6 +653,13 @@ static struct fault_info __refdata debug_fault_info[] = {
 	{ do_bad,	SIGBUS,		0,		"unknown 7"		},
 };
 
+/*
+ * fn should return 0 from any software breakpoint and hw
+ * breakpoint/watchpoint handler if it does not expect a single step stage
+ * and 1 if it expects single step followed by its execution. A single step
+ * handler should always return 0. All handler should return a -ve error in
+ * any other case.
+ */
 void __init hook_debug_fault_code(int nr,
 				  int (*fn)(unsigned long, unsigned int, struct pt_regs *),
 				  int sig, int code, const char *name)
@@ -665,6 +672,8 @@ void __init hook_debug_fault_code(int nr,
 	debug_fault_info[nr].name	= name;
 }
 
+static DEFINE_PER_CPU(bool, irq_enable_needed);
+
 asmlinkage int __exception do_debug_exception(unsigned long addr,
 					      unsigned int esr,
 					      struct pt_regs *regs)
@@ -672,6 +681,7 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
 	const struct fault_info *inf = debug_fault_info + DBG_ESR_EVT(esr);
 	struct siginfo info;
 	int rv;
+	bool *irq_en_needed = this_cpu_ptr(&irq_enable_needed);
 
 	/*
 	 * Tell lockdep we disabled irqs in entry.S. Do nothing if they were
@@ -680,9 +690,8 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
 	if (interrupts_enabled(regs))
 		trace_hardirqs_off();
 
-	if (!inf->fn(addr, esr, regs)) {
-		rv = 1;
-	} else {
+	rv = inf->fn(addr, esr, regs);
+	if (rv < 0) {
 		pr_alert("Unhandled debug exception: %s (0x%08x) at 0x%016lx\n",
 			 inf->name, esr, addr);
 
@@ -691,7 +700,12 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
 		info.si_code  = inf->code;
 		info.si_addr  = (void __user *)addr;
 		arm64_notify_die("", regs, &info, 0);
-		rv = 0;
+	} else if (rv == 1 && interrupts_enabled(regs)) {
+		regs->pstate |= PSR_I_BIT;
+		*irq_en_needed = true;
+	} else if (rv == 0 && *irq_en_needed) {
+		regs->pstate &= ~PSR_I_BIT;
+		*irq_en_needed = false;
 	}
 
 	if (interrupts_enabled(regs))
-- 
2.9.3

[PATCH V2 4/4] arm64: disable irq between breakpoint and step exception

[Pratyush Anand]

If an interrupt is generated between breakpoint and step handler then
step handler can not get correct step address. This situation can easily
be invoked by samples/hw_breakpoint/data_breakpoint.c. It can also be
reproduced if we insert any printk() statement or dump_stack() in perf
overflow_handler. So, it seems that perf is working fine just luckily.
If the CPU which is handling perf breakpoint handler receives any
interrupt then, perf step handler will not execute sanely.

This patch improves do_debug_exception() handling, which enforces now,
that exception handler function:
- should return 0 for any software breakpoint and hw
breakpoint/watchpoint handler if it does not expect a single step stage
- should return 1 if it expects single step.
- A single step handler should always return 0.
- All handler should return a -ve error in any other case.

Now, we can know in do_debug_exception() that whether a step exception
will be followed or not. If there will a step exception then disable
irq. Re-enable it after single step handling.

Signed-off-by: Pratyush Anand <panand@redhat.com>
---
 arch/arm64/kernel/debug-monitors.c |  3 +++
 arch/arm64/kernel/hw_breakpoint.c  |  4 ++--
 arch/arm64/mm/fault.c              | 22 ++++++++++++++++++----
 3 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c
index d618e25c3de1..16f29f853b54 100644
--- a/arch/arm64/kernel/debug-monitors.c
+++ b/arch/arm64/kernel/debug-monitors.c
@@ -325,6 +325,9 @@ static int brk_handler(unsigned long addr, unsigned int esr,
 		return -EFAULT;
 	}
 
+	if (kernel_active_single_step() || test_thread_flag(TIF_SINGLESTEP))
+		return 1;
+
 	return 0;
 }
 NOKPROBE_SYMBOL(brk_handler);
diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c
index 9a73f85ab9ad..d39b8039c70e 100644
--- a/arch/arm64/kernel/hw_breakpoint.c
+++ b/arch/arm64/kernel/hw_breakpoint.c
@@ -697,7 +697,7 @@ static int breakpoint_handler(unsigned long unused, unsigned int esr,
 		}
 	}
 
-	return 0;
+	return 1;
 }
 NOKPROBE_SYMBOL(breakpoint_handler);
 
@@ -840,7 +840,7 @@ static int watchpoint_handler(unsigned long addr, unsigned int esr,
 		}
 	}
 
-	return 0;
+	return 1;
 }
 NOKPROBE_SYMBOL(watchpoint_handler);
 
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 37b95dff0b07..ce5290dacba3 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -653,6 +653,13 @@ static struct fault_info __refdata debug_fault_info[] = {
 	{ do_bad,	SIGBUS,		0,		"unknown 7"		},
 };
 
+/*
+ * fn should return 0 from any software breakpoint and hw
+ * breakpoint/watchpoint handler if it does not expect a single step stage
+ * and 1 if it expects single step followed by its execution. A single step
+ * handler should always return 0. All handler should return a -ve error in
+ * any other case.
+ */
 void __init hook_debug_fault_code(int nr,
 				  int (*fn)(unsigned long, unsigned int, struct pt_regs *),
 				  int sig, int code, const char *name)
@@ -665,6 +672,8 @@ void __init hook_debug_fault_code(int nr,
 	debug_fault_info[nr].name	= name;
 }
 
+static DEFINE_PER_CPU(bool, irq_enable_needed);
+
 asmlinkage int __exception do_debug_exception(unsigned long addr,
 					      unsigned int esr,
 					      struct pt_regs *regs)
@@ -672,6 +681,7 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
 	const struct fault_info *inf = debug_fault_info + DBG_ESR_EVT(esr);
 	struct siginfo info;
 	int rv;
+	bool *irq_en_needed = this_cpu_ptr(&irq_enable_needed);
 
 	/*
 	 * Tell lockdep we disabled irqs in entry.S. Do nothing if they were
@@ -680,9 +690,8 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
 	if (interrupts_enabled(regs))
 		trace_hardirqs_off();
 
-	if (!inf->fn(addr, esr, regs)) {
-		rv = 1;
-	} else {
+	rv = inf->fn(addr, esr, regs);
+	if (rv < 0) {
 		pr_alert("Unhandled debug exception: %s (0x%08x)@0x%016lx\n",
 			 inf->name, esr, addr);
 
@@ -691,7 +700,12 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
 		info.si_code  = inf->code;
 		info.si_addr  = (void __user *)addr;
 		arm64_notify_die("", regs, &info, 0);
-		rv = 0;
+	} else if (rv == 1 && interrupts_enabled(regs)) {
+		regs->pstate |= PSR_I_BIT;
+		*irq_en_needed = true;
+	} else if (rv == 0 && *irq_en_needed) {
+		regs->pstate &= ~PSR_I_BIT;
+		*irq_en_needed = false;
 	}
 
 	if (interrupts_enabled(regs))
-- 
2.9.3

Re: [PATCH V2 0/4] ARM64: Fix irq generation between breakpoint and step exception

[Pratyush Anand]

On Friday 07 July 2017 05:33 PM, Pratyush Anand wrote:
> v1 was here http://marc.info/?l=linux-arm-kernel&m=149910958418708&w=2
>
> v1 -> v2:
> - patch 1 of v1 has been modified to patch 1-3 of v2.
> - Introduced a new event attribute step_needed and implemented
>   hw_breakpoint_needs_single_step() (patch 1)
> - Replaced usage of is_default_overflow_handler() with
>   hw_breakpoint_needs_single_step(). (patch 2)
> - Modified sample test to set set step_needed bit field (patch 3)
>
> samples/hw_breakpoint/data_breakpoint.c passes with x86_64 but fails with
> ARM64. Even though it has been NAKed previously on upstream [1, 2], I have
> tried to come up with patches which can resolve it for ARM64 as well.
>
> I noticed that even perf step exception can go into an infinite loop if CPU
> receives an interrupt while executing breakpoint/watchpoint handler. So,
> event though we are not concerned about above test, we will have to find a
> solution for the perf issue.
>
> This patchset attempts to resolve both the issue. Please review.

Any comments/feedback?

>
> [1] http://marc.info/?l=linux-arm-kernel&m=149580777524910&w=2
> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2016-April/425266.html
>
> Pratyush Anand (4):
>   hw_breakpoint: Add step_needed event attribute
>   arm64: use hw_breakpoint_needs_single_step() to decide if step is
>     needed
>   hw-breakpoint: sample test: set step_needed bit field
>   arm64: disable irq between breakpoint and step exception
>
>  arch/arm64/kernel/debug-monitors.c      |  3 +++
>  arch/arm64/kernel/hw_breakpoint.c       | 10 +++++-----
>  arch/arm64/mm/fault.c                   | 22 ++++++++++++++++++----
>  include/linux/hw_breakpoint.h           |  6 ++++++
>  include/uapi/linux/perf_event.h         |  3 ++-
>  kernel/events/core.c                    |  2 ++
>  samples/hw_breakpoint/data_breakpoint.c |  1 +
>  tools/include/uapi/linux/perf_event.h   |  3 ++-
>  8 files changed, 39 insertions(+), 11 deletions(-)
>

--
Pratyush

[PATCH V2 0/4] ARM64: Fix irq generation between breakpoint and step exception

[Pratyush Anand]

On Friday 07 July 2017 05:33 PM, Pratyush Anand wrote:
> v1 was here http://marc.info/?l=linux-arm-kernel&m=149910958418708&w=2
>
> v1 -> v2:
> - patch 1 of v1 has been modified to patch 1-3 of v2.
> - Introduced a new event attribute step_needed and implemented
>   hw_breakpoint_needs_single_step() (patch 1)
> - Replaced usage of is_default_overflow_handler() with
>   hw_breakpoint_needs_single_step(). (patch 2)
> - Modified sample test to set set step_needed bit field (patch 3)
>
> samples/hw_breakpoint/data_breakpoint.c passes with x86_64 but fails with
> ARM64. Even though it has been NAKed previously on upstream [1, 2], I have
> tried to come up with patches which can resolve it for ARM64 as well.
>
> I noticed that even perf step exception can go into an infinite loop if CPU
> receives an interrupt while executing breakpoint/watchpoint handler. So,
> event though we are not concerned about above test, we will have to find a
> solution for the perf issue.
>
> This patchset attempts to resolve both the issue. Please review.

Any comments/feedback?

>
> [1] http://marc.info/?l=linux-arm-kernel&m=149580777524910&w=2
> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2016-April/425266.html
>
> Pratyush Anand (4):
>   hw_breakpoint: Add step_needed event attribute
>   arm64: use hw_breakpoint_needs_single_step() to decide if step is
>     needed
>   hw-breakpoint: sample test: set step_needed bit field
>   arm64: disable irq between breakpoint and step exception
>
>  arch/arm64/kernel/debug-monitors.c      |  3 +++
>  arch/arm64/kernel/hw_breakpoint.c       | 10 +++++-----
>  arch/arm64/mm/fault.c                   | 22 ++++++++++++++++++----
>  include/linux/hw_breakpoint.h           |  6 ++++++
>  include/uapi/linux/perf_event.h         |  3 ++-
>  kernel/events/core.c                    |  2 ++
>  samples/hw_breakpoint/data_breakpoint.c |  1 +
>  tools/include/uapi/linux/perf_event.h   |  3 ++-
>  8 files changed, 39 insertions(+), 11 deletions(-)
>

--
Pratyush

Re: [PATCH V2 4/4] arm64: disable irq between breakpoint and step exception

[Will Deacon]

On Fri, Jul 07, 2017 at 05:34:00PM +0530, Pratyush Anand wrote:
> If an interrupt is generated between breakpoint and step handler then
> step handler can not get correct step address. This situation can easily
> be invoked by samples/hw_breakpoint/data_breakpoint.c. It can also be
> reproduced if we insert any printk() statement or dump_stack() in perf
> overflow_handler. So, it seems that perf is working fine just luckily.
> If the CPU which is handling perf breakpoint handler receives any
> interrupt then, perf step handler will not execute sanely.
> 
> This patch improves do_debug_exception() handling, which enforces now,
> that exception handler function:
> - should return 0 for any software breakpoint and hw
> breakpoint/watchpoint handler if it does not expect a single step stage
> - should return 1 if it expects single step.
> - A single step handler should always return 0.
> - All handler should return a -ve error in any other case.
> 
> Now, we can know in do_debug_exception() that whether a step exception
> will be followed or not. If there will a step exception then disable
> irq. Re-enable it after single step handling.

AFAICT, this is only a problem for kernel-mode breakpoints where we end up
stepping into the interrupt handler when trying to step over a breakpoint.

We'd probably be better off getting all users of kernel step (kprobes, kgdb
and perf) to run the step with irqs disabled, but I still have reservations
about that:

  http://lists.infradead.org/pipermail/linux-arm-kernel/2017-May/508066.html
  http://lists.infradead.org/pipermail/linux-arm-kernel/2017-June/510814.html

Wouldn't it be better to follow kprobes/kgdb and have perf run the step with
irqs disabled?

Will

[PATCH V2 4/4] arm64: disable irq between breakpoint and step exception

[Will Deacon]

On Fri, Jul 07, 2017 at 05:34:00PM +0530, Pratyush Anand wrote:
> If an interrupt is generated between breakpoint and step handler then
> step handler can not get correct step address. This situation can easily
> be invoked by samples/hw_breakpoint/data_breakpoint.c. It can also be
> reproduced if we insert any printk() statement or dump_stack() in perf
> overflow_handler. So, it seems that perf is working fine just luckily.
> If the CPU which is handling perf breakpoint handler receives any
> interrupt then, perf step handler will not execute sanely.
> 
> This patch improves do_debug_exception() handling, which enforces now,
> that exception handler function:
> - should return 0 for any software breakpoint and hw
> breakpoint/watchpoint handler if it does not expect a single step stage
> - should return 1 if it expects single step.
> - A single step handler should always return 0.
> - All handler should return a -ve error in any other case.
> 
> Now, we can know in do_debug_exception() that whether a step exception
> will be followed or not. If there will a step exception then disable
> irq. Re-enable it after single step handling.

AFAICT, this is only a problem for kernel-mode breakpoints where we end up
stepping into the interrupt handler when trying to step over a breakpoint.

We'd probably be better off getting all users of kernel step (kprobes, kgdb
and perf) to run the step with irqs disabled, but I still have reservations
about that:

  http://lists.infradead.org/pipermail/linux-arm-kernel/2017-May/508066.html
  http://lists.infradead.org/pipermail/linux-arm-kernel/2017-June/510814.html

Wouldn't it be better to follow kprobes/kgdb and have perf run the step with
irqs disabled?

Will

Re: [PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Will Deacon]

On Fri, Jul 07, 2017 at 05:33:57PM +0530, Pratyush Anand wrote:
> Architecture like ARM64 currently allows to use default hw breakpoint
> single step handler only to perf. However, some other users like few
> systemtap tests or kernel test in
> samples/hw_breakpoint/data_breakpoint.c can also work with default step
> handler implementation. At the same time, some other like GDB/ptrace may
> implement their own step handler.
> 
> Therefore, this patch introduces a new perf_event_attr bit field, so
> that arch specific code(specially on arm64) can make a decision to
> enable single stepping.
> 
> Any architecture which is not using this field will not have any
> side effect.
> 
> Signed-off-by: Pratyush Anand <panand@redhat.com>
> ---
>  include/linux/hw_breakpoint.h         | 6 ++++++
>  include/uapi/linux/perf_event.h       | 3 ++-
>  kernel/events/core.c                  | 2 ++
>  tools/include/uapi/linux/perf_event.h | 3 ++-
>  4 files changed, 12 insertions(+), 2 deletions(-)
> 
> diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
> index 0464c85e63fd..6173ae048cbc 100644
> --- a/include/linux/hw_breakpoint.h
> +++ b/include/linux/hw_breakpoint.h
> @@ -38,6 +38,12 @@ static inline int hw_breakpoint_type(struct perf_event *bp)
>  	return bp->attr.bp_type;
>  }
>  
> +static inline bool
> +hw_breakpoint_needs_single_step(struct perf_event *bp)
> +{
> +	return bp->attr.step_needed;
> +}
> +
>  static inline unsigned long hw_breakpoint_len(struct perf_event *bp)
>  {
>  	return bp->attr.bp_len;
> diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> index b1c0b187acfe..00935808de0d 100644
> --- a/include/uapi/linux/perf_event.h
> +++ b/include/uapi/linux/perf_event.h
> @@ -345,7 +345,8 @@ struct perf_event_attr {
>  				context_switch :  1, /* context switch data */
>  				write_backward :  1, /* Write ring buffer from end to beginning */
>  				namespaces     :  1, /* include namespaces data */
> -				__reserved_1   : 35;
> +				step_needed    :  1, /* Use arch step handler */
> +				__reserved_1   : 34;

This needs documenting properly, as I really have no idea how userspace is
going to use it sensibley, especially as you silently overwrite it in some
cases below.

Will

>  
>  	union {
>  		__u32		wakeup_events;	  /* wakeup every n events */
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index 6c4e523dc1e2..220e26941475 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -9444,9 +9444,11 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
>  	} else if (is_write_backward(event)){
>  		event->overflow_handler = perf_event_output_backward;
>  		event->overflow_handler_context = NULL;
> +		event->attr.step_needed = 1;
>  	} else {
>  		event->overflow_handler = perf_event_output_forward;
>  		event->overflow_handler_context = NULL;
> +		event->attr.step_needed = 1;
>  	}
>  
>  	perf_event__state_init(event);
> diff --git a/tools/include/uapi/linux/perf_event.h b/tools/include/uapi/linux/perf_event.h
> index b1c0b187acfe..00935808de0d 100644
> --- a/tools/include/uapi/linux/perf_event.h
> +++ b/tools/include/uapi/linux/perf_event.h
> @@ -345,7 +345,8 @@ struct perf_event_attr {
>  				context_switch :  1, /* context switch data */
>  				write_backward :  1, /* Write ring buffer from end to beginning */
>  				namespaces     :  1, /* include namespaces data */
> -				__reserved_1   : 35;
> +				step_needed    :  1, /* Use arch step handler */
> +				__reserved_1   : 34;
>  
>  	union {
>  		__u32		wakeup_events;	  /* wakeup every n events */
> -- 
> 2.9.3
> 

[PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Will Deacon]

On Fri, Jul 07, 2017 at 05:33:57PM +0530, Pratyush Anand wrote:
> Architecture like ARM64 currently allows to use default hw breakpoint
> single step handler only to perf. However, some other users like few
> systemtap tests or kernel test in
> samples/hw_breakpoint/data_breakpoint.c can also work with default step
> handler implementation. At the same time, some other like GDB/ptrace may
> implement their own step handler.
> 
> Therefore, this patch introduces a new perf_event_attr bit field, so
> that arch specific code(specially on arm64) can make a decision to
> enable single stepping.
> 
> Any architecture which is not using this field will not have any
> side effect.
> 
> Signed-off-by: Pratyush Anand <panand@redhat.com>
> ---
>  include/linux/hw_breakpoint.h         | 6 ++++++
>  include/uapi/linux/perf_event.h       | 3 ++-
>  kernel/events/core.c                  | 2 ++
>  tools/include/uapi/linux/perf_event.h | 3 ++-
>  4 files changed, 12 insertions(+), 2 deletions(-)
> 
> diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
> index 0464c85e63fd..6173ae048cbc 100644
> --- a/include/linux/hw_breakpoint.h
> +++ b/include/linux/hw_breakpoint.h
> @@ -38,6 +38,12 @@ static inline int hw_breakpoint_type(struct perf_event *bp)
>  	return bp->attr.bp_type;
>  }
>  
> +static inline bool
> +hw_breakpoint_needs_single_step(struct perf_event *bp)
> +{
> +	return bp->attr.step_needed;
> +}
> +
>  static inline unsigned long hw_breakpoint_len(struct perf_event *bp)
>  {
>  	return bp->attr.bp_len;
> diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> index b1c0b187acfe..00935808de0d 100644
> --- a/include/uapi/linux/perf_event.h
> +++ b/include/uapi/linux/perf_event.h
> @@ -345,7 +345,8 @@ struct perf_event_attr {
>  				context_switch :  1, /* context switch data */
>  				write_backward :  1, /* Write ring buffer from end to beginning */
>  				namespaces     :  1, /* include namespaces data */
> -				__reserved_1   : 35;
> +				step_needed    :  1, /* Use arch step handler */
> +				__reserved_1   : 34;

This needs documenting properly, as I really have no idea how userspace is
going to use it sensibley, especially as you silently overwrite it in some
cases below.

Will

>  
>  	union {
>  		__u32		wakeup_events;	  /* wakeup every n events */
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index 6c4e523dc1e2..220e26941475 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -9444,9 +9444,11 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
>  	} else if (is_write_backward(event)){
>  		event->overflow_handler = perf_event_output_backward;
>  		event->overflow_handler_context = NULL;
> +		event->attr.step_needed = 1;
>  	} else {
>  		event->overflow_handler = perf_event_output_forward;
>  		event->overflow_handler_context = NULL;
> +		event->attr.step_needed = 1;
>  	}
>  
>  	perf_event__state_init(event);
> diff --git a/tools/include/uapi/linux/perf_event.h b/tools/include/uapi/linux/perf_event.h
> index b1c0b187acfe..00935808de0d 100644
> --- a/tools/include/uapi/linux/perf_event.h
> +++ b/tools/include/uapi/linux/perf_event.h
> @@ -345,7 +345,8 @@ struct perf_event_attr {
>  				context_switch :  1, /* context switch data */
>  				write_backward :  1, /* Write ring buffer from end to beginning */
>  				namespaces     :  1, /* include namespaces data */
> -				__reserved_1   : 35;
> +				step_needed    :  1, /* Use arch step handler */
> +				__reserved_1   : 34;
>  
>  	union {
>  		__u32		wakeup_events;	  /* wakeup every n events */
> -- 
> 2.9.3
> 

Re: [PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Peter Zijlstra]

On Tue, Jul 25, 2017 at 02:27:38PM +0100, Will Deacon wrote:
> On Fri, Jul 07, 2017 at 05:33:57PM +0530, Pratyush Anand wrote:
> > Architecture like ARM64 currently allows to use default hw breakpoint
> > single step handler only to perf. However, some other users like few
> > systemtap tests or kernel test in
> > samples/hw_breakpoint/data_breakpoint.c can also work with default step
> > handler implementation. At the same time, some other like GDB/ptrace may
> > implement their own step handler.
> > 
> > Therefore, this patch introduces a new perf_event_attr bit field, so
> > that arch specific code(specially on arm64) can make a decision to
> > enable single stepping.
> > 
> > Any architecture which is not using this field will not have any
> > side effect.

> > diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> > index b1c0b187acfe..00935808de0d 100644
> > --- a/include/uapi/linux/perf_event.h
> > +++ b/include/uapi/linux/perf_event.h
> > @@ -345,7 +345,8 @@ struct perf_event_attr {
> >  				context_switch :  1, /* context switch data */
> >  				write_backward :  1, /* Write ring buffer from end to beginning */
> >  				namespaces     :  1, /* include namespaces data */
> > -				__reserved_1   : 35;
> > +				step_needed    :  1, /* Use arch step handler */
> > +				__reserved_1   : 34;
> 
> This needs documenting properly, as I really have no idea how userspace is
> going to use it sensibley, especially as you silently overwrite it in some
> cases below.

This is not something userspace _can_ use sensibly afaict. Therefore it
should probably not live here.

[PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Peter Zijlstra]

On Tue, Jul 25, 2017 at 02:27:38PM +0100, Will Deacon wrote:
> On Fri, Jul 07, 2017 at 05:33:57PM +0530, Pratyush Anand wrote:
> > Architecture like ARM64 currently allows to use default hw breakpoint
> > single step handler only to perf. However, some other users like few
> > systemtap tests or kernel test in
> > samples/hw_breakpoint/data_breakpoint.c can also work with default step
> > handler implementation. At the same time, some other like GDB/ptrace may
> > implement their own step handler.
> > 
> > Therefore, this patch introduces a new perf_event_attr bit field, so
> > that arch specific code(specially on arm64) can make a decision to
> > enable single stepping.
> > 
> > Any architecture which is not using this field will not have any
> > side effect.

> > diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> > index b1c0b187acfe..00935808de0d 100644
> > --- a/include/uapi/linux/perf_event.h
> > +++ b/include/uapi/linux/perf_event.h
> > @@ -345,7 +345,8 @@ struct perf_event_attr {
> >  				context_switch :  1, /* context switch data */
> >  				write_backward :  1, /* Write ring buffer from end to beginning */
> >  				namespaces     :  1, /* include namespaces data */
> > -				__reserved_1   : 35;
> > +				step_needed    :  1, /* Use arch step handler */
> > +				__reserved_1   : 34;
> 
> This needs documenting properly, as I really have no idea how userspace is
> going to use it sensibley, especially as you silently overwrite it in some
> cases below.

This is not something userspace _can_ use sensibly afaict. Therefore it
should probably not live here.

Re: [PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Mark Rutland]

On Tue, Jul 25, 2017 at 04:14:23PM +0200, Peter Zijlstra wrote:
> On Tue, Jul 25, 2017 at 02:27:38PM +0100, Will Deacon wrote:
> > On Fri, Jul 07, 2017 at 05:33:57PM +0530, Pratyush Anand wrote:
> > > Architecture like ARM64 currently allows to use default hw breakpoint
> > > single step handler only to perf. However, some other users like few
> > > systemtap tests or kernel test in
> > > samples/hw_breakpoint/data_breakpoint.c can also work with default step
> > > handler implementation. At the same time, some other like GDB/ptrace may
> > > implement their own step handler.
> > > 
> > > Therefore, this patch introduces a new perf_event_attr bit field, so
> > > that arch specific code(specially on arm64) can make a decision to
> > > enable single stepping.
> > > 
> > > Any architecture which is not using this field will not have any
> > > side effect.
> 
> > > diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> > > index b1c0b187acfe..00935808de0d 100644
> > > --- a/include/uapi/linux/perf_event.h
> > > +++ b/include/uapi/linux/perf_event.h
> > > @@ -345,7 +345,8 @@ struct perf_event_attr {
> > >  				context_switch :  1, /* context switch data */
> > >  				write_backward :  1, /* Write ring buffer from end to beginning */
> > >  				namespaces     :  1, /* include namespaces data */
> > > -				__reserved_1   : 35;
> > > +				step_needed    :  1, /* Use arch step handler */
> > > +				__reserved_1   : 34;
> > 
> > This needs documenting properly, as I really have no idea how userspace is
> > going to use it sensibley, especially as you silently overwrite it in some
> > cases below.
> 
> This is not something userspace _can_ use sensibly afaict. Therefore it
> should probably not live here.

Indeed. When I suggested this, I meant that it would be a
kernel-internal flag, and not UAPI.

Thanks,
Mark.

[PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Mark Rutland]

On Tue, Jul 25, 2017 at 04:14:23PM +0200, Peter Zijlstra wrote:
> On Tue, Jul 25, 2017 at 02:27:38PM +0100, Will Deacon wrote:
> > On Fri, Jul 07, 2017 at 05:33:57PM +0530, Pratyush Anand wrote:
> > > Architecture like ARM64 currently allows to use default hw breakpoint
> > > single step handler only to perf. However, some other users like few
> > > systemtap tests or kernel test in
> > > samples/hw_breakpoint/data_breakpoint.c can also work with default step
> > > handler implementation. At the same time, some other like GDB/ptrace may
> > > implement their own step handler.
> > > 
> > > Therefore, this patch introduces a new perf_event_attr bit field, so
> > > that arch specific code(specially on arm64) can make a decision to
> > > enable single stepping.
> > > 
> > > Any architecture which is not using this field will not have any
> > > side effect.
> 
> > > diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> > > index b1c0b187acfe..00935808de0d 100644
> > > --- a/include/uapi/linux/perf_event.h
> > > +++ b/include/uapi/linux/perf_event.h
> > > @@ -345,7 +345,8 @@ struct perf_event_attr {
> > >  				context_switch :  1, /* context switch data */
> > >  				write_backward :  1, /* Write ring buffer from end to beginning */
> > >  				namespaces     :  1, /* include namespaces data */
> > > -				__reserved_1   : 35;
> > > +				step_needed    :  1, /* Use arch step handler */
> > > +				__reserved_1   : 34;
> > 
> > This needs documenting properly, as I really have no idea how userspace is
> > going to use it sensibley, especially as you silently overwrite it in some
> > cases below.
> 
> This is not something userspace _can_ use sensibly afaict. Therefore it
> should probably not live here.

Indeed. When I suggested this, I meant that it would be a
kernel-internal flag, and not UAPI.

Thanks,
Mark.

Re: [PATCH V2 4/4] arm64: disable irq between breakpoint and step exception

[Pratyush Anand]

Hi Will,

Thanks for your review.

On Tuesday 25 July 2017 06:55 PM, Will Deacon wrote:
> On Fri, Jul 07, 2017 at 05:34:00PM +0530, Pratyush Anand wrote:
>> If an interrupt is generated between breakpoint and step handler then
>> step handler can not get correct step address. This situation can easily
>> be invoked by samples/hw_breakpoint/data_breakpoint.c. It can also be
>> reproduced if we insert any printk() statement or dump_stack() in perf
>> overflow_handler. So, it seems that perf is working fine just luckily.
>> If the CPU which is handling perf breakpoint handler receives any
>> interrupt then, perf step handler will not execute sanely.
>>
>> This patch improves do_debug_exception() handling, which enforces now,
>> that exception handler function:
>> - should return 0 for any software breakpoint and hw
>> breakpoint/watchpoint handler if it does not expect a single step stage
>> - should return 1 if it expects single step.
>> - A single step handler should always return 0.
>> - All handler should return a -ve error in any other case.
>>
>> Now, we can know in do_debug_exception() that whether a step exception
>> will be followed or not. If there will a step exception then disable
>> irq. Re-enable it after single step handling.
> 
> AFAICT, this is only a problem for kernel-mode breakpoints where we end up
> stepping into the interrupt handler when trying to step over a breakpoint.

I think yes.

> 
> We'd probably be better off getting all users of kernel step (kprobes, kgdb
> and perf) to run the step with irqs disabled,


That should be doable. We can easily manage all of them in 
do_debug_exception() if individual brk handlers return correct value as per 
the rule mentioned in the commit log of this patch.

I think, I can take care of kprobes and kgdb as well in next version of patch.

> but I still have reservations
> about that:

So, IIUC, you have concern about faulting of a instruction being stepped. 
Since we will have a notion of *irq_en_needed*, so I think, if needed we can 
re-enable interrupt in fault handler do_mem_abort().

Whats your opinion here?

> 
>    http://lists.infradead.org/pipermail/linux-arm-kernel/2017-May/508066.html
>    http://lists.infradead.org/pipermail/linux-arm-kernel/2017-June/510814.html
>
> Wouldn't it be better to follow kprobes/kgdb and have perf run the step with
> irqs disabled?
-- 
Regards
Pratyush

[PATCH V2 4/4] arm64: disable irq between breakpoint and step exception

[Pratyush Anand]

Hi Will,

Thanks for your review.

On Tuesday 25 July 2017 06:55 PM, Will Deacon wrote:
> On Fri, Jul 07, 2017 at 05:34:00PM +0530, Pratyush Anand wrote:
>> If an interrupt is generated between breakpoint and step handler then
>> step handler can not get correct step address. This situation can easily
>> be invoked by samples/hw_breakpoint/data_breakpoint.c. It can also be
>> reproduced if we insert any printk() statement or dump_stack() in perf
>> overflow_handler. So, it seems that perf is working fine just luckily.
>> If the CPU which is handling perf breakpoint handler receives any
>> interrupt then, perf step handler will not execute sanely.
>>
>> This patch improves do_debug_exception() handling, which enforces now,
>> that exception handler function:
>> - should return 0 for any software breakpoint and hw
>> breakpoint/watchpoint handler if it does not expect a single step stage
>> - should return 1 if it expects single step.
>> - A single step handler should always return 0.
>> - All handler should return a -ve error in any other case.
>>
>> Now, we can know in do_debug_exception() that whether a step exception
>> will be followed or not. If there will a step exception then disable
>> irq. Re-enable it after single step handling.
> 
> AFAICT, this is only a problem for kernel-mode breakpoints where we end up
> stepping into the interrupt handler when trying to step over a breakpoint.

I think yes.

> 
> We'd probably be better off getting all users of kernel step (kprobes, kgdb
> and perf) to run the step with irqs disabled,


That should be doable. We can easily manage all of them in 
do_debug_exception() if individual brk handlers return correct value as per 
the rule mentioned in the commit log of this patch.

I think, I can take care of kprobes and kgdb as well in next version of patch.

> but I still have reservations
> about that:

So, IIUC, you have concern about faulting of a instruction being stepped. 
Since we will have a notion of *irq_en_needed*, so I think, if needed we can 
re-enable interrupt in fault handler do_mem_abort().

Whats your opinion here?

> 
>    http://lists.infradead.org/pipermail/linux-arm-kernel/2017-May/508066.html
>    http://lists.infradead.org/pipermail/linux-arm-kernel/2017-June/510814.html
>
> Wouldn't it be better to follow kprobes/kgdb and have perf run the step with
> irqs disabled?
-- 
Regards
Pratyush

Re: [PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Pratyush Anand]

On Tuesday 25 July 2017 06:57 PM, Will Deacon wrote:
> On Fri, Jul 07, 2017 at 05:33:57PM +0530, Pratyush Anand wrote:
>> Architecture like ARM64 currently allows to use default hw breakpoint
>> single step handler only to perf. However, some other users like few
>> systemtap tests or kernel test in
>> samples/hw_breakpoint/data_breakpoint.c can also work with default step
>> handler implementation. At the same time, some other like GDB/ptrace may
>> implement their own step handler.
>>
>> Therefore, this patch introduces a new perf_event_attr bit field, so
>> that arch specific code(specially on arm64) can make a decision to
>> enable single stepping.
>>
>> Any architecture which is not using this field will not have any
>> side effect.
>>
>> Signed-off-by: Pratyush Anand <panand@redhat.com>
>> ---
>>   include/linux/hw_breakpoint.h         | 6 ++++++
>>   include/uapi/linux/perf_event.h       | 3 ++-
>>   kernel/events/core.c                  | 2 ++
>>   tools/include/uapi/linux/perf_event.h | 3 ++-
>>   4 files changed, 12 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
>> index 0464c85e63fd..6173ae048cbc 100644
>> --- a/include/linux/hw_breakpoint.h
>> +++ b/include/linux/hw_breakpoint.h
>> @@ -38,6 +38,12 @@ static inline int hw_breakpoint_type(struct perf_event *bp)
>>   	return bp->attr.bp_type;
>>   }
>>   
>> +static inline bool
>> +hw_breakpoint_needs_single_step(struct perf_event *bp)
>> +{
>> +	return bp->attr.step_needed;
>> +}
>> +
>>   static inline unsigned long hw_breakpoint_len(struct perf_event *bp)
>>   {
>>   	return bp->attr.bp_len;
>> diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
>> index b1c0b187acfe..00935808de0d 100644
>> --- a/include/uapi/linux/perf_event.h
>> +++ b/include/uapi/linux/perf_event.h
>> @@ -345,7 +345,8 @@ struct perf_event_attr {
>>   				context_switch :  1, /* context switch data */
>>   				write_backward :  1, /* Write ring buffer from end to beginning */
>>   				namespaces     :  1, /* include namespaces data */
>> -				__reserved_1   : 35;
>> +				step_needed    :  1, /* Use arch step handler */
>> +				__reserved_1   : 34;
> 
> This needs documenting properly, as I really have no idea how userspace is
> going to use it sensibley, especially as you silently overwrite it in some
> cases below.

I too had thought to put it under include/linux/perf_event.h : struct 
perf_event. But, see hw_break_module_init() which does not have knowledge of 
this structure, and we need to have some way so that none-perf kernel module 
implementation can tell that it needs default arch step handler.

Do you see any alternative?

-- 
Regards
Pratyush

[PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Pratyush Anand]

On Tuesday 25 July 2017 06:57 PM, Will Deacon wrote:
> On Fri, Jul 07, 2017 at 05:33:57PM +0530, Pratyush Anand wrote:
>> Architecture like ARM64 currently allows to use default hw breakpoint
>> single step handler only to perf. However, some other users like few
>> systemtap tests or kernel test in
>> samples/hw_breakpoint/data_breakpoint.c can also work with default step
>> handler implementation. At the same time, some other like GDB/ptrace may
>> implement their own step handler.
>>
>> Therefore, this patch introduces a new perf_event_attr bit field, so
>> that arch specific code(specially on arm64) can make a decision to
>> enable single stepping.
>>
>> Any architecture which is not using this field will not have any
>> side effect.
>>
>> Signed-off-by: Pratyush Anand <panand@redhat.com>
>> ---
>>   include/linux/hw_breakpoint.h         | 6 ++++++
>>   include/uapi/linux/perf_event.h       | 3 ++-
>>   kernel/events/core.c                  | 2 ++
>>   tools/include/uapi/linux/perf_event.h | 3 ++-
>>   4 files changed, 12 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
>> index 0464c85e63fd..6173ae048cbc 100644
>> --- a/include/linux/hw_breakpoint.h
>> +++ b/include/linux/hw_breakpoint.h
>> @@ -38,6 +38,12 @@ static inline int hw_breakpoint_type(struct perf_event *bp)
>>   	return bp->attr.bp_type;
>>   }
>>   
>> +static inline bool
>> +hw_breakpoint_needs_single_step(struct perf_event *bp)
>> +{
>> +	return bp->attr.step_needed;
>> +}
>> +
>>   static inline unsigned long hw_breakpoint_len(struct perf_event *bp)
>>   {
>>   	return bp->attr.bp_len;
>> diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
>> index b1c0b187acfe..00935808de0d 100644
>> --- a/include/uapi/linux/perf_event.h
>> +++ b/include/uapi/linux/perf_event.h
>> @@ -345,7 +345,8 @@ struct perf_event_attr {
>>   				context_switch :  1, /* context switch data */
>>   				write_backward :  1, /* Write ring buffer from end to beginning */
>>   				namespaces     :  1, /* include namespaces data */
>> -				__reserved_1   : 35;
>> +				step_needed    :  1, /* Use arch step handler */
>> +				__reserved_1   : 34;
> 
> This needs documenting properly, as I really have no idea how userspace is
> going to use it sensibley, especially as you silently overwrite it in some
> cases below.

I too had thought to put it under include/linux/perf_event.h : struct 
perf_event. But, see hw_break_module_init() which does not have knowledge of 
this structure, and we need to have some way so that none-perf kernel module 
implementation can tell that it needs default arch step handler.

Do you see any alternative?

-- 
Regards
Pratyush

Re: [PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Peter Zijlstra]

On Wed, Jul 26, 2017 at 11:12:03AM +0530, Pratyush Anand wrote:
> I too had thought to put it under include/linux/perf_event.h : struct
> perf_event. But, see hw_break_module_init() which does not have knowledge of
> this structure, and we need to have some way so that none-perf kernel module
> implementation can tell that it needs default arch step handler.
> 
> Do you see any alternative?

Fix the hw_breakpoint interface?

[PATCH V2 1/4] hw_breakpoint: Add step_needed event attribute

[Peter Zijlstra]

On Wed, Jul 26, 2017 at 11:12:03AM +0530, Pratyush Anand wrote:
> I too had thought to put it under include/linux/perf_event.h : struct
> perf_event. But, see hw_break_module_init() which does not have knowledge of
> this structure, and we need to have some way so that none-perf kernel module
> implementation can tell that it needs default arch step handler.
> 
> Do you see any alternative?

Fix the hw_breakpoint interface?