[PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Lecopzer Chen]

These series implement Perf NMI funxtionality and depends on
Pseudo NMI [1] which has been upstreamed.

In arm64 with GICv3, Pseudo NMI was implemented for NMI-like interruts.
That can be extended to Perf NMI which is the prerequisite for hard-lockup
detector which had already a standard interface inside Linux.

Thus the first step we need to implement perf NMI interface and make sure
it works fine.

Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the link [2]
did.

[1] https://lkml.org/lkml/2019/1/31/535
[2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq


Lecopzer Chen (3):
  arm_pmu: Add support for perf NMI interrupts registration
  arm64: perf: Support NMI context for perf event ISR
  arm64: Kconfig: Add support for the Perf NMI

 arch/arm64/Kconfig             | 10 +++++++
 arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
 drivers/perf/arm_pmu.c         | 51 ++++++++++++++++++++++++++++++----
 include/linux/perf/arm_pmu.h   |  6 ++++
 4 files changed, 88 insertions(+), 15 deletions(-)

-- 
2.25.1

[PATCH 1/3] arm_pmu: Add support for perf NMI interrupts registration

[Lecopzer Chen]

Register perf interrupts by request_nmi()/percpu_nmi() when both
ARM64_PSEUDO_NMI and ARM64_PSEUDO_NMI_PERF are enabled and nmi
cpufreature is active.

Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
---
 drivers/perf/arm_pmu.c       | 51 +++++++++++++++++++++++++++++++-----
 include/linux/perf/arm_pmu.h |  6 +++++
 2 files changed, 51 insertions(+), 6 deletions(-)

diff --git a/drivers/perf/arm_pmu.c b/drivers/perf/arm_pmu.c
index df352b334ea7..fa37b72d19e2 100644
--- a/drivers/perf/arm_pmu.c
+++ b/drivers/perf/arm_pmu.c
@@ -559,6 +559,48 @@ void armpmu_free_irq(int irq, int cpu)
 	per_cpu(cpu_irq, cpu) = 0;
 }
 
+static void armpmu_prepare_percpu_nmi_other(void *info)
+{
+	/*
+	 * We don't need to disable preemption since smp_call_function()
+	 * did this for us.
+	 */
+	prepare_percpu_nmi((uintptr_t) info);
+}
+
+static int _armpmu_request_irq(unsigned int irq, irq_handler_t handler,
+			       unsigned long flags, int cpu)
+{
+	if (armpmu_support_nmi())
+		return request_nmi(irq, handler, flags, "arm-pmu",
+				   per_cpu_ptr(&cpu_armpmu, cpu));
+	return request_irq(irq, handler, flags, "arm-pmu",
+			   per_cpu_ptr(&cpu_armpmu, cpu));
+}
+
+static int _armpmu_request_percpu_irq(unsigned int irq, irq_handler_t handler)
+{
+	if (armpmu_support_nmi()) {
+		int err;
+
+		err = request_percpu_nmi(irq, handler, "arm-pmu",
+					 &cpu_armpmu);
+		if (err)
+			return err;
+
+		preempt_disable();
+		err = prepare_percpu_nmi(irq);
+		if (err) {
+			return err;
+			preempt_enable();
+		}
+		smp_call_function(armpmu_prepare_percpu_nmi_other,
+				  (void *)(uintptr_t) irq, true);
+		preempt_enable();
+	}
+	return request_percpu_irq(irq, handler, "arm-pmu",
+				  &cpu_armpmu);
+}
+
 int armpmu_request_irq(int irq, int cpu)
 {
 	int err = 0;
@@ -582,12 +624,9 @@ int armpmu_request_irq(int irq, int cpu)
 			    IRQF_NO_THREAD;
 
 		irq_set_status_flags(irq, IRQ_NOAUTOEN);
-		err = request_irq(irq, handler, irq_flags, "arm-pmu",
-				  per_cpu_ptr(&cpu_armpmu, cpu));
-	} else if (armpmu_count_irq_users(irq) == 0) {
-		err = request_percpu_irq(irq, handler, "arm-pmu",
-					 &cpu_armpmu);
-	}
+		err = _armpmu_request_irq(irq, handler, irq_flags, cpu);
+	} else if (armpmu_count_irq_users(irq) == 0)
+		err = _armpmu_request_percpu_irq(irq, handler);
 
 	if (err)
 		goto err_out;
diff --git a/include/linux/perf/arm_pmu.h b/include/linux/perf/arm_pmu.h
index 5b616dde9a4c..5b878b5a22aa 100644
--- a/include/linux/perf/arm_pmu.h
+++ b/include/linux/perf/arm_pmu.h
@@ -160,6 +160,12 @@ int arm_pmu_acpi_probe(armpmu_init_fn init_fn);
 static inline int arm_pmu_acpi_probe(armpmu_init_fn init_fn) { return 0; }
 #endif
 
+static inline bool armpmu_support_nmi(void)
+{
+	return IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI_PERF) &&
+	       system_uses_irq_prio_masking();
+}
+
 /* Internal functions only for core arm_pmu code */
 struct arm_pmu *armpmu_alloc(void);
 struct arm_pmu *armpmu_alloc_atomic(void);
-- 
2.25.1

[PATCH 2/3] arm64: perf: Support NMI context for perf event ISR

[Lecopzer Chen]

Perf ISR doesn't support for NMI context, thus add some necessary
condition-if to handle NMI context:

- We should not hold pmu_lock since it may have already been acquired
before NMI triggered.
- irq_work should not run at NMI context.

Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
---
 arch/arm64/kernel/perf_event.c | 36 +++++++++++++++++++++++++---------
 1 file changed, 27 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c
index 4d7879484cec..94b404509f02 100644
--- a/arch/arm64/kernel/perf_event.c
+++ b/arch/arm64/kernel/perf_event.c
@@ -313,6 +313,23 @@ static inline bool armv8pmu_event_is_chained(struct perf_event *event)
 	       (idx != ARMV8_IDX_CYCLE_COUNTER);
 }
 
+/*
+ * NMI Perf interrupts may be triggered during kernel holding
+ * same lock.
+ * Avoid acquiring lock again in NMI context.
+ */
+#define armv8pmu_lock(lock, flags)				\
+	do {							\
+		if (!in_nmi())					\
+			raw_spin_lock_irqsave(lock, flags);	\
+	} while (0)
+
+#define armv8pmu_unlock(lock, flags)				\
+	do {							\
+		if (!in_nmi())					\
+			raw_spin_unlock_irqrestore(lock, flags);\
+	} while (0)
+
 /*
  * ARMv8 low level PMU access
  */
@@ -589,7 +606,7 @@ static void armv8pmu_enable_event(struct perf_event *event)
 	 * Enable counter and interrupt, and set the counter to count
 	 * the event that we're interested in.
 	 */
-	raw_spin_lock_irqsave(&events->pmu_lock, flags);
+	armv8pmu_lock(&events->pmu_lock, flags);
 
 	/*
 	 * Disable counter
@@ -611,7 +628,7 @@ static void armv8pmu_enable_event(struct perf_event *event)
 	 */
 	armv8pmu_enable_event_counter(event);
 
-	raw_spin_unlock_irqrestore(&events->pmu_lock, flags);
+	armv8pmu_unlock(&events->pmu_lock, flags);
 }
 
 static void armv8pmu_disable_event(struct perf_event *event)
@@ -623,7 +640,7 @@ static void armv8pmu_disable_event(struct perf_event *event)
 	/*
 	 * Disable counter and interrupt
 	 */
-	raw_spin_lock_irqsave(&events->pmu_lock, flags);
+	armv8pmu_lock(&events->pmu_lock, flags);
 
 	/*
 	 * Disable counter
@@ -635,7 +652,7 @@ static void armv8pmu_disable_event(struct perf_event *event)
 	 */
 	armv8pmu_disable_event_irq(event);
 
-	raw_spin_unlock_irqrestore(&events->pmu_lock, flags);
+	armv8pmu_unlock(&events->pmu_lock, flags);
 }
 
 static void armv8pmu_start(struct arm_pmu *cpu_pmu)
@@ -643,10 +660,10 @@ static void armv8pmu_start(struct arm_pmu *cpu_pmu)
 	unsigned long flags;
 	struct pmu_hw_events *events = this_cpu_ptr(cpu_pmu->hw_events);
 
-	raw_spin_lock_irqsave(&events->pmu_lock, flags);
+	armv8pmu_lock(&events->pmu_lock, flags);
 	/* Enable all counters */
 	armv8pmu_pmcr_write(armv8pmu_pmcr_read() | ARMV8_PMU_PMCR_E);
-	raw_spin_unlock_irqrestore(&events->pmu_lock, flags);
+	armv8pmu_unlock(&events->pmu_lock, flags);
 }
 
 static void armv8pmu_stop(struct arm_pmu *cpu_pmu)
@@ -654,10 +671,10 @@ static void armv8pmu_stop(struct arm_pmu *cpu_pmu)
 	unsigned long flags;
 	struct pmu_hw_events *events = this_cpu_ptr(cpu_pmu->hw_events);
 
-	raw_spin_lock_irqsave(&events->pmu_lock, flags);
+	armv8pmu_lock(&events->pmu_lock, flags);
 	/* Disable all counters */
 	armv8pmu_pmcr_write(armv8pmu_pmcr_read() & ~ARMV8_PMU_PMCR_E);
-	raw_spin_unlock_irqrestore(&events->pmu_lock, flags);
+	armv8pmu_unlock(&events->pmu_lock, flags);
 }
 
 static irqreturn_t armv8pmu_handle_irq(struct arm_pmu *cpu_pmu)
@@ -722,7 +739,8 @@ static irqreturn_t armv8pmu_handle_irq(struct arm_pmu *cpu_pmu)
 	 * platforms that can have the PMU interrupts raised as an NMI, this
 	 * will not work.
 	 */
-	irq_work_run();
+	if (!armpmu_support_nmi())
+		irq_work_run();
 
 	return IRQ_HANDLED;
 }
-- 
2.25.1

[PATCH 3/3] arm64: Kconfig: Add support for the Perf NMI

[Lecopzer Chen]

This is an extending function for Pseudo NMI that registering
Perf events interrupts as NMI.

It's helpful for sampling irq-off context when using perf.

Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
---
 arch/arm64/Kconfig | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 40fb05d96c60..f89c169771a0 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1679,6 +1679,16 @@ config ARM64_PSEUDO_NMI
 	  If unsure, say N
 
 if ARM64_PSEUDO_NMI
+config ARM64_PSEUDO_NMI_PERF
+	bool "Register Perf interrupts as Pseudo NMI"
+	depends on HW_PERF_EVENTS
+	depends on ARM_PMU
+	select HAVE_PERF_EVENTS_NMI
+	help
+	  This registers Perf interrupts to NMI when Pseudo NMI is active.
+	  This option is helpful when you need to debug any context disabled
+	  irq and get more inforamtion.
+
+	  If unsure, say N
+
 config ARM64_DEBUG_PRIORITY_MASKING
 	bool "Debug interrupt priority masking"
 	help
-- 
2.25.1

Re: [PATCH 1/3] arm_pmu: Add support for perf NMI interrupts registration

[Lecopzer Chen]

There was some mistakes when merging this patch.
The free nmi part is not present :(

The following part will be added in V2 next weekend.

diff --git a/drivers/perf/arm_pmu.c b/drivers/perf/arm_pmu.c
index fa37b72d19e2..aa9ed09e5303 100644
--- a/drivers/perf/arm_pmu.c
+++ b/drivers/perf/arm_pmu.c
@@ -544,6 +544,38 @@ static int armpmu_count_irq_users(const int irq)
        return count;
 }

+static void armpmu_teardown_percpu_nmi_other(void* info)
+{
+       /*
+        * We don't need to disable preemption since smp_call_function()
+        * did this for us.
+        */
+       teardown_percpu_nmi((uintptr_t) info);
+}
+
+static void _armpmu_free_irq(unsigned int irq, void *dev_id)
+{
+       if (armpmu_support_nmi())
+               free_nmi(irq, dev_id);
+       else
+               free_irq(irq, dev_id);
+}
+
+static void _armpmu_free_percpu_irq(unsigned int irq, void __percpu *dev_id)
+{
+       if (armpmu_support_nmi()) {
+               preempt_disable();
+               teardown_percpu_nmi(irq);
+               smp_call_function(armpmu_teardown_percpu_nmi_other,
+                                 (void *)(uintptr_t) irq, true);
+               preempt_enable();
+
+               free_percpu_nmi(irq, dev_id);
+       }
+       else
+               free_percpu_irq(irq, dev_id);
+}
+
 void armpmu_free_irq(int irq, int cpu)
 {
        if (per_cpu(cpu_irq, cpu) == 0)
@@ -552,9 +584,9 @@ void armpmu_free_irq(int irq, int cpu)
                return;

        if (!irq_is_percpu_devid(irq))
-               free_irq(irq, per_cpu_ptr(&cpu_armpmu, cpu));
+               _armpmu_free_irq(irq, per_cpu_ptr(&cpu_armpmu, cpu));
        else if (armpmu_count_irq_users(irq) == 1)
-               free_percpu_irq(irq, &cpu_armpmu);
+               _armpmu_free_percpu_irq(irq, &cpu_armpmu);

        per_cpu(cpu_irq, cpu) = 0;
 }




Thanks,
Lecopzer

Lecopzer Chen <lecopzer@gmail.com> 於 2020年5月16日 週六 下午8:50寫道：
>
> Register perf interrupts by request_nmi()/percpu_nmi() when both
> ARM64_PSEUDO_NMI and ARM64_PSEUDO_NMI_PERF are enabled and nmi
> cpufreature is active.
>
> Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
> ---
>  drivers/perf/arm_pmu.c       | 51 +++++++++++++++++++++++++++++++-----
>  include/linux/perf/arm_pmu.h |  6 +++++
>  2 files changed, 51 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/perf/arm_pmu.c b/drivers/perf/arm_pmu.c
> index df352b334ea7..fa37b72d19e2 100644
> --- a/drivers/perf/arm_pmu.c
> +++ b/drivers/perf/arm_pmu.c
> @@ -559,6 +559,48 @@ void armpmu_free_irq(int irq, int cpu)
>         per_cpu(cpu_irq, cpu) = 0;
>  }
>
> +static void armpmu_prepare_percpu_nmi_other(void *info)
> +{
> +       /*
> +        * We don't need to disable preemption since smp_call_function()
> +        * did this for us.
> +        */
> +       prepare_percpu_nmi((uintptr_t) info);
> +}
> +
> +static int _armpmu_request_irq(unsigned int irq, irq_handler_t handler,
> +                              unsigned long flags, int cpu)
> +{
> +       if (armpmu_support_nmi())
> +               return request_nmi(irq, handler, flags, "arm-pmu",
> +                                  per_cpu_ptr(&cpu_armpmu, cpu));
> +       return request_irq(irq, handler, flags, "arm-pmu",
> +                          per_cpu_ptr(&cpu_armpmu, cpu));
> +}
> +
> +static int _armpmu_request_percpu_irq(unsigned int irq, irq_handler_t handler)
> +{
> +       if (armpmu_support_nmi()) {
> +               int err;
> +
> +               err = request_percpu_nmi(irq, handler, "arm-pmu",
> +                                        &cpu_armpmu);
> +               if (err)
> +                       return err;
> +
> +               preempt_disable();
> +               err = prepare_percpu_nmi(irq);
> +               if (err) {
> +                       return err;
> +                       preempt_enable();
> +               }
> +               smp_call_function(armpmu_prepare_percpu_nmi_other,
> +                                 (void *)(uintptr_t) irq, true);
> +               preempt_enable();
> +       }
> +       return request_percpu_irq(irq, handler, "arm-pmu",
> +                                 &cpu_armpmu);
> +}
> +
>  int armpmu_request_irq(int irq, int cpu)
>  {
>         int err = 0;
> @@ -582,12 +624,9 @@ int armpmu_request_irq(int irq, int cpu)
>                             IRQF_NO_THREAD;
>
>                 irq_set_status_flags(irq, IRQ_NOAUTOEN);
> -               err = request_irq(irq, handler, irq_flags, "arm-pmu",
> -                                 per_cpu_ptr(&cpu_armpmu, cpu));
> -       } else if (armpmu_count_irq_users(irq) == 0) {
> -               err = request_percpu_irq(irq, handler, "arm-pmu",
> -                                        &cpu_armpmu);
> -       }
> +               err = _armpmu_request_irq(irq, handler, irq_flags, cpu);
> +       } else if (armpmu_count_irq_users(irq) == 0)
> +               err = _armpmu_request_percpu_irq(irq, handler);
>
>         if (err)
>                 goto err_out;
> diff --git a/include/linux/perf/arm_pmu.h b/include/linux/perf/arm_pmu.h
> index 5b616dde9a4c..5b878b5a22aa 100644
> --- a/include/linux/perf/arm_pmu.h
> +++ b/include/linux/perf/arm_pmu.h
> @@ -160,6 +160,12 @@ int arm_pmu_acpi_probe(armpmu_init_fn init_fn);
>  static inline int arm_pmu_acpi_probe(armpmu_init_fn init_fn) { return 0; }
>  #endif
>
> +static inline bool armpmu_support_nmi(void)
> +{
> +       return IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI_PERF) &&
> +              system_uses_irq_prio_masking();
> +}
> +
>  /* Internal functions only for core arm_pmu code */
>  struct arm_pmu *armpmu_alloc(void);
>  struct arm_pmu *armpmu_alloc_atomic(void);
> --
> 2.25.1
>

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Sumit Garg]

+ Julien

Hi Lecopzer,

On Sat, 16 May 2020 at 18:20, Lecopzer Chen <lecopzer@gmail.com> wrote:
>
> These series implement Perf NMI funxtionality and depends on
> Pseudo NMI [1] which has been upstreamed.
>
> In arm64 with GICv3, Pseudo NMI was implemented for NMI-like interruts.
> That can be extended to Perf NMI which is the prerequisite for hard-lockup
> detector which had already a standard interface inside Linux.
>
> Thus the first step we need to implement perf NMI interface and make sure
> it works fine.
>

This is something that is already implemented via Julien's patch-set
[1]. Its v4 has been floating since July, 2019 and I couldn't find any
major blocking comments but not sure why things haven't progressed
further.

Maybe Julien or Arm maintainers can provide updates on existing
patch-set [1] and how we should proceed further with this interesting
feature.

And regarding hard-lockup detection, I have been able to enable it
based on perf NMI events using Julien's perf patch-set [1]. Have a
look at the patch here [2].

[1] https://patchwork.kernel.org/cover/11047407/
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html

-Sumit

> Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the link [2]
> did.
>
> [1] https://lkml.org/lkml/2019/1/31/535
> [2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq
>
>
> Lecopzer Chen (3):
>   arm_pmu: Add support for perf NMI interrupts registration
>   arm64: perf: Support NMI context for perf event ISR
>   arm64: Kconfig: Add support for the Perf NMI
>
>  arch/arm64/Kconfig             | 10 +++++++
>  arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
>  drivers/perf/arm_pmu.c         | 51 ++++++++++++++++++++++++++++++----
>  include/linux/perf/arm_pmu.h   |  6 ++++
>  4 files changed, 88 insertions(+), 15 deletions(-)
>
> --
> 2.25.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Lecopzer Chen]

HI Sumit,

Thanks for your information.

I've already implemented IPI (same as you did [1], little difference
in detail), hardlockup detector and perf in last year(2019) for
debuggability.
And now we tend to upstream to reduce kernel maintaining effort.
I'm glad if someone in ARM can do this work :)

Hi Julien,

Does any Arm maintainers can proceed this action?
This is really useful in debugging.
Thank you!!



[1] https://lkml.org/lkml/2020/4/24/328


Lecopzer

Sumit Garg <sumit.garg@linaro.org> 於 2020年5月18日 週一 下午1:46寫道：
>
> + Julien
>
> Hi Lecopzer,
>
> On Sat, 16 May 2020 at 18:20, Lecopzer Chen <lecopzer@gmail.com> wrote:
> >
> > These series implement Perf NMI funxtionality and depends on
> > Pseudo NMI [1] which has been upstreamed.
> >
> > In arm64 with GICv3, Pseudo NMI was implemented for NMI-like interruts.
> > That can be extended to Perf NMI which is the prerequisite for hard-lockup
> > detector which had already a standard interface inside Linux.
> >
> > Thus the first step we need to implement perf NMI interface and make sure
> > it works fine.
> >
>
> This is something that is already implemented via Julien's patch-set
> [1]. Its v4 has been floating since July, 2019 and I couldn't find any
> major blocking comments but not sure why things haven't progressed
> further.
>
> Maybe Julien or Arm maintainers can provide updates on existing
> patch-set [1] and how we should proceed further with this interesting
> feature.
>
> And regarding hard-lockup detection, I have been able to enable it
> based on perf NMI events using Julien's perf patch-set [1]. Have a
> look at the patch here [2].
>
> [1] https://patchwork.kernel.org/cover/11047407/
> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html
>
> -Sumit
>
> > Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the link [2]
> > did.
> >
> > [1] https://lkml.org/lkml/2019/1/31/535
> > [2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq
> >
> >
> > Lecopzer Chen (3):
> >   arm_pmu: Add support for perf NMI interrupts registration
> >   arm64: perf: Support NMI context for perf event ISR
> >   arm64: Kconfig: Add support for the Perf NMI
> >
> >  arch/arm64/Kconfig             | 10 +++++++
> >  arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
> >  drivers/perf/arm_pmu.c         | 51 ++++++++++++++++++++++++++++++----
> >  include/linux/perf/arm_pmu.h   |  6 ++++
> >  4 files changed, 88 insertions(+), 15 deletions(-)
> >
> > --
> > 2.25.1
> >
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Mark Rutland]

Hi all,

On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
> HI Sumit,
> 
> Thanks for your information.
> 
> I've already implemented IPI (same as you did [1], little difference
> in detail), hardlockup detector and perf in last year(2019) for
> debuggability.
> And now we tend to upstream to reduce kernel maintaining effort.
> I'm glad if someone in ARM can do this work :)
> 
> Hi Julien,
> 
> Does any Arm maintainers can proceed this action?

Alexandru (Cc'd) has been rebasing and reworking Julien's patches, which
is my preferred approach.

I understand that's not quite ready for posting since he's investigating
some of the nastier subtleties (e.g. mutual exclusion with the NMI), but
maybe we can put the work-in-progress patches somewhere in the mean
time.

Alexandru, do you have an idea of what needs to be done, and/or when you
expect you could post that?

Thanks,
Mark.

> This is really useful in debugging.
> Thank you!!
> 
> 
> 
> [1] https://lkml.org/lkml/2020/4/24/328
> 
> 
> Lecopzer
> 
> Sumit Garg <sumit.garg@linaro.org> 於 2020年5月18日 週一 下午1:46寫道：
> >
> > + Julien
> >
> > Hi Lecopzer,
> >
> > On Sat, 16 May 2020 at 18:20, Lecopzer Chen <lecopzer@gmail.com> wrote:
> > >
> > > These series implement Perf NMI funxtionality and depends on
> > > Pseudo NMI [1] which has been upstreamed.
> > >
> > > In arm64 with GICv3, Pseudo NMI was implemented for NMI-like interruts.
> > > That can be extended to Perf NMI which is the prerequisite for hard-lockup
> > > detector which had already a standard interface inside Linux.
> > >
> > > Thus the first step we need to implement perf NMI interface and make sure
> > > it works fine.
> > >
> >
> > This is something that is already implemented via Julien's patch-set
> > [1]. Its v4 has been floating since July, 2019 and I couldn't find any
> > major blocking comments but not sure why things haven't progressed
> > further.
> >
> > Maybe Julien or Arm maintainers can provide updates on existing
> > patch-set [1] and how we should proceed further with this interesting
> > feature.
> >
> > And regarding hard-lockup detection, I have been able to enable it
> > based on perf NMI events using Julien's perf patch-set [1]. Have a
> > look at the patch here [2].
> >
> > [1] https://patchwork.kernel.org/cover/11047407/
> > [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html
> >
> > -Sumit
> >
> > > Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the link [2]
> > > did.
> > >
> > > [1] https://lkml.org/lkml/2019/1/31/535
> > > [2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq
> > >
> > >
> > > Lecopzer Chen (3):
> > >   arm_pmu: Add support for perf NMI interrupts registration
> > >   arm64: perf: Support NMI context for perf event ISR
> > >   arm64: Kconfig: Add support for the Perf NMI
> > >
> > >  arch/arm64/Kconfig             | 10 +++++++
> > >  arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
> > >  drivers/perf/arm_pmu.c         | 51 ++++++++++++++++++++++++++++++----
> > >  include/linux/perf/arm_pmu.h   |  6 ++++
> > >  4 files changed, 88 insertions(+), 15 deletions(-)
> > >
> > > --
> > > 2.25.1
> > >
> > >
> > > _______________________________________________
> > > linux-arm-kernel mailing list
> > > linux-arm-kernel@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Alexandru Elisei]

Hi,

On 5/18/20 11:45 AM, Mark Rutland wrote:
> Hi all,
>
> On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
>> HI Sumit,
>>
>> Thanks for your information.
>>
>> I've already implemented IPI (same as you did [1], little difference
>> in detail), hardlockup detector and perf in last year(2019) for
>> debuggability.
>> And now we tend to upstream to reduce kernel maintaining effort.
>> I'm glad if someone in ARM can do this work :)
>>
>> Hi Julien,
>>
>> Does any Arm maintainers can proceed this action?
> Alexandru (Cc'd) has been rebasing and reworking Julien's patches, which
> is my preferred approach.
>
> I understand that's not quite ready for posting since he's investigating
> some of the nastier subtleties (e.g. mutual exclusion with the NMI), but
> maybe we can put the work-in-progress patches somewhere in the mean
> time.
>
> Alexandru, do you have an idea of what needs to be done, and/or when you
> expect you could post that?

I'm currently working on rebasing the patches on top of 5.7-rc5, when I have
something usable I'll post a link (should be a couple of days). After that I will
address the review comments, and I plan to do a thorough testing because I'm not
100% confident that some of the assumptions around the locks that were removed are
correct. My guess is this will take a few weeks.

Thanks,
Alex
>
> Thanks,
> Mark.
>
>> This is really useful in debugging.
>> Thank you!!
>>
>>
>>
>> [1] https://lkml.org/lkml/2020/4/24/328
>>
>>
>> Lecopzer
>>
>> Sumit Garg <sumit.garg@linaro.org> 於 2020年5月18日 週一 下午1:46寫道：
>>> + Julien
>>>
>>> Hi Lecopzer,
>>>
>>> On Sat, 16 May 2020 at 18:20, Lecopzer Chen <lecopzer@gmail.com> wrote:
>>>> These series implement Perf NMI funxtionality and depends on
>>>> Pseudo NMI [1] which has been upstreamed.
>>>>
>>>> In arm64 with GICv3, Pseudo NMI was implemented for NMI-like interruts.
>>>> That can be extended to Perf NMI which is the prerequisite for hard-lockup
>>>> detector which had already a standard interface inside Linux.
>>>>
>>>> Thus the first step we need to implement perf NMI interface and make sure
>>>> it works fine.
>>>>
>>> This is something that is already implemented via Julien's patch-set
>>> [1]. Its v4 has been floating since July, 2019 and I couldn't find any
>>> major blocking comments but not sure why things haven't progressed
>>> further.
>>>
>>> Maybe Julien or Arm maintainers can provide updates on existing
>>> patch-set [1] and how we should proceed further with this interesting
>>> feature.
>>>
>>> And regarding hard-lockup detection, I have been able to enable it
>>> based on perf NMI events using Julien's perf patch-set [1]. Have a
>>> look at the patch here [2].
>>>
>>> [1] https://patchwork.kernel.org/cover/11047407/
>>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html
>>>
>>> -Sumit
>>>
>>>> Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the link [2]
>>>> did.
>>>>
>>>> [1] https://lkml.org/lkml/2019/1/31/535
>>>> [2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq
>>>>
>>>>
>>>> Lecopzer Chen (3):
>>>>   arm_pmu: Add support for perf NMI interrupts registration
>>>>   arm64: perf: Support NMI context for perf event ISR
>>>>   arm64: Kconfig: Add support for the Perf NMI
>>>>
>>>>  arch/arm64/Kconfig             | 10 +++++++
>>>>  arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
>>>>  drivers/perf/arm_pmu.c         | 51 ++++++++++++++++++++++++++++++----
>>>>  include/linux/perf/arm_pmu.h   |  6 ++++
>>>>  4 files changed, 88 insertions(+), 15 deletions(-)
>>>>
>>>> --
>>>> 2.25.1
>>>>
>>>>
>>>> _______________________________________________
>>>> linux-arm-kernel mailing list
>>>> linux-arm-kernel@lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Sumit Garg]

On Mon, 18 May 2020 at 16:47, Alexandru Elisei <alexandru.elisei@arm.com> wrote:
>
> Hi,
>
> On 5/18/20 11:45 AM, Mark Rutland wrote:
> > Hi all,
> >
> > On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
> >> HI Sumit,
> >>
> >> Thanks for your information.
> >>
> >> I've already implemented IPI (same as you did [1], little difference
> >> in detail), hardlockup detector and perf in last year(2019) for
> >> debuggability.
> >> And now we tend to upstream to reduce kernel maintaining effort.
> >> I'm glad if someone in ARM can do this work :)
> >>
> >> Hi Julien,
> >>
> >> Does any Arm maintainers can proceed this action?
> > Alexandru (Cc'd) has been rebasing and reworking Julien's patches, which
> > is my preferred approach.
> >
> > I understand that's not quite ready for posting since he's investigating
> > some of the nastier subtleties (e.g. mutual exclusion with the NMI), but
> > maybe we can put the work-in-progress patches somewhere in the mean
> > time.
> >
> > Alexandru, do you have an idea of what needs to be done, and/or when you
> > expect you could post that?
>
> I'm currently working on rebasing the patches on top of 5.7-rc5, when I have
> something usable I'll post a link (should be a couple of days). After that I will
> address the review comments, and I plan to do a thorough testing because I'm not
> 100% confident that some of the assumptions around the locks that were removed are
> correct. My guess is this will take a few weeks.
>

Thanks Mark, Alex for the status updates on perf NMI feature.

Alex,

As the hard-lockup detection patch [1] has a dependency on perf NMI
patch-set, I will rebase and test hard-lockup detector when you have
got a working tree. But due to the dependency, I think patch [1]
should be accepted along with perf NMI patch-set. So would you be open
to include this patch as part of your series?

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html

-Sumit

> Thanks,
> Alex
> >
> > Thanks,
> > Mark.
> >
> >> This is really useful in debugging.
> >> Thank you!!
> >>
> >>
> >>
> >> [1] https://lkml.org/lkml/2020/4/24/328
> >>
> >>
> >> Lecopzer
> >>
> >> Sumit Garg <sumit.garg@linaro.org> 於 2020年5月18日 週一 下午1:46寫道：
> >>> + Julien
> >>>
> >>> Hi Lecopzer,
> >>>
> >>> On Sat, 16 May 2020 at 18:20, Lecopzer Chen <lecopzer@gmail.com> wrote:
> >>>> These series implement Perf NMI funxtionality and depends on
> >>>> Pseudo NMI [1] which has been upstreamed.
> >>>>
> >>>> In arm64 with GICv3, Pseudo NMI was implemented for NMI-like interruts.
> >>>> That can be extended to Perf NMI which is the prerequisite for hard-lockup
> >>>> detector which had already a standard interface inside Linux.
> >>>>
> >>>> Thus the first step we need to implement perf NMI interface and make sure
> >>>> it works fine.
> >>>>
> >>> This is something that is already implemented via Julien's patch-set
> >>> [1]. Its v4 has been floating since July, 2019 and I couldn't find any
> >>> major blocking comments but not sure why things haven't progressed
> >>> further.
> >>>
> >>> Maybe Julien or Arm maintainers can provide updates on existing
> >>> patch-set [1] and how we should proceed further with this interesting
> >>> feature.
> >>>
> >>> And regarding hard-lockup detection, I have been able to enable it
> >>> based on perf NMI events using Julien's perf patch-set [1]. Have a
> >>> look at the patch here [2].
> >>>
> >>> [1] https://patchwork.kernel.org/cover/11047407/
> >>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html
> >>>
> >>> -Sumit
> >>>
> >>>> Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the link [2]
> >>>> did.
> >>>>
> >>>> [1] https://lkml.org/lkml/2019/1/31/535
> >>>> [2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq
> >>>>
> >>>>
> >>>> Lecopzer Chen (3):
> >>>>   arm_pmu: Add support for perf NMI interrupts registration
> >>>>   arm64: perf: Support NMI context for perf event ISR
> >>>>   arm64: Kconfig: Add support for the Perf NMI
> >>>>
> >>>>  arch/arm64/Kconfig             | 10 +++++++
> >>>>  arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
> >>>>  drivers/perf/arm_pmu.c         | 51 ++++++++++++++++++++++++++++++----
> >>>>  include/linux/perf/arm_pmu.h   |  6 ++++
> >>>>  4 files changed, 88 insertions(+), 15 deletions(-)
> >>>>
> >>>> --
> >>>> 2.25.1
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> linux-arm-kernel mailing list
> >>>> linux-arm-kernel@lists.infradead.org
> >>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Mark Rutland]

On Mon, May 18, 2020 at 07:39:23PM +0530, Sumit Garg wrote:
> On Mon, 18 May 2020 at 16:47, Alexandru Elisei <alexandru.elisei@arm.com> wrote:
> > On 5/18/20 11:45 AM, Mark Rutland wrote:
> > > On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
> > >> HI Sumit,
> > >>
> > >> Thanks for your information.
> > >>
> > >> I've already implemented IPI (same as you did [1], little difference
> > >> in detail), hardlockup detector and perf in last year(2019) for
> > >> debuggability.
> > >> And now we tend to upstream to reduce kernel maintaining effort.
> > >> I'm glad if someone in ARM can do this work :)
> > >>
> > >> Hi Julien,
> > >>
> > >> Does any Arm maintainers can proceed this action?
> > > Alexandru (Cc'd) has been rebasing and reworking Julien's patches, which
> > > is my preferred approach.
> > >
> > > I understand that's not quite ready for posting since he's investigating
> > > some of the nastier subtleties (e.g. mutual exclusion with the NMI), but
> > > maybe we can put the work-in-progress patches somewhere in the mean
> > > time.
> > >
> > > Alexandru, do you have an idea of what needs to be done, and/or when you
> > > expect you could post that?
> >
> > I'm currently working on rebasing the patches on top of 5.7-rc5, when I have
> > something usable I'll post a link (should be a couple of days). After that I will
> > address the review comments, and I plan to do a thorough testing because I'm not
> > 100% confident that some of the assumptions around the locks that were removed are
> > correct. My guess is this will take a few weeks.
> >
> 
> Thanks Mark, Alex for the status updates on perf NMI feature.
> 
> Alex,
> 
> As the hard-lockup detection patch [1] has a dependency on perf NMI
> patch-set, I will rebase and test hard-lockup detector when you have
> got a working tree. But due to the dependency, I think patch [1]
> should be accepted along with perf NMI patch-set. So would you be open
> to include this patch as part of your series?
> 
> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html

While it depends on the perf NMI bits, I don't think it makes sense to
tie that into the series given it's trying to achieve something very
different.

I think that should be reposted separately once the perf NMI bits are in
shape.

Thanks,
Mark.

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Sumit Garg]

On Mon, 18 May 2020 at 19:49, Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Mon, May 18, 2020 at 07:39:23PM +0530, Sumit Garg wrote:
> > On Mon, 18 May 2020 at 16:47, Alexandru Elisei <alexandru.elisei@arm.com> wrote:
> > > On 5/18/20 11:45 AM, Mark Rutland wrote:
> > > > On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
> > > >> HI Sumit,
> > > >>
> > > >> Thanks for your information.
> > > >>
> > > >> I've already implemented IPI (same as you did [1], little difference
> > > >> in detail), hardlockup detector and perf in last year(2019) for
> > > >> debuggability.
> > > >> And now we tend to upstream to reduce kernel maintaining effort.
> > > >> I'm glad if someone in ARM can do this work :)
> > > >>
> > > >> Hi Julien,
> > > >>
> > > >> Does any Arm maintainers can proceed this action?
> > > > Alexandru (Cc'd) has been rebasing and reworking Julien's patches, which
> > > > is my preferred approach.
> > > >
> > > > I understand that's not quite ready for posting since he's investigating
> > > > some of the nastier subtleties (e.g. mutual exclusion with the NMI), but
> > > > maybe we can put the work-in-progress patches somewhere in the mean
> > > > time.
> > > >
> > > > Alexandru, do you have an idea of what needs to be done, and/or when you
> > > > expect you could post that?
> > >
> > > I'm currently working on rebasing the patches on top of 5.7-rc5, when I have
> > > something usable I'll post a link (should be a couple of days). After that I will
> > > address the review comments, and I plan to do a thorough testing because I'm not
> > > 100% confident that some of the assumptions around the locks that were removed are
> > > correct. My guess is this will take a few weeks.
> > >
> >
> > Thanks Mark, Alex for the status updates on perf NMI feature.
> >
> > Alex,
> >
> > As the hard-lockup detection patch [1] has a dependency on perf NMI
> > patch-set, I will rebase and test hard-lockup detector when you have
> > got a working tree. But due to the dependency, I think patch [1]
> > should be accepted along with perf NMI patch-set. So would you be open
> > to include this patch as part of your series?
> >
> > [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html
>
> While it depends on the perf NMI bits, I don't think it makes sense to
> tie that into the series given it's trying to achieve something very
> different.
>
> I think that should be reposted separately once the perf NMI bits are in
> shape.

Okay, fair enough. Will keep it as a separate patch then.

-Sumit

>
> Thanks,
> Mark.

RE: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Song Bao Hua]

> 
> On 5/18/20 11:45 AM, Mark Rutland wrote:
> > Hi all,
> >
> > On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
> >> HI Sumit,
> >>
> >> Thanks for your information.
> >>
> >> I've already implemented IPI (same as you did [1], little difference
> >> in detail), hardlockup detector and perf in last year(2019) for
> >> debuggability.
> >> And now we tend to upstream to reduce kernel maintaining effort.
> >> I'm glad if someone in ARM can do this work :)
> >>
> >> Hi Julien,
> >>
> >> Does any Arm maintainers can proceed this action?
> > Alexandru (Cc'd) has been rebasing and reworking Julien's patches,
> > which is my preferred approach.
> >
> > I understand that's not quite ready for posting since he's
> > investigating some of the nastier subtleties (e.g. mutual exclusion
> > with the NMI), but maybe we can put the work-in-progress patches
> > somewhere in the mean time.
> >
> > Alexandru, do you have an idea of what needs to be done, and/or when
> > you expect you could post that?
> 
> I'm currently working on rebasing the patches on top of 5.7-rc5, when I have
> something usable I'll post a link (should be a couple of days). After that I will
> address the review comments, and I plan to do a thorough testing because I'm
> not 100% confident that some of the assumptions around the locks that were
> removed are correct. My guess is this will take a few weeks.

+1
I would be awesome if perf NMI patches could be re-activated. Right now, it seems it is hard to
do "perf annotate" on a kernel function with local_irq disabled.

func()
{
local_irq_save();
.....
....
local_irq_restore();
return;
}

Perf will report all cycles are used by the last moment of the func().

Thanks,
Barry

> 
> Thanks,
> Alex
> >
> > Thanks,
> > Mark.
> >
> >> This is really useful in debugging.
> >> Thank you!!
> >>
> >>
> >>
> >> [1] https://lkml.org/lkml/2020/4/24/328
> >>
> >>
> >> Lecopzer
> >>
> >> Sumit Garg <sumit.garg@linaro.org> 於 2020年5月18日 週一 下午
> 1:46寫道：
> >>> + Julien
> >>>
> >>> Hi Lecopzer,
> >>>
> >>> On Sat, 16 May 2020 at 18:20, Lecopzer Chen <lecopzer@gmail.com>
> wrote:
> >>>> These series implement Perf NMI funxtionality and depends on Pseudo
> >>>> NMI [1] which has been upstreamed.
> >>>>
> >>>> In arm64 with GICv3, Pseudo NMI was implemented for NMI-like
> interruts.
> >>>> That can be extended to Perf NMI which is the prerequisite for
> >>>> hard-lockup detector which had already a standard interface inside Linux.
> >>>>
> >>>> Thus the first step we need to implement perf NMI interface and
> >>>> make sure it works fine.
> >>>>
> >>> This is something that is already implemented via Julien's patch-set
> >>> [1]. Its v4 has been floating since July, 2019 and I couldn't find
> >>> any major blocking comments but not sure why things haven't
> >>> progressed further.
> >>>
> >>> Maybe Julien or Arm maintainers can provide updates on existing
> >>> patch-set [1] and how we should proceed further with this
> >>> interesting feature.
> >>>
> >>> And regarding hard-lockup detection, I have been able to enable it
> >>> based on perf NMI events using Julien's perf patch-set [1]. Have a
> >>> look at the patch here [2].
> >>>
> >>> [1] https://patchwork.kernel.org/cover/11047407/
> >>> [2]
> >>> http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/73222
> >>> 7.html
> >>>
> >>> -Sumit
> >>>
> >>>> Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the
> >>>> link [2] did.
> >>>>
> >>>> [1] https://lkml.org/lkml/2019/1/31/535
> >>>> [2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq
> >>>>
> >>>>
> >>>> Lecopzer Chen (3):
> >>>>   arm_pmu: Add support for perf NMI interrupts registration
> >>>>   arm64: perf: Support NMI context for perf event ISR
> >>>>   arm64: Kconfig: Add support for the Perf NMI
> >>>>
> >>>>  arch/arm64/Kconfig             | 10 +++++++
> >>>>  arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
> >>>>  drivers/perf/arm_pmu.c         | 51
> ++++++++++++++++++++++++++++++----
> >>>>  include/linux/perf/arm_pmu.h   |  6 ++++
> >>>>  4 files changed, 88 insertions(+), 15 deletions(-)
> >>>>
> >>>> --
> >>>> 2.25.1

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Alexandru Elisei]

Hi,

On 5/18/20 12:17 PM, Alexandru Elisei wrote:
> Hi,
>
> On 5/18/20 11:45 AM, Mark Rutland wrote:
>> Hi all,
>>
>> On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
>>> HI Sumit,
>>>
>>> Thanks for your information.
>>>
>>> I've already implemented IPI (same as you did [1], little difference
>>> in detail), hardlockup detector and perf in last year(2019) for
>>> debuggability.
>>> And now we tend to upstream to reduce kernel maintaining effort.
>>> I'm glad if someone in ARM can do this work :)
>>>
>>> Hi Julien,
>>>
>>> Does any Arm maintainers can proceed this action?
>> Alexandru (Cc'd) has been rebasing and reworking Julien's patches, which
>> is my preferred approach.
>>
>> I understand that's not quite ready for posting since he's investigating
>> some of the nastier subtleties (e.g. mutual exclusion with the NMI), but
>> maybe we can put the work-in-progress patches somewhere in the mean
>> time.
>>
>> Alexandru, do you have an idea of what needs to be done, and/or when you
>> expect you could post that?
> I'm currently working on rebasing the patches on top of 5.7-rc5, when I have
> something usable I'll post a link (should be a couple of days). After that I will
> address the review comments, and I plan to do a thorough testing because I'm not
> 100% confident that some of the assumptions around the locks that were removed are
> correct. My guess is this will take a few weeks.

Pushed a WIP branch on linux-arm.org [1]:

git clone -b WIP-pmu-nmi git://linux-arm.org/linux-ae

Practically untested, I only did perf record on a defconfig kernel running on the
model.

[1] http://www.linux-arm.org/git?p=linux-ae.git;a=shortlog;h=refs/heads/WIP-pmu-nmi

Thanks,
Alex
>
> Thanks,
> Alex
>> Thanks,
>> Mark.
>>
>>> This is really useful in debugging.
>>> Thank you!!
>>>
>>>
>>>
>>> [1] https://lkml.org/lkml/2020/4/24/328
>>>
>>>
>>> Lecopzer
>>>
>>> Sumit Garg <sumit.garg@linaro.org> 於 2020年5月18日 週一 下午1:46寫道：
>>>> + Julien
>>>>
>>>> Hi Lecopzer,
>>>>
>>>> On Sat, 16 May 2020 at 18:20, Lecopzer Chen <lecopzer@gmail.com> wrote:
>>>>> These series implement Perf NMI funxtionality and depends on
>>>>> Pseudo NMI [1] which has been upstreamed.
>>>>>
>>>>> In arm64 with GICv3, Pseudo NMI was implemented for NMI-like interruts.
>>>>> That can be extended to Perf NMI which is the prerequisite for hard-lockup
>>>>> detector which had already a standard interface inside Linux.
>>>>>
>>>>> Thus the first step we need to implement perf NMI interface and make sure
>>>>> it works fine.
>>>>>
>>>> This is something that is already implemented via Julien's patch-set
>>>> [1]. Its v4 has been floating since July, 2019 and I couldn't find any
>>>> major blocking comments but not sure why things haven't progressed
>>>> further.
>>>>
>>>> Maybe Julien or Arm maintainers can provide updates on existing
>>>> patch-set [1] and how we should proceed further with this interesting
>>>> feature.
>>>>
>>>> And regarding hard-lockup detection, I have been able to enable it
>>>> based on perf NMI events using Julien's perf patch-set [1]. Have a
>>>> look at the patch here [2].
>>>>
>>>> [1] https://patchwork.kernel.org/cover/11047407/
>>>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/732227.html
>>>>
>>>> -Sumit
>>>>
>>>>> Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the link [2]
>>>>> did.
>>>>>
>>>>> [1] https://lkml.org/lkml/2019/1/31/535
>>>>> [2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq
>>>>>
>>>>>
>>>>> Lecopzer Chen (3):
>>>>>   arm_pmu: Add support for perf NMI interrupts registration
>>>>>   arm64: perf: Support NMI context for perf event ISR
>>>>>   arm64: Kconfig: Add support for the Perf NMI
>>>>>
>>>>>  arch/arm64/Kconfig             | 10 +++++++
>>>>>  arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
>>>>>  drivers/perf/arm_pmu.c         | 51 ++++++++++++++++++++++++++++++----
>>>>>  include/linux/perf/arm_pmu.h   |  6 ++++
>>>>>  4 files changed, 88 insertions(+), 15 deletions(-)
>>>>>
>>>>> --
>>>>> 2.25.1
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> linux-arm-kernel mailing list
>>>>> linux-arm-kernel@lists.infradead.org
>>>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

RE: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Song Bao Hua (Barry Song)]

> -----Original Message-----
> From: linux-arm-kernel [mailto:linux-arm-kernel-bounces@lists.infradead.org]
> On Behalf Of Alexandru Elisei
> Sent: Wednesday, May 20, 2020 10:31 PM> 
> Hi,
> 
> On 5/18/20 12:17 PM, Alexandru Elisei wrote:
> > Hi,
> >
> > On 5/18/20 11:45 AM, Mark Rutland wrote:
> >> Hi all,
> >>
> >> On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
> >>> HI Sumit,
> >>>
> >>> Thanks for your information.
> >>>
> >>> I've already implemented IPI (same as you did [1], little difference
> >>> in detail), hardlockup detector and perf in last year(2019) for
> >>> debuggability.
> >>> And now we tend to upstream to reduce kernel maintaining effort.
> >>> I'm glad if someone in ARM can do this work :)
> >>>
> >>> Hi Julien,
> >>>
> >>> Does any Arm maintainers can proceed this action?
> >> Alexandru (Cc'd) has been rebasing and reworking Julien's patches,
> >> which is my preferred approach.
> >>
> >> I understand that's not quite ready for posting since he's
> >> investigating some of the nastier subtleties (e.g. mutual exclusion
> >> with the NMI), but maybe we can put the work-in-progress patches
> >> somewhere in the mean time.
> >>
> >> Alexandru, do you have an idea of what needs to be done, and/or when
> >> you expect you could post that?
> > I'm currently working on rebasing the patches on top of 5.7-rc5, when
> > I have something usable I'll post a link (should be a couple of days).
> > After that I will address the review comments, and I plan to do a
> > thorough testing because I'm not 100% confident that some of the
> > assumptions around the locks that were removed are correct. My guess is
> this will take a few weeks.
> 
> Pushed a WIP branch on linux-arm.org [1]:
> 
> git clone -b WIP-pmu-nmi git://linux-arm.org/linux-ae
> 
> Practically untested, I only did perf record on a defconfig kernel running on the
> model.
> 
> [1]
> http://www.linux-arm.org/git?p=linux-ae.git;a=shortlog;h=refs/heads/WIP-pm
> u-nmi

Fortunately, it does work. I used this tree to perf annotate arm_smmu_cmdq_issue_cmdlist() which
is completely disabling IRQ. Luckily, it reports correct data. Before that, it reported all time was spent by
the code which enabled IRQ .


Barry

> 
> Thanks,
> Alex
> >
> > Thanks,
> > Alex
> >> Thanks,
> >> Mark.
> >>
> >>> This is really useful in debugging.
> >>> Thank you!!
> >>>
> >>>
> >>>
> >>> [1] https://lkml.org/lkml/2020/4/24/328
> >>>
> >>>
> >>> Lecopzer
> >>>
> >>> Sumit Garg <sumit.garg@linaro.org> 於 2020年5月18日 週一 下午
> 1:46寫道：
> >>>> + Julien
> >>>>
> >>>> Hi Lecopzer,
> >>>>
> >>>> On Sat, 16 May 2020 at 18:20, Lecopzer Chen <lecopzer@gmail.com>
> wrote:
> >>>>> These series implement Perf NMI funxtionality and depends on
> >>>>> Pseudo NMI [1] which has been upstreamed.
> >>>>>
> >>>>> In arm64 with GICv3, Pseudo NMI was implemented for NMI-like
> interruts.
> >>>>> That can be extended to Perf NMI which is the prerequisite for
> >>>>> hard-lockup detector which had already a standard interface inside
> Linux.
> >>>>>
> >>>>> Thus the first step we need to implement perf NMI interface and
> >>>>> make sure it works fine.
> >>>>>
> >>>> This is something that is already implemented via Julien's
> >>>> patch-set [1]. Its v4 has been floating since July, 2019 and I
> >>>> couldn't find any major blocking comments but not sure why things
> >>>> haven't progressed further.
> >>>>
> >>>> Maybe Julien or Arm maintainers can provide updates on existing
> >>>> patch-set [1] and how we should proceed further with this
> >>>> interesting feature.
> >>>>
> >>>> And regarding hard-lockup detection, I have been able to enable it
> >>>> based on perf NMI events using Julien's perf patch-set [1]. Have a
> >>>> look at the patch here [2].
> >>>>
> >>>> [1] https://patchwork.kernel.org/cover/11047407/
> >>>> [2]
> >>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2020-May/7322
> >>>> 27.html
> >>>>
> >>>> -Sumit
> >>>>
> >>>>> Perf NMI has been test by dd if=/dev/urandom of=/dev/null like the
> >>>>> link [2] did.
> >>>>>
> >>>>> [1] https://lkml.org/lkml/2019/1/31/535
> >>>>> [2] https://www.linaro.org/blog/debugging-arm-kernels-using-nmifiq
> >>>>>
> >>>>>
> >>>>> Lecopzer Chen (3):
> >>>>>   arm_pmu: Add support for perf NMI interrupts registration
> >>>>>   arm64: perf: Support NMI context for perf event ISR
> >>>>>   arm64: Kconfig: Add support for the Perf NMI
> >>>>>
> >>>>>  arch/arm64/Kconfig             | 10 +++++++
> >>>>>  arch/arm64/kernel/perf_event.c | 36 ++++++++++++++++++------
> >>>>>  drivers/perf/arm_pmu.c         | 51
> ++++++++++++++++++++++++++++++----
> >>>>>  include/linux/perf/arm_pmu.h   |  6 ++++
> >>>>>  4 files changed, 88 insertions(+), 15 deletions(-)
> >>>>>
> >>>>> --
> >>>>> 2.25.1

Re: [PATCH 0/3] arm64: perf: Add support for Perf NMI interrupts

[Alexandru Elisei]

Hi,

On 5/21/20 4:00 AM, Song Bao Hua (Barry Song) wrote:

>
>> -----Original Message-----
>> From: linux-arm-kernel [mailto:linux-arm-kernel-bounces@lists.infradead.org]
>> On Behalf Of Alexandru Elisei
>> Sent: Wednesday, May 20, 2020 10:31 PM> 
>> Hi,
>>
>> On 5/18/20 12:17 PM, Alexandru Elisei wrote:
>>> Hi,
>>>
>>> On 5/18/20 11:45 AM, Mark Rutland wrote:
>>>> Hi all,
>>>>
>>>> On Mon, May 18, 2020 at 02:26:00PM +0800, Lecopzer Chen wrote:
>>>>> HI Sumit,
>>>>>
>>>>> Thanks for your information.
>>>>>
>>>>> I've already implemented IPI (same as you did [1], little difference
>>>>> in detail), hardlockup detector and perf in last year(2019) for
>>>>> debuggability.
>>>>> And now we tend to upstream to reduce kernel maintaining effort.
>>>>> I'm glad if someone in ARM can do this work :)
>>>>>
>>>>> Hi Julien,
>>>>>
>>>>> Does any Arm maintainers can proceed this action?
>>>> Alexandru (Cc'd) has been rebasing and reworking Julien's patches,
>>>> which is my preferred approach.
>>>>
>>>> I understand that's not quite ready for posting since he's
>>>> investigating some of the nastier subtleties (e.g. mutual exclusion
>>>> with the NMI), but maybe we can put the work-in-progress patches
>>>> somewhere in the mean time.
>>>>
>>>> Alexandru, do you have an idea of what needs to be done, and/or when
>>>> you expect you could post that?
>>> I'm currently working on rebasing the patches on top of 5.7-rc5, when
>>> I have something usable I'll post a link (should be a couple of days).
>>> After that I will address the review comments, and I plan to do a
>>> thorough testing because I'm not 100% confident that some of the
>>> assumptions around the locks that were removed are correct. My guess is
>> this will take a few weeks.
>>
>> Pushed a WIP branch on linux-arm.org [1]:
>>
>> git clone -b WIP-pmu-nmi git://linux-arm.org/linux-ae
>>
>> Practically untested, I only did perf record on a defconfig kernel running on the
>> model.
>>
>> [1]
>> http://www.linux-arm.org/git?p=linux-ae.git;a=shortlog;h=refs/heads/WIP-pm
>> u-nmi
> Fortunately, it does work. I used this tree to perf annotate arm_smmu_cmdq_issue_cmdlist() which
> is completely disabling IRQ. Luckily, it reports correct data. Before that, it reported all time was spent by
> the code which enabled IRQ .

That's good news that it works for you, thanks for letting me know.

Alex