[PATCH v8 00/26] arm64: provide pseudo NMI with GICv3

[PATCH v8 00/26] arm64: provide pseudo NMI with GICv3

[Julien Thierry]

Hi,

This series is a continuation of the work started by Daniel [1]. The goal
is to use GICv3 interrupt priorities to simulate an NMI.

The patches depend on the core API for NMIs patches [2]. Both series can
be found on this branch:
git clone http://linux-arm.org/linux-jt.git -b v4.20-pseudo-nmi


To achieve this, set two priorities, one for standard interrupts and
another, higher priority, for NMIs. Whenever we want to disable interrupts,
we mask the standard priority instead so NMIs can still be raised. Some
corner cases though still require to actually mask all interrupts
effectively disabling the NMI.

Daniel Thompson ran some benchmarks [3] on a previous version showing a
small (<1%) performance drop when using interrupt priorities on Cortex-A53
and GIC-500.

Currently, only PPIs and SPIs can be set as NMIs. IPIs being currently
hardcoded IRQ numbers, there isn't a generic interface to set SGIs as NMI
for now. LPIs being controlled by the ITS cannot be delivered as NMI.
When an NMI is active on a CPU, no other NMI can be triggered on the CPU.

Requirements to use this:
- Have GICv3
- SCR_EL3.FIQ is set to 1 when linux runs or have single security state
- Select Kernel Feature -> Support for NMI-like interrupts
- Provide "enable_pseudo_nmi" on the kernel command line


* Patch 1 fixes an existing issue with current NMI contexts in arm64
* Patches 2 and 3 are just a bit of cleanup
* Patch 4 introduces a CPU feature to check if priority masking should be
  used
* Patches 5 and 6 add the support for priority masking in GICv3 driver
* Patches 7 to 13 add the support for priority masking the arch/arm64
  code
* Patches 14 and 15 allow us to apply alternatives earlier in the boot
  process
* Patches 16 to 18 starts the PMR masking on cpu startup and provides
  primitives for arm64 GICv3 driver to perform priority masking
* Patches 19 to 22 Add support for pseudo-NMIs in GICv3 driver
* Patches 23 to 25 Add support for receiving NMIs in arch/arm64
* Patch 26 adds the build config and command line option to enable
  pseudo-NMIs


Changes since v7[4]:
* Rebase on v5.0-rc1
* Move default arch_nmi_enter/exit definition to fix build issues
* Override flags to preserve accross EFI services calls


[1] http://www.spinics.net/lists/arm-kernel/msg525077.html
[2] https://lkml.org/lkml/2018/11/12/2113
[3] https://lkml.org/lkml/2018/7/20/803
[4] https://www.spinics.net/lists/arm-kernel/msg695086.html

Cheers,

Julien

-->

Daniel Thompson (1):
  arm64: alternative: Apply alternatives early in boot process

Julien Thierry (25):
  arm64: Fix HCR.TGE status for NMI contexts
  arm64: Remove unused daif related functions/macros
  arm64: cpufeature: Set SYSREG_GIC_CPUIF as a boot system feature
  arm64: cpufeature: Add cpufeature for IRQ priority masking
  arm/arm64: gic-v3: Add PMR and RPR accessors
  irqchip/gic-v3: Switch to PMR masking before calling IRQ handler
  arm64: ptrace: Provide definitions for PMR values
  arm64: Make PMR part of task context
  arm64: Unmask PMR before going idle
  arm64: kvm: Unmask PMR before entering guest
  efi: Let architectures decide the flags that should be saved/restored
  arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking
  arm64: daifflags: Include PMR in daifflags restore operations
  arm64: alternative: Allow alternative status checking per cpufeature
  irqchip/gic-v3: Factor group0 detection into functions
  arm64: Switch to PMR masking when starting CPUs
  arm64: gic-v3: Implement arch support for priority masking
  irqchip/gic-v3: Detect if GIC can support pseudo-NMIs
  irqchip/gic-v3: Handle pseudo-NMIs
  irqchip/gic: Add functions to access irq priorities
  irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI
  arm64: Handle serror in NMI context
  arm64: Skip preemption when exiting an NMI
  arm64: Skip irqflags tracing for NMI in IRQs disabled context
  arm64: Enable the support of pseudo-NMIs

 Documentation/admin-guide/kernel-parameters.txt |   6 +
 Documentation/arm64/booting.txt                 |   5 +
 arch/arm/include/asm/arch_gicv3.h               |  33 ++++
 arch/arm64/Kconfig                              |  14 ++
 arch/arm64/include/asm/alternative.h            |   4 +-
 arch/arm64/include/asm/arch_gicv3.h             |  32 +++
 arch/arm64/include/asm/assembler.h              |  10 +-
 arch/arm64/include/asm/cpucaps.h                |   3 +-
 arch/arm64/include/asm/cpufeature.h             |  10 +
 arch/arm64/include/asm/daifflags.h              |  41 ++--
 arch/arm64/include/asm/efi.h                    |  11 ++
 arch/arm64/include/asm/hardirq.h                |  28 +++
 arch/arm64/include/asm/irqflags.h               | 123 +++++++++---
 arch/arm64/include/asm/kvm_host.h               |  12 ++
 arch/arm64/include/asm/processor.h              |   3 +
 arch/arm64/include/asm/ptrace.h                 |  26 ++-
 arch/arm64/kernel/alternative.c                 |  60 +++++-
 arch/arm64/kernel/asm-offsets.c                 |   1 +
 arch/arm64/kernel/cpufeature.c                  |  42 +++-
 arch/arm64/kernel/entry.S                       |  43 ++++
 arch/arm64/kernel/irq.c                         |   3 +
 arch/arm64/kernel/process.c                     |  51 +++++
 arch/arm64/kernel/smp.c                         |  33 ++++
 arch/arm64/kernel/traps.c                       |   8 +-
 arch/arm64/kvm/hyp/switch.c                     |  16 ++
 arch/arm64/mm/proc.S                            |  11 --
 drivers/firmware/efi/runtime-wrappers.c         |  17 +-
 drivers/irqchip/irq-gic-common.c                |  10 +
 drivers/irqchip/irq-gic-common.h                |   2 +
 drivers/irqchip/irq-gic-v3.c                    | 252 +++++++++++++++++++++---
 include/linux/efi.h                             |   5 +-
 include/linux/hardirq.h                         |   7 +
 32 files changed, 807 insertions(+), 115 deletions(-)

--
1.9.1

[PATCH v8 00/26] arm64: provide pseudo NMI with GICv3

[Julien Thierry]

Hi,

This series is a continuation of the work started by Daniel [1]. The goal
is to use GICv3 interrupt priorities to simulate an NMI.

The patches depend on the core API for NMIs patches [2]. Both series can
be found on this branch:
git clone http://linux-arm.org/linux-jt.git -b v4.20-pseudo-nmi


To achieve this, set two priorities, one for standard interrupts and
another, higher priority, for NMIs. Whenever we want to disable interrupts,
we mask the standard priority instead so NMIs can still be raised. Some
corner cases though still require to actually mask all interrupts
effectively disabling the NMI.

Daniel Thompson ran some benchmarks [3] on a previous version showing a
small (<1%) performance drop when using interrupt priorities on Cortex-A53
and GIC-500.

Currently, only PPIs and SPIs can be set as NMIs. IPIs being currently
hardcoded IRQ numbers, there isn't a generic interface to set SGIs as NMI
for now. LPIs being controlled by the ITS cannot be delivered as NMI.
When an NMI is active on a CPU, no other NMI can be triggered on the CPU.

Requirements to use this:
- Have GICv3
- SCR_EL3.FIQ is set to 1 when linux runs or have single security state
- Select Kernel Feature -> Support for NMI-like interrupts
- Provide "enable_pseudo_nmi" on the kernel command line


* Patch 1 fixes an existing issue with current NMI contexts in arm64
* Patches 2 and 3 are just a bit of cleanup
* Patch 4 introduces a CPU feature to check if priority masking should be
  used
* Patches 5 and 6 add the support for priority masking in GICv3 driver
* Patches 7 to 13 add the support for priority masking the arch/arm64
  code
* Patches 14 and 15 allow us to apply alternatives earlier in the boot
  process
* Patches 16 to 18 starts the PMR masking on cpu startup and provides
  primitives for arm64 GICv3 driver to perform priority masking
* Patches 19 to 22 Add support for pseudo-NMIs in GICv3 driver
* Patches 23 to 25 Add support for receiving NMIs in arch/arm64
* Patch 26 adds the build config and command line option to enable
  pseudo-NMIs


Changes since v7[4]:
* Rebase on v5.0-rc1
* Move default arch_nmi_enter/exit definition to fix build issues
* Override flags to preserve accross EFI services calls


[1] http://www.spinics.net/lists/arm-kernel/msg525077.html
[2] https://lkml.org/lkml/2018/11/12/2113
[3] https://lkml.org/lkml/2018/7/20/803
[4] https://www.spinics.net/lists/arm-kernel/msg695086.html

Cheers,

Julien

-->

Daniel Thompson (1):
  arm64: alternative: Apply alternatives early in boot process

Julien Thierry (25):
  arm64: Fix HCR.TGE status for NMI contexts
  arm64: Remove unused daif related functions/macros
  arm64: cpufeature: Set SYSREG_GIC_CPUIF as a boot system feature
  arm64: cpufeature: Add cpufeature for IRQ priority masking
  arm/arm64: gic-v3: Add PMR and RPR accessors
  irqchip/gic-v3: Switch to PMR masking before calling IRQ handler
  arm64: ptrace: Provide definitions for PMR values
  arm64: Make PMR part of task context
  arm64: Unmask PMR before going idle
  arm64: kvm: Unmask PMR before entering guest
  efi: Let architectures decide the flags that should be saved/restored
  arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking
  arm64: daifflags: Include PMR in daifflags restore operations
  arm64: alternative: Allow alternative status checking per cpufeature
  irqchip/gic-v3: Factor group0 detection into functions
  arm64: Switch to PMR masking when starting CPUs
  arm64: gic-v3: Implement arch support for priority masking
  irqchip/gic-v3: Detect if GIC can support pseudo-NMIs
  irqchip/gic-v3: Handle pseudo-NMIs
  irqchip/gic: Add functions to access irq priorities
  irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI
  arm64: Handle serror in NMI context
  arm64: Skip preemption when exiting an NMI
  arm64: Skip irqflags tracing for NMI in IRQs disabled context
  arm64: Enable the support of pseudo-NMIs

 Documentation/admin-guide/kernel-parameters.txt |   6 +
 Documentation/arm64/booting.txt                 |   5 +
 arch/arm/include/asm/arch_gicv3.h               |  33 ++++
 arch/arm64/Kconfig                              |  14 ++
 arch/arm64/include/asm/alternative.h            |   4 +-
 arch/arm64/include/asm/arch_gicv3.h             |  32 +++
 arch/arm64/include/asm/assembler.h              |  10 +-
 arch/arm64/include/asm/cpucaps.h                |   3 +-
 arch/arm64/include/asm/cpufeature.h             |  10 +
 arch/arm64/include/asm/daifflags.h              |  41 ++--
 arch/arm64/include/asm/efi.h                    |  11 ++
 arch/arm64/include/asm/hardirq.h                |  28 +++
 arch/arm64/include/asm/irqflags.h               | 123 +++++++++---
 arch/arm64/include/asm/kvm_host.h               |  12 ++
 arch/arm64/include/asm/processor.h              |   3 +
 arch/arm64/include/asm/ptrace.h                 |  26 ++-
 arch/arm64/kernel/alternative.c                 |  60 +++++-
 arch/arm64/kernel/asm-offsets.c                 |   1 +
 arch/arm64/kernel/cpufeature.c                  |  42 +++-
 arch/arm64/kernel/entry.S                       |  43 ++++
 arch/arm64/kernel/irq.c                         |   3 +
 arch/arm64/kernel/process.c                     |  51 +++++
 arch/arm64/kernel/smp.c                         |  33 ++++
 arch/arm64/kernel/traps.c                       |   8 +-
 arch/arm64/kvm/hyp/switch.c                     |  16 ++
 arch/arm64/mm/proc.S                            |  11 --
 drivers/firmware/efi/runtime-wrappers.c         |  17 +-
 drivers/irqchip/irq-gic-common.c                |  10 +
 drivers/irqchip/irq-gic-common.h                |   2 +
 drivers/irqchip/irq-gic-v3.c                    | 252 +++++++++++++++++++++---
 include/linux/efi.h                             |   5 +-
 include/linux/hardirq.h                         |   7 +
 32 files changed, 807 insertions(+), 115 deletions(-)

--
1.9.1

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Julien Thierry]

When using VHE, the host needs to clear HCR_EL2.TGE bit in order
to interract with guest TLBs, switching from EL2&0 translation regime
to EL1&0.

However, some non-maskable asynchronous event could happen while TGE is
cleared like SDEI. Because of this address translation operations
relying on EL2&0 translation regime could fail (tlb invalidation,
userspace access, ...).

Fix this by properly setting HCR_EL2.TGE when entering NMI context and
clear it if necessary when returning to the interrupted context.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: linux-arch@vger.kernel.org
Cc: stable@vger.kernel.org
---
 arch/arm64/include/asm/hardirq.h | 28 ++++++++++++++++++++++++++++
 arch/arm64/kernel/irq.c          |  3 +++
 include/linux/hardirq.h          |  7 +++++++
 3 files changed, 38 insertions(+)

diff --git a/arch/arm64/include/asm/hardirq.h b/arch/arm64/include/asm/hardirq.h
index 1473fc2..94b7481 100644
--- a/arch/arm64/include/asm/hardirq.h
+++ b/arch/arm64/include/asm/hardirq.h
@@ -19,6 +19,7 @@
 #include <linux/cache.h>
 #include <linux/threads.h>
 #include <asm/irq.h>
+#include <asm/kvm_arm.h>
 
 #define NR_IPI	7
 
@@ -37,6 +38,33 @@
 
 #define __ARCH_IRQ_EXIT_IRQS_DISABLED	1
 
+struct nmi_ctx {
+	u64 hcr;
+};
+
+DECLARE_PER_CPU(struct nmi_ctx, nmi_contexts);
+
+#define arch_nmi_enter()							\
+	do {									\
+		if (is_kernel_in_hyp_mode()) {					\
+			struct nmi_ctx *nmi_ctx = this_cpu_ptr(&nmi_contexts);	\
+			nmi_ctx->hcr = read_sysreg(hcr_el2);			\
+			if (!(nmi_ctx->hcr & HCR_TGE)) {			\
+				write_sysreg(nmi_ctx->hcr | HCR_TGE, hcr_el2);	\
+				isb();						\
+			}							\
+		}								\
+	} while (0)
+
+#define arch_nmi_exit()								\
+	do {									\
+		if (is_kernel_in_hyp_mode()) {					\
+			struct nmi_ctx *nmi_ctx = this_cpu_ptr(&nmi_contexts);	\
+			if (!(nmi_ctx->hcr & HCR_TGE))				\
+				write_sysreg(nmi_ctx->hcr, hcr_el2);		\
+		}								\
+	} while (0)
+
 static inline void ack_bad_irq(unsigned int irq)
 {
 	extern unsigned long irq_err_count;
diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c
index 780a12f..92fa817 100644
--- a/arch/arm64/kernel/irq.c
+++ b/arch/arm64/kernel/irq.c
@@ -33,6 +33,9 @@
 
 unsigned long irq_err_count;
 
+/* Only access this in an NMI enter/exit */
+DEFINE_PER_CPU(struct nmi_ctx, nmi_contexts);
+
 DEFINE_PER_CPU(unsigned long *, irq_stack_ptr);
 
 int arch_show_interrupts(struct seq_file *p, int prec)
diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h
index 0fbbcdf..da0af63 100644
--- a/include/linux/hardirq.h
+++ b/include/linux/hardirq.h
@@ -60,8 +60,14 @@ static inline void rcu_nmi_exit(void)
  */
 extern void irq_exit(void);
 
+#ifndef arch_nmi_enter
+#define arch_nmi_enter()	do { } while (0)
+#define arch_nmi_exit()		do { } while (0)
+#endif
+
 #define nmi_enter()						\
 	do {							\
+		arch_nmi_enter();				\
 		printk_nmi_enter();				\
 		lockdep_off();					\
 		ftrace_nmi_enter();				\
@@ -80,6 +86,7 @@ static inline void rcu_nmi_exit(void)
 		ftrace_nmi_exit();				\
 		lockdep_on();					\
 		printk_nmi_exit();				\
+		arch_nmi_exit();				\
 	} while (0)
 
 #endif /* LINUX_HARDIRQ_H */
-- 
1.9.1

[PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Julien Thierry]

When using VHE, the host needs to clear HCR_EL2.TGE bit in order
to interract with guest TLBs, switching from EL2&0 translation regime
to EL1&0.

However, some non-maskable asynchronous event could happen while TGE is
cleared like SDEI. Because of this address translation operations
relying on EL2&0 translation regime could fail (tlb invalidation,
userspace access, ...).

Fix this by properly setting HCR_EL2.TGE when entering NMI context and
clear it if necessary when returning to the interrupted context.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: linux-arch@vger.kernel.org
Cc: stable@vger.kernel.org
---
 arch/arm64/include/asm/hardirq.h | 28 ++++++++++++++++++++++++++++
 arch/arm64/kernel/irq.c          |  3 +++
 include/linux/hardirq.h          |  7 +++++++
 3 files changed, 38 insertions(+)

diff --git a/arch/arm64/include/asm/hardirq.h b/arch/arm64/include/asm/hardirq.h
index 1473fc2..94b7481 100644
--- a/arch/arm64/include/asm/hardirq.h
+++ b/arch/arm64/include/asm/hardirq.h
@@ -19,6 +19,7 @@
 #include <linux/cache.h>
 #include <linux/threads.h>
 #include <asm/irq.h>
+#include <asm/kvm_arm.h>
 
 #define NR_IPI	7
 
@@ -37,6 +38,33 @@
 
 #define __ARCH_IRQ_EXIT_IRQS_DISABLED	1
 
+struct nmi_ctx {
+	u64 hcr;
+};
+
+DECLARE_PER_CPU(struct nmi_ctx, nmi_contexts);
+
+#define arch_nmi_enter()							\
+	do {									\
+		if (is_kernel_in_hyp_mode()) {					\
+			struct nmi_ctx *nmi_ctx = this_cpu_ptr(&nmi_contexts);	\
+			nmi_ctx->hcr = read_sysreg(hcr_el2);			\
+			if (!(nmi_ctx->hcr & HCR_TGE)) {			\
+				write_sysreg(nmi_ctx->hcr | HCR_TGE, hcr_el2);	\
+				isb();						\
+			}							\
+		}								\
+	} while (0)
+
+#define arch_nmi_exit()								\
+	do {									\
+		if (is_kernel_in_hyp_mode()) {					\
+			struct nmi_ctx *nmi_ctx = this_cpu_ptr(&nmi_contexts);	\
+			if (!(nmi_ctx->hcr & HCR_TGE))				\
+				write_sysreg(nmi_ctx->hcr, hcr_el2);		\
+		}								\
+	} while (0)
+
 static inline void ack_bad_irq(unsigned int irq)
 {
 	extern unsigned long irq_err_count;
diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c
index 780a12f..92fa817 100644
--- a/arch/arm64/kernel/irq.c
+++ b/arch/arm64/kernel/irq.c
@@ -33,6 +33,9 @@
 
 unsigned long irq_err_count;
 
+/* Only access this in an NMI enter/exit */
+DEFINE_PER_CPU(struct nmi_ctx, nmi_contexts);
+
 DEFINE_PER_CPU(unsigned long *, irq_stack_ptr);
 
 int arch_show_interrupts(struct seq_file *p, int prec)
diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h
index 0fbbcdf..da0af63 100644
--- a/include/linux/hardirq.h
+++ b/include/linux/hardirq.h
@@ -60,8 +60,14 @@ static inline void rcu_nmi_exit(void)
  */
 extern void irq_exit(void);
 
+#ifndef arch_nmi_enter
+#define arch_nmi_enter()	do { } while (0)
+#define arch_nmi_exit()		do { } while (0)
+#endif
+
 #define nmi_enter()						\
 	do {							\
+		arch_nmi_enter();				\
 		printk_nmi_enter();				\
 		lockdep_off();					\
 		ftrace_nmi_enter();				\
@@ -80,6 +86,7 @@ static inline void rcu_nmi_exit(void)
 		ftrace_nmi_exit();				\
 		lockdep_on();					\
 		printk_nmi_exit();				\
+		arch_nmi_exit();				\
 	} while (0)
 
 #endif /* LINUX_HARDIRQ_H */
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 02/26] arm64: Remove unused daif related functions/macros

[Julien Thierry]

There are some helpers to modify PSR.[DAIF] bits that are not referenced
anywhere. The less these bits are available outside of local_irq_*
functions the better.

Get rid of those unused helpers.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
---
 arch/arm64/include/asm/assembler.h | 10 +---------
 arch/arm64/include/asm/daifflags.h | 10 ----------
 2 files changed, 1 insertion(+), 19 deletions(-)

diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 4feb611..7acf243 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -62,16 +62,8 @@
 	.endm
 
 /*
- * Enable and disable interrupts.
+ * Save/restore interrupts.
  */
-	.macro	disable_irq
-	msr	daifset, #2
-	.endm
-
-	.macro	enable_irq
-	msr	daifclr, #2
-	.endm
-
 	.macro	save_and_disable_irq, flags
 	mrs	\flags, daif
 	msr	daifset, #2
diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h
index 8d91f22..546bc39 100644
--- a/arch/arm64/include/asm/daifflags.h
+++ b/arch/arm64/include/asm/daifflags.h
@@ -43,16 +43,6 @@ static inline unsigned long local_daif_save(void)
 	return flags;
 }
 
-static inline void local_daif_unmask(void)
-{
-	trace_hardirqs_on();
-	asm volatile(
-		"msr	daifclr, #0xf		// local_daif_unmask"
-		:
-		:
-		: "memory");
-}
-
 static inline void local_daif_restore(unsigned long flags)
 {
 	if (!arch_irqs_disabled_flags(flags))
-- 
1.9.1

[PATCH v8 02/26] arm64: Remove unused daif related functions/macros

[Julien Thierry]

There are some helpers to modify PSR.[DAIF] bits that are not referenced
anywhere. The less these bits are available outside of local_irq_*
functions the better.

Get rid of those unused helpers.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
---
 arch/arm64/include/asm/assembler.h | 10 +---------
 arch/arm64/include/asm/daifflags.h | 10 ----------
 2 files changed, 1 insertion(+), 19 deletions(-)

diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 4feb611..7acf243 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -62,16 +62,8 @@
 	.endm
 
 /*
- * Enable and disable interrupts.
+ * Save/restore interrupts.
  */
-	.macro	disable_irq
-	msr	daifset, #2
-	.endm
-
-	.macro	enable_irq
-	msr	daifclr, #2
-	.endm
-
 	.macro	save_and_disable_irq, flags
 	mrs	\flags, daif
 	msr	daifset, #2
diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h
index 8d91f22..546bc39 100644
--- a/arch/arm64/include/asm/daifflags.h
+++ b/arch/arm64/include/asm/daifflags.h
@@ -43,16 +43,6 @@ static inline unsigned long local_daif_save(void)
 	return flags;
 }
 
-static inline void local_daif_unmask(void)
-{
-	trace_hardirqs_on();
-	asm volatile(
-		"msr	daifclr, #0xf		// local_daif_unmask"
-		:
-		:
-		: "memory");
-}
-
 static inline void local_daif_restore(unsigned long flags)
 {
 	if (!arch_irqs_disabled_flags(flags))
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 03/26] arm64: cpufeature: Set SYSREG_GIC_CPUIF as a boot system feature

[Julien Thierry]

It is not supported to have some CPUs using GICv3 sysreg CPU interface
while some others do not.

Once ICC_SRE_EL1.SRE is set on a CPU, the bit cannot be cleared. Since
matching this feature require setting ICC_SRE_EL1.SRE, it cannot be
turned off if found on a CPU.

Set the feature as STRICT_BOOT, if boot CPU has it, all other CPUs are
required to have it.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/cpufeature.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 4f27239..9542d01 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1202,7 +1202,7 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 	{
 		.desc = "GIC system register CPU interface",
 		.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
-		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
 		.matches = has_useable_gicv3_cpuif,
 		.sys_reg = SYS_ID_AA64PFR0_EL1,
 		.field_pos = ID_AA64PFR0_GIC_SHIFT,
-- 
1.9.1

[PATCH v8 03/26] arm64: cpufeature: Set SYSREG_GIC_CPUIF as a boot system feature

[Julien Thierry]

It is not supported to have some CPUs using GICv3 sysreg CPU interface
while some others do not.

Once ICC_SRE_EL1.SRE is set on a CPU, the bit cannot be cleared. Since
matching this feature require setting ICC_SRE_EL1.SRE, it cannot be
turned off if found on a CPU.

Set the feature as STRICT_BOOT, if boot CPU has it, all other CPUs are
required to have it.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/cpufeature.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 4f27239..9542d01 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1202,7 +1202,7 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 	{
 		.desc = "GIC system register CPU interface",
 		.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
-		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
 		.matches = has_useable_gicv3_cpuif,
 		.sys_reg = SYS_ID_AA64PFR0_EL1,
 		.field_pos = ID_AA64PFR0_GIC_SHIFT,
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 04/26] arm64: cpufeature: Add cpufeature for IRQ priority masking

[Julien Thierry]

Add a cpufeature indicating whether a cpu supports masking interrupts
by priority.

The feature will be properly enabled in a later patch.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/cpucaps.h    |  3 ++-
 arch/arm64/include/asm/cpufeature.h |  6 ++++++
 arch/arm64/kernel/cpufeature.c      | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 82e9099..f6a76e4 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -60,7 +60,8 @@
 #define ARM64_HAS_ADDRESS_AUTH_IMP_DEF		39
 #define ARM64_HAS_GENERIC_AUTH_ARCH		40
 #define ARM64_HAS_GENERIC_AUTH_IMP_DEF		41
+#define ARM64_HAS_IRQ_PRIO_MASKING		42
 
-#define ARM64_NCAPS				42
+#define ARM64_NCAPS				43
 
 #endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index dfcfba7..89c3f31 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -612,6 +612,12 @@ static inline bool system_supports_generic_auth(void)
 		 cpus_have_const_cap(ARM64_HAS_GENERIC_AUTH_IMP_DEF));
 }
 
+static inline bool system_uses_irq_prio_masking(void)
+{
+	return IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) &&
+	       cpus_have_const_cap(ARM64_HAS_IRQ_PRIO_MASKING);
+}
+
 #define ARM64_SSBD_UNKNOWN		-1
 #define ARM64_SSBD_FORCE_DISABLE	0
 #define ARM64_SSBD_KERNEL		1
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9542d01..c426302 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1198,6 +1198,14 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 }
 #endif /* CONFIG_ARM64_PTR_AUTH */
 
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry,
+				   int scope)
+{
+	return false;
+}
+#endif
+
 static const struct arm64_cpu_capabilities arm64_features[] = {
 	{
 		.desc = "GIC system register CPU interface",
@@ -1475,6 +1483,21 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 		.matches = has_cpuid_feature,
 	},
 #endif /* CONFIG_ARM64_PTR_AUTH */
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+	{
+		/*
+		 * Depends on having GICv3
+		 */
+		.desc = "IRQ priority masking",
+		.capability = ARM64_HAS_IRQ_PRIO_MASKING,
+		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
+		.matches = can_use_gic_priorities,
+		.sys_reg = SYS_ID_AA64PFR0_EL1,
+		.field_pos = ID_AA64PFR0_GIC_SHIFT,
+		.sign = FTR_UNSIGNED,
+		.min_field_value = 1,
+	},
+#endif
 	{},
 };
 
-- 
1.9.1

[PATCH v8 04/26] arm64: cpufeature: Add cpufeature for IRQ priority masking

[Julien Thierry]

Add a cpufeature indicating whether a cpu supports masking interrupts
by priority.

The feature will be properly enabled in a later patch.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/cpucaps.h    |  3 ++-
 arch/arm64/include/asm/cpufeature.h |  6 ++++++
 arch/arm64/kernel/cpufeature.c      | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 82e9099..f6a76e4 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -60,7 +60,8 @@
 #define ARM64_HAS_ADDRESS_AUTH_IMP_DEF		39
 #define ARM64_HAS_GENERIC_AUTH_ARCH		40
 #define ARM64_HAS_GENERIC_AUTH_IMP_DEF		41
+#define ARM64_HAS_IRQ_PRIO_MASKING		42
 
-#define ARM64_NCAPS				42
+#define ARM64_NCAPS				43
 
 #endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index dfcfba7..89c3f31 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -612,6 +612,12 @@ static inline bool system_supports_generic_auth(void)
 		 cpus_have_const_cap(ARM64_HAS_GENERIC_AUTH_IMP_DEF));
 }
 
+static inline bool system_uses_irq_prio_masking(void)
+{
+	return IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) &&
+	       cpus_have_const_cap(ARM64_HAS_IRQ_PRIO_MASKING);
+}
+
 #define ARM64_SSBD_UNKNOWN		-1
 #define ARM64_SSBD_FORCE_DISABLE	0
 #define ARM64_SSBD_KERNEL		1
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9542d01..c426302 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1198,6 +1198,14 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 }
 #endif /* CONFIG_ARM64_PTR_AUTH */
 
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry,
+				   int scope)
+{
+	return false;
+}
+#endif
+
 static const struct arm64_cpu_capabilities arm64_features[] = {
 	{
 		.desc = "GIC system register CPU interface",
@@ -1475,6 +1483,21 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 		.matches = has_cpuid_feature,
 	},
 #endif /* CONFIG_ARM64_PTR_AUTH */
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+	{
+		/*
+		 * Depends on having GICv3
+		 */
+		.desc = "IRQ priority masking",
+		.capability = ARM64_HAS_IRQ_PRIO_MASKING,
+		.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
+		.matches = can_use_gic_priorities,
+		.sys_reg = SYS_ID_AA64PFR0_EL1,
+		.field_pos = ID_AA64PFR0_GIC_SHIFT,
+		.sign = FTR_UNSIGNED,
+		.min_field_value = 1,
+	},
+#endif
 	{},
 };
 
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 05/26] arm/arm64: gic-v3: Add PMR and RPR accessors

[Julien Thierry]

Add helper functions to access system registers related to interrupt
priorities: PMR and RPR.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h   | 16 ++++++++++++++++
 arch/arm64/include/asm/arch_gicv3.h | 15 +++++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index 0bd5307..bef0b5d 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -34,6 +34,7 @@
 #define ICC_SRE				__ACCESS_CP15(c12, 0, c12, 5)
 #define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
 #define ICC_BPR1			__ACCESS_CP15(c12, 0, c12, 3)
+#define ICC_RPR				__ACCESS_CP15(c12, 0, c11, 3)
 
 #define __ICC_AP0Rx(x)			__ACCESS_CP15(c12, 0, c8, 4 | x)
 #define ICC_AP0R0			__ICC_AP0Rx(0)
@@ -245,6 +246,21 @@ static inline void gic_write_bpr1(u32 val)
 	write_sysreg(val, ICC_BPR1);
 }
 
+static inline u32 gic_read_pmr(void)
+{
+	return read_sysreg(ICC_PMR);
+}
+
+static inline void gic_write_pmr(u32 val)
+{
+	write_sysreg(val, ICC_PMR);
+}
+
+static inline u32 gic_read_rpr(void)
+{
+	return read_sysreg(ICC_RPR);
+}
+
 /*
  * Even in 32bit systems that use LPAE, there is no guarantee that the I/O
  * interface provides true 64bit atomic accesses, so using strd/ldrd doesn't
diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index e278f94..37193e2 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -114,6 +114,21 @@ static inline void gic_write_bpr1(u32 val)
 	write_sysreg_s(val, SYS_ICC_BPR1_EL1);
 }
 
+static inline u32 gic_read_pmr(void)
+{
+	return read_sysreg_s(SYS_ICC_PMR_EL1);
+}
+
+static inline void gic_write_pmr(u32 val)
+{
+	write_sysreg_s(val, SYS_ICC_PMR_EL1);
+}
+
+static inline u32 gic_read_rpr(void)
+{
+	return read_sysreg_s(SYS_ICC_RPR_EL1);
+}
+
 #define gic_read_typer(c)		readq_relaxed(c)
 #define gic_write_irouter(v, c)		writeq_relaxed(v, c)
 #define gic_read_lpir(c)		readq_relaxed(c)
-- 
1.9.1

[PATCH v8 05/26] arm/arm64: gic-v3: Add PMR and RPR accessors

[Julien Thierry]

Add helper functions to access system registers related to interrupt
priorities: PMR and RPR.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h   | 16 ++++++++++++++++
 arch/arm64/include/asm/arch_gicv3.h | 15 +++++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index 0bd5307..bef0b5d 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -34,6 +34,7 @@
 #define ICC_SRE				__ACCESS_CP15(c12, 0, c12, 5)
 #define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
 #define ICC_BPR1			__ACCESS_CP15(c12, 0, c12, 3)
+#define ICC_RPR				__ACCESS_CP15(c12, 0, c11, 3)
 
 #define __ICC_AP0Rx(x)			__ACCESS_CP15(c12, 0, c8, 4 | x)
 #define ICC_AP0R0			__ICC_AP0Rx(0)
@@ -245,6 +246,21 @@ static inline void gic_write_bpr1(u32 val)
 	write_sysreg(val, ICC_BPR1);
 }
 
+static inline u32 gic_read_pmr(void)
+{
+	return read_sysreg(ICC_PMR);
+}
+
+static inline void gic_write_pmr(u32 val)
+{
+	write_sysreg(val, ICC_PMR);
+}
+
+static inline u32 gic_read_rpr(void)
+{
+	return read_sysreg(ICC_RPR);
+}
+
 /*
  * Even in 32bit systems that use LPAE, there is no guarantee that the I/O
  * interface provides true 64bit atomic accesses, so using strd/ldrd doesn't
diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index e278f94..37193e2 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -114,6 +114,21 @@ static inline void gic_write_bpr1(u32 val)
 	write_sysreg_s(val, SYS_ICC_BPR1_EL1);
 }
 
+static inline u32 gic_read_pmr(void)
+{
+	return read_sysreg_s(SYS_ICC_PMR_EL1);
+}
+
+static inline void gic_write_pmr(u32 val)
+{
+	write_sysreg_s(val, SYS_ICC_PMR_EL1);
+}
+
+static inline u32 gic_read_rpr(void)
+{
+	return read_sysreg_s(SYS_ICC_RPR_EL1);
+}
+
 #define gic_read_typer(c)		readq_relaxed(c)
 #define gic_write_irouter(v, c)		writeq_relaxed(v, c)
 #define gic_read_lpir(c)		readq_relaxed(c)
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 06/26] irqchip/gic-v3: Switch to PMR masking before calling IRQ handler

[Julien Thierry]

Mask the IRQ priority through PMR and re-enable IRQs at CPU level,
allowing only higher priority interrupts to be received during interrupt
handling.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h   | 17 +++++++++++++++++
 arch/arm64/include/asm/arch_gicv3.h | 17 +++++++++++++++++
 drivers/irqchip/irq-gic-v3.c        |  5 +++++
 3 files changed, 39 insertions(+)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index bef0b5d..f6f485f 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -363,5 +363,22 @@ static inline void gits_write_vpendbaser(u64 val, void * __iomem addr)
 
 #define gits_read_vpendbaser(c)		__gic_readq_nonatomic(c)
 
+static inline bool gic_prio_masking_enabled(void)
+{
+	return false;
+}
+
+static inline void gic_pmr_mask_irqs(void)
+{
+	/* Should not get called. */
+	WARN_ON_ONCE(true);
+}
+
+static inline void gic_arch_enable_irqs(void)
+{
+	/* Should not get called. */
+	WARN_ON_ONCE(true);
+}
+
 #endif /* !__ASSEMBLY__ */
 #endif /* !__ASM_ARCH_GICV3_H */
diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index 37193e2..b5f8142 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -155,5 +155,22 @@ static inline u32 gic_read_rpr(void)
 #define gits_write_vpendbaser(v, c)	writeq_relaxed(v, c)
 #define gits_read_vpendbaser(c)		readq_relaxed(c)
 
+static inline bool gic_prio_masking_enabled(void)
+{
+	return system_uses_irq_prio_masking();
+}
+
+static inline void gic_pmr_mask_irqs(void)
+{
+	/* Should not get called yet. */
+	WARN_ON_ONCE(true);
+}
+
+static inline void gic_arch_enable_irqs(void)
+{
+	/* Should not get called yet. */
+	WARN_ON_ONCE(true);
+}
+
 #endif /* __ASSEMBLY__ */
 #endif /* __ASM_ARCH_GICV3_H */
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 0868a9d..8148a92 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -356,6 +356,11 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 
 	irqnr = gic_read_iar();
 
+	if (gic_prio_masking_enabled()) {
+		gic_pmr_mask_irqs();
+		gic_arch_enable_irqs();
+	}
+
 	if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) {
 		int err;
 
-- 
1.9.1

[PATCH v8 06/26] irqchip/gic-v3: Switch to PMR masking before calling IRQ handler

[Julien Thierry]

Mask the IRQ priority through PMR and re-enable IRQs at CPU level,
allowing only higher priority interrupts to be received during interrupt
handling.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/include/asm/arch_gicv3.h   | 17 +++++++++++++++++
 arch/arm64/include/asm/arch_gicv3.h | 17 +++++++++++++++++
 drivers/irqchip/irq-gic-v3.c        |  5 +++++
 3 files changed, 39 insertions(+)

diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h
index bef0b5d..f6f485f 100644
--- a/arch/arm/include/asm/arch_gicv3.h
+++ b/arch/arm/include/asm/arch_gicv3.h
@@ -363,5 +363,22 @@ static inline void gits_write_vpendbaser(u64 val, void * __iomem addr)
 
 #define gits_read_vpendbaser(c)		__gic_readq_nonatomic(c)
 
+static inline bool gic_prio_masking_enabled(void)
+{
+	return false;
+}
+
+static inline void gic_pmr_mask_irqs(void)
+{
+	/* Should not get called. */
+	WARN_ON_ONCE(true);
+}
+
+static inline void gic_arch_enable_irqs(void)
+{
+	/* Should not get called. */
+	WARN_ON_ONCE(true);
+}
+
 #endif /* !__ASSEMBLY__ */
 #endif /* !__ASM_ARCH_GICV3_H */
diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index 37193e2..b5f8142 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -155,5 +155,22 @@ static inline u32 gic_read_rpr(void)
 #define gits_write_vpendbaser(v, c)	writeq_relaxed(v, c)
 #define gits_read_vpendbaser(c)		readq_relaxed(c)
 
+static inline bool gic_prio_masking_enabled(void)
+{
+	return system_uses_irq_prio_masking();
+}
+
+static inline void gic_pmr_mask_irqs(void)
+{
+	/* Should not get called yet. */
+	WARN_ON_ONCE(true);
+}
+
+static inline void gic_arch_enable_irqs(void)
+{
+	/* Should not get called yet. */
+	WARN_ON_ONCE(true);
+}
+
 #endif /* __ASSEMBLY__ */
 #endif /* __ASM_ARCH_GICV3_H */
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 0868a9d..8148a92 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -356,6 +356,11 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 
 	irqnr = gic_read_iar();
 
+	if (gic_prio_masking_enabled()) {
+		gic_pmr_mask_irqs();
+		gic_arch_enable_irqs();
+	}
+
 	if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) {
 		int err;
 
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 07/26] arm64: ptrace: Provide definitions for PMR values

[Julien Thierry]

Introduce fixed values for PMR that are going to be used to mask and
unmask interrupts by priority.

The current priority given to GIC interrupts is 0xa0, so clearing PMR's
most significant bit is enough to mask interrupts.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 arch/arm64/include/asm/ptrace.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
index fce22c4..05cf913 100644
--- a/arch/arm64/include/asm/ptrace.h
+++ b/arch/arm64/include/asm/ptrace.h
@@ -25,6 +25,18 @@
 #define CurrentEL_EL1		(1 << 2)
 #define CurrentEL_EL2		(2 << 2)
 
+/*
+ * PMR values used to mask/unmask interrupts.
+ *
+ * GIC priority masking works as follows: if an IRQ's priority is a higher value
+ * than the value held in PMR, that interrupt is masked. A lower value of PMR
+ * means more IRQ priorities are masked.
+ *
+ * To mask priorities, we clear the most significant bit of PMR.
+ */
+#define GIC_PRIO_IRQON		0xf0
+#define GIC_PRIO_IRQOFF		(GIC_PRIO_IRQON & ~0x80)
+
 /* Additional SPSR bits not exposed in the UABI */
 #define PSR_IL_BIT		(1 << 20)
 
-- 
1.9.1

[PATCH v8 07/26] arm64: ptrace: Provide definitions for PMR values

[Julien Thierry]

Introduce fixed values for PMR that are going to be used to mask and
unmask interrupts by priority.

The current priority given to GIC interrupts is 0xa0, so clearing PMR's
most significant bit is enough to mask interrupts.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 arch/arm64/include/asm/ptrace.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
index fce22c4..05cf913 100644
--- a/arch/arm64/include/asm/ptrace.h
+++ b/arch/arm64/include/asm/ptrace.h
@@ -25,6 +25,18 @@
 #define CurrentEL_EL1		(1 << 2)
 #define CurrentEL_EL2		(2 << 2)
 
+/*
+ * PMR values used to mask/unmask interrupts.
+ *
+ * GIC priority masking works as follows: if an IRQ's priority is a higher value
+ * than the value held in PMR, that interrupt is masked. A lower value of PMR
+ * means more IRQ priorities are masked.
+ *
+ * To mask priorities, we clear the most significant bit of PMR.
+ */
+#define GIC_PRIO_IRQON		0xf0
+#define GIC_PRIO_IRQOFF		(GIC_PRIO_IRQON & ~0x80)
+
 /* Additional SPSR bits not exposed in the UABI */
 #define PSR_IL_BIT		(1 << 20)
 
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 08/26] arm64: Make PMR part of task context

[Julien Thierry]

In order to replace PSR.I interrupt disabling/enabling with ICC_PMR_EL1
interrupt masking, ICC_PMR_EL1 needs to be saved/restored when
taking/returning from an exception. This mimics the way hardware saves
and restores PSR.I bit in spsr_el1 for exceptions and ERET.

Add PMR to the registers to save in the pt_regs struct upon kernel entry,
and restore it before ERET. Also, initialize it to a sane value when
creating new tasks.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Dave Martin <Dave.Martin@arm.com>
---
 arch/arm64/include/asm/processor.h |  3 +++
 arch/arm64/include/asm/ptrace.h    | 14 +++++++++++---
 arch/arm64/kernel/asm-offsets.c    |  1 +
 arch/arm64/kernel/entry.S          | 14 ++++++++++++++
 arch/arm64/kernel/process.c        |  6 ++++++
 5 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index f1a7ab1..5d9ce62 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -191,6 +191,9 @@ static inline void start_thread_common(struct pt_regs *regs, unsigned long pc)
 	memset(regs, 0, sizeof(*regs));
 	forget_syscall(regs);
 	regs->pc = pc;
+
+	if (system_uses_irq_prio_masking())
+		regs->pmr_save = GIC_PRIO_IRQON;
 }
 
 static inline void start_thread(struct pt_regs *regs, unsigned long pc,
diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
index 05cf913..43e5df5 100644
--- a/arch/arm64/include/asm/ptrace.h
+++ b/arch/arm64/include/asm/ptrace.h
@@ -19,6 +19,8 @@
 #ifndef __ASM_PTRACE_H
 #define __ASM_PTRACE_H
 
+#include <asm/cpufeature.h>
+
 #include <uapi/asm/ptrace.h>
 
 /* Current Exception Level values, as contained in CurrentEL */
@@ -179,7 +181,8 @@ struct pt_regs {
 #endif
 
 	u64 orig_addr_limit;
-	u64 unused;	// maintain 16 byte alignment
+	/* Only valid when ARM64_HAS_IRQ_PRIO_MASKING is enabled. */
+	u64 pmr_save;
 	u64 stackframe[2];
 };
 
@@ -214,8 +217,13 @@ static inline void forget_syscall(struct pt_regs *regs)
 #define processor_mode(regs) \
 	((regs)->pstate & PSR_MODE_MASK)
 
-#define interrupts_enabled(regs) \
-	(!((regs)->pstate & PSR_I_BIT))
+#define irqs_priority_unmasked(regs)					\
+	(system_uses_irq_prio_masking() ?				\
+		(regs)->pmr_save == GIC_PRIO_IRQON :			\
+		true)
+
+#define interrupts_enabled(regs)			\
+	(!((regs)->pstate & PSR_I_BIT) && irqs_priority_unmasked(regs))
 
 #define fast_interrupts_enabled(regs) \
 	(!((regs)->pstate & PSR_F_BIT))
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 65b8afc..90ab2cf 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -81,6 +81,7 @@ int main(void)
   DEFINE(S_ORIG_X0,		offsetof(struct pt_regs, orig_x0));
   DEFINE(S_SYSCALLNO,		offsetof(struct pt_regs, syscallno));
   DEFINE(S_ORIG_ADDR_LIMIT,	offsetof(struct pt_regs, orig_addr_limit));
+  DEFINE(S_PMR_SAVE,		offsetof(struct pt_regs, pmr_save));
   DEFINE(S_STACKFRAME,		offsetof(struct pt_regs, stackframe));
   DEFINE(S_FRAME_SIZE,		sizeof(struct pt_regs));
   BLANK();
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 0ec0c46..35a47f6 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -249,6 +249,12 @@ alternative_else_nop_endif
 	msr	sp_el0, tsk
 	.endif
 
+	/* Save pmr */
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+	mrs_s	x20, SYS_ICC_PMR_EL1
+	str	x20, [sp, #S_PMR_SAVE]
+alternative_else_nop_endif
+
 	/*
 	 * Registers that may be useful after this macro is invoked:
 	 *
@@ -269,6 +275,14 @@ alternative_else_nop_endif
 	/* No need to restore UAO, it will be restored from SPSR_EL1 */
 	.endif
 
+	/* Restore pmr */
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+	ldr	x20, [sp, #S_PMR_SAVE]
+	msr_s	SYS_ICC_PMR_EL1, x20
+	/* Ensure priority change is seen by redistributor */
+	dsb	sy
+alternative_else_nop_endif
+
 	ldp	x21, x22, [sp, #S_PC]		// load ELR, SPSR
 	.if	\el == 0
 	ct_user_enter
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index a0f985a..6d410fc 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -232,6 +232,9 @@ void __show_regs(struct pt_regs *regs)
 
 	printk("sp : %016llx\n", sp);
 
+	if (system_uses_irq_prio_masking())
+		printk("pmr_save: %08llx\n", regs->pmr_save);
+
 	i = top_reg;
 
 	while (i >= 0) {
@@ -363,6 +366,9 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start,
 		if (arm64_get_ssbd_state() == ARM64_SSBD_FORCE_DISABLE)
 			childregs->pstate |= PSR_SSBS_BIT;
 
+		if (system_uses_irq_prio_masking())
+			childregs->pmr_save = GIC_PRIO_IRQON;
+
 		p->thread.cpu_context.x19 = stack_start;
 		p->thread.cpu_context.x20 = stk_sz;
 	}
-- 
1.9.1

[PATCH v8 08/26] arm64: Make PMR part of task context

[Julien Thierry]

In order to replace PSR.I interrupt disabling/enabling with ICC_PMR_EL1
interrupt masking, ICC_PMR_EL1 needs to be saved/restored when
taking/returning from an exception. This mimics the way hardware saves
and restores PSR.I bit in spsr_el1 for exceptions and ERET.

Add PMR to the registers to save in the pt_regs struct upon kernel entry,
and restore it before ERET. Also, initialize it to a sane value when
creating new tasks.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Dave Martin <Dave.Martin@arm.com>
---
 arch/arm64/include/asm/processor.h |  3 +++
 arch/arm64/include/asm/ptrace.h    | 14 +++++++++++---
 arch/arm64/kernel/asm-offsets.c    |  1 +
 arch/arm64/kernel/entry.S          | 14 ++++++++++++++
 arch/arm64/kernel/process.c        |  6 ++++++
 5 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index f1a7ab1..5d9ce62 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -191,6 +191,9 @@ static inline void start_thread_common(struct pt_regs *regs, unsigned long pc)
 	memset(regs, 0, sizeof(*regs));
 	forget_syscall(regs);
 	regs->pc = pc;
+
+	if (system_uses_irq_prio_masking())
+		regs->pmr_save = GIC_PRIO_IRQON;
 }
 
 static inline void start_thread(struct pt_regs *regs, unsigned long pc,
diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
index 05cf913..43e5df5 100644
--- a/arch/arm64/include/asm/ptrace.h
+++ b/arch/arm64/include/asm/ptrace.h
@@ -19,6 +19,8 @@
 #ifndef __ASM_PTRACE_H
 #define __ASM_PTRACE_H
 
+#include <asm/cpufeature.h>
+
 #include <uapi/asm/ptrace.h>
 
 /* Current Exception Level values, as contained in CurrentEL */
@@ -179,7 +181,8 @@ struct pt_regs {
 #endif
 
 	u64 orig_addr_limit;
-	u64 unused;	// maintain 16 byte alignment
+	/* Only valid when ARM64_HAS_IRQ_PRIO_MASKING is enabled. */
+	u64 pmr_save;
 	u64 stackframe[2];
 };
 
@@ -214,8 +217,13 @@ static inline void forget_syscall(struct pt_regs *regs)
 #define processor_mode(regs) \
 	((regs)->pstate & PSR_MODE_MASK)
 
-#define interrupts_enabled(regs) \
-	(!((regs)->pstate & PSR_I_BIT))
+#define irqs_priority_unmasked(regs)					\
+	(system_uses_irq_prio_masking() ?				\
+		(regs)->pmr_save == GIC_PRIO_IRQON :			\
+		true)
+
+#define interrupts_enabled(regs)			\
+	(!((regs)->pstate & PSR_I_BIT) && irqs_priority_unmasked(regs))
 
 #define fast_interrupts_enabled(regs) \
 	(!((regs)->pstate & PSR_F_BIT))
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 65b8afc..90ab2cf 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -81,6 +81,7 @@ int main(void)
   DEFINE(S_ORIG_X0,		offsetof(struct pt_regs, orig_x0));
   DEFINE(S_SYSCALLNO,		offsetof(struct pt_regs, syscallno));
   DEFINE(S_ORIG_ADDR_LIMIT,	offsetof(struct pt_regs, orig_addr_limit));
+  DEFINE(S_PMR_SAVE,		offsetof(struct pt_regs, pmr_save));
   DEFINE(S_STACKFRAME,		offsetof(struct pt_regs, stackframe));
   DEFINE(S_FRAME_SIZE,		sizeof(struct pt_regs));
   BLANK();
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 0ec0c46..35a47f6 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -249,6 +249,12 @@ alternative_else_nop_endif
 	msr	sp_el0, tsk
 	.endif
 
+	/* Save pmr */
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+	mrs_s	x20, SYS_ICC_PMR_EL1
+	str	x20, [sp, #S_PMR_SAVE]
+alternative_else_nop_endif
+
 	/*
 	 * Registers that may be useful after this macro is invoked:
 	 *
@@ -269,6 +275,14 @@ alternative_else_nop_endif
 	/* No need to restore UAO, it will be restored from SPSR_EL1 */
 	.endif
 
+	/* Restore pmr */
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+	ldr	x20, [sp, #S_PMR_SAVE]
+	msr_s	SYS_ICC_PMR_EL1, x20
+	/* Ensure priority change is seen by redistributor */
+	dsb	sy
+alternative_else_nop_endif
+
 	ldp	x21, x22, [sp, #S_PC]		// load ELR, SPSR
 	.if	\el == 0
 	ct_user_enter
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index a0f985a..6d410fc 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -232,6 +232,9 @@ void __show_regs(struct pt_regs *regs)
 
 	printk("sp : %016llx\n", sp);
 
+	if (system_uses_irq_prio_masking())
+		printk("pmr_save: %08llx\n", regs->pmr_save);
+
 	i = top_reg;
 
 	while (i >= 0) {
@@ -363,6 +366,9 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start,
 		if (arm64_get_ssbd_state() == ARM64_SSBD_FORCE_DISABLE)
 			childregs->pstate |= PSR_SSBS_BIT;
 
+		if (system_uses_irq_prio_masking())
+			childregs->pmr_save = GIC_PRIO_IRQON;
+
 		p->thread.cpu_context.x19 = stack_start;
 		p->thread.cpu_context.x20 = stk_sz;
 	}
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 09/26] arm64: Unmask PMR before going idle

[Julien Thierry]

CPU does not received signals for interrupts with a priority masked by
ICC_PMR_EL1. This means the CPU might not come back from a WFI
instruction.

Make sure ICC_PMR_EL1 does not mask interrupts when doing a WFI.

Since the logic of cpu_do_idle is becoming a bit more complex than just
two instructions, lets turn it from ASM to C.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 arch/arm64/kernel/process.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 arch/arm64/mm/proc.S        | 11 -----------
 2 files changed, 45 insertions(+), 11 deletions(-)

diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 6d410fc..f05b63f 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -51,6 +51,7 @@
 #include <linux/thread_info.h>
 
 #include <asm/alternative.h>
+#include <asm/arch_gicv3.h>
 #include <asm/compat.h>
 #include <asm/cacheflush.h>
 #include <asm/exec.h>
@@ -74,6 +75,50 @@
 
 void (*arm_pm_restart)(enum reboot_mode reboot_mode, const char *cmd);
 
+static inline void __cpu_do_idle(void)
+{
+	dsb(sy);
+	wfi();
+}
+
+static inline void __cpu_do_idle_irqprio(void)
+{
+	unsigned long pmr;
+	unsigned long daif_bits;
+
+	daif_bits = read_sysreg(daif);
+	write_sysreg(daif_bits | PSR_I_BIT, daif);
+
+	/*
+	 * Unmask PMR before going idle to make sure interrupts can
+	 * be raised.
+	 */
+	pmr = gic_read_pmr();
+	gic_write_pmr(GIC_PRIO_IRQON);
+
+	__cpu_do_idle();
+
+	gic_write_pmr(pmr);
+	write_sysreg(daif_bits, daif);
+}
+
+/*
+ *	cpu_do_idle()
+ *
+ *	Idle the processor (wait for interrupt).
+ *
+ *	If the CPU supports priority masking we must do additional work to
+ *	ensure that interrupts are not masked at the PMR (because the core will
+ *	not wake up if we block the wake up signal in the interrupt controller).
+ */
+void cpu_do_idle(void)
+{
+	if (system_uses_irq_prio_masking())
+		__cpu_do_idle_irqprio();
+	else
+		__cpu_do_idle();
+}
+
 /*
  * This is our default idle handler.
  */
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 73886a5..3ea4f3b 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -55,17 +55,6 @@
 
 #define MAIR(attr, mt)	((attr) << ((mt) * 8))
 
-/*
- *	cpu_do_idle()
- *
- *	Idle the processor (wait for interrupt).
- */
-ENTRY(cpu_do_idle)
-	dsb	sy				// WFI may enter a low-power mode
-	wfi
-	ret
-ENDPROC(cpu_do_idle)
-
 #ifdef CONFIG_CPU_PM
 /**
  * cpu_do_suspend - save CPU registers context
-- 
1.9.1

[PATCH v8 09/26] arm64: Unmask PMR before going idle

[Julien Thierry]

CPU does not received signals for interrupts with a priority masked by
ICC_PMR_EL1. This means the CPU might not come back from a WFI
instruction.

Make sure ICC_PMR_EL1 does not mask interrupts when doing a WFI.

Since the logic of cpu_do_idle is becoming a bit more complex than just
two instructions, lets turn it from ASM to C.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 arch/arm64/kernel/process.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 arch/arm64/mm/proc.S        | 11 -----------
 2 files changed, 45 insertions(+), 11 deletions(-)

diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 6d410fc..f05b63f 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -51,6 +51,7 @@
 #include <linux/thread_info.h>
 
 #include <asm/alternative.h>
+#include <asm/arch_gicv3.h>
 #include <asm/compat.h>
 #include <asm/cacheflush.h>
 #include <asm/exec.h>
@@ -74,6 +75,50 @@
 
 void (*arm_pm_restart)(enum reboot_mode reboot_mode, const char *cmd);
 
+static inline void __cpu_do_idle(void)
+{
+	dsb(sy);
+	wfi();
+}
+
+static inline void __cpu_do_idle_irqprio(void)
+{
+	unsigned long pmr;
+	unsigned long daif_bits;
+
+	daif_bits = read_sysreg(daif);
+	write_sysreg(daif_bits | PSR_I_BIT, daif);
+
+	/*
+	 * Unmask PMR before going idle to make sure interrupts can
+	 * be raised.
+	 */
+	pmr = gic_read_pmr();
+	gic_write_pmr(GIC_PRIO_IRQON);
+
+	__cpu_do_idle();
+
+	gic_write_pmr(pmr);
+	write_sysreg(daif_bits, daif);
+}
+
+/*
+ *	cpu_do_idle()
+ *
+ *	Idle the processor (wait for interrupt).
+ *
+ *	If the CPU supports priority masking we must do additional work to
+ *	ensure that interrupts are not masked at the PMR (because the core will
+ *	not wake up if we block the wake up signal in the interrupt controller).
+ */
+void cpu_do_idle(void)
+{
+	if (system_uses_irq_prio_masking())
+		__cpu_do_idle_irqprio();
+	else
+		__cpu_do_idle();
+}
+
 /*
  * This is our default idle handler.
  */
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 73886a5..3ea4f3b 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -55,17 +55,6 @@
 
 #define MAIR(attr, mt)	((attr) << ((mt) * 8))
 
-/*
- *	cpu_do_idle()
- *
- *	Idle the processor (wait for interrupt).
- */
-ENTRY(cpu_do_idle)
-	dsb	sy				// WFI may enter a low-power mode
-	wfi
-	ret
-ENDPROC(cpu_do_idle)
-
 #ifdef CONFIG_CPU_PM
 /**
  * cpu_do_suspend - save CPU registers context
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 10/26] arm64: kvm: Unmask PMR before entering guest

[Julien Thierry]

Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This
means that hypervisor will not receive masked interrupts while running a
guest.

Avoid this by making sure ICC_PMR_EL1 is unmasked when we enter a guest.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Christoffer Dall <christoffer.dall@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: kvmarm@lists.cs.columbia.edu
---
 arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++
 arch/arm64/kvm/hyp/switch.c       | 16 ++++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 7732d0b..a1f9f55 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -24,6 +24,7 @@
 
 #include <linux/types.h>
 #include <linux/kvm_types.h>
+#include <asm/arch_gicv3.h>
 #include <asm/cpufeature.h>
 #include <asm/daifflags.h>
 #include <asm/fpsimd.h>
@@ -474,6 +475,17 @@ static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)
 static inline void kvm_arm_vhe_guest_enter(void)
 {
 	local_daif_mask();
+
+	/*
+	 * Having IRQs masked via PMR when entering the guest means the GIC
+	 * will not signal the CPU of interrupts of lower priority, and the
+	 * only way to get out will be via guest exceptions.
+	 * Naturally, we want to avoid this.
+	 */
+	if (system_uses_irq_prio_masking()) {
+		gic_write_pmr(GIC_PRIO_IRQON);
+		dsb(sy);
+	}
 }
 
 static inline void kvm_arm_vhe_guest_exit(void)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index b0b1478..6a4c2d6 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -22,6 +22,7 @@
 
 #include <kvm/arm_psci.h>
 
+#include <asm/arch_gicv3.h>
 #include <asm/cpufeature.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_emulate.h>
@@ -521,6 +522,17 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
 	struct kvm_cpu_context *guest_ctxt;
 	u64 exit_code;
 
+	/*
+	 * Having IRQs masked via PMR when entering the guest means the GIC
+	 * will not signal the CPU of interrupts of lower priority, and the
+	 * only way to get out will be via guest exceptions.
+	 * Naturally, we want to avoid this.
+	 */
+	if (system_uses_irq_prio_masking()) {
+		gic_write_pmr(GIC_PRIO_IRQON);
+		dsb(sy);
+	}
+
 	vcpu = kern_hyp_va(vcpu);
 
 	host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
@@ -573,6 +585,10 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
 	 */
 	__debug_switch_to_host(vcpu);
 
+	/* Returning to host will clear PSR.I, remask PMR if needed */
+	if (system_uses_irq_prio_masking())
+		gic_write_pmr(GIC_PRIO_IRQOFF);
+
 	return exit_code;
 }
 
-- 
1.9.1

[PATCH v8 10/26] arm64: kvm: Unmask PMR before entering guest

[Julien Thierry]

Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This
means that hypervisor will not receive masked interrupts while running a
guest.

Avoid this by making sure ICC_PMR_EL1 is unmasked when we enter a guest.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Christoffer Dall <christoffer.dall@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: kvmarm@lists.cs.columbia.edu
---
 arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++
 arch/arm64/kvm/hyp/switch.c       | 16 ++++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 7732d0b..a1f9f55 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -24,6 +24,7 @@
 
 #include <linux/types.h>
 #include <linux/kvm_types.h>
+#include <asm/arch_gicv3.h>
 #include <asm/cpufeature.h>
 #include <asm/daifflags.h>
 #include <asm/fpsimd.h>
@@ -474,6 +475,17 @@ static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)
 static inline void kvm_arm_vhe_guest_enter(void)
 {
 	local_daif_mask();
+
+	/*
+	 * Having IRQs masked via PMR when entering the guest means the GIC
+	 * will not signal the CPU of interrupts of lower priority, and the
+	 * only way to get out will be via guest exceptions.
+	 * Naturally, we want to avoid this.
+	 */
+	if (system_uses_irq_prio_masking()) {
+		gic_write_pmr(GIC_PRIO_IRQON);
+		dsb(sy);
+	}
 }
 
 static inline void kvm_arm_vhe_guest_exit(void)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index b0b1478..6a4c2d6 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -22,6 +22,7 @@
 
 #include <kvm/arm_psci.h>
 
+#include <asm/arch_gicv3.h>
 #include <asm/cpufeature.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_emulate.h>
@@ -521,6 +522,17 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
 	struct kvm_cpu_context *guest_ctxt;
 	u64 exit_code;
 
+	/*
+	 * Having IRQs masked via PMR when entering the guest means the GIC
+	 * will not signal the CPU of interrupts of lower priority, and the
+	 * only way to get out will be via guest exceptions.
+	 * Naturally, we want to avoid this.
+	 */
+	if (system_uses_irq_prio_masking()) {
+		gic_write_pmr(GIC_PRIO_IRQON);
+		dsb(sy);
+	}
+
 	vcpu = kern_hyp_va(vcpu);
 
 	host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
@@ -573,6 +585,10 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
 	 */
 	__debug_switch_to_host(vcpu);
 
+	/* Returning to host will clear PSR.I, remask PMR if needed */
+	if (system_uses_irq_prio_masking())
+		gic_write_pmr(GIC_PRIO_IRQOFF);
+
 	return exit_code;
 }
 
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 11/26] efi: Let architectures decide the flags that should be saved/restored

[Julien Thierry]

Currently, irqflags are saved before calling runtime services and
checked for mismatch on return.

Provide a pair of overridable macros to save and restore (if needed) the
state that need to be preserved on return from a runtime service.
This allows to check for flags that are not necesarly related to
irqflags.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
---
 drivers/firmware/efi/runtime-wrappers.c | 17 +++++++++++++++--
 include/linux/efi.h                     |  5 +++--
 2 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/firmware/efi/runtime-wrappers.c b/drivers/firmware/efi/runtime-wrappers.c
index 8903b9c..2f4b68b 100644
--- a/drivers/firmware/efi/runtime-wrappers.c
+++ b/drivers/firmware/efi/runtime-wrappers.c
@@ -89,11 +89,24 @@
 	efi_rts_work.status;						\
 })
 
+#ifndef arch_efi_save_flags
+#define arch_efi_save_flags(state_flags)	local_save_flags(state_flags)
+#define arch_efi_restore_flags(state_flags)	local_irq_restore(state_flags)
+#endif
+
+inline unsigned long efi_call_virt_save_flags(void)
+{
+	unsigned long flags;
+
+	arch_efi_save_flags(flags);
+	return flags;
+}
+
 void efi_call_virt_check_flags(unsigned long flags, const char *call)
 {
 	unsigned long cur_flags, mismatch;
 
-	local_save_flags(cur_flags);
+	cur_flags = efi_call_virt_save_flags();
 
 	mismatch = flags ^ cur_flags;
 	if (!WARN_ON_ONCE(mismatch & ARCH_EFI_IRQ_FLAGS_MASK))
@@ -102,7 +115,7 @@ void efi_call_virt_check_flags(unsigned long flags, const char *call)
 	add_taint(TAINT_FIRMWARE_WORKAROUND, LOCKDEP_NOW_UNRELIABLE);
 	pr_err_ratelimited(FW_BUG "IRQ flags corrupted (0x%08lx=>0x%08lx) by EFI %s\n",
 			   flags, cur_flags, call);
-	local_irq_restore(flags);
+	arch_efi_restore_flags(flags);
 }
 
 /*
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 45ff763..bd80b7e 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1607,6 +1607,7 @@ efi_status_t efi_setup_gop(efi_system_table_t *sys_table_arg,
 
 bool efi_runtime_disabled(void);
 extern void efi_call_virt_check_flags(unsigned long flags, const char *call);
+extern unsigned long efi_call_virt_save_flags(void);
 
 enum efi_secureboot_mode {
 	efi_secureboot_mode_unset,
@@ -1652,7 +1653,7 @@ enum efi_secureboot_mode {
 									\
 	arch_efi_call_virt_setup();					\
 									\
-	local_save_flags(__flags);					\
+	__flags = efi_call_virt_save_flags();				\
 	__s = arch_efi_call_virt(p, f, args);				\
 	efi_call_virt_check_flags(__flags, __stringify(f));		\
 									\
@@ -1667,7 +1668,7 @@ enum efi_secureboot_mode {
 									\
 	arch_efi_call_virt_setup();					\
 									\
-	local_save_flags(__flags);					\
+	__flags = efi_call_virt_save_flags();				\
 	arch_efi_call_virt(p, f, args);					\
 	efi_call_virt_check_flags(__flags, __stringify(f));		\
 									\
-- 
1.9.1

[PATCH v8 11/26] efi: Let architectures decide the flags that should be saved/restored

[Julien Thierry]

Currently, irqflags are saved before calling runtime services and
checked for mismatch on return.

Provide a pair of overridable macros to save and restore (if needed) the
state that need to be preserved on return from a runtime service.
This allows to check for flags that are not necesarly related to
irqflags.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
---
 drivers/firmware/efi/runtime-wrappers.c | 17 +++++++++++++++--
 include/linux/efi.h                     |  5 +++--
 2 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/firmware/efi/runtime-wrappers.c b/drivers/firmware/efi/runtime-wrappers.c
index 8903b9c..2f4b68b 100644
--- a/drivers/firmware/efi/runtime-wrappers.c
+++ b/drivers/firmware/efi/runtime-wrappers.c
@@ -89,11 +89,24 @@
 	efi_rts_work.status;						\
 })
 
+#ifndef arch_efi_save_flags
+#define arch_efi_save_flags(state_flags)	local_save_flags(state_flags)
+#define arch_efi_restore_flags(state_flags)	local_irq_restore(state_flags)
+#endif
+
+inline unsigned long efi_call_virt_save_flags(void)
+{
+	unsigned long flags;
+
+	arch_efi_save_flags(flags);
+	return flags;
+}
+
 void efi_call_virt_check_flags(unsigned long flags, const char *call)
 {
 	unsigned long cur_flags, mismatch;
 
-	local_save_flags(cur_flags);
+	cur_flags = efi_call_virt_save_flags();
 
 	mismatch = flags ^ cur_flags;
 	if (!WARN_ON_ONCE(mismatch & ARCH_EFI_IRQ_FLAGS_MASK))
@@ -102,7 +115,7 @@ void efi_call_virt_check_flags(unsigned long flags, const char *call)
 	add_taint(TAINT_FIRMWARE_WORKAROUND, LOCKDEP_NOW_UNRELIABLE);
 	pr_err_ratelimited(FW_BUG "IRQ flags corrupted (0x%08lx=>0x%08lx) by EFI %s\n",
 			   flags, cur_flags, call);
-	local_irq_restore(flags);
+	arch_efi_restore_flags(flags);
 }
 
 /*
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 45ff763..bd80b7e 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1607,6 +1607,7 @@ efi_status_t efi_setup_gop(efi_system_table_t *sys_table_arg,
 
 bool efi_runtime_disabled(void);
 extern void efi_call_virt_check_flags(unsigned long flags, const char *call);
+extern unsigned long efi_call_virt_save_flags(void);
 
 enum efi_secureboot_mode {
 	efi_secureboot_mode_unset,
@@ -1652,7 +1653,7 @@ enum efi_secureboot_mode {
 									\
 	arch_efi_call_virt_setup();					\
 									\
-	local_save_flags(__flags);					\
+	__flags = efi_call_virt_save_flags();				\
 	__s = arch_efi_call_virt(p, f, args);				\
 	efi_call_virt_check_flags(__flags, __stringify(f));		\
 									\
@@ -1667,7 +1668,7 @@ enum efi_secureboot_mode {
 									\
 	arch_efi_call_virt_setup();					\
 									\
-	local_save_flags(__flags);					\
+	__flags = efi_call_virt_save_flags();				\
 	arch_efi_call_virt(p, f, args);					\
 	efi_call_virt_check_flags(__flags, __stringify(f));		\
 									\
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

Instead disabling interrupts by setting the PSR.I bit, use a priority
higher than the one used for interrupts to mask them via PMR.

When using PMR to disable interrupts, the value of PMR will be used
instead of PSR.[DAIF] for the irqflags.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Oleg Nesterov <oleg@redhat.com>
---
 arch/arm64/include/asm/efi.h      |  11 ++++
 arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
 2 files changed, 106 insertions(+), 28 deletions(-)

diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
index 7ed3208..134ff6e 100644
--- a/arch/arm64/include/asm/efi.h
+++ b/arch/arm64/include/asm/efi.h
@@ -44,6 +44,17 @@
 
 #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
 
+#define arch_efi_save_flags(state_flags)		\
+	do {						\
+		(state_flags) =	read_sysreg(daif);	\
+	} while (0)
+
+#define arch_efi_restore_flags(state_flags)		\
+	do {						\
+		write_sysreg(state_flags, daif);	\
+	} while (0)
+
+
 /* arch specific definitions used by the stub code */
 
 /*
diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
index 24692ed..fa3b06f 100644
--- a/arch/arm64/include/asm/irqflags.h
+++ b/arch/arm64/include/asm/irqflags.h
@@ -18,7 +18,9 @@
 
 #ifdef __KERNEL__
 
+#include <asm/alternative.h>
 #include <asm/ptrace.h>
+#include <asm/sysreg.h>
 
 /*
  * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
@@ -36,47 +38,96 @@
 /*
  * CPU interrupt mask handling.
  */
-static inline unsigned long arch_local_irq_save(void)
-{
-	unsigned long flags;
-	asm volatile(
-		"mrs	%0, daif		// arch_local_irq_save\n"
-		"msr	daifset, #2"
-		: "=r" (flags)
-		:
-		: "memory");
-	return flags;
-}
-
 static inline void arch_local_irq_enable(void)
 {
-	asm volatile(
-		"msr	daifclr, #2		// arch_local_irq_enable"
-		:
+	unsigned long unmasked = GIC_PRIO_IRQON;
+
+	asm volatile(ALTERNATIVE(
+		"msr	daifclr, #2		// arch_local_irq_enable\n"
+		"nop",
+		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
+		"dsb	sy",
+		ARM64_HAS_IRQ_PRIO_MASKING)
 		:
+		: "r" (unmasked)
 		: "memory");
 }
 
 static inline void arch_local_irq_disable(void)
 {
-	asm volatile(
-		"msr	daifset, #2		// arch_local_irq_disable"
-		:
+	unsigned long masked = GIC_PRIO_IRQOFF;
+
+	asm volatile(ALTERNATIVE(
+		"msr	daifset, #2		// arch_local_irq_disable",
+		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ", %0",
+		ARM64_HAS_IRQ_PRIO_MASKING)
 		:
+		: "r" (masked)
 		: "memory");
 }
 
 /*
+ * Having two ways to control interrupt status is a bit complicated. Some
+ * locations like exception entries will have PSR.I bit set by the architecture
+ * while PMR is unmasked.
+ * We need the irqflags to represent that interrupts are disabled in such cases.
+ *
+ * For this, we lower the value read from PMR when the I bit is set so it is
+ * considered as an irq masking priority. (With PMR, lower value means masking
+ * more interrupts).
+ */
+#define _get_irqflags(daif_bits, pmr)					\
+({									\
+	unsigned long flags;						\
+									\
+	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
+	asm volatile(ALTERNATIVE(					\
+		"mov	%0, %1\n"					\
+		"nop\n"							\
+		"nop",							\
+		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
+		"mvn	%0, %0\n"					\
+		"and	%0, %0, %2",					\
+		ARM64_HAS_IRQ_PRIO_MASKING)				\
+		: "=&r" (flags)						\
+		: "r" (daif_bits), "r" (pmr)				\
+		: "memory");						\
+									\
+	flags;								\
+})
+
+/*
  * Save the current interrupt enable state.
  */
 static inline unsigned long arch_local_save_flags(void)
 {
-	unsigned long flags;
-	asm volatile(
-		"mrs	%0, daif		// arch_local_save_flags"
-		: "=r" (flags)
+	unsigned long daif_bits;
+	unsigned long pmr; // Only used if alternative is on
+
+	daif_bits = read_sysreg(daif);
+
+	// Get PMR
+	asm volatile(ALTERNATIVE(
+			"nop",
+			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
+			ARM64_HAS_IRQ_PRIO_MASKING)
+		: "=&r" (pmr)
 		:
 		: "memory");
+
+	return _get_irqflags(daif_bits, pmr);
+}
+
+#undef _get_irqflags
+
+static inline unsigned long arch_local_irq_save(void)
+{
+	unsigned long flags;
+
+	flags = arch_local_save_flags();
+
+	arch_local_irq_disable();
+
 	return flags;
 }
 
@@ -85,16 +136,32 @@ static inline unsigned long arch_local_save_flags(void)
  */
 static inline void arch_local_irq_restore(unsigned long flags)
 {
-	asm volatile(
-		"msr	daif, %0		// arch_local_irq_restore"
-	:
-	: "r" (flags)
-	: "memory");
+	asm volatile(ALTERNATIVE(
+			"msr	daif, %0\n"
+			"nop",
+			"msr_s	" __stringify(SYS_ICC_PMR_EL1) ", %0\n"
+			"dsb	sy",
+			ARM64_HAS_IRQ_PRIO_MASKING)
+		: "+r" (flags)
+		:
+		: "memory");
 }
 
 static inline int arch_irqs_disabled_flags(unsigned long flags)
 {
-	return flags & PSR_I_BIT;
+	int res;
+
+	asm volatile(ALTERNATIVE(
+			"and	%w0, %w1, #" __stringify(PSR_I_BIT) "\n"
+			"nop",
+			"cmp	%w1, #" __stringify(GIC_PRIO_IRQOFF) "\n"
+			"cset	%w0, ls",
+			ARM64_HAS_IRQ_PRIO_MASKING)
+		: "=&r" (res)
+		: "r" ((int) flags)
+		: "memory");
+
+	return res;
 }
 #endif
 #endif
-- 
1.9.1

[PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

Instead disabling interrupts by setting the PSR.I bit, use a priority
higher than the one used for interrupts to mask them via PMR.

When using PMR to disable interrupts, the value of PMR will be used
instead of PSR.[DAIF] for the irqflags.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Oleg Nesterov <oleg@redhat.com>
---
 arch/arm64/include/asm/efi.h      |  11 ++++
 arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
 2 files changed, 106 insertions(+), 28 deletions(-)

diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
index 7ed3208..134ff6e 100644
--- a/arch/arm64/include/asm/efi.h
+++ b/arch/arm64/include/asm/efi.h
@@ -44,6 +44,17 @@
 
 #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
 
+#define arch_efi_save_flags(state_flags)		\
+	do {						\
+		(state_flags) =	read_sysreg(daif);	\
+	} while (0)
+
+#define arch_efi_restore_flags(state_flags)		\
+	do {						\
+		write_sysreg(state_flags, daif);	\
+	} while (0)
+
+
 /* arch specific definitions used by the stub code */
 
 /*
diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
index 24692ed..fa3b06f 100644
--- a/arch/arm64/include/asm/irqflags.h
+++ b/arch/arm64/include/asm/irqflags.h
@@ -18,7 +18,9 @@
 
 #ifdef __KERNEL__
 
+#include <asm/alternative.h>
 #include <asm/ptrace.h>
+#include <asm/sysreg.h>
 
 /*
  * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
@@ -36,47 +38,96 @@
 /*
  * CPU interrupt mask handling.
  */
-static inline unsigned long arch_local_irq_save(void)
-{
-	unsigned long flags;
-	asm volatile(
-		"mrs	%0, daif		// arch_local_irq_save\n"
-		"msr	daifset, #2"
-		: "=r" (flags)
-		:
-		: "memory");
-	return flags;
-}
-
 static inline void arch_local_irq_enable(void)
 {
-	asm volatile(
-		"msr	daifclr, #2		// arch_local_irq_enable"
-		:
+	unsigned long unmasked = GIC_PRIO_IRQON;
+
+	asm volatile(ALTERNATIVE(
+		"msr	daifclr, #2		// arch_local_irq_enable\n"
+		"nop",
+		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
+		"dsb	sy",
+		ARM64_HAS_IRQ_PRIO_MASKING)
 		:
+		: "r" (unmasked)
 		: "memory");
 }
 
 static inline void arch_local_irq_disable(void)
 {
-	asm volatile(
-		"msr	daifset, #2		// arch_local_irq_disable"
-		:
+	unsigned long masked = GIC_PRIO_IRQOFF;
+
+	asm volatile(ALTERNATIVE(
+		"msr	daifset, #2		// arch_local_irq_disable",
+		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ", %0",
+		ARM64_HAS_IRQ_PRIO_MASKING)
 		:
+		: "r" (masked)
 		: "memory");
 }
 
 /*
+ * Having two ways to control interrupt status is a bit complicated. Some
+ * locations like exception entries will have PSR.I bit set by the architecture
+ * while PMR is unmasked.
+ * We need the irqflags to represent that interrupts are disabled in such cases.
+ *
+ * For this, we lower the value read from PMR when the I bit is set so it is
+ * considered as an irq masking priority. (With PMR, lower value means masking
+ * more interrupts).
+ */
+#define _get_irqflags(daif_bits, pmr)					\
+({									\
+	unsigned long flags;						\
+									\
+	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
+	asm volatile(ALTERNATIVE(					\
+		"mov	%0, %1\n"					\
+		"nop\n"							\
+		"nop",							\
+		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
+		"mvn	%0, %0\n"					\
+		"and	%0, %0, %2",					\
+		ARM64_HAS_IRQ_PRIO_MASKING)				\
+		: "=&r" (flags)						\
+		: "r" (daif_bits), "r" (pmr)				\
+		: "memory");						\
+									\
+	flags;								\
+})
+
+/*
  * Save the current interrupt enable state.
  */
 static inline unsigned long arch_local_save_flags(void)
 {
-	unsigned long flags;
-	asm volatile(
-		"mrs	%0, daif		// arch_local_save_flags"
-		: "=r" (flags)
+	unsigned long daif_bits;
+	unsigned long pmr; // Only used if alternative is on
+
+	daif_bits = read_sysreg(daif);
+
+	// Get PMR
+	asm volatile(ALTERNATIVE(
+			"nop",
+			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
+			ARM64_HAS_IRQ_PRIO_MASKING)
+		: "=&r" (pmr)
 		:
 		: "memory");
+
+	return _get_irqflags(daif_bits, pmr);
+}
+
+#undef _get_irqflags
+
+static inline unsigned long arch_local_irq_save(void)
+{
+	unsigned long flags;
+
+	flags = arch_local_save_flags();
+
+	arch_local_irq_disable();
+
 	return flags;
 }
 
@@ -85,16 +136,32 @@ static inline unsigned long arch_local_save_flags(void)
  */
 static inline void arch_local_irq_restore(unsigned long flags)
 {
-	asm volatile(
-		"msr	daif, %0		// arch_local_irq_restore"
-	:
-	: "r" (flags)
-	: "memory");
+	asm volatile(ALTERNATIVE(
+			"msr	daif, %0\n"
+			"nop",
+			"msr_s	" __stringify(SYS_ICC_PMR_EL1) ", %0\n"
+			"dsb	sy",
+			ARM64_HAS_IRQ_PRIO_MASKING)
+		: "+r" (flags)
+		:
+		: "memory");
 }
 
 static inline int arch_irqs_disabled_flags(unsigned long flags)
 {
-	return flags & PSR_I_BIT;
+	int res;
+
+	asm volatile(ALTERNATIVE(
+			"and	%w0, %w1, #" __stringify(PSR_I_BIT) "\n"
+			"nop",
+			"cmp	%w1, #" __stringify(GIC_PRIO_IRQOFF) "\n"
+			"cset	%w0, ls",
+			ARM64_HAS_IRQ_PRIO_MASKING)
+		: "=&r" (res)
+		: "r" ((int) flags)
+		: "memory");
+
+	return res;
 }
 #endif
 #endif
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 13/26] arm64: daifflags: Include PMR in daifflags restore operations

[Julien Thierry]

The addition of PMR should not bypass the semantics of daifflags.

When DA_F are set, I bit is also set as no interrupts (even of higher
priority) is allowed.

When DA_F are cleared, I bit is cleared and interrupt enabling/disabling
goes through ICC_PMR_EL1.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
---
 arch/arm64/include/asm/daifflags.h | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h
index 546bc39..1fd390e 100644
--- a/arch/arm64/include/asm/daifflags.h
+++ b/arch/arm64/include/asm/daifflags.h
@@ -18,6 +18,8 @@
 
 #include <linux/irqflags.h>
 
+#include <asm/cpufeature.h>
+
 #define DAIF_PROCCTX		0
 #define DAIF_PROCCTX_NOIRQ	PSR_I_BIT
 
@@ -36,7 +38,13 @@ static inline unsigned long local_daif_save(void)
 {
 	unsigned long flags;
 
-	flags = arch_local_save_flags();
+	flags = read_sysreg(daif);
+
+	if (system_uses_irq_prio_masking()) {
+		/* If IRQs are masked with PMR, reflect it in the flags */
+		if (read_sysreg_s(SYS_ICC_PMR_EL1) <= GIC_PRIO_IRQOFF)
+			flags |= PSR_I_BIT;
+	}
 
 	local_daif_mask();
 
@@ -45,12 +53,27 @@ static inline unsigned long local_daif_save(void)
 
 static inline void local_daif_restore(unsigned long flags)
 {
-	if (!arch_irqs_disabled_flags(flags))
+	bool irq_disabled = flags & PSR_I_BIT;
+
+	if (!irq_disabled) {
 		trace_hardirqs_on();
 
-	arch_local_irq_restore(flags);
+		if (system_uses_irq_prio_masking())
+			arch_local_irq_enable();
+	} else if (!(flags & PSR_A_BIT)) {
+		/*
+		 * If interrupts are disabled but we can take
+		 * asynchronous errors, we can take NMIs
+		 */
+		if (system_uses_irq_prio_masking()) {
+			flags &= ~PSR_I_BIT;
+			arch_local_irq_disable();
+		}
+	}
+
+	write_sysreg(flags, daif);
 
-	if (arch_irqs_disabled_flags(flags))
+	if (irq_disabled)
 		trace_hardirqs_off();
 }
 
-- 
1.9.1

[PATCH v8 13/26] arm64: daifflags: Include PMR in daifflags restore operations

[Julien Thierry]

The addition of PMR should not bypass the semantics of daifflags.

When DA_F are set, I bit is also set as no interrupts (even of higher
priority) is allowed.

When DA_F are cleared, I bit is cleared and interrupt enabling/disabling
goes through ICC_PMR_EL1.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
---
 arch/arm64/include/asm/daifflags.h | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h
index 546bc39..1fd390e 100644
--- a/arch/arm64/include/asm/daifflags.h
+++ b/arch/arm64/include/asm/daifflags.h
@@ -18,6 +18,8 @@
 
 #include <linux/irqflags.h>
 
+#include <asm/cpufeature.h>
+
 #define DAIF_PROCCTX		0
 #define DAIF_PROCCTX_NOIRQ	PSR_I_BIT
 
@@ -36,7 +38,13 @@ static inline unsigned long local_daif_save(void)
 {
 	unsigned long flags;
 
-	flags = arch_local_save_flags();
+	flags = read_sysreg(daif);
+
+	if (system_uses_irq_prio_masking()) {
+		/* If IRQs are masked with PMR, reflect it in the flags */
+		if (read_sysreg_s(SYS_ICC_PMR_EL1) <= GIC_PRIO_IRQOFF)
+			flags |= PSR_I_BIT;
+	}
 
 	local_daif_mask();
 
@@ -45,12 +53,27 @@ static inline unsigned long local_daif_save(void)
 
 static inline void local_daif_restore(unsigned long flags)
 {
-	if (!arch_irqs_disabled_flags(flags))
+	bool irq_disabled = flags & PSR_I_BIT;
+
+	if (!irq_disabled) {
 		trace_hardirqs_on();
 
-	arch_local_irq_restore(flags);
+		if (system_uses_irq_prio_masking())
+			arch_local_irq_enable();
+	} else if (!(flags & PSR_A_BIT)) {
+		/*
+		 * If interrupts are disabled but we can take
+		 * asynchronous errors, we can take NMIs
+		 */
+		if (system_uses_irq_prio_masking()) {
+			flags &= ~PSR_I_BIT;
+			arch_local_irq_disable();
+		}
+	}
+
+	write_sysreg(flags, daif);
 
-	if (arch_irqs_disabled_flags(flags))
+	if (irq_disabled)
 		trace_hardirqs_off();
 }
 
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 14/26] arm64: alternative: Allow alternative status checking per cpufeature

[Julien Thierry]

In preparation for the application of alternatives at different points
during the boot process, provide the possibility to check whether
alternatives for a feature of interest was already applied instead of
having a global boolean for all alternatives.

Make VHE enablement code check for the VHE feature instead of considering
all alternatives.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Cc: Christoffer Dall <Christoffer.Dall@arm.com>
---
 arch/arm64/include/asm/alternative.h |  3 +--
 arch/arm64/kernel/alternative.c      | 21 +++++++++++++++++----
 arch/arm64/kernel/cpufeature.c       |  2 +-
 3 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h
index 4b650ec..9806a23 100644
--- a/arch/arm64/include/asm/alternative.h
+++ b/arch/arm64/include/asm/alternative.h
@@ -14,8 +14,6 @@
 #include <linux/stddef.h>
 #include <linux/stringify.h>
 
-extern int alternatives_applied;
-
 struct alt_instr {
 	s32 orig_offset;	/* offset to original instruction */
 	s32 alt_offset;		/* offset to replacement instruction */
@@ -28,6 +26,7 @@ typedef void (*alternative_cb_t)(struct alt_instr *alt,
 				 __le32 *origptr, __le32 *updptr, int nr_inst);
 
 void __init apply_alternatives_all(void);
+bool alternative_is_applied(u16 cpufeature);
 
 #ifdef CONFIG_MODULES
 void apply_alternatives_module(void *start, size_t length);
diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c
index b5d6039..c947d22 100644
--- a/arch/arm64/kernel/alternative.c
+++ b/arch/arm64/kernel/alternative.c
@@ -32,13 +32,23 @@
 #define ALT_ORIG_PTR(a)		__ALT_PTR(a, orig_offset)
 #define ALT_REPL_PTR(a)		__ALT_PTR(a, alt_offset)
 
-int alternatives_applied;
+static int all_alternatives_applied;
+
+static DECLARE_BITMAP(applied_alternatives, ARM64_NCAPS);
 
 struct alt_region {
 	struct alt_instr *begin;
 	struct alt_instr *end;
 };
 
+bool alternative_is_applied(u16 cpufeature)
+{
+	if (WARN_ON(cpufeature >= ARM64_NCAPS))
+		return false;
+
+	return test_bit(cpufeature, applied_alternatives);
+}
+
 /*
  * Check if the target PC is within an alternative block.
  */
@@ -192,6 +202,9 @@ static void __apply_alternatives(void *alt_region, bool is_module)
 		dsb(ish);
 		__flush_icache_all();
 		isb();
+
+		/* We applied all that was available */
+		bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
 	}
 }
 
@@ -208,14 +221,14 @@ static int __apply_alternatives_multi_stop(void *unused)
 
 	/* We always have a CPU 0 at this point (__init) */
 	if (smp_processor_id()) {
-		while (!READ_ONCE(alternatives_applied))
+		while (!READ_ONCE(all_alternatives_applied))
 			cpu_relax();
 		isb();
 	} else {
-		BUG_ON(alternatives_applied);
+		BUG_ON(all_alternatives_applied);
 		__apply_alternatives(&region, false);
 		/* Barriers provided by the cache flushing */
-		WRITE_ONCE(alternatives_applied, 1);
+		WRITE_ONCE(all_alternatives_applied, 1);
 	}
 
 	return 0;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c426302..84fa5be 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1113,7 +1113,7 @@ static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
 	 * that, freshly-onlined CPUs will set tpidr_el2, so we don't need to
 	 * do anything here.
 	 */
-	if (!alternatives_applied)
+	if (!alternative_is_applied(ARM64_HAS_VIRT_HOST_EXTN))
 		write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
 }
 #endif
-- 
1.9.1

[PATCH v8 14/26] arm64: alternative: Allow alternative status checking per cpufeature

[Julien Thierry]

In preparation for the application of alternatives at different points
during the boot process, provide the possibility to check whether
alternatives for a feature of interest was already applied instead of
having a global boolean for all alternatives.

Make VHE enablement code check for the VHE feature instead of considering
all alternatives.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Cc: Christoffer Dall <Christoffer.Dall@arm.com>
---
 arch/arm64/include/asm/alternative.h |  3 +--
 arch/arm64/kernel/alternative.c      | 21 +++++++++++++++++----
 arch/arm64/kernel/cpufeature.c       |  2 +-
 3 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h
index 4b650ec..9806a23 100644
--- a/arch/arm64/include/asm/alternative.h
+++ b/arch/arm64/include/asm/alternative.h
@@ -14,8 +14,6 @@
 #include <linux/stddef.h>
 #include <linux/stringify.h>
 
-extern int alternatives_applied;
-
 struct alt_instr {
 	s32 orig_offset;	/* offset to original instruction */
 	s32 alt_offset;		/* offset to replacement instruction */
@@ -28,6 +26,7 @@ typedef void (*alternative_cb_t)(struct alt_instr *alt,
 				 __le32 *origptr, __le32 *updptr, int nr_inst);
 
 void __init apply_alternatives_all(void);
+bool alternative_is_applied(u16 cpufeature);
 
 #ifdef CONFIG_MODULES
 void apply_alternatives_module(void *start, size_t length);
diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c
index b5d6039..c947d22 100644
--- a/arch/arm64/kernel/alternative.c
+++ b/arch/arm64/kernel/alternative.c
@@ -32,13 +32,23 @@
 #define ALT_ORIG_PTR(a)		__ALT_PTR(a, orig_offset)
 #define ALT_REPL_PTR(a)		__ALT_PTR(a, alt_offset)
 
-int alternatives_applied;
+static int all_alternatives_applied;
+
+static DECLARE_BITMAP(applied_alternatives, ARM64_NCAPS);
 
 struct alt_region {
 	struct alt_instr *begin;
 	struct alt_instr *end;
 };
 
+bool alternative_is_applied(u16 cpufeature)
+{
+	if (WARN_ON(cpufeature >= ARM64_NCAPS))
+		return false;
+
+	return test_bit(cpufeature, applied_alternatives);
+}
+
 /*
  * Check if the target PC is within an alternative block.
  */
@@ -192,6 +202,9 @@ static void __apply_alternatives(void *alt_region, bool is_module)
 		dsb(ish);
 		__flush_icache_all();
 		isb();
+
+		/* We applied all that was available */
+		bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
 	}
 }
 
@@ -208,14 +221,14 @@ static int __apply_alternatives_multi_stop(void *unused)
 
 	/* We always have a CPU 0 at this point (__init) */
 	if (smp_processor_id()) {
-		while (!READ_ONCE(alternatives_applied))
+		while (!READ_ONCE(all_alternatives_applied))
 			cpu_relax();
 		isb();
 	} else {
-		BUG_ON(alternatives_applied);
+		BUG_ON(all_alternatives_applied);
 		__apply_alternatives(&region, false);
 		/* Barriers provided by the cache flushing */
-		WRITE_ONCE(alternatives_applied, 1);
+		WRITE_ONCE(all_alternatives_applied, 1);
 	}
 
 	return 0;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c426302..84fa5be 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1113,7 +1113,7 @@ static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
 	 * that, freshly-onlined CPUs will set tpidr_el2, so we don't need to
 	 * do anything here.
 	 */
-	if (!alternatives_applied)
+	if (!alternative_is_applied(ARM64_HAS_VIRT_HOST_EXTN))
 		write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
 }
 #endif
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Julien Thierry]

From: Daniel Thompson <daniel.thompson@linaro.org>

Currently alternatives are applied very late in the boot process (and
a long time after we enable scheduling). Some alternative sequences,
such as those that alter the way CPU context is stored, must be applied
much earlier in the boot sequence.

Introduce apply_boot_alternatives() to allow some alternatives to be
applied immediately after we detect the CPU features of the boot CPU.

Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
[julien.thierry@arm.com: rename to fit new cpufeature framework better,
			 apply BOOT_SCOPE feature early in boot]
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Christoffer Dall <christoffer.dall@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/alternative.h |  1 +
 arch/arm64/include/asm/cpufeature.h  |  4 ++++
 arch/arm64/kernel/alternative.c      | 43 +++++++++++++++++++++++++++++++-----
 arch/arm64/kernel/cpufeature.c       |  6 +++++
 arch/arm64/kernel/smp.c              |  7 ++++++
 5 files changed, 56 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h
index 9806a23..b9f8d78 100644
--- a/arch/arm64/include/asm/alternative.h
+++ b/arch/arm64/include/asm/alternative.h
@@ -25,6 +25,7 @@ struct alt_instr {
 typedef void (*alternative_cb_t)(struct alt_instr *alt,
 				 __le32 *origptr, __le32 *updptr, int nr_inst);
 
+void __init apply_boot_alternatives(void);
 void __init apply_alternatives_all(void);
 bool alternative_is_applied(u16 cpufeature);
 
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 89c3f31..e505e1f 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
 extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
 extern struct static_key_false arm64_const_caps_ready;
 
+/* ARM64 CAPS + alternative_cb */
+#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
+extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
+
 #define for_each_available_cap(cap)		\
 	for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
 
diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c
index c947d22..a9b4677 100644
--- a/arch/arm64/kernel/alternative.c
+++ b/arch/arm64/kernel/alternative.c
@@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start, u64 end)
 	} while (cur += d_size, cur < end);
 }
 
-static void __apply_alternatives(void *alt_region, bool is_module)
+static void __apply_alternatives(void *alt_region,  bool is_module,
+				 unsigned long *feature_mask)
 {
 	struct alt_instr *alt;
 	struct alt_region *region = alt_region;
@@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region, bool is_module)
 	for (alt = region->begin; alt < region->end; alt++) {
 		int nr_inst;
 
+		if (!test_bit(alt->cpufeature, feature_mask))
+			continue;
+
 		/* Use ARM64_CB_PATCH as an unconditional patch */
 		if (alt->cpufeature < ARM64_CB_PATCH &&
 		    !cpus_have_cap(alt->cpufeature))
@@ -203,8 +207,11 @@ static void __apply_alternatives(void *alt_region, bool is_module)
 		__flush_icache_all();
 		isb();
 
-		/* We applied all that was available */
-		bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
+		/* Ignore ARM64_CB bit from feature mask */
+		bitmap_or(applied_alternatives, applied_alternatives,
+			  feature_mask, ARM64_NCAPS);
+		bitmap_and(applied_alternatives, applied_alternatives,
+			   cpu_hwcaps, ARM64_NCAPS);
 	}
 }
 
@@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void *unused)
 			cpu_relax();
 		isb();
 	} else {
+		DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
+
+		bitmap_complement(remaining_capabilities, boot_capabilities,
+				  ARM64_NPATCHABLE);
+
 		BUG_ON(all_alternatives_applied);
-		__apply_alternatives(&region, false);
+		__apply_alternatives(&region, false, remaining_capabilities);
 		/* Barriers provided by the cache flushing */
 		WRITE_ONCE(all_alternatives_applied, 1);
 	}
@@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
 	stop_machine(__apply_alternatives_multi_stop, NULL, cpu_online_mask);
 }
 
+/*
+ * This is called very early in the boot process (directly after we run
+ * a feature detect on the boot CPU). No need to worry about other CPUs
+ * here.
+ */
+void __init apply_boot_alternatives(void)
+{
+	struct alt_region region = {
+		.begin	= (struct alt_instr *)__alt_instructions,
+		.end	= (struct alt_instr *)__alt_instructions_end,
+	};
+
+	/* If called on non-boot cpu things could go wrong */
+	WARN_ON(smp_processor_id() != 0);
+
+	__apply_alternatives(&region, false, &boot_capabilities[0]);
+}
+
 #ifdef CONFIG_MODULES
 void apply_alternatives_module(void *start, size_t length)
 {
@@ -247,7 +277,10 @@ void apply_alternatives_module(void *start, size_t length)
 		.begin	= start,
 		.end	= start + length,
 	};
+	DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
+
+	bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
 
-	__apply_alternatives(&region, true);
+	__apply_alternatives(&region, true, &all_capabilities[0]);
 }
 #endif
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 84fa5be..71c8d4f 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -54,6 +54,9 @@
 EXPORT_SYMBOL(cpu_hwcaps);
 static struct arm64_cpu_capabilities const __ro_after_init *cpu_hwcaps_ptrs[ARM64_NCAPS];
 
+/* Need also bit for ARM64_CB_PATCH */
+DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
+
 /*
  * Flag to indicate if we have computed the system wide
  * capabilities based on the boot time active CPUs. This
@@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16 scope_mask)
 		if (caps->desc)
 			pr_info("detected: %s\n", caps->desc);
 		cpus_set_cap(caps->capability);
+
+		if (caps->type & SCOPE_BOOT_CPU)
+			set_bit(caps->capability, boot_capabilities);
 	}
 }
 
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 1598d6f..a944edd 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -419,6 +419,13 @@ void __init smp_prepare_boot_cpu(void)
 	 */
 	jump_label_init();
 	cpuinfo_store_boot_cpu();
+
+	/*
+	 * We now know enough about the boot CPU to apply the
+	 * alternatives that cannot wait until interrupt handling
+	 * and/or scheduling is enabled.
+	 */
+	apply_boot_alternatives();
 }
 
 static u64 __init of_get_cpu_mpidr(struct device_node *dn)
-- 
1.9.1

[PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Julien Thierry]

From: Daniel Thompson <daniel.thompson@linaro.org>

Currently alternatives are applied very late in the boot process (and
a long time after we enable scheduling). Some alternative sequences,
such as those that alter the way CPU context is stored, must be applied
much earlier in the boot sequence.

Introduce apply_boot_alternatives() to allow some alternatives to be
applied immediately after we detect the CPU features of the boot CPU.

Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
[julien.thierry@arm.com: rename to fit new cpufeature framework better,
			 apply BOOT_SCOPE feature early in boot]
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Christoffer Dall <christoffer.dall@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/alternative.h |  1 +
 arch/arm64/include/asm/cpufeature.h  |  4 ++++
 arch/arm64/kernel/alternative.c      | 43 +++++++++++++++++++++++++++++++-----
 arch/arm64/kernel/cpufeature.c       |  6 +++++
 arch/arm64/kernel/smp.c              |  7 ++++++
 5 files changed, 56 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h
index 9806a23..b9f8d78 100644
--- a/arch/arm64/include/asm/alternative.h
+++ b/arch/arm64/include/asm/alternative.h
@@ -25,6 +25,7 @@ struct alt_instr {
 typedef void (*alternative_cb_t)(struct alt_instr *alt,
 				 __le32 *origptr, __le32 *updptr, int nr_inst);
 
+void __init apply_boot_alternatives(void);
 void __init apply_alternatives_all(void);
 bool alternative_is_applied(u16 cpufeature);
 
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 89c3f31..e505e1f 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
 extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
 extern struct static_key_false arm64_const_caps_ready;
 
+/* ARM64 CAPS + alternative_cb */
+#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
+extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
+
 #define for_each_available_cap(cap)		\
 	for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
 
diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c
index c947d22..a9b4677 100644
--- a/arch/arm64/kernel/alternative.c
+++ b/arch/arm64/kernel/alternative.c
@@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start, u64 end)
 	} while (cur += d_size, cur < end);
 }
 
-static void __apply_alternatives(void *alt_region, bool is_module)
+static void __apply_alternatives(void *alt_region,  bool is_module,
+				 unsigned long *feature_mask)
 {
 	struct alt_instr *alt;
 	struct alt_region *region = alt_region;
@@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region, bool is_module)
 	for (alt = region->begin; alt < region->end; alt++) {
 		int nr_inst;
 
+		if (!test_bit(alt->cpufeature, feature_mask))
+			continue;
+
 		/* Use ARM64_CB_PATCH as an unconditional patch */
 		if (alt->cpufeature < ARM64_CB_PATCH &&
 		    !cpus_have_cap(alt->cpufeature))
@@ -203,8 +207,11 @@ static void __apply_alternatives(void *alt_region, bool is_module)
 		__flush_icache_all();
 		isb();
 
-		/* We applied all that was available */
-		bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
+		/* Ignore ARM64_CB bit from feature mask */
+		bitmap_or(applied_alternatives, applied_alternatives,
+			  feature_mask, ARM64_NCAPS);
+		bitmap_and(applied_alternatives, applied_alternatives,
+			   cpu_hwcaps, ARM64_NCAPS);
 	}
 }
 
@@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void *unused)
 			cpu_relax();
 		isb();
 	} else {
+		DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
+
+		bitmap_complement(remaining_capabilities, boot_capabilities,
+				  ARM64_NPATCHABLE);
+
 		BUG_ON(all_alternatives_applied);
-		__apply_alternatives(&region, false);
+		__apply_alternatives(&region, false, remaining_capabilities);
 		/* Barriers provided by the cache flushing */
 		WRITE_ONCE(all_alternatives_applied, 1);
 	}
@@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
 	stop_machine(__apply_alternatives_multi_stop, NULL, cpu_online_mask);
 }
 
+/*
+ * This is called very early in the boot process (directly after we run
+ * a feature detect on the boot CPU). No need to worry about other CPUs
+ * here.
+ */
+void __init apply_boot_alternatives(void)
+{
+	struct alt_region region = {
+		.begin	= (struct alt_instr *)__alt_instructions,
+		.end	= (struct alt_instr *)__alt_instructions_end,
+	};
+
+	/* If called on non-boot cpu things could go wrong */
+	WARN_ON(smp_processor_id() != 0);
+
+	__apply_alternatives(&region, false, &boot_capabilities[0]);
+}
+
 #ifdef CONFIG_MODULES
 void apply_alternatives_module(void *start, size_t length)
 {
@@ -247,7 +277,10 @@ void apply_alternatives_module(void *start, size_t length)
 		.begin	= start,
 		.end	= start + length,
 	};
+	DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
+
+	bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
 
-	__apply_alternatives(&region, true);
+	__apply_alternatives(&region, true, &all_capabilities[0]);
 }
 #endif
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 84fa5be..71c8d4f 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -54,6 +54,9 @@
 EXPORT_SYMBOL(cpu_hwcaps);
 static struct arm64_cpu_capabilities const __ro_after_init *cpu_hwcaps_ptrs[ARM64_NCAPS];
 
+/* Need also bit for ARM64_CB_PATCH */
+DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
+
 /*
  * Flag to indicate if we have computed the system wide
  * capabilities based on the boot time active CPUs. This
@@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16 scope_mask)
 		if (caps->desc)
 			pr_info("detected: %s\n", caps->desc);
 		cpus_set_cap(caps->capability);
+
+		if (caps->type & SCOPE_BOOT_CPU)
+			set_bit(caps->capability, boot_capabilities);
 	}
 }
 
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 1598d6f..a944edd 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -419,6 +419,13 @@ void __init smp_prepare_boot_cpu(void)
 	 */
 	jump_label_init();
 	cpuinfo_store_boot_cpu();
+
+	/*
+	 * We now know enough about the boot CPU to apply the
+	 * alternatives that cannot wait until interrupt handling
+	 * and/or scheduling is enabled.
+	 */
+	apply_boot_alternatives();
 }
 
 static u64 __init of_get_cpu_mpidr(struct device_node *dn)
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 16/26] irqchip/gic-v3: Factor group0 detection into functions

[Julien Thierry]

The code to detect whether Linux has access to group0 interrupts can
prove useful in other parts of the driver.

Provide a separate function to do this.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 55 +++++++++++++++++++++++++++++---------------
 1 file changed, 36 insertions(+), 19 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 8148a92..da547e0 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -400,6 +400,39 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 	}
 }
 
+static u32 gic_get_pribits(void)
+{
+	u32 pribits;
+
+	pribits = gic_read_ctlr();
+	pribits &= ICC_CTLR_EL1_PRI_BITS_MASK;
+	pribits >>= ICC_CTLR_EL1_PRI_BITS_SHIFT;
+	pribits++;
+
+	return pribits;
+}
+
+static bool gic_has_group0(void)
+{
+	u32 val;
+
+	/*
+	 * Let's find out if Group0 is under control of EL3 or not by
+	 * setting the highest possible, non-zero priority in PMR.
+	 *
+	 * If SCR_EL3.FIQ is set, the priority gets shifted down in
+	 * order for the CPU interface to set bit 7, and keep the
+	 * actual priority in the non-secure range. In the process, it
+	 * looses the least significant bit and the actual priority
+	 * becomes 0x80. Reading it back returns 0, indicating that
+	 * we're don't have access to Group0.
+	 */
+	gic_write_pmr(BIT(8 - gic_get_pribits()));
+	val = gic_read_pmr();
+
+	return val != 0;
+}
+
 static void __init gic_dist_init(void)
 {
 	unsigned int i;
@@ -541,7 +574,7 @@ static void gic_cpu_sys_reg_init(void)
 	u64 mpidr = cpu_logical_map(cpu);
 	u64 need_rss = MPIDR_RS(mpidr);
 	bool group0;
-	u32 val, pribits;
+	u32 pribits;
 
 	/*
 	 * Need to check that the SRE bit has actually been set. If
@@ -553,25 +586,9 @@ static void gic_cpu_sys_reg_init(void)
 	if (!gic_enable_sre())
 		pr_err("GIC: unable to set SRE (disabled at EL2), panic ahead\n");
 
-	pribits = gic_read_ctlr();
-	pribits &= ICC_CTLR_EL1_PRI_BITS_MASK;
-	pribits >>= ICC_CTLR_EL1_PRI_BITS_SHIFT;
-	pribits++;
+	pribits = gic_get_pribits();
 
-	/*
-	 * Let's find out if Group0 is under control of EL3 or not by
-	 * setting the highest possible, non-zero priority in PMR.
-	 *
-	 * If SCR_EL3.FIQ is set, the priority gets shifted down in
-	 * order for the CPU interface to set bit 7, and keep the
-	 * actual priority in the non-secure range. In the process, it
-	 * looses the least significant bit and the actual priority
-	 * becomes 0x80. Reading it back returns 0, indicating that
-	 * we're don't have access to Group0.
-	 */
-	write_gicreg(BIT(8 - pribits), ICC_PMR_EL1);
-	val = read_gicreg(ICC_PMR_EL1);
-	group0 = val != 0;
+	group0 = gic_has_group0();
 
 	/* Set priority mask register */
 	write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
-- 
1.9.1

[PATCH v8 16/26] irqchip/gic-v3: Factor group0 detection into functions

[Julien Thierry]

The code to detect whether Linux has access to group0 interrupts can
prove useful in other parts of the driver.

Provide a separate function to do this.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 55 +++++++++++++++++++++++++++++---------------
 1 file changed, 36 insertions(+), 19 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 8148a92..da547e0 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -400,6 +400,39 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 	}
 }
 
+static u32 gic_get_pribits(void)
+{
+	u32 pribits;
+
+	pribits = gic_read_ctlr();
+	pribits &= ICC_CTLR_EL1_PRI_BITS_MASK;
+	pribits >>= ICC_CTLR_EL1_PRI_BITS_SHIFT;
+	pribits++;
+
+	return pribits;
+}
+
+static bool gic_has_group0(void)
+{
+	u32 val;
+
+	/*
+	 * Let's find out if Group0 is under control of EL3 or not by
+	 * setting the highest possible, non-zero priority in PMR.
+	 *
+	 * If SCR_EL3.FIQ is set, the priority gets shifted down in
+	 * order for the CPU interface to set bit 7, and keep the
+	 * actual priority in the non-secure range. In the process, it
+	 * looses the least significant bit and the actual priority
+	 * becomes 0x80. Reading it back returns 0, indicating that
+	 * we're don't have access to Group0.
+	 */
+	gic_write_pmr(BIT(8 - gic_get_pribits()));
+	val = gic_read_pmr();
+
+	return val != 0;
+}
+
 static void __init gic_dist_init(void)
 {
 	unsigned int i;
@@ -541,7 +574,7 @@ static void gic_cpu_sys_reg_init(void)
 	u64 mpidr = cpu_logical_map(cpu);
 	u64 need_rss = MPIDR_RS(mpidr);
 	bool group0;
-	u32 val, pribits;
+	u32 pribits;
 
 	/*
 	 * Need to check that the SRE bit has actually been set. If
@@ -553,25 +586,9 @@ static void gic_cpu_sys_reg_init(void)
 	if (!gic_enable_sre())
 		pr_err("GIC: unable to set SRE (disabled at EL2), panic ahead\n");
 
-	pribits = gic_read_ctlr();
-	pribits &= ICC_CTLR_EL1_PRI_BITS_MASK;
-	pribits >>= ICC_CTLR_EL1_PRI_BITS_SHIFT;
-	pribits++;
+	pribits = gic_get_pribits();
 
-	/*
-	 * Let's find out if Group0 is under control of EL3 or not by
-	 * setting the highest possible, non-zero priority in PMR.
-	 *
-	 * If SCR_EL3.FIQ is set, the priority gets shifted down in
-	 * order for the CPU interface to set bit 7, and keep the
-	 * actual priority in the non-secure range. In the process, it
-	 * looses the least significant bit and the actual priority
-	 * becomes 0x80. Reading it back returns 0, indicating that
-	 * we're don't have access to Group0.
-	 */
-	write_gicreg(BIT(8 - pribits), ICC_PMR_EL1);
-	val = read_gicreg(ICC_PMR_EL1);
-	group0 = val != 0;
+	group0 = gic_has_group0();
 
 	/* Set priority mask register */
 	write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 17/26] arm64: Switch to PMR masking when starting CPUs

[Julien Thierry]

Once the boot CPU has been prepared or a new secondary CPU has been
brought up, use ICC_PMR_EL1 to mask interrupts on that CPU and clear
PSR.I bit.

Since ICC_PMR_EL1 is initialized at CPU bringup, avoid overwriting
it in the GICv3 driver.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/smp.c      | 26 ++++++++++++++++++++++++++
 drivers/irqchip/irq-gic-v3.c |  8 +++++++-
 2 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index a944edd..824de70 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -35,6 +35,7 @@
 #include <linux/smp.h>
 #include <linux/seq_file.h>
 #include <linux/irq.h>
+#include <linux/irqchip/arm-gic-v3.h>
 #include <linux/percpu.h>
 #include <linux/clockchips.h>
 #include <linux/completion.h>
@@ -180,6 +181,24 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle)
 	return ret;
 }
 
+static void init_gic_priority_masking(void)
+{
+	u32 cpuflags;
+
+	if (WARN_ON(!gic_enable_sre()))
+		return;
+
+	cpuflags = read_sysreg(daif);
+
+	WARN_ON(!(cpuflags & PSR_I_BIT));
+
+	gic_write_pmr(GIC_PRIO_IRQOFF);
+
+	/* We can only unmask PSR.I if we can take aborts */
+	if (!(cpuflags & PSR_A_BIT))
+		write_sysreg(cpuflags & ~PSR_I_BIT, daif);
+}
+
 /*
  * This is the secondary CPU boot entry.  We're using this CPUs
  * idle thread stack, but a set of temporary page tables.
@@ -206,6 +225,9 @@ asmlinkage notrace void secondary_start_kernel(void)
 	 */
 	cpu_uninstall_idmap();
 
+	if (system_uses_irq_prio_masking())
+		init_gic_priority_masking();
+
 	preempt_disable();
 	trace_hardirqs_off();
 
@@ -426,6 +448,10 @@ void __init smp_prepare_boot_cpu(void)
 	 * and/or scheduling is enabled.
 	 */
 	apply_boot_alternatives();
+
+	/* Conditionally switch to GIC PMR for interrupt masking */
+	if (system_uses_irq_prio_masking())
+		init_gic_priority_masking();
 }
 
 static u64 __init of_get_cpu_mpidr(struct device_node *dn)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index da547e0..5a703ae 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -415,6 +415,9 @@ static u32 gic_get_pribits(void)
 static bool gic_has_group0(void)
 {
 	u32 val;
+	u32 old_pmr;
+
+	old_pmr = gic_read_pmr();
 
 	/*
 	 * Let's find out if Group0 is under control of EL3 or not by
@@ -430,6 +433,8 @@ static bool gic_has_group0(void)
 	gic_write_pmr(BIT(8 - gic_get_pribits()));
 	val = gic_read_pmr();
 
+	gic_write_pmr(old_pmr);
+
 	return val != 0;
 }
 
@@ -591,7 +596,8 @@ static void gic_cpu_sys_reg_init(void)
 	group0 = gic_has_group0();
 
 	/* Set priority mask register */
-	write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
+	if (!gic_prio_masking_enabled())
+		write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
 
 	/*
 	 * Some firmwares hand over to the kernel with the BPR changed from
-- 
1.9.1

[PATCH v8 17/26] arm64: Switch to PMR masking when starting CPUs

[Julien Thierry]

Once the boot CPU has been prepared or a new secondary CPU has been
brought up, use ICC_PMR_EL1 to mask interrupts on that CPU and clear
PSR.I bit.

Since ICC_PMR_EL1 is initialized at CPU bringup, avoid overwriting
it in the GICv3 driver.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/smp.c      | 26 ++++++++++++++++++++++++++
 drivers/irqchip/irq-gic-v3.c |  8 +++++++-
 2 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index a944edd..824de70 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -35,6 +35,7 @@
 #include <linux/smp.h>
 #include <linux/seq_file.h>
 #include <linux/irq.h>
+#include <linux/irqchip/arm-gic-v3.h>
 #include <linux/percpu.h>
 #include <linux/clockchips.h>
 #include <linux/completion.h>
@@ -180,6 +181,24 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle)
 	return ret;
 }
 
+static void init_gic_priority_masking(void)
+{
+	u32 cpuflags;
+
+	if (WARN_ON(!gic_enable_sre()))
+		return;
+
+	cpuflags = read_sysreg(daif);
+
+	WARN_ON(!(cpuflags & PSR_I_BIT));
+
+	gic_write_pmr(GIC_PRIO_IRQOFF);
+
+	/* We can only unmask PSR.I if we can take aborts */
+	if (!(cpuflags & PSR_A_BIT))
+		write_sysreg(cpuflags & ~PSR_I_BIT, daif);
+}
+
 /*
  * This is the secondary CPU boot entry.  We're using this CPUs
  * idle thread stack, but a set of temporary page tables.
@@ -206,6 +225,9 @@ asmlinkage notrace void secondary_start_kernel(void)
 	 */
 	cpu_uninstall_idmap();
 
+	if (system_uses_irq_prio_masking())
+		init_gic_priority_masking();
+
 	preempt_disable();
 	trace_hardirqs_off();
 
@@ -426,6 +448,10 @@ void __init smp_prepare_boot_cpu(void)
 	 * and/or scheduling is enabled.
 	 */
 	apply_boot_alternatives();
+
+	/* Conditionally switch to GIC PMR for interrupt masking */
+	if (system_uses_irq_prio_masking())
+		init_gic_priority_masking();
 }
 
 static u64 __init of_get_cpu_mpidr(struct device_node *dn)
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index da547e0..5a703ae 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -415,6 +415,9 @@ static u32 gic_get_pribits(void)
 static bool gic_has_group0(void)
 {
 	u32 val;
+	u32 old_pmr;
+
+	old_pmr = gic_read_pmr();
 
 	/*
 	 * Let's find out if Group0 is under control of EL3 or not by
@@ -430,6 +433,8 @@ static bool gic_has_group0(void)
 	gic_write_pmr(BIT(8 - gic_get_pribits()));
 	val = gic_read_pmr();
 
+	gic_write_pmr(old_pmr);
+
 	return val != 0;
 }
 
@@ -591,7 +596,8 @@ static void gic_cpu_sys_reg_init(void)
 	group0 = gic_has_group0();
 
 	/* Set priority mask register */
-	write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
+	if (!gic_prio_masking_enabled())
+		write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
 
 	/*
 	 * Some firmwares hand over to the kernel with the BPR changed from
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 18/26] arm64: gic-v3: Implement arch support for priority masking

[Julien Thierry]

Implement architecture specific primitive allowing the GICv3 driver to
use priorities to mask interrupts.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 arch/arm64/include/asm/arch_gicv3.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index b5f8142..14b41dd 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -22,6 +22,7 @@
 
 #ifndef __ASSEMBLY__
 
+#include <linux/irqchip/arm-gic-common.h>
 #include <linux/stringify.h>
 #include <asm/barrier.h>
 #include <asm/cacheflush.h>
@@ -162,14 +163,13 @@ static inline bool gic_prio_masking_enabled(void)
 
 static inline void gic_pmr_mask_irqs(void)
 {
-	/* Should not get called yet. */
-	WARN_ON_ONCE(true);
+	BUILD_BUG_ON(GICD_INT_DEF_PRI <= GIC_PRIO_IRQOFF);
+	gic_write_pmr(GIC_PRIO_IRQOFF);
 }
 
 static inline void gic_arch_enable_irqs(void)
 {
-	/* Should not get called yet. */
-	WARN_ON_ONCE(true);
+	asm volatile ("msr daifclr, #2" : : : "memory");
 }
 
 #endif /* __ASSEMBLY__ */
-- 
1.9.1

[PATCH v8 18/26] arm64: gic-v3: Implement arch support for priority masking

[Julien Thierry]

Implement architecture specific primitive allowing the GICv3 driver to
use priorities to mask interrupts.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 arch/arm64/include/asm/arch_gicv3.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h
index b5f8142..14b41dd 100644
--- a/arch/arm64/include/asm/arch_gicv3.h
+++ b/arch/arm64/include/asm/arch_gicv3.h
@@ -22,6 +22,7 @@
 
 #ifndef __ASSEMBLY__
 
+#include <linux/irqchip/arm-gic-common.h>
 #include <linux/stringify.h>
 #include <asm/barrier.h>
 #include <asm/cacheflush.h>
@@ -162,14 +163,13 @@ static inline bool gic_prio_masking_enabled(void)
 
 static inline void gic_pmr_mask_irqs(void)
 {
-	/* Should not get called yet. */
-	WARN_ON_ONCE(true);
+	BUILD_BUG_ON(GICD_INT_DEF_PRI <= GIC_PRIO_IRQOFF);
+	gic_write_pmr(GIC_PRIO_IRQOFF);
 }
 
 static inline void gic_arch_enable_irqs(void)
 {
-	/* Should not get called yet. */
-	WARN_ON_ONCE(true);
+	asm volatile ("msr daifclr, #2" : : : "memory");
 }
 
 #endif /* __ASSEMBLY__ */
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 19/26] irqchip/gic-v3: Detect if GIC can support pseudo-NMIs

[Julien Thierry]

The values non secure EL1 needs to use for PMR and RPR registers depends on
the value of SCR_EL3.FIQ.

The values non secure EL1 sees from the distributor and redistributor
depend on whether security is enabled for the GIC or not.

To avoid having to deal with two sets of values for PMR
masking/unmasking, only enable pseudo-NMIs when GIC has non-secure view
of priorities.

Also, add firmware requirements related to SCR_EL3.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/arm64/booting.txt |  5 ++++
 drivers/irqchip/irq-gic-v3.c    | 58 ++++++++++++++++++++++++++++++++++++-----
 2 files changed, 57 insertions(+), 6 deletions(-)

diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 8df9f46..fbab7e2 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -188,6 +188,11 @@ Before jumping into the kernel, the following conditions must be met:
   the kernel image will be entered must be initialised by software at a
   higher exception level to prevent execution in an UNKNOWN state.
 
+  - SCR_EL3.FIQ must have the same value across all CPUs the kernel is
+    executing on.
+  - The value of SCR_EL3.FIQ must be the same as the one present at boot
+    time whenever the kernel is executing.
+
   For systems with a GICv3 interrupt controller to be used in v3 mode:
   - If EL3 is present:
     ICC_SRE_EL3.Enable (bit 3) must be initialiased to 0b1.
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 5a703ae..5374b43 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -66,6 +66,31 @@ struct gic_chip_data {
 static struct gic_chip_data gic_data __read_mostly;
 static DEFINE_STATIC_KEY_TRUE(supports_deactivate_key);
 
+/*
+ * The behaviours of RPR and PMR registers differ depending on the value of
+ * SCR_EL3.FIQ, and the behaviour of non-secure priority registers of the
+ * distributor and redistributors depends on whether security is enabled in the
+ * GIC.
+ *
+ * When security is enabled, non-secure priority values from the (re)distributor
+ * are presented to the GIC CPUIF as follow:
+ *     (GIC_(R)DIST_PRI[irq] >> 1) | 0x80;
+ *
+ * If SCR_EL3.FIQ == 1, the values writen to/read from PMR and RPR at non-secure
+ * EL1 are subject to a similar operation thus matching the priorities presented
+ * from the (re)distributor when security is enabled.
+ *
+ * see GICv3/GICv4 Architecture Specification (IHI0069D):
+ * - section 4.8.1 Non-secure accesses to register fields for Secure interrupt
+ *   priorities.
+ * - Figure 4-7 Secure read of the priority field for a Non-secure Group 1
+ *   interrupt.
+ *
+ * For now, we only support pseudo-NMIs if we have non-secure view of
+ * priorities.
+ */
+static DEFINE_STATIC_KEY_FALSE(supports_pseudo_nmis);
+
 static struct gic_kvm_info gic_v3_kvm_info;
 static DEFINE_PER_CPU(bool, has_rss);
 
@@ -232,6 +257,12 @@ static void gic_unmask_irq(struct irq_data *d)
 	gic_poke_irq(d, GICD_ISENABLER);
 }
 
+static inline bool gic_supports_nmi(void)
+{
+	return IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) &&
+	       static_branch_likely(&supports_pseudo_nmis);
+}
+
 static int gic_irq_set_irqchip_state(struct irq_data *d,
 				     enum irqchip_irq_state which, bool val)
 {
@@ -573,6 +604,12 @@ static void gic_update_vlpi_properties(void)
 		!gic_data.rdists.has_direct_lpi ? "no " : "");
 }
 
+/* Check whether it's single security state view */
+static inline bool gic_dist_security_disabled(void)
+{
+	return readl_relaxed(gic_data.dist_base + GICD_CTLR) & GICD_CTLR_DS;
+}
+
 static void gic_cpu_sys_reg_init(void)
 {
 	int i, cpu = smp_processor_id();
@@ -598,6 +635,9 @@ static void gic_cpu_sys_reg_init(void)
 	/* Set priority mask register */
 	if (!gic_prio_masking_enabled())
 		write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
+	else if (gic_supports_nmi() && group0)
+		/* Mismatch configuration with boot CPU */
+		WARN_ON(!gic_dist_security_disabled());
 
 	/*
 	 * Some firmwares hand over to the kernel with the BPR changed from
@@ -852,12 +892,6 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
 #endif
 
 #ifdef CONFIG_CPU_PM
-/* Check whether it's single security state view */
-static bool gic_dist_security_disabled(void)
-{
-	return readl_relaxed(gic_data.dist_base + GICD_CTLR) & GICD_CTLR_DS;
-}
-
 static int gic_cpu_pm_notifier(struct notifier_block *self,
 			       unsigned long cmd, void *v)
 {
@@ -1110,6 +1144,11 @@ static bool gic_enable_quirk_msm8996(void *data)
 	return true;
 }
 
+static void gic_enable_nmi_support(void)
+{
+	static_branch_enable(&supports_pseudo_nmis);
+}
+
 static int __init gic_init_bases(void __iomem *dist_base,
 				 struct redist_region *rdist_regs,
 				 u32 nr_redist_regions,
@@ -1179,6 +1218,13 @@ static int __init gic_init_bases(void __iomem *dist_base,
 		its_cpu_init();
 	}
 
+	if (gic_prio_masking_enabled()) {
+		if (!gic_has_group0() || gic_dist_security_disabled())
+			gic_enable_nmi_support();
+		else
+			pr_warn("SCR_EL3.FIQ is cleared, cannot enable use of pseudo-NMIs\n");
+	}
+
 	return 0;
 
 out_free:
-- 
1.9.1

[PATCH v8 19/26] irqchip/gic-v3: Detect if GIC can support pseudo-NMIs

[Julien Thierry]

The values non secure EL1 needs to use for PMR and RPR registers depends on
the value of SCR_EL3.FIQ.

The values non secure EL1 sees from the distributor and redistributor
depend on whether security is enabled for the GIC or not.

To avoid having to deal with two sets of values for PMR
masking/unmasking, only enable pseudo-NMIs when GIC has non-secure view
of priorities.

Also, add firmware requirements related to SCR_EL3.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/arm64/booting.txt |  5 ++++
 drivers/irqchip/irq-gic-v3.c    | 58 ++++++++++++++++++++++++++++++++++++-----
 2 files changed, 57 insertions(+), 6 deletions(-)

diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 8df9f46..fbab7e2 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -188,6 +188,11 @@ Before jumping into the kernel, the following conditions must be met:
   the kernel image will be entered must be initialised by software at a
   higher exception level to prevent execution in an UNKNOWN state.
 
+  - SCR_EL3.FIQ must have the same value across all CPUs the kernel is
+    executing on.
+  - The value of SCR_EL3.FIQ must be the same as the one present at boot
+    time whenever the kernel is executing.
+
   For systems with a GICv3 interrupt controller to be used in v3 mode:
   - If EL3 is present:
     ICC_SRE_EL3.Enable (bit 3) must be initialiased to 0b1.
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 5a703ae..5374b43 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -66,6 +66,31 @@ struct gic_chip_data {
 static struct gic_chip_data gic_data __read_mostly;
 static DEFINE_STATIC_KEY_TRUE(supports_deactivate_key);
 
+/*
+ * The behaviours of RPR and PMR registers differ depending on the value of
+ * SCR_EL3.FIQ, and the behaviour of non-secure priority registers of the
+ * distributor and redistributors depends on whether security is enabled in the
+ * GIC.
+ *
+ * When security is enabled, non-secure priority values from the (re)distributor
+ * are presented to the GIC CPUIF as follow:
+ *     (GIC_(R)DIST_PRI[irq] >> 1) | 0x80;
+ *
+ * If SCR_EL3.FIQ == 1, the values writen to/read from PMR and RPR at non-secure
+ * EL1 are subject to a similar operation thus matching the priorities presented
+ * from the (re)distributor when security is enabled.
+ *
+ * see GICv3/GICv4 Architecture Specification (IHI0069D):
+ * - section 4.8.1 Non-secure accesses to register fields for Secure interrupt
+ *   priorities.
+ * - Figure 4-7 Secure read of the priority field for a Non-secure Group 1
+ *   interrupt.
+ *
+ * For now, we only support pseudo-NMIs if we have non-secure view of
+ * priorities.
+ */
+static DEFINE_STATIC_KEY_FALSE(supports_pseudo_nmis);
+
 static struct gic_kvm_info gic_v3_kvm_info;
 static DEFINE_PER_CPU(bool, has_rss);
 
@@ -232,6 +257,12 @@ static void gic_unmask_irq(struct irq_data *d)
 	gic_poke_irq(d, GICD_ISENABLER);
 }
 
+static inline bool gic_supports_nmi(void)
+{
+	return IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) &&
+	       static_branch_likely(&supports_pseudo_nmis);
+}
+
 static int gic_irq_set_irqchip_state(struct irq_data *d,
 				     enum irqchip_irq_state which, bool val)
 {
@@ -573,6 +604,12 @@ static void gic_update_vlpi_properties(void)
 		!gic_data.rdists.has_direct_lpi ? "no " : "");
 }
 
+/* Check whether it's single security state view */
+static inline bool gic_dist_security_disabled(void)
+{
+	return readl_relaxed(gic_data.dist_base + GICD_CTLR) & GICD_CTLR_DS;
+}
+
 static void gic_cpu_sys_reg_init(void)
 {
 	int i, cpu = smp_processor_id();
@@ -598,6 +635,9 @@ static void gic_cpu_sys_reg_init(void)
 	/* Set priority mask register */
 	if (!gic_prio_masking_enabled())
 		write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1);
+	else if (gic_supports_nmi() && group0)
+		/* Mismatch configuration with boot CPU */
+		WARN_ON(!gic_dist_security_disabled());
 
 	/*
 	 * Some firmwares hand over to the kernel with the BPR changed from
@@ -852,12 +892,6 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
 #endif
 
 #ifdef CONFIG_CPU_PM
-/* Check whether it's single security state view */
-static bool gic_dist_security_disabled(void)
-{
-	return readl_relaxed(gic_data.dist_base + GICD_CTLR) & GICD_CTLR_DS;
-}
-
 static int gic_cpu_pm_notifier(struct notifier_block *self,
 			       unsigned long cmd, void *v)
 {
@@ -1110,6 +1144,11 @@ static bool gic_enable_quirk_msm8996(void *data)
 	return true;
 }
 
+static void gic_enable_nmi_support(void)
+{
+	static_branch_enable(&supports_pseudo_nmis);
+}
+
 static int __init gic_init_bases(void __iomem *dist_base,
 				 struct redist_region *rdist_regs,
 				 u32 nr_redist_regions,
@@ -1179,6 +1218,13 @@ static int __init gic_init_bases(void __iomem *dist_base,
 		its_cpu_init();
 	}
 
+	if (gic_prio_masking_enabled()) {
+		if (!gic_has_group0() || gic_dist_security_disabled())
+			gic_enable_nmi_support();
+		else
+			pr_warn("SCR_EL3.FIQ is cleared, cannot enable use of pseudo-NMIs\n");
+	}
+
 	return 0;
 
 out_free:
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 20/26] irqchip/gic-v3: Handle pseudo-NMIs

[Julien Thierry]

Provide a higher priority to be used for pseudo-NMIs. When such an
interrupt is received, keep interrupts fully disabled at CPU level to
prevent receiving other pseudo-NMIs while handling the current one.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 5374b43..4df1e94 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -41,6 +41,8 @@
 
 #include "irq-gic-common.h"
 
+#define GICD_INT_NMI_PRI	(GICD_INT_DEF_PRI & ~0x80)
+
 #define FLAGS_WORKAROUND_GICR_WAKER_MSM8996	(1ULL << 0)
 
 struct redist_region {
@@ -381,12 +383,45 @@ static u64 gic_mpidr_to_affinity(unsigned long mpidr)
 	return aff;
 }
 
+static inline void gic_deactivate_unexpected_irq(u32 irqnr)
+{
+	if (static_branch_likely(&supports_deactivate_key)) {
+		if (irqnr < 8192)
+			gic_write_dir(irqnr);
+	} else {
+		gic_write_eoir(irqnr);
+	}
+}
+
+static inline void gic_handle_nmi(u32 irqnr, struct pt_regs *regs)
+{
+	int err;
+
+	if (static_branch_likely(&supports_deactivate_key))
+		gic_write_eoir(irqnr);
+	/*
+	 * Leave the PSR.I bit set to prevent other NMIs to be
+	 * received while handling this one.
+	 * PSR.I will be restored when we ERET to the
+	 * interrupted context.
+	 */
+	err = handle_domain_nmi(gic_data.domain, irqnr, regs);
+	if (err)
+		gic_deactivate_unexpected_irq(irqnr);
+}
+
 static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
 {
 	u32 irqnr;
 
 	irqnr = gic_read_iar();
 
+	if (gic_supports_nmi() &&
+	    unlikely(gic_read_rpr() == GICD_INT_NMI_PRI)) {
+		gic_handle_nmi(irqnr, regs);
+		return;
+	}
+
 	if (gic_prio_masking_enabled()) {
 		gic_pmr_mask_irqs();
 		gic_arch_enable_irqs();
@@ -403,12 +438,7 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 		err = handle_domain_irq(gic_data.domain, irqnr, regs);
 		if (err) {
 			WARN_ONCE(true, "Unexpected interrupt received!\n");
-			if (static_branch_likely(&supports_deactivate_key)) {
-				if (irqnr < 8192)
-					gic_write_dir(irqnr);
-			} else {
-				gic_write_eoir(irqnr);
-			}
+			gic_deactivate_unexpected_irq(irqnr);
 		}
 		return;
 	}
-- 
1.9.1

[PATCH v8 20/26] irqchip/gic-v3: Handle pseudo-NMIs

[Julien Thierry]

Provide a higher priority to be used for pseudo-NMIs. When such an
interrupt is received, keep interrupts fully disabled at CPU level to
prevent receiving other pseudo-NMIs while handling the current one.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 5374b43..4df1e94 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -41,6 +41,8 @@
 
 #include "irq-gic-common.h"
 
+#define GICD_INT_NMI_PRI	(GICD_INT_DEF_PRI & ~0x80)
+
 #define FLAGS_WORKAROUND_GICR_WAKER_MSM8996	(1ULL << 0)
 
 struct redist_region {
@@ -381,12 +383,45 @@ static u64 gic_mpidr_to_affinity(unsigned long mpidr)
 	return aff;
 }
 
+static inline void gic_deactivate_unexpected_irq(u32 irqnr)
+{
+	if (static_branch_likely(&supports_deactivate_key)) {
+		if (irqnr < 8192)
+			gic_write_dir(irqnr);
+	} else {
+		gic_write_eoir(irqnr);
+	}
+}
+
+static inline void gic_handle_nmi(u32 irqnr, struct pt_regs *regs)
+{
+	int err;
+
+	if (static_branch_likely(&supports_deactivate_key))
+		gic_write_eoir(irqnr);
+	/*
+	 * Leave the PSR.I bit set to prevent other NMIs to be
+	 * received while handling this one.
+	 * PSR.I will be restored when we ERET to the
+	 * interrupted context.
+	 */
+	err = handle_domain_nmi(gic_data.domain, irqnr, regs);
+	if (err)
+		gic_deactivate_unexpected_irq(irqnr);
+}
+
 static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
 {
 	u32 irqnr;
 
 	irqnr = gic_read_iar();
 
+	if (gic_supports_nmi() &&
+	    unlikely(gic_read_rpr() == GICD_INT_NMI_PRI)) {
+		gic_handle_nmi(irqnr, regs);
+		return;
+	}
+
 	if (gic_prio_masking_enabled()) {
 		gic_pmr_mask_irqs();
 		gic_arch_enable_irqs();
@@ -403,12 +438,7 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
 		err = handle_domain_irq(gic_data.domain, irqnr, regs);
 		if (err) {
 			WARN_ONCE(true, "Unexpected interrupt received!\n");
-			if (static_branch_likely(&supports_deactivate_key)) {
-				if (irqnr < 8192)
-					gic_write_dir(irqnr);
-			} else {
-				gic_write_eoir(irqnr);
-			}
+			gic_deactivate_unexpected_irq(irqnr);
 		}
 		return;
 	}
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 21/26] irqchip/gic: Add functions to access irq priorities

[Julien Thierry]

Add accessors to the GIC distributor/redistributors priority registers.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-common.c | 10 ++++++++++
 drivers/irqchip/irq-gic-common.h |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c
index 3c93c6f..04eadbc 100644
--- a/drivers/irqchip/irq-gic-common.c
+++ b/drivers/irqchip/irq-gic-common.c
@@ -110,6 +110,16 @@ int gic_configure_irq(unsigned int irq, unsigned int type,
 	return ret;
 }
 
+void gic_set_irq_prio(unsigned int irq, void __iomem *base, u8 prio)
+{
+	writeb_relaxed(prio, base + GIC_DIST_PRI + irq);
+}
+
+u8 gic_get_irq_prio(unsigned int irq, void __iomem *base)
+{
+	return readb_relaxed(base + GIC_DIST_PRI + irq);
+}
+
 void gic_dist_config(void __iomem *base, int gic_irqs,
 		     void (*sync_access)(void))
 {
diff --git a/drivers/irqchip/irq-gic-common.h b/drivers/irqchip/irq-gic-common.h
index 97e58fb..f1c6f5a 100644
--- a/drivers/irqchip/irq-gic-common.h
+++ b/drivers/irqchip/irq-gic-common.h
@@ -38,6 +38,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
 		void *data);
 void gic_enable_of_quirks(const struct device_node *np,
 			  const struct gic_quirk *quirks, void *data);
+void gic_set_irq_prio(unsigned int irq, void __iomem *base, u8 prio);
+u8 gic_get_irq_prio(unsigned int irq, void __iomem *base);
 
 void gic_set_kvm_info(const struct gic_kvm_info *info);
 
-- 
1.9.1

[PATCH v8 21/26] irqchip/gic: Add functions to access irq priorities

[Julien Thierry]

Add accessors to the GIC distributor/redistributors priority registers.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-common.c | 10 ++++++++++
 drivers/irqchip/irq-gic-common.h |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c
index 3c93c6f..04eadbc 100644
--- a/drivers/irqchip/irq-gic-common.c
+++ b/drivers/irqchip/irq-gic-common.c
@@ -110,6 +110,16 @@ int gic_configure_irq(unsigned int irq, unsigned int type,
 	return ret;
 }
 
+void gic_set_irq_prio(unsigned int irq, void __iomem *base, u8 prio)
+{
+	writeb_relaxed(prio, base + GIC_DIST_PRI + irq);
+}
+
+u8 gic_get_irq_prio(unsigned int irq, void __iomem *base)
+{
+	return readb_relaxed(base + GIC_DIST_PRI + irq);
+}
+
 void gic_dist_config(void __iomem *base, int gic_irqs,
 		     void (*sync_access)(void))
 {
diff --git a/drivers/irqchip/irq-gic-common.h b/drivers/irqchip/irq-gic-common.h
index 97e58fb..f1c6f5a 100644
--- a/drivers/irqchip/irq-gic-common.h
+++ b/drivers/irqchip/irq-gic-common.h
@@ -38,6 +38,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
 		void *data);
 void gic_enable_of_quirks(const struct device_node *np,
 			  const struct gic_quirk *quirks, void *data);
+void gic_set_irq_prio(unsigned int irq, void __iomem *base, u8 prio);
+u8 gic_get_irq_prio(unsigned int irq, void __iomem *base);
 
 void gic_set_kvm_info(const struct gic_kvm_info *info);
 
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 22/26] irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI

[Julien Thierry]

Implement NMI callbacks for GICv3 irqchip. Install NMI safe handlers
when setting up interrupt line as NMI.

Only SPIs and PPIs are allowed to be set up as NMI.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 84 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 84 insertions(+)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 4df1e94..447d8ab 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -27,6 +27,7 @@
 #include <linux/of_address.h>
 #include <linux/of_irq.h>
 #include <linux/percpu.h>
+#include <linux/refcount.h>
 #include <linux/slab.h>
 
 #include <linux/irqchip.h>
@@ -93,6 +94,9 @@ struct gic_chip_data {
  */
 static DEFINE_STATIC_KEY_FALSE(supports_pseudo_nmis);
 
+/* ppi_nmi_refs[n] == number of cpus having ppi[n + 16] set as NMI */
+static refcount_t ppi_nmi_refs[16];
+
 static struct gic_kvm_info gic_v3_kvm_info;
 static DEFINE_PER_CPU(bool, has_rss);
 
@@ -320,6 +324,72 @@ static int gic_irq_get_irqchip_state(struct irq_data *d,
 	return 0;
 }
 
+static int gic_irq_nmi_setup(struct irq_data *d)
+{
+	struct irq_desc *desc = irq_to_desc(d->irq);
+
+	if (!gic_supports_nmi())
+		return -EINVAL;
+
+	if (gic_peek_irq(d, GICD_ISENABLER)) {
+		pr_err("Cannot set NMI property of enabled IRQ %u\n", d->irq);
+		return -EINVAL;
+	}
+
+	/*
+	 * A secondary irq_chip should be in charge of LPI request,
+	 * it should not be possible to get there
+	 */
+	if (WARN_ON(gic_irq(d) >= 8192))
+		return -EINVAL;
+
+	/* desc lock should already be held */
+	if (gic_irq(d) < 32) {
+		/* Setting up PPI as NMI, only switch handler for first NMI */
+		if (!refcount_inc_not_zero(&ppi_nmi_refs[gic_irq(d) - 16])) {
+			refcount_set(&ppi_nmi_refs[gic_irq(d) - 16], 1);
+			desc->handle_irq = handle_percpu_devid_fasteoi_nmi;
+		}
+	} else {
+		desc->handle_irq = handle_fasteoi_nmi;
+	}
+
+	gic_set_irq_prio(gic_irq(d), gic_dist_base(d), GICD_INT_NMI_PRI);
+
+	return 0;
+}
+
+static void gic_irq_nmi_teardown(struct irq_data *d)
+{
+	struct irq_desc *desc = irq_to_desc(d->irq);
+
+	if (WARN_ON(!gic_supports_nmi()))
+		return;
+
+	if (gic_peek_irq(d, GICD_ISENABLER)) {
+		pr_err("Cannot set NMI property of enabled IRQ %u\n", d->irq);
+		return;
+	}
+
+	/*
+	 * A secondary irq_chip should be in charge of LPI request,
+	 * it should not be possible to get there
+	 */
+	if (WARN_ON(gic_irq(d) >= 8192))
+		return;
+
+	/* desc lock should already be held */
+	if (gic_irq(d) < 32) {
+		/* Tearing down NMI, only switch handler for last NMI */
+		if (refcount_dec_and_test(&ppi_nmi_refs[gic_irq(d) - 16]))
+			desc->handle_irq = handle_percpu_devid_irq;
+	} else {
+		desc->handle_irq = handle_fasteoi_irq;
+	}
+
+	gic_set_irq_prio(gic_irq(d), gic_dist_base(d), GICD_INT_DEF_PRI);
+}
+
 static void gic_eoi_irq(struct irq_data *d)
 {
 	gic_write_eoir(gic_irq(d));
@@ -958,6 +1028,8 @@ static inline void gic_cpu_pm_init(void) { }
 	.irq_set_affinity	= gic_set_affinity,
 	.irq_get_irqchip_state	= gic_irq_get_irqchip_state,
 	.irq_set_irqchip_state	= gic_irq_set_irqchip_state,
+	.irq_nmi_setup		= gic_irq_nmi_setup,
+	.irq_nmi_teardown	= gic_irq_nmi_teardown,
 	.flags			= IRQCHIP_SET_TYPE_MASKED |
 				  IRQCHIP_SKIP_SET_WAKE |
 				  IRQCHIP_MASK_ON_SUSPEND,
@@ -973,6 +1045,8 @@ static inline void gic_cpu_pm_init(void) { }
 	.irq_get_irqchip_state	= gic_irq_get_irqchip_state,
 	.irq_set_irqchip_state	= gic_irq_set_irqchip_state,
 	.irq_set_vcpu_affinity	= gic_irq_set_vcpu_affinity,
+	.irq_nmi_setup		= gic_irq_nmi_setup,
+	.irq_nmi_teardown	= gic_irq_nmi_teardown,
 	.flags			= IRQCHIP_SET_TYPE_MASKED |
 				  IRQCHIP_SKIP_SET_WAKE |
 				  IRQCHIP_MASK_ON_SUSPEND,
@@ -1176,7 +1250,17 @@ static bool gic_enable_quirk_msm8996(void *data)
 
 static void gic_enable_nmi_support(void)
 {
+	int i;
+
+	for (i = 0; i < 16; i++)
+		refcount_set(&ppi_nmi_refs[i], 0);
+
 	static_branch_enable(&supports_pseudo_nmis);
+
+	if (static_branch_likely(&supports_deactivate_key))
+		gic_eoimode1_chip.flags |= IRQCHIP_SUPPORTS_NMI;
+	else
+		gic_chip.flags |= IRQCHIP_SUPPORTS_NMI;
 }
 
 static int __init gic_init_bases(void __iomem *dist_base,
-- 
1.9.1

[PATCH v8 22/26] irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI

[Julien Thierry]

Implement NMI callbacks for GICv3 irqchip. Install NMI safe handlers
when setting up interrupt line as NMI.

Only SPIs and PPIs are allowed to be set up as NMI.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 84 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 84 insertions(+)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 4df1e94..447d8ab 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -27,6 +27,7 @@
 #include <linux/of_address.h>
 #include <linux/of_irq.h>
 #include <linux/percpu.h>
+#include <linux/refcount.h>
 #include <linux/slab.h>
 
 #include <linux/irqchip.h>
@@ -93,6 +94,9 @@ struct gic_chip_data {
  */
 static DEFINE_STATIC_KEY_FALSE(supports_pseudo_nmis);
 
+/* ppi_nmi_refs[n] == number of cpus having ppi[n + 16] set as NMI */
+static refcount_t ppi_nmi_refs[16];
+
 static struct gic_kvm_info gic_v3_kvm_info;
 static DEFINE_PER_CPU(bool, has_rss);
 
@@ -320,6 +324,72 @@ static int gic_irq_get_irqchip_state(struct irq_data *d,
 	return 0;
 }
 
+static int gic_irq_nmi_setup(struct irq_data *d)
+{
+	struct irq_desc *desc = irq_to_desc(d->irq);
+
+	if (!gic_supports_nmi())
+		return -EINVAL;
+
+	if (gic_peek_irq(d, GICD_ISENABLER)) {
+		pr_err("Cannot set NMI property of enabled IRQ %u\n", d->irq);
+		return -EINVAL;
+	}
+
+	/*
+	 * A secondary irq_chip should be in charge of LPI request,
+	 * it should not be possible to get there
+	 */
+	if (WARN_ON(gic_irq(d) >= 8192))
+		return -EINVAL;
+
+	/* desc lock should already be held */
+	if (gic_irq(d) < 32) {
+		/* Setting up PPI as NMI, only switch handler for first NMI */
+		if (!refcount_inc_not_zero(&ppi_nmi_refs[gic_irq(d) - 16])) {
+			refcount_set(&ppi_nmi_refs[gic_irq(d) - 16], 1);
+			desc->handle_irq = handle_percpu_devid_fasteoi_nmi;
+		}
+	} else {
+		desc->handle_irq = handle_fasteoi_nmi;
+	}
+
+	gic_set_irq_prio(gic_irq(d), gic_dist_base(d), GICD_INT_NMI_PRI);
+
+	return 0;
+}
+
+static void gic_irq_nmi_teardown(struct irq_data *d)
+{
+	struct irq_desc *desc = irq_to_desc(d->irq);
+
+	if (WARN_ON(!gic_supports_nmi()))
+		return;
+
+	if (gic_peek_irq(d, GICD_ISENABLER)) {
+		pr_err("Cannot set NMI property of enabled IRQ %u\n", d->irq);
+		return;
+	}
+
+	/*
+	 * A secondary irq_chip should be in charge of LPI request,
+	 * it should not be possible to get there
+	 */
+	if (WARN_ON(gic_irq(d) >= 8192))
+		return;
+
+	/* desc lock should already be held */
+	if (gic_irq(d) < 32) {
+		/* Tearing down NMI, only switch handler for last NMI */
+		if (refcount_dec_and_test(&ppi_nmi_refs[gic_irq(d) - 16]))
+			desc->handle_irq = handle_percpu_devid_irq;
+	} else {
+		desc->handle_irq = handle_fasteoi_irq;
+	}
+
+	gic_set_irq_prio(gic_irq(d), gic_dist_base(d), GICD_INT_DEF_PRI);
+}
+
 static void gic_eoi_irq(struct irq_data *d)
 {
 	gic_write_eoir(gic_irq(d));
@@ -958,6 +1028,8 @@ static inline void gic_cpu_pm_init(void) { }
 	.irq_set_affinity	= gic_set_affinity,
 	.irq_get_irqchip_state	= gic_irq_get_irqchip_state,
 	.irq_set_irqchip_state	= gic_irq_set_irqchip_state,
+	.irq_nmi_setup		= gic_irq_nmi_setup,
+	.irq_nmi_teardown	= gic_irq_nmi_teardown,
 	.flags			= IRQCHIP_SET_TYPE_MASKED |
 				  IRQCHIP_SKIP_SET_WAKE |
 				  IRQCHIP_MASK_ON_SUSPEND,
@@ -973,6 +1045,8 @@ static inline void gic_cpu_pm_init(void) { }
 	.irq_get_irqchip_state	= gic_irq_get_irqchip_state,
 	.irq_set_irqchip_state	= gic_irq_set_irqchip_state,
 	.irq_set_vcpu_affinity	= gic_irq_set_vcpu_affinity,
+	.irq_nmi_setup		= gic_irq_nmi_setup,
+	.irq_nmi_teardown	= gic_irq_nmi_teardown,
 	.flags			= IRQCHIP_SET_TYPE_MASKED |
 				  IRQCHIP_SKIP_SET_WAKE |
 				  IRQCHIP_MASK_ON_SUSPEND,
@@ -1176,7 +1250,17 @@ static bool gic_enable_quirk_msm8996(void *data)
 
 static void gic_enable_nmi_support(void)
 {
+	int i;
+
+	for (i = 0; i < 16; i++)
+		refcount_set(&ppi_nmi_refs[i], 0);
+
 	static_branch_enable(&supports_pseudo_nmis);
+
+	if (static_branch_likely(&supports_deactivate_key))
+		gic_eoimode1_chip.flags |= IRQCHIP_SUPPORTS_NMI;
+	else
+		gic_chip.flags |= IRQCHIP_SUPPORTS_NMI;
 }
 
 static int __init gic_init_bases(void __iomem *dist_base,
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 23/26] arm64: Handle serror in NMI context

[Julien Thierry]

Per definition of the daifflags, Serrors can occur during any interrupt
context, that includes NMI contexts. Trying to nmi_enter in an nmi context
will crash.

Skip nmi_enter/nmi_exit when serror occurred during an NMI.

Suggested-by: James Morse <james.morse@arm.com>
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: James Morse <james.morse@arm.com>
---
 arch/arm64/kernel/traps.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 4e2fb87..8ad119c 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -898,13 +898,17 @@ bool arm64_is_fatal_ras_serror(struct pt_regs *regs, unsigned int esr)
 
 asmlinkage void do_serror(struct pt_regs *regs, unsigned int esr)
 {
-	nmi_enter();
+	const bool was_in_nmi = in_nmi();
+
+	if (!was_in_nmi)
+		nmi_enter();
 
 	/* non-RAS errors are not containable */
 	if (!arm64_is_ras_serror(esr) || arm64_is_fatal_ras_serror(regs, esr))
 		arm64_serror_panic(regs, esr);
 
-	nmi_exit();
+	if (!was_in_nmi)
+		nmi_exit();
 }
 
 void __pte_error(const char *file, int line, unsigned long val)
-- 
1.9.1

[PATCH v8 23/26] arm64: Handle serror in NMI context

[Julien Thierry]

Per definition of the daifflags, Serrors can occur during any interrupt
context, that includes NMI contexts. Trying to nmi_enter in an nmi context
will crash.

Skip nmi_enter/nmi_exit when serror occurred during an NMI.

Suggested-by: James Morse <james.morse@arm.com>
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: James Morse <james.morse@arm.com>
---
 arch/arm64/kernel/traps.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 4e2fb87..8ad119c 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -898,13 +898,17 @@ bool arm64_is_fatal_ras_serror(struct pt_regs *regs, unsigned int esr)
 
 asmlinkage void do_serror(struct pt_regs *regs, unsigned int esr)
 {
-	nmi_enter();
+	const bool was_in_nmi = in_nmi();
+
+	if (!was_in_nmi)
+		nmi_enter();
 
 	/* non-RAS errors are not containable */
 	if (!arm64_is_ras_serror(esr) || arm64_is_fatal_ras_serror(regs, esr))
 		arm64_serror_panic(regs, esr);
 
-	nmi_exit();
+	if (!was_in_nmi)
+		nmi_exit();
 }
 
 void __pte_error(const char *file, int line, unsigned long val)
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 24/26] arm64: Skip preemption when exiting an NMI

[Julien Thierry]

Handling of an NMI should not set any TIF flags. For NMIs received from
EL0 the current exit path is safe to use.

However, an NMI received at EL1 could have interrupted some task context
that has set the TIF_NEED_RESCHED flag. Preempting a task should not
happen as a result of an NMI.

Skip preemption after handling an NMI from EL1.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/entry.S | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 35a47f6..a0b0a22 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -624,6 +624,14 @@ el1_irq:
 
 #ifdef CONFIG_PREEMPT
 	ldr	x24, [tsk, #TSK_TI_PREEMPT]	// get preempt count
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+	/*
+	 * DA_F were cleared at start of handling. If anything is set in DAIF,
+	 * we come back from an NMI, so skip preemption
+	 */
+	mrs	x0, daif
+	orr	x24, x24, x0
+alternative_else_nop_endif
 	cbnz	x24, 1f				// preempt count != 0
 	bl	el1_preempt
 1:
-- 
1.9.1

[PATCH v8 24/26] arm64: Skip preemption when exiting an NMI

[Julien Thierry]

Handling of an NMI should not set any TIF flags. For NMIs received from
EL0 the current exit path is safe to use.

However, an NMI received at EL1 could have interrupted some task context
that has set the TIF_NEED_RESCHED flag. Preempting a task should not
happen as a result of an NMI.

Skip preemption after handling an NMI from EL1.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/entry.S | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 35a47f6..a0b0a22 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -624,6 +624,14 @@ el1_irq:
 
 #ifdef CONFIG_PREEMPT
 	ldr	x24, [tsk, #TSK_TI_PREEMPT]	// get preempt count
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+	/*
+	 * DA_F were cleared at start of handling. If anything is set in DAIF,
+	 * we come back from an NMI, so skip preemption
+	 */
+	mrs	x0, daif
+	orr	x24, x24, x0
+alternative_else_nop_endif
 	cbnz	x24, 1f				// preempt count != 0
 	bl	el1_preempt
 1:
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 25/26] arm64: Skip irqflags tracing for NMI in IRQs disabled context

[Julien Thierry]

When an NMI is raised while interrupts where disabled, the IRQ tracing
already is in the correct state (i.e. hardirqs_off) and should be left
as such when returning to the interrupted context.

Check whether PMR was masking interrupts when the NMI was raised and
skip IRQ tracing if necessary.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/entry.S | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index a0b0a22..bd0b078 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -617,7 +617,18 @@ el1_irq:
 	kernel_entry 1
 	enable_da_f
 #ifdef CONFIG_TRACE_IRQFLAGS
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+	ldr	x20, [sp, #S_PMR_SAVE]
+alternative_else
+	mov	x20, #GIC_PRIO_IRQON
+alternative_endif
+	cmp	x20, #GIC_PRIO_IRQOFF
+	/* Irqs were disabled, don't trace */
+	b.ls	1f
+#endif
 	bl	trace_hardirqs_off
+1:
 #endif
 
 	irq_handler
@@ -637,8 +648,18 @@ alternative_else_nop_endif
 1:
 #endif
 #ifdef CONFIG_TRACE_IRQFLAGS
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+	/*
+	 * if IRQs were disabled when we received the interrupt, we have an NMI
+	 * and we are not re-enabling interrupt upon eret. Skip tracing.
+	 */
+	cmp	x20, #GIC_PRIO_IRQOFF
+	b.ls	1f
+#endif
 	bl	trace_hardirqs_on
+1:
 #endif
+
 	kernel_exit 1
 ENDPROC(el1_irq)
 
-- 
1.9.1

[PATCH v8 25/26] arm64: Skip irqflags tracing for NMI in IRQs disabled context

[Julien Thierry]

When an NMI is raised while interrupts where disabled, the IRQ tracing
already is in the correct state (i.e. hardirqs_off) and should be left
as such when returning to the interrupted context.

Check whether PMR was masking interrupts when the NMI was raised and
skip IRQ tracing if necessary.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/entry.S | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index a0b0a22..bd0b078 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -617,7 +617,18 @@ el1_irq:
 	kernel_entry 1
 	enable_da_f
 #ifdef CONFIG_TRACE_IRQFLAGS
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+alternative_if ARM64_HAS_IRQ_PRIO_MASKING
+	ldr	x20, [sp, #S_PMR_SAVE]
+alternative_else
+	mov	x20, #GIC_PRIO_IRQON
+alternative_endif
+	cmp	x20, #GIC_PRIO_IRQOFF
+	/* Irqs were disabled, don't trace */
+	b.ls	1f
+#endif
 	bl	trace_hardirqs_off
+1:
 #endif
 
 	irq_handler
@@ -637,8 +648,18 @@ alternative_else_nop_endif
 1:
 #endif
 #ifdef CONFIG_TRACE_IRQFLAGS
+#ifdef CONFIG_ARM64_PSEUDO_NMI
+	/*
+	 * if IRQs were disabled when we received the interrupt, we have an NMI
+	 * and we are not re-enabling interrupt upon eret. Skip tracing.
+	 */
+	cmp	x20, #GIC_PRIO_IRQOFF
+	b.ls	1f
+#endif
 	bl	trace_hardirqs_on
+1:
 #endif
+
 	kernel_exit 1
 ENDPROC(el1_irq)
 
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 26/26] arm64: Enable the support of pseudo-NMIs

[Julien Thierry]

Add a build option and a command line parameter to build and enable the
support of pseudo-NMIs.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 Documentation/admin-guide/kernel-parameters.txt |  6 ++++++
 arch/arm64/Kconfig                              | 14 ++++++++++++++
 arch/arm64/kernel/cpufeature.c                  | 11 ++++++++++-
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index b799bcf..173e2cc 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1197,6 +1197,12 @@
 			to discrete, to make X server driver able to add WB
 			entry later. This parameter enables that.
 
+	enable_pseudo_nmi [ARM64]
+			Enables support for pseudo-NMIs in the kernel. This
+			requires both the kernel to be built with
+			CONFIG_ARM64_PSEUDO_NMI and to be running on a
+			platform with GICv3.
+
 	enable_timer_pin_1 [X86]
 			Enable PIN 1 of APIC timer
 			Can be useful to work around chipset bugs
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index a4168d3..8d84bfd 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1328,6 +1328,20 @@ config ARM64_MODULE_PLTS
 	bool
 	select HAVE_MOD_ARCH_SPECIFIC
 
+config ARM64_PSEUDO_NMI
+	bool "Support for NMI-like interrupts"
+	select CONFIG_ARM_GIC_V3
+	help
+	  Adds support for mimicking Non-Maskable Interrupts through the use of
+	  GIC interrupt priority. This support requires version 3 or later of
+	  Arm GIC.
+
+	  This high priority configuration for interrupts need to be
+	  explicitly enabled through the new kernel parameter
+	  "enable_pseudo_nmi".
+
+	  If unsure, say N
+
 config RELOCATABLE
 	bool
 	help
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 71c8d4f..9df1217 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1202,10 +1202,19 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 #endif /* CONFIG_ARM64_PTR_AUTH */
 
 #ifdef CONFIG_ARM64_PSEUDO_NMI
+static bool enable_pseudo_nmi;
+
+static int __init early_enable_pseudo_nmi(char *p)
+{
+	enable_pseudo_nmi = true;
+	return 0;
+}
+early_param("enable_pseudo_nmi", early_enable_pseudo_nmi);
+
 static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry,
 				   int scope)
 {
-	return false;
+	return enable_pseudo_nmi && has_useable_gicv3_cpuif(entry, scope);
 }
 #endif
 
-- 
1.9.1

[PATCH v8 26/26] arm64: Enable the support of pseudo-NMIs

[Julien Thierry]

Add a build option and a command line parameter to build and enable the
support of pseudo-NMIs.

Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
 Documentation/admin-guide/kernel-parameters.txt |  6 ++++++
 arch/arm64/Kconfig                              | 14 ++++++++++++++
 arch/arm64/kernel/cpufeature.c                  | 11 ++++++++++-
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index b799bcf..173e2cc 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1197,6 +1197,12 @@
 			to discrete, to make X server driver able to add WB
 			entry later. This parameter enables that.
 
+	enable_pseudo_nmi [ARM64]
+			Enables support for pseudo-NMIs in the kernel. This
+			requires both the kernel to be built with
+			CONFIG_ARM64_PSEUDO_NMI and to be running on a
+			platform with GICv3.
+
 	enable_timer_pin_1 [X86]
 			Enable PIN 1 of APIC timer
 			Can be useful to work around chipset bugs
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index a4168d3..8d84bfd 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1328,6 +1328,20 @@ config ARM64_MODULE_PLTS
 	bool
 	select HAVE_MOD_ARCH_SPECIFIC
 
+config ARM64_PSEUDO_NMI
+	bool "Support for NMI-like interrupts"
+	select CONFIG_ARM_GIC_V3
+	help
+	  Adds support for mimicking Non-Maskable Interrupts through the use of
+	  GIC interrupt priority. This support requires version 3 or later of
+	  Arm GIC.
+
+	  This high priority configuration for interrupts need to be
+	  explicitly enabled through the new kernel parameter
+	  "enable_pseudo_nmi".
+
+	  If unsure, say N
+
 config RELOCATABLE
 	bool
 	help
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 71c8d4f..9df1217 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1202,10 +1202,19 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 #endif /* CONFIG_ARM64_PTR_AUTH */
 
 #ifdef CONFIG_ARM64_PSEUDO_NMI
+static bool enable_pseudo_nmi;
+
+static int __init early_enable_pseudo_nmi(char *p)
+{
+	enable_pseudo_nmi = true;
+	return 0;
+}
+early_param("enable_pseudo_nmi", early_enable_pseudo_nmi);
+
 static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry,
 				   int scope)
 {
-	return false;
+	return enable_pseudo_nmi && has_useable_gicv3_cpuif(entry, scope);
 }
 #endif
 
-- 
1.9.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Suzuki K Poulose]

Hi Julien,

On 08/01/2019 14:07, Julien Thierry wrote:
> From: Daniel Thompson <daniel.thompson@linaro.org>
> 
> Currently alternatives are applied very late in the boot process (and
> a long time after we enable scheduling). Some alternative sequences,
> such as those that alter the way CPU context is stored, must be applied
> much earlier in the boot sequence.
> 
> Introduce apply_boot_alternatives() to allow some alternatives to be
> applied immediately after we detect the CPU features of the boot CPU.
> 
> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
> [julien.thierry@arm.com: rename to fit new cpufeature framework better,
> 			 apply BOOT_SCOPE feature early in boot]
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Christoffer Dall <christoffer.dall@arm.com>
> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
>   arch/arm64/include/asm/alternative.h |  1 +
>   arch/arm64/include/asm/cpufeature.h  |  4 ++++
>   arch/arm64/kernel/alternative.c      | 43 +++++++++++++++++++++++++++++++-----
>   arch/arm64/kernel/cpufeature.c       |  6 +++++
>   arch/arm64/kernel/smp.c              |  7 ++++++
>   5 files changed, 56 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h
> index 9806a23..b9f8d78 100644
> --- a/arch/arm64/include/asm/alternative.h
> +++ b/arch/arm64/include/asm/alternative.h
> @@ -25,6 +25,7 @@ struct alt_instr {
>   typedef void (*alternative_cb_t)(struct alt_instr *alt,
>   				 __le32 *origptr, __le32 *updptr, int nr_inst);
>   
> +void __init apply_boot_alternatives(void);
>   void __init apply_alternatives_all(void);
>   bool alternative_is_applied(u16 cpufeature);
>   
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 89c3f31..e505e1f 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
>   extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
>   extern struct static_key_false arm64_const_caps_ready;
>   
> +/* ARM64 CAPS + alternative_cb */
> +#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
> +extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
> +
>   #define for_each_available_cap(cap)		\
>   	for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
>   
> diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c
> index c947d22..a9b4677 100644
> --- a/arch/arm64/kernel/alternative.c
> +++ b/arch/arm64/kernel/alternative.c
> @@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start, u64 end)
>   	} while (cur += d_size, cur < end);
>   }
>   
> -static void __apply_alternatives(void *alt_region, bool is_module)
> +static void __apply_alternatives(void *alt_region,  bool is_module,
> +				 unsigned long *feature_mask)
>   {
>   	struct alt_instr *alt;
>   	struct alt_region *region = alt_region;
> @@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region, bool is_module)
>   	for (alt = region->begin; alt < region->end; alt++) {
>   		int nr_inst;
>   
> +		if (!test_bit(alt->cpufeature, feature_mask))
> +			continue;
> +
>   		/* Use ARM64_CB_PATCH as an unconditional patch */
>   		if (alt->cpufeature < ARM64_CB_PATCH &&
>   		    !cpus_have_cap(alt->cpufeature))
> @@ -203,8 +207,11 @@ static void __apply_alternatives(void *alt_region, bool is_module)
>   		__flush_icache_all();
>   		isb();
>   
> -		/* We applied all that was available */
> -		bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
> +		/* Ignore ARM64_CB bit from feature mask */
> +		bitmap_or(applied_alternatives, applied_alternatives,
> +			  feature_mask, ARM64_NCAPS);
> +		bitmap_and(applied_alternatives, applied_alternatives,
> +			   cpu_hwcaps, ARM64_NCAPS);
>   	}
>   }
>   
> @@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void *unused)
>   			cpu_relax();
>   		isb();
>   	} else {
> +		DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
> +
> +		bitmap_complement(remaining_capabilities, boot_capabilities,
> +				  ARM64_NPATCHABLE);
> +
>   		BUG_ON(all_alternatives_applied);
> -		__apply_alternatives(&region, false);
> +		__apply_alternatives(&region, false, remaining_capabilities);
>   		/* Barriers provided by the cache flushing */
>   		WRITE_ONCE(all_alternatives_applied, 1);
>   	}
> @@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
>   	stop_machine(__apply_alternatives_multi_stop, NULL, cpu_online_mask);
>   }
>   
> +/*
> + * This is called very early in the boot process (directly after we run
> + * a feature detect on the boot CPU). No need to worry about other CPUs
> + * here.
> + */
> +void __init apply_boot_alternatives(void)
> +{
> +	struct alt_region region = {
> +		.begin	= (struct alt_instr *)__alt_instructions,
> +		.end	= (struct alt_instr *)__alt_instructions_end,
> +	};
> +
> +	/* If called on non-boot cpu things could go wrong */
> +	WARN_ON(smp_processor_id() != 0);
> +
> +	__apply_alternatives(&region, false, &boot_capabilities[0]);
> +}
> +
>   #ifdef CONFIG_MODULES
>   void apply_alternatives_module(void *start, size_t length)
>   {
> @@ -247,7 +277,10 @@ void apply_alternatives_module(void *start, size_t length)
>   		.begin	= start,
>   		.end	= start + length,
>   	};
> +	DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
> +
> +	bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
>   
> -	__apply_alternatives(&region, true);
> +	__apply_alternatives(&region, true, &all_capabilities[0]);
>   }
>   #endif
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 84fa5be..71c8d4f 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -54,6 +54,9 @@
>   EXPORT_SYMBOL(cpu_hwcaps);
>   static struct arm64_cpu_capabilities const __ro_after_init *cpu_hwcaps_ptrs[ARM64_NCAPS];
>   
> +/* Need also bit for ARM64_CB_PATCH */
> +DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
> +
>   /*
>    * Flag to indicate if we have computed the system wide
>    * capabilities based on the boot time active CPUs. This
> @@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16 scope_mask)
>   		if (caps->desc)
>   			pr_info("detected: %s\n", caps->desc);
>   		cpus_set_cap(caps->capability);
> +
> +		if (caps->type & SCOPE_BOOT_CPU)

You may want to do :
		if (scope_mask & SCOPE_BOOT_CPU)

for a tighter check to ensure this doesn't update the boot_capabilities
after we have applied the boot_scope alternatives and miss applying the
alternatives for those, should someone add a multi-scope (i.e SCOPE_BOOT_CPU and
something else) capability (even by mistake).

With that:

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Re: [PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Suzuki K Poulose]

Hi Julien,

On 08/01/2019 14:07, Julien Thierry wrote:
> From: Daniel Thompson <daniel.thompson@linaro.org>
> 
> Currently alternatives are applied very late in the boot process (and
> a long time after we enable scheduling). Some alternative sequences,
> such as those that alter the way CPU context is stored, must be applied
> much earlier in the boot sequence.
> 
> Introduce apply_boot_alternatives() to allow some alternatives to be
> applied immediately after we detect the CPU features of the boot CPU.
> 
> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
> [julien.thierry@arm.com: rename to fit new cpufeature framework better,
> 			 apply BOOT_SCOPE feature early in boot]
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Christoffer Dall <christoffer.dall@arm.com>
> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
> ---
>   arch/arm64/include/asm/alternative.h |  1 +
>   arch/arm64/include/asm/cpufeature.h  |  4 ++++
>   arch/arm64/kernel/alternative.c      | 43 +++++++++++++++++++++++++++++++-----
>   arch/arm64/kernel/cpufeature.c       |  6 +++++
>   arch/arm64/kernel/smp.c              |  7 ++++++
>   5 files changed, 56 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h
> index 9806a23..b9f8d78 100644
> --- a/arch/arm64/include/asm/alternative.h
> +++ b/arch/arm64/include/asm/alternative.h
> @@ -25,6 +25,7 @@ struct alt_instr {
>   typedef void (*alternative_cb_t)(struct alt_instr *alt,
>   				 __le32 *origptr, __le32 *updptr, int nr_inst);
>   
> +void __init apply_boot_alternatives(void);
>   void __init apply_alternatives_all(void);
>   bool alternative_is_applied(u16 cpufeature);
>   
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 89c3f31..e505e1f 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
>   extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
>   extern struct static_key_false arm64_const_caps_ready;
>   
> +/* ARM64 CAPS + alternative_cb */
> +#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
> +extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
> +
>   #define for_each_available_cap(cap)		\
>   	for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
>   
> diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c
> index c947d22..a9b4677 100644
> --- a/arch/arm64/kernel/alternative.c
> +++ b/arch/arm64/kernel/alternative.c
> @@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start, u64 end)
>   	} while (cur += d_size, cur < end);
>   }
>   
> -static void __apply_alternatives(void *alt_region, bool is_module)
> +static void __apply_alternatives(void *alt_region,  bool is_module,
> +				 unsigned long *feature_mask)
>   {
>   	struct alt_instr *alt;
>   	struct alt_region *region = alt_region;
> @@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region, bool is_module)
>   	for (alt = region->begin; alt < region->end; alt++) {
>   		int nr_inst;
>   
> +		if (!test_bit(alt->cpufeature, feature_mask))
> +			continue;
> +
>   		/* Use ARM64_CB_PATCH as an unconditional patch */
>   		if (alt->cpufeature < ARM64_CB_PATCH &&
>   		    !cpus_have_cap(alt->cpufeature))
> @@ -203,8 +207,11 @@ static void __apply_alternatives(void *alt_region, bool is_module)
>   		__flush_icache_all();
>   		isb();
>   
> -		/* We applied all that was available */
> -		bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
> +		/* Ignore ARM64_CB bit from feature mask */
> +		bitmap_or(applied_alternatives, applied_alternatives,
> +			  feature_mask, ARM64_NCAPS);
> +		bitmap_and(applied_alternatives, applied_alternatives,
> +			   cpu_hwcaps, ARM64_NCAPS);
>   	}
>   }
>   
> @@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void *unused)
>   			cpu_relax();
>   		isb();
>   	} else {
> +		DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
> +
> +		bitmap_complement(remaining_capabilities, boot_capabilities,
> +				  ARM64_NPATCHABLE);
> +
>   		BUG_ON(all_alternatives_applied);
> -		__apply_alternatives(&region, false);
> +		__apply_alternatives(&region, false, remaining_capabilities);
>   		/* Barriers provided by the cache flushing */
>   		WRITE_ONCE(all_alternatives_applied, 1);
>   	}
> @@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
>   	stop_machine(__apply_alternatives_multi_stop, NULL, cpu_online_mask);
>   }
>   
> +/*
> + * This is called very early in the boot process (directly after we run
> + * a feature detect on the boot CPU). No need to worry about other CPUs
> + * here.
> + */
> +void __init apply_boot_alternatives(void)
> +{
> +	struct alt_region region = {
> +		.begin	= (struct alt_instr *)__alt_instructions,
> +		.end	= (struct alt_instr *)__alt_instructions_end,
> +	};
> +
> +	/* If called on non-boot cpu things could go wrong */
> +	WARN_ON(smp_processor_id() != 0);
> +
> +	__apply_alternatives(&region, false, &boot_capabilities[0]);
> +}
> +
>   #ifdef CONFIG_MODULES
>   void apply_alternatives_module(void *start, size_t length)
>   {
> @@ -247,7 +277,10 @@ void apply_alternatives_module(void *start, size_t length)
>   		.begin	= start,
>   		.end	= start + length,
>   	};
> +	DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
> +
> +	bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
>   
> -	__apply_alternatives(&region, true);
> +	__apply_alternatives(&region, true, &all_capabilities[0]);
>   }
>   #endif
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 84fa5be..71c8d4f 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -54,6 +54,9 @@
>   EXPORT_SYMBOL(cpu_hwcaps);
>   static struct arm64_cpu_capabilities const __ro_after_init *cpu_hwcaps_ptrs[ARM64_NCAPS];
>   
> +/* Need also bit for ARM64_CB_PATCH */
> +DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
> +
>   /*
>    * Flag to indicate if we have computed the system wide
>    * capabilities based on the boot time active CPUs. This
> @@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16 scope_mask)
>   		if (caps->desc)
>   			pr_info("detected: %s\n", caps->desc);
>   		cpus_set_cap(caps->capability);
> +
> +		if (caps->type & SCOPE_BOOT_CPU)

You may want to do :
		if (scope_mask & SCOPE_BOOT_CPU)

for a tighter check to ensure this doesn't update the boot_capabilities
after we have applied the boot_scope alternatives and miss applying the
alternatives for those, should someone add a multi-scope (i.e SCOPE_BOOT_CPU and
something else) capability (even by mistake).

With that:

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Julien Thierry]

Hi Suzuki,

On 08/01/2019 14:51, Suzuki K Poulose wrote:
> Hi Julien,
> 
> On 08/01/2019 14:07, Julien Thierry wrote:
>> From: Daniel Thompson <daniel.thompson@linaro.org>
>>
>> Currently alternatives are applied very late in the boot process (and
>> a long time after we enable scheduling). Some alternative sequences,
>> such as those that alter the way CPU context is stored, must be applied
>> much earlier in the boot sequence.
>>
>> Introduce apply_boot_alternatives() to allow some alternatives to be
>> applied immediately after we detect the CPU features of the boot CPU.
>>
>> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
>> [julien.thierry@arm.com: rename to fit new cpufeature framework better,
>>              apply BOOT_SCOPE feature early in boot]
>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Cc: Christoffer Dall <christoffer.dall@arm.com>
>> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>>   arch/arm64/include/asm/alternative.h |  1 +
>>   arch/arm64/include/asm/cpufeature.h  |  4 ++++
>>   arch/arm64/kernel/alternative.c      | 43
>> +++++++++++++++++++++++++++++++-----
>>   arch/arm64/kernel/cpufeature.c       |  6 +++++
>>   arch/arm64/kernel/smp.c              |  7 ++++++
>>   5 files changed, 56 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/alternative.h
>> b/arch/arm64/include/asm/alternative.h
>> index 9806a23..b9f8d78 100644
>> --- a/arch/arm64/include/asm/alternative.h
>> +++ b/arch/arm64/include/asm/alternative.h
>> @@ -25,6 +25,7 @@ struct alt_instr {
>>   typedef void (*alternative_cb_t)(struct alt_instr *alt,
>>                    __le32 *origptr, __le32 *updptr, int nr_inst);
>>   +void __init apply_boot_alternatives(void);
>>   void __init apply_alternatives_all(void);
>>   bool alternative_is_applied(u16 cpufeature);
>>   diff --git a/arch/arm64/include/asm/cpufeature.h
>> b/arch/arm64/include/asm/cpufeature.h
>> index 89c3f31..e505e1f 100644
>> --- a/arch/arm64/include/asm/cpufeature.h
>> +++ b/arch/arm64/include/asm/cpufeature.h
>> @@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const
>> struct arm64_cpu_capabilities *cap)
>>   extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
>>   extern struct static_key_false arm64_const_caps_ready;
>>   +/* ARM64 CAPS + alternative_cb */
>> +#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
>> +extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>> +
>>   #define for_each_available_cap(cap)        \
>>       for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
>>   diff --git a/arch/arm64/kernel/alternative.c
>> b/arch/arm64/kernel/alternative.c
>> index c947d22..a9b4677 100644
>> --- a/arch/arm64/kernel/alternative.c
>> +++ b/arch/arm64/kernel/alternative.c
>> @@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start,
>> u64 end)
>>       } while (cur += d_size, cur < end);
>>   }
>>   -static void __apply_alternatives(void *alt_region, bool is_module)
>> +static void __apply_alternatives(void *alt_region,  bool is_module,
>> +                 unsigned long *feature_mask)
>>   {
>>       struct alt_instr *alt;
>>       struct alt_region *region = alt_region;
>> @@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region,
>> bool is_module)
>>       for (alt = region->begin; alt < region->end; alt++) {
>>           int nr_inst;
>>   +        if (!test_bit(alt->cpufeature, feature_mask))
>> +            continue;
>> +
>>           /* Use ARM64_CB_PATCH as an unconditional patch */
>>           if (alt->cpufeature < ARM64_CB_PATCH &&
>>               !cpus_have_cap(alt->cpufeature))
>> @@ -203,8 +207,11 @@ static void __apply_alternatives(void
>> *alt_region, bool is_module)
>>           __flush_icache_all();
>>           isb();
>>   -        /* We applied all that was available */
>> -        bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
>> +        /* Ignore ARM64_CB bit from feature mask */
>> +        bitmap_or(applied_alternatives, applied_alternatives,
>> +              feature_mask, ARM64_NCAPS);
>> +        bitmap_and(applied_alternatives, applied_alternatives,
>> +               cpu_hwcaps, ARM64_NCAPS);
>>       }
>>   }
>>   @@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void
>> *unused)
>>               cpu_relax();
>>           isb();
>>       } else {
>> +        DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
>> +
>> +        bitmap_complement(remaining_capabilities, boot_capabilities,
>> +                  ARM64_NPATCHABLE);
>> +
>>           BUG_ON(all_alternatives_applied);
>> -        __apply_alternatives(&region, false);
>> +        __apply_alternatives(&region, false, remaining_capabilities);
>>           /* Barriers provided by the cache flushing */
>>           WRITE_ONCE(all_alternatives_applied, 1);
>>       }
>> @@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
>>       stop_machine(__apply_alternatives_multi_stop, NULL,
>> cpu_online_mask);
>>   }
>>   +/*
>> + * This is called very early in the boot process (directly after we run
>> + * a feature detect on the boot CPU). No need to worry about other CPUs
>> + * here.
>> + */
>> +void __init apply_boot_alternatives(void)
>> +{
>> +    struct alt_region region = {
>> +        .begin    = (struct alt_instr *)__alt_instructions,
>> +        .end    = (struct alt_instr *)__alt_instructions_end,
>> +    };
>> +
>> +    /* If called on non-boot cpu things could go wrong */
>> +    WARN_ON(smp_processor_id() != 0);
>> +
>> +    __apply_alternatives(&region, false, &boot_capabilities[0]);
>> +}
>> +
>>   #ifdef CONFIG_MODULES
>>   void apply_alternatives_module(void *start, size_t length)
>>   {
>> @@ -247,7 +277,10 @@ void apply_alternatives_module(void *start,
>> size_t length)
>>           .begin    = start,
>>           .end    = start + length,
>>       };
>> +    DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
>> +
>> +    bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
>>   -    __apply_alternatives(&region, true);
>> +    __apply_alternatives(&region, true, &all_capabilities[0]);
>>   }
>>   #endif
>> diff --git a/arch/arm64/kernel/cpufeature.c
>> b/arch/arm64/kernel/cpufeature.c
>> index 84fa5be..71c8d4f 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -54,6 +54,9 @@
>>   EXPORT_SYMBOL(cpu_hwcaps);
>>   static struct arm64_cpu_capabilities const __ro_after_init
>> *cpu_hwcaps_ptrs[ARM64_NCAPS];
>>   +/* Need also bit for ARM64_CB_PATCH */
>> +DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>> +
>>   /*
>>    * Flag to indicate if we have computed the system wide
>>    * capabilities based on the boot time active CPUs. This
>> @@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16 scope_mask)
>>           if (caps->desc)
>>               pr_info("detected: %s\n", caps->desc);
>>           cpus_set_cap(caps->capability);
>> +
>> +        if (caps->type & SCOPE_BOOT_CPU)
> 
> You may want to do :
>         if (scope_mask & SCOPE_BOOT_CPU)
> 
> for a tighter check to ensure this doesn't update the boot_capabilities
> after we have applied the boot_scope alternatives and miss applying the
> alternatives for those, should someone add a multi-scope (i.e
> SCOPE_BOOT_CPU and
> something else) capability (even by mistake).
> 

But a multi-scope capability containing SCOPE_BOOT_CPU should already
get updated for setup_boot_cpu_capabilities. Capabilities marked with
SCOPE_BOOT_CPU need to be enabled on the boot CPU or not at all.

Shouldn't the call to caps->matches() fail for a boot feature that was
not found on the boot cpu?

Also, you made the opposite suggestion 4 version ago with a more
worrying scenario :) :
https://lkml.org/lkml/2018/5/25/208

Otherwise, if my assumption above is wrong, it means the check should
probably be:
	if (caps->type & SCOPE_BOOT_CPU && scope_mask & SCOPE_BOOT_CPU)

But my current understanding is that we don't need that.

> With that:
> 
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Let me know if I can keep your tag or if I indeed need to change the
condition.

Thanks,

-- 
Julien Thierry

Re: [PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Julien Thierry]

Hi Suzuki,

On 08/01/2019 14:51, Suzuki K Poulose wrote:
> Hi Julien,
> 
> On 08/01/2019 14:07, Julien Thierry wrote:
>> From: Daniel Thompson <daniel.thompson@linaro.org>
>>
>> Currently alternatives are applied very late in the boot process (and
>> a long time after we enable scheduling). Some alternative sequences,
>> such as those that alter the way CPU context is stored, must be applied
>> much earlier in the boot sequence.
>>
>> Introduce apply_boot_alternatives() to allow some alternatives to be
>> applied immediately after we detect the CPU features of the boot CPU.
>>
>> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
>> [julien.thierry@arm.com: rename to fit new cpufeature framework better,
>>              apply BOOT_SCOPE feature early in boot]
>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Cc: Christoffer Dall <christoffer.dall@arm.com>
>> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
>> ---
>>   arch/arm64/include/asm/alternative.h |  1 +
>>   arch/arm64/include/asm/cpufeature.h  |  4 ++++
>>   arch/arm64/kernel/alternative.c      | 43
>> +++++++++++++++++++++++++++++++-----
>>   arch/arm64/kernel/cpufeature.c       |  6 +++++
>>   arch/arm64/kernel/smp.c              |  7 ++++++
>>   5 files changed, 56 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/alternative.h
>> b/arch/arm64/include/asm/alternative.h
>> index 9806a23..b9f8d78 100644
>> --- a/arch/arm64/include/asm/alternative.h
>> +++ b/arch/arm64/include/asm/alternative.h
>> @@ -25,6 +25,7 @@ struct alt_instr {
>>   typedef void (*alternative_cb_t)(struct alt_instr *alt,
>>                    __le32 *origptr, __le32 *updptr, int nr_inst);
>>   +void __init apply_boot_alternatives(void);
>>   void __init apply_alternatives_all(void);
>>   bool alternative_is_applied(u16 cpufeature);
>>   diff --git a/arch/arm64/include/asm/cpufeature.h
>> b/arch/arm64/include/asm/cpufeature.h
>> index 89c3f31..e505e1f 100644
>> --- a/arch/arm64/include/asm/cpufeature.h
>> +++ b/arch/arm64/include/asm/cpufeature.h
>> @@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const
>> struct arm64_cpu_capabilities *cap)
>>   extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
>>   extern struct static_key_false arm64_const_caps_ready;
>>   +/* ARM64 CAPS + alternative_cb */
>> +#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
>> +extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>> +
>>   #define for_each_available_cap(cap)        \
>>       for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
>>   diff --git a/arch/arm64/kernel/alternative.c
>> b/arch/arm64/kernel/alternative.c
>> index c947d22..a9b4677 100644
>> --- a/arch/arm64/kernel/alternative.c
>> +++ b/arch/arm64/kernel/alternative.c
>> @@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start,
>> u64 end)
>>       } while (cur += d_size, cur < end);
>>   }
>>   -static void __apply_alternatives(void *alt_region, bool is_module)
>> +static void __apply_alternatives(void *alt_region,  bool is_module,
>> +                 unsigned long *feature_mask)
>>   {
>>       struct alt_instr *alt;
>>       struct alt_region *region = alt_region;
>> @@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region,
>> bool is_module)
>>       for (alt = region->begin; alt < region->end; alt++) {
>>           int nr_inst;
>>   +        if (!test_bit(alt->cpufeature, feature_mask))
>> +            continue;
>> +
>>           /* Use ARM64_CB_PATCH as an unconditional patch */
>>           if (alt->cpufeature < ARM64_CB_PATCH &&
>>               !cpus_have_cap(alt->cpufeature))
>> @@ -203,8 +207,11 @@ static void __apply_alternatives(void
>> *alt_region, bool is_module)
>>           __flush_icache_all();
>>           isb();
>>   -        /* We applied all that was available */
>> -        bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
>> +        /* Ignore ARM64_CB bit from feature mask */
>> +        bitmap_or(applied_alternatives, applied_alternatives,
>> +              feature_mask, ARM64_NCAPS);
>> +        bitmap_and(applied_alternatives, applied_alternatives,
>> +               cpu_hwcaps, ARM64_NCAPS);
>>       }
>>   }
>>   @@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void
>> *unused)
>>               cpu_relax();
>>           isb();
>>       } else {
>> +        DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
>> +
>> +        bitmap_complement(remaining_capabilities, boot_capabilities,
>> +                  ARM64_NPATCHABLE);
>> +
>>           BUG_ON(all_alternatives_applied);
>> -        __apply_alternatives(&region, false);
>> +        __apply_alternatives(&region, false, remaining_capabilities);
>>           /* Barriers provided by the cache flushing */
>>           WRITE_ONCE(all_alternatives_applied, 1);
>>       }
>> @@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
>>       stop_machine(__apply_alternatives_multi_stop, NULL,
>> cpu_online_mask);
>>   }
>>   +/*
>> + * This is called very early in the boot process (directly after we run
>> + * a feature detect on the boot CPU). No need to worry about other CPUs
>> + * here.
>> + */
>> +void __init apply_boot_alternatives(void)
>> +{
>> +    struct alt_region region = {
>> +        .begin    = (struct alt_instr *)__alt_instructions,
>> +        .end    = (struct alt_instr *)__alt_instructions_end,
>> +    };
>> +
>> +    /* If called on non-boot cpu things could go wrong */
>> +    WARN_ON(smp_processor_id() != 0);
>> +
>> +    __apply_alternatives(&region, false, &boot_capabilities[0]);
>> +}
>> +
>>   #ifdef CONFIG_MODULES
>>   void apply_alternatives_module(void *start, size_t length)
>>   {
>> @@ -247,7 +277,10 @@ void apply_alternatives_module(void *start,
>> size_t length)
>>           .begin    = start,
>>           .end    = start + length,
>>       };
>> +    DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
>> +
>> +    bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
>>   -    __apply_alternatives(&region, true);
>> +    __apply_alternatives(&region, true, &all_capabilities[0]);
>>   }
>>   #endif
>> diff --git a/arch/arm64/kernel/cpufeature.c
>> b/arch/arm64/kernel/cpufeature.c
>> index 84fa5be..71c8d4f 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -54,6 +54,9 @@
>>   EXPORT_SYMBOL(cpu_hwcaps);
>>   static struct arm64_cpu_capabilities const __ro_after_init
>> *cpu_hwcaps_ptrs[ARM64_NCAPS];
>>   +/* Need also bit for ARM64_CB_PATCH */
>> +DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>> +
>>   /*
>>    * Flag to indicate if we have computed the system wide
>>    * capabilities based on the boot time active CPUs. This
>> @@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16 scope_mask)
>>           if (caps->desc)
>>               pr_info("detected: %s\n", caps->desc);
>>           cpus_set_cap(caps->capability);
>> +
>> +        if (caps->type & SCOPE_BOOT_CPU)
> 
> You may want to do :
>         if (scope_mask & SCOPE_BOOT_CPU)
> 
> for a tighter check to ensure this doesn't update the boot_capabilities
> after we have applied the boot_scope alternatives and miss applying the
> alternatives for those, should someone add a multi-scope (i.e
> SCOPE_BOOT_CPU and
> something else) capability (even by mistake).
> 

But a multi-scope capability containing SCOPE_BOOT_CPU should already
get updated for setup_boot_cpu_capabilities. Capabilities marked with
SCOPE_BOOT_CPU need to be enabled on the boot CPU or not at all.

Shouldn't the call to caps->matches() fail for a boot feature that was
not found on the boot cpu?

Also, you made the opposite suggestion 4 version ago with a more
worrying scenario :) :
https://lkml.org/lkml/2018/5/25/208

Otherwise, if my assumption above is wrong, it means the check should
probably be:
	if (caps->type & SCOPE_BOOT_CPU && scope_mask & SCOPE_BOOT_CPU)

But my current understanding is that we don't need that.

> With that:
> 
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Let me know if I can keep your tag or if I indeed need to change the
condition.

Thanks,

-- 
Julien Thierry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> Instead disabling interrupts by setting the PSR.I bit, use a priority
> higher than the one used for interrupts to mask them via PMR.
> 
> When using PMR to disable interrupts, the value of PMR will be used
> instead of PSR.[DAIF] for the irqflags.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> ---
>  arch/arm64/include/asm/efi.h      |  11 ++++
>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>  2 files changed, 106 insertions(+), 28 deletions(-)

[...]

> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> index 24692ed..fa3b06f 100644
> --- a/arch/arm64/include/asm/irqflags.h
> +++ b/arch/arm64/include/asm/irqflags.h
> @@ -18,7 +18,9 @@

[...]

>  static inline void arch_local_irq_enable(void)
>  {
> -	asm volatile(
> -		"msr	daifclr, #2		// arch_local_irq_enable"
> -		:
> +	unsigned long unmasked = GIC_PRIO_IRQON;
> +
> +	asm volatile(ALTERNATIVE(
> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
> +		"nop",
> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
> +		"dsb	sy",

I'm still not convinced these dsbs are needed.

Without the dsb, we are probably not guaranteed to take a pending
interrupt _immediately_ on unmasking, but I'm not sure that's a
problem.

What goes wrong if we omit them?

(My attempts to answer these questions using the GIC architecture spec
have met with limited success so far...)

[...]

Cheers
---Dave

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> Instead disabling interrupts by setting the PSR.I bit, use a priority
> higher than the one used for interrupts to mask them via PMR.
> 
> When using PMR to disable interrupts, the value of PMR will be used
> instead of PSR.[DAIF] for the irqflags.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> ---
>  arch/arm64/include/asm/efi.h      |  11 ++++
>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>  2 files changed, 106 insertions(+), 28 deletions(-)

[...]

> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> index 24692ed..fa3b06f 100644
> --- a/arch/arm64/include/asm/irqflags.h
> +++ b/arch/arm64/include/asm/irqflags.h
> @@ -18,7 +18,9 @@

[...]

>  static inline void arch_local_irq_enable(void)
>  {
> -	asm volatile(
> -		"msr	daifclr, #2		// arch_local_irq_enable"
> -		:
> +	unsigned long unmasked = GIC_PRIO_IRQON;
> +
> +	asm volatile(ALTERNATIVE(
> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
> +		"nop",
> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
> +		"dsb	sy",

I'm still not convinced these dsbs are needed.

Without the dsb, we are probably not guaranteed to take a pending
interrupt _immediately_ on unmasking, but I'm not sure that's a
problem.

What goes wrong if we omit them?

(My attempts to answer these questions using the GIC architecture spec
have met with limited success so far...)

[...]

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Marc Zyngier]

On 08/01/2019 15:40, Dave Martin wrote:
> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>> Instead disabling interrupts by setting the PSR.I bit, use a priority
>> higher than the one used for interrupts to mask them via PMR.
>>
>> When using PMR to disable interrupts, the value of PMR will be used
>> instead of PSR.[DAIF] for the irqflags.
>>
>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Cc: Oleg Nesterov <oleg@redhat.com>
>> ---
>>  arch/arm64/include/asm/efi.h      |  11 ++++
>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>>  2 files changed, 106 insertions(+), 28 deletions(-)
> 
> [...]
> 
>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
>> index 24692ed..fa3b06f 100644
>> --- a/arch/arm64/include/asm/irqflags.h
>> +++ b/arch/arm64/include/asm/irqflags.h
>> @@ -18,7 +18,9 @@
> 
> [...]
> 
>>  static inline void arch_local_irq_enable(void)
>>  {
>> -	asm volatile(
>> -		"msr	daifclr, #2		// arch_local_irq_enable"
>> -		:
>> +	unsigned long unmasked = GIC_PRIO_IRQON;
>> +
>> +	asm volatile(ALTERNATIVE(
>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
>> +		"nop",
>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
>> +		"dsb	sy",
> 
> I'm still not convinced these dsbs are needed.
> 
> Without the dsb, we are probably not guaranteed to take a pending
> interrupt _immediately_ on unmasking, but I'm not sure that's a
> problem.
> 
> What goes wrong if we omit them?

Then the GIC doesn't know it can now deliver interrupts of a lower
priority. Only a dsb can guarantee that the GIC's view of PMR will get
updated.

See 9.1.6 (Observability of the effects of accesses to the GIC
registers), which states:

<quote>
Architectural execution of a DSB instruction guarantees that
— The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
associated Redistributor.
</quote>

So yes, DSB is required.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Marc Zyngier]

On 08/01/2019 15:40, Dave Martin wrote:
> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>> Instead disabling interrupts by setting the PSR.I bit, use a priority
>> higher than the one used for interrupts to mask them via PMR.
>>
>> When using PMR to disable interrupts, the value of PMR will be used
>> instead of PSR.[DAIF] for the irqflags.
>>
>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Cc: Oleg Nesterov <oleg@redhat.com>
>> ---
>>  arch/arm64/include/asm/efi.h      |  11 ++++
>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>>  2 files changed, 106 insertions(+), 28 deletions(-)
> 
> [...]
> 
>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
>> index 24692ed..fa3b06f 100644
>> --- a/arch/arm64/include/asm/irqflags.h
>> +++ b/arch/arm64/include/asm/irqflags.h
>> @@ -18,7 +18,9 @@
> 
> [...]
> 
>>  static inline void arch_local_irq_enable(void)
>>  {
>> -	asm volatile(
>> -		"msr	daifclr, #2		// arch_local_irq_enable"
>> -		:
>> +	unsigned long unmasked = GIC_PRIO_IRQON;
>> +
>> +	asm volatile(ALTERNATIVE(
>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
>> +		"nop",
>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
>> +		"dsb	sy",
> 
> I'm still not convinced these dsbs are needed.
> 
> Without the dsb, we are probably not guaranteed to take a pending
> interrupt _immediately_ on unmasking, but I'm not sure that's a
> problem.
> 
> What goes wrong if we omit them?

Then the GIC doesn't know it can now deliver interrupts of a lower
priority. Only a dsb can guarantee that the GIC's view of PMR will get
updated.

See 9.1.6 (Observability of the effects of accesses to the GIC
registers), which states:

<quote>
Architectural execution of a DSB instruction guarantees that
— The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
associated Redistributor.
</quote>

So yes, DSB is required.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
> On 08/01/2019 15:40, Dave Martin wrote:
> > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >> Instead disabling interrupts by setting the PSR.I bit, use a priority
> >> higher than the one used for interrupts to mask them via PMR.
> >>
> >> When using PMR to disable interrupts, the value of PMR will be used
> >> instead of PSR.[DAIF] for the irqflags.
> >>
> >> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> >> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> >> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >> Cc: Will Deacon <will.deacon@arm.com>
> >> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >> Cc: Oleg Nesterov <oleg@redhat.com>
> >> ---
> >>  arch/arm64/include/asm/efi.h      |  11 ++++
> >>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
> >>  2 files changed, 106 insertions(+), 28 deletions(-)
> > 
> > [...]
> > 
> >> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> >> index 24692ed..fa3b06f 100644
> >> --- a/arch/arm64/include/asm/irqflags.h
> >> +++ b/arch/arm64/include/asm/irqflags.h
> >> @@ -18,7 +18,9 @@
> > 
> > [...]
> > 
> >>  static inline void arch_local_irq_enable(void)
> >>  {
> >> -	asm volatile(
> >> -		"msr	daifclr, #2		// arch_local_irq_enable"
> >> -		:
> >> +	unsigned long unmasked = GIC_PRIO_IRQON;
> >> +
> >> +	asm volatile(ALTERNATIVE(
> >> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
> >> +		"nop",
> >> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
> >> +		"dsb	sy",
> > 
> > I'm still not convinced these dsbs are needed.
> > 
> > Without the dsb, we are probably not guaranteed to take a pending
> > interrupt _immediately_ on unmasking, but I'm not sure that's a
> > problem.
> > 
> > What goes wrong if we omit them?
> 
> Then the GIC doesn't know it can now deliver interrupts of a lower
> priority. Only a dsb can guarantee that the GIC's view of PMR will get
> updated.
> 
> See 9.1.6 (Observability of the effects of accesses to the GIC
> registers), which states:
> 
> <quote>
> Architectural execution of a DSB instruction guarantees that
> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
> associated Redistributor.
> </quote>
> 
> So yes, DSB is required.

But it says neither what is means for the PMR write to be "observed by
the redistributor", nor whether the DSB is required for the
redistributor to observe the write at all.  (So, is an implementation
allowed to cached in the CPU interface indefinitely until forcibly
flushed to the redistributor by a DSB, and in any case can the write's
reaching the distributor in finite time or not have any effect that we
care about in this case?).


My reason for querying this is that temporary local masking of classes
of interrupts seems an obvious use case for the PMR, and the DSB
requirement flies rather in the face of this.


Have we seen hardware where interrupts may stall forever upstream of the
CPU interface after a PMR write, until a dsb is executed by the CPU?

If so that is sad, but I guess we have to live with it.

Also, is it ever important in Linux that a pending interrupt be taken
immediately upon unmasking (and how do we know that said interrupt is
pending)?  If not, we don't care precisely when such interrupts are
pended to the PE, just that such an interrupt cannot be taken before
the PMR write that unmasks it.  It would be insane for the self-
synchronization of PMR writes to lack this guarantee (and a DSB after
the PMR write would do no good anyway in that case).

Happy to be put right -- I'm doubtless showing my ignorance here!

Cheers
---Dave

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
> On 08/01/2019 15:40, Dave Martin wrote:
> > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >> Instead disabling interrupts by setting the PSR.I bit, use a priority
> >> higher than the one used for interrupts to mask them via PMR.
> >>
> >> When using PMR to disable interrupts, the value of PMR will be used
> >> instead of PSR.[DAIF] for the irqflags.
> >>
> >> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> >> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> >> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >> Cc: Will Deacon <will.deacon@arm.com>
> >> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >> Cc: Oleg Nesterov <oleg@redhat.com>
> >> ---
> >>  arch/arm64/include/asm/efi.h      |  11 ++++
> >>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
> >>  2 files changed, 106 insertions(+), 28 deletions(-)
> > 
> > [...]
> > 
> >> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> >> index 24692ed..fa3b06f 100644
> >> --- a/arch/arm64/include/asm/irqflags.h
> >> +++ b/arch/arm64/include/asm/irqflags.h
> >> @@ -18,7 +18,9 @@
> > 
> > [...]
> > 
> >>  static inline void arch_local_irq_enable(void)
> >>  {
> >> -	asm volatile(
> >> -		"msr	daifclr, #2		// arch_local_irq_enable"
> >> -		:
> >> +	unsigned long unmasked = GIC_PRIO_IRQON;
> >> +
> >> +	asm volatile(ALTERNATIVE(
> >> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
> >> +		"nop",
> >> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
> >> +		"dsb	sy",
> > 
> > I'm still not convinced these dsbs are needed.
> > 
> > Without the dsb, we are probably not guaranteed to take a pending
> > interrupt _immediately_ on unmasking, but I'm not sure that's a
> > problem.
> > 
> > What goes wrong if we omit them?
> 
> Then the GIC doesn't know it can now deliver interrupts of a lower
> priority. Only a dsb can guarantee that the GIC's view of PMR will get
> updated.
> 
> See 9.1.6 (Observability of the effects of accesses to the GIC
> registers), which states:
> 
> <quote>
> Architectural execution of a DSB instruction guarantees that
> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
> associated Redistributor.
> </quote>
> 
> So yes, DSB is required.

But it says neither what is means for the PMR write to be "observed by
the redistributor", nor whether the DSB is required for the
redistributor to observe the write at all.  (So, is an implementation
allowed to cached in the CPU interface indefinitely until forcibly
flushed to the redistributor by a DSB, and in any case can the write's
reaching the distributor in finite time or not have any effect that we
care about in this case?).


My reason for querying this is that temporary local masking of classes
of interrupts seems an obvious use case for the PMR, and the DSB
requirement flies rather in the face of this.


Have we seen hardware where interrupts may stall forever upstream of the
CPU interface after a PMR write, until a dsb is executed by the CPU?

If so that is sad, but I guess we have to live with it.

Also, is it ever important in Linux that a pending interrupt be taken
immediately upon unmasking (and how do we know that said interrupt is
pending)?  If not, we don't care precisely when such interrupts are
pended to the PE, just that such an interrupt cannot be taken before
the PMR write that unmasks it.  It would be insane for the self-
synchronization of PMR writes to lack this guarantee (and a DSB after
the PMR write would do no good anyway in that case).

Happy to be put right -- I'm doubtless showing my ignorance here!

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Marc Zyngier]

On 08/01/2019 16:45, Dave Martin wrote:
> On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
>> On 08/01/2019 15:40, Dave Martin wrote:
>>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>>>> Instead disabling interrupts by setting the PSR.I bit, use a priority
>>>> higher than the one used for interrupts to mask them via PMR.
>>>>
>>>> When using PMR to disable interrupts, the value of PMR will be used
>>>> instead of PSR.[DAIF] for the irqflags.
>>>>
>>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>>>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
>>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>>>> Cc: Will Deacon <will.deacon@arm.com>
>>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>> Cc: Oleg Nesterov <oleg@redhat.com>
>>>> ---
>>>>  arch/arm64/include/asm/efi.h      |  11 ++++
>>>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>>>>  2 files changed, 106 insertions(+), 28 deletions(-)
>>>
>>> [...]
>>>
>>>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
>>>> index 24692ed..fa3b06f 100644
>>>> --- a/arch/arm64/include/asm/irqflags.h
>>>> +++ b/arch/arm64/include/asm/irqflags.h
>>>> @@ -18,7 +18,9 @@
>>>
>>> [...]
>>>
>>>>  static inline void arch_local_irq_enable(void)
>>>>  {
>>>> -	asm volatile(
>>>> -		"msr	daifclr, #2		// arch_local_irq_enable"
>>>> -		:
>>>> +	unsigned long unmasked = GIC_PRIO_IRQON;
>>>> +
>>>> +	asm volatile(ALTERNATIVE(
>>>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
>>>> +		"nop",
>>>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
>>>> +		"dsb	sy",
>>>
>>> I'm still not convinced these dsbs are needed.
>>>
>>> Without the dsb, we are probably not guaranteed to take a pending
>>> interrupt _immediately_ on unmasking, but I'm not sure that's a
>>> problem.
>>>
>>> What goes wrong if we omit them?
>>
>> Then the GIC doesn't know it can now deliver interrupts of a lower
>> priority. Only a dsb can guarantee that the GIC's view of PMR will get
>> updated.
>>
>> See 9.1.6 (Observability of the effects of accesses to the GIC
>> registers), which states:
>>
>> <quote>
>> Architectural execution of a DSB instruction guarantees that
>> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
>> associated Redistributor.
>> </quote>
>>
>> So yes, DSB is required.
> 
> But it says neither what is means for the PMR write to be "observed by
> the redistributor", nor whether the DSB is required for the
> redistributor to observe the write at all.

Well, it seems pretty clear to me that if the redistributor doesn't
observe the PMR value, it is unlikely to change its interpretation of
it· And conversely, the redistributor is allowed to sit pretty and not
give you any interrupt until you are actually telling it that something
has changed.

I really think that for once, the spec is pretty unambiguous about what
is required.

> (So, is an implementation
> allowed to cached in the CPU interface indefinitely until forcibly
> flushed to the redistributor by a DSB, and in any case can the write's
> reaching the distributor in finite time or not have any effect that we
> care about in this case?).

Nothing in the spec says that the system register write will magically
trickle down to the redistributor in the absence of a DSB.

> My reason for querying this is that temporary local masking of classes
> of interrupts seems an obvious use case for the PMR, and the DSB
> requirement flies rather in the face of this.

Are you implying that the GIC architecture should have any form of
sanity and be useful for general purpose software? Think again! ;-)

The PMR behavior you are describing only works in a single direction
(from low to high priority), because the CPU interface has to perform
some filtering. In the opposite direction, you need the big hammer.

> Have we seen hardware where interrupts may stall forever upstream of the
> CPU interface after a PMR write, until a dsb is executed by the CPU?

Yes. You even have to have a DSB right after a read of IAR to avoid
loosing interrupts. The short story is that there is hardly any
synchronization between redistributor and CPU interface. Implementations
are allowed a more closely coupled design, but that's not what the
architecture mandates.

> If so that is sad, but I guess we have to live with it.
> 
> Also, is it ever important in Linux that a pending interrupt be taken
> immediately upon unmasking (and how do we know that said interrupt is
> pending)?  If not, we don't care precisely when such interrupts are
> pended to the PE, just that such an interrupt cannot be taken before
> the PMR write that unmasks it.  It would be insane for the self-
> synchronization of PMR writes to lack this guarantee (and a DSB after
> the PMR write would do no good anyway in that case).

RT folks are usually quite picky on when they see their interrupts
firing. I can also imagine the following scenario:

	set_pmr(allow all interrupts)
	WFI

where things stop rather abruptly if this is the only CPU in the system.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Marc Zyngier]

On 08/01/2019 16:45, Dave Martin wrote:
> On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
>> On 08/01/2019 15:40, Dave Martin wrote:
>>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>>>> Instead disabling interrupts by setting the PSR.I bit, use a priority
>>>> higher than the one used for interrupts to mask them via PMR.
>>>>
>>>> When using PMR to disable interrupts, the value of PMR will be used
>>>> instead of PSR.[DAIF] for the irqflags.
>>>>
>>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>>>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
>>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>>>> Cc: Will Deacon <will.deacon@arm.com>
>>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>> Cc: Oleg Nesterov <oleg@redhat.com>
>>>> ---
>>>>  arch/arm64/include/asm/efi.h      |  11 ++++
>>>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>>>>  2 files changed, 106 insertions(+), 28 deletions(-)
>>>
>>> [...]
>>>
>>>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
>>>> index 24692ed..fa3b06f 100644
>>>> --- a/arch/arm64/include/asm/irqflags.h
>>>> +++ b/arch/arm64/include/asm/irqflags.h
>>>> @@ -18,7 +18,9 @@
>>>
>>> [...]
>>>
>>>>  static inline void arch_local_irq_enable(void)
>>>>  {
>>>> -	asm volatile(
>>>> -		"msr	daifclr, #2		// arch_local_irq_enable"
>>>> -		:
>>>> +	unsigned long unmasked = GIC_PRIO_IRQON;
>>>> +
>>>> +	asm volatile(ALTERNATIVE(
>>>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
>>>> +		"nop",
>>>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
>>>> +		"dsb	sy",
>>>
>>> I'm still not convinced these dsbs are needed.
>>>
>>> Without the dsb, we are probably not guaranteed to take a pending
>>> interrupt _immediately_ on unmasking, but I'm not sure that's a
>>> problem.
>>>
>>> What goes wrong if we omit them?
>>
>> Then the GIC doesn't know it can now deliver interrupts of a lower
>> priority. Only a dsb can guarantee that the GIC's view of PMR will get
>> updated.
>>
>> See 9.1.6 (Observability of the effects of accesses to the GIC
>> registers), which states:
>>
>> <quote>
>> Architectural execution of a DSB instruction guarantees that
>> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
>> associated Redistributor.
>> </quote>
>>
>> So yes, DSB is required.
> 
> But it says neither what is means for the PMR write to be "observed by
> the redistributor", nor whether the DSB is required for the
> redistributor to observe the write at all.

Well, it seems pretty clear to me that if the redistributor doesn't
observe the PMR value, it is unlikely to change its interpretation of
it· And conversely, the redistributor is allowed to sit pretty and not
give you any interrupt until you are actually telling it that something
has changed.

I really think that for once, the spec is pretty unambiguous about what
is required.

> (So, is an implementation
> allowed to cached in the CPU interface indefinitely until forcibly
> flushed to the redistributor by a DSB, and in any case can the write's
> reaching the distributor in finite time or not have any effect that we
> care about in this case?).

Nothing in the spec says that the system register write will magically
trickle down to the redistributor in the absence of a DSB.

> My reason for querying this is that temporary local masking of classes
> of interrupts seems an obvious use case for the PMR, and the DSB
> requirement flies rather in the face of this.

Are you implying that the GIC architecture should have any form of
sanity and be useful for general purpose software? Think again! ;-)

The PMR behavior you are describing only works in a single direction
(from low to high priority), because the CPU interface has to perform
some filtering. In the opposite direction, you need the big hammer.

> Have we seen hardware where interrupts may stall forever upstream of the
> CPU interface after a PMR write, until a dsb is executed by the CPU?

Yes. You even have to have a DSB right after a read of IAR to avoid
loosing interrupts. The short story is that there is hardly any
synchronization between redistributor and CPU interface. Implementations
are allowed a more closely coupled design, but that's not what the
architecture mandates.

> If so that is sad, but I guess we have to live with it.
> 
> Also, is it ever important in Linux that a pending interrupt be taken
> immediately upon unmasking (and how do we know that said interrupt is
> pending)?  If not, we don't care precisely when such interrupts are
> pended to the PE, just that such an interrupt cannot be taken before
> the PMR write that unmasks it.  It would be insane for the self-
> synchronization of PMR writes to lack this guarantee (and a DSB after
> the PMR write would do no good anyway in that case).

RT folks are usually quite picky on when they see their interrupts
firing. I can also imagine the following scenario:

	set_pmr(allow all interrupts)
	WFI

where things stop rather abruptly if this is the only CPU in the system.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Suzuki K Poulose]

On 08/01/2019 15:20, Julien Thierry wrote:
> Hi Suzuki,
> 
> On 08/01/2019 14:51, Suzuki K Poulose wrote:
>> Hi Julien,
>>
>> On 08/01/2019 14:07, Julien Thierry wrote:
>>> From: Daniel Thompson <daniel.thompson@linaro.org>
>>>
>>> Currently alternatives are applied very late in the boot process (and
>>> a long time after we enable scheduling). Some alternative sequences,
>>> such as those that alter the way CPU context is stored, must be applied
>>> much earlier in the boot sequence.
>>>
>>> Introduce apply_boot_alternatives() to allow some alternatives to be
>>> applied immediately after we detect the CPU features of the boot CPU.
>>>
>>> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
>>> [julien.thierry@arm.com: rename to fit new cpufeature framework better,
>>>               apply BOOT_SCOPE feature early in boot]
>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>>> Cc: Will Deacon <will.deacon@arm.com>
>>> Cc: Christoffer Dall <christoffer.dall@arm.com>
>>> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
>>> ---
>>>    arch/arm64/include/asm/alternative.h |  1 +
>>>    arch/arm64/include/asm/cpufeature.h  |  4 ++++
>>>    arch/arm64/kernel/alternative.c      | 43
>>> +++++++++++++++++++++++++++++++-----
>>>    arch/arm64/kernel/cpufeature.c       |  6 +++++
>>>    arch/arm64/kernel/smp.c              |  7 ++++++
>>>    5 files changed, 56 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/alternative.h
>>> b/arch/arm64/include/asm/alternative.h
>>> index 9806a23..b9f8d78 100644
>>> --- a/arch/arm64/include/asm/alternative.h
>>> +++ b/arch/arm64/include/asm/alternative.h
>>> @@ -25,6 +25,7 @@ struct alt_instr {
>>>    typedef void (*alternative_cb_t)(struct alt_instr *alt,
>>>                     __le32 *origptr, __le32 *updptr, int nr_inst);
>>>    +void __init apply_boot_alternatives(void);
>>>    void __init apply_alternatives_all(void);
>>>    bool alternative_is_applied(u16 cpufeature);
>>>    diff --git a/arch/arm64/include/asm/cpufeature.h
>>> b/arch/arm64/include/asm/cpufeature.h
>>> index 89c3f31..e505e1f 100644
>>> --- a/arch/arm64/include/asm/cpufeature.h
>>> +++ b/arch/arm64/include/asm/cpufeature.h
>>> @@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const
>>> struct arm64_cpu_capabilities *cap)
>>>    extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
>>>    extern struct static_key_false arm64_const_caps_ready;
>>>    +/* ARM64 CAPS + alternative_cb */
>>> +#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
>>> +extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>>> +
>>>    #define for_each_available_cap(cap)        \
>>>        for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
>>>    diff --git a/arch/arm64/kernel/alternative.c
>>> b/arch/arm64/kernel/alternative.c
>>> index c947d22..a9b4677 100644
>>> --- a/arch/arm64/kernel/alternative.c
>>> +++ b/arch/arm64/kernel/alternative.c
>>> @@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start,
>>> u64 end)
>>>        } while (cur += d_size, cur < end);
>>>    }
>>>    -static void __apply_alternatives(void *alt_region, bool is_module)
>>> +static void __apply_alternatives(void *alt_region,  bool is_module,
>>> +                 unsigned long *feature_mask)
>>>    {
>>>        struct alt_instr *alt;
>>>        struct alt_region *region = alt_region;
>>> @@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region,
>>> bool is_module)
>>>        for (alt = region->begin; alt < region->end; alt++) {
>>>            int nr_inst;
>>>    +        if (!test_bit(alt->cpufeature, feature_mask))
>>> +            continue;
>>> +
>>>            /* Use ARM64_CB_PATCH as an unconditional patch */
>>>            if (alt->cpufeature < ARM64_CB_PATCH &&
>>>                !cpus_have_cap(alt->cpufeature))
>>> @@ -203,8 +207,11 @@ static void __apply_alternatives(void
>>> *alt_region, bool is_module)
>>>            __flush_icache_all();
>>>            isb();
>>>    -        /* We applied all that was available */
>>> -        bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
>>> +        /* Ignore ARM64_CB bit from feature mask */
>>> +        bitmap_or(applied_alternatives, applied_alternatives,
>>> +              feature_mask, ARM64_NCAPS);
>>> +        bitmap_and(applied_alternatives, applied_alternatives,
>>> +               cpu_hwcaps, ARM64_NCAPS);
>>>        }
>>>    }
>>>    @@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void
>>> *unused)
>>>                cpu_relax();
>>>            isb();
>>>        } else {
>>> +        DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
>>> +
>>> +        bitmap_complement(remaining_capabilities, boot_capabilities,
>>> +                  ARM64_NPATCHABLE);
>>> +
>>>            BUG_ON(all_alternatives_applied);
>>> -        __apply_alternatives(&region, false);
>>> +        __apply_alternatives(&region, false, remaining_capabilities);
>>>            /* Barriers provided by the cache flushing */
>>>            WRITE_ONCE(all_alternatives_applied, 1);
>>>        }
>>> @@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
>>>        stop_machine(__apply_alternatives_multi_stop, NULL,
>>> cpu_online_mask);
>>>    }
>>>    +/*
>>> + * This is called very early in the boot process (directly after we run
>>> + * a feature detect on the boot CPU). No need to worry about other CPUs
>>> + * here.
>>> + */
>>> +void __init apply_boot_alternatives(void)
>>> +{
>>> +    struct alt_region region = {
>>> +        .begin    = (struct alt_instr *)__alt_instructions,
>>> +        .end    = (struct alt_instr *)__alt_instructions_end,
>>> +    };
>>> +
>>> +    /* If called on non-boot cpu things could go wrong */
>>> +    WARN_ON(smp_processor_id() != 0);
>>> +
>>> +    __apply_alternatives(&region, false, &boot_capabilities[0]);
>>> +}
>>> +
>>>    #ifdef CONFIG_MODULES
>>>    void apply_alternatives_module(void *start, size_t length)
>>>    {
>>> @@ -247,7 +277,10 @@ void apply_alternatives_module(void *start,
>>> size_t length)
>>>            .begin    = start,
>>>            .end    = start + length,
>>>        };
>>> +    DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
>>> +
>>> +    bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
>>>    -    __apply_alternatives(&region, true);
>>> +    __apply_alternatives(&region, true, &all_capabilities[0]);
>>>    }
>>>    #endif
>>> diff --git a/arch/arm64/kernel/cpufeature.c
>>> b/arch/arm64/kernel/cpufeature.c
>>> index 84fa5be..71c8d4f 100644
>>> --- a/arch/arm64/kernel/cpufeature.c
>>> +++ b/arch/arm64/kernel/cpufeature.c
>>> @@ -54,6 +54,9 @@
>>>    EXPORT_SYMBOL(cpu_hwcaps);
>>>    static struct arm64_cpu_capabilities const __ro_after_init
>>> *cpu_hwcaps_ptrs[ARM64_NCAPS];
>>>    +/* Need also bit for ARM64_CB_PATCH */
>>> +DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>>> +
>>>    /*
>>>     * Flag to indicate if we have computed the system wide
>>>     * capabilities based on the boot time active CPUs. This
>>> @@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16 scope_mask)
>>>            if (caps->desc)
>>>                pr_info("detected: %s\n", caps->desc);
>>>            cpus_set_cap(caps->capability);
>>> +
>>> +        if (caps->type & SCOPE_BOOT_CPU)
>>
>> You may want to do :
>>          if (scope_mask & SCOPE_BOOT_CPU)
>>
>> for a tighter check to ensure this doesn't update the boot_capabilities
>> after we have applied the boot_scope alternatives and miss applying the
>> alternatives for those, should someone add a multi-scope (i.e
>> SCOPE_BOOT_CPU and
>> something else) capability (even by mistake).
>>
> 
> But a multi-scope capability containing SCOPE_BOOT_CPU should already
> get updated for setup_boot_cpu_capabilities. Capabilities marked with
> SCOPE_BOOT_CPU need to be enabled on the boot CPU or not at all.

Yes, you're right. It is not normal to have multiple SCOPE for a "capability".
But if someone comes with such a cap, we may miss handling this case. It is
always better to be safer.

> 
> Shouldn't the call to caps->matches() fail for a boot feature that was
> not found on the boot cpu?
> 
> Also, you made the opposite suggestion 4 version ago with a more
> worrying scenario :) :
> https://lkml.org/lkml/2018/5/25/208

Ah, you're right. I missed that. We need the additional check as you mention
below.

> 
> Otherwise, if my assumption above is wrong, it means the check should
> probably be:
> 	if (caps->type & SCOPE_BOOT_CPU && scope_mask & SCOPE_BOOT_CPU)

Yes, this is what we want.


> But my current understanding is that we don't need that.
> 
>> With that:
>>
>> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>


Cheers
Suzuki

Re: [PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Suzuki K Poulose]

On 08/01/2019 15:20, Julien Thierry wrote:
> Hi Suzuki,
> 
> On 08/01/2019 14:51, Suzuki K Poulose wrote:
>> Hi Julien,
>>
>> On 08/01/2019 14:07, Julien Thierry wrote:
>>> From: Daniel Thompson <daniel.thompson@linaro.org>
>>>
>>> Currently alternatives are applied very late in the boot process (and
>>> a long time after we enable scheduling). Some alternative sequences,
>>> such as those that alter the way CPU context is stored, must be applied
>>> much earlier in the boot sequence.
>>>
>>> Introduce apply_boot_alternatives() to allow some alternatives to be
>>> applied immediately after we detect the CPU features of the boot CPU.
>>>
>>> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
>>> [julien.thierry@arm.com: rename to fit new cpufeature framework better,
>>>               apply BOOT_SCOPE feature early in boot]
>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>>> Cc: Will Deacon <will.deacon@arm.com>
>>> Cc: Christoffer Dall <christoffer.dall@arm.com>
>>> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
>>> ---
>>>    arch/arm64/include/asm/alternative.h |  1 +
>>>    arch/arm64/include/asm/cpufeature.h  |  4 ++++
>>>    arch/arm64/kernel/alternative.c      | 43
>>> +++++++++++++++++++++++++++++++-----
>>>    arch/arm64/kernel/cpufeature.c       |  6 +++++
>>>    arch/arm64/kernel/smp.c              |  7 ++++++
>>>    5 files changed, 56 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/alternative.h
>>> b/arch/arm64/include/asm/alternative.h
>>> index 9806a23..b9f8d78 100644
>>> --- a/arch/arm64/include/asm/alternative.h
>>> +++ b/arch/arm64/include/asm/alternative.h
>>> @@ -25,6 +25,7 @@ struct alt_instr {
>>>    typedef void (*alternative_cb_t)(struct alt_instr *alt,
>>>                     __le32 *origptr, __le32 *updptr, int nr_inst);
>>>    +void __init apply_boot_alternatives(void);
>>>    void __init apply_alternatives_all(void);
>>>    bool alternative_is_applied(u16 cpufeature);
>>>    diff --git a/arch/arm64/include/asm/cpufeature.h
>>> b/arch/arm64/include/asm/cpufeature.h
>>> index 89c3f31..e505e1f 100644
>>> --- a/arch/arm64/include/asm/cpufeature.h
>>> +++ b/arch/arm64/include/asm/cpufeature.h
>>> @@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const
>>> struct arm64_cpu_capabilities *cap)
>>>    extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
>>>    extern struct static_key_false arm64_const_caps_ready;
>>>    +/* ARM64 CAPS + alternative_cb */
>>> +#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
>>> +extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>>> +
>>>    #define for_each_available_cap(cap)        \
>>>        for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
>>>    diff --git a/arch/arm64/kernel/alternative.c
>>> b/arch/arm64/kernel/alternative.c
>>> index c947d22..a9b4677 100644
>>> --- a/arch/arm64/kernel/alternative.c
>>> +++ b/arch/arm64/kernel/alternative.c
>>> @@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start,
>>> u64 end)
>>>        } while (cur += d_size, cur < end);
>>>    }
>>>    -static void __apply_alternatives(void *alt_region, bool is_module)
>>> +static void __apply_alternatives(void *alt_region,  bool is_module,
>>> +                 unsigned long *feature_mask)
>>>    {
>>>        struct alt_instr *alt;
>>>        struct alt_region *region = alt_region;
>>> @@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region,
>>> bool is_module)
>>>        for (alt = region->begin; alt < region->end; alt++) {
>>>            int nr_inst;
>>>    +        if (!test_bit(alt->cpufeature, feature_mask))
>>> +            continue;
>>> +
>>>            /* Use ARM64_CB_PATCH as an unconditional patch */
>>>            if (alt->cpufeature < ARM64_CB_PATCH &&
>>>                !cpus_have_cap(alt->cpufeature))
>>> @@ -203,8 +207,11 @@ static void __apply_alternatives(void
>>> *alt_region, bool is_module)
>>>            __flush_icache_all();
>>>            isb();
>>>    -        /* We applied all that was available */
>>> -        bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
>>> +        /* Ignore ARM64_CB bit from feature mask */
>>> +        bitmap_or(applied_alternatives, applied_alternatives,
>>> +              feature_mask, ARM64_NCAPS);
>>> +        bitmap_and(applied_alternatives, applied_alternatives,
>>> +               cpu_hwcaps, ARM64_NCAPS);
>>>        }
>>>    }
>>>    @@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void
>>> *unused)
>>>                cpu_relax();
>>>            isb();
>>>        } else {
>>> +        DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
>>> +
>>> +        bitmap_complement(remaining_capabilities, boot_capabilities,
>>> +                  ARM64_NPATCHABLE);
>>> +
>>>            BUG_ON(all_alternatives_applied);
>>> -        __apply_alternatives(&region, false);
>>> +        __apply_alternatives(&region, false, remaining_capabilities);
>>>            /* Barriers provided by the cache flushing */
>>>            WRITE_ONCE(all_alternatives_applied, 1);
>>>        }
>>> @@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
>>>        stop_machine(__apply_alternatives_multi_stop, NULL,
>>> cpu_online_mask);
>>>    }
>>>    +/*
>>> + * This is called very early in the boot process (directly after we run
>>> + * a feature detect on the boot CPU). No need to worry about other CPUs
>>> + * here.
>>> + */
>>> +void __init apply_boot_alternatives(void)
>>> +{
>>> +    struct alt_region region = {
>>> +        .begin    = (struct alt_instr *)__alt_instructions,
>>> +        .end    = (struct alt_instr *)__alt_instructions_end,
>>> +    };
>>> +
>>> +    /* If called on non-boot cpu things could go wrong */
>>> +    WARN_ON(smp_processor_id() != 0);
>>> +
>>> +    __apply_alternatives(&region, false, &boot_capabilities[0]);
>>> +}
>>> +
>>>    #ifdef CONFIG_MODULES
>>>    void apply_alternatives_module(void *start, size_t length)
>>>    {
>>> @@ -247,7 +277,10 @@ void apply_alternatives_module(void *start,
>>> size_t length)
>>>            .begin    = start,
>>>            .end    = start + length,
>>>        };
>>> +    DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
>>> +
>>> +    bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
>>>    -    __apply_alternatives(&region, true);
>>> +    __apply_alternatives(&region, true, &all_capabilities[0]);
>>>    }
>>>    #endif
>>> diff --git a/arch/arm64/kernel/cpufeature.c
>>> b/arch/arm64/kernel/cpufeature.c
>>> index 84fa5be..71c8d4f 100644
>>> --- a/arch/arm64/kernel/cpufeature.c
>>> +++ b/arch/arm64/kernel/cpufeature.c
>>> @@ -54,6 +54,9 @@
>>>    EXPORT_SYMBOL(cpu_hwcaps);
>>>    static struct arm64_cpu_capabilities const __ro_after_init
>>> *cpu_hwcaps_ptrs[ARM64_NCAPS];
>>>    +/* Need also bit for ARM64_CB_PATCH */
>>> +DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>>> +
>>>    /*
>>>     * Flag to indicate if we have computed the system wide
>>>     * capabilities based on the boot time active CPUs. This
>>> @@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16 scope_mask)
>>>            if (caps->desc)
>>>                pr_info("detected: %s\n", caps->desc);
>>>            cpus_set_cap(caps->capability);
>>> +
>>> +        if (caps->type & SCOPE_BOOT_CPU)
>>
>> You may want to do :
>>          if (scope_mask & SCOPE_BOOT_CPU)
>>
>> for a tighter check to ensure this doesn't update the boot_capabilities
>> after we have applied the boot_scope alternatives and miss applying the
>> alternatives for those, should someone add a multi-scope (i.e
>> SCOPE_BOOT_CPU and
>> something else) capability (even by mistake).
>>
> 
> But a multi-scope capability containing SCOPE_BOOT_CPU should already
> get updated for setup_boot_cpu_capabilities. Capabilities marked with
> SCOPE_BOOT_CPU need to be enabled on the boot CPU or not at all.

Yes, you're right. It is not normal to have multiple SCOPE for a "capability".
But if someone comes with such a cap, we may miss handling this case. It is
always better to be safer.

> 
> Shouldn't the call to caps->matches() fail for a boot feature that was
> not found on the boot cpu?
> 
> Also, you made the opposite suggestion 4 version ago with a more
> worrying scenario :) :
> https://lkml.org/lkml/2018/5/25/208

Ah, you're right. I missed that. We need the additional check as you mention
below.

> 
> Otherwise, if my assumption above is wrong, it means the check should
> probably be:
> 	if (caps->type & SCOPE_BOOT_CPU && scope_mask & SCOPE_BOOT_CPU)

Yes, this is what we want.


> But my current understanding is that we don't need that.
> 
>> With that:
>>
>> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>


Cheers
Suzuki

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

On 08/01/2019 16:45, Dave Martin wrote:
> On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
>> On 08/01/2019 15:40, Dave Martin wrote:
>>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>>>> Instead disabling interrupts by setting the PSR.I bit, use a priority
>>>> higher than the one used for interrupts to mask them via PMR.
>>>>
>>>> When using PMR to disable interrupts, the value of PMR will be used
>>>> instead of PSR.[DAIF] for the irqflags.
>>>>
>>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>>>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
>>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>>>> Cc: Will Deacon <will.deacon@arm.com>
>>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>> Cc: Oleg Nesterov <oleg@redhat.com>
>>>> ---
>>>>  arch/arm64/include/asm/efi.h      |  11 ++++
>>>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>>>>  2 files changed, 106 insertions(+), 28 deletions(-)
>>>
>>> [...]
>>>
>>>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
>>>> index 24692ed..fa3b06f 100644
>>>> --- a/arch/arm64/include/asm/irqflags.h
>>>> +++ b/arch/arm64/include/asm/irqflags.h
>>>> @@ -18,7 +18,9 @@
>>>
>>> [...]
>>>
>>>>  static inline void arch_local_irq_enable(void)
>>>>  {
>>>> -	asm volatile(
>>>> -		"msr	daifclr, #2		// arch_local_irq_enable"
>>>> -		:
>>>> +	unsigned long unmasked = GIC_PRIO_IRQON;
>>>> +
>>>> +	asm volatile(ALTERNATIVE(
>>>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
>>>> +		"nop",
>>>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
>>>> +		"dsb	sy",
>>>
>>> I'm still not convinced these dsbs are needed.
>>>
>>> Without the dsb, we are probably not guaranteed to take a pending
>>> interrupt _immediately_ on unmasking, but I'm not sure that's a
>>> problem.
>>>
>>> What goes wrong if we omit them?
>>
>> Then the GIC doesn't know it can now deliver interrupts of a lower
>> priority. Only a dsb can guarantee that the GIC's view of PMR will get
>> updated.
>>
>> See 9.1.6 (Observability of the effects of accesses to the GIC
>> registers), which states:
>>
>> <quote>
>> Architectural execution of a DSB instruction guarantees that
>> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
>> associated Redistributor.
>> </quote>
>>
>> So yes, DSB is required.
> 
> But it says neither what is means for the PMR write to be "observed by
> the redistributor", nor whether the DSB is required for the
> redistributor to observe the write at all.  (So, is an implementation
> allowed to cached in the CPU interface indefinitely until forcibly
> flushed to the redistributor by a DSB, and in any case can the write's
> reaching the distributor in finite time or not have any effect that we
> care about in this case?).
> 
> 
> My reason for querying this is that temporary local masking of classes
> of interrupts seems an obvious use case for the PMR, and the DSB
> requirement flies rather in the face of this.
> 
> 
> Have we seen hardware where interrupts may stall forever upstream of the
> CPU interface after a PMR write, until a dsb is executed by the CPU?
> 

I don't have too much GICv3 hardware at hand but the one I tested
*seems* to work without the DSB. But of course this does not mean it is
correct even on that hardware.

As you said it is not clear what is meant by "observed by the
redistributor", it can mean that the redistributor is allowed to stop
forwarding interrupts of lower priority or it can mean that it always
forwards them as the CPU interface is required to prevent masked
priority interrupts to be signaled to the CPU. The hardware I'm using
might be in the latter case... or not...



> If so that is sad, but I guess we have to live with it.
> 
> Also, is it ever important in Linux that a pending interrupt be taken
> immediately upon unmasking (and how do we know that said interrupt is
> pending)?  If not, we don't care precisely when such interrupts are
> pended to the PE, just that such an interrupt cannot be taken before
> the PMR write that unmasks it.  It would be insane for the self-
> synchronization of PMR writes to lack this guarantee (and a DSB after
> the PMR write would do no good anyway in that case).
> 

The first thing that comes to mind for this would be the RT world, I'm
not sure it would it would be nice to have interrupts "actually
unmasked" at some random time in the future.

Otherwise, aren't there some parts of the kernel that expect being able
to take interrupts? (memory allocation comes to mind). What happens if
the interrupt might not happen?

Also, code that does:

while (!poll_update_from_irq()) {
	local_irq_disable();

	// do stuff

	local_irq_enable();
}

Might never see interrupts. It is not clear to me whether that is the
case for the loop in do_idle() and the check for need_resched().

I don't know if drivers might have such code patterns.

Cheers,

-- 
Julien Thierry

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

On 08/01/2019 16:45, Dave Martin wrote:
> On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
>> On 08/01/2019 15:40, Dave Martin wrote:
>>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>>>> Instead disabling interrupts by setting the PSR.I bit, use a priority
>>>> higher than the one used for interrupts to mask them via PMR.
>>>>
>>>> When using PMR to disable interrupts, the value of PMR will be used
>>>> instead of PSR.[DAIF] for the irqflags.
>>>>
>>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>>>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
>>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>>>> Cc: Will Deacon <will.deacon@arm.com>
>>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>> Cc: Oleg Nesterov <oleg@redhat.com>
>>>> ---
>>>>  arch/arm64/include/asm/efi.h      |  11 ++++
>>>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>>>>  2 files changed, 106 insertions(+), 28 deletions(-)
>>>
>>> [...]
>>>
>>>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
>>>> index 24692ed..fa3b06f 100644
>>>> --- a/arch/arm64/include/asm/irqflags.h
>>>> +++ b/arch/arm64/include/asm/irqflags.h
>>>> @@ -18,7 +18,9 @@
>>>
>>> [...]
>>>
>>>>  static inline void arch_local_irq_enable(void)
>>>>  {
>>>> -	asm volatile(
>>>> -		"msr	daifclr, #2		// arch_local_irq_enable"
>>>> -		:
>>>> +	unsigned long unmasked = GIC_PRIO_IRQON;
>>>> +
>>>> +	asm volatile(ALTERNATIVE(
>>>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
>>>> +		"nop",
>>>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
>>>> +		"dsb	sy",
>>>
>>> I'm still not convinced these dsbs are needed.
>>>
>>> Without the dsb, we are probably not guaranteed to take a pending
>>> interrupt _immediately_ on unmasking, but I'm not sure that's a
>>> problem.
>>>
>>> What goes wrong if we omit them?
>>
>> Then the GIC doesn't know it can now deliver interrupts of a lower
>> priority. Only a dsb can guarantee that the GIC's view of PMR will get
>> updated.
>>
>> See 9.1.6 (Observability of the effects of accesses to the GIC
>> registers), which states:
>>
>> <quote>
>> Architectural execution of a DSB instruction guarantees that
>> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
>> associated Redistributor.
>> </quote>
>>
>> So yes, DSB is required.
> 
> But it says neither what is means for the PMR write to be "observed by
> the redistributor", nor whether the DSB is required for the
> redistributor to observe the write at all.  (So, is an implementation
> allowed to cached in the CPU interface indefinitely until forcibly
> flushed to the redistributor by a DSB, and in any case can the write's
> reaching the distributor in finite time or not have any effect that we
> care about in this case?).
> 
> 
> My reason for querying this is that temporary local masking of classes
> of interrupts seems an obvious use case for the PMR, and the DSB
> requirement flies rather in the face of this.
> 
> 
> Have we seen hardware where interrupts may stall forever upstream of the
> CPU interface after a PMR write, until a dsb is executed by the CPU?
> 

I don't have too much GICv3 hardware at hand but the one I tested
*seems* to work without the DSB. But of course this does not mean it is
correct even on that hardware.

As you said it is not clear what is meant by "observed by the
redistributor", it can mean that the redistributor is allowed to stop
forwarding interrupts of lower priority or it can mean that it always
forwards them as the CPU interface is required to prevent masked
priority interrupts to be signaled to the CPU. The hardware I'm using
might be in the latter case... or not...



> If so that is sad, but I guess we have to live with it.
> 
> Also, is it ever important in Linux that a pending interrupt be taken
> immediately upon unmasking (and how do we know that said interrupt is
> pending)?  If not, we don't care precisely when such interrupts are
> pended to the PE, just that such an interrupt cannot be taken before
> the PMR write that unmasks it.  It would be insane for the self-
> synchronization of PMR writes to lack this guarantee (and a DSB after
> the PMR write would do no good anyway in that case).
> 

The first thing that comes to mind for this would be the RT world, I'm
not sure it would it would be nice to have interrupts "actually
unmasked" at some random time in the future.

Otherwise, aren't there some parts of the kernel that expect being able
to take interrupts? (memory allocation comes to mind). What happens if
the interrupt might not happen?

Also, code that does:

while (!poll_update_from_irq()) {
	local_irq_disable();

	// do stuff

	local_irq_enable();
}

Might never see interrupts. It is not clear to me whether that is the
case for the loop in do_idle() and the check for need_resched().

I don't know if drivers might have such code patterns.

Cheers,

-- 
Julien Thierry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 05:16:43PM +0000, Marc Zyngier wrote:
> On 08/01/2019 16:45, Dave Martin wrote:
> > On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
> >> On 08/01/2019 15:40, Dave Martin wrote:
> >>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >>>> Instead disabling interrupts by setting the PSR.I bit, use a priority
> >>>> higher than the one used for interrupts to mask them via PMR.
> >>>>
> >>>> When using PMR to disable interrupts, the value of PMR will be used
> >>>> instead of PSR.[DAIF] for the irqflags.
> >>>>
> >>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> >>>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> >>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >>>> Cc: Will Deacon <will.deacon@arm.com>
> >>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >>>> Cc: Oleg Nesterov <oleg@redhat.com>
> >>>> ---
> >>>>  arch/arm64/include/asm/efi.h      |  11 ++++
> >>>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
> >>>>  2 files changed, 106 insertions(+), 28 deletions(-)
> >>>
> >>> [...]
> >>>
> >>>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> >>>> index 24692ed..fa3b06f 100644
> >>>> --- a/arch/arm64/include/asm/irqflags.h
> >>>> +++ b/arch/arm64/include/asm/irqflags.h
> >>>> @@ -18,7 +18,9 @@
> >>>
> >>> [...]
> >>>
> >>>>  static inline void arch_local_irq_enable(void)
> >>>>  {
> >>>> -	asm volatile(
> >>>> -		"msr	daifclr, #2		// arch_local_irq_enable"
> >>>> -		:
> >>>> +	unsigned long unmasked = GIC_PRIO_IRQON;
> >>>> +
> >>>> +	asm volatile(ALTERNATIVE(
> >>>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
> >>>> +		"nop",
> >>>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
> >>>> +		"dsb	sy",
> >>>
> >>> I'm still not convinced these dsbs are needed.
> >>>
> >>> Without the dsb, we are probably not guaranteed to take a pending
> >>> interrupt _immediately_ on unmasking, but I'm not sure that's a
> >>> problem.
> >>>
> >>> What goes wrong if we omit them?
> >>
> >> Then the GIC doesn't know it can now deliver interrupts of a lower
> >> priority. Only a dsb can guarantee that the GIC's view of PMR will get
> >> updated.
> >>
> >> See 9.1.6 (Observability of the effects of accesses to the GIC
> >> registers), which states:
> >>
> >> <quote>
> >> Architectural execution of a DSB instruction guarantees that
> >> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
> >> associated Redistributor.
> >> </quote>
> >>
> >> So yes, DSB is required.
> > 
> > But it says neither what is means for the PMR write to be "observed by
> > the redistributor", nor whether the DSB is required for the
> > redistributor to observe the write at all.
> 
> Well, it seems pretty clear to me that if the redistributor doesn't
> observe the PMR value, it is unlikely to change its interpretation of
> it· And conversely, the redistributor is allowed to sit pretty and not
> give you any interrupt until you are actually telling it that something
> has changed.
> 
> I really think that for once, the spec is pretty unambiguous about what
> is required.

I think that there is some scope for clarification, but it sounds like
that doesn't impact this series.

> > (So, is an implementation
> > allowed to cached in the CPU interface indefinitely until forcibly
> > flushed to the redistributor by a DSB, and in any case can the write's
> > reaching the distributor in finite time or not have any effect that we
> > care about in this case?).
> 
> Nothing in the spec says that the system register write will magically
> trickle down to the redistributor in the absence of a DSB.
> 
> > My reason for querying this is that temporary local masking of classes
> > of interrupts seems an obvious use case for the PMR, and the DSB
> > requirement flies rather in the face of this.
> 
> Are you implying that the GIC architecture should have any form of
> sanity and be useful for general purpose software? Think again! ;-)
> 
> The PMR behavior you are describing only works in a single direction
> (from low to high priority), because the CPU interface has to perform
> some filtering. In the opposite direction, you need the big hammer.
> 
> > Have we seen hardware where interrupts may stall forever upstream of the
> > CPU interface after a PMR write, until a dsb is executed by the CPU?
> 
> Yes. You even have to have a DSB right after a read of IAR to avoid
> loosing interrupts. The short story is that there is hardly any
> synchronization between redistributor and CPU interface. Implementations
> are allowed a more closely coupled design, but that's not what the
> architecture mandates.

Well I guess that pretty much wraps it up!

If implementations require it, then we obviously need to have it.

> > Also, is it ever important in Linux that a pending interrupt be taken
> > immediately upon unmasking (and how do we know that said interrupt is
> > pending)?  If not, we don't care precisely when such interrupts are
> > pended to the PE, just that such an interrupt cannot be taken before
> > the PMR write that unmasks it.  It would be insane for the self-
> > synchronization of PMR writes to lack this guarantee (and a DSB after
> > the PMR write would do no good anyway in that case).
> 
> RT folks are usually quite picky on when they see their interrupts
> firing. I can also imagine the following scenario:
> 
> 	set_pmr(allow all interrupts)
> 	WFI
> 
> where things stop rather abruptly if this is the only CPU in the system.

This is a rather special case (and this case is indeed handled specially
by this series).  Here we don't need to expedite interrupt delivery,
but we want to make sure that the logic that will wake us back up knows
what it's supposed to be doing before we start powering things off.

However, I can see that for RT purposes explicit synchronisation on
interrupt unmasking may provided a more bounded interrupt blackout that
simply waiting for synchronisation to happen in the background (though
that might have better throughput).  Anyway, this is academic since
we have to have the synchyronisation anyway.

I will consider myself corrected and stop trolling...

Cheers
---Dave

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 05:16:43PM +0000, Marc Zyngier wrote:
> On 08/01/2019 16:45, Dave Martin wrote:
> > On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
> >> On 08/01/2019 15:40, Dave Martin wrote:
> >>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >>>> Instead disabling interrupts by setting the PSR.I bit, use a priority
> >>>> higher than the one used for interrupts to mask them via PMR.
> >>>>
> >>>> When using PMR to disable interrupts, the value of PMR will be used
> >>>> instead of PSR.[DAIF] for the irqflags.
> >>>>
> >>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> >>>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> >>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >>>> Cc: Will Deacon <will.deacon@arm.com>
> >>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >>>> Cc: Oleg Nesterov <oleg@redhat.com>
> >>>> ---
> >>>>  arch/arm64/include/asm/efi.h      |  11 ++++
> >>>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
> >>>>  2 files changed, 106 insertions(+), 28 deletions(-)
> >>>
> >>> [...]
> >>>
> >>>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> >>>> index 24692ed..fa3b06f 100644
> >>>> --- a/arch/arm64/include/asm/irqflags.h
> >>>> +++ b/arch/arm64/include/asm/irqflags.h
> >>>> @@ -18,7 +18,9 @@
> >>>
> >>> [...]
> >>>
> >>>>  static inline void arch_local_irq_enable(void)
> >>>>  {
> >>>> -	asm volatile(
> >>>> -		"msr	daifclr, #2		// arch_local_irq_enable"
> >>>> -		:
> >>>> +	unsigned long unmasked = GIC_PRIO_IRQON;
> >>>> +
> >>>> +	asm volatile(ALTERNATIVE(
> >>>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
> >>>> +		"nop",
> >>>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
> >>>> +		"dsb	sy",
> >>>
> >>> I'm still not convinced these dsbs are needed.
> >>>
> >>> Without the dsb, we are probably not guaranteed to take a pending
> >>> interrupt _immediately_ on unmasking, but I'm not sure that's a
> >>> problem.
> >>>
> >>> What goes wrong if we omit them?
> >>
> >> Then the GIC doesn't know it can now deliver interrupts of a lower
> >> priority. Only a dsb can guarantee that the GIC's view of PMR will get
> >> updated.
> >>
> >> See 9.1.6 (Observability of the effects of accesses to the GIC
> >> registers), which states:
> >>
> >> <quote>
> >> Architectural execution of a DSB instruction guarantees that
> >> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
> >> associated Redistributor.
> >> </quote>
> >>
> >> So yes, DSB is required.
> > 
> > But it says neither what is means for the PMR write to be "observed by
> > the redistributor", nor whether the DSB is required for the
> > redistributor to observe the write at all.
> 
> Well, it seems pretty clear to me that if the redistributor doesn't
> observe the PMR value, it is unlikely to change its interpretation of
> it· And conversely, the redistributor is allowed to sit pretty and not
> give you any interrupt until you are actually telling it that something
> has changed.
> 
> I really think that for once, the spec is pretty unambiguous about what
> is required.

I think that there is some scope for clarification, but it sounds like
that doesn't impact this series.

> > (So, is an implementation
> > allowed to cached in the CPU interface indefinitely until forcibly
> > flushed to the redistributor by a DSB, and in any case can the write's
> > reaching the distributor in finite time or not have any effect that we
> > care about in this case?).
> 
> Nothing in the spec says that the system register write will magically
> trickle down to the redistributor in the absence of a DSB.
> 
> > My reason for querying this is that temporary local masking of classes
> > of interrupts seems an obvious use case for the PMR, and the DSB
> > requirement flies rather in the face of this.
> 
> Are you implying that the GIC architecture should have any form of
> sanity and be useful for general purpose software? Think again! ;-)
> 
> The PMR behavior you are describing only works in a single direction
> (from low to high priority), because the CPU interface has to perform
> some filtering. In the opposite direction, you need the big hammer.
> 
> > Have we seen hardware where interrupts may stall forever upstream of the
> > CPU interface after a PMR write, until a dsb is executed by the CPU?
> 
> Yes. You even have to have a DSB right after a read of IAR to avoid
> loosing interrupts. The short story is that there is hardly any
> synchronization between redistributor and CPU interface. Implementations
> are allowed a more closely coupled design, but that's not what the
> architecture mandates.

Well I guess that pretty much wraps it up!

If implementations require it, then we obviously need to have it.

> > Also, is it ever important in Linux that a pending interrupt be taken
> > immediately upon unmasking (and how do we know that said interrupt is
> > pending)?  If not, we don't care precisely when such interrupts are
> > pended to the PE, just that such an interrupt cannot be taken before
> > the PMR write that unmasks it.  It would be insane for the self-
> > synchronization of PMR writes to lack this guarantee (and a DSB after
> > the PMR write would do no good anyway in that case).
> 
> RT folks are usually quite picky on when they see their interrupts
> firing. I can also imagine the following scenario:
> 
> 	set_pmr(allow all interrupts)
> 	WFI
> 
> where things stop rather abruptly if this is the only CPU in the system.

This is a rather special case (and this case is indeed handled specially
by this series).  Here we don't need to expedite interrupt delivery,
but we want to make sure that the logic that will wake us back up knows
what it's supposed to be doing before we start powering things off.

However, I can see that for RT purposes explicit synchronisation on
interrupt unmasking may provided a more bounded interrupt blackout that
simply waiting for synchronisation to happen in the background (though
that might have better throughput).  Anyway, this is academic since
we have to have the synchyronisation anyway.

I will consider myself corrected and stop trolling...

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 05:58:59PM +0000, Julien Thierry wrote:
> 
> 
> On 08/01/2019 16:45, Dave Martin wrote:
> > On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
> >> On 08/01/2019 15:40, Dave Martin wrote:
> >>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >>>> Instead disabling interrupts by setting the PSR.I bit, use a priority
> >>>> higher than the one used for interrupts to mask them via PMR.
> >>>>
> >>>> When using PMR to disable interrupts, the value of PMR will be used
> >>>> instead of PSR.[DAIF] for the irqflags.
> >>>>
> >>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> >>>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> >>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >>>> Cc: Will Deacon <will.deacon@arm.com>
> >>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >>>> Cc: Oleg Nesterov <oleg@redhat.com>
> >>>> ---
> >>>>  arch/arm64/include/asm/efi.h      |  11 ++++
> >>>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
> >>>>  2 files changed, 106 insertions(+), 28 deletions(-)
> >>>
> >>> [...]
> >>>
> >>>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> >>>> index 24692ed..fa3b06f 100644
> >>>> --- a/arch/arm64/include/asm/irqflags.h
> >>>> +++ b/arch/arm64/include/asm/irqflags.h
> >>>> @@ -18,7 +18,9 @@
> >>>
> >>> [...]
> >>>
> >>>>  static inline void arch_local_irq_enable(void)
> >>>>  {
> >>>> -	asm volatile(
> >>>> -		"msr	daifclr, #2		// arch_local_irq_enable"
> >>>> -		:
> >>>> +	unsigned long unmasked = GIC_PRIO_IRQON;
> >>>> +
> >>>> +	asm volatile(ALTERNATIVE(
> >>>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
> >>>> +		"nop",
> >>>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
> >>>> +		"dsb	sy",
> >>>
> >>> I'm still not convinced these dsbs are needed.
> >>>
> >>> Without the dsb, we are probably not guaranteed to take a pending
> >>> interrupt _immediately_ on unmasking, but I'm not sure that's a
> >>> problem.
> >>>
> >>> What goes wrong if we omit them?
> >>
> >> Then the GIC doesn't know it can now deliver interrupts of a lower
> >> priority. Only a dsb can guarantee that the GIC's view of PMR will get
> >> updated.
> >>
> >> See 9.1.6 (Observability of the effects of accesses to the GIC
> >> registers), which states:
> >>
> >> <quote>
> >> Architectural execution of a DSB instruction guarantees that
> >> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
> >> associated Redistributor.
> >> </quote>
> >>
> >> So yes, DSB is required.
> > 
> > But it says neither what is means for the PMR write to be "observed by
> > the redistributor", nor whether the DSB is required for the
> > redistributor to observe the write at all.  (So, is an implementation
> > allowed to cached in the CPU interface indefinitely until forcibly
> > flushed to the redistributor by a DSB, and in any case can the write's
> > reaching the distributor in finite time or not have any effect that we
> > care about in this case?).
> > 
> > 
> > My reason for querying this is that temporary local masking of classes
> > of interrupts seems an obvious use case for the PMR, and the DSB
> > requirement flies rather in the face of this.
> > 
> > 
> > Have we seen hardware where interrupts may stall forever upstream of the
> > CPU interface after a PMR write, until a dsb is executed by the CPU?
> > 
> 
> I don't have too much GICv3 hardware at hand but the one I tested
> *seems* to work without the DSB. But of course this does not mean it is
> correct even on that hardware.
> 
> As you said it is not clear what is meant by "observed by the
> redistributor", it can mean that the redistributor is allowed to stop
> forwarding interrupts of lower priority or it can mean that it always
> forwards them as the CPU interface is required to prevent masked
> priority interrupts to be signaled to the CPU. The hardware I'm using
> might be in the latter case... or not...
> 
> 
> 
> > If so that is sad, but I guess we have to live with it.
> > 
> > Also, is it ever important in Linux that a pending interrupt be taken
> > immediately upon unmasking (and how do we know that said interrupt is
> > pending)?  If not, we don't care precisely when such interrupts are
> > pended to the PE, just that such an interrupt cannot be taken before
> > the PMR write that unmasks it.  It would be insane for the self-
> > synchronization of PMR writes to lack this guarantee (and a DSB after
> > the PMR write would do no good anyway in that case).
> > 
> 
> The first thing that comes to mind for this would be the RT world, I'm
> not sure it would it would be nice to have interrupts "actually
> unmasked" at some random time in the future.

If the PMR write were to start propagating immediately and DSB merely
waited for it to reach the redistributor, then there would be no
random delay -- rather, the DSB would generate some pointless system
load and prevent any useful work being done in the interim, without
actually speeding anything up.

Of course, it sounds like the GIC spec doesn't mandate such an
implementation, so we can't rely on this.

> Otherwise, aren't there some parts of the kernel that expect being able
> to take interrupts? (memory allocation comes to mind). What happens if
> the interrupt might not happen?
> 
> Also, code that does:
> 
> while (!poll_update_from_irq()) {
> 	local_irq_disable();
> 
> 	// do stuff
> 
> 	local_irq_enable();
> }
> 
> Might never see interrupts. It is not clear to me whether that is the
> case for the loop in do_idle() and the check for need_resched().
> 
> I don't know if drivers might have such code patterns.

For appropriate self-synchronising behaviour in PMR writes, this
problem is solvable: if you have a pending PMR write followed by a
PMR write with a more restrictive bound (so disabling some interrupts)
then the second write could be stalled until the first has synchronised,
providing an opportunity to take pending interrupts before they would
get masked.

There might be implementations that do something like this, but I guess
we can't rely on this either.

I'll defer to the experts...

Cheers
---Dave

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 05:58:59PM +0000, Julien Thierry wrote:
> 
> 
> On 08/01/2019 16:45, Dave Martin wrote:
> > On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote:
> >> On 08/01/2019 15:40, Dave Martin wrote:
> >>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >>>> Instead disabling interrupts by setting the PSR.I bit, use a priority
> >>>> higher than the one used for interrupts to mask them via PMR.
> >>>>
> >>>> When using PMR to disable interrupts, the value of PMR will be used
> >>>> instead of PSR.[DAIF] for the irqflags.
> >>>>
> >>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> >>>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> >>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >>>> Cc: Will Deacon <will.deacon@arm.com>
> >>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >>>> Cc: Oleg Nesterov <oleg@redhat.com>
> >>>> ---
> >>>>  arch/arm64/include/asm/efi.h      |  11 ++++
> >>>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
> >>>>  2 files changed, 106 insertions(+), 28 deletions(-)
> >>>
> >>> [...]
> >>>
> >>>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> >>>> index 24692ed..fa3b06f 100644
> >>>> --- a/arch/arm64/include/asm/irqflags.h
> >>>> +++ b/arch/arm64/include/asm/irqflags.h
> >>>> @@ -18,7 +18,9 @@
> >>>
> >>> [...]
> >>>
> >>>>  static inline void arch_local_irq_enable(void)
> >>>>  {
> >>>> -	asm volatile(
> >>>> -		"msr	daifclr, #2		// arch_local_irq_enable"
> >>>> -		:
> >>>> +	unsigned long unmasked = GIC_PRIO_IRQON;
> >>>> +
> >>>> +	asm volatile(ALTERNATIVE(
> >>>> +		"msr	daifclr, #2		// arch_local_irq_enable\n"
> >>>> +		"nop",
> >>>> +		"msr_s  " __stringify(SYS_ICC_PMR_EL1) ",%0\n"
> >>>> +		"dsb	sy",
> >>>
> >>> I'm still not convinced these dsbs are needed.
> >>>
> >>> Without the dsb, we are probably not guaranteed to take a pending
> >>> interrupt _immediately_ on unmasking, but I'm not sure that's a
> >>> problem.
> >>>
> >>> What goes wrong if we omit them?
> >>
> >> Then the GIC doesn't know it can now deliver interrupts of a lower
> >> priority. Only a dsb can guarantee that the GIC's view of PMR will get
> >> updated.
> >>
> >> See 9.1.6 (Observability of the effects of accesses to the GIC
> >> registers), which states:
> >>
> >> <quote>
> >> Architectural execution of a DSB instruction guarantees that
> >> — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the
> >> associated Redistributor.
> >> </quote>
> >>
> >> So yes, DSB is required.
> > 
> > But it says neither what is means for the PMR write to be "observed by
> > the redistributor", nor whether the DSB is required for the
> > redistributor to observe the write at all.  (So, is an implementation
> > allowed to cached in the CPU interface indefinitely until forcibly
> > flushed to the redistributor by a DSB, and in any case can the write's
> > reaching the distributor in finite time or not have any effect that we
> > care about in this case?).
> > 
> > 
> > My reason for querying this is that temporary local masking of classes
> > of interrupts seems an obvious use case for the PMR, and the DSB
> > requirement flies rather in the face of this.
> > 
> > 
> > Have we seen hardware where interrupts may stall forever upstream of the
> > CPU interface after a PMR write, until a dsb is executed by the CPU?
> > 
> 
> I don't have too much GICv3 hardware at hand but the one I tested
> *seems* to work without the DSB. But of course this does not mean it is
> correct even on that hardware.
> 
> As you said it is not clear what is meant by "observed by the
> redistributor", it can mean that the redistributor is allowed to stop
> forwarding interrupts of lower priority or it can mean that it always
> forwards them as the CPU interface is required to prevent masked
> priority interrupts to be signaled to the CPU. The hardware I'm using
> might be in the latter case... or not...
> 
> 
> 
> > If so that is sad, but I guess we have to live with it.
> > 
> > Also, is it ever important in Linux that a pending interrupt be taken
> > immediately upon unmasking (and how do we know that said interrupt is
> > pending)?  If not, we don't care precisely when such interrupts are
> > pended to the PE, just that such an interrupt cannot be taken before
> > the PMR write that unmasks it.  It would be insane for the self-
> > synchronization of PMR writes to lack this guarantee (and a DSB after
> > the PMR write would do no good anyway in that case).
> > 
> 
> The first thing that comes to mind for this would be the RT world, I'm
> not sure it would it would be nice to have interrupts "actually
> unmasked" at some random time in the future.

If the PMR write were to start propagating immediately and DSB merely
waited for it to reach the redistributor, then there would be no
random delay -- rather, the DSB would generate some pointless system
load and prevent any useful work being done in the interim, without
actually speeding anything up.

Of course, it sounds like the GIC spec doesn't mandate such an
implementation, so we can't rely on this.

> Otherwise, aren't there some parts of the kernel that expect being able
> to take interrupts? (memory allocation comes to mind). What happens if
> the interrupt might not happen?
> 
> Also, code that does:
> 
> while (!poll_update_from_irq()) {
> 	local_irq_disable();
> 
> 	// do stuff
> 
> 	local_irq_enable();
> }
> 
> Might never see interrupts. It is not clear to me whether that is the
> case for the loop in do_idle() and the check for need_resched().
> 
> I don't know if drivers might have such code patterns.

For appropriate self-synchronising behaviour in PMR writes, this
problem is solvable: if you have a pending PMR write followed by a
PMR write with a more restrictive bound (so disabling some interrupts)
then the second write could be stalled until the first has synchronised,
providing an opportunity to take pending interrupts before they would
get masked.

There might be implementations that do something like this, but I guess
we can't rely on this either.

I'll defer to the experts...

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Julien Thierry]

On 08/01/2019 17:40, Suzuki K Poulose wrote:
> 
> 
> On 08/01/2019 15:20, Julien Thierry wrote:
>> Hi Suzuki,
>>
>> On 08/01/2019 14:51, Suzuki K Poulose wrote:
>>> Hi Julien,
>>>
>>> On 08/01/2019 14:07, Julien Thierry wrote:
>>>> From: Daniel Thompson <daniel.thompson@linaro.org>
>>>>
>>>> Currently alternatives are applied very late in the boot process (and
>>>> a long time after we enable scheduling). Some alternative sequences,
>>>> such as those that alter the way CPU context is stored, must be applied
>>>> much earlier in the boot sequence.
>>>>
>>>> Introduce apply_boot_alternatives() to allow some alternatives to be
>>>> applied immediately after we detect the CPU features of the boot CPU.
>>>>
>>>> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
>>>> [julien.thierry@arm.com: rename to fit new cpufeature framework better,
>>>>               apply BOOT_SCOPE feature early in boot]
>>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>>>> Cc: Will Deacon <will.deacon@arm.com>
>>>> Cc: Christoffer Dall <christoffer.dall@arm.com>
>>>> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
>>>> ---
>>>>    arch/arm64/include/asm/alternative.h |  1 +
>>>>    arch/arm64/include/asm/cpufeature.h  |  4 ++++
>>>>    arch/arm64/kernel/alternative.c      | 43
>>>> +++++++++++++++++++++++++++++++-----
>>>>    arch/arm64/kernel/cpufeature.c       |  6 +++++
>>>>    arch/arm64/kernel/smp.c              |  7 ++++++
>>>>    5 files changed, 56 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/arch/arm64/include/asm/alternative.h
>>>> b/arch/arm64/include/asm/alternative.h
>>>> index 9806a23..b9f8d78 100644
>>>> --- a/arch/arm64/include/asm/alternative.h
>>>> +++ b/arch/arm64/include/asm/alternative.h
>>>> @@ -25,6 +25,7 @@ struct alt_instr {
>>>>    typedef void (*alternative_cb_t)(struct alt_instr *alt,
>>>>                     __le32 *origptr, __le32 *updptr, int nr_inst);
>>>>    +void __init apply_boot_alternatives(void);
>>>>    void __init apply_alternatives_all(void);
>>>>    bool alternative_is_applied(u16 cpufeature);
>>>>    diff --git a/arch/arm64/include/asm/cpufeature.h
>>>> b/arch/arm64/include/asm/cpufeature.h
>>>> index 89c3f31..e505e1f 100644
>>>> --- a/arch/arm64/include/asm/cpufeature.h
>>>> +++ b/arch/arm64/include/asm/cpufeature.h
>>>> @@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const
>>>> struct arm64_cpu_capabilities *cap)
>>>>    extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
>>>>    extern struct static_key_false arm64_const_caps_ready;
>>>>    +/* ARM64 CAPS + alternative_cb */
>>>> +#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
>>>> +extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>>>> +
>>>>    #define for_each_available_cap(cap)        \
>>>>        for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
>>>>    diff --git a/arch/arm64/kernel/alternative.c
>>>> b/arch/arm64/kernel/alternative.c
>>>> index c947d22..a9b4677 100644
>>>> --- a/arch/arm64/kernel/alternative.c
>>>> +++ b/arch/arm64/kernel/alternative.c
>>>> @@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start,
>>>> u64 end)
>>>>        } while (cur += d_size, cur < end);
>>>>    }
>>>>    -static void __apply_alternatives(void *alt_region, bool is_module)
>>>> +static void __apply_alternatives(void *alt_region,  bool is_module,
>>>> +                 unsigned long *feature_mask)
>>>>    {
>>>>        struct alt_instr *alt;
>>>>        struct alt_region *region = alt_region;
>>>> @@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region,
>>>> bool is_module)
>>>>        for (alt = region->begin; alt < region->end; alt++) {
>>>>            int nr_inst;
>>>>    +        if (!test_bit(alt->cpufeature, feature_mask))
>>>> +            continue;
>>>> +
>>>>            /* Use ARM64_CB_PATCH as an unconditional patch */
>>>>            if (alt->cpufeature < ARM64_CB_PATCH &&
>>>>                !cpus_have_cap(alt->cpufeature))
>>>> @@ -203,8 +207,11 @@ static void __apply_alternatives(void
>>>> *alt_region, bool is_module)
>>>>            __flush_icache_all();
>>>>            isb();
>>>>    -        /* We applied all that was available */
>>>> -        bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
>>>> +        /* Ignore ARM64_CB bit from feature mask */
>>>> +        bitmap_or(applied_alternatives, applied_alternatives,
>>>> +              feature_mask, ARM64_NCAPS);
>>>> +        bitmap_and(applied_alternatives, applied_alternatives,
>>>> +               cpu_hwcaps, ARM64_NCAPS);
>>>>        }
>>>>    }
>>>>    @@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void
>>>> *unused)
>>>>                cpu_relax();
>>>>            isb();
>>>>        } else {
>>>> +        DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
>>>> +
>>>> +        bitmap_complement(remaining_capabilities, boot_capabilities,
>>>> +                  ARM64_NPATCHABLE);
>>>> +
>>>>            BUG_ON(all_alternatives_applied);
>>>> -        __apply_alternatives(&region, false);
>>>> +        __apply_alternatives(&region, false, remaining_capabilities);
>>>>            /* Barriers provided by the cache flushing */
>>>>            WRITE_ONCE(all_alternatives_applied, 1);
>>>>        }
>>>> @@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
>>>>        stop_machine(__apply_alternatives_multi_stop, NULL,
>>>> cpu_online_mask);
>>>>    }
>>>>    +/*
>>>> + * This is called very early in the boot process (directly after we
>>>> run
>>>> + * a feature detect on the boot CPU). No need to worry about other
>>>> CPUs
>>>> + * here.
>>>> + */
>>>> +void __init apply_boot_alternatives(void)
>>>> +{
>>>> +    struct alt_region region = {
>>>> +        .begin    = (struct alt_instr *)__alt_instructions,
>>>> +        .end    = (struct alt_instr *)__alt_instructions_end,
>>>> +    };
>>>> +
>>>> +    /* If called on non-boot cpu things could go wrong */
>>>> +    WARN_ON(smp_processor_id() != 0);
>>>> +
>>>> +    __apply_alternatives(&region, false, &boot_capabilities[0]);
>>>> +}
>>>> +
>>>>    #ifdef CONFIG_MODULES
>>>>    void apply_alternatives_module(void *start, size_t length)
>>>>    {
>>>> @@ -247,7 +277,10 @@ void apply_alternatives_module(void *start,
>>>> size_t length)
>>>>            .begin    = start,
>>>>            .end    = start + length,
>>>>        };
>>>> +    DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
>>>> +
>>>> +    bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
>>>>    -    __apply_alternatives(&region, true);
>>>> +    __apply_alternatives(&region, true, &all_capabilities[0]);
>>>>    }
>>>>    #endif
>>>> diff --git a/arch/arm64/kernel/cpufeature.c
>>>> b/arch/arm64/kernel/cpufeature.c
>>>> index 84fa5be..71c8d4f 100644
>>>> --- a/arch/arm64/kernel/cpufeature.c
>>>> +++ b/arch/arm64/kernel/cpufeature.c
>>>> @@ -54,6 +54,9 @@
>>>>    EXPORT_SYMBOL(cpu_hwcaps);
>>>>    static struct arm64_cpu_capabilities const __ro_after_init
>>>> *cpu_hwcaps_ptrs[ARM64_NCAPS];
>>>>    +/* Need also bit for ARM64_CB_PATCH */
>>>> +DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>>>> +
>>>>    /*
>>>>     * Flag to indicate if we have computed the system wide
>>>>     * capabilities based on the boot time active CPUs. This
>>>> @@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16
>>>> scope_mask)
>>>>            if (caps->desc)
>>>>                pr_info("detected: %s\n", caps->desc);
>>>>            cpus_set_cap(caps->capability);
>>>> +
>>>> +        if (caps->type & SCOPE_BOOT_CPU)
>>>
>>> You may want to do :
>>>          if (scope_mask & SCOPE_BOOT_CPU)
>>>
>>> for a tighter check to ensure this doesn't update the boot_capabilities
>>> after we have applied the boot_scope alternatives and miss applying the
>>> alternatives for those, should someone add a multi-scope (i.e
>>> SCOPE_BOOT_CPU and
>>> something else) capability (even by mistake).
>>>
>>
>> But a multi-scope capability containing SCOPE_BOOT_CPU should already
>> get updated for setup_boot_cpu_capabilities. Capabilities marked with
>> SCOPE_BOOT_CPU need to be enabled on the boot CPU or not at all.
> 
> Yes, you're right. It is not normal to have multiple SCOPE for a
> "capability".
> But if someone comes with such a cap, we may miss handling this case. It is
> always better to be safer.
> 
>>
>> Shouldn't the call to caps->matches() fail for a boot feature that was
>> not found on the boot cpu?
>>
>> Also, you made the opposite suggestion 4 version ago with a more
>> worrying scenario :) :
>> https://lkml.org/lkml/2018/5/25/208
> 
> Ah, you're right. I missed that. We need the additional check as you
> mention
> below.
> 
>>
>> Otherwise, if my assumption above is wrong, it means the check should
>> probably be:
>>     if (caps->type & SCOPE_BOOT_CPU && scope_mask & SCOPE_BOOT_CPU)
> 
> Yes, this is what we want.

Yes you're right, the behaviour I was thinking of were the ID register,
where there views are altered for following CPUs when system wide
features are missing on the boot CPU. But ->matches() callbacks don't
only rely on ID registers.

I'll add that for the next version.

Thanks,

-- 
Julien Thierry

Re: [PATCH v8 15/26] arm64: alternative: Apply alternatives early in boot process

[Julien Thierry]

On 08/01/2019 17:40, Suzuki K Poulose wrote:
> 
> 
> On 08/01/2019 15:20, Julien Thierry wrote:
>> Hi Suzuki,
>>
>> On 08/01/2019 14:51, Suzuki K Poulose wrote:
>>> Hi Julien,
>>>
>>> On 08/01/2019 14:07, Julien Thierry wrote:
>>>> From: Daniel Thompson <daniel.thompson@linaro.org>
>>>>
>>>> Currently alternatives are applied very late in the boot process (and
>>>> a long time after we enable scheduling). Some alternative sequences,
>>>> such as those that alter the way CPU context is stored, must be applied
>>>> much earlier in the boot sequence.
>>>>
>>>> Introduce apply_boot_alternatives() to allow some alternatives to be
>>>> applied immediately after we detect the CPU features of the boot CPU.
>>>>
>>>> Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
>>>> [julien.thierry@arm.com: rename to fit new cpufeature framework better,
>>>>               apply BOOT_SCOPE feature early in boot]
>>>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>>>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>>>> Cc: Will Deacon <will.deacon@arm.com>
>>>> Cc: Christoffer Dall <christoffer.dall@arm.com>
>>>> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
>>>> ---
>>>>    arch/arm64/include/asm/alternative.h |  1 +
>>>>    arch/arm64/include/asm/cpufeature.h  |  4 ++++
>>>>    arch/arm64/kernel/alternative.c      | 43
>>>> +++++++++++++++++++++++++++++++-----
>>>>    arch/arm64/kernel/cpufeature.c       |  6 +++++
>>>>    arch/arm64/kernel/smp.c              |  7 ++++++
>>>>    5 files changed, 56 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/arch/arm64/include/asm/alternative.h
>>>> b/arch/arm64/include/asm/alternative.h
>>>> index 9806a23..b9f8d78 100644
>>>> --- a/arch/arm64/include/asm/alternative.h
>>>> +++ b/arch/arm64/include/asm/alternative.h
>>>> @@ -25,6 +25,7 @@ struct alt_instr {
>>>>    typedef void (*alternative_cb_t)(struct alt_instr *alt,
>>>>                     __le32 *origptr, __le32 *updptr, int nr_inst);
>>>>    +void __init apply_boot_alternatives(void);
>>>>    void __init apply_alternatives_all(void);
>>>>    bool alternative_is_applied(u16 cpufeature);
>>>>    diff --git a/arch/arm64/include/asm/cpufeature.h
>>>> b/arch/arm64/include/asm/cpufeature.h
>>>> index 89c3f31..e505e1f 100644
>>>> --- a/arch/arm64/include/asm/cpufeature.h
>>>> +++ b/arch/arm64/include/asm/cpufeature.h
>>>> @@ -391,6 +391,10 @@ static inline int cpucap_default_scope(const
>>>> struct arm64_cpu_capabilities *cap)
>>>>    extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
>>>>    extern struct static_key_false arm64_const_caps_ready;
>>>>    +/* ARM64 CAPS + alternative_cb */
>>>> +#define ARM64_NPATCHABLE (ARM64_NCAPS + 1)
>>>> +extern DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>>>> +
>>>>    #define for_each_available_cap(cap)        \
>>>>        for_each_set_bit(cap, cpu_hwcaps, ARM64_NCAPS)
>>>>    diff --git a/arch/arm64/kernel/alternative.c
>>>> b/arch/arm64/kernel/alternative.c
>>>> index c947d22..a9b4677 100644
>>>> --- a/arch/arm64/kernel/alternative.c
>>>> +++ b/arch/arm64/kernel/alternative.c
>>>> @@ -155,7 +155,8 @@ static void clean_dcache_range_nopatch(u64 start,
>>>> u64 end)
>>>>        } while (cur += d_size, cur < end);
>>>>    }
>>>>    -static void __apply_alternatives(void *alt_region, bool is_module)
>>>> +static void __apply_alternatives(void *alt_region,  bool is_module,
>>>> +                 unsigned long *feature_mask)
>>>>    {
>>>>        struct alt_instr *alt;
>>>>        struct alt_region *region = alt_region;
>>>> @@ -165,6 +166,9 @@ static void __apply_alternatives(void *alt_region,
>>>> bool is_module)
>>>>        for (alt = region->begin; alt < region->end; alt++) {
>>>>            int nr_inst;
>>>>    +        if (!test_bit(alt->cpufeature, feature_mask))
>>>> +            continue;
>>>> +
>>>>            /* Use ARM64_CB_PATCH as an unconditional patch */
>>>>            if (alt->cpufeature < ARM64_CB_PATCH &&
>>>>                !cpus_have_cap(alt->cpufeature))
>>>> @@ -203,8 +207,11 @@ static void __apply_alternatives(void
>>>> *alt_region, bool is_module)
>>>>            __flush_icache_all();
>>>>            isb();
>>>>    -        /* We applied all that was available */
>>>> -        bitmap_copy(applied_alternatives, cpu_hwcaps, ARM64_NCAPS);
>>>> +        /* Ignore ARM64_CB bit from feature mask */
>>>> +        bitmap_or(applied_alternatives, applied_alternatives,
>>>> +              feature_mask, ARM64_NCAPS);
>>>> +        bitmap_and(applied_alternatives, applied_alternatives,
>>>> +               cpu_hwcaps, ARM64_NCAPS);
>>>>        }
>>>>    }
>>>>    @@ -225,8 +232,13 @@ static int __apply_alternatives_multi_stop(void
>>>> *unused)
>>>>                cpu_relax();
>>>>            isb();
>>>>        } else {
>>>> +        DECLARE_BITMAP(remaining_capabilities, ARM64_NPATCHABLE);
>>>> +
>>>> +        bitmap_complement(remaining_capabilities, boot_capabilities,
>>>> +                  ARM64_NPATCHABLE);
>>>> +
>>>>            BUG_ON(all_alternatives_applied);
>>>> -        __apply_alternatives(&region, false);
>>>> +        __apply_alternatives(&region, false, remaining_capabilities);
>>>>            /* Barriers provided by the cache flushing */
>>>>            WRITE_ONCE(all_alternatives_applied, 1);
>>>>        }
>>>> @@ -240,6 +252,24 @@ void __init apply_alternatives_all(void)
>>>>        stop_machine(__apply_alternatives_multi_stop, NULL,
>>>> cpu_online_mask);
>>>>    }
>>>>    +/*
>>>> + * This is called very early in the boot process (directly after we
>>>> run
>>>> + * a feature detect on the boot CPU). No need to worry about other
>>>> CPUs
>>>> + * here.
>>>> + */
>>>> +void __init apply_boot_alternatives(void)
>>>> +{
>>>> +    struct alt_region region = {
>>>> +        .begin    = (struct alt_instr *)__alt_instructions,
>>>> +        .end    = (struct alt_instr *)__alt_instructions_end,
>>>> +    };
>>>> +
>>>> +    /* If called on non-boot cpu things could go wrong */
>>>> +    WARN_ON(smp_processor_id() != 0);
>>>> +
>>>> +    __apply_alternatives(&region, false, &boot_capabilities[0]);
>>>> +}
>>>> +
>>>>    #ifdef CONFIG_MODULES
>>>>    void apply_alternatives_module(void *start, size_t length)
>>>>    {
>>>> @@ -247,7 +277,10 @@ void apply_alternatives_module(void *start,
>>>> size_t length)
>>>>            .begin    = start,
>>>>            .end    = start + length,
>>>>        };
>>>> +    DECLARE_BITMAP(all_capabilities, ARM64_NPATCHABLE);
>>>> +
>>>> +    bitmap_fill(all_capabilities, ARM64_NPATCHABLE);
>>>>    -    __apply_alternatives(&region, true);
>>>> +    __apply_alternatives(&region, true, &all_capabilities[0]);
>>>>    }
>>>>    #endif
>>>> diff --git a/arch/arm64/kernel/cpufeature.c
>>>> b/arch/arm64/kernel/cpufeature.c
>>>> index 84fa5be..71c8d4f 100644
>>>> --- a/arch/arm64/kernel/cpufeature.c
>>>> +++ b/arch/arm64/kernel/cpufeature.c
>>>> @@ -54,6 +54,9 @@
>>>>    EXPORT_SYMBOL(cpu_hwcaps);
>>>>    static struct arm64_cpu_capabilities const __ro_after_init
>>>> *cpu_hwcaps_ptrs[ARM64_NCAPS];
>>>>    +/* Need also bit for ARM64_CB_PATCH */
>>>> +DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
>>>> +
>>>>    /*
>>>>     * Flag to indicate if we have computed the system wide
>>>>     * capabilities based on the boot time active CPUs. This
>>>> @@ -1672,6 +1675,9 @@ static void update_cpu_capabilities(u16
>>>> scope_mask)
>>>>            if (caps->desc)
>>>>                pr_info("detected: %s\n", caps->desc);
>>>>            cpus_set_cap(caps->capability);
>>>> +
>>>> +        if (caps->type & SCOPE_BOOT_CPU)
>>>
>>> You may want to do :
>>>          if (scope_mask & SCOPE_BOOT_CPU)
>>>
>>> for a tighter check to ensure this doesn't update the boot_capabilities
>>> after we have applied the boot_scope alternatives and miss applying the
>>> alternatives for those, should someone add a multi-scope (i.e
>>> SCOPE_BOOT_CPU and
>>> something else) capability (even by mistake).
>>>
>>
>> But a multi-scope capability containing SCOPE_BOOT_CPU should already
>> get updated for setup_boot_cpu_capabilities. Capabilities marked with
>> SCOPE_BOOT_CPU need to be enabled on the boot CPU or not at all.
> 
> Yes, you're right. It is not normal to have multiple SCOPE for a
> "capability".
> But if someone comes with such a cap, we may miss handling this case. It is
> always better to be safer.
> 
>>
>> Shouldn't the call to caps->matches() fail for a boot feature that was
>> not found on the boot cpu?
>>
>> Also, you made the opposite suggestion 4 version ago with a more
>> worrying scenario :) :
>> https://lkml.org/lkml/2018/5/25/208
> 
> Ah, you're right. I missed that. We need the additional check as you
> mention
> below.
> 
>>
>> Otherwise, if my assumption above is wrong, it means the check should
>> probably be:
>>     if (caps->type & SCOPE_BOOT_CPU && scope_mask & SCOPE_BOOT_CPU)
> 
> Yes, this is what we want.

Yes you're right, the behaviour I was thinking of were the ID register,
where there views are altered for following CPUs when system wide
features are missing on the boot CPU. But ->matches() callbacks don't
only rely on ID registers.

I'll add that for the next version.

Thanks,

-- 
Julien Thierry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:19PM +0000, Julien Thierry wrote:
> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
> to interract with guest TLBs, switching from EL2&0 translation regime
> to EL1&0.
> 
> However, some non-maskable asynchronous event could happen while TGE is
> cleared like SDEI. Because of this address translation operations
> relying on EL2&0 translation regime could fail (tlb invalidation,
> userspace access, ...).

Why would an NMI context need to access user space? (just curious what
breaks exactly without this patch; otherwise it looks fine)

-- 
Catalin

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:19PM +0000, Julien Thierry wrote:
> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
> to interract with guest TLBs, switching from EL2&0 translation regime
> to EL1&0.
> 
> However, some non-maskable asynchronous event could happen while TGE is
> cleared like SDEI. Because of this address translation operations
> relying on EL2&0 translation regime could fail (tlb invalidation,
> userspace access, ...).

Why would an NMI context need to access user space? (just curious what
breaks exactly without this patch; otherwise it looks fine)

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 07/26] arm64: ptrace: Provide definitions for PMR values

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:25PM +0000, Julien Thierry wrote:
> Introduce fixed values for PMR that are going to be used to mask and
> unmask interrupts by priority.
> 
> The current priority given to GIC interrupts is 0xa0, so clearing PMR's
> most significant bit is enough to mask interrupts.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH v8 07/26] arm64: ptrace: Provide definitions for PMR values

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:25PM +0000, Julien Thierry wrote:
> Introduce fixed values for PMR that are going to be used to mask and
> unmask interrupts by priority.
> 
> The current priority given to GIC interrupts is 0xa0, so clearing PMR's
> most significant bit is enough to mask interrupts.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Julien Thierry]

On 14/01/2019 15:56, Catalin Marinas wrote:
> On Tue, Jan 08, 2019 at 02:07:19PM +0000, Julien Thierry wrote:
>> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
>> to interract with guest TLBs, switching from EL2&0 translation regime
>> to EL1&0.
>>
>> However, some non-maskable asynchronous event could happen while TGE is
>> cleared like SDEI. Because of this address translation operations
>> relying on EL2&0 translation regime could fail (tlb invalidation,
>> userspace access, ...).
> 
> Why would an NMI context need to access user space? (just curious what
> breaks exactly without this patch; otherwise it looks fine)

If I remember correctly, the SDEI interrupt might perform cache
maintenance with EL2&0 translation regime, but James can probably give
more detail (or correct me if I'm wrong).

Otherwise, if we decide to use the pseudo NMI for profiling with perf, I
believe the perf interrupt can access user space (although I'm not
completely sure whether that might be to record profiling data in
buffers shared with user space or something else).

Thanks,

-- 
Julien Thierry

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Julien Thierry]

On 14/01/2019 15:56, Catalin Marinas wrote:
> On Tue, Jan 08, 2019 at 02:07:19PM +0000, Julien Thierry wrote:
>> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
>> to interract with guest TLBs, switching from EL2&0 translation regime
>> to EL1&0.
>>
>> However, some non-maskable asynchronous event could happen while TGE is
>> cleared like SDEI. Because of this address translation operations
>> relying on EL2&0 translation regime could fail (tlb invalidation,
>> userspace access, ...).
> 
> Why would an NMI context need to access user space? (just curious what
> breaks exactly without this patch; otherwise it looks fine)

If I remember correctly, the SDEI interrupt might perform cache
maintenance with EL2&0 translation regime, but James can probably give
more detail (or correct me if I'm wrong).

Otherwise, if we decide to use the pseudo NMI for profiling with perf, I
believe the perf interrupt can access user space (although I'm not
completely sure whether that might be to record profiling data in
buffers shared with user space or something else).

Thanks,

-- 
Julien Thierry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[James Morse]

Hi guys,

On 14/01/2019 16:12, Julien Thierry wrote:
> On 14/01/2019 15:56, Catalin Marinas wrote:
>> On Tue, Jan 08, 2019 at 02:07:19PM +0000, Julien Thierry wrote:
>>> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
>>> to interract with guest TLBs, switching from EL2&0 translation regime
>>> to EL1&0.
>>>
>>> However, some non-maskable asynchronous event could happen while TGE is
>>> cleared like SDEI. Because of this address translation operations
>>> relying on EL2&0 translation regime could fail (tlb invalidation,
>>> userspace access, ...).
>>
>> Why would an NMI context need to access user space? (just curious what
>> breaks exactly without this patch; otherwise it looks fine)
> 
> If I remember correctly, the SDEI interrupt might perform cache
> maintenance with EL2&0 translation regime, but James can probably give
> more detail (or correct me if I'm wrong).

Yup, spot on.
The APEI driver has to map/unmap memory using the fixmap. If it interrupts a
guest, the TLB maintenance would affect EL1&0 instead.


> Otherwise, if we decide to use the pseudo NMI for profiling with perf, I
> believe the perf interrupt can access user space (although I'm not
> completely sure whether that might be to record profiling data in
> buffers shared with user space or something else).

It does a stack walk, I think its the PERF_SAMPLE_CALLCHAIN feature, and the
code is:
arch/arm64/kernel/perf_callchain.c::user_backtrace()


Thanks,

James

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[James Morse]

Hi guys,

On 14/01/2019 16:12, Julien Thierry wrote:
> On 14/01/2019 15:56, Catalin Marinas wrote:
>> On Tue, Jan 08, 2019 at 02:07:19PM +0000, Julien Thierry wrote:
>>> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
>>> to interract with guest TLBs, switching from EL2&0 translation regime
>>> to EL1&0.
>>>
>>> However, some non-maskable asynchronous event could happen while TGE is
>>> cleared like SDEI. Because of this address translation operations
>>> relying on EL2&0 translation regime could fail (tlb invalidation,
>>> userspace access, ...).
>>
>> Why would an NMI context need to access user space? (just curious what
>> breaks exactly without this patch; otherwise it looks fine)
> 
> If I remember correctly, the SDEI interrupt might perform cache
> maintenance with EL2&0 translation regime, but James can probably give
> more detail (or correct me if I'm wrong).

Yup, spot on.
The APEI driver has to map/unmap memory using the fixmap. If it interrupts a
guest, the TLB maintenance would affect EL1&0 instead.


> Otherwise, if we decide to use the pseudo NMI for profiling with perf, I
> believe the perf interrupt can access user space (although I'm not
> completely sure whether that might be to record profiling data in
> buffers shared with user space or something else).

It does a stack walk, I think its the PERF_SAMPLE_CALLCHAIN feature, and the
code is:
arch/arm64/kernel/perf_callchain.c::user_backtrace()


Thanks,

James

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Sasha Levin]

Hi,

[This is an automated email]

This commit has been processed because it contains a -stable tag.
The stable tag indicates that it's relevant for the following trees: all

The bot has tested the following trees: v4.20.2, v4.19.15, v4.14.93, v4.9.150, v4.4.170, v3.18.132.

v4.20.2: Build OK!
v4.19.15: Build OK!
v4.14.93: Build OK!
v4.9.150: Failed to apply! Possible dependencies:
    096683724cb2 ("arm64: unwind: avoid percpu indirection for irq stack")
    34be98f4944f ("arm64: kernel: remove {THREAD,IRQ_STACK}_START_SP")
    a9ea0017ebe8 ("arm64: factor out current_stack_pointer")
    c02433dd6de3 ("arm64: split thread_info from task stack")
    c7365330753c ("arm64: unwind: disregard frame.sp when validating frame pointer")
    dbc9344a68e5 ("arm64: clean up THREAD_* definitions")
    f60ad4edcf07 ("arm64: clean up irq stack definitions")
    f60fe78f1332 ("arm64: use an irq stack pointer")

v4.4.170: Failed to apply! Possible dependencies:
    096683724cb2 ("arm64: unwind: avoid percpu indirection for irq stack")
    0a8ea52c3eb1 ("arm64: Add HAVE_REGS_AND_STACK_ACCESS_API feature")
    132cd887b5c5 ("arm64: Modify stack trace and dump for use with irq_stack")
    1ffe199b1c9b ("arm64: when walking onto the task stack, check sp & fp are in current->stack")
    20380bb390a4 ("arm64: ftrace: fix a stack tracer's output under function graph tracer")
    7596abf2e566 ("arm64: irq: fix walking from irq stack to task stack")
    8e23dacd12a4 ("arm64: Add do_softirq_own_stack() and enable irq_stacks")
    971c67ce37cf ("arm64: reduce stack use in irq_handler")
    a80a0eb70c35 ("arm64: make irq_stack_ptr more robust")
    c7365330753c ("arm64: unwind: disregard frame.sp when validating frame pointer")
    f60ad4edcf07 ("arm64: clean up irq stack definitions")
    f60fe78f1332 ("arm64: use an irq stack pointer")
    fe13f95b7200 ("arm64: pass a task parameter to unwind_frame()")

v3.18.132: Failed to apply! Possible dependencies:
    020295b4cb5b ("ACPI / processor: Make it possible to get CPU hardware ID via GICC")
    132cd887b5c5 ("arm64: Modify stack trace and dump for use with irq_stack")
    13ca62b243f6 ("ACPI: Fix minor syntax issues in processor_core.c")
    37655163ce1a ("ARM64 / ACPI: Get RSDP and ACPI boot-time tables")
    587064b610c7 ("arm64: Add framework for legacy instruction emulation")
    652261a7a86c ("ACPI: fix acpi_os_ioremap for arm64")
    828aef376d7a ("ACPI / processor: Introduce phys_cpuid_t for CPU hardware ID")
    96f0e00378d4 ("ARM: add basic support for on-demand backtrace of other CPUs")
    af2c632e234f ("arm64/include/asm: Fixed a warning about 'struct pt_regs'")
    af8f3f514d19 ("ACPI / processor: Convert apic_id to phys_id to make it arch agnostic")
    b4ff8389ed14 ("xen/events: Always allocate legacy interrupts on PV guests")
    d02dc27db0dc ("ACPI / processor: Rename acpi_(un)map_lsapic() to acpi_(un)map_cpu()")
    d60fc3892c4d ("irqchip: Add GICv2 specific ACPI boot support")
    ecf5636dcd59 ("ACPI: Add interfaces to parse IOAPIC ID for IOAPIC hotplug")
    f60ad4edcf07 ("arm64: clean up irq stack definitions")
    f60fe78f1332 ("arm64: use an irq stack pointer")


How should we proceed with this patch?

--
Thanks,
Sasha

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Sasha Levin]

Hi,

[This is an automated email]

This commit has been processed because it contains a -stable tag.
The stable tag indicates that it's relevant for the following trees: all

The bot has tested the following trees: v4.20.2, v4.19.15, v4.14.93, v4.9.150, v4.4.170, v3.18.132.

v4.20.2: Build OK!
v4.19.15: Build OK!
v4.14.93: Build OK!
v4.9.150: Failed to apply! Possible dependencies:
    096683724cb2 ("arm64: unwind: avoid percpu indirection for irq stack")
    34be98f4944f ("arm64: kernel: remove {THREAD,IRQ_STACK}_START_SP")
    a9ea0017ebe8 ("arm64: factor out current_stack_pointer")
    c02433dd6de3 ("arm64: split thread_info from task stack")
    c7365330753c ("arm64: unwind: disregard frame.sp when validating frame pointer")
    dbc9344a68e5 ("arm64: clean up THREAD_* definitions")
    f60ad4edcf07 ("arm64: clean up irq stack definitions")
    f60fe78f1332 ("arm64: use an irq stack pointer")

v4.4.170: Failed to apply! Possible dependencies:
    096683724cb2 ("arm64: unwind: avoid percpu indirection for irq stack")
    0a8ea52c3eb1 ("arm64: Add HAVE_REGS_AND_STACK_ACCESS_API feature")
    132cd887b5c5 ("arm64: Modify stack trace and dump for use with irq_stack")
    1ffe199b1c9b ("arm64: when walking onto the task stack, check sp & fp are in current->stack")
    20380bb390a4 ("arm64: ftrace: fix a stack tracer's output under function graph tracer")
    7596abf2e566 ("arm64: irq: fix walking from irq stack to task stack")
    8e23dacd12a4 ("arm64: Add do_softirq_own_stack() and enable irq_stacks")
    971c67ce37cf ("arm64: reduce stack use in irq_handler")
    a80a0eb70c35 ("arm64: make irq_stack_ptr more robust")
    c7365330753c ("arm64: unwind: disregard frame.sp when validating frame pointer")
    f60ad4edcf07 ("arm64: clean up irq stack definitions")
    f60fe78f1332 ("arm64: use an irq stack pointer")
    fe13f95b7200 ("arm64: pass a task parameter to unwind_frame()")

v3.18.132: Failed to apply! Possible dependencies:
    020295b4cb5b ("ACPI / processor: Make it possible to get CPU hardware ID via GICC")
    132cd887b5c5 ("arm64: Modify stack trace and dump for use with irq_stack")
    13ca62b243f6 ("ACPI: Fix minor syntax issues in processor_core.c")
    37655163ce1a ("ARM64 / ACPI: Get RSDP and ACPI boot-time tables")
    587064b610c7 ("arm64: Add framework for legacy instruction emulation")
    652261a7a86c ("ACPI: fix acpi_os_ioremap for arm64")
    828aef376d7a ("ACPI / processor: Introduce phys_cpuid_t for CPU hardware ID")
    96f0e00378d4 ("ARM: add basic support for on-demand backtrace of other CPUs")
    af2c632e234f ("arm64/include/asm: Fixed a warning about 'struct pt_regs'")
    af8f3f514d19 ("ACPI / processor: Convert apic_id to phys_id to make it arch agnostic")
    b4ff8389ed14 ("xen/events: Always allocate legacy interrupts on PV guests")
    d02dc27db0dc ("ACPI / processor: Rename acpi_(un)map_lsapic() to acpi_(un)map_cpu()")
    d60fc3892c4d ("irqchip: Add GICv2 specific ACPI boot support")
    ecf5636dcd59 ("ACPI: Add interfaces to parse IOAPIC ID for IOAPIC hotplug")
    f60ad4edcf07 ("arm64: clean up irq stack definitions")
    f60fe78f1332 ("arm64: use an irq stack pointer")


How should we proceed with this patch?

--
Thanks,
Sasha

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Catalin Marinas]

Hi Julien,

On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> + * Having two ways to control interrupt status is a bit complicated. Some
> + * locations like exception entries will have PSR.I bit set by the architecture
> + * while PMR is unmasked.
> + * We need the irqflags to represent that interrupts are disabled in such cases.
> + *
> + * For this, we lower the value read from PMR when the I bit is set so it is
> + * considered as an irq masking priority. (With PMR, lower value means masking
> + * more interrupts).
> + */
> +#define _get_irqflags(daif_bits, pmr)					\
> +({									\
> +	unsigned long flags;						\
> +									\
> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
> +	asm volatile(ALTERNATIVE(					\
> +		"mov	%0, %1\n"					\
> +		"nop\n"							\
> +		"nop",							\
> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
> +		"mvn	%0, %0\n"					\
> +		"and	%0, %0, %2",					\
> +		ARM64_HAS_IRQ_PRIO_MASKING)				\

Can you write the last two instructions as a single:

		bic	%0, %2, %0

> +		: "=&r" (flags)						\
> +		: "r" (daif_bits), "r" (pmr)				\
> +		: "memory");						\
> +									\
> +	flags;								\
> +})
> +
> +/*
>   * Save the current interrupt enable state.
>   */
>  static inline unsigned long arch_local_save_flags(void)
>  {
> -	unsigned long flags;
> -	asm volatile(
> -		"mrs	%0, daif		// arch_local_save_flags"
> -		: "=r" (flags)
> +	unsigned long daif_bits;
> +	unsigned long pmr; // Only used if alternative is on
> +
> +	daif_bits = read_sysreg(daif);
> +
> +	// Get PMR

Nitpick: don't use C++ (or arm asm) comment style in C code.

> +	asm volatile(ALTERNATIVE(
> +			"nop",
> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> +			ARM64_HAS_IRQ_PRIO_MASKING)
> +		: "=&r" (pmr)
>  		:
>  		: "memory");
> +
> +	return _get_irqflags(daif_bits, pmr);
> +}

I find this confusing spread over two inline asm statements. IIUC, you
want something like below (it could be written as inline asm but I need
to understand it first):

	daif_bits = read_sysreg(daif);

	if (system_uses_irq_prio_masking()) {
		pmr = read_gicreg(ICC_PMR_EL1);
		flags = pmr & ~(daif_bits & PSR_I_BIT);
	} else {
		flags = daif_bits;
	}

	return flags;

In the case where the interrupts are disabled at the PSR level, is the
PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
Something like:

	flags = read_sysreg(daif);

	if (system_uses_irq_prio_masking())
		flags = flags & PSR_I_BIT ?
			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);

-- 
Catalin

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Catalin Marinas]

Hi Julien,

On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> + * Having two ways to control interrupt status is a bit complicated. Some
> + * locations like exception entries will have PSR.I bit set by the architecture
> + * while PMR is unmasked.
> + * We need the irqflags to represent that interrupts are disabled in such cases.
> + *
> + * For this, we lower the value read from PMR when the I bit is set so it is
> + * considered as an irq masking priority. (With PMR, lower value means masking
> + * more interrupts).
> + */
> +#define _get_irqflags(daif_bits, pmr)					\
> +({									\
> +	unsigned long flags;						\
> +									\
> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
> +	asm volatile(ALTERNATIVE(					\
> +		"mov	%0, %1\n"					\
> +		"nop\n"							\
> +		"nop",							\
> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
> +		"mvn	%0, %0\n"					\
> +		"and	%0, %0, %2",					\
> +		ARM64_HAS_IRQ_PRIO_MASKING)				\

Can you write the last two instructions as a single:

		bic	%0, %2, %0

> +		: "=&r" (flags)						\
> +		: "r" (daif_bits), "r" (pmr)				\
> +		: "memory");						\
> +									\
> +	flags;								\
> +})
> +
> +/*
>   * Save the current interrupt enable state.
>   */
>  static inline unsigned long arch_local_save_flags(void)
>  {
> -	unsigned long flags;
> -	asm volatile(
> -		"mrs	%0, daif		// arch_local_save_flags"
> -		: "=r" (flags)
> +	unsigned long daif_bits;
> +	unsigned long pmr; // Only used if alternative is on
> +
> +	daif_bits = read_sysreg(daif);
> +
> +	// Get PMR

Nitpick: don't use C++ (or arm asm) comment style in C code.

> +	asm volatile(ALTERNATIVE(
> +			"nop",
> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> +			ARM64_HAS_IRQ_PRIO_MASKING)
> +		: "=&r" (pmr)
>  		:
>  		: "memory");
> +
> +	return _get_irqflags(daif_bits, pmr);
> +}

I find this confusing spread over two inline asm statements. IIUC, you
want something like below (it could be written as inline asm but I need
to understand it first):

	daif_bits = read_sysreg(daif);

	if (system_uses_irq_prio_masking()) {
		pmr = read_gicreg(ICC_PMR_EL1);
		flags = pmr & ~(daif_bits & PSR_I_BIT);
	} else {
		flags = daif_bits;
	}

	return flags;

In the case where the interrupts are disabled at the PSR level, is the
PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
Something like:

	flags = read_sysreg(daif);

	if (system_uses_irq_prio_masking())
		flags = flags & PSR_I_BIT ?
			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 08/26] arm64: Make PMR part of task context

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:26PM +0000, Julien Thierry wrote:
> In order to replace PSR.I interrupt disabling/enabling with ICC_PMR_EL1
> interrupt masking, ICC_PMR_EL1 needs to be saved/restored when
> taking/returning from an exception. This mimics the way hardware saves
> and restores PSR.I bit in spsr_el1 for exceptions and ERET.
> 
> Add PMR to the registers to save in the pt_regs struct upon kernel entry,
> and restore it before ERET. Also, initialize it to a sane value when
> creating new tasks.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Dave Martin <Dave.Martin@arm.com>

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH v8 08/26] arm64: Make PMR part of task context

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:26PM +0000, Julien Thierry wrote:
> In order to replace PSR.I interrupt disabling/enabling with ICC_PMR_EL1
> interrupt masking, ICC_PMR_EL1 needs to be saved/restored when
> taking/returning from an exception. This mimics the way hardware saves
> and restores PSR.I bit in spsr_el1 for exceptions and ERET.
> 
> Add PMR to the registers to save in the pt_regs struct upon kernel entry,
> and restore it before ERET. Also, initialize it to a sane value when
> creating new tasks.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Dave Martin <Dave.Martin@arm.com>

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 09/26] arm64: Unmask PMR before going idle

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:27PM +0000, Julien Thierry wrote:
> CPU does not received signals for interrupts with a priority masked by
> ICC_PMR_EL1. This means the CPU might not come back from a WFI
> instruction.
> 
> Make sure ICC_PMR_EL1 does not mask interrupts when doing a WFI.
> 
> Since the logic of cpu_do_idle is becoming a bit more complex than just
> two instructions, lets turn it from ASM to C.

I haven't checked all the call paths, so asking here: does the core code
normally call arch_cpu_idle() with IRQs off?

-- 
Catalin

Re: [PATCH v8 09/26] arm64: Unmask PMR before going idle

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:27PM +0000, Julien Thierry wrote:
> CPU does not received signals for interrupts with a priority masked by
> ICC_PMR_EL1. This means the CPU might not come back from a WFI
> instruction.
> 
> Make sure ICC_PMR_EL1 does not mask interrupts when doing a WFI.
> 
> Since the logic of cpu_do_idle is becoming a bit more complex than just
> two instructions, lets turn it from ASM to C.

I haven't checked all the call paths, so asking here: does the core code
normally call arch_cpu_idle() with IRQs off?

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 10/26] arm64: kvm: Unmask PMR before entering guest

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:28PM +0000, Julien Thierry wrote:
> Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This
> means that hypervisor will not receive masked interrupts while running a
> guest.
> 
> Avoid this by making sure ICC_PMR_EL1 is unmasked when we enter a guest.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Cc: Christoffer Dall <christoffer.dall@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: kvmarm@lists.cs.columbia.edu

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH v8 10/26] arm64: kvm: Unmask PMR before entering guest

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:28PM +0000, Julien Thierry wrote:
> Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This
> means that hypervisor will not receive masked interrupts while running a
> guest.
> 
> Avoid this by making sure ICC_PMR_EL1 is unmasked when we enter a guest.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Cc: Christoffer Dall <christoffer.dall@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: kvmarm@lists.cs.columbia.edu

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 11/26] efi: Let architectures decide the flags that should be saved/restored

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:29PM +0000, Julien Thierry wrote:
> Currently, irqflags are saved before calling runtime services and
> checked for mismatch on return.
> 
> Provide a pair of overridable macros to save and restore (if needed) the
> state that need to be preserved on return from a runtime service.
> This allows to check for flags that are not necesarly related to
> irqflags.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH v8 11/26] efi: Let architectures decide the flags that should be saved/restored

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:29PM +0000, Julien Thierry wrote:
> Currently, irqflags are saved before calling runtime services and
> checked for mismatch on return.
> 
> Provide a pair of overridable macros to save and restore (if needed) the
> state that need to be preserved on return from a runtime service.
> This allows to check for flags that are not necesarly related to
> irqflags.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> Instead disabling interrupts by setting the PSR.I bit, use a priority
> higher than the one used for interrupts to mask them via PMR.
> 
> When using PMR to disable interrupts, the value of PMR will be used
> instead of PSR.[DAIF] for the irqflags.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> ---
>  arch/arm64/include/asm/efi.h      |  11 ++++
>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>  2 files changed, 106 insertions(+), 28 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
> index 7ed3208..134ff6e 100644
> --- a/arch/arm64/include/asm/efi.h
> +++ b/arch/arm64/include/asm/efi.h
> @@ -44,6 +44,17 @@
>  
>  #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
>  
> +#define arch_efi_save_flags(state_flags)		\
> +	do {						\
> +		(state_flags) =	read_sysreg(daif);	\
> +	} while (0)
> +
> +#define arch_efi_restore_flags(state_flags)		\
> +	do {						\
> +		write_sysreg(state_flags, daif);	\
> +	} while (0)
> +
> +

Randomly commenting a few minor nits as I glance down my mailbox...

There's no need to protect single statements with do { } while(0).

Just protect an expression statement that could be misparsed with ( ).

->

#define arch_efi_save_flags(state_flags) ((state_flags) = read_sysreg(daif))
#define arch_efi_restore_flags(state_flags) write_sysreg(state_flags, daif)

[...]

> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> index 24692ed..fa3b06f 100644
> --- a/arch/arm64/include/asm/irqflags.h
> +++ b/arch/arm64/include/asm/irqflags.h
> @@ -18,7 +18,9 @@
>  
>  #ifdef __KERNEL__
>  
> +#include <asm/alternative.h>
>  #include <asm/ptrace.h>
> +#include <asm/sysreg.h>
>  
>  /*
>   * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
> @@ -36,47 +38,96 @@

[...]

>  /*
> + * Having two ways to control interrupt status is a bit complicated. Some
> + * locations like exception entries will have PSR.I bit set by the architecture
> + * while PMR is unmasked.
> + * We need the irqflags to represent that interrupts are disabled in such cases.
> + *
> + * For this, we lower the value read from PMR when the I bit is set so it is
> + * considered as an irq masking priority. (With PMR, lower value means masking
> + * more interrupts).
> + */
> +#define _get_irqflags(daif_bits, pmr)					\
> +({									\
> +	unsigned long flags;						\
> +									\
> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
> +	asm volatile(ALTERNATIVE(					\
> +		"mov	%0, %1\n"					\
> +		"nop\n"							\
> +		"nop",							\
> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
> +		"mvn	%0, %0\n"					\
> +		"and	%0, %0, %2",					\
> +		ARM64_HAS_IRQ_PRIO_MASKING)				\
> +		: "=&r" (flags)						\
> +		: "r" (daif_bits), "r" (pmr)				\
> +		: "memory");						\
> +									\
> +	flags;								\
> +})

Nit: does this need to be a macro?

({ ... }) is mildly gross and it's preferable to avoid it if the code
works just as well without...

pmr would need to be passed as a pointer, with "r" (*pmr) in the asm,
but I think it would compile down to precisely the same code.

> +
> +/*
>   * Save the current interrupt enable state.
>   */
>  static inline unsigned long arch_local_save_flags(void)
>  {
> -	unsigned long flags;
> -	asm volatile(
> -		"mrs	%0, daif		// arch_local_save_flags"
> -		: "=r" (flags)
> +	unsigned long daif_bits;
> +	unsigned long pmr; // Only used if alternative is on
> +
> +	daif_bits = read_sysreg(daif);
> +
> +	// Get PMR
> +	asm volatile(ALTERNATIVE(
> +			"nop",
> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> +			ARM64_HAS_IRQ_PRIO_MASKING)
> +		: "=&r" (pmr)

Why earlyclobber?

>  		:
>  		: "memory");

[...]

> @@ -85,16 +136,32 @@ static inline unsigned long arch_local_save_flags(void)

[...]

>  static inline int arch_irqs_disabled_flags(unsigned long flags)
>  {
> -	return flags & PSR_I_BIT;
> +	int res;
> +
> +	asm volatile(ALTERNATIVE(
> +			"and	%w0, %w1, #" __stringify(PSR_I_BIT) "\n"
> +			"nop",
> +			"cmp	%w1, #" __stringify(GIC_PRIO_IRQOFF) "\n"
> +			"cset	%w0, ls",
> +			ARM64_HAS_IRQ_PRIO_MASKING)
> +		: "=&r" (res)

Why earlyclobber?  %0 is not written before the reading of any input
argument so far as I can see, in either alternative.

Cheers
---Dave

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> Instead disabling interrupts by setting the PSR.I bit, use a priority
> higher than the one used for interrupts to mask them via PMR.
> 
> When using PMR to disable interrupts, the value of PMR will be used
> instead of PSR.[DAIF] for the irqflags.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> ---
>  arch/arm64/include/asm/efi.h      |  11 ++++
>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>  2 files changed, 106 insertions(+), 28 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
> index 7ed3208..134ff6e 100644
> --- a/arch/arm64/include/asm/efi.h
> +++ b/arch/arm64/include/asm/efi.h
> @@ -44,6 +44,17 @@
>  
>  #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
>  
> +#define arch_efi_save_flags(state_flags)		\
> +	do {						\
> +		(state_flags) =	read_sysreg(daif);	\
> +	} while (0)
> +
> +#define arch_efi_restore_flags(state_flags)		\
> +	do {						\
> +		write_sysreg(state_flags, daif);	\
> +	} while (0)
> +
> +

Randomly commenting a few minor nits as I glance down my mailbox...

There's no need to protect single statements with do { } while(0).

Just protect an expression statement that could be misparsed with ( ).

->

#define arch_efi_save_flags(state_flags) ((state_flags) = read_sysreg(daif))
#define arch_efi_restore_flags(state_flags) write_sysreg(state_flags, daif)

[...]

> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> index 24692ed..fa3b06f 100644
> --- a/arch/arm64/include/asm/irqflags.h
> +++ b/arch/arm64/include/asm/irqflags.h
> @@ -18,7 +18,9 @@
>  
>  #ifdef __KERNEL__
>  
> +#include <asm/alternative.h>
>  #include <asm/ptrace.h>
> +#include <asm/sysreg.h>
>  
>  /*
>   * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
> @@ -36,47 +38,96 @@

[...]

>  /*
> + * Having two ways to control interrupt status is a bit complicated. Some
> + * locations like exception entries will have PSR.I bit set by the architecture
> + * while PMR is unmasked.
> + * We need the irqflags to represent that interrupts are disabled in such cases.
> + *
> + * For this, we lower the value read from PMR when the I bit is set so it is
> + * considered as an irq masking priority. (With PMR, lower value means masking
> + * more interrupts).
> + */
> +#define _get_irqflags(daif_bits, pmr)					\
> +({									\
> +	unsigned long flags;						\
> +									\
> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
> +	asm volatile(ALTERNATIVE(					\
> +		"mov	%0, %1\n"					\
> +		"nop\n"							\
> +		"nop",							\
> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
> +		"mvn	%0, %0\n"					\
> +		"and	%0, %0, %2",					\
> +		ARM64_HAS_IRQ_PRIO_MASKING)				\
> +		: "=&r" (flags)						\
> +		: "r" (daif_bits), "r" (pmr)				\
> +		: "memory");						\
> +									\
> +	flags;								\
> +})

Nit: does this need to be a macro?

({ ... }) is mildly gross and it's preferable to avoid it if the code
works just as well without...

pmr would need to be passed as a pointer, with "r" (*pmr) in the asm,
but I think it would compile down to precisely the same code.

> +
> +/*
>   * Save the current interrupt enable state.
>   */
>  static inline unsigned long arch_local_save_flags(void)
>  {
> -	unsigned long flags;
> -	asm volatile(
> -		"mrs	%0, daif		// arch_local_save_flags"
> -		: "=r" (flags)
> +	unsigned long daif_bits;
> +	unsigned long pmr; // Only used if alternative is on
> +
> +	daif_bits = read_sysreg(daif);
> +
> +	// Get PMR
> +	asm volatile(ALTERNATIVE(
> +			"nop",
> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> +			ARM64_HAS_IRQ_PRIO_MASKING)
> +		: "=&r" (pmr)

Why earlyclobber?

>  		:
>  		: "memory");

[...]

> @@ -85,16 +136,32 @@ static inline unsigned long arch_local_save_flags(void)

[...]

>  static inline int arch_irqs_disabled_flags(unsigned long flags)
>  {
> -	return flags & PSR_I_BIT;
> +	int res;
> +
> +	asm volatile(ALTERNATIVE(
> +			"and	%w0, %w1, #" __stringify(PSR_I_BIT) "\n"
> +			"nop",
> +			"cmp	%w1, #" __stringify(GIC_PRIO_IRQOFF) "\n"
> +			"cset	%w0, ls",
> +			ARM64_HAS_IRQ_PRIO_MASKING)
> +		: "=&r" (res)

Why earlyclobber?  %0 is not written before the reading of any input
argument so far as I can see, in either alternative.

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 13/26] arm64: daifflags: Include PMR in daifflags restore operations

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:31PM +0000, Julien Thierry wrote:
> The addition of PMR should not bypass the semantics of daifflags.
> 
> When DA_F are set, I bit is also set as no interrupts (even of higher
> priority) is allowed.
> 
> When DA_F are cleared, I bit is cleared and interrupt enabling/disabling
> goes through ICC_PMR_EL1.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: James Morse <james.morse@arm.com>

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH v8 13/26] arm64: daifflags: Include PMR in daifflags restore operations

[Catalin Marinas]

On Tue, Jan 08, 2019 at 02:07:31PM +0000, Julien Thierry wrote:
> The addition of PMR should not bypass the semantics of daifflags.
> 
> When DA_F are set, I bit is also set as no interrupts (even of higher
> priority) is allowed.
> 
> When DA_F are cleared, I bit is cleared and interrupt enabling/disabling
> goes through ICC_PMR_EL1.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: James Morse <james.morse@arm.com>

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

Hi Catalin,

On 18/01/2019 16:09, Catalin Marinas wrote:
> Hi Julien,
> 
> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>> + * Having two ways to control interrupt status is a bit complicated. Some
>> + * locations like exception entries will have PSR.I bit set by the architecture
>> + * while PMR is unmasked.
>> + * We need the irqflags to represent that interrupts are disabled in such cases.
>> + *
>> + * For this, we lower the value read from PMR when the I bit is set so it is
>> + * considered as an irq masking priority. (With PMR, lower value means masking
>> + * more interrupts).
>> + */
>> +#define _get_irqflags(daif_bits, pmr)					\
>> +({									\
>> +	unsigned long flags;						\
>> +									\
>> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
>> +	asm volatile(ALTERNATIVE(					\
>> +		"mov	%0, %1\n"					\
>> +		"nop\n"							\
>> +		"nop",							\
>> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
>> +		"mvn	%0, %0\n"					\
>> +		"and	%0, %0, %2",					\
>> +		ARM64_HAS_IRQ_PRIO_MASKING)				\
> 
> Can you write the last two instructions as a single:
> 
> 		bic	%0, %2, %0

Yes, makes sense. Although we won't need it anymore with your suggestion
below.

> 
>> +		: "=&r" (flags)						\
>> +		: "r" (daif_bits), "r" (pmr)				\
>> +		: "memory");						\
>> +									\
>> +	flags;								\
>> +})
>> +
>> +/*
>>   * Save the current interrupt enable state.
>>   */
>>  static inline unsigned long arch_local_save_flags(void)
>>  {
>> -	unsigned long flags;
>> -	asm volatile(
>> -		"mrs	%0, daif		// arch_local_save_flags"
>> -		: "=r" (flags)
>> +	unsigned long daif_bits;
>> +	unsigned long pmr; // Only used if alternative is on
>> +
>> +	daif_bits = read_sysreg(daif);
>> +
>> +	// Get PMR
> 
> Nitpick: don't use C++ (or arm asm) comment style in C code.

Noted.

> 
>> +	asm volatile(ALTERNATIVE(
>> +			"nop",
>> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
>> +			ARM64_HAS_IRQ_PRIO_MASKING)
>> +		: "=&r" (pmr)
>>  		:
>>  		: "memory");
>> +
>> +	return _get_irqflags(daif_bits, pmr);
>> +}
> 
> I find this confusing spread over two inline asm statements. IIUC, you
> want something like below (it could be written as inline asm but I need
> to understand it first):
> 
> 	daif_bits = read_sysreg(daif);
> 
> 	if (system_uses_irq_prio_masking()) {
> 		pmr = read_gicreg(ICC_PMR_EL1);
> 		flags = pmr & ~(daif_bits & PSR_I_BIT);
> 	} else {
> 		flags = daif_bits;
> 	}
> 
> 	return flags;
> 
> In the case where the interrupts are disabled at the PSR level, is the
> PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
> Something like:
> 
> 	flags = read_sysreg(daif);
> 
> 	if (system_uses_irq_prio_masking())
> 		flags = flags & PSR_I_BIT ?
> 			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);
> 

You're right, returning GIC_PRIO_IRQOFF should be good enough (it is
actually what happens in this version because GIC_PRIO_IRQOFF ==
GIC_PRIO_IRQON & ~PSR_I_BIT happens to be true). Your suggestion would
make things easier to reason about. Maybe something like:


static inline unsigned long arch_local_save_flags(void)
{
	unsigned long daif_bits;
	unsigned long prio_off = GIC_PRIO_IRQOFF;

	daif_bits = read_sysreg(daif);

	asm volatile(ALTERNATIVE(
		"mov	%0, %1\n"
		"nop\n"
		"nop",
		"mrs	%0, SYS_ICC_PMR_EL1\n"
		"ands	%1, %1, PSR_I_BIT\n"
		"csel	%0, %0, %2, eq")
	: "=&r" (flags)
	: "r" (daif_bits), "r" (prio_off)
	: "memory");

	return flags;
}

(Looks like it removes one nop from the alternative as well, unless I
messed up something)

Does that seem better to you?

Thanks,

-- 
Julien Thierry

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

Hi Catalin,

On 18/01/2019 16:09, Catalin Marinas wrote:
> Hi Julien,
> 
> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>> + * Having two ways to control interrupt status is a bit complicated. Some
>> + * locations like exception entries will have PSR.I bit set by the architecture
>> + * while PMR is unmasked.
>> + * We need the irqflags to represent that interrupts are disabled in such cases.
>> + *
>> + * For this, we lower the value read from PMR when the I bit is set so it is
>> + * considered as an irq masking priority. (With PMR, lower value means masking
>> + * more interrupts).
>> + */
>> +#define _get_irqflags(daif_bits, pmr)					\
>> +({									\
>> +	unsigned long flags;						\
>> +									\
>> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
>> +	asm volatile(ALTERNATIVE(					\
>> +		"mov	%0, %1\n"					\
>> +		"nop\n"							\
>> +		"nop",							\
>> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
>> +		"mvn	%0, %0\n"					\
>> +		"and	%0, %0, %2",					\
>> +		ARM64_HAS_IRQ_PRIO_MASKING)				\
> 
> Can you write the last two instructions as a single:
> 
> 		bic	%0, %2, %0

Yes, makes sense. Although we won't need it anymore with your suggestion
below.

> 
>> +		: "=&r" (flags)						\
>> +		: "r" (daif_bits), "r" (pmr)				\
>> +		: "memory");						\
>> +									\
>> +	flags;								\
>> +})
>> +
>> +/*
>>   * Save the current interrupt enable state.
>>   */
>>  static inline unsigned long arch_local_save_flags(void)
>>  {
>> -	unsigned long flags;
>> -	asm volatile(
>> -		"mrs	%0, daif		// arch_local_save_flags"
>> -		: "=r" (flags)
>> +	unsigned long daif_bits;
>> +	unsigned long pmr; // Only used if alternative is on
>> +
>> +	daif_bits = read_sysreg(daif);
>> +
>> +	// Get PMR
> 
> Nitpick: don't use C++ (or arm asm) comment style in C code.

Noted.

> 
>> +	asm volatile(ALTERNATIVE(
>> +			"nop",
>> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
>> +			ARM64_HAS_IRQ_PRIO_MASKING)
>> +		: "=&r" (pmr)
>>  		:
>>  		: "memory");
>> +
>> +	return _get_irqflags(daif_bits, pmr);
>> +}
> 
> I find this confusing spread over two inline asm statements. IIUC, you
> want something like below (it could be written as inline asm but I need
> to understand it first):
> 
> 	daif_bits = read_sysreg(daif);
> 
> 	if (system_uses_irq_prio_masking()) {
> 		pmr = read_gicreg(ICC_PMR_EL1);
> 		flags = pmr & ~(daif_bits & PSR_I_BIT);
> 	} else {
> 		flags = daif_bits;
> 	}
> 
> 	return flags;
> 
> In the case where the interrupts are disabled at the PSR level, is the
> PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
> Something like:
> 
> 	flags = read_sysreg(daif);
> 
> 	if (system_uses_irq_prio_masking())
> 		flags = flags & PSR_I_BIT ?
> 			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);
> 

You're right, returning GIC_PRIO_IRQOFF should be good enough (it is
actually what happens in this version because GIC_PRIO_IRQOFF ==
GIC_PRIO_IRQON & ~PSR_I_BIT happens to be true). Your suggestion would
make things easier to reason about. Maybe something like:


static inline unsigned long arch_local_save_flags(void)
{
	unsigned long daif_bits;
	unsigned long prio_off = GIC_PRIO_IRQOFF;

	daif_bits = read_sysreg(daif);

	asm volatile(ALTERNATIVE(
		"mov	%0, %1\n"
		"nop\n"
		"nop",
		"mrs	%0, SYS_ICC_PMR_EL1\n"
		"ands	%1, %1, PSR_I_BIT\n"
		"csel	%0, %0, %2, eq")
	: "=&r" (flags)
	: "r" (daif_bits), "r" (prio_off)
	: "memory");

	return flags;
}

(Looks like it removes one nop from the alternative as well, unless I
messed up something)

Does that seem better to you?

Thanks,

-- 
Julien Thierry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 09/26] arm64: Unmask PMR before going idle

[Julien Thierry]

On 18/01/2019 16:23, Catalin Marinas wrote:
> On Tue, Jan 08, 2019 at 02:07:27PM +0000, Julien Thierry wrote:
>> CPU does not received signals for interrupts with a priority masked by
>> ICC_PMR_EL1. This means the CPU might not come back from a WFI
>> instruction.
>>
>> Make sure ICC_PMR_EL1 does not mask interrupts when doing a WFI.
>>
>> Since the logic of cpu_do_idle is becoming a bit more complex than just
>> two instructions, lets turn it from ASM to C.
> 
> I haven't checked all the call paths, so asking here: does the core code
> normally call arch_cpu_idle() with IRQs off?
> 

If you look at do_idle() in kernel/sched/idle.c, it does:

	while (!need_resched()) {
		[...]
		local_irq_disable();
		arch_cpu_idle_enter();

		[...]
		} else {
			cpuidle_idle_call();
		}
	}


Then cpuidle_idle_call() will call default_idle_call() (which calls
arch_cpu_idle()) either directly or via the cpuidle driver. My
understanding is that it always does the call with interrupts disabled.

Thanks,

-- 
Julien Thierry

Re: [PATCH v8 09/26] arm64: Unmask PMR before going idle

[Julien Thierry]

On 18/01/2019 16:23, Catalin Marinas wrote:
> On Tue, Jan 08, 2019 at 02:07:27PM +0000, Julien Thierry wrote:
>> CPU does not received signals for interrupts with a priority masked by
>> ICC_PMR_EL1. This means the CPU might not come back from a WFI
>> instruction.
>>
>> Make sure ICC_PMR_EL1 does not mask interrupts when doing a WFI.
>>
>> Since the logic of cpu_do_idle is becoming a bit more complex than just
>> two instructions, lets turn it from ASM to C.
> 
> I haven't checked all the call paths, so asking here: does the core code
> normally call arch_cpu_idle() with IRQs off?
> 

If you look at do_idle() in kernel/sched/idle.c, it does:

	while (!need_resched()) {
		[...]
		local_irq_disable();
		arch_cpu_idle_enter();

		[...]
		} else {
			cpuidle_idle_call();
		}
	}


Then cpuidle_idle_call() will call default_idle_call() (which calls
arch_cpu_idle()) either directly or via the cpuidle driver. My
understanding is that it always does the call with interrupts disabled.

Thanks,

-- 
Julien Thierry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

On 18/01/2019 16:35, Dave Martin wrote:
> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>> Instead disabling interrupts by setting the PSR.I bit, use a priority
>> higher than the one used for interrupts to mask them via PMR.
>>
>> When using PMR to disable interrupts, the value of PMR will be used
>> instead of PSR.[DAIF] for the irqflags.
>>
>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Cc: Oleg Nesterov <oleg@redhat.com>
>> ---
>>  arch/arm64/include/asm/efi.h      |  11 ++++
>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>>  2 files changed, 106 insertions(+), 28 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
>> index 7ed3208..134ff6e 100644
>> --- a/arch/arm64/include/asm/efi.h
>> +++ b/arch/arm64/include/asm/efi.h
>> @@ -44,6 +44,17 @@
>>  
>>  #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
>>  
>> +#define arch_efi_save_flags(state_flags)		\
>> +	do {						\
>> +		(state_flags) =	read_sysreg(daif);	\
>> +	} while (0)
>> +
>> +#define arch_efi_restore_flags(state_flags)		\
>> +	do {						\
>> +		write_sysreg(state_flags, daif);	\
>> +	} while (0)
>> +
>> +
> 
> Randomly commenting a few minor nits as I glance down my mailbox...
> 
> There's no need to protect single statements with do { } while(0).
> 
> Just protect an expression statement that could be misparsed with ( ).
> 
> ->
> 
> #define arch_efi_save_flags(state_flags) ((state_flags) = read_sysreg(daif))

For the efi_save_flags(), I wanted to avoid it getting used as an
expression.

Would casting the assignment expression to (void) be acceptable?

> #define arch_efi_restore_flags(state_flags) write_sysreg(state_flags, daif)

For this one, write_sysreg() is already a statement, so yes, I
definitely don't need a do { } while (0) here.

> 
> [...]
> 
>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
>> index 24692ed..fa3b06f 100644
>> --- a/arch/arm64/include/asm/irqflags.h
>> +++ b/arch/arm64/include/asm/irqflags.h
>> @@ -18,7 +18,9 @@
>>  
>>  #ifdef __KERNEL__
>>  
>> +#include <asm/alternative.h>
>>  #include <asm/ptrace.h>
>> +#include <asm/sysreg.h>
>>  
>>  /*
>>   * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
>> @@ -36,47 +38,96 @@
> 
> [...]
> 
>>  /*
>> + * Having two ways to control interrupt status is a bit complicated. Some
>> + * locations like exception entries will have PSR.I bit set by the architecture
>> + * while PMR is unmasked.
>> + * We need the irqflags to represent that interrupts are disabled in such cases.
>> + *
>> + * For this, we lower the value read from PMR when the I bit is set so it is
>> + * considered as an irq masking priority. (With PMR, lower value means masking
>> + * more interrupts).
>> + */
>> +#define _get_irqflags(daif_bits, pmr)					\
>> +({									\
>> +	unsigned long flags;						\
>> +									\
>> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
>> +	asm volatile(ALTERNATIVE(					\
>> +		"mov	%0, %1\n"					\
>> +		"nop\n"							\
>> +		"nop",							\
>> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
>> +		"mvn	%0, %0\n"					\
>> +		"and	%0, %0, %2",					\
>> +		ARM64_HAS_IRQ_PRIO_MASKING)				\
>> +		: "=&r" (flags)						\
>> +		: "r" (daif_bits), "r" (pmr)				\
>> +		: "memory");						\
>> +									\
>> +	flags;								\
>> +})
> 
> Nit: does this need to be a macro?
> 
> ({ ... }) is mildly gross and it's preferable to avoid it if the code
> works just as well without...
> 
> pmr would need to be passed as a pointer, with "r" (*pmr) in the asm,
> but I think it would compile down to precisely the same code.
> 

The only motivation for it to be a macro was to be able to #undef it
after its use.

But with Catalin's suggestion, looks like we can makes things simple and
avoid having a separate macro/function.

>> +
>> +/*
>>   * Save the current interrupt enable state.
>>   */
>>  static inline unsigned long arch_local_save_flags(void)
>>  {
>> -	unsigned long flags;
>> -	asm volatile(
>> -		"mrs	%0, daif		// arch_local_save_flags"
>> -		: "=r" (flags)
>> +	unsigned long daif_bits;
>> +	unsigned long pmr; // Only used if alternative is on
>> +
>> +	daif_bits = read_sysreg(daif);
>> +
>> +	// Get PMR
>> +	asm volatile(ALTERNATIVE(
>> +			"nop",
>> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
>> +			ARM64_HAS_IRQ_PRIO_MASKING)
>> +		: "=&r" (pmr)
> 
> Why earlyclobber?
>>>  		:
>>  		: "memory");
> 
> [...]
> 
>> @@ -85,16 +136,32 @@ static inline unsigned long arch_local_save_flags(void)
> 
> [...]
> 
>>  static inline int arch_irqs_disabled_flags(unsigned long flags)
>>  {
>> -	return flags & PSR_I_BIT;
>> +	int res;
>> +
>> +	asm volatile(ALTERNATIVE(
>> +			"and	%w0, %w1, #" __stringify(PSR_I_BIT) "\n"
>> +			"nop",
>> +			"cmp	%w1, #" __stringify(GIC_PRIO_IRQOFF) "\n"
>> +			"cset	%w0, ls",
>> +			ARM64_HAS_IRQ_PRIO_MASKING)
>> +		: "=&r" (res)
> 
> Why earlyclobber?  %0 is not written before the reading of any input
> argument so far as I can see, in either alternative.
> 

I didn't really understand what the earlyclobber semantic was, thanks
for explaining it.

Thanks,

-- 
Julien Thierry

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

On 18/01/2019 16:35, Dave Martin wrote:
> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>> Instead disabling interrupts by setting the PSR.I bit, use a priority
>> higher than the one used for interrupts to mask them via PMR.
>>
>> When using PMR to disable interrupts, the value of PMR will be used
>> instead of PSR.[DAIF] for the irqflags.
>>
>> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
>> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Cc: Oleg Nesterov <oleg@redhat.com>
>> ---
>>  arch/arm64/include/asm/efi.h      |  11 ++++
>>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
>>  2 files changed, 106 insertions(+), 28 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
>> index 7ed3208..134ff6e 100644
>> --- a/arch/arm64/include/asm/efi.h
>> +++ b/arch/arm64/include/asm/efi.h
>> @@ -44,6 +44,17 @@
>>  
>>  #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
>>  
>> +#define arch_efi_save_flags(state_flags)		\
>> +	do {						\
>> +		(state_flags) =	read_sysreg(daif);	\
>> +	} while (0)
>> +
>> +#define arch_efi_restore_flags(state_flags)		\
>> +	do {						\
>> +		write_sysreg(state_flags, daif);	\
>> +	} while (0)
>> +
>> +
> 
> Randomly commenting a few minor nits as I glance down my mailbox...
> 
> There's no need to protect single statements with do { } while(0).
> 
> Just protect an expression statement that could be misparsed with ( ).
> 
> ->
> 
> #define arch_efi_save_flags(state_flags) ((state_flags) = read_sysreg(daif))

For the efi_save_flags(), I wanted to avoid it getting used as an
expression.

Would casting the assignment expression to (void) be acceptable?

> #define arch_efi_restore_flags(state_flags) write_sysreg(state_flags, daif)

For this one, write_sysreg() is already a statement, so yes, I
definitely don't need a do { } while (0) here.

> 
> [...]
> 
>> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
>> index 24692ed..fa3b06f 100644
>> --- a/arch/arm64/include/asm/irqflags.h
>> +++ b/arch/arm64/include/asm/irqflags.h
>> @@ -18,7 +18,9 @@
>>  
>>  #ifdef __KERNEL__
>>  
>> +#include <asm/alternative.h>
>>  #include <asm/ptrace.h>
>> +#include <asm/sysreg.h>
>>  
>>  /*
>>   * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
>> @@ -36,47 +38,96 @@
> 
> [...]
> 
>>  /*
>> + * Having two ways to control interrupt status is a bit complicated. Some
>> + * locations like exception entries will have PSR.I bit set by the architecture
>> + * while PMR is unmasked.
>> + * We need the irqflags to represent that interrupts are disabled in such cases.
>> + *
>> + * For this, we lower the value read from PMR when the I bit is set so it is
>> + * considered as an irq masking priority. (With PMR, lower value means masking
>> + * more interrupts).
>> + */
>> +#define _get_irqflags(daif_bits, pmr)					\
>> +({									\
>> +	unsigned long flags;						\
>> +									\
>> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
>> +	asm volatile(ALTERNATIVE(					\
>> +		"mov	%0, %1\n"					\
>> +		"nop\n"							\
>> +		"nop",							\
>> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
>> +		"mvn	%0, %0\n"					\
>> +		"and	%0, %0, %2",					\
>> +		ARM64_HAS_IRQ_PRIO_MASKING)				\
>> +		: "=&r" (flags)						\
>> +		: "r" (daif_bits), "r" (pmr)				\
>> +		: "memory");						\
>> +									\
>> +	flags;								\
>> +})
> 
> Nit: does this need to be a macro?
> 
> ({ ... }) is mildly gross and it's preferable to avoid it if the code
> works just as well without...
> 
> pmr would need to be passed as a pointer, with "r" (*pmr) in the asm,
> but I think it would compile down to precisely the same code.
> 

The only motivation for it to be a macro was to be able to #undef it
after its use.

But with Catalin's suggestion, looks like we can makes things simple and
avoid having a separate macro/function.

>> +
>> +/*
>>   * Save the current interrupt enable state.
>>   */
>>  static inline unsigned long arch_local_save_flags(void)
>>  {
>> -	unsigned long flags;
>> -	asm volatile(
>> -		"mrs	%0, daif		// arch_local_save_flags"
>> -		: "=r" (flags)
>> +	unsigned long daif_bits;
>> +	unsigned long pmr; // Only used if alternative is on
>> +
>> +	daif_bits = read_sysreg(daif);
>> +
>> +	// Get PMR
>> +	asm volatile(ALTERNATIVE(
>> +			"nop",
>> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
>> +			ARM64_HAS_IRQ_PRIO_MASKING)
>> +		: "=&r" (pmr)
> 
> Why earlyclobber?
>>>  		:
>>  		: "memory");
> 
> [...]
> 
>> @@ -85,16 +136,32 @@ static inline unsigned long arch_local_save_flags(void)
> 
> [...]
> 
>>  static inline int arch_irqs_disabled_flags(unsigned long flags)
>>  {
>> -	return flags & PSR_I_BIT;
>> +	int res;
>> +
>> +	asm volatile(ALTERNATIVE(
>> +			"and	%w0, %w1, #" __stringify(PSR_I_BIT) "\n"
>> +			"nop",
>> +			"cmp	%w1, #" __stringify(GIC_PRIO_IRQOFF) "\n"
>> +			"cset	%w0, ls",
>> +			ARM64_HAS_IRQ_PRIO_MASKING)
>> +		: "=&r" (res)
> 
> Why earlyclobber?  %0 is not written before the reading of any input
> argument so far as I can see, in either alternative.
> 

I didn't really understand what the earlyclobber semantic was, thanks
for explaining it.

Thanks,

-- 
Julien Thierry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Catalin Marinas]

On Fri, Jan 18, 2019 at 04:57:32PM +0000, Julien Thierry wrote:
> On 18/01/2019 16:09, Catalin Marinas wrote:
> > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >> +	asm volatile(ALTERNATIVE(
> >> +			"nop",
> >> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> >> +			ARM64_HAS_IRQ_PRIO_MASKING)
> >> +		: "=&r" (pmr)
> >>  		:
> >>  		: "memory");
> >> +
> >> +	return _get_irqflags(daif_bits, pmr);
> >> +}
> > 
> > I find this confusing spread over two inline asm statements. IIUC, you
> > want something like below (it could be written as inline asm but I need
> > to understand it first):
> > 
> > 	daif_bits = read_sysreg(daif);
> > 
> > 	if (system_uses_irq_prio_masking()) {
> > 		pmr = read_gicreg(ICC_PMR_EL1);
> > 		flags = pmr & ~(daif_bits & PSR_I_BIT);
> > 	} else {
> > 		flags = daif_bits;
> > 	}
> > 
> > 	return flags;
> > 
> > In the case where the interrupts are disabled at the PSR level, is the
> > PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
> > Something like:
> > 
> > 	flags = read_sysreg(daif);
> > 
> > 	if (system_uses_irq_prio_masking())
> > 		flags = flags & PSR_I_BIT ?
> > 			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);
> > 
> 
> You're right, returning GIC_PRIO_IRQOFF should be good enough (it is
> actually what happens in this version because GIC_PRIO_IRQOFF ==
> GIC_PRIO_IRQON & ~PSR_I_BIT happens to be true).

This wasn't entirely clear to me, I got confused by:

+       BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));  \

and I thought there isn't necessarily an equality between the two.

> Your suggestion would
> make things easier to reason about. Maybe something like:
> 
> 
> static inline unsigned long arch_local_save_flags(void)
> {
> 	unsigned long daif_bits;
> 	unsigned long prio_off = GIC_PRIO_IRQOFF;
> 
> 	daif_bits = read_sysreg(daif);
> 
> 	asm volatile(ALTERNATIVE(
> 		"mov	%0, %1\n"
> 		"nop\n"
> 		"nop",
> 		"mrs	%0, SYS_ICC_PMR_EL1\n"
> 		"ands	%1, %1, PSR_I_BIT\n"
> 		"csel	%0, %0, %2, eq")
> 	: "=&r" (flags)
> 	: "r" (daif_bits), "r" (prio_off)
> 	: "memory");
> 
> 	return flags;
> }

It looks fine. If you turn the BUILD_BUG_ON into a !=, you could
probably simplify the asm a bit (though the number of instructions
generated would probably be the same). Untested:

static inline unsigned long arch_local_save_flags(void)
{
	unsigned long flags;

	flags = read_sysreg(daif);

	asm volatile(ALTERNATIVE(
		"nop",
		"bic	%0, %1, %2")
	: "=&r" (flags)
	: "r" (flags & PSR_I_BIT), "r" (GIC_PRIO_IRQOFF)
	: "memory");

	return flags;
}

-- 
Catalin

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Catalin Marinas]

On Fri, Jan 18, 2019 at 04:57:32PM +0000, Julien Thierry wrote:
> On 18/01/2019 16:09, Catalin Marinas wrote:
> > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >> +	asm volatile(ALTERNATIVE(
> >> +			"nop",
> >> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> >> +			ARM64_HAS_IRQ_PRIO_MASKING)
> >> +		: "=&r" (pmr)
> >>  		:
> >>  		: "memory");
> >> +
> >> +	return _get_irqflags(daif_bits, pmr);
> >> +}
> > 
> > I find this confusing spread over two inline asm statements. IIUC, you
> > want something like below (it could be written as inline asm but I need
> > to understand it first):
> > 
> > 	daif_bits = read_sysreg(daif);
> > 
> > 	if (system_uses_irq_prio_masking()) {
> > 		pmr = read_gicreg(ICC_PMR_EL1);
> > 		flags = pmr & ~(daif_bits & PSR_I_BIT);
> > 	} else {
> > 		flags = daif_bits;
> > 	}
> > 
> > 	return flags;
> > 
> > In the case where the interrupts are disabled at the PSR level, is the
> > PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
> > Something like:
> > 
> > 	flags = read_sysreg(daif);
> > 
> > 	if (system_uses_irq_prio_masking())
> > 		flags = flags & PSR_I_BIT ?
> > 			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);
> > 
> 
> You're right, returning GIC_PRIO_IRQOFF should be good enough (it is
> actually what happens in this version because GIC_PRIO_IRQOFF ==
> GIC_PRIO_IRQON & ~PSR_I_BIT happens to be true).

This wasn't entirely clear to me, I got confused by:

+       BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));  \

and I thought there isn't necessarily an equality between the two.

> Your suggestion would
> make things easier to reason about. Maybe something like:
> 
> 
> static inline unsigned long arch_local_save_flags(void)
> {
> 	unsigned long daif_bits;
> 	unsigned long prio_off = GIC_PRIO_IRQOFF;
> 
> 	daif_bits = read_sysreg(daif);
> 
> 	asm volatile(ALTERNATIVE(
> 		"mov	%0, %1\n"
> 		"nop\n"
> 		"nop",
> 		"mrs	%0, SYS_ICC_PMR_EL1\n"
> 		"ands	%1, %1, PSR_I_BIT\n"
> 		"csel	%0, %0, %2, eq")
> 	: "=&r" (flags)
> 	: "r" (daif_bits), "r" (prio_off)
> 	: "memory");
> 
> 	return flags;
> }

It looks fine. If you turn the BUILD_BUG_ON into a !=, you could
probably simplify the asm a bit (though the number of instructions
generated would probably be the same). Untested:

static inline unsigned long arch_local_save_flags(void)
{
	unsigned long flags;

	flags = read_sysreg(daif);

	asm volatile(ALTERNATIVE(
		"nop",
		"bic	%0, %1, %2")
	: "=&r" (flags)
	: "r" (flags & PSR_I_BIT), "r" (GIC_PRIO_IRQOFF)
	: "memory");

	return flags;
}

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Catalin Marinas]

On Fri, Jan 18, 2019 at 05:30:02PM +0000, Catalin Marinas wrote:
> On Fri, Jan 18, 2019 at 04:57:32PM +0000, Julien Thierry wrote:
> > On 18/01/2019 16:09, Catalin Marinas wrote:
> > > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> > >> +	asm volatile(ALTERNATIVE(
> > >> +			"nop",
> > >> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> > >> +			ARM64_HAS_IRQ_PRIO_MASKING)
> > >> +		: "=&r" (pmr)
> > >>  		:
> > >>  		: "memory");
> > >> +
> > >> +	return _get_irqflags(daif_bits, pmr);
> > >> +}
> > > 
> > > I find this confusing spread over two inline asm statements. IIUC, you
> > > want something like below (it could be written as inline asm but I need
> > > to understand it first):
> > > 
> > > 	daif_bits = read_sysreg(daif);
> > > 
> > > 	if (system_uses_irq_prio_masking()) {
> > > 		pmr = read_gicreg(ICC_PMR_EL1);
> > > 		flags = pmr & ~(daif_bits & PSR_I_BIT);
> > > 	} else {
> > > 		flags = daif_bits;
> > > 	}
> > > 
> > > 	return flags;
> > > 
> > > In the case where the interrupts are disabled at the PSR level, is the
> > > PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
> > > Something like:
> > > 
> > > 	flags = read_sysreg(daif);
> > > 
> > > 	if (system_uses_irq_prio_masking())
> > > 		flags = flags & PSR_I_BIT ?
> > > 			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);
> > > 
> > 
> > You're right, returning GIC_PRIO_IRQOFF should be good enough (it is
> > actually what happens in this version because GIC_PRIO_IRQOFF ==
> > GIC_PRIO_IRQON & ~PSR_I_BIT happens to be true).
> 
> This wasn't entirely clear to me, I got confused by:
> 
> +       BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));  \
> 
> and I thought there isn't necessarily an equality between the two.
> 
> > Your suggestion would
> > make things easier to reason about. Maybe something like:
> > 
> > 
> > static inline unsigned long arch_local_save_flags(void)
> > {
> > 	unsigned long daif_bits;
> > 	unsigned long prio_off = GIC_PRIO_IRQOFF;
> > 
> > 	daif_bits = read_sysreg(daif);
> > 
> > 	asm volatile(ALTERNATIVE(
> > 		"mov	%0, %1\n"
> > 		"nop\n"
> > 		"nop",
> > 		"mrs	%0, SYS_ICC_PMR_EL1\n"
> > 		"ands	%1, %1, PSR_I_BIT\n"
> > 		"csel	%0, %0, %2, eq")
> > 	: "=&r" (flags)
> > 	: "r" (daif_bits), "r" (prio_off)
> > 	: "memory");
> > 
> > 	return flags;
> > }
> 
> It looks fine. If you turn the BUILD_BUG_ON into a !=, you could
> probably simplify the asm a bit (though the number of instructions
> generated would probably be the same). Untested:
> 
> static inline unsigned long arch_local_save_flags(void)
> {
> 	unsigned long flags;
> 
> 	flags = read_sysreg(daif);
> 
> 	asm volatile(ALTERNATIVE(
> 		"nop",
> 		"bic	%0, %1, %2")
> 	: "=&r" (flags)
> 	: "r" (flags & PSR_I_BIT), "r" (GIC_PRIO_IRQOFF)
> 	: "memory");

Ah, I missed a read from SYS_ICC_PMR_EL1 here. Anyway, the idea was that
you don't need to set prio_off to a variable, just pass "r" (constant)
here and the compiler does the trick.

-- 
Catalin

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Catalin Marinas]

On Fri, Jan 18, 2019 at 05:30:02PM +0000, Catalin Marinas wrote:
> On Fri, Jan 18, 2019 at 04:57:32PM +0000, Julien Thierry wrote:
> > On 18/01/2019 16:09, Catalin Marinas wrote:
> > > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> > >> +	asm volatile(ALTERNATIVE(
> > >> +			"nop",
> > >> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> > >> +			ARM64_HAS_IRQ_PRIO_MASKING)
> > >> +		: "=&r" (pmr)
> > >>  		:
> > >>  		: "memory");
> > >> +
> > >> +	return _get_irqflags(daif_bits, pmr);
> > >> +}
> > > 
> > > I find this confusing spread over two inline asm statements. IIUC, you
> > > want something like below (it could be written as inline asm but I need
> > > to understand it first):
> > > 
> > > 	daif_bits = read_sysreg(daif);
> > > 
> > > 	if (system_uses_irq_prio_masking()) {
> > > 		pmr = read_gicreg(ICC_PMR_EL1);
> > > 		flags = pmr & ~(daif_bits & PSR_I_BIT);
> > > 	} else {
> > > 		flags = daif_bits;
> > > 	}
> > > 
> > > 	return flags;
> > > 
> > > In the case where the interrupts are disabled at the PSR level, is the
> > > PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
> > > Something like:
> > > 
> > > 	flags = read_sysreg(daif);
> > > 
> > > 	if (system_uses_irq_prio_masking())
> > > 		flags = flags & PSR_I_BIT ?
> > > 			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);
> > > 
> > 
> > You're right, returning GIC_PRIO_IRQOFF should be good enough (it is
> > actually what happens in this version because GIC_PRIO_IRQOFF ==
> > GIC_PRIO_IRQON & ~PSR_I_BIT happens to be true).
> 
> This wasn't entirely clear to me, I got confused by:
> 
> +       BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));  \
> 
> and I thought there isn't necessarily an equality between the two.
> 
> > Your suggestion would
> > make things easier to reason about. Maybe something like:
> > 
> > 
> > static inline unsigned long arch_local_save_flags(void)
> > {
> > 	unsigned long daif_bits;
> > 	unsigned long prio_off = GIC_PRIO_IRQOFF;
> > 
> > 	daif_bits = read_sysreg(daif);
> > 
> > 	asm volatile(ALTERNATIVE(
> > 		"mov	%0, %1\n"
> > 		"nop\n"
> > 		"nop",
> > 		"mrs	%0, SYS_ICC_PMR_EL1\n"
> > 		"ands	%1, %1, PSR_I_BIT\n"
> > 		"csel	%0, %0, %2, eq")
> > 	: "=&r" (flags)
> > 	: "r" (daif_bits), "r" (prio_off)
> > 	: "memory");
> > 
> > 	return flags;
> > }
> 
> It looks fine. If you turn the BUILD_BUG_ON into a !=, you could
> probably simplify the asm a bit (though the number of instructions
> generated would probably be the same). Untested:
> 
> static inline unsigned long arch_local_save_flags(void)
> {
> 	unsigned long flags;
> 
> 	flags = read_sysreg(daif);
> 
> 	asm volatile(ALTERNATIVE(
> 		"nop",
> 		"bic	%0, %1, %2")
> 	: "=&r" (flags)
> 	: "r" (flags & PSR_I_BIT), "r" (GIC_PRIO_IRQOFF)
> 	: "memory");

Ah, I missed a read from SYS_ICC_PMR_EL1 here. Anyway, the idea was that
you don't need to set prio_off to a variable, just pass "r" (constant)
here and the compiler does the trick.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Fri, Jan 18, 2019 at 05:27:29PM +0000, Julien Thierry wrote:
> 
> 
> On 18/01/2019 16:35, Dave Martin wrote:
> > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >> Instead disabling interrupts by setting the PSR.I bit, use a priority
> >> higher than the one used for interrupts to mask them via PMR.
> >>
> >> When using PMR to disable interrupts, the value of PMR will be used
> >> instead of PSR.[DAIF] for the irqflags.
> >>
> >> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> >> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> >> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >> Cc: Will Deacon <will.deacon@arm.com>
> >> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >> Cc: Oleg Nesterov <oleg@redhat.com>
> >> ---
> >>  arch/arm64/include/asm/efi.h      |  11 ++++
> >>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
> >>  2 files changed, 106 insertions(+), 28 deletions(-)
> >>
> >> diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
> >> index 7ed3208..134ff6e 100644
> >> --- a/arch/arm64/include/asm/efi.h
> >> +++ b/arch/arm64/include/asm/efi.h
> >> @@ -44,6 +44,17 @@
> >>  
> >>  #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
> >>  
> >> +#define arch_efi_save_flags(state_flags)		\
> >> +	do {						\
> >> +		(state_flags) =	read_sysreg(daif);	\
> >> +	} while (0)
> >> +
> >> +#define arch_efi_restore_flags(state_flags)		\
> >> +	do {						\
> >> +		write_sysreg(state_flags, daif);	\
> >> +	} while (0)
> >> +
> >> +
> > 
> > Randomly commenting a few minor nits as I glance down my mailbox...
> > 
> > There's no need to protect single statements with do { } while(0).
> > 
> > Just protect an expression statement that could be misparsed with ( ).
> > 
> > ->
> > 
> > #define arch_efi_save_flags(state_flags) ((state_flags) = read_sysreg(daif))
> 
> For the efi_save_flags(), I wanted to avoid it getting used as an
> expression.
> 
> Would casting the assignment expression to (void) be acceptable?

Yep, that's a common way of achieving that, so

	((void)((state_flags) = read_sysreg(daif)))

should be OK.

> > #define arch_efi_restore_flags(state_flags) write_sysreg(state_flags, daif)
> 
> For this one, write_sysreg() is already a statement, so yes, I
> definitely don't need a do { } while (0) here.
> 
> > 
> > [...]
> > 
> >> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> >> index 24692ed..fa3b06f 100644
> >> --- a/arch/arm64/include/asm/irqflags.h
> >> +++ b/arch/arm64/include/asm/irqflags.h
> >> @@ -18,7 +18,9 @@
> >>  
> >>  #ifdef __KERNEL__
> >>  
> >> +#include <asm/alternative.h>
> >>  #include <asm/ptrace.h>
> >> +#include <asm/sysreg.h>
> >>  
> >>  /*
> >>   * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
> >> @@ -36,47 +38,96 @@
> > 
> > [...]
> > 
> >>  /*
> >> + * Having two ways to control interrupt status is a bit complicated. Some
> >> + * locations like exception entries will have PSR.I bit set by the architecture
> >> + * while PMR is unmasked.
> >> + * We need the irqflags to represent that interrupts are disabled in such cases.
> >> + *
> >> + * For this, we lower the value read from PMR when the I bit is set so it is
> >> + * considered as an irq masking priority. (With PMR, lower value means masking
> >> + * more interrupts).
> >> + */
> >> +#define _get_irqflags(daif_bits, pmr)					\
> >> +({									\
> >> +	unsigned long flags;						\
> >> +									\
> >> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
> >> +	asm volatile(ALTERNATIVE(					\
> >> +		"mov	%0, %1\n"					\
> >> +		"nop\n"							\
> >> +		"nop",							\
> >> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
> >> +		"mvn	%0, %0\n"					\
> >> +		"and	%0, %0, %2",					\
> >> +		ARM64_HAS_IRQ_PRIO_MASKING)				\
> >> +		: "=&r" (flags)						\
> >> +		: "r" (daif_bits), "r" (pmr)				\
> >> +		: "memory");						\
> >> +									\
> >> +	flags;								\
> >> +})
> > 
> > Nit: does this need to be a macro?
> > 
> > ({ ... }) is mildly gross and it's preferable to avoid it if the code
> > works just as well without...
> > 
> > pmr would need to be passed as a pointer, with "r" (*pmr) in the asm,
> > but I think it would compile down to precisely the same code.
> > 
> 
> The only motivation for it to be a macro was to be able to #undef it
> after its use.
> 
> But with Catalin's suggestion, looks like we can makes things simple and
> avoid having a separate macro/function.

Fair enough.  This wasn't a big deal in any case.

> 
> >> +
> >> +/*
> >>   * Save the current interrupt enable state.
> >>   */
> >>  static inline unsigned long arch_local_save_flags(void)
> >>  {
> >> -	unsigned long flags;
> >> -	asm volatile(
> >> -		"mrs	%0, daif		// arch_local_save_flags"
> >> -		: "=r" (flags)
> >> +	unsigned long daif_bits;
> >> +	unsigned long pmr; // Only used if alternative is on
> >> +
> >> +	daif_bits = read_sysreg(daif);
> >> +
> >> +	// Get PMR
> >> +	asm volatile(ALTERNATIVE(
> >> +			"nop",
> >> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> >> +			ARM64_HAS_IRQ_PRIO_MASKING)
> >> +		: "=&r" (pmr)
> > 
> > Why earlyclobber?
> >>>  		:
> >>  		: "memory");
> > 
> > [...]
> > 
> >> @@ -85,16 +136,32 @@ static inline unsigned long arch_local_save_flags(void)
> > 
> > [...]
> > 
> >>  static inline int arch_irqs_disabled_flags(unsigned long flags)
> >>  {
> >> -	return flags & PSR_I_BIT;
> >> +	int res;
> >> +
> >> +	asm volatile(ALTERNATIVE(
> >> +			"and	%w0, %w1, #" __stringify(PSR_I_BIT) "\n"
> >> +			"nop",
> >> +			"cmp	%w1, #" __stringify(GIC_PRIO_IRQOFF) "\n"
> >> +			"cset	%w0, ls",
> >> +			ARM64_HAS_IRQ_PRIO_MASKING)
> >> +		: "=&r" (res)
> > 
> > Why earlyclobber?  %0 is not written before the reading of any input
> > argument so far as I can see, in either alternative.
> > 
> 
> I didn't really understand what the earlyclobber semantic was, thanks
> for explaining it.

It basically means "you can't put any input argument in the same
register, because it can get overwritten by some output argument before
use".

When unsure, it's safer to include "&", but it may cause the compiler to
allocate registers suboptimally.

In the above,

	"mov %w0, #" __stringify(PSR_I_BIT) "\n"
	"and %w0, %w0, %w1"

would require the earlyclobber, so that the compiler can't allocate the
same register for %0 and %1.

But in your code, allocating the same register looks safe.  (Don't take
my word for it, though!)

Cheers
---Dave

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Dave Martin]

On Fri, Jan 18, 2019 at 05:27:29PM +0000, Julien Thierry wrote:
> 
> 
> On 18/01/2019 16:35, Dave Martin wrote:
> > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
> >> Instead disabling interrupts by setting the PSR.I bit, use a priority
> >> higher than the one used for interrupts to mask them via PMR.
> >>
> >> When using PMR to disable interrupts, the value of PMR will be used
> >> instead of PSR.[DAIF] for the irqflags.
> >>
> >> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> >> Suggested-by: Daniel Thompson <daniel.thompson@linaro.org>
> >> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >> Cc: Will Deacon <will.deacon@arm.com>
> >> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >> Cc: Oleg Nesterov <oleg@redhat.com>
> >> ---
> >>  arch/arm64/include/asm/efi.h      |  11 ++++
> >>  arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++---------
> >>  2 files changed, 106 insertions(+), 28 deletions(-)
> >>
> >> diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h
> >> index 7ed3208..134ff6e 100644
> >> --- a/arch/arm64/include/asm/efi.h
> >> +++ b/arch/arm64/include/asm/efi.h
> >> @@ -44,6 +44,17 @@
> >>  
> >>  #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT)
> >>  
> >> +#define arch_efi_save_flags(state_flags)		\
> >> +	do {						\
> >> +		(state_flags) =	read_sysreg(daif);	\
> >> +	} while (0)
> >> +
> >> +#define arch_efi_restore_flags(state_flags)		\
> >> +	do {						\
> >> +		write_sysreg(state_flags, daif);	\
> >> +	} while (0)
> >> +
> >> +
> > 
> > Randomly commenting a few minor nits as I glance down my mailbox...
> > 
> > There's no need to protect single statements with do { } while(0).
> > 
> > Just protect an expression statement that could be misparsed with ( ).
> > 
> > ->
> > 
> > #define arch_efi_save_flags(state_flags) ((state_flags) = read_sysreg(daif))
> 
> For the efi_save_flags(), I wanted to avoid it getting used as an
> expression.
> 
> Would casting the assignment expression to (void) be acceptable?

Yep, that's a common way of achieving that, so

	((void)((state_flags) = read_sysreg(daif)))

should be OK.

> > #define arch_efi_restore_flags(state_flags) write_sysreg(state_flags, daif)
> 
> For this one, write_sysreg() is already a statement, so yes, I
> definitely don't need a do { } while (0) here.
> 
> > 
> > [...]
> > 
> >> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
> >> index 24692ed..fa3b06f 100644
> >> --- a/arch/arm64/include/asm/irqflags.h
> >> +++ b/arch/arm64/include/asm/irqflags.h
> >> @@ -18,7 +18,9 @@
> >>  
> >>  #ifdef __KERNEL__
> >>  
> >> +#include <asm/alternative.h>
> >>  #include <asm/ptrace.h>
> >> +#include <asm/sysreg.h>
> >>  
> >>  /*
> >>   * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and
> >> @@ -36,47 +38,96 @@
> > 
> > [...]
> > 
> >>  /*
> >> + * Having two ways to control interrupt status is a bit complicated. Some
> >> + * locations like exception entries will have PSR.I bit set by the architecture
> >> + * while PMR is unmasked.
> >> + * We need the irqflags to represent that interrupts are disabled in such cases.
> >> + *
> >> + * For this, we lower the value read from PMR when the I bit is set so it is
> >> + * considered as an irq masking priority. (With PMR, lower value means masking
> >> + * more interrupts).
> >> + */
> >> +#define _get_irqflags(daif_bits, pmr)					\
> >> +({									\
> >> +	unsigned long flags;						\
> >> +									\
> >> +	BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));	\
> >> +	asm volatile(ALTERNATIVE(					\
> >> +		"mov	%0, %1\n"					\
> >> +		"nop\n"							\
> >> +		"nop",							\
> >> +		"and	%0, %1, #" __stringify(PSR_I_BIT) "\n"		\
> >> +		"mvn	%0, %0\n"					\
> >> +		"and	%0, %0, %2",					\
> >> +		ARM64_HAS_IRQ_PRIO_MASKING)				\
> >> +		: "=&r" (flags)						\
> >> +		: "r" (daif_bits), "r" (pmr)				\
> >> +		: "memory");						\
> >> +									\
> >> +	flags;								\
> >> +})
> > 
> > Nit: does this need to be a macro?
> > 
> > ({ ... }) is mildly gross and it's preferable to avoid it if the code
> > works just as well without...
> > 
> > pmr would need to be passed as a pointer, with "r" (*pmr) in the asm,
> > but I think it would compile down to precisely the same code.
> > 
> 
> The only motivation for it to be a macro was to be able to #undef it
> after its use.
> 
> But with Catalin's suggestion, looks like we can makes things simple and
> avoid having a separate macro/function.

Fair enough.  This wasn't a big deal in any case.

> 
> >> +
> >> +/*
> >>   * Save the current interrupt enable state.
> >>   */
> >>  static inline unsigned long arch_local_save_flags(void)
> >>  {
> >> -	unsigned long flags;
> >> -	asm volatile(
> >> -		"mrs	%0, daif		// arch_local_save_flags"
> >> -		: "=r" (flags)
> >> +	unsigned long daif_bits;
> >> +	unsigned long pmr; // Only used if alternative is on
> >> +
> >> +	daif_bits = read_sysreg(daif);
> >> +
> >> +	// Get PMR
> >> +	asm volatile(ALTERNATIVE(
> >> +			"nop",
> >> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
> >> +			ARM64_HAS_IRQ_PRIO_MASKING)
> >> +		: "=&r" (pmr)
> > 
> > Why earlyclobber?
> >>>  		:
> >>  		: "memory");
> > 
> > [...]
> > 
> >> @@ -85,16 +136,32 @@ static inline unsigned long arch_local_save_flags(void)
> > 
> > [...]
> > 
> >>  static inline int arch_irqs_disabled_flags(unsigned long flags)
> >>  {
> >> -	return flags & PSR_I_BIT;
> >> +	int res;
> >> +
> >> +	asm volatile(ALTERNATIVE(
> >> +			"and	%w0, %w1, #" __stringify(PSR_I_BIT) "\n"
> >> +			"nop",
> >> +			"cmp	%w1, #" __stringify(GIC_PRIO_IRQOFF) "\n"
> >> +			"cset	%w0, ls",
> >> +			ARM64_HAS_IRQ_PRIO_MASKING)
> >> +		: "=&r" (res)
> > 
> > Why earlyclobber?  %0 is not written before the reading of any input
> > argument so far as I can see, in either alternative.
> > 
> 
> I didn't really understand what the earlyclobber semantic was, thanks
> for explaining it.

It basically means "you can't put any input argument in the same
register, because it can get overwritten by some output argument before
use".

When unsure, it's safer to include "&", but it may cause the compiler to
allocate registers suboptimally.

In the above,

	"mov %w0, #" __stringify(PSR_I_BIT) "\n"
	"and %w0, %w0, %w1"

would require the earlyclobber, so that the compiler can't allocate the
same register for %0 and %1.

But in your code, allocating the same register looks safe.  (Don't take
my word for it, though!)

Cheers
---Dave

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

On 18/01/2019 17:33, Catalin Marinas wrote:
> On Fri, Jan 18, 2019 at 05:30:02PM +0000, Catalin Marinas wrote:
>> On Fri, Jan 18, 2019 at 04:57:32PM +0000, Julien Thierry wrote:
>>> On 18/01/2019 16:09, Catalin Marinas wrote:
>>>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>>>>> +	asm volatile(ALTERNATIVE(
>>>>> +			"nop",
>>>>> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
>>>>> +			ARM64_HAS_IRQ_PRIO_MASKING)
>>>>> +		: "=&r" (pmr)
>>>>>  		:
>>>>>  		: "memory");
>>>>> +
>>>>> +	return _get_irqflags(daif_bits, pmr);
>>>>> +}
>>>>
>>>> I find this confusing spread over two inline asm statements. IIUC, you
>>>> want something like below (it could be written as inline asm but I need
>>>> to understand it first):
>>>>
>>>> 	daif_bits = read_sysreg(daif);
>>>>
>>>> 	if (system_uses_irq_prio_masking()) {
>>>> 		pmr = read_gicreg(ICC_PMR_EL1);
>>>> 		flags = pmr & ~(daif_bits & PSR_I_BIT);
>>>> 	} else {
>>>> 		flags = daif_bits;
>>>> 	}
>>>>
>>>> 	return flags;
>>>>
>>>> In the case where the interrupts are disabled at the PSR level, is the
>>>> PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
>>>> Something like:
>>>>
>>>> 	flags = read_sysreg(daif);
>>>>
>>>> 	if (system_uses_irq_prio_masking())
>>>> 		flags = flags & PSR_I_BIT ?
>>>> 			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);
>>>>
>>>
>>> You're right, returning GIC_PRIO_IRQOFF should be good enough (it is
>>> actually what happens in this version because GIC_PRIO_IRQOFF ==
>>> GIC_PRIO_IRQON & ~PSR_I_BIT happens to be true).
>>
>> This wasn't entirely clear to me, I got confused by:
>>
>> +       BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));  \
>>
>> and I thought there isn't necessarily an equality between the two.
>>
>>> Your suggestion would
>>> make things easier to reason about. Maybe something like:
>>>
>>>
>>> static inline unsigned long arch_local_save_flags(void)
>>> {
>>> 	unsigned long daif_bits;
>>> 	unsigned long prio_off = GIC_PRIO_IRQOFF;
>>>
>>> 	daif_bits = read_sysreg(daif);
>>>
>>> 	asm volatile(ALTERNATIVE(
>>> 		"mov	%0, %1\n"
>>> 		"nop\n"
>>> 		"nop",
>>> 		"mrs	%0, SYS_ICC_PMR_EL1\n"
>>> 		"ands	%1, %1, PSR_I_BIT\n"
>>> 		"csel	%0, %0, %2, eq")
>>> 	: "=&r" (flags)
>>> 	: "r" (daif_bits), "r" (prio_off)
>>> 	: "memory");
>>>
>>> 	return flags;
>>> }
>>
>> It looks fine. If you turn the BUILD_BUG_ON into a !=, you could
>> probably simplify the asm a bit (though the number of instructions
>> generated would probably be the same). Untested:
>>
>> static inline unsigned long arch_local_save_flags(void)
>> {
>> 	unsigned long flags;
>>
>> 	flags = read_sysreg(daif);
>>
>> 	asm volatile(ALTERNATIVE(
>> 		"nop",
>> 		"bic	%0, %1, %2")
>> 	: "=&r" (flags)
>> 	: "r" (flags & PSR_I_BIT), "r" (GIC_PRIO_IRQOFF)
>> 	: "memory");
> 
> Ah, I missed a read from SYS_ICC_PMR_EL1 here. Anyway, the idea was that
> you don't need to set prio_off to a variable, just pass "r" (constant)
> here and the compiler does the trick.
> 

I see, thanks. I'll avoid that superfluous variable.

Thanks,

-- 
Julien Thierry

Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking

[Julien Thierry]

On 18/01/2019 17:33, Catalin Marinas wrote:
> On Fri, Jan 18, 2019 at 05:30:02PM +0000, Catalin Marinas wrote:
>> On Fri, Jan 18, 2019 at 04:57:32PM +0000, Julien Thierry wrote:
>>> On 18/01/2019 16:09, Catalin Marinas wrote:
>>>> On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote:
>>>>> +	asm volatile(ALTERNATIVE(
>>>>> +			"nop",
>>>>> +			"mrs_s	%0, " __stringify(SYS_ICC_PMR_EL1),
>>>>> +			ARM64_HAS_IRQ_PRIO_MASKING)
>>>>> +		: "=&r" (pmr)
>>>>>  		:
>>>>>  		: "memory");
>>>>> +
>>>>> +	return _get_irqflags(daif_bits, pmr);
>>>>> +}
>>>>
>>>> I find this confusing spread over two inline asm statements. IIUC, you
>>>> want something like below (it could be written as inline asm but I need
>>>> to understand it first):
>>>>
>>>> 	daif_bits = read_sysreg(daif);
>>>>
>>>> 	if (system_uses_irq_prio_masking()) {
>>>> 		pmr = read_gicreg(ICC_PMR_EL1);
>>>> 		flags = pmr & ~(daif_bits & PSR_I_BIT);
>>>> 	} else {
>>>> 		flags = daif_bits;
>>>> 	}
>>>>
>>>> 	return flags;
>>>>
>>>> In the case where the interrupts are disabled at the PSR level, is the
>>>> PMR value still relevant? Could we just return the GIC_PRIO_IRQOFF?
>>>> Something like:
>>>>
>>>> 	flags = read_sysreg(daif);
>>>>
>>>> 	if (system_uses_irq_prio_masking())
>>>> 		flags = flags & PSR_I_BIT ?
>>>> 			GIC_PRIO_IRQOFF : read_gicreg(ICC_PMR_EL1);
>>>>
>>>
>>> You're right, returning GIC_PRIO_IRQOFF should be good enough (it is
>>> actually what happens in this version because GIC_PRIO_IRQOFF ==
>>> GIC_PRIO_IRQON & ~PSR_I_BIT happens to be true).
>>
>> This wasn't entirely clear to me, I got confused by:
>>
>> +       BUILD_BUG_ON(GIC_PRIO_IRQOFF < (GIC_PRIO_IRQON & ~PSR_I_BIT));  \
>>
>> and I thought there isn't necessarily an equality between the two.
>>
>>> Your suggestion would
>>> make things easier to reason about. Maybe something like:
>>>
>>>
>>> static inline unsigned long arch_local_save_flags(void)
>>> {
>>> 	unsigned long daif_bits;
>>> 	unsigned long prio_off = GIC_PRIO_IRQOFF;
>>>
>>> 	daif_bits = read_sysreg(daif);
>>>
>>> 	asm volatile(ALTERNATIVE(
>>> 		"mov	%0, %1\n"
>>> 		"nop\n"
>>> 		"nop",
>>> 		"mrs	%0, SYS_ICC_PMR_EL1\n"
>>> 		"ands	%1, %1, PSR_I_BIT\n"
>>> 		"csel	%0, %0, %2, eq")
>>> 	: "=&r" (flags)
>>> 	: "r" (daif_bits), "r" (prio_off)
>>> 	: "memory");
>>>
>>> 	return flags;
>>> }
>>
>> It looks fine. If you turn the BUILD_BUG_ON into a !=, you could
>> probably simplify the asm a bit (though the number of instructions
>> generated would probably be the same). Untested:
>>
>> static inline unsigned long arch_local_save_flags(void)
>> {
>> 	unsigned long flags;
>>
>> 	flags = read_sysreg(daif);
>>
>> 	asm volatile(ALTERNATIVE(
>> 		"nop",
>> 		"bic	%0, %1, %2")
>> 	: "=&r" (flags)
>> 	: "r" (flags & PSR_I_BIT), "r" (GIC_PRIO_IRQOFF)
>> 	: "memory");
> 
> Ah, I missed a read from SYS_ICC_PMR_EL1 here. Anyway, the idea was that
> you don't need to set prio_off to a variable, just pass "r" (constant)
> here and the compiler does the trick.
> 

I see, thanks. I'll avoid that superfluous variable.

Thanks,

-- 
Julien Thierry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Marc Zyngier]

On Tue, 08 Jan 2019 14:07:19 +0000,
Julien Thierry <julien.thierry@arm.com> wrote:
> 
> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
> to interract with guest TLBs, switching from EL2&0 translation regime
> to EL1&0.
> 
> However, some non-maskable asynchronous event could happen while TGE is
> cleared like SDEI. Because of this address translation operations
> relying on EL2&0 translation regime could fail (tlb invalidation,
> userspace access, ...).
> 
> Fix this by properly setting HCR_EL2.TGE when entering NMI context and
> clear it if necessary when returning to the interrupted context.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: James Morse <james.morse@arm.com>
> Cc: linux-arch@vger.kernel.org
> Cc: stable@vger.kernel.org

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks,

	M.

-- 
Jazz is not dead, it just smell funny.

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Marc Zyngier]

On Tue, 08 Jan 2019 14:07:19 +0000,
Julien Thierry <julien.thierry@arm.com> wrote:
> 
> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
> to interract with guest TLBs, switching from EL2&0 translation regime
> to EL1&0.
> 
> However, some non-maskable asynchronous event could happen while TGE is
> cleared like SDEI. Because of this address translation operations
> relying on EL2&0 translation regime could fail (tlb invalidation,
> userspace access, ...).
> 
> Fix this by properly setting HCR_EL2.TGE when entering NMI context and
> clear it if necessary when returning to the interrupted context.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: James Morse <james.morse@arm.com>
> Cc: linux-arch@vger.kernel.org
> Cc: stable@vger.kernel.org

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks,

	M.

-- 
Jazz is not dead, it just smell funny.

Re: [PATCH v8 01/26] arm64: Fix HCR.TGE status for NMI contexts

[Marc Zyngier]

On Tue, 08 Jan 2019 14:07:19 +0000,
Julien Thierry <julien.thierry@arm.com> wrote:
> 
> When using VHE, the host needs to clear HCR_EL2.TGE bit in order
> to interract with guest TLBs, switching from EL2&0 translation regime
> to EL1&0.
> 
> However, some non-maskable asynchronous event could happen while TGE is
> cleared like SDEI. Because of this address translation operations
> relying on EL2&0 translation regime could fail (tlb invalidation,
> userspace access, ...).
> 
> Fix this by properly setting HCR_EL2.TGE when entering NMI context and
> clear it if necessary when returning to the interrupted context.
> 
> Signed-off-by: Julien Thierry <julien.thierry@arm.com>
> Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: James Morse <james.morse@arm.com>
> Cc: linux-arch@vger.kernel.org
> Cc: stable@vger.kernel.org

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks,

	M.

-- 
Jazz is not dead, it just smell funny.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel