From: Julien Thierry <julien.thierry@arm.com> To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, daniel.thompson@linaro.org, joel@joelfernandes.org, marc.zyngier@arm.com, christoffer.dall@arm.com, james.morse@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, mark.rutland@arm.com, Julien Thierry <julien.thierry@arm.com>, Oleg Nesterov <oleg@redhat.com>, Dave Martin <Dave.Martin@arm.com> Subject: [PATCH v7 08/25] arm64: Make PMR part of task context Date: Wed, 12 Dec 2018 16:47:08 +0000 [thread overview] Message-ID: <1544633245-6036-9-git-send-email-julien.thierry@arm.com> (raw) In-Reply-To: <1544633245-6036-1-git-send-email-julien.thierry@arm.com> In order to replace PSR.I interrupt disabling/enabling with ICC_PMR_EL1 interrupt masking, ICC_PMR_EL1 needs to be saved/restored when taking/returning from an exception. This mimics the way hardware saves and restores PSR.I bit in spsr_el1 for exceptions and ERET. Add PMR to the registers to save in the pt_regs struct upon kernel entry, and restore it before ERET. Also, initialize it to a sane value when creating new tasks. Signed-off-by: Julien Thierry <julien.thierry@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Dave Martin <Dave.Martin@arm.com> --- arch/arm64/include/asm/processor.h | 3 +++ arch/arm64/include/asm/ptrace.h | 14 +++++++++++--- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/entry.S | 14 ++++++++++++++ arch/arm64/kernel/process.c | 6 ++++++ 5 files changed, 35 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 6b0d4df..27c5618 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -168,6 +168,9 @@ static inline void start_thread_common(struct pt_regs *regs, unsigned long pc) memset(regs, 0, sizeof(*regs)); forget_syscall(regs); regs->pc = pc; + + if (system_uses_irq_prio_masking()) + regs->pmr_save = GIC_PRIO_IRQON; } static inline void start_thread(struct pt_regs *regs, unsigned long pc, diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 05cf913..43e5df5 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -19,6 +19,8 @@ #ifndef __ASM_PTRACE_H #define __ASM_PTRACE_H +#include <asm/cpufeature.h> + #include <uapi/asm/ptrace.h> /* Current Exception Level values, as contained in CurrentEL */ @@ -179,7 +181,8 @@ struct pt_regs { #endif u64 orig_addr_limit; - u64 unused; // maintain 16 byte alignment + /* Only valid when ARM64_HAS_IRQ_PRIO_MASKING is enabled. */ + u64 pmr_save; u64 stackframe[2]; }; @@ -214,8 +217,13 @@ static inline void forget_syscall(struct pt_regs *regs) #define processor_mode(regs) \ ((regs)->pstate & PSR_MODE_MASK) -#define interrupts_enabled(regs) \ - (!((regs)->pstate & PSR_I_BIT)) +#define irqs_priority_unmasked(regs) \ + (system_uses_irq_prio_masking() ? \ + (regs)->pmr_save == GIC_PRIO_IRQON : \ + true) + +#define interrupts_enabled(regs) \ + (!((regs)->pstate & PSR_I_BIT) && irqs_priority_unmasked(regs)) #define fast_interrupts_enabled(regs) \ (!((regs)->pstate & PSR_F_BIT)) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 323aeb5..bab4122 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -78,6 +78,7 @@ int main(void) DEFINE(S_ORIG_X0, offsetof(struct pt_regs, orig_x0)); DEFINE(S_SYSCALLNO, offsetof(struct pt_regs, syscallno)); DEFINE(S_ORIG_ADDR_LIMIT, offsetof(struct pt_regs, orig_addr_limit)); + DEFINE(S_PMR_SAVE, offsetof(struct pt_regs, pmr_save)); DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe)); DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs)); BLANK(); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 039144e..2804c81 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -249,6 +249,12 @@ alternative_else_nop_endif msr sp_el0, tsk .endif + /* Save pmr */ +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + mrs_s x20, SYS_ICC_PMR_EL1 + str x20, [sp, #S_PMR_SAVE] +alternative_else_nop_endif + /* * Registers that may be useful after this macro is invoked: * @@ -269,6 +275,14 @@ alternative_else_nop_endif /* No need to restore UAO, it will be restored from SPSR_EL1 */ .endif + /* Restore pmr */ +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + ldr x20, [sp, #S_PMR_SAVE] + msr_s SYS_ICC_PMR_EL1, x20 + /* Ensure priority change is seen by redistributor */ + dsb sy +alternative_else_nop_endif + ldp x21, x22, [sp, #S_PC] // load ELR, SPSR .if \el == 0 ct_user_enter diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index d9a4c2d..34f495b 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -231,6 +231,9 @@ void __show_regs(struct pt_regs *regs) printk("sp : %016llx\n", sp); + if (system_uses_irq_prio_masking()) + printk("pmr_save: %08llx\n", regs->pmr_save); + i = top_reg; while (i >= 0) { @@ -362,6 +365,9 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start, if (arm64_get_ssbd_state() == ARM64_SSBD_FORCE_DISABLE) childregs->pstate |= PSR_SSBS_BIT; + if (system_uses_irq_prio_masking()) + childregs->pmr_save = GIC_PRIO_IRQON; + p->thread.cpu_context.x19 = stack_start; p->thread.cpu_context.x20 = stk_sz; } -- 1.9.1
WARNING: multiple messages have this Message-ID (diff)
From: Julien Thierry <julien.thierry@arm.com> To: linux-arm-kernel@lists.infradead.org Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry <julien.thierry@arm.com>, marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, Oleg Nesterov <oleg@redhat.com>, joel@joelfernandes.org, Dave Martin <Dave.Martin@arm.com> Subject: [PATCH v7 08/25] arm64: Make PMR part of task context Date: Wed, 12 Dec 2018 16:47:08 +0000 [thread overview] Message-ID: <1544633245-6036-9-git-send-email-julien.thierry@arm.com> (raw) In-Reply-To: <1544633245-6036-1-git-send-email-julien.thierry@arm.com> In order to replace PSR.I interrupt disabling/enabling with ICC_PMR_EL1 interrupt masking, ICC_PMR_EL1 needs to be saved/restored when taking/returning from an exception. This mimics the way hardware saves and restores PSR.I bit in spsr_el1 for exceptions and ERET. Add PMR to the registers to save in the pt_regs struct upon kernel entry, and restore it before ERET. Also, initialize it to a sane value when creating new tasks. Signed-off-by: Julien Thierry <julien.thierry@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Dave Martin <Dave.Martin@arm.com> --- arch/arm64/include/asm/processor.h | 3 +++ arch/arm64/include/asm/ptrace.h | 14 +++++++++++--- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/entry.S | 14 ++++++++++++++ arch/arm64/kernel/process.c | 6 ++++++ 5 files changed, 35 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 6b0d4df..27c5618 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -168,6 +168,9 @@ static inline void start_thread_common(struct pt_regs *regs, unsigned long pc) memset(regs, 0, sizeof(*regs)); forget_syscall(regs); regs->pc = pc; + + if (system_uses_irq_prio_masking()) + regs->pmr_save = GIC_PRIO_IRQON; } static inline void start_thread(struct pt_regs *regs, unsigned long pc, diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 05cf913..43e5df5 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -19,6 +19,8 @@ #ifndef __ASM_PTRACE_H #define __ASM_PTRACE_H +#include <asm/cpufeature.h> + #include <uapi/asm/ptrace.h> /* Current Exception Level values, as contained in CurrentEL */ @@ -179,7 +181,8 @@ struct pt_regs { #endif u64 orig_addr_limit; - u64 unused; // maintain 16 byte alignment + /* Only valid when ARM64_HAS_IRQ_PRIO_MASKING is enabled. */ + u64 pmr_save; u64 stackframe[2]; }; @@ -214,8 +217,13 @@ static inline void forget_syscall(struct pt_regs *regs) #define processor_mode(regs) \ ((regs)->pstate & PSR_MODE_MASK) -#define interrupts_enabled(regs) \ - (!((regs)->pstate & PSR_I_BIT)) +#define irqs_priority_unmasked(regs) \ + (system_uses_irq_prio_masking() ? \ + (regs)->pmr_save == GIC_PRIO_IRQON : \ + true) + +#define interrupts_enabled(regs) \ + (!((regs)->pstate & PSR_I_BIT) && irqs_priority_unmasked(regs)) #define fast_interrupts_enabled(regs) \ (!((regs)->pstate & PSR_F_BIT)) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 323aeb5..bab4122 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -78,6 +78,7 @@ int main(void) DEFINE(S_ORIG_X0, offsetof(struct pt_regs, orig_x0)); DEFINE(S_SYSCALLNO, offsetof(struct pt_regs, syscallno)); DEFINE(S_ORIG_ADDR_LIMIT, offsetof(struct pt_regs, orig_addr_limit)); + DEFINE(S_PMR_SAVE, offsetof(struct pt_regs, pmr_save)); DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe)); DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs)); BLANK(); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 039144e..2804c81 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -249,6 +249,12 @@ alternative_else_nop_endif msr sp_el0, tsk .endif + /* Save pmr */ +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + mrs_s x20, SYS_ICC_PMR_EL1 + str x20, [sp, #S_PMR_SAVE] +alternative_else_nop_endif + /* * Registers that may be useful after this macro is invoked: * @@ -269,6 +275,14 @@ alternative_else_nop_endif /* No need to restore UAO, it will be restored from SPSR_EL1 */ .endif + /* Restore pmr */ +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + ldr x20, [sp, #S_PMR_SAVE] + msr_s SYS_ICC_PMR_EL1, x20 + /* Ensure priority change is seen by redistributor */ + dsb sy +alternative_else_nop_endif + ldp x21, x22, [sp, #S_PC] // load ELR, SPSR .if \el == 0 ct_user_enter diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index d9a4c2d..34f495b 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -231,6 +231,9 @@ void __show_regs(struct pt_regs *regs) printk("sp : %016llx\n", sp); + if (system_uses_irq_prio_masking()) + printk("pmr_save: %08llx\n", regs->pmr_save); + i = top_reg; while (i >= 0) { @@ -362,6 +365,9 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start, if (arm64_get_ssbd_state() == ARM64_SSBD_FORCE_DISABLE) childregs->pstate |= PSR_SSBS_BIT; + if (system_uses_irq_prio_masking()) + childregs->pmr_save = GIC_PRIO_IRQON; + p->thread.cpu_context.x19 = stack_start; p->thread.cpu_context.x20 = stk_sz; } -- 1.9.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2018-12-12 16:48 UTC|newest] Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-12-12 16:47 [PATCH v7 00/25] arm64: provide pseudo NMI with GICv3 Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 01/25] arm64: Fix HCR.TGE status for NMI contexts Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 21:39 ` Sasha Levin 2018-12-12 21:39 ` Sasha Levin 2018-12-17 8:49 ` Julien Thierry 2018-12-17 8:49 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 02/25] arm64: Remove unused daif related functions/macros Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 03/25] arm64: cpufeature: Set SYSREG_GIC_CPUIF as a boot system feature Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 04/25] arm64: cpufeature: Add cpufeature for IRQ priority masking Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 05/25] arm/arm64: gic-v3: Add PMR and RPR accessors Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 06/25] irqchip/gic-v3: Switch to PMR masking before calling IRQ handler Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 07/25] arm64: ptrace: Provide definitions for PMR values Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` Julien Thierry [this message] 2018-12-12 16:47 ` [PATCH v7 08/25] arm64: Make PMR part of task context Julien Thierry 2018-12-12 16:47 ` [PATCH v7 09/25] arm64: Unmask PMR before going idle Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 10/25] arm64: kvm: Unmask PMR before entering guest Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 11/25] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 17:27 ` Ard Biesheuvel 2018-12-12 17:27 ` Ard Biesheuvel 2018-12-12 17:59 ` Julien Thierry 2018-12-12 17:59 ` Julien Thierry 2018-12-12 18:10 ` Ard Biesheuvel 2018-12-12 18:10 ` Ard Biesheuvel 2018-12-13 8:54 ` Julien Thierry 2018-12-13 8:54 ` Julien Thierry 2018-12-13 11:35 ` Ard Biesheuvel 2018-12-13 11:35 ` Ard Biesheuvel 2018-12-13 12:02 ` Julien Thierry 2018-12-13 12:02 ` Julien Thierry 2018-12-13 15:03 ` Julien Thierry 2018-12-13 15:03 ` Julien Thierry 2018-12-14 15:23 ` Julien Thierry 2018-12-14 15:23 ` Julien Thierry 2018-12-14 15:49 ` Ard Biesheuvel 2018-12-14 15:49 ` Ard Biesheuvel 2018-12-14 16:40 ` Julien Thierry 2018-12-14 16:40 ` Julien Thierry 2018-12-19 17:01 ` Julien Thierry 2018-12-19 17:01 ` Julien Thierry 2018-12-20 17:53 ` Ard Biesheuvel 2018-12-20 17:53 ` Ard Biesheuvel 2018-12-21 10:25 ` Julien Thierry 2018-12-21 10:25 ` Julien Thierry 2018-12-16 14:47 ` Jian-Lin Chen 2018-12-16 14:47 ` Jian-Lin Chen 2018-12-17 9:26 ` Julien Thierry 2018-12-17 9:26 ` Julien Thierry 2018-12-18 8:36 ` Jian-Lin Chen 2018-12-18 8:36 ` Jian-Lin Chen 2018-12-12 16:47 ` [PATCH v7 12/25] arm64: daifflags: Include PMR in daifflags restore operations Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 13/25] arm64: alternative: Allow alternative status checking per cpufeature Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 14/25] arm64: alternative: Apply alternatives early in boot process Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 15/25] irqchip/gic-v3: Factor group0 detection into functions Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 16/25] arm64: Switch to PMR masking when starting CPUs Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 17/25] arm64: gic-v3: Implement arch support for priority masking Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 18/25] irqchip/gic-v3: Detect if GIC can support pseudo-NMIs Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 19/25] irqchip/gic-v3: Handle pseudo-NMIs Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 20/25] irqchip/gic: Add functions to access irq priorities Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 21/25] irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 22/25] arm64: Handle serror in NMI context Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 23/25] arm64: Skip preemption when exiting an NMI Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 24/25] arm64: Skip irqflags tracing for NMI in IRQs disabled context Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:47 ` [PATCH v7 25/25] arm64: Enable the support of pseudo-NMIs Julien Thierry 2018-12-12 16:47 ` Julien Thierry 2018-12-12 16:52 ` [PATCH v7 00/25] arm64: provide pseudo NMI with GICv3 Julien Thierry 2018-12-12 16:52 ` Julien Thierry
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1544633245-6036-9-git-send-email-julien.thierry@arm.com \ --to=julien.thierry@arm.com \ --cc=Dave.Martin@arm.com \ --cc=catalin.marinas@arm.com \ --cc=christoffer.dall@arm.com \ --cc=daniel.thompson@linaro.org \ --cc=james.morse@arm.com \ --cc=joel@joelfernandes.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=marc.zyngier@arm.com \ --cc=mark.rutland@arm.com \ --cc=oleg@redhat.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.