From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D664C2D0CF for ; Mon, 23 Dec 2019 15:26:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 518C120709 for ; Mon, 23 Dec 2019 15:26:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=c-s.fr header.i=@c-s.fr header.b="TpqudzOM" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727140AbfLWP0X (ORCPT ); Mon, 23 Dec 2019 10:26:23 -0500 Received: from pegase1.c-s.fr ([93.17.236.30]:4385 "EHLO pegase1.c-s.fr" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726903AbfLWP0U (ORCPT ); Mon, 23 Dec 2019 10:26:20 -0500 Received: from localhost (mailhub1-ext [192.168.12.233]) by localhost (Postfix) with ESMTP id 47hNVF12djz9vJyy; Mon, 23 Dec 2019 16:26:13 +0100 (CET) Authentication-Results: localhost; dkim=pass reason="1024-bit key; insecure key" header.d=c-s.fr header.i=@c-s.fr header.b=TpqudzOM; dkim-adsp=pass; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [192.168.12.234]) (amavisd-new, port 10024) with ESMTP id zIN-liUu8jfd; Mon, 23 Dec 2019 16:26:13 +0100 (CET) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 47hNVD6zPtz9vJyw; Mon, 23 Dec 2019 16:26:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=c-s.fr; s=mail; t=1577114773; bh=CGg5lFjG79+f3AndFF6HVcEwxv4O9U6o4GFVGWYCcUM=; h=In-Reply-To:References:From:Subject:To:Cc:Date:From; b=TpqudzOMTrzBq6NdKEoT6ie4xE8cDofbv/TXPN8mm3jpmcvS9HGZ9rmIJDf8H2qn4 AlcfawkPHFYVGYFzJ0lHOyy94MW4kWjGWoHlrl2TsuNML9KyNvCGtrA0bg+DrYFqkE reBWPEeuMk/xDhJf2lp/RErt9oRP4/+nb9DD/jzE= Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 2120E8B7D1; Mon, 23 Dec 2019 16:26:18 +0100 (CET) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id vWNFPbebVhiN; Mon, 23 Dec 2019 16:26:18 +0100 (CET) Received: from po16098vm.idsi0.si.c-s.fr (po15451.idsi0.si.c-s.fr [172.25.230.100]) by messagerie.si.c-s.fr (Postfix) with ESMTP id EFBDF8B7A1; Mon, 23 Dec 2019 16:26:17 +0100 (CET) Received: by po16098vm.idsi0.si.c-s.fr (Postfix, from userid 0) id 05519637D8; Mon, 23 Dec 2019 15:26:17 +0000 (UTC) Message-Id: In-Reply-To: References: From: Christophe Leroy Subject: [RFC PATCH 7/8] powerpc/32: use IRQ stack immediately on IRQ exception To: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman Cc: linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org Date: Mon, 23 Dec 2019 15:26:17 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Exception entries run of kernel thread stack, then do_IRQ() switches to the IRQ stack. Instead of doing a first step of the thread stack, increasing the risk of stack overflow and spending time switch stacks two times when coming from userspace, set the stack to IRQ stack immediately in the EXCEPTION entry. In the same way as ARM64, consider that when the stack pointer is not within the kernel thread stack, it means it is already on IRQ stack. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/head_32.S | 2 +- arch/powerpc/kernel/head_32.h | 32 +++++++++++++++++++++++++++++--- arch/powerpc/kernel/head_40x.S | 2 +- arch/powerpc/kernel/head_8xx.S | 2 +- 4 files changed, 32 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kernel/head_32.S b/arch/powerpc/kernel/head_32.S index 4a24f8f026c7..0c36fba5b861 100644 --- a/arch/powerpc/kernel/head_32.S +++ b/arch/powerpc/kernel/head_32.S @@ -332,7 +332,7 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_HPTE_TABLE) EXC_XFER_LITE(0x400, handle_page_fault) /* External interrupt */ - EXCEPTION(0x500, HardwareInterrupt, do_IRQ, EXC_XFER_LITE) + EXCEPTION_IRQ(0x500, HardwareInterrupt, __do_irq, EXC_XFER_LITE) /* Alignment exception */ . = 0x600 diff --git a/arch/powerpc/kernel/head_32.h b/arch/powerpc/kernel/head_32.h index 8abc7783dbe5..f9e77e51723e 100644 --- a/arch/powerpc/kernel/head_32.h +++ b/arch/powerpc/kernel/head_32.h @@ -11,21 +11,41 @@ * task's thread_struct. */ -.macro EXCEPTION_PROLOG +.macro EXCEPTION_PROLOG is_irq=0 mtspr SPRN_SPRG_SCRATCH0,r10 mtspr SPRN_SPRG_SCRATCH1,r11 mfcr r10 - EXCEPTION_PROLOG_1 + EXCEPTION_PROLOG_1 is_irq=\is_irq EXCEPTION_PROLOG_2 .endm -.macro EXCEPTION_PROLOG_1 +.macro EXCEPTION_PROLOG_1 is_irq=0 mfspr r11,SPRN_SRR1 /* check whether user or kernel */ andi. r11,r11,MSR_PR + .if \is_irq + bne 2f + mfspr r11, SPRN_SPRG_THREAD + lwz r11, TASK_STACK - THREAD(r11) + xor r11, r11, r1 + cmplwi cr7, r11, THREAD_SIZE - 1 + tophys(r11, r1) /* use tophys(r1) if not thread stack */ + bgt cr7, 1f +2: +#ifdef CONFIG_SMP + mfspr r11, SPRN_SPRG_THREAD + lwz r11, TASK_CPU - THREAD(r11) + slwi r11, r11, 3 + addis r11, r11, (hardirq_ctx - PAGE_OFFSET)@ha +#else + lis r11, (hardirq_ctx - PAGE_OFFSET)@ha +#endif + lwz r11, (hardirq_ctx - PAGE_OFFSET)@l(r11) + .else tophys(r11,r1) /* use tophys(r1) if kernel */ beq 1f mfspr r11,SPRN_SPRG_THREAD lwz r11,TASK_STACK-THREAD(r11) + .endif addi r11,r11,THREAD_SIZE tophys(r11,r11) 1: subi r11,r11,INT_FRAME_SIZE /* alloc exc. frame */ @@ -171,6 +191,12 @@ addi r3,r1,STACK_FRAME_OVERHEAD; \ xfer(n, hdlr) +#define EXCEPTION_IRQ(n, label, hdlr, xfer) \ + START_EXCEPTION(n, label) \ + EXCEPTION_PROLOG is_irq=1; \ + addi r3,r1,STACK_FRAME_OVERHEAD; \ + xfer(n, hdlr) + #define EXC_XFER_TEMPLATE(hdlr, trap, msr, tfer, ret) \ li r10,trap; \ stw r10,_TRAP(r11); \ diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 4511fc1549f7..dd236f596c0b 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -315,7 +315,7 @@ _ENTRY(crit_srr1) EXC_XFER_LITE(0x400, handle_page_fault) /* 0x0500 - External Interrupt Exception */ - EXCEPTION(0x0500, HardwareInterrupt, do_IRQ, EXC_XFER_LITE) + EXCEPTION_IRQ(0x0500, HardwareInterrupt, __do_irq, EXC_XFER_LITE) /* 0x0600 - Alignment Exception */ START_EXCEPTION(0x0600, Alignment) diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index 19f583e18402..5a6cdbc89e26 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -150,7 +150,7 @@ DataAccess: InstructionAccess: /* External interrupt */ - EXCEPTION(0x500, HardwareInterrupt, do_IRQ, EXC_XFER_LITE) + EXCEPTION_IRQ(0x500, HardwareInterrupt, __do_irq, EXC_XFER_LITE) /* Alignment exception */ . = 0x600 -- 2.13.3