From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47F08CCA487 for ; Wed, 6 Jul 2022 14:03:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233398AbiGFODK (ORCPT ); Wed, 6 Jul 2022 10:03:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229678AbiGFODG (ORCPT ); Wed, 6 Jul 2022 10:03:06 -0400 Received: from mail-yw1-f170.google.com (mail-yw1-f170.google.com [209.85.128.170]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B90D718E13; Wed, 6 Jul 2022 07:03:04 -0700 (PDT) Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-2ef5380669cso140859827b3.9; Wed, 06 Jul 2022 07:03:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WBBLwGm3VvIAnm4uSuSyiWShri4w+bgQrIyP7HIMtP0=; b=BWYOrQ+sRmqafGTok0X9D3MjBgtmyg/jJ6UU/mGm95R2O4YIyU3LyRJLu85uPBtlO4 nieCudb8Kc25IevUlq2fVk0h26HUxn9M9/XKbFL7dyFO5eOWW6pW+cgNiiH/ojMJHPUC RcHr8rvHrjLbGxIm8qTquOju9+xMDpcPWyHk3dY7fFwxUJHZZrx9XOL1wy3gCcCb6mrA /gdbXoz7eqddOdatE/209+U//88wG4J1eUpyJgFfl6tjBWgUy78K6h1KDvWdh8VDE1iA gnpVABcdDzDjuJyEN8Tr2lDwCz3hndD0mNiUVLfAuVUCNd6sEi426TQr4nZrMr0Mb1MI 8j4g== X-Gm-Message-State: AJIora97c7r7OlScDvg7zdGxAiQ758qsuSnSyiRZeFNJtr2ULfv8V4bW YZe/I6CydShxM7ImtybBFR8yIM5w+yUBlQam2lQ= X-Google-Smtp-Source: AGRyM1shXjwii2Xu/qgQ1B1HadHYSMjqitGmbFlHxf4P7RndZwyBhPC2CaCRixiXqBuDMvjO7JXHYGnoD2iVhXDPhek= X-Received: by 2002:a81:a184:0:b0:31c:b00e:b5c4 with SMTP id y126-20020a81a184000000b0031cb00eb5c4mr15058481ywg.149.1657116183677; Wed, 06 Jul 2022 07:03:03 -0700 (PDT) MIME-Version: 1.0 References: <20220608142723.103523089@infradead.org> <20220608144517.188449351@infradead.org> In-Reply-To: <20220608144517.188449351@infradead.org> From: "Rafael J. Wysocki" Date: Wed, 6 Jul 2022 16:02:52 +0200 Message-ID: Subject: Re: [PATCH 20/36] arch/idle: Change arch_cpu_idle() IRQ behaviour To: Peter Zijlstra Cc: rth@twiddle.net, ink@jurassic.park.msu.ru, mattst88@gmail.com, vgupta@kernel.org, Russell King - ARM Linux , ulli.kroll@googlemail.com, Linus Walleij , Shawn Guo , Sascha Hauer , Sascha Hauer , Fabio Estevam , dl-linux-imx , Tony Lindgren , Kevin Hilman , Catalin Marinas , Will Deacon , Guo Ren , bcain@quicinc.com, Huacai Chen , kernel@xen0n.name, Geert Uytterhoeven , sammy@sammy.net, Michal Simek , Thomas Bogendoerfer , dinguyen@kernel.org, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Stafford Horne , James Bottomley , Helge Deller , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Yoshinori Sato , Rich Felker , David Miller , Richard Weinberger , anton.ivanov@cambridgegreys.com, Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "the arch/x86 maintainers" , "H. Peter Anvin" , acme@kernel.org, Mark Rutland , Alexander Shishkin , jolsa@kernel.org, namhyung@kernel.org, Juergen Gross , srivatsa@csail.mit.edu, amakhalov@vmware.com, pv-drivers@vmware.com, Boris Ostrovsky , Chris Zankel , jcmvbkbc@gmail.com, "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Michael Turquette , Stephen Boyd , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , Andy Gross , Bjorn Andersson , Anup Patel , Thierry Reding , Jon Hunter , Jacob Pan , Arnd Bergmann , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Steven Rostedt , Petr Mladek , senozhatsky@chromium.org, John Ogness , "Paul E. McKenney" , Frederic Weisbecker , quic_neeraju@quicinc.com, Josh Triplett , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Benjamin Segall , Mel Gorman , Daniel Bristot de Oliveira , vschneid@redhat.com, jpoimboe@kernel.org, linux-alpha@vger.kernel.org, Linux Kernel Mailing List , linux-snps-arc@lists.infradead.org, Linux ARM , Linux OMAP Mailing List , linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, linux-m68k , "open list:BROADCOM NVRAM DRIVER" , openrisc@lists.librecores.org, Parisc List , linuxppc-dev , linux-riscv , linux-s390@vger.kernel.org, Linux-sh list , sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-perf-users@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, linux-xtensa@linux-xtensa.org, ACPI Devel Maling List , Linux PM , linux-clk , linux-arm-msm , linux-tegra , linux-arch , rcu@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org On Wed, Jun 8, 2022 at 4:46 PM Peter Zijlstra wrote: > > Current arch_cpu_idle() is called with IRQs disabled, but will return > with IRQs enabled. > > However, the very first thing the generic code does after calling > arch_cpu_idle() is raw_local_irq_disable(). This means that > architectures that can idle with IRQs disabled end up doing a > pointless 'enable-disable' dance. > > Therefore, push this IRQ disabling into the idle function, meaning > that those architectures can avoid the pointless IRQ state flipping. > > Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki > --- > arch/alpha/kernel/process.c | 1 - > arch/arc/kernel/process.c | 3 +++ > arch/arm/kernel/process.c | 1 - > arch/arm/mach-gemini/board-dt.c | 3 ++- > arch/arm64/kernel/idle.c | 1 - > arch/csky/kernel/process.c | 1 - > arch/csky/kernel/smp.c | 2 +- > arch/hexagon/kernel/process.c | 1 - > arch/ia64/kernel/process.c | 1 + > arch/microblaze/kernel/process.c | 1 - > arch/mips/kernel/idle.c | 8 +++----- > arch/nios2/kernel/process.c | 1 - > arch/openrisc/kernel/process.c | 1 + > arch/parisc/kernel/process.c | 2 -- > arch/powerpc/kernel/idle.c | 5 ++--- > arch/riscv/kernel/process.c | 1 - > arch/s390/kernel/idle.c | 1 - > arch/sh/kernel/idle.c | 1 + > arch/sparc/kernel/leon_pmc.c | 4 ++++ > arch/sparc/kernel/process_32.c | 1 - > arch/sparc/kernel/process_64.c | 3 ++- > arch/um/kernel/process.c | 1 - > arch/x86/coco/tdx/tdx.c | 3 +++ > arch/x86/kernel/process.c | 15 ++++----------- > arch/xtensa/kernel/process.c | 1 + > kernel/sched/idle.c | 2 -- > 26 files changed, 28 insertions(+), 37 deletions(-) > > --- a/arch/alpha/kernel/process.c > +++ b/arch/alpha/kernel/process.c > @@ -57,7 +57,6 @@ EXPORT_SYMBOL(pm_power_off); > void arch_cpu_idle(void) > { > wtint(0); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_dead(void) > --- a/arch/arc/kernel/process.c > +++ b/arch/arc/kernel/process.c > @@ -114,6 +114,8 @@ void arch_cpu_idle(void) > "sleep %0 \n" > : > :"I"(arg)); /* can't be "r" has to be embedded const */ > + > + raw_local_irq_disable(); > } > > #else /* ARC700 */ > @@ -122,6 +124,7 @@ void arch_cpu_idle(void) > { > /* sleep, but enable both set E1/E2 (levels of interrupts) before committing */ > __asm__ __volatile__("sleep 0x3 \n"); > + raw_local_irq_disable(); > } > > #endif > --- a/arch/arm/kernel/process.c > +++ b/arch/arm/kernel/process.c > @@ -78,7 +78,6 @@ void arch_cpu_idle(void) > arm_pm_idle(); > else > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_prepare(void) > --- a/arch/arm/mach-gemini/board-dt.c > +++ b/arch/arm/mach-gemini/board-dt.c > @@ -42,8 +42,9 @@ static void gemini_idle(void) > */ > > /* FIXME: Enabling interrupts here is racy! */ > - local_irq_enable(); > + raw_local_irq_enable(); > cpu_do_idle(); > + raw_local_irq_disable(); > } > > static void __init gemini_init_machine(void) > --- a/arch/arm64/kernel/idle.c > +++ b/arch/arm64/kernel/idle.c > @@ -42,5 +42,4 @@ void noinstr arch_cpu_idle(void) > * tricks > */ > cpu_do_idle(); > - raw_local_irq_enable(); > } > --- a/arch/csky/kernel/process.c > +++ b/arch/csky/kernel/process.c > @@ -101,6 +101,5 @@ void arch_cpu_idle(void) > #ifdef CONFIG_CPU_PM_STOP > asm volatile("stop\n"); > #endif > - raw_local_irq_enable(); > } > #endif > --- a/arch/csky/kernel/smp.c > +++ b/arch/csky/kernel/smp.c > @@ -314,7 +314,7 @@ void arch_cpu_idle_dead(void) > while (!secondary_stack) > arch_cpu_idle(); > > - local_irq_disable(); > + raw_local_irq_disable(); > > asm volatile( > "mov sp, %0\n" > --- a/arch/hexagon/kernel/process.c > +++ b/arch/hexagon/kernel/process.c > @@ -44,7 +44,6 @@ void arch_cpu_idle(void) > { > __vmwait(); > /* interrupts wake us up, but irqs are still disabled */ > - raw_local_irq_enable(); > } > > /* > --- a/arch/ia64/kernel/process.c > +++ b/arch/ia64/kernel/process.c > @@ -241,6 +241,7 @@ void arch_cpu_idle(void) > (*mark_idle)(1); > > raw_safe_halt(); > + raw_local_irq_disable(); > > if (mark_idle) > (*mark_idle)(0); > --- a/arch/microblaze/kernel/process.c > +++ b/arch/microblaze/kernel/process.c > @@ -138,5 +138,4 @@ int dump_fpu(struct pt_regs *regs, elf_f > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > --- a/arch/mips/kernel/idle.c > +++ b/arch/mips/kernel/idle.c > @@ -33,13 +33,13 @@ static void __cpuidle r3081_wait(void) > { > unsigned long cfg = read_c0_conf(); > write_c0_conf(cfg | R30XX_CONF_HALT); > - raw_local_irq_enable(); > } > > void __cpuidle r4k_wait(void) > { > raw_local_irq_enable(); > __r4k_wait(); > + raw_local_irq_disable(); > } > > /* > @@ -57,7 +57,6 @@ void __cpuidle r4k_wait_irqoff(void) > " .set arch=r4000 \n" > " wait \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -77,7 +76,6 @@ static void __cpuidle rm7k_wait_irqoff(v > " wait \n" > " mtc0 $1, $12 # stalls until W stage \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -103,6 +101,8 @@ static void __cpuidle au1k_wait(void) > " nop \n" > " .set pop \n" > : : "r" (au1k_wait), "r" (c0status)); > + > + raw_local_irq_disable(); > } > > static int __initdata nowait; > @@ -245,8 +245,6 @@ void arch_cpu_idle(void) > { > if (cpu_wait) > cpu_wait(); > - else > - raw_local_irq_enable(); > } > > #ifdef CONFIG_CPU_IDLE > --- a/arch/nios2/kernel/process.c > +++ b/arch/nios2/kernel/process.c > @@ -33,7 +33,6 @@ EXPORT_SYMBOL(pm_power_off); > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > > /* > --- a/arch/openrisc/kernel/process.c > +++ b/arch/openrisc/kernel/process.c > @@ -102,6 +102,7 @@ void arch_cpu_idle(void) > raw_local_irq_enable(); > if (mfspr(SPR_UPR) & SPR_UPR_PMP) > mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME); > + raw_local_irq_disable(); > } > > void (*pm_power_off)(void) = NULL; > --- a/arch/parisc/kernel/process.c > +++ b/arch/parisc/kernel/process.c > @@ -187,8 +187,6 @@ void arch_cpu_idle_dead(void) > > void __cpuidle arch_cpu_idle(void) > { > - raw_local_irq_enable(); > - > /* nop on real hardware, qemu will idle sleep. */ > asm volatile("or %%r10,%%r10,%%r10\n":::); > } > --- a/arch/powerpc/kernel/idle.c > +++ b/arch/powerpc/kernel/idle.c > @@ -51,10 +51,9 @@ void arch_cpu_idle(void) > * Some power_save functions return with > * interrupts enabled, some don't. > */ > - if (irqs_disabled()) > - raw_local_irq_enable(); > + if (!irqs_disabled()) > + raw_local_irq_disable(); > } else { > - raw_local_irq_enable(); > /* > * Go into low thread priority and possibly > * low power mode. > --- a/arch/riscv/kernel/process.c > +++ b/arch/riscv/kernel/process.c > @@ -39,7 +39,6 @@ extern asmlinkage void ret_from_kernel_t > void arch_cpu_idle(void) > { > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void __show_regs(struct pt_regs *regs) > --- a/arch/s390/kernel/idle.c > +++ b/arch/s390/kernel/idle.c > @@ -66,7 +66,6 @@ void arch_cpu_idle(void) > idle->idle_count++; > account_idle_time(cputime_to_nsecs(idle_time)); > raw_write_seqcount_end(&idle->seqcount); > - raw_local_irq_enable(); > } > > static ssize_t show_idle_count(struct device *dev, > --- a/arch/sh/kernel/idle.c > +++ b/arch/sh/kernel/idle.c > @@ -25,6 +25,7 @@ void default_idle(void) > raw_local_irq_enable(); > /* Isn't this racy ? */ > cpu_sleep(); > + raw_local_irq_disable(); > clear_bl_bit(); > } > > --- a/arch/sparc/kernel/leon_pmc.c > +++ b/arch/sparc/kernel/leon_pmc.c > @@ -57,6 +57,8 @@ static void pmc_leon_idle_fixup(void) > "lda [%0] %1, %%g0\n" > : > : "r"(address), "i"(ASI_LEON_BYPASS)); > + > + raw_local_irq_disable(); > } > > /* > @@ -70,6 +72,8 @@ static void pmc_leon_idle(void) > > /* For systems without power-down, this will be no-op */ > __asm__ __volatile__ ("wr %g0, %asr19\n\t"); > + > + raw_local_irq_disable(); > } > > /* Install LEON Power Down function */ > --- a/arch/sparc/kernel/process_32.c > +++ b/arch/sparc/kernel/process_32.c > @@ -71,7 +71,6 @@ void arch_cpu_idle(void) > { > if (sparc_idle) > (*sparc_idle)(); > - raw_local_irq_enable(); > } > > /* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */ > --- a/arch/sparc/kernel/process_64.c > +++ b/arch/sparc/kernel/process_64.c > @@ -59,7 +59,6 @@ void arch_cpu_idle(void) > { > if (tlb_type != hypervisor) { > touch_nmi_watchdog(); > - raw_local_irq_enable(); > } else { > unsigned long pstate; > > @@ -90,6 +89,8 @@ void arch_cpu_idle(void) > "wrpr %0, %%g0, %%pstate" > : "=&r" (pstate) > : "i" (PSTATE_IE)); > + > + raw_local_irq_disable(); > } > } > > --- a/arch/um/kernel/process.c > +++ b/arch/um/kernel/process.c > @@ -216,7 +216,6 @@ void arch_cpu_idle(void) > { > cpu_tasks[current_thread_info()->cpu].pid = os_getpid(); > um_idle_sleep(); > - raw_local_irq_enable(); > } > > int __cant_sleep(void) { > --- a/arch/x86/coco/tdx/tdx.c > +++ b/arch/x86/coco/tdx/tdx.c > @@ -178,6 +178,9 @@ void __cpuidle tdx_safe_halt(void) > */ > if (__halt(irq_disabled, do_sti)) > WARN_ONCE(1, "HLT instruction emulation failed\n"); > + > + /* XXX I can't make sense of what @do_sti actually does */ > + raw_local_irq_disable(); > } > > static bool read_msr(struct pt_regs *regs) > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -699,6 +699,7 @@ EXPORT_SYMBOL(boot_option_idle_override) > void __cpuidle default_idle(void) > { > raw_safe_halt(); > + raw_local_irq_disable(); > } > #if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE) > EXPORT_SYMBOL(default_idle); > @@ -804,13 +805,7 @@ static void amd_e400_idle(void) > > default_idle(); > > - /* > - * The switch back from broadcast mode needs to be called with > - * interrupts disabled. > - */ > - raw_local_irq_disable(); > tick_broadcast_exit(); > - raw_local_irq_enable(); > } > > /* > @@ -849,12 +844,10 @@ static __cpuidle void mwait_idle(void) > } > > __monitor((void *)¤t_thread_info()->flags, 0, 0); > - if (!need_resched()) > + if (!need_resched()) { > __sti_mwait(0, 0); > - else > - raw_local_irq_enable(); > - } else { > - raw_local_irq_enable(); > + raw_local_irq_disable(); > + } > } > __current_clr_polling(); > } > --- a/arch/xtensa/kernel/process.c > +++ b/arch/xtensa/kernel/process.c > @@ -183,6 +183,7 @@ void coprocessor_flush_release_all(struc > void arch_cpu_idle(void) > { > platform_idle(); > + raw_local_irq_disable(); > } > > /* > --- a/kernel/sched/idle.c > +++ b/kernel/sched/idle.c > @@ -79,7 +79,6 @@ void __weak arch_cpu_idle_dead(void) { } > void __weak arch_cpu_idle(void) > { > cpu_idle_force_poll = 1; > - raw_local_irq_enable(); > } > > /** > @@ -96,7 +95,6 @@ void __cpuidle default_idle_call(void) > > cpuidle_rcu_enter(); > arch_cpu_idle(); > - raw_local_irq_disable(); > cpuidle_rcu_exit(); > > start_critical_timings(); > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5ABAECCA48B for ; Wed, 6 Jul 2022 14:03:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8V1OtLgYcOUa+sZ2IRl43FMvU+94kUDCB7zsHeV9lhc=; b=UeNDqFAo84ENXI GmxY5fVdxo650lTyve2WZCMg7DPQutOyJNEAkwEJ/D/PQB3BfM/QKEfkTd2aOf4TqsPEbjqMSDWsx IochD2subNeCZTwfYfioKV4nRjgzPh5Qa3DQ0pZC9bUjjm2iw/KGJs23m1ZSYTh/X7DVWy9un3M9w x/HEad3DUUpWAyrP1bUzF14YsGfD+Dyith4FN5JDFK9/XI+J/Ig2jIUrkqjGbkJ8cxcP8Rl2L+v9x 3dx2IVpxpT3XWVtwM5Vp7l8r9HJVBmibFdUCU5OaRucb4fFE5zhSZ3B7FMqu31XlpspWyi5inV15N ST3DZKb1P4VYzGQkGhcg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o95cT-00AVxO-MT; Wed, 06 Jul 2022 14:03:09 +0000 Received: from mail-yw1-f182.google.com ([209.85.128.182]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o95cP-00AVvR-Q0; Wed, 06 Jul 2022 14:03:08 +0000 Received: by mail-yw1-f182.google.com with SMTP id 00721157ae682-31c89653790so89902857b3.13; Wed, 06 Jul 2022 07:03:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WBBLwGm3VvIAnm4uSuSyiWShri4w+bgQrIyP7HIMtP0=; b=Suv+XS600hsJ/7EeZpN6J0ljoxDWvEGyq88jk6AgUPfWp9rDk8lD8kdLeFi8EphBgr vs1jh3B17JQAs5ryHYdBKuXYsvHgVg/NTAJg8s3Fs/9U/sb8yU1OimiJy2WVreh9I/Cx pU48rDD1RTzV2HfDPzGc6Vl0l7NU4ZZRi331hcg6bkuJoLRL/K4+X1F0N3mn26KFuwYQ Y7JooYbLrO9bQlXMq73L10gaSlfYt9c02V7n2foTVerus61dm04jIrr5LsxfRp5qHr3w luLvRvjJuip5VLoj7OsLiV5dnPCbAhkyF7Zi3dFx8lmOPdLDMjKGJo/z16HRa2VAhZ3g 4m3Q== X-Gm-Message-State: AJIora/wBO5RdK5Gm1R2L26VfxJx8pWrWKWb7AmGW/LhrzSrGP4rnxwJ tM7no6bRLcZsS0+zE/S31e6KPdWhlLBcHZsYyfU= X-Google-Smtp-Source: AGRyM1shXjwii2Xu/qgQ1B1HadHYSMjqitGmbFlHxf4P7RndZwyBhPC2CaCRixiXqBuDMvjO7JXHYGnoD2iVhXDPhek= X-Received: by 2002:a81:a184:0:b0:31c:b00e:b5c4 with SMTP id y126-20020a81a184000000b0031cb00eb5c4mr15058481ywg.149.1657116183677; Wed, 06 Jul 2022 07:03:03 -0700 (PDT) MIME-Version: 1.0 References: <20220608142723.103523089@infradead.org> <20220608144517.188449351@infradead.org> In-Reply-To: <20220608144517.188449351@infradead.org> From: "Rafael J. Wysocki" Date: Wed, 6 Jul 2022 16:02:52 +0200 Message-ID: Subject: Re: [PATCH 20/36] arch/idle: Change arch_cpu_idle() IRQ behaviour To: Peter Zijlstra Cc: rth@twiddle.net, ink@jurassic.park.msu.ru, mattst88@gmail.com, vgupta@kernel.org, Russell King - ARM Linux , ulli.kroll@googlemail.com, Linus Walleij , Shawn Guo , Sascha Hauer , Sascha Hauer , Fabio Estevam , dl-linux-imx , Tony Lindgren , Kevin Hilman , Catalin Marinas , Will Deacon , Guo Ren , bcain@quicinc.com, Huacai Chen , kernel@xen0n.name, Geert Uytterhoeven , sammy@sammy.net, Michal Simek , Thomas Bogendoerfer , dinguyen@kernel.org, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Stafford Horne , James Bottomley , Helge Deller , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Yoshinori Sato , Rich Felker , David Miller , Richard Weinberger , anton.ivanov@cambridgegreys.com, Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "the arch/x86 maintainers" , "H. Peter Anvin" , acme@kernel.org, Mark Rutland , Alexander Shishkin , jolsa@kernel.org, namhyung@kernel.org, Juergen Gross , srivatsa@csail.mit.edu, amakhalov@vmware.com, pv-drivers@vmware.com, Boris Ostrovsky , Chris Zankel , jcmvbkbc@gmail.com, "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Michael Turquette , Stephen Boyd , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , Andy Gross , Bjorn Andersson , Anup Patel , Thierry Reding , Jon Hunter , Jacob Pan , Arnd Bergmann , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Steven Rostedt , Petr Mladek , senozhatsky@chromium.org, John Ogness , "Paul E. McKenney" , Frederic Weisbecker , quic_neeraju@quicinc.com, Josh Triplett , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Benjamin Segall , Mel Gorman , Daniel Bristot de Oliveira , vschneid@redhat.com, jpoimboe@kernel.org, linux-alpha@vger.kernel.org, Linux Kernel Mailing List , linux-snps-arc@lists.infradead.org, Linux ARM , Linux OMAP Mailing List , linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, linux-m68k , "open list:BROADCOM NVRAM DRIVER" , openrisc@lists.librecores.org, Parisc List , linuxppc-dev , linux-riscv , linux-s390@vger.kernel.org, Linux-sh list , sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-perf-users@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, linux-xtensa@linux-xtensa.org, ACPI Devel Maling List , Linux PM , linux-clk , linux-arm-msm , linux-tegra , linux-arch , rcu@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220706_070305_879188_2C952C0E X-CRM114-Status: GOOD ( 30.55 ) X-BeenThere: linux-snps-arc@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Linux on Synopsys ARC Processors List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-snps-arc" Errors-To: linux-snps-arc-bounces+linux-snps-arc=archiver.kernel.org@lists.infradead.org On Wed, Jun 8, 2022 at 4:46 PM Peter Zijlstra wrote: > > Current arch_cpu_idle() is called with IRQs disabled, but will return > with IRQs enabled. > > However, the very first thing the generic code does after calling > arch_cpu_idle() is raw_local_irq_disable(). This means that > architectures that can idle with IRQs disabled end up doing a > pointless 'enable-disable' dance. > > Therefore, push this IRQ disabling into the idle function, meaning > that those architectures can avoid the pointless IRQ state flipping. > > Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki > --- > arch/alpha/kernel/process.c | 1 - > arch/arc/kernel/process.c | 3 +++ > arch/arm/kernel/process.c | 1 - > arch/arm/mach-gemini/board-dt.c | 3 ++- > arch/arm64/kernel/idle.c | 1 - > arch/csky/kernel/process.c | 1 - > arch/csky/kernel/smp.c | 2 +- > arch/hexagon/kernel/process.c | 1 - > arch/ia64/kernel/process.c | 1 + > arch/microblaze/kernel/process.c | 1 - > arch/mips/kernel/idle.c | 8 +++----- > arch/nios2/kernel/process.c | 1 - > arch/openrisc/kernel/process.c | 1 + > arch/parisc/kernel/process.c | 2 -- > arch/powerpc/kernel/idle.c | 5 ++--- > arch/riscv/kernel/process.c | 1 - > arch/s390/kernel/idle.c | 1 - > arch/sh/kernel/idle.c | 1 + > arch/sparc/kernel/leon_pmc.c | 4 ++++ > arch/sparc/kernel/process_32.c | 1 - > arch/sparc/kernel/process_64.c | 3 ++- > arch/um/kernel/process.c | 1 - > arch/x86/coco/tdx/tdx.c | 3 +++ > arch/x86/kernel/process.c | 15 ++++----------- > arch/xtensa/kernel/process.c | 1 + > kernel/sched/idle.c | 2 -- > 26 files changed, 28 insertions(+), 37 deletions(-) > > --- a/arch/alpha/kernel/process.c > +++ b/arch/alpha/kernel/process.c > @@ -57,7 +57,6 @@ EXPORT_SYMBOL(pm_power_off); > void arch_cpu_idle(void) > { > wtint(0); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_dead(void) > --- a/arch/arc/kernel/process.c > +++ b/arch/arc/kernel/process.c > @@ -114,6 +114,8 @@ void arch_cpu_idle(void) > "sleep %0 \n" > : > :"I"(arg)); /* can't be "r" has to be embedded const */ > + > + raw_local_irq_disable(); > } > > #else /* ARC700 */ > @@ -122,6 +124,7 @@ void arch_cpu_idle(void) > { > /* sleep, but enable both set E1/E2 (levels of interrupts) before committing */ > __asm__ __volatile__("sleep 0x3 \n"); > + raw_local_irq_disable(); > } > > #endif > --- a/arch/arm/kernel/process.c > +++ b/arch/arm/kernel/process.c > @@ -78,7 +78,6 @@ void arch_cpu_idle(void) > arm_pm_idle(); > else > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_prepare(void) > --- a/arch/arm/mach-gemini/board-dt.c > +++ b/arch/arm/mach-gemini/board-dt.c > @@ -42,8 +42,9 @@ static void gemini_idle(void) > */ > > /* FIXME: Enabling interrupts here is racy! */ > - local_irq_enable(); > + raw_local_irq_enable(); > cpu_do_idle(); > + raw_local_irq_disable(); > } > > static void __init gemini_init_machine(void) > --- a/arch/arm64/kernel/idle.c > +++ b/arch/arm64/kernel/idle.c > @@ -42,5 +42,4 @@ void noinstr arch_cpu_idle(void) > * tricks > */ > cpu_do_idle(); > - raw_local_irq_enable(); > } > --- a/arch/csky/kernel/process.c > +++ b/arch/csky/kernel/process.c > @@ -101,6 +101,5 @@ void arch_cpu_idle(void) > #ifdef CONFIG_CPU_PM_STOP > asm volatile("stop\n"); > #endif > - raw_local_irq_enable(); > } > #endif > --- a/arch/csky/kernel/smp.c > +++ b/arch/csky/kernel/smp.c > @@ -314,7 +314,7 @@ void arch_cpu_idle_dead(void) > while (!secondary_stack) > arch_cpu_idle(); > > - local_irq_disable(); > + raw_local_irq_disable(); > > asm volatile( > "mov sp, %0\n" > --- a/arch/hexagon/kernel/process.c > +++ b/arch/hexagon/kernel/process.c > @@ -44,7 +44,6 @@ void arch_cpu_idle(void) > { > __vmwait(); > /* interrupts wake us up, but irqs are still disabled */ > - raw_local_irq_enable(); > } > > /* > --- a/arch/ia64/kernel/process.c > +++ b/arch/ia64/kernel/process.c > @@ -241,6 +241,7 @@ void arch_cpu_idle(void) > (*mark_idle)(1); > > raw_safe_halt(); > + raw_local_irq_disable(); > > if (mark_idle) > (*mark_idle)(0); > --- a/arch/microblaze/kernel/process.c > +++ b/arch/microblaze/kernel/process.c > @@ -138,5 +138,4 @@ int dump_fpu(struct pt_regs *regs, elf_f > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > --- a/arch/mips/kernel/idle.c > +++ b/arch/mips/kernel/idle.c > @@ -33,13 +33,13 @@ static void __cpuidle r3081_wait(void) > { > unsigned long cfg = read_c0_conf(); > write_c0_conf(cfg | R30XX_CONF_HALT); > - raw_local_irq_enable(); > } > > void __cpuidle r4k_wait(void) > { > raw_local_irq_enable(); > __r4k_wait(); > + raw_local_irq_disable(); > } > > /* > @@ -57,7 +57,6 @@ void __cpuidle r4k_wait_irqoff(void) > " .set arch=r4000 \n" > " wait \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -77,7 +76,6 @@ static void __cpuidle rm7k_wait_irqoff(v > " wait \n" > " mtc0 $1, $12 # stalls until W stage \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -103,6 +101,8 @@ static void __cpuidle au1k_wait(void) > " nop \n" > " .set pop \n" > : : "r" (au1k_wait), "r" (c0status)); > + > + raw_local_irq_disable(); > } > > static int __initdata nowait; > @@ -245,8 +245,6 @@ void arch_cpu_idle(void) > { > if (cpu_wait) > cpu_wait(); > - else > - raw_local_irq_enable(); > } > > #ifdef CONFIG_CPU_IDLE > --- a/arch/nios2/kernel/process.c > +++ b/arch/nios2/kernel/process.c > @@ -33,7 +33,6 @@ EXPORT_SYMBOL(pm_power_off); > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > > /* > --- a/arch/openrisc/kernel/process.c > +++ b/arch/openrisc/kernel/process.c > @@ -102,6 +102,7 @@ void arch_cpu_idle(void) > raw_local_irq_enable(); > if (mfspr(SPR_UPR) & SPR_UPR_PMP) > mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME); > + raw_local_irq_disable(); > } > > void (*pm_power_off)(void) = NULL; > --- a/arch/parisc/kernel/process.c > +++ b/arch/parisc/kernel/process.c > @@ -187,8 +187,6 @@ void arch_cpu_idle_dead(void) > > void __cpuidle arch_cpu_idle(void) > { > - raw_local_irq_enable(); > - > /* nop on real hardware, qemu will idle sleep. */ > asm volatile("or %%r10,%%r10,%%r10\n":::); > } > --- a/arch/powerpc/kernel/idle.c > +++ b/arch/powerpc/kernel/idle.c > @@ -51,10 +51,9 @@ void arch_cpu_idle(void) > * Some power_save functions return with > * interrupts enabled, some don't. > */ > - if (irqs_disabled()) > - raw_local_irq_enable(); > + if (!irqs_disabled()) > + raw_local_irq_disable(); > } else { > - raw_local_irq_enable(); > /* > * Go into low thread priority and possibly > * low power mode. > --- a/arch/riscv/kernel/process.c > +++ b/arch/riscv/kernel/process.c > @@ -39,7 +39,6 @@ extern asmlinkage void ret_from_kernel_t > void arch_cpu_idle(void) > { > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void __show_regs(struct pt_regs *regs) > --- a/arch/s390/kernel/idle.c > +++ b/arch/s390/kernel/idle.c > @@ -66,7 +66,6 @@ void arch_cpu_idle(void) > idle->idle_count++; > account_idle_time(cputime_to_nsecs(idle_time)); > raw_write_seqcount_end(&idle->seqcount); > - raw_local_irq_enable(); > } > > static ssize_t show_idle_count(struct device *dev, > --- a/arch/sh/kernel/idle.c > +++ b/arch/sh/kernel/idle.c > @@ -25,6 +25,7 @@ void default_idle(void) > raw_local_irq_enable(); > /* Isn't this racy ? */ > cpu_sleep(); > + raw_local_irq_disable(); > clear_bl_bit(); > } > > --- a/arch/sparc/kernel/leon_pmc.c > +++ b/arch/sparc/kernel/leon_pmc.c > @@ -57,6 +57,8 @@ static void pmc_leon_idle_fixup(void) > "lda [%0] %1, %%g0\n" > : > : "r"(address), "i"(ASI_LEON_BYPASS)); > + > + raw_local_irq_disable(); > } > > /* > @@ -70,6 +72,8 @@ static void pmc_leon_idle(void) > > /* For systems without power-down, this will be no-op */ > __asm__ __volatile__ ("wr %g0, %asr19\n\t"); > + > + raw_local_irq_disable(); > } > > /* Install LEON Power Down function */ > --- a/arch/sparc/kernel/process_32.c > +++ b/arch/sparc/kernel/process_32.c > @@ -71,7 +71,6 @@ void arch_cpu_idle(void) > { > if (sparc_idle) > (*sparc_idle)(); > - raw_local_irq_enable(); > } > > /* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */ > --- a/arch/sparc/kernel/process_64.c > +++ b/arch/sparc/kernel/process_64.c > @@ -59,7 +59,6 @@ void arch_cpu_idle(void) > { > if (tlb_type != hypervisor) { > touch_nmi_watchdog(); > - raw_local_irq_enable(); > } else { > unsigned long pstate; > > @@ -90,6 +89,8 @@ void arch_cpu_idle(void) > "wrpr %0, %%g0, %%pstate" > : "=&r" (pstate) > : "i" (PSTATE_IE)); > + > + raw_local_irq_disable(); > } > } > > --- a/arch/um/kernel/process.c > +++ b/arch/um/kernel/process.c > @@ -216,7 +216,6 @@ void arch_cpu_idle(void) > { > cpu_tasks[current_thread_info()->cpu].pid = os_getpid(); > um_idle_sleep(); > - raw_local_irq_enable(); > } > > int __cant_sleep(void) { > --- a/arch/x86/coco/tdx/tdx.c > +++ b/arch/x86/coco/tdx/tdx.c > @@ -178,6 +178,9 @@ void __cpuidle tdx_safe_halt(void) > */ > if (__halt(irq_disabled, do_sti)) > WARN_ONCE(1, "HLT instruction emulation failed\n"); > + > + /* XXX I can't make sense of what @do_sti actually does */ > + raw_local_irq_disable(); > } > > static bool read_msr(struct pt_regs *regs) > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -699,6 +699,7 @@ EXPORT_SYMBOL(boot_option_idle_override) > void __cpuidle default_idle(void) > { > raw_safe_halt(); > + raw_local_irq_disable(); > } > #if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE) > EXPORT_SYMBOL(default_idle); > @@ -804,13 +805,7 @@ static void amd_e400_idle(void) > > default_idle(); > > - /* > - * The switch back from broadcast mode needs to be called with > - * interrupts disabled. > - */ > - raw_local_irq_disable(); > tick_broadcast_exit(); > - raw_local_irq_enable(); > } > > /* > @@ -849,12 +844,10 @@ static __cpuidle void mwait_idle(void) > } > > __monitor((void *)¤t_thread_info()->flags, 0, 0); > - if (!need_resched()) > + if (!need_resched()) { > __sti_mwait(0, 0); > - else > - raw_local_irq_enable(); > - } else { > - raw_local_irq_enable(); > + raw_local_irq_disable(); > + } > } > __current_clr_polling(); > } > --- a/arch/xtensa/kernel/process.c > +++ b/arch/xtensa/kernel/process.c > @@ -183,6 +183,7 @@ void coprocessor_flush_release_all(struc > void arch_cpu_idle(void) > { > platform_idle(); > + raw_local_irq_disable(); > } > > /* > --- a/kernel/sched/idle.c > +++ b/kernel/sched/idle.c > @@ -79,7 +79,6 @@ void __weak arch_cpu_idle_dead(void) { } > void __weak arch_cpu_idle(void) > { > cpu_idle_force_poll = 1; > - raw_local_irq_enable(); > } > > /** > @@ -96,7 +95,6 @@ void __cpuidle default_idle_call(void) > > cpuidle_rcu_enter(); > arch_cpu_idle(); > - raw_local_irq_disable(); > cpuidle_rcu_exit(); > > start_critical_timings(); > > _______________________________________________ linux-snps-arc mailing list linux-snps-arc@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-snps-arc From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0646FCCA488 for ; Wed, 6 Jul 2022 14:03:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 74BAF4091D; Wed, 6 Jul 2022 14:03:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 74BAF4091D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RLuAW5Kg745n; Wed, 6 Jul 2022 14:03:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 241374033C; Wed, 6 Jul 2022 14:03:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 241374033C Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E2EA3C0035; Wed, 6 Jul 2022 14:03:07 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 71A27C002D for ; Wed, 6 Jul 2022 14:03:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 38F7B40B33 for ; Wed, 6 Jul 2022 14:03:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 38F7B40B33 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XAOOXbBfROdj for ; Wed, 6 Jul 2022 14:03:05 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 1384140AF6 Received: from mail-yw1-f174.google.com (mail-yw1-f174.google.com [209.85.128.174]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1384140AF6 for ; Wed, 6 Jul 2022 14:03:05 +0000 (UTC) Received: by mail-yw1-f174.google.com with SMTP id 00721157ae682-31cf1adbf92so14147607b3.4 for ; Wed, 06 Jul 2022 07:03:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WBBLwGm3VvIAnm4uSuSyiWShri4w+bgQrIyP7HIMtP0=; b=3Rxf7We9ai+GkWsEpXkoQjqQWa9+sYq/OAmYN97vGxuxw+k2jN0O0BEq3vjtNkf8ra RrxF+ynrozUh4A+4poPMZOE3EqwWS+BSqhQaaFgSDXwFVWtM4dtlE0/8rmBOz8scOUoE JZogDD2nDPPwN9p1azBYIIfMNWQTe3YogKs3kpy09Fp2rWvmUW2PMeG1PZ67C98WC/Rk YZ7gqW5VU7WUrJlAQI2MrFxF/UBDK/QcE5tLLB4q9YNXB9iLX2BG0EmRFPzJ/7PxhIx9 71V7mZpuZYVtvOAWzKLItsVe/M2/Qa9gANLmfVI1xL0sD9LcnZ57IITMuUSsUxKCZsMk +pQw== X-Gm-Message-State: AJIora/iOecKEvXkIhN6Yw8zUA98R16vIN7LtwYk3yFxorKWJCirf4+T LPceZDy5Jy2VR7OZDPyatQ5kAbpyTXoL/80YAs0= X-Google-Smtp-Source: AGRyM1shXjwii2Xu/qgQ1B1HadHYSMjqitGmbFlHxf4P7RndZwyBhPC2CaCRixiXqBuDMvjO7JXHYGnoD2iVhXDPhek= X-Received: by 2002:a81:a184:0:b0:31c:b00e:b5c4 with SMTP id y126-20020a81a184000000b0031cb00eb5c4mr15058481ywg.149.1657116183677; Wed, 06 Jul 2022 07:03:03 -0700 (PDT) MIME-Version: 1.0 References: <20220608142723.103523089@infradead.org> <20220608144517.188449351@infradead.org> In-Reply-To: <20220608144517.188449351@infradead.org> From: "Rafael J. Wysocki" Date: Wed, 6 Jul 2022 16:02:52 +0200 Message-ID: Subject: Re: [PATCH 20/36] arch/idle: Change arch_cpu_idle() IRQ behaviour To: Peter Zijlstra Cc: Juri Lelli , "Rafael J. Wysocki" , Benjamin Herrenschmidt , Linus Walleij , Benjamin Segall , Guo Ren , Pavel Machek , Alexander Gordeev , linux-arch , Vincent Guittot , Michael Ellerman , Huacai Chen , ACPI Devel Maling List , Andy Gross , Geert Uytterhoeven , dl-linux-imx , Catalin Marinas , xen-devel@lists.xenproject.org, mattst88@gmail.com, Michael Turquette , sammy@sammy.net, Petr Mladek , Linux PM , Lai Jiangshan , Sascha Hauer , linux-um@lists.infradead.org, acme@kernel.org, Thomas Gleixner , Linux OMAP Mailing List , Dietmar Eggemann , rth@twiddle.net, Greg Kroah-Hartman , Linux Kernel Mailing List , linux-perf-users@vger.kernel.org, senozhatsky@chromium.org, Sven Schnelle , jolsa@kernel.org, Paul Mackerras , Mark Rutland , linux-ia64@vger.kernel.org, Dave Hansen , virtualization@lists.linux-foundation.org, James Bottomley , jcmvbkbc@gmail.com, Thierry Reding , kernel@xen0n.name, quic_neeraju@quicinc.com, linux-s390@vger.kernel.org, vschneid@redhat.com, John Ogness , Yoshinori Sato , Linux-sh list , Fabio Estevam , Helge Deller , Daniel Lezcano , Jon Hunter , Mathieu Desnoyers , Frederic Weisbecker , Len Brown , linux-xtensa@linux-xtensa.org, Sascha Hauer , Vasily Gorbik , linux-arm-msm , linux-alpha@vger.kernel.org, linux-m68k , Stafford Horne , Linux ARM , Chris Zankel , Stephen Boyd , dinguyen@kernel.org, Daniel Bristot de Oliveira , Alexander Shishkin , Lorenzo Pieralisi , Rasmus Villemoes , Joel Fernandes , Will Deacon , Boris Ostrovsky , Kevin Hilman , linux-csky@vger.kernel.org, pv-drivers@vmware.com, linux-snps-arc@lists.infradead.org, Mel Gorman , Jacob Pan , Arnd Bergmann , ulli.kroll@googlemail.com, vgupta@kernel.org, linux-clk , Josh Triplett , Steven Rostedt , rcu@vger.kernel.org, Borislav Petkov , bcain@quicinc.com, Thomas Bogendoerfer , Parisc List , Sudeep Holla , Shawn Guo , David Miller , Rich Felker , Tony Lindgren , amakhalov@vmware.com, Bjorn Andersson , "H. Peter Anvin" , sparclinux@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-riscv , anton.ivanov@cambridgegreys.com, jonas@southpole.se, Yury Norov , Richard Weinberger , the arch/x86 maintainers , Russell King - ARM Linux , Ingo Molnar , Albert Ou , "Paul E. McKenney" , Heiko Carstens , stefan.kristiansson@saunalahti.fi, openrisc@lists.librecores.org, Paul Walmsley , linux-tegra , namhyung@kernel.org, Andy Shevchenko , jpoimboe@kernel.org, Juergen Gross , Michal Simek , "open list:BROADCOM NVRAM DRIVER" , Palmer Dabbelt , Anup Patel , ink@jurassic.park.msu.ru, Johannes Berg , linuxppc-dev X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" On Wed, Jun 8, 2022 at 4:46 PM Peter Zijlstra wrote: > > Current arch_cpu_idle() is called with IRQs disabled, but will return > with IRQs enabled. > > However, the very first thing the generic code does after calling > arch_cpu_idle() is raw_local_irq_disable(). This means that > architectures that can idle with IRQs disabled end up doing a > pointless 'enable-disable' dance. > > Therefore, push this IRQ disabling into the idle function, meaning > that those architectures can avoid the pointless IRQ state flipping. > > Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki > --- > arch/alpha/kernel/process.c | 1 - > arch/arc/kernel/process.c | 3 +++ > arch/arm/kernel/process.c | 1 - > arch/arm/mach-gemini/board-dt.c | 3 ++- > arch/arm64/kernel/idle.c | 1 - > arch/csky/kernel/process.c | 1 - > arch/csky/kernel/smp.c | 2 +- > arch/hexagon/kernel/process.c | 1 - > arch/ia64/kernel/process.c | 1 + > arch/microblaze/kernel/process.c | 1 - > arch/mips/kernel/idle.c | 8 +++----- > arch/nios2/kernel/process.c | 1 - > arch/openrisc/kernel/process.c | 1 + > arch/parisc/kernel/process.c | 2 -- > arch/powerpc/kernel/idle.c | 5 ++--- > arch/riscv/kernel/process.c | 1 - > arch/s390/kernel/idle.c | 1 - > arch/sh/kernel/idle.c | 1 + > arch/sparc/kernel/leon_pmc.c | 4 ++++ > arch/sparc/kernel/process_32.c | 1 - > arch/sparc/kernel/process_64.c | 3 ++- > arch/um/kernel/process.c | 1 - > arch/x86/coco/tdx/tdx.c | 3 +++ > arch/x86/kernel/process.c | 15 ++++----------- > arch/xtensa/kernel/process.c | 1 + > kernel/sched/idle.c | 2 -- > 26 files changed, 28 insertions(+), 37 deletions(-) > > --- a/arch/alpha/kernel/process.c > +++ b/arch/alpha/kernel/process.c > @@ -57,7 +57,6 @@ EXPORT_SYMBOL(pm_power_off); > void arch_cpu_idle(void) > { > wtint(0); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_dead(void) > --- a/arch/arc/kernel/process.c > +++ b/arch/arc/kernel/process.c > @@ -114,6 +114,8 @@ void arch_cpu_idle(void) > "sleep %0 \n" > : > :"I"(arg)); /* can't be "r" has to be embedded const */ > + > + raw_local_irq_disable(); > } > > #else /* ARC700 */ > @@ -122,6 +124,7 @@ void arch_cpu_idle(void) > { > /* sleep, but enable both set E1/E2 (levels of interrupts) before committing */ > __asm__ __volatile__("sleep 0x3 \n"); > + raw_local_irq_disable(); > } > > #endif > --- a/arch/arm/kernel/process.c > +++ b/arch/arm/kernel/process.c > @@ -78,7 +78,6 @@ void arch_cpu_idle(void) > arm_pm_idle(); > else > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_prepare(void) > --- a/arch/arm/mach-gemini/board-dt.c > +++ b/arch/arm/mach-gemini/board-dt.c > @@ -42,8 +42,9 @@ static void gemini_idle(void) > */ > > /* FIXME: Enabling interrupts here is racy! */ > - local_irq_enable(); > + raw_local_irq_enable(); > cpu_do_idle(); > + raw_local_irq_disable(); > } > > static void __init gemini_init_machine(void) > --- a/arch/arm64/kernel/idle.c > +++ b/arch/arm64/kernel/idle.c > @@ -42,5 +42,4 @@ void noinstr arch_cpu_idle(void) > * tricks > */ > cpu_do_idle(); > - raw_local_irq_enable(); > } > --- a/arch/csky/kernel/process.c > +++ b/arch/csky/kernel/process.c > @@ -101,6 +101,5 @@ void arch_cpu_idle(void) > #ifdef CONFIG_CPU_PM_STOP > asm volatile("stop\n"); > #endif > - raw_local_irq_enable(); > } > #endif > --- a/arch/csky/kernel/smp.c > +++ b/arch/csky/kernel/smp.c > @@ -314,7 +314,7 @@ void arch_cpu_idle_dead(void) > while (!secondary_stack) > arch_cpu_idle(); > > - local_irq_disable(); > + raw_local_irq_disable(); > > asm volatile( > "mov sp, %0\n" > --- a/arch/hexagon/kernel/process.c > +++ b/arch/hexagon/kernel/process.c > @@ -44,7 +44,6 @@ void arch_cpu_idle(void) > { > __vmwait(); > /* interrupts wake us up, but irqs are still disabled */ > - raw_local_irq_enable(); > } > > /* > --- a/arch/ia64/kernel/process.c > +++ b/arch/ia64/kernel/process.c > @@ -241,6 +241,7 @@ void arch_cpu_idle(void) > (*mark_idle)(1); > > raw_safe_halt(); > + raw_local_irq_disable(); > > if (mark_idle) > (*mark_idle)(0); > --- a/arch/microblaze/kernel/process.c > +++ b/arch/microblaze/kernel/process.c > @@ -138,5 +138,4 @@ int dump_fpu(struct pt_regs *regs, elf_f > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > --- a/arch/mips/kernel/idle.c > +++ b/arch/mips/kernel/idle.c > @@ -33,13 +33,13 @@ static void __cpuidle r3081_wait(void) > { > unsigned long cfg = read_c0_conf(); > write_c0_conf(cfg | R30XX_CONF_HALT); > - raw_local_irq_enable(); > } > > void __cpuidle r4k_wait(void) > { > raw_local_irq_enable(); > __r4k_wait(); > + raw_local_irq_disable(); > } > > /* > @@ -57,7 +57,6 @@ void __cpuidle r4k_wait_irqoff(void) > " .set arch=r4000 \n" > " wait \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -77,7 +76,6 @@ static void __cpuidle rm7k_wait_irqoff(v > " wait \n" > " mtc0 $1, $12 # stalls until W stage \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -103,6 +101,8 @@ static void __cpuidle au1k_wait(void) > " nop \n" > " .set pop \n" > : : "r" (au1k_wait), "r" (c0status)); > + > + raw_local_irq_disable(); > } > > static int __initdata nowait; > @@ -245,8 +245,6 @@ void arch_cpu_idle(void) > { > if (cpu_wait) > cpu_wait(); > - else > - raw_local_irq_enable(); > } > > #ifdef CONFIG_CPU_IDLE > --- a/arch/nios2/kernel/process.c > +++ b/arch/nios2/kernel/process.c > @@ -33,7 +33,6 @@ EXPORT_SYMBOL(pm_power_off); > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > > /* > --- a/arch/openrisc/kernel/process.c > +++ b/arch/openrisc/kernel/process.c > @@ -102,6 +102,7 @@ void arch_cpu_idle(void) > raw_local_irq_enable(); > if (mfspr(SPR_UPR) & SPR_UPR_PMP) > mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME); > + raw_local_irq_disable(); > } > > void (*pm_power_off)(void) = NULL; > --- a/arch/parisc/kernel/process.c > +++ b/arch/parisc/kernel/process.c > @@ -187,8 +187,6 @@ void arch_cpu_idle_dead(void) > > void __cpuidle arch_cpu_idle(void) > { > - raw_local_irq_enable(); > - > /* nop on real hardware, qemu will idle sleep. */ > asm volatile("or %%r10,%%r10,%%r10\n":::); > } > --- a/arch/powerpc/kernel/idle.c > +++ b/arch/powerpc/kernel/idle.c > @@ -51,10 +51,9 @@ void arch_cpu_idle(void) > * Some power_save functions return with > * interrupts enabled, some don't. > */ > - if (irqs_disabled()) > - raw_local_irq_enable(); > + if (!irqs_disabled()) > + raw_local_irq_disable(); > } else { > - raw_local_irq_enable(); > /* > * Go into low thread priority and possibly > * low power mode. > --- a/arch/riscv/kernel/process.c > +++ b/arch/riscv/kernel/process.c > @@ -39,7 +39,6 @@ extern asmlinkage void ret_from_kernel_t > void arch_cpu_idle(void) > { > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void __show_regs(struct pt_regs *regs) > --- a/arch/s390/kernel/idle.c > +++ b/arch/s390/kernel/idle.c > @@ -66,7 +66,6 @@ void arch_cpu_idle(void) > idle->idle_count++; > account_idle_time(cputime_to_nsecs(idle_time)); > raw_write_seqcount_end(&idle->seqcount); > - raw_local_irq_enable(); > } > > static ssize_t show_idle_count(struct device *dev, > --- a/arch/sh/kernel/idle.c > +++ b/arch/sh/kernel/idle.c > @@ -25,6 +25,7 @@ void default_idle(void) > raw_local_irq_enable(); > /* Isn't this racy ? */ > cpu_sleep(); > + raw_local_irq_disable(); > clear_bl_bit(); > } > > --- a/arch/sparc/kernel/leon_pmc.c > +++ b/arch/sparc/kernel/leon_pmc.c > @@ -57,6 +57,8 @@ static void pmc_leon_idle_fixup(void) > "lda [%0] %1, %%g0\n" > : > : "r"(address), "i"(ASI_LEON_BYPASS)); > + > + raw_local_irq_disable(); > } > > /* > @@ -70,6 +72,8 @@ static void pmc_leon_idle(void) > > /* For systems without power-down, this will be no-op */ > __asm__ __volatile__ ("wr %g0, %asr19\n\t"); > + > + raw_local_irq_disable(); > } > > /* Install LEON Power Down function */ > --- a/arch/sparc/kernel/process_32.c > +++ b/arch/sparc/kernel/process_32.c > @@ -71,7 +71,6 @@ void arch_cpu_idle(void) > { > if (sparc_idle) > (*sparc_idle)(); > - raw_local_irq_enable(); > } > > /* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */ > --- a/arch/sparc/kernel/process_64.c > +++ b/arch/sparc/kernel/process_64.c > @@ -59,7 +59,6 @@ void arch_cpu_idle(void) > { > if (tlb_type != hypervisor) { > touch_nmi_watchdog(); > - raw_local_irq_enable(); > } else { > unsigned long pstate; > > @@ -90,6 +89,8 @@ void arch_cpu_idle(void) > "wrpr %0, %%g0, %%pstate" > : "=&r" (pstate) > : "i" (PSTATE_IE)); > + > + raw_local_irq_disable(); > } > } > > --- a/arch/um/kernel/process.c > +++ b/arch/um/kernel/process.c > @@ -216,7 +216,6 @@ void arch_cpu_idle(void) > { > cpu_tasks[current_thread_info()->cpu].pid = os_getpid(); > um_idle_sleep(); > - raw_local_irq_enable(); > } > > int __cant_sleep(void) { > --- a/arch/x86/coco/tdx/tdx.c > +++ b/arch/x86/coco/tdx/tdx.c > @@ -178,6 +178,9 @@ void __cpuidle tdx_safe_halt(void) > */ > if (__halt(irq_disabled, do_sti)) > WARN_ONCE(1, "HLT instruction emulation failed\n"); > + > + /* XXX I can't make sense of what @do_sti actually does */ > + raw_local_irq_disable(); > } > > static bool read_msr(struct pt_regs *regs) > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -699,6 +699,7 @@ EXPORT_SYMBOL(boot_option_idle_override) > void __cpuidle default_idle(void) > { > raw_safe_halt(); > + raw_local_irq_disable(); > } > #if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE) > EXPORT_SYMBOL(default_idle); > @@ -804,13 +805,7 @@ static void amd_e400_idle(void) > > default_idle(); > > - /* > - * The switch back from broadcast mode needs to be called with > - * interrupts disabled. > - */ > - raw_local_irq_disable(); > tick_broadcast_exit(); > - raw_local_irq_enable(); > } > > /* > @@ -849,12 +844,10 @@ static __cpuidle void mwait_idle(void) > } > > __monitor((void *)¤t_thread_info()->flags, 0, 0); > - if (!need_resched()) > + if (!need_resched()) { > __sti_mwait(0, 0); > - else > - raw_local_irq_enable(); > - } else { > - raw_local_irq_enable(); > + raw_local_irq_disable(); > + } > } > __current_clr_polling(); > } > --- a/arch/xtensa/kernel/process.c > +++ b/arch/xtensa/kernel/process.c > @@ -183,6 +183,7 @@ void coprocessor_flush_release_all(struc > void arch_cpu_idle(void) > { > platform_idle(); > + raw_local_irq_disable(); > } > > /* > --- a/kernel/sched/idle.c > +++ b/kernel/sched/idle.c > @@ -79,7 +79,6 @@ void __weak arch_cpu_idle_dead(void) { } > void __weak arch_cpu_idle(void) > { > cpu_idle_force_poll = 1; > - raw_local_irq_enable(); > } > > /** > @@ -96,7 +95,6 @@ void __cpuidle default_idle_call(void) > > cpuidle_rcu_enter(); > arch_cpu_idle(); > - raw_local_irq_disable(); > cpuidle_rcu_exit(); > > start_critical_timings(); > > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF6E6C433EF for ; Wed, 6 Jul 2022 14:05:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2S3Xlu6PJoNGw0KpZcc1NTQouJunOc8omU4wlmjifAE=; b=Oqm37eXBMYgJIO Y8CwZYCmpYdHMmLZz+AMjOIirUnl+Xw+JHgl+D7QXDXxPGx51aaDvW6coX/HL/8fZWqNxp5bobv8v EgC47tfeazioP/y5PHM1NKZQkRnktc+1pgb18TSg/CbrGClUMYX197Mrlx5lG0p9L42gFcpca9Pyv 4KxqA7SvVC9QxHOfFqdvvxbDK3Dywy8f7MK8TMTniT/c8Q3mQHt2BEQsqaZ4kWiZsHVqeCVmKl1ra gQQvMR7cYEe4tUd1euPmyigqoOKupJdIM+qQTbnplOeHEB2xkqVE6vKRzEyI+vPgaqoJUEglidjxg x/HSvzzDFKLgDzFE9vWQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o95eh-00AWX6-3J; Wed, 06 Jul 2022 14:05:27 +0000 Received: from mail-yw1-f182.google.com ([209.85.128.182]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o95cP-00AVvR-Q0; Wed, 06 Jul 2022 14:03:08 +0000 Received: by mail-yw1-f182.google.com with SMTP id 00721157ae682-31c89653790so89902857b3.13; Wed, 06 Jul 2022 07:03:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WBBLwGm3VvIAnm4uSuSyiWShri4w+bgQrIyP7HIMtP0=; b=Suv+XS600hsJ/7EeZpN6J0ljoxDWvEGyq88jk6AgUPfWp9rDk8lD8kdLeFi8EphBgr vs1jh3B17JQAs5ryHYdBKuXYsvHgVg/NTAJg8s3Fs/9U/sb8yU1OimiJy2WVreh9I/Cx pU48rDD1RTzV2HfDPzGc6Vl0l7NU4ZZRi331hcg6bkuJoLRL/K4+X1F0N3mn26KFuwYQ Y7JooYbLrO9bQlXMq73L10gaSlfYt9c02V7n2foTVerus61dm04jIrr5LsxfRp5qHr3w luLvRvjJuip5VLoj7OsLiV5dnPCbAhkyF7Zi3dFx8lmOPdLDMjKGJo/z16HRa2VAhZ3g 4m3Q== X-Gm-Message-State: AJIora/wBO5RdK5Gm1R2L26VfxJx8pWrWKWb7AmGW/LhrzSrGP4rnxwJ tM7no6bRLcZsS0+zE/S31e6KPdWhlLBcHZsYyfU= X-Google-Smtp-Source: AGRyM1shXjwii2Xu/qgQ1B1HadHYSMjqitGmbFlHxf4P7RndZwyBhPC2CaCRixiXqBuDMvjO7JXHYGnoD2iVhXDPhek= X-Received: by 2002:a81:a184:0:b0:31c:b00e:b5c4 with SMTP id y126-20020a81a184000000b0031cb00eb5c4mr15058481ywg.149.1657116183677; Wed, 06 Jul 2022 07:03:03 -0700 (PDT) MIME-Version: 1.0 References: <20220608142723.103523089@infradead.org> <20220608144517.188449351@infradead.org> In-Reply-To: <20220608144517.188449351@infradead.org> From: "Rafael J. Wysocki" Date: Wed, 6 Jul 2022 16:02:52 +0200 Message-ID: Subject: Re: [PATCH 20/36] arch/idle: Change arch_cpu_idle() IRQ behaviour To: Peter Zijlstra Cc: rth@twiddle.net, ink@jurassic.park.msu.ru, mattst88@gmail.com, vgupta@kernel.org, Russell King - ARM Linux , ulli.kroll@googlemail.com, Linus Walleij , Shawn Guo , Sascha Hauer , Sascha Hauer , Fabio Estevam , dl-linux-imx , Tony Lindgren , Kevin Hilman , Catalin Marinas , Will Deacon , Guo Ren , bcain@quicinc.com, Huacai Chen , kernel@xen0n.name, Geert Uytterhoeven , sammy@sammy.net, Michal Simek , Thomas Bogendoerfer , dinguyen@kernel.org, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Stafford Horne , James Bottomley , Helge Deller , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Yoshinori Sato , Rich Felker , David Miller , Richard Weinberger , anton.ivanov@cambridgegreys.com, Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "the arch/x86 maintainers" , "H. Peter Anvin" , acme@kernel.org, Mark Rutland , Alexander Shishkin , jolsa@kernel.org, namhyung@kernel.org, Juergen Gross , srivatsa@csail.mit.edu, amakhalov@vmware.com, pv-drivers@vmware.com, Boris Ostrovsky , Chris Zankel , jcmvbkbc@gmail.com, "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Michael Turquette , Stephen Boyd , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , Andy Gross , Bjorn Andersson , Anup Patel , Thierry Reding , Jon Hunter , Jacob Pan , Arnd Bergmann , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Steven Rostedt , Petr Mladek , senozhatsky@chromium.org, John Ogness , "Paul E. McKenney" , Frederic Weisbecker , quic_neeraju@quicinc.com, Josh Triplett , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Benjamin Segall , Mel Gorman , Daniel Bristot de Oliveira , vschneid@redhat.com, jpoimboe@kernel.org, linux-alpha@vger.kernel.org, Linux Kernel Mailing List , linux-snps-arc@lists.infradead.org, Linux ARM , Linux OMAP Mailing List , linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, linux-m68k , "open list:BROADCOM NVRAM DRIVER" , openrisc@lists.librecores.org, Parisc List , linuxppc-dev , linux-riscv , linux-s390@vger.kernel.org, Linux-sh list , sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-perf-users@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, linux-xtensa@linux-xtensa.org, ACPI Devel Maling List , Linux PM , linux-clk , linux-arm-msm , linux-tegra , linux-arch , rcu@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220706_070305_879188_2C952C0E X-CRM114-Status: GOOD ( 30.55 ) X-Mailman-Approved-At: Wed, 06 Jul 2022 07:05:16 -0700 X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On Wed, Jun 8, 2022 at 4:46 PM Peter Zijlstra wrote: > > Current arch_cpu_idle() is called with IRQs disabled, but will return > with IRQs enabled. > > However, the very first thing the generic code does after calling > arch_cpu_idle() is raw_local_irq_disable(). This means that > architectures that can idle with IRQs disabled end up doing a > pointless 'enable-disable' dance. > > Therefore, push this IRQ disabling into the idle function, meaning > that those architectures can avoid the pointless IRQ state flipping. > > Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki > --- > arch/alpha/kernel/process.c | 1 - > arch/arc/kernel/process.c | 3 +++ > arch/arm/kernel/process.c | 1 - > arch/arm/mach-gemini/board-dt.c | 3 ++- > arch/arm64/kernel/idle.c | 1 - > arch/csky/kernel/process.c | 1 - > arch/csky/kernel/smp.c | 2 +- > arch/hexagon/kernel/process.c | 1 - > arch/ia64/kernel/process.c | 1 + > arch/microblaze/kernel/process.c | 1 - > arch/mips/kernel/idle.c | 8 +++----- > arch/nios2/kernel/process.c | 1 - > arch/openrisc/kernel/process.c | 1 + > arch/parisc/kernel/process.c | 2 -- > arch/powerpc/kernel/idle.c | 5 ++--- > arch/riscv/kernel/process.c | 1 - > arch/s390/kernel/idle.c | 1 - > arch/sh/kernel/idle.c | 1 + > arch/sparc/kernel/leon_pmc.c | 4 ++++ > arch/sparc/kernel/process_32.c | 1 - > arch/sparc/kernel/process_64.c | 3 ++- > arch/um/kernel/process.c | 1 - > arch/x86/coco/tdx/tdx.c | 3 +++ > arch/x86/kernel/process.c | 15 ++++----------- > arch/xtensa/kernel/process.c | 1 + > kernel/sched/idle.c | 2 -- > 26 files changed, 28 insertions(+), 37 deletions(-) > > --- a/arch/alpha/kernel/process.c > +++ b/arch/alpha/kernel/process.c > @@ -57,7 +57,6 @@ EXPORT_SYMBOL(pm_power_off); > void arch_cpu_idle(void) > { > wtint(0); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_dead(void) > --- a/arch/arc/kernel/process.c > +++ b/arch/arc/kernel/process.c > @@ -114,6 +114,8 @@ void arch_cpu_idle(void) > "sleep %0 \n" > : > :"I"(arg)); /* can't be "r" has to be embedded const */ > + > + raw_local_irq_disable(); > } > > #else /* ARC700 */ > @@ -122,6 +124,7 @@ void arch_cpu_idle(void) > { > /* sleep, but enable both set E1/E2 (levels of interrupts) before committing */ > __asm__ __volatile__("sleep 0x3 \n"); > + raw_local_irq_disable(); > } > > #endif > --- a/arch/arm/kernel/process.c > +++ b/arch/arm/kernel/process.c > @@ -78,7 +78,6 @@ void arch_cpu_idle(void) > arm_pm_idle(); > else > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_prepare(void) > --- a/arch/arm/mach-gemini/board-dt.c > +++ b/arch/arm/mach-gemini/board-dt.c > @@ -42,8 +42,9 @@ static void gemini_idle(void) > */ > > /* FIXME: Enabling interrupts here is racy! */ > - local_irq_enable(); > + raw_local_irq_enable(); > cpu_do_idle(); > + raw_local_irq_disable(); > } > > static void __init gemini_init_machine(void) > --- a/arch/arm64/kernel/idle.c > +++ b/arch/arm64/kernel/idle.c > @@ -42,5 +42,4 @@ void noinstr arch_cpu_idle(void) > * tricks > */ > cpu_do_idle(); > - raw_local_irq_enable(); > } > --- a/arch/csky/kernel/process.c > +++ b/arch/csky/kernel/process.c > @@ -101,6 +101,5 @@ void arch_cpu_idle(void) > #ifdef CONFIG_CPU_PM_STOP > asm volatile("stop\n"); > #endif > - raw_local_irq_enable(); > } > #endif > --- a/arch/csky/kernel/smp.c > +++ b/arch/csky/kernel/smp.c > @@ -314,7 +314,7 @@ void arch_cpu_idle_dead(void) > while (!secondary_stack) > arch_cpu_idle(); > > - local_irq_disable(); > + raw_local_irq_disable(); > > asm volatile( > "mov sp, %0\n" > --- a/arch/hexagon/kernel/process.c > +++ b/arch/hexagon/kernel/process.c > @@ -44,7 +44,6 @@ void arch_cpu_idle(void) > { > __vmwait(); > /* interrupts wake us up, but irqs are still disabled */ > - raw_local_irq_enable(); > } > > /* > --- a/arch/ia64/kernel/process.c > +++ b/arch/ia64/kernel/process.c > @@ -241,6 +241,7 @@ void arch_cpu_idle(void) > (*mark_idle)(1); > > raw_safe_halt(); > + raw_local_irq_disable(); > > if (mark_idle) > (*mark_idle)(0); > --- a/arch/microblaze/kernel/process.c > +++ b/arch/microblaze/kernel/process.c > @@ -138,5 +138,4 @@ int dump_fpu(struct pt_regs *regs, elf_f > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > --- a/arch/mips/kernel/idle.c > +++ b/arch/mips/kernel/idle.c > @@ -33,13 +33,13 @@ static void __cpuidle r3081_wait(void) > { > unsigned long cfg = read_c0_conf(); > write_c0_conf(cfg | R30XX_CONF_HALT); > - raw_local_irq_enable(); > } > > void __cpuidle r4k_wait(void) > { > raw_local_irq_enable(); > __r4k_wait(); > + raw_local_irq_disable(); > } > > /* > @@ -57,7 +57,6 @@ void __cpuidle r4k_wait_irqoff(void) > " .set arch=r4000 \n" > " wait \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -77,7 +76,6 @@ static void __cpuidle rm7k_wait_irqoff(v > " wait \n" > " mtc0 $1, $12 # stalls until W stage \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -103,6 +101,8 @@ static void __cpuidle au1k_wait(void) > " nop \n" > " .set pop \n" > : : "r" (au1k_wait), "r" (c0status)); > + > + raw_local_irq_disable(); > } > > static int __initdata nowait; > @@ -245,8 +245,6 @@ void arch_cpu_idle(void) > { > if (cpu_wait) > cpu_wait(); > - else > - raw_local_irq_enable(); > } > > #ifdef CONFIG_CPU_IDLE > --- a/arch/nios2/kernel/process.c > +++ b/arch/nios2/kernel/process.c > @@ -33,7 +33,6 @@ EXPORT_SYMBOL(pm_power_off); > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > > /* > --- a/arch/openrisc/kernel/process.c > +++ b/arch/openrisc/kernel/process.c > @@ -102,6 +102,7 @@ void arch_cpu_idle(void) > raw_local_irq_enable(); > if (mfspr(SPR_UPR) & SPR_UPR_PMP) > mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME); > + raw_local_irq_disable(); > } > > void (*pm_power_off)(void) = NULL; > --- a/arch/parisc/kernel/process.c > +++ b/arch/parisc/kernel/process.c > @@ -187,8 +187,6 @@ void arch_cpu_idle_dead(void) > > void __cpuidle arch_cpu_idle(void) > { > - raw_local_irq_enable(); > - > /* nop on real hardware, qemu will idle sleep. */ > asm volatile("or %%r10,%%r10,%%r10\n":::); > } > --- a/arch/powerpc/kernel/idle.c > +++ b/arch/powerpc/kernel/idle.c > @@ -51,10 +51,9 @@ void arch_cpu_idle(void) > * Some power_save functions return with > * interrupts enabled, some don't. > */ > - if (irqs_disabled()) > - raw_local_irq_enable(); > + if (!irqs_disabled()) > + raw_local_irq_disable(); > } else { > - raw_local_irq_enable(); > /* > * Go into low thread priority and possibly > * low power mode. > --- a/arch/riscv/kernel/process.c > +++ b/arch/riscv/kernel/process.c > @@ -39,7 +39,6 @@ extern asmlinkage void ret_from_kernel_t > void arch_cpu_idle(void) > { > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void __show_regs(struct pt_regs *regs) > --- a/arch/s390/kernel/idle.c > +++ b/arch/s390/kernel/idle.c > @@ -66,7 +66,6 @@ void arch_cpu_idle(void) > idle->idle_count++; > account_idle_time(cputime_to_nsecs(idle_time)); > raw_write_seqcount_end(&idle->seqcount); > - raw_local_irq_enable(); > } > > static ssize_t show_idle_count(struct device *dev, > --- a/arch/sh/kernel/idle.c > +++ b/arch/sh/kernel/idle.c > @@ -25,6 +25,7 @@ void default_idle(void) > raw_local_irq_enable(); > /* Isn't this racy ? */ > cpu_sleep(); > + raw_local_irq_disable(); > clear_bl_bit(); > } > > --- a/arch/sparc/kernel/leon_pmc.c > +++ b/arch/sparc/kernel/leon_pmc.c > @@ -57,6 +57,8 @@ static void pmc_leon_idle_fixup(void) > "lda [%0] %1, %%g0\n" > : > : "r"(address), "i"(ASI_LEON_BYPASS)); > + > + raw_local_irq_disable(); > } > > /* > @@ -70,6 +72,8 @@ static void pmc_leon_idle(void) > > /* For systems without power-down, this will be no-op */ > __asm__ __volatile__ ("wr %g0, %asr19\n\t"); > + > + raw_local_irq_disable(); > } > > /* Install LEON Power Down function */ > --- a/arch/sparc/kernel/process_32.c > +++ b/arch/sparc/kernel/process_32.c > @@ -71,7 +71,6 @@ void arch_cpu_idle(void) > { > if (sparc_idle) > (*sparc_idle)(); > - raw_local_irq_enable(); > } > > /* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */ > --- a/arch/sparc/kernel/process_64.c > +++ b/arch/sparc/kernel/process_64.c > @@ -59,7 +59,6 @@ void arch_cpu_idle(void) > { > if (tlb_type != hypervisor) { > touch_nmi_watchdog(); > - raw_local_irq_enable(); > } else { > unsigned long pstate; > > @@ -90,6 +89,8 @@ void arch_cpu_idle(void) > "wrpr %0, %%g0, %%pstate" > : "=&r" (pstate) > : "i" (PSTATE_IE)); > + > + raw_local_irq_disable(); > } > } > > --- a/arch/um/kernel/process.c > +++ b/arch/um/kernel/process.c > @@ -216,7 +216,6 @@ void arch_cpu_idle(void) > { > cpu_tasks[current_thread_info()->cpu].pid = os_getpid(); > um_idle_sleep(); > - raw_local_irq_enable(); > } > > int __cant_sleep(void) { > --- a/arch/x86/coco/tdx/tdx.c > +++ b/arch/x86/coco/tdx/tdx.c > @@ -178,6 +178,9 @@ void __cpuidle tdx_safe_halt(void) > */ > if (__halt(irq_disabled, do_sti)) > WARN_ONCE(1, "HLT instruction emulation failed\n"); > + > + /* XXX I can't make sense of what @do_sti actually does */ > + raw_local_irq_disable(); > } > > static bool read_msr(struct pt_regs *regs) > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -699,6 +699,7 @@ EXPORT_SYMBOL(boot_option_idle_override) > void __cpuidle default_idle(void) > { > raw_safe_halt(); > + raw_local_irq_disable(); > } > #if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE) > EXPORT_SYMBOL(default_idle); > @@ -804,13 +805,7 @@ static void amd_e400_idle(void) > > default_idle(); > > - /* > - * The switch back from broadcast mode needs to be called with > - * interrupts disabled. > - */ > - raw_local_irq_disable(); > tick_broadcast_exit(); > - raw_local_irq_enable(); > } > > /* > @@ -849,12 +844,10 @@ static __cpuidle void mwait_idle(void) > } > > __monitor((void *)¤t_thread_info()->flags, 0, 0); > - if (!need_resched()) > + if (!need_resched()) { > __sti_mwait(0, 0); > - else > - raw_local_irq_enable(); > - } else { > - raw_local_irq_enable(); > + raw_local_irq_disable(); > + } > } > __current_clr_polling(); > } > --- a/arch/xtensa/kernel/process.c > +++ b/arch/xtensa/kernel/process.c > @@ -183,6 +183,7 @@ void coprocessor_flush_release_all(struc > void arch_cpu_idle(void) > { > platform_idle(); > + raw_local_irq_disable(); > } > > /* > --- a/kernel/sched/idle.c > +++ b/kernel/sched/idle.c > @@ -79,7 +79,6 @@ void __weak arch_cpu_idle_dead(void) { } > void __weak arch_cpu_idle(void) > { > cpu_idle_force_poll = 1; > - raw_local_irq_enable(); > } > > /** > @@ -96,7 +95,6 @@ void __cpuidle default_idle_call(void) > > cpuidle_rcu_enter(); > arch_cpu_idle(); > - raw_local_irq_disable(); > cpuidle_rcu_exit(); > > start_critical_timings(); > > _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.librecores.org (lists.librecores.org [88.198.125.70]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCC18CCA482 for ; Wed, 6 Jul 2022 21:55:31 +0000 (UTC) Received: from [172.31.1.100] (localhost.localdomain [127.0.0.1]) by mail.librecores.org (Postfix) with ESMTP id 0981D24929; Wed, 6 Jul 2022 23:55:31 +0200 (CEST) Received: from mail-yw1-f170.google.com (mail-yw1-f170.google.com [209.85.128.170]) by mail.librecores.org (Postfix) with ESMTPS id 0EAB324929 for ; Wed, 6 Jul 2022 16:03:05 +0200 (CEST) Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-31caffa4a45so69329047b3.3 for ; Wed, 06 Jul 2022 07:03:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WBBLwGm3VvIAnm4uSuSyiWShri4w+bgQrIyP7HIMtP0=; b=4Z7RrglK7/VQ280QS6F0DBYaANQYFjxIM3sOyHQBjl3pRRgEQXqqXvzcJBT3ZocM5j 5sTQ3tgMbOKXbygkG67fIeL9SDR1P5QilQlaEUv6X4NzCJO6Wr/K1tlh/UUWIAWP2QFj nyyjLv2ESsPZnZaBMTPj5IdjeRSJSq5e+xaEReoOTKolx7p6eWtnsJlhkI7giTYVtETM /pJ8NtEj/wWubkAI7XTegbA87+/cti1UBDXbzXYfw/zjmQKbw12dHxjX8EKygm1qv6NW 7LSsKusKxLWvgAQcU2PIX5mATdITUMBZxG/DHaFCD2sIwJgv5m+L+0bziRHxzB4Mbwhc 9Yjw== X-Gm-Message-State: AJIora90vYYJUZ8nW5+w22UBPgJSTpZ16M4G2xvnv8SG+UKsxFpn/yRH LdZAfh452wASHSu9+OaosQckzsOJWtNdxkAIHts= X-Google-Smtp-Source: AGRyM1shXjwii2Xu/qgQ1B1HadHYSMjqitGmbFlHxf4P7RndZwyBhPC2CaCRixiXqBuDMvjO7JXHYGnoD2iVhXDPhek= X-Received: by 2002:a81:a184:0:b0:31c:b00e:b5c4 with SMTP id y126-20020a81a184000000b0031cb00eb5c4mr15058481ywg.149.1657116183677; Wed, 06 Jul 2022 07:03:03 -0700 (PDT) MIME-Version: 1.0 References: <20220608142723.103523089@infradead.org> <20220608144517.188449351@infradead.org> In-Reply-To: <20220608144517.188449351@infradead.org> From: "Rafael J. Wysocki" Date: Wed, 6 Jul 2022 16:02:52 +0200 Message-ID: Subject: Re: [PATCH 20/36] arch/idle: Change arch_cpu_idle() IRQ behaviour To: Peter Zijlstra Content-Type: text/plain; charset="UTF-8" X-Mailman-Approved-At: Wed, 06 Jul 2022 23:55:27 +0200 X-BeenThere: openrisc@lists.librecores.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Discussion around the OpenRISC processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juri Lelli , "Rafael J. Wysocki" , Benjamin Herrenschmidt , Linus Walleij , Benjamin Segall , Guo Ren , Pavel Machek , Alexander Gordeev , srivatsa@csail.mit.edu, linux-arch , Vincent Guittot , Michael Ellerman , Huacai Chen , ACPI Devel Maling List , Andy Gross , dl-linux-imx , Catalin Marinas , xen-devel@lists.xenproject.org, mattst88@gmail.com, Christian Borntraeger , Michael Turquette , sammy@sammy.net, Petr Mladek , Linux PM , Lai Jiangshan , Sascha Hauer , linux-um@lists.infradead.org, acme@kernel.org, Thomas Gleixner , Linux OMAP Mailing List , Dietmar Eggemann , rth@twiddle.net, Greg Kroah-Hartman , Linux Kernel Mailing List , linux-perf-users@vger.kernel.org, senozhatsky@chromium.org, Sven Schnelle , jolsa@kernel.org, Paul Mackerras , Mark Rutland , linux-ia64@vger.kernel.org, Dave Hansen , virtualization@lists.linux-foundation.org, James Bottomley , jcmvbkbc@gmail.com, Thierry Reding , kernel@xen0n.name, quic_neeraju@quicinc.com, linux-s390@vger.kernel.org, vschneid@redhat.com, John Ogness , Yoshinori Sato , Linux-sh list , Fabio Estevam , Helge Deller , Daniel Lezcano , Jon Hunter , Mathieu Desnoyers , Frederic Weisbecker , Len Brown , linux-xtensa@linux-xtensa.org, Sascha Hauer , Vasily Gorbik , linux-arm-msm , linux-alpha@vger.kernel.org, linux-m68k , Linux ARM , Chris Zankel , Stephen Boyd , dinguyen@kernel.org, Daniel Bristot de Oliveira , Alexander Shishkin , Lorenzo Pieralisi , Rasmus Villemoes , Joel Fernandes , Will Deacon , Boris Ostrovsky , Kevin Hilman , linux-csky@vger.kernel.org, pv-drivers@vmware.com, linux-snps-arc@lists.infradead.org, Mel Gorman , Jacob Pan , Arnd Bergmann , ulli.kroll@googlemail.com, vgupta@kernel.org, linux-clk , Josh Triplett , Steven Rostedt , rcu@vger.kernel.org, Borislav Petkov , bcain@quicinc.com, Thomas Bogendoerfer , Parisc List , Sudeep Holla , Shawn Guo , David Miller , Rich Felker , Tony Lindgren , amakhalov@vmware.com, Bjorn Andersson , "H. Peter Anvin" , sparclinux@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-riscv , anton.ivanov@cambridgegreys.com, jonas@southpole.se, Yury Norov , Richard Weinberger , the arch/x86 maintainers , Russell King - ARM Linux , Ingo Molnar , Albert Ou , "Paul E. McKenney" , Heiko Carstens , openrisc@lists.librecores.org, Paul Walmsley , linux-tegra , namhyung@kernel.org, Andy Shevchenko , jpoimboe@kernel.org, Juergen Gross , Michal Simek , "open list:BROADCOM NVRAM DRIVER" , Palmer Dabbelt , Anup Patel , ink@jurassic.park.msu.ru, Johannes Berg , linuxppc-dev Errors-To: openrisc-bounces@lists.librecores.org Sender: "OpenRISC" On Wed, Jun 8, 2022 at 4:46 PM Peter Zijlstra wrote: > > Current arch_cpu_idle() is called with IRQs disabled, but will return > with IRQs enabled. > > However, the very first thing the generic code does after calling > arch_cpu_idle() is raw_local_irq_disable(). This means that > architectures that can idle with IRQs disabled end up doing a > pointless 'enable-disable' dance. > > Therefore, push this IRQ disabling into the idle function, meaning > that those architectures can avoid the pointless IRQ state flipping. > > Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki > --- > arch/alpha/kernel/process.c | 1 - > arch/arc/kernel/process.c | 3 +++ > arch/arm/kernel/process.c | 1 - > arch/arm/mach-gemini/board-dt.c | 3 ++- > arch/arm64/kernel/idle.c | 1 - > arch/csky/kernel/process.c | 1 - > arch/csky/kernel/smp.c | 2 +- > arch/hexagon/kernel/process.c | 1 - > arch/ia64/kernel/process.c | 1 + > arch/microblaze/kernel/process.c | 1 - > arch/mips/kernel/idle.c | 8 +++----- > arch/nios2/kernel/process.c | 1 - > arch/openrisc/kernel/process.c | 1 + > arch/parisc/kernel/process.c | 2 -- > arch/powerpc/kernel/idle.c | 5 ++--- > arch/riscv/kernel/process.c | 1 - > arch/s390/kernel/idle.c | 1 - > arch/sh/kernel/idle.c | 1 + > arch/sparc/kernel/leon_pmc.c | 4 ++++ > arch/sparc/kernel/process_32.c | 1 - > arch/sparc/kernel/process_64.c | 3 ++- > arch/um/kernel/process.c | 1 - > arch/x86/coco/tdx/tdx.c | 3 +++ > arch/x86/kernel/process.c | 15 ++++----------- > arch/xtensa/kernel/process.c | 1 + > kernel/sched/idle.c | 2 -- > 26 files changed, 28 insertions(+), 37 deletions(-) > > --- a/arch/alpha/kernel/process.c > +++ b/arch/alpha/kernel/process.c > @@ -57,7 +57,6 @@ EXPORT_SYMBOL(pm_power_off); > void arch_cpu_idle(void) > { > wtint(0); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_dead(void) > --- a/arch/arc/kernel/process.c > +++ b/arch/arc/kernel/process.c > @@ -114,6 +114,8 @@ void arch_cpu_idle(void) > "sleep %0 \n" > : > :"I"(arg)); /* can't be "r" has to be embedded const */ > + > + raw_local_irq_disable(); > } > > #else /* ARC700 */ > @@ -122,6 +124,7 @@ void arch_cpu_idle(void) > { > /* sleep, but enable both set E1/E2 (levels of interrupts) before committing */ > __asm__ __volatile__("sleep 0x3 \n"); > + raw_local_irq_disable(); > } > > #endif > --- a/arch/arm/kernel/process.c > +++ b/arch/arm/kernel/process.c > @@ -78,7 +78,6 @@ void arch_cpu_idle(void) > arm_pm_idle(); > else > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_prepare(void) > --- a/arch/arm/mach-gemini/board-dt.c > +++ b/arch/arm/mach-gemini/board-dt.c > @@ -42,8 +42,9 @@ static void gemini_idle(void) > */ > > /* FIXME: Enabling interrupts here is racy! */ > - local_irq_enable(); > + raw_local_irq_enable(); > cpu_do_idle(); > + raw_local_irq_disable(); > } > > static void __init gemini_init_machine(void) > --- a/arch/arm64/kernel/idle.c > +++ b/arch/arm64/kernel/idle.c > @@ -42,5 +42,4 @@ void noinstr arch_cpu_idle(void) > * tricks > */ > cpu_do_idle(); > - raw_local_irq_enable(); > } > --- a/arch/csky/kernel/process.c > +++ b/arch/csky/kernel/process.c > @@ -101,6 +101,5 @@ void arch_cpu_idle(void) > #ifdef CONFIG_CPU_PM_STOP > asm volatile("stop\n"); > #endif > - raw_local_irq_enable(); > } > #endif > --- a/arch/csky/kernel/smp.c > +++ b/arch/csky/kernel/smp.c > @@ -314,7 +314,7 @@ void arch_cpu_idle_dead(void) > while (!secondary_stack) > arch_cpu_idle(); > > - local_irq_disable(); > + raw_local_irq_disable(); > > asm volatile( > "mov sp, %0\n" > --- a/arch/hexagon/kernel/process.c > +++ b/arch/hexagon/kernel/process.c > @@ -44,7 +44,6 @@ void arch_cpu_idle(void) > { > __vmwait(); > /* interrupts wake us up, but irqs are still disabled */ > - raw_local_irq_enable(); > } > > /* > --- a/arch/ia64/kernel/process.c > +++ b/arch/ia64/kernel/process.c > @@ -241,6 +241,7 @@ void arch_cpu_idle(void) > (*mark_idle)(1); > > raw_safe_halt(); > + raw_local_irq_disable(); > > if (mark_idle) > (*mark_idle)(0); > --- a/arch/microblaze/kernel/process.c > +++ b/arch/microblaze/kernel/process.c > @@ -138,5 +138,4 @@ int dump_fpu(struct pt_regs *regs, elf_f > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > --- a/arch/mips/kernel/idle.c > +++ b/arch/mips/kernel/idle.c > @@ -33,13 +33,13 @@ static void __cpuidle r3081_wait(void) > { > unsigned long cfg = read_c0_conf(); > write_c0_conf(cfg | R30XX_CONF_HALT); > - raw_local_irq_enable(); > } > > void __cpuidle r4k_wait(void) > { > raw_local_irq_enable(); > __r4k_wait(); > + raw_local_irq_disable(); > } > > /* > @@ -57,7 +57,6 @@ void __cpuidle r4k_wait_irqoff(void) > " .set arch=r4000 \n" > " wait \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -77,7 +76,6 @@ static void __cpuidle rm7k_wait_irqoff(v > " wait \n" > " mtc0 $1, $12 # stalls until W stage \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -103,6 +101,8 @@ static void __cpuidle au1k_wait(void) > " nop \n" > " .set pop \n" > : : "r" (au1k_wait), "r" (c0status)); > + > + raw_local_irq_disable(); > } > > static int __initdata nowait; > @@ -245,8 +245,6 @@ void arch_cpu_idle(void) > { > if (cpu_wait) > cpu_wait(); > - else > - raw_local_irq_enable(); > } > > #ifdef CONFIG_CPU_IDLE > --- a/arch/nios2/kernel/process.c > +++ b/arch/nios2/kernel/process.c > @@ -33,7 +33,6 @@ EXPORT_SYMBOL(pm_power_off); > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > > /* > --- a/arch/openrisc/kernel/process.c > +++ b/arch/openrisc/kernel/process.c > @@ -102,6 +102,7 @@ void arch_cpu_idle(void) > raw_local_irq_enable(); > if (mfspr(SPR_UPR) & SPR_UPR_PMP) > mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME); > + raw_local_irq_disable(); > } > > void (*pm_power_off)(void) = NULL; > --- a/arch/parisc/kernel/process.c > +++ b/arch/parisc/kernel/process.c > @@ -187,8 +187,6 @@ void arch_cpu_idle_dead(void) > > void __cpuidle arch_cpu_idle(void) > { > - raw_local_irq_enable(); > - > /* nop on real hardware, qemu will idle sleep. */ > asm volatile("or %%r10,%%r10,%%r10\n":::); > } > --- a/arch/powerpc/kernel/idle.c > +++ b/arch/powerpc/kernel/idle.c > @@ -51,10 +51,9 @@ void arch_cpu_idle(void) > * Some power_save functions return with > * interrupts enabled, some don't. > */ > - if (irqs_disabled()) > - raw_local_irq_enable(); > + if (!irqs_disabled()) > + raw_local_irq_disable(); > } else { > - raw_local_irq_enable(); > /* > * Go into low thread priority and possibly > * low power mode. > --- a/arch/riscv/kernel/process.c > +++ b/arch/riscv/kernel/process.c > @@ -39,7 +39,6 @@ extern asmlinkage void ret_from_kernel_t > void arch_cpu_idle(void) > { > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void __show_regs(struct pt_regs *regs) > --- a/arch/s390/kernel/idle.c > +++ b/arch/s390/kernel/idle.c > @@ -66,7 +66,6 @@ void arch_cpu_idle(void) > idle->idle_count++; > account_idle_time(cputime_to_nsecs(idle_time)); > raw_write_seqcount_end(&idle->seqcount); > - raw_local_irq_enable(); > } > > static ssize_t show_idle_count(struct device *dev, > --- a/arch/sh/kernel/idle.c > +++ b/arch/sh/kernel/idle.c > @@ -25,6 +25,7 @@ void default_idle(void) > raw_local_irq_enable(); > /* Isn't this racy ? */ > cpu_sleep(); > + raw_local_irq_disable(); > clear_bl_bit(); > } > > --- a/arch/sparc/kernel/leon_pmc.c > +++ b/arch/sparc/kernel/leon_pmc.c > @@ -57,6 +57,8 @@ static void pmc_leon_idle_fixup(void) > "lda [%0] %1, %%g0\n" > : > : "r"(address), "i"(ASI_LEON_BYPASS)); > + > + raw_local_irq_disable(); > } > > /* > @@ -70,6 +72,8 @@ static void pmc_leon_idle(void) > > /* For systems without power-down, this will be no-op */ > __asm__ __volatile__ ("wr %g0, %asr19\n\t"); > + > + raw_local_irq_disable(); > } > > /* Install LEON Power Down function */ > --- a/arch/sparc/kernel/process_32.c > +++ b/arch/sparc/kernel/process_32.c > @@ -71,7 +71,6 @@ void arch_cpu_idle(void) > { > if (sparc_idle) > (*sparc_idle)(); > - raw_local_irq_enable(); > } > > /* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */ > --- a/arch/sparc/kernel/process_64.c > +++ b/arch/sparc/kernel/process_64.c > @@ -59,7 +59,6 @@ void arch_cpu_idle(void) > { > if (tlb_type != hypervisor) { > touch_nmi_watchdog(); > - raw_local_irq_enable(); > } else { > unsigned long pstate; > > @@ -90,6 +89,8 @@ void arch_cpu_idle(void) > "wrpr %0, %%g0, %%pstate" > : "=&r" (pstate) > : "i" (PSTATE_IE)); > + > + raw_local_irq_disable(); > } > } > > --- a/arch/um/kernel/process.c > +++ b/arch/um/kernel/process.c > @@ -216,7 +216,6 @@ void arch_cpu_idle(void) > { > cpu_tasks[current_thread_info()->cpu].pid = os_getpid(); > um_idle_sleep(); > - raw_local_irq_enable(); > } > > int __cant_sleep(void) { > --- a/arch/x86/coco/tdx/tdx.c > +++ b/arch/x86/coco/tdx/tdx.c > @@ -178,6 +178,9 @@ void __cpuidle tdx_safe_halt(void) > */ > if (__halt(irq_disabled, do_sti)) > WARN_ONCE(1, "HLT instruction emulation failed\n"); > + > + /* XXX I can't make sense of what @do_sti actually does */ > + raw_local_irq_disable(); > } > > static bool read_msr(struct pt_regs *regs) > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -699,6 +699,7 @@ EXPORT_SYMBOL(boot_option_idle_override) > void __cpuidle default_idle(void) > { > raw_safe_halt(); > + raw_local_irq_disable(); > } > #if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE) > EXPORT_SYMBOL(default_idle); > @@ -804,13 +805,7 @@ static void amd_e400_idle(void) > > default_idle(); > > - /* > - * The switch back from broadcast mode needs to be called with > - * interrupts disabled. > - */ > - raw_local_irq_disable(); > tick_broadcast_exit(); > - raw_local_irq_enable(); > } > > /* > @@ -849,12 +844,10 @@ static __cpuidle void mwait_idle(void) > } > > __monitor((void *)¤t_thread_info()->flags, 0, 0); > - if (!need_resched()) > + if (!need_resched()) { > __sti_mwait(0, 0); > - else > - raw_local_irq_enable(); > - } else { > - raw_local_irq_enable(); > + raw_local_irq_disable(); > + } > } > __current_clr_polling(); > } > --- a/arch/xtensa/kernel/process.c > +++ b/arch/xtensa/kernel/process.c > @@ -183,6 +183,7 @@ void coprocessor_flush_release_all(struc > void arch_cpu_idle(void) > { > platform_idle(); > + raw_local_irq_disable(); > } > > /* > --- a/kernel/sched/idle.c > +++ b/kernel/sched/idle.c > @@ -79,7 +79,6 @@ void __weak arch_cpu_idle_dead(void) { } > void __weak arch_cpu_idle(void) > { > cpu_idle_force_poll = 1; > - raw_local_irq_enable(); > } > > /** > @@ -96,7 +95,6 @@ void __cpuidle default_idle_call(void) > > cpuidle_rcu_enter(); > arch_cpu_idle(); > - raw_local_irq_disable(); > cpuidle_rcu_exit(); > > start_critical_timings(); > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E07B9C43334 for ; Wed, 6 Jul 2022 22:20:45 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4LdYs83Dckz3fFG for ; Thu, 7 Jul 2022 08:20:44 +1000 (AEST) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=209.85.128.176; helo=mail-yw1-f176.google.com; envelope-from=rjwysocki@gmail.com; receiver=) Received: from mail-yw1-f176.google.com (mail-yw1-f176.google.com [209.85.128.176]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4LdLpy36xJz3bk0 for ; Thu, 7 Jul 2022 00:03:06 +1000 (AEST) Received: by mail-yw1-f176.google.com with SMTP id 00721157ae682-317a66d62dfso141052987b3.7 for ; Wed, 06 Jul 2022 07:03:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WBBLwGm3VvIAnm4uSuSyiWShri4w+bgQrIyP7HIMtP0=; b=Yz7O2Nley7/vcb9PiICW4XhYqMEjP0I/ZL4B2LNUzaMNFxKVAJeZ5G9Exv/kA4ZUYb X6F7bQFjBvFHa7mnPn9GSsGJ5IqPJqP0qIz8Jkl0+l5z/aaeZAjwCkapwz1hO70abNjx oj5R9RRDSPpsqyVD/WfZLpBlzWZhlyh6gzUe9NZEJiltSI3J7WOmgojf9O70ldyhK5Dn O5k6shZA6KBD//Xnvs4jaGjlx+xY4mNR7/7M0jvlHSnoYQv15HsBQvYY+PI5V0J6embU wKTLPSKJD/steLbv9aX2ZX/LS49+d8bM8OlPn7aLMWaIWbclG+7lYp2sblhXIUetOiYI uYJw== X-Gm-Message-State: AJIora+eVKMTwvFX1E7FsBDMg/ArJIPxl8cDwq5fH8+AHcpbeWibK9yH boMoSqUmIbJMSzgJO5yDKKmy3Td/AjQ55bzgcZo= X-Google-Smtp-Source: AGRyM1shXjwii2Xu/qgQ1B1HadHYSMjqitGmbFlHxf4P7RndZwyBhPC2CaCRixiXqBuDMvjO7JXHYGnoD2iVhXDPhek= X-Received: by 2002:a81:a184:0:b0:31c:b00e:b5c4 with SMTP id y126-20020a81a184000000b0031cb00eb5c4mr15058481ywg.149.1657116183677; Wed, 06 Jul 2022 07:03:03 -0700 (PDT) MIME-Version: 1.0 References: <20220608142723.103523089@infradead.org> <20220608144517.188449351@infradead.org> In-Reply-To: <20220608144517.188449351@infradead.org> From: "Rafael J. Wysocki" Date: Wed, 6 Jul 2022 16:02:52 +0200 Message-ID: Subject: Re: [PATCH 20/36] arch/idle: Change arch_cpu_idle() IRQ behaviour To: Peter Zijlstra Content-Type: text/plain; charset="UTF-8" X-Mailman-Approved-At: Thu, 07 Jul 2022 08:18:29 +1000 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juri Lelli , "Rafael J. Wysocki" , Linus Walleij , Benjamin Segall , Guo Ren , Pavel Machek , Alexander Gordeev , srivatsa@csail.mit.edu, linux-arch , Vincent Guittot , Huacai Chen , ACPI Devel Maling List , Andy Gross , Geert Uytterhoeven , dl-linux-imx , Catalin Marinas , xen-devel@lists.xenproject.org, mattst88@gmail.com, Christian Borntraeger , Michael Turquette , sammy@sammy.net, Petr Mladek , Linux PM , Lai Jiangshan , Sascha Hauer , linux-um@lists.infradead.org, acme@kernel.org, Thomas Gleixner , Linux OMAP Mailing List , Dietmar Eggemann , rth@twiddle.net, Greg Kroah-Hartman , Linux Kernel Mailing List , linux-perf-users@vger.kernel.org, senozhatsky@chromium.org, Sven Schnelle , jolsa@kernel.org, Paul Mackerras , Mark Rutland , linux-ia64@vger.kernel.org, Dave Hansen , virtualization@lists.linux-foundation.org, James Bottomley , jcmvbkbc@gmail.com, Thierry Reding , kernel@xen0n.name, quic_neeraju@quicinc.com, linux-s390@vger.kernel.org, vschneid@redhat.com, John Ogness , Yoshinori Sato , Linux-sh list , Fabio Estevam , Helge Deller , Daniel Lezcano , Jon H unter , Mathieu Desnoyers , Frederic Weisbecker , Len Brown , linux-xtensa@linux-xtensa.org, Sascha Hauer , Vasily Gorbik , linux-arm-msm , linux-alpha@vger.kernel.org, linux-m68k , Stafford Horne , Linux ARM , Chris Zankel , Stephen Boyd , dinguyen@kernel.org, Daniel Bristot de Oliveira , Alexander Shishkin , Lorenzo Pieralisi , Rasmus Villemoes , Joel Fernandes , Will Deacon , Boris Ostrovsky , Kevin Hilman , linux-csky@vger.kernel.org, pv-drivers@vmware.com, linux-snps-arc@lists.infradead.org, Mel Gorman , Jacob Pan , Arnd Bergmann , ulli.kroll@googlemail.com, vgupta@kernel.org, linux-clk , Josh Triplett , Steven Rostedt , rcu@vger.kernel.org, Borislav Petkov , bcain@quicinc.com, Thomas Bogendoerfer , Parisc List , Sudeep Holla , Shawn Guo , David Miller , Rich Felker , Tony Lindgren , amakhalov@vmware.com, Bjorn Andersson , "H. Peter Anvin" , sparclinux@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-riscv , anton.ivanov@cambridgegreys.com, jonas@southpole.se, Yury Norov , Richard Weinberger , the arch/x86 maintainers , Russell King - ARM Linux , Ingo Molnar , Al bert Ou , "Paul E. McKenney" < paulmck@kernel.org>, Heiko Carstens , stefan.kristiansson@saunalahti.fi, openrisc@lists.librecores.org, Paul Walmsley , linux-tegra , namhyung@kernel.org, Andy Shevchenko , jpoimboe@kernel.org, Juergen Gross , Michal Simek , "open list:BROADCOM NVRAM DRIVER" , Palmer Dabbelt , Anup Patel , ink@jurassic.park.msu.ru, Johannes Berg , linuxppc-dev Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed, Jun 8, 2022 at 4:46 PM Peter Zijlstra wrote: > > Current arch_cpu_idle() is called with IRQs disabled, but will return > with IRQs enabled. > > However, the very first thing the generic code does after calling > arch_cpu_idle() is raw_local_irq_disable(). This means that > architectures that can idle with IRQs disabled end up doing a > pointless 'enable-disable' dance. > > Therefore, push this IRQ disabling into the idle function, meaning > that those architectures can avoid the pointless IRQ state flipping. > > Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki > --- > arch/alpha/kernel/process.c | 1 - > arch/arc/kernel/process.c | 3 +++ > arch/arm/kernel/process.c | 1 - > arch/arm/mach-gemini/board-dt.c | 3 ++- > arch/arm64/kernel/idle.c | 1 - > arch/csky/kernel/process.c | 1 - > arch/csky/kernel/smp.c | 2 +- > arch/hexagon/kernel/process.c | 1 - > arch/ia64/kernel/process.c | 1 + > arch/microblaze/kernel/process.c | 1 - > arch/mips/kernel/idle.c | 8 +++----- > arch/nios2/kernel/process.c | 1 - > arch/openrisc/kernel/process.c | 1 + > arch/parisc/kernel/process.c | 2 -- > arch/powerpc/kernel/idle.c | 5 ++--- > arch/riscv/kernel/process.c | 1 - > arch/s390/kernel/idle.c | 1 - > arch/sh/kernel/idle.c | 1 + > arch/sparc/kernel/leon_pmc.c | 4 ++++ > arch/sparc/kernel/process_32.c | 1 - > arch/sparc/kernel/process_64.c | 3 ++- > arch/um/kernel/process.c | 1 - > arch/x86/coco/tdx/tdx.c | 3 +++ > arch/x86/kernel/process.c | 15 ++++----------- > arch/xtensa/kernel/process.c | 1 + > kernel/sched/idle.c | 2 -- > 26 files changed, 28 insertions(+), 37 deletions(-) > > --- a/arch/alpha/kernel/process.c > +++ b/arch/alpha/kernel/process.c > @@ -57,7 +57,6 @@ EXPORT_SYMBOL(pm_power_off); > void arch_cpu_idle(void) > { > wtint(0); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_dead(void) > --- a/arch/arc/kernel/process.c > +++ b/arch/arc/kernel/process.c > @@ -114,6 +114,8 @@ void arch_cpu_idle(void) > "sleep %0 \n" > : > :"I"(arg)); /* can't be "r" has to be embedded const */ > + > + raw_local_irq_disable(); > } > > #else /* ARC700 */ > @@ -122,6 +124,7 @@ void arch_cpu_idle(void) > { > /* sleep, but enable both set E1/E2 (levels of interrupts) before committing */ > __asm__ __volatile__("sleep 0x3 \n"); > + raw_local_irq_disable(); > } > > #endif > --- a/arch/arm/kernel/process.c > +++ b/arch/arm/kernel/process.c > @@ -78,7 +78,6 @@ void arch_cpu_idle(void) > arm_pm_idle(); > else > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_prepare(void) > --- a/arch/arm/mach-gemini/board-dt.c > +++ b/arch/arm/mach-gemini/board-dt.c > @@ -42,8 +42,9 @@ static void gemini_idle(void) > */ > > /* FIXME: Enabling interrupts here is racy! */ > - local_irq_enable(); > + raw_local_irq_enable(); > cpu_do_idle(); > + raw_local_irq_disable(); > } > > static void __init gemini_init_machine(void) > --- a/arch/arm64/kernel/idle.c > +++ b/arch/arm64/kernel/idle.c > @@ -42,5 +42,4 @@ void noinstr arch_cpu_idle(void) > * tricks > */ > cpu_do_idle(); > - raw_local_irq_enable(); > } > --- a/arch/csky/kernel/process.c > +++ b/arch/csky/kernel/process.c > @@ -101,6 +101,5 @@ void arch_cpu_idle(void) > #ifdef CONFIG_CPU_PM_STOP > asm volatile("stop\n"); > #endif > - raw_local_irq_enable(); > } > #endif > --- a/arch/csky/kernel/smp.c > +++ b/arch/csky/kernel/smp.c > @@ -314,7 +314,7 @@ void arch_cpu_idle_dead(void) > while (!secondary_stack) > arch_cpu_idle(); > > - local_irq_disable(); > + raw_local_irq_disable(); > > asm volatile( > "mov sp, %0\n" > --- a/arch/hexagon/kernel/process.c > +++ b/arch/hexagon/kernel/process.c > @@ -44,7 +44,6 @@ void arch_cpu_idle(void) > { > __vmwait(); > /* interrupts wake us up, but irqs are still disabled */ > - raw_local_irq_enable(); > } > > /* > --- a/arch/ia64/kernel/process.c > +++ b/arch/ia64/kernel/process.c > @@ -241,6 +241,7 @@ void arch_cpu_idle(void) > (*mark_idle)(1); > > raw_safe_halt(); > + raw_local_irq_disable(); > > if (mark_idle) > (*mark_idle)(0); > --- a/arch/microblaze/kernel/process.c > +++ b/arch/microblaze/kernel/process.c > @@ -138,5 +138,4 @@ int dump_fpu(struct pt_regs *regs, elf_f > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > --- a/arch/mips/kernel/idle.c > +++ b/arch/mips/kernel/idle.c > @@ -33,13 +33,13 @@ static void __cpuidle r3081_wait(void) > { > unsigned long cfg = read_c0_conf(); > write_c0_conf(cfg | R30XX_CONF_HALT); > - raw_local_irq_enable(); > } > > void __cpuidle r4k_wait(void) > { > raw_local_irq_enable(); > __r4k_wait(); > + raw_local_irq_disable(); > } > > /* > @@ -57,7 +57,6 @@ void __cpuidle r4k_wait_irqoff(void) > " .set arch=r4000 \n" > " wait \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -77,7 +76,6 @@ static void __cpuidle rm7k_wait_irqoff(v > " wait \n" > " mtc0 $1, $12 # stalls until W stage \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -103,6 +101,8 @@ static void __cpuidle au1k_wait(void) > " nop \n" > " .set pop \n" > : : "r" (au1k_wait), "r" (c0status)); > + > + raw_local_irq_disable(); > } > > static int __initdata nowait; > @@ -245,8 +245,6 @@ void arch_cpu_idle(void) > { > if (cpu_wait) > cpu_wait(); > - else > - raw_local_irq_enable(); > } > > #ifdef CONFIG_CPU_IDLE > --- a/arch/nios2/kernel/process.c > +++ b/arch/nios2/kernel/process.c > @@ -33,7 +33,6 @@ EXPORT_SYMBOL(pm_power_off); > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > > /* > --- a/arch/openrisc/kernel/process.c > +++ b/arch/openrisc/kernel/process.c > @@ -102,6 +102,7 @@ void arch_cpu_idle(void) > raw_local_irq_enable(); > if (mfspr(SPR_UPR) & SPR_UPR_PMP) > mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME); > + raw_local_irq_disable(); > } > > void (*pm_power_off)(void) = NULL; > --- a/arch/parisc/kernel/process.c > +++ b/arch/parisc/kernel/process.c > @@ -187,8 +187,6 @@ void arch_cpu_idle_dead(void) > > void __cpuidle arch_cpu_idle(void) > { > - raw_local_irq_enable(); > - > /* nop on real hardware, qemu will idle sleep. */ > asm volatile("or %%r10,%%r10,%%r10\n":::); > } > --- a/arch/powerpc/kernel/idle.c > +++ b/arch/powerpc/kernel/idle.c > @@ -51,10 +51,9 @@ void arch_cpu_idle(void) > * Some power_save functions return with > * interrupts enabled, some don't. > */ > - if (irqs_disabled()) > - raw_local_irq_enable(); > + if (!irqs_disabled()) > + raw_local_irq_disable(); > } else { > - raw_local_irq_enable(); > /* > * Go into low thread priority and possibly > * low power mode. > --- a/arch/riscv/kernel/process.c > +++ b/arch/riscv/kernel/process.c > @@ -39,7 +39,6 @@ extern asmlinkage void ret_from_kernel_t > void arch_cpu_idle(void) > { > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void __show_regs(struct pt_regs *regs) > --- a/arch/s390/kernel/idle.c > +++ b/arch/s390/kernel/idle.c > @@ -66,7 +66,6 @@ void arch_cpu_idle(void) > idle->idle_count++; > account_idle_time(cputime_to_nsecs(idle_time)); > raw_write_seqcount_end(&idle->seqcount); > - raw_local_irq_enable(); > } > > static ssize_t show_idle_count(struct device *dev, > --- a/arch/sh/kernel/idle.c > +++ b/arch/sh/kernel/idle.c > @@ -25,6 +25,7 @@ void default_idle(void) > raw_local_irq_enable(); > /* Isn't this racy ? */ > cpu_sleep(); > + raw_local_irq_disable(); > clear_bl_bit(); > } > > --- a/arch/sparc/kernel/leon_pmc.c > +++ b/arch/sparc/kernel/leon_pmc.c > @@ -57,6 +57,8 @@ static void pmc_leon_idle_fixup(void) > "lda [%0] %1, %%g0\n" > : > : "r"(address), "i"(ASI_LEON_BYPASS)); > + > + raw_local_irq_disable(); > } > > /* > @@ -70,6 +72,8 @@ static void pmc_leon_idle(void) > > /* For systems without power-down, this will be no-op */ > __asm__ __volatile__ ("wr %g0, %asr19\n\t"); > + > + raw_local_irq_disable(); > } > > /* Install LEON Power Down function */ > --- a/arch/sparc/kernel/process_32.c > +++ b/arch/sparc/kernel/process_32.c > @@ -71,7 +71,6 @@ void arch_cpu_idle(void) > { > if (sparc_idle) > (*sparc_idle)(); > - raw_local_irq_enable(); > } > > /* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */ > --- a/arch/sparc/kernel/process_64.c > +++ b/arch/sparc/kernel/process_64.c > @@ -59,7 +59,6 @@ void arch_cpu_idle(void) > { > if (tlb_type != hypervisor) { > touch_nmi_watchdog(); > - raw_local_irq_enable(); > } else { > unsigned long pstate; > > @@ -90,6 +89,8 @@ void arch_cpu_idle(void) > "wrpr %0, %%g0, %%pstate" > : "=&r" (pstate) > : "i" (PSTATE_IE)); > + > + raw_local_irq_disable(); > } > } > > --- a/arch/um/kernel/process.c > +++ b/arch/um/kernel/process.c > @@ -216,7 +216,6 @@ void arch_cpu_idle(void) > { > cpu_tasks[current_thread_info()->cpu].pid = os_getpid(); > um_idle_sleep(); > - raw_local_irq_enable(); > } > > int __cant_sleep(void) { > --- a/arch/x86/coco/tdx/tdx.c > +++ b/arch/x86/coco/tdx/tdx.c > @@ -178,6 +178,9 @@ void __cpuidle tdx_safe_halt(void) > */ > if (__halt(irq_disabled, do_sti)) > WARN_ONCE(1, "HLT instruction emulation failed\n"); > + > + /* XXX I can't make sense of what @do_sti actually does */ > + raw_local_irq_disable(); > } > > static bool read_msr(struct pt_regs *regs) > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -699,6 +699,7 @@ EXPORT_SYMBOL(boot_option_idle_override) > void __cpuidle default_idle(void) > { > raw_safe_halt(); > + raw_local_irq_disable(); > } > #if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE) > EXPORT_SYMBOL(default_idle); > @@ -804,13 +805,7 @@ static void amd_e400_idle(void) > > default_idle(); > > - /* > - * The switch back from broadcast mode needs to be called with > - * interrupts disabled. > - */ > - raw_local_irq_disable(); > tick_broadcast_exit(); > - raw_local_irq_enable(); > } > > /* > @@ -849,12 +844,10 @@ static __cpuidle void mwait_idle(void) > } > > __monitor((void *)¤t_thread_info()->flags, 0, 0); > - if (!need_resched()) > + if (!need_resched()) { > __sti_mwait(0, 0); > - else > - raw_local_irq_enable(); > - } else { > - raw_local_irq_enable(); > + raw_local_irq_disable(); > + } > } > __current_clr_polling(); > } > --- a/arch/xtensa/kernel/process.c > +++ b/arch/xtensa/kernel/process.c > @@ -183,6 +183,7 @@ void coprocessor_flush_release_all(struc > void arch_cpu_idle(void) > { > platform_idle(); > + raw_local_irq_disable(); > } > > /* > --- a/kernel/sched/idle.c > +++ b/kernel/sched/idle.c > @@ -79,7 +79,6 @@ void __weak arch_cpu_idle_dead(void) { } > void __weak arch_cpu_idle(void) > { > cpu_idle_force_poll = 1; > - raw_local_irq_enable(); > } > > /** > @@ -96,7 +95,6 @@ void __cpuidle default_idle_call(void) > > cpuidle_rcu_enter(); > arch_cpu_idle(); > - raw_local_irq_disable(); > cpuidle_rcu_exit(); > > start_critical_timings(); > > From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Rafael J. Wysocki" Date: Wed, 06 Jul 2022 14:02:52 +0000 Subject: Re: [PATCH 20/36] arch/idle: Change arch_cpu_idle() IRQ behaviour Message-Id: List-Id: References: <20220608142723.103523089@infradead.org> <20220608144517.188449351@infradead.org> In-Reply-To: <20220608144517.188449351@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Peter Zijlstra Cc: rth@twiddle.net, ink@jurassic.park.msu.ru, mattst88@gmail.com, vgupta@kernel.org, Russell King - ARM Linux , ulli.kroll@googlemail.com, Linus Walleij , Shawn Guo , Sascha Hauer , Sascha Hauer , Fabio Estevam , dl-linux-imx , Tony Lindgren , Kevin Hilman , Catalin Marinas , Will Deacon , Guo Ren , bcain@quicinc.com, Huacai Chen , kernel@xen0n.name, Geert Uytterhoeven , sammy@sammy.net, Michal Simek , Thomas Bogendoerfer , dinguyen@kernel.org, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Stafford Horne , James Bottomley , Helge Deller , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Yoshinori Sato , Rich Felker , David Miller , Richard Weinberger , anton.ivanov@cambridgegreys.com, Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , the arch/x86 maintainers , "H. Peter Anvin" , acme@kernel.org, Mark Rutland , Alexander Shishkin , jolsa@kernel.org, namhyung@kernel.org, Juergen Gross , srivatsa@csail.mit.edu, amakhalov@vmware.com, pv-drivers@vmware.com, Boris Ostrovsky , Chris Zankel , jcmvbkbc@gmail.com, "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Michael Turquette , Stephen Boyd , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , Andy Gross , Bjorn Andersson , Anup Patel , Thierry Reding , Jon Hunter , Jacob Pan , Arnd Bergmann , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Steven Rostedt , Petr Mladek , senozhatsky@chromium.org, John Ogness , "Paul E. McKenney" , Frederic Weisbecker , quic_neeraju@quicinc.com, Josh Triplett , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Benjamin Segall , Mel Gorman , Daniel Bristot de Oliveira , vschneid@redhat.com, jpoimboe@kernel.org, linux-alpha@vger.kernel.org, Linux Kernel Mailing List , linux-snps-arc@lists.infradead.org, Linux ARM , Linux OMAP Mailing List , linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, linux-m68k , "open list:BROADCOM NVRAM DRIVER" , openrisc@lists.librecores.org, Parisc List , linuxppc-dev , linux-riscv , linux-s390@vger.kernel.org, Linux-sh list , sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-perf-users@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, linux-xtensa@linux-xtensa.org, ACPI Devel Maling List , Linux PM , linux-clk , linux-arm-msm , linux-tegra , linux-arch , rcu@vger.kernel.org On Wed, Jun 8, 2022 at 4:46 PM Peter Zijlstra wrote: > > Current arch_cpu_idle() is called with IRQs disabled, but will return > with IRQs enabled. > > However, the very first thing the generic code does after calling > arch_cpu_idle() is raw_local_irq_disable(). This means that > architectures that can idle with IRQs disabled end up doing a > pointless 'enable-disable' dance. > > Therefore, push this IRQ disabling into the idle function, meaning > that those architectures can avoid the pointless IRQ state flipping. > > Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki > --- > arch/alpha/kernel/process.c | 1 - > arch/arc/kernel/process.c | 3 +++ > arch/arm/kernel/process.c | 1 - > arch/arm/mach-gemini/board-dt.c | 3 ++- > arch/arm64/kernel/idle.c | 1 - > arch/csky/kernel/process.c | 1 - > arch/csky/kernel/smp.c | 2 +- > arch/hexagon/kernel/process.c | 1 - > arch/ia64/kernel/process.c | 1 + > arch/microblaze/kernel/process.c | 1 - > arch/mips/kernel/idle.c | 8 +++----- > arch/nios2/kernel/process.c | 1 - > arch/openrisc/kernel/process.c | 1 + > arch/parisc/kernel/process.c | 2 -- > arch/powerpc/kernel/idle.c | 5 ++--- > arch/riscv/kernel/process.c | 1 - > arch/s390/kernel/idle.c | 1 - > arch/sh/kernel/idle.c | 1 + > arch/sparc/kernel/leon_pmc.c | 4 ++++ > arch/sparc/kernel/process_32.c | 1 - > arch/sparc/kernel/process_64.c | 3 ++- > arch/um/kernel/process.c | 1 - > arch/x86/coco/tdx/tdx.c | 3 +++ > arch/x86/kernel/process.c | 15 ++++----------- > arch/xtensa/kernel/process.c | 1 + > kernel/sched/idle.c | 2 -- > 26 files changed, 28 insertions(+), 37 deletions(-) > > --- a/arch/alpha/kernel/process.c > +++ b/arch/alpha/kernel/process.c > @@ -57,7 +57,6 @@ EXPORT_SYMBOL(pm_power_off); > void arch_cpu_idle(void) > { > wtint(0); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_dead(void) > --- a/arch/arc/kernel/process.c > +++ b/arch/arc/kernel/process.c > @@ -114,6 +114,8 @@ void arch_cpu_idle(void) > "sleep %0 \n" > : > :"I"(arg)); /* can't be "r" has to be embedded const */ > + > + raw_local_irq_disable(); > } > > #else /* ARC700 */ > @@ -122,6 +124,7 @@ void arch_cpu_idle(void) > { > /* sleep, but enable both set E1/E2 (levels of interrupts) before committing */ > __asm__ __volatile__("sleep 0x3 \n"); > + raw_local_irq_disable(); > } > > #endif > --- a/arch/arm/kernel/process.c > +++ b/arch/arm/kernel/process.c > @@ -78,7 +78,6 @@ void arch_cpu_idle(void) > arm_pm_idle(); > else > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_prepare(void) > --- a/arch/arm/mach-gemini/board-dt.c > +++ b/arch/arm/mach-gemini/board-dt.c > @@ -42,8 +42,9 @@ static void gemini_idle(void) > */ > > /* FIXME: Enabling interrupts here is racy! */ > - local_irq_enable(); > + raw_local_irq_enable(); > cpu_do_idle(); > + raw_local_irq_disable(); > } > > static void __init gemini_init_machine(void) > --- a/arch/arm64/kernel/idle.c > +++ b/arch/arm64/kernel/idle.c > @@ -42,5 +42,4 @@ void noinstr arch_cpu_idle(void) > * tricks > */ > cpu_do_idle(); > - raw_local_irq_enable(); > } > --- a/arch/csky/kernel/process.c > +++ b/arch/csky/kernel/process.c > @@ -101,6 +101,5 @@ void arch_cpu_idle(void) > #ifdef CONFIG_CPU_PM_STOP > asm volatile("stop\n"); > #endif > - raw_local_irq_enable(); > } > #endif > --- a/arch/csky/kernel/smp.c > +++ b/arch/csky/kernel/smp.c > @@ -314,7 +314,7 @@ void arch_cpu_idle_dead(void) > while (!secondary_stack) > arch_cpu_idle(); > > - local_irq_disable(); > + raw_local_irq_disable(); > > asm volatile( > "mov sp, %0\n" > --- a/arch/hexagon/kernel/process.c > +++ b/arch/hexagon/kernel/process.c > @@ -44,7 +44,6 @@ void arch_cpu_idle(void) > { > __vmwait(); > /* interrupts wake us up, but irqs are still disabled */ > - raw_local_irq_enable(); > } > > /* > --- a/arch/ia64/kernel/process.c > +++ b/arch/ia64/kernel/process.c > @@ -241,6 +241,7 @@ void arch_cpu_idle(void) > (*mark_idle)(1); > > raw_safe_halt(); > + raw_local_irq_disable(); > > if (mark_idle) > (*mark_idle)(0); > --- a/arch/microblaze/kernel/process.c > +++ b/arch/microblaze/kernel/process.c > @@ -138,5 +138,4 @@ int dump_fpu(struct pt_regs *regs, elf_f > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > --- a/arch/mips/kernel/idle.c > +++ b/arch/mips/kernel/idle.c > @@ -33,13 +33,13 @@ static void __cpuidle r3081_wait(void) > { > unsigned long cfg = read_c0_conf(); > write_c0_conf(cfg | R30XX_CONF_HALT); > - raw_local_irq_enable(); > } > > void __cpuidle r4k_wait(void) > { > raw_local_irq_enable(); > __r4k_wait(); > + raw_local_irq_disable(); > } > > /* > @@ -57,7 +57,6 @@ void __cpuidle r4k_wait_irqoff(void) > " .set arch=r4000 \n" > " wait \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -77,7 +76,6 @@ static void __cpuidle rm7k_wait_irqoff(v > " wait \n" > " mtc0 $1, $12 # stalls until W stage \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -103,6 +101,8 @@ static void __cpuidle au1k_wait(void) > " nop \n" > " .set pop \n" > : : "r" (au1k_wait), "r" (c0status)); > + > + raw_local_irq_disable(); > } > > static int __initdata nowait; > @@ -245,8 +245,6 @@ void arch_cpu_idle(void) > { > if (cpu_wait) > cpu_wait(); > - else > - raw_local_irq_enable(); > } > > #ifdef CONFIG_CPU_IDLE > --- a/arch/nios2/kernel/process.c > +++ b/arch/nios2/kernel/process.c > @@ -33,7 +33,6 @@ EXPORT_SYMBOL(pm_power_off); > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > > /* > --- a/arch/openrisc/kernel/process.c > +++ b/arch/openrisc/kernel/process.c > @@ -102,6 +102,7 @@ void arch_cpu_idle(void) > raw_local_irq_enable(); > if (mfspr(SPR_UPR) & SPR_UPR_PMP) > mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME); > + raw_local_irq_disable(); > } > > void (*pm_power_off)(void) = NULL; > --- a/arch/parisc/kernel/process.c > +++ b/arch/parisc/kernel/process.c > @@ -187,8 +187,6 @@ void arch_cpu_idle_dead(void) > > void __cpuidle arch_cpu_idle(void) > { > - raw_local_irq_enable(); > - > /* nop on real hardware, qemu will idle sleep. */ > asm volatile("or %%r10,%%r10,%%r10\n":::); > } > --- a/arch/powerpc/kernel/idle.c > +++ b/arch/powerpc/kernel/idle.c > @@ -51,10 +51,9 @@ void arch_cpu_idle(void) > * Some power_save functions return with > * interrupts enabled, some don't. > */ > - if (irqs_disabled()) > - raw_local_irq_enable(); > + if (!irqs_disabled()) > + raw_local_irq_disable(); > } else { > - raw_local_irq_enable(); > /* > * Go into low thread priority and possibly > * low power mode. > --- a/arch/riscv/kernel/process.c > +++ b/arch/riscv/kernel/process.c > @@ -39,7 +39,6 @@ extern asmlinkage void ret_from_kernel_t > void arch_cpu_idle(void) > { > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void __show_regs(struct pt_regs *regs) > --- a/arch/s390/kernel/idle.c > +++ b/arch/s390/kernel/idle.c > @@ -66,7 +66,6 @@ void arch_cpu_idle(void) > idle->idle_count++; > account_idle_time(cputime_to_nsecs(idle_time)); > raw_write_seqcount_end(&idle->seqcount); > - raw_local_irq_enable(); > } > > static ssize_t show_idle_count(struct device *dev, > --- a/arch/sh/kernel/idle.c > +++ b/arch/sh/kernel/idle.c > @@ -25,6 +25,7 @@ void default_idle(void) > raw_local_irq_enable(); > /* Isn't this racy ? */ > cpu_sleep(); > + raw_local_irq_disable(); > clear_bl_bit(); > } > > --- a/arch/sparc/kernel/leon_pmc.c > +++ b/arch/sparc/kernel/leon_pmc.c > @@ -57,6 +57,8 @@ static void pmc_leon_idle_fixup(void) > "lda [%0] %1, %%g0\n" > : > : "r"(address), "i"(ASI_LEON_BYPASS)); > + > + raw_local_irq_disable(); > } > > /* > @@ -70,6 +72,8 @@ static void pmc_leon_idle(void) > > /* For systems without power-down, this will be no-op */ > __asm__ __volatile__ ("wr %g0, %asr19\n\t"); > + > + raw_local_irq_disable(); > } > > /* Install LEON Power Down function */ > --- a/arch/sparc/kernel/process_32.c > +++ b/arch/sparc/kernel/process_32.c > @@ -71,7 +71,6 @@ void arch_cpu_idle(void) > { > if (sparc_idle) > (*sparc_idle)(); > - raw_local_irq_enable(); > } > > /* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */ > --- a/arch/sparc/kernel/process_64.c > +++ b/arch/sparc/kernel/process_64.c > @@ -59,7 +59,6 @@ void arch_cpu_idle(void) > { > if (tlb_type != hypervisor) { > touch_nmi_watchdog(); > - raw_local_irq_enable(); > } else { > unsigned long pstate; > > @@ -90,6 +89,8 @@ void arch_cpu_idle(void) > "wrpr %0, %%g0, %%pstate" > : "=&r" (pstate) > : "i" (PSTATE_IE)); > + > + raw_local_irq_disable(); > } > } > > --- a/arch/um/kernel/process.c > +++ b/arch/um/kernel/process.c > @@ -216,7 +216,6 @@ void arch_cpu_idle(void) > { > cpu_tasks[current_thread_info()->cpu].pid = os_getpid(); > um_idle_sleep(); > - raw_local_irq_enable(); > } > > int __cant_sleep(void) { > --- a/arch/x86/coco/tdx/tdx.c > +++ b/arch/x86/coco/tdx/tdx.c > @@ -178,6 +178,9 @@ void __cpuidle tdx_safe_halt(void) > */ > if (__halt(irq_disabled, do_sti)) > WARN_ONCE(1, "HLT instruction emulation failed\n"); > + > + /* XXX I can't make sense of what @do_sti actually does */ > + raw_local_irq_disable(); > } > > static bool read_msr(struct pt_regs *regs) > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -699,6 +699,7 @@ EXPORT_SYMBOL(boot_option_idle_override) > void __cpuidle default_idle(void) > { > raw_safe_halt(); > + raw_local_irq_disable(); > } > #if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE) > EXPORT_SYMBOL(default_idle); > @@ -804,13 +805,7 @@ static void amd_e400_idle(void) > > default_idle(); > > - /* > - * The switch back from broadcast mode needs to be called with > - * interrupts disabled. > - */ > - raw_local_irq_disable(); > tick_broadcast_exit(); > - raw_local_irq_enable(); > } > > /* > @@ -849,12 +844,10 @@ static __cpuidle void mwait_idle(void) > } > > __monitor((void *)¤t_thread_info()->flags, 0, 0); > - if (!need_resched()) > + if (!need_resched()) { > __sti_mwait(0, 0); > - else > - raw_local_irq_enable(); > - } else { > - raw_local_irq_enable(); > + raw_local_irq_disable(); > + } > } > __current_clr_polling(); > } > --- a/arch/xtensa/kernel/process.c > +++ b/arch/xtensa/kernel/process.c > @@ -183,6 +183,7 @@ void coprocessor_flush_release_all(struc > void arch_cpu_idle(void) > { > platform_idle(); > + raw_local_irq_disable(); > } > > /* > --- a/kernel/sched/idle.c > +++ b/kernel/sched/idle.c > @@ -79,7 +79,6 @@ void __weak arch_cpu_idle_dead(void) { } > void __weak arch_cpu_idle(void) > { > cpu_idle_force_poll = 1; > - raw_local_irq_enable(); > } > > /** > @@ -96,7 +95,6 @@ void __cpuidle default_idle_call(void) > > cpuidle_rcu_enter(); > arch_cpu_idle(); > - raw_local_irq_disable(); > cpuidle_rcu_exit(); > > start_critical_timings(); > > From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Rafael J. Wysocki" Subject: Re: [PATCH 20/36] arch/idle: Change arch_cpu_idle() IRQ behaviour Date: Wed, 6 Jul 2022 16:02:52 +0200 Message-ID: References: <20220608142723.103523089@infradead.org> <20220608144517.188449351@infradead.org> Mime-Version: 1.0 Return-path: In-Reply-To: <20220608144517.188449351@infradead.org> List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Peter Zijlstra Cc: rth@twiddle.net, ink@jurassic.park.msu.ru, mattst88@gmail.com, vgupta@kernel.org, Russell King - ARM Linux , ulli.kroll@googlemail.com, Linus Walleij , Shawn Guo , Sascha Hauer , Sascha Hauer , Fabio Estevam , dl-linux-imx , Tony Lindgren , Kevin Hilman , Catalin Marinas , Will Deacon , Guo Ren , bcain@quicinc.com, Huacai Chen , kernel@xen0n.name, Geert Uytterhoeven , sammy@sammy.net, Michal Simek , Thomas Bogendoerfer On Wed, Jun 8, 2022 at 4:46 PM Peter Zijlstra wrote: > > Current arch_cpu_idle() is called with IRQs disabled, but will return > with IRQs enabled. > > However, the very first thing the generic code does after calling > arch_cpu_idle() is raw_local_irq_disable(). This means that > architectures that can idle with IRQs disabled end up doing a > pointless 'enable-disable' dance. > > Therefore, push this IRQ disabling into the idle function, meaning > that those architectures can avoid the pointless IRQ state flipping. > > Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki > --- > arch/alpha/kernel/process.c | 1 - > arch/arc/kernel/process.c | 3 +++ > arch/arm/kernel/process.c | 1 - > arch/arm/mach-gemini/board-dt.c | 3 ++- > arch/arm64/kernel/idle.c | 1 - > arch/csky/kernel/process.c | 1 - > arch/csky/kernel/smp.c | 2 +- > arch/hexagon/kernel/process.c | 1 - > arch/ia64/kernel/process.c | 1 + > arch/microblaze/kernel/process.c | 1 - > arch/mips/kernel/idle.c | 8 +++----- > arch/nios2/kernel/process.c | 1 - > arch/openrisc/kernel/process.c | 1 + > arch/parisc/kernel/process.c | 2 -- > arch/powerpc/kernel/idle.c | 5 ++--- > arch/riscv/kernel/process.c | 1 - > arch/s390/kernel/idle.c | 1 - > arch/sh/kernel/idle.c | 1 + > arch/sparc/kernel/leon_pmc.c | 4 ++++ > arch/sparc/kernel/process_32.c | 1 - > arch/sparc/kernel/process_64.c | 3 ++- > arch/um/kernel/process.c | 1 - > arch/x86/coco/tdx/tdx.c | 3 +++ > arch/x86/kernel/process.c | 15 ++++----------- > arch/xtensa/kernel/process.c | 1 + > kernel/sched/idle.c | 2 -- > 26 files changed, 28 insertions(+), 37 deletions(-) > > --- a/arch/alpha/kernel/process.c > +++ b/arch/alpha/kernel/process.c > @@ -57,7 +57,6 @@ EXPORT_SYMBOL(pm_power_off); > void arch_cpu_idle(void) > { > wtint(0); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_dead(void) > --- a/arch/arc/kernel/process.c > +++ b/arch/arc/kernel/process.c > @@ -114,6 +114,8 @@ void arch_cpu_idle(void) > "sleep %0 \n" > : > :"I"(arg)); /* can't be "r" has to be embedded const */ > + > + raw_local_irq_disable(); > } > > #else /* ARC700 */ > @@ -122,6 +124,7 @@ void arch_cpu_idle(void) > { > /* sleep, but enable both set E1/E2 (levels of interrupts) before committing */ > __asm__ __volatile__("sleep 0x3 \n"); > + raw_local_irq_disable(); > } > > #endif > --- a/arch/arm/kernel/process.c > +++ b/arch/arm/kernel/process.c > @@ -78,7 +78,6 @@ void arch_cpu_idle(void) > arm_pm_idle(); > else > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void arch_cpu_idle_prepare(void) > --- a/arch/arm/mach-gemini/board-dt.c > +++ b/arch/arm/mach-gemini/board-dt.c > @@ -42,8 +42,9 @@ static void gemini_idle(void) > */ > > /* FIXME: Enabling interrupts here is racy! */ > - local_irq_enable(); > + raw_local_irq_enable(); > cpu_do_idle(); > + raw_local_irq_disable(); > } > > static void __init gemini_init_machine(void) > --- a/arch/arm64/kernel/idle.c > +++ b/arch/arm64/kernel/idle.c > @@ -42,5 +42,4 @@ void noinstr arch_cpu_idle(void) > * tricks > */ > cpu_do_idle(); > - raw_local_irq_enable(); > } > --- a/arch/csky/kernel/process.c > +++ b/arch/csky/kernel/process.c > @@ -101,6 +101,5 @@ void arch_cpu_idle(void) > #ifdef CONFIG_CPU_PM_STOP > asm volatile("stop\n"); > #endif > - raw_local_irq_enable(); > } > #endif > --- a/arch/csky/kernel/smp.c > +++ b/arch/csky/kernel/smp.c > @@ -314,7 +314,7 @@ void arch_cpu_idle_dead(void) > while (!secondary_stack) > arch_cpu_idle(); > > - local_irq_disable(); > + raw_local_irq_disable(); > > asm volatile( > "mov sp, %0\n" > --- a/arch/hexagon/kernel/process.c > +++ b/arch/hexagon/kernel/process.c > @@ -44,7 +44,6 @@ void arch_cpu_idle(void) > { > __vmwait(); > /* interrupts wake us up, but irqs are still disabled */ > - raw_local_irq_enable(); > } > > /* > --- a/arch/ia64/kernel/process.c > +++ b/arch/ia64/kernel/process.c > @@ -241,6 +241,7 @@ void arch_cpu_idle(void) > (*mark_idle)(1); > > raw_safe_halt(); > + raw_local_irq_disable(); > > if (mark_idle) > (*mark_idle)(0); > --- a/arch/microblaze/kernel/process.c > +++ b/arch/microblaze/kernel/process.c > @@ -138,5 +138,4 @@ int dump_fpu(struct pt_regs *regs, elf_f > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > --- a/arch/mips/kernel/idle.c > +++ b/arch/mips/kernel/idle.c > @@ -33,13 +33,13 @@ static void __cpuidle r3081_wait(void) > { > unsigned long cfg = read_c0_conf(); > write_c0_conf(cfg | R30XX_CONF_HALT); > - raw_local_irq_enable(); > } > > void __cpuidle r4k_wait(void) > { > raw_local_irq_enable(); > __r4k_wait(); > + raw_local_irq_disable(); > } > > /* > @@ -57,7 +57,6 @@ void __cpuidle r4k_wait_irqoff(void) > " .set arch=r4000 \n" > " wait \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -77,7 +76,6 @@ static void __cpuidle rm7k_wait_irqoff(v > " wait \n" > " mtc0 $1, $12 # stalls until W stage \n" > " .set pop \n"); > - raw_local_irq_enable(); > } > > /* > @@ -103,6 +101,8 @@ static void __cpuidle au1k_wait(void) > " nop \n" > " .set pop \n" > : : "r" (au1k_wait), "r" (c0status)); > + > + raw_local_irq_disable(); > } > > static int __initdata nowait; > @@ -245,8 +245,6 @@ void arch_cpu_idle(void) > { > if (cpu_wait) > cpu_wait(); > - else > - raw_local_irq_enable(); > } > > #ifdef CONFIG_CPU_IDLE > --- a/arch/nios2/kernel/process.c > +++ b/arch/nios2/kernel/process.c > @@ -33,7 +33,6 @@ EXPORT_SYMBOL(pm_power_off); > > void arch_cpu_idle(void) > { > - raw_local_irq_enable(); > } > > /* > --- a/arch/openrisc/kernel/process.c > +++ b/arch/openrisc/kernel/process.c > @@ -102,6 +102,7 @@ void arch_cpu_idle(void) > raw_local_irq_enable(); > if (mfspr(SPR_UPR) & SPR_UPR_PMP) > mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME); > + raw_local_irq_disable(); > } > > void (*pm_power_off)(void) = NULL; > --- a/arch/parisc/kernel/process.c > +++ b/arch/parisc/kernel/process.c > @@ -187,8 +187,6 @@ void arch_cpu_idle_dead(void) > > void __cpuidle arch_cpu_idle(void) > { > - raw_local_irq_enable(); > - > /* nop on real hardware, qemu will idle sleep. */ > asm volatile("or %%r10,%%r10,%%r10\n":::); > } > --- a/arch/powerpc/kernel/idle.c > +++ b/arch/powerpc/kernel/idle.c > @@ -51,10 +51,9 @@ void arch_cpu_idle(void) > * Some power_save functions return with > * interrupts enabled, some don't. > */ > - if (irqs_disabled()) > - raw_local_irq_enable(); > + if (!irqs_disabled()) > + raw_local_irq_disable(); > } else { > - raw_local_irq_enable(); > /* > * Go into low thread priority and possibly > * low power mode. > --- a/arch/riscv/kernel/process.c > +++ b/arch/riscv/kernel/process.c > @@ -39,7 +39,6 @@ extern asmlinkage void ret_from_kernel_t > void arch_cpu_idle(void) > { > cpu_do_idle(); > - raw_local_irq_enable(); > } > > void __show_regs(struct pt_regs *regs) > --- a/arch/s390/kernel/idle.c > +++ b/arch/s390/kernel/idle.c > @@ -66,7 +66,6 @@ void arch_cpu_idle(void) > idle->idle_count++; > account_idle_time(cputime_to_nsecs(idle_time)); > raw_write_seqcount_end(&idle->seqcount); > - raw_local_irq_enable(); > } > > static ssize_t show_idle_count(struct device *dev, > --- a/arch/sh/kernel/idle.c > +++ b/arch/sh/kernel/idle.c > @@ -25,6 +25,7 @@ void default_idle(void) > raw_local_irq_enable(); > /* Isn't this racy ? */ > cpu_sleep(); > + raw_local_irq_disable(); > clear_bl_bit(); > } > > --- a/arch/sparc/kernel/leon_pmc.c > +++ b/arch/sparc/kernel/leon_pmc.c > @@ -57,6 +57,8 @@ static void pmc_leon_idle_fixup(void) > "lda [%0] %1, %%g0\n" > : > : "r"(address), "i"(ASI_LEON_BYPASS)); > + > + raw_local_irq_disable(); > } > > /* > @@ -70,6 +72,8 @@ static void pmc_leon_idle(void) > > /* For systems without power-down, this will be no-op */ > __asm__ __volatile__ ("wr %g0, %asr19\n\t"); > + > + raw_local_irq_disable(); > } > > /* Install LEON Power Down function */ > --- a/arch/sparc/kernel/process_32.c > +++ b/arch/sparc/kernel/process_32.c > @@ -71,7 +71,6 @@ void arch_cpu_idle(void) > { > if (sparc_idle) > (*sparc_idle)(); > - raw_local_irq_enable(); > } > > /* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */ > --- a/arch/sparc/kernel/process_64.c > +++ b/arch/sparc/kernel/process_64.c > @@ -59,7 +59,6 @@ void arch_cpu_idle(void) > { > if (tlb_type != hypervisor) { > touch_nmi_watchdog(); > - raw_local_irq_enable(); > } else { > unsigned long pstate; > > @@ -90,6 +89,8 @@ void arch_cpu_idle(void) > "wrpr %0, %%g0, %%pstate" > : "=&r" (pstate) > : "i" (PSTATE_IE)); > + > + raw_local_irq_disable(); > } > } > > --- a/arch/um/kernel/process.c > +++ b/arch/um/kernel/process.c > @@ -216,7 +216,6 @@ void arch_cpu_idle(void) > { > cpu_tasks[current_thread_info()->cpu].pid = os_getpid(); > um_idle_sleep(); > - raw_local_irq_enable(); > } > > int __cant_sleep(void) { > --- a/arch/x86/coco/tdx/tdx.c > +++ b/arch/x86/coco/tdx/tdx.c > @@ -178,6 +178,9 @@ void __cpuidle tdx_safe_halt(void) > */ > if (__halt(irq_disabled, do_sti)) > WARN_ONCE(1, "HLT instruction emulation failed\n"); > + > + /* XXX I can't make sense of what @do_sti actually does */ > + raw_local_irq_disable(); > } > > static bool read_msr(struct pt_regs *regs) > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -699,6 +699,7 @@ EXPORT_SYMBOL(boot_option_idle_override) > void __cpuidle default_idle(void) > { > raw_safe_halt(); > + raw_local_irq_disable(); > } > #if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE) > EXPORT_SYMBOL(default_idle); > @@ -804,13 +805,7 @@ static void amd_e400_idle(void) > > default_idle(); > > - /* > - * The switch back from broadcast mode needs to be called with > - * interrupts disabled. > - */ > - raw_local_irq_disable(); > tick_broadcast_exit(); > - raw_local_irq_enable(); > } > > /* > @@ -849,12 +844,10 @@ static __cpuidle void mwait_idle(void) > } > > __monitor((void *)¤t_thread_info()->flags, 0, 0); > - if (!need_resched()) > + if (!need_resched()) { > __sti_mwait(0, 0); > - else > - raw_local_irq_enable(); > - } else { > - raw_local_irq_enable(); > + raw_local_irq_disable(); > + } > } > __current_clr_polling(); > } > --- a/arch/xtensa/kernel/process.c > +++ b/arch/xtensa/kernel/process.c > @@ -183,6 +183,7 @@ void coprocessor_flush_release_all(struc > void arch_cpu_idle(void) > { > platform_idle(); > + raw_local_irq_disable(); > } > > /* > --- a/kernel/sched/idle.c > +++ b/kernel/sched/idle.c > @@ -79,7 +79,6 @@ void __weak arch_cpu_idle_dead(void) { } > void __weak arch_cpu_idle(void) > { > cpu_idle_force_poll = 1; > - raw_local_irq_enable(); > } > > /** > @@ -96,7 +95,6 @@ void __cpuidle default_idle_call(void) > > cpuidle_rcu_enter(); > arch_cpu_idle(); > - raw_local_irq_disable(); > cpuidle_rcu_exit(); > > start_critical_timings(); > >