From: Mark Rutland <mark.rutland@arm.com> To: Peter Zijlstra <peterz@infradead.org> Cc: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, luto@kernel.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>, Deep Shah <sdeep@vmware.com>, "VMware, Inc." <pv-drivers@vmware.com>, Boris Ostrovsky <boris.ostrovsky@oracle.com>, Stefano Stabellini <sstabellini@kernel.org> Subject: Re: [PATCH v2 05/12] x86: rework arch_local_irq_restore() to not use popf Date: Wed, 9 Dec 2020 18:15:14 +0000 [thread overview] Message-ID: <20201209181514.GA14235@C02TD0UTHF1T.local> (raw) In-Reply-To: <20201120115943.GD3021@hirez.programming.kicks-ass.net> On Fri, Nov 20, 2020 at 12:59:43PM +0100, Peter Zijlstra wrote: > On Fri, Nov 20, 2020 at 12:46:23PM +0100, Juergen Gross wrote: > > +static __always_inline void arch_local_irq_restore(unsigned long flags) > > +{ > > + if (!arch_irqs_disabled_flags(flags)) > > + arch_local_irq_enable(); > > +} > > If someone were to write horrible code like: > > local_irq_disable(); > local_irq_save(flags); > local_irq_enable(); > local_irq_restore(flags); > > we'd be up some creek without a paddle... now I don't _think_ we have > genius code like that, but I'd feel saver if we can haz an assertion in > there somewhere... I've cobbled that together locally (i'll post it momentarily), and gave it a spin on both arm64 and x86, whereupon it exploded at boot time on x86. In arch/x86/kernel/apic/io_apic.c's timer_irq_works() we do: local_irq_save(flags); local_irq_enable(); [ trigger an IRQ here ] local_irq_restore(flags); ... and in check_timer() we call that a number of times after either a local_irq_save() or local_irq_disable(), eventually trailing with a local_irq_disable() that will balance things up before calling local_irq_restore(). I guess that timer_irq_works() should instead do: local_irq_save(flags); local_irq_enable(); ... local_irq_disable(); local_irq_restore(flags); ... assuming we consider that legitimate? With that, and all the calls to local_irq_disable() in check_timer() removed (diff below) I get a clean boot under QEMU with the assertion hacked in and DEBUG_LOCKDEP enabled. Thanks Mark. ---->8---- diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c index 7b3c7e0d4a09..e79e665a3aeb 100644 --- a/arch/x86/kernel/apic/io_apic.c +++ b/arch/x86/kernel/apic/io_apic.c @@ -1631,6 +1631,7 @@ static int __init timer_irq_works(void) else delay_without_tsc(); + local_irq_disable(); local_irq_restore(flags); /* @@ -2191,7 +2192,6 @@ static inline void __init check_timer(void) goto out; } panic_if_irq_remap("timer doesn't work through Interrupt-remapped IO-APIC"); - local_irq_disable(); clear_IO_APIC_pin(apic1, pin1); if (!no_pin1) apic_printk(APIC_QUIET, KERN_ERR "..MP-BIOS bug: " @@ -2215,7 +2215,6 @@ static inline void __init check_timer(void) /* * Cleanup, just in case ... */ - local_irq_disable(); legacy_pic->mask(0); clear_IO_APIC_pin(apic2, pin2); apic_printk(APIC_QUIET, KERN_INFO "....... failed.\n"); @@ -2232,7 +2231,6 @@ static inline void __init check_timer(void) apic_printk(APIC_QUIET, KERN_INFO "..... works.\n"); goto out; } - local_irq_disable(); legacy_pic->mask(0); apic_write(APIC_LVT0, APIC_LVT_MASKED | APIC_DM_FIXED | cfg->vector); apic_printk(APIC_QUIET, KERN_INFO "..... failed.\n"); @@ -2251,7 +2249,6 @@ static inline void __init check_timer(void) apic_printk(APIC_QUIET, KERN_INFO "..... works.\n"); goto out; } - local_irq_disable(); apic_printk(APIC_QUIET, KERN_INFO "..... failed :(.\n"); if (apic_is_x2apic_enabled()) apic_printk(APIC_QUIET, KERN_INFO
WARNING: multiple messages have this Message-ID (diff)
From: Mark Rutland <mark.rutland@arm.com> To: Peter Zijlstra <peterz@infradead.org> Cc: Juergen Gross <jgross@suse.com>, Stefano Stabellini <sstabellini@kernel.org>, "VMware, Inc." <pv-drivers@vmware.com>, x86@kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, luto@kernel.org, "H. Peter Anvin" <hpa@zytor.com>, xen-devel@lists.xenproject.org, Thomas Gleixner <tglx@linutronix.de>, Boris Ostrovsky <boris.ostrovsky@oracle.com> Subject: Re: [PATCH v2 05/12] x86: rework arch_local_irq_restore() to not use popf Date: Wed, 9 Dec 2020 18:15:14 +0000 [thread overview] Message-ID: <20201209181514.GA14235@C02TD0UTHF1T.local> (raw) In-Reply-To: <20201120115943.GD3021@hirez.programming.kicks-ass.net> On Fri, Nov 20, 2020 at 12:59:43PM +0100, Peter Zijlstra wrote: > On Fri, Nov 20, 2020 at 12:46:23PM +0100, Juergen Gross wrote: > > +static __always_inline void arch_local_irq_restore(unsigned long flags) > > +{ > > + if (!arch_irqs_disabled_flags(flags)) > > + arch_local_irq_enable(); > > +} > > If someone were to write horrible code like: > > local_irq_disable(); > local_irq_save(flags); > local_irq_enable(); > local_irq_restore(flags); > > we'd be up some creek without a paddle... now I don't _think_ we have > genius code like that, but I'd feel saver if we can haz an assertion in > there somewhere... I've cobbled that together locally (i'll post it momentarily), and gave it a spin on both arm64 and x86, whereupon it exploded at boot time on x86. In arch/x86/kernel/apic/io_apic.c's timer_irq_works() we do: local_irq_save(flags); local_irq_enable(); [ trigger an IRQ here ] local_irq_restore(flags); ... and in check_timer() we call that a number of times after either a local_irq_save() or local_irq_disable(), eventually trailing with a local_irq_disable() that will balance things up before calling local_irq_restore(). I guess that timer_irq_works() should instead do: local_irq_save(flags); local_irq_enable(); ... local_irq_disable(); local_irq_restore(flags); ... assuming we consider that legitimate? With that, and all the calls to local_irq_disable() in check_timer() removed (diff below) I get a clean boot under QEMU with the assertion hacked in and DEBUG_LOCKDEP enabled. Thanks Mark. ---->8---- diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c index 7b3c7e0d4a09..e79e665a3aeb 100644 --- a/arch/x86/kernel/apic/io_apic.c +++ b/arch/x86/kernel/apic/io_apic.c @@ -1631,6 +1631,7 @@ static int __init timer_irq_works(void) else delay_without_tsc(); + local_irq_disable(); local_irq_restore(flags); /* @@ -2191,7 +2192,6 @@ static inline void __init check_timer(void) goto out; } panic_if_irq_remap("timer doesn't work through Interrupt-remapped IO-APIC"); - local_irq_disable(); clear_IO_APIC_pin(apic1, pin1); if (!no_pin1) apic_printk(APIC_QUIET, KERN_ERR "..MP-BIOS bug: " @@ -2215,7 +2215,6 @@ static inline void __init check_timer(void) /* * Cleanup, just in case ... */ - local_irq_disable(); legacy_pic->mask(0); clear_IO_APIC_pin(apic2, pin2); apic_printk(APIC_QUIET, KERN_INFO "....... failed.\n"); @@ -2232,7 +2231,6 @@ static inline void __init check_timer(void) apic_printk(APIC_QUIET, KERN_INFO "..... works.\n"); goto out; } - local_irq_disable(); legacy_pic->mask(0); apic_write(APIC_LVT0, APIC_LVT_MASKED | APIC_DM_FIXED | cfg->vector); apic_printk(APIC_QUIET, KERN_INFO "..... failed.\n"); @@ -2251,7 +2249,6 @@ static inline void __init check_timer(void) apic_printk(APIC_QUIET, KERN_INFO "..... works.\n"); goto out; } - local_irq_disable(); apic_printk(APIC_QUIET, KERN_INFO "..... failed :(.\n"); if (apic_is_x2apic_enabled()) apic_printk(APIC_QUIET, KERN_INFO _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2020-12-09 18:16 UTC|newest] Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-20 11:46 [PATCH v2 00/12] x86: major paravirt cleanup Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-20 11:46 ` [PATCH v2 01/12] x86/xen: use specific Xen pv interrupt entry for MCE Juergen Gross 2020-12-09 21:03 ` Thomas Gleixner 2020-11-20 11:46 ` [PATCH v2 02/12] x86/xen: use specific Xen pv interrupt entry for DF Juergen Gross 2020-12-09 21:03 ` Thomas Gleixner 2020-11-20 11:46 ` [PATCH v2 03/12] x86/pv: switch SWAPGS to ALTERNATIVE Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-27 11:31 ` Borislav Petkov 2020-11-27 11:31 ` Borislav Petkov 2020-12-09 21:07 ` Thomas Gleixner 2020-12-09 21:07 ` Thomas Gleixner 2020-11-20 11:46 ` [PATCH v2 04/12] x86/xen: drop USERGS_SYSRET64 paravirt call Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-12-02 12:32 ` Borislav Petkov 2020-12-02 12:32 ` Borislav Petkov 2020-12-02 14:48 ` Jürgen Groß 2020-12-02 14:48 ` Jürgen Groß via Virtualization 2020-12-02 17:08 ` Borislav Petkov 2020-12-02 17:08 ` Borislav Petkov 2020-11-20 11:46 ` [PATCH v2 05/12] x86: rework arch_local_irq_restore() to not use popf Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-20 11:59 ` Peter Zijlstra 2020-11-20 11:59 ` Peter Zijlstra 2020-11-20 12:05 ` Jürgen Groß 2020-11-20 12:05 ` Jürgen Groß via Virtualization 2020-11-22 6:55 ` Jürgen Groß 2020-11-22 6:55 ` Jürgen Groß via Virtualization 2020-11-22 21:44 ` Andy Lutomirski 2020-11-22 21:44 ` Andy Lutomirski 2020-11-22 21:44 ` Andy Lutomirski 2020-11-23 5:21 ` Jürgen Groß 2020-11-23 5:21 ` Jürgen Groß via Virtualization 2020-11-23 15:15 ` Andy Lutomirski 2020-11-23 15:15 ` Andy Lutomirski 2020-12-09 13:27 ` Mark Rutland 2020-12-09 13:27 ` Mark Rutland 2020-12-09 14:02 ` Mark Rutland 2020-12-09 14:02 ` Mark Rutland 2020-12-09 14:05 ` Jürgen Groß 2020-12-09 14:05 ` Jürgen Groß via Virtualization 2020-12-09 18:15 ` Mark Rutland [this message] 2020-12-09 18:15 ` Mark Rutland 2020-12-09 18:54 ` Thomas Gleixner 2020-12-09 18:54 ` Thomas Gleixner 2020-12-10 11:10 ` Mark Rutland 2020-12-10 11:10 ` Mark Rutland 2020-12-10 20:15 ` x86/ioapic: Cleanup the timer_works() irqflags mess Thomas Gleixner 2020-12-10 20:15 ` Thomas Gleixner 2020-12-10 22:04 ` [tip: x86/apic] " tip-bot2 for Thomas Gleixner 2020-12-11 5:10 ` Jürgen Groß 2020-12-11 5:10 ` Jürgen Groß via Virtualization 2020-11-27 2:20 ` [x86] 97e8f0134a: fio.write_iops 8.6% improvement kernel test robot 2020-11-20 11:46 ` [PATCH v2 06/12] x86/paravirt: switch time pvops functions to use static_call() Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-20 12:01 ` Peter Zijlstra 2020-11-20 12:01 ` Peter Zijlstra 2020-11-20 12:07 ` Jürgen Groß 2020-11-20 12:07 ` Jürgen Groß via Virtualization 2020-11-20 11:46 ` [PATCH v2 07/12] x86: add new features for paravirt patching Juergen Gross 2020-12-08 18:43 ` Borislav Petkov 2020-12-09 7:30 ` Jürgen Groß 2020-12-09 12:03 ` Borislav Petkov 2020-12-09 12:22 ` Jürgen Groß 2020-12-10 17:58 ` Borislav Petkov 2020-11-20 11:46 ` [PATCH v2 08/12] x86/paravirt: remove no longer needed 32-bit pvops cruft Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-20 12:08 ` Peter Zijlstra 2020-11-20 12:08 ` Peter Zijlstra 2020-11-20 12:16 ` Jürgen Groß 2020-11-20 12:16 ` Jürgen Groß via Virtualization 2020-11-20 16:52 ` Arvind Sankar 2020-11-20 11:46 ` [PATCH v2 09/12] x86/paravirt: switch iret pvops to ALTERNATIVE Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-20 11:46 ` [PATCH v2 10/12] x86/paravirt: add new macros PVOP_ALT* supporting pvops in ALTERNATIVEs Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-20 11:46 ` [PATCH v2 11/12] x86/paravirt: switch functions with custom code to ALTERNATIVE Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-20 15:46 ` kernel test robot 2020-11-20 15:46 ` kernel test robot 2020-11-20 15:46 ` kernel test robot 2020-11-25 15:46 ` [x86/paravirt] fd8d46a7a2: kernel-selftests.livepatch.test-callbacks.sh.fail kernel test robot 2020-11-20 11:46 ` [PATCH v2 12/12] x86/paravirt: have only one paravirt patch function Juergen Gross 2020-11-20 11:46 ` Juergen Gross via Virtualization 2020-11-20 14:18 ` kernel test robot 2020-11-20 14:18 ` kernel test robot 2020-11-20 14:18 ` kernel test robot 2020-11-20 12:53 ` [PATCH v2 00/12] x86: major paravirt cleanup Peter Zijlstra 2020-11-20 12:53 ` Peter Zijlstra 2020-11-23 13:43 ` Peter Zijlstra 2020-11-23 13:43 ` Peter Zijlstra 2020-12-15 11:42 ` Jürgen Groß 2020-12-15 11:42 ` Jürgen Groß via Virtualization 2020-12-15 14:18 ` Peter Zijlstra 2020-12-15 14:18 ` Peter Zijlstra 2020-12-15 14:54 ` Peter Zijlstra 2020-12-15 14:54 ` Peter Zijlstra 2020-12-15 15:07 ` Jürgen Groß 2020-12-15 15:07 ` Jürgen Groß via Virtualization 2020-12-16 0:38 ` Josh Poimboeuf 2020-12-16 0:38 ` Josh Poimboeuf 2020-12-16 8:40 ` Peter Zijlstra 2020-12-16 8:40 ` Peter Zijlstra 2020-12-16 16:56 ` Josh Poimboeuf 2020-12-16 16:56 ` Josh Poimboeuf 2020-12-16 17:58 ` Peter Zijlstra 2020-12-16 17:58 ` Peter Zijlstra
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20201209181514.GA14235@C02TD0UTHF1T.local \ --to=mark.rutland@arm.com \ --cc=boris.ostrovsky@oracle.com \ --cc=bp@alien8.de \ --cc=hpa@zytor.com \ --cc=jgross@suse.com \ --cc=linux-kernel@vger.kernel.org \ --cc=luto@kernel.org \ --cc=mingo@redhat.com \ --cc=peterz@infradead.org \ --cc=pv-drivers@vmware.com \ --cc=sdeep@vmware.com \ --cc=sstabellini@kernel.org \ --cc=tglx@linutronix.de \ --cc=virtualization@lists.linux-foundation.org \ --cc=x86@kernel.org \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.