[patch 00/41] x86: Cleanup IDT code

[patch 00/41] x86: Cleanup IDT code

[Thomas Gleixner]

The IDT code in x86 is a convoluted mess and contains quite some
bogosities.

One of the main offenders is the extra tracing IDT which has been
introduced to reduce the impact of tracing on the exception vectors when
tracepoints are disabled.

This results in an unholy macro mess which magically creates IDT entries by
prepending the exception handler name by trace_.

For most exceptions the tracepoint overhead in the disabled case is two
NOP5, which are negligible compared to the heavy work of the exception
handler itself. Modern CPUs are pretty good in optimizing NOPs.

The only two handlers where the overhead matters are the pagefault handler
and the reschedule IPI, where an active tracepoint introduces more extra
work. These two can be handled with separate static keys which reduces the
impact to a minimum.

As a result the whole tracing IDT machinery can be removed.

The second part of the series cleans up the handling of IDT/GDT/LDT/TSS
descriptors to get rid of macro based initialization via the obscure raw
32bit union fields a and b.

The last part moves the setup of the IDT into a separate file, where all
gate related code is moved to. The gate setup is converted to a table based
mechanism so the resulting code is smaller and better understandable.

The series applies on top of

 git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/apic

and is available as a git branch from

 git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.x86/apic

There is more cleanup to come in the near future, which aims at replacing
the braindamaged vector assignment mechanics, which has quite some
shortcomings.

Thanks,

	tglx
---
 arch/x86/boot/compressed/eboot.c         |    8 
 arch/x86/entry/entry_32.S                |   20 -
 arch/x86/entry/entry_64.S                |   26 --
 arch/x86/entry/vdso/vma.c                |    2 
 arch/x86/include/asm/desc.h              |  246 +++-----------------
 arch/x86/include/asm/desc_defs.h         |  122 +++++-----
 arch/x86/include/asm/entry_arch.h        |   14 -
 arch/x86/include/asm/hw_irq.h            |   20 -
 arch/x86/include/asm/irq.h               |    4 
 arch/x86/include/asm/segment.h           |    4 
 arch/x86/include/asm/trace/exceptions.h  |    8 
 arch/x86/include/asm/trace/irq_vectors.h |   24 +-
 arch/x86/include/asm/traps.h             |   10 
 arch/x86/include/asm/xen/hypercall.h     |    6 
 arch/x86/kernel/Makefile                 |    3 
 arch/x86/kernel/apic/apic.c              |   70 +----
 arch/x86/kernel/apic/vector.c            |    2 
 arch/x86/kernel/cpu/common.c             |    9 
 arch/x86/kernel/cpu/mcheck/mce_amd.c     |   16 -
 arch/x86/kernel/cpu/mcheck/therm_throt.c |   20 -
 arch/x86/kernel/cpu/mcheck/threshold.c   |   16 -
 arch/x86/kernel/cpu/mshyperv.c           |    9 
 arch/x86/kernel/head32.c                 |    4 
 arch/x86/kernel/head64.c                 |    6 
 arch/x86/kernel/head_32.S                |   42 ---
 arch/x86/kernel/irq.c                    |   29 --
 arch/x86/kernel/irq_work.c               |   16 -
 arch/x86/kernel/irqinit.c                |  102 --------
 arch/x86/kernel/kvm.c                    |    4 
 arch/x86/kernel/machine_kexec_32.c       |   14 -
 arch/x86/kernel/reboot.c                 |    4 
 arch/x86/kernel/setup.c                  |    4 
 arch/x86/kernel/setup_percpu.c           |    9 
 arch/x86/kernel/smp.c                    |   81 +-----
 arch/x86/kernel/tls.c                    |    2 
 arch/x86/kernel/tracepoint.c             |   57 +---
 arch/x86/kernel/traps.c                  |  108 ---------
 arch/x86/kvm/vmx.c                       |    2 
 arch/x86/math-emu/fpu_entry.c            |   11 
 arch/x86/math-emu/fpu_system.h           |   48 +++-
 arch/x86/math-emu/get_address.c          |   17 -
 arch/x86/mm/fault.c                      |   47 +--
 arch/x86/xen/enlighten_pv.c              |   14 -
 b/arch/x86/include/asm/trace/common.h    |   16 +
 b/arch/x86/kernel/eisa.c                 |   18 +
 b/arch/x86/kernel/idt.c                  |  365 +++++++++++++++++++++++++++++++
 drivers/xen/events/events_base.c         |    6 
 47 files changed, 718 insertions(+), 967 deletions(-)

[patch 01/41] x86/irq: Remove vector_used_by_percpu_irq()

[Thomas Gleixner]

[-- Attachment #1: x86-irq--Remove-vector_used_by_percpu_irq--.patch --]
[-- Type: text/plain, Size: 1314 bytes --]

Last user (lguest) is gone. Remove it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/irq.h |    1 -
 arch/x86/kernel/irq.c      |    2 --
 arch/x86/kernel/irqinit.c  |   12 ------------
 3 files changed, 15 deletions(-)

--- a/arch/x86/include/asm/irq.h
+++ b/arch/x86/include/asm/irq.h
@@ -44,7 +44,6 @@ extern __visible unsigned int do_IRQ(str
 
 /* Interrupt vector management */
 extern DECLARE_BITMAP(used_vectors, NR_VECTORS);
-extern int vector_used_by_percpu_irq(unsigned int vector);
 
 extern void init_ISA_irqs(void);
 
--- a/arch/x86/kernel/irq.c
+++ b/arch/x86/kernel/irq.c
@@ -346,8 +346,6 @@ EXPORT_SYMBOL_GPL(kvm_set_posted_intr_wa
 	set_irq_regs(old_regs);
 }
 
-EXPORT_SYMBOL_GPL(vector_used_by_percpu_irq);
-
 #ifdef CONFIG_HOTPLUG_CPU
 
 /* These two declarations are only used in check_irq_vectors_for_cpu_disable()
--- a/arch/x86/kernel/irqinit.c
+++ b/arch/x86/kernel/irqinit.c
@@ -55,18 +55,6 @@ DEFINE_PER_CPU(vector_irq_t, vector_irq)
 	[0 ... NR_VECTORS - 1] = VECTOR_UNUSED,
 };
 
-int vector_used_by_percpu_irq(unsigned int vector)
-{
-	int cpu;
-
-	for_each_online_cpu(cpu) {
-		if (!IS_ERR_OR_NULL(per_cpu(vector_irq, cpu)[vector]))
-			return 1;
-	}
-
-	return 0;
-}
-
 void __init init_ISA_irqs(void)
 {
 	struct irq_chip *chip = legacy_pic->chip;

[patch 02/41] x86/irq: Unexport used_vectors

[Thomas Gleixner]

[-- Attachment #1: x86-irq--Unexport-used_vectors.patch --]
[-- Type: text/plain, Size: 461 bytes --]

No modular users.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/traps.c |    1 -
 1 file changed, 1 deletion(-)

--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -83,7 +83,6 @@ gate_desc debug_idt_table[NR_VECTORS] __
 gate_desc idt_table[NR_VECTORS] __page_aligned_bss;
 
 DECLARE_BITMAP(used_vectors, NR_VECTORS);
-EXPORT_SYMBOL_GPL(used_vectors);
 
 static inline void cond_local_irq_enable(struct pt_regs *regs)
 {

[patch 03/41] x86/irq: Move ifdeffery to header file

[Thomas Gleixner]

[-- Attachment #1: x86-irq--Move-ifdeffery-to-header-file.patch --]
[-- Type: text/plain, Size: 1084 bytes --]

Having this in the middle of code is beyond ugly.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |    5 +++++
 arch/x86/kernel/irqinit.c   |    3 ---
 2 files changed, 5 insertions(+), 3 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -482,7 +482,12 @@ static inline void _set_gate(int gate, u
 				0, 0, __KERNEL_CS);			\
 	} while (0)
 
+#ifdef CONFIG_X86_LOCAL_APIC
 extern int first_system_vector;
+#else
+#define first_system_vector		NR_VECTORS
+#endif
+
 /* used_vectors is BITMAP for irq is not managed by percpu vector_irq */
 extern unsigned long used_vectors[];
 
--- a/arch/x86/kernel/irqinit.c
+++ b/arch/x86/kernel/irqinit.c
@@ -169,9 +169,6 @@ void __init native_init_IRQ(void)
 	 * 'special' SMP interrupts)
 	 */
 	i = FIRST_EXTERNAL_VECTOR;
-#ifndef CONFIG_X86_LOCAL_APIC
-#define first_system_vector NR_VECTORS
-#endif
 	for_each_clear_bit_from(i, used_vectors, first_system_vector) {
 		/* IA32_SYSCALL_VECTOR could be used in trap_init already. */
 		set_intr_gate(i, irq_entries_start +

[patch 04/41] x86/boot: Move EISA setup to a proper place

[Thomas Gleixner]

[-- Attachment #1: x86-boot--Move-EISA-setup-to-a-proper-place.patch --]
[-- Type: text/plain, Size: 2147 bytes --]

EISA has absolutely nothing to do with traps. The EISA bus detection does
not need to run in the very early boot. It's good enough to run it before
the EISA bus and drivers are initialized.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/Makefile |    1 +
 arch/x86/kernel/eisa.c   |   18 ++++++++++++++++++
 arch/x86/kernel/traps.c  |   13 -------------
 3 files changed, 19 insertions(+), 13 deletions(-)

Index: b/arch/x86/kernel/Makefile
===================================================================
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -111,6 +111,7 @@ obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= parav
 obj-$(CONFIG_PARAVIRT_CLOCK)	+= pvclock.o
 obj-$(CONFIG_X86_PMEM_LEGACY_DEVICE) += pmem.o
 
+obj-$(CONFIG_EISA)		+= eisa.o
 obj-$(CONFIG_PCSPKR_PLATFORM)	+= pcspeaker.o
 
 obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
Index: b/arch/x86/kernel/eisa.c
===================================================================
--- /dev/null
+++ b/arch/x86/kernel/eisa.c
@@ -0,0 +1,18 @@
+/*
+ * EISA specific code
+ *
+ * This file is licensed under the GPL V2
+ */
+#include <linux/ioport.h>
+#include <linux/eisa.h>
+
+static __init int eisa_bus_probe(void)
+{
+	void __iomem *p = ioremap(0x0FFFD9, 4);
+
+	if (readl(p) == 'E' + ('I'<<8) + ('S'<<16) + ('A'<<24))
+		EISA_bus = 1;
+	iounmap(p, 4);
+	return 0;
+}
+subsys_initcall(eisa_bus_probe);
Index: b/arch/x86/kernel/traps.c
===================================================================
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -38,11 +38,6 @@
 #include <linux/smp.h>
 #include <linux/io.h>
 
-#ifdef CONFIG_EISA
-#include <linux/ioport.h>
-#include <linux/eisa.h>
-#endif
-
 #if defined(CONFIG_EDAC)
 #include <linux/edac.h>
 #endif
@@ -969,14 +964,6 @@ void __init trap_init(void)
 {
 	int i;
 
-#ifdef CONFIG_EISA
-	void __iomem *p = early_ioremap(0x0FFFD9, 4);
-
-	if (readl(p) == 'E' + ('I'<<8) + ('S'<<16) + ('A'<<24))
-		EISA_bus = 1;
-	early_iounmap(p, 4);
-#endif
-
 	set_intr_gate(X86_TRAP_DE, divide_error);
 	set_intr_gate_ist(X86_TRAP_NMI, &nmi, NMI_STACK);
 	/* int4 can be called from all */

[patch 05/41] x86/tracing: Introduce a static key for exception tracing

[Thomas Gleixner]

[-- Attachment #1: x86-tracing--Introduce-a-static-key-for-exception-tracing.patch --]
[-- Type: text/plain, Size: 3019 bytes --]

Switching the IDT just for avoiding tracepoints creates a complete
impenetrable macro/inline/ifdef mess.

There is no point in avoiding tracepoints for most of the traps/exceptions.
For the more expensive tracepoints, like pagefaults, this can be handled with
an explicit static key.

Preparatory patch to remove the tracing idt.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/trace/common.h      |   15 +++++++++++++++
 arch/x86/include/asm/trace/exceptions.h  |    4 +---
 arch/x86/include/asm/trace/irq_vectors.h |    4 +---
 arch/x86/kernel/tracepoint.c             |    9 ++++++++-
 4 files changed, 25 insertions(+), 7 deletions(-)

--- /dev/null
+++ b/arch/x86/include/asm/trace/common.h
@@ -0,0 +1,15 @@
+#ifndef _ASM_TRACE_COMMON_H
+#define _ASM_TRACE_COMMON_H
+
+extern int trace_irq_vector_regfunc(void);
+extern void trace_irq_vector_unregfunc(void);
+
+#ifdef CONFIG_TRACING
+DECLARE_STATIC_KEY_FALSE(trace_irqvectors_key);
+#define trace_irqvectors_enabled()			\
+	static_branch_unlikely(&trace_irqvectors_key)
+#else
+static inline bool trace_irqvectors_enabled(void) { return false; }
+#endif
+
+#endif
--- a/arch/x86/include/asm/trace/exceptions.h
+++ b/arch/x86/include/asm/trace/exceptions.h
@@ -5,9 +5,7 @@
 #define _TRACE_PAGE_FAULT_H
 
 #include <linux/tracepoint.h>
-
-extern int trace_irq_vector_regfunc(void);
-extern void trace_irq_vector_unregfunc(void);
+#include <asm/trace/common.h>
 
 DECLARE_EVENT_CLASS(x86_exceptions,
 
--- a/arch/x86/include/asm/trace/irq_vectors.h
+++ b/arch/x86/include/asm/trace/irq_vectors.h
@@ -5,9 +5,7 @@
 #define _TRACE_IRQ_VECTORS_H
 
 #include <linux/tracepoint.h>
-
-extern int trace_irq_vector_regfunc(void);
-extern void trace_irq_vector_unregfunc(void);
+#include <asm/trace/common.h>
 
 DECLARE_EVENT_CLASS(x86_irq_vector,
 
--- a/arch/x86/kernel/tracepoint.c
+++ b/arch/x86/kernel/tracepoint.c
@@ -4,9 +4,11 @@
  * Copyright (C) 2013 Seiji Aguchi <seiji.aguchi@hds.com>
  *
  */
+#include <linux/jump_label.h>
+#include <linux/atomic.h>
+
 #include <asm/hw_irq.h>
 #include <asm/desc.h>
-#include <linux/atomic.h>
 
 atomic_t trace_idt_ctr = ATOMIC_INIT(0);
 struct desc_ptr trace_idt_descr = { NR_VECTORS * 16 - 1,
@@ -15,6 +17,7 @@ struct desc_ptr trace_idt_descr = { NR_V
 /* No need to be aligned, but done to keep all IDTs defined the same way. */
 gate_desc trace_idt_table[NR_VECTORS] __page_aligned_bss;
 
+DEFINE_STATIC_KEY_FALSE(trace_irqvectors_key);
 static int trace_irq_vector_refcount;
 static DEFINE_MUTEX(irq_vector_mutex);
 
@@ -36,6 +39,8 @@ static void switch_idt(void *arg)
 
 int trace_irq_vector_regfunc(void)
 {
+	static_branch_inc(&trace_irqvectors_key);
+
 	mutex_lock(&irq_vector_mutex);
 	if (!trace_irq_vector_refcount) {
 		set_trace_idt_ctr(1);
@@ -49,6 +54,8 @@ int trace_irq_vector_regfunc(void)
 
 void trace_irq_vector_unregfunc(void)
 {
+	static_branch_dec(&trace_irqvectors_key);
+
 	mutex_lock(&irq_vector_mutex);
 	trace_irq_vector_refcount--;
 	if (!trace_irq_vector_refcount) {

[patch 06/41] x86/traps: Simplify pagefault tracing logic

[Thomas Gleixner]

[-- Attachment #1: x86-traps--Simplyfy-pagefault-tracing-logic.patch --]
[-- Type: text/plain, Size: 6273 bytes --]

Make use of the new irqvector tracing static key and remove the duplicated
trace_do_pagefault() implementation.

If irq vector tracing is disabled, then the overhead of this is a single
NOP5, which is a reasonable tradeoff to avoid duplicated code and the
unholy macro mess.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/entry/entry_32.S    |    8 -------
 arch/x86/entry/entry_64.S    |   13 -----------
 arch/x86/include/asm/traps.h |   10 ---------
 arch/x86/kernel/kvm.c        |    2 -
 arch/x86/mm/fault.c          |   47 ++++++++++---------------------------------
 5 files changed, 15 insertions(+), 65 deletions(-)

--- a/arch/x86/entry/entry_32.S
+++ b/arch/x86/entry/entry_32.S
@@ -891,14 +891,6 @@ BUILD_INTERRUPT3(hyperv_callback_vector,
 
 #endif /* CONFIG_HYPERV */
 
-#ifdef CONFIG_TRACING
-ENTRY(trace_page_fault)
-	ASM_CLAC
-	pushl	$trace_do_page_fault
-	jmp	common_exception
-END(trace_page_fault)
-#endif
-
 ENTRY(page_fault)
 	ASM_CLAC
 	pushl	$do_page_fault
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -829,17 +829,6 @@ ENTRY(\sym)
 END(\sym)
 .endm
 
-#ifdef CONFIG_TRACING
-.macro trace_idtentry sym do_sym has_error_code:req
-idtentry trace(\sym) trace(\do_sym) has_error_code=\has_error_code
-idtentry \sym \do_sym has_error_code=\has_error_code
-.endm
-#else
-.macro trace_idtentry sym do_sym has_error_code:req
-idtentry \sym \do_sym has_error_code=\has_error_code
-.endm
-#endif
-
 idtentry divide_error			do_divide_error			has_error_code=0
 idtentry overflow			do_overflow			has_error_code=0
 idtentry bounds				do_bounds			has_error_code=0
@@ -1004,7 +993,7 @@ idtentry xen_stack_segment	do_stack_segm
 #endif
 
 idtentry general_protection	do_general_protection	has_error_code=1
-trace_idtentry page_fault	do_page_fault		has_error_code=1
+idtentry page_fault		do_page_fault		has_error_code=1
 
 #ifdef CONFIG_KVM_GUEST
 idtentry async_page_fault	do_async_page_fault	has_error_code=1
--- a/arch/x86/include/asm/traps.h
+++ b/arch/x86/include/asm/traps.h
@@ -39,7 +39,6 @@ asmlinkage void machine_check(void);
 asmlinkage void simd_coprocessor_error(void);
 
 #ifdef CONFIG_TRACING
-asmlinkage void trace_page_fault(void);
 #define trace_stack_segment stack_segment
 #define trace_divide_error divide_error
 #define trace_bounds bounds
@@ -54,6 +53,7 @@ asmlinkage void trace_page_fault(void);
 #define trace_alignment_check alignment_check
 #define trace_simd_coprocessor_error simd_coprocessor_error
 #define trace_async_page_fault async_page_fault
+#define trace_page_fault page_fault
 #endif
 
 dotraplinkage void do_divide_error(struct pt_regs *, long);
@@ -74,14 +74,6 @@ asmlinkage struct pt_regs *sync_regs(str
 #endif
 dotraplinkage void do_general_protection(struct pt_regs *, long);
 dotraplinkage void do_page_fault(struct pt_regs *, unsigned long);
-#ifdef CONFIG_TRACING
-dotraplinkage void trace_do_page_fault(struct pt_regs *, unsigned long);
-#else
-static inline void trace_do_page_fault(struct pt_regs *regs, unsigned long error)
-{
-	do_page_fault(regs, error);
-}
-#endif
 dotraplinkage void do_spurious_interrupt_bug(struct pt_regs *, long);
 dotraplinkage void do_coprocessor_error(struct pt_regs *, long);
 dotraplinkage void do_alignment_check(struct pt_regs *, long);
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -263,7 +263,7 @@ do_async_page_fault(struct pt_regs *regs
 
 	switch (kvm_read_and_reset_pf_reason()) {
 	default:
-		trace_do_page_fault(regs, error_code);
+		do_page_fault(regs, error_code);
 		break;
 	case KVM_PV_REASON_PAGE_NOT_PRESENT:
 		/* page is swapped out by the host. */
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1254,10 +1254,6 @@ static inline bool smap_violation(int er
  * This routine handles page faults.  It determines the address,
  * and the problem, and then passes it off to one of the appropriate
  * routines.
- *
- * This function must have noinline because both callers
- * {,trace_}do_page_fault() have notrace on. Having this an actual function
- * guarantees there's a function trace entry.
  */
 static noinline void
 __do_page_fault(struct pt_regs *regs, unsigned long error_code,
@@ -1490,27 +1486,6 @@ static noinline void
 }
 NOKPROBE_SYMBOL(__do_page_fault);
 
-dotraplinkage void notrace
-do_page_fault(struct pt_regs *regs, unsigned long error_code)
-{
-	unsigned long address = read_cr2(); /* Get the faulting address */
-	enum ctx_state prev_state;
-
-	/*
-	 * We must have this function tagged with __kprobes, notrace and call
-	 * read_cr2() before calling anything else. To avoid calling any kind
-	 * of tracing machinery before we've observed the CR2 value.
-	 *
-	 * exception_{enter,exit}() contain all sorts of tracepoints.
-	 */
-
-	prev_state = exception_enter();
-	__do_page_fault(regs, error_code, address);
-	exception_exit(prev_state);
-}
-NOKPROBE_SYMBOL(do_page_fault);
-
-#ifdef CONFIG_TRACING
 static nokprobe_inline void
 trace_page_fault_entries(unsigned long address, struct pt_regs *regs,
 			 unsigned long error_code)
@@ -1522,21 +1497,23 @@ trace_page_fault_entries(unsigned long a
 }
 
 dotraplinkage void notrace
-trace_do_page_fault(struct pt_regs *regs, unsigned long error_code)
+do_page_fault(struct pt_regs *regs, unsigned long error_code)
 {
+	unsigned long address = read_cr2(); /* Get the faulting address */
+	enum ctx_state prev_state;
+
 	/*
-	 * The exception_enter and tracepoint processing could
-	 * trigger another page faults (user space callchain
-	 * reading) and destroy the original cr2 value, so read
-	 * the faulting address now.
+	 * We must have this function tagged with __kprobes, notrace and call
+	 * read_cr2() before calling anything else. To avoid calling any kind
+	 * of tracing machinery before we've observed the CR2 value.
+	 *
+	 * exception_{enter,exit}() contain all sorts of tracepoints.
 	 */
-	unsigned long address = read_cr2();
-	enum ctx_state prev_state;
+	if (trace_irqvectors_enabled())
+		trace_page_fault_entries(address, regs, error_code);
 
 	prev_state = exception_enter();
-	trace_page_fault_entries(address, regs, error_code);
 	__do_page_fault(regs, error_code, address);
 	exception_exit(prev_state);
 }
-NOKPROBE_SYMBOL(trace_do_page_fault);
-#endif /* CONFIG_TRACING */
+NOKPROBE_SYMBOL(do_page_fault);

[patch 07/41] x86/apic: Remove the duplicated tracing version of local_timer_interrupt

[Thomas Gleixner]

[-- Attachment #1: x86-apic--Remove-the-duplicated-tracing-version-of-local_timer_interrupt.patch --]
[-- Type: text/plain, Size: 2144 bytes --]

The two NOP5 are noise in the rest of the work which is done by the timer
interrupt and modern CPUs are pretty good in optimizing nops.

Get rid of the interrupt handler duplication and move the tracepoints into
the regular handler.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/hw_irq.h |    2 +-
 arch/x86/kernel/apic/apic.c   |   19 -------------------
 2 files changed, 1 insertion(+), 20 deletions(-)

--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -48,7 +48,6 @@ extern asmlinkage void call_function_sin
 
 #ifdef CONFIG_TRACING
 /* Interrupt handlers registered during init_IRQ */
-extern void trace_apic_timer_interrupt(void);
 extern void trace_x86_platform_ipi(void);
 extern void trace_error_interrupt(void);
 extern void trace_irq_work_interrupt(void);
@@ -59,6 +58,7 @@ extern void trace_threshold_interrupt(vo
 extern void trace_deferred_error_interrupt(void);
 extern void trace_call_function_interrupt(void);
 extern void trace_call_function_single_interrupt(void);
+#define trace_apic_timer_interrupt	apic_timer_interrupt
 #define trace_irq_move_cleanup_interrupt  irq_move_cleanup_interrupt
 #define trace_reboot_interrupt  reboot_interrupt
 #define trace_kvm_posted_intr_ipi kvm_posted_intr_ipi
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1040,25 +1040,6 @@ static void local_apic_timer_interrupt(v
 	 * interrupt lock, which is the WrongThing (tm) to do.
 	 */
 	entering_ack_irq();
-	local_apic_timer_interrupt();
-	exiting_irq();
-
-	set_irq_regs(old_regs);
-}
-
-__visible void __irq_entry smp_trace_apic_timer_interrupt(struct pt_regs *regs)
-{
-	struct pt_regs *old_regs = set_irq_regs(regs);
-
-	/*
-	 * NOTE! We'd better ACK the irq immediately,
-	 * because timer handling can be slow.
-	 *
-	 * update_process_times() expects us to have done irq_enter().
-	 * Besides, if we don't timer interrupts ignore the global
-	 * interrupt lock, which is the WrongThing (tm) to do.
-	 */
-	entering_ack_irq();
 	trace_local_timer_entry(LOCAL_TIMER_VECTOR);
 	local_apic_timer_interrupt();
 	trace_local_timer_exit(LOCAL_TIMER_VECTOR);

[patch 08/41] x86/apic: Use this_cpu_ptr in local_timer_interrupt

[Thomas Gleixner]

[-- Attachment #1: x86-apic--Use-this_cpu_ptr-in-local_timer_interrupt.patch --]
[-- Type: text/plain, Size: 1003 bytes --]

Accessing the per cpu data via per_cpu(, smp_processor_id()) is
pointless. Use this_cpu_ptr() instead.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/apic/apic.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -990,8 +990,7 @@ void setup_secondary_APIC_clock(void)
  */
 static void local_apic_timer_interrupt(void)
 {
-	int cpu = smp_processor_id();
-	struct clock_event_device *evt = &per_cpu(lapic_events, cpu);
+	struct clock_event_device *evt = this_cpu_ptr(&lapic_events);
 
 	/*
 	 * Normally we should not be here till LAPIC has been initialized but
@@ -1005,7 +1004,8 @@ static void local_apic_timer_interrupt(v
 	 * spurious.
 	 */
 	if (!evt->event_handler) {
-		pr_warning("Spurious LAPIC timer interrupt on cpu %d\n", cpu);
+		pr_warning("Spurious LAPIC timer interrupt on cpu %d\n",
+			   smp_processor_id());
 		/* Switch it off */
 		lapic_timer_shutdown(evt);
 		return;

[patch 09/41] x86/irq: Get rid of duplicated trace_x86_platform_ipi() code

[Thomas Gleixner]

[-- Attachment #1: x86-rq--Get-rid-of-duplicated-trace_x86_platform_ipi.patch --]
[-- Type: text/plain, Size: 2331 bytes --]

Two NOP5 are really a good tradeoff vs. the unholy IDT switching mess,
which duplicates code all over the place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/hw_irq.h |    2 +-
 arch/x86/kernel/irq.c         |   25 +++++--------------------
 2 files changed, 6 insertions(+), 21 deletions(-)

--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -48,7 +48,6 @@ extern asmlinkage void call_function_sin
 
 #ifdef CONFIG_TRACING
 /* Interrupt handlers registered during init_IRQ */
-extern void trace_x86_platform_ipi(void);
 extern void trace_error_interrupt(void);
 extern void trace_irq_work_interrupt(void);
 extern void trace_spurious_interrupt(void);
@@ -58,6 +57,7 @@ extern void trace_threshold_interrupt(vo
 extern void trace_deferred_error_interrupt(void);
 extern void trace_call_function_interrupt(void);
 extern void trace_call_function_single_interrupt(void);
+#define trace_x86_platform_ipi	x86_platform_ipi
 #define trace_apic_timer_interrupt	apic_timer_interrupt
 #define trace_irq_move_cleanup_interrupt  irq_move_cleanup_interrupt
 #define trace_reboot_interrupt  reboot_interrupt
--- a/arch/x86/kernel/irq.c
+++ b/arch/x86/kernel/irq.c
@@ -262,20 +262,16 @@ u64 arch_irq_stat(void)
 /*
  * Handler for X86_PLATFORM_IPI_VECTOR.
  */
-void __smp_x86_platform_ipi(void)
-{
-	inc_irq_stat(x86_platform_ipis);
-
-	if (x86_platform_ipi_callback)
-		x86_platform_ipi_callback();
-}
-
 __visible void __irq_entry smp_x86_platform_ipi(struct pt_regs *regs)
 {
 	struct pt_regs *old_regs = set_irq_regs(regs);
 
 	entering_ack_irq();
-	__smp_x86_platform_ipi();
+	trace_x86_platform_ipi_entry(X86_PLATFORM_IPI_VECTOR);
+	inc_irq_stat(x86_platform_ipis);
+	if (x86_platform_ipi_callback)
+		x86_platform_ipi_callback();
+	trace_x86_platform_ipi_exit(X86_PLATFORM_IPI_VECTOR);
 	exiting_irq();
 	set_irq_regs(old_regs);
 }
@@ -334,17 +330,6 @@ EXPORT_SYMBOL_GPL(kvm_set_posted_intr_wa
 }
 #endif
 
-__visible void __irq_entry smp_trace_x86_platform_ipi(struct pt_regs *regs)
-{
-	struct pt_regs *old_regs = set_irq_regs(regs);
-
-	entering_ack_irq();
-	trace_x86_platform_ipi_entry(X86_PLATFORM_IPI_VECTOR);
-	__smp_x86_platform_ipi();
-	trace_x86_platform_ipi_exit(X86_PLATFORM_IPI_VECTOR);
-	exiting_irq();
-	set_irq_regs(old_regs);
-}
 
 #ifdef CONFIG_HOTPLUG_CPU
 

[patch 10/41] x86/apic: Remove the duplicated tracing versions of interrupts

[Thomas Gleixner]

[-- Attachment #1: x86-apic--Remove-the-duplicated-tracing-versions_of_interrupts.patch --]
[-- Type: text/plain, Size: 3882 bytes --]

The error and the spurious interrupt are really rare events and not at all
so performance sensitive that two NOP5s can not be tolerated when tracing
is disabled.

Remove the nonsense.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/hw_irq.h |    4 +--
 arch/x86/kernel/apic/apic.c   |   43 +++++++++---------------------------------
 2 files changed, 12 insertions(+), 35 deletions(-)

--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -48,15 +48,15 @@ extern asmlinkage void call_function_sin
 
 #ifdef CONFIG_TRACING
 /* Interrupt handlers registered during init_IRQ */
-extern void trace_error_interrupt(void);
 extern void trace_irq_work_interrupt(void);
-extern void trace_spurious_interrupt(void);
 extern void trace_thermal_interrupt(void);
 extern void trace_reschedule_interrupt(void);
 extern void trace_threshold_interrupt(void);
 extern void trace_deferred_error_interrupt(void);
 extern void trace_call_function_interrupt(void);
 extern void trace_call_function_single_interrupt(void);
+#define trace_error_interrupt error_interrupt
+#define trace_spurious_interrupt spurious_interrupt
 #define trace_x86_platform_ipi	x86_platform_ipi
 #define trace_apic_timer_interrupt	apic_timer_interrupt
 #define trace_irq_move_cleanup_interrupt  irq_move_cleanup_interrupt
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1901,10 +1901,14 @@ void __init register_lapic_address(unsig
 /*
  * This interrupt should _never_ happen with our APIC/SMP architecture
  */
-static void __smp_spurious_interrupt(u8 vector)
+__visible void __irq_entry smp_spurious_interrupt(struct pt_regs *regs)
 {
+	u8 vector = ~regs->orig_ax;
 	u32 v;
 
+	entering_irq();
+	trace_spurious_apic_entry(vector);
+
 	/*
 	 * Check if this really is a spurious interrupt and ACK it
 	 * if it is a vectored one.  Just in case...
@@ -1919,22 +1923,7 @@ static void __smp_spurious_interrupt(u8
 	/* see sw-dev-man vol 3, chapter 7.4.13.5 */
 	pr_info("spurious APIC interrupt through vector %02x on CPU#%d, "
 		"should never happen.\n", vector, smp_processor_id());
-}
 
-__visible void __irq_entry smp_spurious_interrupt(struct pt_regs *regs)
-{
-	entering_irq();
-	__smp_spurious_interrupt(~regs->orig_ax);
-	exiting_irq();
-}
-
-__visible void __irq_entry smp_trace_spurious_interrupt(struct pt_regs *regs)
-{
-	u8 vector = ~regs->orig_ax;
-
-	entering_irq();
-	trace_spurious_apic_entry(vector);
-	__smp_spurious_interrupt(vector);
 	trace_spurious_apic_exit(vector);
 	exiting_irq();
 }
@@ -1942,10 +1931,8 @@ static void __smp_spurious_interrupt(u8
 /*
  * This interrupt should never happen with our APIC/SMP architecture
  */
-static void __smp_error_interrupt(struct pt_regs *regs)
+__visible void __irq_entry smp_error_interrupt(struct pt_regs *regs)
 {
-	u32 v;
-	u32 i = 0;
 	static const char * const error_interrupt_reason[] = {
 		"Send CS error",		/* APIC Error Bit 0 */
 		"Receive CS error",		/* APIC Error Bit 1 */
@@ -1956,6 +1943,10 @@ static void __smp_error_interrupt(struct
 		"Received illegal vector",	/* APIC Error Bit 6 */
 		"Illegal register address",	/* APIC Error Bit 7 */
 	};
+	u32 v, i = 0;
+
+	entering_irq();
+	trace_error_apic_entry(ERROR_APIC_VECTOR);
 
 	/* First tickle the hardware, only then report what went on. -- REW */
 	if (lapic_get_maxlvt() > 3)	/* Due to the Pentium erratum 3AP. */
@@ -1977,20 +1968,6 @@ static void __smp_error_interrupt(struct
 
 	apic_printk(APIC_DEBUG, KERN_CONT "\n");
 
-}
-
-__visible void __irq_entry smp_error_interrupt(struct pt_regs *regs)
-{
-	entering_irq();
-	__smp_error_interrupt(regs);
-	exiting_irq();
-}
-
-__visible void __irq_entry smp_trace_error_interrupt(struct pt_regs *regs)
-{
-	entering_irq();
-	trace_error_apic_entry(ERROR_APIC_VECTOR);
-	__smp_error_interrupt(regs);
 	trace_error_apic_exit(ERROR_APIC_VECTOR);
 	exiting_irq();
 }

[patch 11/41] x86/irqwork: Get rid of duplicated tracing interrupt code

[Thomas Gleixner]

[-- Attachment #1: x86-irqwork--Get-rid-of-duplicated-tracing-interrupt-code.patch --]
[-- Type: text/plain, Size: 1771 bytes --]

Two NOP5 are a reasonable tradeoff to avoid duplicated code and the
requirement to switch the IDT.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/hw_irq.h |    2 +-
 arch/x86/kernel/irq_work.c    |   16 ++--------------
 2 files changed, 3 insertions(+), 15 deletions(-)

--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -48,13 +48,13 @@ extern asmlinkage void call_function_sin
 
 #ifdef CONFIG_TRACING
 /* Interrupt handlers registered during init_IRQ */
-extern void trace_irq_work_interrupt(void);
 extern void trace_thermal_interrupt(void);
 extern void trace_reschedule_interrupt(void);
 extern void trace_threshold_interrupt(void);
 extern void trace_deferred_error_interrupt(void);
 extern void trace_call_function_interrupt(void);
 extern void trace_call_function_single_interrupt(void);
+#define trace_irq_work_interrupt irq_work_interrupt
 #define trace_error_interrupt error_interrupt
 #define trace_spurious_interrupt spurious_interrupt
 #define trace_x86_platform_ipi	x86_platform_ipi
--- a/arch/x86/kernel/irq_work.c
+++ b/arch/x86/kernel/irq_work.c
@@ -11,24 +11,12 @@
 #include <asm/trace/irq_vectors.h>
 #include <linux/interrupt.h>
 
-static inline void __smp_irq_work_interrupt(void)
-{
-	inc_irq_stat(apic_irq_work_irqs);
-	irq_work_run();
-}
-
 __visible void __irq_entry smp_irq_work_interrupt(struct pt_regs *regs)
 {
 	ipi_entering_ack_irq();
-	__smp_irq_work_interrupt();
-	exiting_irq();
-}
-
-__visible void __irq_entry smp_trace_irq_work_interrupt(struct pt_regs *regs)
-{
-	ipi_entering_ack_irq();
+	inc_irq_stat(apic_irq_work_irqs);
 	trace_irq_work_entry(IRQ_WORK_VECTOR);
-	__smp_irq_work_interrupt();
+	irq_work_run();
 	trace_irq_work_exit(IRQ_WORK_VECTOR);
 	exiting_irq();
 }

[patch 12/41] x86/mce: Remove duplicated tracing interrupt code

[Thomas Gleixner]

[-- Attachment #1: x86-mce--Remove-duplicated-tracing-interrupt-code.patch --]
[-- Type: text/plain, Size: 3909 bytes --]

Machine checks are not really high frequency events. The extra two NOP5 for
the disabled tracepoints are noise vs. the heavy lifting which needs to be
done in the MCE handler.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/hw_irq.h            |    6 +++---
 arch/x86/kernel/cpu/mcheck/mce_amd.c     |   16 ++--------------
 arch/x86/kernel/cpu/mcheck/therm_throt.c |   20 +++-----------------
 arch/x86/kernel/cpu/mcheck/threshold.c   |   16 ++--------------
 4 files changed, 10 insertions(+), 48 deletions(-)

--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -48,12 +48,12 @@ extern asmlinkage void call_function_sin
 
 #ifdef CONFIG_TRACING
 /* Interrupt handlers registered during init_IRQ */
-extern void trace_thermal_interrupt(void);
 extern void trace_reschedule_interrupt(void);
-extern void trace_threshold_interrupt(void);
-extern void trace_deferred_error_interrupt(void);
 extern void trace_call_function_interrupt(void);
 extern void trace_call_function_single_interrupt(void);
+#define trace_thermal_interrupt	thermal_interrupt
+#define trace_threshold_interrupt	threshold_interrupt
+#define trace_deferred_error_interrupt	deferred_error_interrupt
 #define trace_irq_work_interrupt irq_work_interrupt
 #define trace_error_interrupt error_interrupt
 #define trace_spurious_interrupt spurious_interrupt
--- a/arch/x86/kernel/cpu/mcheck/mce_amd.c
+++ b/arch/x86/kernel/cpu/mcheck/mce_amd.c
@@ -776,24 +776,12 @@ static void __log_error(unsigned int ban
 	mce_log(&m);
 }
 
-static inline void __smp_deferred_error_interrupt(void)
-{
-	inc_irq_stat(irq_deferred_error_count);
-	deferred_error_int_vector();
-}
-
 asmlinkage __visible void __irq_entry smp_deferred_error_interrupt(void)
 {
 	entering_irq();
-	__smp_deferred_error_interrupt();
-	exiting_ack_irq();
-}
-
-asmlinkage __visible void __irq_entry smp_trace_deferred_error_interrupt(void)
-{
-	entering_irq();
 	trace_deferred_error_apic_entry(DEFERRED_ERROR_VECTOR);
-	__smp_deferred_error_interrupt();
+	inc_irq_stat(irq_deferred_error_count);
+	deferred_error_int_vector();
 	trace_deferred_error_apic_exit(DEFERRED_ERROR_VECTOR);
 	exiting_ack_irq();
 }
--- a/arch/x86/kernel/cpu/mcheck/therm_throt.c
+++ b/arch/x86/kernel/cpu/mcheck/therm_throt.c
@@ -390,26 +390,12 @@ static void unexpected_thermal_interrupt
 
 static void (*smp_thermal_vector)(void) = unexpected_thermal_interrupt;
 
-static inline void __smp_thermal_interrupt(void)
-{
-	inc_irq_stat(irq_thermal_count);
-	smp_thermal_vector();
-}
-
-asmlinkage __visible void __irq_entry
-smp_thermal_interrupt(struct pt_regs *regs)
-{
-	entering_irq();
-	__smp_thermal_interrupt();
-	exiting_ack_irq();
-}
-
-asmlinkage __visible void __irq_entry
-smp_trace_thermal_interrupt(struct pt_regs *regs)
+asmlinkage __visible void __irq_entry smp_thermal_interrupt(struct pt_regs *r)
 {
 	entering_irq();
+	inc_irq_stat(irq_thermal_count);
 	trace_thermal_apic_entry(THERMAL_APIC_VECTOR);
-	__smp_thermal_interrupt();
+	smp_thermal_vector();
 	trace_thermal_apic_exit(THERMAL_APIC_VECTOR);
 	exiting_ack_irq();
 }
--- a/arch/x86/kernel/cpu/mcheck/threshold.c
+++ b/arch/x86/kernel/cpu/mcheck/threshold.c
@@ -17,24 +17,12 @@ static void default_threshold_interrupt(
 
 void (*mce_threshold_vector)(void) = default_threshold_interrupt;
 
-static inline void __smp_threshold_interrupt(void)
-{
-	inc_irq_stat(irq_threshold_count);
-	mce_threshold_vector();
-}
-
 asmlinkage __visible void __irq_entry smp_threshold_interrupt(void)
 {
 	entering_irq();
-	__smp_threshold_interrupt();
-	exiting_ack_irq();
-}
-
-asmlinkage __visible void __irq_entry smp_trace_threshold_interrupt(void)
-{
-	entering_irq();
 	trace_threshold_apic_entry(THRESHOLD_APIC_VECTOR);
-	__smp_threshold_interrupt();
+	inc_irq_stat(irq_threshold_count);
+	mce_threshold_vector();
 	trace_threshold_apic_exit(THRESHOLD_APIC_VECTOR);
 	exiting_ack_irq();
 }

[patch 13/41] x86/smp: Remove pointless duplicated interrupt code

[Thomas Gleixner]

[-- Attachment #1: x86-smp--Remove-pointless-duplicated-interrupt-code.patch --]
[-- Type: text/plain, Size: 2965 bytes --]

Two NOP5 are really a good tradeoff vs. the unholy IDT switching mess,
which duplicates code all over the place. The rescheduling interrupt gets
optimized in a later step.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/hw_irq.h |    4 +--
 arch/x86/kernel/smp.c         |   43 ++++++------------------------------------
 2 files changed, 9 insertions(+), 38 deletions(-)

--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -49,8 +49,8 @@ extern asmlinkage void call_function_sin
 #ifdef CONFIG_TRACING
 /* Interrupt handlers registered during init_IRQ */
 extern void trace_reschedule_interrupt(void);
-extern void trace_call_function_interrupt(void);
-extern void trace_call_function_single_interrupt(void);
+#define trace_call_function_interrupt	call_function_interrupt
+#define trace_call_function_single_interrupt	call_function_single_interrupt
 #define trace_thermal_interrupt	thermal_interrupt
 #define trace_threshold_interrupt	threshold_interrupt
 #define trace_deferred_error_interrupt	deferred_error_interrupt
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -281,57 +281,28 @@ static inline void __smp_reschedule_inte
 	 */
 	ipi_entering_ack_irq();
 	trace_reschedule_entry(RESCHEDULE_VECTOR);
-	__smp_reschedule_interrupt();
+	inc_irq_stat(irq_resched_count);
+	scheduler_ipi();
 	trace_reschedule_exit(RESCHEDULE_VECTOR);
 	exiting_irq();
-	/*
-	 * KVM uses this interrupt to force a cpu out of guest mode
-	 */
-}
-
-static inline void __smp_call_function_interrupt(void)
-{
-	generic_smp_call_function_interrupt();
-	inc_irq_stat(irq_call_count);
 }
 
 __visible void __irq_entry smp_call_function_interrupt(struct pt_regs *regs)
 {
 	ipi_entering_ack_irq();
-	__smp_call_function_interrupt();
-	exiting_irq();
-}
-
-__visible void __irq_entry
-smp_trace_call_function_interrupt(struct pt_regs *regs)
-{
-	ipi_entering_ack_irq();
 	trace_call_function_entry(CALL_FUNCTION_VECTOR);
-	__smp_call_function_interrupt();
-	trace_call_function_exit(CALL_FUNCTION_VECTOR);
-	exiting_irq();
-}
-
-static inline void __smp_call_function_single_interrupt(void)
-{
-	generic_smp_call_function_single_interrupt();
 	inc_irq_stat(irq_call_count);
-}
-
-__visible void __irq_entry
-smp_call_function_single_interrupt(struct pt_regs *regs)
-{
-	ipi_entering_ack_irq();
-	__smp_call_function_single_interrupt();
+	generic_smp_call_function_interrupt();
+	trace_call_function_exit(CALL_FUNCTION_VECTOR);
 	exiting_irq();
 }
 
-__visible void __irq_entry
-smp_trace_call_function_single_interrupt(struct pt_regs *regs)
+__visible void __irq_entry smp_call_function_single_interrupt(struct pt_regs *r)
 {
 	ipi_entering_ack_irq();
 	trace_call_function_single_entry(CALL_FUNCTION_SINGLE_VECTOR);
-	__smp_call_function_single_interrupt();
+	inc_irq_stat(irq_call_count);
+	generic_smp_call_function_single_interrupt();
 	trace_call_function_single_exit(CALL_FUNCTION_SINGLE_VECTOR);
 	exiting_irq();
 }

[patch 14/41] x86/smp: Use static key for reschedule interrupt tracing

[Thomas Gleixner]

[-- Attachment #1: x86-smp--Use-static-key-for-reschedule-interrupt-tracing.patch --]
[-- Type: text/plain, Size: 2443 bytes --]

It's worth to avoid the extra irq_enter()/irq_exit() pair in the case that
the reschedule interrupt tracepoints are disabled.

Use the static key which indicates that exception tracing is enabled. For
now this key is global. It will be optimized in a later step.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/hw_irq.h |    2 +-
 arch/x86/kernel/smp.c         |   40 ++++++++++++++++------------------------
 2 files changed, 17 insertions(+), 25 deletions(-)

--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -48,7 +48,7 @@ extern asmlinkage void call_function_sin
 
 #ifdef CONFIG_TRACING
 /* Interrupt handlers registered during init_IRQ */
-extern void trace_reschedule_interrupt(void);
+#define trace_reschedule_interrupt	reschedule_interrupt
 #define trace_call_function_interrupt	call_function_interrupt
 #define trace_call_function_single_interrupt	call_function_single_interrupt
 #define trace_thermal_interrupt	thermal_interrupt
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -254,37 +254,29 @@ static void native_stop_other_cpus(int w
 }
 
 /*
- * Reschedule call back.
+ * Reschedule call back. KVM uses this interrupt to force a cpu out of
+ * guest mode
  */
-static inline void __smp_reschedule_interrupt(void)
-{
-	inc_irq_stat(irq_resched_count);
-	scheduler_ipi();
-}
-
 __visible void __irq_entry smp_reschedule_interrupt(struct pt_regs *regs)
 {
 	ack_APIC_irq();
-	__smp_reschedule_interrupt();
-	/*
-	 * KVM uses this interrupt to force a cpu out of guest mode
-	 */
-}
 
-__visible void __irq_entry smp_trace_reschedule_interrupt(struct pt_regs *regs)
-{
-	/*
-	 * Need to call irq_enter() before calling the trace point.
-	 * __smp_reschedule_interrupt() calls irq_enter/exit() too (in
-	 * scheduler_ipi(). This is OK, since those functions are allowed
-	 * to nest.
-	 */
-	ipi_entering_ack_irq();
-	trace_reschedule_entry(RESCHEDULE_VECTOR);
+	if (trace_irqvectors_enabled()) {
+		/*
+		 * scheduler_ipi() might call irq_enter() as well, but
+		 * nested calls are fine.
+		 */
+		irq_enter();
+		trace_reschedule_entry(RESCHEDULE_VECTOR);
+	}
+
 	inc_irq_stat(irq_resched_count);
 	scheduler_ipi();
-	trace_reschedule_exit(RESCHEDULE_VECTOR);
-	exiting_irq();
+
+	if (trace_irqvectors_enabled()) {
+		trace_reschedule_exit(RESCHEDULE_VECTOR);
+		irq_exit();
+	}
 }
 
 __visible void __irq_entry smp_call_function_interrupt(struct pt_regs *regs)

[patch 15/41] x86/idt: Remove tracing idt completely

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Remove-tracing-idt-completely.patch --]
[-- Type: text/plain, Size: 10508 bytes --]

No more users of the tracing idt. All exception tracepoints have been moved
into the regular handlers. Get rid of the mess which shouldn't have been
created in the first place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/entry/entry_32.S         |   12 -----
 arch/x86/entry/entry_64.S         |   13 ------
 arch/x86/include/asm/desc.h       |   82 ++------------------------------------
 arch/x86/include/asm/entry_arch.h |   14 ++----
 arch/x86/include/asm/hw_irq.h     |   20 ---------
 arch/x86/include/asm/segment.h    |    3 -
 arch/x86/kernel/tracepoint.c      |   43 -------------------
 arch/x86/kernel/traps.c           |    6 --
 8 files changed, 14 insertions(+), 179 deletions(-)

--- a/arch/x86/entry/entry_32.S
+++ b/arch/x86/entry/entry_32.S
@@ -673,16 +673,8 @@ ENTRY(name)				\
 	jmp	ret_from_intr;		\
 ENDPROC(name)
 
-
-#ifdef CONFIG_TRACING
-# define TRACE_BUILD_INTERRUPT(name, nr)	BUILD_INTERRUPT3(trace_##name, nr, smp_trace_##name)
-#else
-# define TRACE_BUILD_INTERRUPT(name, nr)
-#endif
-
 #define BUILD_INTERRUPT(name, nr)		\
 	BUILD_INTERRUPT3(name, nr, smp_##name);	\
-	TRACE_BUILD_INTERRUPT(name, nr)
 
 /* The include is where all of the SMP etc. interrupts come from */
 #include <asm/entry_arch.h>
@@ -880,14 +872,14 @@ ENTRY(xen_failsafe_callback)
 ENDPROC(xen_failsafe_callback)
 
 BUILD_INTERRUPT3(xen_hvm_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
-		xen_evtchn_do_upcall)
+		 xen_evtchn_do_upcall)
 
 #endif /* CONFIG_XEN */
 
 #if IS_ENABLED(CONFIG_HYPERV)
 
 BUILD_INTERRUPT3(hyperv_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
-	hyperv_vector_handler)
+		 hyperv_vector_handler)
 
 #endif /* CONFIG_HYPERV */
 
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -748,18 +748,6 @@ ENTRY(\sym)
 END(\sym)
 .endm
 
-#ifdef CONFIG_TRACING
-#define trace(sym) trace_##sym
-#define smp_trace(sym) smp_trace_##sym
-
-.macro trace_apicinterrupt num sym
-apicinterrupt3 \num trace(\sym) smp_trace(\sym)
-.endm
-#else
-.macro trace_apicinterrupt num sym do_sym
-.endm
-#endif
-
 /* Make sure APIC interrupt handlers end up in the irqentry section: */
 #if defined(CONFIG_FUNCTION_GRAPH_TRACER) || defined(CONFIG_KASAN)
 # define PUSH_SECTION_IRQENTRY	.pushsection .irqentry.text, "ax"
@@ -772,7 +760,6 @@ apicinterrupt3 \num trace(\sym) smp_trac
 .macro apicinterrupt num sym do_sym
 PUSH_SECTION_IRQENTRY
 apicinterrupt3 \num \sym \do_sym
-trace_apicinterrupt \num \sym
 POP_SECTION_IRQENTRY
 .endm
 
--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -420,35 +420,7 @@ static inline void set_nmi_gate(int gate
 }
 #endif
 
-#ifdef CONFIG_TRACING
-extern struct desc_ptr trace_idt_descr;
-extern gate_desc trace_idt_table[];
-static inline void write_trace_idt_entry(int entry, const gate_desc *gate)
-{
-	write_idt_entry(trace_idt_table, entry, gate);
-}
-
-static inline void _trace_set_gate(int gate, unsigned type, void *addr,
-				   unsigned dpl, unsigned ist, unsigned seg)
-{
-	gate_desc s;
-
-	pack_gate(&s, type, (unsigned long)addr, dpl, ist, seg);
-	/*
-	 * does not need to be atomic because it is only done once at
-	 * setup time
-	 */
-	write_trace_idt_entry(gate, &s);
-}
-#else
-static inline void write_trace_idt_entry(int entry, const gate_desc *gate)
-{
-}
-
-#define _trace_set_gate(gate, type, addr, dpl, ist, seg)
-#endif
-
-static inline void _set_gate(int gate, unsigned type, void *addr,
+static inline void _set_gate(int gate, unsigned type, const void *addr,
 			     unsigned dpl, unsigned ist, unsigned seg)
 {
 	gate_desc s;
@@ -459,28 +431,13 @@ static inline void _set_gate(int gate, u
 	 * setup time
 	 */
 	write_idt_entry(idt_table, gate, &s);
-	write_trace_idt_entry(gate, &s);
 }
 
-/*
- * This needs to use 'idt_table' rather than 'idt', and
- * thus use the _nonmapped_ version of the IDT, as the
- * Pentium F0 0F bugfix can have resulted in the mapped
- * IDT being write-protected.
- */
-#define set_intr_gate_notrace(n, addr)					\
-	do {								\
-		BUG_ON((unsigned)n > 0xFF);				\
-		_set_gate(n, GATE_INTERRUPT, (void *)addr, 0, 0,	\
-			  __KERNEL_CS);					\
-	} while (0)
-
-#define set_intr_gate(n, addr)						\
-	do {								\
-		set_intr_gate_notrace(n, addr);				\
-		_trace_set_gate(n, GATE_INTERRUPT, (void *)trace_##addr,\
-				0, 0, __KERNEL_CS);			\
-	} while (0)
+static inline void set_intr_gate(unsigned int n, const void *addr)
+{
+	BUG_ON((unsigned)n > 0xFF);
+	_set_gate(n, GATE_INTERRUPT, addr, 0, 0, __KERNEL_CS);
+}
 
 #ifdef CONFIG_X86_LOCAL_APIC
 extern int first_system_vector;
@@ -572,31 +529,6 @@ static inline void load_debug_idt(void)
 }
 #endif
 
-#ifdef CONFIG_TRACING
-extern atomic_t trace_idt_ctr;
-static inline bool is_trace_idt_enabled(void)
-{
-	if (atomic_read(&trace_idt_ctr))
-		return true;
-
-	return false;
-}
-
-static inline void load_trace_idt(void)
-{
-	load_idt((const struct desc_ptr *)&trace_idt_descr);
-}
-#else
-static inline bool is_trace_idt_enabled(void)
-{
-	return false;
-}
-
-static inline void load_trace_idt(void)
-{
-}
-#endif
-
 /*
  * The load_current_idt() must be called with interrupts disabled
  * to avoid races. That way the IDT will always be set back to the expected
@@ -608,8 +540,6 @@ static inline void load_current_idt(void
 {
 	if (is_debug_idt_enabled())
 		load_debug_idt();
-	else if (is_trace_idt_enabled())
-		load_trace_idt();
 	else
 		load_idt((const struct desc_ptr *)&idt_descr);
 }
--- a/arch/x86/include/asm/entry_arch.h
+++ b/arch/x86/include/asm/entry_arch.h
@@ -13,20 +13,16 @@
 BUILD_INTERRUPT(reschedule_interrupt,RESCHEDULE_VECTOR)
 BUILD_INTERRUPT(call_function_interrupt,CALL_FUNCTION_VECTOR)
 BUILD_INTERRUPT(call_function_single_interrupt,CALL_FUNCTION_SINGLE_VECTOR)
-BUILD_INTERRUPT3(irq_move_cleanup_interrupt, IRQ_MOVE_CLEANUP_VECTOR,
-		 smp_irq_move_cleanup_interrupt)
-BUILD_INTERRUPT3(reboot_interrupt, REBOOT_VECTOR, smp_reboot_interrupt)
+BUILD_INTERRUPT(irq_move_cleanup_interrupt, IRQ_MOVE_CLEANUP_VECTOR)
+BUILD_INTERRUPT(reboot_interrupt, REBOOT_VECTOR)
 #endif
 
 BUILD_INTERRUPT(x86_platform_ipi, X86_PLATFORM_IPI_VECTOR)
 
 #ifdef CONFIG_HAVE_KVM
-BUILD_INTERRUPT3(kvm_posted_intr_ipi, POSTED_INTR_VECTOR,
-		 smp_kvm_posted_intr_ipi)
-BUILD_INTERRUPT3(kvm_posted_intr_wakeup_ipi, POSTED_INTR_WAKEUP_VECTOR,
-		 smp_kvm_posted_intr_wakeup_ipi)
-BUILD_INTERRUPT3(kvm_posted_intr_nested_ipi, POSTED_INTR_NESTED_VECTOR,
-		 smp_kvm_posted_intr_nested_ipi)
+BUILD_INTERRUPT(kvm_posted_intr_ipi, POSTED_INTR_VECTOR)
+BUILD_INTERRUPT(kvm_posted_intr_wakeup_ipi, POSTED_INTR_WAKEUP_VECTOR)
+BUILD_INTERRUPT(kvm_posted_intr_nested_ipi, POSTED_INTR_NESTED_VECTOR)
 #endif
 
 /*
--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -46,26 +46,6 @@ extern asmlinkage void deferred_error_in
 extern asmlinkage void call_function_interrupt(void);
 extern asmlinkage void call_function_single_interrupt(void);
 
-#ifdef CONFIG_TRACING
-/* Interrupt handlers registered during init_IRQ */
-#define trace_reschedule_interrupt	reschedule_interrupt
-#define trace_call_function_interrupt	call_function_interrupt
-#define trace_call_function_single_interrupt	call_function_single_interrupt
-#define trace_thermal_interrupt	thermal_interrupt
-#define trace_threshold_interrupt	threshold_interrupt
-#define trace_deferred_error_interrupt	deferred_error_interrupt
-#define trace_irq_work_interrupt irq_work_interrupt
-#define trace_error_interrupt error_interrupt
-#define trace_spurious_interrupt spurious_interrupt
-#define trace_x86_platform_ipi	x86_platform_ipi
-#define trace_apic_timer_interrupt	apic_timer_interrupt
-#define trace_irq_move_cleanup_interrupt  irq_move_cleanup_interrupt
-#define trace_reboot_interrupt  reboot_interrupt
-#define trace_kvm_posted_intr_ipi kvm_posted_intr_ipi
-#define trace_kvm_posted_intr_wakeup_ipi kvm_posted_intr_wakeup_ipi
-#define trace_kvm_posted_intr_nested_ipi kvm_posted_intr_nested_ipi
-#endif /* CONFIG_TRACING */
-
 #ifdef	CONFIG_X86_LOCAL_APIC
 struct irq_data;
 struct pci_dev;
--- a/arch/x86/include/asm/segment.h
+++ b/arch/x86/include/asm/segment.h
@@ -238,9 +238,6 @@
 #ifndef __ASSEMBLY__
 
 extern const char early_idt_handler_array[NUM_EXCEPTION_VECTORS][EARLY_IDT_HANDLER_SIZE];
-#ifdef CONFIG_TRACING
-# define trace_early_idt_handler_array early_idt_handler_array
-#endif
 
 /*
  * Load a segment. Fall back on loading the zero segment if something goes
--- a/arch/x86/kernel/tracepoint.c
+++ b/arch/x86/kernel/tracepoint.c
@@ -10,58 +10,15 @@
 #include <asm/hw_irq.h>
 #include <asm/desc.h>
 
-atomic_t trace_idt_ctr = ATOMIC_INIT(0);
-struct desc_ptr trace_idt_descr = { NR_VECTORS * 16 - 1,
-				(unsigned long) trace_idt_table };
-
-/* No need to be aligned, but done to keep all IDTs defined the same way. */
-gate_desc trace_idt_table[NR_VECTORS] __page_aligned_bss;
-
 DEFINE_STATIC_KEY_FALSE(trace_irqvectors_key);
-static int trace_irq_vector_refcount;
-static DEFINE_MUTEX(irq_vector_mutex);
-
-static void set_trace_idt_ctr(int val)
-{
-	atomic_set(&trace_idt_ctr, val);
-	/* Ensure the trace_idt_ctr is set before sending IPI */
-	wmb();
-}
-
-static void switch_idt(void *arg)
-{
-	unsigned long flags;
-
-	local_irq_save(flags);
-	load_current_idt();
-	local_irq_restore(flags);
-}
 
 int trace_irq_vector_regfunc(void)
 {
 	static_branch_inc(&trace_irqvectors_key);
-
-	mutex_lock(&irq_vector_mutex);
-	if (!trace_irq_vector_refcount) {
-		set_trace_idt_ctr(1);
-		smp_call_function(switch_idt, NULL, 0);
-		switch_idt(NULL);
-	}
-	trace_irq_vector_refcount++;
-	mutex_unlock(&irq_vector_mutex);
 	return 0;
 }
 
 void trace_irq_vector_unregfunc(void)
 {
 	static_branch_dec(&trace_irqvectors_key);
-
-	mutex_lock(&irq_vector_mutex);
-	trace_irq_vector_refcount--;
-	if (!trace_irq_vector_refcount) {
-		set_trace_idt_ctr(0);
-		smp_call_function(switch_idt, NULL, 0);
-		switch_idt(NULL);
-	}
-	mutex_unlock(&irq_vector_mutex);
 }
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -939,12 +939,8 @@ void __init early_trap_init(void)
 	 * stack.  Using the original stack works well enough at this
 	 * early stage. DEBUG_STACK will be equipped after cpu_init() in
 	 * trap_init().
-	 *
-	 * We don't need to set trace_idt_table like set_intr_gate(),
-	 * since we don't have trace_debug and it will be reset to
-	 * 'debug' in trap_init() by set_intr_gate_ist().
 	 */
-	set_intr_gate_notrace(X86_TRAP_DB, debug);
+	set_intr_gate(X86_TRAP_DB, debug);
 	/* int3 can be called from all */
 	set_system_intr_gate(X86_TRAP_BP, &int3);
 #ifdef CONFIG_X86_32

[patch 16/41] x86/tracing: Disentangle pagefault and resched IPI tracing key

[Thomas Gleixner]

[-- Attachment #1: x86-tracing--Distangle-pagefault-and-resched-IPI.patch --]
[-- Type: text/plain, Size: 6334 bytes --]

The pagefault and the resched IPI handler are the only ones where it is
worth to optimize the code further in case tracepoints are disabled. But it
makes no sense to have a single static key for both.

Seperate the static keys so the facilities are handled seperately.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/trace/common.h      |   15 ++++++++-------
 arch/x86/include/asm/trace/exceptions.h  |    6 ++++--
 arch/x86/include/asm/trace/irq_vectors.h |   22 ++++++++++++++++------
 arch/x86/kernel/smp.c                    |    4 ++--
 arch/x86/kernel/tracepoint.c             |   27 ++++++++++++++++++++++-----
 arch/x86/mm/fault.c                      |    2 +-
 6 files changed, 53 insertions(+), 23 deletions(-)

Index: b/arch/x86/include/asm/trace/common.h
===================================================================
--- a/arch/x86/include/asm/trace/common.h
+++ b/arch/x86/include/asm/trace/common.h
@@ -1,15 +1,16 @@
 #ifndef _ASM_TRACE_COMMON_H
 #define _ASM_TRACE_COMMON_H
 
-extern int trace_irq_vector_regfunc(void);
-extern void trace_irq_vector_unregfunc(void);
-
 #ifdef CONFIG_TRACING
-DECLARE_STATIC_KEY_FALSE(trace_irqvectors_key);
-#define trace_irqvectors_enabled()			\
-	static_branch_unlikely(&trace_irqvectors_key)
+DECLARE_STATIC_KEY_FALSE(trace_pagefault_key);
+#define trace_pagefault_enabled()			\
+	static_branch_unlikely(&trace_pagefault_key)
+DECLARE_STATIC_KEY_FALSE(trace_resched_ipi_key);
+#define trace_resched_ipi_enabled()			\
+	static_branch_unlikely(&trace_resched_ipi_key)
 #else
-static inline bool trace_irqvectors_enabled(void) { return false; }
+static inline bool trace_pagefault_enabled(void) { return false; }
+static inline bool trace_resched_ipi_enabled(void) { return false; }
 #endif
 
 #endif
Index: b/arch/x86/include/asm/trace/exceptions.h
===================================================================
--- a/arch/x86/include/asm/trace/exceptions.h
+++ b/arch/x86/include/asm/trace/exceptions.h
@@ -7,6 +7,9 @@
 #include <linux/tracepoint.h>
 #include <asm/trace/common.h>
 
+extern int trace_pagefault_reg(void);
+extern void trace_pagefault_unreg(void);
+
 DECLARE_EVENT_CLASS(x86_exceptions,
 
 	TP_PROTO(unsigned long address, struct pt_regs *regs,
@@ -35,8 +38,7 @@ DEFINE_EVENT_FN(x86_exceptions, name,
 	TP_PROTO(unsigned long address,	struct pt_regs *regs,	\
 		 unsigned long error_code),			\
 	TP_ARGS(address, regs, error_code),			\
-	trace_irq_vector_regfunc,				\
-	trace_irq_vector_unregfunc);
+	trace_pagefault_reg, trace_pagefault_unreg);
 
 DEFINE_PAGE_FAULT_EVENT(page_fault_user);
 DEFINE_PAGE_FAULT_EVENT(page_fault_kernel);
Index: b/arch/x86/include/asm/trace/irq_vectors.h
===================================================================
--- a/arch/x86/include/asm/trace/irq_vectors.h
+++ b/arch/x86/include/asm/trace/irq_vectors.h
@@ -7,6 +7,9 @@
 #include <linux/tracepoint.h>
 #include <asm/trace/common.h>
 
+extern int trace_resched_ipi_reg(void);
+extern void trace_resched_ipi_unreg(void);
+
 DECLARE_EVENT_CLASS(x86_irq_vector,
 
 	TP_PROTO(int vector),
@@ -26,15 +29,22 @@ DECLARE_EVENT_CLASS(x86_irq_vector,
 #define DEFINE_IRQ_VECTOR_EVENT(name)		\
 DEFINE_EVENT_FN(x86_irq_vector, name##_entry,	\
 	TP_PROTO(int vector),			\
+	TP_ARGS(vector), NULL, NULL);		\
+DEFINE_EVENT_FN(x86_irq_vector, name##_exit,	\
+	TP_PROTO(int vector),			\
+	TP_ARGS(vector), NULL, NULL);
+
+#define DEFINE_RESCHED_IPI_EVENT(name)		\
+DEFINE_EVENT_FN(x86_irq_vector, name##_entry,	\
+	TP_PROTO(int vector),			\
 	TP_ARGS(vector),			\
-	trace_irq_vector_regfunc,		\
-	trace_irq_vector_unregfunc);		\
+	trace_resched_ipi_reg,			\
+	trace_resched_ipi_unreg);		\
 DEFINE_EVENT_FN(x86_irq_vector, name##_exit,	\
 	TP_PROTO(int vector),			\
 	TP_ARGS(vector),			\
-	trace_irq_vector_regfunc,		\
-	trace_irq_vector_unregfunc);
-
+	trace_resched_ipi_reg,			\
+	trace_resched_ipi_unreg);
 
 /*
  * local_timer - called when entering/exiting a local timer interrupt
@@ -45,7 +55,7 @@ DEFINE_IRQ_VECTOR_EVENT(local_timer);
 /*
  * reschedule - called when entering/exiting a reschedule vector handler
  */
-DEFINE_IRQ_VECTOR_EVENT(reschedule);
+DEFINE_RESCHED_IPI_EVENT(reschedule);
 
 /*
  * spurious_apic - called when entering/exiting a spurious apic vector handler
Index: b/arch/x86/kernel/smp.c
===================================================================
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -261,7 +261,7 @@ static void native_stop_other_cpus(int w
 {
 	ack_APIC_irq();
 
-	if (trace_irqvectors_enabled()) {
+	if (trace_resched_ipi_enabled()) {
 		/*
 		 * scheduler_ipi() might call irq_enter() as well, but
 		 * nested calls are fine.
@@ -273,7 +273,7 @@ static void native_stop_other_cpus(int w
 	inc_irq_stat(irq_resched_count);
 	scheduler_ipi();
 
-	if (trace_irqvectors_enabled()) {
+	if (trace_resched_ipi_enabled()) {
 		trace_reschedule_exit(RESCHEDULE_VECTOR);
 		irq_exit();
 	}
Index: b/arch/x86/kernel/tracepoint.c
===================================================================
--- a/arch/x86/kernel/tracepoint.c
+++ b/arch/x86/kernel/tracepoint.c
@@ -10,15 +10,32 @@
 #include <asm/hw_irq.h>
 #include <asm/desc.h>
 
-DEFINE_STATIC_KEY_FALSE(trace_irqvectors_key);
+DEFINE_STATIC_KEY_FALSE(trace_pagefault_key);
 
-int trace_irq_vector_regfunc(void)
+int trace_pagefault_reg(void)
 {
-	static_branch_inc(&trace_irqvectors_key);
+	static_branch_inc(&trace_pagefault_key);
 	return 0;
 }
 
-void trace_irq_vector_unregfunc(void)
+void trace_pagefault_unreg(void)
 {
-	static_branch_dec(&trace_irqvectors_key);
+	static_branch_dec(&trace_pagefault_key);
 }
+
+#ifdef CONFIG_SMP
+
+DEFINE_STATIC_KEY_FALSE(trace_resched_ipi_key);
+
+int trace_resched_ipi_reg(void)
+{
+	static_branch_inc(&trace_resched_ipi_key);
+	return 0;
+}
+
+void trace_resched_ipi_unreg(void)
+{
+	static_branch_dec(&trace_resched_ipi_key);
+}
+
+#endif
Index: b/arch/x86/mm/fault.c
===================================================================
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1509,7 +1509,7 @@ do_page_fault(struct pt_regs *regs, unsi
 	 *
 	 * exception_{enter,exit}() contain all sorts of tracepoints.
 	 */
-	if (trace_irqvectors_enabled())
+	if (trace_pagefault_enabled())
 		trace_page_fault_entries(address, regs, error_code);
 
 	prev_state = exception_enter();

[patch 17/41] x86/idt: Unify gate_struct handling for 32/64bit

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Unify-gate_struct-handling-for-32-64bit.patch --]
[-- Type: text/plain, Size: 7337 bytes --]

The first 32bits of gate struct are the same for 32 and 64 bit. The 32bit
version uses desc_struct and no designated data structure, so we need
different accessors for 32 and 64 bit. Aside of that the macros which are
necessary to build the 32bit gate descriptor are horrible to read.

Unify the gate structs and switch all code fiddling with it over.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/boot/compressed/eboot.c |    8 ++---
 arch/x86/include/asm/desc.h      |   45 +++++++++++++-----------------
 arch/x86/include/asm/desc_defs.h |   57 +++++++++++++++++++++++++--------------
 arch/x86/kvm/vmx.c               |    2 -
 arch/x86/xen/enlighten_pv.c      |   12 ++++----
 5 files changed, 67 insertions(+), 57 deletions(-)

--- a/arch/x86/boot/compressed/eboot.c
+++ b/arch/x86/boot/compressed/eboot.c
@@ -1058,7 +1058,7 @@ struct boot_params *efi_main(struct efi_
 		desc->s = DESC_TYPE_CODE_DATA;
 		desc->dpl = 0;
 		desc->p = 1;
-		desc->limit = 0xf;
+		desc->limit1 = 0xf;
 		desc->avl = 0;
 		desc->l = 0;
 		desc->d = SEG_OP_SIZE_32BIT;
@@ -1078,7 +1078,7 @@ struct boot_params *efi_main(struct efi_
 	desc->s = DESC_TYPE_CODE_DATA;
 	desc->dpl = 0;
 	desc->p = 1;
-	desc->limit = 0xf;
+	desc->limit1 = 0xf;
 	desc->avl = 0;
 	if (IS_ENABLED(CONFIG_X86_64)) {
 		desc->l = 1;
@@ -1099,7 +1099,7 @@ struct boot_params *efi_main(struct efi_
 	desc->s = DESC_TYPE_CODE_DATA;
 	desc->dpl = 0;
 	desc->p = 1;
-	desc->limit = 0xf;
+	desc->limit1 = 0xf;
 	desc->avl = 0;
 	desc->l = 0;
 	desc->d = SEG_OP_SIZE_32BIT;
@@ -1116,7 +1116,7 @@ struct boot_params *efi_main(struct efi_
 		desc->s = 0;
 		desc->dpl = 0;
 		desc->p = 1;
-		desc->limit = 0x0;
+		desc->limit1 = 0x0;
 		desc->avl = 0;
 		desc->l = 0;
 		desc->d = 0;
--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -83,33 +83,25 @@ static inline phys_addr_t get_cpu_gdt_pa
 	return per_cpu_ptr_to_phys(get_cpu_gdt_rw(cpu));
 }
 
-#ifdef CONFIG_X86_64
-
 static inline void pack_gate(gate_desc *gate, unsigned type, unsigned long func,
 			     unsigned dpl, unsigned ist, unsigned seg)
 {
-	gate->offset_low	= PTR_LOW(func);
+	gate->offset_low	= (u16) func;
+	gate->bits.p		= 1;
+	gate->bits.dpl		= dpl;
+	gate->bits.zero		= 0;
+	gate->bits.type		= type;
+	gate->offset_middle	= (u16) (func >> 16);
+#ifdef CONFIG_X86_64
 	gate->segment		= __KERNEL_CS;
-	gate->ist		= ist;
-	gate->p			= 1;
-	gate->dpl		= dpl;
-	gate->zero0		= 0;
-	gate->zero1		= 0;
-	gate->type		= type;
-	gate->offset_middle	= PTR_MIDDLE(func);
-	gate->offset_high	= PTR_HIGH(func);
-}
-
+	gate->bits.ist		= ist;
+	gate->reserved		= 0;
+	gate->offset_high	= (u32) (func >> 32);
 #else
-static inline void pack_gate(gate_desc *gate, unsigned char type,
-			     unsigned long base, unsigned dpl, unsigned flags,
-			     unsigned short seg)
-{
-	gate->a = (seg << 16) | (base & 0xffff);
-	gate->b = (base & 0xffff0000) | (((0x80 | type | (dpl << 5)) & 0xff) << 8);
-}
-
+	gate->segment		= seg;
+	gate->bits.ist		= 0;
 #endif
+}
 
 static inline int desc_empty(const void *ptr)
 {
@@ -185,7 +177,8 @@ static inline void pack_descriptor(struc
 }
 
 
-static inline void set_tssldt_descriptor(void *d, unsigned long addr, unsigned type, unsigned size)
+static inline void set_tssldt_descriptor(void *d, unsigned long addr,
+					 unsigned type, unsigned size)
 {
 #ifdef CONFIG_X86_64
 	struct ldttss_desc64 *desc = d;
@@ -193,13 +186,13 @@ static inline void set_tssldt_descriptor
 	memset(desc, 0, sizeof(*desc));
 
 	desc->limit0		= size & 0xFFFF;
-	desc->base0		= PTR_LOW(addr);
-	desc->base1		= PTR_MIDDLE(addr) & 0xFF;
+	desc->base0		= (u16) addr;
+	desc->base1		= (addr >> 16) & 0xFF;
 	desc->type		= type;
 	desc->p			= 1;
 	desc->limit1		= (size >> 16) & 0xF;
-	desc->base2		= (PTR_MIDDLE(addr) >> 8) & 0xFF;
-	desc->base3		= PTR_HIGH(addr);
+	desc->base2		= (addr >> 24) & 0xFF;
+	desc->base3		= (u32) (addr >> 32);
 #else
 	pack_descriptor((struct desc_struct *)d, addr, size, 0x80 | type, 0);
 #endif
--- a/arch/x86/include/asm/desc_defs.h
+++ b/arch/x86/include/asm/desc_defs.h
@@ -47,20 +47,6 @@ enum {
 	GATE_TASK = 0x5,
 };
 
-/* 16byte gate */
-struct gate_struct64 {
-	u16 offset_low;
-	u16 segment;
-	unsigned ist : 3, zero0 : 5, type : 5, dpl : 2, p : 1;
-	u16 offset_middle;
-	u32 offset_high;
-	u32 zero1;
-} __attribute__((packed));
-
-#define PTR_LOW(x) ((unsigned long long)(x) & 0xFFFF)
-#define PTR_MIDDLE(x) (((unsigned long long)(x) >> 16) & 0xFFFF)
-#define PTR_HIGH(x) ((unsigned long long)(x) >> 32)
-
 enum {
 	DESC_TSS = 0x9,
 	DESC_LDT = 0x2,
@@ -77,20 +63,51 @@ struct ldttss_desc64 {
 	u32 zero1;
 } __attribute__((packed));
 
+
 #ifdef CONFIG_X86_64
-typedef struct gate_struct64 gate_desc;
 typedef struct ldttss_desc64 ldt_desc;
 typedef struct ldttss_desc64 tss_desc;
-#define gate_offset(g) ((g).offset_low | ((unsigned long)(g).offset_middle << 16) | ((unsigned long)(g).offset_high << 32))
-#define gate_segment(g) ((g).segment)
 #else
-typedef struct desc_struct gate_desc;
 typedef struct desc_struct ldt_desc;
 typedef struct desc_struct tss_desc;
-#define gate_offset(g)		(((g).b & 0xffff0000) | ((g).a & 0x0000ffff))
-#define gate_segment(g)		((g).a >> 16)
 #endif
 
+struct idt_bits {
+	u16		ist	: 3,
+			zero	: 5,
+			type	: 5,
+			dpl	: 2,
+			p	: 1;
+} __attribute__((packed));
+
+struct gate_struct {
+	u16		offset_low;
+	u16		segment;
+	struct idt_bits	bits;
+	u16		offset_middle;
+#ifdef CONFIG_X86_64
+	u32		offset_high;
+	u32		reserved;
+#endif
+} __attribute__((packed));
+
+typedef struct gate_struct gate_desc;
+
+static inline unsigned long gate_offset(const gate_desc *g)
+{
+#ifdef CONFIG_X86_64
+	return g->offset_low | ((unsigned long)g->offset_middle << 16) |
+		((unsigned long) g->offset_high << 32);
+#else
+	return g->offset_low | ((unsigned long)g->offset_middle << 16);
+#endif
+}
+
+static inline unsigned long gate_segment(const gate_desc *g)
+{
+	return g->segment;
+}
+
 struct desc_ptr {
 	unsigned short size;
 	unsigned long address;
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -8786,7 +8786,7 @@ static void vmx_handle_external_intr(str
 
 		vector =  exit_intr_info & INTR_INFO_VECTOR_MASK;
 		desc = (gate_desc *)vmx->host_idt_base + vector;
-		entry = gate_offset(*desc);
+		entry = gate_offset(desc);
 		asm volatile(
 #ifdef CONFIG_X86_64
 			"mov %%" _ASM_SP ", %[sp]\n\t"
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -584,12 +584,12 @@ static int cvt_gate_to_trap(int vector,
 {
 	unsigned long addr;
 
-	if (val->type != GATE_TRAP && val->type != GATE_INTERRUPT)
+	if (val->bits.type != GATE_TRAP && val->bits.type != GATE_INTERRUPT)
 		return 0;
 
 	info->vector = vector;
 
-	addr = gate_offset(*val);
+	addr = gate_offset(val);
 #ifdef CONFIG_X86_64
 	/*
 	 * Look for known traps using IST, and substitute them
@@ -622,16 +622,16 @@ static int cvt_gate_to_trap(int vector,
 		;
 	else {
 		/* Some other trap using IST? */
-		if (WARN_ON(val->ist != 0))
+		if (WARN_ON(val->bits.ist != 0))
 			return 0;
 	}
 #endif	/* CONFIG_X86_64 */
 	info->address = addr;
 
-	info->cs = gate_segment(*val);
-	info->flags = val->dpl;
+	info->cs = gate_segment(val);
+	info->flags = val->bits.dpl;
 	/* interrupt gates clear IF */
-	if (val->type == GATE_INTERRUPT)
+	if (val->bits.type == GATE_INTERRUPT)
 		info->flags |= 1 << 2;
 
 	return 1;

[patch 18/41] x86/percpu: Use static initializer for GDT entry

[Thomas Gleixner]

[-- Attachment #1: x86-percpu--Use-static-initializer-for-gdt-entry.patch --]
[-- Type: text/plain, Size: 905 bytes --]

The IDT cleanup is about to remove pack_descriptor(). The GDT setup for the
percpu storage can be achieved with the static initializer as well. Replace
it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/setup_percpu.c |    9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

--- a/arch/x86/kernel/setup_percpu.c
+++ b/arch/x86/kernel/setup_percpu.c
@@ -155,13 +155,10 @@ static void __init pcpup_populate_pte(un
 static inline void setup_percpu_segment(int cpu)
 {
 #ifdef CONFIG_X86_32
-	struct desc_struct gdt;
+	struct desc_struct d = GDT_ENTRY_INIT(0x8092, per_cpu_offset(cpu),
+					      0xFFFFF);
 
-	pack_descriptor(&gdt, per_cpu_offset(cpu), 0xFFFFF,
-			0x2 | DESCTYPE_S, 0x8);
-	gdt.s = 1;
-	write_gdt_entry(get_cpu_gdt_rw(cpu),
-			GDT_ENTRY_PERCPU, &gdt, DESCTYPE_S);
+	write_gdt_entry(get_cpu_gdt_rw(cpu), GDT_ENTRY_PERCPU, &d, DESCTYPE_S);
 #endif
 }
 

[patch 19/41] x86/fpu: Use bitfield accessors for desc_struct

[Thomas Gleixner]

[-- Attachment #1: x86-fpu--Use-bitfield-accessors-for-desc_struct.patch --]
[-- Type: text/plain, Size: 5043 bytes --]

desc_struct is a union of u32 fields and bitfields. The access to the u32
fields is done with magic macros.

Convert it to use the bitfields and replace the macro magic with parseable
inline functions.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/math-emu/fpu_entry.c   |   11 ++++-----
 arch/x86/math-emu/fpu_system.h  |   48 ++++++++++++++++++++++++++++++----------
 arch/x86/math-emu/get_address.c |   17 +++++++-------
 3 files changed, 51 insertions(+), 25 deletions(-)

Index: b/arch/x86/math-emu/fpu_entry.c
===================================================================
--- a/arch/x86/math-emu/fpu_entry.c
+++ b/arch/x86/math-emu/fpu_entry.c
@@ -147,7 +147,7 @@ void math_emulate(struct math_emu_info *
 		}
 
 		code_descriptor = FPU_get_ldt_descriptor(FPU_CS);
-		if (SEG_D_SIZE(code_descriptor)) {
+		if (code_descriptor.d) {
 			/* The above test may be wrong, the book is not clear */
 			/* Segmented 32 bit protected mode */
 			addr_modes.default_mode = SEG32;
@@ -155,11 +155,10 @@ void math_emulate(struct math_emu_info *
 			/* 16 bit protected mode */
 			addr_modes.default_mode = PM16;
 		}
-		FPU_EIP += code_base = SEG_BASE_ADDR(code_descriptor);
-		code_limit = code_base
-		    + (SEG_LIMIT(code_descriptor) +
-		       1) * SEG_GRANULARITY(code_descriptor)
-		    - 1;
+		FPU_EIP += code_base = seg_get_base(&code_descriptor);
+		code_limit = seg_get_limit(&code_descriptor) + 1;
+		code_limit *= seg_get_granularity(&code_descriptor);
+		code_limit += code_base - 1;
 		if (code_limit < code_base)
 			code_limit = 0xffffffff;
 	}
Index: b/arch/x86/math-emu/fpu_system.h
===================================================================
--- a/arch/x86/math-emu/fpu_system.h
+++ b/arch/x86/math-emu/fpu_system.h
@@ -34,17 +34,43 @@ static inline struct desc_struct FPU_get
 	return ret;
 }
 
-#define SEG_D_SIZE(x)		((x).b & (3 << 21))
-#define SEG_G_BIT(x)		((x).b & (1 << 23))
-#define SEG_GRANULARITY(x)	(((x).b & (1 << 23)) ? 4096 : 1)
-#define SEG_286_MODE(x)		((x).b & ( 0xff000000 | 0xf0000 | (1 << 23)))
-#define SEG_BASE_ADDR(s)	(((s).b & 0xff000000) \
-				 | (((s).b & 0xff) << 16) | ((s).a >> 16))
-#define SEG_LIMIT(s)		(((s).b & 0xff0000) | ((s).a & 0xffff))
-#define SEG_EXECUTE_ONLY(s)	(((s).b & ((1 << 11) | (1 << 9))) == (1 << 11))
-#define SEG_WRITE_PERM(s)	(((s).b & ((1 << 11) | (1 << 9))) == (1 << 9))
-#define SEG_EXPAND_DOWN(s)	(((s).b & ((1 << 11) | (1 << 10))) \
-				 == (1 << 10))
+#define SEG_TYPE_WRITABLE	(1U << 1)
+#define SEG_TYPE_EXPANDS_DOWN	(1U << 2)
+#define SEG_TYPE_EXECUTE	(1U << 3)
+#define SEG_TYPE_EXPAND_MASK	(SEG_TYPE_EXPANDS_DOWN | SEG_TYPE_EXECUTE)
+#define SEG_TYPE_EXECUTE_MASK	(SEG_TYPE_WRITABLE | SEG_TYPE_EXECUTE)
+
+static inline unsigned long seg_get_base(struct desc_struct *d)
+{
+	unsigned long base = (unsigned long)d->base2 << 24;
+
+	return base | ((unsigned long)d->base1 << 16) | d->base0;
+}
+
+static inline unsigned long seg_get_limit(struct desc_struct *d)
+{
+	return ((unsigned long)d->limit << 16) | d->limit0;
+}
+
+static inline unsigned long seg_get_granularity(struct desc_struct *d)
+{
+	return d->g ? 4096 : 1;
+}
+
+static inline bool seg_expands_down(struct desc_struct *d)
+{
+	return (d->type & SEG_TYPE_EXPAND_MASK) == SEG_TYPE_EXPANDS_DOWN;
+}
+
+static inline bool seg_execute_only(struct desc_struct *d)
+{
+	return (d->type & SEG_TYPE_EXECUTE_MASK) == SEG_TYPE_EXECUTE;
+}
+
+static inline bool seg_writable(struct desc_struct *d)
+{
+	return (d->type & SEG_TYPE_EXECUTE_MASK) == SEG_TYPE_WRITABLE;
+}
 
 #define I387			(&current->thread.fpu.state)
 #define FPU_info		(I387->soft.info)
Index: b/arch/x86/math-emu/get_address.c
===================================================================
--- a/arch/x86/math-emu/get_address.c
+++ b/arch/x86/math-emu/get_address.c
@@ -159,17 +159,18 @@ static long pm_address(u_char FPU_modrm,
 	}
 
 	descriptor = FPU_get_ldt_descriptor(addr->selector);
-	base_address = SEG_BASE_ADDR(descriptor);
+	base_address = seg_get_base(&descriptor);
 	address = base_address + offset;
-	limit = base_address
-	    + (SEG_LIMIT(descriptor) + 1) * SEG_GRANULARITY(descriptor) - 1;
+	limit = seg_get_limit(&descriptor) + 1;
+	limit *= seg_get_granularity(&descriptor);
+	limit += base_address - 1;
 	if (limit < base_address)
 		limit = 0xffffffff;
 
-	if (SEG_EXPAND_DOWN(descriptor)) {
-		if (SEG_G_BIT(descriptor))
+	if (seg_expands_down(&descriptor)) {
+		if (descriptor.g) {
 			seg_top = 0xffffffff;
-		else {
+		} else {
 			seg_top = base_address + (1 << 20);
 			if (seg_top < base_address)
 				seg_top = 0xffffffff;
@@ -182,8 +183,8 @@ static long pm_address(u_char FPU_modrm,
 		    (address > limit) || (address < base_address) ? 0 :
 		    ((limit - address) >= 254 ? 255 : limit - address + 1);
 	}
-	if (SEG_EXECUTE_ONLY(descriptor) ||
-	    (!SEG_WRITE_PERM(descriptor) && (FPU_modrm & FPU_WRITE_BIT))) {
+	if (seg_execute_only(&descriptor) ||
+	    (!seg_writable(&descriptor) && (FPU_modrm & FPU_WRITE_BIT))) {
 		access_limit = 0;
 	}
 	return address;

[patch 20/41] x86: Replace access to desc_struct:a/b fields

[Thomas Gleixner]

[-- Attachment #1: x86--Replace-access-to-desc_struct-a-b-fields.patch --]
[-- Type: text/plain, Size: 2166 bytes --]

The union inside of desc_struct which allows access to the raw u32 parts of
the descriptors. This raw access part is about to go away.

Replace the few code parts which access those fields.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Juergen Gross <jgross@suse.com>
---
 arch/x86/include/asm/xen/hypercall.h |    6 ++++--
 arch/x86/kernel/tls.c                |    2 +-
 arch/x86/xen/enlighten_pv.c          |    2 +-
 3 files changed, 6 insertions(+), 4 deletions(-)

Index: b/arch/x86/include/asm/xen/hypercall.h
===================================================================
--- a/arch/x86/include/asm/xen/hypercall.h
+++ b/arch/x86/include/asm/xen/hypercall.h
@@ -552,6 +552,8 @@ static inline void
 MULTI_update_descriptor(struct multicall_entry *mcl, u64 maddr,
 			struct desc_struct desc)
 {
+	u32 *p = (u32 *) &desc;
+
 	mcl->op = __HYPERVISOR_update_descriptor;
 	if (sizeof(maddr) == sizeof(long)) {
 		mcl->args[0] = maddr;
@@ -559,8 +561,8 @@ MULTI_update_descriptor(struct multicall
 	} else {
 		mcl->args[0] = maddr;
 		mcl->args[1] = maddr >> 32;
-		mcl->args[2] = desc.a;
-		mcl->args[3] = desc.b;
+		mcl->args[2] = *p++;
+		mcl->args[3] = *p;
 	}
 
 	trace_xen_mc_entry(mcl, sizeof(maddr) == sizeof(long) ? 2 : 4);
Index: b/arch/x86/kernel/tls.c
===================================================================
--- a/arch/x86/kernel/tls.c
+++ b/arch/x86/kernel/tls.c
@@ -93,7 +93,7 @@ static void set_tls_desc(struct task_str
 
 	while (n-- > 0) {
 		if (LDT_empty(info) || LDT_zero(info)) {
-			desc->a = desc->b = 0;
+			memset(desc, 0, sizeof(*desc));
 		} else {
 			fill_ldt(desc, info);
 
Index: b/arch/x86/xen/enlighten_pv.c
===================================================================
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -494,7 +494,7 @@ static void __init xen_load_gdt_boot(con
 static inline bool desc_equal(const struct desc_struct *d1,
 			      const struct desc_struct *d2)
 {
-	return d1->a == d2->a && d1->b == d2->b;
+	return memcmp(d1, d2, sizeof(*d1));
 }
 
 static void load_TLS_descriptor(struct thread_struct *t,

[patch 21/41] x86/gdt: Use bitfields for initialization

[Thomas Gleixner]

[-- Attachment #1: x86-gdt--Use-bitfields-for-initialization.patch --]
[-- Type: text/plain, Size: 5463 bytes --]

The GDT entry related code uses partially bitfields and macros which
initialize the two 16 bit parts of the entry by magic shift and mask
operations.

Clean it up and use the bitfields to initialize and access entries.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/entry/vdso/vma.c        |    2 -
 arch/x86/include/asm/desc.h      |   26 ++++++++++++++---------
 arch/x86/include/asm/desc_defs.h |   44 +++++++++++++++++----------------------
 arch/x86/math-emu/fpu_system.h   |    2 -
 4 files changed, 38 insertions(+), 36 deletions(-)

Index: b/arch/x86/entry/vdso/vma.c
===================================================================
--- a/arch/x86/entry/vdso/vma.c
+++ b/arch/x86/entry/vdso/vma.c
@@ -351,7 +351,7 @@ static void vgetcpu_cpu_init(void *arg)
 	 * and 8 bits for the node)
 	 */
 	d.limit0 = cpu | ((node & 0xf) << 12);
-	d.limit = node >> 4;
+	d.limit1 = node >> 4;
 	d.type = 5;		/* RO data, expand down, accessed */
 	d.dpl = 3;		/* Visible to user code */
 	d.s = 1;		/* Not a system segment */
Index: b/arch/x86/include/asm/desc.h
===================================================================
--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -22,7 +22,7 @@ static inline void fill_ldt(struct desc_
 	desc->s			= 1;
 	desc->dpl		= 0x3;
 	desc->p			= info->seg_not_present ^ 1;
-	desc->limit		= (info->limit & 0xf0000) >> 16;
+	desc->limit1		= (info->limit & 0xf0000) >> 16;
 	desc->avl		= info->useable;
 	desc->d			= info->seg_32bit;
 	desc->g			= info->limit_in_pages;
@@ -169,14 +169,20 @@ static inline void pack_descriptor(struc
 				   unsigned long limit, unsigned char type,
 				   unsigned char flags)
 {
-	desc->a = ((base & 0xffff) << 16) | (limit & 0xffff);
-	desc->b = (base & 0xff000000) | ((base & 0xff0000) >> 16) |
-		(limit & 0x000f0000) | ((type & 0xff) << 8) |
-		((flags & 0xf) << 20);
-	desc->p = 1;
+	desc->limit0		= (u16) limit;
+	desc->base0		= (u16) base;
+	desc->base1		= (base >> 16) & 0xFF;
+	desc->type		= type & 0x0F;
+	desc->s			= 0;
+	desc->dpl		= 0;
+	desc->p			= 1;
+	desc->limit1		= (limit >> 16) & 0xF;
+	desc->avl		= (flags >> 0) & 0x01;
+	desc->l			= (flags >> 1) & 0x01;
+	desc->d			= (flags >> 2) & 0x01;
+	desc->g			= (flags >> 3) & 0x01;
 }
 
-
 static inline void set_tssldt_descriptor(void *d, unsigned long addr,
 					 unsigned type, unsigned size)
 {
@@ -194,7 +200,7 @@ static inline void set_tssldt_descriptor
 	desc->base2		= (addr >> 24) & 0xFF;
 	desc->base3		= (u32) (addr >> 32);
 #else
-	pack_descriptor((struct desc_struct *)d, addr, size, 0x80 | type, 0);
+	pack_descriptor((struct desc_struct *)d, addr, size, type, 0);
 #endif
 }
 
@@ -394,13 +400,13 @@ static inline void set_desc_base(struct
 
 static inline unsigned long get_desc_limit(const struct desc_struct *desc)
 {
-	return desc->limit0 | (desc->limit << 16);
+	return desc->limit0 | (desc->limit1 << 16);
 }
 
 static inline void set_desc_limit(struct desc_struct *desc, unsigned long limit)
 {
 	desc->limit0 = limit & 0xffff;
-	desc->limit = (limit >> 16) & 0xf;
+	desc->limit1 = (limit >> 16) & 0xf;
 }
 
 #ifdef CONFIG_X86_64
Index: b/arch/x86/include/asm/desc_defs.h
===================================================================
--- a/arch/x86/include/asm/desc_defs.h
+++ b/arch/x86/include/asm/desc_defs.h
@@ -11,34 +11,30 @@
 
 #include <linux/types.h>
 
-/*
- * FIXME: Accessing the desc_struct through its fields is more elegant,
- * and should be the one valid thing to do. However, a lot of open code
- * still touches the a and b accessors, and doing this allow us to do it
- * incrementally. We keep the signature as a struct, rather than a union,
- * so we can get rid of it transparently in the future -- glommer
- */
 /* 8 byte segment descriptor */
 struct desc_struct {
-	union {
-		struct {
-			unsigned int a;
-			unsigned int b;
-		};
-		struct {
-			u16 limit0;
-			u16 base0;
-			unsigned base1: 8, type: 4, s: 1, dpl: 2, p: 1;
-			unsigned limit: 4, avl: 1, l: 1, d: 1, g: 1, base2: 8;
-		};
-	};
+	u16	limit0;
+	u16	base0;
+	u16	base1: 8, type: 4, s: 1, dpl: 2, p: 1;
+	u16	limit1: 4, avl: 1, l: 1, d: 1, g: 1, base2: 8;
 } __attribute__((packed));
 
-#define GDT_ENTRY_INIT(flags, base, limit) { { { \
-		.a = ((limit) & 0xffff) | (((base) & 0xffff) << 16), \
-		.b = (((base) & 0xff0000) >> 16) | (((flags) & 0xf0ff) << 8) | \
-			((limit) & 0xf0000) | ((base) & 0xff000000), \
-	} } }
+#define GDT_ENTRY_INIT(flags, base, limit)			\
+	{							\
+		.limit0		= (u16) (limit),		\
+		.limit1		= ((limit) >> 16) & 0x0F,	\
+		.base0		= (u16) (base),			\
+		.base1		= ((base) >> 16) & 0xFF,	\
+		.base2		= ((base) >> 24) & 0xFF,	\
+		.type		= (flags & 0x0f),		\
+		.s		= (flags >> 4) & 0x01,		\
+		.dpl		= (flags >> 5) & 0x03,		\
+		.p		= (flags >> 7) & 0x01,		\
+		.avl		= (flags >> 12) & 0x01,		\
+		.l		= (flags >> 13) & 0x01,		\
+		.d		= (flags >> 14) & 0x01,		\
+		.g		= (flags >> 15) & 0x01,		\
+	}
 
 enum {
 	GATE_INTERRUPT = 0xE,
Index: b/arch/x86/math-emu/fpu_system.h
===================================================================
--- a/arch/x86/math-emu/fpu_system.h
+++ b/arch/x86/math-emu/fpu_system.h
@@ -49,7 +49,7 @@ static inline unsigned long seg_get_base
 
 static inline unsigned long seg_get_limit(struct desc_struct *d)
 {
-	return ((unsigned long)d->limit << 16) | d->limit0;
+	return ((unsigned long)d->limit1 << 16) | d->limit0;
 }
 
 static inline unsigned long seg_get_granularity(struct desc_struct *d)

[patch 22/41] x86/ldttss: Cleanup 32bit descriptors

[Thomas Gleixner]

[-- Attachment #1: x86-ldttss--Cleanup-32bit-descriptors.patch --]
[-- Type: text/plain, Size: 2660 bytes --]

Like the IDT descriptors the LDT/TSS descriptors are pointlessly different
on 32 and 64 bit.

Unify them and get rid of the duplicated code.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h      |   26 +++-----------------------
 arch/x86/include/asm/desc_defs.h |   27 ++++++++++++---------------
 2 files changed, 15 insertions(+), 38 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -165,42 +165,22 @@ native_write_gdt_entry(struct desc_struc
 	memcpy(&gdt[entry], desc, size);
 }
 
-static inline void pack_descriptor(struct desc_struct *desc, unsigned long base,
-				   unsigned long limit, unsigned char type,
-				   unsigned char flags)
-{
-	desc->limit0		= (u16) limit;
-	desc->base0		= (u16) base;
-	desc->base1		= (base >> 16) & 0xFF;
-	desc->type		= type & 0x0F;
-	desc->s			= 0;
-	desc->dpl		= 0;
-	desc->p			= 1;
-	desc->limit1		= (limit >> 16) & 0xF;
-	desc->avl		= (flags >> 0) & 0x01;
-	desc->l			= (flags >> 1) & 0x01;
-	desc->d			= (flags >> 2) & 0x01;
-	desc->g			= (flags >> 3) & 0x01;
-}
-
 static inline void set_tssldt_descriptor(void *d, unsigned long addr,
 					 unsigned type, unsigned size)
 {
-#ifdef CONFIG_X86_64
-	struct ldttss_desc64 *desc = d;
+	struct ldttss_desc *desc = d;
 
 	memset(desc, 0, sizeof(*desc));
 
-	desc->limit0		= size & 0xFFFF;
+	desc->limit0		= (u16) size;
 	desc->base0		= (u16) addr;
 	desc->base1		= (addr >> 16) & 0xFF;
 	desc->type		= type;
 	desc->p			= 1;
 	desc->limit1		= (size >> 16) & 0xF;
 	desc->base2		= (addr >> 24) & 0xFF;
+#ifdef CONFIG_X86_64
 	desc->base3		= (u32) (addr >> 32);
-#else
-	pack_descriptor((struct desc_struct *)d, addr, size, type, 0);
 #endif
 }
 
--- a/arch/x86/include/asm/desc_defs.h
+++ b/arch/x86/include/asm/desc_defs.h
@@ -49,24 +49,21 @@ enum {
 	DESCTYPE_S = 0x10,	/* !system */
 };
 
-/* LDT or TSS descriptor in the GDT. 16 bytes. */
-struct ldttss_desc64 {
-	u16 limit0;
-	u16 base0;
-	unsigned base1 : 8, type : 5, dpl : 2, p : 1;
-	unsigned limit1 : 4, zero0 : 3, g : 1, base2 : 8;
-	u32 base3;
-	u32 zero1;
-} __attribute__((packed));
-
+/* LDT or TSS descriptor in the GDT. */
+struct ldttss_desc {
+	u16	limit0;
+	u16	base0;
 
+	u16	base1 : 8, type : 5, dpl : 2, p : 1;
+	u16	limit1 : 4, zero0 : 3, g : 1, base2 : 8;
 #ifdef CONFIG_X86_64
-typedef struct ldttss_desc64 ldt_desc;
-typedef struct ldttss_desc64 tss_desc;
-#else
-typedef struct desc_struct ldt_desc;
-typedef struct desc_struct tss_desc;
+	u32	base3;
+	u32	zero1;
 #endif
+} __attribute__((packed));
+
+typedef struct ldttss_desc ldt_desc;
+typedef struct ldttss_desc tss_desc;
 
 struct idt_bits {
 	u16		ist	: 3,

[patch 23/41] x86/idt: Create file for IDT related code

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Create-file-for-idt-related-code.patch --]
[-- Type: text/plain, Size: 2924 bytes --]

IDT related code lives in different places. Create a new source file to
hold it.

Move the idt_tables and descriptors to it for a start. Follow up patches
will gradually move more code over.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/Makefile     |    2 +-
 arch/x86/kernel/cpu/common.c |    9 ---------
 arch/x86/kernel/idt.c        |   26 ++++++++++++++++++++++++++
 arch/x86/kernel/traps.c      |    6 ------
 4 files changed, 27 insertions(+), 16 deletions(-)

--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -42,7 +42,7 @@ CFLAGS_irq.o := -I$(src)/../include/asm/
 
 obj-y			:= process_$(BITS).o signal.o
 obj-$(CONFIG_COMPAT)	+= signal_compat.o
-obj-y			+= traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o
+obj-y			+= traps.o idt.o irq.o irq_$(BITS).o dumpstack_$(BITS).o
 obj-y			+= time.o ioport.o dumpstack.o nmi.o
 obj-$(CONFIG_MODIFY_LDT_SYSCALL)	+= ldt.o
 obj-y			+= setup.o x86_init.o i8259.o irqinit.o jump_label.o
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1289,15 +1289,6 @@ static __init int setup_disablecpuid(cha
 __setup("clearcpuid=", setup_disablecpuid);
 
 #ifdef CONFIG_X86_64
-struct desc_ptr idt_descr __ro_after_init = {
-	.size = NR_VECTORS * 16 - 1,
-	.address = (unsigned long) idt_table,
-};
-const struct desc_ptr debug_idt_descr = {
-	.size = NR_VECTORS * 16 - 1,
-	.address = (unsigned long) debug_idt_table,
-};
-
 DEFINE_PER_CPU_FIRST(union irq_stack_union,
 		     irq_stack_union) __aligned(PAGE_SIZE) __visible;
 
--- /dev/null
+++ b/arch/x86/kernel/idt.c
@@ -0,0 +1,26 @@
+/*
+ * Interrupt descriptor table related code
+ *
+ * This file is licensed under the GPL V2
+ */
+#include <linux/interrupt.h>
+
+#include <asm/desc.h>
+
+/* Must be page-aligned because the real IDT is used in a fixmap. */
+gate_desc idt_table[IDT_ENTRIES] __page_aligned_bss;
+
+#ifdef CONFIG_X86_64
+/* No need to be aligned, but done to keep all IDTs defined the same way. */
+gate_desc debug_idt_table[IDT_ENTRIES] __page_aligned_bss;
+
+struct desc_ptr idt_descr __ro_after_init = {
+	.size		= IDT_ENTRIES * 16 - 1,
+	.address	= (unsigned long) idt_table,
+};
+
+const struct desc_ptr debug_idt_descr = {
+	.size		= IDT_ENTRIES * 16 - 1,
+	.address	= (unsigned long) debug_idt_table,
+};
+#endif
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -65,18 +65,12 @@
 #include <asm/x86_init.h>
 #include <asm/pgalloc.h>
 #include <asm/proto.h>
-
-/* No need to be aligned, but done to keep all IDTs defined the same way. */
-gate_desc debug_idt_table[NR_VECTORS] __page_aligned_bss;
 #else
 #include <asm/processor-flags.h>
 #include <asm/setup.h>
 #include <asm/proto.h>
 #endif
 
-/* Must be page-aligned because the real IDT is used in a fixmap. */
-gate_desc idt_table[NR_VECTORS] __page_aligned_bss;
-
 DECLARE_BITMAP(used_vectors, NR_VECTORS);
 
 static inline void cond_local_irq_enable(struct pt_regs *regs)

[patch 24/41] x86/idt: Move 32bit idt_descr to C code

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Move-32bit-idt_descr-to-C-code.patch --]
[-- Type: text/plain, Size: 1753 bytes --]

32bit has the idt_descr sitting in the low level assembly entry code. There
is no reason for that. Move it into the C file and use the 64bit version of
it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/head_32.S |    6 ------
 arch/x86/kernel/idt.c     |   10 +++++-----
 2 files changed, 5 insertions(+), 11 deletions(-)

--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -626,7 +626,6 @@ ENTRY(initial_stack)
 
 	.data
 .globl boot_gdt_descr
-.globl idt_descr
 
 	ALIGN
 # early boot GDT descriptor (must use 1:1 address mapping)
@@ -635,11 +634,6 @@ ENTRY(initial_stack)
 	.word __BOOT_DS+7
 	.long boot_gdt - __PAGE_OFFSET
 
-	.word 0				# 32-bit align idt_desc.address
-idt_descr:
-	.word IDT_ENTRIES*8-1		# idt contains 256 entries
-	.long idt_table
-
 # boot GDT descriptor (later on used by CPU#0):
 	.word 0				# 32 bit align gdt_desc.address
 ENTRY(early_gdt_descr)
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -10,15 +10,15 @@
 /* Must be page-aligned because the real IDT is used in a fixmap. */
 gate_desc idt_table[IDT_ENTRIES] __page_aligned_bss;
 
-#ifdef CONFIG_X86_64
-/* No need to be aligned, but done to keep all IDTs defined the same way. */
-gate_desc debug_idt_table[IDT_ENTRIES] __page_aligned_bss;
-
 struct desc_ptr idt_descr __ro_after_init = {
-	.size		= IDT_ENTRIES * 16 - 1,
+	.size		= (IDT_ENTRIES * 2 * sizeof(unsigned long)) - 1,
 	.address	= (unsigned long) idt_table,
 };
 
+#ifdef CONFIG_X86_64
+/* No need to be aligned, but done to keep all IDTs defined the same way. */
+gate_desc debug_idt_table[IDT_ENTRIES] __page_aligned_bss;
+
 const struct desc_ptr debug_idt_descr = {
 	.size		= IDT_ENTRIES * 16 - 1,
 	.address	= (unsigned long) debug_idt_table,

[patch 25/41] x86/idt: Remove unused set_trap_gate()

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Remove-unused-set_trap_gate--.patch --]
[-- Type: text/plain, Size: 786 bytes --]

This inline is not used at all.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |   12 ------------
 1 file changed, 12 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -455,18 +455,6 @@ static inline void set_system_intr_gate(
 	_set_gate(n, GATE_INTERRUPT, addr, 0x3, 0, __KERNEL_CS);
 }
 
-static inline void set_system_trap_gate(unsigned int n, void *addr)
-{
-	BUG_ON((unsigned)n > 0xFF);
-	_set_gate(n, GATE_TRAP, addr, 0x3, 0, __KERNEL_CS);
-}
-
-static inline void set_trap_gate(unsigned int n, void *addr)
-{
-	BUG_ON((unsigned)n > 0xFF);
-	_set_gate(n, GATE_TRAP, addr, 0, 0, __KERNEL_CS);
-}
-
 static inline void set_task_gate(unsigned int n, unsigned int gdt_entry)
 {
 	BUG_ON((unsigned)n > 0xFF);

[patch 26/41] x86/idt: Consolidate IDT invalidation

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Consolidate-idt-invalidation.patch --]
[-- Type: text/plain, Size: 2416 bytes --]

kexec and reboot have both code to invalidate IDT. Create a common function
and use it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h        |    3 +++
 arch/x86/kernel/idt.c              |   11 +++++++++++
 arch/x86/kernel/machine_kexec_32.c |   14 +-------------
 arch/x86/kernel/reboot.c           |    4 +---
 4 files changed, 16 insertions(+), 16 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -512,4 +512,7 @@ static inline void load_current_idt(void
 	else
 		load_idt((const struct desc_ptr *)&idt_descr);
 }
+
+extern void idt_invalidate(void *addr);
+
 #endif /* _ASM_X86_DESC_H */
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -24,3 +24,14 @@ const struct desc_ptr debug_idt_descr =
 	.address	= (unsigned long) debug_idt_table,
 };
 #endif
+
+/**
+ * idt_invalidate - Invalidate interrupt descriptor table
+ * @addr:	The virtual address of the 'invalid' IDT
+ */
+void idt_invalidate(void *addr)
+{
+	struct desc_ptr idt = { .address = (unsigned long) addr, .size = 0 };
+
+	load_idt(&idt);
+}
--- a/arch/x86/kernel/machine_kexec_32.c
+++ b/arch/x86/kernel/machine_kexec_32.c
@@ -26,18 +26,6 @@
 #include <asm/set_memory.h>
 #include <asm/debugreg.h>
 
-static void set_idt(void *newidt, __u16 limit)
-{
-	struct desc_ptr curidt;
-
-	/* ia32 supports unaliged loads & stores */
-	curidt.size    = limit;
-	curidt.address = (unsigned long)newidt;
-
-	load_idt(&curidt);
-}
-
-
 static void set_gdt(void *newgdt, __u16 limit)
 {
 	struct desc_ptr curgdt;
@@ -245,7 +233,7 @@ void machine_kexec(struct kimage *image)
 	 * If you want to load them you must set up your own idt & gdt.
 	 */
 	set_gdt(phys_to_virt(0), 0);
-	set_idt(phys_to_virt(0), 0);
+	idt_invalidate(phys_to_virt(0));
 
 	/* now call it */
 	image->start = relocate_kernel_ptr((unsigned long)image->head,
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -38,8 +38,6 @@
 void (*pm_power_off)(void);
 EXPORT_SYMBOL(pm_power_off);
 
-static const struct desc_ptr no_idt = {};
-
 /*
  * This is set if we need to go through the 'emergency' path.
  * When machine_emergency_restart() is called, we may be on
@@ -638,7 +636,7 @@ static void native_machine_emergency_res
 			break;
 
 		case BOOT_TRIPLE:
-			load_idt(&no_idt);
+			idt_invalidate(NULL);
 			__asm__ __volatile__("int3");
 
 			/* We're probably dead after this, but... */

[patch 27/41] x86/idt: Move early IDT handler setup to IDT code

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Move-early-idt-handler-setup-to-idt-code.patch --]
[-- Type: text/plain, Size: 2098 bytes --]

The early IDT handler setup is done in C entry code for 64 bit and in ASM
entry code for 32 bit. Move the 64bit variant to the IDT code so it can be
shared with 32bit in the next step.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |    9 +++++++++
 arch/x86/kernel/head64.c    |    6 +-----
 arch/x86/kernel/idt.c       |   12 ++++++++++++
 3 files changed, 22 insertions(+), 5 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -511,6 +511,15 @@ static inline void load_current_idt(void
 		load_idt((const struct desc_ptr *)&idt_descr);
 }
 
+extern void idt_setup_early_handler(void);
+extern void idt_setup_early_traps(void);
+
+#ifdef CONFIG_X86_64
+extern void idt_setup_early_pf(void);
+#else
+static inline void idt_setup_early_pf(void) { }
+#endif
+
 extern void idt_invalidate(void *addr);
 
 #endif /* _ASM_X86_DESC_H */
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -311,8 +311,6 @@ static void __init copy_bootdata(char *r
 
 asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
 {
-	int i;
-
 	/*
 	 * Build-time sanity checks on the kernel image and module
 	 * area mappings. (these are purely build-time and produce no code)
@@ -345,9 +343,7 @@ asmlinkage __visible void __init x86_64_
 
 	kasan_early_init();
 
-	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++)
-		set_intr_gate(i, early_idt_handler_array[i]);
-	load_idt((const struct desc_ptr *)&idt_descr);
+	idt_setup_early_handler();
 
 	copy_bootdata(__va(real_mode_data));
 
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -26,6 +26,18 @@ const struct desc_ptr debug_idt_descr =
 #endif
 
 /**
+ * idt_setup_early_handler - Initializes the idt table with early handlers
+ */
+void __init idt_setup_early_handler(void)
+{
+	int i;
+
+	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++)
+		set_intr_gate(i, early_idt_handler_array[i]);
+	load_idt((const struct desc_ptr *)&idt_descr);
+}
+
+/**
  * idt_invalidate - Invalidate interrupt descriptor table
  * @addr:	The virtual address of the 'invalid' IDT
  */

[patch 28/41] x86/idt: Move early IDT setup out of 32bit asm

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Move-early-idt-setup-out-of-32bit-asm.patch --]
[-- Type: text/plain, Size: 3717 bytes --]

The early IDT setup can be done in C code like it's done on 64 bit. Reuse
the 64 bit version.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/segment.h |    1 +
 arch/x86/kernel/head32.c       |    4 ++++
 arch/x86/kernel/head_32.S      |   36 ++----------------------------------
 arch/x86/kernel/idt.c          |    4 ++++
 4 files changed, 11 insertions(+), 34 deletions(-)

Index: b/arch/x86/include/asm/segment.h
===================================================================
--- a/arch/x86/include/asm/segment.h
+++ b/arch/x86/include/asm/segment.h
@@ -238,6 +238,7 @@
 #ifndef __ASSEMBLY__
 
 extern const char early_idt_handler_array[NUM_EXCEPTION_VECTORS][EARLY_IDT_HANDLER_SIZE];
+extern void early_ignore_irq(void);
 
 /*
  * Load a segment. Fall back on loading the zero segment if something goes
Index: b/arch/x86/kernel/head32.c
===================================================================
--- a/arch/x86/kernel/head32.c
+++ b/arch/x86/kernel/head32.c
@@ -10,6 +10,7 @@
 #include <linux/mm.h>
 #include <linux/memblock.h>
 
+#include <asm/desc.h>
 #include <asm/setup.h>
 #include <asm/sections.h>
 #include <asm/e820/api.h>
@@ -30,6 +31,9 @@ static void __init i386_default_early_se
 asmlinkage __visible void __init i386_start_kernel(void)
 {
 	cr4_init_shadow();
+
+	idt_setup_early_handler();
+
 	sanitize_boot_params(&boot_params);
 
 	x86_early_init_platform_quirks();
Index: b/arch/x86/kernel/head_32.S
===================================================================
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -345,7 +345,6 @@ ENTRY(startup_32_smp)
 	movl %eax,%cr0
 
 	lgdt early_gdt_descr
-	lidt idt_descr
 	ljmp $(__KERNEL_CS),$1f
 1:	movl $(__KERNEL_DS),%eax	# reload all the segment registers
 	movl %eax,%ss			# after changing gdt.
@@ -378,37 +377,6 @@ ENDPROC(startup_32_smp)
  */
 __INIT
 setup_once:
-	/*
-	 * Set up a idt with 256 interrupt gates that push zero if there
-	 * is no error code and then jump to early_idt_handler_common.
-	 * It doesn't actually load the idt - that needs to be done on
-	 * each CPU. Interrupts are enabled elsewhere, when we can be
-	 * relatively sure everything is ok.
-	 */
-
-	movl $idt_table,%edi
-	movl $early_idt_handler_array,%eax
-	movl $NUM_EXCEPTION_VECTORS,%ecx
-1:
-	movl %eax,(%edi)
-	movl %eax,4(%edi)
-	/* interrupt gate, dpl=0, present */
-	movl $(0x8E000000 + __KERNEL_CS),2(%edi)
-	addl $EARLY_IDT_HANDLER_SIZE,%eax
-	addl $8,%edi
-	loop 1b
-
-	movl $256 - NUM_EXCEPTION_VECTORS,%ecx
-	movl $ignore_int,%edx
-	movl $(__KERNEL_CS << 16),%eax
-	movw %dx,%ax		/* selector = 0x0010 = cs */
-	movw $0x8E00,%dx	/* interrupt gate - dpl=0, present */
-2:
-	movl %eax,(%edi)
-	movl %edx,4(%edi)
-	addl $8,%edi
-	loop 2b
-
 #ifdef CONFIG_CC_STACKPROTECTOR
 	/*
 	 * Configure the stack canary. The linker can't handle this by
@@ -498,7 +466,7 @@ ENDPROC(early_idt_handler_common)
 
 /* This is the default interrupt "handler" :-) */
 	ALIGN
-ignore_int:
+ENTRY(early_ignore_irq)
 	cld
 #ifdef CONFIG_PRINTK
 	pushl %eax
@@ -533,7 +501,7 @@ ENDPROC(early_idt_handler_common)
 hlt_loop:
 	hlt
 	jmp hlt_loop
-ENDPROC(ignore_int)
+ENDPROC(early_ignore_irq)
 __INITDATA
 	.align 4
 GLOBAL(early_recursion_flag)
Index: b/arch/x86/kernel/idt.c
===================================================================
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -34,6 +34,10 @@ void __init idt_setup_early_handler(void
 
 	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++)
 		set_intr_gate(i, early_idt_handler_array[i]);
+#ifdef CONFIG_X86_32
+	for ( ; i < NR_VECTORS; i++)
+		set_intr_gate(i, early_ignore_irq);
+#endif
 	load_idt((const struct desc_ptr *)&idt_descr);
 }
 

[patch 29/41] x86/idt: Prepare for table based init

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Prepare-for-table-based-init.patch --]
[-- Type: text/plain, Size: 2617 bytes --]

The IDT setup code is handled in several places. All of them use variants
of set_intr_gate() inlines. This can be done with a table based
initialization, which allows to reduce the inline zoo and puts all IDT
related code and information into a single place.

Add the infrastructure.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/idt.c |   67 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

Index: b/arch/x86/kernel/idt.c
===================================================================
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -5,8 +5,49 @@
  */
 #include <linux/interrupt.h>
 
+#include <asm/traps.h>
+#include <asm/proto.h>
 #include <asm/desc.h>
 
+struct idt_data {
+	unsigned int	vector;
+	unsigned int	segment;
+	struct idt_bits	bits;
+	const void	*addr;
+};
+
+#define DPL0		0x0
+#define DPL3		0x3
+
+#define DEFAULT_STACK	0
+
+#define G(_vector, _addr, _ist, _type, _dpl, _segment)	\
+	{						\
+		.vector		= _vector,		\
+		.bits.ist	= _ist,			\
+		.bits.type	= _type,		\
+		.bits.dpl	= _dpl,			\
+		.bits.p		= 1,			\
+		.addr		= _addr,		\
+		.segment	= _segment,		\
+	}
+
+/* Interrupt gate */
+#define INTG(_vector, _addr)				\
+	G(_vector, _addr, DEFAULT_STACK, GATE_INTERRUPT, DPL0, __KERNEL_CS)
+
+/* System interrupt gate */
+#define SYSG(_vector, _addr)				\
+	G(_vector, _addr, DEFAULT_STACK, GATE_INTERRUPT, DPL3, __KERNEL_CS)
+
+/* Interrupt gate with interrupt stack */
+#define ISTG(_vector, _addr, _ist)			\
+	G(_vector, _addr, _ist, GATE_INTERRUPT, DPL0, __KERNEL_CS)
+
+/* Task gate */
+#define TSKG(_vector, _gdt)				\
+	G(_vector, NULL, DEFAULT_STACK, GATE_TASK, DPL0, _gdt << 3)
+
 /* Must be page-aligned because the real IDT is used in a fixmap. */
 gate_desc idt_table[IDT_ENTRIES] __page_aligned_bss;
 
@@ -25,6 +66,32 @@ const struct desc_ptr debug_idt_descr =
 };
 #endif
 
+static inline void idt_init_desc(gate_desc *gate, const struct idt_data *d)
+{
+	unsigned long addr = (unsigned long) d->addr;
+
+	gate->offset_low	= (u16) addr;
+	gate->segment		= (u16) d->segment;
+	gate->bits		= d->bits;
+	gate->offset_middle	= (u16) (addr >> 16);
+#ifdef CONFIG_X86_64
+	gate->offset_high	= (u32) (addr >> 32);
+	gate->reserved		= 0;
+#endif
+}
+
+static __init void
+idt_setup_from_table(gate_desc *idt, const struct idt_data *t, int size)
+{
+	gate_desc desc;
+
+	for (; size > 0; t++, size--) {
+		idt_init_desc(&desc, t);
+		set_bit(t->vector, used_vectors);
+		write_idt_entry(idt, t->vector, &desc);
+	}
+}
+
 /**
  * idt_setup_early_handler - Initializes the idt table with early handlers
  */

[patch 30/41] x86/idt: Switch early trap init to IDT tables

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Switch-early-trao-init-to-tables.patch --]
[-- Type: text/plain, Size: 4061 bytes --]

Add the initialization table for the early trap setup and replace the early
trap init code.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/idt.c   |   52 ++++++++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/setup.c |    4 +--
 arch/x86/kernel/traps.c |   27 ------------------------
 3 files changed, 54 insertions(+), 29 deletions(-)

Index: b/arch/x86/kernel/idt.c
===================================================================
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -48,6 +48,28 @@ struct idt_data {
 #define TSKG(_vector, _gdt)				\
 	G(_vector, NULL, DEFAULT_STACK, GATE_TASK, DPL0, _gdt << 3)
 
+/*
+ * Early traps running on the DEFAULT_STACK because the other interrupt
+ * stacks work only after cpu_init().
+ */
+static const __initdata struct idt_data early_idts[] = {
+	INTG(X86_TRAP_DB,		debug),
+	SYSG(X86_TRAP_BP,		int3),
+#ifdef CONFIG_X86_32
+	INTG(X86_TRAP_PF,		page_fault),
+#endif
+};
+
+#ifdef CONFIG_X86_64
+/*
+ * Early traps running on the DEFAULT_STACK because the other interrupt
+ * stacks work only after cpu_init().
+ */
+static const __initdata struct idt_data early_pf_idts[] = {
+	INTG(X86_TRAP_PF,		page_fault),
+};
+#endif
+
 /* Must be page-aligned because the real IDT is used in a fixmap. */
 gate_desc idt_table[IDT_ENTRIES] __page_aligned_bss;
 
@@ -93,6 +115,36 @@ idt_setup_from_table(gate_desc *idt, con
 }
 
 /**
+ * idt_setup_early_traps - Initialize the idt table with early traps
+ *
+ * On X8664 these traps do not use interrupt stacks as they can't work
+ * before cpu_init() is invoked and sets up TSS. The IST variants are
+ * installed after that.
+ */
+void __init idt_setup_early_traps(void)
+{
+	idt_setup_from_table(idt_table, early_idts, ARRAY_SIZE(early_idts));
+}
+
+#ifdef CONFIG_X86_64
+/**
+ * idt_setup_early_pf - Initialize the idt table with early pagefault handler
+ *
+ * On X8664 this does not use interrupt stacks as they can't work before
+ * cpu_init() is invoked and sets up TSS. The IST variant is installed
+ * after that.
+ *
+ * FIXME: Why is 32bit and 64bit installing the PF handler at different
+ * places in the early setup code?
+ */
+void __init idt_setup_early_pf(void)
+{
+	idt_setup_from_table(idt_table, early_pf_idts,
+			     ARRAY_SIZE(early_pf_idts));
+}
+#endif
+
+/**
  * idt_setup_early_handler - Initializes the idt table with early handlers
  */
 void __init idt_setup_early_handler(void)
Index: b/arch/x86/kernel/setup.c
===================================================================
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -891,7 +891,7 @@ void __init setup_arch(char **cmdline_p)
 	 */
 	olpc_ofw_detect();
 
-	early_trap_init();
+	idt_setup_early_traps();
 	early_cpu_init();
 	early_ioremap_init();
 
@@ -1162,7 +1162,7 @@ void __init setup_arch(char **cmdline_p)
 
 	init_mem_mapping();
 
-	early_trap_pf_init();
+	idt_setup_early_pf();
 
 	/*
 	 * Update mmu_cr4_features (and, indirectly, trampoline_cr4_features)
Index: b/arch/x86/kernel/traps.c
===================================================================
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -923,33 +923,6 @@ dotraplinkage void do_iret_error(struct
 }
 #endif
 
-/* Set of traps needed for early debugging. */
-void __init early_trap_init(void)
-{
-	/*
-	 * Don't use IST to set DEBUG_STACK as it doesn't work until TSS
-	 * is ready in cpu_init() <-- trap_init(). Before trap_init(),
-	 * CPU runs at ring 0 so it is impossible to hit an invalid
-	 * stack.  Using the original stack works well enough at this
-	 * early stage. DEBUG_STACK will be equipped after cpu_init() in
-	 * trap_init().
-	 */
-	set_intr_gate(X86_TRAP_DB, debug);
-	/* int3 can be called from all */
-	set_system_intr_gate(X86_TRAP_BP, &int3);
-#ifdef CONFIG_X86_32
-	set_intr_gate(X86_TRAP_PF, page_fault);
-#endif
-	load_idt(&idt_descr);
-}
-
-void __init early_trap_pf_init(void)
-{
-#ifdef CONFIG_X86_64
-	set_intr_gate(X86_TRAP_PF, page_fault);
-#endif
-}
-
 void __init trap_init(void)
 {
 	int i;

[patch 31/41] x86/idt: Move debug stack init to table based

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Move-debug-stack-init-to-table-based.patch --]
[-- Type: text/plain, Size: 2502 bytes --]

Add the debug_idt init table and make use of it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |    2 ++
 arch/x86/kernel/idt.c       |   23 +++++++++++++++++++++++
 arch/x86/kernel/traps.c     |    6 +-----
 3 files changed, 26 insertions(+), 5 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -516,8 +516,10 @@ extern void idt_setup_early_traps(void);
 
 #ifdef CONFIG_X86_64
 extern void idt_setup_early_pf(void);
+extern void idt_setup_debugidt_traps(void);
 #else
 static inline void idt_setup_early_pf(void) { }
+static inline void idt_setup_debugidt_traps(void) { }
 #endif
 
 extern void idt_invalidate(void *addr);
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -68,6 +68,15 @@ static const __initdata struct idt_data
 static const __initdata struct idt_data early_pf_idts[] = {
 	INTG(X86_TRAP_PF,		page_fault),
 };
+
+/*
+ * Override for the debug_idt. Same as the default, but with interrupt
+ * stack set to DEFAULT_STACK (0). Required for NMI trap handling.
+ */
+static const __initdata struct idt_data dbg_idts[] = {
+	INTG(X86_TRAP_DB,	debug),
+	INTG(X86_TRAP_BP,	int3),
+};
 #endif
 
 /* Must be page-aligned because the real IDT is used in a fixmap. */
@@ -82,6 +91,10 @@ struct desc_ptr idt_descr __ro_after_ini
 /* No need to be aligned, but done to keep all IDTs defined the same way. */
 gate_desc debug_idt_table[IDT_ENTRIES] __page_aligned_bss;
 
+/*
+ * Override for the debug_idt. Same as the default, but with interrupt
+ * stack set to DEFAULT_STACK (0). Required for NMI trap handling.
+ */
 const struct desc_ptr debug_idt_descr = {
 	.size		= IDT_ENTRIES * 16 - 1,
 	.address	= (unsigned long) debug_idt_table,
@@ -142,6 +155,16 @@ void __init idt_setup_early_pf(void)
 	idt_setup_from_table(idt_table, early_pf_idts,
 			     ARRAY_SIZE(early_pf_idts));
 }
+
+/**
+ * idt_setup_debugidt_traps - Initialize the debug idt table with debug traps
+ */
+void __init idt_setup_debugidt_traps(void)
+{
+	memcpy(&debug_idt_table, &idt_table, IDT_ENTRIES * 16);
+
+	idt_setup_from_table(debug_idt_table, dbg_idts, ARRAY_SIZE(dbg_idts));
+}
 #endif
 
 /**
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -990,9 +990,5 @@ void __init trap_init(void)
 
 	x86_init.irqs.trap_init();
 
-#ifdef CONFIG_X86_64
-	memcpy(&debug_idt_table, &idt_table, IDT_ENTRIES * 16);
-	set_nmi_gate(X86_TRAP_DB, &debug);
-	set_nmi_gate(X86_TRAP_BP, &int3);
-#endif
+	idt_setup_debugidt_traps();
 }

[patch 32/41] x86/idt: Move ist stack based traps to table init

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Move-ist-stack-based-traps-to-table-init.patch --]
[-- Type: text/plain, Size: 2426 bytes --]

Initialize the IST based traps via a table

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |    2 ++
 arch/x86/kernel/idt.c       |   22 ++++++++++++++++++++++
 arch/x86/kernel/traps.c     |    9 +--------
 3 files changed, 25 insertions(+), 8 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -516,9 +516,11 @@ extern void idt_setup_early_traps(void);
 
 #ifdef CONFIG_X86_64
 extern void idt_setup_early_pf(void);
+extern void idt_setup_ist_traps(void);
 extern void idt_setup_debugidt_traps(void);
 #else
 static inline void idt_setup_early_pf(void) { }
+static inline void idt_setup_ist_traps(void) { }
 static inline void idt_setup_debugidt_traps(void) { }
 #endif
 
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -92,6 +92,20 @@ struct desc_ptr idt_descr __ro_after_ini
 gate_desc debug_idt_table[IDT_ENTRIES] __page_aligned_bss;
 
 /*
+ * The exceptions which use Interrupt stacks. They are setup after
+ * cpu_init() when the TSS has been initialized.
+ */
+static const __initdata struct idt_data ist_idts[] = {
+	ISTG(X86_TRAP_DB,	debug,		DEBUG_STACK),
+	ISTG(X86_TRAP_NMI,	nmi,		NMI_STACK),
+	ISTG(X86_TRAP_BP,	int3,		DEBUG_STACK),
+	ISTG(X86_TRAP_DF,	double_fault,	DOUBLEFAULT_STACK),
+#ifdef CONFIG_X86_MCE
+	ISTG(X86_TRAP_MC,	&machine_check,	MCE_STACK),
+#endif
+};
+
+/*
  * Override for the debug_idt. Same as the default, but with interrupt
  * stack set to DEFAULT_STACK (0). Required for NMI trap handling.
  */
@@ -157,6 +171,14 @@ void __init idt_setup_early_pf(void)
 }
 
 /**
+ * idt_setup_ist_traps - Initialize the idt table with traps using IST
+ */
+void __init idt_setup_ist_traps(void)
+{
+	idt_setup_from_table(idt_table, ist_idts, ARRAY_SIZE(ist_idts));
+}
+
+/**
  * idt_setup_debugidt_traps - Initialize the debug idt table with debug traps
  */
 void __init idt_setup_debugidt_traps(void)
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -979,14 +979,7 @@ void __init trap_init(void)
 	 */
 	cpu_init();
 
-	/*
-	 * X86_TRAP_DB and X86_TRAP_BP have been set
-	 * in early_trap_init(). However, ITS works only after
-	 * cpu_init() loads TSS. See comments in early_trap_init().
-	 */
-	set_intr_gate_ist(X86_TRAP_DB, &debug, DEBUG_STACK);
-	/* int3 can be called from all */
-	set_system_intr_gate_ist(X86_TRAP_BP, &int3, DEBUG_STACK);
+	idt_setup_ist_traps();
 
 	x86_init.irqs.trap_init();
 

[patch 33/41] x86/idt: Move regular trap init to tables

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Move-regular-trap-init-to-tables.patch --]
[-- Type: text/plain, Size: 4433 bytes --]

Initialize the regular traps with a table.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |    1 
 arch/x86/kernel/idt.c       |   51 ++++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/traps.c     |   42 +-----------------------------------
 3 files changed, 54 insertions(+), 40 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -513,6 +513,7 @@ static inline void load_current_idt(void
 
 extern void idt_setup_early_handler(void);
 extern void idt_setup_early_traps(void);
+extern void idt_setup_traps(void);
 
 #ifdef CONFIG_X86_64
 extern void idt_setup_early_pf(void);
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -60,6 +60,49 @@ static const __initdata struct idt_data
 #endif
 };
 
+/*
+ * The default IDT entries which are set up in trap_init() before
+ * cpu_init() is invoked. Interrupt stacks cannot be used at that point and
+ * the traps which use them are reinitialized with IST after cpu_init() has
+ * set up TSS.
+ */
+static const __initdata struct idt_data def_idts[] = {
+	INTG(X86_TRAP_DE,		divide_error),
+	INTG(X86_TRAP_NMI,		nmi),
+	INTG(X86_TRAP_BR,		bounds),
+	INTG(X86_TRAP_UD,		invalid_op),
+	INTG(X86_TRAP_NM,		device_not_available),
+	INTG(X86_TRAP_OLD_MF,		coprocessor_segment_overrun),
+	INTG(X86_TRAP_TS,		invalid_TSS),
+	INTG(X86_TRAP_NP,		segment_not_present),
+	INTG(X86_TRAP_SS,		stack_segment),
+	INTG(X86_TRAP_GP,		general_protection),
+	INTG(X86_TRAP_SPURIOUS,		spurious_interrupt_bug),
+	INTG(X86_TRAP_MF,		coprocessor_error),
+	INTG(X86_TRAP_AC,		alignment_check),
+	INTG(X86_TRAP_XF,		simd_coprocessor_error),
+
+#ifdef CONFIG_X86_32
+	TSKG(X86_TRAP_DF,		GDT_ENTRY_DOUBLEFAULT_TSS),
+#else
+	INTG(X86_TRAP_DF,		double_fault),
+#endif
+	INTG(X86_TRAP_DB,		debug),
+	INTG(X86_TRAP_NMI,		nmi),
+	INTG(X86_TRAP_BP,		int3),
+
+#ifdef CONFIG_X86_MCE
+	INTG(X86_TRAP_MC,		&machine_check),
+#endif
+
+	SYSG(X86_TRAP_OF,		overflow),
+#if defined(CONFIG_IA32_EMULATION)
+	SYSG(IA32_SYSCALL_VECTOR,	entry_INT80_compat),
+#elif defined(CONFIG_X86_32)
+	SYSG(IA32_SYSCALL_VECTOR,	entry_INT80_32),
+#endif
+};
+
 #ifdef CONFIG_X86_64
 /*
  * Early traps running on the DEFAULT_STACK because the other interrupt
@@ -153,6 +196,14 @@ void __init idt_setup_early_traps(void)
 	idt_setup_from_table(idt_table, early_idts, ARRAY_SIZE(early_idts));
 }
 
+/**
+ * idt_setup_traps - Initialize the idt table with default traps
+ */
+void __init idt_setup_traps(void)
+{
+	idt_setup_from_table(idt_table, def_idts, ARRAY_SIZE(def_idts));
+}
+
 #ifdef CONFIG_X86_64
 /**
  * idt_setup_early_pf - Initialize the idt table with early pagefault handler
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -925,46 +925,8 @@ dotraplinkage void do_iret_error(struct
 
 void __init trap_init(void)
 {
-	int i;
-
-	set_intr_gate(X86_TRAP_DE, divide_error);
-	set_intr_gate_ist(X86_TRAP_NMI, &nmi, NMI_STACK);
-	/* int4 can be called from all */
-	set_system_intr_gate(X86_TRAP_OF, &overflow);
-	set_intr_gate(X86_TRAP_BR, bounds);
-	set_intr_gate(X86_TRAP_UD, invalid_op);
-	set_intr_gate(X86_TRAP_NM, device_not_available);
-#ifdef CONFIG_X86_32
-	set_task_gate(X86_TRAP_DF, GDT_ENTRY_DOUBLEFAULT_TSS);
-#else
-	set_intr_gate_ist(X86_TRAP_DF, &double_fault, DOUBLEFAULT_STACK);
-#endif
-	set_intr_gate(X86_TRAP_OLD_MF, coprocessor_segment_overrun);
-	set_intr_gate(X86_TRAP_TS, invalid_TSS);
-	set_intr_gate(X86_TRAP_NP, segment_not_present);
-	set_intr_gate(X86_TRAP_SS, stack_segment);
-	set_intr_gate(X86_TRAP_GP, general_protection);
-	set_intr_gate(X86_TRAP_SPURIOUS, spurious_interrupt_bug);
-	set_intr_gate(X86_TRAP_MF, coprocessor_error);
-	set_intr_gate(X86_TRAP_AC, alignment_check);
-#ifdef CONFIG_X86_MCE
-	set_intr_gate_ist(X86_TRAP_MC, &machine_check, MCE_STACK);
-#endif
-	set_intr_gate(X86_TRAP_XF, simd_coprocessor_error);
-
-	/* Reserve all the builtin and the syscall vector: */
-	for (i = 0; i < FIRST_EXTERNAL_VECTOR; i++)
-		set_bit(i, used_vectors);
-
-#ifdef CONFIG_IA32_EMULATION
-	set_system_intr_gate(IA32_SYSCALL_VECTOR, entry_INT80_compat);
-	set_bit(IA32_SYSCALL_VECTOR, used_vectors);
-#endif
-
-#ifdef CONFIG_X86_32
-	set_system_intr_gate(IA32_SYSCALL_VECTOR, entry_INT80_32);
-	set_bit(IA32_SYSCALL_VECTOR, used_vectors);
-#endif
+	/* Initial trap setup. No IST stacks used. */
+	idt_setup_traps();
 
 	/*
 	 * Set the IDT descriptor to a fixed read-only location, so that the

[patch 34/41] x86/idt: Move APIC gate initialization to tables

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Move-apic-gate-initialization-to-tables.patch --]
[-- Type: text/plain, Size: 5160 bytes --]

Replace the APIC/SMP vector gate initialization with the table based
mechanism.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |    1 
 arch/x86/kernel/idt.c       |   48 ++++++++++++++++++++++++++++++
 arch/x86/kernel/irqinit.c   |   69 --------------------------------------------
 3 files changed, 50 insertions(+), 68 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -514,6 +514,7 @@ static inline void load_current_idt(void
 extern void idt_setup_early_handler(void);
 extern void idt_setup_early_traps(void);
 extern void idt_setup_traps(void);
+extern void idt_setup_apic_and_irq_gates(void);
 
 #ifdef CONFIG_X86_64
 extern void idt_setup_early_pf(void);
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -103,6 +103,46 @@ static const __initdata struct idt_data
 #endif
 };
 
+/*
+ * The APIC and SMP idt entries
+ */
+static const __initdata struct idt_data apic_idts[] = {
+#ifdef CONFIG_SMP
+	INTG(RESCHEDULE_VECTOR,		reschedule_interrupt),
+	INTG(CALL_FUNCTION_VECTOR,	call_function_interrupt),
+	INTG(CALL_FUNCTION_SINGLE_VECTOR, call_function_single_interrupt),
+	INTG(IRQ_MOVE_CLEANUP_VECTOR,	irq_move_cleanup_interrupt),
+	INTG(REBOOT_VECTOR,		reboot_interrupt),
+#endif
+
+#ifdef CONFIG_X86_THERMAL_VECTOR
+	INTG(THERMAL_APIC_VECTOR,	thermal_interrupt),
+#endif
+
+#ifdef CONFIG_X86_MCE_THRESHOLD
+	INTG(THRESHOLD_APIC_VECTOR,	threshold_interrupt),
+#endif
+
+#ifdef CONFIG_X86_MCE_AMD
+	INTG(DEFERRED_ERROR_VECTOR,	deferred_error_interrupt),
+#endif
+
+#ifdef CONFIG_X86_LOCAL_APIC
+	INTG(LOCAL_TIMER_VECTOR,	apic_timer_interrupt),
+	INTG(X86_PLATFORM_IPI_VECTOR,	x86_platform_ipi),
+# ifdef CONFIG_HAVE_KVM
+	INTG(POSTED_INTR_VECTOR,	kvm_posted_intr_ipi),
+	INTG(POSTED_INTR_WAKEUP_VECTOR, kvm_posted_intr_wakeup_ipi),
+	INTG(POSTED_INTR_NESTED_VECTOR, kvm_posted_intr_nested_ipi),
+# endif
+# ifdef CONFIG_IRQ_WORK
+	INTG(IRQ_WORK_VECTOR,		irq_work_interrupt),
+# endif
+	INTG(SPURIOUS_APIC_VECTOR,	spurious_interrupt),
+	INTG(ERROR_APIC_VECTOR,		error_interrupt),
+#endif
+};
+
 #ifdef CONFIG_X86_64
 /*
  * Early traps running on the DEFAULT_STACK because the other interrupt
@@ -241,6 +281,14 @@ void __init idt_setup_debugidt_traps(voi
 #endif
 
 /**
+ * idt_setup_apic_and_irq_gates - Setup APIC/SMP and normal interrupt gates
+ */
+void __init idt_setup_apic_and_irq_gates(void)
+{
+	idt_setup_from_table(idt_table, apic_idts, ARRAY_SIZE(apic_idts));
+}
+
+/**
  * idt_setup_early_handler - Initializes the idt table with early handlers
  */
 void __init idt_setup_early_handler(void)
--- a/arch/x86/kernel/irqinit.c
+++ b/arch/x86/kernel/irqinit.c
@@ -87,73 +87,6 @@ void __init init_IRQ(void)
 	x86_init.irqs.intr_init();
 }
 
-static void __init smp_intr_init(void)
-{
-#ifdef CONFIG_SMP
-	/*
-	 * The reschedule interrupt is a CPU-to-CPU reschedule-helper
-	 * IPI, driven by wakeup.
-	 */
-	alloc_intr_gate(RESCHEDULE_VECTOR, reschedule_interrupt);
-
-	/* IPI for generic function call */
-	alloc_intr_gate(CALL_FUNCTION_VECTOR, call_function_interrupt);
-
-	/* IPI for generic single function call */
-	alloc_intr_gate(CALL_FUNCTION_SINGLE_VECTOR,
-			call_function_single_interrupt);
-
-	/* Low priority IPI to cleanup after moving an irq */
-	set_intr_gate(IRQ_MOVE_CLEANUP_VECTOR, irq_move_cleanup_interrupt);
-	set_bit(IRQ_MOVE_CLEANUP_VECTOR, used_vectors);
-
-	/* IPI used for rebooting/stopping */
-	alloc_intr_gate(REBOOT_VECTOR, reboot_interrupt);
-#endif /* CONFIG_SMP */
-}
-
-static void __init apic_intr_init(void)
-{
-	smp_intr_init();
-
-#ifdef CONFIG_X86_THERMAL_VECTOR
-	alloc_intr_gate(THERMAL_APIC_VECTOR, thermal_interrupt);
-#endif
-#ifdef CONFIG_X86_MCE_THRESHOLD
-	alloc_intr_gate(THRESHOLD_APIC_VECTOR, threshold_interrupt);
-#endif
-
-#ifdef CONFIG_X86_MCE_AMD
-	alloc_intr_gate(DEFERRED_ERROR_VECTOR, deferred_error_interrupt);
-#endif
-
-#ifdef CONFIG_X86_LOCAL_APIC
-	/* self generated IPI for local APIC timer */
-	alloc_intr_gate(LOCAL_TIMER_VECTOR, apic_timer_interrupt);
-
-	/* IPI for X86 platform specific use */
-	alloc_intr_gate(X86_PLATFORM_IPI_VECTOR, x86_platform_ipi);
-#ifdef CONFIG_HAVE_KVM
-	/* IPI for KVM to deliver posted interrupt */
-	alloc_intr_gate(POSTED_INTR_VECTOR, kvm_posted_intr_ipi);
-	/* IPI for KVM to deliver interrupt to wake up tasks */
-	alloc_intr_gate(POSTED_INTR_WAKEUP_VECTOR, kvm_posted_intr_wakeup_ipi);
-	/* IPI for KVM to deliver nested posted interrupt */
-	alloc_intr_gate(POSTED_INTR_NESTED_VECTOR, kvm_posted_intr_nested_ipi);
-#endif
-
-	/* IPI vectors for APIC spurious and error interrupts */
-	alloc_intr_gate(SPURIOUS_APIC_VECTOR, spurious_interrupt);
-	alloc_intr_gate(ERROR_APIC_VECTOR, error_interrupt);
-
-	/* IRQ work interrupts: */
-# ifdef CONFIG_IRQ_WORK
-	alloc_intr_gate(IRQ_WORK_VECTOR, irq_work_interrupt);
-# endif
-
-#endif
-}
-
 void __init native_init_IRQ(void)
 {
 	int i;
@@ -161,7 +94,7 @@ void __init native_init_IRQ(void)
 	/* Execute any quirks before the call gates are initialised: */
 	x86_init.irqs.pre_vector_init();
 
-	apic_intr_init();
+	idt_setup_apic_and_irq_gates();
 
 	/*
 	 * Cover the whole vector space, no vector can escape

[patch 35/41] x86/idt: Move interrupt gate initialization to IDT code

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Move-interrupt-gate-initialization-to-idt-code.patch --]
[-- Type: text/plain, Size: 1982 bytes --]

Move the gate intialization from interrupt init to the IDT code so all IDT
related operations are at a single place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/idt.c     |   18 ++++++++++++++++++
 arch/x86/kernel/irqinit.c |   18 ------------------
 2 files changed, 18 insertions(+), 18 deletions(-)

--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -285,7 +285,25 @@ void __init idt_setup_debugidt_traps(voi
  */
 void __init idt_setup_apic_and_irq_gates(void)
 {
+	int i = FIRST_EXTERNAL_VECTOR;
+	void *entry;
+
 	idt_setup_from_table(idt_table, apic_idts, ARRAY_SIZE(apic_idts));
+
+	for_each_clear_bit_from(i, used_vectors, first_system_vector) {
+		entry = irq_entries_start + 8 * (i - FIRST_EXTERNAL_VECTOR);
+		set_intr_gate(i, entry);
+	}
+
+	for_each_clear_bit_from(i, used_vectors, NR_VECTORS) {
+#ifdef CONFIG_X86_LOCAL_APIC
+		set_bit(i, used_vectors);
+		set_intr_gate(i, spurious_interrupt);
+#else
+		entry = irq_entries_start + 8 * (i - FIRST_EXTERNAL_VECTOR);
+		set_intr_gate(i, entry);
+#endif
+	}
 }
 
 /**
--- a/arch/x86/kernel/irqinit.c
+++ b/arch/x86/kernel/irqinit.c
@@ -89,29 +89,11 @@ void __init init_IRQ(void)
 
 void __init native_init_IRQ(void)
 {
-	int i;
-
 	/* Execute any quirks before the call gates are initialised: */
 	x86_init.irqs.pre_vector_init();
 
 	idt_setup_apic_and_irq_gates();
 
-	/*
-	 * Cover the whole vector space, no vector can escape
-	 * us. (some of these will be overridden and become
-	 * 'special' SMP interrupts)
-	 */
-	i = FIRST_EXTERNAL_VECTOR;
-	for_each_clear_bit_from(i, used_vectors, first_system_vector) {
-		/* IA32_SYSCALL_VECTOR could be used in trap_init already. */
-		set_intr_gate(i, irq_entries_start +
-				8 * (i - FIRST_EXTERNAL_VECTOR));
-	}
-#ifdef CONFIG_X86_LOCAL_APIC
-	for_each_clear_bit_from(i, used_vectors, NR_VECTORS)
-		set_intr_gate(i, spurious_interrupt);
-#endif
-
 	if (!acpi_ioapic && !of_ioapic && nr_legacy_irqs())
 		setup_irq(2, &irq2);
 

[patch 36/41] x86/idt: Remove unused functions/inlines

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Remove-unused-functions-inlines.patch --]
[-- Type: text/plain, Size: 1688 bytes --]

The IDT related inlines are not longer used. Remove them.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |   36 ------------------------------------
 1 file changed, 36 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -389,16 +389,6 @@ static inline void set_desc_limit(struct
 	desc->limit1 = (limit >> 16) & 0xf;
 }
 
-#ifdef CONFIG_X86_64
-static inline void set_nmi_gate(int gate, void *addr)
-{
-	gate_desc s;
-
-	pack_gate(&s, GATE_INTERRUPT, (unsigned long)addr, 0, 0, __KERNEL_CS);
-	write_idt_entry(debug_idt_table, gate, &s);
-}
-#endif
-
 static inline void _set_gate(int gate, unsigned type, const void *addr,
 			     unsigned dpl, unsigned ist, unsigned seg)
 {
@@ -444,32 +434,6 @@ static inline void alloc_system_vector(i
 		set_intr_gate(n, addr);				\
 	} while (0)
 
-/*
- * This routine sets up an interrupt gate at directory privilege level 3.
- */
-static inline void set_system_intr_gate(unsigned int n, void *addr)
-{
-	BUG_ON((unsigned)n > 0xFF);
-	_set_gate(n, GATE_INTERRUPT, addr, 0x3, 0, __KERNEL_CS);
-}
-
-static inline void set_task_gate(unsigned int n, unsigned int gdt_entry)
-{
-	BUG_ON((unsigned)n > 0xFF);
-	_set_gate(n, GATE_TASK, (void *)0, 0, 0, (gdt_entry<<3));
-}
-
-static inline void set_intr_gate_ist(int n, void *addr, unsigned ist)
-{
-	BUG_ON((unsigned)n > 0xFF);
-	_set_gate(n, GATE_INTERRUPT, addr, 0, ist, __KERNEL_CS);
-}
-
-static inline void set_system_intr_gate_ist(int n, void *addr, unsigned ist)
-{
-	BUG_ON((unsigned)n > 0xFF);
-	_set_gate(n, GATE_INTERRUPT, addr, 0x3, ist, __KERNEL_CS);
-}
 
 #ifdef CONFIG_X86_64
 DECLARE_PER_CPU(u32, debug_idt_ctr);

[patch 37/41] x86/irq: Get rid of the first_system_vector bogisity

[Thomas Gleixner]

[-- Attachment #1: x86-irq--Get-rid-of-the-first_system_vector-bogisity.patch --]
[-- Type: text/plain, Size: 2875 bytes --]

This variable is beyond pointless. Nothing allocates a vector via
alloc_gate() below FIRST_SYSTEM_VECTOR. So nothing can change
first_system_vector.

If there is a need for a gate below FIRST_SYSTEM_VECTOR then it can be
added to the vector defines and FIRST_SYSTEM_VECTOR adjusted accordingly.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h   |   11 ++---------
 arch/x86/kernel/apic/apic.c   |    2 --
 arch/x86/kernel/apic/vector.c |    2 +-
 arch/x86/kernel/idt.c         |    2 +-
 arch/x86/kernel/irq.c         |    2 +-
 5 files changed, 5 insertions(+), 14 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -408,21 +408,14 @@ static inline void set_intr_gate(unsigne
 	_set_gate(n, GATE_INTERRUPT, addr, 0, 0, __KERNEL_CS);
 }
 
-#ifdef CONFIG_X86_LOCAL_APIC
-extern int first_system_vector;
-#else
-#define first_system_vector		NR_VECTORS
-#endif
-
 /* used_vectors is BITMAP for irq is not managed by percpu vector_irq */
 extern unsigned long used_vectors[];
 
-static inline void alloc_system_vector(int vector)
+static inline void alloc_system_vector(unsigned int vector)
 {
+	BUG_ON(vector < FIRST_SYSTEM_VECTOR);
 	if (!test_bit(vector, used_vectors)) {
 		set_bit(vector, used_vectors);
-		if (first_system_vector > vector)
-			first_system_vector = vector;
 	} else {
 		BUG();
 	}
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -177,8 +177,6 @@ static int disable_apic_timer __initdata
 int local_apic_timer_c2_ok;
 EXPORT_SYMBOL_GPL(local_apic_timer_c2_ok);
 
-int first_system_vector = FIRST_SYSTEM_VECTOR;
-
 /*
  * Debug level, exported for io_apic.c
  */
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -166,7 +166,7 @@ static int __assign_irq_vector(int irq,
 		offset = current_offset;
 next:
 		vector += 16;
-		if (vector >= first_system_vector) {
+		if (vector >= FIRST_SYSTEM_VECTOR) {
 			offset = (offset + 1) % 16;
 			vector = FIRST_EXTERNAL_VECTOR + offset;
 		}
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -290,7 +290,7 @@ void __init idt_setup_apic_and_irq_gates
 
 	idt_setup_from_table(idt_table, apic_idts, ARRAY_SIZE(apic_idts));
 
-	for_each_clear_bit_from(i, used_vectors, first_system_vector) {
+	for_each_clear_bit_from(i, used_vectors, FIRST_SYSTEM_VECTOR) {
 		entry = irq_entries_start + 8 * (i - FIRST_EXTERNAL_VECTOR);
 		set_intr_gate(i, entry);
 	}
--- a/arch/x86/kernel/irq.c
+++ b/arch/x86/kernel/irq.c
@@ -414,7 +414,7 @@ int check_irq_vectors_for_cpu_disable(vo
 		 * this w/o holding vector_lock.
 		 */
 		for (vector = FIRST_EXTERNAL_VECTOR;
-		     vector < first_system_vector; vector++) {
+		     vector < FIRST_SYSTEM_VECTOR; vector++) {
 			if (!test_bit(vector, used_vectors) &&
 			    IS_ERR_OR_NULL(per_cpu(vector_irq, cpu)[vector])) {
 				if (++count == this_count)

[patch 38/41] x86/idt: Deinline setup functions

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Deinline-setup-functions.patch --]
[-- Type: text/plain, Size: 4445 bytes --]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/desc.h |   37 ++-----------------------------------
 arch/x86/kernel/idt.c       |   42 +++++++++++++++++++++++++++++++++---------
 2 files changed, 35 insertions(+), 44 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -389,45 +389,12 @@ static inline void set_desc_limit(struct
 	desc->limit1 = (limit >> 16) & 0xf;
 }
 
-static inline void _set_gate(int gate, unsigned type, const void *addr,
-			     unsigned dpl, unsigned ist, unsigned seg)
-{
-	gate_desc s;
-
-	pack_gate(&s, type, (unsigned long)addr, dpl, ist, seg);
-	/*
-	 * does not need to be atomic because it is only done once at
-	 * setup time
-	 */
-	write_idt_entry(idt_table, gate, &s);
-}
-
-static inline void set_intr_gate(unsigned int n, const void *addr)
-{
-	BUG_ON((unsigned)n > 0xFF);
-	_set_gate(n, GATE_INTERRUPT, addr, 0, 0, __KERNEL_CS);
-}
+void set_intr_gate(unsigned int n, const void *addr);
+void alloc_intr_gate(unsigned int n, const void *addr);
 
 /* used_vectors is BITMAP for irq is not managed by percpu vector_irq */
 extern unsigned long used_vectors[];
 
-static inline void alloc_system_vector(unsigned int vector)
-{
-	BUG_ON(vector < FIRST_SYSTEM_VECTOR);
-	if (!test_bit(vector, used_vectors)) {
-		set_bit(vector, used_vectors);
-	} else {
-		BUG();
-	}
-}
-
-#define alloc_intr_gate(n, addr)				\
-	do {							\
-		alloc_system_vector(n);				\
-		set_intr_gate(n, addr);				\
-	} while (0)
-
-
 #ifdef CONFIG_X86_64
 DECLARE_PER_CPU(u32, debug_idt_ctr);
 static inline bool is_debug_idt_enabled(void)
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -212,15 +212,16 @@ static inline void idt_init_desc(gate_de
 #endif
 }
 
-static __init void
-idt_setup_from_table(gate_desc *idt, const struct idt_data *t, int size)
+static void
+idt_setup_from_table(gate_desc *idt, const struct idt_data *t, int size, bool sys)
 {
 	gate_desc desc;
 
 	for (; size > 0; t++, size--) {
 		idt_init_desc(&desc, t);
-		set_bit(t->vector, used_vectors);
 		write_idt_entry(idt, t->vector, &desc);
+		if (sys)
+			set_bit(t->vector, used_vectors);
 	}
 }
 
@@ -233,7 +234,7 @@ idt_setup_from_table(gate_desc *idt, con
  */
 void __init idt_setup_early_traps(void)
 {
-	idt_setup_from_table(idt_table, early_idts, ARRAY_SIZE(early_idts));
+	idt_setup_from_table(idt_table, early_idts, ARRAY_SIZE(early_idts), true);
 }
 
 /**
@@ -241,7 +242,7 @@ void __init idt_setup_early_traps(void)
  */
 void __init idt_setup_traps(void)
 {
-	idt_setup_from_table(idt_table, def_idts, ARRAY_SIZE(def_idts));
+	idt_setup_from_table(idt_table, def_idts, ARRAY_SIZE(def_idts), true);
 }
 
 #ifdef CONFIG_X86_64
@@ -258,7 +259,7 @@ void __init idt_setup_traps(void)
 void __init idt_setup_early_pf(void)
 {
 	idt_setup_from_table(idt_table, early_pf_idts,
-			     ARRAY_SIZE(early_pf_idts));
+			     ARRAY_SIZE(early_pf_idts), true);
 }
 
 /**
@@ -266,7 +267,7 @@ void __init idt_setup_early_pf(void)
  */
 void __init idt_setup_ist_traps(void)
 {
-	idt_setup_from_table(idt_table, ist_idts, ARRAY_SIZE(ist_idts));
+	idt_setup_from_table(idt_table, ist_idts, ARRAY_SIZE(ist_idts), true);
 }
 
 /**
@@ -276,7 +277,7 @@ void __init idt_setup_debugidt_traps(voi
 {
 	memcpy(&debug_idt_table, &idt_table, IDT_ENTRIES * 16);
 
-	idt_setup_from_table(debug_idt_table, dbg_idts, ARRAY_SIZE(dbg_idts));
+	idt_setup_from_table(debug_idt_table, dbg_idts, ARRAY_SIZE(dbg_idts), false);
 }
 #endif
 
@@ -288,7 +289,7 @@ void __init idt_setup_apic_and_irq_gates
 	int i = FIRST_EXTERNAL_VECTOR;
 	void *entry;
 
-	idt_setup_from_table(idt_table, apic_idts, ARRAY_SIZE(apic_idts));
+	idt_setup_from_table(idt_table, apic_idts, ARRAY_SIZE(apic_idts), true);
 
 	for_each_clear_bit_from(i, used_vectors, FIRST_SYSTEM_VECTOR) {
 		entry = irq_entries_start + 8 * (i - FIRST_EXTERNAL_VECTOR);
@@ -332,3 +333,26 @@ void idt_invalidate(void *addr)
 
 	load_idt(&idt);
 }
+
+void set_intr_gate(unsigned int n, const void *addr)
+{
+	struct idt_data data;
+
+	BUG_ON(n > 0xFF);
+
+	memset(&data, 0, sizeof(data));
+	data.vector	= n;
+	data.addr	= addr;
+	data.segment	= __KERNEL_CS;
+	data.bits.type	= GATE_INTERRUPT;
+	data.bits.p	= 1;
+
+	idt_setup_from_table(idt_table, &data, 1, false);
+}
+
+void alloc_intr_gate(unsigned int n, const void *addr)
+{
+	BUG_ON(test_bit(n, used_vectors) || n < FIRST_SYSTEM_VECTOR);
+	set_bit(n, used_vectors);
+	set_intr_gate(n, addr);
+}

[patch 39/41] x86/idt: Simplify alloc_intr_gate

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Simplify-alloc_intr_gate.patch --]
[-- Type: text/plain, Size: 2157 bytes --]

The only users of alloc_intr_gate() are hypervisors, which both check the
used_vectors bitmap whether they have allocated the gate already. Move that
check into alloc_intr_gate() and simplify the users.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Juergen Gross <jgross@suse.com>
---
 arch/x86/kernel/cpu/mshyperv.c   |    9 ++-------
 arch/x86/kernel/idt.c            |    6 +++---
 drivers/xen/events/events_base.c |    6 ++----
 3 files changed, 7 insertions(+), 14 deletions(-)

--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -59,13 +59,8 @@ void hyperv_vector_handler(struct pt_reg
 void hv_setup_vmbus_irq(void (*handler)(void))
 {
 	vmbus_handler = handler;
-	/*
-	 * Setup the IDT for hypervisor callback. Prevent reallocation
-	 * at module reload.
-	 */
-	if (!test_bit(HYPERVISOR_CALLBACK_VECTOR, used_vectors))
-		alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR,
-				hyperv_callback_vector);
+	/* Setup the IDT for hypervisor callback */
+	alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, hyperv_callback_vector);
 }
 
 void hv_remove_vmbus_irq(void)
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -352,7 +352,7 @@ void set_intr_gate(unsigned int n, const
 
 void alloc_intr_gate(unsigned int n, const void *addr)
 {
-	BUG_ON(test_bit(n, used_vectors) || n < FIRST_SYSTEM_VECTOR);
-	set_bit(n, used_vectors);
-	set_intr_gate(n, addr);
+	BUG_ON(n < FIRST_SYSTEM_VECTOR);
+	if (!test_and_set_bit(n, used_vectors))
+		set_intr_gate(n, addr);
 }
--- a/drivers/xen/events/events_base.c
+++ b/drivers/xen/events/events_base.c
@@ -1653,10 +1653,8 @@ void xen_callback_vector(void)
 			return;
 		}
 		pr_info("Xen HVM callback vector for event delivery is enabled\n");
-		/* in the restore case the vector has already been allocated */
-		if (!test_bit(HYPERVISOR_CALLBACK_VECTOR, used_vectors))
-			alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR,
-					xen_hvm_callback_vector);
+		alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR,
+				xen_hvm_callback_vector);
 	}
 }
 #else

[patch 40/41] x86/idt: Hide set_intr_gate()

[Thomas Gleixner]

[-- Attachment #1: x86-idt--Hide-set_intr_gate.patch --]
[-- Type: text/plain, Size: 2455 bytes --]

set_intr_gate() is an internal function of the IDT code. The only user left
is the KVM code which replaces the pagefault handler eventually.

Provide an explicit update_intr_gate() function and make set_intr_gate()
static. While at it replace the magic number 14 in the KVM code with the
proper trap define.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/include/asm/desc.h |    2 +-
 arch/x86/kernel/idt.c       |   33 ++++++++++++++++++++-------------
 arch/x86/kernel/kvm.c       |    2 +-
 3 files changed, 22 insertions(+), 15 deletions(-)

--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -389,7 +389,7 @@ static inline void set_desc_limit(struct
 	desc->limit1 = (limit >> 16) & 0xf;
 }
 
-void set_intr_gate(unsigned int n, const void *addr);
+void update_intr_gate(unsigned int n, const void *addr);
 void alloc_intr_gate(unsigned int n, const void *addr);
 
 /* used_vectors is BITMAP for irq is not managed by percpu vector_irq */
--- a/arch/x86/kernel/idt.c
+++ b/arch/x86/kernel/idt.c
@@ -225,6 +225,22 @@ idt_setup_from_table(gate_desc *idt, con
 	}
 }
 
+static void set_intr_gate(unsigned int n, const void *addr)
+{
+	struct idt_data data;
+
+	BUG_ON(n > 0xFF);
+
+	memset(&data, 0, sizeof(data));
+	data.vector	= n;
+	data.addr	= addr;
+	data.segment	= __KERNEL_CS;
+	data.bits.type	= GATE_INTERRUPT;
+	data.bits.p	= 1;
+
+	idt_setup_from_table(idt_table, &data, 1, false);
+}
+
 /**
  * idt_setup_early_traps - Initialize the idt table with early traps
  *
@@ -334,20 +350,11 @@ void idt_invalidate(void *addr)
 	load_idt(&idt);
 }
 
-void set_intr_gate(unsigned int n, const void *addr)
+void __init update_intr_gate(unsigned int n, const void *addr)
 {
-	struct idt_data data;
-
-	BUG_ON(n > 0xFF);
-
-	memset(&data, 0, sizeof(data));
-	data.vector	= n;
-	data.addr	= addr;
-	data.segment	= __KERNEL_CS;
-	data.bits.type	= GATE_INTERRUPT;
-	data.bits.p	= 1;
-
-	idt_setup_from_table(idt_table, &data, 1, false);
+	if (WARN_ON_ONCE(test_bit(n, used_vectors)))
+		return;
+	set_intr_gate(n, addr);
 }
 
 void alloc_intr_gate(unsigned int n, const void *addr)
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -455,7 +455,7 @@ static int kvm_cpu_down_prepare(unsigned
 
 static void __init kvm_apf_trap_init(void)
 {
-	set_intr_gate(14, async_page_fault);
+	update_intr_gate(X86_TRAP_PF, async_page_fault);
 }
 
 void __init kvm_guest_init(void)

[patch 41/41] x86/irq: Remove duplicated used_vectors definition

[Thomas Gleixner]

[-- Attachment #1: x86-irq--Remove-duplicated-used_vectors-definition.patch --]
[-- Type: text/plain, Size: 471 bytes --]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/include/asm/irq.h |    3 ---
 1 file changed, 3 deletions(-)

--- a/arch/x86/include/asm/irq.h
+++ b/arch/x86/include/asm/irq.h
@@ -42,9 +42,6 @@ extern bool handle_irq(struct irq_desc *
 
 extern __visible unsigned int do_IRQ(struct pt_regs *regs);
 
-/* Interrupt vector management */
-extern DECLARE_BITMAP(used_vectors, NR_VECTORS);
-
 extern void init_ISA_irqs(void);
 
 #ifdef CONFIG_X86_LOCAL_APIC

Re: [patch 40/41] x86/idt: Hide set_intr_gate()

[Paolo Bonzini]

On 25/08/2017 12:31, Thomas Gleixner wrote:
> set_intr_gate() is an internal function of the IDT code. The only user left
> is the KVM code which replaces the pagefault handler eventually.
> 
> Provide an explicit update_intr_gate() function and make set_intr_gate()
> static. While at it replace the magic number 14 in the KVM code with the
> proper trap define.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Paolo Bonzini <pbonzini@redhat.com>


Acked-by: Paolo Bonzini <pbonzini@redhat.com>

Thanks,

Paolo

Re: [patch 20/41] x86: Replace access to desc_struct:a/b fields

[Juergen Gross]

On 25/08/17 12:31, Thomas Gleixner wrote:
> The union inside of desc_struct which allows access to the raw u32 parts of
> the descriptors. This raw access part is about to go away.
> 
> Replace the few code parts which access those fields.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: Juergen Gross <jgross@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

Re: [patch 39/41] x86/idt: Simplify alloc_intr_gate

[Juergen Gross]

On 25/08/17 12:31, Thomas Gleixner wrote:
> The only users of alloc_intr_gate() are hypervisors, which both check the
> used_vectors bitmap whether they have allocated the gate already. Move that
> check into alloc_intr_gate() and simplify the users.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: "K. Y. Srinivasan" <kys@microsoft.com>
> Cc: Stephen Hemminger <sthemmin@microsoft.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: Juergen Gross <jgross@suse.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

Re: [patch 04/41] x86/boot: Move EISA setup to a proper place

[Thomas Gleixner]

On Fri, 25 Aug 2017, Thomas Gleixner wrote:
> +static __init int eisa_bus_probe(void)
> +{
> +	void __iomem *p = ioremap(0x0FFFD9, 4);
> +
> +	if (readl(p) == 'E' + ('I'<<8) + ('S'<<16) + ('A'<<24))
> +		EISA_bus = 1;
> +	iounmap(p, 4);

That should obviously be:

     iounmap(p);

But as a side note:

Can't we finally get rid of this EISA cruft alltogether?

Thanks,

	tglx

Re: [patch 03/41] x86/irq: Move ifdeffery to header file

[Thomas Gleixner]

On Fri, 25 Aug 2017, Thomas Gleixner wrote:

> Having this in the middle of code is beyond ugly.

Bah, I know why this was in the middle of the code.

I reorder the series so the whole first_system_vector nonsense gets removed
early.

Thanks,

	tglx

Re: [patch 20/41] x86: Replace access to desc_struct:a/b fields

[Boris Ostrovsky]

>  
> Index: b/arch/x86/xen/enlighten_pv.c
> ===================================================================
> --- a/arch/x86/xen/enlighten_pv.c
> +++ b/arch/x86/xen/enlighten_pv.c
> @@ -494,7 +494,7 @@ static void __init xen_load_gdt_boot(con
>  static inline bool desc_equal(const struct desc_struct *d1,
>  			      const struct desc_struct *d2)
>  {
> -	return d1->a == d2->a && d1->b == d2->b;
> +	return memcmp(d1, d2, sizeof(*d1));
>  }

Shouldn't this be !memcmp() ?

-boris

Re: [patch 20/41] x86: Replace access to desc_struct:a/b fields

[Thomas Gleixner]

On Fri, 25 Aug 2017, Boris Ostrovsky wrote:

> 
> >  
> > Index: b/arch/x86/xen/enlighten_pv.c
> > ===================================================================
> > --- a/arch/x86/xen/enlighten_pv.c
> > +++ b/arch/x86/xen/enlighten_pv.c
> > @@ -494,7 +494,7 @@ static void __init xen_load_gdt_boot(con
> >  static inline bool desc_equal(const struct desc_struct *d1,
> >  			      const struct desc_struct *d2)
> >  {
> > -	return d1->a == d2->a && d1->b == d2->b;
> > +	return memcmp(d1, d2, sizeof(*d1));
> >  }
> 
> Shouldn't this be !memcmp() ?

Bah yes.

Re: [patch 20/41] x86: Replace access to desc_struct:a/b fields

[Steven Rostedt]

On Fri, 25 Aug 2017 16:02:02 +0200 (CEST)
Thomas Gleixner <tglx@linutronix.de> wrote:

> On Fri, 25 Aug 2017, Boris Ostrovsky wrote:
> 
> >   
> > >  
> > > Index: b/arch/x86/xen/enlighten_pv.c
> > > ===================================================================
> > > --- a/arch/x86/xen/enlighten_pv.c
> > > +++ b/arch/x86/xen/enlighten_pv.c
> > > @@ -494,7 +494,7 @@ static void __init xen_load_gdt_boot(con
> > >  static inline bool desc_equal(const struct desc_struct *d1,
> > >  			      const struct desc_struct *d2)
> > >  {
> > > -	return d1->a == d2->a && d1->b == d2->b;
> > > +	return memcmp(d1, d2, sizeof(*d1));
> > >  }  
> > 
> > Shouldn't this be !memcmp() ?  
> 
> Bah yes.

This is why I'm one of those that like to add the "== 0" to it. Because
I always get this wrong :-p

-- Steve

Re: [patch 30/41] x86/idt: Switch early trap init to IDT tables

[Andy Lutomirski]

On Fri, Aug 25, 2017 at 3:31 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> Add the initialization table for the early trap setup and replace the early
> trap init code.

Nice.

Re: [patch 06/41] x86/traps: Simplify pagefault tracing logic

[Andy Lutomirski]

On Fri, Aug 25, 2017 at 3:31 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> Make use of the new irqvector tracing static key and remove the duplicated
> trace_do_pagefault() implementation.
>
> If irq vector tracing is disabled, then the overhead of this is a single
> NOP5, which is a reasonable tradeoff to avoid duplicated code and the
> unholy macro mess.

This is great.  Thanks!

Re: [patch 06/41] x86/traps: Simplify pagefault tracing logic

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:09 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

>  dotraplinkage void notrace
> -trace_do_page_fault(struct pt_regs *regs, unsigned long error_code)
> +do_page_fault(struct pt_regs *regs, unsigned long error_code)
>  {
> +	unsigned long address = read_cr2(); /* Get the faulting address */
> +	enum ctx_state prev_state;
> +
>  	/*
> -	 * The exception_enter and tracepoint processing could
> -	 * trigger another page faults (user space callchain
> -	 * reading) and destroy the original cr2 value, so read
> -	 * the faulting address now.
> +	 * We must have this function tagged with __kprobes, notrace and call
> +	 * read_cr2() before calling anything else. To avoid calling any kind
> +	 * of tracing machinery before we've observed the CR2 value.
> +	 *
> +	 * exception_{enter,exit}() contain all sorts of tracepoints.
>  	 */
> -	unsigned long address = read_cr2();
> -	enum ctx_state prev_state;
> +	if (trace_irqvectors_enabled())
> +		trace_page_fault_entries(address, regs, error_code);
>  
>  	prev_state = exception_enter();
> -	trace_page_fault_entries(address, regs, error_code);

I believe you need to keep the tracing after the "exception_enter()", as
for NO_HZ_FULL, that enables RCU again, and the tracepoint needs RCU
enabled.

-- Steve


>  	__do_page_fault(regs, error_code, address);
>  	exception_exit(prev_state);
>  }
> -NOKPROBE_SYMBOL(trace_do_page_fault);
> -#endif /* CONFIG_TRACING */
> +NOKPROBE_SYMBOL(do_page_fault);
> 

Re: [patch 06/41] x86/traps: Simplify pagefault tracing logic

[Thomas Gleixner]

On Fri, 25 Aug 2017, Steven Rostedt wrote:

> On Fri, 25 Aug 2017 12:31:09 +0200
> Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> >  dotraplinkage void notrace
> > -trace_do_page_fault(struct pt_regs *regs, unsigned long error_code)
> > +do_page_fault(struct pt_regs *regs, unsigned long error_code)
> >  {
> > +	unsigned long address = read_cr2(); /* Get the faulting address */
> > +	enum ctx_state prev_state;
> > +
> >  	/*
> > -	 * The exception_enter and tracepoint processing could
> > -	 * trigger another page faults (user space callchain
> > -	 * reading) and destroy the original cr2 value, so read
> > -	 * the faulting address now.
> > +	 * We must have this function tagged with __kprobes, notrace and call
> > +	 * read_cr2() before calling anything else. To avoid calling any kind
> > +	 * of tracing machinery before we've observed the CR2 value.
> > +	 *
> > +	 * exception_{enter,exit}() contain all sorts of tracepoints.
> >  	 */
> > -	unsigned long address = read_cr2();
> > -	enum ctx_state prev_state;
> > +	if (trace_irqvectors_enabled())
> > +		trace_page_fault_entries(address, regs, error_code);
> >  
> >  	prev_state = exception_enter();
> > -	trace_page_fault_entries(address, regs, error_code);
> 
> I believe you need to keep the tracing after the "exception_enter()", as
> for NO_HZ_FULL, that enables RCU again, and the tracepoint needs RCU
> enabled.

Indeed. Thanks for spotting it.

Re: [patch 05/41] x86/tracing: Introduce a static key for exception tracing

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:08 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> Switching the IDT just for avoiding tracepoints creates a complete
> impenetrable macro/inline/ifdef mess.
> 
> There is no point in avoiding tracepoints for most of the traps/exceptions.
> For the more expensive tracepoints, like pagefaults, this can be handled with
> an explicit static key.
> 
> Preparatory patch to remove the tracing idt.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  arch/x86/include/asm/trace/common.h      |   15 +++++++++++++++
>  arch/x86/include/asm/trace/exceptions.h  |    4 +---
>  arch/x86/include/asm/trace/irq_vectors.h |    4 +---
>  arch/x86/kernel/tracepoint.c             |    9 ++++++++-
>  4 files changed, 25 insertions(+), 7 deletions(-)
> 
> --- /dev/null
> +++ b/arch/x86/include/asm/trace/common.h

Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

-- Steve

Re: [patch 07/41] x86/apic: Remove the duplicated tracing version of local_timer_interrupt

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:10 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> The two NOP5 are noise in the rest of the work which is done by the timer
> interrupt and modern CPUs are pretty good in optimizing nops.
> 
> Get rid of the interrupt handler duplication and move the tracepoints into
> the regular handler.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

-- Steve

Re: [patch 09/41] x86/irq: Get rid of duplicated trace_x86_platform_ipi() code

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:12 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> Two NOP5 are really a good tradeoff vs. the unholy IDT switching mess,
> which duplicates code all over the place.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

-- Steve

> ---
>  arch/x86/include/asm/hw_irq.h |    2 +-
>  arch/x86/kernel/irq.c         |   25 +++++--------------------
>  2 files changed, 6 insertions(+), 21 deletions(-)
> 

Re: [patch 10/41] x86/apic: Remove the duplicated tracing versions of interrupts

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:13 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> The error and the spurious interrupt are really rare events and not at all
> so performance sensitive that two NOP5s can not be tolerated when tracing
> is disabled.

Just a note. I'm sure if we disassembled it, it may be a little more
work done than just two NOPs, as parameter passing to the tracepoints
sometimes leak out of the static jump block. It's moot on this patch,
but other irqs with fast paths may need to be looked at.

> 
> Remove the nonsense.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  arch/x86/include/asm/hw_irq.h |    4 +--
>  arch/x86/kernel/apic/apic.c   |   43 +++++++++---------------------------------
>  2 files changed, 12 insertions(+), 35 deletions(-)
> 

Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

-- Steve

Re: [patch 11/41] x86/irqwork: Get rid of duplicated tracing interrupt code

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:14 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:


> --- a/arch/x86/kernel/irq_work.c
> +++ b/arch/x86/kernel/irq_work.c
> @@ -11,24 +11,12 @@
>  #include <asm/trace/irq_vectors.h>
>  #include <linux/interrupt.h>
>  
> -static inline void __smp_irq_work_interrupt(void)
> -{
> -	inc_irq_stat(apic_irq_work_irqs);
> -	irq_work_run();
> -}
> -
>  __visible void __irq_entry smp_irq_work_interrupt(struct pt_regs *regs)
>  {
>  	ipi_entering_ack_irq();
> -	__smp_irq_work_interrupt();
> -	exiting_irq();
> -}
> -
> -__visible void __irq_entry smp_trace_irq_work_interrupt(struct pt_regs *regs)
> -{
> -	ipi_entering_ack_irq();
> +	inc_irq_stat(apic_irq_work_irqs);
>  	trace_irq_work_entry(IRQ_WORK_VECTOR);
> -	__smp_irq_work_interrupt();

This changes from what we use to have:

Original:

	ipi_entering_ack_irq();
	trace_irq_work_entry(IRQ_WORK_VECTOR);
	inc_irq_stat(apic_irq_work_irqs);
	irq_work_run();

Patched:

	ipi_entering_ack_irq();
	inc_irq_stat(apic_irq_work_irqs);
	trace_irq_work_entry(IRQ_WORK_VECTOR);
	irq_work_run();

Is there a reason for the swap? It's not mentioned in the change log.

Also, tracepoints can issue new irq works. I don't think that would be
a problem here, as interrupts are disabled in this path. Just wanted to
point it out.

-- Steve


> +	irq_work_run();
>  	trace_irq_work_exit(IRQ_WORK_VECTOR);
>  	exiting_irq();
>  }
> 

Re: [patch 12/41] x86/mce: Remove duplicated tracing interrupt code

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:15 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:


> --- a/arch/x86/kernel/cpu/mcheck/mce_amd.c
> +++ b/arch/x86/kernel/cpu/mcheck/mce_amd.c
> @@ -776,24 +776,12 @@ static void __log_error(unsigned int ban
>  	mce_log(&m);
>  }
>  
> -static inline void __smp_deferred_error_interrupt(void)
> -{
> -	inc_irq_stat(irq_deferred_error_count);
> -	deferred_error_int_vector();
> -}
> -
>  asmlinkage __visible void __irq_entry smp_deferred_error_interrupt(void)
>  {
>  	entering_irq();
> -	__smp_deferred_error_interrupt();
> -	exiting_ack_irq();
> -}
> -
> -asmlinkage __visible void __irq_entry smp_trace_deferred_error_interrupt(void)
> -{
> -	entering_irq();
>  	trace_deferred_error_apic_entry(DEFERRED_ERROR_VECTOR);
> -	__smp_deferred_error_interrupt();
> +	inc_irq_stat(irq_deferred_error_count);

Here we do the inc_irq_stat() after the tracepoint (same as the
original).

> +	deferred_error_int_vector();
>  	trace_deferred_error_apic_exit(DEFERRED_ERROR_VECTOR);
>  	exiting_ack_irq();
>  }
> --- a/arch/x86/kernel/cpu/mcheck/therm_throt.c
> +++ b/arch/x86/kernel/cpu/mcheck/therm_throt.c
> @@ -390,26 +390,12 @@ static void unexpected_thermal_interrupt
>  
>  static void (*smp_thermal_vector)(void) = unexpected_thermal_interrupt;
>  
> -static inline void __smp_thermal_interrupt(void)
> -{
> -	inc_irq_stat(irq_thermal_count);
> -	smp_thermal_vector();
> -}
> -
> -asmlinkage __visible void __irq_entry
> -smp_thermal_interrupt(struct pt_regs *regs)
> -{
> -	entering_irq();
> -	__smp_thermal_interrupt();
> -	exiting_ack_irq();
> -}
> -
> -asmlinkage __visible void __irq_entry
> -smp_trace_thermal_interrupt(struct pt_regs *regs)
> +asmlinkage __visible void __irq_entry smp_thermal_interrupt(struct pt_regs *r)
>  {
>  	entering_irq();
> +	inc_irq_stat(irq_thermal_count);
>  	trace_thermal_apic_entry(THERMAL_APIC_VECTOR);

Here we do the inc_irq_stat() after the tracepoint (swapped from the
original). Was that intentional?

> -	__smp_thermal_interrupt();
> +	smp_thermal_vector();
>  	trace_thermal_apic_exit(THERMAL_APIC_VECTOR);
>  	exiting_ack_irq();
>  }
> --- a/arch/x86/kernel/cpu/mcheck/threshold.c
> +++ b/arch/x86/kernel/cpu/mcheck/threshold.c
> @@ -17,24 +17,12 @@ static void default_threshold_interrupt(
>  
>  void (*mce_threshold_vector)(void) = default_threshold_interrupt;
>  
> -static inline void __smp_threshold_interrupt(void)
> -{
> -	inc_irq_stat(irq_threshold_count);
> -	mce_threshold_vector();
> -}
> -
>  asmlinkage __visible void __irq_entry smp_threshold_interrupt(void)
>  {
>  	entering_irq();
> -	__smp_threshold_interrupt();
> -	exiting_ack_irq();
> -}
> -
> -asmlinkage __visible void __irq_entry smp_trace_threshold_interrupt(void)
> -{
> -	entering_irq();
>  	trace_threshold_apic_entry(THRESHOLD_APIC_VECTOR);
> -	__smp_threshold_interrupt();
> +	inc_irq_stat(irq_threshold_count);

Back to inc_irq_stat() after the tracepoint.

-- Steve

> +	mce_threshold_vector();
>  	trace_threshold_apic_exit(THRESHOLD_APIC_VECTOR);
>  	exiting_ack_irq();
>  }
> 

Re: [patch 12/41] x86/mce: Remove duplicated tracing interrupt code

[Thomas Gleixner]

On Fri, 25 Aug 2017, Steven Rostedt wrote:
> On Fri, 25 Aug 2017 12:31:15 +0200
> Thomas Gleixner <tglx@linutronix.de> wrote:
> >  	trace_deferred_error_apic_entry(DEFERRED_ERROR_VECTOR);
> > -	__smp_deferred_error_interrupt();
> > +	inc_irq_stat(irq_deferred_error_count);
> 
> Here we do the inc_irq_stat() after the tracepoint (same as the
> original).
> >  	entering_irq();
> > +	inc_irq_stat(irq_thermal_count);
> >  	trace_thermal_apic_entry(THERMAL_APIC_VECTOR);
> 
> Here we do the inc_irq_stat() after the tracepoint (swapped from the
> original). Was that intentional?

No. Not that it matters at all, but I can make it consistent everywhere.

Re: [patch 13/41] x86/smp: Remove pointless duplicated interrupt code

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:16 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> -static inline void __smp_call_function_interrupt(void)
> -{
> -	generic_smp_call_function_interrupt();
> -	inc_irq_stat(irq_call_count);

Interesting. The origin code calls the
generic_smp_call_function_interrupt() and then calls inc_irq_stat().
Not sure it matters, but just stating this because...

>  }
>  
>  __visible void __irq_entry smp_call_function_interrupt(struct pt_regs *regs)
>  {
>  	ipi_entering_ack_irq();
> -	__smp_call_function_interrupt();
> -	exiting_irq();
> -}
> -
> -__visible void __irq_entry
> -smp_trace_call_function_interrupt(struct pt_regs *regs)
> -{
> -	ipi_entering_ack_irq();
>  	trace_call_function_entry(CALL_FUNCTION_VECTOR);
> -	__smp_call_function_interrupt();
> -	trace_call_function_exit(CALL_FUNCTION_VECTOR);
> -	exiting_irq();
> -}
> -
> -static inline void __smp_call_function_single_interrupt(void)
> -{
> -	generic_smp_call_function_single_interrupt();
>  	inc_irq_stat(irq_call_count);
> -}
> -
> -__visible void __irq_entry
> -smp_call_function_single_interrupt(struct pt_regs *regs)
> -{
> -	ipi_entering_ack_irq();
> -	__smp_call_function_single_interrupt();
> +	generic_smp_call_function_interrupt();

here and

> +	trace_call_function_exit(CALL_FUNCTION_VECTOR);
>  	exiting_irq();
>  }
>  
> -__visible void __irq_entry
> -smp_trace_call_function_single_interrupt(struct pt_regs *regs)
> +__visible void __irq_entry smp_call_function_single_interrupt(struct pt_regs *r)
>  {
>  	ipi_entering_ack_irq();
>  	trace_call_function_single_entry(CALL_FUNCTION_SINGLE_VECTOR);
> -	__smp_call_function_single_interrupt();
> +	inc_irq_stat(irq_call_count);
> +	generic_smp_call_function_single_interrupt();

here, you reverse it.

Should at least be mentioned in the change log.

-- Steve

>  	trace_call_function_single_exit(CALL_FUNCTION_SINGLE_VECTOR);
>  	exiting_irq();
>  }
> 

Re: [patch 14/41] x86/smp: Use static key for reschedule interrupt tracing

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:17 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:


> -__visible void __irq_entry smp_trace_reschedule_interrupt(struct pt_regs *regs)
> -{
> -	/*
> -	 * Need to call irq_enter() before calling the trace point.
> -	 * __smp_reschedule_interrupt() calls irq_enter/exit() too (in
> -	 * scheduler_ipi(). This is OK, since those functions are allowed
> -	 * to nest.
> -	 */
> -	ipi_entering_ack_irq();
> -	trace_reschedule_entry(RESCHEDULE_VECTOR);
> +	if (trace_irqvectors_enabled()) {
> +		/*
> +		 * scheduler_ipi() might call irq_enter() as well, but
> +		 * nested calls are fine.
> +		 */
> +		irq_enter();
> +		trace_reschedule_entry(RESCHEDULE_VECTOR);
> +	}
> +
>  	inc_irq_stat(irq_resched_count);
>  	scheduler_ipi();
> -	trace_reschedule_exit(RESCHEDULE_VECTOR);
> -	exiting_irq();
> +
> +	if (trace_irqvectors_enabled()) {
> +		trace_reschedule_exit(RESCHEDULE_VECTOR);
> +		irq_exit();

This is racy. If the static key gets changed on one and not the other,
we lose the irq_enter/irq_exit match.

-- Steve


> +	}
>  }
>  
>  __visible void __irq_entry smp_call_function_interrupt(struct pt_regs *regs)
> 

Re: [patch 14/41] x86/smp: Use static key for reschedule interrupt tracing

[Thomas Gleixner]

On Fri, 25 Aug 2017, Steven Rostedt wrote:
> On Fri, 25 Aug 2017 12:31:17 +0200
> Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> 
> > -__visible void __irq_entry smp_trace_reschedule_interrupt(struct pt_regs *regs)
> > -{
> > -	/*
> > -	 * Need to call irq_enter() before calling the trace point.
> > -	 * __smp_reschedule_interrupt() calls irq_enter/exit() too (in
> > -	 * scheduler_ipi(). This is OK, since those functions are allowed
> > -	 * to nest.
> > -	 */
> > -	ipi_entering_ack_irq();
> > -	trace_reschedule_entry(RESCHEDULE_VECTOR);
> > +	if (trace_irqvectors_enabled()) {
> > +		/*
> > +		 * scheduler_ipi() might call irq_enter() as well, but
> > +		 * nested calls are fine.
> > +		 */
> > +		irq_enter();
> > +		trace_reschedule_entry(RESCHEDULE_VECTOR);
> > +	}
> > +
> >  	inc_irq_stat(irq_resched_count);
> >  	scheduler_ipi();
> > -	trace_reschedule_exit(RESCHEDULE_VECTOR);
> > -	exiting_irq();
> > +
> > +	if (trace_irqvectors_enabled()) {
> > +		trace_reschedule_exit(RESCHEDULE_VECTOR);
> > +		irq_exit();
> 
> This is racy. If the static key gets changed on one and not the other,
> we lose the irq_enter/irq_exit match.

Indeed. Good point. Fixed up version below.

Thanks,

	tglx
8<---------------------
--- a/arch/x86/include/asm/hw_irq.h
+++ b/arch/x86/include/asm/hw_irq.h
@@ -48,7 +48,7 @@ extern asmlinkage void call_function_sin
 
 #ifdef CONFIG_TRACING
 /* Interrupt handlers registered during init_IRQ */
-extern void trace_reschedule_interrupt(void);
+#define trace_reschedule_interrupt	reschedule_interrupt
 #define trace_call_function_interrupt	call_function_interrupt
 #define trace_call_function_single_interrupt	call_function_single_interrupt
 #define trace_thermal_interrupt	thermal_interrupt
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -254,37 +254,27 @@ static void native_stop_other_cpus(int w
 }
 
 /*
- * Reschedule call back.
+ * Reschedule call back. KVM uses this interrupt to force a cpu out of
+ * guest mode
  */
-static inline void __smp_reschedule_interrupt(void)
-{
-	inc_irq_stat(irq_resched_count);
-	scheduler_ipi();
-}
-
 __visible void __irq_entry smp_reschedule_interrupt(struct pt_regs *regs)
 {
 	ack_APIC_irq();
-	__smp_reschedule_interrupt();
-	/*
-	 * KVM uses this interrupt to force a cpu out of guest mode
-	 */
-}
-
-__visible void __irq_entry smp_trace_reschedule_interrupt(struct pt_regs *regs)
-{
-	/*
-	 * Need to call irq_enter() before calling the trace point.
-	 * __smp_reschedule_interrupt() calls irq_enter/exit() too (in
-	 * scheduler_ipi(). This is OK, since those functions are allowed
-	 * to nest.
-	 */
-	ipi_entering_ack_irq();
-	trace_reschedule_entry(RESCHEDULE_VECTOR);
 	inc_irq_stat(irq_resched_count);
+
+	if (trace_irqvectors_enabled()) {
+		/*
+		 * scheduler_ipi() might call irq_enter() as well, but
+		 * nested calls are fine.
+		 */
+		irq_enter();
+		trace_reschedule_entry(RESCHEDULE_VECTOR);
+		scheduler_ipi();
+		trace_reschedule_exit(RESCHEDULE_VECTOR);
+		irq_exit();
+		return;
+	}
 	scheduler_ipi();
-	trace_reschedule_exit(RESCHEDULE_VECTOR);
-	exiting_irq();
 }
 
 __visible void __irq_entry smp_call_function_interrupt(struct pt_regs *regs)

Re: [patch 14/41] x86/smp: Use static key for reschedule interrupt tracing

[Steven Rostedt]

On Fri, 25 Aug 2017 22:07:33 +0200 (CEST)
Thomas Gleixner <tglx@linutronix.de> wrote:

> Fixed up version below.
> 

Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

-- Steve

> Thanks,
> 
> 	tglx
> 8<---------------------
> --- a/arch/x86/include/asm/hw_irq.h
> +++ b/arch/x86/include/asm/hw_irq.h
> @@ -48,7 +48,7 @@ extern asmlinkage void call_function_sin
>  
>  #ifdef CONFIG_TRACING
>  /* Interrupt handlers registered during init_IRQ */
> -extern void trace_reschedule_interrupt(void);
> +#define trace_reschedule_interrupt	reschedule_interrupt
>  #define trace_call_function_interrupt	call_function_interrupt
>  #define trace_call_function_single_interrupt	call_function_single_interrupt
>  #define trace_thermal_interrupt	thermal_interrupt
> --- a/arch/x86/kernel/smp.c
> +++ b/arch/x86/kernel/smp.c
> @@ -254,37 +254,27 @@ static void native_stop_other_cpus(int w
>  }
>  
>  /*
> - * Reschedule call back.
> + * Reschedule call back. KVM uses this interrupt to force a cpu out of
> + * guest mode
>   */
> -static inline void __smp_reschedule_interrupt(void)
> -{
> -	inc_irq_stat(irq_resched_count);
> -	scheduler_ipi();
> -}
> -
>  __visible void __irq_entry smp_reschedule_interrupt(struct pt_regs *regs)
>  {
>  	ack_APIC_irq();
> -	__smp_reschedule_interrupt();
> -	/*
> -	 * KVM uses this interrupt to force a cpu out of guest mode
> -	 */
> -}
> -
> -__visible void __irq_entry smp_trace_reschedule_interrupt(struct pt_regs *regs)
> -{
> -	/*
> -	 * Need to call irq_enter() before calling the trace point.
> -	 * __smp_reschedule_interrupt() calls irq_enter/exit() too (in
> -	 * scheduler_ipi(). This is OK, since those functions are allowed
> -	 * to nest.
> -	 */
> -	ipi_entering_ack_irq();
> -	trace_reschedule_entry(RESCHEDULE_VECTOR);
>  	inc_irq_stat(irq_resched_count);
> +
> +	if (trace_irqvectors_enabled()) {
> +		/*
> +		 * scheduler_ipi() might call irq_enter() as well, but
> +		 * nested calls are fine.
> +		 */
> +		irq_enter();
> +		trace_reschedule_entry(RESCHEDULE_VECTOR);
> +		scheduler_ipi();
> +		trace_reschedule_exit(RESCHEDULE_VECTOR);
> +		irq_exit();
> +		return;
> +	}
>  	scheduler_ipi();
> -	trace_reschedule_exit(RESCHEDULE_VECTOR);
> -	exiting_irq();
>  }
>  
>  __visible void __irq_entry smp_call_function_interrupt(struct pt_regs *regs)
> 

Re: [patch 15/41] x86/idt: Remove tracing idt completely

[Steven Rostedt]

On Fri, 25 Aug 2017 12:31:18 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:

> No more users of the tracing idt. All exception tracepoints have been moved
> into the regular handlers. Get rid of the mess which shouldn't have been
> created in the first place.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  arch/x86/entry/entry_32.S         |   12 -----
>  arch/x86/entry/entry_64.S         |   13 ------
>  arch/x86/include/asm/desc.h       |   82 ++------------------------------------
>  arch/x86/include/asm/entry_arch.h |   14 ++----
>  arch/x86/include/asm/hw_irq.h     |   20 ---------
>  arch/x86/include/asm/segment.h    |    3 -
>  arch/x86/kernel/tracepoint.c      |   43 -------------------
>  arch/x86/kernel/traps.c           |    6 --
>  8 files changed, 14 insertions(+), 179 deletions(-)

> --- a/arch/x86/include/asm/entry_arch.h
> +++ b/arch/x86/include/asm/entry_arch.h
> @@ -13,20 +13,16 @@
>  BUILD_INTERRUPT(reschedule_interrupt,RESCHEDULE_VECTOR)
>  BUILD_INTERRUPT(call_function_interrupt,CALL_FUNCTION_VECTOR)
>  BUILD_INTERRUPT(call_function_single_interrupt,CALL_FUNCTION_SINGLE_VECTOR)
> -BUILD_INTERRUPT3(irq_move_cleanup_interrupt, IRQ_MOVE_CLEANUP_VECTOR,
> -		 smp_irq_move_cleanup_interrupt)
> -BUILD_INTERRUPT3(reboot_interrupt, REBOOT_VECTOR, smp_reboot_interrupt)
> +BUILD_INTERRUPT(irq_move_cleanup_interrupt, IRQ_MOVE_CLEANUP_VECTOR)
> +BUILD_INTERRUPT(reboot_interrupt, REBOOT_VECTOR)
>  #endif
>  
>  BUILD_INTERRUPT(x86_platform_ipi, X86_PLATFORM_IPI_VECTOR)
>  
>  #ifdef CONFIG_HAVE_KVM
> -BUILD_INTERRUPT3(kvm_posted_intr_ipi, POSTED_INTR_VECTOR,
> -		 smp_kvm_posted_intr_ipi)
> -BUILD_INTERRUPT3(kvm_posted_intr_wakeup_ipi, POSTED_INTR_WAKEUP_VECTOR,
> -		 smp_kvm_posted_intr_wakeup_ipi)
> -BUILD_INTERRUPT3(kvm_posted_intr_nested_ipi, POSTED_INTR_NESTED_VECTOR,
> -		 smp_kvm_posted_intr_nested_ipi)
> +BUILD_INTERRUPT(kvm_posted_intr_ipi, POSTED_INTR_VECTOR)
> +BUILD_INTERRUPT(kvm_posted_intr_wakeup_ipi, POSTED_INTR_WAKEUP_VECTOR)
> +BUILD_INTERRUPT(kvm_posted_intr_nested_ipi, POSTED_INTR_NESTED_VECTOR)
>  #endif


This looks like a different change than what is specified in the change
log. It looks like a clean up. Shouldn't it be a separate patch?

-- Steve

Re: [patch 15/41] x86/idt: Remove tracing idt completely

[Thomas Gleixner]

On Fri, 25 Aug 2017, Steven Rostedt wrote:
> On Fri, 25 Aug 2017 12:31:18 +0200
> Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> > No more users of the tracing idt. All exception tracepoints have been moved
> > into the regular handlers. Get rid of the mess which shouldn't have been
> > created in the first place.
> > 
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > ---
> >  arch/x86/entry/entry_32.S         |   12 -----
> >  arch/x86/entry/entry_64.S         |   13 ------
> >  arch/x86/include/asm/desc.h       |   82 ++------------------------------------
> >  arch/x86/include/asm/entry_arch.h |   14 ++----
> >  arch/x86/include/asm/hw_irq.h     |   20 ---------
> >  arch/x86/include/asm/segment.h    |    3 -
> >  arch/x86/kernel/tracepoint.c      |   43 -------------------
> >  arch/x86/kernel/traps.c           |    6 --
> >  8 files changed, 14 insertions(+), 179 deletions(-)
> 
> > --- a/arch/x86/include/asm/entry_arch.h
> > +++ b/arch/x86/include/asm/entry_arch.h
> > @@ -13,20 +13,16 @@
> >  BUILD_INTERRUPT(reschedule_interrupt,RESCHEDULE_VECTOR)
> >  BUILD_INTERRUPT(call_function_interrupt,CALL_FUNCTION_VECTOR)
> >  BUILD_INTERRUPT(call_function_single_interrupt,CALL_FUNCTION_SINGLE_VECTOR)
> > -BUILD_INTERRUPT3(irq_move_cleanup_interrupt, IRQ_MOVE_CLEANUP_VECTOR,
> > -		 smp_irq_move_cleanup_interrupt)
> > -BUILD_INTERRUPT3(reboot_interrupt, REBOOT_VECTOR, smp_reboot_interrupt)
> > +BUILD_INTERRUPT(irq_move_cleanup_interrupt, IRQ_MOVE_CLEANUP_VECTOR)
> > +BUILD_INTERRUPT(reboot_interrupt, REBOOT_VECTOR)
> >  #endif
> >  
> >  BUILD_INTERRUPT(x86_platform_ipi, X86_PLATFORM_IPI_VECTOR)
> >  
> >  #ifdef CONFIG_HAVE_KVM
> > -BUILD_INTERRUPT3(kvm_posted_intr_ipi, POSTED_INTR_VECTOR,
> > -		 smp_kvm_posted_intr_ipi)
> > -BUILD_INTERRUPT3(kvm_posted_intr_wakeup_ipi, POSTED_INTR_WAKEUP_VECTOR,
> > -		 smp_kvm_posted_intr_wakeup_ipi)
> > -BUILD_INTERRUPT3(kvm_posted_intr_nested_ipi, POSTED_INTR_NESTED_VECTOR,
> > -		 smp_kvm_posted_intr_nested_ipi)
> > +BUILD_INTERRUPT(kvm_posted_intr_ipi, POSTED_INTR_VECTOR)
> > +BUILD_INTERRUPT(kvm_posted_intr_wakeup_ipi, POSTED_INTR_WAKEUP_VECTOR)
> > +BUILD_INTERRUPT(kvm_posted_intr_nested_ipi, POSTED_INTR_NESTED_VECTOR)
> >  #endif
> 
> 
> This looks like a different change than what is specified in the change
> log. It looks like a clean up. Shouldn't it be a separate patch?

Let me split it up.

Re: [patch 15/41] x86/idt: Remove tracing idt completely

[Thomas Gleixner]

On Fri, 25 Aug 2017, Thomas Gleixner wrote:
> On Fri, 25 Aug 2017, Steven Rostedt wrote:
> > On Fri, 25 Aug 2017 12:31:18 +0200
> > Thomas Gleixner <tglx@linutronix.de> wrote:
> > 
> > > No more users of the tracing idt. All exception tracepoints have been moved
> > > into the regular handlers. Get rid of the mess which shouldn't have been
> > > created in the first place.
> > > 
> > > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > > ---
> > >  arch/x86/entry/entry_32.S         |   12 -----
> > >  arch/x86/entry/entry_64.S         |   13 ------
> > >  arch/x86/include/asm/desc.h       |   82 ++------------------------------------
> > >  arch/x86/include/asm/entry_arch.h |   14 ++----
> > >  arch/x86/include/asm/hw_irq.h     |   20 ---------
> > >  arch/x86/include/asm/segment.h    |    3 -
> > >  arch/x86/kernel/tracepoint.c      |   43 -------------------
> > >  arch/x86/kernel/traps.c           |    6 --
> > >  8 files changed, 14 insertions(+), 179 deletions(-)
> > 
> > > --- a/arch/x86/include/asm/entry_arch.h
> > > +++ b/arch/x86/include/asm/entry_arch.h
> > > @@ -13,20 +13,16 @@
> > >  BUILD_INTERRUPT(reschedule_interrupt,RESCHEDULE_VECTOR)
> > >  BUILD_INTERRUPT(call_function_interrupt,CALL_FUNCTION_VECTOR)
> > >  BUILD_INTERRUPT(call_function_single_interrupt,CALL_FUNCTION_SINGLE_VECTOR)
> > > -BUILD_INTERRUPT3(irq_move_cleanup_interrupt, IRQ_MOVE_CLEANUP_VECTOR,
> > > -		 smp_irq_move_cleanup_interrupt)
> > > -BUILD_INTERRUPT3(reboot_interrupt, REBOOT_VECTOR, smp_reboot_interrupt)
> > > +BUILD_INTERRUPT(irq_move_cleanup_interrupt, IRQ_MOVE_CLEANUP_VECTOR)
> > > +BUILD_INTERRUPT(reboot_interrupt, REBOOT_VECTOR)
> > >  #endif
> > >  
> > >  BUILD_INTERRUPT(x86_platform_ipi, X86_PLATFORM_IPI_VECTOR)
> > >  
> > >  #ifdef CONFIG_HAVE_KVM
> > > -BUILD_INTERRUPT3(kvm_posted_intr_ipi, POSTED_INTR_VECTOR,
> > > -		 smp_kvm_posted_intr_ipi)
> > > -BUILD_INTERRUPT3(kvm_posted_intr_wakeup_ipi, POSTED_INTR_WAKEUP_VECTOR,
> > > -		 smp_kvm_posted_intr_wakeup_ipi)
> > > -BUILD_INTERRUPT3(kvm_posted_intr_nested_ipi, POSTED_INTR_NESTED_VECTOR,
> > > -		 smp_kvm_posted_intr_nested_ipi)
> > > +BUILD_INTERRUPT(kvm_posted_intr_ipi, POSTED_INTR_VECTOR)
> > > +BUILD_INTERRUPT(kvm_posted_intr_wakeup_ipi, POSTED_INTR_WAKEUP_VECTOR)
> > > +BUILD_INTERRUPT(kvm_posted_intr_nested_ipi, POSTED_INTR_NESTED_VECTOR)
> > >  #endif
> > 
> > 
> > This looks like a different change than what is specified in the change
> > log. It looks like a clean up. Shouldn't it be a separate patch?
> 
> Let me split it up.

OTOH, the BUILD_INTERRUPT3 users were explicit to avoid the
TRACE_BUILD_INTERRUPT() extra which was hidden behind BUILD_INTERRUPT. So
yes, it's a cleanup, but related.

Thanks,

	tglx

Re: [patch 10/41] x86/apic: Remove the duplicated tracing versions of interrupts

[Peter Zijlstra]

On Fri, Aug 25, 2017 at 11:49:47AM -0400, Steven Rostedt wrote:
> On Fri, 25 Aug 2017 12:31:13 +0200
> Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> > The error and the spurious interrupt are really rare events and not at all
> > so performance sensitive that two NOP5s can not be tolerated when tracing
> > is disabled.
> 
> Just a note. I'm sure if we disassembled it, it may be a little more
> work done than just two NOPs, as parameter passing to the tracepoints
> sometimes leak out of the static jump block. It's moot on this patch,
> but other irqs with fast paths may need to be looked at.

Is that something we can fix with the trace macros?

They have a general shape of:

#define trace_foo(args...)
	if (static_branch_unlikely(&foo_enabled)) {
		__trace_foo(args...);
	}

Right? And I suppose I see why the compiler would want to sometimes lift
stuff out of the branch block, but we'd really like it not to do that.
Would putting a barrier() in front of __trace_foo() help?