[patch 0/6] AMD C1E aware idle support

[patch 0/6] AMD C1E aware idle support

[Thomas Gleixner]

AMD CPUs with C1E support are currently excluded from high resolution
timers and NOHZ support. The reason is that C1E is a BIOS controlled
C3 power state which switches off TSC and the local APIC timer. The
ACPI C-State control manages the TSC/local APIC timer wreckage, but
this does not include the C1 based ("halt" instruction) C1E mode. The
BIOS/SMM controlled C1E state works on most systems even without
enabling ACPI C-State control.

The fact that a system has C1E support enabled is advertised in a MSR,
but the time during boot when the C1E bit is set by the BIOS varies:

1) Boot CPU has already C1E bit set
2) Secondary CPU sets C1E bit
3) C1E bit is set after ACPI C-State query

Case #1 and #2 are covered by the current implementation, but case #3
results in a complete system lockup due to missing timer
interrupts. The current solution is to disable the local APIC timer
and use the PIT in broadcast mode. This restricts the C1E enabled
systems to periodic timer mode.

The following patch series implements a C1E aware idle function which
also covers the late C1E enablement (case #3):

The function is selected during boot for CPUs which have possibly C1E
support. The function checks the MSR which contains the C1E active
bits before executing the halt instruction. When one of the C1E active
bits is set, it makes the system C1E aware by enabling the timer
broadcast mechanism for all CPUs. For high resolution timer and/or
nohz enabled systems it calls the oneshot timer broadcast mechanism
before executing the halt instruction. This is the same mechanism
which is used in the ACPI C-State control for C2/C3 power states.

On my C1E affected X2 box these patches reduce the wakeups/sec down to
20 according to powertop.

The patches work fine on systems which are not affected by the dreaded
ATI chipset timer wreckage. On those which have the problem, the box
needs help from the keyboard to continue working.

The x86 changes for .27 contain a complete overhaul of the affected
code, but this is out of scope for this patchset.

For those who are interested to test those patches on top of 2.6.26-rc
I extracted a patch and added it to the c1e series. It's available
from:

http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
or
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2

@Macej: I bisected your patches and the commit which solves the mysterious
hangs is:

x86: I/O APIC: timer through 8259A second-chance
(7e3530cd98a0c6ab38f5898e855a5beffab26561 in linux-2.6-tip.git)

That's the patch which you worried about possible impacts, but it
seems that it actually fixes the stupid timer irq issue finally. I
have tested it on various machines which had timer irq problems in the
past and they all run smothly. Great work!

Thanks,

	tglx

-- 

[patch 1/6] x86: simplify idle selection

[Thomas Gleixner]

[-- Attachment #1: 0001-391c1a465c0bad7a2c5ab6c46fd5da7fcaee4adc.patch --]
[-- Type: text/plain, Size: 3006 bytes --]

default_idle is selected in cpu_idle(), when no other idle routine is
selected. Select it in select_idle_routine() when mwait is not
selected.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andreas Herrmann <andreas.herrmann3@amd.com>

---
 arch/x86/kernel/process.c    |   18 +++++++-----------
 arch/x86/kernel/process_32.c |    7 +------
 arch/x86/kernel/process_64.c |    7 ++-----
 3 files changed, 10 insertions(+), 22 deletions(-)

Index: linux-2.6/arch/x86/kernel/process.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process.c
+++ linux-2.6/arch/x86/kernel/process.c
@@ -139,27 +139,23 @@ static int __cpuinit mwait_usable(const 
 
 void __cpuinit select_idle_routine(const struct cpuinfo_x86 *c)
 {
-	static int selected;
-
-	if (selected)
-		return;
 #ifdef CONFIG_X86_SMP
 	if (pm_idle == poll_idle && smp_num_siblings > 1) {
 		printk(KERN_WARNING "WARNING: polling idle and HT enabled,"
 			" performance may degrade.\n");
 	}
 #endif
+	if (pm_idle)
+		return;
+
 	if (cpu_has(c, X86_FEATURE_MWAIT) && mwait_usable(c)) {
 		/*
-		 * Skip, if setup has overridden idle.
 		 * One CPU supports mwait => All CPUs supports mwait
 		 */
-		if (!pm_idle) {
-			printk(KERN_INFO "using mwait in idle threads.\n");
-			pm_idle = mwait_idle;
-		}
-	}
-	selected = 1;
+		printk(KERN_INFO "using mwait in idle threads.\n");
+		pm_idle = mwait_idle;
+	} else
+		pm_idle = default_idle;
 }
 
 static int __init idle_setup(char *str)
Index: linux-2.6/arch/x86/kernel/process_32.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process_32.c
+++ linux-2.6/arch/x86/kernel/process_32.c
@@ -168,24 +168,19 @@ void cpu_idle(void)
 	while (1) {
 		tick_nohz_stop_sched_tick();
 		while (!need_resched()) {
-			void (*idle)(void);
 
 			check_pgt_cache();
 			rmb();
-			idle = pm_idle;
 
 			if (rcu_pending(cpu))
 				rcu_check_callbacks(cpu, 0);
 
-			if (!idle)
-				idle = default_idle;
-
 			if (cpu_is_offline(cpu))
 				play_dead();
 
 			local_irq_disable();
 			__get_cpu_var(irq_stat).idle_timestamp = jiffies;
-			idle();
+			pm_idle();
 		}
 		tick_nohz_restart_sched_tick();
 		preempt_enable_no_resched();
Index: linux-2.6/arch/x86/kernel/process_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process_64.c
+++ linux-2.6/arch/x86/kernel/process_64.c
@@ -150,12 +150,9 @@ void cpu_idle(void)
 	while (1) {
 		tick_nohz_stop_sched_tick();
 		while (!need_resched()) {
-			void (*idle)(void);
 
 			rmb();
-			idle = pm_idle;
-			if (!idle)
-				idle = default_idle;
+
 			if (cpu_is_offline(smp_processor_id()))
 				play_dead();
 			/*
@@ -165,7 +162,7 @@ void cpu_idle(void)
 			 */
 			local_irq_disable();
 			enter_idle();
-			idle();
+			pm_idle();
 			/* In many cases the interrupt that ended idle
 			   has already called exit_idle. But some idle
 			   loops can be woken up without interrupt. */

-- 

[patch 2/6] x86: cleanup C1E enabled detection

[Thomas Gleixner]

[-- Attachment #1: 0002-f9fd9011178b2797d28884836c6b60022bcfd359.patch --]
[-- Type: text/plain, Size: 2700 bytes --]

Rename the MSR to INT_PENDING_MSG, which is the name in the data sheet
as well. Move the C1E mask to the header file.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andreas Herrmann <andreas.herrmann3@amd.com>

---
 arch/x86/kernel/cpu/amd.c   |    5 ++---
 arch/x86/kernel/setup_64.c  |    5 ++---
 include/asm-x86/msr-index.h |    4 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

Index: linux-2.6/arch/x86/kernel/cpu/amd.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/amd.c
+++ linux-2.6/arch/x86/kernel/cpu/amd.c
@@ -25,7 +25,6 @@ extern void vide(void);
 __asm__(".align 4\nvide: ret");
 
 #ifdef CONFIG_X86_LOCAL_APIC
-#define ENABLE_C1E_MASK         0x18000000
 #define CPUID_PROCESSOR_SIGNATURE       1
 #define CPUID_XFAM              0x0ff00000
 #define CPUID_XFAM_K8           0x00000000
@@ -45,8 +44,8 @@ static __cpuinit int amd_apic_timer_brok
 			break;
 	case CPUID_XFAM_10H:
 	case CPUID_XFAM_11H:
-		rdmsr(MSR_K8_ENABLE_C1E, lo, hi);
-		if (lo & ENABLE_C1E_MASK) {
+		rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
+		if (lo & K8_INTP_C1E_ACTIVE_MASK) {
 			if (smp_processor_id() != boot_cpu_physical_apicid)
 				printk(KERN_INFO "AMD C1E detected late. "
 				       "	Force timer broadcast.\n");
Index: linux-2.6/arch/x86/kernel/setup_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/setup_64.c
+++ linux-2.6/arch/x86/kernel/setup_64.c
@@ -682,7 +682,6 @@ static void __cpuinit early_init_amd_mc(
 #endif
 }
 
-#define ENABLE_C1E_MASK		0x18000000
 #define CPUID_PROCESSOR_SIGNATURE	1
 #define CPUID_XFAM		0x0ff00000
 #define CPUID_XFAM_K8		0x00000000
@@ -702,8 +701,8 @@ static __cpuinit int amd_apic_timer_brok
 			break;
 	case CPUID_XFAM_10H:
 	case CPUID_XFAM_11H:
-		rdmsr(MSR_K8_ENABLE_C1E, lo, hi);
-		if (lo & ENABLE_C1E_MASK)
+		rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
+		if (lo & K8_INTP_C1E_ACTIVE_MASK)
 			return 1;
 		break;
 	default:
Index: linux-2.6/include/asm-x86/msr-index.h
===================================================================
--- linux-2.6.orig/include/asm-x86/msr-index.h
+++ linux-2.6/include/asm-x86/msr-index.h
@@ -111,7 +111,9 @@
 #define MSR_K8_TOP_MEM2			0xc001001d
 #define MSR_K8_SYSCFG			0xc0010010
 #define MSR_K8_HWCR			0xc0010015
-#define MSR_K8_ENABLE_C1E		0xc0010055
+#define MSR_K8_INT_PENDING_MSG		0xc0010055
+/* C1E active bits in int pending message */
+#define K8_INTP_C1E_ACTIVE_MASK		0x18000000
 #define MSR_K8_TSEG_ADDR		0xc0010112
 #define K8_MTRRFIXRANGE_DRAM_ENABLE	0x00040000 /* MtrrFixDramEn bit    */
 #define K8_MTRRFIXRANGE_DRAM_MODIFY	0x00080000 /* MtrrFixDramModEn bit */

-- 

[patch 3/6] x86: use cpuinfo to check for interrupt pending message msr

[Thomas Gleixner]

[-- Attachment #1: 0003-1635146fac398a95d5320ac8484f393f18786190.patch --]
[-- Type: text/plain, Size: 4126 bytes --]

No need to do a cpuid(1) again. The cpuinfo structure has all
necessary information already.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andreas Herrmann <andreas.herrmann3@amd.com>

---
 arch/x86/kernel/cpu/amd.c  |   41 +++++++++++++++--------------------------
 arch/x86/kernel/setup_64.c |   38 +++++++++++++++-----------------------
 2 files changed, 30 insertions(+), 49 deletions(-)

Index: linux-2.6/arch/x86/kernel/cpu/amd.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/amd.c
+++ linux-2.6/arch/x86/kernel/cpu/amd.c
@@ -25,35 +25,24 @@ extern void vide(void);
 __asm__(".align 4\nvide: ret");
 
 #ifdef CONFIG_X86_LOCAL_APIC
-#define CPUID_PROCESSOR_SIGNATURE       1
-#define CPUID_XFAM              0x0ff00000
-#define CPUID_XFAM_K8           0x00000000
-#define CPUID_XFAM_10H          0x00100000
-#define CPUID_XFAM_11H          0x00200000
-#define CPUID_XMOD              0x000f0000
-#define CPUID_XMOD_REV_F        0x00040000
 
 /* AMD systems with C1E don't have a working lAPIC timer. Check for that. */
-static __cpuinit int amd_apic_timer_broken(void)
+static __cpuinit int amd_apic_timer_broken(struct cpuinfo_x86 *c)
 {
 	u32 lo, hi;
-	u32 eax = cpuid_eax(CPUID_PROCESSOR_SIGNATURE);
-	switch (eax & CPUID_XFAM) {
-	case CPUID_XFAM_K8:
-		if ((eax & CPUID_XMOD) < CPUID_XMOD_REV_F)
-			break;
-	case CPUID_XFAM_10H:
-	case CPUID_XFAM_11H:
-		rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
-		if (lo & K8_INTP_C1E_ACTIVE_MASK) {
-			if (smp_processor_id() != boot_cpu_physical_apicid)
-				printk(KERN_INFO "AMD C1E detected late. "
-				       "	Force timer broadcast.\n");
-			return 1;
-		}
-		break;
-	default:
-		/* err on the side of caution */
+
+	if (c->x86 < 0x0F)
+		return 0;
+
+	/* Family 0x0f models < rev F do not have this MSR */
+	if (c->x86 == 0x0f && c->x86_model < 0x40)
+		return 0;
+
+	rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
+	if (lo & K8_INTP_C1E_ACTIVE_MASK) {
+		if (smp_processor_id() != boot_cpu_physical_apicid)
+			printk(KERN_INFO "AMD C1E detected late. "
+			       "Force timer broadcast.\n");
 		return 1;
 	}
 	return 0;
@@ -297,7 +286,7 @@ static void __cpuinit init_amd(struct cp
 	}
 
 #ifdef CONFIG_X86_LOCAL_APIC
-	if (amd_apic_timer_broken())
+	if (amd_apic_timer_broken(c))
 		local_apic_timer_disabled = 1;
 #endif
 
Index: linux-2.6/arch/x86/kernel/setup_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/setup_64.c
+++ linux-2.6/arch/x86/kernel/setup_64.c
@@ -682,31 +682,23 @@ static void __cpuinit early_init_amd_mc(
 #endif
 }
 
-#define CPUID_PROCESSOR_SIGNATURE	1
-#define CPUID_XFAM		0x0ff00000
-#define CPUID_XFAM_K8		0x00000000
-#define CPUID_XFAM_10H		0x00100000
-#define CPUID_XFAM_11H		0x00200000
-#define CPUID_XMOD		0x000f0000
-#define CPUID_XMOD_REV_F	0x00040000
-
 /* AMD systems with C1E don't have a working lAPIC timer. Check for that. */
-static __cpuinit int amd_apic_timer_broken(void)
+static __cpuinit int amd_apic_timer_broken(struct cpuinfo_x86 *c)
 {
-	u32 lo, hi, eax = cpuid_eax(CPUID_PROCESSOR_SIGNATURE);
+	u32 lo, hi;
 
-	switch (eax & CPUID_XFAM) {
-	case CPUID_XFAM_K8:
-		if ((eax & CPUID_XMOD) < CPUID_XMOD_REV_F)
-			break;
-	case CPUID_XFAM_10H:
-	case CPUID_XFAM_11H:
-		rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
-		if (lo & K8_INTP_C1E_ACTIVE_MASK)
-			return 1;
-		break;
-	default:
-		/* err on the side of caution */
+	if (c->x86 < 0x0F)
+		return 0;
+
+	/* Family 0x0f models < rev F do not have this MSR */
+	if (c->x86 == 0x0f && c->x86_model < 0x40)
+		return 0;
+
+	rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
+	if (lo & K8_INTP_C1E_ACTIVE_MASK) {
+		if (smp_processor_id() != boot_cpu_physical_apicid)
+			printk(KERN_INFO "AMD C1E detected late. "
+			       "Force timer broadcast.\n");
 		return 1;
 	}
 	return 0;
@@ -789,7 +781,7 @@ static void __cpuinit init_amd(struct cp
 	if (c->x86 == 0x10)
 		fam10h_check_enable_mmcfg();
 
-	if (amd_apic_timer_broken())
+	if (amd_apic_timer_broken(c))
 		disable_apic_timer = 1;
 
 	if (c == &boot_cpu_data && c->x86 >= 0xf && c->x86 <= 0x11) {

-- 

[patch 4/6] x86: use cpuid to check MWAIT support for C1

[Thomas Gleixner]

[-- Attachment #1: 0004-eb744213eda636f876ec8f9de12169350a5f44f8.patch --]
[-- Type: text/plain, Size: 1538 bytes --]

cpuid(0x05) provides extended information about MWAIT in EDX when bit
0 of ECX is set. Bit 4-7 of EDX determine whether MWAIT is supported
for C1. C1E enabled CPUs have these bits set to 0.

Based on an ealier patch from Andi Kleen.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andreas Herrmann <andreas.herrmann3@amd.com>

---
 arch/x86/kernel/process.c |   28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

Index: linux-2.6/arch/x86/kernel/process.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process.c
+++ linux-2.6/arch/x86/kernel/process.c
@@ -122,19 +122,31 @@ static void poll_idle(void)
  *
  * idle=mwait overrides this decision and forces the usage of mwait.
  */
+
+#define MWAIT_INFO			0x05
+#define MWAIT_ECX_EXTENDED_INFO		0x01
+#define MWAIT_EDX_C1			0xf0
+
 static int __cpuinit mwait_usable(const struct cpuinfo_x86 *c)
 {
+	u32 eax, ebx, ecx, edx;
+
 	if (force_mwait)
 		return 1;
 
-	if (c->x86_vendor == X86_VENDOR_AMD) {
-		switch(c->x86) {
-		case 0x10:
-		case 0x11:
-			return 0;
-		}
-	}
-	return 1;
+	if (c->cpuid_level < MWAIT_INFO)
+		return 0;
+
+	cpuid(MWAIT_INFO, &eax, &ebx, &ecx, &edx);
+	/* Check, whether EDX has extended info about MWAIT */
+	if (!(ecx & MWAIT_ECX_EXTENDED_INFO))
+		return 1;
+
+	/*
+	 * edx enumerates MONITOR/MWAIT extensions. Check, whether C1
+	 * supports MWAIT
+	 */
+	return edx & MWAIT_EDX_C1;
 }
 
 void __cpuinit select_idle_routine(const struct cpuinfo_x86 *c)

-- 

[patch 5/6] x86: move more common idle functions/variables to process.c

[Thomas Gleixner]

[-- Attachment #1: 0005-0f619dcd9614c003426c472f1fe8227db7cc1a42.patch --]
[-- Type: text/plain, Size: 4819 bytes --]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andreas Herrmann <andreas.herrmann3@amd.com>

---
 arch/x86/kernel/process.c    |   70 +++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/process_32.c |   54 ---------------------------------
 arch/x86/kernel/process_64.c |   28 -----------------
 3 files changed, 70 insertions(+), 82 deletions(-)

Index: linux-2.6/arch/x86/kernel/process.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process.c
+++ linux-2.6/arch/x86/kernel/process.c
@@ -45,6 +45,76 @@ void arch_task_cache_init(void)
 				  SLAB_PANIC, NULL);
 }
 
+/*
+ * Idle related variables and functions
+ */
+unsigned long boot_option_idle_override = 0;
+EXPORT_SYMBOL(boot_option_idle_override);
+
+/*
+ * Powermanagement idle function, if any..
+ */
+void (*pm_idle)(void);
+EXPORT_SYMBOL(pm_idle);
+
+#ifdef CONFIG_X86_32
+/*
+ * This halt magic was a workaround for ancient floppy DMA
+ * wreckage. It should be safe to remove.
+ */
+static int hlt_counter;
+void disable_hlt(void)
+{
+	hlt_counter++;
+}
+EXPORT_SYMBOL(disable_hlt);
+
+void enable_hlt(void)
+{
+	hlt_counter--;
+}
+EXPORT_SYMBOL(enable_hlt);
+
+static inline int hlt_use_halt(void)
+{
+	return !hlt_counter && boot_cpu_data.hlt_works_ok;
+}
+#else
+static inline int hlt_use_halt(void)
+{
+	return 1;
+}
+#endif
+
+/*
+ * We use this if we don't have any better
+ * idle routine..
+ */
+void default_idle(void)
+{
+	if (hlt_use_halt()) {
+		current_thread_info()->status &= ~TS_POLLING;
+		/*
+		 * TS_POLLING-cleared state must be visible before we
+		 * test NEED_RESCHED:
+		 */
+		smp_mb();
+
+		if (!need_resched())
+			safe_halt();	/* enables interrupts racelessly */
+		else
+			local_irq_enable();
+		current_thread_info()->status |= TS_POLLING;
+	} else {
+		local_irq_enable();
+		/* loop is done by the caller */
+		cpu_relax();
+	}
+}
+#ifdef CONFIG_APM_MODULE
+EXPORT_SYMBOL(default_idle);
+#endif
+
 static void do_nothing(void *unused)
 {
 }
Index: linux-2.6/arch/x86/kernel/process_32.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process_32.c
+++ linux-2.6/arch/x86/kernel/process_32.c
@@ -58,11 +58,6 @@
 
 asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
 
-static int hlt_counter;
-
-unsigned long boot_option_idle_override = 0;
-EXPORT_SYMBOL(boot_option_idle_override);
-
 DEFINE_PER_CPU(struct task_struct *, current_task) = &init_task;
 EXPORT_PER_CPU_SYMBOL(current_task);
 
@@ -77,55 +72,6 @@ unsigned long thread_saved_pc(struct tas
 	return ((unsigned long *)tsk->thread.sp)[3];
 }
 
-/*
- * Powermanagement idle function, if any..
- */
-void (*pm_idle)(void);
-EXPORT_SYMBOL(pm_idle);
-
-void disable_hlt(void)
-{
-	hlt_counter++;
-}
-
-EXPORT_SYMBOL(disable_hlt);
-
-void enable_hlt(void)
-{
-	hlt_counter--;
-}
-
-EXPORT_SYMBOL(enable_hlt);
-
-/*
- * We use this if we don't have any better
- * idle routine..
- */
-void default_idle(void)
-{
-	if (!hlt_counter && boot_cpu_data.hlt_works_ok) {
-		current_thread_info()->status &= ~TS_POLLING;
-		/*
-		 * TS_POLLING-cleared state must be visible before we
-		 * test NEED_RESCHED:
-		 */
-		smp_mb();
-
-		if (!need_resched())
-			safe_halt();	/* enables interrupts racelessly */
-		else
-			local_irq_enable();
-		current_thread_info()->status |= TS_POLLING;
-	} else {
-		local_irq_enable();
-		/* loop is done by the caller */
-		cpu_relax();
-	}
-}
-#ifdef CONFIG_APM_MODULE
-EXPORT_SYMBOL(default_idle);
-#endif
-
 #ifdef CONFIG_HOTPLUG_CPU
 #include <asm/nmi.h>
 /* We don't actually take CPU down, just spin without interrupts. */
Index: linux-2.6/arch/x86/kernel/process_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process_64.c
+++ linux-2.6/arch/x86/kernel/process_64.c
@@ -56,15 +56,6 @@ asmlinkage extern void ret_from_fork(voi
 
 unsigned long kernel_thread_flags = CLONE_VM | CLONE_UNTRACED;
 
-unsigned long boot_option_idle_override = 0;
-EXPORT_SYMBOL(boot_option_idle_override);
-
-/*
- * Powermanagement idle function, if any..
- */
-void (*pm_idle)(void);
-EXPORT_SYMBOL(pm_idle);
-
 static ATOMIC_NOTIFIER_HEAD(idle_notifier);
 
 void idle_notifier_register(struct notifier_block *n)
@@ -94,25 +85,6 @@ void exit_idle(void)
 	__exit_idle();
 }
 
-/*
- * We use this if we don't have any better
- * idle routine..
- */
-void default_idle(void)
-{
-	current_thread_info()->status &= ~TS_POLLING;
-	/*
-	 * TS_POLLING-cleared state must be visible before we
-	 * test NEED_RESCHED:
-	 */
-	smp_mb();
-	if (!need_resched())
-		safe_halt();	/* enables interrupts racelessly */
-	else
-		local_irq_enable();
-	current_thread_info()->status |= TS_POLLING;
-}
-
 #ifdef CONFIG_HOTPLUG_CPU
 DECLARE_PER_CPU(int, cpu_state);
 

-- 

[patch 6/6] x86: add c1e aware idle function

[Thomas Gleixner]

[-- Attachment #1: 0006-31a3761dc561e16ed154bb099ef79acee4457ff8.patch --]
[-- Type: text/plain, Size: 9778 bytes --]

C1E on AMD machines is like C3 but without control from the OS. Up to
now we disabled the local apic timer for those machines as it stops
when the CPU goes into C1E. This excludes those machines from high
resolution timers / dynamic ticks, which hurts especially the X2 based
laptops.

The current boot time C1E detection has another more serious flaw:
some BIOSes do not enable C1E until the ACPI processor module is
loaded. This causes systems to stop working after that point.

To work nicely with C1E enabled machines we use a separate idle
function, which checks on idle entry whether C1E was enabled in the
Interrupt Pending Message MSR. This allows us to do timer broadcasting
for C1E and covers the late enablement of C1E as well.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andreas Herrmann <andreas.herrmann3@amd.com>

---
 arch/x86/kernel/apic_32.c    |    5 +--
 arch/x86/kernel/apic_64.c    |   26 ----------------
 arch/x86/kernel/cpu/amd.c    |   30 -------------------
 arch/x86/kernel/process.c    |   66 +++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/setup_64.c   |   25 ----------------
 include/asm-x86/apic.h       |    3 -
 kernel/time/tick-broadcast.c |    6 ++-
 7 files changed, 73 insertions(+), 88 deletions(-)

Index: linux-2.6/arch/x86/kernel/apic_32.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic_32.c
+++ linux-2.6/arch/x86/kernel/apic_32.c
@@ -64,9 +64,8 @@ static int enable_local_apic __initdata;
 
 /* Local APIC timer verification ok */
 static int local_apic_timer_verify_ok;
-/* Disable local APIC timer from the kernel commandline or via dmi quirk
-   or using CPU MSR check */
-int local_apic_timer_disabled;
+/* Disable local APIC timer from the kernel commandline or via dmi quirk */
+static int local_apic_timer_disabled;
 /* Local APIC timer works in C2 */
 int local_apic_timer_c2_ok;
 EXPORT_SYMBOL_GPL(local_apic_timer_c2_ok);
Index: linux-2.6/arch/x86/kernel/apic_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic_64.c
+++ linux-2.6/arch/x86/kernel/apic_64.c
@@ -43,7 +43,7 @@
 #include <mach_ipi.h>
 #include <mach_apic.h>
 
-int disable_apic_timer __cpuinitdata;
+static int disable_apic_timer __cpuinitdata;
 static int apic_calibrate_pmtmr __initdata;
 int disable_apic;
 
@@ -422,32 +422,8 @@ void __init setup_boot_APIC_clock(void)
 	setup_APIC_timer();
 }
 
-/*
- * AMD C1E enabled CPUs have a real nasty problem: Some BIOSes set the
- * C1E flag only in the secondary CPU, so when we detect the wreckage
- * we already have enabled the boot CPU local apic timer. Check, if
- * disable_apic_timer is set and the DUMMY flag is cleared. If yes,
- * set the DUMMY flag again and force the broadcast mode in the
- * clockevents layer.
- */
-static void __cpuinit check_boot_apic_timer_broadcast(void)
-{
-	if (!disable_apic_timer ||
-	    (lapic_clockevent.features & CLOCK_EVT_FEAT_DUMMY))
-		return;
-
-	printk(KERN_INFO "AMD C1E detected late. Force timer broadcast.\n");
-	lapic_clockevent.features |= CLOCK_EVT_FEAT_DUMMY;
-
-	local_irq_enable();
-	clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_FORCE,
-			   &boot_cpu_physical_apicid);
-	local_irq_disable();
-}
-
 void __cpuinit setup_secondary_APIC_clock(void)
 {
-	check_boot_apic_timer_broadcast();
 	setup_APIC_timer();
 }
 
Index: linux-2.6/arch/x86/kernel/cpu/amd.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/amd.c
+++ linux-2.6/arch/x86/kernel/cpu/amd.c
@@ -24,31 +24,6 @@
 extern void vide(void);
 __asm__(".align 4\nvide: ret");
 
-#ifdef CONFIG_X86_LOCAL_APIC
-
-/* AMD systems with C1E don't have a working lAPIC timer. Check for that. */
-static __cpuinit int amd_apic_timer_broken(struct cpuinfo_x86 *c)
-{
-	u32 lo, hi;
-
-	if (c->x86 < 0x0F)
-		return 0;
-
-	/* Family 0x0f models < rev F do not have this MSR */
-	if (c->x86 == 0x0f && c->x86_model < 0x40)
-		return 0;
-
-	rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
-	if (lo & K8_INTP_C1E_ACTIVE_MASK) {
-		if (smp_processor_id() != boot_cpu_physical_apicid)
-			printk(KERN_INFO "AMD C1E detected late. "
-			       "Force timer broadcast.\n");
-		return 1;
-	}
-	return 0;
-}
-#endif
-
 int force_mwait __cpuinitdata;
 
 static void __cpuinit early_init_amd(struct cpuinfo_x86 *c)
@@ -285,11 +260,6 @@ static void __cpuinit init_amd(struct cp
 			num_cache_leaves = 3;
 	}
 
-#ifdef CONFIG_X86_LOCAL_APIC
-	if (amd_apic_timer_broken(c))
-		local_apic_timer_disabled = 1;
-#endif
-
 	/* K6s reports MCEs but don't actually have all the MSRs */
 	if (c->x86 < 6)
 		clear_cpu_cap(c, X86_FEATURE_MCE);
Index: linux-2.6/arch/x86/kernel/process.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process.c
+++ linux-2.6/arch/x86/kernel/process.c
@@ -6,6 +6,7 @@
 #include <linux/sched.h>
 #include <linux/module.h>
 #include <linux/pm.h>
+#include <linux/clockchips.h>
 
 struct kmem_cache *task_xstate_cachep;
 
@@ -219,6 +220,68 @@ static int __cpuinit mwait_usable(const 
 	return edx & MWAIT_EDX_C1;
 }
 
+/*
+ * Check for AMD CPUs, which have potentially C1E support
+ */
+static int __cpuinit check_c1e_idle(const struct cpuinfo_x86 *c)
+{
+	if (c->x86_vendor != X86_VENDOR_AMD)
+		return 0;
+
+	if (c->x86 < 0x0F)
+		return 0;
+
+	/* Family 0x0f models < rev F do not have C1E */
+	if (c->x86 == 0x0f && c->x86_model < 0x40)
+		return 0;
+
+	return 1;
+}
+
+/*
+ * C1E aware idle routine. We check for C1E active in the interrupt
+ * pending message MSR. If we detect C1E, then we handle it the same
+ * way as C3 power states (local apic timer and TSC stop)
+ */
+static void c1e_idle(void)
+{
+	static cpumask_t c1e_mask = CPU_MASK_NONE;
+	static int c1e_detected;
+
+	if (need_resched())
+		return;
+
+	if (!c1e_detected) {
+		u32 lo, hi;
+
+		rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
+		if (lo & K8_INTP_C1E_ACTIVE_MASK) {
+			c1e_detected = 1;
+			mark_tsc_unstable("TSC halt in C1E");
+			printk(KERN_INFO "System has C1E enabled\n");
+		}
+	}
+
+	if (c1e_detected) {
+		int cpu = smp_processor_id();
+
+		if (!cpu_isset(cpu, c1e_mask)) {
+			cpu_set(cpu, c1e_mask);
+			/* Force broadcast so ACPI can not interfere */
+			clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_FORCE,
+					   &cpu);
+			printk(KERN_INFO "Switch to broadcast mode on CPU%d\n",
+			       cpu);
+		}
+		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
+		default_idle();
+		local_irq_disable();
+		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu);
+		local_irq_enable();
+	} else
+		default_idle();
+}
+
 void __cpuinit select_idle_routine(const struct cpuinfo_x86 *c)
 {
 #ifdef CONFIG_X86_SMP
@@ -236,6 +299,9 @@ void __cpuinit select_idle_routine(const
 		 */
 		printk(KERN_INFO "using mwait in idle threads.\n");
 		pm_idle = mwait_idle;
+	} else if (check_c1e_idle(c)) {
+		printk(KERN_INFO "using C1E aware idle routine.\n");
+		pm_idle = c1e_idle;
 	} else
 		pm_idle = default_idle;
 }
Index: linux-2.6/arch/x86/kernel/setup_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/setup_64.c
+++ linux-2.6/arch/x86/kernel/setup_64.c
@@ -682,28 +682,6 @@ static void __cpuinit early_init_amd_mc(
 #endif
 }
 
-/* AMD systems with C1E don't have a working lAPIC timer. Check for that. */
-static __cpuinit int amd_apic_timer_broken(struct cpuinfo_x86 *c)
-{
-	u32 lo, hi;
-
-	if (c->x86 < 0x0F)
-		return 0;
-
-	/* Family 0x0f models < rev F do not have this MSR */
-	if (c->x86 == 0x0f && c->x86_model < 0x40)
-		return 0;
-
-	rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
-	if (lo & K8_INTP_C1E_ACTIVE_MASK) {
-		if (smp_processor_id() != boot_cpu_physical_apicid)
-			printk(KERN_INFO "AMD C1E detected late. "
-			       "Force timer broadcast.\n");
-		return 1;
-	}
-	return 0;
-}
-
 static void __cpuinit early_init_amd(struct cpuinfo_x86 *c)
 {
 	early_init_amd_mc(c);
@@ -781,9 +759,6 @@ static void __cpuinit init_amd(struct cp
 	if (c->x86 == 0x10)
 		fam10h_check_enable_mmcfg();
 
-	if (amd_apic_timer_broken(c))
-		disable_apic_timer = 1;
-
 	if (c == &boot_cpu_data && c->x86 >= 0xf && c->x86 <= 0x11) {
 		unsigned long long tseg;
 
Index: linux-2.6/include/asm-x86/apic.h
===================================================================
--- linux-2.6.orig/include/asm-x86/apic.h
+++ linux-2.6/include/asm-x86/apic.h
@@ -38,12 +38,9 @@ extern void generic_apic_probe(void);
 extern int apic_verbosity;
 extern int timer_over_8254;
 extern int local_apic_timer_c2_ok;
-extern int local_apic_timer_disabled;
 
-extern int apic_runs_main_timer;
 extern int ioapic_force;
 extern int disable_apic;
-extern int disable_apic_timer;
 
 /*
  * Basic functions accessing APICs.
Index: linux-2.6/kernel/time/tick-broadcast.c
===================================================================
--- linux-2.6.orig/kernel/time/tick-broadcast.c
+++ linux-2.6/kernel/time/tick-broadcast.c
@@ -30,6 +30,7 @@
 struct tick_device tick_broadcast_device;
 static cpumask_t tick_broadcast_mask;
 static DEFINE_SPINLOCK(tick_broadcast_lock);
+static int tick_broadcast_force;
 
 #ifdef CONFIG_TICK_ONESHOT
 static void tick_broadcast_clear_oneshot(int cpu);
@@ -232,10 +233,11 @@ static void tick_do_broadcast_on_off(voi
 						     CLOCK_EVT_MODE_SHUTDOWN);
 		}
 		if (*reason == CLOCK_EVT_NOTIFY_BROADCAST_FORCE)
-			dev->features |= CLOCK_EVT_FEAT_DUMMY;
+			tick_broadcast_force = 1;
 		break;
 	case CLOCK_EVT_NOTIFY_BROADCAST_OFF:
-		if (cpu_isset(cpu, tick_broadcast_mask)) {
+		if (!tick_broadcast_force &&
+		    cpu_isset(cpu, tick_broadcast_mask)) {
 			cpu_clear(cpu, tick_broadcast_mask);
 			if (td->mode == TICKDEV_MODE_PERIODIC)
 				tick_setup_periodic(dev, 0);

-- 

Re: [patch 0/6] AMD C1E aware idle support

[Rafael J. Wysocki]

On Thursday, 12 of June 2008, Thomas Gleixner wrote:
> AMD CPUs with C1E support are currently excluded from high resolution
> timers and NOHZ support. The reason is that C1E is a BIOS controlled
> C3 power state which switches off TSC and the local APIC timer. The
> ACPI C-State control manages the TSC/local APIC timer wreckage, but
> this does not include the C1 based ("halt" instruction) C1E mode. The
> BIOS/SMM controlled C1E state works on most systems even without
> enabling ACPI C-State control.
> 
> The fact that a system has C1E support enabled is advertised in a MSR,
> but the time during boot when the C1E bit is set by the BIOS varies:
> 
> 1) Boot CPU has already C1E bit set
> 2) Secondary CPU sets C1E bit
> 3) C1E bit is set after ACPI C-State query
> 
> Case #1 and #2 are covered by the current implementation, but case #3
> results in a complete system lockup due to missing timer
> interrupts. The current solution is to disable the local APIC timer
> and use the PIT in broadcast mode. This restricts the C1E enabled
> systems to periodic timer mode.
> 
> The following patch series implements a C1E aware idle function which
> also covers the late C1E enablement (case #3):
> 
> The function is selected during boot for CPUs which have possibly C1E
> support. The function checks the MSR which contains the C1E active
> bits before executing the halt instruction. When one of the C1E active
> bits is set, it makes the system C1E aware by enabling the timer
> broadcast mechanism for all CPUs. For high resolution timer and/or
> nohz enabled systems it calls the oneshot timer broadcast mechanism
> before executing the halt instruction. This is the same mechanism
> which is used in the ACPI C-State control for C2/C3 power states.
> 
> On my C1E affected X2 box these patches reduce the wakeups/sec down to
> 20 according to powertop.
> 
> The patches work fine on systems which are not affected by the dreaded
> ATI chipset timer wreckage. On those which have the problem, the box
> needs help from the keyboard to continue working.
> 
> The x86 changes for .27 contain a complete overhaul of the affected
> code, but this is out of scope for this patchset.
> 
> For those who are interested to test those patches on top of 2.6.26-rc
> I extracted a patch and added it to the c1e series. It's available
> from:
> 
> http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
> or
> http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2

I guess the x86-next-apic-timer-fixes.patch is a combination of all patches in
this series.  Is that correct?

Thanks,
Rafael

Re: [patch 0/6] AMD C1E aware idle support

[Ingo Molnar]

* Rafael J. Wysocki <rjw@sisk.pl> wrote:

> > http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
> > or
> > http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2
> 
> I guess the x86-next-apic-timer-fixes.patch is a combination of all 
> patches in this series.  Is that correct?

you can also try out those patches via running tip/master. (the patches 
from Thomas are queued up in the tip/x86/cpu topic branch)

	Ingo

Re: [patch 0/6] AMD C1E aware idle support

[Thomas Gleixner]

On Thu, 12 Jun 2008, Rafael J. Wysocki wrote:
> On Thursday, 12 of June 2008, Thomas Gleixner wrote:
> > For those who are interested to test those patches on top of 2.6.26-rc
> > I extracted a patch and added it to the c1e series. It's available
> > from:
> > 
> > http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
> > or
> > http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2
> 
> I guess the x86-next-apic-timer-fixes.patch is a combination of all patches in
> this series.  Is that correct?

Yes. It's the combi patch of everything in the x86/apic branch of
linux-2.6-tip.git.

Thanks,
	tglx

Re: [patch 0/6] AMD C1E aware idle support

[Andreas Herrmann]

On Thu, Jun 12, 2008 at 10:28:32AM -0000, Thomas Gleixner wrote:
> ...
> The following patch series implements a C1E aware idle function which
> also covers the late C1E enablement (case #3):
> ...

Great work!
>From a first glance and test it looks really good.
Just about 10 wakeups from idle on a C1E enabled box here.

Will do some deeper review and more tests later.

> The patches work fine on systems which are not affected by the dreaded
> ATI chipset timer wreckage. On those which have the problem, the box
> needs help from the keyboard to continue working.

I am currently not aware of that timer wreckage problem.
Are you referring to one of the HPET related bugzillas at Kernel Bug
Tracker?
If not, do you have a pointer to more information regarding this?
(Say, problem description and lspci -vxxx output for such a system.)


Thanks,

Andreas

Re: [patch 0/6] AMD C1E aware idle support

[Thomas Gleixner]

On Thu, 12 Jun 2008, Andreas Herrmann wrote:
> On Thu, Jun 12, 2008 at 10:28:32AM -0000, Thomas Gleixner wrote:
> > ...
> > The following patch series implements a C1E aware idle function which
> > also covers the late C1E enablement (case #3):
> > ...
>
> Great work!

Thanks.

> >From a first glance and test it looks really good.
> Just about 10 wakeups from idle on a C1E enabled box here.
> 
> Will do some deeper review and more tests later.
> 
> > The patches work fine on systems which are not affected by the dreaded
> > ATI chipset timer wreckage. On those which have the problem, the box
> > needs help from the keyboard to continue working.
> 
> I am currently not aware of that timer wreckage problem.
> Are you referring to one of the HPET related bugzillas at Kernel Bug
> Tracker?

There might be related ones. Need to look.

> If not, do you have a pointer to more information regarding this?
> (Say, problem description and lspci -vxxx output for such a system.)

Usually those systems have the:

..MP-BIOS bug: 8254 timer not connected to IO-APIC

and the fixup machinery in current mainline is slightly insufficient.

The result on such a system is that with current mainline plus the C1E
idle changes (just do not apply the apic patch from the tarball),
those systems keep missing timer interrupts. I have not yet decoded
the root cause, but the patch which is finally fixing that is:

http://git.kernel.org/?p=linux/kernel/git/tip/linux-2.6-tip.git;a=commit;h=7e3530cd98a0c6ab38f5898e855a5beffab26561

lspci output below.

Thanks,

	tglx
---

00:00.0 Host bridge: ATI Technologies Inc RS690 Host Bridge
	Subsystem: Fujitsu Siemens Computers Unknown device 111d
	Flags: bus master, 66MHz, medium devsel, latency 64
	Memory at <ignored> (64-bit, non-prefetchable)
00: 02 10 10 79 06 00 20 22 00 00 00 06 00 40 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 e0
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 1d 11
30: 00 00 00 00 c4 00 00 00 00 00 00 00 00 00 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 42 20 05 00
50: 34 17 1d 11 00 00 00 00 00 00 00 00 00 00 00 00
60: 80 00 00 00 87 04 00 00 00 02 20 00 d8 d0 01 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 40 44 00 00 95 00 00 03 20 11 10 00 02 27 00 10
90: 00 00 00 38 40 c4 40 e3 00 c0 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 07 01 00 00 49 01 10 07
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 08 00 80 01 60 20 11 11 d0 00 00 00
d0: 25 05 65 00 02 00 00 00 00 00 00 00 00 00 00 00
e0: 10 00 00 00 01 30 03 80 7f 00 00 00 00 00 00 00
f0: 00 00 00 00 00 80 80 00 00 00 00 00 00 00 00 00

00:01.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (Internal gfx) (prog-if 00 [Normal decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=64
	I/O behind bridge: 00009000-00009fff
	Memory behind bridge: f8200000-f83fffff
	Prefetchable memory behind bridge: 00000000f0000000-00000000f7ffffff
	Capabilities: [44] HyperTransport: MSI Mapping Enable+ Fixed+
	Capabilities: [b0] Subsystem: ATI Technologies Inc RS690 PCI to PCI Bridge (Internal gfx)
	Kernel modules: shpchp
00: 02 10 12 79 07 00 30 02 00 00 04 06 00 40 01 00
10: 00 00 00 00 00 00 00 00 00 01 01 40 91 91 20 02
20: 20 f8 30 f8 01 f0 f1 f7 00 00 00 00 00 00 00 00
30: 00 00 00 00 44 00 00 00 00 00 00 00 ff 00 0c 00
40: 00 00 00 00 08 b0 03 a8 00 00 00 00 02 10 12 79
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 0d 00 00 00 02 10 12 79 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:06.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (PCI Express Port 2) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=02, subordinate=07, sec-latency=0
	I/O behind bridge: 0000a000-0000afff
	Memory behind bridge: f8400000-f84fffff
	Prefetchable memory behind bridge: 00000000f8000000-00000000f80fffff
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Root Port (Slot+), MSI 00
	Capabilities: [80] Message Signalled Interrupts: Mask- 64bit- Queue=0/0 Enable+
	Capabilities: [b0] Subsystem: Fujitsu Siemens Computers Unknown device 111d
	Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
	Capabilities: [100] Virtual Channel <?>
	Kernel driver in use: pcieport-driver
	Kernel modules: shpchp
00: 02 10 16 79 07 04 10 00 00 00 04 06 08 00 01 00
10: 00 00 00 00 00 00 00 00 00 02 07 00 a1 a1 00 00
20: 40 f8 40 f8 01 f8 01 f8 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 04 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 01 58 03 c8 00 00 00 00 10 80 41 01 20 80 00 00
60: 1f 08 00 00 11 0c 10 03 42 00 11 30 e0 0c 30 00
70: c0 03 48 00 00 00 01 00 00 00 00 00 00 00 00 00
80: 05 b0 01 00 0c 30 e0 fe 49 41 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 0d b8 00 00 34 17 1d 11 08 00 03 a8 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:07.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (PCI Express Port 3) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0
	Bus: primary=00, secondary=08, subordinate=0d, sec-latency=0
	I/O behind bridge: 0000b000-0000bfff
	Memory behind bridge: f8500000-f85fffff
	Prefetchable memory behind bridge: 00000000f8100000-00000000f81fffff
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Root Port (Slot+), MSI 00
	Capabilities: [80] Message Signalled Interrupts: Mask- 64bit- Queue=0/0 Enable+
	Capabilities: [b0] Subsystem: Fujitsu Siemens Computers Unknown device 111d
	Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
	Capabilities: [100] Virtual Channel <?>
	Kernel driver in use: pcieport-driver
	Kernel modules: shpchp
00: 02 10 17 79 07 04 10 00 00 00 04 06 08 00 01 00
10: 00 00 00 00 00 00 00 00 00 08 0d 00 b1 b1 00 00
20: 50 f8 50 f8 11 f8 11 f8 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 04 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 01 58 03 c8 00 00 00 00 10 80 41 01 20 80 00 00
60: 1f 08 00 00 11 0c 10 04 42 00 11 30 80 0c 30 00
70: c0 03 48 00 00 00 01 00 00 00 00 00 00 00 00 00
80: 05 b0 01 00 0c 30 e0 fe 51 41 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 0d b8 00 00 34 17 1d 11 08 00 03 a8 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:12.0 SATA controller: ATI Technologies Inc SB600 Non-Raid-5 SATA (prog-if 01 [AHCI 1.0])
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 22
	I/O ports at 8440 [size=8]
	I/O ports at 8434 [size=4]
	I/O ports at 8438 [size=8]
	I/O ports at 8430 [size=4]
	I/O ports at 8400 [size=16]
	Memory at f890a000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [60] Power Management version 2
	Kernel driver in use: ahci
	Kernel modules: ahci
00: 02 10 80 43 07 00 30 02 00 01 06 01 00 40 00 00
10: 41 84 00 00 35 84 00 00 39 84 00 00 31 84 00 00
20: 01 84 00 00 00 a0 90 f8 00 00 00 00 34 17 d1 10
30: 00 00 00 00 60 00 00 00 00 00 00 00 0b 01 00 00
40: 10 00 83 02 01 00 10 00 01 00 00 00 00 00 00 00
50: 05 00 84 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 01 00 22 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 12 00 10 00 0f 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 06 00 00 2c d6 01 b4 00 d6 01 b4 00
90: d6 01 b4 00 d6 01 b4 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 b8 00 00 00 00 00 00 00 b8 00 00
b0: 00 00 00 00 00 b8 00 00 00 00 00 00 00 b8 00 00
c0: 00 20 00 00 80 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:13.0 USB Controller: ATI Technologies Inc SB600 USB (OHCI0) (prog-if 10 [OHCI])
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 16
	Memory at f8904000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd
00: 02 10 87 43 17 00 a0 02 00 10 03 0c 08 40 80 00
10: 00 40 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 0a 01 00 00
40: 80 1f 00 00 0a 84 b7 18 07 35 00 00 00 00 00 00
50: 00 9c 00 00 00 00 00 00 ff ff ff ff ff 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 ff 00 00 80 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:13.1 USB Controller: ATI Technologies Inc SB600 USB (OHCI1) (prog-if 10 [OHCI])
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 17
	Memory at f8905000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd
00: 02 10 88 43 17 00 a0 02 00 10 03 0c 08 40 00 00
10: 00 50 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 05 02 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:13.2 USB Controller: ATI Technologies Inc SB600 USB (OHCI2) (prog-if 10 [OHCI])
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18
	Memory at f8906000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd
00: 02 10 89 43 17 00 a0 02 00 10 03 0c 08 40 00 00
10: 00 60 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 0b 03 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:13.3 USB Controller: ATI Technologies Inc SB600 USB (OHCI3) (prog-if 10 [OHCI])
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 17
	Memory at f8907000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd
00: 02 10 8a 43 17 00 a0 02 00 10 03 0c 08 40 00 00
10: 00 70 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 05 02 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:13.4 USB Controller: ATI Technologies Inc SB600 USB (OHCI4) (prog-if 10 [OHCI])
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18
	Memory at f8908000 (32-bit, non-prefetchable) [size=4K]
	Kernel driver in use: ohci_hcd
	Kernel modules: ohci-hcd
00: 02 10 8b 43 17 00 a0 02 00 10 03 0c 08 40 00 00
10: 00 80 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 0b 03 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:13.5 USB Controller: ATI Technologies Inc SB600 USB Controller (EHCI) (prog-if 20 [EHCI])
	Subsystem: ATI Technologies Inc SB600 USB Controller (EHCI)
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 19
	Memory at f890a400 (32-bit, non-prefetchable) [size=256]
	Capabilities: [c0] Power Management version 2
	Capabilities: [e4] Debug port: BAR=1 offset=00e0
	Kernel driver in use: ehci_hcd
	Kernel modules: ehci-hcd
00: 02 10 86 43 17 00 b0 02 00 20 03 0c 10 40 00 00
10: 00 a4 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 02 10 86 43
30: 00 00 00 00 c0 00 00 00 00 00 00 00 0b 04 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 40 00 0e 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 01 00 00 01 00 00 08 80 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 e4 02 7e 00 00 40 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 0a 00 e0 20 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 14)
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: 66MHz, medium devsel
	I/O ports at 8410 [size=16]
	Capabilities: [b0] HyperTransport: MSI Mapping Enable- Fixed+
	Kernel driver in use: piix4_smbus
	Kernel modules: i2c-piix4
00: 02 10 85 43 03 00 30 02 14 00 05 0c 00 00 80 00
10: 11 84 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 b0 00 00 00 00 00 00 00 00 00 00 00
40: d4 bb 00 04 00 00 00 00 0f ff 00 00 00 00 00 00
50: f0 0d 00 00 00 00 f0 08 51 80 03 f3 00 00 00 00
60: 01 00 24 00 bf fd 9e 8f 3f 90 00 00 20 00 00 00
70: 00 01 00 00 08 00 c0 fe ff 6f 00 00 00 00 5a 0f
80: 4a 0a 84 7c 00 00 00 00 00 00 00 00 8c 00 00 80
90: 11 84 00 00 ff de f7 00 00 00 00 00 00 00 00 00
a0: 00 00 ff ff ff ff e0 08 14 28 1c 00 16 79 20 18
b0: 08 00 02 a8 00 00 00 00 00 00 00 00 f0 0f 08 1a
c0: 7d db b6 ed 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: d8 0c 00 00 7f 08 00 00 00 00 00 00 02 00 30 01

00:14.1 IDE interface: ATI Technologies Inc SB600 IDE (prog-if 8a [Master SecP PriP])
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 16
	I/O ports at 01f0 [size=8]
	I/O ports at 03f4 [size=1]
	I/O ports at 0170 [size=8]
	I/O ports at 0374 [size=1]
	I/O ports at 8420 [size=16]
	Kernel driver in use: pata_atiixp
	Kernel modules: ata_generic, pata_acpi, pata_atiixp
00: 02 10 8c 43 15 00 20 02 00 8a 01 01 00 40 00 00
10: f1 01 00 00 f5 03 00 00 01 00 00 00 01 00 00 00
20: 21 84 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 ff 01 00 00
40: 5d 5d 00 00 ff ff 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 40 00 10 2c 01 07 01 00 00 00 ff 03 00 00
70: 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia
	Subsystem: Fujitsu Siemens Computers Unknown device 111e
	Flags: bus master, slow devsel, latency 64, IRQ 16
	Memory at f8900000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 2
	Kernel driver in use: HDA Intel
	Kernel modules: snd-hda-intel
00: 02 10 83 43 06 00 10 04 00 00 03 04 08 40 00 00
10: 04 00 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 1e 11
30: 00 00 00 00 50 00 00 00 00 00 00 00 0a 00 00 00
40: 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 01 00 42 c8 00 00 00 00 00 00 00 00 00 00 00 00
60: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:14.3 ISA bridge: ATI Technologies Inc SB600 PCI to LPC Bridge
	Subsystem: Fujitsu Siemens Computers Unknown device 10d1
	Flags: bus master, 66MHz, medium devsel, latency 0
00: 02 10 8d 43 0f 00 20 02 00 00 01 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
40: 04 00 00 00 f7 ff 43 ff 03 ff 42 01 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 06 0e 00 0f 00 f0 ff ff ff
70: 67 45 23 01 00 00 00 00 01 00 00 00 05 00 00 00
80: 08 00 03 a8 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (prog-if 01 [Subtractive decode])
	Flags: bus master, 66MHz, medium devsel, latency 64
	Bus: primary=00, secondary=0e, subordinate=13, sec-latency=64
	Memory behind bridge: f8600000-f86fffff
00: 02 10 84 43 07 00 a0 02 00 01 04 06 00 40 81 00
10: 00 00 00 00 00 00 00 00 00 0e 13 40 f0 00 80 22
20: 60 f8 60 f8 f0 ff 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00
40: 26 00 3c ff 00 00 00 00 0c 01 3f d1 00 00 00 00
50: 01 00 00 00 08 00 03 a8 00 00 00 00 85 00 ff ff
60: ca 0e 17 00 ba 18 10 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 02 06
e0: 00 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
00: 22 10 00 11 00 00 10 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 80 00 00 00 00 00 00 00 00 00 00 00
40: 01 01 01 00 01 01 01 00 01 01 01 00 01 01 01 00
50: 01 01 01 00 01 01 01 00 01 01 01 00 01 01 01 00
60: 00 00 01 00 e4 00 00 00 20 c8 2f 0f 3c 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 08 00 01 21 20 a8 11 11 22 05 35 80 02 00 00 00
90: 78 01 70 01 00 00 ff 00 07 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel
00: 22 10 01 11 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
40: 03 00 00 00 00 00 3f 00 00 00 00 00 01 00 00 00
50: 00 00 00 00 02 00 00 00 00 00 00 00 03 00 00 00
60: 00 00 00 00 04 00 00 00 00 00 00 00 05 00 00 00
70: 00 00 00 00 06 00 00 00 00 00 00 00 07 00 00 00
80: 03 30 f8 00 00 ff ff 00 03 20 f8 00 80 2f f8 00
90: 03 00 f8 00 00 1f f8 00 03 00 f0 00 80 ff f7 00
a0: 03 0a 00 00 00 0b 00 00 03 00 f0 00 00 ff ef 00
b0: 03 00 e0 00 80 ff ef 00 03 00 40 00 00 ff df 00
c0: 00 00 00 00 00 00 00 00 13 10 00 00 00 f0 0f 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 03 00 00 ff 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel
00: 22 10 02 11 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
40: 01 00 00 00 01 01 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: e0 3e 38 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 46 00 00 00 00 00 00 00
80: 01 00 00 00 00 00 00 00 24 c2 aa 5d 20 13 12 00
90: 20 0c 01 00 5a 80 00 a7 39 00 00 80 00 00 00 00
a0: ef 06 00 5d 00 00 00 00 00 00 00 00 00 00 00 00
b0: 7e b6 b5 e7 7e 00 00 00 9b 7e 1a 3f d8 3f 72 78
c0: 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 44 e0 dc e1 12 11 91 6f 21 17 c1 06 88 24 80 87
e0: a4 e2 76 81 2b 3a b7 af 60 4b 5a 04 0a 70 c9 ad
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel
	Capabilities: [f0] Secure device <?>
	Kernel driver in use: k8temp
	Kernel modules: k8temp
00: 22 10 03 11 00 00 10 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 f0 00 00 00 00 00 00 00 00 00 00 00
40: ff 3b 04 00 40 00 10 0a 00 00 00 00 00 00 00 00
50: 00 00 00 05 08 00 00 00 00 00 00 00 80 01 00 94
60: 88 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 01 32 51 21 40 30 50 00 2a 00 08 1b 21 00 00
80: 31 37 07 33 13 31 13 31 00 00 00 00 00 00 00 00
90: 00 00 00 00 26 56 00 00 50 01 d2 85 00 00 00 00
a0: 9e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 01 a7 0d 00 00 00 00 09 25 26 26 00
e0: 00 00 00 00 24 0e 4f 00 01 17 00 00 00 00 00 00
f0: 0f 00 10 00 00 00 00 00 00 00 00 00 82 0f 04 00

01:05.0 VGA compatible controller: ATI Technologies Inc RS690M [Radeon X1200 Series] (prog-if 00 [VGA controller])
	Subsystem: Fujitsu Siemens Computers Unknown device 111d
	Flags: bus master, fast devsel, latency 64, IRQ 10
	Memory at f0000000 (64-bit, prefetchable) [size=128M]
	Memory at f8300000 (64-bit, non-prefetchable) [size=64K]
	I/O ports at 9000 [size=256]
	Memory at f8200000 (32-bit, non-prefetchable) [size=1M]
	Capabilities: [50] Power Management version 2
	Capabilities: [80] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable-
00: 02 10 1f 79 07 00 10 00 00 00 00 03 08 40 80 00
10: 0c 00 00 f0 00 00 00 00 04 00 30 f8 00 00 00 00
20: 01 90 00 00 00 00 20 f8 00 00 00 00 34 17 1d 11
30: 00 00 00 00 50 00 00 00 00 00 00 00 0a 01 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 1d 11
50: 01 80 02 06 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

01:05.2 Audio device: ATI Technologies Inc Radeon X1200 Series Audio Controller
	Subsystem: ATI Technologies Inc Radeon X1200 Series Audio Controller
	Flags: bus master, fast devsel, latency 64, IRQ 17
	Memory at f8310000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 2
	Capabilities: [60] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable-
	Kernel driver in use: HDA Intel
	Kernel modules: snd-hda-intel
00: 02 10 19 79 06 00 18 00 00 00 03 04 08 40 00 00
10: 04 00 31 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 02 10 19 79
30: 00 00 00 00 50 00 00 00 00 00 00 00 0b 02 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 02 10 19 79
50: 01 60 02 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
	Subsystem: Fujitsu Siemens Computers Unknown device 111d
	Flags: bus master, fast devsel, latency 0, IRQ 2301
	I/O ports at a000 [size=256]
	Memory at f8400000 (64-bit, non-prefetchable) [size=4K]
	Memory at f8000000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at f8020000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable+
	Capabilities: [70] Express Endpoint, MSI 01
	Capabilities: [b0] MSI-X: Enable- Mask- TabSize=2
	Capabilities: [d0] Vital Product Data <?>
	Capabilities: [100] Advanced Error Reporting <?>
	Capabilities: [140] Virtual Channel <?>
	Capabilities: [160] Device Serial Number 81-68-10-ec-00-00-00-00
	Kernel driver in use: r8169
	Kernel modules: r8169
00: ec 10 68 81 07 04 10 00 02 00 00 02 10 00 00 00
10: 01 a0 00 00 00 00 00 00 04 00 40 f8 00 00 00 00
20: 0c 00 00 f8 00 00 00 00 00 00 00 00 34 17 1d 11
30: 00 00 00 00 40 00 00 00 00 00 00 00 05 01 00 00
40: 01 50 c3 ff 08 00 00 00 00 00 00 00 00 00 00 00
50: 05 70 81 00 0c 10 e0 fe 00 00 00 00 81 41 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 b0 01 02 c1 86 64 00 10 20 19 00 11 3c 07 00
80: 42 00 11 10 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 11 d0 01 00 04 00 00 00 04 08 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

08:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
	Subsystem: Fujitsu Siemens Computers Unknown device 111d
	Flags: bus master, fast devsel, latency 0, IRQ 2300
	I/O ports at b000 [size=256]
	Memory at f8500000 (64-bit, non-prefetchable) [size=4K]
	Memory at f8100000 (64-bit, prefetchable) [size=64K]
	[virtual] Expansion ROM at f8120000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable+
	Capabilities: [70] Express Endpoint, MSI 01
	Capabilities: [b0] MSI-X: Enable- Mask- TabSize=2
	Capabilities: [d0] Vital Product Data <?>
	Capabilities: [100] Advanced Error Reporting <?>
	Capabilities: [140] Virtual Channel <?>
	Capabilities: [160] Device Serial Number 81-68-10-ec-00-00-00-00
	Kernel driver in use: r8169
	Kernel modules: r8169
00: ec 10 68 81 07 04 10 00 02 00 00 02 10 00 00 00
10: 01 b0 00 00 00 00 00 00 04 00 50 f8 00 00 00 00
20: 0c 00 10 f8 00 00 00 00 00 00 00 00 34 17 1d 11
30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00
40: 01 50 c3 ff 08 00 00 00 00 00 00 00 00 00 00 00
50: 05 70 81 00 0c 30 e0 fe 00 00 00 00 89 41 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 b0 01 02 c1 86 64 00 10 20 19 00 11 3c 07 00
80: 42 00 11 10 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 11 d0 01 00 04 00 00 00 04 08 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0e:04.0 FireWire (IEEE 1394): Agere Systems FW323 (rev 70) (prog-if 10 [OHCI])
	Subsystem: Agere Systems FW323
	Flags: bus master, fast Back2Back, medium devsel, latency 96, IRQ 23
	Memory at f8600000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Power Management version 2
	Kernel driver in use: firewire_ohci
	Kernel modules: firewire-ohci
00: c1 11 11 58 16 02 90 02 70 10 00 0c 08 60 00 00
10: 00 00 60 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 c1 11 11 58
30: 00 00 00 00 44 00 00 00 00 00 00 00 0a 01 0c 18
40: 00 00 00 00 01 00 02 7e 00 80 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Re: [patch 6/6] x86: add c1e aware idle function

[Andrew Morton]

On Thu, 12 Jun 2008 10:29:00 -0000
Thomas Gleixner <tglx@linutronix.de> wrote:

> +static void c1e_idle(void)
> +{
> +	static cpumask_t c1e_mask = CPU_MASK_NONE;
> +	static int c1e_detected;
> +
> +	if (need_resched())
> +		return;
> +
> +	if (!c1e_detected) {
> +		u32 lo, hi;
> +
> +		rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
> +		if (lo & K8_INTP_C1E_ACTIVE_MASK) {
> +			c1e_detected = 1;
> +			mark_tsc_unstable("TSC halt in C1E");
> +			printk(KERN_INFO "System has C1E enabled\n");
> +		}
> +	}
> +
> +	if (c1e_detected) {
> +		int cpu = smp_processor_id();
> +
> +		if (!cpu_isset(cpu, c1e_mask)) {
> +			cpu_set(cpu, c1e_mask);
> +			/* Force broadcast so ACPI can not interfere */
> +			clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_FORCE,
> +					   &cpu);
> +			printk(KERN_INFO "Switch to broadcast mode on CPU%d\n",
> +			       cpu);
> +		}
> +		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
> +		default_idle();
> +		local_irq_disable();
> +		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu);
> +		local_irq_enable();

I worked it out!  It took me a while.

This function is called with local irqs disabled.

default_idle() is entered with local irqs disabled but returns with them
enabled.

clockevents_notify() is supposed to be called with local irqs disabled.

This functions returns with local irqs enabled.

Was I right?  None of any of that is documented anywhere.  But it should
be.


> +	} else
> +		default_idle();
> +}

Re: [patch 6/6] x86: add c1e aware idle function

[Thomas Gleixner]

On Thu, 12 Jun 2008, Andrew Morton wrote:
> On Thu, 12 Jun 2008 10:29:00 -0000
> Thomas Gleixner <tglx@linutronix.de> wrote:
> > +		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
> > +		default_idle();
> > +		local_irq_disable();
> > +		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu);
> > +		local_irq_enable();
> 
> I worked it out!  It took me a while.
> 
> This function is called with local irqs disabled.
> 
> default_idle() is entered with local irqs disabled but returns with them
> enabled.
> 
> clockevents_notify() is supposed to be called with local irqs disabled.
> 
> This functions returns with local irqs enabled.

Damn, you decoded my sekrit.
 
> Was I right?  None of any of that is documented anywhere.  But it should
> be.

Yeah, needs a big comment. Will add one.

Thanks,

	tglx

Re: [patch 3/6] x86: use cpuinfo to check for interrupt pending message msr

[Andreas Herrmann]

On Thu, Jun 12, 2008 at 10:28:47AM -0000, Thomas Gleixner wrote:
> No need to do a cpuid(1) again. The cpuinfo structure has all
> necessary information already.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Andreas Herrmann <andreas.herrmann3@amd.com>
> 
> ---
>  arch/x86/kernel/cpu/amd.c  |   41 +++++++++++++++--------------------------
>  arch/x86/kernel/setup_64.c |   38 +++++++++++++++-----------------------
>  2 files changed, 30 insertions(+), 49 deletions(-)
> 
> Index: linux-2.6/arch/x86/kernel/cpu/amd.c
> ===================================================================
> --- linux-2.6.orig/arch/x86/kernel/cpu/amd.c
> +++ linux-2.6/arch/x86/kernel/cpu/amd.c
> @@ -25,35 +25,24 @@ extern void vide(void);
>  __asm__(".align 4\nvide: ret");
>  
>  #ifdef CONFIG_X86_LOCAL_APIC
> -#define CPUID_PROCESSOR_SIGNATURE       1
> -#define CPUID_XFAM              0x0ff00000
> -#define CPUID_XFAM_K8           0x00000000
> -#define CPUID_XFAM_10H          0x00100000
> -#define CPUID_XFAM_11H          0x00200000
> -#define CPUID_XMOD              0x000f0000
> -#define CPUID_XMOD_REV_F        0x00040000
>  
>  /* AMD systems with C1E don't have a working lAPIC timer. Check for that. */
> -static __cpuinit int amd_apic_timer_broken(void)
> +static __cpuinit int amd_apic_timer_broken(struct cpuinfo_x86 *c)
>  {
>  	u32 lo, hi;
> -	u32 eax = cpuid_eax(CPUID_PROCESSOR_SIGNATURE);
> -	switch (eax & CPUID_XFAM) {
> -	case CPUID_XFAM_K8:
> -		if ((eax & CPUID_XMOD) < CPUID_XMOD_REV_F)
> -			break;
> -	case CPUID_XFAM_10H:
> -	case CPUID_XFAM_11H:
> -		rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
> -		if (lo & K8_INTP_C1E_ACTIVE_MASK) {
> -			if (smp_processor_id() != boot_cpu_physical_apicid)
> -				printk(KERN_INFO "AMD C1E detected late. "
> -				       "	Force timer broadcast.\n");
> -			return 1;
> -		}
> -		break;
> -	default:
> -		/* err on the side of caution */
> +
> +	if (c->x86 < 0x0F)
> +		return 0;
> +
> +	/* Family 0x0f models < rev F do not have this MSR */
> +	if (c->x86 == 0x0f && c->x86_model < 0x40)
> +		return 0;

Just some minor nitpicking.
Older AMD family 0xf CPUs have this Interrupt Pending Message
Register. But they do not support C1E and thus bits 27 and 28 of this
MSR are reserved.


Andreas

Re: [patch 6/6] x86: add c1e aware idle function

[Andrew Morton]

On Fri, 13 Jun 2008 08:02:30 +0200 (CEST) Thomas Gleixner <tglx@linutronix.de> wrote:

> On Thu, 12 Jun 2008, Andrew Morton wrote:
> > On Thu, 12 Jun 2008 10:29:00 -0000
> > Thomas Gleixner <tglx@linutronix.de> wrote:
> > > +		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
> > > +		default_idle();
> > > +		local_irq_disable();
> > > +		clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu);
> > > +		local_irq_enable();
> > 
> > I worked it out!  It took me a while.
> > 
> > This function is called with local irqs disabled.
> > 
> > default_idle() is entered with local irqs disabled but returns with them
> > enabled.
> > 
> > clockevents_notify() is supposed to be called with local irqs disabled.
> > 
> > This functions returns with local irqs enabled.
> 
> Damn, you decoded my sekrit.

The wonders of the GPL.

> > Was I right?  None of any of that is documented anywhere.  But it should
> > be.
> 
> Yeah, needs a big comment. Will add one.
> 

Actually it need lots of little comments.  One at default_idle(), one at
clockevents_notify(), one at whatever-this-function-was.

Because "must be called with local irqs disabled" is as much a part of
a function's interface as "must be passed a foo* and an integer and
returns a bar*".  Ditto "must be called under zot_lock".  But people
often forget these things.

I guess kerneldoc should have a standard "call environment" template. 
Maybe it does, dunno.

Of course, WARN_ON(!irqs_disabled()) and WARN_ON(!spin_is_locked(foo))
is robust documentation.

Re: [patch 0/6] AMD C1E aware idle support

[Rafael J. Wysocki]

On Friday, 13 of June 2008, Rafael J. Wysocki wrote:
> On Thursday, 12 of June 2008, Ingo Molnar wrote:
> > 
> > * Rafael J. Wysocki <rjw@sisk.pl> wrote:
> > 
> > > > http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
> > > > or
> > > > http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2
> > > 
> > > I guess the x86-next-apic-timer-fixes.patch is a combination of all 
> > > patches in this series.  Is that correct?
> > 
> > you can also try out those patches via running tip/master. (the patches 
> > from Thomas are queued up in the tip/x86/cpu topic branch)
> 
> I tested this one eventually. :-)
> 
> It seems to work quite well on the nx6325 that caused us so many headaches in
> the past, even suspend to RAM doesn't break which is a good sign.

Well, unfortunately, an attempt to suspend to RAM after a successful
hibernation-restore cycle finally hanged the machine. :-(

Thanks,
Rafael

Re: [patch 3/6] x86: use cpuinfo to check for interrupt pending message msr

[Thomas Gleixner]

On Fri, 13 Jun 2008, Andreas Herrmann wrote:
> > +	/* Family 0x0f models < rev F do not have this MSR */
> > +	if (c->x86 == 0x0f && c->x86_model < 0x40)
> > +		return 0;
> 
> Just some minor nitpicking.
> Older AMD family 0xf CPUs have this Interrupt Pending Message
> Register. But they do not support C1E and thus bits 27 and 28 of this
> MSR are reserved.

So the check can be simplified to always check the MSR for all 
family >= 0x0f CPUs ?

Thanks,

	tglx

Re: [patch 3/6] x86: use cpuinfo to check for interrupt pending message msr

[Andreas Herrmann]

On Fri, Jun 13, 2008 at 02:38:30PM +0200, Thomas Gleixner wrote:
> On Fri, 13 Jun 2008, Andreas Herrmann wrote:
> > > +	/* Family 0x0f models < rev F do not have this MSR */
> > > +	if (c->x86 == 0x0f && c->x86_model < 0x40)
> > > +		return 0;
> > 
> > Just some minor nitpicking.
> > Older AMD family 0xf CPUs have this Interrupt Pending Message
> > Register. But they do not support C1E and thus bits 27 and 28 of this
> > MSR are reserved.
> 
> So the check can be simplified to always check the MSR for all 
> family >= 0x0f CPUs ?

First of all I thought of changing the comment.

But now that you ask:
Documentation for older K8 CPUs says that reserved bits in that MSR
are "Read as Zero". But otherwise it also says "Software must not
depend on the state of a reserved field ..."

Maybe I am a little paranoid but I would keep the model check.


Andreas

Re: [patch 0/6] AMD C1E aware idle support

[Maciej W. Rozycki]

On Thu, 12 Jun 2008, Thomas Gleixner wrote:

> @Macej: I bisected your patches and the commit which solves the mysterious
> hangs is:
> 
> x86: I/O APIC: timer through 8259A second-chance
> (7e3530cd98a0c6ab38f5898e855a5beffab26561 in linux-2.6-tip.git)
> 
> That's the patch which you worried about possible impacts, but it
> seems that it actually fixes the stupid timer irq issue finally. I
> have tested it on various machines which had timer irq problems in the
> past and they all run smothly. Great work!

 Well, I did hope at least some impact would be positive!  I am glad this 
is the case and thanks for your report as well as appreciation.

  Maciej

Re: [patch 6/6] x86: add c1e aware idle function

[Pavel Machek]

On Thu 2008-06-12 10:29:00, Thomas Gleixner wrote:
> C1E on AMD machines is like C3 but without control from the OS. Up to
> now we disabled the local apic timer for those machines as it stops
> when the CPU goes into C1E. This excludes those machines from high
> resolution timers / dynamic ticks, which hurts especially the X2 based
> laptops.
> 
> The current boot time C1E detection has another more serious flaw:
> some BIOSes do not enable C1E until the ACPI processor module is
> loaded. This causes systems to stop working after that point.
> 
> To work nicely with C1E enabled machines we use a separate idle
> function, which checks on idle entry whether C1E was enabled in the
> Interrupt Pending Message MSR. This allows us to do timer broadcasting

Entering idle is quite a common operation, and reading MSR is quite
slow. Is it possible to do better here?

What happens if ACPI BIOS toggles MSR on all cpus *while* we are
entering idle? This seems inherently racy...
							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [patch 6/6] x86: add c1e aware idle function

[Rafael J. Wysocki]

On Wednesday, 18 of June 2008, Pavel Machek wrote:
> On Thu 2008-06-12 10:29:00, Thomas Gleixner wrote:
> > C1E on AMD machines is like C3 but without control from the OS. Up to
> > now we disabled the local apic timer for those machines as it stops
> > when the CPU goes into C1E. This excludes those machines from high
> > resolution timers / dynamic ticks, which hurts especially the X2 based
> > laptops.
> > 
> > The current boot time C1E detection has another more serious flaw:
> > some BIOSes do not enable C1E until the ACPI processor module is
> > loaded. This causes systems to stop working after that point.
> > 
> > To work nicely with C1E enabled machines we use a separate idle
> > function, which checks on idle entry whether C1E was enabled in the
> > Interrupt Pending Message MSR. This allows us to do timer broadcasting
> 
> Entering idle is quite a common operation, and reading MSR is quite
> slow. Is it possible to do better here?
> 
> What happens if ACPI BIOS toggles MSR on all cpus *while* we are
> entering idle? This seems inherently racy...

Yes, and that fits the picture I'm observing on the nx6325 (see the
"linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).

Thanks,
Rafael

Re: [patch 6/6] x86: add c1e aware idle function

[Thomas Gleixner]

On Wed, 18 Jun 2008, Rafael J. Wysocki wrote:
> On Wednesday, 18 of June 2008, Pavel Machek wrote:
> > On Thu 2008-06-12 10:29:00, Thomas Gleixner wrote:
> > > C1E on AMD machines is like C3 but without control from the OS. Up to
> > > now we disabled the local apic timer for those machines as it stops
> > > when the CPU goes into C1E. This excludes those machines from high
> > > resolution timers / dynamic ticks, which hurts especially the X2 based
> > > laptops.
> > > 
> > > The current boot time C1E detection has another more serious flaw:
> > > some BIOSes do not enable C1E until the ACPI processor module is
> > > loaded. This causes systems to stop working after that point.
> > > 
> > > To work nicely with C1E enabled machines we use a separate idle
> > > function, which checks on idle entry whether C1E was enabled in the
> > > Interrupt Pending Message MSR. This allows us to do timer broadcasting
> > 
> > Entering idle is quite a common operation, and reading MSR is quite
> > slow. Is it possible to do better here?
> > 
> > What happens if ACPI BIOS toggles MSR on all cpus *while* we are
> > entering idle? This seems inherently racy...

Well, on most of the machines we see the C1E bit on the first idle
entry either on CPU0 or on CPU1.

I know of exactly one machine which has the C1E thing enabled late
when the ACPI stuff runs.
 
> Yes, and that fits the picture I'm observing on the nx6325 (see the
> "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).

Hmm, when is the C1E detected on current mainline ? Does the boot CPU
have it or is it when the second CPU comes up ?

Thanks,
	tglx

Re: [patch 6/6] x86: add c1e aware idle function

[Rafael J. Wysocki]

On Wednesday, 18 of June 2008, Thomas Gleixner wrote:
> On Wed, 18 Jun 2008, Rafael J. Wysocki wrote:
> > On Wednesday, 18 of June 2008, Pavel Machek wrote:
> > > On Thu 2008-06-12 10:29:00, Thomas Gleixner wrote:
> > > > C1E on AMD machines is like C3 but without control from the OS. Up to
> > > > now we disabled the local apic timer for those machines as it stops
> > > > when the CPU goes into C1E. This excludes those machines from high
> > > > resolution timers / dynamic ticks, which hurts especially the X2 based
> > > > laptops.
> > > > 
> > > > The current boot time C1E detection has another more serious flaw:
> > > > some BIOSes do not enable C1E until the ACPI processor module is
> > > > loaded. This causes systems to stop working after that point.
> > > > 
> > > > To work nicely with C1E enabled machines we use a separate idle
> > > > function, which checks on idle entry whether C1E was enabled in the
> > > > Interrupt Pending Message MSR. This allows us to do timer broadcasting
> > > 
> > > Entering idle is quite a common operation, and reading MSR is quite
> > > slow. Is it possible to do better here?
> > > 
> > > What happens if ACPI BIOS toggles MSR on all cpus *while* we are
> > > entering idle? This seems inherently racy...
> 
> Well, on most of the machines we see the C1E bit on the first idle
> entry either on CPU0 or on CPU1.
> 
> I know of exactly one machine which has the C1E thing enabled late
> when the ACPI stuff runs.
>  
> > Yes, and that fits the picture I'm observing on the nx6325 (see the
> > "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).
> 
> Hmm, when is the C1E detected on current mainline ? Does the boot CPU
> have it or is it when the second CPU comes up ?

How can I check that?

Rafael

Re: [patch 6/6] x86: add c1e aware idle function

[Thomas Gleixner]

On Thu, 19 Jun 2008, Rafael J. Wysocki wrote:
> On Wednesday, 18 of June 2008, Thomas Gleixner wrote:
> > On Wed, 18 Jun 2008, Rafael J. Wysocki wrote:
> > > On Wednesday, 18 of June 2008, Pavel Machek wrote:
> > > > On Thu 2008-06-12 10:29:00, Thomas Gleixner wrote:
> > > > > C1E on AMD machines is like C3 but without control from the OS. Up to
> > > > > now we disabled the local apic timer for those machines as it stops
> > > > > when the CPU goes into C1E. This excludes those machines from high
> > > > > resolution timers / dynamic ticks, which hurts especially the X2 based
> > > > > laptops.
> > > > > 
> > > > > The current boot time C1E detection has another more serious flaw:
> > > > > some BIOSes do not enable C1E until the ACPI processor module is
> > > > > loaded. This causes systems to stop working after that point.
> > > > > 
> > > > > To work nicely with C1E enabled machines we use a separate idle
> > > > > function, which checks on idle entry whether C1E was enabled in the
> > > > > Interrupt Pending Message MSR. This allows us to do timer broadcasting
> > > > 
> > > > Entering idle is quite a common operation, and reading MSR is quite
> > > > slow. Is it possible to do better here?
> > > > 
> > > > What happens if ACPI BIOS toggles MSR on all cpus *while* we are
> > > > entering idle? This seems inherently racy...
> > 
> > Well, on most of the machines we see the C1E bit on the first idle
> > entry either on CPU0 or on CPU1.
> > 
> > I know of exactly one machine which has the C1E thing enabled late
> > when the ACPI stuff runs.
> >  
> > > Yes, and that fits the picture I'm observing on the nx6325 (see the
> > > "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).
> > 
> > Hmm, when is the C1E detected on current mainline ? Does the boot CPU
> > have it or is it when the second CPU comes up ?
> 
> How can I check that?

When the boot cpu has it then it prints:

"Disabling APIC timer"

when the secondary CPU has it then it prints:

"AMD C1E detected late. Force timer broadcast."

Current mainline does not check for the late ACPI case so I'm pretty
sure that on your box it is detected during early boot way before we
can switch to highres/dyntick.

Thanks,

	tglx

Re: [patch 6/6] x86: add c1e aware idle function

[Rafael J. Wysocki]

On Thursday, 19 of June 2008, Thomas Gleixner wrote:
> On Thu, 19 Jun 2008, Rafael J. Wysocki wrote:
> > On Wednesday, 18 of June 2008, Thomas Gleixner wrote:
> > > On Wed, 18 Jun 2008, Rafael J. Wysocki wrote:
> > > > On Wednesday, 18 of June 2008, Pavel Machek wrote:
> > > > > On Thu 2008-06-12 10:29:00, Thomas Gleixner wrote:
> > > > > > C1E on AMD machines is like C3 but without control from the OS. Up to
> > > > > > now we disabled the local apic timer for those machines as it stops
> > > > > > when the CPU goes into C1E. This excludes those machines from high
> > > > > > resolution timers / dynamic ticks, which hurts especially the X2 based
> > > > > > laptops.
> > > > > > 
> > > > > > The current boot time C1E detection has another more serious flaw:
> > > > > > some BIOSes do not enable C1E until the ACPI processor module is
> > > > > > loaded. This causes systems to stop working after that point.
> > > > > > 
> > > > > > To work nicely with C1E enabled machines we use a separate idle
> > > > > > function, which checks on idle entry whether C1E was enabled in the
> > > > > > Interrupt Pending Message MSR. This allows us to do timer broadcasting
> > > > > 
> > > > > Entering idle is quite a common operation, and reading MSR is quite
> > > > > slow. Is it possible to do better here?
> > > > > 
> > > > > What happens if ACPI BIOS toggles MSR on all cpus *while* we are
> > > > > entering idle? This seems inherently racy...
> > > 
> > > Well, on most of the machines we see the C1E bit on the first idle
> > > entry either on CPU0 or on CPU1.
> > > 
> > > I know of exactly one machine which has the C1E thing enabled late
> > > when the ACPI stuff runs.
> > >  
> > > > Yes, and that fits the picture I'm observing on the nx6325 (see the
> > > > "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).
> > > 
> > > Hmm, when is the C1E detected on current mainline ? Does the boot CPU
> > > have it or is it when the second CPU comes up ?
> > 
> > How can I check that?
> 
> When the boot cpu has it then it prints:
> 
> "Disabling APIC timer"

It does that.

> when the secondary CPU has it then it prints:
> 
> "AMD C1E detected late. Force timer broadcast."
> 
> Current mainline does not check for the late ACPI case so I'm pretty
> sure that on your box it is detected during early boot way before we
> can switch to highres/dyntick.

Yes, that's the case.

Thanks,
Rafael

Re: [patch 0/6] AMD C1E aware idle support

[Len Brown]

On Thu, 12 Jun 2008, Thomas Gleixner wrote:

> AMD CPUs with C1E support are currently excluded from high resolution
> timers and NOHZ support. The reason is that C1E is a BIOS controlled
> C3 power state which switches off TSC and the local APIC timer. The
> ACPI C-State control manages the TSC/local APIC timer wreckage, but
> this does not include the C1 based ("halt" instruction) C1E mode. The
> BIOS/SMM controlled C1E state works on most systems even without
> enabling ACPI C-State control.

What a mess.

What is the measured power savings that justifies this effort?

While I'm okay with platform specific idle states,
I'm not okay with the use of therm C1E here.

C1E has been shipped for many years on Intel processors
and it is completely transparent to the OS.

If AMD now has their own C1E and it breaks the OS,
please call is something like AMD_C1E to make it
totally clear in shared files like process.c
that consulting that variable or running that routine
on Intel hardware would be a Linux bug.

thanks
-Len

Re: [patch 0/6] AMD C1E aware idle support

[Andreas Herrmann]

On Wed, Jun 18, 2008 at 06:47:49PM -0400, Len Brown wrote:
> On Thu, 12 Jun 2008, Thomas Gleixner wrote:
> 
> > AMD CPUs with C1E support are currently excluded from high resolution
> > timers and NOHZ support. The reason is that C1E is a BIOS controlled
> > C3 power state which switches off TSC and the local APIC timer. The
> > ACPI C-State control manages the TSC/local APIC timer wreckage, but
> > this does not include the C1 based ("halt" instruction) C1E mode. The
> > BIOS/SMM controlled C1E state works on most systems even without
> > enabling ACPI C-State control.
> 
> What a mess.
> 
> What is the measured power savings that justifies this effort?

IMHO the power savings are not that important when such a kernel runs
on bare metal:

(In the following C1E denotes "AMD-C1E" of course ;-)

- From what I've seen in some basic tests, power usage was not that
  different between C1E+NOHZ and C1E+250Hz. (I've done tests on a
  Turion X2 (RevF) Laptop, a desktop with Phenom (RevB3), and some
  Turion X2 Ultra (family 0x11) prototype.)

- Just the desktop showed 2-4 Watt larger power usage with C1E+250Hz
  in comparison to C1E+NOHZ.

- The Turion X2 (Ultra) systems showed almost no differences. Just
  when running with C1E+1000Hz the ACPI power usage estimate reported
  by powertop was slightly increased (by <1 Watt). (No differences
  shown on external power meter and removed battery.)

In conclusion, I think the big benefit from that code is not improved
power savings for laptop users but that it allows highres timers and
dynticks for CPUs with C1E.

E.g. this helps a lot with virtualization -- where dynticks is
preferred for the host system.
And if you have lots of guests the number of timer interrupts can
significantly be reduced for the entire system when all guests and the
host use dynticks.

Some further (random) observations/notes from my tests:

- Using C1E (NOHZ or with periodic timer) was usually better than
  running w/o C1E on those machines. (Ok, I didn't expect anything
  else.)

- It did not matter which timer interrupt source was used (PIT or
  HPET) in NOHZ/one-shot mode.

  + One-shot programming of HPET in comparison to PIT was faster (on
    average) by a factor of 2.7 with the chipsets that I've used.

  + Of course using PIT in NOHZ mode resulted in sligtly more
    interrupts as the 16-bit counter overflows at least every 0.055
    seconds - and Linux even uses a maximum delta of about 27ms.

  But overall no measurable difference in power usage was seen.

- "wakeups from idle" as reported by powertop are somehow inaccurate
  (from my point of view). They are calculated as "number of
  interrupts"/"number of online CPUs". Thus powertop shows about 125
  wakeups with 250 HZ on any dual core system or about 250 wakeups
  with 1000Hz on a quad-core system.  But I had no time to look
  further into this.

- Doing battery tests where you fully charge the battery and then
  deeply discharge it are harmful, it will/might decrease full
  capacity of your lithium battery and most probably decrease its
  lifetime. ;-(

> While I'm okay with platform specific idle states,
> I'm not okay with the use of therm C1E here.

Valid point.
I try to address this with some patch.


Regards,

Andreas

[PATCH] x86: emphasize that c1e aware idle stuff is AMD specific

[Andreas Herrmann]

This is to clarify that AMD C1E has nothing to do with Intel's C1E or
C2E.

Furthermore I changed some comment to clarify that currently just AMD
family 0x10 CPUs support MWAIT. AMD family 0x11 has quite similar
power management features like family 0x10 but it has no MONITOR/MWAIT
instructions.

Signed-off-by: Andreas Herrmann <andreas.herrmann3@amd.com>
---
Patch is against current tip/master.
(git describe says: tip-history-2008-06-30_08.08_Mon-441-g7782052
whatever that means ;-)

Regards,
Andreas


 arch/x86/kernel/process.c |   36 ++++++++++++++++++------------------
 1 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index beef5e1..f6da61a 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -189,12 +189,12 @@ static void poll_idle(void)
 /*
  * mwait selection logic:
  *
- * It depends on the CPU. For AMD CPUs that support MWAIT this is
- * wrong. Family 0x10 and 0x11 CPUs will enter C1 on HLT. Powersavings
- * then depend on a clock divisor and current Pstate of the core. If
- * all cores of a processor are in halt state (C1) the processor can
- * enter the C1E (C1 enhanced) state. If mwait is used this will never
- * happen.
+ * It depends on the CPU. For AMD CPUs that support MWAIT - currently
+ * AMD family 0x10 - this is wrong. Those CPUs will enter C1 on
+ * HLT. Powersavings then depend on a clock divisor and current Pstate
+ * of the core. If all cores of a processor are in halt state (C1) the
+ * processor can enter the AMD C1E (C1 enhanced) state. If mwait is
+ * used this will never happen.
  *
  * idle=mwait overrides this decision and forces the usage of mwait.
  */
@@ -219,16 +219,16 @@ static int __cpuinit mwait_usable(const struct cpuinfo_x86 *c)
 		return 1;
 
 	/*
-	 * edx enumeratios MONITOR/MWAIT extensions. Check, whether
-	 * C1  supports MWAIT
+	 * EDX enumerates MONITOR/MWAIT extensions. Check, whether an
+	 * C1 sub-state is supported using MONITOR/MWAIT.
 	 */
 	return (edx & MWAIT_EDX_C1);
 }
 
 /*
- * Check for AMD CPUs, which have potentially C1E support
+ * Check for AMD CPUs, which have potentially AMD's C1E support
  */
-static int __cpuinit check_c1e_idle(const struct cpuinfo_x86 *c)
+static int __cpuinit check_amd_c1e_idle(const struct cpuinfo_x86 *c)
 {
 	if (c->x86_vendor != X86_VENDOR_AMD)
 		return 0;
@@ -236,7 +236,7 @@ static int __cpuinit check_c1e_idle(const struct cpuinfo_x86 *c)
 	if (c->x86 < 0x0F)
 		return 0;
 
-	/* Family 0x0f models < rev F do not have C1E */
+	/* Family 0x0f models < rev F do not have AMD C1E */
 	if (c->x86 == 0x0f && c->x86_model < 0x40)
 		return 0;
 
@@ -244,11 +244,11 @@ static int __cpuinit check_c1e_idle(const struct cpuinfo_x86 *c)
 }
 
 /*
- * C1E aware idle routine. We check for C1E active in the interrupt
+ * AMD C1E aware idle routine. We check for C1E active in the interrupt
  * pending message MSR. If we detect C1E, then we handle it the same
  * way as C3 power states (local apic timer and TSC stop)
  */
-static void c1e_idle(void)
+static void amd_c1e_idle(void)
 {
 	static cpumask_t c1e_mask = CPU_MASK_NONE;
 	static int c1e_detected;
@@ -262,8 +262,8 @@ static void c1e_idle(void)
 		rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
 		if (lo & K8_INTP_C1E_ACTIVE_MASK) {
 			c1e_detected = 1;
-			mark_tsc_unstable("TSC halt in C1E");
-			printk(KERN_INFO "System has C1E enabled\n");
+			mark_tsc_unstable("TSC halts in AMD C1E");
+			printk(KERN_INFO "System has AMD C1E enabled\n");
 		}
 	}
 
@@ -316,9 +316,9 @@ void __cpuinit select_idle_routine(const struct cpuinfo_x86 *c)
 		 */
 		printk(KERN_INFO "using mwait in idle threads.\n");
 		pm_idle = mwait_idle;
-	} else if (check_c1e_idle(c)) {
-		printk(KERN_INFO "using C1E aware idle routine\n");
-		pm_idle = c1e_idle;
+	} else if (check_amd_c1e_idle(c)) {
+		printk(KERN_INFO "using AMD C1E aware idle routine\n");
+		pm_idle = amd_c1e_idle;
 	} else
 		pm_idle = default_idle;
 }
-- 
1.5.6.1

Re: [patch 0/6] AMD C1E aware idle support

[Pavel Machek]

Hi!

> > > AMD CPUs with C1E support are currently excluded from high resolution
> > > timers and NOHZ support. The reason is that C1E is a BIOS controlled
> > > C3 power state which switches off TSC and the local APIC timer. The
> > > ACPI C-State control manages the TSC/local APIC timer wreckage, but
> > > this does not include the C1 based ("halt" instruction) C1E mode. The
> > > BIOS/SMM controlled C1E state works on most systems even without
> > > enabling ACPI C-State control.
> > 
> > What a mess.

Yep, seems like AMD is breaking C1 semantics. Is it even valid from
ACPI spec point of view?

> > What is the measured power savings that justifies this effort?
> 
> IMHO the power savings are not that important when such a kernel runs
> on bare metal:

Ok, so maybe we should disable C1E to work around its misdesign?

It would be certainly nice to have noc1e command line option...

>   But overall no measurable difference in power usage was seen.

							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [patch 0/6] AMD C1E aware idle support

[Andreas Herrmann]

On Tue, Aug 05, 2008 at 07:42:18PM +0200, Pavel Machek wrote:
> > > > AMD CPUs with C1E support are currently excluded from high resolution
> > > > timers and NOHZ support. The reason is that C1E is a BIOS controlled
> > > > C3 power state which switches off TSC and the local APIC timer. The
> > > > ACPI C-State control manages the TSC/local APIC timer wreckage, but
> > > > this does not include the C1 based ("halt" instruction) C1E mode. The
> > > > BIOS/SMM controlled C1E state works on most systems even without
> > > > enabling ACPI C-State control.
> > > 
> > > What a mess.
> 
> Yep, seems like AMD is breaking C1 semantics. Is it even valid from
> ACPI spec point of view?
>
> > > What is the measured power savings that justifies this effort?
> > 
> > IMHO the power savings are not that important when such a kernel runs
> > on bare metal:
> 
> Ok, so maybe we should disable C1E to work around its misdesign?

Disabling C1E is not an option. It saves the most power when you have
a multicore AMD CPU. Neither C2 nor C3 are declared here. AMD C1E is
hardware level power management if all cores are in the C1 state.

If you disable C1E and use a NOHZ kernel your power consumption is
higher than using C1E with a periodic timer (say 250 HZ).

> It would be certainly nice to have noc1e command line option...

The sane way to disable it is in the BIOS -- if your BIOS provides
such an option.
Thomas pointed already out that C1E might be enabled later during
boot. So you would have to reset the respective bits whenever you
enter idle. This is ugly and what would be the benefit of this?


Andreas