[PATCH 0/9] x86: TSC calibration update

[PATCH 0/9] x86: TSC calibration update

[Len Brown]

cpu_khz and tsc_khz initialization can be unreliable and expensive.
They are initialized in tsc_init()/native_calibrate_tsc(), which prints:

pr_info("Detected %lu.%03lu MHz processor\n", cpu_khz...)

native_calibrate_cpu() first tries quick_pit_calibrate(),
which can take over 50.0M cycles to succeed,
or as few as 2.4M cycles to fail.

On failure, pit_calibrate_tsc() is attempted, which can succeed
in as few as 20M cycles, but may consume over 240M cycles
before it fails.

By comparison, on many processors, tsc frequency can be discovered by
table and MSR or CPUID in under 0.002M cycles.

Subsequently tsc_refine_calibration_work() checks our work,
but it takes under 0.004M cycles.

pr_info("Refined TSC clocksource calibration: %lu.%03lu MHz\n", tsc_khz...)

Finally, CPU and TSC frequency are not guaranteed to be identical,
and this series allows cpu_khz and tsc_khz to differ
within a few percent.

cheers,
Len Brown, Intel Open Source Technology Center

this patch set is also available via git:

git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux.git x86


Bin Gao (1):
      x86 tsc: enumerate BXT tsc_khz via CPUID

Len Brown (8):
      x86 tsc_msr: Identify Intel-specific code
      x86 tsc_msr: Remove debugging messages
      x86 tsc_msr: Update comments, expand definitions
      x86 tsc_msr: Correct Silvermont reference clock values
      x86 tsc_msr: Add Airmont reference clock values
      x86 tsc_msr: Extend to include Intel Core Architecture
      x86 tsc_msr: Remove irqoff around MSR-based TSC enumeration
      x86 tsc: enumerate SKL cpu_khz and tsc_khz via CPUID

 arch/x86/include/asm/tsc.h      |   4 +-
 arch/x86/include/asm/x86_init.h |   4 +-
 arch/x86/kernel/tsc.c           |  96 ++++++++++++++++++++++++++++++----
 arch/x86/kernel/tsc_msr.c       | 112 ++++++++++++++++++++++------------------
 arch/x86/kernel/x86_init.c      |   1 +
 5 files changed, 152 insertions(+), 65 deletions(-)

[PATCH 1/9] x86 tsc_msr: Identify Intel-specific code

[Len Brown]

From: Len Brown <len.brown@intel.com>

try_msr_calibrate_tsc() is currently Intel-specific,
and should not execute on any other vendor's parts.

Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/kernel/tsc_msr.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index 92ae6ac..c16e35b 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -86,6 +86,9 @@ unsigned long try_msr_calibrate_tsc(void)
 	unsigned long res;
 	int cpu_index;
 
+	if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
+		return 0;
+
 	cpu_index = match_cpu(boot_cpu_data.x86, boot_cpu_data.x86_model);
 	if (cpu_index < 0)
 		return 0;
-- 
2.8.0.rc4.16.g56331f8

[PATCH 2/9] x86 tsc_msr: Remove debugging messages

[Len Brown]

From: Len Brown <len.brown@intel.com>

Debugging messages are not necessary after all of the
possible hardware failures that never occur.
Instead, this code can simply return 0.

This code also doesn't need to print in the success case.
tsc_init() already prints the TSC frequency,
and apic=debug is available if anybody really is
interested in printing the LAPIC frequency.

Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/kernel/tsc_msr.c | 19 +++----------------
 1 file changed, 3 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index c16e35b..d460ef1 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -76,9 +76,10 @@ static int match_cpu(u8 family, u8 model)
 	(freq_desc_tables[cpu_index].freqs[freq_id])
 
 /*
- * Do MSR calibration only for known/supported CPUs.
+ * MSR-based CPU/TSC frequency discovery for certain CPUs.
  *
- * Returns the calibration value or 0 if MSR calibration failed.
+ * Set global "lapic_timer_frequency" to bus_clock_cycles/jiffy
+ * Return processor base frequency in KHz, or 0 on failure.
  */
 unsigned long try_msr_calibrate_tsc(void)
 {
@@ -100,31 +101,17 @@ unsigned long try_msr_calibrate_tsc(void)
 		rdmsr(MSR_IA32_PERF_STATUS, lo, hi);
 		ratio = (hi >> 8) & 0x1f;
 	}
-	pr_info("Maximum core-clock to bus-clock ratio: 0x%x\n", ratio);
-
-	if (!ratio)
-		goto fail;
 
 	/* Get FSB FREQ ID */
 	rdmsr(MSR_FSB_FREQ, lo, hi);
 	freq_id = lo & 0x7;
 	freq = id_to_freq(cpu_index, freq_id);
-	pr_info("Resolved frequency ID: %u, frequency: %u KHz\n",
-				freq_id, freq);
-	if (!freq)
-		goto fail;
 
 	/* TSC frequency = maximum resolved freq * maximum resolved bus ratio */
 	res = freq * ratio;
-	pr_info("TSC runs at %lu KHz\n", res);
 
 #ifdef CONFIG_X86_LOCAL_APIC
 	lapic_timer_frequency = (freq * 1000) / HZ;
-	pr_info("lapic_timer_frequency = %d\n", lapic_timer_frequency);
 #endif
 	return res;
-
-fail:
-	pr_warn("Fast TSC calibration using MSR failed\n");
-	return 0;
 }
-- 
2.8.0.rc4.16.g56331f8

[PATCH 3/9] x86 tsc_msr: Update comments, expand definitions

[Len Brown]

From: Len Brown <len.brown@intel.com>

Syntax only, no functional change.

Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/kernel/tsc_msr.c | 36 ++++++++++--------------------------
 1 file changed, 10 insertions(+), 26 deletions(-)

diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index d460ef1..3a866bc 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -1,14 +1,5 @@
 /*
- * tsc_msr.c - MSR based TSC calibration on Intel Atom SoC platforms.
- *
- * TSC in Intel Atom SoC runs at a constant rate which can be figured
- * by this formula:
- * <maximum core-clock to bus-clock ratio> * <maximum resolved frequency>
- * See Intel 64 and IA-32 System Programming Guid section 16.12 and 30.11.5
- * for details.
- * Especially some Intel Atom SoCs don't have PIT(i8254) or HPET, so MSR
- * based calibration is the only option.
- *
+ * tsc_msr.c - TSC frequency enumeration via MSR
  *
  * Copyright (C) 2013 Intel Corporation
  * Author: Bin Gao <bin.gao@intel.com>
@@ -22,17 +13,10 @@
 #include <asm/apic.h>
 #include <asm/param.h>
 
-/* CPU reference clock frequency: in KHz */
-#define FREQ_83		83200
-#define FREQ_100	99840
-#define FREQ_133	133200
-#define FREQ_166	166400
-
 #define MAX_NUM_FREQS	8
 
 /*
- * According to Intel 64 and IA-32 System Programming Guide,
- * if MSR_PERF_STAT[31] is set, the maximum resolved bus ratio can be
+ * If MSR_PERF_STAT[31] is set, the maximum resolved bus ratio can be
  * read in MSR_PLATFORM_ID[12:8], otherwise in MSR_PERF_STAT[44:40].
  * Unfortunately some Intel Atom SoCs aren't quite compliant to this,
  * so we need manually differentiate SoC families. This is what the
@@ -47,15 +31,15 @@ struct freq_desc {
 
 static struct freq_desc freq_desc_tables[] = {
 	/* PNW */
-	{ 6, 0x27, 0, { 0, 0, 0, 0, 0, FREQ_100, 0, FREQ_83 } },
+	{ 6, 0x27, 0, { 0, 0, 0, 0, 0, 99840, 0, 83200 } },
 	/* CLV+ */
-	{ 6, 0x35, 0, { 0, FREQ_133, 0, 0, 0, FREQ_100, 0, FREQ_83 } },
-	/* TNG */
-	{ 6, 0x4a, 1, { 0, FREQ_100, FREQ_133, 0, 0, 0, 0, 0 } },
-	/* VLV2 */
-	{ 6, 0x37, 1, { FREQ_83, FREQ_100, FREQ_133, FREQ_166, 0, 0, 0, 0 } },
-	/* ANN */
-	{ 6, 0x5a, 1, { FREQ_83, FREQ_100, FREQ_133, FREQ_100, 0, 0, 0, 0 } },
+	{ 6, 0x35, 0, { 0, 133200, 0, 0, 0, 99840, 0, 83200 } },
+	/* TNG - Intel Atom processor Z3400 series */
+	{ 6, 0x4a, 1, { 0, 99840, 133200, 0, 0, 0, 0, 0 } },
+	/* VLV2 - Intel Atom processor E3000, Z3600, Z3700 series */
+	{ 6, 0x37, 1, { 83200, 99840, 133200, 166400, 0, 0, 0, 0 } },
+	/* ANN - Intel Atom processor Z3500 series */
+	{ 6, 0x5a, 1, { 83200, 99840, 133200, 99840, 0, 0, 0, 0 } },
 };
 
 static int match_cpu(u8 family, u8 model)
-- 
2.8.0.rc4.16.g56331f8

[PATCH 4/9] x86 tsc_msr: Correct Silvermont reference clock values

[Len Brown]

From: Len Brown <len.brown@intel.com>

Atom processors use a 19.2 MHz crystal oscillator.

Early processors generate 100 MHz via 19.2 MHz * 26 / 5 = 99.84 MHz.

Later preocessor generate 100 MHz via 19.2 MHz * 125 / 24 = 100 MHz.

Update the Silvermont-based tables accordingly,
matching the Software Developers Manual.

Also, correct a 166 MHz entry that should have been 116 MHz,
and add a missing 80 MHz entry.

Reported-by: Stephane Gasparini <stephane.gasparini@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/kernel/tsc_msr.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index 3a866bc..19f2a9a 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -35,11 +35,11 @@ static struct freq_desc freq_desc_tables[] = {
 	/* CLV+ */
 	{ 6, 0x35, 0, { 0, 133200, 0, 0, 0, 99840, 0, 83200 } },
 	/* TNG - Intel Atom processor Z3400 series */
-	{ 6, 0x4a, 1, { 0, 99840, 133200, 0, 0, 0, 0, 0 } },
+	{ 6, 0x4a, 1, { 0, 100000, 133300, 0, 0, 0, 0, 0 } },
 	/* VLV2 - Intel Atom processor E3000, Z3600, Z3700 series */
-	{ 6, 0x37, 1, { 83200, 99840, 133200, 166400, 0, 0, 0, 0 } },
+	{ 6, 0x37, 1, { 83300, 100000, 133300, 116700, 80000, 0, 0, 0 } },
 	/* ANN - Intel Atom processor Z3500 series */
-	{ 6, 0x5a, 1, { 83200, 99840, 133200, 99840, 0, 0, 0, 0 } },
+	{ 6, 0x5a, 1, { 83300, 100000, 133300, 100000, 0, 0, 0, 0 } },
 };
 
 static int match_cpu(u8 family, u8 model)
-- 
2.8.0.rc4.16.g56331f8

[PATCH 5/9] x86 tsc_msr: Add Airmont reference clock values

[Len Brown]

From: Len Brown <len.brown@intel.com>

per the Intel 64 and IA-32 Architecture Software Developer's Manual...

Add the reference clock for Intel Atom Processors
Based on the Airmont Microarchitecture.

Reported-by: Stephane Gasparini <stephane.gasparini@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/kernel/tsc_msr.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index 19f2a9a..59c371e 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -13,7 +13,7 @@
 #include <asm/apic.h>
 #include <asm/param.h>
 
-#define MAX_NUM_FREQS	8
+#define MAX_NUM_FREQS	9
 
 /*
  * If MSR_PERF_STAT[31] is set, the maximum resolved bus ratio can be
@@ -40,6 +40,9 @@ static struct freq_desc freq_desc_tables[] = {
 	{ 6, 0x37, 1, { 83300, 100000, 133300, 116700, 80000, 0, 0, 0 } },
 	/* ANN - Intel Atom processor Z3500 series */
 	{ 6, 0x5a, 1, { 83300, 100000, 133300, 100000, 0, 0, 0, 0 } },
+	/* AMT - Intel Atom processor X7-Z8000 and X5-Z8000 series */
+	{ 6, 0x4c, 1, { 83300, 100000, 133300, 116700,
+			80000, 93300, 90000, 88900, 87500 } },
 };
 
 static int match_cpu(u8 family, u8 model)
-- 
2.8.0.rc4.16.g56331f8

[PATCH 6/9] x86 tsc_msr: Extend to include Intel Core Architecture

[Len Brown]

From: Len Brown <len.brown@intel.com>

tsc_msr is used to quickly and reliably
enumerate the CPU/TSC frequencies at boot time
For the Intel Atom Architecture.

Extend tsc_msr to include recent Intel Core Architecture.

As this code discovers BCLK, it also sets lapic_timer_frequency,
which allows LAPIC timer calibration to be skipped,
though it is already skipped on systems with a TSC deadline timer.

Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/kernel/tsc_msr.c | 49 +++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 41 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index 59c371e..c8ea977 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -77,23 +77,56 @@ unsigned long try_msr_calibrate_tsc(void)
 	if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
 		return 0;
 
+	/*
+	 * 100 MHz BCLK Core Architecture -- before SKL.
+	 * De-rate 100Mhz by about 0.25% to account
+	 * for the average effect of spread-spectrum clocking.
+	 */
+	switch (boot_cpu_data.x86_model) {
+
+	case 0x2A:	/* SNB */
+	case 0x3A:	/* IVB */
+		freq = 99773;
+		goto get_ratio;
+	case 0x2D:	/* SNB Xeon */
+	case 0x3E:	/* IVB Xeon */
+		freq = 99760;
+		goto get_ratio;
+	case 0x3C:	/* HSW */
+	case 0x3F:	/* HSW */
+	case 0x45:	/* HSW */
+	case 0x46:	/* HSW */
+	case 0x3D:	/* BDW */
+	case 0x47:	/* BDW */
+	case 0x4F:	/* BDX */
+	case 0x56:	/* BDX-DE */
+		freq = 99769;
+		goto get_ratio;
+	}
+
+	/*
+	 * Atom Architecture
+	 */
 	cpu_index = match_cpu(boot_cpu_data.x86, boot_cpu_data.x86_model);
 	if (cpu_index < 0)
 		return 0;
 
-	if (freq_desc_tables[cpu_index].msr_plat) {
-		rdmsr(MSR_PLATFORM_INFO, lo, hi);
-		ratio = (lo >> 8) & 0x1f;
-	} else {
-		rdmsr(MSR_IA32_PERF_STATUS, lo, hi);
-		ratio = (hi >> 8) & 0x1f;
-	}
-
 	/* Get FSB FREQ ID */
 	rdmsr(MSR_FSB_FREQ, lo, hi);
 	freq_id = lo & 0x7;
 	freq = id_to_freq(cpu_index, freq_id);
 
+	if (!freq_desc_tables[cpu_index].msr_plat) {
+		rdmsr(MSR_IA32_PERF_STATUS, lo, hi);
+		ratio = (hi >> 8) & 0x1f;
+		goto done;
+	}
+
+get_ratio:
+	rdmsr(MSR_PLATFORM_INFO, lo, hi);
+	ratio = (lo >> 8) & 0x1f;
+
+done:
 	/* TSC frequency = maximum resolved freq * maximum resolved bus ratio */
 	res = freq * ratio;
 
-- 
2.8.0.rc4.16.g56331f8

[PATCH 7/9] x86 tsc_msr: Remove irqoff around MSR-based TSC enumeration

[Len Brown]

From: Len Brown <len.brown@intel.com>

Remove the irqoff/irqon around MSR-based TSC enumeration,
as it is not necessary.

Also rename: try_msr_calibrate_tsc() to cpu_khz_from_msr(),
as that better describes what the routine does.

Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/include/asm/tsc.h | 3 +--
 arch/x86/kernel/tsc.c      | 5 +----
 arch/x86/kernel/tsc_msr.c  | 2 +-
 3 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 174c421..d634f2a 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -52,7 +52,6 @@ extern int notsc_setup(char *);
 extern void tsc_save_sched_clock_state(void);
 extern void tsc_restore_sched_clock_state(void);
 
-/* MSR based TSC calibration for Intel Atom SoC platforms */
-unsigned long try_msr_calibrate_tsc(void);
+unsigned long cpu_khz_from_msr(void);
 
 #endif /* _ASM_X86_TSC_H */
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index c9c4c7c..0ffb57f 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -674,10 +674,7 @@ unsigned long native_calibrate_tsc(void)
 	unsigned long flags, latch, ms, fast_calibrate;
 	int hpet = is_hpet_enabled(), i, loopmin;
 
-	/* Calibrate TSC using MSR for Intel Atom SoCs */
-	local_irq_save(flags);
-	fast_calibrate = try_msr_calibrate_tsc();
-	local_irq_restore(flags);
+	fast_calibrate = cpu_khz_from_msr();
 	if (fast_calibrate)
 		return fast_calibrate;
 
diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index c8ea977..b0839c5 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -68,7 +68,7 @@ static int match_cpu(u8 family, u8 model)
  * Set global "lapic_timer_frequency" to bus_clock_cycles/jiffy
  * Return processor base frequency in KHz, or 0 on failure.
  */
-unsigned long try_msr_calibrate_tsc(void)
+unsigned long cpu_khz_from_msr(void)
 {
 	u32 lo, hi, ratio, freq_id, freq;
 	unsigned long res;
-- 
2.8.0.rc4.16.g56331f8

[PATCH 8/9] x86 tsc: enumerate SKL cpu_khz and tsc_khz via CPUID

[Len Brown]

From: Len Brown <len.brown@intel.com>

Skylake CPU base-frequency and TSC frequency may differ
by up to 2%.

Enumerate CPU and TSC frequencies separately, allowing
cpu_khz and tsc_khz to differ.

The existing CPU frequency calibration mechanism is unchanged.
However, CPUID extensions are preferred, when available.

CPUID.0x16 is preferred over MSR and timer calibration
for CPU frequency discovery.

CPUID.0x15 takes precedence over CPU-frequency
for TSC frequency discovery.

Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/include/asm/tsc.h      |  1 +
 arch/x86/include/asm/x86_init.h |  4 ++-
 arch/x86/kernel/tsc.c           | 75 +++++++++++++++++++++++++++++++++++++----
 arch/x86/kernel/x86_init.c      |  1 +
 4 files changed, 73 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index d634f2a..c279502 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -36,6 +36,7 @@ extern void mark_tsc_unstable(char *reason);
 extern int unsynchronized_tsc(void);
 extern int check_tsc_unstable(void);
 extern int check_tsc_disabled(void);
+extern unsigned long native_calibrate_cpu(void);
 extern unsigned long native_calibrate_tsc(void);
 extern unsigned long long native_sched_clock_from_tsc(u64 tsc);
 
diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h
index 1ae89a2..2e5c84d 100644
--- a/arch/x86/include/asm/x86_init.h
+++ b/arch/x86/include/asm/x86_init.h
@@ -143,7 +143,8 @@ struct timespec;
 
 /**
  * struct x86_platform_ops - platform specific runtime functions
- * @calibrate_tsc:		calibrate TSC
+ * @calibrate_cpu:		calibrate CPU
+ * @calibrate_tsc:		calibrate TSC, if different from CPU
  * @get_wallclock:		get time from HW clock like RTC etc.
  * @set_wallclock:		set time back to HW clock
  * @is_untracked_pat_range	exclude from PAT logic
@@ -154,6 +155,7 @@ struct timespec;
  * @apic_post_init:		adjust apic if neeeded
  */
 struct x86_platform_ops {
+	unsigned long (*calibrate_cpu)(void);
 	unsigned long (*calibrate_tsc)(void);
 	void (*get_wallclock)(struct timespec *ts);
 	int (*set_wallclock)(const struct timespec *ts);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 0ffb57f..ca41c30 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -239,7 +239,7 @@ static inline unsigned long long cycles_2_ns(unsigned long long cyc)
 	return ns;
 }
 
-static void set_cyc2ns_scale(unsigned long cpu_khz, int cpu)
+static void set_cyc2ns_scale(unsigned long khz, int cpu)
 {
 	unsigned long long tsc_now, ns_now;
 	struct cyc2ns_data *data;
@@ -248,7 +248,7 @@ static void set_cyc2ns_scale(unsigned long cpu_khz, int cpu)
 	local_irq_save(flags);
 	sched_clock_idle_sleep_event();
 
-	if (!cpu_khz)
+	if (!khz)
 		goto done;
 
 	data = cyc2ns_write_begin(cpu);
@@ -261,7 +261,7 @@ static void set_cyc2ns_scale(unsigned long cpu_khz, int cpu)
 	 * time function is continuous; see the comment near struct
 	 * cyc2ns_data.
 	 */
-	clocks_calc_mult_shift(&data->cyc2ns_mul, &data->cyc2ns_shift, cpu_khz,
+	clocks_calc_mult_shift(&data->cyc2ns_mul, &data->cyc2ns_shift, khz,
 			       NSEC_PER_MSEC, 0);
 
 	/*
@@ -665,15 +665,72 @@ success:
 }
 
 /**
- * native_calibrate_tsc - calibrate the tsc on boot
+ * native_calibrate_tsc
+ * Determine TSC frequency via CPUID, else return 0.
  */
 unsigned long native_calibrate_tsc(void)
 {
+	unsigned int eax_denominator, ebx_numerator, ecx_hz, edx;
+	unsigned int crystal_khz;
+
+	if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
+		return 0;
+
+	if (boot_cpu_data.cpuid_level < 0x15)
+		return 0;
+
+	eax_denominator = ebx_numerator = ecx_hz = edx = 0;
+
+	/* CPUID 15H TSC/Crystal ratio, plus optionally Crystal Hz */
+	cpuid(0x15, &eax_denominator, &ebx_numerator, &ecx_hz, &edx);
+
+	if (ebx_numerator == 0 || eax_denominator == 0)
+		return 0;
+
+	crystal_khz = ecx_hz / 1000;
+
+	if (crystal_khz == 0) {
+		switch (boot_cpu_data.x86_model) {
+		case 0x4E:	/* SKL */
+		case 0x5E:	/* SKL */
+			crystal_khz = 24000;	/* 24 MHz */
+		}
+	}
+
+	return crystal_khz * ebx_numerator / eax_denominator;
+}
+
+static unsigned long cpu_khz_from_cpuid(void)
+{
+	unsigned int eax_base_mhz, ebx_max_mhz, ecx_bus_mhz, edx;
+
+	if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
+		return 0;
+
+	if (boot_cpu_data.cpuid_level < 0x16)
+		return 0;
+
+	eax_base_mhz = ebx_max_mhz = ecx_bus_mhz = edx = 0;
+
+	cpuid(0x16, &eax_base_mhz, &ebx_max_mhz, &ecx_bus_mhz, &edx);
+
+	return eax_base_mhz * 1000;
+}
+
+/**
+ * native_calibrate_cpu - calibrate the cpu on boot
+ */
+unsigned long native_calibrate_cpu(void)
+{
 	u64 tsc1, tsc2, delta, ref1, ref2;
 	unsigned long tsc_pit_min = ULONG_MAX, tsc_ref_min = ULONG_MAX;
 	unsigned long flags, latch, ms, fast_calibrate;
 	int hpet = is_hpet_enabled(), i, loopmin;
 
+	fast_calibrate = cpu_khz_from_cpuid();
+	if (fast_calibrate)
+		return fast_calibrate;
+
 	fast_calibrate = cpu_khz_from_msr();
 	if (fast_calibrate)
 		return fast_calibrate;
@@ -832,8 +889,10 @@ int recalibrate_cpu_khz(void)
 	unsigned long cpu_khz_old = cpu_khz;
 
 	if (cpu_has_tsc) {
+		cpu_khz = x86_platform.calibrate_cpu();
 		tsc_khz = x86_platform.calibrate_tsc();
-		cpu_khz = tsc_khz;
+		if (tsc_khz == 0)
+			tsc_khz = cpu_khz;
 		cpu_data(0).loops_per_jiffy =
 			cpufreq_scale(cpu_data(0).loops_per_jiffy,
 					cpu_khz_old, cpu_khz);
@@ -1244,8 +1303,10 @@ void __init tsc_init(void)
 		return;
 	}
 
+	cpu_khz = x86_platform.calibrate_cpu();
 	tsc_khz = x86_platform.calibrate_tsc();
-	cpu_khz = tsc_khz;
+	if (tsc_khz == 0)
+		tsc_khz = cpu_khz;
 
 	if (!tsc_khz) {
 		mark_tsc_unstable("could not calculate TSC khz");
@@ -1265,7 +1326,7 @@ void __init tsc_init(void)
 	 */
 	for_each_possible_cpu(cpu) {
 		cyc2ns_init(cpu);
-		set_cyc2ns_scale(cpu_khz, cpu);
+		set_cyc2ns_scale(tsc_khz, cpu);
 	}
 
 	if (tsc_disabled > 0)
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index dad5fe9..58b4592 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -92,6 +92,7 @@ static void default_nmi_init(void) { };
 static int default_i8042_detect(void) { return 1; };
 
 struct x86_platform_ops x86_platform = {
+	.calibrate_cpu			= native_calibrate_cpu,
 	.calibrate_tsc			= native_calibrate_tsc,
 	.get_wallclock			= mach_get_cmos_time,
 	.set_wallclock			= mach_set_rtc_mmss,
-- 
2.8.0.rc4.16.g56331f8

[PATCH 9/9] x86 tsc: enumerate BXT tsc_khz via CPUID

[Len Brown]

From: Bin Gao <bin.gao@intel.com>

Hard code the BXT crystal clock (aka ART - Always Running Timer)
to 19.200 MHz, and use CPUID leaf 0x15 to determine the BXT TSC frequency.

Use tsc_khz to sanity check BXT cpu_khz,
which can be erroneous in some configurations.

Signed-off-by: Bin Gao <bin.gao@intel.com>
[lenb: simplified]
Signed-off-by: Len Brown <len.brown@intel.com>
---
 arch/x86/kernel/tsc.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index ca41c30..64dc998 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -693,7 +693,11 @@ unsigned long native_calibrate_tsc(void)
 		switch (boot_cpu_data.x86_model) {
 		case 0x4E:	/* SKL */
 		case 0x5E:	/* SKL */
-			crystal_khz = 24000;	/* 24 MHz */
+			crystal_khz = 24000;	/* 24.0 MHz */
+			break;
+		case 0x5C:	/* BXT */
+			crystal_khz = 19200;	/* 19.2 MHz */
+			break;
 		}
 	}
 
@@ -891,8 +895,12 @@ int recalibrate_cpu_khz(void)
 	if (cpu_has_tsc) {
 		cpu_khz = x86_platform.calibrate_cpu();
 		tsc_khz = x86_platform.calibrate_tsc();
+
 		if (tsc_khz == 0)
 			tsc_khz = cpu_khz;
+		else if (abs(cpu_khz - tsc_khz) * 10 > tsc_khz)
+			cpu_khz = tsc_khz;
+
 		cpu_data(0).loops_per_jiffy =
 			cpufreq_scale(cpu_data(0).loops_per_jiffy,
 					cpu_khz_old, cpu_khz);
@@ -1305,8 +1313,16 @@ void __init tsc_init(void)
 
 	cpu_khz = x86_platform.calibrate_cpu();
 	tsc_khz = x86_platform.calibrate_tsc();
+
+	/*
+	 * Trust non-zero tsc_khz as authorative,
+	 * and use it to sanity check cpu_khz,
+	 * which will be off if system timer is off.
+	 */
 	if (tsc_khz == 0)
 		tsc_khz = cpu_khz;
+	else if (abs(cpu_khz - tsc_khz) * 10 > tsc_khz)
+		cpu_khz = tsc_khz;
 
 	if (!tsc_khz) {
 		mark_tsc_unstable("could not calculate TSC khz");
-- 
2.8.0.rc4.16.g56331f8

Re: [PATCH 0/9] x86: TSC calibration update

[Thomas Gleixner]

Len,

On Thu, 31 Mar 2016, Len Brown wrote:

> cpu_khz and tsc_khz initialization can be unreliable and expensive.
> They are initialized in tsc_init()/native_calibrate_tsc(), which prints:
> 
> pr_info("Detected %lu.%03lu MHz processor\n", cpu_khz...)
> 
> native_calibrate_cpu() first tries quick_pit_calibrate(),
> which can take over 50.0M cycles to succeed,
> or as few as 2.4M cycles to fail.
> 
> On failure, pit_calibrate_tsc() is attempted, which can succeed
> in as few as 20M cycles, but may consume over 240M cycles
> before it fails.
> 
> By comparison, on many processors, tsc frequency can be discovered by
> table and MSR or CPUID in under 0.002M cycles.

I found this in my backlog. Sorry for missing it. 

The series does not apply anymore and the hardcoded family constants want to
be replaced by the new model defines.

Could you please resend?

Thanks,

	tglx