[PATCH 0/5] x86/platform/UV: Update TSC support

[PATCH 0/5] x86/platform/UV: Update TSC support

[mike.travis]

The UV BIOS goes to considerable effort to get the TSC synchronization
accurate across the entire system.  Included in that are multiple chassis
that can have 32+ sockets.  The architecture does support an external
high resolution clock to aid in maintaining this synchronization.

The resulting TSC accuracy set by the UV system BIOS is much better
than the generic kernel TSC ADJUST functions.  This is important for
applications that read the TSC values directly for accessing data bases.

*   Patch 1 disables an assumption made by the kernel tsc sync functions
    that Socket 0 in the system should all have a TSC ADJUST value of
    zero.  This is not correct when the chassis are reset asynchronously
    to each other so which TSC's should be zero is not predictable.

*   Patch 2 prevents the kernel from attempting to fix the TSC if the
    system BIOS has determined that the TSC is not stable.  This prevents
    a slew of useless warning messages.

*   Patch 3 eliminates another avalanche of warning messages from older
    BIOS that did not have the TSC ADJUST MSR (ex. >3000 msgs in a 32
    socket Skylake system).  It now notes this with a single warning
    message and then moves on with fixing them.

*   Patch 4 puts in a facility to disable ART if the art to tsc conversion
    factor is not constant for all sockets.

*   Patch 5 puts a new check in the UV system init to check this new BIOS
    flag that displays the result of the TSC synchronization phase.  Three
    possible states are available, "sync is valid", "sync is unstable", or
    TSC sync state is unavailable in this BIOS.  In the later case, the
    UV kernel init reverts to prior assumptions about TSC sync state.

-- 

[PATCH 1/5] x86/kernel: Add option that TSC on Socket 0 being non-zero is valid

[mike.travis]

[-- Attachment #1: add-base-nonzero --]
[-- Type: text/plain, Size: 3235 bytes --]

Add a flag to indicate and process that TSC counters are on chassis
that reset at different times during system startup.  Therefore which
TSC ADJUST values should be zero is not predictable.

Signed-off-by: Mike Travis <mike.travis@hpe.com>
Reviewed-by: Dimitri Sivanich <dimitri.sivanich@hpe.com>
Reviewed-by: Russ Anderson <russ.anderson@hpe.com>
Reviewed-by: And ew Banman <andrew.abanman@hpe.com>
Reviewed-by: Peter Zijlstra <peterz@infradead.org>
---
 arch/x86/include/asm/tsc.h |    1 +
 arch/x86/kernel/tsc_sync.c |   39 +++++++++++++++++++++++++++++++++++----
 2 files changed, 36 insertions(+), 4 deletions(-)

--- linux.orig/arch/x86/include/asm/tsc.h
+++ linux/arch/x86/include/asm/tsc.h
@@ -35,6 +35,7 @@ extern void tsc_init(void);
 extern void mark_tsc_unstable(char *reason);
 extern int unsynchronized_tsc(void);
 extern int check_tsc_unstable(void);
+extern void mark_tsc_multi_sync_resets(char *reason);
 extern unsigned long native_calibrate_cpu(void);
 extern unsigned long native_calibrate_tsc(void);
 extern unsigned long long native_sched_clock_from_tsc(u64 tsc);
--- linux.orig/arch/x86/kernel/tsc_sync.c
+++ linux/arch/x86/kernel/tsc_sync.c
@@ -30,6 +30,20 @@ struct tsc_adjust {
 
 static DEFINE_PER_CPU(struct tsc_adjust, tsc_adjust);
 
+/*
+ * TSC's on different sockets may be reset asynchronously.
+ * This may cause the TSC ADJUST value on socket 0 to be NOT 0.
+ */
+static bool __read_mostly tsc_multi_sync_resets;
+
+void mark_tsc_multi_sync_resets(char *reason)
+{
+	if (tsc_multi_sync_resets)
+		return;
+	tsc_multi_sync_resets = true;
+	pr_info("tsc: Marking TSC multi sync resets true due to %s\n", reason);
+}
+
 void tsc_verify_tsc_adjust(bool resume)
 {
 	struct tsc_adjust *adj = this_cpu_ptr(&tsc_adjust);
@@ -71,12 +85,22 @@ static void tsc_sanitize_first_cpu(struc
 	 * non zero. We don't do that on non boot cpus because physical
 	 * hotplug should have set the ADJUST register to a value > 0 so
 	 * the TSC is in sync with the already running cpus.
+	 *
+	 * Also don't force the ADJUST value to zero if that is a valid value
+	 * for socket 0 as determined by the system arch.  This is required
+	 * when multiple sockets are reset asynchronously with each other
+	 * and socket 0 may not have an TSC ADJUST value of 0.
 	 */
 	if (bootcpu && bootval != 0) {
-		pr_warn(FW_BUG "TSC ADJUST: CPU%u: %lld force to 0\n", cpu,
-			bootval);
-		wrmsrl(MSR_IA32_TSC_ADJUST, 0);
-		bootval = 0;
+		if (likely(!tsc_multi_sync_resets)) {
+			pr_warn(FW_BUG "TSC ADJUST: CPU%u: %lld force to 0\n",
+				cpu, bootval);
+			wrmsrl(MSR_IA32_TSC_ADJUST, 0);
+			bootval = 0;
+		} else {
+			pr_info("TSC ADJUST: CPU%u: %lld NOT forced to 0\n",
+				cpu, bootval);
+		}
 	}
 	cur->adjusted = bootval;
 }
@@ -118,6 +142,13 @@ bool tsc_store_and_check_tsc_adjust(bool
 	cur->warned = false;
 
 	/*
+	 * If a non-zero TSC value for socket 0 may be valid then the default
+	 * adjusted value cannot assumed to be zero either.
+	 */
+	if (tsc_multi_sync_resets)
+		cur->adjusted = bootval;
+
+	/*
 	 * Check whether this CPU is the first in a package to come up. In
 	 * this case do not check the boot value against another package
 	 * because the new package might have been physically hotplugged,

-- 

[PATCH 2/5] x86/kernel: Skip TSC test and error messages if already unstable

[mike.travis]

[-- Attachment #1: unstable-tsc-disable-check --]
[-- Type: text/plain, Size: 1172 bytes --]

If the TSC has already been determined to be unstable, then checking
TSC ADJUST values is a waste of time and generates unnecessary error
messages.

Signed-off-by: Mike Travis <mike.travis@hpe.com>
Reviewed-by: Dimitri Sivanich <dimitri.sivanich@hpe.com>
Reviewed-by: Russ Anderson <russ.anderson@hpe.com>
Reviewed-by: Peter Zijlstra <peterz@infradead.org>
---
 arch/x86/kernel/tsc_sync.c |    8 ++++++++
 1 file changed, 8 insertions(+)

--- linux.orig/arch/x86/kernel/tsc_sync.c
+++ linux/arch/x86/kernel/tsc_sync.c
@@ -38,6 +38,10 @@ void tsc_verify_tsc_adjust(bool resume)
 	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
 		return;
 
+	/* Skip unnecessary error messages if TSC already unstable */
+	if (check_tsc_unstable())
+		return;
+
 	/* Rate limit the MSR check */
 	if (!resume && time_before(jiffies, adj->nextcheck))
 		return;
@@ -89,6 +93,10 @@ bool tsc_store_and_check_tsc_adjust(bool
 	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
 		return false;
 
+	/* Skip unnecessary error messages if TSC already unstable */
+	if (check_tsc_unstable())
+		return false;
+
 	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
 	cur->bootval = bootval;
 	cur->adjusted = bootval;

-- 

[PATCH 3/5] x86/kernel: Drastically reduce the number of firmware bug warnings

[mike.travis]

[-- Attachment #1: quiet-excessive-warnings --]
[-- Type: text/plain, Size: 1714 bytes --]

Prior to the TSC ADJUST MSR being available, the method to set TSC's in
sync with each other naturally caused a small skew between cpu threads.
This was NOT a firmware bug at the time so introducing a whole avalanche
of alarming warning messages might cause unnecessary concern and customer
complaints. (Example: >3000 msgs in a 32 socket Skylake system.)

Simply report the warning condition, if possible do the necessary fixes,
and move on.

Signed-off-by: Mike Travis <mike.travis@hpe.com>
Reviewed-by: Dimitri Sivanich <dimitri.sivanich@hpe.com>
Reviewed-by: Russ Anderson <russ.anderson@hpe.com>
Reviewed-by: Peter Zijlstra <peterz@infradead.org>
---
 arch/x86/kernel/tsc_sync.c |    9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

--- linux.orig/arch/x86/kernel/tsc_sync.c
+++ linux/arch/x86/kernel/tsc_sync.c
@@ -123,10 +123,9 @@ bool tsc_store_and_check_tsc_adjust(bool
 	 * Compare the boot value and complain if it differs in the
 	 * package.
 	 */
-	if (bootval != ref->bootval) {
-		pr_warn(FW_BUG "TSC ADJUST differs: Reference CPU%u: %lld CPU%u: %lld\n",
-			refcpu, ref->bootval, cpu, bootval);
-	}
+	if (bootval != ref->bootval)
+		printk_once(FW_BUG "TSC ADJUST differs within socket(s), fixing all errors\n");
+
 	/*
 	 * The TSC_ADJUST values in a package must be the same. If the boot
 	 * value on this newly upcoming CPU differs from the adjustment
@@ -134,8 +133,6 @@ bool tsc_store_and_check_tsc_adjust(bool
 	 * adjusted value.
 	 */
 	if (bootval != ref->adjusted) {
-		pr_warn("TSC ADJUST synchronize: Reference CPU%u: %lld CPU%u: %lld\n",
-			refcpu, ref->adjusted, cpu, bootval);
 		cur->adjusted = ref->adjusted;
 		wrmsrl(MSR_IA32_TSC_ADJUST, ref->adjusted);
 	}

-- 

[PATCH 4/5] x86/kernel: Provide a means to disable TSC ART

[mike.travis]

[-- Attachment #1: disable-detect_art --]
[-- Type: text/plain, Size: 1804 bytes --]

On systems where multiple chassis are reset asynchronously there is not a
constant ratio between the ART frequency and the TSC time that can be
provided by the boot cpu.  Provide a means to disable the ART capability
by arches that have this characteristic.

Signed-off-by: Mike Travis <mike.travis@hpe.com>
---
 arch/x86/include/asm/tsc.h |    1 +
 arch/x86/kernel/tsc.c      |   16 +++++++++++++---
 2 files changed, 14 insertions(+), 3 deletions(-)

--- linux.orig/arch/x86/include/asm/tsc.h
+++ linux/arch/x86/include/asm/tsc.h
@@ -36,6 +36,7 @@ extern void mark_tsc_unstable(char *reas
 extern int unsynchronized_tsc(void);
 extern int check_tsc_unstable(void);
 extern void mark_tsc_multi_sync_resets(char *reason);
+extern void mark_tsc_art_disabled(char *reason);
 extern unsigned long native_calibrate_cpu(void);
 extern unsigned long native_calibrate_tsc(void);
 extern unsigned long long native_sched_clock_from_tsc(u64 tsc);
--- linux.orig/arch/x86/kernel/tsc.c
+++ linux/arch/x86/kernel/tsc.c
@@ -955,15 +955,25 @@ core_initcall(cpufreq_register_tsc_scali
 #define ART_CPUID_LEAF (0x15)
 #define ART_MIN_DENOMINATOR (1)
 
-
 /*
- * If ART is present detect the numerator:denominator to convert to TSC
+ * If ART is present detect the numerator:denominator to convert to TSC.
+ * The CPU ART capability can be disabled.
  */
+static bool tsc_art_disabled;
+
+void mark_tsc_art_disabled(char *reason)
+{
+	if (tsc_art_disabled)
+		return;
+	tsc_art_disabled = true;
+	pr_info("TSC: ART disabled due to %s\n", reason);
+}
+
 static void detect_art(void)
 {
 	unsigned int unused[2];
 
-	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
+	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF || tsc_art_disabled)
 		return;
 
 	/* Don't enable ART in a VM, non-stop TSC and TSC_ADJUST required */

-- 

[PATCH 5/5] x86/platform/UV: Add check of TSC state set by UV BIOS

[mike.travis]

[-- Attachment #1: uv-set-tsc-valid-state --]
[-- Type: text/plain, Size: 4060 bytes --]

Insert a check early in UV system startup that checks whether BIOS was
able to obtain satisfactory TSC Sync stability.  If not, it usually
is caused by an error in the external TSC clock generation source.
In this case the best fallback is to use the builtin hardware RTC
as the kernel will not be able to set an accurate TSC sync either.

Signed-off-by: Mike Travis <mike.travis@hpe.com>
Reviewed-by: Dimitri Sivanich <dimitri.sivanich@hpe.com>
Reviewed-by: Russ Anderson <russ.anderson@hpe.com>
Reviewed-by: Andrew Banman <andrew.abanman@hpe.com>
---
 arch/x86/include/asm/uv/uv_hub.h   |   23 +++++++++++++----
 arch/x86/kernel/apic/x2apic_uv_x.c |   48 +++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 5 deletions(-)

--- linux.orig/arch/x86/include/asm/uv/uv_hub.h
+++ linux/arch/x86/include/asm/uv/uv_hub.h
@@ -776,23 +776,36 @@ static inline int uv_num_possible_blades
 extern void uv_nmi_setup(void);
 extern void uv_nmi_setup_hubless(void);
 
+/* BIOS/Kernel flags exchange MMR */
+#define UVH_BIOS_KERNEL_MMR		UVH_SCRATCH5
+#define UVH_BIOS_KERNEL_MMR_ALIAS	UVH_SCRATCH5_ALIAS
+#define UVH_BIOS_KERNEL_MMR_ALIAS_2	UVH_SCRATCH5_ALIAS_2
+
+/* TSC sync valid, set by BIOS */
+#define UVH_TSC_SYNC_MMR	UVH_BIOS_KERNEL_MMR
+#define UVH_TSC_SYNC_SHIFT	10
+#define UVH_TSC_SYNC_SHIFT_UV2K	16	/* UV2/3k have different bits */
+#define UVH_TSC_SYNC_MASK	3	/* 0011 */
+#define UVH_TSC_SYNC_VALID	3	/* 0011 */
+#define UVH_TSC_SYNC_INVALID	2	/* 0010 */
+
 /* BMC sets a bit this MMR non-zero before sending an NMI */
-#define UVH_NMI_MMR		UVH_SCRATCH5
-#define UVH_NMI_MMR_CLEAR	UVH_SCRATCH5_ALIAS
+#define UVH_NMI_MMR		UVH_BIOS_KERNEL_MMR
+#define UVH_NMI_MMR_CLEAR	UVH_BIOS_KERNEL_MMR_ALIAS
 #define UVH_NMI_MMR_SHIFT	63
-#define	UVH_NMI_MMR_TYPE	"SCRATCH5"
+#define UVH_NMI_MMR_TYPE	"SCRATCH5"
 
 /* Newer SMM NMI handler, not present in all systems */
 #define UVH_NMI_MMRX		UVH_EVENT_OCCURRED0
 #define UVH_NMI_MMRX_CLEAR	UVH_EVENT_OCCURRED0_ALIAS
 #define UVH_NMI_MMRX_SHIFT	UVH_EVENT_OCCURRED0_EXTIO_INT0_SHFT
-#define	UVH_NMI_MMRX_TYPE	"EXTIO_INT0"
+#define UVH_NMI_MMRX_TYPE	"EXTIO_INT0"
 
 /* Non-zero indicates newer SMM NMI handler present */
 #define UVH_NMI_MMRX_SUPPORTED	UVH_EXTIO_INT0_BROADCAST
 
 /* Indicates to BIOS that we want to use the newer SMM NMI handler */
-#define UVH_NMI_MMRX_REQ	UVH_SCRATCH5_ALIAS_2
+#define UVH_NMI_MMRX_REQ	UVH_BIOS_KERNEL_MMR_ALIAS_2
 #define UVH_NMI_MMRX_REQ_SHIFT	62
 
 struct uv_hub_nmi_s {
--- linux.orig/arch/x86/kernel/apic/x2apic_uv_x.c
+++ linux/arch/x86/kernel/apic/x2apic_uv_x.c
@@ -154,6 +154,53 @@ static int __init early_get_pnodeid(void
 	return pnode;
 }
 
+static void uv_tsc_check_sync(void)
+{
+	u64 mmr;
+	int sync_state;
+	int mmr_shift;
+	char *state;
+	bool valid;
+
+	/* Accommodate different UV arch BIOSes */
+	mmr = uv_early_read_mmr(UVH_TSC_SYNC_MMR);
+	mmr_shift =
+		is_uv1_hub() ? 0 :
+		is_uv2_hub() ? UVH_TSC_SYNC_SHIFT_UV2K : UVH_TSC_SYNC_SHIFT;
+	if (mmr_shift)
+		sync_state = (mmr >> mmr_shift) & UVH_TSC_SYNC_MASK;
+	else
+		sync_state = 0;
+
+	switch (sync_state) {
+	case UVH_TSC_SYNC_VALID:
+		state = "in sync";
+		valid = true;
+		break;
+
+	case UVH_TSC_SYNC_INVALID:
+		state = "unstable";
+		valid = false;
+		break;
+	default:
+		state = "unknown: assuming valid";
+		valid = true;
+		break;
+	}
+	pr_info("UV: TSC sync state from BIOS:0%d(%s)\n", sync_state, state);
+
+	/*
+	 * Mark flag that says TSC != 0 is valid for socket 0.
+	 * Mark flag that says ART is not reliable on multi-chassis UV.
+	 */
+	if (valid) {
+		mark_tsc_multi_sync_resets("UV BIOS");
+		mark_tsc_art_disabled("UV BIOS");
+	} else {
+		mark_tsc_unstable("UV BIOS");
+	}
+}
+
 /* [Copied from arch/x86/kernel/cpu/topology.c:detect_extended_topology()] */
 
 #define SMT_LEVEL			0	/* Leaf 0xb SMT level */
@@ -288,6 +335,7 @@ static int __init uv_acpi_madt_oem_check
 	}
 
 	pr_info("UV: OEM IDs %s/%s, System/HUB Types %d/%d, uv_apic %d\n", oem_id, oem_table_id, uv_system_type, uv_min_hub_revision_id, uv_apic);
+	uv_tsc_check_sync();
 
 	return uv_apic;
 

-- 

Re: [PATCH 4/5] x86/kernel: Provide a means to disable TSC ART

[Peter Zijlstra]

On Mon, Oct 02, 2017 at 10:12:18AM -0500, mike.travis@hpe.com wrote:
>  static void detect_art(void)
>  {
>  	unsigned int unused[2];
>  
> -	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
> +	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF || tsc_art_disabled)
>  		return;
>  
>  	/* Don't enable ART in a VM, non-stop TSC and TSC_ADJUST required */


So why can't we use is_uv_system() here an for the tsc_adjust thing?

Also (and I hate the name) tsc_multi_sync_resets is the reason you
cannot use ART, I don't think it makes sense to introduce yet another
knob.

Re: [PATCH 4/5] x86/kernel: Provide a means to disable TSC ART

[Mike Travis]

On 10/4/2017 2:27 AM, Peter Zijlstra wrote:
> On Mon, Oct 02, 2017 at 10:12:18AM -0500, mike.travis@hpe.com wrote:
>>   static void detect_art(void)
>>   {
>>   	unsigned int unused[2];
>>   
>> -	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
>> +	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF || tsc_art_disabled)
>>   		return;
>>   
>>   	/* Don't enable ART in a VM, non-stop TSC and TSC_ADJUST required */
> 
> 
> So why can't we use is_uv_system() here an for the tsc_adjust thing?

I could.  I just thought that there may be future system arches that
need the same facility?
> 
> Also (and I hate the name) tsc_multi_sync_resets is the reason you
> cannot use ART, I don't think it makes sense to introduce yet another
> knob.
> 
Okay.  I wasn't sure if there might be different causes for wanting one
condition over the other.  So does this mean change this later test to
use is_uv_system() or tsc_multi_sync_resets?

Thanks.

Re: [PATCH 4/5] x86/kernel: Provide a means to disable TSC ART

[Thomas Gleixner]

On Wed, 4 Oct 2017, Mike Travis wrote:
> On 10/4/2017 2:27 AM, Peter Zijlstra wrote:
> > On Mon, Oct 02, 2017 at 10:12:18AM -0500, mike.travis@hpe.com wrote:
> > >   static void detect_art(void)
> > >   {
> > >   	unsigned int unused[2];
> > >   -	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
> > > +	if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF || tsc_art_disabled)
> > >   		return;
> > >     	/* Don't enable ART in a VM, non-stop TSC and TSC_ADJUST
> > > required */
> > 
> > 
> > So why can't we use is_uv_system() here an for the tsc_adjust thing?
> 
> I could.  I just thought that there may be future system arches that
> need the same facility?
> > 
> > Also (and I hate the name) tsc_multi_sync_resets is the reason you
> > cannot use ART, I don't think it makes sense to introduce yet another
> > knob.
> > 
> Okay.  I wasn't sure if there might be different causes for wanting one
> condition over the other.  So does this mean change this later test to
> use is_uv_system() or tsc_multi_sync_resets?

tsc_multi_sync_resets because that's the reason to disable ART as not all
sockets have the same TSC <-> ART offset.

Btw, your changelog is slightly wrong here:

> On systems where multiple chassis are reset asynchronously there is not a
> constant ratio between the ART frequency and the TSC time that can be
> provided by the boot cpu.

The ratio is actually the same as the frequencies should be the
same. Though the offset in that equation:

                            n
      TSC = offset + ART * ---
                            d

is different because that offset is basically TSC_ADJUST.

Thanks,

	tglx

Re: [PATCH 2/5] x86/kernel: Skip TSC test and error messages if already unstable

[Mike Travis]

On 10/12/2017 8:22 AM, Thomas Gleixner wrote:
> On Thu, 12 Oct 2017, Mike Travis wrote:
>> On 10/12/2017 4:17 AM, Thomas Gleixner wrote:
>>> On Thu, 5 Oct 2017, mike.travis@hpe.com wrote:
>>>> @@ -89,6 +93,10 @@ bool tsc_store_and_check_tsc_adjust(bool
>>>>    	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
>>>>    		return false;
>>>>    +	/* Skip unnecessary error messages if TSC already unstable */
>>>> +	if (check_tsc_unstable())
>>>> +		return false;
>>>> +
>>>>    	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
>>>>    	cur->bootval = bootval;
>>>>    	cur->adjusted = bootval;
>>>
>>> This hunk rejects and I really can't figure out against which tree that
>>> would apply.
>>
>> My current merge tree happens to be 4.13.0-rc1 which was the latest when I
>> started this patch submission.  I can update my merge tree and reapply if need
>> be?
> 
> Please send patches always against top of tree and not some random ancient
> version of it.
> 

I pulled in 4.14.0-rc4 and reapplied the patches and they went in
without conflict.  Hopefully it will be the same for you.

(And dealt with your question about applying the same change to
the !CONFIG_SMP case.)

Thanks,
Mike

[PATCH 2/5] x86/kernel: Skip TSC test and error messages if already unstable

[mike.travis]

[-- Attachment #1: unstable-tsc-disable-check --]
[-- Type: text/plain, Size: 1270 bytes --]

If the TSC has already been determined to be unstable, then checking
TSC ADJUST values is a waste of time and generates unnecessary error
messages.

Signed-off-by: Mike Travis <mike.travis@hpe.com>
Reviewed-by: Dimitri Sivanich <dimitri.sivanich@hpe.com>
Reviewed-by: Russ Anderson <russ.anderson@hpe.com>
Reviewed-by: Peter Zijlstra <peterz@infradead.org>
---
v2: Add check_tsc_unstable to !CONFIG_SMP case.
    Patches merged against 4.14.0-rc4
---
 arch/x86/kernel/tsc_sync.c |    8 ++++++++
 1 file changed, 8 insertions(+)

--- linux.orig/arch/x86/kernel/tsc_sync.c
+++ linux/arch/x86/kernel/tsc_sync.c
@@ -52,6 +52,10 @@ void tsc_verify_tsc_adjust(bool resume)
 	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
 		return;
 
+	/* Skip unnecessary error messages if TSC already unstable */
+	if (check_tsc_unstable())
+		return;
+
 	/* Rate limit the MSR check */
 	if (!resume && time_before(jiffies, adj->nextcheck))
 		return;
@@ -114,6 +118,10 @@ bool __init tsc_store_and_check_tsc_adju
 	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
 		return false;
 
+	/* Skip unnecessary error messages if TSC already unstable */
+	if (check_tsc_unstable())
+		return false;
+
 	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
 	cur->bootval = bootval;
 	cur->nextcheck = jiffies + HZ;

-- 

Re: [PATCH 2/5] x86/kernel: Skip TSC test and error messages if already unstable

[Thomas Gleixner]

On Thu, 12 Oct 2017, Mike Travis wrote:
> On 10/12/2017 4:17 AM, Thomas Gleixner wrote:
> > On Thu, 5 Oct 2017, mike.travis@hpe.com wrote:
> > > @@ -89,6 +93,10 @@ bool tsc_store_and_check_tsc_adjust(bool
> > >   	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
> > >   		return false;
> > >   +	/* Skip unnecessary error messages if TSC already unstable */
> > > +	if (check_tsc_unstable())
> > > +		return false;
> > > +
> > >   	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
> > >   	cur->bootval = bootval;
> > >   	cur->adjusted = bootval;
> > 
> > This hunk rejects and I really can't figure out against which tree that
> > would apply.
> 
> My current merge tree happens to be 4.13.0-rc1 which was the latest when I
> started this patch submission.  I can update my merge tree and reapply if need
> be?

Please send patches always against top of tree and not some random ancient
version of it.

> > Btw, there are two incarnations of tsc_store_and_check_tsc_adjust().
> > Shouldn't the !SMP variant get the same treatment?
> 
> I could add it though I'm not sure the point?  If it's only one CPU would
> TSC's being out of sync become a question?

Well, this is about TSC_ADJUST and if BIOS/SMM fiddles with TSC_ADJUST
behind the kernels back, then our timekeeping is buggered. So we better
check that.

Thanks,

	tglx

Re: [PATCH 2/5] x86/kernel: Skip TSC test and error messages if already unstable

[Mike Travis]

On 10/12/2017 4:17 AM, Thomas Gleixner wrote:
> On Thu, 5 Oct 2017, mike.travis@hpe.com wrote:
> 
>> If the TSC has already been determined to be unstable, then checking
>> TSC ADJUST values is a waste of time and generates unnecessary error
>> messages.
>>
>> Signed-off-by: Mike Travis <mike.travis@hpe.com>
>> Reviewed-by: Dimitri Sivanich <dimitri.sivanich@hpe.com>
>> Reviewed-by: Russ Anderson <russ.anderson@hpe.com>
>> Reviewed-by: Peter Zijlstra <peterz@infradead.org>
>> ---
>>   arch/x86/kernel/tsc_sync.c |    8 ++++++++
>>   1 file changed, 8 insertions(+)
>>
>> --- linux.orig/arch/x86/kernel/tsc_sync.c
>> +++ linux/arch/x86/kernel/tsc_sync.c
>> @@ -38,6 +38,10 @@ void tsc_verify_tsc_adjust(bool resume)
>>   	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
>>   		return;
>>   
>> +	/* Skip unnecessary error messages if TSC already unstable */
>> +	if (check_tsc_unstable())
>> +		return;
>> +
>>   	/* Rate limit the MSR check */
>>   	if (!resume && time_before(jiffies, adj->nextcheck))
>>   		return;
>> @@ -89,6 +93,10 @@ bool tsc_store_and_check_tsc_adjust(bool
>>   	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
>>   		return false;
>>   
>> +	/* Skip unnecessary error messages if TSC already unstable */
>> +	if (check_tsc_unstable())
>> +		return false;
>> +
>>   	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
>>   	cur->bootval = bootval;
>>   	cur->adjusted = bootval;
> 
> This hunk rejects and I really can't figure out against which tree that
> would apply.

My current merge tree happens to be 4.13.0-rc1 which was the latest when 
I started this patch submission.  I can update my merge tree and reapply 
if need be?

> 
> Btw, there are two incarnations of tsc_store_and_check_tsc_adjust().
> Shouldn't the !SMP variant get the same treatment?

I could add it though I'm not sure the point?  If it's only one CPU 
would TSC's being out of sync become a question?
> 
> Thanks,
> 
> 	tglx
> 

Re: [PATCH 2/5] x86/kernel: Skip TSC test and error messages if already unstable

[Thomas Gleixner]

On Thu, 5 Oct 2017, mike.travis@hpe.com wrote:

> If the TSC has already been determined to be unstable, then checking
> TSC ADJUST values is a waste of time and generates unnecessary error
> messages.
> 
> Signed-off-by: Mike Travis <mike.travis@hpe.com>
> Reviewed-by: Dimitri Sivanich <dimitri.sivanich@hpe.com>
> Reviewed-by: Russ Anderson <russ.anderson@hpe.com>
> Reviewed-by: Peter Zijlstra <peterz@infradead.org>
> ---
>  arch/x86/kernel/tsc_sync.c |    8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> --- linux.orig/arch/x86/kernel/tsc_sync.c
> +++ linux/arch/x86/kernel/tsc_sync.c
> @@ -38,6 +38,10 @@ void tsc_verify_tsc_adjust(bool resume)
>  	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
>  		return;
>  
> +	/* Skip unnecessary error messages if TSC already unstable */
> +	if (check_tsc_unstable())
> +		return;
> +
>  	/* Rate limit the MSR check */
>  	if (!resume && time_before(jiffies, adj->nextcheck))
>  		return;
> @@ -89,6 +93,10 @@ bool tsc_store_and_check_tsc_adjust(bool
>  	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
>  		return false;
>  
> +	/* Skip unnecessary error messages if TSC already unstable */
> +	if (check_tsc_unstable())
> +		return false;
> +
>  	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
>  	cur->bootval = bootval;
>  	cur->adjusted = bootval;

This hunk rejects and I really can't figure out against which tree that
would apply.

Btw, there are two incarnations of tsc_store_and_check_tsc_adjust().
Shouldn't the !SMP variant get the same treatment?

Thanks,

	tglx

[PATCH 2/5] x86/kernel: Skip TSC test and error messages if already unstable

[mike.travis]

[-- Attachment #1: unstable-tsc-disable-check --]
[-- Type: text/plain, Size: 1172 bytes --]

If the TSC has already been determined to be unstable, then checking
TSC ADJUST values is a waste of time and generates unnecessary error
messages.

Signed-off-by: Mike Travis <mike.travis@hpe.com>
Reviewed-by: Dimitri Sivanich <dimitri.sivanich@hpe.com>
Reviewed-by: Russ Anderson <russ.anderson@hpe.com>
Reviewed-by: Peter Zijlstra <peterz@infradead.org>
---
 arch/x86/kernel/tsc_sync.c |    8 ++++++++
 1 file changed, 8 insertions(+)

--- linux.orig/arch/x86/kernel/tsc_sync.c
+++ linux/arch/x86/kernel/tsc_sync.c
@@ -38,6 +38,10 @@ void tsc_verify_tsc_adjust(bool resume)
 	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
 		return;
 
+	/* Skip unnecessary error messages if TSC already unstable */
+	if (check_tsc_unstable())
+		return;
+
 	/* Rate limit the MSR check */
 	if (!resume && time_before(jiffies, adj->nextcheck))
 		return;
@@ -89,6 +93,10 @@ bool tsc_store_and_check_tsc_adjust(bool
 	if (!boot_cpu_has(X86_FEATURE_TSC_ADJUST))
 		return false;
 
+	/* Skip unnecessary error messages if TSC already unstable */
+	if (check_tsc_unstable())
+		return false;
+
 	rdmsrl(MSR_IA32_TSC_ADJUST, bootval);
 	cur->bootval = bootval;
 	cur->adjusted = bootval;

--