[PATCH 1/2] mwait_idle: Disable Baytrail Core and Module C6 auto-demotion

[PATCH 1/2] mwait_idle: Disable Baytrail Core and Module C6 auto-demotion

[Ross Lagerwall]

From: Len Brown <len.brown@intel.com>

Power efficiency improves on Baytrail (Intel Atom Processor E3000)
when C6 auto-demotion is disabled.

Based on work by Srinidhi Kasagar <srinidhi.kasagar@intel.com>.

Signed-off-by: Len Brown <len.brown@intel.com>
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
---
 xen/arch/x86/cpu/mwait-idle.c   | 7 +++++++
 xen/include/asm-x86/msr-index.h | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
index c2c8889..2c5864e 100644
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -88,6 +88,7 @@ struct idle_cpu {
 	 * Indicate which enable bits to clear here.
 	 */
 	unsigned long auto_demotion_disable_flags;
+	bool_t byt_auto_demotion_disable_flag;
 	bool_t disable_promotion_to_c1e;
 };
 
@@ -569,6 +570,7 @@ static const struct idle_cpu idle_cpu_snb = {
 static const struct idle_cpu idle_cpu_byt = {
 	.state_table = byt_cstates,
 	.disable_promotion_to_c1e = 1,
+	.byt_auto_demotion_disable_flag = 1,
 };
 
 static const struct idle_cpu idle_cpu_ivb = {
@@ -767,6 +769,11 @@ static int mwait_idle_cpu_init(struct notifier_block *nfb,
 	if (icpu->auto_demotion_disable_flags)
 		on_selected_cpus(cpumask_of(cpu), auto_demotion_disable, NULL, 1);
 
+	if (icpu->byt_auto_demotion_disable_flag) {
+		wrmsrl(MSR_CC6_DEMOTION_POLICY_CONFIG, 0);
+		wrmsrl(MSR_MC6_DEMOTION_POLICY_CONFIG, 0);
+	}
+
 	if (icpu->disable_promotion_to_c1e)
 		on_selected_cpus(cpumask_of(cpu), c1e_promotion_disable, NULL, 1);
 
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 2056501..7e13742 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -102,6 +102,9 @@
 #define CMCI_EN 			(1UL<<30)
 #define CMCI_THRESHOLD_MASK		0x7FFF
 
+#define MSR_CC6_DEMOTION_POLICY_CONFIG	0x00000668
+#define MSR_MC6_DEMOTION_POLICY_CONFIG	0x00000669
+
 #define MSR_AMD64_MC0_MASK		0xc0010044
 
 #define MSR_IA32_MCx_CTL(x)		(MSR_IA32_MC0_CTL + 4*(x))
-- 
1.9.3

[PATCH 2/2] mwait_idle: Broadwell support

[Ross Lagerwall]

From: Len Brown <len.brown@intel.com>

Broadwell (BDW) is similar to Haswell (HSW), the preceding processor generation.

Currently, the only difference in their C-state tables is that PC3 max exit latency
is 33usec on HSW and 40usec on BDW.

Signed-off-by: Len Brown <len.brown@intel.com>
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
---
 xen/arch/x86/cpu/mwait-idle.c | 60 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
index 2c5864e..4d5d31d 100644
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -398,6 +398,58 @@ static const struct cpuidle_state hsw_cstates[] = {
 	{}
 };
 
+static struct cpuidle_state bdw_cstates[] = {
+	{
+		.name = "C1-BDW",
+		.flags = MWAIT2flg(0x00),
+		.exit_latency = 2,
+		.target_residency = 2,
+        },
+	{
+		.name = "C1E-BDW",
+		.flags = MWAIT2flg(0x01),
+		.exit_latency = 10,
+		.target_residency = 20,
+        },
+	{
+		.name = "C3-BDW",
+		.flags = MWAIT2flg(0x10) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 40,
+		.target_residency = 100,
+        },
+	{
+		.name = "C6-BDW",
+		.flags = MWAIT2flg(0x20) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 133,
+		.target_residency = 400,
+        },
+	{
+		.name = "C7s-BDW",
+		.flags = MWAIT2flg(0x32) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 166,
+		.target_residency = 500,
+        },
+	{
+		.name = "C8-BDW",
+		.flags = MWAIT2flg(0x40) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 300,
+		.target_residency = 900,
+        },
+	{
+		.name = "C9-BDW",
+		.flags = MWAIT2flg(0x50) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 600,
+		.target_residency = 1800,
+        },
+	{
+		.name = "C10-BDW",
+		.flags = MWAIT2flg(0x60) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 2600,
+		.target_residency = 7700,
+        },
+        {}
+};
+
 static const struct cpuidle_state atom_cstates[] = {
 	{
 		.name = "C1E-ATM",
@@ -588,6 +640,11 @@ static const struct idle_cpu idle_cpu_hsw = {
 	.disable_promotion_to_c1e = 1,
 };
 
+static const struct idle_cpu idle_cpu_bdw = {
+	.state_table = bdw_cstates,
+	.disable_promotion_to_c1e = 1,
+};
+
 static const struct idle_cpu idle_cpu_avn = {
 	.state_table = avn_cstates,
 	.disable_promotion_to_c1e = 1,
@@ -619,6 +676,9 @@ static struct intel_idle_id {
 	ICPU(0x45, hsw),
 	ICPU(0x46, hsw),
 	ICPU(0x4d, avn),
+	ICPU(0x3d, bdw),
+	ICPU(0x4f, bdw),
+	ICPU(0x56, bdw),
 	{}
 };
 
-- 
1.9.3

Re: [PATCH 1/2] mwait_idle: Disable Baytrail Core and Module C6 auto-demotion

[Jan Beulich]

[-- Attachment #1: Type: text/plain, Size: 2795 bytes --]

>>> On 08.09.14 at 14:42, <ross.lagerwall@citrix.com> wrote:
> From: Len Brown <len.brown@intel.com>
> 
> Power efficiency improves on Baytrail (Intel Atom Processor E3000)
> when C6 auto-demotion is disabled.
> 
> Based on work by Srinidhi Kasagar <srinidhi.kasagar@intel.com>.
> 
> Signed-off-by: Len Brown <len.brown@intel.com>
> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>

So we raced on pulling these over - I did so late last week too and
just didn't get to send them out yet. On this one I in fact doubt
(and sent a respective query to Len; no response yet) that the MSR
handling is. Hence my ported over patch is a little different:

x86/mwait-idle: disable Baytrail Core and Module C6 auto-demotion

Power efficiency improves on Baytrail (Intel Atom Processor E3000)
when Linux disables C6 auto-demotion.

Based on work by Srinidhi Kasagar <srinidhi.kasagar@intel.com>.

Signed-off-by: Len Brown <len.brown@intel.com>

Do the MSR writes on all CPUs rather than just the current one.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -483,4 +483,7 @@
 #define _MSR_MISC_FEATURES_CPUID_FAULTING	0
 #define MSR_MISC_FEATURES_CPUID_FAULTING	(1ULL << _MSR_MISC_FEATURES_CPUID_FAULTING)
 
+#define MSR_CC6_DEMOTION_POLICY_CONFIG	0x00000668
+#define MSR_MC6_DEMOTION_POLICY_CONFIG	0x00000669
+
 #endif /* __ASM_MSR_INDEX_H */
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -88,6 +88,7 @@ struct idle_cpu {
 	 * Indicate which enable bits to clear here.
 	 */
 	unsigned long auto_demotion_disable_flags;
+	bool_t byt_auto_demotion_disable_flag;
 	bool_t disable_promotion_to_c1e;
 };
 
@@ -539,6 +540,12 @@ static void auto_demotion_disable(void *
 	wrmsrl(MSR_NHM_SNB_PKG_CST_CFG_CTL, msr_bits);
 }
 
+static void byt_auto_demotion_disable(void *dummy)
+{
+	wrmsrl(MSR_CC6_DEMOTION_POLICY_CONFIG, 0);
+	wrmsrl(MSR_MC6_DEMOTION_POLICY_CONFIG, 0);
+}
+
 static void c1e_promotion_disable(void *dummy)
 {
 	u64 msr_bits;
@@ -571,6 +578,7 @@ static const struct idle_cpu idle_cpu_sn
 static const struct idle_cpu idle_cpu_byt = {
 	.state_table = byt_cstates,
 	.disable_promotion_to_c1e = 1,
+	.byt_auto_demotion_disable_flag = 1,
 };
 
 static const struct idle_cpu idle_cpu_ivb = {
@@ -769,6 +777,9 @@ static int mwait_idle_cpu_init(struct no
 	if (icpu->auto_demotion_disable_flags)
 		on_selected_cpus(cpumask_of(cpu), auto_demotion_disable, NULL, 1);
 
+	if (icpu->byt_auto_demotion_disable_flag)
+		on_selected_cpus(cpumask_of(cpu), byt_auto_demotion_disable, NULL, 1);
+
 	if (icpu->disable_promotion_to_c1e)
 		on_selected_cpus(cpumask_of(cpu), c1e_promotion_disable, NULL, 1);
 




[-- Attachment #2: x86-mwait-idle-Baytrail-no-demotion.patch --]
[-- Type: text/plain, Size: 2116 bytes --]

x86/mwait-idle: disable Baytrail Core and Module C6 auto-demotion

Power efficiency improves on Baytrail (Intel Atom Processor E3000)
when Linux disables C6 auto-demotion.

Based on work by Srinidhi Kasagar <srinidhi.kasagar@intel.com>.

Signed-off-by: Len Brown <len.brown@intel.com>

Do the MSR writes on all CPUs rather than just the current one.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -483,4 +483,7 @@
 #define _MSR_MISC_FEATURES_CPUID_FAULTING	0
 #define MSR_MISC_FEATURES_CPUID_FAULTING	(1ULL << _MSR_MISC_FEATURES_CPUID_FAULTING)
 
+#define MSR_CC6_DEMOTION_POLICY_CONFIG	0x00000668
+#define MSR_MC6_DEMOTION_POLICY_CONFIG	0x00000669
+
 #endif /* __ASM_MSR_INDEX_H */
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -88,6 +88,7 @@ struct idle_cpu {
 	 * Indicate which enable bits to clear here.
 	 */
 	unsigned long auto_demotion_disable_flags;
+	bool_t byt_auto_demotion_disable_flag;
 	bool_t disable_promotion_to_c1e;
 };
 
@@ -539,6 +540,12 @@ static void auto_demotion_disable(void *
 	wrmsrl(MSR_NHM_SNB_PKG_CST_CFG_CTL, msr_bits);
 }
 
+static void byt_auto_demotion_disable(void *dummy)
+{
+	wrmsrl(MSR_CC6_DEMOTION_POLICY_CONFIG, 0);
+	wrmsrl(MSR_MC6_DEMOTION_POLICY_CONFIG, 0);
+}
+
 static void c1e_promotion_disable(void *dummy)
 {
 	u64 msr_bits;
@@ -571,6 +578,7 @@ static const struct idle_cpu idle_cpu_sn
 static const struct idle_cpu idle_cpu_byt = {
 	.state_table = byt_cstates,
 	.disable_promotion_to_c1e = 1,
+	.byt_auto_demotion_disable_flag = 1,
 };
 
 static const struct idle_cpu idle_cpu_ivb = {
@@ -769,6 +777,9 @@ static int mwait_idle_cpu_init(struct no
 	if (icpu->auto_demotion_disable_flags)
 		on_selected_cpus(cpumask_of(cpu), auto_demotion_disable, NULL, 1);
 
+	if (icpu->byt_auto_demotion_disable_flag)
+		on_selected_cpus(cpumask_of(cpu), byt_auto_demotion_disable, NULL, 1);
+
 	if (icpu->disable_promotion_to_c1e)
 		on_selected_cpus(cpumask_of(cpu), c1e_promotion_disable, NULL, 1);
 

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 2/2] mwait_idle: Broadwell support

[Jan Beulich]

[-- Attachment #1: Type: text/plain, Size: 2830 bytes --]

>>> On 08.09.14 at 14:42, <ross.lagerwall@citrix.com> wrote:
> From: Len Brown <len.brown@intel.com>
> 
> Broadwell (BDW) is similar to Haswell (HSW), the preceding processor 
> generation.
> 
> Currently, the only difference in their C-state tables is that PC3 max exit 
> latency
> is 33usec on HSW and 40usec on BDW.
> 
> Signed-off-by: Len Brown <len.brown@intel.com>
> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>

Here yours is almost identical to mine - yours lacking a "const" and
having some indentation issues.

Jan

x86/mwait-idle: Broadwell support

Broadwell (BDW) is similar to Haswell (HSW), the preceding processor generation.

Currently, the only difference in their C-state tables is that PC3 max exit latency
is 33usec on HSW and 40usec on BDW.

Signed-off-by: Len Brown <len.brown@intel.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -398,6 +398,58 @@ static const struct cpuidle_state hsw_cs
 	{}
 };
 
+static const struct cpuidle_state bdw_cstates[] = {
+	{
+		.name = "C1-BDW",
+		.flags = MWAIT2flg(0x00),
+		.exit_latency = 2,
+		.target_residency = 2,
+	},
+	{
+		.name = "C1E-BDW",
+		.flags = MWAIT2flg(0x01),
+		.exit_latency = 10,
+		.target_residency = 20,
+	},
+	{
+		.name = "C3-BDW",
+		.flags = MWAIT2flg(0x10) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 40,
+		.target_residency = 100,
+	},
+	{
+		.name = "C6-BDW",
+		.flags = MWAIT2flg(0x20) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 133,
+		.target_residency = 400,
+	},
+	{
+		.name = "C7s-BDW",
+		.flags = MWAIT2flg(0x32) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 166,
+		.target_residency = 500,
+	},
+	{
+		.name = "C8-BDW",
+		.flags = MWAIT2flg(0x40) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 300,
+		.target_residency = 900,
+	},
+	{
+		.name = "C9-BDW",
+		.flags = MWAIT2flg(0x50) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 600,
+		.target_residency = 1800,
+	},
+	{
+		.name = "C10-BDW",
+		.flags = MWAIT2flg(0x60) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 2600,
+		.target_residency = 7700,
+	},
+	{}
+};
+
 static const struct cpuidle_state atom_cstates[] = {
 	{
 		.name = "C1E-ATM",
@@ -596,6 +648,11 @@ static const struct idle_cpu idle_cpu_hs
 	.disable_promotion_to_c1e = 1,
 };
 
+static const struct idle_cpu idle_cpu_bdw = {
+	.state_table = bdw_cstates,
+	.disable_promotion_to_c1e = 1,
+};
+
 static const struct idle_cpu idle_cpu_avn = {
 	.state_table = avn_cstates,
 	.disable_promotion_to_c1e = 1,
@@ -627,6 +684,9 @@ static struct intel_idle_id {
 	ICPU(0x45, hsw),
 	ICPU(0x46, hsw),
 	ICPU(0x4d, avn),
+	ICPU(0x3d, bdw),
+	ICPU(0x4f, bdw),
+	ICPU(0x56, bdw),
 	{}
 };
 




[-- Attachment #2: x86-mwait-idle-Broadwell.patch --]
[-- Type: text/plain, Size: 2266 bytes --]

x86/mwait-idle: Broadwell support

Broadwell (BDW) is similar to Haswell (HSW), the preceding processor generation.

Currently, the only difference in their C-state tables is that PC3 max exit latency
is 33usec on HSW and 40usec on BDW.

Signed-off-by: Len Brown <len.brown@intel.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -398,6 +398,58 @@ static const struct cpuidle_state hsw_cs
 	{}
 };
 
+static const struct cpuidle_state bdw_cstates[] = {
+	{
+		.name = "C1-BDW",
+		.flags = MWAIT2flg(0x00),
+		.exit_latency = 2,
+		.target_residency = 2,
+	},
+	{
+		.name = "C1E-BDW",
+		.flags = MWAIT2flg(0x01),
+		.exit_latency = 10,
+		.target_residency = 20,
+	},
+	{
+		.name = "C3-BDW",
+		.flags = MWAIT2flg(0x10) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 40,
+		.target_residency = 100,
+	},
+	{
+		.name = "C6-BDW",
+		.flags = MWAIT2flg(0x20) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 133,
+		.target_residency = 400,
+	},
+	{
+		.name = "C7s-BDW",
+		.flags = MWAIT2flg(0x32) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 166,
+		.target_residency = 500,
+	},
+	{
+		.name = "C8-BDW",
+		.flags = MWAIT2flg(0x40) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 300,
+		.target_residency = 900,
+	},
+	{
+		.name = "C9-BDW",
+		.flags = MWAIT2flg(0x50) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 600,
+		.target_residency = 1800,
+	},
+	{
+		.name = "C10-BDW",
+		.flags = MWAIT2flg(0x60) | CPUIDLE_FLAG_TLB_FLUSHED,
+		.exit_latency = 2600,
+		.target_residency = 7700,
+	},
+	{}
+};
+
 static const struct cpuidle_state atom_cstates[] = {
 	{
 		.name = "C1E-ATM",
@@ -596,6 +648,11 @@ static const struct idle_cpu idle_cpu_hs
 	.disable_promotion_to_c1e = 1,
 };
 
+static const struct idle_cpu idle_cpu_bdw = {
+	.state_table = bdw_cstates,
+	.disable_promotion_to_c1e = 1,
+};
+
 static const struct idle_cpu idle_cpu_avn = {
 	.state_table = avn_cstates,
 	.disable_promotion_to_c1e = 1,
@@ -627,6 +684,9 @@ static struct intel_idle_id {
 	ICPU(0x45, hsw),
 	ICPU(0x46, hsw),
 	ICPU(0x4d, avn),
+	ICPU(0x3d, bdw),
+	ICPU(0x4f, bdw),
+	ICPU(0x56, bdw),
 	{}
 };
 

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 1/2] mwait_idle: Disable Baytrail Core and Module C6 auto-demotion

[Ross Lagerwall]

On 09/08/2014 02:22 PM, Jan Beulich wrote:
>>>> On 08.09.14 at 14:42, <ross.lagerwall@citrix.com> wrote:
>> From: Len Brown <len.brown@intel.com>
>>
>> Power efficiency improves on Baytrail (Intel Atom Processor E3000)
>> when C6 auto-demotion is disabled.
>>
>> Based on work by Srinidhi Kasagar <srinidhi.kasagar@intel.com>.
>>
>> Signed-off-by: Len Brown <len.brown@intel.com>
>> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
>
> So we raced on pulling these over - I did so late last week too and
> just didn't get to send them out yet. On this one I in fact doubt
> (and sent a respective query to Len; no response yet) that the MSR
> handling is. Hence my ported over patch is a little different:
>

According to Intel's manual, those two registers have package scope, 
i.e. all processor cores in the package share the same MSR, so 
presumably the original commit is OK.

-- 
Ross Lagerwall

Re: [PATCH 1/2] mwait_idle: Disable Baytrail Core and Module C6 auto-demotion

[Jan Beulich]

>>> On 08.09.14 at 16:02, <ross.lagerwall@citrix.com> wrote:
> On 09/08/2014 02:22 PM, Jan Beulich wrote:
>>>>> On 08.09.14 at 14:42, <ross.lagerwall@citrix.com> wrote:
>>> From: Len Brown <len.brown@intel.com>
>>>
>>> Power efficiency improves on Baytrail (Intel Atom Processor E3000)
>>> when C6 auto-demotion is disabled.
>>>
>>> Based on work by Srinidhi Kasagar <srinidhi.kasagar@intel.com>.
>>>
>>> Signed-off-by: Len Brown <len.brown@intel.com>
>>> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
>>
>> So we raced on pulling these over - I did so late last week too and
>> just didn't get to send them out yet. On this one I in fact doubt
>> (and sent a respective query to Len; no response yet) that the MSR
>> handling is. Hence my ported over patch is a little different:
>>
> 
> According to Intel's manual, those two registers have package scope, 
> i.e. all processor cores in the package share the same MSR, so 
> presumably the original commit is OK.

Provided none of these can occur in multi-socket systems (which
seems likely to be the case for Atoms, but I'm not finally certain,
and there's definitely no harm in doing this redundantly).

Jan

Re: [PATCH 2/2] mwait_idle: Broadwell support

[Chen, Tiejun]

Did you guys validate this in real machine? Or any potential side affect?

When I do IGD GFX passthrough with qemu-xen-traditional, I found in the 
boot phase of VM, the target will reboot. After I revert this everything 
is fine.

Note I don't add anything into codes. Here I just build Xen unstable 4.5 
to use qemu-xen-traditional. And additionally, my BDW is based on 
stepping E, and guest is WindXP.

Thanks
Tiejun

On 2014/9/8 20:42, Ross Lagerwall wrote:
> From: Len Brown <len.brown@intel.com>
>
> Broadwell (BDW) is similar to Haswell (HSW), the preceding processor generation.
>
> Currently, the only difference in their C-state tables is that PC3 max exit latency
> is 33usec on HSW and 40usec on BDW.
>
> Signed-off-by: Len Brown <len.brown@intel.com>
> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
> ---
>   xen/arch/x86/cpu/mwait-idle.c | 60 +++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 60 insertions(+)
>
> diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
> index 2c5864e..4d5d31d 100644
> --- a/xen/arch/x86/cpu/mwait-idle.c
> +++ b/xen/arch/x86/cpu/mwait-idle.c
> @@ -398,6 +398,58 @@ static const struct cpuidle_state hsw_cstates[] = {
>   	{}
>   };
>
> +static struct cpuidle_state bdw_cstates[] = {
> +	{
> +		.name = "C1-BDW",
> +		.flags = MWAIT2flg(0x00),
> +		.exit_latency = 2,
> +		.target_residency = 2,
> +        },
> +	{
> +		.name = "C1E-BDW",
> +		.flags = MWAIT2flg(0x01),
> +		.exit_latency = 10,
> +		.target_residency = 20,
> +        },
> +	{
> +		.name = "C3-BDW",
> +		.flags = MWAIT2flg(0x10) | CPUIDLE_FLAG_TLB_FLUSHED,
> +		.exit_latency = 40,
> +		.target_residency = 100,
> +        },
> +	{
> +		.name = "C6-BDW",
> +		.flags = MWAIT2flg(0x20) | CPUIDLE_FLAG_TLB_FLUSHED,
> +		.exit_latency = 133,
> +		.target_residency = 400,
> +        },
> +	{
> +		.name = "C7s-BDW",
> +		.flags = MWAIT2flg(0x32) | CPUIDLE_FLAG_TLB_FLUSHED,
> +		.exit_latency = 166,
> +		.target_residency = 500,
> +        },
> +	{
> +		.name = "C8-BDW",
> +		.flags = MWAIT2flg(0x40) | CPUIDLE_FLAG_TLB_FLUSHED,
> +		.exit_latency = 300,
> +		.target_residency = 900,
> +        },
> +	{
> +		.name = "C9-BDW",
> +		.flags = MWAIT2flg(0x50) | CPUIDLE_FLAG_TLB_FLUSHED,
> +		.exit_latency = 600,
> +		.target_residency = 1800,
> +        },
> +	{
> +		.name = "C10-BDW",
> +		.flags = MWAIT2flg(0x60) | CPUIDLE_FLAG_TLB_FLUSHED,
> +		.exit_latency = 2600,
> +		.target_residency = 7700,
> +        },
> +        {}
> +};
> +
>   static const struct cpuidle_state atom_cstates[] = {
>   	{
>   		.name = "C1E-ATM",
> @@ -588,6 +640,11 @@ static const struct idle_cpu idle_cpu_hsw = {
>   	.disable_promotion_to_c1e = 1,
>   };
>
> +static const struct idle_cpu idle_cpu_bdw = {
> +	.state_table = bdw_cstates,
> +	.disable_promotion_to_c1e = 1,
> +};
> +
>   static const struct idle_cpu idle_cpu_avn = {
>   	.state_table = avn_cstates,
>   	.disable_promotion_to_c1e = 1,
> @@ -619,6 +676,9 @@ static struct intel_idle_id {
>   	ICPU(0x45, hsw),
>   	ICPU(0x46, hsw),
>   	ICPU(0x4d, avn),
> +	ICPU(0x3d, bdw),
> +	ICPU(0x4f, bdw),
> +	ICPU(0x56, bdw),
>   	{}
>   };
>
>

Re: [PATCH 2/2] mwait_idle: Broadwell support

[Jan Beulich]

>>> On 16.10.14 at 14:54, <tiejun.chen@intel.com> wrote:
> Did you guys validate this in real machine? Or any potential side affect?
> 
> When I do IGD GFX passthrough with qemu-xen-traditional, I found in the 
> boot phase of VM, the target will reboot. After I revert this everything 
> is fine.

Please be more precise. What is "target" here? Host? Guest?
Also, reporting issues just verbally (i.e. without any logs or other
actual technical information) rarely makes a lot of sense.

> Note I don't add anything into codes. Here I just build Xen unstable 4.5 
> to use qemu-xen-traditional. And additionally, my BDW is based on 
> stepping E, and guest is WindXP.

But the change is only about C-state handling for a particular
CPU model. There's really nothing being added that doesn't
already work fine elsewhere. I.e. this patch being the apparent
culprit of a problem you see makes it relatively likely that your
hardware is having some issue.

Jan

Re: [PATCH 2/2] mwait_idle: Broadwell support

[Chen, Tiejun]

On 2014/10/16 20:54, Chen, Tiejun wrote:
> Did you guys validate this in real machine? Or any potential side affect?

Any comments?

>
> When I do IGD GFX passthrough with qemu-xen-traditional, I found in the
> boot phase of VM, the target will reboot. After I revert this everything

Especially after reboot APCI can't be parsed correctly then Xen will 
disable VT-d. So I have to power off the target then power on to recover 
this.

> is fine.
>
> Note I don't add anything into codes. Here I just build Xen unstable 4.5
> to use qemu-xen-traditional. And additionally, my BDW is based on
> stepping E, and guest is WindXP.

s/WindXP/Wind7

Thanks
Tiejun

>
> Thanks
> Tiejun
>
> On 2014/9/8 20:42, Ross Lagerwall wrote:
>> From: Len Brown <len.brown@intel.com>
>>
>> Broadwell (BDW) is similar to Haswell (HSW), the preceding processor
>> generation.
>>
>> Currently, the only difference in their C-state tables is that PC3 max
>> exit latency
>> is 33usec on HSW and 40usec on BDW.
>>
>> Signed-off-by: Len Brown <len.brown@intel.com>
>> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
>> ---
>>   xen/arch/x86/cpu/mwait-idle.c | 60
>> +++++++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 60 insertions(+)
>>
>> diff --git a/xen/arch/x86/cpu/mwait-idle.c
>> b/xen/arch/x86/cpu/mwait-idle.c
>> index 2c5864e..4d5d31d 100644
>> --- a/xen/arch/x86/cpu/mwait-idle.c
>> +++ b/xen/arch/x86/cpu/mwait-idle.c
>> @@ -398,6 +398,58 @@ static const struct cpuidle_state hsw_cstates[] = {
>>       {}
>>   };
>>
>> +static struct cpuidle_state bdw_cstates[] = {
>> +    {
>> +        .name = "C1-BDW",
>> +        .flags = MWAIT2flg(0x00),
>> +        .exit_latency = 2,
>> +        .target_residency = 2,
>> +        },
>> +    {
>> +        .name = "C1E-BDW",
>> +        .flags = MWAIT2flg(0x01),
>> +        .exit_latency = 10,
>> +        .target_residency = 20,
>> +        },
>> +    {
>> +        .name = "C3-BDW",
>> +        .flags = MWAIT2flg(0x10) | CPUIDLE_FLAG_TLB_FLUSHED,
>> +        .exit_latency = 40,
>> +        .target_residency = 100,
>> +        },
>> +    {
>> +        .name = "C6-BDW",
>> +        .flags = MWAIT2flg(0x20) | CPUIDLE_FLAG_TLB_FLUSHED,
>> +        .exit_latency = 133,
>> +        .target_residency = 400,
>> +        },
>> +    {
>> +        .name = "C7s-BDW",
>> +        .flags = MWAIT2flg(0x32) | CPUIDLE_FLAG_TLB_FLUSHED,
>> +        .exit_latency = 166,
>> +        .target_residency = 500,
>> +        },
>> +    {
>> +        .name = "C8-BDW",
>> +        .flags = MWAIT2flg(0x40) | CPUIDLE_FLAG_TLB_FLUSHED,
>> +        .exit_latency = 300,
>> +        .target_residency = 900,
>> +        },
>> +    {
>> +        .name = "C9-BDW",
>> +        .flags = MWAIT2flg(0x50) | CPUIDLE_FLAG_TLB_FLUSHED,
>> +        .exit_latency = 600,
>> +        .target_residency = 1800,
>> +        },
>> +    {
>> +        .name = "C10-BDW",
>> +        .flags = MWAIT2flg(0x60) | CPUIDLE_FLAG_TLB_FLUSHED,
>> +        .exit_latency = 2600,
>> +        .target_residency = 7700,
>> +        },
>> +        {}
>> +};
>> +
>>   static const struct cpuidle_state atom_cstates[] = {
>>       {
>>           .name = "C1E-ATM",
>> @@ -588,6 +640,11 @@ static const struct idle_cpu idle_cpu_hsw = {
>>       .disable_promotion_to_c1e = 1,
>>   };
>>
>> +static const struct idle_cpu idle_cpu_bdw = {
>> +    .state_table = bdw_cstates,
>> +    .disable_promotion_to_c1e = 1,
>> +};
>> +
>>   static const struct idle_cpu idle_cpu_avn = {
>>       .state_table = avn_cstates,
>>       .disable_promotion_to_c1e = 1,
>> @@ -619,6 +676,9 @@ static struct intel_idle_id {
>>       ICPU(0x45, hsw),
>>       ICPU(0x46, hsw),
>>       ICPU(0x4d, avn),
>> +    ICPU(0x3d, bdw),
>> +    ICPU(0x4f, bdw),
>> +    ICPU(0x56, bdw),
>>       {}
>>   };
>>
>>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

Re: [PATCH 2/2] mwait_idle: Broadwell support

[Chen, Tiejun]

On 2014/10/16 22:06, Jan Beulich wrote:
>>>> On 16.10.14 at 14:54, <tiejun.chen@intel.com> wrote:
>> Did you guys validate this in real machine? Or any potential side affect?
>>
>> When I do IGD GFX passthrough with qemu-xen-traditional, I found in the
>> boot phase of VM, the target will reboot. After I revert this everything
>> is fine.
>
> Please be more precise. What is "target" here? Host? Guest?

The real target is BDW based on stepping E. When I use 
qemu-xen-traditional to validate IGD passthrough like this,

gfx_passthru=1
pci=["00:02.0", "00:14.0"]

Here the device at "00:02.0" is IGD device, another is USB controller.

And we should pass 'no-sharept' to disable shared EPT table since you 
know there is a well-know RMRR problem I'm trying to fix.

During the Windows7 guest VM boot phase, the physical machine reboot 
directly.

> Also, reporting issues just verbally (i.e. without any logs or other
> actual technical information) rarely makes a lot of sense.

As I said above the machine reboot directly. I can't see any output.

>
>> Note I don't add anything into codes. Here I just build Xen unstable 4.5
>> to use qemu-xen-traditional. And additionally, my BDW is based on
>> stepping E, and guest is WindXP.

Any other technical information you guys want to know?

>
> But the change is only about C-state handling for a particular
> CPU model. There's really nothing being added that doesn't
> already work fine elsewhere. I.e. this patch being the apparent
> culprit of a problem you see makes it relatively likely that your
> hardware is having some issue.

Yes I also suspect this point but I tried two machine, I still can see 
the same problem. Maybe this is related to CPU stepping, BIOS or some 
errata we may ignore previously. So I just post this to ask if anybody 
know this in detail.

Thanks
Tiejun
>
> Jan
>
>

Re: [PATCH 2/2] mwait_idle: Broadwell support

[Jan Beulich]

>>> On 17.10.14 at 03:22, <tiejun.chen@intel.com> wrote:
> On 2014/10/16 22:06, Jan Beulich wrote:
>> Also, reporting issues just verbally (i.e. without any logs or other
>> actual technical information) rarely makes a lot of sense.
> 
> As I said above the machine reboot directly. I can't see any output.

Triple faults are extremely rare. Are you sure you have
"sync_console noreboot" in place on your hypervisor command line
(and of course a serial console properly set up)?

Jan

Re: [PATCH 2/2] mwait_idle: Broadwell support

[Chen, Tiejun]

On 2014/10/17 16:40, Jan Beulich wrote:
>>>> On 17.10.14 at 03:22, <tiejun.chen@intel.com> wrote:
>> On 2014/10/16 22:06, Jan Beulich wrote:
>>> Also, reporting issues just verbally (i.e. without any logs or other
>>> actual technical information) rarely makes a lot of sense.
>>
>> As I said above the machine reboot directly. I can't see any output.
>
> Triple faults are extremely rare. Are you sure you have
> "sync_console noreboot" in place on your hypervisor command line
> (and of course a serial console properly set up)?

Looks current BIOS can't output anything so I will try this once I get 
the latest.

Thanks
Tiejun