[PATCH 0/3] X86: Deliver Hyper-V interrupts on a seperate IDT vector

[PATCH 0/3] X86: Deliver Hyper-V interrupts on a seperate IDT vector

[K. Y. Srinivasan]

This patch-set implements the functionality to deliver Hyper-V VMBUS interrupts via
a special IDT entry. Xen emulates Hyper-V and we have added code in this patch-set to
properly manage this emulation when Linux is running on Hyper-V.

K. Y. Srinivasan (2):
  X86: Add a check to catch Xen emulation of Hyper-V
  X86: Handle Hyper-V vmbus interrupts as special hypervisor interrupts

Olaf Hering (1):
  x86: Hyper-V: register clocksource only if its advertised

 arch/x86/include/asm/irq_vectors.h |    4 +-
 arch/x86/include/asm/mshyperv.h    |    4 +++
 arch/x86/kernel/cpu/mshyperv.c     |   48 +++++++++++++++++++++++++++++++++++-
 arch/x86/kernel/entry_32.S         |    9 ++++++-
 arch/x86/kernel/entry_64.S         |    7 ++++-
 drivers/xen/events.c               |    7 +++--
 6 files changed, 71 insertions(+), 8 deletions(-)

-- 
1.7.4.1

[PATCH 1/3] x86: Hyper-V: register clocksource only if its advertised

[K. Y. Srinivasan]

From: Olaf Hering <[mailto:olaf@aepfle.de]>

Enable hyperv_clocksource only if its advertised as a feature.
XenServer 6 returns the signature which is checked in
ms_hyperv_platform(), but it does not offer all features. Currently the
clocksource is enabled unconditionally in ms_hyperv_init_platform(), and
the result is a hanging guest.

Hyper-V spec Bit 1 indicates the availability of Partition Reference
Counter.  Register the clocksource only if this bit is set.

The guest in question prints this in dmesg:
 [    0.000000] Hypervisor detected: Microsoft HyperV
 [    0.000000] HyperV: features 0x70, hints 0x0

This bug can be reproduced easily be setting 'viridian=1' in a HVM domU
.cfg file. A workaround without this patch is to boot the HVM guest with
'clocksource=jiffies'.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
Cc: stable@kernel.org
Cc: Greg KH <gregkh@linuxfoundation.org>
---
 arch/x86/kernel/cpu/mshyperv.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 0a630dd..646d192 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -68,7 +68,8 @@ static void __init ms_hyperv_init_platform(void)
 	printk(KERN_INFO "HyperV: features 0x%x, hints 0x%x\n",
 	       ms_hyperv.features, ms_hyperv.hints);
 
-	clocksource_register_hz(&hyperv_cs, NSEC_PER_SEC/100);
+	if (ms_hyperv.features & HV_X64_MSR_TIME_REF_COUNT_AVAILABLE)
+		clocksource_register_hz(&hyperv_cs, NSEC_PER_SEC/100);
 }
 
 const __refconst struct hypervisor_x86 x86_hyper_ms_hyperv = {
-- 
1.7.4.1

[PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[K. Y. Srinivasan]

Xen emulates Hyper-V to host enlightened Windows. Looks like this
emulation may be turned on by default even for Linux guests. Check and
fail Hyper-V detection if we are on Xen.

Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
---
 arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 646d192..4dab317 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
 	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
 		return false;
 
+	/*
+	 * Xen emulates Hyper-V to support enlightened Windows.
+	 * Check to see first if we are on a Xen Hypervisor.
+	 */
+	if (xen_cpuid_base())
+		return false;
+
 	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
 	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);
 
-- 
1.7.4.1

[PATCH 3/3] X86: Handle Hyper-V vmbus interrupts as special hypervisor interrupts

[K. Y. Srinivasan]

Starting with win8, vmbus interrupts can be delivered on any VCPU in the guest
and furthermore can be concurrently active on multiple VCPUs. Support this
interrupt delivery model by setting up a separate IDT entry for Hyper-V vmbus.
interrupts. I would like to thank Jan Beulich <JBeulich@suse.com> and
Thomas Gleixner <tglx@linutronix.de>, for their help.

In this version of the patch, based on the feedback, I have merged the IDT
vector for Xen and Hyper-V and made the necessary adjustments.

Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
---
 arch/x86/include/asm/irq_vectors.h |    4 +-
 arch/x86/include/asm/mshyperv.h    |    4 +++
 arch/x86/kernel/cpu/mshyperv.c     |   38 ++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/entry_32.S         |    9 +++++++-
 arch/x86/kernel/entry_64.S         |    7 +++++-
 drivers/xen/events.c               |    7 +++--
 6 files changed, 62 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_vectors.h
index 1508e51..aac5fa6 100644
--- a/arch/x86/include/asm/irq_vectors.h
+++ b/arch/x86/include/asm/irq_vectors.h
@@ -109,8 +109,8 @@
 
 #define UV_BAU_MESSAGE			0xf5
 
-/* Xen vector callback to receive events in a HVM domain */
-#define XEN_HVM_EVTCHN_CALLBACK		0xf3
+/* Vector on which hypervisor callbacks will be delivered */
+#define HYPERVISOR_CALLBACK_VECTOR	0xf3
 
 /*
  * Local APIC timer IRQ vector is on a different priority level,
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 79ce568..c2934be 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -11,4 +11,8 @@ struct ms_hyperv_info {
 
 extern struct ms_hyperv_info ms_hyperv;
 
+void hyperv_callback_vector(void);
+void hyperv_vector_handler(struct pt_regs *regs);
+void hv_register_vmbus_handler(int irq, irq_handler_t handler);
+
 #endif
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 4dab317..e48a01cb 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -14,10 +14,15 @@
 #include <linux/time.h>
 #include <linux/clocksource.h>
 #include <linux/module.h>
+#include <linux/hardirq.h>
+#include <linux/interrupt.h>
 #include <asm/processor.h>
 #include <asm/hypervisor.h>
 #include <asm/hyperv.h>
 #include <asm/mshyperv.h>
+#include <asm/desc.h>
+#include <asm/idle.h>
+#include <asm/irq_regs.h>
 
 struct ms_hyperv_info ms_hyperv;
 EXPORT_SYMBOL_GPL(ms_hyperv);
@@ -77,6 +82,12 @@ static void __init ms_hyperv_init_platform(void)
 
 	if (ms_hyperv.features & HV_X64_MSR_TIME_REF_COUNT_AVAILABLE)
 		clocksource_register_hz(&hyperv_cs, NSEC_PER_SEC/100);
+#if IS_ENABLED(CONFIG_HYPERV)
+	/*
+	 * Setup the IDT for hypervisor callback.
+	 */
+	alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, hyperv_callback_vector);
+#endif
 }
 
 const __refconst struct hypervisor_x86 x86_hyper_ms_hyperv = {
@@ -85,3 +96,30 @@ const __refconst struct hypervisor_x86 x86_hyper_ms_hyperv = {
 	.init_platform		= ms_hyperv_init_platform,
 };
 EXPORT_SYMBOL(x86_hyper_ms_hyperv);
+
+static int vmbus_irq = -1;
+static irq_handler_t vmbus_isr;
+
+void hv_register_vmbus_handler(int irq, irq_handler_t handler)
+{
+	vmbus_irq = irq;
+	vmbus_isr = handler;
+}
+EXPORT_SYMBOL_GPL(hv_register_vmbus_handler);
+
+void hyperv_vector_handler(struct pt_regs *regs)
+{
+	struct pt_regs *old_regs = set_irq_regs(regs);
+	struct irq_desc *desc;
+
+	irq_enter();
+	exit_idle();
+
+	desc = irq_to_desc(vmbus_irq);
+
+	if (desc)
+		generic_handle_irq_desc(vmbus_irq, desc);
+
+	irq_exit();
+	set_irq_regs(old_regs);
+}
diff --git a/arch/x86/kernel/entry_32.S b/arch/x86/kernel/entry_32.S
index 352e5a9..8f3e2de 100644
--- a/arch/x86/kernel/entry_32.S
+++ b/arch/x86/kernel/entry_32.S
@@ -1046,11 +1046,18 @@ ENTRY(xen_failsafe_callback)
 	_ASM_EXTABLE(4b,9b)
 ENDPROC(xen_failsafe_callback)
 
-BUILD_INTERRUPT3(xen_hvm_callback_vector, XEN_HVM_EVTCHN_CALLBACK,
+BUILD_INTERRUPT3(xen_hvm_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
 		xen_evtchn_do_upcall)
 
 #endif	/* CONFIG_XEN */
 
+#if IS_ENABLED(CONFIG_HYPERV)
+
+BUILD_INTERRUPT3(hyperv_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
+	hyperv_vector_handler)
+
+#endif /* CONFIG_HYPERV */
+
 #ifdef CONFIG_FUNCTION_TRACER
 #ifdef CONFIG_DYNAMIC_FTRACE
 
diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S
index d5113c3..c1d01e6 100644
--- a/arch/x86/kernel/entry_64.S
+++ b/arch/x86/kernel/entry_64.S
@@ -1446,11 +1446,16 @@ ENTRY(xen_failsafe_callback)
 	CFI_ENDPROC
 END(xen_failsafe_callback)
 
-apicinterrupt XEN_HVM_EVTCHN_CALLBACK \
+apicinterrupt HYPERVISOR_CALLBACK_VECTOR \
 	xen_hvm_callback_vector xen_evtchn_do_upcall
 
 #endif /* CONFIG_XEN */
 
+#if IS_ENABLED(CONFIG_HYPERV)
+apicinterrupt HYPERVISOR_CALLBACK_VECTOR \
+	hyperv_callback_vector hyperv_vector_handler
+#endif /* CONFIG_HYPERV */
+
 /*
  * Some functions should be protected against kprobes
  */
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 4293c57..a54401e 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -1787,7 +1787,7 @@ void xen_callback_vector(void)
 	int rc;
 	uint64_t callback_via;
 	if (xen_have_vector_callback) {
-		callback_via = HVM_CALLBACK_VECTOR(XEN_HVM_EVTCHN_CALLBACK);
+		callback_via = HVM_CALLBACK_VECTOR(HYPERVISOR_CALLBACK_VECTOR);
 		rc = xen_set_callback_via(callback_via);
 		if (rc) {
 			printk(KERN_ERR "Request for Xen HVM callback vector"
@@ -1798,8 +1798,9 @@ void xen_callback_vector(void)
 		printk(KERN_INFO "Xen HVM callback vector for event delivery is "
 				"enabled\n");
 		/* in the restore case the vector has already been allocated */
-		if (!test_bit(XEN_HVM_EVTCHN_CALLBACK, used_vectors))
-			alloc_intr_gate(XEN_HVM_EVTCHN_CALLBACK, xen_hvm_callback_vector);
+		if (!test_bit(HYPERVISOR_CALLBACK_VECTOR, used_vectors))
+			alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR,
+					xen_hvm_callback_vector);
 	}
 }
 #else
-- 
1.7.4.1

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Jan Beulich]

>>> On 30.01.13 at 01:51, "K. Y. Srinivasan" <kys@microsoft.com> wrote:
> Xen emulates Hyper-V to host enlightened Windows. Looks like this
> emulation may be turned on by default even for Linux guests. Check and
> fail Hyper-V detection if we are on Xen.
> 
> Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> ---
>  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
>  1 files changed, 7 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 646d192..4dab317 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
>  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
>  		return false;
>  
> +	/*
> +	 * Xen emulates Hyper-V to support enlightened Windows.
> +	 * Check to see first if we are on a Xen Hypervisor.
> +	 */
> +	if (xen_cpuid_base())
> +		return false;
> +
>  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
>  	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);

I'm not convinced that's the right approach - any hypervisor
could do similar emulation, and hence you either want to make
sure you run on Hyper-V (by excluding all others), or you
tolerate using the emulation (which may require syncing up with
the other guest implementations so that shared resources don't
get used by two parties).

I also wonder whether using the Hyper-V emulation (where
useful, there might not be anything right now, but this may
change going forward) when no Xen support is configured
wouldn't be better than not using anything...

Jan

Re: [PATCH 3/3] X86: Handle Hyper-V vmbus interrupts as special hypervisor interrupts

[Jan Beulich]

>>> On 30.01.13 at 01:51, "K. Y. Srinivasan" <kys@microsoft.com> wrote:
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -14,10 +14,15 @@
>  #include <linux/time.h>
>  #include <linux/clocksource.h>
>  #include <linux/module.h>
> +#include <linux/hardirq.h>
> +#include <linux/interrupt.h>
>  #include <asm/processor.h>
>  #include <asm/hypervisor.h>
>  #include <asm/hyperv.h>
>  #include <asm/mshyperv.h>
> +#include <asm/desc.h>
> +#include <asm/idle.h>
> +#include <asm/irq_regs.h>
>  
>  struct ms_hyperv_info ms_hyperv;
>  EXPORT_SYMBOL_GPL(ms_hyperv);
> @@ -77,6 +82,12 @@ static void __init ms_hyperv_init_platform(void)
>  
>  	if (ms_hyperv.features & HV_X64_MSR_TIME_REF_COUNT_AVAILABLE)
>  		clocksource_register_hz(&hyperv_cs, NSEC_PER_SEC/100);
> +#if IS_ENABLED(CONFIG_HYPERV)
> +	/*
> +	 * Setup the IDT for hypervisor callback.
> +	 */
> +	alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, hyperv_callback_vector);
> +#endif
>  }
>  
>  const __refconst struct hypervisor_x86 x86_hyper_ms_hyperv = {
> @@ -85,3 +96,30 @@ const __refconst struct hypervisor_x86 x86_hyper_ms_hyperv = {
>  	.init_platform		= ms_hyperv_init_platform,
>  };
>  EXPORT_SYMBOL(x86_hyper_ms_hyperv);
> +
> +static int vmbus_irq = -1;
> +static irq_handler_t vmbus_isr;
> +
> +void hv_register_vmbus_handler(int irq, irq_handler_t handler)
> +{
> +	vmbus_irq = irq;
> +	vmbus_isr = handler;
> +}
> +EXPORT_SYMBOL_GPL(hv_register_vmbus_handler);
> +
> +void hyperv_vector_handler(struct pt_regs *regs)
> +{
> +	struct pt_regs *old_regs = set_irq_regs(regs);
> +	struct irq_desc *desc;
> +
> +	irq_enter();
> +	exit_idle();
> +
> +	desc = irq_to_desc(vmbus_irq);
> +
> +	if (desc)
> +		generic_handle_irq_desc(vmbus_irq, desc);
> +
> +	irq_exit();
> +	set_irq_regs(old_regs);
> +}

This function appears to be dead code when !CONFIG_HYPERV,
because ...

> --- a/arch/x86/kernel/entry_32.S
> +++ b/arch/x86/kernel/entry_32.S
> @@ -1046,11 +1046,18 @@ ENTRY(xen_failsafe_callback)
>  	_ASM_EXTABLE(4b,9b)
>  ENDPROC(xen_failsafe_callback)
>  
> -BUILD_INTERRUPT3(xen_hvm_callback_vector, XEN_HVM_EVTCHN_CALLBACK,
> +BUILD_INTERRUPT3(xen_hvm_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
>  		xen_evtchn_do_upcall)
>  
>  #endif	/* CONFIG_XEN */
>  
> +#if IS_ENABLED(CONFIG_HYPERV)
> +
> +BUILD_INTERRUPT3(hyperv_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
> +	hyperv_vector_handler)
> +
> +#endif /* CONFIG_HYPERV */
> +
>  #ifdef CONFIG_FUNCTION_TRACER
>  #ifdef CONFIG_DYNAMIC_FTRACE
>  

... the consumers here and below are conditional.

I also wonder why arch/x86/kernel/cpu/mshyperv.c is being built
at all when !CONFIG_HYPERV (which would eliminate the need
for the conditional the patch adds to ms_hyperv_init_platform()).

Jan

> diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S
> index d5113c3..c1d01e6 100644
> --- a/arch/x86/kernel/entry_64.S
> +++ b/arch/x86/kernel/entry_64.S
> @@ -1446,11 +1446,16 @@ ENTRY(xen_failsafe_callback)
>  	CFI_ENDPROC
>  END(xen_failsafe_callback)
>  
> -apicinterrupt XEN_HVM_EVTCHN_CALLBACK \
> +apicinterrupt HYPERVISOR_CALLBACK_VECTOR \
>  	xen_hvm_callback_vector xen_evtchn_do_upcall
>  
>  #endif /* CONFIG_XEN */
>  
> +#if IS_ENABLED(CONFIG_HYPERV)
> +apicinterrupt HYPERVISOR_CALLBACK_VECTOR \
> +	hyperv_callback_vector hyperv_vector_handler
> +#endif /* CONFIG_HYPERV */
> +
>  /*
>   * Some functions should be protected against kprobes
>   */

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Wednesday, January 30, 2013 3:53 AM
> To: KY Srinivasan
> Cc: olaf@aepfle.de; bp@alien8.de; apw@canonical.com; x86@kernel.org;
> tglx@linutronix.de; devel@linuxdriverproject.org; gregkh@linuxfoundation.org;
> jasowang@redhat.com; linux-kernel@vger.kernel.org; hpa@zytor.com
> Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> >>> On 30.01.13 at 01:51, "K. Y. Srinivasan" <kys@microsoft.com> wrote:
> > Xen emulates Hyper-V to host enlightened Windows. Looks like this
> > emulation may be turned on by default even for Linux guests. Check and
> > fail Hyper-V detection if we are on Xen.
> >
> > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > ---
> >  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
> >  1 files changed, 7 insertions(+), 0 deletions(-)
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c
> b/arch/x86/kernel/cpu/mshyperv.c
> > index 646d192..4dab317 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
> >  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
> >  		return false;
> >
> > +	/*
> > +	 * Xen emulates Hyper-V to support enlightened Windows.
> > +	 * Check to see first if we are on a Xen Hypervisor.
> > +	 */
> > +	if (xen_cpuid_base())
> > +		return false;
> > +
> >  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
> >  	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);
> 
> I'm not convinced that's the right approach - any hypervisor
> could do similar emulation, and hence you either want to make
> sure you run on Hyper-V (by excluding all others), or you
> tolerate using the emulation (which may require syncing up with
> the other guest implementations so that shared resources don't
> get used by two parties).
> 
> I also wonder whether using the Hyper-V emulation (where
> useful, there might not be anything right now, but this may
> change going forward) when no Xen support is configured
> wouldn't be better than not using anything...

Jan,

Presumably, Hyper-V emulation is only to run enlightened Windows. The issue with
Xen is not that it emulates Hyper-V, but this emulation is turned on while running Linux.
That is the reason I chose to check for Xen. Would you prefer a DMI check for the Hyper-V
platform. 

Regards,

K. Y

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[H. Peter Anvin]

On 01/30/2013 10:12 AM, KY Srinivasan wrote:
>>
>> I'm not convinced that's the right approach - any hypervisor
>> could do similar emulation, and hence you either want to make
>> sure you run on Hyper-V (by excluding all others), or you
>> tolerate using the emulation (which may require syncing up with
>> the other guest implementations so that shared resources don't
>> get used by two parties).
>>
>> I also wonder whether using the Hyper-V emulation (where
>> useful, there might not be anything right now, but this may
>> change going forward) when no Xen support is configured
>> wouldn't be better than not using anything...
>
> Jan,
>
> Presumably, Hyper-V emulation is only to run enlightened Windows. The issue with
> Xen is not that it emulates Hyper-V, but this emulation is turned on while running Linux.
> That is the reason I chose to check for Xen. Would you prefer a DMI check for the Hyper-V
> platform.
>

The real issue here is anyone going to run a kernel:

1. which has Xen support *disabled*,
2. which has HyperV support *enabled*,
3. on top of Xen,
4. with Xen HyperV support enabled.

The fundamental problem seems to be that Xen HyperV support is "just 
good enough to run Windows", but in fact has some serious shortcomings 
which breaks Linux guests that don't have Xen support (if they do, the 
Xen support kicks in instead.)

Am I right?

	-hpa


-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Jan Beulich]

>>> On 30.01.13 at 19:12, KY Srinivasan <kys@microsoft.com> wrote:
> Presumably, Hyper-V emulation is only to run enlightened Windows. The issue 
> with
> Xen is not that it emulates Hyper-V, but this emulation is turned on while 
> running Linux.
> That is the reason I chose to check for Xen. Would you prefer a DMI check 
> for the Hyper-V
> platform. 

I consider DMI checks to be too fragile here - in particular with
the eventual passing through of host DMI attributes to guests,
this sets you up for mistakes. Instead, I would envision you
scanning the whole CPUID range "reserved" for virtualization
(starting at 0x40000000) and see whether you get back
anything other than the Hyper-V identification (much like the
way xen_cpuid_base() scans for the Xen range). Of course
that's under the premise that you're right in assuming Hyper-V
would never emulate any other hypervisor's interface.

Jan

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Thursday, January 31, 2013 2:38 AM
> To: KY Srinivasan
> Cc: olaf@aepfle.de; bp@alien8.de; apw@canonical.com; x86@kernel.org;
> tglx@linutronix.de; devel@linuxdriverproject.org; gregkh@linuxfoundation.org;
> jasowang@redhat.com; linux-kernel@vger.kernel.org; hpa@zytor.com
> Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> >>> On 30.01.13 at 19:12, KY Srinivasan <kys@microsoft.com> wrote:
> > Presumably, Hyper-V emulation is only to run enlightened Windows. The issue
> > with
> > Xen is not that it emulates Hyper-V, but this emulation is turned on while
> > running Linux.
> > That is the reason I chose to check for Xen. Would you prefer a DMI check
> > for the Hyper-V
> > platform.
> 
> I consider DMI checks to be too fragile here - in particular with
> the eventual passing through of host DMI attributes to guests,
> this sets you up for mistakes. Instead, I would envision you
> scanning the whole CPUID range "reserved" for virtualization
> (starting at 0x40000000) and see whether you get back
> anything other than the Hyper-V identification (much like the
> way xen_cpuid_base() scans for the Xen range). Of course
> that's under the premise that you're right in assuming Hyper-V
> would never emulate any other hypervisor's interface.

Agreed; I will make the appropriate changes as you have recommended.

Regards,

K. Y

RE: [PATCH 3/3] X86: Handle Hyper-V vmbus interrupts as special hypervisor interrupts

[KY Srinivasan]

> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Wednesday, January 30, 2013 3:59 AM
> To: KY Srinivasan
> Cc: olaf@aepfle.de; bp@alien8.de; apw@canonical.com; x86@kernel.org;
> tglx@linutronix.de; devel@linuxdriverproject.org; gregkh@linuxfoundation.org;
> jasowang@redhat.com; linux-kernel@vger.kernel.org; hpa@zytor.com
> Subject: Re: [PATCH 3/3] X86: Handle Hyper-V vmbus interrupts as special
> hypervisor interrupts
> 
> >>> On 30.01.13 at 01:51, "K. Y. Srinivasan" <kys@microsoft.com> wrote:
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -14,10 +14,15 @@
> >  #include <linux/time.h>
> >  #include <linux/clocksource.h>
> >  #include <linux/module.h>
> > +#include <linux/hardirq.h>
> > +#include <linux/interrupt.h>
> >  #include <asm/processor.h>
> >  #include <asm/hypervisor.h>
> >  #include <asm/hyperv.h>
> >  #include <asm/mshyperv.h>
> > +#include <asm/desc.h>
> > +#include <asm/idle.h>
> > +#include <asm/irq_regs.h>
> >
> >  struct ms_hyperv_info ms_hyperv;
> >  EXPORT_SYMBOL_GPL(ms_hyperv);
> > @@ -77,6 +82,12 @@ static void __init ms_hyperv_init_platform(void)
> >
> >  	if (ms_hyperv.features & HV_X64_MSR_TIME_REF_COUNT_AVAILABLE)
> >  		clocksource_register_hz(&hyperv_cs, NSEC_PER_SEC/100);
> > +#if IS_ENABLED(CONFIG_HYPERV)
> > +	/*
> > +	 * Setup the IDT for hypervisor callback.
> > +	 */
> > +	alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR,
> hyperv_callback_vector);
> > +#endif
> >  }
> >
> >  const __refconst struct hypervisor_x86 x86_hyper_ms_hyperv = {
> > @@ -85,3 +96,30 @@ const __refconst struct hypervisor_x86
> x86_hyper_ms_hyperv = {
> >  	.init_platform		= ms_hyperv_init_platform,
> >  };
> >  EXPORT_SYMBOL(x86_hyper_ms_hyperv);
> > +
> > +static int vmbus_irq = -1;
> > +static irq_handler_t vmbus_isr;
> > +
> > +void hv_register_vmbus_handler(int irq, irq_handler_t handler)
> > +{
> > +	vmbus_irq = irq;
> > +	vmbus_isr = handler;
> > +}
> > +EXPORT_SYMBOL_GPL(hv_register_vmbus_handler);
> > +
> > +void hyperv_vector_handler(struct pt_regs *regs)
> > +{
> > +	struct pt_regs *old_regs = set_irq_regs(regs);
> > +	struct irq_desc *desc;
> > +
> > +	irq_enter();
> > +	exit_idle();
> > +
> > +	desc = irq_to_desc(vmbus_irq);
> > +
> > +	if (desc)
> > +		generic_handle_irq_desc(vmbus_irq, desc);
> > +
> > +	irq_exit();
> > +	set_irq_regs(old_regs);
> > +}
> 
> This function appears to be dead code when !CONFIG_HYPERV,
> because ...

I will make the necessary adjustments to deal with this.

> 
> > --- a/arch/x86/kernel/entry_32.S
> > +++ b/arch/x86/kernel/entry_32.S
> > @@ -1046,11 +1046,18 @@ ENTRY(xen_failsafe_callback)
> >  	_ASM_EXTABLE(4b,9b)
> >  ENDPROC(xen_failsafe_callback)
> >
> > -BUILD_INTERRUPT3(xen_hvm_callback_vector,
> XEN_HVM_EVTCHN_CALLBACK,
> > +BUILD_INTERRUPT3(xen_hvm_callback_vector,
> HYPERVISOR_CALLBACK_VECTOR,
> >  		xen_evtchn_do_upcall)
> >
> >  #endif	/* CONFIG_XEN */
> >
> > +#if IS_ENABLED(CONFIG_HYPERV)
> > +
> > +BUILD_INTERRUPT3(hyperv_callback_vector,
> HYPERVISOR_CALLBACK_VECTOR,
> > +	hyperv_vector_handler)
> > +
> > +#endif /* CONFIG_HYPERV */
> > +
> >  #ifdef CONFIG_FUNCTION_TRACER
> >  #ifdef CONFIG_DYNAMIC_FTRACE
> >
> 
> ... the consumers here and below are conditional.
> 
> I also wonder why arch/x86/kernel/cpu/mshyperv.c is being built
> at all when !CONFIG_HYPERV (which would eliminate the need
> for the conditional the patch adds to ms_hyperv_init_platform()).

Linux is usable on Hyper-V in full emulation mode with the Hyper-V specific
clocksource plugged in. The current code permits that.

K. Y 

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: devel [mailto:devel-bounces@linuxdriverproject.org] On Behalf Of KY
> Srinivasan
> Sent: Thursday, January 31, 2013 9:46 AM
> To: Jan Beulich
> Cc: olaf@aepfle.de; gregkh@linuxfoundation.org; jasowang@redhat.com;
> x86@kernel.org; linux-kernel@vger.kernel.org; bp@alien8.de; hpa@zytor.com;
> apw@canonical.com; devel@linuxdriverproject.org; tglx@linutronix.de
> Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> 
> 
> > -----Original Message-----
> > From: Jan Beulich [mailto:JBeulich@suse.com]
> > Sent: Thursday, January 31, 2013 2:38 AM
> > To: KY Srinivasan
> > Cc: olaf@aepfle.de; bp@alien8.de; apw@canonical.com; x86@kernel.org;
> > tglx@linutronix.de; devel@linuxdriverproject.org;
> gregkh@linuxfoundation.org;
> > jasowang@redhat.com; linux-kernel@vger.kernel.org; hpa@zytor.com
> > Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> >
> > >>> On 30.01.13 at 19:12, KY Srinivasan <kys@microsoft.com> wrote:
> > > Presumably, Hyper-V emulation is only to run enlightened Windows. The
> issue
> > > with
> > > Xen is not that it emulates Hyper-V, but this emulation is turned on while
> > > running Linux.
> > > That is the reason I chose to check for Xen. Would you prefer a DMI check
> > > for the Hyper-V
> > > platform.
> >
> > I consider DMI checks to be too fragile here - in particular with
> > the eventual passing through of host DMI attributes to guests,
> > this sets you up for mistakes. Instead, I would envision you
> > scanning the whole CPUID range "reserved" for virtualization
> > (starting at 0x40000000) and see whether you get back
> > anything other than the Hyper-V identification (much like the
> > way xen_cpuid_base() scans for the Xen range). Of course
> > that's under the premise that you're right in assuming Hyper-V
> > would never emulate any other hypervisor's interface.
> 
> Agreed; I will make the appropriate changes as you have recommended.

Jan,

Are there any published standards in terms of how the CPUID space should be populated in the range from 0x40000000 to 0x40010000. Specifically, unless the standard mandates that all ranges unused by a given hypervisor would return a known value, how can this code be used to detect the presence of an unknown hypervisor. Hyper-V is going to return the Hyper-V string at 0x40000000. So, I was planning to scan starting at 0x40000100. Clearly, I can check for a specific hypervisor that I know causes a problem for Hyper-V (as I have currently done by checking for Xen). How can I check for the presence of yet to be created Hypervisors that may emulate Hyper-V by scanning the CPUID space. I am almost tempted to say that Xen is the special case and the patch I have submitted addresses that. If a new (or existing hypervisor) plans to do what Xen is doing, perhaps we can dissuade them from doing that or we can fix that within the general framework we have here.

Regards,

K. Y
> _______________________________________________
> devel mailing list
> devel@linuxdriverproject.org
> http://driverdev.linuxdriverproject.org/mailman/listinfo/devel
> 

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Jan Beulich]

>>> On 31.01.13 at 16:53, KY Srinivasan <kys@microsoft.com> wrote:
> Are there any published standards in terms of how the CPUID space should be 
> populated in the range from 0x40000000 to 0x40010000. Specifically, unless 

I recall having seen this range being marked as reserved for
hypervisor use somewhere, but I don't remember where that was.

> the standard mandates that all ranges unused by a given hypervisor would 
> return a known value, how can this code be used to detect the presence of an 
> unknown hypervisor. Hyper-V is going to return the Hyper-V string at 
> 0x40000000. So, I was planning to scan starting at 0x40000100. Clearly, I can 
> check for a specific hypervisor that I know causes a problem for Hyper-V (as I 
> have currently done by checking for Xen). How can I check for the presence of 
> yet to be created Hypervisors that may emulate Hyper-V by scanning the CPUID 
> space. I am almost tempted to say that Xen is the special case and the patch 
> I have submitted addresses that. If a new (or existing hypervisor) plans to 
> do what Xen is doing, perhaps we can dissuade them from doing that or we can 
> fix that within the general framework we have here.

I'd simply preset ECX, EDX, and EBX to zero, and check whether
they change on any EAX input divisible by 256, skipping the one
range you know Hyper-V sits in.

Jan

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[H. Peter Anvin]

On 01/31/2013 08:45 AM, Jan Beulich wrote:
>>>> On 31.01.13 at 16:53, KY Srinivasan <kys@microsoft.com> wrote:
>> Are there any published standards in terms of how the CPUID space should be
>> populated in the range from 0x40000000 to 0x40010000. Specifically, unless
>
> I recall having seen this range being marked as reserved for
> hypervisor use somewhere, but I don't remember where that was.
>

AFAIK Intel reserves 0x40000000 to 0x4FFFFFFF for hypervisor use. 
Typically multivendor CPUID utilities expect "blocks" of 64K but that is 
not in any spec that I know of.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[H. Peter Anvin]

On 01/30/2013 12:53 AM, Jan Beulich wrote:
> 
> I'm not convinced that's the right approach - any hypervisor
> could do similar emulation, and hence you either want to make
> sure you run on Hyper-V (by excluding all others), or you
> tolerate using the emulation (which may require syncing up with
> the other guest implementations so that shared resources don't
> get used by two parties).
> 
> I also wonder whether using the Hyper-V emulation (where
> useful, there might not be anything right now, but this may
> change going forward) when no Xen support is configured
> wouldn't be better than not using anything...
> 

I'm confused about "the right approach" here is.  As far as I
understand, this only can affect a Xen guest where HyperV guest support
is enabled but not Xen support, and only because Xen emulates HyperV but
does so incorrectly.

This is a Xen bug, and as such it makes sense to reject Xen
specifically.  If another hypervisor emulates HyperV and does so
correctly there seems to be no reason to reject it.

	-hpa

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Stefano Stabellini]

On Thu, 31 Jan 2013, H. Peter Anvin wrote:
> On 01/30/2013 12:53 AM, Jan Beulich wrote:
> > 
> > I'm not convinced that's the right approach - any hypervisor
> > could do similar emulation, and hence you either want to make
> > sure you run on Hyper-V (by excluding all others), or you
> > tolerate using the emulation (which may require syncing up with
> > the other guest implementations so that shared resources don't
> > get used by two parties).
> > 
> > I also wonder whether using the Hyper-V emulation (where
> > useful, there might not be anything right now, but this may
> > change going forward) when no Xen support is configured
> > wouldn't be better than not using anything...
> > 
> 
> I'm confused about "the right approach" here is.  As far as I
> understand, this only can affect a Xen guest where HyperV guest support
> is enabled but not Xen support, and only because Xen emulates HyperV but
> does so incorrectly.
> 
> This is a Xen bug, and as such it makes sense to reject Xen
> specifically.  If another hypervisor emulates HyperV and does so
> correctly there seems to be no reason to reject it.

I don't think so.

AFAIK originally there were features exported as flags and Xen doesn't
turn on the flags that correspond to features that are not implemented.
The problem here is that Hyper-V is about to introduce a feature without
a flag that is not implemented by Xen (see "X86: Deliver Hyper-V
interrupts on a separate IDT vector").
K.Y. please confirm if I got this right.

If I were the Microsoft engineer implementing this feature, no matter
what Xen does or does not, I would also make sure that there is a
corresponding flag for it, because in my experience they avoid future
headaches.
I wonder what happens if you run Linux with Hyper-V support on an old
Hyper-V host that doesn't support vector injection.

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Stefano Stabellini [mailto:stefano.stabellini@eu.citrix.com]
> Sent: Friday, February 01, 2013 8:20 AM
> To: H. Peter Anvin
> Cc: Jan Beulich; KY Srinivasan; olaf@aepfle.de; bp@alien8.de;
> apw@canonical.com; x86@kernel.org; tglx@linutronix.de;
> devel@linuxdriverproject.org; gregkh@linuxfoundation.org;
> jasowang@redhat.com; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> On Thu, 31 Jan 2013, H. Peter Anvin wrote:
> > On 01/30/2013 12:53 AM, Jan Beulich wrote:
> > >
> > > I'm not convinced that's the right approach - any hypervisor
> > > could do similar emulation, and hence you either want to make
> > > sure you run on Hyper-V (by excluding all others), or you
> > > tolerate using the emulation (which may require syncing up with
> > > the other guest implementations so that shared resources don't
> > > get used by two parties).
> > >
> > > I also wonder whether using the Hyper-V emulation (where
> > > useful, there might not be anything right now, but this may
> > > change going forward) when no Xen support is configured
> > > wouldn't be better than not using anything...
> > >
> >
> > I'm confused about "the right approach" here is.  As far as I
> > understand, this only can affect a Xen guest where HyperV guest support
> > is enabled but not Xen support, and only because Xen emulates HyperV but
> > does so incorrectly.
> >
> > This is a Xen bug, and as such it makes sense to reject Xen
> > specifically.  If another hypervisor emulates HyperV and does so
> > correctly there seems to be no reason to reject it.
> 
> I don't think so.
> 
> AFAIK originally there were features exported as flags and Xen doesn't
> turn on the flags that correspond to features that are not implemented.
> The problem here is that Hyper-V is about to introduce a feature without
> a flag that is not implemented by Xen (see "X86: Deliver Hyper-V
> interrupts on a separate IDT vector").
> K.Y. please confirm if I got this right.

I am not sure I can agree with you here. There are two discriminating factors
here: (a) Hypervisor check and (b) Feature check. Not every feature of the
hypervisor can be surfaced as feature bit and furthermore, just because a feature
bit is turned on, it does not necessarily mean that the feature is to be used. For instance,
let us say that Windows guests begin to use the "partition counter" and Xen chooses
to implement that to better support Windows. This does not mean that while hosting
Linux on Xen, you want to plug in a clock source based on the emulated
"partition counter". This is what would happen in the code we have today.

Other Hypervisors emulating Hyper-V do not have this problem and Xen would not either
if the emulation bit is selectively turned on (only while running Windows) or if Xen were allowed
to check first ahead of Hyper-V (unconditionally) in the hypervisor detection code. As Peter pointed out, we 
have this problem because of the unique situation with Xen.

In any event, I am not going to further argue this issue; this last round of patches I sent out,
fixes the issue for Xen. Jan wants me to make this check more general. While I don't think
we need to do that, I will see if I can do it. I am checking to see if MSFT guarantees that Hyper-V
would initialize the unused CPUID space to 0. If this is the case, I will implement the check
Jan has suggested; if not, we have to live with the Xen specific check that I currently have.

> 
> If I were the Microsoft engineer implementing this feature, no matter
> what Xen does or does not, I would also make sure that there is a
> corresponding flag for it, because in my experience they avoid future
> headaches.
> I wonder what happens if you run Linux with Hyper-V support on an old
> Hyper-V host that doesn't support vector injection.
> 

To answer your specific question, this feature of being able to distribute vmbus 
interrupt load across all VCPUs in the guest is a win8 and beyond feature. On prior
hosts, all interrupts will be delivered on the boot CPU. VMBUS, as part of connecting with
the hosts determines host supported protocol version and decides how it wants to
program the hypervisor with regards to interrupt delivery. Even though we might setup
an IDT entry for delivering the hypervisor interrupt, if the host is a pre-win8 host, the vmbus
driver will program the hypervisor to deliver the interrupt on the boot CPU via a legacy interrupt
vector.

Regards,

K. Y

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: devel [mailto:devel-bounces@linuxdriverproject.org] On Behalf Of KY
> Srinivasan
> Sent: Friday, February 01, 2013 10:11 AM
> To: Stefano Stabellini; H. Peter Anvin
> Cc: olaf@aepfle.de; gregkh@linuxfoundation.org; jasowang@redhat.com;
> x86@kernel.org; linux-kernel@vger.kernel.org; bp@alien8.de; Jan Beulich;
> apw@canonical.com; devel@linuxdriverproject.org; tglx@linutronix.de
> Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> 
> 
> > -----Original Message-----
> > From: Stefano Stabellini [mailto:stefano.stabellini@eu.citrix.com]
> > Sent: Friday, February 01, 2013 8:20 AM
> > To: H. Peter Anvin
> > Cc: Jan Beulich; KY Srinivasan; olaf@aepfle.de; bp@alien8.de;
> > apw@canonical.com; x86@kernel.org; tglx@linutronix.de;
> > devel@linuxdriverproject.org; gregkh@linuxfoundation.org;
> > jasowang@redhat.com; linux-kernel@vger.kernel.org
> > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> >
> > On Thu, 31 Jan 2013, H. Peter Anvin wrote:
> > > On 01/30/2013 12:53 AM, Jan Beulich wrote:
> > > >
> > > > I'm not convinced that's the right approach - any hypervisor
> > > > could do similar emulation, and hence you either want to make
> > > > sure you run on Hyper-V (by excluding all others), or you
> > > > tolerate using the emulation (which may require syncing up with
> > > > the other guest implementations so that shared resources don't
> > > > get used by two parties).
> > > >
> > > > I also wonder whether using the Hyper-V emulation (where
> > > > useful, there might not be anything right now, but this may
> > > > change going forward) when no Xen support is configured
> > > > wouldn't be better than not using anything...
> > > >
> > >
> > > I'm confused about "the right approach" here is.  As far as I
> > > understand, this only can affect a Xen guest where HyperV guest support
> > > is enabled but not Xen support, and only because Xen emulates HyperV but
> > > does so incorrectly.
> > >
> > > This is a Xen bug, and as such it makes sense to reject Xen
> > > specifically.  If another hypervisor emulates HyperV and does so
> > > correctly there seems to be no reason to reject it.
> >
> > I don't think so.
> >
> > AFAIK originally there were features exported as flags and Xen doesn't
> > turn on the flags that correspond to features that are not implemented.
> > The problem here is that Hyper-V is about to introduce a feature without
> > a flag that is not implemented by Xen (see "X86: Deliver Hyper-V
> > interrupts on a separate IDT vector").
> > K.Y. please confirm if I got this right.
> 
> I am not sure I can agree with you here. There are two discriminating factors
> here: (a) Hypervisor check and (b) Feature check. Not every feature of the
> hypervisor can be surfaced as feature bit and furthermore, just because a
> feature
> bit is turned on, it does not necessarily mean that the feature is to be used. For
> instance,
> let us say that Windows guests begin to use the "partition counter" and Xen
> chooses
> to implement that to better support Windows. This does not mean that while
> hosting
> Linux on Xen, you want to plug in a clock source based on the emulated
> "partition counter". This is what would happen in the code we have today.
> 
> Other Hypervisors emulating Hyper-V do not have this problem and Xen would
> not either
> if the emulation bit is selectively turned on (only while running Windows) or if
> Xen were allowed
> to check first ahead of Hyper-V (unconditionally) in the hypervisor detection
> code. As Peter pointed out, we
> have this problem because of the unique situation with Xen.
> 
> In any event, I am not going to further argue this issue; this last round of patches I
> sent out,
> fixes the issue for Xen. Jan wants me to make this check more general. While I
> don't think
> we need to do that, I will see if I can do it. I am checking to see if MSFT
> guarantees that Hyper-V
> would initialize the unused CPUID space to 0. If this is the case, I will implement
> the check
> Jan has suggested; if not, we have to live with the Xen specific check that I
> currently have.

I checked with the Hyper-V guys and I am told that there is no guarantee that Hyper-V
would not use some other range in the CPUID space in the future. So, I will keep the Xen
specific check that I had in this version. I will add the appropriate compilation switches to
take care of the "dead code" and resubmit the patches.

Regards,


K. Y
> 
> >
> > If I were the Microsoft engineer implementing this feature, no matter
> > what Xen does or does not, I would also make sure that there is a
> > corresponding flag for it, because in my experience they avoid future
> > headaches.
> > I wonder what happens if you run Linux with Hyper-V support on an old
> > Hyper-V host that doesn't support vector injection.
> >
> 
> To answer your specific question, this feature of being able to distribute vmbus
> interrupt load across all VCPUs in the guest is a win8 and beyond feature. On prior
> hosts, all interrupts will be delivered on the boot CPU. VMBUS, as part of
> connecting with
> the hosts determines host supported protocol version and decides how it wants
> to
> program the hypervisor with regards to interrupt delivery. Even though we might
> setup
> an IDT entry for delivering the hypervisor interrupt, if the host is a pre-win8 host,
> the vmbus
> driver will program the hypervisor to deliver the interrupt on the boot CPU via a
> legacy interrupt
> vector.
> 
> Regards,
> 
> K. Y
> 
> _______________________________________________
> devel mailing list
> devel@linuxdriverproject.org
> http://driverdev.linuxdriverproject.org/mailman/listinfo/devel
> 

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: devel [mailto:devel-bounces@linuxdriverproject.org] On Behalf Of KY
> Srinivasan
> Sent: Thursday, April 18, 2013 9:23 AM
> To: Michael S. Tsirkin
> Cc: olaf@aepfle.de; gregkh@linuxfoundation.org; jasowang@redhat.com;
> x86@kernel.org; linux-kernel@vger.kernel.org; bp@alien8.de; Jan Beulich;
> hpa@zytor.com; apw@canonical.com; devel@linuxdriverproject.org;
> tglx@linutronix.de
> Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> 
> 
> > -----Original Message-----
> > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > Sent: Thursday, April 18, 2013 3:48 AM
> > To: KY Srinivasan
> > Cc: Jan Beulich; olaf@aepfle.de; bp@alien8.de; apw@canonical.com;
> > x86@kernel.org; tglx@linutronix.de; devel@linuxdriverproject.org;
> > gregkh@linuxfoundation.org; jasowang@redhat.com; linux-
> > kernel@vger.kernel.org; hpa@zytor.com
> > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> >
> > On Wed, Apr 17, 2013 at 04:28:36PM +0000, KY Srinivasan wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Jan Beulich [mailto:JBeulich@suse.com]
> > > > Sent: Wednesday, April 17, 2013 11:51 AM
> > > > To: KY Srinivasan; Michael S. Tsirkin
> > > > Cc: olaf@aepfle.de; bp@alien8.de; apw@canonical.com; x86@kernel.org;
> > > > tglx@linutronix.de; devel@linuxdriverproject.org;
> > gregkh@linuxfoundation.org;
> > > > jasowang@redhat.com; linux-kernel@vger.kernel.org; hpa@zytor.com
> > > > Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-
> V
> > > >
> > > > >>> On 17.04.13 at 17:31, KY Srinivasan <kys@microsoft.com> wrote:
> > > > > If Xen were to change where it would not unconditionally emulate Hyper-
> V,
> > I
> > > > > would not be opposed to taking
> > > > > this check out.
> > > >
> > > > But it doesn't do this unconditionally, only upon admin request.
> > >
> > > >From the discussion we had a couple of months ago, the default setting was
> to
> > enable
> > > Hyper-V emulation for all guests. If this is not the case, we ought to be able to
> > drop this.
> > > However, I think it is not reasonable to add additional checks (in addition to
> > hypervisor check)
> > > to customize the run-time in the guest for the specific Hypervisor.
> > >
> > > Regards,
> > >
> > > K. Y
> >
> > Parse error. What are you trying to say? I'm just saying it's best to do
> > things in the way that make it possible for Xen to implement hyperv in a
> > more complete way in the future and have things just work
> > and in a way that does not change guest/hypervisor interface.
> 
> There is no error to parse here. I need to setup the vmbus interrupt before I
> can contact the host. As I said I am fine with dropping this check if Xen behavior is
> changed appropriately. We have a mechanism for hypervisor detection and a
> mechanism for discovering features within a hypervisor. Once we confirm the
> existence of a specific hypervisor, we ought to be able to do what is needed for
> that hypervisor (unless the hypervisor feature flag precludes it) without having to
> go through hoops to discover if we are in a hypervisor emulation mode where
> the emulation
> may not be complete. In this particular case, the need for VMBUS related
> infrastructure cannot be
> currently determined via the feature detection mechanism.
> 
> For features that cannot be detected via CPUID feature bits, we can use some
> other CPUID based
> mechanism to discriminate. For instance in the case under discussion, I could
> install
> hyperv_callback_vector only if we are on an environment that supports the
> vmbus infrastructure.
> Unfortunately there is no positive test for this. The best we can do is a negative
> test. I could move the
> Xen check and conditionally install the hyperv_callback vector.

Michael,

I think I can address the problem you have raised. I will send a patch out soon.

Regards,

K. Y
> 
> Regards,
> 
> K. Y
> 
> 
> >
> > > >
> > > > Jan
> > > >
> > > >
> > >
> >
> 
> 
> _______________________________________________
> devel mailing list
> devel@linuxdriverproject.org
> http://driverdev.linuxdriverproject.org/mailman/listinfo/devel
> 

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Michael S. Tsirkin [mailto:mst@redhat.com]
> Sent: Thursday, April 18, 2013 3:48 AM
> To: KY Srinivasan
> Cc: Jan Beulich; olaf@aepfle.de; bp@alien8.de; apw@canonical.com;
> x86@kernel.org; tglx@linutronix.de; devel@linuxdriverproject.org;
> gregkh@linuxfoundation.org; jasowang@redhat.com; linux-
> kernel@vger.kernel.org; hpa@zytor.com
> Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> On Wed, Apr 17, 2013 at 04:28:36PM +0000, KY Srinivasan wrote:
> >
> >
> > > -----Original Message-----
> > > From: Jan Beulich [mailto:JBeulich@suse.com]
> > > Sent: Wednesday, April 17, 2013 11:51 AM
> > > To: KY Srinivasan; Michael S. Tsirkin
> > > Cc: olaf@aepfle.de; bp@alien8.de; apw@canonical.com; x86@kernel.org;
> > > tglx@linutronix.de; devel@linuxdriverproject.org;
> gregkh@linuxfoundation.org;
> > > jasowang@redhat.com; linux-kernel@vger.kernel.org; hpa@zytor.com
> > > Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > >
> > > >>> On 17.04.13 at 17:31, KY Srinivasan <kys@microsoft.com> wrote:
> > > > If Xen were to change where it would not unconditionally emulate Hyper-V,
> I
> > > > would not be opposed to taking
> > > > this check out.
> > >
> > > But it doesn't do this unconditionally, only upon admin request.
> >
> > >From the discussion we had a couple of months ago, the default setting was to
> enable
> > Hyper-V emulation for all guests. If this is not the case, we ought to be able to
> drop this.
> > However, I think it is not reasonable to add additional checks (in addition to
> hypervisor check)
> > to customize the run-time in the guest for the specific Hypervisor.
> >
> > Regards,
> >
> > K. Y
> 
> Parse error. What are you trying to say? I'm just saying it's best to do
> things in the way that make it possible for Xen to implement hyperv in a
> more complete way in the future and have things just work
> and in a way that does not change guest/hypervisor interface.

There is no error to parse here. I need to setup the vmbus interrupt before I
can contact the host. As I said I am fine with dropping this check if Xen behavior is
changed appropriately. We have a mechanism for hypervisor detection and a 
mechanism for discovering features within a hypervisor. Once we confirm the
existence of a specific hypervisor, we ought to be able to do what is needed for
that hypervisor (unless the hypervisor feature flag precludes it) without having to
go through hoops to discover if we are in a hypervisor emulation mode where the emulation
may not be complete. In this particular case, the need for VMBUS related infrastructure cannot be
currently determined via the feature detection mechanism. 

For features that cannot be detected via CPUID feature bits, we can use some other CPUID based
mechanism to discriminate. For instance in the case under discussion, I could  install
hyperv_callback_vector only if we are on an environment that supports the vmbus infrastructure.
Unfortunately there is no positive test for this. The best we can do is a negative test. I could move the 
Xen check and conditionally install the hyperv_callback vector. 

Regards,

K. Y
 
 
> 
> > >
> > > Jan
> > >
> > >
> >
> 

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Michael S. Tsirkin]

On Wed, Apr 17, 2013 at 04:28:36PM +0000, KY Srinivasan wrote:
> 
> 
> > -----Original Message-----
> > From: Jan Beulich [mailto:JBeulich@suse.com]
> > Sent: Wednesday, April 17, 2013 11:51 AM
> > To: KY Srinivasan; Michael S. Tsirkin
> > Cc: olaf@aepfle.de; bp@alien8.de; apw@canonical.com; x86@kernel.org;
> > tglx@linutronix.de; devel@linuxdriverproject.org; gregkh@linuxfoundation.org;
> > jasowang@redhat.com; linux-kernel@vger.kernel.org; hpa@zytor.com
> > Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > 
> > >>> On 17.04.13 at 17:31, KY Srinivasan <kys@microsoft.com> wrote:
> > > If Xen were to change where it would not unconditionally emulate Hyper-V, I
> > > would not be opposed to taking
> > > this check out.
> > 
> > But it doesn't do this unconditionally, only upon admin request.
> 
> >From the discussion we had a couple of months ago, the default setting was to enable
> Hyper-V emulation for all guests. If this is not the case, we ought to be able to drop this.
> However, I think it is not reasonable to add additional checks (in addition to hypervisor check)
> to customize the run-time in the guest for the specific Hypervisor.
> 
> Regards,
> 
> K. Y

Parse error. What are you trying to say? I'm just saying it's best to do
things in the way that make it possible for Xen to implement hyperv in a
more complete way in the future and have things just work
and in a way that does not change guest/hypervisor interface.

> > 
> > Jan
> > 
> > 
> 

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Wednesday, April 17, 2013 11:51 AM
> To: KY Srinivasan; Michael S. Tsirkin
> Cc: olaf@aepfle.de; bp@alien8.de; apw@canonical.com; x86@kernel.org;
> tglx@linutronix.de; devel@linuxdriverproject.org; gregkh@linuxfoundation.org;
> jasowang@redhat.com; linux-kernel@vger.kernel.org; hpa@zytor.com
> Subject: RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> >>> On 17.04.13 at 17:31, KY Srinivasan <kys@microsoft.com> wrote:
> > If Xen were to change where it would not unconditionally emulate Hyper-V, I
> > would not be opposed to taking
> > this check out.
> 
> But it doesn't do this unconditionally, only upon admin request.

>From the discussion we had a couple of months ago, the default setting was to enable
Hyper-V emulation for all guests. If this is not the case, we ought to be able to drop this.
However, I think it is not reasonable to add additional checks (in addition to hypervisor check)
to customize the run-time in the guest for the specific Hypervisor.

Regards,

K. Y
> 
> Jan
> 
> 

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Jan Beulich]

>>> On 17.04.13 at 17:31, KY Srinivasan <kys@microsoft.com> wrote:
> If Xen were to change where it would not unconditionally emulate Hyper-V, I 
> would not be opposed to taking
> this check out.

But it doesn't do this unconditionally, only upon admin request.

Jan

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Michael S. Tsirkin [mailto:mst@redhat.com]
> Sent: Wednesday, April 17, 2013 9:49 AM
> To: KY Srinivasan
> Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> JBeulich@suse.com; bp@alien8.de
> Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> On Wed, Apr 17, 2013 at 02:34:57PM +0000, KY Srinivasan wrote:
> >
> >
> > > -----Original Message-----
> > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > Sent: Wednesday, April 17, 2013 9:25 AM
> > > To: KY Srinivasan
> > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> kernel@vger.kernel.org;
> > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > JBeulich@suse.com; bp@alien8.de
> > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > >
> > > On Wed, Apr 17, 2013 at 02:12:12PM +0000, KY Srinivasan wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > > > Sent: Wednesday, April 17, 2013 8:49 AM
> > > > > To: KY Srinivasan
> > > > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> > > kernel@vger.kernel.org;
> > > > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > > > JBeulich@suse.com; bp@alien8.de
> > > > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of
> Hyper-V
> > > > >
> > > > > On Wed, Apr 17, 2013 at 01:20:58PM +0000, KY Srinivasan wrote:
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > > > > > Sent: Wednesday, April 17, 2013 3:06 AM
> > > > > > > To: KY Srinivasan
> > > > > > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> > > > > kernel@vger.kernel.org;
> > > > > > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > > > > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > > > > > JBeulich@suse.com; bp@alien8.de
> > > > > > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of
> > > Hyper-V
> > > > > > >
> > > > > > > On Sun, Feb 03, 2013 at 05:22:38PM -0800, K. Y. Srinivasan wrote:
> > > > > > > > Xen emulates Hyper-V to host enlightened Windows. Looks like this
> > > > > > > > emulation may be turned on by default even for Linux guests. Check
> > > and
> > > > > > > > fail Hyper-V detection if we are on Xen.
> > > > > > > >
> > > > > > > > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > > > > > >
> > > > > > > I'm very curious what's the specific bug that is fixed here?
> > > > > > > I went over the original discussion in
> > > > > > > https://patchwork.kernel.org/patch/2064331/
> > > > > > > and that's still not clear to me. Is there a configuration
> > > > > > > that is broken without this patch but starts working with
> > > > > > > this patch?
> > > > > > >
> > > > > > > It seems that one might want to use hyper-v emulation e.g. to test
> > > > > > > hyper-v code without using windows, so the functionality
> > > > > > > that this patch disables is not completely useless,
> > > > > > > so there should be a good reason for disabling it.
> > > > > > >
> > > > > > > Could you enlighten me please? :)
> > > > > >
> > > > > > Michael,
> > > > > >
> > > > > > If I recall correctly, the issue here was that Xen was enabling
> > > > > > Hyper-V emulation un-conditionally even for Linux guests.
> > > > >
> > > > > Okay so basically some Xen specific strangeness.
> > > > >
> > > > > > Clearly,
> > > > > > this was not the intention to begin with. This check was added to
> > > > > > ensure that while hosting Linux we would not un-intentionally enable
> > > > > > Hyper-V emulation in Xen.
> > > > >
> > > > > What if you want to intentionally enable Hyper-V emulation in Xen?
> > > > > There's no way to do this now, right?
> > > > >
> > > > > > Not all Hyper-V features are tagged with a
> > > > > > CPUID feature bit and so it was agreed that it would be better to
> > > > > > disable Hyper-V emulation when hosting Linux on Xen.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > K. Y
> > > > >
> > > > > I was just asking whether there are any specific examples of a bug
> > > > > or is this change done just in case?
> > > >
> > > > Ideally, if Hyper-V could be emulated in a plug compatible way, in that the
> guest
> > > would not know
> > > > the difference as to whether Hyper-V was being emulated or indeed it was
> > > Hyper-V underneath, I would agree with you.
> > > > Even in this case, I would argue that we would somehow give preference to
> the
> > > native implementation rather
> > > > than the emulated environment. Having done the initial emulation of
> Hyper-V
> > > on Xen, I know the current emulation of
> > > > Hyper-V is restricted to emulating a few features advertised via the CPUID
> bits.
> > > If I recall correctly, Xen did not implement
> > > > the enlightened TLB flush scheme. Furthermore, none of the Hyper-V
> > > emulations implement the I/O related infrastructure.
> > > > For Linux to run efficiently on Hyper-V, the I/O paths are significantly more
> > > important than what Hyper-V emulation there is
> > > > in Xen and I suspect in KVM. So, I don't think we want to run Linux on
> Xen/KVM
> > > that is emulating Hyper-V (cpuids).
> > >
> > > No, it's a Xen specific bug.  KVM in a hyper-v mode still emulates PCI
> > > so can still use KVM PV devices which are all PCI devices. So there's no
> problem
> > > there.
> >
> > Even Xen does not have a problem here; Xen would install its PV drivers in the
> guest.
> > So, the I/O infrastructure that would be installed in the guest would be based
> on the
> > host.
> > >
> > > Also, why override this in the guest? With time Xen hyper-v emulation
> > > might improve, meanwhile it's always possible to disable this in
> > > the hypervisor.
> > >
> > > > The specific case I ran into that required this patch was when I moved the
> > > VMBUS interrupts to be delivered via a special
> > > > hypervisor vector (to support full interrupt distribution amongst all VCPUs in
> > > the guest). This feature is not tagged in the CPUID
> > > > feature bits and without this check, I would install an interrupt handler for
> the
> > > hypervisor vector that clearly did not make
> > > > any sense in anything other than Hyper-V. I suspect there will be more such
> > > situations as we go forward.
> > > >
> > > > Regards,
> > > >
> > > > K. Y
> > >
> > > Why? Because there's no VMBUS on Xen? So check whether VMBUS exists?
> > On Linux, Hyper-V detection is used to load hyper-V related drivers (VMBUS
> etc.). For the case in
> > point, it was not clear to me how I could install the vmbus interrupt handler in a
> way that would
> > make sense when Xen was unconditionally emulating Hyper-V.
> >
> > Regards,
> >
> > K. Y
> 
> For example, check whether there is anything on the vmbus?
I think the right solution to this problem is to ensure that the guest does not have to do what you are proposing.
Based on the hypervisor detection, we load the drivers and setup the infrastructure to support these drivers.
There is no independent way of verifying if the host is Windows other than the hypervisor detection code.

If Xen were to change where it would not unconditionally emulate Hyper-V, I would not be opposed to taking
this check out.

K. Y
> 
> > >
> > > > >
> > > > > > >
> > > > > > > > ---
> > > > > > > >  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
> > > > > > > >  1 files changed, 7 insertions(+), 0 deletions(-)
> > > > > > > >
> > > > > > > > diff --git a/arch/x86/kernel/cpu/mshyperv.c
> > > > > > > b/arch/x86/kernel/cpu/mshyperv.c
> > > > > > > > index 646d192..4dab317 100644
> > > > > > > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > > > > > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > > > > > > @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
> > > > > > > >  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
> > > > > > > >  		return false;
> > > > > > > >
> > > > > > > > +	/*
> > > > > > > > +	 * Xen emulates Hyper-V to support enlightened Windows.
> > > > > > > > +	 * Check to see first if we are on a Xen Hypervisor.
> > > > > > > > +	 */
> > > > > > > > +	if (xen_cpuid_base())
> > > > > > > > +		return false;
> > > > > > > > +
> > > > > > > >  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
> > > > > > > >  	      &eax, &hyp_signature[0], &hyp_signature[1],
> > > &hyp_signature[2]);
> > > > > > > >
> > > > > > > > --
> > > > > > > > 1.7.4.1
> > > > > > > >
> > > > > > > > --
> > > > > > > > To unsubscribe from this list: send the line "unsubscribe linux-
> kernel" in
> > > > > > > > the body of a message to majordomo@vger.kernel.org
> > > > > > > > More majordomo info at  http://vger.kernel.org/majordomo-
> info.html
> > > > > > > > Please read the FAQ at  http://www.tux.org/lkml/
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[H. Peter Anvin]

On 04/17/2013 07:09 AM, Jan Beulich wrote:
> 
> Iirc it was/is XenServer which enable Hyper-V emulation for all HVM
> guests, which clearly is the wrong thing. I was personally also not
> really in agreement with the override in the kernel, but it was
> decided to do it that way at that point in time. As a consequence,
> I don't object this to be reverted.
> 

I was not exactly happy with the idea either.  I'm sorry if I mixed up
Xen and KVM.

Furthermore, as I recall, it only affected kernels where Xen was
compiled out, or something like that.

It would definitely be best to fix the HyperV driver to be more
robust... that may be important across HyperV releases, too.

	-hpa


-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Michael S. Tsirkin [mailto:mst@redhat.com]
> Sent: Wednesday, April 17, 2013 9:25 AM
> To: KY Srinivasan
> Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> JBeulich@suse.com; bp@alien8.de
> Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> On Wed, Apr 17, 2013 at 02:12:12PM +0000, KY Srinivasan wrote:
> >
> >
> > > -----Original Message-----
> > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > Sent: Wednesday, April 17, 2013 8:49 AM
> > > To: KY Srinivasan
> > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> kernel@vger.kernel.org;
> > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > JBeulich@suse.com; bp@alien8.de
> > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > >
> > > On Wed, Apr 17, 2013 at 01:20:58PM +0000, KY Srinivasan wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > > > Sent: Wednesday, April 17, 2013 3:06 AM
> > > > > To: KY Srinivasan
> > > > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> > > kernel@vger.kernel.org;
> > > > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > > > JBeulich@suse.com; bp@alien8.de
> > > > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of
> Hyper-V
> > > > >
> > > > > On Sun, Feb 03, 2013 at 05:22:38PM -0800, K. Y. Srinivasan wrote:
> > > > > > Xen emulates Hyper-V to host enlightened Windows. Looks like this
> > > > > > emulation may be turned on by default even for Linux guests. Check
> and
> > > > > > fail Hyper-V detection if we are on Xen.
> > > > > >
> > > > > > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > > > >
> > > > > I'm very curious what's the specific bug that is fixed here?
> > > > > I went over the original discussion in
> > > > > https://patchwork.kernel.org/patch/2064331/
> > > > > and that's still not clear to me. Is there a configuration
> > > > > that is broken without this patch but starts working with
> > > > > this patch?
> > > > >
> > > > > It seems that one might want to use hyper-v emulation e.g. to test
> > > > > hyper-v code without using windows, so the functionality
> > > > > that this patch disables is not completely useless,
> > > > > so there should be a good reason for disabling it.
> > > > >
> > > > > Could you enlighten me please? :)
> > > >
> > > > Michael,
> > > >
> > > > If I recall correctly, the issue here was that Xen was enabling
> > > > Hyper-V emulation un-conditionally even for Linux guests.
> > >
> > > Okay so basically some Xen specific strangeness.
> > >
> > > > Clearly,
> > > > this was not the intention to begin with. This check was added to
> > > > ensure that while hosting Linux we would not un-intentionally enable
> > > > Hyper-V emulation in Xen.
> > >
> > > What if you want to intentionally enable Hyper-V emulation in Xen?
> > > There's no way to do this now, right?
> > >
> > > > Not all Hyper-V features are tagged with a
> > > > CPUID feature bit and so it was agreed that it would be better to
> > > > disable Hyper-V emulation when hosting Linux on Xen.
> > > >
> > > > Regards,
> > > >
> > > > K. Y
> > >
> > > I was just asking whether there are any specific examples of a bug
> > > or is this change done just in case?
> >
> > Ideally, if Hyper-V could be emulated in a plug compatible way, in that the guest
> would not know
> > the difference as to whether Hyper-V was being emulated or indeed it was
> Hyper-V underneath, I would agree with you.
> > Even in this case, I would argue that we would somehow give preference to the
> native implementation rather
> > than the emulated environment. Having done the initial emulation of Hyper-V
> on Xen, I know the current emulation of
> > Hyper-V is restricted to emulating a few features advertised via the CPUID bits.
> If I recall correctly, Xen did not implement
> > the enlightened TLB flush scheme. Furthermore, none of the Hyper-V
> emulations implement the I/O related infrastructure.
> > For Linux to run efficiently on Hyper-V, the I/O paths are significantly more
> important than what Hyper-V emulation there is
> > in Xen and I suspect in KVM. So, I don't think we want to run Linux on Xen/KVM
> that is emulating Hyper-V (cpuids).
> 
> No, it's a Xen specific bug.  KVM in a hyper-v mode still emulates PCI
> so can still use KVM PV devices which are all PCI devices. So there's no problem
> there.

Even Xen does not have a problem here; Xen would install its PV drivers in the guest.
So, the I/O infrastructure that would be installed in the guest would be based on the 
host.
> 
> Also, why override this in the guest? With time Xen hyper-v emulation
> might improve, meanwhile it's always possible to disable this in
> the hypervisor.
> 
> > The specific case I ran into that required this patch was when I moved the
> VMBUS interrupts to be delivered via a special
> > hypervisor vector (to support full interrupt distribution amongst all VCPUs in
> the guest). This feature is not tagged in the CPUID
> > feature bits and without this check, I would install an interrupt handler for the
> hypervisor vector that clearly did not make
> > any sense in anything other than Hyper-V. I suspect there will be more such
> situations as we go forward.
> >
> > Regards,
> >
> > K. Y
> 
> Why? Because there's no VMBUS on Xen? So check whether VMBUS exists?
On Linux, Hyper-V detection is used to load hyper-V related drivers (VMBUS etc.). For the case in
point, it was not clear to me how I could install the vmbus interrupt handler in a way that would
make sense when Xen was unconditionally emulating Hyper-V. 

Regards,

K. Y
> 
> > >
> > > > >
> > > > > > ---
> > > > > >  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
> > > > > >  1 files changed, 7 insertions(+), 0 deletions(-)
> > > > > >
> > > > > > diff --git a/arch/x86/kernel/cpu/mshyperv.c
> > > > > b/arch/x86/kernel/cpu/mshyperv.c
> > > > > > index 646d192..4dab317 100644
> > > > > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > > > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > > > > @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
> > > > > >  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
> > > > > >  		return false;
> > > > > >
> > > > > > +	/*
> > > > > > +	 * Xen emulates Hyper-V to support enlightened Windows.
> > > > > > +	 * Check to see first if we are on a Xen Hypervisor.
> > > > > > +	 */
> > > > > > +	if (xen_cpuid_base())
> > > > > > +		return false;
> > > > > > +
> > > > > >  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
> > > > > >  	      &eax, &hyp_signature[0], &hyp_signature[1],
> &hyp_signature[2]);
> > > > > >
> > > > > > --
> > > > > > 1.7.4.1
> > > > > >
> > > > > > --
> > > > > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > > > > > the body of a message to majordomo@vger.kernel.org
> > > > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > > > > Please read the FAQ at  http://www.tux.org/lkml/
> > > > >
> > > >
> > >
> >
> 

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Michael S. Tsirkin [mailto:mst@redhat.com]
> Sent: Wednesday, April 17, 2013 8:49 AM
> To: KY Srinivasan
> Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> JBeulich@suse.com; bp@alien8.de
> Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> On Wed, Apr 17, 2013 at 01:20:58PM +0000, KY Srinivasan wrote:
> >
> >
> > > -----Original Message-----
> > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > Sent: Wednesday, April 17, 2013 3:06 AM
> > > To: KY Srinivasan
> > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> kernel@vger.kernel.org;
> > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > JBeulich@suse.com; bp@alien8.de
> > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > >
> > > On Sun, Feb 03, 2013 at 05:22:38PM -0800, K. Y. Srinivasan wrote:
> > > > Xen emulates Hyper-V to host enlightened Windows. Looks like this
> > > > emulation may be turned on by default even for Linux guests. Check and
> > > > fail Hyper-V detection if we are on Xen.
> > > >
> > > > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > >
> > > I'm very curious what's the specific bug that is fixed here?
> > > I went over the original discussion in
> > > https://patchwork.kernel.org/patch/2064331/
> > > and that's still not clear to me. Is there a configuration
> > > that is broken without this patch but starts working with
> > > this patch?
> > >
> > > It seems that one might want to use hyper-v emulation e.g. to test
> > > hyper-v code without using windows, so the functionality
> > > that this patch disables is not completely useless,
> > > so there should be a good reason for disabling it.
> > >
> > > Could you enlighten me please? :)
> >
> > Michael,
> >
> > If I recall correctly, the issue here was that Xen was enabling
> > Hyper-V emulation un-conditionally even for Linux guests.
> 
> Okay so basically some Xen specific strangeness.
> 
> > Clearly,
> > this was not the intention to begin with. This check was added to
> > ensure that while hosting Linux we would not un-intentionally enable
> > Hyper-V emulation in Xen.
> 
> What if you want to intentionally enable Hyper-V emulation in Xen?
> There's no way to do this now, right?
> 
> > Not all Hyper-V features are tagged with a
> > CPUID feature bit and so it was agreed that it would be better to
> > disable Hyper-V emulation when hosting Linux on Xen.
> >
> > Regards,
> >
> > K. Y
> 
> I was just asking whether there are any specific examples of a bug
> or is this change done just in case?

Ideally, if Hyper-V could be emulated in a plug compatible way, in that the guest would not know
the difference as to whether Hyper-V was being emulated or indeed it was Hyper-V underneath, I would agree with you.
Even in this case, I would argue that we would somehow give preference to the native implementation rather
than the emulated environment. Having done the initial emulation of Hyper-V on Xen, I know the current emulation of 
Hyper-V is restricted to emulating a few features advertised via the CPUID bits. If I recall correctly, Xen did not implement
the enlightened TLB flush scheme. Furthermore, none of the Hyper-V emulations implement the I/O related infrastructure.
For Linux to run efficiently on Hyper-V, the I/O paths are significantly more important than what Hyper-V emulation there is
in Xen and I suspect in KVM. So, I don't think we want to run Linux on Xen/KVM that is emulating Hyper-V (cpuids).
The specific case I ran into that required this patch was when I moved the VMBUS interrupts to be delivered via a special
hypervisor vector (to support full interrupt distribution amongst all VCPUs in the guest). This feature is not tagged in the CPUID
feature bits and without this check, I would install an interrupt handler for the hypervisor vector that clearly did not make
any sense in anything other than Hyper-V. I suspect there will be more such situations as we go forward.

Regards,

K. Y 
> 
> > >
> > > > ---
> > > >  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
> > > >  1 files changed, 7 insertions(+), 0 deletions(-)
> > > >
> > > > diff --git a/arch/x86/kernel/cpu/mshyperv.c
> > > b/arch/x86/kernel/cpu/mshyperv.c
> > > > index 646d192..4dab317 100644
> > > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > > @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
> > > >  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
> > > >  		return false;
> > > >
> > > > +	/*
> > > > +	 * Xen emulates Hyper-V to support enlightened Windows.
> > > > +	 * Check to see first if we are on a Xen Hypervisor.
> > > > +	 */
> > > > +	if (xen_cpuid_base())
> > > > +		return false;
> > > > +
> > > >  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
> > > >  	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);
> > > >
> > > > --
> > > > 1.7.4.1
> > > >
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > > > the body of a message to majordomo@vger.kernel.org
> > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > > Please read the FAQ at  http://www.tux.org/lkml/
> > >
> >
> 

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Jan Beulich]

>>> On 17.04.13 at 15:01, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, Apr 17, 2013 at 02:52:42PM +0100, Jan Beulich wrote:
>> >>> On 17.04.13 at 15:20, KY Srinivasan <kys@microsoft.com> wrote:
>> > If I recall correctly, the issue here was that Xen was enabling Hyper-V 
>> > emulation un-conditionally even for Linux guests.
>> 
>> To make this a little more precise - Xen is doing so only when the
>> guest config tells it to.
> 
> So it's something explicitly requested for this guest?  Then I don't
> understand why does not make sense to override it in guest.

Iirc it was/is XenServer which enable Hyper-V emulation for all HVM
guests, which clearly is the wrong thing. I was personally also not
really in agreement with the override in the kernel, but it was
decided to do it that way at that point in time. As a consequence,
I don't object this to be reverted.

Jan

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Jan Beulich]

>>> On 17.04.13 at 15:20, KY Srinivasan <kys@microsoft.com> wrote:
> If I recall correctly, the issue here was that Xen was enabling Hyper-V 
> emulation un-conditionally even for Linux guests.

To make this a little more precise - Xen is doing so only when the
guest config tells it to.

Jan

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Michael S. Tsirkin]

On Wed, Apr 17, 2013 at 02:34:57PM +0000, KY Srinivasan wrote:
> 
> 
> > -----Original Message-----
> > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > Sent: Wednesday, April 17, 2013 9:25 AM
> > To: KY Srinivasan
> > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > JBeulich@suse.com; bp@alien8.de
> > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > 
> > On Wed, Apr 17, 2013 at 02:12:12PM +0000, KY Srinivasan wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > > Sent: Wednesday, April 17, 2013 8:49 AM
> > > > To: KY Srinivasan
> > > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> > kernel@vger.kernel.org;
> > > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > > JBeulich@suse.com; bp@alien8.de
> > > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > > >
> > > > On Wed, Apr 17, 2013 at 01:20:58PM +0000, KY Srinivasan wrote:
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > > > > Sent: Wednesday, April 17, 2013 3:06 AM
> > > > > > To: KY Srinivasan
> > > > > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> > > > kernel@vger.kernel.org;
> > > > > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > > > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > > > > JBeulich@suse.com; bp@alien8.de
> > > > > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of
> > Hyper-V
> > > > > >
> > > > > > On Sun, Feb 03, 2013 at 05:22:38PM -0800, K. Y. Srinivasan wrote:
> > > > > > > Xen emulates Hyper-V to host enlightened Windows. Looks like this
> > > > > > > emulation may be turned on by default even for Linux guests. Check
> > and
> > > > > > > fail Hyper-V detection if we are on Xen.
> > > > > > >
> > > > > > > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > > > > >
> > > > > > I'm very curious what's the specific bug that is fixed here?
> > > > > > I went over the original discussion in
> > > > > > https://patchwork.kernel.org/patch/2064331/
> > > > > > and that's still not clear to me. Is there a configuration
> > > > > > that is broken without this patch but starts working with
> > > > > > this patch?
> > > > > >
> > > > > > It seems that one might want to use hyper-v emulation e.g. to test
> > > > > > hyper-v code without using windows, so the functionality
> > > > > > that this patch disables is not completely useless,
> > > > > > so there should be a good reason for disabling it.
> > > > > >
> > > > > > Could you enlighten me please? :)
> > > > >
> > > > > Michael,
> > > > >
> > > > > If I recall correctly, the issue here was that Xen was enabling
> > > > > Hyper-V emulation un-conditionally even for Linux guests.
> > > >
> > > > Okay so basically some Xen specific strangeness.
> > > >
> > > > > Clearly,
> > > > > this was not the intention to begin with. This check was added to
> > > > > ensure that while hosting Linux we would not un-intentionally enable
> > > > > Hyper-V emulation in Xen.
> > > >
> > > > What if you want to intentionally enable Hyper-V emulation in Xen?
> > > > There's no way to do this now, right?
> > > >
> > > > > Not all Hyper-V features are tagged with a
> > > > > CPUID feature bit and so it was agreed that it would be better to
> > > > > disable Hyper-V emulation when hosting Linux on Xen.
> > > > >
> > > > > Regards,
> > > > >
> > > > > K. Y
> > > >
> > > > I was just asking whether there are any specific examples of a bug
> > > > or is this change done just in case?
> > >
> > > Ideally, if Hyper-V could be emulated in a plug compatible way, in that the guest
> > would not know
> > > the difference as to whether Hyper-V was being emulated or indeed it was
> > Hyper-V underneath, I would agree with you.
> > > Even in this case, I would argue that we would somehow give preference to the
> > native implementation rather
> > > than the emulated environment. Having done the initial emulation of Hyper-V
> > on Xen, I know the current emulation of
> > > Hyper-V is restricted to emulating a few features advertised via the CPUID bits.
> > If I recall correctly, Xen did not implement
> > > the enlightened TLB flush scheme. Furthermore, none of the Hyper-V
> > emulations implement the I/O related infrastructure.
> > > For Linux to run efficiently on Hyper-V, the I/O paths are significantly more
> > important than what Hyper-V emulation there is
> > > in Xen and I suspect in KVM. So, I don't think we want to run Linux on Xen/KVM
> > that is emulating Hyper-V (cpuids).
> > 
> > No, it's a Xen specific bug.  KVM in a hyper-v mode still emulates PCI
> > so can still use KVM PV devices which are all PCI devices. So there's no problem
> > there.
> 
> Even Xen does not have a problem here; Xen would install its PV drivers in the guest.
> So, the I/O infrastructure that would be installed in the guest would be based on the 
> host.
> > 
> > Also, why override this in the guest? With time Xen hyper-v emulation
> > might improve, meanwhile it's always possible to disable this in
> > the hypervisor.
> > 
> > > The specific case I ran into that required this patch was when I moved the
> > VMBUS interrupts to be delivered via a special
> > > hypervisor vector (to support full interrupt distribution amongst all VCPUs in
> > the guest). This feature is not tagged in the CPUID
> > > feature bits and without this check, I would install an interrupt handler for the
> > hypervisor vector that clearly did not make
> > > any sense in anything other than Hyper-V. I suspect there will be more such
> > situations as we go forward.
> > >
> > > Regards,
> > >
> > > K. Y
> > 
> > Why? Because there's no VMBUS on Xen? So check whether VMBUS exists?
> On Linux, Hyper-V detection is used to load hyper-V related drivers (VMBUS etc.). For the case in
> point, it was not clear to me how I could install the vmbus interrupt handler in a way that would
> make sense when Xen was unconditionally emulating Hyper-V. 
> 
> Regards,
> 
> K. Y

For example, check whether there is anything on the vmbus?

> > 
> > > >
> > > > > >
> > > > > > > ---
> > > > > > >  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
> > > > > > >  1 files changed, 7 insertions(+), 0 deletions(-)
> > > > > > >
> > > > > > > diff --git a/arch/x86/kernel/cpu/mshyperv.c
> > > > > > b/arch/x86/kernel/cpu/mshyperv.c
> > > > > > > index 646d192..4dab317 100644
> > > > > > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > > > > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > > > > > @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
> > > > > > >  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
> > > > > > >  		return false;
> > > > > > >
> > > > > > > +	/*
> > > > > > > +	 * Xen emulates Hyper-V to support enlightened Windows.
> > > > > > > +	 * Check to see first if we are on a Xen Hypervisor.
> > > > > > > +	 */
> > > > > > > +	if (xen_cpuid_base())
> > > > > > > +		return false;
> > > > > > > +
> > > > > > >  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
> > > > > > >  	      &eax, &hyp_signature[0], &hyp_signature[1],
> > &hyp_signature[2]);
> > > > > > >
> > > > > > > --
> > > > > > > 1.7.4.1
> > > > > > >
> > > > > > > --
> > > > > > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > > > > > > the body of a message to majordomo@vger.kernel.org
> > > > > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > > > > > Please read the FAQ at  http://www.tux.org/lkml/
> > > > > >
> > > > >
> > > >
> > >
> > 
> 

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Michael S. Tsirkin]

On Wed, Apr 17, 2013 at 02:12:12PM +0000, KY Srinivasan wrote:
> 
> 
> > -----Original Message-----
> > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > Sent: Wednesday, April 17, 2013 8:49 AM
> > To: KY Srinivasan
> > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > JBeulich@suse.com; bp@alien8.de
> > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > 
> > On Wed, Apr 17, 2013 at 01:20:58PM +0000, KY Srinivasan wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > > > Sent: Wednesday, April 17, 2013 3:06 AM
> > > > To: KY Srinivasan
> > > > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-
> > kernel@vger.kernel.org;
> > > > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > > > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > > > JBeulich@suse.com; bp@alien8.de
> > > > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > > >
> > > > On Sun, Feb 03, 2013 at 05:22:38PM -0800, K. Y. Srinivasan wrote:
> > > > > Xen emulates Hyper-V to host enlightened Windows. Looks like this
> > > > > emulation may be turned on by default even for Linux guests. Check and
> > > > > fail Hyper-V detection if we are on Xen.
> > > > >
> > > > > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > > >
> > > > I'm very curious what's the specific bug that is fixed here?
> > > > I went over the original discussion in
> > > > https://patchwork.kernel.org/patch/2064331/
> > > > and that's still not clear to me. Is there a configuration
> > > > that is broken without this patch but starts working with
> > > > this patch?
> > > >
> > > > It seems that one might want to use hyper-v emulation e.g. to test
> > > > hyper-v code without using windows, so the functionality
> > > > that this patch disables is not completely useless,
> > > > so there should be a good reason for disabling it.
> > > >
> > > > Could you enlighten me please? :)
> > >
> > > Michael,
> > >
> > > If I recall correctly, the issue here was that Xen was enabling
> > > Hyper-V emulation un-conditionally even for Linux guests.
> > 
> > Okay so basically some Xen specific strangeness.
> > 
> > > Clearly,
> > > this was not the intention to begin with. This check was added to
> > > ensure that while hosting Linux we would not un-intentionally enable
> > > Hyper-V emulation in Xen.
> > 
> > What if you want to intentionally enable Hyper-V emulation in Xen?
> > There's no way to do this now, right?
> > 
> > > Not all Hyper-V features are tagged with a
> > > CPUID feature bit and so it was agreed that it would be better to
> > > disable Hyper-V emulation when hosting Linux on Xen.
> > >
> > > Regards,
> > >
> > > K. Y
> > 
> > I was just asking whether there are any specific examples of a bug
> > or is this change done just in case?
> 
> Ideally, if Hyper-V could be emulated in a plug compatible way, in that the guest would not know
> the difference as to whether Hyper-V was being emulated or indeed it was Hyper-V underneath, I would agree with you.
> Even in this case, I would argue that we would somehow give preference to the native implementation rather
> than the emulated environment. Having done the initial emulation of Hyper-V on Xen, I know the current emulation of 
> Hyper-V is restricted to emulating a few features advertised via the CPUID bits. If I recall correctly, Xen did not implement
> the enlightened TLB flush scheme. Furthermore, none of the Hyper-V emulations implement the I/O related infrastructure.
> For Linux to run efficiently on Hyper-V, the I/O paths are significantly more important than what Hyper-V emulation there is
> in Xen and I suspect in KVM. So, I don't think we want to run Linux on Xen/KVM that is emulating Hyper-V (cpuids).

No, it's a Xen specific bug.  KVM in a hyper-v mode still emulates PCI
so can still use KVM PV devices which are all PCI devices. So there's no problem there.

Also, why override this in the guest? With time Xen hyper-v emulation
might improve, meanwhile it's always possible to disable this in
the hypervisor.

> The specific case I ran into that required this patch was when I moved the VMBUS interrupts to be delivered via a special
> hypervisor vector (to support full interrupt distribution amongst all VCPUs in the guest). This feature is not tagged in the CPUID
> feature bits and without this check, I would install an interrupt handler for the hypervisor vector that clearly did not make
> any sense in anything other than Hyper-V. I suspect there will be more such situations as we go forward.
> 
> Regards,
> 
> K. Y 

Why? Because there's no VMBUS on Xen? So check whether VMBUS exists?

> > 
> > > >
> > > > > ---
> > > > >  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
> > > > >  1 files changed, 7 insertions(+), 0 deletions(-)
> > > > >
> > > > > diff --git a/arch/x86/kernel/cpu/mshyperv.c
> > > > b/arch/x86/kernel/cpu/mshyperv.c
> > > > > index 646d192..4dab317 100644
> > > > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > > > @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
> > > > >  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
> > > > >  		return false;
> > > > >
> > > > > +	/*
> > > > > +	 * Xen emulates Hyper-V to support enlightened Windows.
> > > > > +	 * Check to see first if we are on a Xen Hypervisor.
> > > > > +	 */
> > > > > +	if (xen_cpuid_base())
> > > > > +		return false;
> > > > > +
> > > > >  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
> > > > >  	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);
> > > > >
> > > > > --
> > > > > 1.7.4.1
> > > > >
> > > > > --
> > > > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > > > > the body of a message to majordomo@vger.kernel.org
> > > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > > > Please read the FAQ at  http://www.tux.org/lkml/
> > > >
> > >
> > 
> 

RE: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: Michael S. Tsirkin [mailto:mst@redhat.com]
> Sent: Wednesday, April 17, 2013 3:06 AM
> To: KY Srinivasan
> Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> JBeulich@suse.com; bp@alien8.de
> Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> 
> On Sun, Feb 03, 2013 at 05:22:38PM -0800, K. Y. Srinivasan wrote:
> > Xen emulates Hyper-V to host enlightened Windows. Looks like this
> > emulation may be turned on by default even for Linux guests. Check and
> > fail Hyper-V detection if we are on Xen.
> >
> > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> 
> I'm very curious what's the specific bug that is fixed here?
> I went over the original discussion in
> https://patchwork.kernel.org/patch/2064331/
> and that's still not clear to me. Is there a configuration
> that is broken without this patch but starts working with
> this patch?
> 
> It seems that one might want to use hyper-v emulation e.g. to test
> hyper-v code without using windows, so the functionality
> that this patch disables is not completely useless,
> so there should be a good reason for disabling it.
> 
> Could you enlighten me please? :)

Michael,

If I recall correctly, the issue here was that Xen was enabling Hyper-V emulation un-conditionally even for Linux guests. Clearly, this was not the intention to begin with. This check was added to ensure that while hosting Linux we would not un-intentionally enable Hyper-V emulation in Xen. Not all Hyper-V features are tagged with a CPUID feature bit and so it was agreed that it would be better to disable Hyper-V emulation when hosting Linux on Xen.

Regards,

K. Y
> 
> > ---
> >  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
> >  1 files changed, 7 insertions(+), 0 deletions(-)
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c
> b/arch/x86/kernel/cpu/mshyperv.c
> > index 646d192..4dab317 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
> >  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
> >  		return false;
> >
> > +	/*
> > +	 * Xen emulates Hyper-V to support enlightened Windows.
> > +	 * Check to see first if we are on a Xen Hypervisor.
> > +	 */
> > +	if (xen_cpuid_base())
> > +		return false;
> > +
> >  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
> >  	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);
> >
> > --
> > 1.7.4.1
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Michael S. Tsirkin]

On Wed, Apr 17, 2013 at 04:01:02PM +0300, Michael S. Tsirkin wrote:
> On Wed, Apr 17, 2013 at 02:52:42PM +0100, Jan Beulich wrote:
> > >>> On 17.04.13 at 15:20, KY Srinivasan <kys@microsoft.com> wrote:
> > > If I recall correctly, the issue here was that Xen was enabling Hyper-V 
> > > emulation un-conditionally even for Linux guests.
> > 
> > To make this a little more precise - Xen is doing so only when the
> > guest config tells it to.
> > 
> > Jan
> 
> So it's something explicitly requested for this guest?  Then I don't
> understand why does not make sense to override it in guest.

Typo, should have been "why does *it* make sense to override it in
guest.

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Michael S. Tsirkin]

On Wed, Apr 17, 2013 at 02:52:42PM +0100, Jan Beulich wrote:
> >>> On 17.04.13 at 15:20, KY Srinivasan <kys@microsoft.com> wrote:
> > If I recall correctly, the issue here was that Xen was enabling Hyper-V 
> > emulation un-conditionally even for Linux guests.
> 
> To make this a little more precise - Xen is doing so only when the
> guest config tells it to.
> 
> Jan

So it's something explicitly requested for this guest?  Then I don't
understand why does not make sense to override it in guest.

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Michael S. Tsirkin]

On Wed, Apr 17, 2013 at 01:20:58PM +0000, KY Srinivasan wrote:
> 
> 
> > -----Original Message-----
> > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > Sent: Wednesday, April 17, 2013 3:06 AM
> > To: KY Srinivasan
> > Cc: x86@kernel.org; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.org;
> > devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com;
> > jasowang@redhat.com; tglx@linutronix.de; hpa@zytor.com;
> > JBeulich@suse.com; bp@alien8.de
> > Subject: Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V
> > 
> > On Sun, Feb 03, 2013 at 05:22:38PM -0800, K. Y. Srinivasan wrote:
> > > Xen emulates Hyper-V to host enlightened Windows. Looks like this
> > > emulation may be turned on by default even for Linux guests. Check and
> > > fail Hyper-V detection if we are on Xen.
> > >
> > > Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> > 
> > I'm very curious what's the specific bug that is fixed here?
> > I went over the original discussion in
> > https://patchwork.kernel.org/patch/2064331/
> > and that's still not clear to me. Is there a configuration
> > that is broken without this patch but starts working with
> > this patch?
> > 
> > It seems that one might want to use hyper-v emulation e.g. to test
> > hyper-v code without using windows, so the functionality
> > that this patch disables is not completely useless,
> > so there should be a good reason for disabling it.
> > 
> > Could you enlighten me please? :)
> 
> Michael,
> 
> If I recall correctly, the issue here was that Xen was enabling
> Hyper-V emulation un-conditionally even for Linux guests.

Okay so basically some Xen specific strangeness.

> Clearly,
> this was not the intention to begin with. This check was added to
> ensure that while hosting Linux we would not un-intentionally enable
> Hyper-V emulation in Xen.

What if you want to intentionally enable Hyper-V emulation in Xen?
There's no way to do this now, right?

> Not all Hyper-V features are tagged with a
> CPUID feature bit and so it was agreed that it would be better to
> disable Hyper-V emulation when hosting Linux on Xen.
> 
> Regards,
> 
> K. Y

I was just asking whether there are any specific examples of a bug
or is this change done just in case?

> > 
> > > ---
> > >  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
> > >  1 files changed, 7 insertions(+), 0 deletions(-)
> > >
> > > diff --git a/arch/x86/kernel/cpu/mshyperv.c
> > b/arch/x86/kernel/cpu/mshyperv.c
> > > index 646d192..4dab317 100644
> > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
> > >  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
> > >  		return false;
> > >
> > > +	/*
> > > +	 * Xen emulates Hyper-V to support enlightened Windows.
> > > +	 * Check to see first if we are on a Xen Hypervisor.
> > > +	 */
> > > +	if (xen_cpuid_base())
> > > +		return false;
> > > +
> > >  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
> > >  	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);
> > >
> > > --
> > > 1.7.4.1
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > Please read the FAQ at  http://www.tux.org/lkml/
> > 
> 

Re: [PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[Michael S. Tsirkin]

On Sun, Feb 03, 2013 at 05:22:38PM -0800, K. Y. Srinivasan wrote:
> Xen emulates Hyper-V to host enlightened Windows. Looks like this
> emulation may be turned on by default even for Linux guests. Check and
> fail Hyper-V detection if we are on Xen.
> 
> Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>

I'm very curious what's the specific bug that is fixed here?
I went over the original discussion in
https://patchwork.kernel.org/patch/2064331/
and that's still not clear to me. Is there a configuration
that is broken without this patch but starts working with
this patch?

It seems that one might want to use hyper-v emulation e.g. to test
hyper-v code without using windows, so the functionality
that this patch disables is not completely useless,
so there should be a good reason for disabling it.

Could you enlighten me please? :)

> ---
>  arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
>  1 files changed, 7 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 646d192..4dab317 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
>  	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
>  		return false;
>  
> +	/*
> +	 * Xen emulates Hyper-V to support enlightened Windows.
> +	 * Check to see first if we are on a Xen Hypervisor.
> +	 */
> +	if (xen_cpuid_base())
> +		return false;
> +
>  	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
>  	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);
>  
> -- 
> 1.7.4.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

[PATCH 2/3] X86: Add a check to catch Xen emulation of Hyper-V

[K. Y. Srinivasan]

Xen emulates Hyper-V to host enlightened Windows. Looks like this
emulation may be turned on by default even for Linux guests. Check and
fail Hyper-V detection if we are on Xen.

Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
---
 arch/x86/kernel/cpu/mshyperv.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 646d192..4dab317 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -30,6 +30,13 @@ static bool __init ms_hyperv_platform(void)
 	if (!boot_cpu_has(X86_FEATURE_HYPERVISOR))
 		return false;
 
+	/*
+	 * Xen emulates Hyper-V to support enlightened Windows.
+	 * Check to see first if we are on a Xen Hypervisor.
+	 */
+	if (xen_cpuid_base())
+		return false;
+
 	cpuid(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS,
 	      &eax, &hyp_signature[0], &hyp_signature[1], &hyp_signature[2]);
 
-- 
1.7.4.1