Linux-HyperV Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH v3] x86/hyperv: make vapic support x2apic mode
@ 2019-10-09 14:50 Roman Kagan
  2019-10-09 15:27 ` Vitaly Kuznetsov
  0 siblings, 1 reply; 3+ messages in thread
From: Roman Kagan @ 2019-10-09 14:50 UTC (permalink / raw)
  To: Michael Kelley, Lan Tianyu, Joerg Roedel, K. Y. Srinivasan,
	Haiyang Zhang, Stephen Hemminger, Sasha Levin, Thomas Gleixner,
	Ingo Molnar, Borislav Petkov, H. Peter Anvin, x86, linux-hyperv,
	linux-kernel
  Cc: kvm, Vitaly Kuznetsov

Now that there's Hyper-V IOMMU driver, Linux can switch to x2apic mode
when supported by the vcpus.

However, the apic access functions for Hyper-V enlightened apic assume
xapic mode only.

As a result, Linux fails to bring up secondary cpus when run as a guest
in QEMU/KVM with both hv_apic and x2apic enabled.

According to Michael Kelley, when in x2apic mode, the Hyper-V synthetic
apic MSRs behave exactly the same as the corresponding architectural
x2apic MSRs, so there's no need to override the apic accessors.  The
only exception is hv_apic_eoi_write, which benefits from lazy EOI when
available; however, its implementation works for both xapic and x2apic
modes.

Fixes: 29217a474683 ("iommu/hyper-v: Add Hyper-V stub IOMMU driver")
Fixes: 6b48cb5f8347 ("X86/Hyper-V: Enlighten APIC access")
Cc: stable@vger.kernel.org
Suggested-by: Michael Kelley <mikelley@microsoft.com>
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
---
v2 -> v3:
- do not introduce x2apic-capable hv_apic accessors; leave original
  x2apic accessors instead

v1 -> v2:
- add ifdefs to handle !CONFIG_X86_X2APIC

 arch/x86/hyperv/hv_apic.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
index 5c056b8aebef..26eeff5bd535 100644
--- a/arch/x86/hyperv/hv_apic.c
+++ b/arch/x86/hyperv/hv_apic.c
@@ -261,10 +261,19 @@ void __init hv_apic_init(void)
 
 	if (ms_hyperv.hints & HV_X64_APIC_ACCESS_RECOMMENDED) {
 		pr_info("Hyper-V: Using MSR based APIC access\n");
+		/*
+		 * With x2apic, architectural x2apic MSRs are equivalent to the
+		 * respective synthetic MSRs, so there's no need to override
+		 * the apic accessors.  The only exception is
+		 * hv_apic_eoi_write, because it benefits from lazy EOI when
+		 * available, but it works for both xapic and x2apic modes.
+		 */
 		apic_set_eoi_write(hv_apic_eoi_write);
-		apic->read      = hv_apic_read;
-		apic->write     = hv_apic_write;
-		apic->icr_write = hv_apic_icr_write;
-		apic->icr_read  = hv_apic_icr_read;
+		if (!x2apic_enabled()) {
+			apic->read      = hv_apic_read;
+			apic->write     = hv_apic_write;
+			apic->icr_write = hv_apic_icr_write;
+			apic->icr_read  = hv_apic_icr_read;
+		}
 	}
 }
-- 
2.21.0


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH v3] x86/hyperv: make vapic support x2apic mode
  2019-10-09 14:50 [PATCH v3] x86/hyperv: make vapic support x2apic mode Roman Kagan
@ 2019-10-09 15:27 ` Vitaly Kuznetsov
  2019-10-09 15:41   ` Michael Kelley
  0 siblings, 1 reply; 3+ messages in thread
From: Vitaly Kuznetsov @ 2019-10-09 15:27 UTC (permalink / raw)
  To: Roman Kagan, Michael Kelley
  Cc: kvm\,
	Lan Tianyu, Joerg Roedel, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Sasha Levin, Thomas Gleixner, Ingo Molnar,
	Borislav Petkov, H. Peter Anvin, x86\, linux-hyperv\,
	linux-kernel\

Roman Kagan <rkagan@virtuozzo.com> writes:

> Now that there's Hyper-V IOMMU driver, Linux can switch to x2apic mode
> when supported by the vcpus.
>
> However, the apic access functions for Hyper-V enlightened apic assume
> xapic mode only.
>
> As a result, Linux fails to bring up secondary cpus when run as a guest
> in QEMU/KVM with both hv_apic and x2apic enabled.
>
> According to Michael Kelley, when in x2apic mode, the Hyper-V synthetic
> apic MSRs behave exactly the same as the corresponding architectural
> x2apic MSRs, so there's no need to override the apic accessors.  The
> only exception is hv_apic_eoi_write, which benefits from lazy EOI when
> available; however, its implementation works for both xapic and x2apic
> modes.
>
> Fixes: 29217a474683 ("iommu/hyper-v: Add Hyper-V stub IOMMU driver")
> Fixes: 6b48cb5f8347 ("X86/Hyper-V: Enlighten APIC access")
> Cc: stable@vger.kernel.org
> Suggested-by: Michael Kelley <mikelley@microsoft.com>
> Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
> ---
> v2 -> v3:
> - do not introduce x2apic-capable hv_apic accessors; leave original
>   x2apic accessors instead
>
> v1 -> v2:
> - add ifdefs to handle !CONFIG_X86_X2APIC
>
>  arch/x86/hyperv/hv_apic.c | 17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
> index 5c056b8aebef..26eeff5bd535 100644
> --- a/arch/x86/hyperv/hv_apic.c
> +++ b/arch/x86/hyperv/hv_apic.c
> @@ -261,10 +261,19 @@ void __init hv_apic_init(void)
>  
>  	if (ms_hyperv.hints & HV_X64_APIC_ACCESS_RECOMMENDED) {
>  		pr_info("Hyper-V: Using MSR based APIC access\n");

This pr_info() becomes a bit misleading in x2apic mode, maybe do
something like

pr_info("Hyper-V: using Enlightened APIC (%s mode)",
        x2apic_enabled() ? "x2apic" : "xapic");

> +		/*
> +		 * With x2apic, architectural x2apic MSRs are equivalent to the
> +		 * respective synthetic MSRs, so there's no need to override
> +		 * the apic accessors.  The only exception is
> +		 * hv_apic_eoi_write, because it benefits from lazy EOI when
> +		 * available, but it works for both xapic and x2apic modes.
> +		 */
>  		apic_set_eoi_write(hv_apic_eoi_write);
> -		apic->read      = hv_apic_read;
> -		apic->write     = hv_apic_write;
> -		apic->icr_write = hv_apic_icr_write;
> -		apic->icr_read  = hv_apic_icr_read;
> +		if (!x2apic_enabled()) {
> +			apic->read      = hv_apic_read;
> +			apic->write     = hv_apic_write;
> +			apic->icr_write = hv_apic_icr_write;
> +			apic->icr_read  = hv_apic_icr_read;
> +		}
>  	}
>  }

Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>

-- 
Vitaly

^ permalink raw reply	[flat|nested] 3+ messages in thread

* RE: [PATCH v3] x86/hyperv: make vapic support x2apic mode
  2019-10-09 15:27 ` Vitaly Kuznetsov
@ 2019-10-09 15:41   ` Michael Kelley
  0 siblings, 0 replies; 3+ messages in thread
From: Michael Kelley @ 2019-10-09 15:41 UTC (permalink / raw)
  To: vkuznets, Roman Kagan
  Cc: kvm, Tianyu Lan, Joerg Roedel, KY Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Sasha Levin, Thomas Gleixner, Ingo Molnar,
	Borislav Petkov, H. Peter Anvin, x86, linux-hyperv, linux-kernel

From: Vitaly Kuznetsov <vkuznets@redhat.com> Sent: Wednesday, October 9, 2019 8:27 AM
> 
> Roman Kagan <rkagan@virtuozzo.com> writes:
> 
> > Now that there's Hyper-V IOMMU driver, Linux can switch to x2apic mode
> > when supported by the vcpus.
> >
> > However, the apic access functions for Hyper-V enlightened apic assume
> > xapic mode only.
> >
> > As a result, Linux fails to bring up secondary cpus when run as a guest
> > in QEMU/KVM with both hv_apic and x2apic enabled.
> >
> > According to Michael Kelley, when in x2apic mode, the Hyper-V synthetic
> > apic MSRs behave exactly the same as the corresponding architectural
> > x2apic MSRs, so there's no need to override the apic accessors.  The
> > only exception is hv_apic_eoi_write, which benefits from lazy EOI when
> > available; however, its implementation works for both xapic and x2apic
> > modes.
> >
> > Fixes: 29217a474683 ("iommu/hyper-v: Add Hyper-V stub IOMMU driver")
> > Fixes: 6b48cb5f8347 ("X86/Hyper-V: Enlighten APIC access")
> > Cc: stable@vger.kernel.org
> > Suggested-by: Michael Kelley <mikelley@microsoft.com>
> > Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
> > ---
> > v2 -> v3:
> > - do not introduce x2apic-capable hv_apic accessors; leave original
> >   x2apic accessors instead
> >
> > v1 -> v2:
> > - add ifdefs to handle !CONFIG_X86_X2APIC
> >
> >  arch/x86/hyperv/hv_apic.c | 17 +++++++++++++----
> >  1 file changed, 13 insertions(+), 4 deletions(-)
> >
> > diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
> > index 5c056b8aebef..26eeff5bd535 100644
> > --- a/arch/x86/hyperv/hv_apic.c
> > +++ b/arch/x86/hyperv/hv_apic.c
> > @@ -261,10 +261,19 @@ void __init hv_apic_init(void)
> >
> >  	if (ms_hyperv.hints & HV_X64_APIC_ACCESS_RECOMMENDED) {
> >  		pr_info("Hyper-V: Using MSR based APIC access\n");
> 
> This pr_info() becomes a bit misleading in x2apic mode, maybe do
> something like
> 
> pr_info("Hyper-V: using Enlightened APIC (%s mode)",
>         x2apic_enabled() ? "x2apic" : "xapic");

Yes, I like this.  But tweak the capitalization of the message:

pr_info("Hyper-V: Using enlightened APIC (%s mode)",

> 
> > +		/*
> > +		 * With x2apic, architectural x2apic MSRs are equivalent to the
> > +		 * respective synthetic MSRs, so there's no need to override
> > +		 * the apic accessors.  The only exception is
> > +		 * hv_apic_eoi_write, because it benefits from lazy EOI when
> > +		 * available, but it works for both xapic and x2apic modes.
> > +		 */
> >  		apic_set_eoi_write(hv_apic_eoi_write);
> > -		apic->read      = hv_apic_read;
> > -		apic->write     = hv_apic_write;
> > -		apic->icr_write = hv_apic_icr_write;
> > -		apic->icr_read  = hv_apic_icr_read;
> > +		if (!x2apic_enabled()) {
> > +			apic->read      = hv_apic_read;
> > +			apic->write     = hv_apic_write;
> > +			apic->icr_write = hv_apic_icr_write;
> > +			apic->icr_read  = hv_apic_icr_read;
> > +		}
> >  	}
> >  }
> 
> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> 
Reviewed-by: Michael Kelley <mikelley@microsoft.com>

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-09 14:50 [PATCH v3] x86/hyperv: make vapic support x2apic mode Roman Kagan
2019-10-09 15:27 ` Vitaly Kuznetsov
2019-10-09 15:41   ` Michael Kelley

Linux-HyperV Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-hyperv/0 linux-hyperv/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-hyperv linux-hyperv/ https://lore.kernel.org/linux-hyperv \
		linux-hyperv@vger.kernel.org linux-hyperv@archiver.kernel.org
	public-inbox-index linux-hyperv

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-hyperv


AGPL code for this site: git clone https://public-inbox.org/ public-inbox