linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] x86/hyperv: make vapic support x2apic mode
@ 2019-09-30 17:33 Roman Kagan
  2019-09-30 22:44 ` kbuild test robot
  2019-09-30 23:04 ` kbuild test robot
  0 siblings, 2 replies; 4+ messages in thread
From: Roman Kagan @ 2019-09-30 17:33 UTC (permalink / raw)
  To: Michael Kelley, Lan Tianyu, Joerg Roedel, K. Y. Srinivasan,
	Haiyang Zhang, Stephen Hemminger, Sasha Levin, Thomas Gleixner,
	Ingo Molnar, Borislav Petkov, H. Peter Anvin, x86, linux-hyperv,
	linux-kernel
  Cc: kvm

Now that there's Hyper-V IOMMU driver, Linux can switch to x2apic mode
when supported by the vcpus.

However, the apic access functions for Hyper-V enlightened apic assume
xapic mode only.

As a result, Linux fails to bring up secondary cpus when run as a guest
in QEMU/KVM with both hv_apic and x2apic enabled.

I didn't manage to make my instance of Hyper-V expose x2apic to the
guest; nor does Hyper-V spec document the expected behavior.  However,
a Windows guest running in QEMU/KVM with hv_apic and x2apic and a big
number of vcpus (so that it turns on x2apic mode) does use enlightened
apic MSRs passing unshifted 32bit destination id and falls back to the
regular x2apic MSRs for less frequently used apic fields.

So implement the same behavior, by replacing enlightened apic access
functions (only those where it makes a difference) with their
x2apic-aware versions when x2apic is in use.

Fixes: 29217a474683 ("iommu/hyper-v: Add Hyper-V stub IOMMU driver")
Fixes: 6b48cb5f8347 ("X86/Hyper-V: Enlighten APIC access")
Cc: stable@vger.kernel.org
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
---
 arch/x86/hyperv/hv_apic.c | 48 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 45 insertions(+), 3 deletions(-)

diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
index 5c056b8aebef..9564fec00375 100644
--- a/arch/x86/hyperv/hv_apic.c
+++ b/arch/x86/hyperv/hv_apic.c
@@ -53,6 +53,11 @@ static void hv_apic_icr_write(u32 low, u32 id)
 	wrmsrl(HV_X64_MSR_ICR, reg_val);
 }
 
+static void hv_x2apic_icr_write(u32 low, u32 id)
+{
+	wrmsr(HV_X64_MSR_ICR, low, id);
+}
+
 static u32 hv_apic_read(u32 reg)
 {
 	u32 reg_val, hi;
@@ -70,6 +75,23 @@ static u32 hv_apic_read(u32 reg)
 	}
 }
 
+static u32 hv_x2apic_read(u32 reg)
+{
+	u32 reg_val, hi;
+
+	switch (reg) {
+	case APIC_EOI:
+		rdmsr(HV_X64_MSR_EOI, reg_val, hi);
+		return reg_val;
+	case APIC_TASKPRI:
+		rdmsr(HV_X64_MSR_TPR, reg_val, hi);
+		return reg_val;
+
+	default:
+		return native_apic_msr_read(reg);
+	}
+}
+
 static void hv_apic_write(u32 reg, u32 val)
 {
 	switch (reg) {
@@ -84,6 +106,20 @@ static void hv_apic_write(u32 reg, u32 val)
 	}
 }
 
+static void hv_x2apic_write(u32 reg, u32 val)
+{
+	switch (reg) {
+	case APIC_EOI:
+		wrmsr(HV_X64_MSR_EOI, val, 0);
+		break;
+	case APIC_TASKPRI:
+		wrmsr(HV_X64_MSR_TPR, val, 0);
+		break;
+	default:
+		native_apic_msr_write(reg, val);
+	}
+}
+
 static void hv_apic_eoi_write(u32 reg, u32 val)
 {
 	struct hv_vp_assist_page *hvp = hv_vp_assist_page[smp_processor_id()];
@@ -262,9 +298,15 @@ void __init hv_apic_init(void)
 	if (ms_hyperv.hints & HV_X64_APIC_ACCESS_RECOMMENDED) {
 		pr_info("Hyper-V: Using MSR based APIC access\n");
 		apic_set_eoi_write(hv_apic_eoi_write);
-		apic->read      = hv_apic_read;
-		apic->write     = hv_apic_write;
-		apic->icr_write = hv_apic_icr_write;
+		if (x2apic_enabled()) {
+			apic->read      = hv_x2apic_read;
+			apic->write     = hv_x2apic_write;
+			apic->icr_write = hv_x2apic_icr_write;
+		} else {
+			apic->read      = hv_apic_read;
+			apic->write     = hv_apic_write;
+			apic->icr_write = hv_apic_icr_write;
+		}
 		apic->icr_read  = hv_apic_icr_read;
 	}
 }
-- 
2.21.0


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [PATCH] x86/hyperv: make vapic support x2apic mode
  2019-09-30 17:33 [PATCH] x86/hyperv: make vapic support x2apic mode Roman Kagan
@ 2019-09-30 22:44 ` kbuild test robot
  2019-10-01  9:18   ` Roman Kagan
  2019-09-30 23:04 ` kbuild test robot
  1 sibling, 1 reply; 4+ messages in thread
From: kbuild test robot @ 2019-09-30 22:44 UTC (permalink / raw)
  To: Roman Kagan
  Cc: kbuild-all, Michael Kelley, Lan Tianyu, Joerg Roedel,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
	x86, linux-hyperv, linux-kernel, kvm

[-- Attachment #1: Type: text/plain, Size: 2817 bytes --]

Hi Roman,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on tip/x86/core]
[cannot apply to v5.4-rc1 next-20190930]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Roman-Kagan/x86-hyperv-make-vapic-support-x2apic-mode/20191001-044238
config: x86_64-randconfig-e004-201939 (attached as .config)
compiler: gcc-7 (Debian 7.4.0-13) 7.4.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   arch/x86/hyperv/hv_apic.c: In function 'hv_x2apic_read':
>> arch/x86/hyperv/hv_apic.c:91:10: error: implicit declaration of function 'native_apic_msr_read'; did you mean 'native_apic_icr_read'? [-Werror=implicit-function-declaration]
      return native_apic_msr_read(reg);
             ^~~~~~~~~~~~~~~~~~~~
             native_apic_icr_read
   arch/x86/hyperv/hv_apic.c: In function 'hv_x2apic_write':
>> arch/x86/hyperv/hv_apic.c:119:3: error: implicit declaration of function 'native_apic_msr_write'; did you mean 'native_apic_icr_write'? [-Werror=implicit-function-declaration]
      native_apic_msr_write(reg, val);
      ^~~~~~~~~~~~~~~~~~~~~
      native_apic_icr_write
   cc1: some warnings being treated as errors

vim +91 arch/x86/hyperv/hv_apic.c

    77	
    78	static u32 hv_x2apic_read(u32 reg)
    79	{
    80		u32 reg_val, hi;
    81	
    82		switch (reg) {
    83		case APIC_EOI:
    84			rdmsr(HV_X64_MSR_EOI, reg_val, hi);
    85			return reg_val;
    86		case APIC_TASKPRI:
    87			rdmsr(HV_X64_MSR_TPR, reg_val, hi);
    88			return reg_val;
    89	
    90		default:
  > 91			return native_apic_msr_read(reg);
    92		}
    93	}
    94	
    95	static void hv_apic_write(u32 reg, u32 val)
    96	{
    97		switch (reg) {
    98		case APIC_EOI:
    99			wrmsr(HV_X64_MSR_EOI, val, 0);
   100			break;
   101		case APIC_TASKPRI:
   102			wrmsr(HV_X64_MSR_TPR, val, 0);
   103			break;
   104		default:
   105			native_apic_mem_write(reg, val);
   106		}
   107	}
   108	
   109	static void hv_x2apic_write(u32 reg, u32 val)
   110	{
   111		switch (reg) {
   112		case APIC_EOI:
   113			wrmsr(HV_X64_MSR_EOI, val, 0);
   114			break;
   115		case APIC_TASKPRI:
   116			wrmsr(HV_X64_MSR_TPR, val, 0);
   117			break;
   118		default:
 > 119			native_apic_msr_write(reg, val);
   120		}
   121	}
   122	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 31452 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] x86/hyperv: make vapic support x2apic mode
  2019-09-30 17:33 [PATCH] x86/hyperv: make vapic support x2apic mode Roman Kagan
  2019-09-30 22:44 ` kbuild test robot
@ 2019-09-30 23:04 ` kbuild test robot
  1 sibling, 0 replies; 4+ messages in thread
From: kbuild test robot @ 2019-09-30 23:04 UTC (permalink / raw)
  To: Roman Kagan
  Cc: kbuild-all, Michael Kelley, Lan Tianyu, Joerg Roedel,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
	x86, linux-hyperv, linux-kernel, kvm

[-- Attachment #1: Type: text/plain, Size: 2681 bytes --]

Hi Roman,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on tip/x86/core]
[cannot apply to v5.4-rc1 next-20190930]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Roman-Kagan/x86-hyperv-make-vapic-support-x2apic-mode/20191001-044238
config: x86_64-randconfig-s0-201939 (attached as .config)
compiler: gcc-5 (Ubuntu 5.5.0-12ubuntu1) 5.5.0 20171010
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   arch/x86//hyperv/hv_apic.c: In function 'hv_x2apic_read':
>> arch/x86//hyperv/hv_apic.c:91:10: error: implicit declaration of function 'native_apic_msr_read' [-Werror=implicit-function-declaration]
      return native_apic_msr_read(reg);
             ^
   arch/x86//hyperv/hv_apic.c: In function 'hv_x2apic_write':
>> arch/x86//hyperv/hv_apic.c:119:3: error: implicit declaration of function 'native_apic_msr_write' [-Werror=implicit-function-declaration]
      native_apic_msr_write(reg, val);
      ^
   cc1: some warnings being treated as errors

vim +/native_apic_msr_read +91 arch/x86//hyperv/hv_apic.c

    77	
    78	static u32 hv_x2apic_read(u32 reg)
    79	{
    80		u32 reg_val, hi;
    81	
    82		switch (reg) {
    83		case APIC_EOI:
    84			rdmsr(HV_X64_MSR_EOI, reg_val, hi);
    85			return reg_val;
    86		case APIC_TASKPRI:
    87			rdmsr(HV_X64_MSR_TPR, reg_val, hi);
    88			return reg_val;
    89	
    90		default:
  > 91			return native_apic_msr_read(reg);
    92		}
    93	}
    94	
    95	static void hv_apic_write(u32 reg, u32 val)
    96	{
    97		switch (reg) {
    98		case APIC_EOI:
    99			wrmsr(HV_X64_MSR_EOI, val, 0);
   100			break;
   101		case APIC_TASKPRI:
   102			wrmsr(HV_X64_MSR_TPR, val, 0);
   103			break;
   104		default:
   105			native_apic_mem_write(reg, val);
   106		}
   107	}
   108	
   109	static void hv_x2apic_write(u32 reg, u32 val)
   110	{
   111		switch (reg) {
   112		case APIC_EOI:
   113			wrmsr(HV_X64_MSR_EOI, val, 0);
   114			break;
   115		case APIC_TASKPRI:
   116			wrmsr(HV_X64_MSR_TPR, val, 0);
   117			break;
   118		default:
 > 119			native_apic_msr_write(reg, val);
   120		}
   121	}
   122	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 30710 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] x86/hyperv: make vapic support x2apic mode
  2019-09-30 22:44 ` kbuild test robot
@ 2019-10-01  9:18   ` Roman Kagan
  0 siblings, 0 replies; 4+ messages in thread
From: Roman Kagan @ 2019-10-01  9:18 UTC (permalink / raw)
  To: kbuild test robot
  Cc: kbuild-all, Michael Kelley, Lan Tianyu, Joerg Roedel,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
	x86, linux-hyperv, linux-kernel, kvm

On Tue, Oct 01, 2019 at 06:44:08AM +0800, kbuild test robot wrote:
> url:    https://github.com/0day-ci/linux/commits/Roman-Kagan/x86-hyperv-make-vapic-support-x2apic-mode/20191001-044238
> config: x86_64-randconfig-e004-201939 (attached as .config)
> compiler: gcc-7 (Debian 7.4.0-13) 7.4.0
> reproduce:
>         # save the attached .config to linux build tree
>         make ARCH=x86_64 
> 
> If you fix the issue, kindly add following tag
> Reported-by: kbuild test robot <lkp@intel.com>
> 
> All errors (new ones prefixed by >>):
> 
>    arch/x86/hyperv/hv_apic.c: In function 'hv_x2apic_read':
> >> arch/x86/hyperv/hv_apic.c:91:10: error: implicit declaration of function 'native_apic_msr_read'; did you mean 'native_apic_icr_read'? [-Werror=implicit-function-declaration]
>       return native_apic_msr_read(reg);
>              ^~~~~~~~~~~~~~~~~~~~
>              native_apic_icr_read
>    arch/x86/hyperv/hv_apic.c: In function 'hv_x2apic_write':
> >> arch/x86/hyperv/hv_apic.c:119:3: error: implicit declaration of function 'native_apic_msr_write'; did you mean 'native_apic_icr_write'? [-Werror=implicit-function-declaration]
>       native_apic_msr_write(reg, val);
>       ^~~~~~~~~~~~~~~~~~~~~
>       native_apic_icr_write
>    cc1: some warnings being treated as errors

Oops, !CONFIG_X86_X2APIC needs to be handled.  Will post v2.

Roman.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2019-10-01  9:19 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-30 17:33 [PATCH] x86/hyperv: make vapic support x2apic mode Roman Kagan
2019-09-30 22:44 ` kbuild test robot
2019-10-01  9:18   ` Roman Kagan
2019-09-30 23:04 ` kbuild test robot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).