* [PATCH 0/4] x86/xen: do some paravirt cleanup
@ 2020-11-16 15:22 ` Juergen Gross via Virtualization
0 siblings, 0 replies; 24+ messages in thread
From: Juergen Gross via Virtualization @ 2020-11-16 15:22 UTC (permalink / raw)
To: xen-devel, x86, linux-kernel, virtualization
Cc: Juergen Gross, Stefano Stabellini, VMware, Inc.,
Ingo Molnar, Borislav Petkov, Andy Lutomirski, H. Peter Anvin,
Thomas Gleixner, Boris Ostrovsky
Eliminate the usergs_sysret64 paravirt call completely and switch
the swapgs one to use ALTERNATIVE instead. This requires to fix the
IST based exception entries for Xen PV to use the same mechanism as
NMI and debug exception already do.
Juergen Gross (4):
x86/xen: use specific Xen pv interrupt entry for MCE
x86/xen: use specific Xen pv interrupt entry for DF
x86/pv: switch SWAPGS to ALTERNATIVE
x86/xen: drop USERGS_SYSRET64 paravirt call
arch/x86/entry/entry_64.S | 32 ++++++++++++---------------
arch/x86/include/asm/idtentry.h | 6 +++++
arch/x86/include/asm/irqflags.h | 26 +++++++---------------
arch/x86/include/asm/paravirt.h | 25 ---------------------
arch/x86/include/asm/paravirt_types.h | 10 ---------
arch/x86/kernel/asm-offsets_64.c | 3 ---
arch/x86/kernel/paravirt.c | 6 +----
arch/x86/kernel/paravirt_patch.c | 7 ------
arch/x86/xen/enlighten_pv.c | 28 ++++++++++++++++++-----
arch/x86/xen/xen-asm.S | 24 ++------------------
arch/x86/xen/xen-ops.h | 2 --
11 files changed, 53 insertions(+), 116 deletions(-)
--
2.26.2
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 1/4] x86/xen: use specific Xen pv interrupt entry for MCE
2020-11-16 15:22 ` Juergen Gross via Virtualization
(?)
@ 2020-11-16 15:22 ` Juergen Gross
-1 siblings, 0 replies; 24+ messages in thread
From: Juergen Gross @ 2020-11-16 15:22 UTC (permalink / raw)
To: xen-devel, x86, linux-kernel
Cc: Juergen Gross, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, Boris Ostrovsky, Stefano Stabellini
Xen PV guests don't use IST. For machine check interrupts switch to
the same model as debug interrupts.
Signed-off-by: Juergen Gross <jgross@suse.com>
---
arch/x86/include/asm/idtentry.h | 3 +++
arch/x86/xen/enlighten_pv.c | 16 +++++++++++++++-
arch/x86/xen/xen-asm.S | 2 +-
3 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h
index b2442eb0ac2f..3505c0396fa5 100644
--- a/arch/x86/include/asm/idtentry.h
+++ b/arch/x86/include/asm/idtentry.h
@@ -588,6 +588,9 @@ DECLARE_IDTENTRY_MCE(X86_TRAP_MC, exc_machine_check);
#else
DECLARE_IDTENTRY_RAW(X86_TRAP_MC, exc_machine_check);
#endif
+#ifdef CONFIG_XEN_PV
+DECLARE_IDTENTRY_RAW(X86_TRAP_MC, xenpv_exc_machine_check);
+#endif
#endif
/* NMI */
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 4409306364dc..9f5e44c1f70a 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -583,6 +583,20 @@ DEFINE_IDTENTRY_RAW(xenpv_exc_debug)
exc_debug(regs);
}
+#ifdef CONFIG_X86_MCE
+DEFINE_IDTENTRY_RAW(xenpv_exc_machine_check)
+{
+ /*
+ * There's no IST on Xen PV, but we still need to dispatch
+ * to the correct handler.
+ */
+ if (user_mode(regs))
+ noist_exc_machine_check(regs);
+ else
+ exc_machine_check(regs);
+}
+#endif
+
struct trap_array_entry {
void (*orig)(void);
void (*xen)(void);
@@ -603,7 +617,7 @@ static struct trap_array_entry trap_array[] = {
TRAP_ENTRY_REDIR(exc_debug, true ),
TRAP_ENTRY(exc_double_fault, true ),
#ifdef CONFIG_X86_MCE
- TRAP_ENTRY(exc_machine_check, true ),
+ TRAP_ENTRY_REDIR(exc_machine_check, true ),
#endif
TRAP_ENTRY_REDIR(exc_nmi, true ),
TRAP_ENTRY(exc_int3, false ),
diff --git a/arch/x86/xen/xen-asm.S b/arch/x86/xen/xen-asm.S
index 1cb0e84b9161..bc2586730a5b 100644
--- a/arch/x86/xen/xen-asm.S
+++ b/arch/x86/xen/xen-asm.S
@@ -172,7 +172,7 @@ xen_pv_trap asm_exc_spurious_interrupt_bug
xen_pv_trap asm_exc_coprocessor_error
xen_pv_trap asm_exc_alignment_check
#ifdef CONFIG_X86_MCE
-xen_pv_trap asm_exc_machine_check
+xen_pv_trap asm_xenpv_exc_machine_check
#endif /* CONFIG_X86_MCE */
xen_pv_trap asm_exc_simd_coprocessor_error
#ifdef CONFIG_IA32_EMULATION
--
2.26.2
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 2/4] x86/xen: use specific Xen pv interrupt entry for DF
2020-11-16 15:22 ` Juergen Gross via Virtualization
(?)
(?)
@ 2020-11-16 15:22 ` Juergen Gross
2020-11-16 16:12 ` Andy Lutomirski
-1 siblings, 1 reply; 24+ messages in thread
From: Juergen Gross @ 2020-11-16 15:22 UTC (permalink / raw)
To: xen-devel, x86, linux-kernel
Cc: Juergen Gross, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, Boris Ostrovsky, Stefano Stabellini
Xen PV guests don't use IST. For double fault interrupts switch to
the same model as NMI.
Signed-off-by: Juergen Gross <jgross@suse.com>
---
arch/x86/include/asm/idtentry.h | 3 +++
arch/x86/xen/enlighten_pv.c | 8 +++++++-
arch/x86/xen/xen-asm.S | 2 +-
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h
index 3505c0396fa5..b35825392547 100644
--- a/arch/x86/include/asm/idtentry.h
+++ b/arch/x86/include/asm/idtentry.h
@@ -611,6 +611,9 @@ DECLARE_IDTENTRY_RAW(X86_TRAP_DB, xenpv_exc_debug);
/* #DF */
DECLARE_IDTENTRY_DF(X86_TRAP_DF, exc_double_fault);
+#ifdef CONFIG_XEN_PV
+DECLARE_IDTENTRY_RAW_ERRORCODE(X86_TRAP_DF, xenpv_exc_double_fault);
+#endif
/* #VC */
#ifdef CONFIG_AMD_MEM_ENCRYPT
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 9f5e44c1f70a..803fbcb398c4 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -571,6 +571,12 @@ DEFINE_IDTENTRY_RAW(xenpv_exc_nmi)
exc_nmi(regs);
}
+DEFINE_IDTENTRY_RAW_ERRORCODE(xenpv_exc_double_fault)
+{
+ /* On Xen PV, DF doesn't use IST. The C part is the sane as native. */
+ exc_double_fault(regs, error_code);
+}
+
DEFINE_IDTENTRY_RAW(xenpv_exc_debug)
{
/*
@@ -615,7 +621,7 @@ struct trap_array_entry {
static struct trap_array_entry trap_array[] = {
TRAP_ENTRY_REDIR(exc_debug, true ),
- TRAP_ENTRY(exc_double_fault, true ),
+ TRAP_ENTRY_REDIR(exc_double_fault, true ),
#ifdef CONFIG_X86_MCE
TRAP_ENTRY_REDIR(exc_machine_check, true ),
#endif
diff --git a/arch/x86/xen/xen-asm.S b/arch/x86/xen/xen-asm.S
index bc2586730a5b..1d054c915046 100644
--- a/arch/x86/xen/xen-asm.S
+++ b/arch/x86/xen/xen-asm.S
@@ -161,7 +161,7 @@ xen_pv_trap asm_exc_overflow
xen_pv_trap asm_exc_bounds
xen_pv_trap asm_exc_invalid_op
xen_pv_trap asm_exc_device_not_available
-xen_pv_trap asm_exc_double_fault
+xen_pv_trap asm_xenpv_exc_double_fault
xen_pv_trap asm_exc_coproc_segment_overrun
xen_pv_trap asm_exc_invalid_tss
xen_pv_trap asm_exc_segment_not_present
--
2.26.2
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH 2/4] x86/xen: use specific Xen pv interrupt entry for DF
2020-11-16 15:22 ` [PATCH 2/4] x86/xen: use specific Xen pv interrupt entry for DF Juergen Gross
@ 2020-11-16 16:12 ` Andy Lutomirski
0 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:12 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, X86 ML, LKML, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Boris Ostrovsky,
Stefano Stabellini
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> Xen PV guests don't use IST. For double fault interrupts switch to
> the same model as NMI.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
> arch/x86/include/asm/idtentry.h | 3 +++
> arch/x86/xen/enlighten_pv.c | 8 +++++++-
> arch/x86/xen/xen-asm.S | 2 +-
> 3 files changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h
> index 3505c0396fa5..b35825392547 100644
> --- a/arch/x86/include/asm/idtentry.h
> +++ b/arch/x86/include/asm/idtentry.h
> @@ -611,6 +611,9 @@ DECLARE_IDTENTRY_RAW(X86_TRAP_DB, xenpv_exc_debug);
>
> /* #DF */
> DECLARE_IDTENTRY_DF(X86_TRAP_DF, exc_double_fault);
> +#ifdef CONFIG_XEN_PV
> +DECLARE_IDTENTRY_RAW_ERRORCODE(X86_TRAP_DF, xenpv_exc_double_fault);
> +#endif
>
> /* #VC */
> #ifdef CONFIG_AMD_MEM_ENCRYPT
> diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
> index 9f5e44c1f70a..803fbcb398c4 100644
> --- a/arch/x86/xen/enlighten_pv.c
> +++ b/arch/x86/xen/enlighten_pv.c
> @@ -571,6 +571,12 @@ DEFINE_IDTENTRY_RAW(xenpv_exc_nmi)
> exc_nmi(regs);
> }
>
> +DEFINE_IDTENTRY_RAW_ERRORCODE(xenpv_exc_double_fault)
> +{
> + /* On Xen PV, DF doesn't use IST. The C part is the sane as native. */
I would like to think that code is sane, but you probably meant "same".
> + exc_double_fault(regs, error_code);
> +}
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH 2/4] x86/xen: use specific Xen pv interrupt entry for DF
@ 2020-11-16 16:12 ` Andy Lutomirski
0 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:12 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, X86 ML, LKML, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Boris Ostrovsky,
Stefano Stabellini
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> Xen PV guests don't use IST. For double fault interrupts switch to
> the same model as NMI.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
> arch/x86/include/asm/idtentry.h | 3 +++
> arch/x86/xen/enlighten_pv.c | 8 +++++++-
> arch/x86/xen/xen-asm.S | 2 +-
> 3 files changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h
> index 3505c0396fa5..b35825392547 100644
> --- a/arch/x86/include/asm/idtentry.h
> +++ b/arch/x86/include/asm/idtentry.h
> @@ -611,6 +611,9 @@ DECLARE_IDTENTRY_RAW(X86_TRAP_DB, xenpv_exc_debug);
>
> /* #DF */
> DECLARE_IDTENTRY_DF(X86_TRAP_DF, exc_double_fault);
> +#ifdef CONFIG_XEN_PV
> +DECLARE_IDTENTRY_RAW_ERRORCODE(X86_TRAP_DF, xenpv_exc_double_fault);
> +#endif
>
> /* #VC */
> #ifdef CONFIG_AMD_MEM_ENCRYPT
> diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
> index 9f5e44c1f70a..803fbcb398c4 100644
> --- a/arch/x86/xen/enlighten_pv.c
> +++ b/arch/x86/xen/enlighten_pv.c
> @@ -571,6 +571,12 @@ DEFINE_IDTENTRY_RAW(xenpv_exc_nmi)
> exc_nmi(regs);
> }
>
> +DEFINE_IDTENTRY_RAW_ERRORCODE(xenpv_exc_double_fault)
> +{
> + /* On Xen PV, DF doesn't use IST. The C part is the sane as native. */
I would like to think that code is sane, but you probably meant "same".
> + exc_double_fault(regs, error_code);
> +}
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH 2/4] x86/xen: use specific Xen pv interrupt entry for DF
2020-11-16 16:12 ` Andy Lutomirski
(?)
@ 2020-11-16 16:14 ` Jürgen Groß
-1 siblings, 0 replies; 24+ messages in thread
From: Jürgen Groß @ 2020-11-16 16:14 UTC (permalink / raw)
To: Andy Lutomirski
Cc: xen-devel, X86 ML, LKML, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Boris Ostrovsky,
Stefano Stabellini
[-- Attachment #1.1.1: Type: text/plain, Size: 1647 bytes --]
On 16.11.20 17:12, Andy Lutomirski wrote:
> On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>>
>> Xen PV guests don't use IST. For double fault interrupts switch to
>> the same model as NMI.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>> arch/x86/include/asm/idtentry.h | 3 +++
>> arch/x86/xen/enlighten_pv.c | 8 +++++++-
>> arch/x86/xen/xen-asm.S | 2 +-
>> 3 files changed, 11 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h
>> index 3505c0396fa5..b35825392547 100644
>> --- a/arch/x86/include/asm/idtentry.h
>> +++ b/arch/x86/include/asm/idtentry.h
>> @@ -611,6 +611,9 @@ DECLARE_IDTENTRY_RAW(X86_TRAP_DB, xenpv_exc_debug);
>>
>> /* #DF */
>> DECLARE_IDTENTRY_DF(X86_TRAP_DF, exc_double_fault);
>> +#ifdef CONFIG_XEN_PV
>> +DECLARE_IDTENTRY_RAW_ERRORCODE(X86_TRAP_DF, xenpv_exc_double_fault);
>> +#endif
>>
>> /* #VC */
>> #ifdef CONFIG_AMD_MEM_ENCRYPT
>> diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
>> index 9f5e44c1f70a..803fbcb398c4 100644
>> --- a/arch/x86/xen/enlighten_pv.c
>> +++ b/arch/x86/xen/enlighten_pv.c
>> @@ -571,6 +571,12 @@ DEFINE_IDTENTRY_RAW(xenpv_exc_nmi)
>> exc_nmi(regs);
>> }
>>
>> +DEFINE_IDTENTRY_RAW_ERRORCODE(xenpv_exc_double_fault)
>> +{
>> + /* On Xen PV, DF doesn't use IST. The C part is the sane as native. */
>
> I would like to think that code is sane, but you probably meant "same".
Oh, this is the result of copy and paste. Now we have two sane
functions. :-)
Juergen
[-- Attachment #1.1.2: OpenPGP_0xB0DE9DD628BF132F.asc --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 3/4] x86/pv: switch SWAPGS to ALTERNATIVE
2020-11-16 15:22 ` Juergen Gross via Virtualization
@ 2020-11-16 15:23 ` Juergen Gross via Virtualization
-1 siblings, 0 replies; 24+ messages in thread
From: Juergen Gross @ 2020-11-16 15:23 UTC (permalink / raw)
To: xen-devel, x86, linux-kernel, virtualization
Cc: Juergen Gross, Andy Lutomirski, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Deep Shah, VMware, Inc.,
Boris Ostrovsky, Stefano Stabellini
SWAPGS is used only for interrupts coming from user mode or for
returning to user mode. So there is no reason to use the PARAVIRT
framework, as it can easily be replaced by an ALTERNATIVE depending
on X86_FEATURE_XENPV.
There are several instances using the PV-aware SWAPGS macro in paths
which are never executed in a Xen PV guest. Replace those with the
plain swapgs instruction. For SWAPGS_UNSAFE_STACK the same applies.
Signed-off-by: Juergen Gross <jgross@suse.com>
---
arch/x86/entry/entry_64.S | 10 +++++-----
arch/x86/include/asm/irqflags.h | 20 ++++++++------------
arch/x86/include/asm/paravirt.h | 20 --------------------
arch/x86/include/asm/paravirt_types.h | 2 --
arch/x86/kernel/asm-offsets_64.c | 1 -
arch/x86/kernel/paravirt.c | 1 -
arch/x86/kernel/paravirt_patch.c | 3 ---
arch/x86/xen/enlighten_pv.c | 3 ---
8 files changed, 13 insertions(+), 47 deletions(-)
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index cad08703c4ad..a876204a73e0 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -669,7 +669,7 @@ native_irq_return_ldt:
*/
pushq %rdi /* Stash user RDI */
- SWAPGS /* to kernel GS */
+ swapgs /* to kernel GS */
SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi /* to kernel CR3 */
movq PER_CPU_VAR(espfix_waddr), %rdi
@@ -699,7 +699,7 @@ native_irq_return_ldt:
orq PER_CPU_VAR(espfix_stack), %rax
SWITCH_TO_USER_CR3_STACK scratch_reg=%rdi
- SWAPGS /* to user GS */
+ swapgs /* to user GS */
popq %rdi /* Restore user RDI */
movq %rax, %rsp
@@ -943,7 +943,7 @@ SYM_CODE_START_LOCAL(paranoid_entry)
ret
.Lparanoid_entry_swapgs:
- SWAPGS
+ swapgs
/*
* The above SAVE_AND_SWITCH_TO_KERNEL_CR3 macro doesn't do an
@@ -1001,7 +1001,7 @@ SYM_CODE_START_LOCAL(paranoid_exit)
jnz restore_regs_and_return_to_kernel
/* We are returning to a context with user GSBASE */
- SWAPGS_UNSAFE_STACK
+ swapgs
jmp restore_regs_and_return_to_kernel
SYM_CODE_END(paranoid_exit)
@@ -1426,7 +1426,7 @@ nmi_no_fsgsbase:
jnz nmi_restore
nmi_swapgs:
- SWAPGS_UNSAFE_STACK
+ swapgs
nmi_restore:
POP_REGS
diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h
index 2dfc8d380dab..8c86edefa115 100644
--- a/arch/x86/include/asm/irqflags.h
+++ b/arch/x86/include/asm/irqflags.h
@@ -131,18 +131,6 @@ static __always_inline unsigned long arch_local_irq_save(void)
#define SAVE_FLAGS(x) pushfq; popq %rax
#endif
-#define SWAPGS swapgs
-/*
- * Currently paravirt can't handle swapgs nicely when we
- * don't have a stack we can rely on (such as a user space
- * stack). So we either find a way around these or just fault
- * and emulate if a guest tries to call swapgs directly.
- *
- * Either way, this is a good way to document that we don't
- * have a reliable stack. x86_64 only.
- */
-#define SWAPGS_UNSAFE_STACK swapgs
-
#define INTERRUPT_RETURN jmp native_iret
#define USERGS_SYSRET64 \
swapgs; \
@@ -170,6 +158,14 @@ static __always_inline int arch_irqs_disabled(void)
return arch_irqs_disabled_flags(flags);
}
+#else
+#ifdef CONFIG_X86_64
+#ifdef CONFIG_XEN_PV
+#define SWAPGS ALTERNATIVE "swapgs", "", X86_FEATURE_XENPV
+#else
+#define SWAPGS swapgs
+#endif
+#endif
#endif /* !__ASSEMBLY__ */
#endif
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index d25cc6830e89..5647bcdba776 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -776,26 +776,6 @@ extern void default_banner(void);
#ifdef CONFIG_X86_64
#ifdef CONFIG_PARAVIRT_XXL
-/*
- * If swapgs is used while the userspace stack is still current,
- * there's no way to call a pvop. The PV replacement *must* be
- * inlined, or the swapgs instruction must be trapped and emulated.
- */
-#define SWAPGS_UNSAFE_STACK \
- PARA_SITE(PARA_PATCH(PV_CPU_swapgs), swapgs)
-
-/*
- * Note: swapgs is very special, and in practise is either going to be
- * implemented with a single "swapgs" instruction or something very
- * special. Either way, we don't need to save any registers for
- * it.
- */
-#define SWAPGS \
- PARA_SITE(PARA_PATCH(PV_CPU_swapgs), \
- ANNOTATE_RETPOLINE_SAFE; \
- call PARA_INDIRECT(pv_ops+PV_CPU_swapgs); \
- )
-
#define USERGS_SYSRET64 \
PARA_SITE(PARA_PATCH(PV_CPU_usergs_sysret64), \
ANNOTATE_RETPOLINE_SAFE; \
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index 0fad9f61c76a..903d71884fa2 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -169,8 +169,6 @@ struct pv_cpu_ops {
frame set up. */
void (*iret)(void);
- void (*swapgs)(void);
-
void (*start_context_switch)(struct task_struct *prev);
void (*end_context_switch)(struct task_struct *next);
#endif
diff --git a/arch/x86/kernel/asm-offsets_64.c b/arch/x86/kernel/asm-offsets_64.c
index 828be792231e..1354bc30614d 100644
--- a/arch/x86/kernel/asm-offsets_64.c
+++ b/arch/x86/kernel/asm-offsets_64.c
@@ -15,7 +15,6 @@ int main(void)
#ifdef CONFIG_PARAVIRT_XXL
OFFSET(PV_CPU_usergs_sysret64, paravirt_patch_template,
cpu.usergs_sysret64);
- OFFSET(PV_CPU_swapgs, paravirt_patch_template, cpu.swapgs);
#ifdef CONFIG_DEBUG_ENTRY
OFFSET(PV_IRQ_save_fl, paravirt_patch_template, irq.save_fl);
#endif
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 6c3407ba6ee9..5e5fcf5c376d 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -312,7 +312,6 @@ struct paravirt_patch_template pv_ops = {
.cpu.usergs_sysret64 = native_usergs_sysret64,
.cpu.iret = native_iret,
- .cpu.swapgs = native_swapgs,
#ifdef CONFIG_X86_IOPL_IOPERM
.cpu.invalidate_io_bitmap = native_tss_invalidate_io_bitmap,
diff --git a/arch/x86/kernel/paravirt_patch.c b/arch/x86/kernel/paravirt_patch.c
index ace6e334cb39..7c518b08aa3c 100644
--- a/arch/x86/kernel/paravirt_patch.c
+++ b/arch/x86/kernel/paravirt_patch.c
@@ -28,7 +28,6 @@ struct patch_xxl {
const unsigned char irq_restore_fl[2];
const unsigned char cpu_wbinvd[2];
const unsigned char cpu_usergs_sysret64[6];
- const unsigned char cpu_swapgs[3];
const unsigned char mov64[3];
};
@@ -43,7 +42,6 @@ static const struct patch_xxl patch_data_xxl = {
.cpu_wbinvd = { 0x0f, 0x09 }, // wbinvd
.cpu_usergs_sysret64 = { 0x0f, 0x01, 0xf8,
0x48, 0x0f, 0x07 }, // swapgs; sysretq
- .cpu_swapgs = { 0x0f, 0x01, 0xf8 }, // swapgs
.mov64 = { 0x48, 0x89, 0xf8 }, // mov %rdi, %rax
};
@@ -86,7 +84,6 @@ unsigned int native_patch(u8 type, void *insn_buff, unsigned long addr,
PATCH_CASE(mmu, write_cr3, xxl, insn_buff, len);
PATCH_CASE(cpu, usergs_sysret64, xxl, insn_buff, len);
- PATCH_CASE(cpu, swapgs, xxl, insn_buff, len);
PATCH_CASE(cpu, wbinvd, xxl, insn_buff, len);
#endif
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 803fbcb398c4..82030d49f4f7 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1085,9 +1085,6 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = {
#endif
.io_delay = xen_io_delay,
- /* Xen takes care of %gs when switching to usermode for us */
- .swapgs = paravirt_nop,
-
.start_context_switch = paravirt_start_context_switch,
.end_context_switch = xen_end_context_switch,
};
--
2.26.2
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 3/4] x86/pv: switch SWAPGS to ALTERNATIVE
@ 2020-11-16 15:23 ` Juergen Gross via Virtualization
0 siblings, 0 replies; 24+ messages in thread
From: Juergen Gross via Virtualization @ 2020-11-16 15:23 UTC (permalink / raw)
To: xen-devel, x86, linux-kernel, virtualization
Cc: Juergen Gross, Stefano Stabellini, VMware, Inc.,
Ingo Molnar, Borislav Petkov, Andy Lutomirski, H. Peter Anvin,
Thomas Gleixner, Boris Ostrovsky
SWAPGS is used only for interrupts coming from user mode or for
returning to user mode. So there is no reason to use the PARAVIRT
framework, as it can easily be replaced by an ALTERNATIVE depending
on X86_FEATURE_XENPV.
There are several instances using the PV-aware SWAPGS macro in paths
which are never executed in a Xen PV guest. Replace those with the
plain swapgs instruction. For SWAPGS_UNSAFE_STACK the same applies.
Signed-off-by: Juergen Gross <jgross@suse.com>
---
arch/x86/entry/entry_64.S | 10 +++++-----
arch/x86/include/asm/irqflags.h | 20 ++++++++------------
arch/x86/include/asm/paravirt.h | 20 --------------------
arch/x86/include/asm/paravirt_types.h | 2 --
arch/x86/kernel/asm-offsets_64.c | 1 -
arch/x86/kernel/paravirt.c | 1 -
arch/x86/kernel/paravirt_patch.c | 3 ---
arch/x86/xen/enlighten_pv.c | 3 ---
8 files changed, 13 insertions(+), 47 deletions(-)
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index cad08703c4ad..a876204a73e0 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -669,7 +669,7 @@ native_irq_return_ldt:
*/
pushq %rdi /* Stash user RDI */
- SWAPGS /* to kernel GS */
+ swapgs /* to kernel GS */
SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi /* to kernel CR3 */
movq PER_CPU_VAR(espfix_waddr), %rdi
@@ -699,7 +699,7 @@ native_irq_return_ldt:
orq PER_CPU_VAR(espfix_stack), %rax
SWITCH_TO_USER_CR3_STACK scratch_reg=%rdi
- SWAPGS /* to user GS */
+ swapgs /* to user GS */
popq %rdi /* Restore user RDI */
movq %rax, %rsp
@@ -943,7 +943,7 @@ SYM_CODE_START_LOCAL(paranoid_entry)
ret
.Lparanoid_entry_swapgs:
- SWAPGS
+ swapgs
/*
* The above SAVE_AND_SWITCH_TO_KERNEL_CR3 macro doesn't do an
@@ -1001,7 +1001,7 @@ SYM_CODE_START_LOCAL(paranoid_exit)
jnz restore_regs_and_return_to_kernel
/* We are returning to a context with user GSBASE */
- SWAPGS_UNSAFE_STACK
+ swapgs
jmp restore_regs_and_return_to_kernel
SYM_CODE_END(paranoid_exit)
@@ -1426,7 +1426,7 @@ nmi_no_fsgsbase:
jnz nmi_restore
nmi_swapgs:
- SWAPGS_UNSAFE_STACK
+ swapgs
nmi_restore:
POP_REGS
diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h
index 2dfc8d380dab..8c86edefa115 100644
--- a/arch/x86/include/asm/irqflags.h
+++ b/arch/x86/include/asm/irqflags.h
@@ -131,18 +131,6 @@ static __always_inline unsigned long arch_local_irq_save(void)
#define SAVE_FLAGS(x) pushfq; popq %rax
#endif
-#define SWAPGS swapgs
-/*
- * Currently paravirt can't handle swapgs nicely when we
- * don't have a stack we can rely on (such as a user space
- * stack). So we either find a way around these or just fault
- * and emulate if a guest tries to call swapgs directly.
- *
- * Either way, this is a good way to document that we don't
- * have a reliable stack. x86_64 only.
- */
-#define SWAPGS_UNSAFE_STACK swapgs
-
#define INTERRUPT_RETURN jmp native_iret
#define USERGS_SYSRET64 \
swapgs; \
@@ -170,6 +158,14 @@ static __always_inline int arch_irqs_disabled(void)
return arch_irqs_disabled_flags(flags);
}
+#else
+#ifdef CONFIG_X86_64
+#ifdef CONFIG_XEN_PV
+#define SWAPGS ALTERNATIVE "swapgs", "", X86_FEATURE_XENPV
+#else
+#define SWAPGS swapgs
+#endif
+#endif
#endif /* !__ASSEMBLY__ */
#endif
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index d25cc6830e89..5647bcdba776 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -776,26 +776,6 @@ extern void default_banner(void);
#ifdef CONFIG_X86_64
#ifdef CONFIG_PARAVIRT_XXL
-/*
- * If swapgs is used while the userspace stack is still current,
- * there's no way to call a pvop. The PV replacement *must* be
- * inlined, or the swapgs instruction must be trapped and emulated.
- */
-#define SWAPGS_UNSAFE_STACK \
- PARA_SITE(PARA_PATCH(PV_CPU_swapgs), swapgs)
-
-/*
- * Note: swapgs is very special, and in practise is either going to be
- * implemented with a single "swapgs" instruction or something very
- * special. Either way, we don't need to save any registers for
- * it.
- */
-#define SWAPGS \
- PARA_SITE(PARA_PATCH(PV_CPU_swapgs), \
- ANNOTATE_RETPOLINE_SAFE; \
- call PARA_INDIRECT(pv_ops+PV_CPU_swapgs); \
- )
-
#define USERGS_SYSRET64 \
PARA_SITE(PARA_PATCH(PV_CPU_usergs_sysret64), \
ANNOTATE_RETPOLINE_SAFE; \
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index 0fad9f61c76a..903d71884fa2 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -169,8 +169,6 @@ struct pv_cpu_ops {
frame set up. */
void (*iret)(void);
- void (*swapgs)(void);
-
void (*start_context_switch)(struct task_struct *prev);
void (*end_context_switch)(struct task_struct *next);
#endif
diff --git a/arch/x86/kernel/asm-offsets_64.c b/arch/x86/kernel/asm-offsets_64.c
index 828be792231e..1354bc30614d 100644
--- a/arch/x86/kernel/asm-offsets_64.c
+++ b/arch/x86/kernel/asm-offsets_64.c
@@ -15,7 +15,6 @@ int main(void)
#ifdef CONFIG_PARAVIRT_XXL
OFFSET(PV_CPU_usergs_sysret64, paravirt_patch_template,
cpu.usergs_sysret64);
- OFFSET(PV_CPU_swapgs, paravirt_patch_template, cpu.swapgs);
#ifdef CONFIG_DEBUG_ENTRY
OFFSET(PV_IRQ_save_fl, paravirt_patch_template, irq.save_fl);
#endif
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 6c3407ba6ee9..5e5fcf5c376d 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -312,7 +312,6 @@ struct paravirt_patch_template pv_ops = {
.cpu.usergs_sysret64 = native_usergs_sysret64,
.cpu.iret = native_iret,
- .cpu.swapgs = native_swapgs,
#ifdef CONFIG_X86_IOPL_IOPERM
.cpu.invalidate_io_bitmap = native_tss_invalidate_io_bitmap,
diff --git a/arch/x86/kernel/paravirt_patch.c b/arch/x86/kernel/paravirt_patch.c
index ace6e334cb39..7c518b08aa3c 100644
--- a/arch/x86/kernel/paravirt_patch.c
+++ b/arch/x86/kernel/paravirt_patch.c
@@ -28,7 +28,6 @@ struct patch_xxl {
const unsigned char irq_restore_fl[2];
const unsigned char cpu_wbinvd[2];
const unsigned char cpu_usergs_sysret64[6];
- const unsigned char cpu_swapgs[3];
const unsigned char mov64[3];
};
@@ -43,7 +42,6 @@ static const struct patch_xxl patch_data_xxl = {
.cpu_wbinvd = { 0x0f, 0x09 }, // wbinvd
.cpu_usergs_sysret64 = { 0x0f, 0x01, 0xf8,
0x48, 0x0f, 0x07 }, // swapgs; sysretq
- .cpu_swapgs = { 0x0f, 0x01, 0xf8 }, // swapgs
.mov64 = { 0x48, 0x89, 0xf8 }, // mov %rdi, %rax
};
@@ -86,7 +84,6 @@ unsigned int native_patch(u8 type, void *insn_buff, unsigned long addr,
PATCH_CASE(mmu, write_cr3, xxl, insn_buff, len);
PATCH_CASE(cpu, usergs_sysret64, xxl, insn_buff, len);
- PATCH_CASE(cpu, swapgs, xxl, insn_buff, len);
PATCH_CASE(cpu, wbinvd, xxl, insn_buff, len);
#endif
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 803fbcb398c4..82030d49f4f7 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1085,9 +1085,6 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = {
#endif
.io_delay = xen_io_delay,
- /* Xen takes care of %gs when switching to usermode for us */
- .swapgs = paravirt_nop,
-
.start_context_switch = paravirt_start_context_switch,
.end_context_switch = xen_end_context_switch,
};
--
2.26.2
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH 3/4] x86/pv: switch SWAPGS to ALTERNATIVE
2020-11-16 15:23 ` Juergen Gross via Virtualization
(?)
@ 2020-11-16 16:17 ` Andy Lutomirski
-1 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:17 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, X86 ML, LKML, Linux Virtualization, Andy Lutomirski,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Deep Shah, VMware, Inc.,
Boris Ostrovsky, Stefano Stabellini
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> SWAPGS is used only for interrupts coming from user mode or for
> returning to user mode. So there is no reason to use the PARAVIRT
> framework, as it can easily be replaced by an ALTERNATIVE depending
> on X86_FEATURE_XENPV.
>
> There are several instances using the PV-aware SWAPGS macro in paths
> which are never executed in a Xen PV guest. Replace those with the
> plain swapgs instruction. For SWAPGS_UNSAFE_STACK the same applies.
Acked-by: Andy Lutomirski <luto@kernel.org>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 3/4] x86/pv: switch SWAPGS to ALTERNATIVE
@ 2020-11-16 16:17 ` Andy Lutomirski
0 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:17 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, X86 ML, LKML, Linux Virtualization, Andy Lutomirski,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Deep Shah, VMware, Inc.,
Boris Ostrovsky, Stefano Stabellini
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> SWAPGS is used only for interrupts coming from user mode or for
> returning to user mode. So there is no reason to use the PARAVIRT
> framework, as it can easily be replaced by an ALTERNATIVE depending
> on X86_FEATURE_XENPV.
>
> There are several instances using the PV-aware SWAPGS macro in paths
> which are never executed in a Xen PV guest. Replace those with the
> plain swapgs instruction. For SWAPGS_UNSAFE_STACK the same applies.
Acked-by: Andy Lutomirski <luto@kernel.org>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 3/4] x86/pv: switch SWAPGS to ALTERNATIVE
@ 2020-11-16 16:17 ` Andy Lutomirski
0 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:17 UTC (permalink / raw)
To: Juergen Gross
Cc: Stefano Stabellini, VMware, Inc.,
X86 ML, LKML, Linux Virtualization, Ingo Molnar, Borislav Petkov,
Andy Lutomirski, H. Peter Anvin, xen-devel, Thomas Gleixner,
Boris Ostrovsky
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> SWAPGS is used only for interrupts coming from user mode or for
> returning to user mode. So there is no reason to use the PARAVIRT
> framework, as it can easily be replaced by an ALTERNATIVE depending
> on X86_FEATURE_XENPV.
>
> There are several instances using the PV-aware SWAPGS macro in paths
> which are never executed in a Xen PV guest. Replace those with the
> plain swapgs instruction. For SWAPGS_UNSAFE_STACK the same applies.
Acked-by: Andy Lutomirski <luto@kernel.org>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
2020-11-16 15:22 ` Juergen Gross via Virtualization
@ 2020-11-16 15:23 ` Juergen Gross via Virtualization
-1 siblings, 0 replies; 24+ messages in thread
From: Juergen Gross @ 2020-11-16 15:23 UTC (permalink / raw)
To: xen-devel, x86, linux-kernel, virtualization
Cc: Juergen Gross, Andy Lutomirski, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Deep Shah, VMware, Inc.,
Boris Ostrovsky, Stefano Stabellini
USERGS_SYSRET64 is used to return from a syscall via sysret, but
a Xen PV guest will nevertheless use the iret hypercall, as there
is no sysret PV hypercall defined.
So instead of testing all the prerequisites for doing a sysret and
then mangling the stack for Xen PV again for doing an iret just use
the iret exit from the beginning.
This can easily be done via an ALTERNATIVE like it is done for the
sysenter compat case already.
While at it remove to stale sysret32 remnants.
Signed-off-by: Juergen Gross <jgross@suse.com>
---
arch/x86/entry/entry_64.S | 22 +++++++++-------------
arch/x86/include/asm/irqflags.h | 6 ------
arch/x86/include/asm/paravirt.h | 5 -----
arch/x86/include/asm/paravirt_types.h | 8 --------
arch/x86/kernel/asm-offsets_64.c | 2 --
arch/x86/kernel/paravirt.c | 5 +----
arch/x86/kernel/paravirt_patch.c | 4 ----
arch/x86/xen/enlighten_pv.c | 1 -
arch/x86/xen/xen-asm.S | 20 --------------------
arch/x86/xen/xen-ops.h | 2 --
10 files changed, 10 insertions(+), 65 deletions(-)
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index a876204a73e0..df865eebd3d7 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -46,14 +46,6 @@
.code64
.section .entry.text, "ax"
-#ifdef CONFIG_PARAVIRT_XXL
-SYM_CODE_START(native_usergs_sysret64)
- UNWIND_HINT_EMPTY
- swapgs
- sysretq
-SYM_CODE_END(native_usergs_sysret64)
-#endif /* CONFIG_PARAVIRT_XXL */
-
/*
* 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.
*
@@ -123,12 +115,15 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)
* Try to use SYSRET instead of IRET if we're returning to
* a completely clean 64-bit userspace context. If we're not,
* go to the slow exit path.
+ * In the Xen PV case we must use iret anyway.
*/
- movq RCX(%rsp), %rcx
- movq RIP(%rsp), %r11
- cmpq %rcx, %r11 /* SYSRET requires RCX == RIP */
- jne swapgs_restore_regs_and_return_to_usermode
+ ALTERNATIVE __stringify( \
+ movq RCX(%rsp), %rcx; \
+ movq RIP(%rsp), %r11; \
+ cmpq %rcx, %r11; /* SYSRET requires RCX == RIP */ \
+ jne swapgs_restore_regs_and_return_to_usermode), \
+ "jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV
/*
* On Intel CPUs, SYSRET with non-canonical RCX/RIP will #GP
@@ -215,7 +210,8 @@ syscall_return_via_sysret:
popq %rdi
popq %rsp
- USERGS_SYSRET64
+ swapgs
+ sysretq
SYM_CODE_END(entry_SYSCALL_64)
/*
diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h
index 8c86edefa115..e585a4705b8d 100644
--- a/arch/x86/include/asm/irqflags.h
+++ b/arch/x86/include/asm/irqflags.h
@@ -132,12 +132,6 @@ static __always_inline unsigned long arch_local_irq_save(void)
#endif
#define INTERRUPT_RETURN jmp native_iret
-#define USERGS_SYSRET64 \
- swapgs; \
- sysretq;
-#define USERGS_SYSRET32 \
- swapgs; \
- sysretl
#else
#define INTERRUPT_RETURN iret
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 5647bcdba776..8121cf9b8d81 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -776,11 +776,6 @@ extern void default_banner(void);
#ifdef CONFIG_X86_64
#ifdef CONFIG_PARAVIRT_XXL
-#define USERGS_SYSRET64 \
- PARA_SITE(PARA_PATCH(PV_CPU_usergs_sysret64), \
- ANNOTATE_RETPOLINE_SAFE; \
- jmp PARA_INDIRECT(pv_ops+PV_CPU_usergs_sysret64);)
-
#ifdef CONFIG_DEBUG_ENTRY
#define SAVE_FLAGS(clobbers) \
PARA_SITE(PARA_PATCH(PV_IRQ_save_fl), \
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index 903d71884fa2..55d8b7950e61 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -157,14 +157,6 @@ struct pv_cpu_ops {
u64 (*read_pmc)(int counter);
- /*
- * Switch to usermode gs and return to 64-bit usermode using
- * sysret. Only used in 64-bit kernels to return to 64-bit
- * processes. Usermode register state, including %rsp, must
- * already be restored.
- */
- void (*usergs_sysret64)(void);
-
/* Normal iret. Jump to this with the standard iret stack
frame set up. */
void (*iret)(void);
diff --git a/arch/x86/kernel/asm-offsets_64.c b/arch/x86/kernel/asm-offsets_64.c
index 1354bc30614d..b14533af7676 100644
--- a/arch/x86/kernel/asm-offsets_64.c
+++ b/arch/x86/kernel/asm-offsets_64.c
@@ -13,8 +13,6 @@ int main(void)
{
#ifdef CONFIG_PARAVIRT
#ifdef CONFIG_PARAVIRT_XXL
- OFFSET(PV_CPU_usergs_sysret64, paravirt_patch_template,
- cpu.usergs_sysret64);
#ifdef CONFIG_DEBUG_ENTRY
OFFSET(PV_IRQ_save_fl, paravirt_patch_template, irq.save_fl);
#endif
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 5e5fcf5c376d..18560b71e717 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -135,8 +135,7 @@ unsigned paravirt_patch_default(u8 type, void *insn_buff,
else if (opfunc == _paravirt_ident_64)
ret = paravirt_patch_ident_64(insn_buff, len);
- else if (type == PARAVIRT_PATCH(cpu.iret) ||
- type == PARAVIRT_PATCH(cpu.usergs_sysret64))
+ else if (type == PARAVIRT_PATCH(cpu.iret))
/* If operation requires a jmp, then jmp */
ret = paravirt_patch_jmp(insn_buff, opfunc, addr, len);
#endif
@@ -170,7 +169,6 @@ static u64 native_steal_clock(int cpu)
/* These are in entry.S */
extern void native_iret(void);
-extern void native_usergs_sysret64(void);
static struct resource reserve_ioports = {
.start = 0,
@@ -310,7 +308,6 @@ struct paravirt_patch_template pv_ops = {
.cpu.load_sp0 = native_load_sp0,
- .cpu.usergs_sysret64 = native_usergs_sysret64,
.cpu.iret = native_iret,
#ifdef CONFIG_X86_IOPL_IOPERM
diff --git a/arch/x86/kernel/paravirt_patch.c b/arch/x86/kernel/paravirt_patch.c
index 7c518b08aa3c..2fada2c347c9 100644
--- a/arch/x86/kernel/paravirt_patch.c
+++ b/arch/x86/kernel/paravirt_patch.c
@@ -27,7 +27,6 @@ struct patch_xxl {
const unsigned char mmu_write_cr3[3];
const unsigned char irq_restore_fl[2];
const unsigned char cpu_wbinvd[2];
- const unsigned char cpu_usergs_sysret64[6];
const unsigned char mov64[3];
};
@@ -40,8 +39,6 @@ static const struct patch_xxl patch_data_xxl = {
.mmu_write_cr3 = { 0x0f, 0x22, 0xdf }, // mov %rdi, %cr3
.irq_restore_fl = { 0x57, 0x9d }, // push %rdi; popfq
.cpu_wbinvd = { 0x0f, 0x09 }, // wbinvd
- .cpu_usergs_sysret64 = { 0x0f, 0x01, 0xf8,
- 0x48, 0x0f, 0x07 }, // swapgs; sysretq
.mov64 = { 0x48, 0x89, 0xf8 }, // mov %rdi, %rax
};
@@ -83,7 +80,6 @@ unsigned int native_patch(u8 type, void *insn_buff, unsigned long addr,
PATCH_CASE(mmu, read_cr3, xxl, insn_buff, len);
PATCH_CASE(mmu, write_cr3, xxl, insn_buff, len);
- PATCH_CASE(cpu, usergs_sysret64, xxl, insn_buff, len);
PATCH_CASE(cpu, wbinvd, xxl, insn_buff, len);
#endif
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 82030d49f4f7..2170553f524a 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1060,7 +1060,6 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = {
.read_pmc = xen_read_pmc,
.iret = xen_iret,
- .usergs_sysret64 = xen_sysret64,
.load_tr_desc = paravirt_nop,
.set_ldt = xen_set_ldt,
diff --git a/arch/x86/xen/xen-asm.S b/arch/x86/xen/xen-asm.S
index 1d054c915046..c0630fd9f44e 100644
--- a/arch/x86/xen/xen-asm.S
+++ b/arch/x86/xen/xen-asm.S
@@ -214,26 +214,6 @@ SYM_CODE_START(xen_iret)
jmp hypercall_iret
SYM_CODE_END(xen_iret)
-SYM_CODE_START(xen_sysret64)
- /*
- * We're already on the usermode stack at this point, but
- * still with the kernel gs, so we can easily switch back.
- *
- * tss.sp2 is scratch space.
- */
- movq %rsp, PER_CPU_VAR(cpu_tss_rw + TSS_sp2)
- movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp
-
- pushq $__USER_DS
- pushq PER_CPU_VAR(cpu_tss_rw + TSS_sp2)
- pushq %r11
- pushq $__USER_CS
- pushq %rcx
-
- pushq $VGCF_in_syscall
- jmp hypercall_iret
-SYM_CODE_END(xen_sysret64)
-
/*
* Xen handles syscall callbacks much like ordinary exceptions, which
* means we have:
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index 9546c3384c75..b2fd80a01a36 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -138,8 +138,6 @@ __visible unsigned long xen_read_cr2_direct(void);
/* These are not functions, and cannot be called normally */
__visible void xen_iret(void);
-__visible void xen_sysret32(void);
-__visible void xen_sysret64(void);
extern int xen_panic_handler_init(void);
--
2.26.2
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
@ 2020-11-16 15:23 ` Juergen Gross via Virtualization
0 siblings, 0 replies; 24+ messages in thread
From: Juergen Gross via Virtualization @ 2020-11-16 15:23 UTC (permalink / raw)
To: xen-devel, x86, linux-kernel, virtualization
Cc: Juergen Gross, Stefano Stabellini, VMware, Inc.,
Ingo Molnar, Borislav Petkov, Andy Lutomirski, H. Peter Anvin,
Thomas Gleixner, Boris Ostrovsky
USERGS_SYSRET64 is used to return from a syscall via sysret, but
a Xen PV guest will nevertheless use the iret hypercall, as there
is no sysret PV hypercall defined.
So instead of testing all the prerequisites for doing a sysret and
then mangling the stack for Xen PV again for doing an iret just use
the iret exit from the beginning.
This can easily be done via an ALTERNATIVE like it is done for the
sysenter compat case already.
While at it remove to stale sysret32 remnants.
Signed-off-by: Juergen Gross <jgross@suse.com>
---
arch/x86/entry/entry_64.S | 22 +++++++++-------------
arch/x86/include/asm/irqflags.h | 6 ------
arch/x86/include/asm/paravirt.h | 5 -----
arch/x86/include/asm/paravirt_types.h | 8 --------
arch/x86/kernel/asm-offsets_64.c | 2 --
arch/x86/kernel/paravirt.c | 5 +----
arch/x86/kernel/paravirt_patch.c | 4 ----
arch/x86/xen/enlighten_pv.c | 1 -
arch/x86/xen/xen-asm.S | 20 --------------------
arch/x86/xen/xen-ops.h | 2 --
10 files changed, 10 insertions(+), 65 deletions(-)
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index a876204a73e0..df865eebd3d7 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -46,14 +46,6 @@
.code64
.section .entry.text, "ax"
-#ifdef CONFIG_PARAVIRT_XXL
-SYM_CODE_START(native_usergs_sysret64)
- UNWIND_HINT_EMPTY
- swapgs
- sysretq
-SYM_CODE_END(native_usergs_sysret64)
-#endif /* CONFIG_PARAVIRT_XXL */
-
/*
* 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.
*
@@ -123,12 +115,15 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)
* Try to use SYSRET instead of IRET if we're returning to
* a completely clean 64-bit userspace context. If we're not,
* go to the slow exit path.
+ * In the Xen PV case we must use iret anyway.
*/
- movq RCX(%rsp), %rcx
- movq RIP(%rsp), %r11
- cmpq %rcx, %r11 /* SYSRET requires RCX == RIP */
- jne swapgs_restore_regs_and_return_to_usermode
+ ALTERNATIVE __stringify( \
+ movq RCX(%rsp), %rcx; \
+ movq RIP(%rsp), %r11; \
+ cmpq %rcx, %r11; /* SYSRET requires RCX == RIP */ \
+ jne swapgs_restore_regs_and_return_to_usermode), \
+ "jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV
/*
* On Intel CPUs, SYSRET with non-canonical RCX/RIP will #GP
@@ -215,7 +210,8 @@ syscall_return_via_sysret:
popq %rdi
popq %rsp
- USERGS_SYSRET64
+ swapgs
+ sysretq
SYM_CODE_END(entry_SYSCALL_64)
/*
diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h
index 8c86edefa115..e585a4705b8d 100644
--- a/arch/x86/include/asm/irqflags.h
+++ b/arch/x86/include/asm/irqflags.h
@@ -132,12 +132,6 @@ static __always_inline unsigned long arch_local_irq_save(void)
#endif
#define INTERRUPT_RETURN jmp native_iret
-#define USERGS_SYSRET64 \
- swapgs; \
- sysretq;
-#define USERGS_SYSRET32 \
- swapgs; \
- sysretl
#else
#define INTERRUPT_RETURN iret
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 5647bcdba776..8121cf9b8d81 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -776,11 +776,6 @@ extern void default_banner(void);
#ifdef CONFIG_X86_64
#ifdef CONFIG_PARAVIRT_XXL
-#define USERGS_SYSRET64 \
- PARA_SITE(PARA_PATCH(PV_CPU_usergs_sysret64), \
- ANNOTATE_RETPOLINE_SAFE; \
- jmp PARA_INDIRECT(pv_ops+PV_CPU_usergs_sysret64);)
-
#ifdef CONFIG_DEBUG_ENTRY
#define SAVE_FLAGS(clobbers) \
PARA_SITE(PARA_PATCH(PV_IRQ_save_fl), \
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index 903d71884fa2..55d8b7950e61 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -157,14 +157,6 @@ struct pv_cpu_ops {
u64 (*read_pmc)(int counter);
- /*
- * Switch to usermode gs and return to 64-bit usermode using
- * sysret. Only used in 64-bit kernels to return to 64-bit
- * processes. Usermode register state, including %rsp, must
- * already be restored.
- */
- void (*usergs_sysret64)(void);
-
/* Normal iret. Jump to this with the standard iret stack
frame set up. */
void (*iret)(void);
diff --git a/arch/x86/kernel/asm-offsets_64.c b/arch/x86/kernel/asm-offsets_64.c
index 1354bc30614d..b14533af7676 100644
--- a/arch/x86/kernel/asm-offsets_64.c
+++ b/arch/x86/kernel/asm-offsets_64.c
@@ -13,8 +13,6 @@ int main(void)
{
#ifdef CONFIG_PARAVIRT
#ifdef CONFIG_PARAVIRT_XXL
- OFFSET(PV_CPU_usergs_sysret64, paravirt_patch_template,
- cpu.usergs_sysret64);
#ifdef CONFIG_DEBUG_ENTRY
OFFSET(PV_IRQ_save_fl, paravirt_patch_template, irq.save_fl);
#endif
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 5e5fcf5c376d..18560b71e717 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -135,8 +135,7 @@ unsigned paravirt_patch_default(u8 type, void *insn_buff,
else if (opfunc == _paravirt_ident_64)
ret = paravirt_patch_ident_64(insn_buff, len);
- else if (type == PARAVIRT_PATCH(cpu.iret) ||
- type == PARAVIRT_PATCH(cpu.usergs_sysret64))
+ else if (type == PARAVIRT_PATCH(cpu.iret))
/* If operation requires a jmp, then jmp */
ret = paravirt_patch_jmp(insn_buff, opfunc, addr, len);
#endif
@@ -170,7 +169,6 @@ static u64 native_steal_clock(int cpu)
/* These are in entry.S */
extern void native_iret(void);
-extern void native_usergs_sysret64(void);
static struct resource reserve_ioports = {
.start = 0,
@@ -310,7 +308,6 @@ struct paravirt_patch_template pv_ops = {
.cpu.load_sp0 = native_load_sp0,
- .cpu.usergs_sysret64 = native_usergs_sysret64,
.cpu.iret = native_iret,
#ifdef CONFIG_X86_IOPL_IOPERM
diff --git a/arch/x86/kernel/paravirt_patch.c b/arch/x86/kernel/paravirt_patch.c
index 7c518b08aa3c..2fada2c347c9 100644
--- a/arch/x86/kernel/paravirt_patch.c
+++ b/arch/x86/kernel/paravirt_patch.c
@@ -27,7 +27,6 @@ struct patch_xxl {
const unsigned char mmu_write_cr3[3];
const unsigned char irq_restore_fl[2];
const unsigned char cpu_wbinvd[2];
- const unsigned char cpu_usergs_sysret64[6];
const unsigned char mov64[3];
};
@@ -40,8 +39,6 @@ static const struct patch_xxl patch_data_xxl = {
.mmu_write_cr3 = { 0x0f, 0x22, 0xdf }, // mov %rdi, %cr3
.irq_restore_fl = { 0x57, 0x9d }, // push %rdi; popfq
.cpu_wbinvd = { 0x0f, 0x09 }, // wbinvd
- .cpu_usergs_sysret64 = { 0x0f, 0x01, 0xf8,
- 0x48, 0x0f, 0x07 }, // swapgs; sysretq
.mov64 = { 0x48, 0x89, 0xf8 }, // mov %rdi, %rax
};
@@ -83,7 +80,6 @@ unsigned int native_patch(u8 type, void *insn_buff, unsigned long addr,
PATCH_CASE(mmu, read_cr3, xxl, insn_buff, len);
PATCH_CASE(mmu, write_cr3, xxl, insn_buff, len);
- PATCH_CASE(cpu, usergs_sysret64, xxl, insn_buff, len);
PATCH_CASE(cpu, wbinvd, xxl, insn_buff, len);
#endif
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 82030d49f4f7..2170553f524a 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1060,7 +1060,6 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = {
.read_pmc = xen_read_pmc,
.iret = xen_iret,
- .usergs_sysret64 = xen_sysret64,
.load_tr_desc = paravirt_nop,
.set_ldt = xen_set_ldt,
diff --git a/arch/x86/xen/xen-asm.S b/arch/x86/xen/xen-asm.S
index 1d054c915046..c0630fd9f44e 100644
--- a/arch/x86/xen/xen-asm.S
+++ b/arch/x86/xen/xen-asm.S
@@ -214,26 +214,6 @@ SYM_CODE_START(xen_iret)
jmp hypercall_iret
SYM_CODE_END(xen_iret)
-SYM_CODE_START(xen_sysret64)
- /*
- * We're already on the usermode stack at this point, but
- * still with the kernel gs, so we can easily switch back.
- *
- * tss.sp2 is scratch space.
- */
- movq %rsp, PER_CPU_VAR(cpu_tss_rw + TSS_sp2)
- movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp
-
- pushq $__USER_DS
- pushq PER_CPU_VAR(cpu_tss_rw + TSS_sp2)
- pushq %r11
- pushq $__USER_CS
- pushq %rcx
-
- pushq $VGCF_in_syscall
- jmp hypercall_iret
-SYM_CODE_END(xen_sysret64)
-
/*
* Xen handles syscall callbacks much like ordinary exceptions, which
* means we have:
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index 9546c3384c75..b2fd80a01a36 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -138,8 +138,6 @@ __visible unsigned long xen_read_cr2_direct(void);
/* These are not functions, and cannot be called normally */
__visible void xen_iret(void);
-__visible void xen_sysret32(void);
-__visible void xen_sysret64(void);
extern int xen_panic_handler_init(void);
--
2.26.2
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
2020-11-16 15:23 ` Juergen Gross via Virtualization
(?)
@ 2020-11-16 16:28 ` Andy Lutomirski
-1 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:28 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, X86 ML, LKML, Linux Virtualization, Andy Lutomirski,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Deep Shah, VMware, Inc.,
Boris Ostrovsky, Stefano Stabellini
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> USERGS_SYSRET64 is used to return from a syscall via sysret, but
> a Xen PV guest will nevertheless use the iret hypercall, as there
> is no sysret PV hypercall defined.
>
> So instead of testing all the prerequisites for doing a sysret and
> then mangling the stack for Xen PV again for doing an iret just use
> the iret exit from the beginning.
>
> This can easily be done via an ALTERNATIVE like it is done for the
> sysenter compat case already.
>
> While at it remove to stale sysret32 remnants.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Andy Lutomirski <luto@kernel.org>
FWIW, you've lost the VGCF_in_syscall optimization. Let me see if I
can give it back to you better.
> ---
> arch/x86/entry/entry_64.S | 22 +++++++++-------------
> arch/x86/include/asm/irqflags.h | 6 ------
> arch/x86/include/asm/paravirt.h | 5 -----
> arch/x86/include/asm/paravirt_types.h | 8 --------
> arch/x86/kernel/asm-offsets_64.c | 2 --
> arch/x86/kernel/paravirt.c | 5 +----
> arch/x86/kernel/paravirt_patch.c | 4 ----
> arch/x86/xen/enlighten_pv.c | 1 -
> arch/x86/xen/xen-asm.S | 20 --------------------
> arch/x86/xen/xen-ops.h | 2 --
> 10 files changed, 10 insertions(+), 65 deletions(-)
>
> diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
> index a876204a73e0..df865eebd3d7 100644
> --- a/arch/x86/entry/entry_64.S
> +++ b/arch/x86/entry/entry_64.S
> @@ -46,14 +46,6 @@
> .code64
> .section .entry.text, "ax"
>
> -#ifdef CONFIG_PARAVIRT_XXL
> -SYM_CODE_START(native_usergs_sysret64)
> - UNWIND_HINT_EMPTY
> - swapgs
> - sysretq
> -SYM_CODE_END(native_usergs_sysret64)
> -#endif /* CONFIG_PARAVIRT_XXL */
> -
> /*
> * 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.
> *
> @@ -123,12 +115,15 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)
> * Try to use SYSRET instead of IRET if we're returning to
> * a completely clean 64-bit userspace context. If we're not,
> * go to the slow exit path.
> + * In the Xen PV case we must use iret anyway.
> */
> - movq RCX(%rsp), %rcx
> - movq RIP(%rsp), %r11
>
> - cmpq %rcx, %r11 /* SYSRET requires RCX == RIP */
> - jne swapgs_restore_regs_and_return_to_usermode
> + ALTERNATIVE __stringify( \
> + movq RCX(%rsp), %rcx; \
> + movq RIP(%rsp), %r11; \
> + cmpq %rcx, %r11; /* SYSRET requires RCX == RIP */ \
> + jne swapgs_restore_regs_and_return_to_usermode), \
> + "jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV
I'm not in love with this save-a-few-bytes stringify, but I can live with it.
--Andy
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
@ 2020-11-16 16:28 ` Andy Lutomirski
0 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:28 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, X86 ML, LKML, Linux Virtualization, Andy Lutomirski,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Deep Shah, VMware, Inc.,
Boris Ostrovsky, Stefano Stabellini
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> USERGS_SYSRET64 is used to return from a syscall via sysret, but
> a Xen PV guest will nevertheless use the iret hypercall, as there
> is no sysret PV hypercall defined.
>
> So instead of testing all the prerequisites for doing a sysret and
> then mangling the stack for Xen PV again for doing an iret just use
> the iret exit from the beginning.
>
> This can easily be done via an ALTERNATIVE like it is done for the
> sysenter compat case already.
>
> While at it remove to stale sysret32 remnants.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Andy Lutomirski <luto@kernel.org>
FWIW, you've lost the VGCF_in_syscall optimization. Let me see if I
can give it back to you better.
> ---
> arch/x86/entry/entry_64.S | 22 +++++++++-------------
> arch/x86/include/asm/irqflags.h | 6 ------
> arch/x86/include/asm/paravirt.h | 5 -----
> arch/x86/include/asm/paravirt_types.h | 8 --------
> arch/x86/kernel/asm-offsets_64.c | 2 --
> arch/x86/kernel/paravirt.c | 5 +----
> arch/x86/kernel/paravirt_patch.c | 4 ----
> arch/x86/xen/enlighten_pv.c | 1 -
> arch/x86/xen/xen-asm.S | 20 --------------------
> arch/x86/xen/xen-ops.h | 2 --
> 10 files changed, 10 insertions(+), 65 deletions(-)
>
> diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
> index a876204a73e0..df865eebd3d7 100644
> --- a/arch/x86/entry/entry_64.S
> +++ b/arch/x86/entry/entry_64.S
> @@ -46,14 +46,6 @@
> .code64
> .section .entry.text, "ax"
>
> -#ifdef CONFIG_PARAVIRT_XXL
> -SYM_CODE_START(native_usergs_sysret64)
> - UNWIND_HINT_EMPTY
> - swapgs
> - sysretq
> -SYM_CODE_END(native_usergs_sysret64)
> -#endif /* CONFIG_PARAVIRT_XXL */
> -
> /*
> * 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.
> *
> @@ -123,12 +115,15 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)
> * Try to use SYSRET instead of IRET if we're returning to
> * a completely clean 64-bit userspace context. If we're not,
> * go to the slow exit path.
> + * In the Xen PV case we must use iret anyway.
> */
> - movq RCX(%rsp), %rcx
> - movq RIP(%rsp), %r11
>
> - cmpq %rcx, %r11 /* SYSRET requires RCX == RIP */
> - jne swapgs_restore_regs_and_return_to_usermode
> + ALTERNATIVE __stringify( \
> + movq RCX(%rsp), %rcx; \
> + movq RIP(%rsp), %r11; \
> + cmpq %rcx, %r11; /* SYSRET requires RCX == RIP */ \
> + jne swapgs_restore_regs_and_return_to_usermode), \
> + "jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV
I'm not in love with this save-a-few-bytes stringify, but I can live with it.
--Andy
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
@ 2020-11-16 16:28 ` Andy Lutomirski
0 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:28 UTC (permalink / raw)
To: Juergen Gross
Cc: Stefano Stabellini, VMware, Inc.,
X86 ML, LKML, Linux Virtualization, Ingo Molnar, Borislav Petkov,
Andy Lutomirski, H. Peter Anvin, xen-devel, Thomas Gleixner,
Boris Ostrovsky
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> USERGS_SYSRET64 is used to return from a syscall via sysret, but
> a Xen PV guest will nevertheless use the iret hypercall, as there
> is no sysret PV hypercall defined.
>
> So instead of testing all the prerequisites for doing a sysret and
> then mangling the stack for Xen PV again for doing an iret just use
> the iret exit from the beginning.
>
> This can easily be done via an ALTERNATIVE like it is done for the
> sysenter compat case already.
>
> While at it remove to stale sysret32 remnants.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Andy Lutomirski <luto@kernel.org>
FWIW, you've lost the VGCF_in_syscall optimization. Let me see if I
can give it back to you better.
> ---
> arch/x86/entry/entry_64.S | 22 +++++++++-------------
> arch/x86/include/asm/irqflags.h | 6 ------
> arch/x86/include/asm/paravirt.h | 5 -----
> arch/x86/include/asm/paravirt_types.h | 8 --------
> arch/x86/kernel/asm-offsets_64.c | 2 --
> arch/x86/kernel/paravirt.c | 5 +----
> arch/x86/kernel/paravirt_patch.c | 4 ----
> arch/x86/xen/enlighten_pv.c | 1 -
> arch/x86/xen/xen-asm.S | 20 --------------------
> arch/x86/xen/xen-ops.h | 2 --
> 10 files changed, 10 insertions(+), 65 deletions(-)
>
> diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
> index a876204a73e0..df865eebd3d7 100644
> --- a/arch/x86/entry/entry_64.S
> +++ b/arch/x86/entry/entry_64.S
> @@ -46,14 +46,6 @@
> .code64
> .section .entry.text, "ax"
>
> -#ifdef CONFIG_PARAVIRT_XXL
> -SYM_CODE_START(native_usergs_sysret64)
> - UNWIND_HINT_EMPTY
> - swapgs
> - sysretq
> -SYM_CODE_END(native_usergs_sysret64)
> -#endif /* CONFIG_PARAVIRT_XXL */
> -
> /*
> * 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.
> *
> @@ -123,12 +115,15 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)
> * Try to use SYSRET instead of IRET if we're returning to
> * a completely clean 64-bit userspace context. If we're not,
> * go to the slow exit path.
> + * In the Xen PV case we must use iret anyway.
> */
> - movq RCX(%rsp), %rcx
> - movq RIP(%rsp), %r11
>
> - cmpq %rcx, %r11 /* SYSRET requires RCX == RIP */
> - jne swapgs_restore_regs_and_return_to_usermode
> + ALTERNATIVE __stringify( \
> + movq RCX(%rsp), %rcx; \
> + movq RIP(%rsp), %r11; \
> + cmpq %rcx, %r11; /* SYSRET requires RCX == RIP */ \
> + jne swapgs_restore_regs_and_return_to_usermode), \
> + "jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV
I'm not in love with this save-a-few-bytes stringify, but I can live with it.
--Andy
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
2020-11-16 16:28 ` Andy Lutomirski
@ 2020-11-17 8:19 ` Jürgen Groß via Virtualization
-1 siblings, 0 replies; 24+ messages in thread
From: Jürgen Groß @ 2020-11-17 8:19 UTC (permalink / raw)
To: Andy Lutomirski
Cc: xen-devel, X86 ML, LKML, Linux Virtualization, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, H. Peter Anvin, Deep Shah, VMware,
Inc.,
Boris Ostrovsky, Stefano Stabellini
[-- Attachment #1.1.1: Type: text/plain, Size: 969 bytes --]
On 16.11.20 17:28, Andy Lutomirski wrote:
> On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>>
>> USERGS_SYSRET64 is used to return from a syscall via sysret, but
>> a Xen PV guest will nevertheless use the iret hypercall, as there
>> is no sysret PV hypercall defined.
>>
>> So instead of testing all the prerequisites for doing a sysret and
>> then mangling the stack for Xen PV again for doing an iret just use
>> the iret exit from the beginning.
>>
>> This can easily be done via an ALTERNATIVE like it is done for the
>> sysenter compat case already.
>>
>> While at it remove to stale sysret32 remnants.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>
> Acked-by: Andy Lutomirski <luto@kernel.org>
>
> FWIW, you've lost the VGCF_in_syscall optimization. Let me see if I
> can give it back to you better.
Ah, right.
Nevertheless a simple kernel build is about 0.5% faster with this
patch.
Juergen
[-- Attachment #1.1.2: OpenPGP_0xB0DE9DD628BF132F.asc --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
@ 2020-11-17 8:19 ` Jürgen Groß via Virtualization
0 siblings, 0 replies; 24+ messages in thread
From: Jürgen Groß via Virtualization @ 2020-11-17 8:19 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Stefano Stabellini, VMware, Inc.,
X86 ML, LKML, Linux Virtualization, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, xen-devel, Thomas Gleixner, Boris Ostrovsky
[-- Attachment #1.1.1.1: Type: text/plain, Size: 969 bytes --]
On 16.11.20 17:28, Andy Lutomirski wrote:
> On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>>
>> USERGS_SYSRET64 is used to return from a syscall via sysret, but
>> a Xen PV guest will nevertheless use the iret hypercall, as there
>> is no sysret PV hypercall defined.
>>
>> So instead of testing all the prerequisites for doing a sysret and
>> then mangling the stack for Xen PV again for doing an iret just use
>> the iret exit from the beginning.
>>
>> This can easily be done via an ALTERNATIVE like it is done for the
>> sysenter compat case already.
>>
>> While at it remove to stale sysret32 remnants.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>
> Acked-by: Andy Lutomirski <luto@kernel.org>
>
> FWIW, you've lost the VGCF_in_syscall optimization. Let me see if I
> can give it back to you better.
Ah, right.
Nevertheless a simple kernel build is about 0.5% faster with this
patch.
Juergen
[-- Attachment #1.1.1.2: OpenPGP_0xB0DE9DD628BF132F.asc --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
2020-11-16 15:23 ` Juergen Gross via Virtualization
(?)
@ 2020-11-16 16:30 ` Andy Lutomirski
-1 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:30 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, X86 ML, LKML, Linux Virtualization, Andy Lutomirski,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Deep Shah, VMware, Inc.,
Boris Ostrovsky, Stefano Stabellini
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> USERGS_SYSRET64 is used to return from a syscall via sysret, but
> a Xen PV guest will nevertheless use the iret hypercall, as there
> is no sysret PV hypercall defined.
>
> So instead of testing all the prerequisites for doing a sysret and
> then mangling the stack for Xen PV again for doing an iret just use
> the iret exit from the beginning.
>
> This can easily be done via an ALTERNATIVE like it is done for the
> sysenter compat case already.
>
> While at it remove to stale sysret32 remnants.
s/to/the/
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
@ 2020-11-16 16:30 ` Andy Lutomirski
0 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:30 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, X86 ML, LKML, Linux Virtualization, Andy Lutomirski,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
Deep Shah, VMware, Inc.,
Boris Ostrovsky, Stefano Stabellini
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> USERGS_SYSRET64 is used to return from a syscall via sysret, but
> a Xen PV guest will nevertheless use the iret hypercall, as there
> is no sysret PV hypercall defined.
>
> So instead of testing all the prerequisites for doing a sysret and
> then mangling the stack for Xen PV again for doing an iret just use
> the iret exit from the beginning.
>
> This can easily be done via an ALTERNATIVE like it is done for the
> sysenter compat case already.
>
> While at it remove to stale sysret32 remnants.
s/to/the/
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 4/4] x86/xen: drop USERGS_SYSRET64 paravirt call
@ 2020-11-16 16:30 ` Andy Lutomirski
0 siblings, 0 replies; 24+ messages in thread
From: Andy Lutomirski @ 2020-11-16 16:30 UTC (permalink / raw)
To: Juergen Gross
Cc: Stefano Stabellini, VMware, Inc.,
X86 ML, LKML, Linux Virtualization, Ingo Molnar, Borislav Petkov,
Andy Lutomirski, H. Peter Anvin, xen-devel, Thomas Gleixner,
Boris Ostrovsky
On Mon, Nov 16, 2020 at 7:23 AM Juergen Gross <jgross@suse.com> wrote:
>
> USERGS_SYSRET64 is used to return from a syscall via sysret, but
> a Xen PV guest will nevertheless use the iret hypercall, as there
> is no sysret PV hypercall defined.
>
> So instead of testing all the prerequisites for doing a sysret and
> then mangling the stack for Xen PV again for doing an iret just use
> the iret exit from the beginning.
>
> This can easily be done via an ALTERNATIVE like it is done for the
> sysenter compat case already.
>
> While at it remove to stale sysret32 remnants.
s/to/the/
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 0/4] x86/xen: do some paravirt cleanup
2020-11-16 15:22 ` Juergen Gross via Virtualization
@ 2020-11-17 10:00 ` Peter Zijlstra
-1 siblings, 0 replies; 24+ messages in thread
From: Peter Zijlstra @ 2020-11-17 10:00 UTC (permalink / raw)
To: Juergen Gross
Cc: xen-devel, x86, linux-kernel, virtualization, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, H. Peter Anvin, Boris Ostrovsky,
Stefano Stabellini, Andy Lutomirski, Deep Shah, VMware, Inc.
On Mon, Nov 16, 2020 at 04:22:57PM +0100, Juergen Gross wrote:
> Eliminate the usergs_sysret64 paravirt call completely and switch
> the swapgs one to use ALTERNATIVE instead. This requires to fix the
> IST based exception entries for Xen PV to use the same mechanism as
> NMI and debug exception already do.
>
> Juergen Gross (4):
> x86/xen: use specific Xen pv interrupt entry for MCE
> x86/xen: use specific Xen pv interrupt entry for DF
> x86/pv: switch SWAPGS to ALTERNATIVE
> x86/xen: drop USERGS_SYSRET64 paravirt call
Looks 'sane' :-))
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 0/4] x86/xen: do some paravirt cleanup
@ 2020-11-17 10:00 ` Peter Zijlstra
0 siblings, 0 replies; 24+ messages in thread
From: Peter Zijlstra @ 2020-11-17 10:00 UTC (permalink / raw)
To: Juergen Gross
Cc: Stefano Stabellini, VMware, Inc.,
x86, linux-kernel, virtualization, Ingo Molnar, Borislav Petkov,
Andy Lutomirski, H. Peter Anvin, xen-devel, Thomas Gleixner,
Boris Ostrovsky
On Mon, Nov 16, 2020 at 04:22:57PM +0100, Juergen Gross wrote:
> Eliminate the usergs_sysret64 paravirt call completely and switch
> the swapgs one to use ALTERNATIVE instead. This requires to fix the
> IST based exception entries for Xen PV to use the same mechanism as
> NMI and debug exception already do.
>
> Juergen Gross (4):
> x86/xen: use specific Xen pv interrupt entry for MCE
> x86/xen: use specific Xen pv interrupt entry for DF
> x86/pv: switch SWAPGS to ALTERNATIVE
> x86/xen: drop USERGS_SYSRET64 paravirt call
Looks 'sane' :-))
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 24+ messages in thread