From: Juergen Gross <jgross@suse.com> To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Cc: Juergen Gross <jgross@suse.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>, Deep Shah <sdeep@vmware.com>, "VMware, Inc." <pv-drivers@vmware.com> Subject: [PATCH v5 06/12] x86: add new features for paravirt patching Date: Mon, 8 Mar 2021 13:28:38 +0100 [thread overview] Message-ID: <20210308122844.30488-7-jgross@suse.com> (raw) In-Reply-To: <20210308122844.30488-1-jgross@suse.com> For being able to switch paravirt patching from special cased custom code sequences to ALTERNATIVE handling some X86_FEATURE_* are needed as new features. This enables to have the standard indirect pv call as the default code and to patch that with the non-Xen custom code sequence via ALTERNATIVE patching later. Make sure paravirt patching is performed before alternative patching. Signed-off-by: Juergen Gross <jgross@suse.com> --- V3: - add comment (Boris Petkov) - no negative features (Boris Petkov) V4: - move paravirt_set_cap() to paravirt-spinlocks.c --- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/include/asm/paravirt.h | 10 ++++++++++ arch/x86/kernel/alternative.c | 30 ++++++++++++++++++++++++++-- arch/x86/kernel/paravirt-spinlocks.c | 9 +++++++++ 4 files changed, 49 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index cc96e26d69f7..b440c950246d 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -236,6 +236,8 @@ #define X86_FEATURE_EPT_AD ( 8*32+17) /* Intel Extended Page Table access-dirty bit */ #define X86_FEATURE_VMCALL ( 8*32+18) /* "" Hypervisor supports the VMCALL instruction */ #define X86_FEATURE_VMW_VMMCALL ( 8*32+19) /* "" VMware prefers VMMCALL hypercall instruction */ +#define X86_FEATURE_PVUNLOCK ( 8*32+20) /* "" PV unlock function */ +#define X86_FEATURE_VCPUPREEMPT ( 8*32+21) /* "" PV vcpu_is_preempted function */ /* Intel-defined CPU features, CPUID level 0x00000007:0 (EBX), word 9 */ #define X86_FEATURE_FSGSBASE ( 9*32+ 0) /* RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/ diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 1e45b46fae84..8c354099d9c3 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -47,6 +47,10 @@ static inline u64 paravirt_steal_clock(int cpu) return static_call(pv_steal_clock)(cpu); } +#ifdef CONFIG_PARAVIRT_SPINLOCKS +void __init paravirt_set_cap(void); +#endif + /* The paravirtualized I/O functions */ static inline void slow_down_io(void) { @@ -811,5 +815,11 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct *mm) { } #endif + +#ifndef CONFIG_PARAVIRT_SPINLOCKS +static inline void paravirt_set_cap(void) +{ +} +#endif #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_PARAVIRT_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 1296a90aa5b8..ab9ad729fc5a 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -28,6 +28,7 @@ #include <asm/insn.h> #include <asm/io.h> #include <asm/fixmap.h> +#include <asm/paravirt.h> int __read_mostly alternatives_patched; @@ -724,6 +725,33 @@ void __init alternative_instructions(void) * patching. */ + /* + * Paravirt patching and alternative patching can be combined to + * replace a function call with a short direct code sequence (e.g. + * by setting a constant return value instead of doing that in an + * external function). + * In order to make this work the following sequence is required: + * 1. set (artificial) features depending on used paravirt + * functions which can later influence alternative patching + * 2. apply paravirt patching (generally replacing an indirect + * function call with a direct one) + * 3. apply alternative patching (e.g. replacing a direct function + * call with a custom code sequence) + * Doing paravirt patching after alternative patching would clobber + * the optimization of the custom code with a function call again. + */ + paravirt_set_cap(); + + /* + * First patch paravirt functions, such that we overwrite the indirect + * call with the direct call. + */ + apply_paravirt(__parainstructions, __parainstructions_end); + + /* + * Then patch alternatives, such that those paravirt calls that are in + * alternatives can be overwritten by their immediate fragments. + */ apply_alternatives(__alt_instructions, __alt_instructions_end); #ifdef CONFIG_SMP @@ -742,8 +770,6 @@ void __init alternative_instructions(void) } #endif - apply_paravirt(__parainstructions, __parainstructions_end); - restart_nmi(); alternatives_patched = 1; } diff --git a/arch/x86/kernel/paravirt-spinlocks.c b/arch/x86/kernel/paravirt-spinlocks.c index 4f75d0cf6305..9e1ea99ad9df 100644 --- a/arch/x86/kernel/paravirt-spinlocks.c +++ b/arch/x86/kernel/paravirt-spinlocks.c @@ -32,3 +32,12 @@ bool pv_is_native_vcpu_is_preempted(void) return pv_ops.lock.vcpu_is_preempted.func == __raw_callee_save___native_vcpu_is_preempted; } + +void __init paravirt_set_cap(void) +{ + if (!pv_is_native_spin_unlock()) + setup_force_cpu_cap(X86_FEATURE_PVUNLOCK); + + if (!pv_is_native_vcpu_is_preempted()) + setup_force_cpu_cap(X86_FEATURE_VCPUPREEMPT); +} -- 2.26.2
WARNING: multiple messages have this Message-ID (diff)
From: Juergen Gross via Virtualization <virtualization@lists.linux-foundation.org> To: xen-devel@lists.xenproject.org, x86@kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Cc: Juergen Gross <jgross@suse.com>, "VMware, Inc." <pv-drivers@vmware.com>, Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>, Thomas Gleixner <tglx@linutronix.de> Subject: [PATCH v5 06/12] x86: add new features for paravirt patching Date: Mon, 8 Mar 2021 13:28:38 +0100 [thread overview] Message-ID: <20210308122844.30488-7-jgross@suse.com> (raw) In-Reply-To: <20210308122844.30488-1-jgross@suse.com> For being able to switch paravirt patching from special cased custom code sequences to ALTERNATIVE handling some X86_FEATURE_* are needed as new features. This enables to have the standard indirect pv call as the default code and to patch that with the non-Xen custom code sequence via ALTERNATIVE patching later. Make sure paravirt patching is performed before alternative patching. Signed-off-by: Juergen Gross <jgross@suse.com> --- V3: - add comment (Boris Petkov) - no negative features (Boris Petkov) V4: - move paravirt_set_cap() to paravirt-spinlocks.c --- arch/x86/include/asm/cpufeatures.h | 2 ++ arch/x86/include/asm/paravirt.h | 10 ++++++++++ arch/x86/kernel/alternative.c | 30 ++++++++++++++++++++++++++-- arch/x86/kernel/paravirt-spinlocks.c | 9 +++++++++ 4 files changed, 49 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index cc96e26d69f7..b440c950246d 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -236,6 +236,8 @@ #define X86_FEATURE_EPT_AD ( 8*32+17) /* Intel Extended Page Table access-dirty bit */ #define X86_FEATURE_VMCALL ( 8*32+18) /* "" Hypervisor supports the VMCALL instruction */ #define X86_FEATURE_VMW_VMMCALL ( 8*32+19) /* "" VMware prefers VMMCALL hypercall instruction */ +#define X86_FEATURE_PVUNLOCK ( 8*32+20) /* "" PV unlock function */ +#define X86_FEATURE_VCPUPREEMPT ( 8*32+21) /* "" PV vcpu_is_preempted function */ /* Intel-defined CPU features, CPUID level 0x00000007:0 (EBX), word 9 */ #define X86_FEATURE_FSGSBASE ( 9*32+ 0) /* RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/ diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 1e45b46fae84..8c354099d9c3 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -47,6 +47,10 @@ static inline u64 paravirt_steal_clock(int cpu) return static_call(pv_steal_clock)(cpu); } +#ifdef CONFIG_PARAVIRT_SPINLOCKS +void __init paravirt_set_cap(void); +#endif + /* The paravirtualized I/O functions */ static inline void slow_down_io(void) { @@ -811,5 +815,11 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct *mm) { } #endif + +#ifndef CONFIG_PARAVIRT_SPINLOCKS +static inline void paravirt_set_cap(void) +{ +} +#endif #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_PARAVIRT_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 1296a90aa5b8..ab9ad729fc5a 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -28,6 +28,7 @@ #include <asm/insn.h> #include <asm/io.h> #include <asm/fixmap.h> +#include <asm/paravirt.h> int __read_mostly alternatives_patched; @@ -724,6 +725,33 @@ void __init alternative_instructions(void) * patching. */ + /* + * Paravirt patching and alternative patching can be combined to + * replace a function call with a short direct code sequence (e.g. + * by setting a constant return value instead of doing that in an + * external function). + * In order to make this work the following sequence is required: + * 1. set (artificial) features depending on used paravirt + * functions which can later influence alternative patching + * 2. apply paravirt patching (generally replacing an indirect + * function call with a direct one) + * 3. apply alternative patching (e.g. replacing a direct function + * call with a custom code sequence) + * Doing paravirt patching after alternative patching would clobber + * the optimization of the custom code with a function call again. + */ + paravirt_set_cap(); + + /* + * First patch paravirt functions, such that we overwrite the indirect + * call with the direct call. + */ + apply_paravirt(__parainstructions, __parainstructions_end); + + /* + * Then patch alternatives, such that those paravirt calls that are in + * alternatives can be overwritten by their immediate fragments. + */ apply_alternatives(__alt_instructions, __alt_instructions_end); #ifdef CONFIG_SMP @@ -742,8 +770,6 @@ void __init alternative_instructions(void) } #endif - apply_paravirt(__parainstructions, __parainstructions_end); - restart_nmi(); alternatives_patched = 1; } diff --git a/arch/x86/kernel/paravirt-spinlocks.c b/arch/x86/kernel/paravirt-spinlocks.c index 4f75d0cf6305..9e1ea99ad9df 100644 --- a/arch/x86/kernel/paravirt-spinlocks.c +++ b/arch/x86/kernel/paravirt-spinlocks.c @@ -32,3 +32,12 @@ bool pv_is_native_vcpu_is_preempted(void) return pv_ops.lock.vcpu_is_preempted.func == __raw_callee_save___native_vcpu_is_preempted; } + +void __init paravirt_set_cap(void) +{ + if (!pv_is_native_spin_unlock()) + setup_force_cpu_cap(X86_FEATURE_PVUNLOCK); + + if (!pv_is_native_vcpu_is_preempted()) + setup_force_cpu_cap(X86_FEATURE_VCPUPREEMPT); +} -- 2.26.2 _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2021-03-08 12:29 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-03-08 12:28 [PATCH v5 00/12] x86: major paravirt cleanup Juergen Gross 2021-03-08 12:28 ` Juergen Gross 2021-03-08 12:28 ` Juergen Gross via Virtualization 2021-03-08 12:28 ` [PATCH v5 01/12] staticcall: move struct static_call_key definition to static_call_types.h Juergen Gross 2021-03-08 16:27 ` Peter Zijlstra 2021-03-08 12:28 ` [PATCH v5 02/12] x86/paravirt: switch time pvops functions to use static_call() Juergen Gross 2021-03-08 12:28 ` Juergen Gross 2021-03-08 12:28 ` Juergen Gross via Virtualization 2021-03-08 17:00 ` Boris Ostrovsky 2021-03-08 17:00 ` Boris Ostrovsky 2021-03-08 17:00 ` Boris Ostrovsky 2021-03-09 6:14 ` Jürgen Groß 2021-03-09 6:14 ` Jürgen Groß 2021-03-09 6:14 ` Jürgen Groß via Virtualization 2021-03-08 12:28 ` [PATCH v5 03/12] x86/alternative: drop feature parameter from ALTINSTR_REPLACEMENT() Juergen Gross 2021-03-08 12:28 ` [PATCH v5 04/12] x86/alternative: support not-feature Juergen Gross 2021-03-08 12:28 ` [PATCH v5 05/12] x86/alternative: support ALTERNATIVE_TERNARY Juergen Gross 2021-03-08 12:28 ` Juergen Gross [this message] 2021-03-08 12:28 ` [PATCH v5 06/12] x86: add new features for paravirt patching Juergen Gross via Virtualization 2021-03-08 12:28 ` [PATCH v5 07/12] x86/paravirt: remove no longer needed 32-bit pvops cruft Juergen Gross 2021-03-08 12:28 ` Juergen Gross via Virtualization 2021-03-08 12:28 ` [PATCH v5 08/12] x86/paravirt: simplify paravirt macros Juergen Gross 2021-03-08 12:28 ` Juergen Gross via Virtualization 2021-03-08 12:28 ` [PATCH v5 09/12] x86/paravirt: switch iret pvops to ALTERNATIVE Juergen Gross 2021-03-08 12:28 ` Juergen Gross via Virtualization 2021-03-08 12:28 ` [PATCH v5 10/12] x86/paravirt: add new macros PVOP_ALT* supporting pvops in ALTERNATIVEs Juergen Gross 2021-03-08 12:28 ` Juergen Gross via Virtualization 2021-03-08 12:28 ` [PATCH v5 11/12] x86/paravirt: switch functions with custom code to ALTERNATIVE Juergen Gross 2021-03-08 12:28 ` Juergen Gross via Virtualization 2021-03-08 18:30 ` Borislav Petkov 2021-03-08 18:30 ` Borislav Petkov 2021-03-09 6:21 ` Jürgen Groß 2021-03-09 6:21 ` Jürgen Groß via Virtualization 2021-03-08 12:28 ` [PATCH v5 12/12] x86/paravirt: have only one paravirt patch function Juergen Gross 2021-03-08 12:28 ` Juergen Gross via Virtualization 2021-03-08 16:31 ` [PATCH v5 00/12] x86: major paravirt cleanup Peter Zijlstra 2021-03-08 16:31 ` Peter Zijlstra 2021-03-08 16:31 ` Peter Zijlstra
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210308122844.30488-7-jgross@suse.com \ --to=jgross@suse.com \ --cc=bp@alien8.de \ --cc=hpa@zytor.com \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@redhat.com \ --cc=pv-drivers@vmware.com \ --cc=sdeep@vmware.com \ --cc=tglx@linutronix.de \ --cc=virtualization@lists.linux-foundation.org \ --cc=x86@kernel.org \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.