From: Babu Moger <babu.moger@amd.com>
To: Borislav Petkov <bp@alien8.de>, Hugh Dickins <hughd@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
Jim Mattson <jmattson@google.com>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Wanpeng Li <wanpengli@tencent.com>,
kvm list <kvm@vger.kernel.org>, Joerg Roedel <joro@8bytes.org>,
the arch/x86 maintainers <x86@kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Ingo Molnar <mingo@redhat.com>, "H . Peter Anvin" <hpa@zytor.com>,
Thomas Gleixner <tglx@linutronix.de>,
Makarand Sonare <makarandsonare@google.com>,
Sean Christopherson <seanjc@google.com>
Subject: Re: [PATCH] x86/tlb: Flush global mappings when KAISER is disabled
Date: Thu, 25 Mar 2021 10:13:10 -0500 [thread overview]
Message-ID: <c853578c-1a47-6fb0-8bf8-9c9f5c991e30@amd.com> (raw)
In-Reply-To: <20210325102959.GD31322@zn.tnic>
On 3/25/21 5:29 AM, Borislav Petkov wrote:
> Ok,
>
> I tried to be as specific as possible in the commit message so that we
> don't forget. Please lemme know if I've missed something.
>
> Babu, Jim, I'd appreciate it if you ran this to confirm.
>
> Thx.
>
> ---
> From: Borislav Petkov <bp@suse.de>
> Date: Thu, 25 Mar 2021 11:02:31 +0100
>
> Jim Mattson reported that Debian 9 guests using a 4.9-stable kernel
> are exploding during alternatives patching:
>
> kernel BUG at /build/linux-dqnRSc/linux-4.9.228/arch/x86/kernel/alternative.c:709!
> invalid opcode: 0000 [#1] SMP
> Modules linked in:
> CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.9.0-13-amd64 #1 Debian 4.9.228-1
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Call Trace:
> swap_entry_free
> swap_entry_free
> text_poke_bp
> swap_entry_free
> arch_jump_label_transform
> set_debug_rodata
> __jump_label_update
> static_key_slow_inc
> frontswap_register_ops
> init_zswap
> init_frontswap
> do_one_initcall
> set_debug_rodata
> kernel_init_freeable
> rest_init
> kernel_init
> ret_from_fork
>
> triggering the BUG_ON in text_poke() which verifies whether patched
> instruction bytes have actually landed at the destination.
>
> Further debugging showed that the TLB flush before that check is
> insufficient because there could be global mappings left in the TLB,
> leading to a stale mapping getting used.
>
> I say "global mappings" because the hardware configuration is a new one:
> machine is an AMD, which means, KAISER/PTI doesn't need to be enabled
> there, which also means there's no user/kernel pagetables split and
> therefore the TLB can have global mappings.
>
> And the configuration is new one for a second reason: because that AMD
> machine supports PCID and INVPCID, which leads the CPU detection code to
> set the synthetic X86_FEATURE_INVPCID_SINGLE flag.
>
> Now, __native_flush_tlb_single() does invalidate global mappings when
> X86_FEATURE_INVPCID_SINGLE is *not* set and returns.
>
> When X86_FEATURE_INVPCID_SINGLE is set, however, it invalidates the
> requested address from both PCIDs in the KAISER-enabled case. But if
> KAISER is not enabled and the machine has global mappings in the TLB,
> then those global mappings do not get invalidated, which would lead to
> the above mismatch from using a stale TLB entry.
>
> So make sure to flush those global mappings in the KAISER disabled case.
>
> Co-debugged by Babu Moger <babu.moger@amd.com>.
>
> Reported-by: Jim Mattson <jmattson@google.com>
> Signed-off-by: Borislav Petkov <bp@suse.de>
> Link: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flkml.kernel.org%2Fr%2FCALMp9eRDSW66%252BXvbHVF4ohL7XhThoPoT0BrB0TcS0cgk%3DdkcBg%40mail.gmail.com&data=04%7C01%7Cbabu.moger%40amd.com%7Cf4e0aacf81744dc8be4408d8ef78f2cf%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637522650066097649%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1c4MQ9I9KrLxWLqghGCI%2BC%2Bvs0c9vYaNC5d%2FiYL0oMA%3D&reserved=0
> ---
> arch/x86/include/asm/tlbflush.h | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
> index f5ca15622dc9..2bfa4deb8cae 100644
> --- a/arch/x86/include/asm/tlbflush.h
> +++ b/arch/x86/include/asm/tlbflush.h
> @@ -245,12 +245,15 @@ static inline void __native_flush_tlb_single(unsigned long addr)
> * ASID. But, userspace flushes are probably much more
> * important performance-wise.
> *
> - * Make sure to do only a single invpcid when KAISER is
> - * disabled and we have only a single ASID.
> + * In the KAISER disabled case, do an INVLPG to make sure
> + * the mapping is flushed in case it is a global one.
> */
> - if (kaiser_enabled)
> + if (kaiser_enabled) {
> invpcid_flush_one(X86_CR3_PCID_ASID_USER, addr);
> - invpcid_flush_one(X86_CR3_PCID_ASID_KERN, addr);
> + invpcid_flush_one(X86_CR3_PCID_ASID_KERN, addr);
> + } else {
> + asm volatile("invlpg (%0)" ::"r" (addr) : "memory");
> + }
> }
>
> static inline void __flush_tlb_all(void)
>
Thanks Boris. As you updated the patch little bit since yesterday, I
retested them again both on host and guest kernel. They are looking good.
Tested-by: Babu Moger <babu.moger@amd.com>
next prev parent reply other threads:[~2021-03-25 15:14 UTC|newest]
Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-11 19:27 [PATCH v6 00/12] SVM cleanup and INVPCID feature support Babu Moger
2020-09-11 19:27 ` [PATCH v6 01/12] KVM: SVM: Introduce vmcb_(set_intercept/clr_intercept/_is_intercept) Babu Moger
2020-09-11 19:28 ` [PATCH v6 02/12] KVM: SVM: Change intercept_cr to generic intercepts Babu Moger
2020-09-11 19:28 ` [PATCH v6 03/12] KVM: SVM: Change intercept_dr " Babu Moger
2020-09-11 19:28 ` [PATCH v6 04/12] KVM: SVM: Modify intercept_exceptions " Babu Moger
2020-09-12 16:52 ` Paolo Bonzini
2020-09-14 15:06 ` Sean Christopherson
2020-09-22 13:39 ` Paolo Bonzini
2020-09-22 19:11 ` Babu Moger
2020-09-23 2:43 ` Paolo Bonzini
2020-09-23 13:35 ` Babu Moger
2020-09-11 19:28 ` [PATCH v6 05/12] KVM: SVM: Modify 64 bit intercept field to two 32 bit vectors Babu Moger
2020-09-11 19:28 ` [PATCH v6 06/12] KVM: SVM: Add new intercept vector in vmcb_control_area Babu Moger
2020-09-11 19:28 ` [PATCH v6 07/12] KVM: nSVM: Cleanup nested_state data structure Babu Moger
2020-09-11 19:28 ` [PATCH v6 08/12] KVM: SVM: Remove set_cr_intercept, clr_cr_intercept and is_cr_intercept Babu Moger
2020-09-11 19:28 ` [PATCH v6 09/12] KVM: SVM: Remove set_exception_intercept and clr_exception_intercept Babu Moger
2020-09-11 19:29 ` [PATCH v6 10/12] KVM: X86: Rename and move the function vmx_handle_memory_failure to x86.c Babu Moger
2020-09-11 19:29 ` [PATCH v6 11/12] KVM: X86: Move handling of INVPCID types to x86 Babu Moger
2020-09-11 19:29 ` [PATCH v6 12/12] KVM:SVM: Enable INVPCID feature on AMD Babu Moger
2020-09-12 17:08 ` [PATCH v6 00/12] SVM cleanup and INVPCID feature support Paolo Bonzini
2020-09-14 15:05 ` Sean Christopherson
2020-09-14 18:33 ` Babu Moger
2021-01-19 23:01 ` Jim Mattson
2021-01-19 23:45 ` Babu Moger
2021-01-20 21:14 ` Jim Mattson
2021-01-20 21:45 ` Babu Moger
2021-01-21 3:10 ` Babu Moger
2021-01-21 23:51 ` Babu Moger
2021-01-23 1:52 ` Babu Moger
2021-02-24 0:13 ` Jim Mattson
2021-02-24 22:17 ` Babu Moger
2021-03-10 1:04 ` Babu Moger
2021-03-10 9:08 ` Paolo Bonzini
2021-03-10 14:55 ` Babu Moger
2021-03-10 14:58 ` Babu Moger
2021-03-10 15:31 ` Paolo Bonzini
2021-03-11 1:21 ` Babu Moger
2021-03-11 20:07 ` Borislav Petkov
2021-03-11 20:32 ` Borislav Petkov
2021-03-11 20:57 ` Babu Moger
2021-03-11 21:40 ` Borislav Petkov
2021-03-11 22:04 ` Babu Moger
2021-03-11 22:15 ` Babu Moger
2021-03-11 23:52 ` Borislav Petkov
2021-03-12 14:53 ` Babu Moger
2021-03-12 16:12 ` Babu Moger
2021-03-24 21:21 ` Borislav Petkov
2021-03-24 21:59 ` Paolo Bonzini
2021-03-25 0:05 ` Hugh Dickins
2021-03-25 2:43 ` Hugh Dickins
2021-03-25 9:56 ` Borislav Petkov
2021-03-25 10:29 ` [PATCH] x86/tlb: Flush global mappings when KAISER is disabled Borislav Petkov
2021-03-25 10:52 ` Paolo Bonzini
2021-03-25 15:13 ` Babu Moger [this message]
2021-03-25 16:33 ` Hugh Dickins
2021-03-25 19:00 ` Jim Mattson
2021-03-25 20:09 ` Borislav Petkov
2021-03-25 20:36 ` Sasha Levin
2021-03-25 23:19 ` Sasha Levin
2021-03-25 23:56 ` Ben Hutchings
2021-03-11 21:23 ` [PATCH v6 00/12] SVM cleanup and INVPCID feature support Jim Mattson
2021-03-11 21:36 ` Borislav Petkov
2021-03-11 21:50 ` Babu Moger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c853578c-1a47-6fb0-8bf8-9c9f5c991e30@amd.com \
--to=babu.moger@amd.com \
--cc=bp@alien8.de \
--cc=hpa@zytor.com \
--cc=hughd@google.com \
--cc=jmattson@google.com \
--cc=joro@8bytes.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=makarandsonare@google.com \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=vkuznets@redhat.com \
--cc=wanpengli@tencent.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).