From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F30E3C433E0 for ; Tue, 21 Jul 2020 11:10:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CBAB620709 for ; Tue, 21 Jul 2020 11:10:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="c4ItvpAD"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="nRUVF8Bt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729837AbgGULKF (ORCPT ); Tue, 21 Jul 2020 07:10:05 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:37356 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726415AbgGULIm (ORCPT ); Tue, 21 Jul 2020 07:08:42 -0400 Message-Id: <20200721110808.780082657@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1595329720; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: references:references; bh=kg3tYFw5AM7077WcbqjxPP5BJjGJU17MR3Uz7axOyXw=; b=c4ItvpAD+jEAYDMaM8CvqGjpJ1kou8wkrO27IgPLluOEIgovJSyG3dnU/nguKm9fO1b74i 76mQrxgHi21/SsHwsSBKD06bypEgprDn6xBaF7nbidbbw8H95PT6WLPA7YD0B64Ynhmc6n wfBvbhefw+2QsPuVbWAwoqVGKCRk3G7Urnl7toyk9FbqQB/7dag+vacFOYKfmEBPAyw8SL 8DXwkzaVbd56CCueM/+EmDfUs3nlcqTOvB8SjNtWCpPUEHYpnBkZp1BDEu8glvLVvwDQFA BcvgXDfs6Q4DXOmZQyt6Fi7QLxoG3Q5m6p4L7n5l7dLBx+QztyNLc/poClOI9g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1595329720; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: references:references; bh=kg3tYFw5AM7077WcbqjxPP5BJjGJU17MR3Uz7axOyXw=; b=nRUVF8BtNSGJLxY3HfxEYfqnSovH+ZlRJHtiSBqiUckM7+qxmvdGCaYJYabrjan0pEOHA4 9niox1G8UtV4CSAQ== Date: Tue, 21 Jul 2020 12:57:11 +0200 From: Thomas Gleixner To: LKML Cc: x86@kernel.org, linux-arch@vger.kernel.org, Will Deacon , Arnd Bergmann , Mark Rutland , Kees Cook , Keno Fischer , Paolo Bonzini , kvm@vger.kernel.org, Gabriel Krisman Bertazi Subject: [patch V4 05/15] entry: Provide infrastructure for work before exiting to guest mode References: <20200721105706.030914876@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-transfer-encoding: 8-bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Thomas Gleixner Entering a guest is similar to exiting to user space. Pending work like handling signals, rescheduling, task work etc. needs to be handled before that. Provide generic infrastructure to avoid duplication of the same handling code all over the place. The exit to guest mode handling is different from the exit to usermode handling, e.g. vs. rseq and live patching, so a separate function is used. The initial list of work items handled is: TIF_SIGPENDING, TIF_NEED_RESCHED, TIF_NOTIFY_RESUME Architecture specific TIF flags can be added via defines in the architecture specific include files. The calling convention is also different from the syscall/interrupt entry functions as KVM invokes this from the outer vcpu_run() loop with interrupts and preemption disabled. To prevent missing a pending work item it invokes a check for pending TIF work from interrupt disabled code right before exiting to guest mode. The lockdep, RCU and tracing state handling is also done directly around the switch to and from guest mode. Signed-off-by: Thomas Gleixner --- V3: Reworked and simplified version adopted to recent X86 and KVM changes V2: Moved KVM specific functions to kvm (Paolo) Added lockdep assert (Andy) Dropped live patching from enter guest mode work (Miroslav) --- include/linux/entry-kvm.h | 80 ++++++++++++++++++++++++++++++++++++++++++++++ include/linux/kvm_host.h | 8 ++++ kernel/entry/Makefile | 3 + kernel/entry/kvm.c | 51 +++++++++++++++++++++++++++++ virt/kvm/Kconfig | 3 + 5 files changed, 144 insertions(+), 1 deletion(-) --- /dev/null +++ b/include/linux/entry-kvm.h @@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __LINUX_ENTRYKVM_H +#define __LINUX_ENTRYKVM_H + +#include + +/* Exit to guest mode work */ +#ifdef CONFIG_KVM_EXIT_TO_GUEST_WORK + +#ifndef ARCH_EXIT_TO_GUEST_MODE_WORK +# define ARCH_EXIT_TO_GUEST_MODE_WORK (0) +#endif + +#define EXIT_TO_GUEST_MODE_WORK \ + (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ + _TIF_NOTIFY_RESUME | ARCH_EXIT_TO_GUEST_MODE_WORK) + +struct kvm_vcpu; + +/** + * arch_exit_to_guest_mode_work - Architecture specific exit to guest mode + * work function. + * @vcpu: Pointer to current's VCPU data + * @ti_work: Cached TIF flags gathered in exit_to_guest_mode() + * + * Invoked from exit_to_guest_mode_work(). Defaults to NOOP. Can be + * replaced by architecture specific code. + */ +static inline int arch_exit_to_guest_mode_work(struct kvm_vcpu *vcpu, + unsigned long ti_work); + +#ifndef arch_exit_to_guest_mode_work +static inline int arch_exit_to_guest_mode_work(struct kvm_vcpu *vcpu, + unsigned long ti_work) +{ + return 0; +} +#endif + +/** + * exit_to_guest_mode - Check and handle pending work which needs to be + * handled before returning to guest mode + * @vcpu: Pointer to current's VCPU data + * + * Returns: 0 or an error code + */ +int exit_to_guest_mode(struct kvm_vcpu *vcpu); + +/** + * __exit_to_guest_mode_work_pending - Check if work is pending + * + * Returns: True if work pending, False otherwise. + * + * Bare variant of exit_to_guest_mode_work_pending(). Can be called from + * interrupt enabled code for racy quick checks with care. + */ +static inline bool __exit_to_guest_mode_work_pending(void) +{ + unsigned long ti_work = READ_ONCE(current_thread_info()->flags); + + return !!(ti_work & EXIT_TO_GUEST_MODE_WORK); +} + +/** + * exit_to_guest_mode_work_pending - Check if work is pending which needs to be + * handled before returning to guest mode + * + * Returns: True if work pending, False otherwise. + * + * Has to be invoked with interrupts disabled before the transition to + * guest mode. + */ +static inline bool exit_to_guest_mode_work_pending(void) +{ + lockdep_assert_irqs_disabled(); + return __exit_to_guest_mode_work_pending(); +} +#endif /* CONFIG_KVM_EXIT_TO_GUEST_WORK */ + +#endif --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1439,4 +1439,12 @@ int kvm_vm_create_worker_thread(struct k uintptr_t data, const char *name, struct task_struct **thread_ptr); +#ifdef CONFIG_KVM_EXIT_TO_GUEST_WORK +static inline void kvm_handle_signal_exit(struct kvm_vcpu *vcpu) +{ + vcpu->run->exit_reason = KVM_EXIT_INTR; + vcpu->stat.signal_exits++; +} +#endif /* CONFIG_KVM_EXIT_TO_GUEST_WORK */ + #endif --- a/kernel/entry/Makefile +++ b/kernel/entry/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_GENERIC_ENTRY) += common.o +obj-$(CONFIG_GENERIC_ENTRY) += common.o +obj-$(CONFIG_KVM_EXIT_TO_GUEST_WORK) += kvm.o --- /dev/null +++ b/kernel/entry/kvm.c @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +static int exit_to_guest_mode_work(struct kvm_vcpu *vcpu, unsigned long ti_work) +{ + do { + int ret; + + if (ti_work & _TIF_SIGPENDING) { + kvm_handle_signal_exit(vcpu); + return -EINTR; + } + + if (ti_work & _TIF_NEED_RESCHED) + schedule(); + + if (ti_work & _TIF_NOTIFY_RESUME) { + clear_thread_flag(TIF_NOTIFY_RESUME); + tracehook_notify_resume(NULL); + } + + ret = arch_exit_to_guest_mode_work(vcpu, ti_work); + if (ret) + return ret; + + ti_work = READ_ONCE(current_thread_info()->flags); + } while (ti_work & EXIT_TO_GUEST_MODE_WORK || need_resched()); + return 0; +} + +int exit_to_guest_mode(struct kvm_vcpu *vcpu) +{ + unsigned long ti_work; + + /* + * This is invoked from the outer guest loop with interrupts and + * preemption enabled. + * + * KVM invokes exit_to_guest_mode_work_pending() with interrupts + * disabled in the inner loop before going into guest mode. No need + * to disable interrupts here. + */ + ti_work = READ_ONCE(current_thread_info()->flags); + if (!(ti_work & EXIT_TO_GUEST_MODE_WORK)) + return 0; + + return exit_to_guest_mode_work(vcpu, ti_work); +} +EXPORT_SYMBOL_GPL(exit_to_guest_mode); --- a/virt/kvm/Kconfig +++ b/virt/kvm/Kconfig @@ -60,3 +60,6 @@ config HAVE_KVM_VCPU_RUN_PID_CHANGE config HAVE_KVM_NO_POLL bool + +config KVM_EXIT_TO_GUEST_WORK + bool