From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753509AbdLMPLF (ORCPT ); Wed, 13 Dec 2017 10:11:05 -0500 Received: from mx1.redhat.com ([209.132.183.28]:44858 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752550AbdLMPKn (ORCPT ); Wed, 13 Dec 2017 10:10:43 -0500 From: Vitaly Kuznetsov To: kvm@vger.kernel.org, x86@kernel.org Cc: Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , "Michael Kelley (EOSG)" , Andy Lutomirski , Mohammed Gamal , Cathy Avery , Roman Kagan , linux-kernel@vger.kernel.org, devel@linuxdriverproject.org Subject: [PATCH v2 4/7] x86/hyper-v: redirect reenlightment notifications on CPU offlining Date: Wed, 13 Dec 2017 16:09:42 +0100 Message-Id: <20171213150945.24054-5-vkuznets@redhat.com> In-Reply-To: <20171213150945.24054-1-vkuznets@redhat.com> References: <20171213150945.24054-1-vkuznets@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 13 Dec 2017 15:10:43 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org It is very unlikely for CPUs to get offlined when we run on Hyper-V as we have a protection in vmbus module which prevents it when we have any VMBus devices assigned. This, however, may change in future if an option to reassign an already active channel will be added. It is also possible to run without any Hyper-V devices of have a CPU with no assigned channels. Reassign reenlightenment notifications to some other active CPU when the CPU which is assigned to get them goes offline. Signed-off-by: Vitaly Kuznetsov --- arch/x86/hyperv/hv_init.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 347be4de3edd..38118ca38d97 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -191,6 +191,36 @@ void clear_hv_tscchange_cb(void) } EXPORT_SYMBOL_GPL(clear_hv_tscchange_cb); +static int hv_cpu_die(unsigned int cpu) +{ + struct hv_reenlightenment_control re_ctrl; + int i; + static DEFINE_SPINLOCK(lock); + + if (hv_reenlightenment_cb == NULL) + return 0; + + /* Make sure the CPU we migrate to is not going away too */ + spin_lock(&lock); + rdmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, *((u64 *)&re_ctrl)); + if (re_ctrl.target_vp == hv_vp_index[cpu]) { + /* Find some other online CPU */ + for_each_online_cpu(i) { + if (i == cpu) + continue; + + re_ctrl.target_vp = hv_vp_index[i]; + wrmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, + *((u64 *)&re_ctrl)); + + break; + } + } + spin_unlock(&lock); + + return 0; +} + /* * This function is to be invoked early in the boot sequence after the * hypervisor has been detected. @@ -220,7 +250,7 @@ void hyperv_init(void) return; if (cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/hyperv_init:online", - hv_cpu_init, NULL) < 0) + hv_cpu_init, hv_cpu_die) < 0) goto free_vp_index; /* -- 2.14.3 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vitaly Kuznetsov Subject: [PATCH v2 4/7] x86/hyper-v: redirect reenlightment notifications on CPU offlining Date: Wed, 13 Dec 2017 16:09:42 +0100 Message-ID: <20171213150945.24054-5-vkuznets@redhat.com> References: <20171213150945.24054-1-vkuznets@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: Stephen Hemminger , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Haiyang Zhang , linux-kernel@vger.kernel.org, devel@linuxdriverproject.org, "Michael Kelley \(EOSG\)" , Ingo Molnar , Roman Kagan , Andy Lutomirski , "H. Peter Anvin" , Paolo Bonzini , Thomas Gleixner , Mohammed Gamal To: kvm@vger.kernel.org, x86@kernel.org Return-path: In-Reply-To: <20171213150945.24054-1-vkuznets@redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: driverdev-devel-bounces@linuxdriverproject.org Sender: "devel" List-Id: kvm.vger.kernel.org It is very unlikely for CPUs to get offlined when we run on Hyper-V as we have a protection in vmbus module which prevents it when we have any VMBus devices assigned. This, however, may change in future if an option to reassign an already active channel will be added. It is also possible to run without any Hyper-V devices of have a CPU with no assigned channels. Reassign reenlightenment notifications to some other active CPU when the CPU which is assigned to get them goes offline. Signed-off-by: Vitaly Kuznetsov --- arch/x86/hyperv/hv_init.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 347be4de3edd..38118ca38d97 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -191,6 +191,36 @@ void clear_hv_tscchange_cb(void) } EXPORT_SYMBOL_GPL(clear_hv_tscchange_cb); +static int hv_cpu_die(unsigned int cpu) +{ + struct hv_reenlightenment_control re_ctrl; + int i; + static DEFINE_SPINLOCK(lock); + + if (hv_reenlightenment_cb == NULL) + return 0; + + /* Make sure the CPU we migrate to is not going away too */ + spin_lock(&lock); + rdmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, *((u64 *)&re_ctrl)); + if (re_ctrl.target_vp == hv_vp_index[cpu]) { + /* Find some other online CPU */ + for_each_online_cpu(i) { + if (i == cpu) + continue; + + re_ctrl.target_vp = hv_vp_index[i]; + wrmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, + *((u64 *)&re_ctrl)); + + break; + } + } + spin_unlock(&lock); + + return 0; +} + /* * This function is to be invoked early in the boot sequence after the * hypervisor has been detected. @@ -220,7 +250,7 @@ void hyperv_init(void) return; if (cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/hyperv_init:online", - hv_cpu_init, NULL) < 0) + hv_cpu_init, hv_cpu_die) < 0) goto free_vp_index; /* -- 2.14.3