From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D041C4727E for ; Fri, 25 Sep 2020 17:25:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B7BA2074B for ; Fri, 25 Sep 2020 17:25:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729535AbgIYRZm (ORCPT ); Fri, 25 Sep 2020 13:25:42 -0400 Received: from mga12.intel.com ([192.55.52.136]:56940 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726401AbgIYRZl (ORCPT ); Fri, 25 Sep 2020 13:25:41 -0400 IronPort-SDR: l1s3r0WbxLYtXok1tXsAkk5l77ri6PGiu6v87JN3ELxhOm0gEePkb7rVTvwWnBIaOEREeM4NVS SeF+sGjrFlkA== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="141004363" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="141004363" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 10:12:35 -0700 IronPort-SDR: cpapzdKeorbGJAJcicHTqvjLBQC0HcUDOXmfyns5mrzobWjC3ddmjR9HRB2DUcOw18nqoDh4KK uLzfmR0kQukQ== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="336814163" Received: from sjchrist-coffee.jf.intel.com (HELO linux.intel.com) ([10.54.74.160]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 10:12:35 -0700 Date: Fri, 25 Sep 2020 10:12:33 -0700 From: Sean Christopherson To: Vitaly Kuznetsov Cc: Paolo Bonzini , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Marc Zyngier , James Morse , Julien Thierry , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Huacai Chen , Aleksandar Markovic , linux-mips@vger.kernel.org, Paul Mackerras , kvm-ppc@vger.kernel.org, Christian Borntraeger , Janosch Frank , David Hildenbrand , Cornelia Huck , Claudio Imbrenda Subject: Re: [RFC PATCH 3/3] KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the VM Message-ID: <20200925171233.GC31528@linux.intel.com> References: <20200923224530.17735-1-sean.j.christopherson@intel.com> <20200923224530.17735-4-sean.j.christopherson@intel.com> <878scze4l5.fsf@vitty.brq.redhat.com> <20200924181134.GB9649@linux.intel.com> <87k0wichht.fsf@vitty.brq.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87k0wichht.fsf@vitty.brq.redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Sep 25, 2020 at 11:50:38AM +0200, Vitaly Kuznetsov wrote: > Sean Christopherson writes: > > > On Thu, Sep 24, 2020 at 02:34:14PM +0200, Vitaly Kuznetsov wrote: > >> Sean Christopherson writes: > >> > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > >> > index 6f9a0c6d5dc5..810d46ab0a47 100644 > >> > --- a/arch/x86/kvm/vmx/vmx.c > >> > +++ b/arch/x86/kvm/vmx/vmx.c > >> > @@ -4985,14 +4986,13 @@ static int handle_cr(struct kvm_vcpu *vcpu) > >> > } > >> > break; > >> > case 2: /* clts */ > >> > - WARN_ONCE(1, "Guest should always own CR0.TS"); > >> > - vmx_set_cr0(vcpu, kvm_read_cr0_bits(vcpu, ~X86_CR0_TS)); > >> > - trace_kvm_cr_write(0, kvm_read_cr0(vcpu)); > >> > - return kvm_skip_emulated_instruction(vcpu); > >> > + KVM_BUG(1, vcpu->kvm, "Guest always owns CR0.TS"); > >> > + return -EIO; > >> > case 1: /*mov from cr*/ > >> > switch (cr) { > >> > case 3: > >> > WARN_ON_ONCE(enable_unrestricted_guest); > >> > + > >> > >> Here, were you intended to replace WARN_ON_ONCE() with KVM_BUG_ON() or > >> this is just a stray newline added? > > > > I think it's just a stray newline. At one point I had converted this to a > > KVM_BUG_ON(), but then reversed direction because it's not fatal to the guest, > > i.e. KVM should continue to function even though it's spuriously intercepting > > CR3 loads. > > > > Which, rereading this patch, completely contradicts the KVM_BUG() for CLTS. > > > > That's probably something we should sort out in this RFC: is KVM_BUG() only > > to be used if the bug is fatal/dangerous, or should it be used any time the > > error is definitely a KVM (or hardware) bug. > > Personally, I'm feeling adventurous so my vote goes to the later :-) > Whenever a KVM bug was discovered by a VM it's much safer to stop > executing it as who knows what the implications might be? Not necessarily, e.g. terminating the VM may corrupt the VM's file system, which is less safe, for lack of a better word, from the VM's perspective. > In this particular case I can think of a nested scenario when L1 didn't > ask for CR3 intercept but L0 is still injecting it. It is not fatal by > itself but probably there is bug in calculating intercepts in L0 so > if we're getting something extra maybe we're also missing some? And this > doesn't sound good at all. Hmm, but by that argument this scenario would fall into the "dangerous" part of "bug is fatal/dangerous". I guess my opinion is that we should set a fairly high bar for using KVM_BUG() so that KVM can be aggressive in shutting down. > > In theory, it should be impossible to reach this again as "r = -EIO" will > > bounce this out to userspace, the common checks to deny all ioctls() will > > prevent reinvoking KVM_RUN. > > Do we actually want to prevent *all* ioctls? E.g. when 'vm bugged' > condition is triggered userspace may want to extract some information to > assist debugging but even things like KVM_GET_[S]REGS will just return > -EIO. I'm not sure it is generally safe to enable *everything* (except > for KVM_RUN which should definitely be forbidden) so maybe your approach > is preferable. The answer to this probably depends on the answer to the first question of when it's appropriate to use KVM_BUG(). E.g. if we limit usage to fatal or dangrous cases, then blocking all ioctls() is probably the right thing do do. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3507C4363D for ; Fri, 25 Sep 2020 17:14:21 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73C6920936 for ; Fri, 25 Sep 2020 17:14:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="2mGweO9Q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73C6920936 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=eop6FUPXFT4w+u+hOAgq7fXys5dSc3JDEe4vY507sz4=; b=2mGweO9QFT/e8la+f20ElFgyk aDpqgd3hGyUuTC99rZHNmDBx2yWqcmr6kE3KErK0+T6HRRT/LOa6UiWwlsmpg8d2btr4zVVH4yyLD Uo2Venxy2iTCpCWutBOPRJD0gBpqFONm2iuhS1RZDEl0EmE4asDF6Kb+gQ14cdqXEmH1GV/VqL7pd Cmh/3XZqp3lflC7mPx3HFkQYifacAYNjL114aHfMjA8gxS78DLXZpPKKpPxWvnZdBU5Cf4Vl4o8UO k1wUn2y8th+96Kpc0prfMRnjqVSCj9rr1B/wRXiTOdMhkmXV+kfpMJI1zb+L0cohCqM/zSXI8RszQ oa8iXORkg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kLrGy-0007Zc-Sp; Fri, 25 Sep 2020 17:12:40 +0000 Received: from mga02.intel.com ([134.134.136.20]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kLrGw-0007ZE-3G for linux-arm-kernel@lists.infradead.org; Fri, 25 Sep 2020 17:12:39 +0000 IronPort-SDR: IFXUvjPsmP16TzPioZ8KJkWlzlt6mdSjLotczgz31YotleDplQz0gkwzP5hkjk6CvabHmKW0KX rQgK2NF7CoAg== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="149238259" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="149238259" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 10:12:35 -0700 IronPort-SDR: cpapzdKeorbGJAJcicHTqvjLBQC0HcUDOXmfyns5mrzobWjC3ddmjR9HRB2DUcOw18nqoDh4KK uLzfmR0kQukQ== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="336814163" Received: from sjchrist-coffee.jf.intel.com (HELO linux.intel.com) ([10.54.74.160]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 10:12:35 -0700 Date: Fri, 25 Sep 2020 10:12:33 -0700 From: Sean Christopherson To: Vitaly Kuznetsov Subject: Re: [RFC PATCH 3/3] KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the VM Message-ID: <20200925171233.GC31528@linux.intel.com> References: <20200923224530.17735-1-sean.j.christopherson@intel.com> <20200923224530.17735-4-sean.j.christopherson@intel.com> <878scze4l5.fsf@vitty.brq.redhat.com> <20200924181134.GB9649@linux.intel.com> <87k0wichht.fsf@vitty.brq.redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87k0wichht.fsf@vitty.brq.redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200925_131238_268351_DC532B21 X-CRM114-Status: GOOD ( 34.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Cornelia Huck , Wanpeng Li , Janosch Frank , kvm@vger.kernel.org, Suzuki K Poulose , Marc Zyngier , Joerg Roedel , David Hildenbrand , linux-kernel@vger.kernel.org, kvm-ppc@vger.kernel.org, linux-mips@vger.kernel.org, Paul Mackerras , Christian Borntraeger , Aleksandar Markovic , James Morse , linux-arm-kernel@lists.infradead.org, Huacai Chen , Paolo Bonzini , Claudio Imbrenda , Julien Thierry , Jim Mattson Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Sep 25, 2020 at 11:50:38AM +0200, Vitaly Kuznetsov wrote: > Sean Christopherson writes: > > > On Thu, Sep 24, 2020 at 02:34:14PM +0200, Vitaly Kuznetsov wrote: > >> Sean Christopherson writes: > >> > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > >> > index 6f9a0c6d5dc5..810d46ab0a47 100644 > >> > --- a/arch/x86/kvm/vmx/vmx.c > >> > +++ b/arch/x86/kvm/vmx/vmx.c > >> > @@ -4985,14 +4986,13 @@ static int handle_cr(struct kvm_vcpu *vcpu) > >> > } > >> > break; > >> > case 2: /* clts */ > >> > - WARN_ONCE(1, "Guest should always own CR0.TS"); > >> > - vmx_set_cr0(vcpu, kvm_read_cr0_bits(vcpu, ~X86_CR0_TS)); > >> > - trace_kvm_cr_write(0, kvm_read_cr0(vcpu)); > >> > - return kvm_skip_emulated_instruction(vcpu); > >> > + KVM_BUG(1, vcpu->kvm, "Guest always owns CR0.TS"); > >> > + return -EIO; > >> > case 1: /*mov from cr*/ > >> > switch (cr) { > >> > case 3: > >> > WARN_ON_ONCE(enable_unrestricted_guest); > >> > + > >> > >> Here, were you intended to replace WARN_ON_ONCE() with KVM_BUG_ON() or > >> this is just a stray newline added? > > > > I think it's just a stray newline. At one point I had converted this to a > > KVM_BUG_ON(), but then reversed direction because it's not fatal to the guest, > > i.e. KVM should continue to function even though it's spuriously intercepting > > CR3 loads. > > > > Which, rereading this patch, completely contradicts the KVM_BUG() for CLTS. > > > > That's probably something we should sort out in this RFC: is KVM_BUG() only > > to be used if the bug is fatal/dangerous, or should it be used any time the > > error is definitely a KVM (or hardware) bug. > > Personally, I'm feeling adventurous so my vote goes to the later :-) > Whenever a KVM bug was discovered by a VM it's much safer to stop > executing it as who knows what the implications might be? Not necessarily, e.g. terminating the VM may corrupt the VM's file system, which is less safe, for lack of a better word, from the VM's perspective. > In this particular case I can think of a nested scenario when L1 didn't > ask for CR3 intercept but L0 is still injecting it. It is not fatal by > itself but probably there is bug in calculating intercepts in L0 so > if we're getting something extra maybe we're also missing some? And this > doesn't sound good at all. Hmm, but by that argument this scenario would fall into the "dangerous" part of "bug is fatal/dangerous". I guess my opinion is that we should set a fairly high bar for using KVM_BUG() so that KVM can be aggressive in shutting down. > > In theory, it should be impossible to reach this again as "r = -EIO" will > > bounce this out to userspace, the common checks to deny all ioctls() will > > prevent reinvoking KVM_RUN. > > Do we actually want to prevent *all* ioctls? E.g. when 'vm bugged' > condition is triggered userspace may want to extract some information to > assist debugging but even things like KVM_GET_[S]REGS will just return > -EIO. I'm not sure it is generally safe to enable *everything* (except > for KVM_RUN which should definitely be forbidden) so maybe your approach > is preferable. The answer to this probably depends on the answer to the first question of when it's appropriate to use KVM_BUG(). E.g. if we limit usage to fatal or dangrous cases, then blocking all ioctls() is probably the right thing do do. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sean Christopherson Date: Fri, 25 Sep 2020 17:12:33 +0000 Subject: Re: [RFC PATCH 3/3] KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the VM Message-Id: <20200925171233.GC31528@linux.intel.com> List-Id: References: <20200923224530.17735-1-sean.j.christopherson@intel.com> <20200923224530.17735-4-sean.j.christopherson@intel.com> <878scze4l5.fsf@vitty.brq.redhat.com> <20200924181134.GB9649@linux.intel.com> <87k0wichht.fsf@vitty.brq.redhat.com> In-Reply-To: <87k0wichht.fsf@vitty.brq.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Vitaly Kuznetsov Cc: Paolo Bonzini , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Marc Zyngier , James Morse , Julien Thierry , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Huacai Chen , Aleksandar Markovic , linux-mips@vger.kernel.org, Paul Mackerras , kvm-ppc@vger.kernel.org, Christian Borntraeger , Janosch Frank , David Hildenbrand , Cornelia Huck , Claudio Imbrenda On Fri, Sep 25, 2020 at 11:50:38AM +0200, Vitaly Kuznetsov wrote: > Sean Christopherson writes: > > > On Thu, Sep 24, 2020 at 02:34:14PM +0200, Vitaly Kuznetsov wrote: > >> Sean Christopherson writes: > >> > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > >> > index 6f9a0c6d5dc5..810d46ab0a47 100644 > >> > --- a/arch/x86/kvm/vmx/vmx.c > >> > +++ b/arch/x86/kvm/vmx/vmx.c > >> > @@ -4985,14 +4986,13 @@ static int handle_cr(struct kvm_vcpu *vcpu) > >> > } > >> > break; > >> > case 2: /* clts */ > >> > - WARN_ONCE(1, "Guest should always own CR0.TS"); > >> > - vmx_set_cr0(vcpu, kvm_read_cr0_bits(vcpu, ~X86_CR0_TS)); > >> > - trace_kvm_cr_write(0, kvm_read_cr0(vcpu)); > >> > - return kvm_skip_emulated_instruction(vcpu); > >> > + KVM_BUG(1, vcpu->kvm, "Guest always owns CR0.TS"); > >> > + return -EIO; > >> > case 1: /*mov from cr*/ > >> > switch (cr) { > >> > case 3: > >> > WARN_ON_ONCE(enable_unrestricted_guest); > >> > + > >> > >> Here, were you intended to replace WARN_ON_ONCE() with KVM_BUG_ON() or > >> this is just a stray newline added? > > > > I think it's just a stray newline. At one point I had converted this to a > > KVM_BUG_ON(), but then reversed direction because it's not fatal to the guest, > > i.e. KVM should continue to function even though it's spuriously intercepting > > CR3 loads. > > > > Which, rereading this patch, completely contradicts the KVM_BUG() for CLTS. > > > > That's probably something we should sort out in this RFC: is KVM_BUG() only > > to be used if the bug is fatal/dangerous, or should it be used any time the > > error is definitely a KVM (or hardware) bug. > > Personally, I'm feeling adventurous so my vote goes to the later :-) > Whenever a KVM bug was discovered by a VM it's much safer to stop > executing it as who knows what the implications might be? Not necessarily, e.g. terminating the VM may corrupt the VM's file system, which is less safe, for lack of a better word, from the VM's perspective. > In this particular case I can think of a nested scenario when L1 didn't > ask for CR3 intercept but L0 is still injecting it. It is not fatal by > itself but probably there is bug in calculating intercepts in L0 so > if we're getting something extra maybe we're also missing some? And this > doesn't sound good at all. Hmm, but by that argument this scenario would fall into the "dangerous" part of "bug is fatal/dangerous". I guess my opinion is that we should set a fairly high bar for using KVM_BUG() so that KVM can be aggressive in shutting down. > > In theory, it should be impossible to reach this again as "r = -EIO" will > > bounce this out to userspace, the common checks to deny all ioctls() will > > prevent reinvoking KVM_RUN. > > Do we actually want to prevent *all* ioctls? E.g. when 'vm bugged' > condition is triggered userspace may want to extract some information to > assist debugging but even things like KVM_GET_[S]REGS will just return > -EIO. I'm not sure it is generally safe to enable *everything* (except > for KVM_RUN which should definitely be forbidden) so maybe your approach > is preferable. The answer to this probably depends on the answer to the first question of when it's appropriate to use KVM_BUG(). E.g. if we limit usage to fatal or dangrous cases, then blocking all ioctls() is probably the right thing do do.