Re: kernel BUG at arch/x86/kvm/x86.c:361! on AMD CPU

From: Dmitry Vyukov <dvyukov@google.com>
To: Borislav Petkov <bp@alien8.de>
Cc: George Kennedy <george.kennedy@oracle.com>,
	Sean Christopherson <sean.j.christopherson@intel.com>,
	Joerg Roedel <joro@8bytes.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
	KVM list <kvm@vger.kernel.org>,
	syzkaller <syzkaller@googlegroups.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>
Subject: Re: kernel BUG at arch/x86/kvm/x86.c:361! on AMD CPU
Date: Sun, 23 Jun 2019 15:15:10 +0200	[thread overview]
Message-ID: <CACT4Y+ZKcbcZmMs_PMJuofieLeugkFjx6CDQertWYhxiF9aTvg@mail.gmail.com> (raw)
In-Reply-To: <20190618182733.GD26346@zn.tnic>

On Tue, Jun 18, 2019 at 8:27 PM Borislav Petkov <bp@alien8.de> wrote:
>
> On Tue, Jun 18, 2019 at 08:01:06PM +0200, Dmitry Vyukov wrote:
> > I am not a KVM folk either, but FWIW syzkaller is capable of creating
> > a double-nested VM.
>
> Aaaha, there it is. :)
>
> > The code is somewhat VMX-specific, but it should
> > be capable at least executing some SVM instructions inside of guest.
> > This code setups VM to run a given instruction sequences (should be generic):
> > https://github.com/google/syzkaller/blob/34bf9440bd06034f86b5d9ac8afbf078129cbdae/executor/common_kvm_amd64.h
> > The instruction generator is based on Intel XED so it may be somewhat
> > Intel-biased, but at least I see some mentions of SVM there:
> > https://raw.githubusercontent.com/google/syzkaller/34bf9440bd06034f86b5d9ac8afbf078129cbdae/pkg/ifuzz/gen/all-enc-instructions.txt
>
> Right, and that right there looks wrong:
>
> ICLASS    : VMLOAD
> CPL       : 3
> CATEGORY  : SYSTEM
> EXTENSION : SVM
> ATTRIBUTES: PROTECTED_MODE
> PATTERN   : 0x0F 0x01 MOD[0b11] MOD=3 REG[0b011] RM[0b010]
> OPERANDS  : REG0=OrAX():r:IMPL
>
> That is, *if* "CPL: 3" above means in XED context that VMLOAD is
> supposed to be run in CPL3, then this is wrong because VMLOAD #GPs if
> CPL was not 0. Ditto for VMRUN and a couple of others.
>
> Perhaps that support was added at some point but not really run on AMD
> hw yet...

Interesting. I've updated to the latest Intel XED:
https://github.com/google/syzkaller/commit/472f0082fd8a2f82b85ab0682086e10b71529a51

And I actually see a number of changes around this for the VMX instructions:

 ICLASS    : VMPTRLD
-CPL       : 3
+CPL       : 0

 ICLASS    : VMXON
-CPL       : 3
+CPL       : 0

 ICLASS    : VMLAUNCH
-CPL       : 3
+CPL       : 0

But VMLOAD is still marked as CPL 3.
Perhaps it's something to fix in Intel XED. But for syzkaller it
should not matter much, it's a fuzzer and it will do all kinds of
crazy non-conforming stuff. These CPLs are only used as hints at best
if at all. It should sure try CPL=0 instructions in CPL3 regardless.