UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    5e60366d Merge tag 'fallthrough-fixes-clang-5.11-rc1' of g..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=11c7046b500000
kernel config:  https://syzkaller.appspot.com/x/.config?x=db720fe37a6a41d8
dashboard link: https://syzkaller.appspot.com/bug?extid=e87846c48bf72bc85311
compiler:       gcc (GCC) 10.1.0-syz 20200507
userspace arch: i386

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+e87846c48bf72bc85311@syzkaller.appspotmail.com

================================================================================
UBSAN: shift-out-of-bounds in arch/x86/kvm/mmu.h:52:16
shift exponent 64 is too large for 64-bit type 'long long unsigned int'
CPU: 1 PID: 11156 Comm: syz-executor.1 Not tainted 5.10.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0x107/0x163 lib/dump_stack.c:120
 ubsan_epilogue+0xb/0x5a lib/ubsan.c:148
 __ubsan_handle_shift_out_of_bounds.cold+0xb1/0x181 lib/ubsan.c:395
 rsvd_bits arch/x86/kvm/mmu.h:52 [inline]
 kvm_vcpu_after_set_cpuid.cold+0x35/0x3a arch/x86/kvm/cpuid.c:181
 kvm_vcpu_ioctl_set_cpuid+0x28e/0x970 arch/x86/kvm/cpuid.c:273
 kvm_arch_vcpu_ioctl+0x1091/0x2d70 arch/x86/kvm/x86.c:4699
 kvm_vcpu_ioctl+0x7b9/0xdb0 arch/x86/kvm/../../../virt/kvm/kvm_main.c:3386
 kvm_vcpu_compat_ioctl+0x1a2/0x340 arch/x86/kvm/../../../virt/kvm/kvm_main.c:3430
 __do_compat_sys_ioctl+0x1d3/0x230 fs/ioctl.c:842
 do_syscall_32_irqs_on arch/x86/entry/common.c:78 [inline]
 __do_fast_syscall_32+0x56/0x80 arch/x86/entry/common.c:137
 do_fast_syscall_32+0x2f/0x70 arch/x86/entry/common.c:160
 entry_SYSENTER_compat_after_hwframe+0x4d/0x5c
RIP: 0023:0xf7fe8549
Code: b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10 08 03 74 d8 01 00 00 00 00 00 00 00 00 00 00 00 00 00 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 eb 0d 90 90 90 90 90 90 90 90 90 90 90 90
RSP: 002b:00000000f55e20cc EFLAGS: 00000296 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000000004008ae8a
RDX: 00000000200000c0 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
================================================================================


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

Re: UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

[Jim Mattson]

It looks like userspace can possibly induce this by providing guest
CPUID information with a "physical address width" of 64 in leaf
0x80000008.

Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
CPUID.80000008H:EAX? Better would be to return an error for
out-of-range values, but I understand that the kvm community's stance
is that, in general, guest CPUID information should not be validated
by kvm.

On Tue, Dec 22, 2020 at 12:36 AM syzbot
<syzbot+e87846c48bf72bc85311@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit:    5e60366d Merge tag 'fallthrough-fixes-clang-5.11-rc1' of g..
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=11c7046b500000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=db720fe37a6a41d8
> dashboard link: https://syzkaller.appspot.com/bug?extid=e87846c48bf72bc85311
> compiler:       gcc (GCC) 10.1.0-syz 20200507
> userspace arch: i386
>
> Unfortunately, I don't have any reproducer for this issue yet.
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+e87846c48bf72bc85311@syzkaller.appspotmail.com
>
> ================================================================================
> UBSAN: shift-out-of-bounds in arch/x86/kvm/mmu.h:52:16
> shift exponent 64 is too large for 64-bit type 'long long unsigned int'
> CPU: 1 PID: 11156 Comm: syz-executor.1 Not tainted 5.10.0-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:79 [inline]
>  dump_stack+0x107/0x163 lib/dump_stack.c:120
>  ubsan_epilogue+0xb/0x5a lib/ubsan.c:148
>  __ubsan_handle_shift_out_of_bounds.cold+0xb1/0x181 lib/ubsan.c:395
>  rsvd_bits arch/x86/kvm/mmu.h:52 [inline]
>  kvm_vcpu_after_set_cpuid.cold+0x35/0x3a arch/x86/kvm/cpuid.c:181
>  kvm_vcpu_ioctl_set_cpuid+0x28e/0x970 arch/x86/kvm/cpuid.c:273
>  kvm_arch_vcpu_ioctl+0x1091/0x2d70 arch/x86/kvm/x86.c:4699
>  kvm_vcpu_ioctl+0x7b9/0xdb0 arch/x86/kvm/../../../virt/kvm/kvm_main.c:3386
>  kvm_vcpu_compat_ioctl+0x1a2/0x340 arch/x86/kvm/../../../virt/kvm/kvm_main.c:3430
>  __do_compat_sys_ioctl+0x1d3/0x230 fs/ioctl.c:842
>  do_syscall_32_irqs_on arch/x86/entry/common.c:78 [inline]
>  __do_fast_syscall_32+0x56/0x80 arch/x86/entry/common.c:137
>  do_fast_syscall_32+0x2f/0x70 arch/x86/entry/common.c:160
>  entry_SYSENTER_compat_after_hwframe+0x4d/0x5c
> RIP: 0023:0xf7fe8549
> Code: b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10 08 03 74 d8 01 00 00 00 00 00 00 00 00 00 00 00 00 00 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 eb 0d 90 90 90 90 90 90 90 90 90 90 90 90
> RSP: 002b:00000000f55e20cc EFLAGS: 00000296 ORIG_RAX: 0000000000000036
> RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000000004008ae8a
> RDX: 00000000200000c0 RSI: 0000000000000000 RDI: 0000000000000000
> RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> ================================================================================
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@googlegroups.com.
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

Re: UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

[Sean Christopherson]

On Mon, Jan 11, 2021, Jim Mattson wrote:
> It looks like userspace can possibly induce this by providing guest
> CPUID information with a "physical address width" of 64 in leaf
> 0x80000008.

It was actually the opposite, where userspace provides '0' and caused '63 - 0 + 1'
to overflow.  KVM controls the upper bound, and rsvd_bits() explicitly handles
'end < start', so an absurdly large maxpa is handled correctly.

Aleady fixed by Paolo in commit 2f80d502d627 ("KVM: x86: fix shift out of bounds
reported by UBSAN").

> Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
> CPUID.80000008H:EAX? Better would be to return an error for
> out-of-range values, but I understand that the kvm community's stance
> is that, in general, guest CPUID information should not be validated
> by kvm.

And rob Paolo of his crazy^Wbrilliant bit math shenanigans? :-D

Re: UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

[Paolo Bonzini]

On 12/01/21 00:01, Sean Christopherson wrote:
>> Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
>> CPUID.80000008H:EAX?

The low 6 bits I guess---yes, that would make sense and it would have 
also fixed the bug.

(Nevertheless it's a good idea to make rsvd_bits more robust as well, as 
in the commit that Sean referenced.

Paolo

Re: UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

[Sean Christopherson]

On Tue, Jan 12, 2021, Paolo Bonzini wrote:
> On 12/01/21 00:01, Sean Christopherson wrote:
> > > Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
> > > CPUID.80000008H:EAX?
> 
> The low 6 bits I guess---yes, that would make sense and it would have also
> fixed the bug.

No, that wouldn't have fixed this specific bug.  In this case, the issue was
CPUID.80000008H:AL == 0; masking off bits 7:6 wouldn't have changed anything.

And, masking bits 7:6 is architecturally wrong.  Both the SDM and APM state that
bits 7:0 contain the number of PA bits.

KVM could reject guest.MAXPA > host.MAXPA, but arbitrarily dropping bits would
be wrong.

Re: UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

[Paolo Bonzini]

On 12/01/21 17:53, Sean Christopherson wrote:
> On Tue, Jan 12, 2021, Paolo Bonzini wrote:
>> On 12/01/21 00:01, Sean Christopherson wrote:
>>>> Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
>>>> CPUID.80000008H:EAX?
>>
>> The low 6 bits I guess---yes, that would make sense and it would have also
>> fixed the bug.
> 
> No, that wouldn't have fixed this specific bug.  In this case, the issue was
> CPUID.80000008H:AL == 0; masking off bits 7:6 wouldn't have changed anything.

Right.

> And, masking bits 7:6 is architecturally wrong.  Both the SDM and APM state that
> bits 7:0 contain the number of PA bits.

They cannot be higher than 52, therefore bits 7:6 are (architecturally) 
always zero.  In other words, I interpret "bit 7:0 contain the number of 
PA bits" as "you need not do an '& 63' yourself", which is basically the 
opposite of "bit 7:6 might be nonzero".  If masking made any difference, 
it would be outside the spec already.

In fact another possibility to avoid UB is to do "& 63" of both s and e 
in rsvd_bits.  This would also be masking bits 7:6 of the CPUID leaf, 
just done differently.

Paolo

> KVM could reject guest.MAXPA > host.MAXPA, but arbitrarily dropping bits would
> be wrong.
> 

Re: UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

[Sean Christopherson]

On Wed, Jan 13, 2021, Paolo Bonzini wrote:
> On 12/01/21 17:53, Sean Christopherson wrote:
> > And, masking bits 7:6 is architecturally wrong.  Both the SDM and APM state that
> > bits 7:0 contain the number of PA bits.
> 
> They cannot be higher than 52,

Drat, I was going to argue that it could be >52 with a new paging mode, but both
the SDM and APM explicitly call out 52 as the max.  Spending cycles on the stuff
that really matters here... :-)

> therefore bits 7:6 are (architecturally)
> always zero.  In other words, I interpret "bit 7:0 contain the number of PA
> bits" as "you need not do an '& 63' yourself", which is basically the
> opposite of "bit 7:6 might be nonzero".  If masking made any difference, it
> would be outside the spec already.
> 
> In fact another possibility to avoid UB is to do "& 63" of both s and e in
> rsvd_bits.  This would also be masking bits 7:6 of the CPUID leaf, just done
> differently.

Hmm, 'e' is hardcoded in all call sites except kvm_mmu_reset_all_pte_masks(),
and so long as 'e <= 63' holds true, 's &= 63' is unnecessary.  What if we add
compile-time asserts on hardcoded values, and mask 'e' for the rare case where
the upper bound isn't hardcoded?  That way bogus things like rsvd_bits(63, 65)
will fail the build.

diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h
index 581925e476d6..261be1d2032b 100644
--- a/arch/x86/kvm/mmu.h
+++ b/arch/x86/kvm/mmu.h
@@ -44,8 +44,15 @@
 #define PT32_ROOT_LEVEL 2
 #define PT32E_ROOT_LEVEL 3

-static inline u64 rsvd_bits(int s, int e)
+static __always_inline u64 rsvd_bits(int s, int e)
 {
+       BUILD_BUG_ON(__builtin_constant_p(e) && __builtin_constant_p(s) && e < s);
+
+       if (__builtin_constant_p(e))
+               BUILD_BUG_ON(e > 63);
+       else
+               e &= 63;
+
        if (e < s)
                return 0;