From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DATE_IN_PAST_03_06, DKIMWL_WL_MED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B65FC388F7 for ; Wed, 28 Oct 2020 22:05:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 34A772470E for ; Wed, 28 Oct 2020 22:05:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hmu6Uyd1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730017AbgJ1WFe (ORCPT ); Wed, 28 Oct 2020 18:05:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729449AbgJ1WFc (ORCPT ); Wed, 28 Oct 2020 18:05:32 -0400 Received: from mail-ot1-x343.google.com (mail-ot1-x343.google.com [IPv6:2607:f8b0:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49FA8C0613D1 for ; Wed, 28 Oct 2020 15:05:32 -0700 (PDT) Received: by mail-ot1-x343.google.com with SMTP id j21so542448ota.13 for ; Wed, 28 Oct 2020 15:05:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=dpvqDFTkgoj21BdY1pTBOrwAo1cSX7C2DrwyEeBthb0=; b=hmu6Uyd1Gc7QqAmRGDaehKNzIEHFFVZG/PPYkq4ZaUmooqpshdaE0gVQJL47IyDUs0 aa2GXsroHFjXApjXlR2NG81QyRF6TWy8OJ5Ruho6lV2jn0OOiJwvcsnUyKGGbE4Qun6p yLV+PoWlWh/lpdxMCeKORAUXgoGP5QKakKGhvfB0kHShumuv9SKYcLJLacUFkVePZY/I kZGqUs4Bky+kPkKzU8jeJpqdHvRisSNEuDmsTftWGirndVwGm3n8ES2p2NUNTlrAiH7k N/gb0RjAJy5/x+EklKbygR1hyS3FlR8Tmu9bNer1/vqXlrkKvwQEXxwz9qaCfprZswEk 3YKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=dpvqDFTkgoj21BdY1pTBOrwAo1cSX7C2DrwyEeBthb0=; b=BORDhEXqq9jqeO9PRRls+fvdHVc4m7GVj4PfjeEALWSZK5RO4PzNBr1uaUuluDsmcp t5bCFcxu9C03GCk0rmRTRWCiL9Pvk2HViG5O+BSNMpif1+v0DkTyFzLfwxeXPtXwbXNj 7HbFrubnAXrYK9h99Nemp6wegUmbbV40m7jXItXvnr9RNlJH+1HC0wz3FpO2P3zuMaOD DgiqWH0AQWnL0jYAz7Cd/YkLQzG07ONDTUqwJjXIzg/2T5PLOehw+iv+K1cN3eaVrH0S rG3kIijNrKY2jBATNxpFxZhqPkD57vLKcRqEtZbAMnCFQU+h0ilv/FOXMAcjpSRBiHJy 0WBw== X-Gm-Message-State: AOAM530SBX0tO9dyri4RQ9cXbscKu1fsMuf9EVOB4ztdPwoiogLJRmAr kidGIO7wFyvVk0rPcr6kx4WoTqzKZuBB4fZN/psS/QvF/w9CFYZM X-Google-Smtp-Source: ABdhPJwg81J95bq/E6nPimq6xCcVhmcL1hYDUWbH2VVxCaOeVS3rOgpWN0zdv6OGnLhkSCFQJPP7LbhuFVKXhlqIBBM= X-Received: by 2002:a25:d906:: with SMTP id q6mr531935ybg.316.1603908196352; Wed, 28 Oct 2020 11:03:16 -0700 (PDT) MIME-Version: 1.0 References: <20200907131613.12703-64-joro@8bytes.org> <159972972598.20229.12880317872521101289.tip-bot2@tip-bot2> <20201028094952.GI22179@suse.de> In-Reply-To: <20201028094952.GI22179@suse.de> From: Erdem Aktas Date: Wed, 28 Oct 2020 11:03:05 -0700 Message-ID: Subject: Re: [tip: x86/seves] x86/kvm: Add KVM-specific VMMCALL handling under SEV-ES To: Joerg Roedel Cc: linux-kernel@vger.kernel.org, "Lendacky, Thomas" , linux-tip-commits@vger.kernel.org, Borislav Petkov , x86 Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I might be missing something here but I think what you say is only correct for the kvm_hypercall4 cases. All other functions use a smaller number of registers. #VC blindly assumes that all those registers are used in the vmcall and exposes them. Here are some examples: For example in the kvm_hypercall2 only rax, rbx, and rcx should be exposed. apic_id address is leaked with rdx when this hypercall is used in kvm_kick_cpu function. RSI is never used. I am not sure what value will be exposed to VMM in this case: 54 static inline long kvm_hypercall2(unsigned int nr, unsigned long p1, 55 unsigned long p2) 56 { 57 long ret; 58 asm volatile(KVM_HYPERCALL 59 : "=a"(ret) 60 : "a"(nr), "b"(p1), "c"(p2) 61 : "memory"); 62 return ret; 63 } And this function is called in : 820 static void kvm_kick_cpu(int cpu) 821 { 822 int apicid; 823 unsigned long flags = 0; 824 825 apicid = per_cpu(x86_cpu_to_apicid, cpu); 826 kvm_hypercall2(KVM_HC_KICK_CPU, flags, apicid); 827 } looking to what it is compiled in my machine : 151215 ffffffff8105def0 : 151216 { 151217 ffffffff8105def0: e8 fb 9e ff ff callq ffffffff81057df0 <__fentry__> 151218 apicid = per_cpu(x86_cpu_to_apicid, cpu); 151219 ffffffff8105def5: 48 63 ff movslq %edi,%rdi 151220 { 151221 ffffffff8105def8: 53 push %rbx 151222 apicid = per_cpu(x86_cpu_to_apicid, cpu); 151223 ffffffff8105def9: 48 c7 c0 58 16 01 00 mov $0x11658,%rax 151224 151225 static inline long kvm_hypercall2(unsigned int nr, unsigned long p1, 151226 unsigned long p2) 151227 { 151228 long ret; 151229 asm volatile(KVM_HYPERCALL 151230 ffffffff8105df00: 31 db xor %ebx,%ebx 151231 ffffffff8105df02: 48 8b 14 fd 00 19 cb mov -0x7e34e700(,%rdi,8),%rdx 151232 ffffffff8105df09: 81 151233 kvm_hypercall2(KVM_HC_KICK_CPU, flags, apicid); 151234 ffffffff8105df0a: 0f b7 0c 02 movzwl (%rdx,%rax,1),%ecx 151235 ffffffff8105df0e: b8 05 00 00 00 mov $0x5,%eax 151236 ffffffff8105df13: 0f 01 c1 vmcall 151237 } 151238 ffffffff8105df16: 5b pop %rbx 151239 ffffffff8105df17: c3 retq Similarly kvm_hypercall1 only need 2 registers to expose: 44 static inline long kvm_hypercall1(unsigned int nr, unsigned long p1) 45 { 46 long ret; 47 asm volatile(KVM_HYPERCALL 48 : "=a"(ret) 49 : "a"(nr), "b"(p1) 50 : "memory"); 51 return ret; 52 } And an example where it is used: 562 static void kvm_smp_send_call_func_ipi(const struct cpumask *mask) 563 { 564 int cpu; 565 566 native_send_call_func_ipi(mask); 567 568 /* Make sure other vCPUs get a chance to run if they need to. */ 569 for_each_cpu(cpu, mask) { 570 if (vcpu_is_preempted(cpu)) { 571 kvm_hypercall1(KVM_HC_SCHED_YIELD, per_cpu(x86_cpu_to_apicid, cpu)); 572 break; 573 } 574 } 575 } If we look at the function decompiled in my platform, here x86_cpu_to_apicid address is leaked in rdx. RSI also leaks some information from kvm_smp_send_call_function_ipi function. RCX is not used so it might include something from a higher caller. 151243 ffffffff8105df20 : 151244 { 151245 ffffffff8105df20: e8 cb 9e ff ff callq ffffffff81057df0 <__fentry__> 151246 ffffffff8105df25: 53 push %rbx 151247 ffffffff8105df26: 48 89 fb mov %rdi,%rbx 151248 native_send_call_func_ipi(mask); 151249 ffffffff8105df29: e8 a2 45 ff ff callq ffffffff810524d0 151250 for_each_cpu(cpu, mask) { 151251 ffffffff8105df2e: 41 b8 ff ff ff ff mov $0xffffffff,%r8d 151252 ffffffff8105df34: eb 0e jmp ffffffff8105df44 151253 return PVOP_CALLEE1(bool, lock.vcpu_is_preempted, cpu); 151254 ffffffff8105df36: 49 63 f8 movslq %r8d,%rdi 151255 ffffffff8105df39: ff 14 25 90 93 02 82 callq *0xffffffff82029390 151256 if (vcpu_is_preempted(cpu)) { 151257 ffffffff8105df40: 84 c0 test %al,%al 151258 ffffffff8105df42: 75 18 jne ffffffff8105df5c 151259 for_each_cpu(cpu, mask) { 151260 ffffffff8105df44: 44 89 c7 mov %r8d,%edi 151261 ffffffff8105df47: 48 89 de mov %rbx,%rsi 151262 ffffffff8105df4a: e8 61 44 39 00 callq ffffffff813f23b0 151263 ffffffff8105df4f: 3b 05 2f 7e 12 01 cmp 0x1127e2f(%rip),%eax # ffffffff82185d84 151264 ffffffff8105df55: 41 89 c0 mov %eax,%r8d 151265 ffffffff8105df58: 72 dc jb ffffffff8105df36 151266 } 151267 ffffffff8105df5a: 5b pop %rbx 151268 ffffffff8105df5b: c3 retq 151269 kvm_hypercall1(KVM_HC_SCHED_YIELD, per_cpu(x86_cpu_to_apicid, cpu)); 151270 ffffffff8105df5c: 48 8b 14 fd 00 19 cb mov -0x7e34e700(,%rdi,8),%rdx 151271 ffffffff8105df63: 81 151272 ffffffff8105df64: 48 c7 c0 58 16 01 00 mov $0x11658,%rax 151273 ffffffff8105df6b: 0f b7 1c 02 movzwl (%rdx,%rax,1),%ebx 151274 asm volatile(KVM_HYPERCALL 151275 ffffffff8105df6f: b8 0b 00 00 00 mov $0xb,%eax 151276 ffffffff8105df74: 0f 01 c1 vmcall 151277 } I am not sure how those leaked registers can be used, but depending on which function call hypercall[0-3], there will be some leak. -Erdem On Wed, Oct 28, 2020 at 2:49 AM Joerg Roedel wrote: > > On Tue, Oct 27, 2020 at 04:14:15PM -0700, Erdem Aktas wrote: > > It seems to me that the kvm_sev_es_hcall_prepare is leaking more > > information than it is needed. Is this an expected behavior? > > What exactly is leaked? The kvm hypercall uses RAX, RBX, RCX, RDX and > RSI for parameters. > > Regards, > > Joerg