From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B48ABC433E1 for ; Wed, 19 Aug 2020 21:26:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 939EB2067C for ; Wed, 19 Aug 2020 21:26:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="sACXdxPy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727858AbgHSV0Q (ORCPT ); Wed, 19 Aug 2020 17:26:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727125AbgHSV0M (ORCPT ); Wed, 19 Aug 2020 17:26:12 -0400 Received: from mail-ot1-x344.google.com (mail-ot1-x344.google.com [IPv6:2607:f8b0:4864:20::344]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81B8DC061384 for ; Wed, 19 Aug 2020 14:26:12 -0700 (PDT) Received: by mail-ot1-x344.google.com with SMTP id k12so20308089otr.1 for ; Wed, 19 Aug 2020 14:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=nIUorchxEqSUbrKFSXauXIdexeZ+y78f0sS0GWpnDMU=; b=sACXdxPyni7vFFPMXlG14vAyrsQ90Tzwn5qsmxLoYZtf+qU+nGF5JvdDQYNRRoDuZO Kdi+LQIkpqVrO3VICfaF0IHleMBXVxC2ptqvIukbkJcKIyqVl8CslrKIRek2ZK/MD2cG 6uR8RacEpULjcicv4UhibrqMJ4v1cIjjGTf+2VZXALVYwnC1VJcO+4wqI4HwFcEzkF6h wePJKUDBmOqtCZqOPSa0rSbJ4WSqzky0hac6zBBcPKiIC/0m4sGrXfHDPu4dxNXe6e39 pbMCs17AzP8t8Nc1EYOxFXRKzKuc84XtOSMC+5GlUBnHnAs1AEi0r8EupqomGOjLAjPc Ss0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=nIUorchxEqSUbrKFSXauXIdexeZ+y78f0sS0GWpnDMU=; b=R+jNJOUTjWkD73j5fvnOfUtx13yxxf3bjvqBCoRbTnFdcvT3loehkqALUDFhAZ+wgd OBU3ZFhankZFP0PA2fFRrDmlLcajgkKUH3gd3JIHBEDjfTUqDPnFAs9OTD5mo2g5NZ1J 9rbXNDDhfLoDt4J/R4fHmK+HUNV/1HggTtyurmIJAAqlorA4JcQ8suZnNeieQX7sSe9Q EkfRofH1SuOl+qOrchBjCHnj7JT0D3+WceHw7YMurglzL8Ga+eTA0ncKl7aAkatAzla5 yaiRT3AHnFr7EcN/GGKzeswltGYxBupXz0/W23HpB0wtK3kuskYLHUucfD3u0xLfnrtu RUjA== X-Gm-Message-State: AOAM533g9RWPOehqAtVQbSzdh3NrJZYTLGnlr+dcVbeZD5CFaGcDw9lt sfkDzxK3jIBK+wJozG2jZVj4qE0szkOa/uZQkV1SCQ== X-Google-Smtp-Source: ABdhPJzDEhYWtgbxFIoNbIl6nvUXg6m1nwP+Cma508GkraFee4w2YnO9D79RG930cyfq42GIoecoDC3qxMMYBsZFJeo= X-Received: by 2002:a9d:ae9:: with SMTP id 96mr19469993otq.241.1597872371444; Wed, 19 Aug 2020 14:26:11 -0700 (PDT) MIME-Version: 1.0 References: <20200803211423.29398-1-graf@amazon.com> In-Reply-To: <20200803211423.29398-1-graf@amazon.com> From: Jim Mattson Date: Wed, 19 Aug 2020 14:25:59 -0700 Message-ID: Subject: Re: [PATCH v4 0/3] Allow user space to restrict and augment MSR emulation To: Alexander Graf Cc: Paolo Bonzini , Jonathan Corbet , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Joerg Roedel , KarimAllah Raslan , Aaron Lewis , kvm list , linux-doc@vger.kernel.org, LKML Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Aug 3, 2020 at 2:14 PM Alexander Graf wrote: > > While tying to add support for the MSR_CORE_THREAD_COUNT MSR in KVM, > I realized that we were still in a world where user space has no control > over what happens with MSR emulation in KVM. > > That is bad for multiple reasons. In my case, I wanted to emulate the > MSR in user space, because it's a CPU specific register that does not > exist on older CPUs and that really only contains informational data that > is on the package level, so it's a natural fit for user space to provide > it. > > However, it is also bad on a platform compatibility level. Currrently, > KVM has no way to expose different MSRs based on the selected target CPU > type. > > This patch set introduces a way for user space to indicate to KVM which > MSRs should be handled in kernel space. With that, we can solve part of > the platform compatibility story. Or at least we can not handle AMD specific > MSRs on an Intel platform and vice versa. > > In addition, it introduces a way for user space to get into the loop > when an MSR access would generate a #GP fault, such as when KVM finds an > MSR that is not handled by the in-kernel MSR emulation or when the guest > is trying to access reserved registers. > > In combination with the allow list, the user space trapping allows us > to emulate arbitrary MSRs in user space, paving the way for target CPU > specific MSR implementations from user space. This is somewhat misleading. If you don't modify the MSR permission bitmaps, as Aaron has done, you cannot emulate *arbitrary* MSRs in userspace. You can only emulate MSRs that kvm is going to intercept. Moreover, since the set of intercepted MSRs evolves over time, this isn't a stable API. > v1 -> v2: > > - s/ETRAP_TO_USER_SPACE/ENOENT/g > - deflect all #GP injection events to user space, not just unknown MSRs. > That was we can also deflect allowlist errors later > - fix emulator case > - new patch: KVM: x86: Introduce allow list for MSR emulation > - new patch: KVM: selftests: Add test for user space MSR handling > > v2 -> v3: > > - return r if r == X86EMUL_IO_NEEDED > - s/KVM_EXIT_RDMSR/KVM_EXIT_X86_RDMSR/g > - s/KVM_EXIT_WRMSR/KVM_EXIT_X86_WRMSR/g > - Use complete_userspace_io logic instead of reply field > - Simplify trapping code > - document flags for KVM_X86_ADD_MSR_ALLOWLIST > - generalize exit path, always unlock when returning > - s/KVM_CAP_ADD_MSR_ALLOWLIST/KVM_CAP_X86_MSR_ALLOWLIST/g > - Add KVM_X86_CLEAR_MSR_ALLOWLIST > - Add test to clear whitelist > - Adjust to reply-less API > - Fix asserts > - Actually trap on MSR_IA32_POWER_CTL writes > > v3 -> v4: > > - Mention exit reasons in re-enter mandatory section of API documentation > - Clear padding bytes > - Generalize get/set deflect functions > - Remove redundant pending_user_msr field > - lock allow check and clearing > - free bitmaps on clear > > Alexander Graf (3): > KVM: x86: Deflect unknown MSR accesses to user space > KVM: x86: Introduce allow list for MSR emulation > KVM: selftests: Add test for user space MSR handling > > Documentation/virt/kvm/api.rst | 157 ++++++++++- > arch/x86/include/asm/kvm_host.h | 13 + > arch/x86/include/uapi/asm/kvm.h | 15 + > arch/x86/kvm/emulate.c | 18 +- > arch/x86/kvm/x86.c | 259 +++++++++++++++++- > include/trace/events/kvm.h | 2 +- > include/uapi/linux/kvm.h | 15 + > tools/testing/selftests/kvm/Makefile | 1 + > .../selftests/kvm/x86_64/user_msr_test.c | 221 +++++++++++++++ > 9 files changed, 692 insertions(+), 9 deletions(-) > create mode 100644 tools/testing/selftests/kvm/x86_64/user_msr_test.c > > -- > 2.17.1 > > > > > Amazon Development Center Germany GmbH > Krausenstr. 38 > 10117 Berlin > Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss > Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B > Sitz: Berlin > Ust-ID: DE 289 237 879 > > >