From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71433C31E5B for ; Mon, 17 Jun 2019 18:53:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4077120B1F for ; Mon, 17 Jun 2019 18:53:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1560797619; bh=lKC3JNmm/M1af0c1PRbg2Uh820jqPX+NRIxEKPybN6M=; h=References:In-Reply-To:From:Date:Subject:To:Cc:List-ID:From; b=Up9PkiRUR6NJj6R5X6nu+UJ0CBvWBOboXN/z97bOg5W2SbwNca30H/r+rEQIe5Y6F zIM5SobnbextDD+L+t7t7blcrRjeGqlxYZOy2oh+sISNHJebVZX3F2zadDVr5N80qq jUtKVcU+vVV41PzZeAYlKq3LmYF5xcFYxt9Pf/PE= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726761AbfFQSxi (ORCPT ); Mon, 17 Jun 2019 14:53:38 -0400 Received: from mail.kernel.org ([198.145.29.99]:48008 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725772AbfFQSxh (ORCPT ); Mon, 17 Jun 2019 14:53:37 -0400 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AF26321655 for ; Mon, 17 Jun 2019 18:53:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1560797617; bh=lKC3JNmm/M1af0c1PRbg2Uh820jqPX+NRIxEKPybN6M=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=0Dr1Iy1V0hAJ3rnm6kc3K5SbhQQtt9OipXmix3nmhNMADb4NoPh18+yKuKBvN4EU4 Z6dOzZ56nlA9rH2kDsoMZcP6Qdi3MpfFZ6Jbga/CQDaoTLaiImVUds/GSgaOjiZNTe slkiFT+xALD+x9RZbCyPNafq4xEcyCCYbAgcNYR8= Received: by mail-wr1-f41.google.com with SMTP id p13so11128131wru.10 for ; Mon, 17 Jun 2019 11:53:36 -0700 (PDT) X-Gm-Message-State: APjAAAUbmHLXoQQs08DaN+FxAhMzRfHQafGrtAmIGDZJErgjzc8KwKWE OSAmvkijO/eNadv3dEMbmcu08EQIiOeR4pq0YyFH1A== X-Google-Smtp-Source: APXvYqyNAFrO0PJ/PQ/nZtVT7qVJ0gCZKAFFrDHCa57/0Vwr7NK90EHDsldsfHv0mzTzjWiWbLBIz0Xs4+/j24iLBNA= X-Received: by 2002:a5d:6207:: with SMTP id y7mr56496191wru.265.1560797615195; Mon, 17 Jun 2019 11:53:35 -0700 (PDT) MIME-Version: 1.0 References: <58788f05-04c3-e71c-12c3-0123be55012c@amazon.com> <63b1b249-6bc7-ffd9-99db-d36dd3f1a962@intel.com> <698ca264-123d-46ae-c165-ed62ea149896@intel.com> <5AA8BF10-8987-4FCB-870C-667A5228D97B@gmail.com> <20190617184536.GB11017@char.us.oracle.com> In-Reply-To: <20190617184536.GB11017@char.us.oracle.com> From: Andy Lutomirski Date: Mon, 17 Jun 2019 11:53:22 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFC 00/10] Process-local memory allocations for hiding KVM secrets To: Konrad Rzeszutek Wilk Cc: Dave Hansen , Nadav Amit , Andy Lutomirski , Alexander Graf , Thomas Gleixner , Marius Hillenbrand , kvm list , LKML , Kernel Hardening , Linux-MM , Alexander Graf , David Woodhouse , "the arch/x86 maintainers" , Peter Zijlstra Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Jun 17, 2019 at 11:44 AM Konrad Rzeszutek Wilk wrote: > > On Mon, Jun 17, 2019 at 11:07:45AM -0700, Dave Hansen wrote: > > On 6/17/19 9:53 AM, Nadav Amit wrote: > > >>> For anyone following along at home, I'm going to go off into crazy > > >>> per-cpu-pgds speculation mode now... Feel free to stop reading now. :) > > >>> > > >>> But, I was thinking we could get away with not doing this on _every_ > > >>> context switch at least. For instance, couldn't 'struct tlb_context' > > >>> have PGD pointer (or two with PTI) in addition to the TLB info? That > > >>> way we only do the copying when we change the context. Or does that tie > > >>> the implementation up too much with PCIDs? > > >> Hmm, that seems entirely reasonable. I think the nasty bit would be > > >> figuring out all the interactions with PV TLB flushing. PV TLB > > >> flushes already don't play so well with PCID tracking, and this will > > >> make it worse. We probably need to rewrite all that code regardless. > > > How is PCID (as you implemented) related to TLB flushing of kernel (not > > > user) PTEs? These kernel PTEs would be global, so they would be invalidated > > > from all the address-spaces using INVLPG, I presume. No? > > > > The idea is that you have a per-cpu address space. Certain kernel > > virtual addresses would map to different physical address based on where > > you are running. Each of the physical addresses would be "owned" by a > > single CPU and would, by convention, never use a PGD that mapped an > > address unless that CPU that "owned" it. > > > > In that case, you never really invalidate those addresses. > > But you would need to invalidate if the process moved to another CPU, correct? > There's nothing to invalidate. It's a different CPU with a different TLB. The big problem is that you have a choice. Either you can have one PGD per (mm, cpu) or you just have one or a few PGDs per CPU and you change them every time you change processes. Dave's idea to have one or two per (cpu, asid) is right, though. It means we have a decent chance of context switching without rewriting the whole thing, and it also means we don't need to write to the one that's currently loaded when we switch CR3. The latter could plausibly be important enough that we'd want to pretend we're using PCID even if we're not.