From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57962C433DF for ; Wed, 27 May 2020 21:22:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1831D207D8 for ; Wed, 27 May 2020 21:22:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="j2ZgK8Ew" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1831D207D8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8A6058001A; Wed, 27 May 2020 17:22:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8578680010; Wed, 27 May 2020 17:22:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76D988001A; Wed, 27 May 2020 17:22:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0140.hostedemail.com [216.40.44.140]) by kanga.kvack.org (Postfix) with ESMTP id 6370E80010 for ; Wed, 27 May 2020 17:22:11 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 2438B8248076 for ; Wed, 27 May 2020 21:22:11 +0000 (UTC) X-FDA: 76863772062.25.skin59_4d313f194264f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id F2EA31804E3B5 for ; Wed, 27 May 2020 21:22:10 +0000 (UTC) X-HE-Tag: skin59_4d313f194264f X-Filterd-Recvd-Size: 5373 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Wed, 27 May 2020 21:22:10 +0000 (UTC) Received: from kernel.org (unknown [87.71.78.142]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 612E52078C; Wed, 27 May 2020 21:22:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1590614529; bh=ZfN6TjCih4KRWkakKKG5b5uklqfy1UErCB1lMEJOcfo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=j2ZgK8EwwFrGozUSttXAmBtw0/ayQsFW5FCRyot+Db9RnnnICgh26zIKdwlOtXgep 2EUvcj6Zr4e0h0wGRqXx9com35n6GlA4rF8kTsDjfz5KwKqryTTeihjFRrSjvVfgCF Q6EnS/w7tfSWhWnJJg4ICc5DWPCfTn2ITCaFND10= Date: Thu, 28 May 2020 00:22:00 +0300 From: Mike Rapoport To: Dave Hansen Cc: Liran Alon , "Kirill A. Shutemov" , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Rientjes , Andrea Arcangeli , Kees Cook , Will Drewry , "Edgecombe, Rick P" , "Kleen, Andi" , x86@kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: Re: [RFC 00/16] KVM protected memory extension Message-ID: <20200527212200.GH48741@kernel.org> References: <20200522125214.31348-1-kirill.shutemov@linux.intel.com> <42685c32-a7a9-b971-0cf4-e8af8d9a40c6@oracle.com> <20200526061721.GB48741@kernel.org> <8866ff79-e8fd-685d-9a1d-72acff5bf6bb@oracle.com> <20200526113844.GC48741@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: F2EA31804E3B5 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 27, 2020 at 08:45:33AM -0700, Dave Hansen wrote: > On 5/26/20 4:38 AM, Mike Rapoport wrote: > > On Tue, May 26, 2020 at 01:16:14PM +0300, Liran Alon wrote: > >> On 26/05/2020 9:17, Mike Rapoport wrote: > >>> On Mon, May 25, 2020 at 04:47:18PM +0300, Liran Alon wrote: > >>>> On 22/05/2020 15:51, Kirill A. Shutemov wrote: > >>>> > >>> Out of curiosity, do we actually have some numbers for the "non-trivial > >>> performance cost"? For instance for KVM usecase? > >>> > >> Dig into XPFO mailing-list discussions to find out... > >> I just remember that this was one of the main concerns regarding XPFO. > > > > The XPFO benchmarks measure total XPFO cost, and huge share of it comes > > from TLB shootdowns. > > Yes, TLB shootdown when pages transition between owners is huge. The > XPFO folks did a lot of work to try to optimize some of this overhead > away. But, it's still a concern. > > The concern with XPFO was that it could affect *all* application page > allocation. This approach cheats a bit and only goes after guest VM > pages. It's significantly more work to allocate a page and map it into > a guest than it is to, for instance, allocate an anonymous user page. > That means that the *additional* overhead of things like this for guest > memory matter a lot less. > > > It's not exactly measurement of the imapct of the direct map > > fragmentation to workload running inside a vitrual machine. > > While the VM *itself* is running, there is zero overhead. The host > direct map is not used at *all*. The guest and host TLB entries share > the same space in the TLB so there could be some increased pressure on > the TLB, but that's a really secondary effect. It would also only occur > if the guest exits and the host runs and starts evicting TLB entries. > > The other effects I could think of would be when the guest exits and the > host is doing some work for the guest, like emulation or something. The > host would see worse TLB behavior because the host is using the > (fragmented) direct map. > > But, both of those things require VMEXITs. The more exits, the more > overhead you _might_ observe. What I've been hearing from KVM folks is > that exits are getting more and more rare and the hardware designers are > working hard to minimize them. Right, when guest stays in the guest mode, there is no overhead. But guests still exit sometimes and I was wondering if anybody had measured difference in the overhead with different page size used for the host's direct map. My guesstimate is that the overhead will not differ much for most workloads. But still, it's still interesting to *know* what is it. > That's especially good news because it means that even if the > situation > isn't perfect, it's only bound to get *better* over time, not worse. The processors have been aggressively improving performance for decades and see where are we know because of it ;-) -- Sincerely yours, Mike.