From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C306C4740C for ; Mon, 9 Sep 2019 15:11:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E2F2620863 for ; Mon, 9 Sep 2019 15:11:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388089AbfIIPLi (ORCPT ); Mon, 9 Sep 2019 11:11:38 -0400 Received: from mga07.intel.com ([134.134.136.100]:37550 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727674AbfIIPLh (ORCPT ); Mon, 9 Sep 2019 11:11:37 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Sep 2019 08:11:36 -0700 X-IronPort-AV: E=Sophos;i="5.64,486,1559545200"; d="scan'208";a="175007390" Received: from ahduyck-desk1.jf.intel.com ([10.7.198.76]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Sep 2019 08:11:36 -0700 Message-ID: <0ca58fea280b51b83e7b42e2087128789bc9448d.camel@linux.intel.com> Subject: Re: [PATCH v9 1/8] mm: Add per-cpu logic to page shuffling From: Alexander Duyck To: David Hildenbrand , Alexander Duyck , virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, mst@redhat.com, catalin.marinas@arm.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, willy@infradead.org, mhocko@kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, will@kernel.org, linux-arm-kernel@lists.infradead.org, osalvador@suse.de Cc: yang.zhang.wz@gmail.com, pagupta@redhat.com, konrad.wilk@oracle.com, nitesh@redhat.com, riel@surriel.com, lcapitulino@redhat.com, wei.w.wang@intel.com, aarcange@redhat.com, ying.huang@intel.com, pbonzini@redhat.com, dan.j.williams@intel.com, fengguang.wu@intel.com, kirill.shutemov@linux.intel.com Date: Mon, 09 Sep 2019 08:11:36 -0700 In-Reply-To: <0df2e5d0-af92-04b4-aa7d-891387874039@redhat.com> References: <20190907172225.10910.34302.stgit@localhost.localdomain> <20190907172512.10910.74435.stgit@localhost.localdomain> <0df2e5d0-af92-04b4-aa7d-891387874039@redhat.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.30.5 (3.30.5-1.fc29) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 2019-09-09 at 10:14 +0200, David Hildenbrand wrote: > On 07.09.19 19:25, Alexander Duyck wrote: > > From: Alexander Duyck > > > > Change the logic used to generate randomness in the suffle path so that we > > can avoid cache line bouncing. The previous logic was sharing the offset > > and entropy word between all CPUs. As such this can result in cache line > > bouncing and will ultimately hurt performance when enabled. > > So, usually we perform such changes if there is real evidence. Do you > have any such performance numbers to back your claims? I'll have to go rerun the test to get the exact numbers. The reason this came up is that my original test was spanning NUMA nodes and that made this more expensive as a result since the memory was both not local to the CPU and was being updated by multiple sockets. I will try building a pair of host kernels with shuffling enabled and this patch applied to one and can add that data to the patch description. - Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2277C4740A for ; Mon, 9 Sep 2019 15:11:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70D4920863 for ; Mon, 9 Sep 2019 15:11:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70D4920863 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1C61C6B0008; Mon, 9 Sep 2019 11:11:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 176436B000A; Mon, 9 Sep 2019 11:11:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 08B386B000C; Mon, 9 Sep 2019 11:11:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id D77FC6B0008 for ; Mon, 9 Sep 2019 11:11:39 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 82AE5181AC9BF for ; Mon, 9 Sep 2019 15:11:39 +0000 (UTC) X-FDA: 75915721518.03.rain81_82ce8dc3dd109 X-HE-Tag: rain81_82ce8dc3dd109 X-Filterd-Recvd-Size: 3062 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Mon, 9 Sep 2019 15:11:38 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Sep 2019 08:11:36 -0700 X-IronPort-AV: E=Sophos;i="5.64,486,1559545200"; d="scan'208";a="175007390" Received: from ahduyck-desk1.jf.intel.com ([10.7.198.76]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Sep 2019 08:11:36 -0700 Message-ID: <0ca58fea280b51b83e7b42e2087128789bc9448d.camel@linux.intel.com> Subject: Re: [PATCH v9 1/8] mm: Add per-cpu logic to page shuffling From: Alexander Duyck To: David Hildenbrand , Alexander Duyck , virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, mst@redhat.com, catalin.marinas@arm.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, willy@infradead.org, mhocko@kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, will@kernel.org, linux-arm-kernel@lists.infradead.org, osalvador@suse.de Cc: yang.zhang.wz@gmail.com, pagupta@redhat.com, konrad.wilk@oracle.com, nitesh@redhat.com, riel@surriel.com, lcapitulino@redhat.com, wei.w.wang@intel.com, aarcange@redhat.com, ying.huang@intel.com, pbonzini@redhat.com, dan.j.williams@intel.com, fengguang.wu@intel.com, kirill.shutemov@linux.intel.com Date: Mon, 09 Sep 2019 08:11:36 -0700 In-Reply-To: <0df2e5d0-af92-04b4-aa7d-891387874039@redhat.com> References: <20190907172225.10910.34302.stgit@localhost.localdomain> <20190907172512.10910.74435.stgit@localhost.localdomain> <0df2e5d0-af92-04b4-aa7d-891387874039@redhat.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.30.5 (3.30.5-1.fc29) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 2019-09-09 at 10:14 +0200, David Hildenbrand wrote: > On 07.09.19 19:25, Alexander Duyck wrote: > > From: Alexander Duyck > > > > Change the logic used to generate randomness in the suffle path so that we > > can avoid cache line bouncing. The previous logic was sharing the offset > > and entropy word between all CPUs. As such this can result in cache line > > bouncing and will ultimately hurt performance when enabled. > > So, usually we perform such changes if there is real evidence. Do you > have any such performance numbers to back your claims? I'll have to go rerun the test to get the exact numbers. The reason this came up is that my original test was spanning NUMA nodes and that made this more expensive as a result since the memory was both not local to the CPU and was being updated by multiple sockets. I will try building a pair of host kernels with shuffling enabled and this patch applied to one and can add that data to the patch description. - Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77534C4740A for ; Mon, 9 Sep 2019 15:11:45 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C50520863 for ; Mon, 9 Sep 2019 15:11:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Rj53v8nk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C50520863 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Date:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zJRLJ8o43hAeObzL7cOewsaAfB0eiuYNIGMNZrw55rY=; b=Rj53v8nk6KXofT bGAs788yCYejYDIg8+9YPDRpuYfgpxy1iHU3GzRkKjWcyNR+i/IShtJKwXKuidfxlmaZaxwSHEcyL hQWvq/ZfZmHY2NXvAkxehlFlK6CRXc2EWG1aW2cgh5o0Lu9UqwNuGtO+qrzBAr1RvhKZUkUFQ+TS8 j650cuuHL1vy0Pj+9SLJmkhxsWaw8t7aKjxdbbtz0A/3gLX3kiD3xlTXxF8ckAyMV7pjJ53K0G5xC S0D6KCMX4rBdgKebe+sfp+1D9I/nIhrDVMK9TPeeuLIBaTY96erxjC4iZcsxAXP2zhJwXtVDTbZln s8kC9HhshGoYycz5WoNg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i7LKP-0006Jv-Fy; Mon, 09 Sep 2019 15:11:41 +0000 Received: from mga02.intel.com ([134.134.136.20]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1i7LKM-0006J5-4c for linux-arm-kernel@lists.infradead.org; Mon, 09 Sep 2019 15:11:39 +0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Sep 2019 08:11:36 -0700 X-IronPort-AV: E=Sophos;i="5.64,486,1559545200"; d="scan'208";a="175007390" Received: from ahduyck-desk1.jf.intel.com ([10.7.198.76]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Sep 2019 08:11:36 -0700 Message-ID: <0ca58fea280b51b83e7b42e2087128789bc9448d.camel@linux.intel.com> Subject: Re: [PATCH v9 1/8] mm: Add per-cpu logic to page shuffling From: Alexander Duyck To: David Hildenbrand , Alexander Duyck , virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, mst@redhat.com, catalin.marinas@arm.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, willy@infradead.org, mhocko@kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, will@kernel.org, linux-arm-kernel@lists.infradead.org, osalvador@suse.de Date: Mon, 09 Sep 2019 08:11:36 -0700 In-Reply-To: <0df2e5d0-af92-04b4-aa7d-891387874039@redhat.com> References: <20190907172225.10910.34302.stgit@localhost.localdomain> <20190907172512.10910.74435.stgit@localhost.localdomain> <0df2e5d0-af92-04b4-aa7d-891387874039@redhat.com> User-Agent: Evolution 3.30.5 (3.30.5-1.fc29) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190909_081138_216590_8061F0B2 X-CRM114-Status: GOOD ( 11.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yang.zhang.wz@gmail.com, pagupta@redhat.com, riel@surriel.com, konrad.wilk@oracle.com, ying.huang@intel.com, lcapitulino@redhat.com, wei.w.wang@intel.com, aarcange@redhat.com, nitesh@redhat.com, pbonzini@redhat.com, dan.j.williams@intel.com, fengguang.wu@intel.com, kirill.shutemov@linux.intel.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 2019-09-09 at 10:14 +0200, David Hildenbrand wrote: > On 07.09.19 19:25, Alexander Duyck wrote: > > From: Alexander Duyck > > > > Change the logic used to generate randomness in the suffle path so that we > > can avoid cache line bouncing. The previous logic was sharing the offset > > and entropy word between all CPUs. As such this can result in cache line > > bouncing and will ultimately hurt performance when enabled. > > So, usually we perform such changes if there is real evidence. Do you > have any such performance numbers to back your claims? I'll have to go rerun the test to get the exact numbers. The reason this came up is that my original test was spanning NUMA nodes and that made this more expensive as a result since the memory was both not local to the CPU and was being updated by multiple sockets. I will try building a pair of host kernels with shuffling enabled and this patch applied to one and can add that data to the patch description. - Alex _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: virtio-dev-return-6083-cohuck=redhat.com@lists.oasis-open.org Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: Received: from lists.oasis-open.org (oasis-open.org [10.110.1.242]) by lists.oasis-open.org (Postfix) with ESMTP id 32CCC98434A for ; Mon, 9 Sep 2019 15:11:39 +0000 (UTC) Message-ID: <0ca58fea280b51b83e7b42e2087128789bc9448d.camel@linux.intel.com> From: Alexander Duyck Date: Mon, 09 Sep 2019 08:11:36 -0700 In-Reply-To: <0df2e5d0-af92-04b4-aa7d-891387874039@redhat.com> References: <20190907172225.10910.34302.stgit@localhost.localdomain> <20190907172512.10910.74435.stgit@localhost.localdomain> <0df2e5d0-af92-04b4-aa7d-891387874039@redhat.com> Content-Type: text/plain; charset="UTF-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [virtio-dev] Re: [PATCH v9 1/8] mm: Add per-cpu logic to page shuffling To: David Hildenbrand , Alexander Duyck , virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, mst@redhat.com, catalin.marinas@arm.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, willy@infradead.org, mhocko@kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, will@kernel.org, linux-arm-kernel@lists.infradead.org, osalvador@suse.de Cc: yang.zhang.wz@gmail.com, pagupta@redhat.com, konrad.wilk@oracle.com, nitesh@redhat.com, riel@surriel.com, lcapitulino@redhat.com, wei.w.wang@intel.com, aarcange@redhat.com, ying.huang@intel.com, pbonzini@redhat.com, dan.j.williams@intel.com, fengguang.wu@intel.com, kirill.shutemov@linux.intel.com List-ID: On Mon, 2019-09-09 at 10:14 +0200, David Hildenbrand wrote: > On 07.09.19 19:25, Alexander Duyck wrote: > > From: Alexander Duyck > > > > Change the logic used to generate randomness in the suffle path so that we > > can avoid cache line bouncing. The previous logic was sharing the offset > > and entropy word between all CPUs. As such this can result in cache line > > bouncing and will ultimately hurt performance when enabled. > > So, usually we perform such changes if there is real evidence. Do you > have any such performance numbers to back your claims? I'll have to go rerun the test to get the exact numbers. The reason this came up is that my original test was spanning NUMA nodes and that made this more expensive as a result since the memory was both not local to the CPU and was being updated by multiple sockets. I will try building a pair of host kernels with shuffling enabled and this patch applied to one and can add that data to the patch description. - Alex --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org