From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0705C433ED for ; Thu, 8 Apr 2021 13:46:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4C1DE610FA for ; Thu, 8 Apr 2021 13:46:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C1DE610FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D4EBD6B0078; Thu, 8 Apr 2021 09:46:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D000B6B007E; Thu, 8 Apr 2021 09:46:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B9CFF6B0080; Thu, 8 Apr 2021 09:46:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id 9AAF96B0078 for ; Thu, 8 Apr 2021 09:46:20 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 45C298248047 for ; Thu, 8 Apr 2021 13:46:20 +0000 (UTC) X-FDA: 78009324120.34.9C0E23C Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf15.hostedemail.com (Postfix) with ESMTP id CDDE2A0003A8 for ; Thu, 8 Apr 2021 13:46:17 +0000 (UTC) IronPort-SDR: hgcDPPDWoaF15tjCYWsOIsmQIutmU6DW7kqzr2A11IdOw9bWAX2bUGLx5SisZRIYovqkHGl1n0 e1DPiLVNqIKQ== X-IronPort-AV: E=McAfee;i="6000,8403,9948"; a="193582990" X-IronPort-AV: E=Sophos;i="5.82,206,1613462400"; d="scan'208";a="193582990" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2021 06:46:17 -0700 IronPort-SDR: PFuqVYXQVDMRqvIwcoIv55qr11MROY/k3kS/4SijUECqUDT5YvCwG7rd5losMtsAnji4W+GcuE /P1/Sjzi0x5A== X-IronPort-AV: E=Sophos;i="5.82,206,1613462400"; d="scan'208";a="415776505" Received: from yhuang6-desk1.sh.intel.com (HELO yhuang6-desk1.ccr.corp.intel.com) ([10.239.13.1]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2021 06:46:13 -0700 From: "Huang, Ying" To: Mel Gorman , Andrew Morton Cc: , , Peter Zijlstra , "Peter Xu" , Johannes Weiner , "Vlastimil Babka" , Matthew Wilcox , Will Deacon , Michel Lespinasse , Arjun Roy , "Kirill A. Shutemov" Subject: Re: [PATCH -V2] NUMA balancing: reduce TLB flush via delaying mapping on hint page fault References: <20210402082717.3525316-1-ying.huang@intel.com> <20210407082728.GA15768@suse.de> Date: Thu, 08 Apr 2021 21:46:11 +0800 In-Reply-To: <20210407082728.GA15768@suse.de> (Mel Gorman's message of "Wed, 7 Apr 2021 09:27:28 +0100") Message-ID: <87lf9syjmk.fsf@yhuang6-desk1.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: CDDE2A0003A8 X-Stat-Signature: 8jnx1hb8j7onf4mhty8mfxc7rxbw7g8y Received-SPF: none (intel.com>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mga03.intel.com; client-ip=134.134.136.65 X-HE-DKIM-Result: none/none X-HE-Tag: 1617889577-671529 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Mel Gorman writes: > On Fri, Apr 02, 2021 at 04:27:17PM +0800, Huang Ying wrote: >> With NUMA balancing, in hint page fault handler, the faulting page >> will be migrated to the accessing node if necessary. During the >> migration, TLB will be shot down on all CPUs that the process has run >> on recently. Because in the hint page fault handler, the PTE will be >> made accessible before the migration is tried. The overhead of TLB >> shooting down can be high, so it's better to be avoided if possible. >> In fact, if we delay mapping the page until migration, that can be >> avoided. This is what this patch doing. >> >> >> > > Thanks, I think this is ok for Andrew to pick up to see if anything > bisects to this commit but it's a low risk. > > Reviewed-by: Mel Gorman > > More notes; > > This is not a universal win given that not all workloads exhibit the > pattern where accesses occur in parallel threads between when a page > is marked accessible and when it is migrated. The impact of the patch > appears to be neutral for those workloads. For workloads that do exhibit > the pattern, there is a small gain with a reduction in interrupts as > advertised unlike v1 of the patch. Further tests are running to confirm > the reduction is in TLB shootdown interrupts but I'm reasonably confident > that will be the case. Gains are typically small and the load described in > the changelog appears to be a best case scenario but a 1-5% gain in some > other workloads is still an improvement. There is still the possibility > that some workloads will unnecessarily stall as a result of the patch > for slightly longer periods of time but that is a relatively low risk > and will be difficult to detect. If I'm wrong, a bisection will find it. Hi, Mel, Thanks! Hi, Andrew, I found that V2 cannot apply on top of latest mmotm, so I send V3 as follows. In case you need it. https://lore.kernel.org/lkml/20210408132236.1175607-1-ying.huang@intel.com/ Best Regards, Huang, Ying