From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 314A4C433E1 for ; Sat, 27 Jun 2020 03:47:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E7AB121473 for ; Sat, 27 Jun 2020 03:47:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="bUr8sMsx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E7AB121473 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5149A6B0031; Fri, 26 Jun 2020 23:47:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C6D16B0036; Fri, 26 Jun 2020 23:47:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B68F6B005A; Fri, 26 Jun 2020 23:47:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id 208BE6B0031 for ; Fri, 26 Jun 2020 23:47:07 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A52DA1EE6 for ; Sat, 27 Jun 2020 03:47:06 +0000 (UTC) X-FDA: 76973606052.07.value22_1f02cb126e5a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id 7E9A61803F9A9 for ; Sat, 27 Jun 2020 03:47:06 +0000 (UTC) X-HE-Tag: value22_1f02cb126e5a X-Filterd-Recvd-Size: 3142 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 03:47:06 +0000 (UTC) Received: from X1 (nat-ab2241.sltdut.senawave.net [162.218.216.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E39712073E; Sat, 27 Jun 2020 03:47:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1593229625; bh=7M/aFJ4oCOEcKIXVhUAL7PKvd2xl0X++eFKfo6b95LY=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=bUr8sMsxHrVrBaccYyd85cCqLAtEnJmUNoxmm8PZWhwZHcc1jK7BN4eRWytH9bhos OmlGTTxNjjajBsmYLsSq7wQIeFy3Q+XzmFVun188tt8rZHUzX0XziHzXUHD6TPRkYX 9Lv0CH1dAtiKWACeliqstPy+PlbMQcAC2ldm8GnQ= Date: Fri, 26 Jun 2020 20:47:04 -0700 From: Andrew Morton To: Matthew Wilcox Cc: Tim Chen , Vladimir Davydov , Johannes Weiner , Michal Hocko , Dave Hansen , Ying Huang , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [Patch] mm: Increase pagevec size on large system Message-Id: <20200626204704.f023988699421db00e9bdab7@linux-foundation.org> In-Reply-To: <20200627031304.GC25039@casper.infradead.org> References: <20200627031304.GC25039@casper.infradead.org> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 7E9A61803F9A9 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, 27 Jun 2020 04:13:04 +0100 Matthew Wilcox wrote: > On Fri, Jun 26, 2020 at 02:23:03PM -0700, Tim Chen wrote: > > Enlarge the pagevec size to 31 to reduce LRU lock contention for > > large systems. > > > > The LRU lock contention is reduced from 8.9% of total CPU cycles > > to 2.2% of CPU cyles. And the pmbench throughput increases > > from 88.8 Mpages/sec to 95.1 Mpages/sec. > > The downside here is that pagevecs are often stored on the stack (eg > truncate_inode_pages_range()) as well as being used for the LRU list. > On a 64-bit system, this increases the stack usage from 128 to 256 bytes > for this array. > > I wonder if we could do something where we transform the ones on the > stack to DECLARE_STACK_PAGEVEC(pvec), and similarly DECLARE_LRU_PAGEVEC > the ones used for the LRUs. There's plenty of space in the header to > add an unsigned char sz, delete PAGEVEC_SIZE and make it an variable > length struct. > > Or maybe our stacks are now big enough that we just don't care. > What do you think? And I wonder how useful CONFIG_NR_CPUS is for making this decision. Presumably a lot of general-purpose kernel builds have CONFIG_NR_CPUS much larger than the actual number of CPUs. I can't think of much of a fix for this, apart from making it larger on all kernels, Is there a downside to this?