From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9151CC47254 for ; Thu, 30 Apr 2020 21:31:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 50EF020873 for ; Thu, 30 Apr 2020 21:31:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="n3v24GDL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 50EF020873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C74528E0005; Thu, 30 Apr 2020 17:31:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BFDF78E0001; Thu, 30 Apr 2020 17:31:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC4F48E0005; Thu, 30 Apr 2020 17:31:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 903CB8E0001 for ; Thu, 30 Apr 2020 17:31:34 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4C1FA180AD811 for ; Thu, 30 Apr 2020 21:31:34 +0000 (UTC) X-FDA: 76765818108.21.loaf32_2ac46b6661c32 X-HE-Tag: loaf32_2ac46b6661c32 X-Filterd-Recvd-Size: 3942 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Thu, 30 Apr 2020 21:31:33 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 07C8820870; Thu, 30 Apr 2020 21:31:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588282292; bh=Bkl834xA9CcEf8hJwN2hSps66VvWfQ0obMpJUC2DhsI=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=n3v24GDLU0OAacwQ8azR4ZyAH5L3zSqPATNrbvEhpRTC9Q9/nuG17Q0Cck8wscapG Y1g5a4dTXh/93Ig5fF8vwETzF+pKorxdKNJqO2MqxYSJq/wQZy7YVk8PNlgT7rWxht JJU7Zj83UBZFTuWnzZGeHq02EE6uLPPSRoKVL7Qg= Date: Thu, 30 Apr 2020 14:31:31 -0700 From: Andrew Morton To: Daniel Jordan Cc: Herbert Xu , Steffen Klassert , Alex Williamson , Alexander Duyck , Dan Williams , Dave Hansen , David Hildenbrand , Jason Gunthorpe , Jonathan Corbet , Josh Triplett , Kirill Tkhai , Michal Hocko , Pavel Machek , Pavel Tatashin , Peter Zijlstra , Randy Dunlap , Shile Zhang , Tejun Heo , Zi Yan , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/7] padata: parallelize deferred page init Message-Id: <20200430143131.7b8ff07f022ed879305de82f@linux-foundation.org> In-Reply-To: <20200430201125.532129-1-daniel.m.jordan@oracle.com> References: <20200430201125.532129-1-daniel.m.jordan@oracle.com> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.31; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, 30 Apr 2020 16:11:18 -0400 Daniel Jordan wrote: > Sometimes the kernel doesn't take full advantage of system memory > bandwidth, leading to a single CPU spending excessive time in > initialization paths where the data scales with memory size. > > Multithreading naturally addresses this problem, and this series is the > first step. > > It extends padata, a framework that handles many parallel singlethreaded > jobs, to handle multithreaded jobs as well by adding support for > splitting up the work evenly, specifying a minimum amount of work that's > appropriate for one helper thread to do, load balancing between helpers, > and coordinating them. More documentation in patches 4 and 7. > > The first user is deferred struct page init, a large bottleneck in > kernel boot--actually the largest for us and likely others too. This > path doesn't require concurrency limits, resource control, or priority > adjustments like future users will (vfio, hugetlb fallocate, munmap) > because it happens during boot when the system is otherwise idle and > waiting on page init to finish. > > This has been tested on a variety of x86 systems and speeds up kernel > boot by 6% to 49% by making deferred init 63% to 91% faster. How long is this up-to-91% in seconds? If it's 91% of a millisecond then not impressed. If it's 91% of two weeks then better :) Relatedly, how important is boot time on these large machines anyway? They presumably have lengthy uptimes so boot time is relatively unimportant? IOW, can you please explain more fully why this patchset is valuable to our users?