From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56009C4724C for ; Thu, 30 Apr 2020 21:41:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EE7EB206D9 for ; Thu, 30 Apr 2020 21:41:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="ejzlSjOS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE7EB206D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 716628E0005; Thu, 30 Apr 2020 17:41:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C5E98E0001; Thu, 30 Apr 2020 17:41:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5DC368E0005; Thu, 30 Apr 2020 17:41:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id 45E308E0001 for ; Thu, 30 Apr 2020 17:41:37 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 01ECB824934B for ; Thu, 30 Apr 2020 21:41:37 +0000 (UTC) X-FDA: 76765843434.05.nose26_827803cb92124 X-HE-Tag: nose26_827803cb92124 X-Filterd-Recvd-Size: 5508 Received: from mail-ed1-f66.google.com (mail-ed1-f66.google.com [209.85.208.66]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Thu, 30 Apr 2020 21:41:36 +0000 (UTC) Received: by mail-ed1-f66.google.com with SMTP id d16so5791229edq.7 for ; Thu, 30 Apr 2020 14:41:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hkpaJwE+XdImJwiRCGmMrHpnH9Do497AqONxguPuLh4=; b=ejzlSjOSQT9fOgnWL9BlmOywT9YUrObwmrAsjURuygWHxwV8OQVp72xlXJtZ02/TVw DIT/agnijUbVoAX7BK86VAyjiWzMyGXM2KTuZI6UNemsiX1Cq2rx0pT3HwreM1CaOKlk g255rxIrtopXIP8n5gl+jKtXRgHpDyi++7rWe+DxBDgBZCFAwnPQpsWVH19peAeY9FzY yaIFdv6+/raAyg7urd5JvKL/fqOUW4UB7EqADLBzSUJ+f8Aahl2Zz0zKvpxAEb37wx7V DTqNs130Y7jUegFidoOJ9vnmbx17b9H5MOSZ5HwfoY9hpRUnZWiWdAn+WzCJoM0ygalG kz4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hkpaJwE+XdImJwiRCGmMrHpnH9Do497AqONxguPuLh4=; b=SGnlfSE4nqRQ/IbxGEzHXvZys/fDApHjg3d0wOx7hezx+pDy1YVkhFsdyeHelSgrGF Gms6lFediY0GZ276lVySo97Go5dXpRHTSsgjGjXclfkG88V420y+z+i+vHu2tcJewQgx cl9OsVQ8eSrVZwmAiDSdumUw8vurzwxqAkSlzbmVbHmFe153mugyzpqmHVyuX3uzlZwy ra7+C0OcPPnFv72lR/9mSa6/jTi/gS2qhFJ4c00L9WAtC2i8JpQcptzDjSzsBh+5zMSR d2xNO6V+YWRCda7JSuK+cx5qwBy0TtSGTs/p5ArjirV0fRQ4MMLe2pFhu2UTeUSFhtxe pBZw== X-Gm-Message-State: AGi0Pub8XnRidel7kON2KYwDfuKVJ1bhIXsRs8SMKnRAGTh+JEMEx+qT oAAHGSpemd757P4pTDWyZ4AbT67wvDR0o9GDhxQkkA== X-Google-Smtp-Source: APiQypJ/NZIDe1nyo+ttBHz9E+jzzV+NreYGwdIKd9gp9L17jS4zp63ur9u6iJDTRLJh3/QP04vldfbKOZe8y5H1q9E= X-Received: by 2002:a50:c28a:: with SMTP id o10mr979777edf.85.1588282895115; Thu, 30 Apr 2020 14:41:35 -0700 (PDT) MIME-Version: 1.0 References: <20200430201125.532129-1-daniel.m.jordan@oracle.com> <20200430143131.7b8ff07f022ed879305de82f@linux-foundation.org> In-Reply-To: <20200430143131.7b8ff07f022ed879305de82f@linux-foundation.org> From: Pavel Tatashin Date: Thu, 30 Apr 2020 17:40:59 -0400 Message-ID: Subject: Re: [PATCH 0/7] padata: parallelize deferred page init To: Andrew Morton Cc: Daniel Jordan , Herbert Xu , Steffen Klassert , Alex Williamson , Alexander Duyck , Dan Williams , Dave Hansen , David Hildenbrand , Jason Gunthorpe , Jonathan Corbet , Josh Triplett , Kirill Tkhai , Michal Hocko , Pavel Machek , Peter Zijlstra , Randy Dunlap , Shile Zhang , Tejun Heo , Zi Yan , linux-crypto@vger.kernel.org, linux-mm , LKML Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Apr 30, 2020 at 5:31 PM Andrew Morton wrote: > > On Thu, 30 Apr 2020 16:11:18 -0400 Daniel Jordan wrote: > > > Sometimes the kernel doesn't take full advantage of system memory > > bandwidth, leading to a single CPU spending excessive time in > > initialization paths where the data scales with memory size. > > > > Multithreading naturally addresses this problem, and this series is the > > first step. > > > > It extends padata, a framework that handles many parallel singlethreaded > > jobs, to handle multithreaded jobs as well by adding support for > > splitting up the work evenly, specifying a minimum amount of work that's > > appropriate for one helper thread to do, load balancing between helpers, > > and coordinating them. More documentation in patches 4 and 7. > > > > The first user is deferred struct page init, a large bottleneck in > > kernel boot--actually the largest for us and likely others too. This > > path doesn't require concurrency limits, resource control, or priority > > adjustments like future users will (vfio, hugetlb fallocate, munmap) > > because it happens during boot when the system is otherwise idle and > > waiting on page init to finish. > > > > This has been tested on a variety of x86 systems and speeds up kernel > > boot by 6% to 49% by making deferred init 63% to 91% faster. > > How long is this up-to-91% in seconds? If it's 91% of a millisecond > then not impressed. If it's 91% of two weeks then better :) > > Relatedly, how important is boot time on these large machines anyway? > They presumably have lengthy uptimes so boot time is relatively > unimportant? Large machines indeed have a lengthy uptime, but they also can host a large number of VMs meaning that downtime of the host increases the downtime of VMs in cloud environments. Some VMs might be very sensible to downtime: game servers, traders, etc. > > IOW, can you please explain more fully why this patchset is valuable to > our users?