From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 027C2C18E5A for ; Mon, 9 Mar 2020 12:26:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B8AE62072A for ; Mon, 9 Mar 2020 12:26:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B8AE62072A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4DA466B0003; Mon, 9 Mar 2020 08:26:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 48B106B0006; Mon, 9 Mar 2020 08:26:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3796A6B0007; Mon, 9 Mar 2020 08:26:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0211.hostedemail.com [216.40.44.211]) by kanga.kvack.org (Postfix) with ESMTP id 201BB6B0003 for ; Mon, 9 Mar 2020 08:26:50 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B7A39181AEF07 for ; Mon, 9 Mar 2020 12:26:49 +0000 (UTC) X-FDA: 76575747738.05.desk36_75c5e7237f903 X-HE-Tag: desk36_75c5e7237f903 X-Filterd-Recvd-Size: 6233 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Mon, 9 Mar 2020 12:26:49 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id v9so10819572wrf.10 for ; Mon, 09 Mar 2020 05:26:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=iKhaaNVti6SfxxTFgbT+erOBA/XhG2I7RA0RYVnVfIc=; b=mrpXhv8vsHlE0O1MamM1pg2oG9WBRUDIhh/qvswExfz1RJadhpxoiNYslHUHKVFtFZ DiuqixpZT/t4MAWI4fhASUQ0STo7C4r/vUUg+NC5WAawWINB8aK8wqx8vlHWw5EIDqYW 5lWKUTJHusWMTYODv2ATsVAY+Mt3isMeGxuz2z1eCjCtb92JKU19Xc0E0ZuAIvSXOsvt s9XJvlLnVOLeq3S6QcJhUISB/DjQE/LkqScljg1BD+XLHWzFHhR0L6ieNhV3QaCE7RFP QA4mV7ue0vW2q75tMd+5YhPbTg3tim3p2dUrWBDQ9lqsSkkCsdIAT3mbsUjO6wTCxh0M 6StQ== X-Gm-Message-State: ANhLgQ0RQMjhz2E2XXtbFIB+DLXpbi2w+F5PtvYZtH3d6n6BCk7wkldW LaAjyYPVC7Jnx+ei3UmcFxw= X-Google-Smtp-Source: ADFU+vtPwmhVB9+ErnryVhj01l+eUA6nQE5i1+BIr2Uuo9ns096ZP99v5nPot69toVGyNrdmnPm+8A== X-Received: by 2002:adf:ed06:: with SMTP id a6mr7689322wro.346.1583756808305; Mon, 09 Mar 2020 05:26:48 -0700 (PDT) Received: from localhost (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id t193sm597696wmt.14.2020.03.09.05.26.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Mar 2020 05:26:47 -0700 (PDT) Date: Mon, 9 Mar 2020 13:26:46 +0100 From: Michal Hocko To: "Kirill A. Shutemov" Cc: Cannon Matthews , Mike Kravetz , Andrew Morton , Matthew Wilcox , David Rientjes , Greg Thelen , Salman Qazi , linux-mm@kvack.org, linux-kernel@vger.kernel.org, ak@linux.intel.com, x86@kernel.org Subject: Re: [PATCH] mm: clear 1G pages with streaming stores on x86 Message-ID: <20200309122646.GM8447@dhcp22.suse.cz> References: <20200307010353.172991-1-cannonmatthews@google.com> <20200309000820.f37opzmppm67g6et@box> <20200309090630.GC8447@dhcp22.suse.cz> <20200309113658.bctbw35e73ahhgbu@box> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200309113658.bctbw35e73ahhgbu@box> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon 09-03-20 14:36:58, Kirill A. Shutemov wrote: > On Mon, Mar 09, 2020 at 10:06:30AM +0100, Michal Hocko wrote: > > On Mon 09-03-20 03:08:20, Kirill A. Shutemov wrote: > > > On Fri, Mar 06, 2020 at 05:03:53PM -0800, Cannon Matthews wrote: > > > > Reimplement clear_gigantic_page() to clear gigabytes pages using the > > > > non-temporal streaming store instructions that bypass the cache > > > > (movnti), since an entire 1GiB region will not fit in the cache anyway. > > > > > > > > Doing an mlock() on a 512GiB 1G-hugetlb region previously would take on > > > > average 134 seconds, about 260ms/GiB which is quite slow. Using `movnti` > > > > and optimizing the control flow over the constituent small pages, this > > > > can be improved roughly by a factor of 3-4x, with the 512GiB mlock() > > > > taking only 34 seconds on average, or 67ms/GiB. > > > > > > > > The assembly code for the __clear_page_nt routine is more or less > > > > taken directly from the output of gcc with -O3 for this function with > > > > some tweaks to support arbitrary sizes and moving memory barriers: > > > > > > > > void clear_page_nt_64i (void *page) > > > > { > > > > for (int i = 0; i < GiB /sizeof(long long int); ++i) > > > > { > > > > _mm_stream_si64 (((long long int*)page) + i, 0); > > > > } > > > > sfence(); > > > > } > > > > > > > > Tested: > > > > Time to `mlock()` a 512GiB region on broadwell CPU > > > > AVG time (s) % imp. ms/page > > > > clear_page_erms 133.584 - 261 > > > > clear_page_nt 34.154 74.43% 67 > > > > > > Some macrobenchmark would be great too. > > > > > > > An earlier version of this code was sent as an RFC patch ~July 2018 > > > > https://patchwork.kernel.org/patch/10543193/ but never merged. > > > > > > Andi and I tried to use MOVNTI for large/gigantic page clearing back in > > > 2012[1]. Maybe it can be useful. > > > > > > That patchset is somewhat more complex trying to keep the memory around > > > the fault address hot in cache. In theory it should help to reduce latency > > > on the first access to the memory. > > > > > > I was not able to get convincing numbers back then for the hardware of the > > > time. Maybe it's better now. > > > > > > https://lore.kernel.org/r/1345470757-12005-1-git-send-email-kirill.shutemov@linux.intel.com > > > > Thanks for the reminder. I've had only a very vague recollection. Your > > series had a much wider scope indeed. Since then we have gained > > process_huge_page which tries to optimize normal huge pages. > > > > Gigantic huge pages are a bit different. They are much less dynamic from > > the usage POV in my experience. Micro-optimizations for the first access > > tends to not matter at all as it is usually pre-allocation scenario. > > The page got cleared not on reservation, but on allocation, including page > fault time. Keeping the page around the fault address can still be > beneficial. You are right of course. What I meant to say that GB pages backed workloads I have seen tend to pre-allocate during the startup so they do not realy on lazy initialization duing #PF. This is slightly easier to handle for resource that is essentially impossible to get on-demand so an early failure is easier to handle. If there are workloads which can benefit from page fault microptimizations then all good but this can be done on top and demonstrate by numbers. It is much more easier to demonstrate the speed up on pre-initialization workloads. That's all I wanted to say here. -- Michal Hocko SUSE Labs