From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 772C1C433DB for ; Fri, 15 Jan 2021 07:23:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E3464221FE for ; Fri, 15 Jan 2021 07:23:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E3464221FE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2C53B8D0062; Fri, 15 Jan 2021 02:23:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 29B6A8D0023; Fri, 15 Jan 2021 02:23:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B2C38D0062; Fri, 15 Jan 2021 02:23:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0218.hostedemail.com [216.40.44.218]) by kanga.kvack.org (Postfix) with ESMTP id 050988D0023 for ; Fri, 15 Jan 2021 02:23:20 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C33333638 for ; Fri, 15 Jan 2021 07:23:19 +0000 (UTC) X-FDA: 77707168518.05.ghost01_0c01a702752d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id A6B59180FDDAC for ; Fri, 15 Jan 2021 07:23:19 +0000 (UTC) X-HE-Tag: ghost01_0c01a702752d X-Filterd-Recvd-Size: 4045 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Fri, 15 Jan 2021 07:23:18 +0000 (UTC) IronPort-SDR: YHKBw4gMoaYHGalkq0C85ZbiEDu+3G0vRpsxJp6ODzDyzsp9l6eBw0+fEYaKnXUCegO3NL3Jbo IFpZtjvolkkQ== X-IronPort-AV: E=McAfee;i="6000,8403,9864"; a="157690005" X-IronPort-AV: E=Sophos;i="5.79,348,1602572400"; d="scan'208";a="157690005" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jan 2021 23:23:11 -0800 IronPort-SDR: rI+g70nMjeju82zmMHGK73x/doUFYZSAe2vGJuYhomHwkIvTWuVo0O7Ct7zsxNcIwXaSIhdfNP DZ8veDKLVzNw== X-IronPort-AV: E=Sophos;i="5.79,348,1602572400"; d="scan'208";a="382568970" Received: from xingzhen-mobl.ccr.corp.intel.com (HELO [10.255.30.191]) ([10.255.30.191]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jan 2021 23:23:10 -0800 To: linux-mm@kvack.org, LKML Cc: Dave Hansen , Tony , Tim C Chen , "Huang, Ying" , "Du, Julie" From: Xing Zhengjun Subject: Test report for kernel direct mapping performance Message-ID: <213b4567-46ce-f116-9cdf-bbd0c884eb3c@linux.intel.com> Date: Fri, 15 Jan 2021 15:23:07 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.011718, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, There is currently a bit of a debate about the kernel direct map. Does using 2M/1G pages aggressively for the kernel direct map help performance? Or, is it an old optimization which is not as helpful on modern CPUs as it was in the old days? What is the penalty of a kernel feature that heavily demotes this mapping from larger to smaller pages? We did a set of runs with 1G and 2M pages enabled /disabled and saw the changes. [Conclusions] Assuming that this was a good representative set of workloads and that the data are good, for server usage, we conclude that the existing aggressive use of 1G mappings is a good choice since it represents the best in a plurality of the workloads. However, in a *majority* of cases, another mapping size (2M or 4k) potentially offers a performance improvement. This leads us to conclude that although 1G mappings are a good default choice, there is no compelling evidence that it must be the only choice, or that folks deriving benefits (like hardening) from smaller mapping sizes should avoid the smaller mapping sizes. [Summary of results] 1. The test was done on server platforms with 11 benchmarks. For the 4 different server platforms tested, each with three different maximums kernel mapping sizes: 4k, 2M, and 1G. Each system has enough memory to effectively deploy 1G mappings. For the 11 different benchmarks were used, not every benchmark was run on every system, there was a total of 259 tests. 2. For each benchmark/system combination, the 1G mapping had the highest performance for 45% of the tests, 2M for ~30%, and 4k for~20%. 3. From the average delta, among 1G/2M/4K, 4K gets the lowest performance in all the 4 test machines, while 1G gets the best performance on 2 test machines and 2M gets the best performance on the other 2 machines. 4. By testing with machine memory from 256G to 512G, we observed that the larger memory will lead to the performance better for 1G page size. With Large memory, Will-it-scale/vm-scalability/unixbench/reaim/hackbench shows 1G has the best performance, while kbuild/memtier/netperf shows 4K has the best performance. For more details please see the following web link: https://01.org/sites/default/files/documentation/test_report_for_kernel_direct_mapping_performance_0.pdf -- Zhengjun Xing