From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DF85C433E0 for ; Thu, 28 Jan 2021 10:55:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8A4986146D for ; Thu, 28 Jan 2021 10:55:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8A4986146D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=fb.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0689C6B0005; Thu, 28 Jan 2021 05:55:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 01B056B0006; Thu, 28 Jan 2021 05:55:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4CC36B006C; Thu, 28 Jan 2021 05:55:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0132.hostedemail.com [216.40.44.132]) by kanga.kvack.org (Postfix) with ESMTP id CBEE96B0005 for ; Thu, 28 Jan 2021 05:55:43 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 910061EE6 for ; Thu, 28 Jan 2021 10:55:43 +0000 (UTC) X-FDA: 77754878166.19.vase50_620da9c2759f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 69D361AD1B2 for ; Thu, 28 Jan 2021 10:55:43 +0000 (UTC) X-HE-Tag: vase50_620da9c2759f X-Filterd-Recvd-Size: 8942 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Thu, 28 Jan 2021 10:55:42 +0000 (UTC) Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 10SArwRn024395 for ; Thu, 28 Jan 2021 02:55:41 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=A2Tds/vdqZp0W528lsjA2sQmhnxQty6Eo+Ky3FKtWtE=; b=HfMwLRLfJQXySddE9kQAHvFqP7pZ0fA0N3UBffzRdXABjXNeY873msBdbmHikuwvzsOa jWpgpj/O2ZAi1WJpibUw0PBj4JytMCplibjJ+uabt17YJltlUyAU0A6rJGYEVHRjw7Fw gTRdT1W/GF3lEnJmjbpk0PVoYenJHyaz5gs= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 36awcprbp0-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 28 Jan 2021 02:55:41 -0800 Received: from intmgw001.05.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::e) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1979.3; Thu, 28 Jan 2021 02:55:39 -0800 Received: from devvm1945.atn0.facebook.com (localhost [127.0.0.1]) by devvm1945.atn0.facebook.com (Postfix) with ESMTP id 640EC2598E84; Thu, 28 Jan 2021 02:49:40 -0800 (PST) Received: (from saravanand@localhost) by devvm1945.atn0.facebook.com (8.15.2/8.15.2/Submit) id 10SAnbaX2916913; Thu, 28 Jan 2021 02:49:37 -0800 X-Authentication-Warning: devvm1945.atn0.facebook.com: saravanand set sender to saravanand@fb.com using -f From: Saravanan D To: , , , , , CC: , , , , , Saravanan D Subject: [PATCH V5] x86/mm: Tracking linear mapping split events Date: Thu, 28 Jan 2021 02:49:34 -0800 Message-ID: <20210128104934.2916679-1-saravanand@fb.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20210128045153.GW308988@casper.infradead.org> References: <20210128045153.GW308988@casper.infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-FB-Internal: Safe Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343,18.0.737 definitions=2021-01-28_05:2021-01-27,2021-01-28 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 malwarescore=0 phishscore=0 bulkscore=0 mlxlogscore=999 priorityscore=1501 spamscore=0 impostorscore=0 lowpriorityscore=0 mlxscore=0 adultscore=0 suspectscore=0 clxscore=1011 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101280054 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To help with debugging the sluggishness caused by TLB miss/reload, we introduce monotonic lifetime hugepage split event counts since system state: SYSTEM_RUNNING to be displayed as part of /proc/vmstat in x86 servers The lifetime split event information will be displayed at the bottom of /proc/vmstat .... swap_ra 0 swap_ra_hit 0 direct_map_level2_splits 94 direct_map_level3_splits 4 nr_unstable 0 .... One of the many lasting (as we don't coalesce back) sources for huge page splits is tracing as the granular page attribute/permission changes would force the kernel to split code segments mapped to huge pages to smaller ones thereby increasing the probability of TLB miss/reload even after tracing has been stopped. Documentation regarding linear mapping split events added to admin-guide as requested in V3 of the patch. Signed-off-by: Saravanan D --- .../admin-guide/mm/direct_mapping_splits.rst | 59 +++++++++++++++++++ Documentation/admin-guide/mm/index.rst | 1 + arch/x86/mm/pat/set_memory.c | 8 +++ include/linux/vm_event_item.h | 4 ++ mm/vmstat.c | 4 ++ 5 files changed, 76 insertions(+) create mode 100644 Documentation/admin-guide/mm/direct_mapping_splits.rs= t diff --git a/Documentation/admin-guide/mm/direct_mapping_splits.rst b/Doc= umentation/admin-guide/mm/direct_mapping_splits.rst new file mode 100644 index 000000000000..298751391deb --- /dev/null +++ b/Documentation/admin-guide/mm/direct_mapping_splits.rst @@ -0,0 +1,59 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Direct Mapping Splits +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Kernel maps all of physical memory in linear/direct mapped pages with +translation of virtual kernel address to physical address is achieved +through a simple subtraction of offset. CPUs maintain a cache of these +translations on fast caches called TLBs. CPU architectures like x86 allo= w +direct mapping large portions of memory into hugepages (2M, 1G, etc) in +various page table levels. + +Maintaining huge direct mapped pages greatly reduces TLB miss pressure. +The splintering of huge direct pages into smaller ones does result in +a measurable performance hit caused by frequent TLB miss and reloads. + +One of the many lasting (as we don't coalesce back) sources for huge pag= e +splits is tracing as the granular page attribute/permission changes woul= d +force the kernel to split code segments mapped to hugepages to smaller +ones thus increasing the probability of TLB miss/reloads even after +tracing has been stopped. + +On x86 systems, we can track the splitting of huge direct mapped pages +through lifetime event counters in ``/proc/vmstat`` + + direct_map_level2_splits xxx + direct_map_level3_splits yyy + +where: + +direct_map_level2_splits + are 2M/4M hugepage split events +direct_map_level3_splits + are 1G hugepage split events + +The distribution of direct mapped system memory in various page sizes +post splits can be viewed through ``/proc/meminfo`` whose output +will include the following lines depending upon supporting CPU +architecture + + DirectMap4k: xxxxx kB + DirectMap2M: yyyyy kB + DirectMap1G: zzzzz kB + +where: + +DirectMap4k + is the total amount of direct mapped memory (in kB) + accessed through 4k pages +DirectMap2M + is the total amount of direct mapped memory (in kB) + accessed through 2M pages +DirectMap1G + is the total amount of direct mapped memory (in kB) + accessed through 1G pages + + +-- Saravanan D, Jan 27, 2021 diff --git a/Documentation/admin-guide/mm/index.rst b/Documentation/admin= -guide/mm/index.rst index 4b14d8b50e9e..9439780f3f07 100644 --- a/Documentation/admin-guide/mm/index.rst +++ b/Documentation/admin-guide/mm/index.rst @@ -38,3 +38,4 @@ the Linux memory management. soft-dirty transhuge userfaultfd + direct_mapping_splits diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 16f878c26667..a7b3c5f1d316 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -16,6 +16,8 @@ #include #include #include +#include +#include =20 #include #include @@ -91,6 +93,12 @@ static void split_page_count(int level) return; =20 direct_pages_count[level]--; + if (system_state =3D=3D SYSTEM_RUNNING) { + if (level =3D=3D PG_LEVEL_2M) + count_vm_event(DIRECT_MAP_LEVEL2_SPLIT); + else if (level =3D=3D PG_LEVEL_1G) + count_vm_event(DIRECT_MAP_LEVEL3_SPLIT); + } direct_pages_count[level - 1] +=3D PTRS_PER_PTE; } =20 diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.= h index 18e75974d4e3..7c06c2bdc33b 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -120,6 +120,10 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOU= T, #ifdef CONFIG_SWAP SWAP_RA, SWAP_RA_HIT, +#endif +#ifdef CONFIG_X86 + DIRECT_MAP_LEVEL2_SPLIT, + DIRECT_MAP_LEVEL3_SPLIT, #endif NR_VM_EVENT_ITEMS }; diff --git a/mm/vmstat.c b/mm/vmstat.c index f8942160fc95..a43ac4ac98a2 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1350,6 +1350,10 @@ const char * const vmstat_text[] =3D { "swap_ra", "swap_ra_hit", #endif +#ifdef CONFIG_X86 + "direct_map_level2_splits", + "direct_map_level3_splits", +#endif #endif /* CONFIG_VM_EVENT_COUNTERS || CONFIG_MEMCG */ }; #endif /* CONFIG_PROC_FS || CONFIG_SYSFS || CONFIG_NUMA || CONFIG_MEMCG = */ --=20 2.24.1