From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4B27C433EF for ; Mon, 4 Jul 2022 10:58:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233785AbiGDK6c (ORCPT ); Mon, 4 Jul 2022 06:58:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233601AbiGDK62 (ORCPT ); Mon, 4 Jul 2022 06:58:28 -0400 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2FE2B2F for ; Mon, 4 Jul 2022 03:58:26 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=guanghuifeng@linux.alibaba.com;NM=1;PH=DS;RN=21;SR=0;TI=SMTPD_---0VIKu7Cb_1656932300; Received: from 30.225.28.131(mailfrom:guanghuifeng@linux.alibaba.com fp:SMTPD_---0VIKu7Cb_1656932300) by smtp.aliyun-inc.com; Mon, 04 Jul 2022 18:58:21 +0800 Message-ID: <73f0c53b-fd17-c5e9-3773-1d71e564eb50@linux.alibaba.com> Date: Mon, 4 Jul 2022 18:58:20 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: [PATCH v4] arm64: mm: fix linear mem mapping access performance degradation To: Will Deacon Cc: baolin.wang@linux.alibaba.com, catalin.marinas@arm.com, akpm@linux-foundation.org, david@redhat.com, jianyong.wu@arm.com, james.morse@arm.com, quic_qiancai@quicinc.com, christophe.leroy@csgroup.eu, jonathan@marek.ca, mark.rutland@arm.com, thunder.leizhen@huawei.com, anshuman.khandual@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, rppt@kernel.org, geert+renesas@glider.be, ardb@kernel.org, linux-mm@kvack.org, yaohongbo@linux.alibaba.com, alikernel-developer@linux.alibaba.com References: <1656777473-73887-1-git-send-email-guanghuifeng@linux.alibaba.com> <20220704103523.GC31437@willie-the-truck> From: "guanghui.fgh" In-Reply-To: <20220704103523.GC31437@willie-the-truck> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 在 2022/7/4 18:35, Will Deacon 写道: > On Sat, Jul 02, 2022 at 11:57:53PM +0800, Guanghui Feng wrote: >> The arm64 can build 2M/1G block/sectiion mapping. When using DMA/DMA32 zone >> (enable crashkernel, disable rodata full, disable kfence), the mem_map will >> use non block/section mapping(for crashkernel requires to shrink the region >> in page granularity). But it will degrade performance when doing larging >> continuous mem access in kernel(memcpy/memmove, etc). > > Hmm. It seems a bit silly to me that we take special care to unmap the > crashkernel from the linear map even when can_set_direct_map() is false, as > we won't be protecting the main kernel at all! > > Why don't we just leave the crashkernel mapped if !can_set_direct_map() > and then this problem just goes away? > > Will This question had been asked lask week. 1.Quoted messages from arch/arm64/mm/init.c "Memory reservation for crash kernel either done early or deferred depending on DMA memory zones configs (ZONE_DMA) -- In absence of ZONE_DMA configs arm64_dma_phys_limit initialized here instead of max_zone_phys(). This lets early reservation of crash kernel memory which has a dependency on arm64_dma_phys_limit. Reserving memory early for crash kernel allows linear creation of block mappings (greater than page-granularity) for all the memory bank rangs. In this scheme a comparatively quicker boot is observed. If ZONE_DMA configs are defined, crash kernel memory reservation is delayed until DMA zone memory range size initialization performed in zone_sizes_init(). The defer is necessary to steer clear of DMA zone memory range to avoid overlap allocation. [[[ So crash kernel memory boundaries are not known when mapping all bank memory ranges, which otherwise means not possible to exclude crash kernel range from creating block mappings so page-granularity mappings are created for the entire memory range. ]]]" Namely, the init order: memblock init--->linear mem mapping(4k mapping for crashkernel, requirinig page-granularity changing))--->zone dma limit--->reserve crashkernel. So when enable ZONE DMA and using crashkernel, the mem mapping using 4k mapping. 2.As mentioned above, when linear mem use 4k mapping simply, there is high dtlb miss(degrade performance). This patch use block/section mapping as far as possible with performance improvement. 3.This patch reserve crashkernel as same as the history(ZONE DMA & crashkernel reserving order), and only change the linear mem mapping to block/section mapping. Init order: memblock init--->linear mem mapping(block/section mapping for linear mem mapping))--->zone dma limit--->reserve crashkernel--->[[[only]]] rebuild 4k pagesize mapping for crashkernel mem With this method, there will use block/section mapping as far as possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 472EAC433EF for ; Mon, 4 Jul 2022 11:31:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vNefVNFkKeQPjCyl18ADXTpnlzRtf0HyUQL2uf3PXsY=; b=g2Hn872+BSJ05T I6kcQJ/q9pFvjcqN8DGD3JGyPNTVxHOY0zxhwHvtzAMTA2vPfrYFWjYlT5MtLv8PbTvb6mAnlTSar 14JZTCur18CaFyr+3TIoJhiGYjC3QxORH21SiCNf9S+OyRVhbzUe5Caci2D8n69uTgvZTNTYdsciE YVP8cejIcUAdEL9WUw6PrVp7Umpt/GdSIn+nKrAilfnW8HoFQAdHCbO8v5Ysb3R5qFZpHHD41tKb1 vx88wHXzfNfVkCq+BmAY5GTO7734jWHJdDPV2f3VG5rUI5YlJY/axiPs6jm22pxe2V6q3RtvMAqud s0+tY2GQnd07JhSB3EUg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o8KHM-007l7O-Cj; Mon, 04 Jul 2022 11:30:12 +0000 Received: from out199-4.us.a.mail.aliyun.com ([47.90.199.4]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o8Jmi-007LNV-8q for linux-arm-kernel@lists.infradead.org; Mon, 04 Jul 2022 10:58:35 +0000 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=guanghuifeng@linux.alibaba.com;NM=1;PH=DS;RN=21;SR=0;TI=SMTPD_---0VIKu7Cb_1656932300; Received: from 30.225.28.131(mailfrom:guanghuifeng@linux.alibaba.com fp:SMTPD_---0VIKu7Cb_1656932300) by smtp.aliyun-inc.com; Mon, 04 Jul 2022 18:58:21 +0800 Message-ID: <73f0c53b-fd17-c5e9-3773-1d71e564eb50@linux.alibaba.com> Date: Mon, 4 Jul 2022 18:58:20 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: [PATCH v4] arm64: mm: fix linear mem mapping access performance degradation To: Will Deacon Cc: baolin.wang@linux.alibaba.com, catalin.marinas@arm.com, akpm@linux-foundation.org, david@redhat.com, jianyong.wu@arm.com, james.morse@arm.com, quic_qiancai@quicinc.com, christophe.leroy@csgroup.eu, jonathan@marek.ca, mark.rutland@arm.com, thunder.leizhen@huawei.com, anshuman.khandual@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, rppt@kernel.org, geert+renesas@glider.be, ardb@kernel.org, linux-mm@kvack.org, yaohongbo@linux.alibaba.com, alikernel-developer@linux.alibaba.com References: <1656777473-73887-1-git-send-email-guanghuifeng@linux.alibaba.com> <20220704103523.GC31437@willie-the-truck> From: "guanghui.fgh" In-Reply-To: <20220704103523.GC31437@willie-the-truck> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220704_035832_660733_61CECFB3 X-CRM114-Status: GOOD ( 12.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org CgrlnKggMjAyMi83LzQgMTg6MzUsIFdpbGwgRGVhY29uIOWGmemBkzoKPiBPbiBTYXQsIEp1bCAw MiwgMjAyMiBhdCAxMTo1Nzo1M1BNICswODAwLCBHdWFuZ2h1aSBGZW5nIHdyb3RlOgo+PiBUaGUg YXJtNjQgY2FuIGJ1aWxkIDJNLzFHIGJsb2NrL3NlY3RpaW9uIG1hcHBpbmcuIFdoZW4gdXNpbmcg RE1BL0RNQTMyIHpvbmUKPj4gKGVuYWJsZSBjcmFzaGtlcm5lbCwgZGlzYWJsZSByb2RhdGEgZnVs bCwgZGlzYWJsZSBrZmVuY2UpLCB0aGUgbWVtX21hcCB3aWxsCj4+IHVzZSBub24gYmxvY2svc2Vj dGlvbiBtYXBwaW5nKGZvciBjcmFzaGtlcm5lbCByZXF1aXJlcyB0byBzaHJpbmsgdGhlIHJlZ2lv bgo+PiBpbiBwYWdlIGdyYW51bGFyaXR5KS4gQnV0IGl0IHdpbGwgZGVncmFkZSBwZXJmb3JtYW5j ZSB3aGVuIGRvaW5nIGxhcmdpbmcKPj4gY29udGludW91cyBtZW0gYWNjZXNzIGluIGtlcm5lbCht ZW1jcHkvbWVtbW92ZSwgZXRjKS4KPiAKPiBIbW0uIEl0IHNlZW1zIGEgYml0IHNpbGx5IHRvIG1l IHRoYXQgd2UgdGFrZSBzcGVjaWFsIGNhcmUgdG8gdW5tYXAgdGhlCj4gY3Jhc2hrZXJuZWwgZnJv bSB0aGUgbGluZWFyIG1hcCBldmVuIHdoZW4gY2FuX3NldF9kaXJlY3RfbWFwKCkgaXMgZmFsc2Us IGFzCj4gd2Ugd29uJ3QgYmUgcHJvdGVjdGluZyB0aGUgbWFpbiBrZXJuZWwgYXQgYWxsIQo+IAo+ IFdoeSBkb24ndCB3ZSBqdXN0IGxlYXZlIHRoZSBjcmFzaGtlcm5lbCBtYXBwZWQgaWYgIWNhbl9z ZXRfZGlyZWN0X21hcCgpCj4gYW5kIHRoZW4gdGhpcyBwcm9ibGVtIGp1c3QgZ29lcyBhd2F5Pwo+ IAo+IFdpbGwKClRoaXMgcXVlc3Rpb24gaGFkIGJlZW4gYXNrZWQgbGFzayB3ZWVrLgoKCjEuUXVv dGVkIG1lc3NhZ2VzIGZyb20gYXJjaC9hcm02NC9tbS9pbml0LmMKCiJNZW1vcnkgcmVzZXJ2YXRp b24gZm9yIGNyYXNoIGtlcm5lbCBlaXRoZXIgZG9uZSBlYXJseSBvciBkZWZlcnJlZApkZXBlbmRp bmcgb24gRE1BIG1lbW9yeSB6b25lcyBjb25maWdzIChaT05FX0RNQSkgLS0KCkluIGFic2VuY2Ug b2YgWk9ORV9ETUEgY29uZmlncyBhcm02NF9kbWFfcGh5c19saW1pdCBpbml0aWFsaXplZApoZXJl IGluc3RlYWQgb2YgbWF4X3pvbmVfcGh5cygpLiAgVGhpcyBsZXRzIGVhcmx5IHJlc2VydmF0aW9u IG9mCmNyYXNoIGtlcm5lbCBtZW1vcnkgd2hpY2ggaGFzIGEgZGVwZW5kZW5jeSBvbiBhcm02NF9k bWFfcGh5c19saW1pdC4KUmVzZXJ2aW5nIG1lbW9yeSBlYXJseSBmb3IgY3Jhc2gga2VybmVsIGFs bG93cyBsaW5lYXIgY3JlYXRpb24gb2YgYmxvY2sKbWFwcGluZ3MgKGdyZWF0ZXIgdGhhbiBwYWdl LWdyYW51bGFyaXR5KSBmb3IgYWxsIHRoZSBtZW1vcnkgYmFuayByYW5ncy4KSW4gdGhpcyBzY2hl bWUgYSBjb21wYXJhdGl2ZWx5IHF1aWNrZXIgYm9vdCBpcyBvYnNlcnZlZC4KCklmIFpPTkVfRE1B IGNvbmZpZ3MgYXJlIGRlZmluZWQsIGNyYXNoIGtlcm5lbCBtZW1vcnkgcmVzZXJ2YXRpb24KaXMg ZGVsYXllZCB1bnRpbCBETUEgem9uZSBtZW1vcnkgcmFuZ2Ugc2l6ZSBpbml0aWFsaXphdGlvbiBw ZXJmb3JtZWQgaW4Kem9uZV9zaXplc19pbml0KCkuICBUaGUgZGVmZXIgaXMgbmVjZXNzYXJ5IHRv IHN0ZWVyIGNsZWFyIG9mIERNQSB6b25lCm1lbW9yeSByYW5nZSB0byBhdm9pZCBvdmVybGFwIGFs bG9jYXRpb24uCgpbW1sKU28gY3Jhc2gga2VybmVsIG1lbW9yeSBib3VuZGFyaWVzIGFyZSBub3Qg a25vd24gd2hlbiBtYXBwaW5nIGFsbCBiYW5rIAptZW1vcnkgcmFuZ2VzLCB3aGljaCBvdGhlcndp c2UgbWVhbnMgbm90IHBvc3NpYmxlIHRvIGV4Y2x1ZGUgY3Jhc2ggCmtlcm5lbCByYW5nZSBmcm9t IGNyZWF0aW5nIGJsb2NrIG1hcHBpbmdzIHNvIHBhZ2UtZ3JhbnVsYXJpdHkgbWFwcGluZ3MgCmFy ZSBjcmVhdGVkIGZvciB0aGUgZW50aXJlIG1lbW9yeSByYW5nZS4KXV1dIgoKTmFtZWx5LCB0aGUg aW5pdCBvcmRlcjogbWVtYmxvY2sgaW5pdC0tLT5saW5lYXIgbWVtIG1hcHBpbmcoNGsgbWFwcGlu ZyAKZm9yIGNyYXNoa2VybmVsLCByZXF1aXJpbmlnIHBhZ2UtZ3JhbnVsYXJpdHkgY2hhbmdpbmcp KS0tLT56b25lIGRtYSAKbGltaXQtLS0+cmVzZXJ2ZSBjcmFzaGtlcm5lbC4KU28gd2hlbiBlbmFi bGUgWk9ORSBETUEgYW5kIHVzaW5nIGNyYXNoa2VybmVsLCB0aGUgbWVtIG1hcHBpbmcgdXNpbmcg NGsgCm1hcHBpbmcuCgoKMi5BcyBtZW50aW9uZWQgYWJvdmUsIHdoZW4gbGluZWFyIG1lbSB1c2Ug NGsgbWFwcGluZyBzaW1wbHksIHRoZXJlIGlzIApoaWdoIGR0bGIgbWlzcyhkZWdyYWRlIHBlcmZv cm1hbmNlKS4KVGhpcyBwYXRjaCB1c2UgYmxvY2svc2VjdGlvbiBtYXBwaW5nIGFzIGZhciBhcyBw b3NzaWJsZSB3aXRoIHBlcmZvcm1hbmNlIAppbXByb3ZlbWVudC4KCjMuVGhpcyBwYXRjaCByZXNl cnZlIGNyYXNoa2VybmVsIGFzIHNhbWUgYXMgdGhlIGhpc3RvcnkoWk9ORSBETUEgJiAKY3Jhc2hr ZXJuZWwgcmVzZXJ2aW5nIG9yZGVyKSwgYW5kIG9ubHkgY2hhbmdlIHRoZSBsaW5lYXIgbWVtIG1h cHBpbmcgdG8gCmJsb2NrL3NlY3Rpb24gbWFwcGluZy4KSW5pdCBvcmRlcjogbWVtYmxvY2sgaW5p dC0tLT5saW5lYXIgbWVtIG1hcHBpbmcoYmxvY2svc2VjdGlvbiBtYXBwaW5nIApmb3IgbGluZWFy IG1lbSBtYXBwaW5nKSktLS0+em9uZSBkbWEgbGltaXQtLS0+cmVzZXJ2ZSAKY3Jhc2hrZXJuZWwt LS0+W1tbb25seV1dXSByZWJ1aWxkIDRrIHBhZ2VzaXplIG1hcHBpbmcgZm9yIGNyYXNoa2VybmVs IG1lbQoKV2l0aCB0aGlzIG1ldGhvZCwgdGhlcmUgd2lsbCB1c2UgYmxvY2svc2VjdGlvbiBtYXBw aW5nIGFzIGZhciBhcyBwb3NzaWJsZS4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1r ZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWls bWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK