From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D71AC43334 for ; Mon, 4 Jul 2022 11:14:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234202AbiGDLOQ (ORCPT ); Mon, 4 Jul 2022 07:14:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231339AbiGDLOP (ORCPT ); Mon, 4 Jul 2022 07:14:15 -0400 Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0237CFD39 for ; Mon, 4 Jul 2022 04:14:14 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id B5369CE13FD for ; Mon, 4 Jul 2022 11:14:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 658D0C341CA; Mon, 4 Jul 2022 11:14:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656933250; bh=klOYLUi5vKbP7lLXPlg3VywRiC6zi5lv+GFCDUacxQ4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=nQ9jXCF2NYWAAv8fe8XVgTNnCLyXBOTaZWTiQOC4cwuzd8Q4vJ+UD0H1EWlmZn+Ck VZ1wdXcXsKfGvIK1NX5keAaCcxU4FLZpumoeUJBb7WH6YrLlnHl8EY1mpQyF9ymn1h M/OSILsEjYJy8wuNBLxNqxKwyKPchvfsX6Ig/fkSgAZlOP05GKWJ+jnYUcq1L5s3tq cRBMp667H+23Pqt4EEsMI/BauNcg8EXzDUFTz+CPFQu/7wCimx6HCDkxqM68WbMTQd mSCMXzb3LHG8BBlOVcU9ctEl4BmOZdAfsYOehzZvl1RbLv+oG4GkSQo6RdGsMCzm6L sm68TYRAf8/Dw== Date: Mon, 4 Jul 2022 12:14:02 +0100 From: Will Deacon To: "guanghui.fgh" Cc: baolin.wang@linux.alibaba.com, catalin.marinas@arm.com, akpm@linux-foundation.org, david@redhat.com, jianyong.wu@arm.com, james.morse@arm.com, quic_qiancai@quicinc.com, christophe.leroy@csgroup.eu, jonathan@marek.ca, mark.rutland@arm.com, thunder.leizhen@huawei.com, anshuman.khandual@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, rppt@kernel.org, geert+renesas@glider.be, ardb@kernel.org, linux-mm@kvack.org, yaohongbo@linux.alibaba.com, alikernel-developer@linux.alibaba.com Subject: Re: [PATCH v4] arm64: mm: fix linear mem mapping access performance degradation Message-ID: <20220704111402.GA31553@willie-the-truck> References: <1656777473-73887-1-git-send-email-guanghuifeng@linux.alibaba.com> <20220704103523.GC31437@willie-the-truck> <73f0c53b-fd17-c5e9-3773-1d71e564eb50@linux.alibaba.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <73f0c53b-fd17-c5e9-3773-1d71e564eb50@linux.alibaba.com> User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Jul 04, 2022 at 06:58:20PM +0800, guanghui.fgh wrote: > > > 在 2022/7/4 18:35, Will Deacon 写道: > > On Sat, Jul 02, 2022 at 11:57:53PM +0800, Guanghui Feng wrote: > > > The arm64 can build 2M/1G block/sectiion mapping. When using DMA/DMA32 zone > > > (enable crashkernel, disable rodata full, disable kfence), the mem_map will > > > use non block/section mapping(for crashkernel requires to shrink the region > > > in page granularity). But it will degrade performance when doing larging > > > continuous mem access in kernel(memcpy/memmove, etc). > > > > Hmm. It seems a bit silly to me that we take special care to unmap the > > crashkernel from the linear map even when can_set_direct_map() is false, as > > we won't be protecting the main kernel at all! > > > > Why don't we just leave the crashkernel mapped if !can_set_direct_map() > > and then this problem just goes away? > > > > Will > > This question had been asked lask week. Sorry, I didn't spot that. Please could you link me to the conversation, as I'm still unable to find it in my inbox? > 1.Quoted messages from arch/arm64/mm/init.c > > "Memory reservation for crash kernel either done early or deferred > depending on DMA memory zones configs (ZONE_DMA) -- > > In absence of ZONE_DMA configs arm64_dma_phys_limit initialized > here instead of max_zone_phys(). This lets early reservation of > crash kernel memory which has a dependency on arm64_dma_phys_limit. > Reserving memory early for crash kernel allows linear creation of block > mappings (greater than page-granularity) for all the memory bank rangs. > In this scheme a comparatively quicker boot is observed. > > If ZONE_DMA configs are defined, crash kernel memory reservation > is delayed until DMA zone memory range size initialization performed in > zone_sizes_init(). The defer is necessary to steer clear of DMA zone > memory range to avoid overlap allocation. > > [[[ > So crash kernel memory boundaries are not known when mapping all bank memory > ranges, which otherwise means not possible to exclude crash kernel range > from creating block mappings so page-granularity mappings are created for > the entire memory range. > ]]]" > > Namely, the init order: memblock init--->linear mem mapping(4k mapping for > crashkernel, requirinig page-granularity changing))--->zone dma > limit--->reserve crashkernel. > So when enable ZONE DMA and using crashkernel, the mem mapping using 4k > mapping. Yes, I understand that is how things work today but I'm saying that we may as well leave the crashkernel mapped (at block granularity) if !can_set_direct_map() and then I think your patch becomes a lot simpler. Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B67AC43334 for ; Mon, 4 Jul 2022 11:37:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2j4ZEvlrjMP6PB533/xAENcFVK1LdRMkMPfxBtfWuuc=; b=MBOZiUgvxcLYFj MRYa4YxjdJzeXNDa9KCDNGJT2j2+vNKhwHmGhVS/GErsGqvUMI8VQBzdr0UCsnnj9w1XJa/xtAD9Z pcg6qUE0+q/XrnRe+wRTS6yuWqZKHrg7f2AcA6dOQPm+z1RnL3JGH5NK/9NSnuXMJApcombHG5R+H ZLwTZSZ05TEUfeBtTvPBTQvggTtOsDQUoFRmcn0Q/qKaXJgk7gfysaDHrv5rMPXS8nqgEYfX2DcDW AsEfq2FzEA17af/0b1/yqLjJUAB7MujTvGbaAnVMbLrAnTVf+Lu/pMqLxwunXPiwTlX57hGXuLUtT jzPV9Ftzs49rbOmcbXBQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o8KNR-007qth-30; Mon, 04 Jul 2022 11:36:29 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o8K1x-007Ya3-D3 for linux-arm-kernel@lists.infradead.org; Mon, 04 Jul 2022 11:14:19 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 66E5FB80E88; Mon, 4 Jul 2022 11:14:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 658D0C341CA; Mon, 4 Jul 2022 11:14:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656933250; bh=klOYLUi5vKbP7lLXPlg3VywRiC6zi5lv+GFCDUacxQ4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=nQ9jXCF2NYWAAv8fe8XVgTNnCLyXBOTaZWTiQOC4cwuzd8Q4vJ+UD0H1EWlmZn+Ck VZ1wdXcXsKfGvIK1NX5keAaCcxU4FLZpumoeUJBb7WH6YrLlnHl8EY1mpQyF9ymn1h M/OSILsEjYJy8wuNBLxNqxKwyKPchvfsX6Ig/fkSgAZlOP05GKWJ+jnYUcq1L5s3tq cRBMp667H+23Pqt4EEsMI/BauNcg8EXzDUFTz+CPFQu/7wCimx6HCDkxqM68WbMTQd mSCMXzb3LHG8BBlOVcU9ctEl4BmOZdAfsYOehzZvl1RbLv+oG4GkSQo6RdGsMCzm6L sm68TYRAf8/Dw== Date: Mon, 4 Jul 2022 12:14:02 +0100 From: Will Deacon To: "guanghui.fgh" Cc: baolin.wang@linux.alibaba.com, catalin.marinas@arm.com, akpm@linux-foundation.org, david@redhat.com, jianyong.wu@arm.com, james.morse@arm.com, quic_qiancai@quicinc.com, christophe.leroy@csgroup.eu, jonathan@marek.ca, mark.rutland@arm.com, thunder.leizhen@huawei.com, anshuman.khandual@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, rppt@kernel.org, geert+renesas@glider.be, ardb@kernel.org, linux-mm@kvack.org, yaohongbo@linux.alibaba.com, alikernel-developer@linux.alibaba.com Subject: Re: [PATCH v4] arm64: mm: fix linear mem mapping access performance degradation Message-ID: <20220704111402.GA31553@willie-the-truck> References: <1656777473-73887-1-git-send-email-guanghuifeng@linux.alibaba.com> <20220704103523.GC31437@willie-the-truck> <73f0c53b-fd17-c5e9-3773-1d71e564eb50@linux.alibaba.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <73f0c53b-fd17-c5e9-3773-1d71e564eb50@linux.alibaba.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220704_041417_780517_977BA9FC X-CRM114-Status: GOOD ( 29.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gTW9uLCBKdWwgMDQsIDIwMjIgYXQgMDY6NTg6MjBQTSArMDgwMCwgZ3VhbmdodWkuZmdoIHdy b3RlOgo+IAo+IAo+IOWcqCAyMDIyLzcvNCAxODozNSwgV2lsbCBEZWFjb24g5YaZ6YGTOgo+ID4g T24gU2F0LCBKdWwgMDIsIDIwMjIgYXQgMTE6NTc6NTNQTSArMDgwMCwgR3VhbmdodWkgRmVuZyB3 cm90ZToKPiA+ID4gVGhlIGFybTY0IGNhbiBidWlsZCAyTS8xRyBibG9jay9zZWN0aWlvbiBtYXBw aW5nLiBXaGVuIHVzaW5nIERNQS9ETUEzMiB6b25lCj4gPiA+IChlbmFibGUgY3Jhc2hrZXJuZWws IGRpc2FibGUgcm9kYXRhIGZ1bGwsIGRpc2FibGUga2ZlbmNlKSwgdGhlIG1lbV9tYXAgd2lsbAo+ ID4gPiB1c2Ugbm9uIGJsb2NrL3NlY3Rpb24gbWFwcGluZyhmb3IgY3Jhc2hrZXJuZWwgcmVxdWly ZXMgdG8gc2hyaW5rIHRoZSByZWdpb24KPiA+ID4gaW4gcGFnZSBncmFudWxhcml0eSkuIEJ1dCBp dCB3aWxsIGRlZ3JhZGUgcGVyZm9ybWFuY2Ugd2hlbiBkb2luZyBsYXJnaW5nCj4gPiA+IGNvbnRp bnVvdXMgbWVtIGFjY2VzcyBpbiBrZXJuZWwobWVtY3B5L21lbW1vdmUsIGV0YykuCj4gPiAKPiA+ IEhtbS4gSXQgc2VlbXMgYSBiaXQgc2lsbHkgdG8gbWUgdGhhdCB3ZSB0YWtlIHNwZWNpYWwgY2Fy ZSB0byB1bm1hcCB0aGUKPiA+IGNyYXNoa2VybmVsIGZyb20gdGhlIGxpbmVhciBtYXAgZXZlbiB3 aGVuIGNhbl9zZXRfZGlyZWN0X21hcCgpIGlzIGZhbHNlLCBhcwo+ID4gd2Ugd29uJ3QgYmUgcHJv dGVjdGluZyB0aGUgbWFpbiBrZXJuZWwgYXQgYWxsIQo+ID4gCj4gPiBXaHkgZG9uJ3Qgd2UganVz dCBsZWF2ZSB0aGUgY3Jhc2hrZXJuZWwgbWFwcGVkIGlmICFjYW5fc2V0X2RpcmVjdF9tYXAoKQo+ ID4gYW5kIHRoZW4gdGhpcyBwcm9ibGVtIGp1c3QgZ29lcyBhd2F5Pwo+ID4gCj4gPiBXaWxsCj4g Cj4gVGhpcyBxdWVzdGlvbiBoYWQgYmVlbiBhc2tlZCBsYXNrIHdlZWsuCgpTb3JyeSwgSSBkaWRu J3Qgc3BvdCB0aGF0LiBQbGVhc2UgY291bGQgeW91IGxpbmsgbWUgdG8gdGhlIGNvbnZlcnNhdGlv biwgYXMKSSdtIHN0aWxsIHVuYWJsZSB0byBmaW5kIGl0IGluIG15IGluYm94PwoKPiAxLlF1b3Rl ZCBtZXNzYWdlcyBmcm9tIGFyY2gvYXJtNjQvbW0vaW5pdC5jCj4gCj4gIk1lbW9yeSByZXNlcnZh dGlvbiBmb3IgY3Jhc2gga2VybmVsIGVpdGhlciBkb25lIGVhcmx5IG9yIGRlZmVycmVkCj4gZGVw ZW5kaW5nIG9uIERNQSBtZW1vcnkgem9uZXMgY29uZmlncyAoWk9ORV9ETUEpIC0tCj4gCj4gSW4g YWJzZW5jZSBvZiBaT05FX0RNQSBjb25maWdzIGFybTY0X2RtYV9waHlzX2xpbWl0IGluaXRpYWxp emVkCj4gaGVyZSBpbnN0ZWFkIG9mIG1heF96b25lX3BoeXMoKS4gIFRoaXMgbGV0cyBlYXJseSBy ZXNlcnZhdGlvbiBvZgo+IGNyYXNoIGtlcm5lbCBtZW1vcnkgd2hpY2ggaGFzIGEgZGVwZW5kZW5j eSBvbiBhcm02NF9kbWFfcGh5c19saW1pdC4KPiBSZXNlcnZpbmcgbWVtb3J5IGVhcmx5IGZvciBj cmFzaCBrZXJuZWwgYWxsb3dzIGxpbmVhciBjcmVhdGlvbiBvZiBibG9jawo+IG1hcHBpbmdzIChn cmVhdGVyIHRoYW4gcGFnZS1ncmFudWxhcml0eSkgZm9yIGFsbCB0aGUgbWVtb3J5IGJhbmsgcmFu Z3MuCj4gSW4gdGhpcyBzY2hlbWUgYSBjb21wYXJhdGl2ZWx5IHF1aWNrZXIgYm9vdCBpcyBvYnNl cnZlZC4KPiAKPiBJZiBaT05FX0RNQSBjb25maWdzIGFyZSBkZWZpbmVkLCBjcmFzaCBrZXJuZWwg bWVtb3J5IHJlc2VydmF0aW9uCj4gaXMgZGVsYXllZCB1bnRpbCBETUEgem9uZSBtZW1vcnkgcmFu Z2Ugc2l6ZSBpbml0aWFsaXphdGlvbiBwZXJmb3JtZWQgaW4KPiB6b25lX3NpemVzX2luaXQoKS4g IFRoZSBkZWZlciBpcyBuZWNlc3NhcnkgdG8gc3RlZXIgY2xlYXIgb2YgRE1BIHpvbmUKPiBtZW1v cnkgcmFuZ2UgdG8gYXZvaWQgb3ZlcmxhcCBhbGxvY2F0aW9uLgo+IAo+IFtbWwo+IFNvIGNyYXNo IGtlcm5lbCBtZW1vcnkgYm91bmRhcmllcyBhcmUgbm90IGtub3duIHdoZW4gbWFwcGluZyBhbGwg YmFuayBtZW1vcnkKPiByYW5nZXMsIHdoaWNoIG90aGVyd2lzZSBtZWFucyBub3QgcG9zc2libGUg dG8gZXhjbHVkZSBjcmFzaCBrZXJuZWwgcmFuZ2UKPiBmcm9tIGNyZWF0aW5nIGJsb2NrIG1hcHBp bmdzIHNvIHBhZ2UtZ3JhbnVsYXJpdHkgbWFwcGluZ3MgYXJlIGNyZWF0ZWQgZm9yCj4gdGhlIGVu dGlyZSBtZW1vcnkgcmFuZ2UuCj4gXV1dIgo+IAo+IE5hbWVseSwgdGhlIGluaXQgb3JkZXI6IG1l bWJsb2NrIGluaXQtLS0+bGluZWFyIG1lbSBtYXBwaW5nKDRrIG1hcHBpbmcgZm9yCj4gY3Jhc2hr ZXJuZWwsIHJlcXVpcmluaWcgcGFnZS1ncmFudWxhcml0eSBjaGFuZ2luZykpLS0tPnpvbmUgZG1h Cj4gbGltaXQtLS0+cmVzZXJ2ZSBjcmFzaGtlcm5lbC4KPiBTbyB3aGVuIGVuYWJsZSBaT05FIERN QSBhbmQgdXNpbmcgY3Jhc2hrZXJuZWwsIHRoZSBtZW0gbWFwcGluZyB1c2luZyA0awo+IG1hcHBp bmcuCgpZZXMsIEkgdW5kZXJzdGFuZCB0aGF0IGlzIGhvdyB0aGluZ3Mgd29yayB0b2RheSBidXQg SSdtIHNheWluZyB0aGF0IHdlIG1heQphcyB3ZWxsIGxlYXZlIHRoZSBjcmFzaGtlcm5lbCBtYXBw ZWQgKGF0IGJsb2NrIGdyYW51bGFyaXR5KSBpZgohY2FuX3NldF9kaXJlY3RfbWFwKCkgYW5kIHRo ZW4gSSB0aGluayB5b3VyIHBhdGNoIGJlY29tZXMgYSBsb3Qgc2ltcGxlci4KCldpbGwKCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJu ZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRw Oi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK