From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AD87C4338F for ; Wed, 4 Aug 2021 10:39:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3DC2E60E96 for ; Wed, 4 Aug 2021 10:39:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3DC2E60E96 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=armlinux.org.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UV85MgvvBEhCnQwOxSmm9+0nxeqRZtM8QmaOKWlC7vE=; b=ke0uGQ1ztc1SO6 Tis5DKTFBi6kbp+yqZsNb9M9jVWi0/nps0J9fIPfvJxsO1S9IvlHsa3HuBNAtKfJBjPCKPLa2rKxr VT0Dwu+THu+UyK6gmnWSjK6UVE4/RkYbOLwSObBiugJM2y4ozoiAJeRO8tY+yunXY+3E/Xu6su2cr iBZmxbZSx6iT0IqQ4S2P+hHssvTvJQAUvcl1VcEC0sw66pNxvbr5GTOBkdqHmerQCoaBa1DI+o61Q EMQgas+1r76Gn4meXkD/6LxMAFcpBapmsylUvQCUVgDBaD2RWys6w42U+bh64B7YPGaZbjoGb6VvK fh6Qz1WUWoNM6aQYa5eA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mBEGu-005kmb-3o; Wed, 04 Aug 2021 10:37:12 +0000 Received: from pandora.armlinux.org.uk ([2001:4d48:ad52:32c8:5054:ff:fe00:142]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mBEGp-005kgS-Fb for linux-arm-kernel@lists.infradead.org; Wed, 04 Aug 2021 10:37:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=uJBqoVHuj4xIngws3p309iMuLOuYjLcL6RqZGwScV6U=; b=BO8qX5rvVMaOLM5XBbJUwymZl 7GmtCw6FL1CXsf7L0gWQ9a3oc1X2jSky1NXaPHmMN81JyBDiSWMoL0NUVsf40hrLlsZ11dw/O3Pxc rdoiNJuiammV/LETUNZcAugaZxB773eL8Buwl/OwD8B9jUHgwniMTpjXnsKjas0G4EgfIImuLxZA6 U1LUTBsr7BTX2awVnHSDXWFVUKViS04mUx9LKsMgMQIM68YDJ39lqUuz4gnqdNv+w64bNfmh5c9RP ZdY/5PJM1mP5Z7xTuc4cy7G1zYemPVxuEFIXh0dLL2aJQDzZXyC+sXOEcl/3JtH6a0qtd/Vq6ZIVR 5GbBf9D7A==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:46932) by pandora.armlinux.org.uk with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mBEGi-000062-Q4; Wed, 04 Aug 2021 11:37:00 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.92) (envelope-from ) id 1mBEGg-0004io-Fe; Wed, 04 Aug 2021 11:36:58 +0100 Date: Wed, 4 Aug 2021 11:36:58 +0100 From: "Russell King (Oracle)" To: Linus Walleij Cc: Grygorii Strashko , Nishanth Menon , santosh.shilimkar@oracle.com, Santosh Shilimkar , Linux ARM , Arnd Bergmann , Florian Fainelli , Geert Uytterhoeven , Ard Biesheuvel , Suman Anna , Tero Kristo Subject: Re: [PATCH 3/3] ARM: Map the lowmem and kernel separately Message-ID: <20210804103658.GP22278@shell.armlinux.org.uk> References: <20210517145719.110966-1-linus.walleij@linaro.org> <20210517145719.110966-4-linus.walleij@linaro.org> <20210730202259.dbifugsvry5vjdr7@earthen> <20210803104932.GM22278@shell.armlinux.org.uk> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210804_033707_554726_EEC2C7F3 X-CRM114-Status: GOOD ( 36.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Aug 04, 2021 at 12:05:54PM +0200, Linus Walleij wrote: > On Tue, Aug 3, 2021 at 12:49 PM Russell King (Oracle) > wrote: > > > > It should be called on all keystone 2 platforms by default (LAPE is default mode for K2). > > > > > > Huh. Below as I remember it: > > > - K2 starts using memory window (aliased memory) at 00 8000 0000 (No IO coherency is supported) > > > - K2, early at boot, is switching to LPAE memory window 08 0000 0000 (IO coherency is supported) > > > so all, already mapped memory, has to be fixed - phys addresses. > > > > If I remember correctly, the code that fixes that up assumes that > > (a) the kernel is mapped using section mappings in the lowmem mapping > > at PAGE_OFFSET.._end-1 > > (b) that no other RAM is mapped (iow, it's called before the kernel > > starts building the real page mappings in paging_init()). > > > > It looks to me like Linus missed the code in arch/arm/mm/pv-fixup-asm.S > > and as the kernel is no longer mapped in the lowmem mapping, this > > likely writes a load of entries in the page tables that are random... > > I looked into this! > > Indeed early_mm_init() is called before paging_init() and early_mm_init() > calls lpae_pgtables_remap_asm() in pv-fixup-asm.S to add the offset > to all the section mappings over the kernel. > > The section mappings we have at this point are coming from head.S, > so those get modified with the offset. > > What I don't understand is what map_lowmem() was doing with the > kernel mappings on the Keystone 2 before my patch. Essentially, what happened with Keystone 2 is that the head.S code does its usual thing, mapping the kernel using the physical address that the early code is running at using section mappings, before then switching the MMU on and running from the virtual address space. At this point, the phys<->virt conversions work for this physical address space (which, on Keystone 2 is the low physical space which isn't coherent with devices.) early_mm_init() gets called, we spot we're on Keystone 2, and we initiate the change - the MMU needs a break-before-make for these mappings, and as we're using these mappings to execute our code, we have to turn the MMU off to do the update. We update the phys<->virt conversion, and then fixup the page tables. lpae_pgtables_remap() is passed the offset between the low physical address and the high physical address, as well as the low physical address of the page tables. (since when we turn the MMU off, all we have access to are the low physical mapping.) The structure of the LPAE page tables is: - First 4k - L1 entries. - Following 32k - L2 entries mapped as sections. In pv-fixup-asm.S: ldr r6, =(_end - 1) add r7, r2, #0x1000 add r6, r7, r6, lsr #SECTION_SHIFT - L2_ORDER add r7, r7, #PAGE_OFFSET >> (SECTION_SHIFT - L2_ORDER) r7 is the low physical address in the L2 entries, r6 is the last (inclusive) entry. Essentially, all we do is add "offset" to every section mapping corresponding to addresses between PAGE_OFFSET and _end. We also do that for the boot data (two section mappings) at FDT_FIXED_BASE, the four L1 entries, and the two TTBR physical address register values. > If it actually overwrites the kernel mapping with a new one, it will > undo what lpae_pgtables_remap_asm() has done without applying > the offset. > > So I suppose map_lowmem(), due to the memory ranges passed, > would just bail the kernel mappings and leave them as-is. By the time map_lowmem() gets called, the phys<->virt mappings will have been modified throughout the kernel so that virtual addresses in the direct mapping will translate to the high physical addresses. So, things like __pa(__init_end) in map_kernel() will translate __init_end to the high physical address rather than the low physical address. My guess would be the problem is that kernel_sec_start and kernel_sec_end are not being adjusted, and are still pointing at the low physical addresses rather than the high ones. Consequently, kernel_x_start and kernel_nx_end points at the low phys space, but kernel_x_end and kernel_nx_start points to the high phys space. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel