From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92B17C432BE for ; Tue, 24 Aug 2021 18:30:48 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5912361357 for ; Tue, 24 Aug 2021 18:30:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5912361357 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=u4wQlaDUNdOZCMFKwwfPJxMOqzdpIwB1mYuRxMHrPFw=; b=f+TylWmkyeo1G1 QVOGTNGrsKYUbnUGcK22oiR6onoEot0rpshYb7nu7pZpsOXCUrNX35vieJ5i2zSqpvq/ldq7/hy/m qrpdHWYrq/3R+lBOocubbASkMYe6vPMJjsCoMYvFudWv6yOjvGZdOKDsulGI30WCT9/o3ecxIsBaj danppyqPpdeKFm8dX+Gly0pQxwCAAak1Z9aNCs6JZ7MYY2BL4IaM3ID+Tg7I2OHZhX7zSqvPs5yvc 56a9qWBh0A4Z6D62GLuwMYrjAVA4Dn9d9zurQgcbUa1yxKuSix6Fd1eWCtLshNL8+5q5McnSAlZYC YB2NQc4TqNTq/p0cvSoQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mIb9k-004MYf-8a; Tue, 24 Aug 2021 18:28:16 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mIb9g-004MWd-8t for linux-arm-kernel@lists.infradead.org; Tue, 24 Aug 2021 18:28:14 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0548661357; Tue, 24 Aug 2021 18:28:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1629829691; bh=qejIvxeKJm0/TqXoKGNLd7McXlorFN4Ly2FywuAeM9k=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=gW+926tqq97MY1AfJ/rW64ngE8zXCk3TIGUdVRIUCHgRb/S2uzQiw8fQazz5hdg43 5Bd5fWg4E5qwsHqrjZBH0my9FoADc9atxhEeD2FKnLoZw03JAnP0xtBmkR2bNMeYD/ mVG+nVsbMc/W+8f9scHCgU/nimS1iWyrMZ/KAIm1PAJGO426/anRg/XDrEaaM6BkOr okRWQEQ8BGVnyjTSJ8P+CoRfkwrdGFbGQdvYzIZjD84crXLh4yu5/D8lEVCUx2v7HD 1X4mp3EmkdV5Kmr7v8zjTkNFpc8NZoKFm15yuJh99dcwS6SvvA2bbxDt5A6OJ9eFos 2iNnj6t6MHJOg== Date: Tue, 24 Aug 2021 21:28:05 +0300 From: Mike Rapoport To: Catalin Marinas Cc: Alex Bee , Will Deacon , Andrew Morton , Anshuman Khandual , Linux Kernel Mailing List , linux-mm@kvack.org, Linux ARM , Robin Murphy Subject: Re: [BUG 5.14] arm64/mm: dma memory mapping fails (in some cases) Message-ID: References: <20210824173741.GC623@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210824173741.GC623@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210824_112812_399293_AB42003F X-CRM114-Status: GOOD ( 44.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Aug 24, 2021 at 06:37:41PM +0100, Catalin Marinas wrote: > Hi Alex, > = > Thanks for the report. > = > On Tue, Aug 24, 2021 at 03:40:47PM +0200, Alex Bee wrote: > > it seems there is a regression in arm64 memory mapping in 5.14, since it > > fails on Rockchip RK3328 when the pl330 dmac tries to map with: > > = > > [=A0=A0=A0 8.921909] ------------[ cut here ]------------ > > [=A0=A0=A0 8.921940] WARNING: CPU: 2 PID: 373 at kernel/dma/mapping.c:2= 35 dma_map_resource+0x68/0xc0 > > [=A0=A0=A0 8.921973] Modules linked in: spi_rockchip(+) fuse > > [=A0=A0=A0 8.921996] CPU: 2 PID: 373 Comm: systemd-udevd Not tainted 5.= 14.0-rc7 #1 > > [=A0=A0=A0 8.922004] Hardware name: Pine64 Rock64 (DT) > > [=A0=A0=A0 8.922011] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO BTYPE= =3D--) > > [=A0=A0=A0 8.922018] pc : dma_map_resource+0x68/0xc0 > > [=A0=A0=A0 8.922026] lr : pl330_prep_slave_fifo+0x78/0xd0 > > [=A0=A0=A0 8.922040] sp : ffff800012102ae0 > > [=A0=A0=A0 8.922043] x29: ffff800012102ae0 x28: ffff000005c94800 x27: 0= 000000000000000 > > [=A0=A0=A0 8.922056] x26: ffff000000566bd0 x25: 0000000000000001 x24: 0= 000000000000001 > > [=A0=A0=A0 8.922067] x23: 0000000000000002 x22: ffff000000628c00 x21: 0= 000000000000001 > > [=A0=A0=A0 8.922078] x20: ffff000000566bd0 x19: 0000000000000001 x18: 0= 000000000000000 > > [=A0=A0=A0 8.922089] x17: 0000000000000000 x16: 0000000000000000 x15: 0= 000000000000000 > > [=A0=A0=A0 8.922100] x14: 0000000000000277 x13: 0000000000000001 x12: 0= 000000000000000 > > [=A0=A0=A0 8.922111] x11: 0000000000000001 x10: 00000000000008e0 x9 : f= fff800012102a80 > > [=A0=A0=A0 8.922123] x8 : ffff000000d14b80 x7 : ffff0000fe7b12f0 x6 : f= fff0000fe7b1100 > > [=A0=A0=A0 8.922134] x5 : fffffc000000000f x4 : 0000000000000000 x3 : 0= 000000000000001 > > [=A0=A0=A0 8.922145] x2 : 0000000000000001 x1 : 00000000ff190800 x0 : f= fff000000628c00 > > [=A0=A0=A0 8.922158] Call trace: > > [=A0=A0=A0 8.922163]=A0 dma_map_resource+0x68/0xc0 > > [=A0=A0=A0 8.922173]=A0 pl330_prep_slave_sg+0x58/0x220 > > [=A0=A0=A0 8.922181]=A0 rockchip_spi_prepare_dma+0xd8/0x2c0 [spi_rockch= ip] > > [=A0=A0=A0 8.922208]=A0 rockchip_spi_transfer_one+0x294/0x3d8 [spi_rock= chip] > [...] > > Note: This does not relate to the spi driver - when disabling this devi= ce in > > the device tree it fails for any other (i2s, for instance) which uses d= ma. > > Commenting out the failing check at [1], however, helps and the mapping > > works again. > Do you know which address dma_map_resource() is trying to map (maybe > add some printk())? It's not supposed to map RAM, hence the warning. > Random guess, the address is 0xff190800 (based on the x1 above but the > regs might as well be mangled). 0xff190800 will cause this warning for sure. It has a memory map, but it is not RAM so old version of pfn_valid() would return 0 and the new one returns 1. = > > I tried to follow the recent changes for arm64 mm which could relate to= the > > check failing at [1] and reverting > > =A0 commit 16c9afc77660 ("arm64/mm: drop HAVE_ARCH_PFN_VALID") > > helps and makes it work again, but I'm 100% uncertain if that commit is > > really the culprit. > > = > > Note, that the firmware (legacy u-boot) injects memory configuration in= the > > device tree as follows: > > = > > /memreserve/=A0=A0=A0 0x00000000fcefc000 0x000000000000d000; > > / { > > .. > > =A0=A0=A0 compatible =3D "pine64,rock64\0rockchip,rk3328"; > > .. > > =A0=A0=A0 memory { > > =A0=A0=A0 =A0=A0=A0 reg =3D <0x00 0x200000 0x00 0xfee00000 0x00 0x00 0x= 00 0x00>; > > =A0=A0=A0 =A0=A0=A0 device_type =3D "memory"; > > =A0=A0=A0 }; > > = > > .. > > } > = > Either pfn_valid() gets confused in 5.14 or something is wrong with the > DT. I have a suspicion it's the former since reverting the above commit > makes it disappear. I think pfn_valid() actually behaves as expected but the caller is wrong because pfn_valid !=3D RAM (this applies btw to !arm64 as well). /* Don't allow RAM to be mapped */ if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr)))) return DMA_MAPPING_ERROR; Alex, can you please try this patch: diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index 2b06a809d0b9..4715e9641a29 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -232,7 +232,7 @@ dma_addr_t dma_map_resource(struct device *dev, phys_ad= dr_t phys_addr, return DMA_MAPPING_ERROR; = /* Don't allow RAM to be mapped */ - if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr)))) + if (WARN_ON_ONCE(!memblock_is_memory(phys_addr))) return DMA_MAPPING_ERROR; = if (dma_map_direct(dev, ops)) = > > So: there is a "hole" in the mappable memory and reading the commit mes= sage > > of > > =A0 commit a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify > > pfn_valid()") > > suggests, there was a change for that case recently. > = > I think the change from the arm64 pfn_valid() to the generic one is > avoiding the call to memblock_is_memory(). I wonder whether pfn_valid() > returns true just because we have a struct page available but the memory > may have been reserved. > = > Cc'ing Mike R. > = > > I also noticed there is a diff in the kernel log regarding memory init = up > > until 5.13.12 it says > > = > > [=A0=A0=A0 0.000000] Zone ranges: > > [=A0=A0=A0 0.000000]=A0=A0 DMA=A0=A0=A0=A0=A0 [mem 0x0000000000200000-0= x00000000feffffff] > > [=A0=A0=A0 0.000000]=A0=A0 DMA32=A0=A0=A0 empty > > [=A0=A0=A0 0.000000]=A0=A0 Normal=A0=A0 empty > > [=A0=A0=A0 0.000000] Movable zone start for each node > > [=A0=A0=A0 0.000000] Early memory node ranges > > [=A0=A0=A0 0.000000]=A0=A0 node=A0=A0 0: [mem 0x0000000000200000-0x0000= 0000feffffff] > > [=A0=A0=A0 0.000000] Initmem setup node 0 [mem 0x0000000000200000-0x000= 00000feffffff] > > [=A0=A0=A0 0.000000] On node 0 totalpages: 1043968 > > [=A0=A0=A0 0.000000]=A0=A0 DMA zone: 16312 pages used for memmap > > [=A0=A0=A0 0.000000]=A0=A0 DMA zone: 0 pages reserved > > [=A0=A0=A0 0.000000]=A0=A0 DMA zone: 1043968 pages, LIFO batch:63 > > = > > In contrary in 5.14-rc7 it says: > > = > > [=A0=A0=A0 0.000000] Zone ranges: > > [=A0=A0=A0 0.000000]=A0=A0 DMA=A0=A0=A0=A0=A0 [mem 0x0000000000200000-0= x00000000feffffff] > > [=A0=A0=A0 0.000000]=A0=A0 DMA32=A0=A0=A0 empty > > [=A0=A0=A0 0.000000]=A0=A0 Normal=A0=A0 empty > > [=A0=A0=A0 0.000000] Movable zone start for each node > > [=A0=A0=A0 0.000000] Early memory node ranges > > [=A0=A0=A0 0.000000]=A0=A0 node=A0=A0 0: [mem 0x0000000000200000-0x0000= 0000feffffff] > > [=A0=A0=A0 0.000000] Initmem setup node 0 [mem 0x0000000000200000-0x000= 00000feffffff] > > [=A0=A0=A0 0.000000] On node 0, zone DMA: 512 pages in unavailable rang= es > > [=A0=A0=A0 0.000000] On node 0, zone DMA: 4096 pages in unavailable ran= ges > > = > > (note the "unavailable ranges") > > I'm uncertain again here, if that diff is expected behavior because of = those > > recent mm changes for arm64. > > = > > After reverting > > =A0 commit 16c9afc77660 ("arm64/mm: drop HAVE_ARCH_PFN_VALID") > > the log changes to > > = > > [=A0=A0=A0 0.000000] Zone ranges: > > [=A0=A0=A0 0.000000]=A0=A0 DMA=A0=A0=A0=A0=A0 [mem 0x0000000000200000-0= x00000000feffffff] > > [=A0=A0=A0 0.000000]=A0=A0 DMA32=A0=A0=A0 empty > > [=A0=A0=A0 0.000000]=A0=A0 Normal=A0=A0 empty > > [=A0=A0=A0 0.000000] Movable zone start for each node > > [=A0=A0=A0 0.000000] Early memory node ranges > > [=A0=A0=A0 0.000000]=A0=A0 node=A0=A0 0: [mem 0x0000000000200000-0x0000= 0000feffffff] > > [=A0=A0=A0 0.000000] Initmem setup node 0 [mem > > 0x0000000000200000-0x00000000feffffff] > > = > > (no DMA zones here) > > = > > As you might have noticed I have _zero_ clue about memory mapping and d= ma > > subsystem - so let me know if there is any more information needed for = that > > and thanks for your help. > = > Adding Robin as well, he has a better clue than us on DMA ;). > = > > Alex > > = > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/= tree/kernel/dma/mapping.c?id=3De22ce8eb631bdc47a4a4ea7ecf4e4ba499db4f93#n235 > = > -- = > Catalin -- = Sincerely yours, Mike. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel