From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CBC6C433F5 for ; Fri, 17 Sep 2021 21:22:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B2941611C8 for ; Fri, 17 Sep 2021 21:22:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B2941611C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 431D96B0071; Fri, 17 Sep 2021 17:22:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E1466B0072; Fri, 17 Sep 2021 17:22:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25C9E900002; Fri, 17 Sep 2021 17:22:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id 13DE76B0071 for ; Fri, 17 Sep 2021 17:22:56 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BEB4482499A8 for ; Fri, 17 Sep 2021 21:22:55 +0000 (UTC) X-FDA: 78598340310.38.6F8B231 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 5B116F000205 for ; Fri, 17 Sep 2021 21:22:55 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 989F361164; Fri, 17 Sep 2021 21:22:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1631913774; bh=Uu9dNO8DrgIzGfRQBRb7gNnqOOCyAqAxFtQDy7L9y2s=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=KF8gycHviV29usRa2ZvPMSqOY3SjVrmgWv+0TvHYvp8An/CV0MggYAE9e3Mnm3K4j uZZX3r2BCBGNMND/k9jJ8x3c+KQGNepPpyPO6PknCXYUIeKt/3xpE/wwc7vtJzGWVK CJ29ckcCOv6pC+zcA810V8EEFGRqjzqJslCQQfsrNs64qmamK8bOdVzEMPEiMyrfJU eTn/hRFLqdHTG6vdPtywGn+kXu9qzJCCq5Pn1hsWOPEapr6DMo2Y1CGT/EWDBhqQoZ hGnZ19TtmfCbojaa8is0Zv/QVmQDmEAI/kQ+YS3y0yVyGVS4isqvmorNyQbTwwN2XP hl9KdGpI6OTKQ== Date: Sat, 18 Sep 2021 00:22:47 +0300 From: Mike Rapoport To: Catalin Marinas Cc: David Hildenbrand , Robin Murphy , Alex Bee , Will Deacon , Andrew Morton , Anshuman Khandual , Linux Kernel Mailing List , linux-mm@kvack.org, Linux ARM , Christoph Hellwig Subject: Re: [BUG 5.14] arm64/mm: dma memory mapping fails (in some cases) Message-ID: References: <20210824173741.GC623@arm.com> <0908ce39-7e30-91fa-68ef-11620f9596ae@arm.com> <60a11eba-2910-3b5f-ef96-97d4556c1596@redhat.com> <20210825102044.GA3420@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210825102044.GA3420@arm.com> X-Stat-Signature: bpsmzmucos4zn86ngcx7q8bs8kk75ji1 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=KF8gycHv; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of rppt@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=rppt@kernel.org X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 5B116F000205 X-HE-Tag: 1631913775-901151 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 25, 2021 at 11:20:46AM +0100, Catalin Marinas wrote: > + hch > > On Tue, Aug 24, 2021 at 08:59:22PM +0200, David Hildenbrand wrote: > > On 24.08.21 20:46, Robin Murphy wrote: > > > On 2021-08-24 19:28, Mike Rapoport wrote: > > > > On Tue, Aug 24, 2021 at 06:37:41PM +0100, Catalin Marinas wrote: > > > > > On Tue, Aug 24, 2021 at 03:40:47PM +0200, Alex Bee wrote: > > > > > > it seems there is a regression in arm64 memory mapping in 5.14, since it > > > > > > fails on Rockchip RK3328 when the pl330 dmac tries to map with: > > > > > > > > > > > > ------------[ cut here ]------------ > > > > > > WARNING: CPU: 2 PID: 373 at kernel/dma/mapping.c:235 dma_map_resource+0x68/0xc0 > > > > > > Modules linked in: spi_rockchip(+) fuse > > > > > > CPU: 2 PID: 373 Comm: systemd-udevd Not tainted 5.14.0-rc7 #1 > > > > > > Hardware name: Pine64 Rock64 (DT) > > > > > > pstate: 80000005 (Nzcv daif -PAN -UAO -TCO BTYPE=--) > > > > > > pc : dma_map_resource+0x68/0xc0 > > > > > > lr : pl330_prep_slave_fifo+0x78/0xd0 > > > > > > sp : ffff800012102ae0 > > > > > > x29: ffff800012102ae0 x28: ffff000005c94800 x27: 0000000000000000 > > > > > > x26: ffff000000566bd0 x25: 0000000000000001 x24: 0000000000000001 > > > > > > x23: 0000000000000002 x22: ffff000000628c00 x21: 0000000000000001 > > > > > > x20: ffff000000566bd0 x19: 0000000000000001 x18: 0000000000000000 > > > > > > x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 > > > > > > x14: 0000000000000277 x13: 0000000000000001 x12: 0000000000000000 > > > > > > x11: 0000000000000001 x10: 00000000000008e0 x9 : ffff800012102a80 > > > > > > x8 : ffff000000d14b80 x7 : ffff0000fe7b12f0 x6 : ffff0000fe7b1100 > > > > > > x5 : fffffc000000000f x4 : 0000000000000000 x3 : 0000000000000001 > > > > > > x2 : 0000000000000001 x1 : 00000000ff190800 x0 : ffff000000628c00 > > > > > > Call trace: > > > > > > dma_map_resource+0x68/0xc0 > > > > > > pl330_prep_slave_sg+0x58/0x220 > > > > > > rockchip_spi_prepare_dma+0xd8/0x2c0 [spi_rockchip] > > > > > > rockchip_spi_transfer_one+0x294/0x3d8 [spi_rockchip] > > > > > [...] > > > > > > Note: This does not relate to the spi driver - when disabling this device in > > > > > > the device tree it fails for any other (i2s, for instance) which uses dma. > > > > > > Commenting out the failing check at [1], however, helps and the mapping > > > > > > works again. > > > > > > > > > Do you know which address dma_map_resource() is trying to map (maybe > > > > > add some printk())? It's not supposed to map RAM, hence the warning. > > > > > Random guess, the address is 0xff190800 (based on the x1 above but the > > > > > regs might as well be mangled). > > > > > > > > 0xff190800 will cause this warning for sure. It has a memory map, but it is > > > > not RAM so old version of pfn_valid() would return 0 and the new one > > > > returns 1. > > > > > > How does that happen, though? It's not a memory address, and it's not > > > even within the bounds of anywhere there should or could be memory. This > > > SoC has a simple memory map - everything from 0 to 0xfeffffff goes to > > > the DRAM controller (which may not all be populated, and may have pieces > > > carved out by secure firmware), while 0xff000000-0xffffffff is MMIO. Why > > > do we have pages (or at least the assumption of pages) for somewhere > > > which by all rights should not have them? > > > > Simple: we allocate the vmemmap for whole sections (e.g., 128 MiB) to avoid > > any such hacks. If there is a memory hole, it gets a memmap as well. > > > > Tricking pfn_valid() into returning "false" where we actually have a memmap > > only makes it look like there is no memmap; but there is one, and > > it's PG_reserved. > > I can see the documentation for pfn_valid() does not claim anything more > than the presence of an memmap entry. But I wonder whether the confusion > is wider-spread than just the DMA code. At a quick grep, try_ram_remap() > assumes __va() can be used on pfn_valid(), though I suspect it relies on > the calling function to check that the resource was RAM. The arm64 > kern_addr_valid() returns true based on pfn_valid() and kcore.c uses > standard memcpy on it, which wouldn't work for I/O (should we change > this check to pfn_is_map_memory() for arm64?). > > > > > > Either pfn_valid() gets confused in 5.14 or something is wrong with the > > > > > DT. I have a suspicion it's the former since reverting the above commit > > > > > makes it disappear. > > > > > > > > I think pfn_valid() actually behaves as expected but the caller is wrong > > > > because pfn_valid != RAM (this applies btw to !arm64 as well). > > > > > > > > /* Don't allow RAM to be mapped */ > > > > if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr)))) > > > > return DMA_MAPPING_ERROR; > > > > > > > > Alex, can you please try this patch: > > > > > > That will certainly paper over the issue, but it's avoiding the question > > > of what went wrong with the memory map in the first place. The comment > > > is indeed a bit inaccurate, but ultimately dma_map_resource() exists for > > > addresses that would be wrong to pass to dma_map_page(), so I believe > > > pfn_valid() is still the correct check. > > > > If we want to check for RAM, pfn_valid() would be wrong. If we want to check > > for "is there a memmap, for whatever lives or does not live there", > > pfn_valid() is the right check. > > So what should the DMA code use instead? Last time we needed something > similar, the recommendation was to use pfn_to_online_page(). Mike is > suggesting memblock_is_memory(). I did some digging and it seems that the most "generic" way to check if a page is in RAM is page_is_ram(). It's not 100% bullet proof as it'll give false negatives for architectures that do not register "System RAM", but those are not using dma_map_resource() anyway and, apparently, never would. Alex, can you test this patch please? >From 35586e9dbbdb47962cb0f3803bc650d77867905a Mon Sep 17 00:00:00 2001 From: Mike Rapoport Date: Sat, 18 Sep 2021 00:01:08 +0300 Subject: [PATCH] dma-mapping: use page_is_ram to ensure RAM is not mapped dma_map_resource() uses pfn_valid() to ensure the range is not RAM. However, pfn_valid() only checks for availability of the memory map for a PFN but it does not ensure that the PFN is actually backed by RAM. Replace pfn_valid() with page_is_ram() that does verify whether a PFN is in RAM or not. Signed-off-by: Mike Rapoport --- kernel/dma/mapping.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index 7ee5284bff58..dc6064a213a2 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -296,7 +296,7 @@ dma_addr_t dma_map_resource(struct device *dev, phys_addr_t phys_addr, return DMA_MAPPING_ERROR; /* Don't allow RAM to be mapped */ - if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr)))) + if (WARN_ON_ONCE(page_is_ram(PHYS_PFN(phys_addr)))) return DMA_MAPPING_ERROR; if (dma_map_direct(dev, ops)) -- 2.28.0 > Given how later we are in the -rc cycle, I suggest we revert Anshuman's > commit 16c9afc77660 ("arm64/mm: drop HAVE_ARCH_PFN_VALID") and try to > assess the implications in 5.15 (the patch doesn't seem to have the > arm64 maintainers' ack anyway ;)). > > Thanks. > > -- > Catalin -- Sincerely yours, Mike.