From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4DD3C432BE for ; Wed, 25 Aug 2021 11:14:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A2ACC6108E for ; Wed, 25 Aug 2021 11:14:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A2ACC6108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:Cc:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=wNKK7x/z+SyeJXi5OL2Q4j+jHWcz9iqG0Rct87pL2eQ=; b=LIpyWtFBvxfV4qC1ji1jhAmgjA TI5nc5+D7urXVq60R5QldfucdELfBlIO2/L2qLU0IoM6pxf8kWO9BNKdLhy9WurzHzHkvRf65Si2i PCObBJwbHWwNUNcJeUe0ysx4Hn56Shyzz0Ej6OjID/Q4/p4Ew/YhZ8ajZyrqlgETg0QC7WruJVdq1 AcJPHa7/TPkU9e7aIeIqJzIkBUSaML6uDaLCUEkosuMSXs2XcRHka9At7ne+EXFKhCT+sutRUHZZy OanCD4X6J0pZGe6sV0nlCuRE8MQrgVmdnKuDFTspbN/NdWHviUYLQX660nH5Vmpjxvzy2afANRBVO OsWFj2Pw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mIqpq-006m3f-Ho; Wed, 25 Aug 2021 11:12:46 +0000 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mIqpm-006m2e-9c for linux-arm-kernel@lists.infradead.org; Wed, 25 Aug 2021 11:12:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1629889960; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mlZrdwxyxNGVrCxVZHW8/Pp6CcGFK5pPfiHHHiWvgYs=; b=Mxg/VdKwcUtXCnWKz3B/0y/m2OSIWiTt8VlK2Lm5vZs/nVXrFWUJA6zo6HjZudD7BNxuiI ow5s5cmW0T8FlHTpa5Ee4KAAH60jvQZBnLKC1ZTLx7GVJfAqlinI0t9nqpHeJvZ8Ag3ygc qKcI1O4bVa+zNjlE9/x2k5Q55oy+lgM= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-199-xIB5czgVPcGLNpftdTX-MA-1; Wed, 25 Aug 2021 07:12:39 -0400 X-MC-Unique: xIB5czgVPcGLNpftdTX-MA-1 Received: by mail-wm1-f69.google.com with SMTP id z18-20020a1c7e120000b02902e69f6fa2e0so5175598wmc.9 for ; Wed, 25 Aug 2021 04:12:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=mlZrdwxyxNGVrCxVZHW8/Pp6CcGFK5pPfiHHHiWvgYs=; b=F5/fyR53GNkCUxcqH2HmpZ96UuvQ8XNgRtrUkt4JzAY0OLbzghcLhaZG+33D8xssei b/CfJD18nBxxrwZlsSvYAq1xHg9t8RgZHDMRZo7BkBBNGv3gYfVzQenEwyId+VIrA1m5 YpNjkUfvMlrvsDGQAaLQKULQf9hFzATT9GZel9UhM2kOtMzV4l6PDEM11iJj6BzYMYMM kp/CUaL8IfgdA+PEmAM1mclbHUDyKPAavYkOMymjVZc3KlqrzBUZ736kvnhOKoRHfdrr TV2vJdF9ZypLUaojTS7Q58EACGf18jho96BSva1AyTTAH4aknyXHugCaJIoGko90seuW osMg== X-Gm-Message-State: AOAM530ABRUw3MMpiskIhYXs0P7txP6sATYS4n+gSzq6/zDY/a5ewTV9 LVHvtfXsUJ/XIJpZ9LhWBFNopyzE/qLZQ+/qOrrg4xFjAnI77IqjqGGhvP6EXOOMfKco3oT+I9r WfrqBnlWR9mUaNQoQkhA5I38X3cJ77e72EeQ= X-Received: by 2002:a1c:a7d2:: with SMTP id q201mr8673062wme.77.1629889958426; Wed, 25 Aug 2021 04:12:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwPYsoqyixurCYV2/i4Ak4XtssI6+XlDP4VkKl4oAoJCywccuN7PduZAQrM/yipMWB60N8Lrw== X-Received: by 2002:a1c:a7d2:: with SMTP id q201mr8673042wme.77.1629889958223; Wed, 25 Aug 2021 04:12:38 -0700 (PDT) Received: from [192.168.3.132] (p4ff23d6b.dip0.t-ipconnect.de. [79.242.61.107]) by smtp.gmail.com with ESMTPSA id b12sm25073911wrx.72.2021.08.25.04.12.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 25 Aug 2021 04:12:37 -0700 (PDT) Subject: Re: [BUG 5.14] arm64/mm: dma memory mapping fails (in some cases) To: Catalin Marinas Cc: Robin Murphy , Mike Rapoport , Alex Bee , Will Deacon , Andrew Morton , Anshuman Khandual , Linux Kernel Mailing List , linux-mm@kvack.org, Linux ARM , Christoph Hellwig References: <20210824173741.GC623@arm.com> <0908ce39-7e30-91fa-68ef-11620f9596ae@arm.com> <60a11eba-2910-3b5f-ef96-97d4556c1596@redhat.com> <20210825102044.GA3420@arm.com> <20210825105510.GB3420@arm.com> From: David Hildenbrand Organization: Red Hat Message-ID: <547785ff-e02f-df28-7f9c-9ad4f5b3cc77@redhat.com> Date: Wed, 25 Aug 2021 13:12:37 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210825105510.GB3420@arm.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210825_041242_553717_192397AB X-CRM114-Status: GOOD ( 34.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 25.08.21 12:55, Catalin Marinas wrote: > On Wed, Aug 25, 2021 at 12:38:31PM +0200, David Hildenbrand wrote: >> On 25.08.21 12:20, Catalin Marinas wrote: >>> On Tue, Aug 24, 2021 at 08:59:22PM +0200, David Hildenbrand wrote: >>>> On 24.08.21 20:46, Robin Murphy wrote: >>>>> On 2021-08-24 19:28, Mike Rapoport wrote: >>>>>> On Tue, Aug 24, 2021 at 06:37:41PM +0100, Catalin Marinas wrote: >>>>>>> On Tue, Aug 24, 2021 at 03:40:47PM +0200, Alex Bee wrote: >>>>>>>> it seems there is a regression in arm64 memory mapping in 5.14, since it >>>>>>>> fails on Rockchip RK3328 when the pl330 dmac tries to map with: >>>>>>>> >>>>>>>> ------------[ cut here ]------------ >>>>>>>> WARNING: CPU: 2 PID: 373 at kernel/dma/mapping.c:235 dma_map_resource+0x68/0xc0 >>>>>>>> Modules linked in: spi_rockchip(+) fuse >>>>>>>> CPU: 2 PID: 373 Comm: systemd-udevd Not tainted 5.14.0-rc7 #1 >>>>>>>> Hardware name: Pine64 Rock64 (DT) >>>>>>>> pstate: 80000005 (Nzcv daif -PAN -UAO -TCO BTYPE=--) >>>>>>>> pc : dma_map_resource+0x68/0xc0 >>>>>>>> lr : pl330_prep_slave_fifo+0x78/0xd0 >>>>>>>> sp : ffff800012102ae0 >>>>>>>> x29: ffff800012102ae0 x28: ffff000005c94800 x27: 0000000000000000 >>>>>>>> x26: ffff000000566bd0 x25: 0000000000000001 x24: 0000000000000001 >>>>>>>> x23: 0000000000000002 x22: ffff000000628c00 x21: 0000000000000001 >>>>>>>> x20: ffff000000566bd0 x19: 0000000000000001 x18: 0000000000000000 >>>>>>>> x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 >>>>>>>> x14: 0000000000000277 x13: 0000000000000001 x12: 0000000000000000 >>>>>>>> x11: 0000000000000001 x10: 00000000000008e0 x9 : ffff800012102a80 >>>>>>>> x8 : ffff000000d14b80 x7 : ffff0000fe7b12f0 x6 : ffff0000fe7b1100 >>>>>>>> x5 : fffffc000000000f x4 : 0000000000000000 x3 : 0000000000000001 >>>>>>>> x2 : 0000000000000001 x1 : 00000000ff190800 x0 : ffff000000628c00 >>>>>>>> Call trace: >>>>>>>> dma_map_resource+0x68/0xc0 >>>>>>>> pl330_prep_slave_sg+0x58/0x220 >>>>>>>> rockchip_spi_prepare_dma+0xd8/0x2c0 [spi_rockchip] >>>>>>>> rockchip_spi_transfer_one+0x294/0x3d8 [spi_rockchip] >>>>>>> [...] >>>>>>>> Note: This does not relate to the spi driver - when disabling this device in >>>>>>>> the device tree it fails for any other (i2s, for instance) which uses dma. >>>>>>>> Commenting out the failing check at [1], however, helps and the mapping >>>>>>>> works again. >>>>>> >>>>>>> Do you know which address dma_map_resource() is trying to map (maybe >>>>>>> add some printk())? It's not supposed to map RAM, hence the warning. >>>>>>> Random guess, the address is 0xff190800 (based on the x1 above but the >>>>>>> regs might as well be mangled). >>>>>> >>>>>> 0xff190800 will cause this warning for sure. It has a memory map, but it is >>>>>> not RAM so old version of pfn_valid() would return 0 and the new one >>>>>> returns 1. >>>>> >>>>> How does that happen, though? It's not a memory address, and it's not >>>>> even within the bounds of anywhere there should or could be memory. This >>>>> SoC has a simple memory map - everything from 0 to 0xfeffffff goes to >>>>> the DRAM controller (which may not all be populated, and may have pieces >>>>> carved out by secure firmware), while 0xff000000-0xffffffff is MMIO. Why >>>>> do we have pages (or at least the assumption of pages) for somewhere >>>>> which by all rights should not have them? >>>> >>>> Simple: we allocate the vmemmap for whole sections (e.g., 128 MiB) to avoid >>>> any such hacks. If there is a memory hole, it gets a memmap as well. >>>> >>>> Tricking pfn_valid() into returning "false" where we actually have a memmap >>>> only makes it look like there is no memmap; but there is one, and >>>> it's PG_reserved. >>> >>> I can see the documentation for pfn_valid() does not claim anything more >>> than the presence of an memmap entry. But I wonder whether the confusion >>> is wider-spread than just the DMA code. At a quick grep, try_ram_remap() >>> assumes __va() can be used on pfn_valid(), though I suspect it relies on >>> the calling function to check that the resource was RAM. The arm64 >>> kern_addr_valid() returns true based on pfn_valid() and kcore.c uses >>> standard memcpy on it, which wouldn't work for I/O (should we change >>> this check to pfn_is_map_memory() for arm64?). >> >> kern_addr_valid() checks that there is a direct map entry, and that the >> mapped address has a valid mmap. (copied from x86-64) > > It checks that there is a va->pa mapping, not necessarily in the linear > map as it walks the page tables. So for some I/O range that happens to > be mapped but which was in close proximity to RAM so that pfn_valid() is > true, kern_addr_valid() would return true. I don't thin that was the > intention. > >> Would you expect to have a direct map for memory holes and similar (IOW, >> !System RAM)? > > No, but we with the generic pfn_valid(), it may return true for mapped > MMIO (with different attributes than the direct map). Ah, right. But can we actually run into that via kcore? kcore builds the RAM list via walk_system_ram_range(), IOW the resource tree. And we end up calling kern_addr_valid() only on KCORE_RAM, KCORE_VMEMMAP and KCORE_TEXT. Not saying that kern_addr_valid() shouldn't be improved. -- Thanks, David / dhildenb _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel