From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6CDFC4338F for ; Tue, 24 Aug 2021 18:59:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5220B610FD for ; Tue, 24 Aug 2021 18:59:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5220B610FD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id B9ADA8D0001; Tue, 24 Aug 2021 14:59:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B239E6B0071; Tue, 24 Aug 2021 14:59:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99D168D0001; Tue, 24 Aug 2021 14:59:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7E14C6B006C for ; Tue, 24 Aug 2021 14:59:27 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2F47F25F48 for ; Tue, 24 Aug 2021 18:59:27 +0000 (UTC) X-FDA: 78510887574.38.95DE983 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf02.hostedemail.com (Postfix) with ESMTP id A4C027001702 for ; Tue, 24 Aug 2021 18:59:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1629831566; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jPwWpNthpZ403/8gfxfxEAu75PwoWMuxmw+b7PTl2R4=; b=YzTXyPt3igrn7vyesz0ecmPoZ9/3HxVD6vABDG6MToZa2Bkm2e73NGZMeKLc8U7nRymGgx hXYp8f8e3gEVRZJwugd4jKLpTZiuFvc3AxnnNGzs0R1OGKJTt76+MjJ7X0gU5RkYxaZkab BJCfr6t+qAoAC47bxxr5nsxP9dPEN5E= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-589-mZj28i4VN9affHnvm2FlRg-1; Tue, 24 Aug 2021 14:59:24 -0400 X-MC-Unique: mZj28i4VN9affHnvm2FlRg-1 Received: by mail-wm1-f70.google.com with SMTP id k5-20020a7bc3050000b02901e081f69d80so4402652wmj.8 for ; Tue, 24 Aug 2021 11:59:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=jPwWpNthpZ403/8gfxfxEAu75PwoWMuxmw+b7PTl2R4=; b=T/2BstB0+5u0ZLF5X0JuvG2z1o6iwCFbQYUAKpyL3Z7k5Xod2EtXfbYBzRkwE+PH5j DDxaKIduS5U//Qwdp0MfSOr8Ol48uaplEroounJMcT7fp+aQ3yjZm2yq58JRaLjn6b6q uH1pfHRDhBbGVVi72UqI5IQlJJQluiHqmQOIHqXi4xgBQzOVnhf4iaxggSWOLSWJHL99 M/BzTSoaBw58ZlX/zjkXTh+PIbQkMQa0EA17QXcCI4VIpPczmUj3bK6Tbier4t9g1WGd uw6Sy8UqpYhWILd1BukIdUjbV1toZX+9HnZyzm1XHaARVwMEGmYndWnmami51/g1P4CW RYNQ== X-Gm-Message-State: AOAM531pk0FQxbaubW96BjUuBS6NFl/Hm+cDfAzvRTOK1ztmuF1NrITf YSoB6qhkksw/TQ56CZMgHs5t/E3Bb00ZYzSDP/MFIQ7SZeDE5hXkSr17sGIr59qvShqwtIOYGYf MSLbirIeA+LI= X-Received: by 2002:a5d:69cf:: with SMTP id s15mr10821282wrw.403.1629831563794; Tue, 24 Aug 2021 11:59:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxyUUsO3aM/Aw8JKywUQlqQx+NxbxwYJB/N2X7y2N6F418U2YWeqhz0sdmAI60nhne1xFmJKg== X-Received: by 2002:a5d:69cf:: with SMTP id s15mr10821263wrw.403.1629831563543; Tue, 24 Aug 2021 11:59:23 -0700 (PDT) Received: from [192.168.3.132] (p4ff23c4d.dip0.t-ipconnect.de. [79.242.60.77]) by smtp.gmail.com with ESMTPSA id y6sm8969058wrm.54.2021.08.24.11.59.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 24 Aug 2021 11:59:23 -0700 (PDT) Subject: Re: [BUG 5.14] arm64/mm: dma memory mapping fails (in some cases) To: Robin Murphy , Mike Rapoport , Catalin Marinas Cc: Alex Bee , Will Deacon , Andrew Morton , Anshuman Khandual , Linux Kernel Mailing List , linux-mm@kvack.org, Linux ARM References: <20210824173741.GC623@arm.com> <0908ce39-7e30-91fa-68ef-11620f9596ae@arm.com> From: David Hildenbrand Organization: Red Hat Message-ID: <60a11eba-2910-3b5f-ef96-97d4556c1596@redhat.com> Date: Tue, 24 Aug 2021 20:59:22 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <0908ce39-7e30-91fa-68ef-11620f9596ae@arm.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=YzTXyPt3; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf02.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=david@redhat.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A4C027001702 X-Stat-Signature: iheupj43g3k6mo6fmkgiqp8yucko7gj4 X-HE-Tag: 1629831566-827456 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 24.08.21 20:46, Robin Murphy wrote: > On 2021-08-24 19:28, Mike Rapoport wrote: >> On Tue, Aug 24, 2021 at 06:37:41PM +0100, Catalin Marinas wrote: >>> Hi Alex, >>> >>> Thanks for the report. >>> >>> On Tue, Aug 24, 2021 at 03:40:47PM +0200, Alex Bee wrote: >>>> it seems there is a regression in arm64 memory mapping in 5.14, sinc= e it >>>> fails on Rockchip RK3328 when the pl330 dmac tries to map with: >>>> >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.921909] --= ----------[ cut here ]------------ >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.921940] WA= RNING: CPU: 2 PID: 373 at kernel/dma/mapping.c:235 dma_map_resource+0x68/= 0xc0 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.921973] Mo= dules linked in: spi_rockchip(+) fuse >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.921996] CP= U: 2 PID: 373 Comm: systemd-udevd Not tainted 5.14.0-rc7 #1 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922004] Ha= rdware name: Pine64 Rock64 (DT) >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922011] ps= tate: 80000005 (Nzcv daif -PAN -UAO -TCO BTYPE=3D--) >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922018] pc= : dma_map_resource+0x68/0xc0 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922026] lr= : pl330_prep_slave_fifo+0x78/0xd0 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922040] sp= : ffff800012102ae0 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922043] x2= 9: ffff800012102ae0 x28: ffff000005c94800 x27: 0000000000000000 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922056] x2= 6: ffff000000566bd0 x25: 0000000000000001 x24: 0000000000000001 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922067] x2= 3: 0000000000000002 x22: ffff000000628c00 x21: 0000000000000001 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922078] x2= 0: ffff000000566bd0 x19: 0000000000000001 x18: 0000000000000000 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922089] x1= 7: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922100] x1= 4: 0000000000000277 x13: 0000000000000001 x12: 0000000000000000 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922111] x1= 1: 0000000000000001 x10: 00000000000008e0 x9 : ffff800012102a80 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922123] x8= : ffff000000d14b80 x7 : ffff0000fe7b12f0 x6 : ffff0000fe7b1100 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922134] x5= : fffffc000000000f x4 : 0000000000000000 x3 : 0000000000000001 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922145] x2= : 0000000000000001 x1 : 00000000ff190800 x0 : ffff000000628c00 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922158] Ca= ll trace: >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922163]=C3= =AF=C2=BF=C2=BD dma_map_resource+0x68/0xc0 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922173]=C3= =AF=C2=BF=C2=BD pl330_prep_slave_sg+0x58/0x220 >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922181]=C3= =AF=C2=BF=C2=BD rockchip_spi_prepare_dma+0xd8/0x2c0 [spi_rockchip] >>>> [=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD=C3=AF=C2=BF=C2=BD 8.922208]=C3= =AF=C2=BF=C2=BD rockchip_spi_transfer_one+0x294/0x3d8 [spi_rockchip] >>> [...] >>>> Note: This does not relate to the spi driver - when disabling this d= evice in >>>> the device tree it fails for any other (i2s, for instance) which use= s dma. >>>> Commenting out the failing check at [1], however, helps and the mapp= ing >>>> works again. >> >>> Do you know which address dma_map_resource() is trying to map (maybe >>> add some printk())? It's not supposed to map RAM, hence the warning. >>> Random guess, the address is 0xff190800 (based on the x1 above but th= e >>> regs might as well be mangled). >> >> 0xff190800 will cause this warning for sure. It has a memory map, but = it is >> not RAM so old version of pfn_valid() would return 0 and the new one >> returns 1. >=20 > How does that happen, though? It's not a memory address, and it's not > even within the bounds of anywhere there should or could be memory. Thi= s > SoC has a simple memory map - everything from 0 to 0xfeffffff goes to > the DRAM controller (which may not all be populated, and may have piece= s > carved out by secure firmware), while 0xff000000-0xffffffff is MMIO. Wh= y > do we have pages (or at least the assumption of pages) for somewhere > which by all rights should not have them? Simple: we allocate the vmemmap for whole sections (e.g., 128 MiB) to=20 avoid any such hacks. If there is a memory hole, it gets a memmap as well= . Tricking pfn_valid() into returning "false" where we actually have a=20 memmap only makes it look like there is no memmap; but there is one, and it's PG_reserved. [...] >>> Either pfn_valid() gets confused in 5.14 or something is wrong with t= he >>> DT. I have a suspicion it's the former since reverting the above comm= it >>> makes it disappear. >> >> I think pfn_valid() actually behaves as expected but the caller is wro= ng >> because pfn_valid !=3D RAM (this applies btw to !arm64 as well). >> >> /* Don't allow RAM to be mapped */ >> if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr)))) >> return DMA_MAPPING_ERROR; >> >> Alex, can you please try this patch: >=20 > That will certainly paper over the issue, but it's avoiding the questio= n > of what went wrong with the memory map in the first place. The comment > is indeed a bit inaccurate, but ultimately dma_map_resource() exists fo= r > addresses that would be wrong to pass to dma_map_page(), so I believe > pfn_valid() is still the correct check. If we want to check for RAM, pfn_valid() would be wrong. If we want to=20 check for "is there a memmap, for whatever lives or does not live=20 there", pfn_valid() is the right check. --=20 Thanks, David / dhildenb