From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAC51C2BBCF for ; Fri, 18 Dec 2020 08:33:21 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 97EC220936 for ; Fri, 18 Dec 2020 08:33:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 97EC220936 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=uniontech.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 69EA589B66; Fri, 18 Dec 2020 08:32:18 +0000 (UTC) Received: from regular1.263xmail.com (regular1.263xmail.com [211.150.70.197]) by gabe.freedesktop.org (Postfix) with ESMTPS id C67DB891EB for ; Fri, 18 Dec 2020 06:14:51 +0000 (UTC) Received: from localhost (unknown [192.168.167.69]) by regular1.263xmail.com (Postfix) with ESMTP id 9D11D1BBD; Fri, 18 Dec 2020 14:14:46 +0800 (CST) X-MAIL-GRAY: 0 X-MAIL-DELIVERY: 1 X-ADDR-CHECKED4: 1 X-ANTISPAM-LEVEL: 2 X-SKE-CHECKED: 1 X-ABS-CHECKED: 1 Received: from firstlove-e500.uniontech.com (unknown [58.246.122.242]) by smtp.263.net (postfix) whith ESMTP id P9716T140427743995648S1608272085341061_; Fri, 18 Dec 2020 14:14:46 +0800 (CST) X-IP-DOMAINF: 1 X-UNIQUE-TAG: X-RL-SENDER: chenli@uniontech.com X-SENDER: chenli@uniontech.com X-LOGIN-NAME: chenli@uniontech.com X-FST-TO: robin.murphy@arm.com X-SENDER-IP: 58.246.122.242 X-ATTACHMENT-NUM: 0 X-System-Flag: 0 Date: Fri, 18 Dec 2020 14:14:52 +0800 Message-ID: <87wnxfy71f.wl-chenli@uniontech.com> From: Chen Li To: Robin Murphy Subject: Re: [PATCH] drm/[amdgpu|radeon]: fix memset on io mem In-Reply-To: <159c72db-1316-6155-2209-8e0e9a7f5224@arm.com> References: <877dpiz4sf.wl-chenli@uniontech.com> <4277816d-db00-7e81-a2fb-069aeee18e8b@amd.com> <875z51zwsq.wl-chenli@uniontech.com> <90b625e2-2409-d13b-2456-483ad4eef18f@amd.com> <873605z1du.wl-chenli@uniontech.com> <7920fd29-3f95-2109-07ee-15659e80dc40@amd.com> <159c72db-1316-6155-2209-8e0e9a7f5224@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-Mailman-Approved-At: Fri, 18 Dec 2020 08:32:02 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Deucher , Christian =?ISO-8859-1?Q?K=F6nig?= , dri-devel@lists.freedesktop.org, Chen Li Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Thu, 17 Dec 2020 21:45:06 +0800, Robin Murphy wrote: > = > On 2020-12-17 10:25, Christian K=F6nig wrote: > > Am 17.12.20 um 02:07 schrieb Chen Li: > >> On Wed, 16 Dec 2020 22:19:11 +0800, > >> Christian K=F6nig wrote: > >>> Am 16.12.20 um 14:48 schrieb Chen Li: > >>>> On Wed, 16 Dec 2020 15:59:37 +0800, > >>>> Christian K=F6nig wrote: > >>>>> [SNIP] > >>>> Hi, Christian. I'm not sure why this change is a hack here. I cannot= see > >>>> the problem and wll be grateful if you give more explainations. > >>> __memset is supposed to work on those addresses, otherwise you can't = use the > >>> e8860 on your arm64 system. > >> If __memset is supposed to work on those adresses, why this > >> commit(https://nam11.safelinks.protection.outlook.com/?url=3Dhttps%3A%= 2F%2Fgithub.com%2Ftorvalds%2Flinux%2Fcommit%2Fba0b2275a6781b2f3919d931d6332= 9b5548f6d5f&data=3D04%7C01%7Cchristian.koenig%40amd.com%7C4ed3c07588874= 6b7f41408d8a22811c5%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C6374376402= 74023350%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi= I6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DHhWxUaLo3WpzoV6hjV%2BG1HICaIOXw= soNpzv5tNMNg8A%3D&reserved=3D0) > >> is needed? (I also notice drm/radeon didn't take this change though) j= ust out > >> of curiosity. > > = > > We generally accept those patches as cleanup in the kernel with the hop= e that > > we can find a way to work around the userspace restrictions. > > = > > But when you also have this issue in userspace then there isn't much we= can do > > for you. > > = > >>> Replacing the the direct write in the kernel with calls to writel() or > >>> memset_io() will fix that temporary, but you have a more general prob= lem > >>> here. > >> I cannot see what's the more general problem here :( u mean performanc= e? > > = > > No, not performance. See standards like OpenGL, Vulkan as well as VA-AP= I and > > VDPAU require that you can mmap() device memory and execute memset/memc= py on > > the memory from userspace. > > = > > If your ARM base board can't do that for some then you can't use the ha= rdware > > with that board. > = > If the VRAM lives in a prefetchable PCI bar then on most sane Arm-based s= ystems > I believe it should be able to mmap() to userspace with the Normal memory= type, > where unaligned accesses and such are allowed, as opposed to the Device m= emory > type intended for MMIO mappings, which has more restrictions but stricter > ordering guarantees. = Hi, Robin. I cannot understand it allow unaligned accesses. prefetchable PC= I bar should also be mmio, and accesses will end with device memory, so why= does this allow unaligned access? > Regardless of what happens elsewhere though, if something is mapped *into= the > kernel* with ioremap(), then it is fundamentally wrong per the kernel mem= ory > model to reference that mapping directly without using I/O accessors. Tha= t is > not specific to any individual architecture, and Sparse should be screami= ng > about it already. I guess in this case the UVD code needs to pay more att= ention > to whether radeon_bo_kmap() ends up going via ttm_bo_ioremap() or not. > = > (I'm assuming the initial fault was memset() with 0 trying to perform "DC= ZVA" > on a Device-type mapping from ioremap() - FYI a stacktrace on its own wit= hout > the rest of the error dump showing what actually triggered it isn't overly > useful) > = > Robin. why it may be 'DC ZVA'? I'm not sure the pc in initial kernel fault memset,= but I capture the userspace crash pc: stp(128bit) or str with neon(also 12= 8bit) to render node(/dev/dri/renderD128). = > >>>>> For amdgpu I suggest that we allocate the UVD message in GTT instea= d of > >>>>> VRAM > >>>>> since we don't have the hardware restriction for that on the new > >>>>> generations. > >>>>> = > >>>> Thanks, I will try to dig into deeper. But what's the "hardware > >>>> restriction" meaning here? I'm not familiar with video driver stack = and amd > >>>> gpu, sorry. > >>> On older hardware (AGP days) the buffer had to be in VRAM (MMIO) memo= ry, but > >>> on > >>> modern system GTT (system memory) works as well. > >> IIUC, e8860 can use amdgpu(I use radeon now) beause its device id 6822= is in > >> amdgpu's table. But I cannot tell whether e8860 has iommu, and I canno= t find > >> iommu from lspci, so graphics translation table may not work here? > > = > > That is not related to IOMMU. IOMMU is a feature of the CPU/motherboard= . This > > is implemented using GTT, e.g. the VM page tables inside the GPU. > > = > > And yes it should work I will prepare a patch for it. > > = > >>>>> BTW: How does userspace work on arm64 then? The driver stack usuall= y only > >>>>> works > >>>>> if mmio can be mapped directly. > >>>> I also post two usespace issue on mesa, and you may be interested wi= th > >>>> them: > >>>> https://nam11.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F= %2Fgitlab.freedesktop.org%2Fmesa%2Fmesa%2F-%2Fissues%2F3954&data=3D04%7= C01%7Cchristian.koenig%40amd.com%7C4ed3c075888746b7f41408d8a22811c5%7C3dd89= 61fe4884e608e11a82d994e183d%7C0%7C0%7C637437640274023350%7CUnknown%7CTWFpbG= Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C= 1000&sdata=3DZR7pDS%2BCLUuMjCeKcMAXfHtbczt8WdUwSeLZCuHfCHw%3D&reser= ved=3D0 = > >>>> https://nam11.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F= %2Fgitlab.freedesktop.org%2Fmesa%2Fmesa%2F-%2Fissues%2F3951&data=3D04%7= C01%7Cchristian.koenig%40amd.com%7C4ed3c075888746b7f41408d8a22811c5%7C3dd89= 61fe4884e608e11a82d994e183d%7C0%7C0%7C637437640274033344%7CUnknown%7CTWFpbG= Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C= 1000&sdata=3DjAJo3aG2I1oIDTZXWhNgcKoKbd6tTdiAtc7vE4hJJPY%3D&reserve= d=3D0 = > >>>> I paste some virtual memory map in userspace there. (and the two pro= blems > >>>> do bother me quite a long time.) > >>> I don't really see a solution for those problems. > >>> = > >>> See it is perfectly valid for an application to memset/memcpy on mmap= ed MMIO > >>> space which comes from OpenGL or Vulkan. > >>> = > >>> So your CPU simply won't work with the hardware. We could work around= that > >>> with > >>> a couple of hacks, but this is a pretty much general problem. > >>> = > >>> Regards, > >>> Christian. > >> Thanks! Can you provid some details about these hacks? Should I post a= nother > >> issue on the mail list? > > = > > Adjust the kernel and/or user space to never map VRAM to the CPU. > > = > > This violates the OpenGL/Vulkan specification in some ways. So not sure= if > > that will work or not. > > = > > Regards, > > Christian. > > _______________________________________________ > > dri-devel mailing list > > dri-devel@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/dri-devel > = > = _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel