From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9342C433E0 for ; Sat, 27 Mar 2021 15:31:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A1663619BB for ; Sat, 27 Mar 2021 15:31:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230213AbhC0PbJ (ORCPT ); Sat, 27 Mar 2021 11:31:09 -0400 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:55929 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230086AbhC0PbF (ORCPT ); Sat, 27 Mar 2021 11:31:05 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id DC64858043D; Sat, 27 Mar 2021 11:31:04 -0400 (EDT) Received: from imap21 ([10.202.2.71]) by compute3.internal (MEProxy); Sat, 27 Mar 2021 11:31:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svenpeter.dev; h=mime-version:message-id:in-reply-to:references:date:from:to :cc:subject:content-type; s=fm1; bh=x2m3pCDfFJux8o5j+dJWq42C3lpC 8BI/9Q73L1x239U=; b=UDIqOyXp4GWRmI2pzioG2zuL3bKG4h5t1oaqrMF+A9/U WD6G5vyt+h7rjWV/2rwtSLbM79Vc6xOB+j5gjjCrs5su1k2oL+8cfR/ZTns61zkY /ZgRT37gx/5oJkS578CTTr4NyZoB20uA0Pixc3tqyAEaRNWy+oFtztYFIzWZe+WV gZMIYrwa3xPb63OrvhNqjMn8AO4NetnpDRAIE/gx1ae8Y0VJbJUdRI0Wuf9v0MM0 7qby21vBt8sfsJ1HUeBUUYqYnp+ZpczL2CZrfHaifor5yodI6/JUM7iAW3YPo2tP mf5gEGpTAbMa9WFRQ/FgmLTJV7MNsF6Z5hrTanQmHg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=x2m3pC DfFJux8o5j+dJWq42C3lpC8BI/9Q73L1x239U=; b=Qm1zA1/fn0kTeXiwtso6EV aQkQ4AY2qawCC2OVtoUViubYu+WsOw1jQwFekQ8xdoF5uuCZdryL939THsvpPPzU 4rOR3Km4Mng0Iq+Ah9/Q10bk8RibabwJ0Sys1zcKzV6bdMnIEJ4TNDNqF+sOHcoI p6qNu/yWphfM6qoNqG/cKhLRIrlKOCvar1RjqtpS/tgLAfBgyaKS3DreVCoWKyjY eseRKUzeRP4tHuKd1o5v1/9OpTJU5Wq/9axhSQ9Lk9ZkMekn2FggipTDkym9eBmW 9RnEFvi5h71ATTWWJy6ftG2jq7msXnnLBGd+dIFfAuRtnxw65itdDxKZLyy7DTPw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudehgedgjeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesthdtredtreertdenucfhrhhomhepfdfuvhgv nhcurfgvthgvrhdfuceoshhvvghnsehsvhgvnhhpvghtvghrrdguvghvqeenucggtffrrg htthgvrhhnpefgieegieffuefhtedtjefgteejteefleefgfefgfdvvddtgffhffduhedv feekffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hsvhgvnhesshhvvghnphgvthgvrhdruggvvh X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 732A051C005E; Sat, 27 Mar 2021 11:31:02 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.5.0-alpha0-273-g8500d2492d-fm-20210323.002-g8500d249 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <20210320151903.60759-1-sven@svenpeter.dev> <20210323205346.GA1283560@robh.at.kernel.org> <43685c67-6d9c-4e72-b320-0462c2273bf0@www.fastmail.com> <9f06872d-f0ec-43c3-9b53-d144337100b3@www.fastmail.com> <45faaadd-eda7-464f-96ff-7324f566669e@www.fastmail.com> Date: Sat, 27 Mar 2021 16:30:41 +0100 From: "Sven Peter" To: "Arnd Bergmann" Cc: "Robin Murphy" , "Mark Kettenis" , "Rob Herring" , "open list:IOMMU DRIVERS" , "Joerg Roedel" , "Will Deacon" , "Hector Martin" , "Marc Zyngier" , "Mohamed Mediouni" , "Stan Skowronek" , "Linux ARM" , "Linux Kernel Mailing List" , DTML Subject: Re: [PATCH 0/3] Apple M1 DART IOMMU driver Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Mar 26, 2021, at 20:59, Arnd Bergmann wrote: > On Fri, Mar 26, 2021 at 6:51 PM Sven Peter wrote: > > dart-sio: 0021c000 fbde4000 (at least their Secure Enclave/TPM co-processor) > > Same here: > dart-sio { > vm-base = <0x0>; > vm-size = <0xfc000000>; > pio-vm-base = <0xfd000000>; > pio-vm-size = <0x2000000>; > pio-granularity = <0x1000000>; > } > > There are clearly two distinct ranges that split up the 4GB space again, > with a small hole of 16MB (==pio-granularity) at the end of each range. > > The "pio" name might indicate that this is a range of addresses that > can be programmed to point at I/O registers in another device, rather > than pointing to RAM. > > Arnd > Very interesting observation! Mark and I have discussed this a little bit further on IRC. Mark also successfully used the PCIe DARTs with a DMA window outside of the one specified by vm-base/vm-size in the ADT. I believe that the (pio-)vm-base/size properties merely specify the ranges their allocator uses and do not describe actual hardware limitations. Mark also suggested that they might reserve memory at the beginning to find bugs similar to how one might not allow to map memory at 0x0. I have also done a few more experiments and figured out that if I put the IOMMU into bypass mode (which doesn't seem to work for all IOMMUs/master combinations which is why I'll leave it out of this series for now until I figure out more details) I *can* use the full address space. I think the limitation is therefore imposed by the translation hardware inside the IOMMU and not by the bus/the interconnect. If that's correct I think the right place to enforce this is to just limit the aperture inside the DART driver to a 32bit address space whenever address translation is enabled. Thanks, Sven From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_RED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D574C433DB for ; Sat, 27 Mar 2021 15:31:13 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DDA6D61991 for ; Sat, 27 Mar 2021 15:31:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DDA6D61991 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lists.linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9EEB64063C; Sat, 27 Mar 2021 15:31:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FmOV7dg-9Xvu; Sat, 27 Mar 2021 15:31:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTP id 3B1EB40628; Sat, 27 Mar 2021 15:31:11 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 003F1C000B; Sat, 27 Mar 2021 15:31:11 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1FC78C000A for ; Sat, 27 Mar 2021 15:31:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0722784D91 for ; Sat, 27 Mar 2021 15:31:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=svenpeter.dev header.b="UDIqOyXp"; dkim=pass (2048-bit key) header.d=messagingengine.com header.b="Qm1zA1/f" Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 175zorJ0XEf2 for ; Sat, 27 Mar 2021 15:31:08 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com [66.111.4.230]) by smtp1.osuosl.org (Postfix) with ESMTPS id EBE9784D8C for ; Sat, 27 Mar 2021 15:31:07 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id DC64858043D; Sat, 27 Mar 2021 11:31:04 -0400 (EDT) Received: from imap21 ([10.202.2.71]) by compute3.internal (MEProxy); Sat, 27 Mar 2021 11:31:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svenpeter.dev; h=mime-version:message-id:in-reply-to:references:date:from:to :cc:subject:content-type; s=fm1; bh=x2m3pCDfFJux8o5j+dJWq42C3lpC 8BI/9Q73L1x239U=; b=UDIqOyXp4GWRmI2pzioG2zuL3bKG4h5t1oaqrMF+A9/U WD6G5vyt+h7rjWV/2rwtSLbM79Vc6xOB+j5gjjCrs5su1k2oL+8cfR/ZTns61zkY /ZgRT37gx/5oJkS578CTTr4NyZoB20uA0Pixc3tqyAEaRNWy+oFtztYFIzWZe+WV gZMIYrwa3xPb63OrvhNqjMn8AO4NetnpDRAIE/gx1ae8Y0VJbJUdRI0Wuf9v0MM0 7qby21vBt8sfsJ1HUeBUUYqYnp+ZpczL2CZrfHaifor5yodI6/JUM7iAW3YPo2tP mf5gEGpTAbMa9WFRQ/FgmLTJV7MNsF6Z5hrTanQmHg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=x2m3pC DfFJux8o5j+dJWq42C3lpC8BI/9Q73L1x239U=; b=Qm1zA1/fn0kTeXiwtso6EV aQkQ4AY2qawCC2OVtoUViubYu+WsOw1jQwFekQ8xdoF5uuCZdryL939THsvpPPzU 4rOR3Km4Mng0Iq+Ah9/Q10bk8RibabwJ0Sys1zcKzV6bdMnIEJ4TNDNqF+sOHcoI p6qNu/yWphfM6qoNqG/cKhLRIrlKOCvar1RjqtpS/tgLAfBgyaKS3DreVCoWKyjY eseRKUzeRP4tHuKd1o5v1/9OpTJU5Wq/9axhSQ9Lk9ZkMekn2FggipTDkym9eBmW 9RnEFvi5h71ATTWWJy6ftG2jq7msXnnLBGd+dIFfAuRtnxw65itdDxKZLyy7DTPw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudehgedgjeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesthdtredtreertdenucfhrhhomhepfdfuvhgv nhcurfgvthgvrhdfuceoshhvvghnsehsvhgvnhhpvghtvghrrdguvghvqeenucggtffrrg htthgvrhhnpefgieegieffuefhtedtjefgteejteefleefgfefgfdvvddtgffhffduhedv feekffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hsvhgvnhesshhvvghnphgvthgvrhdruggvvh X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 732A051C005E; Sat, 27 Mar 2021 11:31:02 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.5.0-alpha0-273-g8500d2492d-fm-20210323.002-g8500d249 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <20210320151903.60759-1-sven@svenpeter.dev> <20210323205346.GA1283560@robh.at.kernel.org> <43685c67-6d9c-4e72-b320-0462c2273bf0@www.fastmail.com> <9f06872d-f0ec-43c3-9b53-d144337100b3@www.fastmail.com> <45faaadd-eda7-464f-96ff-7324f566669e@www.fastmail.com> Date: Sat, 27 Mar 2021 16:30:41 +0100 To: "Arnd Bergmann" Subject: Re: [PATCH 0/3] Apple M1 DART IOMMU driver Cc: Rob Herring , DTML , Will Deacon , Hector Martin , Linux Kernel Mailing List , "open list:IOMMU DRIVERS" , Marc Zyngier , Mohamed Mediouni , Stan Skowronek , Robin Murphy , Linux ARM , Mark Kettenis X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sven Peter via iommu Reply-To: Sven Peter Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Fri, Mar 26, 2021, at 20:59, Arnd Bergmann wrote: > On Fri, Mar 26, 2021 at 6:51 PM Sven Peter wrote: > > dart-sio: 0021c000 fbde4000 (at least their Secure Enclave/TPM co-processor) > > Same here: > dart-sio { > vm-base = <0x0>; > vm-size = <0xfc000000>; > pio-vm-base = <0xfd000000>; > pio-vm-size = <0x2000000>; > pio-granularity = <0x1000000>; > } > > There are clearly two distinct ranges that split up the 4GB space again, > with a small hole of 16MB (==pio-granularity) at the end of each range. > > The "pio" name might indicate that this is a range of addresses that > can be programmed to point at I/O registers in another device, rather > than pointing to RAM. > > Arnd > Very interesting observation! Mark and I have discussed this a little bit further on IRC. Mark also successfully used the PCIe DARTs with a DMA window outside of the one specified by vm-base/vm-size in the ADT. I believe that the (pio-)vm-base/size properties merely specify the ranges their allocator uses and do not describe actual hardware limitations. Mark also suggested that they might reserve memory at the beginning to find bugs similar to how one might not allow to map memory at 0x0. I have also done a few more experiments and figured out that if I put the IOMMU into bypass mode (which doesn't seem to work for all IOMMUs/master combinations which is why I'll leave it out of this series for now until I figure out more details) I *can* use the full address space. I think the limitation is therefore imposed by the translation hardware inside the IOMMU and not by the bus/the interconnect. If that's correct I think the right place to enforce this is to just limit the aperture inside the DART driver to a 32bit address space whenever address translation is enabled. Thanks, Sven _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C431C433DB for ; Sat, 27 Mar 2021 15:32:59 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0994260235 for ; Sat, 27 Mar 2021 15:32:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0994260235 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=svenpeter.dev Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Subject:Cc:To:From:Date:References:In-Reply-To: Message-Id:Mime-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+uMY4w1/sAxPagNJmJdfRs1fp1Xu2qOzOYKsTKSzjXU=; b=Fw3vH5eMtOxC6bg69oED9w5rQ nkrymLHYeW9gBGjGwJPCSfs7jY+Tgoit4oQ5tfNmTyU8v5CaGK+hocW6la4xAkZW4/EuMjpdN6WG5 0VG97UVJknbsVylItvCPWslsu8c2I/NsEMPwh9Puk/rGHtM3WjVULtXnQuxcZ3vo5Z3XM2to41Gal /4obUrHH/cICjUh6wO8+3MbFKypcfoUAfP6n4jV2iWl+5fH1D1kthG7CtWKEhVqtuIy3yq3ucssa7 bf7+TURBaLy8AUbhtB0UrZnNF1vIRA+3eVt4UeLPoWuDUyNMzK2mPqpV8szWiYLMlXlBbrLis5I+V jYl58iMCw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lQAu9-005E3b-Ua; Sat, 27 Mar 2021 15:31:14 +0000 Received: from new4-smtp.messagingengine.com ([66.111.4.230]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lQAu4-005E3F-Ru for linux-arm-kernel@lists.infradead.org; Sat, 27 Mar 2021 15:31:11 +0000 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id DC64858043D; Sat, 27 Mar 2021 11:31:04 -0400 (EDT) Received: from imap21 ([10.202.2.71]) by compute3.internal (MEProxy); Sat, 27 Mar 2021 11:31:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=svenpeter.dev; h=mime-version:message-id:in-reply-to:references:date:from:to :cc:subject:content-type; s=fm1; bh=x2m3pCDfFJux8o5j+dJWq42C3lpC 8BI/9Q73L1x239U=; b=UDIqOyXp4GWRmI2pzioG2zuL3bKG4h5t1oaqrMF+A9/U WD6G5vyt+h7rjWV/2rwtSLbM79Vc6xOB+j5gjjCrs5su1k2oL+8cfR/ZTns61zkY /ZgRT37gx/5oJkS578CTTr4NyZoB20uA0Pixc3tqyAEaRNWy+oFtztYFIzWZe+WV gZMIYrwa3xPb63OrvhNqjMn8AO4NetnpDRAIE/gx1ae8Y0VJbJUdRI0Wuf9v0MM0 7qby21vBt8sfsJ1HUeBUUYqYnp+ZpczL2CZrfHaifor5yodI6/JUM7iAW3YPo2tP mf5gEGpTAbMa9WFRQ/FgmLTJV7MNsF6Z5hrTanQmHg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=x2m3pC DfFJux8o5j+dJWq42C3lpC8BI/9Q73L1x239U=; b=Qm1zA1/fn0kTeXiwtso6EV aQkQ4AY2qawCC2OVtoUViubYu+WsOw1jQwFekQ8xdoF5uuCZdryL939THsvpPPzU 4rOR3Km4Mng0Iq+Ah9/Q10bk8RibabwJ0Sys1zcKzV6bdMnIEJ4TNDNqF+sOHcoI p6qNu/yWphfM6qoNqG/cKhLRIrlKOCvar1RjqtpS/tgLAfBgyaKS3DreVCoWKyjY eseRKUzeRP4tHuKd1o5v1/9OpTJU5Wq/9axhSQ9Lk9ZkMekn2FggipTDkym9eBmW 9RnEFvi5h71ATTWWJy6ftG2jq7msXnnLBGd+dIFfAuRtnxw65itdDxKZLyy7DTPw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudehgedgjeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesthdtredtreertdenucfhrhhomhepfdfuvhgv nhcurfgvthgvrhdfuceoshhvvghnsehsvhgvnhhpvghtvghrrdguvghvqeenucggtffrrg htthgvrhhnpefgieegieffuefhtedtjefgteejteefleefgfefgfdvvddtgffhffduhedv feekffenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hsvhgvnhesshhvvghnphgvthgvrhdruggvvh X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 732A051C005E; Sat, 27 Mar 2021 11:31:02 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.5.0-alpha0-273-g8500d2492d-fm-20210323.002-g8500d249 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <20210320151903.60759-1-sven@svenpeter.dev> <20210323205346.GA1283560@robh.at.kernel.org> <43685c67-6d9c-4e72-b320-0462c2273bf0@www.fastmail.com> <9f06872d-f0ec-43c3-9b53-d144337100b3@www.fastmail.com> <45faaadd-eda7-464f-96ff-7324f566669e@www.fastmail.com> Date: Sat, 27 Mar 2021 16:30:41 +0100 From: "Sven Peter" To: "Arnd Bergmann" Cc: "Robin Murphy" , "Mark Kettenis" , "Rob Herring" , "open list:IOMMU DRIVERS" , "Joerg Roedel" , "Will Deacon" , "Hector Martin" , "Marc Zyngier" , "Mohamed Mediouni" , "Stan Skowronek" , "Linux ARM" , "Linux Kernel Mailing List" , DTML Subject: Re: [PATCH 0/3] Apple M1 DART IOMMU driver X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210327_153109_408152_44590AD8 X-CRM114-Status: GOOD ( 14.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Mar 26, 2021, at 20:59, Arnd Bergmann wrote: > On Fri, Mar 26, 2021 at 6:51 PM Sven Peter wrote: > > dart-sio: 0021c000 fbde4000 (at least their Secure Enclave/TPM co-processor) > > Same here: > dart-sio { > vm-base = <0x0>; > vm-size = <0xfc000000>; > pio-vm-base = <0xfd000000>; > pio-vm-size = <0x2000000>; > pio-granularity = <0x1000000>; > } > > There are clearly two distinct ranges that split up the 4GB space again, > with a small hole of 16MB (==pio-granularity) at the end of each range. > > The "pio" name might indicate that this is a range of addresses that > can be programmed to point at I/O registers in another device, rather > than pointing to RAM. > > Arnd > Very interesting observation! Mark and I have discussed this a little bit further on IRC. Mark also successfully used the PCIe DARTs with a DMA window outside of the one specified by vm-base/vm-size in the ADT. I believe that the (pio-)vm-base/size properties merely specify the ranges their allocator uses and do not describe actual hardware limitations. Mark also suggested that they might reserve memory at the beginning to find bugs similar to how one might not allow to map memory at 0x0. I have also done a few more experiments and figured out that if I put the IOMMU into bypass mode (which doesn't seem to work for all IOMMUs/master combinations which is why I'll leave it out of this series for now until I figure out more details) I *can* use the full address space. I think the limitation is therefore imposed by the translation hardware inside the IOMMU and not by the bus/the interconnect. If that's correct I think the right place to enforce this is to just limit the aperture inside the DART driver to a 32bit address space whenever address translation is enabled. Thanks, Sven _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel