From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF7ADC2B9F7 for ; Wed, 26 May 2021 12:13:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9EB98613CE for ; Wed, 26 May 2021 12:13:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234699AbhEZMPI (ORCPT ); Wed, 26 May 2021 08:15:08 -0400 Received: from mail.kernel.org ([198.145.29.99]:50730 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233784AbhEZMPF (ORCPT ); Wed, 26 May 2021 08:15:05 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DB6A613D6; Wed, 26 May 2021 12:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1622031214; bh=6cgAAAry3KXjHYICFlfgug7HP7MtfiezqAbh5vyjbAk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dD4raOXl4m/1dmUYxCXd+9Zhh/XbrtQC52Qny8ZwVBaqmrNGpiCc5ZIU1b4WjGm8H uJa3jIZybN6M6kTMpAfuccsLT4/wRQbg2MVSjmE1yL01mC3lcM16NVvL5naGjFRQ57 RUc8/qmG6+LKY4mt6jNuAHGXfU60PflHdIRn81RXVX/L8quJSpzl4wYs9fdYRMYSyz TDHUTCxqGOaTMeMxI6d2iUlniRYHOiu06+32s75wchffQ8s3cbbJwvz7Jm6fWP5KI+ RznUS6G8WImqfu6sIHm02R4cmQSm1/c2dk4BLfXSL89hrwovsmQduosVw5UITnC7eb z+DGv0xJWhG6g== Date: Wed, 26 May 2021 13:13:23 +0100 From: Will Deacon To: Claire Chang Cc: Rob Herring , mpe@ellerman.id.au, Joerg Roedel , Frank Rowand , Konrad Rzeszutek Wilk , boris.ostrovsky@oracle.com, jgross@suse.com, Christoph Hellwig , Marek Szyprowski , benh@kernel.crashing.org, paulus@samba.org, "list@263.net:IOMMU DRIVERS" , sstabellini@kernel.org, Robin Murphy , grant.likely@arm.com, xypron.glpk@gmx.de, Thierry Reding , mingo@kernel.org, bauerman@linux.ibm.com, peterz@infradead.org, Greg KH , Saravana Kannan , "Rafael J . Wysocki" , heikki.krogerus@linux.intel.com, Andy Shevchenko , Randy Dunlap , Dan Williams , Bartosz Golaszewski , linux-devicetree , lkml , linuxppc-dev@lists.ozlabs.org, xen-devel@lists.xenproject.org, Nicolas Boichat , Jim Quinlan , tfiga@chromium.org, bskeggs@redhat.com, bhelgaas@google.com, chris@chris-wilson.co.uk, daniel@ffwll.ch, airlied@linux.ie, dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, jani.nikula@linux.intel.com, jxgao@google.com, joonas.lahtinen@linux.intel.com, linux-pci@vger.kernel.org, maarten.lankhorst@linux.intel.com, matthew.auld@intel.com, rodrigo.vivi@intel.com, thomas.hellstrom@linux.intel.com Subject: Re: [PATCH v7 14/15] dt-bindings: of: Add restricted DMA pool Message-ID: <20210526121322.GA19313@willie-the-truck> References: <20210518064215.2856977-1-tientzu@chromium.org> <20210518064215.2856977-15-tientzu@chromium.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210518064215.2856977-15-tientzu@chromium.org> User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Claire, On Tue, May 18, 2021 at 02:42:14PM +0800, Claire Chang wrote: > Introduce the new compatible string, restricted-dma-pool, for restricted > DMA. One can specify the address and length of the restricted DMA memory > region by restricted-dma-pool in the reserved-memory node. > > Signed-off-by: Claire Chang > --- > .../reserved-memory/reserved-memory.txt | 27 +++++++++++++++++++ > 1 file changed, 27 insertions(+) > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > index e8d3096d922c..284aea659015 100644 > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > @@ -51,6 +51,23 @@ compatible (optional) - standard definition > used as a shared pool of DMA buffers for a set of devices. It can > be used by an operating system to instantiate the necessary pool > management subsystem if necessary. > + - restricted-dma-pool: This indicates a region of memory meant to be > + used as a pool of restricted DMA buffers for a set of devices. The > + memory region would be the only region accessible to those devices. > + When using this, the no-map and reusable properties must not be set, > + so the operating system can create a virtual mapping that will be used > + for synchronization. The main purpose for restricted DMA is to > + mitigate the lack of DMA access control on systems without an IOMMU, > + which could result in the DMA accessing the system memory at > + unexpected times and/or unexpected addresses, possibly leading to data > + leakage or corruption. The feature on its own provides a basic level > + of protection against the DMA overwriting buffer contents at > + unexpected times. However, to protect against general data leakage and > + system memory corruption, the system needs to provide way to lock down > + the memory access, e.g., MPU. Note that since coherent allocation > + needs remapping, one must set up another device coherent pool by > + shared-dma-pool and use dma_alloc_from_dev_coherent instead for atomic > + coherent allocation. > - vendor specific string in the form ,[-] > no-map (optional) - empty property > - Indicates the operating system must not create a virtual mapping > @@ -120,6 +137,11 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > compatible = "acme,multimedia-memory"; > reg = <0x77000000 0x4000000>; > }; > + > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > + compatible = "restricted-dma-pool"; > + reg = <0x50000000 0x400000>; > + }; nit: You need to update the old text that states "This example defines 3 contiguous regions ...". > }; > > /* ... */ > @@ -138,4 +160,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > memory-region = <&multimedia_reserved>; > /* ... */ > }; > + > + pcie_device: pcie_device@0,0 { > + memory-region = <&restricted_dma_mem_reserved>; > + /* ... */ > + }; I still don't understand how this works for individual PCIe devices -- how is dev->of_node set to point at the node you have above? I tried adding the memory-region to the host controller instead, and then I see it crop up in dmesg: | pci-host-generic 40000000.pci: assigned reserved memory node restricted_dma_mem_reserved but none of the actual PCI devices end up with 'dma_io_tlb_mem' set, and so the restricted DMA area is not used. In fact, swiotlb isn't used at all. What am I missing to make this work with PCIe devices? Thanks, Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5312BC2B9F7 for ; Wed, 26 May 2021 12:14:08 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B7FEB613CE for ; Wed, 26 May 2021 12:14:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B7FEB613CE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4FqqcZ2vFBz308x for ; Wed, 26 May 2021 22:14:06 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=dD4raOXl; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=198.145.29.99; helo=mail.kernel.org; envelope-from=will@kernel.org; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=dD4raOXl; dkim-atps=neutral Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4Fqqc06rqqz2xfk for ; Wed, 26 May 2021 22:13:36 +1000 (AEST) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DB6A613D6; Wed, 26 May 2021 12:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1622031214; bh=6cgAAAry3KXjHYICFlfgug7HP7MtfiezqAbh5vyjbAk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dD4raOXl4m/1dmUYxCXd+9Zhh/XbrtQC52Qny8ZwVBaqmrNGpiCc5ZIU1b4WjGm8H uJa3jIZybN6M6kTMpAfuccsLT4/wRQbg2MVSjmE1yL01mC3lcM16NVvL5naGjFRQ57 RUc8/qmG6+LKY4mt6jNuAHGXfU60PflHdIRn81RXVX/L8quJSpzl4wYs9fdYRMYSyz TDHUTCxqGOaTMeMxI6d2iUlniRYHOiu06+32s75wchffQ8s3cbbJwvz7Jm6fWP5KI+ RznUS6G8WImqfu6sIHm02R4cmQSm1/c2dk4BLfXSL89hrwovsmQduosVw5UITnC7eb z+DGv0xJWhG6g== Date: Wed, 26 May 2021 13:13:23 +0100 From: Will Deacon To: Claire Chang Subject: Re: [PATCH v7 14/15] dt-bindings: of: Add restricted DMA pool Message-ID: <20210526121322.GA19313@willie-the-truck> References: <20210518064215.2856977-1-tientzu@chromium.org> <20210518064215.2856977-15-tientzu@chromium.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210518064215.2856977-15-tientzu@chromium.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: heikki.krogerus@linux.intel.com, thomas.hellstrom@linux.intel.com, peterz@infradead.org, joonas.lahtinen@linux.intel.com, dri-devel@lists.freedesktop.org, chris@chris-wilson.co.uk, grant.likely@arm.com, paulus@samba.org, Frank Rowand , mingo@kernel.org, Marek Szyprowski , sstabellini@kernel.org, Saravana Kannan , Joerg Roedel , "Rafael J . Wysocki" , Christoph Hellwig , Bartosz Golaszewski , bskeggs@redhat.com, linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org, Thierry Reding , intel-gfx@lists.freedesktop.org, matthew.auld@intel.com, linux-devicetree , jxgao@google.com, daniel@ffwll.ch, Konrad Rzeszutek Wilk , maarten.lankhorst@linux.intel.com, airlied@linux.ie, Dan Williams , linuxppc-dev@lists.ozlabs.org, jani.nikula@linux.intel.com, Rob Herring , rodrigo.vivi@intel.com, bhelgaas@google.com, boris.ostrovsky@oracle.com, Andy Shevchenko , jgross@suse.com, Nicolas Boichat , Greg KH , Randy Dunlap , lkml , tfiga@chromium.org, "list@263.net:IOMMU DRIVERS" , Jim Quinlan , xypron.glpk@gmx.de, Robin Murphy , bauerman@linux.ibm.com Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Hi Claire, On Tue, May 18, 2021 at 02:42:14PM +0800, Claire Chang wrote: > Introduce the new compatible string, restricted-dma-pool, for restricted > DMA. One can specify the address and length of the restricted DMA memory > region by restricted-dma-pool in the reserved-memory node. > > Signed-off-by: Claire Chang > --- > .../reserved-memory/reserved-memory.txt | 27 +++++++++++++++++++ > 1 file changed, 27 insertions(+) > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > index e8d3096d922c..284aea659015 100644 > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > @@ -51,6 +51,23 @@ compatible (optional) - standard definition > used as a shared pool of DMA buffers for a set of devices. It can > be used by an operating system to instantiate the necessary pool > management subsystem if necessary. > + - restricted-dma-pool: This indicates a region of memory meant to be > + used as a pool of restricted DMA buffers for a set of devices. The > + memory region would be the only region accessible to those devices. > + When using this, the no-map and reusable properties must not be set, > + so the operating system can create a virtual mapping that will be used > + for synchronization. The main purpose for restricted DMA is to > + mitigate the lack of DMA access control on systems without an IOMMU, > + which could result in the DMA accessing the system memory at > + unexpected times and/or unexpected addresses, possibly leading to data > + leakage or corruption. The feature on its own provides a basic level > + of protection against the DMA overwriting buffer contents at > + unexpected times. However, to protect against general data leakage and > + system memory corruption, the system needs to provide way to lock down > + the memory access, e.g., MPU. Note that since coherent allocation > + needs remapping, one must set up another device coherent pool by > + shared-dma-pool and use dma_alloc_from_dev_coherent instead for atomic > + coherent allocation. > - vendor specific string in the form ,[-] > no-map (optional) - empty property > - Indicates the operating system must not create a virtual mapping > @@ -120,6 +137,11 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > compatible = "acme,multimedia-memory"; > reg = <0x77000000 0x4000000>; > }; > + > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > + compatible = "restricted-dma-pool"; > + reg = <0x50000000 0x400000>; > + }; nit: You need to update the old text that states "This example defines 3 contiguous regions ...". > }; > > /* ... */ > @@ -138,4 +160,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > memory-region = <&multimedia_reserved>; > /* ... */ > }; > + > + pcie_device: pcie_device@0,0 { > + memory-region = <&restricted_dma_mem_reserved>; > + /* ... */ > + }; I still don't understand how this works for individual PCIe devices -- how is dev->of_node set to point at the node you have above? I tried adding the memory-region to the host controller instead, and then I see it crop up in dmesg: | pci-host-generic 40000000.pci: assigned reserved memory node restricted_dma_mem_reserved but none of the actual PCI devices end up with 'dma_io_tlb_mem' set, and so the restricted DMA area is not used. In fact, swiotlb isn't used at all. What am I missing to make this work with PCIe devices? Thanks, Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_RED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08B83C47088 for ; Wed, 26 May 2021 12:13:39 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ADC9061440 for ; Wed, 26 May 2021 12:13:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ADC9061440 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 78AAC60686; Wed, 26 May 2021 12:13:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7yv9Q3c3DtRm; Wed, 26 May 2021 12:13:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTP id 49D9A6061C; Wed, 26 May 2021 12:13:37 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 207ACC000E; Wed, 26 May 2021 12:13:37 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id D4E91C0001 for ; Wed, 26 May 2021 12:13:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id B938D4044D for ; Wed, 26 May 2021 12:13:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=kernel.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b5c0T6TSKQKm for ; Wed, 26 May 2021 12:13:34 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp4.osuosl.org (Postfix) with ESMTPS id 691DC40424 for ; Wed, 26 May 2021 12:13:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DB6A613D6; Wed, 26 May 2021 12:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1622031214; bh=6cgAAAry3KXjHYICFlfgug7HP7MtfiezqAbh5vyjbAk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dD4raOXl4m/1dmUYxCXd+9Zhh/XbrtQC52Qny8ZwVBaqmrNGpiCc5ZIU1b4WjGm8H uJa3jIZybN6M6kTMpAfuccsLT4/wRQbg2MVSjmE1yL01mC3lcM16NVvL5naGjFRQ57 RUc8/qmG6+LKY4mt6jNuAHGXfU60PflHdIRn81RXVX/L8quJSpzl4wYs9fdYRMYSyz TDHUTCxqGOaTMeMxI6d2iUlniRYHOiu06+32s75wchffQ8s3cbbJwvz7Jm6fWP5KI+ RznUS6G8WImqfu6sIHm02R4cmQSm1/c2dk4BLfXSL89hrwovsmQduosVw5UITnC7eb z+DGv0xJWhG6g== Date: Wed, 26 May 2021 13:13:23 +0100 From: Will Deacon To: Claire Chang Subject: Re: [PATCH v7 14/15] dt-bindings: of: Add restricted DMA pool Message-ID: <20210526121322.GA19313@willie-the-truck> References: <20210518064215.2856977-1-tientzu@chromium.org> <20210518064215.2856977-15-tientzu@chromium.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210518064215.2856977-15-tientzu@chromium.org> User-Agent: Mutt/1.10.1 (2018-07-13) Cc: heikki.krogerus@linux.intel.com, thomas.hellstrom@linux.intel.com, peterz@infradead.org, benh@kernel.crashing.org, joonas.lahtinen@linux.intel.com, dri-devel@lists.freedesktop.org, chris@chris-wilson.co.uk, grant.likely@arm.com, paulus@samba.org, Frank Rowand , mingo@kernel.org, sstabellini@kernel.org, Saravana Kannan , mpe@ellerman.id.au, "Rafael J . Wysocki" , Christoph Hellwig , Bartosz Golaszewski , bskeggs@redhat.com, linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org, Thierry Reding , intel-gfx@lists.freedesktop.org, matthew.auld@intel.com, linux-devicetree , jxgao@google.com, daniel@ffwll.ch, Konrad Rzeszutek Wilk , maarten.lankhorst@linux.intel.com, airlied@linux.ie, Dan Williams , linuxppc-dev@lists.ozlabs.org, jani.nikula@linux.intel.com, Rob Herring , rodrigo.vivi@intel.com, bhelgaas@google.com, boris.ostrovsky@oracle.com, Andy Shevchenko , jgross@suse.com, Nicolas Boichat , Greg KH , Randy Dunlap , lkml , "list@263.net:IOMMU DRIVERS" , Jim Quinlan , xypron.glpk@gmx.de, Robin Murphy , bauerman@linux.ibm.com X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" Hi Claire, On Tue, May 18, 2021 at 02:42:14PM +0800, Claire Chang wrote: > Introduce the new compatible string, restricted-dma-pool, for restricted > DMA. One can specify the address and length of the restricted DMA memory > region by restricted-dma-pool in the reserved-memory node. > > Signed-off-by: Claire Chang > --- > .../reserved-memory/reserved-memory.txt | 27 +++++++++++++++++++ > 1 file changed, 27 insertions(+) > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > index e8d3096d922c..284aea659015 100644 > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > @@ -51,6 +51,23 @@ compatible (optional) - standard definition > used as a shared pool of DMA buffers for a set of devices. It can > be used by an operating system to instantiate the necessary pool > management subsystem if necessary. > + - restricted-dma-pool: This indicates a region of memory meant to be > + used as a pool of restricted DMA buffers for a set of devices. The > + memory region would be the only region accessible to those devices. > + When using this, the no-map and reusable properties must not be set, > + so the operating system can create a virtual mapping that will be used > + for synchronization. The main purpose for restricted DMA is to > + mitigate the lack of DMA access control on systems without an IOMMU, > + which could result in the DMA accessing the system memory at > + unexpected times and/or unexpected addresses, possibly leading to data > + leakage or corruption. The feature on its own provides a basic level > + of protection against the DMA overwriting buffer contents at > + unexpected times. However, to protect against general data leakage and > + system memory corruption, the system needs to provide way to lock down > + the memory access, e.g., MPU. Note that since coherent allocation > + needs remapping, one must set up another device coherent pool by > + shared-dma-pool and use dma_alloc_from_dev_coherent instead for atomic > + coherent allocation. > - vendor specific string in the form ,[-] > no-map (optional) - empty property > - Indicates the operating system must not create a virtual mapping > @@ -120,6 +137,11 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > compatible = "acme,multimedia-memory"; > reg = <0x77000000 0x4000000>; > }; > + > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > + compatible = "restricted-dma-pool"; > + reg = <0x50000000 0x400000>; > + }; nit: You need to update the old text that states "This example defines 3 contiguous regions ...". > }; > > /* ... */ > @@ -138,4 +160,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > memory-region = <&multimedia_reserved>; > /* ... */ > }; > + > + pcie_device: pcie_device@0,0 { > + memory-region = <&restricted_dma_mem_reserved>; > + /* ... */ > + }; I still don't understand how this works for individual PCIe devices -- how is dev->of_node set to point at the node you have above? I tried adding the memory-region to the host controller instead, and then I see it crop up in dmesg: | pci-host-generic 40000000.pci: assigned reserved memory node restricted_dma_mem_reserved but none of the actual PCI devices end up with 'dma_io_tlb_mem' set, and so the restricted DMA area is not used. In fact, swiotlb isn't used at all. What am I missing to make this work with PCIe devices? Thanks, Will _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7E18C47088 for ; Wed, 26 May 2021 12:13:35 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A32B2613DC for ; Wed, 26 May 2021 12:13:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A32B2613DC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0E1546E1B4; Wed, 26 May 2021 12:13:35 +0000 (UTC) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by gabe.freedesktop.org (Postfix) with ESMTPS id 44FE96E1B4; Wed, 26 May 2021 12:13:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DB6A613D6; Wed, 26 May 2021 12:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1622031214; bh=6cgAAAry3KXjHYICFlfgug7HP7MtfiezqAbh5vyjbAk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dD4raOXl4m/1dmUYxCXd+9Zhh/XbrtQC52Qny8ZwVBaqmrNGpiCc5ZIU1b4WjGm8H uJa3jIZybN6M6kTMpAfuccsLT4/wRQbg2MVSjmE1yL01mC3lcM16NVvL5naGjFRQ57 RUc8/qmG6+LKY4mt6jNuAHGXfU60PflHdIRn81RXVX/L8quJSpzl4wYs9fdYRMYSyz TDHUTCxqGOaTMeMxI6d2iUlniRYHOiu06+32s75wchffQ8s3cbbJwvz7Jm6fWP5KI+ RznUS6G8WImqfu6sIHm02R4cmQSm1/c2dk4BLfXSL89hrwovsmQduosVw5UITnC7eb z+DGv0xJWhG6g== Date: Wed, 26 May 2021 13:13:23 +0100 From: Will Deacon To: Claire Chang Subject: Re: [PATCH v7 14/15] dt-bindings: of: Add restricted DMA pool Message-ID: <20210526121322.GA19313@willie-the-truck> References: <20210518064215.2856977-1-tientzu@chromium.org> <20210518064215.2856977-15-tientzu@chromium.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210518064215.2856977-15-tientzu@chromium.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: heikki.krogerus@linux.intel.com, thomas.hellstrom@linux.intel.com, peterz@infradead.org, dri-devel@lists.freedesktop.org, chris@chris-wilson.co.uk, grant.likely@arm.com, paulus@samba.org, Frank Rowand , mingo@kernel.org, Marek Szyprowski , sstabellini@kernel.org, Saravana Kannan , mpe@ellerman.id.au, Joerg Roedel , "Rafael J . Wysocki" , Christoph Hellwig , Bartosz Golaszewski , bskeggs@redhat.com, linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org, Thierry Reding , intel-gfx@lists.freedesktop.org, matthew.auld@intel.com, linux-devicetree , jxgao@google.com, Konrad Rzeszutek Wilk , airlied@linux.ie, Dan Williams , linuxppc-dev@lists.ozlabs.org, Rob Herring , rodrigo.vivi@intel.com, bhelgaas@google.com, boris.ostrovsky@oracle.com, Andy Shevchenko , jgross@suse.com, Nicolas Boichat , Greg KH , Randy Dunlap , lkml , tfiga@chromium.org, "list@263.net:IOMMU DRIVERS" , Jim Quinlan , xypron.glpk@gmx.de, Robin Murphy , bauerman@linux.ibm.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Hi Claire, On Tue, May 18, 2021 at 02:42:14PM +0800, Claire Chang wrote: > Introduce the new compatible string, restricted-dma-pool, for restricted > DMA. One can specify the address and length of the restricted DMA memory > region by restricted-dma-pool in the reserved-memory node. > > Signed-off-by: Claire Chang > --- > .../reserved-memory/reserved-memory.txt | 27 +++++++++++++++++++ > 1 file changed, 27 insertions(+) > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > index e8d3096d922c..284aea659015 100644 > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > @@ -51,6 +51,23 @@ compatible (optional) - standard definition > used as a shared pool of DMA buffers for a set of devices. It can > be used by an operating system to instantiate the necessary pool > management subsystem if necessary. > + - restricted-dma-pool: This indicates a region of memory meant to be > + used as a pool of restricted DMA buffers for a set of devices. The > + memory region would be the only region accessible to those devices. > + When using this, the no-map and reusable properties must not be set, > + so the operating system can create a virtual mapping that will be used > + for synchronization. The main purpose for restricted DMA is to > + mitigate the lack of DMA access control on systems without an IOMMU, > + which could result in the DMA accessing the system memory at > + unexpected times and/or unexpected addresses, possibly leading to data > + leakage or corruption. The feature on its own provides a basic level > + of protection against the DMA overwriting buffer contents at > + unexpected times. However, to protect against general data leakage and > + system memory corruption, the system needs to provide way to lock down > + the memory access, e.g., MPU. Note that since coherent allocation > + needs remapping, one must set up another device coherent pool by > + shared-dma-pool and use dma_alloc_from_dev_coherent instead for atomic > + coherent allocation. > - vendor specific string in the form ,[-] > no-map (optional) - empty property > - Indicates the operating system must not create a virtual mapping > @@ -120,6 +137,11 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > compatible = "acme,multimedia-memory"; > reg = <0x77000000 0x4000000>; > }; > + > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > + compatible = "restricted-dma-pool"; > + reg = <0x50000000 0x400000>; > + }; nit: You need to update the old text that states "This example defines 3 contiguous regions ...". > }; > > /* ... */ > @@ -138,4 +160,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > memory-region = <&multimedia_reserved>; > /* ... */ > }; > + > + pcie_device: pcie_device@0,0 { > + memory-region = <&restricted_dma_mem_reserved>; > + /* ... */ > + }; I still don't understand how this works for individual PCIe devices -- how is dev->of_node set to point at the node you have above? I tried adding the memory-region to the host controller instead, and then I see it crop up in dmesg: | pci-host-generic 40000000.pci: assigned reserved memory node restricted_dma_mem_reserved but none of the actual PCI devices end up with 'dma_io_tlb_mem' set, and so the restricted DMA area is not used. In fact, swiotlb isn't used at all. What am I missing to make this work with PCIe devices? Thanks, Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B75CC47089 for ; Wed, 26 May 2021 12:46:45 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4A1296101E for ; Wed, 26 May 2021 12:46:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A1296101E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E401A6E4A5; Wed, 26 May 2021 12:46:44 +0000 (UTC) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by gabe.freedesktop.org (Postfix) with ESMTPS id 44FE96E1B4; Wed, 26 May 2021 12:13:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DB6A613D6; Wed, 26 May 2021 12:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1622031214; bh=6cgAAAry3KXjHYICFlfgug7HP7MtfiezqAbh5vyjbAk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dD4raOXl4m/1dmUYxCXd+9Zhh/XbrtQC52Qny8ZwVBaqmrNGpiCc5ZIU1b4WjGm8H uJa3jIZybN6M6kTMpAfuccsLT4/wRQbg2MVSjmE1yL01mC3lcM16NVvL5naGjFRQ57 RUc8/qmG6+LKY4mt6jNuAHGXfU60PflHdIRn81RXVX/L8quJSpzl4wYs9fdYRMYSyz TDHUTCxqGOaTMeMxI6d2iUlniRYHOiu06+32s75wchffQ8s3cbbJwvz7Jm6fWP5KI+ RznUS6G8WImqfu6sIHm02R4cmQSm1/c2dk4BLfXSL89hrwovsmQduosVw5UITnC7eb z+DGv0xJWhG6g== Date: Wed, 26 May 2021 13:13:23 +0100 From: Will Deacon To: Claire Chang Message-ID: <20210526121322.GA19313@willie-the-truck> References: <20210518064215.2856977-1-tientzu@chromium.org> <20210518064215.2856977-15-tientzu@chromium.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210518064215.2856977-15-tientzu@chromium.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Mailman-Approved-At: Wed, 26 May 2021 12:46:43 +0000 Subject: Re: [Intel-gfx] [PATCH v7 14/15] dt-bindings: of: Add restricted DMA pool X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: heikki.krogerus@linux.intel.com, thomas.hellstrom@linux.intel.com, peterz@infradead.org, benh@kernel.crashing.org, dri-devel@lists.freedesktop.org, chris@chris-wilson.co.uk, grant.likely@arm.com, paulus@samba.org, Frank Rowand , mingo@kernel.org, Marek Szyprowski , sstabellini@kernel.org, Saravana Kannan , mpe@ellerman.id.au, Joerg Roedel , "Rafael J . Wysocki" , Christoph Hellwig , Bartosz Golaszewski , bskeggs@redhat.com, linux-pci@vger.kernel.org, xen-devel@lists.xenproject.org, Thierry Reding , intel-gfx@lists.freedesktop.org, matthew.auld@intel.com, linux-devicetree , jxgao@google.com, Konrad Rzeszutek Wilk , airlied@linux.ie, Dan Williams , linuxppc-dev@lists.ozlabs.org, Rob Herring , bhelgaas@google.com, boris.ostrovsky@oracle.com, Andy Shevchenko , jgross@suse.com, Nicolas Boichat , Greg KH , Randy Dunlap , lkml , tfiga@chromium.org, "list@263.net:IOMMU DRIVERS" , Jim Quinlan , xypron.glpk@gmx.de, Robin Murphy , bauerman@linux.ibm.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Hi Claire, On Tue, May 18, 2021 at 02:42:14PM +0800, Claire Chang wrote: > Introduce the new compatible string, restricted-dma-pool, for restricted > DMA. One can specify the address and length of the restricted DMA memory > region by restricted-dma-pool in the reserved-memory node. > > Signed-off-by: Claire Chang > --- > .../reserved-memory/reserved-memory.txt | 27 +++++++++++++++++++ > 1 file changed, 27 insertions(+) > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > index e8d3096d922c..284aea659015 100644 > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > @@ -51,6 +51,23 @@ compatible (optional) - standard definition > used as a shared pool of DMA buffers for a set of devices. It can > be used by an operating system to instantiate the necessary pool > management subsystem if necessary. > + - restricted-dma-pool: This indicates a region of memory meant to be > + used as a pool of restricted DMA buffers for a set of devices. The > + memory region would be the only region accessible to those devices. > + When using this, the no-map and reusable properties must not be set, > + so the operating system can create a virtual mapping that will be used > + for synchronization. The main purpose for restricted DMA is to > + mitigate the lack of DMA access control on systems without an IOMMU, > + which could result in the DMA accessing the system memory at > + unexpected times and/or unexpected addresses, possibly leading to data > + leakage or corruption. The feature on its own provides a basic level > + of protection against the DMA overwriting buffer contents at > + unexpected times. However, to protect against general data leakage and > + system memory corruption, the system needs to provide way to lock down > + the memory access, e.g., MPU. Note that since coherent allocation > + needs remapping, one must set up another device coherent pool by > + shared-dma-pool and use dma_alloc_from_dev_coherent instead for atomic > + coherent allocation. > - vendor specific string in the form ,[-] > no-map (optional) - empty property > - Indicates the operating system must not create a virtual mapping > @@ -120,6 +137,11 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > compatible = "acme,multimedia-memory"; > reg = <0x77000000 0x4000000>; > }; > + > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > + compatible = "restricted-dma-pool"; > + reg = <0x50000000 0x400000>; > + }; nit: You need to update the old text that states "This example defines 3 contiguous regions ...". > }; > > /* ... */ > @@ -138,4 +160,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > memory-region = <&multimedia_reserved>; > /* ... */ > }; > + > + pcie_device: pcie_device@0,0 { > + memory-region = <&restricted_dma_mem_reserved>; > + /* ... */ > + }; I still don't understand how this works for individual PCIe devices -- how is dev->of_node set to point at the node you have above? I tried adding the memory-region to the host controller instead, and then I see it crop up in dmesg: | pci-host-generic 40000000.pci: assigned reserved memory node restricted_dma_mem_reserved but none of the actual PCI devices end up with 'dma_io_tlb_mem' set, and so the restricted DMA area is not used. In fact, swiotlb isn't used at all. What am I missing to make this work with PCIe devices? Thanks, Will _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx