From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08F87C433DB for ; Wed, 10 Mar 2021 21:41:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C033464FD7 for ; Wed, 10 Mar 2021 21:41:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231783AbhCJVlQ (ORCPT ); Wed, 10 Mar 2021 16:41:16 -0500 Received: from mail.kernel.org ([198.145.29.99]:42890 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231778AbhCJVky (ORCPT ); Wed, 10 Mar 2021 16:40:54 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 731E764FE8; Wed, 10 Mar 2021 21:40:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615412454; bh=lljbEm8y6jNcxvIMoB+J261BTyFdid0U0j/1mSpgrns=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=eDXX9I0rBca4D2age7wm+DtVv6geacxhoDXnt5SysKVhym9aYBoOULiDV+ts4rOLP jxqSr5B0vX0/o/NShM/j+kwlnb4hGRATSr0mEhy2fTgb4b/f9TFkN0sucrguP/wUnN VpsdkpA2+bkS7oY10zU2zq+j59iH8lNhIYLI61shYJGWLghrHuyig30tcjLo3fTgbB 3DEVFLncbcg+C2B9HtVOo3SUvsmkNa+xgZXWZi3ZI5a2G3Gno+oUiLdB0wFYwj7D9V li5Sma9VH3O9evfauK7N3UwWSXURG5BK+1LBH0BYK5Vbx589vAqinspwZjQAwTmkXH mjJKT2LCYnF3Q== Received: by mail-qk1-f179.google.com with SMTP id m186so1917402qke.12; Wed, 10 Mar 2021 13:40:54 -0800 (PST) X-Gm-Message-State: AOAM532bmvbOE5P49KZOS0cD2KWJvsod2ISvDBzPVJPC80auwWE+hToR AA4UccCh9R29pvbb73fNZY1LaaCoz6jH/BN8fQ== X-Google-Smtp-Source: ABdhPJwvOHv4sjXRoS+j4qWhL9MVaGJgYMXKyxBDilpUrwUXgxjhTnULwJ7kIho0KdWOn0m+qzW6exxxNwLS3x7GxZU= X-Received: by 2002:a05:620a:1001:: with SMTP id z1mr4659415qkj.364.1615412453005; Wed, 10 Mar 2021 13:40:53 -0800 (PST) MIME-Version: 1.0 References: <20210209062131.2300005-1-tientzu@chromium.org> <20210209062131.2300005-14-tientzu@chromium.org> <20210310160747.GA29834@willie-the-truck> In-Reply-To: <20210310160747.GA29834@willie-the-truck> From: Rob Herring Date: Wed, 10 Mar 2021 14:40:41 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v4 13/14] dt-bindings: of: Add restricted DMA pool To: Will Deacon Cc: Claire Chang , Michael Ellerman , Joerg Roedel , Frank Rowand , Konrad Rzeszutek Wilk , Boris Ostrovsky , Juergen Gross , Christoph Hellwig , Marek Szyprowski , Benjamin Herrenschmidt , Paul Mackerras , "list@263.net:IOMMU DRIVERS" , Stefano Stabellini , Robin Murphy , Grant Likely , Heinrich Schuchardt , Thierry Reding , Ingo Molnar , Thiago Jung Bauermann , Peter Zijlstra , Greg KH , Saravana Kannan , "Rafael J . Wysocki" , Heikki Krogerus , Andy Shevchenko , Randy Dunlap , Dan Williams , Bartosz Golaszewski , linux-devicetree , lkml , linuxppc-dev , xen-devel , Nicolas Boichat , Jim Quinlan Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Mar 10, 2021 at 9:08 AM Will Deacon wrote: > > Hi Claire, > > On Tue, Feb 09, 2021 at 02:21:30PM +0800, Claire Chang wrote: > > Introduce the new compatible string, restricted-dma-pool, for restricted > > DMA. One can specify the address and length of the restricted DMA memory > > region by restricted-dma-pool in the reserved-memory node. > > > > Signed-off-by: Claire Chang > > --- > > .../reserved-memory/reserved-memory.txt | 24 +++++++++++++++++++ > > 1 file changed, 24 insertions(+) > > > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > index e8d3096d922c..fc9a12c2f679 100644 > > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > @@ -51,6 +51,20 @@ compatible (optional) - standard definition > > used as a shared pool of DMA buffers for a set of devices. It can > > be used by an operating system to instantiate the necessary pool > > management subsystem if necessary. > > + - restricted-dma-pool: This indicates a region of memory meant to be > > + used as a pool of restricted DMA buffers for a set of devices. The > > + memory region would be the only region accessible to those devices. > > + When using this, the no-map and reusable properties must not be set, > > + so the operating system can create a virtual mapping that will be used > > + for synchronization. The main purpose for restricted DMA is to > > + mitigate the lack of DMA access control on systems without an IOMMU, > > + which could result in the DMA accessing the system memory at > > + unexpected times and/or unexpected addresses, possibly leading to data > > + leakage or corruption. The feature on its own provides a basic level > > + of protection against the DMA overwriting buffer contents at > > + unexpected times. However, to protect against general data leakage and > > + system memory corruption, the system needs to provide way to lock down > > + the memory access, e.g., MPU. > > As far as I can tell, these pools work with both static allocations (which > seem to match your use-case where firmware has preconfigured the DMA ranges) > but also with dynamic allocations where a 'size' property is present instead > of the 'reg' property and the kernel is responsible for allocating the > reservation during boot. Am I right and, if so, is that deliberate? I believe so. I'm not keen on having size only reservations in DT. Yes, we allowed that already, but that's back from the days of needing large CMA carveouts to be reserved early in boot. I've read that the kernel is much better now at contiguous allocations, so do we really need this in DT anymore? > I ask because I think that would potentially be useful to us for the > Protected KVM work, where we need to bounce virtio memory accesses via > guest-determined windows because the guest memory is generally inaccessible > to the host. We've been hacking this using a combination of "swiotlb=force" > and set_memory_{decrypted,encrypted}() but it would be much better to > leverage the stuff you have here. > > Also: > > > + > > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > > + compatible = "restricted-dma-pool"; > > + reg = <0x50000000 0x400000>; > > + }; > > }; > > > > /* ... */ > > @@ -138,4 +157,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > > memory-region = <&multimedia_reserved>; > > /* ... */ > > }; > > + > > + pcie_device: pcie_device@0,0 { > > + memory-region = <&restricted_dma_mem_reserved>; > > + /* ... */ > > + }; > > I find this example a bit weird, as I didn't think we usually had DT nodes > for PCI devices; rather they are discovered as a result of probing config > space. Is the idea that you have one reserved memory region attached to the > RC and all the PCI devices below that share the region, or is there a need > for a mapping mechanism? We can have DT nodes for PCI. AIUI, IBM power systems always do. For FDT, it's only if there are extra non-discoverable resources. It's particularly fun when it's resources which need to be enabled for the PCI device to be discovered. That seems to be a growing problem as PCI becomes more common on embedded systems. Rob From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1EF2C433E0 for ; Wed, 10 Mar 2021 21:41:55 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0200464FC4 for ; Wed, 10 Mar 2021 21:41:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0200464FC4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4DwlsF1vPfz3cW2 for ; Thu, 11 Mar 2021 08:41:53 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=eDXX9I0r; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=198.145.29.99; helo=mail.kernel.org; envelope-from=robh+dt@kernel.org; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=eDXX9I0r; dkim-atps=neutral Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4Dwlrl1bClz30Jd for ; Thu, 11 Mar 2021 08:41:26 +1100 (AEDT) Received: by mail.kernel.org (Postfix) with ESMTPSA id A1FF064FE7 for ; Wed, 10 Mar 2021 21:40:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615412454; bh=lljbEm8y6jNcxvIMoB+J261BTyFdid0U0j/1mSpgrns=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=eDXX9I0rBca4D2age7wm+DtVv6geacxhoDXnt5SysKVhym9aYBoOULiDV+ts4rOLP jxqSr5B0vX0/o/NShM/j+kwlnb4hGRATSr0mEhy2fTgb4b/f9TFkN0sucrguP/wUnN VpsdkpA2+bkS7oY10zU2zq+j59iH8lNhIYLI61shYJGWLghrHuyig30tcjLo3fTgbB 3DEVFLncbcg+C2B9HtVOo3SUvsmkNa+xgZXWZi3ZI5a2G3Gno+oUiLdB0wFYwj7D9V li5Sma9VH3O9evfauK7N3UwWSXURG5BK+1LBH0BYK5Vbx589vAqinspwZjQAwTmkXH mjJKT2LCYnF3Q== Received: by mail-qk1-f170.google.com with SMTP id x10so18546082qkm.8 for ; Wed, 10 Mar 2021 13:40:54 -0800 (PST) X-Gm-Message-State: AOAM531pLRl1TKT2RMDnIWEO3FGv9giLeWB3gIjmcPj8lVchodXAA8+y iccF0NeGKOYXXXtZ6f1uN4TB8Tx4mZuYCEqilw== X-Google-Smtp-Source: ABdhPJwvOHv4sjXRoS+j4qWhL9MVaGJgYMXKyxBDilpUrwUXgxjhTnULwJ7kIho0KdWOn0m+qzW6exxxNwLS3x7GxZU= X-Received: by 2002:a05:620a:1001:: with SMTP id z1mr4659415qkj.364.1615412453005; Wed, 10 Mar 2021 13:40:53 -0800 (PST) MIME-Version: 1.0 References: <20210209062131.2300005-1-tientzu@chromium.org> <20210209062131.2300005-14-tientzu@chromium.org> <20210310160747.GA29834@willie-the-truck> In-Reply-To: <20210310160747.GA29834@willie-the-truck> From: Rob Herring Date: Wed, 10 Mar 2021 14:40:41 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v4 13/14] dt-bindings: of: Add restricted DMA pool To: Will Deacon Content-Type: text/plain; charset="UTF-8" X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Heikki Krogerus , Peter Zijlstra , Grant Likely , Paul Mackerras , Frank Rowand , Ingo Molnar , Marek Szyprowski , Stefano Stabellini , Saravana Kannan , Joerg Roedel , "Rafael J . Wysocki" , Christoph Hellwig , Bartosz Golaszewski , xen-devel , Thierry Reding , linux-devicetree , Konrad Rzeszutek Wilk , Dan Williams , linuxppc-dev , Nicolas Boichat , Claire Chang , Boris Ostrovsky , Andy Shevchenko , Juergen Gross , Greg KH , Randy Dunlap , lkml , "list@263.net:IOMMU DRIVERS" , Jim Quinlan , Heinrich Schuchardt , Robin Murphy , Thiago Jung Bauermann Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed, Mar 10, 2021 at 9:08 AM Will Deacon wrote: > > Hi Claire, > > On Tue, Feb 09, 2021 at 02:21:30PM +0800, Claire Chang wrote: > > Introduce the new compatible string, restricted-dma-pool, for restricted > > DMA. One can specify the address and length of the restricted DMA memory > > region by restricted-dma-pool in the reserved-memory node. > > > > Signed-off-by: Claire Chang > > --- > > .../reserved-memory/reserved-memory.txt | 24 +++++++++++++++++++ > > 1 file changed, 24 insertions(+) > > > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > index e8d3096d922c..fc9a12c2f679 100644 > > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > @@ -51,6 +51,20 @@ compatible (optional) - standard definition > > used as a shared pool of DMA buffers for a set of devices. It can > > be used by an operating system to instantiate the necessary pool > > management subsystem if necessary. > > + - restricted-dma-pool: This indicates a region of memory meant to be > > + used as a pool of restricted DMA buffers for a set of devices. The > > + memory region would be the only region accessible to those devices. > > + When using this, the no-map and reusable properties must not be set, > > + so the operating system can create a virtual mapping that will be used > > + for synchronization. The main purpose for restricted DMA is to > > + mitigate the lack of DMA access control on systems without an IOMMU, > > + which could result in the DMA accessing the system memory at > > + unexpected times and/or unexpected addresses, possibly leading to data > > + leakage or corruption. The feature on its own provides a basic level > > + of protection against the DMA overwriting buffer contents at > > + unexpected times. However, to protect against general data leakage and > > + system memory corruption, the system needs to provide way to lock down > > + the memory access, e.g., MPU. > > As far as I can tell, these pools work with both static allocations (which > seem to match your use-case where firmware has preconfigured the DMA ranges) > but also with dynamic allocations where a 'size' property is present instead > of the 'reg' property and the kernel is responsible for allocating the > reservation during boot. Am I right and, if so, is that deliberate? I believe so. I'm not keen on having size only reservations in DT. Yes, we allowed that already, but that's back from the days of needing large CMA carveouts to be reserved early in boot. I've read that the kernel is much better now at contiguous allocations, so do we really need this in DT anymore? > I ask because I think that would potentially be useful to us for the > Protected KVM work, where we need to bounce virtio memory accesses via > guest-determined windows because the guest memory is generally inaccessible > to the host. We've been hacking this using a combination of "swiotlb=force" > and set_memory_{decrypted,encrypted}() but it would be much better to > leverage the stuff you have here. > > Also: > > > + > > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > > + compatible = "restricted-dma-pool"; > > + reg = <0x50000000 0x400000>; > > + }; > > }; > > > > /* ... */ > > @@ -138,4 +157,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > > memory-region = <&multimedia_reserved>; > > /* ... */ > > }; > > + > > + pcie_device: pcie_device@0,0 { > > + memory-region = <&restricted_dma_mem_reserved>; > > + /* ... */ > > + }; > > I find this example a bit weird, as I didn't think we usually had DT nodes > for PCI devices; rather they are discovered as a result of probing config > space. Is the idea that you have one reserved memory region attached to the > RC and all the PCI devices below that share the region, or is there a need > for a mapping mechanism? We can have DT nodes for PCI. AIUI, IBM power systems always do. For FDT, it's only if there are extra non-discoverable resources. It's particularly fun when it's resources which need to be enabled for the PCI device to be discovered. That seems to be a growing problem as PCI becomes more common on embedded systems. Rob From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22778C433DB for ; Wed, 10 Mar 2021 21:41:23 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7643E64E33 for ; Wed, 10 Mar 2021 21:41:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7643E64E33 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1C2404EC69; Wed, 10 Mar 2021 21:41:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sqJoS2pLYUOV; Wed, 10 Mar 2021 21:41:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTP id D121447135; Wed, 10 Mar 2021 21:41:04 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B0133C000B; Wed, 10 Mar 2021 21:41:04 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id A2B4AC0001 for ; Wed, 10 Mar 2021 21:41:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 94F22414D4 for ; Wed, 10 Mar 2021 21:41:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=kernel.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dPefin-Svjmp for ; Wed, 10 Mar 2021 21:40:58 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp2.osuosl.org (Postfix) with ESMTPS id DE58D40001 for ; Wed, 10 Mar 2021 21:40:56 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 02CB664FD7 for ; Wed, 10 Mar 2021 21:40:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615412454; bh=lljbEm8y6jNcxvIMoB+J261BTyFdid0U0j/1mSpgrns=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=eDXX9I0rBca4D2age7wm+DtVv6geacxhoDXnt5SysKVhym9aYBoOULiDV+ts4rOLP jxqSr5B0vX0/o/NShM/j+kwlnb4hGRATSr0mEhy2fTgb4b/f9TFkN0sucrguP/wUnN VpsdkpA2+bkS7oY10zU2zq+j59iH8lNhIYLI61shYJGWLghrHuyig30tcjLo3fTgbB 3DEVFLncbcg+C2B9HtVOo3SUvsmkNa+xgZXWZi3ZI5a2G3Gno+oUiLdB0wFYwj7D9V li5Sma9VH3O9evfauK7N3UwWSXURG5BK+1LBH0BYK5Vbx589vAqinspwZjQAwTmkXH mjJKT2LCYnF3Q== Received: by mail-qk1-f177.google.com with SMTP id a9so18515072qkn.13 for ; Wed, 10 Mar 2021 13:40:53 -0800 (PST) X-Gm-Message-State: AOAM532avGFBN4xRAIaCPaXqg4Ha29/jkrF8az+4irbrgKif6/ynDMAF 2iv5FQZUmitTgR4n6BiG+5Mt5wzFldaUGVtkrQ== X-Google-Smtp-Source: ABdhPJwvOHv4sjXRoS+j4qWhL9MVaGJgYMXKyxBDilpUrwUXgxjhTnULwJ7kIho0KdWOn0m+qzW6exxxNwLS3x7GxZU= X-Received: by 2002:a05:620a:1001:: with SMTP id z1mr4659415qkj.364.1615412453005; Wed, 10 Mar 2021 13:40:53 -0800 (PST) MIME-Version: 1.0 References: <20210209062131.2300005-1-tientzu@chromium.org> <20210209062131.2300005-14-tientzu@chromium.org> <20210310160747.GA29834@willie-the-truck> In-Reply-To: <20210310160747.GA29834@willie-the-truck> From: Rob Herring Date: Wed, 10 Mar 2021 14:40:41 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v4 13/14] dt-bindings: of: Add restricted DMA pool To: Will Deacon Cc: Heikki Krogerus , Peter Zijlstra , Benjamin Herrenschmidt , Grant Likely , Paul Mackerras , Frank Rowand , Ingo Molnar , Stefano Stabellini , Saravana Kannan , Michael Ellerman , "Rafael J . Wysocki" , Christoph Hellwig , Bartosz Golaszewski , xen-devel , Thierry Reding , linux-devicetree , Konrad Rzeszutek Wilk , Dan Williams , linuxppc-dev , Nicolas Boichat , Claire Chang , Boris Ostrovsky , Andy Shevchenko , Juergen Gross , Greg KH , Randy Dunlap , lkml , "list@263.net:IOMMU DRIVERS" , Jim Quinlan , Heinrich Schuchardt , Robin Murphy , Thiago Jung Bauermann X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Wed, Mar 10, 2021 at 9:08 AM Will Deacon wrote: > > Hi Claire, > > On Tue, Feb 09, 2021 at 02:21:30PM +0800, Claire Chang wrote: > > Introduce the new compatible string, restricted-dma-pool, for restricted > > DMA. One can specify the address and length of the restricted DMA memory > > region by restricted-dma-pool in the reserved-memory node. > > > > Signed-off-by: Claire Chang > > --- > > .../reserved-memory/reserved-memory.txt | 24 +++++++++++++++++++ > > 1 file changed, 24 insertions(+) > > > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > index e8d3096d922c..fc9a12c2f679 100644 > > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > @@ -51,6 +51,20 @@ compatible (optional) - standard definition > > used as a shared pool of DMA buffers for a set of devices. It can > > be used by an operating system to instantiate the necessary pool > > management subsystem if necessary. > > + - restricted-dma-pool: This indicates a region of memory meant to be > > + used as a pool of restricted DMA buffers for a set of devices. The > > + memory region would be the only region accessible to those devices. > > + When using this, the no-map and reusable properties must not be set, > > + so the operating system can create a virtual mapping that will be used > > + for synchronization. The main purpose for restricted DMA is to > > + mitigate the lack of DMA access control on systems without an IOMMU, > > + which could result in the DMA accessing the system memory at > > + unexpected times and/or unexpected addresses, possibly leading to data > > + leakage or corruption. The feature on its own provides a basic level > > + of protection against the DMA overwriting buffer contents at > > + unexpected times. However, to protect against general data leakage and > > + system memory corruption, the system needs to provide way to lock down > > + the memory access, e.g., MPU. > > As far as I can tell, these pools work with both static allocations (which > seem to match your use-case where firmware has preconfigured the DMA ranges) > but also with dynamic allocations where a 'size' property is present instead > of the 'reg' property and the kernel is responsible for allocating the > reservation during boot. Am I right and, if so, is that deliberate? I believe so. I'm not keen on having size only reservations in DT. Yes, we allowed that already, but that's back from the days of needing large CMA carveouts to be reserved early in boot. I've read that the kernel is much better now at contiguous allocations, so do we really need this in DT anymore? > I ask because I think that would potentially be useful to us for the > Protected KVM work, where we need to bounce virtio memory accesses via > guest-determined windows because the guest memory is generally inaccessible > to the host. We've been hacking this using a combination of "swiotlb=force" > and set_memory_{decrypted,encrypted}() but it would be much better to > leverage the stuff you have here. > > Also: > > > + > > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > > + compatible = "restricted-dma-pool"; > > + reg = <0x50000000 0x400000>; > > + }; > > }; > > > > /* ... */ > > @@ -138,4 +157,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > > memory-region = <&multimedia_reserved>; > > /* ... */ > > }; > > + > > + pcie_device: pcie_device@0,0 { > > + memory-region = <&restricted_dma_mem_reserved>; > > + /* ... */ > > + }; > > I find this example a bit weird, as I didn't think we usually had DT nodes > for PCI devices; rather they are discovered as a result of probing config > space. Is the idea that you have one reserved memory region attached to the > RC and all the PCI devices below that share the region, or is there a need > for a mapping mechanism? We can have DT nodes for PCI. AIUI, IBM power systems always do. For FDT, it's only if there are extra non-discoverable resources. It's particularly fun when it's resources which need to be enabled for the PCI device to be discovered. That seems to be a growing problem as PCI becomes more common on embedded systems. Rob _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B0EAC433E0 for ; Wed, 10 Mar 2021 21:41:24 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 256B164FC4 for ; Wed, 10 Mar 2021 21:41:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 256B164FC4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.96311.182078 (Exim 4.92) (envelope-from ) id 1lK6Ze-0003Y8-N7; Wed, 10 Mar 2021 21:40:58 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 96311.182078; Wed, 10 Mar 2021 21:40:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lK6Ze-0003Y1-Jd; Wed, 10 Mar 2021 21:40:58 +0000 Received: by outflank-mailman (input) for mailman id 96311; Wed, 10 Mar 2021 21:40:57 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lK6Zd-0003Xw-Ga for xen-devel@lists.xenproject.org; Wed, 10 Mar 2021 21:40:57 +0000 Received: from mail.kernel.org (unknown [198.145.29.99]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 4a31c3e4-121d-4c83-8697-8ca6b797bd45; Wed, 10 Mar 2021 21:40:55 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B7BBC64FD6 for ; Wed, 10 Mar 2021 21:40:54 +0000 (UTC) Received: by mail-qt1-f177.google.com with SMTP id d11so14252870qtx.9 for ; Wed, 10 Mar 2021 13:40:54 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4a31c3e4-121d-4c83-8697-8ca6b797bd45 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615412454; bh=lljbEm8y6jNcxvIMoB+J261BTyFdid0U0j/1mSpgrns=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=eDXX9I0rBca4D2age7wm+DtVv6geacxhoDXnt5SysKVhym9aYBoOULiDV+ts4rOLP jxqSr5B0vX0/o/NShM/j+kwlnb4hGRATSr0mEhy2fTgb4b/f9TFkN0sucrguP/wUnN VpsdkpA2+bkS7oY10zU2zq+j59iH8lNhIYLI61shYJGWLghrHuyig30tcjLo3fTgbB 3DEVFLncbcg+C2B9HtVOo3SUvsmkNa+xgZXWZi3ZI5a2G3Gno+oUiLdB0wFYwj7D9V li5Sma9VH3O9evfauK7N3UwWSXURG5BK+1LBH0BYK5Vbx589vAqinspwZjQAwTmkXH mjJKT2LCYnF3Q== X-Gm-Message-State: AOAM533xQqmKjpjvGpHhhd4e9D9o1KGB5HEXeqnGFrRi34C+R0+N7WdE YWH61Jp5K4jBvqAjUW9+WVufP/96wr8URiXcjA== X-Google-Smtp-Source: ABdhPJwvOHv4sjXRoS+j4qWhL9MVaGJgYMXKyxBDilpUrwUXgxjhTnULwJ7kIho0KdWOn0m+qzW6exxxNwLS3x7GxZU= X-Received: by 2002:a05:620a:1001:: with SMTP id z1mr4659415qkj.364.1615412453005; Wed, 10 Mar 2021 13:40:53 -0800 (PST) MIME-Version: 1.0 References: <20210209062131.2300005-1-tientzu@chromium.org> <20210209062131.2300005-14-tientzu@chromium.org> <20210310160747.GA29834@willie-the-truck> In-Reply-To: <20210310160747.GA29834@willie-the-truck> From: Rob Herring Date: Wed, 10 Mar 2021 14:40:41 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v4 13/14] dt-bindings: of: Add restricted DMA pool To: Will Deacon Cc: Claire Chang , Michael Ellerman , Joerg Roedel , Frank Rowand , Konrad Rzeszutek Wilk , Boris Ostrovsky , Juergen Gross , Christoph Hellwig , Marek Szyprowski , Benjamin Herrenschmidt , Paul Mackerras , "list@263.net:IOMMU DRIVERS" , Stefano Stabellini , Robin Murphy , Grant Likely , Heinrich Schuchardt , Thierry Reding , Ingo Molnar , Thiago Jung Bauermann , Peter Zijlstra , Greg KH , Saravana Kannan , "Rafael J . Wysocki" , Heikki Krogerus , Andy Shevchenko , Randy Dunlap , Dan Williams , Bartosz Golaszewski , linux-devicetree , lkml , linuxppc-dev , xen-devel , Nicolas Boichat , Jim Quinlan Content-Type: text/plain; charset="UTF-8" On Wed, Mar 10, 2021 at 9:08 AM Will Deacon wrote: > > Hi Claire, > > On Tue, Feb 09, 2021 at 02:21:30PM +0800, Claire Chang wrote: > > Introduce the new compatible string, restricted-dma-pool, for restricted > > DMA. One can specify the address and length of the restricted DMA memory > > region by restricted-dma-pool in the reserved-memory node. > > > > Signed-off-by: Claire Chang > > --- > > .../reserved-memory/reserved-memory.txt | 24 +++++++++++++++++++ > > 1 file changed, 24 insertions(+) > > > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > index e8d3096d922c..fc9a12c2f679 100644 > > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt > > @@ -51,6 +51,20 @@ compatible (optional) - standard definition > > used as a shared pool of DMA buffers for a set of devices. It can > > be used by an operating system to instantiate the necessary pool > > management subsystem if necessary. > > + - restricted-dma-pool: This indicates a region of memory meant to be > > + used as a pool of restricted DMA buffers for a set of devices. The > > + memory region would be the only region accessible to those devices. > > + When using this, the no-map and reusable properties must not be set, > > + so the operating system can create a virtual mapping that will be used > > + for synchronization. The main purpose for restricted DMA is to > > + mitigate the lack of DMA access control on systems without an IOMMU, > > + which could result in the DMA accessing the system memory at > > + unexpected times and/or unexpected addresses, possibly leading to data > > + leakage or corruption. The feature on its own provides a basic level > > + of protection against the DMA overwriting buffer contents at > > + unexpected times. However, to protect against general data leakage and > > + system memory corruption, the system needs to provide way to lock down > > + the memory access, e.g., MPU. > > As far as I can tell, these pools work with both static allocations (which > seem to match your use-case where firmware has preconfigured the DMA ranges) > but also with dynamic allocations where a 'size' property is present instead > of the 'reg' property and the kernel is responsible for allocating the > reservation during boot. Am I right and, if so, is that deliberate? I believe so. I'm not keen on having size only reservations in DT. Yes, we allowed that already, but that's back from the days of needing large CMA carveouts to be reserved early in boot. I've read that the kernel is much better now at contiguous allocations, so do we really need this in DT anymore? > I ask because I think that would potentially be useful to us for the > Protected KVM work, where we need to bounce virtio memory accesses via > guest-determined windows because the guest memory is generally inaccessible > to the host. We've been hacking this using a combination of "swiotlb=force" > and set_memory_{decrypted,encrypted}() but it would be much better to > leverage the stuff you have here. > > Also: > > > + > > + restricted_dma_mem_reserved: restricted_dma_mem_reserved { > > + compatible = "restricted-dma-pool"; > > + reg = <0x50000000 0x400000>; > > + }; > > }; > > > > /* ... */ > > @@ -138,4 +157,9 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB). > > memory-region = <&multimedia_reserved>; > > /* ... */ > > }; > > + > > + pcie_device: pcie_device@0,0 { > > + memory-region = <&restricted_dma_mem_reserved>; > > + /* ... */ > > + }; > > I find this example a bit weird, as I didn't think we usually had DT nodes > for PCI devices; rather they are discovered as a result of probing config > space. Is the idea that you have one reserved memory region attached to the > RC and all the PCI devices below that share the region, or is there a need > for a mapping mechanism? We can have DT nodes for PCI. AIUI, IBM power systems always do. For FDT, it's only if there are extra non-discoverable resources. It's particularly fun when it's resources which need to be enabled for the PCI device to be discovered. That seems to be a growing problem as PCI becomes more common on embedded systems. Rob