From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 55CCCC433EF for ; Tue, 4 Jan 2022 17:06:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id CEB838124F; Tue, 4 Jan 2022 17:06:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Al12aa2JqC7E; Tue, 4 Jan 2022 17:06:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id A14DB80C44; Tue, 4 Jan 2022 17:06:37 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 71D72C002F; Tue, 4 Jan 2022 17:06:37 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 866AAC001E for ; Tue, 4 Jan 2022 17:06:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 56AF1414C8 for ; Tue, 4 Jan 2022 17:06:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=kernel.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vQgzajXcXuUV for ; Tue, 4 Jan 2022 17:06:34 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by smtp4.osuosl.org (Postfix) with ESMTPS id A184B414C6 for ; Tue, 4 Jan 2022 17:06:34 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8754461521; Tue, 4 Jan 2022 17:06:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9BDC2C36AED; Tue, 4 Jan 2022 17:06:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641315993; bh=s92lHlV9B1y8sS9ImfgWApRrW8AUqJJMtnEF5Ev1DqY=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=j7P0q9nXAg/o99t9HPCrgze9DFgTvDvgqJsT5ucepqaFZzZff9B4kgE0E/Iqt202m L9p+eS2zTii/RLR6lzZ7htDhw14iYE4hswug3mZH7T9hlrkYXa/slzlRyQwI8hyElJ kW9V+c6DJGh+Vge2J0+PHqhruF9lMTh5G1Oduf9DGVAyX1Ukf12kHr7qVIVgUKRvni yavpxTvIr66Xgvtl4/wr2SK7CdAE86V54P4RmLuc9dtiUbCff0pAFG2Ay793w7USA6 H7bbx5IrpRSSb39U839eTI4eqy6XRo3BPK1wHuLN9qAA2xjCT3c388KS+nMcJw+qNd Kw0Vb9qYzStzg== Date: Tue, 4 Jan 2022 11:06:31 -0600 From: Bjorn Helgaas To: Lu Baolu Subject: Re: [PATCH v5 09/14] PCI: portdrv: Suppress kernel DMA ownership auto-claiming Message-ID: <20220104170631.GA99771@bhelgaas> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220104015644.2294354-10-baolu.lu@linux.intel.com> Cc: Stuart Yoder , rafael@kernel.org, David Airlie , linux-pci@vger.kernel.org, Thierry Reding , Diana Craciun , Dmitry Osipenko , Will Deacon , Ashok Raj , Jonathan Hunter , Christoph Hellwig , Jason Gunthorpe , Kevin Tian , Chaitanya Kulkarni , Alex Williamson , kvm@vger.kernel.org, Bjorn Helgaas , Dan Williams , Greg Kroah-Hartman , Cornelia Huck , linux-kernel@vger.kernel.org, Li Yang , iommu@lists.linux-foundation.org, Jacob jun Pan , Daniel Vetter , Robin Murphy X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Tue, Jan 04, 2022 at 09:56:39AM +0800, Lu Baolu wrote: > If a switch lacks ACS P2P Request Redirect, a device below the switch can > bypass the IOMMU and DMA directly to other devices below the switch, so > all the downstream devices must be in the same IOMMU group as the switch > itself. Help me think through what's going on here. IIUC, we put devices in the same IOMMU group when they can interfere with each other in any way (DMA, config access, etc). (We said "DMA" above, but I guess this would also apply to config requests, right?) *This* patch doesn't check for any ACS features. Can you connect the dots for me? I guess the presence or absence of P2P Request Redirect determines the size of the IOMMU group. And the following says something about what is allowed in the group? And .no_kernel_api_dma allows an exception to the general rule? > The existing vfio framework allows the portdrv driver to be bound > to the bridge while its downstream devices are assigned to user space. I.e., the existing VFIO framework allows a switch to be in the same IOMMU group as the devices below it, even though the switch has a kernel driver and the other devices may have userspace drivers? Is this a function of VFIO design or of the IOMMU driver? > The pci_dma_configure() marks the iommu_group as containing only devices > with kernel drivers that manage DMA. Avoid this default behavior for the > portdrv driver in order for compatibility with the current vfio policy. I assume "IOMMU group" means the same as "iommu_group"; maybe we can use one of them consistently? > Suggested-by: Jason Gunthorpe > Suggested-by: Kevin Tian > Signed-off-by: Lu Baolu > --- > drivers/pci/pcie/portdrv_pci.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/pci/pcie/portdrv_pci.c b/drivers/pci/pcie/portdrv_pci.c > index 35eca6277a96..2116f821c005 100644 > --- a/drivers/pci/pcie/portdrv_pci.c > +++ b/drivers/pci/pcie/portdrv_pci.c > @@ -202,6 +202,8 @@ static struct pci_driver pcie_portdriver = { > > .err_handler = &pcie_portdrv_err_handler, > > + .no_kernel_api_dma = true, > + > .driver.pm = PCIE_PORTDRV_PM_OPS, > }; > > -- > 2.25.1 > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A34DC433FE for ; Tue, 4 Jan 2022 17:07:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234262AbiADRHC (ORCPT ); Tue, 4 Jan 2022 12:07:02 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:40260 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234250AbiADRGf (ORCPT ); Tue, 4 Jan 2022 12:06:35 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 314BAB811B1; Tue, 4 Jan 2022 17:06:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9BDC2C36AED; Tue, 4 Jan 2022 17:06:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641315993; bh=s92lHlV9B1y8sS9ImfgWApRrW8AUqJJMtnEF5Ev1DqY=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=j7P0q9nXAg/o99t9HPCrgze9DFgTvDvgqJsT5ucepqaFZzZff9B4kgE0E/Iqt202m L9p+eS2zTii/RLR6lzZ7htDhw14iYE4hswug3mZH7T9hlrkYXa/slzlRyQwI8hyElJ kW9V+c6DJGh+Vge2J0+PHqhruF9lMTh5G1Oduf9DGVAyX1Ukf12kHr7qVIVgUKRvni yavpxTvIr66Xgvtl4/wr2SK7CdAE86V54P4RmLuc9dtiUbCff0pAFG2Ay793w7USA6 H7bbx5IrpRSSb39U839eTI4eqy6XRo3BPK1wHuLN9qAA2xjCT3c388KS+nMcJw+qNd Kw0Vb9qYzStzg== Date: Tue, 4 Jan 2022 11:06:31 -0600 From: Bjorn Helgaas To: Lu Baolu Cc: Greg Kroah-Hartman , Joerg Roedel , Alex Williamson , Bjorn Helgaas , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Dan Williams , rafael@kernel.org, Diana Craciun , Cornelia Huck , Eric Auger , Liu Yi L , Jacob jun Pan , Chaitanya Kulkarni , Stuart Yoder , Laurentiu Tudor , Thierry Reding , David Airlie , Daniel Vetter , Jonathan Hunter , Li Yang , Dmitry Osipenko , iommu@lists.linux-foundation.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v5 09/14] PCI: portdrv: Suppress kernel DMA ownership auto-claiming Message-ID: <20220104170631.GA99771@bhelgaas> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220104015644.2294354-10-baolu.lu@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jan 04, 2022 at 09:56:39AM +0800, Lu Baolu wrote: > If a switch lacks ACS P2P Request Redirect, a device below the switch can > bypass the IOMMU and DMA directly to other devices below the switch, so > all the downstream devices must be in the same IOMMU group as the switch > itself. Help me think through what's going on here. IIUC, we put devices in the same IOMMU group when they can interfere with each other in any way (DMA, config access, etc). (We said "DMA" above, but I guess this would also apply to config requests, right?) *This* patch doesn't check for any ACS features. Can you connect the dots for me? I guess the presence or absence of P2P Request Redirect determines the size of the IOMMU group. And the following says something about what is allowed in the group? And .no_kernel_api_dma allows an exception to the general rule? > The existing vfio framework allows the portdrv driver to be bound > to the bridge while its downstream devices are assigned to user space. I.e., the existing VFIO framework allows a switch to be in the same IOMMU group as the devices below it, even though the switch has a kernel driver and the other devices may have userspace drivers? Is this a function of VFIO design or of the IOMMU driver? > The pci_dma_configure() marks the iommu_group as containing only devices > with kernel drivers that manage DMA. Avoid this default behavior for the > portdrv driver in order for compatibility with the current vfio policy. I assume "IOMMU group" means the same as "iommu_group"; maybe we can use one of them consistently? > Suggested-by: Jason Gunthorpe > Suggested-by: Kevin Tian > Signed-off-by: Lu Baolu > --- > drivers/pci/pcie/portdrv_pci.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/pci/pcie/portdrv_pci.c b/drivers/pci/pcie/portdrv_pci.c > index 35eca6277a96..2116f821c005 100644 > --- a/drivers/pci/pcie/portdrv_pci.c > +++ b/drivers/pci/pcie/portdrv_pci.c > @@ -202,6 +202,8 @@ static struct pci_driver pcie_portdriver = { > > .err_handler = &pcie_portdrv_err_handler, > > + .no_kernel_api_dma = true, > + > .driver.pm = PCIE_PORTDRV_PM_OPS, > }; > > -- > 2.25.1 >