From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030510AbbKDPac (ORCPT ); Wed, 4 Nov 2015 10:30:32 -0500 Received: from foss.arm.com ([217.140.101.70]:60960 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030368AbbKDPa3 (ORCPT ); Wed, 4 Nov 2015 10:30:29 -0500 Date: Wed, 4 Nov 2015 15:30:29 +0000 From: Will Deacon To: Phil Edworthy Cc: "Liviu.Dudau@arm.com" , Lorenzo Pieralisi , Arnd Bergmann , "linux-pci@vger.kernel.org" , Magnus , "linux-kernel@vger.kernel.org" , Bjorn Helgaas , "linux-arm-kernel@lists.infradead.org" Subject: Re: PCIe host controller behind IOMMU on ARM Message-ID: <20151104153028.GR5405@arm.com> References: <20151104142412.GS963@e106497-lin.cambridge.arm.com> <20151104150147.GT963@e106497-lin.cambridge.arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Nov 04, 2015 at 03:19:13PM +0000, Phil Edworthy wrote: > On 04 November 2015 15:02, Liviu wrote: > > On Wed, Nov 04, 2015 at 02:48:38PM +0000, Phil Edworthy wrote: > > > Sure, though since this is bog standard Intel PCIe ethernet card which works > > > fine when the IOMMU is effectively unused, I don’t think there is a problem > > > with that. > > > > > > The driver for the PCIe controller sets up the IOMMU mapping ok when I > > > do a test call to dma_alloc_coherent() in the controller's driver. i.e. when I > > > do this, it ends up in arm_iommu_alloc_attrs(), which calls > > > __iommu_alloc_buffer() and __alloc_iova(). > > > > > > When an endpoint driver allocates and maps a dma coherent buffer it > > > also needs to end up in arm_iommu_alloc_attrs(), but it doesn't. > > > > Why do you think that? Remember that the only thing attached to the IOMMU is > > the > > host controller. The endpoint is on the PCIe bus, which gets a different > > translation > > that the IOMMU knows nothing about. If it helps you to visualise it better, think > > of the host controller as another IOMMU device. It's the ops of the host > > controller > > that should be invoked, not the IOMMU's. > Ok, that makes sense. I'll have a think and poke it a bit more... Take a look at of_iommu_configure, which is currently lacking support for PCI devices. It should be using a variant on the device-tree bindings already in use for describing MSI device IDs, so that we can translate the RequesterID of the endpoint into an ID that the IOMMU can understand. Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 4 Nov 2015 15:30:29 +0000 From: Will Deacon To: Phil Edworthy Cc: "Liviu.Dudau@arm.com" , Lorenzo Pieralisi , Arnd Bergmann , "linux-pci@vger.kernel.org" , Magnus , "linux-kernel@vger.kernel.org" , Bjorn Helgaas , "linux-arm-kernel@lists.infradead.org" Subject: Re: PCIe host controller behind IOMMU on ARM Message-ID: <20151104153028.GR5405@arm.com> References: <20151104142412.GS963@e106497-lin.cambridge.arm.com> <20151104150147.GT963@e106497-lin.cambridge.arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 In-Reply-To: Sender: linux-kernel-owner@vger.kernel.org List-ID: On Wed, Nov 04, 2015 at 03:19:13PM +0000, Phil Edworthy wrote: > On 04 November 2015 15:02, Liviu wrote: > > On Wed, Nov 04, 2015 at 02:48:38PM +0000, Phil Edworthy wrote: > > > Sure, though since this is bog standard Intel PCIe ethernet card which works > > > fine when the IOMMU is effectively unused, I don’t think there is a problem > > > with that. > > > > > > The driver for the PCIe controller sets up the IOMMU mapping ok when I > > > do a test call to dma_alloc_coherent() in the controller's driver. i.e. when I > > > do this, it ends up in arm_iommu_alloc_attrs(), which calls > > > __iommu_alloc_buffer() and __alloc_iova(). > > > > > > When an endpoint driver allocates and maps a dma coherent buffer it > > > also needs to end up in arm_iommu_alloc_attrs(), but it doesn't. > > > > Why do you think that? Remember that the only thing attached to the IOMMU is > > the > > host controller. The endpoint is on the PCIe bus, which gets a different > > translation > > that the IOMMU knows nothing about. If it helps you to visualise it better, think > > of the host controller as another IOMMU device. It's the ops of the host > > controller > > that should be invoked, not the IOMMU's. > Ok, that makes sense. I'll have a think and poke it a bit more... Take a look at of_iommu_configure, which is currently lacking support for PCI devices. It should be using a variant on the device-tree bindings already in use for describing MSI device IDs, so that we can translate the RequesterID of the endpoint into an ID that the IOMMU can understand. Will From mboxrd@z Thu Jan 1 00:00:00 1970 From: will.deacon@arm.com (Will Deacon) Date: Wed, 4 Nov 2015 15:30:29 +0000 Subject: PCIe host controller behind IOMMU on ARM In-Reply-To: References: <20151104142412.GS963@e106497-lin.cambridge.arm.com> <20151104150147.GT963@e106497-lin.cambridge.arm.com> Message-ID: <20151104153028.GR5405@arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Wed, Nov 04, 2015 at 03:19:13PM +0000, Phil Edworthy wrote: > On 04 November 2015 15:02, Liviu wrote: > > On Wed, Nov 04, 2015 at 02:48:38PM +0000, Phil Edworthy wrote: > > > Sure, though since this is bog standard Intel PCIe ethernet card which works > > > fine when the IOMMU is effectively unused, I don?t think there is a problem > > > with that. > > > > > > The driver for the PCIe controller sets up the IOMMU mapping ok when I > > > do a test call to dma_alloc_coherent() in the controller's driver. i.e. when I > > > do this, it ends up in arm_iommu_alloc_attrs(), which calls > > > __iommu_alloc_buffer() and __alloc_iova(). > > > > > > When an endpoint driver allocates and maps a dma coherent buffer it > > > also needs to end up in arm_iommu_alloc_attrs(), but it doesn't. > > > > Why do you think that? Remember that the only thing attached to the IOMMU is > > the > > host controller. The endpoint is on the PCIe bus, which gets a different > > translation > > that the IOMMU knows nothing about. If it helps you to visualise it better, think > > of the host controller as another IOMMU device. It's the ops of the host > > controller > > that should be invoked, not the IOMMU's. > Ok, that makes sense. I'll have a think and poke it a bit more... Take a look at of_iommu_configure, which is currently lacking support for PCI devices. It should be using a variant on the device-tree bindings already in use for describing MSI device IDs, so that we can translate the RequesterID of the endpoint into an ID that the IOMMU can understand. Will