From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41841) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XgYNk-0000kH-2l for qemu-devel@nongnu.org; Tue, 21 Oct 2014 08:17:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XgYNf-0005Tq-Nz for qemu-devel@nongnu.org; Tue, 21 Oct 2014 08:17:44 -0400 Date: Tue, 21 Oct 2014 15:20:34 +0300 From: "Michael S. Tsirkin" Message-ID: <20141021122034.GB23466@redhat.com> References: <1413844443-28894-1-git-send-email-knut.omang@oracle.com> <1413869211.17541.24.camel@ori.omang.mine.nu> <1413884154.8961.6.camel@ori.omang.mine.nu> <54464042.7000408@suse.de> <20141021112603.GA23199@redhat.com> <1413891453.8961.17.camel@ori.omang.mine.nu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1413891453.8961.17.camel@ori.omang.mine.nu> Subject: Re: [Qemu-devel] [PATCH 0/2] intel_iommu: Add support for translation for devices behind bridges List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Knut Omang Cc: "jroedel@suse.de" , Alex Williamson , Marcel Apfelbaum , Michael Tokarev , Mark Cave-Ayland , Alexander Graf , "qemu-devel@nongnu.org" , Andreas =?iso-8859-1?Q?F=E4rber?= , =?iso-8859-1?Q?Herv=E9?= Poussineau , Anthony Liguori , Stefan Weil , "qemu-ppc@nongnu.org" , Markus Armbruster , Richard Henderson On Tue, Oct 21, 2014 at 01:37:33PM +0200, Knut Omang wrote: > On Tue, 2014-10-21 at 14:26 +0300, Michael S. Tsirkin wrote: > > On Tue, Oct 21, 2014 at 01:15:14PM +0200, Alexander Graf wrote: > > > > > > > > > On 21.10.14 11:35, Knut Omang wrote: > > > > On Tue, 2014-10-21 at 11:07 +0200, Alexander Graf wrote: > > > >> > > > >> > > > >>> Am 21.10.2014 um 07:26 schrieb Knut Omang : > > > >>> > > > >>>> On Tue, 2014-10-21 at 01:29 +0200, Alexander Graf wrote: > > > >>>> > > > >>>> > > > >>>>> Am 21.10.2014 um 00:34 schrieb Knut Omang : > > > >>>>> > > > >>>>> This patch set changes the data structure used to handle address spaces within > > > >>>>> the emulated Intel iommu to support traversal also if bus numbers are dynamically > > > >>>>> allocated, as is the case for devices that sit behind root ports or downstream switches. > > > >>>>> This means that we cannot use bus number as index, instead a QLIST is used. > > > >>>>> > > > >>>>> This requires a change in the API for setup of IOMMUs which is taken care of by > > > >>>>> the first patch. The second patch implements the fix. > > > >>>> > > > >>>> Are you sure that this works on real hardware? How does that one > > > >>>> communicate sub-bridge liodns to the iommu? How do they get indexed > > > >>>> from software? > > > >>> > > > >>> I do not claim to fully understand the details of how this is > > > >>> implemented in hardware, but I believe the implementation I propose here > > > >>> should be functionally equivalent to what the Intel IOMMU offers, and > > > >>> similar to the original implementation here, except that the data > > > >>> structure is valid also before enumeration when behind buses. > > > >> > > > >> Can you please give me a pointer to the vt-d spec's section that explains iommu behavior behind bridges? > > > >> > > > >> I've also added Alex W who has played with PCI bridges behind iommus quite a bit recently. > > > >> > > > >>> > > > >>> After enumeration, the only difference would be that during > > > >>> invalidation, there is a list search for the right bus rather than an > > > >>> index lookup as before, slightly less efficient but at the benefit of > > > >>> being independent of bus numbering during setup. > > > >> > > > >> I don't think the implementation is bad, I'm just not sure that it follows the spec, > > > >> so I want to confirm :). > > > > > > > > http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf > > > > > > So if I understand that document correctly, a PCIe / PCI-X bridge can > > > swizzle the requester id depending on a device behind itself. PCI > > > bridges can not - there everything behind the bridge will appear as if > > > the DMA originated from the bridge device. > > > > > > So conceptually, PCIe / PCI-X bridges should probably be the ones > > > converting requester IDs. > > > > > > > > > Alex > > > > To avoid confusion, when you say PCIe / PCI-X you really mean > > PCI-Express-to-PCI/PCI-X. > > > > I think you got this right. > > > > I'd like to add that regular PCIe to PCIe bridges just forward everything > > without changes. This applies to root complex and downstream ports. > > My test case for this patch set has been just the root port/downstream > port case, where as you say the requester ID seen by the device in the > root port is always the same as the one used/enumerated by the host. > > Knut > So pci/pci and pcie/pci cases are still TBD. Again, if we can't make it in time for 2.2, I'm fine with disallowing these configurations. -- MST