From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754457AbcFHVGP (ORCPT ); Wed, 8 Jun 2016 17:06:15 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56891 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751796AbcFHVGN convert rfc822-to-8bit (ORCPT ); Wed, 8 Jun 2016 17:06:13 -0400 Date: Wed, 8 Jun 2016 15:06:09 -0600 From: Alex Williamson To: Auger Eric Cc: eric.auger@st.com, robin.murphy@arm.com, will.deacon@arm.com, joro@8bytes.org, tglx@linutronix.de, jason@lakedaemon.net, marc.zyngier@arm.com, christoffer.dall@linaro.org, linux-arm-kernel@lists.infradead.org, patches@linaro.org, linux-kernel@vger.kernel.org, Bharat.Bhushan@freescale.com, pranav.sawargaonkar@gmail.com, p.fedin@samsung.com, iommu@lists.linux-foundation.org, Jean-Philippe.Brucker@arm.com, julien.grall@arm.com, yehuday@marvell.com Subject: Re: [PATCH v9 0/7] KVM PCIe/MSI passthrough on ARM/ARM64: kernel part 3/3: vfio changes Message-ID: <20160608150609.7e28d63d@ul30vt.home> In-Reply-To: <875b5791-f7c9-97ca-46de-4b1474fe65e0@linaro.org> References: <1462362858-2925-1-git-send-email-eric.auger@linaro.org> <573F34CA.5080308@linaro.org> <875b5791-f7c9-97ca-46de-4b1474fe65e0@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 08 Jun 2016 21:06:12 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 8 Jun 2016 10:29:35 +0200 Auger Eric wrote: > Dear all, > Le 20/05/2016 à 18:01, Eric Auger a écrit : > > Alex, Robin, > > > > While my 3 part series primarily addresses the problematic of mapping > > MSI doorbells into arm-smmu, it fails in : > > > > 1) determining whether the MSI controller is downstream or upstream to > > the IOMMU, > > => indicates whether the MSI doorbell must be mapped > > => participates in the decision about 2) > > > > 2) determining whether it is safe to assign a PCIe device. > > > > I think we share this understanding with Robin. All above of course > > stands for ARM. > > > > I get stuck with those 2 issues and I have few questions about iommu > > group setup, PCIe, iommu dt/ACPI description. I would be grateful to you > > if you could answer part of those questions and advise about the > > strategy to fix those. > > gentle reminder about the questions below; hope I did not miss any reply. > If anybody has some time to spent on this topic... > > > > > Best Regards > > > > Eric > > > > QUESTIONS: > > > > 1) Robin, you pointed some host controllers which also are MSI > > controllers > > (http://thread.gmane.org/gmane.linux.kernel.pci/47174/focus=47268). In > > that case MSIs never reach the IOMMU. I failed in finding anything about > > MSIs in PCIe ACS spec. What should be the iommu groups in that > > situation. Isn't the upstreamed code able to see some DMA transfers are > > not properly isolated and alias devices in the same group? According to > > your security warning, Alex, I would think the code does not recognize > > it, can you confirm please? > my current understanding is end points would be in separate groups (assuming > ACS support) although MSI controller frame is not properly protected. We don't currently consider MSI differently from other DMA and we don't currently have any sort of concept of a device within the intermediate fabric as being a DMA target. We expect fabric devices to only be transaction routers. We use ACS to determine whether there's any possibility of DMA being redirected before it reaches the IOMMU, but it seems that a DMA being consumed by an interrupt controller before it reaches the IOMMU would be another cause for an isolation breach. > > 2) can other PCIe components be MSI controllers? I'm not even entirely sure what this means. Would a DMA write from an endpoint target the MMIO space of an intermediate, fabric device? > > 3) Am I obliged to consider arbitrary topologies where an MSI controller > > stands between the PCIe host and the iommu? in the PCIe space or > > platform space? If this only relates to PCIe couldn' I check if an MSI > > controller exists in the PCIe tree? > In my last series, I consider the assignment of platform device unsafe as > soon as there is a GICv2m. This is a change in the user experience compared to > what we have before. If the MSI controller is downstream of our DMA translation, it doesn't seem like we have much choice but to mark it unsafe. The endpoint is fully able to attempt to exploit it. > > 4) Robin suggested in a private thread to enumerate through a list of > > "registered" doorbells and if any belongs to an unsafe MSI controller, > > consider the assignment is unsafe. This would be a first step before > > doing something more complex. Alex, would that be acceptable to you for > > issue #2? > I implemented this technique in my last series waiting for more discussion > on 4, 5. Seems sufficient. I don't mind taking a broad swing versus all the extra complexity of defining which devices are safe vs unsafe. > > 5) About issue #1: don't we miss tools in dt/ACPI to describe the > > location of the iommu on ARM? This is not needed on x86 because > > irq_remapping and IOMMU are at the same place but my understanding is > > that it is on ARM where > > - there is no connection between the MSI controller - which implements > > irq remapping - and the iommu > > - MSI are conveyed on the same address space as standard memory > > transactions. It seems pretty dubious to me to have fixed address, unprotected MSI controllers sitting in the DMA space of a device before IOMMU translation. Seems like you not only need to mark interrupts as unsafe, but exclude the address space of the MSI controller from the available IOVA space to the user. > > 6) can't we live with iommu/MSI controller respective location uncertainty? > > > > - in my current series, with the above Xilinx MSI controller, I would > > see there is an arm-smmu requiring mapping behind the PCI host, would > > query the characteristics of the MSI doorbell (not implemented by that > > controller), so no mapping would be done. So it would work I think. > > - However in case we have this topology: PCIe host -> MSI controller > > generally used behind an IOMMU (so registering a doorbell) -> IOMMU, > > this wouldn't work since the doorbell would be mapped. I'm a little confused which direction "behind" is here, but it seems like any time the MSI controller lives in the DMA address space of the endpoint, both interfering with the available IOVA space to the user and potentially an attack vector for the user, we need to call it out as unsafe. Maybe some of them are for exclusive use of the device and the attack vector is relatively contained, but they still affect the IOVA space of the user. Such a configuration might be safe, but as I said I'm not opposed to being pretty liberal in applying the unsafe requirement if the platform has done something unfriendly. Thanks, Alex From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: [PATCH v9 0/7] KVM PCIe/MSI passthrough on ARM/ARM64: kernel part 3/3: vfio changes Date: Wed, 8 Jun 2016 15:06:09 -0600 Message-ID: <20160608150609.7e28d63d@ul30vt.home> References: <1462362858-2925-1-git-send-email-eric.auger@linaro.org> <573F34CA.5080308@linaro.org> <875b5791-f7c9-97ca-46de-4b1474fe65e0@linaro.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <875b5791-f7c9-97ca-46de-4b1474fe65e0-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Auger Eric Cc: julien.grall-5wv7dgnIgG8@public.gmane.org, eric.auger-qxv4g6HH51o@public.gmane.org, jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org, patches-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org, marc.zyngier-5wv7dgnIgG8@public.gmane.org, p.fedin-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org, will.deacon-5wv7dgnIgG8@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, pranav.sawargaonkar-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org, christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org, yehuday-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org List-Id: iommu@lists.linux-foundation.org T24gV2VkLCA4IEp1biAyMDE2IDEwOjI5OjM1ICswMjAwCkF1Z2VyIEVyaWMgPGVyaWMuYXVnZXJA bGluYXJvLm9yZz4gd3JvdGU6Cgo+IERlYXIgYWxsLAo+IExlIDIwLzA1LzIwMTYgw6AgMTg6MDEs IEVyaWMgQXVnZXIgYSDDqWNyaXQgOgo+ID4gQWxleCwgUm9iaW4sCj4gPiAKPiA+IFdoaWxlIG15 IDMgcGFydCBzZXJpZXMgcHJpbWFyaWx5IGFkZHJlc3NlcyB0aGUgcHJvYmxlbWF0aWMgb2YgbWFw cGluZwo+ID4gTVNJIGRvb3JiZWxscyBpbnRvIGFybS1zbW11LCBpdCBmYWlscyBpbiA6Cj4gPiAK PiA+IDEpIGRldGVybWluaW5nIHdoZXRoZXIgdGhlIE1TSSBjb250cm9sbGVyIGlzIGRvd25zdHJl YW0gb3IgdXBzdHJlYW0gdG8KPiA+IHRoZSBJT01NVSwgIAo+ID4gCT0+IGluZGljYXRlcyB3aGV0 aGVyIHRoZSBNU0kgZG9vcmJlbGwgbXVzdCBiZSBtYXBwZWQKPiA+IAk9PiBwYXJ0aWNpcGF0ZXMg aW4gdGhlIGRlY2lzaW9uIGFib3V0IDIpICAKPiA+IAo+ID4gMikgZGV0ZXJtaW5pbmcgd2hldGhl ciBpdCBpcyBzYWZlIHRvIGFzc2lnbiBhIFBDSWUgZGV2aWNlLgo+ID4gCj4gPiBJIHRoaW5rIHdl IHNoYXJlIHRoaXMgdW5kZXJzdGFuZGluZyB3aXRoIFJvYmluLiBBbGwgYWJvdmUgb2YgY291cnNl Cj4gPiBzdGFuZHMgZm9yIEFSTS4KPiA+IAo+ID4gSSBnZXQgc3R1Y2sgd2l0aCB0aG9zZSAyIGlz c3VlcyBhbmQgSSBoYXZlIGZldyBxdWVzdGlvbnMgYWJvdXQgaW9tbXUKPiA+IGdyb3VwIHNldHVw LCBQQ0llLCBpb21tdSBkdC9BQ1BJIGRlc2NyaXB0aW9uLiBJIHdvdWxkIGJlIGdyYXRlZnVsIHRv IHlvdQo+ID4gaWYgeW91IGNvdWxkIGFuc3dlciBwYXJ0IG9mIHRob3NlIHF1ZXN0aW9ucyBhbmQg YWR2aXNlIGFib3V0IHRoZQo+ID4gc3RyYXRlZ3kgdG8gZml4IHRob3NlLiAgCj4gCj4gZ2VudGxl IHJlbWluZGVyIGFib3V0IHRoZSBxdWVzdGlvbnMgYmVsb3c7IGhvcGUgSSBkaWQgbm90IG1pc3Mg YW55IHJlcGx5Lgo+IElmIGFueWJvZHkgaGFzIHNvbWUgdGltZSB0byBzcGVudCBvbiB0aGlzIHRv cGljLi4uCj4gCj4gPiAKPiA+IEJlc3QgUmVnYXJkcwo+ID4gCj4gPiBFcmljCj4gPiAKPiA+IFFV RVNUSU9OUzoKPiA+IAo+ID4gMSkgUm9iaW4sIHlvdSBwb2ludGVkIHNvbWUgaG9zdCBjb250cm9s bGVycyB3aGljaCBhbHNvIGFyZSBNU0kKPiA+IGNvbnRyb2xsZXJzCj4gPiAoaHR0cDovL3RocmVh ZC5nbWFuZS5vcmcvZ21hbmUubGludXgua2VybmVsLnBjaS80NzE3NC9mb2N1cz00NzI2OCkuIElu Cj4gPiB0aGF0IGNhc2UgTVNJcyBuZXZlciByZWFjaCB0aGUgSU9NTVUuIEkgZmFpbGVkIGluIGZp bmRpbmcgYW55dGhpbmcgYWJvdXQKPiA+IE1TSXMgaW4gUENJZSBBQ1Mgc3BlYy4gV2hhdCBzaG91 bGQgYmUgdGhlIGlvbW11IGdyb3VwcyBpbiB0aGF0Cj4gPiBzaXR1YXRpb24uIElzbid0IHRoZSB1 cHN0cmVhbWVkIGNvZGUgYWJsZSB0byBzZWUgc29tZSBETUEgdHJhbnNmZXJzIGFyZQo+ID4gbm90 IHByb3Blcmx5IGlzb2xhdGVkIGFuZCBhbGlhcyBkZXZpY2VzIGluIHRoZSBzYW1lIGdyb3VwPyBB Y2NvcmRpbmcgdG8KPiA+IHlvdXIgc2VjdXJpdHkgd2FybmluZywgQWxleCwgSSB3b3VsZCB0aGlu ayB0aGUgY29kZSBkb2VzIG5vdCByZWNvZ25pemUKPiA+IGl0LCBjYW4geW91IGNvbmZpcm0gcGxl YXNlPyAgCj4gbXkgY3VycmVudCB1bmRlcnN0YW5kaW5nIGlzIGVuZCBwb2ludHMgd291bGQgYmUg aW4gc2VwYXJhdGUgZ3JvdXBzIChhc3N1bWluZwo+IEFDUyBzdXBwb3J0KSBhbHRob3VnaCBNU0kg Y29udHJvbGxlciBmcmFtZSBpcyBub3QgcHJvcGVybHkgcHJvdGVjdGVkLgoKV2UgZG9uJ3QgY3Vy cmVudGx5IGNvbnNpZGVyIE1TSSBkaWZmZXJlbnRseSBmcm9tIG90aGVyIERNQSBhbmQgd2UgZG9u J3QKY3VycmVudGx5IGhhdmUgYW55IHNvcnQgb2YgY29uY2VwdCBvZiBhIGRldmljZSB3aXRoaW4g dGhlIGludGVybWVkaWF0ZQpmYWJyaWMgYXMgYmVpbmcgYSBETUEgdGFyZ2V0LiAgV2UgZXhwZWN0 IGZhYnJpYyBkZXZpY2VzIHRvIG9ubHkgYmUKdHJhbnNhY3Rpb24gcm91dGVycy4gIFdlIHVzZSBB Q1MgdG8gZGV0ZXJtaW5lIHdoZXRoZXIgdGhlcmUncyBhbnkKcG9zc2liaWxpdHkgb2YgRE1BIGJl aW5nIHJlZGlyZWN0ZWQgYmVmb3JlIGl0IHJlYWNoZXMgdGhlIElPTU1VLCBidXQgaXQKc2VlbXMg dGhhdCBhIERNQSBiZWluZyBjb25zdW1lZCBieSBhbiBpbnRlcnJ1cHQgY29udHJvbGxlciBiZWZv cmUgaXQKcmVhY2hlcyB0aGUgSU9NTVUgd291bGQgYmUgYW5vdGhlciBjYXVzZSBmb3IgYW4gaXNv bGF0aW9uIGJyZWFjaC4KIAo+ID4gMikgY2FuIG90aGVyIFBDSWUgY29tcG9uZW50cyBiZSBNU0kg Y29udHJvbGxlcnM/CgpJJ20gbm90IGV2ZW4gZW50aXJlbHkgc3VyZSB3aGF0IHRoaXMgbWVhbnMu ICBXb3VsZCBhIERNQSB3cml0ZSBmcm9tIGFuCmVuZHBvaW50IHRhcmdldCB0aGUgTU1JTyBzcGFj ZSBvZiBhbiBpbnRlcm1lZGlhdGUsIGZhYnJpYyBkZXZpY2U/CiAKPiA+IDMpIEFtIEkgb2JsaWdl ZCB0byBjb25zaWRlciBhcmJpdHJhcnkgdG9wb2xvZ2llcyB3aGVyZSBhbiBNU0kgY29udHJvbGxl cgo+ID4gc3RhbmRzIGJldHdlZW4gdGhlIFBDSWUgaG9zdCBhbmQgdGhlIGlvbW11PyBpbiB0aGUg UENJZSBzcGFjZSBvcgo+ID4gcGxhdGZvcm0gc3BhY2U/IElmIHRoaXMgb25seSByZWxhdGVzIHRv IFBDSWUgY291bGRuJyBJIGNoZWNrIGlmIGFuIE1TSQo+ID4gY29udHJvbGxlciBleGlzdHMgaW4g dGhlIFBDSWUgdHJlZT8gIAo+IEluIG15IGxhc3Qgc2VyaWVzLCBJIGNvbnNpZGVyIHRoZSBhc3Np Z25tZW50IG9mIHBsYXRmb3JtIGRldmljZSB1bnNhZmUgYXMKPiBzb29uIGFzIHRoZXJlIGlzIGEg R0lDdjJtLiBUaGlzIGlzIGEgY2hhbmdlIGluIHRoZSB1c2VyIGV4cGVyaWVuY2UgY29tcGFyZWQg dG8KPiB3aGF0IHdlIGhhdmUgYmVmb3JlLgoKSWYgdGhlIE1TSSBjb250cm9sbGVyIGlzIGRvd25z dHJlYW0gb2Ygb3VyIERNQSB0cmFuc2xhdGlvbiwgaXQgZG9lc24ndApzZWVtIGxpa2Ugd2UgaGF2 ZSBtdWNoIGNob2ljZSBidXQgdG8gbWFyayBpdCB1bnNhZmUuICBUaGUgZW5kcG9pbnQgaXMKZnVs bHkgYWJsZSB0byBhdHRlbXB0IHRvIGV4cGxvaXQgaXQuCiAKPiA+IDQpIFJvYmluIHN1Z2dlc3Rl ZCBpbiBhIHByaXZhdGUgdGhyZWFkIHRvIGVudW1lcmF0ZSB0aHJvdWdoIGEgbGlzdCBvZgo+ID4g InJlZ2lzdGVyZWQiIGRvb3JiZWxscyBhbmQgaWYgYW55IGJlbG9uZ3MgdG8gYW4gdW5zYWZlIE1T SSBjb250cm9sbGVyLAo+ID4gY29uc2lkZXIgdGhlIGFzc2lnbm1lbnQgaXMgdW5zYWZlLiBUaGlz IHdvdWxkIGJlIGEgZmlyc3Qgc3RlcCBiZWZvcmUKPiA+IGRvaW5nIHNvbWV0aGluZyBtb3JlIGNv bXBsZXguIEFsZXgsIHdvdWxkIHRoYXQgYmUgYWNjZXB0YWJsZSB0byB5b3UgZm9yCj4gPiBpc3N1 ZSAjMj8gIAo+IEkgaW1wbGVtZW50ZWQgdGhpcyB0ZWNobmlxdWUgaW4gbXkgbGFzdCBzZXJpZXMg d2FpdGluZyBmb3IgbW9yZSBkaXNjdXNzaW9uCj4gb24gNCwgNS4KClNlZW1zIHN1ZmZpY2llbnQu ICBJIGRvbid0IG1pbmQgdGFraW5nIGEgYnJvYWQgc3dpbmcgdmVyc3VzIGFsbCB0aGUKZXh0cmEg Y29tcGxleGl0eSBvZiBkZWZpbmluZyB3aGljaCBkZXZpY2VzIGFyZSBzYWZlIHZzIHVuc2FmZS4K IAo+ID4gNSkgQWJvdXQgaXNzdWUgIzE6IGRvbid0IHdlIG1pc3MgdG9vbHMgaW4gZHQvQUNQSSB0 byBkZXNjcmliZSB0aGUKPiA+IGxvY2F0aW9uIG9mIHRoZSBpb21tdSBvbiBBUk0/IFRoaXMgaXMg bm90IG5lZWRlZCBvbiB4ODYgYmVjYXVzZQo+ID4gaXJxX3JlbWFwcGluZyBhbmQgSU9NTVUgYXJl IGF0IHRoZSBzYW1lIHBsYWNlIGJ1dCBteSB1bmRlcnN0YW5kaW5nIGlzCj4gPiB0aGF0IGl0IGlz IG9uIEFSTSB3aGVyZQo+ID4gLSB0aGVyZSBpcyBubyBjb25uZWN0aW9uIGJldHdlZW4gdGhlIE1T SSBjb250cm9sbGVyIC0gd2hpY2ggaW1wbGVtZW50cwo+ID4gaXJxIHJlbWFwcGluZyAtIGFuZCB0 aGUgaW9tbXUKPiA+IC0gTVNJIGFyZSBjb252ZXllZCBvbiB0aGUgc2FtZSBhZGRyZXNzIHNwYWNl IGFzIHN0YW5kYXJkIG1lbW9yeQo+ID4gdHJhbnNhY3Rpb25zLgoKSXQgc2VlbXMgcHJldHR5IGR1 YmlvdXMgdG8gbWUgdG8gaGF2ZSBmaXhlZCBhZGRyZXNzLCB1bnByb3RlY3RlZCBNU0kKY29udHJv bGxlcnMgc2l0dGluZyBpbiB0aGUgRE1BIHNwYWNlIG9mIGEgZGV2aWNlIGJlZm9yZSBJT01NVQp0 cmFuc2xhdGlvbi4gIFNlZW1zIGxpa2UgeW91IG5vdCBvbmx5IG5lZWQgdG8gbWFyayBpbnRlcnJ1 cHRzIGFzCnVuc2FmZSwgYnV0IGV4Y2x1ZGUgdGhlIGFkZHJlc3Mgc3BhY2Ugb2YgdGhlIE1TSSBj b250cm9sbGVyIGZyb20gdGhlCmF2YWlsYWJsZSBJT1ZBIHNwYWNlIHRvIHRoZSB1c2VyLgogCj4g PiA2KSAgY2FuJ3Qgd2UgbGl2ZSB3aXRoIGlvbW11L01TSSBjb250cm9sbGVyIHJlc3BlY3RpdmUg bG9jYXRpb24gdW5jZXJ0YWludHk/Cj4gPiAKPiA+IC0gaW4gbXkgY3VycmVudCBzZXJpZXMsIHdp dGggdGhlIGFib3ZlIFhpbGlueCBNU0kgY29udHJvbGxlciwgSSB3b3VsZAo+ID4gc2VlIHRoZXJl IGlzIGFuIGFybS1zbW11IHJlcXVpcmluZyBtYXBwaW5nIGJlaGluZCB0aGUgUENJIGhvc3QsIHdv dWxkCj4gPiBxdWVyeSB0aGUgY2hhcmFjdGVyaXN0aWNzIG9mIHRoZSBNU0kgZG9vcmJlbGwgKG5v dCBpbXBsZW1lbnRlZCBieSB0aGF0Cj4gPiBjb250cm9sbGVyKSwgc28gbm8gbWFwcGluZyB3b3Vs ZCBiZSBkb25lLiBTbyBpdCB3b3VsZCB3b3JrIEkgdGhpbmsuCj4gPiAtIEhvd2V2ZXIgaW4gY2Fz ZSB3ZSBoYXZlIHRoaXMgdG9wb2xvZ3k6IFBDSWUgaG9zdCAtPiBNU0kgY29udHJvbGxlcgo+ID4g Z2VuZXJhbGx5IHVzZWQgYmVoaW5kIGFuIElPTU1VIChzbyByZWdpc3RlcmluZyBhIGRvb3JiZWxs KSAtPiBJT01NVSwKPiA+IHRoaXMgd291bGRuJ3Qgd29yayBzaW5jZSB0aGUgZG9vcmJlbGwgd291 bGQgYmUgbWFwcGVkLiAgCgpJJ20gYSBsaXR0bGUgY29uZnVzZWQgd2hpY2ggZGlyZWN0aW9uICJi ZWhpbmQiIGlzIGhlcmUsIGJ1dCBpdCBzZWVtcwpsaWtlIGFueSB0aW1lIHRoZSBNU0kgY29udHJv bGxlciBsaXZlcyBpbiB0aGUgRE1BIGFkZHJlc3Mgc3BhY2Ugb2YgdGhlCmVuZHBvaW50LCBib3Ro IGludGVyZmVyaW5nIHdpdGggdGhlIGF2YWlsYWJsZSBJT1ZBIHNwYWNlIHRvIHRoZSB1c2VyCmFu ZCBwb3RlbnRpYWxseSBhbiBhdHRhY2sgdmVjdG9yIGZvciB0aGUgdXNlciwgd2UgbmVlZCB0byBj YWxsIGl0IG91dAphcyB1bnNhZmUuICBNYXliZSBzb21lIG9mIHRoZW0gYXJlIGZvciBleGNsdXNp dmUgdXNlIG9mIHRoZSBkZXZpY2UgYW5kCnRoZSBhdHRhY2sgdmVjdG9yIGlzIHJlbGF0aXZlbHkg Y29udGFpbmVkLCBidXQgdGhleSBzdGlsbCBhZmZlY3QgdGhlCklPVkEgc3BhY2Ugb2YgdGhlIHVz ZXIuICBTdWNoIGEgY29uZmlndXJhdGlvbiBtaWdodCBiZSBzYWZlLCBidXQgYXMgSQpzYWlkIEkn bSBub3Qgb3Bwb3NlZCB0byBiZWluZyBwcmV0dHkgbGliZXJhbCBpbiBhcHBseWluZyB0aGUgdW5z YWZlCnJlcXVpcmVtZW50IGlmIHRoZSBwbGF0Zm9ybSBoYXMgZG9uZSBzb21ldGhpbmcgdW5mcmll bmRseS4gIFRoYW5rcywKCkFsZXgKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11QGxpc3RzLmxpbnV4LWZvdW5kYXRp b24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZv L2lvbW11 From mboxrd@z Thu Jan 1 00:00:00 1970 From: alex.williamson@redhat.com (Alex Williamson) Date: Wed, 8 Jun 2016 15:06:09 -0600 Subject: [PATCH v9 0/7] KVM PCIe/MSI passthrough on ARM/ARM64: kernel part 3/3: vfio changes In-Reply-To: <875b5791-f7c9-97ca-46de-4b1474fe65e0@linaro.org> References: <1462362858-2925-1-git-send-email-eric.auger@linaro.org> <573F34CA.5080308@linaro.org> <875b5791-f7c9-97ca-46de-4b1474fe65e0@linaro.org> Message-ID: <20160608150609.7e28d63d@ul30vt.home> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Wed, 8 Jun 2016 10:29:35 +0200 Auger Eric wrote: > Dear all, > Le 20/05/2016 ? 18:01, Eric Auger a ?crit : > > Alex, Robin, > > > > While my 3 part series primarily addresses the problematic of mapping > > MSI doorbells into arm-smmu, it fails in : > > > > 1) determining whether the MSI controller is downstream or upstream to > > the IOMMU, > > => indicates whether the MSI doorbell must be mapped > > => participates in the decision about 2) > > > > 2) determining whether it is safe to assign a PCIe device. > > > > I think we share this understanding with Robin. All above of course > > stands for ARM. > > > > I get stuck with those 2 issues and I have few questions about iommu > > group setup, PCIe, iommu dt/ACPI description. I would be grateful to you > > if you could answer part of those questions and advise about the > > strategy to fix those. > > gentle reminder about the questions below; hope I did not miss any reply. > If anybody has some time to spent on this topic... > > > > > Best Regards > > > > Eric > > > > QUESTIONS: > > > > 1) Robin, you pointed some host controllers which also are MSI > > controllers > > (http://thread.gmane.org/gmane.linux.kernel.pci/47174/focus=47268). In > > that case MSIs never reach the IOMMU. I failed in finding anything about > > MSIs in PCIe ACS spec. What should be the iommu groups in that > > situation. Isn't the upstreamed code able to see some DMA transfers are > > not properly isolated and alias devices in the same group? According to > > your security warning, Alex, I would think the code does not recognize > > it, can you confirm please? > my current understanding is end points would be in separate groups (assuming > ACS support) although MSI controller frame is not properly protected. We don't currently consider MSI differently from other DMA and we don't currently have any sort of concept of a device within the intermediate fabric as being a DMA target. We expect fabric devices to only be transaction routers. We use ACS to determine whether there's any possibility of DMA being redirected before it reaches the IOMMU, but it seems that a DMA being consumed by an interrupt controller before it reaches the IOMMU would be another cause for an isolation breach. > > 2) can other PCIe components be MSI controllers? I'm not even entirely sure what this means. Would a DMA write from an endpoint target the MMIO space of an intermediate, fabric device? > > 3) Am I obliged to consider arbitrary topologies where an MSI controller > > stands between the PCIe host and the iommu? in the PCIe space or > > platform space? If this only relates to PCIe couldn' I check if an MSI > > controller exists in the PCIe tree? > In my last series, I consider the assignment of platform device unsafe as > soon as there is a GICv2m. This is a change in the user experience compared to > what we have before. If the MSI controller is downstream of our DMA translation, it doesn't seem like we have much choice but to mark it unsafe. The endpoint is fully able to attempt to exploit it. > > 4) Robin suggested in a private thread to enumerate through a list of > > "registered" doorbells and if any belongs to an unsafe MSI controller, > > consider the assignment is unsafe. This would be a first step before > > doing something more complex. Alex, would that be acceptable to you for > > issue #2? > I implemented this technique in my last series waiting for more discussion > on 4, 5. Seems sufficient. I don't mind taking a broad swing versus all the extra complexity of defining which devices are safe vs unsafe. > > 5) About issue #1: don't we miss tools in dt/ACPI to describe the > > location of the iommu on ARM? This is not needed on x86 because > > irq_remapping and IOMMU are at the same place but my understanding is > > that it is on ARM where > > - there is no connection between the MSI controller - which implements > > irq remapping - and the iommu > > - MSI are conveyed on the same address space as standard memory > > transactions. It seems pretty dubious to me to have fixed address, unprotected MSI controllers sitting in the DMA space of a device before IOMMU translation. Seems like you not only need to mark interrupts as unsafe, but exclude the address space of the MSI controller from the available IOVA space to the user. > > 6) can't we live with iommu/MSI controller respective location uncertainty? > > > > - in my current series, with the above Xilinx MSI controller, I would > > see there is an arm-smmu requiring mapping behind the PCI host, would > > query the characteristics of the MSI doorbell (not implemented by that > > controller), so no mapping would be done. So it would work I think. > > - However in case we have this topology: PCIe host -> MSI controller > > generally used behind an IOMMU (so registering a doorbell) -> IOMMU, > > this wouldn't work since the doorbell would be mapped. I'm a little confused which direction "behind" is here, but it seems like any time the MSI controller lives in the DMA address space of the endpoint, both interfering with the available IOVA space to the user and potentially an attack vector for the user, we need to call it out as unsafe. Maybe some of them are for exclusive use of the device and the attack vector is relatively contained, but they still affect the IOVA space of the user. Such a configuration might be safe, but as I said I'm not opposed to being pretty liberal in applying the unsafe requirement if the platform has done something unfriendly. Thanks, Alex