From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752651AbcEMJSK (ORCPT ); Fri, 13 May 2016 05:18:10 -0400 Received: from smtp-out4.electric.net ([192.162.216.181]:56908 "EHLO smtp-out4.electric.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752594AbcEMJSD convert rfc822-to-8bit (ORCPT ); Fri, 13 May 2016 05:18:03 -0400 From: David Laight To: "'Alex Williamson'" , "Tian, Kevin" CC: Yongji Xie , "kvm@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-pci@vger.kernel.org" , "linuxppc-dev@lists.ozlabs.org" , "iommu@lists.linux-foundation.org" , "bhelgaas@google.com" , "aik@ozlabs.ru" , "benh@kernel.crashing.org" , "paulus@samba.org" , "mpe@ellerman.id.au" , "joro@8bytes.org" , "warrier@linux.vnet.ibm.com" , "zhong@linux.vnet.ibm.com" , "nikunj@linux.vnet.ibm.com" , "eric.auger@linaro.org" , "will.deacon@arm.com" , "gwshan@linux.vnet.ibm.com" , "alistair@popple.id.au" , "ruscur@russell.cc" Subject: RE: [PATCH 5/5] vfio-pci: Allow to mmap MSI-X table if interrupt remapping is supported Thread-Topic: [PATCH 5/5] vfio-pci: Allow to mmap MSI-X table if interrupt remapping is supported Thread-Index: AQHRoIMDw+bjSjW8wkusiJ7fWPCsmJ+muJEA///6CQCAAAPhgIAAFBmAgANG2ICAABQKQIAADzIAgAAJQgCAAC9YgIAI3coAgACdsoCAAJ4zAIAAEQkAgAAqpICAANhUgIAAkuIAgAAyJACAAEYjcA== Date: Fri, 13 May 2016 09:16:18 +0000 Message-ID: <063D6719AE5E284EB5DD2968C1650D6D5F4BBEDD@AcuExch.aculab.com> References: <1461761010-5452-1-git-send-email-xyjxie@linux.vnet.ibm.com> <063D6719AE5E284EB5DD2968C1650D6D5F4B52B5@AcuExch.aculab.com> <4be013bc-e81b-84c5-06d3-e1b3f46b3227@linux.vnet.ibm.com> <20160505090513.56886c12@t450s.home> <20160511095331.18436241@t450s.home> <20160511202042.77593861@t450s.home> <20160512114735.7ec61bd3@t450s.home> <20160512233246.347b8b3c@t450s.home> In-Reply-To: <20160512233246.347b8b3c@t450s.home> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.202.99.200] Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 8BIT MIME-Version: 1.0 X-Outbound-IP: 213.249.233.130 X-Env-From: David.Laight@ACULAB.COM X-PolicySMART: 3396946, 3397078 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Alex Williamson [mailto:alex.williamson@redhat.com] > Sent: 13 May 2016 06:33 ... > Simply denying direct writes to the vector table or preventing mapping > of the vector table into the user address space does not provide any > tangible form of protection. Many devices make use of window registers > that allow backdoors to arbitrary device registers. Some drivers even > use this as the primary means for configuring MSI-X, which makes them > incompatible with device assignment without device specific quirks to > enable virtualization of these paths. We have one fgpa based PCIe slave where the device driver has to read the MSI-X table and then write the value to other fpga registers so that the logic can generate the correct PCIe write cycle when an interrupt is requested. The MSI-X table itself is only as a PCIe slave. We also have host accessible DMA controllers that the device driver uses to copy data to kernel memory. These could easily be used to generate arbitrary MSI-X requests. As I've said earlier it is almost certainly possible to get any ethernet hardware to perform something similar. So without hardware that is able to limit the memory and MSI-X that each PCIe endpoint can access I believe that if a virtualisation system gives a guest kernel direct access to a PCIe devices it gives the guest kernel the ability to raise and MSI-X interrupt and read/write any physical memory. (I've not looked at the cpu virtualisation support, but do know what the PCIe devices can do.) More interestingly, probably the 'worst' thing (from a security point of view) that changing the MSI-X table lets you do is a write to an arbitrary physical memory address. David From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Laight Subject: RE: [PATCH 5/5] vfio-pci: Allow to mmap MSI-X table if interrupt remapping is supported Date: Fri, 13 May 2016 09:16:18 +0000 Message-ID: <063D6719AE5E284EB5DD2968C1650D6D5F4BBEDD@AcuExch.aculab.com> References: <1461761010-5452-1-git-send-email-xyjxie@linux.vnet.ibm.com> <063D6719AE5E284EB5DD2968C1650D6D5F4B52B5@AcuExch.aculab.com> <4be013bc-e81b-84c5-06d3-e1b3f46b3227@linux.vnet.ibm.com> <20160505090513.56886c12@t450s.home> <20160511095331.18436241@t450s.home> <20160511202042.77593861@t450s.home> <20160512114735.7ec61bd3@t450s.home> <20160512233246.347b8b3c@t450s.home> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: "alistair-Y4h6yKqj69EXC2x5gXVKYQ@public.gmane.org" , "nikunj-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org" , "zhong-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org" , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "eric.auger-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org" , "aik-sLpHqDYs0B2HXe+LvDLADg@public.gmane.org" , "linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "ruscur-3Su/lFKaw5ejKv3TNrM5DQ@public.gmane.org" , "will.deacon-5wv7dgnIgG8@public.gmane.org" , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "gwshan-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org" , "warrier-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org" , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , Yongji Xie , "mpe-Gsx/Oe8HsFggBc27wqDAHg@public.gmane.org" , "benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org" , "bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org" , "paulus@samba.o To: 'Alex Williamson' , "Tian, Kevin" Return-path: In-Reply-To: <20160512233246.347b8b3c-1yVPhWWZRC1BDLzU/O5InQ@public.gmane.org> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org List-Id: kvm.vger.kernel.org From: Alex Williamson [mailto:alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org] > Sent: 13 May 2016 06:33 ... > Simply denying direct writes to the vector table or preventing mapping > of the vector table into the user address space does not provide any > tangible form of protection. Many devices make use of window registers > that allow backdoors to arbitrary device registers. Some drivers even > use this as the primary means for configuring MSI-X, which makes them > incompatible with device assignment without device specific quirks to > enable virtualization of these paths. We have one fgpa based PCIe slave where the device driver has to read the MSI-X table and then write the value to other fpga registers so that the logic can generate the correct PCIe write cycle when an interrupt is requested. The MSI-X table itself is only as a PCIe slave. We also have host accessible DMA controllers that the device driver uses to copy data to kernel memory. These could easily be used to generate arbitrary MSI-X requests. As I've said earlier it is almost certainly possible to get any ethernet hardware to perform something similar. So without hardware that is able to limit the memory and MSI-X that each PCIe endpoint can access I believe that if a virtualisation system gives a guest kernel direct access to a PCIe devices it gives the guest kernel the ability to raise and MSI-X interrupt and read/write any physical memory. (I've not looked at the cpu virtualisation support, but do know what the PCIe devices can do.) More interestingly, probably the 'worst' thing (from a security point of view) that changing the MSI-X table lets you do is a write to an arbitrary physical memory address. David From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out4.electric.net (smtp-out4.electric.net [192.162.216.192]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3r5kmB3vGSzDq7n for ; Fri, 13 May 2016 19:18:02 +1000 (AEST) From: David Laight To: 'Alex Williamson' , "Tian, Kevin" CC: Yongji Xie , "kvm@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-pci@vger.kernel.org" , "linuxppc-dev@lists.ozlabs.org" , "iommu@lists.linux-foundation.org" , "bhelgaas@google.com" , "aik@ozlabs.ru" , "benh@kernel.crashing.org" , "paulus@samba.org" , "mpe@ellerman.id.au" , "joro@8bytes.org" , "warrier@linux.vnet.ibm.com" , "zhong@linux.vnet.ibm.com" , "nikunj@linux.vnet.ibm.com" , "eric.auger@linaro.org" , "will.deacon@arm.com" , "gwshan@linux.vnet.ibm.com" , "alistair@popple.id.au" , "ruscur@russell.cc" Subject: RE: [PATCH 5/5] vfio-pci: Allow to mmap MSI-X table if interrupt remapping is supported Date: Fri, 13 May 2016 09:16:18 +0000 Message-ID: <063D6719AE5E284EB5DD2968C1650D6D5F4BBEDD@AcuExch.aculab.com> References: <1461761010-5452-1-git-send-email-xyjxie@linux.vnet.ibm.com> <063D6719AE5E284EB5DD2968C1650D6D5F4B52B5@AcuExch.aculab.com> <4be013bc-e81b-84c5-06d3-e1b3f46b3227@linux.vnet.ibm.com> <20160505090513.56886c12@t450s.home> <20160511095331.18436241@t450s.home> <20160511202042.77593861@t450s.home> <20160512114735.7ec61bd3@t450s.home> <20160512233246.347b8b3c@t450s.home> In-Reply-To: <20160512233246.347b8b3c@t450s.home> Content-Type: text/plain; charset="Windows-1252" MIME-Version: 1.0 List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Alex Williamson [mailto:alex.williamson@redhat.com] > Sent: 13 May 2016 06:33 ... > Simply denying direct writes to the vector table or preventing mapping > of the vector table into the user address space does not provide any > tangible form of protection. Many devices make use of window registers > that allow backdoors to arbitrary device registers. Some drivers even > use this as the primary means for configuring MSI-X, which makes them > incompatible with device assignment without device specific quirks to > enable virtualization of these paths. We have one fgpa based PCIe slave where the device driver has to read the MSI-X table and then write the value to other fpga registers so that the logic can generate the correct PCIe write cycle when an interrupt is requested. The MSI-X table itself is only as a PCIe slave. We also have host accessible DMA controllers that the device driver uses to copy data to kernel memory. These could easily be used to generate arbitrary MSI-X requests. As I've said earlier it is almost certainly possible to get any ethernet hardware to perform something similar. So without hardware that is able to limit the memory and MSI-X that each PCIe endpoint can access I believe that if a virtualisation system gives a guest kernel direct access to a PCIe devices it gives the guest kernel the ability to raise and MSI-X interrupt and read/write any physical memory. (I've not looked at the cpu virtualisation support, but do know what the PCIe devices can do.) More interestingly, probably the 'worst' thing (from a security point of vi= ew) that changing the MSI-X table lets you do is a write to an arbitrary physical memory address. David