From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops domU kernel with PCI passthrough Date: Fri, 12 Nov 2010 10:56:59 -0500 Message-ID: <20101112155659.GA5529@dumpdata.com> References: <20101111190351.GB15530@dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Lin, Ray" Cc: Xen-devel , Dante Cinco List-Id: xen-devel@lists.xenproject.org On Thu, Nov 11, 2010 at 12:42:03PM -0700, Lin, Ray wrote: > > Konrad, > > See my response in red. Please don't top post. > > > -Ray > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Konrad Rzeszutek Wilk > Sent: Thursday, November 11, 2010 11:04 AM > To: Dante Cinco > Cc: Xen-devel > Subject: Re: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops domU kernel with PCI passthrough > > On Thu, Nov 11, 2010 at 10:31:48AM -0800, Dante Cinco wrote: > > Konrad, > > > > Without swiotlb=force, I don't see "PCI-DMA: Using software bounce > > buffering for IO" in /var/log/kern.log. > > > > With iommu=soft and without swiotlb=force, I see the "software bounce > > buffering" in /var/log/kern.log and an NMI (see below) when I load the > > kernel module drivers. I made sure the NMI is reproducible and not a > > What is the kernel module doing to cause this? DMA? ??? What did it do? > > one-time event. > > So doing 64-bit DMA causes an NMI. Do you have the Hypervisor's IOMMU VT-d enabled or disabled? (iommu=off,verbose) If you turn it off does this work? > > We have IOMMU VT-d enabled. If we turn it off (iommu=off,verbose), the DMA doesn't work properly and the driver code is unable to detect the source of interrupt. The interrupts of our device would be disabled by kernel eventually due to nobody services the interrupts for more than 100000 times. That does not sound right. You should be able to use the PCI passthrough without the IOMMU. Since it is an interrupt issue it sounds like that you are using x2APIC and that is enabled without the IOMMU. Had you tried disabling IOMMU and x2apic? (this is all on the hypervisor line?)