Re: [alsa-devel] USB transfer_buffer allocations on 64bit systems

[Alan Stern <stern@rowland.harvard.edu>]

On Tue, 11 May 2010, FUJITA Tomonori wrote:

> > > > > At one point we tried an experiment, printing out the buffer and DMA 
> > > > > addresses.  I don't recall seeing anything obviously wrong, but if an 
> > > > > IOMMU was in use then that might not mean anything.  Is it possible 
> > > > > that the IOMMU mappings sometimes get messed up for addresses above 4 
> > > > > GB?
> > > > 
> > > > You mean that an IOMMU could allocate an address above 4GB wrongly? If
> > > > so, IIRC, all the IOMMU implementations use dev->dma_mask and
> > > > dev->coherent_dma_mask properly. And the DMA address space of the
> > > > majority of IOMMUs are limited less than 4GB.
> > > 
> > > The Intel IOMMU code will use dev->dma_mask and dev->coherent_dma_mask
> > > properly. It is not limited to 4GiB, but it will tend to give virtual
> > > DMA addresses below 4GiB even when a device is capable of more; it'll
> > > only give out higher addresses when the address space below 4GiB is
> > > exhausted.
> > 
> > What I meant was: Is it possible that the IOMMU code will return a
> > virtual DMA address before 4 GB but will somehow forget to actually map
> > that address to the data buffer?
> 
> Then, the IOMMU is completely broken. Then, we would get tons of DMA
> bugs not only about USB, I guess. So I'm not sure.

Yes, you're right about that.

> > The problem goes away when Pedro boots with mem=4G.  And the dma_mask
> > value is set properly (in fact, the ehci-hcd driver currently doesn't
> > use 64-bit DMA at all).
> > 
> > If anyone wants to see the debug log entries showing the buffer and DMA
> > addresses, they are attached to this email message:
> > 
> > 	http://marc.info/?l=linux-kernel&m=127076841801054&w=2
> > 
> > Either the data isn't getting written to the buffer correctly or else
> > the buffer isn't getting sent to the device correctly.  Can anybody
> > suggest a means of determining which is the case?
> 
> I can't say anything about this log that including only DMA addresses.
> I'm not familiar with how the USB core does DMA stuff. And the USB
> stack design that the USB core does DMA stuff (allocating, mappings,
> etc) makes debugging DMA issues really difficult.

The DMA stuff is simple enough in this case.  The urb->transfer_buffer
address is passed to dma_map_single(), and the DMA address it returns
is stored in urb->transfer_dma.  Those are the two values printed out
by the debugging patch.

Alan Stern