Re: Was: Re: [RFC PATCH] fix problems with NETIF_F_HIGHDMA in networking, Now: SWIOTLB dynamic allocation

[FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>]

On Mon, 1 Mar 2010 15:12:41 -0600
Robert Hancock <hancockrwd@gmail.com> wrote:

> On Mon, Mar 1, 2010 at 10:34 AM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> > On Sun, Feb 28, 2010 at 12:16:28AM -0800, David Miller wrote:
> >> From: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
> >> Date: Sun, 28 Feb 2010 03:38:19 +0900
> >>
> >> > When I proposed such approach (always use swiotlb) before, IIRC,
> >> > the objections were:
> >> >
> >> > - better to make allocation respect dma_mask. (I don't think that this
> >> >   approach is possible since we don't know which device handles data
> >> >   later when we allocate memory).
> >>
> >> And such objects might end up being processed by multiple devices with
> >> different DMA restrictions.
> >>
> >> > - swiotlb is not good for small systems since it allocates too much
> >> >   memory (we can fix this though).
> >>
> >> Indeed.
> >
> > What would be a good mechanism for this? Enumerating all of the PCI
> > devices to find out which ones are 32-bit and then allocate some chunk
> > of memory based on the amount of them? say, 1MB per card?
> >
> > Or maybe a simpler one - figure out how many pages we have an allocate
> > based on some sliding rule (say, 8MB for under 512MB, 16MB between 512MB
> > and 2GB, and 32MB for 2GB to 4GB, and after that the full 64MB?)
> 
> Why do we need to allocate SWIOTLB if your highest memory address is
> under 4GB? You can just disable it in that case, like x86_64 does.

Unfortunately, we need to do because the goal is removing the block
layer bounce (and other bounce mechanisms in sub systems).

For example, even if highest memory address is under 4GB, you must
provide bounce buffer for ISA devices (and some devices for strange
DMA restrictions such as under 1GB).