Re: Was: Re: [RFC PATCH] fix problems with NETIF_F_HIGHDMA in networking, Now: SWIOTLB dynamic allocation

[FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>]

On Mon, 1 Mar 2010 11:34:37 -0500
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> On Sun, Feb 28, 2010 at 12:16:28AM -0800, David Miller wrote:
> > From: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
> > Date: Sun, 28 Feb 2010 03:38:19 +0900
> > 
> > > When I proposed such approach (always use swiotlb) before, IIRC,
> > > the objections were:
> > > 
> > > - better to make allocation respect dma_mask. (I don't think that this
> > >   approach is possible since we don't know which device handles data
> > >   later when we allocate memory).
> > 
> > And such objects might end up being processed by multiple devices with
> > different DMA restrictions.
> > 
> > > - swiotlb is not good for small systems since it allocates too much
> > >   memory (we can fix this though).
> > 
> > Indeed.
> 
> What would be a good mechanism for this? Enumerating all of the PCI
> devices to find out which ones are 32-bit and then allocate some chunk
> of memory based on the amount of them? say, 1MB per card?
> 
> Or maybe a simpler one - figure out how many pages we have an allocate
> based on some sliding rule (say, 8MB for under 512MB, 16MB between 512MB
> and 2GB, and 32MB for 2GB to 4GB, and after that the full 64MB?)

Hmm, have you read the above objection from embedded system people ? :)

We can't pre-allocate such lots of memory (several MB). And we don't
need to.

We have to pre-allocate some for the block layer (we have to guarantee
that we can handle one request at least), however we don't need to
pre-allocate for the rest (including the network stack).

This guarantee is a bit difficult since the dma_map_* doesn't know who
is at the upper layer; the dma_map_* cannot know if it needs to
allocate from the pre-allocated pool or not. I thought about adding
'dma attribute flag' to struct device (to be exact, struct
device_dma_parameters).