Re: [PATCH 1/2] dma-mapping: add a dma_ops_bypass flag to struct device

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 23/03/2020 19:37, Christoph Hellwig wrote:
> On Mon, Mar 23, 2020 at 12:28:34PM +1100, Alexey Kardashevskiy wrote:
> 
> [full quote deleted, please follow proper quoting rules]
> 
>>> +static bool dma_alloc_direct(struct device *dev, const struct dma_map_ops *ops)
>>> +{
>>> +	if (!ops)
>>> +		return true;
>>> +
>>> +	/*
>>> +	 * Allows IOMMU drivers to bypass dynamic translations if the DMA mask
>>> +	 * is large enough.
>>> +	 */
>>> +	if (dev->dma_ops_bypass) {
>>> +		if (min_not_zero(dev->coherent_dma_mask, dev->bus_dma_limit) >=
>>> +				dma_direct_get_required_mask(dev))
>>> +			return true;
>>> +	}
>>
>>
>> Why not do this in dma_map_direct() as well?
> 
> Mostly beacuse it is a relatively expensive operation, including a
> fls64.

Ah, ok.

>> Or simply have just one dma_map_direct()?
> 
> What do you mean with that?

I mean use dma_alloc_direct() instead of dma_map_direct() everywhere,
you explained just above.

> 
>> And one more general question - we need a way to use non-direct IOMMU
>> for RAM above certain limit.
>>
>> Let's say we have a system with:
>> 0 .. 0x1.0000.0000
>> 0x100.0000.0000 .. 0x101.0000.0000
>>
>> 2x4G, each is 1TB aligned. And we can map directly only the first 4GB
>> (because of the maximum IOMMU table size) but not the other. And 1:1 on
>> that "pseries" is done with offset=0x0800.0000.0000.0000.
>>
>> So we want to check every bus address against dev->bus_dma_limit, not
>> dev->coherent_dma_mask. In the example above I'd set bus_dma_limit to
>> 0x0800.0001.0000.0000 and 1:1 mapping for the second 4GB would not be
>> tried. Does this sound reasonable? Thanks,
> 
> bus_dma_limit is just another limiting factor applied on top of
> coherent_dma_mask or dma_mask respectively.

This is not enough for the task: in my example, I'd set bus limit to
0x0800.0001.0000.0000 but this would disable bypass for all RAM
addresses - the first and the second 4GB blocks.


-- 
Alexey