Re: [PATCH 00/11] of: Fix DMA configuration for non-DT masters

[Robin Murphy <robin.murphy@arm.com>]

On 25/09/2019 17:16, Rob Herring wrote:
> On Wed, Sep 25, 2019 at 10:30 AM Nicolas Saenz Julienne
> <nsaenzjulienne@suse.de> wrote:
>>
>> On Wed, 2019-09-25 at 16:09 +0100, Robin Murphy wrote:
>>> On 25/09/2019 15:52, Nicolas Saenz Julienne wrote:
>>>> On Tue, 2019-09-24 at 16:59 -0500, Rob Herring wrote:
>>>>> On Tue, Sep 24, 2019 at 1:12 PM Nicolas Saenz Julienne
>>>>> <nsaenzjulienne@suse.de> wrote:
>>>>>> Hi All,
>>>>>> this series tries to address one of the issues blocking us from
>>>>>> upstreaming Broadcom's STB PCIe controller[1]. Namely, the fact that
>>>>>> devices not represented in DT which sit behind a PCI bus fail to get the
>>>>>> bus' DMA addressing constraints.
>>>>>>
>>>>>> This is due to the fact that of_dma_configure() assumes it's receiving a
>>>>>> DT node representing the device being configured, as opposed to the PCIe
>>>>>> bridge node we currently pass. This causes the code to directly jump
>>>>>> into PCI's parent node when checking for 'dma-ranges' and misses
>>>>>> whatever was set there.
>>>>>>
>>>>>> To address this I create a new API in OF - inspired from Robin Murphys
>>>>>> original proposal[2] - which accepts a bus DT node as it's input in
>>>>>> order to configure a device's DMA constraints. The changes go deep into
>>>>>> of/address.c's implementation, as a device being having a DT node
>>>>>> assumption was pretty strong.
>>>>>>
>>>>>> On top of this work, I also cleaned up of_dma_configure() removing its
>>>>>> redundant arguments and creating an alternative function for the special
>>>>>> cases
>>>>>> not applicable to either the above case or the default usage.
>>>>>>
>>>>>> IMO the resulting functions are more explicit. They will probably
>>>>>> surface some hacky usages that can be properly fixed as I show with the
>>>>>> DT fixes on the Layerscape platform.
>>>>>>
>>>>>> This was also tested on a Raspberry Pi 4 with a custom PCIe driver and
>>>>>> on a Seattle AMD board.
>>>>>
>>>>> Humm, I've been working on this issue too. Looks similar though yours
>>>>> has a lot more churn and there's some other bugs I've found.
>>>>
>>>> That's good news, and yes now that I see it, some stuff on my series is
>>>> overly
>>>> complicated. Specially around of_translate_*().
>>>>
>>>> On top of that, you removed in of_dma_get_range():
>>>>
>>>> -   /*
>>>> -    * At least empty ranges has to be defined for parent node if
>>>> -    * DMA is supported
>>>> -    */
>>>> -   if (!ranges)
>>>> -           break;
>>>>
>>>> Which I assumed was bound to the standard and makes things easier.
>>>>
>>>>> Can you test out this branch[1]. I don't have any h/w needing this,
>>>>> but wrote a unittest and tested with modified QEMU.
>>>>
>>>> I reviewed everything, I did find a minor issue, see the patch attached.
>>>
>>> WRT that patch, the original intent of "force_dma" was purely to
>>> consider a device DMA-capable regardless of the presence of
>>> "dma-ranges". Expecting of_dma_configure() to do anything for a non-OF
>>> device has always been bogus - magic paravirt devices which appear out
>>> of nowhere and expect to be treated as genuine DMA masters are a
>>> separate problem that we haven't really approached yet.
>>
>> I agree it's clearly abusing the function. I have no problem with the behaviour
>> change if it's OK with you.

Thinking about it, you could probably just remove that call from the Xen 
DRM driver now anyway - since the dma-direct rework, we lost the ability 
to set dma_dummy_ops by default, and NULL ops now represent what it 
(presumably) wants.

>> Robin, have you looked into supporting multiple dma-ranges? It's the next thing
>> we need for BCM STB's PCIe. I'll have a go at it myself if nothing is in the
>> works already.
> 
> Multiple dma-ranges as far as configuring inbound windows should work
> already other than the bug when there's any parent translation. But if
> you mean supporting multiple DMA offsets and masks per device in the
> DMA API, there's nothing in the works yet.

There's also the in-between step of making of_dma_get_range() return a 
size based on all the dma-ranges entries rather than only the first one 
- otherwise, something like [1] can lead to pretty unworkable default 
masks. We implemented that when doing acpi_dma_get_range(), it's just 
that the OF counterpart never caught up.

Robin.

[1] 
http://linux-arm.org/git?p=linux-rm.git;a=commitdiff;h=a2814af56b3486c2985a95540a88d8f9fa3a699f