Re: [XEN RFC PATCH 04/40] xen/arm: return default DMA bit width when platform is not set

[Julien Grall <julien@xen.org>]

On 20/08/2021 03:04, Wei Chen wrote:
> Hi Julien,

Hi Wei,

>> -----Original Message-----
>> From: Julien Grall <julien@xen.org>
>> Sent: 2021年8月19日 21:28
>> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org;
>> sstabellini@kernel.org; jbeulich@suse.com
>> Cc: Bertrand Marquis <Bertrand.Marquis@arm.com>
>> Subject: Re: [XEN RFC PATCH 04/40] xen/arm: return default DMA bit width
>> when platform is not set
>>
>> Hi,
>>
>> On 11/08/2021 11:23, Wei Chen wrote:
>>> From: Hongda Deng <Hongda.Deng@arm.com>
>>>
>>> In current code, arch_get_dma_bitsize will return 32 when platorm
>>> or platform->dma_bitsize is not set. It's not resonable, for Arm,
>>
>> s/resonable/reasonable/
>>
> 
> Ok
> 
>>> we don't require to reserve DMA memory. So we set dma_bitsize always
>>> be 0. In NO-NUMA system, arch_get_dma_bitsize will not be invoked,
>>> so dma_bitsize will not be overrided by this function.
>>
>> arch_get_dma_bitsize() is also used to allocate dom0 memory. We need to
>> be able to allocate some DMA-able memory that can be used by every devices.
>>
>>> But in NUMA
>>> system, once the online nodes are greater than 1, this function will
>>> be invoked. The dma_bitsize will be limited to 32. That means, only
>>> first 4GB memory can be used for DMA. But that's against our hardware
>>> design. We don't have that kind of restriction on hardware.
>>
>> What do you mean by "hardware design"? Are you referring to the server
>> you boot Xen on?
>>
> 
> Yes. I will change it to some neutral words. something like:
> "But that could not reflect some hardware's real DMA ability. They may not
> have kind of restriction on hardware." ?

The thing is DMA ability is not about the platform itself. It is more 
about the devices (this could just be a PCI card you just plugged). What 
you seem to suggest is no-one will ever plug such card on your platform. 
Is that correct?

> 
> 
>> Anyway, there are plenty of platform out that have devices which can't
>> DMA into memory above 32-bit. On RPI, this is even lower (30-bit).
>>
>> So I would be cautious to change the default limit.
>>
> 
> How about return 0 when platform doesn't specify the limit?
> In my opinion, arbitrary to give 32 on AArch64 doesn't make sense.
We have to care about the common use-case. We added this restriction a 
few years ago because we had a few reports of users using only 32-bit 
DMA capable device.

> But as you mentioned, if Xen is running on a platform with DMA limitation,
> but we have not listed this platform in Xen supported list, Xen cannot
> get DMA limit from platform->dma_bitsize. In this case, return 0 will
> also cause some issue.
> 
>> At the moment, the only place on Arm where we need DMA-able memory is
>> for dom0. This is allocated at boot and can't change afterwards (for now).
>>
> 
> For Dom0, we squash the patch#5 into this patch?

Let me answer with another question. Why should we modify the Arm code 
rather than the common code? IOW...

> 
>> So I would explore to remove the NUMA check for drop the DMA zone. FAOD,
>> both suggestion are for Arm only. For x86, they need to be kept.
>>
> 
> Without introducing new flag, such as lowmem_for_dma, it's a little
> hard to skip the numa node check. Unless we crudely add #ifdef ARCH to
> common code, which is not what we want to see ...
>        if ( !dma_bitsize && (num_online_nodes() > 1) )
>            dma_bitsize = arch_get_dma_bitsize();

... Why do you think we need this check on Arm when NUMA is enabled?

We can discuss how to remove it once this is answered.

Cheers,

-- 
Julien Grall