[PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable without EXPERT

[PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable without EXPERT

[peng.fan]

From: Peng Fan <peng.fan@nxp.com>

commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32")
enables both ZONE_DMA and ZONE_DMA32. The lower 1GB memory
will be occupied by ZONE_DMA, this will cause CMA allocation
fail on some platforms, because CMA area could not across
different type of memory zones.

Make CONFIG_ZONE_DMA configurable without EXPERT option
could let people build non debug kernel image with
CONFIG_ZONE_DMA disabled.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 arch/arm64/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 217e12ff2115..c4ba8bf011e1 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -270,7 +270,7 @@ config GENERIC_CALIBRATE_DELAY
 	def_bool y
 
 config ZONE_DMA
-	bool "Support DMA zone" if EXPERT
+	bool "Support DMA zone"
 	default y
 
 config ZONE_DMA32
-- 
2.16.4

RE: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable without EXPERT

[Peng Fan]

> Subject: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable
> without EXPERT

Gentle ping..

Thanks,
Peng.

> 
> From: Peng Fan <peng.fan@nxp.com>
> 
> commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32")
> enables both ZONE_DMA and ZONE_DMA32. The lower 1GB memory will be
> occupied by ZONE_DMA, this will cause CMA allocation fail on some platforms,
> because CMA area could not across different type of memory zones.
> 
> Make CONFIG_ZONE_DMA configurable without EXPERT option could let
> people build non debug kernel image with CONFIG_ZONE_DMA disabled.
> 
> Signed-off-by: Peng Fan <peng.fan@nxp.com>
> ---
>  arch/arm64/Kconfig | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index
> 217e12ff2115..c4ba8bf011e1 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -270,7 +270,7 @@ config GENERIC_CALIBRATE_DELAY
>  	def_bool y
> 
>  config ZONE_DMA
> -	bool "Support DMA zone" if EXPERT
> +	bool "Support DMA zone"
>  	default y
> 
>  config ZONE_DMA32
> --
> 2.16.4

Re: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable without EXPERT

[Catalin Marinas]

On Tue, Mar 10, 2020 at 08:48:46PM +0800, peng.fan@nxp.com wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32")
> enables both ZONE_DMA and ZONE_DMA32. The lower 1GB memory
> will be occupied by ZONE_DMA, this will cause CMA allocation
> fail on some platforms, because CMA area could not across
> different type of memory zones.
> 
> Make CONFIG_ZONE_DMA configurable without EXPERT option
> could let people build non debug kernel image with
> CONFIG_ZONE_DMA disabled.

While I see why you need to toggle this feature, I'd rather try to
figure out whether there is a better solution that does not break the
single kernel image aim (i.e. the same config works for all supported
SoCs).

When we decided to go ahead with a static 1GB ZONE_DMA for Raspberry Pi
4, we thought that other platforms would be fine, ZONE_DMA32 allocations
fall back to ZONE_DMA. We missed the large CMA case.

I see a few potential options:

a) Ensure the CMA is contained within a single zone. How large is it in
   your case? Is it allocated by the kernel dynamically or a fixed start
   set by the boot loader?

b) Change the CMA allocator to allow spanning multiple zones (last time
   I looked it wasn't trivial since it relied on some per-zone lock).

c) Make ZONE_DMA dynamic on arm64 and only enable it if RPi4.

-- 
Catalin

RE: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable without EXPERT

[Peng Fan]

> Subject: Re: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable
> without EXPERT
> 
> On Tue, Mar 10, 2020 at 08:48:46PM +0800, peng.fan@nxp.com wrote:
> > From: Peng Fan <peng.fan@nxp.com>
> >
> > commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32")
> > enables both ZONE_DMA and ZONE_DMA32. The lower 1GB memory will
> be
> > occupied by ZONE_DMA, this will cause CMA allocation fail on some
> > platforms, because CMA area could not across different type of memory
> > zones.
> >
> > Make CONFIG_ZONE_DMA configurable without EXPERT option could let
> > people build non debug kernel image with CONFIG_ZONE_DMA disabled.
> 
> While I see why you need to toggle this feature, I'd rather try to figure out
> whether there is a better solution that does not break the single kernel image
> aim (i.e. the same config works for all supported SoCs).
> 
> When we decided to go ahead with a static 1GB ZONE_DMA for Raspberry Pi
> 4, we thought that other platforms would be fine, ZONE_DMA32 allocations
> fall back to ZONE_DMA. We missed the large CMA case.
> 
> I see a few potential options:
> 
> a) Ensure the CMA is contained within a single zone. 

This will break legacy dts with new version kernel.

How large is it in
>    your case? 

It is 1GB

Is it allocated by the kernel dynamically or a fixed start
>    set by the boot loader?

We use alloc-ranges and size in kernel dts.

But there is only 2GB DRAM in the board.

> 
> b) Change the CMA allocator to allow spanning multiple zones (last time
>    I looked it wasn't trivial since it relied on some per-zone lock).
> 
> c) Make ZONE_DMA dynamic on arm64 and only enable it if RPi4.

Option c seems a bit easier to me :)

I will try to explore both, but if you have time to help, that would be
appreciated.

Thanks,
Peng.
> 
> --
> Catalin

Re: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable without EXPERT

[Catalin Marinas]

On Wed, Mar 25, 2020 at 12:34:15AM +0000, Peng Fan wrote:
> > On Tue, Mar 10, 2020 at 08:48:46PM +0800, peng.fan@nxp.com wrote:
> > > From: Peng Fan <peng.fan@nxp.com>
> > >
> > > commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32")
> > > enables both ZONE_DMA and ZONE_DMA32. The lower 1GB memory will be
> > > occupied by ZONE_DMA, this will cause CMA allocation fail on some
> > > platforms, because CMA area could not across different type of memory
> > > zones.
> > >
> > > Make CONFIG_ZONE_DMA configurable without EXPERT option could let
> > > people build non debug kernel image with CONFIG_ZONE_DMA disabled.
> > 
> > While I see why you need to toggle this feature, I'd rather try to figure out
> > whether there is a better solution that does not break the single kernel image
> > aim (i.e. the same config works for all supported SoCs).
> > 
> > When we decided to go ahead with a static 1GB ZONE_DMA for Raspberry Pi
> > 4, we thought that other platforms would be fine, ZONE_DMA32 allocations
> > fall back to ZONE_DMA. We missed the large CMA case.
> > 
> > I see a few potential options:
> > 
> > a) Ensure the CMA is contained within a single zone. 
> 
> This will break legacy dts with new version kernel.
> 
> > How large is it in your case? 
> 
> It is 1GB
> 
> > Is it allocated by the kernel dynamically or a fixed start set by
> > the boot loader?
> 
> We use alloc-ranges and size in kernel dts.
> 
> But there is only 2GB DRAM in the board.

So I guess without changing the dts, option (a) doesn't really work.

> > b) Change the CMA allocator to allow spanning multiple zones (last time
> >    I looked it wasn't trivial since it relied on some per-zone lock).
> > 
> > c) Make ZONE_DMA dynamic on arm64 and only enable it if RPi4.
> 
> Option c seems a bit easier to me :)
> 
> I will try to explore both, but if you have time to help, that would be
> appreciated.

I don't have time but option (c) was already discussed and there are
patches from Nicolas on the list:

https://lore.kernel.org/linux-arm-kernel/20190820145821.27214-5-nsaenzjulienne@suse.de/

The above series was checking whether the platform is RPi4 and limiting
the ZONE_DMA size to 1GB (otherwise 4GB with ZONE_DMA32 empty). We ended
up with a static 1GB for ZONE_DMA but we missed the fact that it may
break existing platforms.

So I don't think it would be too hard to revive the above series (most
of it was already merged).

-- 
Catalin

RE: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable without EXPERT

[Peng Fan]

> Subject: Re: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable
> without EXPERT
> 
> On Wed, Mar 25, 2020 at 12:34:15AM +0000, Peng Fan wrote:
> > > On Tue, Mar 10, 2020 at 08:48:46PM +0800, peng.fan@nxp.com wrote:
> > > > From: Peng Fan <peng.fan@nxp.com>
> > > >
> > > > commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and
> ZONE_DMA32")
> > > > enables both ZONE_DMA and ZONE_DMA32. The lower 1GB memory
> will be
> > > > occupied by ZONE_DMA, this will cause CMA allocation fail on some
> > > > platforms, because CMA area could not across different type of
> > > > memory zones.
> > > >
> > > > Make CONFIG_ZONE_DMA configurable without EXPERT option could
> let
> > > > people build non debug kernel image with CONFIG_ZONE_DMA
> disabled.
> > >
> > > While I see why you need to toggle this feature, I'd rather try to
> > > figure out whether there is a better solution that does not break
> > > the single kernel image aim (i.e. the same config works for all supported
> SoCs).
> > >
> > > When we decided to go ahead with a static 1GB ZONE_DMA for
> Raspberry
> > > Pi 4, we thought that other platforms would be fine, ZONE_DMA32
> > > allocations fall back to ZONE_DMA. We missed the large CMA case.
> > >
> > > I see a few potential options:
> > >
> > > a) Ensure the CMA is contained within a single zone.
> >
> > This will break legacy dts with new version kernel.
> >
> > > How large is it in your case?
> >
> > It is 1GB
> >
> > > Is it allocated by the kernel dynamically or a fixed start set by
> > > the boot loader?
> >
> > We use alloc-ranges and size in kernel dts.
> >
> > But there is only 2GB DRAM in the board.
> 
> So I guess without changing the dts, option (a) doesn't really work.
> 
> > > b) Change the CMA allocator to allow spanning multiple zones (last time
> > >    I looked it wasn't trivial since it relied on some per-zone lock).
> > >
> > > c) Make ZONE_DMA dynamic on arm64 and only enable it if RPi4.
> >
> > Option c seems a bit easier to me :)
> >
> > I will try to explore both, but if you have time to help, that would
> > be appreciated.
> 
> I don't have time but option (c) was already discussed and there are patches
> from Nicolas on the list:
> 
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.ke
> rnel.org%2Flinux-arm-kernel%2F20190820145821.27214-5-nsaenzjulienne%
> 40suse.de%2F&amp;data=02%7C01%7Cpeng.fan%40nxp.com%7C6403ddf37
> 89b452ae5ee08d7d0a5a659%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0
> %7C0%7C637207282191026738&amp;sdata=t2cZ9HTCcRuaL9RO4kD%2BzN
> 2n4VqM%2F66zYNZIOComCVs%3D&amp;reserved=0
> 
> The above series was checking whether the platform is RPi4 and limiting the
> ZONE_DMA size to 1GB (otherwise 4GB with ZONE_DMA32 empty). We
> ended up with a static 1GB for ZONE_DMA but we missed the fact that it may
> break existing platforms.


Thanks for the information. I'll check the patchset and work out something
proper to fix the issue I met.

Thanks,
Peng.

> 
> So I don't think it would be too hard to revive the above series (most of it was
> already merged).
> 
> --
> Catalin

Re: [PATCH] arm64: mm: make CONFIG_ZONE_DMA configurable without EXPERT

[Nicolas Saenz Julienne]

[-- Attachment #1: Type: text/plain, Size: 2414 bytes --]

Hi Peng,

On Wed, 2020-03-25 at 12:30 +0000, Peng Fan wrote:
> > > > I see a few potential options:
> > > > 
> > > > a) Ensure the CMA is contained within a single zone.
> > > 
> > > This will break legacy dts with new version kernel.
> > > 
> > > > How large is it in your case?
> > > 
> > > It is 1GB
> > > 
> > > > Is it allocated by the kernel dynamically or a fixed start set by
> > > > the boot loader?
> > > 
> > > We use alloc-ranges and size in kernel dts.
> > > 
> > > But there is only 2GB DRAM in the board.
> > 
> > So I guess without changing the dts, option (a) doesn't really work.
> > 
> > > > b) Change the CMA allocator to allow spanning multiple zones (last time
> > > >    I looked it wasn't trivial since it relied on some per-zone lock).

I like this as a solution, ultimately why should CMA be linked to a specific
zone. Also, crossing the ZONE_DMA/ZONE_DMA32 boundary shouldn't be an issue
since we already default to ZONE_DMA32 as the default area for CMA.

That said, easier said than done.

> > > > c) Make ZONE_DMA dynamic on arm64 and only enable it if RPi4.
> > > 
> > > Option c seems a bit easier to me :)
> > > 
> > > I will try to explore both, but if you have time to help, that would
> > > be appreciated.
> > 
> > I don't have time but option (c) was already discussed and there are patches
> > from Nicolas on the list:
> > 
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.ke
> > rnel.org%2Flinux-arm-kernel%2F20190820145821.27214-5-nsaenzjulienne%
> > 40suse.de%2F&data=02%7C01%7Cpeng.fan%40nxp.com%7C6403ddf37
> > 89b452ae5ee08d7d0a5a659%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0
> > %7C0%7C637207282191026738&sdata=t2cZ9HTCcRuaL9RO4kD%2BzN
> > 2n4VqM%2F66zYNZIOComCVs%3D&reserved=0
> > 
> > The above series was checking whether the platform is RPi4 and limiting the
> > ZONE_DMA size to 1GB (otherwise 4GB with ZONE_DMA32 empty). We
> > ended up with a static 1GB for ZONE_DMA but we missed the fact that it may
> > break existing platforms.
> 
> Thanks for the information. I'll check the patchset and work out something
> proper to fix the issue I met.

Just an FYI, I'm follwing the conversation, and will try to look into as soon
as I get some free time. In the meantime if you want me to test/review anything
on the Raspberry Pi, I'll be happy to.

Regards,
Nicolas


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]