Running/building an 32 bits kernel on A53 CPU?

Running/building an 32 bits kernel on A53 CPU?

[Joakim Tjernlund]

In order to reduce RAM usage we are wondering if running an 32 bits kernel on ARM A53 CPU is a good idea?

Is it possible to build such a kernel? If so, how to go about it?
Should one use ARCH=arm and a 32 bits toolchain or
ARCH=arm64 and 64 bits toolchain and adding 32 bits compile options?

 Jocke 
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Running/building an 32 bits kernel on A53 CPU?

[Catalin Marinas]

On Fri, Oct 29, 2021 at 08:27:35AM +0000, Joakim Tjernlund wrote:
> In order to reduce RAM usage we are wondering if running an 32 bits
> kernel on ARM A53 CPU is a good idea?

No. Really.

What's the RAM size on your SoC?

> Is it possible to build such a kernel? If so, how to go about it?
> Should one use ARCH=arm and a 32 bits toolchain or
> ARCH=arm64 and 64 bits toolchain and adding 32 bits compile options?

If you want a 32-bit kernel, you'd have to use ARCH=arm and a 32-bit
toolchain. This would give you an ARMv7 kernel that can run on ARMv8
hardware. But you'd miss errata workarounds for example and any other
features that come with ARMv8.

I don't recommend this at all. If you have some specific embedded
system, you can disable kernel features you don't need to make it
smaller. Also note that you can still run 32-bit applications with a
64-bit kernel.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Running/building an 32 bits kernel on A53 CPU?

[Joakim Tjernlund]

On Fri, 2021-10-29 at 11:09 +0100, Catalin Marinas wrote:
> On Fri, Oct 29, 2021 at 08:27:35AM +0000, Joakim Tjernlund wrote:
> > In order to reduce RAM usage we are wondering if running an 32 bits
> > kernel on ARM A53 CPU is a good idea?
> 
> No. Really.
> 
> What's the RAM size on your SoC?

32MB + 4MB SRAM

> 
> > Is it possible to build such a kernel? If so, how to go about it?
> > Should one use ARCH=arm and a 32 bits toolchain or
> > ARCH=arm64 and 64 bits toolchain and adding 32 bits compile options?
> 
> If you want a 32-bit kernel, you'd have to use ARCH=arm and a 32-bit
> toolchain. This would give you an ARMv7 kernel that can run on ARMv8
> hardware. But you'd miss errata workarounds for example and any other
> features that come with ARMv8.
> 
> I don't recommend this at all. If you have some specific embedded
> system, you can disable kernel features you don't need to make it
> smaller. Also note that you can still run 32-bit applications with a
> 64-bit kernel.
> 
I have built the smallest kernel I can(so I think ATM), disabling all the features we
don't need. We do need IPv4/IPv6 though so that builds size.

ATM the my memory consumption is(without starting any user space):
  Memory: 23924K/32768K available (3584K kernel code, 688K rwdata, 568K rodata, 320K init, 244K bss, 8844K reserved, 0K cma-reserved)

The reserved size is about twice the size of static kernel use, any way to lower the dynamic part?
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Running/building an 32 bits kernel on A53 CPU?

[Robin Murphy]

On 2021-10-29 11:32, Joakim Tjernlund wrote:
> On Fri, 2021-10-29 at 11:09 +0100, Catalin Marinas wrote:
>> On Fri, Oct 29, 2021 at 08:27:35AM +0000, Joakim Tjernlund wrote:
>>> In order to reduce RAM usage we are wondering if running an 32 bits
>>> kernel on ARM A53 CPU is a good idea?
>>
>> No. Really.
>>
>> What's the RAM size on your SoC?
> 
> 32MB + 4MB SRAM
> 
>>
>>> Is it possible to build such a kernel? If so, how to go about it?
>>> Should one use ARCH=arm and a 32 bits toolchain or
>>> ARCH=arm64 and 64 bits toolchain and adding 32 bits compile options?
>>
>> If you want a 32-bit kernel, you'd have to use ARCH=arm and a 32-bit
>> toolchain. This would give you an ARMv7 kernel that can run on ARMv8
>> hardware. But you'd miss errata workarounds for example and any other
>> features that come with ARMv8.
>>
>> I don't recommend this at all. If you have some specific embedded
>> system, you can disable kernel features you don't need to make it
>> smaller. Also note that you can still run 32-bit applications with a
>> 64-bit kernel.
>>
> I have built the smallest kernel I can(so I think ATM), disabling all the features we
> don't need. We do need IPv4/IPv6 though so that builds size.
> 
> ATM the my memory consumption is(without starting any user space):
>    Memory: 23924K/32768K available (3584K kernel code, 688K rwdata, 568K rodata, 320K init, 244K bss, 8844K reserved, 0K cma-reserved)
> 
> The reserved size is about twice the size of static kernel use, any way to lower the dynamic part?

FWIW, if you haven't already I'd certainly reduce CONFIG_NR_CPUS to the 
minimum you need, as oversized percpu structures can add up. I'd also 
try hacking ARCH_DMA_MINALIGN down to 64 - provided your SoC doesn't 
have a system cache with larger lines - as that should save a bit of 
wastage on smaller allocations.

Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Running/building an 32 bits kernel on A53 CPU?

[Joakim Tjernlund]

On Fri, 2021-10-29 at 14:00 +0100, Robin Murphy wrote:
> On 2021-10-29 11:32, Joakim Tjernlund wrote:
> > On Fri, 2021-10-29 at 11:09 +0100, Catalin Marinas wrote:
> > > On Fri, Oct 29, 2021 at 08:27:35AM +0000, Joakim Tjernlund wrote:
> > > > In order to reduce RAM usage we are wondering if running an 32 bits
> > > > kernel on ARM A53 CPU is a good idea?
> > > 
> > > No. Really.
> > > 
> > > What's the RAM size on your SoC?
> > 
> > 32MB + 4MB SRAM
> > 
> > > 
> > > > Is it possible to build such a kernel? If so, how to go about it?
> > > > Should one use ARCH=arm and a 32 bits toolchain or
> > > > ARCH=arm64 and 64 bits toolchain and adding 32 bits compile options?
> > > 
> > > If you want a 32-bit kernel, you'd have to use ARCH=arm and a 32-bit
> > > toolchain. This would give you an ARMv7 kernel that can run on ARMv8
> > > hardware. But you'd miss errata workarounds for example and any other
> > > features that come with ARMv8.
> > > 
> > > I don't recommend this at all. If you have some specific embedded
> > > system, you can disable kernel features you don't need to make it
> > > smaller. Also note that you can still run 32-bit applications with a
> > > 64-bit kernel.
> > > 
> > I have built the smallest kernel I can(so I think ATM), disabling all the features we
> > don't need. We do need IPv4/IPv6 though so that builds size.
> > 
> > ATM the my memory consumption is(without starting any user space):
> >    Memory: 23924K/32768K available (3584K kernel code, 688K rwdata, 568K rodata, 320K init, 244K bss, 8844K reserved, 0K cma-reserved)
> > 
> > The reserved size is about twice the size of static kernel use, any way to lower the dynamic part?
> 
> FWIW, if you haven't already I'd certainly reduce CONFIG_NR_CPUS to the 
> minimum you need, as oversized percpu structures can add up. I'd also 
> try hacking ARCH_DMA_MINALIGN down to 64 - provided your SoC doesn't 
> have a system cache with larger lines - as that should save a bit of 
> wastage on smaller allocations.
> 

Right, I already had CONFIG_NR_CPUS=2 (minimum allowed, why is SMP forced? I got only one core ATM).
The ARCH_DMA_MINALIGN didn't help Reserved but I guess it will help runtime later, thanks

Memory: 23924K/32768K available (3584K kernel code, 682K rwdata, 560K rodata, 320K init, 242K bss, 8844K reserved, 0K cma-reserved)

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Running/building an 32 bits kernel on A53 CPU?

[Joakim Tjernlund]

On Fri, 2021-10-29 at 10:32 +0000, Joakim Tjernlund wrote:
> On Fri, 2021-10-29 at 11:09 +0100, Catalin Marinas wrote:
> > On Fri, Oct 29, 2021 at 08:27:35AM +0000, Joakim Tjernlund wrote:
> > > In order to reduce RAM usage we are wondering if running an 32 bits
> > > kernel on ARM A53 CPU is a good idea?
> > 
> > No. Really.
> > 
> > What's the RAM size on your SoC?
> 
> 32MB + 4MB SRAM
> 
> > 
> > > Is it possible to build such a kernel? If so, how to go about it?
> > > Should one use ARCH=arm and a 32 bits toolchain or
> > > ARCH=arm64 and 64 bits toolchain and adding 32 bits compile options?
> > 
> > If you want a 32-bit kernel, you'd have to use ARCH=arm and a 32-bit
> > toolchain. This would give you an ARMv7 kernel that can run on ARMv8
> > hardware. But you'd miss errata workarounds for example and any other
> > features that come with ARMv8.

Anything else you can think of why 32 bits kernel is a bad idea?
I do wonder why 64 bits A53 enforces SMP? I only have one core in my HW(custom ASIC)

   Jocke
> 
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Running/building an 32 bits kernel on A53 CPU?

[Catalin Marinas]

On Tue, Nov 02, 2021 at 01:20:22PM +0000, Joakim Tjernlund wrote:
> On Fri, 2021-10-29 at 10:32 +0000, Joakim Tjernlund wrote:
> > On Fri, 2021-10-29 at 11:09 +0100, Catalin Marinas wrote:
> > > On Fri, Oct 29, 2021 at 08:27:35AM +0000, Joakim Tjernlund wrote:
> > > > In order to reduce RAM usage we are wondering if running an 32 bits
> > > > kernel on ARM A53 CPU is a good idea?
> > > 
> > > No. Really.
> > > 
> > > What's the RAM size on your SoC?
> > 
> > 32MB + 4MB SRAM
> > 
> > > 
> > > > Is it possible to build such a kernel? If so, how to go about it?
> > > > Should one use ARCH=arm and a 32 bits toolchain or
> > > > ARCH=arm64 and 64 bits toolchain and adding 32 bits compile options?
> > > 
> > > If you want a 32-bit kernel, you'd have to use ARCH=arm and a 32-bit
> > > toolchain. This would give you an ARMv7 kernel that can run on ARMv8
> > > hardware. But you'd miss errata workarounds for example and any other
> > > features that come with ARMv8.
> 
> Anything else you can think of why 32 bits kernel is a bad idea?

32-bit kernels on arm64 hardware are not "officially" supported by the
maintainers (though there are some commercial deployments).

Given the amount of RAM, you might as well give 32-bit uClinux a try ;)
(even less tested).

> I do wonder why 64 bits A53 enforces SMP? I only have one core in my
> HW(custom ASIC)

We removed it because we were not aware of any such UP system. Note that
even with CONFIG_SMP enabled, the kernel still runs fine on a UP system.
I guess you want to get rid of extra code in the final image.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel