Re: [PATCH 9/9] [DNI] ARM: multi_v7_defconfig: Enable CONFIG_ARM_LPAE for multi_v7_config

["Arnd Bergmann" <arnd@arndb.de>]

On Fri, Jan 20, 2023, at 13:43, Alexander Stein wrote:
> Am Donnerstag, 19. Januar 2023, 17:07:30 CET schrieb Arnd Bergmann:
>> On Thu, Jan 19, 2023, at 16:27, Alexander Stein wrote:
>> 
>> In particular, it seems that the memory map of the PCI address
>> spaces is configurable, but only within that area you listed.
>> I see that section "28.4.2 PEX register descriptions" does list
>> a 64-bit prefetchable address space in addition to the 32-bit
>> non-prefetchable memory space, but the 64-bit space is not
>> listed in the DT. It would be a good idea to configure that
>> as well in order for devices to work that need a larger BAR,
>> such as a GPU, but it wouldn't help with fitting the PCIe
>> into non-LPAE 32-bit CPU address space.
>
> I'm not sure if I can follow you here. Do you have some keywords of what's 
> missing there?

Prefetchable_Memory_Base_Register, section 28.4.2.20 in the
document you pointed me to. 

PCIe addressing is usually split up into I/O space (kilobytes of
registers), non-prefetchable memory space (megabytes of registers
and memory and prefetchable 64-bit memory space (gigabytes of
device memory).

The prefetchable space is indicated by bit '30' of the first
word in the ranges property, so if that is configured, you
would see a third line there starting with 0xc2000000 or
0x42000000. Without this, PCIe cards that have prefetchable
BARs fall back to the non-prefetchable one, which may be
too small or less efficient. This is usually only relevant
for framebuffers on a GPU, but there are probably other
devices as well.

>> In the datasheet I also see that the chip theoretically
>> supports 8GB of DDR4, which would definitely put it beyond
>> the highmem limit, even with the 4G:4G memory split. Do you
>> know if there are ls1021a devices with more than 4GB of
>> installed memory?
>
> Where did you find those 8GB? Section 16.2 mentions it supports up to 4 banks/
> chip-selects which I would assume is much more. Also the memory map has a DRAM 
> region 2 for memory region 2-32GB. But yes this exceeds 32bit addressing.
> I'm not aware of ls1021 devices with more than 4GB memory. Our modules only 
> support up to 2GB.

I think I misread this, as section 2.2 mentions you can have
four chip-selects that are limited to either 2GB or 8GB each,
for a theoretical maximum of 26GB. As long as the practical
limit is 4GB or less, I think we're fine here. Linus Walleij
has is working on a prototype for changing the memory
management code to handle up to 4GB of contiguous RAM without
highmem, which will become relevant in the future as we get
rid of highmem support. On this chip, the first 4GB of
installed memory are not contiguous in the physical address
space, so this will need another set of patches on top.

As long as you only use the first chip-select with 2GB
of installed memory, very little will change for you.

It might be worthwhile to check if your system works
correctly with ARM_LPAE=y, VMSPLIT_2G=y and HIGHMEM=n,
which should be the best configuration for your system
anyway and will keep working after highmem gets removed.

    Arnd