Re: Can the grub2.02 improve the limit of initrd size in 64-bit system? (Lennart Sorensen)

Re: Can the grub2.02 improve the limit of initrd size in 64-bit system? (Lennart Sorensen)

[Zhang]

[-- Attachment #1: Type: text/plain, Size: 2185 bytes --]



Yes,I want to an initrd  as big as possible on  the service where ram size is large.
I am sorry that I have a mistake.The right is as follows:
x86_64:
It seems good for initrd size between 0x37feffff-0x7fffffff（2GB）.But the grub seems it is against that.


1、BIOS in legacy mode :
I can load a 1.75G image-initrd in the x86_64 system and the linux boot normal.  In my experiment,  the image-initrd of size is n 0x37feffff-0x7fffffff（2GB）
boot normal.


The limit value 0x7fffffff is that grub_read_file uses the len is singed int when I use the  BIOS in legacy mode to boot linux.
I see that 0x37ffffff(896M) is a line between the normal zone and highmem zone in the 32-bit system. The grub is 32-bit mode and the linux change the 32-bit to 64-bit. So is the reason the grub set GRUB_LINUX_INITRD_MAX_ADDRESS =0x37FFFFFF  because grub and kernel need 32-bit mode?
I know I have enough ram size  and 64-bit kernel , so can I set GRUB_LINUX_INITRD_MAX_ADDRESS =0x7FFFFFFF ?


  if (grub_le_to_cpu16 (linux_params.version) >= 0x0203)
    {
      addr_max = grub_cpu_to_le32 (linux_params.initrd_addr_max);
      /* XXX in reality, Linux specifies a bogus value, so
         it is necessary to make sure that ADDR_MAX does not exceed
         0x3fffffff.  */
     if (addr_max > GRUB_LINUX_INITRD_MAX_ADDRESS)
              addr_max = GRUB_LINUX_INITRD_MAX_ADDRESS;
  else
    addr_max = GRUB_LINUX_INITRD_MAX_ADDRESS;
  if (linux_mem_size != 0 && linux_mem_size < addr_max)
    addr_max = linux_mem_size;

  /* Linux 2.3.xx has a bug in the memory range check, so avoid
     the last page.
     Linux 2.2.xx has a bug in the memory range check, which is
     worse than that of Linux 2.3.xx, so avoid the last 64kb.  */
  addr_max -= 0x10000;



32-bit system:
DMA zone 0-16M
Normal zone:16M-896M
highmenm zone :896M-1G




2、Does EFI support the 64-bit boot protocol?
UEFI  boot:
It seems that the grub2.02 is 64-bit because the grub support 64-bit data. The function of grub_read_file can read the file above 2GB.
when I use a 2G image-initrd ,it boots normal. But when I use a 3G image-initrd, it shows "error: out of memory".
So is it a limit in malloc_in_rang function?






[-- Attachment #2: Type: text/html, Size: 4320 bytes --]

Re: Can the grub2.02 improve the limit of initrd size in 64-bit system? (Lennart Sorensen)

[Vladimir 'phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 194 bytes --]

On Thu, 1 Aug 2019, 05:20 Zhang, <zhang_uestc1@163.com> wrote:

>
> Yes,I want to an initrd  as big as possible on  the service where ram size
> is large.
>
Why? Please detail your usecase

>
>

[-- Attachment #2: Type: text/html, Size: 868 bytes --]

Re: Can the grub2.02 improve the limit of initrd size in 64-bit system?

[Zhang]

[-- Attachment #1: Type: text/plain, Size: 2261 bytes --]





Yes,I want to an initrd  as big as possible on  the service where ram size is large. Maybe the initrd can include files > 1GB.


I am sorry that I have a mistake.The right is as follows:
x86_64:
It seems good for initrd size between 0x37feffff-0x7fffffff（2GB）.But the grub seems it is against that.


1、BIOS in legacy mode :
I can load a 1.75G image-initrd in the x86_64 system and the linux boot normal.  In my experiment,  the image-initrd of size is n 0x37feffff-0x7fffffff（2GB）
which can boot normal.


The limit value 0x7fffffff is that grub_read_file uses the len is signed int (<2GB)when I use the  BIOS in legacy mode to boot linux.
I see that 0x37ffffff(896M) is a line between the normal zone and highmem zone in the 32-bit system. The grub is 32-bit mode and the linux changes the 32-bit to 64-bit. So is the reason that the grub sets GRUB_LINUX_INITRD_MAX_ADDRESS =0x37FFFFFF  because grub and kernel need 32-bit mode?
I know I have enough ram size  and 64-bit kernel , so can I set GRUB_LINUX_INITRD_MAX_ADDRESS =0x7FFFFFFF ?


  if (grub_le_to_cpu16 (linux_params.version) >= 0x0203)
    {
      addr_max = grub_cpu_to_le32 (linux_params.initrd_addr_max);
      /* XXX in reality, Linux specifies a bogus value, so
         it is necessary to make sure that ADDR_MAX does not exceed
         0x3fffffff.  */
     if (addr_max > GRUB_LINUX_INITRD_MAX_ADDRESS)
              addr_max = GRUB_LINUX_INITRD_MAX_ADDRESS;
  else
    addr_max = GRUB_LINUX_INITRD_MAX_ADDRESS;
  if (linux_mem_size != 0 && linux_mem_size < addr_max)
    addr_max = linux_mem_size;

  /* Linux 2.3.xx has a bug in the memory range check, so avoid
     the last page.
     Linux 2.2.xx has a bug in the memory range check, which is
     worse than that of Linux 2.3.xx, so avoid the last 64kb.  */
  addr_max -= 0x10000;



32-bit system:
DMA zone 0-16M
Normal zone:16M-896M
highmenm zone :896M-1G




2、Does EFI support the 64-bit boot protocol?
UEFI  boot:
It seems that the grub2.02 is 64-bit because the grub supports 64-bit data. The function of grub_read_file can read the file above 2GB.
when I use a 2G image-initrd ,it boots normal. But when I use a 3G image-initrd, it shows "error: out of memory".
So is it a limit in malloc_in_rang function?










 

[-- Attachment #2: Type: text/html, Size: 4677 bytes --]

Re: Can the grub2.02 improve the limit of initrd size in 64-bit system?

[Vladimir 'phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

пт, 2 авг. 2019 г., 08:45 Zhang <zhang_uestc1@163.com>:

>
>
> Yes,I want to an initrd  as big as possible on  the service where ram size
> is large. Maybe the initrd can include files > 1GB.
>
> I am sorry that I have a mistake.The right is as follows:
> x86_64:
>
> *It seems good for initrd size between 0x37feffff-0x7fffffff（2GB）.But the grub **seems it is against that.*
>
>
This is still not a usecase. You can't ask people to invest resources into
implementing something without having at least a usecase.

[-- Attachment #2: Type: text/html, Size: 1241 bytes --]

Re: Can the grub2.02 improve the limit of initrd size in 64-bit system?

[Chris Murphy]

On Wed, Jul 31, 2019 at 2:18 PM Vladimir 'phcoder' Serbinenko
<phcoder@gmail.com> wrote:
>
> > So 32 bit arm requires kernel + initrd less than 512MB, 64 bit
> > arm requires kernel + inirrd less than 32GB.  If I read the code
> > correctly, booting x86 in efi mode the max initrd file allocation is 1GB
> > (0x3fffffff).  Seems the code is shared for 32 and 64 bit EFI and doesn't
> > allow anything more on 64 bit than 32 bit.  Of course no normal system
> > would ever need more so no point implementing it.
> >
> > A 1GB ramdisk would be plenty to start a system that could then load more
> > ramdisks if it really wanted to.  Could any sensible use case exist for
> > loading an initrd that large?
>
> I agree. I don't think that implementing support of > 1GiB ramdisk
> makes sense unless we have a usecase in mind.
>
> Original reporter: can you please ellaborate why you need such a large
> ramdisk and why you can't let kernel mount additional images?

And also for such a large file, is there sufficient I/O support for
quickly reading it? I vaguely recall some work had to be done in the
kernel to get it to support faster storage devices such as NVMe.

A possible use case would be resuming from a hibernation image.

-- 
Chris Murphy

Re: Can the grub2.02 improve the limit of initrd size in 64-bit system?

[Vladimir 'phcoder' Serbinenko]

> So 32 bit arm requires kernel + initrd less than 512MB, 64 bit
> arm requires kernel + inirrd less than 32GB.  If I read the code
> correctly, booting x86 in efi mode the max initrd file allocation is 1GB
> (0x3fffffff).  Seems the code is shared for 32 and 64 bit EFI and doesn't
> allow anything more on 64 bit than 32 bit.  Of course no normal system
> would ever need more so no point implementing it.
>
> A 1GB ramdisk would be plenty to start a system that could then load more
> ramdisks if it really wanted to.  Could any sensible use case exist for
> loading an initrd that large?

I agree. I don't think that implementing support of > 1GiB ramdisk
makes sense unless we have a usecase in mind.

Original reporter: can you please ellaborate why you need such a large
ramdisk and why you can't let kernel mount additional images?

Re: Can the grub2.02 improve the limit of initrd size in 64-bit system?

[Lennart Sorensen]

On Tue, Jul 30, 2019 at 10:50:23AM +0800, 电子科技大学-张旭霞 wrote:
> Hi All,
> 
> environment : X86-64 , kernel 3.16.0(64-bit support),grub2.02
> 
> I compile the grub2.02 by default. And the grub boots the linux kernel with 32-bit boot protocol.(Documents/X86/boot.txt)
> 
> I want to use the initrd as big as possible, but it is limited as follows.
> 
> But when I make a image-initrd that size is 1.75G(no compression), the grub can boot linux normal .
> 
> It seems good for initrd size between 0x37feffff-0x3fffffff.But the grub seems it is against that.
> 
> So I have some questions:
> 
> 1、Why it is necessary to make sure that ADDR_MAX does not exceed 0x3fffffff ? Has it a relationship about the 32-bit kernel?
> 
> 2、Can I  set addr_max=0x3fFFFFFF when (grub_le_to_cpu16 (linux_params.version) >= 0x0203?
> 
> include/grub/i386/linux.h
> 
> #define GRUB_LINUX_INITRD_MAX_ADDRESS   0x37FFFFFF

Interesting that on arm the address offset is allowed to be 32GB on 64bit
arm.  x86 does not appear to have any special handling for 64 bit yet.

So 32 bit arm requires kernel + initrd less than 512MB, 64 bit
arm requires kernel + inirrd less than 32GB.  If I read the code
correctly, booting x86 in efi mode the max initrd file allocation is 1GB
(0x3fffffff).  Seems the code is shared for 32 and 64 bit EFI and doesn't
allow anything more on 64 bit than 32 bit.  Of course no normal system
would ever need more so no point implementing it.

A 1GB ramdisk would be plenty to start a system that could then load more
ramdisks if it really wanted to.  Could any sensible use case exist for
loading an initrd that large?

-- 
Len Sorensen

Can the grub2.02 improve the limit of initrd size in 64-bit system?

[电子科技大学-张旭霞]

[-- Attachment #1: Type: text/plain, Size: 1374 bytes --]

Hi All,

environment : X86-64 , kernel 3.16.0(64-bit support),grub2.02

 

I compile the grub2.02 by default. And the grub boots the linux kernel with 32-bit boot protocol.(Documents/X86/boot.txt)

 

I want to use the initrd as big as possible, but it is limited as follows.

But when I make a image-initrd that size is 1.75G(no compression), the grub can boot linux normal .

It seems good for initrd size between 0x37feffff-0x3fffffff.But the grub seems it is against that.

 

So I have some questions:

1、Why it is necessary to make sure that ADDR_MAX does not exceed 0x3fffffff ? Has it a relationship about the 32-bit kernel?

2、Can I  set addr_max=0x3fFFFFFF when (grub_le_to_cpu16 (linux_params.version) >= 0x0203?

 

 

include/grub/i386/linux.h

#define GRUB_LINUX_INITRD_MAX_ADDRESS   0x37FFFFFF

 

grub-core/loader/i386/linux.c

grub_cmd_initrd:

  /* Get the highest address available for the initrd.  */

  if (grub_le_to_cpu16 (linux_params.version) >= 0x0203)

    {

      addr_max = grub_cpu_to_le32 (linux_params.initrd_addr_max);

      /* XXX in reality, Linux specifies a bogus value, so

         it is necessary to make sure that ADDR_MAX does not exceed

         0x3fffffff.  */

     if (addr_max > GRUB_LINUX_INITRD_MAX_ADDRESS)

        addr_max = GRUB_LINUX_INITRD_MAX_ADDRESS;

    }

  else

    addr_max = GRUB_LINUX_INITRD_MAX_ADDRESS;

[-- Attachment #2: Type: text/html, Size: 4214 bytes --]