Grub2: add UEFI support for accessing memory address above 4GB.

Grub2: add UEFI support for accessing memory address above 4GB.

[Yufuping]

[-- Attachment #1: Type: text/plain, Size: 314 bytes --]


Hi all,

Who can add the new feature for grub2:
Add UEFI support for accessing memory address above 4GB.
When using grub2 as PXE downloading engine, grub2 can get initrd file from network and put it to memory above 4GB.
The feature should support UEFI BIOS boot mode.
Thanks!

Best Regard,
Yanjianlong

[-- Attachment #2: Type: text/html, Size: 3306 bytes --]

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Leif Lindholm]

On Tue, Mar 07, 2017 at 01:55:01AM +0000, Yufuping wrote:
> Who can add the new feature for grub2:
> Add UEFI support for accessing memory address above 4GB.

Presumably you mean for x86_64?
Since GRUB supports all 5 architectures currently supported by the
UEFI specification, 3 of which are 64-bit, it is useful to be a bit
more precise.

> When using grub2 as PXE downloading engine, grub2 can get initrd
> file from network and put it to memory above 4GB.

I can think of nothing particularly related to PXE here.
The x86_64 port currently sets GRUB_EFI_MAX_USABLE_ADDRESS to
0xffffffff or 0x7fffffff, depending on toolchain configuration.

ARM64 sets it to 0xffffffffffffULL, and that works fine.

I seem to recall that the x86_64 port was being restricted due to
known bad firmware encountered in the past. It could be that it would
be worth adding an option to configure for enabling access to higher
addresses, alternatively for retaining compatibility with the broken
systems.

> The feature should support UEFI BIOS boot mode.

I do not understand this statement.

/
    Leif

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Vladimir 'phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1988 bytes --]

On Tue, Mar 7, 2017, 08:15 Leif Lindholm <leif.lindholm@linaro.org> wrote:

> On Tue, Mar 07, 2017 at 01:55:01AM +0000, Yufuping wrote:
> > Who can add the new feature for grub2:
> > Add UEFI support for accessing memory address above 4GB.
>
> Presumably you mean for x86_64?
> Since GRUB supports all 5 architectures currently supported by the
> UEFI specification, 3 of which are 64-bit, it is useful to be a bit
> more precise.
>
> > When using grub2 as PXE downloading engine, grub2 can get initrd
> > file from network and put it to memory above 4GB.
>
I'd like to know more about the usecase. Generally you should avoid
downloading or loading too large files in bootloader. I.a. TFTP protocol
has problems with files over about 100MIB. Generally you should download
only kernel + initrd and rest of the system should be on iSCSI or NFS.

>
> I can think of nothing particularly related to PXE here.
> The x86_64 port currently sets GRUB_EFI_MAX_USABLE_ADDRESS to
> 0xffffffff or 0x7fffffff, depending on toolchain configuration.
>
> ARM64 sets it to 0xffffffffffffULL, and that works fine.
>
> I seem to recall that the x86_64 port was being restricted due to
> known bad firmware encountered in the past. It could be that it would
> be worth adding an option to configure for enabling access to higher
> addresses, alternatively for retaining compatibility with the broken
> systems.
>
I'm opposed to a config option for this. We don't want to have several
variants of grub binaries for the same platform. If we want to support
>4GiB memory we should detect the buggy firmware on runtime. It's pretty
easy: buggy firmware didn't map memory above 4GiB. We can then either avoid
memory above 4GiB or map it ourselves.

>
> > The feature should support UEFI BIOS boot mode.
>
> I do not understand this statement.
>
> /
>     Leif
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>

[-- Attachment #2: Type: text/html, Size: 3496 bytes --]

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Michel Hermier]

[-- Attachment #1: Type: text/plain, Size: 2404 bytes --]

Le 7 mars 2017 17:24, "Vladimir 'phcoder' Serbinenko" <phcoder@gmail.com> a
écrit :



On Tue, Mar 7, 2017, 08:15 Leif Lindholm <leif.lindholm@linaro.org> wrote:

> On Tue, Mar 07, 2017 at 01:55:01AM +0000, Yufuping wrote:
> > Who can add the new feature for grub2:
> > Add UEFI support for accessing memory address above 4GB.
>
> Presumably you mean for x86_64?
> Since GRUB supports all 5 architectures currently supported by the
> UEFI specification, 3 of which are 64-bit, it is useful to be a bit
> more precise.
>
> > When using grub2 as PXE downloading engine, grub2 can get initrd
> > file from network and put it to memory above 4GB.
>
I'd like to know more about the usecase. Generally you should avoid
downloading or loading too large files in bootloader. I.a. TFTP protocol
has problems with files over about 100MIB. Generally you should download
only kernel + initrd and rest of the system should be on iSCSI or NFS.

>
> I can think of nothing particularly related to PXE here.
> The x86_64 port currently sets GRUB_EFI_MAX_USABLE_ADDRESS to
> 0xffffffff or 0x7fffffff, depending on toolchain configuration.
>
> ARM64 sets it to 0xffffffffffffULL, and that works fine.
>
> I seem to recall that the x86_64 port was being restricted due to
> known bad firmware encountered in the past. It could be that it would
> be worth adding an option to configure for enabling access to higher
> addresses, alternatively for retaining compatibility with the broken
> systems.
>
I'm opposed to a config option for this. We don't want to have several
variants of grub binaries for the same platform. If we want to support
>4GiB memory we should detect the buggy firmware on runtime. It's pretty
easy: buggy firmware didn't map memory above 4GiB. We can then either avoid
memory above 4GiB or map it ourselves.


What about a dynamic variable instead or at least accessible from script?
So a user could redefine a value of any kind.


> > The feature should support UEFI BIOS boot mode.
>
> I do not understand this statement.
>
> /
>     Leif
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[-- Attachment #2: Type: text/html, Size: 5484 bytes --]

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Vladimir 'phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2795 bytes --]

On Tue, Mar 7, 2017, 09:09 Michel Hermier <michel.hermier@gmail.com> wrote:

>
>
> Le 7 mars 2017 17:24, "Vladimir 'phcoder' Serbinenko" <phcoder@gmail.com>
> a écrit :
>
>
>
> On Tue, Mar 7, 2017, 08:15 Leif Lindholm <leif.lindholm@linaro.org> wrote:
>
> On Tue, Mar 07, 2017 at 01:55:01AM +0000, Yufuping wrote:
> > Who can add the new feature for grub2:
> > Add UEFI support for accessing memory address above 4GB.
>
> Presumably you mean for x86_64?
> Since GRUB supports all 5 architectures currently supported by the
> UEFI specification, 3 of which are 64-bit, it is useful to be a bit
> more precise.
>
> > When using grub2 as PXE downloading engine, grub2 can get initrd
> > file from network and put it to memory above 4GB.
>
> I'd like to know more about the usecase. Generally you should avoid
> downloading or loading too large files in bootloader. I.a. TFTP protocol
> has problems with files over about 100MIB. Generally you should download
> only kernel + initrd and rest of the system should be on iSCSI or NFS.
>
>
> I can think of nothing particularly related to PXE here.
> The x86_64 port currently sets GRUB_EFI_MAX_USABLE_ADDRESS to
> 0xffffffff or 0x7fffffff, depending on toolchain configuration.
>
> ARM64 sets it to 0xffffffffffffULL, and that works fine.
>
> I seem to recall that the x86_64 port was being restricted due to
> known bad firmware encountered in the past. It could be that it would
> be worth adding an option to configure for enabling access to higher
> addresses, alternatively for retaining compatibility with the broken
> systems.
>
> I'm opposed to a config option for this. We don't want to have several
> variants of grub binaries for the same platform. If we want to support
> >4GiB memory we should detect the buggy firmware on runtime. It's pretty
> easy: buggy firmware didn't map memory above 4GiB. We can then either avoid
> memory above 4GiB or map it ourselves.
>
>
> What about a dynamic variable instead or at least accessible from script?
> So a user could redefine a value of any kind.
>
What if advantage compared to automatic detection. And still, I want to
know about usecase

>
>
> > The feature should support UEFI BIOS boot mode.
>
> I do not understand this statement.
>
> /
>     Leif
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>

[-- Attachment #2: Type: text/html, Size: 8501 bytes --]

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Michel Hermier]

[-- Attachment #1: Type: text/plain, Size: 3255 bytes --]

Le 7 mars 2017 18:22, "Vladimir 'phcoder' Serbinenko" <phcoder@gmail.com> a
écrit :



On Tue, Mar 7, 2017, 09:09 Michel Hermier <michel.hermier@gmail.com> wrote:

>
>
> Le 7 mars 2017 17:24, "Vladimir 'phcoder' Serbinenko" <phcoder@gmail.com>
> a écrit :
>
>
>
> On Tue, Mar 7, 2017, 08:15 Leif Lindholm <leif.lindholm@linaro.org> wrote:
>
> On Tue, Mar 07, 2017 at 01:55:01AM +0000, Yufuping wrote:
> > Who can add the new feature for grub2:
> > Add UEFI support for accessing memory address above 4GB.
>
> Presumably you mean for x86_64?
> Since GRUB supports all 5 architectures currently supported by the
> UEFI specification, 3 of which are 64-bit, it is useful to be a bit
> more precise.
>
> > When using grub2 as PXE downloading engine, grub2 can get initrd
> > file from network and put it to memory above 4GB.
>
> I'd like to know more about the usecase. Generally you should avoid
> downloading or loading too large files in bootloader. I.a. TFTP protocol
> has problems with files over about 100MIB. Generally you should download
> only kernel + initrd and rest of the system should be on iSCSI or NFS.
>
>
> I can think of nothing particularly related to PXE here.
> The x86_64 port currently sets GRUB_EFI_MAX_USABLE_ADDRESS to
> 0xffffffff or 0x7fffffff, depending on toolchain configuration.
>
> ARM64 sets it to 0xffffffffffffULL, and that works fine.
>
> I seem to recall that the x86_64 port was being restricted due to
> known bad firmware encountered in the past. It could be that it would
> be worth adding an option to configure for enabling access to higher
> addresses, alternatively for retaining compatibility with the broken
> systems.
>
> I'm opposed to a config option for this. We don't want to have several
> variants of grub binaries for the same platform. If we want to support
> >4GiB memory we should detect the buggy firmware on runtime. It's pretty
> easy: buggy firmware didn't map memory above 4GiB. We can then either avoid
> memory above 4GiB or map it ourselves.
>
>
> What about a dynamic variable instead or at least accessible from script?
> So a user could redefine a value of any kind.
>
What if advantage compared to automatic detection. And still, I want to
know about usecase


Because I don't trust automatic detection. Even if one say it is 200% safe,
there is allways that machine that nobody heard of that will fail. So
having user being able to force some values is usually a good idea.


>
> > The feature should support UEFI BIOS boot mode.
>
> I do not understand this statement.
>
> /
>     Leif
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[-- Attachment #2: Type: text/html, Size: 12529 bytes --]

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Vladimir 'phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 3905 bytes --]

On Tue, Mar 7, 2017, 10:13 Michel Hermier <michel.hermier@gmail.com> wrote:

>
>
> Le 7 mars 2017 18:22, "Vladimir 'phcoder' Serbinenko" <phcoder@gmail.com>
> a écrit :
>
>
>
> On Tue, Mar 7, 2017, 09:09 Michel Hermier <michel.hermier@gmail.com>
> wrote:
>
>
>
> Le 7 mars 2017 17:24, "Vladimir 'phcoder' Serbinenko" <phcoder@gmail.com>
> a écrit :
>
>
>
> On Tue, Mar 7, 2017, 08:15 Leif Lindholm <leif.lindholm@linaro.org> wrote:
>
> On Tue, Mar 07, 2017 at 01:55:01AM +0000, Yufuping wrote:
> > Who can add the new feature for grub2:
> > Add UEFI support for accessing memory address above 4GB.
>
> Presumably you mean for x86_64?
> Since GRUB supports all 5 architectures currently supported by the
> UEFI specification, 3 of which are 64-bit, it is useful to be a bit
> more precise.
>
> > When using grub2 as PXE downloading engine, grub2 can get initrd
> > file from network and put it to memory above 4GB.
>
> I'd like to know more about the usecase. Generally you should avoid
> downloading or loading too large files in bootloader. I.a. TFTP protocol
> has problems with files over about 100MIB. Generally you should download
> only kernel + initrd and rest of the system should be on iSCSI or NFS.
>
>
> I can think of nothing particularly related to PXE here.
> The x86_64 port currently sets GRUB_EFI_MAX_USABLE_ADDRESS to
> 0xffffffff or 0x7fffffff, depending on toolchain configuration.
>
> ARM64 sets it to 0xffffffffffffULL, and that works fine.
>
> I seem to recall that the x86_64 port was being restricted due to
> known bad firmware encountered in the past. It could be that it would
> be worth adding an option to configure for enabling access to higher
> addresses, alternatively for retaining compatibility with the broken
> systems.
>
> I'm opposed to a config option for this. We don't want to have several
> variants of grub binaries for the same platform. If we want to support
> >4GiB memory we should detect the buggy firmware on runtime. It's pretty
> easy: buggy firmware didn't map memory above 4GiB. We can then either avoid
> memory above 4GiB or map it ourselves.
>
>
> What about a dynamic variable instead or at least accessible from script?
> So a user could redefine a value of any kind.
>
> What if advantage compared to automatic detection. And still, I want to
> know about usecase
>
>
> Because I don't trust automatic detection. Even if one say it is 200%
> safe, there is allways that machine that nobody heard of that will fail. So
> having user being able to force some values is usually a good idea.
>
This bug is well understood. There is no guesswork involved in detecting
unmapped ranges. And generally allowing overrides results in that everybody
uses overrides rather than reporting and fixing problems. Also exposing
this kind of details in higher levels is a whole different can of worms.
Allocatable ranges are determined before you reach any parser.

>
>
>
> > The feature should support UEFI BIOS boot mode.
>
> I do not understand this statement.
>
> /
>     Leif
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>

[-- Attachment #2: Type: text/html, Size: 18144 bytes --]

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Brendan Trotter]

[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]

Hi,

> Le 7 mars 2017 17:24, "Vladimir 'phcoder' Serbinenko" <phcoder@gmail.com>
a écrit :
>
> I'd like to know more about the usecase. Generally you should avoid
downloading or loading too large files in bootloader. I.a. TFTP protocol
has problems with files over about 100MIB. Generally you should download
only kernel + initrd and rest of the system should be on iSCSI or NFS.

"Ancient TFTP" (with 512-byte blocks and 16-bit block numbers that aren't
allowed to roll over) has problems with files over 32 MiB. "Modern
TFTP" (with larger/negotiated block size and 16-bit block numbers that are
allowed to roll over) has no limit at all.


> On Tue, Mar 7, 2017, 09:09 Michel Hermier <michel.hermier@gmail.com>
wrote:
>
> Because I don't trust automatic detection. Even if one say it is 200%
safe, there is allways that machine that nobody heard of that will fail. So
having user being able to force some values is usually a good idea.

What do you think is more reliable: well designed auto-detection code that
was written and tested by competent developer/s that know exactly what
they're doing and why; or a random user who failed read the documentation,
thought it did something else, screwed everything up, then blames you for
their mistake, then blames you for giving them the ability to make the
mistake?


Cheers,

Brendan

[-- Attachment #2: Type: text/html, Size: 1837 bytes --]

Re: Grub2: add UEFI support for accessing memory address above 4GB.

[Michel Hermier]

[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]

Le 8 mars 2017 10:58 AM, "Brendan Trotter" <btrotter@gmail.com> a écrit :

Hi,


> Le 7 mars 2017 17:24, "Vladimir 'phcoder' Serbinenko" <phcoder@gmail.com>
a écrit :
>
> I'd like to know more about the usecase. Generally you should avoid
downloading or loading too large files in bootloader. I.a. TFTP protocol
has problems with files over about 100MIB. Generally you should download
only kernel + initrd and rest of the system should be on iSCSI or NFS.

"Ancient TFTP" (with 512-byte blocks and 16-bit block numbers that aren't
allowed to roll over) has problems with files over 32 MiB. "Modern
TFTP" (with larger/negotiated block size and 16-bit block numbers that are
allowed to roll over) has no limit at all.


> On Tue, Mar 7, 2017, 09:09 Michel Hermier <michel.hermier@gmail.com>
wrote:
>
> Because I don't trust automatic detection. Even if one say it is 200%
safe, there is allways that machine that nobody heard of that will fail. So
having user being able to force some values is usually a good idea.

What do you think is more reliable: well designed auto-detection code that
was written and tested by competent developer/s that know exactly what
they're doing and why; or a random user who failed read the documentation,
thought it did something else, screwed everything up, then blames you for
their mistake, then blames you for giving them the ability to make the
mistake?


Something in the middle, have a well tuned detection, and user defined
variable (with a taint mechanism, that *blame/inform* the user that he use
a sensible option)

[-- Attachment #2: Type: text/html, Size: 2413 bytes --]