All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC] New object format for grub2
@ 2009-07-08 11:31 Bean
  2009-07-08 14:27 ` Marco Gerards
  0 siblings, 1 reply; 8+ messages in thread
From: Bean @ 2009-07-08 11:31 UTC (permalink / raw)
  To: The development of GRUB 2

Hi,

Why another format ? Here are some reason:

The current object format is ELF. Most unix based os use ELF as native
object format, but there are minor difference, and gcc may add extra
segment which is of no use for grub. Mingw/cygwin uses PE, and we need
to convert it to ELF. Platform like OSX also requires special
treatment. All these make the build system complicated, so we might
just use an unified format specific to grub.

The ELF format is designed to be compatible with many system, it's
neither compact nor easy to parse. If we invent a new format, we
should make it simple.

There are extra data useful to grub, such as module dependence list.
It's possible to piggyback it using special segment of ELF, but it's
not very clean. We could specify the information explicitly in our own
format.

Here are my proposed format:

struct grub_obj_header
{
  grub_uint32_t magic;
  grub_uint32_t version;
  grub_uint32_t mod_attr;
  grub_uint32_t mod_deps;
  grub_uint32_t string_table;
  grub_uint32_t symbol_table;
  grub_uint32_t init_func;
  grub_uint32_t fini_func;
  struct grub_obj_segment segments[0];
} __attribute__((packed));

magic: magic number "GOBJ"
version: version number
mod_attr: offset to the module attributes
mod_deps: offset to the module dependent list
string_table: offset to the string table
symbol_table: offset to the symbol table
init_func: init function offset, init function must be in segment 0
fini_func: fini function offset, fini function must be in segment 0
segments: segment list, ends when segment type is 0

segment structure, the fields should be self explaining

struct grub_obj_segment
{
  grub_uint16_t type;
  grub_uint16_t align;
  grub_uint32_t offset;
  grub_uint32_t raw_size;
  grub_uint32_t mem_size;
} __attribute__((packed));

symbol table defines import, export and unnamed symbols, it's the
catenation of structure grub_obj_symbol, and ends when segment field
is 0xff:

#define GRUB_OBJ_SYM_EXPORT	0
#define GRUB_OBJ_SYM_PC32		1
#define GRUB_OBJ_SYM_ABS32	2
#define GRUB_OBJ_SYM_PC64		3
#define GRUB_OBJ_SYM_ABS64	4

struct grub_obj_symbol
{
  grub_uint8_t segment;
  grub_uint8_t type;
  grub_uint16_t name;
  grub_uint32_t offset;
} __attribute__((packed));

name is the offset to the string table, if name == 0, it's an unnamed symbol.

mod_attr is a series of null-terminated string, ended with an extra 0,
it can be used to store information, such as:

fs:fat
command:ls
handler.script:sh

we could use it to generate the *.lst files, which is more reliable
than sed the source files.

mod_deps is a series of null-terminated string, ended with an extra 0.
To make it easier to modify, mod_deps should be at the end of the
module file.

-- 
Bean



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] New object format for grub2
  2009-07-08 11:31 [RFC] New object format for grub2 Bean
@ 2009-07-08 14:27 ` Marco Gerards
  2009-07-08 15:37   ` Bean
                     ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Marco Gerards @ 2009-07-08 14:27 UTC (permalink / raw)
  To: The development of GRUB 2

Hi,

Bean <bean123ch@gmail.com> writes:

> Why another format ? Here are some reason:
>
> The current object format is ELF. Most unix based os use ELF as native
> object format, but there are minor difference, and gcc may add extra
> segment which is of no use for grub. Mingw/cygwin uses PE, and we need
> to convert it to ELF. Platform like OSX also requires special
> treatment. All these make the build system complicated, so we might
> just use an unified format specific to grub.
>
> The ELF format is designed to be compatible with many system, it's
> neither compact nor easy to parse. If we invent a new format, we
> should make it simple.

There is another problem with this: the maintainance burden.  People
know ELF, we have ELF and people will know ELF in the future.
Furthermore, as far as I am concerned, GNU/Linux is our main
platform.  I do not mind supporting windows or so and we can support
it in a sane way, but changing our binary formats for it is one step
too far for me...

--
Marco




^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] New object format for grub2
  2009-07-08 14:27 ` Marco Gerards
@ 2009-07-08 15:37   ` Bean
  2009-07-08 18:51   ` Pavel Roskin
  2009-07-10 17:27   ` Robert Millan
  2 siblings, 0 replies; 8+ messages in thread
From: Bean @ 2009-07-08 15:37 UTC (permalink / raw)
  To: The development of GRUB 2

On Wed, Jul 8, 2009 at 10:27 PM, Marco Gerards<mgerards@xs4all.nl> wrote:
> Hi,
>
> Bean <bean123ch@gmail.com> writes:
>
>> Why another format ? Here are some reason:
>>
>> The current object format is ELF. Most unix based os use ELF as native
>> object format, but there are minor difference, and gcc may add extra
>> segment which is of no use for grub. Mingw/cygwin uses PE, and we need
>> to convert it to ELF. Platform like OSX also requires special
>> treatment. All these make the build system complicated, so we might
>> just use an unified format specific to grub.
>>
>> The ELF format is designed to be compatible with many system, it's
>> neither compact nor easy to parse. If we invent a new format, we
>> should make it simple.
>
> There is another problem with this: the maintainance burden.  People
> know ELF, we have ELF and people will know ELF in the future.
> Furthermore, as far as I am concerned, GNU/Linux is our main
> platform.  I do not mind supporting windows or so and we can support
> it in a sane way, but changing our binary formats for it is one step
> too far for me...

Hi,

An important issue to consider is the size of kernel. As of now,
core.img is approaching the upper limit. And the size of ELF header is
quite significant. For example, the size of hello.mod is 1132, but the
actually code is only 120 !  Also, the size of dynamic loader is quite
big, about 3K, which is the fourth largest part in the kernel, right
after misc, startup and symlist.

And with customized format, it's easier to implement features like
library, which can help to reduce kernel size further.

-- 
Bean



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] New object format for grub2
  2009-07-08 14:27 ` Marco Gerards
  2009-07-08 15:37   ` Bean
@ 2009-07-08 18:51   ` Pavel Roskin
  2009-07-08 22:57     ` Vladimir 'phcoder' Serbinenko
  2009-07-10 17:27   ` Robert Millan
  2 siblings, 1 reply; 8+ messages in thread
From: Pavel Roskin @ 2009-07-08 18:51 UTC (permalink / raw)
  To: The development of GRUB 2

Hello, Marco!

On Wed, 2009-07-08 at 16:27 +0200, Marco Gerards wrote:

> There is another problem with this: the maintainance burden.  People
> know ELF, we have ELF and people will know ELF in the future.
> Furthermore, as far as I am concerned, GNU/Linux is our main
> platform.  I do not mind supporting windows or so and we can support
> it in a sane way, but changing our binary formats for it is one step
> too far for me...

There is actually a maintenance burden that is caused by the need to
support the ELF format.  We need to strip some sections from the modules
for GRUB to process them correctly.

Support for "secondary" platforms means that the build system should
recognize of compile tools to convert their native formats to ELF.  It's
more complex than having one converter for all platforms that is
completely under our control.

In addition to the problem on the core.img size, we have a problem of
the size of all the core plus modules.  GRUB already barely fits the
boot partitions on PowerPC, and it won't fit if yaboot is to be
preserved.  Soon it will push the limits of a 1.44Mb floppy (without any
kernels, mind you).  And then there is an issue with writing GRUB on a
ROM.

I'm not exactly enthusiastic about another major change when we need
stabilization, but the maintenance argument really doesn't apply here.

-- 
Regards,
Pavel Roskin



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] New object format for grub2
  2009-07-08 18:51   ` Pavel Roskin
@ 2009-07-08 22:57     ` Vladimir 'phcoder' Serbinenko
  2009-07-08 23:21       ` Pavel Roskin
  2009-07-10 17:33       ` Robert Millan
  0 siblings, 2 replies; 8+ messages in thread
From: Vladimir 'phcoder' Serbinenko @ 2009-07-08 22:57 UTC (permalink / raw)
  To: The development of GRUB 2

On Wed, Jul 8, 2009 at 8:51 PM, Pavel Roskin<proski@gnu.org> wrote:
> Hello, Marco!
>
> On Wed, 2009-07-08 at 16:27 +0200, Marco Gerards wrote:
>
>> There is another problem with this: the maintainance burden.  People
>> know ELF, we have ELF and people will know ELF in the future.
>> Furthermore, as far as I am concerned, GNU/Linux is our main
>> platform.  I do not mind supporting windows or so and we can support
>> it in a sane way, but changing our binary formats for it is one step
>> too far for me...
>
> There is actually a maintenance burden that is caused by the need to
> support the ELF format.  We need to strip some sections from the modules
> for GRUB to process them correctly.
>
What about SELF? (coreboot format) If it's sane I would prefer to
share a format between these 2 projects.
> Support for "secondary" platforms means that the build system should
> recognize of compile tools to convert their native formats to ELF.  It's
> more complex than having one converter for all platforms that is
> completely under our control.
>
> In addition to the problem on the core.img size, we have a problem of
> the size of all the core plus modules.  GRUB already barely fits the
> boot partitions on PowerPC, and it won't fit if yaboot is to be
> preserved.  Soon it will push the limits of a 1.44Mb floppy (without any
> kernels, mind you).  And then there is an issue with writing GRUB on a
> ROM.
>
> I'm not exactly enthusiastic about another major change when we need
> stabilization, but the maintenance argument really doesn't apply here.
>
> --
> Regards,
> Pavel Roskin
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>



-- 
Regards
Vladimir 'phcoder' Serbinenko

Personal git repository: http://repo.or.cz/w/grub2/phcoder.git



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] New object format for grub2
  2009-07-08 22:57     ` Vladimir 'phcoder' Serbinenko
@ 2009-07-08 23:21       ` Pavel Roskin
  2009-07-10 17:33       ` Robert Millan
  1 sibling, 0 replies; 8+ messages in thread
From: Pavel Roskin @ 2009-07-08 23:21 UTC (permalink / raw)
  To: The development of GRUB 2

On Thu, 2009-07-09 at 00:57 +0200, Vladimir 'phcoder' Serbinenko wrote:

> What about SELF? (coreboot format) If it's sane I would prefer to
> share a format between these 2 projects.

The description is here: http://www.coreboot.org/SELF

I think the biggest problem is lack of support for any relocations.
Even if we resolve all symbols against the kernel, modules still may
depend on other modules.

Either we preallocate all modules to fixed places in memory, or we'll
need to use the PARAMS section to hold relocations, magic numbers,
versions and so on.

My impression in that the SELF format is too dumb and we don't want to
go there.

-- 
Regards,
Pavel Roskin



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] New object format for grub2
  2009-07-08 14:27 ` Marco Gerards
  2009-07-08 15:37   ` Bean
  2009-07-08 18:51   ` Pavel Roskin
@ 2009-07-10 17:27   ` Robert Millan
  2 siblings, 0 replies; 8+ messages in thread
From: Robert Millan @ 2009-07-10 17:27 UTC (permalink / raw)
  To: The development of GRUB 2

On Wed, Jul 08, 2009 at 04:27:08PM +0200, Marco Gerards wrote:
> Hi,
> 
> Bean <bean123ch@gmail.com> writes:
> 
> > Why another format ? Here are some reason:
> >
> > The current object format is ELF. Most unix based os use ELF as native
> > object format, but there are minor difference, and gcc may add extra
> > segment which is of no use for grub. Mingw/cygwin uses PE, and we need
> > to convert it to ELF. Platform like OSX also requires special
> > treatment. All these make the build system complicated, so we might
> > just use an unified format specific to grub.
> >
> > The ELF format is designed to be compatible with many system, it's
> > neither compact nor easy to parse. If we invent a new format, we
> > should make it simple.
> 
> There is another problem with this: the maintainance burden.  People
> know ELF, we have ELF and people will know ELF in the future.
> Furthermore, as far as I am concerned, GNU/Linux is our main
> platform.  I do not mind supporting windows or so and we can support
> it in a sane way, but changing our binary formats for it is one step
> too far for me...

Hi,

Even if changing our binary format were deemed a good idea, I think this
is very bad timing for changing GRUB's core in such significant ways.

Remember that the majority of GRUB users are still running GRUB Legacy,
and we've just got started in migrating them.  Ubuntu has committed to
GRUB 2, and IMO we should try to give them a stable codebase for their
release.

This sort of changes tends to cause regressions.  Notice the hiddenmenu
one, introduced with the script engine split, which is still unsolved.
Hiddenmenu is what Ubuntu is currently using with GRUB Legacy.  It's
important that we can provide them with something equivalent.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC] New object format for grub2
  2009-07-08 22:57     ` Vladimir 'phcoder' Serbinenko
  2009-07-08 23:21       ` Pavel Roskin
@ 2009-07-10 17:33       ` Robert Millan
  1 sibling, 0 replies; 8+ messages in thread
From: Robert Millan @ 2009-07-10 17:33 UTC (permalink / raw)
  To: The development of GRUB 2

On Thu, Jul 09, 2009 at 12:57:43AM +0200, Vladimir 'phcoder' Serbinenko wrote:
> On Wed, Jul 8, 2009 at 8:51 PM, Pavel Roskin<proski@gnu.org> wrote:
> > Hello, Marco!
> >
> > On Wed, 2009-07-08 at 16:27 +0200, Marco Gerards wrote:
> >
> >> There is another problem with this: the maintainance burden.  People
> >> know ELF, we have ELF and people will know ELF in the future.
> >> Furthermore, as far as I am concerned, GNU/Linux is our main
> >> platform.  I do not mind supporting windows or so and we can support
> >> it in a sane way, but changing our binary formats for it is one step
> >> too far for me...
> >
> > There is actually a maintenance burden that is caused by the need to
> > support the ELF format.  We need to strip some sections from the modules
> > for GRUB to process them correctly.
> >
> What about SELF? (coreboot format) If it's sane I would prefer to
> share a format between these 2 projects.

I think the most important advantage of using ELF binaries is that it was
widely available & very mature utilities to work with them.

I don't know SELF in detail, but if it prevents us from e.g. running objdump
on our binaries then I think it's a bad idea.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."



^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2009-07-10 17:34 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-07-08 11:31 [RFC] New object format for grub2 Bean
2009-07-08 14:27 ` Marco Gerards
2009-07-08 15:37   ` Bean
2009-07-08 18:51   ` Pavel Roskin
2009-07-08 22:57     ` Vladimir 'phcoder' Serbinenko
2009-07-08 23:21       ` Pavel Roskin
2009-07-10 17:33       ` Robert Millan
2009-07-10 17:27   ` Robert Millan

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.