Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3

Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3

[Etienne Lorrain]

Vivek Goyal wrote:
>How do I know which program header is real mode code and the boot loader
> is not supposed to load it? May be PT_LOAD header with physical addr 0?
> What happens if changes happen and down the line we start compiling 
> real mode code for non-zero address?

 Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode
cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything
different (like with virtual address at 0xC0000000) is Linux protected mode.
Considering the linker used it is always the 4th program header, before there
were only 3 program header,third one stay the NOTE one.

> Hence I think keeping real mode code out of vmlinux might prove to be a good idea.

 Well, ELF is a format made to describe what to load, at which addres, and I assume
everyone recognise that some ELF section shall not be loaded, like the debug information.
Gujin accepts having the real-mode treated like a program text and so in the program header,
or like extra memory block with content like the symbol table used by a debugger, I do not
know what is best so have implemented the two.

> Secondly, if you compile real mode code with vmlinux, what would be the
> entry point for this ELF file? Real mode entry? Then I have not way to
> find out from ELF headers where is the protected mode entry point and
> I can not do use this vmlinux with kexec/kdump.

 I did not touch the kernel entry point, the real mode entry point is at offset zero
in the real-mode program header/section table.

> OTOH, now bzImage is relocatable. Is this image going to be relocatable?
> How do we take care of that?

 Just I did not completely understand why you need relocation (and the announce
was when I was in holidays far away). I know a simple kernel is needed to do
some debugging when the main kernel has crashed, this kernel is better loaded
at for instance 16 Mbytes. You probably do not want the same kernel as the main
one because it may crash the same way, and you start a loop - and even then
if you have exactly the same kernel it is easier to use the same write-protected
block of memory with different data sections.
 But I probably do not understand the problem so do not know what to write.

  Thanks,
 Etienne.


	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com

Re: Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3

[Vivek Goyal]

On Wed, Feb 07, 2007 at 02:55:39PM +0000, Etienne Lorrain wrote:
> Vivek Goyal wrote:
> >How do I know which program header is real mode code and the boot loader
> > is not supposed to load it? May be PT_LOAD header with physical addr 0?
> > What happens if changes happen and down the line we start compiling 
> > real mode code for non-zero address?
> 
>  Yes, any PT_LOAD below 64 Kbytes can only be real mode, and real-mode
> cannot be loaded higher, and cannot be bigger than 640 Kbytes, anything
> different (like with virtual address at 0xC0000000) is Linux protected mode.
> Considering the linker used it is always the 4th program header, before there
> were only 3 program header,third one stay the NOTE one.
> 

Well this is all a lot of speculation. This is not standard way of retrieving
information from ELF. Number of program headers finally created keep on
changing. Previously it was left to the linker and now people have changed
it to 3 by specifically using PHDR directive.
  
> > Hence I think keeping real mode code out of vmlinux might prove to be a good idea.
> 
>  Well, ELF is a format made to describe what to load, at which addres, and I assume
> everyone recognise that some ELF section shall not be loaded, like the debug information.
> Gujin accepts having the real-mode treated like a program text and so in the program header,
> or like extra memory block with content like the symbol table used by a debugger, I do not
> know what is best so have implemented the two.
> 
> > Secondly, if you compile real mode code with vmlinux, what would be the
> > entry point for this ELF file? Real mode entry? Then I have not way to
> > find out from ELF headers where is the protected mode entry point and
> > I can not do use this vmlinux with kexec/kdump.
> 
>  I did not touch the kernel entry point, the real mode entry point is at offset zero
> in the real-mode program header/section table.
> 
> > OTOH, now bzImage is relocatable. Is this image going to be relocatable?
> > How do we take care of that?
> 
>  Just I did not completely understand why you need relocation (and the announce
> was when I was in holidays far away). I know a simple kernel is needed to do
> some debugging when the main kernel has crashed, this kernel is better loaded
> at for instance 16 Mbytes. You probably do not want the same kernel as the main
> one because it may crash the same way, and you start a loop - and even then
> if you have exactly the same kernel it is easier to use the same write-protected
> block of memory with different data sections.
>  But I probably do not understand the problem so do not know what to write.
> 

As long as one can make sure that test kernel boots (commonly the case with
distros), same kernel can be used as capture kernel too. So the idea here
is to be able to use same kernel binary as production kernel and capture
kernel hence distros don't have to ship an additional kernel binary compiled
for a different physical addr just for dump capturing purposes.

Currently relocation information is extracted from vmlinux and packed in
final bzImage after some processing. After execution of real mode code
and once the image is decompressed, all the relocations are performed and
then control is transferred to kernel.

Thanks
Vivek

Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3

[Etienne Lorrain]

Eric W. Biederman wrote:
> Etienne Lorrain <etienne_lorrain@yahoo.fr> writes:
> >  New features are more that this real-mode function can return an error
> >  to the bootloader to tell something to the user, so the user can select
> >  another kernel with the right processor, another video mode... with
> >  clean error messages - not a crash dump because this assembly
> >  instruction is not for that processor.
>
> Having an error handling compatibility that is backwards compatible sounds
> interesting.

 Well, if the function called at offset 0 in the real-mode section return
 non zero, that is probably an error - that is Gujin interface, but do not ask
 me to change other bootloaders to handle that.
 This function is called with few parameters, one is a string pointer
 if the function do not want to print the problem report itself.
 The function is called with an offset of zero in its own intel segment.

> >   I am still saying that the bootloader knows the root filesystem to
> >  be used by the kernel it loads, and that ELF is a clean format to
> >  store different sections to be loaded into memory at predefined
> >  addresses.
> 
> Yes.  Although when you think sections instead of segments I'm a little
> worried.

  Well, for my defense, if Gujin do not find the real mode code in the program
 header it search for it in the section table, if the program header cannot be
 modified for some reason. There may be some issues if that ELF file is
 "stripped" but it seems to work on my side.

> >  Also there isn't any more kernel size limit.
>
> I think as HPA points out we have gotten past that a long time
> ago with the bzImage format.

  There was still a test not that long ago (in the code I am not executing) - but
 I was wrong to bring this point forward.

> With the right delicacy, and preserving backwards compatibility
> with existing bootloaders I think we can achieve things.

  Well, you can generate with the proposed patch and boot with SYSLINUX,
 Grub and LILO, but trying to clean Linux real-mode code is where I begun.

> The big issue is that sometimes bootloaders are a little bit brittle.
> Eric

  I like to think mine is more stable - maybe I am a dreamer...
  I do not compete on network booting neither.

  Etienne.


	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com

Re: Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3

[Eric W. Biederman]

Etienne Lorrain <etienne_lorrain@yahoo.fr> writes:

> Eric W. Biederman wrote:
>> Etienne Lorrain <etienne_lorrain@yahoo.fr> writes:
>> >  New features are more that this real-mode function can return an error
>> >  to the bootloader to tell something to the user, so the user can select
>> >  another kernel with the right processor, another video mode... with
>> >  clean error messages - not a crash dump because this assembly
>> >  instruction is not for that processor.
>>
>> Having an error handling compatibility that is backwards compatible sounds
>> interesting.
>
>  Well, if the function called at offset 0 in the real-mode section return
>  non zero, that is probably an error - that is Gujin interface, but do not ask
>  me to change other bootloaders to handle that.
>  This function is called with few parameters, one is a string pointer
>  if the function do not want to print the problem report itself.
>  The function is called with an offset of zero in its own intel
>  segment.

So I think there are some backwards compatibility issues with your
real mode code interface.  Setting a new flag that says to return instead
of printing an error message and halting would be one way to handle
this.

Is your real mode C code section relative such that it can be loaded
at different addresses and still work?

>> >   I am still saying that the bootloader knows the root filesystem to
>> >  be used by the kernel it loads, and that ELF is a clean format to
>> >  store different sections to be loaded into memory at predefined
>> >  addresses.
>> 
>> Yes.  Although when you think sections instead of segments I'm a little
>> worried.
>
>   Well, for my defense, if Gujin do not find the real mode code in the program
>  header it search for it in the section table, if the program header cannot be
>  modified for some reason. There may be some issues if that ELF file is
>  "stripped" but it seems to work on my side.

The program header is for executable loaders the section header is for
linkers and the section header is optional in PT_LOAD and PT_DYN
executables.  Use of the section header by a loader is a bug.

>> >  Also there isn't any more kernel size limit.
>>
>> I think as HPA points out we have gotten past that a long time
>> ago with the bzImage format.
>
>   There was still a test not that long ago (in the code I am not executing) -
> but
>  I was wrong to bring this point forward.

There have been limitations but mostly with respect to page table size
and the like but they were not limitations a bootloader had to deal with.

>> With the right delicacy, and preserving backwards compatibility
>> with existing bootloaders I think we can achieve things.
>
>   Well, you can generate with the proposed patch and boot with SYSLINUX,
>  Grub and LILO, but trying to clean Linux real-mode code is where I begun.

I haven't looked yet.  But I believe this is something that we can do,
so long as cleaning and feature enhancement are not mixed in a bad way.

>> The big issue is that sometimes bootloaders are a little bit brittle.
>> Eric
>
>   I like to think mine is more stable - maybe I am a dreamer...
>   I do not compete on network booting neither.

Phrasing this another way.  What we can do with the interface is
determined our interface to existing bootloaders and what bootloaders
exist.  Basically it is the rule that we need to preserve backwards
compatibility and changes generally need to be evolutionary ones.

Eric

Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3

[Etienne Lorrain]

H. Peter Anvin
> Etienne Lorrain wrote:
> > Also there isn't any more kernel size limit.
> 
> What kernel size limit?

  Sorry - there isn't any limit for the kernel size, for a long time.
  Etienne.


	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com