How vmlinux is recognized?

How vmlinux is recognized?

[Vikram Narayanan]

Hi,

Sorry if this question is stupid. 
How the vmlinux (an ELF executable) is recognized by the processor?
What are the files that are responsible for this?

Thanks,
Vikram

How vmlinux is recognized?

[Dave Hylands]

Hi Vikram,

On Wed, May 11, 2011 at 11:06 AM, Vikram Narayanan <vikram186@gmail.com> wrote:
> Hi,
>
> Sorry if this question is stupid.
> How the vmlinux (an ELF executable) is recognized by the processor?
> What are the files that are responsible for this?

Well the short answer is that it isn't.

The ELF file is normally just one stage of the process. You still need
to extract a binary from the ELF, and the binary contains the raw
executable code that the processor uses.

Normally the boot loader will extract a binary (perhaps from an ELF,
or perhaps from a raw binary image) and this is what the processor
sees.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 12:47 AM, Dave Hylands <dhylands@gmail.com> wrote:
> Hi Vikram,
>
> On Wed, May 11, 2011 at 11:06 AM, Vikram Narayanan <vikram186@gmail.com> wrote:
>> Hi,
>>
>> Sorry if this question is stupid.
>> How the vmlinux (an ELF executable) is recognized by the processor?
>> What are the files that are responsible for this?
>
> Well the short answer is that it isn't.
>
> The ELF file is normally just one stage of the process. You still need
> to extract a binary from the ELF, and the binary contains the raw
> executable code that the processor uses.
>
> Normally the boot loader will extract a binary (perhaps from an ELF,
> or perhaps from a raw binary image) and this is what the processor
> sees.
So in case of x86, say Grub will be taking care of this extraction. Right?
If, so the grub code will have the mechanisms for extracting the raw
binary from ELF.
Am i right?

-
Thanks,
Vikram

How vmlinux is recognized?

[Mulyadi Santosa]

On Thu, May 12, 2011 at 02:31, Vikram Narayanan <vikram186@gmail.com> wrote:
> So in case of x86, say Grub will be taking care of this extraction. Right?
> If, so the grub code will have the mechanisms for extracting the raw
> binary from ELF.
> Am i right?

you mean, vmlinuz right? the bzImage right? not the vmlinux....
because that's the one GRUB handles...not vmlinux one...

well, in that case, see this first:
$ file -k -z  /boot/vmlinuz-2.6.32-31-generic

/boot/vmlinuz-2.6.32-31-generic: Linux kernel x86 boot executable
bzImage, version 2.6.32-31-generic (buildd at rothe, RO-rootFS, root_dev
0x801, swap_dev 0x3, Normal VGA\012- x86 boot sector, code offset 0x5

i am sure you will get idea based upon the above file identification,
on what vmlinuz is and how is it supposed to be treated by boot
loader....
-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 1:15 AM, Mulyadi Santosa
<mulyadi.santosa@gmail.com> wrote:
> On Thu, May 12, 2011 at 02:31, Vikram Narayanan <vikram186@gmail.com> wrote:
>> So in case of x86, say Grub will be taking care of this extraction. Right?
>> If, so the grub code will have the mechanisms for extracting the raw
>> binary from ELF.
>> Am i right?
>
> you mean, vmlinuz right? the bzImage right? not the vmlinux....
> because that's the one GRUB handles...not vmlinux one...

The vmlinux is an ELF binary. right? If so, Who does the unpacking of
raw binary image from that ELF?

> well, in that case, see this first:
> $ file -k -z ?/boot/vmlinuz-2.6.32-31-generic

This is the compressed one. The uncompression is done by the kernel
and not by the grub, If I am not wrong.

> /boot/vmlinuz-2.6.32-31-generic: Linux kernel x86 boot executable
> bzImage, version 2.6.32-31-generic (buildd at rothe, RO-rootFS, root_dev
> 0x801, swap_dev 0x3, Normal VGA\012- x86 boot sector, code offset 0x5
>
> i am sure you will get idea based upon the above file identification,
> on what vmlinuz is and how is it supposed to be treated by boot
> loader....
I am still confused :(

Thanks,
Vikram

How vmlinux is recognized?

[Mulyadi Santosa]

On Thu, May 12, 2011 at 03:04, Vikram Narayanan <vikram186@gmail.com> wrote:
> The vmlinux is an ELF binary. right? If so, Who does the unpacking of
> raw binary image from that ELF?

why do you put concern on vmlinux anyway? boot loader loads vmlinuz,
not vmlinux....
-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 1:39 AM, Mulyadi Santosa
<mulyadi.santosa@gmail.com> wrote:
> On Thu, May 12, 2011 at 03:04, Vikram Narayanan <vikram186@gmail.com> wrote:
>> The vmlinux is an ELF binary. right? If so, Who does the unpacking of
>> raw binary image from that ELF?
>
> why do you put concern on vmlinux anyway? boot loader loads vmlinuz,
> not vmlinux....
Yes. I agree. But how who converts the ELF binary to raw binary so
that the processor understands. Or how is it actually done?

Thanks,
Vikram

How vmlinux is recognized?

[Mulyadi Santosa]

On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
> Yes. I agree. But how who converts the ELF binary to raw binary so
> that the processor understands. Or how is it actually done?

OK I try my best to understand your question :)

i think I got it...you probably guessed that vmlinux created first,
then vmlinuz... AFAIK, it's the other way around...or more precisely,
not both.

After final phase of final kernel image creation, it will go into
making bootable image first. in order to do that, first it will be
compressed 1st. These days, gz is the choice.

So, it is gzipped..and the boot loading code is appended in front of
it... there, you get vmlinuz.

And vmlinux? developers usually use vmlinux as symbol file... and the
way it is created, back to the above phase, is by linking it according
to the accompanying elf linker script. Finally, ELF that contains
kernel is there.

Another guess, maybe you wanna know how to extract the kernel code
from ELF image? then why so? that is indeed the kernel image
itself...it is just appended ELF headers, sections and so on just to
represent ELF construction. But it is not behaving like standart ELF
binary i.e the entry point is not main() but IIRC start_kernel or
something like that.

that helps you?
-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

How vmlinux is recognized?

[Manohar Vanga]

Hi Vikram,

How the vmlinux (an ELF executable) is recognized by the processor?
>

ELF is just a file format. That is, the machine instructions and data are
stored in a specific format. The _processor_ simply recognizes machine
instructions and this needs to be taken from the ELF file and loaded into
memory (the instruction pointer is then pointed to the place the
instructions were loaded).

The format is simply a set of rules defined in the specification (a pretty
nice introduction is available at www.skyfree.org/linux/references/*ELF*
_Format.pdf <http://www.skyfree.org/linux/references/ELF_Format.pdf>). For
example, when you ask a Linux kernel to execute an ELF file, it has code to
know how to decode the information and place it into memory (see
fs/binfmt_elf.c).

As for the vmlinux file specifically, the Wikipedia page on vmlinux (
http://en.wikipedia.org/wiki/Vmlinux) seems like a good start. As shown
above with Linux, GRUB needs to have a way to decode whatever format is
passed to it (bzImage).

The kernel however places the unzipping code into the bzImage itself so that
it is loaded into memory by the bootloader and is then run. This code then
unzips the kernel. This way, GRUB doesn't need to know how to decode ELF
files and the job is left to the kernel code. You can see
arch/x86/boot/Makefile and look for the bzImage target to see what files
constitute the bzImage. I may be wrong about this with regard to newer
kernels so I hope others correct me in this case. Another great explanation
is by Alessandro Rubini at:

http://www.ibiblio.org/oswg/oswg-nightly/oswg/en_US.ISO_8859-1/articles/alessandro-rubini/boot/boot/zimage.html

Hope this helped! :-)

-- 
/manohar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110511/b7913e0b/attachment.html 

How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
<mulyadi.santosa@gmail.com> wrote:
> On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
>> Yes. I agree. But how who converts the ELF binary to raw binary so
>> that the processor understands. Or how is it actually done?
>
> OK I try my best to understand your question :)
>
> i think I got it...you probably guessed that vmlinux created first,
> then vmlinuz... AFAIK, it's the other way around...or more precisely,
> not both.

I think you got it wrong. I will try to put my question more elaborately.
1) The system is on and BIOS code runs. It gives the control to the
boot loader, say GRUB.
2) Grub picks up the kernel from the specific partition. (i.e a
vmlinuz image), which denotes that it is compressed.
3) There are uncompression routines in the kernel itself, If I am not
wrong. So the kernel uncompresses itself.
4) Now the uncompressed thing is the vmlinux image, right?
5) The vmlinux is in ELF format. Correct?
6) If the OS boots and if u try to run an ELF file, the loader knows
how to load that in the RAM. (I mean it knows how to interpret the ELF
format)
7) Coming back to the vmlinux image, Who takes care of the loading activity.?
8) Who recognizes that the image is ELF format and do the necessary
things accordingly.?

Hope I have my question clear now.

-
Thanks,
Vikram

How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 2:03 AM, Manohar Vanga <manohar.vanga@gmail.com> wrote:
> Hi Vikram,
>
>> How the vmlinux (an ELF executable) is recognized by the processor?
>
> ELF is just a file format. That is, the machine instructions and data are
> stored in a specific format. The _processor_ simply recognizes machine
> instructions and this needs to be taken from the ELF file and loaded into
> memory (the instruction pointer is then pointed to the place the
> instructions were loaded).

Hope everyone here got my question wrong. I am aware that ELF is a
format and there will be specific loader for loading ELF files. Please
refer to the previous reply.

Thanks,
Vikram

How vmlinux is recognized?

[Sudheer Divakaran]

Hi Vikram,

On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan <vikram186@gmail.com> wrote:
> On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
> <mulyadi.santosa@gmail.com> wrote:
>> On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
>>> Yes. I agree. But how who converts the ELF binary to raw binary so
>>> that the processor understands. Or how is it actually done?
>>
>> OK I try my best to understand your question :)
>>
>> i think I got it...you probably guessed that vmlinux created first,
>> then vmlinuz... AFAIK, it's the other way around...or more precisely,
>> not both.
>
> I think you got it wrong. I will try to put my question more elaborately.
> 1) The system is on and BIOS code runs. It gives the control to the
> boot loader, say GRUB.
> 2) Grub picks up the kernel from the specific partition. (i.e a
> vmlinuz image), which denotes that it is compressed.
> 3) There are uncompression routines in the kernel itself, If I am not
> wrong. So the kernel uncompresses itself.
> 4) Now the uncompressed thing is the vmlinux image, right?
> 5) The vmlinux is in ELF format. Correct?
> 6) If the OS boots and if u try to run an ELF file, the loader knows
> how to load that in the RAM. (I mean it knows how to interpret the ELF
> format)
> 7) Coming back to the vmlinux image, Who takes care of the loading activity.?
> 8) Who recognizes that the image is ELF format and do the necessary
> things accordingly.?
>
> Hope I have my question clear now.
>



If understand your question correctly, you believe that the
uncompressed kernel is in elf format. correct?. it is in binary
format, so elf interpretation is not required, #5 is wrong.

You can see this by building the kernel using 'make V=1'  and note the
following line in the output,

"arch/x86/boot/tools/build arch/x86/boot/setup.bin
arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage"

means bzImage is made out of two binary files extracted from the elf images.
-- 
Thanks
Sudheer

How vmlinux is recognized?

[Sudheer Divakaran]

On Thu, May 12, 2011 at 10:02 AM, Sudheer Divakaran
<inbox1.sudheer@gmail.com> wrote:
> Hi Vikram,
>
> On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan <vikram186@gmail.com> wrote:
>> On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
>> <mulyadi.santosa@gmail.com> wrote:
>>> On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com> wrote:
>>>> Yes. I agree. But how who converts the ELF binary to raw binary so
>>>> that the processor understands. Or how is it actually done?
>>>
>>> OK I try my best to understand your question :)
>>>
>>> i think I got it...you probably guessed that vmlinux created first,
>>> then vmlinuz... AFAIK, it's the other way around...or more precisely,
>>> not both.
>>
>> I think you got it wrong. I will try to put my question more elaborately.
>> 1) The system is on and BIOS code runs. It gives the control to the
>> boot loader, say GRUB.
>> 2) Grub picks up the kernel from the specific partition. (i.e a
>> vmlinuz image), which denotes that it is compressed.
>> 3) There are uncompression routines in the kernel itself, If I am not
>> wrong. So the kernel uncompresses itself.
>> 4) Now the uncompressed thing is the vmlinux image, right?
>> 5) The vmlinux is in ELF format. Correct?
>> 6) If the OS boots and if u try to run an ELF file, the loader knows
>> how to load that in the RAM. (I mean it knows how to interpret the ELF
>> format)
>> 7) Coming back to the vmlinux image, Who takes care of the loading activity.?
>> 8) Who recognizes that the image is ELF format and do the necessary
>> things accordingly.?
>>
>> Hope I have my question clear now.
>>
>
>
>
> If understand your question correctly, you believe that the
> uncompressed kernel is in elf format. correct?. it is in binary
> format, so elf interpretation is not required, #5 is wrong.
>
> You can see this by building the kernel using 'make V=1' ?and note the
> following line in the output,
>
> "arch/x86/boot/tools/build arch/x86/boot/setup.bin
> arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage"
>
> means bzImage is made out of two binary files extracted from the elf images.

One more info I want to clarify is,  vmlinux.bin mentioned in the
above snippet contains the compressed binary image and some other
routines. Just go through the 'make V=1' output, you can see that the
build process is actually compressing binary file extracted from the
vmlinux elf image, which is again combined with some object files,
creates another elf and again extracts the binary and finally combined
with the setup.bin to create the final bzImage. So, elf interpretation
 doesn't happen on the uncompressed code.

-- 
Thanks
Sudheer

How vmlinux is recognized?

[luca ellero]

On 12/05/2011 8.21, Sudheer Divakaran wrote:
> On Thu, May 12, 2011 at 10:02 AM, Sudheer Divakaran
> <inbox1.sudheer@gmail.com>  wrote:
>> Hi Vikram,
>>
>> On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan<vikram186@gmail.com>  wrote:
>>> On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
>>> <mulyadi.santosa@gmail.com>  wrote:
>>>> On Thu, May 12, 2011 at 03:11, Vikram Narayanan<vikram186@gmail.com>  wrote:
>>>>> Yes. I agree. But how who converts the ELF binary to raw binary so
>>>>> that the processor understands. Or how is it actually done?
>>>>
>>>> OK I try my best to understand your question :)
>>>>
>>>> i think I got it...you probably guessed that vmlinux created first,
>>>> then vmlinuz... AFAIK, it's the other way around...or more precisely,
>>>> not both.
>>>
>>> I think you got it wrong. I will try to put my question more elaborately.
>>> 1) The system is on and BIOS code runs. It gives the control to the
>>> boot loader, say GRUB.
>>> 2) Grub picks up the kernel from the specific partition. (i.e a
>>> vmlinuz image), which denotes that it is compressed.
>>> 3) There are uncompression routines in the kernel itself, If I am not
>>> wrong. So the kernel uncompresses itself.
>>> 4) Now the uncompressed thing is the vmlinux image, right?
>>> 5) The vmlinux is in ELF format. Correct?
>>> 6) If the OS boots and if u try to run an ELF file, the loader knows
>>> how to load that in the RAM. (I mean it knows how to interpret the ELF
>>> format)
>>> 7) Coming back to the vmlinux image, Who takes care of the loading activity.?
>>> 8) Who recognizes that the image is ELF format and do the necessary
>>> things accordingly.?
>>>
>>> Hope I have my question clear now.
>>>
>>
>>
>>
>> If understand your question correctly, you believe that the
>> uncompressed kernel is in elf format. correct?. it is in binary
>> format, so elf interpretation is not required, #5 is wrong.
>>
>> You can see this by building the kernel using 'make V=1'  and note the
>> following line in the output,
>>
>> "arch/x86/boot/tools/build arch/x86/boot/setup.bin
>> arch/x86/boot/vmlinux.bin CURRENT>  arch/x86/boot/bzImage"
>>
>> means bzImage is made out of two binary files extracted from the elf images.
>
> One more info I want to clarify is,  vmlinux.bin mentioned in the
> above snippet contains the compressed binary image and some other
> routines. Just go through the 'make V=1' output, you can see that the
> build process is actually compressing binary file extracted from the
> vmlinux elf image, which is again combined with some object files,
> creates another elf and again extracts the binary and finally combined
> with the setup.bin to create the final bzImage. So, elf interpretation
>   doesn't happen on the uncompressed code.
>


Let's put some order here. The image that almost all bootloaders use is 
arch/x86/boot/bzImage which is made of a setup binary file (executable) 
joined with some compressed code (the real kernel) which is uncompressed 
in memory by the setup binary.

The big suggestion I can give is to check the hidden files which end 
with .cmd. There is one of these for every object created by the 
compilation process. For example there is a file called .bzImage.cmd 
which tell you how bzImage was made:

arch/x86/boot/tools/build -b arch/x86/boot/setup.bin 
arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage

NOTE: I refer to a quite old kernel here, it's likely that the 
compilation process has changed somehow.

You can proceed now in reverse order to find how bzImage was made (if I 
understand correctly that is the one you are interested in).

Here is how is made on my kernel tree:

bzImage:
arch/x86/boot/tools/build -b arch/x86/boot/setup.bin 
arch/x86/boot/vmlinux.bin CURRENT > arch/x86/boot/bzImage

arch/x86/boot/vmlinux.bin:
objcopy  -O binary -R .note -R .comment -S 
arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin

arch/x86/boot/compressed/vmlinux:
ld -m elf_i386   -T arch/x86/boot/compressed/vmlinux_32.lds 
arch/x86/boot/compressed/head_32.o arch/x86/boot/compressed/misc.o 
arch/x86/boot/compressed/piggy.o -o arch/x86/boot/compressed/vmlinux

arch/x86/boot/compressed/piggy.o:
ld -m elf_i386   -r --format binary --oformat elf32-i386 -T 
arch/x86/boot/compressed/vmlinux.scr 
arch/x86/boot/compressed/vmlinux.bin.gz -o arch/x86/boot/compressed/piggy.o

arch/x86/boot/compressed/vmlinux.bin.gz:
gzip -f -9 < arch/x86/boot/compressed/vmlinux.bin > 
arch/x86/boot/compressed/vmlinux.bin.gz

arch/x86/boot/compressed/vmlinux.bin:
objcopy  -O binary -R .note -R .comment -S vmlinux 
arch/x86/boot/compressed/vmlinux.bin

vmlinux:
ld -m elf_i386 --build-id -o vmlinux -T arch/x86/kernel/vmlinux.lds 
arch/x86/kernel/head_32.o arch/x86/kernel/init_task.o  init/built-in.o 
--start-group  usr/built-in.o  arch/x86/kernel/built-in.o 
arch/x86/mm/built-in.o  arch/x86/mach-default/built-in.o 
arch/x86/crypto/built-in.o  arch/x86/vdso/built-in.o  kernel/built-in.o 
  mm/built-in.o  fs/built-in.o  ipc/built-in.o  security/built-in.o 
crypto/built-in.o  block/built-in.o  lib/lib.a  arch/x86/lib/lib.a 
lib/built-in.o  arch/x86/lib/built-in.o  drivers/built-in.o 
sound/built-in.o  arch/x86/math-emu/built-in.o  arch/x86/pci/built-in.o 
  arch/x86/power/built-in.o  net/built-in.o --end-group .tmp_kallsyms2.o

NOTE: there are 2 vmlinux (vmlinux and arch\x86\boot\compressed\vmlinux) 
and 2 vmlinux.bin (arch\x86\boot\compressed\vmlinux.bin and 
arch\x86\boot\vmlinux.bin) which are not the same


Hope this helps
Regards
Luca Ellero

How vmlinux is recognized?

[Mulyadi Santosa]

Hi...

On Thu, May 12, 2011 at 10:32, Vikram Narayanan <vikram186@gmail.com> wrote:
> I think you got it wrong. I will try to put my question more elaborately.
> 1) The system is on and BIOS code runs. It gives the control to the
> boot loader, say GRUB.
> 2) Grub picks up the kernel from the specific partition. (i.e a
> vmlinuz image), which denotes that it is compressed.
> 3) There are uncompression routines in the kernel itself, If I am not
> wrong. So the kernel uncompresses itself.
> 4) Now the uncompressed thing is the vmlinux image, right?

nope... it's a binary....but not ELF...and that's not even named
vmlinux or similar to vmlinux...

> 5) The vmlinux is in ELF format. Correct?
yes.... but see above...


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

How vmlinux is recognized?

[अनुज]

Hi All

On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan <vikram186@gmail.com>wrote:

> On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
> <mulyadi.santosa@gmail.com> wrote:
> > On Thu, May 12, 2011 at 03:11, Vikram Narayanan <vikram186@gmail.com>
> wrote:
> >> Yes. I agree. But how who converts the ELF binary to raw binary so
> >> that the processor understands. Or how is it actually done?
> >
> > OK I try my best to understand your question :)
> >
> > i think I got it...you probably guessed that vmlinux created first,
> > then vmlinuz... AFAIK, it's the other way around...or more precisely,
> > not both.
>
> I think you got it wrong. I will try to put my question more elaborately.
> 1) The system is on and BIOS code runs. It gives the control to the
> boot loader, say GRUB.
> 2) Grub picks up the kernel from the specific partition. (i.e a
> vmlinuz image), which denotes that it is compressed.
> 3) There are uncompression routines in the kernel itself, If I am not
> wrong. So the kernel uncompresses itself.
> 4) Now the uncompressed thing is the vmlinux image, right?
> 5) The vmlinux is in ELF format. Correct?
>

I Guess Yes.

6) If the OS boots and if u try to run an ELF file, the loader knows
> how to load that in the RAM. (I mean it knows how to interpret the ELF
> format)
>

See the multi-boot specification. GRUB is a multi-boot compliant boot loader


> 7) Coming back to the vmlinux image, Who takes care of the loading
> activity.?
>

GRUB

8) Who recognizes that the image is ELF format and do the necessary
> things accordingly.?
>

GRUB

>
> Hope I have my question clear now.
>
> -
> Thanks,
> Vikram
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 
Anuj Aggarwal

 .''`.
: :? :   # apt-get install hakuna-matata
`. `'`
   `-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110512/b902e0b4/attachment-0001.html 

How vmlinux is recognized?

[mindentropy]

> This way, GRUB doesn't need to know how to decode ELF
> files and the job is left to the kernel code. 

GRUB has a elf decoder, but it should have multiboot header.
http://osdev.berlios.de/grub.html#multiboot

Hope this answers your doubt.

How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 1:47 PM, Mulyadi Santosa
<mulyadi.santosa@gmail.com> wrote:
> Hi...
>
> On Thu, May 12, 2011 at 10:32, Vikram Narayanan <vikram186@gmail.com> wrote:
>> I think you got it wrong. I will try to put my question more elaborately.
>> 1) The system is on and BIOS code runs. It gives the control to the
>> boot loader, say GRUB.
>> 2) Grub picks up the kernel from the specific partition. (i.e a
>> vmlinuz image), which denotes that it is compressed.
>> 3) There are uncompression routines in the kernel itself, If I am not
>> wrong. So the kernel uncompresses itself.
>> 4) Now the uncompressed thing is the vmlinux image, right?
>
> nope... it's a binary....but not ELF...and that's not even named
> vmlinux or similar to vmlinux...
>
>> 5) The vmlinux is in ELF format. Correct?
> yes.... but see above...
>
Thanks for all your explanations. So the uncompressed one is _NOT_ an
ELF file, but a raw binary. So it doesn't need any interpretation.
Hope this is right.

So when compiling the kernel, what is the purpose of the other
files(mentioned below)
linux-2.6/vmlinux - ELF executable, not stripped
linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
one which is inside the bzImage)
linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped
linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped

Thanks,
Vikram

How vmlinux is recognized?

[Dave Hylands]

Hi Vikram,

...snip...
> So when compiling the kernel, what is the purpose of the other
> files(mentioned below)
> linux-2.6/vmlinux - ELF executable, not stripped
> linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
> one which is inside the bzImage)
> linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped
> linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped

Take luca's email and start at the bottom working towards the top.

linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file.
A binary is then extracted from this to create
arch/x86/boot/compressed/vmlinux.bin
This binary is then compressed to produce
arch/x86/boot/compressed/vmlinux.bin.gz
This gzipped binary is then converted into an object file (which just
contains the gzipped data) but now we're back to having an ELF file
called arch/x86/boot/compressed/piggy.o
The linker then compiles a decompressor (misc.o) and piggy.o together
to produce arch/x86/boot/compressed/vmlinux (an ELF file).
objcopy is used again to convert this ELF into a binary:
arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin
Finally, the binary is compressed to produce bzImage.

So what you get is a compressed binary which contains a decompressor
and another compressed binary, this inner compressed binary being the
kernel.

GRUB loads bzImage into memory and decompresses it and then executes
the resulting binary.
This binary starts with a decompressor which then decompresses the
kernel, and executes the resulting binary.
This binary may relocate itself (probably depends on the architecture)
to a different spot in memory, and then runs.
The kernel is now running.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

How vmlinux is recognized?

[Vikram Narayanan]

On Fri, May 13, 2011 at 7:12 AM, Dave Hylands <dhylands@gmail.com> wrote:
>
> Hi Vikram,
>
> ...snip...
> > So when compiling the kernel, what is the purpose of the other
> > files(mentioned below)
> > linux-2.6/vmlinux - ELF executable, not stripped
> > linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
> > one which is inside the bzImage)
> > linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped
> > linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped
>
> Take luca's email and start at the bottom working towards the top.
>
> linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file.
> A binary is then extracted from this to create
> arch/x86/boot/compressed/vmlinux.bin
> This binary is then compressed to produce
> arch/x86/boot/compressed/vmlinux.bin.gz
> This gzipped binary is then converted into an object file (which just
> contains the gzipped data) but now we're back to having an ELF file
> called arch/x86/boot/compressed/piggy.o
> The linker then compiles a decompressor (misc.o) and piggy.o together
> to produce arch/x86/boot/compressed/vmlinux (an ELF file).
> objcopy is used again to convert this ELF into a binary:
> arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin
> Finally, the binary is compressed to produce bzImage.
>
> So what you get is a compressed binary which contains a decompressor
> and another compressed binary, this inner compressed binary being the
> kernel.
>
> GRUB loads bzImage into memory and decompresses it and then executes
> the resulting binary.
> This binary starts with a decompressor which then decompresses the
> kernel, and executes the resulting binary.
> This binary may relocate itself (probably depends on the architecture)
> to a different spot in memory, and then runs.
> The kernel is now running.
Thanks for the detailed explanation. Clarified. :)

-
Thanks
Vikram

How vmlinux is recognized?

[Peter Teoh]

I loved this reply.......can I annotate it with references to the linux
kernel sources?

On Fri, May 13, 2011 at 9:42 AM, Dave Hylands <dhylands@gmail.com> wrote:

> Hi Vikram,
>
> ...snip...
> > So when compiling the kernel, what is the purpose of the other
> > files(mentioned below)
> > linux-2.6/vmlinux - ELF executable, not stripped
> > linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
> > one which is inside the bzImage)
> > linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped
> > linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped
>
> Take luca's email and start at the bottom working towards the top.
>
> linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file.
> A binary is then extracted from this to create
> arch/x86/boot/compressed/vmlinux.bin
>

yes:

See ./arch/x86/boot/Makefile


>  This binary is then compressed to produce
> arch/x86/boot/compressed/vmlinux.bin.gz
>

See ./arch/x86/boot/compressed/Makefile


> This gzipped binary is then converted into an object file (which just
> contains the gzipped data) but now we're back to having an ELF file
>

./arch/x86/boot/compressed/mkpiggy.c is compiled into a commandline binary -
mkpiggy which will generate the piggy.o.


> called arch/x86/boot/compressed/piggy.o
> The linker then compiles a decompressor (misc.o) and piggy.o together
>

Yes, the routine is called "decompress_kernel", residing inside
./arch/x86/boot/compressed/misc.c.   And this routine is called
from ./arch/x86/boot/compressed/head_32.S (or head_64.S) and at runtime, the
gzipped data is decompressed and immediately jumped into (perhaps after some
relocation if needed):

/*
 * Do the decompression, and jump to the new kernel..
 */
        leal    z_extract_offset_negative(%ebx), %ebp
                                /* push arguments for decompress_kernel: */
        pushl   %ebp            /* output address */
        pushl   $z_input_len    /* input_len */
        leal    input_data(%ebx), %eax
        pushl   %eax            /* input_data */
        leal    boot_heap(%ebx), %eax
        pushl   %eax            /* heap area */
        pushl   %esi            /* real mode pointer */
        call    decompress_kernel
        addl    $20, %esp


 to produce arch/x86/boot/compressed/vmlinux (an ELF file).
> objcopy is used again to convert this ELF into a binary:
> arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin
> Finally, the binary is compressed to produce bzImage.
>

Inside arch/x86/boot/Makefile:

Creating the vmlinux.bin from vmlinux via objcopy (note that this operation
will throw all relocation information):

$(obj)/vmlinux.bin: $(obj)/compressed/vmlinux FORCE
        $(call if_changed,objcopy)

And then packing together linearly to form the "bzImage" (output from make):

make -f scripts/Makefile.build obj=arch/x86/boot arch/x86/boot/bzImage

make -f scripts/Makefile.build obj=arch/x86/boot/compressed
arch/x86/boot/compressed/vmlinux

arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin
CURRENT > arch/x86/boot/bzImage


> So what you get is a compressed binary which contains a decompressor
> and another compressed binary, this inner compressed binary being the
> kernel.
>
> GRUB loads bzImage into memory and decompresses it and then executes
> the resulting binary.
>

To be more precise, grub will load bzImage and jump into the startup_32
function located in arch/x86/boot/compressed/head_32.S at the following
fixed address (from source code):

/*
 *  head.S contains the 32-bit startup code.
 *
 * NOTE!!! Startup happens at absolute address 0x00001000, which is also
where
 * the page directory will exist. The startup code will be overwritten by
 * the page directory. [According to comments etc elsewhere on a compressed
 * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC]
 *
 * Page 0 is deliberately kept safe, since System Management Mode code in
 * laptops may need to access the BIOS data stored there.  This is also
 * useful for future device drivers that either access the BIOS via VM86
 * mode.
 */

More info:

http://books.google.com/books?id=e8BbHxVhzFAC&pg=PA1224&lpg=PA1224&dq=grub+head_32.S&source=bl&ots=0MSdKwBoM6&sig=2RyEpprl25zueiqi332TQHLIj0E&hl=en&ei=y5vQTY7eBNDNrQeI3bTCCg&sa=X&oi=book_result&ct=result&resnum=3&ved=0CCkQ6AEwAg#v=onepage&q=grub%20head_32.S&f=false


> This binary starts with a decompressor which then decompresses the
> kernel, and executes the resulting binary.
> This binary may relocate itself (probably depends on the architecture)
> to a different spot in memory, and then runs.
> The kernel is now running.
>
> --
> Dave Hylands
> Shuswap, BC, Canada
> http://www.davehylands.com
>
> _______________________________________________
> K
>

-- 
Regards,
Peter Teoh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110516/5393cbf8/attachment-0001.html 

How vmlinux is recognized?

[Vikram Narayanan]

On Mon, May 16, 2011 at 9:14 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
> I loved this reply.......can I annotate it with references to the linux
> kernel sources?
>
> On Fri, May 13, 2011 at 9:42 AM, Dave Hylands <dhylands@gmail.com> wrote:
>>
>> Hi Vikram,
>>
>> ...snip...
>> > So when compiling the kernel, what is the purpose of the other
>> > files(mentioned below)
>> > linux-2.6/vmlinux - ELF executable, not stripped
>> > linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
>> > one which is inside the bzImage)
>> > linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable,
>> > stripped
>> > linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not
>> > stripped
>>
>> Take luca's email and start at the bottom working towards the top.
>>
>> linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file.
>> A binary is then extracted from this to create
>> arch/x86/boot/compressed/vmlinux.bin
>
> yes:
> See ./arch/x86/boot/Makefile
>
>>
>> This binary is then compressed to produce
>> arch/x86/boot/compressed/vmlinux.bin.gz
>
> See ./arch/x86/boot/compressed/Makefile
>
>>
>> This gzipped binary is then converted into an object file (which just
>> contains the gzipped data) but now we're back to having an ELF file
>
> ./arch/x86/boot/compressed/mkpiggy.c is compiled into a commandline binary -
> mkpiggy which will generate the piggy.o.
>
>>
>> called arch/x86/boot/compressed/piggy.o
>> The linker then compiles a decompressor (misc.o) and piggy.o together
>
> Yes, the routine is called "decompress_kernel", residing inside
> ./arch/x86/boot/compressed/misc.c. ? And this routine is called
> from?./arch/x86/boot/compressed/head_32.S (or head_64.S) and at runtime, the
> gzipped data is decompressed and immediately jumped into (perhaps after some
> relocation if needed):
> /*
> ?* Do the decompression, and jump to the new kernel..
> ?*/
> ?? ? ? ?leal ? ?z_extract_offset_negative(%ebx), %ebp
> ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/* push arguments for decompress_kernel: */
> ?? ? ? ?pushl ? %ebp ? ? ? ? ? ?/* output address */
> ?? ? ? ?pushl ? $z_input_len ? ?/* input_len */
> ?? ? ? ?leal ? ?input_data(%ebx), %eax
> ?? ? ? ?pushl ? %eax ? ? ? ? ? ?/* input_data */
> ?? ? ? ?leal ? ?boot_heap(%ebx), %eax
> ?? ? ? ?pushl ? %eax ? ? ? ? ? ?/* heap area */
> ?? ? ? ?pushl ? %esi ? ? ? ? ? ?/* real mode pointer */
> ?? ? ? ?call ? ?decompress_kernel
> ?? ? ? ?addl ? ?$20, %esp
>
>> to produce arch/x86/boot/compressed/vmlinux (an ELF file).
>> objcopy is used again to convert this ELF into a binary:
>> arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin
>> Finally, the binary is compressed to produce bzImage.
>
> Inside arch/x86/boot/Makefile:
> Creating the vmlinux.bin from vmlinux via objcopy (note that this operation
> will throw all relocation information):
> $(obj)/vmlinux.bin: $(obj)/compressed/vmlinux FORCE
> ?? ? ? ?$(call if_changed,objcopy)
> And then packing together linearly to form the "bzImage" (output from make):
> make -f scripts/Makefile.build obj=arch/x86/boot arch/x86/boot/bzImage
> make -f scripts/Makefile.build obj=arch/x86/boot/compressed
> arch/x86/boot/compressed/vmlinux
> arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin
> CURRENT > arch/x86/boot/bzImage
>>
>> So what you get is a compressed binary which contains a decompressor
>> and another compressed binary, this inner compressed binary being the
>> kernel.
>>
>> GRUB loads bzImage into memory and decompresses it and then executes
>> the resulting binary.
>
> To be more precise, grub will load bzImage and jump into the startup_32
> function located in arch/x86/boot/compressed/head_32.S at the following
> fixed address (from source code):
> /*
> ?* ?head.S contains the 32-bit startup code.
> ?*
> ?* NOTE!!! Startup happens at absolute address 0x00001000, which is also
> where
> ?* the page directory will exist. The startup code will be overwritten by
> ?* the page directory. [According to comments etc elsewhere on a compressed
> ?* kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC]
> ?*
> ?* Page 0 is deliberately kept safe, since System Management Mode code in
> ?* laptops may need to access the BIOS data stored there. ?This is also
> ?* useful for future device drivers that either access the BIOS via VM86
> ?* mode.
> ?*/
> More info:
> http://books.google.com/books?id=e8BbHxVhzFAC&pg=PA1224&lpg=PA1224&dq=grub+head_32.S&source=bl&ots=0MSdKwBoM6&sig=2RyEpprl25zueiqi332TQHLIj0E&hl=en&ei=y5vQTY7eBNDNrQeI3bTCCg&sa=X&oi=book_result&ct=result&resnum=3&ved=0CCkQ6AEwAg#v=onepage&q=grub%20head_32.S&f=false
>
>>
>> This binary starts with a decompressor which then decompresses the
>> kernel, and executes the resulting binary.
>> This binary may relocate itself (probably depends on the architecture)
>> to a different spot in memory, and then runs.
>> The kernel is now running.
>>
>> --
>> Dave Hylands
>> Shuswap, BC, Canada
>> http://www.davehylands.com
>>
>> _______________________________________________
>> K
>
> --
> Regards,
> Peter Teoh
>
That was a great explanation. Thanks a lot. I think this will be very
much useful for people who want to know how things work in the
background.

-
Thanks,
Vikram