[Qemu-devel] qemu-user: relocating target code weakness

[Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

Hi!
  I am working on a project based on qemu-user. More exactly it is 
qemu-ppc (version 0.13.0) with x86 host.
All the project and documentation about qemu will be open for everybody 
as it is a project for my university that is a public one..
I have the need to relocate the target code in the memory space to some 
other starting address.
So I went inside linux-user/elfload.c: load_elf_binary and there I found 
many things that according to me are someway buggy or just "weak" ..
Of course I am only a student, and all of this was far beyond my 
knowledge .. I am studying like a crazy for understanding all of this.. 
elf stuff, linking and loading .. and I may be just confused.. 
nevertheless I think it is worth of some attention by some of you gurus 
outta there :)
First of all:
> info->start_mmap = (abi_ulong)ELF_START_MMAP;
What is this? what is start_mmap supposed to point at at the end? Why 
that static value is chosen at the beginning?

Then there is a block of code protected by the define:
> #if defined(CONFIG_USE_GUEST_BASE)
this define appears to be enabled in my code though I do not use any 
special parameter for qemu-ppc.. What is it supposed to do?
I mean I guess it wants to reserve a memory area for the target code, 
starting at a user defined address (guest_base).. and all of that is 
done in linux-user/main.c: main, through a proper command line 
parameter, but then the guest_base address is not more used in the code 
of qemu-user o.O
Inside the code, protected by that define in load_elf_binary, seems to 
check if the system can reserve the requested area of program address 
space, otherwise find a suitable address and set again guest_base to 
that address.. but oddly that variable is never more taken in any 
consideration then!!!
Going down the code I notice that for executable target binaries the 
flag MAP_FIXED for mmap is set.. and in the requested starting address 
the variable load_bias is always zero for executable files, so that the 
starting address is simply the p_vaddr of the various program 
segments..what does it mean? Simple! A target executable code is ALWAYS 
loaded at the same address of the program address space it would be if 
run on a target machine (not on the host machine through qemu)..far 
enough? No for me!!
When executing in the original machine the binary was created for, its 
OS creates an address space only for the target.. that means all the 
address are free for it, and it can be loaded at any address chosen by 
the link editor when creating the binary.. but here in qemu-user it 
shares the address space with qemu-user code itself (and in my case with 
all other I added to it!) so those addresses can be not free at all!!
Isn't that a big weakness??
I went a little forward, though with my poor skills.. I forced a bias, 
created assembly PPC simple test binaries and had a look at what happens 
when a shift of the start address is required (after fixing some 
variables that did not consider fine such possibility):
The target global variables addresses are hard coded into the target 
binary.. nothing in the process of elf loading or then in TCG do 
anything for patching those addresses when there is a shift of the 
starting code.. the result is simply got: segmentation fault when trying 
to access those variables!
  Moreover the way the various variables like start_code, start_data, 
end_data, elf_brk were set was very odd to me:
It was like while cycling through all the program segments they could be 
set for any new segment checked!
Now I must admit, through all my studying of ELF, I did not understand 
yet (I'd appreciate very much if you can tell me it!!!) if it is 
possible to have multiple code segments or multiple data segments in an 
executable binary..
What I did was to check if the code had the PF_X flag or PF_W flag for 
distinguishing whether it was a code segment or a data segment (hoping 
there was only one of each), and set those variables accordingly..
This approach fails anyway when I use PIC code (I am trying to use PIC 
code now as a solution for relocating code), it seems there are many 
code segments and many writable segments, and I can't understand how to 
set properly start_code, end_code.. etc there.
Another thing I had to edit while setting those variable was making them 
consider as address not more p_vaddr, but the actual loading address 
(error) ..
If any of you is interested in solving this problem I can provide all my 
logs for various tests, source code for the test ppc binaries for 
testing everything and of course my elfload.c
Thank you in advance for any support!
Best regards,
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mulyadi Santosa]

Pardon me for any possibly stupid answer...so here we come :)

On Mon, Jan 24, 2011 at 18:34, Stefano Bonifazi
<stefboombastic@gmail.com> wrote:
> First of all:
>>
>> info->start_mmap = (abi_ulong)ELF_START_MMAP;
>
> What is this?

Start address in which address the ELF code section, in reality (not
under qemu-user) should be mapped. in x86 32 bit it's 08048000, you
can check it by yourself by executing:
cat /proc/self/maps

however, as you already know, it might be relocated elsewhere if the
code is PIC or PIE (Position Independent Executable).

I wrote an article about understanding ELF years ago, here is the URL:
http://www.linuxforums.org/articles/understanding-elf-using-readelf-and-objdump_125.html

>what is start_mmap supposed to point at at the end? Why that
> static value is chosen at the beginning?

in qemu 0.13.0, linux-user/elfload.c, lines around 1728 to 1761, you
will find that start_mmap is kinda an address that is the result of
mmap operation that will be filled with the code in code section. It
will be a "hint" on which address the guest code really points when
dealing with memory address.

I dare to guess that, every code executed in the guest...when
referring to virtual address, will be get substracted by the delta of
ELF_START_MMAP and real_base.

 It's like "hey, it's written A in the code, but it's mapped in B, ok
so the delta is A-B, call it X. Then every address in the code should
be substracted (or added, depending on how you see it) with X, then it
will point in qemu mmaped VMA"

See the codes, read it slowly, you wil get the idea.
I am claiming I know it 100%, but I think that's the way it is

PS: IMHO the real guru is still the one and only  Fabrice Bellard, too
bad he's out of qemu.

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

Hi!
Thank you for answering me!

> Start address in which address the ELF code section, in reality (not
> under qemu-user) should be mapped. in x86 32 bit it's 08048000, you
> can check it by yourself by executing:
> cat /proc/self/maps
>
Wait, like that I don't understand anything more.. As far as I 
understood when calling mmap from within a process, its result, when no 
error happen, would be the virtual address where the mapping starts.. so 
if I get 0x10000000 from mmap why should it be 08048000 instead??
Inside load_elf_binary the call to mmap has elf_ppnt->p_vaddr as 
required starting address (plus alignment) and MAP_FIXED flag.. so what 
I get is elf_ppnt->p_vaddr as starting map address (if the call does not 
fail) .. so I do not understand where does that address ELF_START_MMAP 
get any role?
> however, as you already know, it might be relocated elsewhere if the
> code is PIC or PIE (Position Independent Executable).
>
Well PIC target code, inside load_elf_binary would have elf_ex.e_type == 
ET_DYN
so the mmap would be not flagged with MAP_FIXED, and the starting 
address of the mapping would be chosen by the memory manager .. again 
ELF_START_MMAP is not used...
> I wrote an article about understanding ELF years ago, here is the URL:
> http://www.linuxforums.org/articles/understanding-elf-using-readelf-and-objdump_125.html
>
Will you believe me if I tell you I printed that document some hours ago 
without noticing you were the author? :)
I downloaded and read so much material from the Internet for studying 
this stuff!!
>> what is start_mmap supposed to point at at the end? Why that
>> static value is chosen at the beginning?
> in qemu 0.13.0, linux-user/elfload.c, lines around 1728 to 1761, you
> will find that start_mmap is kinda an address that is the result of
> mmap operation that will be filled with the code in code section. It
> will be a "hint" on which address the guest code really points when
> dealing with memory address.
>
No, not for my understanding: that code for me searches for a suitable 
mapping area, I can't see info->start_mmap being set anywhere else.. 
surely not among those lines of code
> I dare to guess that, every code executed in the guest...when
> referring to virtual address, will be get substracted by the delta of
> ELF_START_MMAP and real_base.
>
>   It's like "hey, it's written A in the code, but it's mapped in B, ok
> so the delta is A-B, call it X. Then every address in the code should
> be substracted (or added, depending on how you see it) with X, then it
> will point in qemu mmaped VMA"
>
Wrong, at least for what I did understand, and from the tests I did.. I 
compared different input - output assembly code, hardcoded target binary 
addresses like those of static global variables (not of shared modules) 
are not changed at all, producing segmentation faults when forcing a 
shift of the mapping..
> See the codes, read it slowly, you wil get the idea.
I did, I do, I am changing the code for testing it.. fixing it.. but I 
can't see any relocation
> PS: IMHO the real guru is still the one and only  Fabrice Bellard, too
> bad he's out of qemu.
>
How is it possible that the creator of all of this is out?

Thank you very much for your help!
Best regards!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Richard Henderson]

On 01/24/2011 03:34 AM, Stefano Bonifazi wrote:
> I am working on a project based on qemu-user. More exactly it is
> qemu-ppc (version 0.13.0) with x86 host. All the project and
> documentation about qemu will be open for everybody as it is a
> project for my university that is a public one.. I have the need to
> relocate the target code in the memory space to some other starting
> address. So I went inside linux-user/elfload.c: load_elf_binary and
> there I found many things that according to me are someway buggy or
> just "weak" ..

Yes.  Have a look at 

  http://lists.gnu.org/archive/html/qemu-devel/2010-07/msg01626.html

where I tried to clean this up last year.  The patch never got properly
reviewed, however.

All that said, unless you have an executable that's been properly 
prepared for relocation, e.g. an ET_DYN binary instead of a normal
ET_EXEC binary, you will *not* have enough information to do what
you're suggesting.


r~

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mike Frysinger]

On Mon, Jan 24, 2011 at 13:16, Stefano Bonifazi wrote:
>> Start address in which address the ELF code section, in reality (not
>> under qemu-user) should be mapped. in x86 32 bit it's 08048000, you
>> can check it by yourself by executing:
>> cat /proc/self/maps
>
> Wait, like that I don't understand anything more.. As far as I understood
> when calling mmap from within a process, its result, when no error happen,
> would be the virtual address where the mapping starts.. so if I get
> 0x10000000 from mmap why should it be 08048000 instead??

he is telling you what ELF_START_MMAP is all about.  it is the base
address that the linux kernel for that architecture will start giving
out addresses.  so when running Linux on an x86 system, the first
mmap() a process does will start at 0x80000000 and move up.

although looking at the elfload code quickly, it doesnt seem that this
is really used anymore.  so it probably can be ignored.

> Inside load_elf_binary the call to mmap has elf_ppnt->p_vaddr as required
> starting address (plus alignment) and MAP_FIXED flag.

review the mmap() man page ... MAP_FIXED is always a *suggestion* and
never a requirement.  the app must check the return value to see what
the kernel actually gave it.
-mike

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

Hi!
  Thank you for your answer!

> he is telling you what ELF_START_MMAP is all about.  it is the base
> address that the linux kernel for that architecture will start giving
> out addresses.  so when running Linux on an x86 system, the first
> mmap() a process does will start at 0x80000000 and move up.
>
> although looking at the elfload code quickly, it doesnt seem that this
> is really used anymore.  so it probably can be ignored.
Uhmm that makes sense and explains many things like why they can simply 
consider always possible that the target is allocated at 
elf_ppnt->p_vaddr, without risking it to clash with qemu-user code..

>> Inside load_elf_binary the call to mmap has elf_ppnt->p_vaddr as required
>> starting address (plus alignment) and MAP_FIXED flag.
> review the mmap() man page ... MAP_FIXED is always a *suggestion* and
> never a requirement.  the app must check the return value to see what
> the kernel actually gave it.
Sorry, wrong!

*MAP_FIXED*
               Don't interpret/addr/  as a hint: place the mapping at exactly that
               address. [] If the specified address cannot be used,*mmap*() will fail.[]

http://www.kernel.org/doc/man-pages/online/pages/man2/mmap.2.html

Thank you!
Best regards!
Stefano B.




[-- Attachment #2: Type: text/html, Size: 2122 bytes --]

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mike Frysinger]

On Mon, Jan 24, 2011 at 15:58, Stefano Bonifazi wrote:
>> review the mmap() man page ... MAP_FIXED is always a *suggestion* and
>> never a requirement.  the app must check the return value to see what
>> the kernel actually gave it.
>
> Sorry, wrong!
>
> MAP_FIXED
>               Don't interpret addr as a hint: place the mapping at exactly
> that
>               address. [] If the specified address cannot be used, mmap()
> will fail.[]

err, yes.  i was thinking the specified address when MAP_FIXED isnt
used.  since qemu itself has its own mappings, there is a slightly
greater possibility of the simulated program making a MAP_FIXED
request that clashes with qemu itself, or with addresses that are free
in the simulated arch's ABI but reserved in the host processor's ABI.
but that can happen with the app running natively too, so any app not
handling MAP_FIXED failures is buggy and not qemu's problem.
-mike

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

> Yes.  Have a look at
>
>    http://lists.gnu.org/archive/html/qemu-devel/2010-07/msg01626.html
>
> where I tried to clean this up last year.  The patch never got properly
> reviewed, however.
>
> All that said, unless you have an executable that's been properly
> prepared for relocation, e.g. an ET_DYN binary instead of a normal
> ET_EXEC binary, you will *not* have enough information to do what
> you're suggesting.
>
>
> r~
Wow wonderful! So you fixed the code for PIC (ET_DYN) support? I won't 
have any problem in producing the target code into pie format, as I am 
writing, compiling and linking my target code by my own ;)
A noob question, how can I get your sources? Is there a simpler solution 
than "copy&paste" all the code from your messages into patches and then 
applying them? Can you just send your sources by email? Or can I 
download them from a site?
I am very eager to study your code and to try it :)
Thank you very much!!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

> but that can happen with the app running natively too, so any app not
> handling MAP_FIXED failures is buggy and not qemu's problem.
> -mike
How? For what I learned each process executing on a OS with an mmu sees 
its virtual address space, and normally only its code is loaded there 
(well I am learning that the dynamic linker also inject into that space 
shared library code used by the process code, but of course a good 
dynamic linker would prevent clashes!) so how can it happen that it can 
clash with anything??
If I remember fine, at the time of DOS there were some addresses 
reserved for the OS, and then only one executable could run at time, but 
in modern time with virtual addressing I don't think it is possible 
other than in scenarios alike ours..
Regards,
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mike Frysinger]

On Mon, Jan 24, 2011 at 16:52, Stefano Bonifazi wrote:
>> but that can happen with the app running natively too, so any app not
>> handling MAP_FIXED failures is buggy and not qemu's problem.
>
> How? For what I learned each process executing on a OS with an mmu sees its
> virtual address space, and normally only its code is loaded there (well I am
> learning that the dynamic linker also inject into that space shared library
> code used by the process code, but of course a good dynamic linker would
> prevent clashes!)

how can the ldso possibly prevent clashes ?  it has no idea what
addresses an app will ask for at runtime.

plus, if the kernel is employing ASLR (which isnt uncommon nowadays),
the load addresses could be anywhere.
-mike

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

> how can the ldso possibly prevent clashes ?  it has no idea what
> addresses an app will ask for at runtime.
>
> plus, if the kernel is employing ASLR (which isnt uncommon nowadays),
> the load addresses could be anywhere.
> -mike
Well not alone, in my mind ld.so asks the memory manager through calling 
mmap where it can map the shared modules inside the process address 
space.. the memory manager should know what addresses are free and which 
are taken .. then when the process code requires dynamic memory 
allocation, it does it again through the memory manager (i.e. malloc) 
avoiding of allocating memory where shared modules have been loaded into..
Again with ASLR I think the mmu should be aware of all used memory 
slots, avoiding conflicts..
Well your view of random possible clashes maybe is what happens or 
happened in Windows systems :D But Linux is supposed to be different, 
isn't it?
Best regards,
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mike Frysinger]

On Mon, Jan 24, 2011 at 17:24, Stefano Bonifazi wrote:
>> how can the ldso possibly prevent clashes ?  it has no idea what
>> addresses an app will ask for at runtime.
>>
>> plus, if the kernel is employing ASLR (which isnt uncommon nowadays),
>> the load addresses could be anywhere.
>
> Well not alone, in my mind ld.so asks the memory manager through calling
> mmap where it can map the shared modules inside the process address space..
> the memory manager should know what addresses are free and which are taken
> .. then when the process code requires dynamic memory allocation, it does it
> again through the memory manager (i.e. malloc) avoiding of allocating memory
> where shared modules have been loaded into..
> Again with ASLR I think the mmu should be aware of all used memory slots,
> avoiding conflicts..

i'm not saying conflicts arent noticed.  i'm saying that there is no
way for the surrounding system (ldso/linux mm/...) to know ahead of
time what the app is going to request with MAP_FIXED and thus prevent
possible future conflicts.
-mike

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mike Frysinger]

On Mon, Jan 24, 2011 at 16:44, Stefano Bonifazi wrote:
>>   http://lists.gnu.org/archive/html/qemu-devel/2010-07/msg01626.html
>
> A noob question, how can I get your sources? Is there a simpler solution
> than "copy&paste" all the code from your messages into patches and then
> applying them? Can you just send your sources by email? Or can I download
> them from a site?

weird ... usually the gnu archives have a "download raw text" option
so you can get a mbox file to import into your mailer

you could use the gmane.org nntp gateway to get all the raw e-mails though:
    nntp://news.gmane.org/gmane.comp.emulators.qemu
-mike

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mike Frysinger]

On Mon, Jan 24, 2011 at 11:29, Mulyadi Santosa wrote:
> I wrote an article about understanding ELF years ago, here is the URL:
> http://www.linuxforums.org/articles/understanding-elf-using-readelf-and-objdump_125.html

i also highly recommend Linkers & Loaders:
    http://linker.iecc.com/

some pieces might be a little dated, but buy & large, it's still relevant
-mike

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mike Frysinger]

On Mon, Jan 24, 2011 at 19:06, Mike Frysinger wrote:
> On Mon, Jan 24, 2011 at 11:29, Mulyadi Santosa wrote:
>> I wrote an article about understanding ELF years ago, here is the URL:
>> http://www.linuxforums.org/articles/understanding-elf-using-readelf-and-objdump_125.html
>
> i also highly recommend Linkers & Loaders:
>    http://linker.iecc.com/
>
> some pieces might be a little dated, but buy & large, it's still relevant

Ian Lance Taylor (author of the GOLD linker) maintains a blog where he
posts well written and in-depth articles on various toolchain pieces:
http://www.airs.com/blog/archives/category/programming

this one talks about the GOT/PLT for ELF on a MMU:
http://www.airs.com/blog/archives/41
-mike

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Richard Henderson]

On 01/24/2011 01:44 PM, Stefano Bonifazi wrote:
> Wow wonderful! So you fixed the code for PIC (ET_DYN) support? 

Yes.

> how can I get your sources?

I was mistaken -- a later version of the patch set was in fact merged.
I simply forgot to delete my working branch afterward.


r~

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 12:32 AM, Mike Frysinger wrote:
> On Mon, Jan 24, 2011 at 16:44, Stefano Bonifazi wrote:
>>>    http://lists.gnu.org/archive/html/qemu-devel/2010-07/msg01626.html
>> A noob question, how can I get your sources? Is there a simpler solution
>> than "copy&paste" all the code from your messages into patches and then
>> applying them? Can you just send your sources by email? Or can I download
>> them from a site?
> weird ... usually the gnu archives have a "download raw text" option
> so you can get a mbox file to import into your mailer
>
> you could use the gmane.org nntp gateway to get all the raw e-mails though:
>      nntp://news.gmane.org/gmane.comp.emulators.qemu
> -mike
Hi!
I didn't mean the messages sources, I am already using nntp with 
thunderbird, I meant whether there is an option for getting the .c, .h 
sources, instead of copying the patches and apply them .. sorry but I am 
really new with this
Thank you!
Regards!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 01:06 AM, Mike Frysinger wrote:
> On Mon, Jan 24, 2011 at 11:29, Mulyadi Santosa wrote:
>> I wrote an article about understanding ELF years ago, here is the URL:
>> http://www.linuxforums.org/articles/understanding-elf-using-readelf-and-objdump_125.html
> i also highly recommend Linkers&  Loaders:
>      http://linker.iecc.com/
>
> some pieces might be a little dated, but buy&  large, it's still relevant
> -mike
Thank you for this suggestion! Of course it was also one of the first I got:
> http://www.iecc.com/linker/
Here I got it for free!
Best regards!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 02:36 AM, Richard Henderson wrote:
> On 01/24/2011 01:44 PM, Stefano Bonifazi wrote:
>> Wow wonderful! So you fixed the code for PIC (ET_DYN) support?
> Yes.
>
>> how can I get your sources?
> I was mistaken -- a later version of the patch set was in fact merged.
> I simply forgot to delete my working branch afterward.
>
>
> r~
Sorry.. merged with qemu? What version? I have qemu.0.13.0 and there are 
no your fixes..
How can I get the fixed qemu sources?

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 01:18 AM, Mike Frysinger wrote:
> On Mon, Jan 24, 2011 at 19:06, Mike Frysinger wrote:
>> On Mon, Jan 24, 2011 at 11:29, Mulyadi Santosa wrote:
>>> I wrote an article about understanding ELF years ago, here is the URL:
>>> http://www.linuxforums.org/articles/understanding-elf-using-readelf-and-objdump_125.html
>> i also highly recommend Linkers&  Loaders:
>>     http://linker.iecc.com/
>>
>> some pieces might be a little dated, but buy&  large, it's still relevant
> Ian Lance Taylor (author of the GOLD linker) maintains a blog where he
> posts well written and in-depth articles on various toolchain pieces:
> http://www.airs.com/blog/archives/category/programming
>
> this one talks about the GOT/PLT for ELF on a MMU:
> http://www.airs.com/blog/archives/41
> -mike
Great! Thank you!
Good that I ordered new ink for my printer :)
Best regards!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Mike Frysinger]

On Tue, Jan 25, 2011 at 03:47, Stefano Bonifazi wrote:
> On 01/25/2011 02:36 AM, Richard Henderson wrote:
>> On 01/24/2011 01:44 PM, Stefano Bonifazi wrote:
>>> Wow wonderful! So you fixed the code for PIC (ET_DYN) support?
>>
>> Yes.
>>
>>> how can I get your sources?
>>
>> I was mistaken -- a later version of the patch set was in fact merged.
>> I simply forgot to delete my working branch afterward.
>
> Sorry.. merged with qemu? What version? I have qemu.0.13.0 and there are no
> your fixes..
> How can I get the fixed qemu sources?

you probably want to use the latest git tree
http://git.qemu.org/qemu.git/
-mike

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

>> Sorry.. merged with qemu? What version? I have qemu.0.13.0 and there are no
>> your fixes..
>> How can I get the fixed qemu sources?
> you probably want to use the latest git tree
> http://git.qemu.org/qemu.git/
> -mike
Wow man! I got your work through the git!
Very good job! Now everything is much clearer there!
I am happy to see that you did many edits I also did without knowing 
your work :)
Surely you had the sureness of touch doing everything new.. Instead I 
was always afraid to destroy what worked before, so I moved very slowly :[
Anyway thank you again!
If something keeps being wrong I know who to ask for help to! :)
Best regards!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 09:53 AM, Mike Frysinger wrote:
> On Tue, Jan 25, 2011 at 03:47, Stefano Bonifazi wrote:
>> On 01/25/2011 02:36 AM, Richard Henderson wrote:
>>> On 01/24/2011 01:44 PM, Stefano Bonifazi wrote:
>>>> Wow wonderful! So you fixed the code for PIC (ET_DYN) support?
>>> Yes.
>>>
>>>> how can I get your sources?
>>> I was mistaken -- a later version of the patch set was in fact merged.
>>> I simply forgot to delete my working branch afterward.
>> Sorry.. merged with qemu? What version? I have qemu.0.13.0 and there are no
>> your fixes..
>> How can I get the fixed qemu sources?
> you probably want to use the latest git tree
> http://git.qemu.org/qemu.git/
> -mike
Hi!
  I think there is still a "bug" I corrected:
You keep getting the name of the dynamic linker from the PT_INTERP 
program segment, but that gives you the absolute position of the dynamic 
linker inside the machine the target binary was created in.
Here qemu-user is an emultaor used for running that binary into another 
machine..
Qemu default to usr/gnemul/ the path where the libs for the target 
machines are stored...
So we need to patch the absolute dynamic linker name with the proper path..
I dunno maybe this can be my first little contribute to qemu family ;)
Regards,
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 09:53 AM, Mike Frysinger wrote:
> On Tue, Jan 25, 2011 at 03:47, Stefano Bonifazi wrote:
>> On 01/25/2011 02:36 AM, Richard Henderson wrote:
>>> On 01/24/2011 01:44 PM, Stefano Bonifazi wrote:
>>>> Wow wonderful! So you fixed the code for PIC (ET_DYN) support?
>>> Yes.
>>>
>>>> how can I get your sources?
>>> I was mistaken -- a later version of the patch set was in fact merged.
>>> I simply forgot to delete my working branch afterward.
>> Sorry.. merged with qemu? What version? I have qemu.0.13.0 and there are no
>> your fixes..
>> How can I get the fixed qemu sources?
> you probably want to use the latest git tree
> http://git.qemu.org/qemu.git/
> -mike
Hi again! :)
  debugging a test pie code using qemu-ppc with your (and my little one) 
fixes I got this:
> start_brk   0x00000000
> end_code    0x400102e0
> start_code  0x40000000
> start_data  0x4001024c
> end_data    0x400102e0
> start_stack 0x40811438
> brk         0x400102e4
> entry       0x40828c24
that is start_brk is 0
As far as I understood brk is the .bss section, that is unitialized data 
area, am I right?
If not pls could you be so kind to explain me what it is, or pointing me 
to some document where it is explained.. among all material I am 
studying about linkers, relocations, I never found brk other than in qemu..
Once i am sure what it is I'll try to fix (if needed.. for me it is too 
strange it is zero) it..
Thank you again!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Richard Henderson]

On 01/25/2011 02:47 AM, Stefano Bonifazi wrote:
> You keep getting the name of the dynamic linker from the PT_INTERP program segment, but that gives you the absolute position of the dynamic linker inside the machine the target binary was created in.
> Here qemu-user is an emultaor used for running that binary into another machine..
> Qemu default to usr/gnemul/ the path where the libs for the target machines are stored...
> So we need to patch the absolute dynamic linker name with the proper path..

In load_elf_interp,

    fd = open(path(filename), O_RDONLY);

Notice the path function call.  That does the translation into gnemul, given
the proper configure option, or -L command-line option.


r~

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Richard Henderson]

On 01/25/2011 03:06 AM, Stefano Bonifazi wrote:
>> start_brk   0x00000000
>> end_code    0x400102e0
>> start_code  0x40000000
>> start_data  0x4001024c
>> end_data    0x400102e0
>> start_stack 0x40811438
>> brk         0x400102e4
>> entry       0x40828c24
> that is start_brk is 0
> As far as I understood brk is the .bss section, that is unitialized data area, am I right?

Not quite.  It's normally the beginning of the heap, after the bss section.

That said, it looks like start_brk is a dead variable.  It's written only by the
FLAT loader, and nothing at all reads it, except for this debugging dump.

The "real" value is the "brk" variable, which is indeed set to a plausible looking value.


r~

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 05:22 PM, Richard Henderson wrote:
> On 01/25/2011 02:47 AM, Stefano Bonifazi wrote:
>> You keep getting the name of the dynamic linker from the PT_INTERP program segment, but that gives you the absolute position of the dynamic linker inside the machine the target binary was created in.
>> Here qemu-user is an emultaor used for running that binary into another machine..
>> Qemu default to usr/gnemul/ the path where the libs for the target machines are stored...
>> So we need to patch the absolute dynamic linker name with the proper path..
> In load_elf_interp,
>
>      fd = open(path(filename), O_RDONLY);
>
> Notice the path function call.  That does the translation into gnemul, given
> the proper configure option, or -L command-line option.
>
>
> r~
Hi!
  Strange, I have all the target libs in the default host usr/gnemul 
folder, but your fixed qemu still complained for not finding ld.so.1 
until I fixed the code..
I'll try to check why "path" doesn't work!
Thank you!
Best regards!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 05:26 PM, Richard Henderson wrote:
> On 01/25/2011 03:06 AM, Stefano Bonifazi wrote:
>>> start_brk   0x00000000
>>> end_code    0x400102e0
>>> start_code  0x40000000
>>> start_data  0x4001024c
>>> end_data    0x400102e0
>>> start_stack 0x40811438
>>> brk         0x400102e4
>>> entry       0x40828c24
>> that is start_brk is 0
>> As far as I understood brk is the .bss section, that is unitialized data area, am I right?
> Not quite.  It's normally the beginning of the heap, after the bss section.
>
> That said, it looks like start_brk is a dead variable.  It's written only by the
> FLAT loader, and nothing at all reads it, except for this debugging dump.
>
> The "real" value is the "brk" variable, which is indeed set to a plausible looking value.
>
>
> r~
Thank you!! I really missed that.. I've always wondered where the heap was!
So brk and start_brk are the same .. the latter just being used 
previously and now forgotten there?
Is there a heap end address? Is it possible to set it someway? It would 
be really very helpful for me!!
Thank you again!
Best regards!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Lluís]

Stefano Bonifazi writes:

> Is there a heap end address? Is it possible to set it someway? It
> would be really very helpful for me!!

See the man page for brk(2):

    "Calling sbrk() with an increment of 0 can be used to find the
     current location of the program break."


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/25/2011 02:36 AM, Richard Henderson wrote:
> On 01/24/2011 01:44 PM, Stefano Bonifazi wrote:
>> Wow wonderful! So you fixed the code for PIC (ET_DYN) support?
> Yes.
>
>> how can I get your sources?
> I was mistaken -- a later version of the patch set was in fact merged.
> I simply forgot to delete my working branch afterward.
>
>
> r~
Hi!
  I tested succesfully the sources with your fixes (though it remains 
the little problem of ld.so.1 path)!
Even my final goal of having more than one instance of qemu-user running 
in the same address space worked fine using pie code after your fixes!
Man I really own you a big thank you! :)
Though I was on the right way, I'd surely take very long for doing the 
same fixes, and I have got a very short deadline!
You almost saved me! :)
I dunno where you are from, but if you happen to visit Rome you have a 
dinner paid! :D
Again many thanks and best regards!
Stefano B.

P.S. Please just answer that last question, whether it is possible to 
have a variable showing the upper bound of heap (some brk_end) for a 
target process

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Richard Henderson]

On 01/26/2011 03:07 AM, Stefano Bonifazi wrote:
> P.S. Please just answer that last question, whether it is possible to
> have a variable showing the upper bound of heap (some brk_end) for a
> target process

No, the heap grows until it reaches some other memory mapped entity.


r~

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/26/2011 04:38 PM, Richard Henderson wrote:
> On 01/26/2011 03:07 AM, Stefano Bonifazi wrote:
>> P.S. Please just answer that last question, whether it is possible to
>> have a variable showing the upper bound of heap (some brk_end) for a
>> target process
> No, the heap grows until it reaches some other memory mapped entity.
>
>
> r~
Thank you again! :)
Best regards!
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Lluís]

Richard Henderson writes:

> On 01/26/2011 03:07 AM, Stefano Bonifazi wrote:
>> P.S. Please just answer that last question, whether it is possible to
>> have a variable showing the upper bound of heap (some brk_end) for a
>> target process

> No, the heap grows until it reaches some other memory mapped entity.

>From man brk(2) :

     "sbrk() increments the program's data space by increment bytes.
      Calling sbrk() with an increment of 0 can be used to find the
      current location of the program break."

I already sent this to the list in a previous mail, but it seems you
overlooked it as you were not an explicit recipient.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Richard Henderson]

On 01/26/2011 12:17 PM, Lluís wrote:
> Richard Henderson writes:
> 
>> On 01/26/2011 03:07 AM, Stefano Bonifazi wrote:
>>> P.S. Please just answer that last question, whether it is possible to
>>> have a variable showing the upper bound of heap (some brk_end) for a
>>> target process
> 
>> No, the heap grows until it reaches some other memory mapped entity.
> 
>>From man brk(2) :
> 
>      "sbrk() increments the program's data space by increment bytes.
>       Calling sbrk() with an increment of 0 can be used to find the
>       current location of the program break."
> 
> I already sent this to the list in a previous mail, but it seems you
> overlooked it as you were not an explicit recipient.

That's the current top of the heap.  I answered the question as if
it was asking for the maximum top of the heap.


r~

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Stefano Bonifazi]

On 01/26/2011 09:19 PM, Richard Henderson wrote:
> On 01/26/2011 12:17 PM, Lluís wrote:
>> Richard Henderson writes:
>>
>>> On 01/26/2011 03:07 AM, Stefano Bonifazi wrote:
>>>> P.S. Please just answer that last question, whether it is possible to
>>>> have a variable showing the upper bound of heap (some brk_end) for a
>>>> target process
>>> No, the heap grows until it reaches some other memory mapped entity.
>> > From man brk(2) :
>>
>>       "sbrk() increments the program's data space by increment bytes.
>>        Calling sbrk() with an increment of 0 can be used to find the
>>        current location of the program break."
>>
>> I already sent this to the list in a previous mail, but it seems you
>> overlooked it as you were not an explicit recipient.
> That's the current top of the heap.  I answered the question as if
> it was asking for the maximum top of the heap.
>
>
> r~
Yup!
I am concerned what can be the highest address used as heap by a target 
binary, because I am creating multiple instances of qemu-user, and I 
wanna prevent the target process images and heaps to overlap..
Good enough to know the mmu will stop giving memory when meeting another 
memory mapped region..
Moreover I think I'd have to call sbrk from within the target binary for 
resizing its heap, whereas I wanna control it from qemu-user that is 
loading, then translating and executing it..
@Luis: Yup sorry for not answering the previous post of yours, I thought 
I'd receive emails always when somebody answered a post I created at the 
beginning!

Thank you!
Regards,
Stefano B.

Re: [Qemu-devel] qemu-user: relocating target code weakness

[Lluís]

Richard Henderson writes:

> On 01/26/2011 12:17 PM, Lluís wrote:
>> Richard Henderson writes:
>> 
>>> On 01/26/2011 03:07 AM, Stefano Bonifazi wrote:
>>>> P.S. Please just answer that last question, whether it is possible to
>>>> have a variable showing the upper bound of heap (some brk_end) for a
>>>> target process
>> 
>>> No, the heap grows until it reaches some other memory mapped entity.
>> 
>>> From man brk(2) :
>> 
>> "sbrk() increments the program's data space by increment bytes.
>> Calling sbrk() with an increment of 0 can be used to find the
>> current location of the program break."
>> 
>> I already sent this to the list in a previous mail, but it seems you
>> overlooked it as you were not an explicit recipient.

> That's the current top of the heap.  I answered the question as if
> it was asking for the maximum top of the heap.

Ah! I didn't understand it like that, sorry.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth