[Qemu-devel] Fwd: Re: QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Qemu-devel] Fwd: Re: QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

[-- Attachment #1: Type: text/plain, Size: 834 bytes --]




-------- Перенаправленное сообщение --------
Тема: 	Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of 
loop-programs
Дата: 	Thu, 03 Sep 2015 18:22:16 +0300
От: 	Sergey Smolov <smolov@ispras.ru>
Кому: 	Richard Henderson <rth@twiddle.net>



Richard Henderson писал 2015-09-03 17:28:
> On 09/02/2015 11:33 PM, Peter Maydell wrote:
>> You might find the 'exec' logging helpful for tracking
>> which translated blocks get executed.
>
> Even then you will of course only log the head of unlinked chains of
> blocks.
>
>
> r~

Do you think it is possible to implement another QEMU logger which will
make a record for every executed block, and these records will be in the
form of input assembler (in my case, Aarch64)?

Thanks,

Sergey Smolov




[-- Attachment #2: Type: text/html, Size: 1978 bytes --]

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Peter Maydell]

On 3 September 2015 at 15:31, Sergey Smolov <smolov@ispras.ru> wrote:
> Do you think it is possible to implement another QEMU logger which will
> make a record for every executed block,

Yes (this would just need to disable the TB linking optimisation,
which we've discussed providing a debug toggle for in another
thread).

>  and these records will be in the
> form of input assembler (in my case, Aarch64)?

No. At execution time we don't have the input assembler.
You need to match up the TB addresses in the execution
logs with the translation logs to find the input asm.

thanks
-- PMM

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

03.09.2015 19:35, Peter Maydell пишет:
> On 3 September 2015 at 15:31, Sergey Smolov <smolov@ispras.ru> wrote:
>> Do you think it is possible to implement another QEMU logger which will
>> make a record for every executed block,
> Yes (this would just need to disable the TB linking optimisation,
> which we've discussed providing a debug toggle for in another
> thread).

Sorry, are you talking about this patch: 
https://lists.gnu.org/archive/html/qemu-devel/2014-03/msg05603.html?

Will this be added to QEMU master in the nearest future?

Thanks,
Sergey Smolov

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

>
> 03.09.2015 19:35, Peter Maydell пишет:
>> On 3 September 2015 at 15:31, Sergey Smolov <smolov@ispras.ru> wrote:
>>> Do you think it is possible to implement another QEMU logger which will
>>> make a record for every executed block,
>> Yes (this would just need to disable the TB linking optimisation,
>> which we've discussed providing a debug toggle for in another
>> thread).

Ok, I've implemented a mapping between disassembled forms of 
instructions and executed TBs.
Now my logger does "loop unrolling" successfully.

Thank you very much for your help!

Sergey Smolov

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Christopher Covington]

Hi Sergey,

On 09/04/2015 12:38 PM, Sergey Smolov wrote:
> 
>>
>> 03.09.2015 19:35, Peter Maydell пишет:
>>> On 3 September 2015 at 15:31, Sergey Smolov <smolov@ispras.ru> wrote:
>>>> Do you think it is possible to implement another QEMU logger which will
>>>> make a record for every executed block,
>>> Yes (this would just need to disable the TB linking optimisation,
>>> which we've discussed providing a debug toggle for in another
>>> thread).
> 
> Ok, I've implemented a mapping between disassembled forms of instructions and
> executed TBs.
> Now my logger does "loop unrolling" successfully.

This sounds like it solves the same issue as -d nochain but in what's probably
a more time efficient manner. Are you able to share your implementation?

Thanks,
Christopher Covington

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

Hi Christopher,

18.09.2015 02:02, Christopher Covington пишет:
> Hi Sergey,
>
> On 09/04/2015 12:38 PM, Sergey Smolov wrote:
>>> 03.09.2015 19:35, Peter Maydell пишет:
>>>> On 3 September 2015 at 15:31, Sergey Smolov <smolov@ispras.ru> wrote:
>>>>> Do you think it is possible to implement another QEMU logger which will
>>>>> make a record for every executed block,
>>>> Yes (this would just need to disable the TB linking optimisation,
>>>> which we've discussed providing a debug toggle for in another
>>>> thread).
>> Ok, I've implemented a mapping between disassembled forms of instructions and
>> executed TBs.
>> Now my logger does "loop unrolling" successfully.
> This sounds like it solves the same issue as -d nochain but in what's probably
> a more time efficient manner. Are you able to share your implementation?
>
> Thanks,
> Christopher Covington
>

In which way should I share it? Am I need to create a patch and send it 
to mailing list?

Sergey Smolov

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Christopher Covington]

On 09/18/2015 04:15 AM, Sergey Smolov wrote:
> Hi Christopher,
> 
> 18.09.2015 02:02, Christopher Covington пишет:
>> Hi Sergey,
>>
>> On 09/04/2015 12:38 PM, Sergey Smolov wrote:
>>>> 03.09.2015 19:35, Peter Maydell пишет:
>>>>> On 3 September 2015 at 15:31, Sergey Smolov <smolov@ispras.ru> wrote:
>>>>>> Do you think it is possible to implement another QEMU logger which will
>>>>>> make a record for every executed block,
>>>>> Yes (this would just need to disable the TB linking optimisation,
>>>>> which we've discussed providing a debug toggle for in another
>>>>> thread).
>>> Ok, I've implemented a mapping between disassembled forms of instructions and
>>> executed TBs.
>>> Now my logger does "loop unrolling" successfully.
>> This sounds like it solves the same issue as -d nochain but in what's probably
>> a more time efficient manner. Are you able to share your implementation?

> In which way should I share it? Am I need to create a patch and send it to
> mailing list?

That would be ideal. If you're not familiar with the process, just let me know
and I'd be happy to help.

Thanks,
Christopher Covington

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

Hi Christopher,

I've send my patch to the mailing list. Sorry for the great delay in 
answers.


18.09.2015 18:26, Christopher Covington пишет:
> On 09/18/2015 04:15 AM, Sergey Smolov wrote:
>> Hi Christopher,
>>
>> 18.09.2015 02:02, Christopher Covington пишет:
>>> Hi Sergey,
>>>
>>> On 09/04/2015 12:38 PM, Sergey Smolov wrote:
>>>>> 03.09.2015 19:35, Peter Maydell пишет:
>>>>>> On 3 September 2015 at 15:31, Sergey Smolov <smolov@ispras.ru> wrote:
>>>>>>> Do you think it is possible to implement another QEMU logger which will
>>>>>>> make a record for every executed block,
>>>>>> Yes (this would just need to disable the TB linking optimisation,
>>>>>> which we've discussed providing a debug toggle for in another
>>>>>> thread).
>>>> Ok, I've implemented a mapping between disassembled forms of instructions and
>>>> executed TBs.
>>>> Now my logger does "loop unrolling" successfully.
>>> This sounds like it solves the same issue as -d nochain but in what's probably
>>> a more time efficient manner. Are you able to share your implementation?
>> In which way should I share it? Am I need to create a patch and send it to
>> mailing list?
> That would be ideal. If you're not familiar with the process, just let me know
> and I'd be happy to help.
>
> Thanks,
> Christopher Covington
>

[Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

Hello, List!

I've found that while running Aarch64 assembler programs on QEMU with 
"-d in_asm" option enabled I receive not complete logs. For example, on 
the following assembler program which contains a loop :

[assembler]
.globl _start
bl _start
_start:
label1:
   movz  x0, 10
   subs x0, x0, 1
   b.eq label2
   b label1
label2:
hlt #57005
[/assembler]

I receive the following log which does not include "loop unrolling":
[log]
----------------
IN:
0x0000000000000000:  94000001      bl #+0x4 (addr 0x4)

----------------
IN:
0x0000000000000004:  d2800140      mov x0, #0xa

----------------
IN:
0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)

----------------
IN:
0x000000000000000c:  54000040      b.eq #+0x8 (addr 0x14)

----------------
IN:
0x0000000000000010:  17fffffd      b #-0xc (addr 0x4)


[/log]

It seems that QEMU skips some internal instructions when generates 
"in_asm" log. How to eliminate this?

Thanks in advance!

Sincerely yours,
Sergey Smolov

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

02.09.2015 16:55, Sergey Smolov пишет:
> Hello, List!
>
> I've found that while running Aarch64 assembler programs on QEMU with 
> "-d in_asm" option enabled I receive not complete logs. For example, 
> on the following assembler program which contains a loop :
>
> [assembler]
> .globl _start
> bl _start
> _start:
> label1:
>   movz  x0, 10
>   subs x0, x0, 1
>   b.eq label2
>   b label1
> label2:
> hlt #57005
> [/assembler]
>
> I receive the following log which does not include "loop unrolling":
> [log]
> ----------------
> IN:
> 0x0000000000000000:  94000001      bl #+0x4 (addr 0x4)
>
> ----------------
> IN:
> 0x0000000000000004:  d2800140      mov x0, #0xa
>
> ----------------
> IN:
> 0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)
>
> ----------------
> IN:
> 0x000000000000000c:  54000040      b.eq #+0x8 (addr 0x14)
>
> ----------------
> IN:
> 0x0000000000000010:  17fffffd      b #-0xc (addr 0x4)
>
>
> [/log]
>
> It seems that QEMU skips some internal instructions when generates 
> "in_asm" log. How to eliminate this?
>
> Thanks in advance!
>
> Sincerely yours,
> Sergey Smolov

Sorry, the correct assembler code is:

.globl _start
bl _start
_start:
movz  x0, 10
label1:
   subs x0, x0, 1
   b.eq label2
   b label1
label2:
hlt #57005

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Peter Maydell]

On 2 September 2015 at 14:50, Sergey Smolov <smolov@ispras.ru> wrote:
> 02.09.2015 16:55, Sergey Smolov пишет:
>> It seems that QEMU skips some internal instructions when generates
>> "in_asm" log. How to eliminate this?

It might help if you said what you thought was missing.

-- PMM

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

Peter Maydell писал 2015-09-02 19:39:
> On 2 September 2015 at 14:50, Sergey Smolov <smolov@ispras.ru> wrote:
>> 02.09.2015 16:55, Sergey Smolov пишет:
>>> It seems that QEMU skips some internal instructions when generates
>>> "in_asm" log. How to eliminate this?
> 
> It might help if you said what you thought was missing.
> 
> -- PMM

I mean that in this example QEMU does not write to log "intermediate" 
SUBS instructions which appear in loop-unrolling process.
For me it woulb be ok if QEMU generate the following in_asm log:

[log]
IN:
0x0000000000000000:  94000001      bl #+0x4 (addr 0x4)

----------------
IN:
0x0000000000000004:  d2800140      mov x0, #0xa

----------------
IN:
0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)

----------------
IN:
0x000000000000000c:  54000040      b.eq #+0x8 (addr 0x14)

----------------
IN:
0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)

----------------
IN:
0x000000000000000c:  54000040      b.eq #+0x8 (addr 0x14)

----------------
IN:
0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)

----------------
IN:
0x000000000000000c:  54000040      b.eq #+0x8 (addr 0x14)

...
<repeat SUBS and B.EQ 8 times, if we enable also "cpu" logging mode here 
we will see that X1 register value decrements on each iteration>
...
----------------
IN:
0x0000000000000010:  17fffffd      b #-0xc (addr 0x4)
[/log]

In terms of functional programming languages (like C), my idea is to 
generate for the following program:

for (i = 0; i < 2; i++) {
    func(i);
}

the log like:
func(0)
func(1)

but QEMU generates log that contains exactly one call of "func".

Sergey Smolov

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Peter Maydell]

On 3 September 2015 at 06:27, Sergey Smolov <smolov@ispras.ru> wrote:
> I mean that in this example QEMU does not write to log "intermediate" SUBS
> instructions which appear in loop-unrolling process.
> For me it woulb be ok if QEMU generate the following in_asm log:
>
> [log]
> IN:
> 0x0000000000000000:  94000001      bl #+0x4 (addr 0x4)
>
> ----------------
> IN:
> 0x0000000000000004:  d2800140      mov x0, #0xa
>
> ----------------
> IN:
> 0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)
>
> ----------------
> IN:
> 0x000000000000000c:  54000040      b.eq #+0x8 (addr 0x14)
>
> ----------------
> IN:
> 0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)

QEMU's in_asm logging doesn't work like that. We log the
instructions that we *translate*, not instructions that we
*execute*. Generally we translate basic blocks and then
cache them so they can be executed multiple times. (It's
not actually guaranteed that a translated block will ever
be executed, though it almost always will be.) The
logging of input instructions happens at the translate phase.
When we execute we don't have any information about the
guest instructions involved.

This has nothing to do with "loop-unrolling", which QEMU
doesn't try to do at all.

You might find the 'exec' logging helpful for tracking
which translated blocks get executed.

thanks
-- PMM

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Sergey Smolov]

[-- Attachment #1: Type: text/plain, Size: 2377 bytes --]



03.09.2015 10:33, Peter Maydell пишет:
> On 3 September 2015 at 06:27, Sergey Smolov <smolov@ispras.ru> wrote:
>> I mean that in this example QEMU does not write to log "intermediate" SUBS
>> instructions which appear in loop-unrolling process.
>> For me it woulb be ok if QEMU generate the following in_asm log:
>>
>> [log]
>> IN:
>> 0x0000000000000000:  94000001      bl #+0x4 (addr 0x4)
>>
>> ----------------
>> IN:
>> 0x0000000000000004:  d2800140      mov x0, #0xa
>>
>> ----------------
>> IN:
>> 0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)
>>
>> ----------------
>> IN:
>> 0x000000000000000c:  54000040      b.eq #+0x8 (addr 0x14)
>>
>> ----------------
>> IN:
>> 0x0000000000000008:  f1000400      subs x0, x0, #0x1 (1)
> QEMU's in_asm logging doesn't work like that. We log the
> instructions that we *translate*, not instructions that we
> *execute*. Generally we translate basic blocks and then
> cache them so they can be executed multiple times. (It's
> not actually guaranteed that a translated block will ever
> be executed, though it almost always will be.) The
> logging of input instructions happens at the translate phase.
> When we execute we don't have any information about the
> guest instructions involved.
>
> This has nothing to do with "loop-unrolling", which QEMU
> doesn't try to do at all.
>
> You might find the 'exec' logging helpful for tracking
> which translated blocks get executed.
>
> thanks
> -- PMM
>

Peter,

Thank you for your answer!

I've tested 'exec' logging and it does not contain unrolled llops too. 
For my sample it looks like:

[log]
Trace 0x7f999385a000 [0000000000000000]
Trace 0x7f999385a050 [0000000000000004]
Trace 0x7f999385a090 [*0000000000000008*]
Trace 0x7f999385a120 [000000000000000c]
Trace 0x7f999385a190 [0000000000000010]
Trace 0x7f999385a090 [*0000000000000008*]
Trace 0x7f999385a1d0 [0000000000000014]
Trace 0x7f999385a210 [0000000000000200]
Trace 0x7f999385a210 [0000000000000200]
...
[/log]

and contains only two records related to 'subs' instruction (it is 
situated on 0x0000000000000008 addresses whcih i've highlighted).

Also, is it true that after translation to internal representation 
(TCG-representation, yes?) there is no possibility at all to restore 
guest instructions?

Thanks,

Sergey Smolov

[-- Attachment #2: Type: text/html, Size: 3101 bytes --]

Re: [Qemu-devel] QEMU+Aarch64: in_asm log skips instructions of loop-programs

[Richard Henderson]

On 09/02/2015 11:33 PM, Peter Maydell wrote:
> You might find the 'exec' logging helpful for tracking
> which translated blocks get executed.

Even then you will of course only log the head of unlinked chains of blocks.


r~