[Qemu-devel] [PATCH v2 0/3] Enable MTTCG on PPC64

[Qemu-devel] [PATCH v2 0/3] Enable MTTCG on PPC64

[Nikunj A Dadhania]

The series enables Multi-Threaded TCG on PPC64

Patch 01: Use atomic_cmpxchg in store conditional
      02: Handle first write to page during atomic operation
      03: Generate memory barriers for sync/isync and load/store conditional

Patches are based on ppc-for-2.10

Changelog:
v1:
* Rewrote store_conditional as suggested by Richard

Tested using following:
./ppc64-softmmu/qemu-system-ppc64 -cpu POWER8 -vga none -nographic -machine pseries,usb=off -m 2G  -smp 8,cores=8,threads=1 -accel tcg,thread=multi  f23.img

Todo:
* Implement lqarx and stqcx
* Enable other machine types and PPC32.
* More testing for corner cases.

Nikunj A Dadhania (3):
  target/ppc: Emulate LL/SC using cmpxchg helpers
  cputlb: handle first atomic write to the page
  target/ppc: Generate fence operations

 cputlb.c               |  8 +++++++-
 target/ppc/translate.c | 37 +++++++++++++++++++++++++++++++------
 2 files changed, 38 insertions(+), 7 deletions(-)

-- 
2.9.3

[Qemu-devel] [PATCH v2 1/3] target/ppc: Emulate LL/SC using cmpxchg helpers

[Nikunj A Dadhania]

Emulating LL/SC with cmpxchg is not correct, since it can suffer from
the ABA problem. However, portable parallel code is written assuming
only cmpxchg which means that in practice this is a viable alternative.

Signed-off-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
---
 target/ppc/translate.c | 29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/target/ppc/translate.c b/target/ppc/translate.c
index f40b5a1..50b6d4d 100644
--- a/target/ppc/translate.c
+++ b/target/ppc/translate.c
@@ -73,6 +73,7 @@ static TCGv cpu_cfar;
 #endif
 static TCGv cpu_xer, cpu_so, cpu_ov, cpu_ca, cpu_ov32, cpu_ca32;
 static TCGv cpu_reserve;
+static TCGv cpu_reserve_val;
 static TCGv cpu_fpscr;
 static TCGv_i32 cpu_access_type;
 
@@ -181,6 +182,9 @@ void ppc_translate_init(void)
     cpu_reserve = tcg_global_mem_new(cpu_env,
                                      offsetof(CPUPPCState, reserve_addr),
                                      "reserve_addr");
+    cpu_reserve_val = tcg_global_mem_new(cpu_env,
+                                     offsetof(CPUPPCState, reserve_val),
+                                     "reserve_val");
 
     cpu_fpscr = tcg_global_mem_new(cpu_env,
                                    offsetof(CPUPPCState, fpscr), "fpscr");
@@ -3023,7 +3027,7 @@ static void gen_##name(DisasContext *ctx)                            \
     }                                                                \
     tcg_gen_qemu_ld_tl(gpr, t0, ctx->mem_idx, memop);                \
     tcg_gen_mov_tl(cpu_reserve, t0);                                 \
-    tcg_gen_st_tl(gpr, cpu_env, offsetof(CPUPPCState, reserve_val)); \
+    tcg_gen_mov_tl(cpu_reserve_val, gpr);                            \
     tcg_temp_free(t0);                                               \
 }
 
@@ -3155,14 +3159,27 @@ static void gen_conditional_store(DisasContext *ctx, TCGv EA,
 static void gen_conditional_store(DisasContext *ctx, TCGv EA,
                                   int reg, int memop)
 {
-    TCGLabel *l1;
+    TCGLabel *l1 = gen_new_label();
+    TCGLabel *l2 = gen_new_label();
+    TCGv t0;
 
-    tcg_gen_trunc_tl_i32(cpu_crf[0], cpu_so);
-    l1 = gen_new_label();
     tcg_gen_brcond_tl(TCG_COND_NE, EA, cpu_reserve, l1);
-    tcg_gen_ori_i32(cpu_crf[0], cpu_crf[0], CRF_EQ);
-    tcg_gen_qemu_st_tl(cpu_gpr[reg], EA, ctx->mem_idx, memop);
+
+    t0 = tcg_temp_new();
+    tcg_gen_atomic_cmpxchg_tl(t0, cpu_reserve, cpu_reserve_val,
+                              cpu_gpr[reg], ctx->mem_idx,
+                              DEF_MEMOP(memop) | MO_ALIGN);
+    tcg_gen_setcond_tl(TCG_COND_EQ, t0, t0, cpu_reserve_val);
+    tcg_gen_shli_tl(t0, t0, CRF_EQ_BIT);
+    tcg_gen_or_tl(t0, t0, cpu_so);
+    tcg_gen_trunc_tl_i32(cpu_crf[0], t0);
+    tcg_temp_free(t0);
+    tcg_gen_br(l2);
+
     gen_set_label(l1);
+    tcg_gen_trunc_tl_i32(cpu_crf[0], cpu_so);
+
+    gen_set_label(l2);
     tcg_gen_movi_tl(cpu_reserve, -1);
 }
 #endif
-- 
2.9.3

Re: [Qemu-devel] [PATCH v2 1/3] target/ppc: Emulate LL/SC using cmpxchg helpers

[Richard Henderson]

On 04/06/2017 11:07 PM, Nikunj A Dadhania wrote:
> Emulating LL/SC with cmpxchg is not correct, since it can suffer from
> the ABA problem. However, portable parallel code is written assuming
> only cmpxchg which means that in practice this is a viable alternative.
>
> Signed-off-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
> ---
>  target/ppc/translate.c | 29 +++++++++++++++++++++++------
>  1 file changed, 23 insertions(+), 6 deletions(-)

Reviewed-by: Richard Henderson <rth@twiddle.net>


r~

[Qemu-devel] [PATCH v2 2/3] cputlb: handle first atomic write to the page

[Nikunj A Dadhania]

In case where the conditional write is the first write to the page,
TLB_NOTDIRTY will be set and stop_the_world is triggered. Handle this as
a special case and set the dirty bit. After that fall through to the
actual atomic instruction below.

Signed-off-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
Reviewed-by: Richard Henderson <rth@twiddle.net>
---
 cputlb.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/cputlb.c b/cputlb.c
index f5d056c..743776a 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -930,7 +930,13 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
         tlb_addr = tlbe->addr_write;
     }
 
-    /* Notice an IO access, or a notdirty page.  */
+    /* Check notdirty */
+    if (unlikely(tlb_addr & TLB_NOTDIRTY)) {
+        tlb_set_dirty(ENV_GET_CPU(env), addr);
+        tlb_addr = tlb_addr & ~TLB_NOTDIRTY;
+    }
+
+    /* Notice an IO access  */
     if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
         /* There's really nothing that can be done to
            support this apart from stop-the-world.  */
-- 
2.9.3

[Qemu-devel] [PATCH v2 3/3] target/ppc: Generate fence operations

[Nikunj A Dadhania]

Signed-off-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
---
 target/ppc/translate.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/target/ppc/translate.c b/target/ppc/translate.c
index 50b6d4d..4a1f24a 100644
--- a/target/ppc/translate.c
+++ b/target/ppc/translate.c
@@ -2971,6 +2971,7 @@ static void gen_stswx(DisasContext *ctx)
 /* eieio */
 static void gen_eieio(DisasContext *ctx)
 {
+    tcg_gen_mb(TCG_MO_LD_ST | TCG_BAR_SC);
 }
 
 #if !defined(CONFIG_USER_ONLY)
@@ -3008,6 +3009,7 @@ static void gen_isync(DisasContext *ctx)
     if (!ctx->pr) {
         gen_check_tlb_flush(ctx, false);
     }
+    tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
     gen_stop_exception(ctx);
 }
 
@@ -3028,6 +3030,7 @@ static void gen_##name(DisasContext *ctx)                            \
     tcg_gen_qemu_ld_tl(gpr, t0, ctx->mem_idx, memop);                \
     tcg_gen_mov_tl(cpu_reserve, t0);                                 \
     tcg_gen_mov_tl(cpu_reserve_val, gpr);                            \
+    tcg_gen_mb(TCG_MO_ALL | TCG_BAR_LDAQ);                           \
     tcg_temp_free(t0);                                               \
 }
 
@@ -3177,6 +3180,10 @@ static void gen_conditional_store(DisasContext *ctx, TCGv EA,
     tcg_gen_br(l2);
 
     gen_set_label(l1);
+
+    /* Address mismatch implies failure.  But we still need to provide the
+       memory barrier semantics of the instruction.  */
+    tcg_gen_mb(TCG_MO_ALL | TCG_BAR_STRL);
     tcg_gen_trunc_tl_i32(cpu_crf[0], cpu_so);
 
     gen_set_label(l2);
@@ -3308,6 +3315,7 @@ static void gen_sync(DisasContext *ctx)
     if (((l == 2) || !(ctx->insns_flags & PPC_64B)) && !ctx->pr) {
         gen_check_tlb_flush(ctx, true);
     }
+    tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
 }
 
 /* wait */
-- 
2.9.3

Re: [Qemu-devel] [PATCH v2 3/3] target/ppc: Generate fence operations

[Richard Henderson]

On 04/06/2017 11:07 PM, Nikunj A Dadhania wrote:
> Signed-off-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>
> ---
>  target/ppc/translate.c | 8 ++++++++
>  1 file changed, 8 insertions(+)

Reviewed-by: Richard Henderson <rth@twiddle.net>


r~

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[luigi burdo]

Tested on PowerMac G5 Quad  and 380% of system load and working on

Fedora 25 PPC64 host and Ubuntu Mate 17.04 guest  (patched the 2.9 rc3)


The machine configuration was this

sudo ./qemu-system-ppc64 -cpu POWER8 -vga none -machine pseries-2.5,usb=off -m 2G  -smp 4,cores=4,threads=1 -accel tcg,thread=multi  -kerne vmlinuz    -append root=/dev/sda   -device ich9-ahci,id=ahci   -device ide-drive,drive=disk0 -drive file=/dev/sda4,if=none,id=disk0   -net nic,model=pcnet -net user -soundhw hda  -display sdl -vga virtio


vga virtio working too

here a shot

https://scontent-mxp1-1.xx.fbcdn.net/v/t1.0-9/17796379_10208795258860396_7825547329794577576_n.jpg?oh=526d6ddeb67c817053582d5b9ee56c71&oe=594D7BDF


Thanks

Luigi

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[G 3]

On Apr 7, 2017, at 1:49 PM, luigi burdo wrote:

> Tested on PowerMac G5 Quad  and 380% of system load and working on
> Fedora 25 PPC64 host and Ubuntu Mate 17.04 guest  (patched the 2.9  
> rc3)
>
> The machine configuration was this
>
> sudo ./qemu-system-ppc64 -cpu POWER8 -vga none -machine  
> pseries-2.5,usb=off -m 2G  -smp 4,cores=4,threads=1 -accel  
> tcg,thread=multi  -kerne vmlinuz    -append root=/dev/sda   -device  
> ich9-ahci,id=ahci   -device ide-drive,drive=disk0 -drive file=/dev/ 
> sda4,if=none,id=disk0   -net nic,model=pcnet -net user -soundhw  
> hda  -display sdl -vga virtio
>
> vga virtio working too
> here a shot
> https://scontent-mxp1-1.xx.fbcdn.net/v/ 
> t1.0-9/17796379_10208795258860396_7825547329794577576_n.jpg? 
> oh=526d6ddeb67c817053582d5b9ee56c71&oe=594D7BDF
>
> Thanks
> Luigi

Do you have any timings? Did the guest run faster?

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[luigi burdo]

Hi i dint made much tests,

but for sure all is faster compared one thread only.

Fore sure tcg need to be optimized (in all emulated architectures) compared some old commercial emulators but all is better than before.

I will made more tests tomorrow and report.

ciao

Luigi


Do you have any timings? Did the guest run faster?

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[luigi burdo]

Hi, on info is mttcg using an amouth of ram for cpu caching and translating operations like was did in past by emulators like virtualpc,realpc, bluelabel or softwindows?

in case of yes is possible increase it from the command line?

Thanks

Luigi

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[Alex Bennée]

luigi burdo <intermediadc@hotmail.com> writes:

> Hi, on info is mttcg using an amouth of ram for cpu caching and
> translating operations like was did in past by emulators like
> virtualpc,realpc, bluelabel or softwindows?

It's a fixed buffer. You can see if it being flushed by running:

  info jit

On the QEMU monitor console.

>
> in case of yes is possible increase it from the command line?
>
> Thanks
>
> Luigi


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 0/3] Enable MTTCG on PPC64

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

On Fri, Apr 07, 2017 at 11:37:49AM +0530, Nikunj A Dadhania wrote:
> The series enables Multi-Threaded TCG on PPC64
> 
> Patch 01: Use atomic_cmpxchg in store conditional
>       02: Handle first write to page during atomic operation
>       03: Generate memory barriers for sync/isync and load/store conditional
> 
> Patches are based on ppc-for-2.10

Applied to ppc-for-2.10.  Anyone object to that for 2/3, which isn't
within ppc code?

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[luigi burdo]

Hi David and Nikuji,

can i suggest to remove the message:


Guest not yet converted to MTTCG - you may get unexpected results
where the mttcg is enabled?

another thing im finding  is this message
Guest expects a stronger memory ordering than the host provides
This may cause strange/hard to debug errors


I have 8 gb on my machine and this message come if i gave 512mb on the guest too.

Is possible know where the mttcg is already enabled?

i see on x86,i386,arm too if i set thread=multy the qemu start using the other host cores is on this guest system enabled?

if yes i have the same messages of ppc64 too :

Guest not yet converted to MTTCG - you may get unexpected results

and

Guest expects a stronger memory ordering than the host provides
This may cause strange/hard to debug errors


Ciao

Luigi


Applied to ppc-for-2.10.  Anyone object to that for 2/3, which isn't
within ppc code?

--
David Gibson
<http://www.ozlabs.org/~dgibson>

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[Alex Bennée]

luigi burdo <intermediadc@hotmail.com> writes:

> Hi David and Nikuji,
>
> can i suggest to remove the message:
>
>
> Guest not yet converted to MTTCG - you may get unexpected results
> where the mttcg is enabled?

Have you declared the memory ordering for the guest?

>
> another thing im finding  is this message
> Guest expects a stronger memory ordering than the host provides
> This may cause strange/hard to debug errors

See ca759f9e387db87e1719911f019bc60c74be9ed8 for an example.

>
>
> I have 8 gb on my machine and this message come if i gave 512mb on the guest too.
>
> Is possible know where the mttcg is already enabled?
>
> i see on x86,i386,arm too if i set thread=multy the qemu start using the other host cores is on this guest system enabled?
>
> if yes i have the same messages of ppc64 too :
>
> Guest not yet converted to MTTCG - you may get unexpected results
>
> and
>
> Guest expects a stronger memory ordering than the host provides
> This may cause strange/hard to debug errors
>
>
> Ciao
>
> Luigi
>
>
> Applied to ppc-for-2.10.  Anyone object to that for 2/3, which isn't
> within ppc code?


--
Alex Bennée

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[luigi burdo]

Hi Alex,

>Have you declared the memory ordering for the guest?

Nope didnt know was necessary i just add the standard -m 2047


>See ca759f9e387db87e1719911f019bc60c74be9ed8 for an example.

watching it about

Thanks
Luigi

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[Nikunj A Dadhania]

Alex Bennée <alex.bennee@linaro.org> writes:

> luigi burdo <intermediadc@hotmail.com> writes:
>
>> Hi David and Nikuji,
>>
>> can i suggest to remove the message:
>>
>>
>> Guest not yet converted to MTTCG - you may get unexpected results
>> where the mttcg is enabled?
>
> Have you declared the memory ordering for the guest?

No, I havent done that yet, will send a patch.

>> another thing im finding  is this message
>> Guest expects a stronger memory ordering than the host provides
>> This may cause strange/hard to debug errors
>
> See ca759f9e387db87e1719911f019bc60c74be9ed8 for an example.

Regards
Nikunj

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[Alex Bennée]

Nikunj A Dadhania <nikunj@linux.vnet.ibm.com> writes:

> Alex Bennée <alex.bennee@linaro.org> writes:
>
>> luigi burdo <intermediadc@hotmail.com> writes:
>>
>>> Hi David and Nikuji,
>>>
>>> can i suggest to remove the message:
>>>
>>>
>>> Guest not yet converted to MTTCG - you may get unexpected results
>>> where the mttcg is enabled?
>>
>> Have you declared the memory ordering for the guest?
>
> No, I havent done that yet, will send a patch.

You also need to update configure so mttcg="yes" (resulting in
TARGET_SUPPORTS_MTTCG being set for the build).
>
>>> another thing im finding  is this message
>>> Guest expects a stronger memory ordering than the host provides
>>> This may cause strange/hard to debug errors
>>
>> See ca759f9e387db87e1719911f019bc60c74be9ed8 for an example.
>
> Regards
> Nikunj


--
Alex Bennée

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2 0/3] Enable MTTCG on PPC64

[Nikunj A Dadhania]

Alex Bennée <alex.bennee@linaro.org> writes:

> Nikunj A Dadhania <nikunj@linux.vnet.ibm.com> writes:
>
>> Alex Bennée <alex.bennee@linaro.org> writes:
>>
>>> luigi burdo <intermediadc@hotmail.com> writes:
>>>
>>>> Hi David and Nikuji,
>>>>
>>>> can i suggest to remove the message:
>>>>
>>>>
>>>> Guest not yet converted to MTTCG - you may get unexpected results
>>>> where the mttcg is enabled?
>>>
>>> Have you declared the memory ordering for the guest?
>>
>> No, I havent done that yet, will send a patch.
>
> You also need to update configure so mttcg="yes" (resulting in
> TARGET_SUPPORTS_MTTCG being set for the build).

Yes, I have that patch for ppc64, will send it to the list.

Regards
Nikunj

Re: [Qemu-devel] [PATCH v2 0/3] Enable MTTCG on PPC64

[Richard Henderson]

On 04/07/2017 11:51 PM, David Gibson wrote:
> On Fri, Apr 07, 2017 at 11:37:49AM +0530, Nikunj A Dadhania wrote:
>> The series enables Multi-Threaded TCG on PPC64
>>
>> Patch 01: Use atomic_cmpxchg in store conditional
>>       02: Handle first write to page during atomic operation
>>       03: Generate memory barriers for sync/isync and load/store conditional
>>
>> Patches are based on ppc-for-2.10
>
> Applied to ppc-for-2.10.  Anyone object to that for 2/3, which isn't
> within ppc code?

Please go ahead.  If there was another queue it might have gone in, it would 
probably be tcg-next, and I don't have anything else pending at the moment.


r~