* [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up
@ 2016-04-28 21:33 Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 01/10] tcg: Clean up direct block chaining data fields Sergey Fedorov
` (10 more replies)
0 siblings, 11 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson
From: Sergey Fedorov <serge.fdrv@gmail.com>
This series combines a set of patches which is meant to improve overall code
structure and readability of the direct block chaining mechanism. The other
point is to make a step towards thread safety of TB chainig.
This series is based on commit: 1d02fa9e045b ("translate-all: Adjust 256mb
testing for mips64") from git://github.com/rth7680/qemu.git tcg-next and is
available at git://github.com/sergefdrv/qemu.git tb-chaining-cleanup-v5
Summary of changes:
Changes in v5:
* Fixed rebase conflicts
* Don't check for in_superpage() in target-alpha/translate.c for
user-mode [PATCH v5 10/10]
Changes in v4:
* Removed assert from tb_add_jump() [PATCH v4 02/10]
* Added comment on TB stuff synchronization [PATCH v4 04/10]
* Documented tcg_gen_goto_tb() and moved its usage notes there
[PATCH v4 09/10] and [PATCH v4 10/10]
* Cc'ed usermode maintainers in commit message [PATCH v4 10/10]
Changes in v3:
* New patch to clean up safety checks [PATCH v3 09/10]
* New patch to eliminate unneeded checks in user-mode [PATCH v3 10/10]
Changes in v2:
* Eliminated duplicate dereference of 'ptb' in tb_jmp_remove() [PATCH v2 2/8]
* Tweaked a comment [PATCH v2 4/8]
* Complete rewrite [PATCH v2 5/8]
* Tweaked a comment; eliminated duplicate dereference of 'ptb' in
tb_jmp_unlink() [PATCH v2 8/8]
Sergey Fedorov (10):
tcg: Clean up direct block chaining data fields
tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB
tcg: Rearrange tb_link_page() to avoid forward declaration
tcg: Init TB's direct jumps before making it visible
tcg: Clarify thread safety check in tb_add_jump()
tcg: Rename tb_jmp_remove() to tb_remove_from_jmp_list()
tcg: Extract removing of jumps to TB from tb_phys_invalidate()
tcg: Clean up tb_jmp_unlink()
tcg: Clean up direct block chaining safety checks
tcg: Allow goto_tb to any target PC in user mode
cpu-exec.c | 7 +-
include/exec/exec-all.h | 69 ++++++----
target-alpha/translate.c | 4 +
target-arm/translate-a64.c | 2 +
target-arm/translate.c | 17 ++-
target-cris/translate.c | 16 ++-
target-i386/translate.c | 23 ++--
target-lm32/translate.c | 21 ++-
target-m68k/translate.c | 18 ++-
target-microblaze/translate.c | 15 ++-
target-mips/translate.c | 20 ++-
target-moxie/translate.c | 21 ++-
target-openrisc/translate.c | 20 ++-
target-ppc/translate.c | 20 ++-
target-s390x/translate.c | 17 ++-
target-sh4/translate.c | 21 ++-
target-sparc/translate.c | 24 +++-
target-tricore/translate.c | 20 ++-
target-unicore32/translate.c | 16 ++-
target-xtensa/translate.c | 4 +
tcg/aarch64/tcg-target.inc.c | 7 +-
tcg/arm/tcg-target.inc.c | 8 +-
tcg/i386/tcg-target.inc.c | 8 +-
tcg/ia64/tcg-target.inc.c | 6 +-
tcg/mips/tcg-target.inc.c | 8 +-
tcg/ppc/tcg-target.inc.c | 6 +-
tcg/s390/tcg-target.inc.c | 11 +-
tcg/sparc/tcg-target.inc.c | 9 +-
tcg/tcg-op.h | 13 ++
tcg/tcg.h | 6 +-
tcg/tci/tcg-target.inc.c | 10 +-
translate-all.c | 301 ++++++++++++++++++++++--------------------
32 files changed, 472 insertions(+), 296 deletions(-)
--
2.8.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 01/10] tcg: Clean up direct block chaining data fields
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 02/10] tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB Sergey Fedorov
` (9 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov,
Claudio Fontana, Andrzej Zaborowski, Aurelien Jarno,
Vassili Karpov (malc),
Alexander Graf, Blue Swirl, Stefan Weil, qemu-arm
From: Sergey Fedorov <serge.fdrv@gmail.com>
Briefly describe in a comment how direct block chaining is done. It
should help in understanding of the following data fields.
Rename some fields in TranslationBlock and TCGContext structures to
better reflect their purpose (dropping excessive 'tb_' prefix in
TranslationBlock but keeping it in TCGContext):
tb_next_offset => jmp_reset_offset
tb_jmp_offset => jmp_insn_offset
tb_next => jmp_target_addr
jmp_next => jmp_list_next
jmp_first => jmp_list_first
Avoid using a magic constant as an invalid offset which is used to
indicate that there's no n-th jump generated.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
Changes in v5:
* Fixed rebase conflicts
include/exec/exec-all.h | 44 ++++++++++++++++++++++++--------------
tcg/aarch64/tcg-target.inc.c | 7 +++---
tcg/arm/tcg-target.inc.c | 8 +++----
tcg/i386/tcg-target.inc.c | 8 +++----
tcg/ia64/tcg-target.inc.c | 6 +++---
tcg/mips/tcg-target.inc.c | 8 +++----
tcg/ppc/tcg-target.inc.c | 6 +++---
tcg/s390/tcg-target.inc.c | 11 +++++-----
tcg/sparc/tcg-target.inc.c | 9 ++++----
tcg/tcg.h | 6 +++---
tcg/tci/tcg-target.inc.c | 10 ++++-----
translate-all.c | 51 +++++++++++++++++++++++---------------------
12 files changed, 96 insertions(+), 78 deletions(-)
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 6c113a3d2010..445d946d84fb 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -259,20 +259,32 @@ struct TranslationBlock {
struct TranslationBlock *page_next[2];
tb_page_addr_t page_addr[2];
- /* the following data are used to directly call another TB from
- the code of this one. */
- uint16_t tb_next_offset[2]; /* offset of original jump target */
+ /* The following data are used to directly call another TB from
+ * the code of this one. This can be done either by emitting direct or
+ * indirect native jump instructions. These jumps are reset so that the TB
+ * just continue its execution. The TB can be linked to another one by
+ * setting one of the jump targets (or patching the jump instruction). Only
+ * two of such jumps are supported.
+ */
+ uint16_t jmp_reset_offset[2]; /* offset of original jump target */
+#define TB_JMP_RESET_OFFSET_INVALID 0xffff /* indicates no jump generated */
#ifdef USE_DIRECT_JUMP
- uint16_t tb_jmp_offset[2]; /* offset of jump instruction */
+ uint16_t jmp_insn_offset[2]; /* offset of native jump instruction */
#else
- uintptr_t tb_next[2]; /* address of jump generated code */
+ uintptr_t jmp_target_addr[2]; /* target address for indirect jump */
#endif
- /* list of TBs jumping to this one. This is a circular list using
- the two least significant bits of the pointers to tell what is
- the next pointer: 0 = jmp_next[0], 1 = jmp_next[1], 2 =
- jmp_first */
- struct TranslationBlock *jmp_next[2];
- struct TranslationBlock *jmp_first;
+ /* Each TB has an assosiated circular list of TBs jumping to this one.
+ * jmp_list_first points to the first TB jumping to this one.
+ * jmp_list_next is used to point to the next TB in a list.
+ * Since each TB can have two jumps, it can participate in two lists.
+ * The two least significant bits of a pointer are used to choose which
+ * data field holds a pointer to the next TB:
+ * 0 => jmp_list_next[0], 1 => jmp_list_next[1], 2 => jmp_list_first.
+ * In other words, 0/1 tells which jump is used in the pointed TB,
+ * and 2 means that this is a pointer back to the target TB of this list.
+ */
+ struct TranslationBlock *jmp_list_next[2];
+ struct TranslationBlock *jmp_list_first;
};
#include "qemu/thread.h"
@@ -340,7 +352,7 @@ void tb_set_jmp_target1(uintptr_t jmp_addr, uintptr_t addr);
static inline void tb_set_jmp_target(TranslationBlock *tb,
int n, uintptr_t addr)
{
- uint16_t offset = tb->tb_jmp_offset[n];
+ uint16_t offset = tb->jmp_insn_offset[n];
tb_set_jmp_target1((uintptr_t)(tb->tc_ptr + offset), addr);
}
@@ -350,7 +362,7 @@ static inline void tb_set_jmp_target(TranslationBlock *tb,
static inline void tb_set_jmp_target(TranslationBlock *tb,
int n, uintptr_t addr)
{
- tb->tb_next[n] = addr;
+ tb->jmp_target_addr[n] = addr;
}
#endif
@@ -359,7 +371,7 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
TranslationBlock *tb_next)
{
/* NOTE: this test is only needed for thread safety */
- if (!tb->jmp_next[n]) {
+ if (!tb->jmp_list_next[n]) {
qemu_log_mask_and_addr(CPU_LOG_EXEC, tb->pc,
"Linking TBs %p [" TARGET_FMT_lx
"] index %d -> %p [" TARGET_FMT_lx "]\n",
@@ -369,8 +381,8 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
tb_set_jmp_target(tb, n, (uintptr_t)tb_next->tc_ptr);
/* add in TB jmp circular list */
- tb->jmp_next[n] = tb_next->jmp_first;
- tb_next->jmp_first = (TranslationBlock *)((uintptr_t)(tb) | (n));
+ tb->jmp_list_next[n] = tb_next->jmp_list_first;
+ tb_next->jmp_list_first = (TranslationBlock *)((uintptr_t)tb | n);
}
}
diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c
index 88183c830fb0..1447f7c216d9 100644
--- a/tcg/aarch64/tcg-target.inc.c
+++ b/tcg/aarch64/tcg-target.inc.c
@@ -1306,12 +1306,13 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
#ifndef USE_DIRECT_JUMP
#error "USE_DIRECT_JUMP required for aarch64"
#endif
- tcg_debug_assert(s->tb_jmp_offset != NULL); /* consistency for USE_DIRECT_JUMP */
- s->tb_jmp_offset[a0] = tcg_current_code_size(s);
+ /* consistency for USE_DIRECT_JUMP */
+ tcg_debug_assert(s->tb_jmp_insn_offset != NULL);
+ s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s);
/* actual branch destination will be patched by
aarch64_tb_set_jmp_target later, beware retranslation. */
tcg_out_goto_noaddr(s);
- s->tb_next_offset[a0] = tcg_current_code_size(s);
+ s->tb_jmp_reset_offset[a0] = tcg_current_code_size(s);
break;
case INDEX_op_br:
diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c
index 977baa05c516..f9f54c64c6af 100644
--- a/tcg/arm/tcg-target.inc.c
+++ b/tcg/arm/tcg-target.inc.c
@@ -1665,17 +1665,17 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc,
tcg_out_goto(s, COND_AL, tb_ret_addr);
break;
case INDEX_op_goto_tb:
- if (s->tb_jmp_offset) {
+ if (s->tb_jmp_insn_offset) {
/* Direct jump method */
- s->tb_jmp_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s);
tcg_out_b_noaddr(s, COND_AL);
} else {
/* Indirect jump method */
- intptr_t ptr = (intptr_t)(s->tb_next + args[0]);
+ intptr_t ptr = (intptr_t)(s->tb_jmp_target_addr + args[0]);
tcg_out_movi32(s, COND_AL, TCG_REG_R0, ptr & ~0xfff);
tcg_out_ld32_12(s, COND_AL, TCG_REG_PC, TCG_REG_R0, ptr & 0xfff);
}
- s->tb_next_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_reset_offset[args[0]] = tcg_current_code_size(s);
break;
case INDEX_op_br:
tcg_out_goto_label(s, COND_AL, arg_label(args[0]));
diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c
index 8d242a666591..317484cb5db3 100644
--- a/tcg/i386/tcg-target.inc.c
+++ b/tcg/i386/tcg-target.inc.c
@@ -1790,7 +1790,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc,
tcg_out_jmp(s, tb_ret_addr);
break;
case INDEX_op_goto_tb:
- if (s->tb_jmp_offset) {
+ if (s->tb_jmp_insn_offset) {
/* direct jump method */
int gap;
/* jump displacement must be aligned for atomic patching;
@@ -1801,14 +1801,14 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc,
tcg_out_nopn(s, gap - 1);
}
tcg_out8(s, OPC_JMP_long); /* jmp im */
- s->tb_jmp_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s);
tcg_out32(s, 0);
} else {
/* indirect jump method */
tcg_out_modrm_offset(s, OPC_GRP5, EXT5_JMPN_Ev, -1,
- (intptr_t)(s->tb_next + args[0]));
+ (intptr_t)(s->tb_jmp_target_addr + args[0]));
}
- s->tb_next_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_reset_offset[args[0]] = tcg_current_code_size(s);
break;
case INDEX_op_br:
tcg_out_jxx(s, JCC_JMP, arg_label(args[0]), 0);
diff --git a/tcg/ia64/tcg-target.inc.c b/tcg/ia64/tcg-target.inc.c
index 7557e6a9d41c..395223e34003 100644
--- a/tcg/ia64/tcg-target.inc.c
+++ b/tcg/ia64/tcg-target.inc.c
@@ -881,13 +881,13 @@ static void tcg_out_exit_tb(TCGContext *s, tcg_target_long arg)
static inline void tcg_out_goto_tb(TCGContext *s, TCGArg arg)
{
- if (s->tb_jmp_offset) {
+ if (s->tb_jmp_insn_offset) {
/* direct jump method */
tcg_abort();
} else {
/* indirect jump method */
tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R2,
- (tcg_target_long)(s->tb_next + arg));
+ (tcg_target_long)(s->tb_jmp_target_addr + arg));
tcg_out_bundle(s, MmI,
tcg_opc_m1 (TCG_REG_P0, OPC_LD8_M1,
TCG_REG_R2, TCG_REG_R2),
@@ -900,7 +900,7 @@ static inline void tcg_out_goto_tb(TCGContext *s, TCGArg arg)
tcg_opc_b4 (TCG_REG_P0, OPC_BR_SPTK_MANY_B4,
TCG_REG_B6));
}
- s->tb_next_offset[arg] = tcg_current_code_size(s);
+ s->tb_jmp_reset_offset[arg] = tcg_current_code_size(s);
}
static inline void tcg_out_jmp(TCGContext *s, TCGArg addr)
diff --git a/tcg/mips/tcg-target.inc.c b/tcg/mips/tcg-target.inc.c
index ee98e24c8cca..b0440b9c5a79 100644
--- a/tcg/mips/tcg-target.inc.c
+++ b/tcg/mips/tcg-target.inc.c
@@ -1397,19 +1397,19 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc,
}
break;
case INDEX_op_goto_tb:
- if (s->tb_jmp_offset) {
+ if (s->tb_jmp_insn_offset) {
/* direct jump method */
- s->tb_jmp_offset[a0] = tcg_current_code_size(s);
+ s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s);
/* Avoid clobbering the address during retranslation. */
tcg_out32(s, OPC_J | (*(uint32_t *)s->code_ptr & 0x3ffffff));
} else {
/* indirect jump method */
tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP0, TCG_REG_ZERO,
- (uintptr_t)(s->tb_next + a0));
+ (uintptr_t)(s->tb_jmp_target_addr + a0));
tcg_out_opc_reg(s, OPC_JR, 0, TCG_TMP0, 0);
}
tcg_out_nop(s);
- s->tb_next_offset[a0] = tcg_current_code_size(s);
+ s->tb_jmp_reset_offset[a0] = tcg_current_code_size(s);
break;
case INDEX_op_br:
tcg_out_brcond(s, TCG_COND_EQ, TCG_REG_ZERO, TCG_REG_ZERO,
diff --git a/tcg/ppc/tcg-target.inc.c b/tcg/ppc/tcg-target.inc.c
index 039fa77748c2..da100528ab6f 100644
--- a/tcg/ppc/tcg-target.inc.c
+++ b/tcg/ppc/tcg-target.inc.c
@@ -1902,14 +1902,14 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg *args,
tcg_out_b(s, 0, tb_ret_addr);
break;
case INDEX_op_goto_tb:
- tcg_debug_assert(s->tb_jmp_offset);
+ tcg_debug_assert(s->tb_jmp_insn_offset);
/* Direct jump. */
#ifdef __powerpc64__
/* Ensure the next insns are 8-byte aligned. */
if ((uintptr_t)s->code_ptr & 7) {
tcg_out32(s, NOP);
}
- s->tb_jmp_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s);
/* To be replaced by either a branch+nop or a load into TMP1. */
s->code_ptr += 2;
tcg_out32(s, MTSPR | RS(TCG_REG_TMP1) | CTR);
@@ -1918,7 +1918,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg *args,
/* To be replaced by a branch. */
s->code_ptr++;
#endif
- s->tb_next_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_reset_offset[args[0]] = tcg_current_code_size(s);
break;
case INDEX_op_br:
{
diff --git a/tcg/s390/tcg-target.inc.c b/tcg/s390/tcg-target.inc.c
index 8e8064ce73e7..e0a60e618c56 100644
--- a/tcg/s390/tcg-target.inc.c
+++ b/tcg/s390/tcg-target.inc.c
@@ -1717,7 +1717,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc,
break;
case INDEX_op_goto_tb:
- if (s->tb_jmp_offset) {
+ if (s->tb_jmp_insn_offset) {
/* branch displacement must be aligned for atomic patching;
* see if we need to add extra nop before branch
*/
@@ -1725,15 +1725,16 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc,
tcg_out16(s, NOP);
}
tcg_out16(s, RIL_BRCL | (S390_CC_ALWAYS << 4));
- s->tb_jmp_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s);
s->code_ptr += 2;
} else {
- /* load address stored at s->tb_next + args[0] */
- tcg_out_ld_abs(s, TCG_TYPE_PTR, TCG_TMP0, s->tb_next + args[0]);
+ /* load address stored at s->tb_jmp_target_addr + args[0] */
+ tcg_out_ld_abs(s, TCG_TYPE_PTR, TCG_TMP0,
+ s->tb_jmp_target_addr + args[0]);
/* and go there */
tcg_out_insn(s, RR, BCR, S390_CC_ALWAYS, TCG_TMP0);
}
- s->tb_next_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_reset_offset[args[0]] = tcg_current_code_size(s);
break;
OP_32_64(ld8u):
diff --git a/tcg/sparc/tcg-target.inc.c b/tcg/sparc/tcg-target.inc.c
index c6479e25076b..9938a5085eac 100644
--- a/tcg/sparc/tcg-target.inc.c
+++ b/tcg/sparc/tcg-target.inc.c
@@ -1229,18 +1229,19 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
}
break;
case INDEX_op_goto_tb:
- if (s->tb_jmp_offset) {
+ if (s->tb_jmp_insn_offset) {
/* direct jump method */
- s->tb_jmp_offset[a0] = tcg_current_code_size(s);
+ s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s);
/* Make sure to preserve links during retranslation. */
tcg_out32(s, CALL | (*s->code_ptr & ~INSN_OP(-1)));
} else {
/* indirect jump method */
- tcg_out_ld_ptr(s, TCG_REG_T1, (uintptr_t)(s->tb_next + a0));
+ tcg_out_ld_ptr(s, TCG_REG_T1,
+ (uintptr_t)(s->tb_jmp_target_addr + a0));
tcg_out_arithi(s, TCG_REG_G0, TCG_REG_T1, 0, JMPL);
}
tcg_out_nop(s);
- s->tb_next_offset[a0] = tcg_current_code_size(s);
+ s->tb_jmp_reset_offset[a0] = tcg_current_code_size(s);
break;
case INDEX_op_br:
tcg_out_bpcc(s, COND_A, BPCC_PT, arg_label(a0));
diff --git a/tcg/tcg.h b/tcg/tcg.h
index 40c8fbe2ae64..ea4ff02308fc 100644
--- a/tcg/tcg.h
+++ b/tcg/tcg.h
@@ -510,9 +510,9 @@ struct TCGContext {
/* goto_tb support */
tcg_insn_unit *code_buf;
- uintptr_t *tb_next;
- uint16_t *tb_next_offset;
- uint16_t *tb_jmp_offset; /* != NULL if USE_DIRECT_JUMP */
+ uint16_t *tb_jmp_reset_offset; /* tb->jmp_reset_offset */
+ uint16_t *tb_jmp_insn_offset; /* tb->jmp_insn_offset if USE_DIRECT_JUMP */
+ uintptr_t *tb_jmp_target_addr; /* tb->jmp_target_addr if !USE_DIRECT_JUMP */
/* liveness analysis */
uint16_t *op_dead_args; /* for each operation, each bit tells if the
diff --git a/tcg/tci/tcg-target.inc.c b/tcg/tci/tcg-target.inc.c
index 85eeb5de246a..fa74d5278eaa 100644
--- a/tcg/tci/tcg-target.inc.c
+++ b/tcg/tci/tcg-target.inc.c
@@ -553,19 +553,19 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg *args,
tcg_out64(s, args[0]);
break;
case INDEX_op_goto_tb:
- if (s->tb_jmp_offset) {
+ if (s->tb_jmp_insn_offset) {
/* Direct jump method. */
- tcg_debug_assert(args[0] < ARRAY_SIZE(s->tb_jmp_offset));
+ tcg_debug_assert(args[0] < ARRAY_SIZE(s->tb_jmp_insn_offset));
/* Align for atomic patching and thread safety */
s->code_ptr = QEMU_ALIGN_PTR_UP(s->code_ptr, 4);
- s->tb_jmp_offset[args[0]] = tcg_current_code_size(s);
+ s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s);
tcg_out32(s, 0);
} else {
/* Indirect jump method. */
TODO();
}
- tcg_debug_assert(args[0] < ARRAY_SIZE(s->tb_next_offset));
- s->tb_next_offset[args[0]] = tcg_current_code_size(s);
+ tcg_debug_assert(args[0] < ARRAY_SIZE(s->tb_jmp_reset_offset));
+ s->tb_jmp_reset_offset[args[0]] = tcg_current_code_size(s);
break;
case INDEX_op_br:
tci_out_label(s, arg_label(args[0]));
diff --git a/translate-all.c b/translate-all.c
index 79a515dde74a..c6613d13c980 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -931,7 +931,7 @@ static inline void tb_jmp_remove(TranslationBlock *tb, int n)
TranslationBlock *tb1, **ptb;
unsigned int n1;
- ptb = &tb->jmp_next[n];
+ ptb = &tb->jmp_list_next[n];
tb1 = *ptb;
if (tb1) {
/* find tb(n) in circular list */
@@ -943,15 +943,15 @@ static inline void tb_jmp_remove(TranslationBlock *tb, int n)
break;
}
if (n1 == 2) {
- ptb = &tb1->jmp_first;
+ ptb = &tb1->jmp_list_first;
} else {
- ptb = &tb1->jmp_next[n1];
+ ptb = &tb1->jmp_list_next[n1];
}
}
/* now we can suppress tb(n) from the list */
- *ptb = tb->jmp_next[n];
+ *ptb = tb->jmp_list_next[n];
- tb->jmp_next[n] = NULL;
+ tb->jmp_list_next[n] = NULL;
}
}
@@ -959,7 +959,8 @@ static inline void tb_jmp_remove(TranslationBlock *tb, int n)
another TB */
static inline void tb_reset_jump(TranslationBlock *tb, int n)
{
- tb_set_jmp_target(tb, n, (uintptr_t)(tb->tc_ptr + tb->tb_next_offset[n]));
+ uintptr_t addr = (uintptr_t)(tb->tc_ptr + tb->jmp_reset_offset[n]);
+ tb_set_jmp_target(tb, n, addr);
}
/* invalidate one TB */
@@ -1003,19 +1004,21 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
tb_jmp_remove(tb, 1);
/* suppress any remaining jumps to this TB */
- tb1 = tb->jmp_first;
+ tb1 = tb->jmp_list_first;
for (;;) {
n1 = (uintptr_t)tb1 & 3;
if (n1 == 2) {
break;
}
tb1 = (TranslationBlock *)((uintptr_t)tb1 & ~3);
- tb2 = tb1->jmp_next[n1];
+ tb2 = tb1->jmp_list_next[n1];
tb_reset_jump(tb1, n1);
- tb1->jmp_next[n1] = NULL;
+ tb1->jmp_list_next[n1] = NULL;
tb1 = tb2;
}
- tb->jmp_first = (TranslationBlock *)((uintptr_t)tb | 2); /* fail safe */
+
+ /* fail safe */
+ tb->jmp_list_first = (TranslationBlock *)((uintptr_t)tb | 2);
tcg_ctx.tb_ctx.tb_phys_invalidate_count++;
}
@@ -1100,15 +1103,15 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
trace_translate_block(tb, tb->pc, tb->tc_ptr);
/* generate machine code */
- tb->tb_next_offset[0] = 0xffff;
- tb->tb_next_offset[1] = 0xffff;
- tcg_ctx.tb_next_offset = tb->tb_next_offset;
+ tb->jmp_reset_offset[0] = TB_JMP_RESET_OFFSET_INVALID;
+ tb->jmp_reset_offset[1] = TB_JMP_RESET_OFFSET_INVALID;
+ tcg_ctx.tb_jmp_reset_offset = tb->jmp_reset_offset;
#ifdef USE_DIRECT_JUMP
- tcg_ctx.tb_jmp_offset = tb->tb_jmp_offset;
- tcg_ctx.tb_next = NULL;
+ tcg_ctx.tb_jmp_insn_offset = tb->jmp_insn_offset;
+ tcg_ctx.tb_jmp_target_addr = NULL;
#else
- tcg_ctx.tb_jmp_offset = NULL;
- tcg_ctx.tb_next = tb->tb_next;
+ tcg_ctx.tb_jmp_insn_offset = NULL;
+ tcg_ctx.tb_jmp_target_addr = tb->jmp_target_addr;
#endif
#ifdef CONFIG_PROFILER
@@ -1489,15 +1492,15 @@ static void tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
tb->page_addr[1] = -1;
}
- tb->jmp_first = (TranslationBlock *)((uintptr_t)tb | 2);
- tb->jmp_next[0] = NULL;
- tb->jmp_next[1] = NULL;
+ tb->jmp_list_first = (TranslationBlock *)((uintptr_t)tb | 2);
+ tb->jmp_list_next[0] = NULL;
+ tb->jmp_list_next[1] = NULL;
/* init original jump addresses */
- if (tb->tb_next_offset[0] != 0xffff) {
+ if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) {
tb_reset_jump(tb, 0);
}
- if (tb->tb_next_offset[1] != 0xffff) {
+ if (tb->jmp_reset_offset[1] != TB_JMP_RESET_OFFSET_INVALID) {
tb_reset_jump(tb, 1);
}
@@ -1690,9 +1693,9 @@ void dump_exec_info(FILE *f, fprintf_function cpu_fprintf)
if (tb->page_addr[1] != -1) {
cross_page++;
}
- if (tb->tb_next_offset[0] != 0xffff) {
+ if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) {
direct_jmp_count++;
- if (tb->tb_next_offset[1] != 0xffff) {
+ if (tb->jmp_reset_offset[1] != TB_JMP_RESET_OFFSET_INVALID) {
direct_jmp2_count++;
}
}
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 02/10] tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 01/10] tcg: Clean up direct block chaining data fields Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 03/10] tcg: Rearrange tb_link_page() to avoid forward declaration Sergey Fedorov
` (8 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov
From: Sergey Fedorov <serge.fdrv@gmail.com>
These fields do not contain pure pointers to a TranslationBlock
structure. So uintptr_t is the most appropriate type for them.
Also put some asserts to assure that the two least significant bits of
the pointer are always zero before assigning it to jmp_list_first.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
Changes in v4:
* Removed assert from tb_add_jump()
* Minor commit message rewording
Changes in v2:
* Eliminated duplicate dereference of 'ptb' in tb_jmp_remove()
include/exec/exec-all.h | 12 +++++++-----
translate-all.c | 38 ++++++++++++++++++++------------------
2 files changed, 27 insertions(+), 23 deletions(-)
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 445d946d84fb..64c2a660bcb2 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -277,14 +277,16 @@ struct TranslationBlock {
* jmp_list_first points to the first TB jumping to this one.
* jmp_list_next is used to point to the next TB in a list.
* Since each TB can have two jumps, it can participate in two lists.
- * The two least significant bits of a pointer are used to choose which
- * data field holds a pointer to the next TB:
+ * jmp_list_first and jmp_list_next are 4-byte aligned pointers to a
+ * TranslationBlock structure, but the two least significant bits of
+ * them are used to encode which data field of the pointed TB should
+ * be used to traverse the list further from that TB:
* 0 => jmp_list_next[0], 1 => jmp_list_next[1], 2 => jmp_list_first.
* In other words, 0/1 tells which jump is used in the pointed TB,
* and 2 means that this is a pointer back to the target TB of this list.
*/
- struct TranslationBlock *jmp_list_next[2];
- struct TranslationBlock *jmp_list_first;
+ uintptr_t jmp_list_next[2];
+ uintptr_t jmp_list_first;
};
#include "qemu/thread.h"
@@ -382,7 +384,7 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
/* add in TB jmp circular list */
tb->jmp_list_next[n] = tb_next->jmp_list_first;
- tb_next->jmp_list_first = (TranslationBlock *)((uintptr_t)tb | n);
+ tb_next->jmp_list_first = (uintptr_t)tb | n;
}
}
diff --git a/translate-all.c b/translate-all.c
index c6613d13c980..2fb16466c15e 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -928,17 +928,17 @@ static inline void tb_page_remove(TranslationBlock **ptb, TranslationBlock *tb)
static inline void tb_jmp_remove(TranslationBlock *tb, int n)
{
- TranslationBlock *tb1, **ptb;
+ TranslationBlock *tb1;
+ uintptr_t *ptb, ntb;
unsigned int n1;
ptb = &tb->jmp_list_next[n];
- tb1 = *ptb;
- if (tb1) {
+ if (*ptb) {
/* find tb(n) in circular list */
for (;;) {
- tb1 = *ptb;
- n1 = (uintptr_t)tb1 & 3;
- tb1 = (TranslationBlock *)((uintptr_t)tb1 & ~3);
+ ntb = *ptb;
+ n1 = ntb & 3;
+ tb1 = (TranslationBlock *)(ntb & ~3);
if (n1 == n && tb1 == tb) {
break;
}
@@ -951,7 +951,7 @@ static inline void tb_jmp_remove(TranslationBlock *tb, int n)
/* now we can suppress tb(n) from the list */
*ptb = tb->jmp_list_next[n];
- tb->jmp_list_next[n] = NULL;
+ tb->jmp_list_next[n] = (uintptr_t)NULL;
}
}
@@ -970,7 +970,7 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
PageDesc *p;
unsigned int h, n1;
tb_page_addr_t phys_pc;
- TranslationBlock *tb1, *tb2;
+ uintptr_t tb1, tb2;
/* remove the TB from the hash list */
phys_pc = tb->page_addr[0] + (tb->pc & ~TARGET_PAGE_MASK);
@@ -1006,19 +1006,20 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
/* suppress any remaining jumps to this TB */
tb1 = tb->jmp_list_first;
for (;;) {
- n1 = (uintptr_t)tb1 & 3;
+ TranslationBlock *tmp_tb;
+ n1 = tb1 & 3;
if (n1 == 2) {
break;
}
- tb1 = (TranslationBlock *)((uintptr_t)tb1 & ~3);
- tb2 = tb1->jmp_list_next[n1];
- tb_reset_jump(tb1, n1);
- tb1->jmp_list_next[n1] = NULL;
+ tmp_tb = (TranslationBlock *)(tb1 & ~3);
+ tb2 = tmp_tb->jmp_list_next[n1];
+ tb_reset_jump(tmp_tb, n1);
+ tmp_tb->jmp_list_next[n1] = (uintptr_t)NULL;
tb1 = tb2;
}
- /* fail safe */
- tb->jmp_list_first = (TranslationBlock *)((uintptr_t)tb | 2);
+ assert(((uintptr_t)tb & 3) == 0);
+ tb->jmp_list_first = (uintptr_t)tb | 2; /* fail safe */
tcg_ctx.tb_ctx.tb_phys_invalidate_count++;
}
@@ -1492,9 +1493,10 @@ static void tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
tb->page_addr[1] = -1;
}
- tb->jmp_list_first = (TranslationBlock *)((uintptr_t)tb | 2);
- tb->jmp_list_next[0] = NULL;
- tb->jmp_list_next[1] = NULL;
+ assert(((uintptr_t)tb & 3) == 0);
+ tb->jmp_list_first = (uintptr_t)tb | 2;
+ tb->jmp_list_next[0] = (uintptr_t)NULL;
+ tb->jmp_list_next[1] = (uintptr_t)NULL;
/* init original jump addresses */
if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) {
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 03/10] tcg: Rearrange tb_link_page() to avoid forward declaration
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 01/10] tcg: Clean up direct block chaining data fields Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 02/10] tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 04/10] tcg: Init TB's direct jumps before making it visible Sergey Fedorov
` (7 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov
From: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
translate-all.c | 204 ++++++++++++++++++++++++++++----------------------------
1 file changed, 101 insertions(+), 103 deletions(-)
diff --git a/translate-all.c b/translate-all.c
index 2fb16466c15e..4a58af43a430 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -153,8 +153,6 @@ void tb_lock_reset(void)
#endif
}
-static void tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
- tb_page_addr_t phys_page2);
static TranslationBlock *tb_find_pc(uintptr_t tc_ptr);
void cpu_gen_init(void)
@@ -1053,6 +1051,107 @@ static void build_page_bitmap(PageDesc *p)
}
}
+/* add the tb in the target page and protect it if necessary
+ *
+ * Called with mmap_lock held for user-mode emulation.
+ */
+static inline void tb_alloc_page(TranslationBlock *tb,
+ unsigned int n, tb_page_addr_t page_addr)
+{
+ PageDesc *p;
+#ifndef CONFIG_USER_ONLY
+ bool page_already_protected;
+#endif
+
+ tb->page_addr[n] = page_addr;
+ p = page_find_alloc(page_addr >> TARGET_PAGE_BITS, 1);
+ tb->page_next[n] = p->first_tb;
+#ifndef CONFIG_USER_ONLY
+ page_already_protected = p->first_tb != NULL;
+#endif
+ p->first_tb = (TranslationBlock *)((uintptr_t)tb | n);
+ invalidate_page_bitmap(p);
+
+#if defined(CONFIG_USER_ONLY)
+ if (p->flags & PAGE_WRITE) {
+ target_ulong addr;
+ PageDesc *p2;
+ int prot;
+
+ /* force the host page as non writable (writes will have a
+ page fault + mprotect overhead) */
+ page_addr &= qemu_host_page_mask;
+ prot = 0;
+ for (addr = page_addr; addr < page_addr + qemu_host_page_size;
+ addr += TARGET_PAGE_SIZE) {
+
+ p2 = page_find(addr >> TARGET_PAGE_BITS);
+ if (!p2) {
+ continue;
+ }
+ prot |= p2->flags;
+ p2->flags &= ~PAGE_WRITE;
+ }
+ mprotect(g2h(page_addr), qemu_host_page_size,
+ (prot & PAGE_BITS) & ~PAGE_WRITE);
+#ifdef DEBUG_TB_INVALIDATE
+ printf("protecting code page: 0x" TARGET_FMT_lx "\n",
+ page_addr);
+#endif
+ }
+#else
+ /* if some code is already present, then the pages are already
+ protected. So we handle the case where only the first TB is
+ allocated in a physical page */
+ if (!page_already_protected) {
+ tlb_protect_code(page_addr);
+ }
+#endif
+}
+
+/* add a new TB and link it to the physical page tables. phys_page2 is
+ * (-1) to indicate that only one page contains the TB.
+ *
+ * Called with mmap_lock held for user-mode emulation.
+ */
+static void tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
+ tb_page_addr_t phys_page2)
+{
+ unsigned int h;
+ TranslationBlock **ptb;
+
+ /* add in the physical hash table */
+ h = tb_phys_hash_func(phys_pc);
+ ptb = &tcg_ctx.tb_ctx.tb_phys_hash[h];
+ tb->phys_hash_next = *ptb;
+ *ptb = tb;
+
+ /* add in the page list */
+ tb_alloc_page(tb, 0, phys_pc & TARGET_PAGE_MASK);
+ if (phys_page2 != -1) {
+ tb_alloc_page(tb, 1, phys_page2);
+ } else {
+ tb->page_addr[1] = -1;
+ }
+
+ assert(((uintptr_t)tb & 3) == 0);
+ tb->jmp_list_first = (uintptr_t)tb | 2;
+ tb->jmp_list_next[0] = (uintptr_t)NULL;
+ tb->jmp_list_next[1] = (uintptr_t)NULL;
+
+ /* init original jump addresses */
+ if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) {
+ tb_reset_jump(tb, 0);
+ }
+ if (tb->jmp_reset_offset[1] != TB_JMP_RESET_OFFSET_INVALID) {
+ tb_reset_jump(tb, 1);
+ }
+
+#ifdef DEBUG_TB_CHECK
+ tb_page_check();
+#endif
+}
+
/* Called with mmap_lock held for user mode emulation. */
TranslationBlock *tb_gen_code(CPUState *cpu,
target_ulong pc, target_ulong cs_base,
@@ -1410,107 +1509,6 @@ static void tb_invalidate_phys_page(tb_page_addr_t addr,
}
#endif
-/* add the tb in the target page and protect it if necessary
- *
- * Called with mmap_lock held for user-mode emulation.
- */
-static inline void tb_alloc_page(TranslationBlock *tb,
- unsigned int n, tb_page_addr_t page_addr)
-{
- PageDesc *p;
-#ifndef CONFIG_USER_ONLY
- bool page_already_protected;
-#endif
-
- tb->page_addr[n] = page_addr;
- p = page_find_alloc(page_addr >> TARGET_PAGE_BITS, 1);
- tb->page_next[n] = p->first_tb;
-#ifndef CONFIG_USER_ONLY
- page_already_protected = p->first_tb != NULL;
-#endif
- p->first_tb = (TranslationBlock *)((uintptr_t)tb | n);
- invalidate_page_bitmap(p);
-
-#if defined(CONFIG_USER_ONLY)
- if (p->flags & PAGE_WRITE) {
- target_ulong addr;
- PageDesc *p2;
- int prot;
-
- /* force the host page as non writable (writes will have a
- page fault + mprotect overhead) */
- page_addr &= qemu_host_page_mask;
- prot = 0;
- for (addr = page_addr; addr < page_addr + qemu_host_page_size;
- addr += TARGET_PAGE_SIZE) {
-
- p2 = page_find(addr >> TARGET_PAGE_BITS);
- if (!p2) {
- continue;
- }
- prot |= p2->flags;
- p2->flags &= ~PAGE_WRITE;
- }
- mprotect(g2h(page_addr), qemu_host_page_size,
- (prot & PAGE_BITS) & ~PAGE_WRITE);
-#ifdef DEBUG_TB_INVALIDATE
- printf("protecting code page: 0x" TARGET_FMT_lx "\n",
- page_addr);
-#endif
- }
-#else
- /* if some code is already present, then the pages are already
- protected. So we handle the case where only the first TB is
- allocated in a physical page */
- if (!page_already_protected) {
- tlb_protect_code(page_addr);
- }
-#endif
-}
-
-/* add a new TB and link it to the physical page tables. phys_page2 is
- * (-1) to indicate that only one page contains the TB.
- *
- * Called with mmap_lock held for user-mode emulation.
- */
-static void tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
- tb_page_addr_t phys_page2)
-{
- unsigned int h;
- TranslationBlock **ptb;
-
- /* add in the physical hash table */
- h = tb_phys_hash_func(phys_pc);
- ptb = &tcg_ctx.tb_ctx.tb_phys_hash[h];
- tb->phys_hash_next = *ptb;
- *ptb = tb;
-
- /* add in the page list */
- tb_alloc_page(tb, 0, phys_pc & TARGET_PAGE_MASK);
- if (phys_page2 != -1) {
- tb_alloc_page(tb, 1, phys_page2);
- } else {
- tb->page_addr[1] = -1;
- }
-
- assert(((uintptr_t)tb & 3) == 0);
- tb->jmp_list_first = (uintptr_t)tb | 2;
- tb->jmp_list_next[0] = (uintptr_t)NULL;
- tb->jmp_list_next[1] = (uintptr_t)NULL;
-
- /* init original jump addresses */
- if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) {
- tb_reset_jump(tb, 0);
- }
- if (tb->jmp_reset_offset[1] != TB_JMP_RESET_OFFSET_INVALID) {
- tb_reset_jump(tb, 1);
- }
-
-#ifdef DEBUG_TB_CHECK
- tb_page_check();
-#endif
-}
-
/* find the TB 'tb' such that tb[0].tc_ptr <= tc_ptr <
tb[1].tc_ptr. Return NULL if not found */
static TranslationBlock *tb_find_pc(uintptr_t tc_ptr)
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 04/10] tcg: Init TB's direct jumps before making it visible
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
` (2 preceding siblings ...)
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 03/10] tcg: Rearrange tb_link_page() to avoid forward declaration Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 05/10] tcg: Clarify thread safety check in tb_add_jump() Sergey Fedorov
` (6 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov
From: Sergey Fedorov <serge.fdrv@gmail.com>
Initialize TB's direct jump list data fields and reset the jumps before
tb_link_page() puts it into the physical hash table and the physical
page list. So TB is completely initialized before it becomes visible.
This is pure rearrangement of code to a more suitable place, though it
could be a preparation for relaxing the locking scheme in future.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
Changes in v4:
* Added comment on TB stuff synchronization
* Expanded commit message
Changes in v2:
* Tweaked a comment
translate-all.c | 32 +++++++++++++++++++-------------
1 file changed, 19 insertions(+), 13 deletions(-)
diff --git a/translate-all.c b/translate-all.c
index 4a58af43a430..1dc1a73c1d55 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -1134,19 +1134,6 @@ static void tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
tb->page_addr[1] = -1;
}
- assert(((uintptr_t)tb & 3) == 0);
- tb->jmp_list_first = (uintptr_t)tb | 2;
- tb->jmp_list_next[0] = (uintptr_t)NULL;
- tb->jmp_list_next[1] = (uintptr_t)NULL;
-
- /* init original jump addresses */
- if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) {
- tb_reset_jump(tb, 0);
- }
- if (tb->jmp_reset_offset[1] != TB_JMP_RESET_OFFSET_INVALID) {
- tb_reset_jump(tb, 1);
- }
-
#ifdef DEBUG_TB_CHECK
tb_page_check();
#endif
@@ -1255,12 +1242,31 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
ROUND_UP((uintptr_t)gen_code_buf + gen_code_size + search_size,
CODE_GEN_ALIGN);
+ /* init jump list */
+ assert(((uintptr_t)tb & 3) == 0);
+ tb->jmp_list_first = (uintptr_t)tb | 2;
+ tb->jmp_list_next[0] = (uintptr_t)NULL;
+ tb->jmp_list_next[1] = (uintptr_t)NULL;
+
+ /* init original jump addresses wich has been set during tcg_gen_code() */
+ if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) {
+ tb_reset_jump(tb, 0);
+ }
+ if (tb->jmp_reset_offset[1] != TB_JMP_RESET_OFFSET_INVALID) {
+ tb_reset_jump(tb, 1);
+ }
+
/* check next page if needed */
virt_page2 = (pc + tb->size - 1) & TARGET_PAGE_MASK;
phys_page2 = -1;
if ((pc & TARGET_PAGE_MASK) != virt_page2) {
phys_page2 = get_page_addr_code(env, virt_page2);
}
+ /* As long as consistency of the TB stuff is provided by tb_lock in user
+ * mode and is implicit in single-threaded softmmu emulation, no explicit
+ * memory barrier is required before tb_link_page() makes the TB visible
+ * through the physical hash table and physical page list.
+ */
tb_link_page(tb, phys_pc, phys_page2);
return tb;
}
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 05/10] tcg: Clarify thread safety check in tb_add_jump()
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
` (3 preceding siblings ...)
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 04/10] tcg: Init TB's direct jumps before making it visible Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 06/10] tcg: Rename tb_jmp_remove() to tb_remove_from_jmp_list() Sergey Fedorov
` (5 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov
From: Sergey Fedorov <serge.fdrv@gmail.com>
The check is to make sure that another thread hasn't already done the
same while we were outside of tb_lock. Mention this in a comment.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
Changes in v2:
* Typo fixed in the commit title
* Complete rewrite of the commit body and the patch based on Paolo's comments
include/exec/exec-all.h | 29 ++++++++++++++++-------------
1 file changed, 16 insertions(+), 13 deletions(-)
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 64c2a660bcb2..06da1bcc45bf 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -372,20 +372,23 @@ static inline void tb_set_jmp_target(TranslationBlock *tb,
static inline void tb_add_jump(TranslationBlock *tb, int n,
TranslationBlock *tb_next)
{
- /* NOTE: this test is only needed for thread safety */
- if (!tb->jmp_list_next[n]) {
- qemu_log_mask_and_addr(CPU_LOG_EXEC, tb->pc,
- "Linking TBs %p [" TARGET_FMT_lx
- "] index %d -> %p [" TARGET_FMT_lx "]\n",
- tb->tc_ptr, tb->pc, n,
- tb_next->tc_ptr, tb_next->pc);
- /* patch the native jump address */
- tb_set_jmp_target(tb, n, (uintptr_t)tb_next->tc_ptr);
-
- /* add in TB jmp circular list */
- tb->jmp_list_next[n] = tb_next->jmp_list_first;
- tb_next->jmp_list_first = (uintptr_t)tb | n;
+ if (tb->jmp_list_next[n]) {
+ /* Another thread has already done this while we were
+ * outside of the lock; nothing to do in this case */
+ return;
}
+ qemu_log_mask_and_addr(CPU_LOG_EXEC, tb->pc,
+ "Linking TBs %p [" TARGET_FMT_lx
+ "] index %d -> %p [" TARGET_FMT_lx "]\n",
+ tb->tc_ptr, tb->pc, n,
+ tb_next->tc_ptr, tb_next->pc);
+
+ /* patch the native jump address */
+ tb_set_jmp_target(tb, n, (uintptr_t)tb_next->tc_ptr);
+
+ /* add in TB jmp circular list */
+ tb->jmp_list_next[n] = tb_next->jmp_list_first;
+ tb_next->jmp_list_first = (uintptr_t)tb | n;
}
/* GETRA is the true target of the return instruction that we'll execute,
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 06/10] tcg: Rename tb_jmp_remove() to tb_remove_from_jmp_list()
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
` (4 preceding siblings ...)
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 05/10] tcg: Clarify thread safety check in tb_add_jump() Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 07/10] tcg: Extract removing of jumps to TB from tb_phys_invalidate() Sergey Fedorov
` (4 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov
From: Sergey Fedorov <serge.fdrv@gmail.com>
tb_jmp_remove() was only used to remove the TB from a list of all TBs
jumping to the same TB which is n-th jump destination of the given TB.
Put a comment briefly describing the function behavior and rename it to
better reflect its purpose.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
translate-all.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/translate-all.c b/translate-all.c
index 1dc1a73c1d55..5e057baaa346 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -924,7 +924,8 @@ static inline void tb_page_remove(TranslationBlock **ptb, TranslationBlock *tb)
}
}
-static inline void tb_jmp_remove(TranslationBlock *tb, int n)
+/* remove the TB from a list of TBs jumping to the n-th jump target of the TB */
+static inline void tb_remove_from_jmp_list(TranslationBlock *tb, int n)
{
TranslationBlock *tb1;
uintptr_t *ptb, ntb;
@@ -998,8 +999,8 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
}
/* suppress this TB from the two jump lists */
- tb_jmp_remove(tb, 0);
- tb_jmp_remove(tb, 1);
+ tb_remove_from_jmp_list(tb, 0);
+ tb_remove_from_jmp_list(tb, 1);
/* suppress any remaining jumps to this TB */
tb1 = tb->jmp_list_first;
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 07/10] tcg: Extract removing of jumps to TB from tb_phys_invalidate()
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
` (5 preceding siblings ...)
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 06/10] tcg: Rename tb_jmp_remove() to tb_remove_from_jmp_list() Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 08/10] tcg: Clean up tb_jmp_unlink() Sergey Fedorov
` (3 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov
From: Sergey Fedorov <serge.fdrv@gmail.com>
Move the code for removing jumps to a TB out of tb_phys_invalidate() to
a separate static inline function tb_jmp_unlink(). This simplifies
tb_phys_invalidate() and improves code structure.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
translate-all.c | 44 ++++++++++++++++++++++++++------------------
1 file changed, 26 insertions(+), 18 deletions(-)
diff --git a/translate-all.c b/translate-all.c
index 5e057baaa346..9a57aaba3d57 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -962,14 +962,37 @@ static inline void tb_reset_jump(TranslationBlock *tb, int n)
tb_set_jmp_target(tb, n, addr);
}
+/* remove any jumps to the TB */
+static inline void tb_jmp_unlink(TranslationBlock *tb)
+{
+ uintptr_t tb1, tb2;
+ unsigned int n1;
+
+ tb1 = tb->jmp_list_first;
+ for (;;) {
+ TranslationBlock *tmp_tb;
+ n1 = tb1 & 3;
+ if (n1 == 2) {
+ break;
+ }
+ tmp_tb = (TranslationBlock *)(tb1 & ~3);
+ tb2 = tmp_tb->jmp_list_next[n1];
+ tb_reset_jump(tmp_tb, n1);
+ tmp_tb->jmp_list_next[n1] = (uintptr_t)NULL;
+ tb1 = tb2;
+ }
+
+ assert(((uintptr_t)tb & 3) == 0);
+ tb->jmp_list_first = (uintptr_t)tb | 2; /* fail safe */
+}
+
/* invalidate one TB */
void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
{
CPUState *cpu;
PageDesc *p;
- unsigned int h, n1;
+ unsigned int h;
tb_page_addr_t phys_pc;
- uintptr_t tb1, tb2;
/* remove the TB from the hash list */
phys_pc = tb->page_addr[0] + (tb->pc & ~TARGET_PAGE_MASK);
@@ -1003,22 +1026,7 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
tb_remove_from_jmp_list(tb, 1);
/* suppress any remaining jumps to this TB */
- tb1 = tb->jmp_list_first;
- for (;;) {
- TranslationBlock *tmp_tb;
- n1 = tb1 & 3;
- if (n1 == 2) {
- break;
- }
- tmp_tb = (TranslationBlock *)(tb1 & ~3);
- tb2 = tmp_tb->jmp_list_next[n1];
- tb_reset_jump(tmp_tb, n1);
- tmp_tb->jmp_list_next[n1] = (uintptr_t)NULL;
- tb1 = tb2;
- }
-
- assert(((uintptr_t)tb & 3) == 0);
- tb->jmp_list_first = (uintptr_t)tb | 2; /* fail safe */
+ tb_jmp_unlink(tb);
tcg_ctx.tb_ctx.tb_phys_invalidate_count++;
}
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 08/10] tcg: Clean up tb_jmp_unlink()
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
` (6 preceding siblings ...)
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 07/10] tcg: Extract removing of jumps to TB from tb_phys_invalidate() Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 09/10] tcg: Clean up direct block chaining safety checks Sergey Fedorov
` (2 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov
From: Sergey Fedorov <serge.fdrv@gmail.com>
Unify the code of this function with tb_jmp_remove_from_list(). Making
these functions similar improves their readability. Also this could be a
step towards making this function thread-safe.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
Changes in v2:
* Tweaked a comment to better describe jmp_list_{next|first} usage
* Eliminated duplicate dereference of 'ptb' in tb_jmp_unlink()
translate-all.c | 21 +++++++++------------
1 file changed, 9 insertions(+), 12 deletions(-)
diff --git a/translate-all.c b/translate-all.c
index 9a57aaba3d57..d679ad129e10 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -965,25 +965,22 @@ static inline void tb_reset_jump(TranslationBlock *tb, int n)
/* remove any jumps to the TB */
static inline void tb_jmp_unlink(TranslationBlock *tb)
{
- uintptr_t tb1, tb2;
+ TranslationBlock *tb1;
+ uintptr_t *ptb, ntb;
unsigned int n1;
- tb1 = tb->jmp_list_first;
+ ptb = &tb->jmp_list_first;
for (;;) {
- TranslationBlock *tmp_tb;
- n1 = tb1 & 3;
+ ntb = *ptb;
+ n1 = ntb & 3;
+ tb1 = (TranslationBlock *)(ntb & ~3);
if (n1 == 2) {
break;
}
- tmp_tb = (TranslationBlock *)(tb1 & ~3);
- tb2 = tmp_tb->jmp_list_next[n1];
- tb_reset_jump(tmp_tb, n1);
- tmp_tb->jmp_list_next[n1] = (uintptr_t)NULL;
- tb1 = tb2;
+ tb_reset_jump(tb1, n1);
+ *ptb = tb1->jmp_list_next[n1];
+ tb1->jmp_list_next[n1] = (uintptr_t)NULL;
}
-
- assert(((uintptr_t)tb & 3) == 0);
- tb->jmp_list_first = (uintptr_t)tb | 2; /* fail safe */
}
/* invalidate one TB */
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 09/10] tcg: Clean up direct block chaining safety checks
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
` (7 preceding siblings ...)
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 08/10] tcg: Clean up tb_jmp_unlink() Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 10/10] tcg: Allow goto_tb to any target PC in user mode Sergey Fedorov
2016-04-29 16:46 ` [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Richard Henderson
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov,
Peter Maydell, Edgar E. Iglesias, Eduardo Habkost,
Alexander Graf, qemu-arm
From: Sergey Fedorov <serge.fdrv@gmail.com>
We don't take care of direct jumps when address mapping changes. Thus we
must be sure to generate direct jumps so that they always keep valid
even if address mapping changes. Luckily, we can only allow to execute a
TB if it was generated from the pages which match with current mapping.
Document tcg_gen_goto_tb() declaration and note the reason for
destination PC limitations.
Some targets with variable length instructions allow TB to straddle a
page boundary. However, we make sure that both of TB pages match the
current address mapping when looking up TBs. So it is safe to do direct
jumps into the both pages. Correct the checks for some of those targets.
Given that, we can safely patch a TB which spans two pages. Remove the
unnecessary check in cpu_exec() and allow such TBs to be patched.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
Changes in v4:
* Documented tcg_gen_goto_tb() declaration
* Destination PC check explanatory comments moved to tcg_gen_goto_tb()
declaration comment
* Updated commit message
cpu-exec.c | 7 ++-----
target-arm/translate.c | 3 ++-
target-cris/translate.c | 4 +++-
target-i386/translate.c | 2 +-
target-m68k/translate.c | 2 +-
target-s390x/translate.c | 2 +-
tcg/tcg-op.h | 10 ++++++++++
7 files changed, 20 insertions(+), 10 deletions(-)
diff --git a/cpu-exec.c b/cpu-exec.c
index debc65ca691a..f984dc71cbf1 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -508,11 +508,8 @@ int cpu_exec(CPUState *cpu)
next_tb = 0;
tcg_ctx.tb_ctx.tb_invalidated_flag = 0;
}
- /* see if we can patch the calling TB. When the TB
- spans two pages, we cannot safely do a direct
- jump. */
- if (next_tb != 0 && tb->page_addr[1] == -1
- && !qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
+ /* See if we can patch the calling TB. */
+ if (next_tb != 0 && !qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
tb_add_jump((TranslationBlock *)(next_tb & ~TB_EXIT_MASK),
next_tb & TB_EXIT_MASK, tb);
}
diff --git a/target-arm/translate.c b/target-arm/translate.c
index 940ec8d981d1..34196a821772 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -4054,7 +4054,8 @@ static inline void gen_goto_tb(DisasContext *s, int n, target_ulong dest)
TranslationBlock *tb;
tb = s->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
+ if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
+ ((s->pc - 1) & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
tcg_gen_goto_tb(n);
gen_set_pc_im(s, dest);
tcg_gen_exit_tb((uintptr_t)tb + n);
diff --git a/target-cris/translate.c b/target-cris/translate.c
index a73176c118b0..9c8ff8f2308a 100644
--- a/target-cris/translate.c
+++ b/target-cris/translate.c
@@ -524,7 +524,9 @@ static void gen_goto_tb(DisasContext *dc, int n, target_ulong dest)
{
TranslationBlock *tb;
tb = dc->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
+
+ if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
+ (dc->ppc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
tcg_gen_goto_tb(n);
tcg_gen_movi_tl(env_pc, dest);
tcg_gen_exit_tb((uintptr_t)tb + n);
diff --git a/target-i386/translate.c b/target-i386/translate.c
index 3a32f65f8de8..058d85a1abf7 100644
--- a/target-i386/translate.c
+++ b/target-i386/translate.c
@@ -2094,7 +2094,7 @@ static inline void gen_goto_tb(DisasContext *s, int tb_num, target_ulong eip)
tb = s->tb;
/* NOTE: we handle the case where the TB spans two pages here */
if ((pc & TARGET_PAGE_MASK) == (tb->pc & TARGET_PAGE_MASK) ||
- (pc & TARGET_PAGE_MASK) == ((s->pc - 1) & TARGET_PAGE_MASK)) {
+ (pc & TARGET_PAGE_MASK) == (s->pc_start & TARGET_PAGE_MASK)) {
/* jump to same page: we can use a direct jump */
tcg_gen_goto_tb(tb_num);
gen_jmp_im(eip);
diff --git a/target-m68k/translate.c b/target-m68k/translate.c
index 7560c3a80841..e2ce6c615e07 100644
--- a/target-m68k/translate.c
+++ b/target-m68k/translate.c
@@ -861,7 +861,7 @@ static void gen_jmp_tb(DisasContext *s, int n, uint32_t dest)
if (unlikely(s->singlestep_enabled)) {
gen_exception(s, dest, EXCP_DEBUG);
} else if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
- (s->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
+ (s->insn_pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
tcg_gen_goto_tb(n);
tcg_gen_movi_i32(QREG_PC, dest);
tcg_gen_exit_tb((uintptr_t)tb + n);
diff --git a/target-s390x/translate.c b/target-s390x/translate.c
index c871ef2bb302..c5179fe05d7e 100644
--- a/target-s390x/translate.c
+++ b/target-s390x/translate.c
@@ -610,7 +610,7 @@ static int use_goto_tb(DisasContext *s, uint64_t dest)
{
/* NOTE: we handle the case where the TB spans two pages here */
return (((dest & TARGET_PAGE_MASK) == (s->tb->pc & TARGET_PAGE_MASK)
- || (dest & TARGET_PAGE_MASK) == ((s->pc - 1) & TARGET_PAGE_MASK))
+ || (dest & TARGET_PAGE_MASK) == (s->pc & TARGET_PAGE_MASK))
&& !s->singlestep_enabled
&& !(s->tb->cflags & CF_LAST_IO)
&& !(s->tb->flags & FLAG_MASK_PER));
diff --git a/tcg/tcg-op.h b/tcg/tcg-op.h
index c446d3dc7293..ace39619ef89 100644
--- a/tcg/tcg-op.h
+++ b/tcg/tcg-op.h
@@ -753,6 +753,16 @@ static inline void tcg_gen_exit_tb(uintptr_t val)
tcg_gen_op1i(INDEX_op_exit_tb, val);
}
+/**
+ * tcg_gen_goto_tb() - output goto_tb TCG operation
+ * @idx: Direct jump slot index (0 or 1)
+ *
+ * See tcg/README for more info about this TCG operation.
+ *
+ * NOTE: Direct jumps with goto_tb are only safe within the pages this TB
+ * resides in because we don't take care of direct jumps when address mapping
+ * changes, e.g. in tlb_flush().
+ */
void tcg_gen_goto_tb(unsigned idx);
#if TARGET_LONG_BITS == 32
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH v5 10/10] tcg: Allow goto_tb to any target PC in user mode
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
` (8 preceding siblings ...)
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 09/10] tcg: Clean up direct block chaining safety checks Sergey Fedorov
@ 2016-04-28 21:33 ` Sergey Fedorov
2016-04-29 16:46 ` [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Richard Henderson
10 siblings, 0 replies; 12+ messages in thread
From: Sergey Fedorov @ 2016-04-28 21:33 UTC (permalink / raw)
To: qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini,
Peter Crosthwaite, Richard Henderson, Sergey Fedorov,
Riku Voipio, Blue Swirl, Peter Maydell, Edgar E. Iglesias,
Eduardo Habkost, Michael Walle, Aurelien Jarno, Leon Alrae,
Anthony Green, Jia Liu, Alexander Graf, Mark Cave-Ayland,
Bastian Koppelmann, Guan Xuetao, Max Filippov, qemu-arm,
qemu-ppc
From: Sergey Fedorov <serge.fdrv@gmail.com>
In user mode, there's only a static address translation, TBs are always
invalidated properly and direct jumps are reset when mapping change.
Thus the destination address is always valid for direct jumps and
there's no need to restrict it to the pages the TB resides in.
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Cc: Riku Voipio <riku.voipio@iki.fi>
Cc: Blue Swirl <blauwirbel@gmail.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
Changes in v5:
* Don't check for in_superpage() in target-alpha/translate.c for
user-mode
Changes in v4:
* Explanatory comments moved to tcg_gen_goto_tb()
declaration comment
* Cc'ed usermode maintainers in commit message excplicitly
target-alpha/translate.c | 4 ++++
target-arm/translate-a64.c | 2 ++
target-arm/translate.c | 18 ++++++++++++------
target-cris/translate.c | 18 ++++++++++++------
target-i386/translate.c | 23 ++++++++++++++---------
target-lm32/translate.c | 21 +++++++++++++++------
target-m68k/translate.c | 18 ++++++++++++------
target-microblaze/translate.c | 15 +++++++++++----
target-mips/translate.c | 20 +++++++++++++++-----
target-moxie/translate.c | 21 +++++++++++++++------
target-openrisc/translate.c | 20 +++++++++++++++-----
target-ppc/translate.c | 20 +++++++++++++++-----
target-s390x/translate.c | 17 +++++++++++------
target-sh4/translate.c | 21 +++++++++++++++------
target-sparc/translate.c | 24 +++++++++++++++++-------
target-tricore/translate.c | 20 +++++++++++++++-----
target-unicore32/translate.c | 16 +++++++++++-----
target-xtensa/translate.c | 4 ++++
tcg/tcg-op.h | 9 ++++++---
19 files changed, 221 insertions(+), 90 deletions(-)
diff --git a/target-alpha/translate.c b/target-alpha/translate.c
index 5b86992dd367..8c2183a418e3 100644
--- a/target-alpha/translate.c
+++ b/target-alpha/translate.c
@@ -460,12 +460,16 @@ static bool use_goto_tb(DisasContext *ctx, uint64_t dest)
|| ctx->singlestep_enabled || singlestep) {
return false;
}
+#ifndef CONFIG_USER_ONLY
/* If the destination is in the superpage, the page perms can't change. */
if (in_superpage(ctx, dest)) {
return true;
}
/* Check for the dest on the same page as the start of the TB. */
return ((ctx->tb->pc ^ dest) & TARGET_PAGE_MASK) == 0;
+#else
+ return true;
+#endif
}
static ExitStatus gen_bdirect(DisasContext *ctx, int ra, int32_t disp)
diff --git a/target-arm/translate-a64.c b/target-arm/translate-a64.c
index b13cff756ad1..476d677f22d7 100644
--- a/target-arm/translate-a64.c
+++ b/target-arm/translate-a64.c
@@ -274,10 +274,12 @@ static inline bool use_goto_tb(DisasContext *s, int n, uint64_t dest)
return false;
}
+#ifndef CONFIG_USER_ONLY
/* Only link tbs from inside the same guest page */
if ((s->tb->pc & TARGET_PAGE_MASK) != (dest & TARGET_PAGE_MASK)) {
return false;
}
+#endif
return true;
}
diff --git a/target-arm/translate.c b/target-arm/translate.c
index 34196a821772..a43b1f61cf77 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -4049,16 +4049,22 @@ static int disas_vfp_insn(DisasContext *s, uint32_t insn)
return 0;
}
-static inline void gen_goto_tb(DisasContext *s, int n, target_ulong dest)
+static inline bool use_goto_tb(DisasContext *s, target_ulong dest)
{
- TranslationBlock *tb;
+#ifndef CONFIG_USER_ONLY
+ return (s->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
+ ((s->pc - 1) & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
- tb = s->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
- ((s->pc - 1) & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
+static inline void gen_goto_tb(DisasContext *s, int n, target_ulong dest)
+{
+ if (use_goto_tb(s, dest)) {
tcg_gen_goto_tb(n);
gen_set_pc_im(s, dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)s->tb + n);
} else {
gen_set_pc_im(s, dest);
tcg_gen_exit_tb(0);
diff --git a/target-cris/translate.c b/target-cris/translate.c
index 9c8ff8f2308a..f28b1999a786 100644
--- a/target-cris/translate.c
+++ b/target-cris/translate.c
@@ -520,16 +520,22 @@ static void t_gen_cc_jmp(TCGv pc_true, TCGv pc_false)
gen_set_label(l1);
}
-static void gen_goto_tb(DisasContext *dc, int n, target_ulong dest)
+static inline bool use_goto_tb(DisasContext *dc, target_ulong dest)
{
- TranslationBlock *tb;
- tb = dc->tb;
+#ifndef CONFIG_USER_ONLY
+ return (dc->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
+ (dc->ppc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
- (dc->ppc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
+static void gen_goto_tb(DisasContext *dc, int n, target_ulong dest)
+{
+ if (use_goto_tb(dc, dest)) {
tcg_gen_goto_tb(n);
tcg_gen_movi_tl(env_pc, dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)dc->tb + n);
} else {
tcg_gen_movi_tl(env_pc, dest);
tcg_gen_exit_tb(0);
diff --git a/target-i386/translate.c b/target-i386/translate.c
index 058d85a1abf7..868c26244ba7 100644
--- a/target-i386/translate.c
+++ b/target-i386/translate.c
@@ -2085,20 +2085,25 @@ static inline int insn_const_size(TCGMemOp ot)
}
}
+static inline bool use_goto_tb(DisasContext *s, target_ulong pc)
+{
+#ifndef CONFIG_USER_ONLY
+ return (pc & TARGET_PAGE_MASK) == (s->tb->pc & TARGET_PAGE_MASK) ||
+ (pc & TARGET_PAGE_MASK) == (s->pc_start & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
static inline void gen_goto_tb(DisasContext *s, int tb_num, target_ulong eip)
{
- TranslationBlock *tb;
- target_ulong pc;
-
- pc = s->cs_base + eip;
- tb = s->tb;
- /* NOTE: we handle the case where the TB spans two pages here */
- if ((pc & TARGET_PAGE_MASK) == (tb->pc & TARGET_PAGE_MASK) ||
- (pc & TARGET_PAGE_MASK) == (s->pc_start & TARGET_PAGE_MASK)) {
+ target_ulong pc = s->cs_base + eip;
+
+ if (use_goto_tb(s, pc)) {
/* jump to same page: we can use a direct jump */
tcg_gen_goto_tb(tb_num);
gen_jmp_im(eip);
- tcg_gen_exit_tb((uintptr_t)tb + tb_num);
+ tcg_gen_exit_tb((uintptr_t)s->tb + tb_num);
} else {
/* jump to another page: currently not optimized */
gen_jmp_im(eip);
diff --git a/target-lm32/translate.c b/target-lm32/translate.c
index 256a51f8498f..dd972f5b8c59 100644
--- a/target-lm32/translate.c
+++ b/target-lm32/translate.c
@@ -133,16 +133,25 @@ static inline void t_gen_illegal_insn(DisasContext *dc)
gen_helper_ill(cpu_env);
}
-static void gen_goto_tb(DisasContext *dc, int n, target_ulong dest)
+static inline bool use_goto_tb(DisasContext *dc, target_ulong dest)
{
- TranslationBlock *tb;
+ if (unlikely(dc->singlestep_enabled)) {
+ return false;
+ }
+
+#ifndef CONFIG_USER_ONLY
+ return (dc->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
- tb = dc->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) &&
- likely(!dc->singlestep_enabled)) {
+static void gen_goto_tb(DisasContext *dc, int n, target_ulong dest)
+{
+ if (use_goto_tb(dc, dest)) {
tcg_gen_goto_tb(n);
tcg_gen_movi_tl(cpu_pc, dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)dc->tb + n);
} else {
tcg_gen_movi_tl(cpu_pc, dest);
if (dc->singlestep_enabled) {
diff --git a/target-m68k/translate.c b/target-m68k/translate.c
index e2ce6c615e07..e46356e44c78 100644
--- a/target-m68k/translate.c
+++ b/target-m68k/translate.c
@@ -852,19 +852,25 @@ static inline void gen_addr_fault(DisasContext *s)
} \
} while (0)
+static inline bool use_goto_tb(DisasContext *s, uint32_t dest)
+{
+#ifndef CONFIG_USER_ONLY
+ return (s->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
+ (s->insn_pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
/* Generate a jump to an immediate address. */
static void gen_jmp_tb(DisasContext *s, int n, uint32_t dest)
{
- TranslationBlock *tb;
-
- tb = s->tb;
if (unlikely(s->singlestep_enabled)) {
gen_exception(s, dest, EXCP_DEBUG);
- } else if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) ||
- (s->insn_pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
+ } else if (use_goto_tb(s, dest)) {
tcg_gen_goto_tb(n);
tcg_gen_movi_i32(QREG_PC, dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)s->tb + n);
} else {
gen_jmp_im(s, dest);
tcg_gen_exit_tb(0);
diff --git a/target-microblaze/translate.c b/target-microblaze/translate.c
index f944965a14e1..a7a8ac8f995f 100644
--- a/target-microblaze/translate.c
+++ b/target-microblaze/translate.c
@@ -124,14 +124,21 @@ static inline void t_gen_raise_exception(DisasContext *dc, uint32_t index)
dc->is_jmp = DISAS_UPDATE;
}
+static inline bool use_goto_tb(DisasContext *dc, target_ulong dest)
+{
+#ifndef CONFIG_USER_ONLY
+ return (dc->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
static void gen_goto_tb(DisasContext *dc, int n, target_ulong dest)
{
- TranslationBlock *tb;
- tb = dc->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
+ if (use_goto_tb(dc, dest)) {
tcg_gen_goto_tb(n);
tcg_gen_movi_tl(cpu_SR[SR_PC], dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)dc->tb + n);
} else {
tcg_gen_movi_tl(cpu_SR[SR_PC], dest);
tcg_gen_exit_tb(0);
diff --git a/target-mips/translate.c b/target-mips/translate.c
index a3a05ec66dd2..ddfb9244d7e3 100644
--- a/target-mips/translate.c
+++ b/target-mips/translate.c
@@ -4191,15 +4191,25 @@ static void gen_trap (DisasContext *ctx, uint32_t opc,
tcg_temp_free(t1);
}
+static inline bool use_goto_tb(DisasContext *ctx, target_ulong dest)
+{
+ if (unlikely(ctx->singlestep_enabled)) {
+ return false;
+ }
+
+#ifndef CONFIG_USER_ONLY
+ return (ctx->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
static inline void gen_goto_tb(DisasContext *ctx, int n, target_ulong dest)
{
- TranslationBlock *tb;
- tb = ctx->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) &&
- likely(!ctx->singlestep_enabled)) {
+ if (use_goto_tb(ctx, dest)) {
tcg_gen_goto_tb(n);
gen_save_pc(dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)ctx->tb + n);
} else {
gen_save_pc(dest);
if (ctx->singlestep_enabled) {
diff --git a/target-moxie/translate.c b/target-moxie/translate.c
index a437e2ab6026..58200c25d3f4 100644
--- a/target-moxie/translate.c
+++ b/target-moxie/translate.c
@@ -121,17 +121,26 @@ void moxie_translate_init(void)
done_init = 1;
}
+static inline bool use_goto_tb(DisasContext *ctx, target_ulong dest)
+{
+ if (unlikely(ctx->singlestep_enabled)) {
+ return false;
+ }
+
+#ifndef CONFIG_USER_ONLY
+ return (ctx->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
static inline void gen_goto_tb(CPUMoxieState *env, DisasContext *ctx,
int n, target_ulong dest)
{
- TranslationBlock *tb;
- tb = ctx->tb;
-
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) &&
- !ctx->singlestep_enabled) {
+ if (use_goto_tb(ctx, dest)) {
tcg_gen_goto_tb(n);
tcg_gen_movi_i32(cpu_pc, dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)ctx->tb + n);
} else {
tcg_gen_movi_i32(cpu_pc, dest);
if (ctx->singlestep_enabled) {
diff --git a/target-openrisc/translate.c b/target-openrisc/translate.c
index 5d0ab442a872..d4f1f260e425 100644
--- a/target-openrisc/translate.c
+++ b/target-openrisc/translate.c
@@ -190,15 +190,25 @@ static void check_ov64s(DisasContext *dc)
}
#endif*/
+static inline bool use_goto_tb(DisasContext *dc, target_ulong dest)
+{
+ if (unlikely(dc->singlestep_enabled)) {
+ return false;
+ }
+
+#ifndef CONFIG_USER_ONLY
+ return (dc->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
static void gen_goto_tb(DisasContext *dc, int n, target_ulong dest)
{
- TranslationBlock *tb;
- tb = dc->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) &&
- likely(!dc->singlestep_enabled)) {
+ if (use_goto_tb(dc, dest)) {
tcg_gen_movi_tl(cpu_pc, dest);
tcg_gen_goto_tb(n);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)dc->tb + n);
} else {
tcg_gen_movi_tl(cpu_pc, dest);
if (dc->singlestep_enabled) {
diff --git a/target-ppc/translate.c b/target-ppc/translate.c
index b3860ecdea9c..d485d7c7cb18 100644
--- a/target-ppc/translate.c
+++ b/target-ppc/translate.c
@@ -3822,19 +3822,29 @@ static inline void gen_update_cfar(DisasContext *ctx, target_ulong nip)
#endif
}
+static inline bool use_goto_tb(DisasContext *ctx, target_ulong dest)
+{
+ if (unlikely(ctx->singlestep_enabled)) {
+ return false;
+ }
+
+#ifndef CONFIG_USER_ONLY
+ return (ctx->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
/*** Branch ***/
static inline void gen_goto_tb(DisasContext *ctx, int n, target_ulong dest)
{
- TranslationBlock *tb;
- tb = ctx->tb;
if (NARROW_MODE(ctx)) {
dest = (uint32_t) dest;
}
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) &&
- likely(!ctx->singlestep_enabled)) {
+ if (use_goto_tb(ctx, dest)) {
tcg_gen_goto_tb(n);
tcg_gen_movi_tl(cpu_nip, dest & ~3);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)ctx->tb + n);
} else {
tcg_gen_movi_tl(cpu_nip, dest & ~3);
if (unlikely(ctx->singlestep_enabled)) {
diff --git a/target-s390x/translate.c b/target-s390x/translate.c
index c5179fe05d7e..e99eb5cb0169 100644
--- a/target-s390x/translate.c
+++ b/target-s390x/translate.c
@@ -608,12 +608,17 @@ static void gen_op_calc_cc(DisasContext *s)
static int use_goto_tb(DisasContext *s, uint64_t dest)
{
- /* NOTE: we handle the case where the TB spans two pages here */
- return (((dest & TARGET_PAGE_MASK) == (s->tb->pc & TARGET_PAGE_MASK)
- || (dest & TARGET_PAGE_MASK) == (s->pc & TARGET_PAGE_MASK))
- && !s->singlestep_enabled
- && !(s->tb->cflags & CF_LAST_IO)
- && !(s->tb->flags & FLAG_MASK_PER));
+ if (unlikely(s->singlestep_enabled) ||
+ (s->tb->cflags & CF_LAST_IO) ||
+ (s->tb->flags & FLAG_MASK_PER)) {
+ return false;
+ }
+#ifndef CONFIG_USER_ONLY
+ return (dest & TARGET_PAGE_MASK) == (s->tb->pc & TARGET_PAGE_MASK) ||
+ (dest & TARGET_PAGE_MASK) == (s->pc & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
}
static void account_noninline_branch(DisasContext *s, int cc_op)
diff --git a/target-sh4/translate.c b/target-sh4/translate.c
index 7c189680a7a4..53f782c05467 100644
--- a/target-sh4/translate.c
+++ b/target-sh4/translate.c
@@ -205,17 +205,26 @@ static void gen_write_sr(TCGv src)
tcg_gen_andi_i32(cpu_sr_t, cpu_sr_t, 1);
}
-static void gen_goto_tb(DisasContext * ctx, int n, target_ulong dest)
+static inline bool use_goto_tb(DisasContext *ctx, target_ulong dest)
{
- TranslationBlock *tb;
- tb = ctx->tb;
+ if (unlikely(ctx->singlestep_enabled)) {
+ return false;
+ }
+
+#ifndef CONFIG_USER_ONLY
+ return (ctx->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) &&
- !ctx->singlestep_enabled) {
+static void gen_goto_tb(DisasContext *ctx, int n, target_ulong dest)
+{
+ if (use_goto_tb(ctx, dest)) {
/* Use a direct jump if in same page and singlestep not enabled */
tcg_gen_goto_tb(n);
tcg_gen_movi_i32(cpu_pc, dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)ctx->tb + n);
} else {
tcg_gen_movi_i32(cpu_pc, dest);
if (ctx->singlestep_enabled)
diff --git a/target-sparc/translate.c b/target-sparc/translate.c
index 7998ff57bf09..d154e3f7b633 100644
--- a/target-sparc/translate.c
+++ b/target-sparc/translate.c
@@ -303,20 +303,30 @@ static inline TCGv gen_dest_gpr(DisasContext *dc, int reg)
}
}
+static inline bool use_goto_tb(DisasContext *s, target_ulong pc,
+ target_ulong npc)
+{
+ if (unlikely(s->singlestep)) {
+ return false;
+ }
+
+#ifndef CONFIG_USER_ONLY
+ return (pc & TARGET_PAGE_MASK) == (s->tb->pc & TARGET_PAGE_MASK) &&
+ (npc & TARGET_PAGE_MASK) == (s->tb->pc & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
static inline void gen_goto_tb(DisasContext *s, int tb_num,
target_ulong pc, target_ulong npc)
{
- TranslationBlock *tb;
-
- tb = s->tb;
- if ((pc & TARGET_PAGE_MASK) == (tb->pc & TARGET_PAGE_MASK) &&
- (npc & TARGET_PAGE_MASK) == (tb->pc & TARGET_PAGE_MASK) &&
- !s->singlestep) {
+ if (use_goto_tb(s, pc, npc)) {
/* jump to same page: we can use a direct jump */
tcg_gen_goto_tb(tb_num);
tcg_gen_movi_tl(cpu_pc, pc);
tcg_gen_movi_tl(cpu_npc, npc);
- tcg_gen_exit_tb((uintptr_t)tb + tb_num);
+ tcg_gen_exit_tb((uintptr_t)s->tb + tb_num);
} else {
/* jump to another page: currently not optimized */
tcg_gen_movi_tl(cpu_pc, pc);
diff --git a/target-tricore/translate.c b/target-tricore/translate.c
index 912bf226bedc..0237e7bea835 100644
--- a/target-tricore/translate.c
+++ b/target-tricore/translate.c
@@ -3236,15 +3236,25 @@ static inline void gen_save_pc(target_ulong pc)
tcg_gen_movi_tl(cpu_PC, pc);
}
+static inline bool use_goto_tb(DisasContext *ctx, target_ulong dest)
+{
+ if (unlikely(ctx->singlestep_enabled)) {
+ return false;
+ }
+
+#ifndef CONFIG_USER_ONLY
+ return (ctx->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
+
static inline void gen_goto_tb(DisasContext *ctx, int n, target_ulong dest)
{
- TranslationBlock *tb;
- tb = ctx->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK) &&
- likely(!ctx->singlestep_enabled)) {
+ if (use_goto_tb(ctx, dest)) {
tcg_gen_goto_tb(n);
gen_save_pc(dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)ctx->tb + n);
} else {
gen_save_pc(dest);
if (ctx->singlestep_enabled) {
diff --git a/target-unicore32/translate.c b/target-unicore32/translate.c
index 39af3af05f15..307f7b205924 100644
--- a/target-unicore32/translate.c
+++ b/target-unicore32/translate.c
@@ -1089,15 +1089,21 @@ static void disas_ucf64_insn(CPUUniCore32State *env, DisasContext *s, uint32_t i
}
}
-static inline void gen_goto_tb(DisasContext *s, int n, uint32_t dest)
+static inline bool use_goto_tb(DisasContext *s, uint32_t dest)
{
- TranslationBlock *tb;
+#ifndef CONFIG_USER_ONLY
+ return (s->tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK);
+#else
+ return true;
+#endif
+}
- tb = s->tb;
- if ((tb->pc & TARGET_PAGE_MASK) == (dest & TARGET_PAGE_MASK)) {
+static inline void gen_goto_tb(DisasContext *s, int n, uint32_t dest)
+{
+ if (use_goto_tb(s, dest)) {
tcg_gen_goto_tb(n);
gen_set_pc_im(dest);
- tcg_gen_exit_tb((uintptr_t)tb + n);
+ tcg_gen_exit_tb((uintptr_t)s->tb + n);
} else {
gen_set_pc_im(dest);
tcg_gen_exit_tb(0);
diff --git a/target-xtensa/translate.c b/target-xtensa/translate.c
index 989448846902..9eac56e2a5bc 100644
--- a/target-xtensa/translate.c
+++ b/target-xtensa/translate.c
@@ -418,9 +418,11 @@ static void gen_jump(DisasContext *dc, TCGv dest)
static void gen_jumpi(DisasContext *dc, uint32_t dest, int slot)
{
TCGv_i32 tmp = tcg_const_i32(dest);
+#ifndef CONFIG_USER_ONLY
if (((dc->tb->pc ^ dest) & TARGET_PAGE_MASK) != 0) {
slot = -1;
}
+#endif
gen_jump_slot(dc, tmp, slot);
tcg_temp_free(tmp);
}
@@ -446,9 +448,11 @@ static void gen_callw(DisasContext *dc, int callinc, TCGv_i32 dest)
static void gen_callwi(DisasContext *dc, int callinc, uint32_t dest, int slot)
{
TCGv_i32 tmp = tcg_const_i32(dest);
+#ifndef CONFIG_USER_ONLY
if (((dc->tb->pc ^ dest) & TARGET_PAGE_MASK) != 0) {
slot = -1;
}
+#endif
gen_callw_slot(dc, callinc, tmp, slot);
tcg_temp_free(tmp);
}
diff --git a/tcg/tcg-op.h b/tcg/tcg-op.h
index ace39619ef89..f217e8074715 100644
--- a/tcg/tcg-op.h
+++ b/tcg/tcg-op.h
@@ -759,9 +759,12 @@ static inline void tcg_gen_exit_tb(uintptr_t val)
*
* See tcg/README for more info about this TCG operation.
*
- * NOTE: Direct jumps with goto_tb are only safe within the pages this TB
- * resides in because we don't take care of direct jumps when address mapping
- * changes, e.g. in tlb_flush().
+ * NOTE: In softmmu emulation, direct jumps with goto_tb are only safe within
+ * the pages this TB resides in because we don't take care of direct jumps when
+ * address mapping changes, e.g. in tlb_flush(). In user mode, there's only a
+ * static address translation, so the destination address is always valid, TBs
+ * are always invalidated properly, and direct jumps are reset when mapping
+ * changes.
*/
void tcg_gen_goto_tb(unsigned idx);
--
2.8.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
` (9 preceding siblings ...)
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 10/10] tcg: Allow goto_tb to any target PC in user mode Sergey Fedorov
@ 2016-04-29 16:46 ` Richard Henderson
10 siblings, 0 replies; 12+ messages in thread
From: Richard Henderson @ 2016-04-29 16:46 UTC (permalink / raw)
To: Sergey Fedorov, qemu-devel
Cc: Alex Bennée, Sergey Fedorov, Paolo Bonzini, Peter Crosthwaite
On 04/28/2016 02:33 PM, Sergey Fedorov wrote:
> From: Sergey Fedorov <serge.fdrv@gmail.com>
>
> This series combines a set of patches which is meant to improve overall code
> structure and readability of the direct block chaining mechanism. The other
> point is to make a step towards thread safety of TB chainig.
>
> This series is based on commit: 1d02fa9e045b ("translate-all: Adjust 256mb
> testing for mips64") from git://github.com/rth7680/qemu.git tcg-next and is
> available at git://github.com/sergefdrv/qemu.git tb-chaining-cleanup-v5
>
> Summary of changes:
> Changes in v5:
> * Fixed rebase conflicts
> * Don't check for in_superpage() in target-alpha/translate.c for
> user-mode [PATCH v5 10/10]
> Changes in v4:
> * Removed assert from tb_add_jump() [PATCH v4 02/10]
> * Added comment on TB stuff synchronization [PATCH v4 04/10]
> * Documented tcg_gen_goto_tb() and moved its usage notes there
> [PATCH v4 09/10] and [PATCH v4 10/10]
> * Cc'ed usermode maintainers in commit message [PATCH v4 10/10]
> Changes in v3:
> * New patch to clean up safety checks [PATCH v3 09/10]
> * New patch to eliminate unneeded checks in user-mode [PATCH v3 10/10]
> Changes in v2:
> * Eliminated duplicate dereference of 'ptb' in tb_jmp_remove() [PATCH v2 2/8]
> * Tweaked a comment [PATCH v2 4/8]
> * Complete rewrite [PATCH v2 5/8]
> * Tweaked a comment; eliminated duplicate dereference of 'ptb' in
> tb_jmp_unlink() [PATCH v2 8/8]
>
> Sergey Fedorov (10):
> tcg: Clean up direct block chaining data fields
> tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB
> tcg: Rearrange tb_link_page() to avoid forward declaration
> tcg: Init TB's direct jumps before making it visible
> tcg: Clarify thread safety check in tb_add_jump()
> tcg: Rename tb_jmp_remove() to tb_remove_from_jmp_list()
> tcg: Extract removing of jumps to TB from tb_phys_invalidate()
> tcg: Clean up tb_jmp_unlink()
> tcg: Clean up direct block chaining safety checks
> tcg: Allow goto_tb to any target PC in user mode
Applied to tcg-next.
r~
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2016-04-29 16:46 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-28 21:33 [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 01/10] tcg: Clean up direct block chaining data fields Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 02/10] tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 03/10] tcg: Rearrange tb_link_page() to avoid forward declaration Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 04/10] tcg: Init TB's direct jumps before making it visible Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 05/10] tcg: Clarify thread safety check in tb_add_jump() Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 06/10] tcg: Rename tb_jmp_remove() to tb_remove_from_jmp_list() Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 07/10] tcg: Extract removing of jumps to TB from tb_phys_invalidate() Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 08/10] tcg: Clean up tb_jmp_unlink() Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 09/10] tcg: Clean up direct block chaining safety checks Sergey Fedorov
2016-04-28 21:33 ` [Qemu-devel] [PATCH v5 10/10] tcg: Allow goto_tb to any target PC in user mode Sergey Fedorov
2016-04-29 16:46 ` [Qemu-devel] [PATCH v5 00/10] tcg: Direct block chaining clean-up Richard Henderson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.