* [PATCH 0/9] cputlb: Various cleanups
@ 2020-01-09 2:48 Richard Henderson
2020-01-09 2:48 ` [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked Richard Henderson
` (9 more replies)
0 siblings, 10 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:48 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
I had a conversation with Alistair Francis at KVM forum about
being able to represent ASIDs "properly". This lead to the idea
that target-specific code might be able to cache TLBs outside of
the "main" NB_MMU_MODES -- possibly thousands of them.
This goes nowhere near that far. But it does begin edging toward
the possibility of having a
struct CPUTLBSaved {
CPUTLBDesc d;
CPUTLBDescFast f;
};
by moving some of the most basic routines to use CPUTLBDesc and
CPUTLBDescFast directly instead of always using an mmu_idx.
I'm not sure how much time I'll have to go further along these
lines, but what I have so far still looks like a cleanup.
r~
Richard Henderson (9):
cputlb: Merge tlb_table_flush_by_mmuidx into
tlb_flush_one_mmuidx_locked
cputlb: Make tlb_n_entries private to cputlb.c
cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb
cputlb: Hoist tlb portions in tlb_mmu_resize_locked
cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked
cputlb: Split out tlb_mmu_flush_locked
cputlb: Partially merge tlb_dyn_init into tlb_init
cputlb: Initialize tlbs as flushed
cputlb: Hoist timestamp outside of loops over tlbs
include/exec/cpu_ldst.h | 5 --
accel/tcg/cputlb.c | 120 +++++++++++++++++++++-------------------
2 files changed, 64 insertions(+), 61 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
@ 2020-01-09 2:48 ` Richard Henderson
2020-01-20 0:34 ` Alistair Francis
2020-01-20 13:34 ` Alex Bennée
2020-01-09 2:49 ` [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c Richard Henderson
` (8 subsequent siblings)
9 siblings, 2 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:48 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
There is only one caller for tlb_table_flush_by_mmuidx. Place
the result at the earlier line number, due to an expected user
in the near future.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cputlb.c | 19 +++++++------------
1 file changed, 7 insertions(+), 12 deletions(-)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index a991ea2964..1a81886e58 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -224,11 +224,16 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
}
}
-static inline void tlb_table_flush_by_mmuidx(CPUArchState *env, int mmu_idx)
+static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
{
tlb_mmu_resize_locked(env, mmu_idx);
- memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
env_tlb(env)->d[mmu_idx].n_used_entries = 0;
+ env_tlb(env)->d[mmu_idx].large_page_addr = -1;
+ env_tlb(env)->d[mmu_idx].large_page_mask = -1;
+ env_tlb(env)->d[mmu_idx].vindex = 0;
+ memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
+ memset(env_tlb(env)->d[mmu_idx].vtable, -1,
+ sizeof(env_tlb(env)->d[0].vtable));
}
static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
@@ -289,16 +294,6 @@ void tlb_flush_counts(size_t *pfull, size_t *ppart, size_t *pelide)
*pelide = elide;
}
-static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
-{
- tlb_table_flush_by_mmuidx(env, mmu_idx);
- env_tlb(env)->d[mmu_idx].large_page_addr = -1;
- env_tlb(env)->d[mmu_idx].large_page_mask = -1;
- env_tlb(env)->d[mmu_idx].vindex = 0;
- memset(env_tlb(env)->d[mmu_idx].vtable, -1,
- sizeof(env_tlb(env)->d[0].vtable));
-}
-
static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
{
CPUArchState *env = cpu->env_ptr;
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
2020-01-09 2:48 ` [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked Richard Henderson
@ 2020-01-09 2:49 ` Richard Henderson
2020-01-20 0:36 ` Alistair Francis
` (2 more replies)
2020-01-09 2:49 ` [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb Richard Henderson
` (7 subsequent siblings)
9 siblings, 3 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:49 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
There are no users of this function outside cputlb.c,
and its interface will change in the next patch.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/cpu_ldst.h | 5 -----
accel/tcg/cputlb.c | 5 +++++
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h
index a46116167c..53de19753a 100644
--- a/include/exec/cpu_ldst.h
+++ b/include/exec/cpu_ldst.h
@@ -234,11 +234,6 @@ static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx,
return (addr >> TARGET_PAGE_BITS) & size_mask;
}
-static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
-{
- return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
-}
-
/* Find the TLB entry corresponding to the mmu_idx + address pair. */
static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx,
target_ulong addr)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 1a81886e58..e4a8ed9534 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -80,6 +80,11 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
#define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
+static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
+{
+ return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
+}
+
static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
{
return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
2020-01-09 2:48 ` [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked Richard Henderson
2020-01-09 2:49 ` [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c Richard Henderson
@ 2020-01-09 2:49 ` Richard Henderson
2020-01-20 0:37 ` Alistair Francis
` (3 more replies)
2020-01-09 2:49 ` [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked Richard Henderson
` (6 subsequent siblings)
9 siblings, 4 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:49 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
We do not need the entire CPUArchState to compute these values.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cputlb.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index e4a8ed9534..49c605b6d8 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -80,14 +80,14 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
#define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
-static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
+static inline size_t tlb_n_entries(CPUTLBDescFast *fast)
{
- return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
+ return (fast->mask >> CPU_TLB_ENTRY_BITS) + 1;
}
-static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
+static inline size_t sizeof_tlb(CPUTLBDescFast *fast)
{
- return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
+ return fast->mask + (1 << CPU_TLB_ENTRY_BITS);
}
static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
@@ -156,7 +156,7 @@ static void tlb_dyn_init(CPUArchState *env)
static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
{
CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
- size_t old_size = tlb_n_entries(env, mmu_idx);
+ size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
size_t rate;
size_t new_size = old_size;
int64_t now = get_clock_realtime();
@@ -236,7 +236,8 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
env_tlb(env)->d[mmu_idx].large_page_addr = -1;
env_tlb(env)->d[mmu_idx].large_page_mask = -1;
env_tlb(env)->d[mmu_idx].vindex = 0;
- memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
+ memset(env_tlb(env)->f[mmu_idx].table, -1,
+ sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
memset(env_tlb(env)->d[mmu_idx].vtable, -1,
sizeof(env_tlb(env)->d[0].vtable));
}
@@ -622,7 +623,7 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length)
qemu_spin_lock(&env_tlb(env)->c.lock);
for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) {
unsigned int i;
- unsigned int n = tlb_n_entries(env, mmu_idx);
+ unsigned int n = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
for (i = 0; i < n; i++) {
tlb_reset_dirty_range_locked(&env_tlb(env)->f[mmu_idx].table[i],
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
` (2 preceding siblings ...)
2020-01-09 2:49 ` [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb Richard Henderson
@ 2020-01-09 2:49 ` Richard Henderson
2020-01-20 8:58 ` Philippe Mathieu-Daudé
` (2 more replies)
2020-01-09 2:49 ` [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked Richard Henderson
` (5 subsequent siblings)
9 siblings, 3 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:49 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
No functional change, but the smaller expressions make
the code easier to read.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cputlb.c | 35 +++++++++++++++++------------------
1 file changed, 17 insertions(+), 18 deletions(-)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 49c605b6d8..c7dc1dc85a 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -115,8 +115,8 @@ static void tlb_dyn_init(CPUArchState *env)
/**
* tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
- * @env: CPU that owns the TLB
- * @mmu_idx: MMU index of the TLB
+ * @desc: The CPUTLBDesc portion of the TLB
+ * @fast: The CPUTLBDescFast portion of the same TLB
*
* Called with tlb_lock_held.
*
@@ -153,10 +153,9 @@ static void tlb_dyn_init(CPUArchState *env)
* high), since otherwise we are likely to have a significant amount of
* conflict misses.
*/
-static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
+static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
{
- CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
- size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
+ size_t old_size = tlb_n_entries(fast);
size_t rate;
size_t new_size = old_size;
int64_t now = get_clock_realtime();
@@ -198,14 +197,15 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
return;
}
- g_free(env_tlb(env)->f[mmu_idx].table);
- g_free(env_tlb(env)->d[mmu_idx].iotlb);
+ g_free(fast->table);
+ g_free(desc->iotlb);
tlb_window_reset(desc, now, 0);
/* desc->n_used_entries is cleared by the caller */
- env_tlb(env)->f[mmu_idx].mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
- env_tlb(env)->f[mmu_idx].table = g_try_new(CPUTLBEntry, new_size);
- env_tlb(env)->d[mmu_idx].iotlb = g_try_new(CPUIOTLBEntry, new_size);
+ fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
+ fast->table = g_try_new(CPUTLBEntry, new_size);
+ desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
+
/*
* If the allocations fail, try smaller sizes. We just freed some
* memory, so going back to half of new_size has a good chance of working.
@@ -213,25 +213,24 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
* allocations to fail though, so we progressively reduce the allocation
* size, aborting if we cannot even allocate the smallest TLB we support.
*/
- while (env_tlb(env)->f[mmu_idx].table == NULL ||
- env_tlb(env)->d[mmu_idx].iotlb == NULL) {
+ while (fast->table == NULL || desc->iotlb == NULL) {
if (new_size == (1 << CPU_TLB_DYN_MIN_BITS)) {
error_report("%s: %s", __func__, strerror(errno));
abort();
}
new_size = MAX(new_size >> 1, 1 << CPU_TLB_DYN_MIN_BITS);
- env_tlb(env)->f[mmu_idx].mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
+ fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
- g_free(env_tlb(env)->f[mmu_idx].table);
- g_free(env_tlb(env)->d[mmu_idx].iotlb);
- env_tlb(env)->f[mmu_idx].table = g_try_new(CPUTLBEntry, new_size);
- env_tlb(env)->d[mmu_idx].iotlb = g_try_new(CPUIOTLBEntry, new_size);
+ g_free(fast->table);
+ g_free(desc->iotlb);
+ fast->table = g_try_new(CPUTLBEntry, new_size);
+ desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
}
}
static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
{
- tlb_mmu_resize_locked(env, mmu_idx);
+ tlb_mmu_resize_locked(&env_tlb(env)->d[mmu_idx], &env_tlb(env)->f[mmu_idx]);
env_tlb(env)->d[mmu_idx].n_used_entries = 0;
env_tlb(env)->d[mmu_idx].large_page_addr = -1;
env_tlb(env)->d[mmu_idx].large_page_mask = -1;
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
` (3 preceding siblings ...)
2020-01-09 2:49 ` [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked Richard Henderson
@ 2020-01-09 2:49 ` Richard Henderson
2020-01-20 8:58 ` Philippe Mathieu-Daudé
` (2 more replies)
2020-01-09 2:49 ` [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked Richard Henderson
` (4 subsequent siblings)
9 siblings, 3 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:49 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
No functional change, but the smaller expressions make
the code easier to read.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cputlb.c | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index c7dc1dc85a..eff427f137 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -230,15 +230,16 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
{
- tlb_mmu_resize_locked(&env_tlb(env)->d[mmu_idx], &env_tlb(env)->f[mmu_idx]);
- env_tlb(env)->d[mmu_idx].n_used_entries = 0;
- env_tlb(env)->d[mmu_idx].large_page_addr = -1;
- env_tlb(env)->d[mmu_idx].large_page_mask = -1;
- env_tlb(env)->d[mmu_idx].vindex = 0;
- memset(env_tlb(env)->f[mmu_idx].table, -1,
- sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
- memset(env_tlb(env)->d[mmu_idx].vtable, -1,
- sizeof(env_tlb(env)->d[0].vtable));
+ CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
+ CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
+
+ tlb_mmu_resize_locked(desc, fast);
+ desc->n_used_entries = 0;
+ desc->large_page_addr = -1;
+ desc->large_page_mask = -1;
+ desc->vindex = 0;
+ memset(fast->table, -1, sizeof_tlb(fast));
+ memset(desc->vtable, -1, sizeof(desc->vtable));
}
static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
` (4 preceding siblings ...)
2020-01-09 2:49 ` [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked Richard Henderson
@ 2020-01-09 2:49 ` Richard Henderson
2020-01-20 8:59 ` Philippe Mathieu-Daudé
` (2 more replies)
2020-01-09 2:49 ` [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init Richard Henderson
` (3 subsequent siblings)
9 siblings, 3 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:49 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
We will want to be able to flush a tlb without resizing.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cputlb.c | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index eff427f137..e60e501334 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -228,12 +228,8 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
}
}
-static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
+static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
{
- CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
- CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
-
- tlb_mmu_resize_locked(desc, fast);
desc->n_used_entries = 0;
desc->large_page_addr = -1;
desc->large_page_mask = -1;
@@ -242,6 +238,15 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
memset(desc->vtable, -1, sizeof(desc->vtable));
}
+static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
+{
+ CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
+ CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
+
+ tlb_mmu_resize_locked(desc, fast);
+ tlb_mmu_flush_locked(desc, fast);
+}
+
static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
{
env_tlb(env)->d[mmu_idx].n_used_entries++;
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
` (5 preceding siblings ...)
2020-01-09 2:49 ` [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked Richard Henderson
@ 2020-01-09 2:49 ` Richard Henderson
2020-01-20 9:01 ` Philippe Mathieu-Daudé
` (2 more replies)
2020-01-09 2:49 ` [PATCH 8/9] cputlb: Initialize tlbs as flushed Richard Henderson
` (2 subsequent siblings)
9 siblings, 3 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:49 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
Merge into the only caller, but at the same time split
out tlb_mmu_init to initialize a single tlb entry.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cputlb.c | 33 ++++++++++++++++-----------------
1 file changed, 16 insertions(+), 17 deletions(-)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index e60e501334..c7c34b185b 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -97,22 +97,6 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
desc->window_max_entries = max_entries;
}
-static void tlb_dyn_init(CPUArchState *env)
-{
- int i;
-
- for (i = 0; i < NB_MMU_MODES; i++) {
- CPUTLBDesc *desc = &env_tlb(env)->d[i];
- size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS;
-
- tlb_window_reset(desc, get_clock_realtime(), 0);
- desc->n_used_entries = 0;
- env_tlb(env)->f[i].mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
- env_tlb(env)->f[i].table = g_new(CPUTLBEntry, n_entries);
- env_tlb(env)->d[i].iotlb = g_new(CPUIOTLBEntry, n_entries);
- }
-}
-
/**
* tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
* @desc: The CPUTLBDesc portion of the TLB
@@ -247,6 +231,17 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
tlb_mmu_flush_locked(desc, fast);
}
+static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
+{
+ size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS;
+
+ tlb_window_reset(desc, now, 0);
+ desc->n_used_entries = 0;
+ fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
+ fast->table = g_new(CPUTLBEntry, n_entries);
+ desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
+}
+
static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
{
env_tlb(env)->d[mmu_idx].n_used_entries++;
@@ -260,13 +255,17 @@ static inline void tlb_n_used_entries_dec(CPUArchState *env, uintptr_t mmu_idx)
void tlb_init(CPUState *cpu)
{
CPUArchState *env = cpu->env_ptr;
+ int64_t now = get_clock_realtime();
+ int i;
qemu_spin_init(&env_tlb(env)->c.lock);
/* Ensure that cpu_reset performs a full flush. */
env_tlb(env)->c.dirty = ALL_MMUIDX_BITS;
- tlb_dyn_init(env);
+ for (i = 0; i < NB_MMU_MODES; i++) {
+ tlb_mmu_init(&env_tlb(env)->d[i], &env_tlb(env)->f[i], now);
+ }
}
/* flush_all_helper: run fn across all cpus
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 8/9] cputlb: Initialize tlbs as flushed
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
` (6 preceding siblings ...)
2020-01-09 2:49 ` [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init Richard Henderson
@ 2020-01-09 2:49 ` Richard Henderson
2020-01-20 14:33 ` Alex Bennée
2020-01-20 23:01 ` Alistair Francis
2020-01-09 2:49 ` [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs Richard Henderson
2020-01-20 23:03 ` [PATCH 0/9] cputlb: Various cleanups Alistair Francis
9 siblings, 2 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:49 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
There's little point in leaving these data structures half initialized,
and relying on a flush to be done during reset.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cputlb.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index c7c34b185b..761e9d44d7 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -240,6 +240,7 @@ static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
fast->table = g_new(CPUTLBEntry, n_entries);
desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
+ tlb_mmu_flush_locked(desc, fast);
}
static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
@@ -260,8 +261,8 @@ void tlb_init(CPUState *cpu)
qemu_spin_init(&env_tlb(env)->c.lock);
- /* Ensure that cpu_reset performs a full flush. */
- env_tlb(env)->c.dirty = ALL_MMUIDX_BITS;
+ /* All tlbs are initialized flushed. */
+ env_tlb(env)->c.dirty = 0;
for (i = 0; i < NB_MMU_MODES; i++) {
tlb_mmu_init(&env_tlb(env)->d[i], &env_tlb(env)->f[i], now);
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
` (7 preceding siblings ...)
2020-01-09 2:49 ` [PATCH 8/9] cputlb: Initialize tlbs as flushed Richard Henderson
@ 2020-01-09 2:49 ` Richard Henderson
2020-01-20 9:02 ` Philippe Mathieu-Daudé
` (2 more replies)
2020-01-20 23:03 ` [PATCH 0/9] cputlb: Various cleanups Alistair Francis
9 siblings, 3 replies; 37+ messages in thread
From: Richard Henderson @ 2020-01-09 2:49 UTC (permalink / raw)
To: qemu-devel; +Cc: alistair.francis
Do not call get_clock_realtime() in tlb_mmu_resize_locked,
but hoist outside of any loop over a set of tlbs. This is
only two (indirect) callers, tlb_flush_by_mmuidx_async_work
and tlb_flush_page_locked, so not onerous.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cputlb.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 761e9d44d7..9f6cb36921 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -137,12 +137,12 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
* high), since otherwise we are likely to have a significant amount of
* conflict misses.
*/
-static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
+static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
+ int64_t now)
{
size_t old_size = tlb_n_entries(fast);
size_t rate;
size_t new_size = old_size;
- int64_t now = get_clock_realtime();
int64_t window_len_ms = 100;
int64_t window_len_ns = window_len_ms * 1000 * 1000;
bool window_expired = now > desc->window_begin_ns + window_len_ns;
@@ -222,12 +222,13 @@ static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
memset(desc->vtable, -1, sizeof(desc->vtable));
}
-static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
+static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx,
+ int64_t now)
{
CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
- tlb_mmu_resize_locked(desc, fast);
+ tlb_mmu_resize_locked(desc, fast, now);
tlb_mmu_flush_locked(desc, fast);
}
@@ -310,6 +311,7 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
CPUArchState *env = cpu->env_ptr;
uint16_t asked = data.host_int;
uint16_t all_dirty, work, to_clean;
+ int64_t now = get_clock_realtime();
assert_cpu_is_self(cpu);
@@ -324,7 +326,7 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
for (work = to_clean; work != 0; work &= work - 1) {
int mmu_idx = ctz32(work);
- tlb_flush_one_mmuidx_locked(env, mmu_idx);
+ tlb_flush_one_mmuidx_locked(env, mmu_idx, now);
}
qemu_spin_unlock(&env_tlb(env)->c.lock);
@@ -446,7 +448,7 @@ static void tlb_flush_page_locked(CPUArchState *env, int midx,
tlb_debug("forcing full flush midx %d ("
TARGET_FMT_lx "/" TARGET_FMT_lx ")\n",
midx, lp_addr, lp_mask);
- tlb_flush_one_mmuidx_locked(env, midx);
+ tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
} else {
if (tlb_flush_entry_locked(tlb_entry(env, midx, page), page)) {
tlb_n_used_entries_dec(env, midx);
--
2.20.1
^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked
2020-01-09 2:48 ` [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked Richard Henderson
@ 2020-01-20 0:34 ` Alistair Francis
2020-01-20 13:34 ` Alex Bennée
1 sibling, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 0:34 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:49 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> There is only one caller for tlb_table_flush_by_mmuidx. Place
> the result at the earlier line number, due to an expected user
> in the near future.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> accel/tcg/cputlb.c | 19 +++++++------------
> 1 file changed, 7 insertions(+), 12 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index a991ea2964..1a81886e58 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -224,11 +224,16 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> }
> }
>
> -static inline void tlb_table_flush_by_mmuidx(CPUArchState *env, int mmu_idx)
> +static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> {
> tlb_mmu_resize_locked(env, mmu_idx);
> - memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
> env_tlb(env)->d[mmu_idx].n_used_entries = 0;
> + env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> + env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> + env_tlb(env)->d[mmu_idx].vindex = 0;
> + memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
> + memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> + sizeof(env_tlb(env)->d[0].vtable));
> }
>
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> @@ -289,16 +294,6 @@ void tlb_flush_counts(size_t *pfull, size_t *ppart, size_t *pelide)
> *pelide = elide;
> }
>
> -static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> -{
> - tlb_table_flush_by_mmuidx(env, mmu_idx);
> - env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> - env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> - env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> - sizeof(env_tlb(env)->d[0].vtable));
> -}
> -
> static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
> {
> CPUArchState *env = cpu->env_ptr;
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c
2020-01-09 2:49 ` [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c Richard Henderson
@ 2020-01-20 0:36 ` Alistair Francis
2020-01-20 8:53 ` Philippe Mathieu-Daudé
2020-01-20 13:34 ` Alex Bennée
2 siblings, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 0:36 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:49 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> There are no users of this function outside cputlb.c,
> and its interface will change in the next patch.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> include/exec/cpu_ldst.h | 5 -----
> accel/tcg/cputlb.c | 5 +++++
> 2 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h
> index a46116167c..53de19753a 100644
> --- a/include/exec/cpu_ldst.h
> +++ b/include/exec/cpu_ldst.h
> @@ -234,11 +234,6 @@ static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx,
> return (addr >> TARGET_PAGE_BITS) & size_mask;
> }
>
> -static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> -{
> - return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> -}
> -
> /* Find the TLB entry corresponding to the mmu_idx + address pair. */
> static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx,
> target_ulong addr)
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 1a81886e58..e4a8ed9534 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -80,6 +80,11 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
> QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
> #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
>
> +static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> +{
> + return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> +}
> +
> static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
> {
> return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb
2020-01-09 2:49 ` [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb Richard Henderson
@ 2020-01-20 0:37 ` Alistair Francis
2020-01-20 8:54 ` Philippe Mathieu-Daudé
` (2 subsequent siblings)
3 siblings, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 0:37 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:49 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> We do not need the entire CPUArchState to compute these values.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> accel/tcg/cputlb.c | 15 ++++++++-------
> 1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index e4a8ed9534..49c605b6d8 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -80,14 +80,14 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
> QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
> #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
>
> -static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> +static inline size_t tlb_n_entries(CPUTLBDescFast *fast)
> {
> - return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> + return (fast->mask >> CPU_TLB_ENTRY_BITS) + 1;
> }
>
> -static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
> +static inline size_t sizeof_tlb(CPUTLBDescFast *fast)
> {
> - return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
> + return fast->mask + (1 << CPU_TLB_ENTRY_BITS);
> }
>
> static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> @@ -156,7 +156,7 @@ static void tlb_dyn_init(CPUArchState *env)
> static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> {
> CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - size_t old_size = tlb_n_entries(env, mmu_idx);
> + size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
> size_t rate;
> size_t new_size = old_size;
> int64_t now = get_clock_realtime();
> @@ -236,7 +236,8 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
> + memset(env_tlb(env)->f[mmu_idx].table, -1,
> + sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
> memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> sizeof(env_tlb(env)->d[0].vtable));
> }
> @@ -622,7 +623,7 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length)
> qemu_spin_lock(&env_tlb(env)->c.lock);
> for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) {
> unsigned int i;
> - unsigned int n = tlb_n_entries(env, mmu_idx);
> + unsigned int n = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
>
> for (i = 0; i < n; i++) {
> tlb_reset_dirty_range_locked(&env_tlb(env)->f[mmu_idx].table[i],
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c
2020-01-09 2:49 ` [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c Richard Henderson
2020-01-20 0:36 ` Alistair Francis
@ 2020-01-20 8:53 ` Philippe Mathieu-Daudé
2020-01-20 13:34 ` Alex Bennée
2 siblings, 0 replies; 37+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-20 8:53 UTC (permalink / raw)
To: Richard Henderson, qemu-devel; +Cc: alistair.francis
On 1/9/20 3:49 AM, Richard Henderson wrote:
> There are no users of this function outside cputlb.c,
> and its interface will change in the next patch.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> include/exec/cpu_ldst.h | 5 -----
> accel/tcg/cputlb.c | 5 +++++
> 2 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h
> index a46116167c..53de19753a 100644
> --- a/include/exec/cpu_ldst.h
> +++ b/include/exec/cpu_ldst.h
> @@ -234,11 +234,6 @@ static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx,
> return (addr >> TARGET_PAGE_BITS) & size_mask;
> }
>
> -static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> -{
> - return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> -}
> -
> /* Find the TLB entry corresponding to the mmu_idx + address pair. */
> static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx,
> target_ulong addr)
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 1a81886e58..e4a8ed9534 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -80,6 +80,11 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
> QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
> #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
>
> +static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> +{
> + return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> +}
> +
> static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
> {
> return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb
2020-01-09 2:49 ` [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb Richard Henderson
2020-01-20 0:37 ` Alistair Francis
@ 2020-01-20 8:54 ` Philippe Mathieu-Daudé
2020-01-20 13:36 ` Alex Bennée
2020-01-20 13:39 ` Alex Bennée
3 siblings, 0 replies; 37+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-20 8:54 UTC (permalink / raw)
To: Richard Henderson, qemu-devel; +Cc: alistair.francis
On 1/9/20 3:49 AM, Richard Henderson wrote:
> We do not need the entire CPUArchState to compute these values.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> accel/tcg/cputlb.c | 15 ++++++++-------
> 1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index e4a8ed9534..49c605b6d8 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -80,14 +80,14 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
> QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
> #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
>
> -static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> +static inline size_t tlb_n_entries(CPUTLBDescFast *fast)
> {
> - return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> + return (fast->mask >> CPU_TLB_ENTRY_BITS) + 1;
> }
>
> -static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
> +static inline size_t sizeof_tlb(CPUTLBDescFast *fast)
> {
> - return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
> + return fast->mask + (1 << CPU_TLB_ENTRY_BITS);
> }
>
> static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> @@ -156,7 +156,7 @@ static void tlb_dyn_init(CPUArchState *env)
> static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> {
> CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - size_t old_size = tlb_n_entries(env, mmu_idx);
> + size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
> size_t rate;
> size_t new_size = old_size;
> int64_t now = get_clock_realtime();
> @@ -236,7 +236,8 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
> + memset(env_tlb(env)->f[mmu_idx].table, -1,
> + sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
> memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> sizeof(env_tlb(env)->d[0].vtable));
> }
> @@ -622,7 +623,7 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length)
> qemu_spin_lock(&env_tlb(env)->c.lock);
> for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) {
> unsigned int i;
> - unsigned int n = tlb_n_entries(env, mmu_idx);
> + unsigned int n = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
>
> for (i = 0; i < n; i++) {
> tlb_reset_dirty_range_locked(&env_tlb(env)->f[mmu_idx].table[i],
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked
2020-01-09 2:49 ` [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked Richard Henderson
@ 2020-01-20 8:58 ` Philippe Mathieu-Daudé
2020-01-20 12:05 ` Alistair Francis
2020-01-20 13:40 ` Alex Bennée
2 siblings, 0 replies; 37+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-20 8:58 UTC (permalink / raw)
To: Richard Henderson, qemu-devel; +Cc: alistair.francis
On 1/9/20 3:49 AM, Richard Henderson wrote:
> No functional change, but the smaller expressions make
> the code easier to read.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> accel/tcg/cputlb.c | 35 +++++++++++++++++------------------
> 1 file changed, 17 insertions(+), 18 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 49c605b6d8..c7dc1dc85a 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -115,8 +115,8 @@ static void tlb_dyn_init(CPUArchState *env)
>
> /**
> * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
> - * @env: CPU that owns the TLB
> - * @mmu_idx: MMU index of the TLB
> + * @desc: The CPUTLBDesc portion of the TLB
> + * @fast: The CPUTLBDescFast portion of the same TLB
> *
> * Called with tlb_lock_held.
> *
> @@ -153,10 +153,9 @@ static void tlb_dyn_init(CPUArchState *env)
> * high), since otherwise we are likely to have a significant amount of
> * conflict misses.
> */
> -static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> {
> - CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
> + size_t old_size = tlb_n_entries(fast);
> size_t rate;
> size_t new_size = old_size;
> int64_t now = get_clock_realtime();
> @@ -198,14 +197,15 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> return;
> }
>
> - g_free(env_tlb(env)->f[mmu_idx].table);
> - g_free(env_tlb(env)->d[mmu_idx].iotlb);
> + g_free(fast->table);
> + g_free(desc->iotlb);
>
> tlb_window_reset(desc, now, 0);
> /* desc->n_used_entries is cleared by the caller */
> - env_tlb(env)->f[mmu_idx].mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> - env_tlb(env)->f[mmu_idx].table = g_try_new(CPUTLBEntry, new_size);
> - env_tlb(env)->d[mmu_idx].iotlb = g_try_new(CPUIOTLBEntry, new_size);
> + fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> + fast->table = g_try_new(CPUTLBEntry, new_size);
> + desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
> +
> /*
> * If the allocations fail, try smaller sizes. We just freed some
> * memory, so going back to half of new_size has a good chance of working.
> @@ -213,25 +213,24 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> * allocations to fail though, so we progressively reduce the allocation
> * size, aborting if we cannot even allocate the smallest TLB we support.
> */
> - while (env_tlb(env)->f[mmu_idx].table == NULL ||
> - env_tlb(env)->d[mmu_idx].iotlb == NULL) {
> + while (fast->table == NULL || desc->iotlb == NULL) {
> if (new_size == (1 << CPU_TLB_DYN_MIN_BITS)) {
> error_report("%s: %s", __func__, strerror(errno));
> abort();
> }
> new_size = MAX(new_size >> 1, 1 << CPU_TLB_DYN_MIN_BITS);
> - env_tlb(env)->f[mmu_idx].mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> + fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
>
> - g_free(env_tlb(env)->f[mmu_idx].table);
> - g_free(env_tlb(env)->d[mmu_idx].iotlb);
> - env_tlb(env)->f[mmu_idx].table = g_try_new(CPUTLBEntry, new_size);
> - env_tlb(env)->d[mmu_idx].iotlb = g_try_new(CPUIOTLBEntry, new_size);
> + g_free(fast->table);
> + g_free(desc->iotlb);
> + fast->table = g_try_new(CPUTLBEntry, new_size);
> + desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
> }
> }
>
> static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> {
> - tlb_mmu_resize_locked(env, mmu_idx);
> + tlb_mmu_resize_locked(&env_tlb(env)->d[mmu_idx], &env_tlb(env)->f[mmu_idx]);
> env_tlb(env)->d[mmu_idx].n_used_entries = 0;
> env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> env_tlb(env)->d[mmu_idx].large_page_mask = -1;
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked
2020-01-09 2:49 ` [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked Richard Henderson
@ 2020-01-20 8:58 ` Philippe Mathieu-Daudé
2020-01-20 13:41 ` Alex Bennée
2020-01-20 22:56 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-20 8:58 UTC (permalink / raw)
To: Richard Henderson, qemu-devel; +Cc: alistair.francis
On 1/9/20 3:49 AM, Richard Henderson wrote:
> No functional change, but the smaller expressions make
> the code easier to read.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> accel/tcg/cputlb.c | 19 ++++++++++---------
> 1 file changed, 10 insertions(+), 9 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index c7dc1dc85a..eff427f137 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -230,15 +230,16 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
>
> static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> {
> - tlb_mmu_resize_locked(&env_tlb(env)->d[mmu_idx], &env_tlb(env)->f[mmu_idx]);
> - env_tlb(env)->d[mmu_idx].n_used_entries = 0;
> - env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> - env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> - env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->f[mmu_idx].table, -1,
> - sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
> - memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> - sizeof(env_tlb(env)->d[0].vtable));
> + CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> + CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> +
> + tlb_mmu_resize_locked(desc, fast);
> + desc->n_used_entries = 0;
> + desc->large_page_addr = -1;
> + desc->large_page_mask = -1;
> + desc->vindex = 0;
> + memset(fast->table, -1, sizeof_tlb(fast));
> + memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked
2020-01-09 2:49 ` [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked Richard Henderson
@ 2020-01-20 8:59 ` Philippe Mathieu-Daudé
2020-01-20 13:41 ` Alex Bennée
2020-01-20 22:58 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-20 8:59 UTC (permalink / raw)
To: Richard Henderson, qemu-devel; +Cc: alistair.francis
On 1/9/20 3:49 AM, Richard Henderson wrote:
> We will want to be able to flush a tlb without resizing.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> accel/tcg/cputlb.c | 15 ++++++++++-----
> 1 file changed, 10 insertions(+), 5 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index eff427f137..e60e501334 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -228,12 +228,8 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> }
> }
>
> -static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> {
> - CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> -
> - tlb_mmu_resize_locked(desc, fast);
> desc->n_used_entries = 0;
> desc->large_page_addr = -1;
> desc->large_page_mask = -1;
> @@ -242,6 +238,15 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> +static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +{
> + CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> + CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> +
> + tlb_mmu_resize_locked(desc, fast);
> + tlb_mmu_flush_locked(desc, fast);
> +}
> +
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> {
> env_tlb(env)->d[mmu_idx].n_used_entries++;
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init
2020-01-09 2:49 ` [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init Richard Henderson
@ 2020-01-20 9:01 ` Philippe Mathieu-Daudé
2020-01-20 14:33 ` Alex Bennée
2020-01-20 23:00 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-20 9:01 UTC (permalink / raw)
To: Richard Henderson, qemu-devel; +Cc: alistair.francis
On 1/9/20 3:49 AM, Richard Henderson wrote:
> Merge into the only caller, but at the same time split
> out tlb_mmu_init to initialize a single tlb entry.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> accel/tcg/cputlb.c | 33 ++++++++++++++++-----------------
> 1 file changed, 16 insertions(+), 17 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index e60e501334..c7c34b185b 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -97,22 +97,6 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> desc->window_max_entries = max_entries;
> }
>
> -static void tlb_dyn_init(CPUArchState *env)
> -{
> - int i;
> -
> - for (i = 0; i < NB_MMU_MODES; i++) {
> - CPUTLBDesc *desc = &env_tlb(env)->d[i];
> - size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS;
> -
> - tlb_window_reset(desc, get_clock_realtime(), 0);
> - desc->n_used_entries = 0;
> - env_tlb(env)->f[i].mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
> - env_tlb(env)->f[i].table = g_new(CPUTLBEntry, n_entries);
> - env_tlb(env)->d[i].iotlb = g_new(CPUIOTLBEntry, n_entries);
> - }
> -}
> -
> /**
> * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
> * @desc: The CPUTLBDesc portion of the TLB
> @@ -247,6 +231,17 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> tlb_mmu_flush_locked(desc, fast);
> }
>
> +static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
> +{
> + size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS;
> +
> + tlb_window_reset(desc, now, 0);
> + desc->n_used_entries = 0;
> + fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
> + fast->table = g_new(CPUTLBEntry, n_entries);
> + desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
> +}
> +
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> {
> env_tlb(env)->d[mmu_idx].n_used_entries++;
> @@ -260,13 +255,17 @@ static inline void tlb_n_used_entries_dec(CPUArchState *env, uintptr_t mmu_idx)
> void tlb_init(CPUState *cpu)
> {
> CPUArchState *env = cpu->env_ptr;
> + int64_t now = get_clock_realtime();
> + int i;
>
> qemu_spin_init(&env_tlb(env)->c.lock);
>
> /* Ensure that cpu_reset performs a full flush. */
> env_tlb(env)->c.dirty = ALL_MMUIDX_BITS;
>
> - tlb_dyn_init(env);
> + for (i = 0; i < NB_MMU_MODES; i++) {
> + tlb_mmu_init(&env_tlb(env)->d[i], &env_tlb(env)->f[i], now);
> + }
> }
>
> /* flush_all_helper: run fn across all cpus
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs
2020-01-09 2:49 ` [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs Richard Henderson
@ 2020-01-20 9:02 ` Philippe Mathieu-Daudé
2020-01-20 14:36 ` Alex Bennée
2020-01-20 23:02 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-20 9:02 UTC (permalink / raw)
To: Richard Henderson, qemu-devel; +Cc: alistair.francis
On 1/9/20 3:49 AM, Richard Henderson wrote:
> Do not call get_clock_realtime() in tlb_mmu_resize_locked,
> but hoist outside of any loop over a set of tlbs. This is
> only two (indirect) callers, tlb_flush_by_mmuidx_async_work
> and tlb_flush_page_locked, so not onerous.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> accel/tcg/cputlb.c | 14 ++++++++------
> 1 file changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 761e9d44d7..9f6cb36921 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -137,12 +137,12 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> * high), since otherwise we are likely to have a significant amount of
> * conflict misses.
> */
> -static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> +static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
> + int64_t now)
> {
> size_t old_size = tlb_n_entries(fast);
> size_t rate;
> size_t new_size = old_size;
> - int64_t now = get_clock_realtime();
> int64_t window_len_ms = 100;
> int64_t window_len_ns = window_len_ms * 1000 * 1000;
> bool window_expired = now > desc->window_begin_ns + window_len_ns;
> @@ -222,12 +222,13 @@ static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> -static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx,
> + int64_t now)
> {
> CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
>
> - tlb_mmu_resize_locked(desc, fast);
> + tlb_mmu_resize_locked(desc, fast, now);
> tlb_mmu_flush_locked(desc, fast);
> }
>
> @@ -310,6 +311,7 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
> CPUArchState *env = cpu->env_ptr;
> uint16_t asked = data.host_int;
> uint16_t all_dirty, work, to_clean;
> + int64_t now = get_clock_realtime();
>
> assert_cpu_is_self(cpu);
>
> @@ -324,7 +326,7 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
>
> for (work = to_clean; work != 0; work &= work - 1) {
> int mmu_idx = ctz32(work);
> - tlb_flush_one_mmuidx_locked(env, mmu_idx);
> + tlb_flush_one_mmuidx_locked(env, mmu_idx, now);
> }
>
> qemu_spin_unlock(&env_tlb(env)->c.lock);
> @@ -446,7 +448,7 @@ static void tlb_flush_page_locked(CPUArchState *env, int midx,
> tlb_debug("forcing full flush midx %d ("
> TARGET_FMT_lx "/" TARGET_FMT_lx ")\n",
> midx, lp_addr, lp_mask);
> - tlb_flush_one_mmuidx_locked(env, midx);
> + tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
> } else {
> if (tlb_flush_entry_locked(tlb_entry(env, midx, page), page)) {
> tlb_n_used_entries_dec(env, midx);
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked
2020-01-09 2:49 ` [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked Richard Henderson
2020-01-20 8:58 ` Philippe Mathieu-Daudé
@ 2020-01-20 12:05 ` Alistair Francis
2020-01-20 13:40 ` Alex Bennée
2 siblings, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 12:05 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:52 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> No functional change, but the smaller expressions make
> the code easier to read.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> accel/tcg/cputlb.c | 35 +++++++++++++++++------------------
> 1 file changed, 17 insertions(+), 18 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 49c605b6d8..c7dc1dc85a 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -115,8 +115,8 @@ static void tlb_dyn_init(CPUArchState *env)
>
> /**
> * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
> - * @env: CPU that owns the TLB
> - * @mmu_idx: MMU index of the TLB
> + * @desc: The CPUTLBDesc portion of the TLB
> + * @fast: The CPUTLBDescFast portion of the same TLB
> *
> * Called with tlb_lock_held.
> *
> @@ -153,10 +153,9 @@ static void tlb_dyn_init(CPUArchState *env)
> * high), since otherwise we are likely to have a significant amount of
> * conflict misses.
> */
> -static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> {
> - CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
> + size_t old_size = tlb_n_entries(fast);
> size_t rate;
> size_t new_size = old_size;
> int64_t now = get_clock_realtime();
> @@ -198,14 +197,15 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> return;
> }
>
> - g_free(env_tlb(env)->f[mmu_idx].table);
> - g_free(env_tlb(env)->d[mmu_idx].iotlb);
> + g_free(fast->table);
> + g_free(desc->iotlb);
>
> tlb_window_reset(desc, now, 0);
> /* desc->n_used_entries is cleared by the caller */
> - env_tlb(env)->f[mmu_idx].mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> - env_tlb(env)->f[mmu_idx].table = g_try_new(CPUTLBEntry, new_size);
> - env_tlb(env)->d[mmu_idx].iotlb = g_try_new(CPUIOTLBEntry, new_size);
> + fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> + fast->table = g_try_new(CPUTLBEntry, new_size);
> + desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
> +
> /*
> * If the allocations fail, try smaller sizes. We just freed some
> * memory, so going back to half of new_size has a good chance of working.
> @@ -213,25 +213,24 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> * allocations to fail though, so we progressively reduce the allocation
> * size, aborting if we cannot even allocate the smallest TLB we support.
> */
> - while (env_tlb(env)->f[mmu_idx].table == NULL ||
> - env_tlb(env)->d[mmu_idx].iotlb == NULL) {
> + while (fast->table == NULL || desc->iotlb == NULL) {
> if (new_size == (1 << CPU_TLB_DYN_MIN_BITS)) {
> error_report("%s: %s", __func__, strerror(errno));
> abort();
> }
> new_size = MAX(new_size >> 1, 1 << CPU_TLB_DYN_MIN_BITS);
> - env_tlb(env)->f[mmu_idx].mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> + fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
>
> - g_free(env_tlb(env)->f[mmu_idx].table);
> - g_free(env_tlb(env)->d[mmu_idx].iotlb);
> - env_tlb(env)->f[mmu_idx].table = g_try_new(CPUTLBEntry, new_size);
> - env_tlb(env)->d[mmu_idx].iotlb = g_try_new(CPUIOTLBEntry, new_size);
> + g_free(fast->table);
> + g_free(desc->iotlb);
> + fast->table = g_try_new(CPUTLBEntry, new_size);
> + desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
> }
> }
>
> static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> {
> - tlb_mmu_resize_locked(env, mmu_idx);
> + tlb_mmu_resize_locked(&env_tlb(env)->d[mmu_idx], &env_tlb(env)->f[mmu_idx]);
> env_tlb(env)->d[mmu_idx].n_used_entries = 0;
> env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked
2020-01-09 2:48 ` [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked Richard Henderson
2020-01-20 0:34 ` Alistair Francis
@ 2020-01-20 13:34 ` Alex Bennée
1 sibling, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 13:34 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> There is only one caller for tlb_table_flush_by_mmuidx. Place
> the result at the earlier line number, due to an expected user
> in the near future.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> accel/tcg/cputlb.c | 19 +++++++------------
> 1 file changed, 7 insertions(+), 12 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index a991ea2964..1a81886e58 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -224,11 +224,16 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> }
> }
>
> -static inline void tlb_table_flush_by_mmuidx(CPUArchState *env, int mmu_idx)
> +static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> {
> tlb_mmu_resize_locked(env, mmu_idx);
> - memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
> env_tlb(env)->d[mmu_idx].n_used_entries = 0;
> + env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> + env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> + env_tlb(env)->d[mmu_idx].vindex = 0;
> + memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
> + memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> + sizeof(env_tlb(env)->d[0].vtable));
> }
>
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> @@ -289,16 +294,6 @@ void tlb_flush_counts(size_t *pfull, size_t *ppart, size_t *pelide)
> *pelide = elide;
> }
>
> -static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> -{
> - tlb_table_flush_by_mmuidx(env, mmu_idx);
> - env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> - env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> - env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> - sizeof(env_tlb(env)->d[0].vtable));
> -}
> -
> static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
> {
> CPUArchState *env = cpu->env_ptr;
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c
2020-01-09 2:49 ` [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c Richard Henderson
2020-01-20 0:36 ` Alistair Francis
2020-01-20 8:53 ` Philippe Mathieu-Daudé
@ 2020-01-20 13:34 ` Alex Bennée
2 siblings, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 13:34 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> There are no users of this function outside cputlb.c,
> and its interface will change in the next patch.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> include/exec/cpu_ldst.h | 5 -----
> accel/tcg/cputlb.c | 5 +++++
> 2 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h
> index a46116167c..53de19753a 100644
> --- a/include/exec/cpu_ldst.h
> +++ b/include/exec/cpu_ldst.h
> @@ -234,11 +234,6 @@ static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx,
> return (addr >> TARGET_PAGE_BITS) & size_mask;
> }
>
> -static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> -{
> - return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> -}
> -
> /* Find the TLB entry corresponding to the mmu_idx + address pair. */
> static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx,
> target_ulong addr)
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 1a81886e58..e4a8ed9534 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -80,6 +80,11 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
> QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
> #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
>
> +static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> +{
> + return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> +}
> +
> static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
> {
> return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb
2020-01-09 2:49 ` [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb Richard Henderson
2020-01-20 0:37 ` Alistair Francis
2020-01-20 8:54 ` Philippe Mathieu-Daudé
@ 2020-01-20 13:36 ` Alex Bennée
2020-01-20 13:39 ` Alex Bennée
3 siblings, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 13:36 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> We do not need the entire CPUArchState to compute these values.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> accel/tcg/cputlb.c | 15 ++++++++-------
> 1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index e4a8ed9534..49c605b6d8 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -80,14 +80,14 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
> QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
> #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
>
> -static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> +static inline size_t tlb_n_entries(CPUTLBDescFast *fast)
> {
> - return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> + return (fast->mask >> CPU_TLB_ENTRY_BITS) + 1;
> }
>
> -static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
> +static inline size_t sizeof_tlb(CPUTLBDescFast *fast)
> {
> - return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
> + return fast->mask + (1 << CPU_TLB_ENTRY_BITS);
> }
>
> static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> @@ -156,7 +156,7 @@ static void tlb_dyn_init(CPUArchState *env)
> static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> {
> CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - size_t old_size = tlb_n_entries(env, mmu_idx);
> + size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
> size_t rate;
> size_t new_size = old_size;
> int64_t now = get_clock_realtime();
> @@ -236,7 +236,8 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
> + memset(env_tlb(env)->f[mmu_idx].table, -1,
> + sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
> memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> sizeof(env_tlb(env)->d[0].vtable));
> }
> @@ -622,7 +623,7 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length)
> qemu_spin_lock(&env_tlb(env)->c.lock);
> for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) {
> unsigned int i;
> - unsigned int n = tlb_n_entries(env, mmu_idx);
> + unsigned int n = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
>
> for (i = 0; i < n; i++) {
> tlb_reset_dirty_range_locked(&env_tlb(env)->f[mmu_idx].table[i],
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb
2020-01-09 2:49 ` [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb Richard Henderson
` (2 preceding siblings ...)
2020-01-20 13:36 ` Alex Bennée
@ 2020-01-20 13:39 ` Alex Bennée
3 siblings, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 13:39 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> We do not need the entire CPUArchState to compute these values.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> accel/tcg/cputlb.c | 15 ++++++++-------
> 1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index e4a8ed9534..49c605b6d8 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -80,14 +80,14 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data));
> QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16);
> #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1)
>
> -static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx)
> +static inline size_t tlb_n_entries(CPUTLBDescFast *fast)
> {
> - return (env_tlb(env)->f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS) + 1;
> + return (fast->mask >> CPU_TLB_ENTRY_BITS) + 1;
> }
>
> -static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx)
> +static inline size_t sizeof_tlb(CPUTLBDescFast *fast)
> {
> - return env_tlb(env)->f[mmu_idx].mask + (1 << CPU_TLB_ENTRY_BITS);
> + return fast->mask + (1 << CPU_TLB_ENTRY_BITS);
> }
>
> static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> @@ -156,7 +156,7 @@ static void tlb_dyn_init(CPUArchState *env)
> static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> {
> CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - size_t old_size = tlb_n_entries(env, mmu_idx);
> + size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
> size_t rate;
> size_t new_size = old_size;
> int64_t now = get_clock_realtime();
> @@ -236,7 +236,8 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->f[mmu_idx].table, -1, sizeof_tlb(env, mmu_idx));
> + memset(env_tlb(env)->f[mmu_idx].table, -1,
> + sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
> memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> sizeof(env_tlb(env)->d[0].vtable));
> }
> @@ -622,7 +623,7 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length)
> qemu_spin_lock(&env_tlb(env)->c.lock);
> for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) {
> unsigned int i;
> - unsigned int n = tlb_n_entries(env, mmu_idx);
> + unsigned int n = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
>
> for (i = 0; i < n; i++) {
> tlb_reset_dirty_range_locked(&env_tlb(env)->f[mmu_idx].table[i],
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked
2020-01-09 2:49 ` [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked Richard Henderson
2020-01-20 8:58 ` Philippe Mathieu-Daudé
2020-01-20 12:05 ` Alistair Francis
@ 2020-01-20 13:40 ` Alex Bennée
2 siblings, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 13:40 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> No functional change, but the smaller expressions make
> the code easier to read.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> accel/tcg/cputlb.c | 35 +++++++++++++++++------------------
> 1 file changed, 17 insertions(+), 18 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 49c605b6d8..c7dc1dc85a 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -115,8 +115,8 @@ static void tlb_dyn_init(CPUArchState *env)
>
> /**
> * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
> - * @env: CPU that owns the TLB
> - * @mmu_idx: MMU index of the TLB
> + * @desc: The CPUTLBDesc portion of the TLB
> + * @fast: The CPUTLBDescFast portion of the same TLB
> *
> * Called with tlb_lock_held.
> *
> @@ -153,10 +153,9 @@ static void tlb_dyn_init(CPUArchState *env)
> * high), since otherwise we are likely to have a significant amount of
> * conflict misses.
> */
> -static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> {
> - CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - size_t old_size = tlb_n_entries(&env_tlb(env)->f[mmu_idx]);
> + size_t old_size = tlb_n_entries(fast);
> size_t rate;
> size_t new_size = old_size;
> int64_t now = get_clock_realtime();
> @@ -198,14 +197,15 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> return;
> }
>
> - g_free(env_tlb(env)->f[mmu_idx].table);
> - g_free(env_tlb(env)->d[mmu_idx].iotlb);
> + g_free(fast->table);
> + g_free(desc->iotlb);
>
> tlb_window_reset(desc, now, 0);
> /* desc->n_used_entries is cleared by the caller */
> - env_tlb(env)->f[mmu_idx].mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> - env_tlb(env)->f[mmu_idx].table = g_try_new(CPUTLBEntry, new_size);
> - env_tlb(env)->d[mmu_idx].iotlb = g_try_new(CPUIOTLBEntry, new_size);
> + fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> + fast->table = g_try_new(CPUTLBEntry, new_size);
> + desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
> +
> /*
> * If the allocations fail, try smaller sizes. We just freed some
> * memory, so going back to half of new_size has a good chance of working.
> @@ -213,25 +213,24 @@ static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx)
> * allocations to fail though, so we progressively reduce the allocation
> * size, aborting if we cannot even allocate the smallest TLB we support.
> */
> - while (env_tlb(env)->f[mmu_idx].table == NULL ||
> - env_tlb(env)->d[mmu_idx].iotlb == NULL) {
> + while (fast->table == NULL || desc->iotlb == NULL) {
> if (new_size == (1 << CPU_TLB_DYN_MIN_BITS)) {
> error_report("%s: %s", __func__, strerror(errno));
> abort();
> }
> new_size = MAX(new_size >> 1, 1 << CPU_TLB_DYN_MIN_BITS);
> - env_tlb(env)->f[mmu_idx].mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
> + fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
>
> - g_free(env_tlb(env)->f[mmu_idx].table);
> - g_free(env_tlb(env)->d[mmu_idx].iotlb);
> - env_tlb(env)->f[mmu_idx].table = g_try_new(CPUTLBEntry, new_size);
> - env_tlb(env)->d[mmu_idx].iotlb = g_try_new(CPUIOTLBEntry, new_size);
> + g_free(fast->table);
> + g_free(desc->iotlb);
> + fast->table = g_try_new(CPUTLBEntry, new_size);
> + desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
> }
> }
>
> static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> {
> - tlb_mmu_resize_locked(env, mmu_idx);
> + tlb_mmu_resize_locked(&env_tlb(env)->d[mmu_idx], &env_tlb(env)->f[mmu_idx]);
> env_tlb(env)->d[mmu_idx].n_used_entries = 0;
> env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> env_tlb(env)->d[mmu_idx].large_page_mask = -1;
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked
2020-01-09 2:49 ` [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked Richard Henderson
2020-01-20 8:58 ` Philippe Mathieu-Daudé
@ 2020-01-20 13:41 ` Alex Bennée
2020-01-20 22:56 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 13:41 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> No functional change, but the smaller expressions make
> the code easier to read.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> accel/tcg/cputlb.c | 19 ++++++++++---------
> 1 file changed, 10 insertions(+), 9 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index c7dc1dc85a..eff427f137 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -230,15 +230,16 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
>
> static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> {
> - tlb_mmu_resize_locked(&env_tlb(env)->d[mmu_idx], &env_tlb(env)->f[mmu_idx]);
> - env_tlb(env)->d[mmu_idx].n_used_entries = 0;
> - env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> - env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> - env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->f[mmu_idx].table, -1,
> - sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
> - memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> - sizeof(env_tlb(env)->d[0].vtable));
> + CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> + CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> +
> + tlb_mmu_resize_locked(desc, fast);
> + desc->n_used_entries = 0;
> + desc->large_page_addr = -1;
> + desc->large_page_mask = -1;
> + desc->vindex = 0;
> + memset(fast->table, -1, sizeof_tlb(fast));
> + memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked
2020-01-09 2:49 ` [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked Richard Henderson
2020-01-20 8:59 ` Philippe Mathieu-Daudé
@ 2020-01-20 13:41 ` Alex Bennée
2020-01-20 22:58 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 13:41 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> We will want to be able to flush a tlb without resizing.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> accel/tcg/cputlb.c | 15 ++++++++++-----
> 1 file changed, 10 insertions(+), 5 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index eff427f137..e60e501334 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -228,12 +228,8 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> }
> }
>
> -static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> {
> - CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> -
> - tlb_mmu_resize_locked(desc, fast);
> desc->n_used_entries = 0;
> desc->large_page_addr = -1;
> desc->large_page_mask = -1;
> @@ -242,6 +238,15 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> +static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +{
> + CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> + CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> +
> + tlb_mmu_resize_locked(desc, fast);
> + tlb_mmu_flush_locked(desc, fast);
> +}
> +
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> {
> env_tlb(env)->d[mmu_idx].n_used_entries++;
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init
2020-01-09 2:49 ` [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init Richard Henderson
2020-01-20 9:01 ` Philippe Mathieu-Daudé
@ 2020-01-20 14:33 ` Alex Bennée
2020-01-20 23:00 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 14:33 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> Merge into the only caller, but at the same time split
> out tlb_mmu_init to initialize a single tlb entry.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> accel/tcg/cputlb.c | 33 ++++++++++++++++-----------------
> 1 file changed, 16 insertions(+), 17 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index e60e501334..c7c34b185b 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -97,22 +97,6 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> desc->window_max_entries = max_entries;
> }
>
> -static void tlb_dyn_init(CPUArchState *env)
> -{
> - int i;
> -
> - for (i = 0; i < NB_MMU_MODES; i++) {
> - CPUTLBDesc *desc = &env_tlb(env)->d[i];
> - size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS;
> -
> - tlb_window_reset(desc, get_clock_realtime(), 0);
> - desc->n_used_entries = 0;
> - env_tlb(env)->f[i].mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
> - env_tlb(env)->f[i].table = g_new(CPUTLBEntry, n_entries);
> - env_tlb(env)->d[i].iotlb = g_new(CPUIOTLBEntry, n_entries);
> - }
> -}
> -
> /**
> * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
> * @desc: The CPUTLBDesc portion of the TLB
> @@ -247,6 +231,17 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> tlb_mmu_flush_locked(desc, fast);
> }
>
> +static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
> +{
> + size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS;
> +
> + tlb_window_reset(desc, now, 0);
> + desc->n_used_entries = 0;
> + fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
> + fast->table = g_new(CPUTLBEntry, n_entries);
> + desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
> +}
> +
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> {
> env_tlb(env)->d[mmu_idx].n_used_entries++;
> @@ -260,13 +255,17 @@ static inline void tlb_n_used_entries_dec(CPUArchState *env, uintptr_t mmu_idx)
> void tlb_init(CPUState *cpu)
> {
> CPUArchState *env = cpu->env_ptr;
> + int64_t now = get_clock_realtime();
> + int i;
>
> qemu_spin_init(&env_tlb(env)->c.lock);
>
> /* Ensure that cpu_reset performs a full flush. */
> env_tlb(env)->c.dirty = ALL_MMUIDX_BITS;
>
> - tlb_dyn_init(env);
> + for (i = 0; i < NB_MMU_MODES; i++) {
> + tlb_mmu_init(&env_tlb(env)->d[i], &env_tlb(env)->f[i], now);
> + }
> }
>
> /* flush_all_helper: run fn across all cpus
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 8/9] cputlb: Initialize tlbs as flushed
2020-01-09 2:49 ` [PATCH 8/9] cputlb: Initialize tlbs as flushed Richard Henderson
@ 2020-01-20 14:33 ` Alex Bennée
2020-01-20 23:01 ` Alistair Francis
1 sibling, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 14:33 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> There's little point in leaving these data structures half initialized,
> and relying on a flush to be done during reset.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> accel/tcg/cputlb.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index c7c34b185b..761e9d44d7 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -240,6 +240,7 @@ static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
> fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
> fast->table = g_new(CPUTLBEntry, n_entries);
> desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
> + tlb_mmu_flush_locked(desc, fast);
> }
>
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> @@ -260,8 +261,8 @@ void tlb_init(CPUState *cpu)
>
> qemu_spin_init(&env_tlb(env)->c.lock);
>
> - /* Ensure that cpu_reset performs a full flush. */
> - env_tlb(env)->c.dirty = ALL_MMUIDX_BITS;
> + /* All tlbs are initialized flushed. */
> + env_tlb(env)->c.dirty = 0;
>
> for (i = 0; i < NB_MMU_MODES; i++) {
> tlb_mmu_init(&env_tlb(env)->d[i], &env_tlb(env)->f[i], now);
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs
2020-01-09 2:49 ` [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs Richard Henderson
2020-01-20 9:02 ` Philippe Mathieu-Daudé
@ 2020-01-20 14:36 ` Alex Bennée
2020-01-20 23:02 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Alex Bennée @ 2020-01-20 14:36 UTC (permalink / raw)
To: Richard Henderson; +Cc: alistair.francis, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> Do not call get_clock_realtime() in tlb_mmu_resize_locked,
> but hoist outside of any loop over a set of tlbs. This is
> only two (indirect) callers, tlb_flush_by_mmuidx_async_work
> and tlb_flush_page_locked, so not onerous.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> accel/tcg/cputlb.c | 14 ++++++++------
> 1 file changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 761e9d44d7..9f6cb36921 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -137,12 +137,12 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> * high), since otherwise we are likely to have a significant amount of
> * conflict misses.
> */
> -static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> +static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
> + int64_t now)
> {
> size_t old_size = tlb_n_entries(fast);
> size_t rate;
> size_t new_size = old_size;
> - int64_t now = get_clock_realtime();
> int64_t window_len_ms = 100;
> int64_t window_len_ns = window_len_ms * 1000 * 1000;
> bool window_expired = now > desc->window_begin_ns + window_len_ns;
> @@ -222,12 +222,13 @@ static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> -static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx,
> + int64_t now)
> {
> CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
>
> - tlb_mmu_resize_locked(desc, fast);
> + tlb_mmu_resize_locked(desc, fast, now);
> tlb_mmu_flush_locked(desc, fast);
> }
>
> @@ -310,6 +311,7 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
> CPUArchState *env = cpu->env_ptr;
> uint16_t asked = data.host_int;
> uint16_t all_dirty, work, to_clean;
> + int64_t now = get_clock_realtime();
>
> assert_cpu_is_self(cpu);
>
> @@ -324,7 +326,7 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
>
> for (work = to_clean; work != 0; work &= work - 1) {
> int mmu_idx = ctz32(work);
> - tlb_flush_one_mmuidx_locked(env, mmu_idx);
> + tlb_flush_one_mmuidx_locked(env, mmu_idx, now);
> }
>
> qemu_spin_unlock(&env_tlb(env)->c.lock);
> @@ -446,7 +448,7 @@ static void tlb_flush_page_locked(CPUArchState *env, int midx,
> tlb_debug("forcing full flush midx %d ("
> TARGET_FMT_lx "/" TARGET_FMT_lx ")\n",
> midx, lp_addr, lp_mask);
> - tlb_flush_one_mmuidx_locked(env, midx);
> + tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
> } else {
> if (tlb_flush_entry_locked(tlb_entry(env, midx, page), page)) {
> tlb_n_used_entries_dec(env, midx);
--
Alex Bennée
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked
2020-01-09 2:49 ` [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked Richard Henderson
2020-01-20 8:58 ` Philippe Mathieu-Daudé
2020-01-20 13:41 ` Alex Bennée
@ 2020-01-20 22:56 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 22:56 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:55 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> No functional change, but the smaller expressions make
> the code easier to read.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> accel/tcg/cputlb.c | 19 ++++++++++---------
> 1 file changed, 10 insertions(+), 9 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index c7dc1dc85a..eff427f137 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -230,15 +230,16 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
>
> static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> {
> - tlb_mmu_resize_locked(&env_tlb(env)->d[mmu_idx], &env_tlb(env)->f[mmu_idx]);
> - env_tlb(env)->d[mmu_idx].n_used_entries = 0;
> - env_tlb(env)->d[mmu_idx].large_page_addr = -1;
> - env_tlb(env)->d[mmu_idx].large_page_mask = -1;
> - env_tlb(env)->d[mmu_idx].vindex = 0;
> - memset(env_tlb(env)->f[mmu_idx].table, -1,
> - sizeof_tlb(&env_tlb(env)->f[mmu_idx]));
> - memset(env_tlb(env)->d[mmu_idx].vtable, -1,
> - sizeof(env_tlb(env)->d[0].vtable));
> + CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> + CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> +
> + tlb_mmu_resize_locked(desc, fast);
> + desc->n_used_entries = 0;
> + desc->large_page_addr = -1;
> + desc->large_page_mask = -1;
> + desc->vindex = 0;
> + memset(fast->table, -1, sizeof_tlb(fast));
> + memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked
2020-01-09 2:49 ` [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked Richard Henderson
2020-01-20 8:59 ` Philippe Mathieu-Daudé
2020-01-20 13:41 ` Alex Bennée
@ 2020-01-20 22:58 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 22:58 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:52 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> We will want to be able to flush a tlb without resizing.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> accel/tcg/cputlb.c | 15 ++++++++++-----
> 1 file changed, 10 insertions(+), 5 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index eff427f137..e60e501334 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -228,12 +228,8 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> }
> }
>
> -static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> {
> - CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> - CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> -
> - tlb_mmu_resize_locked(desc, fast);
> desc->n_used_entries = 0;
> desc->large_page_addr = -1;
> desc->large_page_mask = -1;
> @@ -242,6 +238,15 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> +static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +{
> + CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> + CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
> +
> + tlb_mmu_resize_locked(desc, fast);
> + tlb_mmu_flush_locked(desc, fast);
> +}
> +
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> {
> env_tlb(env)->d[mmu_idx].n_used_entries++;
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init
2020-01-09 2:49 ` [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init Richard Henderson
2020-01-20 9:01 ` Philippe Mathieu-Daudé
2020-01-20 14:33 ` Alex Bennée
@ 2020-01-20 23:00 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 23:00 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:52 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Merge into the only caller, but at the same time split
> out tlb_mmu_init to initialize a single tlb entry.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> accel/tcg/cputlb.c | 33 ++++++++++++++++-----------------
> 1 file changed, 16 insertions(+), 17 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index e60e501334..c7c34b185b 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -97,22 +97,6 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> desc->window_max_entries = max_entries;
> }
>
> -static void tlb_dyn_init(CPUArchState *env)
> -{
> - int i;
> -
> - for (i = 0; i < NB_MMU_MODES; i++) {
> - CPUTLBDesc *desc = &env_tlb(env)->d[i];
> - size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS;
> -
> - tlb_window_reset(desc, get_clock_realtime(), 0);
> - desc->n_used_entries = 0;
> - env_tlb(env)->f[i].mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
> - env_tlb(env)->f[i].table = g_new(CPUTLBEntry, n_entries);
> - env_tlb(env)->d[i].iotlb = g_new(CPUIOTLBEntry, n_entries);
> - }
> -}
> -
> /**
> * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
> * @desc: The CPUTLBDesc portion of the TLB
> @@ -247,6 +231,17 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> tlb_mmu_flush_locked(desc, fast);
> }
>
> +static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
> +{
> + size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS;
> +
> + tlb_window_reset(desc, now, 0);
> + desc->n_used_entries = 0;
> + fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
> + fast->table = g_new(CPUTLBEntry, n_entries);
> + desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
> +}
> +
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> {
> env_tlb(env)->d[mmu_idx].n_used_entries++;
> @@ -260,13 +255,17 @@ static inline void tlb_n_used_entries_dec(CPUArchState *env, uintptr_t mmu_idx)
> void tlb_init(CPUState *cpu)
> {
> CPUArchState *env = cpu->env_ptr;
> + int64_t now = get_clock_realtime();
> + int i;
>
> qemu_spin_init(&env_tlb(env)->c.lock);
>
> /* Ensure that cpu_reset performs a full flush. */
> env_tlb(env)->c.dirty = ALL_MMUIDX_BITS;
>
> - tlb_dyn_init(env);
> + for (i = 0; i < NB_MMU_MODES; i++) {
> + tlb_mmu_init(&env_tlb(env)->d[i], &env_tlb(env)->f[i], now);
> + }
> }
>
> /* flush_all_helper: run fn across all cpus
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 8/9] cputlb: Initialize tlbs as flushed
2020-01-09 2:49 ` [PATCH 8/9] cputlb: Initialize tlbs as flushed Richard Henderson
2020-01-20 14:33 ` Alex Bennée
@ 2020-01-20 23:01 ` Alistair Francis
1 sibling, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 23:01 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:57 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> There's little point in leaving these data structures half initialized,
> and relying on a flush to be done during reset.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> accel/tcg/cputlb.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index c7c34b185b..761e9d44d7 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -240,6 +240,7 @@ static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
> fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
> fast->table = g_new(CPUTLBEntry, n_entries);
> desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
> + tlb_mmu_flush_locked(desc, fast);
> }
>
> static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx)
> @@ -260,8 +261,8 @@ void tlb_init(CPUState *cpu)
>
> qemu_spin_init(&env_tlb(env)->c.lock);
>
> - /* Ensure that cpu_reset performs a full flush. */
> - env_tlb(env)->c.dirty = ALL_MMUIDX_BITS;
> + /* All tlbs are initialized flushed. */
> + env_tlb(env)->c.dirty = 0;
>
> for (i = 0; i < NB_MMU_MODES; i++) {
> tlb_mmu_init(&env_tlb(env)->d[i], &env_tlb(env)->f[i], now);
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs
2020-01-09 2:49 ` [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs Richard Henderson
2020-01-20 9:02 ` Philippe Mathieu-Daudé
2020-01-20 14:36 ` Alex Bennée
@ 2020-01-20 23:02 ` Alistair Francis
2 siblings, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 23:02 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:55 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Do not call get_clock_realtime() in tlb_mmu_resize_locked,
> but hoist outside of any loop over a set of tlbs. This is
> only two (indirect) callers, tlb_flush_by_mmuidx_async_work
> and tlb_flush_page_locked, so not onerous.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> accel/tcg/cputlb.c | 14 ++++++++------
> 1 file changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 761e9d44d7..9f6cb36921 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -137,12 +137,12 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
> * high), since otherwise we are likely to have a significant amount of
> * conflict misses.
> */
> -static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> +static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
> + int64_t now)
> {
> size_t old_size = tlb_n_entries(fast);
> size_t rate;
> size_t new_size = old_size;
> - int64_t now = get_clock_realtime();
> int64_t window_len_ms = 100;
> int64_t window_len_ns = window_len_ms * 1000 * 1000;
> bool window_expired = now > desc->window_begin_ns + window_len_ns;
> @@ -222,12 +222,13 @@ static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast)
> memset(desc->vtable, -1, sizeof(desc->vtable));
> }
>
> -static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx)
> +static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx,
> + int64_t now)
> {
> CPUTLBDesc *desc = &env_tlb(env)->d[mmu_idx];
> CPUTLBDescFast *fast = &env_tlb(env)->f[mmu_idx];
>
> - tlb_mmu_resize_locked(desc, fast);
> + tlb_mmu_resize_locked(desc, fast, now);
> tlb_mmu_flush_locked(desc, fast);
> }
>
> @@ -310,6 +311,7 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
> CPUArchState *env = cpu->env_ptr;
> uint16_t asked = data.host_int;
> uint16_t all_dirty, work, to_clean;
> + int64_t now = get_clock_realtime();
>
> assert_cpu_is_self(cpu);
>
> @@ -324,7 +326,7 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
>
> for (work = to_clean; work != 0; work &= work - 1) {
> int mmu_idx = ctz32(work);
> - tlb_flush_one_mmuidx_locked(env, mmu_idx);
> + tlb_flush_one_mmuidx_locked(env, mmu_idx, now);
> }
>
> qemu_spin_unlock(&env_tlb(env)->c.lock);
> @@ -446,7 +448,7 @@ static void tlb_flush_page_locked(CPUArchState *env, int midx,
> tlb_debug("forcing full flush midx %d ("
> TARGET_FMT_lx "/" TARGET_FMT_lx ")\n",
> midx, lp_addr, lp_mask);
> - tlb_flush_one_mmuidx_locked(env, midx);
> + tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
> } else {
> if (tlb_flush_entry_locked(tlb_entry(env, midx, page), page)) {
> tlb_n_used_entries_dec(env, midx);
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 0/9] cputlb: Various cleanups
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
` (8 preceding siblings ...)
2020-01-09 2:49 ` [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs Richard Henderson
@ 2020-01-20 23:03 ` Alistair Francis
9 siblings, 0 replies; 37+ messages in thread
From: Alistair Francis @ 2020-01-20 23:03 UTC (permalink / raw)
To: Richard Henderson; +Cc: Alistair Francis, qemu-devel@nongnu.org Developers
On Thu, Jan 9, 2020 at 12:49 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> I had a conversation with Alistair Francis at KVM forum about
> being able to represent ASIDs "properly". This lead to the idea
> that target-specific code might be able to cache TLBs outside of
> the "main" NB_MMU_MODES -- possibly thousands of them.
>
> This goes nowhere near that far. But it does begin edging toward
> the possibility of having a
>
> struct CPUTLBSaved {
> CPUTLBDesc d;
> CPUTLBDescFast f;
> };
>
> by moving some of the most basic routines to use CPUTLBDesc and
> CPUTLBDescFast directly instead of always using an mmu_idx.
>
> I'm not sure how much time I'll have to go further along these
> lines, but what I have so far still looks like a cleanup.
Thanks for helping with this!
Unfortunately I haven't had a chance to dig into this myself.
Alistair
>
>
> r~
>
>
> Richard Henderson (9):
> cputlb: Merge tlb_table_flush_by_mmuidx into
> tlb_flush_one_mmuidx_locked
> cputlb: Make tlb_n_entries private to cputlb.c
> cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb
> cputlb: Hoist tlb portions in tlb_mmu_resize_locked
> cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked
> cputlb: Split out tlb_mmu_flush_locked
> cputlb: Partially merge tlb_dyn_init into tlb_init
> cputlb: Initialize tlbs as flushed
> cputlb: Hoist timestamp outside of loops over tlbs
>
> include/exec/cpu_ldst.h | 5 --
> accel/tcg/cputlb.c | 120 +++++++++++++++++++++-------------------
> 2 files changed, 64 insertions(+), 61 deletions(-)
>
> --
> 2.20.1
>
>
^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2020-01-20 23:12 UTC | newest]
Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-09 2:48 [PATCH 0/9] cputlb: Various cleanups Richard Henderson
2020-01-09 2:48 ` [PATCH 1/9] cputlb: Merge tlb_table_flush_by_mmuidx into tlb_flush_one_mmuidx_locked Richard Henderson
2020-01-20 0:34 ` Alistair Francis
2020-01-20 13:34 ` Alex Bennée
2020-01-09 2:49 ` [PATCH 2/9] cputlb: Make tlb_n_entries private to cputlb.c Richard Henderson
2020-01-20 0:36 ` Alistair Francis
2020-01-20 8:53 ` Philippe Mathieu-Daudé
2020-01-20 13:34 ` Alex Bennée
2020-01-09 2:49 ` [PATCH 3/9] cputlb: Pass CPUTLBDescFast to tlb_n_entries and sizeof_tlb Richard Henderson
2020-01-20 0:37 ` Alistair Francis
2020-01-20 8:54 ` Philippe Mathieu-Daudé
2020-01-20 13:36 ` Alex Bennée
2020-01-20 13:39 ` Alex Bennée
2020-01-09 2:49 ` [PATCH 4/9] cputlb: Hoist tlb portions in tlb_mmu_resize_locked Richard Henderson
2020-01-20 8:58 ` Philippe Mathieu-Daudé
2020-01-20 12:05 ` Alistair Francis
2020-01-20 13:40 ` Alex Bennée
2020-01-09 2:49 ` [PATCH 5/9] cputlb: Hoist tlb portions in tlb_flush_one_mmuidx_locked Richard Henderson
2020-01-20 8:58 ` Philippe Mathieu-Daudé
2020-01-20 13:41 ` Alex Bennée
2020-01-20 22:56 ` Alistair Francis
2020-01-09 2:49 ` [PATCH 6/9] cputlb: Split out tlb_mmu_flush_locked Richard Henderson
2020-01-20 8:59 ` Philippe Mathieu-Daudé
2020-01-20 13:41 ` Alex Bennée
2020-01-20 22:58 ` Alistair Francis
2020-01-09 2:49 ` [PATCH 7/9] cputlb: Partially merge tlb_dyn_init into tlb_init Richard Henderson
2020-01-20 9:01 ` Philippe Mathieu-Daudé
2020-01-20 14:33 ` Alex Bennée
2020-01-20 23:00 ` Alistair Francis
2020-01-09 2:49 ` [PATCH 8/9] cputlb: Initialize tlbs as flushed Richard Henderson
2020-01-20 14:33 ` Alex Bennée
2020-01-20 23:01 ` Alistair Francis
2020-01-09 2:49 ` [PATCH 9/9] cputlb: Hoist timestamp outside of loops over tlbs Richard Henderson
2020-01-20 9:02 ` Philippe Mathieu-Daudé
2020-01-20 14:36 ` Alex Bennée
2020-01-20 23:02 ` Alistair Francis
2020-01-20 23:03 ` [PATCH 0/9] cputlb: Various cleanups Alistair Francis
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).