* [PATCH v3 00/16] target/arm: partial vector cleanup
@ 2020-05-08 15:21 Richard Henderson
2020-05-08 15:21 ` [PATCH v3 01/16] target/arm: Create gen_gvec_[us]sra Richard Henderson
` (16 more replies)
0 siblings, 17 replies; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
This is not complete, but shows the direction I'd like to go.
Version 2 extracts more bits from my sve2 branch. There's
still more to pull back, especially for crypto_helper.c, where
there are also tail clearing bugs to fix.
Version 3 rebases on master, which has some of the arm neon
decodetree conversion applied.
r~
Richard Henderson (16):
target/arm: Create gen_gvec_[us]sra
target/arm: Create gen_gvec_{u,s}{rshr,rsra}
target/arm: Create gen_gvec_{sri,sli}
target/arm: Remove unnecessary range check for VSHL
target/arm: Tidy handle_vec_simd_shri
target/arm: Create gen_gvec_{ceq,clt,cle,cgt,cge}0
target/arm: Create gen_gvec_{mla,mls}
target/arm: Swap argument order for VSHL during decode
target/arm: Create gen_gvec_{cmtst,ushl,sshl}
target/arm: Create gen_gvec_{uqadd,sqadd,uqsub,sqsub}
target/arm: Remove fp_status from helper_{recpe,rsqrte}_u32
target/arm: Create gen_gvec_{qrdmla,qrdmls}
target/arm: Pass pointer to qc to qrdmla/qrdmls
target/arm: Clear tail in gvec_fmul_idx_*, gvec_fmla_idx_*
target/arm: Vectorize SABD/UABD
target/arm: Vectorize SABA/UABA
target/arm/helper.h | 71 +-
target/arm/translate.h | 84 +-
target/arm/neon-dp.decode | 9 +-
target/arm/neon_helper.c | 10 -
target/arm/translate-a64.c | 210 +---
target/arm/translate-neon.inc.c | 59 +-
target/arm/translate.c | 1908 ++++++++++++++++++++-----------
target/arm/vec_helper.c | 233 +++-
target/arm/vfp_helper.c | 4 +-
9 files changed, 1667 insertions(+), 921 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v3 01/16] target/arm: Create gen_gvec_[us]sra
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 13:20 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra} Richard Henderson
` (15 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
The functions eliminate duplication of the special cases for
this operation. They match up with the GVecGen2iFn typedef.
Add out-of-line helpers. We got away with only having inline
expanders because the neon vector size is only 16 bytes, and
we know that the inline expansion will always succeed.
When we reuse this for SVE, tcg-gvec-op may decide to use an
out-of-line helper due to longer vector lengths.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/helper.h | 10 +++
target/arm/translate.h | 7 +-
target/arm/translate-a64.c | 15 +---
target/arm/translate.c | 161 ++++++++++++++++++++++---------------
target/arm/vec_helper.c | 25 ++++++
5 files changed, 139 insertions(+), 79 deletions(-)
diff --git a/target/arm/helper.h b/target/arm/helper.h
index 5817626b20..9bc162345c 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -691,6 +691,16 @@ DEF_HELPER_FLAGS_4(gvec_pmull_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
DEF_HELPER_FLAGS_4(neon_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_ssra_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_ssra_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_ssra_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_ssra_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_3(gvec_usra_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_usra_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_usra_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_usra_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
#ifdef TARGET_AARCH64
#include "helper-a64.h"
#include "helper-sve.h"
diff --git a/target/arm/translate.h b/target/arm/translate.h
index cb7925ea46..1839a59a8e 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -285,8 +285,6 @@ extern const GVecGen3 mls_op[4];
extern const GVecGen3 cmtst_op[4];
extern const GVecGen3 sshl_op[4];
extern const GVecGen3 ushl_op[4];
-extern const GVecGen2i ssra_op[4];
-extern const GVecGen2i usra_op[4];
extern const GVecGen2i sri_op[4];
extern const GVecGen2i sli_op[4];
extern const GVecGen4 uqadd_op[4];
@@ -299,6 +297,11 @@ void gen_sshl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
void gen_ushl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
void gen_sshl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
+void gen_gvec_ssra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_usra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+
/*
* Forward to the isar_feature_* tests given a DisasContext pointer.
*/
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 62e5729904..315de9a9b6 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -10188,19 +10188,8 @@ static void handle_vec_simd_shri(DisasContext *s, bool is_q, bool is_u,
switch (opcode) {
case 0x02: /* SSRA / USRA (accumulate) */
- if (is_u) {
- /* Shift count same as element size produces zero to add. */
- if (shift == 8 << size) {
- goto done;
- }
- gen_gvec_op2i(s, is_q, rd, rn, shift, &usra_op[size]);
- } else {
- /* Shift count same as element size produces all sign to add. */
- if (shift == 8 << size) {
- shift -= 1;
- }
- gen_gvec_op2i(s, is_q, rd, rn, shift, &ssra_op[size]);
- }
+ gen_gvec_fn2i(s, is_q, rd, rn, shift,
+ is_u ? gen_gvec_usra : gen_gvec_ssra, size);
return;
case 0x08: /* SRI */
/* Shift count same as element size is valid but does nothing. */
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 74fac1d09c..c18140f2e6 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -3874,33 +3874,51 @@ static void gen_ssra_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
tcg_gen_add_vec(vece, d, d, a);
}
-static const TCGOpcode vecop_list_ssra[] = {
- INDEX_op_sari_vec, INDEX_op_add_vec, 0
-};
+void gen_gvec_ssra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_sari_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen2i ops[4] = {
+ { .fni8 = gen_ssra8_i64,
+ .fniv = gen_ssra_vec,
+ .fno = gen_helper_gvec_ssra_b,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fni8 = gen_ssra16_i64,
+ .fniv = gen_ssra_vec,
+ .fno = gen_helper_gvec_ssra_h,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_ssra32_i32,
+ .fniv = gen_ssra_vec,
+ .fno = gen_helper_gvec_ssra_s,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_ssra64_i64,
+ .fniv = gen_ssra_vec,
+ .fno = gen_helper_gvec_ssra_b,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_64 },
+ };
-const GVecGen2i ssra_op[4] = {
- { .fni8 = gen_ssra8_i64,
- .fniv = gen_ssra_vec,
- .load_dest = true,
- .opt_opc = vecop_list_ssra,
- .vece = MO_8 },
- { .fni8 = gen_ssra16_i64,
- .fniv = gen_ssra_vec,
- .load_dest = true,
- .opt_opc = vecop_list_ssra,
- .vece = MO_16 },
- { .fni4 = gen_ssra32_i32,
- .fniv = gen_ssra_vec,
- .load_dest = true,
- .opt_opc = vecop_list_ssra,
- .vece = MO_32 },
- { .fni8 = gen_ssra64_i64,
- .fniv = gen_ssra_vec,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .opt_opc = vecop_list_ssra,
- .load_dest = true,
- .vece = MO_64 },
-};
+ /* tszimm encoding produces immediates in the range [1..esize]. */
+ tcg_debug_assert(shift > 0);
+ tcg_debug_assert(shift <= (8 << vece));
+
+ /*
+ * Shifts larger than the element size are architecturally valid.
+ * Signed results in all sign bits.
+ */
+ shift = MIN(shift, (8 << vece) - 1);
+ tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
+}
static void gen_usra8_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
{
@@ -3932,33 +3950,55 @@ static void gen_usra_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
tcg_gen_add_vec(vece, d, d, a);
}
-static const TCGOpcode vecop_list_usra[] = {
- INDEX_op_shri_vec, INDEX_op_add_vec, 0
-};
+void gen_gvec_usra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_shri_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen2i ops[4] = {
+ { .fni8 = gen_usra8_i64,
+ .fniv = gen_usra_vec,
+ .fno = gen_helper_gvec_usra_b,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_8, },
+ { .fni8 = gen_usra16_i64,
+ .fniv = gen_usra_vec,
+ .fno = gen_helper_gvec_usra_h,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_16, },
+ { .fni4 = gen_usra32_i32,
+ .fniv = gen_usra_vec,
+ .fno = gen_helper_gvec_usra_s,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_32, },
+ { .fni8 = gen_usra64_i64,
+ .fniv = gen_usra_vec,
+ .fno = gen_helper_gvec_usra_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_64, },
+ };
-const GVecGen2i usra_op[4] = {
- { .fni8 = gen_usra8_i64,
- .fniv = gen_usra_vec,
- .load_dest = true,
- .opt_opc = vecop_list_usra,
- .vece = MO_8, },
- { .fni8 = gen_usra16_i64,
- .fniv = gen_usra_vec,
- .load_dest = true,
- .opt_opc = vecop_list_usra,
- .vece = MO_16, },
- { .fni4 = gen_usra32_i32,
- .fniv = gen_usra_vec,
- .load_dest = true,
- .opt_opc = vecop_list_usra,
- .vece = MO_32, },
- { .fni8 = gen_usra64_i64,
- .fniv = gen_usra_vec,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .load_dest = true,
- .opt_opc = vecop_list_usra,
- .vece = MO_64, },
-};
+ /* tszimm encoding produces immediates in the range [1..esize]. */
+ tcg_debug_assert(shift > 0);
+ tcg_debug_assert(shift <= (8 << vece));
+
+ /*
+ * Shifts larger than the element size are architecturally valid.
+ * Unsigned results in all zeros as input to accumulate: nop.
+ */
+ if (shift < (8 << vece)) {
+ tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
+ } else {
+ /* Nop, but we do need to clear the tail. */
+ tcg_gen_gvec_mov(vece, rd_ofs, rd_ofs, opr_sz, max_sz);
+ }
+}
static void gen_shr8_ins_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
{
@@ -5220,19 +5260,12 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
case 1: /* VSRA */
/* Right shift comes here negative. */
shift = -shift;
- /* Shifts larger than the element size are architecturally
- * valid. Unsigned results in all zeros; signed results
- * in all sign bits.
- */
- if (!u) {
- tcg_gen_gvec_2i(rd_ofs, rm_ofs, vec_size, vec_size,
- MIN(shift, (8 << size) - 1),
- &ssra_op[size]);
- } else if (shift >= 8 << size) {
- /* rd += 0 */
+ if (u) {
+ gen_gvec_usra(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
} else {
- tcg_gen_gvec_2i(rd_ofs, rm_ofs, vec_size, vec_size,
- shift, &usra_op[size]);
+ gen_gvec_ssra(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
}
return 0;
diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index 3d534188a8..230085b35e 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -899,6 +899,31 @@ void HELPER(gvec_sqsub_d)(void *vd, void *vq, void *vn,
clear_tail(d, oprsz, simd_maxsz(desc));
}
+
+#define DO_SRA(NAME, TYPE) \
+void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \
+{ \
+ intptr_t i, oprsz = simd_oprsz(desc); \
+ int shift = simd_data(desc); \
+ TYPE *d = vd, *n = vn; \
+ for (i = 0; i < oprsz / sizeof(TYPE); i++) { \
+ d[i] += n[i] >> shift; \
+ } \
+ clear_tail(d, oprsz, simd_maxsz(desc)); \
+}
+
+DO_SRA(gvec_ssra_b, int8_t)
+DO_SRA(gvec_ssra_h, int16_t)
+DO_SRA(gvec_ssra_s, int32_t)
+DO_SRA(gvec_ssra_d, int64_t)
+
+DO_SRA(gvec_usra_b, uint8_t)
+DO_SRA(gvec_usra_h, uint16_t)
+DO_SRA(gvec_usra_s, uint32_t)
+DO_SRA(gvec_usra_d, uint64_t)
+
+#undef DO_SRA
+
/*
* Convert float16 to float32, raising no exceptions and
* preserving exceptional values, including SNaN.
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra}
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
2020-05-08 15:21 ` [PATCH v3 01/16] target/arm: Create gen_gvec_[us]sra Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 13:09 ` Peter Maydell
2020-05-12 13:51 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 03/16] target/arm: Create gen_gvec_{sri,sli} Richard Henderson
` (14 subsequent siblings)
16 siblings, 2 replies; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Create vectorized versions of handle_shri_with_rndacc
for shift+round and shift+round+accumulate. Add out-of-line
helpers in preparation for longer vector lengths from SVE.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/helper.h | 20 ++
target/arm/translate.h | 9 +
target/arm/translate-a64.c | 11 +-
target/arm/translate.c | 461 +++++++++++++++++++++++++++++++++++--
target/arm/vec_helper.c | 50 ++++
5 files changed, 525 insertions(+), 26 deletions(-)
diff --git a/target/arm/helper.h b/target/arm/helper.h
index 9bc162345c..aeb1f52455 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -701,6 +701,26 @@ DEF_HELPER_FLAGS_3(gvec_usra_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
DEF_HELPER_FLAGS_3(gvec_usra_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
DEF_HELPER_FLAGS_3(gvec_usra_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_srshr_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_srshr_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_srshr_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_srshr_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_3(gvec_urshr_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_urshr_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_urshr_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_urshr_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_3(gvec_srsra_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_srsra_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_srsra_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_srsra_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_3(gvec_ursra_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_ursra_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_ursra_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_ursra_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
#ifdef TARGET_AARCH64
#include "helper-a64.h"
#include "helper-sve.h"
diff --git a/target/arm/translate.h b/target/arm/translate.h
index 1839a59a8e..1db3b43a61 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -302,6 +302,15 @@ void gen_gvec_ssra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
void gen_gvec_usra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_srshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_urshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_srsra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_ursra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+
/*
* Forward to the isar_feature_* tests given a DisasContext pointer.
*/
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 315de9a9b6..50949d306b 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -10218,10 +10218,15 @@ static void handle_vec_simd_shri(DisasContext *s, bool is_q, bool is_u,
return;
case 0x04: /* SRSHR / URSHR (rounding) */
- break;
+ gen_gvec_fn2i(s, is_q, rd, rn, shift,
+ is_u ? gen_gvec_urshr : gen_gvec_srshr, size);
+ return;
+
case 0x06: /* SRSRA / URSRA (accum + rounding) */
- accumulate = true;
- break;
+ gen_gvec_fn2i(s, is_q, rd, rn, shift,
+ is_u ? gen_gvec_ursra : gen_gvec_srsra, size);
+ return;
+
default:
g_assert_not_reached();
}
diff --git a/target/arm/translate.c b/target/arm/translate.c
index c18140f2e6..19bd514a84 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -4000,6 +4000,422 @@ void gen_gvec_usra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
}
}
+/*
+ * Shift one less than the requested amount, and the low bit is
+ * the rounding bit. For the 8 and 16-bit operations, because we
+ * mask the low bit, we can perform a normal integer shift instead
+ * of a vector shift.
+ */
+static void gen_srshr8_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ tcg_gen_shri_i64(t, a, sh - 1);
+ tcg_gen_andi_i64(t, t, dup_const(MO_8, 1));
+ tcg_gen_vec_sar8i_i64(d, a, sh);
+ tcg_gen_vec_add8_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_srshr16_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ tcg_gen_shri_i64(t, a, sh - 1);
+ tcg_gen_andi_i64(t, t, dup_const(MO_16, 1));
+ tcg_gen_vec_sar16i_i64(d, a, sh);
+ tcg_gen_vec_add16_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_srshr32_i32(TCGv_i32 d, TCGv_i32 a, int32_t sh)
+{
+ TCGv_i32 t = tcg_temp_new_i32();
+
+ tcg_gen_extract_i32(t, a, sh - 1, 1);
+ tcg_gen_sari_i32(d, a, sh);
+ tcg_gen_add_i32(d, d, t);
+ tcg_temp_free_i32(t);
+}
+
+static void gen_srshr64_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ tcg_gen_extract_i64(t, a, sh - 1, 1);
+ tcg_gen_sari_i64(d, a, sh);
+ tcg_gen_add_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_srshr_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
+{
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+ TCGv_vec ones = tcg_temp_new_vec_matching(d);
+
+ tcg_gen_shri_vec(vece, t, a, sh - 1);
+ tcg_gen_dupi_vec(vece, ones, 1);
+ tcg_gen_and_vec(vece, t, t, ones);
+ tcg_gen_sari_vec(vece, d, a, sh);
+ tcg_gen_add_vec(vece, d, d, t);
+
+ tcg_temp_free_vec(t);
+ tcg_temp_free_vec(ones);
+}
+
+void gen_gvec_srshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_shri_vec, INDEX_op_sari_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen2i ops[4] = {
+ { .fni8 = gen_srshr8_i64,
+ .fniv = gen_srshr_vec,
+ .fno = gen_helper_gvec_srshr_b,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fni8 = gen_srshr16_i64,
+ .fniv = gen_srshr_vec,
+ .fno = gen_helper_gvec_srshr_h,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_srshr32_i32,
+ .fniv = gen_srshr_vec,
+ .fno = gen_helper_gvec_srshr_s,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_srshr64_i64,
+ .fniv = gen_srshr_vec,
+ .fno = gen_helper_gvec_srshr_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+
+ /* tszimm encoding produces immediates in the range [1..esize] */
+ tcg_debug_assert(shift > 0);
+ tcg_debug_assert(shift <= (8 << vece));
+
+ if (shift == (8 << vece)) {
+ /*
+ * Shifts larger than the element size are architecturally valid.
+ * Signed results in all sign bits. With rounding, this produces
+ * (-1 + 1) >> 1 == 0, or (0 + 1) >> 1 == 0.
+ * I.e. always zero.
+ */
+ tcg_gen_gvec_dup_imm(vece, rd_ofs, opr_sz, max_sz, 0);
+ } else {
+ tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
+ }
+}
+
+static void gen_srsra8_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ gen_srshr8_i64(t, a, sh);
+ tcg_gen_vec_add8_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_srsra16_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ gen_srshr16_i64(t, a, sh);
+ tcg_gen_vec_add16_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_srsra32_i32(TCGv_i32 d, TCGv_i32 a, int32_t sh)
+{
+ TCGv_i32 t = tcg_temp_new_i32();
+
+ gen_srshr32_i32(t, a, sh);
+ tcg_gen_add_i32(d, d, t);
+ tcg_temp_free_i32(t);
+}
+
+static void gen_srsra64_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ gen_srshr64_i64(t, a, sh);
+ tcg_gen_add_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_srsra_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
+{
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+ gen_srshr_vec(vece, t, a, sh);
+ tcg_gen_add_vec(vece, d, d, t);
+ tcg_temp_free_vec(t);
+}
+
+void gen_gvec_srsra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_shri_vec, INDEX_op_sari_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen2i ops[4] = {
+ { .fni8 = gen_srsra8_i64,
+ .fniv = gen_srsra_vec,
+ .fno = gen_helper_gvec_srsra_b,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_8 },
+ { .fni8 = gen_srsra16_i64,
+ .fniv = gen_srsra_vec,
+ .fno = gen_helper_gvec_srsra_h,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_16 },
+ { .fni4 = gen_srsra32_i32,
+ .fniv = gen_srsra_vec,
+ .fno = gen_helper_gvec_srsra_s,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_32 },
+ { .fni8 = gen_srsra64_i64,
+ .fniv = gen_srsra_vec,
+ .fno = gen_helper_gvec_srsra_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_64 },
+ };
+
+ /* tszimm encoding produces immediates in the range [1..esize] */
+ tcg_debug_assert(shift > 0);
+ tcg_debug_assert(shift <= (8 << vece));
+
+ /*
+ * Shifts larger than the element size are architecturally valid.
+ * Signed results in all sign bits. With rounding, this produces
+ * (-1 + 1) >> 1 == 0, or (0 + 1) >> 1 == 0.
+ * I.e. always zero. With accumulation, this leaves D unchanged.
+ */
+ if (shift == (8 << vece)) {
+ /* Nop, but we do need to clear the tail. */
+ tcg_gen_gvec_mov(vece, rd_ofs, rd_ofs, opr_sz, max_sz);
+ } else {
+ tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
+ }
+}
+
+static void gen_urshr8_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ tcg_gen_shri_i64(t, a, sh - 1);
+ tcg_gen_andi_i64(t, t, dup_const(MO_8, 1));
+ tcg_gen_vec_shr8i_i64(d, a, sh);
+ tcg_gen_vec_add8_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_urshr16_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ tcg_gen_shri_i64(t, a, sh - 1);
+ tcg_gen_andi_i64(t, t, dup_const(MO_16, 1));
+ tcg_gen_vec_shr16i_i64(d, a, sh);
+ tcg_gen_vec_add16_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_urshr32_i32(TCGv_i32 d, TCGv_i32 a, int32_t sh)
+{
+ TCGv_i32 t = tcg_temp_new_i32();
+
+ tcg_gen_extract_i32(t, a, sh - 1, 1);
+ tcg_gen_shri_i32(d, a, sh);
+ tcg_gen_add_i32(d, d, t);
+ tcg_temp_free_i32(t);
+}
+
+static void gen_urshr64_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ tcg_gen_extract_i64(t, a, sh - 1, 1);
+ tcg_gen_shri_i64(d, a, sh);
+ tcg_gen_add_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_urshr_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t shift)
+{
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+ TCGv_vec ones = tcg_temp_new_vec_matching(d);
+
+ tcg_gen_shri_vec(vece, t, a, shift - 1);
+ tcg_gen_dupi_vec(vece, ones, 1);
+ tcg_gen_and_vec(vece, t, t, ones);
+ tcg_gen_shri_vec(vece, d, a, shift);
+ tcg_gen_add_vec(vece, d, d, t);
+
+ tcg_temp_free_vec(t);
+ tcg_temp_free_vec(ones);
+}
+
+void gen_gvec_urshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_shri_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen2i ops[4] = {
+ { .fni8 = gen_urshr8_i64,
+ .fniv = gen_urshr_vec,
+ .fno = gen_helper_gvec_urshr_b,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fni8 = gen_urshr16_i64,
+ .fniv = gen_urshr_vec,
+ .fno = gen_helper_gvec_urshr_h,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_urshr32_i32,
+ .fniv = gen_urshr_vec,
+ .fno = gen_helper_gvec_urshr_s,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_urshr64_i64,
+ .fniv = gen_urshr_vec,
+ .fno = gen_helper_gvec_urshr_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+
+ /* tszimm encoding produces immediates in the range [1..esize] */
+ tcg_debug_assert(shift > 0);
+ tcg_debug_assert(shift <= (8 << vece));
+
+ if (shift == (8 << vece)) {
+ /*
+ * Shifts larger than the element size are architecturally valid.
+ * Unsigned results in zero. With rounding, this produces a
+ * copy of the most significant bit.
+ */
+ tcg_gen_gvec_shri(vece, rd_ofs, rm_ofs, shift - 1, opr_sz, max_sz);
+ } else {
+ tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
+ }
+}
+
+static void gen_ursra8_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ if (sh == 8) {
+ tcg_gen_vec_shr8i_i64(t, a, 7);
+ } else {
+ gen_urshr8_i64(t, a, sh);
+ }
+ tcg_gen_vec_add8_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_ursra16_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ if (sh == 16) {
+ tcg_gen_vec_shr16i_i64(t, a, 15);
+ } else {
+ gen_urshr16_i64(t, a, sh);
+ }
+ tcg_gen_vec_add16_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_ursra32_i32(TCGv_i32 d, TCGv_i32 a, int32_t sh)
+{
+ TCGv_i32 t = tcg_temp_new_i32();
+
+ if (sh == 32) {
+ tcg_gen_shri_i32(t, a, 31);
+ } else {
+ gen_urshr32_i32(t, a, sh);
+ }
+ tcg_gen_add_i32(d, d, t);
+ tcg_temp_free_i32(t);
+}
+
+static void gen_ursra64_i64(TCGv_i64 d, TCGv_i64 a, int64_t sh)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ if (sh == 64) {
+ tcg_gen_shri_i64(t, a, 63);
+ } else {
+ gen_urshr64_i64(t, a, sh);
+ }
+ tcg_gen_add_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_ursra_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
+{
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+ if (sh == (8 << vece)) {
+ tcg_gen_shri_vec(vece, t, a, sh - 1);
+ } else {
+ gen_urshr_vec(vece, t, a, sh);
+ }
+ tcg_gen_add_vec(vece, d, d, t);
+ tcg_temp_free_vec(t);
+}
+
+void gen_gvec_ursra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_shri_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen2i ops[4] = {
+ { .fni8 = gen_ursra8_i64,
+ .fniv = gen_ursra_vec,
+ .fno = gen_helper_gvec_ursra_b,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_8 },
+ { .fni8 = gen_ursra16_i64,
+ .fniv = gen_ursra_vec,
+ .fno = gen_helper_gvec_ursra_h,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_16 },
+ { .fni4 = gen_ursra32_i32,
+ .fniv = gen_ursra_vec,
+ .fno = gen_helper_gvec_ursra_s,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_32 },
+ { .fni8 = gen_ursra64_i64,
+ .fniv = gen_ursra_vec,
+ .fno = gen_helper_gvec_ursra_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_64 },
+ };
+
+ /* tszimm encoding produces immediates in the range [1..esize] */
+ tcg_debug_assert(shift > 0);
+ tcg_debug_assert(shift <= (8 << vece));
+
+ tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
+}
+
static void gen_shr8_ins_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
{
uint64_t mask = dup_const(MO_8, 0xff >> shift);
@@ -5269,6 +5685,28 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
}
return 0;
+ case 2: /* VRSHR */
+ /* Right shift comes here negative. */
+ shift = -shift;
+ if (u) {
+ gen_gvec_urshr(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
+ } else {
+ gen_gvec_srshr(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
+ }
+ return 0;
+
+ case 3: /* VRSRA */
+ if (u) {
+ gen_gvec_ursra(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
+ } else {
+ gen_gvec_srsra(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
+ }
+ return 0;
+
case 4: /* VSRI */
if (!u) {
return 1;
@@ -5320,13 +5758,6 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
neon_load_reg64(cpu_V0, rm + pass);
tcg_gen_movi_i64(cpu_V1, imm);
switch (op) {
- case 2: /* VRSHR */
- case 3: /* VRSRA */
- if (u)
- gen_helper_neon_rshl_u64(cpu_V0, cpu_V0, cpu_V1);
- else
- gen_helper_neon_rshl_s64(cpu_V0, cpu_V0, cpu_V1);
- break;
case 6: /* VQSHLU */
gen_helper_neon_qshlu_s64(cpu_V0, cpu_env,
cpu_V0, cpu_V1);
@@ -5343,11 +5774,6 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
default:
g_assert_not_reached();
}
- if (op == 3) {
- /* Accumulate. */
- neon_load_reg64(cpu_V1, rd + pass);
- tcg_gen_add_i64(cpu_V0, cpu_V0, cpu_V1);
- }
neon_store_reg64(cpu_V0, rd + pass);
} else { /* size < 3 */
/* Operands in T0 and T1. */
@@ -5355,10 +5781,6 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
tmp2 = tcg_temp_new_i32();
tcg_gen_movi_i32(tmp2, imm);
switch (op) {
- case 2: /* VRSHR */
- case 3: /* VRSRA */
- GEN_NEON_INTEGER_OP(rshl);
- break;
case 6: /* VQSHLU */
switch (size) {
case 0:
@@ -5384,13 +5806,6 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
g_assert_not_reached();
}
tcg_temp_free_i32(tmp2);
-
- if (op == 3) {
- /* Accumulate. */
- tmp2 = neon_load_reg(rd, pass);
- gen_neon_add(size, tmp, tmp2);
- tcg_temp_free_i32(tmp2);
- }
neon_store_reg(rd, pass, tmp);
}
} /* for pass */
diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index 230085b35e..fd8b2bff49 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -924,6 +924,56 @@ DO_SRA(gvec_usra_d, uint64_t)
#undef DO_SRA
+#define DO_RSHR(NAME, TYPE) \
+void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \
+{ \
+ intptr_t i, oprsz = simd_oprsz(desc); \
+ int shift = simd_data(desc); \
+ TYPE *d = vd, *n = vn; \
+ for (i = 0; i < oprsz / sizeof(TYPE); i++) { \
+ TYPE tmp = n[i] >> (shift - 1); \
+ d[i] = (tmp >> 1) + (tmp & 1); \
+ } \
+ clear_tail(d, oprsz, simd_maxsz(desc)); \
+}
+
+DO_RSHR(gvec_srshr_b, int8_t)
+DO_RSHR(gvec_srshr_h, int16_t)
+DO_RSHR(gvec_srshr_s, int32_t)
+DO_RSHR(gvec_srshr_d, int64_t)
+
+DO_RSHR(gvec_urshr_b, uint8_t)
+DO_RSHR(gvec_urshr_h, uint16_t)
+DO_RSHR(gvec_urshr_s, uint32_t)
+DO_RSHR(gvec_urshr_d, uint64_t)
+
+#undef DO_RSHR
+
+#define DO_RSRA(NAME, TYPE) \
+void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \
+{ \
+ intptr_t i, oprsz = simd_oprsz(desc); \
+ int shift = simd_data(desc); \
+ TYPE *d = vd, *n = vn; \
+ for (i = 0; i < oprsz / sizeof(TYPE); i++) { \
+ TYPE tmp = n[i] >> (shift - 1); \
+ d[i] += (tmp >> 1) + (tmp & 1); \
+ } \
+ clear_tail(d, oprsz, simd_maxsz(desc)); \
+}
+
+DO_RSRA(gvec_srsra_b, int8_t)
+DO_RSRA(gvec_srsra_h, int16_t)
+DO_RSRA(gvec_srsra_s, int32_t)
+DO_RSRA(gvec_srsra_d, int64_t)
+
+DO_RSRA(gvec_ursra_b, uint8_t)
+DO_RSRA(gvec_ursra_h, uint16_t)
+DO_RSRA(gvec_ursra_s, uint32_t)
+DO_RSRA(gvec_ursra_d, uint64_t)
+
+#undef DO_RSRA
+
/*
* Convert float16 to float32, raising no exceptions and
* preserving exceptional values, including SNaN.
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 03/16] target/arm: Create gen_gvec_{sri,sli}
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
2020-05-08 15:21 ` [PATCH v3 01/16] target/arm: Create gen_gvec_[us]sra Richard Henderson
2020-05-08 15:21 ` [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra} Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 13:52 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 04/16] target/arm: Remove unnecessary range check for VSHL Richard Henderson
` (13 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
The functions eliminate duplication of the special cases for
this operation. They match up with the GVecGen2iFn typedef.
Add out-of-line helpers. We got away with only having inline
expanders because the neon vector size is only 16 bytes, and
we know that the inline expansion will always succeed.
When we reuse this for SVE, tcg-gvec-op may decide to use an
out-of-line helper due to longer vector lengths.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/helper.h | 10 ++
target/arm/translate.h | 7 +-
target/arm/translate-a64.c | 20 +---
target/arm/translate.c | 186 +++++++++++++++++++++----------------
target/arm/vec_helper.c | 38 ++++++++
5 files changed, 160 insertions(+), 101 deletions(-)
diff --git a/target/arm/helper.h b/target/arm/helper.h
index aeb1f52455..33c76192d2 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -721,6 +721,16 @@ DEF_HELPER_FLAGS_3(gvec_ursra_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
DEF_HELPER_FLAGS_3(gvec_ursra_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
DEF_HELPER_FLAGS_3(gvec_ursra_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_sri_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_sri_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_sri_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_sri_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_3(gvec_sli_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_sli_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_sli_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_sli_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
#ifdef TARGET_AARCH64
#include "helper-a64.h"
#include "helper-sve.h"
diff --git a/target/arm/translate.h b/target/arm/translate.h
index 1db3b43a61..fa5c3f12b9 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -285,8 +285,6 @@ extern const GVecGen3 mls_op[4];
extern const GVecGen3 cmtst_op[4];
extern const GVecGen3 sshl_op[4];
extern const GVecGen3 ushl_op[4];
-extern const GVecGen2i sri_op[4];
-extern const GVecGen2i sli_op[4];
extern const GVecGen4 uqadd_op[4];
extern const GVecGen4 sqadd_op[4];
extern const GVecGen4 uqsub_op[4];
@@ -311,6 +309,11 @@ void gen_gvec_srsra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
void gen_gvec_ursra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sri(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sli(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+
/*
* Forward to the isar_feature_* tests given a DisasContext pointer.
*/
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 50949d306b..2d7dad6c3f 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -585,16 +585,6 @@ static void gen_gvec_op2(DisasContext *s, bool is_q, int rd,
is_q ? 16 : 8, vec_full_reg_size(s), gvec_op);
}
-/* Expand a 2-operand + immediate AdvSIMD vector operation using
- * an op descriptor.
- */
-static void gen_gvec_op2i(DisasContext *s, bool is_q, int rd,
- int rn, int64_t imm, const GVecGen2i *gvec_op)
-{
- tcg_gen_gvec_2i(vec_full_reg_offset(s, rd), vec_full_reg_offset(s, rn),
- is_q ? 16 : 8, vec_full_reg_size(s), imm, gvec_op);
-}
-
/* Expand a 3-operand AdvSIMD vector operation using an op descriptor. */
static void gen_gvec_op3(DisasContext *s, bool is_q, int rd,
int rn, int rm, const GVecGen3 *gvec_op)
@@ -10191,12 +10181,9 @@ static void handle_vec_simd_shri(DisasContext *s, bool is_q, bool is_u,
gen_gvec_fn2i(s, is_q, rd, rn, shift,
is_u ? gen_gvec_usra : gen_gvec_ssra, size);
return;
+
case 0x08: /* SRI */
- /* Shift count same as element size is valid but does nothing. */
- if (shift == 8 << size) {
- goto done;
- }
- gen_gvec_op2i(s, is_q, rd, rn, shift, &sri_op[size]);
+ gen_gvec_fn2i(s, is_q, rd, rn, shift, gen_gvec_sri, size);
return;
case 0x00: /* SSHR / USHR */
@@ -10247,7 +10234,6 @@ static void handle_vec_simd_shri(DisasContext *s, bool is_q, bool is_u,
}
tcg_temp_free_i64(tcg_round);
- done:
clear_vec_high(s, is_q, rd);
}
@@ -10272,7 +10258,7 @@ static void handle_vec_simd_shli(DisasContext *s, bool is_q, bool insert,
}
if (insert) {
- gen_gvec_op2i(s, is_q, rd, rn, shift, &sli_op[size]);
+ gen_gvec_fn2i(s, is_q, rd, rn, shift, gen_gvec_sli, size);
} else {
gen_gvec_fn2i(s, is_q, rd, rn, shift, tcg_gen_gvec_shli, size);
}
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 19bd514a84..e221d0c959 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -4454,47 +4454,62 @@ static void gen_shr64_ins_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
static void gen_shr_ins_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
{
- if (sh == 0) {
- tcg_gen_mov_vec(d, a);
- } else {
- TCGv_vec t = tcg_temp_new_vec_matching(d);
- TCGv_vec m = tcg_temp_new_vec_matching(d);
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+ TCGv_vec m = tcg_temp_new_vec_matching(d);
- tcg_gen_dupi_vec(vece, m, MAKE_64BIT_MASK((8 << vece) - sh, sh));
- tcg_gen_shri_vec(vece, t, a, sh);
- tcg_gen_and_vec(vece, d, d, m);
- tcg_gen_or_vec(vece, d, d, t);
+ tcg_gen_dupi_vec(vece, m, MAKE_64BIT_MASK((8 << vece) - sh, sh));
+ tcg_gen_shri_vec(vece, t, a, sh);
+ tcg_gen_and_vec(vece, d, d, m);
+ tcg_gen_or_vec(vece, d, d, t);
- tcg_temp_free_vec(t);
- tcg_temp_free_vec(m);
- }
+ tcg_temp_free_vec(t);
+ tcg_temp_free_vec(m);
}
-static const TCGOpcode vecop_list_sri[] = { INDEX_op_shri_vec, 0 };
+void gen_gvec_sri(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = { INDEX_op_shri_vec, 0 };
+ const GVecGen2i ops[4] = {
+ { .fni8 = gen_shr8_ins_i64,
+ .fniv = gen_shr_ins_vec,
+ .fno = gen_helper_gvec_sri_b,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fni8 = gen_shr16_ins_i64,
+ .fniv = gen_shr_ins_vec,
+ .fno = gen_helper_gvec_sri_h,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_shr32_ins_i32,
+ .fniv = gen_shr_ins_vec,
+ .fno = gen_helper_gvec_sri_s,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_shr64_ins_i64,
+ .fniv = gen_shr_ins_vec,
+ .fno = gen_helper_gvec_sri_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
-const GVecGen2i sri_op[4] = {
- { .fni8 = gen_shr8_ins_i64,
- .fniv = gen_shr_ins_vec,
- .load_dest = true,
- .opt_opc = vecop_list_sri,
- .vece = MO_8 },
- { .fni8 = gen_shr16_ins_i64,
- .fniv = gen_shr_ins_vec,
- .load_dest = true,
- .opt_opc = vecop_list_sri,
- .vece = MO_16 },
- { .fni4 = gen_shr32_ins_i32,
- .fniv = gen_shr_ins_vec,
- .load_dest = true,
- .opt_opc = vecop_list_sri,
- .vece = MO_32 },
- { .fni8 = gen_shr64_ins_i64,
- .fniv = gen_shr_ins_vec,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .load_dest = true,
- .opt_opc = vecop_list_sri,
- .vece = MO_64 },
-};
+ /* tszimm encoding produces immediates in the range [1..esize]. */
+ tcg_debug_assert(shift > 0);
+ tcg_debug_assert(shift <= (8 << vece));
+
+ /* Shift of esize leaves destination unchanged. */
+ if (shift < (8 << vece)) {
+ tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
+ } else {
+ /* Nop, but we do need to clear the tail. */
+ tcg_gen_gvec_mov(vece, rd_ofs, rd_ofs, opr_sz, max_sz);
+ }
+}
static void gen_shl8_ins_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
{
@@ -4532,47 +4547,60 @@ static void gen_shl64_ins_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
static void gen_shl_ins_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
{
- if (sh == 0) {
- tcg_gen_mov_vec(d, a);
- } else {
- TCGv_vec t = tcg_temp_new_vec_matching(d);
- TCGv_vec m = tcg_temp_new_vec_matching(d);
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+ TCGv_vec m = tcg_temp_new_vec_matching(d);
- tcg_gen_dupi_vec(vece, m, MAKE_64BIT_MASK(0, sh));
- tcg_gen_shli_vec(vece, t, a, sh);
- tcg_gen_and_vec(vece, d, d, m);
- tcg_gen_or_vec(vece, d, d, t);
+ tcg_gen_shli_vec(vece, t, a, sh);
+ tcg_gen_dupi_vec(vece, m, MAKE_64BIT_MASK(0, sh));
+ tcg_gen_and_vec(vece, d, d, m);
+ tcg_gen_or_vec(vece, d, d, t);
- tcg_temp_free_vec(t);
- tcg_temp_free_vec(m);
- }
+ tcg_temp_free_vec(t);
+ tcg_temp_free_vec(m);
}
-static const TCGOpcode vecop_list_sli[] = { INDEX_op_shli_vec, 0 };
+void gen_gvec_sli(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = { INDEX_op_shli_vec, 0 };
+ const GVecGen2i ops[4] = {
+ { .fni8 = gen_shl8_ins_i64,
+ .fniv = gen_shl_ins_vec,
+ .fno = gen_helper_gvec_sli_b,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fni8 = gen_shl16_ins_i64,
+ .fniv = gen_shl_ins_vec,
+ .fno = gen_helper_gvec_sli_h,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_shl32_ins_i32,
+ .fniv = gen_shl_ins_vec,
+ .fno = gen_helper_gvec_sli_s,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_shl64_ins_i64,
+ .fniv = gen_shl_ins_vec,
+ .fno = gen_helper_gvec_sli_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
-const GVecGen2i sli_op[4] = {
- { .fni8 = gen_shl8_ins_i64,
- .fniv = gen_shl_ins_vec,
- .load_dest = true,
- .opt_opc = vecop_list_sli,
- .vece = MO_8 },
- { .fni8 = gen_shl16_ins_i64,
- .fniv = gen_shl_ins_vec,
- .load_dest = true,
- .opt_opc = vecop_list_sli,
- .vece = MO_16 },
- { .fni4 = gen_shl32_ins_i32,
- .fniv = gen_shl_ins_vec,
- .load_dest = true,
- .opt_opc = vecop_list_sli,
- .vece = MO_32 },
- { .fni8 = gen_shl64_ins_i64,
- .fniv = gen_shl_ins_vec,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .load_dest = true,
- .opt_opc = vecop_list_sli,
- .vece = MO_64 },
-};
+ /* tszimm encoding produces immediates in the range [0..esize-1]. */
+ tcg_debug_assert(shift >= 0);
+ tcg_debug_assert(shift < (8 << vece));
+
+ if (shift == 0) {
+ tcg_gen_gvec_mov(vece, rd_ofs, rm_ofs, opr_sz, max_sz);
+ } else {
+ tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
+ }
+}
static void gen_mla8_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
{
@@ -5713,20 +5741,14 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
}
/* Right shift comes here negative. */
shift = -shift;
- /* Shift out of range leaves destination unchanged. */
- if (shift < 8 << size) {
- tcg_gen_gvec_2i(rd_ofs, rm_ofs, vec_size, vec_size,
- shift, &sri_op[size]);
- }
+ gen_gvec_sri(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
return 0;
case 5: /* VSHL, VSLI */
if (u) { /* VSLI */
- /* Shift out of range leaves destination unchanged. */
- if (shift < 8 << size) {
- tcg_gen_gvec_2i(rd_ofs, rm_ofs, vec_size,
- vec_size, shift, &sli_op[size]);
- }
+ gen_gvec_sli(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
} else { /* VSHL */
/* Shifts larger than the element size are
* architecturally valid and results in zero.
diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index fd8b2bff49..096fea67ef 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -974,6 +974,44 @@ DO_RSRA(gvec_ursra_d, uint64_t)
#undef DO_RSRA
+#define DO_SRI(NAME, TYPE) \
+void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \
+{ \
+ intptr_t i, oprsz = simd_oprsz(desc); \
+ int shift = simd_data(desc); \
+ TYPE *d = vd, *n = vn; \
+ for (i = 0; i < oprsz / sizeof(TYPE); i++) { \
+ d[i] = deposit64(d[i], 0, sizeof(TYPE) * 8 - shift, n[i] >> shift); \
+ } \
+ clear_tail(d, oprsz, simd_maxsz(desc)); \
+}
+
+DO_SRI(gvec_sri_b, uint8_t)
+DO_SRI(gvec_sri_h, uint16_t)
+DO_SRI(gvec_sri_s, uint32_t)
+DO_SRI(gvec_sri_d, uint64_t)
+
+#undef DO_SRI
+
+#define DO_SLI(NAME, TYPE) \
+void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \
+{ \
+ intptr_t i, oprsz = simd_oprsz(desc); \
+ int shift = simd_data(desc); \
+ TYPE *d = vd, *n = vn; \
+ for (i = 0; i < oprsz / sizeof(TYPE); i++) { \
+ d[i] = deposit64(d[i], shift, sizeof(TYPE) * 8 - shift, n[i]); \
+ } \
+ clear_tail(d, oprsz, simd_maxsz(desc)); \
+}
+
+DO_SLI(gvec_sli_b, uint8_t)
+DO_SLI(gvec_sli_h, uint16_t)
+DO_SLI(gvec_sli_s, uint32_t)
+DO_SLI(gvec_sli_d, uint64_t)
+
+#undef DO_SLI
+
/*
* Convert float16 to float32, raising no exceptions and
* preserving exceptional values, including SNaN.
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 04/16] target/arm: Remove unnecessary range check for VSHL
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (2 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 03/16] target/arm: Create gen_gvec_{sri,sli} Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 13:53 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 05/16] target/arm: Tidy handle_vec_simd_shri Richard Henderson
` (12 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
In 1dc8425e551, while converting to gvec, I added an extra range check
against the shift count. This was unnecessary because the encoding of
the shift count produces 0 to the element size - 1.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/translate.c | 12 ++----------
1 file changed, 2 insertions(+), 10 deletions(-)
diff --git a/target/arm/translate.c b/target/arm/translate.c
index e221d0c959..967108b3f4 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -5750,16 +5750,8 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
gen_gvec_sli(size, rd_ofs, rm_ofs, shift,
vec_size, vec_size);
} else { /* VSHL */
- /* Shifts larger than the element size are
- * architecturally valid and results in zero.
- */
- if (shift >= 8 << size) {
- tcg_gen_gvec_dup_imm(size, rd_ofs,
- vec_size, vec_size, 0);
- } else {
- tcg_gen_gvec_shli(size, rd_ofs, rm_ofs, shift,
- vec_size, vec_size);
- }
+ tcg_gen_gvec_shli(size, rd_ofs, rm_ofs, shift,
+ vec_size, vec_size);
}
return 0;
}
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 05/16] target/arm: Tidy handle_vec_simd_shri
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (3 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 04/16] target/arm: Remove unnecessary range check for VSHL Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 13:56 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 06/16] target/arm: Create gen_gvec_{ceq,clt,cle,cgt,cge}0 Richard Henderson
` (11 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Now that we've converted all cases to gvec, there is quite a bit
of dead code at the end of the function. Remove it.
Sink the call to gen_gvec_fn2i to the end, loading a function
pointer within the switch statement.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/translate-a64.c | 56 ++++++++++----------------------------
1 file changed, 14 insertions(+), 42 deletions(-)
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 2d7dad6c3f..d5e77f34a7 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -10155,16 +10155,7 @@ static void handle_vec_simd_shri(DisasContext *s, bool is_q, bool is_u,
int size = 32 - clz32(immh) - 1;
int immhb = immh << 3 | immb;
int shift = 2 * (8 << size) - immhb;
- bool accumulate = false;
- int dsize = is_q ? 128 : 64;
- int esize = 8 << size;
- int elements = dsize/esize;
- MemOp memop = size | (is_u ? 0 : MO_SIGN);
- TCGv_i64 tcg_rn = new_tmp_a64(s);
- TCGv_i64 tcg_rd = new_tmp_a64(s);
- TCGv_i64 tcg_round;
- uint64_t round_const;
- int i;
+ GVecGen2iFn *gvec_fn;
if (extract32(immh, 3, 1) && !is_q) {
unallocated_encoding(s);
@@ -10178,13 +10169,12 @@ static void handle_vec_simd_shri(DisasContext *s, bool is_q, bool is_u,
switch (opcode) {
case 0x02: /* SSRA / USRA (accumulate) */
- gen_gvec_fn2i(s, is_q, rd, rn, shift,
- is_u ? gen_gvec_usra : gen_gvec_ssra, size);
- return;
+ gvec_fn = is_u ? gen_gvec_usra : gen_gvec_ssra;
+ break;
case 0x08: /* SRI */
- gen_gvec_fn2i(s, is_q, rd, rn, shift, gen_gvec_sri, size);
- return;
+ gvec_fn = gen_gvec_sri;
+ break;
case 0x00: /* SSHR / USHR */
if (is_u) {
@@ -10192,49 +10182,31 @@ static void handle_vec_simd_shri(DisasContext *s, bool is_q, bool is_u,
/* Shift count the same size as element size produces zero. */
tcg_gen_gvec_dup_imm(size, vec_full_reg_offset(s, rd),
is_q ? 16 : 8, vec_full_reg_size(s), 0);
- } else {
- gen_gvec_fn2i(s, is_q, rd, rn, shift, tcg_gen_gvec_shri, size);
+ return;
}
+ gvec_fn = tcg_gen_gvec_shri;
} else {
/* Shift count the same size as element size produces all sign. */
if (shift == 8 << size) {
shift -= 1;
}
- gen_gvec_fn2i(s, is_q, rd, rn, shift, tcg_gen_gvec_sari, size);
+ gvec_fn = tcg_gen_gvec_sari;
}
- return;
+ break;
case 0x04: /* SRSHR / URSHR (rounding) */
- gen_gvec_fn2i(s, is_q, rd, rn, shift,
- is_u ? gen_gvec_urshr : gen_gvec_srshr, size);
- return;
+ gvec_fn = is_u ? gen_gvec_urshr : gen_gvec_srshr;
+ break;
case 0x06: /* SRSRA / URSRA (accum + rounding) */
- gen_gvec_fn2i(s, is_q, rd, rn, shift,
- is_u ? gen_gvec_ursra : gen_gvec_srsra, size);
- return;
+ gvec_fn = is_u ? gen_gvec_ursra : gen_gvec_srsra;
+ break;
default:
g_assert_not_reached();
}
- round_const = 1ULL << (shift - 1);
- tcg_round = tcg_const_i64(round_const);
-
- for (i = 0; i < elements; i++) {
- read_vec_element(s, tcg_rn, rn, i, memop);
- if (accumulate) {
- read_vec_element(s, tcg_rd, rd, i, memop);
- }
-
- handle_shri_with_rndacc(tcg_rd, tcg_rn, tcg_round,
- accumulate, is_u, size, shift);
-
- write_vec_element(s, tcg_rd, rd, i, size);
- }
- tcg_temp_free_i64(tcg_round);
-
- clear_vec_high(s, is_q, rd);
+ gen_gvec_fn2i(s, is_q, rd, rn, shift, gvec_fn, size);
}
/* SHL/SLI - Vector shift left */
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 06/16] target/arm: Create gen_gvec_{ceq,clt,cle,cgt,cge}0
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (4 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 05/16] target/arm: Tidy handle_vec_simd_shri Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:10 ` [PATCH v3 06/16] target/arm: Create gen_gvec_{ceq, clt, cle, cgt, cge}0 Peter Maydell
2020-05-08 15:21 ` [PATCH v3 07/16] target/arm: Create gen_gvec_{mla,mls} Richard Henderson
` (10 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.
Macro-ize the 5 nearly identical comparisons.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/translate.h | 16 ++-
target/arm/translate-a64.c | 22 ++--
target/arm/translate.c | 254 ++++++++-----------------------------
3 files changed, 74 insertions(+), 218 deletions(-)
diff --git a/target/arm/translate.h b/target/arm/translate.h
index fa5c3f12b9..e35c812cc5 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -275,11 +275,17 @@ static inline void gen_swstep_exception(DisasContext *s, int isv, int ex)
uint64_t vfp_expand_imm(int size, uint8_t imm8);
/* Vector operations shared between ARM and AArch64. */
-extern const GVecGen2 ceq0_op[4];
-extern const GVecGen2 clt0_op[4];
-extern const GVecGen2 cgt0_op[4];
-extern const GVecGen2 cle0_op[4];
-extern const GVecGen2 cge0_op[4];
+void gen_gvec_ceq0(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_clt0(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_cgt0(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_cle0(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_cge0(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ uint32_t opr_sz, uint32_t max_sz);
+
extern const GVecGen3 mla_op[4];
extern const GVecGen3 mls_op[4];
extern const GVecGen3 cmtst_op[4];
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index d5e77f34a7..fef93dc27a 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -577,14 +577,6 @@ static void gen_gvec_fn4(DisasContext *s, bool is_q, int rd, int rn, int rm,
is_q ? 16 : 8, vec_full_reg_size(s));
}
-/* Expand a 2-operand AdvSIMD vector operation using an op descriptor. */
-static void gen_gvec_op2(DisasContext *s, bool is_q, int rd,
- int rn, const GVecGen2 *gvec_op)
-{
- tcg_gen_gvec_2(vec_full_reg_offset(s, rd), vec_full_reg_offset(s, rn),
- is_q ? 16 : 8, vec_full_reg_size(s), gvec_op);
-}
-
/* Expand a 3-operand AdvSIMD vector operation using an op descriptor. */
static void gen_gvec_op3(DisasContext *s, bool is_q, int rd,
int rn, int rm, const GVecGen3 *gvec_op)
@@ -12310,13 +12302,21 @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn)
}
break;
case 0x8: /* CMGT, CMGE */
- gen_gvec_op2(s, is_q, rd, rn, u ? &cge0_op[size] : &cgt0_op[size]);
+ if (u) {
+ gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cge0, size);
+ } else {
+ gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cgt0, size);
+ }
return;
case 0x9: /* CMEQ, CMLE */
- gen_gvec_op2(s, is_q, rd, rn, u ? &cle0_op[size] : &ceq0_op[size]);
+ if (u) {
+ gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_cle0, size);
+ } else {
+ gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_ceq0, size);
+ }
return;
case 0xa: /* CMLT */
- gen_gvec_op2(s, is_q, rd, rn, &clt0_op[size]);
+ gen_gvec_fn2(s, is_q, rd, rn, gen_gvec_clt0, size);
return;
case 0xb:
if (u) { /* ABS, NEG */
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 967108b3f4..45df3743f6 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -3645,204 +3645,59 @@ static int do_v81_helper(DisasContext *s, gen_helper_gvec_3_ptr *fn,
return 1;
}
-static void gen_ceq0_i32(TCGv_i32 d, TCGv_i32 a)
-{
- tcg_gen_setcondi_i32(TCG_COND_EQ, d, a, 0);
- tcg_gen_neg_i32(d, d);
-}
-
-static void gen_ceq0_i64(TCGv_i64 d, TCGv_i64 a)
-{
- tcg_gen_setcondi_i64(TCG_COND_EQ, d, a, 0);
- tcg_gen_neg_i64(d, d);
-}
-
-static void gen_ceq0_vec(unsigned vece, TCGv_vec d, TCGv_vec a)
-{
- TCGv_vec zero = tcg_const_zeros_vec_matching(d);
- tcg_gen_cmp_vec(TCG_COND_EQ, vece, d, a, zero);
- tcg_temp_free_vec(zero);
-}
+#define GEN_CMP0(NAME, COND) \
+ static void gen_##NAME##0_i32(TCGv_i32 d, TCGv_i32 a) \
+ { \
+ tcg_gen_setcondi_i32(COND, d, a, 0); \
+ tcg_gen_neg_i32(d, d); \
+ } \
+ static void gen_##NAME##0_i64(TCGv_i64 d, TCGv_i64 a) \
+ { \
+ tcg_gen_setcondi_i64(COND, d, a, 0); \
+ tcg_gen_neg_i64(d, d); \
+ } \
+ static void gen_##NAME##0_vec(unsigned vece, TCGv_vec d, TCGv_vec a) \
+ { \
+ TCGv_vec zero = tcg_const_zeros_vec_matching(d); \
+ tcg_gen_cmp_vec(COND, vece, d, a, zero); \
+ tcg_temp_free_vec(zero); \
+ } \
+ void gen_gvec_##NAME##0(unsigned vece, uint32_t d, uint32_t m, \
+ uint32_t opr_sz, uint32_t max_sz) \
+ { \
+ const GVecGen2 op[4] = { \
+ { .fno = gen_helper_gvec_##NAME##0_b, \
+ .fniv = gen_##NAME##0_vec, \
+ .opt_opc = vecop_list_cmp, \
+ .vece = MO_8 }, \
+ { .fno = gen_helper_gvec_##NAME##0_h, \
+ .fniv = gen_##NAME##0_vec, \
+ .opt_opc = vecop_list_cmp, \
+ .vece = MO_16 }, \
+ { .fni4 = gen_##NAME##0_i32, \
+ .fniv = gen_##NAME##0_vec, \
+ .opt_opc = vecop_list_cmp, \
+ .vece = MO_32 }, \
+ { .fni8 = gen_##NAME##0_i64, \
+ .fniv = gen_##NAME##0_vec, \
+ .opt_opc = vecop_list_cmp, \
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64, \
+ .vece = MO_64 }, \
+ }; \
+ tcg_gen_gvec_2(d, m, opr_sz, max_sz, &op[vece]); \
+ }
static const TCGOpcode vecop_list_cmp[] = {
INDEX_op_cmp_vec, 0
};
-const GVecGen2 ceq0_op[4] = {
- { .fno = gen_helper_gvec_ceq0_b,
- .fniv = gen_ceq0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_8 },
- { .fno = gen_helper_gvec_ceq0_h,
- .fniv = gen_ceq0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_16 },
- { .fni4 = gen_ceq0_i32,
- .fniv = gen_ceq0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_32 },
- { .fni8 = gen_ceq0_i64,
- .fniv = gen_ceq0_vec,
- .opt_opc = vecop_list_cmp,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .vece = MO_64 },
-};
+GEN_CMP0(ceq, TCG_COND_EQ)
+GEN_CMP0(cle, TCG_COND_LE)
+GEN_CMP0(cge, TCG_COND_GE)
+GEN_CMP0(clt, TCG_COND_LT)
+GEN_CMP0(cgt, TCG_COND_GT)
-static void gen_cle0_i32(TCGv_i32 d, TCGv_i32 a)
-{
- tcg_gen_setcondi_i32(TCG_COND_LE, d, a, 0);
- tcg_gen_neg_i32(d, d);
-}
-
-static void gen_cle0_i64(TCGv_i64 d, TCGv_i64 a)
-{
- tcg_gen_setcondi_i64(TCG_COND_LE, d, a, 0);
- tcg_gen_neg_i64(d, d);
-}
-
-static void gen_cle0_vec(unsigned vece, TCGv_vec d, TCGv_vec a)
-{
- TCGv_vec zero = tcg_const_zeros_vec_matching(d);
- tcg_gen_cmp_vec(TCG_COND_LE, vece, d, a, zero);
- tcg_temp_free_vec(zero);
-}
-
-const GVecGen2 cle0_op[4] = {
- { .fno = gen_helper_gvec_cle0_b,
- .fniv = gen_cle0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_8 },
- { .fno = gen_helper_gvec_cle0_h,
- .fniv = gen_cle0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_16 },
- { .fni4 = gen_cle0_i32,
- .fniv = gen_cle0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_32 },
- { .fni8 = gen_cle0_i64,
- .fniv = gen_cle0_vec,
- .opt_opc = vecop_list_cmp,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .vece = MO_64 },
-};
-
-static void gen_cge0_i32(TCGv_i32 d, TCGv_i32 a)
-{
- tcg_gen_setcondi_i32(TCG_COND_GE, d, a, 0);
- tcg_gen_neg_i32(d, d);
-}
-
-static void gen_cge0_i64(TCGv_i64 d, TCGv_i64 a)
-{
- tcg_gen_setcondi_i64(TCG_COND_GE, d, a, 0);
- tcg_gen_neg_i64(d, d);
-}
-
-static void gen_cge0_vec(unsigned vece, TCGv_vec d, TCGv_vec a)
-{
- TCGv_vec zero = tcg_const_zeros_vec_matching(d);
- tcg_gen_cmp_vec(TCG_COND_GE, vece, d, a, zero);
- tcg_temp_free_vec(zero);
-}
-
-const GVecGen2 cge0_op[4] = {
- { .fno = gen_helper_gvec_cge0_b,
- .fniv = gen_cge0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_8 },
- { .fno = gen_helper_gvec_cge0_h,
- .fniv = gen_cge0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_16 },
- { .fni4 = gen_cge0_i32,
- .fniv = gen_cge0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_32 },
- { .fni8 = gen_cge0_i64,
- .fniv = gen_cge0_vec,
- .opt_opc = vecop_list_cmp,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .vece = MO_64 },
-};
-
-static void gen_clt0_i32(TCGv_i32 d, TCGv_i32 a)
-{
- tcg_gen_setcondi_i32(TCG_COND_LT, d, a, 0);
- tcg_gen_neg_i32(d, d);
-}
-
-static void gen_clt0_i64(TCGv_i64 d, TCGv_i64 a)
-{
- tcg_gen_setcondi_i64(TCG_COND_LT, d, a, 0);
- tcg_gen_neg_i64(d, d);
-}
-
-static void gen_clt0_vec(unsigned vece, TCGv_vec d, TCGv_vec a)
-{
- TCGv_vec zero = tcg_const_zeros_vec_matching(d);
- tcg_gen_cmp_vec(TCG_COND_LT, vece, d, a, zero);
- tcg_temp_free_vec(zero);
-}
-
-const GVecGen2 clt0_op[4] = {
- { .fno = gen_helper_gvec_clt0_b,
- .fniv = gen_clt0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_8 },
- { .fno = gen_helper_gvec_clt0_h,
- .fniv = gen_clt0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_16 },
- { .fni4 = gen_clt0_i32,
- .fniv = gen_clt0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_32 },
- { .fni8 = gen_clt0_i64,
- .fniv = gen_clt0_vec,
- .opt_opc = vecop_list_cmp,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .vece = MO_64 },
-};
-
-static void gen_cgt0_i32(TCGv_i32 d, TCGv_i32 a)
-{
- tcg_gen_setcondi_i32(TCG_COND_GT, d, a, 0);
- tcg_gen_neg_i32(d, d);
-}
-
-static void gen_cgt0_i64(TCGv_i64 d, TCGv_i64 a)
-{
- tcg_gen_setcondi_i64(TCG_COND_GT, d, a, 0);
- tcg_gen_neg_i64(d, d);
-}
-
-static void gen_cgt0_vec(unsigned vece, TCGv_vec d, TCGv_vec a)
-{
- TCGv_vec zero = tcg_const_zeros_vec_matching(d);
- tcg_gen_cmp_vec(TCG_COND_GT, vece, d, a, zero);
- tcg_temp_free_vec(zero);
-}
-
-const GVecGen2 cgt0_op[4] = {
- { .fno = gen_helper_gvec_cgt0_b,
- .fniv = gen_cgt0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_8 },
- { .fno = gen_helper_gvec_cgt0_h,
- .fniv = gen_cgt0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_16 },
- { .fni4 = gen_cgt0_i32,
- .fniv = gen_cgt0_vec,
- .opt_opc = vecop_list_cmp,
- .vece = MO_32 },
- { .fni8 = gen_cgt0_i64,
- .fniv = gen_cgt0_vec,
- .opt_opc = vecop_list_cmp,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .vece = MO_64 },
-};
+#undef GEN_CMP0
static void gen_ssra8_i64(TCGv_i64 d, TCGv_i64 a, int64_t shift)
{
@@ -6770,24 +6625,19 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
break;
case NEON_2RM_VCEQ0:
- tcg_gen_gvec_2(rd_ofs, rm_ofs, vec_size,
- vec_size, &ceq0_op[size]);
+ gen_gvec_ceq0(size, rd_ofs, rm_ofs, vec_size, vec_size);
break;
case NEON_2RM_VCGT0:
- tcg_gen_gvec_2(rd_ofs, rm_ofs, vec_size,
- vec_size, &cgt0_op[size]);
+ gen_gvec_cgt0(size, rd_ofs, rm_ofs, vec_size, vec_size);
break;
case NEON_2RM_VCLE0:
- tcg_gen_gvec_2(rd_ofs, rm_ofs, vec_size,
- vec_size, &cle0_op[size]);
+ gen_gvec_cle0(size, rd_ofs, rm_ofs, vec_size, vec_size);
break;
case NEON_2RM_VCGE0:
- tcg_gen_gvec_2(rd_ofs, rm_ofs, vec_size,
- vec_size, &cge0_op[size]);
+ gen_gvec_cge0(size, rd_ofs, rm_ofs, vec_size, vec_size);
break;
case NEON_2RM_VCLT0:
- tcg_gen_gvec_2(rd_ofs, rm_ofs, vec_size,
- vec_size, &clt0_op[size]);
+ gen_gvec_clt0(size, rd_ofs, rm_ofs, vec_size, vec_size);
break;
default:
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 07/16] target/arm: Create gen_gvec_{mla,mls}
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (5 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 06/16] target/arm: Create gen_gvec_{ceq,clt,cle,cgt,cge}0 Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:11 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 08/16] target/arm: Swap argument order for VSHL during decode Richard Henderson
` (9 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/translate.h | 7 +-
target/arm/translate-a64.c | 4 +-
target/arm/translate-neon.inc.c | 16 +----
target/arm/translate.c | 117 +++++++++++++++++---------------
4 files changed, 71 insertions(+), 73 deletions(-)
diff --git a/target/arm/translate.h b/target/arm/translate.h
index e35c812cc5..9354ceba35 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -286,8 +286,11 @@ void gen_gvec_cle0(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
void gen_gvec_cge0(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
uint32_t opr_sz, uint32_t max_sz);
-extern const GVecGen3 mla_op[4];
-extern const GVecGen3 mls_op[4];
+void gen_gvec_mla(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_mls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
extern const GVecGen3 cmtst_op[4];
extern const GVecGen3 sshl_op[4];
extern const GVecGen3 ushl_op[4];
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index fef93dc27a..ab9df12e44 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -11226,9 +11226,9 @@ static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
return;
case 0x12: /* MLA, MLS */
if (u) {
- gen_gvec_op3(s, is_q, rd, rn, rm, &mls_op[size]);
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_mls, size);
} else {
- gen_gvec_op3(s, is_q, rd, rn, rm, &mla_op[size]);
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_mla, size);
}
return;
case 0x11:
diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c
index 50b77b6d71..aefeff498a 100644
--- a/target/arm/translate-neon.inc.c
+++ b/target/arm/translate-neon.inc.c
@@ -632,6 +632,8 @@ DO_3SAME_NO_SZ_3(VMAX_U, tcg_gen_gvec_umax)
DO_3SAME_NO_SZ_3(VMIN_S, tcg_gen_gvec_smin)
DO_3SAME_NO_SZ_3(VMIN_U, tcg_gen_gvec_umin)
DO_3SAME_NO_SZ_3(VMUL, tcg_gen_gvec_mul)
+DO_3SAME_NO_SZ_3(VMLA, gen_gvec_mla)
+DO_3SAME_NO_SZ_3(VMLS, gen_gvec_mls)
#define DO_3SAME_CMP(INSN, COND) \
static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
@@ -685,20 +687,6 @@ static bool trans_VMUL_p_3s(DisasContext *s, arg_3same *a)
return do_3same(s, a, gen_VMUL_p_3s);
}
-#define DO_3SAME_GVEC3_NO_SZ_3(INSN, OPARRAY) \
- static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
- uint32_t rn_ofs, uint32_t rm_ofs, \
- uint32_t oprsz, uint32_t maxsz) \
- { \
- tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, \
- oprsz, maxsz, &OPARRAY[vece]); \
- } \
- DO_3SAME_NO_SZ_3(INSN, gen_##INSN##_3s)
-
-
-DO_3SAME_GVEC3_NO_SZ_3(VMLA, mla_op)
-DO_3SAME_GVEC3_NO_SZ_3(VMLS, mls_op)
-
#define DO_3SAME_GVEC3_SHIFT(INSN, OPARRAY) \
static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
uint32_t rn_ofs, uint32_t rm_ofs, \
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 45df3743f6..52b6d32cf1 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -4520,62 +4520,69 @@ static void gen_mls_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
/* Note that while NEON does not support VMLA and VMLS as 64-bit ops,
* these tables are shared with AArch64 which does support them.
*/
+void gen_gvec_mla(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_mul_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen3 ops[4] = {
+ { .fni4 = gen_mla8_i32,
+ .fniv = gen_mla_vec,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fni4 = gen_mla16_i32,
+ .fniv = gen_mla_vec,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_mla32_i32,
+ .fniv = gen_mla_vec,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_mla64_i64,
+ .fniv = gen_mla_vec,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
-static const TCGOpcode vecop_list_mla[] = {
- INDEX_op_mul_vec, INDEX_op_add_vec, 0
-};
-
-static const TCGOpcode vecop_list_mls[] = {
- INDEX_op_mul_vec, INDEX_op_sub_vec, 0
-};
-
-const GVecGen3 mla_op[4] = {
- { .fni4 = gen_mla8_i32,
- .fniv = gen_mla_vec,
- .load_dest = true,
- .opt_opc = vecop_list_mla,
- .vece = MO_8 },
- { .fni4 = gen_mla16_i32,
- .fniv = gen_mla_vec,
- .load_dest = true,
- .opt_opc = vecop_list_mla,
- .vece = MO_16 },
- { .fni4 = gen_mla32_i32,
- .fniv = gen_mla_vec,
- .load_dest = true,
- .opt_opc = vecop_list_mla,
- .vece = MO_32 },
- { .fni8 = gen_mla64_i64,
- .fniv = gen_mla_vec,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .load_dest = true,
- .opt_opc = vecop_list_mla,
- .vece = MO_64 },
-};
-
-const GVecGen3 mls_op[4] = {
- { .fni4 = gen_mls8_i32,
- .fniv = gen_mls_vec,
- .load_dest = true,
- .opt_opc = vecop_list_mls,
- .vece = MO_8 },
- { .fni4 = gen_mls16_i32,
- .fniv = gen_mls_vec,
- .load_dest = true,
- .opt_opc = vecop_list_mls,
- .vece = MO_16 },
- { .fni4 = gen_mls32_i32,
- .fniv = gen_mls_vec,
- .load_dest = true,
- .opt_opc = vecop_list_mls,
- .vece = MO_32 },
- { .fni8 = gen_mls64_i64,
- .fniv = gen_mls_vec,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .load_dest = true,
- .opt_opc = vecop_list_mls,
- .vece = MO_64 },
-};
+void gen_gvec_mls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_mul_vec, INDEX_op_sub_vec, 0
+ };
+ static const GVecGen3 ops[4] = {
+ { .fni4 = gen_mls8_i32,
+ .fniv = gen_mls_vec,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fni4 = gen_mls16_i32,
+ .fniv = gen_mls_vec,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_mls32_i32,
+ .fniv = gen_mls_vec,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_mls64_i64,
+ .fniv = gen_mls_vec,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .load_dest = true,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
/* CMTST : test is "if (X & Y != 0)". */
static void gen_cmtst_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 08/16] target/arm: Swap argument order for VSHL during decode
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (6 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 07/16] target/arm: Create gen_gvec_{mla,mls} Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:14 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 09/16] target/arm: Create gen_gvec_{cmtst,ushl,sshl} Richard Henderson
` (8 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Rather than perform the argument swap during code generation,
perform it during decode. This means it doesn't have to be
special cased later, and we can share code with aarch64 code
generation. Hopefully the decode comment addresses any confusion
that might arise in between.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/neon-dp.decode | 9 +++++++--
target/arm/translate-neon.inc.c | 3 +--
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode
index ec3a92fe75..6b0b6566d6 100644
--- a/target/arm/neon-dp.decode
+++ b/target/arm/neon-dp.decode
@@ -65,8 +65,13 @@ VCGT_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 0 .... @3same
VCGE_S_3s 1111 001 0 0 . .. .... .... 0011 . . . 1 .... @3same
VCGE_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 1 .... @3same
-VSHL_S_3s 1111 001 0 0 . .. .... .... 0100 . . . 0 .... @3same
-VSHL_U_3s 1111 001 1 0 . .. .... .... 0100 . . . 0 .... @3same
+# The shift operations are of the form Vd = Vm << Vn.
+# By reversing the names of the fields here, we can use standard expanders.
+@3same_rev .... ... . . . size:2 .... .... .... . q:1 . . .... \
+ &3same vn=%vm_dp vm=%vn_dp vd=%vd_dp
+
+VSHL_S_3s 1111 001 0 0 . .. .... .... 0100 . . . 0 .... @3same_rev
+VSHL_U_3s 1111 001 1 0 . .. .... .... 0100 . . . 0 .... @3same_rev
VMAX_S_3s 1111 001 0 0 . .. .... .... 0110 . . . 0 .... @3same
VMAX_U_3s 1111 001 1 0 . .. .... .... 0110 . . . 0 .... @3same
diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c
index aefeff498a..416302bcc7 100644
--- a/target/arm/translate-neon.inc.c
+++ b/target/arm/translate-neon.inc.c
@@ -692,8 +692,7 @@ static bool trans_VMUL_p_3s(DisasContext *s, arg_3same *a)
uint32_t rn_ofs, uint32_t rm_ofs, \
uint32_t oprsz, uint32_t maxsz) \
{ \
- /* Note the operation is vshl vd,vm,vn */ \
- tcg_gen_gvec_3(rd_ofs, rm_ofs, rn_ofs, \
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, \
oprsz, maxsz, &OPARRAY[vece]); \
} \
DO_3SAME(INSN, gen_##INSN##_3s)
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 09/16] target/arm: Create gen_gvec_{cmtst,ushl,sshl}
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (7 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 08/16] target/arm: Swap argument order for VSHL during decode Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:16 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 10/16] target/arm: Create gen_gvec_{uqadd, sqadd, uqsub, sqsub} Richard Henderson
` (7 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/translate.h | 10 ++-
target/arm/translate-a64.c | 18 ++--
target/arm/translate-neon.inc.c | 23 +----
target/arm/translate.c | 146 +++++++++++++++++---------------
4 files changed, 95 insertions(+), 102 deletions(-)
diff --git a/target/arm/translate.h b/target/arm/translate.h
index 9354ceba35..a02a54cabf 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -291,9 +291,13 @@ void gen_gvec_mla(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
void gen_gvec_mls(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
-extern const GVecGen3 cmtst_op[4];
-extern const GVecGen3 sshl_op[4];
-extern const GVecGen3 ushl_op[4];
+void gen_gvec_cmtst(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_ushl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
extern const GVecGen4 uqadd_op[4];
extern const GVecGen4 sqadd_op[4];
extern const GVecGen4 uqsub_op[4];
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index ab9df12e44..3956c19ed8 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -577,15 +577,6 @@ static void gen_gvec_fn4(DisasContext *s, bool is_q, int rd, int rn, int rm,
is_q ? 16 : 8, vec_full_reg_size(s));
}
-/* Expand a 3-operand AdvSIMD vector operation using an op descriptor. */
-static void gen_gvec_op3(DisasContext *s, bool is_q, int rd,
- int rn, int rm, const GVecGen3 *gvec_op)
-{
- tcg_gen_gvec_3(vec_full_reg_offset(s, rd), vec_full_reg_offset(s, rn),
- vec_full_reg_offset(s, rm), is_q ? 16 : 8,
- vec_full_reg_size(s), gvec_op);
-}
-
/* Expand a 3-operand operation using an out-of-line helper. */
static void gen_gvec_op3_ool(DisasContext *s, bool is_q, int rd,
int rn, int rm, int data, gen_helper_gvec_3 *fn)
@@ -11193,8 +11184,11 @@ static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
(u ? uqsub_op : sqsub_op) + size);
return;
case 0x08: /* SSHL, USHL */
- gen_gvec_op3(s, is_q, rd, rn, rm,
- u ? &ushl_op[size] : &sshl_op[size]);
+ if (u) {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_ushl, size);
+ } else {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sshl, size);
+ }
return;
case 0x0c: /* SMAX, UMAX */
if (u) {
@@ -11233,7 +11227,7 @@ static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
return;
case 0x11:
if (!u) { /* CMTST */
- gen_gvec_op3(s, is_q, rd, rn, rm, &cmtst_op[size]);
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_cmtst, size);
return;
}
/* else CMEQ */
diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c
index 416302bcc7..e16475c212 100644
--- a/target/arm/translate-neon.inc.c
+++ b/target/arm/translate-neon.inc.c
@@ -603,6 +603,8 @@ DO_3SAME(VBIC, tcg_gen_gvec_andc)
DO_3SAME(VORR, tcg_gen_gvec_or)
DO_3SAME(VORN, tcg_gen_gvec_orc)
DO_3SAME(VEOR, tcg_gen_gvec_xor)
+DO_3SAME(VSHL_S, gen_gvec_sshl)
+DO_3SAME(VSHL_U, gen_gvec_ushl)
/* These insns are all gvec_bitsel but with the inputs in various orders. */
#define DO_3SAME_BITSEL(INSN, O1, O2, O3) \
@@ -634,6 +636,7 @@ DO_3SAME_NO_SZ_3(VMIN_U, tcg_gen_gvec_umin)
DO_3SAME_NO_SZ_3(VMUL, tcg_gen_gvec_mul)
DO_3SAME_NO_SZ_3(VMLA, gen_gvec_mla)
DO_3SAME_NO_SZ_3(VMLS, gen_gvec_mls)
+DO_3SAME_NO_SZ_3(VTST, gen_gvec_cmtst)
#define DO_3SAME_CMP(INSN, COND) \
static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
@@ -650,13 +653,6 @@ DO_3SAME_CMP(VCGE_S, TCG_COND_GE)
DO_3SAME_CMP(VCGE_U, TCG_COND_GEU)
DO_3SAME_CMP(VCEQ, TCG_COND_EQ)
-static void gen_VTST_3s(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
- uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz)
-{
- tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, &cmtst_op[vece]);
-}
-DO_3SAME_NO_SZ_3(VTST, gen_VTST_3s)
-
#define DO_3SAME_GVEC4(INSN, OPARRAY) \
static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
uint32_t rn_ofs, uint32_t rm_ofs, \
@@ -686,16 +682,3 @@ static bool trans_VMUL_p_3s(DisasContext *s, arg_3same *a)
}
return do_3same(s, a, gen_VMUL_p_3s);
}
-
-#define DO_3SAME_GVEC3_SHIFT(INSN, OPARRAY) \
- static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
- uint32_t rn_ofs, uint32_t rm_ofs, \
- uint32_t oprsz, uint32_t maxsz) \
- { \
- tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, \
- oprsz, maxsz, &OPARRAY[vece]); \
- } \
- DO_3SAME(INSN, gen_##INSN##_3s)
-
-DO_3SAME_GVEC3_SHIFT(VSHL_S, sshl_op)
-DO_3SAME_GVEC3_SHIFT(VSHL_U, ushl_op)
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 52b6d32cf1..e366281274 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -4606,27 +4606,31 @@ static void gen_cmtst_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
tcg_gen_cmp_vec(TCG_COND_NE, vece, d, d, a);
}
-static const TCGOpcode vecop_list_cmtst[] = { INDEX_op_cmp_vec, 0 };
-
-const GVecGen3 cmtst_op[4] = {
- { .fni4 = gen_helper_neon_tst_u8,
- .fniv = gen_cmtst_vec,
- .opt_opc = vecop_list_cmtst,
- .vece = MO_8 },
- { .fni4 = gen_helper_neon_tst_u16,
- .fniv = gen_cmtst_vec,
- .opt_opc = vecop_list_cmtst,
- .vece = MO_16 },
- { .fni4 = gen_cmtst_i32,
- .fniv = gen_cmtst_vec,
- .opt_opc = vecop_list_cmtst,
- .vece = MO_32 },
- { .fni8 = gen_cmtst_i64,
- .fniv = gen_cmtst_vec,
- .prefer_i64 = TCG_TARGET_REG_BITS == 64,
- .opt_opc = vecop_list_cmtst,
- .vece = MO_64 },
-};
+void gen_gvec_cmtst(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = { INDEX_op_cmp_vec, 0 };
+ static const GVecGen3 ops[4] = {
+ { .fni4 = gen_helper_neon_tst_u8,
+ .fniv = gen_cmtst_vec,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fni4 = gen_helper_neon_tst_u16,
+ .fniv = gen_cmtst_vec,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_cmtst_i32,
+ .fniv = gen_cmtst_vec,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_cmtst_i64,
+ .fniv = gen_cmtst_vec,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
void gen_ushl_i32(TCGv_i32 dst, TCGv_i32 src, TCGv_i32 shift)
{
@@ -4744,29 +4748,33 @@ static void gen_ushl_vec(unsigned vece, TCGv_vec dst,
tcg_temp_free_vec(rsh);
}
-static const TCGOpcode ushl_list[] = {
- INDEX_op_neg_vec, INDEX_op_shlv_vec,
- INDEX_op_shrv_vec, INDEX_op_cmp_vec, 0
-};
-
-const GVecGen3 ushl_op[4] = {
- { .fniv = gen_ushl_vec,
- .fno = gen_helper_gvec_ushl_b,
- .opt_opc = ushl_list,
- .vece = MO_8 },
- { .fniv = gen_ushl_vec,
- .fno = gen_helper_gvec_ushl_h,
- .opt_opc = ushl_list,
- .vece = MO_16 },
- { .fni4 = gen_ushl_i32,
- .fniv = gen_ushl_vec,
- .opt_opc = ushl_list,
- .vece = MO_32 },
- { .fni8 = gen_ushl_i64,
- .fniv = gen_ushl_vec,
- .opt_opc = ushl_list,
- .vece = MO_64 },
-};
+void gen_gvec_ushl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_neg_vec, INDEX_op_shlv_vec,
+ INDEX_op_shrv_vec, INDEX_op_cmp_vec, 0
+ };
+ static const GVecGen3 ops[4] = {
+ { .fniv = gen_ushl_vec,
+ .fno = gen_helper_gvec_ushl_b,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fniv = gen_ushl_vec,
+ .fno = gen_helper_gvec_ushl_h,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_ushl_i32,
+ .fniv = gen_ushl_vec,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_ushl_i64,
+ .fniv = gen_ushl_vec,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
void gen_sshl_i32(TCGv_i32 dst, TCGv_i32 src, TCGv_i32 shift)
{
@@ -4878,29 +4886,33 @@ static void gen_sshl_vec(unsigned vece, TCGv_vec dst,
tcg_temp_free_vec(tmp);
}
-static const TCGOpcode sshl_list[] = {
- INDEX_op_neg_vec, INDEX_op_umin_vec, INDEX_op_shlv_vec,
- INDEX_op_sarv_vec, INDEX_op_cmp_vec, INDEX_op_cmpsel_vec, 0
-};
-
-const GVecGen3 sshl_op[4] = {
- { .fniv = gen_sshl_vec,
- .fno = gen_helper_gvec_sshl_b,
- .opt_opc = sshl_list,
- .vece = MO_8 },
- { .fniv = gen_sshl_vec,
- .fno = gen_helper_gvec_sshl_h,
- .opt_opc = sshl_list,
- .vece = MO_16 },
- { .fni4 = gen_sshl_i32,
- .fniv = gen_sshl_vec,
- .opt_opc = sshl_list,
- .vece = MO_32 },
- { .fni8 = gen_sshl_i64,
- .fniv = gen_sshl_vec,
- .opt_opc = sshl_list,
- .vece = MO_64 },
-};
+void gen_gvec_sshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_neg_vec, INDEX_op_umin_vec, INDEX_op_shlv_vec,
+ INDEX_op_sarv_vec, INDEX_op_cmp_vec, INDEX_op_cmpsel_vec, 0
+ };
+ static const GVecGen3 ops[4] = {
+ { .fniv = gen_sshl_vec,
+ .fno = gen_helper_gvec_sshl_b,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fniv = gen_sshl_vec,
+ .fno = gen_helper_gvec_sshl_h,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_sshl_i32,
+ .fniv = gen_sshl_vec,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_sshl_i64,
+ .fniv = gen_sshl_vec,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
static void gen_uqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
TCGv_vec a, TCGv_vec b)
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 10/16] target/arm: Create gen_gvec_{uqadd, sqadd, uqsub, sqsub}
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (8 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 09/16] target/arm: Create gen_gvec_{cmtst,ushl,sshl} Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:18 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 11/16] target/arm: Remove fp_status from helper_{recpe, rsqrte}_u32 Richard Henderson
` (6 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/translate.h | 13 +-
target/arm/translate-a64.c | 22 ++-
target/arm/translate-neon.inc.c | 19 +--
target/arm/translate.c | 228 +++++++++++++++++---------------
4 files changed, 147 insertions(+), 135 deletions(-)
diff --git a/target/arm/translate.h b/target/arm/translate.h
index a02a54cabf..4e1778c5e0 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -298,16 +298,21 @@ void gen_gvec_sshl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
void gen_gvec_ushl(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
-extern const GVecGen4 uqadd_op[4];
-extern const GVecGen4 sqadd_op[4];
-extern const GVecGen4 uqsub_op[4];
-extern const GVecGen4 sqsub_op[4];
void gen_cmtst_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
void gen_ushl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
void gen_sshl_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b);
void gen_ushl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
void gen_sshl_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b);
+void gen_gvec_uqadd_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sqadd_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_uqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
void gen_gvec_ssra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
int64_t shift, uint32_t opr_sz, uint32_t max_sz);
void gen_gvec_usra(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 3956c19ed8..ea5f6ceadc 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -11168,20 +11168,18 @@ static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
switch (opcode) {
case 0x01: /* SQADD, UQADD */
- tcg_gen_gvec_4(vec_full_reg_offset(s, rd),
- offsetof(CPUARMState, vfp.qc),
- vec_full_reg_offset(s, rn),
- vec_full_reg_offset(s, rm),
- is_q ? 16 : 8, vec_full_reg_size(s),
- (u ? uqadd_op : sqadd_op) + size);
+ if (u) {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uqadd_qc, size);
+ } else {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sqadd_qc, size);
+ }
return;
case 0x05: /* SQSUB, UQSUB */
- tcg_gen_gvec_4(vec_full_reg_offset(s, rd),
- offsetof(CPUARMState, vfp.qc),
- vec_full_reg_offset(s, rn),
- vec_full_reg_offset(s, rm),
- is_q ? 16 : 8, vec_full_reg_size(s),
- (u ? uqsub_op : sqsub_op) + size);
+ if (u) {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uqsub_qc, size);
+ } else {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sqsub_qc, size);
+ }
return;
case 0x08: /* SSHL, USHL */
if (u) {
diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c
index e16475c212..099491b16f 100644
--- a/target/arm/translate-neon.inc.c
+++ b/target/arm/translate-neon.inc.c
@@ -605,6 +605,10 @@ DO_3SAME(VORN, tcg_gen_gvec_orc)
DO_3SAME(VEOR, tcg_gen_gvec_xor)
DO_3SAME(VSHL_S, gen_gvec_sshl)
DO_3SAME(VSHL_U, gen_gvec_ushl)
+DO_3SAME(VQADD_S, gen_gvec_sqadd_qc)
+DO_3SAME(VQADD_U, gen_gvec_uqadd_qc)
+DO_3SAME(VQSUB_S, gen_gvec_sqsub_qc)
+DO_3SAME(VQSUB_U, gen_gvec_uqsub_qc)
/* These insns are all gvec_bitsel but with the inputs in various orders. */
#define DO_3SAME_BITSEL(INSN, O1, O2, O3) \
@@ -653,21 +657,6 @@ DO_3SAME_CMP(VCGE_S, TCG_COND_GE)
DO_3SAME_CMP(VCGE_U, TCG_COND_GEU)
DO_3SAME_CMP(VCEQ, TCG_COND_EQ)
-#define DO_3SAME_GVEC4(INSN, OPARRAY) \
- static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \
- uint32_t rn_ofs, uint32_t rm_ofs, \
- uint32_t oprsz, uint32_t maxsz) \
- { \
- tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc), \
- rn_ofs, rm_ofs, oprsz, maxsz, &OPARRAY[vece]); \
- } \
- DO_3SAME(INSN, gen_##INSN##_3s)
-
-DO_3SAME_GVEC4(VQADD_S, sqadd_op)
-DO_3SAME_GVEC4(VQADD_U, uqadd_op)
-DO_3SAME_GVEC4(VQSUB_S, sqsub_op)
-DO_3SAME_GVEC4(VQSUB_U, uqsub_op)
-
static void gen_VMUL_p_3s(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz)
{
diff --git a/target/arm/translate.c b/target/arm/translate.c
index e366281274..72c3aab544 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -4925,32 +4925,37 @@ static void gen_uqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
tcg_temp_free_vec(x);
}
-static const TCGOpcode vecop_list_uqadd[] = {
- INDEX_op_usadd_vec, INDEX_op_cmp_vec, INDEX_op_add_vec, 0
-};
-
-const GVecGen4 uqadd_op[4] = {
- { .fniv = gen_uqadd_vec,
- .fno = gen_helper_gvec_uqadd_b,
- .write_aofs = true,
- .opt_opc = vecop_list_uqadd,
- .vece = MO_8 },
- { .fniv = gen_uqadd_vec,
- .fno = gen_helper_gvec_uqadd_h,
- .write_aofs = true,
- .opt_opc = vecop_list_uqadd,
- .vece = MO_16 },
- { .fniv = gen_uqadd_vec,
- .fno = gen_helper_gvec_uqadd_s,
- .write_aofs = true,
- .opt_opc = vecop_list_uqadd,
- .vece = MO_32 },
- { .fniv = gen_uqadd_vec,
- .fno = gen_helper_gvec_uqadd_d,
- .write_aofs = true,
- .opt_opc = vecop_list_uqadd,
- .vece = MO_64 },
-};
+void gen_gvec_uqadd_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_usadd_vec, INDEX_op_cmp_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen4 ops[4] = {
+ { .fniv = gen_uqadd_vec,
+ .fno = gen_helper_gvec_uqadd_b,
+ .write_aofs = true,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fniv = gen_uqadd_vec,
+ .fno = gen_helper_gvec_uqadd_h,
+ .write_aofs = true,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fniv = gen_uqadd_vec,
+ .fno = gen_helper_gvec_uqadd_s,
+ .write_aofs = true,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fniv = gen_uqadd_vec,
+ .fno = gen_helper_gvec_uqadd_d,
+ .write_aofs = true,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc),
+ rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
static void gen_sqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
TCGv_vec a, TCGv_vec b)
@@ -4963,32 +4968,37 @@ static void gen_sqadd_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
tcg_temp_free_vec(x);
}
-static const TCGOpcode vecop_list_sqadd[] = {
- INDEX_op_ssadd_vec, INDEX_op_cmp_vec, INDEX_op_add_vec, 0
-};
-
-const GVecGen4 sqadd_op[4] = {
- { .fniv = gen_sqadd_vec,
- .fno = gen_helper_gvec_sqadd_b,
- .opt_opc = vecop_list_sqadd,
- .write_aofs = true,
- .vece = MO_8 },
- { .fniv = gen_sqadd_vec,
- .fno = gen_helper_gvec_sqadd_h,
- .opt_opc = vecop_list_sqadd,
- .write_aofs = true,
- .vece = MO_16 },
- { .fniv = gen_sqadd_vec,
- .fno = gen_helper_gvec_sqadd_s,
- .opt_opc = vecop_list_sqadd,
- .write_aofs = true,
- .vece = MO_32 },
- { .fniv = gen_sqadd_vec,
- .fno = gen_helper_gvec_sqadd_d,
- .opt_opc = vecop_list_sqadd,
- .write_aofs = true,
- .vece = MO_64 },
-};
+void gen_gvec_sqadd_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_ssadd_vec, INDEX_op_cmp_vec, INDEX_op_add_vec, 0
+ };
+ static const GVecGen4 ops[4] = {
+ { .fniv = gen_sqadd_vec,
+ .fno = gen_helper_gvec_sqadd_b,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_8 },
+ { .fniv = gen_sqadd_vec,
+ .fno = gen_helper_gvec_sqadd_h,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_16 },
+ { .fniv = gen_sqadd_vec,
+ .fno = gen_helper_gvec_sqadd_s,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_32 },
+ { .fniv = gen_sqadd_vec,
+ .fno = gen_helper_gvec_sqadd_d,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc),
+ rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
static void gen_uqsub_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
TCGv_vec a, TCGv_vec b)
@@ -5001,32 +5011,37 @@ static void gen_uqsub_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
tcg_temp_free_vec(x);
}
-static const TCGOpcode vecop_list_uqsub[] = {
- INDEX_op_ussub_vec, INDEX_op_cmp_vec, INDEX_op_sub_vec, 0
-};
-
-const GVecGen4 uqsub_op[4] = {
- { .fniv = gen_uqsub_vec,
- .fno = gen_helper_gvec_uqsub_b,
- .opt_opc = vecop_list_uqsub,
- .write_aofs = true,
- .vece = MO_8 },
- { .fniv = gen_uqsub_vec,
- .fno = gen_helper_gvec_uqsub_h,
- .opt_opc = vecop_list_uqsub,
- .write_aofs = true,
- .vece = MO_16 },
- { .fniv = gen_uqsub_vec,
- .fno = gen_helper_gvec_uqsub_s,
- .opt_opc = vecop_list_uqsub,
- .write_aofs = true,
- .vece = MO_32 },
- { .fniv = gen_uqsub_vec,
- .fno = gen_helper_gvec_uqsub_d,
- .opt_opc = vecop_list_uqsub,
- .write_aofs = true,
- .vece = MO_64 },
-};
+void gen_gvec_uqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_ussub_vec, INDEX_op_cmp_vec, INDEX_op_sub_vec, 0
+ };
+ static const GVecGen4 ops[4] = {
+ { .fniv = gen_uqsub_vec,
+ .fno = gen_helper_gvec_uqsub_b,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_8 },
+ { .fniv = gen_uqsub_vec,
+ .fno = gen_helper_gvec_uqsub_h,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_16 },
+ { .fniv = gen_uqsub_vec,
+ .fno = gen_helper_gvec_uqsub_s,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_32 },
+ { .fniv = gen_uqsub_vec,
+ .fno = gen_helper_gvec_uqsub_d,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc),
+ rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
static void gen_sqsub_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
TCGv_vec a, TCGv_vec b)
@@ -5039,32 +5054,37 @@ static void gen_sqsub_vec(unsigned vece, TCGv_vec t, TCGv_vec sat,
tcg_temp_free_vec(x);
}
-static const TCGOpcode vecop_list_sqsub[] = {
- INDEX_op_sssub_vec, INDEX_op_cmp_vec, INDEX_op_sub_vec, 0
-};
-
-const GVecGen4 sqsub_op[4] = {
- { .fniv = gen_sqsub_vec,
- .fno = gen_helper_gvec_sqsub_b,
- .opt_opc = vecop_list_sqsub,
- .write_aofs = true,
- .vece = MO_8 },
- { .fniv = gen_sqsub_vec,
- .fno = gen_helper_gvec_sqsub_h,
- .opt_opc = vecop_list_sqsub,
- .write_aofs = true,
- .vece = MO_16 },
- { .fniv = gen_sqsub_vec,
- .fno = gen_helper_gvec_sqsub_s,
- .opt_opc = vecop_list_sqsub,
- .write_aofs = true,
- .vece = MO_32 },
- { .fniv = gen_sqsub_vec,
- .fno = gen_helper_gvec_sqsub_d,
- .opt_opc = vecop_list_sqsub,
- .write_aofs = true,
- .vece = MO_64 },
-};
+void gen_gvec_sqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_sssub_vec, INDEX_op_cmp_vec, INDEX_op_sub_vec, 0
+ };
+ static const GVecGen4 ops[4] = {
+ { .fniv = gen_sqsub_vec,
+ .fno = gen_helper_gvec_sqsub_b,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_8 },
+ { .fniv = gen_sqsub_vec,
+ .fno = gen_helper_gvec_sqsub_h,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_16 },
+ { .fniv = gen_sqsub_vec,
+ .fno = gen_helper_gvec_sqsub_s,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_32 },
+ { .fniv = gen_sqsub_vec,
+ .fno = gen_helper_gvec_sqsub_d,
+ .opt_opc = vecop_list,
+ .write_aofs = true,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_4(rd_ofs, offsetof(CPUARMState, vfp.qc),
+ rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
/* Translate a NEON data processing instruction. Return nonzero if the
instruction is invalid.
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 11/16] target/arm: Remove fp_status from helper_{recpe, rsqrte}_u32
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (9 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 10/16] target/arm: Create gen_gvec_{uqadd, sqadd, uqsub, sqsub} Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:19 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 12/16] target/arm: Create gen_gvec_{qrdmla,qrdmls} Richard Henderson
` (5 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
These operations do not touch fp_status.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/helper.h | 4 ++--
target/arm/translate-a64.c | 5 ++---
target/arm/translate.c | 12 ++----------
target/arm/vfp_helper.c | 4 ++--
4 files changed, 8 insertions(+), 17 deletions(-)
diff --git a/target/arm/helper.h b/target/arm/helper.h
index 33c76192d2..aed3050965 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -211,8 +211,8 @@ DEF_HELPER_FLAGS_2(recpe_f64, TCG_CALL_NO_RWG, f64, f64, ptr)
DEF_HELPER_FLAGS_2(rsqrte_f16, TCG_CALL_NO_RWG, f16, f16, ptr)
DEF_HELPER_FLAGS_2(rsqrte_f32, TCG_CALL_NO_RWG, f32, f32, ptr)
DEF_HELPER_FLAGS_2(rsqrte_f64, TCG_CALL_NO_RWG, f64, f64, ptr)
-DEF_HELPER_2(recpe_u32, i32, i32, ptr)
-DEF_HELPER_FLAGS_2(rsqrte_u32, TCG_CALL_NO_RWG, i32, i32, ptr)
+DEF_HELPER_FLAGS_1(recpe_u32, TCG_CALL_NO_RWG, i32, i32)
+DEF_HELPER_FLAGS_1(rsqrte_u32, TCG_CALL_NO_RWG, i32, i32)
DEF_HELPER_FLAGS_4(neon_tbl, TCG_CALL_NO_RWG, i32, i32, i32, ptr, i32)
DEF_HELPER_3(shl_cc, i32, env, i32, i32)
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index ea5f6ceadc..367fa403ae 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -9699,7 +9699,7 @@ static void handle_2misc_reciprocal(DisasContext *s, int opcode,
switch (opcode) {
case 0x3c: /* URECPE */
- gen_helper_recpe_u32(tcg_res, tcg_op, fpst);
+ gen_helper_recpe_u32(tcg_res, tcg_op);
break;
case 0x3d: /* FRECPE */
gen_helper_recpe_f32(tcg_res, tcg_op, fpst);
@@ -12244,7 +12244,6 @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn)
unallocated_encoding(s);
return;
}
- need_fpstatus = true;
break;
case 0x1e: /* FRINT32Z */
case 0x1f: /* FRINT64Z */
@@ -12412,7 +12411,7 @@ static void disas_simd_two_reg_misc(DisasContext *s, uint32_t insn)
gen_helper_rints_exact(tcg_res, tcg_op, tcg_fpstatus);
break;
case 0x7c: /* URSQRTE */
- gen_helper_rsqrte_u32(tcg_res, tcg_op, tcg_fpstatus);
+ gen_helper_rsqrte_u32(tcg_res, tcg_op);
break;
case 0x1e: /* FRINT32Z */
case 0x5e: /* FRINT32X */
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 72c3aab544..676701143b 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -6873,19 +6873,11 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
break;
}
case NEON_2RM_VRECPE:
- {
- TCGv_ptr fpstatus = get_fpstatus_ptr(1);
- gen_helper_recpe_u32(tmp, tmp, fpstatus);
- tcg_temp_free_ptr(fpstatus);
+ gen_helper_recpe_u32(tmp, tmp);
break;
- }
case NEON_2RM_VRSQRTE:
- {
- TCGv_ptr fpstatus = get_fpstatus_ptr(1);
- gen_helper_rsqrte_u32(tmp, tmp, fpstatus);
- tcg_temp_free_ptr(fpstatus);
+ gen_helper_rsqrte_u32(tmp, tmp);
break;
- }
case NEON_2RM_VRECPE_F:
{
TCGv_ptr fpstatus = get_fpstatus_ptr(1);
diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c
index 930d6e747f..a792661166 100644
--- a/target/arm/vfp_helper.c
+++ b/target/arm/vfp_helper.c
@@ -1023,7 +1023,7 @@ float64 HELPER(rsqrte_f64)(float64 input, void *fpstp)
return make_float64(val);
}
-uint32_t HELPER(recpe_u32)(uint32_t a, void *fpstp)
+uint32_t HELPER(recpe_u32)(uint32_t a)
{
/* float_status *s = fpstp; */
int input, estimate;
@@ -1038,7 +1038,7 @@ uint32_t HELPER(recpe_u32)(uint32_t a, void *fpstp)
return deposit32(0, (32 - 9), 9, estimate);
}
-uint32_t HELPER(rsqrte_u32)(uint32_t a, void *fpstp)
+uint32_t HELPER(rsqrte_u32)(uint32_t a)
{
int estimate;
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 12/16] target/arm: Create gen_gvec_{qrdmla,qrdmls}
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (10 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 11/16] target/arm: Remove fp_status from helper_{recpe, rsqrte}_u32 Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:20 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 13/16] target/arm: Pass pointer to qc to qrdmla/qrdmls Richard Henderson
` (4 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Provide a functional interface for the vector expansion.
This fits better with the existing set of helpers that
we provide for other operations.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/translate.h | 5 ++++
target/arm/translate-a64.c | 34 ++----------------------
target/arm/translate.c | 54 +++++++++++++++++++-------------------
3 files changed, 34 insertions(+), 59 deletions(-)
diff --git a/target/arm/translate.h b/target/arm/translate.h
index 4e1778c5e0..aea8a9759d 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -332,6 +332,11 @@ void gen_gvec_sri(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
void gen_gvec_sli(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
int64_t shift, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sqrdmlah_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sqrdmlsh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
/*
* Forward to the isar_feature_* tests given a DisasContext pointer.
*/
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 367fa403ae..4577df3cf4 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -587,18 +587,6 @@ static void gen_gvec_op3_ool(DisasContext *s, bool is_q, int rd,
is_q ? 16 : 8, vec_full_reg_size(s), data, fn);
}
-/* Expand a 3-operand + env pointer operation using
- * an out-of-line helper.
- */
-static void gen_gvec_op3_env(DisasContext *s, bool is_q, int rd,
- int rn, int rm, gen_helper_gvec_3_ptr *fn)
-{
- tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd),
- vec_full_reg_offset(s, rn),
- vec_full_reg_offset(s, rm), cpu_env,
- is_q ? 16 : 8, vec_full_reg_size(s), 0, fn);
-}
-
/* Expand a 3-operand + fpstatus pointer + simd data value operation using
* an out-of-line helper.
*/
@@ -11693,29 +11681,11 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn)
switch (opcode) {
case 0x0: /* SQRDMLAH (vector) */
- switch (size) {
- case 1:
- gen_gvec_op3_env(s, is_q, rd, rn, rm, gen_helper_gvec_qrdmlah_s16);
- break;
- case 2:
- gen_gvec_op3_env(s, is_q, rd, rn, rm, gen_helper_gvec_qrdmlah_s32);
- break;
- default:
- g_assert_not_reached();
- }
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sqrdmlah_qc, size);
return;
case 0x1: /* SQRDMLSH (vector) */
- switch (size) {
- case 1:
- gen_gvec_op3_env(s, is_q, rd, rn, rm, gen_helper_gvec_qrdmlsh_s16);
- break;
- case 2:
- gen_gvec_op3_env(s, is_q, rd, rn, rm, gen_helper_gvec_qrdmlsh_s32);
- break;
- default:
- g_assert_not_reached();
- }
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sqrdmlsh_qc, size);
return;
case 0x2: /* SDOT / UDOT */
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 676701143b..8321644f25 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -3629,20 +3629,26 @@ static const uint8_t neon_2rm_sizes[] = {
[NEON_2RM_VCVT_UF] = 0x4,
};
-
-/* Expand v8.1 simd helper. */
-static int do_v81_helper(DisasContext *s, gen_helper_gvec_3_ptr *fn,
- int q, int rd, int rn, int rm)
+void gen_gvec_sqrdmlah_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
{
- if (dc_isar_feature(aa32_rdm, s)) {
- int opr_sz = (1 + q) * 8;
- tcg_gen_gvec_3_ptr(vfp_reg_offset(1, rd),
- vfp_reg_offset(1, rn),
- vfp_reg_offset(1, rm), cpu_env,
- opr_sz, opr_sz, 0, fn);
- return 0;
- }
- return 1;
+ static gen_helper_gvec_3_ptr * const fns[2] = {
+ gen_helper_gvec_qrdmlah_s16, gen_helper_gvec_qrdmlah_s32
+ };
+ tcg_debug_assert(vece >= 1 && vece <= 2);
+ tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, cpu_env,
+ opr_sz, max_sz, 0, fns[vece - 1]);
+}
+
+void gen_gvec_sqrdmlsh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static gen_helper_gvec_3_ptr * const fns[2] = {
+ gen_helper_gvec_qrdmlsh_s16, gen_helper_gvec_qrdmlsh_s32
+ };
+ tcg_debug_assert(vece >= 1 && vece <= 2);
+ tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, cpu_env,
+ opr_sz, max_sz, 0, fns[vece - 1]);
}
#define GEN_CMP0(NAME, COND) \
@@ -5197,13 +5203,10 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
break; /* VPADD */
}
/* VQRDMLAH */
- switch (size) {
- case 1:
- return do_v81_helper(s, gen_helper_gvec_qrdmlah_s16,
- q, rd, rn, rm);
- case 2:
- return do_v81_helper(s, gen_helper_gvec_qrdmlah_s32,
- q, rd, rn, rm);
+ if (dc_isar_feature(aa32_rdm, s) && (size == 1 || size == 2)) {
+ gen_gvec_sqrdmlah_qc(size, rd_ofs, rn_ofs, rm_ofs,
+ vec_size, vec_size);
+ return 0;
}
return 1;
@@ -5216,13 +5219,10 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
break;
}
/* VQRDMLSH */
- switch (size) {
- case 1:
- return do_v81_helper(s, gen_helper_gvec_qrdmlsh_s16,
- q, rd, rn, rm);
- case 2:
- return do_v81_helper(s, gen_helper_gvec_qrdmlsh_s32,
- q, rd, rn, rm);
+ if (dc_isar_feature(aa32_rdm, s) && (size == 1 || size == 2)) {
+ gen_gvec_sqrdmlsh_qc(size, rd_ofs, rn_ofs, rm_ofs,
+ vec_size, vec_size);
+ return 0;
}
return 1;
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 13/16] target/arm: Pass pointer to qc to qrdmla/qrdmls
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (11 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 12/16] target/arm: Create gen_gvec_{qrdmla,qrdmls} Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:28 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 14/16] target/arm: Clear tail in gvec_fmul_idx_*, gvec_fmla_idx_* Richard Henderson
` (3 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Pass a pointer directly to env->vfp.qc[0], rather than env.
This will allow SVE2, which does not modify QC, to pass a
pointer to dummy storage.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/translate.c | 18 ++++++++---
target/arm/vec_helper.c | 70 +++++++++++++++++++++++------------------
2 files changed, 54 insertions(+), 34 deletions(-)
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 8321644f25..0e2cf6028a 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -3629,6 +3629,18 @@ static const uint8_t neon_2rm_sizes[] = {
[NEON_2RM_VCVT_UF] = 0x4,
};
+static void gen_gvec_fn3_qc(uint32_t rd_ofs, uint32_t rn_ofs, uint32_t rm_ofs,
+ uint32_t opr_sz, uint32_t max_sz,
+ gen_helper_gvec_3_ptr *fn)
+{
+ TCGv_ptr qc_ptr = tcg_temp_new_ptr();
+
+ tcg_gen_addi_ptr(qc_ptr, cpu_env, offsetof(CPUARMState, vfp.qc));
+ tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, qc_ptr,
+ opr_sz, max_sz, 0, fn);
+ tcg_temp_free_ptr(qc_ptr);
+}
+
void gen_gvec_sqrdmlah_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
{
@@ -3636,8 +3648,7 @@ void gen_gvec_sqrdmlah_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
gen_helper_gvec_qrdmlah_s16, gen_helper_gvec_qrdmlah_s32
};
tcg_debug_assert(vece >= 1 && vece <= 2);
- tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, cpu_env,
- opr_sz, max_sz, 0, fns[vece - 1]);
+ gen_gvec_fn3_qc(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, fns[vece - 1]);
}
void gen_gvec_sqrdmlsh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
@@ -3647,8 +3658,7 @@ void gen_gvec_sqrdmlsh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
gen_helper_gvec_qrdmlsh_s16, gen_helper_gvec_qrdmlsh_s32
};
tcg_debug_assert(vece >= 1 && vece <= 2);
- tcg_gen_gvec_3_ptr(rd_ofs, rn_ofs, rm_ofs, cpu_env,
- opr_sz, max_sz, 0, fns[vece - 1]);
+ gen_gvec_fn3_qc(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, fns[vece - 1]);
}
#define GEN_CMP0(NAME, COND) \
diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index 096fea67ef..6aa2ca0827 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -36,8 +36,6 @@
#define H4(x) (x)
#endif
-#define SET_QC() env->vfp.qc[0] = 1
-
static void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz)
{
uint64_t *d = vd + opr_sz;
@@ -49,8 +47,8 @@ static void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz)
}
/* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */
-static uint16_t inl_qrdmlah_s16(CPUARMState *env, int16_t src1,
- int16_t src2, int16_t src3)
+static int16_t inl_qrdmlah_s16(int16_t src1, int16_t src2,
+ int16_t src3, uint32_t *sat)
{
/* Simplify:
* = ((a3 << 16) + ((e1 * e2) << 1) + (1 << 15)) >> 16
@@ -60,7 +58,7 @@ static uint16_t inl_qrdmlah_s16(CPUARMState *env, int16_t src1,
ret = ((int32_t)src3 << 15) + ret + (1 << 14);
ret >>= 15;
if (ret != (int16_t)ret) {
- SET_QC();
+ *sat = 1;
ret = (ret < 0 ? -0x8000 : 0x7fff);
}
return ret;
@@ -69,30 +67,30 @@ static uint16_t inl_qrdmlah_s16(CPUARMState *env, int16_t src1,
uint32_t HELPER(neon_qrdmlah_s16)(CPUARMState *env, uint32_t src1,
uint32_t src2, uint32_t src3)
{
- uint16_t e1 = inl_qrdmlah_s16(env, src1, src2, src3);
- uint16_t e2 = inl_qrdmlah_s16(env, src1 >> 16, src2 >> 16, src3 >> 16);
+ uint32_t *sat = &env->vfp.qc[0];
+ uint16_t e1 = inl_qrdmlah_s16(src1, src2, src3, sat);
+ uint16_t e2 = inl_qrdmlah_s16(src1 >> 16, src2 >> 16, src3 >> 16, sat);
return deposit32(e1, 16, 16, e2);
}
void HELPER(gvec_qrdmlah_s16)(void *vd, void *vn, void *vm,
- void *ve, uint32_t desc)
+ void *vq, uint32_t desc)
{
uintptr_t opr_sz = simd_oprsz(desc);
int16_t *d = vd;
int16_t *n = vn;
int16_t *m = vm;
- CPUARMState *env = ve;
uintptr_t i;
for (i = 0; i < opr_sz / 2; ++i) {
- d[i] = inl_qrdmlah_s16(env, n[i], m[i], d[i]);
+ d[i] = inl_qrdmlah_s16(n[i], m[i], d[i], vq);
}
clear_tail(d, opr_sz, simd_maxsz(desc));
}
/* Signed saturating rounding doubling multiply-subtract high half, 16-bit */
-static uint16_t inl_qrdmlsh_s16(CPUARMState *env, int16_t src1,
- int16_t src2, int16_t src3)
+static int16_t inl_qrdmlsh_s16(int16_t src1, int16_t src2,
+ int16_t src3, uint32_t *sat)
{
/* Similarly, using subtraction:
* = ((a3 << 16) - ((e1 * e2) << 1) + (1 << 15)) >> 16
@@ -102,7 +100,7 @@ static uint16_t inl_qrdmlsh_s16(CPUARMState *env, int16_t src1,
ret = ((int32_t)src3 << 15) - ret + (1 << 14);
ret >>= 15;
if (ret != (int16_t)ret) {
- SET_QC();
+ *sat = 1;
ret = (ret < 0 ? -0x8000 : 0x7fff);
}
return ret;
@@ -111,85 +109,97 @@ static uint16_t inl_qrdmlsh_s16(CPUARMState *env, int16_t src1,
uint32_t HELPER(neon_qrdmlsh_s16)(CPUARMState *env, uint32_t src1,
uint32_t src2, uint32_t src3)
{
- uint16_t e1 = inl_qrdmlsh_s16(env, src1, src2, src3);
- uint16_t e2 = inl_qrdmlsh_s16(env, src1 >> 16, src2 >> 16, src3 >> 16);
+ uint32_t *sat = &env->vfp.qc[0];
+ uint16_t e1 = inl_qrdmlsh_s16(src1, src2, src3, sat);
+ uint16_t e2 = inl_qrdmlsh_s16(src1 >> 16, src2 >> 16, src3 >> 16, sat);
return deposit32(e1, 16, 16, e2);
}
void HELPER(gvec_qrdmlsh_s16)(void *vd, void *vn, void *vm,
- void *ve, uint32_t desc)
+ void *vq, uint32_t desc)
{
uintptr_t opr_sz = simd_oprsz(desc);
int16_t *d = vd;
int16_t *n = vn;
int16_t *m = vm;
- CPUARMState *env = ve;
uintptr_t i;
for (i = 0; i < opr_sz / 2; ++i) {
- d[i] = inl_qrdmlsh_s16(env, n[i], m[i], d[i]);
+ d[i] = inl_qrdmlsh_s16(n[i], m[i], d[i], vq);
}
clear_tail(d, opr_sz, simd_maxsz(desc));
}
/* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */
-uint32_t HELPER(neon_qrdmlah_s32)(CPUARMState *env, int32_t src1,
- int32_t src2, int32_t src3)
+static int32_t inl_qrdmlah_s32(int32_t src1, int32_t src2,
+ int32_t src3, uint32_t *sat)
{
/* Simplify similarly to int_qrdmlah_s16 above. */
int64_t ret = (int64_t)src1 * src2;
ret = ((int64_t)src3 << 31) + ret + (1 << 30);
ret >>= 31;
if (ret != (int32_t)ret) {
- SET_QC();
+ *sat = 1;
ret = (ret < 0 ? INT32_MIN : INT32_MAX);
}
return ret;
}
+uint32_t HELPER(neon_qrdmlah_s32)(CPUARMState *env, int32_t src1,
+ int32_t src2, int32_t src3)
+{
+ uint32_t *sat = &env->vfp.qc[0];
+ return inl_qrdmlah_s32(src1, src2, src3, sat);
+}
+
void HELPER(gvec_qrdmlah_s32)(void *vd, void *vn, void *vm,
- void *ve, uint32_t desc)
+ void *vq, uint32_t desc)
{
uintptr_t opr_sz = simd_oprsz(desc);
int32_t *d = vd;
int32_t *n = vn;
int32_t *m = vm;
- CPUARMState *env = ve;
uintptr_t i;
for (i = 0; i < opr_sz / 4; ++i) {
- d[i] = helper_neon_qrdmlah_s32(env, n[i], m[i], d[i]);
+ d[i] = inl_qrdmlah_s32(n[i], m[i], d[i], vq);
}
clear_tail(d, opr_sz, simd_maxsz(desc));
}
/* Signed saturating rounding doubling multiply-subtract high half, 32-bit */
-uint32_t HELPER(neon_qrdmlsh_s32)(CPUARMState *env, int32_t src1,
- int32_t src2, int32_t src3)
+static int32_t inl_qrdmlsh_s32(int32_t src1, int32_t src2,
+ int32_t src3, uint32_t *sat)
{
/* Simplify similarly to int_qrdmlsh_s16 above. */
int64_t ret = (int64_t)src1 * src2;
ret = ((int64_t)src3 << 31) - ret + (1 << 30);
ret >>= 31;
if (ret != (int32_t)ret) {
- SET_QC();
+ *sat = 1;
ret = (ret < 0 ? INT32_MIN : INT32_MAX);
}
return ret;
}
+uint32_t HELPER(neon_qrdmlsh_s32)(CPUARMState *env, int32_t src1,
+ int32_t src2, int32_t src3)
+{
+ uint32_t *sat = &env->vfp.qc[0];
+ return inl_qrdmlsh_s32(src1, src2, src3, sat);
+}
+
void HELPER(gvec_qrdmlsh_s32)(void *vd, void *vn, void *vm,
- void *ve, uint32_t desc)
+ void *vq, uint32_t desc)
{
uintptr_t opr_sz = simd_oprsz(desc);
int32_t *d = vd;
int32_t *n = vn;
int32_t *m = vm;
- CPUARMState *env = ve;
uintptr_t i;
for (i = 0; i < opr_sz / 4; ++i) {
- d[i] = helper_neon_qrdmlsh_s32(env, n[i], m[i], d[i]);
+ d[i] = inl_qrdmlsh_s32(n[i], m[i], d[i], vq);
}
clear_tail(d, opr_sz, simd_maxsz(desc));
}
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 14/16] target/arm: Clear tail in gvec_fmul_idx_*, gvec_fmla_idx_*
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (12 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 13/16] target/arm: Pass pointer to qc to qrdmla/qrdmls Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:29 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 15/16] target/arm: Vectorize SABD/UABD Richard Henderson
` (2 subsequent siblings)
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Must clear the tail for AdvSIMD when SVE is enabled.
Fixes: ca40a6e6e39
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/vec_helper.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index 6aa2ca0827..a483841add 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -747,6 +747,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, void *stat, uint32_t desc) \
d[i + j] = TYPE##_mul(n[i + j], mm, stat); \
} \
} \
+ clear_tail(d, oprsz, simd_maxsz(desc)); \
}
DO_MUL_IDX(gvec_fmul_idx_h, float16, H2)
@@ -771,6 +772,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, \
mm, a[i + j], 0, stat); \
} \
} \
+ clear_tail(d, oprsz, simd_maxsz(desc)); \
}
DO_FMLA_IDX(gvec_fmla_idx_h, float16, H2)
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 15/16] target/arm: Vectorize SABD/UABD
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (13 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 14/16] target/arm: Clear tail in gvec_fmul_idx_*, gvec_fmla_idx_* Richard Henderson
@ 2020-05-08 15:21 ` Richard Henderson
2020-05-12 14:40 ` Peter Maydell
2020-05-08 15:22 ` [PATCH v3 16/16] target/arm: Vectorize SABA/UABA Richard Henderson
2020-05-12 12:55 ` [PATCH v3 00/16] target/arm: partial vector cleanup Peter Maydell
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:21 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Include 64-bit element size in preparation for SVE2.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/helper.h | 10 +++
target/arm/translate.h | 5 ++
target/arm/translate-a64.c | 8 ++-
target/arm/translate.c | 133 ++++++++++++++++++++++++++++++++++++-
target/arm/vec_helper.c | 24 +++++++
5 files changed, 176 insertions(+), 4 deletions(-)
diff --git a/target/arm/helper.h b/target/arm/helper.h
index aed3050965..4678d3a6f4 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -731,6 +731,16 @@ DEF_HELPER_FLAGS_3(gvec_sli_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
DEF_HELPER_FLAGS_3(gvec_sli_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
DEF_HELPER_FLAGS_3(gvec_sli_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_sabd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_sabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_sabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_sabd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_4(gvec_uabd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_uabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_uabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_uabd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
#ifdef TARGET_AARCH64
#include "helper-a64.h"
#include "helper-sve.h"
diff --git a/target/arm/translate.h b/target/arm/translate.h
index aea8a9759d..bbfe3d1393 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -337,6 +337,11 @@ void gen_gvec_sqrdmlah_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
void gen_gvec_sqrdmlsh_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_sabd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_uabd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
/*
* Forward to the isar_feature_* tests given a DisasContext pointer.
*/
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 4577df3cf4..54b06553a6 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -11190,6 +11190,13 @@ static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_smin, size);
}
return;
+ case 0xe: /* SABD, UABD */
+ if (u) {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uabd, size);
+ } else {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sabd, size);
+ }
+ return;
case 0x10: /* ADD, SUB */
if (u) {
gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_sub, size);
@@ -11322,7 +11329,6 @@ static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
genenvfn = fns[size][u];
break;
}
- case 0xe: /* SABD, UABD */
case 0xf: /* SABA, UABA */
{
static NeonGenTwoOpFn * const fns[3][2] = {
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 0e2cf6028a..0ed6f49f42 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -5102,6 +5102,126 @@ void gen_gvec_sqsub_qc(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
}
+static void gen_sabd_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+ TCGv_i32 t = tcg_temp_new_i32();
+
+ tcg_gen_sub_i32(t, a, b);
+ tcg_gen_sub_i32(d, b, a);
+ tcg_gen_movcond_i32(TCG_COND_LT, d, a, b, d, t);
+ tcg_temp_free_i32(t);
+}
+
+static void gen_sabd_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ tcg_gen_sub_i64(t, a, b);
+ tcg_gen_sub_i64(d, b, a);
+ tcg_gen_movcond_i64(TCG_COND_LT, d, a, b, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_sabd_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
+{
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+ tcg_gen_smin_vec(vece, t, a, b);
+ tcg_gen_smax_vec(vece, d, a, b);
+ tcg_gen_sub_vec(vece, d, d, t);
+ tcg_temp_free_vec(t);
+}
+
+void gen_gvec_sabd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_sub_vec, INDEX_op_smin_vec, INDEX_op_smax_vec, 0
+ };
+ static const GVecGen3 ops[4] = {
+ { .fniv = gen_sabd_vec,
+ .fno = gen_helper_gvec_sabd_b,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fniv = gen_sabd_vec,
+ .fno = gen_helper_gvec_sabd_h,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_sabd_i32,
+ .fniv = gen_sabd_vec,
+ .fno = gen_helper_gvec_sabd_s,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_sabd_i64,
+ .fniv = gen_sabd_vec,
+ .fno = gen_helper_gvec_sabd_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
+
+static void gen_uabd_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+ TCGv_i32 t = tcg_temp_new_i32();
+
+ tcg_gen_sub_i32(t, a, b);
+ tcg_gen_sub_i32(d, b, a);
+ tcg_gen_movcond_i32(TCG_COND_LTU, d, a, b, d, t);
+ tcg_temp_free_i32(t);
+}
+
+static void gen_uabd_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+
+ tcg_gen_sub_i64(t, a, b);
+ tcg_gen_sub_i64(d, b, a);
+ tcg_gen_movcond_i64(TCG_COND_LTU, d, a, b, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_uabd_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
+{
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+ tcg_gen_umin_vec(vece, t, a, b);
+ tcg_gen_umax_vec(vece, d, a, b);
+ tcg_gen_sub_vec(vece, d, d, t);
+ tcg_temp_free_vec(t);
+}
+
+void gen_gvec_uabd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_sub_vec, INDEX_op_umin_vec, INDEX_op_umax_vec, 0
+ };
+ static const GVecGen3 ops[4] = {
+ { .fniv = gen_uabd_vec,
+ .fno = gen_helper_gvec_uabd_b,
+ .opt_opc = vecop_list,
+ .vece = MO_8 },
+ { .fniv = gen_uabd_vec,
+ .fno = gen_helper_gvec_uabd_h,
+ .opt_opc = vecop_list,
+ .vece = MO_16 },
+ { .fni4 = gen_uabd_i32,
+ .fniv = gen_uabd_vec,
+ .fno = gen_helper_gvec_uabd_s,
+ .opt_opc = vecop_list,
+ .vece = MO_32 },
+ { .fni8 = gen_uabd_i64,
+ .fniv = gen_uabd_vec,
+ .fno = gen_helper_gvec_uabd_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
+
/* Translate a NEON data processing instruction. Return nonzero if the
instruction is invalid.
We process data in a mixture of 32-bit and 64-bit chunks.
@@ -5236,6 +5356,16 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
}
return 1;
+ case NEON_3R_VABD:
+ if (u) {
+ gen_gvec_uabd(size, rd_ofs, rn_ofs, rm_ofs,
+ vec_size, vec_size);
+ } else {
+ gen_gvec_sabd(size, rd_ofs, rn_ofs, rm_ofs,
+ vec_size, vec_size);
+ }
+ return 0;
+
case NEON_3R_VADD_VSUB:
case NEON_3R_LOGIC:
case NEON_3R_VMAX:
@@ -5380,9 +5510,6 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
case NEON_3R_VQRSHL:
GEN_NEON_INTEGER_OP_ENV(qrshl);
break;
- case NEON_3R_VABD:
- GEN_NEON_INTEGER_OP(abd);
- break;
case NEON_3R_VABA:
GEN_NEON_INTEGER_OP(abd);
tcg_temp_free_i32(tmp2);
diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index a483841add..a4972d02fc 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -1407,3 +1407,27 @@ DO_CMP0(gvec_cgt0_h, int16_t, >)
DO_CMP0(gvec_cge0_h, int16_t, >=)
#undef DO_CMP0
+
+#define DO_ABD(NAME, TYPE) \
+void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \
+{ \
+ intptr_t i, opr_sz = simd_oprsz(desc); \
+ TYPE *d = vd, *n = vn, *m = vm; \
+ \
+ for (i = 0; i < opr_sz / sizeof(TYPE); ++i) { \
+ d[i] = n[i] < m[i] ? m[i] - n[i] : n[i] - m[i]; \
+ } \
+ clear_tail(d, opr_sz, simd_maxsz(desc)); \
+}
+
+DO_ABD(gvec_sabd_b, int8_t)
+DO_ABD(gvec_sabd_h, int16_t)
+DO_ABD(gvec_sabd_s, int32_t)
+DO_ABD(gvec_sabd_d, int64_t)
+
+DO_ABD(gvec_uabd_b, uint8_t)
+DO_ABD(gvec_uabd_h, uint16_t)
+DO_ABD(gvec_uabd_s, uint32_t)
+DO_ABD(gvec_uabd_d, uint64_t)
+
+#undef DO_ABD
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v3 16/16] target/arm: Vectorize SABA/UABA
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (14 preceding siblings ...)
2020-05-08 15:21 ` [PATCH v3 15/16] target/arm: Vectorize SABD/UABD Richard Henderson
@ 2020-05-08 15:22 ` Richard Henderson
2020-05-12 14:41 ` Peter Maydell
2020-05-12 12:55 ` [PATCH v3 00/16] target/arm: partial vector cleanup Peter Maydell
16 siblings, 1 reply; 38+ messages in thread
From: Richard Henderson @ 2020-05-08 15:22 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell
Include 64-bit element size in preparation for SVE2.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/helper.h | 17 +++--
target/arm/translate.h | 5 ++
target/arm/neon_helper.c | 10 ---
target/arm/translate-a64.c | 17 ++---
target/arm/translate.c | 134 +++++++++++++++++++++++++++++++++++--
target/arm/vec_helper.c | 24 +++++++
6 files changed, 174 insertions(+), 33 deletions(-)
diff --git a/target/arm/helper.h b/target/arm/helper.h
index 4678d3a6f4..1857f4ee46 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -284,13 +284,6 @@ DEF_HELPER_2(neon_pmax_s8, i32, i32, i32)
DEF_HELPER_2(neon_pmax_u16, i32, i32, i32)
DEF_HELPER_2(neon_pmax_s16, i32, i32, i32)
-DEF_HELPER_2(neon_abd_u8, i32, i32, i32)
-DEF_HELPER_2(neon_abd_s8, i32, i32, i32)
-DEF_HELPER_2(neon_abd_u16, i32, i32, i32)
-DEF_HELPER_2(neon_abd_s16, i32, i32, i32)
-DEF_HELPER_2(neon_abd_u32, i32, i32, i32)
-DEF_HELPER_2(neon_abd_s32, i32, i32, i32)
-
DEF_HELPER_2(neon_shl_u16, i32, i32, i32)
DEF_HELPER_2(neon_shl_s16, i32, i32, i32)
DEF_HELPER_2(neon_rshl_u8, i32, i32, i32)
@@ -741,6 +734,16 @@ DEF_HELPER_FLAGS_4(gvec_uabd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
DEF_HELPER_FLAGS_4(gvec_uabd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
DEF_HELPER_FLAGS_4(gvec_uabd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_saba_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_saba_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_saba_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_saba_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_4(gvec_uaba_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_uaba_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_uaba_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_uaba_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
#ifdef TARGET_AARCH64
#include "helper-a64.h"
#include "helper-sve.h"
diff --git a/target/arm/translate.h b/target/arm/translate.h
index bbfe3d1393..c937dfe9bf 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -342,6 +342,11 @@ void gen_gvec_sabd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
void gen_gvec_uabd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_saba(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+void gen_gvec_uaba(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz);
+
/*
* Forward to the isar_feature_* tests given a DisasContext pointer.
*/
diff --git a/target/arm/neon_helper.c b/target/arm/neon_helper.c
index 448be93fa1..2ef75e04c8 100644
--- a/target/arm/neon_helper.c
+++ b/target/arm/neon_helper.c
@@ -576,16 +576,6 @@ NEON_POP(pmax_s16, neon_s16, 2)
NEON_POP(pmax_u16, neon_u16, 2)
#undef NEON_FN
-#define NEON_FN(dest, src1, src2) \
- dest = (src1 > src2) ? (src1 - src2) : (src2 - src1)
-NEON_VOP(abd_s8, neon_s8, 4)
-NEON_VOP(abd_u8, neon_u8, 4)
-NEON_VOP(abd_s16, neon_s16, 2)
-NEON_VOP(abd_u16, neon_u16, 2)
-NEON_VOP(abd_s32, neon_s32, 1)
-NEON_VOP(abd_u32, neon_u32, 1)
-#undef NEON_FN
-
#define NEON_FN(dest, src1, src2) do { \
int8_t tmp; \
tmp = (int8_t)src2; \
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 54b06553a6..991e451644 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -11197,6 +11197,13 @@ static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_sabd, size);
}
return;
+ case 0xf: /* SABA, UABA */
+ if (u) {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_uaba, size);
+ } else {
+ gen_gvec_fn3(s, is_q, rd, rn, rm, gen_gvec_saba, size);
+ }
+ return;
case 0x10: /* ADD, SUB */
if (u) {
gen_gvec_fn3(s, is_q, rd, rn, rm, tcg_gen_gvec_sub, size);
@@ -11329,16 +11336,6 @@ static void disas_simd_3same_int(DisasContext *s, uint32_t insn)
genenvfn = fns[size][u];
break;
}
- case 0xf: /* SABA, UABA */
- {
- static NeonGenTwoOpFn * const fns[3][2] = {
- { gen_helper_neon_abd_s8, gen_helper_neon_abd_u8 },
- { gen_helper_neon_abd_s16, gen_helper_neon_abd_u16 },
- { gen_helper_neon_abd_s32, gen_helper_neon_abd_u32 },
- };
- genfn = fns[size][u];
- break;
- }
case 0x16: /* SQDMULH, SQRDMULH */
{
static NeonGenTwoOpEnvFn * const fns[2][2] = {
diff --git a/target/arm/translate.c b/target/arm/translate.c
index 0ed6f49f42..bb7a731b21 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -5222,6 +5222,124 @@ void gen_gvec_uabd(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
}
+static void gen_saba_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+ TCGv_i32 t = tcg_temp_new_i32();
+ gen_sabd_i32(t, a, b);
+ tcg_gen_add_i32(d, d, t);
+ tcg_temp_free_i32(t);
+}
+
+static void gen_saba_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+ gen_sabd_i64(t, a, b);
+ tcg_gen_add_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_saba_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
+{
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+ gen_sabd_vec(vece, t, a, b);
+ tcg_gen_add_vec(vece, d, d, t);
+ tcg_temp_free_vec(t);
+}
+
+void gen_gvec_saba(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_sub_vec, INDEX_op_add_vec,
+ INDEX_op_smin_vec, INDEX_op_smax_vec, 0
+ };
+ static const GVecGen3 ops[4] = {
+ { .fniv = gen_saba_vec,
+ .fno = gen_helper_gvec_saba_b,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_8 },
+ { .fniv = gen_saba_vec,
+ .fno = gen_helper_gvec_saba_h,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_16 },
+ { .fni4 = gen_saba_i32,
+ .fniv = gen_saba_vec,
+ .fno = gen_helper_gvec_saba_s,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_32 },
+ { .fni8 = gen_saba_i64,
+ .fniv = gen_saba_vec,
+ .fno = gen_helper_gvec_saba_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
+
+static void gen_uaba_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+ TCGv_i32 t = tcg_temp_new_i32();
+ gen_uabd_i32(t, a, b);
+ tcg_gen_add_i32(d, d, t);
+ tcg_temp_free_i32(t);
+}
+
+static void gen_uaba_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+ TCGv_i64 t = tcg_temp_new_i64();
+ gen_uabd_i64(t, a, b);
+ tcg_gen_add_i64(d, d, t);
+ tcg_temp_free_i64(t);
+}
+
+static void gen_uaba_vec(unsigned vece, TCGv_vec d, TCGv_vec a, TCGv_vec b)
+{
+ TCGv_vec t = tcg_temp_new_vec_matching(d);
+ gen_uabd_vec(vece, t, a, b);
+ tcg_gen_add_vec(vece, d, d, t);
+ tcg_temp_free_vec(t);
+}
+
+void gen_gvec_uaba(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs,
+ uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_sub_vec, INDEX_op_add_vec,
+ INDEX_op_umin_vec, INDEX_op_umax_vec, 0
+ };
+ static const GVecGen3 ops[4] = {
+ { .fniv = gen_uaba_vec,
+ .fno = gen_helper_gvec_uaba_b,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_8 },
+ { .fniv = gen_uaba_vec,
+ .fno = gen_helper_gvec_uaba_h,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_16 },
+ { .fni4 = gen_uaba_i32,
+ .fniv = gen_uaba_vec,
+ .fno = gen_helper_gvec_uaba_s,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_32 },
+ { .fni8 = gen_uaba_i64,
+ .fniv = gen_uaba_vec,
+ .fno = gen_helper_gvec_uaba_d,
+ .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+ .opt_opc = vecop_list,
+ .load_dest = true,
+ .vece = MO_64 },
+ };
+ tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &ops[vece]);
+}
+
/* Translate a NEON data processing instruction. Return nonzero if the
instruction is invalid.
We process data in a mixture of 32-bit and 64-bit chunks.
@@ -5366,6 +5484,16 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
}
return 0;
+ case NEON_3R_VABA:
+ if (u) {
+ gen_gvec_uaba(size, rd_ofs, rn_ofs, rm_ofs,
+ vec_size, vec_size);
+ } else {
+ gen_gvec_saba(size, rd_ofs, rn_ofs, rm_ofs,
+ vec_size, vec_size);
+ }
+ return 0;
+
case NEON_3R_VADD_VSUB:
case NEON_3R_LOGIC:
case NEON_3R_VMAX:
@@ -5510,12 +5638,6 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
case NEON_3R_VQRSHL:
GEN_NEON_INTEGER_OP_ENV(qrshl);
break;
- case NEON_3R_VABA:
- GEN_NEON_INTEGER_OP(abd);
- tcg_temp_free_i32(tmp2);
- tmp2 = neon_load_reg(rd, pass);
- gen_neon_add(size, tmp, tmp2);
- break;
case NEON_3R_VPMAX:
GEN_NEON_INTEGER_OP(pmax);
break;
diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c
index a4972d02fc..fa33df859e 100644
--- a/target/arm/vec_helper.c
+++ b/target/arm/vec_helper.c
@@ -1431,3 +1431,27 @@ DO_ABD(gvec_uabd_s, uint32_t)
DO_ABD(gvec_uabd_d, uint64_t)
#undef DO_ABD
+
+#define DO_ABA(NAME, TYPE) \
+void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \
+{ \
+ intptr_t i, opr_sz = simd_oprsz(desc); \
+ TYPE *d = vd, *n = vn, *m = vm; \
+ \
+ for (i = 0; i < opr_sz / sizeof(TYPE); ++i) { \
+ d[i] += n[i] < m[i] ? m[i] - n[i] : n[i] - m[i]; \
+ } \
+ clear_tail(d, opr_sz, simd_maxsz(desc)); \
+}
+
+DO_ABA(gvec_saba_b, int8_t)
+DO_ABA(gvec_saba_h, int16_t)
+DO_ABA(gvec_saba_s, int32_t)
+DO_ABA(gvec_saba_d, int64_t)
+
+DO_ABA(gvec_uaba_b, uint8_t)
+DO_ABA(gvec_uaba_h, uint16_t)
+DO_ABA(gvec_uaba_s, uint32_t)
+DO_ABA(gvec_uaba_d, uint64_t)
+
+#undef DO_ABA
--
2.20.1
^ permalink raw reply related [flat|nested] 38+ messages in thread
* Re: [PATCH v3 00/16] target/arm: partial vector cleanup
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
` (15 preceding siblings ...)
2020-05-08 15:22 ` [PATCH v3 16/16] target/arm: Vectorize SABA/UABA Richard Henderson
@ 2020-05-12 12:55 ` Peter Maydell
16 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 12:55 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> This is not complete, but shows the direction I'd like to go.
>
> Version 2 extracts more bits from my sve2 branch. There's
> still more to pull back, especially for crypto_helper.c, where
> there are also tail clearing bugs to fix.
>
> Version 3 rebases on master, which has some of the arm neon
> decodetree conversion applied.
Something in this patchset produces a failure on VRSRA:
testcases/aarch32-all-v80/insn_VRSRA__INC.risu.bin
from Alex's set produces different results.
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra}
2020-05-08 15:21 ` [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra} Richard Henderson
@ 2020-05-12 13:09 ` Peter Maydell
2020-05-12 13:46 ` Peter Maydell
2020-05-12 13:51 ` Peter Maydell
1 sibling, 1 reply; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 13:09 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Create vectorized versions of handle_shri_with_rndacc
> for shift+round and shift+round+accumulate. Add out-of-line
> helpers in preparation for longer vector lengths from SVE.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/helper.h | 20 ++
> target/arm/translate.h | 9 +
> target/arm/translate-a64.c | 11 +-
> target/arm/translate.c | 461 +++++++++++++++++++++++++++++++++++--
> target/arm/vec_helper.c | 50 ++++
> 5 files changed, 525 insertions(+), 26 deletions(-)
> +void gen_gvec_srshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
> + int64_t shift, uint32_t opr_sz, uint32_t max_sz)
> +{
> + static const TCGOpcode vecop_list[] = {
> + INDEX_op_shri_vec, INDEX_op_sari_vec, INDEX_op_add_vec, 0
> + };
> + static const GVecGen2i ops[4] = {
> + { .fni8 = gen_srshr8_i64,
> + .fniv = gen_srshr_vec,
> + .fno = gen_helper_gvec_srshr_b,
> + .opt_opc = vecop_list,
> + .vece = MO_8 },
> + { .fni8 = gen_srshr16_i64,
> + .fniv = gen_srshr_vec,
> + .fno = gen_helper_gvec_srshr_h,
> + .opt_opc = vecop_list,
> + .vece = MO_16 },
> + { .fni4 = gen_srshr32_i32,
> + .fniv = gen_srshr_vec,
> + .fno = gen_helper_gvec_srshr_s,
> + .opt_opc = vecop_list,
> + .vece = MO_32 },
> + { .fni8 = gen_srshr64_i64,
> + .fniv = gen_srshr_vec,
> + .fno = gen_helper_gvec_srshr_d,
> + .prefer_i64 = TCG_TARGET_REG_BITS == 64,
> + .opt_opc = vecop_list,
> + .vece = MO_64 },
> + };
> +
> + /* tszimm encoding produces immediates in the range [1..esize] */
> + tcg_debug_assert(shift > 0);
This assert can be triggered:
#4 0x00000000004ee086 in gen_gvec_ursra (vece=3, rd_ofs=3112,
rm_ofs=3296, shift=-6, opr_sz=8,
max_sz=8) at
/home/petmay01/linaro/qemu-from-laptop/qemu/target/arm/translate.c:4413
#5 0x00000000004f17ad in disas_neon_data_insn (s=0x7fffffffe000,
insn=4089066426)
at /home/petmay01/linaro/qemu-from-laptop/qemu/target/arm/translate.c:5702
#6 0x0000000000510dfa in disas_arm_insn (s=0x7fffffffe000, insn=4089066426)
at /home/petmay01/linaro/qemu-from-laptop/qemu/target/arm/translate.c:10803
#7 0x0000000000511b6b in arm_tr_translate_insn
(dcbase=0x7fffffffe000, cpu=0xd04ea0)
at /home/petmay01/linaro/qemu-from-laptop/qemu/target/arm/translate.c:11277
#8 0x000000000045a833 in translator_loop (ops=0xc2ad40
<arm_translator_ops>, db=0x7fffffffe000,
cpu=0xd04ea0, tb=0x7fffe80e3540 <code_gen_buffer+931091>, max_insns=512)
at /home/petmay01/linaro/qemu-from-laptop/qemu/accel/tcg/translator.c:102
#9 0x0000000000512303 in gen_intermediate_code (cpu=0xd04ea0,
tb=0x7fffe80e3540 <code_gen_buffer+931091>, max_insns=512)
at /home/petmay01/linaro/qemu-from-laptop/qemu/target/arm/translate.c:11568
#10 0x0000000000458c01 in tb_gen_code (cpu=0xd04ea0, pc=4268188000,
cs_base=0, flags=1196032,
cflags=-16777216) at
/home/petmay01/linaro/qemu-from-laptop/qemu/accel/tcg/translate-all.c:1718
#11 0x0000000000455f73 in tb_find (cpu=0xd04ea0, last_tb=0x0,
tb_exit=0, cf_mask=0)
at /home/petmay01/linaro/qemu-from-laptop/qemu/accel/tcg/cpu-exec.c:407
#12 0x00000000004566d3 in cpu_exec (cpu=0xd04ea0)
at /home/petmay01/linaro/qemu-from-laptop/qemu/accel/tcg/cpu-exec.c:731
#13 0x000000000049a380 in cpu_loop (env=0xd0d180)
at /home/petmay01/linaro/qemu-from-laptop/qemu/linux-user/arm/cpu_loop.c:219
#14 0x0000000000465637 in main (argc=5, argv=0x7fffffffeb28,
envp=0x7fffffffeb58)
at /home/petmay01/linaro/qemu-from-laptop/qemu/linux-user/main.c:872
> + tcg_debug_assert(shift <= (8 << vece));
> +
> + if (shift == (8 << vece)) {
> + /*
> + * Shifts larger than the element size are architecturally valid.
> + * Signed results in all sign bits. With rounding, this produces
> + * (-1 + 1) >> 1 == 0, or (0 + 1) >> 1 == 0.
> + * I.e. always zero.
> + */
> + tcg_gen_gvec_dup_imm(vece, rd_ofs, opr_sz, max_sz, 0);
> + } else {
> + tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
> + }
> +}
> @@ -5269,6 +5685,28 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
> }
> return 0;
>
> + case 2: /* VRSHR */
> + /* Right shift comes here negative. */
> + shift = -shift;
> + if (u) {
> + gen_gvec_urshr(size, rd_ofs, rm_ofs, shift,
> + vec_size, vec_size);
> + } else {
> + gen_gvec_srshr(size, rd_ofs, rm_ofs, shift,
> + vec_size, vec_size);
> + }
> + return 0;
> +
> + case 3: /* VRSRA */
> + if (u) {
> + gen_gvec_ursra(size, rd_ofs, rm_ofs, shift,
> + vec_size, vec_size);
> + } else {
> + gen_gvec_srsra(size, rd_ofs, rm_ofs, shift,
> + vec_size, vec_size);
> + }
> + return 0;
I think the VRSRA case needs the same "shift = -shift" as VRSHR.
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 01/16] target/arm: Create gen_gvec_[us]sra
2020-05-08 15:21 ` [PATCH v3 01/16] target/arm: Create gen_gvec_[us]sra Richard Henderson
@ 2020-05-12 13:20 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 13:20 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> The functions eliminate duplication of the special cases for
> this operation. They match up with the GVecGen2iFn typedef.
>
> Add out-of-line helpers. We got away with only having inline
> expanders because the neon vector size is only 16 bytes, and
> we know that the inline expansion will always succeed.
> When we reuse this for SVE, tcg-gvec-op may decide to use an
> out-of-line helper due to longer vector lengths.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra}
2020-05-12 13:09 ` Peter Maydell
@ 2020-05-12 13:46 ` Peter Maydell
2020-05-13 2:04 ` Richard Henderson
0 siblings, 1 reply; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 13:46 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Tue, 12 May 2020 at 14:09, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Fri, 8 May 2020 at 16:22, Richard Henderson
> <richard.henderson@linaro.org> wrote:
> >
> > Create vectorized versions of handle_shri_with_rndacc
> > for shift+round and shift+round+accumulate. Add out-of-line
> > helpers in preparation for longer vector lengths from SVE.
> >
> > Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> > + /* tszimm encoding produces immediates in the range [1..esize] */
> > + tcg_debug_assert(shift > 0);
>
> This assert can be triggered:
(well, not this assert, but the equivalent one in gen_gvec_ursra)
> > +static void gen_srshr_vec(unsigned vece, TCGv_vec d, TCGv_vec a, int64_t sh)
> > +{
> > + TCGv_vec t = tcg_temp_new_vec_matching(d);
> > + TCGv_vec ones = tcg_temp_new_vec_matching(d);
> > +
> > + tcg_gen_shri_vec(vece, t, a, sh - 1);
> > + tcg_gen_dupi_vec(vece, ones, 1);
> > + tcg_gen_and_vec(vece, t, t, ones);
> > + tcg_gen_sari_vec(vece, d, a, sh);
> > + tcg_gen_add_vec(vece, d, d, t);
> > +
> > + tcg_temp_free_vec(t);
> > + tcg_temp_free_vec(ones);
> > +}
+void gen_gvec_srshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
+ int64_t shift, uint32_t opr_sz, uint32_t max_sz)
+{
+ static const TCGOpcode vecop_list[] = {
+ INDEX_op_shri_vec, INDEX_op_sari_vec, INDEX_op_add_vec, 0
+ };
Is there documentation somewhere of which vector operations don't
need to be listed in the vecop list? Here gen_srshr_vec() also
uses 'dupi_vec' and 'and_vec', which aren't listed, presumably
because we guarantee them to be implemented? (Hopefully we don't
encounter a future host vector architecture which breaks that
assumption :-))
> > @@ -5269,6 +5685,28 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn)
> > }
> > return 0;
> >
> > + case 2: /* VRSHR */
> > + /* Right shift comes here negative. */
> > + shift = -shift;
> > + if (u) {
> > + gen_gvec_urshr(size, rd_ofs, rm_ofs, shift,
> > + vec_size, vec_size);
> > + } else {
> > + gen_gvec_srshr(size, rd_ofs, rm_ofs, shift,
> > + vec_size, vec_size);
> > + }
> > + return 0;
> > +
> > + case 3: /* VRSRA */
> > + if (u) {
> > + gen_gvec_ursra(size, rd_ofs, rm_ofs, shift,
> > + vec_size, vec_size);
> > + } else {
> > + gen_gvec_srsra(size, rd_ofs, rm_ofs, shift,
> > + vec_size, vec_size);
> > + }
> > + return 0;
>
> I think the VRSRA case needs the same "shift = -shift" as VRSHR.
With this bug fixed,
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra}
2020-05-08 15:21 ` [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra} Richard Henderson
2020-05-12 13:09 ` Peter Maydell
@ 2020-05-12 13:51 ` Peter Maydell
2020-05-13 2:06 ` Richard Henderson
1 sibling, 1 reply; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 13:51 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Create vectorized versions of handle_shri_with_rndacc
> for shift+round and shift+round+accumulate. Add out-of-line
> helpers in preparation for longer vector lengths from SVE.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> + /* tszimm encoding produces immediates in the range [1..esize] */
> + tcg_debug_assert(shift > 0);
> + tcg_debug_assert(shift <= (8 << vece));
> +
> + if (shift == (8 << vece)) {
> + /*
> + * Shifts larger than the element size are architecturally valid.
> + * Signed results in all sign bits. With rounding, this produces
> + * (-1 + 1) >> 1 == 0, or (0 + 1) >> 1 == 0.
> + * I.e. always zero.
> + */
> + tcg_gen_gvec_dup_imm(vece, rd_ofs, opr_sz, max_sz, 0);
Knew I'd forgotten a review comment -- should this "dup_imm to clear
to zeroes" be using a fixed element size rather than 'vece' to avoid
the "dup_imm doesn't handle 128 bits" issue? (Similarly elsewhere.)
> + } else {
> + tcg_gen_gvec_2i(rd_ofs, rm_ofs, opr_sz, max_sz, shift, &ops[vece]);
> + }
> +}
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 03/16] target/arm: Create gen_gvec_{sri,sli}
2020-05-08 15:21 ` [PATCH v3 03/16] target/arm: Create gen_gvec_{sri,sli} Richard Henderson
@ 2020-05-12 13:52 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 13:52 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> The functions eliminate duplication of the special cases for
> this operation. They match up with the GVecGen2iFn typedef.
>
> Add out-of-line helpers. We got away with only having inline
> expanders because the neon vector size is only 16 bytes, and
> we know that the inline expansion will always succeed.
> When we reuse this for SVE, tcg-gvec-op may decide to use an
> out-of-line helper due to longer vector lengths.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 04/16] target/arm: Remove unnecessary range check for VSHL
2020-05-08 15:21 ` [PATCH v3 04/16] target/arm: Remove unnecessary range check for VSHL Richard Henderson
@ 2020-05-12 13:53 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 13:53 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> In 1dc8425e551, while converting to gvec, I added an extra range check
> against the shift count. This was unnecessary because the encoding of
> the shift count produces 0 to the element size - 1.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/translate.c | 12 ++----------
> 1 file changed, 2 insertions(+), 10 deletions(-)
>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 05/16] target/arm: Tidy handle_vec_simd_shri
2020-05-08 15:21 ` [PATCH v3 05/16] target/arm: Tidy handle_vec_simd_shri Richard Henderson
@ 2020-05-12 13:56 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 13:56 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Now that we've converted all cases to gvec, there is quite a bit
> of dead code at the end of the function. Remove it.
>
> Sink the call to gen_gvec_fn2i to the end, loading a function
> pointer within the switch statement.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/translate-a64.c | 56 ++++++++++----------------------------
> 1 file changed, 14 insertions(+), 42 deletions(-)
>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 06/16] target/arm: Create gen_gvec_{ceq, clt, cle, cgt, cge}0
2020-05-08 15:21 ` [PATCH v3 06/16] target/arm: Create gen_gvec_{ceq,clt,cle,cgt,cge}0 Richard Henderson
@ 2020-05-12 14:10 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:10 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Provide a functional interface for the vector expansion.
> This fits better with the existing set of helpers that
> we provide for other operations.
>
> Macro-ize the 5 nearly identical comparisons.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> --
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 07/16] target/arm: Create gen_gvec_{mla,mls}
2020-05-08 15:21 ` [PATCH v3 07/16] target/arm: Create gen_gvec_{mla,mls} Richard Henderson
@ 2020-05-12 14:11 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:11 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Provide a functional interface for the vector expansion.
> This fits better with the existing set of helpers that
> we provide for other operations.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 08/16] target/arm: Swap argument order for VSHL during decode
2020-05-08 15:21 ` [PATCH v3 08/16] target/arm: Swap argument order for VSHL during decode Richard Henderson
@ 2020-05-12 14:14 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:14 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Rather than perform the argument swap during code generation,
> perform it during decode. This means it doesn't have to be
> special cased later, and we can share code with aarch64 code
> generation. Hopefully the decode comment addresses any confusion
> that might arise in between.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> target/arm/neon-dp.decode | 9 +++++++--
> target/arm/translate-neon.inc.c | 3 +--
> 2 files changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode
> index ec3a92fe75..6b0b6566d6 100644
> --- a/target/arm/neon-dp.decode
> +++ b/target/arm/neon-dp.decode
> @@ -65,8 +65,13 @@ VCGT_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 0 .... @3same
> VCGE_S_3s 1111 001 0 0 . .. .... .... 0011 . . . 1 .... @3same
> VCGE_U_3s 1111 001 1 0 . .. .... .... 0011 . . . 1 .... @3same
>
> -VSHL_S_3s 1111 001 0 0 . .. .... .... 0100 . . . 0 .... @3same
> -VSHL_U_3s 1111 001 1 0 . .. .... .... 0100 . . . 0 .... @3same
> +# The shift operations are of the form Vd = Vm << Vn.
> +# By reversing the names of the fields here, we can use standard expanders.
My work-in-progress v2 of the neon decodetree stuff has a slightly more
expanded comment for @3same_rev:
# The _rev suffix indicates that Vn and Vm are reversed. This is
# the case for shifts. In the Arm ARM these insns are documented
# with the Vm and Vn fields in their usual places, but in the
# assembly the operands are listed "backwards", ie in the order
# Dd, Dm, Dn where other insns use Dd, Dn, Dm. For QEMU we choose
# to consider Vm and Vm as being in different fields in the insn,
# which allows us to avoid special-casing shifts in the trans_
# function code (where we would otherwise need to manually swap
# the operands over to call Neon helper functions that are shared
# with AArch64 which does not have this odd reversed-operand situation).
> +@3same_rev .... ... . . . size:2 .... .... .... . q:1 . . .... \
> + &3same vn=%vm_dp vm=%vn_dp vd=%vd_dp
> +
> +VSHL_S_3s 1111 001 0 0 . .. .... .... 0100 . . . 0 .... @3same_rev
> +VSHL_U_3s 1111 001 1 0 . .. .... .... 0100 . . . 0 .... @3same_rev
>
> VMAX_S_3s 1111 001 0 0 . .. .... .... 0110 . . . 0 .... @3same
> VMAX_U_3s 1111 001 1 0 . .. .... .... 0110 . . . 0 .... @3same
> diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c
> index aefeff498a..416302bcc7 100644
> --- a/target/arm/translate-neon.inc.c
> +++ b/target/arm/translate-neon.inc.c
> @@ -692,8 +692,7 @@ static bool trans_VMUL_p_3s(DisasContext *s, arg_3same *a)
> uint32_t rn_ofs, uint32_t rm_ofs, \
> uint32_t oprsz, uint32_t maxsz) \
> { \
> - /* Note the operation is vshl vd,vm,vn */ \
> - tcg_gen_gvec_3(rd_ofs, rm_ofs, rn_ofs, \
> + tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, \
> oprsz, maxsz, &OPARRAY[vece]); \
> } \
> DO_3SAME(INSN, gen_##INSN##_3s)
> --
> 2.20.1
Otherwise
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 09/16] target/arm: Create gen_gvec_{cmtst,ushl,sshl}
2020-05-08 15:21 ` [PATCH v3 09/16] target/arm: Create gen_gvec_{cmtst,ushl,sshl} Richard Henderson
@ 2020-05-12 14:16 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:16 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Provide a functional interface for the vector expansion.
> This fits better with the existing set of helpers that
> we provide for other operations.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 10/16] target/arm: Create gen_gvec_{uqadd, sqadd, uqsub, sqsub}
2020-05-08 15:21 ` [PATCH v3 10/16] target/arm: Create gen_gvec_{uqadd, sqadd, uqsub, sqsub} Richard Henderson
@ 2020-05-12 14:18 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:18 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Provide a functional interface for the vector expansion.
> This fits better with the existing set of helpers that
> we provide for other operations.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> --
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 11/16] target/arm: Remove fp_status from helper_{recpe, rsqrte}_u32
2020-05-08 15:21 ` [PATCH v3 11/16] target/arm: Remove fp_status from helper_{recpe, rsqrte}_u32 Richard Henderson
@ 2020-05-12 14:19 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:19 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> These operations do not touch fp_status.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> diff --git a/target/arm/vfp_helper.c b/target/arm/vfp_helper.c
> index 930d6e747f..a792661166 100644
> --- a/target/arm/vfp_helper.c
> +++ b/target/arm/vfp_helper.c
> @@ -1023,7 +1023,7 @@ float64 HELPER(rsqrte_f64)(float64 input, void *fpstp)
> return make_float64(val);
> }
>
> -uint32_t HELPER(recpe_u32)(uint32_t a, void *fpstp)
> +uint32_t HELPER(recpe_u32)(uint32_t a)
> {
> /* float_status *s = fpstp; */
We should delete this stale commented out line of code too.
> int input, estimate;
> @@ -1038,7 +1038,7 @@ uint32_t HELPER(recpe_u32)(uint32_t a, void *fpstp)
> return deposit32(0, (32 - 9), 9, estimate);
> }
>
> -uint32_t HELPER(rsqrte_u32)(uint32_t a, void *fpstp)
> +uint32_t HELPER(rsqrte_u32)(uint32_t a)
> {
> int estimate;
Otherwise
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 12/16] target/arm: Create gen_gvec_{qrdmla,qrdmls}
2020-05-08 15:21 ` [PATCH v3 12/16] target/arm: Create gen_gvec_{qrdmla,qrdmls} Richard Henderson
@ 2020-05-12 14:20 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:20 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Provide a functional interface for the vector expansion.
> This fits better with the existing set of helpers that
> we provide for other operations.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 13/16] target/arm: Pass pointer to qc to qrdmla/qrdmls
2020-05-08 15:21 ` [PATCH v3 13/16] target/arm: Pass pointer to qc to qrdmla/qrdmls Richard Henderson
@ 2020-05-12 14:28 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:28 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Pass a pointer directly to env->vfp.qc[0], rather than env.
> This will allow SVE2, which does not modify QC, to pass a
> pointer to dummy storage.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */
> -static uint16_t inl_qrdmlah_s16(CPUARMState *env, int16_t src1,
> - int16_t src2, int16_t src3)
> +static int16_t inl_qrdmlah_s16(int16_t src1, int16_t src2,
> + int16_t src3, uint32_t *sat)
Why has the return type changed here ? The new type does make
more sense (there's no behaviour change since the return value
is always assigned to either an int16_t or uint16_t lvalue),
but it also doesn't seem related to the passing-of-QC that
this patch is doing.
> {
> /* Simplify:
> * = ((a3 << 16) + ((e1 * e2) << 1) + (1 << 15)) >> 16
> /* Signed saturating rounding doubling multiply-subtract high half, 16-bit */
> -static uint16_t inl_qrdmlsh_s16(CPUARMState *env, int16_t src1,
> - int16_t src2, int16_t src3)
> +static int16_t inl_qrdmlsh_s16(int16_t src1, int16_t src2,
> + int16_t src3, uint32_t *sat)
Ditto.
Otherwise
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 14/16] target/arm: Clear tail in gvec_fmul_idx_*, gvec_fmla_idx_*
2020-05-08 15:21 ` [PATCH v3 14/16] target/arm: Clear tail in gvec_fmul_idx_*, gvec_fmla_idx_* Richard Henderson
@ 2020-05-12 14:29 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:29 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Must clear the tail for AdvSIMD when SVE is enabled.
>
> Fixes: ca40a6e6e39
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Is it worth adding a "Cc: qemu-stable@nongnu.org" ?
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 15/16] target/arm: Vectorize SABD/UABD
2020-05-08 15:21 ` [PATCH v3 15/16] target/arm: Vectorize SABD/UABD Richard Henderson
@ 2020-05-12 14:40 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:40 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Include 64-bit element size in preparation for SVE2.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 16/16] target/arm: Vectorize SABA/UABA
2020-05-08 15:22 ` [PATCH v3 16/16] target/arm: Vectorize SABA/UABA Richard Henderson
@ 2020-05-12 14:41 ` Peter Maydell
0 siblings, 0 replies; 38+ messages in thread
From: Peter Maydell @ 2020-05-12 14:41 UTC (permalink / raw)
To: Richard Henderson; +Cc: QEMU Developers
On Fri, 8 May 2020 at 16:22, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Include 64-bit element size in preparation for SVE2.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
thanks
-- PMM
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra}
2020-05-12 13:46 ` Peter Maydell
@ 2020-05-13 2:04 ` Richard Henderson
0 siblings, 0 replies; 38+ messages in thread
From: Richard Henderson @ 2020-05-13 2:04 UTC (permalink / raw)
To: Peter Maydell; +Cc: QEMU Developers
On 5/12/20 6:46 AM, Peter Maydell wrote:
> +void gen_gvec_srshr(unsigned vece, uint32_t rd_ofs, uint32_t rm_ofs,
> + int64_t shift, uint32_t opr_sz, uint32_t max_sz)
> +{
> + static const TCGOpcode vecop_list[] = {
> + INDEX_op_shri_vec, INDEX_op_sari_vec, INDEX_op_add_vec, 0
> + };
>
> Is there documentation somewhere of which vector operations don't
> need to be listed in the vecop list? Here gen_srshr_vec() also
> uses 'dupi_vec' and 'and_vec', which aren't listed, presumably
> because we guarantee them to be implemented? (Hopefully we don't
> encounter a future host vector architecture which breaks that
> assumption :-))
Yes, though perhaps not perfectly easy to find: it's at the top of tcg-op-vec.c.
Correct, that the logicals and mov/dupi etc are mandatory and should not be
listed. Moreover, I assert that they are *not* listed, so we get a
CONFIG_DEBUG_TCG check of this list both positive and negative.
I'm going to hope that no future architecture is so irregular as to not
implement logicals. ;-) Or even be as irregular as Intel.
>> I think the VRSRA case needs the same "shift = -shift" as VRSHR.
>
> With this bug fixed,
> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Thanks, fixed.
r~
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra}
2020-05-12 13:51 ` Peter Maydell
@ 2020-05-13 2:06 ` Richard Henderson
0 siblings, 0 replies; 38+ messages in thread
From: Richard Henderson @ 2020-05-13 2:06 UTC (permalink / raw)
To: Peter Maydell; +Cc: QEMU Developers
On 5/12/20 6:51 AM, Peter Maydell wrote:
>> + tcg_gen_gvec_dup_imm(vece, rd_ofs, opr_sz, max_sz, 0);
>
> Knew I'd forgotten a review comment -- should this "dup_imm to clear
> to zeroes" be using a fixed element size rather than 'vece' to avoid
> the "dup_imm doesn't handle 128 bits" issue? (Similarly elsewhere.)
No, because here we cannot have 128-bit elements (not 128-bit vectors).
r~
^ permalink raw reply [flat|nested] 38+ messages in thread
end of thread, other threads:[~2020-05-13 2:07 UTC | newest]
Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-08 15:21 [PATCH v3 00/16] target/arm: partial vector cleanup Richard Henderson
2020-05-08 15:21 ` [PATCH v3 01/16] target/arm: Create gen_gvec_[us]sra Richard Henderson
2020-05-12 13:20 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 02/16] target/arm: Create gen_gvec_{u,s}{rshr,rsra} Richard Henderson
2020-05-12 13:09 ` Peter Maydell
2020-05-12 13:46 ` Peter Maydell
2020-05-13 2:04 ` Richard Henderson
2020-05-12 13:51 ` Peter Maydell
2020-05-13 2:06 ` Richard Henderson
2020-05-08 15:21 ` [PATCH v3 03/16] target/arm: Create gen_gvec_{sri,sli} Richard Henderson
2020-05-12 13:52 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 04/16] target/arm: Remove unnecessary range check for VSHL Richard Henderson
2020-05-12 13:53 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 05/16] target/arm: Tidy handle_vec_simd_shri Richard Henderson
2020-05-12 13:56 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 06/16] target/arm: Create gen_gvec_{ceq,clt,cle,cgt,cge}0 Richard Henderson
2020-05-12 14:10 ` [PATCH v3 06/16] target/arm: Create gen_gvec_{ceq, clt, cle, cgt, cge}0 Peter Maydell
2020-05-08 15:21 ` [PATCH v3 07/16] target/arm: Create gen_gvec_{mla,mls} Richard Henderson
2020-05-12 14:11 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 08/16] target/arm: Swap argument order for VSHL during decode Richard Henderson
2020-05-12 14:14 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 09/16] target/arm: Create gen_gvec_{cmtst,ushl,sshl} Richard Henderson
2020-05-12 14:16 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 10/16] target/arm: Create gen_gvec_{uqadd, sqadd, uqsub, sqsub} Richard Henderson
2020-05-12 14:18 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 11/16] target/arm: Remove fp_status from helper_{recpe, rsqrte}_u32 Richard Henderson
2020-05-12 14:19 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 12/16] target/arm: Create gen_gvec_{qrdmla,qrdmls} Richard Henderson
2020-05-12 14:20 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 13/16] target/arm: Pass pointer to qc to qrdmla/qrdmls Richard Henderson
2020-05-12 14:28 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 14/16] target/arm: Clear tail in gvec_fmul_idx_*, gvec_fmla_idx_* Richard Henderson
2020-05-12 14:29 ` Peter Maydell
2020-05-08 15:21 ` [PATCH v3 15/16] target/arm: Vectorize SABD/UABD Richard Henderson
2020-05-12 14:40 ` Peter Maydell
2020-05-08 15:22 ` [PATCH v3 16/16] target/arm: Vectorize SABA/UABA Richard Henderson
2020-05-12 14:41 ` Peter Maydell
2020-05-12 12:55 ` [PATCH v3 00/16] target/arm: partial vector cleanup Peter Maydell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.