[PATCH v3 1/3] target/riscv: Fix tb->flags FS status

[PATCH v3 1/3] target/riscv: Fix tb->flags FS status

[shihpo.hung]

It was found that running libquantum on riscv-linux qemu produced an
incorrect result. After investigation, FP registers are not saved
during context switch due to incorrect mstatus.FS.

In current implementation tb->flags merges all non-disabled state to
dirty. This means the code in mark_fs_dirty in translate.c that
handles initial and clean states is unreachable.

This patch fixes it and is successfully tested with:
  libquantum

Thanks to Richard for pointing out the actual bug.

v3: remove the redundant condition
v2: root cause FS problem

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/riscv/cpu.h | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index e59343e..de0a8d8 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -293,10 +293,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
 #ifdef CONFIG_USER_ONLY
     *flags = TB_FLAGS_MSTATUS_FS;
 #else
-    *flags = cpu_mmu_index(env, 0);
-    if (riscv_cpu_fp_enabled(env)) {
-        *flags |= TB_FLAGS_MSTATUS_FS;
-    }
+    *flags = cpu_mmu_index(env, 0) | (env->mstatus & MSTATUS_FS);
 #endif
 }
 
-- 
2.7.4

[PATCH v3 2/3] target/riscv: fsd/fsw doesn't dirty FP state

[shihpo.hung]

Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/riscv/insn_trans/trans_rvd.inc.c | 1 -
 target/riscv/insn_trans/trans_rvf.inc.c | 1 -
 2 files changed, 2 deletions(-)

diff --git a/target/riscv/insn_trans/trans_rvd.inc.c b/target/riscv/insn_trans/trans_rvd.inc.c
index 393fa02..ea1044f 100644
--- a/target/riscv/insn_trans/trans_rvd.inc.c
+++ b/target/riscv/insn_trans/trans_rvd.inc.c
@@ -43,7 +43,6 @@ static bool trans_fsd(DisasContext *ctx, arg_fsd *a)
 
     tcg_gen_qemu_st_i64(cpu_fpr[a->rs2], t0, ctx->mem_idx, MO_TEQ);
 
-    mark_fs_dirty(ctx);
     tcg_temp_free(t0);
     return true;
 }
diff --git a/target/riscv/insn_trans/trans_rvf.inc.c b/target/riscv/insn_trans/trans_rvf.inc.c
index 172dbfa..e23cd63 100644
--- a/target/riscv/insn_trans/trans_rvf.inc.c
+++ b/target/riscv/insn_trans/trans_rvf.inc.c
@@ -52,7 +52,6 @@ static bool trans_fsw(DisasContext *ctx, arg_fsw *a)
     tcg_gen_qemu_st_i64(cpu_fpr[a->rs2], t0, ctx->mem_idx, MO_TEUL);
 
     tcg_temp_free(t0);
-    mark_fs_dirty(ctx);
     return true;
 }
 
-- 
2.7.4

[PATCH v3 3/3] target/riscv: update mstatus.SD when FS is set dirty

[shihpo.hung]

remove the check becuase SD bit should summarize FS and XS fields
unconditionally.

Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/riscv/csr.c       | 3 +--
 target/riscv/translate.c | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index da02f9f..0e34c29 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -341,8 +341,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
-    dirty = (riscv_cpu_fp_enabled(env) &&
-             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
+    dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
             ((mstatus & MSTATUS_XS) == MSTATUS_XS);
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;
diff --git a/target/riscv/translate.c b/target/riscv/translate.c
index ab6a891..8e40ed3 100644
--- a/target/riscv/translate.c
+++ b/target/riscv/translate.c
@@ -394,7 +394,7 @@ static void mark_fs_dirty(DisasContext *ctx)
 
     tmp = tcg_temp_new();
     tcg_gen_ld_tl(tmp, cpu_env, offsetof(CPURISCVState, mstatus));
-    tcg_gen_ori_tl(tmp, tmp, MSTATUS_FS);
+    tcg_gen_ori_tl(tmp, tmp, MSTATUS_FS | MSTATUS_SD);
     tcg_gen_st_tl(tmp, cpu_env, offsetof(CPURISCVState, mstatus));
     tcg_temp_free(tmp);
 }
-- 
2.7.4

Re: [PATCH v3 1/3] target/riscv: Fix tb->flags FS status

[Alistair Francis]

On Wed, Jan 15, 2020 at 4:18 PM <shihpo.hung@sifive.com> wrote:
>
> It was found that running libquantum on riscv-linux qemu produced an
> incorrect result. After investigation, FP registers are not saved
> during context switch due to incorrect mstatus.FS.
>
> In current implementation tb->flags merges all non-disabled state to
> dirty. This means the code in mark_fs_dirty in translate.c that
> handles initial and clean states is unreachable.
>
> This patch fixes it and is successfully tested with:
>   libquantum
>
> Thanks to Richard for pointing out the actual bug.
>
> v3: remove the redundant condition
> v2: root cause FS problem
>
> Suggested-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> ---
>  target/riscv/cpu.h | 5 +----
>  1 file changed, 1 insertion(+), 4 deletions(-)
>
> diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
> index e59343e..de0a8d8 100644
> --- a/target/riscv/cpu.h
> +++ b/target/riscv/cpu.h
> @@ -293,10 +293,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
>  #ifdef CONFIG_USER_ONLY
>      *flags = TB_FLAGS_MSTATUS_FS;
>  #else
> -    *flags = cpu_mmu_index(env, 0);
> -    if (riscv_cpu_fp_enabled(env)) {
> -        *flags |= TB_FLAGS_MSTATUS_FS;
> -    }
> +    *flags = cpu_mmu_index(env, 0) | (env->mstatus & MSTATUS_FS);

I don't think this is right, you should use the riscv_cpu_fp_enabled() function.

Right now it's the same as env->mstatus & MSTATUS_FS but when the
Hypervisor extension goes in riscv_cpu_fp_enabled() will be more
complex.

Alistair

>  #endif
>  }
>
> --
> 2.7.4
>
>

Re: [PATCH v3 2/3] target/riscv: fsd/fsw doesn't dirty FP state

[Alistair Francis]

On Wed, Jan 15, 2020 at 4:18 PM <shihpo.hung@sifive.com> wrote:
>
> Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

Reviewed-by: Alistair Francis <alistair.francis@wdc.com>

Alistair

> ---
>  target/riscv/insn_trans/trans_rvd.inc.c | 1 -
>  target/riscv/insn_trans/trans_rvf.inc.c | 1 -
>  2 files changed, 2 deletions(-)
>
> diff --git a/target/riscv/insn_trans/trans_rvd.inc.c b/target/riscv/insn_trans/trans_rvd.inc.c
> index 393fa02..ea1044f 100644
> --- a/target/riscv/insn_trans/trans_rvd.inc.c
> +++ b/target/riscv/insn_trans/trans_rvd.inc.c
> @@ -43,7 +43,6 @@ static bool trans_fsd(DisasContext *ctx, arg_fsd *a)
>
>      tcg_gen_qemu_st_i64(cpu_fpr[a->rs2], t0, ctx->mem_idx, MO_TEQ);
>
> -    mark_fs_dirty(ctx);
>      tcg_temp_free(t0);
>      return true;
>  }
> diff --git a/target/riscv/insn_trans/trans_rvf.inc.c b/target/riscv/insn_trans/trans_rvf.inc.c
> index 172dbfa..e23cd63 100644
> --- a/target/riscv/insn_trans/trans_rvf.inc.c
> +++ b/target/riscv/insn_trans/trans_rvf.inc.c
> @@ -52,7 +52,6 @@ static bool trans_fsw(DisasContext *ctx, arg_fsw *a)
>      tcg_gen_qemu_st_i64(cpu_fpr[a->rs2], t0, ctx->mem_idx, MO_TEUL);
>
>      tcg_temp_free(t0);
> -    mark_fs_dirty(ctx);
>      return true;
>  }
>
> --
> 2.7.4
>
>

Re: [PATCH v3 3/3] target/riscv: update mstatus.SD when FS is set dirty

[Alistair Francis]

On Wed, Jan 15, 2020 at 4:19 PM <shihpo.hung@sifive.com> wrote:
>
> remove the check becuase SD bit should summarize FS and XS fields
> unconditionally.
>
> Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

Reviewed-by: Alistair Francis <alistair.francis@wdc.com>

Alistair

> ---
>  target/riscv/csr.c       | 3 +--
>  target/riscv/translate.c | 2 +-
>  2 files changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/target/riscv/csr.c b/target/riscv/csr.c
> index da02f9f..0e34c29 100644
> --- a/target/riscv/csr.c
> +++ b/target/riscv/csr.c
> @@ -341,8 +341,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
>
>      mstatus = (mstatus & ~mask) | (val & mask);
>
> -    dirty = (riscv_cpu_fp_enabled(env) &&
> -             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
> +    dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
>              ((mstatus & MSTATUS_XS) == MSTATUS_XS);
>      mstatus = set_field(mstatus, MSTATUS_SD, dirty);
>      env->mstatus = mstatus;
> diff --git a/target/riscv/translate.c b/target/riscv/translate.c
> index ab6a891..8e40ed3 100644
> --- a/target/riscv/translate.c
> +++ b/target/riscv/translate.c
> @@ -394,7 +394,7 @@ static void mark_fs_dirty(DisasContext *ctx)
>
>      tmp = tcg_temp_new();
>      tcg_gen_ld_tl(tmp, cpu_env, offsetof(CPURISCVState, mstatus));
> -    tcg_gen_ori_tl(tmp, tmp, MSTATUS_FS);
> +    tcg_gen_ori_tl(tmp, tmp, MSTATUS_FS | MSTATUS_SD);
>      tcg_gen_st_tl(tmp, cpu_env, offsetof(CPURISCVState, mstatus));
>      tcg_temp_free(tmp);
>  }
> --
> 2.7.4
>
>

Re: [PATCH v3 1/3] target/riscv: Fix tb->flags FS status

[ShihPo Hung]

[-- Attachment #1: Type: text/plain, Size: 974 bytes --]

On Wed, Jan 15, 2020 at 2:29 PM Alistair Francis <alistair23@gmail.com>
wrote:

> > -    *flags = cpu_mmu_index(env, 0);
> > -    if (riscv_cpu_fp_enabled(env)) {
> > -        *flags |= TB_FLAGS_MSTATUS_FS;
> > -    }
> > +    *flags = cpu_mmu_index(env, 0) | (env->mstatus & MSTATUS_FS);
>
> I don't think this is right, you should use the riscv_cpu_fp_enabled()
> function.
>
> Right now it's the same as env->mstatus & MSTATUS_FS but when the
> Hypervisor extension goes in riscv_cpu_fp_enabled() will be more
> complex.
>
> Alistair
>
> I agree using riscv_cpu_fp_enabled() to hide the complexity when checking
FP,
but here I only duplicate the FP status (disabled/initial/clean/dirty) to
tb->flags
no matter FP is enabled or not.

Is it still necessary to check this before duplicating it?

I think it is not as long as TB_FLAGS_MSTATUS_FS is equivalent to
MSTATUS_FS.
But I don't know what changes hypervisor extension brings, please correct
me if I am wrong.

ShihPo

[-- Attachment #2: Type: text/html, Size: 1504 bytes --]

Re: [PATCH v3 1/3] target/riscv: Fix tb->flags FS status

[Richard Henderson]

On 1/14/20 8:28 PM, Alistair Francis wrote:
> On Wed, Jan 15, 2020 at 4:18 PM <shihpo.hung@sifive.com> wrote:
>>
>> It was found that running libquantum on riscv-linux qemu produced an
>> incorrect result. After investigation, FP registers are not saved
>> during context switch due to incorrect mstatus.FS.
>>
>> In current implementation tb->flags merges all non-disabled state to
>> dirty. This means the code in mark_fs_dirty in translate.c that
>> handles initial and clean states is unreachable.
>>
>> This patch fixes it and is successfully tested with:
>>   libquantum
>>
>> Thanks to Richard for pointing out the actual bug.
>>
>> v3: remove the redundant condition
>> v2: root cause FS problem
>>
>> Suggested-by: Richard Henderson <richard.henderson@linaro.org>
>> Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com>
>> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
>> ---
>>  target/riscv/cpu.h | 5 +----
>>  1 file changed, 1 insertion(+), 4 deletions(-)
>>
>> diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
>> index e59343e..de0a8d8 100644
>> --- a/target/riscv/cpu.h
>> +++ b/target/riscv/cpu.h
>> @@ -293,10 +293,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
>>  #ifdef CONFIG_USER_ONLY
>>      *flags = TB_FLAGS_MSTATUS_FS;
>>  #else
>> -    *flags = cpu_mmu_index(env, 0);
>> -    if (riscv_cpu_fp_enabled(env)) {
>> -        *flags |= TB_FLAGS_MSTATUS_FS;
>> -    }
>> +    *flags = cpu_mmu_index(env, 0) | (env->mstatus & MSTATUS_FS);
> 
> I don't think this is right, you should use the riscv_cpu_fp_enabled() function.
> 
> Right now it's the same as env->mstatus & MSTATUS_FS but when the
> Hypervisor extension goes in riscv_cpu_fp_enabled() will be more
> complex.

Hmm.  Are you sure something like

  flags |= riscv_cpu_effective_mstatus(env) & MSTATUS_FS;

wouldn't be more appropriate for the hypervisor extension?

I guess I should have another browse through your hv patchset, but I worry now
about bare uses of env->mstatus, if they no longer mean what they appear to mean.


r~

Re: [PATCH v3 1/3] target/riscv: Fix tb->flags FS status

[Alistair Francis]

On Thu, Jan 16, 2020 at 7:46 AM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 1/14/20 8:28 PM, Alistair Francis wrote:
> > On Wed, Jan 15, 2020 at 4:18 PM <shihpo.hung@sifive.com> wrote:
> >>
> >> It was found that running libquantum on riscv-linux qemu produced an
> >> incorrect result. After investigation, FP registers are not saved
> >> during context switch due to incorrect mstatus.FS.
> >>
> >> In current implementation tb->flags merges all non-disabled state to
> >> dirty. This means the code in mark_fs_dirty in translate.c that
> >> handles initial and clean states is unreachable.
> >>
> >> This patch fixes it and is successfully tested with:
> >>   libquantum
> >>
> >> Thanks to Richard for pointing out the actual bug.
> >>
> >> v3: remove the redundant condition
> >> v2: root cause FS problem
> >>
> >> Suggested-by: Richard Henderson <richard.henderson@linaro.org>
> >> Signed-off-by: ShihPo Hung <shihpo.hung@sifive.com>
> >> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> >> ---
> >>  target/riscv/cpu.h | 5 +----
> >>  1 file changed, 1 insertion(+), 4 deletions(-)
> >>
> >> diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
> >> index e59343e..de0a8d8 100644
> >> --- a/target/riscv/cpu.h
> >> +++ b/target/riscv/cpu.h
> >> @@ -293,10 +293,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
> >>  #ifdef CONFIG_USER_ONLY
> >>      *flags = TB_FLAGS_MSTATUS_FS;
> >>  #else
> >> -    *flags = cpu_mmu_index(env, 0);
> >> -    if (riscv_cpu_fp_enabled(env)) {
> >> -        *flags |= TB_FLAGS_MSTATUS_FS;
> >> -    }
> >> +    *flags = cpu_mmu_index(env, 0) | (env->mstatus & MSTATUS_FS);
> >
> > I don't think this is right, you should use the riscv_cpu_fp_enabled() function.
> >
> > Right now it's the same as env->mstatus & MSTATUS_FS but when the
> > Hypervisor extension goes in riscv_cpu_fp_enabled() will be more
> > complex.
>
> Hmm.  Are you sure something like
>
>   flags |= riscv_cpu_effective_mstatus(env) & MSTATUS_FS;
>
> wouldn't be more appropriate for the hypervisor extension?

I was more thinking:

    if (riscv_cpu_fp_enabled(env)) {
        *flags |= env->mstatus & MSTATUS_FS;
    }

as floating point can be disabled from multiple places when we have
the H extension.

>
> I guess I should have another browse through your hv patchset, but I worry now
> about bare uses of env->mstatus, if they no longer mean what they appear to mean.

That was why this was all refacted in the first place as we now need
to check against env->vsstatus as well (depending on virt status).

Alistair

>
>
> r~