[Qemu-devel] [PATCH v1 00/27] Add RISC-V Hypervisor Extension

[Qemu-devel] [PATCH v1 00/27] Add RISC-V Hypervisor Extension

[Alistair Francis]

This patch series adds the RISC-V Hypervisor extension 0.3. This is the
latest draft spec of the Hypervisor extension.

The Hypervisor extension is disabled by default, so this series should
result in no changes to anyone using QEMU unless they enable the
extension. The extention can be enabled with the -cpu property (see
below).

At the moment the spec does not include information about the mstatush
register. As it is not in the spec I haven't added it to QEMU. This
means the extension won't work correctly for 32-bit guests. This should
be a small fix to add the CSR once the spec is updated.

All testing of this implementation has been done by using the baremetal
Xvisor Hypervisor. We are able to run two Linux guests (that's all I
have tried) as guests.

At the moment this spec is in a draft state and is subject to change. As
QEMU is extreamly useful in early bring up I think it makes sense for
QEMU to support non-frozen extensions. I would like to decide with this
series how QEMU will handle all future non-frozen extensions. That is a
standard way that QEMU users can test future RISC-V extensions while
still understanding things will change. One idea is just to disable it by
default, another option is to maybe use the Kconfig to make it a compile
time option which developers can use. Should we also display a warning
when running non-frozen extensions?

Thanks to Anup for doing the initial port of Xvisor. The port is avaliable here:
https://github.com/avpatel/xvisor-next and will run on QEMU.

Also thanks to Atish for implementing the SBI call support in Xvisor and
for lots of help debugging.

To run this yourself:
 1. Apply this patch series to QEMU. The latest branch can be found here:
      https://github.com/alistair23/qemu/tree/mainline/alistair/riscv-hyp-work.next
 2. Get the version of OpenSBI that supports the H extension. This can
    be found here:
      https://github.com/riscv/opensbi/tree/hyp_ext_changes_v1
 3. Build the next release of Xvisor. It is available here:
      https://github.com/avpatel/xvisor-next
 4. Make sure you build the Xvisor tests, see here for details:
      https://github.com/avpatel/xvisor-next/tree/master/tests/riscv/virt64/linux
 5. Run QEMU:
     ./riscv64-softmmu/qemu-system-riscv64 -nographic \
       -machine virt -cpu rv64,h=true\
       -serial mon:stdio -serial null -m 4G \
       -device loader,file=vmm.bin,addr=0x80200000 \
       -kernel fw_jump.elf \
       -initrd vmm-disk-linux.img \
       -append "vmm.console=uart@10000000 vmm.bootcmd=\"vfs mount initrd /;vfs run /boot.xscript;vfs cat /system/banner.txt\""

   Once you get to the prompt you can start the geust by running:
     guest kick guest0
   You can then bind to the serial port using:
     vserial bind guest0/uart0
   Then you can start Linux using:
     autoexec

 This was all tested with the mainline 5.1 kernel. I don't know if it
 will work on older kernels.

So far all of the QEMU work has been tested on Xvisor.

Known Issues/TODO:
 - Add mstatush to support 32-bit Hypervisors
 - Fix the random hang that sometimes appears when running a Hypervisor guest (~5%)

There is also on going work from Anup to port KVM.
We have code complete implementation of RISC-V KVM kernel module and
RISC-V KVMTOOL. Currently, we are debugging KVM on QEMU and we will
send-out RFC PATCHES for KVM in June/July 2019.
The KVM RISC-V kernel module is available in riscv_kvm_v1
branch at: https://github.com/avpatel/linux.git
The KVMTOOL RISC-V port is available in riscv_v1 branch of
https://github.com/avpatel/kvmtool.git

There is very early work on a Xen port as well which is avaliable here:
https://github.com/alistair23/xen/tree/alistair/riscv-port

Changes from RFC:
 - Rebase on latest master
 - Add floating point changes from Hypervisor extension

Alistair Francis (27):
  target/riscv: Don't set write permissions on dirty PTEs
  target/riscv: Add the Hypervisor extension
  target/riscv: Add the virtulisation mode
  target/riscv: Add the force HS exception mode
  target/riscv: Add the Hypervisor CSRs to CPUState
  target/riscv: Dump Hypervisor registers if enabled
  target/riscv: Remove strict perm checking for CSR R/W
  target/riscv: Create function to test if FP is enabled
  target/riscv: Add support for background interrupt setting
  target/riscv: Add Hypervisor CSR access functions
  target/riscv: Add background CSRs accesses
  target/riscv: Add background register swapping function
  target/ricsv: Flush the TLB on virtulisation mode changes
  target/riscv: Generate illegal instruction on WFI when V=1
  riscv: plic: Remove unused interrupt functions
  riscv: plic: Always set sip.SEIP bit for HS
  target/riscv: Add hypvervisor trap support
  target/riscv: Add Hypervisor trap return support
  target/riscv: Add hfence instructions
  target/riscv: Disable guest FP support based on backgrond status
  target/riscv: Mark both sstatus and bsstatus as dirty
  target/riscv: Respect MPRV and SPRV for floating point ops
  target/riscv: Allow specifying MMU stage
  target/riscv: Allow specifying number of MMU stages
  target/riscv: Implement second stage MMU
  target/riscv: Call the second stage MMU in virtualisation mode
  target/riscv: Allow enabling the Hypervisor extension

 hw/riscv/sifive_plic.c                        |  24 +-
 include/hw/riscv/sifive_plic.h                |   3 -
 target/riscv/cpu.c                            |  31 ++
 target/riscv/cpu.h                            |  32 +-
 target/riscv/cpu_bits.h                       |  32 +-
 target/riscv/cpu_helper.c                     | 443 ++++++++++++++++--
 target/riscv/csr.c                            | 216 ++++++++-
 target/riscv/insn32.decode                    |  23 +-
 .../riscv/insn_trans/trans_privileged.inc.c   |  40 ++
 target/riscv/op_helper.c                      |  71 ++-
 target/riscv/translate.c                      |  26 +
 11 files changed, 839 insertions(+), 102 deletions(-)

-- 
2.21.0

[Qemu-devel] [PATCH v1 01/27] target/riscv: Don't set write permissions on dirty PTEs

[Alistair Francis]

Setting write permission on dirty PTEs results in userspace inside a
Hypervisor guest (VU) becoming corrupted. This appears to be becuase it
ends up with write permission in the second stage translation in cases
where we aren't doing a store.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index b1bee3d45d..872835177a 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -326,10 +326,8 @@ restart:
             if ((pte & PTE_X)) {
                 *prot |= PAGE_EXEC;
             }
-            /* add write permission on stores or if the page is already dirty,
-               so that we TLB miss on later writes to update the dirty bit */
-            if ((pte & PTE_W) &&
-                    (access_type == MMU_DATA_STORE || (pte & PTE_D))) {
+            /* add write permission on stores */
+            if ((pte & PTE_W) && (access_type == MMU_DATA_STORE)) {
                 *prot |= PAGE_WRITE;
             }
             return TRANSLATE_SUCCESS;
-- 
2.21.0

[Qemu-devel] [PATCH v1 02/27] target/riscv: Add the Hypervisor extension

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index 8937bda918..3337d1aef3 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -81,6 +81,7 @@
 #define RVC RV('C')
 #define RVS RV('S')
 #define RVU RV('U')
+#define RVH RV('H')
 
 /* S extension denotes that Supervisor mode exists, however it is possible
    to have a core that support S mode but does not have an MMU and there
-- 
2.21.0

[Qemu-devel] [PATCH v1 03/27] target/riscv: Add the virtulisation mode

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.h        |  4 ++++
 target/riscv/cpu_bits.h   |  6 ++++++
 target/riscv/cpu_helper.c | 23 +++++++++++++++++++++++
 3 files changed, 33 insertions(+)

diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index 3337d1aef3..de4843b879 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -132,6 +132,8 @@ struct CPURISCVState {
 
 #ifndef CONFIG_USER_ONLY
     target_ulong priv;
+    /* This contains QEMU specific information about the virt state. */
+    target_ulong virt;
     target_ulong resetvec;
 
     target_ulong mhartid;
@@ -278,6 +280,8 @@ void riscv_cpu_do_interrupt(CPUState *cpu);
 int riscv_cpu_gdb_read_register(CPUState *cpu, uint8_t *buf, int reg);
 int riscv_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg);
 bool riscv_cpu_exec_interrupt(CPUState *cs, int interrupt_request);
+bool riscv_cpu_virt_enabled(CPURISCVState *env);
+void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable);
 int riscv_cpu_mmu_index(CPURISCVState *env, bool ifetch);
 hwaddr riscv_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr);
 void  riscv_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
diff --git a/target/riscv/cpu_bits.h b/target/riscv/cpu_bits.h
index dc9d53d4be..07c95e8d2c 100644
--- a/target/riscv/cpu_bits.h
+++ b/target/riscv/cpu_bits.h
@@ -417,6 +417,12 @@
 #define PRV_H 2 /* Reserved */
 #define PRV_M 3
 
+/* Virtulisation modes */
+#define VIRT_OFF            0
+#define VIRT_ON             1
+#define VIRT_MODE_SHIFT     0
+#define VIRT_MODE_MASK      (1 << VIRT_MODE_SHIFT)
+
 /* RV32 satp CSR field masks */
 #define SATP32_MODE         0x80000000
 #define SATP32_ASID         0x7fc00000
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 872835177a..5912ae63b7 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -71,6 +71,29 @@ bool riscv_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
 
 #if !defined(CONFIG_USER_ONLY)
 
+bool riscv_cpu_virt_enabled(CPURISCVState *env)
+{
+    bool tmp;
+
+    if (!riscv_has_ext(env, RVH)) {
+        return false;
+    }
+
+    tmp = (env->virt & VIRT_MODE_MASK) >> VIRT_MODE_SHIFT;
+
+    return tmp == VIRT_ON;
+}
+
+void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable)
+{
+    if (!riscv_has_ext(env, RVH)) {
+        return;
+    }
+
+    env->virt &= ~VIRT_MODE_MASK;
+    env->virt |= enable << VIRT_MODE_SHIFT;
+}
+
 int riscv_cpu_claim_interrupts(RISCVCPU *cpu, uint32_t interrupts)
 {
     CPURISCVState *env = &cpu->env;
-- 
2.21.0

[Qemu-devel] [PATCH v1 04/27] target/riscv: Add the force HS exception mode

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.h        |  2 ++
 target/riscv/cpu_bits.h   |  6 ++++++
 target/riscv/cpu_helper.c | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+)

diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index de4843b879..eeb3756c91 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -282,6 +282,8 @@ int riscv_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg);
 bool riscv_cpu_exec_interrupt(CPUState *cs, int interrupt_request);
 bool riscv_cpu_virt_enabled(CPURISCVState *env);
 void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable);
+bool riscv_cpu_force_hs_excep_enabled(CPURISCVState *env);
+void riscv_cpu_set_force_hs_excep(CPURISCVState *env, bool enable);
 int riscv_cpu_mmu_index(CPURISCVState *env, bool ifetch);
 hwaddr riscv_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr);
 void  riscv_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
diff --git a/target/riscv/cpu_bits.h b/target/riscv/cpu_bits.h
index 07c95e8d2c..c898bb1102 100644
--- a/target/riscv/cpu_bits.h
+++ b/target/riscv/cpu_bits.h
@@ -423,6 +423,12 @@
 #define VIRT_MODE_SHIFT     0
 #define VIRT_MODE_MASK      (1 << VIRT_MODE_SHIFT)
 
+/* HS-level exceptions modes */
+#define CLEAR_HS_EXCEP        0
+#define FORCE_HS_EXCEP        1
+#define FORCE_HS_EXCEP_SHIFT  1
+#define FORCE_HS_EXCEP_MASK   (1 << FORCE_HS_EXCEP_SHIFT)
+
 /* RV32 satp CSR field masks */
 #define SATP32_MODE         0x80000000
 #define SATP32_ASID         0x7fc00000
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 5912ae63b7..0fdc81f71f 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -94,6 +94,29 @@ void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable)
     env->virt |= enable << VIRT_MODE_SHIFT;
 }
 
+bool riscv_cpu_force_hs_excep_enabled(CPURISCVState *env)
+{
+    bool tmp;
+
+    if (!riscv_has_ext(env, RVH)) {
+        return false;
+    }
+
+    tmp = (env->virt & FORCE_HS_EXCEP_MASK) >> FORCE_HS_EXCEP_SHIFT;
+
+    return tmp == FORCE_HS_EXCEP;
+}
+
+void riscv_cpu_set_force_hs_excep(CPURISCVState *env, bool enable)
+{
+    if (!riscv_has_ext(env, RVH)) {
+        return;
+    }
+
+    env->virt &= ~FORCE_HS_EXCEP_MASK;
+    env->virt |= enable << FORCE_HS_EXCEP_SHIFT;
+}
+
 int riscv_cpu_claim_interrupts(RISCVCPU *cpu, uint32_t interrupts)
 {
     CPURISCVState *env = &cpu->env;
-- 
2.21.0

[Qemu-devel] [PATCH v1 05/27] target/riscv: Add the Hypervisor CSRs to CPUState

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index eeb3756c91..b99d2b7af2 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -169,12 +169,29 @@ struct CPURISCVState {
     target_ulong mcause;
     target_ulong mtval;  /* since: priv-1.10.0 */
 
+    /* Hypervisor CSRs */
+    target_ulong hstatus;
+    target_ulong hedeleg;
+    target_ulong hideleg;
+    target_ulong hgatp;
+
     target_ulong scounteren;
     target_ulong mcounteren;
 
     target_ulong sscratch;
     target_ulong mscratch;
 
+    /* Background CSRs */
+    target_ulong bsstatus;
+    target_ulong bsip;
+    target_ulong bsie;
+    target_ulong bstvec;
+    target_ulong bsscratch;
+    target_ulong bsepc;
+    target_ulong bscause;
+    target_ulong bstval;
+    target_ulong bsatp;
+
     /* temporary htif regs */
     uint64_t mfromhost;
     uint64_t mtohost;
-- 
2.21.0

[Qemu-devel] [PATCH v1 06/27] target/riscv: Dump Hypervisor registers if enabled

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 1d1378bb7f..6111f0f0bc 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -220,14 +220,41 @@ static void riscv_cpu_dump_state(CPUState *cs, FILE *f, int flags)
 #ifndef CONFIG_USER_ONLY
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "mhartid ", env->mhartid);
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "mstatus ", env->mstatus);
+    if (riscv_has_ext(env, RVH)) {
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "hstatus ", env->hstatus);
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "bstatus ", env->bsstatus);
+    }
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "mip     ",
                  (target_ulong)atomic_read(&env->mip));
+    if (riscv_has_ext(env, RVH)) {
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "bsip    ",
+                     (target_ulong)atomic_read(&env->bsip));
+    }
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "mie     ", env->mie);
+    if (riscv_has_ext(env, RVH)) {
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "bsie    ", env->bsie);
+    }
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "mideleg ", env->mideleg);
+    if (riscv_has_ext(env, RVH)) {
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "hideleg ", env->hideleg);
+    }
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "medeleg ", env->medeleg);
+    if (riscv_has_ext(env, RVH)) {
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "hedeleg ", env->hedeleg);
+    }
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "mtvec   ", env->mtvec);
+    if (riscv_has_ext(env, RVH)) {
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "bstvec  ", env->bstvec);
+    }
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "mepc    ", env->mepc);
+    qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "sepc    ", env->sepc);
+    if (riscv_has_ext(env, RVH)) {
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "bsepc   ", env->bsepc);
+    }
     qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "mcause  ", env->mcause);
+    if (riscv_has_ext(env, RVH)) {
+        qemu_fprintf(f, " %s " TARGET_FMT_lx "\n", "bscause ", env->bscause);
+    }
 #endif
 
     for (i = 0; i < 32; i++) {
-- 
2.21.0

[Qemu-devel] [PATCH v1 07/27] target/riscv: Remove strict perm checking for CSR R/W

[Alistair Francis]

The privledge check based on the CSR address mask 0x300 doesn't work
when using Hypervisor extensions so remove the check

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/csr.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index f9e2910643..809c4c09a9 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -776,9 +776,8 @@ int riscv_csrrw(CPURISCVState *env, int csrno, target_ulong *ret_value,
 
     /* check privileges and return -1 if check fails */
 #if !defined(CONFIG_USER_ONLY)
-    int csr_priv = get_field(csrno, 0x300);
     int read_only = get_field(csrno, 0xC00) == 3;
-    if ((write_mask && read_only) || (env->priv < csr_priv)) {
+    if (write_mask && read_only) {
         return -1;
     }
 #endif
-- 
2.21.0

[Qemu-devel] [PATCH v1 08/27] target/riscv: Create function to test if FP is enabled

[Alistair Francis]

Let's creaate a function that tests if floating point support is
enabled. We can then protect all floating point operations based on if
they are enabled.

This patch so far doesn't change anything, it's just preparing for the
Hypervisor support for floating point operations.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.h        |  6 +++++-
 target/riscv/cpu_helper.c | 10 ++++++++++
 target/riscv/csr.c        | 22 +++++++++++++---------
 3 files changed, 28 insertions(+), 10 deletions(-)

diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index b99d2b7af2..eed561d56e 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -301,6 +301,7 @@ bool riscv_cpu_virt_enabled(CPURISCVState *env);
 void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable);
 bool riscv_cpu_force_hs_excep_enabled(CPURISCVState *env);
 void riscv_cpu_set_force_hs_excep(CPURISCVState *env, bool enable);
+bool riscv_cpu_fp_enabled(CPURISCVState *env);
 int riscv_cpu_mmu_index(CPURISCVState *env, bool ifetch);
 hwaddr riscv_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr);
 void  riscv_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
@@ -344,7 +345,10 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
 #ifdef CONFIG_USER_ONLY
     *flags = TB_FLAGS_MSTATUS_FS;
 #else
-    *flags = cpu_mmu_index(env, 0) | (env->mstatus & MSTATUS_FS);
+    *flags = cpu_mmu_index(env, 0);
+    if (riscv_cpu_fp_enabled(env)) {
+        *flags |= env->mstatus & MSTATUS_FS;
+    }
 #endif
 }
 
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 0fdc81f71f..f51139b543 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -117,6 +117,16 @@ void riscv_cpu_set_force_hs_excep(CPURISCVState *env, bool enable)
     env->virt |= enable << FORCE_HS_EXCEP_SHIFT;
 }
 
+/* Return true is floating point support is currently enabled */
+bool riscv_cpu_fp_enabled(CPURISCVState *env)
+{
+    if (env->mstatus & MSTATUS_FS) {
+        return true;
+    }
+
+    return false;
+}
+
 int riscv_cpu_claim_interrupts(RISCVCPU *cpu, uint32_t interrupts)
 {
     CPURISCVState *env = &cpu->env;
diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index 809c4c09a9..4b1308d47c 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -46,7 +46,7 @@ void riscv_set_csr_ops(int csrno, riscv_csr_operations *ops)
 static int fs(CPURISCVState *env, int csrno)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
 #endif
@@ -92,7 +92,7 @@ static int pmp(CPURISCVState *env, int csrno)
 static int read_fflags(CPURISCVState *env, int csrno, target_ulong *val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
 #endif
@@ -103,7 +103,7 @@ static int read_fflags(CPURISCVState *env, int csrno, target_ulong *val)
 static int write_fflags(CPURISCVState *env, int csrno, target_ulong val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
     env->mstatus |= MSTATUS_FS;
@@ -115,7 +115,7 @@ static int write_fflags(CPURISCVState *env, int csrno, target_ulong val)
 static int read_frm(CPURISCVState *env, int csrno, target_ulong *val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
 #endif
@@ -126,7 +126,7 @@ static int read_frm(CPURISCVState *env, int csrno, target_ulong *val)
 static int write_frm(CPURISCVState *env, int csrno, target_ulong val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
     env->mstatus |= MSTATUS_FS;
@@ -138,7 +138,7 @@ static int write_frm(CPURISCVState *env, int csrno, target_ulong val)
 static int read_fcsr(CPURISCVState *env, int csrno, target_ulong *val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
 #endif
@@ -150,7 +150,7 @@ static int read_fcsr(CPURISCVState *env, int csrno, target_ulong *val)
 static int write_fcsr(CPURISCVState *env, int csrno, target_ulong val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
     env->mstatus |= MSTATUS_FS;
@@ -324,8 +324,12 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
-    int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
-                ((mstatus & MSTATUS_XS) == MSTATUS_XS);
+    int dirty = (mstatus & MSTATUS_XS) == MSTATUS_XS;
+
+    if (riscv_cpu_fp_enabled(env)) {
+        dirty |= (mstatus & MSTATUS_FS) == MSTATUS_FS;
+    }
+
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;
 
-- 
2.21.0

[Qemu-devel] [PATCH v1 09/27] target/riscv: Add support for background interrupt setting

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index f51139b543..0116d2499c 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -38,12 +38,27 @@ static int riscv_cpu_local_irq_pending(CPURISCVState *env)
 {
     target_ulong mstatus_mie = get_field(env->mstatus, MSTATUS_MIE);
     target_ulong mstatus_sie = get_field(env->mstatus, MSTATUS_SIE);
+    target_ulong bsstatus_sie = get_field(env->bsstatus, MSTATUS_SIE);
+
     target_ulong pending = atomic_read(&env->mip) & env->mie;
-    target_ulong mie = env->priv < PRV_M || (env->priv == PRV_M && mstatus_mie);
-    target_ulong sie = env->priv < PRV_S || (env->priv == PRV_S && mstatus_sie);
+    target_ulong hspending = atomic_read(&env->bsip) & env->bsie;
+
+    target_ulong mie  = env->priv < PRV_M || (env->priv == PRV_M && mstatus_mie);
+    target_ulong sie  = env->priv < PRV_S || (env->priv == PRV_S && mstatus_sie);
+    target_ulong bsie = env->priv < PRV_S || (env->priv == PRV_S && bsstatus_sie);
+
     target_ulong irqs = (pending & ~env->mideleg & -mie) |
                         (pending &  env->mideleg & -sie);
 
+    if (riscv_cpu_virt_enabled(env)) {
+        target_ulong pending_hs_irq = hspending & -bsie;
+
+        if (pending_hs_irq) {
+            riscv_cpu_set_force_hs_excep(env, FORCE_HS_EXCEP);
+            return ctz64(pending_hs_irq);
+        }
+    }
+
     if (irqs) {
         return ctz64(irqs); /* since non-zero */
     } else {
-- 
2.21.0

[Qemu-devel] [PATCH v1 10/27] target/riscv: Add Hypervisor CSR access functions

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/csr.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index 4b1308d47c..911f83ef51 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -82,6 +82,20 @@ static int smode(CPURISCVState *env, int csrno)
     return -!riscv_has_ext(env, RVS);
 }
 
+static int hmode(CPURISCVState *env, int csrno)
+{
+    if (riscv_has_ext(env, RVS) &&
+        riscv_has_ext(env, RVH)) {
+        /* Hypervisor extension is supported */
+        if ((env->priv == PRV_S && !riscv_cpu_virt_enabled(env)) ||
+            env->priv == PRV_M) {
+            return 0;
+        }
+    }
+
+        return -1;
+}
+
 static int pmp(CPURISCVState *env, int csrno)
 {
     return -!riscv_feature(env, RISCV_FEATURE_PMP);
@@ -736,6 +750,55 @@ static int write_satp(CPURISCVState *env, int csrno, target_ulong val)
     return 0;
 }
 
+/* Hypervisor Extensions */
+static int read_hstatus(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->hstatus;
+    return 0;
+}
+
+static int write_hstatus(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->hstatus = val;
+    return 0;
+}
+
+static int read_hedeleg(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->hedeleg;
+    return 0;
+}
+
+static int write_hedeleg(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->hedeleg = val;
+    return 0;
+}
+
+static int read_hideleg(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->hideleg;
+    return 0;
+}
+
+static int write_hideleg(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->hideleg = val;
+    return 0;
+}
+
+static int read_hgatp(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->hgatp;
+    return 0;
+}
+
+static int write_hgatp(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->hgatp = val;
+    return 0;
+}
+
 /* Physical Memory Protection */
 static int read_pmpcfg(CPURISCVState *env, int csrno, target_ulong *val)
 {
@@ -919,6 +982,11 @@ static riscv_csr_operations csr_ops[CSR_TABLE_SIZE] = {
     /* Supervisor Protection and Translation */
     [CSR_SATP] =                { smode, read_satp,        write_satp        },
 
+    [CSR_HSTATUS] =             { hmode,   read_hstatus,     write_hstatus    },
+    [CSR_HEDELEG] =             { hmode,   read_hedeleg,     write_hedeleg    },
+    [CSR_HIDELEG] =             { hmode,   read_hideleg,     write_hideleg    },
+    [CSR_HGATP] =               { hmode,   read_hgatp,       write_hgatp      },
+
     /* Physical Memory Protection */
     [CSR_PMPCFG0  ... CSR_PMPADDR9] =  { pmp,   read_pmpcfg,  write_pmpcfg   },
     [CSR_PMPADDR0 ... CSR_PMPADDR15] = { pmp,   read_pmpaddr, write_pmpaddr  },
-- 
2.21.0

[Qemu-devel] [PATCH v1 11/27] target/riscv: Add background CSRs accesses

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_bits.h |  11 ++++
 target/riscv/csr.c      | 119 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 130 insertions(+)

diff --git a/target/riscv/cpu_bits.h b/target/riscv/cpu_bits.h
index c898bb1102..9c27727e6f 100644
--- a/target/riscv/cpu_bits.h
+++ b/target/riscv/cpu_bits.h
@@ -169,6 +169,17 @@
 #define CSR_SPTBR           0x180
 #define CSR_SATP            0x180
 
+/* Background CSRs */
+#define CSR_BSSTATUS        0x200
+#define CSR_BSIE            0x204
+#define CSR_BSTVEC          0x205
+#define CSR_BSSCRATCH       0x240
+#define CSR_BSEPC           0x241
+#define CSR_BSCAUSE         0x242
+#define CSR_BSTVAL          0x243
+#define CSR_BSIP            0x244
+#define CSR_BSATP           0x280
+
 /* Physical Memory Protection */
 #define CSR_PMPCFG0         0x3a0
 #define CSR_PMPCFG1         0x3a1
diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index 911f83ef51..c55eea44ec 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -799,6 +799,115 @@ static int write_hgatp(CPURISCVState *env, int csrno, target_ulong val)
     return 0;
 }
 
+/* Background CSR Registers */
+static int read_bsstatus(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->bsstatus;
+    return 0;
+}
+
+static int write_bsstatus(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->bsstatus = val;
+    return 0;
+}
+
+static int read_bsie(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->bsie;
+    return 0;
+}
+
+static int write_bsie(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->bsie = val;
+    return 0;
+}
+
+static int read_bstvec(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->bstvec;
+    return 0;
+}
+
+static int write_bstvec(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->bstvec = val;
+    return 0;
+}
+
+static int read_bsscratch(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->bsscratch;
+    return 0;
+}
+
+static int write_bsscratch(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->bsscratch = val;
+    return 0;
+}
+
+static int read_bsepc(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->bsepc;
+    return 0;
+}
+
+static int write_bsepc(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->bsepc = val;
+    return 0;
+}
+
+static int read_bscause(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->bscause;
+    return 0;
+}
+
+static int write_bscause(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->bscause = val;
+    return 0;
+}
+
+static int read_bstval(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->bstval;
+    return 0;
+}
+
+static int write_bstval(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->bstval = val;
+    return 0;
+}
+
+static int read_bsip(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = (target_ulong)atomic_read(&env->bsip);
+    return 0;
+}
+
+static int write_bsip(CPURISCVState *env, int csrno, target_ulong val)
+{
+    atomic_set(&env->bsip, val);
+    return 0;
+}
+
+static int read_bsatp(CPURISCVState *env, int csrno, target_ulong *val)
+{
+    *val = env->bsatp;
+    return 0;
+}
+
+static int write_bsatp(CPURISCVState *env, int csrno, target_ulong val)
+{
+    env->bsatp = val;
+    return 0;
+}
+
 /* Physical Memory Protection */
 static int read_pmpcfg(CPURISCVState *env, int csrno, target_ulong *val)
 {
@@ -987,6 +1096,16 @@ static riscv_csr_operations csr_ops[CSR_TABLE_SIZE] = {
     [CSR_HIDELEG] =             { hmode,   read_hideleg,     write_hideleg    },
     [CSR_HGATP] =               { hmode,   read_hgatp,       write_hgatp      },
 
+    [CSR_BSSTATUS] =            { hmode,   read_bsstatus,    write_bsstatus   },
+    [CSR_BSIE] =                { hmode,   read_bsie,        write_bsie       },
+    [CSR_BSTVEC] =              { hmode,   read_bstvec,      write_bstvec     },
+    [CSR_BSSCRATCH] =           { hmode,   read_bsscratch,   write_bsscratch  },
+    [CSR_BSEPC] =               { hmode,   read_bsepc,       write_bsepc      },
+    [CSR_BSCAUSE] =             { hmode,   read_bscause,     write_bscause    },
+    [CSR_BSTVAL] =              { hmode,   read_bstval,      write_bstval     },
+    [CSR_BSIP] =                { hmode,   read_bsip,        write_bsip       },
+    [CSR_BSATP] =               { hmode,   read_bsatp,       write_bsatp      },
+
     /* Physical Memory Protection */
     [CSR_PMPCFG0  ... CSR_PMPADDR9] =  { pmp,   read_pmpcfg,  write_pmpcfg   },
     [CSR_PMPADDR0 ... CSR_PMPADDR15] = { pmp,   read_pmpaddr, write_pmpaddr  },
-- 
2.21.0

[Qemu-devel] [PATCH v1 12/27] target/riscv: Add background register swapping function

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.h        |  1 +
 target/riscv/cpu_bits.h   |  5 ++++
 target/riscv/cpu_helper.c | 52 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 58 insertions(+)

diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index eed561d56e..5b3b32dbbc 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -320,6 +320,7 @@ void riscv_cpu_list(void);
 #define cpu_mmu_index riscv_cpu_mmu_index
 
 #ifndef CONFIG_USER_ONLY
+void riscv_cpu_swap_background_regs(CPURISCVState *env);
 int riscv_cpu_claim_interrupts(RISCVCPU *cpu, uint32_t interrupts);
 uint32_t riscv_cpu_update_mip(RISCVCPU *cpu, uint32_t mask, uint32_t value);
 #define BOOL_TO_MASK(x) (-!!(x)) /* helper for riscv_cpu_update_mip value */
diff --git a/target/riscv/cpu_bits.h b/target/riscv/cpu_bits.h
index 9c27727e6f..28117bdd32 100644
--- a/target/riscv/cpu_bits.h
+++ b/target/riscv/cpu_bits.h
@@ -550,3 +550,8 @@
 #define SIP_SSIP                           MIP_SSIP
 #define SIP_STIP                           MIP_STIP
 #define SIP_SEIP                           MIP_SEIP
+
+/* MIE masks */
+#define MIE_SEIE                           (1 << IRQ_S_EXT)
+#define MIE_STIE                           (1 << IRQ_S_TIMER)
+#define MIE_SSIE                           (1 << IRQ_S_SOFT)
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 0116d2499c..5e5029ac0b 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -86,6 +86,58 @@ bool riscv_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
 
 #if !defined(CONFIG_USER_ONLY)
 
+void riscv_cpu_swap_background_regs(CPURISCVState *env)
+{
+    RISCVCPU *cpu = riscv_env_get_cpu(env);
+    target_ulong tmp;
+    target_ulong mstatus_mask = MSTATUS_MXR | MSTATUS_SUM | MSTATUS_FS |
+                                MSTATUS_SPP | MSTATUS_SPIE | MSTATUS_SIE;
+    target_ulong sie_mask = MIE_SEIE | MIE_STIE | MIE_SSIE;
+
+    g_assert(riscv_has_ext(env, RVH));
+
+#if defined(TARGET_RISCV64)
+    mstatus_mask |= MSTATUS64_UXL;
+#endif
+
+    tmp = env->bsstatus & mstatus_mask;
+    env->bsstatus = env->mstatus & mstatus_mask;
+    env->mstatus = (env->mstatus & ~mstatus_mask) | tmp;
+
+    tmp = env->bsie & sie_mask;
+    env->bsie = env->mie & sie_mask;
+    env->mie = (env->mie & ~sie_mask) | tmp;
+
+    tmp = env->bstvec;
+    env->bstvec = env->stvec;
+    env->stvec = tmp;
+
+    tmp = env->bsscratch;
+    env->bsscratch = env->sscratch;
+    env->sscratch = tmp;
+
+    tmp = env->bsepc;
+    env->bsepc = env->sepc;
+    env->sepc = tmp;
+
+    tmp = env->bscause;
+    env->bscause = env->scause;
+    env->scause = tmp;
+
+    tmp = env->bstval;
+    env->bstval = env->sbadaddr;
+    env->sbadaddr = tmp;
+
+    tmp = env->bsatp;
+    env->bsatp = env->satp;
+    env->satp = tmp;
+
+    tmp = (target_ulong)atomic_read(&env->bsip);
+    tmp = riscv_cpu_update_mip(cpu, (MIP_SSIP | MIP_STIP | MIP_SEIP), tmp);
+    tmp &= MIP_SSIP | MIP_STIP | MIP_SEIP;
+    atomic_set(&env->bsip, tmp);
+}
+
 bool riscv_cpu_virt_enabled(CPURISCVState *env)
 {
     bool tmp;
-- 
2.21.0

[Qemu-devel] [PATCH v1 13/27] target/ricsv: Flush the TLB on virtulisation mode changes

[Alistair Francis]

To ensure our TLB isn't out-of-date we flush it on all virt mode
changes. Unlike priv mode this isn't saved in the mmu_idx as all
guests share V=1. The easiest option is just to flush on all changes.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 5e5029ac0b..6d6fff83a3 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -157,6 +157,11 @@ void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable)
         return;
     }
 
+    /* Flush the TLB on all virt mode changes. */
+    if (((env->virt & VIRT_MODE_MASK) >> VIRT_MODE_SHIFT) != enable) {
+        tlb_flush(CPU(riscv_env_get_cpu(env)));
+    }
+
     env->virt &= ~VIRT_MODE_MASK;
     env->virt |= enable << VIRT_MODE_SHIFT;
 }
-- 
2.21.0

[Qemu-devel] [PATCH v1 14/27] target/riscv: Generate illegal instruction on WFI when V=1

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/op_helper.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c
index 644d0fb35f..e08bb8dd5a 100644
--- a/target/riscv/op_helper.c
+++ b/target/riscv/op_helper.c
@@ -130,9 +130,10 @@ void helper_wfi(CPURISCVState *env)
 {
     CPUState *cs = CPU(riscv_env_get_cpu(env));
 
-    if (env->priv == PRV_S &&
+    if ((env->priv == PRV_S &&
         env->priv_ver >= PRIV_VERSION_1_10_0 &&
-        get_field(env->mstatus, MSTATUS_TW)) {
+        get_field(env->mstatus, MSTATUS_TW)) ||
+        riscv_cpu_virt_enabled(env)) {
         riscv_raise_exception(env, RISCV_EXCP_ILLEGAL_INST, GETPC());
     } else {
         cs->halted = 1;
-- 
2.21.0

[Qemu-devel] [PATCH v1 15/27] riscv: plic: Remove unused interrupt functions

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/riscv/sifive_plic.c         | 12 ------------
 include/hw/riscv/sifive_plic.h |  3 ---
 2 files changed, 15 deletions(-)

diff --git a/hw/riscv/sifive_plic.c b/hw/riscv/sifive_plic.c
index 07a032d93d..1e7e4c8d51 100644
--- a/hw/riscv/sifive_plic.c
+++ b/hw/riscv/sifive_plic.c
@@ -159,18 +159,6 @@ static void sifive_plic_update(SiFivePLICState *plic)
     }
 }
 
-void sifive_plic_raise_irq(SiFivePLICState *plic, uint32_t irq)
-{
-    sifive_plic_set_pending(plic, irq, true);
-    sifive_plic_update(plic);
-}
-
-void sifive_plic_lower_irq(SiFivePLICState *plic, uint32_t irq)
-{
-    sifive_plic_set_pending(plic, irq, false);
-    sifive_plic_update(plic);
-}
-
 static uint32_t sifive_plic_claim(SiFivePLICState *plic, uint32_t addrid)
 {
     int i, j;
diff --git a/include/hw/riscv/sifive_plic.h b/include/hw/riscv/sifive_plic.h
index ce8907f6aa..3b8a623919 100644
--- a/include/hw/riscv/sifive_plic.h
+++ b/include/hw/riscv/sifive_plic.h
@@ -69,9 +69,6 @@ typedef struct SiFivePLICState {
     uint32_t aperture_size;
 } SiFivePLICState;
 
-void sifive_plic_raise_irq(SiFivePLICState *plic, uint32_t irq);
-void sifive_plic_lower_irq(SiFivePLICState *plic, uint32_t irq);
-
 DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
     uint32_t num_sources, uint32_t num_priorities,
     uint32_t priority_base, uint32_t pending_base,
-- 
2.21.0

[Qemu-devel] [PATCH v1 16/27] riscv: plic: Always set sip.SEIP bit for HS

[Alistair Francis]

When the PLIC generates an interrupt ensure we always set it for the SIP
CSR that corresponds to the HS (V=0) register.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 hw/riscv/sifive_plic.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/hw/riscv/sifive_plic.c b/hw/riscv/sifive_plic.c
index 1e7e4c8d51..25da29fa3d 100644
--- a/hw/riscv/sifive_plic.c
+++ b/hw/riscv/sifive_plic.c
@@ -147,7 +147,17 @@ static void sifive_plic_update(SiFivePLICState *plic)
             riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_MEIP, BOOL_TO_MASK(level));
             break;
         case PLICMode_S:
-            riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_SEIP, BOOL_TO_MASK(level));
+            if (riscv_cpu_virt_enabled(env)) {
+                if (level) {
+                    atomic_or(&env->bsip, MIP_SEIP);
+                    g_assert(riscv_cpu_virt_enabled(env));
+                } else {
+                    atomic_and(&env->bsip, ~MIP_SEIP);
+                    g_assert(riscv_cpu_virt_enabled(env));
+                }
+            } else {
+                riscv_cpu_update_mip(RISCV_CPU(cpu), MIP_SEIP, BOOL_TO_MASK(level));
+            }
             break;
         default:
             break;
-- 
2.21.0

[Qemu-devel] [PATCH v1 17/27] target/riscv: Add hypvervisor trap support

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_bits.h   |  4 +--
 target/riscv/cpu_helper.c | 71 +++++++++++++++++++++++++++++++++------
 target/riscv/csr.c        |  4 +--
 3 files changed, 65 insertions(+), 14 deletions(-)

diff --git a/target/riscv/cpu_bits.h b/target/riscv/cpu_bits.h
index 28117bdd32..8966c1bff6 100644
--- a/target/riscv/cpu_bits.h
+++ b/target/riscv/cpu_bits.h
@@ -508,8 +508,8 @@
 #define RISCV_EXCP_STORE_AMO_ADDR_MIS      0x6
 #define RISCV_EXCP_STORE_AMO_ACCESS_FAULT  0x7
 #define RISCV_EXCP_U_ECALL                 0x8
-#define RISCV_EXCP_S_ECALL                 0x9
-#define RISCV_EXCP_H_ECALL                 0xa
+#define RISCV_EXCP_HS_ECALL                0x9
+#define RISCV_EXCP_VS_ECALL                0xa
 #define RISCV_EXCP_M_ECALL                 0xb
 #define RISCV_EXCP_INST_PAGE_FAULT         0xc /* since: priv-1.10.0 */
 #define RISCV_EXCP_LOAD_PAGE_FAULT         0xd /* since: priv-1.10.0 */
diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 6d6fff83a3..d1f73396e4 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -605,6 +605,7 @@ void riscv_cpu_do_interrupt(CPUState *cs)
 
     RISCVCPU *cpu = RISCV_CPU(cs);
     CPURISCVState *env = &cpu->env;
+    target_ulong s;
 
     /* cs->exception is 32-bits wide unlike mcause which is XLEN-bits wide
      * so we mask off the MSB and separate into trap type and cause.
@@ -614,13 +615,6 @@ void riscv_cpu_do_interrupt(CPUState *cs)
     target_ulong deleg = async ? env->mideleg : env->medeleg;
     target_ulong tval = 0;
 
-    static const int ecall_cause_map[] = {
-        [PRV_U] = RISCV_EXCP_U_ECALL,
-        [PRV_S] = RISCV_EXCP_S_ECALL,
-        [PRV_H] = RISCV_EXCP_H_ECALL,
-        [PRV_M] = RISCV_EXCP_M_ECALL
-    };
-
     if (!async) {
         /* set tval to badaddr for traps with address information */
         switch (cause) {
@@ -641,7 +635,16 @@ void riscv_cpu_do_interrupt(CPUState *cs)
         /* ecall is dispatched as one cause so translate based on mode */
         if (cause == RISCV_EXCP_U_ECALL) {
             assert(env->priv <= 3);
-            cause = ecall_cause_map[env->priv];
+
+            if (env->priv == PRV_M) {
+                cause = RISCV_EXCP_M_ECALL;
+            } else if (env->priv == PRV_S && riscv_cpu_virt_enabled(env)) {
+                cause = RISCV_EXCP_VS_ECALL;
+            } else if (env->priv == PRV_S && !riscv_cpu_virt_enabled(env)) {
+                cause = RISCV_EXCP_HS_ECALL;
+            } else if (env->priv == PRV_U) {
+                cause = RISCV_EXCP_U_ECALL;
+            }
         }
     }
 
@@ -651,7 +654,42 @@ void riscv_cpu_do_interrupt(CPUState *cs)
     if (env->priv <= PRV_S &&
             cause < TARGET_LONG_BITS && ((deleg >> cause) & 1)) {
         /* handle the trap in S-mode */
-        target_ulong s = env->mstatus;
+        if (riscv_has_ext(env, RVH)) {
+            target_ulong hdeleg = async ? env->hideleg : env->hedeleg;
+
+            if (riscv_cpu_virt_enabled(env) && ((hdeleg >> cause) & 1) &&
+                !riscv_cpu_force_hs_excep_enabled(env)) {
+                /* Trap to VS mode */
+            } else if (riscv_cpu_virt_enabled(env)) {
+                /* Trap into HS mode, from virt */
+                riscv_cpu_swap_background_regs(env);
+                env->hstatus = set_field(env->hstatus, HSTATUS_SP2V,
+                                         get_field(env->hstatus, HSTATUS_SPV));
+                env->hstatus = set_field(env->hstatus, HSTATUS_SP2P,
+                                         get_field(env->mstatus, SSTATUS_SPP));
+                env->hstatus = set_field(env->hstatus, HSTATUS_SPV,
+                                         riscv_cpu_virt_enabled(env));
+
+                if (riscv_cpu_force_hs_excep_enabled(env)) {
+                    env->hstatus = set_field(env->hstatus, HSTATUS_STL, 1);
+                } else {
+                    env->hstatus = set_field(env->hstatus, HSTATUS_STL, 0);
+                }
+
+                riscv_cpu_set_virt_enabled(env, VIRT_OFF);
+                riscv_cpu_set_force_hs_excep(env, CLEAR_HS_EXCEP);
+            } else {
+                /* Trap into HS mode */
+                env->hstatus = set_field(env->hstatus, HSTATUS_SP2V,
+                                         get_field(env->hstatus, HSTATUS_SPV));
+                env->hstatus = set_field(env->hstatus, HSTATUS_SP2P,
+                                         get_field(env->mstatus, SSTATUS_SPP));
+                env->hstatus = set_field(env->hstatus, HSTATUS_SPV,
+                                         riscv_cpu_virt_enabled(env));
+            }
+        }
+
+        s = env->mstatus;
         s = set_field(s, MSTATUS_SPIE, env->priv_ver >= PRIV_VERSION_1_10_0 ?
             get_field(s, MSTATUS_SIE) : get_field(s, MSTATUS_UIE << env->priv));
         s = set_field(s, MSTATUS_SPP, env->priv);
@@ -665,7 +703,20 @@ void riscv_cpu_do_interrupt(CPUState *cs)
         riscv_cpu_set_mode(env, PRV_S);
     } else {
         /* handle the trap in M-mode */
-        target_ulong s = env->mstatus;
+        if (riscv_has_ext(env, RVH)) {
+            if (riscv_cpu_virt_enabled(env)) {
+                riscv_cpu_swap_background_regs(env);
+            }
+            env->mstatus = set_field(env->mstatus, MSTATUS_MPV,
+                                     riscv_cpu_virt_enabled(env));
+            env->mstatus = set_field(env->mstatus, MSTATUS_MTL,
+                                     riscv_cpu_force_hs_excep_enabled(env));
+
+            /* Trapping to M mode, virt is disabled */
+            riscv_cpu_set_virt_enabled(env, VIRT_OFF);
+        }
+
+        s = env->mstatus;
         s = set_field(s, MSTATUS_MPIE, env->priv_ver >= PRIV_VERSION_1_10_0 ?
             get_field(s, MSTATUS_MIE) : get_field(s, MSTATUS_UIE << env->priv));
         s = set_field(s, MSTATUS_MPP, env->priv);
diff --git a/target/riscv/csr.c b/target/riscv/csr.c
index c55eea44ec..9a1e29f39c 100644
--- a/target/riscv/csr.c
+++ b/target/riscv/csr.c
@@ -239,8 +239,8 @@ static const target_ulong delegable_excps =
     (1ULL << (RISCV_EXCP_STORE_AMO_ADDR_MIS)) |
     (1ULL << (RISCV_EXCP_STORE_AMO_ACCESS_FAULT)) |
     (1ULL << (RISCV_EXCP_U_ECALL)) |
-    (1ULL << (RISCV_EXCP_S_ECALL)) |
-    (1ULL << (RISCV_EXCP_H_ECALL)) |
+    (1ULL << (RISCV_EXCP_VS_ECALL)) |
+    (1ULL << (RISCV_EXCP_HS_ECALL)) |
     (1ULL << (RISCV_EXCP_M_ECALL)) |
     (1ULL << (RISCV_EXCP_INST_PAGE_FAULT)) |
     (1ULL << (RISCV_EXCP_LOAD_PAGE_FAULT)) |
-- 
2.21.0

[Qemu-devel] [PATCH v1 18/27] target/riscv: Add Hypervisor trap return support

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/op_helper.c | 66 ++++++++++++++++++++++++++++++++--------
 1 file changed, 54 insertions(+), 12 deletions(-)

diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c
index e08bb8dd5a..60dcd73fc7 100644
--- a/target/riscv/op_helper.c
+++ b/target/riscv/op_helper.c
@@ -73,6 +73,8 @@ target_ulong helper_csrrc(CPURISCVState *env, target_ulong src,
 
 target_ulong helper_sret(CPURISCVState *env, target_ulong cpu_pc_deb)
 {
+    target_ulong prev_priv, prev_virt, mstatus;
+
     if (!(env->priv >= PRV_S)) {
         riscv_raise_exception(env, RISCV_EXCP_ILLEGAL_INST, GETPC());
     }
@@ -87,16 +89,46 @@ target_ulong helper_sret(CPURISCVState *env, target_ulong cpu_pc_deb)
         riscv_raise_exception(env, RISCV_EXCP_ILLEGAL_INST, GETPC());
     }
 
-    target_ulong mstatus = env->mstatus;
-    target_ulong prev_priv = get_field(mstatus, MSTATUS_SPP);
-    mstatus = set_field(mstatus,
-        env->priv_ver >= PRIV_VERSION_1_10_0 ?
-        MSTATUS_SIE : MSTATUS_UIE << prev_priv,
-        get_field(mstatus, MSTATUS_SPIE));
-    mstatus = set_field(mstatus, MSTATUS_SPIE, 0);
-    mstatus = set_field(mstatus, MSTATUS_SPP, PRV_U);
+    mstatus = env->mstatus;
+
+    if (riscv_has_ext(env, RVH) && !riscv_cpu_virt_enabled(env)) {
+        /* We support Hypervisor extensions and virtulisation is disabled */
+        target_ulong hstatus = env->hstatus;
+
+        prev_priv = get_field(mstatus, MSTATUS_SPP);
+        prev_virt = get_field(hstatus, HSTATUS_SPV);
+
+        hstatus = set_field(hstatus, HSTATUS_SPV,
+                                 get_field(hstatus, HSTATUS_SP2V));
+        mstatus = set_field(mstatus, MSTATUS_SPP,
+                            get_field(hstatus, HSTATUS_SP2P));
+        hstatus = set_field(hstatus, HSTATUS_SP2V, 0);
+        hstatus = set_field(hstatus, HSTATUS_SP2P, 0);
+        mstatus = set_field(mstatus, SSTATUS_SIE,
+                            get_field(mstatus, SSTATUS_SPIE));
+        mstatus = set_field(mstatus, SSTATUS_SPIE, 1);
+
+        env->mstatus = mstatus;
+        env->hstatus = hstatus;
+
+        if (prev_virt == VIRT_ON) {
+            riscv_cpu_swap_background_regs(env);
+        }
+
+        riscv_cpu_set_virt_enabled(env, prev_virt);
+    } else {
+        prev_priv = get_field(mstatus, MSTATUS_SPP);
+
+        mstatus = set_field(mstatus,
+            env->priv_ver >= PRIV_VERSION_1_10_0 ?
+            MSTATUS_SIE : MSTATUS_UIE << prev_priv,
+            get_field(mstatus, MSTATUS_SPIE));
+        mstatus = set_field(mstatus, MSTATUS_SPIE, 0);
+        mstatus = set_field(mstatus, MSTATUS_SPP, PRV_U);
+        env->mstatus = mstatus;
+    }
+
     riscv_cpu_set_mode(env, prev_priv);
-    env->mstatus = mstatus;
 
     return retpc;
 }
@@ -114,14 +146,24 @@ target_ulong helper_mret(CPURISCVState *env, target_ulong cpu_pc_deb)
 
     target_ulong mstatus = env->mstatus;
     target_ulong prev_priv = get_field(mstatus, MSTATUS_MPP);
+    target_ulong prev_virt = get_field(mstatus, MSTATUS_MPV);
     mstatus = set_field(mstatus,
         env->priv_ver >= PRIV_VERSION_1_10_0 ?
         MSTATUS_MIE : MSTATUS_UIE << prev_priv,
         get_field(mstatus, MSTATUS_MPIE));
-    mstatus = set_field(mstatus, MSTATUS_MPIE, 0);
-    mstatus = set_field(mstatus, MSTATUS_MPP, PRV_U);
-    riscv_cpu_set_mode(env, prev_priv);
+    mstatus = set_field(mstatus, MSTATUS_MPIE, 1);
+    mstatus = set_field(mstatus, MSTATUS_MPP, 0);
+    mstatus = set_field(mstatus, MSTATUS_MPV, 0);
     env->mstatus = mstatus;
+    riscv_cpu_set_mode(env, prev_priv);
+
+    if (riscv_has_ext(env, RVH)) {
+        if (prev_virt == VIRT_ON) {
+            riscv_cpu_swap_background_regs(env);
+        }
+
+        riscv_cpu_set_virt_enabled(env, prev_virt);
+    }
 
     return retpc;
 }
-- 
2.21.0

[Qemu-devel] [PATCH v1 19/27] target/riscv: Add hfence instructions

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/insn32.decode                    | 23 ++++++-----
 .../riscv/insn_trans/trans_privileged.inc.c   | 40 +++++++++++++++++++
 2 files changed, 54 insertions(+), 9 deletions(-)

diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode
index 77f794ed70..5bbf6add9c 100644
--- a/target/riscv/insn32.decode
+++ b/target/riscv/insn32.decode
@@ -63,20 +63,25 @@
 @r2_rm   .......   ..... ..... ... ..... ....... %rs1 %rm %rd
 @r2      .......   ..... ..... ... ..... ....... %rs1 %rd
 
+@hfence_gvma ....... ..... .....   ... ..... ....... %rs2 %rs1
+@hfence_bvma ....... ..... .....   ... ..... ....... %rs2 %rs1
+
 @sfence_vma ....... ..... .....   ... ..... ....... %rs2 %rs1
 @sfence_vm  ....... ..... .....   ... ..... ....... %rs1
 
 
 # *** Privileged Instructions ***
-ecall      000000000000     00000 000 00000 1110011
-ebreak     000000000001     00000 000 00000 1110011
-uret       0000000    00010 00000 000 00000 1110011
-sret       0001000    00010 00000 000 00000 1110011
-hret       0010000    00010 00000 000 00000 1110011
-mret       0011000    00010 00000 000 00000 1110011
-wfi        0001000    00101 00000 000 00000 1110011
-sfence_vma 0001001    ..... ..... 000 00000 1110011 @sfence_vma
-sfence_vm  0001000    00100 ..... 000 00000 1110011 @sfence_vm
+ecall       000000000000     00000 000 00000 1110011
+ebreak      000000000001     00000 000 00000 1110011
+uret        0000000    00010 00000 000 00000 1110011
+sret        0001000    00010 00000 000 00000 1110011
+hret        0010000    00010 00000 000 00000 1110011
+mret        0011000    00010 00000 000 00000 1110011
+wfi         0001000    00101 00000 000 00000 1110011
+hfence_gvma 1010001    ..... ..... 000 00000 1110011 @hfence_gvma
+hfence_bvma 0010001    ..... ..... 000 00000 1110011 @hfence_bvma
+sfence_vma  0001001    ..... ..... 000 00000 1110011 @sfence_vma
+sfence_vm   0001000    00100 ..... 000 00000 1110011 @sfence_vm
 
 # *** RV32I Base Instruction Set ***
 lui      ....................       ..... 0110111 @u
diff --git a/target/riscv/insn_trans/trans_privileged.inc.c b/target/riscv/insn_trans/trans_privileged.inc.c
index 664d6ba3f2..ac953ad30d 100644
--- a/target/riscv/insn_trans/trans_privileged.inc.c
+++ b/target/riscv/insn_trans/trans_privileged.inc.c
@@ -108,3 +108,43 @@ static bool trans_sfence_vm(DisasContext *ctx, arg_sfence_vm *a)
 #endif
     return false;
 }
+
+static bool trans_hfence_gvma(DisasContext *ctx, arg_sfence_vma *a)
+{
+#ifndef CONFIG_USER_ONLY
+    if (ctx->priv_ver == PRIV_VERSION_1_10_0 &&
+        has_ext(ctx, RVH)) {
+        /* Hpervisor extensions exist */
+        /*
+         * if (env->priv == PRV_M ||
+         *   (env->priv == PRV_S &&
+         *    !riscv_cpu_virt_enabled(env) &&
+         *    get_field(ctx->mstatus_fs, MSTATUS_TVM))) {
+         */
+            gen_helper_tlb_flush(cpu_env);
+            return true;
+        /* } */
+    }
+#endif
+    return false;
+}
+
+static bool trans_hfence_bvma(DisasContext *ctx, arg_sfence_vma *a)
+{
+#ifndef CONFIG_USER_ONLY
+    if (ctx->priv_ver == PRIV_VERSION_1_10_0 &&
+        has_ext(ctx, RVH)) {
+        /* Hpervisor extensions exist */
+        /*
+         * if (env->priv == PRV_M ||
+         *   (env->priv == PRV_S &&
+         *    !riscv_cpu_virt_enabled(env) &&
+         *    get_field(ctx->mstatus_fs, MSTATUS_TVM))) {
+         */
+            gen_helper_tlb_flush(cpu_env);
+            return true;
+        /* } */
+    }
+#endif
+    return false;
+}
-- 
2.21.0

[Qemu-devel] [PATCH v1 20/27] target/riscv: Disable guest FP support based on backgrond status

[Alistair Francis]

When the Hypervisor extension is in use we only enable floating point
support when both status and bsstatus have enabled floating point
support.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index d1f73396e4..b009049cc4 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -193,6 +193,9 @@ void riscv_cpu_set_force_hs_excep(CPURISCVState *env, bool enable)
 bool riscv_cpu_fp_enabled(CPURISCVState *env)
 {
     if (env->mstatus & MSTATUS_FS) {
+        if (riscv_cpu_virt_enabled(env) && !(env->bsstatus & MSTATUS_FS)) {
+            return false;
+        }
         return true;
     }
 
-- 
2.21.0

[Qemu-devel] [PATCH v1 21/27] target/riscv: Mark both sstatus and bsstatus as dirty

[Alistair Francis]

Mark both sstatus and bsstatus as dirty (3).

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/translate.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/target/riscv/translate.c b/target/riscv/translate.c
index 313c27b700..1c3cd1c94b 100644
--- a/target/riscv/translate.c
+++ b/target/riscv/translate.c
@@ -44,6 +44,7 @@ typedef struct DisasContext {
     /* pc_succ_insn points to the instruction following base.pc_next */
     target_ulong pc_succ_insn;
     target_ulong priv_ver;
+    bool virt_enabled;
     uint32_t opcode;
     uint32_t mstatus_fs;
     uint32_t misa;
@@ -397,6 +398,12 @@ static void mark_fs_dirty(DisasContext *ctx)
     tcg_gen_ld_tl(tmp, cpu_env, offsetof(CPURISCVState, mstatus));
     tcg_gen_ori_tl(tmp, tmp, MSTATUS_FS);
     tcg_gen_st_tl(tmp, cpu_env, offsetof(CPURISCVState, mstatus));
+
+    if (ctx->virt_enabled) {
+        tcg_gen_ld_tl(tmp, cpu_env, offsetof(CPURISCVState, bsstatus));
+        tcg_gen_ori_tl(tmp, tmp, MSTATUS_FS);
+        tcg_gen_st_tl(tmp, cpu_env, offsetof(CPURISCVState, bsstatus));
+    }
     tcg_temp_free(tmp);
 }
 #else
@@ -757,6 +764,11 @@ static void riscv_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
     ctx->mem_idx = ctx->base.tb->flags & TB_FLAGS_MMU_MASK;
     ctx->mstatus_fs = ctx->base.tb->flags & TB_FLAGS_MSTATUS_FS;
     ctx->priv_ver = env->priv_ver;
+#if !defined(CONFIG_USER_ONLY)
+    ctx->virt_enabled = riscv_cpu_virt_enabled(env);
+#else
+    ctx->virt_enabled = false;
+#endif
     ctx->misa = env->misa;
     ctx->frm = -1;  /* unknown rounding mode */
 }
-- 
2.21.0

[Qemu-devel] [PATCH v1 22/27] target/riscv: Respect MPRV and SPRV for floating point ops

[Alistair Francis]

Respect the contents of MSTATUS.MPRV and HSTATUS.SPRV when performing
floating point operations when V=0.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/translate.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/target/riscv/translate.c b/target/riscv/translate.c
index 1c3cd1c94b..d4fa7d056d 100644
--- a/target/riscv/translate.c
+++ b/target/riscv/translate.c
@@ -765,7 +765,21 @@ static void riscv_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
     ctx->mstatus_fs = ctx->base.tb->flags & TB_FLAGS_MSTATUS_FS;
     ctx->priv_ver = env->priv_ver;
 #if !defined(CONFIG_USER_ONLY)
-    ctx->virt_enabled = riscv_cpu_virt_enabled(env);
+    if (riscv_has_ext(env, RVH)) {
+        ctx->virt_enabled = riscv_cpu_virt_enabled(env);
+        if (env->priv_ver == PRV_M &&
+            get_field(env->mstatus, MSTATUS_MPRV) &&
+            get_field(env->mstatus, MSTATUS_MPV)) {
+            ctx->virt_enabled = true;
+        } else if (env->priv == PRV_S &&
+                   !riscv_cpu_virt_enabled(env) &&
+                   get_field(env->hstatus, HSTATUS_SPRV) &&
+                   get_field(env->hstatus, HSTATUS_SPV)) {
+            ctx->virt_enabled = true;
+        }
+    } else {
+        ctx->virt_enabled = false;
+    }
 #else
     ctx->virt_enabled = false;
 #endif
-- 
2.21.0

[Qemu-devel] [PATCH v1 23/27] target/riscv: Allow specifying MMU stage

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 35 +++++++++++++++++++++++++++--------
 1 file changed, 27 insertions(+), 8 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index b009049cc4..6cef78a2c7 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -274,10 +274,19 @@ void riscv_cpu_set_mode(CPURISCVState *env, target_ulong newpriv)
  *
  * Adapted from Spike's mmu_t::translate and mmu_t::walk
  *
+ * @env: CPURISCVState
+ * @physical: This will be set to the calculated physical address
+ * @prot: The returned protection attributes
+ * @addr: The virtual address to be translated
+ * @access_type: The type of MMU access
+ * @mmu_idx: Indicates current privilege level
+ * @first_stage: Are we in first stage translation?
+ *               Second stage is used for hypervisor guest translation
  */
 static int get_physical_address(CPURISCVState *env, hwaddr *physical,
                                 int *prot, target_ulong addr,
-                                int access_type, int mmu_idx)
+                                int access_type, int mmu_idx,
+                                bool first_stage)
 {
     /* NOTE: the env->pc value visible here will not be
      * correct, but the value visible to the exception handler
@@ -468,12 +477,20 @@ restart:
 }
 
 static void raise_mmu_exception(CPURISCVState *env, target_ulong address,
-                                MMUAccessType access_type)
+                                MMUAccessType access_type, bool first_stage)
 {
     CPUState *cs = CPU(riscv_env_get_cpu(env));
-    int page_fault_exceptions =
-        (env->priv_ver >= PRIV_VERSION_1_10_0) &&
-        get_field(env->satp, SATP_MODE) != VM_1_10_MBARE;
+    int page_fault_exceptions;
+    if (first_stage) {
+        page_fault_exceptions =
+            (env->priv_ver >= PRIV_VERSION_1_10_0) &&
+            get_field(env->satp, SATP_MODE) != VM_1_10_MBARE;
+            riscv_cpu_set_force_hs_excep(env, CLEAR_HS_EXCEP);
+    } else {
+        page_fault_exceptions =
+            get_field(env->hgatp, HGATP_MODE) != VM_1_10_MBARE;
+            riscv_cpu_set_force_hs_excep(env, FORCE_HS_EXCEP);
+    }
     switch (access_type) {
     case MMU_INST_FETCH:
         cs->exception_index = page_fault_exceptions ?
@@ -500,7 +517,8 @@ hwaddr riscv_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)
     int prot;
     int mmu_idx = cpu_mmu_index(&cpu->env, false);
 
-    if (get_physical_address(&cpu->env, &phys_addr, &prot, addr, 0, mmu_idx)) {
+    if (get_physical_address(&cpu->env, &phys_addr, &prot, addr, 0, mmu_idx,
+                             true)) {
         return -1;
     }
     return phys_addr;
@@ -560,7 +578,8 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
     qemu_log_mask(CPU_LOG_MMU, "%s ad %" VADDR_PRIx " rw %d mmu_idx %d\n",
                   __func__, address, access_type, mmu_idx);
 
-    ret = get_physical_address(env, &pa, &prot, address, access_type, mmu_idx);
+    ret = get_physical_address(env, &pa, &prot, address, access_type, mmu_idx,
+                               true);
 
     qemu_log_mask(CPU_LOG_MMU,
                   "%s address=%" VADDR_PRIx " ret %d physical " TARGET_FMT_plx
@@ -577,7 +596,7 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
     } else if (probe) {
         return false;
     } else {
-        raise_mmu_exception(env, address, access_type);
+        raise_mmu_exception(env, address, access_type, true);
         riscv_raise_exception(env, cs->exception_index, retaddr);
     }
 #else
-- 
2.21.0

[Qemu-devel] [PATCH v1 24/27] target/riscv: Allow specifying number of MMU stages

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 6cef78a2c7..6ff4272da2 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -286,7 +286,7 @@ void riscv_cpu_set_mode(CPURISCVState *env, target_ulong newpriv)
 static int get_physical_address(CPURISCVState *env, hwaddr *physical,
                                 int *prot, target_ulong addr,
                                 int access_type, int mmu_idx,
-                                bool first_stage)
+                                bool first_stage, bool two_stage)
 {
     /* NOTE: the env->pc value visible here will not be
      * correct, but the value visible to the exception handler
@@ -518,9 +518,10 @@ hwaddr riscv_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)
     int mmu_idx = cpu_mmu_index(&cpu->env, false);
 
     if (get_physical_address(&cpu->env, &phys_addr, &prot, addr, 0, mmu_idx,
-                             true)) {
+                             true, false)) {
         return -1;
     }
+
     return phys_addr;
 }
 
@@ -579,7 +580,7 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
                   __func__, address, access_type, mmu_idx);
 
     ret = get_physical_address(env, &pa, &prot, address, access_type, mmu_idx,
-                               true);
+                               true, false);
 
     qemu_log_mask(CPU_LOG_MMU,
                   "%s address=%" VADDR_PRIx " ret %d physical " TARGET_FMT_plx
-- 
2.21.0

[Qemu-devel] [PATCH v1 25/27] target/riscv: Implement second stage MMU

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 87 +++++++++++++++++++++++++++++++++++----
 1 file changed, 78 insertions(+), 9 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 6ff4272da2..ece3eadf66 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -293,13 +293,40 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,
      * (riscv_cpu_do_interrupt) is correct */
 
     int mode = mmu_idx;
-
+    bool use_background = false;
+
+    /*
+     * Check if we should use the background registers for the two
+     * stage translation. We don't need to check if we actually need
+     * two stage translation as that happened before this function
+     * was called. Background registers will be used if the guest has
+     * forced a two stage translation to be on (in HS or M mode).
+     */
     if (mode == PRV_M && access_type != MMU_INST_FETCH) {
         if (get_field(env->mstatus, MSTATUS_MPRV)) {
             mode = get_field(env->mstatus, MSTATUS_MPP);
+
+            if (riscv_has_ext(env, RVH) &&
+                get_field(env->mstatus, MSTATUS_MPV)) {
+                use_background = true;
+            }
+        }
+    }
+
+    if (mode == PRV_S && access_type != MMU_INST_FETCH &&
+        riscv_has_ext(env, RVH) && !riscv_cpu_virt_enabled(env)) {
+        if (get_field(env->hstatus, HSTATUS_SPRV)) {
+            mode = get_field(env->mstatus, SSTATUS_SPP);
+            use_background = true;
         }
     }
 
+    if (first_stage == false) {
+        /* We are in stage 2 translation, this is similar to stage 1. */
+        /* Stage 2 is always taken as U-mode */
+        mode = PRV_U;
+    }
+
     if (mode == PRV_M || !riscv_feature(env, RISCV_FEATURE_MMU)) {
         *physical = addr;
         *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC;
@@ -309,13 +336,30 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,
     *prot = 0;
 
     target_ulong base;
-    int levels, ptidxbits, ptesize, vm, sum;
-    int mxr = get_field(env->mstatus, MSTATUS_MXR);
+    int levels, ptidxbits, ptesize, vm, sum, mxr, widened;
+
+    if (first_stage == true) {
+        mxr = get_field(env->mstatus, MSTATUS_MXR);
+    } else {
+        mxr = get_field(env->bsstatus, MSTATUS_MXR);
+    }
 
     if (env->priv_ver >= PRIV_VERSION_1_10_0) {
-        base = get_field(env->satp, SATP_PPN) << PGSHIFT;
+        if (first_stage == true) {
+            if (use_background) {
+                base = get_field(env->bsatp, SATP_PPN) << PGSHIFT;
+                vm = get_field(env->bsatp, SATP_MODE);
+            } else {
+                base = get_field(env->satp, SATP_PPN) << PGSHIFT;
+                vm = get_field(env->satp, SATP_MODE);
+            }
+            widened = 0;
+        } else {
+            base = get_field(env->hgatp, HGATP_PPN) << PGSHIFT;
+            vm = get_field(env->hgatp, HGATP_MODE);
+            widened = 2;
+        }
         sum = get_field(env->mstatus, MSTATUS_SUM);
-        vm = get_field(env->satp, SATP_MODE);
         switch (vm) {
         case VM_1_10_SV32:
           levels = 2; ptidxbits = 10; ptesize = 4; break;
@@ -333,6 +377,7 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,
           g_assert_not_reached();
         }
     } else {
+        widened = 0;
         base = env->sptbr << PGSHIFT;
         sum = !get_field(env->mstatus, MSTATUS_PUM);
         vm = get_field(env->mstatus, MSTATUS_VM);
@@ -353,7 +398,7 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,
     }
 
     CPUState *cs = CPU(riscv_env_get_cpu(env));
-    int va_bits = PGSHIFT + levels * ptidxbits;
+    int va_bits = PGSHIFT + levels * ptidxbits + widened;
     target_ulong mask = (1L << (TARGET_LONG_BITS - (va_bits - 1))) - 1;
     target_ulong masked_msbs = (addr >> (va_bits - 1)) & mask;
     if (masked_msbs != 0 && masked_msbs != mask) {
@@ -367,11 +412,30 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,
 restart:
 #endif
     for (i = 0; i < levels; i++, ptshift -= ptidxbits) {
-        target_ulong idx = (addr >> (PGSHIFT + ptshift)) &
+        target_ulong idx;
+        if (i == 0) {
+            idx = (addr >> (PGSHIFT + ptshift)) &
+                           ((1 << (ptidxbits + widened)) - 1);
+        } else {
+            idx = (addr >> (PGSHIFT + ptshift)) &
                            ((1 << ptidxbits) - 1);
+        }
 
         /* check that physical address of PTE is legal */
-        target_ulong pte_addr = base + idx * ptesize;
+        target_ulong pte_addr;
+
+        if (two_stage && first_stage) {
+            hwaddr vbase;
+
+            /* Do the second stage translation on the base PTE address. */
+            get_physical_address(env, &vbase, prot, base, access_type,
+                                 mmu_idx, false, true);
+
+            pte_addr = vbase + idx * ptesize;
+        } else {
+            pte_addr = base + idx * ptesize;
+        }
+
 #if defined(TARGET_RISCV32)
         target_ulong pte = ldl_phys(cs->as, pte_addr);
 #elif defined(TARGET_RISCV64)
@@ -457,7 +521,12 @@ restart:
             /* for superpage mappings, make a fake leaf PTE for the TLB's
                benefit. */
             target_ulong vpn = addr >> PGSHIFT;
-            *physical = (ppn | (vpn & ((1L << ptshift) - 1))) << PGSHIFT;
+            if (i == 0) {
+                *physical = (ppn | (vpn & ((1L << (ptshift + widened)) - 1))) <<
+                             PGSHIFT;
+            } else {
+                *physical = (ppn | (vpn & ((1L << ptshift) - 1))) << PGSHIFT;
+            }
 
             /* set permissions on the TLB entry */
             if ((pte & PTE_R) || ((pte & PTE_X) && mxr)) {
-- 
2.21.0

[Qemu-devel] [PATCH v1 26/27] target/riscv: Call the second stage MMU in virtualisation mode

[Alistair Francis]

The qemu_log_mask(CPU_LOG_MMU,... calls trigger false positive
checkpatch errors which are being ignored.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu_helper.c | 118 ++++++++++++++++++++++++++++++++------
 1 file changed, 99 insertions(+), 19 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index ece3eadf66..023368c2ce 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -582,15 +582,23 @@ static void raise_mmu_exception(CPURISCVState *env, target_ulong address,
 hwaddr riscv_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)
 {
     RISCVCPU *cpu = RISCV_CPU(cs);
+    CPURISCVState *env = &cpu->env;
     hwaddr phys_addr;
     int prot;
     int mmu_idx = cpu_mmu_index(&cpu->env, false);
 
-    if (get_physical_address(&cpu->env, &phys_addr, &prot, addr, 0, mmu_idx,
-                             true, false)) {
+    if (get_physical_address(env, &phys_addr, &prot, addr, 0, mmu_idx,
+                             true, riscv_cpu_virt_enabled(env))) {
         return -1;
     }
 
+    if (riscv_cpu_virt_enabled(env)) {
+        if (get_physical_address(env, &phys_addr, &prot, phys_addr,
+                                 0, mmu_idx, false, true)) {
+            return -1;
+        }
+    }
+
     return phys_addr;
 }
 
@@ -641,34 +649,106 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
 #ifndef CONFIG_USER_ONLY
     RISCVCPU *cpu = RISCV_CPU(cs);
     CPURISCVState *env = &cpu->env;
+    vaddr im_address;
     hwaddr pa = 0;
     int prot;
+    bool m_mode_two_stage = false;
+    bool hs_mode_two_stage = false;
     int ret = TRANSLATE_FAIL;
 
     qemu_log_mask(CPU_LOG_MMU, "%s ad %" VADDR_PRIx " rw %d mmu_idx %d\n",
                   __func__, address, access_type, mmu_idx);
 
-    ret = get_physical_address(env, &pa, &prot, address, access_type, mmu_idx,
-                               true, false);
+    /*
+     * Determine if we are in M mode and MPRV is set or in HS mode and SPRV is
+     * set and we want to access a virtulisation address.
+     */
+    if (riscv_has_ext(env, RVH)) {
+        m_mode_two_stage = env->priv == PRV_M &&
+                           access_type != MMU_INST_FETCH &&
+                           get_field(env->mstatus, MSTATUS_MPRV) &&
+                           get_field(env->mstatus, MSTATUS_MPV);
+
+        hs_mode_two_stage = env->priv == PRV_S &&
+                            !riscv_cpu_virt_enabled(env) &&
+                            access_type != MMU_INST_FETCH &&
+                            get_field(env->hstatus, HSTATUS_SPRV) &&
+                            get_field(env->hstatus, HSTATUS_SPV);
+    }
+
+    if (riscv_cpu_virt_enabled(env) || m_mode_two_stage || hs_mode_two_stage) {
+        /* Two stage lookup */
+        ret = get_physical_address(env, &pa, &prot, address, access_type,
+                                   mmu_idx, true, true);
 
-    qemu_log_mask(CPU_LOG_MMU,
-                  "%s address=%" VADDR_PRIx " ret %d physical " TARGET_FMT_plx
-                  " prot %d\n", __func__, address, ret, pa, prot);
+        qemu_log_mask(CPU_LOG_MMU,
+                      "%s 1st-stage address=%" VADDR_PRIx " ret %d physical "
+                      TARGET_FMT_plx " prot %d\n",
+                      __func__, address, ret, pa, prot);
 
-    if (riscv_feature(env, RISCV_FEATURE_PMP) &&
-        !pmp_hart_has_privs(env, pa, TARGET_PAGE_SIZE, 1 << access_type)) {
-        ret = TRANSLATE_FAIL;
-    }
-    if (ret == TRANSLATE_SUCCESS) {
-        tlb_set_page(cs, address & TARGET_PAGE_MASK, pa & TARGET_PAGE_MASK,
-                     prot, mmu_idx, TARGET_PAGE_SIZE);
-        return true;
-    } else if (probe) {
-        return false;
+        if (ret == TRANSLATE_FAIL) {
+            if (!probe) {
+                raise_mmu_exception(env, address, access_type, true);
+                riscv_raise_exception(env, cs->exception_index, retaddr);
+            }
+            return ret;
+        }
+
+        /* Second stage lookup */
+        im_address = pa;
+
+        ret = get_physical_address(env, &pa, &prot, im_address, access_type, mmu_idx,
+                                   false, true);
+
+        qemu_log_mask(CPU_LOG_MMU,
+                "%s 2nd-stage address=%" VADDR_PRIx " ret %d physical "
+                TARGET_FMT_plx " prot %d\n",
+                __func__, im_address, ret, pa, prot);
+
+        if (riscv_feature(env, RISCV_FEATURE_PMP) &&
+            !pmp_hart_has_privs(env, pa, TARGET_PAGE_SIZE, 1 << access_type)) {
+            ret = TRANSLATE_FAIL;
+        }
+
+        if (ret == TRANSLATE_FAIL) {
+            /*
+             * Guest physical address translation failed, this is a HS
+             * level exception
+             */
+            if (!probe) {
+                raise_mmu_exception(env, im_address | (address & (TARGET_PAGE_SIZE - 1)), access_type, false);
+                riscv_raise_exception(env, cs->exception_index, retaddr);
+            }
+            return ret;
+        }
     } else {
-        raise_mmu_exception(env, address, access_type, true);
-        riscv_raise_exception(env, cs->exception_index, retaddr);
+        /* Single stage lookup */
+        ret = get_physical_address(env, &pa, &prot, address, access_type,
+                                   mmu_idx, true, false);
+
+        qemu_log_mask(CPU_LOG_MMU,
+                      "%s address=%" VADDR_PRIx " ret %d physical "
+                      TARGET_FMT_plx " prot %d\n",
+                      __func__, address, ret, pa, prot);
+
+        if (riscv_feature(env, RISCV_FEATURE_PMP) &&
+            !pmp_hart_has_privs(env, pa, TARGET_PAGE_SIZE, 1 << access_type)) {
+            ret = TRANSLATE_FAIL;
+        }
+
+        if (ret == TRANSLATE_FAIL) {
+            if (!probe) {
+                raise_mmu_exception(env, address, access_type, true);
+                riscv_raise_exception(env, cs->exception_index, retaddr);
+            }
+            return ret;
+        }
     }
+
+    tlb_set_page(cs, address & TARGET_PAGE_MASK, pa & TARGET_PAGE_MASK,
+                 prot, mmu_idx, TARGET_PAGE_SIZE);
+    return true;
+
 #else
     switch (access_type) {
     case MMU_INST_FETCH:
-- 
2.21.0

[Qemu-devel] [PATCH v1 27/27] target/riscv: Allow enabling the Hypervisor extension

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 target/riscv/cpu.c | 4 ++++
 target/riscv/cpu.h | 1 +
 2 files changed, 5 insertions(+)

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index 6111f0f0bc..38583e7a6e 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -436,6 +436,9 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)
         if (cpu->cfg.ext_u) {
             target_misa |= RVU;
         }
+        if (cpu->cfg.ext_h) {
+            target_misa |= RVH;
+        }
 
         set_misa(env, RVXLEN | target_misa);
     }
@@ -472,6 +475,7 @@ static Property riscv_cpu_properties[] = {
     DEFINE_PROP_BOOL("c", RISCVCPU, cfg.ext_c, true),
     DEFINE_PROP_BOOL("s", RISCVCPU, cfg.ext_s, true),
     DEFINE_PROP_BOOL("u", RISCVCPU, cfg.ext_u, true),
+    DEFINE_PROP_BOOL("h", RISCVCPU, cfg.ext_h, false),
     DEFINE_PROP_STRING("priv_spec", RISCVCPU, cfg.priv_spec),
     DEFINE_PROP_STRING("user_spec", RISCVCPU, cfg.user_spec),
     DEFINE_PROP_BOOL("mmu", RISCVCPU, cfg.mmu, true),
diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
index 5b3b32dbbc..342e628379 100644
--- a/target/riscv/cpu.h
+++ b/target/riscv/cpu.h
@@ -259,6 +259,7 @@ typedef struct RISCVCPU {
         bool ext_c;
         bool ext_s;
         bool ext_u;
+        bool ext_h;
 
         char *priv_spec;
         char *user_spec;
-- 
2.21.0

Re: [Qemu-devel] [PATCH v1 00/27] Add RISC-V Hypervisor Extension

[Chih-Min Chao]

On Sat, Jun 8, 2019 at 6:03 AM Alistair Francis <alistair.francis@wdc.com>
wrote:

> This patch series adds the RISC-V Hypervisor extension 0.3. This is the
> latest draft spec of the Hypervisor extension.
>
> The Hypervisor extension is disabled by default, so this series should
> result in no changes to anyone using QEMU unless they enable the
> extension. The extention can be enabled with the -cpu property (see
> below).
>
> At the moment the spec does not include information about the mstatush
> register. As it is not in the spec I haven't added it to QEMU. This
> means the extension won't work correctly for 32-bit guests. This should
> be a small fix to add the CSR once the spec is updated.
>
> All testing of this implementation has been done by using the baremetal
> Xvisor Hypervisor. We are able to run two Linux guests (that's all I
> have tried) as guests.
>
> At the moment this spec is in a draft state and is subject to change. As
> QEMU is extreamly useful in early bring up I think it makes sense for
> QEMU to support non-frozen extensions. I would like to decide with this
> series how QEMU will handle all future non-frozen extensions. That is a
> standard way that QEMU users can test future RISC-V extensions while
> still understanding things will change. One idea is just to disable it by
> default, another option is to maybe use the Kconfig to make it a compile
> time option which developers can use. Should we also display a warning
> when running non-frozen extensions?
>
> Thanks to Anup for doing the initial port of Xvisor. The port is avaliable
> here:
> https://github.com/avpatel/xvisor-next and will run on QEMU.
>
> Also thanks to Atish for implementing the SBI call support in Xvisor and
> for lots of help debugging.
>
> To run this yourself:
>  1. Apply this patch series to QEMU. The latest branch can be found here:
>
> https://github.com/alistair23/qemu/tree/mainline/alistair/riscv-hyp-work.next
>  2. Get the version of OpenSBI that supports the H extension. This can
>     be found here:
>       https://github.com/riscv/opensbi/tree/hyp_ext_changes_v1
>  3. Build the next release of Xvisor. It is available here:
>       https://github.com/avpatel/xvisor-next
>  4. Make sure you build the Xvisor tests, see here for details:
>
> https://github.com/avpatel/xvisor-next/tree/master/tests/riscv/virt64/linux
>  5. Run QEMU:
>      ./riscv64-softmmu/qemu-system-riscv64 -nographic \
>        -machine virt -cpu rv64,h=true\
>        -serial mon:stdio -serial null -m 4G \
>        -device loader,file=vmm.bin,addr=0x80200000 \
>        -kernel fw_jump.elf \
>        -initrd vmm-disk-linux.img \
>        -append "vmm.console=uart@10000000 vmm.bootcmd=\"vfs mount initrd
> /;vfs run /boot.xscript;vfs cat /system/banner.txt\""
>
>    Once you get to the prompt you can start the geust by running:
>      guest kick guest0
>    You can then bind to the serial port using:
>      vserial bind guest0/uart0
>    Then you can start Linux using:
>      autoexec
>
>  This was all tested with the mainline 5.1 kernel. I don't know if it
>  will work on older kernels.
>
> So far all of the QEMU work has been tested on Xvisor.
>
> Known Issues/TODO:
>  - Add mstatush to support 32-bit Hypervisors
>  - Fix the random hang that sometimes appears when running a Hypervisor
> guest (~5%)
>
> There is also on going work from Anup to port KVM.
> We have code complete implementation of RISC-V KVM kernel module and
> RISC-V KVMTOOL. Currently, we are debugging KVM on QEMU and we will
> send-out RFC PATCHES for KVM in June/July 2019.
> The KVM RISC-V kernel module is available in riscv_kvm_v1
> branch at: https://github.com/avpatel/linux.git
> The KVMTOOL RISC-V port is available in riscv_v1 branch of
> https://github.com/avpatel/kvmtool.git
>
> There is very early work on a Xen port as well which is avaliable here:
> https://github.com/alistair23/xen/tree/alistair/riscv-port
>
> Changes from RFC:
>  - Rebase on latest master
>  - Add floating point changes from Hypervisor extension
>
> Alistair Francis (27):
>   target/riscv: Don't set write permissions on dirty PTEs
>   target/riscv: Add the Hypervisor extension
>   target/riscv: Add the virtulisation mode
>   target/riscv: Add the force HS exception mode
>   target/riscv: Add the Hypervisor CSRs to CPUState
>   target/riscv: Dump Hypervisor registers if enabled
>   target/riscv: Remove strict perm checking for CSR R/W
>   target/riscv: Create function to test if FP is enabled
>   target/riscv: Add support for background interrupt setting
>   target/riscv: Add Hypervisor CSR access functions
>   target/riscv: Add background CSRs accesses
>   target/riscv: Add background register swapping function
>   target/ricsv: Flush the TLB on virtulisation mode changes
>   target/riscv: Generate illegal instruction on WFI when V=1
>   riscv: plic: Remove unused interrupt functions
>   riscv: plic: Always set sip.SEIP bit for HS
>   target/riscv: Add hypvervisor trap support
>   target/riscv: Add Hypervisor trap return support
>   target/riscv: Add hfence instructions
>   target/riscv: Disable guest FP support based on backgrond status
>   target/riscv: Mark both sstatus and bsstatus as dirty
>   target/riscv: Respect MPRV and SPRV for floating point ops
>   target/riscv: Allow specifying MMU stage
>   target/riscv: Allow specifying number of MMU stages
>   target/riscv: Implement second stage MMU
>   target/riscv: Call the second stage MMU in virtualisation mode
>   target/riscv: Allow enabling the Hypervisor extension
>
>  hw/riscv/sifive_plic.c                        |  24 +-
>  include/hw/riscv/sifive_plic.h                |   3 -
>  target/riscv/cpu.c                            |  31 ++
>  target/riscv/cpu.h                            |  32 +-
>  target/riscv/cpu_bits.h                       |  32 +-
>  target/riscv/cpu_helper.c                     | 443 ++++++++++++++++--
>  target/riscv/csr.c                            | 216 ++++++++-
>  target/riscv/insn32.decode                    |  23 +-
>  .../riscv/insn_trans/trans_privileged.inc.c   |  40 ++
>  target/riscv/op_helper.c                      |  71 ++-
>  target/riscv/translate.c                      |  26 +
>  11 files changed, 839 insertions(+), 102 deletions(-)
>
> --
> 2.21.0
>
>
  tested with Linux kernel v5.2 and related opensbi/xvisor branches
described above.

 Tested-by: Chih-Min Chao <chihmin.chao@sifive.com>

Re: [Qemu-devel] [PATCH v1 00/27] Add RISC-V Hypervisor Extension

[Peter Maydell]

On Fri, 7 Jun 2019 at 23:03, Alistair Francis <alistair.francis@wdc.com> wrote:
> At the moment this spec is in a draft state and is subject to change. As
> QEMU is extreamly useful in early bring up I think it makes sense for
> QEMU to support non-frozen extensions. I would like to decide with this
> series how QEMU will handle all future non-frozen extensions. That is a
> standard way that QEMU users can test future RISC-V extensions while
> still understanding things will change. One idea is just to disable it by
> default, another option is to maybe use the Kconfig to make it a compile
> time option which developers can use. Should we also display a warning
> when running non-frozen extensions?

We had an instance of this for Arm (though in fact the
relevant patches to QEMU didn't end up getting into master
before the spec was finalized in the end). My suggestion
would be at minimum:
 * by default non-frozen extensions should not be provided
 * they should be enabled via a command line option (cpu
   property) whose name starts with "x-", which is our standard
   way of flagging properties that are experimental and subject
   to change or removal in future QEMU versions

That way end-users know they're doing something non-standard
that won't necessarily be supported in future by newer versions
of QEMU, and if people copy recipes/commandlines/random guest
images off old blog posts there'll be a hint that there's a
reason why they don't work on newer QEMU that adheres to the
final spec.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v1 00/27] Add RISC-V Hypervisor Extension

[Alistair Francis]

On Mon, Jul 15, 2019 at 4:50 AM Chih-Min Chao <chihmin.chao@sifive.com> wrote:
>
>
>
> On Sat, Jun 8, 2019 at 6:03 AM Alistair Francis <alistair.francis@wdc.com> wrote:
>>
>> This patch series adds the RISC-V Hypervisor extension 0.3. This is the
>> latest draft spec of the Hypervisor extension.
>>
>> The Hypervisor extension is disabled by default, so this series should
>> result in no changes to anyone using QEMU unless they enable the
>> extension. The extention can be enabled with the -cpu property (see
>> below).
>>
>> At the moment the spec does not include information about the mstatush
>> register. As it is not in the spec I haven't added it to QEMU. This
>> means the extension won't work correctly for 32-bit guests. This should
>> be a small fix to add the CSR once the spec is updated.
>>
>> All testing of this implementation has been done by using the baremetal
>> Xvisor Hypervisor. We are able to run two Linux guests (that's all I
>> have tried) as guests.
>>
>> At the moment this spec is in a draft state and is subject to change. As
>> QEMU is extreamly useful in early bring up I think it makes sense for
>> QEMU to support non-frozen extensions. I would like to decide with this
>> series how QEMU will handle all future non-frozen extensions. That is a
>> standard way that QEMU users can test future RISC-V extensions while
>> still understanding things will change. One idea is just to disable it by
>> default, another option is to maybe use the Kconfig to make it a compile
>> time option which developers can use. Should we also display a warning
>> when running non-frozen extensions?
>>
>> Thanks to Anup for doing the initial port of Xvisor. The port is avaliable here:
>> https://github.com/avpatel/xvisor-next and will run on QEMU.
>>
>> Also thanks to Atish for implementing the SBI call support in Xvisor and
>> for lots of help debugging.
>>
>> To run this yourself:
>>  1. Apply this patch series to QEMU. The latest branch can be found here:
>>       https://github.com/alistair23/qemu/tree/mainline/alistair/riscv-hyp-work.next
>>  2. Get the version of OpenSBI that supports the H extension. This can
>>     be found here:
>>       https://github.com/riscv/opensbi/tree/hyp_ext_changes_v1
>>  3. Build the next release of Xvisor. It is available here:
>>       https://github.com/avpatel/xvisor-next
>>  4. Make sure you build the Xvisor tests, see here for details:
>>       https://github.com/avpatel/xvisor-next/tree/master/tests/riscv/virt64/linux
>>  5. Run QEMU:
>>      ./riscv64-softmmu/qemu-system-riscv64 -nographic \
>>        -machine virt -cpu rv64,h=true\
>>        -serial mon:stdio -serial null -m 4G \
>>        -device loader,file=vmm.bin,addr=0x80200000 \
>>        -kernel fw_jump.elf \
>>        -initrd vmm-disk-linux.img \
>>        -append "vmm.console=uart@10000000 vmm.bootcmd=\"vfs mount initrd /;vfs run /boot.xscript;vfs cat /system/banner.txt\""
>>
>>    Once you get to the prompt you can start the geust by running:
>>      guest kick guest0
>>    You can then bind to the serial port using:
>>      vserial bind guest0/uart0
>>    Then you can start Linux using:
>>      autoexec
>>
>>  This was all tested with the mainline 5.1 kernel. I don't know if it
>>  will work on older kernels.
>>
>> So far all of the QEMU work has been tested on Xvisor.
>>
>> Known Issues/TODO:
>>  - Add mstatush to support 32-bit Hypervisors
>>  - Fix the random hang that sometimes appears when running a Hypervisor guest (~5%)
>>
>> There is also on going work from Anup to port KVM.
>> We have code complete implementation of RISC-V KVM kernel module and
>> RISC-V KVMTOOL. Currently, we are debugging KVM on QEMU and we will
>> send-out RFC PATCHES for KVM in June/July 2019.
>> The KVM RISC-V kernel module is available in riscv_kvm_v1
>> branch at: https://github.com/avpatel/linux.git
>> The KVMTOOL RISC-V port is available in riscv_v1 branch of
>> https://github.com/avpatel/kvmtool.git
>>
>> There is very early work on a Xen port as well which is avaliable here:
>> https://github.com/alistair23/xen/tree/alistair/riscv-port
>>
>> Changes from RFC:
>>  - Rebase on latest master
>>  - Add floating point changes from Hypervisor extension
>>
>> Alistair Francis (27):
>>   target/riscv: Don't set write permissions on dirty PTEs
>>   target/riscv: Add the Hypervisor extension
>>   target/riscv: Add the virtulisation mode
>>   target/riscv: Add the force HS exception mode
>>   target/riscv: Add the Hypervisor CSRs to CPUState
>>   target/riscv: Dump Hypervisor registers if enabled
>>   target/riscv: Remove strict perm checking for CSR R/W
>>   target/riscv: Create function to test if FP is enabled
>>   target/riscv: Add support for background interrupt setting
>>   target/riscv: Add Hypervisor CSR access functions
>>   target/riscv: Add background CSRs accesses
>>   target/riscv: Add background register swapping function
>>   target/ricsv: Flush the TLB on virtulisation mode changes
>>   target/riscv: Generate illegal instruction on WFI when V=1
>>   riscv: plic: Remove unused interrupt functions
>>   riscv: plic: Always set sip.SEIP bit for HS
>>   target/riscv: Add hypvervisor trap support
>>   target/riscv: Add Hypervisor trap return support
>>   target/riscv: Add hfence instructions
>>   target/riscv: Disable guest FP support based on backgrond status
>>   target/riscv: Mark both sstatus and bsstatus as dirty
>>   target/riscv: Respect MPRV and SPRV for floating point ops
>>   target/riscv: Allow specifying MMU stage
>>   target/riscv: Allow specifying number of MMU stages
>>   target/riscv: Implement second stage MMU
>>   target/riscv: Call the second stage MMU in virtualisation mode
>>   target/riscv: Allow enabling the Hypervisor extension
>>
>>  hw/riscv/sifive_plic.c                        |  24 +-
>>  include/hw/riscv/sifive_plic.h                |   3 -
>>  target/riscv/cpu.c                            |  31 ++
>>  target/riscv/cpu.h                            |  32 +-
>>  target/riscv/cpu_bits.h                       |  32 +-
>>  target/riscv/cpu_helper.c                     | 443 ++++++++++++++++--
>>  target/riscv/csr.c                            | 216 ++++++++-
>>  target/riscv/insn32.decode                    |  23 +-
>>  .../riscv/insn_trans/trans_privileged.inc.c   |  40 ++
>>  target/riscv/op_helper.c                      |  71 ++-
>>  target/riscv/translate.c                      |  26 +
>>  11 files changed, 839 insertions(+), 102 deletions(-)
>>
>> --
>> 2.21.0
>>
>
>   tested with Linux kernel v5.2 and related opensbi/xvisor branches described above.
>
>  Tested-by: Chih-Min Chao <chihmin.chao@sifive.com>

Thanks for testing!!!

Unfortunately I don't think this is going to be merged. It is too late
to have this in the 4.1 release and I'm about to start working on
upgrading this to the v0.4 Hypervisor spec. So hopefully we can have
that in 4.2.

Alistair

>

Re: [Qemu-devel] [PATCH v1 00/27] Add RISC-V Hypervisor Extension

[Alistair Francis]

On Mon, Jul 15, 2019 at 5:00 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Fri, 7 Jun 2019 at 23:03, Alistair Francis <alistair.francis@wdc.com> wrote:
> > At the moment this spec is in a draft state and is subject to change. As
> > QEMU is extreamly useful in early bring up I think it makes sense for
> > QEMU to support non-frozen extensions. I would like to decide with this
> > series how QEMU will handle all future non-frozen extensions. That is a
> > standard way that QEMU users can test future RISC-V extensions while
> > still understanding things will change. One idea is just to disable it by
> > default, another option is to maybe use the Kconfig to make it a compile
> > time option which developers can use. Should we also display a warning
> > when running non-frozen extensions?
>
> We had an instance of this for Arm (though in fact the
> relevant patches to QEMU didn't end up getting into master
> before the spec was finalized in the end). My suggestion
> would be at minimum:
>  * by default non-frozen extensions should not be provided

Yep, these are off by default.

>  * they should be enabled via a command line option (cpu
>    property) whose name starts with "x-", which is our standard
>    way of flagging properties that are experimental and subject
>    to change or removal in future QEMU versions

Sounds good, I'll rename the property in the next version.

Alistair

>
> That way end-users know they're doing something non-standard
> that won't necessarily be supported in future by newer versions
> of QEMU, and if people copy recipes/commandlines/random guest
> images off old blog posts there'll be a hint that there's a
> reason why they don't work on newer QEMU that adheres to the
> final spec.
>
> thanks
> -- PMM

Re: [Qemu-devel] [PATCH v1 00/27] Add RISC-V Hypervisor Extension

[Chih-Min Chao]

On Wed, Jul 17, 2019 at 8:17 AM Alistair Francis <alistair23@gmail.com>
wrote:

> On Mon, Jul 15, 2019 at 5:00 AM Peter Maydell <peter.maydell@linaro.org>
> wrote:
> >
> > On Fri, 7 Jun 2019 at 23:03, Alistair Francis <alistair.francis@wdc.com>
> wrote:
> > > At the moment this spec is in a draft state and is subject to change.
> As
> > > QEMU is extreamly useful in early bring up I think it makes sense for
> > > QEMU to support non-frozen extensions. I would like to decide with this
> > > series how QEMU will handle all future non-frozen extensions. That is a
> > > standard way that QEMU users can test future RISC-V extensions while
> > > still understanding things will change. One idea is just to disable it
> by
> > > default, another option is to maybe use the Kconfig to make it a
> compile
> > > time option which developers can use. Should we also display a warning
> > > when running non-frozen extensions?
> >
> > We had an instance of this for Arm (though in fact the
> > relevant patches to QEMU didn't end up getting into master
> > before the spec was finalized in the end). My suggestion
> > would be at minimum:
> >  * by default non-frozen extensions should not be provided
>
> Yep, these are off by default.
>
> >  * they should be enabled via a command line option (cpu
> >    property) whose name starts with "x-", which is our standard
> >    way of flagging properties that are experimental and subject
> >    to change or removal in future QEMU versions
>
> Sounds good, I'll rename the property in the next version.
>
> Alistair
>
> >
> > That way end-users know they're doing something non-standard
> > that won't necessarily be supported in future by newer versions
> > of QEMU, and if people copy recipes/commandlines/random guest
> > images off old blog posts there'll be a hint that there's a
> > reason why they don't work on newer QEMU that adheres to the
> > final spec.
> >
> > thanks
> > -- PMM
>


Hi Peter,

I agree that unfrozen extension should't be merged into master but I think
the patch set is more like RFC version.
Some part of the patches have been merged in into master and will be
available in 4.1.  So my intention is to be
familiar with present h-extension by reviewing the Alistair's great work
and help the work when new patch set are ready
for 4.2 cycle.

chihmin.chao