* [PATCH v4 0/5] Add SBI v0.2 support for KVM
@ 2021-11-05 23:58 Atish Patra
2021-11-05 23:58 ` [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01 Atish Patra
` (4 more replies)
0 siblings, 5 replies; 14+ messages in thread
From: Atish Patra @ 2021-11-05 23:58 UTC (permalink / raw)
To: linux-kernel
Cc: Atish Patra, Anup Patel, Heinrich Schuchardt, kvm-riscv, kvm,
linux-riscv, Palmer Dabbelt, Paul Walmsley, Vincent Chen,
pbonzini, Sean Christopherson
The Supervisor Binary Interface(SBI) specification[1] now defines a
base extension that provides extendability to add future extensions
while maintaining backward compatibility with previous versions.
The new version is defined as 0.2 and older version is marked as 0.1.
This series adds following features to RISC-V Linux KVM.
1. Adds support for SBI v0.2 in KVM
2. SBI Hart state management extension (HSM) in KVM
3. Ordered booting of guest vcpus in guest Linux
This series is based on base KVM series which is already part of the kvm-next[2].
Guest kernel needs to also support SBI v0.2 and HSM extension in Kernel
to boot multiple vcpus. Linux kernel supports both starting v5.7.
In absense of that, guest can only boot 1 vcpu.
Changes from v3->v4:
1. Fixed the commit text title.
2. Removed a redundant memory barrier from patch 4.
3. Replaced preempt_enable/disable with get_cpu/put_cpu.
4. Renamed the exixting implementation as v01 instead of legacy.
Changes from v2->v3:
1. Rebased on the latest merged kvm series.
2. Dropped the reset extension patch because reset extension is not merged in kernel.
However, my tree[3] still contains it in case anybody wants to test it.
Changes from v1->v2:
1. Sent the patch 1 separately as it can merged independently.
2. Added Reset extension functionality.
Tested on Qemu and Rocket core FPGA.
[1] https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc
[3] https://github.com/atishp04/linux/tree/kvm_sbi_v03_reset
[4] https://github.com/atishp04/linux/tree/kvm_sbi_v03
Atish Patra (5):
RISC-V: KVM: Mark the existing SBI implementation as v01
RISC-V: KVM: Reorganize SBI code by moving SBI v0.1 to its own file
RISC-V: KVM: Add SBI v0.2 base extension
RISC-V: KVM: Add v0.1 replacement SBI extensions defined in v02
RISC-V: KVM: Add SBI HSM extension in KVM
arch/riscv/include/asm/kvm_vcpu_sbi.h | 33 +++++
arch/riscv/include/asm/sbi.h | 9 ++
arch/riscv/kvm/Makefile | 4 +
arch/riscv/kvm/vcpu.c | 23 +++
arch/riscv/kvm/vcpu_sbi.c | 206 ++++++++++++--------------
arch/riscv/kvm/vcpu_sbi_base.c | 73 +++++++++
arch/riscv/kvm/vcpu_sbi_hsm.c | 107 +++++++++++++
arch/riscv/kvm/vcpu_sbi_replace.c | 136 +++++++++++++++++
arch/riscv/kvm/vcpu_sbi_v01.c | 129 ++++++++++++++++
9 files changed, 609 insertions(+), 111 deletions(-)
create mode 100644 arch/riscv/include/asm/kvm_vcpu_sbi.h
create mode 100644 arch/riscv/kvm/vcpu_sbi_base.c
create mode 100644 arch/riscv/kvm/vcpu_sbi_hsm.c
create mode 100644 arch/riscv/kvm/vcpu_sbi_replace.c
create mode 100644 arch/riscv/kvm/vcpu_sbi_v01.c
--
2.31.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01
2021-11-05 23:58 [PATCH v4 0/5] Add SBI v0.2 support for KVM Atish Patra
@ 2021-11-05 23:58 ` Atish Patra
2021-11-06 4:27 ` Heinrich Schuchardt
2021-11-06 4:41 ` Anup Patel
2021-11-05 23:58 ` [PATCH v4 2/5] RISC-V: KVM: Reorganize SBI code by moving SBI v0.1 to its own file Atish Patra
` (3 subsequent siblings)
4 siblings, 2 replies; 14+ messages in thread
From: Atish Patra @ 2021-11-05 23:58 UTC (permalink / raw)
To: linux-kernel
Cc: Atish Patra, Anup Patel, Heinrich Schuchardt, kvm-riscv, kvm,
linux-riscv, Palmer Dabbelt, Paul Walmsley, Vincent Chen,
pbonzini, Sean Christopherson
The existing SBI specification impelementation follows v0.1
specification. The latest specification known as v0.2 allows more
scalability and performance improvements.
Rename the existing implementation as v01 and provide a way to allow
future extensions.
Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
arch/riscv/include/asm/kvm_vcpu_sbi.h | 29 +++++
arch/riscv/kvm/vcpu_sbi.c | 147 +++++++++++++++++++++-----
2 files changed, 147 insertions(+), 29 deletions(-)
create mode 100644 arch/riscv/include/asm/kvm_vcpu_sbi.h
diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
new file mode 100644
index 000000000000..1a4cb0db2d0b
--- /dev/null
+++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/**
+ * Copyright (c) 2021 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ * Atish Patra <atish.patra@wdc.com>
+ */
+
+#ifndef __RISCV_KVM_VCPU_SBI_H__
+#define __RISCV_KVM_VCPU_SBI_H__
+
+#define KVM_SBI_VERSION_MAJOR 0
+#define KVM_SBI_VERSION_MINOR 2
+
+struct kvm_vcpu_sbi_extension {
+ unsigned long extid_start;
+ unsigned long extid_end;
+ /**
+ * SBI extension handler. It can be defined for a given extension or group of
+ * extension. But it should always return linux error codes rather than SBI
+ * specific error codes.
+ */
+ int (*handler)(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ unsigned long *out_val, struct kvm_cpu_trap *utrap,
+ bool *exit);
+};
+
+const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid);
+#endif /* __RISCV_KVM_VCPU_SBI_H__ */
diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
index eb3c045edf11..05cab5f27eee 100644
--- a/arch/riscv/kvm/vcpu_sbi.c
+++ b/arch/riscv/kvm/vcpu_sbi.c
@@ -12,9 +12,25 @@
#include <asm/csr.h>
#include <asm/sbi.h>
#include <asm/kvm_vcpu_timer.h>
+#include <asm/kvm_vcpu_sbi.h>
-#define SBI_VERSION_MAJOR 0
-#define SBI_VERSION_MINOR 1
+static int kvm_linux_err_map_sbi(int err)
+{
+ switch (err) {
+ case 0:
+ return SBI_SUCCESS;
+ case -EPERM:
+ return SBI_ERR_DENIED;
+ case -EINVAL:
+ return SBI_ERR_INVALID_PARAM;
+ case -EFAULT:
+ return SBI_ERR_INVALID_ADDRESS;
+ case -EOPNOTSUPP:
+ return SBI_ERR_NOT_SUPPORTED;
+ default:
+ return SBI_ERR_FAILURE;
+ };
+}
static void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu,
struct kvm_run *run)
@@ -72,16 +88,17 @@ static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
}
-int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
+static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ unsigned long *out_val,
+ struct kvm_cpu_trap *utrap,
+ bool *exit)
{
ulong hmask;
- int i, ret = 1;
+ int i, ret = 0;
u64 next_cycle;
struct kvm_vcpu *rvcpu;
- bool next_sepc = true;
struct cpumask cm, hm;
struct kvm *kvm = vcpu->kvm;
- struct kvm_cpu_trap utrap = { 0 };
struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
if (!cp)
@@ -95,8 +112,7 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
* handled in kernel so we forward these to user-space
*/
kvm_riscv_vcpu_sbi_forward(vcpu, run);
- next_sepc = false;
- ret = 0;
+ *exit = true;
break;
case SBI_EXT_0_1_SET_TIMER:
#if __riscv_xlen == 32
@@ -104,47 +120,42 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
#else
next_cycle = (u64)cp->a0;
#endif
- kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
+ ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
break;
case SBI_EXT_0_1_CLEAR_IPI:
- kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
+ ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
break;
case SBI_EXT_0_1_SEND_IPI:
if (cp->a0)
hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
- &utrap);
+ utrap);
else
hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
- if (utrap.scause) {
- utrap.sepc = cp->sepc;
- kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
- next_sepc = false;
+ if (utrap->scause)
break;
- }
+
for_each_set_bit(i, &hmask, BITS_PER_LONG) {
rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
- kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
+ ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
+ if (ret < 0)
+ break;
}
break;
case SBI_EXT_0_1_SHUTDOWN:
kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
- next_sepc = false;
- ret = 0;
+ *exit = true;
break;
case SBI_EXT_0_1_REMOTE_FENCE_I:
case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
if (cp->a0)
hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
- &utrap);
+ utrap);
else
hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
- if (utrap.scause) {
- utrap.sepc = cp->sepc;
- kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
- next_sepc = false;
+ if (utrap->scause)
break;
- }
+
cpumask_clear(&cm);
for_each_set_bit(i, &hmask, BITS_PER_LONG) {
rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
@@ -154,22 +165,100 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
}
riscv_cpuid_to_hartid_mask(&cm, &hm);
if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
- sbi_remote_fence_i(cpumask_bits(&hm));
+ ret = sbi_remote_fence_i(cpumask_bits(&hm));
else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
- sbi_remote_hfence_vvma(cpumask_bits(&hm),
+ ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
cp->a1, cp->a2);
else
- sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
+ ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
cp->a1, cp->a2, cp->a3);
break;
default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
+ .extid_start = SBI_EXT_0_1_SET_TIMER,
+ .extid_end = SBI_EXT_0_1_SHUTDOWN,
+ .handler = kvm_sbi_ext_v01_handler,
+};
+
+static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
+ &vcpu_sbi_ext_v01,
+};
+
+const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid)
+{
+ int i = 0;
+
+ for (i = 0; i < ARRAY_SIZE(sbi_ext); i++) {
+ if (sbi_ext[i]->extid_start <= extid &&
+ sbi_ext[i]->extid_end >= extid)
+ return sbi_ext[i];
+ }
+
+ return NULL;
+}
+
+int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
+{
+ int ret = 1;
+ bool next_sepc = true;
+ bool userspace_exit = false;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ const struct kvm_vcpu_sbi_extension *sbi_ext;
+ struct kvm_cpu_trap utrap = { 0 };
+ unsigned long out_val = 0;
+ bool ext_is_v01 = false;
+
+ if (!cp)
+ return -EINVAL;
+
+ sbi_ext = kvm_vcpu_sbi_find_ext(cp->a7);
+ if (sbi_ext && sbi_ext->handler) {
+ if (cp->a7 >= SBI_EXT_0_1_SET_TIMER &&
+ cp->a7 <= SBI_EXT_0_1_SHUTDOWN)
+ ext_is_v01 = true;
+ ret = sbi_ext->handler(vcpu, run, &out_val, &utrap, &userspace_exit);
+ } else {
/* Return error for unsupported SBI calls */
cp->a0 = SBI_ERR_NOT_SUPPORTED;
- break;
+ goto ecall_done;
}
+ /* Handle special error cases i.e trap, exit or userspace forward */
+ if (utrap.scause) {
+ /* No need to increment sepc or exit ioctl loop */
+ ret = 1;
+ utrap.sepc = cp->sepc;
+ kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
+ next_sepc = false;
+ goto ecall_done;
+ }
+
+ /* Exit ioctl loop or Propagate the error code the guest */
+ if (userspace_exit) {
+ next_sepc = false;
+ ret = 0;
+ } else {
+ /**
+ * SBI extension handler always returns an Linux error code. Convert
+ * it to the SBI specific error code that can be propagated the SBI
+ * caller.
+ */
+ ret = kvm_linux_err_map_sbi(ret);
+ cp->a0 = ret;
+ ret = 1;
+ }
+ecall_done:
if (next_sepc)
cp->sepc += 4;
+ if (!ext_is_v01)
+ cp->a1 = out_val;
return ret;
}
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v4 2/5] RISC-V: KVM: Reorganize SBI code by moving SBI v0.1 to its own file
2021-11-05 23:58 [PATCH v4 0/5] Add SBI v0.2 support for KVM Atish Patra
2021-11-05 23:58 ` [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01 Atish Patra
@ 2021-11-05 23:58 ` Atish Patra
2021-11-06 4:43 ` Anup Patel
2021-11-05 23:58 ` [PATCH v4 3/5] RISC-V: KVM: Add SBI v0.2 base extension Atish Patra
` (2 subsequent siblings)
4 siblings, 1 reply; 14+ messages in thread
From: Atish Patra @ 2021-11-05 23:58 UTC (permalink / raw)
To: linux-kernel
Cc: Atish Patra, Anup Patel, Heinrich Schuchardt, kvm-riscv, kvm,
linux-riscv, Palmer Dabbelt, Paul Walmsley, Vincent Chen,
pbonzini, Sean Christopherson
With SBI v0.2, there may be more SBI extensions in future. It makes more
sense to group related extensions in separate files. Guest kernel will
choose appropriate SBI version dynamically.
Move the existing implementation to a separate file so that it can be
removed in future without much conflict.
Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
arch/riscv/include/asm/kvm_vcpu_sbi.h | 2 +
arch/riscv/kvm/Makefile | 1 +
arch/riscv/kvm/vcpu_sbi.c | 151 +++-----------------------
arch/riscv/kvm/vcpu_sbi_v01.c | 129 ++++++++++++++++++++++
4 files changed, 149 insertions(+), 134 deletions(-)
create mode 100644 arch/riscv/kvm/vcpu_sbi_v01.c
diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
index 1a4cb0db2d0b..704151969ceb 100644
--- a/arch/riscv/include/asm/kvm_vcpu_sbi.h
+++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
@@ -25,5 +25,7 @@ struct kvm_vcpu_sbi_extension {
bool *exit);
};
+void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run);
const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid);
+
#endif /* __RISCV_KVM_VCPU_SBI_H__ */
diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
index 30cdd1df0098..d3d5ff3a6019 100644
--- a/arch/riscv/kvm/Makefile
+++ b/arch/riscv/kvm/Makefile
@@ -23,4 +23,5 @@ kvm-y += vcpu_exit.o
kvm-y += vcpu_fp.o
kvm-y += vcpu_switch.o
kvm-y += vcpu_sbi.o
+kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
kvm-y += vcpu_timer.o
diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
index 05cab5f27eee..06b42f6977e1 100644
--- a/arch/riscv/kvm/vcpu_sbi.c
+++ b/arch/riscv/kvm/vcpu_sbi.c
@@ -9,9 +9,7 @@
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/kvm_host.h>
-#include <asm/csr.h>
#include <asm/sbi.h>
-#include <asm/kvm_vcpu_timer.h>
#include <asm/kvm_vcpu_sbi.h>
static int kvm_linux_err_map_sbi(int err)
@@ -32,8 +30,21 @@ static int kvm_linux_err_map_sbi(int err)
};
}
-static void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu,
- struct kvm_run *run)
+#ifdef CONFIG_RISCV_SBI_V01
+extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01;
+#else
+static const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
+ .extid_start = -1UL,
+ .extid_end = -1UL,
+ .handler = NULL,
+};
+#endif
+
+static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
+ &vcpu_sbi_ext_v01,
+};
+
+void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
{
struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
@@ -71,126 +82,6 @@ int kvm_riscv_vcpu_sbi_return(struct kvm_vcpu *vcpu, struct kvm_run *run)
return 0;
}
-#ifdef CONFIG_RISCV_SBI_V01
-
-static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
- struct kvm_run *run, u32 type)
-{
- int i;
- struct kvm_vcpu *tmp;
-
- kvm_for_each_vcpu(i, tmp, vcpu->kvm)
- tmp->arch.power_off = true;
- kvm_make_all_cpus_request(vcpu->kvm, KVM_REQ_SLEEP);
-
- memset(&run->system_event, 0, sizeof(run->system_event));
- run->system_event.type = type;
- run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
-}
-
-static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
- unsigned long *out_val,
- struct kvm_cpu_trap *utrap,
- bool *exit)
-{
- ulong hmask;
- int i, ret = 0;
- u64 next_cycle;
- struct kvm_vcpu *rvcpu;
- struct cpumask cm, hm;
- struct kvm *kvm = vcpu->kvm;
- struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
-
- if (!cp)
- return -EINVAL;
-
- switch (cp->a7) {
- case SBI_EXT_0_1_CONSOLE_GETCHAR:
- case SBI_EXT_0_1_CONSOLE_PUTCHAR:
- /*
- * The CONSOLE_GETCHAR/CONSOLE_PUTCHAR SBI calls cannot be
- * handled in kernel so we forward these to user-space
- */
- kvm_riscv_vcpu_sbi_forward(vcpu, run);
- *exit = true;
- break;
- case SBI_EXT_0_1_SET_TIMER:
-#if __riscv_xlen == 32
- next_cycle = ((u64)cp->a1 << 32) | (u64)cp->a0;
-#else
- next_cycle = (u64)cp->a0;
-#endif
- ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
- break;
- case SBI_EXT_0_1_CLEAR_IPI:
- ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
- break;
- case SBI_EXT_0_1_SEND_IPI:
- if (cp->a0)
- hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
- utrap);
- else
- hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
- if (utrap->scause)
- break;
-
- for_each_set_bit(i, &hmask, BITS_PER_LONG) {
- rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
- ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
- if (ret < 0)
- break;
- }
- break;
- case SBI_EXT_0_1_SHUTDOWN:
- kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
- *exit = true;
- break;
- case SBI_EXT_0_1_REMOTE_FENCE_I:
- case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
- case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
- if (cp->a0)
- hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
- utrap);
- else
- hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
- if (utrap->scause)
- break;
-
- cpumask_clear(&cm);
- for_each_set_bit(i, &hmask, BITS_PER_LONG) {
- rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
- if (rvcpu->cpu < 0)
- continue;
- cpumask_set_cpu(rvcpu->cpu, &cm);
- }
- riscv_cpuid_to_hartid_mask(&cm, &hm);
- if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
- ret = sbi_remote_fence_i(cpumask_bits(&hm));
- else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
- ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
- cp->a1, cp->a2);
- else
- ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
- cp->a1, cp->a2, cp->a3);
- break;
- default:
- ret = -EINVAL;
- break;
- }
-
- return ret;
-}
-
-const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
- .extid_start = SBI_EXT_0_1_SET_TIMER,
- .extid_end = SBI_EXT_0_1_SHUTDOWN,
- .handler = kvm_sbi_ext_v01_handler,
-};
-
-static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
- &vcpu_sbi_ext_v01,
-};
-
const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid)
{
int i = 0;
@@ -220,9 +111,11 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
sbi_ext = kvm_vcpu_sbi_find_ext(cp->a7);
if (sbi_ext && sbi_ext->handler) {
+#ifdef CONFIG_RISCV_SBI_V01
if (cp->a7 >= SBI_EXT_0_1_SET_TIMER &&
cp->a7 <= SBI_EXT_0_1_SHUTDOWN)
ext_is_v01 = true;
+#endif
ret = sbi_ext->handler(vcpu, run, &out_val, &utrap, &userspace_exit);
} else {
/* Return error for unsupported SBI calls */
@@ -262,13 +155,3 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
return ret;
}
-
-#else
-
-int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
-{
- kvm_riscv_vcpu_sbi_forward(vcpu, run);
- return 0;
-}
-
-#endif
diff --git a/arch/riscv/kvm/vcpu_sbi_v01.c b/arch/riscv/kvm/vcpu_sbi_v01.c
new file mode 100644
index 000000000000..5a8e2afc8a57
--- /dev/null
+++ b/arch/riscv/kvm/vcpu_sbi_v01.c
@@ -0,0 +1,129 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ * Atish Patra <atish.patra@wdc.com>
+ */
+
+#include <linux/errno.h>
+#include <linux/err.h>
+#include <linux/kvm_host.h>
+#include <asm/csr.h>
+#include <asm/sbi.h>
+#include <asm/kvm_vcpu_timer.h>
+#include <asm/kvm_vcpu_sbi.h>
+
+static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
+ struct kvm_run *run, u32 type)
+{
+ int i;
+ struct kvm_vcpu *tmp;
+
+ kvm_for_each_vcpu(i, tmp, vcpu->kvm)
+ tmp->arch.power_off = true;
+ kvm_make_all_cpus_request(vcpu->kvm, KVM_REQ_SLEEP);
+
+ memset(&run->system_event, 0, sizeof(run->system_event));
+ run->system_event.type = type;
+ run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
+}
+
+static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ unsigned long *out_val,
+ struct kvm_cpu_trap *utrap,
+ bool *exit)
+{
+ ulong hmask;
+ int i, ret = 0;
+ u64 next_cycle;
+ struct kvm_vcpu *rvcpu;
+ struct cpumask cm, hm;
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+
+ if (!cp)
+ return -EINVAL;
+
+ switch (cp->a7) {
+ case SBI_EXT_0_1_CONSOLE_GETCHAR:
+ case SBI_EXT_0_1_CONSOLE_PUTCHAR:
+ /*
+ * The CONSOLE_GETCHAR/CONSOLE_PUTCHAR SBI calls cannot be
+ * handled in kernel so we forward these to user-space
+ */
+ kvm_riscv_vcpu_sbi_forward(vcpu, run);
+ *exit = true;
+ break;
+ case SBI_EXT_0_1_SET_TIMER:
+#if __riscv_xlen == 32
+ next_cycle = ((u64)cp->a1 << 32) | (u64)cp->a0;
+#else
+ next_cycle = (u64)cp->a0;
+#endif
+ ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
+ break;
+ case SBI_EXT_0_1_CLEAR_IPI:
+ ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
+ break;
+ case SBI_EXT_0_1_SEND_IPI:
+ if (cp->a0)
+ hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
+ utrap);
+ else
+ hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
+ if (utrap->scause)
+ break;
+
+ for_each_set_bit(i, &hmask, BITS_PER_LONG) {
+ rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
+ ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
+ if (ret < 0)
+ break;
+ }
+ break;
+ case SBI_EXT_0_1_SHUTDOWN:
+ kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
+ *exit = true;
+ break;
+ case SBI_EXT_0_1_REMOTE_FENCE_I:
+ case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
+ case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
+ if (cp->a0)
+ hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
+ utrap);
+ else
+ hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
+ if (utrap->scause)
+ break;
+
+ cpumask_clear(&cm);
+ for_each_set_bit(i, &hmask, BITS_PER_LONG) {
+ rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
+ if (rvcpu->cpu < 0)
+ continue;
+ cpumask_set_cpu(rvcpu->cpu, &cm);
+ }
+ riscv_cpuid_to_hartid_mask(&cm, &hm);
+ if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
+ ret = sbi_remote_fence_i(cpumask_bits(&hm));
+ else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
+ ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
+ cp->a1, cp->a2);
+ else
+ ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
+ cp->a1, cp->a2, cp->a3);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ };
+
+ return ret;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
+ .extid_start = SBI_EXT_0_1_SET_TIMER,
+ .extid_end = SBI_EXT_0_1_SHUTDOWN,
+ .handler = kvm_sbi_ext_v01_handler,
+};
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v4 3/5] RISC-V: KVM: Add SBI v0.2 base extension
2021-11-05 23:58 [PATCH v4 0/5] Add SBI v0.2 support for KVM Atish Patra
2021-11-05 23:58 ` [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01 Atish Patra
2021-11-05 23:58 ` [PATCH v4 2/5] RISC-V: KVM: Reorganize SBI code by moving SBI v0.1 to its own file Atish Patra
@ 2021-11-05 23:58 ` Atish Patra
2021-11-06 4:45 ` Anup Patel
2021-11-05 23:58 ` [PATCH v4 4/5] RISC-V: KVM: Add v0.1 replacement SBI extensions defined in v02 Atish Patra
2021-11-05 23:58 ` [PATCH v4 5/5] RISC-V: KVM: Add SBI HSM extension in KVM Atish Patra
4 siblings, 1 reply; 14+ messages in thread
From: Atish Patra @ 2021-11-05 23:58 UTC (permalink / raw)
To: linux-kernel
Cc: Atish Patra, Anup Patel, Heinrich Schuchardt, kvm-riscv, kvm,
linux-riscv, Palmer Dabbelt, Paul Walmsley, Vincent Chen,
pbonzini, Sean Christopherson
SBI v0.2 base extension defined to allow backward compatibility and
probing of future extensions. This is also the only mandatory SBI
extension that must be implemented by SBI implementors.
Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
arch/riscv/include/asm/kvm_vcpu_sbi.h | 2 +
arch/riscv/include/asm/sbi.h | 8 +++
arch/riscv/kvm/Makefile | 1 +
arch/riscv/kvm/vcpu_sbi.c | 3 +-
arch/riscv/kvm/vcpu_sbi_base.c | 73 +++++++++++++++++++++++++++
5 files changed, 86 insertions(+), 1 deletion(-)
create mode 100644 arch/riscv/kvm/vcpu_sbi_base.c
diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
index 704151969ceb..76e4e17a3e00 100644
--- a/arch/riscv/include/asm/kvm_vcpu_sbi.h
+++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
@@ -9,6 +9,8 @@
#ifndef __RISCV_KVM_VCPU_SBI_H__
#define __RISCV_KVM_VCPU_SBI_H__
+#define KVM_SBI_IMPID 3
+
#define KVM_SBI_VERSION_MAJOR 0
#define KVM_SBI_VERSION_MINOR 2
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 0d42693cb65e..4f9370b6032e 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -27,6 +27,14 @@ enum sbi_ext_id {
SBI_EXT_IPI = 0x735049,
SBI_EXT_RFENCE = 0x52464E43,
SBI_EXT_HSM = 0x48534D,
+
+ /* Experimentals extensions must lie within this range */
+ SBI_EXT_EXPERIMENTAL_START = 0x0800000,
+ SBI_EXT_EXPERIMENTAL_END = 0x08FFFFFF,
+
+ /* Vendor extensions must lie within this range */
+ SBI_EXT_VENDOR_START = 0x09000000,
+ SBI_EXT_VENDOR_END = 0x09FFFFFF,
};
enum sbi_ext_base_fid {
diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
index d3d5ff3a6019..84c02922a329 100644
--- a/arch/riscv/kvm/Makefile
+++ b/arch/riscv/kvm/Makefile
@@ -24,4 +24,5 @@ kvm-y += vcpu_fp.o
kvm-y += vcpu_switch.o
kvm-y += vcpu_sbi.o
kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
+kvm-y += vcpu_sbi_base.o
kvm-y += vcpu_timer.o
diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
index 06b42f6977e1..92b682f4f29e 100644
--- a/arch/riscv/kvm/vcpu_sbi.c
+++ b/arch/riscv/kvm/vcpu_sbi.c
@@ -39,9 +39,10 @@ static const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
.handler = NULL,
};
#endif
-
+extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_base;
static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
&vcpu_sbi_ext_v01,
+ &vcpu_sbi_ext_base,
};
void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
diff --git a/arch/riscv/kvm/vcpu_sbi_base.c b/arch/riscv/kvm/vcpu_sbi_base.c
new file mode 100644
index 000000000000..1aeda3e10e7c
--- /dev/null
+++ b/arch/riscv/kvm/vcpu_sbi_base.c
@@ -0,0 +1,73 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ * Atish Patra <atish.patra@wdc.com>
+ */
+
+#include <linux/errno.h>
+#include <linux/err.h>
+#include <linux/kvm_host.h>
+#include <asm/csr.h>
+#include <asm/sbi.h>
+#include <asm/kvm_vcpu_timer.h>
+#include <asm/kvm_vcpu_sbi.h>
+
+static int kvm_sbi_ext_base_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ unsigned long *out_val,
+ struct kvm_cpu_trap *trap, bool *exit)
+{
+ int ret = 0;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ struct sbiret ecall_ret;
+
+ if (!cp)
+ return -EINVAL;
+
+ switch (cp->a6) {
+ case SBI_EXT_BASE_GET_SPEC_VERSION:
+ *out_val = (KVM_SBI_VERSION_MAJOR <<
+ SBI_SPEC_VERSION_MAJOR_SHIFT) |
+ KVM_SBI_VERSION_MINOR;
+ break;
+ case SBI_EXT_BASE_GET_IMP_ID:
+ *out_val = KVM_SBI_IMPID;
+ break;
+ case SBI_EXT_BASE_GET_IMP_VERSION:
+ *out_val = 0;
+ break;
+ case SBI_EXT_BASE_PROBE_EXT:
+ *out_val = kvm_vcpu_sbi_find_ext(cp->a0) ? 1 : 0;
+ if ((!*out_val) &&
+ ((cp->a0 >= SBI_EXT_EXPERIMENTAL_START &&
+ cp->a0 <= SBI_EXT_EXPERIMENTAL_END) ||
+ ((cp->a0 >= SBI_EXT_VENDOR_START &&
+ cp->a0 <= SBI_EXT_VENDOR_END)))) {
+ /* For experimental/vendor extensions forward to the userspace*/
+ kvm_riscv_vcpu_sbi_forward(vcpu, run);
+ *exit = true;
+ }
+ break;
+ case SBI_EXT_BASE_GET_MVENDORID:
+ case SBI_EXT_BASE_GET_MARCHID:
+ case SBI_EXT_BASE_GET_MIMPID:
+ ecall_ret = sbi_ecall(SBI_EXT_BASE, cp->a6, 0, 0, 0, 0, 0, 0);
+ if (!ecall_ret.error)
+ *out_val = ecall_ret.value;
+ /*TODO: We are unnecessarily converting the error twice */
+ ret = sbi_err_map_linux_errno(ecall_ret.error);
+ break;
+ default:
+ ret = -EOPNOTSUPP;
+ break;
+ }
+
+ return ret;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_base = {
+ .extid_start = SBI_EXT_BASE,
+ .extid_end = SBI_EXT_BASE,
+ .handler = kvm_sbi_ext_base_handler,
+};
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v4 4/5] RISC-V: KVM: Add v0.1 replacement SBI extensions defined in v02
2021-11-05 23:58 [PATCH v4 0/5] Add SBI v0.2 support for KVM Atish Patra
` (2 preceding siblings ...)
2021-11-05 23:58 ` [PATCH v4 3/5] RISC-V: KVM: Add SBI v0.2 base extension Atish Patra
@ 2021-11-05 23:58 ` Atish Patra
2021-11-06 4:46 ` Anup Patel
2021-11-05 23:58 ` [PATCH v4 5/5] RISC-V: KVM: Add SBI HSM extension in KVM Atish Patra
4 siblings, 1 reply; 14+ messages in thread
From: Atish Patra @ 2021-11-05 23:58 UTC (permalink / raw)
To: linux-kernel
Cc: Atish Patra, Anup Patel, Heinrich Schuchardt, kvm-riscv, kvm,
linux-riscv, Palmer Dabbelt, Paul Walmsley, Vincent Chen,
pbonzini, Sean Christopherson
The SBI v0.2 contains some of the improved versions of required v0.1
extensions such as remote fence, timer and IPI.
This patch implements those extensions.
Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
arch/riscv/kvm/Makefile | 1 +
arch/riscv/kvm/vcpu_sbi.c | 7 ++
arch/riscv/kvm/vcpu_sbi_replace.c | 136 ++++++++++++++++++++++++++++++
3 files changed, 144 insertions(+)
create mode 100644 arch/riscv/kvm/vcpu_sbi_replace.c
diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
index 84c02922a329..4757ae158bf3 100644
--- a/arch/riscv/kvm/Makefile
+++ b/arch/riscv/kvm/Makefile
@@ -25,4 +25,5 @@ kvm-y += vcpu_switch.o
kvm-y += vcpu_sbi.o
kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
kvm-y += vcpu_sbi_base.o
+kvm-y += vcpu_sbi_replace.o
kvm-y += vcpu_timer.o
diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
index 92b682f4f29e..3502c6166eba 100644
--- a/arch/riscv/kvm/vcpu_sbi.c
+++ b/arch/riscv/kvm/vcpu_sbi.c
@@ -40,9 +40,16 @@ static const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
};
#endif
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_base;
+extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_time;
+extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_ipi;
+extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_rfence;
+
static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
&vcpu_sbi_ext_v01,
&vcpu_sbi_ext_base,
+ &vcpu_sbi_ext_time,
+ &vcpu_sbi_ext_ipi,
+ &vcpu_sbi_ext_rfence,
};
void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
diff --git a/arch/riscv/kvm/vcpu_sbi_replace.c b/arch/riscv/kvm/vcpu_sbi_replace.c
new file mode 100644
index 000000000000..a80fa7691b14
--- /dev/null
+++ b/arch/riscv/kvm/vcpu_sbi_replace.c
@@ -0,0 +1,136 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ * Atish Patra <atish.patra@wdc.com>
+ */
+
+#include <linux/errno.h>
+#include <linux/err.h>
+#include <linux/kvm_host.h>
+#include <asm/csr.h>
+#include <asm/sbi.h>
+#include <asm/kvm_vcpu_timer.h>
+#include <asm/kvm_vcpu_sbi.h>
+
+static int kvm_sbi_ext_time_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ unsigned long *out_val,
+ struct kvm_cpu_trap *utrap, bool *exit)
+{
+ int ret = 0;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ u64 next_cycle;
+
+ if (!cp || (cp->a6 != SBI_EXT_TIME_SET_TIMER))
+ return -EINVAL;
+
+#if __riscv_xlen == 32
+ next_cycle = ((u64)cp->a1 << 32) | (u64)cp->a0;
+#else
+ next_cycle = (u64)cp->a0;
+#endif
+ kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
+
+ return ret;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_time = {
+ .extid_start = SBI_EXT_TIME,
+ .extid_end = SBI_EXT_TIME,
+ .handler = kvm_sbi_ext_time_handler,
+};
+
+static int kvm_sbi_ext_ipi_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ unsigned long *out_val,
+ struct kvm_cpu_trap *utrap, bool *exit)
+{
+ int i, ret = 0;
+ struct kvm_vcpu *tmp;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ unsigned long hmask = cp->a0;
+ unsigned long hbase = cp->a1;
+
+ if (!cp || (cp->a6 != SBI_EXT_IPI_SEND_IPI))
+ return -EINVAL;
+
+ kvm_for_each_vcpu(i, tmp, vcpu->kvm) {
+ if (hbase != -1UL) {
+ if (tmp->vcpu_id < hbase)
+ continue;
+ if (!(hmask & (1UL << (tmp->vcpu_id - hbase))))
+ continue;
+ }
+ ret = kvm_riscv_vcpu_set_interrupt(tmp, IRQ_VS_SOFT);
+ if (ret < 0)
+ break;
+ }
+
+ return ret;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_ipi = {
+ .extid_start = SBI_EXT_IPI,
+ .extid_end = SBI_EXT_IPI,
+ .handler = kvm_sbi_ext_ipi_handler,
+};
+
+static int kvm_sbi_ext_rfence_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ unsigned long *out_val,
+ struct kvm_cpu_trap *utrap, bool *exit)
+{
+ int i, ret = 0;
+ struct cpumask cm, hm;
+ struct kvm_vcpu *tmp;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ unsigned long hmask = cp->a0;
+ unsigned long hbase = cp->a1;
+ unsigned long funcid = cp->a6;
+
+ if (!cp)
+ return -EINVAL;
+
+ cpumask_clear(&cm);
+ cpumask_clear(&hm);
+ kvm_for_each_vcpu(i, tmp, vcpu->kvm) {
+ if (hbase != -1UL) {
+ if (tmp->vcpu_id < hbase)
+ continue;
+ if (!(hmask & (1UL << (tmp->vcpu_id - hbase))))
+ continue;
+ }
+ if (tmp->cpu < 0)
+ continue;
+ cpumask_set_cpu(tmp->cpu, &cm);
+ }
+
+ riscv_cpuid_to_hartid_mask(&cm, &hm);
+
+ switch (funcid) {
+ case SBI_EXT_RFENCE_REMOTE_FENCE_I:
+ ret = sbi_remote_fence_i(cpumask_bits(&hm));
+ break;
+ case SBI_EXT_RFENCE_REMOTE_SFENCE_VMA:
+ ret = sbi_remote_hfence_vvma(cpumask_bits(&hm), cp->a2, cp->a3);
+ break;
+ case SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID:
+ ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm), cp->a2,
+ cp->a3, cp->a4);
+ break;
+ case SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA:
+ case SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID:
+ case SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA:
+ case SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID:
+ /* TODO: implement for nested hypervisor case */
+ default:
+ ret = -EOPNOTSUPP;
+ }
+
+ return ret;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_rfence = {
+ .extid_start = SBI_EXT_RFENCE,
+ .extid_end = SBI_EXT_RFENCE,
+ .handler = kvm_sbi_ext_rfence_handler,
+};
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v4 5/5] RISC-V: KVM: Add SBI HSM extension in KVM
2021-11-05 23:58 [PATCH v4 0/5] Add SBI v0.2 support for KVM Atish Patra
` (3 preceding siblings ...)
2021-11-05 23:58 ` [PATCH v4 4/5] RISC-V: KVM: Add v0.1 replacement SBI extensions defined in v02 Atish Patra
@ 2021-11-05 23:58 ` Atish Patra
2021-11-06 4:50 ` Anup Patel
4 siblings, 1 reply; 14+ messages in thread
From: Atish Patra @ 2021-11-05 23:58 UTC (permalink / raw)
To: linux-kernel
Cc: Atish Patra, Anup Patel, Heinrich Schuchardt, kvm-riscv, kvm,
linux-riscv, Palmer Dabbelt, Paul Walmsley, Vincent Chen,
pbonzini, Sean Christopherson
SBI HSM extension allows OS to start/stop harts any time. It also allows
ordered booting of harts instead of random booting.
Implement SBI HSM exntesion and designate the vcpu 0 as the boot vcpu id.
All other non-zero non-booting vcpus should be brought up by the OS
implementing HSM extension. If the guest OS doesn't implement HSM
extension, only single vcpu will be available to OS.
Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
arch/riscv/include/asm/sbi.h | 1 +
arch/riscv/kvm/Makefile | 1 +
arch/riscv/kvm/vcpu.c | 23 ++++++++
arch/riscv/kvm/vcpu_sbi.c | 4 ++
arch/riscv/kvm/vcpu_sbi_hsm.c | 107 ++++++++++++++++++++++++++++++++++
5 files changed, 136 insertions(+)
create mode 100644 arch/riscv/kvm/vcpu_sbi_hsm.c
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 4f9370b6032e..79af25c45c8d 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -90,6 +90,7 @@ enum sbi_hsm_hart_status {
#define SBI_ERR_INVALID_PARAM -3
#define SBI_ERR_DENIED -4
#define SBI_ERR_INVALID_ADDRESS -5
+#define SBI_ERR_ALREADY_AVAILABLE -6
extern unsigned long sbi_spec_version;
struct sbiret {
diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
index 4757ae158bf3..aaf181a3d74b 100644
--- a/arch/riscv/kvm/Makefile
+++ b/arch/riscv/kvm/Makefile
@@ -26,4 +26,5 @@ kvm-y += vcpu_sbi.o
kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
kvm-y += vcpu_sbi_base.o
kvm-y += vcpu_sbi_replace.o
+kvm-y += vcpu_sbi_hsm.o
kvm-y += vcpu_timer.o
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index e3d3aed46184..50158867406d 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -53,6 +53,17 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu)
struct kvm_vcpu_csr *reset_csr = &vcpu->arch.guest_reset_csr;
struct kvm_cpu_context *cntx = &vcpu->arch.guest_context;
struct kvm_cpu_context *reset_cntx = &vcpu->arch.guest_reset_context;
+ bool loaded;
+
+ /**
+ * The preemption should be disabled here because it races with
+ * kvm_sched_out/kvm_sched_in(called from preempt notifiers) which
+ * also calls vcpu_load/put.
+ */
+ get_cpu();
+ loaded = (vcpu->cpu != -1);
+ if (loaded)
+ kvm_arch_vcpu_put(vcpu);
memcpy(csr, reset_csr, sizeof(*csr));
@@ -64,6 +75,11 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu)
WRITE_ONCE(vcpu->arch.irqs_pending, 0);
WRITE_ONCE(vcpu->arch.irqs_pending_mask, 0);
+
+ /* Reset the guest CSRs for hotplug usecase */
+ if (loaded)
+ kvm_arch_vcpu_load(vcpu, smp_processor_id());
+ put_cpu();
}
int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
@@ -100,6 +116,13 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
{
+ /**
+ * vcpu with id 0 is the designated boot cpu.
+ * Keep all vcpus with non-zero cpu id in power-off state so that they
+ * can brought to online using SBI HSM extension.
+ */
+ if (vcpu->vcpu_idx != 0)
+ kvm_riscv_vcpu_power_off(vcpu);
}
void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
index 3502c6166eba..bff753b6e4b0 100644
--- a/arch/riscv/kvm/vcpu_sbi.c
+++ b/arch/riscv/kvm/vcpu_sbi.c
@@ -25,6 +25,8 @@ static int kvm_linux_err_map_sbi(int err)
return SBI_ERR_INVALID_ADDRESS;
case -EOPNOTSUPP:
return SBI_ERR_NOT_SUPPORTED;
+ case -EALREADY:
+ return SBI_ERR_ALREADY_AVAILABLE;
default:
return SBI_ERR_FAILURE;
};
@@ -43,6 +45,7 @@ extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_base;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_time;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_ipi;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_rfence;
+extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_hsm;
static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
&vcpu_sbi_ext_v01,
@@ -50,6 +53,7 @@ static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
&vcpu_sbi_ext_time,
&vcpu_sbi_ext_ipi,
&vcpu_sbi_ext_rfence,
+ &vcpu_sbi_ext_hsm,
};
void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
diff --git a/arch/riscv/kvm/vcpu_sbi_hsm.c b/arch/riscv/kvm/vcpu_sbi_hsm.c
new file mode 100644
index 000000000000..aefee16a1560
--- /dev/null
+++ b/arch/riscv/kvm/vcpu_sbi_hsm.c
@@ -0,0 +1,107 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ * Atish Patra <atish.patra@wdc.com>
+ */
+
+#include <linux/errno.h>
+#include <linux/err.h>
+#include <linux/kvm_host.h>
+#include <asm/csr.h>
+#include <asm/sbi.h>
+#include <asm/kvm_vcpu_sbi.h>
+
+static int kvm_sbi_hsm_vcpu_start(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *reset_cntx;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ struct kvm_vcpu *target_vcpu;
+ unsigned long target_vcpuid = cp->a0;
+
+ target_vcpu = kvm_get_vcpu_by_id(vcpu->kvm, target_vcpuid);
+ if (!target_vcpu)
+ return -EINVAL;
+ if (!target_vcpu->arch.power_off)
+ return -EALREADY;
+
+ reset_cntx = &target_vcpu->arch.guest_reset_context;
+ /* start address */
+ reset_cntx->sepc = cp->a1;
+ /* target vcpu id to start */
+ reset_cntx->a0 = target_vcpuid;
+ /* private data passed from kernel */
+ reset_cntx->a1 = cp->a2;
+ kvm_make_request(KVM_REQ_VCPU_RESET, target_vcpu);
+
+ kvm_riscv_vcpu_power_on(target_vcpu);
+
+ return 0;
+}
+
+static int kvm_sbi_hsm_vcpu_stop(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->arch.power_off)
+ return -EINVAL;
+
+ kvm_riscv_vcpu_power_off(vcpu);
+
+ return 0;
+}
+
+static int kvm_sbi_hsm_vcpu_get_status(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ unsigned long target_vcpuid = cp->a0;
+ struct kvm_vcpu *target_vcpu;
+
+ target_vcpu = kvm_get_vcpu_by_id(vcpu->kvm, target_vcpuid);
+ if (!target_vcpu)
+ return -EINVAL;
+ if (!target_vcpu->arch.power_off)
+ return SBI_HSM_HART_STATUS_STARTED;
+ else
+ return SBI_HSM_HART_STATUS_STOPPED;
+}
+
+static int kvm_sbi_ext_hsm_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ unsigned long *out_val,
+ struct kvm_cpu_trap *utrap,
+ bool *exit)
+{
+ int ret = 0;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ struct kvm *kvm = vcpu->kvm;
+ unsigned long funcid = cp->a6;
+
+ if (!cp)
+ return -EINVAL;
+ switch (funcid) {
+ case SBI_EXT_HSM_HART_START:
+ mutex_lock(&kvm->lock);
+ ret = kvm_sbi_hsm_vcpu_start(vcpu);
+ mutex_unlock(&kvm->lock);
+ break;
+ case SBI_EXT_HSM_HART_STOP:
+ ret = kvm_sbi_hsm_vcpu_stop(vcpu);
+ break;
+ case SBI_EXT_HSM_HART_STATUS:
+ ret = kvm_sbi_hsm_vcpu_get_status(vcpu);
+ if (ret >= 0) {
+ *out_val = ret;
+ ret = 0;
+ }
+ break;
+ default:
+ ret = -EOPNOTSUPP;
+ }
+
+ return ret;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_hsm = {
+ .extid_start = SBI_EXT_HSM,
+ .extid_end = SBI_EXT_HSM,
+ .handler = kvm_sbi_ext_hsm_handler,
+};
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01
2021-11-05 23:58 ` [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01 Atish Patra
@ 2021-11-06 4:27 ` Heinrich Schuchardt
2021-11-06 4:35 ` Anup Patel
2021-11-08 16:45 ` Atish Patra
2021-11-06 4:41 ` Anup Patel
1 sibling, 2 replies; 14+ messages in thread
From: Heinrich Schuchardt @ 2021-11-06 4:27 UTC (permalink / raw)
To: Atish Patra, linux-kernel
Cc: Anup Patel, kvm-riscv, kvm, linux-riscv, Palmer Dabbelt,
Paul Walmsley, Vincent Chen, pbonzini, Sean Christopherson
Am 6. November 2021 00:58:48 MEZ schrieb Atish Patra <atish.patra@wdc.com>:
>The existing SBI specification impelementation follows v0.1
>specification. The latest specification known as v0.2 allows more
>scalability and performance improvements.
Isn't 0.3 the current SBI specification version?
Especially the system reset extension would be valuable for KVM.
(This is not meant to stop merging this patch series.)
Best regards
Heinrich
>
>Rename the existing implementation as v01 and provide a way to allow
>future extensions.
>
>Signed-off-by: Atish Patra <atish.patra@wdc.com>
>---
> arch/riscv/include/asm/kvm_vcpu_sbi.h | 29 +++++
> arch/riscv/kvm/vcpu_sbi.c | 147 +++++++++++++++++++++-----
> 2 files changed, 147 insertions(+), 29 deletions(-)
> create mode 100644 arch/riscv/include/asm/kvm_vcpu_sbi.h
>
>diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
>new file mode 100644
>index 000000000000..1a4cb0db2d0b
>--- /dev/null
>+++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
>@@ -0,0 +1,29 @@
>+/* SPDX-License-Identifier: GPL-2.0-only */
>+/**
>+ * Copyright (c) 2021 Western Digital Corporation or its affiliates.
>+ *
>+ * Authors:
>+ * Atish Patra <atish.patra@wdc.com>
>+ */
>+
>+#ifndef __RISCV_KVM_VCPU_SBI_H__
>+#define __RISCV_KVM_VCPU_SBI_H__
>+
>+#define KVM_SBI_VERSION_MAJOR 0
>+#define KVM_SBI_VERSION_MINOR 2
>+
>+struct kvm_vcpu_sbi_extension {
>+ unsigned long extid_start;
>+ unsigned long extid_end;
>+ /**
>+ * SBI extension handler. It can be defined for a given extension or group of
>+ * extension. But it should always return linux error codes rather than SBI
>+ * specific error codes.
>+ */
>+ int (*handler)(struct kvm_vcpu *vcpu, struct kvm_run *run,
>+ unsigned long *out_val, struct kvm_cpu_trap *utrap,
>+ bool *exit);
>+};
>+
>+const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid);
>+#endif /* __RISCV_KVM_VCPU_SBI_H__ */
>diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
>index eb3c045edf11..05cab5f27eee 100644
>--- a/arch/riscv/kvm/vcpu_sbi.c
>+++ b/arch/riscv/kvm/vcpu_sbi.c
>@@ -12,9 +12,25 @@
> #include <asm/csr.h>
> #include <asm/sbi.h>
> #include <asm/kvm_vcpu_timer.h>
>+#include <asm/kvm_vcpu_sbi.h>
>
>-#define SBI_VERSION_MAJOR 0
>-#define SBI_VERSION_MINOR 1
>+static int kvm_linux_err_map_sbi(int err)
>+{
>+ switch (err) {
>+ case 0:
>+ return SBI_SUCCESS;
>+ case -EPERM:
>+ return SBI_ERR_DENIED;
>+ case -EINVAL:
>+ return SBI_ERR_INVALID_PARAM;
>+ case -EFAULT:
>+ return SBI_ERR_INVALID_ADDRESS;
>+ case -EOPNOTSUPP:
>+ return SBI_ERR_NOT_SUPPORTED;
>+ default:
>+ return SBI_ERR_FAILURE;
>+ };
>+}
>
> static void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu,
> struct kvm_run *run)
>@@ -72,16 +88,17 @@ static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
> run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
> }
>
>-int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
>+static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
>+ unsigned long *out_val,
>+ struct kvm_cpu_trap *utrap,
>+ bool *exit)
> {
> ulong hmask;
>- int i, ret = 1;
>+ int i, ret = 0;
> u64 next_cycle;
> struct kvm_vcpu *rvcpu;
>- bool next_sepc = true;
> struct cpumask cm, hm;
> struct kvm *kvm = vcpu->kvm;
>- struct kvm_cpu_trap utrap = { 0 };
> struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
>
> if (!cp)
>@@ -95,8 +112,7 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> * handled in kernel so we forward these to user-space
> */
> kvm_riscv_vcpu_sbi_forward(vcpu, run);
>- next_sepc = false;
>- ret = 0;
>+ *exit = true;
> break;
> case SBI_EXT_0_1_SET_TIMER:
> #if __riscv_xlen == 32
>@@ -104,47 +120,42 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> #else
> next_cycle = (u64)cp->a0;
> #endif
>- kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
>+ ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> break;
> case SBI_EXT_0_1_CLEAR_IPI:
>- kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
>+ ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> break;
> case SBI_EXT_0_1_SEND_IPI:
> if (cp->a0)
> hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
>- &utrap);
>+ utrap);
> else
> hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
>- if (utrap.scause) {
>- utrap.sepc = cp->sepc;
>- kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
>- next_sepc = false;
>+ if (utrap->scause)
> break;
>- }
>+
> for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
>- kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
>+ ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
>+ if (ret < 0)
>+ break;
> }
> break;
> case SBI_EXT_0_1_SHUTDOWN:
> kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
>- next_sepc = false;
>- ret = 0;
>+ *exit = true;
> break;
> case SBI_EXT_0_1_REMOTE_FENCE_I:
> case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
> case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
> if (cp->a0)
> hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
>- &utrap);
>+ utrap);
> else
> hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
>- if (utrap.scause) {
>- utrap.sepc = cp->sepc;
>- kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
>- next_sepc = false;
>+ if (utrap->scause)
> break;
>- }
>+
> cpumask_clear(&cm);
> for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
>@@ -154,22 +165,100 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> }
> riscv_cpuid_to_hartid_mask(&cm, &hm);
> if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
>- sbi_remote_fence_i(cpumask_bits(&hm));
>+ ret = sbi_remote_fence_i(cpumask_bits(&hm));
> else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
>- sbi_remote_hfence_vvma(cpumask_bits(&hm),
>+ ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
> cp->a1, cp->a2);
> else
>- sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
>+ ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> cp->a1, cp->a2, cp->a3);
> break;
> default:
>+ ret = -EINVAL;
>+ break;
>+ }
>+
>+ return ret;
>+}
>+
>+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
>+ .extid_start = SBI_EXT_0_1_SET_TIMER,
>+ .extid_end = SBI_EXT_0_1_SHUTDOWN,
>+ .handler = kvm_sbi_ext_v01_handler,
>+};
>+
>+static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
>+ &vcpu_sbi_ext_v01,
>+};
>+
>+const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid)
>+{
>+ int i = 0;
>+
>+ for (i = 0; i < ARRAY_SIZE(sbi_ext); i++) {
>+ if (sbi_ext[i]->extid_start <= extid &&
>+ sbi_ext[i]->extid_end >= extid)
>+ return sbi_ext[i];
>+ }
>+
>+ return NULL;
>+}
>+
>+int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
>+{
>+ int ret = 1;
>+ bool next_sepc = true;
>+ bool userspace_exit = false;
>+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
>+ const struct kvm_vcpu_sbi_extension *sbi_ext;
>+ struct kvm_cpu_trap utrap = { 0 };
>+ unsigned long out_val = 0;
>+ bool ext_is_v01 = false;
>+
>+ if (!cp)
>+ return -EINVAL;
>+
>+ sbi_ext = kvm_vcpu_sbi_find_ext(cp->a7);
>+ if (sbi_ext && sbi_ext->handler) {
>+ if (cp->a7 >= SBI_EXT_0_1_SET_TIMER &&
>+ cp->a7 <= SBI_EXT_0_1_SHUTDOWN)
>+ ext_is_v01 = true;
>+ ret = sbi_ext->handler(vcpu, run, &out_val, &utrap, &userspace_exit);
>+ } else {
> /* Return error for unsupported SBI calls */
> cp->a0 = SBI_ERR_NOT_SUPPORTED;
>- break;
>+ goto ecall_done;
> }
>
>+ /* Handle special error cases i.e trap, exit or userspace forward */
>+ if (utrap.scause) {
>+ /* No need to increment sepc or exit ioctl loop */
>+ ret = 1;
>+ utrap.sepc = cp->sepc;
>+ kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
>+ next_sepc = false;
>+ goto ecall_done;
>+ }
>+
>+ /* Exit ioctl loop or Propagate the error code the guest */
>+ if (userspace_exit) {
>+ next_sepc = false;
>+ ret = 0;
>+ } else {
>+ /**
>+ * SBI extension handler always returns an Linux error code. Convert
>+ * it to the SBI specific error code that can be propagated the SBI
>+ * caller.
>+ */
>+ ret = kvm_linux_err_map_sbi(ret);
>+ cp->a0 = ret;
>+ ret = 1;
>+ }
>+ecall_done:
> if (next_sepc)
> cp->sepc += 4;
>+ if (!ext_is_v01)
>+ cp->a1 = out_val;
>
> return ret;
> }
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01
2021-11-06 4:27 ` Heinrich Schuchardt
@ 2021-11-06 4:35 ` Anup Patel
2021-11-08 16:45 ` Atish Patra
1 sibling, 0 replies; 14+ messages in thread
From: Anup Patel @ 2021-11-06 4:35 UTC (permalink / raw)
To: Heinrich Schuchardt
Cc: Atish Patra, linux-kernel@vger.kernel.org List, Anup Patel,
kvm-riscv, KVM General, linux-riscv, Palmer Dabbelt,
Paul Walmsley, Vincent Chen, Paolo Bonzini, Sean Christopherson
On Sat, Nov 6, 2021 at 9:58 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Am 6. November 2021 00:58:48 MEZ schrieb Atish Patra <atish.patra@wdc.com>:
> >The existing SBI specification impelementation follows v0.1
> >specification. The latest specification known as v0.2 allows more
> >scalability and performance improvements.
>
> Isn't 0.3 the current SBI specification version?
Yes, 0.3 is the current/latest SBI version but RVI wants SBI spec
to follow the non-ISA process so we are going through this process.
Meanwhile, KVM SBI v0.2 series was already on the list for quite
some time so we will have that included first and KVM SBI v0.3
series will incrementally extend KVM SBI support.
>
> Especially the system reset extension would be valuable for KVM.
Yes, both SRST and PMU extensions in SBI v0.3 are valuable
for KVM so there will be separate patches for it. I am hoping that
SBI v0.3 support in KVM will be possible for Linux-5.17.
Regards,
Anup
>
> (This is not meant to stop merging this patch series.)
>
> Best regards
>
> Heinrich
>
>
> >
> >Rename the existing implementation as v01 and provide a way to allow
> >future extensions.
> >
> >Signed-off-by: Atish Patra <atish.patra@wdc.com>
> >---
> > arch/riscv/include/asm/kvm_vcpu_sbi.h | 29 +++++
> > arch/riscv/kvm/vcpu_sbi.c | 147 +++++++++++++++++++++-----
> > 2 files changed, 147 insertions(+), 29 deletions(-)
> > create mode 100644 arch/riscv/include/asm/kvm_vcpu_sbi.h
> >
> >diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> >new file mode 100644
> >index 000000000000..1a4cb0db2d0b
> >--- /dev/null
> >+++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> >@@ -0,0 +1,29 @@
> >+/* SPDX-License-Identifier: GPL-2.0-only */
> >+/**
> >+ * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> >+ *
> >+ * Authors:
> >+ * Atish Patra <atish.patra@wdc.com>
> >+ */
> >+
> >+#ifndef __RISCV_KVM_VCPU_SBI_H__
> >+#define __RISCV_KVM_VCPU_SBI_H__
> >+
> >+#define KVM_SBI_VERSION_MAJOR 0
> >+#define KVM_SBI_VERSION_MINOR 2
> >+
> >+struct kvm_vcpu_sbi_extension {
> >+ unsigned long extid_start;
> >+ unsigned long extid_end;
> >+ /**
> >+ * SBI extension handler. It can be defined for a given extension or group of
> >+ * extension. But it should always return linux error codes rather than SBI
> >+ * specific error codes.
> >+ */
> >+ int (*handler)(struct kvm_vcpu *vcpu, struct kvm_run *run,
> >+ unsigned long *out_val, struct kvm_cpu_trap *utrap,
> >+ bool *exit);
> >+};
> >+
> >+const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid);
> >+#endif /* __RISCV_KVM_VCPU_SBI_H__ */
> >diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> >index eb3c045edf11..05cab5f27eee 100644
> >--- a/arch/riscv/kvm/vcpu_sbi.c
> >+++ b/arch/riscv/kvm/vcpu_sbi.c
> >@@ -12,9 +12,25 @@
> > #include <asm/csr.h>
> > #include <asm/sbi.h>
> > #include <asm/kvm_vcpu_timer.h>
> >+#include <asm/kvm_vcpu_sbi.h>
> >
> >-#define SBI_VERSION_MAJOR 0
> >-#define SBI_VERSION_MINOR 1
> >+static int kvm_linux_err_map_sbi(int err)
> >+{
> >+ switch (err) {
> >+ case 0:
> >+ return SBI_SUCCESS;
> >+ case -EPERM:
> >+ return SBI_ERR_DENIED;
> >+ case -EINVAL:
> >+ return SBI_ERR_INVALID_PARAM;
> >+ case -EFAULT:
> >+ return SBI_ERR_INVALID_ADDRESS;
> >+ case -EOPNOTSUPP:
> >+ return SBI_ERR_NOT_SUPPORTED;
> >+ default:
> >+ return SBI_ERR_FAILURE;
> >+ };
> >+}
> >
> > static void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu,
> > struct kvm_run *run)
> >@@ -72,16 +88,17 @@ static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
> > run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
> > }
> >
> >-int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> >+static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> >+ unsigned long *out_val,
> >+ struct kvm_cpu_trap *utrap,
> >+ bool *exit)
> > {
> > ulong hmask;
> >- int i, ret = 1;
> >+ int i, ret = 0;
> > u64 next_cycle;
> > struct kvm_vcpu *rvcpu;
> >- bool next_sepc = true;
> > struct cpumask cm, hm;
> > struct kvm *kvm = vcpu->kvm;
> >- struct kvm_cpu_trap utrap = { 0 };
> > struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> >
> > if (!cp)
> >@@ -95,8 +112,7 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > * handled in kernel so we forward these to user-space
> > */
> > kvm_riscv_vcpu_sbi_forward(vcpu, run);
> >- next_sepc = false;
> >- ret = 0;
> >+ *exit = true;
> > break;
> > case SBI_EXT_0_1_SET_TIMER:
> > #if __riscv_xlen == 32
> >@@ -104,47 +120,42 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > #else
> > next_cycle = (u64)cp->a0;
> > #endif
> >- kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> >+ ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> > break;
> > case SBI_EXT_0_1_CLEAR_IPI:
> >- kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> >+ ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> > break;
> > case SBI_EXT_0_1_SEND_IPI:
> > if (cp->a0)
> > hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> >- &utrap);
> >+ utrap);
> > else
> > hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> >- if (utrap.scause) {
> >- utrap.sepc = cp->sepc;
> >- kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> >- next_sepc = false;
> >+ if (utrap->scause)
> > break;
> >- }
> >+
> > for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> > rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> >- kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
> >+ ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
> >+ if (ret < 0)
> >+ break;
> > }
> > break;
> > case SBI_EXT_0_1_SHUTDOWN:
> > kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
> >- next_sepc = false;
> >- ret = 0;
> >+ *exit = true;
> > break;
> > case SBI_EXT_0_1_REMOTE_FENCE_I:
> > case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
> > case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
> > if (cp->a0)
> > hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> >- &utrap);
> >+ utrap);
> > else
> > hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> >- if (utrap.scause) {
> >- utrap.sepc = cp->sepc;
> >- kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> >- next_sepc = false;
> >+ if (utrap->scause)
> > break;
> >- }
> >+
> > cpumask_clear(&cm);
> > for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> > rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> >@@ -154,22 +165,100 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > }
> > riscv_cpuid_to_hartid_mask(&cm, &hm);
> > if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
> >- sbi_remote_fence_i(cpumask_bits(&hm));
> >+ ret = sbi_remote_fence_i(cpumask_bits(&hm));
> > else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
> >- sbi_remote_hfence_vvma(cpumask_bits(&hm),
> >+ ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
> > cp->a1, cp->a2);
> > else
> >- sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> >+ ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> > cp->a1, cp->a2, cp->a3);
> > break;
> > default:
> >+ ret = -EINVAL;
> >+ break;
> >+ }
> >+
> >+ return ret;
> >+}
> >+
> >+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
> >+ .extid_start = SBI_EXT_0_1_SET_TIMER,
> >+ .extid_end = SBI_EXT_0_1_SHUTDOWN,
> >+ .handler = kvm_sbi_ext_v01_handler,
> >+};
> >+
> >+static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> >+ &vcpu_sbi_ext_v01,
> >+};
> >+
> >+const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid)
> >+{
> >+ int i = 0;
> >+
> >+ for (i = 0; i < ARRAY_SIZE(sbi_ext); i++) {
> >+ if (sbi_ext[i]->extid_start <= extid &&
> >+ sbi_ext[i]->extid_end >= extid)
> >+ return sbi_ext[i];
> >+ }
> >+
> >+ return NULL;
> >+}
> >+
> >+int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> >+{
> >+ int ret = 1;
> >+ bool next_sepc = true;
> >+ bool userspace_exit = false;
> >+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> >+ const struct kvm_vcpu_sbi_extension *sbi_ext;
> >+ struct kvm_cpu_trap utrap = { 0 };
> >+ unsigned long out_val = 0;
> >+ bool ext_is_v01 = false;
> >+
> >+ if (!cp)
> >+ return -EINVAL;
> >+
> >+ sbi_ext = kvm_vcpu_sbi_find_ext(cp->a7);
> >+ if (sbi_ext && sbi_ext->handler) {
> >+ if (cp->a7 >= SBI_EXT_0_1_SET_TIMER &&
> >+ cp->a7 <= SBI_EXT_0_1_SHUTDOWN)
> >+ ext_is_v01 = true;
> >+ ret = sbi_ext->handler(vcpu, run, &out_val, &utrap, &userspace_exit);
> >+ } else {
> > /* Return error for unsupported SBI calls */
> > cp->a0 = SBI_ERR_NOT_SUPPORTED;
> >- break;
> >+ goto ecall_done;
> > }
> >
> >+ /* Handle special error cases i.e trap, exit or userspace forward */
> >+ if (utrap.scause) {
> >+ /* No need to increment sepc or exit ioctl loop */
> >+ ret = 1;
> >+ utrap.sepc = cp->sepc;
> >+ kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> >+ next_sepc = false;
> >+ goto ecall_done;
> >+ }
> >+
> >+ /* Exit ioctl loop or Propagate the error code the guest */
> >+ if (userspace_exit) {
> >+ next_sepc = false;
> >+ ret = 0;
> >+ } else {
> >+ /**
> >+ * SBI extension handler always returns an Linux error code. Convert
> >+ * it to the SBI specific error code that can be propagated the SBI
> >+ * caller.
> >+ */
> >+ ret = kvm_linux_err_map_sbi(ret);
> >+ cp->a0 = ret;
> >+ ret = 1;
> >+ }
> >+ecall_done:
> > if (next_sepc)
> > cp->sepc += 4;
> >+ if (!ext_is_v01)
> >+ cp->a1 = out_val;
> >
> > return ret;
> > }
>
>
> --
> kvm-riscv mailing list
> kvm-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01
2021-11-05 23:58 ` [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01 Atish Patra
2021-11-06 4:27 ` Heinrich Schuchardt
@ 2021-11-06 4:41 ` Anup Patel
1 sibling, 0 replies; 14+ messages in thread
From: Anup Patel @ 2021-11-06 4:41 UTC (permalink / raw)
To: Atish Patra
Cc: linux-kernel@vger.kernel.org List, Anup Patel,
Heinrich Schuchardt, kvm-riscv, KVM General, linux-riscv,
Palmer Dabbelt, Paul Walmsley, Vincent Chen, Paolo Bonzini,
Sean Christopherson
On Sat, Nov 6, 2021 at 5:29 AM Atish Patra <atish.patra@wdc.com> wrote:
>
> The existing SBI specification impelementation follows v0.1
> specification. The latest specification known as v0.2 allows more
> scalability and performance improvements.
Please update commit description to not mention v0.2 as
the latest SBI specification.
>
> Rename the existing implementation as v01 and provide a way to allow
> future extensions.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
Otherwise it looks good to me.
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Regards,
Anup
> ---
> arch/riscv/include/asm/kvm_vcpu_sbi.h | 29 +++++
> arch/riscv/kvm/vcpu_sbi.c | 147 +++++++++++++++++++++-----
> 2 files changed, 147 insertions(+), 29 deletions(-)
> create mode 100644 arch/riscv/include/asm/kvm_vcpu_sbi.h
>
> diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> new file mode 100644
> index 000000000000..1a4cb0db2d0b
> --- /dev/null
> +++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/**
> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> + *
> + * Authors:
> + * Atish Patra <atish.patra@wdc.com>
> + */
> +
> +#ifndef __RISCV_KVM_VCPU_SBI_H__
> +#define __RISCV_KVM_VCPU_SBI_H__
> +
> +#define KVM_SBI_VERSION_MAJOR 0
> +#define KVM_SBI_VERSION_MINOR 2
> +
> +struct kvm_vcpu_sbi_extension {
> + unsigned long extid_start;
> + unsigned long extid_end;
> + /**
> + * SBI extension handler. It can be defined for a given extension or group of
> + * extension. But it should always return linux error codes rather than SBI
> + * specific error codes.
> + */
> + int (*handler)(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + unsigned long *out_val, struct kvm_cpu_trap *utrap,
> + bool *exit);
> +};
> +
> +const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid);
> +#endif /* __RISCV_KVM_VCPU_SBI_H__ */
> diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> index eb3c045edf11..05cab5f27eee 100644
> --- a/arch/riscv/kvm/vcpu_sbi.c
> +++ b/arch/riscv/kvm/vcpu_sbi.c
> @@ -12,9 +12,25 @@
> #include <asm/csr.h>
> #include <asm/sbi.h>
> #include <asm/kvm_vcpu_timer.h>
> +#include <asm/kvm_vcpu_sbi.h>
>
> -#define SBI_VERSION_MAJOR 0
> -#define SBI_VERSION_MINOR 1
> +static int kvm_linux_err_map_sbi(int err)
> +{
> + switch (err) {
> + case 0:
> + return SBI_SUCCESS;
> + case -EPERM:
> + return SBI_ERR_DENIED;
> + case -EINVAL:
> + return SBI_ERR_INVALID_PARAM;
> + case -EFAULT:
> + return SBI_ERR_INVALID_ADDRESS;
> + case -EOPNOTSUPP:
> + return SBI_ERR_NOT_SUPPORTED;
> + default:
> + return SBI_ERR_FAILURE;
> + };
> +}
>
> static void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu,
> struct kvm_run *run)
> @@ -72,16 +88,17 @@ static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
> run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
> }
>
> -int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + unsigned long *out_val,
> + struct kvm_cpu_trap *utrap,
> + bool *exit)
> {
> ulong hmask;
> - int i, ret = 1;
> + int i, ret = 0;
> u64 next_cycle;
> struct kvm_vcpu *rvcpu;
> - bool next_sepc = true;
> struct cpumask cm, hm;
> struct kvm *kvm = vcpu->kvm;
> - struct kvm_cpu_trap utrap = { 0 };
> struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
>
> if (!cp)
> @@ -95,8 +112,7 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> * handled in kernel so we forward these to user-space
> */
> kvm_riscv_vcpu_sbi_forward(vcpu, run);
> - next_sepc = false;
> - ret = 0;
> + *exit = true;
> break;
> case SBI_EXT_0_1_SET_TIMER:
> #if __riscv_xlen == 32
> @@ -104,47 +120,42 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> #else
> next_cycle = (u64)cp->a0;
> #endif
> - kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> + ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> break;
> case SBI_EXT_0_1_CLEAR_IPI:
> - kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> + ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> break;
> case SBI_EXT_0_1_SEND_IPI:
> if (cp->a0)
> hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> - &utrap);
> + utrap);
> else
> hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> - if (utrap.scause) {
> - utrap.sepc = cp->sepc;
> - kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> - next_sepc = false;
> + if (utrap->scause)
> break;
> - }
> +
> for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> - kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
> + ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
> + if (ret < 0)
> + break;
> }
> break;
> case SBI_EXT_0_1_SHUTDOWN:
> kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
> - next_sepc = false;
> - ret = 0;
> + *exit = true;
> break;
> case SBI_EXT_0_1_REMOTE_FENCE_I:
> case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
> case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
> if (cp->a0)
> hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> - &utrap);
> + utrap);
> else
> hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> - if (utrap.scause) {
> - utrap.sepc = cp->sepc;
> - kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> - next_sepc = false;
> + if (utrap->scause)
> break;
> - }
> +
> cpumask_clear(&cm);
> for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> @@ -154,22 +165,100 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> }
> riscv_cpuid_to_hartid_mask(&cm, &hm);
> if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
> - sbi_remote_fence_i(cpumask_bits(&hm));
> + ret = sbi_remote_fence_i(cpumask_bits(&hm));
> else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
> - sbi_remote_hfence_vvma(cpumask_bits(&hm),
> + ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
> cp->a1, cp->a2);
> else
> - sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> + ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> cp->a1, cp->a2, cp->a3);
> break;
> default:
> + ret = -EINVAL;
> + break;
> + }
> +
> + return ret;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
> + .extid_start = SBI_EXT_0_1_SET_TIMER,
> + .extid_end = SBI_EXT_0_1_SHUTDOWN,
> + .handler = kvm_sbi_ext_v01_handler,
> +};
> +
> +static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> + &vcpu_sbi_ext_v01,
> +};
> +
> +const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid)
> +{
> + int i = 0;
> +
> + for (i = 0; i < ARRAY_SIZE(sbi_ext); i++) {
> + if (sbi_ext[i]->extid_start <= extid &&
> + sbi_ext[i]->extid_end >= extid)
> + return sbi_ext[i];
> + }
> +
> + return NULL;
> +}
> +
> +int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +{
> + int ret = 1;
> + bool next_sepc = true;
> + bool userspace_exit = false;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + const struct kvm_vcpu_sbi_extension *sbi_ext;
> + struct kvm_cpu_trap utrap = { 0 };
> + unsigned long out_val = 0;
> + bool ext_is_v01 = false;
> +
> + if (!cp)
> + return -EINVAL;
> +
> + sbi_ext = kvm_vcpu_sbi_find_ext(cp->a7);
> + if (sbi_ext && sbi_ext->handler) {
> + if (cp->a7 >= SBI_EXT_0_1_SET_TIMER &&
> + cp->a7 <= SBI_EXT_0_1_SHUTDOWN)
> + ext_is_v01 = true;
> + ret = sbi_ext->handler(vcpu, run, &out_val, &utrap, &userspace_exit);
> + } else {
> /* Return error for unsupported SBI calls */
> cp->a0 = SBI_ERR_NOT_SUPPORTED;
> - break;
> + goto ecall_done;
> }
>
> + /* Handle special error cases i.e trap, exit or userspace forward */
> + if (utrap.scause) {
> + /* No need to increment sepc or exit ioctl loop */
> + ret = 1;
> + utrap.sepc = cp->sepc;
> + kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> + next_sepc = false;
> + goto ecall_done;
> + }
> +
> + /* Exit ioctl loop or Propagate the error code the guest */
> + if (userspace_exit) {
> + next_sepc = false;
> + ret = 0;
> + } else {
> + /**
> + * SBI extension handler always returns an Linux error code. Convert
> + * it to the SBI specific error code that can be propagated the SBI
> + * caller.
> + */
> + ret = kvm_linux_err_map_sbi(ret);
> + cp->a0 = ret;
> + ret = 1;
> + }
> +ecall_done:
> if (next_sepc)
> cp->sepc += 4;
> + if (!ext_is_v01)
> + cp->a1 = out_val;
>
> return ret;
> }
> --
> 2.31.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 2/5] RISC-V: KVM: Reorganize SBI code by moving SBI v0.1 to its own file
2021-11-05 23:58 ` [PATCH v4 2/5] RISC-V: KVM: Reorganize SBI code by moving SBI v0.1 to its own file Atish Patra
@ 2021-11-06 4:43 ` Anup Patel
0 siblings, 0 replies; 14+ messages in thread
From: Anup Patel @ 2021-11-06 4:43 UTC (permalink / raw)
To: Atish Patra
Cc: linux-kernel@vger.kernel.org List, Anup Patel,
Heinrich Schuchardt, kvm-riscv, KVM General, linux-riscv,
Palmer Dabbelt, Paul Walmsley, Vincent Chen, Paolo Bonzini,
Sean Christopherson
On Sat, Nov 6, 2021 at 5:29 AM Atish Patra <atish.patra@wdc.com> wrote:
>
> With SBI v0.2, there may be more SBI extensions in future. It makes more
> sense to group related extensions in separate files. Guest kernel will
> choose appropriate SBI version dynamically.
>
> Move the existing implementation to a separate file so that it can be
> removed in future without much conflict.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
> arch/riscv/include/asm/kvm_vcpu_sbi.h | 2 +
> arch/riscv/kvm/Makefile | 1 +
> arch/riscv/kvm/vcpu_sbi.c | 151 +++-----------------------
> arch/riscv/kvm/vcpu_sbi_v01.c | 129 ++++++++++++++++++++++
> 4 files changed, 149 insertions(+), 134 deletions(-)
> create mode 100644 arch/riscv/kvm/vcpu_sbi_v01.c
>
> diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> index 1a4cb0db2d0b..704151969ceb 100644
> --- a/arch/riscv/include/asm/kvm_vcpu_sbi.h
> +++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> @@ -25,5 +25,7 @@ struct kvm_vcpu_sbi_extension {
> bool *exit);
> };
>
> +void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run);
> const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid);
> +
> #endif /* __RISCV_KVM_VCPU_SBI_H__ */
> diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
> index 30cdd1df0098..d3d5ff3a6019 100644
> --- a/arch/riscv/kvm/Makefile
> +++ b/arch/riscv/kvm/Makefile
> @@ -23,4 +23,5 @@ kvm-y += vcpu_exit.o
> kvm-y += vcpu_fp.o
> kvm-y += vcpu_switch.o
> kvm-y += vcpu_sbi.o
> +kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
> kvm-y += vcpu_timer.o
> diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> index 05cab5f27eee..06b42f6977e1 100644
> --- a/arch/riscv/kvm/vcpu_sbi.c
> +++ b/arch/riscv/kvm/vcpu_sbi.c
> @@ -9,9 +9,7 @@
> #include <linux/errno.h>
> #include <linux/err.h>
> #include <linux/kvm_host.h>
> -#include <asm/csr.h>
> #include <asm/sbi.h>
> -#include <asm/kvm_vcpu_timer.h>
> #include <asm/kvm_vcpu_sbi.h>
>
> static int kvm_linux_err_map_sbi(int err)
> @@ -32,8 +30,21 @@ static int kvm_linux_err_map_sbi(int err)
> };
> }
>
> -static void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu,
> - struct kvm_run *run)
> +#ifdef CONFIG_RISCV_SBI_V01
> +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01;
> +#else
> +static const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
> + .extid_start = -1UL,
> + .extid_end = -1UL,
> + .handler = NULL,
> +};
> +#endif
> +
> +static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> + &vcpu_sbi_ext_v01,
> +};
> +
> +void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
> {
> struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
>
> @@ -71,126 +82,6 @@ int kvm_riscv_vcpu_sbi_return(struct kvm_vcpu *vcpu, struct kvm_run *run)
> return 0;
> }
>
> -#ifdef CONFIG_RISCV_SBI_V01
> -
> -static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
> - struct kvm_run *run, u32 type)
> -{
> - int i;
> - struct kvm_vcpu *tmp;
> -
> - kvm_for_each_vcpu(i, tmp, vcpu->kvm)
> - tmp->arch.power_off = true;
> - kvm_make_all_cpus_request(vcpu->kvm, KVM_REQ_SLEEP);
> -
> - memset(&run->system_event, 0, sizeof(run->system_event));
> - run->system_event.type = type;
> - run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
> -}
> -
> -static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> - unsigned long *out_val,
> - struct kvm_cpu_trap *utrap,
> - bool *exit)
> -{
> - ulong hmask;
> - int i, ret = 0;
> - u64 next_cycle;
> - struct kvm_vcpu *rvcpu;
> - struct cpumask cm, hm;
> - struct kvm *kvm = vcpu->kvm;
> - struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> -
> - if (!cp)
> - return -EINVAL;
> -
> - switch (cp->a7) {
> - case SBI_EXT_0_1_CONSOLE_GETCHAR:
> - case SBI_EXT_0_1_CONSOLE_PUTCHAR:
> - /*
> - * The CONSOLE_GETCHAR/CONSOLE_PUTCHAR SBI calls cannot be
> - * handled in kernel so we forward these to user-space
> - */
> - kvm_riscv_vcpu_sbi_forward(vcpu, run);
> - *exit = true;
> - break;
> - case SBI_EXT_0_1_SET_TIMER:
> -#if __riscv_xlen == 32
> - next_cycle = ((u64)cp->a1 << 32) | (u64)cp->a0;
> -#else
> - next_cycle = (u64)cp->a0;
> -#endif
> - ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> - break;
> - case SBI_EXT_0_1_CLEAR_IPI:
> - ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> - break;
> - case SBI_EXT_0_1_SEND_IPI:
> - if (cp->a0)
> - hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> - utrap);
> - else
> - hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> - if (utrap->scause)
> - break;
> -
> - for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> - rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> - ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
> - if (ret < 0)
> - break;
> - }
> - break;
> - case SBI_EXT_0_1_SHUTDOWN:
> - kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
> - *exit = true;
> - break;
> - case SBI_EXT_0_1_REMOTE_FENCE_I:
> - case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
> - case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
> - if (cp->a0)
> - hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> - utrap);
> - else
> - hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> - if (utrap->scause)
> - break;
> -
> - cpumask_clear(&cm);
> - for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> - rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> - if (rvcpu->cpu < 0)
> - continue;
> - cpumask_set_cpu(rvcpu->cpu, &cm);
> - }
> - riscv_cpuid_to_hartid_mask(&cm, &hm);
> - if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
> - ret = sbi_remote_fence_i(cpumask_bits(&hm));
> - else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
> - ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
> - cp->a1, cp->a2);
> - else
> - ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> - cp->a1, cp->a2, cp->a3);
> - break;
> - default:
> - ret = -EINVAL;
> - break;
> - }
> -
> - return ret;
> -}
> -
> -const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
> - .extid_start = SBI_EXT_0_1_SET_TIMER,
> - .extid_end = SBI_EXT_0_1_SHUTDOWN,
> - .handler = kvm_sbi_ext_v01_handler,
> -};
> -
> -static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> - &vcpu_sbi_ext_v01,
> -};
> -
> const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid)
> {
> int i = 0;
> @@ -220,9 +111,11 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
>
> sbi_ext = kvm_vcpu_sbi_find_ext(cp->a7);
> if (sbi_ext && sbi_ext->handler) {
> +#ifdef CONFIG_RISCV_SBI_V01
> if (cp->a7 >= SBI_EXT_0_1_SET_TIMER &&
> cp->a7 <= SBI_EXT_0_1_SHUTDOWN)
> ext_is_v01 = true;
> +#endif
> ret = sbi_ext->handler(vcpu, run, &out_val, &utrap, &userspace_exit);
> } else {
> /* Return error for unsupported SBI calls */
> @@ -262,13 +155,3 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
>
> return ret;
> }
> -
> -#else
> -
> -int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> -{
> - kvm_riscv_vcpu_sbi_forward(vcpu, run);
> - return 0;
> -}
> -
> -#endif
> diff --git a/arch/riscv/kvm/vcpu_sbi_v01.c b/arch/riscv/kvm/vcpu_sbi_v01.c
> new file mode 100644
> index 000000000000..5a8e2afc8a57
> --- /dev/null
> +++ b/arch/riscv/kvm/vcpu_sbi_v01.c
> @@ -0,0 +1,129 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> + *
> + * Authors:
> + * Atish Patra <atish.patra@wdc.com>
> + */
> +
> +#include <linux/errno.h>
> +#include <linux/err.h>
> +#include <linux/kvm_host.h>
> +#include <asm/csr.h>
> +#include <asm/sbi.h>
> +#include <asm/kvm_vcpu_timer.h>
> +#include <asm/kvm_vcpu_sbi.h>
> +
> +static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
> + struct kvm_run *run, u32 type)
> +{
> + int i;
> + struct kvm_vcpu *tmp;
> +
> + kvm_for_each_vcpu(i, tmp, vcpu->kvm)
> + tmp->arch.power_off = true;
> + kvm_make_all_cpus_request(vcpu->kvm, KVM_REQ_SLEEP);
> +
> + memset(&run->system_event, 0, sizeof(run->system_event));
> + run->system_event.type = type;
> + run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
> +}
> +
> +static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + unsigned long *out_val,
> + struct kvm_cpu_trap *utrap,
> + bool *exit)
> +{
> + ulong hmask;
> + int i, ret = 0;
> + u64 next_cycle;
> + struct kvm_vcpu *rvcpu;
> + struct cpumask cm, hm;
> + struct kvm *kvm = vcpu->kvm;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> +
> + if (!cp)
> + return -EINVAL;
This check on "cp" is unnecessary.
> +
> + switch (cp->a7) {
> + case SBI_EXT_0_1_CONSOLE_GETCHAR:
> + case SBI_EXT_0_1_CONSOLE_PUTCHAR:
> + /*
> + * The CONSOLE_GETCHAR/CONSOLE_PUTCHAR SBI calls cannot be
> + * handled in kernel so we forward these to user-space
> + */
> + kvm_riscv_vcpu_sbi_forward(vcpu, run);
> + *exit = true;
> + break;
> + case SBI_EXT_0_1_SET_TIMER:
> +#if __riscv_xlen == 32
> + next_cycle = ((u64)cp->a1 << 32) | (u64)cp->a0;
> +#else
> + next_cycle = (u64)cp->a0;
> +#endif
> + ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> + break;
> + case SBI_EXT_0_1_CLEAR_IPI:
> + ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> + break;
> + case SBI_EXT_0_1_SEND_IPI:
> + if (cp->a0)
> + hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> + utrap);
> + else
> + hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> + if (utrap->scause)
> + break;
> +
> + for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> + rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> + ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
> + if (ret < 0)
> + break;
> + }
> + break;
> + case SBI_EXT_0_1_SHUTDOWN:
> + kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
> + *exit = true;
> + break;
> + case SBI_EXT_0_1_REMOTE_FENCE_I:
> + case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
> + case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
> + if (cp->a0)
> + hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> + utrap);
> + else
> + hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> + if (utrap->scause)
> + break;
> +
> + cpumask_clear(&cm);
> + for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> + rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> + if (rvcpu->cpu < 0)
> + continue;
> + cpumask_set_cpu(rvcpu->cpu, &cm);
> + }
> + riscv_cpuid_to_hartid_mask(&cm, &hm);
> + if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
> + ret = sbi_remote_fence_i(cpumask_bits(&hm));
> + else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
> + ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
> + cp->a1, cp->a2);
> + else
> + ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> + cp->a1, cp->a2, cp->a3);
> + break;
> + default:
> + ret = -EINVAL;
> + break;
> + };
> +
> + return ret;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
> + .extid_start = SBI_EXT_0_1_SET_TIMER,
> + .extid_end = SBI_EXT_0_1_SHUTDOWN,
> + .handler = kvm_sbi_ext_v01_handler,
> +};
> --
> 2.31.1
>
Otherwise it looks good to me.
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Regards,
Anup
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 3/5] RISC-V: KVM: Add SBI v0.2 base extension
2021-11-05 23:58 ` [PATCH v4 3/5] RISC-V: KVM: Add SBI v0.2 base extension Atish Patra
@ 2021-11-06 4:45 ` Anup Patel
0 siblings, 0 replies; 14+ messages in thread
From: Anup Patel @ 2021-11-06 4:45 UTC (permalink / raw)
To: Atish Patra
Cc: linux-kernel@vger.kernel.org List, Anup Patel,
Heinrich Schuchardt, kvm-riscv, KVM General, linux-riscv,
Palmer Dabbelt, Paul Walmsley, Vincent Chen, Paolo Bonzini,
Sean Christopherson
On Sat, Nov 6, 2021 at 5:29 AM Atish Patra <atish.patra@wdc.com> wrote:
>
> SBI v0.2 base extension defined to allow backward compatibility and
> probing of future extensions. This is also the only mandatory SBI
> extension that must be implemented by SBI implementors.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
> arch/riscv/include/asm/kvm_vcpu_sbi.h | 2 +
> arch/riscv/include/asm/sbi.h | 8 +++
> arch/riscv/kvm/Makefile | 1 +
> arch/riscv/kvm/vcpu_sbi.c | 3 +-
> arch/riscv/kvm/vcpu_sbi_base.c | 73 +++++++++++++++++++++++++++
> 5 files changed, 86 insertions(+), 1 deletion(-)
> create mode 100644 arch/riscv/kvm/vcpu_sbi_base.c
>
> diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> index 704151969ceb..76e4e17a3e00 100644
> --- a/arch/riscv/include/asm/kvm_vcpu_sbi.h
> +++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> @@ -9,6 +9,8 @@
> #ifndef __RISCV_KVM_VCPU_SBI_H__
> #define __RISCV_KVM_VCPU_SBI_H__
>
> +#define KVM_SBI_IMPID 3
> +
> #define KVM_SBI_VERSION_MAJOR 0
> #define KVM_SBI_VERSION_MINOR 2
>
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index 0d42693cb65e..4f9370b6032e 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -27,6 +27,14 @@ enum sbi_ext_id {
> SBI_EXT_IPI = 0x735049,
> SBI_EXT_RFENCE = 0x52464E43,
> SBI_EXT_HSM = 0x48534D,
> +
> + /* Experimentals extensions must lie within this range */
> + SBI_EXT_EXPERIMENTAL_START = 0x0800000,
> + SBI_EXT_EXPERIMENTAL_END = 0x08FFFFFF,
> +
> + /* Vendor extensions must lie within this range */
> + SBI_EXT_VENDOR_START = 0x09000000,
> + SBI_EXT_VENDOR_END = 0x09FFFFFF,
> };
>
> enum sbi_ext_base_fid {
> diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
> index d3d5ff3a6019..84c02922a329 100644
> --- a/arch/riscv/kvm/Makefile
> +++ b/arch/riscv/kvm/Makefile
> @@ -24,4 +24,5 @@ kvm-y += vcpu_fp.o
> kvm-y += vcpu_switch.o
> kvm-y += vcpu_sbi.o
> kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
> +kvm-y += vcpu_sbi_base.o
> kvm-y += vcpu_timer.o
> diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> index 06b42f6977e1..92b682f4f29e 100644
> --- a/arch/riscv/kvm/vcpu_sbi.c
> +++ b/arch/riscv/kvm/vcpu_sbi.c
> @@ -39,9 +39,10 @@ static const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
> .handler = NULL,
> };
> #endif
> -
> +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_base;
> static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> &vcpu_sbi_ext_v01,
> + &vcpu_sbi_ext_base,
> };
>
> void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
> diff --git a/arch/riscv/kvm/vcpu_sbi_base.c b/arch/riscv/kvm/vcpu_sbi_base.c
> new file mode 100644
> index 000000000000..1aeda3e10e7c
> --- /dev/null
> +++ b/arch/riscv/kvm/vcpu_sbi_base.c
> @@ -0,0 +1,73 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> + *
> + * Authors:
> + * Atish Patra <atish.patra@wdc.com>
> + */
> +
> +#include <linux/errno.h>
> +#include <linux/err.h>
> +#include <linux/kvm_host.h>
> +#include <asm/csr.h>
> +#include <asm/sbi.h>
> +#include <asm/kvm_vcpu_timer.h>
> +#include <asm/kvm_vcpu_sbi.h>
> +
> +static int kvm_sbi_ext_base_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + unsigned long *out_val,
> + struct kvm_cpu_trap *trap, bool *exit)
> +{
> + int ret = 0;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + struct sbiret ecall_ret;
> +
> + if (!cp)
> + return -EINVAL;
Drop the check on "cp" here.
> +
> + switch (cp->a6) {
> + case SBI_EXT_BASE_GET_SPEC_VERSION:
> + *out_val = (KVM_SBI_VERSION_MAJOR <<
> + SBI_SPEC_VERSION_MAJOR_SHIFT) |
> + KVM_SBI_VERSION_MINOR;
> + break;
> + case SBI_EXT_BASE_GET_IMP_ID:
> + *out_val = KVM_SBI_IMPID;
> + break;
> + case SBI_EXT_BASE_GET_IMP_VERSION:
> + *out_val = 0;
> + break;
> + case SBI_EXT_BASE_PROBE_EXT:
> + *out_val = kvm_vcpu_sbi_find_ext(cp->a0) ? 1 : 0;
> + if ((!*out_val) &&
> + ((cp->a0 >= SBI_EXT_EXPERIMENTAL_START &&
> + cp->a0 <= SBI_EXT_EXPERIMENTAL_END) ||
> + ((cp->a0 >= SBI_EXT_VENDOR_START &&
> + cp->a0 <= SBI_EXT_VENDOR_END)))) {
> + /* For experimental/vendor extensions forward to the userspace*/
> + kvm_riscv_vcpu_sbi_forward(vcpu, run);
> + *exit = true;
> + }
> + break;
> + case SBI_EXT_BASE_GET_MVENDORID:
> + case SBI_EXT_BASE_GET_MARCHID:
> + case SBI_EXT_BASE_GET_MIMPID:
> + ecall_ret = sbi_ecall(SBI_EXT_BASE, cp->a6, 0, 0, 0, 0, 0, 0);
> + if (!ecall_ret.error)
> + *out_val = ecall_ret.value;
> + /*TODO: We are unnecessarily converting the error twice */
> + ret = sbi_err_map_linux_errno(ecall_ret.error);
> + break;
> + default:
> + ret = -EOPNOTSUPP;
> + break;
> + }
> +
> + return ret;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_base = {
> + .extid_start = SBI_EXT_BASE,
> + .extid_end = SBI_EXT_BASE,
> + .handler = kvm_sbi_ext_base_handler,
> +};
> --
> 2.31.1
>
Otherwise it looks good to me.
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Regards,
Anup
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 4/5] RISC-V: KVM: Add v0.1 replacement SBI extensions defined in v02
2021-11-05 23:58 ` [PATCH v4 4/5] RISC-V: KVM: Add v0.1 replacement SBI extensions defined in v02 Atish Patra
@ 2021-11-06 4:46 ` Anup Patel
0 siblings, 0 replies; 14+ messages in thread
From: Anup Patel @ 2021-11-06 4:46 UTC (permalink / raw)
To: Atish Patra
Cc: linux-kernel@vger.kernel.org List, Anup Patel,
Heinrich Schuchardt, kvm-riscv, KVM General, linux-riscv,
Palmer Dabbelt, Paul Walmsley, Vincent Chen, Paolo Bonzini,
Sean Christopherson
On Sat, Nov 6, 2021 at 5:29 AM Atish Patra <atish.patra@wdc.com> wrote:
>
> The SBI v0.2 contains some of the improved versions of required v0.1
> extensions such as remote fence, timer and IPI.
>
> This patch implements those extensions.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
> arch/riscv/kvm/Makefile | 1 +
> arch/riscv/kvm/vcpu_sbi.c | 7 ++
> arch/riscv/kvm/vcpu_sbi_replace.c | 136 ++++++++++++++++++++++++++++++
> 3 files changed, 144 insertions(+)
> create mode 100644 arch/riscv/kvm/vcpu_sbi_replace.c
>
> diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
> index 84c02922a329..4757ae158bf3 100644
> --- a/arch/riscv/kvm/Makefile
> +++ b/arch/riscv/kvm/Makefile
> @@ -25,4 +25,5 @@ kvm-y += vcpu_switch.o
> kvm-y += vcpu_sbi.o
> kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
> kvm-y += vcpu_sbi_base.o
> +kvm-y += vcpu_sbi_replace.o
> kvm-y += vcpu_timer.o
> diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> index 92b682f4f29e..3502c6166eba 100644
> --- a/arch/riscv/kvm/vcpu_sbi.c
> +++ b/arch/riscv/kvm/vcpu_sbi.c
> @@ -40,9 +40,16 @@ static const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
> };
> #endif
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_base;
> +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_time;
> +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_ipi;
> +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_rfence;
> +
> static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> &vcpu_sbi_ext_v01,
> &vcpu_sbi_ext_base,
> + &vcpu_sbi_ext_time,
> + &vcpu_sbi_ext_ipi,
> + &vcpu_sbi_ext_rfence,
> };
>
> void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
> diff --git a/arch/riscv/kvm/vcpu_sbi_replace.c b/arch/riscv/kvm/vcpu_sbi_replace.c
> new file mode 100644
> index 000000000000..a80fa7691b14
> --- /dev/null
> +++ b/arch/riscv/kvm/vcpu_sbi_replace.c
> @@ -0,0 +1,136 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> + *
> + * Authors:
> + * Atish Patra <atish.patra@wdc.com>
> + */
> +
> +#include <linux/errno.h>
> +#include <linux/err.h>
> +#include <linux/kvm_host.h>
> +#include <asm/csr.h>
> +#include <asm/sbi.h>
> +#include <asm/kvm_vcpu_timer.h>
> +#include <asm/kvm_vcpu_sbi.h>
> +
> +static int kvm_sbi_ext_time_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + unsigned long *out_val,
> + struct kvm_cpu_trap *utrap, bool *exit)
> +{
> + int ret = 0;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + u64 next_cycle;
> +
> + if (!cp || (cp->a6 != SBI_EXT_TIME_SET_TIMER))
> + return -EINVAL;
> +
> +#if __riscv_xlen == 32
> + next_cycle = ((u64)cp->a1 << 32) | (u64)cp->a0;
> +#else
> + next_cycle = (u64)cp->a0;
> +#endif
> + kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> +
> + return ret;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_time = {
> + .extid_start = SBI_EXT_TIME,
> + .extid_end = SBI_EXT_TIME,
> + .handler = kvm_sbi_ext_time_handler,
> +};
> +
> +static int kvm_sbi_ext_ipi_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + unsigned long *out_val,
> + struct kvm_cpu_trap *utrap, bool *exit)
> +{
> + int i, ret = 0;
> + struct kvm_vcpu *tmp;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + unsigned long hmask = cp->a0;
> + unsigned long hbase = cp->a1;
> +
> + if (!cp || (cp->a6 != SBI_EXT_IPI_SEND_IPI))
> + return -EINVAL;
> +
> + kvm_for_each_vcpu(i, tmp, vcpu->kvm) {
> + if (hbase != -1UL) {
> + if (tmp->vcpu_id < hbase)
> + continue;
> + if (!(hmask & (1UL << (tmp->vcpu_id - hbase))))
> + continue;
> + }
> + ret = kvm_riscv_vcpu_set_interrupt(tmp, IRQ_VS_SOFT);
> + if (ret < 0)
> + break;
> + }
> +
> + return ret;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_ipi = {
> + .extid_start = SBI_EXT_IPI,
> + .extid_end = SBI_EXT_IPI,
> + .handler = kvm_sbi_ext_ipi_handler,
> +};
> +
> +static int kvm_sbi_ext_rfence_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + unsigned long *out_val,
> + struct kvm_cpu_trap *utrap, bool *exit)
> +{
> + int i, ret = 0;
> + struct cpumask cm, hm;
> + struct kvm_vcpu *tmp;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + unsigned long hmask = cp->a0;
> + unsigned long hbase = cp->a1;
> + unsigned long funcid = cp->a6;
> +
> + if (!cp)
> + return -EINVAL;
> +
> + cpumask_clear(&cm);
> + cpumask_clear(&hm);
> + kvm_for_each_vcpu(i, tmp, vcpu->kvm) {
> + if (hbase != -1UL) {
> + if (tmp->vcpu_id < hbase)
> + continue;
> + if (!(hmask & (1UL << (tmp->vcpu_id - hbase))))
> + continue;
> + }
> + if (tmp->cpu < 0)
> + continue;
> + cpumask_set_cpu(tmp->cpu, &cm);
> + }
> +
> + riscv_cpuid_to_hartid_mask(&cm, &hm);
> +
> + switch (funcid) {
> + case SBI_EXT_RFENCE_REMOTE_FENCE_I:
> + ret = sbi_remote_fence_i(cpumask_bits(&hm));
> + break;
> + case SBI_EXT_RFENCE_REMOTE_SFENCE_VMA:
> + ret = sbi_remote_hfence_vvma(cpumask_bits(&hm), cp->a2, cp->a3);
> + break;
> + case SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID:
> + ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm), cp->a2,
> + cp->a3, cp->a4);
> + break;
> + case SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA:
> + case SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID:
> + case SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA:
> + case SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID:
> + /* TODO: implement for nested hypervisor case */
> + default:
> + ret = -EOPNOTSUPP;
> + }
> +
> + return ret;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_rfence = {
> + .extid_start = SBI_EXT_RFENCE,
> + .extid_end = SBI_EXT_RFENCE,
> + .handler = kvm_sbi_ext_rfence_handler,
> +};
> --
> 2.31.1
>
Drop checks on "cp" pointer in various functions above.
Otherwise it looks good to me.
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Regards,
Anup
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 5/5] RISC-V: KVM: Add SBI HSM extension in KVM
2021-11-05 23:58 ` [PATCH v4 5/5] RISC-V: KVM: Add SBI HSM extension in KVM Atish Patra
@ 2021-11-06 4:50 ` Anup Patel
0 siblings, 0 replies; 14+ messages in thread
From: Anup Patel @ 2021-11-06 4:50 UTC (permalink / raw)
To: Atish Patra
Cc: linux-kernel@vger.kernel.org List, Anup Patel,
Heinrich Schuchardt, kvm-riscv, KVM General, linux-riscv,
Palmer Dabbelt, Paul Walmsley, Vincent Chen, Paolo Bonzini,
Sean Christopherson
On Sat, Nov 6, 2021 at 5:29 AM Atish Patra <atish.patra@wdc.com> wrote:
>
> SBI HSM extension allows OS to start/stop harts any time. It also allows
> ordered booting of harts instead of random booting.
>
> Implement SBI HSM exntesion and designate the vcpu 0 as the boot vcpu id.
> All other non-zero non-booting vcpus should be brought up by the OS
> implementing HSM extension. If the guest OS doesn't implement HSM
> extension, only single vcpu will be available to OS.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
> arch/riscv/include/asm/sbi.h | 1 +
> arch/riscv/kvm/Makefile | 1 +
> arch/riscv/kvm/vcpu.c | 23 ++++++++
> arch/riscv/kvm/vcpu_sbi.c | 4 ++
> arch/riscv/kvm/vcpu_sbi_hsm.c | 107 ++++++++++++++++++++++++++++++++++
> 5 files changed, 136 insertions(+)
> create mode 100644 arch/riscv/kvm/vcpu_sbi_hsm.c
>
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index 4f9370b6032e..79af25c45c8d 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -90,6 +90,7 @@ enum sbi_hsm_hart_status {
> #define SBI_ERR_INVALID_PARAM -3
> #define SBI_ERR_DENIED -4
> #define SBI_ERR_INVALID_ADDRESS -5
> +#define SBI_ERR_ALREADY_AVAILABLE -6
>
> extern unsigned long sbi_spec_version;
> struct sbiret {
> diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
> index 4757ae158bf3..aaf181a3d74b 100644
> --- a/arch/riscv/kvm/Makefile
> +++ b/arch/riscv/kvm/Makefile
> @@ -26,4 +26,5 @@ kvm-y += vcpu_sbi.o
> kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
> kvm-y += vcpu_sbi_base.o
> kvm-y += vcpu_sbi_replace.o
> +kvm-y += vcpu_sbi_hsm.o
> kvm-y += vcpu_timer.o
> diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
> index e3d3aed46184..50158867406d 100644
> --- a/arch/riscv/kvm/vcpu.c
> +++ b/arch/riscv/kvm/vcpu.c
> @@ -53,6 +53,17 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu)
> struct kvm_vcpu_csr *reset_csr = &vcpu->arch.guest_reset_csr;
> struct kvm_cpu_context *cntx = &vcpu->arch.guest_context;
> struct kvm_cpu_context *reset_cntx = &vcpu->arch.guest_reset_context;
> + bool loaded;
> +
> + /**
> + * The preemption should be disabled here because it races with
> + * kvm_sched_out/kvm_sched_in(called from preempt notifiers) which
> + * also calls vcpu_load/put.
> + */
> + get_cpu();
> + loaded = (vcpu->cpu != -1);
> + if (loaded)
> + kvm_arch_vcpu_put(vcpu);
>
> memcpy(csr, reset_csr, sizeof(*csr));
>
> @@ -64,6 +75,11 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu)
>
> WRITE_ONCE(vcpu->arch.irqs_pending, 0);
> WRITE_ONCE(vcpu->arch.irqs_pending_mask, 0);
> +
> + /* Reset the guest CSRs for hotplug usecase */
> + if (loaded)
> + kvm_arch_vcpu_load(vcpu, smp_processor_id());
> + put_cpu();
> }
>
> int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
> @@ -100,6 +116,13 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>
> void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
> {
> + /**
> + * vcpu with id 0 is the designated boot cpu.
> + * Keep all vcpus with non-zero cpu id in power-off state so that they
> + * can brought to online using SBI HSM extension.
> + */
> + if (vcpu->vcpu_idx != 0)
> + kvm_riscv_vcpu_power_off(vcpu);
> }
>
> void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
> diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> index 3502c6166eba..bff753b6e4b0 100644
> --- a/arch/riscv/kvm/vcpu_sbi.c
> +++ b/arch/riscv/kvm/vcpu_sbi.c
> @@ -25,6 +25,8 @@ static int kvm_linux_err_map_sbi(int err)
> return SBI_ERR_INVALID_ADDRESS;
> case -EOPNOTSUPP:
> return SBI_ERR_NOT_SUPPORTED;
> + case -EALREADY:
> + return SBI_ERR_ALREADY_AVAILABLE;
> default:
> return SBI_ERR_FAILURE;
> };
> @@ -43,6 +45,7 @@ extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_base;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_time;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_ipi;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_rfence;
> +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_hsm;
>
> static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> &vcpu_sbi_ext_v01,
> @@ -50,6 +53,7 @@ static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> &vcpu_sbi_ext_time,
> &vcpu_sbi_ext_ipi,
> &vcpu_sbi_ext_rfence,
> + &vcpu_sbi_ext_hsm,
> };
>
> void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
> diff --git a/arch/riscv/kvm/vcpu_sbi_hsm.c b/arch/riscv/kvm/vcpu_sbi_hsm.c
> new file mode 100644
> index 000000000000..aefee16a1560
> --- /dev/null
> +++ b/arch/riscv/kvm/vcpu_sbi_hsm.c
> @@ -0,0 +1,107 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> + *
> + * Authors:
> + * Atish Patra <atish.patra@wdc.com>
> + */
> +
> +#include <linux/errno.h>
> +#include <linux/err.h>
> +#include <linux/kvm_host.h>
> +#include <asm/csr.h>
> +#include <asm/sbi.h>
> +#include <asm/kvm_vcpu_sbi.h>
> +
> +static int kvm_sbi_hsm_vcpu_start(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *reset_cntx;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + struct kvm_vcpu *target_vcpu;
> + unsigned long target_vcpuid = cp->a0;
> +
> + target_vcpu = kvm_get_vcpu_by_id(vcpu->kvm, target_vcpuid);
> + if (!target_vcpu)
> + return -EINVAL;
> + if (!target_vcpu->arch.power_off)
> + return -EALREADY;
> +
> + reset_cntx = &target_vcpu->arch.guest_reset_context;
> + /* start address */
> + reset_cntx->sepc = cp->a1;
> + /* target vcpu id to start */
> + reset_cntx->a0 = target_vcpuid;
> + /* private data passed from kernel */
> + reset_cntx->a1 = cp->a2;
> + kvm_make_request(KVM_REQ_VCPU_RESET, target_vcpu);
> +
> + kvm_riscv_vcpu_power_on(target_vcpu);
> +
> + return 0;
> +}
> +
> +static int kvm_sbi_hsm_vcpu_stop(struct kvm_vcpu *vcpu)
> +{
> + if (vcpu->arch.power_off)
> + return -EINVAL;
> +
> + kvm_riscv_vcpu_power_off(vcpu);
> +
> + return 0;
> +}
> +
> +static int kvm_sbi_hsm_vcpu_get_status(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + unsigned long target_vcpuid = cp->a0;
> + struct kvm_vcpu *target_vcpu;
> +
> + target_vcpu = kvm_get_vcpu_by_id(vcpu->kvm, target_vcpuid);
> + if (!target_vcpu)
> + return -EINVAL;
> + if (!target_vcpu->arch.power_off)
> + return SBI_HSM_HART_STATUS_STARTED;
> + else
> + return SBI_HSM_HART_STATUS_STOPPED;
> +}
> +
> +static int kvm_sbi_ext_hsm_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + unsigned long *out_val,
> + struct kvm_cpu_trap *utrap,
> + bool *exit)
> +{
> + int ret = 0;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + struct kvm *kvm = vcpu->kvm;
> + unsigned long funcid = cp->a6;
> +
> + if (!cp)
> + return -EINVAL;
Same as other patches. Drop the check on "cp".
> + switch (funcid) {
> + case SBI_EXT_HSM_HART_START:
> + mutex_lock(&kvm->lock);
> + ret = kvm_sbi_hsm_vcpu_start(vcpu);
> + mutex_unlock(&kvm->lock);
> + break;
> + case SBI_EXT_HSM_HART_STOP:
> + ret = kvm_sbi_hsm_vcpu_stop(vcpu);
> + break;
> + case SBI_EXT_HSM_HART_STATUS:
> + ret = kvm_sbi_hsm_vcpu_get_status(vcpu);
> + if (ret >= 0) {
> + *out_val = ret;
> + ret = 0;
> + }
> + break;
> + default:
> + ret = -EOPNOTSUPP;
> + }
> +
> + return ret;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_hsm = {
> + .extid_start = SBI_EXT_HSM,
> + .extid_end = SBI_EXT_HSM,
> + .handler = kvm_sbi_ext_hsm_handler,
> +};
> --
> 2.31.1
>
Otherwise it looks good to me.
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Regards,
Anup
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01
2021-11-06 4:27 ` Heinrich Schuchardt
2021-11-06 4:35 ` Anup Patel
@ 2021-11-08 16:45 ` Atish Patra
1 sibling, 0 replies; 14+ messages in thread
From: Atish Patra @ 2021-11-08 16:45 UTC (permalink / raw)
To: Heinrich Schuchardt
Cc: Atish Patra, linux-kernel@vger.kernel.org List, Anup Patel,
kvm-riscv, kvm, linux-riscv, Palmer Dabbelt, Paul Walmsley,
Vincent Chen, Paolo Bonzini, Sean Christopherson
On Fri, Nov 5, 2021 at 9:28 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Am 6. November 2021 00:58:48 MEZ schrieb Atish Patra <atish.patra@wdc.com>:
> >The existing SBI specification impelementation follows v0.1
> >specification. The latest specification known as v0.2 allows more
> >scalability and performance improvements.
>
> Isn't 0.3 the current SBI specification version?
>
Yes. It was a typo. I will fix it in the next version.
Thanks for catching it.
> Especially the system reset extension would be valuable for KVM.
>
Yup. I do have a patch ready for reset extension. But I did not
include that in this series
because the reset patch for the host kernel is not merged yet. I will
send it separately.
> (This is not meant to stop merging this patch series.)
>
> Best regards
>
> Heinrich
>
>
> >
> >Rename the existing implementation as v01 and provide a way to allow
> >future extensions.
> >
> >Signed-off-by: Atish Patra <atish.patra@wdc.com>
> >---
> > arch/riscv/include/asm/kvm_vcpu_sbi.h | 29 +++++
> > arch/riscv/kvm/vcpu_sbi.c | 147 +++++++++++++++++++++-----
> > 2 files changed, 147 insertions(+), 29 deletions(-)
> > create mode 100644 arch/riscv/include/asm/kvm_vcpu_sbi.h
> >
> >diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> >new file mode 100644
> >index 000000000000..1a4cb0db2d0b
> >--- /dev/null
> >+++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> >@@ -0,0 +1,29 @@
> >+/* SPDX-License-Identifier: GPL-2.0-only */
> >+/**
> >+ * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> >+ *
> >+ * Authors:
> >+ * Atish Patra <atish.patra@wdc.com>
> >+ */
> >+
> >+#ifndef __RISCV_KVM_VCPU_SBI_H__
> >+#define __RISCV_KVM_VCPU_SBI_H__
> >+
> >+#define KVM_SBI_VERSION_MAJOR 0
> >+#define KVM_SBI_VERSION_MINOR 2
> >+
> >+struct kvm_vcpu_sbi_extension {
> >+ unsigned long extid_start;
> >+ unsigned long extid_end;
> >+ /**
> >+ * SBI extension handler. It can be defined for a given extension or group of
> >+ * extension. But it should always return linux error codes rather than SBI
> >+ * specific error codes.
> >+ */
> >+ int (*handler)(struct kvm_vcpu *vcpu, struct kvm_run *run,
> >+ unsigned long *out_val, struct kvm_cpu_trap *utrap,
> >+ bool *exit);
> >+};
> >+
> >+const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid);
> >+#endif /* __RISCV_KVM_VCPU_SBI_H__ */
> >diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> >index eb3c045edf11..05cab5f27eee 100644
> >--- a/arch/riscv/kvm/vcpu_sbi.c
> >+++ b/arch/riscv/kvm/vcpu_sbi.c
> >@@ -12,9 +12,25 @@
> > #include <asm/csr.h>
> > #include <asm/sbi.h>
> > #include <asm/kvm_vcpu_timer.h>
> >+#include <asm/kvm_vcpu_sbi.h>
> >
> >-#define SBI_VERSION_MAJOR 0
> >-#define SBI_VERSION_MINOR 1
> >+static int kvm_linux_err_map_sbi(int err)
> >+{
> >+ switch (err) {
> >+ case 0:
> >+ return SBI_SUCCESS;
> >+ case -EPERM:
> >+ return SBI_ERR_DENIED;
> >+ case -EINVAL:
> >+ return SBI_ERR_INVALID_PARAM;
> >+ case -EFAULT:
> >+ return SBI_ERR_INVALID_ADDRESS;
> >+ case -EOPNOTSUPP:
> >+ return SBI_ERR_NOT_SUPPORTED;
> >+ default:
> >+ return SBI_ERR_FAILURE;
> >+ };
> >+}
> >
> > static void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu,
> > struct kvm_run *run)
> >@@ -72,16 +88,17 @@ static void kvm_sbi_system_shutdown(struct kvm_vcpu *vcpu,
> > run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
> > }
> >
> >-int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> >+static int kvm_sbi_ext_v01_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> >+ unsigned long *out_val,
> >+ struct kvm_cpu_trap *utrap,
> >+ bool *exit)
> > {
> > ulong hmask;
> >- int i, ret = 1;
> >+ int i, ret = 0;
> > u64 next_cycle;
> > struct kvm_vcpu *rvcpu;
> >- bool next_sepc = true;
> > struct cpumask cm, hm;
> > struct kvm *kvm = vcpu->kvm;
> >- struct kvm_cpu_trap utrap = { 0 };
> > struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> >
> > if (!cp)
> >@@ -95,8 +112,7 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > * handled in kernel so we forward these to user-space
> > */
> > kvm_riscv_vcpu_sbi_forward(vcpu, run);
> >- next_sepc = false;
> >- ret = 0;
> >+ *exit = true;
> > break;
> > case SBI_EXT_0_1_SET_TIMER:
> > #if __riscv_xlen == 32
> >@@ -104,47 +120,42 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > #else
> > next_cycle = (u64)cp->a0;
> > #endif
> >- kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> >+ ret = kvm_riscv_vcpu_timer_next_event(vcpu, next_cycle);
> > break;
> > case SBI_EXT_0_1_CLEAR_IPI:
> >- kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> >+ ret = kvm_riscv_vcpu_unset_interrupt(vcpu, IRQ_VS_SOFT);
> > break;
> > case SBI_EXT_0_1_SEND_IPI:
> > if (cp->a0)
> > hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> >- &utrap);
> >+ utrap);
> > else
> > hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> >- if (utrap.scause) {
> >- utrap.sepc = cp->sepc;
> >- kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> >- next_sepc = false;
> >+ if (utrap->scause)
> > break;
> >- }
> >+
> > for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> > rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> >- kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
> >+ ret = kvm_riscv_vcpu_set_interrupt(rvcpu, IRQ_VS_SOFT);
> >+ if (ret < 0)
> >+ break;
> > }
> > break;
> > case SBI_EXT_0_1_SHUTDOWN:
> > kvm_sbi_system_shutdown(vcpu, run, KVM_SYSTEM_EVENT_SHUTDOWN);
> >- next_sepc = false;
> >- ret = 0;
> >+ *exit = true;
> > break;
> > case SBI_EXT_0_1_REMOTE_FENCE_I:
> > case SBI_EXT_0_1_REMOTE_SFENCE_VMA:
> > case SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID:
> > if (cp->a0)
> > hmask = kvm_riscv_vcpu_unpriv_read(vcpu, false, cp->a0,
> >- &utrap);
> >+ utrap);
> > else
> > hmask = (1UL << atomic_read(&kvm->online_vcpus)) - 1;
> >- if (utrap.scause) {
> >- utrap.sepc = cp->sepc;
> >- kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> >- next_sepc = false;
> >+ if (utrap->scause)
> > break;
> >- }
> >+
> > cpumask_clear(&cm);
> > for_each_set_bit(i, &hmask, BITS_PER_LONG) {
> > rvcpu = kvm_get_vcpu_by_id(vcpu->kvm, i);
> >@@ -154,22 +165,100 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > }
> > riscv_cpuid_to_hartid_mask(&cm, &hm);
> > if (cp->a7 == SBI_EXT_0_1_REMOTE_FENCE_I)
> >- sbi_remote_fence_i(cpumask_bits(&hm));
> >+ ret = sbi_remote_fence_i(cpumask_bits(&hm));
> > else if (cp->a7 == SBI_EXT_0_1_REMOTE_SFENCE_VMA)
> >- sbi_remote_hfence_vvma(cpumask_bits(&hm),
> >+ ret = sbi_remote_hfence_vvma(cpumask_bits(&hm),
> > cp->a1, cp->a2);
> > else
> >- sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> >+ ret = sbi_remote_hfence_vvma_asid(cpumask_bits(&hm),
> > cp->a1, cp->a2, cp->a3);
> > break;
> > default:
> >+ ret = -EINVAL;
> >+ break;
> >+ }
> >+
> >+ return ret;
> >+}
> >+
> >+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_v01 = {
> >+ .extid_start = SBI_EXT_0_1_SET_TIMER,
> >+ .extid_end = SBI_EXT_0_1_SHUTDOWN,
> >+ .handler = kvm_sbi_ext_v01_handler,
> >+};
> >+
> >+static const struct kvm_vcpu_sbi_extension *sbi_ext[] = {
> >+ &vcpu_sbi_ext_v01,
> >+};
> >+
> >+const struct kvm_vcpu_sbi_extension *kvm_vcpu_sbi_find_ext(unsigned long extid)
> >+{
> >+ int i = 0;
> >+
> >+ for (i = 0; i < ARRAY_SIZE(sbi_ext); i++) {
> >+ if (sbi_ext[i]->extid_start <= extid &&
> >+ sbi_ext[i]->extid_end >= extid)
> >+ return sbi_ext[i];
> >+ }
> >+
> >+ return NULL;
> >+}
> >+
> >+int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)
> >+{
> >+ int ret = 1;
> >+ bool next_sepc = true;
> >+ bool userspace_exit = false;
> >+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> >+ const struct kvm_vcpu_sbi_extension *sbi_ext;
> >+ struct kvm_cpu_trap utrap = { 0 };
> >+ unsigned long out_val = 0;
> >+ bool ext_is_v01 = false;
> >+
> >+ if (!cp)
> >+ return -EINVAL;
> >+
> >+ sbi_ext = kvm_vcpu_sbi_find_ext(cp->a7);
> >+ if (sbi_ext && sbi_ext->handler) {
> >+ if (cp->a7 >= SBI_EXT_0_1_SET_TIMER &&
> >+ cp->a7 <= SBI_EXT_0_1_SHUTDOWN)
> >+ ext_is_v01 = true;
> >+ ret = sbi_ext->handler(vcpu, run, &out_val, &utrap, &userspace_exit);
> >+ } else {
> > /* Return error for unsupported SBI calls */
> > cp->a0 = SBI_ERR_NOT_SUPPORTED;
> >- break;
> >+ goto ecall_done;
> > }
> >
> >+ /* Handle special error cases i.e trap, exit or userspace forward */
> >+ if (utrap.scause) {
> >+ /* No need to increment sepc or exit ioctl loop */
> >+ ret = 1;
> >+ utrap.sepc = cp->sepc;
> >+ kvm_riscv_vcpu_trap_redirect(vcpu, &utrap);
> >+ next_sepc = false;
> >+ goto ecall_done;
> >+ }
> >+
> >+ /* Exit ioctl loop or Propagate the error code the guest */
> >+ if (userspace_exit) {
> >+ next_sepc = false;
> >+ ret = 0;
> >+ } else {
> >+ /**
> >+ * SBI extension handler always returns an Linux error code. Convert
> >+ * it to the SBI specific error code that can be propagated the SBI
> >+ * caller.
> >+ */
> >+ ret = kvm_linux_err_map_sbi(ret);
> >+ cp->a0 = ret;
> >+ ret = 1;
> >+ }
> >+ecall_done:
> > if (next_sepc)
> > cp->sepc += 4;
> >+ if (!ext_is_v01)
> >+ cp->a1 = out_val;
> >
> > return ret;
> > }
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv
--
Regards,
Atish
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2021-11-08 16:45 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-05 23:58 [PATCH v4 0/5] Add SBI v0.2 support for KVM Atish Patra
2021-11-05 23:58 ` [PATCH v4 1/5] RISC-V: KVM: Mark the existing SBI implementation as v01 Atish Patra
2021-11-06 4:27 ` Heinrich Schuchardt
2021-11-06 4:35 ` Anup Patel
2021-11-08 16:45 ` Atish Patra
2021-11-06 4:41 ` Anup Patel
2021-11-05 23:58 ` [PATCH v4 2/5] RISC-V: KVM: Reorganize SBI code by moving SBI v0.1 to its own file Atish Patra
2021-11-06 4:43 ` Anup Patel
2021-11-05 23:58 ` [PATCH v4 3/5] RISC-V: KVM: Add SBI v0.2 base extension Atish Patra
2021-11-06 4:45 ` Anup Patel
2021-11-05 23:58 ` [PATCH v4 4/5] RISC-V: KVM: Add v0.1 replacement SBI extensions defined in v02 Atish Patra
2021-11-06 4:46 ` Anup Patel
2021-11-05 23:58 ` [PATCH v4 5/5] RISC-V: KVM: Add SBI HSM extension in KVM Atish Patra
2021-11-06 4:50 ` Anup Patel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).