All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side)
@ 2020-05-13  8:42 Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 01/15] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data) Huacai Chen
                   ` (14 more replies)
  0 siblings, 15 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

We are preparing to add KVM support for Loongson-3. VZ extension is
fully supported in Loongson-3A R4+, and we will not care about old CPUs
(at least now). We already have a full functional Linux kernel (based
on Linux-5.4.x LTS) and QEMU (based on 5.0.0) and their git repositories
are here:

QEMU: https://github.com/chenhuacai/qemu
Kernel: https://github.com/chenhuacai/linux

Of course these two repositories need to be rework and not suitable for
upstream (especially the commits need to be splitted). We show them here
is just to tell others what we have done, and how KVM/Loongson will look
like.

Our plan is make the KVM host side be upstream first, and after that,
we will make the KVM guest side and QEMU emulator be upstream.

V1 -> V2:
1, Remove "mips: define pud_index() regardless of page table folding"
   because it has been applied.
2, Make Loongson-specific code be guarded by CONFIG_CPU_LOONGSON64.

V2 -> V3:
1, Emulate a reduced feature list of CPUCFG.
2, Fix all possible checkpatch.pl errors and warnings.

V3 -> V4:
1, Emulate LOONGSON_CFG0/LOONGSON_CFG3 in CPUCFG correctly.
2, Update commit messages to explain Loongson-3 Virtual IPI.
3, Add Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>.

V4 -> V5:
1, Fix a typo.
2, Update MAINTAINERS.

Xing Li(2):
 KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
 KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits

Huacai Chen(13):
 KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16
 KVM: MIPS: Add EVENTFD support which is needed by VHOST
 KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd
 KVM: MIPS: Introduce and use cpu_guest_has_ldpte
 KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3
 KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3
 KVM: MIPS: Add more types of virtual interrupts
 KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support
 KVM: MIPS: Add CPUCFG emulation for Loongson-3
 KVM: MIPS: Add CONFIG6 and DIAG registers emulation
 KVM: MIPS: Add more MMIO load/store instructions emulation
 KVM: MIPS: Enable KVM support for Loongson-3
 MAINTAINERS: Update KVM/MIPS maintainers

Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 MAINTAINERS                          |   4 +-
 arch/mips/Kconfig                    |   1 +
 arch/mips/include/asm/cpu-features.h |   3 +
 arch/mips/include/asm/kvm_host.h     |  52 +++-
 arch/mips/include/asm/mipsregs.h     |   7 +
 arch/mips/include/uapi/asm/inst.h    |  11 +
 arch/mips/kernel/cpu-probe.c         |   2 +
 arch/mips/kvm/Kconfig                |   1 +
 arch/mips/kvm/Makefile               |   5 +-
 arch/mips/kvm/emulate.c              | 503 ++++++++++++++++++++++++++++++++++-
 arch/mips/kvm/entry.c                |  19 +-
 arch/mips/kvm/interrupt.c            |  93 +------
 arch/mips/kvm/interrupt.h            |  14 +-
 arch/mips/kvm/loongson_ipi.c         | 214 +++++++++++++++
 arch/mips/kvm/mips.c                 |  49 +++-
 arch/mips/kvm/tlb.c                  |  41 +++
 arch/mips/kvm/trap_emul.c            |   3 +
 arch/mips/kvm/vz.c                   | 237 ++++++++++++-----
 18 files changed, 1092 insertions(+), 167 deletions(-)
 create mode 100644 arch/mips/kvm/loongson_ipi.c
--
2.7.0

^ permalink raw reply	[flat|nested] 31+ messages in thread

* [PATCH V5 01/15] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-19 11:49   ` Sasha Levin
  2020-05-13  8:42 ` [PATCH V5 02/15] KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits Huacai Chen
                   ` (13 subsequent siblings)
  14 siblings, 1 reply; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Xing Li,
	stable, Huacai Chen

From: Xing Li <lixing@loongson.cn>

The code in decode_config4() of arch/mips/kernel/cpu-probe.c

        asid_mask = MIPS_ENTRYHI_ASID;
        if (config4 & MIPS_CONF4_AE)
                asid_mask |= MIPS_ENTRYHI_ASIDX;
        set_cpu_asid_mask(c, asid_mask);

set asid_mask to cpuinfo->asid_mask.

So in order to support variable ASID_MASK, KVM_ENTRYHI_ASID should also
be changed to cpu_asid_mask(&boot_cpu_data).

Cc: stable@vger.kernel.org
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Xing Li <lixing@loongson.cn>
[Huacai: Change current_cpu_data to boot_cpu_data for optimization]
Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 arch/mips/include/asm/kvm_host.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 2c343c3..a01cee9 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -275,7 +275,7 @@ enum emulation_result {
 #define MIPS3_PG_FRAME		0x3fffffc0
 
 #define VPN2_MASK		0xffffe000
-#define KVM_ENTRYHI_ASID	MIPS_ENTRYHI_ASID
+#define KVM_ENTRYHI_ASID	cpu_asid_mask(&boot_cpu_data)
 #define TLB_IS_GLOBAL(x)	((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
 #define TLB_VPN2(x)		((x).tlb_hi & VPN2_MASK)
 #define TLB_ASID(x)		((x).tlb_hi & KVM_ENTRYHI_ASID)
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 02/15] KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 01/15] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data) Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 03/15] KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16 Huacai Chen
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Xing Li,
	stable, Huacai Chen

From: Xing Li <lixing@loongson.cn>

If a CPU support more than 32bit vmbits (which is true for 64bit CPUs),
VPN2_MASK set to fixed 0xffffe000 will lead to a wrong EntryHi in some
functions such as _kvm_mips_host_tlb_inv().

The cpu_vmbits definition of 32bit CPU in cpu-features.h is 31, so we
still use the old definition.

Cc: stable@vger.kernel.org
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Xing Li <lixing@loongson.cn>
[Huacai: Improve commit messages]
Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 arch/mips/include/asm/kvm_host.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index a01cee9..caa2b936 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -274,7 +274,11 @@ enum emulation_result {
 #define MIPS3_PG_SHIFT		6
 #define MIPS3_PG_FRAME		0x3fffffc0
 
+#if defined(CONFIG_64BIT)
+#define VPN2_MASK		GENMASK(cpu_vmbits - 1, 13)
+#else
 #define VPN2_MASK		0xffffe000
+#endif
 #define KVM_ENTRYHI_ASID	cpu_asid_mask(&boot_cpu_data)
 #define TLB_IS_GLOBAL(x)	((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
 #define TLB_VPN2(x)		((x).tlb_hi & VPN2_MASK)
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 03/15] KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 01/15] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data) Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 02/15] KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 04/15] KVM: MIPS: Add EVENTFD support which is needed by VHOST Huacai Chen
                   ` (11 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

Loongson-3 based machines can have as many as 16 CPUs, and so does
memory slots, so increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16.

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index caa2b936..a7758c0 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -78,8 +78,8 @@
 #define KVM_REG_MIPS_CP0_KSCRATCH6	MIPS_CP0_64(31, 7)
 
 
-#define KVM_MAX_VCPUS		8
-#define KVM_USER_MEM_SLOTS	8
+#define KVM_MAX_VCPUS		16
+#define KVM_USER_MEM_SLOTS	16
 /* memory slots that does not exposed to userspace */
 #define KVM_PRIVATE_MEM_SLOTS	0
 
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 04/15] KVM: MIPS: Add EVENTFD support which is needed by VHOST
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (2 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 03/15] KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16 Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 05/15] KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd Huacai Chen
                   ` (10 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

Add EVENTFD support for KVM/MIPS, which is needed by VHOST. Tested on
Loongson-3 platform.

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/Kconfig     | 1 +
 arch/mips/kvm/Makefile    | 2 +-
 arch/mips/kvm/trap_emul.c | 3 +++
 arch/mips/kvm/vz.c        | 3 +++
 4 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
index b91d145..d697752 100644
--- a/arch/mips/kvm/Kconfig
+++ b/arch/mips/kvm/Kconfig
@@ -22,6 +22,7 @@ config KVM
 	select EXPORT_UASM
 	select PREEMPT_NOTIFIERS
 	select KVM_GENERIC_DIRTYLOG_READ_PROTECT
+	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_VCPU_ASYNC_IOCTL
 	select KVM_MMIO
 	select MMU_NOTIFIER
diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
index 01affc1..0a3cef6 100644
--- a/arch/mips/kvm/Makefile
+++ b/arch/mips/kvm/Makefile
@@ -2,7 +2,7 @@
 # Makefile for KVM support for MIPS
 #
 
-common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
+common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o eventfd.o)
 
 EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
 
diff --git a/arch/mips/kvm/trap_emul.c b/arch/mips/kvm/trap_emul.c
index 5a11e83..f464506b 100644
--- a/arch/mips/kvm/trap_emul.c
+++ b/arch/mips/kvm/trap_emul.c
@@ -529,6 +529,9 @@ static int kvm_trap_emul_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_MIPS_TE:
 		r = 1;
 		break;
+	case KVM_CAP_IOEVENTFD:
+		r = 1;
+		break;
 	default:
 		r = 0;
 		break;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index dde2088..17932ab 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2927,6 +2927,9 @@ static int kvm_vz_check_extension(struct kvm *kvm, long ext)
 		r = 2;
 		break;
 #endif
+	case KVM_CAP_IOEVENTFD:
+		r = 1;
+		break;
 	default:
 		r = 0;
 		break;
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 05/15] KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (3 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 04/15] KVM: MIPS: Add EVENTFD support which is needed by VHOST Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 06/15] KVM: MIPS: Introduce and use cpu_guest_has_ldpte Huacai Chen
                   ` (9 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

Loongson-3 can use lddir/ldpte instuctions to accelerate page table
walking, so use them to lookup gpa_mm.pgd.

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/entry.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 16e1c93..fd71694 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -56,6 +56,7 @@
 #define C0_BADVADDR	8, 0
 #define C0_BADINSTR	8, 1
 #define C0_BADINSTRP	8, 2
+#define C0_PGD		9, 7
 #define C0_ENTRYHI	10, 0
 #define C0_GUESTCTL1	10, 4
 #define C0_STATUS	12, 0
@@ -307,7 +308,10 @@ static void *kvm_mips_build_enter_guest(void *addr)
 
 #ifdef CONFIG_KVM_MIPS_VZ
 	/* Save normal linux process pgd (VZ guarantees pgd_reg is set) */
-	UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
+	if (cpu_has_ldpte)
+		UASM_i_MFC0(&p, K0, C0_PWBASE);
+	else
+		UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
 	UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_pgd), K1);
 
 	/*
@@ -469,8 +473,10 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	u32 *p = addr;
 	struct uasm_label labels[2];
 	struct uasm_reloc relocs[2];
+#ifndef CONFIG_CPU_LOONGSON64
 	struct uasm_label *l = labels;
 	struct uasm_reloc *r = relocs;
+#endif
 
 	memset(labels, 0, sizeof(labels));
 	memset(relocs, 0, sizeof(relocs));
@@ -490,6 +496,16 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	 */
 	preempt_disable();
 
+#ifdef CONFIG_CPU_LOONGSON64
+	UASM_i_MFC0(&p, K1, C0_PGD);
+	uasm_i_lddir(&p, K0, K1, 3);  /* global page dir */
+#ifndef __PAGETABLE_PMD_FOLDED
+	uasm_i_lddir(&p, K1, K0, 1);  /* middle page dir */
+#endif
+	uasm_i_ldpte(&p, K1, 0);      /* even */
+	uasm_i_ldpte(&p, K1, 1);      /* odd */
+	uasm_i_tlbwr(&p);
+#else
 	/*
 	 * Now for the actual refill bit. A lot of this can be common with the
 	 * Linux TLB refill handler, however we don't need to handle so many
@@ -512,6 +528,7 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	build_get_ptep(&p, K0, K1);
 	build_update_entries(&p, K0, K1);
 	build_tlb_write_entry(&p, &l, &r, tlb_random);
+#endif
 
 	preempt_enable();
 
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 06/15] KVM: MIPS: Introduce and use cpu_guest_has_ldpte
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (4 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 05/15] KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-16 15:37   ` Thomas Bogendoerfer
  2020-05-13  8:42 ` [PATCH V5 07/15] KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3 Huacai Chen
                   ` (8 subsequent siblings)
  14 siblings, 1 reply; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

Loongson-3 has lddir/ldpte instructions and their related CP0 registers
are the same as HTW. So we introduce a cpu_guest_has_ldpte flag and use
it to indicate whether we need to save/restore HTW related CP0 registers
(PWBase, PWSize, PWField and PWCtl).

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/cpu-features.h |  3 +++
 arch/mips/kernel/cpu-probe.c         |  1 +
 arch/mips/kvm/vz.c                   | 26 +++++++++++++-------------
 3 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
index 400b123..e127495 100644
--- a/arch/mips/include/asm/cpu-features.h
+++ b/arch/mips/include/asm/cpu-features.h
@@ -659,6 +659,9 @@
 #ifndef cpu_guest_has_htw
 #define cpu_guest_has_htw	(cpu_data[0].guest.options & MIPS_CPU_HTW)
 #endif
+#ifndef cpu_guest_has_ldpte
+#define cpu_guest_has_ldpte	(cpu_data[0].guest.options & MIPS_CPU_LDPTE)
+#endif
 #ifndef cpu_guest_has_mvh
 #define cpu_guest_has_mvh	(cpu_data[0].guest.options & MIPS_CPU_MVH)
 #endif
diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
index a0ef21b..319c250 100644
--- a/arch/mips/kernel/cpu-probe.c
+++ b/arch/mips/kernel/cpu-probe.c
@@ -2004,6 +2004,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
 		 * register, we correct it here.
 		 */
 		c->options |= MIPS_CPU_FTLB | MIPS_CPU_TLBINV | MIPS_CPU_LDPTE;
+		c->guest.options |= MIPS_CPU_LDPTE;
 		c->writecombine = _CACHE_UNCACHED_ACCELERATED;
 		c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
 			MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 17932ab..422cd06 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -1706,7 +1706,7 @@ static unsigned long kvm_vz_num_regs(struct kvm_vcpu *vcpu)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_contextconfig);
 	if (cpu_guest_has_segments)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
-	if (cpu_guest_has_htw)
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_htw);
 	if (cpu_guest_has_maar && !cpu_guest_has_dyn_maar)
 		ret += 1 + ARRAY_SIZE(vcpu->arch.maar);
@@ -1755,7 +1755,7 @@ static int kvm_vz_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices)
 			return -EFAULT;
 		indices += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
 	}
-	if (cpu_guest_has_htw) {
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
 		if (copy_to_user(indices, kvm_vz_get_one_regs_htw,
 				 sizeof(kvm_vz_get_one_regs_htw)))
 			return -EFAULT;
@@ -1878,17 +1878,17 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 		*v = read_gc0_segctl2();
 		break;
 	case KVM_REG_MIPS_CP0_PWBASE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwbase();
 		break;
 	case KVM_REG_MIPS_CP0_PWFIELD:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwfield();
 		break;
 	case KVM_REG_MIPS_CP0_PWSIZE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwsize();
 		break;
@@ -1896,7 +1896,7 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 		*v = (long)read_gc0_wired();
 		break;
 	case KVM_REG_MIPS_CP0_PWCTL:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwctl();
 		break;
@@ -2101,17 +2101,17 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 		write_gc0_segctl2(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWBASE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwbase(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWFIELD:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwfield(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWSIZE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwsize(v);
 		break;
@@ -2119,7 +2119,7 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 		change_gc0_wired(MIPSR6_WIRED_WIRED, v);
 		break;
 	case KVM_REG_MIPS_CP0_PWCTL:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwctl(v);
 		break;
@@ -2580,7 +2580,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	}
 
 	/* restore HTW registers */
-	if (cpu_guest_has_htw) {
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
 		kvm_restore_gc0_pwbase(cop0);
 		kvm_restore_gc0_pwfield(cop0);
 		kvm_restore_gc0_pwsize(cop0);
@@ -2685,8 +2685,8 @@ static int kvm_vz_vcpu_put(struct kvm_vcpu *vcpu, int cpu)
 	}
 
 	/* save HTW registers if enabled in guest */
-	if (cpu_guest_has_htw &&
-	    kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW) {
+	if (cpu_guest_has_ldpte || (cpu_guest_has_htw &&
+	    kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW)) {
 		kvm_save_gc0_pwbase(cop0);
 		kvm_save_gc0_pwfield(cop0);
 		kvm_save_gc0_pwsize(cop0);
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 07/15] KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (5 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 06/15] KVM: MIPS: Introduce and use cpu_guest_has_ldpte Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 08/15] KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3 Huacai Chen
                   ` (7 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

KVM guest has two levels of address translation: guest tlb translates
GVA to GPA, and root tlb translates GPA to HPA. By default guest's CCA
is controlled by guest tlb, but Loongson-3 maintains all cache coherency
by hardware (including multi-core coherency and I/O DMA coherency) so it
prefers all guest mappings be cacheable mappings. Thus, we use root tlb
to control guest's CCA for Loongson-3.

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/vz.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 422cd06..f9fbbc16 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2871,6 +2871,12 @@ static int kvm_vz_hardware_enable(void)
 	if (cpu_has_guestctl2)
 		clear_c0_guestctl2(0x3f << 10);
 
+#ifdef CONFIG_CPU_LOONGSON64
+	/* Control guest CCA attribute */
+	if (cpu_has_csr())
+		csr_writel(csr_readl(0xffffffec) | 0x1, 0xffffffec);
+#endif
+
 	return 0;
 }
 
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 08/15] KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (6 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 07/15] KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3 Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 09/15] KVM: MIPS: Add more types of virtual interrupts Huacai Chen
                   ` (6 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

Loongson-3's indexed cache operations need a node-id in the address,
but in KVM guest the node-id may be incorrect. So, let indexed cache
operations cause guest exit on Loongson-3.

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/vz.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index f9fbbc16..ab320f0 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2853,8 +2853,12 @@ static int kvm_vz_hardware_enable(void)
 	write_c0_guestctl0(MIPS_GCTL0_CP0 |
 			   (MIPS_GCTL0_AT_GUEST << MIPS_GCTL0_AT_SHIFT) |
 			   MIPS_GCTL0_CG | MIPS_GCTL0_CF);
-	if (cpu_has_guestctl0ext)
-		set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+	if (cpu_has_guestctl0ext) {
+		if (current_cpu_type() != CPU_LOONGSON64)
+			set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+		else
+			clear_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+	}
 
 	if (cpu_has_guestid) {
 		write_c0_guestctl1(0);
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 09/15] KVM: MIPS: Add more types of virtual interrupts
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (7 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 08/15] KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3 Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 10/15] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support Huacai Chen
                   ` (5 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

In current implementation, MIPS KVM uses IP2, IP3, IP4 and IP7 for
external interrupt, two kinds of IPIs and timer interrupt respectively,
but Loongson-3 based machines prefer to use IP2, IP3, IP6 and IP7 for
two kinds of external interrupts, IPI and timer interrupt. So we define
two priority-irq mapping tables: kvm_loongson3_priority_to_irq[] for
Loongson-3, and kvm_default_priority_to_irq[] for others. The virtual
interrupt infrastructure is updated to deliver all types of interrupts
from IP2, IP3, IP4, IP6 and IP7.

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/interrupt.c | 93 +++++++----------------------------------------
 arch/mips/kvm/interrupt.h | 14 ++++---
 arch/mips/kvm/mips.c      | 40 ++++++++++++++++++--
 arch/mips/kvm/vz.c        | 53 ++++-----------------------
 4 files changed, 67 insertions(+), 133 deletions(-)

diff --git a/arch/mips/kvm/interrupt.c b/arch/mips/kvm/interrupt.c
index 7257e8b6..d28c2c9c 100644
--- a/arch/mips/kvm/interrupt.c
+++ b/arch/mips/kvm/interrupt.c
@@ -61,27 +61,8 @@ void kvm_mips_queue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * the EXC code will be set when we are actually
 	 * delivering the interrupt:
 	 */
-	switch (intr) {
-	case 2:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
-		/* Queue up an INT exception for the core */
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case 3:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case 4:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_set_c0_guest_cause(vcpu->arch.cop0, 1 << (intr + 8));
+	kvm_mips_queue_irq(vcpu, kvm_irq_to_priority(intr));
 }
 
 void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
@@ -89,26 +70,8 @@ void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
 {
 	int intr = (int)irq->irq;
 
-	switch (intr) {
-	case -2:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case -3:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case -4:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_clear_c0_guest_cause(vcpu->arch.cop0, 1 << (-intr + 8));
+	kvm_mips_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
 }
 
 /* Deliver the interrupt of the corresponding priority, if possible. */
@@ -116,50 +79,20 @@ int kvm_mips_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 			    u32 cause)
 {
 	int allowed = 0;
-	u32 exccode;
+	u32 exccode, ie;
 
 	struct kvm_vcpu_arch *arch = &vcpu->arch;
 	struct mips_coproc *cop0 = vcpu->arch.cop0;
 
-	switch (priority) {
-	case MIPS_EXC_INT_TIMER:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ5)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IO:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ0)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IPI_1:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ1)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IPI_2:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ2)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
+	if (priority == MIPS_EXC_MAX)
+		return 0;
 
-	default:
-		break;
+	ie = 1 << (kvm_priority_to_irq[priority] + 8);
+	if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
+	    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
+	    && (kvm_read_c0_guest_status(cop0) & ie)) {
+		allowed = 1;
+		exccode = EXCCODE_INT;
 	}
 
 	/* Are we allowed to deliver the interrupt ??? */
diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
index 3bf0a49..c3e878c 100644
--- a/arch/mips/kvm/interrupt.h
+++ b/arch/mips/kvm/interrupt.h
@@ -21,11 +21,12 @@
 #define MIPS_EXC_NMI                5
 #define MIPS_EXC_MCHK               6
 #define MIPS_EXC_INT_TIMER          7
-#define MIPS_EXC_INT_IO             8
-#define MIPS_EXC_EXECUTE            9
-#define MIPS_EXC_INT_IPI_1          10
-#define MIPS_EXC_INT_IPI_2          11
-#define MIPS_EXC_MAX                12
+#define MIPS_EXC_INT_IO_1           8
+#define MIPS_EXC_INT_IO_2           9
+#define MIPS_EXC_EXECUTE            10
+#define MIPS_EXC_INT_IPI_1          11
+#define MIPS_EXC_INT_IPI_2          12
+#define MIPS_EXC_MAX                13
 /* XXXSL More to follow */
 
 #define C_TI        (_ULCAST_(1) << 30)
@@ -38,6 +39,9 @@
 #define KVM_MIPS_IRQ_CLEAR_ALL_AT_ONCE   (0)
 #endif
 
+extern u32 *kvm_priority_to_irq;
+u32 kvm_irq_to_priority(u32 irq);
+
 void kvm_mips_queue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
 void kvm_mips_dequeue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
 int kvm_mips_pending_timer(struct kvm_vcpu *vcpu);
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 8f05dd0..5ca122c 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -489,7 +489,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
 	int intr = (int)irq->irq;
 	struct kvm_vcpu *dvcpu = NULL;
 
-	if (intr == 3 || intr == -3 || intr == 4 || intr == -4)
+	if (intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_1] ||
+	    intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_2] ||
+	    intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_1]) ||
+	    intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_2]))
 		kvm_debug("%s: CPU: %d, INTR: %d\n", __func__, irq->cpu,
 			  (int)intr);
 
@@ -498,10 +501,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
 	else
 		dvcpu = vcpu->kvm->vcpus[irq->cpu];
 
-	if (intr == 2 || intr == 3 || intr == 4) {
+	if (intr == 2 || intr == 3 || intr == 4 || intr == 6) {
 		kvm_mips_callbacks->queue_io_int(dvcpu, irq);
 
-	} else if (intr == -2 || intr == -3 || intr == -4) {
+	} else if (intr == -2 || intr == -3 || intr == -4 || intr == -6) {
 		kvm_mips_callbacks->dequeue_io_int(dvcpu, irq);
 	} else {
 		kvm_err("%s: invalid interrupt ioctl (%d:%d)\n", __func__,
@@ -1620,6 +1623,34 @@ static struct notifier_block kvm_mips_csr_die_notifier = {
 	.notifier_call = kvm_mips_csr_die_notify,
 };
 
+static u32 kvm_default_priority_to_irq[MIPS_EXC_MAX] = {
+	[MIPS_EXC_INT_TIMER] = C_IRQ5,
+	[MIPS_EXC_INT_IO_1]  = C_IRQ0,
+	[MIPS_EXC_INT_IPI_1] = C_IRQ1,
+	[MIPS_EXC_INT_IPI_2] = C_IRQ2,
+};
+
+static u32 kvm_loongson3_priority_to_irq[MIPS_EXC_MAX] = {
+	[MIPS_EXC_INT_TIMER] = C_IRQ5,
+	[MIPS_EXC_INT_IO_1]  = C_IRQ0,
+	[MIPS_EXC_INT_IO_2]  = C_IRQ1,
+	[MIPS_EXC_INT_IPI_1] = C_IRQ4,
+};
+
+u32 *kvm_priority_to_irq = kvm_default_priority_to_irq;
+
+u32 kvm_irq_to_priority(u32 irq)
+{
+	int i;
+
+	for (i = MIPS_EXC_INT_TIMER; i < MIPS_EXC_MAX; i++) {
+		if (kvm_priority_to_irq[i] == (1 << (irq + 8)))
+			return i;
+	}
+
+	return MIPS_EXC_MAX;
+}
+
 static int __init kvm_mips_init(void)
 {
 	int ret;
@@ -1638,6 +1669,9 @@ static int __init kvm_mips_init(void)
 	if (ret)
 		return ret;
 
+	if (boot_cpu_type() == CPU_LOONGSON64)
+		kvm_priority_to_irq = kvm_loongson3_priority_to_irq;
+
 	register_die_notifier(&kvm_mips_csr_die_notifier);
 
 	return 0;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index ab320f0..63d5b35 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -225,23 +225,7 @@ static void kvm_vz_queue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * interrupts are asynchronous to vcpu execution therefore defer guest
 	 * cp0 accesses
 	 */
-	switch (intr) {
-	case 2:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case 3:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case 4:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_vz_queue_irq(vcpu, kvm_irq_to_priority(intr));
 }
 
 static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
@@ -253,44 +237,22 @@ static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * interrupts are asynchronous to vcpu execution therefore defer guest
 	 * cp0 accesses
 	 */
-	switch (intr) {
-	case -2:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case -3:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case -4:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_vz_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
 }
 
-static u32 kvm_vz_priority_to_irq[MIPS_EXC_MAX] = {
-	[MIPS_EXC_INT_TIMER] = C_IRQ5,
-	[MIPS_EXC_INT_IO]    = C_IRQ0,
-	[MIPS_EXC_INT_IPI_1] = C_IRQ1,
-	[MIPS_EXC_INT_IPI_2] = C_IRQ2,
-};
-
 static int kvm_vz_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 				 u32 cause)
 {
 	u32 irq = (priority < MIPS_EXC_MAX) ?
-		kvm_vz_priority_to_irq[priority] : 0;
+		kvm_priority_to_irq[priority] : 0;
 
 	switch (priority) {
 	case MIPS_EXC_INT_TIMER:
 		set_gc0_cause(C_TI);
 		break;
 
-	case MIPS_EXC_INT_IO:
+	case MIPS_EXC_INT_IO_1:
+	case MIPS_EXC_INT_IO_2:
 	case MIPS_EXC_INT_IPI_1:
 	case MIPS_EXC_INT_IPI_2:
 		if (cpu_has_guestctl2)
@@ -311,7 +273,7 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 			       u32 cause)
 {
 	u32 irq = (priority < MIPS_EXC_MAX) ?
-		kvm_vz_priority_to_irq[priority] : 0;
+		kvm_priority_to_irq[priority] : 0;
 
 	switch (priority) {
 	case MIPS_EXC_INT_TIMER:
@@ -329,7 +291,8 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 		}
 		break;
 
-	case MIPS_EXC_INT_IO:
+	case MIPS_EXC_INT_IO_1:
+	case MIPS_EXC_INT_IO_2:
 	case MIPS_EXC_INT_IPI_1:
 	case MIPS_EXC_INT_IPI_2:
 		/* Clear GuestCtl2.VIP irq if not using Hardware Clear */
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 10/15] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (8 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 09/15] KVM: MIPS: Add more types of virtual interrupts Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 11/15] KVM: MIPS: Add CPUCFG emulation for Loongson-3 Huacai Chen
                   ` (4 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

This patch add Loongson-3 Virtual IPI interrupt support in the kernel.
The current implementation of IPI emulation in QEMU is based on GIC for
MIPS, but Loongson-3 doesn't use GIC. Furthermore, IPI emulation in QEMU
is too expensive for performance (because of too many context switches
between Host and Guest). With current solution, the IPI delay may even
cause RCU stall warnings in a multi-core Guest. So, we design a faster
solution that emulate IPI interrupt in kernel (only used by Loongson-3
now).

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h |  32 ++++++
 arch/mips/kvm/Makefile           |   3 +
 arch/mips/kvm/emulate.c          |  23 ++++-
 arch/mips/kvm/loongson_ipi.c     | 214 +++++++++++++++++++++++++++++++++++++++
 arch/mips/kvm/mips.c             |   6 ++
 5 files changed, 277 insertions(+), 1 deletion(-)
 create mode 100644 arch/mips/kvm/loongson_ipi.c

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index a7758c0..f165902 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -23,6 +23,8 @@
 #include <asm/inst.h>
 #include <asm/mipsregs.h>
 
+#include <kvm/iodev.h>
+
 /* MIPS KVM register ids */
 #define MIPS_CP0_32(_R, _S)					\
 	(KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
@@ -181,11 +183,39 @@ struct kvm_vcpu_stat {
 struct kvm_arch_memory_slot {
 };
 
+#ifdef CONFIG_CPU_LOONGSON64
+struct ipi_state {
+	uint32_t status;
+	uint32_t en;
+	uint32_t set;
+	uint32_t clear;
+	uint64_t buf[4];
+};
+
+struct loongson_kvm_ipi;
+
+struct ipi_io_device {
+	int node_id;
+	struct loongson_kvm_ipi *ipi;
+	struct kvm_io_device device;
+};
+
+struct loongson_kvm_ipi {
+	spinlock_t lock;
+	struct kvm *kvm;
+	struct ipi_state ipistate[16];
+	struct ipi_io_device dev_ipi[4];
+};
+#endif
+
 struct kvm_arch {
 	/* Guest physical mm */
 	struct mm_struct gpa_mm;
 	/* Mask of CPUs needing GPA ASID flush */
 	cpumask_t asid_flush_mask;
+#ifdef CONFIG_CPU_LOONGSON64
+	struct loongson_kvm_ipi ipi;
+#endif
 };
 
 #define N_MIPS_COPROC_REGS	32
@@ -1133,6 +1163,8 @@ extern int kvm_mips_trans_mtc0(union mips_instruction inst, u32 *opc,
 /* Misc */
 extern void kvm_mips_dump_stats(struct kvm_vcpu *vcpu);
 extern unsigned long kvm_mips_get_ramsize(struct kvm *kvm);
+extern int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
+			     struct kvm_mips_interrupt *irq);
 
 static inline void kvm_arch_hardware_unsetup(void) {}
 static inline void kvm_arch_sync_events(struct kvm *kvm) {}
diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
index 0a3cef6..506c4ac 100644
--- a/arch/mips/kvm/Makefile
+++ b/arch/mips/kvm/Makefile
@@ -13,6 +13,9 @@ kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
 	    fpu.o
 kvm-objs += hypcall.o
 kvm-objs += mmu.o
+ifdef CONFIG_CPU_LOONGSON64
+kvm-objs += loongson_ipi.o
+endif
 
 ifdef CONFIG_KVM_MIPS_VZ
 kvm-objs		+= vz.o
diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
index 8c80333..dfc92e5 100644
--- a/arch/mips/kvm/emulate.c
+++ b/arch/mips/kvm/emulate.c
@@ -1600,6 +1600,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 					     struct kvm_run *run,
 					     struct kvm_vcpu *vcpu)
 {
+	int r;
 	enum emulation_result er;
 	u32 rt;
 	void *data = run->mmio.data;
@@ -1666,9 +1667,18 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 		goto out_fail;
 	}
 
-	run->mmio.is_write = 1;
 	vcpu->mmio_needed = 1;
+	run->mmio.is_write = 1;
 	vcpu->mmio_is_write = 1;
+
+	r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS,
+			run->mmio.phys_addr, run->mmio.len, data);
+
+	if (!r) {
+		vcpu->mmio_needed = 0;
+		return EMULATE_DONE;
+	}
+
 	return EMULATE_DO_MMIO;
 
 out_fail:
@@ -1681,6 +1691,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 					    u32 cause, struct kvm_run *run,
 					    struct kvm_vcpu *vcpu)
 {
+	int r;
 	enum emulation_result er;
 	unsigned long curr_pc;
 	u32 op, rt;
@@ -1745,6 +1756,16 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 
 	run->mmio.is_write = 0;
 	vcpu->mmio_is_write = 0;
+
+	r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS,
+			run->mmio.phys_addr, run->mmio.len, run->mmio.data);
+
+	if (!r) {
+		kvm_mips_complete_mmio_load(vcpu, run);
+		vcpu->mmio_needed = 0;
+		return EMULATE_DONE;
+	}
+
 	return EMULATE_DO_MMIO;
 }
 
diff --git a/arch/mips/kvm/loongson_ipi.c b/arch/mips/kvm/loongson_ipi.c
new file mode 100644
index 00000000..3681fc8
--- /dev/null
+++ b/arch/mips/kvm/loongson_ipi.c
@@ -0,0 +1,214 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Loongson-3 Virtual IPI interrupt support.
+ *
+ * Copyright (C) 2019  Loongson Technologies, Inc.  All rights reserved.
+ *
+ * Authors: Chen Zhu <zhuchen@loongson.cn>
+ * Authors: Huacai Chen <chenhc@lemote.com>
+ */
+
+#include <linux/kvm_host.h>
+
+#define IPI_BASE            0x3ff01000ULL
+
+#define CORE0_STATUS_OFF       0x000
+#define CORE0_EN_OFF           0x004
+#define CORE0_SET_OFF          0x008
+#define CORE0_CLEAR_OFF        0x00c
+#define CORE0_BUF_20           0x020
+#define CORE0_BUF_28           0x028
+#define CORE0_BUF_30           0x030
+#define CORE0_BUF_38           0x038
+
+#define CORE1_STATUS_OFF       0x100
+#define CORE1_EN_OFF           0x104
+#define CORE1_SET_OFF          0x108
+#define CORE1_CLEAR_OFF        0x10c
+#define CORE1_BUF_20           0x120
+#define CORE1_BUF_28           0x128
+#define CORE1_BUF_30           0x130
+#define CORE1_BUF_38           0x138
+
+#define CORE2_STATUS_OFF       0x200
+#define CORE2_EN_OFF           0x204
+#define CORE2_SET_OFF          0x208
+#define CORE2_CLEAR_OFF        0x20c
+#define CORE2_BUF_20           0x220
+#define CORE2_BUF_28           0x228
+#define CORE2_BUF_30           0x230
+#define CORE2_BUF_38           0x238
+
+#define CORE3_STATUS_OFF       0x300
+#define CORE3_EN_OFF           0x304
+#define CORE3_SET_OFF          0x308
+#define CORE3_CLEAR_OFF        0x30c
+#define CORE3_BUF_20           0x320
+#define CORE3_BUF_28           0x328
+#define CORE3_BUF_30           0x330
+#define CORE3_BUF_38           0x338
+
+static int loongson_vipi_read(struct loongson_kvm_ipi *ipi,
+				gpa_t addr, int len, void *val)
+{
+	uint32_t core = (addr >> 8) & 3;
+	uint32_t node = (addr >> 44) & 3;
+	uint32_t id = core + node * 4;
+	uint64_t offset = addr & 0xff;
+	void *pbuf;
+	struct ipi_state *s = &(ipi->ipistate[id]);
+
+	BUG_ON(offset & (len - 1));
+
+	switch (offset) {
+	case CORE0_STATUS_OFF:
+		*(uint64_t *)val = s->status;
+		break;
+
+	case CORE0_EN_OFF:
+		*(uint64_t *)val = s->en;
+		break;
+
+	case CORE0_SET_OFF:
+		*(uint64_t *)val = 0;
+		break;
+
+	case CORE0_CLEAR_OFF:
+		*(uint64_t *)val = 0;
+		break;
+
+	case CORE0_BUF_20 ... CORE0_BUF_38:
+		pbuf = (void *)s->buf + (offset - 0x20);
+		if (len == 8)
+			*(uint64_t *)val = *(uint64_t *)pbuf;
+		else /* Assume len == 4 */
+			*(uint32_t *)val = *(uint32_t *)pbuf;
+		break;
+
+	default:
+		pr_notice("%s with unknown addr %llx\n", __func__, addr);
+		break;
+	}
+
+	return 0;
+}
+
+static int loongson_vipi_write(struct loongson_kvm_ipi *ipi,
+				gpa_t addr, int len, const void *val)
+{
+	uint32_t core = (addr >> 8) & 3;
+	uint32_t node = (addr >> 44) & 3;
+	uint32_t id = core + node * 4;
+	uint64_t data, offset = addr & 0xff;
+	void *pbuf;
+	struct kvm *kvm = ipi->kvm;
+	struct kvm_mips_interrupt irq;
+	struct ipi_state *s = &(ipi->ipistate[id]);
+
+	data = *(uint64_t *)val;
+	BUG_ON(offset & (len - 1));
+
+	switch (offset) {
+	case CORE0_STATUS_OFF:
+		break;
+
+	case CORE0_EN_OFF:
+		s->en = data;
+		break;
+
+	case CORE0_SET_OFF:
+		s->status |= data;
+		irq.cpu = id;
+		irq.irq = 6;
+		kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
+		break;
+
+	case CORE0_CLEAR_OFF:
+		s->status &= ~data;
+		if (!s->status) {
+			irq.cpu = id;
+			irq.irq = -6;
+			kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
+		}
+		break;
+
+	case CORE0_BUF_20 ... CORE0_BUF_38:
+		pbuf = (void *)s->buf + (offset - 0x20);
+		if (len == 8)
+			*(uint64_t *)pbuf = (uint64_t)data;
+		else /* Assume len == 4 */
+			*(uint32_t *)pbuf = (uint32_t)data;
+		break;
+
+	default:
+		pr_notice("%s with unknown addr %llx\n", __func__, addr);
+		break;
+	}
+
+	return 0;
+}
+
+static int kvm_ipi_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			gpa_t addr, int len, void *val)
+{
+	unsigned long flags;
+	struct loongson_kvm_ipi *ipi;
+	struct ipi_io_device *ipi_device;
+
+	ipi_device = container_of(dev, struct ipi_io_device, device);
+	ipi = ipi_device->ipi;
+
+	spin_lock_irqsave(&ipi->lock, flags);
+	loongson_vipi_read(ipi, addr, len, val);
+	spin_unlock_irqrestore(&ipi->lock, flags);
+
+	return 0;
+}
+
+static int kvm_ipi_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			gpa_t addr, int len, const void *val)
+{
+	unsigned long flags;
+	struct loongson_kvm_ipi *ipi;
+	struct ipi_io_device *ipi_device;
+
+	ipi_device = container_of(dev, struct ipi_io_device, device);
+	ipi = ipi_device->ipi;
+
+	spin_lock_irqsave(&ipi->lock, flags);
+	loongson_vipi_write(ipi, addr, len, val);
+	spin_unlock_irqrestore(&ipi->lock, flags);
+
+	return 0;
+}
+
+static const struct kvm_io_device_ops kvm_ipi_ops = {
+	.read     = kvm_ipi_read,
+	.write    = kvm_ipi_write,
+};
+
+void kvm_init_loongson_ipi(struct kvm *kvm)
+{
+	int i;
+	unsigned long addr;
+	struct loongson_kvm_ipi *s;
+	struct kvm_io_device *device;
+
+	s = &kvm->arch.ipi;
+	s->kvm = kvm;
+	spin_lock_init(&s->lock);
+
+	/*
+	 * Initialize IPI device
+	 */
+	for (i = 0; i < 4; i++) {
+		device = &s->dev_ipi[i].device;
+		kvm_iodevice_init(device, &kvm_ipi_ops);
+		addr = (((unsigned long)i) << 44) + IPI_BASE;
+		mutex_lock(&kvm->slots_lock);
+		kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, addr, 0x400, device);
+		mutex_unlock(&kvm->slots_lock);
+		s->dev_ipi[i].ipi = s;
+		s->dev_ipi[i].node_id = i;
+	}
+}
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 5ca122c..ed989ef 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -128,6 +128,8 @@ int kvm_arch_check_processor_compat(void *opaque)
 	return 0;
 }
 
+extern void kvm_init_loongson_ipi(struct kvm *kvm);
+
 int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 {
 	switch (type) {
@@ -147,6 +149,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	if (!kvm->arch.gpa_mm.pgd)
 		return -ENOMEM;
 
+#ifdef CONFIG_CPU_LOONGSON64
+	kvm_init_loongson_ipi(kvm);
+#endif
+
 	return 0;
 }
 
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 11/15] KVM: MIPS: Add CPUCFG emulation for Loongson-3
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (9 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 10/15] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 12/15] KVM: MIPS: Add CONFIG6 and DIAG registers emulation Huacai Chen
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

Loongson-3 overrides lwc2 instructions to implement CPUCFG and CSR
read/write functions. These instructions all cause guest exit so CSR
doesn't benifit KVM guest (and there are always legacy methods to
provide the same functions as CSR). So, we only emulate CPUCFG and let
it return a reduced feature list (which means the virtual CPU doesn't
have any other advanced features, including CSR) in KVM.

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h  |  3 ++
 arch/mips/include/uapi/asm/inst.h | 11 ++++++
 arch/mips/kvm/mips.c              |  3 ++
 arch/mips/kvm/vz.c                | 77 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 94 insertions(+)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index f165902..3fd2f1c 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -173,6 +173,9 @@ struct kvm_vcpu_stat {
 	u64 vz_ghfc_exits;
 	u64 vz_gpa_exits;
 	u64 vz_resvd_exits;
+#ifdef CONFIG_CPU_LOONGSON64
+	u64 vz_cpucfg_exits;
+#endif
 #endif
 	u64 halt_successful_poll;
 	u64 halt_attempted_poll;
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 98f97c8..43d1faa 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -1012,6 +1012,16 @@ struct loongson3_lsdc2_format {	/* Loongson-3 overridden ldc2/sdc2 Load/Store fo
 	;))))))
 };
 
+struct loongson3_lscsr_format {	/* Loongson-3 CPUCFG&CSR read/write format */
+	__BITFIELD_FIELD(unsigned int opcode : 6,
+	__BITFIELD_FIELD(unsigned int rs : 5,
+	__BITFIELD_FIELD(unsigned int fr : 5,
+	__BITFIELD_FIELD(unsigned int rd : 5,
+	__BITFIELD_FIELD(unsigned int fd : 5,
+	__BITFIELD_FIELD(unsigned int func : 6,
+	;))))))
+};
+
 /*
  * MIPS16e instruction formats (16-bit length)
  */
@@ -1114,6 +1124,7 @@ union mips_instruction {
 	struct mm16_r5_format mm16_r5_format;
 	struct loongson3_lswc2_format loongson3_lswc2_format;
 	struct loongson3_lsdc2_format loongson3_lsdc2_format;
+	struct loongson3_lscsr_format loongson3_lscsr_format;
 };
 
 union mips16e_instruction {
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index ed989ef..9362769 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -68,6 +68,9 @@ struct kvm_stats_debugfs_item debugfs_entries[] = {
 	{ "vz_ghfc",	  VCPU_STAT(vz_ghfc_exits),	 KVM_STAT_VCPU },
 	{ "vz_gpa",	  VCPU_STAT(vz_gpa_exits),	 KVM_STAT_VCPU },
 	{ "vz_resvd",	  VCPU_STAT(vz_resvd_exits),	 KVM_STAT_VCPU },
+#ifdef CONFIG_CPU_LOONGSON64
+	{ "vz_cpucfg",	  VCPU_STAT(vz_cpucfg_exits),	 KVM_STAT_VCPU },
+#endif
 #endif
 	{ "halt_successful_poll", VCPU_STAT(halt_successful_poll), KVM_STAT_VCPU },
 	{ "halt_attempted_poll", VCPU_STAT(halt_attempted_poll), KVM_STAT_VCPU },
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 63d5b35..72a62f1 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -29,6 +29,7 @@
 #include <linux/kvm_host.h>
 
 #include "interrupt.h"
+#include "loongson_regs.h"
 
 #include "trace.h"
 
@@ -1092,6 +1093,77 @@ static enum emulation_result kvm_vz_gpsi_cache(union mips_instruction inst,
 	return EMULATE_FAIL;
 }
 
+#ifdef CONFIG_CPU_LOONGSON64
+static enum emulation_result kvm_vz_gpsi_lwc2(union mips_instruction inst,
+					      u32 *opc, u32 cause,
+					      struct kvm_run *run,
+					      struct kvm_vcpu *vcpu)
+{
+	unsigned int rs, rd;
+	unsigned int hostcfg;
+	unsigned long curr_pc;
+	enum emulation_result er = EMULATE_DONE;
+
+	/*
+	 * Update PC and hold onto current PC in case there is
+	 * an error and we want to rollback the PC
+	 */
+	curr_pc = vcpu->arch.pc;
+	er = update_pc(vcpu, cause);
+	if (er == EMULATE_FAIL)
+		return er;
+
+	rs = inst.loongson3_lscsr_format.rs;
+	rd = inst.loongson3_lscsr_format.rd;
+	switch (inst.loongson3_lscsr_format.fr) {
+	case 0x8:  /* Read CPUCFG */
+		++vcpu->stat.vz_cpucfg_exits;
+		hostcfg = read_cpucfg(vcpu->arch.gprs[rs]);
+
+		switch (vcpu->arch.gprs[rs]) {
+		case LOONGSON_CFG0:
+			vcpu->arch.gprs[rd] = 0x14c000;
+			break;
+		case LOONGSON_CFG1:
+			hostcfg &= (LOONGSON_CFG1_FP | LOONGSON_CFG1_MMI |
+				    LOONGSON_CFG1_MSA1 | LOONGSON_CFG1_MSA2 |
+				    LOONGSON_CFG1_SFBP);
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		case LOONGSON_CFG2:
+			hostcfg &= (LOONGSON_CFG2_LEXT1 | LOONGSON_CFG2_LEXT2 |
+				    LOONGSON_CFG2_LEXT3 | LOONGSON_CFG2_LSPW);
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		case LOONGSON_CFG3:
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		default:
+			/* Don't export any other advanced features to guest */
+			vcpu->arch.gprs[rd] = 0;
+			break;
+		}
+		break;
+
+	default:
+		kvm_err("lwc2 emulate not impl %d rs %lx @%lx\n",
+			inst.loongson3_lscsr_format.fr, vcpu->arch.gprs[rs], curr_pc);
+		er = EMULATE_FAIL;
+		break;
+	}
+
+	/* Rollback PC only if emulation was unsuccessful */
+	if (er == EMULATE_FAIL) {
+		kvm_err("[%#lx]%s: unsupported lwc2 instruction 0x%08x 0x%08x\n",
+			curr_pc, __func__, inst.word, inst.loongson3_lscsr_format.fr);
+
+		vcpu->arch.pc = curr_pc;
+	}
+
+	return er;
+}
+#endif
+
 static enum emulation_result kvm_trap_vz_handle_gpsi(u32 cause, u32 *opc,
 						     struct kvm_vcpu *vcpu)
 {
@@ -1121,6 +1193,11 @@ static enum emulation_result kvm_trap_vz_handle_gpsi(u32 cause, u32 *opc,
 		er = kvm_vz_gpsi_cache(inst, opc, cause, run, vcpu);
 		break;
 #endif
+#ifdef CONFIG_CPU_LOONGSON64
+	case lwc2_op:
+		er = kvm_vz_gpsi_lwc2(inst, opc, cause, run, vcpu);
+		break;
+#endif
 	case spec3_op:
 		switch (inst.spec3_format.func) {
 #ifdef CONFIG_CPU_MIPSR6
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 12/15] KVM: MIPS: Add CONFIG6 and DIAG registers emulation
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (10 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 11/15] KVM: MIPS: Add CPUCFG emulation for Loongson-3 Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-14  7:45   ` Aleksandar Markovic
  2020-05-16 15:36   ` Thomas Bogendoerfer
  2020-05-13  8:42 ` [PATCH V5 13/15] KVM: MIPS: Add more MMIO load/store instructions emulation Huacai Chen
                   ` (2 subsequent siblings)
  14 siblings, 2 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

Loongson-3 has CONFIG6 and DIAG registers which need to be emulated.
CONFIG6 is mostly used to enable/disable FTLB and SFB, while DIAG is
mostly used to flush BTB, ITLB, DTLB, VTLB and FTLB.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h |  7 +++++
 arch/mips/include/asm/mipsregs.h |  7 +++++
 arch/mips/kvm/tlb.c              | 41 ++++++++++++++++++++++++++
 arch/mips/kvm/vz.c               | 62 +++++++++++++++++++++++++++++++++++++++-
 4 files changed, 116 insertions(+), 1 deletion(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 3fd2f1c..30b5e33 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -68,9 +68,11 @@
 #define KVM_REG_MIPS_CP0_CONFIG3	MIPS_CP0_32(16, 3)
 #define KVM_REG_MIPS_CP0_CONFIG4	MIPS_CP0_32(16, 4)
 #define KVM_REG_MIPS_CP0_CONFIG5	MIPS_CP0_32(16, 5)
+#define KVM_REG_MIPS_CP0_CONFIG6	MIPS_CP0_32(16, 6)
 #define KVM_REG_MIPS_CP0_CONFIG7	MIPS_CP0_32(16, 7)
 #define KVM_REG_MIPS_CP0_MAARI		MIPS_CP0_64(17, 2)
 #define KVM_REG_MIPS_CP0_XCONTEXT	MIPS_CP0_64(20, 0)
+#define KVM_REG_MIPS_CP0_DIAG		MIPS_CP0_32(22, 0)
 #define KVM_REG_MIPS_CP0_ERROREPC	MIPS_CP0_64(30, 0)
 #define KVM_REG_MIPS_CP0_KSCRATCH1	MIPS_CP0_64(31, 2)
 #define KVM_REG_MIPS_CP0_KSCRATCH2	MIPS_CP0_64(31, 3)
@@ -256,6 +258,7 @@ struct mips_coproc {
 #define MIPS_CP0_WATCH_LO	18
 #define MIPS_CP0_WATCH_HI	19
 #define MIPS_CP0_TLB_XCONTEXT	20
+#define MIPS_CP0_DIAG		22
 #define MIPS_CP0_ECC		26
 #define MIPS_CP0_CACHE_ERR	27
 #define MIPS_CP0_TAG_LO		28
@@ -927,6 +930,10 @@ void kvm_vz_save_guesttlb(struct kvm_mips_tlb *buf, unsigned int index,
 			  unsigned int count);
 void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
 			  unsigned int count);
+#ifdef CONFIG_CPU_LOONGSON64
+void kvm_loongson_clear_guest_vtlb(void);
+void kvm_loongson_clear_guest_ftlb(void);
+#endif
 #endif
 
 void kvm_mips_suspend_mm(int cpu);
diff --git a/arch/mips/include/asm/mipsregs.h b/arch/mips/include/asm/mipsregs.h
index 796fe47..ce40fbf 100644
--- a/arch/mips/include/asm/mipsregs.h
+++ b/arch/mips/include/asm/mipsregs.h
@@ -674,6 +674,9 @@
 #define MIPS_CONF5_CV		(_ULCAST_(1) << 29)
 #define MIPS_CONF5_K		(_ULCAST_(1) << 30)
 
+#define MIPS_CONF6_INTIMER	(_ULCAST_(1) << 6)
+#define MIPS_CONF6_EXTIMER	(_ULCAST_(1) << 7)
+#define MIPS_CONF6_SFBEN	(_ULCAST_(1) << 8)
 #define MIPS_CONF6_SYND		(_ULCAST_(1) << 13)
 /* proAptiv FTLB on/off bit */
 #define MIPS_CONF6_FTLBEN	(_ULCAST_(1) << 15)
@@ -993,6 +996,8 @@
 /* Disable Branch Return Cache */
 #define R10K_DIAG_D_BRC		(_ULCAST_(1) << 22)
 
+/* Flush BTB */
+#define LOONGSON_DIAG_BTB	(_ULCAST_(1) << 1)
 /* Flush ITLB */
 #define LOONGSON_DIAG_ITLB	(_ULCAST_(1) << 2)
 /* Flush DTLB */
@@ -2825,7 +2830,9 @@ __BUILD_SET_C0(status)
 __BUILD_SET_C0(cause)
 __BUILD_SET_C0(config)
 __BUILD_SET_C0(config5)
+__BUILD_SET_C0(config6)
 __BUILD_SET_C0(config7)
+__BUILD_SET_C0(diag)
 __BUILD_SET_C0(intcontrol)
 __BUILD_SET_C0(intctl)
 __BUILD_SET_C0(srsmap)
diff --git a/arch/mips/kvm/tlb.c b/arch/mips/kvm/tlb.c
index 7cd9216..1418715 100644
--- a/arch/mips/kvm/tlb.c
+++ b/arch/mips/kvm/tlb.c
@@ -20,6 +20,7 @@
 
 #include <asm/cpu.h>
 #include <asm/bootinfo.h>
+#include <asm/mipsregs.h>
 #include <asm/mmu_context.h>
 #include <asm/pgtable.h>
 #include <asm/cacheflush.h>
@@ -622,6 +623,46 @@ void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
 }
 EXPORT_SYMBOL_GPL(kvm_vz_load_guesttlb);
 
+#ifdef CONFIG_CPU_LOONGSON64
+void kvm_loongson_clear_guest_vtlb(void)
+{
+	int idx = read_gc0_index();
+
+	/* Set root GuestID for root probe and write of guest TLB entry */
+	set_root_gid_to_guest_gid();
+
+	write_gc0_index(0);
+	guest_tlbinvf();
+	write_gc0_index(idx);
+
+	clear_root_gid();
+	set_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
+}
+EXPORT_SYMBOL_GPL(kvm_loongson_clear_guest_vtlb);
+
+void kvm_loongson_clear_guest_ftlb(void)
+{
+	int i;
+	int idx = read_gc0_index();
+
+	/* Set root GuestID for root probe and write of guest TLB entry */
+	set_root_gid_to_guest_gid();
+
+	for (i = current_cpu_data.tlbsizevtlb;
+	     i < (current_cpu_data.tlbsizevtlb +
+		     current_cpu_data.tlbsizeftlbsets);
+	     i++) {
+		write_gc0_index(i);
+		guest_tlbinvf();
+	}
+	write_gc0_index(idx);
+
+	clear_root_gid();
+	set_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
+}
+EXPORT_SYMBOL_GPL(kvm_loongson_clear_guest_ftlb);
+#endif
+
 #endif
 
 /**
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 72a62f1..73701c3 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -127,6 +127,11 @@ static inline unsigned int kvm_vz_config5_guest_wrmask(struct kvm_vcpu *vcpu)
 	return mask;
 }
 
+static inline unsigned int kvm_vz_config6_guest_wrmask(struct kvm_vcpu *vcpu)
+{
+	return MIPS_CONF6_INTIMER | MIPS_CONF6_EXTIMER | MIPS_CONF6_SYND;
+}
+
 /*
  * VZ optionally allows these additional Config bits to be written by root:
  * Config:	M, [MT]
@@ -181,6 +186,12 @@ static inline unsigned int kvm_vz_config5_user_wrmask(struct kvm_vcpu *vcpu)
 	return kvm_vz_config5_guest_wrmask(vcpu) | MIPS_CONF5_MRP;
 }
 
+static inline unsigned int kvm_vz_config6_user_wrmask(struct kvm_vcpu *vcpu)
+{
+	return kvm_vz_config6_guest_wrmask(vcpu) |
+		MIPS_CONF6_SFBEN | MIPS_CONF6_FTLBEN | MIPS_CONF6_FTLBDIS;
+}
+
 static gpa_t kvm_vz_gva_to_gpa_cb(gva_t gva)
 {
 	/* VZ guest has already converted gva to gpa */
@@ -930,7 +941,8 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				    (sel == 2 ||	/* SRSCtl */
 				     sel == 3)) ||	/* SRSMap */
 				   (rd == MIPS_CP0_CONFIG &&
-				    (sel == 7)) ||	/* Config7 */
+				    (sel == 6 ||	/* Config6 */
+				     sel == 7)) ||	/* Config7 */
 				   (rd == MIPS_CP0_LLADDR &&
 				    (sel == 2) &&	/* MAARI */
 				    cpu_guest_has_maar &&
@@ -938,6 +950,11 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				   (rd == MIPS_CP0_ERRCTL &&
 				    (sel == 0))) {	/* ErrCtl */
 				val = cop0->reg[rd][sel];
+#ifdef CONFIG_CPU_LOONGSON64
+			} else if (rd == MIPS_CP0_DIAG &&
+				   (sel == 0)) {	/* Diag */
+				val = cop0->reg[rd][sel];
+#endif
 			} else {
 				val = 0;
 				er = EMULATE_FAIL;
@@ -1000,9 +1017,40 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				   cpu_guest_has_maar &&
 				   !cpu_guest_has_dyn_maar) {
 				kvm_write_maari(vcpu, val);
+			} else if (rd == MIPS_CP0_CONFIG &&
+				   (sel == 6)) {
+				cop0->reg[rd][sel] = (int)val;
 			} else if (rd == MIPS_CP0_ERRCTL &&
 				   (sel == 0)) {	/* ErrCtl */
 				/* ignore the written value */
+#ifdef CONFIG_CPU_LOONGSON64
+			} else if (rd == MIPS_CP0_DIAG &&
+				   (sel == 0)) {	/* Diag */
+				unsigned long flags;
+
+				local_irq_save(flags);
+				if (val & LOONGSON_DIAG_BTB) {
+					/* Flush BTB */
+					set_c0_diag(LOONGSON_DIAG_BTB);
+				}
+				if (val & LOONGSON_DIAG_ITLB) {
+					/* Flush ITLB */
+					set_c0_diag(LOONGSON_DIAG_ITLB);
+				}
+				if (val & LOONGSON_DIAG_DTLB) {
+					/* Flush DTLB */
+					set_c0_diag(LOONGSON_DIAG_DTLB);
+				}
+				if (val & LOONGSON_DIAG_VTLB) {
+					/* Flush VTLB */
+					kvm_loongson_clear_guest_vtlb();
+				}
+				if (val & LOONGSON_DIAG_FTLB) {
+					/* Flush FTLB */
+					kvm_loongson_clear_guest_ftlb();
+				}
+				local_irq_restore(flags);
+#endif
 			} else {
 				er = EMULATE_FAIL;
 			}
@@ -1692,6 +1740,7 @@ static u64 kvm_vz_get_one_regs[] = {
 	KVM_REG_MIPS_CP0_CONFIG3,
 	KVM_REG_MIPS_CP0_CONFIG4,
 	KVM_REG_MIPS_CP0_CONFIG5,
+	KVM_REG_MIPS_CP0_CONFIG6,
 #ifdef CONFIG_64BIT
 	KVM_REG_MIPS_CP0_XCONTEXT,
 #endif
@@ -2019,6 +2068,9 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 			return -EINVAL;
 		*v = read_gc0_config5();
 		break;
+	case KVM_REG_MIPS_CP0_CONFIG6:
+		*v = kvm_read_sw_gc0_config6(cop0);
+		break;
 	case KVM_REG_MIPS_CP0_MAAR(0) ... KVM_REG_MIPS_CP0_MAAR(0x3f):
 		if (!cpu_guest_has_maar || cpu_guest_has_dyn_maar)
 			return -EINVAL;
@@ -2288,6 +2340,14 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 			write_gc0_config5(v);
 		}
 		break;
+	case KVM_REG_MIPS_CP0_CONFIG6:
+		cur = kvm_read_sw_gc0_config6(cop0);
+		change = (cur ^ v) & kvm_vz_config6_user_wrmask(vcpu);
+		if (change) {
+			v = cur ^ change;
+			kvm_write_sw_gc0_config6(cop0, (int)v);
+		}
+		break;
 	case KVM_REG_MIPS_CP0_MAAR(0) ... KVM_REG_MIPS_CP0_MAAR(0x3f):
 		if (!cpu_guest_has_maar || cpu_guest_has_dyn_maar)
 			return -EINVAL;
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 13/15] KVM: MIPS: Add more MMIO load/store instructions emulation
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (11 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 12/15] KVM: MIPS: Add CONFIG6 and DIAG registers emulation Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 14/15] KVM: MIPS: Enable KVM support for Loongson-3 Huacai Chen
  2020-05-13  8:42 ` [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers Huacai Chen
  14 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

This patch add more MMIO load/store instructions emulation, which can
be observed in QXL and some other device drivers:

1, LWL, LWR, LDW, LDR, SWL, SWR, SDL and SDR for all MIPS;
2, GSLBX, GSLHX, GSLWX, GSLDX, GSSBX, GSSHX, GSSWX and GSSDX for
   Loongson-3.

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/emulate.c | 480 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 470 insertions(+), 10 deletions(-)

diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
index dfc92e5..07b123f 100644
--- a/arch/mips/kvm/emulate.c
+++ b/arch/mips/kvm/emulate.c
@@ -1604,6 +1604,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 	enum emulation_result er;
 	u32 rt;
 	void *data = run->mmio.data;
+	unsigned int imme;
 	unsigned long curr_pc;
 
 	/*
@@ -1661,6 +1662,211 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 			  vcpu->arch.gprs[rt], *(u8 *)data);
 		break;
 
+	case swl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			*(u32 *)data = ((*(u32 *)data) & 0xffffff00) |
+					(vcpu->arch.gprs[rt] >> 24);
+			break;
+		case 1:
+			*(u32 *)data = ((*(u32 *)data) & 0xffff0000) |
+					(vcpu->arch.gprs[rt] >> 16);
+			break;
+		case 2:
+			*(u32 *)data = ((*(u32 *)data) & 0xff000000) |
+					(vcpu->arch.gprs[rt] >> 8);
+			break;
+		case 3:
+			*(u32 *)data = vcpu->arch.gprs[rt];
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SWL: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u32 *)data);
+		break;
+
+	case swr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			*(u32 *)data = vcpu->arch.gprs[rt];
+			break;
+		case 1:
+			*(u32 *)data = ((*(u32 *)data) & 0xff) |
+					(vcpu->arch.gprs[rt] << 8);
+			break;
+		case 2:
+			*(u32 *)data = ((*(u32 *)data) & 0xffff) |
+					(vcpu->arch.gprs[rt] << 16);
+			break;
+		case 3:
+			*(u32 *)data = ((*(u32 *)data) & 0xffffff) |
+					(vcpu->arch.gprs[rt] << 24);
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SWR: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u32 *)data);
+		break;
+
+	case sdl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff00) |
+					((vcpu->arch.gprs[rt] >> 56) & 0xff);
+			break;
+		case 1:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffff0000) |
+					((vcpu->arch.gprs[rt] >> 48) & 0xffff);
+			break;
+		case 2:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffff000000) |
+					((vcpu->arch.gprs[rt] >> 40) & 0xffffff);
+			break;
+		case 3:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffff00000000) |
+					((vcpu->arch.gprs[rt] >> 32) & 0xffffffff);
+			break;
+		case 4:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffff0000000000) |
+					((vcpu->arch.gprs[rt] >> 24) & 0xffffffffff);
+			break;
+		case 5:
+			*(u64 *)data = ((*(u64 *)data) & 0xffff000000000000) |
+					((vcpu->arch.gprs[rt] >> 16) & 0xffffffffffff);
+			break;
+		case 6:
+			*(u64 *)data = ((*(u64 *)data) & 0xff00000000000000) |
+					((vcpu->arch.gprs[rt] >> 8) & 0xffffffffffffff);
+			break;
+		case 7:
+			*(u64 *)data = vcpu->arch.gprs[rt];
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SDL: eaddr: %#lx, gpr: %#lx, data: %llx\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u64 *)data);
+		break;
+
+	case sdr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			*(u64 *)data = vcpu->arch.gprs[rt];
+			break;
+		case 1:
+			*(u64 *)data = ((*(u64 *)data) & 0xff) |
+					(vcpu->arch.gprs[rt] << 8);
+			break;
+		case 2:
+			*(u64 *)data = ((*(u64 *)data) & 0xffff) |
+					(vcpu->arch.gprs[rt] << 16);
+			break;
+		case 3:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffff) |
+					(vcpu->arch.gprs[rt] << 24);
+			break;
+		case 4:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffff) |
+					(vcpu->arch.gprs[rt] << 32);
+			break;
+		case 5:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffff) |
+					(vcpu->arch.gprs[rt] << 40);
+			break;
+		case 6:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffff) |
+					(vcpu->arch.gprs[rt] << 48);
+			break;
+		case 7:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff) |
+					(vcpu->arch.gprs[rt] << 56);
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SDR: eaddr: %#lx, gpr: %#lx, data: %llx\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u64 *)data);
+		break;
+
+#ifdef CONFIG_CPU_LOONGSON64
+	case sdc2_op:
+		rt = inst.loongson3_lsdc2_format.rt;
+		switch (inst.loongson3_lsdc2_format.opcode1) {
+		/*
+		 * Loongson-3 overridden sdc2 instructions.
+		 * opcode1              instruction
+		 *   0x0          gssbx: store 1 bytes from GPR
+		 *   0x1          gsshx: store 2 bytes from GPR
+		 *   0x2          gsswx: store 4 bytes from GPR
+		 *   0x3          gssdx: store 8 bytes from GPR
+		 */
+		case 0x0:
+			run->mmio.len = 1;
+			*(u8 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSBX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u8 *)data);
+			break;
+		case 0x1:
+			run->mmio.len = 2;
+			*(u16 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSSHX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u16 *)data);
+			break;
+		case 0x2:
+			run->mmio.len = 4;
+			*(u32 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSWX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u32 *)data);
+			break;
+		case 0x3:
+			run->mmio.len = 8;
+			*(u64 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSDX: eaddr: %#lx, gpr: %#lx, data: %#llx\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u64 *)data);
+			break;
+		default:
+			kvm_err("Godson Exteneded GS-Store not yet supported (inst=0x%08x)\n",
+				inst.word);
+			break;
+		}
+		break;
+#endif
 	default:
 		kvm_err("Store not yet supported (inst=0x%08x)\n",
 			inst.word);
@@ -1695,6 +1901,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 	enum emulation_result er;
 	unsigned long curr_pc;
 	u32 op, rt;
+	unsigned int imme;
 
 	rt = inst.i_format.rt;
 	op = inst.i_format.opcode;
@@ -1747,6 +1954,162 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 		run->mmio.len = 1;
 		break;
 
+	case lwl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 3;	/* 1 byte */
+			break;
+		case 1:
+			vcpu->mmio_needed = 4;	/* 2 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 5;	/* 3 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 6;	/* 4 bytes */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case lwr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 7;	/* 4 bytes */
+			break;
+		case 1:
+			vcpu->mmio_needed = 8;	/* 3 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 9;	/* 2 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 10;	/* 1 byte */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case ldl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 11;	/* 1 byte */
+			break;
+		case 1:
+			vcpu->mmio_needed = 12;	/* 2 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 13;	/* 3 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 14;	/* 4 bytes */
+			break;
+		case 4:
+			vcpu->mmio_needed = 15;	/* 5 bytes */
+			break;
+		case 5:
+			vcpu->mmio_needed = 16;	/* 6 bytes */
+			break;
+		case 6:
+			vcpu->mmio_needed = 17;	/* 7 bytes */
+			break;
+		case 7:
+			vcpu->mmio_needed = 18;	/* 8 bytes */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case ldr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 19;	/* 8 bytes */
+			break;
+		case 1:
+			vcpu->mmio_needed = 20;	/* 7 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 21;	/* 6 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 22;	/* 5 bytes */
+			break;
+		case 4:
+			vcpu->mmio_needed = 23;	/* 4 bytes */
+			break;
+		case 5:
+			vcpu->mmio_needed = 24;	/* 3 bytes */
+			break;
+		case 6:
+			vcpu->mmio_needed = 25;	/* 2 bytes */
+			break;
+		case 7:
+			vcpu->mmio_needed = 26;	/* 1 byte */
+			break;
+		default:
+			break;
+		}
+		break;
+
+#ifdef CONFIG_CPU_LOONGSON64
+	case ldc2_op:
+		rt = inst.loongson3_lsdc2_format.rt;
+		switch (inst.loongson3_lsdc2_format.opcode1) {
+		/*
+		 * Loongson-3 overridden ldc2 instructions.
+		 * opcode1              instruction
+		 *   0x0          gslbx: store 1 bytes from GPR
+		 *   0x1          gslhx: store 2 bytes from GPR
+		 *   0x2          gslwx: store 4 bytes from GPR
+		 *   0x3          gsldx: store 8 bytes from GPR
+		 */
+		case 0x0:
+			run->mmio.len = 1;
+			vcpu->mmio_needed = 27;	/* signed */
+			break;
+		case 0x1:
+			run->mmio.len = 2;
+			vcpu->mmio_needed = 28;	/* signed */
+			break;
+		case 0x2:
+			run->mmio.len = 4;
+			vcpu->mmio_needed = 29;	/* signed */
+			break;
+		case 0x3:
+			run->mmio.len = 8;
+			vcpu->mmio_needed = 30;	/* signed */
+			break;
+		default:
+			kvm_err("Godson Exteneded GS-Load for float not yet supported (inst=0x%08x)\n",
+				inst.word);
+			break;
+		}
+		break;
+#endif
+
 	default:
 		kvm_err("Load not yet supported (inst=0x%08x)\n",
 			inst.word);
@@ -2612,28 +2975,125 @@ enum emulation_result kvm_mips_complete_mmio_load(struct kvm_vcpu *vcpu,
 
 	switch (run->mmio.len) {
 	case 8:
-		*gpr = *(s64 *)run->mmio.data;
+		switch (vcpu->mmio_needed) {
+		case 11:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xff) << 56);
+			break;
+		case 12:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffff) << 48);
+			break;
+		case 13:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffff) << 40);
+			break;
+		case 14:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffff) << 32);
+			break;
+		case 15:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffff) << 24);
+			break;
+		case 16:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffffff) << 16);
+			break;
+		case 17:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffffffff) << 8);
+			break;
+		case 18:
+		case 19:
+			*gpr = *(s64 *)run->mmio.data;
+			break;
+		case 20:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff00000000000000) |
+				((((*(s64 *)run->mmio.data)) >> 8) & 0xffffffffffffff);
+			break;
+		case 21:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff000000000000) |
+				((((*(s64 *)run->mmio.data)) >> 16) & 0xffffffffffff);
+			break;
+		case 22:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff0000000000) |
+				((((*(s64 *)run->mmio.data)) >> 24) & 0xffffffffff);
+			break;
+		case 23:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff00000000) |
+				((((*(s64 *)run->mmio.data)) >> 32) & 0xffffffff);
+			break;
+		case 24:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff000000) |
+				((((*(s64 *)run->mmio.data)) >> 40) & 0xffffff);
+			break;
+		case 25:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff0000) |
+				((((*(s64 *)run->mmio.data)) >> 48) & 0xffff);
+			break;
+		case 26:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff00) |
+				((((*(s64 *)run->mmio.data)) >> 56) & 0xff);
+			break;
+		default:
+			*gpr = *(s64 *)run->mmio.data;
+		}
 		break;
 
 	case 4:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s32 *)run->mmio.data;
-		else
+		switch (vcpu->mmio_needed) {
+		case 1:
 			*gpr = *(u32 *)run->mmio.data;
+			break;
+		case 2:
+			*gpr = *(s32 *)run->mmio.data;
+			break;
+		case 3:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
+				(((*(s32 *)run->mmio.data) & 0xff) << 24);
+			break;
+		case 4:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
+				(((*(s32 *)run->mmio.data) & 0xffff) << 16);
+			break;
+		case 5:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
+				(((*(s32 *)run->mmio.data) & 0xffffff) << 8);
+			break;
+		case 6:
+		case 7:
+			*gpr = *(s32 *)run->mmio.data;
+			break;
+		case 8:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff000000) |
+				((((*(s32 *)run->mmio.data)) >> 8) & 0xffffff);
+			break;
+		case 9:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff0000) |
+				((((*(s32 *)run->mmio.data)) >> 16) & 0xffff);
+			break;
+		case 10:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff00) |
+				((((*(s32 *)run->mmio.data)) >> 24) & 0xff);
+			break;
+		default:
+			*gpr = *(s32 *)run->mmio.data;
+		}
 		break;
 
 	case 2:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s16 *) run->mmio.data;
-		else
+		if (vcpu->mmio_needed == 1)
 			*gpr = *(u16 *)run->mmio.data;
+		else
+			*gpr = *(s16 *)run->mmio.data;
 
 		break;
 	case 1:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s8 *) run->mmio.data;
+		if (vcpu->mmio_needed == 1)
+			*gpr = *(u8 *)run->mmio.data;
 		else
-			*gpr = *(u8 *) run->mmio.data;
+			*gpr = *(s8 *)run->mmio.data;
 		break;
 	}
 
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 14/15] KVM: MIPS: Enable KVM support for Loongson-3
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (12 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 13/15] KVM: MIPS: Add more MMIO load/store instructions emulation Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-14  7:46   ` Aleksandar Markovic
  2020-05-16 15:35   ` Thomas Bogendoerfer
  2020-05-13  8:42 ` [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers Huacai Chen
  14 siblings, 2 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

This patch enable KVM support for Loongson-3 by selecting HAVE_KVM, but
only enable KVM/VZ on Loongson-3A R4+ (because VZ of early processors
are incomplete). Besides, Loongson-3 support SMP guests, so we clear the
linked load bit of LLAddr in kvm_vz_vcpu_load() if the guest has more
than one VCPUs.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/Kconfig            | 1 +
 arch/mips/kernel/cpu-probe.c | 1 +
 arch/mips/kvm/vz.c           | 2 +-
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index ce1aacc..3a15f2b 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -1405,6 +1405,7 @@ config CPU_LOONGSON64
 	select MIPS_L1_CACHE_SHIFT_6
 	select GPIOLIB
 	select SWIOTLB
+	select HAVE_KVM
 	help
 		The Loongson GSx64(GS264/GS464/GS464E/GS464V) series of processor
 		cores implements the MIPS64R2 instruction set with many extensions,
diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
index 319c250..415fbd2 100644
--- a/arch/mips/kernel/cpu-probe.c
+++ b/arch/mips/kernel/cpu-probe.c
@@ -2008,6 +2008,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
 		c->writecombine = _CACHE_UNCACHED_ACCELERATED;
 		c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
 			MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
+		c->ases &= ~MIPS_ASE_VZ; /* VZ of Loongson-3A2000/3000 is incomplete */
 		break;
 	case PRID_IMP_LOONGSON_64G:
 		c->cputype = CPU_LOONGSON64;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 73701c3..dc753d0 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2697,7 +2697,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	 * prevents a SC on the next VCPU from succeeding by matching a LL on
 	 * the previous VCPU.
 	 */
-	if (cpu_guest_has_rw_llb)
+	if (vcpu->kvm->created_vcpus > 1)
 		write_gc0_lladdr(0);
 
 	return 0;
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
                   ` (13 preceding siblings ...)
  2020-05-13  8:42 ` [PATCH V5 14/15] KVM: MIPS: Enable KVM support for Loongson-3 Huacai Chen
@ 2020-05-13  8:42 ` Huacai Chen
  2020-05-14  7:47   ` Aleksandar Markovic
  2020-05-14 13:33   ` Jiaxun Yang
  14 siblings, 2 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-13  8:42 UTC (permalink / raw)
  To: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen, Jiaxun Yang, Huacai Chen

James Hogan has become inactive for a long time and leaves KVM for MIPS
orphan. I'm working on KVM/Loongson and attempt to make it upstream both
in kernel and QEMU, while Aleksandar Markovic is already a maintainer of
QEMU/MIPS. We are both interested in QEMU/KVM/MIPS, and we have already
made some contributions in kernel and QEMU. If possible, we want to take
the KVM/MIPS maintainership.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 MAINTAINERS | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index e64e5db..59b3f43 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9302,9 +9302,11 @@ F:	include/kvm/arm_*
 F:	virt/kvm/arm/
 
 KERNEL VIRTUAL MACHINE FOR MIPS (KVM/mips)
+M:	Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
+M:	Huacai Chen <chenhc@lemote.com>
 L:	linux-mips@vger.kernel.org
 L:	kvm@vger.kernel.org
-S:	Orphan
+S:	Maintained
 F:	arch/mips/include/asm/kvm*
 F:	arch/mips/include/uapi/asm/kvm*
 F:	arch/mips/kvm/
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 12/15] KVM: MIPS: Add CONFIG6 and DIAG registers emulation
  2020-05-13  8:42 ` [PATCH V5 12/15] KVM: MIPS: Add CONFIG6 and DIAG registers emulation Huacai Chen
@ 2020-05-14  7:45   ` Aleksandar Markovic
  2020-05-16 15:36   ` Thomas Bogendoerfer
  1 sibling, 0 replies; 31+ messages in thread
From: Aleksandar Markovic @ 2020-05-14  7:45 UTC (permalink / raw)
  To: Huacai Chen
  Cc: Paolo Bonzini, Thomas Bogendoerfer, kvm, linux-mips, Fuxin Zhang,
	Huacai Chen, Jiaxun Yang

сре, 13. мај 2020. у 10:47 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3 has CONFIG6 and DIAG registers which need to be emulated.
> CONFIG6 is mostly used to enable/disable FTLB and SFB, while DIAG is
> mostly used to flush BTB, ITLB, DTLB, VTLB and FTLB.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

>  arch/mips/include/asm/kvm_host.h |  7 +++++
>  arch/mips/include/asm/mipsregs.h |  7 +++++
>  arch/mips/kvm/tlb.c              | 41 ++++++++++++++++++++++++++
>  arch/mips/kvm/vz.c               | 62 +++++++++++++++++++++++++++++++++++++++-
>  4 files changed, 116 insertions(+), 1 deletion(-)
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index 3fd2f1c..30b5e33 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -68,9 +68,11 @@
>  #define KVM_REG_MIPS_CP0_CONFIG3       MIPS_CP0_32(16, 3)
>  #define KVM_REG_MIPS_CP0_CONFIG4       MIPS_CP0_32(16, 4)
>  #define KVM_REG_MIPS_CP0_CONFIG5       MIPS_CP0_32(16, 5)
> +#define KVM_REG_MIPS_CP0_CONFIG6       MIPS_CP0_32(16, 6)
>  #define KVM_REG_MIPS_CP0_CONFIG7       MIPS_CP0_32(16, 7)
>  #define KVM_REG_MIPS_CP0_MAARI         MIPS_CP0_64(17, 2)
>  #define KVM_REG_MIPS_CP0_XCONTEXT      MIPS_CP0_64(20, 0)
> +#define KVM_REG_MIPS_CP0_DIAG          MIPS_CP0_32(22, 0)
>  #define KVM_REG_MIPS_CP0_ERROREPC      MIPS_CP0_64(30, 0)
>  #define KVM_REG_MIPS_CP0_KSCRATCH1     MIPS_CP0_64(31, 2)
>  #define KVM_REG_MIPS_CP0_KSCRATCH2     MIPS_CP0_64(31, 3)
> @@ -256,6 +258,7 @@ struct mips_coproc {
>  #define MIPS_CP0_WATCH_LO      18
>  #define MIPS_CP0_WATCH_HI      19
>  #define MIPS_CP0_TLB_XCONTEXT  20
> +#define MIPS_CP0_DIAG          22
>  #define MIPS_CP0_ECC           26
>  #define MIPS_CP0_CACHE_ERR     27
>  #define MIPS_CP0_TAG_LO                28
> @@ -927,6 +930,10 @@ void kvm_vz_save_guesttlb(struct kvm_mips_tlb *buf, unsigned int index,
>                           unsigned int count);
>  void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
>                           unsigned int count);
> +#ifdef CONFIG_CPU_LOONGSON64
> +void kvm_loongson_clear_guest_vtlb(void);
> +void kvm_loongson_clear_guest_ftlb(void);
> +#endif
>  #endif
>
>  void kvm_mips_suspend_mm(int cpu);
> diff --git a/arch/mips/include/asm/mipsregs.h b/arch/mips/include/asm/mipsregs.h
> index 796fe47..ce40fbf 100644
> --- a/arch/mips/include/asm/mipsregs.h
> +++ b/arch/mips/include/asm/mipsregs.h
> @@ -674,6 +674,9 @@
>  #define MIPS_CONF5_CV          (_ULCAST_(1) << 29)
>  #define MIPS_CONF5_K           (_ULCAST_(1) << 30)
>
> +#define MIPS_CONF6_INTIMER     (_ULCAST_(1) << 6)
> +#define MIPS_CONF6_EXTIMER     (_ULCAST_(1) << 7)
> +#define MIPS_CONF6_SFBEN       (_ULCAST_(1) << 8)
>  #define MIPS_CONF6_SYND                (_ULCAST_(1) << 13)
>  /* proAptiv FTLB on/off bit */
>  #define MIPS_CONF6_FTLBEN      (_ULCAST_(1) << 15)
> @@ -993,6 +996,8 @@
>  /* Disable Branch Return Cache */
>  #define R10K_DIAG_D_BRC                (_ULCAST_(1) << 22)
>
> +/* Flush BTB */
> +#define LOONGSON_DIAG_BTB      (_ULCAST_(1) << 1)
>  /* Flush ITLB */
>  #define LOONGSON_DIAG_ITLB     (_ULCAST_(1) << 2)
>  /* Flush DTLB */
> @@ -2825,7 +2830,9 @@ __BUILD_SET_C0(status)
>  __BUILD_SET_C0(cause)
>  __BUILD_SET_C0(config)
>  __BUILD_SET_C0(config5)
> +__BUILD_SET_C0(config6)
>  __BUILD_SET_C0(config7)
> +__BUILD_SET_C0(diag)
>  __BUILD_SET_C0(intcontrol)
>  __BUILD_SET_C0(intctl)
>  __BUILD_SET_C0(srsmap)
> diff --git a/arch/mips/kvm/tlb.c b/arch/mips/kvm/tlb.c
> index 7cd9216..1418715 100644
> --- a/arch/mips/kvm/tlb.c
> +++ b/arch/mips/kvm/tlb.c
> @@ -20,6 +20,7 @@
>
>  #include <asm/cpu.h>
>  #include <asm/bootinfo.h>
> +#include <asm/mipsregs.h>
>  #include <asm/mmu_context.h>
>  #include <asm/pgtable.h>
>  #include <asm/cacheflush.h>
> @@ -622,6 +623,46 @@ void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
>  }
>  EXPORT_SYMBOL_GPL(kvm_vz_load_guesttlb);
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +void kvm_loongson_clear_guest_vtlb(void)
> +{
> +       int idx = read_gc0_index();
> +
> +       /* Set root GuestID for root probe and write of guest TLB entry */
> +       set_root_gid_to_guest_gid();
> +
> +       write_gc0_index(0);
> +       guest_tlbinvf();
> +       write_gc0_index(idx);
> +
> +       clear_root_gid();
> +       set_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
> +}
> +EXPORT_SYMBOL_GPL(kvm_loongson_clear_guest_vtlb);
> +
> +void kvm_loongson_clear_guest_ftlb(void)
> +{
> +       int i;
> +       int idx = read_gc0_index();
> +
> +       /* Set root GuestID for root probe and write of guest TLB entry */
> +       set_root_gid_to_guest_gid();
> +
> +       for (i = current_cpu_data.tlbsizevtlb;
> +            i < (current_cpu_data.tlbsizevtlb +
> +                    current_cpu_data.tlbsizeftlbsets);
> +            i++) {
> +               write_gc0_index(i);
> +               guest_tlbinvf();
> +       }
> +       write_gc0_index(idx);
> +
> +       clear_root_gid();
> +       set_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
> +}
> +EXPORT_SYMBOL_GPL(kvm_loongson_clear_guest_ftlb);
> +#endif
> +
>  #endif
>
>  /**
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index 72a62f1..73701c3 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -127,6 +127,11 @@ static inline unsigned int kvm_vz_config5_guest_wrmask(struct kvm_vcpu *vcpu)
>         return mask;
>  }
>
> +static inline unsigned int kvm_vz_config6_guest_wrmask(struct kvm_vcpu *vcpu)
> +{
> +       return MIPS_CONF6_INTIMER | MIPS_CONF6_EXTIMER | MIPS_CONF6_SYND;
> +}
> +
>  /*
>   * VZ optionally allows these additional Config bits to be written by root:
>   * Config:     M, [MT]
> @@ -181,6 +186,12 @@ static inline unsigned int kvm_vz_config5_user_wrmask(struct kvm_vcpu *vcpu)
>         return kvm_vz_config5_guest_wrmask(vcpu) | MIPS_CONF5_MRP;
>  }
>
> +static inline unsigned int kvm_vz_config6_user_wrmask(struct kvm_vcpu *vcpu)
> +{
> +       return kvm_vz_config6_guest_wrmask(vcpu) |
> +               MIPS_CONF6_SFBEN | MIPS_CONF6_FTLBEN | MIPS_CONF6_FTLBDIS;
> +}
> +
>  static gpa_t kvm_vz_gva_to_gpa_cb(gva_t gva)
>  {
>         /* VZ guest has already converted gva to gpa */
> @@ -930,7 +941,8 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
>                                     (sel == 2 ||        /* SRSCtl */
>                                      sel == 3)) ||      /* SRSMap */
>                                    (rd == MIPS_CP0_CONFIG &&
> -                                   (sel == 7)) ||      /* Config7 */
> +                                   (sel == 6 ||        /* Config6 */
> +                                    sel == 7)) ||      /* Config7 */
>                                    (rd == MIPS_CP0_LLADDR &&
>                                     (sel == 2) &&       /* MAARI */
>                                     cpu_guest_has_maar &&
> @@ -938,6 +950,11 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
>                                    (rd == MIPS_CP0_ERRCTL &&
>                                     (sel == 0))) {      /* ErrCtl */
>                                 val = cop0->reg[rd][sel];
> +#ifdef CONFIG_CPU_LOONGSON64
> +                       } else if (rd == MIPS_CP0_DIAG &&
> +                                  (sel == 0)) {        /* Diag */
> +                               val = cop0->reg[rd][sel];
> +#endif
>                         } else {
>                                 val = 0;
>                                 er = EMULATE_FAIL;
> @@ -1000,9 +1017,40 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
>                                    cpu_guest_has_maar &&
>                                    !cpu_guest_has_dyn_maar) {
>                                 kvm_write_maari(vcpu, val);
> +                       } else if (rd == MIPS_CP0_CONFIG &&
> +                                  (sel == 6)) {
> +                               cop0->reg[rd][sel] = (int)val;
>                         } else if (rd == MIPS_CP0_ERRCTL &&
>                                    (sel == 0)) {        /* ErrCtl */
>                                 /* ignore the written value */
> +#ifdef CONFIG_CPU_LOONGSON64
> +                       } else if (rd == MIPS_CP0_DIAG &&
> +                                  (sel == 0)) {        /* Diag */
> +                               unsigned long flags;
> +
> +                               local_irq_save(flags);
> +                               if (val & LOONGSON_DIAG_BTB) {
> +                                       /* Flush BTB */
> +                                       set_c0_diag(LOONGSON_DIAG_BTB);
> +                               }
> +                               if (val & LOONGSON_DIAG_ITLB) {
> +                                       /* Flush ITLB */
> +                                       set_c0_diag(LOONGSON_DIAG_ITLB);
> +                               }
> +                               if (val & LOONGSON_DIAG_DTLB) {
> +                                       /* Flush DTLB */
> +                                       set_c0_diag(LOONGSON_DIAG_DTLB);
> +                               }
> +                               if (val & LOONGSON_DIAG_VTLB) {
> +                                       /* Flush VTLB */
> +                                       kvm_loongson_clear_guest_vtlb();
> +                               }
> +                               if (val & LOONGSON_DIAG_FTLB) {
> +                                       /* Flush FTLB */
> +                                       kvm_loongson_clear_guest_ftlb();
> +                               }
> +                               local_irq_restore(flags);
> +#endif
>                         } else {
>                                 er = EMULATE_FAIL;
>                         }
> @@ -1692,6 +1740,7 @@ static u64 kvm_vz_get_one_regs[] = {
>         KVM_REG_MIPS_CP0_CONFIG3,
>         KVM_REG_MIPS_CP0_CONFIG4,
>         KVM_REG_MIPS_CP0_CONFIG5,
> +       KVM_REG_MIPS_CP0_CONFIG6,
>  #ifdef CONFIG_64BIT
>         KVM_REG_MIPS_CP0_XCONTEXT,
>  #endif
> @@ -2019,6 +2068,9 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
>                         return -EINVAL;
>                 *v = read_gc0_config5();
>                 break;
> +       case KVM_REG_MIPS_CP0_CONFIG6:
> +               *v = kvm_read_sw_gc0_config6(cop0);
> +               break;
>         case KVM_REG_MIPS_CP0_MAAR(0) ... KVM_REG_MIPS_CP0_MAAR(0x3f):
>                 if (!cpu_guest_has_maar || cpu_guest_has_dyn_maar)
>                         return -EINVAL;
> @@ -2288,6 +2340,14 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
>                         write_gc0_config5(v);
>                 }
>                 break;
> +       case KVM_REG_MIPS_CP0_CONFIG6:
> +               cur = kvm_read_sw_gc0_config6(cop0);
> +               change = (cur ^ v) & kvm_vz_config6_user_wrmask(vcpu);
> +               if (change) {
> +                       v = cur ^ change;
> +                       kvm_write_sw_gc0_config6(cop0, (int)v);
> +               }
> +               break;
>         case KVM_REG_MIPS_CP0_MAAR(0) ... KVM_REG_MIPS_CP0_MAAR(0x3f):
>                 if (!cpu_guest_has_maar || cpu_guest_has_dyn_maar)
>                         return -EINVAL;
> --
> 2.7.0
>

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 14/15] KVM: MIPS: Enable KVM support for Loongson-3
  2020-05-13  8:42 ` [PATCH V5 14/15] KVM: MIPS: Enable KVM support for Loongson-3 Huacai Chen
@ 2020-05-14  7:46   ` Aleksandar Markovic
  2020-05-16 15:35   ` Thomas Bogendoerfer
  1 sibling, 0 replies; 31+ messages in thread
From: Aleksandar Markovic @ 2020-05-14  7:46 UTC (permalink / raw)
  To: Huacai Chen
  Cc: Paolo Bonzini, Thomas Bogendoerfer, kvm, linux-mips, Fuxin Zhang,
	Huacai Chen, Jiaxun Yang

сре, 13. мај 2020. у 10:48 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> This patch enable KVM support for Loongson-3 by selecting HAVE_KVM, but
> only enable KVM/VZ on Loongson-3A R4+ (because VZ of early processors
> are incomplete). Besides, Loongson-3 support SMP guests, so we clear the
> linked load bit of LLAddr in kvm_vz_vcpu_load() if the guest has more
> than one VCPUs.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

>  arch/mips/Kconfig            | 1 +
>  arch/mips/kernel/cpu-probe.c | 1 +
>  arch/mips/kvm/vz.c           | 2 +-
>  3 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
> index ce1aacc..3a15f2b 100644
> --- a/arch/mips/Kconfig
> +++ b/arch/mips/Kconfig
> @@ -1405,6 +1405,7 @@ config CPU_LOONGSON64
>         select MIPS_L1_CACHE_SHIFT_6
>         select GPIOLIB
>         select SWIOTLB
> +       select HAVE_KVM
>         help
>                 The Loongson GSx64(GS264/GS464/GS464E/GS464V) series of processor
>                 cores implements the MIPS64R2 instruction set with many extensions,
> diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
> index 319c250..415fbd2 100644
> --- a/arch/mips/kernel/cpu-probe.c
> +++ b/arch/mips/kernel/cpu-probe.c
> @@ -2008,6 +2008,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
>                 c->writecombine = _CACHE_UNCACHED_ACCELERATED;
>                 c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
>                         MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
> +               c->ases &= ~MIPS_ASE_VZ; /* VZ of Loongson-3A2000/3000 is incomplete */
>                 break;
>         case PRID_IMP_LOONGSON_64G:
>                 c->cputype = CPU_LOONGSON64;
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index 73701c3..dc753d0 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -2697,7 +2697,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>          * prevents a SC on the next VCPU from succeeding by matching a LL on
>          * the previous VCPU.
>          */
> -       if (cpu_guest_has_rw_llb)
> +       if (vcpu->kvm->created_vcpus > 1)
>                 write_gc0_lladdr(0);
>
>         return 0;
> --
> 2.7.0
>

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-13  8:42 ` [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers Huacai Chen
@ 2020-05-14  7:47   ` Aleksandar Markovic
  2020-05-14 13:33   ` Jiaxun Yang
  1 sibling, 0 replies; 31+ messages in thread
From: Aleksandar Markovic @ 2020-05-14  7:47 UTC (permalink / raw)
  To: Huacai Chen
  Cc: Paolo Bonzini, Thomas Bogendoerfer, kvm, linux-mips, Fuxin Zhang,
	Huacai Chen, Jiaxun Yang

сре, 13. мај 2020. у 10:48 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> James Hogan has become inactive for a long time and leaves KVM for MIPS
> orphan. I'm working on KVM/Loongson and attempt to make it upstream both
> in kernel and QEMU, while Aleksandar Markovic is already a maintainer of
> QEMU/MIPS. We are both interested in QEMU/KVM/MIPS, and we have already
> made some contributions in kernel and QEMU. If possible, we want to take
> the KVM/MIPS maintainership.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> ---

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

>  MAINTAINERS | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e64e5db..59b3f43 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -9302,9 +9302,11 @@ F:       include/kvm/arm_*
>  F:     virt/kvm/arm/
>
>  KERNEL VIRTUAL MACHINE FOR MIPS (KVM/mips)
> +M:     Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
> +M:     Huacai Chen <chenhc@lemote.com>
>  L:     linux-mips@vger.kernel.org
>  L:     kvm@vger.kernel.org
> -S:     Orphan
> +S:     Maintained
>  F:     arch/mips/include/asm/kvm*
>  F:     arch/mips/include/uapi/asm/kvm*
>  F:     arch/mips/kvm/
> --
> 2.7.0
>

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-13  8:42 ` [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers Huacai Chen
  2020-05-14  7:47   ` Aleksandar Markovic
@ 2020-05-14 13:33   ` Jiaxun Yang
  2020-05-15  1:50     ` Huacai Chen
  1 sibling, 1 reply; 31+ messages in thread
From: Jiaxun Yang @ 2020-05-14 13:33 UTC (permalink / raw)
  To: Huacai Chen, Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic
  Cc: kvm, linux-mips, Fuxin Zhang, Huacai Chen



于 2020年5月13日 GMT+08:00 下午4:42:46, Huacai Chen <chenhc@lemote.com> 写到:
>James Hogan has become inactive for a long time and leaves KVM for MIPS
>orphan. I'm working on KVM/Loongson and attempt to make it upstream both
>in kernel and QEMU, while Aleksandar Markovic is already a maintainer of
>QEMU/MIPS. We are both interested in QEMU/KVM/MIPS, and we have already
>made some contributions in kernel and QEMU. If possible, we want to take
>the KVM/MIPS maintainership.
>
>Signed-off-by: Huacai Chen <chenhc@lemote.com>
>---

Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>

Huacai is extremely experienced on virtualization,
his PhD thesis is about virtualization too.
He had been working on Loongson kernel for a long period.

Alexander maintained QEMU/MIPS for some years, he is a expert on
QEMU and MIPS architecture.

I believe these guys can effectively bring MIPS/KVM support back to sea level again.

Best wishes!
-- 
Jiaxun Yang

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-14 13:33   ` Jiaxun Yang
@ 2020-05-15  1:50     ` Huacai Chen
  2020-05-15 21:13       ` Thomas Bogendoerfer
  0 siblings, 1 reply; 31+ messages in thread
From: Huacai Chen @ 2020-05-15  1:50 UTC (permalink / raw)
  To: Jiaxun Yang
  Cc: Paolo Bonzini, Thomas Bogendoerfer, Aleksandar Markovic, kvm,
	open list:MIPS, Fuxin Zhang

Hi, Jiaxun,

On Thu, May 14, 2020 at 9:34 PM Jiaxun Yang <jiaxun.yang@flygoat.com> wrote:
>
>
>
> 于 2020年5月13日 GMT+08:00 下午4:42:46, Huacai Chen <chenhc@lemote.com> 写到:
> >James Hogan has become inactive for a long time and leaves KVM for MIPS
> >orphan. I'm working on KVM/Loongson and attempt to make it upstream both
> >in kernel and QEMU, while Aleksandar Markovic is already a maintainer of
> >QEMU/MIPS. We are both interested in QEMU/KVM/MIPS, and we have already
> >made some contributions in kernel and QEMU. If possible, we want to take
> >the KVM/MIPS maintainership.
> >
> >Signed-off-by: Huacai Chen <chenhc@lemote.com>
> >---
>
> Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
>
> Huacai is extremely experienced on virtualization,
> his PhD thesis is about virtualization too.
> He had been working on Loongson kernel for a long period.
>
> Alexander maintained QEMU/MIPS for some years, he is a expert on
> QEMU and MIPS architecture.
>
> I believe these guys can effectively bring MIPS/KVM support back to sea level again.
Thank you very much, and I think you won't be disappointed in us.

>
> Best wishes!
> --
> Jiaxun Yang
Regards,
Huacai

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-15  1:50     ` Huacai Chen
@ 2020-05-15 21:13       ` Thomas Bogendoerfer
  2020-05-16  9:36         ` Huacai Chen
  0 siblings, 1 reply; 31+ messages in thread
From: Thomas Bogendoerfer @ 2020-05-15 21:13 UTC (permalink / raw)
  To: Huacai Chen
  Cc: Jiaxun Yang, Paolo Bonzini, Aleksandar Markovic, kvm,
	open list:MIPS, Fuxin Zhang

On Fri, May 15, 2020 at 09:50:04AM +0800, Huacai Chen wrote:
> On Thu, May 14, 2020 at 9:34 PM Jiaxun Yang <jiaxun.yang@flygoat.com> wrote:
> > 于 2020年5月13日 GMT+08:00 下午4:42:46, Huacai Chen <chenhc@lemote.com> 写到:
> > >James Hogan has become inactive for a long time and leaves KVM for MIPS
> > >orphan. I'm working on KVM/Loongson and attempt to make it upstream both
> > >in kernel and QEMU, while Aleksandar Markovic is already a maintainer of
> > >QEMU/MIPS. We are both interested in QEMU/KVM/MIPS, and we have already
> > >made some contributions in kernel and QEMU. If possible, we want to take
> > >the KVM/MIPS maintainership.
> > >
> > >Signed-off-by: Huacai Chen <chenhc@lemote.com>
> > >---
> >
> > Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> >
> > Huacai is extremely experienced on virtualization,
> > his PhD thesis is about virtualization too.
> > He had been working on Loongson kernel for a long period.
> >
> > Alexander maintained QEMU/MIPS for some years, he is a expert on
> > QEMU and MIPS architecture.
> >
> > I believe these guys can effectively bring MIPS/KVM support back to sea level again.
> Thank you very much, and I think you won't be disappointed in us.

I'm happy to see you taking care of the KVM part. So how is your plan
to handle patches ? Do want to collect them and send pull requests to
me ? Or should I just pick them up with your Acked-by ?

And could you do me a favour and check your mail setup. I always
get failed deliveries:

  chenhc@lemote.com
    SMTP error from remote mailer after end of data:
    host mxbiz1.qq.com [203.205.232.191]: 550 Mail content denied. http://servic
e.exmail.qq.com/cgi-bin/help?subtype=1&&id=20022&&no=1000726

or

  chenhc@lemote.com
    SMTP error from remote mailer after end of data:
    host mxbiz1.qq.com [203.205.232.191]: 550 Ip frequency limited. http://servi
ce.exmail.qq.com/cgi-bin/help?subtype=1&&id=20022&&no=1000725

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-15 21:13       ` Thomas Bogendoerfer
@ 2020-05-16  9:36         ` Huacai Chen
  2020-05-16 11:55           ` Paolo Bonzini
  0 siblings, 1 reply; 31+ messages in thread
From: Huacai Chen @ 2020-05-16  9:36 UTC (permalink / raw)
  To: Thomas Bogendoerfer
  Cc: Jiaxun Yang, Paolo Bonzini, Aleksandar Markovic, kvm,
	open list:MIPS, Fuxin Zhang

Hi, Thomas,

On Sat, May 16, 2020 at 5:14 AM Thomas Bogendoerfer
<tsbogend@alpha.franken.de> wrote:
>
> On Fri, May 15, 2020 at 09:50:04AM +0800, Huacai Chen wrote:
> > On Thu, May 14, 2020 at 9:34 PM Jiaxun Yang <jiaxun.yang@flygoat.com> wrote:
> > > 于 2020年5月13日 GMT+08:00 下午4:42:46, Huacai Chen <chenhc@lemote.com> 写到:
> > > >James Hogan has become inactive for a long time and leaves KVM for MIPS
> > > >orphan. I'm working on KVM/Loongson and attempt to make it upstream both
> > > >in kernel and QEMU, while Aleksandar Markovic is already a maintainer of
> > > >QEMU/MIPS. We are both interested in QEMU/KVM/MIPS, and we have already
> > > >made some contributions in kernel and QEMU. If possible, we want to take
> > > >the KVM/MIPS maintainership.
> > > >
> > > >Signed-off-by: Huacai Chen <chenhc@lemote.com>
> > > >---
> > >
> > > Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> > >
> > > Huacai is extremely experienced on virtualization,
> > > his PhD thesis is about virtualization too.
> > > He had been working on Loongson kernel for a long period.
> > >
> > > Alexander maintained QEMU/MIPS for some years, he is a expert on
> > > QEMU and MIPS architecture.
> > >
> > > I believe these guys can effectively bring MIPS/KVM support back to sea level again.
> > Thank you very much, and I think you won't be disappointed in us.
>
> I'm happy to see you taking care of the KVM part. So how is your plan
> to handle patches ? Do want to collect them and send pull requests to
> me ? Or should I just pick them up with your Acked-by ?
I think we can only use the second method, because both Aleksandar and
me don't have a kernel tree in kernel.org now.

>
> And could you do me a favour and check your mail setup. I always
> get failed deliveries:
>
>   chenhc@lemote.com
>     SMTP error from remote mailer after end of data:
>     host mxbiz1.qq.com [203.205.232.191]: 550 Mail content denied. http://servic
> e.exmail.qq.com/cgi-bin/help?subtype=1&&id=20022&&no=1000726
>
> or
>
>   chenhc@lemote.com
>     SMTP error from remote mailer after end of data:
>     host mxbiz1.qq.com [203.205.232.191]: 550 Ip frequency limited. http://servi
> ce.exmail.qq.com/cgi-bin/help?subtype=1&&id=20022&&no=1000725
I only use chenhc@lemote.com to sign-off my patches, and you can
communicate with me via chenhuacai@gmail.com. Thank you.


>
> Thomas.
>
> --
> Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
> good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-16  9:36         ` Huacai Chen
@ 2020-05-16 11:55           ` Paolo Bonzini
  2020-05-16 15:38             ` Thomas Bogendoerfer
  2020-05-18  8:42             ` Aleksandar Markovic
  0 siblings, 2 replies; 31+ messages in thread
From: Paolo Bonzini @ 2020-05-16 11:55 UTC (permalink / raw)
  To: Huacai Chen, Thomas Bogendoerfer
  Cc: Jiaxun Yang, Aleksandar Markovic, kvm, open list:MIPS, Fuxin Zhang

On 16/05/20 11:36, Huacai Chen wrote:
>> I'm happy to see you taking care of the KVM part. So how is your plan
>> to handle patches ? Do want to collect them and send pull requests to
>> me ? Or should I just pick them up with your Acked-by ?
> I think we can only use the second method, because both Aleksandar and
> me don't have a kernel tree in kernel.org now.

If you don't mind, I generally prefer to have MIPS changs submitted
through the KVM tree, because KVM patches rarely have intrusive changes
in generic arch files.  It's more common to have generic KVM patches
that require touching all architectures.

For 5.8 I don't have anything planned that could cause conflicts, so
this time it doesn't matter; but I can pick these up too if Thomas acks
patches 6, 12 and 14.

Thanks,

Paolo


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 14/15] KVM: MIPS: Enable KVM support for Loongson-3
  2020-05-13  8:42 ` [PATCH V5 14/15] KVM: MIPS: Enable KVM support for Loongson-3 Huacai Chen
  2020-05-14  7:46   ` Aleksandar Markovic
@ 2020-05-16 15:35   ` Thomas Bogendoerfer
  1 sibling, 0 replies; 31+ messages in thread
From: Thomas Bogendoerfer @ 2020-05-16 15:35 UTC (permalink / raw)
  To: Huacai Chen
  Cc: Paolo Bonzini, Aleksandar Markovic, kvm, linux-mips, Fuxin Zhang,
	Huacai Chen, Jiaxun Yang

On Wed, May 13, 2020 at 04:42:45PM +0800, Huacai Chen wrote:
> This patch enable KVM support for Loongson-3 by selecting HAVE_KVM, but
> only enable KVM/VZ on Loongson-3A R4+ (because VZ of early processors
> are incomplete). Besides, Loongson-3 support SMP guests, so we clear the
> linked load bit of LLAddr in kvm_vz_vcpu_load() if the guest has more
> than one VCPUs.
> 
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/Kconfig            | 1 +
>  arch/mips/kernel/cpu-probe.c | 1 +
>  arch/mips/kvm/vz.c           | 2 +-
>  3 files changed, 3 insertions(+), 1 deletion(-)

Acked-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 12/15] KVM: MIPS: Add CONFIG6 and DIAG registers emulation
  2020-05-13  8:42 ` [PATCH V5 12/15] KVM: MIPS: Add CONFIG6 and DIAG registers emulation Huacai Chen
  2020-05-14  7:45   ` Aleksandar Markovic
@ 2020-05-16 15:36   ` Thomas Bogendoerfer
  1 sibling, 0 replies; 31+ messages in thread
From: Thomas Bogendoerfer @ 2020-05-16 15:36 UTC (permalink / raw)
  To: Huacai Chen
  Cc: Paolo Bonzini, Aleksandar Markovic, kvm, linux-mips, Fuxin Zhang,
	Huacai Chen, Jiaxun Yang

On Wed, May 13, 2020 at 04:42:43PM +0800, Huacai Chen wrote:
> Loongson-3 has CONFIG6 and DIAG registers which need to be emulated.
> CONFIG6 is mostly used to enable/disable FTLB and SFB, while DIAG is
> mostly used to flush BTB, ITLB, DTLB, VTLB and FTLB.
> 
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/include/asm/kvm_host.h |  7 +++++
>  arch/mips/include/asm/mipsregs.h |  7 +++++
>  arch/mips/kvm/tlb.c              | 41 ++++++++++++++++++++++++++
>  arch/mips/kvm/vz.c               | 62 +++++++++++++++++++++++++++++++++++++++-
>  4 files changed, 116 insertions(+), 1 deletion(-)

Acked-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 06/15] KVM: MIPS: Introduce and use cpu_guest_has_ldpte
  2020-05-13  8:42 ` [PATCH V5 06/15] KVM: MIPS: Introduce and use cpu_guest_has_ldpte Huacai Chen
@ 2020-05-16 15:37   ` Thomas Bogendoerfer
  0 siblings, 0 replies; 31+ messages in thread
From: Thomas Bogendoerfer @ 2020-05-16 15:37 UTC (permalink / raw)
  To: Huacai Chen
  Cc: Paolo Bonzini, Aleksandar Markovic, kvm, linux-mips, Fuxin Zhang,
	Huacai Chen, Jiaxun Yang

On Wed, May 13, 2020 at 04:42:37PM +0800, Huacai Chen wrote:
> Loongson-3 has lddir/ldpte instructions and their related CP0 registers
> are the same as HTW. So we introduce a cpu_guest_has_ldpte flag and use
> it to indicate whether we need to save/restore HTW related CP0 registers
> (PWBase, PWSize, PWField and PWCtl).
> 
> Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/include/asm/cpu-features.h |  3 +++
>  arch/mips/kernel/cpu-probe.c         |  1 +
>  arch/mips/kvm/vz.c                   | 26 +++++++++++++-------------
>  3 files changed, 17 insertions(+), 13 deletions(-)

Acked-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-16 11:55           ` Paolo Bonzini
@ 2020-05-16 15:38             ` Thomas Bogendoerfer
  2020-05-17  5:50               ` Huacai Chen
  2020-05-18  8:42             ` Aleksandar Markovic
  1 sibling, 1 reply; 31+ messages in thread
From: Thomas Bogendoerfer @ 2020-05-16 15:38 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Huacai Chen, Jiaxun Yang, Aleksandar Markovic, kvm,
	open list:MIPS, Fuxin Zhang

On Sat, May 16, 2020 at 01:55:23PM +0200, Paolo Bonzini wrote:
> On 16/05/20 11:36, Huacai Chen wrote:
> >> I'm happy to see you taking care of the KVM part. So how is your plan
> >> to handle patches ? Do want to collect them and send pull requests to
> >> me ? Or should I just pick them up with your Acked-by ?
> > I think we can only use the second method, because both Aleksandar and
> > me don't have a kernel tree in kernel.org now.
> 
> If you don't mind, I generally prefer to have MIPS changs submitted
> through the KVM tree, because KVM patches rarely have intrusive changes
> in generic arch files.  It's more common to have generic KVM patches
> that require touching all architectures.

works for me.

> For 5.8 I don't have anything planned that could cause conflicts, so
> this time it doesn't matter; but I can pick these up too if Thomas acks
> patches 6, 12 and 14.

I've acked all three pacthes

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-16 15:38             ` Thomas Bogendoerfer
@ 2020-05-17  5:50               ` Huacai Chen
  0 siblings, 0 replies; 31+ messages in thread
From: Huacai Chen @ 2020-05-17  5:50 UTC (permalink / raw)
  To: Thomas Bogendoerfer
  Cc: Paolo Bonzini, Jiaxun Yang, Aleksandar Markovic, kvm,
	open list:MIPS, Fuxin Zhang

Hi, Paolo and Thomas,

On Sat, May 16, 2020 at 11:39 PM Thomas Bogendoerfer
<tsbogend@alpha.franken.de> wrote:
>
> On Sat, May 16, 2020 at 01:55:23PM +0200, Paolo Bonzini wrote:
> > On 16/05/20 11:36, Huacai Chen wrote:
> > >> I'm happy to see you taking care of the KVM part. So how is your plan
> > >> to handle patches ? Do want to collect them and send pull requests to
> > >> me ? Or should I just pick them up with your Acked-by ?
> > > I think we can only use the second method, because both Aleksandar and
> > > me don't have a kernel tree in kernel.org now.
> >
> > If you don't mind, I generally prefer to have MIPS changs submitted
> > through the KVM tree, because KVM patches rarely have intrusive changes
> > in generic arch files.  It's more common to have generic KVM patches
> > that require touching all architectures.
>
> works for me.
>
> > For 5.8 I don't have anything planned that could cause conflicts, so
> > this time it doesn't matter; but I can pick these up too if Thomas acks
> > patches 6, 12 and 14.
>
> I've acked all three pacthes
>
> Thomas.
Thank you very much, I've send V6 instead of V5, because V5 has a
small rebase issue.

>
> --
> Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
> good idea.                                                [ RFC1925, 2.3 ]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers
  2020-05-16 11:55           ` Paolo Bonzini
  2020-05-16 15:38             ` Thomas Bogendoerfer
@ 2020-05-18  8:42             ` Aleksandar Markovic
  1 sibling, 0 replies; 31+ messages in thread
From: Aleksandar Markovic @ 2020-05-18  8:42 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Huacai Chen, Thomas Bogendoerfer, Jiaxun Yang, kvm,
	open list:MIPS, Fuxin Zhang

суб, 16. мај 2020. у 13:55 Paolo Bonzini <pbonzini@redhat.com> је написао/ла:
>
> On 16/05/20 11:36, Huacai Chen wrote:
> >> I'm happy to see you taking care of the KVM part. So how is your plan
> >> to handle patches ? Do want to collect them and send pull requests to
> >> me ? Or should I just pick them up with your Acked-by ?
> > I think we can only use the second method, because both Aleksandar and
> > me don't have a kernel tree in kernel.org now.
>
> If you don't mind, I generally prefer to have MIPS changs submitted
> through the KVM tree, because KVM patches rarely have intrusive changes
> in generic arch files.  It's more common to have generic KVM patches
> that require touching all architectures.
>

To me, Paolo's workflow seems reasonable and efficient from practical
point of view. No unnecessary multitude of repeated series or pull sending,
and no single person is burdened with majority of work being done just by
himself for any given case of series like this one. Also, it looks to me it is
quite efficient wrt possible conflict and rebase issues, that will be taken
care by both KVM/MIPS and KVM/general submitters/maintainers.

I suggest we keep Paolo's workflow plan for this series, but, more
importantly, if you guys all agree, in future as well.

Sincerely,
Aleksandar

> For 5.8 I don't have anything planned that could cause conflicts, so
> this time it doesn't matter; but I can pick these up too if Thomas acks
> patches 6, 12 and 14.
>
> Thanks,
>
> Paolo
>

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [PATCH V5 01/15] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
  2020-05-13  8:42 ` [PATCH V5 01/15] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data) Huacai Chen
@ 2020-05-19 11:49   ` Sasha Levin
  0 siblings, 0 replies; 31+ messages in thread
From: Sasha Levin @ 2020-05-19 11:49 UTC (permalink / raw)
  To: Sasha Levin, Huacai Chen, Xing Li, Paolo Bonzini
  Cc: kvm, linux-mips, stable, stable

Hi

[This is an automated email]

This commit has been processed because it contains a -stable tag.
The stable tag indicates that it's relevant for the following trees: all

The bot has tested the following trees: v5.6.13, v5.4.41, v4.19.123, v4.14.180, v4.9.223, v4.4.223.

v5.6.13: Build OK!
v5.4.41: Build OK!
v4.19.123: Build OK!
v4.14.180: Build OK!
v4.9.223: Build OK!
v4.4.223: Failed to apply! Possible dependencies:
    029499b47738 ("KVM: x86: MMU: Make mmu_set_spte() return emulate value")
    19d194c62b25 ("MIPS: KVM: Simplify TLB_* macros")
    403015b323a2 ("MIPS: KVM: Move non-TLB handling code out of tlb.c")
    7ee0e5b29d27 ("KVM: x86: MMU: Remove unused parameter of __direct_map()")
    9fbfb06a4065 ("MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo*")
    ba049e93aef7 ("kvm: rename pfn_t to kvm_pfn_t")
    bdb7ed8608f8 ("MIPS: KVM: Convert headers to kernel sized types")
    ca64c2beecd4 ("MIPS: KVM: Abstract guest ASID mask")
    caa1faa7aba6 ("MIPS: KVM: Trivial whitespace and style fixes")
    e6207bbea16c ("MIPS: KVM: Use MIPS_ENTRYLO_* defs from mipsregs.h")


NOTE: The patch will not be queued to stable trees until it is upstream.

How should we proceed with this patch?

-- 
Thanks
Sasha

^ permalink raw reply	[flat|nested] 31+ messages in thread

end of thread, other threads:[~2020-05-19 11:49 UTC | newest]

Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-13  8:42 [PATCH V5 00/15] KVM: MIPS: Add Loongson-3 support (Host Side) Huacai Chen
2020-05-13  8:42 ` [PATCH V5 01/15] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data) Huacai Chen
2020-05-19 11:49   ` Sasha Levin
2020-05-13  8:42 ` [PATCH V5 02/15] KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits Huacai Chen
2020-05-13  8:42 ` [PATCH V5 03/15] KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16 Huacai Chen
2020-05-13  8:42 ` [PATCH V5 04/15] KVM: MIPS: Add EVENTFD support which is needed by VHOST Huacai Chen
2020-05-13  8:42 ` [PATCH V5 05/15] KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd Huacai Chen
2020-05-13  8:42 ` [PATCH V5 06/15] KVM: MIPS: Introduce and use cpu_guest_has_ldpte Huacai Chen
2020-05-16 15:37   ` Thomas Bogendoerfer
2020-05-13  8:42 ` [PATCH V5 07/15] KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3 Huacai Chen
2020-05-13  8:42 ` [PATCH V5 08/15] KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3 Huacai Chen
2020-05-13  8:42 ` [PATCH V5 09/15] KVM: MIPS: Add more types of virtual interrupts Huacai Chen
2020-05-13  8:42 ` [PATCH V5 10/15] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support Huacai Chen
2020-05-13  8:42 ` [PATCH V5 11/15] KVM: MIPS: Add CPUCFG emulation for Loongson-3 Huacai Chen
2020-05-13  8:42 ` [PATCH V5 12/15] KVM: MIPS: Add CONFIG6 and DIAG registers emulation Huacai Chen
2020-05-14  7:45   ` Aleksandar Markovic
2020-05-16 15:36   ` Thomas Bogendoerfer
2020-05-13  8:42 ` [PATCH V5 13/15] KVM: MIPS: Add more MMIO load/store instructions emulation Huacai Chen
2020-05-13  8:42 ` [PATCH V5 14/15] KVM: MIPS: Enable KVM support for Loongson-3 Huacai Chen
2020-05-14  7:46   ` Aleksandar Markovic
2020-05-16 15:35   ` Thomas Bogendoerfer
2020-05-13  8:42 ` [PATCH V5 15/15] MAINTAINERS: Update KVM/MIPS maintainers Huacai Chen
2020-05-14  7:47   ` Aleksandar Markovic
2020-05-14 13:33   ` Jiaxun Yang
2020-05-15  1:50     ` Huacai Chen
2020-05-15 21:13       ` Thomas Bogendoerfer
2020-05-16  9:36         ` Huacai Chen
2020-05-16 11:55           ` Paolo Bonzini
2020-05-16 15:38             ` Thomas Bogendoerfer
2020-05-17  5:50               ` Huacai Chen
2020-05-18  8:42             ` Aleksandar Markovic

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.