[PATCH V3 00/14] KVM: MIPS: Add Loongson-3 support (Host Side)

[PATCH V3 00/14] KVM: MIPS: Add Loongson-3 support (Host Side)

[Huacai Chen]

We are preparing to add KVM support for Loongson-3. VZ extension is
fully supported in Loongson-3A R4+, and we will not care about old CPUs
(at least now). We already have a full functional Linux kernel (based
on Linux-5.4.x LTS) and QEMU (based on 5.0.0-rc2) and their git
repositories are here:

QEMU: https://github.com/chenhuacai/qemu
Kernel: https://github.com/chenhuacai/linux

Of course these two repositories need to be rework and not suitable for
upstream (especially the commits need to be splitted). We show them here
is just to tell others what we have done, and how KVM/Loongson will look
like.

Our plan is make the KVM host side be upstream first, and after that,
we will make the KVM guest side and QEMU emulator be upstream.

V1 -> V2:
1, Remove "mips: define pud_index() regardless of page table folding"
   because it has been applied.
2, Make Loongson-specific code be guarded by CONFIG_CPU_LOONGSON64.

V2 -> V3:
1, Emulate a reduced feature list of CPUCFG.
2, Fix all possible checkpatch.pl errors and warnings.

Xing Li(2):
 KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
 KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits

Huacai Chen(12):
 KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16
 KVM: MIPS: Add EVENTFD support which is needed by VHOST
 KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd
 KVM: MIPS: Introduce and use cpu_guest_has_ldpte
 KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3
 KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3
 KVM: MIPS: Add more types of virtual interrupts
 KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support
 KVM: MIPS: Add CPUCFG emulation for Loongson-3
 KVM: MIPS: Add CONFIG6 and DIAG registers emulation
 KVM: MIPS: Add more MMIO load/store instructions emulation
 KVM: MIPS: Enable KVM support for Loongson-3

Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 arch/mips/Kconfig                    |   1 +
 arch/mips/include/asm/cpu-features.h |   3 +
 arch/mips/include/asm/kvm_host.h     |  52 +++-
 arch/mips/include/asm/mipsregs.h     |   7 +
 arch/mips/include/uapi/asm/inst.h    |  11 +
 arch/mips/kernel/cpu-probe.c         |   2 +
 arch/mips/kvm/Kconfig                |   1 +
 arch/mips/kvm/Makefile               |   5 +-
 arch/mips/kvm/emulate.c              | 503 ++++++++++++++++++++++++++++++++++-
 arch/mips/kvm/entry.c                |  19 +-
 arch/mips/kvm/interrupt.c            |  93 +------
 arch/mips/kvm/interrupt.h            |  14 +-
 arch/mips/kvm/loongson_ipi.c         | 214 +++++++++++++++
 arch/mips/kvm/mips.c                 |  49 +++-
 arch/mips/kvm/tlb.c                  |  41 +++
 arch/mips/kvm/trap_emul.c            |   3 +
 arch/mips/kvm/vz.c                   | 235 +++++++++++-----
 17 files changed, 1087 insertions(+), 166 deletions(-)
 create mode 100644 arch/mips/kvm/loongson_ipi.c
--
2.7.0

[PATCH V3 00/14] KVM: MIPS: Add Loongson-3 support (Host Side)

[Huacai Chen]

We are preparing to add KVM support for Loongson-3. VZ extension is
fully supported in Loongson-3A R4+, and we will not care about old CPUs
(at least now). We already have a full functional Linux kernel (based
on Linux-5.4.x LTS) and QEMU (based on 5.0.0-rc2) and their git
repositories are here:

QEMU: https://github.com/chenhuacai/qemu
Kernel: https://github.com/chenhuacai/linux

Of course these two repositories need to be rework and not suitable for
upstream (especially the commits need to be splitted). We show them here
is just to tell others what we have done, and how KVM/Loongson will look
like.

Our plan is make the KVM host side be upstream first, and after that,
we will make the KVM guest side and QEMU emulator be upstream.

V1 -> V2:
1, Remove "mips: define pud_index() regardless of page table folding"
   because it has been applied.
2, Make Loongson-specific code be guarded by CONFIG_CPU_LOONGSON64.

V2 -> V3:
1, Emulate a reduced feature list of CPUCFG.
2, Fix all possible checkpatch.pl errors and warnings.

Xing Li(2):
 KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
 KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits

Huacai Chen(12):
 KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16
 KVM: MIPS: Add EVENTFD support which is needed by VHOST
 KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd
 KVM: MIPS: Introduce and use cpu_guest_has_ldpte
 KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3
 KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3
 KVM: MIPS: Add more types of virtual interrupts
 KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support
 KVM: MIPS: Add CPUCFG emulation for Loongson-3
 KVM: MIPS: Add CONFIG6 and DIAG registers emulation
 KVM: MIPS: Add more MMIO load/store instructions emulation
 KVM: MIPS: Enable KVM support for Loongson-3

Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 arch/mips/Kconfig                    |   1 +
 arch/mips/include/asm/cpu-features.h |   3 +
 arch/mips/include/asm/kvm_host.h     |  52 +++-
 arch/mips/include/asm/mipsregs.h     |   7 +
 arch/mips/include/uapi/asm/inst.h    |  11 +
 arch/mips/kernel/cpu-probe.c         |   2 +
 arch/mips/kvm/Kconfig                |   1 +
 arch/mips/kvm/Makefile               |   5 +-
 arch/mips/kvm/emulate.c              | 503 ++++++++++++++++++++++++++++++++++-
 arch/mips/kvm/entry.c                |  19 +-
 arch/mips/kvm/interrupt.c            |  93 +------
 arch/mips/kvm/interrupt.h            |  14 +-
 arch/mips/kvm/loongson_ipi.c         | 214 +++++++++++++++
 arch/mips/kvm/mips.c                 |  49 +++-
 arch/mips/kvm/tlb.c                  |  41 +++
 arch/mips/kvm/trap_emul.c            |   3 +
 arch/mips/kvm/vz.c                   | 235 +++++++++++-----
 17 files changed, 1087 insertions(+), 166 deletions(-)
 create mode 100644 arch/mips/kvm/loongson_ipi.c
--
2.7.0

[PATCH V3 01/14] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)

[Huacai Chen]

From: Xing Li <lixing@loongson.cn>

The code in decode_config4() of arch/mips/kernel/cpu-probe.c

        asid_mask = MIPS_ENTRYHI_ASID;
        if (config4 & MIPS_CONF4_AE)
                asid_mask |= MIPS_ENTRYHI_ASIDX;
        set_cpu_asid_mask(c, asid_mask);

set asid_mask to cpuinfo->asid_mask.

So in order to support variable ASID_MASK, KVM_ENTRYHI_ASID should also
be changed to cpu_asid_mask(&boot_cpu_data).

Cc: stable@vger.kernel.org
Signed-off-by: Xing Li <lixing@loongson.cn>
[Huacai: Change current_cpu_data to boot_cpu_data for optimization]
Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 arch/mips/include/asm/kvm_host.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 2c343c3..a01cee9 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -275,7 +275,7 @@ enum emulation_result {
 #define MIPS3_PG_FRAME		0x3fffffc0
 
 #define VPN2_MASK		0xffffe000
-#define KVM_ENTRYHI_ASID	MIPS_ENTRYHI_ASID
+#define KVM_ENTRYHI_ASID	cpu_asid_mask(&boot_cpu_data)
 #define TLB_IS_GLOBAL(x)	((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
 #define TLB_VPN2(x)		((x).tlb_hi & VPN2_MASK)
 #define TLB_ASID(x)		((x).tlb_hi & KVM_ENTRYHI_ASID)
-- 
2.7.0

[PATCH V3 01/14] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)

[Huacai Chen]

From: Xing Li <lixing@loongson.cn>

The code in decode_config4() of arch/mips/kernel/cpu-probe.c

        asid_mask = MIPS_ENTRYHI_ASID;
        if (config4 & MIPS_CONF4_AE)
                asid_mask |= MIPS_ENTRYHI_ASIDX;
        set_cpu_asid_mask(c, asid_mask);

set asid_mask to cpuinfo->asid_mask.

So in order to support variable ASID_MASK, KVM_ENTRYHI_ASID should also
be changed to cpu_asid_mask(&boot_cpu_data).

Cc: stable@vger.kernel.org
Signed-off-by: Xing Li <lixing@loongson.cn>
[Huacai: Change current_cpu_data to boot_cpu_data for optimization]
Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 arch/mips/include/asm/kvm_host.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 2c343c3..a01cee9 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -275,7 +275,7 @@ enum emulation_result {
 #define MIPS3_PG_FRAME		0x3fffffc0
 
 #define VPN2_MASK		0xffffe000
-#define KVM_ENTRYHI_ASID	MIPS_ENTRYHI_ASID
+#define KVM_ENTRYHI_ASID	cpu_asid_mask(&boot_cpu_data)
 #define TLB_IS_GLOBAL(x)	((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
 #define TLB_VPN2(x)		((x).tlb_hi & VPN2_MASK)
 #define TLB_ASID(x)		((x).tlb_hi & KVM_ENTRYHI_ASID)
-- 
2.7.0

Re: [PATCH V3 01/14] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)

[Sasha Levin]

Hi

[This is an automated email]

This commit has been processed because it contains a -stable tag.
The stable tag indicates that it's relevant for the following trees: all

The bot has tested the following trees: v5.6.10, v5.4.38, v4.19.120, v4.14.178, v4.9.221, v4.4.221.

v5.6.10: Build OK!
v5.4.38: Build OK!
v4.19.120: Build OK!
v4.14.178: Build OK!
v4.9.221: Build OK!
v4.4.221: Failed to apply! Possible dependencies:
    029499b47738 ("KVM: x86: MMU: Make mmu_set_spte() return emulate value")
    19d194c62b25 ("MIPS: KVM: Simplify TLB_* macros")
    403015b323a2 ("MIPS: KVM: Move non-TLB handling code out of tlb.c")
    7ee0e5b29d27 ("KVM: x86: MMU: Remove unused parameter of __direct_map()")
    9fbfb06a4065 ("MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo*")
    ba049e93aef7 ("kvm: rename pfn_t to kvm_pfn_t")
    bdb7ed8608f8 ("MIPS: KVM: Convert headers to kernel sized types")
    ca64c2beecd4 ("MIPS: KVM: Abstract guest ASID mask")
    caa1faa7aba6 ("MIPS: KVM: Trivial whitespace and style fixes")
    e6207bbea16c ("MIPS: KVM: Use MIPS_ENTRYLO_* defs from mipsregs.h")


NOTE: The patch will not be queued to stable trees until it is upstream.

How should we proceed with this patch?

-- 
Thanks
Sasha

Re: [PATCH V3 01/14] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)

[Sasha Levin]

Hi

[This is an automated email]

This commit has been processed because it contains a -stable tag.
The stable tag indicates that it's relevant for the following trees: all

The bot has tested the following trees: v5.6.10, v5.4.38, v4.19.120, v4.14.178, v4.9.221, v4.4.221.

v5.6.10: Build OK!
v5.4.38: Build OK!
v4.19.120: Build OK!
v4.14.178: Build OK!
v4.9.221: Build OK!
v4.4.221: Failed to apply! Possible dependencies:
    029499b47738 ("KVM: x86: MMU: Make mmu_set_spte() return emulate value")
    19d194c62b25 ("MIPS: KVM: Simplify TLB_* macros")
    403015b323a2 ("MIPS: KVM: Move non-TLB handling code out of tlb.c")
    7ee0e5b29d27 ("KVM: x86: MMU: Remove unused parameter of __direct_map()")
    9fbfb06a4065 ("MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo*")
    ba049e93aef7 ("kvm: rename pfn_t to kvm_pfn_t")
    bdb7ed8608f8 ("MIPS: KVM: Convert headers to kernel sized types")
    ca64c2beecd4 ("MIPS: KVM: Abstract guest ASID mask")
    caa1faa7aba6 ("MIPS: KVM: Trivial whitespace and style fixes")
    e6207bbea16c ("MIPS: KVM: Use MIPS_ENTRYLO_* defs from mipsregs.h")


NOTE: The patch will not be queued to stable trees until it is upstream.

How should we proceed with this patch?

-- 
Thanks
Sasha

Re: [PATCH V3 01/14] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)

[Aleksandar Markovic]

чет, 7. мај 2020. у 01:43 Sasha Levin <sashal@kernel.org> је написао/ла:
>
> Hi
>
> [This is an automated email]
>
> This commit has been processed because it contains a -stable tag.
> The stable tag indicates that it's relevant for the following trees: all
>
> The bot has tested the following trees: v5.6.10, v5.4.38, v4.19.120, v4.14.178, v4.9.221, v4.4.221.
>
> v5.6.10: Build OK!
> v5.4.38: Build OK!
> v4.19.120: Build OK!
> v4.14.178: Build OK!
> v4.9.221: Build OK!
> v4.4.221: Failed to apply! Possible dependencies:
>     029499b47738 ("KVM: x86: MMU: Make mmu_set_spte() return emulate value")
>     19d194c62b25 ("MIPS: KVM: Simplify TLB_* macros")
>     403015b323a2 ("MIPS: KVM: Move non-TLB handling code out of tlb.c")
>     7ee0e5b29d27 ("KVM: x86: MMU: Remove unused parameter of __direct_map()")
>     9fbfb06a4065 ("MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo*")
>     ba049e93aef7 ("kvm: rename pfn_t to kvm_pfn_t")
>     bdb7ed8608f8 ("MIPS: KVM: Convert headers to kernel sized types")
>     ca64c2beecd4 ("MIPS: KVM: Abstract guest ASID mask")
>     caa1faa7aba6 ("MIPS: KVM: Trivial whitespace and style fixes")
>     e6207bbea16c ("MIPS: KVM: Use MIPS_ENTRYLO_* defs from mipsregs.h")
>
>
> NOTE: The patch will not be queued to stable trees until it is upstream.
>
> How should we proceed with this patch?
>

Hi, Sasha.

Please wait for the the review of the whole series. It might also be
the case that this patch will be upstreamed before the whole series,
but please do not rush or skip regular review process.

Thanks for you involvement and efforts!

Aleksandar

> --
> Thanks
> Sasha
>

Re: [PATCH V3 01/14] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)

[Aleksandar Markovic]

чет, 7. мај 2020. у 01:43 Sasha Levin <sashal@kernel.org> је написао/ла:
>
> Hi
>
> [This is an automated email]
>
> This commit has been processed because it contains a -stable tag.
> The stable tag indicates that it's relevant for the following trees: all
>
> The bot has tested the following trees: v5.6.10, v5.4.38, v4.19.120, v4.14.178, v4.9.221, v4.4.221.
>
> v5.6.10: Build OK!
> v5.4.38: Build OK!
> v4.19.120: Build OK!
> v4.14.178: Build OK!
> v4.9.221: Build OK!
> v4.4.221: Failed to apply! Possible dependencies:
>     029499b47738 ("KVM: x86: MMU: Make mmu_set_spte() return emulate value")
>     19d194c62b25 ("MIPS: KVM: Simplify TLB_* macros")
>     403015b323a2 ("MIPS: KVM: Move non-TLB handling code out of tlb.c")
>     7ee0e5b29d27 ("KVM: x86: MMU: Remove unused parameter of __direct_map()")
>     9fbfb06a4065 ("MIPS: KVM: Arrayify struct kvm_mips_tlb::tlb_lo*")
>     ba049e93aef7 ("kvm: rename pfn_t to kvm_pfn_t")
>     bdb7ed8608f8 ("MIPS: KVM: Convert headers to kernel sized types")
>     ca64c2beecd4 ("MIPS: KVM: Abstract guest ASID mask")
>     caa1faa7aba6 ("MIPS: KVM: Trivial whitespace and style fixes")
>     e6207bbea16c ("MIPS: KVM: Use MIPS_ENTRYLO_* defs from mipsregs.h")
>
>
> NOTE: The patch will not be queued to stable trees until it is upstream.
>
> How should we proceed with this patch?
>

Hi, Sasha.

Please wait for the the review of the whole series. It might also be
the case that this patch will be upstreamed before the whole series,
but please do not rush or skip regular review process.

Thanks for you involvement and efforts!

Aleksandar

> --
> Thanks
> Sasha
>

Re: [PATCH V3 01/14] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:06 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> From: Xing Li <lixing@loongson.cn>
>
> The code in decode_config4() of arch/mips/kernel/cpu-probe.c
>
>         asid_mask = MIPS_ENTRYHI_ASID;
>         if (config4 & MIPS_CONF4_AE)
>                 asid_mask |= MIPS_ENTRYHI_ASIDX;
>         set_cpu_asid_mask(c, asid_mask);
>
> set asid_mask to cpuinfo->asid_mask.
>
> So in order to support variable ASID_MASK, KVM_ENTRYHI_ASID should also
> be changed to cpu_asid_mask(&boot_cpu_data).
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Xing Li <lixing@loongson.cn>
> [Huacai: Change current_cpu_data to boot_cpu_data for optimization]
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> ---
>  arch/mips/include/asm/kvm_host.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>

For what is worth:

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index 2c343c3..a01cee9 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -275,7 +275,7 @@ enum emulation_result {
>  #define MIPS3_PG_FRAME         0x3fffffc0
>
>  #define VPN2_MASK              0xffffe000
> -#define KVM_ENTRYHI_ASID       MIPS_ENTRYHI_ASID
> +#define KVM_ENTRYHI_ASID       cpu_asid_mask(&boot_cpu_data)
>  #define TLB_IS_GLOBAL(x)       ((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
>  #define TLB_VPN2(x)            ((x).tlb_hi & VPN2_MASK)
>  #define TLB_ASID(x)            ((x).tlb_hi & KVM_ENTRYHI_ASID)
> --
> 2.7.0
>

Re: [PATCH V3 01/14] KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:06 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> From: Xing Li <lixing@loongson.cn>
>
> The code in decode_config4() of arch/mips/kernel/cpu-probe.c
>
>         asid_mask = MIPS_ENTRYHI_ASID;
>         if (config4 & MIPS_CONF4_AE)
>                 asid_mask |= MIPS_ENTRYHI_ASIDX;
>         set_cpu_asid_mask(c, asid_mask);
>
> set asid_mask to cpuinfo->asid_mask.
>
> So in order to support variable ASID_MASK, KVM_ENTRYHI_ASID should also
> be changed to cpu_asid_mask(&boot_cpu_data).
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Xing Li <lixing@loongson.cn>
> [Huacai: Change current_cpu_data to boot_cpu_data for optimization]
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> ---
>  arch/mips/include/asm/kvm_host.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>

For what is worth:

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index 2c343c3..a01cee9 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -275,7 +275,7 @@ enum emulation_result {
>  #define MIPS3_PG_FRAME         0x3fffffc0
>
>  #define VPN2_MASK              0xffffe000
> -#define KVM_ENTRYHI_ASID       MIPS_ENTRYHI_ASID
> +#define KVM_ENTRYHI_ASID       cpu_asid_mask(&boot_cpu_data)
>  #define TLB_IS_GLOBAL(x)       ((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
>  #define TLB_VPN2(x)            ((x).tlb_hi & VPN2_MASK)
>  #define TLB_ASID(x)            ((x).tlb_hi & KVM_ENTRYHI_ASID)
> --
> 2.7.0
>

[PATCH V3 02/14] KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits

[Huacai Chen]

From: Xing Li <lixing@loongson.cn>

If a CPU support more than 32bit vmbits (which is true for 64bit CPUs),
VPN2_MASK set to fixed 0xffffe000 will lead to a wrong EntryHi in some
functions such as _kvm_mips_host_tlb_inv().

The cpu_vmbits definition of 32bit CPU in cpu-features.h is 31, so we
still use the old definition.

Cc: stable@vger.kernel.org
Signed-off-by: Xing Li <lixing@loongson.cn>
[Huacai: Improve commit messages]
Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 arch/mips/include/asm/kvm_host.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index a01cee9..caa2b936 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -274,7 +274,11 @@ enum emulation_result {
 #define MIPS3_PG_SHIFT		6
 #define MIPS3_PG_FRAME		0x3fffffc0
 
+#if defined(CONFIG_64BIT)
+#define VPN2_MASK		GENMASK(cpu_vmbits - 1, 13)
+#else
 #define VPN2_MASK		0xffffe000
+#endif
 #define KVM_ENTRYHI_ASID	cpu_asid_mask(&boot_cpu_data)
 #define TLB_IS_GLOBAL(x)	((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
 #define TLB_VPN2(x)		((x).tlb_hi & VPN2_MASK)
-- 
2.7.0

[PATCH V3 02/14] KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits

[Huacai Chen]

From: Xing Li <lixing@loongson.cn>

If a CPU support more than 32bit vmbits (which is true for 64bit CPUs),
VPN2_MASK set to fixed 0xffffe000 will lead to a wrong EntryHi in some
functions such as _kvm_mips_host_tlb_inv().

The cpu_vmbits definition of 32bit CPU in cpu-features.h is 31, so we
still use the old definition.

Cc: stable@vger.kernel.org
Signed-off-by: Xing Li <lixing@loongson.cn>
[Huacai: Improve commit messages]
Signed-off-by: Huacai Chen <chenhc@lemote.com>
---
 arch/mips/include/asm/kvm_host.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index a01cee9..caa2b936 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -274,7 +274,11 @@ enum emulation_result {
 #define MIPS3_PG_SHIFT		6
 #define MIPS3_PG_FRAME		0x3fffffc0
 
+#if defined(CONFIG_64BIT)
+#define VPN2_MASK		GENMASK(cpu_vmbits - 1, 13)
+#else
 #define VPN2_MASK		0xffffe000
+#endif
 #define KVM_ENTRYHI_ASID	cpu_asid_mask(&boot_cpu_data)
 #define TLB_IS_GLOBAL(x)	((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
 #define TLB_VPN2(x)		((x).tlb_hi & VPN2_MASK)
-- 
2.7.0

Re: [PATCH V3 02/14] KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:07 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> From: Xing Li <lixing@loongson.cn>
>
> If a CPU support more than 32bit vmbits (which is true for 64bit CPUs),
> VPN2_MASK set to fixed 0xffffe000 will lead to a wrong EntryHi in some
> functions such as _kvm_mips_host_tlb_inv().
>
> The cpu_vmbits definition of 32bit CPU in cpu-features.h is 31, so we
> still use the old definition.
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Xing Li <lixing@loongson.cn>
> [Huacai: Improve commit messages]
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> ---
>  arch/mips/include/asm/kvm_host.h | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index a01cee9..caa2b936 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -274,7 +274,11 @@ enum emulation_result {
>  #define MIPS3_PG_SHIFT         6
>  #define MIPS3_PG_FRAME         0x3fffffc0
>
> +#if defined(CONFIG_64BIT)
> +#define VPN2_MASK              GENMASK(cpu_vmbits - 1, 13)
> +#else
>  #define VPN2_MASK              0xffffe000
> +#endif
>  #define KVM_ENTRYHI_ASID       cpu_asid_mask(&boot_cpu_data)
>  #define TLB_IS_GLOBAL(x)       ((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
>  #define TLB_VPN2(x)            ((x).tlb_hi & VPN2_MASK)
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

Re: [PATCH V3 02/14] KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:07 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> From: Xing Li <lixing@loongson.cn>
>
> If a CPU support more than 32bit vmbits (which is true for 64bit CPUs),
> VPN2_MASK set to fixed 0xffffe000 will lead to a wrong EntryHi in some
> functions such as _kvm_mips_host_tlb_inv().
>
> The cpu_vmbits definition of 32bit CPU in cpu-features.h is 31, so we
> still use the old definition.
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Xing Li <lixing@loongson.cn>
> [Huacai: Improve commit messages]
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> ---
>  arch/mips/include/asm/kvm_host.h | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index a01cee9..caa2b936 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -274,7 +274,11 @@ enum emulation_result {
>  #define MIPS3_PG_SHIFT         6
>  #define MIPS3_PG_FRAME         0x3fffffc0
>
> +#if defined(CONFIG_64BIT)
> +#define VPN2_MASK              GENMASK(cpu_vmbits - 1, 13)
> +#else
>  #define VPN2_MASK              0xffffe000
> +#endif
>  #define KVM_ENTRYHI_ASID       cpu_asid_mask(&boot_cpu_data)
>  #define TLB_IS_GLOBAL(x)       ((x).tlb_lo[0] & (x).tlb_lo[1] & ENTRYLO_G)
>  #define TLB_VPN2(x)            ((x).tlb_hi & VPN2_MASK)
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

[PATCH V3 03/14] KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16

[Huacai Chen]

Loongson-3 based machines can have as many as 16 CPUs, and so does
memory slots, so increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index caa2b936..a7758c0 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -78,8 +78,8 @@
 #define KVM_REG_MIPS_CP0_KSCRATCH6	MIPS_CP0_64(31, 7)
 
 
-#define KVM_MAX_VCPUS		8
-#define KVM_USER_MEM_SLOTS	8
+#define KVM_MAX_VCPUS		16
+#define KVM_USER_MEM_SLOTS	16
 /* memory slots that does not exposed to userspace */
 #define KVM_PRIVATE_MEM_SLOTS	0
 
-- 
2.7.0

[PATCH V3 03/14] KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16

[Huacai Chen]

Loongson-3 based machines can have as many as 16 CPUs, and so does
memory slots, so increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index caa2b936..a7758c0 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -78,8 +78,8 @@
 #define KVM_REG_MIPS_CP0_KSCRATCH6	MIPS_CP0_64(31, 7)
 
 
-#define KVM_MAX_VCPUS		8
-#define KVM_USER_MEM_SLOTS	8
+#define KVM_MAX_VCPUS		16
+#define KVM_USER_MEM_SLOTS	16
 /* memory slots that does not exposed to userspace */
 #define KVM_PRIVATE_MEM_SLOTS	0
 
-- 
2.7.0

Re: [PATCH V3 03/14] KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:08 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3 based machines can have as many as 16 CPUs, and so does
> memory slots, so increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/include/asm/kvm_host.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index caa2b936..a7758c0 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -78,8 +78,8 @@
>  #define KVM_REG_MIPS_CP0_KSCRATCH6     MIPS_CP0_64(31, 7)
>
>
> -#define KVM_MAX_VCPUS          8
> -#define KVM_USER_MEM_SLOTS     8
> +#define KVM_MAX_VCPUS          16
> +#define KVM_USER_MEM_SLOTS     16
>  /* memory slots that does not exposed to userspace */
>  #define KVM_PRIVATE_MEM_SLOTS  0
>
> --
> 2.7.0
>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

Re: [PATCH V3 03/14] KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:08 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3 based machines can have as many as 16 CPUs, and so does
> memory slots, so increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/include/asm/kvm_host.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index caa2b936..a7758c0 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -78,8 +78,8 @@
>  #define KVM_REG_MIPS_CP0_KSCRATCH6     MIPS_CP0_64(31, 7)
>
>
> -#define KVM_MAX_VCPUS          8
> -#define KVM_USER_MEM_SLOTS     8
> +#define KVM_MAX_VCPUS          16
> +#define KVM_USER_MEM_SLOTS     16
>  /* memory slots that does not exposed to userspace */
>  #define KVM_PRIVATE_MEM_SLOTS  0
>
> --
> 2.7.0
>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

[PATCH V3 04/14] KVM: MIPS: Add EVENTFD support which is needed by VHOST

[Huacai Chen]

Add EVENTFD support for KVM/MIPS, which is needed by VHOST. Tested on
Loongson-3 platform.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/Kconfig     | 1 +
 arch/mips/kvm/Makefile    | 2 +-
 arch/mips/kvm/trap_emul.c | 3 +++
 arch/mips/kvm/vz.c        | 3 +++
 4 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
index b91d145..d697752 100644
--- a/arch/mips/kvm/Kconfig
+++ b/arch/mips/kvm/Kconfig
@@ -22,6 +22,7 @@ config KVM
 	select EXPORT_UASM
 	select PREEMPT_NOTIFIERS
 	select KVM_GENERIC_DIRTYLOG_READ_PROTECT
+	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_VCPU_ASYNC_IOCTL
 	select KVM_MMIO
 	select MMU_NOTIFIER
diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
index 01affc1..0a3cef6 100644
--- a/arch/mips/kvm/Makefile
+++ b/arch/mips/kvm/Makefile
@@ -2,7 +2,7 @@
 # Makefile for KVM support for MIPS
 #
 
-common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
+common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o eventfd.o)
 
 EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
 
diff --git a/arch/mips/kvm/trap_emul.c b/arch/mips/kvm/trap_emul.c
index 5a11e83..f464506b 100644
--- a/arch/mips/kvm/trap_emul.c
+++ b/arch/mips/kvm/trap_emul.c
@@ -529,6 +529,9 @@ static int kvm_trap_emul_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_MIPS_TE:
 		r = 1;
 		break;
+	case KVM_CAP_IOEVENTFD:
+		r = 1;
+		break;
 	default:
 		r = 0;
 		break;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index dde2088..17932ab 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2927,6 +2927,9 @@ static int kvm_vz_check_extension(struct kvm *kvm, long ext)
 		r = 2;
 		break;
 #endif
+	case KVM_CAP_IOEVENTFD:
+		r = 1;
+		break;
 	default:
 		r = 0;
 		break;
-- 
2.7.0

[PATCH V3 04/14] KVM: MIPS: Add EVENTFD support which is needed by VHOST

[Huacai Chen]

Add EVENTFD support for KVM/MIPS, which is needed by VHOST. Tested on
Loongson-3 platform.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/Kconfig     | 1 +
 arch/mips/kvm/Makefile    | 2 +-
 arch/mips/kvm/trap_emul.c | 3 +++
 arch/mips/kvm/vz.c        | 3 +++
 4 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
index b91d145..d697752 100644
--- a/arch/mips/kvm/Kconfig
+++ b/arch/mips/kvm/Kconfig
@@ -22,6 +22,7 @@ config KVM
 	select EXPORT_UASM
 	select PREEMPT_NOTIFIERS
 	select KVM_GENERIC_DIRTYLOG_READ_PROTECT
+	select HAVE_KVM_EVENTFD
 	select HAVE_KVM_VCPU_ASYNC_IOCTL
 	select KVM_MMIO
 	select MMU_NOTIFIER
diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
index 01affc1..0a3cef6 100644
--- a/arch/mips/kvm/Makefile
+++ b/arch/mips/kvm/Makefile
@@ -2,7 +2,7 @@
 # Makefile for KVM support for MIPS
 #
 
-common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
+common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o eventfd.o)
 
 EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
 
diff --git a/arch/mips/kvm/trap_emul.c b/arch/mips/kvm/trap_emul.c
index 5a11e83..f464506b 100644
--- a/arch/mips/kvm/trap_emul.c
+++ b/arch/mips/kvm/trap_emul.c
@@ -529,6 +529,9 @@ static int kvm_trap_emul_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_MIPS_TE:
 		r = 1;
 		break;
+	case KVM_CAP_IOEVENTFD:
+		r = 1;
+		break;
 	default:
 		r = 0;
 		break;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index dde2088..17932ab 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2927,6 +2927,9 @@ static int kvm_vz_check_extension(struct kvm *kvm, long ext)
 		r = 2;
 		break;
 #endif
+	case KVM_CAP_IOEVENTFD:
+		r = 1;
+		break;
 	default:
 		r = 0;
 		break;
-- 
2.7.0

Re: [PATCH V3 04/14] KVM: MIPS: Add EVENTFD support which is needed by VHOST

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:09 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Add EVENTFD support for KVM/MIPS, which is needed by VHOST. Tested on
> Loongson-3 platform.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---

Perhaps this patch needs rebasing to land cleanly into master.

But, in general, for what is worth:

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>


>  arch/mips/kvm/Kconfig     | 1 +
>  arch/mips/kvm/Makefile    | 2 +-
>  arch/mips/kvm/trap_emul.c | 3 +++
>  arch/mips/kvm/vz.c        | 3 +++
>  4 files changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
> index b91d145..d697752 100644
> --- a/arch/mips/kvm/Kconfig
> +++ b/arch/mips/kvm/Kconfig
> @@ -22,6 +22,7 @@ config KVM
>         select EXPORT_UASM
>         select PREEMPT_NOTIFIERS
>         select KVM_GENERIC_DIRTYLOG_READ_PROTECT
> +       select HAVE_KVM_EVENTFD
>         select HAVE_KVM_VCPU_ASYNC_IOCTL
>         select KVM_MMIO
>         select MMU_NOTIFIER
> diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> index 01affc1..0a3cef6 100644
> --- a/arch/mips/kvm/Makefile
> +++ b/arch/mips/kvm/Makefile
> @@ -2,7 +2,7 @@
>  # Makefile for KVM support for MIPS
>  #
>
> -common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
> +common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o eventfd.o)
>
>  EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
>
> diff --git a/arch/mips/kvm/trap_emul.c b/arch/mips/kvm/trap_emul.c
> index 5a11e83..f464506b 100644
> --- a/arch/mips/kvm/trap_emul.c
> +++ b/arch/mips/kvm/trap_emul.c
> @@ -529,6 +529,9 @@ static int kvm_trap_emul_check_extension(struct kvm *kvm, long ext)
>         case KVM_CAP_MIPS_TE:
>                 r = 1;
>                 break;
> +       case KVM_CAP_IOEVENTFD:
> +               r = 1;
> +               break;
>         default:
>                 r = 0;
>                 break;
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index dde2088..17932ab 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -2927,6 +2927,9 @@ static int kvm_vz_check_extension(struct kvm *kvm, long ext)
>                 r = 2;
>                 break;
>  #endif
> +       case KVM_CAP_IOEVENTFD:
> +               r = 1;
> +               break;
>         default:
>                 r = 0;
>                 break;
> --
> 2.7.0
>

Re: [PATCH V3 04/14] KVM: MIPS: Add EVENTFD support which is needed by VHOST

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:09 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Add EVENTFD support for KVM/MIPS, which is needed by VHOST. Tested on
> Loongson-3 platform.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---

Perhaps this patch needs rebasing to land cleanly into master.

But, in general, for what is worth:

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>


>  arch/mips/kvm/Kconfig     | 1 +
>  arch/mips/kvm/Makefile    | 2 +-
>  arch/mips/kvm/trap_emul.c | 3 +++
>  arch/mips/kvm/vz.c        | 3 +++
>  4 files changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
> index b91d145..d697752 100644
> --- a/arch/mips/kvm/Kconfig
> +++ b/arch/mips/kvm/Kconfig
> @@ -22,6 +22,7 @@ config KVM
>         select EXPORT_UASM
>         select PREEMPT_NOTIFIERS
>         select KVM_GENERIC_DIRTYLOG_READ_PROTECT
> +       select HAVE_KVM_EVENTFD
>         select HAVE_KVM_VCPU_ASYNC_IOCTL
>         select KVM_MMIO
>         select MMU_NOTIFIER
> diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> index 01affc1..0a3cef6 100644
> --- a/arch/mips/kvm/Makefile
> +++ b/arch/mips/kvm/Makefile
> @@ -2,7 +2,7 @@
>  # Makefile for KVM support for MIPS
>  #
>
> -common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
> +common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o eventfd.o)
>
>  EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
>
> diff --git a/arch/mips/kvm/trap_emul.c b/arch/mips/kvm/trap_emul.c
> index 5a11e83..f464506b 100644
> --- a/arch/mips/kvm/trap_emul.c
> +++ b/arch/mips/kvm/trap_emul.c
> @@ -529,6 +529,9 @@ static int kvm_trap_emul_check_extension(struct kvm *kvm, long ext)
>         case KVM_CAP_MIPS_TE:
>                 r = 1;
>                 break;
> +       case KVM_CAP_IOEVENTFD:
> +               r = 1;
> +               break;
>         default:
>                 r = 0;
>                 break;
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index dde2088..17932ab 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -2927,6 +2927,9 @@ static int kvm_vz_check_extension(struct kvm *kvm, long ext)
>                 r = 2;
>                 break;
>  #endif
> +       case KVM_CAP_IOEVENTFD:
> +               r = 1;
> +               break;
>         default:
>                 r = 0;
>                 break;
> --
> 2.7.0
>

Re: [PATCH V3 04/14] KVM: MIPS: Add EVENTFD support which is needed by VHOST

[Aleksandar Markovic]

пет, 8. мај 2020. у 11:08 Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> је написао/ла:
>
> нед, 3. мај 2020. у 12:09 Huacai Chen <chenhc@lemote.com> је написао/ла:
> >
> > Add EVENTFD support for KVM/MIPS, which is needed by VHOST. Tested on
> > Loongson-3 platform.
> >
> > Signed-off-by: Huacai Chen <chenhc@lemote.com>
> > Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> > ---
>
> Perhaps this patch needs rebasing to land cleanly into master.
>

Or, maybe, my tree was out-of-date. Sorry if that was the case.

Yours,
Aleksandar

> But, in general, for what is worth:
>
> Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
>
>
> >  arch/mips/kvm/Kconfig     | 1 +
> >  arch/mips/kvm/Makefile    | 2 +-
> >  arch/mips/kvm/trap_emul.c | 3 +++
> >  arch/mips/kvm/vz.c        | 3 +++
> >  4 files changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
> > index b91d145..d697752 100644
> > --- a/arch/mips/kvm/Kconfig
> > +++ b/arch/mips/kvm/Kconfig
> > @@ -22,6 +22,7 @@ config KVM
> >         select EXPORT_UASM
> >         select PREEMPT_NOTIFIERS
> >         select KVM_GENERIC_DIRTYLOG_READ_PROTECT
> > +       select HAVE_KVM_EVENTFD
> >         select HAVE_KVM_VCPU_ASYNC_IOCTL
> >         select KVM_MMIO
> >         select MMU_NOTIFIER
> > diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> > index 01affc1..0a3cef6 100644
> > --- a/arch/mips/kvm/Makefile
> > +++ b/arch/mips/kvm/Makefile
> > @@ -2,7 +2,7 @@
> >  # Makefile for KVM support for MIPS
> >  #
> >
> > -common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
> > +common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o eventfd.o)
> >
> >  EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
> >
> > diff --git a/arch/mips/kvm/trap_emul.c b/arch/mips/kvm/trap_emul.c
> > index 5a11e83..f464506b 100644
> > --- a/arch/mips/kvm/trap_emul.c
> > +++ b/arch/mips/kvm/trap_emul.c
> > @@ -529,6 +529,9 @@ static int kvm_trap_emul_check_extension(struct kvm *kvm, long ext)
> >         case KVM_CAP_MIPS_TE:
> >                 r = 1;
> >                 break;
> > +       case KVM_CAP_IOEVENTFD:
> > +               r = 1;
> > +               break;
> >         default:
> >                 r = 0;
> >                 break;
> > diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> > index dde2088..17932ab 100644
> > --- a/arch/mips/kvm/vz.c
> > +++ b/arch/mips/kvm/vz.c
> > @@ -2927,6 +2927,9 @@ static int kvm_vz_check_extension(struct kvm *kvm, long ext)
> >                 r = 2;
> >                 break;
> >  #endif
> > +       case KVM_CAP_IOEVENTFD:
> > +               r = 1;
> > +               break;
> >         default:
> >                 r = 0;
> >                 break;
> > --
> > 2.7.0
> >

Re: [PATCH V3 04/14] KVM: MIPS: Add EVENTFD support which is needed by VHOST

[Aleksandar Markovic]

пет, 8. мај 2020. у 11:08 Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> је написао/ла:
>
> нед, 3. мај 2020. у 12:09 Huacai Chen <chenhc@lemote.com> је написао/ла:
> >
> > Add EVENTFD support for KVM/MIPS, which is needed by VHOST. Tested on
> > Loongson-3 platform.
> >
> > Signed-off-by: Huacai Chen <chenhc@lemote.com>
> > Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> > ---
>
> Perhaps this patch needs rebasing to land cleanly into master.
>

Or, maybe, my tree was out-of-date. Sorry if that was the case.

Yours,
Aleksandar

> But, in general, for what is worth:
>
> Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
>
>
> >  arch/mips/kvm/Kconfig     | 1 +
> >  arch/mips/kvm/Makefile    | 2 +-
> >  arch/mips/kvm/trap_emul.c | 3 +++
> >  arch/mips/kvm/vz.c        | 3 +++
> >  4 files changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
> > index b91d145..d697752 100644
> > --- a/arch/mips/kvm/Kconfig
> > +++ b/arch/mips/kvm/Kconfig
> > @@ -22,6 +22,7 @@ config KVM
> >         select EXPORT_UASM
> >         select PREEMPT_NOTIFIERS
> >         select KVM_GENERIC_DIRTYLOG_READ_PROTECT
> > +       select HAVE_KVM_EVENTFD
> >         select HAVE_KVM_VCPU_ASYNC_IOCTL
> >         select KVM_MMIO
> >         select MMU_NOTIFIER
> > diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> > index 01affc1..0a3cef6 100644
> > --- a/arch/mips/kvm/Makefile
> > +++ b/arch/mips/kvm/Makefile
> > @@ -2,7 +2,7 @@
> >  # Makefile for KVM support for MIPS
> >  #
> >
> > -common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
> > +common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o eventfd.o)
> >
> >  EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
> >
> > diff --git a/arch/mips/kvm/trap_emul.c b/arch/mips/kvm/trap_emul.c
> > index 5a11e83..f464506b 100644
> > --- a/arch/mips/kvm/trap_emul.c
> > +++ b/arch/mips/kvm/trap_emul.c
> > @@ -529,6 +529,9 @@ static int kvm_trap_emul_check_extension(struct kvm *kvm, long ext)
> >         case KVM_CAP_MIPS_TE:
> >                 r = 1;
> >                 break;
> > +       case KVM_CAP_IOEVENTFD:
> > +               r = 1;
> > +               break;
> >         default:
> >                 r = 0;
> >                 break;
> > diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> > index dde2088..17932ab 100644
> > --- a/arch/mips/kvm/vz.c
> > +++ b/arch/mips/kvm/vz.c
> > @@ -2927,6 +2927,9 @@ static int kvm_vz_check_extension(struct kvm *kvm, long ext)
> >                 r = 2;
> >                 break;
> >  #endif
> > +       case KVM_CAP_IOEVENTFD:
> > +               r = 1;
> > +               break;
> >         default:
> >                 r = 0;
> >                 break;
> > --
> > 2.7.0
> >

[PATCH V3 05/14] KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd

[Huacai Chen]

Loongson-3 can use lddir/ldpte instuctions to accelerate page table
walking, so use them to lookup gpa_mm.pgd.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/entry.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 16e1c93..fd71694 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -56,6 +56,7 @@
 #define C0_BADVADDR	8, 0
 #define C0_BADINSTR	8, 1
 #define C0_BADINSTRP	8, 2
+#define C0_PGD		9, 7
 #define C0_ENTRYHI	10, 0
 #define C0_GUESTCTL1	10, 4
 #define C0_STATUS	12, 0
@@ -307,7 +308,10 @@ static void *kvm_mips_build_enter_guest(void *addr)
 
 #ifdef CONFIG_KVM_MIPS_VZ
 	/* Save normal linux process pgd (VZ guarantees pgd_reg is set) */
-	UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
+	if (cpu_has_ldpte)
+		UASM_i_MFC0(&p, K0, C0_PWBASE);
+	else
+		UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
 	UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_pgd), K1);
 
 	/*
@@ -469,8 +473,10 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	u32 *p = addr;
 	struct uasm_label labels[2];
 	struct uasm_reloc relocs[2];
+#ifndef CONFIG_CPU_LOONGSON64
 	struct uasm_label *l = labels;
 	struct uasm_reloc *r = relocs;
+#endif
 
 	memset(labels, 0, sizeof(labels));
 	memset(relocs, 0, sizeof(relocs));
@@ -490,6 +496,16 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	 */
 	preempt_disable();
 
+#ifdef CONFIG_CPU_LOONGSON64
+	UASM_i_MFC0(&p, K1, C0_PGD);
+	uasm_i_lddir(&p, K0, K1, 3);  /* global page dir */
+#ifndef __PAGETABLE_PMD_FOLDED
+	uasm_i_lddir(&p, K1, K0, 1);  /* middle page dir */
+#endif
+	uasm_i_ldpte(&p, K1, 0);      /* even */
+	uasm_i_ldpte(&p, K1, 1);      /* odd */
+	uasm_i_tlbwr(&p);
+#else
 	/*
 	 * Now for the actual refill bit. A lot of this can be common with the
 	 * Linux TLB refill handler, however we don't need to handle so many
@@ -512,6 +528,7 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	build_get_ptep(&p, K0, K1);
 	build_update_entries(&p, K0, K1);
 	build_tlb_write_entry(&p, &l, &r, tlb_random);
+#endif
 
 	preempt_enable();
 
-- 
2.7.0

[PATCH V3 05/14] KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd

[Huacai Chen]

Loongson-3 can use lddir/ldpte instuctions to accelerate page table
walking, so use them to lookup gpa_mm.pgd.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/entry.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 16e1c93..fd71694 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -56,6 +56,7 @@
 #define C0_BADVADDR	8, 0
 #define C0_BADINSTR	8, 1
 #define C0_BADINSTRP	8, 2
+#define C0_PGD		9, 7
 #define C0_ENTRYHI	10, 0
 #define C0_GUESTCTL1	10, 4
 #define C0_STATUS	12, 0
@@ -307,7 +308,10 @@ static void *kvm_mips_build_enter_guest(void *addr)
 
 #ifdef CONFIG_KVM_MIPS_VZ
 	/* Save normal linux process pgd (VZ guarantees pgd_reg is set) */
-	UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
+	if (cpu_has_ldpte)
+		UASM_i_MFC0(&p, K0, C0_PWBASE);
+	else
+		UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
 	UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_pgd), K1);
 
 	/*
@@ -469,8 +473,10 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	u32 *p = addr;
 	struct uasm_label labels[2];
 	struct uasm_reloc relocs[2];
+#ifndef CONFIG_CPU_LOONGSON64
 	struct uasm_label *l = labels;
 	struct uasm_reloc *r = relocs;
+#endif
 
 	memset(labels, 0, sizeof(labels));
 	memset(relocs, 0, sizeof(relocs));
@@ -490,6 +496,16 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	 */
 	preempt_disable();
 
+#ifdef CONFIG_CPU_LOONGSON64
+	UASM_i_MFC0(&p, K1, C0_PGD);
+	uasm_i_lddir(&p, K0, K1, 3);  /* global page dir */
+#ifndef __PAGETABLE_PMD_FOLDED
+	uasm_i_lddir(&p, K1, K0, 1);  /* middle page dir */
+#endif
+	uasm_i_ldpte(&p, K1, 0);      /* even */
+	uasm_i_ldpte(&p, K1, 1);      /* odd */
+	uasm_i_tlbwr(&p);
+#else
 	/*
 	 * Now for the actual refill bit. A lot of this can be common with the
 	 * Linux TLB refill handler, however we don't need to handle so many
@@ -512,6 +528,7 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
 	build_get_ptep(&p, K0, K1);
 	build_update_entries(&p, K0, K1);
 	build_tlb_write_entry(&p, &l, &r, tlb_random);
+#endif
 
 	preempt_enable();
 
-- 
2.7.0

Re: [PATCH V3 05/14] KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:11 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3 can use lddir/ldpte instuctions to accelerate page table
> walking, so use them to lookup gpa_mm.pgd.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/entry.c | 19 ++++++++++++++++++-
>  1 file changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
> index 16e1c93..fd71694 100644
> --- a/arch/mips/kvm/entry.c
> +++ b/arch/mips/kvm/entry.c
> @@ -56,6 +56,7 @@
>  #define C0_BADVADDR    8, 0
>  #define C0_BADINSTR    8, 1
>  #define C0_BADINSTRP   8, 2
> +#define C0_PGD         9, 7
>  #define C0_ENTRYHI     10, 0
>  #define C0_GUESTCTL1   10, 4
>  #define C0_STATUS      12, 0
> @@ -307,7 +308,10 @@ static void *kvm_mips_build_enter_guest(void *addr)
>
>  #ifdef CONFIG_KVM_MIPS_VZ
>         /* Save normal linux process pgd (VZ guarantees pgd_reg is set) */
> -       UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
> +       if (cpu_has_ldpte)
> +               UASM_i_MFC0(&p, K0, C0_PWBASE);
> +       else
> +               UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
>         UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_pgd), K1);
>
>         /*
> @@ -469,8 +473,10 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
>         u32 *p = addr;
>         struct uasm_label labels[2];
>         struct uasm_reloc relocs[2];
> +#ifndef CONFIG_CPU_LOONGSON64
>         struct uasm_label *l = labels;
>         struct uasm_reloc *r = relocs;
> +#endif
>
>         memset(labels, 0, sizeof(labels));
>         memset(relocs, 0, sizeof(relocs));
> @@ -490,6 +496,16 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
>          */
>         preempt_disable();
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +       UASM_i_MFC0(&p, K1, C0_PGD);
> +       uasm_i_lddir(&p, K0, K1, 3);  /* global page dir */
> +#ifndef __PAGETABLE_PMD_FOLDED
> +       uasm_i_lddir(&p, K1, K0, 1);  /* middle page dir */
> +#endif
> +       uasm_i_ldpte(&p, K1, 0);      /* even */
> +       uasm_i_ldpte(&p, K1, 1);      /* odd */
> +       uasm_i_tlbwr(&p);
> +#else
>         /*
>          * Now for the actual refill bit. A lot of this can be common with the
>          * Linux TLB refill handler, however we don't need to handle so many
> @@ -512,6 +528,7 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
>         build_get_ptep(&p, K0, K1);
>         build_update_entries(&p, K0, K1);
>         build_tlb_write_entry(&p, &l, &r, tlb_random);
> +#endif
>
>         preempt_enable();
>
> --
> 2.7.0
>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

Re: [PATCH V3 05/14] KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:11 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3 can use lddir/ldpte instuctions to accelerate page table
> walking, so use them to lookup gpa_mm.pgd.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/entry.c | 19 ++++++++++++++++++-
>  1 file changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
> index 16e1c93..fd71694 100644
> --- a/arch/mips/kvm/entry.c
> +++ b/arch/mips/kvm/entry.c
> @@ -56,6 +56,7 @@
>  #define C0_BADVADDR    8, 0
>  #define C0_BADINSTR    8, 1
>  #define C0_BADINSTRP   8, 2
> +#define C0_PGD         9, 7
>  #define C0_ENTRYHI     10, 0
>  #define C0_GUESTCTL1   10, 4
>  #define C0_STATUS      12, 0
> @@ -307,7 +308,10 @@ static void *kvm_mips_build_enter_guest(void *addr)
>
>  #ifdef CONFIG_KVM_MIPS_VZ
>         /* Save normal linux process pgd (VZ guarantees pgd_reg is set) */
> -       UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
> +       if (cpu_has_ldpte)
> +               UASM_i_MFC0(&p, K0, C0_PWBASE);
> +       else
> +               UASM_i_MFC0(&p, K0, c0_kscratch(), pgd_reg);
>         UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_pgd), K1);
>
>         /*
> @@ -469,8 +473,10 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
>         u32 *p = addr;
>         struct uasm_label labels[2];
>         struct uasm_reloc relocs[2];
> +#ifndef CONFIG_CPU_LOONGSON64
>         struct uasm_label *l = labels;
>         struct uasm_reloc *r = relocs;
> +#endif
>
>         memset(labels, 0, sizeof(labels));
>         memset(relocs, 0, sizeof(relocs));
> @@ -490,6 +496,16 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
>          */
>         preempt_disable();
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +       UASM_i_MFC0(&p, K1, C0_PGD);
> +       uasm_i_lddir(&p, K0, K1, 3);  /* global page dir */
> +#ifndef __PAGETABLE_PMD_FOLDED
> +       uasm_i_lddir(&p, K1, K0, 1);  /* middle page dir */
> +#endif
> +       uasm_i_ldpte(&p, K1, 0);      /* even */
> +       uasm_i_ldpte(&p, K1, 1);      /* odd */
> +       uasm_i_tlbwr(&p);
> +#else
>         /*
>          * Now for the actual refill bit. A lot of this can be common with the
>          * Linux TLB refill handler, however we don't need to handle so many
> @@ -512,6 +528,7 @@ void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler)
>         build_get_ptep(&p, K0, K1);
>         build_update_entries(&p, K0, K1);
>         build_tlb_write_entry(&p, &l, &r, tlb_random);
> +#endif
>
>         preempt_enable();
>
> --
> 2.7.0
>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

[PATCH V3 06/14] KVM: MIPS: Introduce and use cpu_guest_has_ldpte

[Huacai Chen]

Loongson-3 has lddir/ldpte instructions and their related CP0 registers
are the same as HTW. So we introduce a cpu_guest_has_ldpte flag and use
it to indicate whether we need to save/restore HTW related CP0 registers
(PWBase, PWSize, PWField and PWCtl).

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/cpu-features.h |  3 +++
 arch/mips/kernel/cpu-probe.c         |  1 +
 arch/mips/kvm/vz.c                   | 26 +++++++++++++-------------
 3 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
index 400b123..e127495 100644
--- a/arch/mips/include/asm/cpu-features.h
+++ b/arch/mips/include/asm/cpu-features.h
@@ -659,6 +659,9 @@
 #ifndef cpu_guest_has_htw
 #define cpu_guest_has_htw	(cpu_data[0].guest.options & MIPS_CPU_HTW)
 #endif
+#ifndef cpu_guest_has_ldpte
+#define cpu_guest_has_ldpte	(cpu_data[0].guest.options & MIPS_CPU_LDPTE)
+#endif
 #ifndef cpu_guest_has_mvh
 #define cpu_guest_has_mvh	(cpu_data[0].guest.options & MIPS_CPU_MVH)
 #endif
diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
index ca2e6f1..be1b556 100644
--- a/arch/mips/kernel/cpu-probe.c
+++ b/arch/mips/kernel/cpu-probe.c
@@ -2004,6 +2004,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
 		 * register, we correct it here.
 		 */
 		c->options |= MIPS_CPU_FTLB | MIPS_CPU_TLBINV | MIPS_CPU_LDPTE;
+		c->guest.options |= MIPS_CPU_LDPTE;
 		c->writecombine = _CACHE_UNCACHED_ACCELERATED;
 		c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
 			MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 17932ab..422cd06 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -1706,7 +1706,7 @@ static unsigned long kvm_vz_num_regs(struct kvm_vcpu *vcpu)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_contextconfig);
 	if (cpu_guest_has_segments)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
-	if (cpu_guest_has_htw)
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_htw);
 	if (cpu_guest_has_maar && !cpu_guest_has_dyn_maar)
 		ret += 1 + ARRAY_SIZE(vcpu->arch.maar);
@@ -1755,7 +1755,7 @@ static int kvm_vz_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices)
 			return -EFAULT;
 		indices += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
 	}
-	if (cpu_guest_has_htw) {
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
 		if (copy_to_user(indices, kvm_vz_get_one_regs_htw,
 				 sizeof(kvm_vz_get_one_regs_htw)))
 			return -EFAULT;
@@ -1878,17 +1878,17 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 		*v = read_gc0_segctl2();
 		break;
 	case KVM_REG_MIPS_CP0_PWBASE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwbase();
 		break;
 	case KVM_REG_MIPS_CP0_PWFIELD:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwfield();
 		break;
 	case KVM_REG_MIPS_CP0_PWSIZE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwsize();
 		break;
@@ -1896,7 +1896,7 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 		*v = (long)read_gc0_wired();
 		break;
 	case KVM_REG_MIPS_CP0_PWCTL:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwctl();
 		break;
@@ -2101,17 +2101,17 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 		write_gc0_segctl2(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWBASE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwbase(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWFIELD:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwfield(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWSIZE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwsize(v);
 		break;
@@ -2119,7 +2119,7 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 		change_gc0_wired(MIPSR6_WIRED_WIRED, v);
 		break;
 	case KVM_REG_MIPS_CP0_PWCTL:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwctl(v);
 		break;
@@ -2580,7 +2580,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	}
 
 	/* restore HTW registers */
-	if (cpu_guest_has_htw) {
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
 		kvm_restore_gc0_pwbase(cop0);
 		kvm_restore_gc0_pwfield(cop0);
 		kvm_restore_gc0_pwsize(cop0);
@@ -2685,8 +2685,8 @@ static int kvm_vz_vcpu_put(struct kvm_vcpu *vcpu, int cpu)
 	}
 
 	/* save HTW registers if enabled in guest */
-	if (cpu_guest_has_htw &&
-	    kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW) {
+	if (cpu_guest_has_ldpte || (cpu_guest_has_htw &&
+	    kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW)) {
 		kvm_save_gc0_pwbase(cop0);
 		kvm_save_gc0_pwfield(cop0);
 		kvm_save_gc0_pwsize(cop0);
-- 
2.7.0

[PATCH V3 06/14] KVM: MIPS: Introduce and use cpu_guest_has_ldpte

[Huacai Chen]

Loongson-3 has lddir/ldpte instructions and their related CP0 registers
are the same as HTW. So we introduce a cpu_guest_has_ldpte flag and use
it to indicate whether we need to save/restore HTW related CP0 registers
(PWBase, PWSize, PWField and PWCtl).

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/cpu-features.h |  3 +++
 arch/mips/kernel/cpu-probe.c         |  1 +
 arch/mips/kvm/vz.c                   | 26 +++++++++++++-------------
 3 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
index 400b123..e127495 100644
--- a/arch/mips/include/asm/cpu-features.h
+++ b/arch/mips/include/asm/cpu-features.h
@@ -659,6 +659,9 @@
 #ifndef cpu_guest_has_htw
 #define cpu_guest_has_htw	(cpu_data[0].guest.options & MIPS_CPU_HTW)
 #endif
+#ifndef cpu_guest_has_ldpte
+#define cpu_guest_has_ldpte	(cpu_data[0].guest.options & MIPS_CPU_LDPTE)
+#endif
 #ifndef cpu_guest_has_mvh
 #define cpu_guest_has_mvh	(cpu_data[0].guest.options & MIPS_CPU_MVH)
 #endif
diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
index ca2e6f1..be1b556 100644
--- a/arch/mips/kernel/cpu-probe.c
+++ b/arch/mips/kernel/cpu-probe.c
@@ -2004,6 +2004,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
 		 * register, we correct it here.
 		 */
 		c->options |= MIPS_CPU_FTLB | MIPS_CPU_TLBINV | MIPS_CPU_LDPTE;
+		c->guest.options |= MIPS_CPU_LDPTE;
 		c->writecombine = _CACHE_UNCACHED_ACCELERATED;
 		c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
 			MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 17932ab..422cd06 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -1706,7 +1706,7 @@ static unsigned long kvm_vz_num_regs(struct kvm_vcpu *vcpu)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_contextconfig);
 	if (cpu_guest_has_segments)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
-	if (cpu_guest_has_htw)
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte)
 		ret += ARRAY_SIZE(kvm_vz_get_one_regs_htw);
 	if (cpu_guest_has_maar && !cpu_guest_has_dyn_maar)
 		ret += 1 + ARRAY_SIZE(vcpu->arch.maar);
@@ -1755,7 +1755,7 @@ static int kvm_vz_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices)
 			return -EFAULT;
 		indices += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
 	}
-	if (cpu_guest_has_htw) {
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
 		if (copy_to_user(indices, kvm_vz_get_one_regs_htw,
 				 sizeof(kvm_vz_get_one_regs_htw)))
 			return -EFAULT;
@@ -1878,17 +1878,17 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 		*v = read_gc0_segctl2();
 		break;
 	case KVM_REG_MIPS_CP0_PWBASE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwbase();
 		break;
 	case KVM_REG_MIPS_CP0_PWFIELD:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwfield();
 		break;
 	case KVM_REG_MIPS_CP0_PWSIZE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwsize();
 		break;
@@ -1896,7 +1896,7 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 		*v = (long)read_gc0_wired();
 		break;
 	case KVM_REG_MIPS_CP0_PWCTL:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		*v = read_gc0_pwctl();
 		break;
@@ -2101,17 +2101,17 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 		write_gc0_segctl2(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWBASE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwbase(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWFIELD:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwfield(v);
 		break;
 	case KVM_REG_MIPS_CP0_PWSIZE:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwsize(v);
 		break;
@@ -2119,7 +2119,7 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 		change_gc0_wired(MIPSR6_WIRED_WIRED, v);
 		break;
 	case KVM_REG_MIPS_CP0_PWCTL:
-		if (!cpu_guest_has_htw)
+		if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
 			return -EINVAL;
 		write_gc0_pwctl(v);
 		break;
@@ -2580,7 +2580,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	}
 
 	/* restore HTW registers */
-	if (cpu_guest_has_htw) {
+	if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
 		kvm_restore_gc0_pwbase(cop0);
 		kvm_restore_gc0_pwfield(cop0);
 		kvm_restore_gc0_pwsize(cop0);
@@ -2685,8 +2685,8 @@ static int kvm_vz_vcpu_put(struct kvm_vcpu *vcpu, int cpu)
 	}
 
 	/* save HTW registers if enabled in guest */
-	if (cpu_guest_has_htw &&
-	    kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW) {
+	if (cpu_guest_has_ldpte || (cpu_guest_has_htw &&
+	    kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW)) {
 		kvm_save_gc0_pwbase(cop0);
 		kvm_save_gc0_pwfield(cop0);
 		kvm_save_gc0_pwsize(cop0);
-- 
2.7.0

Re: [PATCH V3 06/14] KVM: MIPS: Introduce and use cpu_guest_has_ldpte

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:11 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3 has lddir/ldpte instructions and their related CP0 registers
> are the same as HTW. So we introduce a cpu_guest_has_ldpte flag and use
> it to indicate whether we need to save/restore HTW related CP0 registers
> (PWBase, PWSize, PWField and PWCtl).
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/include/asm/cpu-features.h |  3 +++
>  arch/mips/kernel/cpu-probe.c         |  1 +
>  arch/mips/kvm/vz.c                   | 26 +++++++++++++-------------
>  3 files changed, 17 insertions(+), 13 deletions(-)
>
> diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
> index 400b123..e127495 100644
> --- a/arch/mips/include/asm/cpu-features.h
> +++ b/arch/mips/include/asm/cpu-features.h
> @@ -659,6 +659,9 @@
>  #ifndef cpu_guest_has_htw
>  #define cpu_guest_has_htw      (cpu_data[0].guest.options & MIPS_CPU_HTW)
>  #endif
> +#ifndef cpu_guest_has_ldpte
> +#define cpu_guest_has_ldpte    (cpu_data[0].guest.options & MIPS_CPU_LDPTE)
> +#endif
>  #ifndef cpu_guest_has_mvh
>  #define cpu_guest_has_mvh      (cpu_data[0].guest.options & MIPS_CPU_MVH)
>  #endif
> diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
> index ca2e6f1..be1b556 100644
> --- a/arch/mips/kernel/cpu-probe.c
> +++ b/arch/mips/kernel/cpu-probe.c
> @@ -2004,6 +2004,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
>                  * register, we correct it here.
>                  */
>                 c->options |= MIPS_CPU_FTLB | MIPS_CPU_TLBINV | MIPS_CPU_LDPTE;
> +               c->guest.options |= MIPS_CPU_LDPTE;
>                 c->writecombine = _CACHE_UNCACHED_ACCELERATED;
>                 c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
>                         MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index 17932ab..422cd06 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -1706,7 +1706,7 @@ static unsigned long kvm_vz_num_regs(struct kvm_vcpu *vcpu)
>                 ret += ARRAY_SIZE(kvm_vz_get_one_regs_contextconfig);
>         if (cpu_guest_has_segments)
>                 ret += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
> -       if (cpu_guest_has_htw)
> +       if (cpu_guest_has_htw || cpu_guest_has_ldpte)
>                 ret += ARRAY_SIZE(kvm_vz_get_one_regs_htw);
>         if (cpu_guest_has_maar && !cpu_guest_has_dyn_maar)
>                 ret += 1 + ARRAY_SIZE(vcpu->arch.maar);
> @@ -1755,7 +1755,7 @@ static int kvm_vz_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices)
>                         return -EFAULT;
>                 indices += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
>         }
> -       if (cpu_guest_has_htw) {
> +       if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
>                 if (copy_to_user(indices, kvm_vz_get_one_regs_htw,
>                                  sizeof(kvm_vz_get_one_regs_htw)))
>                         return -EFAULT;
> @@ -1878,17 +1878,17 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
>                 *v = read_gc0_segctl2();
>                 break;
>         case KVM_REG_MIPS_CP0_PWBASE:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 *v = read_gc0_pwbase();
>                 break;
>         case KVM_REG_MIPS_CP0_PWFIELD:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 *v = read_gc0_pwfield();
>                 break;
>         case KVM_REG_MIPS_CP0_PWSIZE:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 *v = read_gc0_pwsize();
>                 break;
> @@ -1896,7 +1896,7 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
>                 *v = (long)read_gc0_wired();
>                 break;
>         case KVM_REG_MIPS_CP0_PWCTL:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 *v = read_gc0_pwctl();
>                 break;
> @@ -2101,17 +2101,17 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
>                 write_gc0_segctl2(v);
>                 break;
>         case KVM_REG_MIPS_CP0_PWBASE:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 write_gc0_pwbase(v);
>                 break;
>         case KVM_REG_MIPS_CP0_PWFIELD:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 write_gc0_pwfield(v);
>                 break;
>         case KVM_REG_MIPS_CP0_PWSIZE:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 write_gc0_pwsize(v);
>                 break;
> @@ -2119,7 +2119,7 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
>                 change_gc0_wired(MIPSR6_WIRED_WIRED, v);
>                 break;
>         case KVM_REG_MIPS_CP0_PWCTL:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 write_gc0_pwctl(v);
>                 break;
> @@ -2580,7 +2580,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>         }
>
>         /* restore HTW registers */
> -       if (cpu_guest_has_htw) {
> +       if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
>                 kvm_restore_gc0_pwbase(cop0);
>                 kvm_restore_gc0_pwfield(cop0);
>                 kvm_restore_gc0_pwsize(cop0);
> @@ -2685,8 +2685,8 @@ static int kvm_vz_vcpu_put(struct kvm_vcpu *vcpu, int cpu)
>         }
>
>         /* save HTW registers if enabled in guest */
> -       if (cpu_guest_has_htw &&
> -           kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW) {
> +       if (cpu_guest_has_ldpte || (cpu_guest_has_htw &&
> +           kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW)) {
>                 kvm_save_gc0_pwbase(cop0);
>                 kvm_save_gc0_pwfield(cop0);
>                 kvm_save_gc0_pwsize(cop0);
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

Re: [PATCH V3 06/14] KVM: MIPS: Introduce and use cpu_guest_has_ldpte

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:11 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3 has lddir/ldpte instructions and their related CP0 registers
> are the same as HTW. So we introduce a cpu_guest_has_ldpte flag and use
> it to indicate whether we need to save/restore HTW related CP0 registers
> (PWBase, PWSize, PWField and PWCtl).
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/include/asm/cpu-features.h |  3 +++
>  arch/mips/kernel/cpu-probe.c         |  1 +
>  arch/mips/kvm/vz.c                   | 26 +++++++++++++-------------
>  3 files changed, 17 insertions(+), 13 deletions(-)
>
> diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
> index 400b123..e127495 100644
> --- a/arch/mips/include/asm/cpu-features.h
> +++ b/arch/mips/include/asm/cpu-features.h
> @@ -659,6 +659,9 @@
>  #ifndef cpu_guest_has_htw
>  #define cpu_guest_has_htw      (cpu_data[0].guest.options & MIPS_CPU_HTW)
>  #endif
> +#ifndef cpu_guest_has_ldpte
> +#define cpu_guest_has_ldpte    (cpu_data[0].guest.options & MIPS_CPU_LDPTE)
> +#endif
>  #ifndef cpu_guest_has_mvh
>  #define cpu_guest_has_mvh      (cpu_data[0].guest.options & MIPS_CPU_MVH)
>  #endif
> diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
> index ca2e6f1..be1b556 100644
> --- a/arch/mips/kernel/cpu-probe.c
> +++ b/arch/mips/kernel/cpu-probe.c
> @@ -2004,6 +2004,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
>                  * register, we correct it here.
>                  */
>                 c->options |= MIPS_CPU_FTLB | MIPS_CPU_TLBINV | MIPS_CPU_LDPTE;
> +               c->guest.options |= MIPS_CPU_LDPTE;
>                 c->writecombine = _CACHE_UNCACHED_ACCELERATED;
>                 c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
>                         MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index 17932ab..422cd06 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -1706,7 +1706,7 @@ static unsigned long kvm_vz_num_regs(struct kvm_vcpu *vcpu)
>                 ret += ARRAY_SIZE(kvm_vz_get_one_regs_contextconfig);
>         if (cpu_guest_has_segments)
>                 ret += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
> -       if (cpu_guest_has_htw)
> +       if (cpu_guest_has_htw || cpu_guest_has_ldpte)
>                 ret += ARRAY_SIZE(kvm_vz_get_one_regs_htw);
>         if (cpu_guest_has_maar && !cpu_guest_has_dyn_maar)
>                 ret += 1 + ARRAY_SIZE(vcpu->arch.maar);
> @@ -1755,7 +1755,7 @@ static int kvm_vz_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices)
>                         return -EFAULT;
>                 indices += ARRAY_SIZE(kvm_vz_get_one_regs_segments);
>         }
> -       if (cpu_guest_has_htw) {
> +       if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
>                 if (copy_to_user(indices, kvm_vz_get_one_regs_htw,
>                                  sizeof(kvm_vz_get_one_regs_htw)))
>                         return -EFAULT;
> @@ -1878,17 +1878,17 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
>                 *v = read_gc0_segctl2();
>                 break;
>         case KVM_REG_MIPS_CP0_PWBASE:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 *v = read_gc0_pwbase();
>                 break;
>         case KVM_REG_MIPS_CP0_PWFIELD:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 *v = read_gc0_pwfield();
>                 break;
>         case KVM_REG_MIPS_CP0_PWSIZE:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 *v = read_gc0_pwsize();
>                 break;
> @@ -1896,7 +1896,7 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
>                 *v = (long)read_gc0_wired();
>                 break;
>         case KVM_REG_MIPS_CP0_PWCTL:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 *v = read_gc0_pwctl();
>                 break;
> @@ -2101,17 +2101,17 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
>                 write_gc0_segctl2(v);
>                 break;
>         case KVM_REG_MIPS_CP0_PWBASE:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 write_gc0_pwbase(v);
>                 break;
>         case KVM_REG_MIPS_CP0_PWFIELD:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 write_gc0_pwfield(v);
>                 break;
>         case KVM_REG_MIPS_CP0_PWSIZE:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 write_gc0_pwsize(v);
>                 break;
> @@ -2119,7 +2119,7 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
>                 change_gc0_wired(MIPSR6_WIRED_WIRED, v);
>                 break;
>         case KVM_REG_MIPS_CP0_PWCTL:
> -               if (!cpu_guest_has_htw)
> +               if (!cpu_guest_has_htw && !cpu_guest_has_ldpte)
>                         return -EINVAL;
>                 write_gc0_pwctl(v);
>                 break;
> @@ -2580,7 +2580,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>         }
>
>         /* restore HTW registers */
> -       if (cpu_guest_has_htw) {
> +       if (cpu_guest_has_htw || cpu_guest_has_ldpte) {
>                 kvm_restore_gc0_pwbase(cop0);
>                 kvm_restore_gc0_pwfield(cop0);
>                 kvm_restore_gc0_pwsize(cop0);
> @@ -2685,8 +2685,8 @@ static int kvm_vz_vcpu_put(struct kvm_vcpu *vcpu, int cpu)
>         }
>
>         /* save HTW registers if enabled in guest */
> -       if (cpu_guest_has_htw &&
> -           kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW) {
> +       if (cpu_guest_has_ldpte || (cpu_guest_has_htw &&
> +           kvm_read_sw_gc0_config3(cop0) & MIPS_CONF3_PW)) {
>                 kvm_save_gc0_pwbase(cop0);
>                 kvm_save_gc0_pwfield(cop0);
>                 kvm_save_gc0_pwsize(cop0);
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

[PATCH V3 07/14] KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3

[Huacai Chen]

KVM guest has two levels of address translation: guest tlb translates
GVA to GPA, and root tlb translates GPA to HPA. By default guest's CCA
is controlled by guest tlb, but Loongson-3 maintains all cache coherency
by hardware (including multi-core coherency and I/O DMA coherency) so it
prefers all guest mappings be cacheable mappings. Thus, we use root tlb
to control guest's CCA for Loongson-3.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/vz.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 422cd06..f9fbbc16 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2871,6 +2871,12 @@ static int kvm_vz_hardware_enable(void)
 	if (cpu_has_guestctl2)
 		clear_c0_guestctl2(0x3f << 10);
 
+#ifdef CONFIG_CPU_LOONGSON64
+	/* Control guest CCA attribute */
+	if (cpu_has_csr())
+		csr_writel(csr_readl(0xffffffec) | 0x1, 0xffffffec);
+#endif
+
 	return 0;
 }
 
-- 
2.7.0

[PATCH V3 07/14] KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3

[Huacai Chen]

KVM guest has two levels of address translation: guest tlb translates
GVA to GPA, and root tlb translates GPA to HPA. By default guest's CCA
is controlled by guest tlb, but Loongson-3 maintains all cache coherency
by hardware (including multi-core coherency and I/O DMA coherency) so it
prefers all guest mappings be cacheable mappings. Thus, we use root tlb
to control guest's CCA for Loongson-3.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/vz.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 422cd06..f9fbbc16 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2871,6 +2871,12 @@ static int kvm_vz_hardware_enable(void)
 	if (cpu_has_guestctl2)
 		clear_c0_guestctl2(0x3f << 10);
 
+#ifdef CONFIG_CPU_LOONGSON64
+	/* Control guest CCA attribute */
+	if (cpu_has_csr())
+		csr_writel(csr_readl(0xffffffec) | 0x1, 0xffffffec);
+#endif
+
 	return 0;
 }
 
-- 
2.7.0

Re: [PATCH V3 07/14] KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:12 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> KVM guest has two levels of address translation: guest tlb translates
> GVA to GPA, and root tlb translates GPA to HPA. By default guest's CCA
> is controlled by guest tlb, but Loongson-3 maintains all cache coherency
> by hardware (including multi-core coherency and I/O DMA coherency) so it
> prefers all guest mappings be cacheable mappings. Thus, we use root tlb
> to control guest's CCA for Loongson-3.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/vz.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index 422cd06..f9fbbc16 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -2871,6 +2871,12 @@ static int kvm_vz_hardware_enable(void)
>         if (cpu_has_guestctl2)
>                 clear_c0_guestctl2(0x3f << 10);
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +       /* Control guest CCA attribute */
> +       if (cpu_has_csr())
> +               csr_writel(csr_readl(0xffffffec) | 0x1, 0xffffffec);
> +#endif
> +
>         return 0;
>  }
>
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

Re: [PATCH V3 07/14] KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:12 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> KVM guest has two levels of address translation: guest tlb translates
> GVA to GPA, and root tlb translates GPA to HPA. By default guest's CCA
> is controlled by guest tlb, but Loongson-3 maintains all cache coherency
> by hardware (including multi-core coherency and I/O DMA coherency) so it
> prefers all guest mappings be cacheable mappings. Thus, we use root tlb
> to control guest's CCA for Loongson-3.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/vz.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index 422cd06..f9fbbc16 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -2871,6 +2871,12 @@ static int kvm_vz_hardware_enable(void)
>         if (cpu_has_guestctl2)
>                 clear_c0_guestctl2(0x3f << 10);
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +       /* Control guest CCA attribute */
> +       if (cpu_has_csr())
> +               csr_writel(csr_readl(0xffffffec) | 0x1, 0xffffffec);
> +#endif
> +
>         return 0;
>  }
>
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

[PATCH V3 08/14] KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3

[Huacai Chen]

Loongson-3's indexed cache operations need a node-id in the address,
but in KVM guest the node-id may be incorrect. So, let indexed cache
operations cause guest exit on Loongson-3.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/vz.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index f9fbbc16..ab320f0 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2853,8 +2853,12 @@ static int kvm_vz_hardware_enable(void)
 	write_c0_guestctl0(MIPS_GCTL0_CP0 |
 			   (MIPS_GCTL0_AT_GUEST << MIPS_GCTL0_AT_SHIFT) |
 			   MIPS_GCTL0_CG | MIPS_GCTL0_CF);
-	if (cpu_has_guestctl0ext)
-		set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+	if (cpu_has_guestctl0ext) {
+		if (current_cpu_type() != CPU_LOONGSON64)
+			set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+		else
+			clear_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+	}
 
 	if (cpu_has_guestid) {
 		write_c0_guestctl1(0);
-- 
2.7.0

[PATCH V3 08/14] KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3

[Huacai Chen]

Loongson-3's indexed cache operations need a node-id in the address,
but in KVM guest the node-id may be incorrect. So, let indexed cache
operations cause guest exit on Loongson-3.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/vz.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index f9fbbc16..ab320f0 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2853,8 +2853,12 @@ static int kvm_vz_hardware_enable(void)
 	write_c0_guestctl0(MIPS_GCTL0_CP0 |
 			   (MIPS_GCTL0_AT_GUEST << MIPS_GCTL0_AT_SHIFT) |
 			   MIPS_GCTL0_CG | MIPS_GCTL0_CF);
-	if (cpu_has_guestctl0ext)
-		set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+	if (cpu_has_guestctl0ext) {
+		if (current_cpu_type() != CPU_LOONGSON64)
+			set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+		else
+			clear_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
+	}
 
 	if (cpu_has_guestid) {
 		write_c0_guestctl1(0);
-- 
2.7.0

Re: [PATCH V3 08/14] KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:13 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3's indexed cache operations need a node-id in the address,
> but in KVM guest the node-id may be incorrect. So, let indexed cache
> operations cause guest exit on Loongson-3.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/vz.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index f9fbbc16..ab320f0 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -2853,8 +2853,12 @@ static int kvm_vz_hardware_enable(void)
>         write_c0_guestctl0(MIPS_GCTL0_CP0 |
>                            (MIPS_GCTL0_AT_GUEST << MIPS_GCTL0_AT_SHIFT) |
>                            MIPS_GCTL0_CG | MIPS_GCTL0_CF);
> -       if (cpu_has_guestctl0ext)
> -               set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
> +       if (cpu_has_guestctl0ext) {
> +               if (current_cpu_type() != CPU_LOONGSON64)
> +                       set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
> +               else
> +                       clear_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
> +       }
>
>         if (cpu_has_guestid) {
>                 write_c0_guestctl1(0);
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

Re: [PATCH V3 08/14] KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:13 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> Loongson-3's indexed cache operations need a node-id in the address,
> but in KVM guest the node-id may be incorrect. So, let indexed cache
> operations cause guest exit on Loongson-3.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/vz.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index f9fbbc16..ab320f0 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -2853,8 +2853,12 @@ static int kvm_vz_hardware_enable(void)
>         write_c0_guestctl0(MIPS_GCTL0_CP0 |
>                            (MIPS_GCTL0_AT_GUEST << MIPS_GCTL0_AT_SHIFT) |
>                            MIPS_GCTL0_CG | MIPS_GCTL0_CF);
> -       if (cpu_has_guestctl0ext)
> -               set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
> +       if (cpu_has_guestctl0ext) {
> +               if (current_cpu_type() != CPU_LOONGSON64)
> +                       set_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
> +               else
> +                       clear_c0_guestctl0ext(MIPS_GCTL0EXT_CGI);
> +       }
>
>         if (cpu_has_guestid) {
>                 write_c0_guestctl1(0);
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

[PATCH V3 09/14] KVM: MIPS: Add more types of virtual interrupts

[Huacai Chen]

In current implementation, MIPS KVM uses IP2, IP3, IP4 and IP7 for
external interrupt, two kinds of IPIs and timer interrupt respectively,
but Loongson-3 based machines prefer to use IP2, IP3, IP6 and IP7 for
two kinds of external interrupts, IPI and timer interrupt. So we define
two priority-irq mapping tables: kvm_loongson3_priority_to_irq[] for
Loongson-3, and kvm_default_priority_to_irq[] for others. The virtual
interrupt infrastructure is updated to deliver all types of interrupts
from IP2, IP3, IP4, IP6 and IP7.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/interrupt.c | 93 +++++++----------------------------------------
 arch/mips/kvm/interrupt.h | 14 ++++---
 arch/mips/kvm/mips.c      | 40 ++++++++++++++++++--
 arch/mips/kvm/vz.c        | 53 ++++-----------------------
 4 files changed, 67 insertions(+), 133 deletions(-)

diff --git a/arch/mips/kvm/interrupt.c b/arch/mips/kvm/interrupt.c
index 7257e8b6..d28c2c9c 100644
--- a/arch/mips/kvm/interrupt.c
+++ b/arch/mips/kvm/interrupt.c
@@ -61,27 +61,8 @@ void kvm_mips_queue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * the EXC code will be set when we are actually
 	 * delivering the interrupt:
 	 */
-	switch (intr) {
-	case 2:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
-		/* Queue up an INT exception for the core */
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case 3:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case 4:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_set_c0_guest_cause(vcpu->arch.cop0, 1 << (intr + 8));
+	kvm_mips_queue_irq(vcpu, kvm_irq_to_priority(intr));
 }
 
 void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
@@ -89,26 +70,8 @@ void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
 {
 	int intr = (int)irq->irq;
 
-	switch (intr) {
-	case -2:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case -3:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case -4:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_clear_c0_guest_cause(vcpu->arch.cop0, 1 << (-intr + 8));
+	kvm_mips_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
 }
 
 /* Deliver the interrupt of the corresponding priority, if possible. */
@@ -116,50 +79,20 @@ int kvm_mips_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 			    u32 cause)
 {
 	int allowed = 0;
-	u32 exccode;
+	u32 exccode, ie;
 
 	struct kvm_vcpu_arch *arch = &vcpu->arch;
 	struct mips_coproc *cop0 = vcpu->arch.cop0;
 
-	switch (priority) {
-	case MIPS_EXC_INT_TIMER:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ5)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IO:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ0)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IPI_1:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ1)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IPI_2:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ2)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
+	if (priority == MIPS_EXC_MAX)
+		return 0;
 
-	default:
-		break;
+	ie = 1 << (kvm_priority_to_irq[priority] + 8);
+	if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
+	    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
+	    && (kvm_read_c0_guest_status(cop0) & ie)) {
+		allowed = 1;
+		exccode = EXCCODE_INT;
 	}
 
 	/* Are we allowed to deliver the interrupt ??? */
diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
index 3bf0a49..c3e878c 100644
--- a/arch/mips/kvm/interrupt.h
+++ b/arch/mips/kvm/interrupt.h
@@ -21,11 +21,12 @@
 #define MIPS_EXC_NMI                5
 #define MIPS_EXC_MCHK               6
 #define MIPS_EXC_INT_TIMER          7
-#define MIPS_EXC_INT_IO             8
-#define MIPS_EXC_EXECUTE            9
-#define MIPS_EXC_INT_IPI_1          10
-#define MIPS_EXC_INT_IPI_2          11
-#define MIPS_EXC_MAX                12
+#define MIPS_EXC_INT_IO_1           8
+#define MIPS_EXC_INT_IO_2           9
+#define MIPS_EXC_EXECUTE            10
+#define MIPS_EXC_INT_IPI_1          11
+#define MIPS_EXC_INT_IPI_2          12
+#define MIPS_EXC_MAX                13
 /* XXXSL More to follow */
 
 #define C_TI        (_ULCAST_(1) << 30)
@@ -38,6 +39,9 @@
 #define KVM_MIPS_IRQ_CLEAR_ALL_AT_ONCE   (0)
 #endif
 
+extern u32 *kvm_priority_to_irq;
+u32 kvm_irq_to_priority(u32 irq);
+
 void kvm_mips_queue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
 void kvm_mips_dequeue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
 int kvm_mips_pending_timer(struct kvm_vcpu *vcpu);
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 8f05dd0..5ca122c 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -489,7 +489,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
 	int intr = (int)irq->irq;
 	struct kvm_vcpu *dvcpu = NULL;
 
-	if (intr == 3 || intr == -3 || intr == 4 || intr == -4)
+	if (intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_1] ||
+	    intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_2] ||
+	    intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_1]) ||
+	    intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_2]))
 		kvm_debug("%s: CPU: %d, INTR: %d\n", __func__, irq->cpu,
 			  (int)intr);
 
@@ -498,10 +501,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
 	else
 		dvcpu = vcpu->kvm->vcpus[irq->cpu];
 
-	if (intr == 2 || intr == 3 || intr == 4) {
+	if (intr == 2 || intr == 3 || intr == 4 || intr == 6) {
 		kvm_mips_callbacks->queue_io_int(dvcpu, irq);
 
-	} else if (intr == -2 || intr == -3 || intr == -4) {
+	} else if (intr == -2 || intr == -3 || intr == -4 || intr == -6) {
 		kvm_mips_callbacks->dequeue_io_int(dvcpu, irq);
 	} else {
 		kvm_err("%s: invalid interrupt ioctl (%d:%d)\n", __func__,
@@ -1620,6 +1623,34 @@ static struct notifier_block kvm_mips_csr_die_notifier = {
 	.notifier_call = kvm_mips_csr_die_notify,
 };
 
+static u32 kvm_default_priority_to_irq[MIPS_EXC_MAX] = {
+	[MIPS_EXC_INT_TIMER] = C_IRQ5,
+	[MIPS_EXC_INT_IO_1]  = C_IRQ0,
+	[MIPS_EXC_INT_IPI_1] = C_IRQ1,
+	[MIPS_EXC_INT_IPI_2] = C_IRQ2,
+};
+
+static u32 kvm_loongson3_priority_to_irq[MIPS_EXC_MAX] = {
+	[MIPS_EXC_INT_TIMER] = C_IRQ5,
+	[MIPS_EXC_INT_IO_1]  = C_IRQ0,
+	[MIPS_EXC_INT_IO_2]  = C_IRQ1,
+	[MIPS_EXC_INT_IPI_1] = C_IRQ4,
+};
+
+u32 *kvm_priority_to_irq = kvm_default_priority_to_irq;
+
+u32 kvm_irq_to_priority(u32 irq)
+{
+	int i;
+
+	for (i = MIPS_EXC_INT_TIMER; i < MIPS_EXC_MAX; i++) {
+		if (kvm_priority_to_irq[i] == (1 << (irq + 8)))
+			return i;
+	}
+
+	return MIPS_EXC_MAX;
+}
+
 static int __init kvm_mips_init(void)
 {
 	int ret;
@@ -1638,6 +1669,9 @@ static int __init kvm_mips_init(void)
 	if (ret)
 		return ret;
 
+	if (boot_cpu_type() == CPU_LOONGSON64)
+		kvm_priority_to_irq = kvm_loongson3_priority_to_irq;
+
 	register_die_notifier(&kvm_mips_csr_die_notifier);
 
 	return 0;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index ab320f0..63d5b35 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -225,23 +225,7 @@ static void kvm_vz_queue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * interrupts are asynchronous to vcpu execution therefore defer guest
 	 * cp0 accesses
 	 */
-	switch (intr) {
-	case 2:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case 3:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case 4:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_vz_queue_irq(vcpu, kvm_irq_to_priority(intr));
 }
 
 static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
@@ -253,44 +237,22 @@ static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * interrupts are asynchronous to vcpu execution therefore defer guest
 	 * cp0 accesses
 	 */
-	switch (intr) {
-	case -2:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case -3:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case -4:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_vz_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
 }
 
-static u32 kvm_vz_priority_to_irq[MIPS_EXC_MAX] = {
-	[MIPS_EXC_INT_TIMER] = C_IRQ5,
-	[MIPS_EXC_INT_IO]    = C_IRQ0,
-	[MIPS_EXC_INT_IPI_1] = C_IRQ1,
-	[MIPS_EXC_INT_IPI_2] = C_IRQ2,
-};
-
 static int kvm_vz_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 				 u32 cause)
 {
 	u32 irq = (priority < MIPS_EXC_MAX) ?
-		kvm_vz_priority_to_irq[priority] : 0;
+		kvm_priority_to_irq[priority] : 0;
 
 	switch (priority) {
 	case MIPS_EXC_INT_TIMER:
 		set_gc0_cause(C_TI);
 		break;
 
-	case MIPS_EXC_INT_IO:
+	case MIPS_EXC_INT_IO_1:
+	case MIPS_EXC_INT_IO_2:
 	case MIPS_EXC_INT_IPI_1:
 	case MIPS_EXC_INT_IPI_2:
 		if (cpu_has_guestctl2)
@@ -311,7 +273,7 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 			       u32 cause)
 {
 	u32 irq = (priority < MIPS_EXC_MAX) ?
-		kvm_vz_priority_to_irq[priority] : 0;
+		kvm_priority_to_irq[priority] : 0;
 
 	switch (priority) {
 	case MIPS_EXC_INT_TIMER:
@@ -329,7 +291,8 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 		}
 		break;
 
-	case MIPS_EXC_INT_IO:
+	case MIPS_EXC_INT_IO_1:
+	case MIPS_EXC_INT_IO_2:
 	case MIPS_EXC_INT_IPI_1:
 	case MIPS_EXC_INT_IPI_2:
 		/* Clear GuestCtl2.VIP irq if not using Hardware Clear */
-- 
2.7.0

[PATCH V3 09/14] KVM: MIPS: Add more types of virtual interrupts

[Huacai Chen]

In current implementation, MIPS KVM uses IP2, IP3, IP4 and IP7 for
external interrupt, two kinds of IPIs and timer interrupt respectively,
but Loongson-3 based machines prefer to use IP2, IP3, IP6 and IP7 for
two kinds of external interrupts, IPI and timer interrupt. So we define
two priority-irq mapping tables: kvm_loongson3_priority_to_irq[] for
Loongson-3, and kvm_default_priority_to_irq[] for others. The virtual
interrupt infrastructure is updated to deliver all types of interrupts
from IP2, IP3, IP4, IP6 and IP7.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/interrupt.c | 93 +++++++----------------------------------------
 arch/mips/kvm/interrupt.h | 14 ++++---
 arch/mips/kvm/mips.c      | 40 ++++++++++++++++++--
 arch/mips/kvm/vz.c        | 53 ++++-----------------------
 4 files changed, 67 insertions(+), 133 deletions(-)

diff --git a/arch/mips/kvm/interrupt.c b/arch/mips/kvm/interrupt.c
index 7257e8b6..d28c2c9c 100644
--- a/arch/mips/kvm/interrupt.c
+++ b/arch/mips/kvm/interrupt.c
@@ -61,27 +61,8 @@ void kvm_mips_queue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * the EXC code will be set when we are actually
 	 * delivering the interrupt:
 	 */
-	switch (intr) {
-	case 2:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
-		/* Queue up an INT exception for the core */
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case 3:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case 4:
-		kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
-		kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_set_c0_guest_cause(vcpu->arch.cop0, 1 << (intr + 8));
+	kvm_mips_queue_irq(vcpu, kvm_irq_to_priority(intr));
 }
 
 void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
@@ -89,26 +70,8 @@ void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
 {
 	int intr = (int)irq->irq;
 
-	switch (intr) {
-	case -2:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case -3:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case -4:
-		kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
-		kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_clear_c0_guest_cause(vcpu->arch.cop0, 1 << (-intr + 8));
+	kvm_mips_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
 }
 
 /* Deliver the interrupt of the corresponding priority, if possible. */
@@ -116,50 +79,20 @@ int kvm_mips_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 			    u32 cause)
 {
 	int allowed = 0;
-	u32 exccode;
+	u32 exccode, ie;
 
 	struct kvm_vcpu_arch *arch = &vcpu->arch;
 	struct mips_coproc *cop0 = vcpu->arch.cop0;
 
-	switch (priority) {
-	case MIPS_EXC_INT_TIMER:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ5)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IO:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ0)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IPI_1:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ1)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
-
-	case MIPS_EXC_INT_IPI_2:
-		if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
-		    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
-		    && (kvm_read_c0_guest_status(cop0) & IE_IRQ2)) {
-			allowed = 1;
-			exccode = EXCCODE_INT;
-		}
-		break;
+	if (priority == MIPS_EXC_MAX)
+		return 0;
 
-	default:
-		break;
+	ie = 1 << (kvm_priority_to_irq[priority] + 8);
+	if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
+	    && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
+	    && (kvm_read_c0_guest_status(cop0) & ie)) {
+		allowed = 1;
+		exccode = EXCCODE_INT;
 	}
 
 	/* Are we allowed to deliver the interrupt ??? */
diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
index 3bf0a49..c3e878c 100644
--- a/arch/mips/kvm/interrupt.h
+++ b/arch/mips/kvm/interrupt.h
@@ -21,11 +21,12 @@
 #define MIPS_EXC_NMI                5
 #define MIPS_EXC_MCHK               6
 #define MIPS_EXC_INT_TIMER          7
-#define MIPS_EXC_INT_IO             8
-#define MIPS_EXC_EXECUTE            9
-#define MIPS_EXC_INT_IPI_1          10
-#define MIPS_EXC_INT_IPI_2          11
-#define MIPS_EXC_MAX                12
+#define MIPS_EXC_INT_IO_1           8
+#define MIPS_EXC_INT_IO_2           9
+#define MIPS_EXC_EXECUTE            10
+#define MIPS_EXC_INT_IPI_1          11
+#define MIPS_EXC_INT_IPI_2          12
+#define MIPS_EXC_MAX                13
 /* XXXSL More to follow */
 
 #define C_TI        (_ULCAST_(1) << 30)
@@ -38,6 +39,9 @@
 #define KVM_MIPS_IRQ_CLEAR_ALL_AT_ONCE   (0)
 #endif
 
+extern u32 *kvm_priority_to_irq;
+u32 kvm_irq_to_priority(u32 irq);
+
 void kvm_mips_queue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
 void kvm_mips_dequeue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
 int kvm_mips_pending_timer(struct kvm_vcpu *vcpu);
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 8f05dd0..5ca122c 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -489,7 +489,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
 	int intr = (int)irq->irq;
 	struct kvm_vcpu *dvcpu = NULL;
 
-	if (intr == 3 || intr == -3 || intr == 4 || intr == -4)
+	if (intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_1] ||
+	    intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_2] ||
+	    intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_1]) ||
+	    intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_2]))
 		kvm_debug("%s: CPU: %d, INTR: %d\n", __func__, irq->cpu,
 			  (int)intr);
 
@@ -498,10 +501,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
 	else
 		dvcpu = vcpu->kvm->vcpus[irq->cpu];
 
-	if (intr == 2 || intr == 3 || intr == 4) {
+	if (intr == 2 || intr == 3 || intr == 4 || intr == 6) {
 		kvm_mips_callbacks->queue_io_int(dvcpu, irq);
 
-	} else if (intr == -2 || intr == -3 || intr == -4) {
+	} else if (intr == -2 || intr == -3 || intr == -4 || intr == -6) {
 		kvm_mips_callbacks->dequeue_io_int(dvcpu, irq);
 	} else {
 		kvm_err("%s: invalid interrupt ioctl (%d:%d)\n", __func__,
@@ -1620,6 +1623,34 @@ static struct notifier_block kvm_mips_csr_die_notifier = {
 	.notifier_call = kvm_mips_csr_die_notify,
 };
 
+static u32 kvm_default_priority_to_irq[MIPS_EXC_MAX] = {
+	[MIPS_EXC_INT_TIMER] = C_IRQ5,
+	[MIPS_EXC_INT_IO_1]  = C_IRQ0,
+	[MIPS_EXC_INT_IPI_1] = C_IRQ1,
+	[MIPS_EXC_INT_IPI_2] = C_IRQ2,
+};
+
+static u32 kvm_loongson3_priority_to_irq[MIPS_EXC_MAX] = {
+	[MIPS_EXC_INT_TIMER] = C_IRQ5,
+	[MIPS_EXC_INT_IO_1]  = C_IRQ0,
+	[MIPS_EXC_INT_IO_2]  = C_IRQ1,
+	[MIPS_EXC_INT_IPI_1] = C_IRQ4,
+};
+
+u32 *kvm_priority_to_irq = kvm_default_priority_to_irq;
+
+u32 kvm_irq_to_priority(u32 irq)
+{
+	int i;
+
+	for (i = MIPS_EXC_INT_TIMER; i < MIPS_EXC_MAX; i++) {
+		if (kvm_priority_to_irq[i] == (1 << (irq + 8)))
+			return i;
+	}
+
+	return MIPS_EXC_MAX;
+}
+
 static int __init kvm_mips_init(void)
 {
 	int ret;
@@ -1638,6 +1669,9 @@ static int __init kvm_mips_init(void)
 	if (ret)
 		return ret;
 
+	if (boot_cpu_type() == CPU_LOONGSON64)
+		kvm_priority_to_irq = kvm_loongson3_priority_to_irq;
+
 	register_die_notifier(&kvm_mips_csr_die_notifier);
 
 	return 0;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index ab320f0..63d5b35 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -225,23 +225,7 @@ static void kvm_vz_queue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * interrupts are asynchronous to vcpu execution therefore defer guest
 	 * cp0 accesses
 	 */
-	switch (intr) {
-	case 2:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case 3:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case 4:
-		kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_vz_queue_irq(vcpu, kvm_irq_to_priority(intr));
 }
 
 static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
@@ -253,44 +237,22 @@ static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
 	 * interrupts are asynchronous to vcpu execution therefore defer guest
 	 * cp0 accesses
 	 */
-	switch (intr) {
-	case -2:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
-		break;
-
-	case -3:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
-		break;
-
-	case -4:
-		kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
-		break;
-
-	default:
-		break;
-	}
-
+	kvm_vz_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
 }
 
-static u32 kvm_vz_priority_to_irq[MIPS_EXC_MAX] = {
-	[MIPS_EXC_INT_TIMER] = C_IRQ5,
-	[MIPS_EXC_INT_IO]    = C_IRQ0,
-	[MIPS_EXC_INT_IPI_1] = C_IRQ1,
-	[MIPS_EXC_INT_IPI_2] = C_IRQ2,
-};
-
 static int kvm_vz_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 				 u32 cause)
 {
 	u32 irq = (priority < MIPS_EXC_MAX) ?
-		kvm_vz_priority_to_irq[priority] : 0;
+		kvm_priority_to_irq[priority] : 0;
 
 	switch (priority) {
 	case MIPS_EXC_INT_TIMER:
 		set_gc0_cause(C_TI);
 		break;
 
-	case MIPS_EXC_INT_IO:
+	case MIPS_EXC_INT_IO_1:
+	case MIPS_EXC_INT_IO_2:
 	case MIPS_EXC_INT_IPI_1:
 	case MIPS_EXC_INT_IPI_2:
 		if (cpu_has_guestctl2)
@@ -311,7 +273,7 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 			       u32 cause)
 {
 	u32 irq = (priority < MIPS_EXC_MAX) ?
-		kvm_vz_priority_to_irq[priority] : 0;
+		kvm_priority_to_irq[priority] : 0;
 
 	switch (priority) {
 	case MIPS_EXC_INT_TIMER:
@@ -329,7 +291,8 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
 		}
 		break;
 
-	case MIPS_EXC_INT_IO:
+	case MIPS_EXC_INT_IO_1:
+	case MIPS_EXC_INT_IO_2:
 	case MIPS_EXC_INT_IPI_1:
 	case MIPS_EXC_INT_IPI_2:
 		/* Clear GuestCtl2.VIP irq if not using Hardware Clear */
-- 
2.7.0

Re: [PATCH V3 09/14] KVM: MIPS: Add more types of virtual interrupts

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:13 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> In current implementation, MIPS KVM uses IP2, IP3, IP4 and IP7 for
> external interrupt, two kinds of IPIs and timer interrupt respectively,
> but Loongson-3 based machines prefer to use IP2, IP3, IP6 and IP7 for
> two kinds of external interrupts, IPI and timer interrupt. So we define
> two priority-irq mapping tables: kvm_loongson3_priority_to_irq[] for
> Loongson-3, and kvm_default_priority_to_irq[] for others. The virtual
> interrupt infrastructure is updated to deliver all types of interrupts
> from IP2, IP3, IP4, IP6 and IP7.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/interrupt.c | 93 +++++++----------------------------------------
>  arch/mips/kvm/interrupt.h | 14 ++++---
>  arch/mips/kvm/mips.c      | 40 ++++++++++++++++++--
>  arch/mips/kvm/vz.c        | 53 ++++-----------------------
>  4 files changed, 67 insertions(+), 133 deletions(-)
>
> diff --git a/arch/mips/kvm/interrupt.c b/arch/mips/kvm/interrupt.c
> index 7257e8b6..d28c2c9c 100644
> --- a/arch/mips/kvm/interrupt.c
> +++ b/arch/mips/kvm/interrupt.c
> @@ -61,27 +61,8 @@ void kvm_mips_queue_io_int_cb(struct kvm_vcpu *vcpu,
>          * the EXC code will be set when we are actually
>          * delivering the interrupt:
>          */
> -       switch (intr) {
> -       case 2:
> -               kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
> -               /* Queue up an INT exception for the core */
> -               kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IO);
> -               break;
> -
> -       case 3:
> -               kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
> -               kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
> -               break;
> -
> -       case 4:
> -               kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
> -               kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
> -               break;
> -
> -       default:
> -               break;
> -       }
> -
> +       kvm_set_c0_guest_cause(vcpu->arch.cop0, 1 << (intr + 8));
> +       kvm_mips_queue_irq(vcpu, kvm_irq_to_priority(intr));
>  }
>
>  void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
> @@ -89,26 +70,8 @@ void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
>  {
>         int intr = (int)irq->irq;
>
> -       switch (intr) {
> -       case -2:
> -               kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
> -               kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
> -               break;
> -
> -       case -3:
> -               kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
> -               kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
> -               break;
> -
> -       case -4:
> -               kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
> -               kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
> -               break;
> -
> -       default:
> -               break;
> -       }
> -
> +       kvm_clear_c0_guest_cause(vcpu->arch.cop0, 1 << (-intr + 8));
> +       kvm_mips_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
>  }
>
>  /* Deliver the interrupt of the corresponding priority, if possible. */
> @@ -116,50 +79,20 @@ int kvm_mips_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
>                             u32 cause)
>  {
>         int allowed = 0;
> -       u32 exccode;
> +       u32 exccode, ie;
>
>         struct kvm_vcpu_arch *arch = &vcpu->arch;
>         struct mips_coproc *cop0 = vcpu->arch.cop0;
>
> -       switch (priority) {
> -       case MIPS_EXC_INT_TIMER:
> -               if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> -                   && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> -                   && (kvm_read_c0_guest_status(cop0) & IE_IRQ5)) {
> -                       allowed = 1;
> -                       exccode = EXCCODE_INT;
> -               }
> -               break;
> -
> -       case MIPS_EXC_INT_IO:
> -               if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> -                   && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> -                   && (kvm_read_c0_guest_status(cop0) & IE_IRQ0)) {
> -                       allowed = 1;
> -                       exccode = EXCCODE_INT;
> -               }
> -               break;
> -
> -       case MIPS_EXC_INT_IPI_1:
> -               if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> -                   && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> -                   && (kvm_read_c0_guest_status(cop0) & IE_IRQ1)) {
> -                       allowed = 1;
> -                       exccode = EXCCODE_INT;
> -               }
> -               break;
> -
> -       case MIPS_EXC_INT_IPI_2:
> -               if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> -                   && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> -                   && (kvm_read_c0_guest_status(cop0) & IE_IRQ2)) {
> -                       allowed = 1;
> -                       exccode = EXCCODE_INT;
> -               }
> -               break;
> +       if (priority == MIPS_EXC_MAX)
> +               return 0;
>
> -       default:
> -               break;
> +       ie = 1 << (kvm_priority_to_irq[priority] + 8);
> +       if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> +           && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> +           && (kvm_read_c0_guest_status(cop0) & ie)) {
> +               allowed = 1;
> +               exccode = EXCCODE_INT;
>         }
>
>         /* Are we allowed to deliver the interrupt ??? */
> diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
> index 3bf0a49..c3e878c 100644
> --- a/arch/mips/kvm/interrupt.h
> +++ b/arch/mips/kvm/interrupt.h
> @@ -21,11 +21,12 @@
>  #define MIPS_EXC_NMI                5
>  #define MIPS_EXC_MCHK               6
>  #define MIPS_EXC_INT_TIMER          7
> -#define MIPS_EXC_INT_IO             8
> -#define MIPS_EXC_EXECUTE            9
> -#define MIPS_EXC_INT_IPI_1          10
> -#define MIPS_EXC_INT_IPI_2          11
> -#define MIPS_EXC_MAX                12
> +#define MIPS_EXC_INT_IO_1           8
> +#define MIPS_EXC_INT_IO_2           9
> +#define MIPS_EXC_EXECUTE            10
> +#define MIPS_EXC_INT_IPI_1          11
> +#define MIPS_EXC_INT_IPI_2          12
> +#define MIPS_EXC_MAX                13
>  /* XXXSL More to follow */
>
>  #define C_TI        (_ULCAST_(1) << 30)
> @@ -38,6 +39,9 @@
>  #define KVM_MIPS_IRQ_CLEAR_ALL_AT_ONCE   (0)
>  #endif
>
> +extern u32 *kvm_priority_to_irq;
> +u32 kvm_irq_to_priority(u32 irq);
> +
>  void kvm_mips_queue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
>  void kvm_mips_dequeue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
>  int kvm_mips_pending_timer(struct kvm_vcpu *vcpu);
> diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> index 8f05dd0..5ca122c 100644
> --- a/arch/mips/kvm/mips.c
> +++ b/arch/mips/kvm/mips.c
> @@ -489,7 +489,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
>         int intr = (int)irq->irq;
>         struct kvm_vcpu *dvcpu = NULL;
>
> -       if (intr == 3 || intr == -3 || intr == 4 || intr == -4)
> +       if (intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_1] ||
> +           intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_2] ||
> +           intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_1]) ||
> +           intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_2]))
>                 kvm_debug("%s: CPU: %d, INTR: %d\n", __func__, irq->cpu,
>                           (int)intr);
>
> @@ -498,10 +501,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
>         else
>                 dvcpu = vcpu->kvm->vcpus[irq->cpu];
>
> -       if (intr == 2 || intr == 3 || intr == 4) {
> +       if (intr == 2 || intr == 3 || intr == 4 || intr == 6) {
>                 kvm_mips_callbacks->queue_io_int(dvcpu, irq);
>
> -       } else if (intr == -2 || intr == -3 || intr == -4) {
> +       } else if (intr == -2 || intr == -3 || intr == -4 || intr == -6) {
>                 kvm_mips_callbacks->dequeue_io_int(dvcpu, irq);
>         } else {
>                 kvm_err("%s: invalid interrupt ioctl (%d:%d)\n", __func__,
> @@ -1620,6 +1623,34 @@ static struct notifier_block kvm_mips_csr_die_notifier = {
>         .notifier_call = kvm_mips_csr_die_notify,
>  };
>
> +static u32 kvm_default_priority_to_irq[MIPS_EXC_MAX] = {
> +       [MIPS_EXC_INT_TIMER] = C_IRQ5,
> +       [MIPS_EXC_INT_IO_1]  = C_IRQ0,
> +       [MIPS_EXC_INT_IPI_1] = C_IRQ1,
> +       [MIPS_EXC_INT_IPI_2] = C_IRQ2,
> +};
> +
> +static u32 kvm_loongson3_priority_to_irq[MIPS_EXC_MAX] = {
> +       [MIPS_EXC_INT_TIMER] = C_IRQ5,
> +       [MIPS_EXC_INT_IO_1]  = C_IRQ0,
> +       [MIPS_EXC_INT_IO_2]  = C_IRQ1,
> +       [MIPS_EXC_INT_IPI_1] = C_IRQ4,
> +};
> +
> +u32 *kvm_priority_to_irq = kvm_default_priority_to_irq;
> +
> +u32 kvm_irq_to_priority(u32 irq)
> +{
> +       int i;
> +
> +       for (i = MIPS_EXC_INT_TIMER; i < MIPS_EXC_MAX; i++) {
> +               if (kvm_priority_to_irq[i] == (1 << (irq + 8)))
> +                       return i;
> +       }
> +
> +       return MIPS_EXC_MAX;
> +}
> +
>  static int __init kvm_mips_init(void)
>  {
>         int ret;
> @@ -1638,6 +1669,9 @@ static int __init kvm_mips_init(void)
>         if (ret)
>                 return ret;
>
> +       if (boot_cpu_type() == CPU_LOONGSON64)
> +               kvm_priority_to_irq = kvm_loongson3_priority_to_irq;
> +
>         register_die_notifier(&kvm_mips_csr_die_notifier);
>
>         return 0;
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index ab320f0..63d5b35 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -225,23 +225,7 @@ static void kvm_vz_queue_io_int_cb(struct kvm_vcpu *vcpu,
>          * interrupts are asynchronous to vcpu execution therefore defer guest
>          * cp0 accesses
>          */
> -       switch (intr) {
> -       case 2:
> -               kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IO);
> -               break;
> -
> -       case 3:
> -               kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
> -               break;
> -
> -       case 4:
> -               kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
> -               break;
> -
> -       default:
> -               break;
> -       }
> -
> +       kvm_vz_queue_irq(vcpu, kvm_irq_to_priority(intr));
>  }
>
>  static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
> @@ -253,44 +237,22 @@ static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
>          * interrupts are asynchronous to vcpu execution therefore defer guest
>          * cp0 accesses
>          */
> -       switch (intr) {
> -       case -2:
> -               kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
> -               break;
> -
> -       case -3:
> -               kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
> -               break;
> -
> -       case -4:
> -               kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
> -               break;
> -
> -       default:
> -               break;
> -       }
> -
> +       kvm_vz_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
>  }
>
> -static u32 kvm_vz_priority_to_irq[MIPS_EXC_MAX] = {
> -       [MIPS_EXC_INT_TIMER] = C_IRQ5,
> -       [MIPS_EXC_INT_IO]    = C_IRQ0,
> -       [MIPS_EXC_INT_IPI_1] = C_IRQ1,
> -       [MIPS_EXC_INT_IPI_2] = C_IRQ2,
> -};
> -
>  static int kvm_vz_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
>                                  u32 cause)
>  {
>         u32 irq = (priority < MIPS_EXC_MAX) ?
> -               kvm_vz_priority_to_irq[priority] : 0;
> +               kvm_priority_to_irq[priority] : 0;
>
>         switch (priority) {
>         case MIPS_EXC_INT_TIMER:
>                 set_gc0_cause(C_TI);
>                 break;
>
> -       case MIPS_EXC_INT_IO:
> +       case MIPS_EXC_INT_IO_1:
> +       case MIPS_EXC_INT_IO_2:
>         case MIPS_EXC_INT_IPI_1:
>         case MIPS_EXC_INT_IPI_2:
>                 if (cpu_has_guestctl2)
> @@ -311,7 +273,7 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
>                                u32 cause)
>  {
>         u32 irq = (priority < MIPS_EXC_MAX) ?
> -               kvm_vz_priority_to_irq[priority] : 0;
> +               kvm_priority_to_irq[priority] : 0;
>
>         switch (priority) {
>         case MIPS_EXC_INT_TIMER:
> @@ -329,7 +291,8 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
>                 }
>                 break;
>
> -       case MIPS_EXC_INT_IO:
> +       case MIPS_EXC_INT_IO_1:
> +       case MIPS_EXC_INT_IO_2:
>         case MIPS_EXC_INT_IPI_1:
>         case MIPS_EXC_INT_IPI_2:
>                 /* Clear GuestCtl2.VIP irq if not using Hardware Clear */
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

Re: [PATCH V3 09/14] KVM: MIPS: Add more types of virtual interrupts

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:13 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> In current implementation, MIPS KVM uses IP2, IP3, IP4 and IP7 for
> external interrupt, two kinds of IPIs and timer interrupt respectively,
> but Loongson-3 based machines prefer to use IP2, IP3, IP6 and IP7 for
> two kinds of external interrupts, IPI and timer interrupt. So we define
> two priority-irq mapping tables: kvm_loongson3_priority_to_irq[] for
> Loongson-3, and kvm_default_priority_to_irq[] for others. The virtual
> interrupt infrastructure is updated to deliver all types of interrupts
> from IP2, IP3, IP4, IP6 and IP7.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/interrupt.c | 93 +++++++----------------------------------------
>  arch/mips/kvm/interrupt.h | 14 ++++---
>  arch/mips/kvm/mips.c      | 40 ++++++++++++++++++--
>  arch/mips/kvm/vz.c        | 53 ++++-----------------------
>  4 files changed, 67 insertions(+), 133 deletions(-)
>
> diff --git a/arch/mips/kvm/interrupt.c b/arch/mips/kvm/interrupt.c
> index 7257e8b6..d28c2c9c 100644
> --- a/arch/mips/kvm/interrupt.c
> +++ b/arch/mips/kvm/interrupt.c
> @@ -61,27 +61,8 @@ void kvm_mips_queue_io_int_cb(struct kvm_vcpu *vcpu,
>          * the EXC code will be set when we are actually
>          * delivering the interrupt:
>          */
> -       switch (intr) {
> -       case 2:
> -               kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
> -               /* Queue up an INT exception for the core */
> -               kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IO);
> -               break;
> -
> -       case 3:
> -               kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
> -               kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
> -               break;
> -
> -       case 4:
> -               kvm_set_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
> -               kvm_mips_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
> -               break;
> -
> -       default:
> -               break;
> -       }
> -
> +       kvm_set_c0_guest_cause(vcpu->arch.cop0, 1 << (intr + 8));
> +       kvm_mips_queue_irq(vcpu, kvm_irq_to_priority(intr));
>  }
>
>  void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
> @@ -89,26 +70,8 @@ void kvm_mips_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
>  {
>         int intr = (int)irq->irq;
>
> -       switch (intr) {
> -       case -2:
> -               kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ0));
> -               kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
> -               break;
> -
> -       case -3:
> -               kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ1));
> -               kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
> -               break;
> -
> -       case -4:
> -               kvm_clear_c0_guest_cause(vcpu->arch.cop0, (C_IRQ2));
> -               kvm_mips_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
> -               break;
> -
> -       default:
> -               break;
> -       }
> -
> +       kvm_clear_c0_guest_cause(vcpu->arch.cop0, 1 << (-intr + 8));
> +       kvm_mips_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
>  }
>
>  /* Deliver the interrupt of the corresponding priority, if possible. */
> @@ -116,50 +79,20 @@ int kvm_mips_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
>                             u32 cause)
>  {
>         int allowed = 0;
> -       u32 exccode;
> +       u32 exccode, ie;
>
>         struct kvm_vcpu_arch *arch = &vcpu->arch;
>         struct mips_coproc *cop0 = vcpu->arch.cop0;
>
> -       switch (priority) {
> -       case MIPS_EXC_INT_TIMER:
> -               if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> -                   && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> -                   && (kvm_read_c0_guest_status(cop0) & IE_IRQ5)) {
> -                       allowed = 1;
> -                       exccode = EXCCODE_INT;
> -               }
> -               break;
> -
> -       case MIPS_EXC_INT_IO:
> -               if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> -                   && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> -                   && (kvm_read_c0_guest_status(cop0) & IE_IRQ0)) {
> -                       allowed = 1;
> -                       exccode = EXCCODE_INT;
> -               }
> -               break;
> -
> -       case MIPS_EXC_INT_IPI_1:
> -               if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> -                   && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> -                   && (kvm_read_c0_guest_status(cop0) & IE_IRQ1)) {
> -                       allowed = 1;
> -                       exccode = EXCCODE_INT;
> -               }
> -               break;
> -
> -       case MIPS_EXC_INT_IPI_2:
> -               if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> -                   && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> -                   && (kvm_read_c0_guest_status(cop0) & IE_IRQ2)) {
> -                       allowed = 1;
> -                       exccode = EXCCODE_INT;
> -               }
> -               break;
> +       if (priority == MIPS_EXC_MAX)
> +               return 0;
>
> -       default:
> -               break;
> +       ie = 1 << (kvm_priority_to_irq[priority] + 8);
> +       if ((kvm_read_c0_guest_status(cop0) & ST0_IE)
> +           && (!(kvm_read_c0_guest_status(cop0) & (ST0_EXL | ST0_ERL)))
> +           && (kvm_read_c0_guest_status(cop0) & ie)) {
> +               allowed = 1;
> +               exccode = EXCCODE_INT;
>         }
>
>         /* Are we allowed to deliver the interrupt ??? */
> diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
> index 3bf0a49..c3e878c 100644
> --- a/arch/mips/kvm/interrupt.h
> +++ b/arch/mips/kvm/interrupt.h
> @@ -21,11 +21,12 @@
>  #define MIPS_EXC_NMI                5
>  #define MIPS_EXC_MCHK               6
>  #define MIPS_EXC_INT_TIMER          7
> -#define MIPS_EXC_INT_IO             8
> -#define MIPS_EXC_EXECUTE            9
> -#define MIPS_EXC_INT_IPI_1          10
> -#define MIPS_EXC_INT_IPI_2          11
> -#define MIPS_EXC_MAX                12
> +#define MIPS_EXC_INT_IO_1           8
> +#define MIPS_EXC_INT_IO_2           9
> +#define MIPS_EXC_EXECUTE            10
> +#define MIPS_EXC_INT_IPI_1          11
> +#define MIPS_EXC_INT_IPI_2          12
> +#define MIPS_EXC_MAX                13
>  /* XXXSL More to follow */
>
>  #define C_TI        (_ULCAST_(1) << 30)
> @@ -38,6 +39,9 @@
>  #define KVM_MIPS_IRQ_CLEAR_ALL_AT_ONCE   (0)
>  #endif
>
> +extern u32 *kvm_priority_to_irq;
> +u32 kvm_irq_to_priority(u32 irq);
> +
>  void kvm_mips_queue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
>  void kvm_mips_dequeue_irq(struct kvm_vcpu *vcpu, unsigned int priority);
>  int kvm_mips_pending_timer(struct kvm_vcpu *vcpu);
> diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> index 8f05dd0..5ca122c 100644
> --- a/arch/mips/kvm/mips.c
> +++ b/arch/mips/kvm/mips.c
> @@ -489,7 +489,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
>         int intr = (int)irq->irq;
>         struct kvm_vcpu *dvcpu = NULL;
>
> -       if (intr == 3 || intr == -3 || intr == 4 || intr == -4)
> +       if (intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_1] ||
> +           intr == kvm_priority_to_irq[MIPS_EXC_INT_IPI_2] ||
> +           intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_1]) ||
> +           intr == (-kvm_priority_to_irq[MIPS_EXC_INT_IPI_2]))
>                 kvm_debug("%s: CPU: %d, INTR: %d\n", __func__, irq->cpu,
>                           (int)intr);
>
> @@ -498,10 +501,10 @@ int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
>         else
>                 dvcpu = vcpu->kvm->vcpus[irq->cpu];
>
> -       if (intr == 2 || intr == 3 || intr == 4) {
> +       if (intr == 2 || intr == 3 || intr == 4 || intr == 6) {
>                 kvm_mips_callbacks->queue_io_int(dvcpu, irq);
>
> -       } else if (intr == -2 || intr == -3 || intr == -4) {
> +       } else if (intr == -2 || intr == -3 || intr == -4 || intr == -6) {
>                 kvm_mips_callbacks->dequeue_io_int(dvcpu, irq);
>         } else {
>                 kvm_err("%s: invalid interrupt ioctl (%d:%d)\n", __func__,
> @@ -1620,6 +1623,34 @@ static struct notifier_block kvm_mips_csr_die_notifier = {
>         .notifier_call = kvm_mips_csr_die_notify,
>  };
>
> +static u32 kvm_default_priority_to_irq[MIPS_EXC_MAX] = {
> +       [MIPS_EXC_INT_TIMER] = C_IRQ5,
> +       [MIPS_EXC_INT_IO_1]  = C_IRQ0,
> +       [MIPS_EXC_INT_IPI_1] = C_IRQ1,
> +       [MIPS_EXC_INT_IPI_2] = C_IRQ2,
> +};
> +
> +static u32 kvm_loongson3_priority_to_irq[MIPS_EXC_MAX] = {
> +       [MIPS_EXC_INT_TIMER] = C_IRQ5,
> +       [MIPS_EXC_INT_IO_1]  = C_IRQ0,
> +       [MIPS_EXC_INT_IO_2]  = C_IRQ1,
> +       [MIPS_EXC_INT_IPI_1] = C_IRQ4,
> +};
> +
> +u32 *kvm_priority_to_irq = kvm_default_priority_to_irq;
> +
> +u32 kvm_irq_to_priority(u32 irq)
> +{
> +       int i;
> +
> +       for (i = MIPS_EXC_INT_TIMER; i < MIPS_EXC_MAX; i++) {
> +               if (kvm_priority_to_irq[i] == (1 << (irq + 8)))
> +                       return i;
> +       }
> +
> +       return MIPS_EXC_MAX;
> +}
> +
>  static int __init kvm_mips_init(void)
>  {
>         int ret;
> @@ -1638,6 +1669,9 @@ static int __init kvm_mips_init(void)
>         if (ret)
>                 return ret;
>
> +       if (boot_cpu_type() == CPU_LOONGSON64)
> +               kvm_priority_to_irq = kvm_loongson3_priority_to_irq;
> +
>         register_die_notifier(&kvm_mips_csr_die_notifier);
>
>         return 0;
> diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
> index ab320f0..63d5b35 100644
> --- a/arch/mips/kvm/vz.c
> +++ b/arch/mips/kvm/vz.c
> @@ -225,23 +225,7 @@ static void kvm_vz_queue_io_int_cb(struct kvm_vcpu *vcpu,
>          * interrupts are asynchronous to vcpu execution therefore defer guest
>          * cp0 accesses
>          */
> -       switch (intr) {
> -       case 2:
> -               kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IO);
> -               break;
> -
> -       case 3:
> -               kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_1);
> -               break;
> -
> -       case 4:
> -               kvm_vz_queue_irq(vcpu, MIPS_EXC_INT_IPI_2);
> -               break;
> -
> -       default:
> -               break;
> -       }
> -
> +       kvm_vz_queue_irq(vcpu, kvm_irq_to_priority(intr));
>  }
>
>  static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
> @@ -253,44 +237,22 @@ static void kvm_vz_dequeue_io_int_cb(struct kvm_vcpu *vcpu,
>          * interrupts are asynchronous to vcpu execution therefore defer guest
>          * cp0 accesses
>          */
> -       switch (intr) {
> -       case -2:
> -               kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IO);
> -               break;
> -
> -       case -3:
> -               kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_1);
> -               break;
> -
> -       case -4:
> -               kvm_vz_dequeue_irq(vcpu, MIPS_EXC_INT_IPI_2);
> -               break;
> -
> -       default:
> -               break;
> -       }
> -
> +       kvm_vz_dequeue_irq(vcpu, kvm_irq_to_priority(-intr));
>  }
>
> -static u32 kvm_vz_priority_to_irq[MIPS_EXC_MAX] = {
> -       [MIPS_EXC_INT_TIMER] = C_IRQ5,
> -       [MIPS_EXC_INT_IO]    = C_IRQ0,
> -       [MIPS_EXC_INT_IPI_1] = C_IRQ1,
> -       [MIPS_EXC_INT_IPI_2] = C_IRQ2,
> -};
> -
>  static int kvm_vz_irq_deliver_cb(struct kvm_vcpu *vcpu, unsigned int priority,
>                                  u32 cause)
>  {
>         u32 irq = (priority < MIPS_EXC_MAX) ?
> -               kvm_vz_priority_to_irq[priority] : 0;
> +               kvm_priority_to_irq[priority] : 0;
>
>         switch (priority) {
>         case MIPS_EXC_INT_TIMER:
>                 set_gc0_cause(C_TI);
>                 break;
>
> -       case MIPS_EXC_INT_IO:
> +       case MIPS_EXC_INT_IO_1:
> +       case MIPS_EXC_INT_IO_2:
>         case MIPS_EXC_INT_IPI_1:
>         case MIPS_EXC_INT_IPI_2:
>                 if (cpu_has_guestctl2)
> @@ -311,7 +273,7 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
>                                u32 cause)
>  {
>         u32 irq = (priority < MIPS_EXC_MAX) ?
> -               kvm_vz_priority_to_irq[priority] : 0;
> +               kvm_priority_to_irq[priority] : 0;
>
>         switch (priority) {
>         case MIPS_EXC_INT_TIMER:
> @@ -329,7 +291,8 @@ static int kvm_vz_irq_clear_cb(struct kvm_vcpu *vcpu, unsigned int priority,
>                 }
>                 break;
>
> -       case MIPS_EXC_INT_IO:
> +       case MIPS_EXC_INT_IO_1:
> +       case MIPS_EXC_INT_IO_2:
>         case MIPS_EXC_INT_IPI_1:
>         case MIPS_EXC_INT_IPI_2:
>                 /* Clear GuestCtl2.VIP irq if not using Hardware Clear */
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

[PATCH V3 10/14] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support

[Huacai Chen]

This patch add Loongson-3 Virtual IPI interrupt support in the kernel,
because emulate it in QEMU is too expensive for performance.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h |  32 ++++++
 arch/mips/kvm/Makefile           |   3 +
 arch/mips/kvm/emulate.c          |  23 ++++-
 arch/mips/kvm/loongson_ipi.c     | 214 +++++++++++++++++++++++++++++++++++++++
 arch/mips/kvm/mips.c             |   6 ++
 5 files changed, 277 insertions(+), 1 deletion(-)
 create mode 100644 arch/mips/kvm/loongson_ipi.c

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index a7758c0..f165902 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -23,6 +23,8 @@
 #include <asm/inst.h>
 #include <asm/mipsregs.h>
 
+#include <kvm/iodev.h>
+
 /* MIPS KVM register ids */
 #define MIPS_CP0_32(_R, _S)					\
 	(KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
@@ -181,11 +183,39 @@ struct kvm_vcpu_stat {
 struct kvm_arch_memory_slot {
 };
 
+#ifdef CONFIG_CPU_LOONGSON64
+struct ipi_state {
+	uint32_t status;
+	uint32_t en;
+	uint32_t set;
+	uint32_t clear;
+	uint64_t buf[4];
+};
+
+struct loongson_kvm_ipi;
+
+struct ipi_io_device {
+	int node_id;
+	struct loongson_kvm_ipi *ipi;
+	struct kvm_io_device device;
+};
+
+struct loongson_kvm_ipi {
+	spinlock_t lock;
+	struct kvm *kvm;
+	struct ipi_state ipistate[16];
+	struct ipi_io_device dev_ipi[4];
+};
+#endif
+
 struct kvm_arch {
 	/* Guest physical mm */
 	struct mm_struct gpa_mm;
 	/* Mask of CPUs needing GPA ASID flush */
 	cpumask_t asid_flush_mask;
+#ifdef CONFIG_CPU_LOONGSON64
+	struct loongson_kvm_ipi ipi;
+#endif
 };
 
 #define N_MIPS_COPROC_REGS	32
@@ -1133,6 +1163,8 @@ extern int kvm_mips_trans_mtc0(union mips_instruction inst, u32 *opc,
 /* Misc */
 extern void kvm_mips_dump_stats(struct kvm_vcpu *vcpu);
 extern unsigned long kvm_mips_get_ramsize(struct kvm *kvm);
+extern int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
+			     struct kvm_mips_interrupt *irq);
 
 static inline void kvm_arch_hardware_unsetup(void) {}
 static inline void kvm_arch_sync_events(struct kvm *kvm) {}
diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
index 0a3cef6..506c4ac 100644
--- a/arch/mips/kvm/Makefile
+++ b/arch/mips/kvm/Makefile
@@ -13,6 +13,9 @@ kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
 	    fpu.o
 kvm-objs += hypcall.o
 kvm-objs += mmu.o
+ifdef CONFIG_CPU_LOONGSON64
+kvm-objs += loongson_ipi.o
+endif
 
 ifdef CONFIG_KVM_MIPS_VZ
 kvm-objs		+= vz.o
diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
index 754094b..3946499 100644
--- a/arch/mips/kvm/emulate.c
+++ b/arch/mips/kvm/emulate.c
@@ -1600,6 +1600,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 					     struct kvm_run *run,
 					     struct kvm_vcpu *vcpu)
 {
+	int r;
 	enum emulation_result er;
 	u32 rt;
 	void *data = run->mmio.data;
@@ -1666,9 +1667,18 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 		goto out_fail;
 	}
 
-	run->mmio.is_write = 1;
 	vcpu->mmio_needed = 1;
+	run->mmio.is_write = 1;
 	vcpu->mmio_is_write = 1;
+
+	r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS,
+			run->mmio.phys_addr, run->mmio.len, data);
+
+	if (!r) {
+		vcpu->mmio_needed = 0;
+		return EMULATE_DONE;
+	}
+
 	return EMULATE_DO_MMIO;
 
 out_fail:
@@ -1681,6 +1691,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 					    u32 cause, struct kvm_run *run,
 					    struct kvm_vcpu *vcpu)
 {
+	int r;
 	enum emulation_result er;
 	unsigned long curr_pc;
 	u32 op, rt;
@@ -1745,6 +1756,16 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 
 	run->mmio.is_write = 0;
 	vcpu->mmio_is_write = 0;
+
+	r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS,
+			run->mmio.phys_addr, run->mmio.len, run->mmio.data);
+
+	if (!r) {
+		kvm_mips_complete_mmio_load(vcpu, run);
+		vcpu->mmio_needed = 0;
+		return EMULATE_DONE;
+	}
+
 	return EMULATE_DO_MMIO;
 }
 
diff --git a/arch/mips/kvm/loongson_ipi.c b/arch/mips/kvm/loongson_ipi.c
new file mode 100644
index 00000000..3681fc8
--- /dev/null
+++ b/arch/mips/kvm/loongson_ipi.c
@@ -0,0 +1,214 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Loongson-3 Virtual IPI interrupt support.
+ *
+ * Copyright (C) 2019  Loongson Technologies, Inc.  All rights reserved.
+ *
+ * Authors: Chen Zhu <zhuchen@loongson.cn>
+ * Authors: Huacai Chen <chenhc@lemote.com>
+ */
+
+#include <linux/kvm_host.h>
+
+#define IPI_BASE            0x3ff01000ULL
+
+#define CORE0_STATUS_OFF       0x000
+#define CORE0_EN_OFF           0x004
+#define CORE0_SET_OFF          0x008
+#define CORE0_CLEAR_OFF        0x00c
+#define CORE0_BUF_20           0x020
+#define CORE0_BUF_28           0x028
+#define CORE0_BUF_30           0x030
+#define CORE0_BUF_38           0x038
+
+#define CORE1_STATUS_OFF       0x100
+#define CORE1_EN_OFF           0x104
+#define CORE1_SET_OFF          0x108
+#define CORE1_CLEAR_OFF        0x10c
+#define CORE1_BUF_20           0x120
+#define CORE1_BUF_28           0x128
+#define CORE1_BUF_30           0x130
+#define CORE1_BUF_38           0x138
+
+#define CORE2_STATUS_OFF       0x200
+#define CORE2_EN_OFF           0x204
+#define CORE2_SET_OFF          0x208
+#define CORE2_CLEAR_OFF        0x20c
+#define CORE2_BUF_20           0x220
+#define CORE2_BUF_28           0x228
+#define CORE2_BUF_30           0x230
+#define CORE2_BUF_38           0x238
+
+#define CORE3_STATUS_OFF       0x300
+#define CORE3_EN_OFF           0x304
+#define CORE3_SET_OFF          0x308
+#define CORE3_CLEAR_OFF        0x30c
+#define CORE3_BUF_20           0x320
+#define CORE3_BUF_28           0x328
+#define CORE3_BUF_30           0x330
+#define CORE3_BUF_38           0x338
+
+static int loongson_vipi_read(struct loongson_kvm_ipi *ipi,
+				gpa_t addr, int len, void *val)
+{
+	uint32_t core = (addr >> 8) & 3;
+	uint32_t node = (addr >> 44) & 3;
+	uint32_t id = core + node * 4;
+	uint64_t offset = addr & 0xff;
+	void *pbuf;
+	struct ipi_state *s = &(ipi->ipistate[id]);
+
+	BUG_ON(offset & (len - 1));
+
+	switch (offset) {
+	case CORE0_STATUS_OFF:
+		*(uint64_t *)val = s->status;
+		break;
+
+	case CORE0_EN_OFF:
+		*(uint64_t *)val = s->en;
+		break;
+
+	case CORE0_SET_OFF:
+		*(uint64_t *)val = 0;
+		break;
+
+	case CORE0_CLEAR_OFF:
+		*(uint64_t *)val = 0;
+		break;
+
+	case CORE0_BUF_20 ... CORE0_BUF_38:
+		pbuf = (void *)s->buf + (offset - 0x20);
+		if (len == 8)
+			*(uint64_t *)val = *(uint64_t *)pbuf;
+		else /* Assume len == 4 */
+			*(uint32_t *)val = *(uint32_t *)pbuf;
+		break;
+
+	default:
+		pr_notice("%s with unknown addr %llx\n", __func__, addr);
+		break;
+	}
+
+	return 0;
+}
+
+static int loongson_vipi_write(struct loongson_kvm_ipi *ipi,
+				gpa_t addr, int len, const void *val)
+{
+	uint32_t core = (addr >> 8) & 3;
+	uint32_t node = (addr >> 44) & 3;
+	uint32_t id = core + node * 4;
+	uint64_t data, offset = addr & 0xff;
+	void *pbuf;
+	struct kvm *kvm = ipi->kvm;
+	struct kvm_mips_interrupt irq;
+	struct ipi_state *s = &(ipi->ipistate[id]);
+
+	data = *(uint64_t *)val;
+	BUG_ON(offset & (len - 1));
+
+	switch (offset) {
+	case CORE0_STATUS_OFF:
+		break;
+
+	case CORE0_EN_OFF:
+		s->en = data;
+		break;
+
+	case CORE0_SET_OFF:
+		s->status |= data;
+		irq.cpu = id;
+		irq.irq = 6;
+		kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
+		break;
+
+	case CORE0_CLEAR_OFF:
+		s->status &= ~data;
+		if (!s->status) {
+			irq.cpu = id;
+			irq.irq = -6;
+			kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
+		}
+		break;
+
+	case CORE0_BUF_20 ... CORE0_BUF_38:
+		pbuf = (void *)s->buf + (offset - 0x20);
+		if (len == 8)
+			*(uint64_t *)pbuf = (uint64_t)data;
+		else /* Assume len == 4 */
+			*(uint32_t *)pbuf = (uint32_t)data;
+		break;
+
+	default:
+		pr_notice("%s with unknown addr %llx\n", __func__, addr);
+		break;
+	}
+
+	return 0;
+}
+
+static int kvm_ipi_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			gpa_t addr, int len, void *val)
+{
+	unsigned long flags;
+	struct loongson_kvm_ipi *ipi;
+	struct ipi_io_device *ipi_device;
+
+	ipi_device = container_of(dev, struct ipi_io_device, device);
+	ipi = ipi_device->ipi;
+
+	spin_lock_irqsave(&ipi->lock, flags);
+	loongson_vipi_read(ipi, addr, len, val);
+	spin_unlock_irqrestore(&ipi->lock, flags);
+
+	return 0;
+}
+
+static int kvm_ipi_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			gpa_t addr, int len, const void *val)
+{
+	unsigned long flags;
+	struct loongson_kvm_ipi *ipi;
+	struct ipi_io_device *ipi_device;
+
+	ipi_device = container_of(dev, struct ipi_io_device, device);
+	ipi = ipi_device->ipi;
+
+	spin_lock_irqsave(&ipi->lock, flags);
+	loongson_vipi_write(ipi, addr, len, val);
+	spin_unlock_irqrestore(&ipi->lock, flags);
+
+	return 0;
+}
+
+static const struct kvm_io_device_ops kvm_ipi_ops = {
+	.read     = kvm_ipi_read,
+	.write    = kvm_ipi_write,
+};
+
+void kvm_init_loongson_ipi(struct kvm *kvm)
+{
+	int i;
+	unsigned long addr;
+	struct loongson_kvm_ipi *s;
+	struct kvm_io_device *device;
+
+	s = &kvm->arch.ipi;
+	s->kvm = kvm;
+	spin_lock_init(&s->lock);
+
+	/*
+	 * Initialize IPI device
+	 */
+	for (i = 0; i < 4; i++) {
+		device = &s->dev_ipi[i].device;
+		kvm_iodevice_init(device, &kvm_ipi_ops);
+		addr = (((unsigned long)i) << 44) + IPI_BASE;
+		mutex_lock(&kvm->slots_lock);
+		kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, addr, 0x400, device);
+		mutex_unlock(&kvm->slots_lock);
+		s->dev_ipi[i].ipi = s;
+		s->dev_ipi[i].node_id = i;
+	}
+}
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 5ca122c..ed989ef 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -128,6 +128,8 @@ int kvm_arch_check_processor_compat(void *opaque)
 	return 0;
 }
 
+extern void kvm_init_loongson_ipi(struct kvm *kvm);
+
 int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 {
 	switch (type) {
@@ -147,6 +149,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	if (!kvm->arch.gpa_mm.pgd)
 		return -ENOMEM;
 
+#ifdef CONFIG_CPU_LOONGSON64
+	kvm_init_loongson_ipi(kvm);
+#endif
+
 	return 0;
 }
 
-- 
2.7.0

[PATCH V3 10/14] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support

[Huacai Chen]

This patch add Loongson-3 Virtual IPI interrupt support in the kernel,
because emulate it in QEMU is too expensive for performance.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h |  32 ++++++
 arch/mips/kvm/Makefile           |   3 +
 arch/mips/kvm/emulate.c          |  23 ++++-
 arch/mips/kvm/loongson_ipi.c     | 214 +++++++++++++++++++++++++++++++++++++++
 arch/mips/kvm/mips.c             |   6 ++
 5 files changed, 277 insertions(+), 1 deletion(-)
 create mode 100644 arch/mips/kvm/loongson_ipi.c

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index a7758c0..f165902 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -23,6 +23,8 @@
 #include <asm/inst.h>
 #include <asm/mipsregs.h>
 
+#include <kvm/iodev.h>
+
 /* MIPS KVM register ids */
 #define MIPS_CP0_32(_R, _S)					\
 	(KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
@@ -181,11 +183,39 @@ struct kvm_vcpu_stat {
 struct kvm_arch_memory_slot {
 };
 
+#ifdef CONFIG_CPU_LOONGSON64
+struct ipi_state {
+	uint32_t status;
+	uint32_t en;
+	uint32_t set;
+	uint32_t clear;
+	uint64_t buf[4];
+};
+
+struct loongson_kvm_ipi;
+
+struct ipi_io_device {
+	int node_id;
+	struct loongson_kvm_ipi *ipi;
+	struct kvm_io_device device;
+};
+
+struct loongson_kvm_ipi {
+	spinlock_t lock;
+	struct kvm *kvm;
+	struct ipi_state ipistate[16];
+	struct ipi_io_device dev_ipi[4];
+};
+#endif
+
 struct kvm_arch {
 	/* Guest physical mm */
 	struct mm_struct gpa_mm;
 	/* Mask of CPUs needing GPA ASID flush */
 	cpumask_t asid_flush_mask;
+#ifdef CONFIG_CPU_LOONGSON64
+	struct loongson_kvm_ipi ipi;
+#endif
 };
 
 #define N_MIPS_COPROC_REGS	32
@@ -1133,6 +1163,8 @@ extern int kvm_mips_trans_mtc0(union mips_instruction inst, u32 *opc,
 /* Misc */
 extern void kvm_mips_dump_stats(struct kvm_vcpu *vcpu);
 extern unsigned long kvm_mips_get_ramsize(struct kvm *kvm);
+extern int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
+			     struct kvm_mips_interrupt *irq);
 
 static inline void kvm_arch_hardware_unsetup(void) {}
 static inline void kvm_arch_sync_events(struct kvm *kvm) {}
diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
index 0a3cef6..506c4ac 100644
--- a/arch/mips/kvm/Makefile
+++ b/arch/mips/kvm/Makefile
@@ -13,6 +13,9 @@ kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
 	    fpu.o
 kvm-objs += hypcall.o
 kvm-objs += mmu.o
+ifdef CONFIG_CPU_LOONGSON64
+kvm-objs += loongson_ipi.o
+endif
 
 ifdef CONFIG_KVM_MIPS_VZ
 kvm-objs		+= vz.o
diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
index 754094b..3946499 100644
--- a/arch/mips/kvm/emulate.c
+++ b/arch/mips/kvm/emulate.c
@@ -1600,6 +1600,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 					     struct kvm_run *run,
 					     struct kvm_vcpu *vcpu)
 {
+	int r;
 	enum emulation_result er;
 	u32 rt;
 	void *data = run->mmio.data;
@@ -1666,9 +1667,18 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 		goto out_fail;
 	}
 
-	run->mmio.is_write = 1;
 	vcpu->mmio_needed = 1;
+	run->mmio.is_write = 1;
 	vcpu->mmio_is_write = 1;
+
+	r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS,
+			run->mmio.phys_addr, run->mmio.len, data);
+
+	if (!r) {
+		vcpu->mmio_needed = 0;
+		return EMULATE_DONE;
+	}
+
 	return EMULATE_DO_MMIO;
 
 out_fail:
@@ -1681,6 +1691,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 					    u32 cause, struct kvm_run *run,
 					    struct kvm_vcpu *vcpu)
 {
+	int r;
 	enum emulation_result er;
 	unsigned long curr_pc;
 	u32 op, rt;
@@ -1745,6 +1756,16 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 
 	run->mmio.is_write = 0;
 	vcpu->mmio_is_write = 0;
+
+	r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS,
+			run->mmio.phys_addr, run->mmio.len, run->mmio.data);
+
+	if (!r) {
+		kvm_mips_complete_mmio_load(vcpu, run);
+		vcpu->mmio_needed = 0;
+		return EMULATE_DONE;
+	}
+
 	return EMULATE_DO_MMIO;
 }
 
diff --git a/arch/mips/kvm/loongson_ipi.c b/arch/mips/kvm/loongson_ipi.c
new file mode 100644
index 00000000..3681fc8
--- /dev/null
+++ b/arch/mips/kvm/loongson_ipi.c
@@ -0,0 +1,214 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Loongson-3 Virtual IPI interrupt support.
+ *
+ * Copyright (C) 2019  Loongson Technologies, Inc.  All rights reserved.
+ *
+ * Authors: Chen Zhu <zhuchen@loongson.cn>
+ * Authors: Huacai Chen <chenhc@lemote.com>
+ */
+
+#include <linux/kvm_host.h>
+
+#define IPI_BASE            0x3ff01000ULL
+
+#define CORE0_STATUS_OFF       0x000
+#define CORE0_EN_OFF           0x004
+#define CORE0_SET_OFF          0x008
+#define CORE0_CLEAR_OFF        0x00c
+#define CORE0_BUF_20           0x020
+#define CORE0_BUF_28           0x028
+#define CORE0_BUF_30           0x030
+#define CORE0_BUF_38           0x038
+
+#define CORE1_STATUS_OFF       0x100
+#define CORE1_EN_OFF           0x104
+#define CORE1_SET_OFF          0x108
+#define CORE1_CLEAR_OFF        0x10c
+#define CORE1_BUF_20           0x120
+#define CORE1_BUF_28           0x128
+#define CORE1_BUF_30           0x130
+#define CORE1_BUF_38           0x138
+
+#define CORE2_STATUS_OFF       0x200
+#define CORE2_EN_OFF           0x204
+#define CORE2_SET_OFF          0x208
+#define CORE2_CLEAR_OFF        0x20c
+#define CORE2_BUF_20           0x220
+#define CORE2_BUF_28           0x228
+#define CORE2_BUF_30           0x230
+#define CORE2_BUF_38           0x238
+
+#define CORE3_STATUS_OFF       0x300
+#define CORE3_EN_OFF           0x304
+#define CORE3_SET_OFF          0x308
+#define CORE3_CLEAR_OFF        0x30c
+#define CORE3_BUF_20           0x320
+#define CORE3_BUF_28           0x328
+#define CORE3_BUF_30           0x330
+#define CORE3_BUF_38           0x338
+
+static int loongson_vipi_read(struct loongson_kvm_ipi *ipi,
+				gpa_t addr, int len, void *val)
+{
+	uint32_t core = (addr >> 8) & 3;
+	uint32_t node = (addr >> 44) & 3;
+	uint32_t id = core + node * 4;
+	uint64_t offset = addr & 0xff;
+	void *pbuf;
+	struct ipi_state *s = &(ipi->ipistate[id]);
+
+	BUG_ON(offset & (len - 1));
+
+	switch (offset) {
+	case CORE0_STATUS_OFF:
+		*(uint64_t *)val = s->status;
+		break;
+
+	case CORE0_EN_OFF:
+		*(uint64_t *)val = s->en;
+		break;
+
+	case CORE0_SET_OFF:
+		*(uint64_t *)val = 0;
+		break;
+
+	case CORE0_CLEAR_OFF:
+		*(uint64_t *)val = 0;
+		break;
+
+	case CORE0_BUF_20 ... CORE0_BUF_38:
+		pbuf = (void *)s->buf + (offset - 0x20);
+		if (len == 8)
+			*(uint64_t *)val = *(uint64_t *)pbuf;
+		else /* Assume len == 4 */
+			*(uint32_t *)val = *(uint32_t *)pbuf;
+		break;
+
+	default:
+		pr_notice("%s with unknown addr %llx\n", __func__, addr);
+		break;
+	}
+
+	return 0;
+}
+
+static int loongson_vipi_write(struct loongson_kvm_ipi *ipi,
+				gpa_t addr, int len, const void *val)
+{
+	uint32_t core = (addr >> 8) & 3;
+	uint32_t node = (addr >> 44) & 3;
+	uint32_t id = core + node * 4;
+	uint64_t data, offset = addr & 0xff;
+	void *pbuf;
+	struct kvm *kvm = ipi->kvm;
+	struct kvm_mips_interrupt irq;
+	struct ipi_state *s = &(ipi->ipistate[id]);
+
+	data = *(uint64_t *)val;
+	BUG_ON(offset & (len - 1));
+
+	switch (offset) {
+	case CORE0_STATUS_OFF:
+		break;
+
+	case CORE0_EN_OFF:
+		s->en = data;
+		break;
+
+	case CORE0_SET_OFF:
+		s->status |= data;
+		irq.cpu = id;
+		irq.irq = 6;
+		kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
+		break;
+
+	case CORE0_CLEAR_OFF:
+		s->status &= ~data;
+		if (!s->status) {
+			irq.cpu = id;
+			irq.irq = -6;
+			kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
+		}
+		break;
+
+	case CORE0_BUF_20 ... CORE0_BUF_38:
+		pbuf = (void *)s->buf + (offset - 0x20);
+		if (len == 8)
+			*(uint64_t *)pbuf = (uint64_t)data;
+		else /* Assume len == 4 */
+			*(uint32_t *)pbuf = (uint32_t)data;
+		break;
+
+	default:
+		pr_notice("%s with unknown addr %llx\n", __func__, addr);
+		break;
+	}
+
+	return 0;
+}
+
+static int kvm_ipi_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			gpa_t addr, int len, void *val)
+{
+	unsigned long flags;
+	struct loongson_kvm_ipi *ipi;
+	struct ipi_io_device *ipi_device;
+
+	ipi_device = container_of(dev, struct ipi_io_device, device);
+	ipi = ipi_device->ipi;
+
+	spin_lock_irqsave(&ipi->lock, flags);
+	loongson_vipi_read(ipi, addr, len, val);
+	spin_unlock_irqrestore(&ipi->lock, flags);
+
+	return 0;
+}
+
+static int kvm_ipi_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			gpa_t addr, int len, const void *val)
+{
+	unsigned long flags;
+	struct loongson_kvm_ipi *ipi;
+	struct ipi_io_device *ipi_device;
+
+	ipi_device = container_of(dev, struct ipi_io_device, device);
+	ipi = ipi_device->ipi;
+
+	spin_lock_irqsave(&ipi->lock, flags);
+	loongson_vipi_write(ipi, addr, len, val);
+	spin_unlock_irqrestore(&ipi->lock, flags);
+
+	return 0;
+}
+
+static const struct kvm_io_device_ops kvm_ipi_ops = {
+	.read     = kvm_ipi_read,
+	.write    = kvm_ipi_write,
+};
+
+void kvm_init_loongson_ipi(struct kvm *kvm)
+{
+	int i;
+	unsigned long addr;
+	struct loongson_kvm_ipi *s;
+	struct kvm_io_device *device;
+
+	s = &kvm->arch.ipi;
+	s->kvm = kvm;
+	spin_lock_init(&s->lock);
+
+	/*
+	 * Initialize IPI device
+	 */
+	for (i = 0; i < 4; i++) {
+		device = &s->dev_ipi[i].device;
+		kvm_iodevice_init(device, &kvm_ipi_ops);
+		addr = (((unsigned long)i) << 44) + IPI_BASE;
+		mutex_lock(&kvm->slots_lock);
+		kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, addr, 0x400, device);
+		mutex_unlock(&kvm->slots_lock);
+		s->dev_ipi[i].ipi = s;
+		s->dev_ipi[i].node_id = i;
+	}
+}
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 5ca122c..ed989ef 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -128,6 +128,8 @@ int kvm_arch_check_processor_compat(void *opaque)
 	return 0;
 }
 
+extern void kvm_init_loongson_ipi(struct kvm *kvm);
+
 int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 {
 	switch (type) {
@@ -147,6 +149,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	if (!kvm->arch.gpa_mm.pgd)
 		return -ENOMEM;
 
+#ifdef CONFIG_CPU_LOONGSON64
+	kvm_init_loongson_ipi(kvm);
+#endif
+
 	return 0;
 }
 
-- 
2.7.0

Re: [PATCH V3 10/14] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:14 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> This patch add Loongson-3 Virtual IPI interrupt support in the kernel,
> because emulate it in QEMU is too expensive for performance.
>

Huacei, hi!

While, in principle, I support this patch, could you please, just for
the sake of everybody being on the same page, describe the
hyphothetical alternative implementation in QEMU, and explain why it
would be more expensive than the solution proposed in this patch.

No need to go into details, I would like to see just the general idea
you had in your mind.

Yours,
Aleksandar


> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/include/asm/kvm_host.h |  32 ++++++
>  arch/mips/kvm/Makefile           |   3 +
>  arch/mips/kvm/emulate.c          |  23 ++++-
>  arch/mips/kvm/loongson_ipi.c     | 214 +++++++++++++++++++++++++++++++++++++++
>  arch/mips/kvm/mips.c             |   6 ++
>  5 files changed, 277 insertions(+), 1 deletion(-)
>  create mode 100644 arch/mips/kvm/loongson_ipi.c
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index a7758c0..f165902 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -23,6 +23,8 @@
>  #include <asm/inst.h>
>  #include <asm/mipsregs.h>
>
> +#include <kvm/iodev.h>
> +
>  /* MIPS KVM register ids */
>  #define MIPS_CP0_32(_R, _S)                                    \
>         (KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
> @@ -181,11 +183,39 @@ struct kvm_vcpu_stat {
>  struct kvm_arch_memory_slot {
>  };
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +struct ipi_state {
> +       uint32_t status;
> +       uint32_t en;
> +       uint32_t set;
> +       uint32_t clear;
> +       uint64_t buf[4];
> +};
> +
> +struct loongson_kvm_ipi;
> +
> +struct ipi_io_device {
> +       int node_id;
> +       struct loongson_kvm_ipi *ipi;
> +       struct kvm_io_device device;
> +};
> +
> +struct loongson_kvm_ipi {
> +       spinlock_t lock;
> +       struct kvm *kvm;
> +       struct ipi_state ipistate[16];
> +       struct ipi_io_device dev_ipi[4];
> +};
> +#endif
> +
>  struct kvm_arch {
>         /* Guest physical mm */
>         struct mm_struct gpa_mm;
>         /* Mask of CPUs needing GPA ASID flush */
>         cpumask_t asid_flush_mask;
> +#ifdef CONFIG_CPU_LOONGSON64
> +       struct loongson_kvm_ipi ipi;
> +#endif
>  };
>
>  #define N_MIPS_COPROC_REGS     32
> @@ -1133,6 +1163,8 @@ extern int kvm_mips_trans_mtc0(union mips_instruction inst, u32 *opc,
>  /* Misc */
>  extern void kvm_mips_dump_stats(struct kvm_vcpu *vcpu);
>  extern unsigned long kvm_mips_get_ramsize(struct kvm *kvm);
> +extern int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
> +                            struct kvm_mips_interrupt *irq);
>
>  static inline void kvm_arch_hardware_unsetup(void) {}
>  static inline void kvm_arch_sync_events(struct kvm *kvm) {}
> diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> index 0a3cef6..506c4ac 100644
> --- a/arch/mips/kvm/Makefile
> +++ b/arch/mips/kvm/Makefile
> @@ -13,6 +13,9 @@ kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
>             fpu.o
>  kvm-objs += hypcall.o
>  kvm-objs += mmu.o
> +ifdef CONFIG_CPU_LOONGSON64
> +kvm-objs += loongson_ipi.o
> +endif
>
>  ifdef CONFIG_KVM_MIPS_VZ
>  kvm-objs               += vz.o
> diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
> index 754094b..3946499 100644
> --- a/arch/mips/kvm/emulate.c
> +++ b/arch/mips/kvm/emulate.c
> @@ -1600,6 +1600,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
>                                              struct kvm_run *run,
>                                              struct kvm_vcpu *vcpu)
>  {
> +       int r;
>         enum emulation_result er;
>         u32 rt;
>         void *data = run->mmio.data;
> @@ -1666,9 +1667,18 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
>                 goto out_fail;
>         }
>
> -       run->mmio.is_write = 1;
>         vcpu->mmio_needed = 1;
> +       run->mmio.is_write = 1;
>         vcpu->mmio_is_write = 1;
> +
> +       r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS,
> +                       run->mmio.phys_addr, run->mmio.len, data);
> +
> +       if (!r) {
> +               vcpu->mmio_needed = 0;
> +               return EMULATE_DONE;
> +       }
> +
>         return EMULATE_DO_MMIO;
>
>  out_fail:
> @@ -1681,6 +1691,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
>                                             u32 cause, struct kvm_run *run,
>                                             struct kvm_vcpu *vcpu)
>  {
> +       int r;
>         enum emulation_result er;
>         unsigned long curr_pc;
>         u32 op, rt;
> @@ -1745,6 +1756,16 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
>
>         run->mmio.is_write = 0;
>         vcpu->mmio_is_write = 0;
> +
> +       r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS,
> +                       run->mmio.phys_addr, run->mmio.len, run->mmio.data);
> +
> +       if (!r) {
> +               kvm_mips_complete_mmio_load(vcpu, run);
> +               vcpu->mmio_needed = 0;
> +               return EMULATE_DONE;
> +       }
> +
>         return EMULATE_DO_MMIO;
>  }
>
> diff --git a/arch/mips/kvm/loongson_ipi.c b/arch/mips/kvm/loongson_ipi.c
> new file mode 100644
> index 00000000..3681fc8
> --- /dev/null
> +++ b/arch/mips/kvm/loongson_ipi.c
> @@ -0,0 +1,214 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Loongson-3 Virtual IPI interrupt support.
> + *
> + * Copyright (C) 2019  Loongson Technologies, Inc.  All rights reserved.
> + *
> + * Authors: Chen Zhu <zhuchen@loongson.cn>
> + * Authors: Huacai Chen <chenhc@lemote.com>
> + */
> +
> +#include <linux/kvm_host.h>
> +
> +#define IPI_BASE            0x3ff01000ULL
> +
> +#define CORE0_STATUS_OFF       0x000
> +#define CORE0_EN_OFF           0x004
> +#define CORE0_SET_OFF          0x008
> +#define CORE0_CLEAR_OFF        0x00c
> +#define CORE0_BUF_20           0x020
> +#define CORE0_BUF_28           0x028
> +#define CORE0_BUF_30           0x030
> +#define CORE0_BUF_38           0x038
> +
> +#define CORE1_STATUS_OFF       0x100
> +#define CORE1_EN_OFF           0x104
> +#define CORE1_SET_OFF          0x108
> +#define CORE1_CLEAR_OFF        0x10c
> +#define CORE1_BUF_20           0x120
> +#define CORE1_BUF_28           0x128
> +#define CORE1_BUF_30           0x130
> +#define CORE1_BUF_38           0x138
> +
> +#define CORE2_STATUS_OFF       0x200
> +#define CORE2_EN_OFF           0x204
> +#define CORE2_SET_OFF          0x208
> +#define CORE2_CLEAR_OFF        0x20c
> +#define CORE2_BUF_20           0x220
> +#define CORE2_BUF_28           0x228
> +#define CORE2_BUF_30           0x230
> +#define CORE2_BUF_38           0x238
> +
> +#define CORE3_STATUS_OFF       0x300
> +#define CORE3_EN_OFF           0x304
> +#define CORE3_SET_OFF          0x308
> +#define CORE3_CLEAR_OFF        0x30c
> +#define CORE3_BUF_20           0x320
> +#define CORE3_BUF_28           0x328
> +#define CORE3_BUF_30           0x330
> +#define CORE3_BUF_38           0x338
> +
> +static int loongson_vipi_read(struct loongson_kvm_ipi *ipi,
> +                               gpa_t addr, int len, void *val)
> +{
> +       uint32_t core = (addr >> 8) & 3;
> +       uint32_t node = (addr >> 44) & 3;
> +       uint32_t id = core + node * 4;
> +       uint64_t offset = addr & 0xff;
> +       void *pbuf;
> +       struct ipi_state *s = &(ipi->ipistate[id]);
> +
> +       BUG_ON(offset & (len - 1));
> +
> +       switch (offset) {
> +       case CORE0_STATUS_OFF:
> +               *(uint64_t *)val = s->status;
> +               break;
> +
> +       case CORE0_EN_OFF:
> +               *(uint64_t *)val = s->en;
> +               break;
> +
> +       case CORE0_SET_OFF:
> +               *(uint64_t *)val = 0;
> +               break;
> +
> +       case CORE0_CLEAR_OFF:
> +               *(uint64_t *)val = 0;
> +               break;
> +
> +       case CORE0_BUF_20 ... CORE0_BUF_38:
> +               pbuf = (void *)s->buf + (offset - 0x20);
> +               if (len == 8)
> +                       *(uint64_t *)val = *(uint64_t *)pbuf;
> +               else /* Assume len == 4 */
> +                       *(uint32_t *)val = *(uint32_t *)pbuf;
> +               break;
> +
> +       default:
> +               pr_notice("%s with unknown addr %llx\n", __func__, addr);
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
> +static int loongson_vipi_write(struct loongson_kvm_ipi *ipi,
> +                               gpa_t addr, int len, const void *val)
> +{
> +       uint32_t core = (addr >> 8) & 3;
> +       uint32_t node = (addr >> 44) & 3;
> +       uint32_t id = core + node * 4;
> +       uint64_t data, offset = addr & 0xff;
> +       void *pbuf;
> +       struct kvm *kvm = ipi->kvm;
> +       struct kvm_mips_interrupt irq;
> +       struct ipi_state *s = &(ipi->ipistate[id]);
> +
> +       data = *(uint64_t *)val;
> +       BUG_ON(offset & (len - 1));
> +
> +       switch (offset) {
> +       case CORE0_STATUS_OFF:
> +               break;
> +
> +       case CORE0_EN_OFF:
> +               s->en = data;
> +               break;
> +
> +       case CORE0_SET_OFF:
> +               s->status |= data;
> +               irq.cpu = id;
> +               irq.irq = 6;
> +               kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
> +               break;
> +
> +       case CORE0_CLEAR_OFF:
> +               s->status &= ~data;
> +               if (!s->status) {
> +                       irq.cpu = id;
> +                       irq.irq = -6;
> +                       kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
> +               }
> +               break;
> +
> +       case CORE0_BUF_20 ... CORE0_BUF_38:
> +               pbuf = (void *)s->buf + (offset - 0x20);
> +               if (len == 8)
> +                       *(uint64_t *)pbuf = (uint64_t)data;
> +               else /* Assume len == 4 */
> +                       *(uint32_t *)pbuf = (uint32_t)data;
> +               break;
> +
> +       default:
> +               pr_notice("%s with unknown addr %llx\n", __func__, addr);
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
> +static int kvm_ipi_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> +                       gpa_t addr, int len, void *val)
> +{
> +       unsigned long flags;
> +       struct loongson_kvm_ipi *ipi;
> +       struct ipi_io_device *ipi_device;
> +
> +       ipi_device = container_of(dev, struct ipi_io_device, device);
> +       ipi = ipi_device->ipi;
> +
> +       spin_lock_irqsave(&ipi->lock, flags);
> +       loongson_vipi_read(ipi, addr, len, val);
> +       spin_unlock_irqrestore(&ipi->lock, flags);
> +
> +       return 0;
> +}
> +
> +static int kvm_ipi_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> +                       gpa_t addr, int len, const void *val)
> +{
> +       unsigned long flags;
> +       struct loongson_kvm_ipi *ipi;
> +       struct ipi_io_device *ipi_device;
> +
> +       ipi_device = container_of(dev, struct ipi_io_device, device);
> +       ipi = ipi_device->ipi;
> +
> +       spin_lock_irqsave(&ipi->lock, flags);
> +       loongson_vipi_write(ipi, addr, len, val);
> +       spin_unlock_irqrestore(&ipi->lock, flags);
> +
> +       return 0;
> +}
> +
> +static const struct kvm_io_device_ops kvm_ipi_ops = {
> +       .read     = kvm_ipi_read,
> +       .write    = kvm_ipi_write,
> +};
> +
> +void kvm_init_loongson_ipi(struct kvm *kvm)
> +{
> +       int i;
> +       unsigned long addr;
> +       struct loongson_kvm_ipi *s;
> +       struct kvm_io_device *device;
> +
> +       s = &kvm->arch.ipi;
> +       s->kvm = kvm;
> +       spin_lock_init(&s->lock);
> +
> +       /*
> +        * Initialize IPI device
> +        */
> +       for (i = 0; i < 4; i++) {
> +               device = &s->dev_ipi[i].device;
> +               kvm_iodevice_init(device, &kvm_ipi_ops);
> +               addr = (((unsigned long)i) << 44) + IPI_BASE;
> +               mutex_lock(&kvm->slots_lock);
> +               kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, addr, 0x400, device);
> +               mutex_unlock(&kvm->slots_lock);
> +               s->dev_ipi[i].ipi = s;
> +               s->dev_ipi[i].node_id = i;
> +       }
> +}
> diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> index 5ca122c..ed989ef 100644
> --- a/arch/mips/kvm/mips.c
> +++ b/arch/mips/kvm/mips.c
> @@ -128,6 +128,8 @@ int kvm_arch_check_processor_compat(void *opaque)
>         return 0;
>  }
>
> +extern void kvm_init_loongson_ipi(struct kvm *kvm);
> +
>  int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>  {
>         switch (type) {
> @@ -147,6 +149,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>         if (!kvm->arch.gpa_mm.pgd)
>                 return -ENOMEM;
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +       kvm_init_loongson_ipi(kvm);
> +#endif
> +
>         return 0;
>  }
>
> --
> 2.7.0
>

Re: [PATCH V3 10/14] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:14 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> This patch add Loongson-3 Virtual IPI interrupt support in the kernel,
> because emulate it in QEMU is too expensive for performance.
>

Huacei, hi!

While, in principle, I support this patch, could you please, just for
the sake of everybody being on the same page, describe the
hyphothetical alternative implementation in QEMU, and explain why it
would be more expensive than the solution proposed in this patch.

No need to go into details, I would like to see just the general idea
you had in your mind.

Yours,
Aleksandar


> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/include/asm/kvm_host.h |  32 ++++++
>  arch/mips/kvm/Makefile           |   3 +
>  arch/mips/kvm/emulate.c          |  23 ++++-
>  arch/mips/kvm/loongson_ipi.c     | 214 +++++++++++++++++++++++++++++++++++++++
>  arch/mips/kvm/mips.c             |   6 ++
>  5 files changed, 277 insertions(+), 1 deletion(-)
>  create mode 100644 arch/mips/kvm/loongson_ipi.c
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index a7758c0..f165902 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -23,6 +23,8 @@
>  #include <asm/inst.h>
>  #include <asm/mipsregs.h>
>
> +#include <kvm/iodev.h>
> +
>  /* MIPS KVM register ids */
>  #define MIPS_CP0_32(_R, _S)                                    \
>         (KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
> @@ -181,11 +183,39 @@ struct kvm_vcpu_stat {
>  struct kvm_arch_memory_slot {
>  };
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +struct ipi_state {
> +       uint32_t status;
> +       uint32_t en;
> +       uint32_t set;
> +       uint32_t clear;
> +       uint64_t buf[4];
> +};
> +
> +struct loongson_kvm_ipi;
> +
> +struct ipi_io_device {
> +       int node_id;
> +       struct loongson_kvm_ipi *ipi;
> +       struct kvm_io_device device;
> +};
> +
> +struct loongson_kvm_ipi {
> +       spinlock_t lock;
> +       struct kvm *kvm;
> +       struct ipi_state ipistate[16];
> +       struct ipi_io_device dev_ipi[4];
> +};
> +#endif
> +
>  struct kvm_arch {
>         /* Guest physical mm */
>         struct mm_struct gpa_mm;
>         /* Mask of CPUs needing GPA ASID flush */
>         cpumask_t asid_flush_mask;
> +#ifdef CONFIG_CPU_LOONGSON64
> +       struct loongson_kvm_ipi ipi;
> +#endif
>  };
>
>  #define N_MIPS_COPROC_REGS     32
> @@ -1133,6 +1163,8 @@ extern int kvm_mips_trans_mtc0(union mips_instruction inst, u32 *opc,
>  /* Misc */
>  extern void kvm_mips_dump_stats(struct kvm_vcpu *vcpu);
>  extern unsigned long kvm_mips_get_ramsize(struct kvm *kvm);
> +extern int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
> +                            struct kvm_mips_interrupt *irq);
>
>  static inline void kvm_arch_hardware_unsetup(void) {}
>  static inline void kvm_arch_sync_events(struct kvm *kvm) {}
> diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> index 0a3cef6..506c4ac 100644
> --- a/arch/mips/kvm/Makefile
> +++ b/arch/mips/kvm/Makefile
> @@ -13,6 +13,9 @@ kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
>             fpu.o
>  kvm-objs += hypcall.o
>  kvm-objs += mmu.o
> +ifdef CONFIG_CPU_LOONGSON64
> +kvm-objs += loongson_ipi.o
> +endif
>
>  ifdef CONFIG_KVM_MIPS_VZ
>  kvm-objs               += vz.o
> diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
> index 754094b..3946499 100644
> --- a/arch/mips/kvm/emulate.c
> +++ b/arch/mips/kvm/emulate.c
> @@ -1600,6 +1600,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
>                                              struct kvm_run *run,
>                                              struct kvm_vcpu *vcpu)
>  {
> +       int r;
>         enum emulation_result er;
>         u32 rt;
>         void *data = run->mmio.data;
> @@ -1666,9 +1667,18 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
>                 goto out_fail;
>         }
>
> -       run->mmio.is_write = 1;
>         vcpu->mmio_needed = 1;
> +       run->mmio.is_write = 1;
>         vcpu->mmio_is_write = 1;
> +
> +       r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS,
> +                       run->mmio.phys_addr, run->mmio.len, data);
> +
> +       if (!r) {
> +               vcpu->mmio_needed = 0;
> +               return EMULATE_DONE;
> +       }
> +
>         return EMULATE_DO_MMIO;
>
>  out_fail:
> @@ -1681,6 +1691,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
>                                             u32 cause, struct kvm_run *run,
>                                             struct kvm_vcpu *vcpu)
>  {
> +       int r;
>         enum emulation_result er;
>         unsigned long curr_pc;
>         u32 op, rt;
> @@ -1745,6 +1756,16 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
>
>         run->mmio.is_write = 0;
>         vcpu->mmio_is_write = 0;
> +
> +       r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS,
> +                       run->mmio.phys_addr, run->mmio.len, run->mmio.data);
> +
> +       if (!r) {
> +               kvm_mips_complete_mmio_load(vcpu, run);
> +               vcpu->mmio_needed = 0;
> +               return EMULATE_DONE;
> +       }
> +
>         return EMULATE_DO_MMIO;
>  }
>
> diff --git a/arch/mips/kvm/loongson_ipi.c b/arch/mips/kvm/loongson_ipi.c
> new file mode 100644
> index 00000000..3681fc8
> --- /dev/null
> +++ b/arch/mips/kvm/loongson_ipi.c
> @@ -0,0 +1,214 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Loongson-3 Virtual IPI interrupt support.
> + *
> + * Copyright (C) 2019  Loongson Technologies, Inc.  All rights reserved.
> + *
> + * Authors: Chen Zhu <zhuchen@loongson.cn>
> + * Authors: Huacai Chen <chenhc@lemote.com>
> + */
> +
> +#include <linux/kvm_host.h>
> +
> +#define IPI_BASE            0x3ff01000ULL
> +
> +#define CORE0_STATUS_OFF       0x000
> +#define CORE0_EN_OFF           0x004
> +#define CORE0_SET_OFF          0x008
> +#define CORE0_CLEAR_OFF        0x00c
> +#define CORE0_BUF_20           0x020
> +#define CORE0_BUF_28           0x028
> +#define CORE0_BUF_30           0x030
> +#define CORE0_BUF_38           0x038
> +
> +#define CORE1_STATUS_OFF       0x100
> +#define CORE1_EN_OFF           0x104
> +#define CORE1_SET_OFF          0x108
> +#define CORE1_CLEAR_OFF        0x10c
> +#define CORE1_BUF_20           0x120
> +#define CORE1_BUF_28           0x128
> +#define CORE1_BUF_30           0x130
> +#define CORE1_BUF_38           0x138
> +
> +#define CORE2_STATUS_OFF       0x200
> +#define CORE2_EN_OFF           0x204
> +#define CORE2_SET_OFF          0x208
> +#define CORE2_CLEAR_OFF        0x20c
> +#define CORE2_BUF_20           0x220
> +#define CORE2_BUF_28           0x228
> +#define CORE2_BUF_30           0x230
> +#define CORE2_BUF_38           0x238
> +
> +#define CORE3_STATUS_OFF       0x300
> +#define CORE3_EN_OFF           0x304
> +#define CORE3_SET_OFF          0x308
> +#define CORE3_CLEAR_OFF        0x30c
> +#define CORE3_BUF_20           0x320
> +#define CORE3_BUF_28           0x328
> +#define CORE3_BUF_30           0x330
> +#define CORE3_BUF_38           0x338
> +
> +static int loongson_vipi_read(struct loongson_kvm_ipi *ipi,
> +                               gpa_t addr, int len, void *val)
> +{
> +       uint32_t core = (addr >> 8) & 3;
> +       uint32_t node = (addr >> 44) & 3;
> +       uint32_t id = core + node * 4;
> +       uint64_t offset = addr & 0xff;
> +       void *pbuf;
> +       struct ipi_state *s = &(ipi->ipistate[id]);
> +
> +       BUG_ON(offset & (len - 1));
> +
> +       switch (offset) {
> +       case CORE0_STATUS_OFF:
> +               *(uint64_t *)val = s->status;
> +               break;
> +
> +       case CORE0_EN_OFF:
> +               *(uint64_t *)val = s->en;
> +               break;
> +
> +       case CORE0_SET_OFF:
> +               *(uint64_t *)val = 0;
> +               break;
> +
> +       case CORE0_CLEAR_OFF:
> +               *(uint64_t *)val = 0;
> +               break;
> +
> +       case CORE0_BUF_20 ... CORE0_BUF_38:
> +               pbuf = (void *)s->buf + (offset - 0x20);
> +               if (len == 8)
> +                       *(uint64_t *)val = *(uint64_t *)pbuf;
> +               else /* Assume len == 4 */
> +                       *(uint32_t *)val = *(uint32_t *)pbuf;
> +               break;
> +
> +       default:
> +               pr_notice("%s with unknown addr %llx\n", __func__, addr);
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
> +static int loongson_vipi_write(struct loongson_kvm_ipi *ipi,
> +                               gpa_t addr, int len, const void *val)
> +{
> +       uint32_t core = (addr >> 8) & 3;
> +       uint32_t node = (addr >> 44) & 3;
> +       uint32_t id = core + node * 4;
> +       uint64_t data, offset = addr & 0xff;
> +       void *pbuf;
> +       struct kvm *kvm = ipi->kvm;
> +       struct kvm_mips_interrupt irq;
> +       struct ipi_state *s = &(ipi->ipistate[id]);
> +
> +       data = *(uint64_t *)val;
> +       BUG_ON(offset & (len - 1));
> +
> +       switch (offset) {
> +       case CORE0_STATUS_OFF:
> +               break;
> +
> +       case CORE0_EN_OFF:
> +               s->en = data;
> +               break;
> +
> +       case CORE0_SET_OFF:
> +               s->status |= data;
> +               irq.cpu = id;
> +               irq.irq = 6;
> +               kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
> +               break;
> +
> +       case CORE0_CLEAR_OFF:
> +               s->status &= ~data;
> +               if (!s->status) {
> +                       irq.cpu = id;
> +                       irq.irq = -6;
> +                       kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
> +               }
> +               break;
> +
> +       case CORE0_BUF_20 ... CORE0_BUF_38:
> +               pbuf = (void *)s->buf + (offset - 0x20);
> +               if (len == 8)
> +                       *(uint64_t *)pbuf = (uint64_t)data;
> +               else /* Assume len == 4 */
> +                       *(uint32_t *)pbuf = (uint32_t)data;
> +               break;
> +
> +       default:
> +               pr_notice("%s with unknown addr %llx\n", __func__, addr);
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
> +static int kvm_ipi_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> +                       gpa_t addr, int len, void *val)
> +{
> +       unsigned long flags;
> +       struct loongson_kvm_ipi *ipi;
> +       struct ipi_io_device *ipi_device;
> +
> +       ipi_device = container_of(dev, struct ipi_io_device, device);
> +       ipi = ipi_device->ipi;
> +
> +       spin_lock_irqsave(&ipi->lock, flags);
> +       loongson_vipi_read(ipi, addr, len, val);
> +       spin_unlock_irqrestore(&ipi->lock, flags);
> +
> +       return 0;
> +}
> +
> +static int kvm_ipi_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> +                       gpa_t addr, int len, const void *val)
> +{
> +       unsigned long flags;
> +       struct loongson_kvm_ipi *ipi;
> +       struct ipi_io_device *ipi_device;
> +
> +       ipi_device = container_of(dev, struct ipi_io_device, device);
> +       ipi = ipi_device->ipi;
> +
> +       spin_lock_irqsave(&ipi->lock, flags);
> +       loongson_vipi_write(ipi, addr, len, val);
> +       spin_unlock_irqrestore(&ipi->lock, flags);
> +
> +       return 0;
> +}
> +
> +static const struct kvm_io_device_ops kvm_ipi_ops = {
> +       .read     = kvm_ipi_read,
> +       .write    = kvm_ipi_write,
> +};
> +
> +void kvm_init_loongson_ipi(struct kvm *kvm)
> +{
> +       int i;
> +       unsigned long addr;
> +       struct loongson_kvm_ipi *s;
> +       struct kvm_io_device *device;
> +
> +       s = &kvm->arch.ipi;
> +       s->kvm = kvm;
> +       spin_lock_init(&s->lock);
> +
> +       /*
> +        * Initialize IPI device
> +        */
> +       for (i = 0; i < 4; i++) {
> +               device = &s->dev_ipi[i].device;
> +               kvm_iodevice_init(device, &kvm_ipi_ops);
> +               addr = (((unsigned long)i) << 44) + IPI_BASE;
> +               mutex_lock(&kvm->slots_lock);
> +               kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, addr, 0x400, device);
> +               mutex_unlock(&kvm->slots_lock);
> +               s->dev_ipi[i].ipi = s;
> +               s->dev_ipi[i].node_id = i;
> +       }
> +}
> diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> index 5ca122c..ed989ef 100644
> --- a/arch/mips/kvm/mips.c
> +++ b/arch/mips/kvm/mips.c
> @@ -128,6 +128,8 @@ int kvm_arch_check_processor_compat(void *opaque)
>         return 0;
>  }
>
> +extern void kvm_init_loongson_ipi(struct kvm *kvm);
> +
>  int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>  {
>         switch (type) {
> @@ -147,6 +149,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>         if (!kvm->arch.gpa_mm.pgd)
>                 return -ENOMEM;
>
> +#ifdef CONFIG_CPU_LOONGSON64
> +       kvm_init_loongson_ipi(kvm);
> +#endif
> +
>         return 0;
>  }
>
> --
> 2.7.0
>

Re: [PATCH V3 10/14] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support

[Huacai Chen]

Hi, Aleksandar,

On Fri, May 8, 2020 at 7:23 PM Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> wrote:
>
> нед, 3. мај 2020. у 12:14 Huacai Chen <chenhc@lemote.com> је написао/ла:
> >
> > This patch add Loongson-3 Virtual IPI interrupt support in the kernel,
> > because emulate it in QEMU is too expensive for performance.
> >
>
> Huacei, hi!
>
> While, in principle, I support this patch, could you please, just for
> the sake of everybody being on the same page, describe the
> hyphothetical alternative implementation in QEMU, and explain why it
> would be more expensive than the solution proposed in this patch.
>
> No need to go into details, I would like to see just the general idea
> you had in your mind.
The current implementation of IPI emulation in QEMU is based on GIC for
MIPS, but Loongson-3 doesn't use GIC. Furthermore, IPI emulation in QEMU
is too expensive for performance (because of too many context switches
between Host and Guest). With current solution, the IPI delay may even
cause RCU stall warnings in a multi-core Guest. So, we design a faster
solution that emulate IPI interrupt in kernel (only used by Loongson-3
now).


Huacai


>
> Yours,
> Aleksandar
>
>
> > Signed-off-by: Huacai Chen <chenhc@lemote.com>
> > Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> > ---
> >  arch/mips/include/asm/kvm_host.h |  32 ++++++
> >  arch/mips/kvm/Makefile           |   3 +
> >  arch/mips/kvm/emulate.c          |  23 ++++-
> >  arch/mips/kvm/loongson_ipi.c     | 214 +++++++++++++++++++++++++++++++++++++++
> >  arch/mips/kvm/mips.c             |   6 ++
> >  5 files changed, 277 insertions(+), 1 deletion(-)
> >  create mode 100644 arch/mips/kvm/loongson_ipi.c
> >
> > diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> > index a7758c0..f165902 100644
> > --- a/arch/mips/include/asm/kvm_host.h
> > +++ b/arch/mips/include/asm/kvm_host.h
> > @@ -23,6 +23,8 @@
> >  #include <asm/inst.h>
> >  #include <asm/mipsregs.h>
> >
> > +#include <kvm/iodev.h>
> > +
> >  /* MIPS KVM register ids */
> >  #define MIPS_CP0_32(_R, _S)                                    \
> >         (KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
> > @@ -181,11 +183,39 @@ struct kvm_vcpu_stat {
> >  struct kvm_arch_memory_slot {
> >  };
> >
> > +#ifdef CONFIG_CPU_LOONGSON64
> > +struct ipi_state {
> > +       uint32_t status;
> > +       uint32_t en;
> > +       uint32_t set;
> > +       uint32_t clear;
> > +       uint64_t buf[4];
> > +};
> > +
> > +struct loongson_kvm_ipi;
> > +
> > +struct ipi_io_device {
> > +       int node_id;
> > +       struct loongson_kvm_ipi *ipi;
> > +       struct kvm_io_device device;
> > +};
> > +
> > +struct loongson_kvm_ipi {
> > +       spinlock_t lock;
> > +       struct kvm *kvm;
> > +       struct ipi_state ipistate[16];
> > +       struct ipi_io_device dev_ipi[4];
> > +};
> > +#endif
> > +
> >  struct kvm_arch {
> >         /* Guest physical mm */
> >         struct mm_struct gpa_mm;
> >         /* Mask of CPUs needing GPA ASID flush */
> >         cpumask_t asid_flush_mask;
> > +#ifdef CONFIG_CPU_LOONGSON64
> > +       struct loongson_kvm_ipi ipi;
> > +#endif
> >  };
> >
> >  #define N_MIPS_COPROC_REGS     32
> > @@ -1133,6 +1163,8 @@ extern int kvm_mips_trans_mtc0(union mips_instruction inst, u32 *opc,
> >  /* Misc */
> >  extern void kvm_mips_dump_stats(struct kvm_vcpu *vcpu);
> >  extern unsigned long kvm_mips_get_ramsize(struct kvm *kvm);
> > +extern int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
> > +                            struct kvm_mips_interrupt *irq);
> >
> >  static inline void kvm_arch_hardware_unsetup(void) {}
> >  static inline void kvm_arch_sync_events(struct kvm *kvm) {}
> > diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> > index 0a3cef6..506c4ac 100644
> > --- a/arch/mips/kvm/Makefile
> > +++ b/arch/mips/kvm/Makefile
> > @@ -13,6 +13,9 @@ kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
> >             fpu.o
> >  kvm-objs += hypcall.o
> >  kvm-objs += mmu.o
> > +ifdef CONFIG_CPU_LOONGSON64
> > +kvm-objs += loongson_ipi.o
> > +endif
> >
> >  ifdef CONFIG_KVM_MIPS_VZ
> >  kvm-objs               += vz.o
> > diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
> > index 754094b..3946499 100644
> > --- a/arch/mips/kvm/emulate.c
> > +++ b/arch/mips/kvm/emulate.c
> > @@ -1600,6 +1600,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
> >                                              struct kvm_run *run,
> >                                              struct kvm_vcpu *vcpu)
> >  {
> > +       int r;
> >         enum emulation_result er;
> >         u32 rt;
> >         void *data = run->mmio.data;
> > @@ -1666,9 +1667,18 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
> >                 goto out_fail;
> >         }
> >
> > -       run->mmio.is_write = 1;
> >         vcpu->mmio_needed = 1;
> > +       run->mmio.is_write = 1;
> >         vcpu->mmio_is_write = 1;
> > +
> > +       r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS,
> > +                       run->mmio.phys_addr, run->mmio.len, data);
> > +
> > +       if (!r) {
> > +               vcpu->mmio_needed = 0;
> > +               return EMULATE_DONE;
> > +       }
> > +
> >         return EMULATE_DO_MMIO;
> >
> >  out_fail:
> > @@ -1681,6 +1691,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
> >                                             u32 cause, struct kvm_run *run,
> >                                             struct kvm_vcpu *vcpu)
> >  {
> > +       int r;
> >         enum emulation_result er;
> >         unsigned long curr_pc;
> >         u32 op, rt;
> > @@ -1745,6 +1756,16 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
> >
> >         run->mmio.is_write = 0;
> >         vcpu->mmio_is_write = 0;
> > +
> > +       r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS,
> > +                       run->mmio.phys_addr, run->mmio.len, run->mmio.data);
> > +
> > +       if (!r) {
> > +               kvm_mips_complete_mmio_load(vcpu, run);
> > +               vcpu->mmio_needed = 0;
> > +               return EMULATE_DONE;
> > +       }
> > +
> >         return EMULATE_DO_MMIO;
> >  }
> >
> > diff --git a/arch/mips/kvm/loongson_ipi.c b/arch/mips/kvm/loongson_ipi.c
> > new file mode 100644
> > index 00000000..3681fc8
> > --- /dev/null
> > +++ b/arch/mips/kvm/loongson_ipi.c
> > @@ -0,0 +1,214 @@
> > +// SPDX-License-Identifier: GPL-2.0-or-later
> > +/*
> > + * Loongson-3 Virtual IPI interrupt support.
> > + *
> > + * Copyright (C) 2019  Loongson Technologies, Inc.  All rights reserved.
> > + *
> > + * Authors: Chen Zhu <zhuchen@loongson.cn>
> > + * Authors: Huacai Chen <chenhc@lemote.com>
> > + */
> > +
> > +#include <linux/kvm_host.h>
> > +
> > +#define IPI_BASE            0x3ff01000ULL
> > +
> > +#define CORE0_STATUS_OFF       0x000
> > +#define CORE0_EN_OFF           0x004
> > +#define CORE0_SET_OFF          0x008
> > +#define CORE0_CLEAR_OFF        0x00c
> > +#define CORE0_BUF_20           0x020
> > +#define CORE0_BUF_28           0x028
> > +#define CORE0_BUF_30           0x030
> > +#define CORE0_BUF_38           0x038
> > +
> > +#define CORE1_STATUS_OFF       0x100
> > +#define CORE1_EN_OFF           0x104
> > +#define CORE1_SET_OFF          0x108
> > +#define CORE1_CLEAR_OFF        0x10c
> > +#define CORE1_BUF_20           0x120
> > +#define CORE1_BUF_28           0x128
> > +#define CORE1_BUF_30           0x130
> > +#define CORE1_BUF_38           0x138
> > +
> > +#define CORE2_STATUS_OFF       0x200
> > +#define CORE2_EN_OFF           0x204
> > +#define CORE2_SET_OFF          0x208
> > +#define CORE2_CLEAR_OFF        0x20c
> > +#define CORE2_BUF_20           0x220
> > +#define CORE2_BUF_28           0x228
> > +#define CORE2_BUF_30           0x230
> > +#define CORE2_BUF_38           0x238
> > +
> > +#define CORE3_STATUS_OFF       0x300
> > +#define CORE3_EN_OFF           0x304
> > +#define CORE3_SET_OFF          0x308
> > +#define CORE3_CLEAR_OFF        0x30c
> > +#define CORE3_BUF_20           0x320
> > +#define CORE3_BUF_28           0x328
> > +#define CORE3_BUF_30           0x330
> > +#define CORE3_BUF_38           0x338
> > +
> > +static int loongson_vipi_read(struct loongson_kvm_ipi *ipi,
> > +                               gpa_t addr, int len, void *val)
> > +{
> > +       uint32_t core = (addr >> 8) & 3;
> > +       uint32_t node = (addr >> 44) & 3;
> > +       uint32_t id = core + node * 4;
> > +       uint64_t offset = addr & 0xff;
> > +       void *pbuf;
> > +       struct ipi_state *s = &(ipi->ipistate[id]);
> > +
> > +       BUG_ON(offset & (len - 1));
> > +
> > +       switch (offset) {
> > +       case CORE0_STATUS_OFF:
> > +               *(uint64_t *)val = s->status;
> > +               break;
> > +
> > +       case CORE0_EN_OFF:
> > +               *(uint64_t *)val = s->en;
> > +               break;
> > +
> > +       case CORE0_SET_OFF:
> > +               *(uint64_t *)val = 0;
> > +               break;
> > +
> > +       case CORE0_CLEAR_OFF:
> > +               *(uint64_t *)val = 0;
> > +               break;
> > +
> > +       case CORE0_BUF_20 ... CORE0_BUF_38:
> > +               pbuf = (void *)s->buf + (offset - 0x20);
> > +               if (len == 8)
> > +                       *(uint64_t *)val = *(uint64_t *)pbuf;
> > +               else /* Assume len == 4 */
> > +                       *(uint32_t *)val = *(uint32_t *)pbuf;
> > +               break;
> > +
> > +       default:
> > +               pr_notice("%s with unknown addr %llx\n", __func__, addr);
> > +               break;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static int loongson_vipi_write(struct loongson_kvm_ipi *ipi,
> > +                               gpa_t addr, int len, const void *val)
> > +{
> > +       uint32_t core = (addr >> 8) & 3;
> > +       uint32_t node = (addr >> 44) & 3;
> > +       uint32_t id = core + node * 4;
> > +       uint64_t data, offset = addr & 0xff;
> > +       void *pbuf;
> > +       struct kvm *kvm = ipi->kvm;
> > +       struct kvm_mips_interrupt irq;
> > +       struct ipi_state *s = &(ipi->ipistate[id]);
> > +
> > +       data = *(uint64_t *)val;
> > +       BUG_ON(offset & (len - 1));
> > +
> > +       switch (offset) {
> > +       case CORE0_STATUS_OFF:
> > +               break;
> > +
> > +       case CORE0_EN_OFF:
> > +               s->en = data;
> > +               break;
> > +
> > +       case CORE0_SET_OFF:
> > +               s->status |= data;
> > +               irq.cpu = id;
> > +               irq.irq = 6;
> > +               kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
> > +               break;
> > +
> > +       case CORE0_CLEAR_OFF:
> > +               s->status &= ~data;
> > +               if (!s->status) {
> > +                       irq.cpu = id;
> > +                       irq.irq = -6;
> > +                       kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
> > +               }
> > +               break;
> > +
> > +       case CORE0_BUF_20 ... CORE0_BUF_38:
> > +               pbuf = (void *)s->buf + (offset - 0x20);
> > +               if (len == 8)
> > +                       *(uint64_t *)pbuf = (uint64_t)data;
> > +               else /* Assume len == 4 */
> > +                       *(uint32_t *)pbuf = (uint32_t)data;
> > +               break;
> > +
> > +       default:
> > +               pr_notice("%s with unknown addr %llx\n", __func__, addr);
> > +               break;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static int kvm_ipi_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> > +                       gpa_t addr, int len, void *val)
> > +{
> > +       unsigned long flags;
> > +       struct loongson_kvm_ipi *ipi;
> > +       struct ipi_io_device *ipi_device;
> > +
> > +       ipi_device = container_of(dev, struct ipi_io_device, device);
> > +       ipi = ipi_device->ipi;
> > +
> > +       spin_lock_irqsave(&ipi->lock, flags);
> > +       loongson_vipi_read(ipi, addr, len, val);
> > +       spin_unlock_irqrestore(&ipi->lock, flags);
> > +
> > +       return 0;
> > +}
> > +
> > +static int kvm_ipi_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> > +                       gpa_t addr, int len, const void *val)
> > +{
> > +       unsigned long flags;
> > +       struct loongson_kvm_ipi *ipi;
> > +       struct ipi_io_device *ipi_device;
> > +
> > +       ipi_device = container_of(dev, struct ipi_io_device, device);
> > +       ipi = ipi_device->ipi;
> > +
> > +       spin_lock_irqsave(&ipi->lock, flags);
> > +       loongson_vipi_write(ipi, addr, len, val);
> > +       spin_unlock_irqrestore(&ipi->lock, flags);
> > +
> > +       return 0;
> > +}
> > +
> > +static const struct kvm_io_device_ops kvm_ipi_ops = {
> > +       .read     = kvm_ipi_read,
> > +       .write    = kvm_ipi_write,
> > +};
> > +
> > +void kvm_init_loongson_ipi(struct kvm *kvm)
> > +{
> > +       int i;
> > +       unsigned long addr;
> > +       struct loongson_kvm_ipi *s;
> > +       struct kvm_io_device *device;
> > +
> > +       s = &kvm->arch.ipi;
> > +       s->kvm = kvm;
> > +       spin_lock_init(&s->lock);
> > +
> > +       /*
> > +        * Initialize IPI device
> > +        */
> > +       for (i = 0; i < 4; i++) {
> > +               device = &s->dev_ipi[i].device;
> > +               kvm_iodevice_init(device, &kvm_ipi_ops);
> > +               addr = (((unsigned long)i) << 44) + IPI_BASE;
> > +               mutex_lock(&kvm->slots_lock);
> > +               kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, addr, 0x400, device);
> > +               mutex_unlock(&kvm->slots_lock);
> > +               s->dev_ipi[i].ipi = s;
> > +               s->dev_ipi[i].node_id = i;
> > +       }
> > +}
> > diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> > index 5ca122c..ed989ef 100644
> > --- a/arch/mips/kvm/mips.c
> > +++ b/arch/mips/kvm/mips.c
> > @@ -128,6 +128,8 @@ int kvm_arch_check_processor_compat(void *opaque)
> >         return 0;
> >  }
> >
> > +extern void kvm_init_loongson_ipi(struct kvm *kvm);
> > +
> >  int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> >  {
> >         switch (type) {
> > @@ -147,6 +149,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> >         if (!kvm->arch.gpa_mm.pgd)
> >                 return -ENOMEM;
> >
> > +#ifdef CONFIG_CPU_LOONGSON64
> > +       kvm_init_loongson_ipi(kvm);
> > +#endif
> > +
> >         return 0;
> >  }
> >
> > --
> > 2.7.0
> >

Re: [PATCH V3 10/14] KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support

[Huacai Chen]

Hi, Aleksandar,

On Fri, May 8, 2020 at 7:23 PM Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> wrote:
>
> нед, 3. мај 2020. у 12:14 Huacai Chen <chenhc@lemote.com> је написао/ла:
> >
> > This patch add Loongson-3 Virtual IPI interrupt support in the kernel,
> > because emulate it in QEMU is too expensive for performance.
> >
>
> Huacei, hi!
>
> While, in principle, I support this patch, could you please, just for
> the sake of everybody being on the same page, describe the
> hyphothetical alternative implementation in QEMU, and explain why it
> would be more expensive than the solution proposed in this patch.
>
> No need to go into details, I would like to see just the general idea
> you had in your mind.
The current implementation of IPI emulation in QEMU is based on GIC for
MIPS, but Loongson-3 doesn't use GIC. Furthermore, IPI emulation in QEMU
is too expensive for performance (because of too many context switches
between Host and Guest). With current solution, the IPI delay may even
cause RCU stall warnings in a multi-core Guest. So, we design a faster
solution that emulate IPI interrupt in kernel (only used by Loongson-3
now).


Huacai


>
> Yours,
> Aleksandar
>
>
> > Signed-off-by: Huacai Chen <chenhc@lemote.com>
> > Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> > ---
> >  arch/mips/include/asm/kvm_host.h |  32 ++++++
> >  arch/mips/kvm/Makefile           |   3 +
> >  arch/mips/kvm/emulate.c          |  23 ++++-
> >  arch/mips/kvm/loongson_ipi.c     | 214 +++++++++++++++++++++++++++++++++++++++
> >  arch/mips/kvm/mips.c             |   6 ++
> >  5 files changed, 277 insertions(+), 1 deletion(-)
> >  create mode 100644 arch/mips/kvm/loongson_ipi.c
> >
> > diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> > index a7758c0..f165902 100644
> > --- a/arch/mips/include/asm/kvm_host.h
> > +++ b/arch/mips/include/asm/kvm_host.h
> > @@ -23,6 +23,8 @@
> >  #include <asm/inst.h>
> >  #include <asm/mipsregs.h>
> >
> > +#include <kvm/iodev.h>
> > +
> >  /* MIPS KVM register ids */
> >  #define MIPS_CP0_32(_R, _S)                                    \
> >         (KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
> > @@ -181,11 +183,39 @@ struct kvm_vcpu_stat {
> >  struct kvm_arch_memory_slot {
> >  };
> >
> > +#ifdef CONFIG_CPU_LOONGSON64
> > +struct ipi_state {
> > +       uint32_t status;
> > +       uint32_t en;
> > +       uint32_t set;
> > +       uint32_t clear;
> > +       uint64_t buf[4];
> > +};
> > +
> > +struct loongson_kvm_ipi;
> > +
> > +struct ipi_io_device {
> > +       int node_id;
> > +       struct loongson_kvm_ipi *ipi;
> > +       struct kvm_io_device device;
> > +};
> > +
> > +struct loongson_kvm_ipi {
> > +       spinlock_t lock;
> > +       struct kvm *kvm;
> > +       struct ipi_state ipistate[16];
> > +       struct ipi_io_device dev_ipi[4];
> > +};
> > +#endif
> > +
> >  struct kvm_arch {
> >         /* Guest physical mm */
> >         struct mm_struct gpa_mm;
> >         /* Mask of CPUs needing GPA ASID flush */
> >         cpumask_t asid_flush_mask;
> > +#ifdef CONFIG_CPU_LOONGSON64
> > +       struct loongson_kvm_ipi ipi;
> > +#endif
> >  };
> >
> >  #define N_MIPS_COPROC_REGS     32
> > @@ -1133,6 +1163,8 @@ extern int kvm_mips_trans_mtc0(union mips_instruction inst, u32 *opc,
> >  /* Misc */
> >  extern void kvm_mips_dump_stats(struct kvm_vcpu *vcpu);
> >  extern unsigned long kvm_mips_get_ramsize(struct kvm *kvm);
> > +extern int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
> > +                            struct kvm_mips_interrupt *irq);
> >
> >  static inline void kvm_arch_hardware_unsetup(void) {}
> >  static inline void kvm_arch_sync_events(struct kvm *kvm) {}
> > diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> > index 0a3cef6..506c4ac 100644
> > --- a/arch/mips/kvm/Makefile
> > +++ b/arch/mips/kvm/Makefile
> > @@ -13,6 +13,9 @@ kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
> >             fpu.o
> >  kvm-objs += hypcall.o
> >  kvm-objs += mmu.o
> > +ifdef CONFIG_CPU_LOONGSON64
> > +kvm-objs += loongson_ipi.o
> > +endif
> >
> >  ifdef CONFIG_KVM_MIPS_VZ
> >  kvm-objs               += vz.o
> > diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
> > index 754094b..3946499 100644
> > --- a/arch/mips/kvm/emulate.c
> > +++ b/arch/mips/kvm/emulate.c
> > @@ -1600,6 +1600,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
> >                                              struct kvm_run *run,
> >                                              struct kvm_vcpu *vcpu)
> >  {
> > +       int r;
> >         enum emulation_result er;
> >         u32 rt;
> >         void *data = run->mmio.data;
> > @@ -1666,9 +1667,18 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
> >                 goto out_fail;
> >         }
> >
> > -       run->mmio.is_write = 1;
> >         vcpu->mmio_needed = 1;
> > +       run->mmio.is_write = 1;
> >         vcpu->mmio_is_write = 1;
> > +
> > +       r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS,
> > +                       run->mmio.phys_addr, run->mmio.len, data);
> > +
> > +       if (!r) {
> > +               vcpu->mmio_needed = 0;
> > +               return EMULATE_DONE;
> > +       }
> > +
> >         return EMULATE_DO_MMIO;
> >
> >  out_fail:
> > @@ -1681,6 +1691,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
> >                                             u32 cause, struct kvm_run *run,
> >                                             struct kvm_vcpu *vcpu)
> >  {
> > +       int r;
> >         enum emulation_result er;
> >         unsigned long curr_pc;
> >         u32 op, rt;
> > @@ -1745,6 +1756,16 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
> >
> >         run->mmio.is_write = 0;
> >         vcpu->mmio_is_write = 0;
> > +
> > +       r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS,
> > +                       run->mmio.phys_addr, run->mmio.len, run->mmio.data);
> > +
> > +       if (!r) {
> > +               kvm_mips_complete_mmio_load(vcpu, run);
> > +               vcpu->mmio_needed = 0;
> > +               return EMULATE_DONE;
> > +       }
> > +
> >         return EMULATE_DO_MMIO;
> >  }
> >
> > diff --git a/arch/mips/kvm/loongson_ipi.c b/arch/mips/kvm/loongson_ipi.c
> > new file mode 100644
> > index 00000000..3681fc8
> > --- /dev/null
> > +++ b/arch/mips/kvm/loongson_ipi.c
> > @@ -0,0 +1,214 @@
> > +// SPDX-License-Identifier: GPL-2.0-or-later
> > +/*
> > + * Loongson-3 Virtual IPI interrupt support.
> > + *
> > + * Copyright (C) 2019  Loongson Technologies, Inc.  All rights reserved.
> > + *
> > + * Authors: Chen Zhu <zhuchen@loongson.cn>
> > + * Authors: Huacai Chen <chenhc@lemote.com>
> > + */
> > +
> > +#include <linux/kvm_host.h>
> > +
> > +#define IPI_BASE            0x3ff01000ULL
> > +
> > +#define CORE0_STATUS_OFF       0x000
> > +#define CORE0_EN_OFF           0x004
> > +#define CORE0_SET_OFF          0x008
> > +#define CORE0_CLEAR_OFF        0x00c
> > +#define CORE0_BUF_20           0x020
> > +#define CORE0_BUF_28           0x028
> > +#define CORE0_BUF_30           0x030
> > +#define CORE0_BUF_38           0x038
> > +
> > +#define CORE1_STATUS_OFF       0x100
> > +#define CORE1_EN_OFF           0x104
> > +#define CORE1_SET_OFF          0x108
> > +#define CORE1_CLEAR_OFF        0x10c
> > +#define CORE1_BUF_20           0x120
> > +#define CORE1_BUF_28           0x128
> > +#define CORE1_BUF_30           0x130
> > +#define CORE1_BUF_38           0x138
> > +
> > +#define CORE2_STATUS_OFF       0x200
> > +#define CORE2_EN_OFF           0x204
> > +#define CORE2_SET_OFF          0x208
> > +#define CORE2_CLEAR_OFF        0x20c
> > +#define CORE2_BUF_20           0x220
> > +#define CORE2_BUF_28           0x228
> > +#define CORE2_BUF_30           0x230
> > +#define CORE2_BUF_38           0x238
> > +
> > +#define CORE3_STATUS_OFF       0x300
> > +#define CORE3_EN_OFF           0x304
> > +#define CORE3_SET_OFF          0x308
> > +#define CORE3_CLEAR_OFF        0x30c
> > +#define CORE3_BUF_20           0x320
> > +#define CORE3_BUF_28           0x328
> > +#define CORE3_BUF_30           0x330
> > +#define CORE3_BUF_38           0x338
> > +
> > +static int loongson_vipi_read(struct loongson_kvm_ipi *ipi,
> > +                               gpa_t addr, int len, void *val)
> > +{
> > +       uint32_t core = (addr >> 8) & 3;
> > +       uint32_t node = (addr >> 44) & 3;
> > +       uint32_t id = core + node * 4;
> > +       uint64_t offset = addr & 0xff;
> > +       void *pbuf;
> > +       struct ipi_state *s = &(ipi->ipistate[id]);
> > +
> > +       BUG_ON(offset & (len - 1));
> > +
> > +       switch (offset) {
> > +       case CORE0_STATUS_OFF:
> > +               *(uint64_t *)val = s->status;
> > +               break;
> > +
> > +       case CORE0_EN_OFF:
> > +               *(uint64_t *)val = s->en;
> > +               break;
> > +
> > +       case CORE0_SET_OFF:
> > +               *(uint64_t *)val = 0;
> > +               break;
> > +
> > +       case CORE0_CLEAR_OFF:
> > +               *(uint64_t *)val = 0;
> > +               break;
> > +
> > +       case CORE0_BUF_20 ... CORE0_BUF_38:
> > +               pbuf = (void *)s->buf + (offset - 0x20);
> > +               if (len == 8)
> > +                       *(uint64_t *)val = *(uint64_t *)pbuf;
> > +               else /* Assume len == 4 */
> > +                       *(uint32_t *)val = *(uint32_t *)pbuf;
> > +               break;
> > +
> > +       default:
> > +               pr_notice("%s with unknown addr %llx\n", __func__, addr);
> > +               break;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static int loongson_vipi_write(struct loongson_kvm_ipi *ipi,
> > +                               gpa_t addr, int len, const void *val)
> > +{
> > +       uint32_t core = (addr >> 8) & 3;
> > +       uint32_t node = (addr >> 44) & 3;
> > +       uint32_t id = core + node * 4;
> > +       uint64_t data, offset = addr & 0xff;
> > +       void *pbuf;
> > +       struct kvm *kvm = ipi->kvm;
> > +       struct kvm_mips_interrupt irq;
> > +       struct ipi_state *s = &(ipi->ipistate[id]);
> > +
> > +       data = *(uint64_t *)val;
> > +       BUG_ON(offset & (len - 1));
> > +
> > +       switch (offset) {
> > +       case CORE0_STATUS_OFF:
> > +               break;
> > +
> > +       case CORE0_EN_OFF:
> > +               s->en = data;
> > +               break;
> > +
> > +       case CORE0_SET_OFF:
> > +               s->status |= data;
> > +               irq.cpu = id;
> > +               irq.irq = 6;
> > +               kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
> > +               break;
> > +
> > +       case CORE0_CLEAR_OFF:
> > +               s->status &= ~data;
> > +               if (!s->status) {
> > +                       irq.cpu = id;
> > +                       irq.irq = -6;
> > +                       kvm_vcpu_ioctl_interrupt(kvm->vcpus[id], &irq);
> > +               }
> > +               break;
> > +
> > +       case CORE0_BUF_20 ... CORE0_BUF_38:
> > +               pbuf = (void *)s->buf + (offset - 0x20);
> > +               if (len == 8)
> > +                       *(uint64_t *)pbuf = (uint64_t)data;
> > +               else /* Assume len == 4 */
> > +                       *(uint32_t *)pbuf = (uint32_t)data;
> > +               break;
> > +
> > +       default:
> > +               pr_notice("%s with unknown addr %llx\n", __func__, addr);
> > +               break;
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static int kvm_ipi_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> > +                       gpa_t addr, int len, void *val)
> > +{
> > +       unsigned long flags;
> > +       struct loongson_kvm_ipi *ipi;
> > +       struct ipi_io_device *ipi_device;
> > +
> > +       ipi_device = container_of(dev, struct ipi_io_device, device);
> > +       ipi = ipi_device->ipi;
> > +
> > +       spin_lock_irqsave(&ipi->lock, flags);
> > +       loongson_vipi_read(ipi, addr, len, val);
> > +       spin_unlock_irqrestore(&ipi->lock, flags);
> > +
> > +       return 0;
> > +}
> > +
> > +static int kvm_ipi_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> > +                       gpa_t addr, int len, const void *val)
> > +{
> > +       unsigned long flags;
> > +       struct loongson_kvm_ipi *ipi;
> > +       struct ipi_io_device *ipi_device;
> > +
> > +       ipi_device = container_of(dev, struct ipi_io_device, device);
> > +       ipi = ipi_device->ipi;
> > +
> > +       spin_lock_irqsave(&ipi->lock, flags);
> > +       loongson_vipi_write(ipi, addr, len, val);
> > +       spin_unlock_irqrestore(&ipi->lock, flags);
> > +
> > +       return 0;
> > +}
> > +
> > +static const struct kvm_io_device_ops kvm_ipi_ops = {
> > +       .read     = kvm_ipi_read,
> > +       .write    = kvm_ipi_write,
> > +};
> > +
> > +void kvm_init_loongson_ipi(struct kvm *kvm)
> > +{
> > +       int i;
> > +       unsigned long addr;
> > +       struct loongson_kvm_ipi *s;
> > +       struct kvm_io_device *device;
> > +
> > +       s = &kvm->arch.ipi;
> > +       s->kvm = kvm;
> > +       spin_lock_init(&s->lock);
> > +
> > +       /*
> > +        * Initialize IPI device
> > +        */
> > +       for (i = 0; i < 4; i++) {
> > +               device = &s->dev_ipi[i].device;
> > +               kvm_iodevice_init(device, &kvm_ipi_ops);
> > +               addr = (((unsigned long)i) << 44) + IPI_BASE;
> > +               mutex_lock(&kvm->slots_lock);
> > +               kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, addr, 0x400, device);
> > +               mutex_unlock(&kvm->slots_lock);
> > +               s->dev_ipi[i].ipi = s;
> > +               s->dev_ipi[i].node_id = i;
> > +       }
> > +}
> > diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> > index 5ca122c..ed989ef 100644
> > --- a/arch/mips/kvm/mips.c
> > +++ b/arch/mips/kvm/mips.c
> > @@ -128,6 +128,8 @@ int kvm_arch_check_processor_compat(void *opaque)
> >         return 0;
> >  }
> >
> > +extern void kvm_init_loongson_ipi(struct kvm *kvm);
> > +
> >  int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> >  {
> >         switch (type) {
> > @@ -147,6 +149,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> >         if (!kvm->arch.gpa_mm.pgd)
> >                 return -ENOMEM;
> >
> > +#ifdef CONFIG_CPU_LOONGSON64
> > +       kvm_init_loongson_ipi(kvm);
> > +#endif
> > +
> >         return 0;
> >  }
> >
> > --
> > 2.7.0
> >

[PATCH V3 11/14] KVM: MIPS: Add CPUCFG emulation for Loongson-3

[Huacai Chen]

Loongson-3 overrides lwc2 instructions to implement CPUCFG and CSR
read/write functions. These instructions all cause guest exit so CSR
doesn't benifit KVM guest (and there are always legacy methods to
provide the same functions as CSR). So, we only emulate CPUCFG and let
it return a reduced feature list (which means the virtual CPU doesn't
have any other advanced features, including CSR) in KVM.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h  |  3 ++
 arch/mips/include/uapi/asm/inst.h | 11 ++++++
 arch/mips/kvm/mips.c              |  3 ++
 arch/mips/kvm/vz.c                | 75 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 92 insertions(+)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index f165902..3fd2f1c 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -173,6 +173,9 @@ struct kvm_vcpu_stat {
 	u64 vz_ghfc_exits;
 	u64 vz_gpa_exits;
 	u64 vz_resvd_exits;
+#ifdef CONFIG_CPU_LOONGSON64
+	u64 vz_cpucfg_exits;
+#endif
 #endif
 	u64 halt_successful_poll;
 	u64 halt_attempted_poll;
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 98f97c8..43d1faa 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -1012,6 +1012,16 @@ struct loongson3_lsdc2_format {	/* Loongson-3 overridden ldc2/sdc2 Load/Store fo
 	;))))))
 };
 
+struct loongson3_lscsr_format {	/* Loongson-3 CPUCFG&CSR read/write format */
+	__BITFIELD_FIELD(unsigned int opcode : 6,
+	__BITFIELD_FIELD(unsigned int rs : 5,
+	__BITFIELD_FIELD(unsigned int fr : 5,
+	__BITFIELD_FIELD(unsigned int rd : 5,
+	__BITFIELD_FIELD(unsigned int fd : 5,
+	__BITFIELD_FIELD(unsigned int func : 6,
+	;))))))
+};
+
 /*
  * MIPS16e instruction formats (16-bit length)
  */
@@ -1114,6 +1124,7 @@ union mips_instruction {
 	struct mm16_r5_format mm16_r5_format;
 	struct loongson3_lswc2_format loongson3_lswc2_format;
 	struct loongson3_lsdc2_format loongson3_lsdc2_format;
+	struct loongson3_lscsr_format loongson3_lscsr_format;
 };
 
 union mips16e_instruction {
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index ed989ef..9362769 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -68,6 +68,9 @@ struct kvm_stats_debugfs_item debugfs_entries[] = {
 	{ "vz_ghfc",	  VCPU_STAT(vz_ghfc_exits),	 KVM_STAT_VCPU },
 	{ "vz_gpa",	  VCPU_STAT(vz_gpa_exits),	 KVM_STAT_VCPU },
 	{ "vz_resvd",	  VCPU_STAT(vz_resvd_exits),	 KVM_STAT_VCPU },
+#ifdef CONFIG_CPU_LOONGSON64
+	{ "vz_cpucfg",	  VCPU_STAT(vz_cpucfg_exits),	 KVM_STAT_VCPU },
+#endif
 #endif
 	{ "halt_successful_poll", VCPU_STAT(halt_successful_poll), KVM_STAT_VCPU },
 	{ "halt_attempted_poll", VCPU_STAT(halt_attempted_poll), KVM_STAT_VCPU },
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 63d5b35..e5c751b 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -29,6 +29,7 @@
 #include <linux/kvm_host.h>
 
 #include "interrupt.h"
+#include "loongson_regs.h"
 
 #include "trace.h"
 
@@ -1092,6 +1093,75 @@ static enum emulation_result kvm_vz_gpsi_cache(union mips_instruction inst,
 	return EMULATE_FAIL;
 }
 
+#ifdef CONFIG_CPU_LOONGSON64
+static enum emulation_result kvm_vz_gpsi_lwc2(union mips_instruction inst,
+					      u32 *opc, u32 cause,
+					      struct kvm_run *run,
+					      struct kvm_vcpu *vcpu)
+{
+	unsigned int rs, rd;
+	unsigned int hostcfg;
+	unsigned long curr_pc;
+	enum emulation_result er = EMULATE_DONE;
+
+	/*
+	 * Update PC and hold onto current PC in case there is
+	 * an error and we want to rollback the PC
+	 */
+	curr_pc = vcpu->arch.pc;
+	er = update_pc(vcpu, cause);
+	if (er == EMULATE_FAIL)
+		return er;
+
+	rs = inst.loongson3_lscsr_format.rs;
+	rd = inst.loongson3_lscsr_format.rd;
+	switch (inst.loongson3_lscsr_format.fr) {
+	case 0x8:  /* Read CPUCFG */
+		++vcpu->stat.vz_cpucfg_exits;
+		hostcfg = read_cpucfg(vcpu->arch.gprs[rs]);
+
+		switch (vcpu->arch.gprs[rs]) {
+		case LOONGSON_CFG1:
+			hostcfg &= (LOONGSON_CFG1_FP | LOONGSON_CFG1_MMI |
+				    LOONGSON_CFG1_MSA1 | LOONGSON_CFG1_MSA2 |
+				    LOONGSON_CFG1_SFBP);
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		case LOONGSON_CFG2:
+			hostcfg &= (LOONGSON_CFG2_LEXT1 | LOONGSON_CFG2_LEXT2 |
+				    LOONGSON_CFG2_LEXT3 | LOONGSON_CFG2_LSPW);
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		case LOONGSON_CFG3:
+			hostcfg &= LOONGSON_CFG3_LCAMP;
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		default:
+			/* Don't export any other advanced features to guest */
+			vcpu->arch.gprs[rd] = 0;
+			break;
+		}
+
+		break;
+	default:
+		kvm_err("lwc2 emulate not impl %d rs %lx @%lx\n",
+			inst.loongson3_lscsr_format.fr, vcpu->arch.gprs[rs], curr_pc);
+		er = EMULATE_FAIL;
+		break;
+	}
+
+	/* Rollback PC only if emulation was unsuccessful */
+	if (er == EMULATE_FAIL) {
+		kvm_err("[%#lx]%s: unsupported lwc2 instruction 0x%08x 0x%08x\n",
+			curr_pc, __func__, inst.word, inst.loongson3_lscsr_format.fr);
+
+		vcpu->arch.pc = curr_pc;
+	}
+
+	return er;
+}
+#endif
+
 static enum emulation_result kvm_trap_vz_handle_gpsi(u32 cause, u32 *opc,
 						     struct kvm_vcpu *vcpu)
 {
@@ -1121,6 +1191,11 @@ static enum emulation_result kvm_trap_vz_handle_gpsi(u32 cause, u32 *opc,
 		er = kvm_vz_gpsi_cache(inst, opc, cause, run, vcpu);
 		break;
 #endif
+#ifdef CONFIG_CPU_LOONGSON64
+	case lwc2_op:
+		er = kvm_vz_gpsi_lwc2(inst, opc, cause, run, vcpu);
+		break;
+#endif
 	case spec3_op:
 		switch (inst.spec3_format.func) {
 #ifdef CONFIG_CPU_MIPSR6
-- 
2.7.0

[PATCH V3 11/14] KVM: MIPS: Add CPUCFG emulation for Loongson-3

[Huacai Chen]

Loongson-3 overrides lwc2 instructions to implement CPUCFG and CSR
read/write functions. These instructions all cause guest exit so CSR
doesn't benifit KVM guest (and there are always legacy methods to
provide the same functions as CSR). So, we only emulate CPUCFG and let
it return a reduced feature list (which means the virtual CPU doesn't
have any other advanced features, including CSR) in KVM.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h  |  3 ++
 arch/mips/include/uapi/asm/inst.h | 11 ++++++
 arch/mips/kvm/mips.c              |  3 ++
 arch/mips/kvm/vz.c                | 75 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 92 insertions(+)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index f165902..3fd2f1c 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -173,6 +173,9 @@ struct kvm_vcpu_stat {
 	u64 vz_ghfc_exits;
 	u64 vz_gpa_exits;
 	u64 vz_resvd_exits;
+#ifdef CONFIG_CPU_LOONGSON64
+	u64 vz_cpucfg_exits;
+#endif
 #endif
 	u64 halt_successful_poll;
 	u64 halt_attempted_poll;
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 98f97c8..43d1faa 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -1012,6 +1012,16 @@ struct loongson3_lsdc2_format {	/* Loongson-3 overridden ldc2/sdc2 Load/Store fo
 	;))))))
 };
 
+struct loongson3_lscsr_format {	/* Loongson-3 CPUCFG&CSR read/write format */
+	__BITFIELD_FIELD(unsigned int opcode : 6,
+	__BITFIELD_FIELD(unsigned int rs : 5,
+	__BITFIELD_FIELD(unsigned int fr : 5,
+	__BITFIELD_FIELD(unsigned int rd : 5,
+	__BITFIELD_FIELD(unsigned int fd : 5,
+	__BITFIELD_FIELD(unsigned int func : 6,
+	;))))))
+};
+
 /*
  * MIPS16e instruction formats (16-bit length)
  */
@@ -1114,6 +1124,7 @@ union mips_instruction {
 	struct mm16_r5_format mm16_r5_format;
 	struct loongson3_lswc2_format loongson3_lswc2_format;
 	struct loongson3_lsdc2_format loongson3_lsdc2_format;
+	struct loongson3_lscsr_format loongson3_lscsr_format;
 };
 
 union mips16e_instruction {
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index ed989ef..9362769 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -68,6 +68,9 @@ struct kvm_stats_debugfs_item debugfs_entries[] = {
 	{ "vz_ghfc",	  VCPU_STAT(vz_ghfc_exits),	 KVM_STAT_VCPU },
 	{ "vz_gpa",	  VCPU_STAT(vz_gpa_exits),	 KVM_STAT_VCPU },
 	{ "vz_resvd",	  VCPU_STAT(vz_resvd_exits),	 KVM_STAT_VCPU },
+#ifdef CONFIG_CPU_LOONGSON64
+	{ "vz_cpucfg",	  VCPU_STAT(vz_cpucfg_exits),	 KVM_STAT_VCPU },
+#endif
 #endif
 	{ "halt_successful_poll", VCPU_STAT(halt_successful_poll), KVM_STAT_VCPU },
 	{ "halt_attempted_poll", VCPU_STAT(halt_attempted_poll), KVM_STAT_VCPU },
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 63d5b35..e5c751b 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -29,6 +29,7 @@
 #include <linux/kvm_host.h>
 
 #include "interrupt.h"
+#include "loongson_regs.h"
 
 #include "trace.h"
 
@@ -1092,6 +1093,75 @@ static enum emulation_result kvm_vz_gpsi_cache(union mips_instruction inst,
 	return EMULATE_FAIL;
 }
 
+#ifdef CONFIG_CPU_LOONGSON64
+static enum emulation_result kvm_vz_gpsi_lwc2(union mips_instruction inst,
+					      u32 *opc, u32 cause,
+					      struct kvm_run *run,
+					      struct kvm_vcpu *vcpu)
+{
+	unsigned int rs, rd;
+	unsigned int hostcfg;
+	unsigned long curr_pc;
+	enum emulation_result er = EMULATE_DONE;
+
+	/*
+	 * Update PC and hold onto current PC in case there is
+	 * an error and we want to rollback the PC
+	 */
+	curr_pc = vcpu->arch.pc;
+	er = update_pc(vcpu, cause);
+	if (er == EMULATE_FAIL)
+		return er;
+
+	rs = inst.loongson3_lscsr_format.rs;
+	rd = inst.loongson3_lscsr_format.rd;
+	switch (inst.loongson3_lscsr_format.fr) {
+	case 0x8:  /* Read CPUCFG */
+		++vcpu->stat.vz_cpucfg_exits;
+		hostcfg = read_cpucfg(vcpu->arch.gprs[rs]);
+
+		switch (vcpu->arch.gprs[rs]) {
+		case LOONGSON_CFG1:
+			hostcfg &= (LOONGSON_CFG1_FP | LOONGSON_CFG1_MMI |
+				    LOONGSON_CFG1_MSA1 | LOONGSON_CFG1_MSA2 |
+				    LOONGSON_CFG1_SFBP);
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		case LOONGSON_CFG2:
+			hostcfg &= (LOONGSON_CFG2_LEXT1 | LOONGSON_CFG2_LEXT2 |
+				    LOONGSON_CFG2_LEXT3 | LOONGSON_CFG2_LSPW);
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		case LOONGSON_CFG3:
+			hostcfg &= LOONGSON_CFG3_LCAMP;
+			vcpu->arch.gprs[rd] = hostcfg;
+			break;
+		default:
+			/* Don't export any other advanced features to guest */
+			vcpu->arch.gprs[rd] = 0;
+			break;
+		}
+
+		break;
+	default:
+		kvm_err("lwc2 emulate not impl %d rs %lx @%lx\n",
+			inst.loongson3_lscsr_format.fr, vcpu->arch.gprs[rs], curr_pc);
+		er = EMULATE_FAIL;
+		break;
+	}
+
+	/* Rollback PC only if emulation was unsuccessful */
+	if (er == EMULATE_FAIL) {
+		kvm_err("[%#lx]%s: unsupported lwc2 instruction 0x%08x 0x%08x\n",
+			curr_pc, __func__, inst.word, inst.loongson3_lscsr_format.fr);
+
+		vcpu->arch.pc = curr_pc;
+	}
+
+	return er;
+}
+#endif
+
 static enum emulation_result kvm_trap_vz_handle_gpsi(u32 cause, u32 *opc,
 						     struct kvm_vcpu *vcpu)
 {
@@ -1121,6 +1191,11 @@ static enum emulation_result kvm_trap_vz_handle_gpsi(u32 cause, u32 *opc,
 		er = kvm_vz_gpsi_cache(inst, opc, cause, run, vcpu);
 		break;
 #endif
+#ifdef CONFIG_CPU_LOONGSON64
+	case lwc2_op:
+		er = kvm_vz_gpsi_lwc2(inst, opc, cause, run, vcpu);
+		break;
+#endif
 	case spec3_op:
 		switch (inst.spec3_format.func) {
 #ifdef CONFIG_CPU_MIPSR6
-- 
2.7.0

[PATCH V3 12/14] KVM: MIPS: Add CONFIG6 and DIAG registers emulation

[Huacai Chen]

Loongson-3 has CONFIG6 and DIAG registers which need to be emulate.
CONFIG6 is mostly used to enable/disable FTLB and SFB, while DIAG is
mostly used to flush BTB, ITLB, DTLB, VTLB and FTLB.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h |  7 +++++
 arch/mips/include/asm/mipsregs.h |  7 +++++
 arch/mips/kvm/tlb.c              | 41 ++++++++++++++++++++++++++
 arch/mips/kvm/vz.c               | 62 +++++++++++++++++++++++++++++++++++++++-
 4 files changed, 116 insertions(+), 1 deletion(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 3fd2f1c..30b5e33 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -68,9 +68,11 @@
 #define KVM_REG_MIPS_CP0_CONFIG3	MIPS_CP0_32(16, 3)
 #define KVM_REG_MIPS_CP0_CONFIG4	MIPS_CP0_32(16, 4)
 #define KVM_REG_MIPS_CP0_CONFIG5	MIPS_CP0_32(16, 5)
+#define KVM_REG_MIPS_CP0_CONFIG6	MIPS_CP0_32(16, 6)
 #define KVM_REG_MIPS_CP0_CONFIG7	MIPS_CP0_32(16, 7)
 #define KVM_REG_MIPS_CP0_MAARI		MIPS_CP0_64(17, 2)
 #define KVM_REG_MIPS_CP0_XCONTEXT	MIPS_CP0_64(20, 0)
+#define KVM_REG_MIPS_CP0_DIAG		MIPS_CP0_32(22, 0)
 #define KVM_REG_MIPS_CP0_ERROREPC	MIPS_CP0_64(30, 0)
 #define KVM_REG_MIPS_CP0_KSCRATCH1	MIPS_CP0_64(31, 2)
 #define KVM_REG_MIPS_CP0_KSCRATCH2	MIPS_CP0_64(31, 3)
@@ -256,6 +258,7 @@ struct mips_coproc {
 #define MIPS_CP0_WATCH_LO	18
 #define MIPS_CP0_WATCH_HI	19
 #define MIPS_CP0_TLB_XCONTEXT	20
+#define MIPS_CP0_DIAG		22
 #define MIPS_CP0_ECC		26
 #define MIPS_CP0_CACHE_ERR	27
 #define MIPS_CP0_TAG_LO		28
@@ -927,6 +930,10 @@ void kvm_vz_save_guesttlb(struct kvm_mips_tlb *buf, unsigned int index,
 			  unsigned int count);
 void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
 			  unsigned int count);
+#ifdef CONFIG_CPU_LOONGSON64
+void kvm_loongson_clear_guest_vtlb(void);
+void kvm_loongson_clear_guest_ftlb(void);
+#endif
 #endif
 
 void kvm_mips_suspend_mm(int cpu);
diff --git a/arch/mips/include/asm/mipsregs.h b/arch/mips/include/asm/mipsregs.h
index 796fe47..ce40fbf 100644
--- a/arch/mips/include/asm/mipsregs.h
+++ b/arch/mips/include/asm/mipsregs.h
@@ -674,6 +674,9 @@
 #define MIPS_CONF5_CV		(_ULCAST_(1) << 29)
 #define MIPS_CONF5_K		(_ULCAST_(1) << 30)
 
+#define MIPS_CONF6_INTIMER	(_ULCAST_(1) << 6)
+#define MIPS_CONF6_EXTIMER	(_ULCAST_(1) << 7)
+#define MIPS_CONF6_SFBEN	(_ULCAST_(1) << 8)
 #define MIPS_CONF6_SYND		(_ULCAST_(1) << 13)
 /* proAptiv FTLB on/off bit */
 #define MIPS_CONF6_FTLBEN	(_ULCAST_(1) << 15)
@@ -993,6 +996,8 @@
 /* Disable Branch Return Cache */
 #define R10K_DIAG_D_BRC		(_ULCAST_(1) << 22)
 
+/* Flush BTB */
+#define LOONGSON_DIAG_BTB	(_ULCAST_(1) << 1)
 /* Flush ITLB */
 #define LOONGSON_DIAG_ITLB	(_ULCAST_(1) << 2)
 /* Flush DTLB */
@@ -2825,7 +2830,9 @@ __BUILD_SET_C0(status)
 __BUILD_SET_C0(cause)
 __BUILD_SET_C0(config)
 __BUILD_SET_C0(config5)
+__BUILD_SET_C0(config6)
 __BUILD_SET_C0(config7)
+__BUILD_SET_C0(diag)
 __BUILD_SET_C0(intcontrol)
 __BUILD_SET_C0(intctl)
 __BUILD_SET_C0(srsmap)
diff --git a/arch/mips/kvm/tlb.c b/arch/mips/kvm/tlb.c
index 7cd9216..1418715 100644
--- a/arch/mips/kvm/tlb.c
+++ b/arch/mips/kvm/tlb.c
@@ -20,6 +20,7 @@
 
 #include <asm/cpu.h>
 #include <asm/bootinfo.h>
+#include <asm/mipsregs.h>
 #include <asm/mmu_context.h>
 #include <asm/pgtable.h>
 #include <asm/cacheflush.h>
@@ -622,6 +623,46 @@ void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
 }
 EXPORT_SYMBOL_GPL(kvm_vz_load_guesttlb);
 
+#ifdef CONFIG_CPU_LOONGSON64
+void kvm_loongson_clear_guest_vtlb(void)
+{
+	int idx = read_gc0_index();
+
+	/* Set root GuestID for root probe and write of guest TLB entry */
+	set_root_gid_to_guest_gid();
+
+	write_gc0_index(0);
+	guest_tlbinvf();
+	write_gc0_index(idx);
+
+	clear_root_gid();
+	set_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
+}
+EXPORT_SYMBOL_GPL(kvm_loongson_clear_guest_vtlb);
+
+void kvm_loongson_clear_guest_ftlb(void)
+{
+	int i;
+	int idx = read_gc0_index();
+
+	/* Set root GuestID for root probe and write of guest TLB entry */
+	set_root_gid_to_guest_gid();
+
+	for (i = current_cpu_data.tlbsizevtlb;
+	     i < (current_cpu_data.tlbsizevtlb +
+		     current_cpu_data.tlbsizeftlbsets);
+	     i++) {
+		write_gc0_index(i);
+		guest_tlbinvf();
+	}
+	write_gc0_index(idx);
+
+	clear_root_gid();
+	set_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
+}
+EXPORT_SYMBOL_GPL(kvm_loongson_clear_guest_ftlb);
+#endif
+
 #endif
 
 /**
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index e5c751b..fc0f8d5 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -127,6 +127,11 @@ static inline unsigned int kvm_vz_config5_guest_wrmask(struct kvm_vcpu *vcpu)
 	return mask;
 }
 
+static inline unsigned int kvm_vz_config6_guest_wrmask(struct kvm_vcpu *vcpu)
+{
+	return MIPS_CONF6_INTIMER | MIPS_CONF6_EXTIMER | MIPS_CONF6_SYND;
+}
+
 /*
  * VZ optionally allows these additional Config bits to be written by root:
  * Config:	M, [MT]
@@ -181,6 +186,12 @@ static inline unsigned int kvm_vz_config5_user_wrmask(struct kvm_vcpu *vcpu)
 	return kvm_vz_config5_guest_wrmask(vcpu) | MIPS_CONF5_MRP;
 }
 
+static inline unsigned int kvm_vz_config6_user_wrmask(struct kvm_vcpu *vcpu)
+{
+	return kvm_vz_config6_guest_wrmask(vcpu) |
+		MIPS_CONF6_SFBEN | MIPS_CONF6_FTLBEN | MIPS_CONF6_FTLBDIS;
+}
+
 static gpa_t kvm_vz_gva_to_gpa_cb(gva_t gva)
 {
 	/* VZ guest has already converted gva to gpa */
@@ -930,7 +941,8 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				    (sel == 2 ||	/* SRSCtl */
 				     sel == 3)) ||	/* SRSMap */
 				   (rd == MIPS_CP0_CONFIG &&
-				    (sel == 7)) ||	/* Config7 */
+				    (sel == 6 ||	/* Config6 */
+				     sel == 7)) ||	/* Config7 */
 				   (rd == MIPS_CP0_LLADDR &&
 				    (sel == 2) &&	/* MAARI */
 				    cpu_guest_has_maar &&
@@ -938,6 +950,11 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				   (rd == MIPS_CP0_ERRCTL &&
 				    (sel == 0))) {	/* ErrCtl */
 				val = cop0->reg[rd][sel];
+#ifdef CONFIG_CPU_LOONGSON64
+			} else if (rd == MIPS_CP0_DIAG &&
+				   (sel == 0)) {	/* Diag */
+				val = cop0->reg[rd][sel];
+#endif
 			} else {
 				val = 0;
 				er = EMULATE_FAIL;
@@ -1000,9 +1017,40 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				   cpu_guest_has_maar &&
 				   !cpu_guest_has_dyn_maar) {
 				kvm_write_maari(vcpu, val);
+			} else if (rd == MIPS_CP0_CONFIG &&
+				   (sel == 6)) {
+				cop0->reg[rd][sel] = (int)val;
 			} else if (rd == MIPS_CP0_ERRCTL &&
 				   (sel == 0)) {	/* ErrCtl */
 				/* ignore the written value */
+#ifdef CONFIG_CPU_LOONGSON64
+			} else if (rd == MIPS_CP0_DIAG &&
+				   (sel == 0)) {	/* Diag */
+				unsigned long flags;
+
+				local_irq_save(flags);
+				if (val & LOONGSON_DIAG_BTB) {
+					/* Flush BTB */
+					set_c0_diag(LOONGSON_DIAG_BTB);
+				}
+				if (val & LOONGSON_DIAG_ITLB) {
+					/* Flush ITLB */
+					set_c0_diag(LOONGSON_DIAG_ITLB);
+				}
+				if (val & LOONGSON_DIAG_DTLB) {
+					/* Flush DTLB */
+					set_c0_diag(LOONGSON_DIAG_DTLB);
+				}
+				if (val & LOONGSON_DIAG_VTLB) {
+					/* Flush VTLB */
+					kvm_loongson_clear_guest_vtlb();
+				}
+				if (val & LOONGSON_DIAG_FTLB) {
+					/* Flush FTLB */
+					kvm_loongson_clear_guest_ftlb();
+				}
+				local_irq_restore(flags);
+#endif
 			} else {
 				er = EMULATE_FAIL;
 			}
@@ -1690,6 +1738,7 @@ static u64 kvm_vz_get_one_regs[] = {
 	KVM_REG_MIPS_CP0_CONFIG3,
 	KVM_REG_MIPS_CP0_CONFIG4,
 	KVM_REG_MIPS_CP0_CONFIG5,
+	KVM_REG_MIPS_CP0_CONFIG6,
 #ifdef CONFIG_64BIT
 	KVM_REG_MIPS_CP0_XCONTEXT,
 #endif
@@ -2017,6 +2066,9 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 			return -EINVAL;
 		*v = read_gc0_config5();
 		break;
+	case KVM_REG_MIPS_CP0_CONFIG6:
+		*v = kvm_read_sw_gc0_config6(cop0);
+		break;
 	case KVM_REG_MIPS_CP0_MAAR(0) ... KVM_REG_MIPS_CP0_MAAR(0x3f):
 		if (!cpu_guest_has_maar || cpu_guest_has_dyn_maar)
 			return -EINVAL;
@@ -2286,6 +2338,14 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 			write_gc0_config5(v);
 		}
 		break;
+	case KVM_REG_MIPS_CP0_CONFIG6:
+		cur = kvm_read_sw_gc0_config6(cop0);
+		change = (cur ^ v) & kvm_vz_config6_user_wrmask(vcpu);
+		if (change) {
+			v = cur ^ change;
+			kvm_write_sw_gc0_config6(cop0, (int)v);
+		}
+		break;
 	case KVM_REG_MIPS_CP0_MAAR(0) ... KVM_REG_MIPS_CP0_MAAR(0x3f):
 		if (!cpu_guest_has_maar || cpu_guest_has_dyn_maar)
 			return -EINVAL;
-- 
2.7.0

[PATCH V3 12/14] KVM: MIPS: Add CONFIG6 and DIAG registers emulation

[Huacai Chen]

Loongson-3 has CONFIG6 and DIAG registers which need to be emulate.
CONFIG6 is mostly used to enable/disable FTLB and SFB, while DIAG is
mostly used to flush BTB, ITLB, DTLB, VTLB and FTLB.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/include/asm/kvm_host.h |  7 +++++
 arch/mips/include/asm/mipsregs.h |  7 +++++
 arch/mips/kvm/tlb.c              | 41 ++++++++++++++++++++++++++
 arch/mips/kvm/vz.c               | 62 +++++++++++++++++++++++++++++++++++++++-
 4 files changed, 116 insertions(+), 1 deletion(-)

diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 3fd2f1c..30b5e33 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -68,9 +68,11 @@
 #define KVM_REG_MIPS_CP0_CONFIG3	MIPS_CP0_32(16, 3)
 #define KVM_REG_MIPS_CP0_CONFIG4	MIPS_CP0_32(16, 4)
 #define KVM_REG_MIPS_CP0_CONFIG5	MIPS_CP0_32(16, 5)
+#define KVM_REG_MIPS_CP0_CONFIG6	MIPS_CP0_32(16, 6)
 #define KVM_REG_MIPS_CP0_CONFIG7	MIPS_CP0_32(16, 7)
 #define KVM_REG_MIPS_CP0_MAARI		MIPS_CP0_64(17, 2)
 #define KVM_REG_MIPS_CP0_XCONTEXT	MIPS_CP0_64(20, 0)
+#define KVM_REG_MIPS_CP0_DIAG		MIPS_CP0_32(22, 0)
 #define KVM_REG_MIPS_CP0_ERROREPC	MIPS_CP0_64(30, 0)
 #define KVM_REG_MIPS_CP0_KSCRATCH1	MIPS_CP0_64(31, 2)
 #define KVM_REG_MIPS_CP0_KSCRATCH2	MIPS_CP0_64(31, 3)
@@ -256,6 +258,7 @@ struct mips_coproc {
 #define MIPS_CP0_WATCH_LO	18
 #define MIPS_CP0_WATCH_HI	19
 #define MIPS_CP0_TLB_XCONTEXT	20
+#define MIPS_CP0_DIAG		22
 #define MIPS_CP0_ECC		26
 #define MIPS_CP0_CACHE_ERR	27
 #define MIPS_CP0_TAG_LO		28
@@ -927,6 +930,10 @@ void kvm_vz_save_guesttlb(struct kvm_mips_tlb *buf, unsigned int index,
 			  unsigned int count);
 void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
 			  unsigned int count);
+#ifdef CONFIG_CPU_LOONGSON64
+void kvm_loongson_clear_guest_vtlb(void);
+void kvm_loongson_clear_guest_ftlb(void);
+#endif
 #endif
 
 void kvm_mips_suspend_mm(int cpu);
diff --git a/arch/mips/include/asm/mipsregs.h b/arch/mips/include/asm/mipsregs.h
index 796fe47..ce40fbf 100644
--- a/arch/mips/include/asm/mipsregs.h
+++ b/arch/mips/include/asm/mipsregs.h
@@ -674,6 +674,9 @@
 #define MIPS_CONF5_CV		(_ULCAST_(1) << 29)
 #define MIPS_CONF5_K		(_ULCAST_(1) << 30)
 
+#define MIPS_CONF6_INTIMER	(_ULCAST_(1) << 6)
+#define MIPS_CONF6_EXTIMER	(_ULCAST_(1) << 7)
+#define MIPS_CONF6_SFBEN	(_ULCAST_(1) << 8)
 #define MIPS_CONF6_SYND		(_ULCAST_(1) << 13)
 /* proAptiv FTLB on/off bit */
 #define MIPS_CONF6_FTLBEN	(_ULCAST_(1) << 15)
@@ -993,6 +996,8 @@
 /* Disable Branch Return Cache */
 #define R10K_DIAG_D_BRC		(_ULCAST_(1) << 22)
 
+/* Flush BTB */
+#define LOONGSON_DIAG_BTB	(_ULCAST_(1) << 1)
 /* Flush ITLB */
 #define LOONGSON_DIAG_ITLB	(_ULCAST_(1) << 2)
 /* Flush DTLB */
@@ -2825,7 +2830,9 @@ __BUILD_SET_C0(status)
 __BUILD_SET_C0(cause)
 __BUILD_SET_C0(config)
 __BUILD_SET_C0(config5)
+__BUILD_SET_C0(config6)
 __BUILD_SET_C0(config7)
+__BUILD_SET_C0(diag)
 __BUILD_SET_C0(intcontrol)
 __BUILD_SET_C0(intctl)
 __BUILD_SET_C0(srsmap)
diff --git a/arch/mips/kvm/tlb.c b/arch/mips/kvm/tlb.c
index 7cd9216..1418715 100644
--- a/arch/mips/kvm/tlb.c
+++ b/arch/mips/kvm/tlb.c
@@ -20,6 +20,7 @@
 
 #include <asm/cpu.h>
 #include <asm/bootinfo.h>
+#include <asm/mipsregs.h>
 #include <asm/mmu_context.h>
 #include <asm/pgtable.h>
 #include <asm/cacheflush.h>
@@ -622,6 +623,46 @@ void kvm_vz_load_guesttlb(const struct kvm_mips_tlb *buf, unsigned int index,
 }
 EXPORT_SYMBOL_GPL(kvm_vz_load_guesttlb);
 
+#ifdef CONFIG_CPU_LOONGSON64
+void kvm_loongson_clear_guest_vtlb(void)
+{
+	int idx = read_gc0_index();
+
+	/* Set root GuestID for root probe and write of guest TLB entry */
+	set_root_gid_to_guest_gid();
+
+	write_gc0_index(0);
+	guest_tlbinvf();
+	write_gc0_index(idx);
+
+	clear_root_gid();
+	set_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
+}
+EXPORT_SYMBOL_GPL(kvm_loongson_clear_guest_vtlb);
+
+void kvm_loongson_clear_guest_ftlb(void)
+{
+	int i;
+	int idx = read_gc0_index();
+
+	/* Set root GuestID for root probe and write of guest TLB entry */
+	set_root_gid_to_guest_gid();
+
+	for (i = current_cpu_data.tlbsizevtlb;
+	     i < (current_cpu_data.tlbsizevtlb +
+		     current_cpu_data.tlbsizeftlbsets);
+	     i++) {
+		write_gc0_index(i);
+		guest_tlbinvf();
+	}
+	write_gc0_index(idx);
+
+	clear_root_gid();
+	set_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
+}
+EXPORT_SYMBOL_GPL(kvm_loongson_clear_guest_ftlb);
+#endif
+
 #endif
 
 /**
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index e5c751b..fc0f8d5 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -127,6 +127,11 @@ static inline unsigned int kvm_vz_config5_guest_wrmask(struct kvm_vcpu *vcpu)
 	return mask;
 }
 
+static inline unsigned int kvm_vz_config6_guest_wrmask(struct kvm_vcpu *vcpu)
+{
+	return MIPS_CONF6_INTIMER | MIPS_CONF6_EXTIMER | MIPS_CONF6_SYND;
+}
+
 /*
  * VZ optionally allows these additional Config bits to be written by root:
  * Config:	M, [MT]
@@ -181,6 +186,12 @@ static inline unsigned int kvm_vz_config5_user_wrmask(struct kvm_vcpu *vcpu)
 	return kvm_vz_config5_guest_wrmask(vcpu) | MIPS_CONF5_MRP;
 }
 
+static inline unsigned int kvm_vz_config6_user_wrmask(struct kvm_vcpu *vcpu)
+{
+	return kvm_vz_config6_guest_wrmask(vcpu) |
+		MIPS_CONF6_SFBEN | MIPS_CONF6_FTLBEN | MIPS_CONF6_FTLBDIS;
+}
+
 static gpa_t kvm_vz_gva_to_gpa_cb(gva_t gva)
 {
 	/* VZ guest has already converted gva to gpa */
@@ -930,7 +941,8 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				    (sel == 2 ||	/* SRSCtl */
 				     sel == 3)) ||	/* SRSMap */
 				   (rd == MIPS_CP0_CONFIG &&
-				    (sel == 7)) ||	/* Config7 */
+				    (sel == 6 ||	/* Config6 */
+				     sel == 7)) ||	/* Config7 */
 				   (rd == MIPS_CP0_LLADDR &&
 				    (sel == 2) &&	/* MAARI */
 				    cpu_guest_has_maar &&
@@ -938,6 +950,11 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				   (rd == MIPS_CP0_ERRCTL &&
 				    (sel == 0))) {	/* ErrCtl */
 				val = cop0->reg[rd][sel];
+#ifdef CONFIG_CPU_LOONGSON64
+			} else if (rd == MIPS_CP0_DIAG &&
+				   (sel == 0)) {	/* Diag */
+				val = cop0->reg[rd][sel];
+#endif
 			} else {
 				val = 0;
 				er = EMULATE_FAIL;
@@ -1000,9 +1017,40 @@ static enum emulation_result kvm_vz_gpsi_cop0(union mips_instruction inst,
 				   cpu_guest_has_maar &&
 				   !cpu_guest_has_dyn_maar) {
 				kvm_write_maari(vcpu, val);
+			} else if (rd == MIPS_CP0_CONFIG &&
+				   (sel == 6)) {
+				cop0->reg[rd][sel] = (int)val;
 			} else if (rd == MIPS_CP0_ERRCTL &&
 				   (sel == 0)) {	/* ErrCtl */
 				/* ignore the written value */
+#ifdef CONFIG_CPU_LOONGSON64
+			} else if (rd == MIPS_CP0_DIAG &&
+				   (sel == 0)) {	/* Diag */
+				unsigned long flags;
+
+				local_irq_save(flags);
+				if (val & LOONGSON_DIAG_BTB) {
+					/* Flush BTB */
+					set_c0_diag(LOONGSON_DIAG_BTB);
+				}
+				if (val & LOONGSON_DIAG_ITLB) {
+					/* Flush ITLB */
+					set_c0_diag(LOONGSON_DIAG_ITLB);
+				}
+				if (val & LOONGSON_DIAG_DTLB) {
+					/* Flush DTLB */
+					set_c0_diag(LOONGSON_DIAG_DTLB);
+				}
+				if (val & LOONGSON_DIAG_VTLB) {
+					/* Flush VTLB */
+					kvm_loongson_clear_guest_vtlb();
+				}
+				if (val & LOONGSON_DIAG_FTLB) {
+					/* Flush FTLB */
+					kvm_loongson_clear_guest_ftlb();
+				}
+				local_irq_restore(flags);
+#endif
 			} else {
 				er = EMULATE_FAIL;
 			}
@@ -1690,6 +1738,7 @@ static u64 kvm_vz_get_one_regs[] = {
 	KVM_REG_MIPS_CP0_CONFIG3,
 	KVM_REG_MIPS_CP0_CONFIG4,
 	KVM_REG_MIPS_CP0_CONFIG5,
+	KVM_REG_MIPS_CP0_CONFIG6,
 #ifdef CONFIG_64BIT
 	KVM_REG_MIPS_CP0_XCONTEXT,
 #endif
@@ -2017,6 +2066,9 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
 			return -EINVAL;
 		*v = read_gc0_config5();
 		break;
+	case KVM_REG_MIPS_CP0_CONFIG6:
+		*v = kvm_read_sw_gc0_config6(cop0);
+		break;
 	case KVM_REG_MIPS_CP0_MAAR(0) ... KVM_REG_MIPS_CP0_MAAR(0x3f):
 		if (!cpu_guest_has_maar || cpu_guest_has_dyn_maar)
 			return -EINVAL;
@@ -2286,6 +2338,14 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
 			write_gc0_config5(v);
 		}
 		break;
+	case KVM_REG_MIPS_CP0_CONFIG6:
+		cur = kvm_read_sw_gc0_config6(cop0);
+		change = (cur ^ v) & kvm_vz_config6_user_wrmask(vcpu);
+		if (change) {
+			v = cur ^ change;
+			kvm_write_sw_gc0_config6(cop0, (int)v);
+		}
+		break;
 	case KVM_REG_MIPS_CP0_MAAR(0) ... KVM_REG_MIPS_CP0_MAAR(0x3f):
 		if (!cpu_guest_has_maar || cpu_guest_has_dyn_maar)
 			return -EINVAL;
-- 
2.7.0

[PATCH V3 13/14] KVM: MIPS: Add more MMIO load/store instructions emulation

[Huacai Chen]

This patch add more MMIO load/store instructions emulation, which can
be observed in QXL and some other device drivers:

1, LWL, LWR, LDW, LDR, SWL, SWR, SDL and SDR for all MIPS;
2, GSLBX, GSLHX, GSLWX, GSLDX, GSSBX, GSSHX, GSSWX and GSSDX for
   Loongson-3.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/emulate.c | 480 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 470 insertions(+), 10 deletions(-)

diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
index 3946499..71316fa 100644
--- a/arch/mips/kvm/emulate.c
+++ b/arch/mips/kvm/emulate.c
@@ -1604,6 +1604,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 	enum emulation_result er;
 	u32 rt;
 	void *data = run->mmio.data;
+	unsigned int imme;
 	unsigned long curr_pc;
 
 	/*
@@ -1661,6 +1662,211 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 			  vcpu->arch.gprs[rt], *(u8 *)data);
 		break;
 
+	case swl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			*(u32 *)data = ((*(u32 *)data) & 0xffffff00) |
+					(vcpu->arch.gprs[rt] >> 24);
+			break;
+		case 1:
+			*(u32 *)data = ((*(u32 *)data) & 0xffff0000) |
+					(vcpu->arch.gprs[rt] >> 16);
+			break;
+		case 2:
+			*(u32 *)data = ((*(u32 *)data) & 0xff000000) |
+					(vcpu->arch.gprs[rt] >> 8);
+			break;
+		case 3:
+			*(u32 *)data = vcpu->arch.gprs[rt];
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SWL: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u32 *)data);
+		break;
+
+	case swr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			*(u32 *)data = vcpu->arch.gprs[rt];
+			break;
+		case 1:
+			*(u32 *)data = ((*(u32 *)data) & 0xff) |
+					(vcpu->arch.gprs[rt] << 8);
+			break;
+		case 2:
+			*(u32 *)data = ((*(u32 *)data) & 0xffff) |
+					(vcpu->arch.gprs[rt] << 16);
+			break;
+		case 3:
+			*(u32 *)data = ((*(u32 *)data) & 0xffffff) |
+					(vcpu->arch.gprs[rt] << 24);
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SWR: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u32 *)data);
+		break;
+
+	case sdl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff00) |
+					((vcpu->arch.gprs[rt] >> 56) & 0xff);
+			break;
+		case 1:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffff0000) |
+					((vcpu->arch.gprs[rt] >> 48) & 0xffff);
+			break;
+		case 2:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffff000000) |
+					((vcpu->arch.gprs[rt] >> 40) & 0xffffff);
+			break;
+		case 3:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffff00000000) |
+					((vcpu->arch.gprs[rt] >> 32) & 0xffffffff);
+			break;
+		case 4:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffff0000000000) |
+					((vcpu->arch.gprs[rt] >> 24) & 0xffffffffff);
+			break;
+		case 5:
+			*(u64 *)data = ((*(u64 *)data) & 0xffff000000000000) |
+					((vcpu->arch.gprs[rt] >> 16) & 0xffffffffffff);
+			break;
+		case 6:
+			*(u64 *)data = ((*(u64 *)data) & 0xff00000000000000) |
+					((vcpu->arch.gprs[rt] >> 8) & 0xffffffffffffff);
+			break;
+		case 7:
+			*(u64 *)data = vcpu->arch.gprs[rt];
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SDL: eaddr: %#lx, gpr: %#lx, data: %llx\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u64 *)data);
+		break;
+
+	case sdr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			*(u64 *)data = vcpu->arch.gprs[rt];
+			break;
+		case 1:
+			*(u64 *)data = ((*(u64 *)data) & 0xff) |
+					(vcpu->arch.gprs[rt] << 8);
+			break;
+		case 2:
+			*(u64 *)data = ((*(u64 *)data) & 0xffff) |
+					(vcpu->arch.gprs[rt] << 16);
+			break;
+		case 3:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffff) |
+					(vcpu->arch.gprs[rt] << 24);
+			break;
+		case 4:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffff) |
+					(vcpu->arch.gprs[rt] << 32);
+			break;
+		case 5:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffff) |
+					(vcpu->arch.gprs[rt] << 40);
+			break;
+		case 6:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffff) |
+					(vcpu->arch.gprs[rt] << 48);
+			break;
+		case 7:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff) |
+					(vcpu->arch.gprs[rt] << 56);
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SDR: eaddr: %#lx, gpr: %#lx, data: %llx\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u64 *)data);
+		break;
+
+#ifdef CONFIG_CPU_LOONGSON64
+	case sdc2_op:
+		rt = inst.loongson3_lsdc2_format.rt;
+		switch (inst.loongson3_lsdc2_format.opcode1) {
+		/*
+		 * Loongson-3 overridden sdc2 instructions.
+		 * opcode1              instruction
+		 *   0x0          gssbx: store 1 bytes from GPR
+		 *   0x1          gsshx: store 2 bytes from GPR
+		 *   0x2          gsswx: store 4 bytes from GPR
+		 *   0x3          gssdx: store 8 bytes from GPR
+		 */
+		case 0x0:
+			run->mmio.len = 1;
+			*(u8 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSBX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u8 *)data);
+			break;
+		case 0x1:
+			run->mmio.len = 2;
+			*(u16 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSSHX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u16 *)data);
+			break;
+		case 0x2:
+			run->mmio.len = 4;
+			*(u32 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSWX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u32 *)data);
+			break;
+		case 0x3:
+			run->mmio.len = 8;
+			*(u64 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSDX: eaddr: %#lx, gpr: %#lx, data: %#llx\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u64 *)data);
+			break;
+		default:
+			kvm_err("Godson Exteneded GS-Store not yet supported (inst=0x%08x)\n",
+				inst.word);
+			break;
+		}
+		break;
+#endif
 	default:
 		kvm_err("Store not yet supported (inst=0x%08x)\n",
 			inst.word);
@@ -1695,6 +1901,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 	enum emulation_result er;
 	unsigned long curr_pc;
 	u32 op, rt;
+	unsigned int imme;
 
 	rt = inst.i_format.rt;
 	op = inst.i_format.opcode;
@@ -1747,6 +1954,162 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 		run->mmio.len = 1;
 		break;
 
+	case lwl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 3;	/* 1 byte */
+			break;
+		case 1:
+			vcpu->mmio_needed = 4;	/* 2 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 5;	/* 3 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 6;	/* 4 bytes */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case lwr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 7;	/* 4 bytes */
+			break;
+		case 1:
+			vcpu->mmio_needed = 8;	/* 3 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 9;	/* 2 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 10;	/* 1 byte */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case ldl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 11;	/* 1 byte */
+			break;
+		case 1:
+			vcpu->mmio_needed = 12;	/* 2 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 13;	/* 3 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 14;	/* 4 bytes */
+			break;
+		case 4:
+			vcpu->mmio_needed = 15;	/* 5 bytes */
+			break;
+		case 5:
+			vcpu->mmio_needed = 16;	/* 6 bytes */
+			break;
+		case 6:
+			vcpu->mmio_needed = 17;	/* 7 bytes */
+			break;
+		case 7:
+			vcpu->mmio_needed = 18;	/* 8 bytes */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case ldr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 19;	/* 8 bytes */
+			break;
+		case 1:
+			vcpu->mmio_needed = 20;	/* 7 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 21;	/* 6 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 22;	/* 5 bytes */
+			break;
+		case 4:
+			vcpu->mmio_needed = 23;	/* 4 bytes */
+			break;
+		case 5:
+			vcpu->mmio_needed = 24;	/* 3 bytes */
+			break;
+		case 6:
+			vcpu->mmio_needed = 25;	/* 2 bytes */
+			break;
+		case 7:
+			vcpu->mmio_needed = 26;	/* 1 byte */
+			break;
+		default:
+			break;
+		}
+		break;
+
+#ifdef CONFIG_CPU_LOONGSON64
+	case ldc2_op:
+		rt = inst.loongson3_lsdc2_format.rt;
+		switch (inst.loongson3_lsdc2_format.opcode1) {
+		/*
+		 * Loongson-3 overridden ldc2 instructions.
+		 * opcode1              instruction
+		 *   0x0          gslbx: store 1 bytes from GPR
+		 *   0x1          gslhx: store 2 bytes from GPR
+		 *   0x2          gslwx: store 4 bytes from GPR
+		 *   0x3          gsldx: store 8 bytes from GPR
+		 */
+		case 0x0:
+			run->mmio.len = 1;
+			vcpu->mmio_needed = 27;	/* signed */
+			break;
+		case 0x1:
+			run->mmio.len = 2;
+			vcpu->mmio_needed = 28;	/* signed */
+			break;
+		case 0x2:
+			run->mmio.len = 4;
+			vcpu->mmio_needed = 29;	/* signed */
+			break;
+		case 0x3:
+			run->mmio.len = 8;
+			vcpu->mmio_needed = 30;	/* signed */
+			break;
+		default:
+			kvm_err("Godson Exteneded GS-Load for float not yet supported (inst=0x%08x)\n",
+				inst.word);
+			break;
+		}
+		break;
+#endif
+
 	default:
 		kvm_err("Load not yet supported (inst=0x%08x)\n",
 			inst.word);
@@ -2612,28 +2975,125 @@ enum emulation_result kvm_mips_complete_mmio_load(struct kvm_vcpu *vcpu,
 
 	switch (run->mmio.len) {
 	case 8:
-		*gpr = *(s64 *)run->mmio.data;
+		switch (vcpu->mmio_needed) {
+		case 11:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xff) << 56);
+			break;
+		case 12:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffff) << 48);
+			break;
+		case 13:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffff) << 40);
+			break;
+		case 14:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffff) << 32);
+			break;
+		case 15:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffff) << 24);
+			break;
+		case 16:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffffff) << 16);
+			break;
+		case 17:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffffffff) << 8);
+			break;
+		case 18:
+		case 19:
+			*gpr = *(s64 *)run->mmio.data;
+			break;
+		case 20:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff00000000000000) |
+				((((*(s64 *)run->mmio.data)) >> 8) & 0xffffffffffffff);
+			break;
+		case 21:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff000000000000) |
+				((((*(s64 *)run->mmio.data)) >> 16) & 0xffffffffffff);
+			break;
+		case 22:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff0000000000) |
+				((((*(s64 *)run->mmio.data)) >> 24) & 0xffffffffff);
+			break;
+		case 23:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff00000000) |
+				((((*(s64 *)run->mmio.data)) >> 32) & 0xffffffff);
+			break;
+		case 24:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff000000) |
+				((((*(s64 *)run->mmio.data)) >> 40) & 0xffffff);
+			break;
+		case 25:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff0000) |
+				((((*(s64 *)run->mmio.data)) >> 48) & 0xffff);
+			break;
+		case 26:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff00) |
+				((((*(s64 *)run->mmio.data)) >> 56) & 0xff);
+			break;
+		default:
+			*gpr = *(s64 *)run->mmio.data;
+		}
 		break;
 
 	case 4:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s32 *)run->mmio.data;
-		else
+		switch (vcpu->mmio_needed) {
+		case 1:
 			*gpr = *(u32 *)run->mmio.data;
+			break;
+		case 2:
+			*gpr = *(s32 *)run->mmio.data;
+			break;
+		case 3:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
+				(((*(s32 *)run->mmio.data) & 0xff) << 24);
+			break;
+		case 4:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
+				(((*(s32 *)run->mmio.data) & 0xffff) << 16);
+			break;
+		case 5:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
+				(((*(s32 *)run->mmio.data) & 0xffffff) << 8);
+			break;
+		case 6:
+		case 7:
+			*gpr = *(s32 *)run->mmio.data;
+			break;
+		case 8:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff000000) |
+				((((*(s32 *)run->mmio.data)) >> 8) & 0xffffff);
+			break;
+		case 9:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff0000) |
+				((((*(s32 *)run->mmio.data)) >> 16) & 0xffff);
+			break;
+		case 10:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff00) |
+				((((*(s32 *)run->mmio.data)) >> 24) & 0xff);
+			break;
+		default:
+			*gpr = *(s32 *)run->mmio.data;
+		}
 		break;
 
 	case 2:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s16 *) run->mmio.data;
-		else
+		if (vcpu->mmio_needed == 1)
 			*gpr = *(u16 *)run->mmio.data;
+		else
+			*gpr = *(s16 *)run->mmio.data;
 
 		break;
 	case 1:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s8 *) run->mmio.data;
+		if (vcpu->mmio_needed == 1)
+			*gpr = *(u8 *)run->mmio.data;
 		else
-			*gpr = *(u8 *) run->mmio.data;
+			*gpr = *(s8 *)run->mmio.data;
 		break;
 	}
 
-- 
2.7.0

[PATCH V3 13/14] KVM: MIPS: Add more MMIO load/store instructions emulation

[Huacai Chen]

This patch add more MMIO load/store instructions emulation, which can
be observed in QXL and some other device drivers:

1, LWL, LWR, LDW, LDR, SWL, SWR, SDL and SDR for all MIPS;
2, GSLBX, GSLHX, GSLWX, GSLDX, GSSBX, GSSHX, GSSWX and GSSDX for
   Loongson-3.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/kvm/emulate.c | 480 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 470 insertions(+), 10 deletions(-)

diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
index 3946499..71316fa 100644
--- a/arch/mips/kvm/emulate.c
+++ b/arch/mips/kvm/emulate.c
@@ -1604,6 +1604,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 	enum emulation_result er;
 	u32 rt;
 	void *data = run->mmio.data;
+	unsigned int imme;
 	unsigned long curr_pc;
 
 	/*
@@ -1661,6 +1662,211 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
 			  vcpu->arch.gprs[rt], *(u8 *)data);
 		break;
 
+	case swl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			*(u32 *)data = ((*(u32 *)data) & 0xffffff00) |
+					(vcpu->arch.gprs[rt] >> 24);
+			break;
+		case 1:
+			*(u32 *)data = ((*(u32 *)data) & 0xffff0000) |
+					(vcpu->arch.gprs[rt] >> 16);
+			break;
+		case 2:
+			*(u32 *)data = ((*(u32 *)data) & 0xff000000) |
+					(vcpu->arch.gprs[rt] >> 8);
+			break;
+		case 3:
+			*(u32 *)data = vcpu->arch.gprs[rt];
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SWL: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u32 *)data);
+		break;
+
+	case swr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			*(u32 *)data = vcpu->arch.gprs[rt];
+			break;
+		case 1:
+			*(u32 *)data = ((*(u32 *)data) & 0xff) |
+					(vcpu->arch.gprs[rt] << 8);
+			break;
+		case 2:
+			*(u32 *)data = ((*(u32 *)data) & 0xffff) |
+					(vcpu->arch.gprs[rt] << 16);
+			break;
+		case 3:
+			*(u32 *)data = ((*(u32 *)data) & 0xffffff) |
+					(vcpu->arch.gprs[rt] << 24);
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SWR: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u32 *)data);
+		break;
+
+	case sdl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff00) |
+					((vcpu->arch.gprs[rt] >> 56) & 0xff);
+			break;
+		case 1:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffff0000) |
+					((vcpu->arch.gprs[rt] >> 48) & 0xffff);
+			break;
+		case 2:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffff000000) |
+					((vcpu->arch.gprs[rt] >> 40) & 0xffffff);
+			break;
+		case 3:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffff00000000) |
+					((vcpu->arch.gprs[rt] >> 32) & 0xffffffff);
+			break;
+		case 4:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffff0000000000) |
+					((vcpu->arch.gprs[rt] >> 24) & 0xffffffffff);
+			break;
+		case 5:
+			*(u64 *)data = ((*(u64 *)data) & 0xffff000000000000) |
+					((vcpu->arch.gprs[rt] >> 16) & 0xffffffffffff);
+			break;
+		case 6:
+			*(u64 *)data = ((*(u64 *)data) & 0xff00000000000000) |
+					((vcpu->arch.gprs[rt] >> 8) & 0xffffffffffffff);
+			break;
+		case 7:
+			*(u64 *)data = vcpu->arch.gprs[rt];
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SDL: eaddr: %#lx, gpr: %#lx, data: %llx\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u64 *)data);
+		break;
+
+	case sdr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			*(u64 *)data = vcpu->arch.gprs[rt];
+			break;
+		case 1:
+			*(u64 *)data = ((*(u64 *)data) & 0xff) |
+					(vcpu->arch.gprs[rt] << 8);
+			break;
+		case 2:
+			*(u64 *)data = ((*(u64 *)data) & 0xffff) |
+					(vcpu->arch.gprs[rt] << 16);
+			break;
+		case 3:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffff) |
+					(vcpu->arch.gprs[rt] << 24);
+			break;
+		case 4:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffff) |
+					(vcpu->arch.gprs[rt] << 32);
+			break;
+		case 5:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffff) |
+					(vcpu->arch.gprs[rt] << 40);
+			break;
+		case 6:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffff) |
+					(vcpu->arch.gprs[rt] << 48);
+			break;
+		case 7:
+			*(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff) |
+					(vcpu->arch.gprs[rt] << 56);
+			break;
+		default:
+			break;
+		}
+
+		kvm_debug("[%#lx] OP_SDR: eaddr: %#lx, gpr: %#lx, data: %llx\n",
+			  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+			  vcpu->arch.gprs[rt], *(u64 *)data);
+		break;
+
+#ifdef CONFIG_CPU_LOONGSON64
+	case sdc2_op:
+		rt = inst.loongson3_lsdc2_format.rt;
+		switch (inst.loongson3_lsdc2_format.opcode1) {
+		/*
+		 * Loongson-3 overridden sdc2 instructions.
+		 * opcode1              instruction
+		 *   0x0          gssbx: store 1 bytes from GPR
+		 *   0x1          gsshx: store 2 bytes from GPR
+		 *   0x2          gsswx: store 4 bytes from GPR
+		 *   0x3          gssdx: store 8 bytes from GPR
+		 */
+		case 0x0:
+			run->mmio.len = 1;
+			*(u8 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSBX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u8 *)data);
+			break;
+		case 0x1:
+			run->mmio.len = 2;
+			*(u16 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSSHX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u16 *)data);
+			break;
+		case 0x2:
+			run->mmio.len = 4;
+			*(u32 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSWX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u32 *)data);
+			break;
+		case 0x3:
+			run->mmio.len = 8;
+			*(u64 *)data = vcpu->arch.gprs[rt];
+
+			kvm_debug("[%#lx] OP_GSSDX: eaddr: %#lx, gpr: %#lx, data: %#llx\n",
+				  vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
+				  vcpu->arch.gprs[rt], *(u64 *)data);
+			break;
+		default:
+			kvm_err("Godson Exteneded GS-Store not yet supported (inst=0x%08x)\n",
+				inst.word);
+			break;
+		}
+		break;
+#endif
 	default:
 		kvm_err("Store not yet supported (inst=0x%08x)\n",
 			inst.word);
@@ -1695,6 +1901,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 	enum emulation_result er;
 	unsigned long curr_pc;
 	u32 op, rt;
+	unsigned int imme;
 
 	rt = inst.i_format.rt;
 	op = inst.i_format.opcode;
@@ -1747,6 +1954,162 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
 		run->mmio.len = 1;
 		break;
 
+	case lwl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 3;	/* 1 byte */
+			break;
+		case 1:
+			vcpu->mmio_needed = 4;	/* 2 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 5;	/* 3 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 6;	/* 4 bytes */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case lwr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x3);
+
+		run->mmio.len = 4;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x3;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 7;	/* 4 bytes */
+			break;
+		case 1:
+			vcpu->mmio_needed = 8;	/* 3 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 9;	/* 2 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 10;	/* 1 byte */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case ldl_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 11;	/* 1 byte */
+			break;
+		case 1:
+			vcpu->mmio_needed = 12;	/* 2 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 13;	/* 3 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 14;	/* 4 bytes */
+			break;
+		case 4:
+			vcpu->mmio_needed = 15;	/* 5 bytes */
+			break;
+		case 5:
+			vcpu->mmio_needed = 16;	/* 6 bytes */
+			break;
+		case 6:
+			vcpu->mmio_needed = 17;	/* 7 bytes */
+			break;
+		case 7:
+			vcpu->mmio_needed = 18;	/* 8 bytes */
+			break;
+		default:
+			break;
+		}
+		break;
+
+	case ldr_op:
+		run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
+					vcpu->arch.host_cp0_badvaddr) & (~0x7);
+
+		run->mmio.len = 8;
+		imme = vcpu->arch.host_cp0_badvaddr & 0x7;
+		switch (imme) {
+		case 0:
+			vcpu->mmio_needed = 19;	/* 8 bytes */
+			break;
+		case 1:
+			vcpu->mmio_needed = 20;	/* 7 bytes */
+			break;
+		case 2:
+			vcpu->mmio_needed = 21;	/* 6 bytes */
+			break;
+		case 3:
+			vcpu->mmio_needed = 22;	/* 5 bytes */
+			break;
+		case 4:
+			vcpu->mmio_needed = 23;	/* 4 bytes */
+			break;
+		case 5:
+			vcpu->mmio_needed = 24;	/* 3 bytes */
+			break;
+		case 6:
+			vcpu->mmio_needed = 25;	/* 2 bytes */
+			break;
+		case 7:
+			vcpu->mmio_needed = 26;	/* 1 byte */
+			break;
+		default:
+			break;
+		}
+		break;
+
+#ifdef CONFIG_CPU_LOONGSON64
+	case ldc2_op:
+		rt = inst.loongson3_lsdc2_format.rt;
+		switch (inst.loongson3_lsdc2_format.opcode1) {
+		/*
+		 * Loongson-3 overridden ldc2 instructions.
+		 * opcode1              instruction
+		 *   0x0          gslbx: store 1 bytes from GPR
+		 *   0x1          gslhx: store 2 bytes from GPR
+		 *   0x2          gslwx: store 4 bytes from GPR
+		 *   0x3          gsldx: store 8 bytes from GPR
+		 */
+		case 0x0:
+			run->mmio.len = 1;
+			vcpu->mmio_needed = 27;	/* signed */
+			break;
+		case 0x1:
+			run->mmio.len = 2;
+			vcpu->mmio_needed = 28;	/* signed */
+			break;
+		case 0x2:
+			run->mmio.len = 4;
+			vcpu->mmio_needed = 29;	/* signed */
+			break;
+		case 0x3:
+			run->mmio.len = 8;
+			vcpu->mmio_needed = 30;	/* signed */
+			break;
+		default:
+			kvm_err("Godson Exteneded GS-Load for float not yet supported (inst=0x%08x)\n",
+				inst.word);
+			break;
+		}
+		break;
+#endif
+
 	default:
 		kvm_err("Load not yet supported (inst=0x%08x)\n",
 			inst.word);
@@ -2612,28 +2975,125 @@ enum emulation_result kvm_mips_complete_mmio_load(struct kvm_vcpu *vcpu,
 
 	switch (run->mmio.len) {
 	case 8:
-		*gpr = *(s64 *)run->mmio.data;
+		switch (vcpu->mmio_needed) {
+		case 11:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xff) << 56);
+			break;
+		case 12:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffff) << 48);
+			break;
+		case 13:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffff) << 40);
+			break;
+		case 14:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffff) << 32);
+			break;
+		case 15:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffff) << 24);
+			break;
+		case 16:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffffff) << 16);
+			break;
+		case 17:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
+				(((*(s64 *)run->mmio.data) & 0xffffffffffffff) << 8);
+			break;
+		case 18:
+		case 19:
+			*gpr = *(s64 *)run->mmio.data;
+			break;
+		case 20:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff00000000000000) |
+				((((*(s64 *)run->mmio.data)) >> 8) & 0xffffffffffffff);
+			break;
+		case 21:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff000000000000) |
+				((((*(s64 *)run->mmio.data)) >> 16) & 0xffffffffffff);
+			break;
+		case 22:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff0000000000) |
+				((((*(s64 *)run->mmio.data)) >> 24) & 0xffffffffff);
+			break;
+		case 23:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff00000000) |
+				((((*(s64 *)run->mmio.data)) >> 32) & 0xffffffff);
+			break;
+		case 24:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff000000) |
+				((((*(s64 *)run->mmio.data)) >> 40) & 0xffffff);
+			break;
+		case 25:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff0000) |
+				((((*(s64 *)run->mmio.data)) >> 48) & 0xffff);
+			break;
+		case 26:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff00) |
+				((((*(s64 *)run->mmio.data)) >> 56) & 0xff);
+			break;
+		default:
+			*gpr = *(s64 *)run->mmio.data;
+		}
 		break;
 
 	case 4:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s32 *)run->mmio.data;
-		else
+		switch (vcpu->mmio_needed) {
+		case 1:
 			*gpr = *(u32 *)run->mmio.data;
+			break;
+		case 2:
+			*gpr = *(s32 *)run->mmio.data;
+			break;
+		case 3:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
+				(((*(s32 *)run->mmio.data) & 0xff) << 24);
+			break;
+		case 4:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
+				(((*(s32 *)run->mmio.data) & 0xffff) << 16);
+			break;
+		case 5:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
+				(((*(s32 *)run->mmio.data) & 0xffffff) << 8);
+			break;
+		case 6:
+		case 7:
+			*gpr = *(s32 *)run->mmio.data;
+			break;
+		case 8:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff000000) |
+				((((*(s32 *)run->mmio.data)) >> 8) & 0xffffff);
+			break;
+		case 9:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff0000) |
+				((((*(s32 *)run->mmio.data)) >> 16) & 0xffff);
+			break;
+		case 10:
+			*gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff00) |
+				((((*(s32 *)run->mmio.data)) >> 24) & 0xff);
+			break;
+		default:
+			*gpr = *(s32 *)run->mmio.data;
+		}
 		break;
 
 	case 2:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s16 *) run->mmio.data;
-		else
+		if (vcpu->mmio_needed == 1)
 			*gpr = *(u16 *)run->mmio.data;
+		else
+			*gpr = *(s16 *)run->mmio.data;
 
 		break;
 	case 1:
-		if (vcpu->mmio_needed == 2)
-			*gpr = *(s8 *) run->mmio.data;
+		if (vcpu->mmio_needed == 1)
+			*gpr = *(u8 *)run->mmio.data;
 		else
-			*gpr = *(u8 *) run->mmio.data;
+			*gpr = *(s8 *)run->mmio.data;
 		break;
 	}
 
-- 
2.7.0

Re: [PATCH V3 13/14] KVM: MIPS: Add more MMIO load/store instructions emulation

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:16 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> This patch add more MMIO load/store instructions emulation, which can
> be observed in QXL and some other device drivers:
>
> 1, LWL, LWR, LDW, LDR, SWL, SWR, SDL and SDR for all MIPS;
> 2, GSLBX, GSLHX, GSLWX, GSLDX, GSSBX, GSSHX, GSSWX and GSSDX for
>    Loongson-3.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/emulate.c | 480 +++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 470 insertions(+), 10 deletions(-)
>
> diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
> index 3946499..71316fa 100644
> --- a/arch/mips/kvm/emulate.c
> +++ b/arch/mips/kvm/emulate.c
> @@ -1604,6 +1604,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
>         enum emulation_result er;
>         u32 rt;
>         void *data = run->mmio.data;
> +       unsigned int imme;
>         unsigned long curr_pc;
>
>         /*
> @@ -1661,6 +1662,211 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
>                           vcpu->arch.gprs[rt], *(u8 *)data);
>                 break;
>
> +       case swl_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x3);
> +               run->mmio.len = 4;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x3;
> +               switch (imme) {
> +               case 0:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xffffff00) |
> +                                       (vcpu->arch.gprs[rt] >> 24);
> +                       break;
> +               case 1:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xffff0000) |
> +                                       (vcpu->arch.gprs[rt] >> 16);
> +                       break;
> +               case 2:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xff000000) |
> +                                       (vcpu->arch.gprs[rt] >> 8);
> +                       break;
> +               case 3:
> +                       *(u32 *)data = vcpu->arch.gprs[rt];
> +                       break;
> +               default:
> +                       break;
> +               }
> +
> +               kvm_debug("[%#lx] OP_SWL: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                         vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                         vcpu->arch.gprs[rt], *(u32 *)data);
> +               break;
> +
> +       case swr_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x3);
> +               run->mmio.len = 4;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x3;
> +               switch (imme) {
> +               case 0:
> +                       *(u32 *)data = vcpu->arch.gprs[rt];
> +                       break;
> +               case 1:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xff) |
> +                                       (vcpu->arch.gprs[rt] << 8);
> +                       break;
> +               case 2:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xffff) |
> +                                       (vcpu->arch.gprs[rt] << 16);
> +                       break;
> +               case 3:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xffffff) |
> +                                       (vcpu->arch.gprs[rt] << 24);
> +                       break;
> +               default:
> +                       break;
> +               }
> +
> +               kvm_debug("[%#lx] OP_SWR: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                         vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                         vcpu->arch.gprs[rt], *(u32 *)data);
> +               break;
> +
> +       case sdl_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x7);
> +
> +               run->mmio.len = 8;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x7;
> +               switch (imme) {
> +               case 0:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff00) |
> +                                       ((vcpu->arch.gprs[rt] >> 56) & 0xff);
> +                       break;
> +               case 1:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffffff0000) |
> +                                       ((vcpu->arch.gprs[rt] >> 48) & 0xffff);
> +                       break;
> +               case 2:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffff000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 40) & 0xffffff);
> +                       break;
> +               case 3:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffff00000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 32) & 0xffffffff);
> +                       break;
> +               case 4:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffff0000000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 24) & 0xffffffffff);
> +                       break;
> +               case 5:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffff000000000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 16) & 0xffffffffffff);
> +                       break;
> +               case 6:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xff00000000000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 8) & 0xffffffffffffff);
> +                       break;
> +               case 7:
> +                       *(u64 *)data = vcpu->arch.gprs[rt];
> +                       break;
> +               default:
> +                       break;
> +               }
> +
> +               kvm_debug("[%#lx] OP_SDL: eaddr: %#lx, gpr: %#lx, data: %llx\n",
> +                         vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                         vcpu->arch.gprs[rt], *(u64 *)data);
> +               break;
> +
> +       case sdr_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x7);
> +
> +               run->mmio.len = 8;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x7;
> +               switch (imme) {
> +               case 0:
> +                       *(u64 *)data = vcpu->arch.gprs[rt];
> +                       break;
> +               case 1:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xff) |
> +                                       (vcpu->arch.gprs[rt] << 8);
> +                       break;
> +               case 2:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffff) |
> +                                       (vcpu->arch.gprs[rt] << 16);
> +                       break;
> +               case 3:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffff) |
> +                                       (vcpu->arch.gprs[rt] << 24);
> +                       break;
> +               case 4:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffff) |
> +                                       (vcpu->arch.gprs[rt] << 32);
> +                       break;
> +               case 5:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffff) |
> +                                       (vcpu->arch.gprs[rt] << 40);
> +                       break;
> +               case 6:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffffff) |
> +                                       (vcpu->arch.gprs[rt] << 48);
> +                       break;
> +               case 7:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff) |
> +                                       (vcpu->arch.gprs[rt] << 56);
> +                       break;
> +               default:
> +                       break;
> +               }
> +
> +               kvm_debug("[%#lx] OP_SDR: eaddr: %#lx, gpr: %#lx, data: %llx\n",
> +                         vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                         vcpu->arch.gprs[rt], *(u64 *)data);
> +               break;
> +
> +#ifdef CONFIG_CPU_LOONGSON64
> +       case sdc2_op:
> +               rt = inst.loongson3_lsdc2_format.rt;
> +               switch (inst.loongson3_lsdc2_format.opcode1) {
> +               /*
> +                * Loongson-3 overridden sdc2 instructions.
> +                * opcode1              instruction
> +                *   0x0          gssbx: store 1 bytes from GPR
> +                *   0x1          gsshx: store 2 bytes from GPR
> +                *   0x2          gsswx: store 4 bytes from GPR
> +                *   0x3          gssdx: store 8 bytes from GPR
> +                */
> +               case 0x0:
> +                       run->mmio.len = 1;
> +                       *(u8 *)data = vcpu->arch.gprs[rt];
> +
> +                       kvm_debug("[%#lx] OP_GSSBX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                                 vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                                 vcpu->arch.gprs[rt], *(u8 *)data);
> +                       break;
> +               case 0x1:
> +                       run->mmio.len = 2;
> +                       *(u16 *)data = vcpu->arch.gprs[rt];
> +
> +                       kvm_debug("[%#lx] OP_GSSSHX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                                 vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                                 vcpu->arch.gprs[rt], *(u16 *)data);
> +                       break;
> +               case 0x2:
> +                       run->mmio.len = 4;
> +                       *(u32 *)data = vcpu->arch.gprs[rt];
> +
> +                       kvm_debug("[%#lx] OP_GSSWX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                                 vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                                 vcpu->arch.gprs[rt], *(u32 *)data);
> +                       break;
> +               case 0x3:
> +                       run->mmio.len = 8;
> +                       *(u64 *)data = vcpu->arch.gprs[rt];
> +
> +                       kvm_debug("[%#lx] OP_GSSDX: eaddr: %#lx, gpr: %#lx, data: %#llx\n",
> +                                 vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                                 vcpu->arch.gprs[rt], *(u64 *)data);
> +                       break;
> +               default:
> +                       kvm_err("Godson Exteneded GS-Store not yet supported (inst=0x%08x)\n",
> +                               inst.word);
> +                       break;
> +               }
> +               break;
> +#endif
>         default:
>                 kvm_err("Store not yet supported (inst=0x%08x)\n",
>                         inst.word);
> @@ -1695,6 +1901,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
>         enum emulation_result er;
>         unsigned long curr_pc;
>         u32 op, rt;
> +       unsigned int imme;
>
>         rt = inst.i_format.rt;
>         op = inst.i_format.opcode;
> @@ -1747,6 +1954,162 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
>                 run->mmio.len = 1;
>                 break;
>
> +       case lwl_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x3);
> +
> +               run->mmio.len = 4;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x3;
> +               switch (imme) {
> +               case 0:
> +                       vcpu->mmio_needed = 3;  /* 1 byte */
> +                       break;
> +               case 1:
> +                       vcpu->mmio_needed = 4;  /* 2 bytes */
> +                       break;
> +               case 2:
> +                       vcpu->mmio_needed = 5;  /* 3 bytes */
> +                       break;
> +               case 3:
> +                       vcpu->mmio_needed = 6;  /* 4 bytes */
> +                       break;
> +               default:
> +                       break;
> +               }
> +               break;
> +
> +       case lwr_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x3);
> +
> +               run->mmio.len = 4;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x3;
> +               switch (imme) {
> +               case 0:
> +                       vcpu->mmio_needed = 7;  /* 4 bytes */
> +                       break;
> +               case 1:
> +                       vcpu->mmio_needed = 8;  /* 3 bytes */
> +                       break;
> +               case 2:
> +                       vcpu->mmio_needed = 9;  /* 2 bytes */
> +                       break;
> +               case 3:
> +                       vcpu->mmio_needed = 10; /* 1 byte */
> +                       break;
> +               default:
> +                       break;
> +               }
> +               break;
> +
> +       case ldl_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x7);
> +
> +               run->mmio.len = 8;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x7;
> +               switch (imme) {
> +               case 0:
> +                       vcpu->mmio_needed = 11; /* 1 byte */
> +                       break;
> +               case 1:
> +                       vcpu->mmio_needed = 12; /* 2 bytes */
> +                       break;
> +               case 2:
> +                       vcpu->mmio_needed = 13; /* 3 bytes */
> +                       break;
> +               case 3:
> +                       vcpu->mmio_needed = 14; /* 4 bytes */
> +                       break;
> +               case 4:
> +                       vcpu->mmio_needed = 15; /* 5 bytes */
> +                       break;
> +               case 5:
> +                       vcpu->mmio_needed = 16; /* 6 bytes */
> +                       break;
> +               case 6:
> +                       vcpu->mmio_needed = 17; /* 7 bytes */
> +                       break;
> +               case 7:
> +                       vcpu->mmio_needed = 18; /* 8 bytes */
> +                       break;
> +               default:
> +                       break;
> +               }
> +               break;
> +
> +       case ldr_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x7);
> +
> +               run->mmio.len = 8;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x7;
> +               switch (imme) {
> +               case 0:
> +                       vcpu->mmio_needed = 19; /* 8 bytes */
> +                       break;
> +               case 1:
> +                       vcpu->mmio_needed = 20; /* 7 bytes */
> +                       break;
> +               case 2:
> +                       vcpu->mmio_needed = 21; /* 6 bytes */
> +                       break;
> +               case 3:
> +                       vcpu->mmio_needed = 22; /* 5 bytes */
> +                       break;
> +               case 4:
> +                       vcpu->mmio_needed = 23; /* 4 bytes */
> +                       break;
> +               case 5:
> +                       vcpu->mmio_needed = 24; /* 3 bytes */
> +                       break;
> +               case 6:
> +                       vcpu->mmio_needed = 25; /* 2 bytes */
> +                       break;
> +               case 7:
> +                       vcpu->mmio_needed = 26; /* 1 byte */
> +                       break;
> +               default:
> +                       break;
> +               }
> +               break;
> +
> +#ifdef CONFIG_CPU_LOONGSON64
> +       case ldc2_op:
> +               rt = inst.loongson3_lsdc2_format.rt;
> +               switch (inst.loongson3_lsdc2_format.opcode1) {
> +               /*
> +                * Loongson-3 overridden ldc2 instructions.
> +                * opcode1              instruction
> +                *   0x0          gslbx: store 1 bytes from GPR
> +                *   0x1          gslhx: store 2 bytes from GPR
> +                *   0x2          gslwx: store 4 bytes from GPR
> +                *   0x3          gsldx: store 8 bytes from GPR
> +                */
> +               case 0x0:
> +                       run->mmio.len = 1;
> +                       vcpu->mmio_needed = 27; /* signed */
> +                       break;
> +               case 0x1:
> +                       run->mmio.len = 2;
> +                       vcpu->mmio_needed = 28; /* signed */
> +                       break;
> +               case 0x2:
> +                       run->mmio.len = 4;
> +                       vcpu->mmio_needed = 29; /* signed */
> +                       break;
> +               case 0x3:
> +                       run->mmio.len = 8;
> +                       vcpu->mmio_needed = 30; /* signed */
> +                       break;
> +               default:
> +                       kvm_err("Godson Exteneded GS-Load for float not yet supported (inst=0x%08x)\n",
> +                               inst.word);
> +                       break;
> +               }
> +               break;
> +#endif
> +
>         default:
>                 kvm_err("Load not yet supported (inst=0x%08x)\n",
>                         inst.word);
> @@ -2612,28 +2975,125 @@ enum emulation_result kvm_mips_complete_mmio_load(struct kvm_vcpu *vcpu,
>
>         switch (run->mmio.len) {
>         case 8:
> -               *gpr = *(s64 *)run->mmio.data;
> +               switch (vcpu->mmio_needed) {
> +               case 11:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xff) << 56);
> +                       break;
> +               case 12:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffff) << 48);
> +                       break;
> +               case 13:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffff) << 40);
> +                       break;
> +               case 14:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffffff) << 32);
> +                       break;
> +               case 15:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffffffff) << 24);
> +                       break;
> +               case 16:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffffffffff) << 16);
> +                       break;
> +               case 17:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffffffffffff) << 8);
> +                       break;
> +               case 18:
> +               case 19:
> +                       *gpr = *(s64 *)run->mmio.data;
> +                       break;
> +               case 20:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff00000000000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 8) & 0xffffffffffffff);
> +                       break;
> +               case 21:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff000000000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 16) & 0xffffffffffff);
> +                       break;
> +               case 22:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff0000000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 24) & 0xffffffffff);
> +                       break;
> +               case 23:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff00000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 32) & 0xffffffff);
> +                       break;
> +               case 24:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 40) & 0xffffff);
> +                       break;
> +               case 25:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff0000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 48) & 0xffff);
> +                       break;
> +               case 26:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff00) |
> +                               ((((*(s64 *)run->mmio.data)) >> 56) & 0xff);
> +                       break;
> +               default:
> +                       *gpr = *(s64 *)run->mmio.data;
> +               }
>                 break;
>
>         case 4:
> -               if (vcpu->mmio_needed == 2)
> -                       *gpr = *(s32 *)run->mmio.data;
> -               else
> +               switch (vcpu->mmio_needed) {
> +               case 1:
>                         *gpr = *(u32 *)run->mmio.data;
> +                       break;
> +               case 2:
> +                       *gpr = *(s32 *)run->mmio.data;
> +                       break;
> +               case 3:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
> +                               (((*(s32 *)run->mmio.data) & 0xff) << 24);
> +                       break;
> +               case 4:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
> +                               (((*(s32 *)run->mmio.data) & 0xffff) << 16);
> +                       break;
> +               case 5:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
> +                               (((*(s32 *)run->mmio.data) & 0xffffff) << 8);
> +                       break;
> +               case 6:
> +               case 7:
> +                       *gpr = *(s32 *)run->mmio.data;
> +                       break;
> +               case 8:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff000000) |
> +                               ((((*(s32 *)run->mmio.data)) >> 8) & 0xffffff);
> +                       break;
> +               case 9:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff0000) |
> +                               ((((*(s32 *)run->mmio.data)) >> 16) & 0xffff);
> +                       break;
> +               case 10:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff00) |
> +                               ((((*(s32 *)run->mmio.data)) >> 24) & 0xff);
> +                       break;
> +               default:
> +                       *gpr = *(s32 *)run->mmio.data;
> +               }
>                 break;
>
>         case 2:
> -               if (vcpu->mmio_needed == 2)
> -                       *gpr = *(s16 *) run->mmio.data;
> -               else
> +               if (vcpu->mmio_needed == 1)
>                         *gpr = *(u16 *)run->mmio.data;
> +               else
> +                       *gpr = *(s16 *)run->mmio.data;
>
>                 break;
>         case 1:
> -               if (vcpu->mmio_needed == 2)
> -                       *gpr = *(s8 *) run->mmio.data;
> +               if (vcpu->mmio_needed == 1)
> +                       *gpr = *(u8 *)run->mmio.data;
>                 else
> -                       *gpr = *(u8 *) run->mmio.data;
> +                       *gpr = *(s8 *)run->mmio.data;
>                 break;
>         }
>
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

Re: [PATCH V3 13/14] KVM: MIPS: Add more MMIO load/store instructions emulation

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:16 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> This patch add more MMIO load/store instructions emulation, which can
> be observed in QXL and some other device drivers:
>
> 1, LWL, LWR, LDW, LDR, SWL, SWR, SDL and SDR for all MIPS;
> 2, GSLBX, GSLHX, GSLWX, GSLDX, GSSBX, GSSHX, GSSWX and GSSDX for
>    Loongson-3.
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
> ---
>  arch/mips/kvm/emulate.c | 480 +++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 470 insertions(+), 10 deletions(-)
>
> diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
> index 3946499..71316fa 100644
> --- a/arch/mips/kvm/emulate.c
> +++ b/arch/mips/kvm/emulate.c
> @@ -1604,6 +1604,7 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
>         enum emulation_result er;
>         u32 rt;
>         void *data = run->mmio.data;
> +       unsigned int imme;
>         unsigned long curr_pc;
>
>         /*
> @@ -1661,6 +1662,211 @@ enum emulation_result kvm_mips_emulate_store(union mips_instruction inst,
>                           vcpu->arch.gprs[rt], *(u8 *)data);
>                 break;
>
> +       case swl_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x3);
> +               run->mmio.len = 4;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x3;
> +               switch (imme) {
> +               case 0:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xffffff00) |
> +                                       (vcpu->arch.gprs[rt] >> 24);
> +                       break;
> +               case 1:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xffff0000) |
> +                                       (vcpu->arch.gprs[rt] >> 16);
> +                       break;
> +               case 2:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xff000000) |
> +                                       (vcpu->arch.gprs[rt] >> 8);
> +                       break;
> +               case 3:
> +                       *(u32 *)data = vcpu->arch.gprs[rt];
> +                       break;
> +               default:
> +                       break;
> +               }
> +
> +               kvm_debug("[%#lx] OP_SWL: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                         vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                         vcpu->arch.gprs[rt], *(u32 *)data);
> +               break;
> +
> +       case swr_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x3);
> +               run->mmio.len = 4;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x3;
> +               switch (imme) {
> +               case 0:
> +                       *(u32 *)data = vcpu->arch.gprs[rt];
> +                       break;
> +               case 1:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xff) |
> +                                       (vcpu->arch.gprs[rt] << 8);
> +                       break;
> +               case 2:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xffff) |
> +                                       (vcpu->arch.gprs[rt] << 16);
> +                       break;
> +               case 3:
> +                       *(u32 *)data = ((*(u32 *)data) & 0xffffff) |
> +                                       (vcpu->arch.gprs[rt] << 24);
> +                       break;
> +               default:
> +                       break;
> +               }
> +
> +               kvm_debug("[%#lx] OP_SWR: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                         vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                         vcpu->arch.gprs[rt], *(u32 *)data);
> +               break;
> +
> +       case sdl_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x7);
> +
> +               run->mmio.len = 8;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x7;
> +               switch (imme) {
> +               case 0:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff00) |
> +                                       ((vcpu->arch.gprs[rt] >> 56) & 0xff);
> +                       break;
> +               case 1:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffffff0000) |
> +                                       ((vcpu->arch.gprs[rt] >> 48) & 0xffff);
> +                       break;
> +               case 2:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffff000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 40) & 0xffffff);
> +                       break;
> +               case 3:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffff00000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 32) & 0xffffffff);
> +                       break;
> +               case 4:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffff0000000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 24) & 0xffffffffff);
> +                       break;
> +               case 5:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffff000000000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 16) & 0xffffffffffff);
> +                       break;
> +               case 6:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xff00000000000000) |
> +                                       ((vcpu->arch.gprs[rt] >> 8) & 0xffffffffffffff);
> +                       break;
> +               case 7:
> +                       *(u64 *)data = vcpu->arch.gprs[rt];
> +                       break;
> +               default:
> +                       break;
> +               }
> +
> +               kvm_debug("[%#lx] OP_SDL: eaddr: %#lx, gpr: %#lx, data: %llx\n",
> +                         vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                         vcpu->arch.gprs[rt], *(u64 *)data);
> +               break;
> +
> +       case sdr_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x7);
> +
> +               run->mmio.len = 8;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x7;
> +               switch (imme) {
> +               case 0:
> +                       *(u64 *)data = vcpu->arch.gprs[rt];
> +                       break;
> +               case 1:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xff) |
> +                                       (vcpu->arch.gprs[rt] << 8);
> +                       break;
> +               case 2:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffff) |
> +                                       (vcpu->arch.gprs[rt] << 16);
> +                       break;
> +               case 3:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffff) |
> +                                       (vcpu->arch.gprs[rt] << 24);
> +                       break;
> +               case 4:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffff) |
> +                                       (vcpu->arch.gprs[rt] << 32);
> +                       break;
> +               case 5:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffff) |
> +                                       (vcpu->arch.gprs[rt] << 40);
> +                       break;
> +               case 6:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffffff) |
> +                                       (vcpu->arch.gprs[rt] << 48);
> +                       break;
> +               case 7:
> +                       *(u64 *)data = ((*(u64 *)data) & 0xffffffffffffff) |
> +                                       (vcpu->arch.gprs[rt] << 56);
> +                       break;
> +               default:
> +                       break;
> +               }
> +
> +               kvm_debug("[%#lx] OP_SDR: eaddr: %#lx, gpr: %#lx, data: %llx\n",
> +                         vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                         vcpu->arch.gprs[rt], *(u64 *)data);
> +               break;
> +
> +#ifdef CONFIG_CPU_LOONGSON64
> +       case sdc2_op:
> +               rt = inst.loongson3_lsdc2_format.rt;
> +               switch (inst.loongson3_lsdc2_format.opcode1) {
> +               /*
> +                * Loongson-3 overridden sdc2 instructions.
> +                * opcode1              instruction
> +                *   0x0          gssbx: store 1 bytes from GPR
> +                *   0x1          gsshx: store 2 bytes from GPR
> +                *   0x2          gsswx: store 4 bytes from GPR
> +                *   0x3          gssdx: store 8 bytes from GPR
> +                */
> +               case 0x0:
> +                       run->mmio.len = 1;
> +                       *(u8 *)data = vcpu->arch.gprs[rt];
> +
> +                       kvm_debug("[%#lx] OP_GSSBX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                                 vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                                 vcpu->arch.gprs[rt], *(u8 *)data);
> +                       break;
> +               case 0x1:
> +                       run->mmio.len = 2;
> +                       *(u16 *)data = vcpu->arch.gprs[rt];
> +
> +                       kvm_debug("[%#lx] OP_GSSSHX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                                 vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                                 vcpu->arch.gprs[rt], *(u16 *)data);
> +                       break;
> +               case 0x2:
> +                       run->mmio.len = 4;
> +                       *(u32 *)data = vcpu->arch.gprs[rt];
> +
> +                       kvm_debug("[%#lx] OP_GSSWX: eaddr: %#lx, gpr: %#lx, data: %#x\n",
> +                                 vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                                 vcpu->arch.gprs[rt], *(u32 *)data);
> +                       break;
> +               case 0x3:
> +                       run->mmio.len = 8;
> +                       *(u64 *)data = vcpu->arch.gprs[rt];
> +
> +                       kvm_debug("[%#lx] OP_GSSDX: eaddr: %#lx, gpr: %#lx, data: %#llx\n",
> +                                 vcpu->arch.pc, vcpu->arch.host_cp0_badvaddr,
> +                                 vcpu->arch.gprs[rt], *(u64 *)data);
> +                       break;
> +               default:
> +                       kvm_err("Godson Exteneded GS-Store not yet supported (inst=0x%08x)\n",
> +                               inst.word);
> +                       break;
> +               }
> +               break;
> +#endif
>         default:
>                 kvm_err("Store not yet supported (inst=0x%08x)\n",
>                         inst.word);
> @@ -1695,6 +1901,7 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
>         enum emulation_result er;
>         unsigned long curr_pc;
>         u32 op, rt;
> +       unsigned int imme;
>
>         rt = inst.i_format.rt;
>         op = inst.i_format.opcode;
> @@ -1747,6 +1954,162 @@ enum emulation_result kvm_mips_emulate_load(union mips_instruction inst,
>                 run->mmio.len = 1;
>                 break;
>
> +       case lwl_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x3);
> +
> +               run->mmio.len = 4;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x3;
> +               switch (imme) {
> +               case 0:
> +                       vcpu->mmio_needed = 3;  /* 1 byte */
> +                       break;
> +               case 1:
> +                       vcpu->mmio_needed = 4;  /* 2 bytes */
> +                       break;
> +               case 2:
> +                       vcpu->mmio_needed = 5;  /* 3 bytes */
> +                       break;
> +               case 3:
> +                       vcpu->mmio_needed = 6;  /* 4 bytes */
> +                       break;
> +               default:
> +                       break;
> +               }
> +               break;
> +
> +       case lwr_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x3);
> +
> +               run->mmio.len = 4;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x3;
> +               switch (imme) {
> +               case 0:
> +                       vcpu->mmio_needed = 7;  /* 4 bytes */
> +                       break;
> +               case 1:
> +                       vcpu->mmio_needed = 8;  /* 3 bytes */
> +                       break;
> +               case 2:
> +                       vcpu->mmio_needed = 9;  /* 2 bytes */
> +                       break;
> +               case 3:
> +                       vcpu->mmio_needed = 10; /* 1 byte */
> +                       break;
> +               default:
> +                       break;
> +               }
> +               break;
> +
> +       case ldl_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x7);
> +
> +               run->mmio.len = 8;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x7;
> +               switch (imme) {
> +               case 0:
> +                       vcpu->mmio_needed = 11; /* 1 byte */
> +                       break;
> +               case 1:
> +                       vcpu->mmio_needed = 12; /* 2 bytes */
> +                       break;
> +               case 2:
> +                       vcpu->mmio_needed = 13; /* 3 bytes */
> +                       break;
> +               case 3:
> +                       vcpu->mmio_needed = 14; /* 4 bytes */
> +                       break;
> +               case 4:
> +                       vcpu->mmio_needed = 15; /* 5 bytes */
> +                       break;
> +               case 5:
> +                       vcpu->mmio_needed = 16; /* 6 bytes */
> +                       break;
> +               case 6:
> +                       vcpu->mmio_needed = 17; /* 7 bytes */
> +                       break;
> +               case 7:
> +                       vcpu->mmio_needed = 18; /* 8 bytes */
> +                       break;
> +               default:
> +                       break;
> +               }
> +               break;
> +
> +       case ldr_op:
> +               run->mmio.phys_addr = kvm_mips_callbacks->gva_to_gpa(
> +                                       vcpu->arch.host_cp0_badvaddr) & (~0x7);
> +
> +               run->mmio.len = 8;
> +               imme = vcpu->arch.host_cp0_badvaddr & 0x7;
> +               switch (imme) {
> +               case 0:
> +                       vcpu->mmio_needed = 19; /* 8 bytes */
> +                       break;
> +               case 1:
> +                       vcpu->mmio_needed = 20; /* 7 bytes */
> +                       break;
> +               case 2:
> +                       vcpu->mmio_needed = 21; /* 6 bytes */
> +                       break;
> +               case 3:
> +                       vcpu->mmio_needed = 22; /* 5 bytes */
> +                       break;
> +               case 4:
> +                       vcpu->mmio_needed = 23; /* 4 bytes */
> +                       break;
> +               case 5:
> +                       vcpu->mmio_needed = 24; /* 3 bytes */
> +                       break;
> +               case 6:
> +                       vcpu->mmio_needed = 25; /* 2 bytes */
> +                       break;
> +               case 7:
> +                       vcpu->mmio_needed = 26; /* 1 byte */
> +                       break;
> +               default:
> +                       break;
> +               }
> +               break;
> +
> +#ifdef CONFIG_CPU_LOONGSON64
> +       case ldc2_op:
> +               rt = inst.loongson3_lsdc2_format.rt;
> +               switch (inst.loongson3_lsdc2_format.opcode1) {
> +               /*
> +                * Loongson-3 overridden ldc2 instructions.
> +                * opcode1              instruction
> +                *   0x0          gslbx: store 1 bytes from GPR
> +                *   0x1          gslhx: store 2 bytes from GPR
> +                *   0x2          gslwx: store 4 bytes from GPR
> +                *   0x3          gsldx: store 8 bytes from GPR
> +                */
> +               case 0x0:
> +                       run->mmio.len = 1;
> +                       vcpu->mmio_needed = 27; /* signed */
> +                       break;
> +               case 0x1:
> +                       run->mmio.len = 2;
> +                       vcpu->mmio_needed = 28; /* signed */
> +                       break;
> +               case 0x2:
> +                       run->mmio.len = 4;
> +                       vcpu->mmio_needed = 29; /* signed */
> +                       break;
> +               case 0x3:
> +                       run->mmio.len = 8;
> +                       vcpu->mmio_needed = 30; /* signed */
> +                       break;
> +               default:
> +                       kvm_err("Godson Exteneded GS-Load for float not yet supported (inst=0x%08x)\n",
> +                               inst.word);
> +                       break;
> +               }
> +               break;
> +#endif
> +
>         default:
>                 kvm_err("Load not yet supported (inst=0x%08x)\n",
>                         inst.word);
> @@ -2612,28 +2975,125 @@ enum emulation_result kvm_mips_complete_mmio_load(struct kvm_vcpu *vcpu,
>
>         switch (run->mmio.len) {
>         case 8:
> -               *gpr = *(s64 *)run->mmio.data;
> +               switch (vcpu->mmio_needed) {
> +               case 11:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xff) << 56);
> +                       break;
> +               case 12:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffff) << 48);
> +                       break;
> +               case 13:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffff) << 40);
> +                       break;
> +               case 14:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffffff) << 32);
> +                       break;
> +               case 15:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffffffff) << 24);
> +                       break;
> +               case 16:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffffffffff) << 16);
> +                       break;
> +               case 17:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
> +                               (((*(s64 *)run->mmio.data) & 0xffffffffffffff) << 8);
> +                       break;
> +               case 18:
> +               case 19:
> +                       *gpr = *(s64 *)run->mmio.data;
> +                       break;
> +               case 20:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff00000000000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 8) & 0xffffffffffffff);
> +                       break;
> +               case 21:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff000000000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 16) & 0xffffffffffff);
> +                       break;
> +               case 22:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff0000000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 24) & 0xffffffffff);
> +                       break;
> +               case 23:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffff00000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 32) & 0xffffffff);
> +                       break;
> +               case 24:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffff000000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 40) & 0xffffff);
> +                       break;
> +               case 25:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffff0000) |
> +                               ((((*(s64 *)run->mmio.data)) >> 48) & 0xffff);
> +                       break;
> +               case 26:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffffffffffff00) |
> +                               ((((*(s64 *)run->mmio.data)) >> 56) & 0xff);
> +                       break;
> +               default:
> +                       *gpr = *(s64 *)run->mmio.data;
> +               }
>                 break;
>
>         case 4:
> -               if (vcpu->mmio_needed == 2)
> -                       *gpr = *(s32 *)run->mmio.data;
> -               else
> +               switch (vcpu->mmio_needed) {
> +               case 1:
>                         *gpr = *(u32 *)run->mmio.data;
> +                       break;
> +               case 2:
> +                       *gpr = *(s32 *)run->mmio.data;
> +                       break;
> +               case 3:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff) |
> +                               (((*(s32 *)run->mmio.data) & 0xff) << 24);
> +                       break;
> +               case 4:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff) |
> +                               (((*(s32 *)run->mmio.data) & 0xffff) << 16);
> +                       break;
> +               case 5:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff) |
> +                               (((*(s32 *)run->mmio.data) & 0xffffff) << 8);
> +                       break;
> +               case 6:
> +               case 7:
> +                       *gpr = *(s32 *)run->mmio.data;
> +                       break;
> +               case 8:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xff000000) |
> +                               ((((*(s32 *)run->mmio.data)) >> 8) & 0xffffff);
> +                       break;
> +               case 9:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffff0000) |
> +                               ((((*(s32 *)run->mmio.data)) >> 16) & 0xffff);
> +                       break;
> +               case 10:
> +                       *gpr = (vcpu->arch.gprs[vcpu->arch.io_gpr] & 0xffffff00) |
> +                               ((((*(s32 *)run->mmio.data)) >> 24) & 0xff);
> +                       break;
> +               default:
> +                       *gpr = *(s32 *)run->mmio.data;
> +               }
>                 break;
>
>         case 2:
> -               if (vcpu->mmio_needed == 2)
> -                       *gpr = *(s16 *) run->mmio.data;
> -               else
> +               if (vcpu->mmio_needed == 1)
>                         *gpr = *(u16 *)run->mmio.data;
> +               else
> +                       *gpr = *(s16 *)run->mmio.data;
>
>                 break;
>         case 1:
> -               if (vcpu->mmio_needed == 2)
> -                       *gpr = *(s8 *) run->mmio.data;
> +               if (vcpu->mmio_needed == 1)
> +                       *gpr = *(u8 *)run->mmio.data;
>                 else
> -                       *gpr = *(u8 *) run->mmio.data;
> +                       *gpr = *(s8 *)run->mmio.data;
>                 break;
>         }
>
> --
> 2.7.0
>

Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

[PATCH V3 14/14] KVM: MIPS: Enable KVM support for Loongson-3

[Huacai Chen]

This patch enable KVM support for Loongson-3 by selecting HAVE_KVM, but
only enable KVM/VZ on Loongson-3A R4+ (because VZ of early processors
are incomplete). Besides, Loongson-3 support SMP guests, so we clear the
linked load bit of LLAddr in kvm_vz_vcpu_load() if the guest has more
than one VCPUs.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/Kconfig            | 1 +
 arch/mips/kernel/cpu-probe.c | 1 +
 arch/mips/kvm/vz.c           | 2 +-
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 9f15539..9c4bdac 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -1470,6 +1470,7 @@ config CPU_LOONGSON64
 	select MIPS_L1_CACHE_SHIFT_6
 	select GPIOLIB
 	select SWIOTLB
+	select HAVE_KVM
 	help
 		The Loongson GSx64(GS264/GS464/GS464E/GS464V) series of processor
 		cores implements the MIPS64R2 instruction set with many extensions,
diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
index be1b556..4432442 100644
--- a/arch/mips/kernel/cpu-probe.c
+++ b/arch/mips/kernel/cpu-probe.c
@@ -2008,6 +2008,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
 		c->writecombine = _CACHE_UNCACHED_ACCELERATED;
 		c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
 			MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
+		c->ases &= ~MIPS_ASE_VZ; /* VZ of Loongson-3A2000/3000 is incomplete */
 		break;
 	case PRID_IMP_LOONGSON_64G:
 		c->cputype = CPU_LOONGSON64;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index fc0f8d5..5f877a9 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2695,7 +2695,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	 * prevents a SC on the next VCPU from succeeding by matching a LL on
 	 * the previous VCPU.
 	 */
-	if (cpu_guest_has_rw_llb)
+	if (vcpu->kvm->created_vcpus > 1)
 		write_gc0_lladdr(0);
 
 	return 0;
-- 
2.7.0

[PATCH V3 14/14] KVM: MIPS: Enable KVM support for Loongson-3

[Huacai Chen]

This patch enable KVM support for Loongson-3 by selecting HAVE_KVM, but
only enable KVM/VZ on Loongson-3A R4+ (because VZ of early processors
are incomplete). Besides, Loongson-3 support SMP guests, so we clear the
linked load bit of LLAddr in kvm_vz_vcpu_load() if the guest has more
than one VCPUs.

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Co-developed-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/Kconfig            | 1 +
 arch/mips/kernel/cpu-probe.c | 1 +
 arch/mips/kvm/vz.c           | 2 +-
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 9f15539..9c4bdac 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -1470,6 +1470,7 @@ config CPU_LOONGSON64
 	select MIPS_L1_CACHE_SHIFT_6
 	select GPIOLIB
 	select SWIOTLB
+	select HAVE_KVM
 	help
 		The Loongson GSx64(GS264/GS464/GS464E/GS464V) series of processor
 		cores implements the MIPS64R2 instruction set with many extensions,
diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
index be1b556..4432442 100644
--- a/arch/mips/kernel/cpu-probe.c
+++ b/arch/mips/kernel/cpu-probe.c
@@ -2008,6 +2008,7 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
 		c->writecombine = _CACHE_UNCACHED_ACCELERATED;
 		c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM |
 			MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2);
+		c->ases &= ~MIPS_ASE_VZ; /* VZ of Loongson-3A2000/3000 is incomplete */
 		break;
 	case PRID_IMP_LOONGSON_64G:
 		c->cputype = CPU_LOONGSON64;
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index fc0f8d5..5f877a9 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -2695,7 +2695,7 @@ static int kvm_vz_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	 * prevents a SC on the next VCPU from succeeding by matching a LL on
 	 * the previous VCPU.
 	 */
-	if (cpu_guest_has_rw_llb)
+	if (vcpu->kvm->created_vcpus > 1)
 		write_gc0_lladdr(0);
 
 	return 0;
-- 
2.7.0

Re: [PATCH V3 00/14] KVM: MIPS: Add Loongson-3 support (Host Side)

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:06 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> We are preparing to add KVM support for Loongson-3. VZ extension is
> fully supported in Loongson-3A R4+, and we will not care about old CPUs
> (at least now). We already have a full functional Linux kernel (based
> on Linux-5.4.x LTS) and QEMU (based on 5.0.0-rc2) and their git
> repositories are here:
>
> QEMU: https://github.com/chenhuacai/qemu
> Kernel: https://github.com/chenhuacai/linux
>
> Of course these two repositories need to be rework and not suitable for
> upstream (especially the commits need to be splitted). We show them here
> is just to tell others what we have done, and how KVM/Loongson will look
> like.
>
> Our plan is make the KVM host side be upstream first, and after that,
> we will make the KVM guest side and QEMU emulator be upstream.
>

Huacei, I absolutely salute this series, as it is, in my opinion, of
tremendous significance not only for Loongson, but also for KVM for
MIPS in general.

As you probably know, James Hogan left KVM for MIPS kernel
maintainership some time ago. It was really too bad, as he was and
still is, from my direct experience, an excellent colleague and
engineer.

KVM for MIPS kernel maintainership left orphaned put me (as the
maintainer for KVM for MIPS in QEMU) in an awkward position.

May I ask that you and me jointly assume KVM for MIPS kernel
maintainership? For me, it makes perfect sense, and I would certainly
enjoy working with you. If you agree, please add such patch at the end
of this series, in its next version.

Yours,
Aleksandar


> V1 -> V2:
> 1, Remove "mips: define pud_index() regardless of page table folding"
>    because it has been applied.
> 2, Make Loongson-specific code be guarded by CONFIG_CPU_LOONGSON64.
>
> V2 -> V3:
> 1, Emulate a reduced feature list of CPUCFG.
> 2, Fix all possible checkpatch.pl errors and warnings.
>
> Xing Li(2):
>  KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
>  KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits
>
> Huacai Chen(12):
>  KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16
>  KVM: MIPS: Add EVENTFD support which is needed by VHOST
>  KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd
>  KVM: MIPS: Introduce and use cpu_guest_has_ldpte
>  KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3
>  KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3
>  KVM: MIPS: Add more types of virtual interrupts
>  KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support
>  KVM: MIPS: Add CPUCFG emulation for Loongson-3
>  KVM: MIPS: Add CONFIG6 and DIAG registers emulation
>  KVM: MIPS: Add more MMIO load/store instructions emulation
>  KVM: MIPS: Enable KVM support for Loongson-3
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> ---
>  arch/mips/Kconfig                    |   1 +
>  arch/mips/include/asm/cpu-features.h |   3 +
>  arch/mips/include/asm/kvm_host.h     |  52 +++-
>  arch/mips/include/asm/mipsregs.h     |   7 +
>  arch/mips/include/uapi/asm/inst.h    |  11 +
>  arch/mips/kernel/cpu-probe.c         |   2 +
>  arch/mips/kvm/Kconfig                |   1 +
>  arch/mips/kvm/Makefile               |   5 +-
>  arch/mips/kvm/emulate.c              | 503 ++++++++++++++++++++++++++++++++++-
>  arch/mips/kvm/entry.c                |  19 +-
>  arch/mips/kvm/interrupt.c            |  93 +------
>  arch/mips/kvm/interrupt.h            |  14 +-
>  arch/mips/kvm/loongson_ipi.c         | 214 +++++++++++++++
>  arch/mips/kvm/mips.c                 |  49 +++-
>  arch/mips/kvm/tlb.c                  |  41 +++
>  arch/mips/kvm/trap_emul.c            |   3 +
>  arch/mips/kvm/vz.c                   | 235 +++++++++++-----
>  17 files changed, 1087 insertions(+), 166 deletions(-)
>  create mode 100644 arch/mips/kvm/loongson_ipi.c
> --
> 2.7.0

Re: [PATCH V3 00/14] KVM: MIPS: Add Loongson-3 support (Host Side)

[Aleksandar Markovic]

нед, 3. мај 2020. у 12:06 Huacai Chen <chenhc@lemote.com> је написао/ла:
>
> We are preparing to add KVM support for Loongson-3. VZ extension is
> fully supported in Loongson-3A R4+, and we will not care about old CPUs
> (at least now). We already have a full functional Linux kernel (based
> on Linux-5.4.x LTS) and QEMU (based on 5.0.0-rc2) and their git
> repositories are here:
>
> QEMU: https://github.com/chenhuacai/qemu
> Kernel: https://github.com/chenhuacai/linux
>
> Of course these two repositories need to be rework and not suitable for
> upstream (especially the commits need to be splitted). We show them here
> is just to tell others what we have done, and how KVM/Loongson will look
> like.
>
> Our plan is make the KVM host side be upstream first, and after that,
> we will make the KVM guest side and QEMU emulator be upstream.
>

Huacei, I absolutely salute this series, as it is, in my opinion, of
tremendous significance not only for Loongson, but also for KVM for
MIPS in general.

As you probably know, James Hogan left KVM for MIPS kernel
maintainership some time ago. It was really too bad, as he was and
still is, from my direct experience, an excellent colleague and
engineer.

KVM for MIPS kernel maintainership left orphaned put me (as the
maintainer for KVM for MIPS in QEMU) in an awkward position.

May I ask that you and me jointly assume KVM for MIPS kernel
maintainership? For me, it makes perfect sense, and I would certainly
enjoy working with you. If you agree, please add such patch at the end
of this series, in its next version.

Yours,
Aleksandar


> V1 -> V2:
> 1, Remove "mips: define pud_index() regardless of page table folding"
>    because it has been applied.
> 2, Make Loongson-specific code be guarded by CONFIG_CPU_LOONGSON64.
>
> V2 -> V3:
> 1, Emulate a reduced feature list of CPUCFG.
> 2, Fix all possible checkpatch.pl errors and warnings.
>
> Xing Li(2):
>  KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
>  KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits
>
> Huacai Chen(12):
>  KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16
>  KVM: MIPS: Add EVENTFD support which is needed by VHOST
>  KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd
>  KVM: MIPS: Introduce and use cpu_guest_has_ldpte
>  KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3
>  KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3
>  KVM: MIPS: Add more types of virtual interrupts
>  KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support
>  KVM: MIPS: Add CPUCFG emulation for Loongson-3
>  KVM: MIPS: Add CONFIG6 and DIAG registers emulation
>  KVM: MIPS: Add more MMIO load/store instructions emulation
>  KVM: MIPS: Enable KVM support for Loongson-3
>
> Signed-off-by: Huacai Chen <chenhc@lemote.com>
> ---
>  arch/mips/Kconfig                    |   1 +
>  arch/mips/include/asm/cpu-features.h |   3 +
>  arch/mips/include/asm/kvm_host.h     |  52 +++-
>  arch/mips/include/asm/mipsregs.h     |   7 +
>  arch/mips/include/uapi/asm/inst.h    |  11 +
>  arch/mips/kernel/cpu-probe.c         |   2 +
>  arch/mips/kvm/Kconfig                |   1 +
>  arch/mips/kvm/Makefile               |   5 +-
>  arch/mips/kvm/emulate.c              | 503 ++++++++++++++++++++++++++++++++++-
>  arch/mips/kvm/entry.c                |  19 +-
>  arch/mips/kvm/interrupt.c            |  93 +------
>  arch/mips/kvm/interrupt.h            |  14 +-
>  arch/mips/kvm/loongson_ipi.c         | 214 +++++++++++++++
>  arch/mips/kvm/mips.c                 |  49 +++-
>  arch/mips/kvm/tlb.c                  |  41 +++
>  arch/mips/kvm/trap_emul.c            |   3 +
>  arch/mips/kvm/vz.c                   | 235 +++++++++++-----
>  17 files changed, 1087 insertions(+), 166 deletions(-)
>  create mode 100644 arch/mips/kvm/loongson_ipi.c
> --
> 2.7.0

Re: [PATCH V3 00/14] KVM: MIPS: Add Loongson-3 support (Host Side)

[Huacai Chen]

Hi, Aleksandar,

On Fri, May 8, 2020 at 7:43 PM Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> wrote:
>
> нед, 3. мај 2020. у 12:06 Huacai Chen <chenhc@lemote.com> је написао/ла:
> >
> > We are preparing to add KVM support for Loongson-3. VZ extension is
> > fully supported in Loongson-3A R4+, and we will not care about old CPUs
> > (at least now). We already have a full functional Linux kernel (based
> > on Linux-5.4.x LTS) and QEMU (based on 5.0.0-rc2) and their git
> > repositories are here:
> >
> > QEMU: https://github.com/chenhuacai/qemu
> > Kernel: https://github.com/chenhuacai/linux
> >
> > Of course these two repositories need to be rework and not suitable for
> > upstream (especially the commits need to be splitted). We show them here
> > is just to tell others what we have done, and how KVM/Loongson will look
> > like.
> >
> > Our plan is make the KVM host side be upstream first, and after that,
> > we will make the KVM guest side and QEMU emulator be upstream.
> >
>
> Huacei, I absolutely salute this series, as it is, in my opinion, of
> tremendous significance not only for Loongson, but also for KVM for
> MIPS in general.
>
> As you probably know, James Hogan left KVM for MIPS kernel
> maintainership some time ago. It was really too bad, as he was and
> still is, from my direct experience, an excellent colleague and
> engineer.
>
> KVM for MIPS kernel maintainership left orphaned put me (as the
> maintainer for KVM for MIPS in QEMU) in an awkward position.
>
> May I ask that you and me jointly assume KVM for MIPS kernel
> maintainership? For me, it makes perfect sense, and I would certainly
> enjoy working with you. If you agree, please add such patch at the end
> of this series, in its next version.
I'm very glad to join maintainership with you, thank you for your invitation.

Regards,
Huacai

>
> Yours,
> Aleksandar
>
>
> > V1 -> V2:
> > 1, Remove "mips: define pud_index() regardless of page table folding"
> >    because it has been applied.
> > 2, Make Loongson-specific code be guarded by CONFIG_CPU_LOONGSON64.
> >
> > V2 -> V3:
> > 1, Emulate a reduced feature list of CPUCFG.
> > 2, Fix all possible checkpatch.pl errors and warnings.
> >
> > Xing Li(2):
> >  KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
> >  KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits
> >
> > Huacai Chen(12):
> >  KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16
> >  KVM: MIPS: Add EVENTFD support which is needed by VHOST
> >  KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd
> >  KVM: MIPS: Introduce and use cpu_guest_has_ldpte
> >  KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3
> >  KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3
> >  KVM: MIPS: Add more types of virtual interrupts
> >  KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support
> >  KVM: MIPS: Add CPUCFG emulation for Loongson-3
> >  KVM: MIPS: Add CONFIG6 and DIAG registers emulation
> >  KVM: MIPS: Add more MMIO load/store instructions emulation
> >  KVM: MIPS: Enable KVM support for Loongson-3
> >
> > Signed-off-by: Huacai Chen <chenhc@lemote.com>
> > ---
> >  arch/mips/Kconfig                    |   1 +
> >  arch/mips/include/asm/cpu-features.h |   3 +
> >  arch/mips/include/asm/kvm_host.h     |  52 +++-
> >  arch/mips/include/asm/mipsregs.h     |   7 +
> >  arch/mips/include/uapi/asm/inst.h    |  11 +
> >  arch/mips/kernel/cpu-probe.c         |   2 +
> >  arch/mips/kvm/Kconfig                |   1 +
> >  arch/mips/kvm/Makefile               |   5 +-
> >  arch/mips/kvm/emulate.c              | 503 ++++++++++++++++++++++++++++++++++-
> >  arch/mips/kvm/entry.c                |  19 +-
> >  arch/mips/kvm/interrupt.c            |  93 +------
> >  arch/mips/kvm/interrupt.h            |  14 +-
> >  arch/mips/kvm/loongson_ipi.c         | 214 +++++++++++++++
> >  arch/mips/kvm/mips.c                 |  49 +++-
> >  arch/mips/kvm/tlb.c                  |  41 +++
> >  arch/mips/kvm/trap_emul.c            |   3 +
> >  arch/mips/kvm/vz.c                   | 235 +++++++++++-----
> >  17 files changed, 1087 insertions(+), 166 deletions(-)
> >  create mode 100644 arch/mips/kvm/loongson_ipi.c
> > --
> > 2.7.0

Re: [PATCH V3 00/14] KVM: MIPS: Add Loongson-3 support (Host Side)

[Huacai Chen]

Hi, Aleksandar,

On Fri, May 8, 2020 at 7:43 PM Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> wrote:
>
> нед, 3. мај 2020. у 12:06 Huacai Chen <chenhc@lemote.com> је написао/ла:
> >
> > We are preparing to add KVM support for Loongson-3. VZ extension is
> > fully supported in Loongson-3A R4+, and we will not care about old CPUs
> > (at least now). We already have a full functional Linux kernel (based
> > on Linux-5.4.x LTS) and QEMU (based on 5.0.0-rc2) and their git
> > repositories are here:
> >
> > QEMU: https://github.com/chenhuacai/qemu
> > Kernel: https://github.com/chenhuacai/linux
> >
> > Of course these two repositories need to be rework and not suitable for
> > upstream (especially the commits need to be splitted). We show them here
> > is just to tell others what we have done, and how KVM/Loongson will look
> > like.
> >
> > Our plan is make the KVM host side be upstream first, and after that,
> > we will make the KVM guest side and QEMU emulator be upstream.
> >
>
> Huacei, I absolutely salute this series, as it is, in my opinion, of
> tremendous significance not only for Loongson, but also for KVM for
> MIPS in general.
>
> As you probably know, James Hogan left KVM for MIPS kernel
> maintainership some time ago. It was really too bad, as he was and
> still is, from my direct experience, an excellent colleague and
> engineer.
>
> KVM for MIPS kernel maintainership left orphaned put me (as the
> maintainer for KVM for MIPS in QEMU) in an awkward position.
>
> May I ask that you and me jointly assume KVM for MIPS kernel
> maintainership? For me, it makes perfect sense, and I would certainly
> enjoy working with you. If you agree, please add such patch at the end
> of this series, in its next version.
I'm very glad to join maintainership with you, thank you for your invitation.

Regards,
Huacai

>
> Yours,
> Aleksandar
>
>
> > V1 -> V2:
> > 1, Remove "mips: define pud_index() regardless of page table folding"
> >    because it has been applied.
> > 2, Make Loongson-specific code be guarded by CONFIG_CPU_LOONGSON64.
> >
> > V2 -> V3:
> > 1, Emulate a reduced feature list of CPUCFG.
> > 2, Fix all possible checkpatch.pl errors and warnings.
> >
> > Xing Li(2):
> >  KVM: MIPS: Define KVM_ENTRYHI_ASID to cpu_asid_mask(&boot_cpu_data)
> >  KVM: MIPS: Fix VPN2_MASK definition for variable cpu_vmbits
> >
> > Huacai Chen(12):
> >  KVM: MIPS: Increase KVM_MAX_VCPUS and KVM_USER_MEM_SLOTS to 16
> >  KVM: MIPS: Add EVENTFD support which is needed by VHOST
> >  KVM: MIPS: Use lddir/ldpte instructions to lookup gpa_mm.pgd
> >  KVM: MIPS: Introduce and use cpu_guest_has_ldpte
> >  KVM: MIPS: Use root tlb to control guest's CCA for Loongson-3
> >  KVM: MIPS: Let indexed cacheops cause guest exit on Loongson-3
> >  KVM: MIPS: Add more types of virtual interrupts
> >  KVM: MIPS: Add Loongson-3 Virtual IPI interrupt support
> >  KVM: MIPS: Add CPUCFG emulation for Loongson-3
> >  KVM: MIPS: Add CONFIG6 and DIAG registers emulation
> >  KVM: MIPS: Add more MMIO load/store instructions emulation
> >  KVM: MIPS: Enable KVM support for Loongson-3
> >
> > Signed-off-by: Huacai Chen <chenhc@lemote.com>
> > ---
> >  arch/mips/Kconfig                    |   1 +
> >  arch/mips/include/asm/cpu-features.h |   3 +
> >  arch/mips/include/asm/kvm_host.h     |  52 +++-
> >  arch/mips/include/asm/mipsregs.h     |   7 +
> >  arch/mips/include/uapi/asm/inst.h    |  11 +
> >  arch/mips/kernel/cpu-probe.c         |   2 +
> >  arch/mips/kvm/Kconfig                |   1 +
> >  arch/mips/kvm/Makefile               |   5 +-
> >  arch/mips/kvm/emulate.c              | 503 ++++++++++++++++++++++++++++++++++-
> >  arch/mips/kvm/entry.c                |  19 +-
> >  arch/mips/kvm/interrupt.c            |  93 +------
> >  arch/mips/kvm/interrupt.h            |  14 +-
> >  arch/mips/kvm/loongson_ipi.c         | 214 +++++++++++++++
> >  arch/mips/kvm/mips.c                 |  49 +++-
> >  arch/mips/kvm/tlb.c                  |  41 +++
> >  arch/mips/kvm/trap_emul.c            |   3 +
> >  arch/mips/kvm/vz.c                   | 235 +++++++++++-----
> >  17 files changed, 1087 insertions(+), 166 deletions(-)
> >  create mode 100644 arch/mips/kvm/loongson_ipi.c
> > --
> > 2.7.0